Adding upstream version 4.64.1.
Signed-off-by: Daniel Baumann <daniel@debian.org>
This commit is contained in:
parent
ee08d9327c
commit
2da88b2fbc
89 changed files with 16770 additions and 0 deletions
62
.pre-commit-config.yaml
Normal file
62
.pre-commit-config.yaml
Normal file
|
@ -0,0 +1,62 @@
|
|||
default_language_version:
|
||||
python: python3
|
||||
repos:
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v4.1.0
|
||||
hooks:
|
||||
- id: check-added-large-files
|
||||
- id: check-case-conflict
|
||||
- id: check-docstring-first
|
||||
- id: check-executables-have-shebangs
|
||||
- id: check-toml
|
||||
- id: check-merge-conflict
|
||||
- id: check-yaml
|
||||
- id: debug-statements
|
||||
- id: end-of-file-fixer
|
||||
- id: mixed-line-ending
|
||||
- id: sort-simple-yaml
|
||||
- id: trailing-whitespace
|
||||
exclude: ^README.rst$
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: todo
|
||||
name: Check TODO
|
||||
language: pygrep
|
||||
entry: WIP
|
||||
args: [-i]
|
||||
types: [text]
|
||||
exclude: ^(.pre-commit-config.yaml|.github/workflows/test.yml)$
|
||||
- id: pytest
|
||||
name: pytest quick
|
||||
language: python
|
||||
entry: pytest
|
||||
args: [-qq, --durations=1, -k=not slow]
|
||||
types: [python]
|
||||
pass_filenames: false
|
||||
additional_dependencies:
|
||||
- numpy
|
||||
- pandas
|
||||
- pytest-timeout
|
||||
- pytest-asyncio
|
||||
- repo: https://gitlab.com/pycqa/flake8
|
||||
rev: 3.9.2
|
||||
hooks:
|
||||
- id: flake8
|
||||
args: [-j8]
|
||||
additional_dependencies:
|
||||
- flake8-broken-line
|
||||
- flake8-bugbear
|
||||
- flake8-comprehensions
|
||||
- flake8-debugger
|
||||
- flake8-isort
|
||||
- flake8-string-format
|
||||
- flake8-type-annotations
|
||||
- repo: https://github.com/PyCQA/isort
|
||||
rev: 5.10.1
|
||||
hooks:
|
||||
- id: isort
|
||||
- repo: https://github.com/kynan/nbstripout
|
||||
rev: 0.5.0
|
||||
hooks:
|
||||
- id: nbstripout
|
||||
args: [--keep-count, --keep-output]
|
11
.zenodo.json
Normal file
11
.zenodo.json
Normal file
|
@ -0,0 +1,11 @@
|
|||
{
|
||||
"title": "tqdm: A fast, Extensible Progress Bar for Python and CLI",
|
||||
"keywords": [
|
||||
"progressbar", "progressmeter", "progress-bar", "meter", "rate", "eta",
|
||||
"console", "terminal", "time", "progress", "bar", "gui", "python",
|
||||
"parallel", "cli", "utilities", "shell", "batch"],
|
||||
"related_identifiers": [
|
||||
{"identifier": "10.21105/joss.01277", "relation": "cites"}],
|
||||
"contributors": [
|
||||
{"name": "tqdm developers", "type": "Other", "affiliation": "tqdm"}]
|
||||
}
|
26
CODE_OF_CONDUCT.md
Normal file
26
CODE_OF_CONDUCT.md
Normal file
|
@ -0,0 +1,26 @@
|
|||
International law supersedes all codes of conduct, including this one.
|
||||
Authors of codes of conduct presuming the authority to create new laws
|
||||
should be brought to justice.
|
||||
|
||||
This being said, if you still insist on wringing advice from us:
|
||||
Be kind. Be professional. Be friendly. Be respectful. Focus on the project.
|
||||
[Report inappropriate conduct](https://help.github.com/en/articles/reporting-abuse-or-spam).
|
||||
|
||||
Contributions which fail to conform to these guidelines may be reworded,
|
||||
censored, and/or reported as abuse without notice.
|
||||
|
||||
## The long explanation
|
||||
|
||||
We celebrate the fact that all contributions come from different humans with
|
||||
their own unique attributes, background and history. However we also place a
|
||||
strong emphasis on neutrality and a simple focus on the project's goals.
|
||||
Contributions are thus particularly welcome when references to gender,
|
||||
ethnicity, religion and so on are avoided as much as possible.
|
||||
This is for inclusiveness, professionality, and also privacy and security.
|
||||
This likely means that apart from usernames (which frequently unavoidably hint
|
||||
about the author's background) there should be no personal information included
|
||||
in contributions.
|
||||
|
||||
Note that we have no contract with contributors. Introducing contributor licence
|
||||
agreements (CLAs) would allow us to enforce additional rules but would also be a
|
||||
scary barrier for new contributors. We don't want to be scary. We love you!
|
369
CONTRIBUTING.md
Normal file
369
CONTRIBUTING.md
Normal file
|
@ -0,0 +1,369 @@
|
|||
# HOW TO CONTRIBUTE TO TQDM
|
||||
|
||||
**TL;DR: Skip to [QUICK DEV SUMMARY]**
|
||||
|
||||
This file describes how to
|
||||
|
||||
- contribute changes to the project, and
|
||||
- upload released to the PyPI repository.
|
||||
|
||||
Most of the management commands have been directly placed inside the
|
||||
Makefile:
|
||||
|
||||
```
|
||||
make [<alias>] # on UNIX-like environments
|
||||
python setup.py make [<alias>] # if make is unavailable
|
||||
```
|
||||
|
||||
The latter depends on [`py-make>=0.1.0`](https://github.com/tqdm/py-make).
|
||||
|
||||
Use the alias `help` (or leave blank) to list all available aliases.
|
||||
|
||||
|
||||
## HOW TO COMMIT CONTRIBUTIONS
|
||||
|
||||
Contributions to the project are made using the "Fork & Pull" model. The
|
||||
typical steps would be:
|
||||
|
||||
1. create an account on [github](https://github.com)
|
||||
2. fork [`tqdm`](https://github.com/tqdm/tqdm)
|
||||
3. make a local clone: `git clone https://github.com/your_account/tqdm.git`
|
||||
4. make changes on the local copy
|
||||
5. test (see below) and commit changes `git commit -a -m "my message"`
|
||||
6. `push` to your GitHub account: `git push origin`
|
||||
7. create a Pull Request (PR) from your GitHub fork
|
||||
(go to your fork's webpage and click on "Pull Request."
|
||||
You can then add a message to describe your proposal.)
|
||||
|
||||
|
||||
## WHAT CODE LAYOUT SHOULD I FOLLOW?
|
||||
|
||||
Don't worry too much - maintainers can help reorganise contributions.
|
||||
However it would be helpful to bear in mind:
|
||||
|
||||
- The standard core of `tqdm`, i.e. [`tqdm.std.tqdm`](tqdm/std.py)
|
||||
+ must have no dependencies apart from pure python built-in standard libraries
|
||||
+ must have negligible impact on performance
|
||||
+ should have 100% coverage by unit tests
|
||||
+ should be appropriately commented
|
||||
+ should have well-formatted docstrings for functions
|
||||
* under 76 chars (incl. initial spaces) to avoid linebreaks in terminal pagers
|
||||
* use two spaces between variable name and colon, specify a type, and most likely state that it's optional: `VAR<space><space>:<space>TYPE[, optional]`
|
||||
* use [default: ...] for default values of keyword arguments
|
||||
+ will not break backward compatibility unless there is a very good reason
|
||||
* e.g. breaking py26 compatibility purely in favour of readability (such as converting `dict(a=1)` to `{'a': 1}`) is not a good enough reason
|
||||
+ API changes should be discussed carefully
|
||||
+ remember, with millions of downloads per month, `tqdm` must be extremely fast and reliable
|
||||
- Any other kind of change may be included in a (possibly new) submodule
|
||||
+ submodules are likely single python files under the main [tqdm/](tqdm/) directory
|
||||
+ submodules extending `tqdm.std.tqdm` or any other module (e.g. [`tqdm.notebook.tqdm`](tqdm/notebook.py), [`tqdm.gui.tqdm`](tqdm/gui.py))
|
||||
+ CLI wrapper `tqdm.cli`
|
||||
* if a newly added `tqdm.std.tqdm` option is not supported by the CLI, append to `tqdm.cli.UNSUPPORTED_OPTS`
|
||||
+ can implement anything from experimental new features to support for third-party libraries such as `pandas`, `numpy`, etc.
|
||||
+ submodule maturity
|
||||
* alpha: experimental; missing unit tests, comments, and/or feedback; raises `tqdm.TqdmExperimentalWarning`
|
||||
* beta: well-used; commented, perhaps still missing tests
|
||||
* stable: >10 users; commented, 80% coverage
|
||||
- `.meta/`
|
||||
+ A "hidden" folder containing helper utilities not strictly part of the `tqdm` distribution itself
|
||||
|
||||
|
||||
## TESTING
|
||||
|
||||
Once again, don't worry too much - tests are automated online, and maintainers
|
||||
can also help.
|
||||
|
||||
To test functionality (such as before submitting a Pull
|
||||
Request), there are a number of unit tests.
|
||||
|
||||
### Standard unit tests
|
||||
|
||||
The standard way to run the tests:
|
||||
|
||||
- install `tox`
|
||||
- `cd` to the root of the `tqdm` directory (in the same folder as this file)
|
||||
- run the following command:
|
||||
|
||||
```
|
||||
[python setup.py] make test
|
||||
# or:
|
||||
tox --skip-missing-interpreters
|
||||
```
|
||||
|
||||
This will build the module and run the tests in a virtual environment.
|
||||
Errors and coverage rates will be output to the console/log. (Ignore missing
|
||||
interpreters errors - these are due to the local machine missing certain
|
||||
versions of Python.)
|
||||
|
||||
Note: to install all versions of the Python interpreter that are specified
|
||||
in [tox.ini](https://github.com/tqdm/tqdm/blob/master/tox.ini),
|
||||
you can use `MiniConda` to install a minimal setup. You must also make sure
|
||||
that each distribution has an alias to call the Python interpreter:
|
||||
`python27` for Python 2.7's interpreter, `python32` for Python 3.2's, etc.
|
||||
|
||||
### Alternative unit tests with pytest
|
||||
|
||||
Alternatively, use `pytest` to run the tests just for the current Python version:
|
||||
|
||||
- install test requirements: `[python setup.py] make install_test`
|
||||
- run the following command:
|
||||
|
||||
```
|
||||
[python setup.py] make alltests
|
||||
```
|
||||
|
||||
|
||||
|
||||
# MANAGE A NEW RELEASE
|
||||
|
||||
This section is intended for the project's maintainers and describes
|
||||
how to build and upload a new release. Once again,
|
||||
`[python setup.py] make [<alias>]` will help.
|
||||
Also consider `pip install`ing development utilities:
|
||||
`[python setup.py] make install_build` at a minimum, or a more thorough `conda env create`.
|
||||
|
||||
|
||||
## Pre-commit Hook
|
||||
|
||||
It's probably a good idea to use the `pre-commit` (`pip install pre-commit`) helper.
|
||||
|
||||
Run `pre-commit install` for convenient local sanity-checking.
|
||||
|
||||
|
||||
## Semantic Versioning
|
||||
|
||||
The `tqdm` repository managers should:
|
||||
|
||||
- follow the [Semantic Versioning](https://semver.org) convention for tagging
|
||||
|
||||
|
||||
## Checking setup.py
|
||||
|
||||
To check that the `setup.py`/`setup.cfg`/`pyproject.toml` file is compliant with PyPI
|
||||
requirements (e.g. version number; reStructuredText in `README.rst`) use:
|
||||
|
||||
```
|
||||
[python setup.py] make testsetup
|
||||
```
|
||||
|
||||
To upload just metadata (including overwriting mistakenly uploaded metadata)
|
||||
to PyPI, use:
|
||||
|
||||
```
|
||||
[python setup.py] make pypimeta
|
||||
```
|
||||
|
||||
|
||||
## Merging Pull Requests
|
||||
|
||||
This section describes how to cleanly merge PRs.
|
||||
|
||||
### 1 Rebase
|
||||
|
||||
From your project repository, merge and test
|
||||
(replace `pr-branch-name` as appropriate):
|
||||
|
||||
```
|
||||
git fetch origin
|
||||
git checkout -b pr-branch-name origin/pr-branch-name
|
||||
git rebase master
|
||||
```
|
||||
|
||||
If there are conflicts:
|
||||
|
||||
```
|
||||
git mergetool
|
||||
git rebase --continue
|
||||
```
|
||||
|
||||
### 2 Push
|
||||
|
||||
Update branch with the rebased history:
|
||||
|
||||
```
|
||||
git push origin pr-branch-name --force
|
||||
```
|
||||
|
||||
Non maintainers can stop here.
|
||||
|
||||
Note: NEVER just `git push --force` (this will push all local branches,
|
||||
overwriting remotes).
|
||||
|
||||
### 3 Merge
|
||||
|
||||
```
|
||||
git checkout master
|
||||
git merge --no-ff pr-branch-name
|
||||
```
|
||||
|
||||
### 4 Test
|
||||
|
||||
```
|
||||
[python setup.py] make alltests
|
||||
```
|
||||
|
||||
### 5 Push to master
|
||||
|
||||
```
|
||||
git push origin master
|
||||
```
|
||||
|
||||
|
||||
## Building a Release and Uploading to PyPI
|
||||
|
||||
Formally publishing requires additional steps: testing and tagging.
|
||||
|
||||
### Test
|
||||
|
||||
Ensure that all online CI tests have passed.
|
||||
|
||||
### Tag
|
||||
|
||||
- ensure the version has been tagged.
|
||||
The tag format is `v{major}.{minor}.{patch}`, for example: `v4.4.1`.
|
||||
The current commit's tag is used in the version checking process.
|
||||
If the current commit is not tagged appropriately, the version will
|
||||
display as `v{major}.{minor}.{patch}.dev{N}+g{commit_hash}`.
|
||||
|
||||
### Upload
|
||||
|
||||
GitHub Actions (GHA) CI should automatically do this after pushing tags.
|
||||
Manual instructions are given below in case of failure.
|
||||
|
||||
Build `tqdm` into a distributable python package:
|
||||
|
||||
```
|
||||
[python setup.py] make build
|
||||
```
|
||||
|
||||
This will generate several builds in the `dist/` folder. On non-windows
|
||||
machines the windows `exe` installer may fail to build. This is normal.
|
||||
|
||||
Finally, upload everything to PyPI. This can be done easily using the
|
||||
[twine](https://github.com/pypa/twine) module:
|
||||
|
||||
```
|
||||
[python setup.py] make pypi
|
||||
```
|
||||
|
||||
Also, the new release can (should) be added to GitHub by creating a new
|
||||
release from the [web interface](https://github.com/tqdm/tqdm/releases);
|
||||
uploading packages from the `dist/` folder
|
||||
created by `[python setup.py] make build`.
|
||||
The [wiki] can be automatically updated with GitHub release notes by
|
||||
running `make` within the wiki repository.
|
||||
|
||||
[wiki]: https://github.com/tqdm/tqdm/wiki
|
||||
|
||||
Docker images may be uploaded to <https://hub.docker.com/r/tqdm/tqdm>.
|
||||
Assuming `docker` is
|
||||
[installed](https://docs.docker.com/install/linux/docker-ce/ubuntu/):
|
||||
|
||||
```
|
||||
make -B docker
|
||||
docker login
|
||||
docker push tqdm/tqdm:latest
|
||||
docker push tqdm/tqdm:$(docker run -i --rm tqdm/tqdm -v)
|
||||
```
|
||||
|
||||
Snaps may be uploaded to <https://snapcraft.io/tqdm>.
|
||||
Assuming `snapcraft` is installed (`snap install snapcraft --classic --beta`):
|
||||
|
||||
```
|
||||
make snap
|
||||
snapcraft login
|
||||
snapcraft push tqdm*.snap --release stable
|
||||
```
|
||||
|
||||
### Notes
|
||||
|
||||
- you can also test on the PyPI test servers `test.pypi.org`
|
||||
before the real deployment
|
||||
- in case of a mistake, you can delete an uploaded release on PyPI, but you
|
||||
cannot re-upload another with the same version number
|
||||
- in case of a mistake in the metadata on PyPI (e.g. bad README),
|
||||
updating just the metadata is possible: `[python setup.py] make pypimeta`
|
||||
|
||||
|
||||
## Updating Websites
|
||||
|
||||
The most important file is `.readme.rst`, which should always be kept up-to-date
|
||||
and in sync with the in-line source documentation. This will affect all of the
|
||||
following:
|
||||
|
||||
- `README.rst` (generated by `mkdocs.py` during `make build`)
|
||||
- The [main repository site](https://github.com/tqdm/tqdm) which automatically
|
||||
serves the latest `README.rst` as well as links to all of GitHub's features.
|
||||
This is the preferred online referral link for `tqdm`.
|
||||
- The [PyPI mirror](https://pypi.org/project/tqdm) which automatically
|
||||
serves the latest release built from `README.rst` as well as links to past
|
||||
releases.
|
||||
- Many external web crawlers.
|
||||
|
||||
Additionally (less maintained), there exists:
|
||||
|
||||
- A [wiki] which is publicly editable.
|
||||
- The [gh-pages project] which is built from the
|
||||
[gh-pages branch](https://github.com/tqdm/tqdm/tree/gh-pages), which is
|
||||
built using [asv](https://github.com/airspeed-velocity/asv).
|
||||
- The [gh-pages root] which is built from a separate
|
||||
[github.io repo](https://github.com/tqdm/tqdm.github.io).
|
||||
|
||||
[gh-pages project]: https://tqdm.github.io/tqdm/
|
||||
[gh-pages root]: https://tqdm.github.io/
|
||||
|
||||
|
||||
## Helper Bots
|
||||
|
||||
There are some helpers in
|
||||
[.github/workflows](https://github.com/tqdm/tqdm/tree/master/.github/workflows)
|
||||
to assist with maintenance.
|
||||
|
||||
- Comment Bot
|
||||
+ allows maintainers to write `/tag vM.m.p commit_hash` in an issue/PR to create a tag
|
||||
- Post Release
|
||||
+ automatically updates the [wiki]
|
||||
+ automatically updates the [gh-pages root]
|
||||
- Benchmark
|
||||
+ automatically updates the [gh-pages project]
|
||||
|
||||
|
||||
## QUICK DEV SUMMARY
|
||||
|
||||
For experienced devs, once happy with local master, follow the steps below.
|
||||
Much is automated so really it's steps 1-5, then 11(a).
|
||||
|
||||
1. test (`[python setup.py] make alltests` or rely on `pre-commit`)
|
||||
2. `git commit [--amend] # -m "bump version"`
|
||||
3. `git push`
|
||||
4. wait for tests to pass
|
||||
a) in case of failure, fix and go back to (1)
|
||||
5. `git tag vM.m.p && git push --tags` or comment `/tag vM.m.p commit_hash`
|
||||
6. **`[AUTO:GHA]`** `[python setup.py] make distclean`
|
||||
7. **`[AUTO:GHA]`** `[python setup.py] make build`
|
||||
8. **`[AUTO:GHA]`** upload to PyPI. either:
|
||||
a) `[python setup.py] make pypi`, or
|
||||
b) `twine upload -s -i $(git config user.signingkey) dist/tqdm-*`
|
||||
9. **`[AUTO:GHA]`** upload to docker hub:
|
||||
a) `make -B docker`
|
||||
b) `docker push tqdm/tqdm:latest`
|
||||
c) `docker push tqdm/tqdm:$(docker run -i --rm tqdm/tqdm -v)`
|
||||
10. **`[AUTO:GHA]`** upload to snapcraft:
|
||||
a) `make snap`, and
|
||||
b) `snapcraft push tqdm*.snap --release stable`
|
||||
11. Wait for GHA to draft a new release on <https://github.com/tqdm/tqdm/releases>
|
||||
a) replace the commit history with helpful release notes, and click publish
|
||||
b) **`[AUTO:GHA]`** attach `dist/tqdm-*` binaries
|
||||
(usually only `*.whl*`)
|
||||
12. **`[SUB][AUTO:GHA-rel]`** run `make` in the `wiki` submodule to update release notes
|
||||
13. **`[SUB][AUTO:GHA-rel]`** run `make deploy` in the `docs` submodule to update website
|
||||
14. **`[SUB][AUTO:GHA-rel]`** accept the automated PR in the `feedstock` submodule to update conda
|
||||
15. **`[AUTO:GHA-rel]`** update the [gh-pages project] benchmarks
|
||||
a) `[python setup.py] make testasvfull`
|
||||
b) `asv gh-pages`
|
||||
|
||||
Key:
|
||||
|
||||
- **`[AUTO:GHA]`**: GitHub Actions CI should automatically do this after `git push --tags` (5)
|
||||
- **`[AUTO:GHA-rel]`**: GitHub Actions CI should automatically do this after release (11a)
|
||||
- **`[SUB]`**: Requires one-time `make submodules` to clone `docs`, `wiki`, and `feedstock`
|
890
DEMO.ipynb
Normal file
890
DEMO.ipynb
Normal file
|
@ -0,0 +1,890 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<h1 align=\"center\">tqdm</h1>\n",
|
||||
"<img src=\"https://img.tqdm.ml/logo.gif\" align=\"left\" />\n",
|
||||
"\n",
|
||||
"[![Py-Versions](https://img.shields.io/pypi/pyversions/tqdm.svg?logo=python&logoColor=white)](https://pypi.org/project/tqdm)|[![Versions](https://img.shields.io/pypi/v/tqdm.svg)](https://tqdm.github.io/releases)|[![Conda-Forge-Status](https://img.shields.io/conda/v/conda-forge/tqdm.svg?label=conda-forge&logo=conda-forge)](https://anaconda.org/conda-forge/tqdm)|[![Docker](https://img.shields.io/badge/docker-pull-blue.svg?logo=docker&logoColor=white)](https://hub.docker.com/r/tqdm/tqdm)|[![Snapcraft](https://img.shields.io/badge/snap-install-82BEA0.svg?logo=snapcraft)](https://snapcraft.io/tqdm)\n",
|
||||
"-|-|-|-|-\n",
|
||||
"\n",
|
||||
"[![Build-Status](https://img.shields.io/github/workflow/status/tqdm/tqdm/Test/master?logo=GitHub)](https://github.com/tqdm/tqdm/actions?query=workflow%3ATest)|[![Coverage-Status](https://img.shields.io/coveralls/github/tqdm/tqdm/master?logo=coveralls)](https://coveralls.io/github/tqdm/tqdm)|[![Branch-Coverage-Status](https://codecov.io/gh/tqdm/tqdm/branch/master/graph/badge.svg)](https://codecov.io/gh/tqdm/tqdm)|[![Codacy-Grade](https://app.codacy.com/project/badge/Grade/3f965571598f44549c7818f29cdcf177)](https://www.codacy.com/gh/tqdm/tqdm/dashboard)|[![Libraries-Rank](https://img.shields.io/librariesio/sourcerank/pypi/tqdm.svg?logo=koding&logoColor=white)](https://libraries.io/pypi/tqdm)|[![PyPI-Downloads](https://img.shields.io/pypi/dm/tqdm.svg?label=pypi%20downloads&logo=PyPI&logoColor=white)](https://pepy.tech/project/tqdm)\n",
|
||||
"-|-|-|-|-|-\n",
|
||||
"\n",
|
||||
"[![DOI](https://img.shields.io/badge/DOI-10.5281/zenodo.595120-blue.svg)](https://doi.org/10.5281/zenodo.595120)|[![LICENCE](https://img.shields.io/pypi/l/tqdm.svg)](https://raw.githubusercontent.com/tqdm/tqdm/master/LICENCE)|[![OpenHub-Status](https://www.openhub.net/p/tqdm/widgets/project_thin_badge?format=gif)](https://www.openhub.net/p/tqdm?ref=Thin+badge)|[![binder-demo](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/tqdm/tqdm/master?filepath=DEMO.ipynb)|[![awesome-python](https://awesome.re/mentioned-badge.svg)](https://github.com/vinta/awesome-python)\n",
|
||||
"-|-|-|-|-\n",
|
||||
"\n",
|
||||
"`tqdm` derives from the Arabic word *taqaddum* (تقدّم) which can mean\n",
|
||||
"\"progress,\" and is an abbreviation for \"I love you so much\" in Spanish\n",
|
||||
"(*te quiero demasiado*).\n",
|
||||
"\n",
|
||||
"Instantly make your loops show a smart progress meter - just wrap any\n",
|
||||
"iterable with `tqdm(iterable)`, and you're done!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm import tqdm\n",
|
||||
"for i in tqdm(range(10000)):\n",
|
||||
" pass"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"`trange(N)` can be also used as a convenient shortcut for\n",
|
||||
"`tqdm(xrange(N))`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm import trange\n",
|
||||
"for i in trange(10000, unit_scale=True, desc=\"hello\", unit=\"epoch\"):\n",
|
||||
" pass"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"![Screenshot](https://img.tqdm.ml/tqdm.gif)|[![Video](https://img.tqdm.ml/video.jpg)](https://tqdm.github.io/video) [![Slides](https://img.tqdm.ml/slides.jpg)](https://tqdm.github.io/PyData2019/slides.html) [![Merch](https://img.tqdm.ml/merch.jpg)](https://tqdm.github.io/merch)\n",
|
||||
"-|-\n",
|
||||
"\n",
|
||||
"It can also be executed as a module with pipes:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! seq 9999999 | tqdm --bytes | wc -l"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"```sh\n",
|
||||
"tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
|
||||
" 44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s]\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Overhead is low -- about 60ns per iteration (80ns with `tqdm.gui`), and\n",
|
||||
"is unit tested against performance regression. By comparison, the\n",
|
||||
"well-established\n",
|
||||
"[ProgressBar](https://github.com/niltonvolpato/python-progressbar) has\n",
|
||||
"an 800ns/iter overhead.\n",
|
||||
"\n",
|
||||
"In addition to its low overhead, `tqdm` uses smart algorithms to predict\n",
|
||||
"the remaining time and to skip unnecessary iteration displays, which\n",
|
||||
"allows for a negligible overhead in most cases.\n",
|
||||
"\n",
|
||||
"`tqdm` works on any platform (Linux, Windows, Mac, FreeBSD, NetBSD,\n",
|
||||
"Solaris/SunOS), in any console or in a GUI, and is also friendly with\n",
|
||||
"IPython/Jupyter notebooks.\n",
|
||||
"\n",
|
||||
"`tqdm` does not require any dependencies (not even `curses`!), just\n",
|
||||
"Python and an environment supporting `carriage return \\r` and\n",
|
||||
"`line feed \\n` control characters.\n",
|
||||
"\n",
|
||||
"---\n",
|
||||
"\n",
|
||||
"## Usage\n",
|
||||
"\n",
|
||||
"`tqdm` is very versatile and can be used in a number of ways.\n",
|
||||
"The three main ones are given below.\n",
|
||||
"\n",
|
||||
"### Iterable-based\n",
|
||||
"\n",
|
||||
"Wrap `tqdm()` around any iterable:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm import tqdm\n",
|
||||
"from time import sleep"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"text = \"\"\n",
|
||||
"for char in tqdm([\"a\", \"b\", \"c\", \"d\"]):\n",
|
||||
" sleep(0.25)\n",
|
||||
" text = text + char"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"`trange(i)` is a special optimised instance of `tqdm(range(i))`:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm import trange\n",
|
||||
"\n",
|
||||
"for i in trange(100):\n",
|
||||
" sleep(0.01)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Instantiation outside of the loop allows for manual control over `tqdm()`:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"pbar = tqdm([\"a\", \"b\", \"c\", \"d\"])\n",
|
||||
"for char in pbar:\n",
|
||||
" sleep(0.25)\n",
|
||||
" pbar.set_description(\"Processing %s\" % char)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Manual\n",
|
||||
"\n",
|
||||
"Manual control of `tqdm()` updates using a `with` statement:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"with tqdm(total=100) as pbar:\n",
|
||||
" for i in range(10):\n",
|
||||
" sleep(0.1)\n",
|
||||
" pbar.update(10)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If the optional variable `total` (or an iterable with `len()`) is\n",
|
||||
"provided, predictive stats are displayed.\n",
|
||||
"\n",
|
||||
"`with` is also optional (you can just assign `tqdm()` to a variable,\n",
|
||||
"but in this case don't forget to `del` or `close()` at the end:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"pbar = tqdm(total=100)\n",
|
||||
"for i in range(10):\n",
|
||||
" sleep(0.1)\n",
|
||||
" pbar.update(10)\n",
|
||||
"pbar.close()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Module"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Perhaps the most wonderful use of `tqdm` is in a script or on the\n",
|
||||
"command line. Simply inserting `tqdm` (or `python -m tqdm`) between\n",
|
||||
"pipes will pass through all `stdin` to `stdout` while printing progress\n",
|
||||
"to `stderr`.\n",
|
||||
"\n",
|
||||
"The example below demonstrated counting the number of lines in all\n",
|
||||
"Python files in the current directory, with timing information included."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! time find . -name '*.py' -type f -exec cat \\{} \\; | wc -l"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! time find . -name '*.py' -type f -exec cat \\{} \\; | tqdm | wc -l"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Note that the usual arguments for `tqdm` can also be specified."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! find . -name '*.py' -type f -exec cat \\{} \\; | tqdm --unit loc --unit-scale --total 4104300 --null"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Backing up a large directory?\n",
|
||||
"\n",
|
||||
"```sh\n",
|
||||
"tar -zcf - docs/ | tqdm --bytes --total `du -sb docs/ | cut -f1` > backup.tgz\n",
|
||||
" 44%|██████████████▊ | 153M/352M [00:14<00:18, 11.0MB/s]\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"This can be beautified further:\n",
|
||||
"\n",
|
||||
"```sh\n",
|
||||
"BYTES=\"$(du -sb docs/ | cut -f1)\"\n",
|
||||
"tar -cf - docs/ \\\n",
|
||||
" | tqdm --bytes --total \"$BYTES\" --desc Processing | gzip \\\n",
|
||||
" | tqdm --bytes --total \"$BYTES\" --desc Compressed --position 1 \\\n",
|
||||
" > ~/backup.tgz\n",
|
||||
"Processing: 100%|██████████████████████| 352M/352M [00:14<00:00, 30.2MB/s]\n",
|
||||
"Compressed: 42%|█████████▎ | 148M/352M [00:14<00:19, 10.9MB/s]\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Or done on a file level using 7-zip:\n",
|
||||
"\n",
|
||||
"```sh\n",
|
||||
"7z a -bd -r backup.7z docs/ | grep Compressing \\\n",
|
||||
" | tqdm --total $(find docs/ -type f | wc -l) --unit files \\\n",
|
||||
" | grep -v Compressing\n",
|
||||
"100%|██████████████████████████▉| 15327/15327 [01:00<00:00, 712.96files/s]\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Documentation"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"tqdm?"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! tqdm --help"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Examples and Advance Usage\n",
|
||||
"\n",
|
||||
"- See the [examples](https://github.com/tqdm/tqdm/tree/master/examples)\n",
|
||||
" folder;\n",
|
||||
"- import the module and run `help()`;\n",
|
||||
"- consult the [wiki](https://github.com/tqdm/tqdm/wiki)\n",
|
||||
" - this has an\n",
|
||||
" [excellent article](https://github.com/tqdm/tqdm/wiki/How-to-make-a-great-Progress-Bar)\n",
|
||||
" on how to make a **great** progressbar;\n",
|
||||
"- check out the [slides from PyData London](https://tqdm.github.io/PyData2019/slides.html), or\n",
|
||||
"- run this file!\n",
|
||||
"\n",
|
||||
"### Description and additional stats\n",
|
||||
"\n",
|
||||
"Custom information can be displayed and updated dynamically on `tqdm` bars\n",
|
||||
"with the `desc` and `postfix` arguments:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm import tqdm, trange\n",
|
||||
"from random import random, randint\n",
|
||||
"from time import sleep\n",
|
||||
"\n",
|
||||
"with trange(10) as t:\n",
|
||||
" for i in t:\n",
|
||||
" # Description will be displayed on the left\n",
|
||||
" t.set_description('GEN %i' % i)\n",
|
||||
" # Postfix will be displayed on the right,\n",
|
||||
" # formatted automatically based on argument's datatype\n",
|
||||
" t.set_postfix(loss=random(), gen=randint(1,999), str='h',\n",
|
||||
" lst=[1, 2])\n",
|
||||
" sleep(0.1)\n",
|
||||
"\n",
|
||||
"with tqdm(total=10, bar_format=\"{postfix[0]} {postfix[1][value]:>8.2g}\",\n",
|
||||
" postfix=[\"Batch\", dict(value=0)]) as t:\n",
|
||||
" for i in range(10):\n",
|
||||
" sleep(0.1)\n",
|
||||
" t.postfix[1][\"value\"] = i / 2\n",
|
||||
" t.update()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Points to remember when using `{postfix[...]}` in the `bar_format` string:\n",
|
||||
"\n",
|
||||
"- `postfix` also needs to be passed as an initial argument in a\n",
|
||||
" compatible format, and\n",
|
||||
"- `postfix` will be auto-converted to a string if it is a `dict`-like\n",
|
||||
" object. To prevent this behaviour, insert an extra item into the\n",
|
||||
" dictionary where the key is not a string.\n",
|
||||
"\n",
|
||||
"Additional `bar_format` parameters may also be defined by overriding\n",
|
||||
"`format_dict`, and the bar itself may be modified using `ascii`:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm import tqdm\n",
|
||||
"from time import sleep\n",
|
||||
"\n",
|
||||
"class TqdmExtraFormat(tqdm):\n",
|
||||
" \"\"\"Provides a `total_time` format parameter\"\"\"\n",
|
||||
" @property\n",
|
||||
" def format_dict(self):\n",
|
||||
" d = super(TqdmExtraFormat, self).format_dict\n",
|
||||
" total_time = d[\"elapsed\"] * (d[\"total\"] or 0) / max(d[\"n\"], 1)\n",
|
||||
" d.update(total_time=self.format_interval(total_time) + \" in total\")\n",
|
||||
" return d\n",
|
||||
"\n",
|
||||
"for i in TqdmExtraFormat(\n",
|
||||
" range(9), ascii=\" .oO0\",\n",
|
||||
" bar_format=\"{total_time}: {percentage:.0f}%|{bar}{r_bar}\"):\n",
|
||||
" if i == 4:\n",
|
||||
" break"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Note that `{bar}` also supports a format specifier `[width][type]`.\n",
|
||||
"\n",
|
||||
"- `width`\n",
|
||||
" + unspecified (default): automatic to fill `ncols`\n",
|
||||
" + `int >= 0`: fixed width overriding `ncols` logic\n",
|
||||
" + `int < 0`: subtract from the automatic default\n",
|
||||
"- `type`\n",
|
||||
" + `a`: ascii (`ascii=True` override)\n",
|
||||
" + `u`: unicode (`ascii=False` override)\n",
|
||||
" + `b`: blank (`ascii=\" \"` override)\n",
|
||||
"\n",
|
||||
"This means a fixed bar with right-justified text may be created by\n",
|
||||
"using: `bar_format=\"{l_bar}{bar:10}|{bar:-10b}right-justified\"`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Nested progress bars\n",
|
||||
"\n",
|
||||
"`tqdm` supports nested progress bars. Here's an example:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm.auto import trange\n",
|
||||
"from time import sleep\n",
|
||||
"\n",
|
||||
"for i in trange(4, desc='1st loop'):\n",
|
||||
" for j in trange(5, desc='2nd loop'):\n",
|
||||
" for k in trange(50, desc='3rd loop', leave=False):\n",
|
||||
" sleep(0.01)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"For manual control over positioning (e.g. for multi-processing use),\n",
|
||||
"you may specify `position=n` where `n=0` for the outermost bar, `n=1`\n",
|
||||
"for the next, and so on. However, it's best to check if tqdm can work\n",
|
||||
"without manual position first.\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"from time import sleep\n",
|
||||
"from tqdm import trange, tqdm\n",
|
||||
"from multiprocessing import Pool, RLock, freeze_support\n",
|
||||
"\n",
|
||||
"L = list(range(9))\n",
|
||||
"\n",
|
||||
"def progresser(n):\n",
|
||||
" interval = 0.001 / (n + 2)\n",
|
||||
" total = 5000\n",
|
||||
" text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
|
||||
" for _ in trange(total, desc=text, position=n):\n",
|
||||
" sleep(interval)\n",
|
||||
"\n",
|
||||
"if __name__ == '__main__':\n",
|
||||
" freeze_support() # for Windows support\n",
|
||||
" tqdm.set_lock(RLock()) # for managing output contention\n",
|
||||
" p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))\n",
|
||||
" p.map(progresser, L)\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Note that in Python 3, `tqdm.write` is thread-safe:\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"from time import sleep\n",
|
||||
"from tqdm import tqdm, trange\n",
|
||||
"from concurrent.futures import ThreadPoolExecutor\n",
|
||||
"\n",
|
||||
"L = list(range(9))\n",
|
||||
"\n",
|
||||
"def progresser(n):\n",
|
||||
" interval = 0.001 / (n + 2)\n",
|
||||
" total = 5000\n",
|
||||
" text = \"#{}, est. {:<04.2}s\".format(n, interval * total)\n",
|
||||
" for _ in trange(total, desc=text):\n",
|
||||
" sleep(interval).auto\n",
|
||||
" if n == 6:\n",
|
||||
" tqdm.write(\"n == 6 completed.\")\n",
|
||||
" tqdm.write(\"`tqdm.write()` is thread-safe in py3!\")\n",
|
||||
"\n",
|
||||
"if __name__ == '__main__':\n",
|
||||
" with ThreadPoolExecutor() as p:\n",
|
||||
" p.map(progresser, L)\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Hooks and callbacks\n",
|
||||
"\n",
|
||||
"`tqdm` can easily support callbacks/hooks and manual updates.\n",
|
||||
"Here's an example with `urllib`:\n",
|
||||
"\n",
|
||||
"**`urllib.urlretrieve` documentation**\n",
|
||||
"\n",
|
||||
"> [...]\n",
|
||||
"> If present, the hook function will be called once\n",
|
||||
"> on establishment of the network connection and once after each block read\n",
|
||||
"> thereafter. The hook will be passed three arguments; a count of blocks\n",
|
||||
"> transferred so far, a block size in bytes, and the total size of the file.\n",
|
||||
"> [...]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import urllib, os\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"urllib = getattr(urllib, 'request', urllib)\n",
|
||||
"\n",
|
||||
"class TqdmUpTo(tqdm):\n",
|
||||
" \"\"\"Provides `update_to(n)` which uses `tqdm.update(delta_n)`.\"\"\"\n",
|
||||
" def update_to(self, b=1, bsize=1, tsize=None):\n",
|
||||
" \"\"\"\n",
|
||||
" b : int, optional\n",
|
||||
" Number of blocks transferred so far [default: 1].\n",
|
||||
" bsize : int, optional\n",
|
||||
" Size of each block (in tqdm units) [default: 1].\n",
|
||||
" tsize : int, optional\n",
|
||||
" Total size (in tqdm units). If [default: None] remains unchanged.\n",
|
||||
" \"\"\"\n",
|
||||
" if tsize is not None:\n",
|
||||
" self.total = tsize\n",
|
||||
" return self.update(b * bsize - self.n) # also sets self.n = b * bsize\n",
|
||||
"\n",
|
||||
"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
|
||||
"with TqdmUpTo(unit='B', unit_scale=True, unit_divisor=1024, miniters=1,\n",
|
||||
" desc=eg_link.split('/')[-1]) as t: # all optional kwargs\n",
|
||||
" urllib.urlretrieve(eg_link, filename=os.devnull,\n",
|
||||
" reporthook=t.update_to, data=None)\n",
|
||||
" t.total = t.n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Inspired by [twine#242](https://github.com/pypa/twine/pull/242).\n",
|
||||
"Functional alternative in\n",
|
||||
"[examples/tqdm_wget.py](https://github.com/tqdm/tqdm/blob/master/examples/tqdm_wget.py).\n",
|
||||
"\n",
|
||||
"It is recommend to use `miniters=1` whenever there is potentially large\n",
|
||||
"differences in iteration speed (e.g. downloading a file over a patchy\n",
|
||||
"connection).\n",
|
||||
"\n",
|
||||
"**Wrapping read/write methods**\n",
|
||||
"\n",
|
||||
"To measure throughput through a file-like object's `read` or `write`\n",
|
||||
"methods, use `CallbackIOWrapper`:\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"from tqdm.utils import CallbackIOWrapper\n",
|
||||
"\n",
|
||||
"with tqdm(total=file_obj.size,\n",
|
||||
" unit='B', unit_scale=True, unit_divisor=1024) as t:\n",
|
||||
" fobj = CallbackIOWrapper(t.update, file_obj, \"read\")\n",
|
||||
" while True:\n",
|
||||
" chunk = fobj.read(chunk_size)\n",
|
||||
" if not chunk:\n",
|
||||
" break\n",
|
||||
" t.reset()\n",
|
||||
" # ... continue to use `t` for something else\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Alternatively, use the even simpler `wrapattr` convenience function,\n",
|
||||
"which would condense both the `urllib` and `CallbackIOWrapper` examples\n",
|
||||
"down to:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import urllib, os\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
|
||||
"response = getattr(urllib, 'request', urllib).urlopen(eg_link)\n",
|
||||
"with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n",
|
||||
" miniters=1, desc=eg_link.split('/')[-1],\n",
|
||||
" total=getattr(response, 'length', None)) as fout:\n",
|
||||
" for chunk in response:\n",
|
||||
" fout.write(chunk)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The `requests` equivalent is nearly identical:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import requests, os\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"eg_link = \"https://caspersci.uk.to/matryoshka.zip\"\n",
|
||||
"response = requests.get(eg_link, stream=True)\n",
|
||||
"with tqdm.wrapattr(open(os.devnull, \"wb\"), \"write\",\n",
|
||||
" miniters=1, desc=eg_link.split('/')[-1],\n",
|
||||
" total=int(response.headers.get('content-length', 0))) as fout:\n",
|
||||
" for chunk in response.iter_content(chunk_size=4096):\n",
|
||||
" fout.write(chunk)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Pandas Integration\n",
|
||||
"\n",
|
||||
"Due to popular demand we've added support for `pandas` -- here's an example\n",
|
||||
"for `DataFrame.progress_apply` and `DataFrameGroupBy.progress_apply`:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import pandas as pd\n",
|
||||
"import numpy as np\n",
|
||||
"from tqdm import tqdm\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))\n",
|
||||
"\n",
|
||||
"# Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm`\n",
|
||||
"# (can use `tqdm.gui.tqdm`, `tqdm.notebook.tqdm`, optional kwargs, etc.)\n",
|
||||
"tqdm.pandas(desc=\"my bar!\")\n",
|
||||
"\n",
|
||||
"# Now you can use `progress_apply` instead of `apply`\n",
|
||||
"# and `progress_map` instead of `map`\n",
|
||||
"df.progress_apply(lambda x: x**2)\n",
|
||||
"# can also groupby:\n",
|
||||
"# df.groupby(0).progress_apply(lambda x: x**2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In case you're interested in how this works (and how to modify it for\n",
|
||||
"your own callbacks), see the\n",
|
||||
"[examples](https://github.com/tqdm/tqdm/tree/master/examples) folder or\n",
|
||||
"import the module and run `help()`.\n",
|
||||
"\n",
|
||||
"### Keras Integration\n",
|
||||
"\n",
|
||||
"A `keras` callback is also available:\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"from tqdm.keras import TqdmCallback\n",
|
||||
"\n",
|
||||
"...\n",
|
||||
"\n",
|
||||
"model.fit(..., verbose=0, callbacks=[TqdmCallback()])\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"### IPython/Jupyter Integration\n",
|
||||
"\n",
|
||||
"IPython/Jupyter is supported via the `tqdm.notebook` submodule:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm.notebook import trange, tqdm\n",
|
||||
"from time import sleep\n",
|
||||
"\n",
|
||||
"for i in trange(3, desc='1st loop'):\n",
|
||||
" for j in tqdm(range(100), desc='2nd loop'):\n",
|
||||
" sleep(0.01)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In addition to `tqdm` features, the submodule provides a native Jupyter\n",
|
||||
"widget (compatible with IPython v1-v4 and Jupyter), fully working nested\n",
|
||||
"bars and colour hints (blue: normal, green: completed, red:\n",
|
||||
"error/interrupt, light blue: no ETA); as demonstrated below.\n",
|
||||
"\n",
|
||||
"![Screenshot-Jupyter3](https://img.tqdm.ml/jupyter-3.gif)\n",
|
||||
"\n",
|
||||
"The `notebook` version supports percentage or pixels for overall width\n",
|
||||
"(e.g.: `ncols='100%'` or `ncols='480px'`).\n",
|
||||
"\n",
|
||||
"It is also possible to let `tqdm` automatically choose between console\n",
|
||||
"or notebook versions by using the `autonotebook` submodule:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm.autonotebook import tqdm\n",
|
||||
"tqdm.pandas()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Note that this will issue a `TqdmExperimentalWarning` if run in a\n",
|
||||
"notebook since it is not meant to be possible to distinguish between\n",
|
||||
"`jupyter notebook` and `jupyter console`. Use `auto` instead of\n",
|
||||
"`autonotebook` to suppress this warning.\n",
|
||||
"\n",
|
||||
"Note that notebooks will display the bar in the cell where it was\n",
|
||||
"created. This may be a different cell from the one where it is used. If\n",
|
||||
"this is not desired, the creation of the bar must be delayed/moved to\n",
|
||||
"the cell where it is desired to be displayed.\n",
|
||||
"\n",
|
||||
"Another possibility is to have a single bar (near the top of the\n",
|
||||
"notebook) which is constantly re-used (using `reset()` rather than\n",
|
||||
"`close()`). For this reason, the notebook version (unlike the CLI\n",
|
||||
"version) does not automatically call `close()` upon `Exception`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm.notebook import tqdm\n",
|
||||
"pbar = tqdm()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# different cell\n",
|
||||
"iterable = range(100)\n",
|
||||
"pbar.reset(total=len(iterable)) # initialise with new `total`\n",
|
||||
"for i in iterable:\n",
|
||||
" pbar.update()\n",
|
||||
"pbar.refresh() # force print final status but don't `close()`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Writing messages\n",
|
||||
"\n",
|
||||
"This is a work in progress (see\n",
|
||||
"[#737](https://github.com/tqdm/tqdm/issues/737)).\n",
|
||||
"\n",
|
||||
"Since `tqdm` uses a simple printing mechanism to display progress bars,\n",
|
||||
"you should not write any message in the terminal using `print()` while a\n",
|
||||
"progressbar is open.\n",
|
||||
"\n",
|
||||
"To write messages in the terminal without any collision with `tqdm` bar\n",
|
||||
"display, a `.write()` method is provided:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from tqdm.auto import tqdm, trange\n",
|
||||
"from time import sleep\n",
|
||||
"\n",
|
||||
"bar = trange(10)\n",
|
||||
"for i in bar:\n",
|
||||
" # Print using tqdm class method .write()\n",
|
||||
" sleep(0.1)\n",
|
||||
" if not (i % 3):\n",
|
||||
" tqdm.write(\"Done task %i\" % i)\n",
|
||||
" # Can also use bar.write()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"By default, this will print to standard output `sys.stdout`. but you can\n",
|
||||
"specify any file-like object using the `file` argument. For example,\n",
|
||||
"this can be used to redirect the messages writing to a log file or class.\n",
|
||||
"\n",
|
||||
"[![README-Hits](https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&style=social&r=https://github.com/tqdm/tqdm&l=https://img.tqdm.ml/favicon.png&f=https://img.tqdm.ml/logo.gif)](https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot&r=https://github.com/tqdm/tqdm&l=https://img.tqdm.ml/favicon.png&f=https://img.tqdm.ml/logo.gif&style=social)|(Since 19 May 2016)\n",
|
||||
"-|-"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Do your own experiments here 👇\n",
|
||||
"\n",
|
||||
"Try `tqdm` youself by adding your code below and running your own experiments."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import tqdm\n",
|
||||
"\n",
|
||||
"# your code here\n",
|
||||
"tqdm."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython"
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
49
LICENCE
Normal file
49
LICENCE
Normal file
|
@ -0,0 +1,49 @@
|
|||
`tqdm` is a product of collaborative work.
|
||||
Unless otherwise stated, all authors (see commit logs) retain copyright
|
||||
for their respective work, and release the work under the MIT licence
|
||||
(text below).
|
||||
|
||||
Exceptions or notable authors are listed below
|
||||
in reverse chronological order:
|
||||
|
||||
* files: *
|
||||
MPLv2.0 2015-2021 (c) Casper da Costa-Luis
|
||||
[casperdcl](https://github.com/casperdcl).
|
||||
* files: tqdm/_tqdm.py
|
||||
MIT 2016 (c) [PR #96] on behalf of Google Inc.
|
||||
* files: tqdm/_tqdm.py setup.py README.rst MANIFEST.in .gitignore
|
||||
MIT 2013 (c) Noam Yorav-Raphael, original author.
|
||||
|
||||
[PR #96]: https://github.com/tqdm/tqdm/pull/96
|
||||
|
||||
|
||||
Mozilla Public Licence (MPL) v. 2.0 - Exhibit A
|
||||
-----------------------------------------------
|
||||
|
||||
This Source Code Form is subject to the terms of the
|
||||
Mozilla Public License, v. 2.0.
|
||||
If a copy of the MPL was not distributed with this project,
|
||||
You can obtain one at https://mozilla.org/MPL/2.0/.
|
||||
|
||||
|
||||
MIT License (MIT)
|
||||
-----------------
|
||||
|
||||
Copyright (c) 2013 noamraph
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||
this software and associated documentation files (the "Software"), to deal in
|
||||
the Software without restriction, including without limitation the rights to
|
||||
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
||||
the Software, and to permit persons to whom the Software is furnished to do so,
|
||||
subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
||||
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
||||
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
191
Makefile
Normal file
191
Makefile
Normal file
|
@ -0,0 +1,191 @@
|
|||
# IMPORTANT: for compatibility with `python setup.py make [alias]`, ensure:
|
||||
# 1. Every alias is preceded by @[+]make (eg: @make alias)
|
||||
# 2. A maximum of one @make alias or command per line
|
||||
# see: https://github.com/tqdm/py-make/issues/1
|
||||
|
||||
.PHONY:
|
||||
alltests
|
||||
all
|
||||
flake8
|
||||
test
|
||||
pytest
|
||||
testsetup
|
||||
testnb
|
||||
testcoverage
|
||||
testperf
|
||||
testtimer
|
||||
distclean
|
||||
coverclean
|
||||
prebuildclean
|
||||
clean
|
||||
toxclean
|
||||
install_dev
|
||||
install
|
||||
build
|
||||
buildupload
|
||||
pypi
|
||||
snap
|
||||
docker
|
||||
help
|
||||
none
|
||||
run
|
||||
|
||||
help:
|
||||
@python setup.py make -p
|
||||
|
||||
alltests:
|
||||
@+make testcoverage
|
||||
@+make testperf
|
||||
@+make flake8
|
||||
@+make testsetup
|
||||
|
||||
all:
|
||||
@+make alltests
|
||||
@+make build
|
||||
|
||||
flake8:
|
||||
@+pre-commit run -a flake8
|
||||
@+pre-commit run -a nbstripout
|
||||
|
||||
test:
|
||||
TOX_SKIP_ENV=perf tox --skip-missing-interpreters -p all
|
||||
tox -e perf
|
||||
|
||||
pytest:
|
||||
pytest
|
||||
|
||||
testsetup:
|
||||
@make README.rst
|
||||
@make tqdm/tqdm.1
|
||||
@make tqdm/completion.sh
|
||||
python setup.py check --metadata --restructuredtext --strict
|
||||
python setup.py make none
|
||||
|
||||
testnb:
|
||||
pytest tests_notebook.ipynb --nbval --nbval-current-env -W=ignore --nbval-sanitize-with=setup.cfg --cov=tqdm.notebook --cov-report=term
|
||||
|
||||
testcoverage:
|
||||
@make coverclean
|
||||
pytest tests_notebook.ipynb --cov=tqdm --cov-report= --nbval --nbval-current-env --nbval-sanitize-with=setup.cfg -W=ignore
|
||||
pytest -k "not perf" --cov=tqdm --cov-report=xml --cov-report=term --cov-append --cov-fail-under=80
|
||||
|
||||
testperf:
|
||||
# do not use coverage (which is extremely slow)
|
||||
pytest -k perf
|
||||
|
||||
testtimer:
|
||||
pytest
|
||||
|
||||
# another performance test, to check evolution across commits
|
||||
testasv:
|
||||
# Test only the last 3 commits (quick test)
|
||||
asv run -j 8 HEAD~3..HEAD
|
||||
@make viewasv
|
||||
|
||||
testasvfull:
|
||||
# Test all the commits since the beginning (full test)
|
||||
asv run --skip-existing-commits -j 8 v1.0.0..HEAD
|
||||
@make testasv
|
||||
|
||||
viewasv:
|
||||
asv publish
|
||||
asv preview
|
||||
|
||||
tqdm/tqdm.1: .meta/.tqdm.1.md tqdm/cli.py tqdm/std.py
|
||||
# TODO: add to mkdocs.py
|
||||
python -m tqdm --help | tail -n+5 |\
|
||||
sed -r -e 's/\\/\\\\/g' \
|
||||
-e 's/^ (--.*)=<(.*)> : (.*)$$/\n\\\1=*\2*\n: \3./' \
|
||||
-e 's/^ (--.*) : (.*)$$/\n\\\1\n: \2./' \
|
||||
-e 's/ (-.*, )(--.*) /\n\1\\\2\n: /' |\
|
||||
cat "$<" - |\
|
||||
pandoc -o "$@" -s -t man
|
||||
|
||||
tqdm/completion.sh: .meta/mkcompletion.py tqdm/std.py tqdm/cli.py
|
||||
@python .meta/mkcompletion.py
|
||||
|
||||
README.rst: .meta/.readme.rst tqdm/std.py tqdm/cli.py
|
||||
@python .meta/mkdocs.py
|
||||
|
||||
snapcraft.yaml: .meta/mksnap.py
|
||||
@python .meta/mksnap.py
|
||||
|
||||
.dockerignore:
|
||||
@+python -c "fd=open('.dockerignore', 'w'); fd.write('*\n!dist/*.whl\n')"
|
||||
|
||||
Dockerfile:
|
||||
@+python -c 'fd=open("Dockerfile", "w"); fd.write("FROM python:3.8-alpine\nCOPY dist/*.whl .\nRUN pip install -U $$(ls ./*.whl) && rm ./*.whl\nENTRYPOINT [\"tqdm\"]\n")'
|
||||
|
||||
distclean:
|
||||
@+make coverclean
|
||||
@+make prebuildclean
|
||||
@+make clean
|
||||
prebuildclean:
|
||||
@+python -c "import shutil; shutil.rmtree('build', True)"
|
||||
@+python -c "import shutil; shutil.rmtree('dist', True)"
|
||||
@+python -c "import shutil; shutil.rmtree('tqdm.egg-info', True)"
|
||||
@+python -c "import shutil; shutil.rmtree('.eggs', True)"
|
||||
@+python -c "import os; os.remove('tqdm/_dist_ver.py') if os.path.exists('tqdm/_dist_ver.py') else None"
|
||||
coverclean:
|
||||
@+python -c "import os; os.remove('.coverage') if os.path.exists('.coverage') else None"
|
||||
@+python -c "import os, glob; [os.remove(i) for i in glob.glob('.coverage.*')]"
|
||||
@+python -c "import shutil; shutil.rmtree('tests/__pycache__', True)"
|
||||
@+python -c "import shutil; shutil.rmtree('benchmarks/__pycache__', True)"
|
||||
@+python -c "import shutil; shutil.rmtree('tqdm/__pycache__', True)"
|
||||
@+python -c "import shutil; shutil.rmtree('tqdm/contrib/__pycache__', True)"
|
||||
@+python -c "import shutil; shutil.rmtree('tqdm/examples/__pycache__', True)"
|
||||
clean:
|
||||
@+python -c "import os, glob; [os.remove(i) for i in glob.glob('*.py[co]')]"
|
||||
@+python -c "import os, glob; [os.remove(i) for i in glob.glob('tests/*.py[co]')]"
|
||||
@+python -c "import os, glob; [os.remove(i) for i in glob.glob('benchmarks/*.py[co]')]"
|
||||
@+python -c "import os, glob; [os.remove(i) for i in glob.glob('tqdm/*.py[co]')]"
|
||||
@+python -c "import os, glob; [os.remove(i) for i in glob.glob('tqdm/contrib/*.py[co]')]"
|
||||
@+python -c "import os, glob; [os.remove(i) for i in glob.glob('tqdm/examples/*.py[co]')]"
|
||||
toxclean:
|
||||
@+python -c "import shutil; shutil.rmtree('.tox', True)"
|
||||
|
||||
|
||||
submodules:
|
||||
git clone git@github.com:tqdm/tqdm.wiki wiki
|
||||
git clone git@github.com:tqdm/tqdm.github.io docs
|
||||
git clone git@github.com:conda-forge/tqdm-feedstock feedstock
|
||||
cd feedstock && git remote add autotick-bot git@github.com:regro-cf-autotick-bot/tqdm-feedstock
|
||||
|
||||
install:
|
||||
python setup.py install
|
||||
install_dev:
|
||||
python setup.py develop --uninstall
|
||||
python setup.py develop
|
||||
install_build:
|
||||
python -m pip install -r .meta/requirements-dev.txt
|
||||
install_test:
|
||||
python -m pip install -r .meta/requirements-test.txt
|
||||
pre-commit install
|
||||
|
||||
build:
|
||||
@make prebuildclean
|
||||
@make testsetup
|
||||
python setup.py sdist bdist_wheel
|
||||
# python setup.py bdist_wininst
|
||||
|
||||
pypi:
|
||||
twine upload dist/*
|
||||
|
||||
buildupload:
|
||||
@make build
|
||||
@make pypi
|
||||
|
||||
snap:
|
||||
@make -B snapcraft.yaml
|
||||
snapcraft
|
||||
docker:
|
||||
@make build
|
||||
@make .dockerignore
|
||||
@make Dockerfile
|
||||
docker build . -t tqdm/tqdm
|
||||
docker tag tqdm/tqdm:latest tqdm/tqdm:$(shell docker run -i --rm tqdm/tqdm -v)
|
||||
none:
|
||||
# used for unit testing
|
||||
|
||||
run:
|
||||
python -Om tqdm --help
|
1503
README.rst
Normal file
1503
README.rst
Normal file
File diff suppressed because it is too large
Load diff
46
environment.yml
Normal file
46
environment.yml
Normal file
|
@ -0,0 +1,46 @@
|
|||
# development environment
|
||||
name: tqdm
|
||||
channels:
|
||||
- conda-forge
|
||||
- defaults
|
||||
dependencies:
|
||||
# base
|
||||
- python=3
|
||||
- pip
|
||||
- ipykernel
|
||||
- ipywidgets
|
||||
- setuptools
|
||||
- setuptools_scm
|
||||
- toml
|
||||
# test env managers
|
||||
- pre-commit
|
||||
- tox
|
||||
- asv
|
||||
# tests (native)
|
||||
- pytest
|
||||
- pytest-cov
|
||||
- pytest-timeout
|
||||
- pytest-asyncio # [py>=3.7]
|
||||
- nbval
|
||||
- coverage
|
||||
# extras
|
||||
- dask # dask
|
||||
- matplotlib # gui
|
||||
- numpy # pandas, keras, contrib.tenumerate
|
||||
- pandas
|
||||
- tensorflow # keras
|
||||
- slack-sdk # contrib.slack
|
||||
- requests # contrib.telegram
|
||||
- rich # rich
|
||||
- argopt # `cd wiki && pymake`
|
||||
- twine # `pymake pypi`
|
||||
- wheel # `setup.py bdist_wheel`
|
||||
# `cd docs && pymake`
|
||||
- mkdocs-material
|
||||
- pydoc-markdown
|
||||
- pygments
|
||||
- pymdown-extensions
|
||||
- pip:
|
||||
- py-make >=0.1.0 # `setup.py make/pymake`
|
||||
- mkdocs-minify-plugin # `cd docs && pymake`
|
||||
- git+https://github.com/tqdm/jsmin@python3-only#egg=jsmin # `cd docs && pymake`
|
121
examples/7zx.py
Normal file
121
examples/7zx.py
Normal file
|
@ -0,0 +1,121 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
"""Usage:
|
||||
7zx.py [--help | options] <zipfiles>...
|
||||
|
||||
Options:
|
||||
-h, --help Print this help and exit
|
||||
-v, --version Print version and exit
|
||||
-c, --compressed Use compressed (instead of uncompressed) file sizes
|
||||
-s, --silent Do not print one row per zip file
|
||||
-y, --yes Assume yes to all queries (for extraction)
|
||||
-D=<level>, --debug=<level>
|
||||
Print various types of debugging information. Choices:
|
||||
CRITICAL|FATAL
|
||||
ERROR
|
||||
WARN(ING)
|
||||
[default: INFO]
|
||||
DEBUG
|
||||
NOTSET
|
||||
-d, --debug-trace Print lots of debugging information (-D NOTSET)
|
||||
"""
|
||||
from __future__ import print_function
|
||||
|
||||
import io
|
||||
import logging
|
||||
import os
|
||||
import pty
|
||||
import re
|
||||
import subprocess # nosec
|
||||
|
||||
from argopt import argopt
|
||||
|
||||
from tqdm import tqdm
|
||||
|
||||
__author__ = "Casper da Costa-Luis <casper.dcl@physics.org>"
|
||||
__licence__ = "MPLv2.0"
|
||||
__version__ = "0.2.2"
|
||||
__license__ = __licence__
|
||||
|
||||
RE_SCN = re.compile(r"([0-9]+)\s+([0-9]+)\s+(.*)$", flags=re.M)
|
||||
|
||||
|
||||
def main():
|
||||
args = argopt(__doc__, version=__version__).parse_args()
|
||||
if args.debug_trace:
|
||||
args.debug = "NOTSET"
|
||||
logging.basicConfig(level=getattr(logging, args.debug, logging.INFO),
|
||||
format='%(levelname)s:%(message)s')
|
||||
log = logging.getLogger(__name__)
|
||||
log.debug(args)
|
||||
|
||||
# Get compressed sizes
|
||||
zips = {}
|
||||
for fn in args.zipfiles:
|
||||
info = subprocess.check_output(["7z", "l", fn]).strip() # nosec
|
||||
finfo = RE_SCN.findall(info) # size|compressed|name
|
||||
|
||||
# builtin test: last line should be total sizes
|
||||
log.debug(finfo)
|
||||
totals = map(int, finfo[-1][:2])
|
||||
# log.debug(totals)
|
||||
for s in range(2): # size|compressed totals
|
||||
totals_s = sum(map(int, (inf[s] for inf in finfo[:-1])))
|
||||
if totals_s != totals[s]:
|
||||
log.warn("%s: individual total %d != 7z total %d",
|
||||
fn, totals_s, totals[s])
|
||||
fcomp = {n: int(c if args.compressed else u) for (u, c, n) in finfo[:-1]}
|
||||
# log.debug(fcomp)
|
||||
# zips : {'zipname' : {'filename' : int(size)}}
|
||||
zips[fn] = fcomp
|
||||
|
||||
# Extract
|
||||
cmd7zx = ["7z", "x", "-bd"]
|
||||
if args.yes:
|
||||
cmd7zx += ["-y"]
|
||||
log.info("Extracting from %d file(s)", len(zips))
|
||||
with tqdm(total=sum(sum(fcomp.values()) for fcomp in zips.values()),
|
||||
unit="B", unit_scale=True) as tall:
|
||||
for fn, fcomp in zips.items():
|
||||
md, sd = pty.openpty()
|
||||
ex = subprocess.Popen( # nosec
|
||||
cmd7zx + [fn],
|
||||
bufsize=1,
|
||||
stdout=md, # subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT)
|
||||
os.close(sd)
|
||||
with io.open(md, mode="rU", buffering=1) as m:
|
||||
with tqdm(total=sum(fcomp.values()), disable=len(zips) < 2,
|
||||
leave=False, unit="B", unit_scale=True) as t:
|
||||
if not hasattr(t, "start_t"): # disabled
|
||||
t.start_t = tall._time()
|
||||
while True:
|
||||
try:
|
||||
l_raw = m.readline()
|
||||
except IOError:
|
||||
break
|
||||
ln = l_raw.strip()
|
||||
if ln.startswith("Extracting"):
|
||||
exname = ln[len("Extracting"):].lstrip()
|
||||
s = fcomp.get(exname, 0) # 0 is likely folders
|
||||
t.update(s)
|
||||
tall.update(s)
|
||||
elif ln:
|
||||
if not any(
|
||||
ln.startswith(i)
|
||||
for i in ("7-Zip ", "p7zip Version ",
|
||||
"Everything is Ok", "Folders: ",
|
||||
"Files: ", "Size: ", "Compressed: ")):
|
||||
if ln.startswith("Processing archive: "):
|
||||
if not args.silent:
|
||||
t.write(t.format_interval(
|
||||
t.start_t - tall.start_t) + ' ' +
|
||||
ln.replace("Processing archive: ", ""))
|
||||
else:
|
||||
t.write(ln)
|
||||
ex.wait()
|
||||
|
||||
|
||||
main.__doc__ = __doc__
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
38
examples/async_coroutines.py
Normal file
38
examples/async_coroutines.py
Normal file
|
@ -0,0 +1,38 @@
|
|||
"""
|
||||
Asynchronous examples using `asyncio`, `async` and `await` on `python>=3.7`.
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
from tqdm.asyncio import tqdm, trange
|
||||
|
||||
|
||||
def count(start=0, step=1):
|
||||
i = start
|
||||
while True:
|
||||
new_start = yield i
|
||||
if new_start is None:
|
||||
i += step
|
||||
else:
|
||||
i = new_start
|
||||
|
||||
|
||||
async def main():
|
||||
N = int(1e6)
|
||||
async for row in tqdm(trange(N, desc="inner"), desc="outer"):
|
||||
if row >= N:
|
||||
break
|
||||
with tqdm(count(), desc="coroutine", total=N + 2) as pbar:
|
||||
async for row in pbar:
|
||||
if row == N:
|
||||
pbar.send(-10)
|
||||
elif row < 0:
|
||||
assert row == -9
|
||||
break
|
||||
# should be ~1sec rather than ~50s due to async scheduling
|
||||
for i in tqdm.as_completed([asyncio.sleep(0.01 * i)
|
||||
for i in range(100, 0, -1)], desc="as_completed"):
|
||||
await i
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
69
examples/coroutine_pipe.py
Normal file
69
examples/coroutine_pipe.py
Normal file
|
@ -0,0 +1,69 @@
|
|||
"""
|
||||
Inserting `tqdm` as a "pipe" in a chain of coroutines.
|
||||
Not to be confused with `asyncio.coroutine`.
|
||||
"""
|
||||
from functools import wraps
|
||||
|
||||
from tqdm.auto import tqdm
|
||||
|
||||
|
||||
def autonext(func):
|
||||
@wraps(func)
|
||||
def inner(*args, **kwargs):
|
||||
res = func(*args, **kwargs)
|
||||
next(res)
|
||||
return res
|
||||
return inner
|
||||
|
||||
|
||||
@autonext
|
||||
def tqdm_pipe(target, **tqdm_kwargs):
|
||||
"""
|
||||
Coroutine chain pipe `send()`ing to `target`.
|
||||
|
||||
This:
|
||||
>>> r = receiver()
|
||||
>>> p = producer(r)
|
||||
>>> next(r)
|
||||
>>> next(p)
|
||||
|
||||
Becomes:
|
||||
>>> r = receiver()
|
||||
>>> t = tqdm.pipe(r)
|
||||
>>> p = producer(t)
|
||||
>>> next(r)
|
||||
>>> next(p)
|
||||
"""
|
||||
with tqdm(**tqdm_kwargs) as pbar:
|
||||
while True:
|
||||
obj = (yield)
|
||||
target.send(obj)
|
||||
pbar.update()
|
||||
|
||||
|
||||
def source(target):
|
||||
for i in ["foo", "bar", "baz", "pythonista", "python", "py"]:
|
||||
target.send(i)
|
||||
target.close()
|
||||
|
||||
|
||||
@autonext
|
||||
def grep(pattern, target):
|
||||
while True:
|
||||
line = (yield)
|
||||
if pattern in line:
|
||||
target.send(line)
|
||||
|
||||
|
||||
@autonext
|
||||
def sink():
|
||||
while True:
|
||||
line = (yield)
|
||||
tqdm.write(line)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
source(
|
||||
tqdm_pipe(
|
||||
grep('python',
|
||||
sink())))
|
11
examples/include_no_requirements.py
Normal file
11
examples/include_no_requirements.py
Normal file
|
@ -0,0 +1,11 @@
|
|||
# How to import tqdm in any frontend without enforcing it as a dependency
|
||||
try:
|
||||
from tqdm.auto import tqdm
|
||||
except ImportError:
|
||||
|
||||
def tqdm(*args, **kwargs):
|
||||
if args:
|
||||
return args[0]
|
||||
return kwargs.get('iterable', None)
|
||||
|
||||
__all__ = ['tqdm']
|
29
examples/pandas_progress_apply.py
Normal file
29
examples/pandas_progress_apply.py
Normal file
|
@ -0,0 +1,29 @@
|
|||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from tqdm.auto import tqdm
|
||||
|
||||
df = pd.DataFrame(np.random.randint(0, 100, (100000, 6)))
|
||||
|
||||
# Register `pandas.progress_apply` and `pandas.Series.map_apply` with `tqdm`
|
||||
# (can use `tqdm.gui.tqdm`, `tqdm.notebook.tqdm`, optional kwargs, etc.)
|
||||
tqdm.pandas(desc="my bar!")
|
||||
|
||||
# Now you can use `progress_apply` instead of `apply`
|
||||
# and `progress_map` instead of `map`
|
||||
df.progress_apply(lambda x: x**2)
|
||||
# can also groupby:
|
||||
# df.groupby(0).progress_apply(lambda x: x**2)
|
||||
|
||||
# -- Source code for `tqdm_pandas` (really simple!)
|
||||
# def tqdm_pandas(t):
|
||||
# from pandas.core.frame import DataFrame
|
||||
# def inner(df, func, *args, **kwargs):
|
||||
# t.total = groups.size // len(groups)
|
||||
# def wrapper(*args, **kwargs):
|
||||
# t.update(1)
|
||||
# return func(*args, **kwargs)
|
||||
# result = df.apply(wrapper, *args, **kwargs)
|
||||
# t.close()
|
||||
# return result
|
||||
# DataFrame.progress_apply = inner
|
252
examples/paper.bib
Normal file
252
examples/paper.bib
Normal file
|
@ -0,0 +1,252 @@
|
|||
@phdthesis{tqdm-ar,
|
||||
author="Maḥmūd Alī Ġūl",
|
||||
title="Early Southern Arabian Languages and Classical Arabic Sources: A Critical Examination of Literary and Lexicographical Sources by Comparison with the Inscriptions",
|
||||
school="{SOAS} University of London",
|
||||
year="1963"
|
||||
}
|
||||
@misc{tqdm-es,
|
||||
year="2009",
|
||||
title="¿Lenguaje sms que significa esto?",
|
||||
url="https://es.answers.yahoo.com/question/index?qid=20090405052137AAF2YBo&guccounter=1",
|
||||
author="{Yahoo Answers}"
|
||||
}
|
||||
@misc{pypi,
|
||||
year="2019",
|
||||
author="{Python Package Index ({PyPI})}",
|
||||
publisher="Python Software Foundation",
|
||||
title="{tqdm}",
|
||||
url="https://pypi.org/project/tqdm/"
|
||||
}
|
||||
@misc{conda,
|
||||
author="Anaconda",
|
||||
year="2019",
|
||||
title="{tqdm} :: Anaconda Cloud",
|
||||
url="https://anaconda.org/conda-forge/tqdm"
|
||||
}
|
||||
@misc{docker,
|
||||
year="2019",
|
||||
author="{Docker Inc.}",
|
||||
title="{tqdm}/{tqdm} - Docker Hub",
|
||||
url="https://hub.docker.com/r/tqdm/tqdm"
|
||||
}
|
||||
@misc{snapcraft,
|
||||
year="2019",
|
||||
author="Snapcraft",
|
||||
title="Installing {tqdm} for Linux using the Snap Store",
|
||||
url="https://snapcraft.io/tqdm"
|
||||
}
|
||||
@article{zenodo,
|
||||
year="2019",
|
||||
author="Casper O. {da Costa-Luis} and {{tqdm} developers}",
|
||||
title="{tqdm} stable",
|
||||
publisher="Zenodo",
|
||||
doi="10.5281/zenodo.595120"
|
||||
}
|
||||
@misc{notebooks,
|
||||
year="2019",
|
||||
author="{Notebooks {AI}}",
|
||||
title="{tqdm}",
|
||||
url="https://notebooks.ai/demo/gh/tqdm/tqdm"
|
||||
}
|
||||
@misc{binder,
|
||||
year="2019",
|
||||
author="Binder",
|
||||
title="{tqdm}",
|
||||
url="https://mybinder.org/v2/gh/tqdm/tqdm/master?filepath=DEMO.ipynb"
|
||||
}
|
||||
@misc{stdout,
|
||||
year="2019",
|
||||
author="{Stack Overflow}",
|
||||
title="Why is printing to stdout so slow? Can it be sped up?",
|
||||
url="https://stackoverflow.com/questions/3857052/why-is-printing-to-stdout-so-slow-can-it-be-sped-up"
|
||||
}
|
||||
@misc{pypi-downloads,
|
||||
year="2019",
|
||||
author="{Python Packaging Authority ({PyPA})}",
|
||||
publisher="Python Software Foundation",
|
||||
title="Analyzing {PyPI} package downloads -- Python Packaging User Guide",
|
||||
url="https://packaging.python.org/guides/analyzing-pypi-package-downloads/"
|
||||
}
|
||||
@misc{keras,
|
||||
year="2019",
|
||||
author="Ben",
|
||||
title="Keras integration with {tqdm} progress bars",
|
||||
url="https://github.com/bstriner/keras-tqdm"
|
||||
}
|
||||
@misc{tqdm-results,
|
||||
year="2019",
|
||||
author="GitHub",
|
||||
title="{tqdm} Code Results",
|
||||
url="https://github.com/search?q=tqdm&type=Code"
|
||||
}
|
||||
@misc{tqdm-dependents,
|
||||
year="2019",
|
||||
author="GitHub",
|
||||
title="{tqdm} dependents",
|
||||
url="https://github.com/tqdm/tqdm/network/dependents"
|
||||
}
|
||||
@misc{lib-io,
|
||||
year="2019",
|
||||
author="Libraries.io",
|
||||
title="{tqdm} on {PyPI}",
|
||||
url="https://libraries.io/pypi/tqdm"
|
||||
}
|
||||
@misc{sourcerank,
|
||||
year="2019",
|
||||
author="Libraries.io",
|
||||
title="SourceRank Breakdown for {tqdm}",
|
||||
url="https://libraries.io/pypi/tqdm/sourcerank"
|
||||
}
|
||||
@misc{sourcerank-descending,
|
||||
year="2019",
|
||||
author="Libraries.io",
|
||||
title="Libraries - The Open Source Discovery Service",
|
||||
url="https://libraries.io/search?order=desc&platforms=PyPI&sort=rank"
|
||||
}
|
||||
@misc{stars,
|
||||
year="2019",
|
||||
author="GitHub",
|
||||
title="{tqdm} Stargazers",
|
||||
url="https://github.com/tqdm/tqdm/stargazers"
|
||||
}
|
||||
@misc{stars-hist,
|
||||
year="2019",
|
||||
author="{timqian}",
|
||||
title="Star history",
|
||||
url="https://timqian.com/star-history/#tqdm/tqdm"
|
||||
}
|
||||
@misc{trend-hist,
|
||||
year="2018",
|
||||
month="June",
|
||||
day="19",
|
||||
author="Nihey Takizawa",
|
||||
title="GitHub Trending History",
|
||||
url="https://github.com/nihey/trending-history/blob/master/histories/Python.md"
|
||||
}
|
||||
@misc{hits,
|
||||
year="2019",
|
||||
title="{tqdm} hits",
|
||||
url="https://caspersci.uk.to/cgi-bin/hits.cgi?q=tqdm&a=plot",
|
||||
author="Casper O. {da Costa-Luis}"
|
||||
}
|
||||
@book{miller,
|
||||
year="2017",
|
||||
author="Preston Miller and Chapin Bryce",
|
||||
title="Python Digital Forensics Cookbook: Effective Python recipes for digital investigations",
|
||||
publisher="Packt Publishing Ltd",
|
||||
isbn="9781783987474"
|
||||
}
|
||||
@book{boxel,
|
||||
year="2017",
|
||||
author="Dan {Van Boxel}",
|
||||
title="Hands-On Deep Learning with TensorFlow",
|
||||
publisher="Packt Publishing",
|
||||
isbn="9781787125827"
|
||||
}
|
||||
@incollection{nandy,
|
||||
year="2018",
|
||||
author="Abhishek Nandy and Manisha Biswas",
|
||||
title="Reinforcement Learning with Keras, TensorFlow, and ChainerRL",
|
||||
booktitle="Reinforcement Learning : With Open AI, TensorFlow and Keras Using Python",
|
||||
publisher="Apress",
|
||||
isbn="9781484232859",
|
||||
pages="129--153",
|
||||
doi="10.1007/978-1-4842-3285-9_5"
|
||||
}
|
||||
@journal{stein,
|
||||
year="2019",
|
||||
author="Helge S. Stein and Dan Guevarra and Paul F. Newhouse and Edwin Soedarmadji and John M. Gregoire",
|
||||
title="Machine learning of optical properties of materials -- predicting spectra from images and images from spectra",
|
||||
journal="Chemical Science",
|
||||
volume="10",
|
||||
issue="1",
|
||||
pages="47--55",
|
||||
doi="10.1039/C8SC03077D"
|
||||
}
|
||||
@journal{cook,
|
||||
year="2018",
|
||||
author="Neil J. Cook and Aleks Scholz and Ray Jayawardhana",
|
||||
title="Very Low-mass Stars and Brown Dwarfs in Upper Scorpius Using Gaia DR1: Mass Function, Disks, and Kinematics",
|
||||
journal="The Astronomical Journal",
|
||||
volume="154",
|
||||
issue="6",
|
||||
pages="256",
|
||||
doi="10.3847/1538-3881/aa9751",
|
||||
url="https://arxiv.org/abs/1710.11625"
|
||||
}
|
||||
@journal{madhikar,
|
||||
year="2018",
|
||||
author="Pranav Madhikar and Jan Åström and Jan Westerholm and Mikko Karttunen",
|
||||
title="CellSim3D: GPU accelerated software for simulations of cellular growth and division in three dimensions",
|
||||
journal="Computer Physics Communications",
|
||||
volume="232",
|
||||
pages="206--213",
|
||||
doi="10.1016/j.cpc.2018.05.024"
|
||||
}
|
||||
@journal{palmer,
|
||||
year="2018",
|
||||
author="Geraint I. Palmer and Vincent A. Knight and Paul R. Harper and Asyl L. Hawa",
|
||||
title="Ciw: An open-source discrete event simulation library",
|
||||
journal="Journal of Simulation",
|
||||
pages="1--15",
|
||||
doi="10.1080/17477778.2018.1473909"
|
||||
}
|
||||
@journal{knight,
|
||||
year="2016",
|
||||
author="Vincent Knight and Owen Campbell and Marc Harper and Karol Langner and James Campbell and Thomas Campbell and Alex Carney and Martin Chorley and Cameron Davidson-Pilon and Kristian Glass and Nikoleta Glynatsi and Tomáš Ehrlich and Martin Jones and Georgios Koutsovoulos and Holly Tibble and Müller Jochen and Geraint Palmer and Piotr Petunov and Paul Slavin and Timothy Standen and Luis Visintini and Karl Molden",
|
||||
title="An open reproducible framework for the study of the iterated prisoner's dilemma",
|
||||
journal="Journal of Open Research Software",
|
||||
volume="4",
|
||||
doi="10.5334/jors.125",
|
||||
url="https://arxiv.org/abs/1604.00896",
|
||||
issn="2049-9647"
|
||||
}
|
||||
@article{moriwaki,
|
||||
title={Mordred: a molecular descriptor calculator},
|
||||
author={Moriwaki, Hirotomo and Tian, Yu-Shi and Kawashita, Norihito and Takagi, Tatsuya},
|
||||
doi={10.1186/s13321-018-0258-y},
|
||||
number={1},
|
||||
volume={10},
|
||||
month={February},
|
||||
year={2018},
|
||||
journal={Journal of cheminformatics},
|
||||
issn={1758-2946},
|
||||
pages={4}
|
||||
}
|
||||
@article{jackson,
|
||||
title={3D for the people: multi-camera motion capture in the field with consumer-grade cameras and open source software},
|
||||
author={Jackson, Brandon E and Evangelista, Dennis J and Ray, Dylan D and hedrick, Tyson L},
|
||||
doi={10.1242/bio.018713},
|
||||
number={9},
|
||||
volume={5},
|
||||
month={September},
|
||||
year={2016},
|
||||
journal={Biology open},
|
||||
issn={2046-6390},
|
||||
pages={1334--1342}
|
||||
}
|
||||
@misc{travis,
|
||||
year="2019",
|
||||
author="{Travis {CI}}",
|
||||
title="tqdm/tqdm build status",
|
||||
url="https://travis-ci.org/tqdm/tqdm"
|
||||
}
|
||||
@misc{code-review,
|
||||
year="2018",
|
||||
author="Wikipedia",
|
||||
title="List of tools for code review",
|
||||
url="https://en.wikipedia.org/wiki/List_of_tools_for_code_review"
|
||||
}
|
||||
@misc{asv,
|
||||
year="2019",
|
||||
author="{{tqdm} developers}",
|
||||
title="airspeed velocity",
|
||||
url="https://tqdm.github.io/tqdm/"
|
||||
}
|
||||
@misc{licence,
|
||||
year="2019",
|
||||
author="{{tqdm} developers}",
|
||||
title="{tqdm} Licence",
|
||||
url="https://github.com/tqdm/tqdm/blob/master/LICENCE",
|
||||
publisher="GitHub"
|
||||
}
|
169
examples/paper.md
Normal file
169
examples/paper.md
Normal file
|
@ -0,0 +1,169 @@
|
|||
---
|
||||
title: '`tqdm`: A Fast, Extensible Progress Meter for Python and CLI'
|
||||
tags:
|
||||
- progressbar
|
||||
- progressmeter
|
||||
- progress-bar
|
||||
- meter
|
||||
- rate
|
||||
- eta
|
||||
- console
|
||||
- terminal
|
||||
- time
|
||||
- progress
|
||||
- bar
|
||||
- gui
|
||||
- python
|
||||
- parallel
|
||||
- cli
|
||||
- utilities
|
||||
- shell
|
||||
- batch
|
||||
authors:
|
||||
- name: Casper O da Costa-Luis
|
||||
orcid: 0000-0002-7211-1557
|
||||
affiliation: 1
|
||||
affiliations:
|
||||
- name: "Independent (Non-affiliated)"
|
||||
index: 1
|
||||
date: 16 February 2019
|
||||
bibliography: paper.bib
|
||||
---
|
||||
|
||||
# Introduction
|
||||
|
||||
**`tqdm`** is a progress bar library designed to be fast and extensible. It is
|
||||
written in Python, though ports in other languages are available. `tqdm` means
|
||||
**progress** in Arabic (*taqadum* [@tqdm-ar]) and is an abbreviation for
|
||||
**I love you so much** in Spanish (*te quiero demasiado* [@tqdm-es]).
|
||||
|
||||
It is a common programming problem to have iterative operations where progress
|
||||
monitoring is desirable or advantageous. Including statements within a `for` loop to `print` out the current iteration number is a common strategy. However, there are many improvements which could be made in such a scenario:
|
||||
|
||||
- preventing excessive printing, such as only displaying every $n$^th^
|
||||
iteration;
|
||||
- displaying iteration rate;
|
||||
- displaying elapsed and estimated completion times, and
|
||||
- showing all of the above on one continuously updating line.
|
||||
|
||||
Addressing all these issues may well take up more developer time and effort than
|
||||
the rest of the content of the loop. Any changes to iteration rates or attempts
|
||||
to re-use the printing logic in a different loop may well result in suboptimal
|
||||
display rates -- displaying every $n$^th^ iteration may be too (in)frequent --
|
||||
requiring manual adjustment of $n$ to fix.
|
||||
|
||||
`tqdm` addresses all of these problems once and for all, taking advantage of
|
||||
Pythonic patterns to make it a trivial task to add visually appealing,
|
||||
customisable progress bars without any significant performance degradation even
|
||||
in the most demanding of scenarios.
|
||||
|
||||
`tqdm` is intended to be used in frontends (giving end users a visual indication
|
||||
of progress of computations or data transfer). It is also useful for developers
|
||||
for debugging purposes, both as a profiling tool and also as a way of displaying
|
||||
logging information of an iterative task (such as error during training of
|
||||
machine learning algorithms). Due to its ease of use, the library is also an
|
||||
ideal candidate for inclusion in Python educational courses. For general (not
|
||||
necessarily Python) purposes, the command-line interface (CLI) mode further
|
||||
presents a useful tool for CLI users and system administrators monitoring data
|
||||
flow through pipes.
|
||||
|
||||
# Features
|
||||
|
||||
Exhaustive documentation may be found on the project's [home
|
||||
page](https://github.com/tqdm/tqdm/#documentation).
|
||||
|
||||
The two basic use cases are within Python code and within a CLI:
|
||||
|
||||
## Python Iterable Wrapper
|
||||
|
||||
`tqdm`'s primary (and original) use is as a wrapper around Python iterables. A
|
||||
simple case would be:
|
||||
|
||||
```python
|
||||
from tqdm import tqdm
|
||||
from time import sleep
|
||||
for i in tqdm(range(100)):
|
||||
sleep(0.1)
|
||||
100%|#########################################| 100/100 [00:10<00:00, 9.95it/s]
|
||||
```
|
||||
|
||||
Supported features include:
|
||||
|
||||
- Display customisation via arguments such as `desc`, `postfix` and `bar_format`
|
||||
- Automatic limiting of display updates to avoid slowing down due to excessive
|
||||
iteration rates [@stdout]
|
||||
- Automatic detection of console width to fill the display
|
||||
- Automatic use of Unicode to render smooth-filling progress bars on supported
|
||||
terminals
|
||||
- Support for custom rendering frontends, including:
|
||||
* Command-line interface
|
||||
* *Jupyter* HTML notebooks
|
||||
* `matplotlib`
|
||||
- Support for custom hooks/callbacks, including:
|
||||
* `pandas`
|
||||
* `keras` [@keras]
|
||||
|
||||
## Command-line Interface (CLI)
|
||||
|
||||
A CLI is also provided, where `tqdm` may be used a pipe:
|
||||
|
||||
```sh
|
||||
# count lines of text in all *.txt files
|
||||
$ cat *.txt | wc -l
|
||||
1075075
|
||||
# same but with continuously updating progress information
|
||||
$ cat *.txt | python3 -m tqdm --unit loc --unit_scale | wc -l
|
||||
1.08Mloc [00:07, 142kloc/s]
|
||||
# same if `total` is known
|
||||
$ cat *.txt | python3 -m tqdm --unit loc --unit_scale --total 1075075 | wc -l
|
||||
100%|#####################################| 1.08/1.08M [00:07<00:00, 142kloc/s]
|
||||
1075075
|
||||
```
|
||||
|
||||
# Availability
|
||||
|
||||
The package supports both Python versions 2 and 3, and is available for download
|
||||
via `conda` [@conda], `pip` [@pypi], `snap` [@snapcraft], `docker` [@docker],
|
||||
and *Zenodo* [@zenodo].
|
||||
Web-based Jupyter interactive demonstrations are also available
|
||||
[@notebooks;@binder]
|
||||
|
||||
Unit tests are run at least weekly on cloud-based continuous integration
|
||||
[@travis], with code style and security issues checked on
|
||||
[Codacy](https://app.codacy.com/project/tqdm/tqdm/dashboard) [@code-review].
|
||||
Coverage is reported on [Coveralls](https://coveralls.io/github/tqdm/tqdm) and
|
||||
[Codecov](https://codecov.io/gh/tqdm/tqdm), and performance is monitored against
|
||||
regression [@asv].
|
||||
|
||||
# Impact
|
||||
|
||||
As of January 2019, `tqdm` has accumulated over 20 million downloads
|
||||
[@pypi-downloads], and 315 thousand code inclusions [@tqdm-results]. Dependants
|
||||
of `tqdm` include 23 thousand repositories [@tqdm-dependents] and 7 thousand
|
||||
libraries [@lib-io]. `tqdm` has a SourceRank of 22 [@sourcerank], placing it in
|
||||
the world's top 20 Python packages as of early 2019 [@sourcerank-descending].
|
||||
|
||||
The source code of `tqdm` is hosted on GitHub, where it has received over 9
|
||||
thousand stars [@stars;@stars-hist], and was top trending repository during a
|
||||
period in December 2015 [@trend-hist]. The documentation has received over 500
|
||||
thousand hits [@hits], with highest rates during weekdays. Historical reading
|
||||
rates have also trended upwards at the end of holiday periods. This implies
|
||||
widespread use in commercial and academic settings.
|
||||
[OpenHub](https://www.openhub.net/p/tqdm) valuates the work according to the
|
||||
constructive cost model (COCOMO) as being worth approximately $50,000.
|
||||
|
||||
The library has also been used in several textbooks [@miller;@boxel;@nandy] and
|
||||
peer-reviewed scientific publications
|
||||
[@stein;@cook;@madhikar;@palmer;@knight;@moriwaki;@jackson].
|
||||
The [`tqdm` wiki](https://github.com/tqdm/tqdm/wiki) also lists other references
|
||||
in public media.
|
||||
|
||||
# Licence
|
||||
|
||||
`tqdm`'s source code is OSS, and all versions are archived at the DOI
|
||||
[10.5281/zenodo.595120](https://doi.org/10.5281/zenodo.595120). The primary
|
||||
maintainer [Casper da Costa-Luis](https://github.com/casperdcl) releases
|
||||
contributions under the terms of the MPLv2.0, while all other contributions are
|
||||
released under the terms of the MIT licence [@licence].
|
||||
|
||||
# References
|
61
examples/parallel_bars.py
Normal file
61
examples/parallel_bars.py
Normal file
|
@ -0,0 +1,61 @@
|
|||
from __future__ import print_function
|
||||
|
||||
import sys
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from functools import partial
|
||||
from multiprocessing import Pool, RLock, freeze_support
|
||||
from random import random
|
||||
from threading import RLock as TRLock
|
||||
from time import sleep
|
||||
|
||||
from tqdm.auto import tqdm, trange
|
||||
from tqdm.contrib.concurrent import process_map, thread_map
|
||||
|
||||
NUM_SUBITERS = 9
|
||||
PY2 = sys.version_info[:1] <= (2,)
|
||||
|
||||
|
||||
def progresser(n, auto_position=True, write_safe=False, blocking=True, progress=False):
|
||||
interval = random() * 0.002 / (NUM_SUBITERS - n + 2) # nosec
|
||||
total = 5000
|
||||
text = "#{0}, est. {1:<04.2}s".format(n, interval * total)
|
||||
for _ in trange(total, desc=text, disable=not progress,
|
||||
lock_args=None if blocking else (False,),
|
||||
position=None if auto_position else n):
|
||||
sleep(interval)
|
||||
# NB: may not clear instances with higher `position` upon completion
|
||||
# since this worker may not know about other bars #796
|
||||
if write_safe:
|
||||
# we think we know about other bars (currently only py3 threading)
|
||||
if n == 6:
|
||||
tqdm.write("n == 6 completed")
|
||||
return n + 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
freeze_support() # for Windows support
|
||||
L = list(range(NUM_SUBITERS))[::-1]
|
||||
|
||||
print("Simple thread mapping")
|
||||
thread_map(partial(progresser, write_safe=not PY2), L, max_workers=4)
|
||||
|
||||
print("Simple process mapping")
|
||||
process_map(partial(progresser), L, max_workers=4)
|
||||
|
||||
print("Manual nesting")
|
||||
for i in trange(16, desc="1"):
|
||||
for _ in trange(16, desc="2 @ %d" % i, leave=i % 2):
|
||||
sleep(0.01)
|
||||
|
||||
print("Multi-processing")
|
||||
tqdm.set_lock(RLock())
|
||||
p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))
|
||||
p.map(partial(progresser, progress=True), L)
|
||||
|
||||
print("Multi-threading")
|
||||
tqdm.set_lock(TRLock())
|
||||
pool_args = {}
|
||||
if not PY2:
|
||||
pool_args.update(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))
|
||||
with ThreadPoolExecutor(**pool_args) as p:
|
||||
p.map(partial(progresser, progress=True, write_safe=not PY2, blocking=False), L)
|
52
examples/redirect_print.py
Normal file
52
examples/redirect_print.py
Normal file
|
@ -0,0 +1,52 @@
|
|||
"""Redirecting writing
|
||||
|
||||
If using a library that can print messages to the console, editing the library
|
||||
by replacing `print()` with `tqdm.write()` may not be desirable.
|
||||
In that case, redirecting `sys.stdout` to `tqdm.write()` is an option.
|
||||
|
||||
To redirect `sys.stdout`, create a file-like class that will write
|
||||
any input string to `tqdm.write()`, and supply the arguments
|
||||
`file=sys.stdout, dynamic_ncols=True`.
|
||||
|
||||
A reusable canonical example is given below:
|
||||
"""
|
||||
from __future__ import print_function
|
||||
|
||||
import contextlib
|
||||
import sys
|
||||
from time import sleep
|
||||
|
||||
from tqdm import tqdm
|
||||
from tqdm.contrib import DummyTqdmFile
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def std_out_err_redirect_tqdm():
|
||||
orig_out_err = sys.stdout, sys.stderr
|
||||
try:
|
||||
# sys.stdout = sys.stderr = DummyTqdmFile(orig_out_err[0])
|
||||
sys.stdout, sys.stderr = map(DummyTqdmFile, orig_out_err)
|
||||
yield orig_out_err[0]
|
||||
# Relay exceptions
|
||||
except Exception as exc:
|
||||
raise exc
|
||||
# Always restore sys.stdout/err if necessary
|
||||
finally:
|
||||
sys.stdout, sys.stderr = orig_out_err
|
||||
|
||||
|
||||
def some_fun(i):
|
||||
print("Fee, fi, fo,".split()[i])
|
||||
|
||||
|
||||
# Redirect stdout to tqdm.write()
|
||||
with std_out_err_redirect_tqdm() as orig_stdout:
|
||||
# tqdm needs the original stdout
|
||||
# and dynamic_ncols=True to autodetect console width
|
||||
for i in tqdm(range(3), file=orig_stdout, dynamic_ncols=True):
|
||||
# order of the following two lines should not matter
|
||||
some_fun(i)
|
||||
sleep(.5)
|
||||
|
||||
# After the `with`, printing is restored
|
||||
print("Done!")
|
65
examples/simple_examples.py
Normal file
65
examples/simple_examples.py
Normal file
|
@ -0,0 +1,65 @@
|
|||
"""
|
||||
# Simple tqdm examples and profiling
|
||||
|
||||
# Benchmark
|
||||
for i in _range(int(1e8)):
|
||||
pass
|
||||
|
||||
# Basic demo
|
||||
import tqdm
|
||||
for i in tqdm.trange(int(1e8)):
|
||||
pass
|
||||
|
||||
# Some decorations
|
||||
import tqdm
|
||||
for i in tqdm.trange(int(1e8), miniters=int(1e6), ascii=True,
|
||||
desc="cool", dynamic_ncols=True):
|
||||
pass
|
||||
|
||||
# Nested bars
|
||||
from tqdm import trange
|
||||
for i in trange(10):
|
||||
for j in trange(int(1e7), leave=False, unit_scale=True):
|
||||
pass
|
||||
|
||||
# Experimental GUI demo
|
||||
import tqdm
|
||||
for i in tqdm.tgrange(int(1e8)):
|
||||
pass
|
||||
|
||||
# Comparison to https://code.google.com/p/python-progressbar/
|
||||
try:
|
||||
from progressbar.progressbar import ProgressBar
|
||||
except ImportError:
|
||||
pass
|
||||
else:
|
||||
for i in ProgressBar()(_range(int(1e8))):
|
||||
pass
|
||||
|
||||
# Dynamic miniters benchmark
|
||||
from tqdm import trange
|
||||
for i in trange(int(1e8), miniters=None, mininterval=0.1, smoothing=0):
|
||||
pass
|
||||
|
||||
# Fixed miniters benchmark
|
||||
from tqdm import trange
|
||||
for i in trange(int(1e8), miniters=4500000, mininterval=0.1, smoothing=0):
|
||||
pass
|
||||
"""
|
||||
|
||||
import re
|
||||
from time import sleep
|
||||
from timeit import timeit
|
||||
|
||||
# Simple demo
|
||||
from tqdm import trange
|
||||
|
||||
for _ in trange(16, leave=True):
|
||||
sleep(0.1)
|
||||
|
||||
# Profiling/overhead tests
|
||||
stmts = filter(None, re.split(r'\n\s*#.*?\n', __doc__))
|
||||
for s in stmts:
|
||||
print(s.replace('import tqdm\n', ''))
|
||||
print(timeit(stmt='try:\n\t_range = xrange'
|
||||
'\nexcept:\n\t_range = range\n' + s, number=1), 'seconds')
|
49
examples/tqdm_requests.py
Normal file
49
examples/tqdm_requests.py
Normal file
|
@ -0,0 +1,49 @@
|
|||
"""An example of wrapping manual tqdm updates for `requests.get`.
|
||||
See also: tqdm_wget.py.
|
||||
|
||||
Usage:
|
||||
tqdm_requests.py [options]
|
||||
|
||||
Options:
|
||||
-h, --help
|
||||
Print this help message and exit
|
||||
-u URL, --url URL : string, optional
|
||||
The url to fetch.
|
||||
[default: https://caspersci.uk.to/matryoshka.zip]
|
||||
-o FILE, --output FILE : string, optional
|
||||
The local file path in which to save the url [default: /dev/null].
|
||||
"""
|
||||
|
||||
from os import devnull
|
||||
|
||||
import requests
|
||||
from docopt import docopt
|
||||
|
||||
from tqdm.auto import tqdm
|
||||
|
||||
opts = docopt(__doc__)
|
||||
|
||||
eg_link = opts['--url']
|
||||
eg_file = eg_link.replace('/', ' ').split()[-1]
|
||||
eg_out = opts['--output'].replace("/dev/null", devnull)
|
||||
|
||||
response = requests.get(eg_link, stream=True)
|
||||
with open(eg_out, "wb") as fout:
|
||||
with tqdm(
|
||||
# all optional kwargs
|
||||
unit='B', unit_scale=True, unit_divisor=1024, miniters=1,
|
||||
desc=eg_file, total=int(response.headers.get('content-length', 0))
|
||||
) as pbar:
|
||||
for chunk in response.iter_content(chunk_size=4096):
|
||||
fout.write(chunk)
|
||||
pbar.update(len(chunk))
|
||||
|
||||
# Even simpler progress by wrapping the output file's `write()`
|
||||
response = requests.get(eg_link, stream=True)
|
||||
with tqdm.wrapattr(
|
||||
open(eg_out, "wb"), "write",
|
||||
unit='B', unit_scale=True, unit_divisor=1024, miniters=1,
|
||||
desc=eg_file, total=int(response.headers.get('content-length', 0))
|
||||
) as fout:
|
||||
for chunk in response.iter_content(chunk_size=4096):
|
||||
fout.write(chunk)
|
113
examples/tqdm_wget.py
Normal file
113
examples/tqdm_wget.py
Normal file
|
@ -0,0 +1,113 @@
|
|||
"""An example of wrapping manual tqdm updates for `urllib` reporthook.
|
||||
See also: tqdm_requests.py.
|
||||
|
||||
# `urllib.urlretrieve` documentation
|
||||
> If present, the hook function will be called once
|
||||
> on establishment of the network connection and once after each block read
|
||||
> thereafter. The hook will be passed three arguments; a count of blocks
|
||||
> transferred so far, a block size in bytes, and the total size of the file.
|
||||
|
||||
Usage:
|
||||
tqdm_wget.py [options]
|
||||
|
||||
Options:
|
||||
-h, --help
|
||||
Print this help message and exit
|
||||
-u URL, --url URL : string, optional
|
||||
The url to fetch.
|
||||
[default: https://caspersci.uk.to/matryoshka.zip]
|
||||
-o FILE, --output FILE : string, optional
|
||||
The local file path in which to save the url [default: /dev/null].
|
||||
"""
|
||||
|
||||
try:
|
||||
from urllib import request as urllib
|
||||
except ImportError: # py2
|
||||
import urllib
|
||||
from os import devnull
|
||||
|
||||
from docopt import docopt
|
||||
|
||||
from tqdm.auto import tqdm
|
||||
|
||||
|
||||
def my_hook(t):
|
||||
"""Wraps tqdm instance.
|
||||
|
||||
Don't forget to close() or __exit__()
|
||||
the tqdm instance once you're done with it (easiest using `with` syntax).
|
||||
|
||||
Example
|
||||
-------
|
||||
|
||||
>>> with tqdm(...) as t:
|
||||
... reporthook = my_hook(t)
|
||||
... urllib.urlretrieve(..., reporthook=reporthook)
|
||||
|
||||
"""
|
||||
last_b = [0]
|
||||
|
||||
def update_to(b=1, bsize=1, tsize=None):
|
||||
"""
|
||||
b : int, optional
|
||||
Number of blocks transferred so far [default: 1].
|
||||
bsize : int, optional
|
||||
Size of each block (in tqdm units) [default: 1].
|
||||
tsize : int, optional
|
||||
Total size (in tqdm units). If [default: None] or -1,
|
||||
remains unchanged.
|
||||
"""
|
||||
if tsize not in (None, -1):
|
||||
t.total = tsize
|
||||
displayed = t.update((b - last_b[0]) * bsize)
|
||||
last_b[0] = b
|
||||
return displayed
|
||||
|
||||
return update_to
|
||||
|
||||
|
||||
class TqdmUpTo(tqdm):
|
||||
"""Alternative Class-based version of the above.
|
||||
|
||||
Provides `update_to(n)` which uses `tqdm.update(delta_n)`.
|
||||
|
||||
Inspired by [twine#242](https://github.com/pypa/twine/pull/242),
|
||||
[here](https://github.com/pypa/twine/commit/42e55e06).
|
||||
"""
|
||||
|
||||
def update_to(self, b=1, bsize=1, tsize=None):
|
||||
"""
|
||||
b : int, optional
|
||||
Number of blocks transferred so far [default: 1].
|
||||
bsize : int, optional
|
||||
Size of each block (in tqdm units) [default: 1].
|
||||
tsize : int, optional
|
||||
Total size (in tqdm units). If [default: None] remains unchanged.
|
||||
"""
|
||||
if tsize is not None:
|
||||
self.total = tsize
|
||||
return self.update(b * bsize - self.n) # also sets self.n = b * bsize
|
||||
|
||||
|
||||
opts = docopt(__doc__)
|
||||
|
||||
eg_link = opts['--url']
|
||||
eg_file = eg_link.replace('/', ' ').split()[-1]
|
||||
eg_out = opts['--output'].replace("/dev/null", devnull)
|
||||
# with tqdm(unit='B', unit_scale=True, unit_divisor=1024, miniters=1,
|
||||
# desc=eg_file) as t: # all optional kwargs
|
||||
# urllib.urlretrieve(eg_link, filename=eg_out,
|
||||
# reporthook=my_hook(t), data=None)
|
||||
with TqdmUpTo(unit='B', unit_scale=True, unit_divisor=1024, miniters=1,
|
||||
desc=eg_file) as t: # all optional kwargs
|
||||
urllib.urlretrieve( # nosec
|
||||
eg_link, filename=eg_out, reporthook=t.update_to, data=None)
|
||||
t.total = t.n
|
||||
|
||||
# Even simpler progress by wrapping the output file's `write()`
|
||||
response = urllib.urlopen(eg_link) # nosec
|
||||
with tqdm.wrapattr(open(eg_out, "wb"), "write",
|
||||
miniters=1, desc=eg_file,
|
||||
total=getattr(response, 'length', None)) as fout:
|
||||
for chunk in response:
|
||||
fout.write(chunk)
|
15
examples/wrapping_generators.py
Normal file
15
examples/wrapping_generators.py
Normal file
|
@ -0,0 +1,15 @@
|
|||
import numpy as np
|
||||
|
||||
from tqdm.contrib import tenumerate, tmap, tzip
|
||||
|
||||
for _ in tenumerate(range(int(1e6)), desc="builtin enumerate"):
|
||||
pass
|
||||
|
||||
for _ in tenumerate(np.random.random((999, 999)), desc="numpy.ndenumerate"):
|
||||
pass
|
||||
|
||||
for _ in tzip(np.arange(1e6), np.arange(1e6) + 1, desc="builtin zip"):
|
||||
pass
|
||||
|
||||
mapped = tmap(lambda x: x + 1, np.arange(1e6), desc="builtin map")
|
||||
assert (np.arange(1e6) + 1 == list(mapped)).all()
|
BIN
logo.png
Normal file
BIN
logo.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 10 KiB |
7
pyproject.toml
Normal file
7
pyproject.toml
Normal file
|
@ -0,0 +1,7 @@
|
|||
[build-system]
|
||||
requires = ["setuptools>=42", "wheel", "setuptools_scm[toml]>=3.4"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[tool.setuptools_scm]
|
||||
write_to = "tqdm/_dist_ver.py"
|
||||
write_to_template = "__version__ = '{version}'\n"
|
159
setup.cfg
Normal file
159
setup.cfg
Normal file
|
@ -0,0 +1,159 @@
|
|||
[metadata]
|
||||
name = tqdm
|
||||
url = https://tqdm.github.io
|
||||
project_urls =
|
||||
Changelog=https://tqdm.github.io/releases
|
||||
Source=https://github.com/tqdm/tqdm
|
||||
Wiki=https://github.com/tqdm/tqdm/wiki
|
||||
maintainer = tqdm developers
|
||||
maintainer_email = python.tqdm@gmail.com
|
||||
license = MPLv2.0, MIT Licences
|
||||
license_file = LICENCE
|
||||
description = Fast, Extensible Progress Meter
|
||||
long_description = file: README.rst
|
||||
long_description_content_type = text/x-rst
|
||||
keywords = progressbar, progressmeter, progress, bar, meter, rate, eta, console, terminal, time
|
||||
platforms = any
|
||||
provides = tqdm
|
||||
classifiers =
|
||||
Development Status :: 5 - Production/Stable
|
||||
Environment :: Console
|
||||
Environment :: MacOS X
|
||||
Environment :: Other Environment
|
||||
Environment :: Win32 (MS Windows)
|
||||
Environment :: X11 Applications
|
||||
Framework :: IPython
|
||||
Framework :: Jupyter
|
||||
Intended Audience :: Developers
|
||||
Intended Audience :: Education
|
||||
Intended Audience :: End Users/Desktop
|
||||
Intended Audience :: Other Audience
|
||||
Intended Audience :: System Administrators
|
||||
License :: OSI Approved :: MIT License
|
||||
License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)
|
||||
Operating System :: MacOS
|
||||
Operating System :: MacOS :: MacOS X
|
||||
Operating System :: Microsoft
|
||||
Operating System :: Microsoft :: MS-DOS
|
||||
Operating System :: Microsoft :: Windows
|
||||
Operating System :: POSIX
|
||||
Operating System :: POSIX :: BSD
|
||||
Operating System :: POSIX :: BSD :: FreeBSD
|
||||
Operating System :: POSIX :: Linux
|
||||
Operating System :: POSIX :: SunOS/Solaris
|
||||
Operating System :: Unix
|
||||
Programming Language :: Python
|
||||
Programming Language :: Python :: 2
|
||||
Programming Language :: Python :: 2.7
|
||||
Programming Language :: Python :: 3
|
||||
Programming Language :: Python :: 3.5
|
||||
Programming Language :: Python :: 3.6
|
||||
Programming Language :: Python :: 3.7
|
||||
Programming Language :: Python :: 3.8
|
||||
Programming Language :: Python :: 3.9
|
||||
Programming Language :: Python :: 3.10
|
||||
Programming Language :: Python :: Implementation
|
||||
Programming Language :: Python :: Implementation :: IronPython
|
||||
Programming Language :: Python :: Implementation :: PyPy
|
||||
Programming Language :: Unix Shell
|
||||
Topic :: Desktop Environment
|
||||
Topic :: Education :: Computer Aided Instruction (CAI)
|
||||
Topic :: Education :: Testing
|
||||
Topic :: Office/Business
|
||||
Topic :: Other/Nonlisted Topic
|
||||
Topic :: Software Development :: Build Tools
|
||||
Topic :: Software Development :: Libraries
|
||||
Topic :: Software Development :: Libraries :: Python Modules
|
||||
Topic :: Software Development :: Pre-processors
|
||||
Topic :: Software Development :: User Interfaces
|
||||
Topic :: System :: Installation/Setup
|
||||
Topic :: System :: Logging
|
||||
Topic :: System :: Monitoring
|
||||
Topic :: System :: Shells
|
||||
Topic :: Terminals
|
||||
Topic :: Utilities
|
||||
|
||||
[options]
|
||||
setup_requires = setuptools>=42; setuptools_scm[toml]>=3.4
|
||||
python_requires = >=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*
|
||||
install_requires =
|
||||
colorama; platform_system == 'Windows'
|
||||
importlib_resources; python_version < "3.7"
|
||||
tests_require = tox
|
||||
include_package_data = True
|
||||
packages = find:
|
||||
|
||||
[options.extras_require]
|
||||
dev = py-make>=0.1.0; twine; wheel
|
||||
slack = slack-sdk
|
||||
telegram = requests
|
||||
notebook = ipywidgets>=6
|
||||
|
||||
[options.entry_points]
|
||||
console_scripts =
|
||||
tqdm=tqdm.cli:main
|
||||
|
||||
[options.packages.find]
|
||||
exclude = benchmarks, tests
|
||||
|
||||
[bdist_wheel]
|
||||
universal = 1
|
||||
|
||||
[flake8]
|
||||
max_line_length = 99
|
||||
exclude = .asv,.eggs,.tox,.ipynb_checkpoints,build,dist,.git,__pycache__
|
||||
|
||||
[pydocstyle]
|
||||
add_ignore = D400,D415
|
||||
|
||||
[yapf]
|
||||
coalesce_brackets = True
|
||||
column_limit = 99
|
||||
each_dict_entry_on_separate_line = False
|
||||
i18n_comment = NOQA
|
||||
space_between_ending_comma_and_closing_bracket = False
|
||||
split_before_named_assigns = False
|
||||
split_before_closing_bracket = False
|
||||
|
||||
[isort]
|
||||
line_length = 99
|
||||
multi_line_output = 4
|
||||
known_first_party = tqdm,tests
|
||||
|
||||
[tool:pytest]
|
||||
timeout = 30
|
||||
log_level = INFO
|
||||
markers =
|
||||
asyncio
|
||||
slow
|
||||
python_files = tests_*.py tests_*.ipynb
|
||||
testpaths = tests
|
||||
addopts = -v --tb=short -rxs -W=error --durations=0 --durations-min=0.1 --asyncio-mode=strict
|
||||
|
||||
[regex1]
|
||||
regex = (?<= )[\s\d.]+(it/s|s/it)
|
||||
replace = ??.??it/s
|
||||
|
||||
[regex2]
|
||||
regex = 00:0[01]<00:0[01]
|
||||
replace = 00:00<00:00
|
||||
|
||||
[coverage:run]
|
||||
branch = True
|
||||
include = tqdm/*
|
||||
omit =
|
||||
tqdm/contrib/bells.py
|
||||
tqdm/contrib/slack.py
|
||||
tqdm/contrib/discord.py
|
||||
tqdm/contrib/telegram.py
|
||||
tqdm/contrib/utils_worker.py
|
||||
relative_files = True
|
||||
disable_warnings = include-ignored
|
||||
|
||||
[coverage:report]
|
||||
show_missing = True
|
||||
|
||||
[egg_info]
|
||||
tag_build =
|
||||
tag_date = 0
|
||||
|
16
setup.py
Executable file
16
setup.py
Executable file
|
@ -0,0 +1,16 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
import sys
|
||||
from os import path
|
||||
|
||||
from setuptools import setup
|
||||
|
||||
src_dir = path.abspath(path.dirname(__file__))
|
||||
if sys.argv[1].lower().strip() == 'make': # exec Makefile commands
|
||||
import pymake
|
||||
fpath = path.join(src_dir, 'Makefile')
|
||||
pymake.main(['-f', fpath] + sys.argv[2:])
|
||||
# Stop to avoid setup.py raising non-standard command error
|
||||
sys.exit(0)
|
||||
|
||||
setup(use_scm_version=True)
|
0
tests/__init__.py
Normal file
0
tests/__init__.py
Normal file
41
tests/conftest.py
Normal file
41
tests/conftest.py
Normal file
|
@ -0,0 +1,41 @@
|
|||
"""Shared pytest config."""
|
||||
import sys
|
||||
|
||||
from pytest import fixture
|
||||
|
||||
from tqdm import tqdm
|
||||
|
||||
|
||||
@fixture(autouse=True)
|
||||
def pretest_posttest():
|
||||
"""Fixture for all tests ensuring environment cleanup"""
|
||||
try:
|
||||
sys.setswitchinterval(1)
|
||||
except AttributeError:
|
||||
sys.setcheckinterval(100) # deprecated
|
||||
|
||||
if getattr(tqdm, "_instances", False):
|
||||
n = len(tqdm._instances)
|
||||
if n:
|
||||
tqdm._instances.clear()
|
||||
raise EnvironmentError(
|
||||
"{0} `tqdm` instances still in existence PRE-test".format(n))
|
||||
yield
|
||||
if getattr(tqdm, "_instances", False):
|
||||
n = len(tqdm._instances)
|
||||
if n:
|
||||
tqdm._instances.clear()
|
||||
raise EnvironmentError(
|
||||
"{0} `tqdm` instances still in existence POST-test".format(n))
|
||||
|
||||
|
||||
if sys.version_info[0] > 2:
|
||||
@fixture
|
||||
def capsysbin(capsysbinary):
|
||||
"""alias for capsysbinary (py3)"""
|
||||
return capsysbinary
|
||||
else:
|
||||
@fixture
|
||||
def capsysbin(capsys):
|
||||
"""alias for capsys (py2)"""
|
||||
return capsys
|
128
tests/py37_asyncio.py
Normal file
128
tests/py37_asyncio.py
Normal file
|
@ -0,0 +1,128 @@
|
|||
import asyncio
|
||||
from functools import partial
|
||||
from sys import platform
|
||||
from time import time
|
||||
|
||||
from tqdm.asyncio import tarange, tqdm_asyncio
|
||||
|
||||
from .tests_tqdm import StringIO, closing, mark
|
||||
|
||||
tqdm = partial(tqdm_asyncio, miniters=0, mininterval=0)
|
||||
trange = partial(tarange, miniters=0, mininterval=0)
|
||||
as_completed = partial(tqdm_asyncio.as_completed, miniters=0, mininterval=0)
|
||||
gather = partial(tqdm_asyncio.gather, miniters=0, mininterval=0)
|
||||
|
||||
|
||||
def count(start=0, step=1):
|
||||
i = start
|
||||
while True:
|
||||
new_start = yield i
|
||||
if new_start is None:
|
||||
i += step
|
||||
else:
|
||||
i = new_start
|
||||
|
||||
|
||||
async def acount(*args, **kwargs):
|
||||
for i in count(*args, **kwargs):
|
||||
yield i
|
||||
|
||||
|
||||
@mark.asyncio
|
||||
async def test_break():
|
||||
"""Test asyncio break"""
|
||||
pbar = tqdm(count())
|
||||
async for _ in pbar:
|
||||
break
|
||||
pbar.close()
|
||||
|
||||
|
||||
@mark.asyncio
|
||||
async def test_generators(capsys):
|
||||
"""Test asyncio generators"""
|
||||
with tqdm(count(), desc="counter") as pbar:
|
||||
async for i in pbar:
|
||||
if i >= 8:
|
||||
break
|
||||
_, err = capsys.readouterr()
|
||||
assert '9it' in err
|
||||
|
||||
with tqdm(acount(), desc="async_counter") as pbar:
|
||||
async for i in pbar:
|
||||
if i >= 8:
|
||||
break
|
||||
_, err = capsys.readouterr()
|
||||
assert '9it' in err
|
||||
|
||||
|
||||
@mark.asyncio
|
||||
async def test_range():
|
||||
"""Test asyncio range"""
|
||||
with closing(StringIO()) as our_file:
|
||||
async for _ in tqdm(range(9), desc="range", file=our_file):
|
||||
pass
|
||||
assert '9/9' in our_file.getvalue()
|
||||
our_file.seek(0)
|
||||
our_file.truncate()
|
||||
|
||||
async for _ in trange(9, desc="trange", file=our_file):
|
||||
pass
|
||||
assert '9/9' in our_file.getvalue()
|
||||
|
||||
|
||||
@mark.asyncio
|
||||
async def test_nested():
|
||||
"""Test asyncio nested"""
|
||||
with closing(StringIO()) as our_file:
|
||||
async for _ in tqdm(trange(9, desc="inner", file=our_file),
|
||||
desc="outer", file=our_file):
|
||||
pass
|
||||
assert 'inner: 100%' in our_file.getvalue()
|
||||
assert 'outer: 100%' in our_file.getvalue()
|
||||
|
||||
|
||||
@mark.asyncio
|
||||
async def test_coroutines():
|
||||
"""Test asyncio coroutine.send"""
|
||||
with closing(StringIO()) as our_file:
|
||||
with tqdm(count(), file=our_file) as pbar:
|
||||
async for i in pbar:
|
||||
if i == 9:
|
||||
pbar.send(-10)
|
||||
elif i < 0:
|
||||
assert i == -9
|
||||
break
|
||||
assert '10it' in our_file.getvalue()
|
||||
|
||||
|
||||
@mark.slow
|
||||
@mark.asyncio
|
||||
@mark.parametrize("tol", [0.2 if platform.startswith("darwin") else 0.1])
|
||||
async def test_as_completed(capsys, tol):
|
||||
"""Test asyncio as_completed"""
|
||||
for retry in range(3):
|
||||
t = time()
|
||||
skew = time() - t
|
||||
for i in as_completed([asyncio.sleep(0.01 * i) for i in range(30, 0, -1)]):
|
||||
await i
|
||||
t = time() - t - 2 * skew
|
||||
try:
|
||||
assert 0.3 * (1 - tol) < t < 0.3 * (1 + tol), t
|
||||
_, err = capsys.readouterr()
|
||||
assert '30/30' in err
|
||||
except AssertionError:
|
||||
if retry == 2:
|
||||
raise
|
||||
|
||||
|
||||
async def double(i):
|
||||
return i * 2
|
||||
|
||||
|
||||
@mark.asyncio
|
||||
async def test_gather(capsys):
|
||||
"""Test asyncio gather"""
|
||||
res = await gather(*map(double, range(30)))
|
||||
_, err = capsys.readouterr()
|
||||
assert '30/30' in err
|
||||
assert res == list(range(0, 30 * 2, 2))
|
11
tests/tests_asyncio.py
Normal file
11
tests/tests_asyncio.py
Normal file
|
@ -0,0 +1,11 @@
|
|||
"""Tests `tqdm.asyncio` on `python>=3.7`."""
|
||||
import sys
|
||||
|
||||
if sys.version_info[:2] > (3, 6):
|
||||
from .py37_asyncio import * # NOQA, pylint: disable=wildcard-import
|
||||
else:
|
||||
from .tests_tqdm import skip
|
||||
try:
|
||||
skip("async not supported", allow_module_level=True)
|
||||
except TypeError:
|
||||
pass
|
49
tests/tests_concurrent.py
Normal file
49
tests/tests_concurrent.py
Normal file
|
@ -0,0 +1,49 @@
|
|||
"""
|
||||
Tests for `tqdm.contrib.concurrent`.
|
||||
"""
|
||||
from pytest import warns
|
||||
|
||||
from tqdm.contrib.concurrent import process_map, thread_map
|
||||
|
||||
from .tests_tqdm import StringIO, TqdmWarning, closing, importorskip, mark, skip
|
||||
|
||||
|
||||
def incr(x):
|
||||
"""Dummy function"""
|
||||
return x + 1
|
||||
|
||||
|
||||
def test_thread_map():
|
||||
"""Test contrib.concurrent.thread_map"""
|
||||
with closing(StringIO()) as our_file:
|
||||
a = range(9)
|
||||
b = [i + 1 for i in a]
|
||||
try:
|
||||
assert thread_map(lambda x: x + 1, a, file=our_file) == b
|
||||
except ImportError as err:
|
||||
skip(str(err))
|
||||
assert thread_map(incr, a, file=our_file) == b
|
||||
|
||||
|
||||
def test_process_map():
|
||||
"""Test contrib.concurrent.process_map"""
|
||||
with closing(StringIO()) as our_file:
|
||||
a = range(9)
|
||||
b = [i + 1 for i in a]
|
||||
try:
|
||||
assert process_map(incr, a, file=our_file) == b
|
||||
except ImportError as err:
|
||||
skip(str(err))
|
||||
|
||||
|
||||
@mark.parametrize("iterables,should_warn", [([], False), (['x'], False), ([()], False),
|
||||
(['x', ()], False), (['x' * 1001], True),
|
||||
(['x' * 100, ('x',) * 1001], True)])
|
||||
def test_chunksize_warning(iterables, should_warn):
|
||||
"""Test contrib.concurrent.process_map chunksize warnings"""
|
||||
patch = importorskip('unittest.mock').patch
|
||||
with patch('tqdm.contrib.concurrent._executor_map'):
|
||||
if should_warn:
|
||||
warns(TqdmWarning, process_map, incr, *iterables)
|
||||
else:
|
||||
process_map(incr, *iterables)
|
71
tests/tests_contrib.py
Normal file
71
tests/tests_contrib.py
Normal file
|
@ -0,0 +1,71 @@
|
|||
"""
|
||||
Tests for `tqdm.contrib`.
|
||||
"""
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from tqdm import tqdm
|
||||
from tqdm.contrib import tenumerate, tmap, tzip
|
||||
|
||||
from .tests_tqdm import StringIO, closing, importorskip
|
||||
|
||||
|
||||
def incr(x):
|
||||
"""Dummy function"""
|
||||
return x + 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("tqdm_kwargs", [{}, {"tqdm_class": tqdm}])
|
||||
def test_enumerate(tqdm_kwargs):
|
||||
"""Test contrib.tenumerate"""
|
||||
with closing(StringIO()) as our_file:
|
||||
a = range(9)
|
||||
assert list(tenumerate(a, file=our_file, **tqdm_kwargs)) == list(enumerate(a))
|
||||
assert list(tenumerate(a, 42, file=our_file, **tqdm_kwargs)) == list(
|
||||
enumerate(a, 42)
|
||||
)
|
||||
with closing(StringIO()) as our_file:
|
||||
_ = list(tenumerate(iter(a), file=our_file, **tqdm_kwargs))
|
||||
assert "100%" not in our_file.getvalue()
|
||||
with closing(StringIO()) as our_file:
|
||||
_ = list(tenumerate(iter(a), file=our_file, total=len(a), **tqdm_kwargs))
|
||||
assert "100%" in our_file.getvalue()
|
||||
|
||||
|
||||
def test_enumerate_numpy():
|
||||
"""Test contrib.tenumerate(numpy.ndarray)"""
|
||||
np = importorskip("numpy")
|
||||
with closing(StringIO()) as our_file:
|
||||
a = np.random.random((42, 7))
|
||||
assert list(tenumerate(a, file=our_file)) == list(np.ndenumerate(a))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("tqdm_kwargs", [{}, {"tqdm_class": tqdm}])
|
||||
def test_zip(tqdm_kwargs):
|
||||
"""Test contrib.tzip"""
|
||||
with closing(StringIO()) as our_file:
|
||||
a = range(9)
|
||||
b = [i + 1 for i in a]
|
||||
if sys.version_info[:1] < (3,):
|
||||
assert tzip(a, b, file=our_file, **tqdm_kwargs) == zip(a, b)
|
||||
else:
|
||||
gen = tzip(a, b, file=our_file, **tqdm_kwargs)
|
||||
assert gen != list(zip(a, b))
|
||||
assert list(gen) == list(zip(a, b))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("tqdm_kwargs", [{}, {"tqdm_class": tqdm}])
|
||||
def test_map(tqdm_kwargs):
|
||||
"""Test contrib.tmap"""
|
||||
with closing(StringIO()) as our_file:
|
||||
a = range(9)
|
||||
b = [i + 1 for i in a]
|
||||
if sys.version_info[:1] < (3,):
|
||||
assert tmap(lambda x: x + 1, a, file=our_file, **tqdm_kwargs) == map(
|
||||
incr, a
|
||||
)
|
||||
else:
|
||||
gen = tmap(lambda x: x + 1, a, file=our_file, **tqdm_kwargs)
|
||||
assert gen != b
|
||||
assert list(gen) == b
|
173
tests/tests_contrib_logging.py
Normal file
173
tests/tests_contrib_logging.py
Normal file
|
@ -0,0 +1,173 @@
|
|||
# pylint: disable=missing-module-docstring, missing-class-docstring
|
||||
# pylint: disable=missing-function-docstring, no-self-use
|
||||
from __future__ import absolute_import
|
||||
|
||||
import logging
|
||||
import logging.handlers
|
||||
import sys
|
||||
from io import StringIO
|
||||
|
||||
import pytest
|
||||
|
||||
from tqdm import tqdm
|
||||
from tqdm.contrib.logging import _get_first_found_console_logging_handler
|
||||
from tqdm.contrib.logging import _TqdmLoggingHandler as TqdmLoggingHandler
|
||||
from tqdm.contrib.logging import logging_redirect_tqdm, tqdm_logging_redirect
|
||||
|
||||
from .tests_tqdm import importorskip
|
||||
|
||||
LOGGER = logging.getLogger(__name__)
|
||||
|
||||
TEST_LOGGING_FORMATTER = logging.Formatter()
|
||||
|
||||
|
||||
class CustomTqdm(tqdm):
|
||||
messages = []
|
||||
|
||||
@classmethod
|
||||
def write(cls, s, **__): # pylint: disable=arguments-differ
|
||||
CustomTqdm.messages.append(s)
|
||||
|
||||
|
||||
class ErrorRaisingTqdm(tqdm):
|
||||
exception_class = RuntimeError
|
||||
|
||||
@classmethod
|
||||
def write(cls, s, **__): # pylint: disable=arguments-differ
|
||||
raise ErrorRaisingTqdm.exception_class('fail fast')
|
||||
|
||||
|
||||
class TestTqdmLoggingHandler:
|
||||
def test_should_call_tqdm_write(self):
|
||||
CustomTqdm.messages = []
|
||||
logger = logging.Logger('test')
|
||||
logger.handlers = [TqdmLoggingHandler(CustomTqdm)]
|
||||
logger.info('test')
|
||||
assert CustomTqdm.messages == ['test']
|
||||
|
||||
def test_should_call_handle_error_if_exception_was_thrown(self):
|
||||
patch = importorskip('unittest.mock').patch
|
||||
logger = logging.Logger('test')
|
||||
ErrorRaisingTqdm.exception_class = RuntimeError
|
||||
handler = TqdmLoggingHandler(ErrorRaisingTqdm)
|
||||
logger.handlers = [handler]
|
||||
with patch.object(handler, 'handleError') as mock:
|
||||
logger.info('test')
|
||||
assert mock.called
|
||||
|
||||
@pytest.mark.parametrize('exception_class', [
|
||||
KeyboardInterrupt,
|
||||
SystemExit
|
||||
])
|
||||
def test_should_not_swallow_certain_exceptions(self, exception_class):
|
||||
logger = logging.Logger('test')
|
||||
ErrorRaisingTqdm.exception_class = exception_class
|
||||
handler = TqdmLoggingHandler(ErrorRaisingTqdm)
|
||||
logger.handlers = [handler]
|
||||
with pytest.raises(exception_class):
|
||||
logger.info('test')
|
||||
|
||||
|
||||
class TestGetFirstFoundConsoleLoggingHandler:
|
||||
def test_should_return_none_for_no_handlers(self):
|
||||
assert _get_first_found_console_logging_handler([]) is None
|
||||
|
||||
def test_should_return_none_without_stream_handler(self):
|
||||
handler = logging.handlers.MemoryHandler(capacity=1)
|
||||
assert _get_first_found_console_logging_handler([handler]) is None
|
||||
|
||||
def test_should_return_none_for_stream_handler_not_stdout_or_stderr(self):
|
||||
handler = logging.StreamHandler(StringIO())
|
||||
assert _get_first_found_console_logging_handler([handler]) is None
|
||||
|
||||
def test_should_return_stream_handler_if_stream_is_stdout(self):
|
||||
handler = logging.StreamHandler(sys.stdout)
|
||||
assert _get_first_found_console_logging_handler([handler]) == handler
|
||||
|
||||
def test_should_return_stream_handler_if_stream_is_stderr(self):
|
||||
handler = logging.StreamHandler(sys.stderr)
|
||||
assert _get_first_found_console_logging_handler([handler]) == handler
|
||||
|
||||
|
||||
class TestRedirectLoggingToTqdm:
|
||||
def test_should_add_and_remove_tqdm_handler(self):
|
||||
logger = logging.Logger('test')
|
||||
with logging_redirect_tqdm(loggers=[logger]):
|
||||
assert len(logger.handlers) == 1
|
||||
assert isinstance(logger.handlers[0], TqdmLoggingHandler)
|
||||
assert not logger.handlers
|
||||
|
||||
def test_should_remove_and_restore_console_handlers(self):
|
||||
logger = logging.Logger('test')
|
||||
stderr_console_handler = logging.StreamHandler(sys.stderr)
|
||||
stdout_console_handler = logging.StreamHandler(sys.stderr)
|
||||
logger.handlers = [stderr_console_handler, stdout_console_handler]
|
||||
with logging_redirect_tqdm(loggers=[logger]):
|
||||
assert len(logger.handlers) == 1
|
||||
assert isinstance(logger.handlers[0], TqdmLoggingHandler)
|
||||
assert logger.handlers == [stderr_console_handler, stdout_console_handler]
|
||||
|
||||
def test_should_inherit_console_logger_formatter(self):
|
||||
logger = logging.Logger('test')
|
||||
formatter = logging.Formatter('custom: %(message)s')
|
||||
console_handler = logging.StreamHandler(sys.stderr)
|
||||
console_handler.setFormatter(formatter)
|
||||
logger.handlers = [console_handler]
|
||||
with logging_redirect_tqdm(loggers=[logger]):
|
||||
assert logger.handlers[0].formatter == formatter
|
||||
|
||||
def test_should_not_remove_stream_handlers_not_for_stdout_or_stderr(self):
|
||||
logger = logging.Logger('test')
|
||||
stream_handler = logging.StreamHandler(StringIO())
|
||||
logger.addHandler(stream_handler)
|
||||
with logging_redirect_tqdm(loggers=[logger]):
|
||||
assert len(logger.handlers) == 2
|
||||
assert logger.handlers[0] == stream_handler
|
||||
assert isinstance(logger.handlers[1], TqdmLoggingHandler)
|
||||
assert logger.handlers == [stream_handler]
|
||||
|
||||
|
||||
class TestTqdmWithLoggingRedirect:
|
||||
def test_should_add_and_remove_handler_from_root_logger_by_default(self):
|
||||
original_handlers = list(logging.root.handlers)
|
||||
with tqdm_logging_redirect(total=1) as pbar:
|
||||
assert isinstance(logging.root.handlers[-1], TqdmLoggingHandler)
|
||||
LOGGER.info('test')
|
||||
pbar.update(1)
|
||||
assert logging.root.handlers == original_handlers
|
||||
|
||||
def test_should_add_and_remove_handler_from_custom_logger(self):
|
||||
logger = logging.Logger('test')
|
||||
with tqdm_logging_redirect(total=1, loggers=[logger]) as pbar:
|
||||
assert len(logger.handlers) == 1
|
||||
assert isinstance(logger.handlers[0], TqdmLoggingHandler)
|
||||
logger.info('test')
|
||||
pbar.update(1)
|
||||
assert not logger.handlers
|
||||
|
||||
def test_should_not_fail_with_logger_without_console_handler(self):
|
||||
logger = logging.Logger('test')
|
||||
logger.handlers = []
|
||||
with tqdm_logging_redirect(total=1, loggers=[logger]):
|
||||
logger.info('test')
|
||||
assert not logger.handlers
|
||||
|
||||
def test_should_format_message(self):
|
||||
logger = logging.Logger('test')
|
||||
console_handler = logging.StreamHandler(sys.stdout)
|
||||
console_handler.setFormatter(logging.Formatter(
|
||||
r'prefix:%(message)s'
|
||||
))
|
||||
logger.handlers = [console_handler]
|
||||
CustomTqdm.messages = []
|
||||
with tqdm_logging_redirect(loggers=[logger], tqdm_class=CustomTqdm):
|
||||
logger.info('test')
|
||||
assert CustomTqdm.messages == ['prefix:test']
|
||||
|
||||
def test_use_root_logger_by_default_and_write_to_custom_tqdm(self):
|
||||
logger = logging.root
|
||||
CustomTqdm.messages = []
|
||||
with tqdm_logging_redirect(total=1, tqdm_class=CustomTqdm) as pbar:
|
||||
assert isinstance(pbar, CustomTqdm)
|
||||
logger.info('test')
|
||||
assert CustomTqdm.messages == ['test']
|
20
tests/tests_dask.py
Normal file
20
tests/tests_dask.py
Normal file
|
@ -0,0 +1,20 @@
|
|||
from __future__ import division
|
||||
|
||||
from time import sleep
|
||||
|
||||
from .tests_tqdm import importorskip, mark
|
||||
|
||||
pytestmark = mark.slow
|
||||
|
||||
|
||||
def test_dask(capsys):
|
||||
"""Test tqdm.dask.TqdmCallback"""
|
||||
ProgressBar = importorskip('tqdm.dask').TqdmCallback
|
||||
dask = importorskip('dask')
|
||||
|
||||
schedule = [dask.delayed(sleep)(i / 10) for i in range(5)]
|
||||
with ProgressBar(desc="computing"):
|
||||
dask.compute(schedule)
|
||||
_, err = capsys.readouterr()
|
||||
assert "computing: " in err
|
||||
assert '5/5' in err
|
7
tests/tests_gui.py
Normal file
7
tests/tests_gui.py
Normal file
|
@ -0,0 +1,7 @@
|
|||
"""Test `tqdm.gui`."""
|
||||
from .tests_tqdm import importorskip
|
||||
|
||||
|
||||
def test_gui_import():
|
||||
"""Test `tqdm.gui` import"""
|
||||
importorskip('tqdm.gui')
|
26
tests/tests_itertools.py
Normal file
26
tests/tests_itertools.py
Normal file
|
@ -0,0 +1,26 @@
|
|||
"""
|
||||
Tests for `tqdm.contrib.itertools`.
|
||||
"""
|
||||
import itertools as it
|
||||
|
||||
from tqdm.contrib.itertools import product
|
||||
|
||||
from .tests_tqdm import StringIO, closing
|
||||
|
||||
|
||||
class NoLenIter(object):
|
||||
def __init__(self, iterable):
|
||||
self._it = iterable
|
||||
|
||||
def __iter__(self):
|
||||
for i in self._it:
|
||||
yield i
|
||||
|
||||
|
||||
def test_product():
|
||||
"""Test contrib.itertools.product"""
|
||||
with closing(StringIO()) as our_file:
|
||||
a = range(9)
|
||||
assert list(product(a, a[::-1], file=our_file)) == list(it.product(a, a[::-1]))
|
||||
|
||||
assert list(product(a, NoLenIter(a), file=our_file)) == list(it.product(a, NoLenIter(a)))
|
93
tests/tests_keras.py
Normal file
93
tests/tests_keras.py
Normal file
|
@ -0,0 +1,93 @@
|
|||
from __future__ import division
|
||||
|
||||
from .tests_tqdm import importorskip, mark
|
||||
|
||||
pytestmark = mark.slow
|
||||
|
||||
|
||||
@mark.filterwarnings("ignore:.*:DeprecationWarning")
|
||||
def test_keras(capsys):
|
||||
"""Test tqdm.keras.TqdmCallback"""
|
||||
TqdmCallback = importorskip('tqdm.keras').TqdmCallback
|
||||
np = importorskip('numpy')
|
||||
try:
|
||||
import keras as K
|
||||
except ImportError:
|
||||
K = importorskip('tensorflow.keras')
|
||||
|
||||
# 1D autoencoder
|
||||
dtype = np.float32
|
||||
model = K.models.Sequential([
|
||||
K.layers.InputLayer((1, 1), dtype=dtype), K.layers.Conv1D(1, 1)])
|
||||
model.compile("adam", "mse")
|
||||
x = np.random.rand(100, 1, 1).astype(dtype)
|
||||
batch_size = 10
|
||||
batches = len(x) / batch_size
|
||||
epochs = 5
|
||||
|
||||
# just epoch (no batch) progress
|
||||
model.fit(
|
||||
x,
|
||||
x,
|
||||
epochs=epochs,
|
||||
batch_size=batch_size,
|
||||
verbose=False,
|
||||
callbacks=[
|
||||
TqdmCallback(
|
||||
epochs,
|
||||
desc="training",
|
||||
data_size=len(x),
|
||||
batch_size=batch_size,
|
||||
verbose=0)])
|
||||
_, res = capsys.readouterr()
|
||||
assert "training: " in res
|
||||
assert "{epochs}/{epochs}".format(epochs=epochs) in res
|
||||
assert "{batches}/{batches}".format(batches=batches) not in res
|
||||
|
||||
# full (epoch and batch) progress
|
||||
model.fit(
|
||||
x,
|
||||
x,
|
||||
epochs=epochs,
|
||||
batch_size=batch_size,
|
||||
verbose=False,
|
||||
callbacks=[
|
||||
TqdmCallback(
|
||||
epochs,
|
||||
desc="training",
|
||||
data_size=len(x),
|
||||
batch_size=batch_size,
|
||||
verbose=2)])
|
||||
_, res = capsys.readouterr()
|
||||
assert "training: " in res
|
||||
assert "{epochs}/{epochs}".format(epochs=epochs) in res
|
||||
assert "{batches}/{batches}".format(batches=batches) in res
|
||||
|
||||
# auto-detect epochs and batches
|
||||
model.fit(
|
||||
x,
|
||||
x,
|
||||
epochs=epochs,
|
||||
batch_size=batch_size,
|
||||
verbose=False,
|
||||
callbacks=[TqdmCallback(desc="training", verbose=2)])
|
||||
_, res = capsys.readouterr()
|
||||
assert "training: " in res
|
||||
assert "{epochs}/{epochs}".format(epochs=epochs) in res
|
||||
assert "{batches}/{batches}".format(batches=batches) in res
|
||||
|
||||
# continue training (start from epoch != 0)
|
||||
initial_epoch = 3
|
||||
model.fit(
|
||||
x,
|
||||
x,
|
||||
initial_epoch=initial_epoch,
|
||||
epochs=epochs,
|
||||
batch_size=batch_size,
|
||||
verbose=False,
|
||||
callbacks=[TqdmCallback(desc="training", verbose=0,
|
||||
miniters=1, mininterval=0, maxinterval=0)])
|
||||
_, res = capsys.readouterr()
|
||||
assert "training: " in res
|
||||
assert "{epochs}/{epochs}".format(epochs=initial_epoch - 1) not in res
|
||||
assert "{epochs}/{epochs}".format(epochs=epochs) in res
|
245
tests/tests_main.py
Normal file
245
tests/tests_main.py
Normal file
|
@ -0,0 +1,245 @@
|
|||
"""Test CLI usage."""
|
||||
import logging
|
||||
import subprocess # nosec
|
||||
import sys
|
||||
from functools import wraps
|
||||
from os import linesep
|
||||
|
||||
from tqdm.cli import TqdmKeyError, TqdmTypeError, main
|
||||
from tqdm.utils import IS_WIN
|
||||
|
||||
from .tests_tqdm import BytesIO, _range, closing, mark, raises
|
||||
|
||||
|
||||
def restore_sys(func):
|
||||
"""Decorates `func(capsysbin)` to save & restore `sys.(stdin|argv)`."""
|
||||
@wraps(func)
|
||||
def inner(capsysbin):
|
||||
"""function requiring capsysbin which may alter `sys.(stdin|argv)`"""
|
||||
_SYS = sys.stdin, sys.argv
|
||||
try:
|
||||
res = func(capsysbin)
|
||||
finally:
|
||||
sys.stdin, sys.argv = _SYS
|
||||
return res
|
||||
|
||||
return inner
|
||||
|
||||
|
||||
def norm(bytestr):
|
||||
"""Normalise line endings."""
|
||||
return bytestr if linesep == "\n" else bytestr.replace(linesep.encode(), b"\n")
|
||||
|
||||
|
||||
@mark.slow
|
||||
def test_pipes():
|
||||
"""Test command line pipes"""
|
||||
ls_out = subprocess.check_output(['ls']) # nosec
|
||||
ls = subprocess.Popen(['ls'], stdout=subprocess.PIPE) # nosec
|
||||
res = subprocess.Popen( # nosec
|
||||
[sys.executable, '-c', 'from tqdm.cli import main; main()'],
|
||||
stdin=ls.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
out, err = res.communicate()
|
||||
assert ls.poll() == 0
|
||||
|
||||
# actual test:
|
||||
assert norm(ls_out) == norm(out)
|
||||
assert b"it/s" in err
|
||||
assert b"Error" not in err
|
||||
|
||||
|
||||
if sys.version_info[:2] >= (3, 8):
|
||||
test_pipes = mark.filterwarnings("ignore:unclosed file:ResourceWarning")(
|
||||
test_pipes)
|
||||
|
||||
|
||||
def test_main_import():
|
||||
"""Test main CLI import"""
|
||||
N = 123
|
||||
_SYS = sys.stdin, sys.argv
|
||||
# test direct import
|
||||
sys.stdin = [str(i).encode() for i in _range(N)]
|
||||
sys.argv = ['', '--desc', 'Test CLI import',
|
||||
'--ascii', 'True', '--unit_scale', 'True']
|
||||
try:
|
||||
import tqdm.__main__ # NOQA, pylint: disable=unused-variable
|
||||
finally:
|
||||
sys.stdin, sys.argv = _SYS
|
||||
|
||||
|
||||
@restore_sys
|
||||
def test_main_bytes(capsysbin):
|
||||
"""Test CLI --bytes"""
|
||||
N = 123
|
||||
|
||||
# test --delim
|
||||
IN_DATA = '\0'.join(map(str, _range(N))).encode()
|
||||
with closing(BytesIO()) as sys.stdin:
|
||||
sys.stdin.write(IN_DATA)
|
||||
# sys.stdin.write(b'\xff') # TODO
|
||||
sys.stdin.seek(0)
|
||||
main(sys.stderr, ['--desc', 'Test CLI delim', '--ascii', 'True',
|
||||
'--delim', r'\0', '--buf_size', '64'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert out == IN_DATA
|
||||
assert str(N) + "it" in err.decode("U8")
|
||||
|
||||
# test --bytes
|
||||
IN_DATA = IN_DATA.replace(b'\0', b'\n')
|
||||
with closing(BytesIO()) as sys.stdin:
|
||||
sys.stdin.write(IN_DATA)
|
||||
sys.stdin.seek(0)
|
||||
main(sys.stderr, ['--ascii', '--bytes=True', '--unit_scale', 'False'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert out == IN_DATA
|
||||
assert str(len(IN_DATA)) + "B" in err.decode("U8")
|
||||
|
||||
|
||||
@mark.skipif(sys.version_info[0] == 2, reason="no caplog on py2")
|
||||
def test_main_log(capsysbin, caplog):
|
||||
"""Test CLI --log"""
|
||||
_SYS = sys.stdin, sys.argv
|
||||
N = 123
|
||||
sys.stdin = [(str(i) + '\n').encode() for i in _range(N)]
|
||||
IN_DATA = b''.join(sys.stdin)
|
||||
try:
|
||||
with caplog.at_level(logging.INFO):
|
||||
main(sys.stderr, ['--log', 'INFO'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert norm(out) == IN_DATA and b"123/123" in err
|
||||
assert not caplog.record_tuples
|
||||
with caplog.at_level(logging.DEBUG):
|
||||
main(sys.stderr, ['--log', 'DEBUG'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert norm(out) == IN_DATA and b"123/123" in err
|
||||
assert caplog.record_tuples
|
||||
finally:
|
||||
sys.stdin, sys.argv = _SYS
|
||||
|
||||
|
||||
@restore_sys
|
||||
def test_main(capsysbin):
|
||||
"""Test misc CLI options"""
|
||||
N = 123
|
||||
sys.stdin = [(str(i) + '\n').encode() for i in _range(N)]
|
||||
IN_DATA = b''.join(sys.stdin)
|
||||
|
||||
# test --tee
|
||||
main(sys.stderr, ['--mininterval', '0', '--miniters', '1'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert norm(out) == IN_DATA and b"123/123" in err
|
||||
assert N <= len(err.split(b"\r")) < N + 5
|
||||
|
||||
len_err = len(err)
|
||||
main(sys.stderr, ['--tee', '--mininterval', '0', '--miniters', '1'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert norm(out) == IN_DATA and b"123/123" in err
|
||||
# spaces to clear intermediate lines could increase length
|
||||
assert len_err + len(norm(out)) <= len(err)
|
||||
|
||||
# test --null
|
||||
main(sys.stderr, ['--null'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert not out and b"123/123" in err
|
||||
|
||||
# test integer --update
|
||||
main(sys.stderr, ['--update'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert norm(out) == IN_DATA
|
||||
assert (str(N // 2 * N) + "it").encode() in err, "expected arithmetic sum formula"
|
||||
|
||||
# test integer --update_to
|
||||
main(sys.stderr, ['--update-to'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert norm(out) == IN_DATA
|
||||
assert (str(N - 1) + "it").encode() in err
|
||||
assert (str(N) + "it").encode() not in err
|
||||
|
||||
with closing(BytesIO()) as sys.stdin:
|
||||
sys.stdin.write(IN_DATA.replace(b'\n', b'D'))
|
||||
|
||||
# test integer --update --delim
|
||||
sys.stdin.seek(0)
|
||||
main(sys.stderr, ['--update', '--delim', 'D'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert out == IN_DATA.replace(b'\n', b'D')
|
||||
assert (str(N // 2 * N) + "it").encode() in err, "expected arithmetic sum"
|
||||
|
||||
# test integer --update_to --delim
|
||||
sys.stdin.seek(0)
|
||||
main(sys.stderr, ['--update-to', '--delim', 'D'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert out == IN_DATA.replace(b'\n', b'D')
|
||||
assert (str(N - 1) + "it").encode() in err
|
||||
assert (str(N) + "it").encode() not in err
|
||||
|
||||
# test float --update_to
|
||||
sys.stdin = [(str(i / 2.0) + '\n').encode() for i in _range(N)]
|
||||
IN_DATA = b''.join(sys.stdin)
|
||||
main(sys.stderr, ['--update-to'])
|
||||
out, err = capsysbin.readouterr()
|
||||
assert norm(out) == IN_DATA
|
||||
assert (str((N - 1) / 2.0) + "it").encode() in err
|
||||
assert (str(N / 2.0) + "it").encode() not in err
|
||||
|
||||
|
||||
@mark.slow
|
||||
@mark.skipif(IS_WIN, reason="no manpages on windows")
|
||||
def test_manpath(tmp_path):
|
||||
"""Test CLI --manpath"""
|
||||
man = tmp_path / "tqdm.1"
|
||||
assert not man.exists()
|
||||
with raises(SystemExit):
|
||||
main(argv=['--manpath', str(tmp_path)])
|
||||
assert man.is_file()
|
||||
|
||||
|
||||
@mark.slow
|
||||
@mark.skipif(IS_WIN, reason="no completion on windows")
|
||||
def test_comppath(tmp_path):
|
||||
"""Test CLI --comppath"""
|
||||
man = tmp_path / "tqdm_completion.sh"
|
||||
assert not man.exists()
|
||||
with raises(SystemExit):
|
||||
main(argv=['--comppath', str(tmp_path)])
|
||||
assert man.is_file()
|
||||
|
||||
# check most important options appear
|
||||
script = man.read_text()
|
||||
opts = {'--help', '--desc', '--total', '--leave', '--ncols', '--ascii',
|
||||
'--dynamic_ncols', '--position', '--bytes', '--nrows', '--delim',
|
||||
'--manpath', '--comppath'}
|
||||
assert all(args in script for args in opts)
|
||||
|
||||
|
||||
@restore_sys
|
||||
def test_exceptions(capsysbin):
|
||||
"""Test CLI Exceptions"""
|
||||
N = 123
|
||||
sys.stdin = [str(i) + '\n' for i in _range(N)]
|
||||
IN_DATA = ''.join(sys.stdin).encode()
|
||||
|
||||
with raises(TqdmKeyError, match="bad_arg_u_ment"):
|
||||
main(sys.stderr, argv=['-ascii', '-unit_scale', '--bad_arg_u_ment', 'foo'])
|
||||
out, _ = capsysbin.readouterr()
|
||||
assert norm(out) == IN_DATA
|
||||
|
||||
with raises(TqdmTypeError, match="invalid_bool_value"):
|
||||
main(sys.stderr, argv=['-ascii', '-unit_scale', 'invalid_bool_value'])
|
||||
out, _ = capsysbin.readouterr()
|
||||
assert norm(out) == IN_DATA
|
||||
|
||||
with raises(TqdmTypeError, match="invalid_int_value"):
|
||||
main(sys.stderr, argv=['-ascii', '--total', 'invalid_int_value'])
|
||||
out, _ = capsysbin.readouterr()
|
||||
assert norm(out) == IN_DATA
|
||||
|
||||
with raises(TqdmKeyError, match="Can only have one of --"):
|
||||
main(sys.stderr, argv=['--update', '--update_to'])
|
||||
out, _ = capsysbin.readouterr()
|
||||
assert norm(out) == IN_DATA
|
||||
|
||||
# test SystemExits
|
||||
for i in ('-h', '--help', '-v', '--version'):
|
||||
with raises(SystemExit):
|
||||
main(argv=[i])
|
7
tests/tests_notebook.py
Normal file
7
tests/tests_notebook.py
Normal file
|
@ -0,0 +1,7 @@
|
|||
from tqdm.notebook import tqdm as tqdm_notebook
|
||||
|
||||
|
||||
def test_notebook_disabled_description():
|
||||
"""Test that set_description works for disabled tqdm_notebook"""
|
||||
with tqdm_notebook(1, disable=True) as t:
|
||||
t.set_description("description")
|
219
tests/tests_pandas.py
Normal file
219
tests/tests_pandas.py
Normal file
|
@ -0,0 +1,219 @@
|
|||
from tqdm import tqdm
|
||||
|
||||
from .tests_tqdm import StringIO, closing, importorskip, mark, skip
|
||||
|
||||
pytestmark = mark.slow
|
||||
|
||||
random = importorskip('numpy.random')
|
||||
rand = random.rand
|
||||
randint = random.randint
|
||||
pd = importorskip('pandas')
|
||||
|
||||
|
||||
def test_pandas_setup():
|
||||
"""Test tqdm.pandas()"""
|
||||
with closing(StringIO()) as our_file:
|
||||
tqdm.pandas(file=our_file, leave=True, ascii=True, total=123)
|
||||
series = pd.Series(randint(0, 50, (100,)))
|
||||
series.progress_apply(lambda x: x + 10)
|
||||
res = our_file.getvalue()
|
||||
assert '100/123' in res
|
||||
|
||||
|
||||
def test_pandas_rolling_expanding():
|
||||
"""Test pandas.(Series|DataFrame).(rolling|expanding)"""
|
||||
with closing(StringIO()) as our_file:
|
||||
tqdm.pandas(file=our_file, leave=True, ascii=True)
|
||||
|
||||
series = pd.Series(randint(0, 50, (123,)))
|
||||
res1 = series.rolling(10).progress_apply(lambda x: 1, raw=True)
|
||||
res2 = series.rolling(10).apply(lambda x: 1, raw=True)
|
||||
assert res1.equals(res2)
|
||||
|
||||
res3 = series.expanding(10).progress_apply(lambda x: 2, raw=True)
|
||||
res4 = series.expanding(10).apply(lambda x: 2, raw=True)
|
||||
assert res3.equals(res4)
|
||||
|
||||
expects = ['114it'] # 123-10+1
|
||||
for exres in expects:
|
||||
our_file.seek(0)
|
||||
if our_file.getvalue().count(exres) < 2:
|
||||
our_file.seek(0)
|
||||
raise AssertionError("\nExpected:\n{0}\nIn:\n{1}\n".format(
|
||||
exres + " at least twice.", our_file.read()))
|
||||
|
||||
|
||||
def test_pandas_series():
|
||||
"""Test pandas.Series.progress_apply and .progress_map"""
|
||||
with closing(StringIO()) as our_file:
|
||||
tqdm.pandas(file=our_file, leave=True, ascii=True)
|
||||
|
||||
series = pd.Series(randint(0, 50, (123,)))
|
||||
res1 = series.progress_apply(lambda x: x + 10)
|
||||
res2 = series.apply(lambda x: x + 10)
|
||||
assert res1.equals(res2)
|
||||
|
||||
res3 = series.progress_map(lambda x: x + 10)
|
||||
res4 = series.map(lambda x: x + 10)
|
||||
assert res3.equals(res4)
|
||||
|
||||
expects = ['100%', '123/123']
|
||||
for exres in expects:
|
||||
our_file.seek(0)
|
||||
if our_file.getvalue().count(exres) < 2:
|
||||
our_file.seek(0)
|
||||
raise AssertionError("\nExpected:\n{0}\nIn:\n{1}\n".format(
|
||||
exres + " at least twice.", our_file.read()))
|
||||
|
||||
|
||||
def test_pandas_data_frame():
|
||||
"""Test pandas.DataFrame.progress_apply and .progress_applymap"""
|
||||
with closing(StringIO()) as our_file:
|
||||
tqdm.pandas(file=our_file, leave=True, ascii=True)
|
||||
df = pd.DataFrame(randint(0, 50, (100, 200)))
|
||||
|
||||
def task_func(x):
|
||||
return x + 1
|
||||
|
||||
# applymap
|
||||
res1 = df.progress_applymap(task_func)
|
||||
res2 = df.applymap(task_func)
|
||||
assert res1.equals(res2)
|
||||
|
||||
# apply unhashable
|
||||
res1 = []
|
||||
df.progress_apply(res1.extend)
|
||||
assert len(res1) == df.size
|
||||
|
||||
# apply
|
||||
for axis in [0, 1, 'index', 'columns']:
|
||||
res3 = df.progress_apply(task_func, axis=axis)
|
||||
res4 = df.apply(task_func, axis=axis)
|
||||
assert res3.equals(res4)
|
||||
|
||||
our_file.seek(0)
|
||||
if our_file.read().count('100%') < 3:
|
||||
our_file.seek(0)
|
||||
raise AssertionError("\nExpected:\n{0}\nIn:\n{1}\n".format(
|
||||
'100% at least three times', our_file.read()))
|
||||
|
||||
# apply_map, apply axis=0, apply axis=1
|
||||
expects = ['20000/20000', '200/200', '100/100']
|
||||
for exres in expects:
|
||||
our_file.seek(0)
|
||||
if our_file.getvalue().count(exres) < 1:
|
||||
our_file.seek(0)
|
||||
raise AssertionError("\nExpected:\n{0}\nIn:\n {1}\n".format(
|
||||
exres + " at least once.", our_file.read()))
|
||||
|
||||
|
||||
def test_pandas_groupby_apply():
|
||||
"""Test pandas.DataFrame.groupby(...).progress_apply"""
|
||||
with closing(StringIO()) as our_file:
|
||||
tqdm.pandas(file=our_file, leave=False, ascii=True)
|
||||
|
||||
df = pd.DataFrame(randint(0, 50, (500, 3)))
|
||||
df.groupby(0).progress_apply(lambda x: None)
|
||||
|
||||
dfs = pd.DataFrame(randint(0, 50, (500, 3)), columns=list('abc'))
|
||||
dfs.groupby(['a']).progress_apply(lambda x: None)
|
||||
|
||||
df2 = df = pd.DataFrame({'a': randint(1, 8, 10000), 'b': rand(10000)})
|
||||
res1 = df2.groupby("a").apply(max)
|
||||
res2 = df2.groupby("a").progress_apply(max)
|
||||
assert res1.equals(res2)
|
||||
|
||||
our_file.seek(0)
|
||||
|
||||
# don't expect final output since no `leave` and
|
||||
# high dynamic `miniters`
|
||||
nexres = '100%|##########|'
|
||||
if nexres in our_file.read():
|
||||
our_file.seek(0)
|
||||
raise AssertionError("\nDid not expect:\n{0}\nIn:{1}\n".format(
|
||||
nexres, our_file.read()))
|
||||
|
||||
with closing(StringIO()) as our_file:
|
||||
tqdm.pandas(file=our_file, leave=True, ascii=True)
|
||||
|
||||
dfs = pd.DataFrame(randint(0, 50, (500, 3)), columns=list('abc'))
|
||||
dfs.loc[0] = [2, 1, 1]
|
||||
dfs['d'] = 100
|
||||
|
||||
expects = ['500/500', '1/1', '4/4', '2/2']
|
||||
dfs.groupby(dfs.index).progress_apply(lambda x: None)
|
||||
dfs.groupby('d').progress_apply(lambda x: None)
|
||||
dfs.groupby(dfs.columns, axis=1).progress_apply(lambda x: None)
|
||||
dfs.groupby([2, 2, 1, 1], axis=1).progress_apply(lambda x: None)
|
||||
|
||||
our_file.seek(0)
|
||||
if our_file.read().count('100%') < 4:
|
||||
our_file.seek(0)
|
||||
raise AssertionError("\nExpected:\n{0}\nIn:\n{1}\n".format(
|
||||
'100% at least four times', our_file.read()))
|
||||
|
||||
for exres in expects:
|
||||
our_file.seek(0)
|
||||
if our_file.getvalue().count(exres) < 1:
|
||||
our_file.seek(0)
|
||||
raise AssertionError("\nExpected:\n{0}\nIn:\n {1}\n".format(
|
||||
exres + " at least once.", our_file.read()))
|
||||
|
||||
|
||||
def test_pandas_leave():
|
||||
"""Test pandas with `leave=True`"""
|
||||
with closing(StringIO()) as our_file:
|
||||
df = pd.DataFrame(randint(0, 100, (1000, 6)))
|
||||
tqdm.pandas(file=our_file, leave=True, ascii=True)
|
||||
df.groupby(0).progress_apply(lambda x: None)
|
||||
|
||||
our_file.seek(0)
|
||||
|
||||
exres = '100%|##########| 100/100'
|
||||
if exres not in our_file.read():
|
||||
our_file.seek(0)
|
||||
raise AssertionError("\nExpected:\n{0}\nIn:{1}\n".format(
|
||||
exres, our_file.read()))
|
||||
|
||||
|
||||
def test_pandas_apply_args_deprecation():
|
||||
"""Test warning info in
|
||||
`pandas.Dataframe(Series).progress_apply(func, *args)`"""
|
||||
try:
|
||||
from tqdm import tqdm_pandas
|
||||
except ImportError as err:
|
||||
skip(str(err))
|
||||
|
||||
with closing(StringIO()) as our_file:
|
||||
tqdm_pandas(tqdm(file=our_file, leave=False, ascii=True, ncols=20))
|
||||
df = pd.DataFrame(randint(0, 50, (500, 3)))
|
||||
df.progress_apply(lambda x: None, 1) # 1 shall cause a warning
|
||||
# Check deprecation message
|
||||
res = our_file.getvalue()
|
||||
assert all(i in res for i in (
|
||||
"TqdmDeprecationWarning", "not supported",
|
||||
"keyword arguments instead"))
|
||||
|
||||
|
||||
def test_pandas_deprecation():
|
||||
"""Test bar object instance as argument deprecation"""
|
||||
try:
|
||||
from tqdm import tqdm_pandas
|
||||
except ImportError as err:
|
||||
skip(str(err))
|
||||
|
||||
with closing(StringIO()) as our_file:
|
||||
tqdm_pandas(tqdm(file=our_file, leave=False, ascii=True, ncols=20))
|
||||
df = pd.DataFrame(randint(0, 50, (500, 3)))
|
||||
df.groupby(0).progress_apply(lambda x: None)
|
||||
# Check deprecation message
|
||||
assert "TqdmDeprecationWarning" in our_file.getvalue()
|
||||
assert "instead of `tqdm_pandas(tqdm(...))`" in our_file.getvalue()
|
||||
|
||||
with closing(StringIO()) as our_file:
|
||||
tqdm_pandas(tqdm, file=our_file, leave=False, ascii=True, ncols=20)
|
||||
df = pd.DataFrame(randint(0, 50, (500, 3)))
|
||||
df.groupby(0).progress_apply(lambda x: None)
|
||||
# Check deprecation message
|
||||
assert "TqdmDeprecationWarning" in our_file.getvalue()
|
||||
assert "instead of `tqdm_pandas(tqdm, ...)`" in our_file.getvalue()
|
325
tests/tests_perf.py
Normal file
325
tests/tests_perf.py
Normal file
|
@ -0,0 +1,325 @@
|
|||
from __future__ import division, print_function
|
||||
|
||||
import sys
|
||||
from contextlib import contextmanager
|
||||
from functools import wraps
|
||||
from time import sleep, time
|
||||
|
||||
# Use relative/cpu timer to have reliable timings when there is a sudden load
|
||||
try:
|
||||
from time import process_time
|
||||
except ImportError:
|
||||
from time import clock
|
||||
process_time = clock
|
||||
|
||||
from tqdm import tqdm, trange
|
||||
|
||||
from .tests_tqdm import _range, importorskip, mark, patch_lock, skip
|
||||
|
||||
pytestmark = mark.slow
|
||||
|
||||
|
||||
def cpu_sleep(t):
|
||||
"""Sleep the given amount of cpu time"""
|
||||
start = process_time()
|
||||
while (process_time() - start) < t:
|
||||
pass
|
||||
|
||||
|
||||
def checkCpuTime(sleeptime=0.2):
|
||||
"""Check if cpu time works correctly"""
|
||||
if checkCpuTime.passed:
|
||||
return True
|
||||
# First test that sleeping does not consume cputime
|
||||
start1 = process_time()
|
||||
sleep(sleeptime)
|
||||
t1 = process_time() - start1
|
||||
|
||||
# secondly check by comparing to cpusleep (where we actually do something)
|
||||
start2 = process_time()
|
||||
cpu_sleep(sleeptime)
|
||||
t2 = process_time() - start2
|
||||
|
||||
if abs(t1) < 0.0001 and t1 < t2 / 10:
|
||||
checkCpuTime.passed = True
|
||||
return True
|
||||
skip("cpu time not reliable on this machine")
|
||||
|
||||
|
||||
checkCpuTime.passed = False
|
||||
|
||||
|
||||
@contextmanager
|
||||
def relative_timer():
|
||||
"""yields a context timer function which stops ticking on exit"""
|
||||
start = process_time()
|
||||
|
||||
def elapser():
|
||||
return process_time() - start
|
||||
|
||||
yield lambda: elapser()
|
||||
spent = elapser()
|
||||
|
||||
def elapser(): # NOQA
|
||||
return spent
|
||||
|
||||
|
||||
def retry_on_except(n=3, check_cpu_time=True):
|
||||
"""decroator for retrying `n` times before raising Exceptions"""
|
||||
def wrapper(func):
|
||||
"""actual decorator"""
|
||||
@wraps(func)
|
||||
def test_inner(*args, **kwargs):
|
||||
"""may skip if `check_cpu_time` fails"""
|
||||
for i in range(1, n + 1):
|
||||
try:
|
||||
if check_cpu_time:
|
||||
checkCpuTime()
|
||||
func(*args, **kwargs)
|
||||
except Exception:
|
||||
if i >= n:
|
||||
raise
|
||||
else:
|
||||
return
|
||||
return test_inner
|
||||
return wrapper
|
||||
|
||||
|
||||
def simple_progress(iterable=None, total=None, file=sys.stdout, desc='',
|
||||
leave=False, miniters=1, mininterval=0.1, width=60):
|
||||
"""Simple progress bar reproducing tqdm's major features"""
|
||||
n = [0] # use a closure
|
||||
start_t = [time()]
|
||||
last_n = [0]
|
||||
last_t = [0]
|
||||
if iterable is not None:
|
||||
total = len(iterable)
|
||||
|
||||
def format_interval(t):
|
||||
mins, s = divmod(int(t), 60)
|
||||
h, m = divmod(mins, 60)
|
||||
if h:
|
||||
return '{0:d}:{1:02d}:{2:02d}'.format(h, m, s)
|
||||
else:
|
||||
return '{0:02d}:{1:02d}'.format(m, s)
|
||||
|
||||
def update_and_print(i=1):
|
||||
n[0] += i
|
||||
if (n[0] - last_n[0]) >= miniters:
|
||||
last_n[0] = n[0]
|
||||
|
||||
if (time() - last_t[0]) >= mininterval:
|
||||
last_t[0] = time() # last_t[0] == current time
|
||||
|
||||
spent = last_t[0] - start_t[0]
|
||||
spent_fmt = format_interval(spent)
|
||||
rate = n[0] / spent if spent > 0 else 0
|
||||
rate_fmt = "%.2fs/it" % (1.0 / rate) if 0.0 < rate < 1.0 else "%.2fit/s" % rate
|
||||
|
||||
frac = n[0] / total
|
||||
percentage = int(frac * 100)
|
||||
eta = (total - n[0]) / rate if rate > 0 else 0
|
||||
eta_fmt = format_interval(eta)
|
||||
|
||||
# full_bar = "#" * int(frac * width)
|
||||
barfill = " " * int((1.0 - frac) * width)
|
||||
bar_length, frac_bar_length = divmod(int(frac * width * 10), 10)
|
||||
full_bar = '#' * bar_length
|
||||
frac_bar = chr(48 + frac_bar_length) if frac_bar_length else ' '
|
||||
|
||||
file.write("\r%s %i%%|%s%s%s| %i/%i [%s<%s, %s]" %
|
||||
(desc, percentage, full_bar, frac_bar, barfill, n[0],
|
||||
total, spent_fmt, eta_fmt, rate_fmt))
|
||||
|
||||
if n[0] == total and leave:
|
||||
file.write("\n")
|
||||
file.flush()
|
||||
|
||||
def update_and_yield():
|
||||
for elt in iterable:
|
||||
yield elt
|
||||
update_and_print()
|
||||
|
||||
update_and_print(0)
|
||||
if iterable is not None:
|
||||
return update_and_yield()
|
||||
else:
|
||||
return update_and_print
|
||||
|
||||
|
||||
def assert_performance(thresh, name_left, time_left, name_right, time_right):
|
||||
"""raises if time_left > thresh * time_right"""
|
||||
if time_left > thresh * time_right:
|
||||
raise ValueError(
|
||||
('{name[0]}: {time[0]:f}, '
|
||||
'{name[1]}: {time[1]:f}, '
|
||||
'ratio {ratio:f} > {thresh:f}').format(
|
||||
name=(name_left, name_right),
|
||||
time=(time_left, time_right),
|
||||
ratio=time_left / time_right, thresh=thresh))
|
||||
|
||||
|
||||
@retry_on_except()
|
||||
def test_iter_basic_overhead():
|
||||
"""Test overhead of iteration based tqdm"""
|
||||
total = int(1e6)
|
||||
|
||||
a = 0
|
||||
with trange(total) as t:
|
||||
with relative_timer() as time_tqdm:
|
||||
for i in t:
|
||||
a += i
|
||||
assert a == (total ** 2 - total) / 2.0
|
||||
|
||||
a = 0
|
||||
with relative_timer() as time_bench:
|
||||
for i in _range(total):
|
||||
a += i
|
||||
sys.stdout.write(str(a))
|
||||
|
||||
assert_performance(3, 'trange', time_tqdm(), 'range', time_bench())
|
||||
|
||||
|
||||
@retry_on_except()
|
||||
def test_manual_basic_overhead():
|
||||
"""Test overhead of manual tqdm"""
|
||||
total = int(1e6)
|
||||
|
||||
with tqdm(total=total * 10, leave=True) as t:
|
||||
a = 0
|
||||
with relative_timer() as time_tqdm:
|
||||
for i in _range(total):
|
||||
a += i
|
||||
t.update(10)
|
||||
|
||||
a = 0
|
||||
with relative_timer() as time_bench:
|
||||
for i in _range(total):
|
||||
a += i
|
||||
sys.stdout.write(str(a))
|
||||
|
||||
assert_performance(5, 'tqdm', time_tqdm(), 'range', time_bench())
|
||||
|
||||
|
||||
def worker(total, blocking=True):
|
||||
def incr_bar(x):
|
||||
for _ in trange(total, lock_args=None if blocking else (False,),
|
||||
miniters=1, mininterval=0, maxinterval=0):
|
||||
pass
|
||||
return x + 1
|
||||
return incr_bar
|
||||
|
||||
|
||||
@retry_on_except()
|
||||
@patch_lock(thread=True)
|
||||
def test_lock_args():
|
||||
"""Test overhead of nonblocking threads"""
|
||||
ThreadPoolExecutor = importorskip('concurrent.futures').ThreadPoolExecutor
|
||||
|
||||
total = 16
|
||||
subtotal = 10000
|
||||
|
||||
with ThreadPoolExecutor() as pool:
|
||||
sys.stderr.write('block ... ')
|
||||
sys.stderr.flush()
|
||||
with relative_timer() as time_tqdm:
|
||||
res = list(pool.map(worker(subtotal, True), range(total)))
|
||||
assert sum(res) == sum(range(total)) + total
|
||||
sys.stderr.write('noblock ... ')
|
||||
sys.stderr.flush()
|
||||
with relative_timer() as time_noblock:
|
||||
res = list(pool.map(worker(subtotal, False), range(total)))
|
||||
assert sum(res) == sum(range(total)) + total
|
||||
|
||||
assert_performance(0.5, 'noblock', time_noblock(), 'tqdm', time_tqdm())
|
||||
|
||||
|
||||
@retry_on_except(10)
|
||||
def test_iter_overhead_hard():
|
||||
"""Test overhead of iteration based tqdm (hard)"""
|
||||
total = int(1e5)
|
||||
|
||||
a = 0
|
||||
with trange(total, leave=True, miniters=1,
|
||||
mininterval=0, maxinterval=0) as t:
|
||||
with relative_timer() as time_tqdm:
|
||||
for i in t:
|
||||
a += i
|
||||
assert a == (total ** 2 - total) / 2.0
|
||||
|
||||
a = 0
|
||||
with relative_timer() as time_bench:
|
||||
for i in _range(total):
|
||||
a += i
|
||||
sys.stdout.write(("%i" % a) * 40)
|
||||
|
||||
assert_performance(130, 'trange', time_tqdm(), 'range', time_bench())
|
||||
|
||||
|
||||
@retry_on_except(10)
|
||||
def test_manual_overhead_hard():
|
||||
"""Test overhead of manual tqdm (hard)"""
|
||||
total = int(1e5)
|
||||
|
||||
with tqdm(total=total * 10, leave=True, miniters=1,
|
||||
mininterval=0, maxinterval=0) as t:
|
||||
a = 0
|
||||
with relative_timer() as time_tqdm:
|
||||
for i in _range(total):
|
||||
a += i
|
||||
t.update(10)
|
||||
|
||||
a = 0
|
||||
with relative_timer() as time_bench:
|
||||
for i in _range(total):
|
||||
a += i
|
||||
sys.stdout.write(("%i" % a) * 40)
|
||||
|
||||
assert_performance(130, 'tqdm', time_tqdm(), 'range', time_bench())
|
||||
|
||||
|
||||
@retry_on_except(10)
|
||||
def test_iter_overhead_simplebar_hard():
|
||||
"""Test overhead of iteration based tqdm vs simple progress bar (hard)"""
|
||||
total = int(1e4)
|
||||
|
||||
a = 0
|
||||
with trange(total, leave=True, miniters=1,
|
||||
mininterval=0, maxinterval=0) as t:
|
||||
with relative_timer() as time_tqdm:
|
||||
for i in t:
|
||||
a += i
|
||||
assert a == (total ** 2 - total) / 2.0
|
||||
|
||||
a = 0
|
||||
s = simple_progress(_range(total), leave=True,
|
||||
miniters=1, mininterval=0)
|
||||
with relative_timer() as time_bench:
|
||||
for i in s:
|
||||
a += i
|
||||
|
||||
assert_performance(10, 'trange', time_tqdm(), 'simple_progress', time_bench())
|
||||
|
||||
|
||||
@retry_on_except(10)
|
||||
def test_manual_overhead_simplebar_hard():
|
||||
"""Test overhead of manual tqdm vs simple progress bar (hard)"""
|
||||
total = int(1e4)
|
||||
|
||||
with tqdm(total=total * 10, leave=True, miniters=1,
|
||||
mininterval=0, maxinterval=0) as t:
|
||||
a = 0
|
||||
with relative_timer() as time_tqdm:
|
||||
for i in _range(total):
|
||||
a += i
|
||||
t.update(10)
|
||||
|
||||
simplebar_update = simple_progress(total=total * 10, leave=True,
|
||||
miniters=1, mininterval=0)
|
||||
a = 0
|
||||
with relative_timer() as time_bench:
|
||||
for i in _range(total):
|
||||
a += i
|
||||
simplebar_update(10)
|
||||
|
||||
assert_performance(10, 'tqdm', time_tqdm(), 'simple_progress', time_bench())
|
10
tests/tests_rich.py
Normal file
10
tests/tests_rich.py
Normal file
|
@ -0,0 +1,10 @@
|
|||
"""Test `tqdm.rich`."""
|
||||
import sys
|
||||
|
||||
from .tests_tqdm import importorskip, mark
|
||||
|
||||
|
||||
@mark.skipif(sys.version_info[:3] < (3, 6, 1), reason="`rich` needs py>=3.6.1")
|
||||
def test_rich_import():
|
||||
"""Test `tqdm.rich` import"""
|
||||
importorskip('tqdm.rich')
|
224
tests/tests_synchronisation.py
Normal file
224
tests/tests_synchronisation.py
Normal file
|
@ -0,0 +1,224 @@
|
|||
from __future__ import division
|
||||
|
||||
import sys
|
||||
from functools import wraps
|
||||
from threading import Event
|
||||
from time import sleep, time
|
||||
|
||||
from tqdm import TMonitor, tqdm, trange
|
||||
|
||||
from .tests_perf import retry_on_except
|
||||
from .tests_tqdm import StringIO, closing, importorskip, patch_lock, skip
|
||||
|
||||
|
||||
class Time(object):
|
||||
"""Fake time class class providing an offset"""
|
||||
offset = 0
|
||||
|
||||
@classmethod
|
||||
def reset(cls):
|
||||
"""zeroes internal offset"""
|
||||
cls.offset = 0
|
||||
|
||||
@classmethod
|
||||
def time(cls):
|
||||
"""time.time() + offset"""
|
||||
return time() + cls.offset
|
||||
|
||||
@staticmethod
|
||||
def sleep(dur):
|
||||
"""identical to time.sleep()"""
|
||||
sleep(dur)
|
||||
|
||||
@classmethod
|
||||
def fake_sleep(cls, dur):
|
||||
"""adds `dur` to internal offset"""
|
||||
cls.offset += dur
|
||||
sleep(0.000001) # sleep to allow interrupt (instead of pass)
|
||||
|
||||
|
||||
def FakeEvent():
|
||||
"""patched `threading.Event` where `wait()` uses `Time.fake_sleep()`"""
|
||||
event = Event() # not a class in py2 so can't inherit
|
||||
|
||||
def wait(timeout=None):
|
||||
"""uses Time.fake_sleep"""
|
||||
if timeout is not None:
|
||||
Time.fake_sleep(timeout)
|
||||
return event.is_set()
|
||||
|
||||
event.wait = wait
|
||||
return event
|
||||
|
||||
|
||||
def patch_sleep(func):
|
||||
"""Temporarily makes TMonitor use Time.fake_sleep"""
|
||||
@wraps(func)
|
||||
def inner(*args, **kwargs):
|
||||
"""restores TMonitor on completion regardless of Exceptions"""
|
||||
TMonitor._test["time"] = Time.time
|
||||
TMonitor._test["Event"] = FakeEvent
|
||||
if tqdm.monitor:
|
||||
assert not tqdm.monitor.get_instances()
|
||||
tqdm.monitor.exit()
|
||||
del tqdm.monitor
|
||||
tqdm.monitor = None
|
||||
try:
|
||||
return func(*args, **kwargs)
|
||||
finally:
|
||||
# Check that class var monitor is deleted if no instance left
|
||||
tqdm.monitor_interval = 10
|
||||
if tqdm.monitor:
|
||||
assert not tqdm.monitor.get_instances()
|
||||
tqdm.monitor.exit()
|
||||
del tqdm.monitor
|
||||
tqdm.monitor = None
|
||||
TMonitor._test.pop("Event")
|
||||
TMonitor._test.pop("time")
|
||||
|
||||
return inner
|
||||
|
||||
|
||||
def cpu_timify(t, timer=Time):
|
||||
"""Force tqdm to use the specified timer instead of system-wide time"""
|
||||
t._time = timer.time
|
||||
t._sleep = timer.fake_sleep
|
||||
t.start_t = t.last_print_t = t._time()
|
||||
return timer
|
||||
|
||||
|
||||
class FakeTqdm(object):
|
||||
_instances = set()
|
||||
get_lock = tqdm.get_lock
|
||||
|
||||
|
||||
def incr(x):
|
||||
return x + 1
|
||||
|
||||
|
||||
def incr_bar(x):
|
||||
with closing(StringIO()) as our_file:
|
||||
for _ in trange(x, lock_args=(False,), file=our_file):
|
||||
pass
|
||||
return incr(x)
|
||||
|
||||
|
||||
@patch_sleep
|
||||
def test_monitor_thread():
|
||||
"""Test dummy monitoring thread"""
|
||||
monitor = TMonitor(FakeTqdm, 10)
|
||||
# Test if alive, then killed
|
||||
assert monitor.report()
|
||||
monitor.exit()
|
||||
assert not monitor.report()
|
||||
assert not monitor.is_alive()
|
||||
del monitor
|
||||
|
||||
|
||||
@patch_sleep
|
||||
def test_monitoring_and_cleanup():
|
||||
"""Test for stalled tqdm instance and monitor deletion"""
|
||||
# Note: should fix miniters for these tests, else with dynamic_miniters
|
||||
# it's too complicated to handle with monitoring update and maxinterval...
|
||||
maxinterval = tqdm.monitor_interval
|
||||
assert maxinterval == 10
|
||||
total = 1000
|
||||
|
||||
with closing(StringIO()) as our_file:
|
||||
with tqdm(total=total, file=our_file, miniters=500, mininterval=0.1,
|
||||
maxinterval=maxinterval) as t:
|
||||
cpu_timify(t, Time)
|
||||
# Do a lot of iterations in a small timeframe
|
||||
# (smaller than monitor interval)
|
||||
Time.fake_sleep(maxinterval / 10) # monitor won't wake up
|
||||
t.update(500)
|
||||
# check that our fixed miniters is still there
|
||||
assert t.miniters <= 500 # TODO: should really be == 500
|
||||
# Then do 1 it after monitor interval, so that monitor kicks in
|
||||
Time.fake_sleep(maxinterval)
|
||||
t.update(1)
|
||||
# Wait for the monitor to get out of sleep's loop and update tqdm.
|
||||
timeend = Time.time()
|
||||
while not (t.monitor.woken >= timeend and t.miniters == 1):
|
||||
Time.fake_sleep(1) # Force awake up if it woken too soon
|
||||
assert t.miniters == 1 # check that monitor corrected miniters
|
||||
# Note: at this point, there may be a race condition: monitor saved
|
||||
# current woken time but Time.sleep() happen just before monitor
|
||||
# sleep. To fix that, either sleep here or increase time in a loop
|
||||
# to ensure that monitor wakes up at some point.
|
||||
|
||||
# Try again but already at miniters = 1 so nothing will be done
|
||||
Time.fake_sleep(maxinterval)
|
||||
t.update(2)
|
||||
timeend = Time.time()
|
||||
while t.monitor.woken < timeend:
|
||||
Time.fake_sleep(1) # Force awake if it woken too soon
|
||||
# Wait for the monitor to get out of sleep's loop and update
|
||||
# tqdm
|
||||
assert t.miniters == 1 # check that monitor corrected miniters
|
||||
|
||||
|
||||
@patch_sleep
|
||||
def test_monitoring_multi():
|
||||
"""Test on multiple bars, one not needing miniters adjustment"""
|
||||
# Note: should fix miniters for these tests, else with dynamic_miniters
|
||||
# it's too complicated to handle with monitoring update and maxinterval...
|
||||
maxinterval = tqdm.monitor_interval
|
||||
assert maxinterval == 10
|
||||
total = 1000
|
||||
|
||||
with closing(StringIO()) as our_file:
|
||||
with tqdm(total=total, file=our_file, miniters=500, mininterval=0.1,
|
||||
maxinterval=maxinterval) as t1:
|
||||
# Set high maxinterval for t2 so monitor does not need to adjust it
|
||||
with tqdm(total=total, file=our_file, miniters=500, mininterval=0.1,
|
||||
maxinterval=1E5) as t2:
|
||||
cpu_timify(t1, Time)
|
||||
cpu_timify(t2, Time)
|
||||
# Do a lot of iterations in a small timeframe
|
||||
Time.fake_sleep(maxinterval / 10)
|
||||
t1.update(500)
|
||||
t2.update(500)
|
||||
assert t1.miniters <= 500 # TODO: should really be == 500
|
||||
assert t2.miniters == 500
|
||||
# Then do 1 it after monitor interval, so that monitor kicks in
|
||||
Time.fake_sleep(maxinterval)
|
||||
t1.update(1)
|
||||
t2.update(1)
|
||||
# Wait for the monitor to get out of sleep and update tqdm
|
||||
timeend = Time.time()
|
||||
while not (t1.monitor.woken >= timeend and t1.miniters == 1):
|
||||
Time.fake_sleep(1)
|
||||
assert t1.miniters == 1 # check that monitor corrected miniters
|
||||
assert t2.miniters == 500 # check that t2 was not adjusted
|
||||
|
||||
|
||||
def test_imap():
|
||||
"""Test multiprocessing.Pool"""
|
||||
try:
|
||||
from multiprocessing import Pool
|
||||
except ImportError as err:
|
||||
skip(str(err))
|
||||
|
||||
pool = Pool()
|
||||
res = list(tqdm(pool.imap(incr, range(100)), disable=True))
|
||||
pool.close()
|
||||
assert res[-1] == 100
|
||||
|
||||
|
||||
# py2: locks won't propagate to incr_bar so may cause `AttributeError`
|
||||
@retry_on_except(n=3 if sys.version_info < (3,) else 1, check_cpu_time=False)
|
||||
@patch_lock(thread=True)
|
||||
def test_threadpool():
|
||||
"""Test concurrent.futures.ThreadPoolExecutor"""
|
||||
ThreadPoolExecutor = importorskip('concurrent.futures').ThreadPoolExecutor
|
||||
|
||||
with ThreadPoolExecutor(8) as pool:
|
||||
try:
|
||||
res = list(tqdm(pool.map(incr_bar, range(100)), disable=True))
|
||||
except AttributeError:
|
||||
if sys.version_info < (3,):
|
||||
skip("not supported on py2")
|
||||
else:
|
||||
raise
|
||||
assert sum(res) == sum(range(1, 101))
|
7
tests/tests_tk.py
Normal file
7
tests/tests_tk.py
Normal file
|
@ -0,0 +1,7 @@
|
|||
"""Test `tqdm.tk`."""
|
||||
from .tests_tqdm import importorskip
|
||||
|
||||
|
||||
def test_tk_import():
|
||||
"""Test `tqdm.tk` import"""
|
||||
importorskip('tqdm.tk')
|
1996
tests/tests_tqdm.py
Normal file
1996
tests/tests_tqdm.py
Normal file
File diff suppressed because it is too large
Load diff
14
tests/tests_version.py
Normal file
14
tests/tests_version.py
Normal file
|
@ -0,0 +1,14 @@
|
|||
"""Test `tqdm.__version__`."""
|
||||
import re
|
||||
from ast import literal_eval
|
||||
|
||||
|
||||
def test_version():
|
||||
"""Test version string"""
|
||||
from tqdm import __version__
|
||||
version_parts = re.split('[.-]', __version__)
|
||||
if __version__ != "UNKNOWN":
|
||||
assert 3 <= len(version_parts), "must have at least Major.minor.patch"
|
||||
assert all(
|
||||
isinstance(literal_eval(i), int) for i in version_parts[:3]
|
||||
), "Version Major.minor.patch must be 3 integers"
|
515
tests_notebook.ipynb
Normal file
515
tests_notebook.ipynb
Normal file
|
@ -0,0 +1,515 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This file is part of the [test suite](./tests) and will be moved there when [nbval#116](https://github.com/computationalmodelling/nbval/issues/116#issuecomment-793148404) is fixed.\n",
|
||||
"\n",
|
||||
"See [DEMO.ipynb](DEMO.ipynb) instead for notebook examples."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from functools import partial\n",
|
||||
"from time import sleep\n",
|
||||
"\n",
|
||||
"from tqdm.notebook import tqdm_notebook\n",
|
||||
"from tqdm.notebook import tnrange\n",
|
||||
"\n",
|
||||
"# avoid displaying widgets by default (pollutes output cells)\n",
|
||||
"tqdm = partial(tqdm_notebook, display=False)\n",
|
||||
"trange = partial(tnrange, display=False)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Help on function display in module tqdm.notebook:\n",
|
||||
"\n",
|
||||
"display(self, msg=None, pos=None, close=False, bar_style=None, check_delay=True)\n",
|
||||
" Use `self.sp` to display `msg` in the specified `pos`.\n",
|
||||
" \n",
|
||||
" Consider overloading this function when inheriting to use e.g.:\n",
|
||||
" `self.some_frontend(**self.format_dict)` instead of `self.sp`.\n",
|
||||
" \n",
|
||||
" Parameters\n",
|
||||
" ----------\n",
|
||||
" msg : str, optional. What to display (default: `repr(self)`).\n",
|
||||
" pos : int, optional. Position to `moveto`\n",
|
||||
" (default: `abs(self.pos)`).\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"help(tqdm_notebook.display)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "7c18c038bf964b55941e228503292506",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
" 0%| | 0/9 [00:00<?, ?it/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"0\n",
|
||||
"1\n",
|
||||
"2\n",
|
||||
"3\n",
|
||||
"4\n",
|
||||
"5\n",
|
||||
"6\n",
|
||||
"7\n",
|
||||
"8\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "e29668be41ca4e40b16fb98572b333a5",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
" 0%| | 0/9 [00:00<?, ?it/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"True"
|
||||
]
|
||||
},
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: basic use\n",
|
||||
"with tqdm_notebook(range(9)) as t:\n",
|
||||
" for i in t:\n",
|
||||
" print(i)\n",
|
||||
"assert t.container.children[1].bar_style == 'success'\n",
|
||||
"\n",
|
||||
"t = tqdm_notebook(total=9)\n",
|
||||
"t.update()\n",
|
||||
"t.refresh()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 11%|█ | 1/9 [00:00<00:00, 17.73it/s]\n",
|
||||
" 20%|██ | 1/5 [00:00<00:00, 341.50it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: reset\n",
|
||||
"print(t)\n",
|
||||
"t.reset(total=5)\n",
|
||||
"t.update(1)\n",
|
||||
"print(t)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: bar_style\n",
|
||||
"assert t.container.children[1].bar_style != 'danger'\n",
|
||||
"t.close()\n",
|
||||
"assert t.container.children[1].bar_style == 'danger'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 0/8 [00:00<?, ?it/s]\n",
|
||||
" 0%| | 0/8 [00:00<?, ?it/s]\n",
|
||||
"1\n",
|
||||
" 0%| | 0/8 [00:00<?, ?it/s]\n",
|
||||
" 0%| | 0/8 [00:00<?, ?it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: repr\n",
|
||||
"with trange(1, 9) as t:\n",
|
||||
" print(t)\n",
|
||||
" print(t.container)\n",
|
||||
" it = iter(t)\n",
|
||||
" print(next(it))\n",
|
||||
" print(t)\n",
|
||||
" print(t.container)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"t = trange(9)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 0/9 [00:00<?, ?it/s]\n",
|
||||
" 0%| | 0/9 [00:00<?, ?it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: display pre\n",
|
||||
"print(t)\n",
|
||||
"print(t.container)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"for i in t:\n",
|
||||
" pass"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"100%|██████████| 9/9 [00:00<00:00, 132.23it/s]\n",
|
||||
"100%|##########| 9/9 [00:00<00:00, 131.02it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: display post\n",
|
||||
"print(t)\n",
|
||||
"print(t.container)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"no total: 0it [00:00, ?it/s]\n",
|
||||
"no total: 1it [00:00, 47.83it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: no total\n",
|
||||
"with tqdm(desc=\"no total\") as t:\n",
|
||||
" print(t)\n",
|
||||
" t.update()\n",
|
||||
" print(t)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 0/9 [00:00<?, ?it/s]\n",
|
||||
" 11%|███▍ | 1/9 [00:00<00:00, 45.06it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: ncols\n",
|
||||
"with trange(9, ncols=66) as t:\n",
|
||||
" print(t)\n",
|
||||
" for i in t:\n",
|
||||
" if i == 1:\n",
|
||||
" break\n",
|
||||
" print(t)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\n",
|
||||
"100%|██████████| 1/1 [00:00<00:00, 54.60it/s]\n",
|
||||
" 0%| | 0/9 [00:00<?, ?it/s]\n",
|
||||
" 11%|█ | 1/9 [00:00<00:00, 57.43it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: leave\n",
|
||||
"def is_hidden(widget):\n",
|
||||
" return ('hidden', False, None) == (\n",
|
||||
" getattr(getattr(widget, \"layout\", None), \"visibility\", 'visible'), # ipyw>=8\n",
|
||||
" getattr(widget, \"visible\", False), getattr(widget, \"_ipython_display_\", None)) # ipyw<8\n",
|
||||
"\n",
|
||||
"assert not is_hidden(t.container)\n",
|
||||
"for total in (1, 9):\n",
|
||||
" with tqdm(total=total, leave=False) as t:\n",
|
||||
" print(t)\n",
|
||||
" t.update()\n",
|
||||
" print(t)\n",
|
||||
" assert total != 1 or is_hidden(t.container)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"0it [00:00, ?it/s]\n",
|
||||
"1it [00:00, 47.87it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: no total\n",
|
||||
"with tqdm() as t:\n",
|
||||
" print(t)\n",
|
||||
" t.update()\n",
|
||||
" print(t)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"disable: None\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\n",
|
||||
"100%|██████████| 1/1 [00:00<00:00, 53.24it/s]\n",
|
||||
" 0%| | 0/9 [00:00<?, ?it/s]\n",
|
||||
" 11%|█ | 1/9 [00:00<00:00, 1752.01it/s]\n",
|
||||
"0it [00:00, ?it/s]\n",
|
||||
"1it [00:00, 35.88it/s]\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\n",
|
||||
"100%|██████████| 1/1 [00:00<00:00, 1880.85it/s]\n",
|
||||
"disable: True\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\n",
|
||||
" 0%| | 0/9 [00:00<?, ?it/s]\n",
|
||||
" 0%| | 0/9 [00:00<?, ?it/s]\n",
|
||||
"0it [00:00, ?it/s]\n",
|
||||
"0it [00:00, ?it/s]\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\n",
|
||||
" 0%| | 0/1 [00:00<?, ?it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: reset and disable\n",
|
||||
"for disable in (None, True):\n",
|
||||
" print(\"disable:\", disable)\n",
|
||||
" with tqdm(total=1, disable=disable) as t:\n",
|
||||
" print(t)\n",
|
||||
" t.update()\n",
|
||||
" print(t)\n",
|
||||
"\n",
|
||||
" t.reset(total=9)\n",
|
||||
" print(t)\n",
|
||||
" t.update()\n",
|
||||
" print(t)\n",
|
||||
" with tqdm(disable=disable) as t:\n",
|
||||
" print(t)\n",
|
||||
" t.update()\n",
|
||||
" print(t)\n",
|
||||
"\n",
|
||||
" t.reset(total=1)\n",
|
||||
" print(t)\n",
|
||||
" t.update()\n",
|
||||
" print(t)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%|| 0/1 [00:00<?, ?it/s]\n",
|
||||
"100%|| 1/1 [00:00<00:00, 32.57it/s]\n",
|
||||
" 0%| \n",
|
||||
"100%|██████████\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: bar_format\n",
|
||||
"with tqdm(total=1, bar_format='{l_bar}{r_bar}') as t:\n",
|
||||
" print(t)\n",
|
||||
" t.update()\n",
|
||||
" print(t)\n",
|
||||
"\n",
|
||||
"with tqdm(total=1, bar_format='{l_bar}{bar}') as t:\n",
|
||||
" print(t)\n",
|
||||
" t.update()\n",
|
||||
" print(t)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 17,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 0%|\u001b[33m \u001b[0m| 0/1 [00:00<?, ?it/s]\n",
|
||||
"100%|\u001b[33m██████████\u001b[0m| 1/1 [00:00<00:00, 47.14it/s]\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: colour\n",
|
||||
"assert t.colour != 'yellow'\n",
|
||||
"with tqdm(total=1, colour='yellow') as t:\n",
|
||||
" print(t)\n",
|
||||
" t.update()\n",
|
||||
" print(t)\n",
|
||||
" assert t.colour == 'yellow'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: delay no trigger\n",
|
||||
"with tqdm_notebook(total=1, delay=10) as t:\n",
|
||||
" t.update()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 19,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "fe102eedbb4f437783fbd0cff32f6613",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"100%|##########| 1/1 [00:00<00:00, 7.68it/s]"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# NBVAL_TEST_NAME: delay trigger\n",
|
||||
"with tqdm_notebook(total=1, delay=0.1) as t:\n",
|
||||
" sleep(0.1)\n",
|
||||
" t.update()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python [conda env:tqdm]",
|
||||
"language": "python",
|
||||
"name": "conda-env-tqdm-py"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython"
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
88
tox.ini
Normal file
88
tox.ini
Normal file
|
@ -0,0 +1,88 @@
|
|||
# Tox (https://tox.testrun.org/) is a tool for running tests
|
||||
# in multiple virtualenvs. This configuration file will run the
|
||||
# test suite on all supported python versions. To use it, "pip install tox"
|
||||
# and then run "tox" from this directory.
|
||||
|
||||
[tox]
|
||||
# deprecation warning: py{27,py2,34,35,36}
|
||||
envlist=py{27,34,35,36,37,38,39,310,py2,py3}{,-tf}{,-keras}, perf, setup.py
|
||||
isolated_build=True
|
||||
|
||||
[gh-actions]
|
||||
python=
|
||||
2.7: py27
|
||||
3.5: py35
|
||||
3.6: py36
|
||||
3.7: py37
|
||||
3.8: py38
|
||||
3.9: py39
|
||||
3.10: py310
|
||||
pypy-2.7: pypy2
|
||||
pypy-3.7: pypy3
|
||||
[gh-actions:env]
|
||||
PLATFORM=
|
||||
ubuntu: tf-keras
|
||||
|
||||
[core]
|
||||
deps=
|
||||
pytest
|
||||
py3{4,5,6}: pytest<7
|
||||
pytest-cov
|
||||
pytest-timeout
|
||||
py3{7,8,9,10}: pytest-asyncio
|
||||
py3{6,7,8,9,10}: ipywidgets
|
||||
py3{7,8,9,10}: git+https://github.com/casperdcl/nbval.git@master#egg=nbval
|
||||
coverage
|
||||
coveralls
|
||||
codecov
|
||||
commands=
|
||||
- coveralls
|
||||
codecov -X pycov -e TOXENV
|
||||
- codacy report -l Python -r coverage.xml --partial
|
||||
|
||||
[testenv]
|
||||
passenv=TOXENV CI GITHUB_* CODECOV_* COVERALLS_* CODACY_* HOME
|
||||
deps=
|
||||
{[core]deps}
|
||||
cython
|
||||
dask[delayed]
|
||||
matplotlib
|
||||
numpy
|
||||
pandas
|
||||
tf: tensorflow!=2.5.0
|
||||
!py27-keras: keras
|
||||
py27-keras: keras<2.5
|
||||
py35-keras: keras<2.7
|
||||
py27-tf: protobuf<3.18
|
||||
py3{6,7,8,9,10}: rich
|
||||
commands=
|
||||
py3{4,5,6}: pytest --cov=tqdm --cov-report=xml --cov-report=term -k "not perf" -o addopts= -v --tb=short -rxs -W=error --durations=0 --durations-min=0.1
|
||||
py3{7,8,9,10}: pytest --cov=tqdm --cov-report= tests_notebook.ipynb --nbval --nbval-current-env -W=ignore --nbval-sanitize-with=setup.cfg
|
||||
py3{7,8,9,10}: pytest --cov=tqdm --cov-report=xml --cov-report=term --cov-append -k "not perf"
|
||||
{[core]commands}
|
||||
allowlist_externals=codacy
|
||||
|
||||
[testenv:py{27,py2}{,-tf}{,-keras}]
|
||||
commands=
|
||||
pytest --cov=tqdm --cov-report=xml --cov-report=term -k "not perf" -o addopts= -v --tb=short -rxs -W=error --durations=10
|
||||
{[core]commands}
|
||||
|
||||
# no cython/numpy/pandas
|
||||
[testenv:py{34,py2,py3}]
|
||||
deps={[core]deps}
|
||||
|
||||
[testenv:perf]
|
||||
deps=
|
||||
pytest
|
||||
pytest-timeout
|
||||
pytest-asyncio
|
||||
commands=pytest -k perf
|
||||
|
||||
[testenv:setup.py]
|
||||
deps=
|
||||
docutils
|
||||
pygments
|
||||
py-make>=0.1.0
|
||||
commands=
|
||||
{envpython} setup.py check --restructuredtext --metadata --strict
|
||||
{envpython} setup.py make none
|
1581
tqdm.egg-info/PKG-INFO
Normal file
1581
tqdm.egg-info/PKG-INFO
Normal file
File diff suppressed because it is too large
Load diff
88
tqdm.egg-info/SOURCES.txt
Normal file
88
tqdm.egg-info/SOURCES.txt
Normal file
|
@ -0,0 +1,88 @@
|
|||
.pre-commit-config.yaml
|
||||
.zenodo.json
|
||||
CODE_OF_CONDUCT.md
|
||||
CONTRIBUTING.md
|
||||
DEMO.ipynb
|
||||
LICENCE
|
||||
Makefile
|
||||
README.rst
|
||||
environment.yml
|
||||
logo.png
|
||||
pyproject.toml
|
||||
setup.cfg
|
||||
setup.py
|
||||
tests_notebook.ipynb
|
||||
tox.ini
|
||||
examples/7zx.py
|
||||
examples/async_coroutines.py
|
||||
examples/coroutine_pipe.py
|
||||
examples/include_no_requirements.py
|
||||
examples/pandas_progress_apply.py
|
||||
examples/paper.bib
|
||||
examples/paper.md
|
||||
examples/parallel_bars.py
|
||||
examples/redirect_print.py
|
||||
examples/simple_examples.py
|
||||
examples/tqdm_requests.py
|
||||
examples/tqdm_wget.py
|
||||
examples/wrapping_generators.py
|
||||
tests/__init__.py
|
||||
tests/conftest.py
|
||||
tests/py37_asyncio.py
|
||||
tests/tests_asyncio.py
|
||||
tests/tests_concurrent.py
|
||||
tests/tests_contrib.py
|
||||
tests/tests_contrib_logging.py
|
||||
tests/tests_dask.py
|
||||
tests/tests_gui.py
|
||||
tests/tests_itertools.py
|
||||
tests/tests_keras.py
|
||||
tests/tests_main.py
|
||||
tests/tests_notebook.py
|
||||
tests/tests_pandas.py
|
||||
tests/tests_perf.py
|
||||
tests/tests_rich.py
|
||||
tests/tests_synchronisation.py
|
||||
tests/tests_tk.py
|
||||
tests/tests_tqdm.py
|
||||
tests/tests_version.py
|
||||
tqdm/__init__.py
|
||||
tqdm/__main__.py
|
||||
tqdm/_dist_ver.py
|
||||
tqdm/_main.py
|
||||
tqdm/_monitor.py
|
||||
tqdm/_tqdm.py
|
||||
tqdm/_tqdm_gui.py
|
||||
tqdm/_tqdm_notebook.py
|
||||
tqdm/_tqdm_pandas.py
|
||||
tqdm/_utils.py
|
||||
tqdm/asyncio.py
|
||||
tqdm/auto.py
|
||||
tqdm/autonotebook.py
|
||||
tqdm/cli.py
|
||||
tqdm/completion.sh
|
||||
tqdm/dask.py
|
||||
tqdm/gui.py
|
||||
tqdm/keras.py
|
||||
tqdm/notebook.py
|
||||
tqdm/rich.py
|
||||
tqdm/std.py
|
||||
tqdm/tk.py
|
||||
tqdm/tqdm.1
|
||||
tqdm/utils.py
|
||||
tqdm/version.py
|
||||
tqdm.egg-info/PKG-INFO
|
||||
tqdm.egg-info/SOURCES.txt
|
||||
tqdm.egg-info/dependency_links.txt
|
||||
tqdm.egg-info/entry_points.txt
|
||||
tqdm.egg-info/requires.txt
|
||||
tqdm.egg-info/top_level.txt
|
||||
tqdm/contrib/__init__.py
|
||||
tqdm/contrib/bells.py
|
||||
tqdm/contrib/concurrent.py
|
||||
tqdm/contrib/discord.py
|
||||
tqdm/contrib/itertools.py
|
||||
tqdm/contrib/logging.py
|
||||
tqdm/contrib/slack.py
|
||||
tqdm/contrib/telegram.py
|
||||
tqdm/contrib/utils_worker.py
|
1
tqdm.egg-info/dependency_links.txt
Normal file
1
tqdm.egg-info/dependency_links.txt
Normal file
|
@ -0,0 +1 @@
|
|||
|
2
tqdm.egg-info/entry_points.txt
Normal file
2
tqdm.egg-info/entry_points.txt
Normal file
|
@ -0,0 +1,2 @@
|
|||
[console_scripts]
|
||||
tqdm = tqdm.cli:main
|
20
tqdm.egg-info/requires.txt
Normal file
20
tqdm.egg-info/requires.txt
Normal file
|
@ -0,0 +1,20 @@
|
|||
|
||||
[:platform_system == "Windows"]
|
||||
colorama
|
||||
|
||||
[:python_version < "3.7"]
|
||||
importlib_resources
|
||||
|
||||
[dev]
|
||||
py-make>=0.1.0
|
||||
twine
|
||||
wheel
|
||||
|
||||
[notebook]
|
||||
ipywidgets>=6
|
||||
|
||||
[slack]
|
||||
slack-sdk
|
||||
|
||||
[telegram]
|
||||
requests
|
1
tqdm.egg-info/top_level.txt
Normal file
1
tqdm.egg-info/top_level.txt
Normal file
|
@ -0,0 +1 @@
|
|||
tqdm
|
41
tqdm/__init__.py
Normal file
41
tqdm/__init__.py
Normal file
|
@ -0,0 +1,41 @@
|
|||
from ._monitor import TMonitor, TqdmSynchronisationWarning
|
||||
from ._tqdm_pandas import tqdm_pandas
|
||||
from .cli import main # TODO: remove in v5.0.0
|
||||
from .gui import tqdm as tqdm_gui # TODO: remove in v5.0.0
|
||||
from .gui import trange as tgrange # TODO: remove in v5.0.0
|
||||
from .std import (
|
||||
TqdmDeprecationWarning, TqdmExperimentalWarning, TqdmKeyError, TqdmMonitorWarning,
|
||||
TqdmTypeError, TqdmWarning, tqdm, trange)
|
||||
from .version import __version__
|
||||
|
||||
__all__ = ['tqdm', 'tqdm_gui', 'trange', 'tgrange', 'tqdm_pandas',
|
||||
'tqdm_notebook', 'tnrange', 'main', 'TMonitor',
|
||||
'TqdmTypeError', 'TqdmKeyError',
|
||||
'TqdmWarning', 'TqdmDeprecationWarning',
|
||||
'TqdmExperimentalWarning',
|
||||
'TqdmMonitorWarning', 'TqdmSynchronisationWarning',
|
||||
'__version__']
|
||||
|
||||
|
||||
def tqdm_notebook(*args, **kwargs): # pragma: no cover
|
||||
"""See tqdm.notebook.tqdm for full documentation"""
|
||||
from warnings import warn
|
||||
|
||||
from .notebook import tqdm as _tqdm_notebook
|
||||
warn("This function will be removed in tqdm==5.0.0\n"
|
||||
"Please use `tqdm.notebook.tqdm` instead of `tqdm.tqdm_notebook`",
|
||||
TqdmDeprecationWarning, stacklevel=2)
|
||||
return _tqdm_notebook(*args, **kwargs)
|
||||
|
||||
|
||||
def tnrange(*args, **kwargs): # pragma: no cover
|
||||
"""
|
||||
A shortcut for `tqdm.notebook.tqdm(xrange(*args), **kwargs)`.
|
||||
On Python3+, `range` is used instead of `xrange`.
|
||||
"""
|
||||
from warnings import warn
|
||||
|
||||
from .notebook import trange as _tnrange
|
||||
warn("Please use `tqdm.notebook.trange` instead of `tqdm.tnrange`",
|
||||
TqdmDeprecationWarning, stacklevel=2)
|
||||
return _tnrange(*args, **kwargs)
|
3
tqdm/__main__.py
Normal file
3
tqdm/__main__.py
Normal file
|
@ -0,0 +1,3 @@
|
|||
from .cli import main
|
||||
|
||||
main()
|
1
tqdm/_dist_ver.py
Normal file
1
tqdm/_dist_ver.py
Normal file
|
@ -0,0 +1 @@
|
|||
__version__ = '4.64.1'
|
9
tqdm/_main.py
Normal file
9
tqdm/_main.py
Normal file
|
@ -0,0 +1,9 @@
|
|||
from warnings import warn
|
||||
|
||||
from .cli import * # NOQA
|
||||
from .cli import __all__ # NOQA
|
||||
from .std import TqdmDeprecationWarning
|
||||
|
||||
warn("This function will be removed in tqdm==5.0.0\n"
|
||||
"Please use `tqdm.cli.*` instead of `tqdm._main.*`",
|
||||
TqdmDeprecationWarning, stacklevel=2)
|
95
tqdm/_monitor.py
Normal file
95
tqdm/_monitor.py
Normal file
|
@ -0,0 +1,95 @@
|
|||
import atexit
|
||||
from threading import Event, Thread, current_thread
|
||||
from time import time
|
||||
from warnings import warn
|
||||
|
||||
__all__ = ["TMonitor", "TqdmSynchronisationWarning"]
|
||||
|
||||
|
||||
class TqdmSynchronisationWarning(RuntimeWarning):
|
||||
"""tqdm multi-thread/-process errors which may cause incorrect nesting
|
||||
but otherwise no adverse effects"""
|
||||
pass
|
||||
|
||||
|
||||
class TMonitor(Thread):
|
||||
"""
|
||||
Monitoring thread for tqdm bars.
|
||||
Monitors if tqdm bars are taking too much time to display
|
||||
and readjusts miniters automatically if necessary.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
tqdm_cls : class
|
||||
tqdm class to use (can be core tqdm or a submodule).
|
||||
sleep_interval : float
|
||||
Time to sleep between monitoring checks.
|
||||
"""
|
||||
_test = {} # internal vars for unit testing
|
||||
|
||||
def __init__(self, tqdm_cls, sleep_interval):
|
||||
Thread.__init__(self)
|
||||
self.daemon = True # kill thread when main killed (KeyboardInterrupt)
|
||||
self.woken = 0 # last time woken up, to sync with monitor
|
||||
self.tqdm_cls = tqdm_cls
|
||||
self.sleep_interval = sleep_interval
|
||||
self._time = self._test.get("time", time)
|
||||
self.was_killed = self._test.get("Event", Event)()
|
||||
atexit.register(self.exit)
|
||||
self.start()
|
||||
|
||||
def exit(self):
|
||||
self.was_killed.set()
|
||||
if self is not current_thread():
|
||||
self.join()
|
||||
return self.report()
|
||||
|
||||
def get_instances(self):
|
||||
# returns a copy of started `tqdm_cls` instances
|
||||
return [i for i in self.tqdm_cls._instances.copy()
|
||||
# Avoid race by checking that the instance started
|
||||
if hasattr(i, 'start_t')]
|
||||
|
||||
def run(self):
|
||||
cur_t = self._time()
|
||||
while True:
|
||||
# After processing and before sleeping, notify that we woke
|
||||
# Need to be done just before sleeping
|
||||
self.woken = cur_t
|
||||
# Sleep some time...
|
||||
self.was_killed.wait(self.sleep_interval)
|
||||
# Quit if killed
|
||||
if self.was_killed.is_set():
|
||||
return
|
||||
# Then monitor!
|
||||
# Acquire lock (to access _instances)
|
||||
with self.tqdm_cls.get_lock():
|
||||
cur_t = self._time()
|
||||
# Check tqdm instances are waiting too long to print
|
||||
instances = self.get_instances()
|
||||
for instance in instances:
|
||||
# Check event in loop to reduce blocking time on exit
|
||||
if self.was_killed.is_set():
|
||||
return
|
||||
# Only if mininterval > 1 (else iterations are just slow)
|
||||
# and last refresh exceeded maxinterval
|
||||
if (
|
||||
instance.miniters > 1
|
||||
and (cur_t - instance.last_print_t) >= instance.maxinterval
|
||||
):
|
||||
# force bypassing miniters on next iteration
|
||||
# (dynamic_miniters adjusts mininterval automatically)
|
||||
instance.miniters = 1
|
||||
# Refresh now! (works only for manual tqdm)
|
||||
instance.refresh(nolock=True)
|
||||
# Remove accidental long-lived strong reference
|
||||
del instance
|
||||
if instances != self.get_instances(): # pragma: nocover
|
||||
warn("Set changed size during iteration" +
|
||||
" (see https://github.com/tqdm/tqdm/issues/481)",
|
||||
TqdmSynchronisationWarning, stacklevel=2)
|
||||
# Remove accidental long-lived strong references
|
||||
del instances
|
||||
|
||||
def report(self):
|
||||
return not self.was_killed.is_set()
|
9
tqdm/_tqdm.py
Normal file
9
tqdm/_tqdm.py
Normal file
|
@ -0,0 +1,9 @@
|
|||
from warnings import warn
|
||||
|
||||
from .std import * # NOQA
|
||||
from .std import __all__ # NOQA
|
||||
from .std import TqdmDeprecationWarning
|
||||
|
||||
warn("This function will be removed in tqdm==5.0.0\n"
|
||||
"Please use `tqdm.std.*` instead of `tqdm._tqdm.*`",
|
||||
TqdmDeprecationWarning, stacklevel=2)
|
9
tqdm/_tqdm_gui.py
Normal file
9
tqdm/_tqdm_gui.py
Normal file
|
@ -0,0 +1,9 @@
|
|||
from warnings import warn
|
||||
|
||||
from .gui import * # NOQA
|
||||
from .gui import __all__ # NOQA
|
||||
from .std import TqdmDeprecationWarning
|
||||
|
||||
warn("This function will be removed in tqdm==5.0.0\n"
|
||||
"Please use `tqdm.gui.*` instead of `tqdm._tqdm_gui.*`",
|
||||
TqdmDeprecationWarning, stacklevel=2)
|
9
tqdm/_tqdm_notebook.py
Normal file
9
tqdm/_tqdm_notebook.py
Normal file
|
@ -0,0 +1,9 @@
|
|||
from warnings import warn
|
||||
|
||||
from .notebook import * # NOQA
|
||||
from .notebook import __all__ # NOQA
|
||||
from .std import TqdmDeprecationWarning
|
||||
|
||||
warn("This function will be removed in tqdm==5.0.0\n"
|
||||
"Please use `tqdm.notebook.*` instead of `tqdm._tqdm_notebook.*`",
|
||||
TqdmDeprecationWarning, stacklevel=2)
|
24
tqdm/_tqdm_pandas.py
Normal file
24
tqdm/_tqdm_pandas.py
Normal file
|
@ -0,0 +1,24 @@
|
|||
import sys
|
||||
|
||||
__author__ = "github.com/casperdcl"
|
||||
__all__ = ['tqdm_pandas']
|
||||
|
||||
|
||||
def tqdm_pandas(tclass, **tqdm_kwargs):
|
||||
"""
|
||||
Registers the given `tqdm` instance with
|
||||
`pandas.core.groupby.DataFrameGroupBy.progress_apply`.
|
||||
"""
|
||||
from tqdm import TqdmDeprecationWarning
|
||||
|
||||
if isinstance(tclass, type) or (getattr(tclass, '__name__', '').startswith(
|
||||
'tqdm_')): # delayed adapter case
|
||||
TqdmDeprecationWarning(
|
||||
"Please use `tqdm.pandas(...)` instead of `tqdm_pandas(tqdm, ...)`.",
|
||||
fp_write=getattr(tqdm_kwargs.get('file', None), 'write', sys.stderr.write))
|
||||
tclass.pandas(**tqdm_kwargs)
|
||||
else:
|
||||
TqdmDeprecationWarning(
|
||||
"Please use `tqdm.pandas(...)` instead of `tqdm_pandas(tqdm(...))`.",
|
||||
fp_write=getattr(tclass.fp, 'write', sys.stderr.write))
|
||||
type(tclass).pandas(deprecated_t=tclass)
|
12
tqdm/_utils.py
Normal file
12
tqdm/_utils.py
Normal file
|
@ -0,0 +1,12 @@
|
|||
from warnings import warn
|
||||
|
||||
from .std import TqdmDeprecationWarning
|
||||
from .utils import ( # NOQA, pylint: disable=unused-import
|
||||
CUR_OS, IS_NIX, IS_WIN, RE_ANSI, Comparable, FormatReplace, SimpleTextIOWrapper, _basestring,
|
||||
_environ_cols_wrapper, _is_ascii, _is_utf, _range, _screen_shape_linux, _screen_shape_tput,
|
||||
_screen_shape_windows, _screen_shape_wrapper, _supports_unicode, _term_move_up, _unich,
|
||||
_unicode, colorama)
|
||||
|
||||
warn("This function will be removed in tqdm==5.0.0\n"
|
||||
"Please use `tqdm.utils.*` instead of `tqdm._utils.*`",
|
||||
TqdmDeprecationWarning, stacklevel=2)
|
93
tqdm/asyncio.py
Normal file
93
tqdm/asyncio.py
Normal file
|
@ -0,0 +1,93 @@
|
|||
"""
|
||||
Asynchronous progressbar decorator for iterators.
|
||||
Includes a default `range` iterator printing to `stderr`.
|
||||
|
||||
Usage:
|
||||
>>> from tqdm.asyncio import trange, tqdm
|
||||
>>> async for i in trange(10):
|
||||
... ...
|
||||
"""
|
||||
import asyncio
|
||||
from sys import version_info
|
||||
|
||||
from .std import tqdm as std_tqdm
|
||||
|
||||
__author__ = {"github.com/": ["casperdcl"]}
|
||||
__all__ = ['tqdm_asyncio', 'tarange', 'tqdm', 'trange']
|
||||
|
||||
|
||||
class tqdm_asyncio(std_tqdm):
|
||||
"""
|
||||
Asynchronous-friendly version of tqdm (Python 3.6+).
|
||||
"""
|
||||
def __init__(self, iterable=None, *args, **kwargs):
|
||||
super(tqdm_asyncio, self).__init__(iterable, *args, **kwargs)
|
||||
self.iterable_awaitable = False
|
||||
if iterable is not None:
|
||||
if hasattr(iterable, "__anext__"):
|
||||
self.iterable_next = iterable.__anext__
|
||||
self.iterable_awaitable = True
|
||||
elif hasattr(iterable, "__next__"):
|
||||
self.iterable_next = iterable.__next__
|
||||
else:
|
||||
self.iterable_iterator = iter(iterable)
|
||||
self.iterable_next = self.iterable_iterator.__next__
|
||||
|
||||
def __aiter__(self):
|
||||
return self
|
||||
|
||||
async def __anext__(self):
|
||||
try:
|
||||
if self.iterable_awaitable:
|
||||
res = await self.iterable_next()
|
||||
else:
|
||||
res = self.iterable_next()
|
||||
self.update()
|
||||
return res
|
||||
except StopIteration:
|
||||
self.close()
|
||||
raise StopAsyncIteration
|
||||
except BaseException:
|
||||
self.close()
|
||||
raise
|
||||
|
||||
def send(self, *args, **kwargs):
|
||||
return self.iterable.send(*args, **kwargs)
|
||||
|
||||
@classmethod
|
||||
def as_completed(cls, fs, *, loop=None, timeout=None, total=None, **tqdm_kwargs):
|
||||
"""
|
||||
Wrapper for `asyncio.as_completed`.
|
||||
"""
|
||||
if total is None:
|
||||
total = len(fs)
|
||||
kwargs = {}
|
||||
if version_info[:2] < (3, 10):
|
||||
kwargs['loop'] = loop
|
||||
yield from cls(asyncio.as_completed(fs, timeout=timeout, **kwargs),
|
||||
total=total, **tqdm_kwargs)
|
||||
|
||||
@classmethod
|
||||
async def gather(cls, *fs, loop=None, timeout=None, total=None, **tqdm_kwargs):
|
||||
"""
|
||||
Wrapper for `asyncio.gather`.
|
||||
"""
|
||||
async def wrap_awaitable(i, f):
|
||||
return i, await f
|
||||
|
||||
ifs = [wrap_awaitable(i, f) for i, f in enumerate(fs)]
|
||||
res = [await f for f in cls.as_completed(ifs, loop=loop, timeout=timeout,
|
||||
total=total, **tqdm_kwargs)]
|
||||
return [i for _, i in sorted(res)]
|
||||
|
||||
|
||||
def tarange(*args, **kwargs):
|
||||
"""
|
||||
A shortcut for `tqdm.asyncio.tqdm(range(*args), **kwargs)`.
|
||||
"""
|
||||
return tqdm_asyncio(range(*args), **kwargs)
|
||||
|
||||
|
||||
# Aliases
|
||||
tqdm = tqdm_asyncio
|
||||
trange = tarange
|
44
tqdm/auto.py
Normal file
44
tqdm/auto.py
Normal file
|
@ -0,0 +1,44 @@
|
|||
"""
|
||||
Enables multiple commonly used features.
|
||||
|
||||
Method resolution order:
|
||||
|
||||
- `tqdm.autonotebook` without import warnings
|
||||
- `tqdm.asyncio` on Python3.6+
|
||||
- `tqdm.std` base class
|
||||
|
||||
Usage:
|
||||
>>> from tqdm.auto import trange, tqdm
|
||||
>>> for i in trange(10):
|
||||
... ...
|
||||
"""
|
||||
import sys
|
||||
import warnings
|
||||
|
||||
from .std import TqdmExperimentalWarning
|
||||
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("ignore", category=TqdmExperimentalWarning)
|
||||
from .autonotebook import tqdm as notebook_tqdm
|
||||
from .autonotebook import trange as notebook_trange
|
||||
|
||||
if sys.version_info[:2] < (3, 6):
|
||||
tqdm = notebook_tqdm
|
||||
trange = notebook_trange
|
||||
else: # Python3.6+
|
||||
from .asyncio import tqdm as asyncio_tqdm
|
||||
from .std import tqdm as std_tqdm
|
||||
|
||||
if notebook_tqdm != std_tqdm:
|
||||
class tqdm(notebook_tqdm, asyncio_tqdm): # pylint: disable=inconsistent-mro
|
||||
pass
|
||||
else:
|
||||
tqdm = asyncio_tqdm
|
||||
|
||||
def trange(*args, **kwargs):
|
||||
"""
|
||||
A shortcut for `tqdm.auto.tqdm(range(*args), **kwargs)`.
|
||||
"""
|
||||
return tqdm(range(*args), **kwargs)
|
||||
|
||||
__all__ = ["tqdm", "trange"]
|
29
tqdm/autonotebook.py
Normal file
29
tqdm/autonotebook.py
Normal file
|
@ -0,0 +1,29 @@
|
|||
"""
|
||||
Automatically choose between `tqdm.notebook` and `tqdm.std`.
|
||||
|
||||
Usage:
|
||||
>>> from tqdm.autonotebook import trange, tqdm
|
||||
>>> for i in trange(10):
|
||||
... ...
|
||||
"""
|
||||
import sys
|
||||
from warnings import warn
|
||||
|
||||
try:
|
||||
get_ipython = sys.modules['IPython'].get_ipython
|
||||
if 'IPKernelApp' not in get_ipython().config: # pragma: no cover
|
||||
raise ImportError("console")
|
||||
from .notebook import WARN_NOIPYW, IProgress
|
||||
if IProgress is None:
|
||||
from .std import TqdmWarning
|
||||
warn(WARN_NOIPYW, TqdmWarning, stacklevel=2)
|
||||
raise ImportError('ipywidgets')
|
||||
except Exception:
|
||||
from .std import tqdm, trange
|
||||
else: # pragma: no cover
|
||||
from .notebook import tqdm, trange
|
||||
from .std import TqdmExperimentalWarning
|
||||
warn("Using `tqdm.autonotebook.tqdm` in notebook mode."
|
||||
" Use `tqdm.tqdm` instead to force console mode"
|
||||
" (e.g. in jupyter console)", TqdmExperimentalWarning, stacklevel=2)
|
||||
__all__ = ["tqdm", "trange"]
|
315
tqdm/cli.py
Normal file
315
tqdm/cli.py
Normal file
|
@ -0,0 +1,315 @@
|
|||
"""
|
||||
Module version for monitoring CLI pipes (`... | python -m tqdm | ...`).
|
||||
"""
|
||||
import logging
|
||||
import re
|
||||
import sys
|
||||
from ast import literal_eval as numeric
|
||||
|
||||
from .std import TqdmKeyError, TqdmTypeError, tqdm
|
||||
from .version import __version__
|
||||
|
||||
__all__ = ["main"]
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def cast(val, typ):
|
||||
log.debug((val, typ))
|
||||
if " or " in typ:
|
||||
for t in typ.split(" or "):
|
||||
try:
|
||||
return cast(val, t)
|
||||
except TqdmTypeError:
|
||||
pass
|
||||
raise TqdmTypeError(val + ' : ' + typ)
|
||||
|
||||
# sys.stderr.write('\ndebug | `val:type`: `' + val + ':' + typ + '`.\n')
|
||||
if typ == 'bool':
|
||||
if (val == 'True') or (val == ''):
|
||||
return True
|
||||
elif val == 'False':
|
||||
return False
|
||||
else:
|
||||
raise TqdmTypeError(val + ' : ' + typ)
|
||||
try:
|
||||
return eval(typ + '("' + val + '")')
|
||||
except Exception:
|
||||
if typ == 'chr':
|
||||
return chr(ord(eval('"' + val + '"'))).encode()
|
||||
else:
|
||||
raise TqdmTypeError(val + ' : ' + typ)
|
||||
|
||||
|
||||
def posix_pipe(fin, fout, delim=b'\\n', buf_size=256,
|
||||
callback=lambda float: None, callback_len=True):
|
||||
"""
|
||||
Params
|
||||
------
|
||||
fin : binary file with `read(buf_size : int)` method
|
||||
fout : binary file with `write` (and optionally `flush`) methods.
|
||||
callback : function(float), e.g.: `tqdm.update`
|
||||
callback_len : If (default: True) do `callback(len(buffer))`.
|
||||
Otherwise, do `callback(data) for data in buffer.split(delim)`.
|
||||
"""
|
||||
fp_write = fout.write
|
||||
|
||||
if not delim:
|
||||
while True:
|
||||
tmp = fin.read(buf_size)
|
||||
|
||||
# flush at EOF
|
||||
if not tmp:
|
||||
getattr(fout, 'flush', lambda: None)()
|
||||
return
|
||||
|
||||
fp_write(tmp)
|
||||
callback(len(tmp))
|
||||
# return
|
||||
|
||||
buf = b''
|
||||
len_delim = len(delim)
|
||||
# n = 0
|
||||
while True:
|
||||
tmp = fin.read(buf_size)
|
||||
|
||||
# flush at EOF
|
||||
if not tmp:
|
||||
if buf:
|
||||
fp_write(buf)
|
||||
if callback_len:
|
||||
# n += 1 + buf.count(delim)
|
||||
callback(1 + buf.count(delim))
|
||||
else:
|
||||
for i in buf.split(delim):
|
||||
callback(i)
|
||||
getattr(fout, 'flush', lambda: None)()
|
||||
return # n
|
||||
|
||||
while True:
|
||||
i = tmp.find(delim)
|
||||
if i < 0:
|
||||
buf += tmp
|
||||
break
|
||||
fp_write(buf + tmp[:i + len(delim)])
|
||||
# n += 1
|
||||
callback(1 if callback_len else (buf + tmp[:i]))
|
||||
buf = b''
|
||||
tmp = tmp[i + len_delim:]
|
||||
|
||||
|
||||
# ((opt, type), ... )
|
||||
RE_OPTS = re.compile(r'\n {8}(\S+)\s{2,}:\s*([^,]+)')
|
||||
# better split method assuming no positional args
|
||||
RE_SHLEX = re.compile(r'\s*(?<!\S)--?([^\s=]+)(\s+|=|$)')
|
||||
|
||||
# TODO: add custom support for some of the following?
|
||||
UNSUPPORTED_OPTS = ('iterable', 'gui', 'out', 'file')
|
||||
|
||||
# The 8 leading spaces are required for consistency
|
||||
CLI_EXTRA_DOC = r"""
|
||||
Extra CLI Options
|
||||
-----------------
|
||||
name : type, optional
|
||||
TODO: find out why this is needed.
|
||||
delim : chr, optional
|
||||
Delimiting character [default: '\n']. Use '\0' for null.
|
||||
N.B.: on Windows systems, Python converts '\n' to '\r\n'.
|
||||
buf_size : int, optional
|
||||
String buffer size in bytes [default: 256]
|
||||
used when `delim` is specified.
|
||||
bytes : bool, optional
|
||||
If true, will count bytes, ignore `delim`, and default
|
||||
`unit_scale` to True, `unit_divisor` to 1024, and `unit` to 'B'.
|
||||
tee : bool, optional
|
||||
If true, passes `stdin` to both `stderr` and `stdout`.
|
||||
update : bool, optional
|
||||
If true, will treat input as newly elapsed iterations,
|
||||
i.e. numbers to pass to `update()`. Note that this is slow
|
||||
(~2e5 it/s) since every input must be decoded as a number.
|
||||
update_to : bool, optional
|
||||
If true, will treat input as total elapsed iterations,
|
||||
i.e. numbers to assign to `self.n`. Note that this is slow
|
||||
(~2e5 it/s) since every input must be decoded as a number.
|
||||
null : bool, optional
|
||||
If true, will discard input (no stdout).
|
||||
manpath : str, optional
|
||||
Directory in which to install tqdm man pages.
|
||||
comppath : str, optional
|
||||
Directory in which to place tqdm completion.
|
||||
log : str, optional
|
||||
CRITICAL|FATAL|ERROR|WARN(ING)|[default: 'INFO']|DEBUG|NOTSET.
|
||||
"""
|
||||
|
||||
|
||||
def main(fp=sys.stderr, argv=None):
|
||||
"""
|
||||
Parameters (internal use only)
|
||||
---------
|
||||
fp : file-like object for tqdm
|
||||
argv : list (default: sys.argv[1:])
|
||||
"""
|
||||
if argv is None:
|
||||
argv = sys.argv[1:]
|
||||
try:
|
||||
log_idx = argv.index('--log')
|
||||
except ValueError:
|
||||
for i in argv:
|
||||
if i.startswith('--log='):
|
||||
logLevel = i[len('--log='):]
|
||||
break
|
||||
else:
|
||||
logLevel = 'INFO'
|
||||
else:
|
||||
# argv.pop(log_idx)
|
||||
# logLevel = argv.pop(log_idx)
|
||||
logLevel = argv[log_idx + 1]
|
||||
logging.basicConfig(level=getattr(logging, logLevel),
|
||||
format="%(levelname)s:%(module)s:%(lineno)d:%(message)s")
|
||||
|
||||
d = tqdm.__init__.__doc__ + CLI_EXTRA_DOC
|
||||
|
||||
opt_types = dict(RE_OPTS.findall(d))
|
||||
# opt_types['delim'] = 'chr'
|
||||
|
||||
for o in UNSUPPORTED_OPTS:
|
||||
opt_types.pop(o)
|
||||
|
||||
log.debug(sorted(opt_types.items()))
|
||||
|
||||
# d = RE_OPTS.sub(r' --\1=<\1> : \2', d)
|
||||
split = RE_OPTS.split(d)
|
||||
opt_types_desc = zip(split[1::3], split[2::3], split[3::3])
|
||||
d = ''.join(('\n --{0} : {2}{3}' if otd[1] == 'bool' else
|
||||
'\n --{0}=<{1}> : {2}{3}').format(
|
||||
otd[0].replace('_', '-'), otd[0], *otd[1:])
|
||||
for otd in opt_types_desc if otd[0] not in UNSUPPORTED_OPTS)
|
||||
|
||||
help_short = "Usage:\n tqdm [--help | options]\n"
|
||||
d = help_short + """
|
||||
Options:
|
||||
-h, --help Print this help and exit.
|
||||
-v, --version Print version and exit.
|
||||
""" + d.strip('\n') + '\n'
|
||||
|
||||
# opts = docopt(d, version=__version__)
|
||||
if any(v in argv for v in ('-v', '--version')):
|
||||
sys.stdout.write(__version__ + '\n')
|
||||
sys.exit(0)
|
||||
elif any(v in argv for v in ('-h', '--help')):
|
||||
sys.stdout.write(d + '\n')
|
||||
sys.exit(0)
|
||||
elif argv and argv[0][:2] != '--':
|
||||
sys.stderr.write(
|
||||
"Error:Unknown argument:{0}\n{1}".format(argv[0], help_short))
|
||||
|
||||
argv = RE_SHLEX.split(' '.join(["tqdm"] + argv))
|
||||
opts = dict(zip(argv[1::3], argv[3::3]))
|
||||
|
||||
log.debug(opts)
|
||||
opts.pop('log', True)
|
||||
|
||||
tqdm_args = {'file': fp}
|
||||
try:
|
||||
for (o, v) in opts.items():
|
||||
o = o.replace('-', '_')
|
||||
try:
|
||||
tqdm_args[o] = cast(v, opt_types[o])
|
||||
except KeyError as e:
|
||||
raise TqdmKeyError(str(e))
|
||||
log.debug('args:' + str(tqdm_args))
|
||||
|
||||
delim_per_char = tqdm_args.pop('bytes', False)
|
||||
update = tqdm_args.pop('update', False)
|
||||
update_to = tqdm_args.pop('update_to', False)
|
||||
if sum((delim_per_char, update, update_to)) > 1:
|
||||
raise TqdmKeyError("Can only have one of --bytes --update --update_to")
|
||||
except Exception:
|
||||
fp.write("\nError:\n" + help_short)
|
||||
stdin, stdout_write = sys.stdin, sys.stdout.write
|
||||
for i in stdin:
|
||||
stdout_write(i)
|
||||
raise
|
||||
else:
|
||||
buf_size = tqdm_args.pop('buf_size', 256)
|
||||
delim = tqdm_args.pop('delim', b'\\n')
|
||||
tee = tqdm_args.pop('tee', False)
|
||||
manpath = tqdm_args.pop('manpath', None)
|
||||
comppath = tqdm_args.pop('comppath', None)
|
||||
if tqdm_args.pop('null', False):
|
||||
class stdout(object):
|
||||
@staticmethod
|
||||
def write(_):
|
||||
pass
|
||||
else:
|
||||
stdout = sys.stdout
|
||||
stdout = getattr(stdout, 'buffer', stdout)
|
||||
stdin = getattr(sys.stdin, 'buffer', sys.stdin)
|
||||
if manpath or comppath:
|
||||
from os import path
|
||||
from shutil import copyfile
|
||||
try: # py<3.7
|
||||
import importlib_resources as resources
|
||||
except ImportError:
|
||||
from importlib import resources
|
||||
|
||||
def cp(name, dst):
|
||||
"""copy resource `name` to `dst`"""
|
||||
if hasattr(resources, 'files'):
|
||||
copyfile(str(resources.files('tqdm') / name), dst)
|
||||
else: # py<3.9
|
||||
with resources.path('tqdm', name) as src:
|
||||
copyfile(str(src), dst)
|
||||
log.info("written:%s", dst)
|
||||
if manpath is not None:
|
||||
cp('tqdm.1', path.join(manpath, 'tqdm.1'))
|
||||
if comppath is not None:
|
||||
cp('completion.sh', path.join(comppath, 'tqdm_completion.sh'))
|
||||
sys.exit(0)
|
||||
if tee:
|
||||
stdout_write = stdout.write
|
||||
fp_write = getattr(fp, 'buffer', fp).write
|
||||
|
||||
class stdout(object): # pylint: disable=function-redefined
|
||||
@staticmethod
|
||||
def write(x):
|
||||
with tqdm.external_write_mode(file=fp):
|
||||
fp_write(x)
|
||||
stdout_write(x)
|
||||
if delim_per_char:
|
||||
tqdm_args.setdefault('unit', 'B')
|
||||
tqdm_args.setdefault('unit_scale', True)
|
||||
tqdm_args.setdefault('unit_divisor', 1024)
|
||||
log.debug(tqdm_args)
|
||||
with tqdm(**tqdm_args) as t:
|
||||
posix_pipe(stdin, stdout, '', buf_size, t.update)
|
||||
elif delim == b'\\n':
|
||||
log.debug(tqdm_args)
|
||||
write = stdout.write
|
||||
if update or update_to:
|
||||
with tqdm(**tqdm_args) as t:
|
||||
if update:
|
||||
def callback(i):
|
||||
t.update(numeric(i.decode()))
|
||||
else: # update_to
|
||||
def callback(i):
|
||||
t.update(numeric(i.decode()) - t.n)
|
||||
for i in stdin:
|
||||
write(i)
|
||||
callback(i)
|
||||
else:
|
||||
for i in tqdm(stdin, **tqdm_args):
|
||||
write(i)
|
||||
else:
|
||||
log.debug(tqdm_args)
|
||||
with tqdm(**tqdm_args) as t:
|
||||
callback_len = False
|
||||
if update:
|
||||
def callback(i):
|
||||
t.update(numeric(i.decode()))
|
||||
elif update_to:
|
||||
def callback(i):
|
||||
t.update(numeric(i.decode()) - t.n)
|
||||
else:
|
||||
callback = t.update
|
||||
callback_len = True
|
||||
posix_pipe(stdin, stdout, delim, buf_size, callback, callback_len)
|
19
tqdm/completion.sh
Executable file
19
tqdm/completion.sh
Executable file
|
@ -0,0 +1,19 @@
|
|||
#!/usr/bin/env bash
|
||||
_tqdm(){
|
||||
local cur prv
|
||||
cur="${COMP_WORDS[COMP_CWORD]}"
|
||||
prv="${COMP_WORDS[COMP_CWORD - 1]}"
|
||||
|
||||
case ${prv} in
|
||||
--bar_format|--buf_size|--colour|--comppath|--delay|--delim|--desc|--initial|--lock_args|--manpath|--maxinterval|--mininterval|--miniters|--ncols|--nrows|--position|--postfix|--smoothing|--total|--unit|--unit_divisor)
|
||||
# await user input
|
||||
;;
|
||||
"--log")
|
||||
COMPREPLY=($(compgen -W 'CRITICAL FATAL ERROR WARN WARNING INFO DEBUG NOTSET' -- ${cur}))
|
||||
;;
|
||||
*)
|
||||
COMPREPLY=($(compgen -W '--ascii --bar_format --buf_size --bytes --colour --comppath --delay --delim --desc --disable --dynamic_ncols --help --initial --leave --lock_args --log --manpath --maxinterval --mininterval --miniters --ncols --nrows --null --position --postfix --smoothing --tee --total --unit --unit_divisor --unit_scale --update --update_to --version --write_bytes -h -v' -- ${cur}))
|
||||
;;
|
||||
esac
|
||||
}
|
||||
complete -F _tqdm tqdm
|
98
tqdm/contrib/__init__.py
Normal file
98
tqdm/contrib/__init__.py
Normal file
|
@ -0,0 +1,98 @@
|
|||
"""
|
||||
Thin wrappers around common functions.
|
||||
|
||||
Subpackages contain potentially unstable extensions.
|
||||
"""
|
||||
import sys
|
||||
from functools import wraps
|
||||
|
||||
from ..auto import tqdm as tqdm_auto
|
||||
from ..std import tqdm
|
||||
from ..utils import ObjectWrapper
|
||||
|
||||
__author__ = {"github.com/": ["casperdcl"]}
|
||||
__all__ = ['tenumerate', 'tzip', 'tmap']
|
||||
|
||||
|
||||
class DummyTqdmFile(ObjectWrapper):
|
||||
"""Dummy file-like that will write to tqdm"""
|
||||
|
||||
def __init__(self, wrapped):
|
||||
super(DummyTqdmFile, self).__init__(wrapped)
|
||||
self._buf = []
|
||||
|
||||
def write(self, x, nolock=False):
|
||||
nl = b"\n" if isinstance(x, bytes) else "\n"
|
||||
pre, sep, post = x.rpartition(nl)
|
||||
if sep:
|
||||
blank = type(nl)()
|
||||
tqdm.write(blank.join(self._buf + [pre, sep]),
|
||||
end=blank, file=self._wrapped, nolock=nolock)
|
||||
self._buf = [post]
|
||||
else:
|
||||
self._buf.append(x)
|
||||
|
||||
def __del__(self):
|
||||
if self._buf:
|
||||
blank = type(self._buf[0])()
|
||||
try:
|
||||
tqdm.write(blank.join(self._buf), end=blank, file=self._wrapped)
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
|
||||
|
||||
def builtin_iterable(func):
|
||||
"""Wraps `func()` output in a `list()` in py2"""
|
||||
if sys.version_info[:1] < (3,):
|
||||
@wraps(func)
|
||||
def inner(*args, **kwargs):
|
||||
return list(func(*args, **kwargs))
|
||||
return inner
|
||||
return func
|
||||
|
||||
|
||||
def tenumerate(iterable, start=0, total=None, tqdm_class=tqdm_auto, **tqdm_kwargs):
|
||||
"""
|
||||
Equivalent of `numpy.ndenumerate` or builtin `enumerate`.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
tqdm_class : [default: tqdm.auto.tqdm].
|
||||
"""
|
||||
try:
|
||||
import numpy as np
|
||||
except ImportError:
|
||||
pass
|
||||
else:
|
||||
if isinstance(iterable, np.ndarray):
|
||||
return tqdm_class(np.ndenumerate(iterable), total=total or iterable.size,
|
||||
**tqdm_kwargs)
|
||||
return enumerate(tqdm_class(iterable, total=total, **tqdm_kwargs), start)
|
||||
|
||||
|
||||
@builtin_iterable
|
||||
def tzip(iter1, *iter2plus, **tqdm_kwargs):
|
||||
"""
|
||||
Equivalent of builtin `zip`.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
tqdm_class : [default: tqdm.auto.tqdm].
|
||||
"""
|
||||
kwargs = tqdm_kwargs.copy()
|
||||
tqdm_class = kwargs.pop("tqdm_class", tqdm_auto)
|
||||
for i in zip(tqdm_class(iter1, **kwargs), *iter2plus):
|
||||
yield i
|
||||
|
||||
|
||||
@builtin_iterable
|
||||
def tmap(function, *sequences, **tqdm_kwargs):
|
||||
"""
|
||||
Equivalent of builtin `map`.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
tqdm_class : [default: tqdm.auto.tqdm].
|
||||
"""
|
||||
for i in tzip(*sequences, **tqdm_kwargs):
|
||||
yield function(*i)
|
26
tqdm/contrib/bells.py
Normal file
26
tqdm/contrib/bells.py
Normal file
|
@ -0,0 +1,26 @@
|
|||
"""
|
||||
Even more features than `tqdm.auto` (all the bells & whistles):
|
||||
|
||||
- `tqdm.auto`
|
||||
- `tqdm.tqdm.pandas`
|
||||
- `tqdm.contrib.telegram`
|
||||
+ uses `${TQDM_TELEGRAM_TOKEN}` and `${TQDM_TELEGRAM_CHAT_ID}`
|
||||
- `tqdm.contrib.discord`
|
||||
+ uses `${TQDM_DISCORD_TOKEN}` and `${TQDM_DISCORD_CHANNEL_ID}`
|
||||
"""
|
||||
__all__ = ['tqdm', 'trange']
|
||||
import warnings
|
||||
from os import getenv
|
||||
|
||||
if getenv("TQDM_SLACK_TOKEN") and getenv("TQDM_SLACK_CHANNEL"):
|
||||
from .slack import tqdm, trange
|
||||
elif getenv("TQDM_TELEGRAM_TOKEN") and getenv("TQDM_TELEGRAM_CHAT_ID"):
|
||||
from .telegram import tqdm, trange
|
||||
elif getenv("TQDM_DISCORD_TOKEN") and getenv("TQDM_DISCORD_CHANNEL_ID"):
|
||||
from .discord import tqdm, trange
|
||||
else:
|
||||
from ..auto import tqdm, trange
|
||||
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("ignore", category=FutureWarning)
|
||||
tqdm.pandas()
|
130
tqdm/contrib/concurrent.py
Normal file
130
tqdm/contrib/concurrent.py
Normal file
|
@ -0,0 +1,130 @@
|
|||
"""
|
||||
Thin wrappers around `concurrent.futures`.
|
||||
"""
|
||||
from __future__ import absolute_import
|
||||
|
||||
from contextlib import contextmanager
|
||||
|
||||
from ..auto import tqdm as tqdm_auto
|
||||
from ..std import TqdmWarning
|
||||
|
||||
try:
|
||||
from operator import length_hint
|
||||
except ImportError:
|
||||
def length_hint(it, default=0):
|
||||
"""Returns `len(it)`, falling back to `default`"""
|
||||
try:
|
||||
return len(it)
|
||||
except TypeError:
|
||||
return default
|
||||
try:
|
||||
from os import cpu_count
|
||||
except ImportError:
|
||||
try:
|
||||
from multiprocessing import cpu_count
|
||||
except ImportError:
|
||||
def cpu_count():
|
||||
return 4
|
||||
import sys
|
||||
|
||||
__author__ = {"github.com/": ["casperdcl"]}
|
||||
__all__ = ['thread_map', 'process_map']
|
||||
|
||||
|
||||
@contextmanager
|
||||
def ensure_lock(tqdm_class, lock_name=""):
|
||||
"""get (create if necessary) and then restore `tqdm_class`'s lock"""
|
||||
old_lock = getattr(tqdm_class, '_lock', None) # don't create a new lock
|
||||
lock = old_lock or tqdm_class.get_lock() # maybe create a new lock
|
||||
lock = getattr(lock, lock_name, lock) # maybe subtype
|
||||
tqdm_class.set_lock(lock)
|
||||
yield lock
|
||||
if old_lock is None:
|
||||
del tqdm_class._lock
|
||||
else:
|
||||
tqdm_class.set_lock(old_lock)
|
||||
|
||||
|
||||
def _executor_map(PoolExecutor, fn, *iterables, **tqdm_kwargs):
|
||||
"""
|
||||
Implementation of `thread_map` and `process_map`.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
tqdm_class : [default: tqdm.auto.tqdm].
|
||||
max_workers : [default: min(32, cpu_count() + 4)].
|
||||
chunksize : [default: 1].
|
||||
lock_name : [default: "":str].
|
||||
"""
|
||||
kwargs = tqdm_kwargs.copy()
|
||||
if "total" not in kwargs:
|
||||
kwargs["total"] = length_hint(iterables[0])
|
||||
tqdm_class = kwargs.pop("tqdm_class", tqdm_auto)
|
||||
max_workers = kwargs.pop("max_workers", min(32, cpu_count() + 4))
|
||||
chunksize = kwargs.pop("chunksize", 1)
|
||||
lock_name = kwargs.pop("lock_name", "")
|
||||
with ensure_lock(tqdm_class, lock_name=lock_name) as lk:
|
||||
pool_kwargs = {'max_workers': max_workers}
|
||||
sys_version = sys.version_info[:2]
|
||||
if sys_version >= (3, 7):
|
||||
# share lock in case workers are already using `tqdm`
|
||||
pool_kwargs.update(initializer=tqdm_class.set_lock, initargs=(lk,))
|
||||
map_args = {}
|
||||
if not (3, 0) < sys_version < (3, 5):
|
||||
map_args.update(chunksize=chunksize)
|
||||
with PoolExecutor(**pool_kwargs) as ex:
|
||||
return list(tqdm_class(ex.map(fn, *iterables, **map_args), **kwargs))
|
||||
|
||||
|
||||
def thread_map(fn, *iterables, **tqdm_kwargs):
|
||||
"""
|
||||
Equivalent of `list(map(fn, *iterables))`
|
||||
driven by `concurrent.futures.ThreadPoolExecutor`.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
tqdm_class : optional
|
||||
`tqdm` class to use for bars [default: tqdm.auto.tqdm].
|
||||
max_workers : int, optional
|
||||
Maximum number of workers to spawn; passed to
|
||||
`concurrent.futures.ThreadPoolExecutor.__init__`.
|
||||
[default: max(32, cpu_count() + 4)].
|
||||
"""
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
return _executor_map(ThreadPoolExecutor, fn, *iterables, **tqdm_kwargs)
|
||||
|
||||
|
||||
def process_map(fn, *iterables, **tqdm_kwargs):
|
||||
"""
|
||||
Equivalent of `list(map(fn, *iterables))`
|
||||
driven by `concurrent.futures.ProcessPoolExecutor`.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
tqdm_class : optional
|
||||
`tqdm` class to use for bars [default: tqdm.auto.tqdm].
|
||||
max_workers : int, optional
|
||||
Maximum number of workers to spawn; passed to
|
||||
`concurrent.futures.ProcessPoolExecutor.__init__`.
|
||||
[default: min(32, cpu_count() + 4)].
|
||||
chunksize : int, optional
|
||||
Size of chunks sent to worker processes; passed to
|
||||
`concurrent.futures.ProcessPoolExecutor.map`. [default: 1].
|
||||
lock_name : str, optional
|
||||
Member of `tqdm_class.get_lock()` to use [default: mp_lock].
|
||||
"""
|
||||
from concurrent.futures import ProcessPoolExecutor
|
||||
if iterables and "chunksize" not in tqdm_kwargs:
|
||||
# default `chunksize=1` has poor performance for large iterables
|
||||
# (most time spent dispatching items to workers).
|
||||
longest_iterable_len = max(map(length_hint, iterables))
|
||||
if longest_iterable_len > 1000:
|
||||
from warnings import warn
|
||||
warn("Iterable length %d > 1000 but `chunksize` is not set."
|
||||
" This may seriously degrade multiprocess performance."
|
||||
" Set `chunksize=1` or more." % longest_iterable_len,
|
||||
TqdmWarning, stacklevel=2)
|
||||
if "lock_name" not in tqdm_kwargs:
|
||||
tqdm_kwargs = tqdm_kwargs.copy()
|
||||
tqdm_kwargs["lock_name"] = "mp_lock"
|
||||
return _executor_map(ProcessPoolExecutor, fn, *iterables, **tqdm_kwargs)
|
125
tqdm/contrib/discord.py
Normal file
125
tqdm/contrib/discord.py
Normal file
|
@ -0,0 +1,125 @@
|
|||
"""
|
||||
Sends updates to a Discord bot.
|
||||
|
||||
Usage:
|
||||
>>> from tqdm.contrib.discord import tqdm, trange
|
||||
>>> for i in trange(10, token='{token}', channel_id='{channel_id}'):
|
||||
... ...
|
||||
|
||||
![screenshot](https://img.tqdm.ml/screenshot-discord.png)
|
||||
"""
|
||||
from __future__ import absolute_import
|
||||
|
||||
import logging
|
||||
from os import getenv
|
||||
|
||||
try:
|
||||
from disco.client import Client, ClientConfig
|
||||
except ImportError:
|
||||
raise ImportError("Please `pip install disco-py`")
|
||||
|
||||
from ..auto import tqdm as tqdm_auto
|
||||
from ..utils import _range
|
||||
from .utils_worker import MonoWorker
|
||||
|
||||
__author__ = {"github.com/": ["casperdcl"]}
|
||||
__all__ = ['DiscordIO', 'tqdm_discord', 'tdrange', 'tqdm', 'trange']
|
||||
|
||||
|
||||
class DiscordIO(MonoWorker):
|
||||
"""Non-blocking file-like IO using a Discord Bot."""
|
||||
def __init__(self, token, channel_id):
|
||||
"""Creates a new message in the given `channel_id`."""
|
||||
super(DiscordIO, self).__init__()
|
||||
config = ClientConfig()
|
||||
config.token = token
|
||||
client = Client(config)
|
||||
self.text = self.__class__.__name__
|
||||
try:
|
||||
self.message = client.api.channels_messages_create(channel_id, self.text)
|
||||
except Exception as e:
|
||||
tqdm_auto.write(str(e))
|
||||
self.message = None
|
||||
|
||||
def write(self, s):
|
||||
"""Replaces internal `message`'s text with `s`."""
|
||||
if not s:
|
||||
s = "..."
|
||||
s = s.replace('\r', '').strip()
|
||||
if s == self.text:
|
||||
return # skip duplicate message
|
||||
message = self.message
|
||||
if message is None:
|
||||
return
|
||||
self.text = s
|
||||
try:
|
||||
future = self.submit(message.edit, '`' + s + '`')
|
||||
except Exception as e:
|
||||
tqdm_auto.write(str(e))
|
||||
else:
|
||||
return future
|
||||
|
||||
|
||||
class tqdm_discord(tqdm_auto):
|
||||
"""
|
||||
Standard `tqdm.auto.tqdm` but also sends updates to a Discord Bot.
|
||||
May take a few seconds to create (`__init__`).
|
||||
|
||||
- create a discord bot (not public, no requirement of OAuth2 code
|
||||
grant, only send message permissions) & invite it to a channel:
|
||||
<https://discordpy.readthedocs.io/en/latest/discord.html>
|
||||
- copy the bot `{token}` & `{channel_id}` and paste below
|
||||
|
||||
>>> from tqdm.contrib.discord import tqdm, trange
|
||||
>>> for i in tqdm(iterable, token='{token}', channel_id='{channel_id}'):
|
||||
... ...
|
||||
"""
|
||||
def __init__(self, *args, **kwargs):
|
||||
"""
|
||||
Parameters
|
||||
----------
|
||||
token : str, required. Discord token
|
||||
[default: ${TQDM_DISCORD_TOKEN}].
|
||||
channel_id : int, required. Discord channel ID
|
||||
[default: ${TQDM_DISCORD_CHANNEL_ID}].
|
||||
mininterval : float, optional.
|
||||
Minimum of [default: 1.5] to avoid rate limit.
|
||||
|
||||
See `tqdm.auto.tqdm.__init__` for other parameters.
|
||||
"""
|
||||
if not kwargs.get('disable'):
|
||||
kwargs = kwargs.copy()
|
||||
logging.getLogger("HTTPClient").setLevel(logging.WARNING)
|
||||
self.dio = DiscordIO(
|
||||
kwargs.pop('token', getenv("TQDM_DISCORD_TOKEN")),
|
||||
kwargs.pop('channel_id', getenv("TQDM_DISCORD_CHANNEL_ID")))
|
||||
kwargs['mininterval'] = max(1.5, kwargs.get('mininterval', 1.5))
|
||||
super(tqdm_discord, self).__init__(*args, **kwargs)
|
||||
|
||||
def display(self, **kwargs):
|
||||
super(tqdm_discord, self).display(**kwargs)
|
||||
fmt = self.format_dict
|
||||
if fmt.get('bar_format', None):
|
||||
fmt['bar_format'] = fmt['bar_format'].replace(
|
||||
'<bar/>', '{bar:10u}').replace('{bar}', '{bar:10u}')
|
||||
else:
|
||||
fmt['bar_format'] = '{l_bar}{bar:10u}{r_bar}'
|
||||
self.dio.write(self.format_meter(**fmt))
|
||||
|
||||
def clear(self, *args, **kwargs):
|
||||
super(tqdm_discord, self).clear(*args, **kwargs)
|
||||
if not self.disable:
|
||||
self.dio.write("")
|
||||
|
||||
|
||||
def tdrange(*args, **kwargs):
|
||||
"""
|
||||
A shortcut for `tqdm.contrib.discord.tqdm(xrange(*args), **kwargs)`.
|
||||
On Python3+, `range` is used instead of `xrange`.
|
||||
"""
|
||||
return tqdm_discord(_range(*args), **kwargs)
|
||||
|
||||
|
||||
# Aliases
|
||||
tqdm = tqdm_discord
|
||||
trange = tdrange
|
37
tqdm/contrib/itertools.py
Normal file
37
tqdm/contrib/itertools.py
Normal file
|
@ -0,0 +1,37 @@
|
|||
"""
|
||||
Thin wrappers around `itertools`.
|
||||
"""
|
||||
from __future__ import absolute_import
|
||||
|
||||
import itertools
|
||||
|
||||
from ..auto import tqdm as tqdm_auto
|
||||
|
||||
__author__ = {"github.com/": ["casperdcl"]}
|
||||
__all__ = ['product']
|
||||
|
||||
|
||||
def product(*iterables, **tqdm_kwargs):
|
||||
"""
|
||||
Equivalent of `itertools.product`.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
tqdm_class : [default: tqdm.auto.tqdm].
|
||||
"""
|
||||
kwargs = tqdm_kwargs.copy()
|
||||
tqdm_class = kwargs.pop("tqdm_class", tqdm_auto)
|
||||
try:
|
||||
lens = list(map(len, iterables))
|
||||
except TypeError:
|
||||
total = None
|
||||
else:
|
||||
total = 1
|
||||
for i in lens:
|
||||
total *= i
|
||||
kwargs.setdefault("total", total)
|
||||
with tqdm_class(**kwargs) as t:
|
||||
it = itertools.product(*iterables)
|
||||
for i in it:
|
||||
yield i
|
||||
t.update()
|
128
tqdm/contrib/logging.py
Normal file
128
tqdm/contrib/logging.py
Normal file
|
@ -0,0 +1,128 @@
|
|||
"""
|
||||
Helper functionality for interoperability with stdlib `logging`.
|
||||
"""
|
||||
from __future__ import absolute_import
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from contextlib import contextmanager
|
||||
|
||||
try:
|
||||
from typing import Iterator, List, Optional, Type # pylint: disable=unused-import
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
from ..std import tqdm as std_tqdm
|
||||
|
||||
|
||||
class _TqdmLoggingHandler(logging.StreamHandler):
|
||||
def __init__(
|
||||
self,
|
||||
tqdm_class=std_tqdm # type: Type[std_tqdm]
|
||||
):
|
||||
super(_TqdmLoggingHandler, self).__init__()
|
||||
self.tqdm_class = tqdm_class
|
||||
|
||||
def emit(self, record):
|
||||
try:
|
||||
msg = self.format(record)
|
||||
self.tqdm_class.write(msg, file=self.stream)
|
||||
self.flush()
|
||||
except (KeyboardInterrupt, SystemExit):
|
||||
raise
|
||||
except: # noqa pylint: disable=bare-except
|
||||
self.handleError(record)
|
||||
|
||||
|
||||
def _is_console_logging_handler(handler):
|
||||
return (isinstance(handler, logging.StreamHandler)
|
||||
and handler.stream in {sys.stdout, sys.stderr})
|
||||
|
||||
|
||||
def _get_first_found_console_logging_handler(handlers):
|
||||
for handler in handlers:
|
||||
if _is_console_logging_handler(handler):
|
||||
return handler
|
||||
|
||||
|
||||
@contextmanager
|
||||
def logging_redirect_tqdm(
|
||||
loggers=None, # type: Optional[List[logging.Logger]],
|
||||
tqdm_class=std_tqdm # type: Type[std_tqdm]
|
||||
):
|
||||
# type: (...) -> Iterator[None]
|
||||
"""
|
||||
Context manager redirecting console logging to `tqdm.write()`, leaving
|
||||
other logging handlers (e.g. log files) unaffected.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
loggers : list, optional
|
||||
Which handlers to redirect (default: [logging.root]).
|
||||
tqdm_class : optional
|
||||
|
||||
Example
|
||||
-------
|
||||
```python
|
||||
import logging
|
||||
from tqdm import trange
|
||||
from tqdm.contrib.logging import logging_redirect_tqdm
|
||||
|
||||
LOG = logging.getLogger(__name__)
|
||||
|
||||
if __name__ == '__main__':
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
with logging_redirect_tqdm():
|
||||
for i in trange(9):
|
||||
if i == 4:
|
||||
LOG.info("console logging redirected to `tqdm.write()`")
|
||||
# logging restored
|
||||
```
|
||||
"""
|
||||
if loggers is None:
|
||||
loggers = [logging.root]
|
||||
original_handlers_list = [logger.handlers for logger in loggers]
|
||||
try:
|
||||
for logger in loggers:
|
||||
tqdm_handler = _TqdmLoggingHandler(tqdm_class)
|
||||
orig_handler = _get_first_found_console_logging_handler(logger.handlers)
|
||||
if orig_handler is not None:
|
||||
tqdm_handler.setFormatter(orig_handler.formatter)
|
||||
tqdm_handler.stream = orig_handler.stream
|
||||
logger.handlers = [
|
||||
handler for handler in logger.handlers
|
||||
if not _is_console_logging_handler(handler)] + [tqdm_handler]
|
||||
yield
|
||||
finally:
|
||||
for logger, original_handlers in zip(loggers, original_handlers_list):
|
||||
logger.handlers = original_handlers
|
||||
|
||||
|
||||
@contextmanager
|
||||
def tqdm_logging_redirect(
|
||||
*args,
|
||||
# loggers=None, # type: Optional[List[logging.Logger]]
|
||||
# tqdm=None, # type: Optional[Type[tqdm.tqdm]]
|
||||
**kwargs
|
||||
):
|
||||
# type: (...) -> Iterator[None]
|
||||
"""
|
||||
Convenience shortcut for:
|
||||
```python
|
||||
with tqdm_class(*args, **tqdm_kwargs) as pbar:
|
||||
with logging_redirect_tqdm(loggers=loggers, tqdm_class=tqdm_class):
|
||||
yield pbar
|
||||
```
|
||||
|
||||
Parameters
|
||||
----------
|
||||
tqdm_class : optional, (default: tqdm.std.tqdm).
|
||||
loggers : optional, list.
|
||||
**tqdm_kwargs : passed to `tqdm_class`.
|
||||
"""
|
||||
tqdm_kwargs = kwargs.copy()
|
||||
loggers = tqdm_kwargs.pop('loggers', None)
|
||||
tqdm_class = tqdm_kwargs.pop('tqdm_class', std_tqdm)
|
||||
with tqdm_class(*args, **tqdm_kwargs) as pbar:
|
||||
with logging_redirect_tqdm(loggers=loggers, tqdm_class=tqdm_class):
|
||||
yield pbar
|
126
tqdm/contrib/slack.py
Normal file
126
tqdm/contrib/slack.py
Normal file
|
@ -0,0 +1,126 @@
|
|||
"""
|
||||
Sends updates to a Slack app.
|
||||
|
||||
Usage:
|
||||
>>> from tqdm.contrib.slack import tqdm, trange
|
||||
>>> for i in trange(10, token='{token}', channel='{channel}'):
|
||||
... ...
|
||||
|
||||
![screenshot](https://img.tqdm.ml/screenshot-slack.png)
|
||||
"""
|
||||
from __future__ import absolute_import
|
||||
|
||||
import logging
|
||||
from os import getenv
|
||||
|
||||
try:
|
||||
from slack_sdk import WebClient
|
||||
except ImportError:
|
||||
raise ImportError("Please `pip install slack-sdk`")
|
||||
|
||||
from ..auto import tqdm as tqdm_auto
|
||||
from ..utils import _range
|
||||
from .utils_worker import MonoWorker
|
||||
|
||||
__author__ = {"github.com/": ["0x2b3bfa0", "casperdcl"]}
|
||||
__all__ = ['SlackIO', 'tqdm_slack', 'tsrange', 'tqdm', 'trange']
|
||||
|
||||
|
||||
class SlackIO(MonoWorker):
|
||||
"""Non-blocking file-like IO using a Slack app."""
|
||||
def __init__(self, token, channel):
|
||||
"""Creates a new message in the given `channel`."""
|
||||
super(SlackIO, self).__init__()
|
||||
self.client = WebClient(token=token)
|
||||
self.text = self.__class__.__name__
|
||||
try:
|
||||
self.message = self.client.chat_postMessage(channel=channel, text=self.text)
|
||||
except Exception as e:
|
||||
tqdm_auto.write(str(e))
|
||||
self.message = None
|
||||
|
||||
def write(self, s):
|
||||
"""Replaces internal `message`'s text with `s`."""
|
||||
if not s:
|
||||
s = "..."
|
||||
s = s.replace('\r', '').strip()
|
||||
if s == self.text:
|
||||
return # skip duplicate message
|
||||
message = self.message
|
||||
if message is None:
|
||||
return
|
||||
self.text = s
|
||||
try:
|
||||
future = self.submit(self.client.chat_update, channel=message['channel'],
|
||||
ts=message['ts'], text='`' + s + '`')
|
||||
except Exception as e:
|
||||
tqdm_auto.write(str(e))
|
||||
else:
|
||||
return future
|
||||
|
||||
|
||||
class tqdm_slack(tqdm_auto):
|
||||
"""
|
||||
Standard `tqdm.auto.tqdm` but also sends updates to a Slack app.
|
||||
May take a few seconds to create (`__init__`).
|
||||
|
||||
- create a Slack app with the `chat:write` scope & invite it to a
|
||||
channel: <https://api.slack.com/authentication/basics>
|
||||
- copy the bot `{token}` & `{channel}` and paste below
|
||||
>>> from tqdm.contrib.slack import tqdm, trange
|
||||
>>> for i in tqdm(iterable, token='{token}', channel='{channel}'):
|
||||
... ...
|
||||
"""
|
||||
def __init__(self, *args, **kwargs):
|
||||
"""
|
||||
Parameters
|
||||
----------
|
||||
token : str, required. Slack token
|
||||
[default: ${TQDM_SLACK_TOKEN}].
|
||||
channel : int, required. Slack channel
|
||||
[default: ${TQDM_SLACK_CHANNEL}].
|
||||
mininterval : float, optional.
|
||||
Minimum of [default: 1.5] to avoid rate limit.
|
||||
|
||||
See `tqdm.auto.tqdm.__init__` for other parameters.
|
||||
"""
|
||||
if not kwargs.get('disable'):
|
||||
kwargs = kwargs.copy()
|
||||
logging.getLogger("HTTPClient").setLevel(logging.WARNING)
|
||||
self.sio = SlackIO(
|
||||
kwargs.pop('token', getenv("TQDM_SLACK_TOKEN")),
|
||||
kwargs.pop('channel', getenv("TQDM_SLACK_CHANNEL")))
|
||||
kwargs['mininterval'] = max(1.5, kwargs.get('mininterval', 1.5))
|
||||
super(tqdm_slack, self).__init__(*args, **kwargs)
|
||||
|
||||
def display(self, **kwargs):
|
||||
super(tqdm_slack, self).display(**kwargs)
|
||||
fmt = self.format_dict
|
||||
if fmt.get('bar_format', None):
|
||||
fmt['bar_format'] = fmt['bar_format'].replace(
|
||||
'<bar/>', '`{bar:10}`').replace('{bar}', '`{bar:10u}`')
|
||||
else:
|
||||
fmt['bar_format'] = '{l_bar}`{bar:10}`{r_bar}'
|
||||
if fmt['ascii'] is False:
|
||||
fmt['ascii'] = [":black_square:", ":small_blue_diamond:", ":large_blue_diamond:",
|
||||
":large_blue_square:"]
|
||||
fmt['ncols'] = 336
|
||||
self.sio.write(self.format_meter(**fmt))
|
||||
|
||||
def clear(self, *args, **kwargs):
|
||||
super(tqdm_slack, self).clear(*args, **kwargs)
|
||||
if not self.disable:
|
||||
self.sio.write("")
|
||||
|
||||
|
||||
def tsrange(*args, **kwargs):
|
||||
"""
|
||||
A shortcut for `tqdm.contrib.slack.tqdm(xrange(*args), **kwargs)`.
|
||||
On Python3+, `range` is used instead of `xrange`.
|
||||
"""
|
||||
return tqdm_slack(_range(*args), **kwargs)
|
||||
|
||||
|
||||
# Aliases
|
||||
tqdm = tqdm_slack
|
||||
trange = tsrange
|
159
tqdm/contrib/telegram.py
Normal file
159
tqdm/contrib/telegram.py
Normal file
|
@ -0,0 +1,159 @@
|
|||
"""
|
||||
Sends updates to a Telegram bot.
|
||||
|
||||
Usage:
|
||||
>>> from tqdm.contrib.telegram import tqdm, trange
|
||||
>>> for i in trange(10, token='{token}', chat_id='{chat_id}'):
|
||||
... ...
|
||||
|
||||
![screenshot](https://img.tqdm.ml/screenshot-telegram.gif)
|
||||
"""
|
||||
from __future__ import absolute_import
|
||||
|
||||
from os import getenv
|
||||
from warnings import warn
|
||||
|
||||
from requests import Session
|
||||
|
||||
from ..auto import tqdm as tqdm_auto
|
||||
from ..std import TqdmWarning
|
||||
from ..utils import _range
|
||||
from .utils_worker import MonoWorker
|
||||
|
||||
__author__ = {"github.com/": ["casperdcl"]}
|
||||
__all__ = ['TelegramIO', 'tqdm_telegram', 'ttgrange', 'tqdm', 'trange']
|
||||
|
||||
|
||||
class TelegramIO(MonoWorker):
|
||||
"""Non-blocking file-like IO using a Telegram Bot."""
|
||||
API = 'https://api.telegram.org/bot'
|
||||
|
||||
def __init__(self, token, chat_id):
|
||||
"""Creates a new message in the given `chat_id`."""
|
||||
super(TelegramIO, self).__init__()
|
||||
self.token = token
|
||||
self.chat_id = chat_id
|
||||
self.session = Session()
|
||||
self.text = self.__class__.__name__
|
||||
self.message_id
|
||||
|
||||
@property
|
||||
def message_id(self):
|
||||
if hasattr(self, '_message_id'):
|
||||
return self._message_id
|
||||
try:
|
||||
res = self.session.post(
|
||||
self.API + '%s/sendMessage' % self.token,
|
||||
data={'text': '`' + self.text + '`', 'chat_id': self.chat_id,
|
||||
'parse_mode': 'MarkdownV2'}).json()
|
||||
except Exception as e:
|
||||
tqdm_auto.write(str(e))
|
||||
else:
|
||||
if res.get('error_code') == 429:
|
||||
warn("Creation rate limit: try increasing `mininterval`.",
|
||||
TqdmWarning, stacklevel=2)
|
||||
else:
|
||||
self._message_id = res['result']['message_id']
|
||||
return self._message_id
|
||||
|
||||
def write(self, s):
|
||||
"""Replaces internal `message_id`'s text with `s`."""
|
||||
if not s:
|
||||
s = "..."
|
||||
s = s.replace('\r', '').strip()
|
||||
if s == self.text:
|
||||
return # avoid duplicate message Bot error
|
||||
message_id = self.message_id
|
||||
if message_id is None:
|
||||
return
|
||||
self.text = s
|
||||
try:
|
||||
future = self.submit(
|
||||
self.session.post, self.API + '%s/editMessageText' % self.token,
|
||||
data={'text': '`' + s + '`', 'chat_id': self.chat_id,
|
||||
'message_id': message_id, 'parse_mode': 'MarkdownV2'})
|
||||
except Exception as e:
|
||||
tqdm_auto.write(str(e))
|
||||
else:
|
||||
return future
|
||||
|
||||
def delete(self):
|
||||
"""Deletes internal `message_id`."""
|
||||
try:
|
||||
future = self.submit(
|
||||
self.session.post, self.API + '%s/deleteMessage' % self.token,
|
||||
data={'chat_id': self.chat_id, 'message_id': self.message_id})
|
||||
except Exception as e:
|
||||
tqdm_auto.write(str(e))
|
||||
else:
|
||||
return future
|
||||
|
||||
|
||||
class tqdm_telegram(tqdm_auto):
|
||||
"""
|
||||
Standard `tqdm.auto.tqdm` but also sends updates to a Telegram Bot.
|
||||
May take a few seconds to create (`__init__`).
|
||||
|
||||
- create a bot <https://core.telegram.org/bots#6-botfather>
|
||||
- copy its `{token}`
|
||||
- add the bot to a chat and send it a message such as `/start`
|
||||
- go to <https://api.telegram.org/bot`{token}`/getUpdates> to find out
|
||||
the `{chat_id}`
|
||||
- paste the `{token}` & `{chat_id}` below
|
||||
|
||||
>>> from tqdm.contrib.telegram import tqdm, trange
|
||||
>>> for i in tqdm(iterable, token='{token}', chat_id='{chat_id}'):
|
||||
... ...
|
||||
"""
|
||||
def __init__(self, *args, **kwargs):
|
||||
"""
|
||||
Parameters
|
||||
----------
|
||||
token : str, required. Telegram token
|
||||
[default: ${TQDM_TELEGRAM_TOKEN}].
|
||||
chat_id : str, required. Telegram chat ID
|
||||
[default: ${TQDM_TELEGRAM_CHAT_ID}].
|
||||
|
||||
See `tqdm.auto.tqdm.__init__` for other parameters.
|
||||
"""
|
||||
if not kwargs.get('disable'):
|
||||
kwargs = kwargs.copy()
|
||||
self.tgio = TelegramIO(
|
||||
kwargs.pop('token', getenv('TQDM_TELEGRAM_TOKEN')),
|
||||
kwargs.pop('chat_id', getenv('TQDM_TELEGRAM_CHAT_ID')))
|
||||
super(tqdm_telegram, self).__init__(*args, **kwargs)
|
||||
|
||||
def display(self, **kwargs):
|
||||
super(tqdm_telegram, self).display(**kwargs)
|
||||
fmt = self.format_dict
|
||||
if fmt.get('bar_format', None):
|
||||
fmt['bar_format'] = fmt['bar_format'].replace(
|
||||
'<bar/>', '{bar:10u}').replace('{bar}', '{bar:10u}')
|
||||
else:
|
||||
fmt['bar_format'] = '{l_bar}{bar:10u}{r_bar}'
|
||||
self.tgio.write(self.format_meter(**fmt))
|
||||
|
||||
def clear(self, *args, **kwargs):
|
||||
super(tqdm_telegram, self).clear(*args, **kwargs)
|
||||
if not self.disable:
|
||||
self.tgio.write("")
|
||||
|
||||
def close(self):
|
||||
if self.disable:
|
||||
return
|
||||
super(tqdm_telegram, self).close()
|
||||
if not (self.leave or (self.leave is None and self.pos == 0)):
|
||||
self.tgio.delete()
|
||||
|
||||
|
||||
def ttgrange(*args, **kwargs):
|
||||
"""
|
||||
A shortcut for `tqdm.contrib.telegram.tqdm(xrange(*args), **kwargs)`.
|
||||
On Python3+, `range` is used instead of `xrange`.
|
||||
"""
|
||||
return tqdm_telegram(_range(*args), **kwargs)
|
||||
|
||||
|
||||
# Aliases
|
||||
tqdm = tqdm_telegram
|
||||
trange = ttgrange
|
40
tqdm/contrib/utils_worker.py
Normal file
40
tqdm/contrib/utils_worker.py
Normal file
|
@ -0,0 +1,40 @@
|
|||
"""
|
||||
IO/concurrency helpers for `tqdm.contrib`.
|
||||
"""
|
||||
from __future__ import absolute_import
|
||||
|
||||
from collections import deque
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
from ..auto import tqdm as tqdm_auto
|
||||
|
||||
__author__ = {"github.com/": ["casperdcl"]}
|
||||
__all__ = ['MonoWorker']
|
||||
|
||||
|
||||
class MonoWorker(object):
|
||||
"""
|
||||
Supports one running task and one waiting task.
|
||||
The waiting task is the most recent submitted (others are discarded).
|
||||
"""
|
||||
def __init__(self):
|
||||
self.pool = ThreadPoolExecutor(max_workers=1)
|
||||
self.futures = deque([], 2)
|
||||
|
||||
def submit(self, func, *args, **kwargs):
|
||||
"""`func(*args, **kwargs)` may replace currently waiting task."""
|
||||
futures = self.futures
|
||||
if len(futures) == futures.maxlen:
|
||||
running = futures.popleft()
|
||||
if not running.done():
|
||||
if len(futures): # clear waiting
|
||||
waiting = futures.pop()
|
||||
waiting.cancel()
|
||||
futures.appendleft(running) # re-insert running
|
||||
try:
|
||||
waiting = self.pool.submit(func, *args, **kwargs)
|
||||
except Exception as e:
|
||||
tqdm_auto.write(str(e))
|
||||
else:
|
||||
futures.append(waiting)
|
||||
return waiting
|
46
tqdm/dask.py
Normal file
46
tqdm/dask.py
Normal file
|
@ -0,0 +1,46 @@
|
|||
from __future__ import absolute_import
|
||||
|
||||
from functools import partial
|
||||
|
||||
from dask.callbacks import Callback
|
||||
|
||||
from .auto import tqdm as tqdm_auto
|
||||
|
||||
__author__ = {"github.com/": ["casperdcl"]}
|
||||
__all__ = ['TqdmCallback']
|
||||
|
||||
|
||||
class TqdmCallback(Callback):
|
||||
"""Dask callback for task progress."""
|
||||
def __init__(self, start=None, pretask=None, tqdm_class=tqdm_auto,
|
||||
**tqdm_kwargs):
|
||||
"""
|
||||
Parameters
|
||||
----------
|
||||
tqdm_class : optional
|
||||
`tqdm` class to use for bars [default: `tqdm.auto.tqdm`].
|
||||
tqdm_kwargs : optional
|
||||
Any other arguments used for all bars.
|
||||
"""
|
||||
super(TqdmCallback, self).__init__(start=start, pretask=pretask)
|
||||
if tqdm_kwargs:
|
||||
tqdm_class = partial(tqdm_class, **tqdm_kwargs)
|
||||
self.tqdm_class = tqdm_class
|
||||
|
||||
def _start_state(self, _, state):
|
||||
self.pbar = self.tqdm_class(total=sum(
|
||||
len(state[k]) for k in ['ready', 'waiting', 'running', 'finished']))
|
||||
|
||||
def _posttask(self, *_, **__):
|
||||
self.pbar.update()
|
||||
|
||||
def _finish(self, *_, **__):
|
||||
self.pbar.close()
|
||||
|
||||
def display(self):
|
||||
"""Displays in the current cell in Notebooks."""
|
||||
container = getattr(self.bar, 'container', None)
|
||||
if container is None:
|
||||
return
|
||||
from .notebook import display
|
||||
display(container)
|
191
tqdm/gui.py
Normal file
191
tqdm/gui.py
Normal file
|
@ -0,0 +1,191 @@
|
|||
"""
|
||||
Matplotlib GUI progressbar decorator for iterators.
|
||||
|
||||
Usage:
|
||||
>>> from tqdm.gui import trange, tqdm
|
||||
>>> for i in trange(10):
|
||||
... ...
|
||||
"""
|
||||
# future division is important to divide integers and get as
|
||||
# a result precise floating numbers (instead of truncated int)
|
||||
from __future__ import absolute_import, division
|
||||
|
||||
import re
|
||||
from warnings import warn
|
||||
|
||||
# to inherit from the tqdm class
|
||||
from .std import TqdmExperimentalWarning
|
||||
from .std import tqdm as std_tqdm
|
||||
# import compatibility functions and utilities
|
||||
from .utils import _range
|
||||
|
||||
__author__ = {"github.com/": ["casperdcl", "lrq3000"]}
|
||||
__all__ = ['tqdm_gui', 'tgrange', 'tqdm', 'trange']
|
||||
|
||||
|
||||
class tqdm_gui(std_tqdm): # pragma: no cover
|
||||
"""Experimental Matplotlib GUI version of tqdm!"""
|
||||
# TODO: @classmethod: write() on GUI?
|
||||
def __init__(self, *args, **kwargs):
|
||||
from collections import deque
|
||||
|
||||
import matplotlib as mpl
|
||||
import matplotlib.pyplot as plt
|
||||
kwargs = kwargs.copy()
|
||||
kwargs['gui'] = True
|
||||
colour = kwargs.pop('colour', 'g')
|
||||
super(tqdm_gui, self).__init__(*args, **kwargs)
|
||||
|
||||
if self.disable:
|
||||
return
|
||||
|
||||
warn("GUI is experimental/alpha", TqdmExperimentalWarning, stacklevel=2)
|
||||
self.mpl = mpl
|
||||
self.plt = plt
|
||||
|
||||
# Remember if external environment uses toolbars
|
||||
self.toolbar = self.mpl.rcParams['toolbar']
|
||||
self.mpl.rcParams['toolbar'] = 'None'
|
||||
|
||||
self.mininterval = max(self.mininterval, 0.5)
|
||||
self.fig, ax = plt.subplots(figsize=(9, 2.2))
|
||||
# self.fig.subplots_adjust(bottom=0.2)
|
||||
total = self.__len__() # avoids TypeError on None #971
|
||||
if total is not None:
|
||||
self.xdata = []
|
||||
self.ydata = []
|
||||
self.zdata = []
|
||||
else:
|
||||
self.xdata = deque([])
|
||||
self.ydata = deque([])
|
||||
self.zdata = deque([])
|
||||
self.line1, = ax.plot(self.xdata, self.ydata, color='b')
|
||||
self.line2, = ax.plot(self.xdata, self.zdata, color='k')
|
||||
ax.set_ylim(0, 0.001)
|
||||
if total is not None:
|
||||
ax.set_xlim(0, 100)
|
||||
ax.set_xlabel("percent")
|
||||
self.fig.legend((self.line1, self.line2), ("cur", "est"),
|
||||
loc='center right')
|
||||
# progressbar
|
||||
self.hspan = plt.axhspan(0, 0.001, xmin=0, xmax=0, color=colour)
|
||||
else:
|
||||
# ax.set_xlim(-60, 0)
|
||||
ax.set_xlim(0, 60)
|
||||
ax.invert_xaxis()
|
||||
ax.set_xlabel("seconds")
|
||||
ax.legend(("cur", "est"), loc='lower left')
|
||||
ax.grid()
|
||||
# ax.set_xlabel('seconds')
|
||||
ax.set_ylabel((self.unit if self.unit else "it") + "/s")
|
||||
if self.unit_scale:
|
||||
plt.ticklabel_format(style='sci', axis='y', scilimits=(0, 0))
|
||||
ax.yaxis.get_offset_text().set_x(-0.15)
|
||||
|
||||
# Remember if external environment is interactive
|
||||
self.wasion = plt.isinteractive()
|
||||
plt.ion()
|
||||
self.ax = ax
|
||||
|
||||
def close(self):
|
||||
if self.disable:
|
||||
return
|
||||
|
||||
self.disable = True
|
||||
|
||||
with self.get_lock():
|
||||
self._instances.remove(self)
|
||||
|
||||
# Restore toolbars
|
||||
self.mpl.rcParams['toolbar'] = self.toolbar
|
||||
# Return to non-interactive mode
|
||||
if not self.wasion:
|
||||
self.plt.ioff()
|
||||
if self.leave:
|
||||
self.display()
|
||||
else:
|
||||
self.plt.close(self.fig)
|
||||
|
||||
def clear(self, *_, **__):
|
||||
pass
|
||||
|
||||
def display(self, *_, **__):
|
||||
n = self.n
|
||||
cur_t = self._time()
|
||||
elapsed = cur_t - self.start_t
|
||||
delta_it = n - self.last_print_n
|
||||
delta_t = cur_t - self.last_print_t
|
||||
|
||||
# Inline due to multiple calls
|
||||
total = self.total
|
||||
xdata = self.xdata
|
||||
ydata = self.ydata
|
||||
zdata = self.zdata
|
||||
ax = self.ax
|
||||
line1 = self.line1
|
||||
line2 = self.line2
|
||||
# instantaneous rate
|
||||
y = delta_it / delta_t
|
||||
# overall rate
|
||||
z = n / elapsed
|
||||
# update line data
|
||||
xdata.append(n * 100.0 / total if total else cur_t)
|
||||
ydata.append(y)
|
||||
zdata.append(z)
|
||||
|
||||
# Discard old values
|
||||
# xmin, xmax = ax.get_xlim()
|
||||
# if (not total) and elapsed > xmin * 1.1:
|
||||
if (not total) and elapsed > 66:
|
||||
xdata.popleft()
|
||||
ydata.popleft()
|
||||
zdata.popleft()
|
||||
|
||||
ymin, ymax = ax.get_ylim()
|
||||
if y > ymax or z > ymax:
|
||||
ymax = 1.1 * y
|
||||
ax.set_ylim(ymin, ymax)
|
||||
ax.figure.canvas.draw()
|
||||
|
||||
if total:
|
||||
line1.set_data(xdata, ydata)
|
||||
line2.set_data(xdata, zdata)
|
||||
try:
|
||||
poly_lims = self.hspan.get_xy()
|
||||
except AttributeError:
|
||||
self.hspan = self.plt.axhspan(0, 0.001, xmin=0, xmax=0, color='g')
|
||||
poly_lims = self.hspan.get_xy()
|
||||
poly_lims[0, 1] = ymin
|
||||
poly_lims[1, 1] = ymax
|
||||
poly_lims[2] = [n / total, ymax]
|
||||
poly_lims[3] = [poly_lims[2, 0], ymin]
|
||||
if len(poly_lims) > 4:
|
||||
poly_lims[4, 1] = ymin
|
||||
self.hspan.set_xy(poly_lims)
|
||||
else:
|
||||
t_ago = [cur_t - i for i in xdata]
|
||||
line1.set_data(t_ago, ydata)
|
||||
line2.set_data(t_ago, zdata)
|
||||
|
||||
d = self.format_dict
|
||||
# remove {bar}
|
||||
d['bar_format'] = (d['bar_format'] or "{l_bar}<bar/>{r_bar}").replace(
|
||||
"{bar}", "<bar/>")
|
||||
msg = self.format_meter(**d)
|
||||
if '<bar/>' in msg:
|
||||
msg = "".join(re.split(r'\|?<bar/>\|?', msg, 1))
|
||||
ax.set_title(msg, fontname="DejaVu Sans Mono", fontsize=11)
|
||||
self.plt.pause(1e-9)
|
||||
|
||||
|
||||
def tgrange(*args, **kwargs):
|
||||
"""
|
||||
A shortcut for `tqdm.gui.tqdm(xrange(*args), **kwargs)`.
|
||||
On Python3+, `range` is used instead of `xrange`.
|
||||
"""
|
||||
return tqdm_gui(_range(*args), **kwargs)
|
||||
|
||||
|
||||
# Aliases
|
||||
tqdm = tqdm_gui
|
||||
trange = tgrange
|
124
tqdm/keras.py
Normal file
124
tqdm/keras.py
Normal file
|
@ -0,0 +1,124 @@
|
|||
from __future__ import absolute_import, division
|
||||
|
||||
from copy import copy
|
||||
from functools import partial
|
||||
|
||||
from .auto import tqdm as tqdm_auto
|
||||
|
||||
try:
|
||||
import keras
|
||||
except (ImportError, AttributeError) as e:
|
||||
try:
|
||||
from tensorflow import keras
|
||||
except ImportError:
|
||||
raise e
|
||||
__author__ = {"github.com/": ["casperdcl"]}
|
||||
__all__ = ['TqdmCallback']
|
||||
|
||||
|
||||
class TqdmCallback(keras.callbacks.Callback):
|
||||
"""Keras callback for epoch and batch progress."""
|
||||
@staticmethod
|
||||
def bar2callback(bar, pop=None, delta=(lambda logs: 1)):
|
||||
def callback(_, logs=None):
|
||||
n = delta(logs)
|
||||
if logs:
|
||||
if pop:
|
||||
logs = copy(logs)
|
||||
[logs.pop(i, 0) for i in pop]
|
||||
bar.set_postfix(logs, refresh=False)
|
||||
bar.update(n)
|
||||
|
||||
return callback
|
||||
|
||||
def __init__(self, epochs=None, data_size=None, batch_size=None, verbose=1,
|
||||
tqdm_class=tqdm_auto, **tqdm_kwargs):
|
||||
"""
|
||||
Parameters
|
||||
----------
|
||||
epochs : int, optional
|
||||
data_size : int, optional
|
||||
Number of training pairs.
|
||||
batch_size : int, optional
|
||||
Number of training pairs per batch.
|
||||
verbose : int
|
||||
0: epoch, 1: batch (transient), 2: batch. [default: 1].
|
||||
Will be set to `0` unless both `data_size` and `batch_size`
|
||||
are given.
|
||||
tqdm_class : optional
|
||||
`tqdm` class to use for bars [default: `tqdm.auto.tqdm`].
|
||||
tqdm_kwargs : optional
|
||||
Any other arguments used for all bars.
|
||||
"""
|
||||
if tqdm_kwargs:
|
||||
tqdm_class = partial(tqdm_class, **tqdm_kwargs)
|
||||
self.tqdm_class = tqdm_class
|
||||
self.epoch_bar = tqdm_class(total=epochs, unit='epoch')
|
||||
self.on_epoch_end = self.bar2callback(self.epoch_bar)
|
||||
if data_size and batch_size:
|
||||
self.batches = batches = (data_size + batch_size - 1) // batch_size
|
||||
else:
|
||||
self.batches = batches = None
|
||||
self.verbose = verbose
|
||||
if verbose == 1:
|
||||
self.batch_bar = tqdm_class(total=batches, unit='batch', leave=False)
|
||||
self.on_batch_end = self.bar2callback(
|
||||
self.batch_bar, pop=['batch', 'size'],
|
||||
delta=lambda logs: logs.get('size', 1))
|
||||
|
||||
def on_train_begin(self, *_, **__):
|
||||
params = self.params.get
|
||||
auto_total = params('epochs', params('nb_epoch', None))
|
||||
if auto_total is not None and auto_total != self.epoch_bar.total:
|
||||
self.epoch_bar.reset(total=auto_total)
|
||||
|
||||
def on_epoch_begin(self, epoch, *_, **__):
|
||||
if self.epoch_bar.n < epoch:
|
||||
ebar = self.epoch_bar
|
||||
ebar.n = ebar.last_print_n = ebar.initial = epoch
|
||||
if self.verbose:
|
||||
params = self.params.get
|
||||
total = params('samples', params(
|
||||
'nb_sample', params('steps', None))) or self.batches
|
||||
if self.verbose == 2:
|
||||
if hasattr(self, 'batch_bar'):
|
||||
self.batch_bar.close()
|
||||
self.batch_bar = self.tqdm_class(
|
||||
total=total, unit='batch', leave=True,
|
||||
unit_scale=1 / (params('batch_size', 1) or 1))
|
||||
self.on_batch_end = self.bar2callback(
|
||||
self.batch_bar, pop=['batch', 'size'],
|
||||
delta=lambda logs: logs.get('size', 1))
|
||||
elif self.verbose == 1:
|
||||
self.batch_bar.unit_scale = 1 / (params('batch_size', 1) or 1)
|
||||
self.batch_bar.reset(total=total)
|
||||
else:
|
||||
raise KeyError('Unknown verbosity')
|
||||
|
||||
def on_train_end(self, *_, **__):
|
||||
if self.verbose:
|
||||
self.batch_bar.close()
|
||||
self.epoch_bar.close()
|
||||
|
||||
def display(self):
|
||||
"""Displays in the current cell in Notebooks."""
|
||||
container = getattr(self.epoch_bar, 'container', None)
|
||||
if container is None:
|
||||
return
|
||||
from .notebook import display
|
||||
display(container)
|
||||
batch_bar = getattr(self, 'batch_bar', None)
|
||||
if batch_bar is not None:
|
||||
display(batch_bar.container)
|
||||
|
||||
@staticmethod
|
||||
def _implements_train_batch_hooks():
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
def _implements_test_batch_hooks():
|
||||
return True
|
||||
|
||||
@staticmethod
|
||||
def _implements_predict_batch_hooks():
|
||||
return True
|
329
tqdm/notebook.py
Normal file
329
tqdm/notebook.py
Normal file
|
@ -0,0 +1,329 @@
|
|||
"""
|
||||
IPython/Jupyter Notebook progressbar decorator for iterators.
|
||||
Includes a default `range` iterator printing to `stderr`.
|
||||
|
||||
Usage:
|
||||
>>> from tqdm.notebook import trange, tqdm
|
||||
>>> for i in trange(10):
|
||||
... ...
|
||||
"""
|
||||
# future division is important to divide integers and get as
|
||||
# a result precise floating numbers (instead of truncated int)
|
||||
from __future__ import absolute_import, division
|
||||
|
||||
# import compatibility functions and utilities
|
||||
import re
|
||||
import sys
|
||||
from weakref import proxy
|
||||
|
||||
# to inherit from the tqdm class
|
||||
from .std import tqdm as std_tqdm
|
||||
from .utils import _range
|
||||
|
||||
if True: # pragma: no cover
|
||||
# import IPython/Jupyter base widget and display utilities
|
||||
IPY = 0
|
||||
try: # IPython 4.x
|
||||
import ipywidgets
|
||||
IPY = 4
|
||||
except ImportError: # IPython 3.x / 2.x
|
||||
IPY = 32
|
||||
import warnings
|
||||
with warnings.catch_warnings():
|
||||
warnings.filterwarnings(
|
||||
'ignore', message=".*The `IPython.html` package has been deprecated.*")
|
||||
try:
|
||||
import IPython.html.widgets as ipywidgets # NOQA: F401
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
try: # IPython 4.x / 3.x
|
||||
if IPY == 32:
|
||||
from IPython.html.widgets import HTML
|
||||
from IPython.html.widgets import FloatProgress as IProgress
|
||||
from IPython.html.widgets import HBox
|
||||
IPY = 3
|
||||
else:
|
||||
from ipywidgets import HTML
|
||||
from ipywidgets import FloatProgress as IProgress
|
||||
from ipywidgets import HBox
|
||||
except ImportError:
|
||||
try: # IPython 2.x
|
||||
from IPython.html.widgets import HTML
|
||||
from IPython.html.widgets import ContainerWidget as HBox
|
||||
from IPython.html.widgets import FloatProgressWidget as IProgress
|
||||
IPY = 2
|
||||
except ImportError:
|
||||
IPY = 0
|
||||
IProgress = None
|
||||
HBox = object
|
||||
|
||||
try:
|
||||
from IPython.display import display # , clear_output
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
# HTML encoding
|
||||
try: # Py3
|
||||
from html import escape
|
||||
except ImportError: # Py2
|
||||
from cgi import escape
|
||||
|
||||
__author__ = {"github.com/": ["lrq3000", "casperdcl", "alexanderkuk"]}
|
||||
__all__ = ['tqdm_notebook', 'tnrange', 'tqdm', 'trange']
|
||||
WARN_NOIPYW = ("IProgress not found. Please update jupyter and ipywidgets."
|
||||
" See https://ipywidgets.readthedocs.io/en/stable"
|
||||
"/user_install.html")
|
||||
|
||||
|
||||
class TqdmHBox(HBox):
|
||||
"""`ipywidgets.HBox` with a pretty representation"""
|
||||
def _json_(self, pretty=None):
|
||||
pbar = getattr(self, 'pbar', None)
|
||||
if pbar is None:
|
||||
return {}
|
||||
d = pbar.format_dict
|
||||
if pretty is not None:
|
||||
d["ascii"] = not pretty
|
||||
return d
|
||||
|
||||
def __repr__(self, pretty=False):
|
||||
pbar = getattr(self, 'pbar', None)
|
||||
if pbar is None:
|
||||
return super(TqdmHBox, self).__repr__()
|
||||
return pbar.format_meter(**self._json_(pretty))
|
||||
|
||||
def _repr_pretty_(self, pp, *_, **__):
|
||||
pp.text(self.__repr__(True))
|
||||
|
||||
|
||||
class tqdm_notebook(std_tqdm):
|
||||
"""
|
||||
Experimental IPython/Jupyter Notebook widget using tqdm!
|
||||
"""
|
||||
@staticmethod
|
||||
def status_printer(_, total=None, desc=None, ncols=None):
|
||||
"""
|
||||
Manage the printing of an IPython/Jupyter Notebook progress bar widget.
|
||||
"""
|
||||
# Fallback to text bar if there's no total
|
||||
# DEPRECATED: replaced with an 'info' style bar
|
||||
# if not total:
|
||||
# return super(tqdm_notebook, tqdm_notebook).status_printer(file)
|
||||
|
||||
# fp = file
|
||||
|
||||
# Prepare IPython progress bar
|
||||
if IProgress is None: # #187 #451 #558 #872
|
||||
raise ImportError(WARN_NOIPYW)
|
||||
if total:
|
||||
pbar = IProgress(min=0, max=total)
|
||||
else: # No total? Show info style bar with no progress tqdm status
|
||||
pbar = IProgress(min=0, max=1)
|
||||
pbar.value = 1
|
||||
pbar.bar_style = 'info'
|
||||
if ncols is None:
|
||||
pbar.layout.width = "20px"
|
||||
|
||||
ltext = HTML()
|
||||
rtext = HTML()
|
||||
if desc:
|
||||
ltext.value = desc
|
||||
container = TqdmHBox(children=[ltext, pbar, rtext])
|
||||
# Prepare layout
|
||||
if ncols is not None: # use default style of ipywidgets
|
||||
# ncols could be 100, "100px", "100%"
|
||||
ncols = str(ncols) # ipywidgets only accepts string
|
||||
try:
|
||||
if int(ncols) > 0: # isnumeric and positive
|
||||
ncols += 'px'
|
||||
except ValueError:
|
||||
pass
|
||||
pbar.layout.flex = '2'
|
||||
container.layout.width = ncols
|
||||
container.layout.display = 'inline-flex'
|
||||
container.layout.flex_flow = 'row wrap'
|
||||
|
||||
return container
|
||||
|
||||
def display(self, msg=None, pos=None,
|
||||
# additional signals
|
||||
close=False, bar_style=None, check_delay=True):
|
||||
# Note: contrary to native tqdm, msg='' does NOT clear bar
|
||||
# goal is to keep all infos if error happens so user knows
|
||||
# at which iteration the loop failed.
|
||||
|
||||
# Clear previous output (really necessary?)
|
||||
# clear_output(wait=1)
|
||||
|
||||
if not msg and not close:
|
||||
d = self.format_dict
|
||||
# remove {bar}
|
||||
d['bar_format'] = (d['bar_format'] or "{l_bar}<bar/>{r_bar}").replace(
|
||||
"{bar}", "<bar/>")
|
||||
msg = self.format_meter(**d)
|
||||
|
||||
ltext, pbar, rtext = self.container.children
|
||||
pbar.value = self.n
|
||||
|
||||
if msg:
|
||||
# html escape special characters (like '&')
|
||||
if '<bar/>' in msg:
|
||||
left, right = map(escape, re.split(r'\|?<bar/>\|?', msg, 1))
|
||||
else:
|
||||
left, right = '', escape(msg)
|
||||
|
||||
# Update description
|
||||
ltext.value = left
|
||||
# never clear the bar (signal: msg='')
|
||||
if right:
|
||||
rtext.value = right
|
||||
|
||||
# Change bar style
|
||||
if bar_style:
|
||||
# Hack-ish way to avoid the danger bar_style being overridden by
|
||||
# success because the bar gets closed after the error...
|
||||
if pbar.bar_style != 'danger' or bar_style != 'success':
|
||||
pbar.bar_style = bar_style
|
||||
|
||||
# Special signal to close the bar
|
||||
if close and pbar.bar_style != 'danger': # hide only if no error
|
||||
try:
|
||||
self.container.close()
|
||||
except AttributeError:
|
||||
self.container.visible = False
|
||||
self.container.layout.visibility = 'hidden' # IPYW>=8
|
||||
|
||||
if check_delay and self.delay > 0 and not self.displayed:
|
||||
display(self.container)
|
||||
self.displayed = True
|
||||
|
||||
@property
|
||||
def colour(self):
|
||||
if hasattr(self, 'container'):
|
||||
return self.container.children[-2].style.bar_color
|
||||
|
||||
@colour.setter
|
||||
def colour(self, bar_color):
|
||||
if hasattr(self, 'container'):
|
||||
self.container.children[-2].style.bar_color = bar_color
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
"""
|
||||
Supports the usual `tqdm.tqdm` parameters as well as those listed below.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
display : Whether to call `display(self.container)` immediately
|
||||
[default: True].
|
||||
"""
|
||||
kwargs = kwargs.copy()
|
||||
# Setup default output
|
||||
file_kwarg = kwargs.get('file', sys.stderr)
|
||||
if file_kwarg is sys.stderr or file_kwarg is None:
|
||||
kwargs['file'] = sys.stdout # avoid the red block in IPython
|
||||
|
||||
# Initialize parent class + avoid printing by using gui=True
|
||||
kwargs['gui'] = True
|
||||
# convert disable = None to False
|
||||
kwargs['disable'] = bool(kwargs.get('disable', False))
|
||||
colour = kwargs.pop('colour', None)
|
||||
display_here = kwargs.pop('display', True)
|
||||
super(tqdm_notebook, self).__init__(*args, **kwargs)
|
||||
if self.disable or not kwargs['gui']:
|
||||
self.disp = lambda *_, **__: None
|
||||
return
|
||||
|
||||
# Get bar width
|
||||
self.ncols = '100%' if self.dynamic_ncols else kwargs.get("ncols", None)
|
||||
|
||||
# Replace with IPython progress bar display (with correct total)
|
||||
unit_scale = 1 if self.unit_scale is True else self.unit_scale or 1
|
||||
total = self.total * unit_scale if self.total else self.total
|
||||
self.container = self.status_printer(self.fp, total, self.desc, self.ncols)
|
||||
self.container.pbar = proxy(self)
|
||||
self.displayed = False
|
||||
if display_here and self.delay <= 0:
|
||||
display(self.container)
|
||||
self.displayed = True
|
||||
self.disp = self.display
|
||||
self.colour = colour
|
||||
|
||||
# Print initial bar state
|
||||
if not self.disable:
|
||||
self.display(check_delay=False)
|
||||
|
||||
def __iter__(self):
|
||||
try:
|
||||
it = super(tqdm_notebook, self).__iter__()
|
||||
for obj in it:
|
||||
# return super(tqdm...) will not catch exception
|
||||
yield obj
|
||||
# NB: except ... [ as ...] breaks IPython async KeyboardInterrupt
|
||||
except: # NOQA
|
||||
self.disp(bar_style='danger')
|
||||
raise
|
||||
# NB: don't `finally: close()`
|
||||
# since this could be a shared bar which the user will `reset()`
|
||||
|
||||
def update(self, n=1):
|
||||
try:
|
||||
return super(tqdm_notebook, self).update(n=n)
|
||||
# NB: except ... [ as ...] breaks IPython async KeyboardInterrupt
|
||||
except: # NOQA
|
||||
# cannot catch KeyboardInterrupt when using manual tqdm
|
||||
# as the interrupt will most likely happen on another statement
|
||||
self.disp(bar_style='danger')
|
||||
raise
|
||||
# NB: don't `finally: close()`
|
||||
# since this could be a shared bar which the user will `reset()`
|
||||
|
||||
def close(self):
|
||||
if self.disable:
|
||||
return
|
||||
super(tqdm_notebook, self).close()
|
||||
# Try to detect if there was an error or KeyboardInterrupt
|
||||
# in manual mode: if n < total, things probably got wrong
|
||||
if self.total and self.n < self.total:
|
||||
self.disp(bar_style='danger', check_delay=False)
|
||||
else:
|
||||
if self.leave:
|
||||
self.disp(bar_style='success', check_delay=False)
|
||||
else:
|
||||
self.disp(close=True, check_delay=False)
|
||||
|
||||
def clear(self, *_, **__):
|
||||
pass
|
||||
|
||||
def reset(self, total=None):
|
||||
"""
|
||||
Resets to 0 iterations for repeated use.
|
||||
|
||||
Consider combining with `leave=True`.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
total : int or float, optional. Total to use for the new bar.
|
||||
"""
|
||||
if self.disable:
|
||||
return super(tqdm_notebook, self).reset(total=total)
|
||||
_, pbar, _ = self.container.children
|
||||
pbar.bar_style = ''
|
||||
if total is not None:
|
||||
pbar.max = total
|
||||
if not self.total and self.ncols is None: # no longer unknown total
|
||||
pbar.layout.width = None # reset width
|
||||
return super(tqdm_notebook, self).reset(total=total)
|
||||
|
||||
|
||||
def tnrange(*args, **kwargs):
|
||||
"""
|
||||
A shortcut for `tqdm.notebook.tqdm(xrange(*args), **kwargs)`.
|
||||
On Python3+, `range` is used instead of `xrange`.
|
||||
"""
|
||||
return tqdm_notebook(_range(*args), **kwargs)
|
||||
|
||||
|
||||
# Aliases
|
||||
tqdm = tqdm_notebook
|
||||
trange = tnrange
|
156
tqdm/rich.py
Normal file
156
tqdm/rich.py
Normal file
|
@ -0,0 +1,156 @@
|
|||
"""
|
||||
`rich.progress` decorator for iterators.
|
||||
|
||||
Usage:
|
||||
>>> from tqdm.rich import trange, tqdm
|
||||
>>> for i in trange(10):
|
||||
... ...
|
||||
"""
|
||||
from __future__ import absolute_import
|
||||
|
||||
from warnings import warn
|
||||
|
||||
from rich.progress import (
|
||||
BarColumn, Progress, ProgressColumn, Text, TimeElapsedColumn, TimeRemainingColumn, filesize)
|
||||
|
||||
from .std import TqdmExperimentalWarning
|
||||
from .std import tqdm as std_tqdm
|
||||
from .utils import _range
|
||||
|
||||
__author__ = {"github.com/": ["casperdcl"]}
|
||||
__all__ = ['tqdm_rich', 'trrange', 'tqdm', 'trange']
|
||||
|
||||
|
||||
class FractionColumn(ProgressColumn):
|
||||
"""Renders completed/total, e.g. '0.5/2.3 G'."""
|
||||
def __init__(self, unit_scale=False, unit_divisor=1000):
|
||||
self.unit_scale = unit_scale
|
||||
self.unit_divisor = unit_divisor
|
||||
super().__init__()
|
||||
|
||||
def render(self, task):
|
||||
"""Calculate common unit for completed and total."""
|
||||
completed = int(task.completed)
|
||||
total = int(task.total)
|
||||
if self.unit_scale:
|
||||
unit, suffix = filesize.pick_unit_and_suffix(
|
||||
total,
|
||||
["", "K", "M", "G", "T", "P", "E", "Z", "Y"],
|
||||
self.unit_divisor,
|
||||
)
|
||||
else:
|
||||
unit, suffix = filesize.pick_unit_and_suffix(total, [""], 1)
|
||||
precision = 0 if unit == 1 else 1
|
||||
return Text(
|
||||
f"{completed/unit:,.{precision}f}/{total/unit:,.{precision}f} {suffix}",
|
||||
style="progress.download")
|
||||
|
||||
|
||||
class RateColumn(ProgressColumn):
|
||||
"""Renders human readable transfer speed."""
|
||||
def __init__(self, unit="", unit_scale=False, unit_divisor=1000):
|
||||
self.unit = unit
|
||||
self.unit_scale = unit_scale
|
||||
self.unit_divisor = unit_divisor
|
||||
super().__init__()
|
||||
|
||||
def render(self, task):
|
||||
"""Show data transfer speed."""
|
||||
speed = task.speed
|
||||
if speed is None:
|
||||
return Text(f"? {self.unit}/s", style="progress.data.speed")
|
||||
if self.unit_scale:
|
||||
unit, suffix = filesize.pick_unit_and_suffix(
|
||||
speed,
|
||||
["", "K", "M", "G", "T", "P", "E", "Z", "Y"],
|
||||
self.unit_divisor,
|
||||
)
|
||||
else:
|
||||
unit, suffix = filesize.pick_unit_and_suffix(speed, [""], 1)
|
||||
precision = 0 if unit == 1 else 1
|
||||
return Text(f"{speed/unit:,.{precision}f} {suffix}{self.unit}/s",
|
||||
style="progress.data.speed")
|
||||
|
||||
|
||||
class tqdm_rich(std_tqdm): # pragma: no cover
|
||||
"""Experimental rich.progress GUI version of tqdm!"""
|
||||
# TODO: @classmethod: write()?
|
||||
def __init__(self, *args, **kwargs):
|
||||
"""
|
||||
This class accepts the following parameters *in addition* to
|
||||
the parameters accepted by `tqdm`.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
progress : tuple, optional
|
||||
arguments for `rich.progress.Progress()`.
|
||||
options : dict, optional
|
||||
keyword arguments for `rich.progress.Progress()`.
|
||||
"""
|
||||
kwargs = kwargs.copy()
|
||||
kwargs['gui'] = True
|
||||
# convert disable = None to False
|
||||
kwargs['disable'] = bool(kwargs.get('disable', False))
|
||||
progress = kwargs.pop('progress', None)
|
||||
options = kwargs.pop('options', {}).copy()
|
||||
super(tqdm_rich, self).__init__(*args, **kwargs)
|
||||
|
||||
if self.disable:
|
||||
return
|
||||
|
||||
warn("rich is experimental/alpha", TqdmExperimentalWarning, stacklevel=2)
|
||||
d = self.format_dict
|
||||
if progress is None:
|
||||
progress = (
|
||||
"[progress.description]{task.description}"
|
||||
"[progress.percentage]{task.percentage:>4.0f}%",
|
||||
BarColumn(bar_width=None),
|
||||
FractionColumn(
|
||||
unit_scale=d['unit_scale'], unit_divisor=d['unit_divisor']),
|
||||
"[", TimeElapsedColumn(), "<", TimeRemainingColumn(),
|
||||
",", RateColumn(unit=d['unit'], unit_scale=d['unit_scale'],
|
||||
unit_divisor=d['unit_divisor']), "]"
|
||||
)
|
||||
options.setdefault('transient', not self.leave)
|
||||
self._prog = Progress(*progress, **options)
|
||||
self._prog.__enter__()
|
||||
self._task_id = self._prog.add_task(self.desc or "", **d)
|
||||
|
||||
def close(self):
|
||||
if self.disable:
|
||||
return
|
||||
super(tqdm_rich, self).close()
|
||||
self._prog.__exit__(None, None, None)
|
||||
|
||||
def clear(self, *_, **__):
|
||||
pass
|
||||
|
||||
def display(self, *_, **__):
|
||||
if not hasattr(self, '_prog'):
|
||||
return
|
||||
self._prog.update(self._task_id, completed=self.n, description=self.desc)
|
||||
|
||||
def reset(self, total=None):
|
||||
"""
|
||||
Resets to 0 iterations for repeated use.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
total : int or float, optional. Total to use for the new bar.
|
||||
"""
|
||||
if hasattr(self, '_prog'):
|
||||
self._prog.reset(total=total)
|
||||
super(tqdm_rich, self).reset(total=total)
|
||||
|
||||
|
||||
def trrange(*args, **kwargs):
|
||||
"""
|
||||
A shortcut for `tqdm.rich.tqdm(xrange(*args), **kwargs)`.
|
||||
On Python3+, `range` is used instead of `xrange`.
|
||||
"""
|
||||
return tqdm_rich(_range(*args), **kwargs)
|
||||
|
||||
|
||||
# Aliases
|
||||
tqdm = tqdm_rich
|
||||
trange = trrange
|
1541
tqdm/std.py
Normal file
1541
tqdm/std.py
Normal file
File diff suppressed because it is too large
Load diff
207
tqdm/tk.py
Normal file
207
tqdm/tk.py
Normal file
|
@ -0,0 +1,207 @@
|
|||
"""
|
||||
Tkinter GUI progressbar decorator for iterators.
|
||||
|
||||
Usage:
|
||||
>>> from tqdm.tk import trange, tqdm
|
||||
>>> for i in trange(10):
|
||||
... ...
|
||||
"""
|
||||
from __future__ import absolute_import, division
|
||||
|
||||
import re
|
||||
import sys
|
||||
from warnings import warn
|
||||
|
||||
try:
|
||||
import tkinter
|
||||
import tkinter.ttk as ttk
|
||||
except ImportError:
|
||||
import Tkinter as tkinter
|
||||
import ttk as ttk
|
||||
|
||||
from .std import TqdmExperimentalWarning, TqdmWarning
|
||||
from .std import tqdm as std_tqdm
|
||||
from .utils import _range
|
||||
|
||||
__author__ = {"github.com/": ["richardsheridan", "casperdcl"]}
|
||||
__all__ = ['tqdm_tk', 'ttkrange', 'tqdm', 'trange']
|
||||
|
||||
|
||||
class tqdm_tk(std_tqdm): # pragma: no cover
|
||||
"""
|
||||
Experimental Tkinter GUI version of tqdm!
|
||||
|
||||
Note: Window interactivity suffers if `tqdm_tk` is not running within
|
||||
a Tkinter mainloop and values are generated infrequently. In this case,
|
||||
consider calling `tqdm_tk.refresh()` frequently in the Tk thread.
|
||||
"""
|
||||
|
||||
# TODO: @classmethod: write()?
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
"""
|
||||
This class accepts the following parameters *in addition* to
|
||||
the parameters accepted by `tqdm`.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
grab : bool, optional
|
||||
Grab the input across all windows of the process.
|
||||
tk_parent : `tkinter.Wm`, optional
|
||||
Parent Tk window.
|
||||
cancel_callback : Callable, optional
|
||||
Create a cancel button and set `cancel_callback` to be called
|
||||
when the cancel or window close button is clicked.
|
||||
"""
|
||||
kwargs = kwargs.copy()
|
||||
kwargs['gui'] = True
|
||||
# convert disable = None to False
|
||||
kwargs['disable'] = bool(kwargs.get('disable', False))
|
||||
self._warn_leave = 'leave' in kwargs
|
||||
grab = kwargs.pop('grab', False)
|
||||
tk_parent = kwargs.pop('tk_parent', None)
|
||||
self._cancel_callback = kwargs.pop('cancel_callback', None)
|
||||
super(tqdm_tk, self).__init__(*args, **kwargs)
|
||||
|
||||
if self.disable:
|
||||
return
|
||||
|
||||
if tk_parent is None: # Discover parent widget
|
||||
try:
|
||||
tk_parent = tkinter._default_root
|
||||
except AttributeError:
|
||||
raise AttributeError(
|
||||
"`tk_parent` required when using `tkinter.NoDefaultRoot()`")
|
||||
if tk_parent is None: # use new default root window as display
|
||||
self._tk_window = tkinter.Tk()
|
||||
else: # some other windows already exist
|
||||
self._tk_window = tkinter.Toplevel()
|
||||
else:
|
||||
self._tk_window = tkinter.Toplevel(tk_parent)
|
||||
|
||||
warn("GUI is experimental/alpha", TqdmExperimentalWarning, stacklevel=2)
|
||||
self._tk_dispatching = self._tk_dispatching_helper()
|
||||
|
||||
self._tk_window.protocol("WM_DELETE_WINDOW", self.cancel)
|
||||
self._tk_window.wm_title(self.desc)
|
||||
self._tk_window.wm_attributes("-topmost", 1)
|
||||
self._tk_window.after(0, lambda: self._tk_window.wm_attributes("-topmost", 0))
|
||||
self._tk_n_var = tkinter.DoubleVar(self._tk_window, value=0)
|
||||
self._tk_text_var = tkinter.StringVar(self._tk_window)
|
||||
pbar_frame = ttk.Frame(self._tk_window, padding=5)
|
||||
pbar_frame.pack()
|
||||
_tk_label = ttk.Label(pbar_frame, textvariable=self._tk_text_var,
|
||||
wraplength=600, anchor="center", justify="center")
|
||||
_tk_label.pack()
|
||||
self._tk_pbar = ttk.Progressbar(
|
||||
pbar_frame, variable=self._tk_n_var, length=450)
|
||||
if self.total is not None:
|
||||
self._tk_pbar.configure(maximum=self.total)
|
||||
else:
|
||||
self._tk_pbar.configure(mode="indeterminate")
|
||||
self._tk_pbar.pack()
|
||||
if self._cancel_callback is not None:
|
||||
_tk_button = ttk.Button(pbar_frame, text="Cancel", command=self.cancel)
|
||||
_tk_button.pack()
|
||||
if grab:
|
||||
self._tk_window.grab_set()
|
||||
|
||||
def close(self):
|
||||
if self.disable:
|
||||
return
|
||||
|
||||
self.disable = True
|
||||
|
||||
with self.get_lock():
|
||||
self._instances.remove(self)
|
||||
|
||||
def _close():
|
||||
self._tk_window.after('idle', self._tk_window.destroy)
|
||||
if not self._tk_dispatching:
|
||||
self._tk_window.update()
|
||||
|
||||
self._tk_window.protocol("WM_DELETE_WINDOW", _close)
|
||||
|
||||
# if leave is set but we are self-dispatching, the left window is
|
||||
# totally unresponsive unless the user manually dispatches
|
||||
if not self.leave:
|
||||
_close()
|
||||
elif not self._tk_dispatching:
|
||||
if self._warn_leave:
|
||||
warn("leave flag ignored if not in tkinter mainloop",
|
||||
TqdmWarning, stacklevel=2)
|
||||
_close()
|
||||
|
||||
def clear(self, *_, **__):
|
||||
pass
|
||||
|
||||
def display(self, *_, **__):
|
||||
self._tk_n_var.set(self.n)
|
||||
d = self.format_dict
|
||||
# remove {bar}
|
||||
d['bar_format'] = (d['bar_format'] or "{l_bar}<bar/>{r_bar}").replace(
|
||||
"{bar}", "<bar/>")
|
||||
msg = self.format_meter(**d)
|
||||
if '<bar/>' in msg:
|
||||
msg = "".join(re.split(r'\|?<bar/>\|?', msg, 1))
|
||||
self._tk_text_var.set(msg)
|
||||
if not self._tk_dispatching:
|
||||
self._tk_window.update()
|
||||
|
||||
def set_description(self, desc=None, refresh=True):
|
||||
self.set_description_str(desc, refresh)
|
||||
|
||||
def set_description_str(self, desc=None, refresh=True):
|
||||
self.desc = desc
|
||||
if not self.disable:
|
||||
self._tk_window.wm_title(desc)
|
||||
if refresh and not self._tk_dispatching:
|
||||
self._tk_window.update()
|
||||
|
||||
def cancel(self):
|
||||
"""
|
||||
`cancel_callback()` followed by `close()`
|
||||
when close/cancel buttons clicked.
|
||||
"""
|
||||
if self._cancel_callback is not None:
|
||||
self._cancel_callback()
|
||||
self.close()
|
||||
|
||||
def reset(self, total=None):
|
||||
"""
|
||||
Resets to 0 iterations for repeated use.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
total : int or float, optional. Total to use for the new bar.
|
||||
"""
|
||||
if hasattr(self, '_tk_pbar'):
|
||||
if total is None:
|
||||
self._tk_pbar.configure(maximum=100, mode="indeterminate")
|
||||
else:
|
||||
self._tk_pbar.configure(maximum=total, mode="determinate")
|
||||
super(tqdm_tk, self).reset(total=total)
|
||||
|
||||
@staticmethod
|
||||
def _tk_dispatching_helper():
|
||||
"""determine if Tkinter mainloop is dispatching events"""
|
||||
codes = {tkinter.mainloop.__code__, tkinter.Misc.mainloop.__code__}
|
||||
for frame in sys._current_frames().values():
|
||||
while frame:
|
||||
if frame.f_code in codes:
|
||||
return True
|
||||
frame = frame.f_back
|
||||
return False
|
||||
|
||||
|
||||
def ttkrange(*args, **kwargs):
|
||||
"""
|
||||
A shortcut for `tqdm.tk.tqdm(xrange(*args), **kwargs)`.
|
||||
On Python3+, `range` is used instead of `xrange`.
|
||||
"""
|
||||
return tqdm_tk(_range(*args), **kwargs)
|
||||
|
||||
|
||||
# Aliases
|
||||
tqdm = tqdm_tk
|
||||
trange = ttkrange
|
316
tqdm/tqdm.1
Normal file
316
tqdm/tqdm.1
Normal file
|
@ -0,0 +1,316 @@
|
|||
.\" Automatically generated by Pandoc 1.19.2
|
||||
.\"
|
||||
.TH "TQDM" "1" "2015\-2021" "tqdm User Manuals" ""
|
||||
.hy
|
||||
.SH NAME
|
||||
.PP
|
||||
tqdm \- fast, extensible progress bar for Python and CLI
|
||||
.SH SYNOPSIS
|
||||
.PP
|
||||
tqdm [\f[I]options\f[]]
|
||||
.SH DESCRIPTION
|
||||
.PP
|
||||
See <https://github.com/tqdm/tqdm>.
|
||||
Can be used as a pipe:
|
||||
.IP
|
||||
.nf
|
||||
\f[C]
|
||||
$\ #\ count\ lines\ of\ code
|
||||
$\ cat\ *.py\ |\ tqdm\ |\ wc\ \-l
|
||||
327it\ [00:00,\ 981773.38it/s]
|
||||
327
|
||||
|
||||
$\ #\ find\ all\ files
|
||||
$\ find\ .\ \-name\ "*.py"\ |\ tqdm\ |\ wc\ \-l
|
||||
432it\ [00:00,\ 833842.30it/s]
|
||||
432
|
||||
|
||||
#\ ...\ and\ more\ info
|
||||
$\ find\ .\ \-name\ \[aq]*.py\[aq]\ \-exec\ wc\ \-l\ \\{}\ \\;\ \\
|
||||
\ \ |\ tqdm\ \-\-total\ 432\ \-\-unit\ files\ \-\-desc\ counting\ \\
|
||||
\ \ |\ awk\ \[aq]{\ sum\ +=\ $1\ };\ END\ {\ print\ sum\ }\[aq]
|
||||
counting:\ 100%|█████████|\ 432/432\ [00:00<00:00,\ 794361.83files/s]
|
||||
131998
|
||||
\f[]
|
||||
.fi
|
||||
.SH OPTIONS
|
||||
.TP
|
||||
.B \-h, \-\-help
|
||||
Print this help and exit.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-v, \-\-version
|
||||
Print version and exit.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-desc=\f[I]desc\f[]
|
||||
str, optional.
|
||||
Prefix for the progressbar.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-total=\f[I]total\f[]
|
||||
int or float, optional.
|
||||
The number of expected iterations.
|
||||
If unspecified, len(iterable) is used if possible.
|
||||
If float("inf") or as a last resort, only basic progress statistics are
|
||||
displayed (no ETA, no progressbar).
|
||||
If \f[C]gui\f[] is True and this parameter needs subsequent updating,
|
||||
specify an initial arbitrary large positive number, e.g.
|
||||
9e9.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-leave
|
||||
bool, optional.
|
||||
If [default: True], keeps all traces of the progressbar upon termination
|
||||
of iteration.
|
||||
If \f[C]None\f[], will leave only if \f[C]position\f[] is \f[C]0\f[].
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-ncols=\f[I]ncols\f[]
|
||||
int, optional.
|
||||
The width of the entire output message.
|
||||
If specified, dynamically resizes the progressbar to stay within this
|
||||
bound.
|
||||
If unspecified, attempts to use environment width.
|
||||
The fallback is a meter width of 10 and no limit for the counter and
|
||||
statistics.
|
||||
If 0, will not print any meter (only stats).
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-mininterval=\f[I]mininterval\f[]
|
||||
float, optional.
|
||||
Minimum progress display update interval [default: 0.1] seconds.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-maxinterval=\f[I]maxinterval\f[]
|
||||
float, optional.
|
||||
Maximum progress display update interval [default: 10] seconds.
|
||||
Automatically adjusts \f[C]miniters\f[] to correspond to
|
||||
\f[C]mininterval\f[] after long display update lag.
|
||||
Only works if \f[C]dynamic_miniters\f[] or monitor thread is enabled.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-miniters=\f[I]miniters\f[]
|
||||
int or float, optional.
|
||||
Minimum progress display update interval, in iterations.
|
||||
If 0 and \f[C]dynamic_miniters\f[], will automatically adjust to equal
|
||||
\f[C]mininterval\f[] (more CPU efficient, good for tight loops).
|
||||
If > 0, will skip display of specified number of iterations.
|
||||
Tweak this and \f[C]mininterval\f[] to get very efficient loops.
|
||||
If your progress is erratic with both fast and slow iterations (network,
|
||||
skipping items, etc) you should set miniters=1.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-ascii=\f[I]ascii\f[]
|
||||
bool or str, optional.
|
||||
If unspecified or False, use unicode (smooth blocks) to fill the meter.
|
||||
The fallback is to use ASCII characters " 123456789#".
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-disable
|
||||
bool, optional.
|
||||
Whether to disable the entire progressbar wrapper [default: False].
|
||||
If set to None, disable on non\-TTY.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-unit=\f[I]unit\f[]
|
||||
str, optional.
|
||||
String that will be used to define the unit of each iteration [default:
|
||||
it].
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-unit\-scale=\f[I]unit_scale\f[]
|
||||
bool or int or float, optional.
|
||||
If 1 or True, the number of iterations will be reduced/scaled
|
||||
automatically and a metric prefix following the International System of
|
||||
Units standard will be added (kilo, mega, etc.) [default: False].
|
||||
If any other non\-zero number, will scale \f[C]total\f[] and \f[C]n\f[].
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-dynamic\-ncols
|
||||
bool, optional.
|
||||
If set, constantly alters \f[C]ncols\f[] and \f[C]nrows\f[] to the
|
||||
environment (allowing for window resizes) [default: False].
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-smoothing=\f[I]smoothing\f[]
|
||||
float, optional.
|
||||
Exponential moving average smoothing factor for speed estimates (ignored
|
||||
in GUI mode).
|
||||
Ranges from 0 (average speed) to 1 (current/instantaneous speed)
|
||||
[default: 0.3].
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-bar\-format=\f[I]bar_format\f[]
|
||||
str, optional.
|
||||
Specify a custom bar string formatting.
|
||||
May impact performance.
|
||||
[default: \[aq]{l_bar}{bar}{r_bar}\[aq]], where l_bar=\[aq]{desc}:
|
||||
{percentage:3.0f}%|\[aq] and r_bar=\[aq]| {n_fmt}/{total_fmt}
|
||||
[{elapsed}<{remaining}, \[aq] \[aq]{rate_fmt}{postfix}]\[aq] Possible
|
||||
vars: l_bar, bar, r_bar, n, n_fmt, total, total_fmt, percentage,
|
||||
elapsed, elapsed_s, ncols, nrows, desc, unit, rate, rate_fmt,
|
||||
rate_noinv, rate_noinv_fmt, rate_inv, rate_inv_fmt, postfix,
|
||||
unit_divisor, remaining, remaining_s, eta.
|
||||
Note that a trailing ": " is automatically removed after {desc} if the
|
||||
latter is empty.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-initial=\f[I]initial\f[]
|
||||
int or float, optional.
|
||||
The initial counter value.
|
||||
Useful when restarting a progress bar [default: 0].
|
||||
If using float, consider specifying \f[C]{n:.3f}\f[] or similar in
|
||||
\f[C]bar_format\f[], or specifying \f[C]unit_scale\f[].
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-position=\f[I]position\f[]
|
||||
int, optional.
|
||||
Specify the line offset to print this bar (starting from 0) Automatic if
|
||||
unspecified.
|
||||
Useful to manage multiple bars at once (eg, from threads).
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-postfix=\f[I]postfix\f[]
|
||||
dict or *, optional.
|
||||
Specify additional stats to display at the end of the bar.
|
||||
Calls \f[C]set_postfix(**postfix)\f[] if possible (dict).
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-unit\-divisor=\f[I]unit_divisor\f[]
|
||||
float, optional.
|
||||
[default: 1000], ignored unless \f[C]unit_scale\f[] is True.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-write\-bytes
|
||||
bool, optional.
|
||||
If (default: None) and \f[C]file\f[] is unspecified, bytes will be
|
||||
written in Python 2.
|
||||
If \f[C]True\f[] will also write bytes.
|
||||
In all other cases will default to unicode.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-lock\-args=\f[I]lock_args\f[]
|
||||
tuple, optional.
|
||||
Passed to \f[C]refresh\f[] for intermediate output (initialisation,
|
||||
iterating, and updating).
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-nrows=\f[I]nrows\f[]
|
||||
int, optional.
|
||||
The screen height.
|
||||
If specified, hides nested bars outside this bound.
|
||||
If unspecified, attempts to use environment height.
|
||||
The fallback is 20.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-colour=\f[I]colour\f[]
|
||||
str, optional.
|
||||
Bar colour (e.g.
|
||||
\[aq]green\[aq], \[aq]#00ff00\[aq]).
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-delay=\f[I]delay\f[]
|
||||
float, optional.
|
||||
Don\[aq]t display until [default: 0] seconds have elapsed.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-delim=\f[I]delim\f[]
|
||||
chr, optional.
|
||||
Delimiting character [default: \[aq]\\n\[aq]].
|
||||
Use \[aq]\\0\[aq] for null.
|
||||
N.B.: on Windows systems, Python converts \[aq]\\n\[aq] to
|
||||
\[aq]\\r\\n\[aq].
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-buf\-size=\f[I]buf_size\f[]
|
||||
int, optional.
|
||||
String buffer size in bytes [default: 256] used when \f[C]delim\f[] is
|
||||
specified.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-bytes
|
||||
bool, optional.
|
||||
If true, will count bytes, ignore \f[C]delim\f[], and default
|
||||
\f[C]unit_scale\f[] to True, \f[C]unit_divisor\f[] to 1024, and
|
||||
\f[C]unit\f[] to \[aq]B\[aq].
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-tee
|
||||
bool, optional.
|
||||
If true, passes \f[C]stdin\f[] to both \f[C]stderr\f[] and
|
||||
\f[C]stdout\f[].
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-update
|
||||
bool, optional.
|
||||
If true, will treat input as newly elapsed iterations, i.e.
|
||||
numbers to pass to \f[C]update()\f[].
|
||||
Note that this is slow (~2e5 it/s) since every input must be decoded as
|
||||
a number.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-update\-to
|
||||
bool, optional.
|
||||
If true, will treat input as total elapsed iterations, i.e.
|
||||
numbers to assign to \f[C]self.n\f[].
|
||||
Note that this is slow (~2e5 it/s) since every input must be decoded as
|
||||
a number.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-null
|
||||
bool, optional.
|
||||
If true, will discard input (no stdout).
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-manpath=\f[I]manpath\f[]
|
||||
str, optional.
|
||||
Directory in which to install tqdm man pages.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-comppath=\f[I]comppath\f[]
|
||||
str, optional.
|
||||
Directory in which to place tqdm completion.
|
||||
.RS
|
||||
.RE
|
||||
.TP
|
||||
.B \-\-log=\f[I]log\f[]
|
||||
str, optional.
|
||||
CRITICAL|FATAL|ERROR|WARN(ING)|[default: \[aq]INFO\[aq]]|DEBUG|NOTSET.
|
||||
.RS
|
||||
.RE
|
||||
.SH AUTHORS
|
||||
tqdm developers <https://github.com/tqdm>.
|
354
tqdm/utils.py
Normal file
354
tqdm/utils.py
Normal file
|
@ -0,0 +1,354 @@
|
|||
"""
|
||||
General helpers required for `tqdm.std`.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from functools import wraps
|
||||
from warnings import warn
|
||||
from weakref import proxy
|
||||
|
||||
# py2/3 compat
|
||||
try:
|
||||
_range = xrange
|
||||
except NameError:
|
||||
_range = range
|
||||
|
||||
try:
|
||||
_unich = unichr
|
||||
except NameError:
|
||||
_unich = chr
|
||||
|
||||
try:
|
||||
_unicode = unicode
|
||||
except NameError:
|
||||
_unicode = str
|
||||
|
||||
try:
|
||||
_basestring = basestring
|
||||
except NameError:
|
||||
_basestring = str
|
||||
|
||||
CUR_OS = sys.platform
|
||||
IS_WIN = any(CUR_OS.startswith(i) for i in ['win32', 'cygwin'])
|
||||
IS_NIX = any(CUR_OS.startswith(i) for i in ['aix', 'linux', 'darwin'])
|
||||
RE_ANSI = re.compile(r"\x1b\[[;\d]*[A-Za-z]")
|
||||
|
||||
try:
|
||||
if IS_WIN:
|
||||
import colorama
|
||||
else:
|
||||
raise ImportError
|
||||
except ImportError:
|
||||
colorama = None
|
||||
else:
|
||||
try:
|
||||
colorama.init(strip=False)
|
||||
except TypeError:
|
||||
colorama.init()
|
||||
|
||||
|
||||
class FormatReplace(object):
|
||||
"""
|
||||
>>> a = FormatReplace('something')
|
||||
>>> "{:5d}".format(a)
|
||||
'something'
|
||||
""" # NOQA: P102
|
||||
def __init__(self, replace=''):
|
||||
self.replace = replace
|
||||
self.format_called = 0
|
||||
|
||||
def __format__(self, _):
|
||||
self.format_called += 1
|
||||
return self.replace
|
||||
|
||||
|
||||
class Comparable(object):
|
||||
"""Assumes child has self._comparable attr/@property"""
|
||||
def __lt__(self, other):
|
||||
return self._comparable < other._comparable
|
||||
|
||||
def __le__(self, other):
|
||||
return (self < other) or (self == other)
|
||||
|
||||
def __eq__(self, other):
|
||||
return self._comparable == other._comparable
|
||||
|
||||
def __ne__(self, other):
|
||||
return not self == other
|
||||
|
||||
def __gt__(self, other):
|
||||
return not self <= other
|
||||
|
||||
def __ge__(self, other):
|
||||
return not self < other
|
||||
|
||||
|
||||
class ObjectWrapper(object):
|
||||
def __getattr__(self, name):
|
||||
return getattr(self._wrapped, name)
|
||||
|
||||
def __setattr__(self, name, value):
|
||||
return setattr(self._wrapped, name, value)
|
||||
|
||||
def wrapper_getattr(self, name):
|
||||
"""Actual `self.getattr` rather than self._wrapped.getattr"""
|
||||
try:
|
||||
return object.__getattr__(self, name)
|
||||
except AttributeError: # py2
|
||||
return getattr(self, name)
|
||||
|
||||
def wrapper_setattr(self, name, value):
|
||||
"""Actual `self.setattr` rather than self._wrapped.setattr"""
|
||||
return object.__setattr__(self, name, value)
|
||||
|
||||
def __init__(self, wrapped):
|
||||
"""
|
||||
Thin wrapper around a given object
|
||||
"""
|
||||
self.wrapper_setattr('_wrapped', wrapped)
|
||||
|
||||
|
||||
class SimpleTextIOWrapper(ObjectWrapper):
|
||||
"""
|
||||
Change only `.write()` of the wrapped object by encoding the passed
|
||||
value and passing the result to the wrapped object's `.write()` method.
|
||||
"""
|
||||
# pylint: disable=too-few-public-methods
|
||||
def __init__(self, wrapped, encoding):
|
||||
super(SimpleTextIOWrapper, self).__init__(wrapped)
|
||||
self.wrapper_setattr('encoding', encoding)
|
||||
|
||||
def write(self, s):
|
||||
"""
|
||||
Encode `s` and pass to the wrapped object's `.write()` method.
|
||||
"""
|
||||
return self._wrapped.write(s.encode(self.wrapper_getattr('encoding')))
|
||||
|
||||
def __eq__(self, other):
|
||||
return self._wrapped == getattr(other, '_wrapped', other)
|
||||
|
||||
|
||||
class DisableOnWriteError(ObjectWrapper):
|
||||
"""
|
||||
Disable the given `tqdm_instance` upon `write()` or `flush()` errors.
|
||||
"""
|
||||
@staticmethod
|
||||
def disable_on_exception(tqdm_instance, func):
|
||||
"""
|
||||
Quietly set `tqdm_instance.miniters=inf` if `func` raises `errno=5`.
|
||||
"""
|
||||
tqdm_instance = proxy(tqdm_instance)
|
||||
|
||||
def inner(*args, **kwargs):
|
||||
try:
|
||||
return func(*args, **kwargs)
|
||||
except OSError as e:
|
||||
if e.errno != 5:
|
||||
raise
|
||||
try:
|
||||
tqdm_instance.miniters = float('inf')
|
||||
except ReferenceError:
|
||||
pass
|
||||
except ValueError as e:
|
||||
if 'closed' not in str(e):
|
||||
raise
|
||||
try:
|
||||
tqdm_instance.miniters = float('inf')
|
||||
except ReferenceError:
|
||||
pass
|
||||
return inner
|
||||
|
||||
def __init__(self, wrapped, tqdm_instance):
|
||||
super(DisableOnWriteError, self).__init__(wrapped)
|
||||
if hasattr(wrapped, 'write'):
|
||||
self.wrapper_setattr(
|
||||
'write', self.disable_on_exception(tqdm_instance, wrapped.write))
|
||||
if hasattr(wrapped, 'flush'):
|
||||
self.wrapper_setattr(
|
||||
'flush', self.disable_on_exception(tqdm_instance, wrapped.flush))
|
||||
|
||||
def __eq__(self, other):
|
||||
return self._wrapped == getattr(other, '_wrapped', other)
|
||||
|
||||
|
||||
class CallbackIOWrapper(ObjectWrapper):
|
||||
def __init__(self, callback, stream, method="read"):
|
||||
"""
|
||||
Wrap a given `file`-like object's `read()` or `write()` to report
|
||||
lengths to the given `callback`
|
||||
"""
|
||||
super(CallbackIOWrapper, self).__init__(stream)
|
||||
func = getattr(stream, method)
|
||||
if method == "write":
|
||||
@wraps(func)
|
||||
def write(data, *args, **kwargs):
|
||||
res = func(data, *args, **kwargs)
|
||||
callback(len(data))
|
||||
return res
|
||||
self.wrapper_setattr('write', write)
|
||||
elif method == "read":
|
||||
@wraps(func)
|
||||
def read(*args, **kwargs):
|
||||
data = func(*args, **kwargs)
|
||||
callback(len(data))
|
||||
return data
|
||||
self.wrapper_setattr('read', read)
|
||||
else:
|
||||
raise KeyError("Can only wrap read/write methods")
|
||||
|
||||
|
||||
def _is_utf(encoding):
|
||||
try:
|
||||
u'\u2588\u2589'.encode(encoding)
|
||||
except UnicodeEncodeError:
|
||||
return False
|
||||
except Exception:
|
||||
try:
|
||||
return encoding.lower().startswith('utf-') or ('U8' == encoding)
|
||||
except Exception:
|
||||
return False
|
||||
else:
|
||||
return True
|
||||
|
||||
|
||||
def _supports_unicode(fp):
|
||||
try:
|
||||
return _is_utf(fp.encoding)
|
||||
except AttributeError:
|
||||
return False
|
||||
|
||||
|
||||
def _is_ascii(s):
|
||||
if isinstance(s, str):
|
||||
for c in s:
|
||||
if ord(c) > 255:
|
||||
return False
|
||||
return True
|
||||
return _supports_unicode(s)
|
||||
|
||||
|
||||
def _screen_shape_wrapper(): # pragma: no cover
|
||||
"""
|
||||
Return a function which returns console dimensions (width, height).
|
||||
Supported: linux, osx, windows, cygwin.
|
||||
"""
|
||||
_screen_shape = None
|
||||
if IS_WIN:
|
||||
_screen_shape = _screen_shape_windows
|
||||
if _screen_shape is None:
|
||||
_screen_shape = _screen_shape_tput
|
||||
if IS_NIX:
|
||||
_screen_shape = _screen_shape_linux
|
||||
return _screen_shape
|
||||
|
||||
|
||||
def _screen_shape_windows(fp): # pragma: no cover
|
||||
try:
|
||||
import struct
|
||||
from ctypes import create_string_buffer, windll
|
||||
from sys import stdin, stdout
|
||||
|
||||
io_handle = -12 # assume stderr
|
||||
if fp == stdin:
|
||||
io_handle = -10
|
||||
elif fp == stdout:
|
||||
io_handle = -11
|
||||
|
||||
h = windll.kernel32.GetStdHandle(io_handle)
|
||||
csbi = create_string_buffer(22)
|
||||
res = windll.kernel32.GetConsoleScreenBufferInfo(h, csbi)
|
||||
if res:
|
||||
(_bufx, _bufy, _curx, _cury, _wattr, left, top, right, bottom,
|
||||
_maxx, _maxy) = struct.unpack("hhhhHhhhhhh", csbi.raw)
|
||||
return right - left, bottom - top # +1
|
||||
except Exception: # nosec
|
||||
pass
|
||||
return None, None
|
||||
|
||||
|
||||
def _screen_shape_tput(*_): # pragma: no cover
|
||||
"""cygwin xterm (windows)"""
|
||||
try:
|
||||
import shlex
|
||||
from subprocess import check_call # nosec
|
||||
return [int(check_call(shlex.split('tput ' + i))) - 1
|
||||
for i in ('cols', 'lines')]
|
||||
except Exception: # nosec
|
||||
pass
|
||||
return None, None
|
||||
|
||||
|
||||
def _screen_shape_linux(fp): # pragma: no cover
|
||||
|
||||
try:
|
||||
from array import array
|
||||
from fcntl import ioctl
|
||||
from termios import TIOCGWINSZ
|
||||
except ImportError:
|
||||
return None, None
|
||||
else:
|
||||
try:
|
||||
rows, cols = array('h', ioctl(fp, TIOCGWINSZ, '\0' * 8))[:2]
|
||||
return cols, rows
|
||||
except Exception:
|
||||
try:
|
||||
return [int(os.environ[i]) - 1 for i in ("COLUMNS", "LINES")]
|
||||
except (KeyError, ValueError):
|
||||
return None, None
|
||||
|
||||
|
||||
def _environ_cols_wrapper(): # pragma: no cover
|
||||
"""
|
||||
Return a function which returns console width.
|
||||
Supported: linux, osx, windows, cygwin.
|
||||
"""
|
||||
warn("Use `_screen_shape_wrapper()(file)[0]` instead of"
|
||||
" `_environ_cols_wrapper()(file)`", DeprecationWarning, stacklevel=2)
|
||||
shape = _screen_shape_wrapper()
|
||||
if not shape:
|
||||
return None
|
||||
|
||||
@wraps(shape)
|
||||
def inner(fp):
|
||||
return shape(fp)[0]
|
||||
|
||||
return inner
|
||||
|
||||
|
||||
def _term_move_up(): # pragma: no cover
|
||||
return '' if (os.name == 'nt') and (colorama is None) else '\x1b[A'
|
||||
|
||||
|
||||
try:
|
||||
# TODO consider using wcswidth third-party package for 0-width characters
|
||||
from unicodedata import east_asian_width
|
||||
except ImportError:
|
||||
_text_width = len
|
||||
else:
|
||||
def _text_width(s):
|
||||
return sum(2 if east_asian_width(ch) in 'FW' else 1 for ch in _unicode(s))
|
||||
|
||||
|
||||
def disp_len(data):
|
||||
"""
|
||||
Returns the real on-screen length of a string which may contain
|
||||
ANSI control codes and wide chars.
|
||||
"""
|
||||
return _text_width(RE_ANSI.sub('', data))
|
||||
|
||||
|
||||
def disp_trim(data, length):
|
||||
"""
|
||||
Trim a string which may contain ANSI control characters.
|
||||
"""
|
||||
if len(data) == disp_len(data):
|
||||
return data[:length]
|
||||
|
||||
ansi_present = bool(RE_ANSI.search(data))
|
||||
while disp_len(data) > length: # carefully delete one char at a time
|
||||
data = data[:-1]
|
||||
if ansi_present and bool(RE_ANSI.search(data)):
|
||||
# assume ANSI reset is required
|
||||
return data if data.endswith("\033[0m") else data + "\033[0m"
|
||||
return data
|
9
tqdm/version.py
Normal file
9
tqdm/version.py
Normal file
|
@ -0,0 +1,9 @@
|
|||
"""`tqdm` version detector. Precedence: installed dist, git, 'UNKNOWN'."""
|
||||
try:
|
||||
from ._dist_ver import __version__
|
||||
except ImportError:
|
||||
try:
|
||||
from setuptools_scm import get_version
|
||||
__version__ = get_version(root='..', relative_to=__file__)
|
||||
except (ImportError, LookupError):
|
||||
__version__ = "UNKNOWN"
|
Loading…
Add table
Reference in a new issue