diff --git a/CHANGELOG.md b/CHANGELOG.md index d3ba70c..602b9ee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,58 +1,56 @@ # Changelog +## Next version + +- [#46](https://github.com/sdss/sdsstools/pull/46) Added several small utilities: + - `Timer()` (already existed, but mentioning for completeness): a context manager to determine the duration of the task executed between `__enter__` and `__exit__`. + - `get_temporary_file_path()`: returns a valid temporary named path. + - `run_in_executor()`: simple wrapper around `asyncio` `run_in_executor` that allows to use `thread` or `process` pool executors and handles arguments and keyword arguments. + - `cancel_task()`: cancels a task, suppressing the `CancelledError`. + ## [1.5.5](https://github.com/sdss/sdsstools/compare/1.5.4...1.5.5) - 2023-12-10 - Relax `astropy` and `numpy` requirements. - ## [1.5.4](https://github.com/sdss/sdsstools/compare/1.5.3...1.5.4) - 2023-12-10 - Fix unpickling of `Configuration` instances. This only seems relevant when trying to pass a `Configuration` object to a `multiprocessing` callback. - ## [1.5.3](https://github.com/sdss/sdsstools/compare/1.5.2...1.5.3) - 2023-12-08 - Vendorise `pydl`'s `yanny` module which can be accessed as `sdsstools.yanny`. - ## [1.5.2](https://github.com/sdss/sdsstools/compare/1.5.1...1.5.2) - 2023-11-30 - Allow periods in `RecursiveDict` assignments but treat them as normal dictionary keys. - Allow retrieving keys with periods in a `RecursiveDict` if they match an existing key. - ## [1.5.1](https://github.com/sdss/sdsstools/compare/1.5.0...1.5.1) - 2023-11-30 -- Fixed an issue assigning dictionaries caused by returning nested dictionaries in a `RecursiveDict` and `RecursiveDict` instances. This has been fixed by doing the conversion at assignment time. A keyword argument, ``propagate_type``, allows to disable that behaviour. - +- Fixed an issue assigning dictionaries caused by returning nested dictionaries in a `RecursiveDict` and `RecursiveDict` instances. This has been fixed by doing the conversion at assignment time. A keyword argument, `propagate_type`, allows to disable that behaviour. ## [1.5.0](https://github.com/sdss/sdsstools/compare/1.4.1...1.5.0) - 2023-11-29 [yanked] - [#44](https://github.com/sdss/sdsstools/issues/44) If `RecursiveDict` or `Configuration` return a dictionary, the dictionary is itself a `RecursiveDict` or `Configuration` object. - [#45](https://github.com/sdss/sdsstools/issues/45) Add support for Python 3.12. - ## [1.4.0](https://github.com/sdss/sdsstools/compare/1.3.2...1.4.0) - 2023-11-16 - Add support for outputting log file in JSON format, via new options in `start_file_logger`. - ## [1.3.2](https://github.com/sdss/sdsstools/compare/1.3.1...1.3.2) - 2023-11-10 - Better typing for `RecursiveDict`. - ## [1.3.1](https://github.com/sdss/sdsstools/compare/1.3.0...1.3.1) - 2023-11-10 - Improved typing for `read_yaml_file`. - ## [1.3.0](https://github.com/sdss/sdsstools/compare/1.2.3...1.3.0) - 2023-11-09 - [#40](https://github.com/sdss/sdsstools/pr/40) Additional options to `start_file_logger` for the `TimedRotatingFileHandler`. - [#41](https://github.com/sdss/sdsstools/pr/41) `Configuration` now supports a recursive getter (e.g., `config['a.b.c']`). - ## [1.2.3](https://github.com/sdss/sdsstools/compare/1.2.2...1.2.3) - 2023-08-25 - Store `rich` console instance when passing it to handler. @@ -60,139 +58,114 @@ - Handle exceptions and log them even within IPython. - Add option to not capture exceptions in logger. - ## [1.2.2](https://github.com/sdss/sdsstools/compare/1.2.1...1.2.2) - 2023-08-25 - Ensure that logging to file happens with rich handler. - ## [1.2.1](https://github.com/sdss/sdsstools/compare/1.2.0...1.2.1) - 2023-08-25 - Improve logging with `rich` and do not use custon exception hook with `RichHandler`. - Lint using `ruff`. - ## [1.2.0](https://github.com/sdss/sdsstools/compare/1.1.0...1.2.0) - 2023-08-21 - [#36](https://github.com/sdss/sdsstools/issues/36) Add the option to use a `RichHandler` handler for console logging instead of the default `StreamHandler`. This can be enabled by passing `use_rich_handler` to `get_logger()`. For now this is not the default but it may become so in a future version. - Update `invoke` to 2.1.3. - ## [1.1.0](https://github.com/sdss/sdsstools/compare/1.0.2...1.1.0) - 2023-06-18 - [#35](https://github.com/sdss/sdsstools/issues/35) Add support to reload a `Configuration`. If the configuration is based on files on disk and those have changed, the files will be read again. - Add the option to print the time of a log message in `StreamFormatter` by doing `log.sh.formatter.print_time = True`. - ## [1.0.2](https://github.com/sdss/sdsstools/compare/1.0.1...1.0.2) - 2022-12-27 - Do not fail CLI if the root of the package cannot be found. - ## [1.0.1](https://github.com/sdss/sdsstools/compare/1.0.0...1.0.1) - 2022-12-27 - Monkeypatch `invoke` to work with Python 3.10+ (see [this issue](https://github.com/pyinvoke/invoke/issues/833#issuecomment-1293148106)). - ## [1.0.0](https://github.com/sdss/sdsstools/compare/0.5.4...1.0.0) - 2022-12-27 - **Python 3.6 and 3.7 are not supported anymore.** - Removed the vendorised copy of `releases`. `releases` is not maintained anymore and should probably not be used. If you want to continue using it, use `0.5.4` or the normal installation of `releases`. - Full support for Python 3.11. - ## [0.5.4](https://github.com/sdss/sdsstools/compare/0.5.3...0.5.4) - 2022-12-27 - Add `.pytest_cache` and `.nox` to directories removed by `sdss clean`. - Add a new task `sdss sjd` that returns the current SJD. - ## [0.5.3](https://github.com/sdss/sdsstools/compare/0.5.2...0.5.3) - 2022-09-08 - Fix annoying extra printout in `get_sjd()`. - ## [0.5.2](https://github.com/sdss/sdsstools/compare/0.5.1...0.5.2) - 2022-06-05 - Fix `get_sjd()` which used the current time instead of UTC time, and had an additional `int()` which screwed up the calculation. - ## [0.5.1](https://github.com/sdss/sdsstools/compare/0.5.0...0.5.1) - 2022-06-04 - Fix the case of YAML files with multiple environment variable substitution or in which the environment variable was not at the beginning of the string. - ## [0.5.0](https://github.com/sdss/sdsstools/compare/0.4.15...0.5.0) - 2022-04-25 - Added a `time` module with a `get_sjd()` function that returns the SDSS-style Modified Julian Day for an observatory. - ## [0.4.15](https://github.com/sdss/sdsstools/compare/0.4.14...0.4.15) - 2022-04-21 - Feature [#30](https://github.com/sdss/sdsstools/issues/30): Pass optional `fmt` on to stream formatter. - Remove use of future typing annotations to continue supporting Python 3.6. - Add GitHub Actions testing for Python 3.9 and 3.10. - ## [0.4.14](https://github.com/sdss/sdsstools/compare/0.4.13...0.4.14) - 2022-02-22 - Relaxed `packaging` to prevent issues with other packages. - ## [0.4.13](https://github.com/sdss/sdsstools/compare/0.4.12...0.4.13) - 2021-08-14 - Make `daemonocle` a required dependency. - Add `daemonize` CLI script. - ## [0.4.12](https://github.com/sdss/sdsstools/compare/0.4.11...0.4.12) - 2021-08-13 - Do not redirect output to log in `DaemonGroup` if we are running in debug mode. - ## [0.4.11](https://github.com/sdss/sdsstools/compare/0.4.10...0.4.11) - 2021-08-12 - Support: allow `pid_file` or `pidfile` in `DeamonGroup` (`pid_file` is recommended since it's the one supported by `daemonocle`). - Allow to set a global log file in `DaemonGroup` and implement log rotation. - ## [0.4.10](https://github.com/sdss/sdsstools/compare/0.4.9...0.4.10) - 2021-03-24 - Bug: force use of real path for the logger file handler path. - Feature [#28](https://github.com/sdss/sdsstools/issues/28): report base config file in `Configuration`. Clear the `Configuration` internal dictionary when `load` is called. - ## [0.4.9](https://github.com/sdss/sdsstools/compare/0.4.8...0.4.9) - 2021-02-24 - Support: relax dependency on `importlib_metadata`. - ## [0.4.8](https://github.com/sdss/sdsstools/compare/0.4.7...0.4.8) - 2021-02-13 - Feature [#26](https://github.com/sdss/sdsstools/issues/26): allow rollover of file logger on `start_file_logger`. - Support: format using `black` and add type hints. - ## [0.4.7](https://github.com/sdss/sdsstools/compare/0.4.6...0.4.7) - 2021-02-12 - Bug [#24](https://github.com/sdss/sdsstools/issues/24): update dependencies so that `importlib_metadata` is installed for Python<=3.7 (it was Python<3.7). - ## [0.4.6](https://github.com/sdss/sdsstools/compare/0.4.5...0.4.6) - 2021-01-21 - Do not remove `name` from the parameters in `DaemonGroup` that are passed to `click.Group`. This made `DaemonGroup` fail with `daemonocle>=1.1.0` in which the signature of `Daemon` includes a new parameter `name`. - ## [0.4.5](https://github.com/sdss/sdsstools/compare/0.4.4...0.4.5) - 2020-11-10 - Bug [#22](https://github.com/sdss/sdsstools/issues/22): fix `read_yaml` for empty YAML files. - ## [0.4.4](https://github.com/sdss/sdsstools/compare/0.4.3...0.4.4) - 2020-11-05 - Support [#20](https://github.com/sdss/sdsstools/issues/20): discontinue use of `pkg_resources` due to its long import time. Move `sdsstools._vendor.releases` to `sdsstools.releases` to avoid having to load it on import. - ## [0.4.3](https://github.com/sdss/sdsstools/compare/0.4.2...0.4.3) - 2020-10-31 - Added `Timer` context manager utility. @@ -200,28 +173,23 @@ - Do not create a logger at the top level of the package. - New option `--log` when starting a daemon, that will redirect stdout and stderr to a log file. - ## [0.4.2](https://github.com/sdss/sdsstools/compare/0.4.1...0.4.2) - 2020-08-19 - Better support for asyncio exception handling. - ## [0.4.1](https://github.com/sdss/sdsstools/compare/0.4.0...0.4.1) - 2020-08-09 - Add signal handler for CLI coroutines. -- Allow to run coroutines in debug mode with the ``cli_coro`` decorator. - +- Allow to run coroutines in debug mode with the `cli_coro` decorator. ## [0.4.0](https://github.com/sdss/sdsstools/compare/0.3.1...0.4.0) - 2020-08-07 - Feature [#15](https://github.com/sdss/sdsstools/issues/15): add a CLI wrapper to create Unix daemons. - ## [0.3.1](https://github.com/sdss/sdsstools/compare/0.3.0...0.3.1) - 2020-08-01 - Remove a leftover print statement in the logger. - ## [0.3.0](https://github.com/sdss/sdsstools/compare/0.2.1...0.3.0) - 2020-08-01 - Support [#4](https://github.com/sdss/sdsstools/issues/4): include `dev` extras in default installation. @@ -230,39 +198,33 @@ - Support [#11](https://github.com/sdss/sdsstools/issues/11): replaced `colored_formatter` with a propper `Formatter` and added tests for logging. - Support [#12](https://github.com/sdss/sdsstools/issues/12): move `color_print` to `sdsstools._vendor`. - ## [0.2.1](https://github.com/sdss/sdsstools/compare/0.2.0...0.2.1) - 2020-07-30 - Store configuration file loaded in `Configuration` as `Configuration.CONFIG_FILE`. - Improve logging of exceptions to the stream logger. - ## [0.2.0](https://github.com/sdss/sdsstools/compare/0.1.12...0.2.0) - 2020-07-29 - Feature [#5](https://github.com/sdss/sdsstools/issues/5): `get_config` now returns an instance of `Configuration`, which can also be used at a lower level. `Configuration` stores the base configuration if it was specified, and can be used to load a new configuration file dynamically. Since `Configuration` subclasses from a dictionary and the functionality in `get_config` has not changed, this should be a non-breaking change. - Support: added initial framework for tests and tests with 98% coverage for `configuration.py`. - Support: add GitHub workflows for linting and testing. - ## [0.1.12](https://github.com/sdss/sdsstools/compare/0.1.11...0.1.12) - 2020-07-16 - Prefer `pyproject.toml` or `setup.cfg` over package metadata to retrieve module version. - ## [0.1.11](https://github.com/sdss/sdsstools/compare/0.1.10...0.1.11) - 2020-07-07 ### Fixed - `TimedRotatingFileHandler` does not accept `mode`. - ## [0.1.10](https://github.com/sdss/sdsstools/compare/0.1.9...0.1.10) - 2020-06-24 ### Added - Option to normalise the version string to PEP 440 by passing `pep_440=True` to `get_package_version`. - ## [0.1.9](https://github.com/sdss/sdsstools/compare/0.1.8...0.1.9) - 2020-05-13 ### Changed @@ -274,7 +236,6 @@ - Handle case where log header is None. - Use `get_config` when there is not a parent package. - ## [0.1.8](https://github.com/sdss/sdsstools/compare/0.1.7...0.1.8) - 2020-05-01 ### Added @@ -286,34 +247,29 @@ - Configuration default path is now `~/.config/sdss//.yaml`. - ## [0.1.7](https://github.com/sdss/sdsstools/compare/0.1.6...0.1.7) - 2020-04-14 ### Changed - Set minimum Python version to 3.6. Add `six` dependency. - ## [0.1.6](https://github.com/sdss/sdsstools/compare/0.1.5...0.1.6) - 2020-03-31 ### Added - Missing `docutils` dependency, needed by releases. - ## [0.1.5](https://github.com/sdss/sdsstools/compare/0.1.4...0.1.5) - 2020-01-23 ### Changed -- Better implementation of ``SDSSLogger.asyncio_exception_handler``. - +- Better implementation of `SDSSLogger.asyncio_exception_handler`. ## [0.1.4](https://github.com/sdss/sdsstools/compare/0.1.3...0.1.4) - 2020-01-23 ### Added -- Added ``SDSSLogger.asyncio_exception_handler`` to report loop exceptions to the logger. - +- Added `SDSSLogger.asyncio_exception_handler` to report loop exceptions to the logger. ## [0.1.3](https://github.com/sdss/sdsstools/compare/0.1.2...0.1.3) - 2020-01-14 @@ -322,14 +278,12 @@ - One call to `PyYAML` was still using `load` instead of `safe_load`. - `read_yaml_file` also accepts a file-like object. - ## [0.1.2](https://github.com/sdss/sdsstools/compare/0.1.1...0.1.2) - 2019-12-31 ### Fixed - The default target for the docs was pointing to `docs/sphinx/_build` instead of `docs/sphinx`. - ## [0.1.1](https://github.com/sdss/sdsstools/compare/0.1.0...0.1.1) - 2019-12-31 ### Added diff --git a/poetry.lock b/poetry.lock index 90b7f59..4c43d80 100644 --- a/poetry.lock +++ b/poetry.lock @@ -118,13 +118,13 @@ test-all = ["astropy[test]", "coverage[toml]", "ipython (>=4.2)", "objgraph", "s [[package]] name = "astropy-iers-data" -version = "0.2023.12.4.0.30.20" +version = "0.2023.12.11.0.31.11" description = "IERS Earth Rotation and Leap Second tables for the astropy core package" optional = false python-versions = ">=3.8" files = [ - {file = "astropy-iers-data-0.2023.12.4.0.30.20.tar.gz", hash = "sha256:2d27bd0af3f09a0ce01dd0e863086a6c6132a6b213a27da561c5fae5424e85c5"}, - {file = "astropy_iers_data-0.2023.12.4.0.30.20-py3-none-any.whl", hash = "sha256:352b2acf7d8abb8de5c1e5c0cc912d6b92dfaea4a1188aad17d94dc6333dc909"}, + {file = "astropy-iers-data-0.2023.12.11.0.31.11.tar.gz", hash = "sha256:30dfae118e3df16a30013becea7231e3b4699e6127595a86bd71b1bd35a0c75b"}, + {file = "astropy_iers_data-0.2023.12.11.0.31.11-py3-none-any.whl", hash = "sha256:dc9b886713a4b8fc0683d760f42eba3fc029918d4c441b1bab60d876010614f0"}, ] [package.extras] @@ -162,29 +162,33 @@ files = [ [[package]] name = "black" -version = "23.11.0" +version = "24.1a1" description = "The uncompromising code formatter." optional = false python-versions = ">=3.8" files = [ - {file = "black-23.11.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:dbea0bb8575c6b6303cc65017b46351dc5953eea5c0a59d7b7e3a2d2f433a911"}, - {file = "black-23.11.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:412f56bab20ac85927f3a959230331de5614aecda1ede14b373083f62ec24e6f"}, - {file = "black-23.11.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d136ef5b418c81660ad847efe0e55c58c8208b77a57a28a503a5f345ccf01394"}, - {file = "black-23.11.0-cp310-cp310-win_amd64.whl", hash = "sha256:6c1cac07e64433f646a9a838cdc00c9768b3c362805afc3fce341af0e6a9ae9f"}, - {file = "black-23.11.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:cf57719e581cfd48c4efe28543fea3d139c6b6f1238b3f0102a9c73992cbb479"}, - {file = "black-23.11.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:698c1e0d5c43354ec5d6f4d914d0d553a9ada56c85415700b81dc90125aac244"}, - {file = "black-23.11.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:760415ccc20f9e8747084169110ef75d545f3b0932ee21368f63ac0fee86b221"}, - {file = "black-23.11.0-cp311-cp311-win_amd64.whl", hash = "sha256:58e5f4d08a205b11800332920e285bd25e1a75c54953e05502052738fe16b3b5"}, - {file = "black-23.11.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:45aa1d4675964946e53ab81aeec7a37613c1cb71647b5394779e6efb79d6d187"}, - {file = "black-23.11.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:4c44b7211a3a0570cc097e81135faa5f261264f4dfaa22bd5ee2875a4e773bd6"}, - {file = "black-23.11.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2a9acad1451632021ee0d146c8765782a0c3846e0e0ea46659d7c4f89d9b212b"}, - {file = "black-23.11.0-cp38-cp38-win_amd64.whl", hash = "sha256:fc7f6a44d52747e65a02558e1d807c82df1d66ffa80a601862040a43ec2e3142"}, - {file = "black-23.11.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:7f622b6822f02bfaf2a5cd31fdb7cd86fcf33dab6ced5185c35f5db98260b055"}, - {file = "black-23.11.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:250d7e60f323fcfc8ea6c800d5eba12f7967400eb6c2d21ae85ad31c204fb1f4"}, - {file = "black-23.11.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5133f5507007ba08d8b7b263c7aa0f931af5ba88a29beacc4b2dc23fcefe9c06"}, - {file = "black-23.11.0-cp39-cp39-win_amd64.whl", hash = "sha256:421f3e44aa67138ab1b9bfbc22ee3780b22fa5b291e4db8ab7eee95200726b07"}, - {file = "black-23.11.0-py3-none-any.whl", hash = "sha256:54caaa703227c6e0c87b76326d0862184729a69b73d3b7305b6288e1d830067e"}, - {file = "black-23.11.0.tar.gz", hash = "sha256:4c68855825ff432d197229846f971bc4d6666ce90492e5b02013bcaca4d9ab05"}, + {file = "black-24.1a1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:3d139b9531e6bb6d129497a46475535d8289dddc861a5b980f908c36597b9817"}, + {file = "black-24.1a1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:2220c470c22476ca9631337b0daae41be2b215599919b19d576a956ad38aca69"}, + {file = "black-24.1a1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2a12829e372563ffff10c18c7aff1ef274da6afbc7bc8ccdb5fcc8ff84cab43f"}, + {file = "black-24.1a1-cp310-cp310-win_amd64.whl", hash = "sha256:d47b6530c55c092a9d841a12c8b3ad838bd639bebf6660a3df9dae83d4ab83c1"}, + {file = "black-24.1a1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:6b594b3ede60182215d258c76de2de64712d2e8424442ff4402276e22684abbe"}, + {file = "black-24.1a1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:915a6b6b916fc66edec886fc71b60284e447d8fa39d22b879af7ae6efccca90f"}, + {file = "black-24.1a1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:cb0a7ea9aa1c108924e31f1204a1e2534af255dbaa24ecbb8c05f47341a7b6f1"}, + {file = "black-24.1a1-cp311-cp311-win_amd64.whl", hash = "sha256:41c0ce5cbdb701900c166bcca08ac941b64cf1d6967509e3caeab126da0ae0d0"}, + {file = "black-24.1a1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:87c8165fad00b03d9c1d400b1dd250479792f49d012807ee45162d323d04fc06"}, + {file = "black-24.1a1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:6e3c74b35ea179bb69440286b81c309a64c34a032746a9eef3399dc3ce671352"}, + {file = "black-24.1a1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d30a018fc03fd1e83c75d40b8a156ef541d0b56b6403b63754e1cc96889849d9"}, + {file = "black-24.1a1-cp312-cp312-win_amd64.whl", hash = "sha256:88d1c60bac2044a409154e895abb9d74c8ff5d034fb70f3e1f7c3ae96206bc0c"}, + {file = "black-24.1a1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:4de8ba5825588017f90e63d7a25fc4df33a6342d1f4d628ad76130d8f4488fc6"}, + {file = "black-24.1a1-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:c86ecd9d3da3d91e96da5f4a43d9c4fe35c5698b0633e91f171ba9468d112a8b"}, + {file = "black-24.1a1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:623efdb54e7290ba75f7b822dfd2d8a47a55e721ae63aab671ccfd46b2ba6c5d"}, + {file = "black-24.1a1-cp38-cp38-win_amd64.whl", hash = "sha256:ec345caf15ae2c61540812500979e92f2989c6b6d4d13d21bdc82908043b3265"}, + {file = "black-24.1a1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:ac226f37fc429b386d6447df6256dc958c28dd602f86f950072febf886995f80"}, + {file = "black-24.1a1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:cad114d8673adab76b3602c28c461c613b7be3da28415500e42aed47415eb561"}, + {file = "black-24.1a1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e8a054dbb8947718820be2ed6953d66b912ec2795f282725efdd08381a11b0d0"}, + {file = "black-24.1a1-cp39-cp39-win_amd64.whl", hash = "sha256:b03cdf8a4e15929adf47e5e40a0ddeea1d63b65cf59c22553c12417a0c7ccbf4"}, + {file = "black-24.1a1-py3-none-any.whl", hash = "sha256:a2c977909557439d0f17dc82adaea84e48374950d53416efc0b8451a594d42c3"}, + {file = "black-24.1a1.tar.gz", hash = "sha256:4a159ae57f239f3f1ef6a78784b00c1c617c7bb188cc351b3017b9e0702df11c"}, ] [package.dependencies] @@ -198,7 +202,7 @@ typing-extensions = {version = ">=4.0.1", markers = "python_version < \"3.11\""} [package.extras] colorama = ["colorama (>=0.4.3)"] -d = ["aiohttp (>=3.7.4)"] +d = ["aiohttp (>=3.7.4)", "aiohttp (>=3.7.4,!=3.9.0)"] jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"] uvloop = ["uvloop (>=0.15.2)"] @@ -1057,13 +1061,13 @@ testing = ["docopt", "pytest (<6.0.0)"] [[package]] name = "pathspec" -version = "0.12.0" +version = "0.12.1" description = "Utility library for gitignore style pattern matching of file paths." optional = false python-versions = ">=3.8" files = [ - {file = "pathspec-0.12.0-py3-none-any.whl", hash = "sha256:f1f8a7eab698c357945c85ed79715e014612b8584faebe209dca4558e2b09513"}, - {file = "pathspec-0.12.0.tar.gz", hash = "sha256:c57e16065a97b7beb175f13c84d27cb05f7b7315741c2fbd5de541042f4ea6e1"}, + {file = "pathspec-0.12.1-py3-none-any.whl", hash = "sha256:a0d503e138a4c123b27490a4f7beda6a01c6f288df0e4a8b79c7eb0dc7b4cc08"}, + {file = "pathspec-0.12.1.tar.gz", hash = "sha256:a482d51503a1ab33b1c67a6c3813a26953dbdc71c31dacaef9a838c4e29f5712"}, ] [[package]] @@ -1148,13 +1152,13 @@ testing = ["pytest", "pytest-benchmark"] [[package]] name = "prompt-toolkit" -version = "3.0.41" +version = "3.0.43" description = "Library for building powerful interactive command lines in Python" optional = false python-versions = ">=3.7.0" files = [ - {file = "prompt_toolkit-3.0.41-py3-none-any.whl", hash = "sha256:f36fe301fafb7470e86aaf90f036eef600a3210be4decf461a5b1ca8403d3cb2"}, - {file = "prompt_toolkit-3.0.41.tar.gz", hash = "sha256:941367d97fc815548822aa26c2a269fdc4eb21e9ec05fc5d447cf09bad5d75f0"}, + {file = "prompt_toolkit-3.0.43-py3-none-any.whl", hash = "sha256:a11a29cb3bf0a28a387fe5122cdb649816a957cd9261dcedf8c9f1fef33eacf6"}, + {file = "prompt_toolkit-3.0.43.tar.gz", hash = "sha256:3527b7af26106cbc65a040bcc84839a3566ec1b051bb0bfe953631e704b0ff7d"}, ] [package.dependencies] @@ -1604,28 +1608,28 @@ jupyter = ["ipywidgets (>=7.5.1,<9)"] [[package]] name = "ruff" -version = "0.1.7" +version = "0.1.8" description = "An extremely fast Python linter and code formatter, written in Rust." optional = false python-versions = ">=3.7" files = [ - {file = "ruff-0.1.7-py3-none-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl", hash = "sha256:7f80496854fdc65b6659c271d2c26e90d4d401e6a4a31908e7e334fab4645aac"}, - {file = "ruff-0.1.7-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:1ea109bdb23c2a4413f397ebd8ac32cb498bee234d4191ae1a310af760e5d287"}, - {file = "ruff-0.1.7-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8b0c2de9dd9daf5e07624c24add25c3a490dbf74b0e9bca4145c632457b3b42a"}, - {file = "ruff-0.1.7-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:69a4bed13bc1d5dabf3902522b5a2aadfebe28226c6269694283c3b0cecb45fd"}, - {file = "ruff-0.1.7-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:de02ca331f2143195a712983a57137c5ec0f10acc4aa81f7c1f86519e52b92a1"}, - {file = "ruff-0.1.7-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:45b38c3f8788a65e6a2cab02e0f7adfa88872696839d9882c13b7e2f35d64c5f"}, - {file = "ruff-0.1.7-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:6c64cb67b2025b1ac6d58e5ffca8f7b3f7fd921f35e78198411237e4f0db8e73"}, - {file = "ruff-0.1.7-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:9dcc6bb2f4df59cb5b4b40ff14be7d57012179d69c6565c1da0d1f013d29951b"}, - {file = "ruff-0.1.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:df2bb4bb6bbe921f6b4f5b6fdd8d8468c940731cb9406f274ae8c5ed7a78c478"}, - {file = "ruff-0.1.7-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:276a89bcb149b3d8c1b11d91aa81898fe698900ed553a08129b38d9d6570e717"}, - {file = "ruff-0.1.7-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:90c958fe950735041f1c80d21b42184f1072cc3975d05e736e8d66fc377119ea"}, - {file = "ruff-0.1.7-py3-none-musllinux_1_2_i686.whl", hash = "sha256:6b05e3b123f93bb4146a761b7a7d57af8cb7384ccb2502d29d736eaade0db519"}, - {file = "ruff-0.1.7-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:290ecab680dce94affebefe0bbca2322a6277e83d4f29234627e0f8f6b4fa9ce"}, - {file = "ruff-0.1.7-py3-none-win32.whl", hash = "sha256:416dfd0bd45d1a2baa3b1b07b1b9758e7d993c256d3e51dc6e03a5e7901c7d80"}, - {file = "ruff-0.1.7-py3-none-win_amd64.whl", hash = "sha256:4af95fd1d3b001fc41325064336db36e3d27d2004cdb6d21fd617d45a172dd96"}, - {file = "ruff-0.1.7-py3-none-win_arm64.whl", hash = "sha256:0683b7bfbb95e6df3c7c04fe9d78f631f8e8ba4868dfc932d43d690698057e2e"}, - {file = "ruff-0.1.7.tar.gz", hash = "sha256:dffd699d07abf54833e5f6cc50b85a6ff043715da8788c4a79bcd4ab4734d306"}, + {file = "ruff-0.1.8-py3-none-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl", hash = "sha256:7de792582f6e490ae6aef36a58d85df9f7a0cfd1b0d4fe6b4fb51803a3ac96fa"}, + {file = "ruff-0.1.8-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:c8e3255afd186c142eef4ec400d7826134f028a85da2146102a1172ecc7c3696"}, + {file = "ruff-0.1.8-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ff78a7583020da124dd0deb835ece1d87bb91762d40c514ee9b67a087940528b"}, + {file = "ruff-0.1.8-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:bd8ee69b02e7bdefe1e5da2d5b6eaaddcf4f90859f00281b2333c0e3a0cc9cd6"}, + {file = "ruff-0.1.8-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a05b0ddd7ea25495e4115a43125e8a7ebed0aa043c3d432de7e7d6e8e8cd6448"}, + {file = "ruff-0.1.8-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:e6f08ca730f4dc1b76b473bdf30b1b37d42da379202a059eae54ec7fc1fbcfed"}, + {file = "ruff-0.1.8-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f35960b02df6b827c1b903091bb14f4b003f6cf102705efc4ce78132a0aa5af3"}, + {file = "ruff-0.1.8-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:7d076717c67b34c162da7c1a5bda16ffc205e0e0072c03745275e7eab888719f"}, + {file = "ruff-0.1.8-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b6a21ab023124eafb7cef6d038f835cb1155cd5ea798edd8d9eb2f8b84be07d9"}, + {file = "ruff-0.1.8-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:ce697c463458555027dfb194cb96d26608abab920fa85213deb5edf26e026664"}, + {file = "ruff-0.1.8-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:db6cedd9ffed55548ab313ad718bc34582d394e27a7875b4b952c2d29c001b26"}, + {file = "ruff-0.1.8-py3-none-musllinux_1_2_i686.whl", hash = "sha256:05ffe9dbd278965271252704eddb97b4384bf58b971054d517decfbf8c523f05"}, + {file = "ruff-0.1.8-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:5daaeaf00ae3c1efec9742ff294b06c3a2a9db8d3db51ee4851c12ad385cda30"}, + {file = "ruff-0.1.8-py3-none-win32.whl", hash = "sha256:e49fbdfe257fa41e5c9e13c79b9e79a23a79bd0e40b9314bc53840f520c2c0b3"}, + {file = "ruff-0.1.8-py3-none-win_amd64.whl", hash = "sha256:f41f692f1691ad87f51708b823af4bb2c5c87c9248ddd3191c8f088e66ce590a"}, + {file = "ruff-0.1.8-py3-none-win_arm64.whl", hash = "sha256:aa8ee4f8440023b0a6c3707f76cadce8657553655dcbb5fc9b2f9bb9bee389f6"}, + {file = "ruff-0.1.8.tar.gz", hash = "sha256:f7ee467677467526cfe135eab86a40a0e8db43117936ac4f9b469ce9cdb3fb62"}, ] [[package]] diff --git a/pyproject.toml b/pyproject.toml index 5db4a95..372ed5a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -92,7 +92,6 @@ omit = [ "*/_vendor/*", "*/cli.py", "*/_tasks.py", - "*/utils.py" ] [build-system] diff --git a/src/sdsstools/__init__.py b/src/sdsstools/__init__.py index d6b8c40..4ef10c3 100644 --- a/src/sdsstools/__init__.py +++ b/src/sdsstools/__init__.py @@ -23,6 +23,7 @@ from .logger import * from .metadata import * from .time import get_sjd +from .utils import * # # This is a hack to allow doing from sdsstools.color_print import color_text diff --git a/src/sdsstools/utils.py b/src/sdsstools/utils.py index c31b599..e07355f 100644 --- a/src/sdsstools/utils.py +++ b/src/sdsstools/utils.py @@ -6,7 +6,18 @@ # @Filename: utils.py # @License: BSD 3-clause (http://www.opensource.org/licenses/BSD-3-Clause) +from __future__ import annotations + +import asyncio +import concurrent.futures +import pathlib +import tempfile import time +from contextlib import suppress +from functools import partial + + +__all__ = ["Timer", "get_temporary_file_path", "run_in_executor", "cancel_task"] class Timer: @@ -30,3 +41,64 @@ def elapsed(self): return self.interval return time.time() - self.start + + +def get_temporary_file_path(*args, create_parents: bool = False, **kwargs): + """Returns a valid path to a temporary file. + + `args` and `kwargs` are directly passed to `tempfile.NamedTemporaryFile`. + If `create_parents`, the parent directories are created if they don't + exist. + + """ + + tmp_log_file = tempfile.NamedTemporaryFile(*args, **kwargs) + tmp_log_file.close() + + tmp_path = pathlib.Path(tmp_log_file.name) + if tmp_path.exists(): + tmp_path.unlink() + elif create_parents: + tmp_path.parent.mkdir(parents=True, exist_ok=True) + + return tmp_path + + +async def run_in_executor(fn, *args, executor="thread", **kwargs): + """Runs a function in an executor. + + In addition to streamlining the use of the executor, this function + catches any warning issued during the execution and reissues them + after the executor is done. This is important when using the + actor log handler since inside the executor there is no loop that + CLU can use to output the warnings. + + In general, note that the function must not try to do anything with + the actor since they run on different loops. + + """ + + fn = partial(fn, *args, **kwargs) + + if executor == "thread": + executor = concurrent.futures.ThreadPoolExecutor + elif executor == "process": + executor = concurrent.futures.ProcessPoolExecutor + else: + raise ValueError("Invalid executor name.") + + with executor() as pool: + result = await asyncio.get_running_loop().run_in_executor(pool, fn) + + return result + + +async def cancel_task(task: asyncio.Future | None): + """Safely cancels a task.""" + + if task is None or task.done(): + return + + task.cancel() + with suppress(asyncio.CancelledError): + await task diff --git a/test/test_metadata.py b/test/test_metadata.py index 83ed822..5e05666 100644 --- a/test/test_metadata.py +++ b/test/test_metadata.py @@ -66,6 +66,6 @@ def test_get_package_version_path(tmp_path): def test_get_package_version_name(): - assert get_package_version(package_name="pip") is not None + assert get_package_version(package_name="click") is not None assert get_package_version(package_name="non-existing-package") is None diff --git a/test/test_utils.py b/test/test_utils.py new file mode 100644 index 0000000..d870c7e --- /dev/null +++ b/test/test_utils.py @@ -0,0 +1,91 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +# +# @Author: José Sánchez-Gallego (gallegoj@uw.edu) +# @Date: 2023-12-13 +# @Filename: test_utils.py +# @License: BSD 3-clause (http://www.opensource.org/licenses/BSD-3-Clause) + +from __future__ import annotations + +import asyncio +import warnings +from time import sleep + +import pytest + +from sdsstools.utils import Timer, cancel_task, get_temporary_file_path, run_in_executor + + +def test_timer(): + with Timer() as timer: + sleep(0.1) + + assert timer.elapsed > 0 + + +def test_timer_no_end(): + with Timer() as timer: + sleep(0.1) + + timer.end = None + assert timer.elapsed > 0 + + +@pytest.mark.parametrize("create_parents", [True, False]) +def test_create_temp_file(create_parents: bool): + tmp_path = get_temporary_file_path(create_parents=create_parents) + + assert not tmp_path.exists() + + if create_parents: + assert tmp_path.parent.exists() + + tmp_path.unlink(missing_ok=True) + + +def _executor_test_function(): + warnings.warn("A warning", UserWarning) + sleep(0.5) + return True + + +@pytest.mark.parametrize("executor", ["thread", "process"]) +async def test_run_in_executor(executor: str): + result = await run_in_executor(_executor_test_function, executor=executor) + + assert result + + +async def test_run_in_executor_bad_executor(): + with pytest.raises(ValueError): + await run_in_executor(_executor_test_function, executor="blah") + + +async def _task(): + await asyncio.sleep(0.5) + return True + + +async def test_cancel_task(): + task = asyncio.create_task(_task()) + + await cancel_task(task) + + assert task.cancelled() + assert task.done() + + +async def test_cancel_task_done(): + task = asyncio.create_task(_task()) + await task + + await cancel_task(task) + + assert not task.cancelled() + assert task.done() + + +async def test_cancel_task_None(): + task = None + await cancel_task(task)