From c9113da4d38c76416785a31caea2b80128beb6b1 Mon Sep 17 00:00:00 2001 From: Liezl Maree <38435167+roomrys@users.noreply.github.com> Date: Fri, 30 Jun 2023 13:04:42 -0700 Subject: [PATCH] SLEAP 1.3.1 (#1373) * Disable data caching by default for SingleImageVideos (#1243) * Disable data caching by default for SingleImageVideos * Remove a couple things that shouldn't be in this PR :) * Centralize video extensions (#1244) * Add video extension support lists to io.video module * Use centralized extension definitions * Remove some unused code * Add some coverage for indirect coverage reduction * Fix single frame GUI increment (#1254) * Fix frame increment for single frame videos * Add test and lint * Add video search to "robustify" (excessively) fragile test (#1262) Add video search to fix fragile test * Organize docks (#1265) * Add `DockWidget` class and subclasses * Create docks in `MainWindow` using `DockWidget` classes * Remove unused imports * Fix references in existing tests * Fix intermittent bug (file-corruption) that has snuck into the tests (#1267) * Remove corrupted test file * Add non-corrupted copy of test file * Rename the file so tests run * Layout docks in a tab configuration (instead of stacked) (#1289) * Add tab with to docks (and use `self.model` in table) * Use `self.model_type` in `create_models` * Increase GUI crop size range from 512 to 832 (#1295) * Fix conversion to numpy array when last frame(s) do not have labels (#1307) * Ensure frames to predict list is unique (#1293) * Ensure frames to predict list is unique * Ensure frames to predict on are ordered correctly * Better frame sorting * Fix GUI resume training (#1314) * Do not choose `top_k` instances if `max_instances` < num centroids (#1313) * Use max_instances as a max without any minimum requirement * Create test (that fails) * Fix test by re-init inference model each call * Do not compute top k if max is greater * Add `--max_instances` to `sleap-track` and GUI (#1305) * Expose --max_instances to sleap-track and in GUI * Lint * Change shorthand to `-n` for both `sleap-export` and `sleap-track` * Add test for creating predictor from cli * Add max instances support to bottom up model (#1306) * Add max instances support to bottom up model * Remove unnecessary attribute setter in CLI parser * Edge case * Expose max_instances for BU to train/infer GUI --------- Co-authored-by: roomrys <38435167+roomrys@users.noreply.github.com> * Update docs for `--max_instances` command * Add test for BU `--max_instances` --------- Co-authored-by: Talmo Pereira * Remove `--labels` and redundant `data_path` (#1326) * Create copy of config info to modify (gui) (#1325) * Fixes GPU memory polling using environment variable filtering (#1272) * add cuda visible gpus to nvidia smi CLI * tests masked none, single and multiple GPUs * deal with comma separated lists and test 2 gpus * clean up system script and better test that actually works * add a case for cuda_visible_devices = [] * add test for gpu order and length in nvidia and tf * test nvidia smi indices and checking gpu memory * fix spaces * fix linting with Black * put skips in tests for git not having nvidia-smi * add doc strings to get_gpu_memory * set CUDA device order to PCI BUS ID * test with no nvidia smi + invalid visible devices * fixed linting again * remove pci env variable, move to other issue --------- Co-authored-by: Eric Leonardis * Set `split_by_inds`, `test_labels`, and `validation_labels` to default (GUI) (#1331) * Allow returning PAF graph during low level inference (#1329) * allow returning the paf graph during low level inference * reformatted with black * Fix `SingleImageVideo` caching (#1330) * Set `SingleImageVideo.caching` as a class attribute * Modify tests for `SingleImageVideo.caching` * Add caching as a preference and menu checkbox * Test `SingleImageVideo.toggle_caching` * Remove GUI elements for `SingleImageVideo.CACHING` * Remove remaining prefs for `SingleImageVideo.CACHING` * Add depreciated comment * Clean-up * Update comments * Bump to 1.3.1 (#1335) * Bump to 1.3.1 * Test build to dev label * Update environment creation (#1366) * First 'working' env for 1.3.1, no GPU... * First working 1.3.1 w/gpu * Sorry, this is the first gpu working env * Fix `imgstore` version-dependent bug * Build and entry points work, but pip packages not installed * Add default channels in condarc.yaml * Rename environment.yaml to .yml * Working build no gpu (documented) * Env working w/gpu using tensorflow from pypi * Working build with GPU! And pip dep * Attempt to fix cannot find wheel ~= 0.35 issue * Run constrain everything * Use minimal conda req for install_requires, inc build number to get it working * Ubuntu build and environments working * Env creation working on M2 * Build and env working on windows (again) * Apple silicon (M2) build working * Pip package working (M2) * Exclude tests and docs from pypi wheel * Add comments to requirements * Get ready for manual build * Update os images, trigger dev build * Retry manual build win and mac * Retry mac manual build (to upload) * Require latest pynwb, remove setuptools * Remove old mac environment * Add twine as a build dependency * Add comments to manual build * Update build.yml * Rename "apple_silicon" to "mac" * Update installation docs (#1365) * Update installation docs * Trigger website build * Update quick install and build docs * Add Mambaforge links * Remove comment about experimental apple silicon support Co-authored-by: Talmo Pereira * Fix minor typo * Change installation docs to reference universal mac env --------- Co-authored-by: Talmo Pereira * More flexible with python version ~=3.7 * Pin py3.7.12 for host and run (good on win) * Add comments for why python ~=3.7 * Update ci workflow (#1371) * Update CI workflow to include mac * Trigger CI workflow * Remove verbose? python version * Migrate to micromamba * Explicitly state environment name * Remove environment chaching (for now) * Use different environment file for mac * Use correct syntax * Add `env_hash` and `bash -l {0}` * Remove env_file * Try different nested variable call via format * Use correct syntax * Update env file reference AS -> mac * Different path to environment file * Checkout repo * Use default shells * Remove unused comments * Fix caching attempt by changing caching key * Remove environment caching * Increase python in no_cuda to 3.7.17 * Less restrictive with python ~3.7 version * More direct installation docs :crossed_fingers: * Increase build numbers for develop branch build --------- Co-authored-by: Talmo Pereira --------- Co-authored-by: Talmo Pereira Co-authored-by: Eric Leonardis Co-authored-by: Eric Leonardis Co-authored-by: Caleb Weinreb --- .conda/README.md | 21 + .conda/bld.bat | 62 +- .conda/build.sh | 48 +- .conda/conda_build_config.yaml | 10 - .conda/condarc.yaml | 5 + .conda/meta.yaml | 104 ++-- .conda_apple_silicon/README.md | 19 - .conda_apple_silicon/build.sh | 51 -- .conda_apple_silicon/conda_build_config.yaml | 9 - .conda_apple_silicon/meta.yaml | 56 -- .conda_mac/README.md | 21 + .conda_mac/build.sh | 12 + .conda_mac/condarc.yaml | 6 + .conda_mac/meta.yaml | 90 +++ .github/ISSUE_TEMPLATE/bug_report.md | 2 +- .github/workflows/build.yml | 75 ++- .github/workflows/build_manual.yml | 80 ++- .github/workflows/ci.yml | 91 ++- .github/workflows/website.yml | 6 +- MANIFEST.in | 5 +- README.rst | 3 +- dev_requirements.txt | 2 + docs/conf.py | 4 +- docs/guides/cli.md | 7 +- docs/installation.md | 226 ++++--- environment.yml | 66 +- environment_apple_silicon.yml | 20 - environment_build.yml | 29 +- environment_mac.yml | 51 +- environment_no_cuda.yml | 61 +- pip_requirements.txt | 34 ++ requirements.txt | 48 +- setup.py | 10 +- sleap/config/pipeline_form.yaml | 34 +- sleap/config/training_editor_form.yaml | 2 +- sleap/gui/app.py | 387 ++---------- sleap/gui/commands.py | 2 +- sleap/gui/dialogs/importvideos.py | 60 +- sleap/gui/dialogs/missingfiles.py | 7 - sleap/gui/learning/dialog.py | 12 +- sleap/gui/learning/receptivefield.py | 17 - sleap/gui/learning/runners.py | 29 +- sleap/gui/state.py | 21 +- sleap/gui/widgets/docks.py | 567 ++++++++++++++++++ sleap/io/dataset.py | 3 +- sleap/io/video.py | 70 ++- sleap/nn/evals.py | 8 +- sleap/nn/inference.py | 140 ++++- sleap/nn/paf_grouping.py | 14 +- sleap/nn/system.py | 32 +- sleap/nn/training.py | 6 +- sleap/version.py | 2 +- .../{robot_siv.slp => small_robot_siv.slp} | Bin .../siv_format_v2/small_robot_siv_caching.slp | Bin 0 -> 16384 bytes tests/fixtures/datasets.py | 21 +- tests/gui/learning/test_dialog.py | 61 +- tests/gui/test_app.py | 25 +- tests/gui/test_dialogs.py | 2 +- tests/gui/test_inference_gui.py | 27 +- tests/gui/test_state.py | 7 +- tests/gui/widgets/test_docks.py | 55 ++ tests/io/test_dataset.py | 9 +- tests/io/test_video.py | 49 +- tests/nn/test_evals.py | 23 +- tests/nn/test_inference.py | 61 +- tests/nn/test_system.py | 77 +++ tests/nn/test_training.py | 42 +- 67 files changed, 2028 insertions(+), 1178 deletions(-) create mode 100644 .conda/README.md delete mode 100644 .conda/conda_build_config.yaml create mode 100644 .conda/condarc.yaml delete mode 100644 .conda_apple_silicon/README.md delete mode 100644 .conda_apple_silicon/build.sh delete mode 100644 .conda_apple_silicon/conda_build_config.yaml delete mode 100644 .conda_apple_silicon/meta.yaml create mode 100644 .conda_mac/README.md create mode 100644 .conda_mac/build.sh create mode 100644 .conda_mac/condarc.yaml create mode 100644 .conda_mac/meta.yaml delete mode 100644 environment_apple_silicon.yml create mode 100644 pip_requirements.txt create mode 100644 sleap/gui/widgets/docks.py rename tests/data/siv_format_v1/{robot_siv.slp => small_robot_siv.slp} (100%) create mode 100644 tests/data/siv_format_v2/small_robot_siv_caching.slp create mode 100644 tests/gui/widgets/test_docks.py diff --git a/.conda/README.md b/.conda/README.md new file mode 100644 index 000000000..65fadd36e --- /dev/null +++ b/.conda/README.md @@ -0,0 +1,21 @@ +This folder defines the conda package build for Linux and Windows. There are runners for both Linux and Windows on GitHub Actions, but it is faster to experiment with builds locally first. + +To build, first go to the base repo directory and install the build environment: + +``` +conda env create -f environment_build.yml -n sleap_build && conda activate sleap_build +``` + +And finally, run the build command pointing to this directory: + +``` +conda build .conda --output-folder build -c conda-forge -c nvidia -c https://conda.anaconda.org/sleap/ -c anaconda +``` + +To install the local package: + +``` +conda create -n sleap_0 -c conda-forge -c nvidia -c ./build -c https://conda.anaconda.org/sleap/ -c anaconda sleap=x.x.x +``` + +replacing x.x.x with the version of SLEAP that you just built. diff --git a/.conda/bld.bat b/.conda/bld.bat index 4cf76b3c5..542b82616 100644 --- a/.conda/bld.bat +++ b/.conda/bld.bat @@ -1,62 +1,14 @@ -@echo off +@REM Install anything that didn't get conda installed via pip. -rem # Install anything that didn't get conda installed via pip. -rem # We need to turn pip index back on because Anaconda turns -rem # it off for some reason. Just pip install -r requirements.txt -rem # doesn't seem to work, tensorflow-gpu, jsonpickle, networkx, -rem # all get installed twice if we do this. pip doesn't see the -rem # conda install of the packages. - -rem # Install the pip dependencies and their dependencies. Conda turns of -rem # pip index and dependencies by default so re-enable them. Had to figure -rem # this out myself, ughhh. +@REM We need to turn pip index back on because Anaconda turns it off for some reason. set PIP_NO_INDEX=False set PIP_NO_DEPENDENCIES=False set PIP_IGNORE_INSTALLED=False -pip install numpy==1.19.5 -pip install six==1.15.0 -pip install imageio==2.15.0 -pip install attrs==21.2.0 -pip install cattrs==1.1.1 -pip install jsonpickle==1.2 -pip install networkx -pip install nixio>=1.5.3 -@REM pip install tensorflow>=2.6.3,<=2.7.1 -@REM pip install h5py>=3.1.0,<=3.6.0 -pip install python-rapidjson -@REM pip install opencv-python-headless>=4.2.0.34,<=4.5.5.62 -@REM pip install opencv-python @ git+https://github.com/talmolab/wrap_opencv-python-headless.git@ede49f6a23a73033216339f29515e59d594ba921 -@REM pip install pandas -pip install psutil -@REM pip install PySide2>=5.13.2,<=5.14.1 -pip install pyzmq -pip install pyyaml -pip install imgaug==0.4.0 -@REM pip install scipy>=1.4.1,<=1.7.3 -pip install scikit-image -pip install scikit-learn==1.0.* -pip install scikit-video -pip install tensorflow-hub -pip install imgstore==0.2.9 -pip install qimage2ndarray==1.9.0 -pip install jsmin -pip install seaborn -pip install pykalman==0.9.5 -pip install segmentation-models==1.0.1 -pip install rich==10.16.1 -pip install certifi==2021.10.8 -pip install pynwb -pip install ndx-pose - -rem # Use and update environment.yml call to install pip dependencies. This is slick. -rem # While environment.yml contains the non pip dependencies, the only thing left -rem # uninstalled should be the pip stuff because that is left out of meta.yml -rem conda env update -f=environment.yml - -rem # Install requires setuptools-scm -pip install setuptools-scm +@REM Install the pip dependencies. Note: Using urls to wheels might be better: +@REM https://docs.conda.io/projects/conda-build/en/stable/user-guide/wheel-files.html) +pip install -r .\requirements.txt -rem # Install sleap itself. -rem # NOTE: This is the recommended way to install packages +@REM Install sleap itself. This does not install the requirements, but will list which +@REM requirements are missing (see "install_requires") when user attempts to install. python setup.py install --single-version-externally-managed --record=record.txt diff --git a/.conda/build.sh b/.conda/build.sh index b2569b8f9..85bbe442f 100644 --- a/.conda/build.sh +++ b/.conda/build.sh @@ -1,51 +1,15 @@ -#!/usr/bin/env bash - # Install anything that didn't get conda installed via pip. -# We need to turn pip index back on because Anaconda turns -# it off for some reason. Just pip install -r requirements.txt -# doesn't seem to work, tensorflow-gpu, jsonpickle, networkx, -# all get installed twice if we do this. pip doesn't see the -# conda install of the packages. +# We need to turn pip index back on because Anaconda turns it off for some reason. export PIP_NO_INDEX=False export PIP_NO_DEPENDENCIES=False export PIP_IGNORE_INSTALLED=False -pip install numpy==1.19.5 -pip install six==1.15.0 -pip install imageio==2.15.0 -pip install attrs==21.2.0 -pip install cattrs==1.1.1 -pip install jsonpickle==1.2 -pip install networkx -pip install nixio>=1.5.3 -# pip install tensorflow>=2.6.3,<=2.7.1 -# pip install h5py>=3.1.0,<=3.6.0 -pip install python-rapidjson -# pip install opencv-python-headless==4.5.5.62 -# pip install git+https://github.com/talmolab/wrap_opencv-python-headless.git@ede49f6a23a73033216339f29515e59d594ba921 -# pip install pandas -pip install psutil -# pip install PySide2>=5.13.2,<=5.14.1 -pip install pyzmq -pip install pyyaml -pip install imgaug==0.4.0 -# pip install scipy>=1.4.1,<=1.7.3 -pip install scikit-image -pip install scikit-learn==1.0.* -pip install scikit-video -pip install tensorflow-hub -pip install imgstore==0.2.9 -pip install qimage2ndarray==1.9.0 -pip install jsmin -pip install seaborn -pip install pykalman==0.9.5 -pip install segmentation-models==1.0.1 -pip install rich==10.16.1 -pip install certifi==2021.10.8 -pip install pynwb -pip install ndx-pose +# Install the pip dependencies. Note: Using urls to wheels might be better: +# https://docs.conda.io/projects/conda-build/en/stable/user-guide/wheel-files.html) +pip install -r ./requirements.txt -pip install setuptools-scm +# Install sleap itself. This does not install the requirements, but will list which +# requirements are missing (see "install_requires") when user attempts to install. python setup.py install --single-version-externally-managed --record=record.txt \ No newline at end of file diff --git a/.conda/conda_build_config.yaml b/.conda/conda_build_config.yaml deleted file mode 100644 index 80fadb65d..000000000 --- a/.conda/conda_build_config.yaml +++ /dev/null @@ -1,10 +0,0 @@ -# We specify the numpy version here for build compatibility. -# -# Ref: https://docs.conda.io/projects/conda-build/en/latest/resources/variants.html#conda-build-variant-config-files - -python: - - 3.7 - -numpy: - - 1.19.5 - # - 1.21.2 \ No newline at end of file diff --git a/.conda/condarc.yaml b/.conda/condarc.yaml new file mode 100644 index 000000000..f9ac6efbe --- /dev/null +++ b/.conda/condarc.yaml @@ -0,0 +1,5 @@ +channels: + - conda-forge + - nvidia + - sleap + - anaconda diff --git a/.conda/meta.yaml b/.conda/meta.yaml index 9356c113c..c80d3b56f 100644 --- a/.conda/meta.yaml +++ b/.conda/meta.yaml @@ -1,6 +1,3 @@ -# Ref: https://docs.conda.io/projects/conda-build/en/latest/resources/define-metadata.html - -# Jinja template: Process setup.py to obtain version and metadata {% set data = load_setup_py_data() %} @@ -15,50 +12,79 @@ about: license: {{ data.get('license') }} summary: {{ data.get('description') }} -build: - number: 4 - source: path: ../ +build: + number: 9 + requirements: host: - - python=3.7 - # - sleap::pyside2=5.14.1 - - conda-forge::numpy=1.19.5 - - sleap::tensorflow=2.6.3 - - conda-forge::pyside2=5.13.2 - - conda-forge::h5py=3.1.0 - - conda-forge::scipy=1.7.3 - - conda-forge::six=1.15.0 - - pillow=8.4.0 - - shapely=1.7.1 - - conda-forge::pandas - - ffmpeg - - qtpy>=2.0.1 - - conda-forge::pip!=22.0.4 + - conda-forge::python ==3.7.12 # Run into _MAX_WINDOWS_WORKERS not found if < + - numpy >=1.19.5,<1.23.0 # Linux likes anaconda, windows likes conda-forge + - conda-forge::cudatoolkit ==11.3.1 + - conda-forge::cudnn=8.2.1 + - nvidia::cuda-nvcc=11.3 + - conda-forge::setuptools + - conda-forge::pip - run: - - python=3.7 - - conda-forge::numpy~=1.19.5 - - sleap::tensorflow=2.6.3 - - conda-forge::pyside2>=5.13.2,<=5.14.1 - - conda-forge::h5py~=3.1.0 - - conda-forge::scipy>=1.4.1,<=1.7.3 - - conda-forge::six~=1.15.0 - - pillow=8.4.0 - - shapely=1.7.1 + # Only the packages above are required to build, but listing them all ensures no + # unnecessary pypi packages are installed via the build script (bld.bat, build.sh) + - conda-forge::attrs ==21.4.0 + - conda-forge::cattrs ==1.1.1 + - conda-forge::h5py ==3.1 # [not win] + - conda-forge::imgaug ==0.4.0 + - conda-forge::jsmin + - conda-forge::jsonpickle ==1.2 + - conda-forge::networkx + - conda-forge::opencv - conda-forge::pandas - - ffmpeg - - qtpy>=2.0.1 - - cudatoolkit=11.3.1 - - cudnn=8.2.1 + - conda-forge::pillow >=8.3.2 + - conda-forge::psutil + - conda-forge::pykalman + - conda-forge::pyside2 >=5.12,<5.14 + - conda-forge::python-rapidjson + - conda-forge::pyyaml + - conda-forge::pyzmq + - conda-forge::qtpy >=2.0.1 + - conda-forge::rich + - conda-forge::scipy >=1.4.1,<=1.9.0 + - conda-forge::scikit-image + - conda-forge::scikit-learn ==1.0 + - conda-forge::scikit-video + - conda-forge::seaborn + run: + - conda-forge::python ==3.7.12 # Run into _MAX_WINDOWS_WORKERS not found if < + - conda-forge::attrs ==21.4.0 + - conda-forge::cattrs ==1.1.1 + - conda-forge::cudatoolkit ==11.3.1 + - conda-forge::cudnn=8.2.1 - nvidia::cuda-nvcc=11.3 - - conda-forge::pip!=22.0.4 - - run_constrained: - - pyqt==9999999999 + - conda-forge::h5py ==3.1 # [not win] + - conda-forge::imgaug ==0.4.0 + - conda-forge::jsmin + - conda-forge::jsonpickle ==1.2 + - conda-forge::networkx + - numpy >=1.19.5,<1.23.0 # Linux likes anaconda, windows likes conda-forge + - conda-forge::opencv + - conda-forge::pandas + - conda-forge::pillow >=8.3.2 + - conda-forge::psutil + - conda-forge::pykalman + - conda-forge::pyside2 >=5.12,<5.14 # To ensure works correctly with QtPy. + - conda-forge::python-rapidjson + - conda-forge::pyyaml + - conda-forge::pyzmq + - conda-forge::qtpy >=2.0.1 + - conda-forge::rich + - conda-forge::scipy >=1.4.1,<=1.9.0 + - conda-forge::scikit-image + - conda-forge::scikit-learn ==1.0 + - conda-forge::scikit-video + - conda-forge::seaborn + - sleap::tensorflow >=2.6.3,<2.11 # No windows GPU support for >2.10, sleap channel has 2.6.3 + - conda-forge::tensorflow-hub test: imports: - - sleap + - sleap \ No newline at end of file diff --git a/.conda_apple_silicon/README.md b/.conda_apple_silicon/README.md deleted file mode 100644 index c66af90f2..000000000 --- a/.conda_apple_silicon/README.md +++ /dev/null @@ -1,19 +0,0 @@ -This folder defines the conda package build for Apple silicon Macs. Until there are aarm64 runners, we have to run this manually on Apple M1 silicon. - -To build, first go to the base repo directory and install the Apple silicon compatible environment: - -``` -conda env create -f environment_apple_silicon.yml -n sleap_build && conda activate sleap_build -``` - -Next, install build dependencies: - -``` -conda install conda-build=3.21.7 && conda install anaconda-client && conda install conda-verify -``` - -And finally, run the build command pointing to this directory: - -``` -conda build .conda_apple_silicon --output-folder build -c https://conda.anaconda.org/sleap/ -c nvidia -c conda-forge -c apple -``` diff --git a/.conda_apple_silicon/build.sh b/.conda_apple_silicon/build.sh deleted file mode 100644 index 9652efb47..000000000 --- a/.conda_apple_silicon/build.sh +++ /dev/null @@ -1,51 +0,0 @@ -#!/usr/bin/env bash - -# Install anything that didn't get conda installed via pip. -# We need to turn pip index back on because Anaconda turns -# it off for some reason. Just pip install -r requirements.txt -# doesn't seem to work, tensorflow-gpu, jsonpickle, networkx, -# all get installed twice if we do this. pip doesn't see the -# conda install of the packages. - -export PIP_NO_INDEX=False -export PIP_NO_DEPENDENCIES=False -export PIP_IGNORE_INSTALLED=False - -# pip install numpy==1.22.3 -pip install attrs==21.4.0 -pip install cattrs==1.1.1 -pip install jsonpickle==1.2 -pip install networkx -# pip install tensorflow>=2.6.3,<2.9.0; platform_machine != 'arm64' -pip install tensorflow-macos==2.9.2 -pip install tensorflow-metal==0.5.0 -# pip install h5py==3.6.0 -pip install python-rapidjson -# pip install opencv-python==4.6.0 -pip install pandas -pip install psutil -# pip install PySide2==5.15.5 -pip install pyzmq -pip install pyyaml -# pip install pillow==8.4.0 -pip install imageio<=2.15.0 -pip install imgaug==0.4.0 -# pip install scipy==1.7.3 -pip install scikit-image -pip install scikit-learn==1.0.* -pip install scikit-video -pip install imgstore==0.2.9 -pip install qimage2ndarray==1.9.0 -pip install jsmin -pip install seaborn -pip install pykalman==0.9.5 -pip install segmentation-models==1.0.1 -pip install rich==10.16.1 -pip install certifi==2021.10.8 -pip install pynwb -pip install ndx-pose - - -pip install setuptools-scm - -python setup.py install --single-version-externally-managed --record=record.txt \ No newline at end of file diff --git a/.conda_apple_silicon/conda_build_config.yaml b/.conda_apple_silicon/conda_build_config.yaml deleted file mode 100644 index 1e9e2addf..000000000 --- a/.conda_apple_silicon/conda_build_config.yaml +++ /dev/null @@ -1,9 +0,0 @@ -# We specify the numpy version here for build compatibility. -# -# Ref: https://docs.conda.io/projects/conda-build/en/latest/resources/variants.html#conda-build-variant-config-files - -python: - - 3.9 - -numpy: - - 1.22.3 \ No newline at end of file diff --git a/.conda_apple_silicon/meta.yaml b/.conda_apple_silicon/meta.yaml deleted file mode 100644 index 250f200dd..000000000 --- a/.conda_apple_silicon/meta.yaml +++ /dev/null @@ -1,56 +0,0 @@ -# Ref: https://docs.conda.io/projects/conda-build/en/latest/resources/define-metadata.html - -# Jinja template: Process setup.py to obtain version and metadata -{% set data = load_setup_py_data() %} - - -package: - # Repeating name because of the following issue: - # https://github.com/conda/conda-build/issues/2475 - name: sleap - version: {{ data.get('version') }} - -about: - home: {{ data.get('url') }} - license: {{ data.get('license') }} - summary: {{ data.get('description') }} - -build: - number: 1 - -source: - path: ../ - -requirements: - host: - - python=3.9 - - shapely=1.7.1 - - conda-forge::h5py=3.6.0 - - conda-forge::numpy=1.22.3 - - scipy=1.7.3 - - pillow=8.4.0 - - apple::tensorflow-deps=2.9.0 - - conda-forge::pyside2=5.15.5 - - conda-forge::opencv=4.6.0 - - qtpy=2.0.1 - - conda-forge::pip!=22.0.4 - - run: - - python=3.9 - - shapely=1.7.1 - - conda-forge::h5py=3.6.0 - - conda-forge::numpy=1.22.3 - - scipy=1.7.3 - - pillow=8.4.0 - - apple::tensorflow-deps=2.9.0 - - conda-forge::pyside2=5.15.5 - - conda-forge::opencv=4.6.0 - - qtpy=2.0.1 - - conda-forge::pip!=22.0.4 - - run_constrained: - - pyqt==9999999999 - -test: - imports: - - sleap diff --git a/.conda_mac/README.md b/.conda_mac/README.md new file mode 100644 index 000000000..06f370b4f --- /dev/null +++ b/.conda_mac/README.md @@ -0,0 +1,21 @@ +This folder defines the conda package build for Apple silicon Macs. Until there are aarm64 runners, we have to run this manually on Apple M1 silicon. + +To build, first go to the base repo directory and install the build environment: + +``` +conda env create -f environment_build.yml -n sleap_build && conda activate sleap_build +``` + +And finally, run the build command pointing to this directory: + +``` +conda build .conda_mac --output-folder build -c conda-forge -c anaconda +``` + +To install the local package: + +``` +conda create -n sleap_0 -c conda-forge -c anaconda -c ./build sleap=x.x.x +``` + +replacing x.x.x with the version of SLEAP that you just built. diff --git a/.conda_mac/build.sh b/.conda_mac/build.sh new file mode 100644 index 000000000..f1299991b --- /dev/null +++ b/.conda_mac/build.sh @@ -0,0 +1,12 @@ +#!/usr/bin/env bash + +# Install anything that didn't get conda installed via pip. +# We need to turn pip index back on because Anaconda turns it off for some reason. + +export PIP_NO_INDEX=False +export PIP_NO_DEPENDENCIES=False +export PIP_IGNORE_INSTALLED=False + +pip install -r requirements.txt + +python setup.py install --single-version-externally-managed --record=record.txt \ No newline at end of file diff --git a/.conda_mac/condarc.yaml b/.conda_mac/condarc.yaml new file mode 100644 index 000000000..df2727c74 --- /dev/null +++ b/.conda_mac/condarc.yaml @@ -0,0 +1,6 @@ +# This file is not used at the moment, but when github actions can be used to build the package, it needs to be listed. +# https://github.com/github/roadmap/issues/528 + +channels: + - conda-forge + - anaconda \ No newline at end of file diff --git a/.conda_mac/meta.yaml b/.conda_mac/meta.yaml new file mode 100644 index 000000000..db2f23215 --- /dev/null +++ b/.conda_mac/meta.yaml @@ -0,0 +1,90 @@ +# Ref: https://docs.conda.io/projects/conda-build/en/latest/resources/define-metadata.html + +# Jinja template: Process setup.py to obtain version and metadata +{% set data = load_setup_py_data() %} + + +package: + # Repeating name because of the following issue: + # https://github.com/conda/conda-build/issues/2475 + name: sleap + version: {{ data.get('version') }} + +about: + home: {{ data.get('url') }} + license: {{ data.get('license') }} + summary: {{ data.get('description') }} + +build: + number: 5 + +source: + path: ../ + +requirements: + host: + - conda-forge::python ~=3.9 + - anaconda::numpy >=1.19.5,<1.23.0 + - conda-forge::setuptools + - conda-forge::packaging + - conda-forge::pip + + # Only the packages above are required to build, but listing them all ensures no + # unnecessary pypi packages are installed via the build script (bld.bat, build.sh) + - conda-forge::attrs >=21.2.0 + - conda-forge::cattrs ==1.1.1 + - conda-forge::h5py + - conda-forge::imgaug ==0.4.0 + - conda-forge::jsmin + - conda-forge::jsonpickle ==1.2 + - conda-forge::keras <2.10.0,>=2.9.0rc0 # Required by tensorflow-macos + - conda-forge::networkx + - conda-forge::opencv + - conda-forge::pandas + - conda-forge::pillow + - conda-forge::psutil + - conda-forge::pykalman + - conda-forge::pyside2 >=5.12 + - conda-forge::python-rapidjson + - conda-forge::pyyaml + - conda-forge::pyzmq + - conda-forge::qtpy >=2.0.1 + - conda-forge::rich + - conda-forge::scipy >=1.4.1,<=1.9.0 + - conda-forge::scikit-image + - conda-forge::scikit-learn ==1.0 + - conda-forge::scikit-video + - conda-forge::seaborn + + run: + - conda-forge::python ~=3.9 + - conda-forge::attrs >=21.2.0 + - conda-forge::cattrs ==1.1.1 + - conda-forge::h5py + - conda-forge::imgaug ==0.4.0 + - conda-forge::jsmin + - conda-forge::jsonpickle ==1.2 + - conda-forge::keras <2.10.0,>=2.9.0rc0 # Required by tensorflow-macos + - conda-forge::networkx + - anaconda::numpy >=1.19.5,<1.23.0 + - conda-forge::opencv + - conda-forge::pandas + - conda-forge::pillow + - conda-forge::psutil + - conda-forge::pykalman + - conda-forge::pyside2 >=5.12 + - conda-forge::python-rapidjson + - conda-forge::pyyaml + - conda-forge::pyzmq + - conda-forge::qtpy >=2.0.1 + - conda-forge::rich + - conda-forge::scipy >=1.4.1,<=1.9.0 + - conda-forge::scikit-image + - conda-forge::scikit-learn ==1.0 + - conda-forge::scikit-video + - conda-forge::seaborn + - conda-forge::tensorflow-hub + +test: + imports: + - sleap diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 8e2261c6d..91680b64c 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -28,7 +28,7 @@ Please include information about how you installed. - OS: - Version(s): - + - SLEAP installation method (listed [here](https://sleap.ai/installation.html#)): - [ ] [Conda from package](https://sleap.ai/installation.html#conda-package) - [ ] [Conda from source](https://sleap.ai/installation.html#conda-from-source) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index ee9f04385..02bc8798b 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -13,8 +13,14 @@ jobs: strategy: fail-fast: false matrix: - os: ["ubuntu-20.04", "windows-2019"] - # os: ["ubuntu-20.04"] + os: ["ubuntu-22.04", "windows-2022", "macos-latest"] + # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrixinclude + include: + # Use this condarc as default + - condarc: .conda/condarc.yaml + # Use special condarc if macos + - os: "macos-latest" + condarc: .conda_mac/condarc.yaml steps: # Setup - uses: actions/checkout@v2 @@ -33,7 +39,8 @@ jobs: python-version: 3.7 use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly! environment-file: environment_build.yml - activate-environment: sleap + condarc-file: ${{ matrix.condarc }} + activate-environment: sleap_ci - name: Print environment info shell: bash -l {0} run: | @@ -42,42 +49,51 @@ jobs: # Build pip wheel (Ubuntu) - name: Build pip wheel (Ubuntu) - if: matrix.os == 'ubuntu-20.04' + if: matrix.os == 'ubuntu-22.04' shell: bash -l {0} run: | python setup.py bdist_wheel # Upload pip wheel (Ubuntu) - name: Upload pip wheel (Ubuntu) - if: matrix.os == 'ubuntu-20.04' + if: matrix.os == 'ubuntu-22.04' env: PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }} shell: bash -l {0} run: | twine upload -u __token__ -p "$PYPI_TOKEN" dist/* --non-interactive --skip-existing --disable-progress-bar - # Build conda package + # Build conda package (Ubuntu) - name: Build conda package (Ubuntu) - if: matrix.os == 'ubuntu-20.04' + if: matrix.os == 'ubuntu-22.04' shell: bash -l {0} run: | - conda build .conda --output-folder build -c https://conda.anaconda.org/sleap/ -c nvidia -c conda-forge + conda build .conda --output-folder build + + # Build conda package (Windows) - name: Build conda package (Windows) - if: matrix.os == 'windows-2019' + if: matrix.os == 'windows-2022' shell: powershell run: | - conda activate sleap - pytest tests/ - conda build .conda --output-folder build -c https://conda.anaconda.org/sleap/ -c nvidia -c conda-forge + conda build .conda --output-folder build + + # Build conda package (Mac) + - name: Build conda package (Mac) + if: matrix.os == 'macos-latest' + shell: bash -l {0} + run: | + conda build .conda_mac --output-folder build - # Upload conda package + # Login to conda (Ubuntu) - name: Login to Anaconda (Ubuntu) - if: matrix.os == 'ubuntu-20.04' + if: matrix.os == 'ubuntu-22.04' env: ANACONDA_LOGIN: ${{ secrets.ANACONDA_LOGIN }} shell: bash -l {0} run: | yes 2>/dev/null | anaconda login --username sleap --password "$ANACONDA_LOGIN" || true + + # Login to conda (Windows) - name: Login to Anaconda (Windows) if: matrix.os == 'windows-2019' env: @@ -85,6 +101,17 @@ jobs: shell: powershell run: | echo "yes" | anaconda login --username sleap --password "$env:ANACONDA_LOGIN" + + # Login to conda (Mac) + - name: Login to Anaconda (Mac) + if: matrix.os == 'macos-latest' + env: + ANACONDA_LOGIN: ${{ secrets.ANACONDA_LOGIN }} + shell: bash -l {0} + run: | + yes 2>/dev/null | anaconda login --username sleap --password "$ANACONDA_LOGIN" || true + + # Upload conda package (Windows) - name: Upload conda package (Windows/main) if: matrix.os == 'windows-2019' && !github.event.release.prerelease shell: powershell @@ -95,16 +122,32 @@ jobs: shell: powershell run: | anaconda -v upload "build\win-64\*.tar.bz2" --label dev + + # Upload conda package (Ubuntu) - name: Upload conda package (Ubuntu/main) - if: matrix.os == 'ubuntu-20.04' && !github.event.release.prerelease + if: matrix.os == 'ubuntu-22.04' && !github.event.release.prerelease shell: bash -l {0} run: | anaconda -v upload build/linux-64/*.tar.bz2 - name: Upload conda package (Ubuntu/dev) - if: matrix.os == 'ubuntu-20.04' && github.event.release.prerelease + if: matrix.os == 'ubuntu-22.04' && github.event.release.prerelease shell: bash -l {0} run: | anaconda -v upload build/linux-64/*.tar.bz2 --label dev + + # Upload conda package (Mac) + - name: Upload conda package (Mac/main) + if: matrix.os == 'macos-latest' && !github.event.release.prerelease + shell: bash -l {0} + run: | + anaconda -v upload build/osx-64/*.tar.bz2 --label dev + - name: Upload conda package (Mac/dev) + if: matrix.os == 'macos-latest' && github.event.release.prerelease + shell: bash -l {0} + run: | + anaconda -v upload build/osx-64/*.tar.bz2 --label dev + + # Logout - name: Logout from Anaconda shell: bash -l {0} run: | diff --git a/.github/workflows/build_manual.yml b/.github/workflows/build_manual.yml index e7302fdac..ab689342d 100644 --- a/.github/workflows/build_manual.yml +++ b/.github/workflows/build_manual.yml @@ -7,6 +7,7 @@ on: push: paths: - '.conda/meta.yaml' + - '.conda_mac/meta.yaml' branches: - develop @@ -17,8 +18,14 @@ jobs: strategy: fail-fast: false matrix: - os: ["ubuntu-20.04", "windows-2019"] - # os: ["ubuntu-20.04"] + os: ["ubuntu-22.04", "windows-2022", "macos-latest"] + # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrixinclude + include: + # Use this condarc as default + - condarc: .conda/condarc.yaml + # Use special condarc if macos + - os: "macos-latest" + condarc: .conda_mac/condarc.yaml steps: # Setup - uses: actions/checkout@v2 @@ -37,72 +44,99 @@ jobs: python-version: 3.7 use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly! environment-file: environment_build.yml - activate-environment: sleap + condarc-file: ${{ matrix.condarc }} + activate-environment: sleap_ci - name: Print environment info shell: bash -l {0} run: | which python conda info - # Build pip wheel (Ubuntu) - - name: Build pip wheel (Ubuntu) - if: matrix.os == 'ubuntu-20.04' + # Build pip wheel (Not Windows) + - name: Build pip wheel (Not Windows) + if: matrix.os != 'windows-2022' shell: bash -l {0} run: | python setup.py bdist_wheel # Upload pip wheel (Ubuntu) - name: Upload pip wheel (Ubuntu) - if: matrix.os == 'ubuntu-20.04' + if: matrix.os == 'ubuntu-22.04' env: PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }} shell: bash -l {0} run: | twine upload -u __token__ -p "$PYPI_TOKEN" dist/* --non-interactive --skip-existing --disable-progress-bar - # Build conda package + # Build conda package (Ubuntu) - name: Build conda package (Ubuntu) - if: matrix.os == 'ubuntu-20.04' + if: matrix.os == 'ubuntu-22.04' shell: bash -l {0} - # sudo apt install xvfb libxkbcommon-x11-0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-xinerama0 libxcb-xfixes0 - # sudo Xvfb :1 -screen 0 1024x768x24 /dev/null | anaconda login --username sleap --password "$ANACONDA_LOGIN" || true + + # Login to conda (Windows) - name: Login to Anaconda (Windows) - if: matrix.os == 'windows-2019' + if: matrix.os == 'windows-2022' env: ANACONDA_LOGIN: ${{ secrets.ANACONDA_LOGIN }} shell: powershell run: | echo "yes" | anaconda login --username sleap --password "$env:ANACONDA_LOGIN" + + # Login to conda (Mac) + - name: Login to Anaconda (Mac) + if: matrix.os == 'macos-latest' + env: + ANACONDA_LOGIN: ${{ secrets.ANACONDA_LOGIN }} + shell: bash -l {0} + run: | + yes 2>/dev/null | anaconda login --username sleap --password "$ANACONDA_LOGIN" || true + + # Upload conda package (Windows) - name: Upload conda package (Windows/dev) - if: matrix.os == 'windows-2019' + if: matrix.os == 'windows-2022' shell: powershell run: | anaconda -v upload "build\win-64\*.tar.bz2" --label dev + + # Upload conda package (Ubuntu) - name: Upload conda package (Ubuntu/dev) - if: matrix.os == 'ubuntu-20.04' + if: matrix.os == 'ubuntu-22.04' shell: bash -l {0} run: | anaconda -v upload build/linux-64/*.tar.bz2 --label dev + + # Upload conda package (Mac) + - name: Upload conda package (Mac/dev) + if: matrix.os == 'macos-latest' + shell: bash -l {0} + run: | + anaconda -v upload build/osx-64/*.tar.bz2 --label dev + - name: Logout from Anaconda shell: bash -l {0} run: | diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 64caf2bac..faa00412e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -26,11 +26,12 @@ on: jobs: type_check: name: Type Check - runs-on: "ubuntu-20.04" + runs-on: "ubuntu-22.04" steps: - - uses: actions/checkout@v1 + - name: Checkout repo + uses: actions/checkout@v3 - name: Set up Python 3.7 - uses: actions/setup-python@v1 + uses: actions/setup-python@v4 with: python-version: 3.7 - name: Install Dependencies @@ -43,11 +44,12 @@ jobs: mypy --follow-imports=skip --ignore-missing-imports sleap tests lint: name: Lint - runs-on: "ubuntu-20.04" + runs-on: "ubuntu-22.04" steps: - - uses: actions/checkout@v1 + - name: Checkout repo + uses: actions/checkout@v3 - name: Set up Python 3.7 - uses: actions/setup-python@v1 + uses: actions/setup-python@v4 with: python-version: 3.7 - name: Install Dependencies @@ -63,58 +65,51 @@ jobs: strategy: fail-fast: false matrix: - os: ["ubuntu-20.04", "windows-2019"] + os: ["ubuntu-22.04", "windows-2022", "macos-latest"] + include: + # Default values + - env_file: environment_no_cuda.yml + - test_args: --durations=-1 tests/ + # Mac specific values + - os: macos-latest + env_file: environment_mac.yml + # Ubuntu specific values + - os: ubuntu-22.04 + test_args: --cov=sleap --cov-report=xml --durations=-1 tests/ steps: - - name: Checkout - uses: actions/checkout@v2 - - name: Cache conda - uses: actions/cache@v1 - env: - # Increase this value to reset cache if environment_no_cuda.yml has not changed - CACHE_NUMBER: 0 - with: - path: ~/conda_pkgs_dir - key: ${{ runner.os }}-conda-${{ env.CACHE_NUMBER }}-${{ hashFiles('environment_no_cuda.yml', 'requirements.txt') }} - - name: Setup Miniconda - # https://github.com/conda-incubator/setup-miniconda - uses: conda-incubator/setup-miniconda@v2.0.1 + - name: Checkout repo + uses: actions/checkout@v3 + + - name: Setup Micromamba + # https://github.com/mamba-org/setup-micromamba + uses: mamba-org/setup-micromamba@v1 with: - python-version: 3.7 - use-only-tar-bz2: true # IMPORTANT: This needs to be set for caching to work properly! - environment-file: environment_no_cuda.yml - activate-environment: sleap_ci + environment-file: ${{ matrix.env_file }} + environment-name: sleap_ci + init-shell: >- + bash + powershell + post-cleanup: all + + # Print environment info - name: Print environment info shell: bash -l {0} run: | which python - conda info - - name: Conda list - if: matrix.os != 'windows-2019' - shell: pwsh - run: conda list - - name: Test with pytest (Windows) - if: matrix.os == 'windows-2019' + micromamba info + micromamba list + pip freeze + + # Test environment + - name: Test with pytest shell: bash -l {0} run: | - pytest --durations=-1 - - name: Test with pytest (Ubuntu) - if: matrix.os == 'ubuntu-20.04' - shell: - bash -l {0} - # https://pytest-qt.readthedocs.io/en/latest/troubleshooting.html#github-actions - # sudo apt-get install xvfb libxkbcommon-x11-0 - # sudo apt install libxkbcommon-x11-0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-xinerama0 libxcb-xfixes0 - # export DISPLAY=":99.0" - # /sbin/start-stop-daemon --start --quiet --pidfile /tmp/custom_xvfb_99.pid --make-pidfile --background --exec /usr/bin/Xvfb -- :99 -screen 0 1920x1200x24 -ac +extension GLX - # xvfb-run pytest --cov=sleap --cov-report=xml tests/ - run: | - sudo apt install xvfb libxkbcommon-x11-0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-xinerama0 libxcb-xfixes0 - sudo Xvfb :1 -screen 0 1024x768x24 =4.0.0 pytest-cov<=3.0.0 diff --git a/docs/conf.py b/docs/conf.py index 3dc40ff71..bc73ae0d7 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -28,7 +28,7 @@ copyright = f"2019–{date.today().year}, Talmo Lab" # The short X.Y version -version = "1.3.0" +version = "1.3.1" # Get the sleap version # with open("../sleap/version.py") as f: @@ -36,7 +36,7 @@ # version = re.search("\d.+(?=['\"])", version_file).group(0) # Release should be the full branch name -release = "v1.3.0" +release = "v1.3.1" html_title = f"SLEAP ({release})" html_short_title = "SLEAP" diff --git a/docs/guides/cli.md b/docs/guides/cli.md index 17500786b..0c08e9b17 100644 --- a/docs/guides/cli.md +++ b/docs/guides/cli.md @@ -102,9 +102,9 @@ optional arguments: -u, --unrag UNRAG Convert ragged tensors into regular tensors with NaN padding. Defaults to True. - -i, --max_instances MAX_INSTANCES + -n, --max_instances MAX_INSTANCES Limit maximum number of instances in multi-instance models. - Defaults to None. + Not available for ID models. Defaults to None. ``` ## Inference and Tracking @@ -178,6 +178,9 @@ optional arguments: if retracking predictions. --no-empty-frames Clear any empty frames that did not have any detected instances before saving to output. + -n, --max_instances MAX_INSTANCES + Limit maximum number of instances in multi-instance models. + Not available for ID models. Defaults to None. --verbosity {none,rich,json} Verbosity of inference progress reporting. 'none' does not output anything during inference, 'rich' displays diff --git a/docs/installation.md b/docs/installation.md index 06eed1a1e..6918597e8 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -1,8 +1,8 @@ # Installation -SLEAP can be installed as a Python package on Windows, Linux and Mac OS X. We currently provide {ref}`experimental support for Apple Silicon Macs `. +SLEAP can be installed as a Python package on Windows, Linux, Mac OS X, and Mac OS Apple Silicon. -SLEAP requires many complex dependencies, so we **strongly** recommend using [Miniconda](https://docs.conda.io/en/latest/miniconda.html) to install it in its own isolated environment. See {ref}`Installing Miniconda` below for more instructions. +SLEAP requires many complex dependencies, so we **strongly** recommend using [Mambaforge](https://mamba.readthedocs.io/en/latest/installation.html) to install it in its own isolated environment. See {ref}`Installing Mambaforge` below for more instructions. The newest version of SLEAP can always be found in the [Releases page](https://github.com/talmolab/sleap/releases). @@ -12,62 +12,124 @@ local: --- ``` -(miniconda)= +````{hint} +Installation requires entering commands in a terminal. To open one: + +**Windows:** Open the *Start menu* and search for the *Miniforge Prompt* (if using Mambaforge) or the *Command Prompt* if not. +```{note} +On Windows, our personal preference is to use alternative terminal apps like [Cmder](https://cmder.net) or [Windows Terminal](https://aka.ms/terminal). +``` -## Installing Miniconda +**Linux:** Launch a new terminal by pressing Ctrl + Alt + T. + +**Mac:** Launch a new terminal by pressing Cmd + Space and searching for _Terminal_. + +```` + +(apple-silicon)= + +### Macs (Pre-Installation) + +SLEAP can be installed on Macs by following these instructions: + +1. Make sure you're on **macOS Monterey** or later, i.e., version 12+. + +2. If you don't have it yet, [install **homebrew**](https://brew.sh/), a convenient package manager for Macs (skip this if you can run `brew` from the terminal): + + ```bash + /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" + ``` + + This might take a little while since it'll also install Xcode (which we'll need later). Once it's finished, your terminal should give you two extra commands to run listed under **Next Steps**. + + ````{note} + We recommend running the commands given in your terminal which will be similar to (but may differ slightly) from the commands below: + ```bash + echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> ~/.zprofile + ``` + + ```bash + eval "$(/opt/homebrew/bin/brew shellenv)" + ``` + + ```` + + Then, close and re-open the terminal for it to take effect. + +3. Install wget, a CLI downloading utility (also makes sure your homebrew setup worked): + + ```bash + brew install wget + ``` + +(mambaforge)= + +## Installing Mambaforge **Anaconda** is a Python environment manager that makes it easy to install SLEAP and its necessary dependencies without affecting other Python software on your computer. -**Miniconda** is a lightweight version of Anaconda that we recommend. To install it: +[**Mambaforge**](https://mamba.readthedocs.io/en/latest/installation.html) is a lightweight installer of Anaconda with speedy package resolution that we recommend. To install it: -1. Go to: https://docs.conda.io/en/latest/miniconda.html#latest-miniconda-installer-links -2. Download the latest version for your OS. -3. Follow the installer instructions. +**On Windows**, just click through the installation steps. -**On Windows**, just click through the installation steps. We recommend using the following settings: +1. Go to: https://github.com/conda-forge/miniforge#mambaforge +2. Download the latest version for your OS. +3. Follow the installer instructions. + +We recommend using the following settings: - Install for: All Users (requires admin privileges) -- Destination folder: `C:\Miniconda3` -- Advanced Options: Add Miniconda3 to the system PATH environment variable -- Advanced Options: Register Miniconda3 as the system Python 3.X - These will make sure that Anaconda is easily accessible from most places on your computer. +- Destination folder: `C:\mambaforge` +- Advanced Options: Add MambaForge to the system PATH environment variable +- Advanced Options: Register MambaForge as the system Python 3.X + These will make sure that MambaForge is easily accessible from most places on your computer. **On Linux**, it might be easier to do this straight from the terminal (Ctrl + Alt + T) with this one-liner: ```bash -wget -nc https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && bash Miniconda3-latest-Linux-x86_64.sh -b && ~/miniconda3/bin/conda init bash +wget -nc https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh && bash Mambaforge-Linux-x86_64.sh -b && ~/mambaforge/bin/conda init bash ``` Restart the terminal after running this command. -**On Macs**, you can run the graphical installer using the pkg file, or this terminal command: +```{note} +For other Linux architectures (arm64 and POWER8/9), replace the `.sh` filenames above with the correct installer name for your architecture. See the Download column in [this table](https://github.com/conda-forge/miniforge#mambaforge) for the correct filename. -```bash -wget -nc https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh && bash Miniconda3-latest-MacOSX-x86_64.sh -b && ~/miniconda3/bin/conda init zsh ``` -## Installation methods +**On Macs (pre-M1)**, you can run the installer using this terminal command: -````{hint} -Installation requires entering commands in a terminal. To open one: +```bash +wget -nc https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-MacOSX-x86_64.sh && bash Mambaforge-MacOSX-x86_64.sh -b && ~/mambaforge/bin/conda init zsh +``` -**Windows:** Open the *Start menu* and search for the *Anaconda Command Prompt* (if using Miniconda) or the *Command Prompt* if not. -```{note} -On Windows, our personal preference is to use alternative terminal apps like [Cmder](https://cmder.net) or [Windows Terminal](https://aka.ms/terminal). +**On Macs (Apple Silicon)**, use this terminal command: + +```bash +wget -nc https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-MacOSX-arm64.sh && bash Mambaforge-MacOSX-arm64.sh -b && ~/mambaforge/bin/conda init zsh ``` -**Linux:** Launch a new terminal by pressing Ctrl + Alt + T. +## Installation methods -**Mac:** Launch a new terminal by pressing Cmd + Space and searching for *Terminal*. -```` +SLEAP can be installed three different ways: via {ref}`conda package`, {ref}`conda from source`, or {ref}`pip package`. Select one of the methods below to install SLEAP. We recommend {ref}`conda package`. + +(condapackage)= ### `conda` package +**Windows** and **Linux** + +```bash +mamba create -y -n sleap -c conda-forge -c nvidia -c sleap -c anaconda sleap=1.3.1 +``` + +**Mac OS X** and **Apple Silicon** + ```bash -conda create -y -n sleap -c sleap -c nvidia -c conda-forge sleap=1.3.0 +mamba create -y -n sleap -c conda-forge -c anaconda -c sleap sleap=1.3.1 ``` -**This is the recommended installation method**. Works on **Windows** and **Linux**. +**This is the recommended installation method**. ```{note} - This comes with CUDA to enable GPU support. All you need is to have an NVIDIA GPU and [updated drivers](https://nvidia.com/drivers). @@ -75,6 +137,8 @@ conda create -y -n sleap -c sleap -c nvidia -c conda-forge sleap=1.3.0 - This will also work in CPU mode if you don't have a GPU on your machine. ``` +(condasource)= + ### `conda` from source 1. First, ensure git is installed: @@ -91,114 +155,94 @@ conda create -y -n sleap -c sleap -c nvidia -c conda-forge sleap=1.3.0 git clone https://github.com/talmolab/sleap && cd sleap ``` -3. Finally, install from the environment file: +3. Finally, install from the environment file (differs based on OS and GPU): + + **Windows** and **Linux** ```bash - conda env create -f environment.yml -n sleap + mamba env create -f environment.yml -n sleap ``` If you do not have a NVIDIA GPU, then you should use the no CUDA environment file: ```bash - conda env create -f environment_no_cuda.yml -n sleap + mamba env create -f environment_no_cuda.yml -n sleap ``` - This works on **Windows**, **Linux** and **Mac OS X** (pre Apple Silicon). This is the **recommended method for development**. - -```{note} -- This installs SLEAP in development mode, which means that edits to the source code will be applied the next time you run SLEAP. -- Change the `-n sleap` in the command to create an environment with a different name (e.g., `-n sleap_develop`). -``` - -### `pip` package + **Mac OS X** and **Apple Silicon** -```bash -pip install sleap==1.3.0 -``` + ```bash + mamba env create -f environment_mac.yml -n sleap + ``` -This works on **any OS except Apple silicon** and on **Google Colab**. + This is the **recommended method for development**. ```{note} -- Requires Python 3.7 or 3.8. -- To enable GPU support, make sure that you have **CUDA Toolkit v11.3** and **cuDNN v8.2** installed. -``` - -```{warning} -This will uninstall existing libraries and potentially install conflicting ones. - -We strongly recommend that you **only use this method if you know what you're doing**! +- This installs SLEAP in development mode, which means that edits to the source code will be applied the next time you run SLEAP. +- Change the `-n sleap` in the command to create an environment with a different name (e.g., `-n sleap_develop`). ``` -(apple-silicon)= +(pippackage)= -### Apple Silicon Macs +### `pip` package -SLEAP can be installed on newer Apple Silicon Macs by following these instructions: +Although you do not need Mambaforge installed to perform a `pip install`, we recommend {ref}`installing Mambaforge` to create a new environment where we can isolate the `pip install`. If you are working on **Google Colab**, skip to step 3 to perform the `pip install` without using a conda environment. -1. In addition to being on an Apple Silicon Mac, make sure you're on **macOS Monterey**, i.e., version 12+. We've tested this on a MacBook Pro (14-inch, 2021) running macOS version 12.0.1. +1. Otherwise, create a new conda environment where we will `pip install sleap`: -2. If you don't have it yet, install **homebrew**, a convenient package manager for Macs (skip this if you can run `brew` from the terminal): + either without GPU support: ```bash - /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" + mamba create --name sleap pip python=3.7.12 ``` - This might take a little while since it'll also install Xcode (which we'll need later). Once it's finished, run this to enable the `brew` command in your shell, then close and re-open the terminal for it to take effect: + or with GPU support: ```bash - echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> ~/.zprofile && eval "$(/opt/homebrew/bin/brew shellenv)" + mamba create --name sleap pip python=3.7.12 cudatoolkit=11.3 cudnn=8.2 ``` -3. Install wget, a CLI downloading utility (also makes sure your homebrew setup worked): +2. Then activate the environment to isolate the `pip install` from other environments on your computer: ```bash - brew install wget + mamba activate sleap ``` -4. Install the **Apple Silicon Mac version of Miniconda** -- this is important, so make sure you don't have the regular Mac version! If you're not sure, type `which conda` and delete the containing directory to uninstall your existing conda. To install the correct Miniconda, just run: - - ```bash - wget -nc https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh && bash Miniconda3-latest-MacOSX-arm64.sh -b && rm Miniconda3-latest-MacOSX-arm64.sh && ~/miniconda3/bin/conda init zsh + ```{warning} + Refrain from installing anything into the `base` environment. Always create a new environment to install new packages. ``` - Then close and re-open the terminal again. - -5. **Download the SLEAP**: +3. Finally, we can perform the `pip install`: ```bash - cd ~ && git clone https://github.com/talmolab/sleap.git sleap && cd sleap + pip install sleap==1.3.1 ``` -6. **Install SLEAP in a conda environment**: + This works on **any OS except Apple silicon** and on **Google Colab**. - ```bash - conda env create -f environment_apple_silicon.yml + ```{note} + - Requires Python 3.7 + - To enable GPU support, make sure that you have **CUDA Toolkit v11.3** and **cuDNN v8.2** installed. ``` - Your Mac will then automatically sign a devil's pact with Apple to install the correct versions of everything on your system. Once the blood sacrifice/installation process completes, SLEAP will be available in an environment called `sleap`. - - _Note:_ This installs SLEAP in development mode, so changes to the source code are immediately applied in case you wanted to mess around with it. You can also just do a `git pull` to update it (no need to re-do any of the previous steps). + ```{warning} + This will uninstall existing libraries and potentially install conflicting ones. -7. **Test it out** by activating the environment and opening the GUI! - ```bash - conda activate sleap - ``` - ```bash - sleap-label + We strongly recommend that you **only use this method if you know what you're doing**! ``` ## Testing that things are working -If you installed using `conda`, first activate the `sleap` environment by opening a terminal and typing: +If you installed using `mamba`, first activate the `sleap` environment by opening a terminal and typing: ```bash -conda activate sleap +mamba activate sleap ``` ````{hint} -Not sure what `conda` environments you already installed? You can get a list of the environments on your system with: +Not sure what `mamba` environments you already installed? You can get a list of the environments on your system with: ``` -conda env list +mamba env list ``` ```` @@ -231,7 +275,7 @@ python -c "import sleap; sleap.versions()" ### GPU support -Assuming you installed using either of the `conda`-based methods on Windows or Linux, SLEAP should automatically have GPU support enabled. +Assuming you installed using either of the `mamba`-based methods on Windows or Linux, SLEAP should automatically have GPU support enabled. To check, verify that SLEAP can detect the GPUs on your system: @@ -286,13 +330,13 @@ We **strongly recommend** installing SLEAP in a fresh environment when updating. To uninstall an existing environment named `sleap`: ```bash -conda env remove -n sleap +mamba env remove -n sleap ``` ````{hint} -Not sure what `conda` environments you already installed? You can get a list of the environments on your system with: +Not sure what `mamba` environments you already installed? You can get a list of the environments on your system with: ```bash -conda env list +mamba env list ``` ```` @@ -308,10 +352,10 @@ If you get any errors or the GUI fails to launch, try running the diagnostics to sleap-diagnostic ``` -If you were not able to get SLEAP installed, activate the conda environment it is in and generate a list of the package versions installed: +If you were not able to get SLEAP installed, activate the mamba environment it is in and generate a list of the package versions installed: ```bash -conda list +mamba list ``` Then, [open a new Issue](https://github.com/talmolab/sleap/issues) providing the versions from either command above, as well as any errors you saw in the console during the installation. Or [start a discussion](https://github.com/talmolab/sleap/discussions) to get help from the community. diff --git a/environment.yml b/environment.yml index 0ab370245..13cece2df 100644 --- a/environment.yml +++ b/environment.yml @@ -1,23 +1,49 @@ +# Use this environment file if your computer has a nvidia GPU and runs Windows or Linux. + name: sleap +channels: + - conda-forge + - nvidia + - sleap + - anaconda + dependencies: - - python=3.7 - - conda-forge::numpy>=1.19.5,<=1.21.5 - # - conda-forge::tensorflow>=2.6.3,<=2.7.1 - - conda-forge::pyside2>=5.13.2,<=5.14.1 - - conda-forge::h5py=3.1.0 - - conda-forge::scipy>=1.4.1,<=1.7.3 - - conda-forge::pillow=8.4.0 - - shapely=1.7.1 - - conda-forge::pandas - - ffmpeg - - cudatoolkit=11.3.1 - - cudnn=8.2.1 - - nvidia::cuda-nvcc=11.3 - # - sleap::tensorflow=2.7.0 - # - sleap::pyside2=5.14.1 - - qtpy>=2.0.1 - - conda-forge::pip - - pip: - - "--editable=." - - "--requirement=./dev_requirements.txt" + # Packages SLEAP uses directly + - conda-forge::attrs >=21.2.0 #,<=21.4.0 + - conda-forge::cattrs ==1.1.1 + - conda-forge::imgaug ==0.4.0 + - conda-forge::jsmin + - conda-forge::jsonpickle ==1.2 + - conda-forge::networkx + - anaconda::numpy >=1.19.5,<1.23.0 + - conda-forge::opencv + - conda-forge::pandas + - conda-forge::pip + - conda-forge::pillow #>=8.3.1,<=8.4.0 + - conda-forge::psutil + - conda-forge::pykalman + - conda-forge::pyside2 >=5.12 # To ensure application works correctly with QtPy. + - conda-forge::python ~=3.7 # Run into _MAX_WINDOWS_WORKERS not found if == + - conda-forge::python-rapidjson + - conda-forge::pyyaml + - conda-forge::pyzmq + - conda-forge::qtpy >=2.0.1 + - conda-forge::rich + - conda-forge::scipy >=1.4.1,<=1.9.0 + - conda-forge::scikit-image + - conda-forge::scikit-learn ==1.0 + - conda-forge::scikit-video + - conda-forge::seaborn + - sleap::tensorflow >=2.6.3,<2.11 # No windows GPU support for >2.10 + - conda-forge::tensorflow-hub + + # Packages required by tensorflow to find/use GPUs + - conda-forge::cudatoolkit ==11.3.1 + # "==" results in package not found + - conda-forge::cudnn=8.2.1 + - nvidia::cuda-nvcc=11.3 + + - pip: + - "--editable=." + - "--requirement=./dev_requirements.txt" diff --git a/environment_apple_silicon.yml b/environment_apple_silicon.yml deleted file mode 100644 index 1002cbd4f..000000000 --- a/environment_apple_silicon.yml +++ /dev/null @@ -1,20 +0,0 @@ -name: sleap - -channels: - - conda-forge - - defaults - -dependencies: - - python=3.9 - - shapely=1.7.1 - - conda-forge::h5py=3.6.0 - - conda-forge::numpy=1.22.3 - - scipy=1.7.3 - - apple::tensorflow-deps=2.9.0 - - conda-forge::pyside2=5.15.5 - - conda-forge::opencv=4.6.0 - - qtpy>=2.0.1 - - pip - - pip: - - "--editable=./" - - "--requirement=./dev_requirements.txt" \ No newline at end of file diff --git a/environment_build.yml b/environment_build.yml index c2d12d846..b7d6c1ac2 100644 --- a/environment_build.yml +++ b/environment_build.yml @@ -1,25 +1,12 @@ -name: sleap +name: sleap_ci + +channels: + - conda-forge + - anaconda dependencies: - - python=3.7 - - conda-forge::numpy=1.19.5 - - sleap::tensorflow=2.6.3 - - conda-forge::pyside2=5.13.2 - - conda-forge::h5py=3.1.0 - - conda-forge::scipy=1.7.3 - - pillow=8.4.0 - - shapely=1.7.1 - - conda-forge::pandas - - ffmpeg - # - cudatoolkit=11.3.1 - # - cudnn=8.2.1 - # - nvidia::cuda-nvcc=11.3 - - protobuf=3.20.* # https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates - - qtpy>=2.0.1 - - conda-build=3.21.7 + # Needed for the build + - conda-build - anaconda-client - conda-verify - - conda-forge::pip!=22.0.4 - - pip: - - "." - - "--requirement=./dev_requirements.txt" \ No newline at end of file + - twine diff --git a/environment_mac.yml b/environment_mac.yml index 21e403992..611715963 100644 --- a/environment_mac.yml +++ b/environment_mac.yml @@ -1,16 +1,41 @@ +# Use this file if your computer runs Mac OS X or Apple Silicon. + name: sleap +channels: + - conda-forge + - anaconda + dependencies: -- python=3.7 -- pillow=8.4.0 -- shapely=1.7.1 -- ffmpeg -- qtpy>=2.0.1 -- anaconda-client -- conda-verify -- conda-forge::pip!=22.0.4 -- pip: - - tensorflow==2.7.0 - - pyside2==5.14.1 - - "--editable=./" - - "--requirement=./dev_requirements.txt" + # Packages SLEAP uses directly + - conda-forge::attrs >=21.2.0 + - conda-forge::cattrs ==1.1.1 + - conda-forge::h5py + - conda-forge::imgaug ==0.4.0 + - conda-forge::jsmin + - conda-forge::jsonpickle ==1.2 + - conda-forge::keras <2.10.0,>=2.9.0rc0 # Required by tensorflow-macos + - conda-forge::networkx + - anaconda::numpy >=1.19.5,<1.23.0 + - conda-forge::opencv + - conda-forge::pandas + - conda-forge::pip + - conda-forge::pillow + - conda-forge::psutil + - conda-forge::pykalman + - conda-forge::pyside2 >=5.12 # To ensure application works correctly with QtPy. + - conda-forge::python ~=3.9 + - conda-forge::python-rapidjson + - conda-forge::pyyaml + - conda-forge::pyzmq + - conda-forge::qtpy >=2.0.1 + - conda-forge::rich + - conda-forge::scipy >=1.4.1,<=1.9.0 + - conda-forge::scikit-image + - conda-forge::scikit-learn ==1.0 + - conda-forge::scikit-video + - conda-forge::seaborn + - conda-forge::tensorflow-hub + - pip: + - "--editable=./" + - "--requirement=./dev_requirements.txt" diff --git a/environment_no_cuda.yml b/environment_no_cuda.yml index dcded8de9..b3b3bdc08 100644 --- a/environment_no_cuda.yml +++ b/environment_no_cuda.yml @@ -1,23 +1,44 @@ +# Use this environment file if your computer does not have a nvidia GPU and runs Windows +# or Linux. This environment file has exactly the same dependencies listed as +# environment.yaml, minus the packages required by tensorflow to find/use GPUs. + name: sleap_ci +channels: + - conda-forge + - sleap + - anaconda + dependencies: - - python=3.7 - - conda-forge::numpy>=1.19.5,<=1.21.5 - # - conda-forge::tensorflow>=2.6.3,<=2.7.1 - - conda-forge::pyside2>=5.13.2,<=5.14.1 - - conda-forge::h5py=3.1.0 - - conda-forge::scipy>=1.4.1,<=1.7.3 - - conda-forge::pillow=8.4.0 - - shapely=1.7.1 - - conda-forge::pandas - - ffmpeg - # - cudatoolkit=11.3.1 - # - cudnn=8.2.1 - # - nvidia::cuda-nvcc=11.3 - # - sleap::tensorflow=2.7.0 - # - sleap::pyside2=5.14.1 - - qtpy>=2.0.1 - - conda-forge::pip # !=22.0.4 - - pip: - - "--editable=." - - "--requirement=./dev_requirements.txt" + # Packages SLEAP uses directly + - conda-forge::attrs >=21.2.0 #,<=21.4.0 + - conda-forge::cattrs ==1.1.1 + - conda-forge::imgaug ==0.4.0 + - conda-forge::jsmin + - conda-forge::jsonpickle ==1.2 + - conda-forge::networkx + - anaconda::numpy >=1.19.5,<1.23.0 + - conda-forge::opencv + - conda-forge::pandas + - conda-forge::pip + - conda-forge::pillow #>=8.3.1,<=8.4.0 + - conda-forge::psutil + - conda-forge::pykalman + - conda-forge::pyside2 >=5.12 # To ensure application works correctly with QtPy. + - conda-forge::python ~=3.7 # Run into _MAX_WINDOWS_WORKERS not found if == + - conda-forge::python-rapidjson + - conda-forge::pyyaml + - conda-forge::pyzmq + - conda-forge::qtpy >=2.0.1 + - conda-forge::rich + - conda-forge::scipy >=1.4.1,<=1.9.0 + - conda-forge::scikit-image + - conda-forge::scikit-learn ==1.0 + - conda-forge::scikit-video + - conda-forge::seaborn + - sleap::tensorflow >=2.6.3,<2.11 # No windows GPU support for >2.10 + - conda-forge::tensorflow-hub + + - pip: + - "--editable=." + - "--requirement=./dev_requirements.txt" diff --git a/pip_requirements.txt b/pip_requirements.txt new file mode 100644 index 000000000..1e6007118 --- /dev/null +++ b/pip_requirements.txt @@ -0,0 +1,34 @@ +# This file contains the full list of dependencies to be installed when only using pypi. +# This file should look very similar to the environment.yml file. Based on the logic in +# setup.py, the packages in requirements.txt will also be installed when running +# pip install sleap[pip]. + +# These are also distrubuted through conda and not pip installed when using conda. +attrs>=21.2.0,<=21.4.0 +cattrs==1.1.1 +# certifi>=2017.4.17,<=2021.10.8 +jsmin +jsonpickle==1.2 +networkx +numpy>=1.19.5,<1.23.0 +opencv-python>=4.2.0,<=4.6.0 +# opencv-python-headless>=4.2.0.34,<=4.5.5.62 +pandas +pillow>=8.3.1,<=8.4.0 +psutil +pykalman==0.9.5 +PySide2>=5.13.2,<=5.14.1; platform_machine != 'arm64' +PySide6; sys_platform == 'darwin' and platform_machine == 'arm64' +python-rapidjson +pyyaml +pyzmq +qtpy>=2.0.1 +rich==10.16.1 +imageio<=2.15.0 +imgaug==0.4.0 +scipy>=1.4.1,<=1.9.0 +scikit-image +scikit-learn ==1.0.* +scikit-video +seaborn +tensorflow-hub diff --git a/requirements.txt b/requirements.txt index e9f4b3e9b..cb0ef45c5 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,38 +1,14 @@ -numpy>=1.19.5,<1.23.0 -attrs>=21.2.0,<=21.4.0 -cattrs==1.1.1 -jsonpickle==1.2 -networkx -tensorflow>=2.6.3,<2.9.0; platform_machine != 'arm64' -tensorflow-hub +# This file contains the minimal requirements to be installed via pip when using conda. + +# No conda packages for these +imgstore<0.3.0 # 0.3.3 results in https://github.com/O365/python-o365/issues/591 +ndx-pose +nixio>=1.5.3 # Constrain put on by @jgrewe from G-Node +qimage2ndarray # ==1.9.0 +segmentation-models tensorflow-macos==2.9.2; sys_platform == 'darwin' and platform_machine == 'arm64' tensorflow-metal==0.5.0; sys_platform == 'darwin' and platform_machine == 'arm64' -h5py>=3.1.0,<=3.7.0 -python-rapidjson -opencv-python>=4.2.0,<=4.6.0 -# opencv-python-headless>=4.2.0.34,<=4.5.5.62 -pandas -psutil -PySide2>=5.13.2,<=5.14.1; platform_machine != 'arm64' -PySide2>=5.13.2,<=5.15.5; sys_platform == 'darwin' and platform_machine == 'arm64' -qtpy>=2.0.1 -pyzmq -pyyaml -pillow>=8.3.1,<=8.4.0 -imageio<=2.15.0 -imgaug==0.4.0 -scipy>=1.4.1,<=1.9.0 -scikit-image -scikit-learn==1.0.* -scikit-video -imgstore==0.2.9 -qimage2ndarray>=1.9.0 -jsmin -seaborn -pykalman==0.9.5 -segmentation-models==1.0.1 -rich==10.16.1 -certifi>=2017.4.17,<=2021.10.8 -pynwb -ndx-pose -nixio>=1.5.3 + +# Conda installing results in https://github.com/h5py/h5py/issues/2037 +h5py<3.2; sys_platform == 'win32' # Newer versions result in error above, linking issue in Linux +pynwb>=2.3.3 # 2.0.0 required by ndx-pose, 2.3.3 fixes importlib-metadata incompatibility diff --git a/setup.py b/setup.py index 1af316128..6145f3a3a 100644 --- a/setup.py +++ b/setup.py @@ -7,6 +7,7 @@ from os import path import re +import sys here = path.abspath(path.dirname(__file__)) @@ -30,9 +31,10 @@ def get_requirements(require_name=None): name="sleap", version=sleap_version, setup_requires=["setuptools_scm"], - install_requires=get_requirements(), + install_requires=get_requirements(), # Minimal requirements if using conda. extras_require={ - "dev": get_requirements("dev"), + "pip": get_requirements("pip"), # For pip install + "dev": get_requirements("pip") + get_requirements("dev"), }, description="SLEAP (Social LEAP Estimates Animal Poses) is a deep learning framework for animal pose tracking.", long_description=long_description, @@ -47,9 +49,9 @@ def get_requirements(require_name=None): url="https://sleap.ai", keywords="deep learning, pose estimation, tracking, neuroscience", license="BSD 3-Clause License", - packages=find_packages(exclude=["tensorflow"]), + packages=find_packages(exclude=["tensorflow", "tests", "tests.*", "docs"]), include_package_data=True, - entry_points={ + entry_points={ "console_scripts": [ "sleap-convert=sleap.io.convert:main", "sleap-render=sleap.io.visuals:main", diff --git a/sleap/config/pipeline_form.yaml b/sleap/config/pipeline_form.yaml index fe5eb4263..77722f0d4 100644 --- a/sleap/config/pipeline_form.yaml +++ b/sleap/config/pipeline_form.yaml @@ -13,6 +13,14 @@ training: a "confidence map" head to predicts the nodes for an entire image and a "part affinity field" head to group the nodes into distinct animal instances.' + - label: Max Instances + name: max_instances + type: optional_int + help: Maximum number of instances per frame. + none_label: No max + default_disabled: true + range: 1,100 + default: 1 - name: model.heads.multi_instance.confmaps.sigma label: Sigma for Nodes @@ -44,7 +52,15 @@ training: This pipeline uses two models: a "centroid" model to locate and crop around each animal in the frame, and a "centered-instance confidence map" model for predicted node locations - for each individual animal predicted by the centroid model.' + for each individual animal predicted by the centroid model.' + - label: Max Instances + name: max_instances + type: optional_int + help: Maximum number of instances per frame. + none_label: No max + default_disabled: true + range: 1,100 + default: 1 - default: 5.0 help: Spread of the Gaussian distribution of the confidence maps as a scalar float. @@ -280,6 +296,14 @@ inference: a "confidence map" head to predicts the nodes for an entire image and a "part affinity field" head to group the nodes into distinct animal instances.' + - label: Max Instances + name: max_instances + type: optional_int + help: Maximum number of instances per frame. + none_label: No max + default_disabled: true + range: 1,100 + default: 1 multi-animal top-down: - type: text @@ -288,6 +312,14 @@ inference: locate and crop around each animal in the frame, and a "centered-instance confidence map" model for predicted node locations for each individual animal predicted by the centroid model.' + - label: Max Instances + name: max_instances + type: optional_int + help: Maximum number of instances per frame. + none_label: No max + default_disabled: true + range: 1,100 + default: 1 multi-animal bottom-up-id: - type: text diff --git a/sleap/config/training_editor_form.yaml b/sleap/config/training_editor_form.yaml index bb4bc6e6c..d10b840a0 100644 --- a/sleap/config/training_editor_form.yaml +++ b/sleap/config/training_editor_form.yaml @@ -20,7 +20,7 @@ data: name: data.instance_cropping.crop_size type: optional_int none_label: Auto - range: 0,512 + range: 0,832 model: - default: unet diff --git a/sleap/gui/app.py b/sleap/gui/app.py index a14fc007a..b82372511 100644 --- a/sleap/gui/app.py +++ b/sleap/gui/app.py @@ -56,39 +56,29 @@ from qtpy import QtCore, QtGui from qtpy.QtCore import Qt, QEvent -from qtpy.QtWidgets import QApplication, QMainWindow, QWidget, QDockWidget -from qtpy.QtWidgets import QVBoxLayout, QHBoxLayout, QGroupBox, QTabWidget -from qtpy.QtWidgets import QLabel, QPushButton, QComboBox +from qtpy.QtWidgets import QApplication, QMainWindow from qtpy.QtWidgets import QMessageBox import sleap from sleap.gui.dialogs.metrics import MetricsTableDialog from sleap.skeleton import Skeleton -from sleap.instance import Instance, LabeledFrame +from sleap.instance import Instance from sleap.io.dataset import Labels +from sleap.io.video import available_video_exts from sleap.info.summary import StatisticSeries from sleap.gui.commands import CommandContext, UpdateTopic -from sleap.gui.widgets.views import CollapsibleWidget +from sleap.gui.widgets.docks import ( + InstancesDock, + SkeletonDock, + SuggestionsDock, + VideosDock, +) from sleap.gui.widgets.video import QtVideoPlayer from sleap.gui.widgets.slider import set_slider_marks_from_labels -from sleap.gui.dataviews import ( - GenericTableView, - VideosTableModel, - SkeletonNodesTableModel, - SkeletonEdgesTableModel, - SuggestionsTableModel, - LabeledFrameTableModel, - SkeletonNodeModel, -) -from sleap.util import ( - parse_uri_path, - decode_preview_image, - get_package_file, - find_files_by_suffix, -) +from sleap.util import parse_uri_path from sleap.gui.dialogs.filedialog import FileDialog -from sleap.gui.dialogs.formbuilder import YamlFormWidget, FormBuilderModalDialog +from sleap.gui.dialogs.formbuilder import FormBuilderModalDialog from sleap.gui.shortcuts import Shortcuts from sleap.gui.dialogs.shortcuts import ShortcutDialog from sleap.gui.state import GuiState @@ -171,6 +161,7 @@ def __init__( self.state["share usage data"] = False self.state["clipboard_track"] = None self.state["clipboard_instance"] = None + self.state.connect("marker size", self.plotFrame) self.state.connect("node label size", self.plotFrame) self.state.connect("show non-visible nodes", self.plotFrame) @@ -275,8 +266,6 @@ def dropEvent(self, event): exts = [Path(f).suffix for f in filenames] - VIDEO_EXTS = (".mp4", ".avi", ".h5") # TODO: make this list global - if len(exts) == 1 and exts[0].lower() == ".slp": if self.state["project_loaded"]: # Merge @@ -285,7 +274,7 @@ def dropEvent(self, event): # Load self.commands.openProject(filename=filenames[0], first_open=True) - elif all([ext.lower() in VIDEO_EXTS for ext in exts]): + elif all([ext.lower() in available_video_exts() for ext in exts]): # Import videos self.commands.showImportVideos(filenames=filenames) @@ -338,7 +327,11 @@ def switch_frame(video): def update_frame_chunk_suggestions(video): """Set upper limit of frame_chunk spinbox to number frames in video.""" - method_layout = self.suggestions_form_widget.form_layout.fields["method"] + method_layout = ( + self.suggestions_dock.suggestions_form_widget.form_layout.fields[ + "method" + ] + ) frame_chunk_layout = method_layout.page_layouts["frame chunk"] frame_to_spinbox = frame_chunk_layout.fields["frame_to"] frame_from_spinbox = frame_chunk_layout.fields["frame_from"] @@ -993,317 +986,13 @@ def wrapped_function(*args): def _create_dock_windows(self): """Create dock windows and connect them to GUI.""" - def _make_dock(name, widgets=[], tab_with=None): - dock = QDockWidget(name) - dock.setObjectName(name + "Dock") - - dock.setAllowedAreas(Qt.LeftDockWidgetArea | Qt.RightDockWidgetArea) - - dock_widget = QWidget() - dock_widget.setObjectName(name + "Widget") - layout = QVBoxLayout() - - for widget in widgets: - layout.addWidget(widget) - - dock_widget.setLayout(layout) - dock.setWidget(dock_widget) - - self.addDockWidget(Qt.RightDockWidgetArea, dock) - self.viewMenu.addAction(dock.toggleViewAction()) - - if tab_with is not None: - self.tabifyDockWidget(tab_with, dock) - - return layout - - def _add_button(to, label, action, key=None): - key = key or label.lower() - btn = QPushButton(label) - btn.clicked.connect(action) - to.addWidget(btn) - self._buttons[key] = btn - return btn - - ####### Videos ####### - videos_layout = _make_dock("Videos") - self.videosTable = GenericTableView( - state=self.state, - row_name="video", - is_activatable=True, - model=VideosTableModel(items=self.labels.videos, context=self.commands), - ellipsis_left=True, - ) - videos_layout.addWidget(self.videosTable) - - hb = QHBoxLayout() - _add_button(hb, "Toggle Grayscale", self.commands.toggleGrayscale) - _add_button(hb, "Show Video", self.videosTable.activateSelected) - _add_button(hb, "Add Videos", self.commands.addVideo) - _add_button(hb, "Remove Video", self.commands.removeVideo) - - hbw = QWidget() - hbw.setLayout(hb) - videos_layout.addWidget(hbw) - - ####### Skeleton ####### - skeleton_layout = _make_dock( - "Skeleton", tab_with=videos_layout.parent().parent() - ) - - gb = CollapsibleWidget("Templates") - vb = QVBoxLayout() - hb = QHBoxLayout() - - skeletons_folder = get_package_file("sleap/skeletons") - skeletons_json_files = find_files_by_suffix( - skeletons_folder, suffix=".json", depth=1 - ) - skeletons_names = [json.name.split(".")[0] for json in skeletons_json_files] - self.skeletonTemplates = QComboBox() - self.skeletonTemplates.addItems(skeletons_names) - self.skeletonTemplates.setEditable(False) - hb.addWidget(self.skeletonTemplates) - _add_button(hb, "Load", self.commands.openSkeletonTemplate) - hbw = QWidget() - hbw.setLayout(hb) - vb.addWidget(hbw) - - hb = QHBoxLayout() - self.skeleton_preview_image = QLabel("Preview Skeleton") - hb.addWidget(self.skeleton_preview_image) - hb.setAlignment(self.skeleton_preview_image, Qt.AlignLeft) - - self.skeleton_description = QLabel( - f'Description: {self.state["skeleton_description"]}' - ) - self.skeleton_description.setWordWrap(True) - hb.addWidget(self.skeleton_description) - hb.setAlignment(self.skeleton_description, Qt.AlignLeft) - - hbw = QWidget() - hbw.setLayout(hb) - vb.addWidget(hbw) - - def updatePreviewImage(preview_image_bytes: bytes): - - # Decode the preview image - preview_image = decode_preview_image(preview_image_bytes) - - # Create a QImage from the Image - preview_image = QtGui.QImage( - preview_image.tobytes(), - preview_image.size[0], - preview_image.size[1], - QtGui.QImage.Format_RGBA8888, # Format for RGBA images (see Image.mode) - ) - - preview_image = QtGui.QPixmap.fromImage(preview_image) - - self.skeleton_preview_image.setPixmap(preview_image) - - gb.set_content_layout(vb) - skeleton_layout.addWidget(gb) - - def update_skeleton_preview(idx: int): - skel = Skeleton.load_json(skeletons_json_files[idx]) - self.state["skeleton_description"] = ( - f"Description: {skel.description}

" - f"Nodes ({len(skel)}): {', '.join(skel.node_names)}" - ) - self.skeleton_description.setText(self.state["skeleton_description"]) - updatePreviewImage(skel.preview_image) - - self.skeletonTemplates.currentIndexChanged.connect(update_skeleton_preview) - update_skeleton_preview(idx=0) - - gb = QGroupBox("Project Skeleton") - vgb = QVBoxLayout() - - nodes_widget = QWidget() - vb = QVBoxLayout() - graph_tabs = QTabWidget() - self.skeletonNodesTable = GenericTableView( - state=self.state, - row_name="node", - model=SkeletonNodesTableModel( - items=self.state["skeleton"], context=self.commands - ), - ) - vb.addWidget(self.skeletonNodesTable) - hb = QHBoxLayout() - _add_button(hb, "New Node", self.commands.newNode) - _add_button(hb, "Delete Node", self.commands.deleteNode) - - hbw = QWidget() - hbw.setLayout(hb) - vb.addWidget(hbw) - nodes_widget.setLayout(vb) - graph_tabs.addTab(nodes_widget, "Nodes") - - def _update_edge_src(): - self.skeletonEdgesDst.model().skeleton = self.state["skeleton"] - - edges_widget = QWidget() - vb = QVBoxLayout() - self.skeletonEdgesTable = GenericTableView( - state=self.state, - row_name="edge", - model=SkeletonEdgesTableModel( - items=self.state["skeleton"], context=self.commands - ), - ) - - vb.addWidget(self.skeletonEdgesTable) - hb = QHBoxLayout() - self.skeletonEdgesSrc = QComboBox() - self.skeletonEdgesSrc.setEditable(False) - self.skeletonEdgesSrc.currentIndexChanged.connect(_update_edge_src) - self.skeletonEdgesSrc.setModel(SkeletonNodeModel(self.state["skeleton"])) - hb.addWidget(self.skeletonEdgesSrc) - hb.addWidget(QLabel("to")) - self.skeletonEdgesDst = QComboBox() - self.skeletonEdgesDst.setEditable(False) - hb.addWidget(self.skeletonEdgesDst) - self.skeletonEdgesDst.setModel( - SkeletonNodeModel( - self.state["skeleton"], lambda: self.skeletonEdgesSrc.currentText() - ) - ) - - def new_edge(): - src_node = self.skeletonEdgesSrc.currentText() - dst_node = self.skeletonEdgesDst.currentText() - self.commands.newEdge(src_node, dst_node) - - _add_button(hb, "Add Edge", new_edge) - _add_button(hb, "Delete Edge", self.commands.deleteEdge) - hbw = QWidget() - hbw.setLayout(hb) - vb.addWidget(hbw) - edges_widget.setLayout(vb) - graph_tabs.addTab(edges_widget, "Edges") - vgb.addWidget(graph_tabs) - - hb = QHBoxLayout() - _add_button(hb, "Load From File...", self.commands.openSkeleton) - _add_button(hb, "Save As...", self.commands.saveSkeleton) - - hbw = QWidget() - hbw.setLayout(hb) - vgb.addWidget(hbw) - - # Add graph tabs to "Project Skeleton" group box - gb.setLayout(vgb) - skeleton_layout.addWidget(gb) - - ####### Suggestions ####### - suggestions_layout = _make_dock( - "Labeling Suggestions", tab_with=videos_layout.parent().parent() - ) - self.suggestionsTable = GenericTableView( - state=self.state, - is_sortable=True, - model=SuggestionsTableModel( - items=self.labels.suggestions, context=self.commands - ), - ) - - suggestions_layout.addWidget(self.suggestionsTable) - - hb = QHBoxLayout() - - _add_button( - hb, - "Add current frame", - self.process_events_then(self.commands.addCurrentFrameAsSuggestion), - "add current frame as suggestion", - ) - - _add_button( - hb, - "Remove", - self.process_events_then(self.commands.removeSuggestion), - "remove suggestion", - ) - - _add_button( - hb, - "Clear all", - self.process_events_then(self.commands.clearSuggestions), - "clear suggestions", - ) - - hbw = QWidget() - hbw.setLayout(hb) - suggestions_layout.addWidget(hbw) - - hb = QHBoxLayout() - - _add_button( - hb, - "Previous", - self.process_events_then(self.commands.prevSuggestedFrame), - "goto previous suggestion", - ) - - self.suggested_count_label = QLabel() - hb.addWidget(self.suggested_count_label) - - _add_button( - hb, - "Next", - self.process_events_then(self.commands.nextSuggestedFrame), - "goto next suggestion", - ) - - hbw = QWidget() - hbw.setLayout(hb) - suggestions_layout.addWidget(hbw) - - self.suggestions_form_widget = YamlFormWidget.from_name( - "suggestions", - title="Generate Suggestions", - ) - self.suggestions_form_widget.mainAction.connect( - self.process_events_then(self.commands.generateSuggestions) - ) - suggestions_layout.addWidget(self.suggestions_form_widget) - - def goto_suggestion(*args): - selected_frame = self.suggestionsTable.getSelectedRowItem() - self.commands.gotoVideoAndFrame( - selected_frame.video, selected_frame.frame_idx - ) - - self.suggestionsTable.doubleClicked.connect(goto_suggestion) - - self.state.connect("suggestion_idx", self.suggestionsTable.selectRow) - - ####### Instances ####### - instances_layout = _make_dock( - "Instances", tab_with=videos_layout.parent().parent() - ) - self.instancesTable = GenericTableView( - state=self.state, - row_name="instance", - name_prefix="", - model=LabeledFrameTableModel( - items=self.state["labeled_frame"], context=self.commands - ), - ) - instances_layout.addWidget(self.instancesTable) - - hb = QHBoxLayout() - _add_button(hb, "New Instance", lambda x: self.commands.newInstance()) - _add_button(hb, "Delete Instance", self.commands.deleteSelectedInstance) - - hbw = QWidget() - hbw.setLayout(hb) - instances_layout.addWidget(hbw) + self.videos_dock = VideosDock(self) + self.skeleton_dock = SkeletonDock(self, tab_with=self.videos_dock) + self.suggestions_dock = SuggestionsDock(self, tab_with=self.videos_dock) + self.instances_dock = InstancesDock(self, tab_with=self.videos_dock) # Bring videos tab forward. - videos_layout.parent().parent().raise_() + self.videos_dock.wgt_layout.parent().parent().raise_() def _load_overlays(self): """Load all standard video overlays.""" @@ -1377,8 +1066,8 @@ def _update_gui_state(self): ) # todo: exclude predicted instances from count has_nodes_selected = ( - self.skeletonEdgesSrc.currentIndex() > -1 - and self.skeletonEdgesDst.currentIndex() > -1 + self.skeleton_dock.skeletonEdgesSrc.currentIndex() > -1 + and self.skeleton_dock.skeletonEdgesDst.currentIndex() > -1 ) control_key_down = QApplication.queryKeyboardModifiers() == Qt.ControlModifier @@ -1415,7 +1104,9 @@ def _update_gui_state(self): self._buttons["show video"].setEnabled(has_selected_video) self._buttons["remove video"].setEnabled(has_selected_video) self._buttons["delete instance"].setEnabled(has_selected_instance) - self.suggestions_form_widget.buttons["generate_button"].setEnabled(has_videos) + self.suggestions_dock.suggestions_form_widget.buttons[ + "generate_button" + ].setEnabled(has_videos) # Update overlays self.overlays["track_labels"].visible = ( @@ -1457,24 +1148,28 @@ def _has_topic(topic_list): self._update_track_menu() if _has_topic([UpdateTopic.video]): - self.videosTable.model().items = self.labels.videos + self.videos_dock.table.model().items = self.labels.videos if _has_topic([UpdateTopic.skeleton]): - self.skeletonNodesTable.model().items = self.state["skeleton"] - self.skeletonEdgesTable.model().items = self.state["skeleton"] - self.skeletonEdgesSrc.model().skeleton = self.state["skeleton"] - self.skeletonEdgesDst.model().skeleton = self.state["skeleton"] + self.skeleton_dock.nodes_table.model().items = self.state["skeleton"] + self.skeleton_dock.edges_table.model().items = self.state["skeleton"] + self.skeleton_dock.skeletonEdgesSrc.model().skeleton = self.state[ + "skeleton" + ] + self.skeleton_dock.skeletonEdgesDst.model().skeleton = self.state[ + "skeleton" + ] if self.labels.skeletons: - self.suggestions_form_widget.set_field_options( + self.suggestions_dock.suggestions_form_widget.set_field_options( "node", self.labels.skeletons[0].node_names ) if _has_topic([UpdateTopic.project, UpdateTopic.on_frame]): - self.instancesTable.model().items = self.state["labeled_frame"] + self.instances_dock.table.model().items = self.state["labeled_frame"] if _has_topic([UpdateTopic.suggestions]): - self.suggestionsTable.model().items = self.labels.suggestions + self.suggestions_dock.table.model().items = self.labels.suggestions if _has_topic([UpdateTopic.project_instances, UpdateTopic.suggestions]): # update count of suggested frames w/ labeled instances @@ -1492,7 +1187,7 @@ def _has_topic(topic_list): suggestion_status_text = ( f"{labeled_count}/{len(suggestion_list)} labeled ({prc:.1f}%)" ) - self.suggested_count_label.setText(suggestion_status_text) + self.suggestions_dock.suggested_count_label.setText(suggestion_status_text) if _has_topic([UpdateTopic.frame, UpdateTopic.project_instances]): self.state["last_interacted_frame"] = self.state["labeled_frame"] diff --git a/sleap/gui/commands.py b/sleap/gui/commands.py index 59c377d39..c453e4e8e 100644 --- a/sleap/gui/commands.py +++ b/sleap/gui/commands.py @@ -2716,7 +2716,7 @@ class RemoveSuggestion(EditCommand): @classmethod def do_action(cls, context: CommandContext, params: dict): - selected_frame = context.app.suggestionsTable.getSelectedRowItem() + selected_frame = context.app.suggestions_dock.table.getSelectedRowItem() if selected_frame is not None: context.labels.remove_suggestion( selected_frame.video, selected_frame.frame_idx diff --git a/sleap/gui/dialogs/importvideos.py b/sleap/gui/dialogs/importvideos.py index 4f396995b..005b75082 100644 --- a/sleap/gui/dialogs/importvideos.py +++ b/sleap/gui/dialogs/importvideos.py @@ -32,7 +32,15 @@ ) from sleap.gui.widgets.video import GraphicsView -from sleap.io.video import Video +from sleap.io.video import ( + Video, + MediaVideo, + HDF5Video, + NumpyVideo, + ImgStoreVideo, + SingleImageVideo, + available_video_exts, +) from sleap.gui.dialogs.filedialog import FileDialog import h5py @@ -67,11 +75,25 @@ def ask( messages = dict() if messages is None else messages if filenames is None: + + any_video_exts = " ".join(["*." + ext for ext in available_video_exts()]) + media_video_exts = " ".join(["*." + ext for ext in MediaVideo.EXTS]) + hdf5_video_exts = " ".join(["*." + ext for ext in HDF5Video.EXTS]) + numpy_video_exts = " ".join(["*." + ext for ext in NumpyVideo.EXTS]) + imgstore_video_exts = " ".join(["*." + ext for ext in ImgStoreVideo.EXTS]) + siv_video_exts = " ".join(["*." + ext for ext in SingleImageVideo.EXTS]) + filenames, filter = FileDialog.openMultiple( None, "Select videos to import...", # dialogue title ".", # initial path - "Any Video (*.h5 *.hd5v *.mp4 *.avi *.json);;HDF5 (*.h5 *.hd5v);;ImgStore (*.json);;Media Video (*.mp4 *.avi);;Any File (*.*)", + f"Any Video ({any_video_exts});;" + f"Media ({media_video_exts});;" + f"HDF5 ({hdf5_video_exts});;" + f"Numpy ({numpy_video_exts});;" + f"ImgStore ({imgstore_video_exts});;" + f"Single image ({siv_video_exts});;" + "Any File (*.*)", ) if len(filenames) > 0: @@ -113,7 +135,7 @@ def __init__( self.import_types = [ { "video_type": "hdf5", - "match": "h5,hdf5", + "match": ",".join(HDF5Video.EXTS), "video_class": Video.from_hdf5, "params": [ { @@ -132,19 +154,19 @@ def __init__( }, { "video_type": "mp4", - "match": "mp4,avi", + "match": ",".join(MediaVideo.EXTS), "video_class": Video.from_media, "params": [{"name": "grayscale", "type": "check"}], }, { "video_type": "imgstore", - "match": "json", + "match": ",".join(ImgStoreVideo.EXTS), "video_class": Video.from_filename, "params": [], }, { "video_type": "single_image", - "match": "jpg,png,tif,jpeg,tiff", + "match": ",".join(SingleImageVideo.EXTS), "video_class": Video.from_filename, "params": [{"name": "grayscale", "type": "check"}], }, @@ -635,29 +657,3 @@ def plot(self, idx=0): # Display image self.view.setImage(image) - - -# if __name__ == "__main__": - -# app = QApplication([]) - -# # import_list = ImportVideos().ask() - -# filenames = [ -# "tests/data/videos/centered_pair_small.mp4", -# "tests/data/videos/small_robot.mp4", -# ] - -# messages = {"tests/data/videos/small_robot.mp4": "Testing messages"} - -# import_list = [] -# importer = ImportParamDialog(filenames, messages=messages) -# importer.accepted.connect(lambda: importer.get_data(import_list)) -# importer.exec_() - -# for import_item in import_list: -# vid = import_item["video_class"](**import_item["params"]) -# print( -# "Imported video data: (%d, %d), %d f, %d c" -# % (vid.width, vid.height, vid.frames, vid.channels) -# ) diff --git a/sleap/gui/dialogs/missingfiles.py b/sleap/gui/dialogs/missingfiles.py index 440a1a3eb..0451e09f9 100644 --- a/sleap/gui/dialogs/missingfiles.py +++ b/sleap/gui/dialogs/missingfiles.py @@ -234,10 +234,3 @@ def headerData( elif orientation == QtCore.Qt.Vertical: return section return None - - -# if __name__ == "__main__": -# app = QtWidgets.QApplication() -# win = MissingFilesDialog(["m:/centered_pair_small.mp4", "m:/small_robot.mp4"]) -# result = win.exec_() -# print(result) diff --git a/sleap/gui/learning/dialog.py b/sleap/gui/learning/dialog.py index 5fdbf43fa..26531872c 100644 --- a/sleap/gui/learning/dialog.py +++ b/sleap/gui/learning/dialog.py @@ -1245,17 +1245,25 @@ def trained_config_info_to_use(self) -> Optional[configs.ConfigFileInfo]: if self._cfg_list_widget is None: return None - trained_config_info: Optional[ + selected_config_info: Optional[ configs.ConfigFileInfo ] = self._cfg_list_widget.getSelectedConfigInfo() - if (trained_config_info is None) or (not trained_config_info.has_trained_model): + if (selected_config_info is None) or ( + not selected_config_info.has_trained_model + ): return None + trained_config_info = configs.ConfigFileInfo.from_config_file( + selected_config_info.path + ) if self.use_trained: trained_config_info.dont_retrain = True else: # Set certain parameters to defaults trained_config = trained_config_info.config + trained_config.data.labels.validation_labels = None + trained_config.data.labels.test_labels = None + trained_config.data.labels.split_by_inds = False trained_config.data.labels.skeletons = [] trained_config.outputs.run_name = None trained_config.outputs.run_name_prefix = "" diff --git a/sleap/gui/learning/receptivefield.py b/sleap/gui/learning/receptivefield.py index 9b9770092..025f09ae9 100644 --- a/sleap/gui/learning/receptivefield.py +++ b/sleap/gui/learning/receptivefield.py @@ -227,20 +227,3 @@ def _set_field_size(self, size: Optional[int] = None, scale: float = 1.0): self.box.setRect( scene_center.x(), scene_center.y(), scaled_box_size, scaled_box_size ) - - -def demo_receptive_field(): - app = QtWidgets.QApplication([]) - - video = Video.from_filename("tests/data/videos/centered_pair_small.mp4") - - win = ReceptiveFieldImageWidget() - win.setImage(video.get_frame(0)) - win._set_field_size(50) - - win.show() - app.exec_() - - -if __name__ == "__main__": - demo_receptive_field() diff --git a/sleap/gui/learning/runners.py b/sleap/gui/learning/runners.py index 83e56d995..460ca7e5a 100644 --- a/sleap/gui/learning/runners.py +++ b/sleap/gui/learning/runners.py @@ -1,6 +1,4 @@ -""" -Run training/inference in background process via CLI. -""" +"""Run training/inference in background process via CLI.""" import abc import attr import os @@ -41,8 +39,7 @@ def kill_process(pid: int): @attr.s(auto_attribs=True) class ItemForInference(abc.ABC): - """ - Abstract base class for item on which we can run inference via CLI. + """Abstract base class for item on which we can run inference via CLI. Must have `path` and `cli_args` properties, used to build CLI call. """ @@ -60,8 +57,7 @@ def cli_args(self) -> List[Text]: @attr.s(auto_attribs=True) class VideoItemForInference(ItemForInference): - """ - Encapsulate data about video on which inference should run. + """Encapsulate data about video on which inference should run. This allows for inference on an arbitrary list of frames from video. @@ -109,7 +105,8 @@ def cli_args(self): # -Y represents endpoint of [X, Y) range but inference cli expects # [X, Y-1] range (so add 1 since negative). - frame_int_list = [i + 1 if i < 0 else i for i in self.frames] + frame_int_list = list(set([i + 1 if i < 0 else i for i in self.frames])) + frame_int_list.sort(reverse=min(frame_int_list) < 0) # Assumes len of 2 if neg. arg_list.extend(("--frames", ",".join(map(str, frame_int_list)))) @@ -118,8 +115,7 @@ def cli_args(self): @attr.s(auto_attribs=True) class DatasetItemForInference(ItemForInference): - """ - Encapsulate data about frame selection based on dataset data. + """Encapsulate data about frame selection based on dataset data. Attributes: labels_path: path to the saved :py:class:`Labels` dataset. @@ -141,7 +137,7 @@ def path(self): @property def cli_args(self): - args_list = ["--labels", self.path] + args_list = [self.path] if self.frame_filter == "user": args_list.append("--only-labeled-frames") elif self.frame_filter == "suggested": @@ -199,18 +195,8 @@ def make_predict_cli_call( ) -> List[Text]: """Makes list of CLI arguments needed for running inference.""" cli_args = ["sleap-track"] - cli_args.extend(item_for_inference.cli_args) - # TODO: encapsulate in inference item class - if ( - not self.trained_job_paths - and "tracking.tracker" in self.inference_params - and self.labels_filename - ): - # No models so we must want to re-track previous predictions - cli_args.extend(("--labels", self.labels_filename)) - # Make path where we'll save predictions (if not specified) if output_path is None: @@ -240,6 +226,7 @@ def make_predict_cli_call( "tracking.target_instance_count", "tracking.kf_init_frame_count", "tracking.robust", + "max_instances", ) for key in optional_items_as_nones: diff --git a/sleap/gui/state.py b/sleap/gui/state.py index e28b7fd11..4c559ad0e 100644 --- a/sleap/gui/state.py +++ b/sleap/gui/state.py @@ -21,7 +21,7 @@ """ import inspect -from typing import Any, Callable, List, Union +from typing import Any, Callable, List, Union, Optional GSVarType = str @@ -83,13 +83,15 @@ def toggle(self, key: GSVarType, default: bool = False): """Toggle boolean value for specified key.""" self[key] = not self.get(key, default=default) - def increment(self, key: GSVarType, step: int = 1, mod: int = 1, default: int = 0): + def increment( + self, key: GSVarType, step: int = 1, mod: Optional[int] = None, default: int = 0 + ): """Increment numeric value for specified key. Args: key: The key. step: What to add to current value. - mod: Wrap value (i.e., apply modulus) if not 1. + mod: Wrap value (i.e., apply modulus) if not None. default: Set value to this if there's no current value for key. Returns: @@ -97,14 +99,15 @@ def increment(self, key: GSVarType, step: int = 1, mod: int = 1, default: int = """ if key not in self._state_vars: self[key] = default - else: - new_value = self.get(key) + step + return + + new_value = self.get(key) + step - # take modulo of value if mod arg is not 1 - if mod != 1: - new_value %= mod + # Wrap the value if it's out of bounds. + if mod is not None: + new_value %= mod - self[key] = new_value + self[key] = new_value def increment_in_list( self, key: GSVarType, value_list: list, reverse: bool = False diff --git a/sleap/gui/widgets/docks.py b/sleap/gui/widgets/docks.py new file mode 100644 index 000000000..ef473ff96 --- /dev/null +++ b/sleap/gui/widgets/docks.py @@ -0,0 +1,567 @@ +"""Module for creating dock widgets for the `MainWindow`.""" + +from typing import Callable, Iterable, List, Optional, Type, Union +from qtpy import QtGui +from qtpy.QtCore import Qt +from qtpy.QtWidgets import ( + QWidget, + QDockWidget, + QMainWindow, + QLabel, + QComboBox, + QGroupBox, + QPushButton, + QTabWidget, + QLayout, + QHBoxLayout, + QVBoxLayout, +) + +from sleap.gui.dataviews import ( + GenericTableView, + GenericTableModel, + LabeledFrameTableModel, + SkeletonEdgesTableModel, + SkeletonNodeModel, + SkeletonNodesTableModel, + SuggestionsTableModel, + VideosTableModel, +) +from sleap.gui.dialogs.formbuilder import YamlFormWidget +from sleap.gui.widgets.views import CollapsibleWidget +from sleap.skeleton import Skeleton +from sleap.util import decode_preview_image, find_files_by_suffix, get_package_file + +# from sleap.gui.app import MainWindow + + +class DockWidget(QDockWidget): + """'Abstract' class for a dockable widget attached to the `MainWindow`.""" + + def __init__( + self, + name: str, + main_window: Optional[QMainWindow] = None, + model_type: Optional[ + Union[Type[GenericTableModel], List[Type[GenericTableModel]]] + ] = None, + widgets: Optional[Iterable[QWidget]] = None, + tab_with: Optional[QLayout] = None, + ): + # Create the dock and add it to the main window. + super().__init__(name) + self.name = name + self.main_window = main_window + self.setup_dock(widgets, tab_with) + + # Create the model and table for the dock. + self.model_type = model_type + if self.model_type is None: + self.model = None + self.table = None + else: + self.model = self.create_models() + self.table = self.create_tables() + + # Lay out the dock widget, adding/creating other widgets if needed. + self.lay_everything_out() + + @property + def wgt_layout(self) -> QVBoxLayout: + return self.widget().layout() + + def setup_dock(self, widgets, tab_with): + """Create a dock widget. + + Args: + widgets: The widgets to add to the dock. + tab_with: The `QLayout` to tabify the `DockWidget` with. + """ + + self.setObjectName(self.name + "Dock") + self.setAllowedAreas(Qt.LeftDockWidgetArea | Qt.RightDockWidgetArea) + + dock_widget = QWidget() + dock_widget.setObjectName(self.name + "Widget") + layout = QVBoxLayout() + + widgets = widgets or [] + for widget in widgets: + layout.addWidget(widget) + + dock_widget.setLayout(layout) + self.setWidget(dock_widget) + + self.add_to_window(self.main_window, tab_with) + + def add_to_window(self, main_window: QMainWindow, tab_with: QVBoxLayout): + """Add the dock to the `MainWindow`. + + Args: + tab_with: The `QLayout` to tabify the `DockWidget` with. + """ + self.main_window = main_window + self.main_window.addDockWidget(Qt.RightDockWidgetArea, self) + self.main_window.viewMenu.addAction(self.toggleViewAction()) + + if tab_with is not None: + self.main_window.tabifyDockWidget(tab_with, self) + + def add_button(self, to: QLayout, label: str, action: Callable, key=None): + key = key or label.lower() + btn = QPushButton(label) + btn.clicked.connect(action) + to.addWidget(btn) + self.main_window._buttons[key] = btn + return btn + + def create_models(self) -> GenericTableModel: + """Create the model for the table in the dock (if any). + + Implement this in the subclass. + Ex: + self.model = self.model_type(items=[], context=self.main_window.commands) + + Returns: + The model. + """ + raise NotImplementedError + + def create_tables(self) -> GenericTableView: + """Add a table to the dock. + + Implement this in the subclass. + Ex: + self.table = GenericTableView( + state=self.main_window.state, + model=self.model or self.create_models(), + ) + self.wgt_layout.addWidget(self.table) + + Returns: + The table widget. + """ + raise NotImplementedError + + def lay_everything_out(self) -> None: + """Lay out the dock widget, adding/creating other widgets if needed. + + Implement this in the subclass. No example as this is extremely custom. + """ + raise NotImplementedError + + +class VideosDock(DockWidget): + """Dock widget for displaying video information.""" + + def __init__( + self, + main_window: Optional[QMainWindow] = None, + ): + super().__init__( + name="Videos", main_window=main_window, model_type=VideosTableModel + ) + + def create_models(self) -> VideosTableModel: + self.model = self.model_type( + items=self.main_window.labels.videos, context=self.main_window.commands + ) + return self.model + + def create_tables(self) -> GenericTableView: + if self.model is None: + self.create_models() + + main_window = self.main_window + self.table = GenericTableView( + state=main_window.state, + row_name="video", + is_activatable=True, + model=self.model, + ellipsis_left=True, + ) + + return self.table + + def create_video_edit_and_nav_buttons(self) -> QWidget: + """Create the buttons for editing and navigating videos in table.""" + main_window = self.main_window + + hb = QHBoxLayout() + self.add_button(hb, "Toggle Grayscale", main_window.commands.toggleGrayscale) + self.add_button(hb, "Show Video", self.table.activateSelected) + self.add_button(hb, "Add Videos", main_window.commands.addVideo) + self.add_button(hb, "Remove Video", main_window.commands.removeVideo) + + hbw = QWidget() + hbw.setLayout(hb) + return hbw + + def lay_everything_out(self): + """Lay out the dock widget, adding/creating other widgets if needed.""" + self.wgt_layout.addWidget(self.table) + + video_edit_and_nav_buttons = self.create_video_edit_and_nav_buttons() + self.wgt_layout.addWidget(video_edit_and_nav_buttons) + + +class SkeletonDock(DockWidget): + """Dock widget for displaying skeleton information.""" + + def __init__( + self, + main_window: Optional[QMainWindow] = None, + tab_with: Optional[QLayout] = None, + ): + self.nodes_model_type = SkeletonNodesTableModel + self.edges_model_type = SkeletonEdgesTableModel + super().__init__( + name="Skeleton", + main_window=main_window, + model_type=[self.nodes_model_type, self.edges_model_type], + tab_with=tab_with, + ) + + def create_models(self) -> GenericTableModel: + main_window = self.main_window + self.nodes_model = self.nodes_model_type( + items=main_window.state["skeleton"], context=main_window.commands + ) + self.edges_model = self.edges_model_type( + items=main_window.state["skeleton"], context=main_window.commands + ) + return [self.nodes_model, self.edges_model] + + def create_tables(self) -> GenericTableView: + if self.model is None: + self.create_models() + + main_window = self.main_window + self.nodes_table = GenericTableView( + state=main_window.state, + row_name="node", + model=self.nodes_model, + ) + + self.edges_table = GenericTableView( + state=main_window.state, + row_name="edge", + model=self.edges_model, + ) + + return [self.nodes_table, self.edges_table] + + def create_project_skeleton_groupbox(self) -> QGroupBox: + """Create the groupbox for the project skeleton.""" + main_window = self.main_window + gb = QGroupBox("Project Skeleton") + vgb = QVBoxLayout() + + nodes_widget = QWidget() + vb = QVBoxLayout() + graph_tabs = QTabWidget() + + vb.addWidget(self.nodes_table) + hb = QHBoxLayout() + self.add_button(hb, "New Node", main_window.commands.newNode) + self.add_button(hb, "Delete Node", main_window.commands.deleteNode) + + hbw = QWidget() + hbw.setLayout(hb) + vb.addWidget(hbw) + nodes_widget.setLayout(vb) + graph_tabs.addTab(nodes_widget, "Nodes") + + def _update_edge_src(): + self.skeletonEdgesDst.model().skeleton = main_window.state["skeleton"] + + edges_widget = QWidget() + + vb = QVBoxLayout() + vb.addWidget(self.edges_table) + + hb = QHBoxLayout() + self.skeletonEdgesSrc = QComboBox() + self.skeletonEdgesSrc.setEditable(False) + self.skeletonEdgesSrc.currentIndexChanged.connect(_update_edge_src) + self.skeletonEdgesSrc.setModel(SkeletonNodeModel(main_window.state["skeleton"])) + hb.addWidget(self.skeletonEdgesSrc) + hb.addWidget(QLabel("to")) + self.skeletonEdgesDst = QComboBox() + self.skeletonEdgesDst.setEditable(False) + hb.addWidget(self.skeletonEdgesDst) + self.skeletonEdgesDst.setModel( + SkeletonNodeModel( + main_window.state["skeleton"], + lambda: self.skeletonEdgesSrc.currentText(), + ) + ) + + def new_edge(): + src_node = self.skeletonEdgesSrc.currentText() + dst_node = self.skeletonEdgesDst.currentText() + main_window.commands.newEdge(src_node, dst_node) + + self.add_button(hb, "Add Edge", new_edge) + self.add_button(hb, "Delete Edge", main_window.commands.deleteEdge) + hbw = QWidget() + hbw.setLayout(hb) + vb.addWidget(hbw) + edges_widget.setLayout(vb) + graph_tabs.addTab(edges_widget, "Edges") + vgb.addWidget(graph_tabs) + + hb = QHBoxLayout() + self.add_button(hb, "Load From File...", main_window.commands.openSkeleton) + self.add_button(hb, "Save As...", main_window.commands.saveSkeleton) + + hbw = QWidget() + hbw.setLayout(hb) + vgb.addWidget(hbw) + + # Add graph tabs to "Project Skeleton" group box + gb.setLayout(vgb) + return gb + + def create_templates_groupbox(self) -> QGroupBox: + """Create the groupbox for the skeleton templates.""" + main_window = self.main_window + + gb = CollapsibleWidget("Templates") + vb = QVBoxLayout() + hb = QHBoxLayout() + + skeletons_folder = get_package_file("sleap/skeletons") + skeletons_json_files = find_files_by_suffix( + skeletons_folder, suffix=".json", depth=1 + ) + skeletons_names = [json.name.split(".")[0] for json in skeletons_json_files] + self.skeleton_templates = QComboBox() + self.skeleton_templates.addItems(skeletons_names) + self.skeleton_templates.setEditable(False) + hb.addWidget(self.skeleton_templates) + self.add_button(hb, "Load", main_window.commands.openSkeletonTemplate) + hbw = QWidget() + hbw.setLayout(hb) + vb.addWidget(hbw) + + hb = QHBoxLayout() + self.skeleton_preview_image = QLabel("Preview Skeleton") + hb.addWidget(self.skeleton_preview_image) + hb.setAlignment(self.skeleton_preview_image, Qt.AlignLeft) + + self.skeleton_description = QLabel( + f'Description: {main_window.state["skeleton_description"]}' + ) + self.skeleton_description.setWordWrap(True) + hb.addWidget(self.skeleton_description) + hb.setAlignment(self.skeleton_description, Qt.AlignLeft) + + hbw = QWidget() + hbw.setLayout(hb) + vb.addWidget(hbw) + + def updatePreviewImage(preview_image_bytes: bytes): + + # Decode the preview image + preview_image = decode_preview_image(preview_image_bytes) + + # Create a QImage from the Image + preview_image = QtGui.QImage( + preview_image.tobytes(), + preview_image.size[0], + preview_image.size[1], + QtGui.QImage.Format_RGBA8888, # Format for RGBA images (see Image.mode) + ) + + preview_image = QtGui.QPixmap.fromImage(preview_image) + + self.skeleton_preview_image.setPixmap(preview_image) + + def update_skeleton_preview(idx: int): + skel = Skeleton.load_json(skeletons_json_files[idx]) + main_window.state["skeleton_description"] = ( + f"Description: {skel.description}

" + f"Nodes ({len(skel)}): {', '.join(skel.node_names)}" + ) + self.skeleton_description.setText(main_window.state["skeleton_description"]) + updatePreviewImage(skel.preview_image) + + self.skeleton_templates.currentIndexChanged.connect(update_skeleton_preview) + update_skeleton_preview(idx=0) + + gb.set_content_layout(vb) + return gb + + def lay_everything_out(self): + """Lay out the dock widget, adding/creating other widgets if needed.""" + templates_gb = self.create_templates_groupbox() + self.wgt_layout.addWidget(templates_gb) + + project_skeleton_groupbox = self.create_project_skeleton_groupbox() + self.wgt_layout.addWidget(project_skeleton_groupbox) + + +class SuggestionsDock(DockWidget): + """Dock widget for displaying suggestions.""" + + def __init__(self, main_window: QMainWindow, tab_with: Optional[QLayout] = None): + super().__init__( + name="Labeling Suggestions", + main_window=main_window, + model_type=SuggestionsTableModel, + tab_with=tab_with, + ) + + def create_models(self) -> SuggestionsTableModel: + self.model = self.model_type( + items=self.main_window.labels.suggestions, context=self.main_window.commands + ) + return self.model + + def create_tables(self) -> GenericTableView: + self.table = GenericTableView( + state=self.main_window.state, + is_sortable=True, + model=self.model, + ) + + # Connect some actions to the table + def goto_suggestion(*args): + selected_frame = self.table.getSelectedRowItem() + self.main_window.commands.gotoVideoAndFrame( + selected_frame.video, selected_frame.frame_idx + ) + + self.table.doubleClicked.connect(goto_suggestion) + self.main_window.state.connect("suggestion_idx", self.table.selectRow) + + return self.table + + def lay_everything_out(self) -> None: + self.wgt_layout.addWidget(self.table) + + table_edit_buttons = self.create_table_edit_buttons() + self.wgt_layout.addWidget(table_edit_buttons) + + table_nav_buttons = self.create_table_nav_buttons() + self.wgt_layout.addWidget(table_nav_buttons) + + self.suggestions_form_widget = self.create_suggestions_form() + self.wgt_layout.addWidget(self.suggestions_form_widget) + + def create_table_nav_buttons(self) -> QWidget: + main_window = self.main_window + hb = QHBoxLayout() + + self.add_button( + hb, + "Previous", + main_window.process_events_then(main_window.commands.prevSuggestedFrame), + "goto previous suggestion", + ) + + self.suggested_count_label = QLabel() + hb.addWidget(self.suggested_count_label) + + self.add_button( + hb, + "Next", + main_window.process_events_then(main_window.commands.nextSuggestedFrame), + "goto next suggestion", + ) + + hbw = QWidget() + hbw.setLayout(hb) + return hbw + + def create_suggestions_form(self) -> QWidget: + main_window = self.main_window + suggestions_form_widget = YamlFormWidget.from_name( + "suggestions", + title="Generate Suggestions", + ) + suggestions_form_widget.mainAction.connect( + main_window.process_events_then(main_window.commands.generateSuggestions) + ) + return suggestions_form_widget + + def create_table_edit_buttons(self) -> QWidget: + main_window = self.main_window + hb = QHBoxLayout() + + self.add_button( + hb, + "Add current frame", + main_window.process_events_then( + main_window.commands.addCurrentFrameAsSuggestion + ), + "add current frame as suggestion", + ) + + self.add_button( + hb, + "Remove", + main_window.process_events_then(main_window.commands.removeSuggestion), + "remove suggestion", + ) + + self.add_button( + hb, + "Clear all", + main_window.process_events_then(main_window.commands.clearSuggestions), + "clear suggestions", + ) + + hbw = QWidget() + hbw.setLayout(hb) + return hbw + + +class InstancesDock(DockWidget): + """Dock widget for displaying instances.""" + + def __init__(self, main_window: QMainWindow, tab_with: Optional[QLayout] = None): + super().__init__( + name="Instances", + main_window=main_window, + model_type=LabeledFrameTableModel, + tab_with=tab_with, + ) + + def create_models(self) -> LabeledFrameTableModel: + self.model = self.model_type( + items=self.main_window.state["labeled_frame"], + context=self.main_window.commands, + ) + return self.model + + def create_tables(self) -> GenericTableView: + self.table = GenericTableView( + state=self.main_window.state, + row_name="instance", + name_prefix="", + model=self.model, + ) + return self.table + + def lay_everything_out(self) -> None: + self.wgt_layout.addWidget(self.table) + + table_edit_buttons = self.create_table_edit_buttons() + self.wgt_layout.addWidget(table_edit_buttons) + + def create_table_edit_buttons(self) -> QWidget: + main_window = self.main_window + + hb = QHBoxLayout() + self.add_button( + hb, "New Instance", lambda x: main_window.commands.newInstance() + ) + self.add_button( + hb, "Delete Instance", main_window.commands.deleteSelectedInstance + ) + + hbw = QWidget() + hbw.setLayout(hb) + return hbw diff --git a/sleap/io/dataset.py b/sleap/io/dataset.py index 4788f64f5..c54ed2755 100644 --- a/sleap/io/dataset.py +++ b/sleap/io/dataset.py @@ -2453,6 +2453,7 @@ def set_track( frame_idxs = [lf.frame_idx for lf in lfs] frame_idxs.sort() first_frame = 0 if all_frames else frame_idxs[0] + last_frame = len(video) - 1 if all_frames else frame_idxs[-1] # Figure out the number of tracks based on number of instances in each frame. # @@ -2476,7 +2477,7 @@ def set_track( # Case 2: We're considering only tracked instances. n_tracks = len(self.tracks) - n_frames = frame_idxs[-1] - first_frame + 1 + n_frames = last_frame - first_frame + 1 n_nodes = len(self.skeleton.nodes) if return_confidence: diff --git a/sleap/io/video.py b/sleap/io/video.py index 1dc2aadd6..f8af330ec 100644 --- a/sleap/io/video.py +++ b/sleap/io/video.py @@ -63,6 +63,8 @@ class HDF5Video: convert_range: Whether we should convert data to [0, 255]-range """ + EXTS = ("h5", "hdf5", "slp") + filename: str = attr.ib(default=None) dataset: str = attr.ib(default=None) input_format: str = attr.ib(default="channels_last") @@ -349,6 +351,8 @@ class MediaVideo: bgr: Whether color channels ordered as (blue, green, red). """ + EXTS = ("mp4", "avi", "mov", "mj2", "mkv") + filename: str = attr.ib() grayscale: bool = attr.ib() bgr: bool = attr.ib(default=True) @@ -514,6 +518,8 @@ class NumpyVideo: * numpy data shape: (frames, height, width, channels) """ + EXTS = ("npy", "npz") + filename: Union[str, np.ndarray] = attr.ib() def __attrs_post_init__(self): @@ -621,6 +627,8 @@ class ImgStoreVideo: indices on :class:`LabeledFrame` objects in the dataset. """ + EXTS = ("json", "yaml") + filename: str = attr.ib(default=None) index_by_original: bool = attr.ib(default=True) _store_ = None @@ -800,6 +808,9 @@ class SingleImageVideo: filenames: Files to load as video. """ + EXTS = ("jpg", "jpeg", "png", "tif", "tiff") + CACHING = False # Deprecated, but keeping functionality for now. + filename: Optional[str] = attr.ib(default=None) filenames: Optional[List[str]] = attr.ib(factory=list) height_: Optional[int] = attr.ib(default=None) @@ -815,7 +826,7 @@ def __attrs_post_init__(self): elif self.filename and not self.filenames: self.filenames = [self.filename] - self.__data = dict() + self.cache_ = dict() self.test_frame_ = None @grayscale.default @@ -845,20 +856,24 @@ def _get_filename(self, idx: int) -> str: raise FileNotFoundError(f"Unable to locate file {idx}: {self.filenames[idx]}") def _load_test_frame(self): - if self.test_frame_ is None: - self.test_frame_ = self._load_idx(0) + test_frame_ = self.test_frame_ + if test_frame_ is None: + test_frame_ = self._load_idx(0) if self._detect_grayscale is True: self.grayscale = bool( - np.alltrue(self.test_frame_[..., 0] == self.test_frame_[..., -1]) + np.alltrue(test_frame_[..., 0] == test_frame_[..., -1]) ) if self.height_ is None: - self.height_ = self.test_frame.shape[0] + self.height_ = test_frame_.shape[0] if self.width_ is None: - self.width_ = self.test_frame.shape[1] + self.width_ = test_frame_.shape[1] if self.channels_ is None: - self.channels_ = self.test_frame.shape[2] + self.channels_ = test_frame_.shape[2] + if self.CACHING: # Deprecated, but keeping functionality for now. + self.test_frame_ = test_frame_ + return test_frame_ def get_idx_from_filename(self, filename: str) -> int: try: @@ -871,8 +886,7 @@ def get_idx_from_filename(self, filename: str) -> int: @property def test_frame(self) -> np.ndarray: - self._load_test_frame() - return self.test_frame_ + return self._load_test_frame() def matches(self, other: "SingleImageVideo") -> bool: """ @@ -928,7 +942,7 @@ def height(self, val): @property def dtype(self): """See :class:`Video`.""" - return self.__data.dtype + return self.cache_.dtype def reset( self, @@ -945,7 +959,7 @@ def reset( f"Cannot specify both filename and filenames for SingleImageVideo." ) elif filename or filenames: - self.__data = dict() + self.cache_ = dict() self.test_frame_ = None self.height_ = height_ self.width_ = width_ @@ -966,10 +980,13 @@ def reset( def get_frame(self, idx: int, grayscale: bool = None) -> np.ndarray: """See :class:`Video`.""" - if idx not in self.__data: - self.__data[idx] = self._load_idx(idx) + if self.CACHING: # Deprecated, but keeping functionality for now. + if idx not in self.cache_: + self.cache_[idx] = self._load_idx(idx) - frame = self.__data[idx] # Manipulate a copy of self.__data[idx] + frame = self.cache_[idx] # Manipulate a copy of self.__data[idx] + else: + frame = self._load_idx(idx) if grayscale is None: grayscale = self.grayscale @@ -1244,16 +1261,16 @@ def from_filename(cls, filename: str, *args, **kwargs) -> "Video": """ filename = Video.fixup_path(filename) - if filename.lower().endswith(("h5", "hdf5", "slp")): + if filename.lower().endswith(HDF5Video.EXTS): backend_class = HDF5Video - elif filename.endswith(("npy")): + elif filename.endswith(NumpyVideo.EXTS): backend_class = NumpyVideo - elif filename.lower().endswith(("mp4", "avi", "mov")): + elif filename.lower().endswith(MediaVideo.EXTS): backend_class = MediaVideo kwargs["dataset"] = "" # prevent serialization from breaking elif os.path.isdir(filename) or "metadata.yaml" in filename: backend_class = ImgStoreVideo - elif filename.lower().endswith(("jpg", "jpeg", "png", "tif", "tiff")): + elif filename.lower().endswith(SingleImageVideo.EXTS): backend_class = SingleImageVideo else: raise ValueError("Could not detect backend for specified filename.") @@ -1593,11 +1610,27 @@ def fixup_path( return path +def available_video_exts() -> Tuple[str]: + """Return tuple of supported video extensions. + + Returns: + Tuple of supported video extensions. + """ + return ( + MediaVideo.EXTS + + HDF5Video.EXTS + + NumpyVideo.EXTS + + SingleImageVideo.EXTS + + ImgStoreVideo.EXTS + ) + + def load_video( filename: str, grayscale: Optional[bool] = None, dataset=Optional[None], channels_first: bool = False, + **kwargs, ) -> Video: """Open a video from disk. @@ -1633,7 +1666,6 @@ def load_video( See also: sleap.io.video.Video """ - kwargs = {} if grayscale is not None: kwargs["grayscale"] = grayscale if dataset is not None: diff --git a/sleap/nn/evals.py b/sleap/nn/evals.py index 670c339c7..ad8990b9f 100644 --- a/sleap/nn/evals.py +++ b/sleap/nn/evals.py @@ -173,10 +173,10 @@ def compute_oks( Ronch & Perona. "Benchmarking and Error Diagnosis in Multi-Instance Pose Estimation." ICCV (2017). """ - if points_gt.ndim != 3 or points_pr.ndim != 3: - raise ValueError( - "Points must be rank-3 with shape (n_instances, n_nodes, n_ed)." - ) + if points_gt.ndim == 2: + points_gt = np.expand_dims(points_gt, axis=0) + if points_pr.ndim == 2: + points_pr = np.expand_dims(points_pr, axis=0) if scale is None: scale = compute_instance_area(points_gt) diff --git a/sleap/nn/inference.py b/sleap/nn/inference.py index 7a186c26b..24c2ce5f5 100644 --- a/sleap/nn/inference.py +++ b/sleap/nn/inference.py @@ -180,6 +180,7 @@ def from_model_paths( integral_patch_size: int = 5, batch_size: int = 4, resize_input_layer: bool = True, + max_instances: Optional[int] = None, ) -> "Predictor": """Create the appropriate `Predictor` subclass from a list of model paths. @@ -198,12 +199,17 @@ def from_model_paths( usage. resize_input_layer: If True, the the input layer of the `tf.Keras.model` is resized to (None, None, None, num_color_channels). + max_instances: If not `None`, discard instances beyond this count when + predicting, regardless of whether filtering is done at the tracking + stage. This is useful for preventing extraneous instances from being + created when tracking is not being applied. Returns: A subclass of `Predictor`. See also: `SingleInstancePredictor`, `TopDownPredictor`, `BottomUpPredictor`, - `MoveNetPredictor` + `MoveNetPredictor`, `TopDownMultiClassPredictor`, + `BottomUpMultiClassPredictor`. """ # Read configs and find model types. if isinstance(model_paths, str): @@ -272,6 +278,7 @@ def from_model_paths( integral_refinement=integral_refinement, integral_patch_size=integral_patch_size, resize_input_layer=resize_input_layer, + max_instances=max_instances, ) elif "multi_instance" in model_types: @@ -282,6 +289,7 @@ def from_model_paths( integral_patch_size=integral_patch_size, batch_size=batch_size, resize_input_layer=resize_input_layer, + max_instances=max_instances, ) elif "multi_class_bottomup" in model_types: @@ -531,7 +539,6 @@ def export_model( unrag_outputs: bool = True, max_instances: Optional[int] = None, ): - """Export a trained SLEAP model as a frozen graph. Initializes model, creates a dummy tracing batch and passes it through the model. The frozen graph is saved along with training meta info. @@ -1104,7 +1111,6 @@ def export_model( os.makedirs(save_path, exist_ok=True) with tempfile.TemporaryDirectory() as tmp_dir: - self.save(tmp_dir, save_format="tf", save_traces=save_traces) imported = tf.saved_model.load(tmp_dir) @@ -1592,7 +1598,6 @@ def export_model( unrag_outputs: bool = True, max_instances: Optional[int] = None, ): - super().export_model( save_path, signatures, @@ -1806,9 +1811,7 @@ def call(self, inputs): n_peaks = tf.shape(centroid_points)[0] if n_peaks > 0: - if self.max_instances is not None: - centroid_points = tf.RaggedTensor.from_value_rowids( centroid_points, crop_sample_inds, nrows=samples ) @@ -1838,21 +1841,35 @@ def call(self, inputs): ) for sample in range(samples): + n_centroids = tf.shape(centroid_vals[sample])[0] + if self.max_instances < n_centroids: + top_points = tf.math.top_k( + centroid_vals[sample], + k=self.max_instances, + ) + top_inds = top_points.indices - top_points = tf.math.top_k( - centroid_vals[sample], k=self.max_instances - ) - top_inds = top_points.indices - - _centroid_vals = _centroid_vals.write( - sample, tf.gather(centroid_vals[sample], top_inds) - ) + _centroid_vals = _centroid_vals.write( + sample, tf.gather(centroid_vals[sample], top_inds) + ) - _centroid_points = _centroid_points.write( - sample, tf.gather(centroid_points[sample], top_inds) - ) + _centroid_points = _centroid_points.write( + sample, tf.gather(centroid_points[sample], top_inds) + ) - _row_ids = _row_ids.write(sample, tf.fill([len(top_inds)], sample)) + _row_ids = _row_ids.write( + sample, tf.fill([len(top_inds)], sample) + ) + else: + _centroid_vals = _centroid_vals.write( + sample, centroid_vals[sample] + ) + _centroid_points = _centroid_points.write( + sample, centroid_points[sample] + ) + _row_ids = _row_ids.write( + sample, tf.fill([n_centroids], sample) + ) centroid_vals = _centroid_vals.concat() centroid_points = _centroid_points.concat() @@ -2237,7 +2254,6 @@ def call( crop_output = self.centroid_crop(example) if isinstance(self.instance_peaks, FindInstancePeaksGroundTruth): - if "instances" in example: peaks_output = self.instance_peaks(example, crop_output) else: @@ -2290,6 +2306,10 @@ class TopDownPredictor(Predictor): head. integral_patch_size: Size of patches to crop around each rough peak for integral refinement as an integer scalar. + max_instances: If not `None`, discard instances beyond this count when + predicting, regardless of whether filtering is done at the tracking stage. + This is useful for preventing extraneous instances from being created when + tracking is not being applied. """ centroid_config: Optional[TrainingJobConfig] = attr.ib(default=None) @@ -2303,6 +2323,7 @@ class TopDownPredictor(Predictor): peak_threshold: float = 0.2 integral_refinement: bool = True integral_patch_size: int = 5 + max_instances: Optional[int] = None def _initialize_inference_model(self): """Initialize the inference model from the trained models and configuration.""" @@ -2328,6 +2349,7 @@ def _initialize_inference_model(self): refinement="integral" if self.integral_refinement else "local", integral_patch_size=self.integral_patch_size, return_confmaps=False, + max_instances=self.max_instances, ) if use_gt_confmap: @@ -2366,6 +2388,7 @@ def from_trained_models( integral_refinement: bool = True, integral_patch_size: int = 5, resize_input_layer: bool = True, + max_instances: Optional[int] = None, ) -> "TopDownPredictor": """Create predictor from saved models. @@ -2389,6 +2412,10 @@ def from_trained_models( integral refinement as an integer scalar. resize_input_layer: If True, the the input layer of the `tf.Keras.model` is resized to (None, None, None, num_color_channels). + max_instances: If not `None`, discard instances beyond this count when + predicting, regardless of whether filtering is done at the tracking + stage. This is useful for preventing extraneous instances from being + created when tracking is not being applied. Returns: An instance of `TopDownPredictor` with the loaded models. @@ -2444,6 +2471,7 @@ def from_trained_models( peak_threshold=peak_threshold, integral_refinement=integral_refinement, integral_patch_size=integral_patch_size, + max_instances=max_instances, ) obj._initialize_inference_model() return obj @@ -2564,7 +2592,6 @@ def _object_builder(): ex["instance_peak_vals"], ex["centroid_vals"], ): - # Loop over instances. predicted_instances = [] for pts, confs, score in zip(points, confidences, scores): @@ -2624,7 +2651,6 @@ def export_model( unrag_outputs: bool = True, max_instances: Optional[int] = None, ): - super().export_model( save_path, signatures, @@ -2682,6 +2708,10 @@ class BottomUpInferenceLayer(InferenceLayer): the predicted instances. This will result in slower inference times since the data must be copied off of the GPU, but is useful for visualizing the raw output of the model. + return_paf_graph: If `True`, the part affinity field graph will be returned + together with the predicted instances. The graph is obtained by parsing the + part affinity fields with the `paf_scorer` instance and is an intermediate + representation used during instance grouping. confmaps_ind: Index of the output tensor of the model corresponding to confidence maps. If `None` (the default), this will be detected automatically by searching for the first tensor that contains @@ -2710,6 +2740,7 @@ def __init__( integral_patch_size: int = 5, return_confmaps: bool = False, return_pafs: bool = False, + return_paf_graph: bool = False, confmaps_ind: Optional[int] = None, pafs_ind: Optional[int] = None, offsets_ind: Optional[int] = None, @@ -2765,6 +2796,7 @@ def __init__( self.integral_patch_size = integral_patch_size self.return_confmaps = return_confmaps self.return_pafs = return_pafs + self.return_paf_graph = return_paf_graph def forward_pass(self, data): """Run preprocessing and model inference on a batch.""" @@ -2865,6 +2897,10 @@ def call(self, data): If `BottomUpInferenceLayer.return_pafs` is `True`, the predicted PAFs will be returned in the `"part_affinity_fields"` key. + + If `BottomUpInferenceLayer.return_paf_graph` is `True`, the predicted PAF + graph will be returned in the `"peaks"`, `"peak_vals"`, `"peak_channel_inds"`, + `"edge_inds"`, `"edge_peak_inds"` and `"line_scores"` keys. """ cms, pafs, offsets = self.forward_pass(data) peaks, peak_vals, peak_channel_inds = self.find_peaks(cms, offsets) @@ -2872,6 +2908,9 @@ def call(self, data): predicted_instances, predicted_peak_scores, predicted_instance_scores, + edge_inds, + edge_peak_inds, + line_scores, ) = self.paf_scorer.predict(pafs, peaks, peak_vals, peak_channel_inds) # Adjust for input scaling. @@ -2891,6 +2930,13 @@ def call(self, data): out["confmaps"] = cms if self.return_pafs: out["part_affinity_fields"] = pafs + if self.return_paf_graph: + out["peaks"] = peaks + out["peak_vals"] = peak_vals + out["peak_channel_inds"] = peak_channel_inds + out["edge_inds"] = edge_inds + out["edge_peak_inds"] = edge_peak_inds + out["line_scores"] = line_scores return out @@ -2986,6 +3032,10 @@ class BottomUpPredictor(Predictor): min_line_scores: Minimum line score (between -1 and 1) required to form a match between candidate point pairs. Useful for rejecting spurious detections when there are no better ones. + max_instances: If not `None`, discard instances beyond this count when + predicting, regardless of whether filtering is done at the tracking stage. + This is useful for preventing extraneous instances from being created when + tracking is not being applied. """ bottomup_config: TrainingJobConfig @@ -3001,6 +3051,7 @@ class BottomUpPredictor(Predictor): dist_penalty_weight: float = 1.0 paf_line_points: int = 10 min_line_scores: float = 0.25 + max_instances: Optional[int] = None def _initialize_inference_model(self): """Initialize the inference model from the trained model and configuration.""" @@ -3047,6 +3098,7 @@ def from_trained_models( paf_line_points: int = 10, min_line_scores: float = 0.25, resize_input_layer: bool = True, + max_instances: Optional[int] = None, ) -> "BottomUpPredictor": """Create predictor from a saved model. @@ -3077,6 +3129,10 @@ def from_trained_models( detections when there are no better ones. resize_input_layer: If True, the the input layer of the `tf.Keras.model` is resized to (None, None, None, num_color_channels). + max_instances: If not `None`, discard instances beyond this count when + predicting, regardless of whether filtering is done at the tracking + stage. This is useful for preventing extraneous instances from being + created when tracking is not being applied. Returns: An instance of `BottomUpPredictor` with the loaded model. @@ -3103,6 +3159,7 @@ def from_trained_models( dist_penalty_weight=dist_penalty_weight, paf_line_points=paf_line_points, min_line_scores=min_line_scores, + max_instances=max_instances, ) obj._initialize_inference_model() return obj @@ -3159,7 +3216,6 @@ def _object_builder(): ex["instance_peak_vals"], ex["instance_scores"], ): - # Loop over instances. predicted_instances = [] for pts, confs, score in zip(points, confidences, scores): @@ -3175,6 +3231,15 @@ def _object_builder(): ) ) + if self.max_instances is not None: + # Filter by score. + predicted_instances = sorted( + predicted_instances, key=lambda x: x.score, reverse=True + ) + predicted_instances = predicted_instances[ + : min(self.max_instances, len(predicted_instances)) + ] + if self.tracker: # Set tracks for predicted instances in this frame. predicted_instances = self.tracker.track( @@ -3668,7 +3733,6 @@ def _object_builder(): ex["instance_peak_vals"], ex["instance_scores"], ): - # Loop over instances. predicted_instances = [] for i, (pts, confs, score) in enumerate( @@ -4048,7 +4112,6 @@ def export_model( tensors: Optional[Dict[str, str]] = None, unrag_outputs: bool = True, ): - self.instance_peaks.optimal_grouping = False super().export_model( @@ -4191,7 +4254,7 @@ def from_trained_models( resized to (None, None, None, num_color_channels). Returns: - An instance of `TopDownPredictor` with the loaded models. + An instance of `TopDownMultiClassPredictor` with the loaded models. One of the two models can be left as `None` to perform inference with ground truth data. This will only work with `LabelsReader` as the provider. @@ -4357,7 +4420,6 @@ def _object_builder(): ex["instance_peak_vals"], ex["instance_scores"], ): - # Loop over instances. predicted_instances = [] for i, (pts, confs, score) in enumerate( @@ -4411,7 +4473,6 @@ def export_model( unrag_outputs: bool = True, max_instances: Optional[int] = None, ): - super().export_model( save_path, signatures, @@ -4583,7 +4644,6 @@ def _initialize_inference_model(self): @property def data_config(self) -> DataConfig: - if self.inference_model is None: self._initialize_inference_model() @@ -4625,7 +4685,6 @@ def from_trained_models( def _make_labeled_frames_from_generator( self, generator: Iterator[Dict[str, np.ndarray]], data_provider: Provider ) -> List[sleap.LabeledFrame]: - skeleton = MOVENET_SKELETON predicted_frames = [] @@ -4693,6 +4752,7 @@ def load_model( disable_gpu_preallocation: bool = True, progress_reporting: str = "rich", resize_input_layer: bool = True, + max_instances: Optional[int] = None, ) -> Predictor: """Load a trained SLEAP model. @@ -4728,6 +4788,10 @@ def load_model( machines where the output is captured to a log file. resize_input_layer: If True, the the input layer of the `tf.Keras.model` is resized to (None, None, None, num_color_channels). + max_instances: If not `None`, discard instances beyond this count when + predicting, regardless of whether filtering is done at the tracking stage. + This is useful for preventing extraneous instances from being created when + tracking is not being applied. Returns: An instance of a `Predictor` based on which model type was detected. @@ -4789,6 +4853,7 @@ def unpack_sleap_model(model_path): integral_refinement=refinement == "integral", batch_size=batch_size, resize_input_layer=resize_input_layer, + max_instances=max_instances, ) predictor.verbosity = progress_reporting if tracker is not None: @@ -4904,12 +4969,12 @@ def _make_export_cli_parser() -> argparse.ArgumentParser: ), ) parser.add_argument( - "-i", + "-n", "--max_instances", type=int, help=( - "Limit maximum number of instances in multi-instance models" - "Defaults to None" + "Limit maximum number of instances in multi-instance models. " + "Not available for ID models. Defaults to None." ), ) @@ -5085,6 +5150,15 @@ def _make_cli_parser() -> argparse.ArgumentParser: default=0.2, help="Minimum confidence map value to consider a peak as valid.", ) + parser.add_argument( + "-n", + "--max_instances", + type=int, + help=( + "Limit maximum number of instances in multi-instance models. " + "Not available for ID models. Defaults to None." + ), + ) # Deprecated legacy args. These will still be parsed for backward compatibility but # are hidden from the CLI help. @@ -5229,6 +5303,7 @@ def _make_predictor_from_cli(args: argparse.Namespace) -> Predictor: batch_size=batch_size, refinement="integral", progress_reporting=args.verbosity, + max_instances=args.max_instances, ) if type(predictor) == BottomUpPredictor: @@ -5329,7 +5404,6 @@ def main(args: Optional[list] = None): # Either run inference (and tracking) or just run tracking if args.models is not None: - # Setup models. predictor = _make_predictor_from_cli(args) predictor.tracker = tracker diff --git a/sleap/nn/paf_grouping.py b/sleap/nn/paf_grouping.py index 4a6a0a157..4091c026b 100644 --- a/sleap/nn/paf_grouping.py +++ b/sleap/nn/paf_grouping.py @@ -613,7 +613,6 @@ def match_candidates_sample( ) for k in range(n_edges): - is_edge_k = tf.squeeze(tf.where(edge_inds_sample == k), axis=1) edge_peak_inds_k = tf.gather(edge_peak_inds_sample, is_edge_k, axis=0) line_scores_k = tf.gather(line_scores_sample, is_edge_k, axis=0) @@ -836,10 +835,8 @@ def assign_connections_to_instances( # Loop through edge types. for edge_type, edge_connections in connections.items(): - # Loop through connections for the current edge. for connection in edge_connections: - # Notation: specific peaks are identified by (node_ind, peak_ind). src_id = PeakID(edge_type.src_node_ind, connection.src_peak_ind) dst_id = PeakID(edge_type.dst_node_ind, connection.dst_peak_ind) @@ -887,7 +884,6 @@ def assign_connections_to_instances( if min_instance_peaks > 0: if isinstance(min_instance_peaks, float): - if n_nodes is None: # Infer number of nodes if not specified. all_node_types = set() @@ -1240,7 +1236,6 @@ def _group_instances_sample( ) for sample in range(n_samples): - # Call sample-wise function in Eager mode. ( predicted_instances_sample, @@ -1700,4 +1695,11 @@ def predict( match_dst_peak_inds, match_line_scores, ) - return predicted_instances, predicted_peak_scores, predicted_instance_scores + return ( + predicted_instances, + predicted_peak_scores, + predicted_instance_scores, + edge_inds, + edge_peak_inds, + line_scores, + ) diff --git a/sleap/nn/system.py b/sleap/nn/system.py index d4a2bfe7c..24b4c14b3 100644 --- a/sleap/nn/system.py +++ b/sleap/nn/system.py @@ -7,6 +7,8 @@ import tensorflow as tf from typing import List, Optional, Text import subprocess +import shutil +import os def get_all_gpus() -> List[tf.config.PhysicalDevice]: @@ -187,18 +189,18 @@ def best_logical_device_name() -> Text: def get_gpu_memory() -> List[int]: - """Return list of available GPU memory. + """Get the available memory on each GPU. Returns: - List of available GPU memory (in MiB) with indices corresponding to GPU indices. + A list of the available memory on each GPU in MiB. - Notes: - This function depends on the `nvidia-smi` system utility. If it cannot be found - in the `PATH`, this function returns an empty list. """ + if shutil.which("nvidia-smi") is None: + return [] + command = [ "nvidia-smi", - "--query-gpu=memory.free", + "--query-gpu=index,memory.free", "--format=csv", ] @@ -207,20 +209,20 @@ def get_gpu_memory() -> List[int]: except (subprocess.SubprocessError, FileNotFoundError): return [] - # Capture subprocess standard output subprocess_result = memory_poll.stdout - - # nvidia-smi returns an ascii encoded byte string separated by newlines (\n) - # Splitting gives a list where the final entry is an empty string. Slice it off - # and finally slice off the csv header in the 0th element. memory_string = subprocess_result.decode("ascii").split("\n")[1:-1] + if "CUDA_VISIBLE_DEVICES" in os.environ.keys(): + cuda_visible_devices = os.environ["CUDA_VISIBLE_DEVICES"].split(",") + else: + cuda_visible_devices = None + memory_list = [] for row in memory_string: - # Removing megabyte text returned by nvidia-smi - available_memory = row.split(" MiB")[0] + gpu_index, available_memory = row.split(", ") + available_memory = available_memory.split(" MiB")[0] - # Append percent of GPU available to GPU ID, assume GPUs returned in index order - memory_list.append(int(available_memory)) + if cuda_visible_devices is None or gpu_index in cuda_visible_devices: + memory_list.append(int(available_memory)) return memory_list diff --git a/sleap/nn/training.py b/sleap/nn/training.py index 7d0d25a56..21beb802b 100644 --- a/sleap/nn/training.py +++ b/sleap/nn/training.py @@ -1844,9 +1844,10 @@ def create_trainer_using_cli(args: Optional[List] = None): parser.add_argument( "--base_checkpoint", type=str, + default=None, help=( "Path to base checkpoint (directory containing best_model.h5) to resume " - "training from." + "training from. Default is None." ), ) parser.add_argument( @@ -1940,7 +1941,8 @@ def create_trainer_using_cli(args: Optional[List] = None): if len(args.video_paths) == 0: args.video_paths = None - job_config.model.base_checkpoint = args.base_checkpoint + if args.base_checkpoint is not None: + job_config.model.base_checkpoint = args.base_checkpoint logger.info("Versions:") sleap.versions() diff --git a/sleap/version.py b/sleap/version.py index 557ec831a..a4e2cec7d 100644 --- a/sleap/version.py +++ b/sleap/version.py @@ -12,7 +12,7 @@ """ -__version__ = "1.3.0" +__version__ = "1.3.1" def versions(): diff --git a/tests/data/siv_format_v1/robot_siv.slp b/tests/data/siv_format_v1/small_robot_siv.slp similarity index 100% rename from tests/data/siv_format_v1/robot_siv.slp rename to tests/data/siv_format_v1/small_robot_siv.slp diff --git a/tests/data/siv_format_v2/small_robot_siv_caching.slp b/tests/data/siv_format_v2/small_robot_siv_caching.slp new file mode 100644 index 0000000000000000000000000000000000000000..864fb9bb531e0ec63e90865b695aa12d61beb64b GIT binary patch literal 16384 zcmeHOPfr_16dz+~U7AXeqzO%{hOSzbN;U=pY3d6^3K55@jY29_Y9h1PgS}zB>+CM1 z2qCWImXDAlQZIZ2jydMoQ^iL>>ZP~zy_xsc_Tsf^E82o)5&P!P`!n;K@$7GA?H5b; z?td_RV_4vo%ZWiTqLO@jOP5&CC2^h}-qrDb zrm%Yciq!dgHmx&W&kF)Pe}AL~=Sc3+y2kzBbyQ7uF8l{_`MdA$WyFx+_4e1<-p+VF zzxk@S9=CrytM$0uOO&ruzgEc2#HYRe@w)!Um%a7=t6F|;CHr>?9*l#~sEimES4B>& z`9Z^u%1)IUIyycNrFbof+4sfl+XP;>)b;y3uPcY2Xu~=I%Il9>grF)zPtr_AD=_Z- zyv#y0EV}}?*YH0?`GKtVkR$u-z`y~GA*!RQx-L^|F+X2;2Hn`Bn*WEubru-~_m^ed zfkZQpMmh_)pK1D?+9qcZFbEg~3<3rLgTQ%2VB1=?E9=s$TBU7k&2gn?H>6dvA{jle5@H-rJt;OrDEt;9NR6p(ry;O(gBVq!-DNW!YwrY zP=>|khU*pr`D{aZ4xP zvQ@e@GnKcVJJqNTnYr7Lsnl)HlWvHWc`HWBDy`XWD52i2)E%!z`JJ8Q++!YhJkK>; z!ujL*WC_(UWke~&;Vv7r)DL&#besUKq#@VQQ5R?&?#Agj(Z!JKXfL1);g))FAeN}e zX_oAhCF*dRWh)-mn#&+y5HJW_XaxG5pK#}SZO_8_2rj8<=QcggQ`iNb2hq`r5GT)5 z_=Jrk|GU!Q(H-xGjBI3sT&qc8`Dy5TswfJ8Mux3?xKXPi5%j&bLeK;z7`7KSeaDMJ zA)0}#mN^4@ju%F@SCLSquC)MoDP;I@K2GIHVaBf$5jcO%=<`?nWab197@mKA1pkcZ zwzVk(92wyLGm}%3Q$X^=b?M3oMi5B;wpDckS&1ZU6rx}Q7$|VkW*v78v}0H%?ZOwab=#z z&az@<0DE!`z5)6TNZyyWzQLm7q}`D(YhJ$4vW>Qy*#CZf9lx5vD60K$ThRB?0#Z5J za$MfWu}MExj<&dG6k=Sc?G&UwRQdE9_LFVa0>4ql4ZVZ=Mc8Mio2i<1jCgK**2iXa z{AL{QdQM!&w-oC505B@U(LoRi@kJjW?HI+j@G1<=3V*>?VFWK@%GCI|KDOQ3-eX?x z=e0kanm7g3m7LdU`?{_}!bYwHJgOp;cFs-|62e42lO-f6(W5J$Fd&xDWZBAiX0! z{FFN>xcU|SuIHr2`6x-_$!<#XUtH;|;Lk-oP@UB#XAm$57z7Lg1_6V>1wi0WBa-iH literal 0 HcmV?d00001 diff --git a/tests/fixtures/datasets.py b/tests/fixtures/datasets.py index 5db1fc4a1..b8d438fb6 100644 --- a/tests/fixtures/datasets.py +++ b/tests/fixtures/datasets.py @@ -19,7 +19,8 @@ TEST_SLP_MIN_LABELS = "tests/data/slp_hdf5/minimal_instance.slp" TEST_MAT_LABELS = "tests/data/mat/labels.mat" TEST_SLP_MIN_LABELS_ROBOT = "tests/data/slp_hdf5/small_robot_minimal.slp" -TEST_SLP_SIV_ROBOT = "tests/data/siv_format_v1/robot_siv.slp" +TEST_SLP_SIV_ROBOT = "tests/data/siv_format_v1/small_robot_siv.slp" +TEST_SLP_SIV_ROBOT_CACHING = "tests/data/siv_format_v2/small_robot_siv_caching.slp" TEST_MIN_TRACKS_2NODE_LABELS = "tests/data/tracks/clip.2node.slp" TEST_MIN_TRACKS_13NODE_LABELS = "tests/data/tracks/clip.slp" TEST_HDF5_PREDICTIONS = "tests/data/hdf5_format_v1/centered_pair_predictions.h5" @@ -60,7 +61,23 @@ def min_labels_robot(): @pytest.fixture def siv_robot(): """Created before grayscale attribute was added to SingleImageVideo backend.""" - return Labels.load_file(TEST_SLP_SIV_ROBOT) + return Labels.load_file(TEST_SLP_SIV_ROBOT, video_search="tests/data/videos/") + + +@pytest.fixture +def siv_robot_caching(): + """Created after caching attribute was added to `SingleImageVideo` backend. + + The typehinting of the `caching` attribute (#1243) caused it to be used by cattrs to + determine which type of dataclass to use. However, the older datasets containing + `SingleImageVideo`s were now being read in as `NumpyVideo`s. Although removing the + typehinting from `caching` seems to do the trick (and never made it into an official + release), this is a fixture to test that datasets created while `caching` was added + into the serialization are read in correctly. + """ + return Labels.load_file( + TEST_SLP_SIV_ROBOT_CACHING, video_search="tests/data/videos/" + ) @pytest.fixture diff --git a/tests/gui/learning/test_dialog.py b/tests/gui/learning/test_dialog.py index 34a4e1c83..3d77c891f 100644 --- a/tests/gui/learning/test_dialog.py +++ b/tests/gui/learning/test_dialog.py @@ -24,18 +24,34 @@ def test_use_hidden_params_from_loaded_config( - qtbot, min_labels_slp, min_bottomup_model_path + qtbot, min_labels_slp, min_bottomup_model_path, tmpdir ): + model_type = Path(min_bottomup_model_path).name + model_path = Path(tmpdir, "models") + model_path.mkdir() + model_path = Path(model_path, model_type) + model_path.mkdir() - model_path = Path(min_bottomup_model_path) + model_dir = str(model_path) + + best_model_path = str(Path(min_bottomup_model_path, "best_model.h5")) + shutil.copy(best_model_path, model_dir) + + cfg_path = str(Path(model_dir, "training_config.json")) + cfg = TrainingJobConfig.load_json(min_bottomup_model_path) + cfg.data.labels.split_by_inds = True + cfg.data.labels.training_inds = [0, 1, 2] + cfg.data.labels.validation_inds = [3, 4, 5] + cfg.data.labels.test_inds = [6, 7, 8] + cfg.filename = cfg_path + + TrainingJobConfig.save_json(cfg, cfg_path) # Create a learning dialog app = MainWindow(no_usage_data=True) ld = LearningDialog( mode="training", - labels_filename=Path( - model_path - ).parent.absolute(), # Hack to get correct config + labels_filename=model_path.parent.absolute(), # Hack to get correct config labels=min_labels_slp, ) @@ -82,6 +98,9 @@ def test_use_hidden_params_from_loaded_config( # Assert that config info list: params_set_in_tab = [f"config.{k}" for k in tab_cfg_key_val_dict.keys()] params_reset = [ + "config.data.labels.validation_labels", + "config.data.labels.test_labels", + "config.data.labels.split_by_inds", "config.data.labels.skeletons", "config.outputs.run_name", "config.outputs.run_name_suffix", @@ -378,3 +397,35 @@ def test_movenet_selection(qtbot, min_dance_labels): # ensure pipeline version matches model type assert pipeline_form_data["_pipeline"] == model + + +def test_immutablilty_of_trained_config_info( + qtbot, min_labels_slp, min_bottomup_model_path, tmpdir +): + + model_path = Path(min_bottomup_model_path) + + # Create a learning dialog + app = MainWindow(no_usage_data=True) + ld = LearningDialog( + mode="training", + labels_filename=model_path.parent.absolute(), # Hack to get correct config + labels=min_labels_slp, + ) + + # Select a loaded config for pipeline form data + bottom_up_tab: TrainingEditorWidget = ld.tabs["multi_instance"] + cfg_list_widget: TrainingConfigFilesWidget = bottom_up_tab._cfg_list_widget + cfg_list_widget.update() + pre_config = bottom_up_tab._cfg_list_widget.getSelectedConfigInfo() + post_config = bottom_up_tab.trained_config_info_to_use + + # Check that the config info is not mutated when calling trained_config_info_to_use + # run_name is just one of many parameters that are changed during call + assert pre_config.config.outputs.run_name is not None + assert post_config.config.outputs.run_name is None + + # Previously, trained_config_info_to_use would mutate the config info and fail when + # saving multiple configs from one config info. + ld.save(output_dir=tmpdir) + ld.save(output_dir=tmpdir) diff --git a/tests/gui/test_app.py b/tests/gui/test_app.py index 4ea5ee996..66b0dafbb 100644 --- a/tests/gui/test_app.py +++ b/tests/gui/test_app.py @@ -33,7 +33,7 @@ def test_app_workflow( assert app.state["skeleton"].nodes[2].name == "c" # Select and delete the third node - app.skeletonNodesTable.selectRowItem(app.state["skeleton"].nodes[2]) + app.skeleton_dock.nodes_table.selectRowItem(app.state["skeleton"].nodes[2]) app.commands.deleteNode() assert len(app.state["skeleton"].nodes) == 2 @@ -60,7 +60,7 @@ def test_app_workflow( assert app.state["skeleton"].get_symmetry("c") is None # Remove an edge - app.skeletonEdgesTable.selectRowItem(dict(source="b", destination="c")) + app.skeleton_dock.edges_table.selectRowItem(dict(source="b", destination="c")) app.commands.deleteEdge() assert len(app.state["skeleton"].edges) == 1 @@ -81,7 +81,9 @@ def assert_frame_chunk_suggestion_ui_updated( assert frame_to_spinbox.maximum() == app.state["video"].num_frames assert frame_from_spinbox.maximum() == app.state["video"].num_frames - method_layout = app.suggestions_form_widget.form_layout.fields["method"] + method_layout = app.suggestions_dock.suggestions_form_widget.form_layout.fields[ + "method" + ] frame_chunk_layout = method_layout.page_layouts["frame chunk"] frame_to_spinbox = frame_chunk_layout.fields["frame_to"] frame_from_spinbox = frame_chunk_layout.fields["frame_from"] @@ -90,8 +92,8 @@ def assert_frame_chunk_suggestion_ui_updated( assert_frame_chunk_suggestion_ui_updated(app, frame_to_spinbox, frame_from_spinbox) # Activate video using table - app.videosTable.selectRowItem(small_robot_mp4_vid) - app.videosTable.activateSelected() + app.videos_dock.table.selectRowItem(small_robot_mp4_vid) + app.videos_dock.table.activateSelected() assert app.state["video"] == small_robot_mp4_vid @@ -99,7 +101,7 @@ def assert_frame_chunk_suggestion_ui_updated( assert_frame_chunk_suggestion_ui_updated(app, frame_to_spinbox, frame_from_spinbox) # Select remaining video - app.videosTable.selectRowItem(small_robot_mp4_vid) + app.videos_dock.table.selectRowItem(small_robot_mp4_vid) assert app.state["selected_video"] == small_robot_mp4_vid # Verify the max of frame_to in frame_chunk is updated @@ -256,13 +258,16 @@ def assert_frame_chunk_suggestion_ui_updated( # The on_data_update function uses labeled frames cache app.on_data_update([UpdateTopic.suggestions]) - assert len(app.suggestionsTable.model().items) == num_samples - assert f"{num_samples}/{num_samples}" in app.suggested_count_label.text() + assert len(app.suggestions_dock.table.model().items) == num_samples + assert ( + f"{num_samples}/{num_samples}" + in app.suggestions_dock.suggested_count_label.text() + ) # Check that frames returned by labeled frames cache are correct prev_idx = -frame_delta for l_suggestion, st_suggestion in list( - zip(app.labels.get_suggestions(), app.suggestionsTable.model().items) + zip(app.labels.get_suggestions(), app.suggestions_dock.table.model().items) ): assert l_suggestion == st_suggestion["SuggestionFrame"] lf = app.labels.get( @@ -302,7 +307,7 @@ def assert_frame_chunk_suggestion_ui_updated( assert len(video_source) == 2 # Remove video 1, keep video 0 - app.videosTable.selectRowItem(small_robot_mp4_vid) + app.videos_dock.table.selectRowItem(small_robot_mp4_vid) assert app.state["selected_video"] == small_robot_mp4_vid app.commands.removeVideo() assert len(app.labels.videos) == 1 diff --git a/tests/gui/test_dialogs.py b/tests/gui/test_dialogs.py index 250e91bdd..4455550fb 100644 --- a/tests/gui/test_dialogs.py +++ b/tests/gui/test_dialogs.py @@ -5,7 +5,7 @@ from pathlib import Path import pytest -from PySide2.QtWidgets import QComboBox +from qtpy.QtWidgets import QComboBox import sleap from sleap.skeleton import Skeleton diff --git a/tests/gui/test_inference_gui.py b/tests/gui/test_inference_gui.py index a533a0001..ef3716895 100644 --- a/tests/gui/test_inference_gui.py +++ b/tests/gui/test_inference_gui.py @@ -41,17 +41,21 @@ def test_scoped_key_dict(): @pytest.mark.parametrize( - "labels_path, video_path", [("labels.slp", "video.mp4"), (None, "video.mp4")] + "labels_path, video_path, max_instances, frames", + [ + ("labels.slp", "video.mp4", None, [0, 1, 2]), + (None, "video.mp4", 1, [0, -1]), + (None, "video.mp4", 2, [1, -4]), + ], ) -def test_inference_cli_builder(labels_path, video_path): - +def test_inference_cli_builder(labels_path, video_path, max_instances, frames): inference_task = runners.InferenceTask( trained_job_paths=["model1", "model2"], - inference_params={"tracking.tracker": "simple"}, + inference_params={"tracking.tracker": "simple", "max_instances": max_instances}, ) item_for_inference = runners.VideoItemForInference( - video=Video.from_filename(video_path), frames=[1, 2, 3], labels_path=labels_path + video=Video.from_filename(video_path), frames=frames, labels_path=labels_path ) cli_args, output_path = inference_task.make_predict_cli_call(item_for_inference) @@ -62,7 +66,20 @@ def test_inference_cli_builder(labels_path, video_path): assert "model1" in cli_args assert "model2" in cli_args assert "--frames" in cli_args + + frames_idx = cli_args.index("--frames") + if -1 in frames: + assert cli_args[frames_idx + 1] == "0" # No redundant frames + elif -4 in frames: + assert cli_args[frames_idx + 1] == "1,-3" # Ordered correctly + else: + assert cli_args[frames_idx + 1] == ",".join([str(f) for f in frames]) assert "--tracking.tracker" in cli_args + assert ( + "--max_instances" in cli_args + if max_instances is not None + else max_instances is None + ) assert output_path.startswith(data_path) assert output_path.endswith("predictions.slp") diff --git a/tests/gui/test_state.py b/tests/gui/test_state.py index ff69b97ed..4f2caae84 100644 --- a/tests/gui/test_state.py +++ b/tests/gui/test_state.py @@ -50,9 +50,14 @@ def set_y_from_val_param_callback(x): assert times_x_changed == 4 assert state["x"] == 2 + # Test incrementing value with modulus of 1 + state.increment("x", mod=1) + assert times_x_changed == 5 + assert state["x"] == 0 + # test emitting callbacks without changing value state.emit("x") - assert times_x_changed == 5 + assert times_x_changed == 6 def test_gui_state_bool(): diff --git a/tests/gui/widgets/test_docks.py b/tests/gui/widgets/test_docks.py new file mode 100644 index 000000000..0bc8f98b2 --- /dev/null +++ b/tests/gui/widgets/test_docks.py @@ -0,0 +1,55 @@ +"""Module for testing dock widgets for the `MainWindow`.""" + +import pytest + +from sleap.gui.app import MainWindow +from sleap.gui.widgets.docks import ( + InstancesDock, + SuggestionsDock, + VideosDock, + SkeletonDock, +) + + +def test_videos_dock(qtbot): + """Test the `DockWidget` class.""" + main_window = MainWindow() + dock = VideosDock(main_window) + + assert dock.name == "Videos" + assert dock.main_window is main_window + assert dock.wgt_layout is dock.widget().layout() + + +def test_skeleton_dock(qtbot): + """Test the `DockWidget` class.""" + main_window = MainWindow() + dock = SkeletonDock(main_window) + + assert dock.name == "Skeleton" + assert dock.main_window is main_window + assert dock.wgt_layout is dock.widget().layout() + + +def test_suggestions_dock(qtbot): + """Test the `DockWidget` class.""" + main_window = MainWindow() + dock = SuggestionsDock(main_window) + + assert dock.name == "Labeling Suggestions" + assert dock.main_window is main_window + assert dock.wgt_layout is dock.widget().layout() + + +def test_instances_dock(qtbot): + """Test the `DockWidget` class.""" + main_window = MainWindow() + dock = InstancesDock(main_window) + + assert dock.name == "Instances" + assert dock.main_window is main_window + assert dock.wgt_layout is dock.widget().layout() + + +if __name__ == "__main__": + pytest.main([f"{__file__}::test_instances_dock"]) diff --git a/tests/io/test_dataset.py b/tests/io/test_dataset.py index a6fa7fdd7..6cc6485dc 100644 --- a/tests/io/test_dataset.py +++ b/tests/io/test_dataset.py @@ -1349,6 +1349,13 @@ def test_labels_numpy(centered_pair_predictions: Labels): trx = centered_pair_predictions.numpy(video=None, all_frames=True, untracked=False) assert trx.shape == (1100, 27, 24, 2) + centered_pair_predictions.remove_frame(centered_pair_predictions[-1]) + trx = centered_pair_predictions.numpy(video=None, all_frames=False, untracked=False) + assert trx.shape == (1098, 27, 24, 2) + + trx = centered_pair_predictions.numpy(video=None, all_frames=True, untracked=False) + assert trx.shape == (1100, 27, 24, 2) + labels_single = Labels( [ LabeledFrame( @@ -1359,7 +1366,7 @@ def test_labels_numpy(centered_pair_predictions: Labels): ) assert labels_single.numpy().shape == (1100, 1, 24, 2) - assert centered_pair_predictions.numpy(untracked=True).shape == (1100, 5, 24, 2) + assert centered_pair_predictions.numpy(untracked=True).shape == (1100, 4, 24, 2) for lf in centered_pair_predictions: for inst in lf: inst.track = None diff --git a/tests/io/test_video.py b/tests/io/test_video.py index 82ddfe7a4..9361f393b 100644 --- a/tests/io/test_video.py +++ b/tests/io/test_video.py @@ -12,7 +12,7 @@ DummyVideo, load_video, ) -from tests.fixtures.datasets import TEST_SLP_SIV_ROBOT + from tests.fixtures.videos import ( TEST_H5_DSET, TEST_H5_INPUT_FORMAT, @@ -495,7 +495,7 @@ def test_reset_video_mp4(small_robot_mp4_vid: Video): assert_video_params(video=video, filename=filename, bgr=True, reset=True) -def test_reset_video_siv(small_robot_single_image_vid: Video, siv_robot: Labels): +def test_reset_video_siv(small_robot_single_image_vid: Video, siv_robot): video = small_robot_single_image_vid filename = video.backend.filename @@ -547,9 +547,50 @@ def test_reset_video_siv(small_robot_single_image_vid: Video, siv_robot: Labels) video.backend.reset(filename=filename, filenames=filenames) assert_video_params(video=video, filenames=filenames, reset=True) - # Test reset does not break deserialization of older slp - labels: Labels = Labels.load_file(TEST_SLP_SIV_ROBOT) + # Test reset does not break deserialization of older slp. + labels = siv_robot # This is actually tested upon passing in the fixture. video: Video = labels.video filename = labels.video.backend.filename labels.video.backend.reset(filename=filename, grayscale=True) assert_video_params(video=video, filenames=filenames, grayscale=True, reset=True) + + +def test_singleimagevideo_caching(siv_robot_caching): + # Test that older `SingleImageVideo` with type-hinted `caching` can be read in. + siv_robot_caching # This is actually tested upon passing in the fixture. + + # The below tests are for depreciated `SingleImageVideo.CACHING` functionality. + + # With caching + filename = siv_robot_caching.video.backend.filename + video = Video.from_filename(filename) + SingleImageVideo.CACHING = True + assert video.backend.test_frame_ is None + assert len(video.backend.cache_) == 0 + assert video.backend.channels_ is None + + assert video.backend.test_frame.shape == (320, 560, 3) + assert video.backend.test_frame_ is not None # Test frame stored! + assert len(video.backend.cache_) == 0 + assert video.backend.channels_ == 3 + + assert video[0].shape == (1, 320, 560, 3) + assert len(video.backend.cache_) == 1 # Loaded frame stored! + + # No caching + video = Video.from_filename(filename) + SingleImageVideo.CACHING = False + assert video.backend.test_frame_ is None + assert len(video.backend.cache_) == 0 + assert video.backend.channels_ is None + + assert video.backend.test_frame.shape == (320, 560, 3) + assert video.backend.test_frame_ is None # Test frame not stored! + assert len(video.backend.cache_) == 0 + assert video.backend.channels_ == 3 + + assert video.shape == (1, 320, 560, 3) + assert video.backend.test_frame_ is None # Test frame not stored! + + assert video[0].shape == (1, 320, 560, 3) + assert len(video.backend.cache_) == 0 # Loaded frame not stored! diff --git a/tests/nn/test_evals.py b/tests/nn/test_evals.py index 297fec36a..0e6a04dfe 100644 --- a/tests/nn/test_evals.py +++ b/tests/nn/test_evals.py @@ -1,11 +1,32 @@ import numpy as np import sleap -from sleap.nn.evals import load_metrics +from sleap.nn.evals import load_metrics, compute_oks sleap.use_cpu_only() +def test_compute_oks(): + inst_gt = np.array([[0, 0], [1, 1], [2, 2]]).astype("float32") + inst_pr = np.array([[0, 0], [1, 1], [2, 2]]).astype("float32") + oks = compute_oks(inst_gt, inst_pr) + np.testing.assert_allclose(oks, 1) + + inst_pr = np.array([[0, 0], [1, 1], [np.nan, np.nan]]).astype("float32") + oks = compute_oks(inst_gt, inst_pr) + np.testing.assert_allclose(oks, 2 / 3) + + inst_gt = np.array([[0, 0], [1, 1], [np.nan, np.nan]]).astype("float32") + inst_pr = np.array([[0, 0], [1, 1], [2, 2]]).astype("float32") + oks = compute_oks(inst_gt, inst_pr) + np.testing.assert_allclose(oks, 1) + + inst_gt = np.array([[0, 0], [1, 1], [np.nan, np.nan]]).astype("float32") + inst_pr = np.array([[0, 0], [1, 1], [np.nan, np.nan]]).astype("float32") + oks = compute_oks(inst_gt, inst_pr) + np.testing.assert_allclose(oks, 1) + + def test_load_metrics(min_centered_instance_model_path): model_path = min_centered_instance_model_path diff --git a/tests/nn/test_inference.py b/tests/nn/test_inference.py index d21433606..9e07b07f8 100644 --- a/tests/nn/test_inference.py +++ b/tests/nn/test_inference.py @@ -20,6 +20,7 @@ InferenceLayer, InferenceModel, Predictor, + _make_predictor_from_cli, get_model_output_stride, find_head, SingleInstanceInferenceLayer, @@ -643,12 +644,20 @@ def test_topdown_predictor_centroid(min_labels, min_centroid_model_path): inds1, inds2 = sleap.nn.utils.match_points(points_gt, points_pr) assert_allclose(points_gt[inds1.numpy()], points_pr[inds2.numpy()], atol=1.5) - # test max_instances (>2 will fail) - predictor.inference_model.centroid_crop.max_instances = 2 - labels_pr = predictor.predict(min_labels) - assert len(labels_pr) == 1 - assert len(labels_pr[0].instances) == 2 +def test_topdown_predictor_centroid_max_instances(min_labels, min_centroid_model_path): + predictor = TopDownPredictor.from_trained_models( + centroid_model_path=min_centroid_model_path + ) + + # Test max_instances <, =, and > than number of expected instances + for i in [1, 2, 3]: + predictor._initialize_inference_model() + predictor.inference_model.centroid_crop.max_instances = i + labels_pr = predictor.predict(min_labels) + + assert len(labels_pr) == 1 + assert len(labels_pr[0].instances) == min(i, 2) def test_topdown_predictor_centroid_high_threshold(min_labels, min_centroid_model_path): @@ -1328,6 +1337,45 @@ def load_instance(labels_in: Labels): assert new_inst.track != old_inst.track +@pytest.mark.parametrize("cmd", ["--max_instances 1", "-n 1"]) +def test_valid_cli_command(cmd): + """Test that sleap-track CLI command is valid.""" + parser = _make_cli_parser() + args = parser.parse_args(cmd.split()) + assert args.max_instances == 1 + + +def test_make_predictor_from_cli( + centered_pair_predictions: Labels, + min_centroid_model_path: str, + min_centered_instance_model_path: str, + min_bottomup_model_path: str, + tmpdir, +): + slp_path = str(Path(tmpdir, "old_slp.slp")) + Labels.save(centered_pair_predictions, slp_path) + + # Create sleap-track command + model_args = [ + f"--model {min_centroid_model_path} --model {min_centered_instance_model_path}", + f"--model {min_bottomup_model_path}", + ] + for model_arg in model_args: + args = ( + f"{slp_path} {model_arg} --video.index 0 --frames 1-3 " + "--cpu --max_instances 5" + ).split() + parser = _make_cli_parser() + args, _ = parser.parse_known_args(args=args) + + # Create predictor + predictor = _make_predictor_from_cli(args=args) + if isinstance(predictor, TopDownPredictor): + assert predictor.inference_model.centroid_crop.max_instances == 5 + elif isinstance(predictor, BottomUpPredictor): + assert predictor.max_instances == 5 + + def test_sleap_track( centered_pair_predictions: Labels, min_centroid_model_path: str, @@ -1335,10 +1383,9 @@ def test_sleap_track( tmpdir, ): slp_path = str(Path(tmpdir, "old_slp.slp")) - labels: Labels = Labels.save(centered_pair_predictions, slp_path) + Labels.save(centered_pair_predictions, slp_path) # Create sleap-track command - args = f"{slp_path} --model {min_centered_instance_model_path} --frames 1-3 --cpu".split() args = ( f"{slp_path} --model {min_centroid_model_path} " f"--model {min_centered_instance_model_path} --video.index 0 --frames 1-3 --cpu" diff --git a/tests/nn/test_system.py b/tests/nn/test_system.py index 7540b6db9..ea835e3c3 100644 --- a/tests/nn/test_system.py +++ b/tests/nn/test_system.py @@ -5,8 +5,85 @@ """ from sleap.nn.system import get_gpu_memory +from sleap.nn.system import get_all_gpus +import os +import pytest +import subprocess +import tensorflow as tf +import shutil def test_get_gpu_memory(): # Make sure this doesn't throw an exception memory = get_gpu_memory() + + +@pytest.mark.parametrize("cuda_visible_devices", ["0", "1", "0,1"]) +def test_get_gpu_memory_visible(cuda_visible_devices): + if shutil.which("nvidia-smi") is None: + pytest.skip("nvidia-smi not available.") + + # Get GPU indices from nvidia-smi + command = ["nvidia-smi", "--query-gpu=index", "--format=csv,noheader"] + nvidia_indices = ( + subprocess.check_output(command).decode("utf-8").strip().split("\n") + ) + + # Set parameterized CUDA visible devices + os.environ["CUDA_VISIBLE_DEVICES"] = cuda_visible_devices + + gpu_memory = get_gpu_memory() + + if nvidia_indices == "0" or nvidia_indices == "1": + assert len(gpu_memory) > 0 + assert len(gpu_memory) == 1 + + elif nvidia_indices == "0,1": + assert len(gpu_memory) > 0 + assert len(gpu_memory) == 2 + + +def test_get_gpu_memory_no_nvidia_smi(): + # Backup current PATH + old_path = os.environ["PATH"] + + # Set PATH to an empty string to simulate that nvidia-smi is not available + os.environ["PATH"] = "" + + memory = get_gpu_memory() + + # Restore the original PATH + os.environ["PATH"] = old_path + + assert memory == [] + + +@pytest.mark.parametrize("cuda_visible_devices", ["invalid", "3,5", "-1"]) +def test_get_gpu_memory_invalid_cuda_visible_devices(cuda_visible_devices): + for value in cuda_visible_devices: + os.environ["CUDA_VISIBLE_DEVICES"] = value + + memory = get_gpu_memory() + + # Cleanup CUDA_VISIBLE_DEVICES variable after the test + os.environ.pop("CUDA_VISIBLE_DEVICES", None) + + assert len(memory) == 0 + + +def test_gpu_order_and_length(): + if shutil.which("nvidia-smi") is None: + pytest.skip("nvidia-smi not available.") + + # Get GPU indices from sleap.nn.system.get_all_gpus + sleap_indices = [int(gpu.name.split(":")[-1]) for gpu in get_all_gpus()] + + # Get GPU indices from nvidia-smi + command = ["nvidia-smi", "--query-gpu=index", "--format=csv,noheader"] + nvidia_indices = ( + subprocess.check_output(command).decode("utf-8").strip().split("\n") + ) + nvidia_indices = [int(idx) for idx in nvidia_indices] + + # Assert that the order and length of GPU indices match + assert sleap_indices == nvidia_indices diff --git a/tests/nn/test_training.py b/tests/nn/test_training.py index b95d177ff..55f404929 100644 --- a/tests/nn/test_training.py +++ b/tests/nn/test_training.py @@ -316,43 +316,47 @@ def test_train_cropping( ) +@pytest.mark.parametrize("base_checkpoint_path", [None, ""]) def test_resume_training_cli( - min_single_instance_robot_model_path: str, small_robot_mp4_path: str, tmp_path: str + base_checkpoint_path, + min_single_instance_robot_model_path: str, + small_robot_mp4_path: str, + tmp_path: str, ): """Test CLI to resume training.""" - - base_checkpoint_path = min_single_instance_robot_model_path - cfg = TrainingJobConfig.load_json( - str(Path(base_checkpoint_path, "training_config.json")) + cfg_dir = min_single_instance_robot_model_path + base_checkpoint_path = ( + min_single_instance_robot_model_path + if base_checkpoint_path is not None + else base_checkpoint_path ) + + cfg = TrainingJobConfig.load_json(str(Path(cfg_dir, "training_config.json"))) cfg.optimization.preload_data = False cfg.optimization.batch_size = 1 cfg.optimization.batches_per_epoch = 2 cfg.optimization.epochs = 1 cfg.outputs.save_outputs = False + cfg.model.base_checkpoint = base_checkpoint_path # Save training config to tmp folder cfg_path = str(Path(tmp_path, "training_config.json")) cfg.save_json(cfg_path) - # TODO (LM): Stop saving absolute paths in labels files! # We need to do this reload because we save absolute paths (for the video). - labels_path = str(Path(base_checkpoint_path, "labels_gt.train.slp")) + labels_path = str(Path(cfg_dir, "labels_gt.train.slp")) labels: Labels = sleap.load_file(labels_path, search_paths=[small_robot_mp4_path]) labels_path = str(Path(tmp_path, "labels_gt.train.slp")) labels.save_file(labels, labels_path) - # Run CLI to resume training - trainer = sleap_train( - [ - cfg_path, - labels_path, - "--base_checkpoint", - base_checkpoint_path, - ] - ) + # Check that base_checkpoint is set correctly (not overridden by CLI) + cli_args = [cfg_path, labels_path] + trainer = sleap_train(cli_args) assert trainer.config.model.base_checkpoint == base_checkpoint_path - # Run CLI without base checkpoint - trainer = sleap_train([cfg_path, labels_path]) - assert trainer.config.model.base_checkpoint is None + # Check that base_checkpoint is set correctly (overridden by CLI) + if base_checkpoint_path is not None: + cli_args = [cfg_path, labels_path, "--base_checkpoint", base_checkpoint_path] + + trainer = sleap_train(cli_args) + assert trainer.config.model.base_checkpoint == base_checkpoint_path