Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to poetry, improve tooling #109

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[*.{py,toml}]
indent_style = space
indent_size = 4

[*.yml,yaml,json]
indent_style = space
indent_size = 2
50 changes: 31 additions & 19 deletions .github/workflows/tests_full.yaml
Original file line number Diff line number Diff line change
@@ -1,41 +1,53 @@
name: "Full set of tests for PRs and master branch"
on:
on:
push:
branches:
- "master"
pull_request:

concurrency:
concurrency:
group: ${{ github.ref }}
cancel-in-progress: true

jobs:
tests:
runs-on: ubuntu-20.04
runs-on: ubuntu-22.04
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
name: Python ${{ matrix.python-version }}
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install Nox
run: pip install nox==2024.03.02
- name: Run tests
run: nox --non-interactive --error-on-missing-interpreter --session "tests-${{ matrix.python-version }}"

- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install Poetry
run: |
pip install poetry==1.8.2
pip install poetry-plugin-export==1.7.0
- name: Install Nox
run: |
pip install nox==2024.03.02
pip install nox-poetry==1.0.3
- name: Run tests
run: nox --non-interactive --error-on-missing-interpreter --session "tests-${{ matrix.python-version }}"

lint:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.9
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: 3.9
python-version: 3.11
- name: Install Poetry
run: |
pip install poetry==1.8.2
pip install poetry-plugin-export==1.7.0
- name: Install Nox
run: pip install nox==2024.03.02
run: |
pip install nox==2024.03.02
pip install nox-poetry==1.0.3
- name: Lint
run: nox --non-interactive --error-on-missing-interpreter --session "lint"
run: nox --non-interactive --error-on-missing-interpreter --session "precommit"
44 changes: 28 additions & 16 deletions .github/workflows/tests_reduced.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ on:
branches-ignore:
- master

concurrency:
concurrency:
group: ${{ github.ref }}
cancel-in-progress: true

jobs:
tests:
runs-on: ubuntu-latest
Expand All @@ -16,25 +16,37 @@ jobs:
python-version: ["3.11"]
name: Python ${{ matrix.python-version }}
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install Nox
run: pip install nox==2024.03.02
- name: Run tests
run: nox --non-interactive --error-on-missing-interpreter --session "tests-${{ matrix.python-version }}"

- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install Poetry
run: |
pip install poetry==1.8.2
pip install poetry-plugin-export==1.7.0
- name: Install Nox
run: |
pip install nox==2024.03.02
pip install nox-poetry==1.0.3
- name: Run tests
run: nox --non-interactive --error-on-missing-interpreter --session "tests-${{ matrix.python-version }}"

lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python 3.9
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: 3.9
python-version: 3.11
- name: Install Poetry
run: |
pip install poetry==1.8.2
pip install poetry-plugin-export==1.7.0
- name: Install Nox
run: pip install nox==2024.03.02
run: |
pip install nox==2024.03.02
pip install nox-poetry==1.0.3
- name: Lint
run: nox --non-interactive --error-on-missing-interpreter --session "lint"
run: nox --non-interactive --error-on-missing-interpreter --session "precommit"
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -113,4 +113,4 @@ venv.bak/
dmypy.json

# Pyre type checker
.pyre/
.pyre/
59 changes: 59 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
repos:
- repo: local
hooks:
- id: black
name: black
entry: black
language: system
types: [python]
require_serial: true
- id: check-added-large-files
name: Check for added large files
entry: check-added-large-files
language: system
- id: check-toml
name: Check Toml
entry: check-toml
language: system
types: [toml]
- id: check-yaml
name: Check Yaml
entry: check-yaml
language: system
types: [yaml]
- id: end-of-file-fixer
name: Fix End of Files
entry: end-of-file-fixer
language: system
types: [text]
stages: [commit, push, manual]
- id: flake8
name: flake8
entry: flake8
language: system
types: [python]
require_serial: true
- id: isort
name: isort
entry: isort
require_serial: true
language: system
types_or: [cython, pyi, python]
args: ["--filter-files"]
- id: pyupgrade
name: pyupgrade
description: Automatically upgrade syntax for newer versions.
entry: pyupgrade
language: system
types: [python]
args: [--py38-plus]
- id: trailing-whitespace
name: Trim Trailing Whitespace
entry: trailing-whitespace-fixer
language: system
types: [text]
stages: [commit, push, manual]
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v2.6.0
hooks:
- id: prettier
20 changes: 10 additions & 10 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,37 +4,37 @@

### Added

* Add argument `--save-stats` allowing to compute dataset statistics and save them as an `.npz` file ([#80](https://github.com/mseitzer/pytorch-fid/pull/80)). The `.npz` file can be used in subsequent FID computations instead of recomputing the dataset statistics. This option can be used in the following way: `python -m pytorch_fid --save-stats path/to/dataset path/to/outputfile`.
- Add argument `--save-stats` allowing to compute dataset statistics and save them as an `.npz` file ([#80](https://github.com/mseitzer/pytorch-fid/pull/80)). The `.npz` file can be used in subsequent FID computations instead of recomputing the dataset statistics. This option can be used in the following way: `python -m pytorch_fid --save-stats path/to/dataset path/to/outputfile`.

### Fixed

* Do not use `os.sched_getaffinity` to get number of available CPUs on Windows, as it is not available there ([232b3b14](https://github.com/mseitzer/pytorch-fid/commit/232b3b1468800102fcceaf6f2bb8977811fc991a), [#84](https://github.com/mseitzer/pytorch-fid/issues/84)).
* Do not use Inception model argument `pretrained`, as it was deprecated in torchvision 0.13 ([#88](https://github.com/mseitzer/pytorch-fid/pull/88)).
- Do not use `os.sched_getaffinity` to get number of available CPUs on Windows, as it is not available there ([232b3b14](https://github.com/mseitzer/pytorch-fid/commit/232b3b1468800102fcceaf6f2bb8977811fc991a), [#84](https://github.com/mseitzer/pytorch-fid/issues/84)).
- Do not use Inception model argument `pretrained`, as it was deprecated in torchvision 0.13 ([#88](https://github.com/mseitzer/pytorch-fid/pull/88)).

## [0.2.1] - 2021-10-10

### Added

* Add argument `--num-workers` to select number of dataloader processes ([#66](https://github.com/mseitzer/pytorch-fid/pull/66)). Defaults to 8 or the number of available CPUs if less than 8 CPUs are available.
- Add argument `--num-workers` to select number of dataloader processes ([#66](https://github.com/mseitzer/pytorch-fid/pull/66)). Defaults to 8 or the number of available CPUs if less than 8 CPUs are available.

### Fixed

* Fixed package setup to work under Windows ([#55](https://github.com/mseitzer/pytorch-fid/pull/55), [#72](https://github.com/mseitzer/pytorch-fid/issues/72))
- Fixed package setup to work under Windows ([#55](https://github.com/mseitzer/pytorch-fid/pull/55), [#72](https://github.com/mseitzer/pytorch-fid/issues/72))

## [0.2.0] - 2020-11-30

### Added

* Load images using a Pytorch dataloader, which should result in a speed-up. ([#47](https://github.com/mseitzer/pytorch-fid/pull/47))
* Support more image extensions ([#53](https://github.com/mseitzer/pytorch-fid/pull/53))
* Improve tooling by setting up Nox, add linting and test support ([#52](https://github.com/mseitzer/pytorch-fid/pull/52))
* Add some unit tests
- Load images using a Pytorch dataloader, which should result in a speed-up. ([#47](https://github.com/mseitzer/pytorch-fid/pull/47))
- Support more image extensions ([#53](https://github.com/mseitzer/pytorch-fid/pull/53))
- Improve tooling by setting up Nox, add linting and test support ([#52](https://github.com/mseitzer/pytorch-fid/pull/52))
- Add some unit tests

## [0.1.1] - 2020-08-16

### Fixed

* Fixed software license string in `setup.py`
- Fixed software license string in `setup.py`

## [0.1.0] - 2020-08-16

Expand Down
13 changes: 9 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ FID is calculated by computing the [Fréchet distance](https://en.wikipedia.org/

Further insights and an independent evaluation of the FID score can be found in [Are GANs Created Equal? A Large-Scale Study](https://arxiv.org/abs/1711.10337).

The weights and the model are exactly the same as in [the official Tensorflow implementation](https://github.com/bioinf-jku/TTUR), and were tested to give very similar results (e.g. `.08` absolute error and `0.0009` relative error on LSUN, using ProGAN generated images). However, due to differences in the image interpolation implementation and library backends, FID results still differ slightly from the original implementation. So if you report FID scores in your paper, and you want them to be *exactly comparable* to FID scores reported in other papers, you should consider using [the official Tensorflow implementation](https://github.com/bioinf-jku/TTUR).
The weights and the model are exactly the same as in [the official Tensorflow implementation](https://github.com/bioinf-jku/TTUR), and were tested to give very similar results (e.g. `.08` absolute error and `0.0009` relative error on LSUN, using ProGAN generated images). However, due to differences in the image interpolation implementation and library backends, FID results still differ slightly from the original implementation. So if you report FID scores in your paper, and you want them to be _exactly comparable_ to FID scores reported in other papers, you should consider using [the official Tensorflow implementation](https://github.com/bioinf-jku/TTUR).

## Installation

Expand All @@ -22,6 +22,7 @@ pip install pytorch-fid
```

Requirements:

- python3
- pytorch
- torchvision
Expand All @@ -32,6 +33,7 @@ Requirements:
## Usage

To compute the FID score between two datasets, where images of each dataset are contained in an individual folder:

```
python -m pytorch_fid path/to/dataset1 path/to/dataset2
```
Expand All @@ -49,14 +51,17 @@ The resulting scores might also no longer correlate with visual quality.

You can select the dimensionality of features to use with the flag `--dims N`, where N is the dimensionality of features.
The choices are:
- 64: first max pooling features
- 192: second max pooling features
- 768: pre-aux classifier features

- 64: first max pooling features
- 192: second max pooling features
- 768: pre-aux classifier features
- 2048: final average pooling features (this is the default)

## Generating a compatible `.npz` archive from a dataset

A frequent use case will be to compare multiple models against an original dataset.
To save training multiple times on the original dataset, there is also the ability to generate a compatible `.npz` archive from a dataset. This is done using any combination of the previously mentioned arguments with the addition of the `--save-stats` flag. For example:

```
python -m pytorch_fid --save-stats path/to/dataset path/to/outputfile
```
Expand Down
Loading
Loading