From 234434af51afea28fedcf7244cf99acf47f6eeae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Caron?= Date: Wed, 31 Jul 2024 20:53:57 +0200 Subject: [PATCH] Fix reporting of solver versions when some solvers are excluded --- CHANGELOG.md | 134 ++++++++++++++++++++++--------------------- qpbenchmark/utils.py | 19 +++--- 2 files changed, 79 insertions(+), 74 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3fed002..7798d1a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,159 +4,163 @@ All notable changes to this project will be documented in this file. ## [Unreleased] +### Fixed + +- Fix reporting of solver versions when some solvers are excluded + ## [2.2.2] - 2024-07-31 ### Added -* Expose exceptions from top-level module +- Expose exceptions from top-level module ### Fixed -* Bump Python version to actual requirement of 3.9 or above -* Handle no-op conversion from dense to dense properly -* Handle no-op conversion from sparse to sparse properly +- Bump Python version to actual requirement of 3.9 or above +- Handle no-op conversion from dense to dense properly +- Handle no-op conversion from sparse to sparse properly ## [2.2.1] - 2024-02-06 ### Changed -* Update qpsolvers to v4.3.1 to fix the Gurobi interface (thanks @563925743) +- Update qpsolvers to v4.3.1 to fix the Gurobi interface (thanks @563925743) ## [2.2.0] - 2024-01-16 ### Added -* Distribute [on conda-forge](https://anaconda.org/conda-forge/qpbenchmark) -* Test set path argument to the main function of the command-line interface +- Distribute [on conda-forge](https://anaconda.org/conda-forge/qpbenchmark) +- Test set path argument to the main function of the command-line interface ### Changed -* Update environment file to install from conda-forge +- Update environment file to install from conda-forge ## [2.1.1] - 2023-12-22 ### Changed -* Clarabel: set ``tol_gap_abs`` settings in ``set_eps_abs`` -* Clarabel: set ``tol_gap_rel`` settings in ``set_eps_rel`` -* Log a warning message when skipping known solver issue -* PIQP: set ``eps_duality_gap_abs`` settings in ``set_eps_abs`` -* PIQP: set ``eps_duality_gap_rel`` settings in ``set_eps_rel`` -* ProxQP: set ``eps_duality_gap_abs`` settings in ``set_eps_abs`` -* ProxQP: set ``eps_duality_gap_rel`` settings in ``set_eps_rel`` +- Clarabel: set ``tol_gap_abs`` settings in ``set_eps_abs`` +- Clarabel: set ``tol_gap_rel`` settings in ``set_eps_rel`` +- Log a warning message when skipping known solver issue +- PIQP: set ``eps_duality_gap_abs`` settings in ``set_eps_abs`` +- PIQP: set ``eps_duality_gap_rel`` settings in ``set_eps_rel`` +- ProxQP: set ``eps_duality_gap_abs`` settings in ``set_eps_abs`` +- ProxQP: set ``eps_duality_gap_rel`` settings in ``set_eps_rel`` ### Fixed -* Handling of known solver issues +- Handling of known solver issues ## [2.1.0] - 2023-12-21 ### Added -* Expose `Problem` from top-level module -* Link in report header to go to results tables directly -* Number of problems in report header -* Utility function to load a problem from file +- Expose `Problem` from top-level module +- Link in report header to go to results tables directly +- Number of problems in report header +- Utility function to load a problem from file ### Changed -* **Breaking:** Remove the cost error from benchmark metrics -* Include Python 3.8 in supported versions +- **Breaking:** Remove the cost error from benchmark metrics +- Include Python 3.8 in supported versions ### Removed -* Move the GitHub Free-for-all test set to [its own repository](https://github.com/qpsolvers/free_for_all_qpbenchmark) -* Move Maros-Meszaros test set to [its own repository](https://github.com/qpsolvers/maros_meszaros_qpbenchmark) +- Move the GitHub Free-for-all test set to [its own repository](https://github.com/qpsolvers/free_for_all_qpbenchmark) +- Move Maros-Meszaros test set to [its own repository](https://github.com/qpsolvers/maros_meszaros_qpbenchmark) ## [2.0.0] - 2023-12-11 ### Added -* New solver: QPALM +- New solver: QPALM ### Changed -* **Breaking:** Rename the project to "qpbenchmark" -* Don't install solvers by default from PyPI -* Move solver issue and timeout listings to test-set themselves +- **Breaking:** Rename the project to "qpbenchmark" +- Don't install solvers by default from PyPI +- Move solver issue and timeout listings to test-set themselves ### Removed -* Module-wide skip solver issue/timeout functions +- Module-wide skip solver issue/timeout functions ## [1.2.0] - 2023-11-27 ### Added -* Citation file to refer to the benchmark in scientific works -* Medium accuracy settings -* Note in the readme on CPU throttling +- Citation file to refer to the benchmark in scientific works +- Medium accuracy settings +- Note in the readme on CPU throttling ## [1.1.0] - 2023-09-08 ### Added -* Check consistency after loading results -* More CPU information in reports -* New solver: HPIPM -* New solver: PIQP +- Check consistency after loading results +- More CPU information in reports +- New solver: HPIPM +- New solver: PIQP ### Changed -* Don't hard-wrap report lines, as it doesn't render well in Discussions -* Improve reporting of shifted geometric mean errors -* Make `cpuinfo` a proper dependency -* Refactor results class to allow finer `check_results` sessions -* Update to qpsolvers v4.0.0 +- Don't hard-wrap report lines, as it doesn't render well in Discussions +- Improve reporting of shifted geometric mean errors +- Make `cpuinfo` a proper dependency +- Refactor results class to allow finer `check_results` sessions +- Update to qpsolvers v4.0.0 ### Fixed -* Correct `None` values to `False` in found column -* Make sure found column has only boolean values +- Correct `None` values to `False` in found column +- Make sure found column has only boolean values ## [1.0.0] - 2023-07-25 ### Added -* Allow non-lowercase solver names in the command line (thanks to @ottapav) -* Command-line tool and standalone test sets (thanks to @ZAKIAkram) +- Allow non-lowercase solver names in the command line (thanks to @ottapav) +- Command-line tool and standalone test sets (thanks to @ZAKIAkram) ### Changed -* Plot: trim solutions that don't fulfill tolerance requirements -* Rename ``hist`` command to ``plot`` -* Update to qpsolvers v3.4.0 +- Plot: trim solutions that don't fulfill tolerance requirements +- Rename ``hist`` command to ``plot`` +- Update to qpsolvers v3.4.0 ### Fixed -* Plot whiskers on solutions beyond tolerance requirements (thanks to @ottapav) +- Plot whiskers on solutions beyond tolerance requirements (thanks to @ottapav) ## [0.1.0-beta] - 2022-01-26 ### Added -* Check dual residual -* Check duality gap -* Document all benchmark functions -* Main script: new ``hist`` plot command -* ProblemNotFound exception -* Results by settings in reports -* Write benchmark version in reports +- Check dual residual +- Check duality gap +- Document all benchmark functions +- Main script: new ``hist`` plot command +- ProblemNotFound exception +- Results by settings in reports +- Write benchmark version in reports ### Changed -* Benchmark script takes test set as first argument -* Maros-Meszaros: empty equality constraints are now set to ``None`` -* Re-run benchmark with ProxQP 0.3.2 -* Refactor Report class and run function -* Report encoding is now UTF-8 -* Switch to qpsolvers v2.7 -* Test set descriptions are now mandatory +- Benchmark script takes test set as first argument +- Maros-Meszaros: empty equality constraints are now set to ``None`` +- Re-run benchmark with ProxQP 0.3.2 +- Refactor Report class and run function +- Report encoding is now UTF-8 +- Switch to qpsolvers v2.7 +- Test set descriptions are now mandatory ### Fixed -* Conform to linter standards -* Sparse matrix conversion +- Conform to linter standards +- Sparse matrix conversion ## [0.1.0-alpha] - 2022-12-21 diff --git a/qpbenchmark/utils.py b/qpbenchmark/utils.py index 249d971..70145f0 100644 --- a/qpbenchmark/utils.py +++ b/qpbenchmark/utils.py @@ -72,15 +72,16 @@ def get_solver_versions(solvers: Set[str]): Returns: Dictionary mapping solver names to their versions. """ - package_name = {solver: solver for solver in solvers} - package_name.update( - { - "gurobi": "gurobipy", - "highs": "highspy", - "hpipm": "hpipm_python", - "proxqp": "proxsuite", - } - ) + diff = { + "gurobi": "gurobipy", + "highs": "highspy", + "hpipm": "hpipm_python", + "proxqp": "proxsuite", + } + package_name = { + solver: solver if solver not in diff else diff[solver] + for solver in solvers + } versions = {} for solver, package in package_name.items(): try: