Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] basic "test all estimators" suite #89

Merged
merged 13 commits into from
Oct 8, 2023
Merged

Conversation

fkiraly
Copy link
Collaborator

@fkiraly fkiraly commented Sep 12, 2023

Adds a TestAllEstimators which checks generic fit/predict capabilities of estimators.

This is partly migrated from sktime, and uses the scikit-base test framework classes (also originally from sktime).

This can also be used as a joint refactor point with sktime and skbase.

Additions and changes:

  • TestAllEstimators class
  • this requires a BaseFixtureGenerator which generates scenarios. For this, the skpro fixture generator now inherits from skbase.
  • Also, scenarios are required. This is not available in skbase yet, so the sktime scenario framework is copied to tests.scenarios.
  • In incremental testing, an override is introduced so all classes are tested if the central test framework module changes.

@codecov-commenter
Copy link

codecov-commenter commented Oct 8, 2023

Codecov Report

Attention: 77 lines in your changes are missing coverage. Please review.

Comparison is base (4d05959) 71.67% compared to head (6129c9a) 71.63%.
Report is 1 commits behind head on main.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #89      +/-   ##
==========================================
- Coverage   71.67%   71.63%   -0.05%     
==========================================
  Files          93       97       +4     
  Lines        4950     5157     +207     
  Branches      900      952      +52     
==========================================
+ Hits         3548     3694     +146     
- Misses       1173     1206      +33     
- Partials      229      257      +28     
Files Coverage Δ
skpro/utils/random_state.py 100.00% <100.00%> (ø)
skpro/tests/scenarios/scenarios_regressor_proba.py 79.48% <79.48%> (ø)
skpro/tests/test_all_estimators.py 88.33% <83.63%> (+4.27%) ⬆️
skpro/tests/scenarios/scenarios_getter.py 47.36% <47.36%> (ø)
skpro/tests/scenarios/scenarios.py 42.02% <42.02%> (ø)

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@fkiraly fkiraly marked this pull request as ready for review October 8, 2023 18:09
@fkiraly fkiraly added the module:tests test framework functionality - only framework, excl specific tests label Oct 8, 2023
@fkiraly fkiraly merged commit b18d08e into main Oct 8, 2023
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement module:tests test framework functionality - only framework, excl specific tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants