Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] merge test_probabilistic_metrics into TestAllDistrMetrics #398

Open
fkiraly opened this issue Jun 17, 2024 · 1 comment
Open

[ENH] merge test_probabilistic_metrics into TestAllDistrMetrics #398

fkiraly opened this issue Jun 17, 2024 · 1 comment
Labels
good first issue Good for newcomers module:metrics&benchmarking metrics and benchmarking modules module:tests test framework functionality - only framework, excl specific tests

Comments

@fkiraly
Copy link
Collaborator

fkiraly commented Jun 17, 2024

The module test_probabilistic_metrics contains tests for interval and quantile distributions with manual collection. These metrics are also automatically collected in TestAllDistrMetrics, which should be the location where these tests are carried out systematically.

Hence, we should move the existing tests in test_probabilistic_metrics to there.

Whether a metric is an interval or quantile metric can be inspectedby reading the "scitype:y_pred" tag.

@fkiraly fkiraly added good first issue Good for newcomers module:metrics&benchmarking metrics and benchmarking modules module:tests test framework functionality - only framework, excl specific tests labels Jun 17, 2024
@fkiraly
Copy link
Collaborator Author

fkiraly commented Jun 17, 2024

FYI @benHeid, turns out skpro is halfway to the point of a fully programmatic test suite for the proba metrics. The quantile and interval metrics will miss out on some interface tests when checked via check_estimator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers module:metrics&benchmarking metrics and benchmarking modules module:tests test framework functionality - only framework, excl specific tests
Projects
None yet
Development

No branches or pull requests

1 participant