Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] remove convert_to_supported function #2217

Open
wants to merge 54 commits into
base: main
Choose a base branch
from

Conversation

icfaust
Copy link
Contributor

@icfaust icfaust commented Dec 9, 2024

Description

As part of the array_api rollout, moving the capability of convert_to_supported in the table creation is desired. As checking for datatypes for oneDAL should occur on the tables, and not on the arrays before they become tables. When they become tables, they can be converted to the supported type via the queue supplied to them. This logic only currently occurs with numpy arrays, as sycl_usm_ndarrays are not allowed in _device_offload.dispatch to be moved off their current device (i.e. when queue devices don't match, an error is thrown). Given they are on their current device, they are already in a supported format. Dlpack will be handled separately in its own PR.


PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.

You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.
For example, PR with docs update doesn't require checkboxes for performance while PR with any change in actual code should have checkboxes and justify how this code change is expected to affect performance (or justification should be self-evident).

Checklist to comply with before moving PR from draft:

PR completeness and readability

  • I have reviewed my changes thoroughly before submitting this pull request.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
  • Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
  • I have added a respective label(s) to PR if I have a permission for that.
  • I have resolved any merge conflicts that might occur with the base branch.

Testing

  • I have run it locally and tested the changes extensively.
  • All CI jobs are green or I have provided justification why they aren't.
  • I have extended testing suite if new functionality was introduced in this PR.

Performance

  • I have measured performance for affected algorithms using scikit-learn_bench and provided at least summary table with measured data, if performance change is expected.
  • I have provided justification why performance has changed or why changes are not expected.
  • I have provided justification why quality metrics have changed or why changes are not expected.
  • I have extended benchmarking suite and provided corresponding scikit-learn_bench PR if new measurable functionality was introduced in this PR.

@icfaust
Copy link
Contributor Author

icfaust commented Dec 9, 2024

/intelci: run

@icfaust
Copy link
Contributor Author

icfaust commented Dec 10, 2024

/intelci: run

@icfaust
Copy link
Contributor Author

icfaust commented Dec 10, 2024

/intelci: run

@icfaust icfaust marked this pull request as ready for review December 11, 2024 11:00
@icfaust
Copy link
Contributor Author

icfaust commented Dec 11, 2024

/intelci: run

Copy link
Contributor

@ahuber21 ahuber21 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the 3.12 error. Message below

__________________ ERROR collecting tests/test_estimators.py ___________________
tests/test_estimators.py:19: in <module>
    import sklearn.utils.estimator_checks
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.12/site-packages/sklearn/utils/estimator_checks.py:89: in <module>
    from ._test_common.instance_generator import (
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.12/site-packages/sklearn/utils/_test_common/instance_generator.py:179: in <module>
    from sklearn.utils._testing import SkipTest
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.12/site-packages/sklearn/utils/_testing.py:318: in <module>
    _check_array_api_dispatch(True)
../_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl/lib/python3.12/site-packages/sklearn/utils/_array_api.py:123: in _check_array_api_dispatch
    raise RuntimeError(
E   RuntimeError: Scikit-learn array API support was enabled but scipy's own support is not enabled. Please set the SCIPY_ARRAY_API=1 environment variable before importing sklearn or scipy. More details at: https://docs.scipy.org/doc/scipy/dev/api-dev/array_api.html

Copy link
Contributor

@ahuber21 ahuber21 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, didn't realize the array API error was a 1.6 issue

@@ -101,21 +101,20 @@ def partial_fit(self, X, y=None, queue=None):

policy = self._get_policy(queue, X)

X = _convert_to_supported(policy, X)
X_table = to_table(X, queue=queue)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see 2 variants of implementation in this PR:

  1. X = to_table(X, queue=queue)
  2. X_table = to_table(X, queue=queue)

Here 2) is used, but it seems X is not needed after the conversion anyway. Why not to use 1) everywhere possible?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, although it looks like generally the name is just matching the name that was previously used after to_table was called before these changes

Copy link
Contributor

@Vika-F Vika-F left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good to me.
There are minor inconsistencies, but not critical. And they were there before the changes.

@ethanglaser
Copy link
Contributor

Why is sklearn 1.6 being installed?

@ethanglaser
Copy link
Contributor

/intelci: run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants