Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement] Store windows basekit with symlink-preserving compression for nightly-build #2938

Merged
merged 11 commits into from
Oct 17, 2024

Conversation

icfaust
Copy link
Contributor

@icfaust icfaust commented Oct 8, 2024

Description

actions/upload-artifact#625 included in 4.4.1 fixes an issue to not preserve symlinks (since fixed in later versions). The Windows Intel oneAPI BaseKit installer uses absolute paths in symlinks rather than relative ones for the latest folders of various oneAPI components. This makes a maintenance burden on sklearnex-side since they are unusable. Therefore the fully path must be known and maintained to the compiler and tbb versionvars.bat files. A rather long and complicated batch script line was added to force these to be relative. This removed the maintenance burden by fixing the latest symlink to work in sklearnex out-of-the-box. Since it is only one line and fixes what hopefully will only be a temporary issue I don't think necessitates its own script. Because of the complication of it, comments were added for future understanding.

This will reduce maintenance burden on sklearnex side by making the CI pipeline run independent of the tbb/mkl/compiler versions


Checklist to comply with before moving PR from draft:

PR completeness and readability

  • I have reviewed my changes thoroughly before submitting this pull request.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
  • Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
  • I have added a respective label(s) to PR if I have a permission for that.
  • I have resolved any merge conflicts that might occur with the base branch.

Testing

  • I have run it locally and tested the changes extensively.
  • All CI jobs are green or I have provided justification why they aren't.
  • I have extended testing suite if new functionality was introduced in this PR.

Performance

  • I have measured performance for affected algorithms using scikit-learn_bench and provided at least summary table with measured data, if performance change is expected.
  • I have provided justification why performance has changed or why changes are not expected.
  • I have provided justification why quality metrics have changed or why changes are not expected.
  • I have extended benchmarking suite and provided corresponding scikit-learn_bench PR if new measurable functionality was introduced in this PR.

@icfaust icfaust added the infra label Oct 8, 2024
@icfaust icfaust marked this pull request as ready for review October 8, 2024 13:04
@icfaust
Copy link
Contributor Author

icfaust commented Oct 10, 2024

Example run using matching main from my fork: https://github.com/icfaust/oneDAL/actions/runs/11263646195

@icfaust icfaust changed the title [enhancement] Store Nightly-CI artifacts using symlink-preserving compression [enhancement] Store windows basekit with symlink-preserving compression for nightly-build Oct 17, 2024
Copy link
Contributor

@Alexandr-Solovev Alexandr-Solovev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just one question why the upload-artifact version has been decreased?

@icfaust
Copy link
Contributor Author

icfaust commented Oct 17, 2024

Latest run showing valid operation of the nightly-build: https://github.com/icfaust/oneDAL/actions/runs/11373066491

@icfaust
Copy link
Contributor Author

icfaust commented Oct 17, 2024

Here is the latest run on my scikit-learn-intelex fork with the successful run https://github.com/icfaust/scikit-learn-intelex/actions/runs/11378786935

@icfaust
Copy link
Contributor Author

icfaust commented Oct 17, 2024

LGTM, just one question why the upload-artifact version has been decreased?

Good question, the version without the later specifiers will now use the latest version of v4, I thought this would be nicer to prevent a large number of renovate-bot PRs for upgrading it. They have corrected the symlink issues with the allowance of relative symlinks allowing for the builds to be as they were, so the version pin to 4.4.0 can be removed.

@icfaust icfaust merged commit ce6bc85 into uxlfoundation:main Oct 17, 2024
18 of 19 checks passed
icfaust added a commit to uxlfoundation/scikit-learn-intelex that referenced this pull request Oct 17, 2024
…L#2938 (#2093)

* Update ci.yml

* Update activate_components.bat

* Update ci.yml

* Update ci.yml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants