Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rocm-related: 5.3.0 -> 5.3.1 & aggregate #198770

Merged
merged 23 commits into from
Nov 6, 2022
Merged

rocm-related: 5.3.0 -> 5.3.1 & aggregate #198770

merged 23 commits into from
Nov 6, 2022

Conversation

Madouura
Copy link
Contributor

@Madouura Madouura commented Oct 31, 2022 •
edited
Loading

Description of changes

Most hashes did not change, version bump to maintain compatibility.
Tracking: #197885

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

This was referenced Oct 31, 2022
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Open
@ofborg ofborg bot requested review from Flakebi, lovesegfault and acowley October 31, 2022 10:07
Flakebi
Flakebi approved these changes Oct 31, 2022
View reviewed changes
Copy link
Member

@Flakebi Flakebi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again!

I tested some OpenCL and hip applications and they worked fine.

@LunNova
Copy link
Member

LunNova commented Oct 31, 2022

Result of nixpkgs-review pr 198770 run on x86_64-linux 1

15 packages built:
  • blender-hip
  • hip
  • llvmPackages_rocm.clang
  • llvmPackages_rocm.clangNoCompilerRt
  • llvmPackages_rocm.llvm
  • rocclr
  • rocm-cmake
  • rocm-comgr
  • rocm-device-libs
  • rocm-opencl-icd
  • rocm-opencl-runtime
  • rocm-runtime
  • rocm-smi
  • rocm-thunk
  • rocminfo

@Madouura Madouura changed the title rocm-related: 5.3.0 -> 5.3.1 rocm-related: 5.3.0 -> 5.3.1 & aggregate Nov 4, 2022
@Madouura
Copy link
Contributor Author

Madouura commented Nov 4, 2022

@lovesegfault Rolled into this PR, since they all rely on this PR anyway.

This was referenced Nov 4, 2022
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
@ofborg ofborg bot added 8.has: package (new) This PR adds a new package and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Nov 4, 2022
@ofborg ofborg bot requested a review from Flakebi November 4, 2022 13:31
lovesegfault
lovesegfault approved these changes Nov 4, 2022
View reviewed changes
Copy link
Member

@lovesegfault lovesegfault left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Will wait on @Flakebi's review to merge

@Madouura
Copy link
Contributor Author

Madouura commented Nov 4, 2022 •
edited
Loading

Rebuilt everything to make sure this PR is safe to merge.

Final result:

nixpkgs on  pr/aggr [$] took 42m42s 
❯ nix-build . -A llvmPackages_rocm.llvm -A llvmPackages_rocm.clang -A hip -A rocclr -A rocm-comgr -A rocm-device-libs -A rocm-opencl-runtime -A rocm-runtime -A rocm-thunk -A rocm-cmake -A rocminfo -A rocm-smi -A rocprim -A hipcub -A rocsparse -A hipsparse -A rccl -A rocfft -A tensile -A rocblas -A miopengemm -A composable_kernel -A clang-ocl -A rocthrust
/nix/store/qn2f45hc8qi4214ss1dcmrxp5vl0iaw1-rocm-llvm-5.3.1
/nix/store/4nvx5fla45198skmxihg83wlw0k8rr57-rocm-llvm-wrapper-5.3.1
/nix/store/9mpxwijw14zkzlz2icalfkh6pb408zmq-hip-5.3.1
/nix/store/sss84hpsbfgm2xd9va9fjb23x7dfwrnj-rocclr-5.3.1
/nix/store/6vaagd85p5zwgmrfyraz1pgirbpaj73r-rocm-comgr-5.3.1
/nix/store/cf91952wf8dy4fjmjyqi5as41gjb3kc2-rocm-device-libs-5.3.1
/nix/store/a47gbycs0lfzrbfhd6s67q65xsq13rn2-rocm-opencl-runtime-5.3.1
/nix/store/9b0rir8yazpb40b1gm2xf5ljyn7ymn34-rocm-runtime-5.3.1
/nix/store/zbj0m25c60v73fr20g113m783xdgrppi-rocm-thunk-5.3.1
/nix/store/d5x36ns4lg734wkbyqbdb6ab7iyi5vda-rocm-cmake-5.3.1
/nix/store/ybbbxlxgwqp4y2jgdkjr1ddjsys9364g-rocminfo-5.3.1
/nix/store/2q6il1lcanja2w39w26r15hxxrzip287-rocm-smi-5.3.1
/nix/store/hgxqja6cjbrirzx46m8cwyx4jdy6v884-rocprim-2.11.0-5.3.1
/nix/store/s66c7hw4fgb8vc80qyjplcl6shjdxfk5-hipcub-2.12.0-5.3.1
/nix/store/a6xnva6sihlh9dav373j54zkhwzdjd3v-rocsparse-2.3.2-5.3.1
/nix/store/9l231q7bwndn2parilii8w164j6ry6c1-hipsparse-2.3.1-5.3.1
/nix/store/jp0gnv8m96m2zd8yy5ayfv4mv16gjd6x-rccl-2.12.10-5.3.1
/nix/store/80biz86ywzh6q1sxvs4xi1j72rywg185-rocfft-1.0.18-5.3.1
/nix/store/3ak9ks0bbxdp20ph0jiyvy5s2519jg0l-python3.10-tensile-4.34.0-5.3.1
/nix/store/xv3dshh1fsbr0yb41qhmrhivy7x57z17-rocblas-2.45.0-5.3.1
/nix/store/cpgzc3n4h9lml6n0y6kfc5hwqvw88l8r-miopengemm-5.3.1
/nix/store/6y2b4y2a94vqv8g1dfzkga7x9r1r1j74-composable_kernel-unstable-2022-11-02
/nix/store/mickbgzbca3x0ckcc3nirmj8vm7lq3yf-clang-ocl-5.3.1
/nix/store/lm5jm9mj548fcjknf3n5b7hyrlifffv8-rocthrust-2.16.0-5.3.1

Also double checked all package roots with find ${pkg_dir}, everything checks out.

@lovesegfault lovesegfault merged commit 1577208 into NixOS:master Nov 6, 2022
@Madouura Madouura deleted the pr/aggr branch November 6, 2022 21:14
Flakebi
Flakebi reviewed Nov 9, 2022
View reviewed changes
Copy link
Member

@Flakebi Flakebi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, just two questions inline.

pkgs/development/libraries/rocsparse/default.nix
Comment on lines +27 to +31
matrices = lib.optionalAttrs (buildTests || buildBenchmarks) import ./deps.nix {
inherit fetchzip;
mirror1 = "https://sparse.tamu.edu/MM";
mirror2 = "https://www.cise.ufl.edu/research/sparse/MM";
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these the same as in hipsparse/deps.nix?
Maybe you could add them as rocsparse.passthrough.matrices and import them in hipsparse instead of duplicating them?

Copy link
Contributor Author

@Madouura Madouura Nov 9, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea, I'll add it to a TODO list in the tracking issue.
If for some reason hipsparse begins using different matricies, as long as some of them are the same we can still do this as well.

pkgs/development/libraries/rocsparse/default.nix
outputs = [
"out"
] ++ lib.optionals (buildTests || buildBenchmarks) [
"test"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can I run the tests? I get

$ result-test/bin/rocsparse-test
"hipErrorNoBinaryForGpu: Unable to find code object for all current devices!"
fish: Job 1, 'result-test/bin/rocsparse-test' terminated by signal SIGABRT (Abort)

I guess it either needs something in my environment or I have to specify the GPU somewhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably yes, see: https://github.com/Madouura/nixos-config/blob/0e4236c18a468abd300b3df5c2657dba6f763ec0/hosts/machines/ura/modules/system/hardware.nix#L6-L9

@vcunat
Copy link
Member

vcunat commented Nov 11, 2022

rocfft produces 3.6 GiB output. That's too large, so Hydra.nixos.org always fails after finishing the costly build. Unless you can improve this, I suggest adding meta.hydraPlatforms = []; so that this pointless build isn't repeated there every time without succeeding.

@Madouura
Copy link
Contributor Author

AFAIK it can't be improved, it's just a ridiculously large build.
IIRC, it's all in /lib too.
I'll submit a PR for it.

@Madouura Madouura mentioned this pull request Nov 11, 2022
Merged
13 tasks
@Madouura
Copy link
Contributor Author

@vcunat #200705

@vcunat
Copy link
Member

vcunat commented Nov 11, 2022

Yes, I looked before, it's gigabytes of so-libraries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Flakebi Flakebi Flakebi approved these changes

@lovesegfault lovesegfault lovesegfault approved these changes

@LunNova LunNova LunNova approved these changes

@matthewbauer matthewbauer Awaiting requested review from matthewbauer

@acowley acowley Awaiting requested review from acowley

Assignees
No one assigned
Labels
8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 10.rebuild-linux: 11-100
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Madouura @LunNova @vcunat @Flakebi @lovesegfault