Enable reproducible comparing tests on jdk21+ macos #3844

sophia-guo · 2024-06-10T15:12:39Z

Depends adoptium/ci-jenkins-pipelines#1136

sophia-guo · 2024-06-10T19:47:47Z

I have tried both static mac and orka. Both no jq installed.

https://ci.adoptium.net/view/Test_grinder/job/Grinder/10327/console
https://ci.adoptium.net/view/Test_grinder/job/Grinder/10328

Error: jq is not installed. Please install jq before running this script

Might be easy to install jq on static mac and orka?

sophia-guo · 2024-06-18T21:19:37Z

https://ci.adoptium.net/view/Test_grinder/job/Grinder/10427/

17:21:04  Xcode 10.3
17:21:04  Build version 10G8
17:21:04  Xcode Build Success - Get Version
17:21:04  ERROR - No Xcode Installation Matching 15.2 Could Be Identified

Does this mean we can't do reproducing on this environment https://ci.adoptium.net/computer/test-macstadium-macos1014-x64-1/ due to the Xcode version not matching. @steelhead31 ?

steelhead31 · 2024-06-18T21:37:23Z

https://ci.adoptium.net/view/Test_grinder/job/Grinder/10427/
17:21:04  Xcode 10.3
17:21:04  Build version 10G8
17:21:04  Xcode Build Success - Get Version
17:21:04  ERROR - No Xcode Installation Matching 15.2 Could Be Identified
Does this mean we can't do reproducing on this environment https://ci.adoptium.net/computer/test-macstadium-macos1014-x64-1/ due to the Xcode version not matching. @steelhead31 ?

Yup, that machines OS I think is too old for Xcode 15.2, and I think it has to be the same version of Xcode for reproducibility to work, @andrew-m-leonard can confirm this I think :)

andrew-m-leonard · 2024-06-19T08:22:02Z

https://ci.adoptium.net/view/Test_grinder/job/Grinder/10427/
17:21:04  Xcode 10.3
17:21:04  Build version 10G8
17:21:04  Xcode Build Success - Get Version
17:21:04  ERROR - No Xcode Installation Matching 15.2 Could Be Identified
Does this mean we can't do reproducing on this environment https://ci.adoptium.net/computer/test-macstadium-macos1014-x64-1/ due to the Xcode version not matching. @steelhead31 ?
Yup, that machines OS I think is too old for Xcode 15.2, and I think it has to be the same version of Xcode for reproducibility to work, @andrew-m-leonard can confirm this I think :)

Yes, correct Xcode 15.2 requires MacOS 13.5 minimum : https://developer.apple.com/support/xcode/

sophia-guo · 2024-06-19T12:46:04Z

So we need extra label for macos to ensure the reproducing tests run on correct environment. Ether xcode15.2 or sw.os.osx.13_5. Two Mac static agents wont work for reproducing tests. https://ci.adoptium.net/label/ci.role.test&&hw.arch.x86&&%28sw.os.osx%7C%7Csw.os.mac%29/

Current orka ones are label as xcode15.0.1&&build&&mac&&x64 so can't work for reproducing either. Should we open an issue in infra to see if it's possible to update orka's os version?

sophia-guo · 2024-06-19T12:54:27Z

Interesting orka ones with label xcode15.0.1&&build&&mac&&x64 can do the reproducing build. https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-mac-x64-temurin/146/ https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-mac-x64-temurin/146/artifact/workspace/target/OpenJDK21U-sbom_x64_mac_hotspot_2024-05-31-22-18.json

andrew-m-leonard · 2024-06-19T13:42:36Z

Interesting orka ones with label xcode15.0.1&&build&&mac&&x64 can do the reproducing build. https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-mac-x64-temurin/146/ https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-mac-x64-temurin/146/artifact/workspace/target/OpenJDK21U-sbom_x64_mac_hotspot_2024-05-31-22-18.json

@sophia-guo so the Orka nodes are using Xcode 15.2, the 15.0.1 label has not been changed just because of the hassle involved! should have been called Xcode15 !

sophia-guo · 2024-06-26T20:13:09Z

orka updated with jq rerun adoptium/infrastructure#3593

https://ci.adoptium.net/view/Test_grinder/job/Grinder/10470/console

sophia-guo · 2024-06-28T20:17:11Z

16:32:23  Make JDK Any Platform Argument List = 
16:32:23  --clean-git-repo --jdk-boot-dir /Users/admin/comp-jdk-build/jdk-20.0.2+9/Contents/Home --target-file-name OpenJDK21U-jdk_x64_mac_hotspot_21.0.4_5-ea.tar.gz --disable-shallow-git-clone -bjdk-21.0.4+5_adopt --create-jre-image --create-sbom --build-reproducible-date "2024-06-25 11:04:23"--user-openjdk-build-root-directory /Users/admin/comp-jdk-build/temurin-build/workspace/build/openjdkbuild/ --freetype-dir bundled --use-jep319-certs --create-debug-image  --configure-args "--disable-warnings-as-errors --openjdk-target=x86_64-apple-darwin --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk --with-extra-cxxflags='-mmacosx-version-min=10.9' --enable-dtrace --with-version-opt=ea"" --build-variant temurin jdk21u 
16:32:23  
16:32:23  Parameters Parsed Successfully
16:32:23  ---------------------------------------------
16:32:23  Building JDK...
16:32:23  sh: line 1: unexpected EOF while looking for matching `"'
16:32:23  sh: line 2: syntax error: unexpected end of file
16:32:23  -----------------------------------
16:32:23  Rebuild_Same_JDK_Reproducibility_Test_Mac_0_FAILED

parsing issue as SBOM --configure-args value format updated with double quotes. Parsing in mac script also need to update.

sophia-guo · 2024-08-16T21:04:31Z

New grinder with latest and label https://ci.adoptium.net/label/ci.role.test&&sw.os.osx&&orka&&hw.arch.x86/ to make sure running on orka

https://ci.adoptium.net/view/Test_grinder/job/Grinder/10769/ - waiting

sophia-guo · 2024-08-20T15:42:57Z

Running with orka, issue message complains the Xcode can't be found.
https://ci.adoptium.net/view/Test_grinder/job/Grinder/10771/console.

Test is running on orka node , which should have Xcode 15.2 available #3844 (comment)
https://ci.adoptium.net/label/ci.role.test&&sw.os.osx&&orka&&hw.arch.x86/

11:27:51  Compiler Version Returned From SBOM: clang (clang/LLVM from Xcode 15.2)
11:27:51  Checking For Xcode Compilers In : /Applications/
11:27:51  Error - An Xcode Installaton Could Not Be Found In The Default Path : /Applications / Xcode 
11:27:51  -----------------------------------
11:27:51  Rebuild_Same_JDK_Reproducibility_Test_Mac_0_FAILED

sophia-guo · 2024-08-20T16:07:57Z

#3844 (comment)

@steelhead31 If it's related with how the Xcode installation is checked? #3844 (comment)

sophia-guo · 2024-08-20T16:10:45Z

rerun on build orka node https://ci.adoptium.net/label/xcode15.0.1&&build&&mac&&x64/ got different error from running on test orka node https://ci.adoptium.net/label/ci.role.test&&sw.os.osx&&orka&&hw.arch.x86/

https://ci.adoptium.net/view/Test_grinder/job/Grinder/10775/console

12:01:08  ---------------------------------------------
12:01:08  Building JDK...
12:01:08  sh: line 1: unexpected EOF while looking for matching `"'
12:01:08  sh: line 2: syntax error: unexpected end of file
12:01:08  -----------------------------------
12:01:08  Rebuild_Same_JDK_Reproducibility_Test_Mac_0_FAILED
12:01:08  -----------------------------------

steelhead31 · 2024-08-21T09:20:09Z

@sophia-guo I've put some debugging in, and it does appear the script is checking correctly, but not finding an Xcode installation on that Orka node..

00:02:46.365 ---------------------------------------------
00:02:46.365 System Architecture: 64-bit (Intel)
00:02:46.365 SBOM & SYSTEM Architectures Match - All OK
00:02:46.365 ---------------------------------------------
00:02:46.365 Checking For Supported Compiler Versions...
00:02:46.365
00:02:46.365 Compiler Version Returned From SBOM: clang (clang/LLVM from Xcode 15.2)
00:02:46.366 Checking For Xcode Compilers In : /Applications/
00:02:46.366 Debug Xcode Find
00:02:46.366 total 0
00:02:46.366 drwxr-xr-x 3 root wheel 96 Oct 20 2023 Utilities
00:02:46.366 lrwxr-xr-x@ 1 root wheel 54 Oct 20 2023 Safari.app -> ../System/Cryptexes/App/System/Applications/Safari.app
00:02:46.366 drwxr-xr-x 3 root wheel 96 Jul 10 03:52 Packages.app
00:02:46.366 End Debug
00:02:46.366 Error - An Xcode Installaton Could Not Be Found In The Default Path : /Applications / Xcode

steelhead31 · 2024-08-21T09:33:53Z

@sophia-guo I've run a grinder here https://ci.adoptium.net/view/Test_grinder/job/Grinder/10782/console with these labels.. ( borrowed from a build job )

xcode15.0.1&&build&&mac&&x64&&!build-orka-macos14-arm64-fwc8f

So this issue is down to the orka test nodes not having Xcode installed... I'll take a look at the other issue now

steelhead31 · 2024-08-21T10:34:04Z

@sophia-guo I've fixed the issue with the repro script, and it now attempts to build, but its complaining about a missing SPEC.mk file..

Error: Cannot locate spec.gmk, given by SPEC=osx_x86-64.

@andrew-m-leonard may also be able to help in my absence.

See the grinder https://ci.adoptium.net/view/Test_grinder/job/Grinder/10791/console

sophia-guo · 2024-09-17T14:59:21Z

@andrew-m-leonard what should we do with option --assemble-exploded-image on mac? Should we just ignore it?

sophia-guo · 2024-09-17T17:50:14Z

https://ci.adoptium.net/view/Test_grinder/job/Grinder/10922/console @andrew-m-leonard . This is the results by reproduced on arm64 by comparing with repro_compare.sh. Anything wrong?

Regarding to run this x64 tests on arm64, I haven't got a good solution for this. Right now we can enable as part of dev.system and set a special rule like dev.system on mac only run on arm64, as there is no other tests in this group right now. But going forward this is not correct.

andrew-m-leonard · 2024-09-19T12:25:12Z

https://ci.adoptium.net/view/Test_grinder/job/Grinder/10922/console @andrew-m-leonard . This is the results by reproduced on arm64 by comparing with repro_compare.sh. Anything wrong?

Regarding to run this x64 tests on arm64, I haven't got a good solution for this. Right now we can enable as part of dev.system and set a special rule like dev.system on mac only run on arm64, as there is no other tests in this group right now. But going forward this is not correct.

@sophia-guo It looks like it's not picking up a good JDK for running the Java compare classes:

18:31:20  Cleaning any Temurin build-scripts release file BUILD_INFO from src_jdk/Contents/Home
18:31:21  Error: Fatal error: Unsupported class file version: 67

sophia-guo · 2024-09-24T21:03:17Z

@sophia-guo It looks like it's not picking up a good JDK for running the Java compare classes:

@andrew-m-leonard I don't quite understand this. Source jdk is copied from build https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk23/job/jdk23-mac-x64-temurin/16/artifact/workspace/target/OpenJDK23U-jdk_x64_mac_hotspot_23_37-ea.tar.gz. We can successfully rebuild the compared one. Both have this issue Error: Fatal error: Unsupported class file version: 67

But the compare results is exactly same as when we use the independent jenkins job (thought it's jdk21) https://ci.adoptium.net/view/ReproducibleBuild/job/jdk21u-mac-x64-temurin_reproduce_compare/74/consoleFull

So I'm thinking this might not be issues from test view?

andrew-m-leonard · 2024-09-25T08:33:56Z

@sophia-guo It looks like it's not picking up a good JDK for running the Java compare classes:

@andrew-m-leonard I don't quite understand this. Source jdk is copied from build https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk23/job/jdk23-mac-x64-temurin/16/artifact/workspace/target/OpenJDK23U-jdk_x64_mac_hotspot_23_37-ea.tar.gz. We can successfully rebuild the compared one. Both have this issue Error: Fatal error: Unsupported class file version: 67

But the compare results is exactly same as when we use the independent jenkins job (thought it's jdk21) https://ci.adoptium.net/view/ReproducibleBuild/job/jdk21u-mac-x64-temurin_reproduce_compare/74/consoleFull

So I'm thinking this might not be issues from test view?

The problem is occuring invoking "javap" to process the System*.class files, here:

temurin-build/tooling/reproducible/repro_common.sh

Line 104 in 13b306f

javap -v -sysinfo -l -p -c -s -constants "$ff" > "$f.javap.tmp"

The System*.class will be compiled with the given JDK internal class version, so for jdk-23 that is "67". I would guess the "javap" being picked up off PATH by the test is not jdk-23, so it will not know how to read a jdk-23(67) class file.
The reason it works for jdk21u is probably because the test is picking up a jdk-21 jdk in the PATH?

To resolve, I think we need to put a copy of the source or target JDK in the PATH, to resolve to a version of "javap" that knows how to process class files of the same internal version.

sophia-guo · 2024-09-27T20:39:26Z

For cross compiling issue discussion is here adoptium/TKG#618

sophia-guo · 2024-10-02T20:24:30Z

Enable aarch64_mac https://ci.adoptium.net/view/Test_grinder/job/Grinder/11050/console #3972

Interesting , test will archive the reproduced jdk and compare result. For other platform it's around 260M, but Mac_aarch64 is around 4.57G. Will download to see what's the difference and cause such a large aritfact. Caused by wrong workdir, updated.

sophia-guo · 2024-10-04T13:35:25Z

Update all javap with testJDK.
https://ci.adoptium.net/view/Test_grinder/job/Grinder/11058/console

Also there are 150 dylib differences, which seems removeMacOSNonComparableData() ( remove the macos dylib non-compararable data) was not called at all. Updated to call removeMacOSNonComparableData() and removeWindowsNonComparableData() before the final patchManifests step.

@andrew-m-leonard removeMacOSNonComparableData() get the error, what's the issue here?

00:18:25  Removing MacOS dylib non-comparable UUID from src_jdk/Contents/Home
00:18:45  Error occurred during initialization of VM
00:18:45  Failed setting boot class path.
00:18:45    FAILED ==> java "temurin.tools.BinRepl" --inFile "src_jdk/Contents/Home/../../Contents/Home/bin/jwebserver" --outFile "src_jdk/Contents/Home/../../Contents/Home/bin/jwebserver" --hex "8C:04:09:F5:4F:35:3A:EB:B0:C1:42:BF:D8:09:D4:D5-AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA:AA" --firstOnly

https://ci.adoptium.net/view/Test_grinder/job/Grinder/11059/console

sophia-guo · 2024-10-08T17:33:24Z

@andrew-m-leonard Build without call removeMacOSNonComparableData
https:// ci.adoptium.net/view/Test_grinder/job/Grinder/11137/console
150 differs

New failures with "${JDK_DIR}"/bin/javap -v -sysinfo -l -p -c -s -constants "$ff" > "$f.javap.tmp"
and "${JDK_DIR}"/bin/javap -v -sysinfo -l -p -c -s -constants "$ff" > "$f.javap.tmp"

./repro_common.sh: line 97: 45700 Killed: 9               "${JDK_DIR}"/bin/javap -v -sysinfo -l -p -c -s -constants "$ff" > "$f.javap.tmp"

sophia-guo · 2024-11-01T03:27:45Z

One question is if it is reproducible and comparable between cross-compiled x64 mac64 and native x64 mac. I've tried locally with xcode 16.1. Using sbom file generated by cross-compiled x64 mac64 . Got following failures, @andrew-m-leonard do you think the failure is caused by using cross-compiled x64 mac64 with aarch64 to compile on x64?

[buildjdk] Creating buildjdk/support/modules_libs/java.base/libsyslookup.dylib from 1 file(s)

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/libfreetype/static/libfreetype.a(ftdbgmem.o) has no symbols

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/libfreetype/static/libfreetype.a(ftmac.o) has no symbols

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/libfreetype/static/libfreetype.a(pngshim.o) has no symbols

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/libfreetype/static/libfreetype.a(sfwoff.o) has no symbols

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/libfreetype/static/libfreetype.a(sfwoff2.o) has no symbols

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/libfreetype/static/libfreetype.a(ttsubpix.o) has no symbols

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/libfreetype/static/libfreetype.a(woff2tags.o) has no symbols

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/libawt/static/libawt.a(debug_assert.o) has no symbols

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/libawt/static/libawt.a(debug_mem.o) has no symbols

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/libawt/static/libawt.a(debug_util.o) has no symbols

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/liblcms/static/liblcms.a(cmshalf.o) has no symbols

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/native/java.desktop/libfontmanager/static/libfontmanager.a(hb-static.o) has no symbols

[buildjdk] Creating buildjdk/support/modules_cmds/java.base/java from 1 file(s)

[buildjdk] Creating buildjdk/support/modules_cmds/java.base/keytool from 1 file(s)

[buildjdk] Creating buildjdk/support/modules_libs/java.base/jspawnhelper from 1 file(s)

[buildjdk] Creating buildjdk/support/modules_cmds/jdk.jlink/jimage from 1 file(s)

[buildjdk] Creating buildjdk/support/modules_cmds/jdk.jdeps/javap from 1 file(s)

[buildjdk] Creating buildjdk/support/modules_cmds/jdk.jlink/jlink from 1 file(s)

[buildjdk] Creating buildjdk/support/modules_cmds/jdk.jdeps/jdeps from 1 file(s)

[buildjdk] Creating buildjdk/support/modules_cmds/jdk.jlink/jmod from 1 file(s)

[buildjdk] Creating buildjdk/support/modules_cmds/jdk.jdeps/jdeprscan from 1 file(s)

[buildjdk] Creating interim java.logging.jmod

[buildjdk] Creating interim java.base.jmod

Exception in thread "main" java.lang.ClassFormatError: StackMapTable format error: bad verification type

	at jdk.jlink/jdk.tools.jmod.Main.main(Main.java:34)

make[4]: *** [/Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/interim-jmods/support/_create_java.logging.jmod_exec.marker] Error 1

make[3]: *** [java.logging-interim-jmod] Error 2

make[3]: *** Waiting for unfinished jobs....

Exception in thread "main" java.lang.ClassFormatError: StackMapTable format error: bad verification type

	at jdk.jlink/jdk.tools.jmod.Main.main(Main.java:34)

make[4]: *** [/Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/support/interim-jmods/support/_create_java.base.jmod_exec.marker] Error 1

make[3]: *** [java.base-interim-jmod] Error 2

make[2]: *** [create-buildjdk-interim-image] Error 2

make[2]: *** Waiting for unfinished jobs....



ERROR: Build failed for targets 'product-images mac-legacy-jre-bundle test-image static-libs-image' in configuration 'macosx-x86_64-server-release' (exit code 2) 



=== Output from failing command(s) repeated here ===

* For target support_interim-jmods_support__create_java.base.jmod_exec:

Exception in thread "main" java.lang.ClassFormatError: StackMapTable format error: bad verification type

	at jdk.jlink/jdk.tools.jmod.Main.main(Main.java:34)

* For target support_interim-jmods_support__create_java.logging.jmod_exec:

Exception in thread "main" java.lang.ClassFormatError: StackMapTable format error: bad verification type

	at jdk.jlink/jdk.tools.jmod.Main.main(Main.java:34)



* All command lines available in /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/make-support/failure-logs.

=== End of repeated output ===



No indication of failed target found.

HELP: Try searching the build log for '] Error'.

HELP: Run 'make doctor' to diagnose build problems.



make[1]: *** [main] Error 2

make: *** [product-images] Error 2

OpenJDK make failed, archiving make failed logs

Copying build.log to TemurinLogsArchive

Copying make-support/failure-logs to TemurinLogsArchive

Archiving and compressing with pigz



real	0m0.511s

user	0m0.005s

sys	0m0.007s

Your archive was created as /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/build/src/build/macosx-x86_64-server-release/OpenJDK.tar.gz

Moving the artifact to location /Users/sophia/DevelopTest/git/temurin-build/tooling/reproducible/comp-jdk-build/temurin-build/workspace/target//OpenJDK21U-makefailurelogs_x64_mac_hotspot_21.0.5_11.tar.gz

archive done.

Failed to make the JDK, exiting

andrew-m-leonard · 2024-11-01T09:51:19Z

One question is if it is reproducible and comparable between cross-compiled x64 mac64 and native x64 mac. I've tried locally with xcode 16.1. Using sbom file generated by cross-compiled x64 mac64 . Got following failures, @andrew-m-leonard do you think the failure is caused by using cross-compiled x64 mac64 with aarch64 to compile on x64?

@sophia-guo mmm not sure, however it maybe Xcode 16.1 issue, as we currently build using Xcode 15.2

sophia-guo · 2024-11-01T20:16:24Z

I've installed xcode 15.2 instead of 16.1. Using cross-compiled x64 mac jdk on arm64 mac SBOM https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.5%2B11/OpenJDK21U-sbom_x64_linux_hotspot_21.0.5_11.json did reproducing on my local x64 mac and did a compare. Result shows except the dylib differences ( known issue #3989) it's indentical.

Files /Users/sophia/DevelopTest/git/src_jdk/Contents/Home/lib/server/libjsig.dylib and /Users/sophia/DevelopTest/git/tar_jdk/Contents/Home/lib/server/libjsig.dylib differ
Files /Users/sophia/DevelopTest/git/src_jdk/Contents/Home/lib/server/libjvm.dylib and /Users/sophia/DevelopTest/git/tar_jdk/Contents/Home/lib/server/libjvm.dylib differ
Number of differences:      146
ReproduciblePercent = 99.86 %

So go back to initial question if it makes sense to do the reproduce a cross compiled x64 mac on aarch64 and compare with native complied jdk on x64 mac?

Another question @andrew-m-leonard would be for mac64 build from temurin ( on aarch64) reproduced comparing jdk should be built on x64 or aarch64? Or it doesn't matter there should be no difference. @sxa has the same question adoptium/infrastructure#3720 (comment)

andrew-m-leonard · 2024-11-04T10:20:32Z

i'd hope make no difference, but we'd need to check

sophia-guo · 2024-11-05T22:56:51Z

With #4025 x64 mac reproducible can be enabled on aarch64 mac, which requires mac x64 special.system only have reproducible comparing tests https://github.com/adoptium/ci-jenkins-pipelines/compare/master...sophia-guo:mac64?expand=1

sophia-guo · 2024-11-06T19:45:10Z

Temporarily enable on macx64 adoptium/ci-jenkins-pipelines#1136

sophia-guo marked this pull request as draft June 10, 2024 15:12

github-actions bot added the macos Issues that affect or relate to the MAC OS label Jun 10, 2024

sophia-guo force-pushed the mac branch 5 times, most recently from 5dd3e43 to 4e6a29f Compare June 10, 2024 19:30

sophia-guo mentioned this pull request Jun 11, 2024

update mac test agents with jq installation adoptium/infrastructure#3593

Closed

sophia-guo force-pushed the mac branch 2 times, most recently from a8cf660 to 524ffcb Compare August 16, 2024 20:49

sophia-guo self-assigned this Aug 16, 2024

sophia-guo force-pushed the mac branch 3 times, most recently from 0bc4005 to 3132741 Compare August 20, 2024 15:40

sophia-guo force-pushed the mac branch 2 times, most recently from 5a2bd7f to 2b72104 Compare September 17, 2024 00:59

sophia-guo force-pushed the mac branch from 6038ffd to a097af1 Compare September 24, 2024 20:14

smlambert mentioned this pull request Sep 27, 2024

Consider running macos testing on aarch64 adoptium/aqa-tests#2171

Closed

Enable max x64 reproducible comparing tests

05f57b6

sophia-guo force-pushed the mac branch from 05093c5 to 05f57b6 Compare November 6, 2024 16:22

sophia-guo mentioned this pull request Nov 6, 2024

Running Temurin mac x64 reproducible comparing on aarch64 mac adoptium/ci-jenkins-pipelines#1136

Merged

github-actions bot added the jenkins Issues that enhance or fix our jenkins server label Nov 6, 2024

sophia-guo requested review from smlambert and andrew-m-leonard November 6, 2024 19:46

smlambert approved these changes Nov 7, 2024

View reviewed changes

andrew-m-leonard approved these changes Nov 7, 2024

View reviewed changes

sophia-guo marked this pull request as ready for review November 7, 2024 16:07

sophia-guo merged commit 8799052 into adoptium:master Nov 7, 2024
10 checks passed

sophia-guo mentioned this pull request Dec 10, 2024

Add special.system for JDK21+ mac64 and reset variables inside the parallel job adoptium/ci-jenkins-pipelines#1160

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable reproducible comparing tests on jdk21+ macos #3844

Enable reproducible comparing tests on jdk21+ macos #3844

sophia-guo commented Jun 10, 2024 •

edited

Loading

sophia-guo commented Jun 10, 2024

sophia-guo commented Jun 18, 2024 •

edited

Loading

steelhead31 commented Jun 18, 2024

andrew-m-leonard commented Jun 19, 2024

sophia-guo commented Jun 19, 2024

sophia-guo commented Jun 19, 2024

andrew-m-leonard commented Jun 19, 2024

sophia-guo commented Jun 26, 2024 •

edited

Loading

sophia-guo commented Jun 28, 2024

sophia-guo commented Aug 16, 2024 •

edited

Loading

sophia-guo commented Aug 20, 2024 •

edited

Loading

sophia-guo commented Aug 20, 2024 •

edited

Loading

sophia-guo commented Aug 20, 2024

steelhead31 commented Aug 21, 2024

steelhead31 commented Aug 21, 2024

steelhead31 commented Aug 21, 2024 •

edited by sophia-guo

Loading

sophia-guo commented Sep 17, 2024 •

edited

Loading

sophia-guo commented Sep 17, 2024 •

edited

Loading

andrew-m-leonard commented Sep 19, 2024

sophia-guo commented Sep 24, 2024

andrew-m-leonard commented Sep 25, 2024 •

edited

Loading

sophia-guo commented Sep 27, 2024

sophia-guo commented Oct 2, 2024 •

edited

Loading

sophia-guo commented Oct 4, 2024 •

edited

Loading

sophia-guo commented Oct 8, 2024 •

edited

Loading

sophia-guo commented Nov 1, 2024

andrew-m-leonard commented Nov 1, 2024

sophia-guo commented Nov 1, 2024

andrew-m-leonard commented Nov 4, 2024

sophia-guo commented Nov 5, 2024 •

edited

Loading

sophia-guo commented Nov 6, 2024

Enable reproducible comparing tests on jdk21+ macos #3844

Enable reproducible comparing tests on jdk21+ macos #3844

Conversation

sophia-guo commented Jun 10, 2024 • edited Loading

sophia-guo commented Jun 10, 2024

sophia-guo commented Jun 18, 2024 • edited Loading

steelhead31 commented Jun 18, 2024

andrew-m-leonard commented Jun 19, 2024

sophia-guo commented Jun 19, 2024

sophia-guo commented Jun 19, 2024

andrew-m-leonard commented Jun 19, 2024

sophia-guo commented Jun 26, 2024 • edited Loading

sophia-guo commented Jun 28, 2024

sophia-guo commented Aug 16, 2024 • edited Loading

sophia-guo commented Aug 20, 2024 • edited Loading

sophia-guo commented Aug 20, 2024 • edited Loading

sophia-guo commented Aug 20, 2024

steelhead31 commented Aug 21, 2024

steelhead31 commented Aug 21, 2024

steelhead31 commented Aug 21, 2024 • edited by sophia-guo Loading

sophia-guo commented Sep 17, 2024 • edited Loading

sophia-guo commented Sep 17, 2024 • edited Loading

andrew-m-leonard commented Sep 19, 2024

sophia-guo commented Sep 24, 2024

andrew-m-leonard commented Sep 25, 2024 • edited Loading

sophia-guo commented Sep 27, 2024

sophia-guo commented Oct 2, 2024 • edited Loading

sophia-guo commented Oct 4, 2024 • edited Loading

sophia-guo commented Oct 8, 2024 • edited Loading

sophia-guo commented Nov 1, 2024

andrew-m-leonard commented Nov 1, 2024

sophia-guo commented Nov 1, 2024

andrew-m-leonard commented Nov 4, 2024

sophia-guo commented Nov 5, 2024 • edited Loading

sophia-guo commented Nov 6, 2024

sophia-guo commented Jun 10, 2024 •

edited

Loading

sophia-guo commented Jun 18, 2024 •

edited

Loading

sophia-guo commented Jun 26, 2024 •

edited

Loading

sophia-guo commented Aug 16, 2024 •

edited

Loading

sophia-guo commented Aug 20, 2024 •

edited

Loading

sophia-guo commented Aug 20, 2024 •

edited

Loading

steelhead31 commented Aug 21, 2024 •

edited by sophia-guo

Loading

sophia-guo commented Sep 17, 2024 •

edited

Loading

sophia-guo commented Sep 17, 2024 •

edited

Loading

andrew-m-leonard commented Sep 25, 2024 •

edited

Loading

sophia-guo commented Oct 2, 2024 •

edited

Loading

sophia-guo commented Oct 4, 2024 •

edited

Loading

sophia-guo commented Oct 8, 2024 •

edited

Loading

sophia-guo commented Nov 5, 2024 •

edited

Loading