Requesting log of the AIX 11.0.19+7 JVM build

[twaldrep]

Are the build logs for a specific Temurin release saved anywhere? I'd like to try to get a full list of xlclang++ compiler options and linker options that are used when building the jvm (specifically the 11.0.19+7 AIX build).

Hopefully I can spot an incompatibility between our compiler/linker options and the JVM's compiler/linker options before submitting a bug.

We have a problem on AIX 7.2 where internal exceptions that get thrown in a native library loaded by the JVM (or even indirectly loaded by the library that the JVM loads) immediately cause a SIGABRT, but the same library successfully throws the exceptions when loaded by our libraries. The exceptions are even successfully thrown if we reference the library in LDR_PRELOAD64 before starting the java application.

Java version: 11.0.19+7

Your operating system and platform: AIX 7.2 TL5 (7200-05-02-2114) on Power9

XL C++ compiler version 16.1.0.10
XL C++ Runtime version 16.1.0.7

[twaldrep]

If a full log is not available, then complete listings of compile options for each object file and linker options used for libjvm.so would suffice.

[karianna]

Have pinged our AIX v-team on slack

[andrew-m-leonard]

@twaldrep I've started an AIX build with BUILD_ARGS: --make-args LOG=cmdlines, it might show what you're looking for:
https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-aix-ppc64-temurin/252/console

[sxa]

In addition to the log, the SBOM files that we produce for the builds may contain relevant information:

• https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.19%2B7/OpenJDK11U-sbom_ppc64_aix_hotspot_11.0.19_7.json
• https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.19%2B7/OpenJDK11U-sbom_ppc64_aix_hotspot_11.0.19_7-metadata.json

For reference we are also building on AIX 7.2 and with xlc 16.01.0000.0011

[twaldrep]

Thanks for the responses, but none of them show the actual xlclang++ command lines executing. All are higher level. I see in the console output that the Makefile was generated. Is there a way to see the output of that or see the actual Makefile itself?

Also, thanks for providing the XLC and AIX versions. Do you know which XL C++ Runtime that you're using? 16.1.0.11 requires (at least) XL C++ Runtime 16.1.0.7. I've tried several combinations.

[andrew-m-leonard]

Thanks for the responses, but none of them show the actual xlclang++ command lines executing. All are higher level. I see in the console output that the Makefile was generated. Is there a way to see the output of that or see the actual Makefile itself?

Also, thanks for providing the XLC and AIX versions. Do you know which XL C++ Runtime that you're using? 16.1.0.11 requires (at least) XL C++ Runtime 16.1.0.7. I've tried several combinations.

@twaldrep Isn't this what you're after?

09:29:14  Exececuting: [cd /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-aix-ppc64-temurin/workspace/build/src && /opt/IBM/xlC/16.1.0/bin/xlclang++ -qmakedep=gcc -MF ./build/aix-ppc64-normal-server-release/hotspot/variant-server/libjvm/objs/c1_Defs.d.tmp -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D_REENTRANT -qtbtable=full -qtune=balanced -qalias=noansi -qstrict -qtls=default -qnortti -qnoeh -qignerrno -qstackprotect -DLIBC=default -DSUPPORTS_CLOCK_MONOTONIC -DAIX -qpic=large -qarch=ppc64 -q64 -DPRODUCT -DTARGET_ARCH_ppc -DINCLUDE_SUFFIX_OS=_aix -DINCLUDE_SUFFIX_CPU=_ppc -DINCLUDE_SUFFIX_COMPILER=_xlc -DTARGET_COMPILER_xlc -DPPC64 -DHOTSPOT_LIB_ARCH='"ppc64"' -DCOMPILER1 -DCOMPILER2 -DINCLUDE_JVMCI=0 -DINCLUDE_CDS=0 -DINCLUDE_AOT=0 -DINCLUDE_ZGC=0 -DINCLUDE_SHENANDOAHGC=0 -DINCLUDE_JFR=0 -I ./build/aix-ppc64-normal-server-release/hotspot/variant-server/gensrc/adfiles -I ./src/hotspot/share -I ./src/hotspot/os/aix -I ./src/hotspot/os/posix -I ./src/hotspot/cpu/ppc -I ./src/hotspot/os_cpu/aix_ppc -I ./build/aix-ppc64-normal-server-release/hotspot/variant-server/gensrc -I ./src/hotspot/share/precompiled -I ./src/hotspot/share/include -I ./src/hotspot/os/posix/include -I ./build/aix-ppc64-normal-server-release/support/modules_include/java.base -I ./build/aix-ppc64-normal-server-release/support/modules_include/java.base/aix -I ./src/java.base/share/native/libjimage -q64 -DDONT_USE_PRECOMPILED_HEADER -g1 -qsuppress=1540-0216 -qsuppress=1540-0198 -qsuppress=1540-1090 -qsuppress=1540-1639 -qsuppress=1540-1088 -qsuppress=1500-010 -O3 -qhot=level=1 -qinline -qinlglue -c -o ./build/aix-ppc64-normal-server-release/hotspot/variant-server/libjvm/objs/c1_Defs.o ./src/hotspot/share/c1/c1_Defs.cpp]

They're in the console output: https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk11u/job/jdk11u-aix-ppc64-temurin/252/consoleFull

[twaldrep]

Thanks. I didn't see that in the console output.

Looking at the command line arguments, something immediately jumps out at me. Why "-qnortti -qnoeh"? That may be messing with the internal exceptions that our C++ library throws when loaded by the JVM to call native methods. The SIGABRT that we see when attempting to throw an exception is what would be expected if we built our library with -qnortti.

Like I had stated in the original question, we are able to throw exceptions in our native library when called by our own launcher without Java involved. We are also able to throw exceptions in our native library when used by Java, but we have the operating system preload the library before execution, like "LDR_PRELOAD64=/path/to/libourlibrary.so java ...".

[andrew-m-leonard]

@twaldrep those options are specified here: https://github.com/adoptium/jdk11u/blob/d286dded5f7714d54fe57b999a5553653d45218b/make/autoconf/flags-cflags.m4#L539
@aixtools Michael, can you shed some light on the compile options used here for AIX ?

[sxa]

Also cc @backwaterred ref the inclusion of -qnortti -qnoeh flag in the openjdk codebase.

[backwaterred]

It looks like they have been in the project for some time, but I'm not certain of the exact date (so I'm not able to see an issue that comments on their addition). I sent an email to the porter's list to see if anyone remembers why they were added. Here's an excerpt of that email with my guess for the reason:

Reading the descriptions for the flags makes me suspect that these were added to improve performance "For improved runtime performance, suppress RTTI information generation with the -qnortti setting." [2] and possibly to reduce code bloat "If your program does not use C++ structured exception handling, you can compile with -qnoeh to prevent generation of code that is not needed by your application."

If you'd like to track that discussion, replies should show up here

[twaldrep]

Did anyone test how these options affect JNI libraries loaded by the JVM? This is potentially a pretty serious issue.

We've been using JNI on multiple platforms (including AIX) for a long time (20 years). We were compiling our libraries using gcc/g++ in previous versions, and we never saw this issue (unfortunately we had several other stability issues which prompted us abandon gcc/g++ on AIX for our upcoming release). After porting our application to the XL C++ compiler 16.1.0.10 (xlclang/xlclang++, specifically), there doesn't appear to be any compiler or linker option that we can set which overrides the SIGABRT that we get when attempting to throw exceptions in our native libraries.

Like I mentioned in earlier comments, the SIGABRT when attempting to throw exceptions does not occur if we use LDR_PRELOAD64 to preload our initial library before java execution. It only happens when the JVM loads our library (I'm assuming with dlopen).

I'm going to write a small test program (basically just a hello world) tomorrow that will be linked as a shared library (*.so). It will includes a native method for loading via Java and a separate C++ main program that loads it. I'll be compiling the main program with -qnortti -qnoeh and then recompiling with -qrtti -qeh in order to test the theory that the loaded library is being affected by the loading program.

[aixtools]

@andrew-m-leonard : re IBM compiler flags. While I could do a study of them, I was mainly active using xlc-v11 and never using C++ or the later xlclang++ interfaces. Sorry I am not an expert is this area ;)

[andrew-m-leonard]

Did anyone test how these options affect JNI libraries loaded by the JVM? This is potentially a pretty serious issue.

We've been using JNI on multiple platforms (including AIX) for a long time (20 years). We were compiling our libraries using gcc/g++ in previous versions, and we never saw this issue (unfortunately we had several other stability issues which prompted us abandon gcc/g++ on AIX for our upcoming release). After porting our application to the XL C++ compiler 16.1.0.10 (xlclang/xlclang++, specifically), there doesn't appear to be any compiler or linker option that we can set which overrides the SIGABRT that we get when attempting to throw exceptions in our native libraries.

Like I mentioned in earlier comments, the SIGABRT when attempting to throw exceptions does not occur if we use LDR_PRELOAD64 to preload our initial library before java execution. It only happens when the JVM loads our library (I'm assuming with dlopen).

I'm going to write a small test program (basically just a hello world) tomorrow that will be linked as a shared library (*.so). It will includes a native method for loading via Java and a separate C++ main program that loads it. I'll be compiling the main program with -qnortti -qnoeh and then recompiling with -qrtti -qeh in order to test the theory that the loaded library is being affected by the loading program.

Sounds a good plan

[backwaterred]

Did anyone test how these options affect JNI libraries loaded by the JVM? This is potentially a pretty serious issue.

It sounds like these have been part of the build since before jdk8, but I'm not sure if anyone has done this sort of testing before.

We were compiling our libraries using gcc/g++ in previous versions, and we never saw this issue.

Does Open XL Clang 17.1 have the options you are looking for? There is an ongoing effort to move to that version in order to support an increased C++ language level. I expect to see support for that compiler early in the jdk21 lifetime.

[twaldrep]

I created a small program which mimics the general behavior of our libraries, i.e....

1. Execute java class with method defined as native
2. JNI native method in C++ library
3. Native method calls an extern "C" method in another C++ library
4. Extern "C" method creates instance of C++ class and calls a method on it
5. Internal instance method calls a static method which throws an exception
6. Exception should be caught by C++ instance method.

This scenario works without fail in the above small test program.

In the above scenario, a SIGABRT occurs in our actual application when the attempt is made to throw the exception when the bottom of the call stack is the JVM. The exception is thrown and caught without issue if we use LDR_PRELOAD64 to preload our JNI library.

The same compiler and linker options are used to build both the small test program and our actual application.

I can only conclude that there must be another factor that is escaping me. We're running AIX 7.2 TL5, with 16.1.0.7 XL C/C++ runtime libraries and 16.1.0.10 XL C/C++ compilers.

@backwaterred XL Clang 17.1 may end up being where we need to be, but leaping from JDK11 to JDK21+ is a massive jump for our very conservative AIX customers. They seriously resisted the move away from JDK8.

[twaldrep]

Sorry for no response in a while, but I'm buried in last minute work of an imminent release. I wrote my little "Hello World" test code and emulated our architecture as best as I could. I used the same compiler and linker arguments as our current code. I reproduced the extern "C" calls across 2 separate shared libraries to reproduce the exact scenario where the exceptions result in SIGABRT in our current code. I could NOT reproduce the SIGABRT when called from Java through a native method using the latest Adoptium JDK 11. An additional ingredient that I'm not seeing must also affecting the outcome.

@backwaterred Our AIX customers tend to be rather cautious when moving to newer versions of software. Are there any chances of moving to XL Clang 17.1 on one of the current JDK LTS tracks (like JDK 17)? We're on Java 11, and it took a LOT to get our AIX customers off of Java 8.

[backwaterred]

@twaldrep, no worries. I understand that clients are generally resistant to bumping versions. There is risk for them, but generally not much to gain. Once these changes land in the current version, I can look at moving it back to previous versions. jdk17 is usually pretty feasible.

[twaldrep]

@backwaterred Yes, JDK17 would be great if possible. Should I assume that will be on the roadmap or is there a formal request that needs to be made?

[backwaterred]

Hi @twaldrep, no extra steps needed. I've made a note to backport any changes that enable XLClang 17.1 to jdk17. That's probably something we wanted to do anyway, and if we know there is someone out there who would benefit then all the more reason to do so :-).

[twaldrep]

@backwaterred That's really great to hear! Thanks. We just dropped a new released of our software in July and anticipate moving to XLCLang 17.1 (on AIX) during the development cycle leading to our next release (which will be around the 2nd quarter of 2024). I hate to ask about dates (primarily because I hate it when people ask me), but do you anticipate your move to XLCLang 17.1 (and subsequent backport to JDK17) during this timeframe?

[backwaterred]

No worries @twaldrep.

Looking into this a bit more--yes I expect we can have that in jdk17 by that time or sooner. The changes will actually be available in the Java release that will GA in Sept 2023, at which point we just need to 'battle test' them before backporting to jdk17. If it's an option to you, it may be a good idea to build that release internally and do some proof of concept work to see if the changes as they are will meet your use case. Feel free to reach out if you encounter any issues. Otherwise, we will continue building and testing them internally and move them back if they stand the test of time.

[twaldrep]

@backwaterred Thanks for the update. Based on our current set of priorities, it sounds like the JDK 21.x release will GA before we get a chance to upgrade our AIX builds XL C++ 17.1. I can internally test with JDK 21.x to see if that resolves our exception issues. If that resolves it, it will just be a matter of saving off the XL C++ 17.1 upgrades until JDK 17 is ready.

Once again, thanks for your help on this issue.

[twaldrep]

@backwaterred Since Oracle changed the TCK licensing agreement, I expect that backporting JDK 17 on AIX to use Open XL C++ 17.1 is negatively impacted. Is that a correct assumption?

[sxa]

I'm not sure why the new TCK version would affect a backport - am I missing something?

[twaldrep]

@sxa Sorry, some context would have helped. Tyler Steele said in a previous comment (Aug 9th) that JDK 21 that will GA in September 23 will be built using Open XL C++ 17.1.1 on AIX. JDK 21 will need to be 'battle tested' in the field before JDK 17 is backported to build with Open XL C++ 17.1.1.

I'm assuming the the TCK licensing delays will push back the 'battle test' in the field phase, thus pushing back the subsequent backport of JDK 17 ... or am I reading way too much into the situation?

[sxa]

Thanks for the clarification. I can confirm that we have not changed the compiler for Temurin JDK21. It is still being built with xlc16. I believe there are still discussions ongoing (not at Temurin specifically) related to the compiler upgrade which have not yet been finalised.

At present the JDK22 development builds are also built with xlc16 (See the logs at https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk/job/jdk-aix-ppc64-temurin/ - we do not have xlc17 installed anywhere on our infrastructure yet) but that would likely be the first to change if it is agreed upstream.

[sxa]

Now that I'm back at my computer I can give some links: openjdk/jdk#14988 is a related upsttream change related to switching to C++17 and it seems unlikely that any such change would be required in the existing LTS releases. While it can be done (and we bumped Temurin JDK11 from xlc 13 to 16) we would generally avoid switching the compiler during a release line, especially since on platforms such as AIX it would introduce new dependencies on the runtime which could cause problems with upgrades.

I'll also note that XLC17 is supported but not required for JDK21

[twaldrep]

@sxa This is pretty upsetting news. See the series of messages between @backwaterred and myself in this issue. Based on the discussion, we purchased IBM Open XL C++ and configured multiple AIX VMs with the compiler installed. We were about to start a project to upgrade from XL C++ 16.1.0 to Open XL C++ 17.1.1 in preparation for testing with JDK 21. We were anticipating that JDK 17 would be backported down the road to build with Open XL C++ 17.1.1 before the release of our next version near the summer of 2024.

Thanks for the links to issue 14988 and the compiler support table for JDK21. I don't see how Open XL C++ 17.1.1 can be called "supported" if it expects xlclang++ to be in the toolchain. Open XL C++ 17.1.1 has a completely different toolchain than XL C++ 16.1.0. Also, the compile options are significantly different. Note that the topic of Open XL C++ 17.1.1 presented by @backwaterred is attractive to us for more than just C++17 support. Our application depends heavily on JNI (both native methods and the JNI Invocation Interface). We've found many issues with XL C++ 16.1.0 related to JNI which affects both the Adoptium Temurin 11.x and IBM Semeru 11.x releases that we've had to work around. We were hoping that Open XL C++ 17.1.1 would resolve at least some of the issues.

[sxa]

@sxa This is pretty upsetting news. See the series of messages between @backwaterred and myself in this issue.

Assuming you're referring to:

The changes will actually be available in the Java release that will GA in Sept 2023, at which point we just need to 'battle test' them before backporting to jdk17. If it's an option to you, it may be a good idea to build that release internally

I believe that was referring to the upstream openjdk code changes allow it to build with XLC 17 as an option, not that Temurin would be taking advantage of that option.

We've found many issues with XL C++ 16.1.0 related to JNI which affects both the Adoptium Temurin 11.x and IBM Semeru 11.x releases that we've had to work around.

Interesting. That's useful input - thanks. As always we will take advice from IBM on such matters. As per the above comment do you have the ability to try and build a copy of openjdk yourself using XLC17 to verify if that does resolve your issues?

[backwaterred]

The changes will actually be available in the Java release that will GA in Sept 2023, at which point we just need to 'battle test' them before backporting to jdk17. If it's an option to you, it may be a good idea to build that release internally

I believe that was referring to the upstream openjdk code changes allow it to build with XLC 17 as an option, not that Temurin would be taking advantage of that option.

Sorry for the confusion caused @twaldrep. @sxa is right that I meant available to be built, not built by Adoptium. I should have realized that responding on the Adoptium GitHub would add some additional context to my response.

[backwaterred]

@twaldrep, I've confirmed that SAP would like to release their 'friendly fork' of OpenJDK called SAPMachine on AIX, and build it with Open XL 17.1. That project is currently under way.

[twaldrep]

@sxa @backwaterred I've been out of band for the past several days working on a customer issue. I just cloned the jdk21 repository and started to resolve missing tool dependencies required by the configure utility. I got as far as installing the boot jdk (had to grab the last available daily JDK 21 build for AIX which was built in August). The configure utility discovered Open XL C++ 17.1.1 (and correctly set ibm-clang++ as the compiler), but it has failed on the following:

checking for c++filt... configure: error[not found]
: Could not find required tool for CXXFILT
configure exiting with result code 1

The IBM documentation for JDK 17.1.1 states the following:

ibm-llvm-cxxfilt
The ibm-llvm-cxxfilt utility is a symbol demangler that decodes mangled names generated by IBM Open XL C/C++. It replaces the c++filt name demangling utility that was shipped in IBM XL C/C++ for AIX® 16.1.0 and earlier releases.

This would seem to be a dealbreaker for building with Open XL C++ 17.1.1. What is the workaround for this? Manually hacking the configure scripts?

[twaldrep]

@sxa @backwaterred Following up on my prior message, it appears that the fix for the "ibm-llvm-cxxfilt vs c++filt issue" was originally committed to jdk repository, and then backported to to jdk21u repository in August. I just finished looking at the pull request history for it. The fix isn't in the jdk21 repository (at least yet).

[sxa]

Nothing will go back to jdk21 ... All future releases of 21 (including 21.0.1 which will appear any time now) will come from jdk21u. That's standard practice for openjdk. The first one is fine from the non-u version.

[twaldrep]

@sxa Thanks for the additional info about the openjdk repository conventions. I have another question. I've now gotten through what I believe to be most of the configure script and have now run into new missing dependencies related to X11 extensions. The error follows:

checking for X11/extensions/shape.h... configure: error: Could not find all X11 headers (shape.h Xrender.h Xrandr.h XTest.h Intrinsic.h). You might be able to fix this by running 'sudo yum install libXtst-devel libXno
t-devel libXrender-devel libXrandr-devel libXi-devel'.
configure exiting with result code 1

Looking at the ansible playbook role that you referenced, it appears that it explicitly removes libXrender-devel, libXrender, libX, libXft, renderproto and the /opt/freeware/include/X11/extensions path. If these are explicitly removed on your build machines, how are these dependencies resolved on your AIX build machines? Are these extensions directly installed from the AIX DVD images or somewhere else?

[sxa]

Hmmm I'll check our machines tomorrow to be sure but It's have thought that the X11 development stuff should come from the AIX CDs and not from the yum toolbox

[twaldrep]

@sxa That may very well be the case. It's possible that our IT engineer didn't include that when setting up our AIX 7.2 VMs. I'd also be curious if the JDK build would be happy with either (original AIX DVD image or the AIX open source toolbox) as long as it can access the headers.

[sxa]

This dropped down my list abit today as we're working on a new release, but yes ours is comeing from the bos filesets:

root@p8-aix1-adopt03:[/root]find /usr /opt -name \*Xrender.h
/usr/lpp/X11/include/X11/extensions/Xrender.h
root@p8-aix1-adopt03:[/root]lslpp -w /usr/lpp/X11/include/X11/extensions/Xrender.h
  File                                        Fileset               Type
  ----------------------------------------------------------------------------
  /usr/lpp/X11/include/X11/extensions/Xrender.h
                                                 X11.adt.ext        File

So you'll want at least X11.adt.ext and X11.adt.include for some of the others like X11/Intrinstic.h too.

[twaldrep]

@sxa Thanks. X11.adt.include is already on the machine, so Intrinsic.h is there. I'm not sure why the error output included that file by name. I have to get our IT tech to install X11.adt.ext.

[twaldrep]

@sxa After our IT engineer resolved the X11.adt.ext missing dependency (X11.adt.include was already there), I was able to fully build the JDK after I resolved an issue with date. As it turns out, the JDK build really wants the GNU version of date, not the AIX version. Below is the result of my build using IBM Open XL C++ 17.1.1.

-bash-5.2$ ./build/aix-ppc64-server-release/jdk/bin/java -version
openjdk version "21.0.1-internal" 2023-10-17
OpenJDK Runtime Environment (build 21.0.1-internal-adhoc.twaldrep.jdk21u)
OpenJDK 64-Bit Server VM (build 21.0.1-internal-adhoc.twaldrep.jdk21u, mixed mode)

"gmake run-test-tier1" failed since I don't have jtreg built on this VM.

@backwaterred Do you think backporting support for building JDK 17 with IBM Open XL 17.1.1 is still planned?

[sxa]

Nice. And good to know that the xlc17 support works! How many extra packages did you end up installing from the AIX toolbox to be able to compete the build? Might be useful for anyone else who tries it :-)

[varada1110]

@sxa After our IT engineer resolved the X11.adt.ext missing dependency (X11.adt.include was already there), I was able to fully build the JDK after I resolved an issue with date. As it turns out, the JDK build really wants the GNU version of date, not the AIX version. Below is the result of my build using IBM Open XL C++ 17.1.1.

-bash-5.2$ ./build/aix-ppc64-server-release/jdk/bin/java -version openjdk version "21.0.1-internal" 2023-10-17 OpenJDK Runtime Environment (build 21.0.1-internal-adhoc.twaldrep.jdk21u) OpenJDK 64-Bit Server VM (build 21.0.1-internal-adhoc.twaldrep.jdk21u, mixed mode)

"gmake run-test-tier1" failed since I don't have jtreg built on this VM.

@backwaterred Do you think backporting support for building JDK 17 with IBM Open XL 17.1.1 is still planned?

Hi @twaldrep, I am working on the backport.

[twaldrep]

@sxa In addition to X11.adt.ext from the CD and soft-linking date into a directory in the front of my path, I had to add the following from the AIX Toolbox for Open Source Software:

• xz-libs
• libffi-devel
• fontconfig-devel
• freetype2
• freetype2-devel
• cups-devel
• unzip
• zip
• coreutils
• autoconf
• grep

There were a couple of cases where a different version of the utility was already on the box, but the JDK build expected arguments that weren't supported. So, I installed the GNU version from the toolbox.

[twaldrep]

Hi @twaldrep, I am working on the backport.

That's really great news! Thanks. Please let me know when you think the backport is stable enough for someone else to try.

[TheRealMDoerr]

@twaldrep, I've confirmed that SAP would like to release their 'friendly fork' of OpenJDK called SAPMachine on AIX, and build it with Open XL 17.1. That project is currently under way.

We have already released a beta: https://github.com/SAP/SapMachine/releases

[twaldrep]

@TheRealMDoerr That's great to hear. Question: I see that the beta is for JDK22. Does SAP plan to attempt to build JDK 17 with IBM Open XL C++ 17.1 compiler once the openjdk build has been backported to support it (according to @varada1110, that project currently in progress)?

[TheRealMDoerr]

JDK21 is also available as beta. Just scroll a bit down. All SapMachine releases for AIX are built with IBM Open XL C++ 17.1.1. We're not planning to release anything older than JDK21 for AIX.

[twaldrep]

@TheRealMDoerr My manager asked the following question: "Will SAP be releasing a production (non-beta) JDK21 built with IBM Open XL C++ 17.1.1 on AIX?"

[TheRealMDoerr]

Not very soon, but we will consider doing that in the future. Note that there are already discussions with IBMers about this topic. It's too early to announce it publicly, yet.

[sxa]

Ultimately we're waiting on a full implementation of the new foreign memory interface on AIX before Temurin, or other java compatible providers, can release a final version of JDK21 on the platform.

[TheRealMDoerr]

The FFI is no longer required to certify JDK21. The related tests have been excluded.

[sxa]

Thanks @TheRealMDoerr !

[twaldrep]

Not very soon, but we will consider doing that in the future. Note that there are already discussions with IBMers about this topic. It's too early to announce it publicly, yet.

Thanks. I'll pass that along.

[TheRealMDoerr]

SapMachine 21.0.2 for AIX has been released. Built with OpenXL 17.1.1.4.
https://sap.github.io/SapMachine/

[twaldrep]

@TheRealMDoerr Thanks. We have already downloaded it and started testing.

Unfortunately, this doesn't fix the crash issue that we are experiencing on AIX when using the JNI Invocation API. All HotSpot-based distributions that we've tested since 11.0.12 (might have happened earlier) crash very early in the JVM initialization on AIX. The same method that we've been using since around 2003 for loading the JVM still works without fail on our other supported platforms (WIndows and Red Hat-compatible Linux distributions). This includes JDK 21 distributions. As a result of this, we're currently having to embed IBM's OpenJ9-based Semeru JRE in our product distribution, but we recommend the HotSpot-based Adoptium builds to our customers who embed our product in their Java applications (which don't need the JNI Invocation API).

I've created a separate issue specifically for the crash. #997

[sxa]

Temurin 21.0.2+13 for AIX is now available from the website or the API at https://api.adoptium.net/v3/binary/latest/21/ga/aix/ppc64/jdk/hotspot/normal/eclipse

As per my earlier comment this is still built with XLC16.

On the basis that there is now a separate issue dedicated to the crash which is the primary concern relating to this issue I suggest we close this one now.

[twaldrep]

@sxa The original reason for this thread was for the SIGABRT which was being produced on AIX when attempting to throw an exception from our native library which was loaded by Java. This was only happening in the hotspot-based distributions. The same was not happening in the equivalent version of OpenJ9 distributions. Regardless of compiler flags used, we've determined that this doesn't happen when our code is built with Open XL C++ 17.1 (even if the JRE is built with XL C++ 16.1). I personally think that the better behavior is due to the updated Open XL C++ runtime which is required by our Open XL C++ 17.1-built code.

Since we're building our next release with the IBM Open XL C++ 17.1 compiler, we're actively attempting to embed the latest SapMachine 21.0.x AIX distribution (which is also built with Open XL C++ 17.1). Unfortunately that didn't resolve a separate crash (which happens with all hotspot-based distributions starting with JDK 11) when attempting to use the JNI Invocation API. A workaround has been provided for the crash in the separate Adoptium support bug #997. Basically, create the non-primordial thread that is used to load the JVM using pthreads instead of std::thread so that we can explicitly set a sufficient stack size. We've tested the workaround, and it works.

So... yes, I agree that this request for log issue has been resolved for us through multiple upgrades (JRE version, C++ compiler version, compiler options, etc.) and change of JRE distribution to SapMachine 21.0.x. It can be closed while the separate bug #997 is being processed.