Update to openssl 3.0.10 #14900

pshipton · 2022-04-11T17:16:09Z

Update the notices file with the new license for v3. Updated the build instructions.

https://www.openssl.org/source/apache-license-2.0.txt

pshipton · 2022-04-11T17:16:33Z

jenkins compile amac jdk11 depends ibmruntimes/openj9-openjdk-jdk11#500

pshipton · 2022-04-11T23:55:54Z

jenkins compile zlinux,win,xmac,aix jdk11 depends ibmruntimes/openj9-openjdk-jdk11#500

pshipton · 2022-04-12T03:11:37Z

It doesn't build on Windows

21:08:51  	"C:/cygwin64/bin/perl.exe" util/mkbuildinf.pl ""cl" /Zi /Fdossl_static.pdb /Gs0 /GF /Gy /MD /W3 /wd4090 /nologo /O2 -D"L_ENDIAN" -D"OPENSSL_PIC"" "VC-WIN64A" > crypto/buildinf.h
21:08:51  Can't open perl script "util": Is a directory

pshipton · 2022-04-12T13:11:36Z

My rebuild for amac is https://openj9-jenkins.osuosl.org/job/Build_JDK11_aarch64_mac_Personal/57
Strangely it got the following.

18:58:12  WARNING: /Users/jenkins/workspace/Build_JDK11_aarch64_mac_Personal/build/macosx-aarch64-normal-server-release/jdk/bin/jmod is loading libcrypto in an unsafe way
18:58:12  JVMDUMP039I Processing dump event "abort", detail "" at 2022/04/12 08:58:12 - please wait.

The build doesn't yet support loading libcrypto.3.dylib (as ibmruntimes/openj9-openjdk-jdk11#499 is not yet merged) so it should be trying to load the old library libcrypto.1.1.dylib, which doesn't exist in the build. Perhaps it found some other library on the machine which is causing this error. I don't think this matters for the changes themselves. openssl3 was built, and copied into the JVM build. If it wasn't copied in, there would have been an error as seen in https://openj9-jenkins.osuosl.org/job/Build_JDK11_aarch64_mac_Personal/54

13:36:18  /Users/jenkins/workspace/Build_JDK11_aarch64_mac_Personal/closed/custom/copy/Copy-java.base.gmk:206: *** Cannot bundle OpenSSL - none of libcrypto.1.1.dylib libcrypto.1.0.0.dylib are present in /Users/jenkins/workspace/Build_JDK11_aarch64_mac_Personal/openssl.  Stop.

pshipton · 2022-04-12T13:13:46Z

@keithc-ca any ideas for fixing #14900 (comment) ?

keithc-ca · 2022-04-12T15:49:10Z

@keithc-ca any ideas for fixing #14900 (comment) ?

I'll have a look.

pshipton · 2022-04-12T16:26:47Z

Windows is the only platform that uses a different compiler on jdk8 vs jdk11.

jenkins compile win,win32 jdk8 depends ibmruntimes/openj9-openjdk-jdk8#559

pshipton · 2022-04-12T16:34:49Z

Windows and AIX are the only platforms that use a different compiler on jdk17 vs jdk11.

jenkins compile win,aix jdk17 depends ibmruntimes/openj9-openjdk-jdk17#88

pshipton · 2022-04-12T17:01:50Z

All the Windows versions have the same basic problem #14900 (comment). If we can fix that, we'll be able to see if anything else is uncovered when using the different compiler versions.

keithc-ca · 2022-04-12T19:36:50Z

Perhaps some different 'special setup' is required for openssl 3 on Windows; see https://github.com/ibmruntimes/openj9-openjdk-jdk11/blob/openj9/closed/openssl.gmk#L30.

pshipton · 2022-04-12T20:13:49Z

Maybe we need a different version of perl.

pshipton · 2022-04-12T20:33:47Z

perl seems fine, it's the cygwin perl. It doesn't have a problem with forward slashes, even when I call it via nmake.

This makefile works

all:
	perl util/test.pl

and so does

all:
	C:\cygwin64\bin\perl.exe util/test.pl

pshipton · 2022-04-12T20:36:40Z

The following makefile doesn't appear to do anything, I suspect it recognized C: and then stops.

all:
	C:/cygwin64/bin/perl.exe util/test.pl

pshipton · 2022-04-12T20:38:08Z

Running c:/cygwin64/bin/perl.exe util/test.pl from cygwin bash works fine.

keithc-ca · 2022-04-12T23:06:43Z

In my local build, the command echoed by nmake is:

"C:/cygwin/bin/perl.exe" util/mkbuildinf.pl ""cl" /Zi /Fdossl_static.pdb /Gs0 /GF /Gy /MD /W3 /wd4090 /nologo /O2 -D"L_ENDIAN" -D"OPENSSL_PIC"" "VC-WIN64A" > crypto/buildinf.h

Notice numerous nested or unbalanced double-quotes. One avenue I think worth exploring is telling nmake to use a shell other than cmd.

pshipton · 2022-04-12T23:29:57Z

That looks very similar to the command in #14900 (comment) except for the cygwin / cygwin64 directory.

Did you also get Can't open perl script "util": Is a directory?

If I have a util/mkbuildinf.pl file, and run that command from bash or cmd, it "works" in that it runs my util/mkbuildinf.pl.

keithc-ca · 2022-04-13T12:52:06Z

Yes.

Can't open perl script "util": Is a directory

keithc-ca · 2022-04-13T15:36:39Z

For the record, other uses of perl fail similarly when invoked by nmake:

C:\space\openssl>nmake libcrypto.rc

Microsoft (R) Program Maintenance Utility Version 14.16.27042.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        "C:/cygwin/bin/perl.exe" util/mkrc.pl libcrypto > libcrypto.rc
Can't open perl script "util": Is a directory
NMAKE : fatal error U1077: '"C:/cygwin/bin/perl.exe"' : return code '0xff'
Stop.

Removing the double quotes fixes it.

https://github.com/openssl/openssl/blob/master/NOTES-PERL.md#perl-on-windows recommends Strawberry Perl; I'll give that a spin to see if it helps.

keithc-ca · 2022-04-13T15:48:13Z

No dice:

C:\space\openssl>nmake libcrypto.rc

Microsoft (R) Program Maintenance Utility Version 14.16.27042.0
Copyright (C) Microsoft Corporation.  All rights reserved.

        "C:\Strawberry\perl\bin\perl.exe" util/mkrc.pl libcrypto > libcrypto.rc
Can't open perl script "util": Permission denied
NMAKE : fatal error U1077: '"C:\Strawberry\perl\bin\perl.exe"' : return code '0xd'
Stop.

keithc-ca · 2022-04-13T16:11:14Z

It's behaving as if nmake is splitting util/mkrc.pl so perl sees
I'm going to see if using the configuration Cgwin-x86_64 fairs better.

keithc-ca · 2022-04-13T17:26:08Z

Using the Cgwin-x86_64 configuration succeded with jdk11 and openssl 3.0.2.
We'll need to verify that this strategy also works for openssl 1.x.

keithc-ca · 2022-04-14T20:43:12Z

These changes should enable updating openssl to 3+

Update OpenSSL build script to support OpenSSL 3+ ibmruntimes/openj9-openjdk-jdk8#561
Update OpenSSL build script to support OpenSSL 3+ ibmruntimes/openj9-openjdk-jdk11#504
Update OpenSSL build script to support OpenSSL 3+ ibmruntimes/openj9-openjdk-jdk17#89
Update OpenSSL build script to support OpenSSL 3+ ibmruntimes/openj9-openjdk-jdk18#41
Update OpenSSL build script to support OpenSSL 3+ ibmruntimes/openj9-openjdk-jdk#425

keithc-ca · 2022-04-21T17:24:16Z

As the description says, this depends on ibmruntimes/openj9-openjdk-jdk11#500, but we'll also need similar changes for other versions, and we'll need all versions to support using openssl 3.x on all platforms (not just Linux) before it would be safe to merge this.

pshipton · 2022-04-22T00:00:04Z

Also, openssl 3 doesn't build on some of the linux platforms, there is an (internal) issue open to update all the machines with the required perl modules. Then we can see if there is another problem.

keithc-ca · 2022-04-22T14:31:03Z

My point is that we can't change --openssl-version globally for all platforms until the other platforms are ready.

keithc-ca · 2022-05-03T15:11:32Z

A new tag for openssl-3.0.3 appeared today.

paulcheeseman · 2023-07-19T08:10:28Z

@pshipton

We need to ensure that the fix for CVE-2023-2975 is included:

https://www.openssl.org/news/secadv/20230714.txt

OpenSSL have not yet shipped a release containing this fix due to the low severity, but the patch is available:

https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=00e2f5eea29994d19293ec4e8c8775ba73678598

keithc-ca · 2023-07-20T18:20:31Z

OpenSSL versions 1.1.1 and 1.0.2 are not affected by this issue.

We don't yet bundle version 3.* so there's nothing OpenJ9 can do to help avoid CVE-2023-2975. OpenJ9 will use 3.* when available, but it will be end users that will have to upgrade an installation of openssl 3.* to avoid that issue.

pshipton · 2023-07-20T18:30:23Z

When we start bundling 3.x for the Oct release we will include the latest CVEs.

paulcheeseman · 2023-07-21T08:23:02Z

We don't yet bundle version 3.* so there's nothing OpenJ9 can do to help avoid CVE-2023-2975

Yes, apologies if I wasn't clear. I meant that when we start bundling 3.x we need to make sure we include this fix. I only mentioned it here because it (currently) needs to be applied manually - it hasn't yet been included in an official OpenSSL release.

pshipton · 2023-07-21T13:11:40Z

For reference, this is what we did for 1.1.1 to include the latest CVE fix.
#17828

pshipton · 2023-08-21T17:23:27Z

jenkins compile all jdk8,jdk11,jdk17

Signed-off-by: Peter Shipton <[email protected]>

pshipton · 2023-08-21T18:52:54Z

jenkins compile all jdk8,jdk11,jdk17

pshipton · 2023-08-21T19:56:17Z

jenkins compile all jdk20

keithc-ca · 2023-08-21T20:31:48Z

Maybe you meant to build jdk21 (jdk20 is no longer supported on the main branches)?

pshipton · 2023-08-21T21:33:49Z

That is true.

jenkins compile all jdk21

pshipton · 2023-08-21T21:35:34Z

PR builds, which passed compilation on all platforms and versions.

https://openj9-jenkins.osuosl.org/job/PullRequest-OpenJ9/4267/
https://openj9-jenkins.osuosl.org/job/PullRequest-OpenJ9/4269/

doc/build-instructions/Build_Instructions_V11.md

See also eclipse-openj9/openj9#14900 Signed-off-by: Peter Shipton <[email protected]>

doc/build-instructions/Build_Instructions_V11.md

keithc-ca · 2023-08-24T15:18:54Z

doc/build-instructions/Build_Instructions_V20.md

@@ -131,7 +131,7 @@ Now fetch additional sources from the Eclipse OpenJ9 project and its clone of Ec
 bash get_source.sh
 ```

-:pencil: **OpenSSL support:** If you want to build an OpenJDK with OpenJ9 binary with OpenSSL support and you do not have a built version of OpenSSL V1.0.2 or v1.1.1 available locally, you must specify `--openssl-version=<version>` where `<version>` is an OpenSSL level like 1.0.2 or 1.1.1. If the specified version of OpenSSL is already available in the standard location (`SRC_DIR/openssl`), `get_source.sh` uses it. Otherwise, the script deletes the content and downloads the specified version of OpenSSL source to the standard location and builds it. If you already have the version of OpenSSL in the standard location but you want a fresh copy, you must delete your current copy.
+:pencil: **OpenSSL support:** If you want to build an OpenJDK with OpenJ9 binary with OpenSSL support and you do not have a built version of OpenSSL v3.x available locally, you must specify `--openssl-version=<version>` where `<version>` is an OpenSSL level like `3.0.10`. If the specified version of OpenSSL is already available in the standard location (`SRC_DIR/openssl`), `get_source.sh` uses it. Otherwise, the script deletes the content and downloads the specified version of OpenSSL source to the standard location and builds it. If you already have the version of OpenSSL in the standard location but you want a fresh copy, you must delete your current copy.


This should be updated for Java 21 (Java 20 doesn't build with the current OpenJ9 repository), but that can be done in a later pull request if you like.

Yes, I'll do it separately.

keithc-ca · 2023-08-24T15:20:34Z

NOTICES.md

 * License: https://github.com/openssl/openssl/blob/OpenSSL_1_1_1-stable/LICENSE
 * Project: https://www.openssl.org/
 * Source: https://github.com/openssl/openssl or https://github.com/ibmruntimes/openssl


Why is this still here?

Just being cautious. OpenJ9 has build instructions for OpenJ9 jenkins builds that set v3 but other builds can do something else. Also on some platforms OpenJ9 uses whatever OpenSSL is found installed. I figure we can remove it sometime down the line when we notice, maybe a couple of years, when OpenSSL 1.1 isn't so widely used.

I'm fine with deferring removal of references to version 1.x, but I don't think we can (or want to) make reference to the license of every version of OpenSSL someone might want to build with. I think we should expect those people to know what they're doing and understand the implications of building against some version and/or repository other than what OpenJ9 (advises and) uses.

I also think the comments

If you built an OpenJDK with OpenJ9 that includes OpenSSL v1.x support ...

are (at best) redundant and should also be removed at the same time we update this file.

It doesn't sound like you are asking me to change anything now, if that's not the case pls clarify. The diff between 1.x and 3.x is OpenSSL changed the license, otherwise we wouldn't have two entries.

I won't object if you want to remove the references to 1.x now, but I'm not asking you to do so.

The only change I'm waiting for is to fix the typo mentioned in #14900 (comment).

Signed-off-by: Peter Shipton <[email protected]>

keithc-ca

No changes have been made that should invalidate the build results mentioned in #14900 (comment).

pshipton marked this pull request as draft April 12, 2022 13:14

keithc-ca mentioned this pull request Apr 29, 2022

Add quotes around perl scripts openssl/openssl#18215

Closed

pshipton force-pushed the openssl3 branch from 015f66d to a991082 Compare May 3, 2022 15:52

pshipton changed the title ~~Update to openssl 3.0.2~~ Update to openssl 3.0.3 May 3, 2022

pshipton force-pushed the openssl3 branch from 60e80d4 to ff414a2 Compare August 21, 2023 17:22

pshipton added the comp:openssl label Aug 21, 2023

Update to openssl 3.0.10

6433e94

Signed-off-by: Peter Shipton <[email protected]>

pshipton force-pushed the openssl3 branch from ff414a2 to 6433e94 Compare August 21, 2023 18:52

pshipton changed the title ~~Update to openssl 3.0.x~~ Update to openssl 3.0.10 Aug 21, 2023

pshipton force-pushed the openssl3 branch from 96089e8 to 95f0be3 Compare August 22, 2023 13:44

pshipton marked this pull request as ready for review August 22, 2023 13:47

pshipton requested a review from keithc-ca August 22, 2023 13:47

keithc-ca reviewed Aug 22, 2023

View reviewed changes

doc/build-instructions/Build_Instructions_V11.md Show resolved Hide resolved

pshipton added a commit to pshipton/openjdk-build that referenced this pull request Aug 22, 2023

Update to openssl 3.0.10

21d7803

See also eclipse-openj9/openj9#14900 Signed-off-by: Peter Shipton <[email protected]>

pshipton mentioned this pull request Aug 22, 2023

Update to openssl 3.0.10 on Win & Mac ibmruntimes/temurin-build#87

Merged

pshipton added a commit to pshipton/openjdk-build that referenced this pull request Aug 23, 2023

Update to openssl 3.0.10 on Win & Mac

bc14b7f

See also eclipse-openj9/openj9#14900 Signed-off-by: Peter Shipton <[email protected]>

keithc-ca requested changes Aug 24, 2023

View reviewed changes

keithc-ca reviewed Aug 24, 2023

View reviewed changes

Update the OpenSSL notices and build instructions

b6a3bb3

Signed-off-by: Peter Shipton <[email protected]>

pshipton force-pushed the openssl3 branch from 95f0be3 to b6a3bb3 Compare August 24, 2023 18:58

keithc-ca approved these changes Aug 24, 2023

View reviewed changes

keithc-ca merged commit ac49658 into eclipse-openj9:master Aug 24, 2023

pshipton mentioned this pull request Sep 5, 2023

Add support for OpenSSL 3.0 #13686

Closed

Update to openssl 3.0.10 #14900

Update to openssl 3.0.10 #14900

Conversation

pshipton commented Apr 11, 2022 • edited Loading

pshipton commented Apr 11, 2022

pshipton commented Apr 11, 2022

pshipton commented Apr 12, 2022 • edited Loading

pshipton commented Apr 12, 2022 • edited Loading

pshipton commented Apr 12, 2022

keithc-ca commented Apr 12, 2022

pshipton commented Apr 12, 2022

pshipton commented Apr 12, 2022

pshipton commented Apr 12, 2022

keithc-ca commented Apr 12, 2022

pshipton commented Apr 12, 2022

pshipton commented Apr 12, 2022

pshipton commented Apr 12, 2022 • edited Loading

pshipton commented Apr 12, 2022

keithc-ca commented Apr 12, 2022

pshipton commented Apr 12, 2022 • edited Loading

keithc-ca commented Apr 13, 2022

keithc-ca commented Apr 13, 2022

keithc-ca commented Apr 13, 2022

keithc-ca commented Apr 13, 2022

keithc-ca commented Apr 13, 2022

keithc-ca commented Apr 14, 2022

keithc-ca commented Apr 21, 2022

pshipton commented Apr 22, 2022

keithc-ca commented Apr 22, 2022

keithc-ca commented May 3, 2022

paulcheeseman commented Jul 19, 2023

keithc-ca commented Jul 20, 2023

pshipton commented Jul 20, 2023 • edited Loading

paulcheeseman commented Jul 21, 2023 • edited Loading

pshipton commented Jul 21, 2023

pshipton commented Aug 21, 2023

pshipton commented Aug 21, 2023

pshipton commented Aug 21, 2023

keithc-ca commented Aug 21, 2023

pshipton commented Aug 21, 2023

pshipton commented Aug 21, 2023 • edited Loading

keithc-ca Aug 24, 2023

Choose a reason for hiding this comment

pshipton Aug 24, 2023

Choose a reason for hiding this comment

keithc-ca Aug 24, 2023

Choose a reason for hiding this comment

pshipton Aug 24, 2023

Choose a reason for hiding this comment

keithc-ca Aug 24, 2023

Choose a reason for hiding this comment

pshipton Aug 24, 2023

Choose a reason for hiding this comment

keithc-ca Aug 24, 2023

Choose a reason for hiding this comment

keithc-ca left a comment

Choose a reason for hiding this comment

pshipton commented Apr 11, 2022 •

edited

Loading

pshipton commented Apr 12, 2022 •

edited

Loading

pshipton commented Apr 12, 2022 •

edited

Loading

pshipton commented Apr 12, 2022 •

edited

Loading

pshipton commented Apr 12, 2022 •

edited

Loading

pshipton commented Jul 20, 2023 •

edited

Loading

paulcheeseman commented Jul 21, 2023 •

edited

Loading

pshipton commented Aug 21, 2023 •

edited

Loading