Skip to content
This repository has been archived by the owner on Oct 14, 2024. It is now read-only.

Promote AdoptOpenJDK Version jdk-11.0.6+10 #132

Closed
1 of 2 tasks
gdams opened this issue Jan 15, 2020 · 42 comments
Closed
1 of 2 tasks

Promote AdoptOpenJDK Version jdk-11.0.6+10 #132

gdams opened this issue Jan 15, 2020 · 42 comments
Assignees
Milestone

Comments

@gdams
Copy link
Member

gdams commented Jan 15, 2020

Java Version:
jdk-11.0.6+10

JVM:

  • OpenJ9
  • Hotspot

@AdoptOpenJDK/tsc please can somebody +1 this request?

@gdams
Copy link
Member Author

gdams commented Jan 15, 2020

Phase 1 ready for publishing (Linux x64, Mac x64, Windows x64/x86). https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk11-pipeline/704/

@jerboaa
Copy link

jerboaa commented Jan 15, 2020

+1

@gdams
Copy link
Member Author

gdams commented Jan 15, 2020

Test failures:

Mac:
extended.system:

HCRLateAttachWorkload_0 => Failed on 4/5 last builds. Raised openjdk-tests/1555 - looks like a long-standing, recurring issue (as seen in deep history view)

Linux
sanity.openjdk:

java/io/StringBufferInputStream/OverflowInRead.java.OverflowInRead => Passed on re-run Grinder/1772
java/math/BigInteger/LargeValueExceptions.java.LargeValueExceptions => Passed on re-run Grinder/1774
java/net/Inet6Address/B6206527.java.B6206527 => Intermittent (issue) Passed on re-run Grinder/1775
java/net/ipv6tests/B6521014.java.B6521014 => Intermittent (issue) Passed on re-run Grinder/1776

special.function:
MBCS_Tests_CLDR_11_ja_JP_linux_0
MBCS_Tests_CLDR_11_ko_KR_linux_0

@jerboaa
Copy link

jerboaa commented Jan 15, 2020

Linux sanity.openjdk:
java/io/StringBufferInputStream/OverflowInRead.java.OverflowInRead
java/math/BigInteger/LargeValueExceptions.java.LargeValueExceptions
java/net/Inet6Address/B6206527.java.B6206527
java/net/ipv6tests/B6521014.java.B6521014

FWIW, these passed for the upstream build:
https://ci.adoptopenjdk.net/view/Test_upstream/job/Test_openjdk11_hs_sanity.openjdk_x86-64_linux_upstream/33/

@smlambert
Copy link

We should run grinders on those, using the same machines (LABEL=hostname) that the upstream jobs ran on, to rule out machine issues.

@gdams
Copy link
Member Author

gdams commented Jan 15, 2020

@smlambert I tried doing that but I think I set the job up wrong? https://ci.adoptopenjdk.net/job/Grinder/1768/console

@adam-thorpe
Copy link

You need TARGET=jdk_custom and LABEL=machine name

@adam-thorpe
Copy link

Oh and sorry, CUSTOM_TARGET needs to be the full path with 'test/jdk' for 11+ at the front and no OverflowInRead at the end eg. test/jdk/java/io/StringBufferInputStream/OverflowInRead.java

@smlambert
Copy link

https://ci.adoptopenjdk.net/job/Grinder/1771/ runs the correct TARGET and CUSTOM_TARGET values, though I did not restrict it to a particular machine by setting LABEL=hostname

@adam-thorpe
Copy link

Shelley you still have the .OverflowInRead at the end

@gdams
Copy link
Member Author

gdams commented Jan 15, 2020

kicked off https://ci.adoptopenjdk.net/job/Grinder/1772/

@smlambert
Copy link

smlambert commented Jan 15, 2020

The 6 failing test targets have been triaged in #132 (comment) and shown to be non-blocking failures, +1 to publish.
Screen Shot 2020-01-15 at 11 46 29 AM

@gdams
Copy link
Member Author

gdams commented Jan 15, 2020

Releasing Phase 1 platforms (Linux x64, Mac x64, Windows x64/x86)

@karianna karianna added this to the January 2020 milestone Jan 16, 2020
@gdams
Copy link
Member Author

gdams commented Jan 16, 2020

@gdams
Copy link
Member Author

gdams commented Jan 16, 2020

Test failures:

AIX:
extended.system:

LockingLoadTest_0
MathLoadTest_all_0

sanity.openjdk:
java/lang/ProcessBuilder/Basic.java => (Fails Intermittently adoptium/aqa-tests#1013, Upstream Bug Report)
java/nio/channels/AsyncCloseAndInterrupt.java => Known to fail intermitently on Linux but NOT AIX (adoptium/aqa-tests#602)

Linux - aarch64: => All intermittently fail on upstream too (see #132 (comment))
java/lang/invoke/VarHandles/VarHandleTestAccessBoolean.java
java/lang/invoke/VarHandles/VarHandleTestAccessDouble.java
java/lang/invoke/VarHandles/VarHandleTestAccessFloat.java
java/lang/invoke/VarHandles/VarHandleTestAccessShort.java
java/lang/invoke/VarHandles/VarHandleTestAccessString.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessByte.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessChar.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessDouble.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessFloat.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessInt.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessLong.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessShort.java

@jerboaa
Copy link

jerboaa commented Jan 16, 2020

Linux - aarch64:
java/lang/invoke/VarHandles/VarHandleTestAccessBoolean.java
java/lang/invoke/VarHandles/VarHandleTestAccessDouble.java
java/lang/invoke/VarHandles/VarHandleTestAccessFloat.java
java/lang/invoke/VarHandles/VarHandleTestAccessShort.java
java/lang/invoke/VarHandles/VarHandleTestAccessString.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessByte.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessChar.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessDouble.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessFloat.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessInt.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessLong.java
java/lang/invoke/VarHandles/VarHandleTestMethodHandleAccessShort.java

We've been seeing those VarHandle test failures on Aarch64 for upstream builds intermittently too. We are going back and forth about this as we a) would need to run jcstress on that hardware to know whether or not these are HW bugs. b) would need access to those machines in order to see if failures reproduce when being run manually. It's yet unclear whether these are actual bugs (in HW or hotspot code).

On known good hardware we (internally) don't see those failures.

@smlambert
Copy link

The extended.system failures on AIX need further investigation. I think they are the same failures that @karianna reported on last release (that we had a follow-up action to look at - but did not complete that action: #119 (comment)).

@smlambert
Copy link

smlambert commented Jan 16, 2020

Investigation of AIX failures: launch the individual test targets in a Grinder against the past releases to see if the failures are seen in those releases.

MathLoadTest_all :
Looks like this has existed for a while all the way back to April 2019 release, grabbed https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.3%2B7/OpenJDK11U-jdk_ppc64_aix_hotspot_11.0.3_7.tar.gz and
see same failure https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/1792

LockingLoadTest uses some of the Math API tests underneath, and is failing with the same error.
LLT testFailure: testTan(net.adoptopenjdk.test.math.MathAPITest): tan(double)[10] :: expected:<1.2786058599594134> but was:<1.2786058599594137>
LLT junit.framework.AssertionFailedError: tan(double)[10] :: expected:<1.2786058599594134> but was:<1.2786058599594137>
LLT at junit.framework.Assert.fail(Assert.java:57)

Verified that it has existed at least as far back as the April 2019 release (Grinder 1793).

Given we have already released with this failure, will not block this release due to that failure (in MathAPITest), but recommend a deeper investigation to see why it passes on openj9 but fails on hotspot (and determine if its a test case that needs to be fixed or a long-standing 'real' issue).

@smlambert
Copy link

smlambert commented Jan 16, 2020

I am now looking at OpenJ9 jdk11 pipelines launched by @lumpfish:

Submitted openj9 jdk11 builds as:
https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk11-pipeline/708/ - xlinux (built with new jitserver feature which requires and extra build configure arg)
Screen Shot 2020-01-16 at 4 39 17 PM

https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk11-pipeline/710/ - xlinuxXL (built with jitserver - had to resubmit owing to problem with build machine)
Screen Shot 2020-01-16 at 8 26 16 PM

https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk11-pipeline/709/ - all other platforms except AIX.
Screen Shot 2020-01-16 at 4 39 50 PM

AIX builds are currently taking very long to run, so don't want to hold up releasing the other platforms
https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk11-pipeline/711/ - AIX . (waiting on machine)

@smlambert
Copy link

smlambert commented Jan 16, 2020

Pipeline 708: same 2 test targets failing from special.functional group as did in the hotspot build.
MBCS_Tests_CLDR_11_ja_JP_linux_0
MBCS_Tests_CLDR_11_ko_KR_linux_0

I believe this to be a test case issue that was previously reported (and thought to be fixed).

adoptium/aqa-tests#1394
and fixed:
adoptium/aqa-tests#1406
but we should raise an issue (or reopen 1394) for these 2 failing targets.

@smlambert
Copy link

smlambert commented Jan 16, 2020

Pipeline 709: only 2 test targets failing from sanity.openjdk against mac_xl build (not seen on mac build):

jdk_lang:
java/lang/invoke/lambda/LogGeneratedClassesTest.java - seen this before (reported in various openj9 issues, and openjdk-tests/585) - Grinder 1794 passes

Deep history for jdk_lang on jdk11 j9 mac_xl has been green for a long time, so if intermittent, very infrequent.

jdk_rmi:
java/rmi/activation/Activatable/checkRegisterInLog/CheckRegisterInLog.java - seen this one before (reported in past issues openj9/4772, openjdk-tests/585) - Grinder 1795 passes

Deep history on jdk_rmi on jdk11 j9 mac_xl also green.

Ran some Grinders (links above) which passed. Deep history shows no previous failures. Will not block on these 2 test failures, though recommend grinder with ITERATIONS=xx to see if can reproduce (perhaps on same machine where failures seen in release build).

@smlambert
Copy link

Pipeline 710: same 2 test targets failing from special.functional group as did in the hotspot build.
MBCS_Tests_CLDR_11_ja_JP_linux_0
MBCS_Tests_CLDR_11_ko_KR_linux_0

Considered non-blocking.

@gdams
Copy link
Member Author

gdams commented Jan 17, 2020

@smlambert am I good to ship the secondary platforms for jdk11 then?

@adam-thorpe
Copy link

https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk11-pipeline/710/ - xlinuxXL (built with jitserver - had to resubmit owing to problem with build machine)

Pipeline job 710 is an xLinux-openj9 build not xLinuxXL-openj9

@andrew-m-leonard
Copy link

jdk-11.0.6+10_openj9-0.18.0 for xLinuxXL being resubmitted, as was misstakenly re-submitted as xLinux last time: Queued:
https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk11-pipeline/713/

@andrew-m-leonard
Copy link

andrew-m-leonard commented Jan 17, 2020

https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk11-pipeline/711/ - AIX:
Failed to build:
Sjavac server failed to initialize: No port file values materialized. Giving up after 60195 ms
Will wait for @sxa555 to analyse

@adam-thorpe
Copy link

adam-thorpe commented Jan 17, 2020

jdk-11.0.6+10_openj9-0.18.0 ppc64leLinuxXL sanity.openjdk job in pipeline 709 failed: https://ci.adoptopenjdk.net/view/Test_openjdk/job/Test_openjdk11_j9_sanity.openjdk_ppc64le_linux_xl/37/. re-ran a grinder: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/1799/

@andrew-m-leonard
Copy link

andrew-m-leonard commented Jan 17, 2020

jdk-11.0.6+10_openj9-0.18.0 summary so far:

Pipelines Ready for Publish, no blocking failures:

  • #708 (xLinux)

Waiting:

  • #709 (all platforms bar AIX) : waiting for pLinux openjdk.sanity re-run which failed to run
  • #711 (AIX) : fails to build... @sxa555

@adam-thorpe
Copy link

pLinuxXL-openj9 openjdk.sanity run passed with no failures: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/1799/

@smlambert
Copy link

To explicitly answer your question #132 (comment) @gdams, yes, I am +1 to shipping 2ndary platforms for jdk11 hotspot.

@gdams
Copy link
Member Author

gdams commented Jan 17, 2020

@smlambert thanks I'll ship them now!

@andrew-m-leonard
Copy link

jdk-11.0.6+10_openj9-0.18.0 summary so far:

Pipelines Ready for Publish, no blocking failures:
- #708 (xLinux)
- #709 (all platforms bar AIX)
Waiting:
- #711 (AIX) : fails to build... @sxa555

@andrew-m-leonard
Copy link

andrew-m-leonard commented Jan 17, 2020

@tsc jdk-11.0.6+10_openj9-0.18.0 on all platforms except AIX, is ready for "Publishing", all tests good with no-blockers. Permission to publish please?

  • #708 (xLinux)
  • #709 (all platforms bar AIX)
  • #713 (xLinuxXL rebuild)

@jerboaa
Copy link

jerboaa commented Jan 17, 2020

+1

@sxa
Copy link
Member

sxa commented Jan 17, 2020

+2

@andrew-m-leonard
Copy link

All platforms bar AIX Published.
AIX build started: https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk11-pipeline/715/

@andrew-m-leonard
Copy link

andrew-m-leonard commented Jan 20, 2020

jdk-11.0.6+10_openj9-0.18.0 AIX test failures:

java/util/logging/LogManagerAppContextDeadlock.java.LogManagerAppContextDeadlock | 11 sec | 1
java/lang/ProcessBuilder/Basic.java#id0.Basic_id0 | 43 sec | 2
java/lang/invoke/condy/CondyNestedResolutionTest.java.CondyNestedResolutionTest | 12 sec | 2
java/nio/channels/AsyncCloseAndInterrupt.java.AsyncCloseAndInterrupt | 16 min | 2
java/util/concurrent/locks/Lock/TimedAcquireLeak.java.TimedAcquireLeak

Re-testing...
Consistently fail...

@andrew-m-leonard
Copy link

@smlambert Are you ok to published with the above failures on AIX?

@smlambert
Copy link

This is same as for the jdk13 builds (my comment #134 (comment)), so we can proceed to publish AIX given the extra sanity.functional tests against openj9 for jdk_lang functionality.

@andrew-m-leonard
Copy link

Thanks @smlambert i'll go ahead and publish

@andrew-m-leonard
Copy link

jdk-11.0.6+10_openj9-0.18.0 all platforms now published

@gdams
Copy link
Member Author

gdams commented Jan 21, 2020

closing as the release is complete

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants