New serialization tests fail on Mandrel for JDK 21 (21.0.5-ea), 24.2

[jerboaa]

The tests added with #279 now fail in weekly CI. Not sure if this is expected. They fail with:

2024-09-21T01:08:31.7516407Z [ERROR] Tests run: 15, Failures: 1, Errors: 0, Skipped: 6, Time elapsed: 524.743 s <<< FAILURE! - in org.graalvm.tests.integration.AppReproducersTest
2024-09-21T01:08:31.7518580Z [ERROR] forSerialization21Test{TestInfo}  Time elapsed: 57.868 s  <<< FAILURE!
2024-09-21T01:28:35.0237823Z [ERROR] Failures: 
2024-09-21T01:28:35.0242514Z [ERROR]   AppReproducersTest.forSerialization21Test:830->forSerialization:879 The output of the HotSpot and native-image runs must be the same. ==> expected: <0 0 [Z [Z true
2024-09-21T01:28:35.0346143Z [ERROR] Tests run: 22, Failures: 1, Errors: 0, Skipped: 6
2024-09-21T01:28:35.0349144Z [INFO] testsuite .......................................... FAILURE [28:54 min]
2024-09-21T01:28:35.0361174Z [ERROR]   mvn <args> -rf :testsuite

Looks to be related to oracle/graal#9581

See: https://github.com/graalvm/mandrel/actions/runs/10968086869/job/30459449251#step:11:21167

[jerboaa]

@Karm Is this supposed to fail?

[Karm]

@jerboaa Hmm, it is supposed to fail as in: yes, it is not fixed. Although it should have been excluded from the execution.

[jerboaa]

Right, thanks!

[zakkak]

Although it should have been excluded from the execution.

Why so? Is it to reduce noise on the CI?

AFAIK we want this tested with 23.1 (JDK 21) and will fix it by backporting the fix of oracle/graal#9581 once available.

[Karm]

Why so? Is it to reduce noise on the CI?

Yes.

AFAIK we want this tested with 23.1

Yes.

Although I have no idea how long it might take. I will ask Vojin for pointers and might try to take a look.

[jerboaa]

Although it should have been excluded from the execution.

Why so? Is it to reduce noise on the CI?

Please exclude the test until a fix is ready. It's not practical to add tests which are known to fail for now. Triage is painful enough as it is.

[jerboaa]

Well it turns out the test also fails on latest graal master (JDK 24-based):

Error:  Failures: 
Error:    AppReproducersTest.forSerialization23Test:843->forSerialization:879 The output of the HotSpot and native-image runs must be the same. ==> expected: <0 0 [Z [Z true
1 0 [[Z [[Z true
2 0 [B [B true
3 0 [[B [[B true
4 0 [C [C true
5 0 [[C [[C true
6 0 [D [D true
7 0 [[D [[D true
8 0 [F [F true
9 0 [[F [[F true
10 0 [I [I true
11 0 [[I [[I true
12 0 [J [J true
13 0 [[J [[J true
14 0 [S [S true
15 0 [[S [[S true
16 0 [Ljava/lang/Boolean; [Ljava/lang/Boolean; true
17 0 [[Ljava/lang/Boolean; [[Ljava/lang/Boolean; true
18 0 [Ljava/lang/Byte; [Ljava/lang/Byte; true
19 0 [[Ljava/lang/Byte; [[Ljava/lang/Byte; true
20 0 [Ljava/lang/Character; [Ljava/lang/Character; true
21 0 [[Ljava/lang/Character; [[Ljava/lang/Character; true
22 0 [Ljava/lang/Double; [Ljava/lang/Double; true
23 0 [[Ljava/lang/Double; [[Ljava/lang/Double; true
24 0 [Ljava/lang/Float; [Ljava/lang/Float; true
25 0 [[Ljava/lang/Float; [[Ljava/lang/Float; true
26 0 [Ljava/lang/Integer; [Ljava/lang/Integer; true
27 0 [[Ljava/lang/Integer; [[Ljava/lang/Integer; true
28 0 [Ljava/lang/Long; [Ljava/lang/Long; true
29 0 [[Ljava/lang/Long; [[Ljava/lang/Long; true
30 0 [Ljava/lang/Short; [Ljava/lang/Short; true
31 0 [[Ljava/lang/Short; [[Ljava/lang/Short; true
32 0 [Ljava/lang/String; [Ljava/lang/String; true
33 0 [[Ljava/lang/String; [[Ljava/lang/String; true
34 0 [Ljava/lang/Void; [Ljava/lang/Void; true
35 0 [[Ljava/lang/Void; [[Ljava/lang/Void; true
36 0 [Ljava/lang/Class; [Ljava/lang/Class; true
37 0 [[Ljava/lang/Class; [[Ljava/lang/Class; true
38 0 [Ljava/lang/Object; [Ljava/lang/Object; true
39 0 [[Ljava/lang/Object; [[Ljava/lang/Object; true
40 0 [Ljava/lang/Number; [Ljava/lang/Number; true
41 0 [[Ljava/lang/Number; [[Ljava/lang/Number; true
42 0 [Ljava/lang/Comparable; [Ljava/lang/Comparable; true
43 0 [[Ljava/lang/Comparable; [[Ljava/lang/Comparable; true
44 0 [Ljava/io/Serializable; [Ljava/io/Serializable; true
45 0 [[Ljava/io/Serializable; [[Ljava/io/Serializable; true
46 0 [Ljava/lang/Cloneable; [Ljava/lang/Cloneable; true
47 0 [[Ljava/lang/Cloneable; [[Ljava/lang/Cloneable; true
48 0 [Ljava/lang/Readable; [Ljava/lang/Readable; true
49 0 [[Ljava/lang/Readable; [[Ljava/lang/Readable; true
50 0 [Ljava/lang/AutoCloseable; [Ljava/lang/AutoCloseable; true
51 0 [[Ljava/lang/AutoCloseable; [[Ljava/lang/AutoCloseable; true
52 0 [Ljava/io/Closeable; [Ljava/io/Closeable; true
53 0 [[Ljava/io/Closeable; [[Ljava/io/Closeable; true
54 0 [Ljava/lang/Appendable; [Ljava/lang/Appendable; true
55 0 [[Ljava/lang/Appendable; [[Ljava/lang/Appendable; true
56 0 [Ljava/io/PrintStream; [Ljava/io/PrintStream; true
57 0 [[Ljava/io/PrintStream; [[Ljava/io/PrintStream; true
58 0 [Ljava/io/PrintWriter; [Ljava/io/PrintWriter; true
59 0 [[Ljava/io/PrintWriter; [[Ljava/io/PrintWriter; true
60 0 [Ljava/lang/Record; [Ljava/lang/Record; true
61 0 [[Ljava/lang/Record; [[Ljava/lang/Record; true
62 0 [Ljava/lang/Enum; [Ljava/lang/Enum; true
63 0 [[Ljava/lang/Enum; [[Ljava/lang/Enum; true
64 0 [[[Ljava/lang/Enum; [[[Ljava/lang/Enum; true
65 0 [[[[Ljava/lang/Enum; [[[[Ljava/lang/Enum; true
66 0 [[[[[Ljava/lang/Enum; [[[[[Ljava/lang/Enum; true
67 0 [Lfor_serialization/Main$Muhehehe; [Lfor_serialization/Main$Muhehehe; true
68 0 [[Lfor_serialization/Main$Muhehehe; [[Lfor_serialization/Main$Muhehehe; true
69 0 [[[Lfor_serialization/Main$Muhehehe; [[[Lfor_serialization/Main$Muhehehe; true
70 0 [Lfor_serialization/Main$MuheheheNested; [Lfor_serialization/Main$MuheheheNested; true
71 0 [[Lfor_serialization/Main$MuheheheNested; [[Lfor_serialization/Main$MuheheheNested; true
72 0 [[[Lfor_serialization/Main$MuheheheNested; [[[Lfor_serialization/Main$MuheheheNested; true
73 0 [Lfor_serialization/Main$MuheheheNested$MuheheheNestedNested; [Lfor_serialization/Main$MuheheheNested$MuheheheNestedNested; true
74 0 [[Lfor_serialization/Main$MuheheheNested$MuheheheNestedNested; [[Lfor_serialization/Main$MuheheheNested$MuheheheNestedNested; true
75 0 [[[Lfor_serialization/Main$MuheheheNested$MuheheheNestedNested; [[[Lfor_serialization/Main$MuheheheNested$MuheheheNestedNested; true

Done.
> but was: <0 0 [Z [Z true
1 0 [[Z [[Z true
2 0 [B [B true
3 0 [[B [[B true
4 0 [C [C true
5 0 [[C [[C true
6 0 [D [D true
7 0 [[D [[D true
8 0 [F [F true
9 0 [[F [[F true
10 0 [I [I true
11 0 [[I [[I true
12 0 [J [J true
13 0 [[J [[J true
14 0 [S [S true
15 0 [[S [[S true
16 0 [Ljava/lang/Boolean; [Ljava/lang/Boolean; true
17 0 [[Ljava/lang/Boolean; [[Ljava/lang/Boolean; true
18 0 [Ljava/lang/Byte; [Ljava/lang/Byte; true
19 0 [[Ljava/lang/Byte; [[Ljava/lang/Byte; true
20 0 [Ljava/lang/Character; [Ljava/lang/Character; true
21 0 [[Ljava/lang/Character; [[Ljava/lang/Character; true
22 0 [Ljava/lang/Double; [Ljava/lang/Double; true
23 0 [[Ljava/lang/Double; [[Ljava/lang/Double; true
24 0 [Ljava/lang/Float; [Ljava/lang/Float; true
25 0 [[Ljava/lang/Float; [[Ljava/lang/Float; true
26 0 [Ljava/lang/Integer; [Ljava/lang/Integer; true
27 0 [[Ljava/lang/Integer; [[Ljava/lang/Integer; true
28 0 [Ljava/lang/Long; [Ljava/lang/Long; true
29 0 [[Ljava/lang/Long; [[Ljava/lang/Long; true
30 0 [Ljava/lang/Short; [Ljava/lang/Short; true
31 0 [[Ljava/lang/Short; [[Ljava/lang/Short; true
Error: Exception in thread "main" com.oracle.svm.core.jdk.UnsupportedFeatureError: SerializationConstructorAccessor class not found for declaringClass: [Ljava.lang.String; (targetConstructorClass: java.lang.Object). Usually adding [Ljava.lang.String; to serialization-config.json fixes the problem.
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:121)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.reflect.serialize.SerializationSupport.getSerializationConstructorAccessor(SerializationSupport.java:228)
	at java.base@24-beta/jdk.internal.reflect.SerializationConstructorAccessorGenerator.generateSerializationConstructor(SerializationConstructorAccessorGenerator.java:53)
	at java.base@24-beta/jdk.internal.reflect.ReflectionFactory.generateConstructor(ReflectionFactory.java:340)
	at java.base@24-beta/jdk.internal.reflect.ReflectionFactory.newConstructorForSerialization(ReflectionFactory.java:253)
	at jdk.unsupported@24-beta/sun.reflect.ReflectionFactory.newConstructorForSerialization(ReflectionFactory.java:100)
	at for_serialization.Main.main(Main.java:147)
	at java.base@24-beta/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)

See: https://github.com/graalvm/mandrel/actions/runs/11043295615/job/30678091813#step:11:21252

[zakkak]

Yes, that's expected as well (see oracle/graal#9581)

[jerboaa]

The only job where this seemingly passes is Mandrel for JDK 23:
https://github.com/graalvm/mandrel/actions/runs/10968086869/job/30459598573

But there I see no sign of it actually being run.

EDIT: It may be a too old run, which didn't have the new test yet.

[jerboaa]

Yes, that's expected as well (see oracle/graal#9581)

Maybe I'm not up to speed, but is this test supposed to pass anywhere yet? This is just generating noise at the moment :-/

[jerboaa]

@Karm Please prioritize this fix since it generates test noise. A possible solution for now until we have fixes would be to just disabling it for now.

[zakkak]

But there I see no sign of it actually being run.

Yes, that's because it doesn't :) see #281 (comment)

Maybe I'm not up to speed, but is this test supposed to pass anywhere yet?

No, it looks like it depends on oracle/graal#9581

@Karm Please prioritize this fix since it generates test noise. A possible solution for now until we have fixes would be to just disabling it for now.

I think we should better disable only the oracle/graal#9581 related tests, see #286

[Karm]

I am sorry for the noise, this should have been disabled. I should have left it with @ Disabled in the first place, not fidgeting with min.max versions.

[jerboaa]

We don't see this anywhere any more. Closing.