Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Valhalla JTREG test failures #13182

Open
tajila opened this issue Jul 12, 2021 · 15 comments
Open

Valhalla JTREG test failures #13182

tajila opened this issue Jul 12, 2021 · 15 comments
Assignees
Labels
comp:vm project:valhalla Used to track Project Valhalla related work test failure
Milestone

Comments

@tajila
Copy link
Contributor

tajila commented Jul 12, 2021

The following are the jtreg tests failures. They can be found in https://github.com/openjdk/valhalla/tree/lworld/test/hotspot/jtreg/runtime/valhalla/inlinetypes

JT Harness : Tests that failed

runtime/valhalla/inlinetypes/CircularityTest.java: Test initialization of static inline fields with circularity
runtime/valhalla/inlinetypes/ClassPrintLayoutDcmd.java: Test the VM.class_print_layout command
runtime/valhalla/inlinetypes/CreationErrorTest.java: Test data movement with inline types
runtime/valhalla/inlinetypes/EmptyInlineTest.java: Test support for empty inline types (no instance fields)
runtime/valhalla/inlinetypes/FlattenableSemanticTest.java: Flattenable field semantic test
runtime/valhalla/inlinetypes/HiddenInlineClassTest.java: Test a hidden inline class.
runtime/valhalla/inlinetypes/InlineOops.java#id4: Test embedding oops into Inline types
runtime/valhalla/inlinetypes/InlineOops.java#id5: Test embedding oops into Inline types
runtime/valhalla/inlinetypes/InlineOops.java#id6: Test embedding oops into Inline types
runtime/valhalla/inlinetypes/InlineOops.java#id7: Test embedding oops into Inline types
runtime/valhalla/inlinetypes/InlineTypeArray.java: Plain array test for Inline Types
runtime/valhalla/inlinetypes/InlineTypeCreation.java: Inline Type creation test
runtime/valhalla/inlinetypes/InlineTypeDensity.java: Heap density test for InlineTypes
runtime/valhalla/inlinetypes/InlineTypeGetField.java: Inline Type get field test
runtime/valhalla/inlinetypes/InlineTypesTest.java: Test data movement with inline types
runtime/valhalla/inlinetypes/MultiANewArrayTest/MultiANewArrayTest.java: test that mismatches in bottom class of multi-dimensional arrays are correctly detected
runtime/valhalla/inlinetypes/ObjectMethods.java: Check object method implemented by the VM behave with inline types
runtime/valhalla/inlinetypes/QuickeningTest.java: Test quickening of getfield and putfield applied to inline fields
runtime/valhalla/inlinetypes/StaticFieldsTest.java: Test circularity in static fields
runtime/valhalla/inlinetypes/TestFieldTypeMismatch.java:
runtime/valhalla/inlinetypes/TestInheritedInlineTypeFields.java: Test if inline field klasses are correctly retrieved for inherited fields
runtime/valhalla/inlinetypes/UninitializedInlineFieldsTest.java: Uninitialized inline fields test
runtime/valhalla/inlinetypes/UnsafeTest.java: unsafe get/put/with inline type
runtime/valhalla/inlinetypes/ValueTearing.java: Test tearing of inline fields and array elements
runtime/valhalla/inlinetypes/VarArgsArray.java: Test if JVM API using varargs work with inline type arrays
runtime/valhalla/inlinetypes/VolatileTest.java: check effect of volatile keyword on flattenable fields
runtime/valhalla/inlinetypes/WithFieldAccessorTest.java: test nestmate access to an inline type's public, protected and private final fields.
runtime/valhalla/inlinetypes/WithFieldNoAccessTest.jcod: Check that IllegalAccessError exceptions get thrown if a class that is not a nestmate of an inline type tries to write to the inline type's final fields.
runtime/valhalla/inlinetypes/classfileparser/BadACCValue.java: test that if a class file has ACC_INLINE set then it must be run with option -XX:+EnableValhalla.
runtime/valhalla/inlinetypes/classfileparser/BadInlineTypes.java: test that the right exceptions get thrown for bad inline type class files.
runtime/valhalla/inlinetypes/primitiveObject/TestPrimitiveObject.java: test that PrimitiveObject interface is injected correctly
runtime/valhalla/inlinetypes/testSupers/TestSuperClasses.java: test that the JVM detects illegal super classes for inline types.
runtime/valhalla/inlinetypes/verifier/VerifierInlineTypes.java: test that the right exceptions get thrown for bad inline type class files. 
runtime/valhalla/inlinetypes/InlineOops.java#id0: Test embedding oops into Inline types
runtime/valhalla/inlinetypes/InlineOops.java#id1: Test embedding oops into Inline types
runtime/valhalla/inlinetypes/InlineOops.java#id2: Test embedding oops into Inline types
runtime/valhalla/inlinetypes/InlineOops.java#id3: Test embedding oops into Inline types
runtime/valhalla/inlinetypes/InlineWithJni.java: test JNI functions with inline types
runtime/valhalla/inlinetypes/TestJNIIsSameObject.java: Test JNI IsSameObject semantic with inline types 
@tajila tajila added comp:vm project:valhalla Used to track Project Valhalla related work labels Jul 12, 2021
@tajila
Copy link
Contributor Author

tajila commented Jul 12, 2021

In order to run the tests I had to:

@tajila
Copy link
Contributor Author

tajila commented Jul 12, 2021

This issue can be broken up into smaller issues to address the specific failures

@tajila tajila added this to the Release 0.31 (Java 18) milestone Jul 12, 2021
@hangshao0
Copy link
Contributor

Looks like some cases should be captured by verifier, but they are not.

@tajila
Copy link
Contributor Author

tajila commented Jul 13, 2021

Looks like some cases should be captured by verifier, but they are not.

Yes, I would look at those issues last since the spec isn't fully finalized. The first priority are failures relating to field layouts or initialization since that is the most settled part of the spec.

@hangshao0
Copy link
Contributor

As mentioned in the meeting today, I discovered some layout differences between OpenJ9 and RI on Atomic VT and volatile VT fields. More details can be found here: #13130 (comment). Result of some tests here could be affected by such difference.

@hangshao0
Copy link
Contributor

As the verifier change is already merged, we can re-run these tests. @ehrenjulzert

@ehrenjulzert
Copy link

ehrenjulzert commented Nov 18, 2022

When running using make test TEST="hotspot_valhalla" , The following tests fail on openj9:

runtime/valhalla/inlinetypes/CircularityTest.java: Test initialization of static inline fields with circularity 
runtime/valhalla/inlinetypes/ClassPrintLayoutDcmd.java: Test the VM.class_print_layout command 
runtime/valhalla/inlinetypes/HiddenInlineClassTest.java: Test a hidden inline class. 
runtime/valhalla/inlinetypes/InlineTypeArray.java: Plain array test for Inline Types 
runtime/valhalla/inlinetypes/InlineTypeDensity.java: Heap density test for InlineTypes 
runtime/valhalla/inlinetypes/InlineTypesTest.java: Test data movement with inline types 
runtime/valhalla/inlinetypes/InlineWithJni.java: test JNI functions with inline types 
runtime/valhalla/inlinetypes/MultiANewArrayTest/MultiANewArrayTest.java: test that mismatches in bottom class of multi-dimensional arrays are correctly detected 
runtime/valhalla/inlinetypes/ObjectMethods.java: Check object method implemented by the VM behave with inline types 
runtime/valhalla/inlinetypes/QuickeningTest.java: Test quickening of getfield and putfield applied to inline fields 
runtime/valhalla/inlinetypes/TestFieldTypeMismatch.java: 
runtime/valhalla/inlinetypes/TestInheritedInlineTypeFields.java: Test if inline field klasses are correctly retrieved for inherited fields 
runtime/valhalla/inlinetypes/TestJNIIsSameObject.java: Test JNI IsSameObject semantic with inline types 
runtime/valhalla/inlinetypes/UnsafeTest.java: unsafe get/put/with inline type 
runtime/valhalla/inlinetypes/ValuePreloadTest.java: 
runtime/valhalla/inlinetypes/VarArgsArray.java: Test if JVM API using varargs work with inline type arrays 
runtime/valhalla/inlinetypes/VolatileTest.java: check effect of volatile keyword on flattenable fields 
runtime/valhalla/inlinetypes/classfileparser/ACC_CFETest.java: test class access rules for abstract classes that have ACC_VALUE set. 
runtime/valhalla/inlinetypes/classfileparser/ACC_ICCETest.java: test that a value class cannot sub-class an identity class 
runtime/valhalla/inlinetypes/classfileparser/BadInlineTypes.java: test that the right exceptions get thrown for bad inline type class files. 
runtime/valhalla/inlinetypes/classfileparser/PrimitiveUsers.java: test that if a class file uses primitive classes, -XX:+EnablePrimitiveClasses must be set. 
runtime/valhalla/inlinetypes/testClassModifiers/TestClassModifiers.java: test that the JVM correctly accepts or rejects classes based on their ACC_VALUE/ACC_IDENTITY modifiers 
runtime/valhalla/inlinetypes/testSupers/TestSuperClasses.java: test that the JVM detects illegal super classes for value object and primitive value types. 
runtime/valhalla/inlinetypes/verifier/VerifierInlineTypes.java: test that the right exceptions get thrown for bad inline type class files. 
runtime/valhalla/inlinetypes/withfieldTests/RunWithfieldTests.java: test scenarios where getfield, putfield, and withfield access the same constant pool field_ref and test other withfield error cases. 
runtime/valhalla/inlinetypes/InlineOops.java#id0: Test embedding oops into Inline types 
runtime/valhalla/inlinetypes/InlineOops.java#id1: Test embedding oops into Inline types 
runtime/valhalla/inlinetypes/InlineOops.java#id2: Test embedding oops into Inline types 
runtime/valhalla/inlinetypes/InlineOops.java#id3: Test embedding oops into Inline types

The following tests are passing on openj9:

runtime/valhalla/inlinetypes/AbstractValueClassTest.java: Test that a super abstract value class is allowed. 
runtime/valhalla/inlinetypes/CheckcastTest.java: checkcast bytecode test 
runtime/valhalla/inlinetypes/ClassInitializationFailuresTest.java: Test several scenarios of class initialization failures 
runtime/valhalla/inlinetypes/CreationErrorTest.java: Test data movement with inline types 
runtime/valhalla/inlinetypes/EmptyInlineTest.java: Test support for empty inline types (no instance fields) 
runtime/valhalla/inlinetypes/FlattenableSemanticTest.java: Flattenable field semantic test 
runtime/valhalla/inlinetypes/Ifacmp.java: if_acmpeq/ne bytecode test 
runtime/valhalla/inlinetypes/InlineTypeCreation.java: Inline Type creation test 
runtime/valhalla/inlinetypes/InlineTypeGetField.java: Inline Type get field test 
runtime/valhalla/inlinetypes/StaticFieldsTest.java: Test circularity in static fields 
runtime/valhalla/inlinetypes/Test8186715.java: test return of buffered inline type passed in argument by caller 
runtime/valhalla/inlinetypes/TestBytecodeLib.java: Check bytecode test library generates the correct code for Valhalla changes to JVMS 
runtime/valhalla/inlinetypes/TestFieldNullability.java: 
runtime/valhalla/inlinetypes/UninitializedInlineFieldsTest.java: Uninitialized inline fields test 
runtime/valhalla/inlinetypes/VDefaultTest.java: vdefault bytecode test 
runtime/valhalla/inlinetypes/WithFieldAccessorTest.java: test nestmate access to an inline type's public, protected and private final fields. 
runtime/valhalla/inlinetypes/WithFieldTest.java: withfield bytecode test 
runtime/valhalla/inlinetypes/verifier/VTAssignability.java: Test basic verifier assignability of inline types. 
runtime/valhalla/inlinetypes/verifier/VTMonitor.java: Test that verifier allows monitor operations on inline types. 
serviceability/jvmti/Valhalla/FieldAccessModify/FieldAccessModify.java: Tests that all FieldAccess and FieldModification notifications are generated for primitive classes. 

All the tests are passing on RI.

@hangshao0
Copy link
Contributor

Talked to Ehren to also try with flattenning enabled. Some tests might pass with with flattenning enabled.

@ehrenjulzert
Copy link

ehrenjulzert commented Nov 18, 2022

When flattening is enabled (with -XX:ValueTypeFlatteningThreshold=999999 and -XX:+EnableArrayFlattening) two additional tests pass:

runtime/valhalla/inlinetypes/QuickeningTest.java: Test quickening of getfield and putfield applied to inline fields 
runtime/valhalla/inlinetypes/TestInheritedInlineTypeFields.java: Test if inline field klasses are correctly retrieved for inherited fields

The following command was used to run the tests with -XX:ValueTypeFlatteningThreshold=999999 and -XX:+EnableArrayFlattening enabled:

/Users/ehren/Documents/openj9-openjdk-jdk.valuetypes/build/macosx-x86_64-server-release/images/jdk/bin/java -jar /Users/ehren/Documents/jtreg/lib/jtreg.jar -agentvm -a -ea -esa -v:fail,error,time,nopass -retain:fail,error,*.dmp,javacore.*,heapdump.*,*.trc -ignore:quiet -timeoutFactor:8 -xml:verify -concurrency:1 -vmoptions:"-XX:ValueTypeFlatteningThreshold=999999 -XX:+EnableArrayFlattening" -nativepath:/Users/ehren/Documents/valhalla/build/macosx-x86_64-server-release/images/test/jdk/jtreg/native -jdk:/Users/ehren/Documents/openj9-openjdk-jdk.valuetypes//build/macosx-x86_64-server-release/images/jdk /Users/ehren/Documents/openj9-openjdk-jdk.valuetypes/test/hotspot/jtreg/runtime/valhalla/inlinetypes

ehrenjulzert pushed a commit to ehrenjulzert/openj9 that referenced this issue Mar 10, 2023
Correctly generate method signatures for lambda
functions that take Q type arguments

For eclipse-openj9#13182 (fixes error in WithFieldAccessorTest)

Signed-off-by: Ehren Julien-Neitzert <[email protected]>
ehrenjulzert pushed a commit to ehrenjulzert/openj9 that referenced this issue Apr 3, 2023
Some entries in the FCC have their clazz field
set to null, these null classes should not be
dereferenced

For eclipse-openj9#13182 (fixes error in UninitializedInlineFieldsTest, as well as other tests)

Signed-off-by: Ehren Julien-Neitzert <[email protected]>
@hangshao0
Copy link
Contributor

hangshao0 commented May 4, 2023

@ehrenjulzert can you update in #13182 (comment) at the beginning of each test if it has been fixed and is now passing ? sth like:

(passed) runtime/valhalla/inlinetypes/UnsafeTest.java: unsafe get/put/with inline type 
(failed) runtime/valhalla/inlinetypes/ValuePreloadTest.java: 

@ehrenjulzert
Copy link

ehrenjulzert commented May 5, 2023

can you update in #13182 (comment)

I updated it, although I decided to format it differently (I listed the passing and failing tests in two separate code blocks). I figured this would make it easier to compare to in the future (since when the tests are run, this is the format that the results are printed in).

ehrenjulzert pushed a commit to ehrenjulzert/openj9 that referenced this issue May 16, 2023
- Create VM_ValueTypeHelpers::copyFlattenableArray
function and call it instead of
VM_ArrayCopyHelpers::referenceArrayCopy when
copying flattened or primitive arrays
- Throw a NPE instead of an ASE in
doReferenceArrayCopy when an attempt is made to
copy a null object into an array of primitive
value types

for eclipse-openj9#13182

Signed-off-by: Ehren Julien-Neitzert <[email protected]>
midronij pushed a commit to midronij/openj9 that referenced this issue May 24, 2023
- Create VM_ValueTypeHelpers::copyFlattenableArray
function and call it instead of
VM_ArrayCopyHelpers::referenceArrayCopy when
copying flattened or primitive arrays
- Throw a NPE instead of an ASE in
doReferenceArrayCopy when an attempt is made to
copy a null object into an array of primitive
value types

for eclipse-openj9#13182

Signed-off-by: Ehren Julien-Neitzert <[email protected]>
midronij pushed a commit to midronij/openj9 that referenced this issue Jun 1, 2023
- Create VM_ValueTypeHelpers::copyFlattenableArray
function and call it instead of
VM_ArrayCopyHelpers::referenceArrayCopy when
copying flattened or primitive arrays
- Throw a NPE instead of an ASE in
doReferenceArrayCopy when an attempt is made to
copy a null object into an array of primitive
value types

for eclipse-openj9#13182

Signed-off-by: Ehren Julien-Neitzert <[email protected]>
@theresa-m
Copy link
Contributor

theresa-m commented Oct 15, 2024

I reran these tests locally (mac) and will start going through them.

The RI passes all except field_layout/NullMarkersTest.java.

OpenJ9 is failing the following runtime tests.

runtime/valhalla/inlinetypes/AnnotationsTests.java: Test of ImplicitlyConstructible, NullRestricted and LooselyConsistentValue annotations
runtime/valhalla/inlinetypes/CircularityTest.java: Test initialization of static inline fields with circularity
runtime/valhalla/inlinetypes/ClassPrintLayoutDcmd.java: Test the VM.class_print_layout command
runtime/valhalla/inlinetypes/Ifacmp.java: if_acmpeq/ne bytecode test
runtime/valhalla/inlinetypes/InlineTypeArray.java: Plain array test for Inline Types
runtime/valhalla/inlinetypes/InlineTypeDensity.java: Heap density test for InlineTypes
runtime/valhalla/inlinetypes/InlineTypesTest.java: Test data movement with inline types
runtime/valhalla/inlinetypes/InlineWithJni.java: test JNI functions with instances of value classes
runtime/valhalla/inlinetypes/MonitorEnterTest.java#id0:
runtime/valhalla/inlinetypes/MonitorEnterTest.java#id1:
runtime/valhalla/inlinetypes/MonitorEnterTest.java#id2:
runtime/valhalla/inlinetypes/NullRestrictedArrayTest.java: Test of VM.newNullRestrictedArray API
runtime/valhalla/inlinetypes/ObjectMethods.java: Check object methods implemented by the VM behave with value types
runtime/valhalla/inlinetypes/PreloadCircularityTest.java:
runtime/valhalla/inlinetypes/QuickeningTest.java: Test quickening of getfield and putfield applied to inline fields
runtime/valhalla/inlinetypes/StaticFieldsTest.java: Test circularity in static fields
runtime/valhalla/inlinetypes/TestInheritedInlineTypeFields.java: Test if inline field klasses are correctly retrieved for inherited fields
runtime/valhalla/inlinetypes/TestJNIIsSameObject.java: Test JNI IsSameObject semantic with inline types
runtime/valhalla/inlinetypes/ValuePreloadTest.java:
runtime/valhalla/inlinetypes/ValueTearing.java: Test tearing of inline fields and array elements
runtime/valhalla/inlinetypes/VolatileTest.java: check effect of volatile keyword on flattenable fields
runtime/valhalla/inlinetypes/classfileparser/IllegalFieldModifiers.java: test that illegal field modifiers are detected correctly
runtime/valhalla/inlinetypes/classfileparser/ValueClassValidation.java: test validation of value classes
runtime/valhalla/inlinetypes/field_layout/FieldAlignmentTest.java#CompressedOops:
runtime/valhalla/inlinetypes/field_layout/FieldAlignmentTest.java#NoCompressedOops:
runtime/valhalla/inlinetypes/field_layout/NullMarkersTest.java#NullMarker64CompressedOops:
runtime/valhalla/inlinetypes/field_layout/NullMarkersTest.java#NullMarker64NoCompressedOops:
runtime/valhalla/inlinetypes/field_layout/NullMarkersTest.java#NullMarker64NoCompressedOopsNoCompressedKlassPointers:
runtime/valhalla/inlinetypes/field_layout/ValueFieldInheritanceTest.java#64bitsCompressedOops:
runtime/valhalla/inlinetypes/field_layout/ValueFieldInheritanceTest.java#64bitsNoCompressedOops:
runtime/valhalla/inlinetypes/field_layout/ValueFieldInheritanceTest.java#64bitsNoCompressedOopsNoCompressKlassPointers:
runtime/valhalla/inlinetypes/verifier/StrictFields.java:

OpenJ9 passing tests:

runtime/valhalla/inlinetypes/ClassInitializationFailuresTest.java: Test several scenarios of class initialization failures
runtime/valhalla/inlinetypes/EmptyInlineTest.java: Test support for empty inline types (no instance fields)
runtime/valhalla/inlinetypes/FlattenableSemanticTest.java: Flattenable field semantic test
runtime/valhalla/inlinetypes/HiddenInlineClassTest.java: Test a hidden inline class.
runtime/valhalla/inlinetypes/InlineTypeCreation.java: Inline Type creation test
runtime/valhalla/inlinetypes/InlineTypeGetField.java: Inline Type get field test
runtime/valhalla/inlinetypes/Test8186715.java: test return of buffered inline type passed in argument by caller
runtime/valhalla/inlinetypes/TestCloneableValue.java:
runtime/valhalla/inlinetypes/TestFieldNullability.java:
runtime/valhalla/inlinetypes/UninitializedInlineFieldsTest.java: Uninitialized inline fields test
runtime/valhalla/inlinetypes/VarArgsArray.java: Test if JVM API using varargs work with inline type arrays
runtime/valhalla/inlinetypes/verifier/VTAssignability.java: Test basic verifier assignability of inline types.
runtime/valhalla/inlinetypes/verifier/VTMonitor.java: Test that verifier allows monitor operations on inline types.

@theresa-m
Copy link
Contributor

I created a draft pr in the extensions repository that includes a script to run these tests, some minimal changes to get them working, and an exclude list of the failures that I'm working on.
ibmruntimes/openj9-openjdk-jdk.valuetypes#14

@theresa-m
Copy link
Contributor

After a first pass I excluded 4 tests that are not compatible with OpenJ9 and was able to get a few more passing by removing checks for hotspot specific error messages. There are 22 remaining failures, see ProblemList-hotspotjtreg.txt for more details.

@theresa-m
Copy link
Contributor

The following tests are still failing. Some of them are not tests that I would expect to pass with J9 (for example c1 tests) however they are not failing in a way I would expect (crashing or unexpected exception) so I've left them on the list to investigate.

compiler/valhalla/inlinetypes/TestArrayCopyWithOops.java #20522
runtime/valhalla/inlinetypes/TestJNIIsSameObject.java #20512
runtime/valhalla/inlinetypes/verifier/StrictFields.java #20472
runtime/valhalla/inlinetypes/StaticFieldsTest.java #13797
runtime/valhalla/inlinetypes/ObjectMethods.java #13789
runtime/valhalla/inlinetypes/AnnotationsTests.java #20510
compiler/valhalla/inlinetypes/TestUnloadedInlineTypeArray.java - default value is set when its not expected
compiler/valhalla/inlinetypes/TestFlatArrayThreshold.java - default value is set when its not expected
runtime/valhalla/inlinetypes/CircularityTest.java - ClassCircularityError not thrown in the right place
compiler/valhalla/inlinetypes/TestCallingConventionC1.java - ClassCircularityError - shouldn't pass on openj9 but shouldn't throw this error
runtime/valhalla/inlinetypes/InlineWithJni.java - segfault
compiler/valhalla/inlinetypes/TestBufferTearing.java - segfault
compiler/valhalla/inlinetypes/TestLarvalState.java - segfault
runtime/valhalla/inlinetypes/TestInheritedInlineTypeFields.java - vm assertion failure
compiler/valhalla/inlinetypes/TestBufferTearingC1.java - this test shouldn't pass on openj9 but it shouldn't crash

Try again once extensions repo is updated for #20372
compiler/valhalla/inlinetypes/TestIntrinsics.java
runtime/valhalla/inlinetypes/InlineTypesTest.java

These tests expect behavior that is inconsistent with https://openjdk.org/jeps/8316779. Ignore until this is resolved.
runtime/valhalla/inlinetypes/NullRestrictedArrayTest.java
runtime/valhalla/inlinetypes/InlineTypeArray.java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:vm project:valhalla Used to track Project Valhalla related work test failure
Projects
None yet
Development

No branches or pull requests

5 participants