Flaky test fixed #2147
Open
Flaky test fixed #2147
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
reta
reviewed
Nov 13, 2024
| @@ -176,7 +178,7 @@ public void testThrows() throws Exception { | |||
| ClassCollector cc = context.get(ClassCollector.class); | |||
| assertEquals(2, cc.getServiceClassNames().size()); | |||
|
|
|||
| final Map<String, Class<?>[]> methods = new HashMap<>(); | |||
| final Map<String, Class<?>[]> methods = new LinkedHashMap<>(); | |||
There was a problem hiding this comment.
There is no dependency on ordering anywhere in the test (AFAICT) @Derrick2000 please clarify where the flakyness was observed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Dear Maintainers,
Our Nondex tool has detected a potential flaky test at
org.apache.cxf.tools.wadlto.jaxrs.JAXRSContainerTest.testThrows. This flakiness is caused by an ordering issue when iterating over a HashMap and using assertArrayEquals in the unit test.The issue with map.entrySet() lies in the fact that HashMap does not guarantee any specific iteration order for its entries. This lack of deterministic ordering introduces an element of randomness to the test execution, causing unreliable test results. Here are some reasons why this can happen:
Non-Deterministic Ordering in HashMap: HashMap inherently has no fixed order for its elements, and this order can change across different runs or Java versions. As a result, iterating over map.entrySet() may yield entries in a different sequence each time, depending on factors like hash code distribution and hash collisions.
Use of assertArrayEquals: The assertArrayEquals method in JUnit (and similar testing frameworks) checks for exact order in array elements. When the test iterates over a HashMap and collects entries into an array for comparison, any change in iteration order will cause assertArrayEquals to fail, even if the contents of the array are logically the same.
Impact on Test Consistency: This ordering inconsistency means that the test can pass or fail unpredictably, depending on the iteration order of the HashMap. Such flaky behavior can cause false negatives, where a test fails not because of a bug in the code under test, but due to the non-deterministic nature of HashMap ordering.
Concurrency and Threading Issues: If there are multiple threads accessing or modifying the HashMap during the test (intentionally or unintentionally), this can further exacerbate flakiness. Any modification to the map’s state while iterating can lead to ConcurrentModificationException or silent changes in iteration order.
Thus, we have suggested: