Provide a session test framework for JUnit 5

[HeikoKlare]

The existing implementation of session tests (including SessionTestRunner, SessionTestSuite and WorkspaceSessionTestSuite) relies on JUnit 3 functionality. Because of the changed way in which tests are set up in JUnit 5, they cannot be used to run session tests with JUnit 5. In addition the session test framework is currently overly complex, as it provides some functionality that is not used (anymore).

This commit introduces a new JUnit 5 extension, the SessionTestExtension, to be used for running test cases of a class in
separate Eclipse application sessions with JUnit 5. This extension intercepts test methods calls to start a separate Eclipse session in which the tests are executed. The results are evaluated in the intercepted method execution in way they were evaluated for existing JUnit-3-based session tests. Session that are expected to fail can be annotated with an added @SessionShouldFail annotation.

Functionality is demonstrated in a manually executed sample session test.

Functionality is demonstrated in a manually executed sample session test as well as by application to the session test in the runtime tests project. Since JUnit 5 test suites are not yet supported, the project is adapted to execute tests without suites.
-> resolved via eclipse-tycho/tycho#3358

In order to get the session test running with Tycho, an adaptation of the test application provided via PDE is necessary, because a faulty class loader is currently used for the test engines during remote execution.
-> resolved via eclipse-pde/eclipse.pde#1047

Contributes to:

• Migrate ResourceTests to JUnit 4/5 #903

[github-actions]

Test Results

 1 722 files   1 722 suites   1h 25m 39s ⏱️
 3 950 tests  3 928 ✅  22 💤 0 ❌
12 447 runs  12 286 ✅ 161 💤 0 ❌

Results for commit ff6f7f4.

♻️ This comment has been updated with latest results.

[HeikoKlare]

The migration of session tests to JUnit 5 requires several dependencies to be added to org.eclipse.core.tests.harness, as they have to be provided to ConfigurationSessionTestSuite to be able to run with JUnit 5. I am not in favor of having all these dependencies in the test harness bundle, but since the only alternative I see would be a separate bundle for the ConfigurationSessionTestSuite and the test harness bundle is only used internally anway, I think that adding these dependencies is acceptable. If anyone objects or has a proposal for improvement, please let me know. Otherwise I would like to merge this soon to be able to proceed the migration away from JUnit 3.

[HeikoKlare]

PR is not ready to be merged because of faulty JUnit test result aggregation. GitHub reports 102 removed tests, which is because (existing, JUnit-3-based) session tests are not processed correctly when executed with a JUnit 5 runner. The test methods are not assigned to the their test class but to the running test suite, and since many session test methods are simply named test1, test2 etc., they "overwrite" each other in the logs when assigned to the same test suite. This particularly applies to the AllSessionTests suite, e.g.:

AutomatedResourceTests org.eclipse.core.tests.resources.session.AllSessionTests - test1

This subsumes multiple executions of a test1 method of all session test classes in AllSessionTests.

[HeikoKlare]

PR is not ready to be merged because of faulty JUnit test result aggregation. GitHub reports 102 removed tests, which is because (existing, JUnit-3-based) session tests are not processed correctly when executed with a JUnit 5 runner. The test methods are not assigned to the their test class but to the running test suite, and since many session test methods are simply named test1, test2 etc., they "overwrite" each other in the logs when assigned to the same test suite.

Has been fixed via #1334 and #1335.

[HeikoKlare]

This PR particularly affects the ConfigurationSessionTest class, which is why I checked all users of this class in the Eclipse SDK. I found all other users of that class to be in the Equinox repository. I executed those tests to validate that they all still run successfully.