Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add howto contribution iteration 1 #5471

Merged
merged 7 commits into from
Aug 5, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/pages/aqa-howto.markdown
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
## How-to Guides

- [Guidance for Creating OpenJDK Test Defects](Guidance-for-Creating-OpenJDK-Test-Defects.md)
- [First test suite contribution quick guide](howto/first-contrib.markdown)
andrlos marked this conversation as resolved.
Show resolved Hide resolved
42 changes: 42 additions & 0 deletions docs/pages/howto/first-contrib.markdown
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
# Feel free to add content and custom Front Matter to this file.
# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults

layout: default
---

# How-To Contribute your first test suite to the aqa-tests repository

This is a quick guide of adding a testsuite to adoptium aqa-vit. The author of this quick guide has so far only made a contribution into functional tests and some of the other test groups might feature different prerequisites.
andrlos marked this conversation as resolved.
Show resolved Hide resolved

## Requirements:

An actual java/temurin compatible testsuite that you want to contribute.
andrlos marked this conversation as resolved.
Show resolved Hide resolved

## Actual steps:

1. First you need to decide what kind of testsuite you are adding. Create an [issue](https://github.com/adoptium/aqa-tests/issues) about contribution to aqa-tests.. here you can discuss with responsible engineers on where to situate your testsuite. Make sure to explain briefly what is the purpose of your testsuite and what specific part of java implementation it tests. Also describe special requirements that your test needs to work (containers, sudo access, graphical display etc..).
andrlos marked this conversation as resolved.
Show resolved Hide resolved

2. ***Fork the aqa-tests*** repository to your own personal fork. We will be executing the testsuite from this fork later.

3. Once you determine in what testgroup your testsuite is gonna end up in, there are now at least two files you need to prepare.
andrlos marked this conversation as resolved.
Show resolved Hide resolved

1. First one is the ***build.xml*** file that sets up the environment for your test. It also clones the repository of your testsuite to the testing folder. A good approach is to look for build.xml files within other testsuite folders and get inspired there. A good simple example of such build.xml is within functional/testHeadlessComponents.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could add a hyperlink to the functional/testHeadlessComponents directory

andrlos marked this conversation as resolved.
Show resolved Hide resolved

2. The second file we need to discuss is the ***playlist.xml***. This one manages the execution of the testsuite itself as well as what platforms it will be scheduled for. Like with the build.xml files it is advisable to consult other playlists in already existing testsuites.
andrlos marked this conversation as resolved.
Show resolved Hide resolved

The one thing that is not immediately obvious but is ***mandatory*** after every execution is the ***usage of $(TEST_STATUS)*** statement after test execution in the ***playlist.xml*** . This is where the decision on how the test is colored is made.
andrlos marked this conversation as resolved.
Show resolved Hide resolved

5. Once you think you have both the build.xml and playlist.xml you can ***test your integration*** with [Grinder JOB](https://ci.adoptium.net/view/Test_grinder/job/Grinder/).
andrlos marked this conversation as resolved.
Show resolved Hide resolved
Click the ***"Build with Parameters" button*** in the left panel (if there is no such option, make sure you are logged in, and have appropriate permissions set for your account).
Here you can configure your run. Main two parameters that are of interest to us are ADOPTOPENJDK_REPO, where you can put URL of your forked aqa-tests repository and ADOPTOPENJDK_BRANCH that specified the branch your integration is on.
The documentation in the grinder configurator is pretty descriptive, so lets just mention a few other key paremeters you might find useful and their usage.
andrlos marked this conversation as resolved.
Show resolved Hide resolved

| Parameter | Purpose |
| --- | --- |
| BUILD_LIST | is the relative path to the folder containing your build.xml and playlist.xml files |
| TARGET | refers to the name you specified in "testCaseName" section in your playlist.xml file |
| KEEP_WORKSPACE | after testrun the workspace is saved for a while for the developer to check it |
| ARCHIVE_TEST_RESULTS | after testrun the test results are by default discarded for passing tests, check for keeping them |

It is advised to check KEEP_WORKSPACE and ARCHIVE_TEST_RESULTS checkboxes for debuggin purposes. Adoptium infrastructure by default does not keep workspaces at all and results in case the tests were successful. In some cases runs can report as successful when it is not even executed, therefore nothing had a chance of failing there. It is a good practice to not only check that the test finishes "green" but also that the tests that were supposed to run actually did.
andrlos marked this conversation as resolved.
Show resolved Hide resolved