How to Run a Grinder Build on Jenkins

How to run a Grinder test job at AdoptOpenJDK

Using AdoptOpenJDK Grinder job to test your new CS or rerun a test. These instructions also apply to Grinder jobs at the openj9 Jenkins server (and internal servers).

• Click Build with Parameters, or go to the Grinder job in your internal Jenkins instance and set your own parameters:
  • ADOPTOPENJDK_REPO : optional: your fork of openjdk-tests
  • ADOPTOPENJDK_BRANCH : optional, your branch off of your fork of openjdk-tests
  • PLATFORM : required, platform on which to run the test (used to determine machines labels to find machines on which to run)
  • JDK_VERSION : (required, JDK_VERSION that matches your SDK. e.g., 8, 9, 10, 11, 12)
  • JDK_IMPL : different JVM implementations (hotspot, openj9, sap, ibm)
  • BUILD_LIST : optional, reduces the amount of directories that will be compiled. e.g., openjdk, system, perf, external, functional. Any sub test directory you wish to compile
  • TARGET : required, which test target you want to run. Top-level targets such as sanity.openjdk, extended.system are inappropriate for Grinders as they contain so many sub-targets and have a long execution time. Test targets are defined in playlist.xml files (example jdk_math or jdk_custom). Refer to testCaseName in the playlist.xml files for the values to use for the TARGET parameter.
  • CUSTOM_TARGET : if TARGET=jdk_custom|hotspot_custom|langtools_custom, you can set this to be the specific test class name. For example test/jdk/java/math/BigInteger/BigIntegerTest.java
  • SDK_RESOURCE : relates to which build you wish to test (nightly, releases, customized)
  • CUSTOMIZED_SDK_URL : Link to download SDK. This is needed when SDK_RESOURCE=customized
    Optional, link to download native test libs. (Links can be separated by a space)
  • EXTRA_OPTIONS : set this to append additional JVM options to the test run
  • JVM_OPTIONS : set this to replace the JVM options of the test run
  • ITERATIONS : number of times to repeat the execution of the test run (on one machine)
  • PARALLEL: several modes of parallelization supported [None|Dynamic|Subdir|NodesByIterations], where None -> "run tests serially", Dynamic -> "try to retrieve test execution times and dynamically calculate how to divide tests across NUM_MACHINES", Subdir -> "for nested test directories like external, divide and run parallel by sub-directories" and NodesByIterations -> "run ITERATIONS of a test across NUM_MACHINES"
  • LABEL : if left blank, any machine matching the PLATFORM labels will be selected
  • OPENJ9_REPO : optional, your fork of the eclipse/openj9 repo
  • OPENJ9_BRANCH : optional, your branch off your fork of eclipse/openj9 repo
  • TEST_FLAG: optional. Only set to JITAAS for testing JITAAS SDK
  • ARCHIVE_TEST_RESULTS: optional. If checked, the test outputs will be archived regardless of the test result

Common scenario: Rerun an openjdk regression test that was failing

• You can find "Rerun in Grinder" links in 2 locations on the job that fails.

  • near the top of the job page, before the test run is complete, the "Rerun in Grinder" link of the entire job is posted. Clicking on that link opens a Grinder with pre-populated fields, but it sets the TARGET as the entire target used by the job (example, TARGET=sanity.openjdk).
  • at the bottom of the console output of the test run. Those links will show the test targets that failed. (example, TARGET=jdk_math which is one of the several targets that are contained in the sanity.openjdk set).
  

• To rerun a single test case in the openjdk group (for example the test case java/math/BigInteger/LargeValueExceptions.java, group jdk_math for example), click on either rerun link described above (so that the most of the other parameters used in the failing tests are pre-populated and then edit any parameters you want to be able to run a particular variation of the previous run. In this case, you can set TARGET=jdk_custom and CUSTOM_TARGET=jdk/test/java/math/BigInteger/LargeValueExceptions.java to rerun just that particular test class. The value set in the CUSTOM_TARGET field should match the relative path in the jdk repo from where the test material is pulled (and that this path is different in jdk8u versus jdk11u where in 11+ the test material is organized under a directory called test rather than one called jdk, so CUSTOM_TARGET=test/jdk/java/math/BigInteger/LargeValueExceptions.java)

• In the case you want to rerun on a particular machine that the test failed on, because the test seems to only fail on particular machines, you could change LABEL=hostname for our example, LABEL=test-godaddy-centos7-x64-2

• As a general rule, if you want a finer granularity, as in, if only want to run a specific test, you can set TARGET=jdk_custom and CUSTOM_TARGET=jdk/test/java/math/BigInteger/LargeValueExceptions.java to rerun that particular test class

• If you want to run a directory of tests, TARGET=jdk_custom and CUSTOM_TARGET=jdk/test/java/math/BigInteger to run the test cases within that directory

Running system_custom in a Grinder:

• For running STF based system tests using system_custom target in a Grinder, please ensure the following is set:

  • BUILD_LIST=system
  • TARGET=system_custom
  • CUSTOM_TARGET=-test=<stf_test_class_name> - Optionally, if test requires arguments, CUSTOM_TARGET=-test=<stf_test_class_name> -test-args="x=a,y=b" (e.g. CUSTOM_TARGET=-test=MathLoadTest -test-args="workload=math,timeLimit=5m")

• Note 1: <stf_test_class_name> should be the name of the STF class to run, not a Playlist target name. For example, if you want to re-run ClassLoadingTest_5m, using system_custom, you can simply copy the last portion of the command line from the playlist (e.g. -test=ClassloadingLoadTest -test-args="timeLimit=5m") for CUSTOM_TARGERT.

• Note 2: For running the MiniMix test on JDK 9+, please specify

  • EXTRA_OPTIONS=--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.io=ALL-UNNAMED