diff --git a/README.md b/README.md index 206cea1e6..e4824a432 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ Eclipse Adoptium makes use of these scripts to build binaries on the build farm This repository contains several useful scripts in order to build OpenJDK personally or at build farm scale via jenkins. For the Temurin project at -Adoptium, this is done with the jenkins instance at https://ci.adotpium.net +Adoptium, this is done with the jenkins instance at https://ci.adoptium.net 1. The `docs` folder contains images and utility scripts to produce up to date documentation. @@ -15,6 +15,13 @@ documentation. (e.g. build | test | checksum | release). 3. The `tools` folder contains `pipelines/` analysis scripts that deliever success/failure trends and build scripts for code-tool dependancies for the build and test process (e.g. asmtools | jcov | jtharness | jtreg | sigtest). +For those who prefer diagrams, there is an overview of the information below +including it's interactions with the scripts in other repositories in our +[docs/ARCHITECTURE.md] file and specific ones on the pipeline types in +[docs/diagram.md]. If you want to set up these pipelines in your own jenkins +instance, see +[this guide](https://github.com/adoptium/ci-jenkins-pipelines/blob/master/docs/UsingOurScripts.md). + ## Overview of pipeline types The starting point on the jenkins instance from the perspective of the @@ -109,11 +116,15 @@ are triggered on demand from a subset of authorized users. In addition to the main pipelines we have "PR tester" jobs that are run on PRs to the pipelines repository in order to ensure they do not have any unintended side effects before they are merged. These are triggered when -comments from authorized users are added into the PR comments. In that +[specific comments from authorized users](https://github.com/adoptium/ci-jenkins-pipelines/blob/master/pipelines/build/prTester/README.md#usage) +are added into the PR. In that folder in jenkins there are separate versions of all of the openjdkXX-pipelines that can be used to run against PRs and will not "pollute" the history of the main pipelines. +More documentation on the PR tester process can be found in +[the prTester documentation](pipelines/build/prTester). + ## Subjobs of the top level pipelines Each of the top level pipelines described above invoke lower level jobs to @@ -133,10 +144,10 @@ should also not be confused with the build-scripts/release folder which contains jobs used for the final publishing of the builds (early access or GA) to github -*Note: jdkXX is generally related to the name of the upstream codebase, which +_Note: jdkXX is generally related to the name of the upstream codebase, which will often have a `u` suffix. At the moment we have a separate set of jobs for non-u and u versions when the upstream codebase changes. TODO: Add note -about the new branch process for jdk23+* +about the new branch process for jdk23+_ Inside the jdkXX folders there are pipelines which perform a build of one variant (e.g. Temurin) for on JDK version on one platform, for example @@ -146,7 +157,9 @@ which are reponsible for running the build using and initiating the tests and other jobs against the completed build if successful. A "Smoke Test" job such as [jdk21u-linux-aarch64-temurin-SmokeTests](https://ci.adoptium.net/job/build-scripts/job/jobs/job/jdk21u/job/jdk21u-linux-aarch64-temurin_SmokeTests/) -(TODO: Links to info about what that contains) which is initiated after the +which is similar to our test jobs but runs the tests from the temurin-build +[buildAndPackage directory](https://github.com/adoptium/temurin-build/tree/master/test/functional/buildAndPackage) +which is initiated after the build perfoms some basic tests against the build artefacts and acts as a gate to kicking off the subsequent steps. Once complete, the openjdkXX-pipelines which run the early access builds will generally invoke @@ -156,9 +169,9 @@ the jobs to publish them as a release in github (e.g. ## Job generation As you can see from the above sections, there are a lot of separate jobs in -jenkins which are used during the build process. These are not created -manually, but are autogenerated using the generator pipelines. TODO: More -information on this is at ... +jenkins which are used during the build process. Since there are so many of +them, these are not created manually, but are autogenerated using an +automatic generator process. The top level [build-pipeline-generator](https://ci.adoptium.net/job/build-scripts/job/utils/job/build-pipeline-generator/) @@ -170,8 +183,9 @@ jobs which use [build_job_generator.groovy](pipelines/build/regeneration/build_job_generator.groovy) to generate the subjobs for each platform/variant combination. Both of these pipelines are triggered on a change (PR merge) to the -ci-jenkins-pipelines repository - +ci-jenkins-pipelines repository. They will pause themselves if a pipeline is +not running so as to avoid interfering with a currently executing pipeline. +T Similarly there is an evaluation-pipeline-generator and evaluation-pipeline_jobs_generator_jdkXX for generating the evaluation jobs, a trestle-pipeline-generator for those jobs, plus release-pipeline-generator @@ -179,15 +193,19 @@ andand release_pipeline_jobs_generator_jdkXX for release jobs (the release generators are not triggered automatically but are re-run manually at certain points during each release cycle -The generators make use of files in -[pipelines/jobs/configurations](https://github.com/adoptium/ci-jenkins-pipelines/tree/master/pipelines/jobs/configurations) -and you can see more details of the format in the README.md in that location: +The following is a brief summary of how the generation jobs work but more +details can be found in the +[generator documentation](pipelines/build/regeneration/README.md) + +The generators make use of the following files in +[pipelines/jobs/configurations](pipelines/jobs/configurations). The README +file in that directory contains more details of the configuration format: + - The `jdkXX.groovy`, `jdkXX_evaluation.groovy`, `jdkXX_release.groovy` to determine which platforms to configure and generate for each version. - The individual platform configurations, such as jenkins labels, are defined by `jdkXX_pipeline_config.groovy` files. For more details on the regeneration process overall see the -[regeneration documentation](https://github.com/adoptium/ci-jenkins-pipelines/blob/master/pipelines/build/regeneration/README.md) -and for more detail on the configuration files see this readme +[regeneration documentation](pipelines/build/regeneration/README.md) ## Metadata files generated with each build diff --git a/pipelines/jobs/configurations/README.md b/pipelines/jobs/configurations/README.md index 5757944a0..fee8431d1 100644 --- a/pipelines/jobs/configurations/README.md +++ b/pipelines/jobs/configurations/README.md @@ -1,8 +1,7 @@ -*Note: The contents of this file was split out from the top level README.md -in this repository* - ## Configuration Files used by the build job generators +*Note: The contents of this file was split out from the top level README.md in this repository* + The [pipelines/jobs/configurations](pipelines/jobs/configurations) directory contains two categories of configuration files that our jenkins pipelines use (Nicknamed [#Build Configs](#build) and [#Nightly Configs](#nightly) for short). To ensure both configurations are not overridden in a race condition scenario by another job, the [job generators](../../build/regeneration/README.md) ensure they remain in the sync with the repository.