From 82ec96f815f5cfc4bed9e54075f35568f6e852ca Mon Sep 17 00:00:00 2001 From: John Oliver Date: Thu, 7 Jun 2018 17:56:17 +0100 Subject: [PATCH 001/101] Bring new_build_scripts branch up to date with master (#365) * Add openjdktest build on Windows (#322) Also update to enable capability of test build triggered per test type. Signed-off-by: Sophia Guo * Add triggers for openjdk9-j9/hs and openjdk10-j9/hs for External Tests Signed-off-by: Mesbah * update release pipeline (#337) * extra fixes to release pipeline (#338) * update release pipeline * extra fixes to build release pipeline * Update openjdk8_release_pipeline.groovy * Add git-hg/updateforesttags.sh (#340) * Add git-hg/updatetags8. Initial version hard coded with tags and pauses for manual merging * Miscellanous updates to tidy up the 'work in progress' version :-) * Second round of updates to tidy up the script * Rename updatetags8.sh to updateforesttags.sh * Add arm64 jdk10 to pipeline, add Win testing to release pipelines Signed-off-by: smlambert * Attempted fix to Travis config for shellcheck (#344) * use correct SAP branch * Docker build doc improvement * formatting * Upate READMEs to reflect broken status of Docker builds * Attempt to fix Travis CI shellcheck install failure * Initial Shellcheck fixes * More shellcheck fixes * More shellcheck fixes for unrelated scripts * More shellcheck fixes * More shellcheck fixes * More shellcheck fixes * Tidy from review * Fix jdk9u Mercurial repo name (#348) * change git tag command * Initial cut of new git-hg script for java 10+ (#349) * use correct SAP branch * Docker build doc improvement * formatting * Upate READMEs to reflect broken status of Docker builds * Attempt to fix Travis CI shellcheck install failure * Initial Shellcheck fixes * More shellcheck fixes * More shellcheck fixes for unrelated scripts * More shellcheck fixes * More shellcheck fixes * More shellcheck fixes * Tidy from review * Initial cut of new git-hg script for java 10+ * minor fixes from review * pass parameters through to function * skip git version check for mirror script * move git checkout before the fetch * only reset if it's not empty * turn pipefail off because we're awesome * turn pipefail back on because we're NOT awesome * Remove bad flag and get compression right * It's all our fault, let us try fetching first * change merge command * add else case to git reset * add brackets to add or * More shellcheck fixes * More shellcheck fixes * More shellcheck fixes * Add dev branch merge capability --- .travis.yml | 7 +- git-hg/add-branch-without-modules.sh | 33 ++-- git-hg/diff-without-getsource.sh | 2 +- git-hg/import-common.sh | 30 ++++ git-hg/mirrorMercurialIntoGit.sh | 102 ++++++++++++ git-hg/setup10.sh | 24 --- git-hg/setup8.sh | 12 -- git-hg/update-without-modules.sh | 14 +- git-hg/updateforesttags.sh | 151 ++++++++++++++++++ pipelines/openjdk10_nightly_pipeline.groovy | 16 +- .../openjdk10_openj9_nightly_pipeline.groovy | 10 +- pipelines/openjdk8_nightly_pipeline.groovy | 2 +- .../openjdk8_openj9_nightly_pipeline.groovy | 2 +- .../openjdk8_openj9_release_pipeline.groovy | 11 +- pipelines/openjdk8_release_pipeline.groovy | 24 +-- pipelines/openjdk9_nightly_pipeline.groovy | 4 +- .../openjdk9_openj9_nightly_pipeline.groovy | 11 +- sbin/build.sh | 3 + 18 files changed, 352 insertions(+), 106 deletions(-) create mode 100755 git-hg/mirrorMercurialIntoGit.sh delete mode 100644 git-hg/setup10.sh create mode 100755 git-hg/updateforesttags.sh diff --git a/.travis.yml b/.travis.yml index 117b3f5f7..d49b8ca4e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -7,8 +7,11 @@ addons: apt: sources: - debian-sid # Grab shellcheck from the Debian repo (o_O) - packages: - - shellcheck + +before_install: + - sudo apt-get install -y cabal-install shunit2 + - cabal update + - cabal install shellcheck script: - bash -c 'shopt -s globstar; shellcheck **/*.sh' diff --git a/git-hg/add-branch-without-modules.sh b/git-hg/add-branch-without-modules.sh index 6c4b3d5ca..2841b4e56 100755 --- a/git-hg/add-branch-without-modules.sh +++ b/git-hg/add-branch-without-modules.sh @@ -13,56 +13,55 @@ # limitations under the License. # -set -euo pipefail +################################################################################ +# add-branch-without-modules +# +# Local setup of the Git clone branches for the OpenJDK10 (updates) mercurial +# repository +# +# Initial repo will be pushed to git@github.com:AdoptOpenJDK/openjdk-jdk10u.git +# +################################################################################ -echo "Common defs" +set -euo pipefail +echo "Import common functionality" # shellcheck disable=SC1091 source import-common.sh echo "Enter hg" - cd hg || exit 1 bpath=$1 branch=$2 echo "Create $bpath" - mkdir -p "$bpath" || exit 1 -echo "Clone $bpath (root)" +echo "git hg clone $bpath (root)" git hg clone "http://hg.openjdk.java.net/$bpath" "$bpath/root" || exit 1 +echo "checkout the $branch" if [ "$branch" != "" ]; then cd "$bpath/root" || exit 1 git-hg checkout "$branch" || exit 1 cd - || exit 1 fi -echo "Exit hg" -echo "Enter combined" - +echo "Enter ../combined" cd ../combined || exit 1 -echo "Branch $bpath" - -pwd - +echo "checkout the master branch" git checkout -b master || exit 1 echo "Add remote for (root)" - git remote add "imports/$bpath/root" "../hg/$bpath/root" || exit 1 echo "Fetch (root)" - git fetch "imports/$bpath/root" || exit 1 echo "Merge (root)" - git merge "imports/$bpath/root/master" -m "Initial merge of (root)" || exit 1 -echo "Push" - +echo "Push the tags to the master branch" git push github master --tags || exit 1 diff --git a/git-hg/diff-without-getsource.sh b/git-hg/diff-without-getsource.sh index 5053a986c..882b1fdde 100755 --- a/git-hg/diff-without-getsource.sh +++ b/git-hg/diff-without-getsource.sh @@ -39,7 +39,7 @@ fi # get latest git tag cd openjdk-git || exit 1 -gitTag=$(git describe --abbrev=0 --tags) || exit 1 +gitTag=$(git describe --tags "$(git rev-list --tags --max-count=1)") || exit 1 cd - || exit 1 cd openjdk-hg || exit 1 diff --git a/git-hg/import-common.sh b/git-hg/import-common.sh index 3f77bdd9a..bb05f0611 100755 --- a/git-hg/import-common.sh +++ b/git-hg/import-common.sh @@ -13,6 +13,36 @@ # limitations under the License. # +################################################################################ # import-common +# +# common functionality for the git-hg scripts +# +################################################################################ + cd "$WORKSPACE" || exit 1 export modules=(corba langtools jaxp jaxws hotspot nashorn jdk) + +function checkGitVersion() { + git --version || exit 1 + GIT_VERSION=$(git --version | awk '{print$NF}') + GIT_MAJOR_VERSION=$(echo "$GIT_VERSION" | cut -d. -f1) + GIT_MINOR_VERSION=$(echo "$GIT_VERSION" | cut -d. -f2) + [ "$GIT_MAJOR_VERSION" -eq 1 ] && echo I need git version 2.16 or later and you have "$GIT_VERSION" && exit 1 + [ "$GIT_MAJOR_VERSION" -eq 2 ] && [ "$GIT_MINOR_VERSION" -lt 16 ] && echo I need git version 2.16 or later and you have "$GIT_VERSION" && exit 1 +} + +function installGitRemoteHg() { + if ! which git-remote-hg 2>/dev/null; then + echo "I need git-remote-hg and could not find it" + echo "Getting it from https://raw.githubusercontent.com/felipec/git-remote-hg/master/git-remote-hg" + mkdir -p "$WORKSPACE/bin" + PATH="$PATH:$WORKSPACE/bin" + wget -O "$WORKSPACE/bin/git-remote-hg https://raw.githubusercontent.com/felipec/git-remote-hg/master/git-remote-hg" + chmod ugo+x "$WORKSPACE/bin/git-remote-hg" + if ! which git-remote-hg 2>/dev/null; then + echo "Still cannot find it, exiting.." + exit 1 + fi + fi +} diff --git a/git-hg/mirrorMercurialIntoGit.sh b/git-hg/mirrorMercurialIntoGit.sh new file mode 100755 index 000000000..ce49213ed --- /dev/null +++ b/git-hg/mirrorMercurialIntoGit.sh @@ -0,0 +1,102 @@ +#!/bin/bash +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +################################################################################ +# mirrorMercurialIntoGit +# +# Local setup of the Git clone of a OpenJDK mercurial repository +# +# Initial repo will be pushed to git@github.com:AdoptOpenJDK/openjdk-$GITHUB_REPO.git +# +# TODO Make the location of the git push a parameter +# +################################################################################ + +set -euo pipefail + +echo "Import common functionality" +# shellcheck disable=SC1091 +source import-common.sh + +# Make sure we're in a valid dir as a workspace +[ -z "$WORKSPACE" ] || WORKSPACE=$PWD +echo "You are in $WORKSPACE" + +# TODO generalise this for the non adopt build farm case +function checkArgs() { + if [ "$1" -lt 1 ]; then + echo Usage: "$0" '[jdk-updates/jdk10u|jdk/jdk] (branch)' + echo "Hg Repo supplied should match a repository in http://hg.openjdk.java.net/" + echo "For example, to get the latest jdk development repo:" + echo "$0 jdk/jdk" + echo "For example, to get the raw-string-literals branch from the amber repo:" + echo "e.g. $0 amber/jdk raw-string-literals" + exit 1 + fi +} + +# Read in the mandatory Mercurial repo, e.g. jdk-updates/jdk10u +# Default to a master branch if one is not given +HG_REPO=$1 +GITHUB_REPO=$(echo "$HG_REPO" | cut -d/ -f2) +BRANCH=${2:-master} + +function cloneGitHubRepo() { + # If we don't have a $GITHUB_REPO locally then clone it from AdoptOpenJDK/openjdk-$GITHUB_REPO.git + if [ ! -d "$GITHUB_REPO" ] ; then + git clone git@github.com:AdoptOpenJDK/openjdk-"$GITHUB_REPO".git "$GITHUB_REPO" || exit 1 + fi +} + +function addMercurialUpstream() { + cd "$GITHUB_REPO" || exit 1 + + git fetch origin + if ! git checkout "$BRANCH" ; then + git checkout -b "$BRANCH" || exit 1 + else + git reset --hard origin/"$BRANCH" || echo "Not resetting as no upstream exists" + fi + + # shellcheck disable=SC2143 + if [ -z "$(git remote -v | grep 'hg')" ] ; then + echo "Initial setup of hg::http://hg.openjdk.java.net/$HG_REPO" + git remote add hg hg::http://hg.openjdk.java.net/"$HG_REPO" + fi +} + +function performMergeFromMercurialIntoGit() { + git fetch hg + git merge hg/"$BRANCH" -m "Merge $BRANCH" || (echo "The automatic update failed, time for manual intervention!" && exit 1) + git gc --auto + git push origin "$BRANCH" --tags || exit 1 +} + +# Merge master into dev as we build off dev at the AdoptOpenJDK Build farm +# dev contains patches that AdoptOpenJDK has beyond upstream OpenJDK +function performMergeIntoDevFromMaster() { + git checkout dev || git checkout -b dev + git rebase master || exit 1 + git push origin dev || exit 1 +} + +# TODO Need to cover always merging mercurial master into our GitHub dev branch +checkArgs $# +#checkGitVersion +installGitRemoteHg +cloneGitHubRepo +addMercurialUpstream +performMergeFromMercurialIntoGit +performMergeIntoDevFromMaster \ No newline at end of file diff --git a/git-hg/setup10.sh b/git-hg/setup10.sh deleted file mode 100644 index aa8e51081..000000000 --- a/git-hg/setup10.sh +++ /dev/null @@ -1,24 +0,0 @@ -#!/bin/bash -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -set -euo pipefail -rm -rf "$WORKSPACE/combined" "$WORKSPACE/hg" -mkdir "$WORKSPACE/combined" -mkdir "$WORKSPACE/hg" -cd "$WORKSPACE/combined" || exit 1 -git init -git checkout -b root-commit || exit 1 -git remote add github git@github.com:AdoptOpenJDK/openjdk-jdk10.git -cd - || exit 1 -bash add-branch-without-modules.sh jdk/jdk10 diff --git a/git-hg/setup8.sh b/git-hg/setup8.sh index a9c03a699..7d5f08e80 100755 --- a/git-hg/setup8.sh +++ b/git-hg/setup8.sh @@ -11,18 +11,6 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -## -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. # set -euo pipefail rm -rf "$WORKSPACE/combined" "$WORKSPACE/hg" diff --git a/git-hg/update-without-modules.sh b/git-hg/update-without-modules.sh index b6d0daa58..d335540e1 100755 --- a/git-hg/update-without-modules.sh +++ b/git-hg/update-without-modules.sh @@ -13,13 +13,12 @@ # limitations under the License. # -echo "Common defs" +echo "Import common functionality" # shellcheck disable=SC1091 source import-common.sh echo "Enter hg" - cd hg || exit 1 # shellcheck disable=SC2035 @@ -38,25 +37,22 @@ do popd echo "Exit hg" - echo "Enter combined" + echo "Enter ../combined" cd ../combined || exit 1 echo "Check out master" - git checkout master || exit 1 - echo "Fetch (root)" - + echo "Fetch (root) of the mercurial forest/repo" git fetch "imports/$hg_root_forest/$hg_repo/root" || exit 1 echo "Merge (root)" - git merge "imports/$hg_root_forest/$hg_repo/root/master" -m "Merge from (root)" --no-ff || exit 1 - echo "Push" - + echo "Push to GitHub" git push github master --tags + echo "Exit back to ../hg" cd ../hg || exit 1 done diff --git a/git-hg/updateforesttags.sh b/git-hg/updateforesttags.sh new file mode 100755 index 000000000..1e4daded4 --- /dev/null +++ b/git-hg/updateforesttags.sh @@ -0,0 +1,151 @@ +#!/bin/bash +# +# =========================================================================== +# (c) Copyright IBM Corp. 2018 All Rights Reserved +# =========================================================================== +# +# This code is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License version 2 only, as +# published by the Free Software Foundation. +# +# This code is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License +# version 2 for more details (a copy is included in the LICENSE file that +# accompanied this code). +# +# You should have received a copy of the GNU General Public License version +# 2 along with this work; if not, see . +# +# =========================================================================== + +if [ $# -lt 1 ]; then + echo Usage: "$0" '[jdk8u|jdk9] (TAGS)' + echo "Version supplied should match a repository in AdoptOpenJDK/openjdk-VERSION" + exit 1 +fi +[ -z "$WORKSPACE" ] || [ ! -d "$WORKSPACE" ] && echo Cannot access \$WORKSPACE: "$WORKSPACE" && exit 2 + +MODULES=(corba langtools jaxp jaxws nashorn jdk hotspot) + +OPENJDK_VERSION="$1" +shift +# shellcheck disable=SC2124 +TAGS="$@" +GITHUB_PROJECT=git@github.com:AdoptOpenJDK +GITHUB_REPO="openjdk-$OPENJDK_VERSION" + +case "$OPENJDK_VERSION" in + jdk8*) HG_REPO=http://hg.openjdk.java.net/jdk8u/jdk8u + [ -z "$TAGS" ] && TAGS="jdk8u144-b34 jdk8u151-b12 jdk8u152-b16 jdk8u161-b12 jdk8u162-b12 jdk8u172-b03 jdk8u172-b11";; + jdk9*) HG_REPO=http://hg.openjdk.java.net/jdk-updates/jdk9u + [ -z "$TAGS" ] && TAGS="jdk-9+181 jdk-9.0.1+11 jdk-9.0.3+9 jdk-9.0.4+11";; + *) Unknown JDK version - only jdk8u and jdk9 are supported; exit 1;; +esac + +# Clean up +rm -rf "${WORKSPACE:?}/$GITHUB_REPO" "${WORKSPACE:?}/openjdk" +mkdir -p "$WORKSPACE/$GITHUB_REPO" "$WORKSPACE/openjdk/mirror" + +git --version || exit 1 +GIT_VERSION=$(git --version | awk '{print$NF}') +GIT_MAJOR_VERSION=$(echo "$GIT_VERSION" | cut -d. -f1) +GIT_MINOR_VERSION=$(echo "$GIT_VERSION" | cut -d. -f2) +[ "$GIT_MAJOR_VERSION" -eq 1 ] && echo I need git version 2.16 or later and you have "$GIT_VERSION" && exit 1 +[ "$GIT_MAJOR_VERSION" -eq 2 ] && [ "$GIT_MINOR_VERSION" -lt 16 ] && echo I need git version 2.16 or later and you have "$GIT_VERSION" && exit 1 + +if ! which git-remote-hg 2>/dev/null; then + echo "I need git-remote-hg and could not find it" + echo "Get it from http://raw.githubusercontent.com/felipec/git-remote-hg/master/git-remote-hg" + exit 1 +fi + +# Clone current AdoptOpenJDK repo +cd "$WORKSPACE/$GITHUB_REPO" || exit 1 +echo "Clone current $GITHUB_REPO" +git clone "$GITHUB_PROJECT/$GITHUB_REPO.git" +cd "$GITHUB_REPO" || exit +git fetch --tags +OLDTAG=$(git describe --abbrev=0 --tags) +echo "Current openjdk level is $OLDTAG" + +# Clone current openjdk from Mercurial +echo "Get base openjdk repository" +cd "$WORKSPACE/openjdk/mirror" || exit 1 +git init +git clone --bare "hg::${HG_REPO}" + +cd "$OPENJDK_VERSION.git" || exit 1 + +git filter-branch -f --index-filter 'git rm -r -f -q --cached --ignore-unmatch .hg .hgignore .hgtags get_source.sh' --prune-empty --tag-name-filter cat -- --all + +cd .. || exit +git pull "$OPENJDK_VERSION" +git fetch --tags "$OPENJDK_VERSION" +for NEWTAG in $TAGS ; do + cd "$WORKSPACE/openjdk/mirror" || exit 1 + git reset --hard "$NEWTAG" + + echo "$(date +%T)": "Updating master branch for $NEWTAG" + cd "$WORKSPACE/$GITHUB_REPO/$GITHUB_REPO" || exit 1 + git checkout master + git fetch "$WORKSPACE/openjdk/mirror" + git merge --allow-unrelated-histories -m "Merge base $NEWTAG" FETCH_HEAD + + for module in "${MODULES[@]}" + do + if [ ! -d "$WORKSPACE/openjdk/$module" ]; then + rm -rf "$WORKSPACE/openjdk/$module" + mkdir "$WORKSPACE/openjdk/$module" + cd "$WORKSPACE/openjdk/$module" || exit 1 + git init + echo "$(date +%T)": "Clone $module" + git clone --bare "hg::${HG_REPO}/$module" || exit 1 + echo "$(date +%T)": "GIT filter on $module" + cd "$module.git" || exit + git filter-branch -f --index-filter "git rm -f -q --cached --ignore-unmatch .hgignore .hgtags && git ls-files -s | sed \"s|\t\\\"*|&$module/|\" | GIT_INDEX_FILE=\$GIT_INDEX_FILE.new git update-index --index-info && mv \"\$GIT_INDEX_FILE.new\" \"\$GIT_INDEX_FILE\"" --prune-empty --tag-name-filter cat -- --all + cd .. || exit + fi + echo "$(date +%T)": "GIT pull/reset on $module at $NEWTAG" + cd "$WORKSPACE/openjdk/$module" || exit 1 + git pull "$module" + git fetch --tags "$module" + git reset --hard "$NEWTAG" + cd "$WORKSPACE/$GITHUB_REPO/$GITHUB_REPO" || exit 1 + git fetch "$WORKSPACE/openjdk/$module" + echo "$(date +%T)": GIT filter on "$module" + if ! git merge --allow-unrelated-histories -m "Merge $module at $NEWTAG" FETCH_HEAD; then + if ! tty; then + echo "Aborting - not running on a real tty therefore cannot allow manual intervention" + exit 10 + else + echo "Please resolve them in another window then press return to continue" + read -r _ + fi + echo Please resolve the conflicts above in "$WORKSPACE/$GITHUB_REPO/$GITHUB_REPO", and press return to continue + read -r _ + fi + done + cd "$WORKSPACE/$GITHUB_REPO/$GITHUB_REPO" || exit 1 + git push origin master + + echo "Pulling in changes to $GITHUB_REPO branch" + git checkout master + git fetch origin master + if ! git merge --allow-unrelated-histories -m "Merge $NEWTAG into $GITHUB_REPO" FETCH_HEAD; then + echo Conflict resolution needed in "$WORKSPACE/$GITHUB_REPO/$GITHUB_REPO" + if ! tty; then + echo "Aborting - not running on a real tty therefore cannot allow manual intervention" + exit 10 + else + echo "Please resolve them in another window then press return to continue" + read -r _ + fi + fi + [ "$NEWTAG" != "HEAD" ] && git tag -f -a "$NEWTAG" -m "Merge $NEWTAG into master" + echo Deleting the old version of the tag from the server if it is present or push will fail + # shellcheck disable=SC2015 + [ "$NEWTAG" != "HEAD" ] && git push origin :refs/tags/"$NEWTAG" || true + [ "$NEWTAG" == "HEAD" ] && git push origin master + [ "$NEWTAG" != "HEAD" ] && git push origin master --tags +done diff --git a/pipelines/openjdk10_nightly_pipeline.groovy b/pipelines/openjdk10_nightly_pipeline.groovy index 448ea6f7a..90d87393c 100644 --- a/pipelines/openjdk10_nightly_pipeline.groovy +++ b/pipelines/openjdk10_nightly_pipeline.groovy @@ -1,14 +1,14 @@ println "building ${JDK_VERSION}" -def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX'] +def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'arm64'] def buildMaps = [:] -buildMaps['Mac'] = [test:true, ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:false, ArchOSs:'x86-64_windows'] -buildMaps['Linux'] = [test:true, ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:true, ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:true, ArchOSs:'ppc64le_linux'] +buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] +buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] +buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] +buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] +buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -def typeTests = ['openjdktest', 'systemtest'] +buildMaps['arm64'] = [test:['openjdktest'], ArchOSs:'arm64_linux'] def jobs = [:] for ( int i = 0; i < buildPlatforms.size(); i++ ) { @@ -25,7 +25,7 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { } if (buildMaps[platform].test) { stage('test') { - typeTests.each { + buildMaps[platform].test.each { build job:"openjdk10_hs_${it}_${archOS}", propagate: false, parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), diff --git a/pipelines/openjdk10_openj9_nightly_pipeline.groovy b/pipelines/openjdk10_openj9_nightly_pipeline.groovy index 45637ac14..e0efe4b39 100644 --- a/pipelines/openjdk10_openj9_nightly_pipeline.groovy +++ b/pipelines/openjdk10_openj9_nightly_pipeline.groovy @@ -2,11 +2,11 @@ println "building ${JDK_VERSION}" def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] def buildMaps = [:] -buildMaps['Linux'] = [test:true, ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:true, ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:true, ArchOSs:'ppc64le_linux'] +buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] +buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] +buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:false, ArchOSs:'x86-64_windows'] +buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] def typeTests = ['openjdktest', 'systemtest'] def jobs = [:] @@ -24,7 +24,7 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { } if (buildMaps[platform].test) { stage('test') { - typeTests.each { + buildMaps[platform].test.each { build job:"openjdk10_j9_${it}_${archOS}", propagate: false, parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), diff --git a/pipelines/openjdk8_nightly_pipeline.groovy b/pipelines/openjdk8_nightly_pipeline.groovy index 47d97d230..ca47e496e 100644 --- a/pipelines/openjdk8_nightly_pipeline.groovy +++ b/pipelines/openjdk8_nightly_pipeline.groovy @@ -3,7 +3,7 @@ println "building ${JDK_VERSION}" def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX'] def buildMaps = [:] buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:false, ArchOSs:'x86-64_windows'] +buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'perftest', 'externaltest'], ArchOSs:'x86-64_linux'] buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] diff --git a/pipelines/openjdk8_openj9_nightly_pipeline.groovy b/pipelines/openjdk8_openj9_nightly_pipeline.groovy index 2d83680d2..216561b26 100644 --- a/pipelines/openjdk8_openj9_nightly_pipeline.groovy +++ b/pipelines/openjdk8_openj9_nightly_pipeline.groovy @@ -6,7 +6,7 @@ buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'perftest', 'externalte buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:false, ArchOSs:'x86-64_windows'] +buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] def jobs = [:] for ( int i = 0; i < buildPlatforms.size(); i++ ) { diff --git a/pipelines/openjdk8_openj9_release_pipeline.groovy b/pipelines/openjdk8_openj9_release_pipeline.groovy index 3416a3330..62ab3094a 100644 --- a/pipelines/openjdk8_openj9_release_pipeline.groovy +++ b/pipelines/openjdk8_openj9_release_pipeline.groovy @@ -2,12 +2,11 @@ println "building ${JDK_VERSION}" def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', 'Windows'] def buildMaps = [:] -buildMaps['Linux'] = [test:true, ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:true, ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:true, ArchOSs:'ppc64le_linux'] +buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] +buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] +buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:false, ArchOSs:'x86-64_windows'] -def typeTests = ['openjdktest', 'systemtest'] +buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] def jobs = [:] for ( int i = 0; i < buildPlatforms.size(); i++ ) { @@ -21,7 +20,7 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { } if (buildMaps[platform].test) { stage('test') { - typeTests.each { + buildMaps[platform].test.each { build job:"openjdk8_j9_${it}_${archOS}", propagate: false, parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJob.getNumber()}"), diff --git a/pipelines/openjdk8_release_pipeline.groovy b/pipelines/openjdk8_release_pipeline.groovy index 747e0b507..e41f5f7f1 100644 --- a/pipelines/openjdk8_release_pipeline.groovy +++ b/pipelines/openjdk8_release_pipeline.groovy @@ -1,14 +1,14 @@ println "building ${JDK_VERSION}" -def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX'] +def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'arm64'] def buildMaps = [:] -buildMaps['Mac'] = [test:true, ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:false, ArchOSs:'x86-64_windows'] -buildMaps['Linux'] = [test:true, ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:true, ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:true, ArchOSs:'ppc64le_linux'] +buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] +buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] +buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] +buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] +buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -def typeTests = ['openjdktest', 'systemtest'] +buildMaps['arm64'] = [test:['openjdktest'], ArchOSs:'arm64_linux'] def jobs = [:] for ( int i = 0; i < buildPlatforms.size(); i++ ) { @@ -18,10 +18,11 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { jobs[platform] = { def buildJob stage('build') { - buildJob = build job: "openjdk8_build_${archOS}" + buildJob = build job: "openjdk8_build_${archOS}", + parameters: [string(name: 'TAG', value: "${JDK_TAG}")] } if (buildMaps[platform].test) { - stage('test') { + buildMaps[platform].test.each { typeTests.each { build job:"openjdk8_hs_${it}_${archOS}", propagate: false, @@ -36,7 +37,6 @@ parallel jobs def checksumJob stage('checksums') { - build job: 'openjdk8_build_checksum' checksumJob = build job: 'openjdk8_build_checksum', parameters: [string(name: 'PRODUCT', value: 'releases')] } @@ -44,8 +44,8 @@ stage('installers') { build job: 'openjdk8_build_installer', parameters: [string(name: 'VERSION', value: "${JDK_VERSION}")] } stage('publish release') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'release'), + build job: 'openjdk_release_tool', + parameters: [string(name: 'REPO', value: 'releases'), string(name: 'TAG', value: "${JDK_TAG}"), string(name: 'VERSION', value: 'jdk8'), string(name: 'CHECKSUM_JOB_NAME', value: "openjdk8_build_checksum"), diff --git a/pipelines/openjdk9_nightly_pipeline.groovy b/pipelines/openjdk9_nightly_pipeline.groovy index 841fabd1f..d999be775 100644 --- a/pipelines/openjdk9_nightly_pipeline.groovy +++ b/pipelines/openjdk9_nightly_pipeline.groovy @@ -3,8 +3,8 @@ println "building ${JDK_VERSION}" def buildPlatforms = ['Mac', 'Linux', 'zLinux', 'ppc64le', 'Windows', 'AIX', 'arm64'] def buildMaps = [:] buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:false, ArchOSs:'x86-64_windows'] -buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] +buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] +buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] diff --git a/pipelines/openjdk9_openj9_nightly_pipeline.groovy b/pipelines/openjdk9_openj9_nightly_pipeline.groovy index 563f85d95..756fee5fd 100644 --- a/pipelines/openjdk9_openj9_nightly_pipeline.groovy +++ b/pipelines/openjdk9_openj9_nightly_pipeline.groovy @@ -2,12 +2,11 @@ println "building ${JDK_VERSION}" def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] def buildMaps = [:] -buildMaps['Linux'] = [test:true, ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:true, ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:true, ArchOSs:'ppc64le_linux'] +buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] +buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] +buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:false, ArchOSs:'x86-64_windows'] -def typeTests = ['openjdktest', 'systemtest'] +buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] def jobs = [:] for ( int i = 0; i < buildPlatforms.size(); i++ ) { @@ -24,7 +23,7 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { } if (buildMaps[platform].test) { stage('test') { - typeTests.each { + buildMaps[platform].test.each { build job:"openjdk9_j9_${it}_${archOS}", propagate: false, parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), diff --git a/sbin/build.sh b/sbin/build.sh index 6f3ed8591..2f56d6b07 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -316,6 +316,7 @@ printJavaVersionString() PRODUCT_HOME=$(ls -d $OPENJDK_DIR/build/*/images/${JDK_PATH}) if [[ -d "$PRODUCT_HOME" ]]; then echo "${good}'$PRODUCT_HOME' found${normal}" + # shellcheck disable=SC2154 echo "${info}" if ! "$PRODUCT_HOME"/bin/java -version; then echo "${error} Error executing 'java' does not exist in '$PRODUCT_HOME'.${normal}" @@ -418,6 +419,8 @@ signRelease() "darwin"*) echo "Signing OSX release" # Login to KeyChain + # shellcheck disable=SC2046 + # shellcheck disable=SC2006 security unlock-keychain -p `cat ~/.password` # Sign all files with the executable permission bit set. FILES=$(find "${OPENJDK_REPO_TAG}" -perm +111 -type f || find "${OPENJDK_REPO_TAG}" -perm /111 -type f) From 5f2309942d768900551f2e88987a46890473f409 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 3 Jul 2018 16:48:37 +0100 Subject: [PATCH 002/101] Bring new_build_scripts up to date with master (#392) * Add openjdktest build on Windows (#322) Also update to enable capability of test build triggered per test type. Signed-off-by: Sophia Guo * Add triggers for openjdk9-j9/hs and openjdk10-j9/hs for External Tests Signed-off-by: Mesbah * update release pipeline (#337) * extra fixes to release pipeline (#338) * update release pipeline * extra fixes to build release pipeline * Update openjdk8_release_pipeline.groovy * Add git-hg/updateforesttags.sh (#340) * Add git-hg/updatetags8. Initial version hard coded with tags and pauses for manual merging * Miscellanous updates to tidy up the 'work in progress' version :-) * Second round of updates to tidy up the script * Rename updatetags8.sh to updateforesttags.sh * Add arm64 jdk10 to pipeline, add Win testing to release pipelines Signed-off-by: smlambert * Attempted fix to Travis config for shellcheck (#344) * use correct SAP branch * Docker build doc improvement * formatting * Upate READMEs to reflect broken status of Docker builds * Attempt to fix Travis CI shellcheck install failure * Initial Shellcheck fixes * More shellcheck fixes * More shellcheck fixes for unrelated scripts * More shellcheck fixes * More shellcheck fixes * More shellcheck fixes * Tidy from review * Fix jdk9u Mercurial repo name (#348) * change git tag command * Initial cut of new git-hg script for java 10+ (#349) * use correct SAP branch * Docker build doc improvement * formatting * Upate READMEs to reflect broken status of Docker builds * Attempt to fix Travis CI shellcheck install failure * Initial Shellcheck fixes * More shellcheck fixes * More shellcheck fixes for unrelated scripts * More shellcheck fixes * More shellcheck fixes * More shellcheck fixes * Tidy from review * Initial cut of new git-hg script for java 10+ * minor fixes from review * pass parameters through to function * skip git version check for mirror script * move git checkout before the fetch * only reset if it's not empty * turn pipefail off because we're awesome * turn pipefail back on because we're NOT awesome * Remove bad flag and get compression right * It's all our fault, let us try fetching first * change merge command * add else case to git reset * add brackets to add or * More shellcheck fixes * More shellcheck fixes * More shellcheck fixes * Add dev branch merge capability * Update release tags for AO8/J9 and AO9/J9 * Update release tags for AO8/J9 and AO9/J9 (#352) * Add Win32 & x64_linux_largeHeap (#355) * Add Win32 & LinuxLargeHeap Signed-off-by: smlambert * Fix typo Signed-off-by: smlambert * add force flag to git checkout for git-hg mirroring prevents untracked files from failing the job * remove git gc command from mirror scripts (#357) * Ensure tag from the source repo instead of oepnjdk-build is used (#360) * WIP: Update arm64 name to be aarch64 (#368) * Add Win32 & LinuxLargeHeap Signed-off-by: smlambert * Fix typo Signed-off-by: smlambert * Change arm64 to aarch64 Signed-off-by: smlambert * Update ArchOSs Signed-off-by: smlambert * No change to openjdk8_nightly for now Signed-off-by: smlambert * Add openjdk8 Signed-off-by: smlambert * Use escape backslash for ABS_PLATFORM_BUILD_ROOT Enable jcov in java8 Signed-off-by: Sophia Guo * Enable jcov for java8 Signed-off-by: Sophia Guo * Allow overriding of openj9 branch name * Fix negated logic in last commit * Pipelines allow alternate branch (#379) * Add ALT_BRANCH parameter to openj9 pipelines * Add ALT_BRANCH parameter to non-openj9 pipelines for consistency * Add timestamps to top-level pipelines (#382) * Add timestamps to top-level pipelines Signed-off-by: smlambert * Update to use yyMMdd, not yyddMM Signed-off-by: smlambert * jcov8 (#383) Signed-off-by: Sophia Guo * add jdk10 release pipeline (#385) * Fix empty string PIPELINE_TIMESTAMP issue Signed-off-by: Sophia Guo * Remove undefined variable ALT_BRANCH (#391) Signed-off-by: Sophia Guo --- git-hg/mirrorMercurialIntoGit.sh | 5 +- makejdk.sh | 6 +- pipelines/openjdk10_nightly_pipeline.groovy | 9 ++- .../openjdk10_openj9_nightly_pipeline.groovy | 6 +- pipelines/openjdk10_release_pipeline.groovy | 56 +++++++++++++++++++ pipelines/openjdk8_nightly_pipeline.groovy | 8 ++- .../openjdk8_openj9_nightly_pipeline.groovy | 12 +++- .../openjdk8_openj9_release_pipeline.groovy | 6 +- pipelines/openjdk8_release_pipeline.groovy | 9 ++- pipelines/openjdk9_nightly_pipeline.groovy | 10 +++- .../openjdk9_openj9_nightly_pipeline.groovy | 8 ++- .../openjdk_amber_nightly_pipeline.groovy | 5 +- sbin/build.sh | 1 + sbin/jtreg.sh | 6 +- 14 files changed, 119 insertions(+), 28 deletions(-) create mode 100644 pipelines/openjdk10_release_pipeline.groovy diff --git a/git-hg/mirrorMercurialIntoGit.sh b/git-hg/mirrorMercurialIntoGit.sh index ce49213ed..91220990c 100755 --- a/git-hg/mirrorMercurialIntoGit.sh +++ b/git-hg/mirrorMercurialIntoGit.sh @@ -64,7 +64,7 @@ function addMercurialUpstream() { cd "$GITHUB_REPO" || exit 1 git fetch origin - if ! git checkout "$BRANCH" ; then + if ! git checkout -f "$BRANCH" ; then git checkout -b "$BRANCH" || exit 1 else git reset --hard origin/"$BRANCH" || echo "Not resetting as no upstream exists" @@ -80,7 +80,6 @@ function addMercurialUpstream() { function performMergeFromMercurialIntoGit() { git fetch hg git merge hg/"$BRANCH" -m "Merge $BRANCH" || (echo "The automatic update failed, time for manual intervention!" && exit 1) - git gc --auto git push origin "$BRANCH" --tags || exit 1 } @@ -99,4 +98,4 @@ installGitRemoteHg cloneGitHubRepo addMercurialUpstream performMergeFromMercurialIntoGit -performMergeIntoDevFromMaster \ No newline at end of file +performMergeIntoDevFromMaster diff --git a/makejdk.sh b/makejdk.sh index 311e0c6de..b6c4a63e0 100755 --- a/makejdk.sh +++ b/makejdk.sh @@ -141,14 +141,14 @@ parseCommandLineArgs() doAnyBuildVariantOverrides() { if [[ "${BUILD_VARIANT}" == "openj9" ]]; then - # current (hoping not final) location of Extensions for OpenJDK9 for OpenJ9 project + # current location of Extensions for OpenJDK9 for OpenJ9 project REPOSITORY="ibmruntimes/openj9-openjdk-${OPENJDK_CORE_VERSION}" - BRANCH="openj9" + [ -z "$BRANCH" ] && BRANCH="openj9" fi if [[ "${BUILD_VARIANT}" == "SapMachine" ]]; then # current location of SAP variant REPOSITORY="SAP/SapMachine" - BRANCH="sapmachine10" # sapmachine10 is the current branch for OpenJDK10 mainline (equivalent to jdk/jdk10) + [ -z "$BRANCH" ] && BRANCH="sapmachine10" # sapmachine10 is the current branch for OpenJDK10 mainline (equivalent to jdk/jdk10) fi } diff --git a/pipelines/openjdk10_nightly_pipeline.groovy b/pipelines/openjdk10_nightly_pipeline.groovy index 90d87393c..891d776a5 100644 --- a/pipelines/openjdk10_nightly_pipeline.groovy +++ b/pipelines/openjdk10_nightly_pipeline.groovy @@ -1,14 +1,16 @@ println "building ${JDK_VERSION}" -def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'arm64'] +def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'aarch64'] def buildMaps = [:] +def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") + buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['arm64'] = [test:['openjdktest'], ArchOSs:'arm64_linux'] +buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] def jobs = [:] for ( int i = 0; i < buildPlatforms.size(); i++ ) { @@ -20,7 +22,8 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { def buildJobNum def checksumJob stage('build') { - buildJob = build job: "openjdk10_build_${archOS}" + buildJob = build job: "openjdk10_build_${archOS}", + parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] buildJobNum = buildJob.getNumber() } if (buildMaps[platform].test) { diff --git a/pipelines/openjdk10_openj9_nightly_pipeline.groovy b/pipelines/openjdk10_openj9_nightly_pipeline.groovy index e0efe4b39..d53536ed4 100644 --- a/pipelines/openjdk10_openj9_nightly_pipeline.groovy +++ b/pipelines/openjdk10_openj9_nightly_pipeline.groovy @@ -2,6 +2,8 @@ println "building ${JDK_VERSION}" def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] def buildMaps = [:] +def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") + buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] @@ -19,7 +21,9 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { def buildJobNum def checksumJob stage('build') { - buildJob = build job: "openjdk10_openj9_build_${archOS}" + buildJob = build job: "openjdk10_openj9_build_${archOS}", + parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), + string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] buildJobNum = buildJob.getNumber() } if (buildMaps[platform].test) { diff --git a/pipelines/openjdk10_release_pipeline.groovy b/pipelines/openjdk10_release_pipeline.groovy new file mode 100644 index 000000000..dcd0958f1 --- /dev/null +++ b/pipelines/openjdk10_release_pipeline.groovy @@ -0,0 +1,56 @@ +println "building ${JDK_VERSION}" + +def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'aarch64'] +def buildMaps = [:] +def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") + +buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] +buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] +buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] +buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] +buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] +buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] +buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] + +def jobs = [:] +for ( int i = 0; i < buildPlatforms.size(); i++ ) { + def index = i + def platform = buildPlatforms[index] + def archOS = buildMaps[platform].ArchOSs + jobs[platform] = { + def buildJob + stage('build') { + buildJob = build job: "openjdk10_build_${archOS}", + parameters: [string(name: 'TAG', value: "${JDK_TAG}"), + string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] + } + if (buildMaps[platform].test) { + buildMaps[platform].test.each { + typeTests.each { + build job:"openjdk10_hs_${it}_${archOS}", + propagate: false, + parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJob.getNumber()}"), + string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_build_${archOS}")] + } + } + } + } +} +parallel jobs + +def checksumJob +stage('checksums') { + checksumJob = build job: 'openjdk10_build_checksum', + parameters: [string(name: 'PRODUCT', value: 'releases')] +} +stage('installers') { + build job: 'openjdk10_build_installer', parameters: [string(name: 'VERSION', value: "${JDK_VERSION}")] +} +stage('publish release') { + build job: 'openjdk_release_tool', + parameters: [string(name: 'REPO', value: 'releases'), + string(name: 'TAG', value: "${JDK_TAG}"), + string(name: 'VERSION', value: 'jdk10'), + string(name: 'CHECKSUM_JOB_NAME', value: "openjdk10_build_checksum"), + string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] +} diff --git a/pipelines/openjdk8_nightly_pipeline.groovy b/pipelines/openjdk8_nightly_pipeline.groovy index ca47e496e..98ab280f4 100644 --- a/pipelines/openjdk8_nightly_pipeline.groovy +++ b/pipelines/openjdk8_nightly_pipeline.groovy @@ -1,13 +1,16 @@ println "building ${JDK_VERSION}" -def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX'] +def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'aarch64'] def buildMaps = [:] +def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") + buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'perftest', 'externaltest'], ArchOSs:'x86-64_linux'] buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] +buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] def jobs = [:] for ( int i = 0; i < buildPlatforms.size(); i++ ) { @@ -19,7 +22,8 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { def buildJobNum def checksumJob stage('build') { - buildJob = build job: "openjdk8_build_${archOS}" + buildJob = build job: "openjdk8_build_${archOS}", + parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] buildJobNum = buildJob.getNumber() } if (buildMaps[platform].test) { diff --git a/pipelines/openjdk8_openj9_nightly_pipeline.groovy b/pipelines/openjdk8_openj9_nightly_pipeline.groovy index 216561b26..09f9ba977 100644 --- a/pipelines/openjdk8_openj9_nightly_pipeline.groovy +++ b/pipelines/openjdk8_openj9_nightly_pipeline.groovy @@ -1,12 +1,16 @@ println "building ${JDK_VERSION}" -def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', 'Windows'] +def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', 'Windows', 'Windows32', 'LinuxXL'] def buildMaps = [:] +def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") + buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'perftest', 'externaltest'], ArchOSs:'x86-64_linux'] +buildMaps['LinuxXL'] = [test:['openjdktest'], ArchOSs:'x86-64_linux_largeHeap'] buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] +buildMaps['Windows32'] = [test:['openjdktest'], ArchOSs:'x86-32_windows'] def jobs = [:] for ( int i = 0; i < buildPlatforms.size(); i++ ) { @@ -18,7 +22,9 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { def buildJobNum def checksumJob stage('build') { - buildJob = build job: "openjdk8_openj9_build_${archOS}" + buildJob = build job: "openjdk8_openj9_build_${archOS}", + parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), + string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] buildJobNum = buildJob.getNumber() } if (buildMaps[platform].test) { @@ -39,7 +45,7 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { stage('publish nightly') { build job: 'openjdk_release_tool', parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: 'jdk8u162-b12'), + string(name: 'TAG', value: 'jdk8u172-b11'), string(name: 'VERSION', value: 'jdk8-openj9'), string(name: 'CHECKSUM_JOB_NAME', value: "openjdk8_openj9_build_checksum"), string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] diff --git a/pipelines/openjdk8_openj9_release_pipeline.groovy b/pipelines/openjdk8_openj9_release_pipeline.groovy index 62ab3094a..80f058251 100644 --- a/pipelines/openjdk8_openj9_release_pipeline.groovy +++ b/pipelines/openjdk8_openj9_release_pipeline.groovy @@ -2,6 +2,8 @@ println "building ${JDK_VERSION}" def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', 'Windows'] def buildMaps = [:] +def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") + buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] @@ -16,7 +18,9 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { jobs[platform] = { def buildJob stage('build') { - buildJob = build job: "openjdk8_openj9_build_${archOS}" + buildJob = build job: "openjdk8_openj9_build_${archOS}", + parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), + string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] } if (buildMaps[platform].test) { stage('test') { diff --git a/pipelines/openjdk8_release_pipeline.groovy b/pipelines/openjdk8_release_pipeline.groovy index e41f5f7f1..efa825756 100644 --- a/pipelines/openjdk8_release_pipeline.groovy +++ b/pipelines/openjdk8_release_pipeline.groovy @@ -1,14 +1,16 @@ println "building ${JDK_VERSION}" -def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'arm64'] +def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'aarch64'] def buildMaps = [:] +def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") + buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['arm64'] = [test:['openjdktest'], ArchOSs:'arm64_linux'] +buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] def jobs = [:] for ( int i = 0; i < buildPlatforms.size(); i++ ) { @@ -19,7 +21,8 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { def buildJob stage('build') { buildJob = build job: "openjdk8_build_${archOS}", - parameters: [string(name: 'TAG', value: "${JDK_TAG}")] + parameters: [string(name: 'TAG', value: "${JDK_TAG}"), + string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] } if (buildMaps[platform].test) { buildMaps[platform].test.each { diff --git a/pipelines/openjdk9_nightly_pipeline.groovy b/pipelines/openjdk9_nightly_pipeline.groovy index d999be775..be5602d5a 100644 --- a/pipelines/openjdk9_nightly_pipeline.groovy +++ b/pipelines/openjdk9_nightly_pipeline.groovy @@ -1,14 +1,16 @@ println "building ${JDK_VERSION}" -def buildPlatforms = ['Mac', 'Linux', 'zLinux', 'ppc64le', 'Windows', 'AIX', 'arm64'] +def buildPlatforms = ['Mac', 'Linux', 'zLinux', 'ppc64le', 'Windows', 'AIX', 'aarch64'] def buildMaps = [:] +def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") + buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['arm64'] = [test:['openjdktest'], ArchOSs:'arm64_linux'] +buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] def jobs = [:] for ( int i = 0; i < buildPlatforms.size(); i++ ) { @@ -20,7 +22,9 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { def buildJobNum def checksumJob stage('build') { - buildJob = build job: "openjdk9_build_${archOS}" + buildJob = build job: "openjdk9_build_${archOS}", + parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] + buildJobNum = buildJob.getNumber() } if (buildMaps[platform].test) { diff --git a/pipelines/openjdk9_openj9_nightly_pipeline.groovy b/pipelines/openjdk9_openj9_nightly_pipeline.groovy index 756fee5fd..00a8f9bcc 100644 --- a/pipelines/openjdk9_openj9_nightly_pipeline.groovy +++ b/pipelines/openjdk9_openj9_nightly_pipeline.groovy @@ -2,6 +2,8 @@ println "building ${JDK_VERSION}" def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] def buildMaps = [:] +def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") + buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] @@ -18,7 +20,9 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { def buildJobNum def checksumJob stage('build') { - buildJob = build job: "openjdk9_openj9_build_${archOS}" + buildJob = build job: "openjdk9_openj9_build_${archOS}", + parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), + string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] buildJobNum = buildJob.getNumber() } if (buildMaps[platform].test) { @@ -39,7 +43,7 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { stage('publish nightly') { build job: 'openjdk_release_tool', parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: 'jdk-9+181'), + string(name: 'TAG', value: 'jdk-9.0.4+12'), string(name: 'VERSION', value: 'jdk9-openj9'), string(name: 'CHECKSUM_JOB_NAME', value: "openjdk9_openj9_build_checksum"), string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] diff --git a/pipelines/openjdk_amber_nightly_pipeline.groovy b/pipelines/openjdk_amber_nightly_pipeline.groovy index fe8932077..bbc24b024 100644 --- a/pipelines/openjdk_amber_nightly_pipeline.groovy +++ b/pipelines/openjdk_amber_nightly_pipeline.groovy @@ -2,6 +2,8 @@ println "building ${JDK_VERSION}" def buildPlatforms = ['Mac', 'Linux', 'Windows'] def buildMaps = [:] +def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyddMMHHmm") + buildMaps['Mac'] = [test:['openjdktest'], ArchOSs:'x86-64_macos'] buildMaps['Windows'] = [test:false, ArchOSs:'x86-64_windows'] buildMaps['Linux'] = [test:['openjdktest'], ArchOSs:'x86-64_linux'] @@ -16,7 +18,8 @@ for ( int i = 0; i < buildPlatforms.size(); i++ ) { def buildJobNum def checksumJob stage('build') { - buildJob = build job: "openjdk_amber_build_${archOS}" + buildJob = build job: "openjdk_amber_build_${archOS}", + parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] buildJobNum = buildJob.getNumber() } if (buildMaps[platform].test) { diff --git a/sbin/build.sh b/sbin/build.sh index 2f56d6b07..04ad65a3a 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -179,6 +179,7 @@ configuringVersionStringParameter() addConfigureArgIfValueIsNotEmpty "--with-build-number=" "${OPENJDK_BUILD_NUMBER}" else if [ -z "$OPENJDK_REPO_TAG" ]; then + cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || echo Cannot change to "${WORKING_DIR}/${OPENJDK_REPO_NAME}" OPENJDK_REPO_TAG=$(getFirstTagFromOpenJDKGitRepo) echo "OpenJDK repo tag is ${OPENJDK_REPO_TAG}" fi diff --git a/sbin/jtreg.sh b/sbin/jtreg.sh index e1904f1a2..34b133d82 100755 --- a/sbin/jtreg.sh +++ b/sbin/jtreg.sh @@ -80,9 +80,9 @@ applyingJCovSettingsToMakefileForTests() echo "Apply JCov settings to Makefile..." cd "$OPENJDK_DIR/jdk/test" || exit pwd - - sed -i "s/-vmoption:-Xmx512m.*/-vmoption:-Xmx512m -xml:verify -jcov\/classes:$(ABS_PLATFORM_BUILD_ROOT)\/jdk\/classes\/ -jcov\/source:$(ABS_PLATFORM_BUILD_ROOT)\/..\/..\/jdk\/src\/java\/share\/classes -jcov\/include:*/" Makefile - + + sed -i "s/-vmoption:-Xmx512m.*/-vmoption:-Xmx512m -xml:verify -jcov\/classes:\$\(ABS_PLATFORM_BUILD_ROOT\)\/images\/j2sdk-image\/jre\/lib\/rt.jar -jcov\/source:\$\(ABS_PLATFORM_BUILD_ROOT\)\/images\/j2sdk-image\/src.zip -jcov\/include:*/" Makefile + cd "$OPENJDK_DIR" || exit } From 4600d598e8f08dbee5cabfd519814587a151748e Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 23 Jul 2018 17:10:04 +0100 Subject: [PATCH 003/101] Bring new build scripts up to date with master (#420) * Add openjdktest build on Windows (#322) Also update to enable capability of test build triggered per test type. Signed-off-by: Sophia Guo * Add triggers for openjdk9-j9/hs and openjdk10-j9/hs for External Tests Signed-off-by: Mesbah * update release pipeline (#337) * extra fixes to release pipeline (#338) * update release pipeline * extra fixes to build release pipeline * Update openjdk8_release_pipeline.groovy * Add git-hg/updateforesttags.sh (#340) * Add git-hg/updatetags8. Initial version hard coded with tags and pauses for manual merging * Miscellanous updates to tidy up the 'work in progress' version :-) * Second round of updates to tidy up the script * Rename updatetags8.sh to updateforesttags.sh * Add arm64 jdk10 to pipeline, add Win testing to release pipelines Signed-off-by: smlambert * Attempted fix to Travis config for shellcheck (#344) * use correct SAP branch * Docker build doc improvement * formatting * Upate READMEs to reflect broken status of Docker builds * Attempt to fix Travis CI shellcheck install failure * Initial Shellcheck fixes * More shellcheck fixes * More shellcheck fixes for unrelated scripts * More shellcheck fixes * More shellcheck fixes * More shellcheck fixes * Tidy from review * Fix jdk9u Mercurial repo name (#348) * change git tag command * Initial cut of new git-hg script for java 10+ (#349) * use correct SAP branch * Docker build doc improvement * formatting * Upate READMEs to reflect broken status of Docker builds * Attempt to fix Travis CI shellcheck install failure * Initial Shellcheck fixes * More shellcheck fixes * More shellcheck fixes for unrelated scripts * More shellcheck fixes * More shellcheck fixes * More shellcheck fixes * Tidy from review * Initial cut of new git-hg script for java 10+ * minor fixes from review * pass parameters through to function * skip git version check for mirror script * move git checkout before the fetch * only reset if it's not empty * turn pipefail off because we're awesome * turn pipefail back on because we're NOT awesome * Remove bad flag and get compression right * It's all our fault, let us try fetching first * change merge command * add else case to git reset * add brackets to add or * More shellcheck fixes * More shellcheck fixes * More shellcheck fixes * Add dev branch merge capability * Update release tags for AO8/J9 and AO9/J9 * Update release tags for AO8/J9 and AO9/J9 (#352) * Add Win32 & x64_linux_largeHeap (#355) * Add Win32 & LinuxLargeHeap Signed-off-by: smlambert * Fix typo Signed-off-by: smlambert * add force flag to git checkout for git-hg mirroring prevents untracked files from failing the job * remove git gc command from mirror scripts (#357) * Ensure tag from the source repo instead of oepnjdk-build is used (#360) * WIP: Update arm64 name to be aarch64 (#368) * Add Win32 & LinuxLargeHeap Signed-off-by: smlambert * Fix typo Signed-off-by: smlambert * Change arm64 to aarch64 Signed-off-by: smlambert * Update ArchOSs Signed-off-by: smlambert * No change to openjdk8_nightly for now Signed-off-by: smlambert * Add openjdk8 Signed-off-by: smlambert * Use escape backslash for ABS_PLATFORM_BUILD_ROOT Enable jcov in java8 Signed-off-by: Sophia Guo * Enable jcov for java8 Signed-off-by: Sophia Guo * Allow overriding of openj9 branch name * Fix negated logic in last commit * Pipelines allow alternate branch (#379) * Add ALT_BRANCH parameter to openj9 pipelines * Add ALT_BRANCH parameter to non-openj9 pipelines for consistency * Add timestamps to top-level pipelines (#382) * Add timestamps to top-level pipelines Signed-off-by: smlambert * Update to use yyMMdd, not yyddMM Signed-off-by: smlambert * jcov8 (#383) Signed-off-by: Sophia Guo * add jdk10 release pipeline (#385) * Fix empty string PIPELINE_TIMESTAMP issue Signed-off-by: Sophia Guo * Remove undefined variable ALT_BRANCH (#391) Signed-off-by: Sophia Guo * Add Linux/arm32 (e.g. raspberry pi) back into openjdk10 pipeline (#395) * Add support for creating JREs * Replace unneccessary // with / in variable replacement * trying again * trying again * trying again * trying again * trying again * shellcheck fixes part I * more shellcheck fixes * more shellcheck fixes * Revert "Add Linux/arm32 (e.g. raspberry pi) back into openjdk10 pipeline (#395)" (#404) This reverts commit 2b924a9cb194aa02c1bd990af652fb1f78593e8f. * Add Linux/arm32 (e.g. raspberry pi) back into openjdk10 pipeline (#395) (#407) * Remove ao10/arm32 testing as jobs do not yet exist * Use pigz to compress instead of gzip if available (#408) * Add arm32 builds to adoptopenjdk9 nightly pipeline (#410) * Pass in jdk version to jtreg (#380) Pass in jdk version to jtreg as jdk test structure is different with different jdk version Signed-off-by: Sophia Guo * Add support for jdk11 (#419) * Add support for jdk11 * correct update source for older versions --- .travis.yml | 17 +++-- README.md | 7 ++- makejdk-any-platform.1 | 7 ++- makejdk-any-platform.sh | 14 ++--- makejdk.sh | 70 +++++++++------------ pipelines/openjdk10_nightly_pipeline.groovy | 3 +- pipelines/openjdk10_release_pipeline.groovy | 3 +- pipelines/openjdk9_nightly_pipeline.groovy | 3 +- sbin/build.sh | 49 +++++++++++---- sbin/jtreg.sh | 17 +++-- 10 files changed, 110 insertions(+), 80 deletions(-) diff --git a/.travis.yml b/.travis.yml index d49b8ca4e..017b57786 100644 --- a/.travis.yml +++ b/.travis.yml @@ -3,18 +3,17 @@ language: bash # Use container-based infrastructure for quicker build start-up sudo: false -addons: - apt: - sources: - - debian-sid # Grab shellcheck from the Debian repo (o_O) +install: -before_install: - - sudo apt-get install -y cabal-install shunit2 - - cabal update - - cabal install shellcheck + # Install a custom version of shellcheck instead of Travis CI's default + - scversion="stable" # or "v0.4.7", or "latest" + - wget "https://storage.googleapis.com/shellcheck/shellcheck-$scversion.linux.x86_64.tar.xz" + - tar --xz -xvf "shellcheck-$scversion.linux.x86_64.tar.xz" + - shellcheck() { "shellcheck-$scversion/shellcheck" "$@"; } + - shellcheck --version script: - - bash -c 'shopt -s globstar; shellcheck **/*.sh' + - shellcheck *.sh matrix: fast_finish: true diff --git a/README.md b/README.md index 7d5ac3af9..6ef6b214f 100644 --- a/README.md +++ b/README.md @@ -37,9 +37,10 @@ Once you have Docker started you can then use the script below to build OpenJDK. Usage: ./makejdk-any-platform.sh --version [version] [options] Versions: - jdk8u - https://github.com/AdoptOpenJDK/openjdk-jdk8u - jdk9 - https://github.com/AdoptOpenJDK/openjdk-jdk9 - jdk10 - https://github.com/AdoptOpenJDK/openjdk-jdk10 + jdk8u - https://github.com/AdoptOpenJDK/openjdk-jdk8u + jdk9u - https://github.com/AdoptOpenJDK/openjdk-jdk9u + jdk10u - https://github.com/AdoptOpenJDK/openjdk-jdk10u + jdk11 - https://github.com/AdoptOpenJDK/openjdk-jdk11 Options: -s, --source specify the location for the source and dependencies to be cloned, defaults to ./openjdk. If it is specified, docker is not used diff --git a/makejdk-any-platform.1 b/makejdk-any-platform.1 index 19a682d3a..c246f24f5 100755 --- a/makejdk-any-platform.1 +++ b/makejdk-any-platform.1 @@ -13,10 +13,13 @@ The simplest way to build OpenJDK using our scripts is to run makejdk-any-platfo https://github.com/AdoptOpenJDK/openjdk-jdk8u .TP .BR \jdk9 -https://github.com/AdoptOpenJDK/openjdk-jdk9 +https://github.com/AdoptOpenJDK/openjdk-jdk9u .TP .BR \jdk10 -https://github.com/AdoptOpenJDK/openjdk-jdk10 +https://github.com/AdoptOpenJDK/openjdk-jdk10u +.TP +.BR \jdk11 +https://github.com/AdoptOpenJDK/openjdk-jdk11 .TP .SH OPTIONS diff --git a/makejdk-any-platform.sh b/makejdk-any-platform.sh index d920f1532..854c7d729 100755 --- a/makejdk-any-platform.sh +++ b/makejdk-any-platform.sh @@ -40,10 +40,10 @@ export REPOSITORY="" counter=0 for i in "$@"; do - let counter++ + (( counter++ )) case "$i" in "--version" | "-v") - let counter++ + (( counter++ )) string="\$$counter" OPENJDK_FOREST_NAME=$(echo "$@" | awk "{print $string}") export OPENJDK_CORE_VERSION=${OPENJDK_FOREST_NAME} @@ -51,21 +51,21 @@ for i in "$@"; do export OPENJDK_CORE_VERSION=${OPENJDK_FOREST_NAME%?} fi # Switch it back to stop it being out of sync with i - let counter-- + (( counter-- )) ;; "--variant" | "-bv") - let counter++ + (( counter++ )) string="\$$counter" BUILD_VARIANT=$(echo "$@" | awk "{print $string}") # Switch it back to stop it being out of sync with i - let counter-- + (( counter-- )) ;; esac done export COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG="false" export COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG="false" -if [ "$OPENJDK_CORE_VERSION" == "jdk9" ] || [ "$OPENJDK_CORE_VERSION" == "jdk10" ] || [ "$OPENJDK_CORE_VERSION" == "amber" ]; then +if [ "$OPENJDK_CORE_VERSION" == "jdk9" ] || [ "$OPENJDK_CORE_VERSION" == "jdk10" ] || [ "$OPENJDK_CORE_VERSION" == "jdk11" ] || [ "$OPENJDK_CORE_VERSION" == "amber" ]; then export JDK_PATH="jdk" export JRE_PATH="jre" if [ "$OPENJDK_CORE_VERSION" == "jdk9" ]; then @@ -79,7 +79,7 @@ elif [ "$OPENJDK_CORE_VERSION" == "jdk8" ]; then export JDK_PATH="j2sdk-image" export JRE_PATH="j2re-image" else - echo "Please specify a version with --version or -v , either jdk9, jdk10 or jdk8, with or without a \'u\' suffix." + echo "Please specify a version with --version or -v , either jdk9, jdk10 or jdk8, with or without a \\'u\\' suffix." man ./makejdk-any-platform.1 exit 1 fi diff --git a/makejdk.sh b/makejdk.sh index b6c4a63e0..82c1b20fd 100755 --- a/makejdk.sh +++ b/makejdk.sh @@ -25,7 +25,7 @@ # You can set the JDK boot directory with the JDK_BOOT_DIR environment variable SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -# shellcheck source=sbin/common-functions.sh +# shellcheck disable=SC1090 source "$SCRIPT_DIR/sbin/common-functions.sh" OPENJDK_REPO_NAME=${OPENJDK_REPO_NAME:-openjdk} @@ -50,6 +50,7 @@ KEEP=false JTREG=false BUILD_VARIANT=${BUILD_VARIANT-:""} USER_SUPPLIED_CONFIGURE_ARGS="" +VERSION="" JVM_VARIANT=${JVM_VARIANT:-server} @@ -68,6 +69,7 @@ sourceFileWithColourCodes() sourceSignalHandler() { + # shellcheck disable=SC1091 source sbin/signalhandler.sh } @@ -122,7 +124,7 @@ parseCommandLineArgs() export FREETYPE=false;; "--version" | "-v" ) - shift;; + VERSION="$1"; shift;; "--freetype-dir" | "-ftd" ) export FREETYPE_DIRECTORY="$1"; shift;; @@ -133,7 +135,7 @@ parseCommandLineArgs() "--configure-args" | "-ca" ) export USER_SUPPLIED_CONFIGURE_ARGS="$1"; shift;; - *) echo >&2 "${error}Invalid option: ${opt}${normal}"; man ./makejdk-any-platform.1; exit 1;; + *) echo >&2 "Invalid option: ${opt}"; man ./makejdk-any-platform.1; exit 1;; esac done } @@ -157,36 +159,34 @@ checkIfDockerIsUsedForBuildingOrNot() # Both a working directory and a target directory provided if [ ! -z "$WORKING_DIR" ] && [ ! -z "$TARGET_DIR" ] ; then # This uses sbin/build.sh directly - echo "${info}Not using Docker, working area will be ${WORKING_DIR}, target for the JDK will be ${TARGET_DIR} ${normal}" + echo "Not using Docker, working area will be ${WORKING_DIR}, target for the JDK will be ${TARGET_DIR}" fi # No working directory and no target directory provided if [ -z "${WORKING_DIR}" ] && [ -z "${TARGET_DIR}" ] ; then - echo "${info}No parameters provided, using Docker. ${normal}" + echo "No parameters provided, using Docker." USE_DOCKER=true elif [ ! -z "$TARGET_DIR" ] && [ -z "$WORKING_DIR" ] ; then # Target directory is defined but the working directory isn't # Calls sbin/build.sh inside of Docker followed by a docker cp command - echo "${info}Using Docker, target directory for the tgz on the host: ${TARGET_DIR}" + echo "Using Docker, target directory for the tgz on the host: ${TARGET_DIR}" USE_DOCKER=true fi } checkIfDockerIsUsedShouldTheContainerBePreserved() { - echo "${info}" if [ "${KEEP}" == "true" ] ; then echo "We'll keep the built Docker container if you're using Docker." else echo "We'll remove the built Docker container if you're using Docker." fi - echo "${normal}" } setDefaultIfBranchIsNotProvided() { if [ -z "$BRANCH" ] ; then - echo "${info}BRANCH is undefined so checking out dev${normal}" + echo "BRANCH is undefined so checking out dev" BRANCH="dev" fi } @@ -194,23 +194,22 @@ setDefaultIfBranchIsNotProvided() setWorkingDirectoryIfProvided() { if [ -z "${WORKING_DIR}" ] ; then - echo "${info}WORKING_DIR is undefined so setting to ${PWD}${normal}." + echo "WORKING_DIR is undefined so setting to ${PWD}." WORKING_DIR=$PWD else - echo "${info}Working dir is ${WORKING_DIR}${normal}." + echo "Working dir is ${WORKING_DIR}." fi } setTargetDirectoryIfProvided() { - echo "${info}" if [ -z "${TARGET_DIR}" ] ; then - echo "${info}TARGET_DIR is undefined so setting to $PWD" + echo "TARGET_DIR is undefined so setting to $PWD" TARGET_DIR=$PWD # Only makes a difference if we're in Docker echo "If you're using Docker the build artifact will not be copied to the host." else - echo "${info}Target directory is ${TARGET_DIR}${normal}" + echo "Target directory is ${TARGET_DIR}" COPY_TO_HOST=true echo "If you're using Docker we'll copy the build artifact to the host." fi @@ -218,16 +217,15 @@ setTargetDirectoryIfProvided() checkOpenJDKGitRepo() { - echo "${git}" - if [ -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ] && ( [ "$OPENJDK_CORE_VERSION" == "jdk8" ] || [ "$OPENJDK_CORE_VERSION" == "jdk9" ] || [ "$OPENJDK_CORE_VERSION" == "jdk10" ]) ; then + if [ -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ] && { [ "$OPENJDK_CORE_VERSION" == "jdk8" ] || [ "$OPENJDK_CORE_VERSION" == "jdk9" ] || [ "$OPENJDK_CORE_VERSION" == "jdk10" ] || [ "$OPENJDK_CORE_VERSION" == "jdk11" ] ; } ; then GIT_VERSION=$(git --git-dir "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" remote -v | grep "${OPENJDK_CORE_VERSION}") echo "${GIT_VERSION}" if [ "$GIT_VERSION" ]; then # The repo is the correct JDK Version cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || return - echo "${info}Will reset the repository at $PWD in 10 seconds...${git}" + echo "Will reset the repository at $PWD in 10 seconds..." sleep 10 - echo "${git}Pulling latest changes from git repo" + echo "Pulling latest changes from git repo" showShallowCloningMessage "fetch" git fetch --all ${SHALLOW_CLONE_OPTION} @@ -245,15 +243,13 @@ checkOpenJDKGitRepo() cd "${WORKING_DIR}" || return elif [ ! -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ] ; then # If it doesn't exist, clone it - echo "${info}Didn't find any existing openjdk repository at WORKING_DIR (set to ${WORKING_DIR}) so cloning the source to openjdk" + echo "Didn't find any existing openjdk repository at WORKING_DIR (set to ${WORKING_DIR}) so cloning the source to openjdk" cloneOpenJDKGitRepo fi - echo "${normal}" } cloneOpenJDKGitRepo() { - echo "${git}" if [[ "$USE_SSH" == "true" ]] ; then GIT_REMOTE_REPO_ADDRESS="git@github.com:${REPOSITORY}.git" else @@ -261,7 +257,7 @@ cloneOpenJDKGitRepo() fi showShallowCloningMessage "cloning" - GIT_CLONE_ARGUMENTS=($SHALLOW_CLONE_OPTION '-b' "$BRANCH" "$GIT_REMOTE_REPO_ADDRESS" "${WORKING_DIR}/${OPENJDK_REPO_NAME}") + GIT_CLONE_ARGUMENTS=("$SHALLOW_CLONE_OPTION" '-b' "$BRANCH" "$GIT_REMOTE_REPO_ADDRESS" "${WORKING_DIR}/${OPENJDK_REPO_NAME}") echo "git clone ${GIT_CLONE_ARGUMENTS[*]}" git clone "${GIT_CLONE_ARGUMENTS[@]}" @@ -281,16 +277,15 @@ cloneOpenJDKGitRepo() # TODO This only works for jdk8u based releases. Will require refactoring when jdk9 enters an update cycle getOpenJDKUpdateAndBuildVersion() { - echo "${git}" if [ -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ]; then # It does exist and it's a repo other than the AdoptOpenJDK one cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || return - echo "${git}Pulling latest tags and getting the latest update version using git fetch -q --tags ${SHALLOW_CLONE_OPTION}" + echo "Pulling latest tags and getting the latest update version using git fetch -q --tags ${SHALLOW_CLONE_OPTION}" git fetch -q --tags "${SHALLOW_CLONE_OPTION}" OPENJDK_REPO_TAG=${TAG:-$(getFirstTagFromOpenJDKGitRepo)} if [[ "${OPENJDK_REPO_TAG}" == "" ]] ; then - echo "${error}Unable to detect git tag" + echo "Unable to detect git tag" exit 1 else echo "OpenJDK repo tag is $OPENJDK_REPO_TAG" @@ -303,16 +298,15 @@ getOpenJDKUpdateAndBuildVersion() fi - echo "${normal}" } showShallowCloningMessage() { mode=$1 if [[ "$SHALLOW_CLONE_OPTION" == "" ]]; then - echo "${info}Git repo ${mode} mode: deep (preserves commit history)${normal}" + echo "Git repo ${mode} mode: deep (preserves commit history)" else - echo "${info}Git repo ${mode} mode: shallow (DOES NOT contain commit history)${normal}" + echo "Git repo ${mode} mode: shallow (DOES NOT contain commit history)" fi } @@ -338,16 +332,16 @@ createPersistentDockerDataVolume() if [[ "$CLEAN_DOCKER_BUILD" == "true" || "$DATA_VOLUME_EXISTS" != "0" ]]; then - echo "${info}Removing old volumes and containers${normal}" + echo "Removing old volumes and containers" docker rm -f "$(docker ps -a --no-trunc | grep $CONTAINER | cut -d' ' -f1)" || true docker volume rm "${DOCKER_SOURCE_VOLUME_NAME}" || true - echo "${info}Creating tmp container and copying src${normal}" + echo "Creating tmp container and copying src" docker volume create --name "${DOCKER_SOURCE_VOLUME_NAME}" docker run -v "${DOCKER_SOURCE_VOLUME_NAME}":/openjdk/build --name "$TMP_CONTAINER_NAME" ubuntu:14.04 /bin/bash docker cp openjdk "$TMP_CONTAINER_NAME":/openjdk/build/ - echo "${info}Removing tmp container${normal}" + echo "Removing tmp container" docker rm -f "$TMP_CONTAINER_NAME" fi } @@ -369,12 +363,12 @@ buildAndTestOpenJDKViaDocker() PATH_BUILD="docker/${OPENJDK_CORE_VERSION}/x86_64/ubuntu" - if [ -z "$(which docker)" ]; then - echo "${error}Error, please install docker and ensure that it is in your path and running!${normal}" + if [ -z "$(command -v docker)" ]; then + echo "Error, please install docker and ensure that it is in your path and running!" exit fi - echo "${info}Using Docker to build the JDK${normal}" + echo "Using Docker to build the JDK" createPersistentDockerDataVolume @@ -389,15 +383,13 @@ buildAndTestOpenJDKViaDocker() if [[ "$KEEP" == "true" ]] ; then # shellcheck disable=SC2086 if [ "$(docker ps -a | grep -c \"$CONTAINER\")" == 0 ]; then - echo "${info}No docker container found so creating '$CONTAINER' ${normal}" + echo "No docker container found so creating '$CONTAINER' " buildDockerContainer fi else - echo "${info}Building as you've not specified -k or --keep" - echo "$good" + echo "Building as you've not specified -k or --keep" docker ps -a | awk '{ print $1,$2 }' | grep "$CONTAINER" | awk '{print $1 }' | xargs -I {} docker rm -f {} buildDockerContainer - echo "$normal" fi mkdir -p "${WORKING_DIR}/target" @@ -420,7 +412,7 @@ testOpenJDKInNativeEnvironmentIfExpected() { if [[ "$JTREG" == "true" ]]; then - "${SCRIPT_DIR}"/sbin/jtreg.sh "${WORKING_DIR}" "${OPENJDK_REPO_NAME}" "${BUILD_FULL_NAME}" "${JTREG_TEST_SUBSETS}" + "${SCRIPT_DIR}"/sbin/jtreg.sh "${WORKING_DIR}" "${OPENJDK_REPO_NAME}" "${BUILD_FULL_NAME}" "${VERSION}" "${JTREG_TEST_SUBSETS}" fi } diff --git a/pipelines/openjdk10_nightly_pipeline.groovy b/pipelines/openjdk10_nightly_pipeline.groovy index 891d776a5..2f62e2c61 100644 --- a/pipelines/openjdk10_nightly_pipeline.groovy +++ b/pipelines/openjdk10_nightly_pipeline.groovy @@ -1,6 +1,6 @@ println "building ${JDK_VERSION}" -def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'aarch64'] +def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'arm32','aarch64'] def buildMaps = [:] def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") @@ -10,6 +10,7 @@ buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOS buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] +buildMaps['arm32'] = [test:false, ArchOSs:'arm32_linux'] buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] def jobs = [:] diff --git a/pipelines/openjdk10_release_pipeline.groovy b/pipelines/openjdk10_release_pipeline.groovy index dcd0958f1..4f0504c35 100644 --- a/pipelines/openjdk10_release_pipeline.groovy +++ b/pipelines/openjdk10_release_pipeline.groovy @@ -1,6 +1,6 @@ println "building ${JDK_VERSION}" -def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'aarch64'] +def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'arm32', 'aarch64'] def buildMaps = [:] def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") @@ -10,6 +10,7 @@ buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux' buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] +buildMaps['arm32'] = [test:false, ArchOSs:'arm32_linux'] buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] def jobs = [:] diff --git a/pipelines/openjdk9_nightly_pipeline.groovy b/pipelines/openjdk9_nightly_pipeline.groovy index be5602d5a..bdc81ff34 100644 --- a/pipelines/openjdk9_nightly_pipeline.groovy +++ b/pipelines/openjdk9_nightly_pipeline.groovy @@ -1,6 +1,6 @@ println "building ${JDK_VERSION}" -def buildPlatforms = ['Mac', 'Linux', 'zLinux', 'ppc64le', 'Windows', 'AIX', 'aarch64'] +def buildPlatforms = ['Mac', 'Linux', 'zLinux', 'ppc64le', 'Windows', 'AIX', 'arm32', 'aarch64'] def buildMaps = [:] def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") @@ -10,6 +10,7 @@ buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOS buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] +buildMaps['arm32'] = [test:false, ArchOSs:'arm32_linux'] buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] def jobs = [:] diff --git a/sbin/build.sh b/sbin/build.sh index 04ad65a3a..ad9932d84 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -28,6 +28,7 @@ JVM_VARIANT="${JVM_VARIANT:-server}" OPENJDK_UPDATE_VERSION="" OPENJDK_BUILD_NUMBER="" OPENJDK_REPO_TAG="" +JRE_TARGET_PATH="" TRIMMED_TAG="" USER_SUPPLIED_CONFIGURE_ARGS="" @@ -353,13 +354,22 @@ removingUnnecessaryFiles() rm -rf "${OPENJDK_REPO_TAG}" || true mv "$JDK_PATH" "${OPENJDK_REPO_TAG}" + JRE_TARGET_PATH="${OPENJDK_REPO_TAG/jdk/jre}" + [ "${JRE_TARGET_PATH}" == "${OPENJDK_REPO_TAG}" ] && JRE_TARGET_PATH="${OPENJDK_REPO_TAG}.jre" + echo "moving ${JRE_PATH} to ${JRE_TARGET_PATH}" + rm -rf "${JRE_TARGET_PATH}" || true + mv "$JRE_PATH" "${JRE_TARGET_PATH}" + # Remove files we don't need rm -rf "${OPENJDK_REPO_TAG}"/demo/applets || true rm -rf "${OPENJDK_REPO_TAG}"/demo/jfc/Font2DTest || true rm -rf "${OPENJDK_REPO_TAG}"/demo/jfc/SwingApplet || true + rm -rf "${JRE_TARGET_PATH}"/demo/applets || true + rm -rf "${JRE_TARGET_PATH}"/demo/jfc/Font2DTest || true + rm -rf "${JRE_TARGET_PATH}"/demo/jfc/SwingApplet || true find . -name "*.diz" -type f -delete || true - echo "Finished removing unnecessary files from ${OPENJDK_REPO_TAG}" + echo "Finished removing unnecessary files from ${OPENJDK_REPO_TAG} and ${JRE_TARGET_PATH}" } makeACopyOfLibFreeFontForMacOSX() { @@ -411,10 +421,10 @@ signRelease() echo "Signing Windows release" signToolPath=${signToolPath:-"/cygdrive/c/Program Files/Microsoft SDKs/Windows/v7.1/Bin/signtool.exe"} # Sign .exe files - FILES=$(find "${OPENJDK_REPO_TAG}" -type f -name '*.exe') + FILES=$(find "${OPENJDK_REPO_TAG}" "${JRE_TARGET_PATH}" -type f -name '*.exe') echo "$FILES" | while read -r f; do "$signToolPath" sign /f "$CERTIFICATE" /p "$SIGN_PASSWORD" /fd SHA256 /t http://timestamp.verisign.com/scripts/timstamp.dll "$f"; done # Sign .dll files - FILES=$(find "${OPENJDK_REPO_TAG}" -type f -name '*.dll') + FILES=$(find "${OPENJDK_REPO_TAG}" "${JRE_TARGET_PATH}" -type f -name '*.dll') echo "$FILES" | while read -r f; do "$signToolPath" sign /f "$CERTIFICATE" /p "$SIGN_PASSWORD" /fd SHA256 /t http://timestamp.verisign.com/scripts/timstamp.dll "$f"; done ;; "darwin"*) @@ -424,7 +434,7 @@ signRelease() # shellcheck disable=SC2006 security unlock-keychain -p `cat ~/.password` # Sign all files with the executable permission bit set. - FILES=$(find "${OPENJDK_REPO_TAG}" -perm +111 -type f || find "${OPENJDK_REPO_TAG}" -perm /111 -type f) + FILES=$(find "${OPENJDK_REPO_TAG}" "${JRE_TARGET_PATH}" ! -perm +111 -type f || find "${OPENJDK_REPO_TAG}" -perm /111 -type f) echo "$FILES" | while read -r f; do codesign -s "$CERTIFICATE" "$f"; done ;; *) @@ -436,34 +446,51 @@ signRelease() createOpenJDKTarArchive() { - echo "Archiving the build OpenJDK image..." + COMPRESS=gzip + if which pigz >/dev/null 2>&1; then COMPRESS=pigz; fi + echo "Archiving the build OpenJDK image and compressing with $COMPRESS" if [ -z "$OPENJDK_REPO_TAG" ]; then OPENJDK_REPO_TAG=$(getFirstTagFromOpenJDKGitRepo) fi - echo "OpenJDK repo tag is ${OPENJDK_REPO_TAG}" + if [ -z "$JRE_TARGET_PATH" ]; then + JRE_TARGET_PATH="${OPENJDK_REPO_TAG/jdk/jre}" + [ "${JRE_TARGET_PATH}" == "${OPENJDK_REPO_TAG}" ] && JRE_TARGET_PATH="${OPENJDK_REPO_TAG}.jre" + fi + + echo "OpenJDK repo tag is ${OPENJDK_REPO_TAG}. JRE path will be ${JRE_TARGET_PATH}" if [ "$USE_DOCKER" == "true" ] ; then - GZIP=-9 tar -czf OpenJDK.tar.gz ./"${OPENJDK_REPO_TAG}" + GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJDK.tar.gz ./"${OPENJDK_REPO_TAG}" + GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJRE.tar.gz ./"${JRE_TARGET_PATH}" EXT=".tar.gz" echo "${good}Moving the artifact to ${TARGET_DIR}${normal}" mv "OpenJDK${EXT}" "${TARGET_DIR}" + mv "OpenJRE${EXT}" "${TARGET_DIR}" else case "${OS_KERNEL_NAME}" in *cygwin*) zip -r -q OpenJDK.zip ./"${OPENJDK_REPO_TAG}" + zip -r -q OpenJRE.zip ./"${JRE_TARGET_PATH}" EXT=".zip" ;; aix) - GZIP=-9 tar -cf - ./"${OPENJDK_REPO_TAG}"/ | gzip -c > OpenJDK.tar.gz + GZIP=-9 tar -cf - ./"${OPENJDK_REPO_TAG}"/ | $COMPRESS -c > OpenJDK.tar.gz + GZIP=-9 tar -cf - ./"${JRE_TARGET_PATH}"/ | $COMPRESS -c > OpenJRE.tar.gz EXT=".tar.gz" ;; *) - GZIP=-9 tar -czf OpenJDK.tar.gz ./"${OPENJDK_REPO_TAG}" + GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJDK.tar.gz ./"${OPENJDK_REPO_TAG}" + GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJRE.tar.gz ./"${JRE_TARGET_PATH}" EXT=".tar.gz" ;; esac echo "${good}Your final ${EXT} was created at ${PWD}${normal}" - - echo "${good}Moving the artifact to ${TARGET_DIR}${normal}" + ls -l + echo "${good}Moving the artifacts to ${TARGET_DIR}${normal}" + # This really ought to be a separate parameter to makejdk_any_platform or + # TARGET_DIR should be a dir name as the name suggests, not a full filename + # This is currnently assuming TARGET_DIT has JDK in the filename, otherwise + # it wiill get overridden + mv "OpenJRE${EXT}" "${TARGET_DIR/JDK/JRE}" mv "OpenJDK${EXT}" "${TARGET_DIR}" fi diff --git a/sbin/jtreg.sh b/sbin/jtreg.sh index 34b133d82..e355b7727 100755 --- a/sbin/jtreg.sh +++ b/sbin/jtreg.sh @@ -20,8 +20,9 @@ source "$SCRIPT_DIR/common-functions.sh" WORKING_DIR=$1 OPENJDK_REPO_NAME=$2 BUILD_FULL_NAME=$3 +VERSION=$4 # shellcheck disable=SC2001 -JTREG_TEST_SUBSETS=$(echo "$4" | sed 's/:/ /') +JTREG_TEST_SUBSETS=$(echo "$5" | sed 's/:/ /') JTREG_VERSION=${JTREG_VERSION:-4.2.0-tip} JTREG_TARGET_FOLDER=${JTREG_TARGET_FOLDER:-jtreg} JOB_NAME=${JOB_NAME:-OpenJDK} @@ -77,12 +78,16 @@ downloadJtregAndSetupEnvironment() applyingJCovSettingsToMakefileForTests() { - echo "Apply JCov settings to Makefile..." - cd "$OPENJDK_DIR/jdk/test" || exit + echo "Apply JCov settings to Makefile..." + if [[ $VERSION == *8* ]]; then + cd "$OPENJDK_DIR/jdk/test" || exit + sed -i "s/-vmoption:-Xmx512m.*/-vmoption:-Xmx512m -xml:verify -jcov\/classes:\$\(ABS_PLATFORM_BUILD_ROOT\)\/images\/j2sdk-image\/jre\/lib\/rt.jar -jcov\/source:\$\(ABS_PLATFORM_BUILD_ROOT\)\/images\/j2sdk-image\/src.zip -jcov\/include:*/" Makefile + else + cd "$OPENJDK_DIR/test" || exit + # TODO pass in correct jcov parameter for jdk9 and up + # sed -i "s/-vmoption:-Xmx512m.*/-vmoption:-Xmx512m -xml:verify -jcov\/classes:\$\(ABS_PLATFORM_BUILD_ROOT\)\/jdk\/classes\/ -jcov\/source:\$\(ABS_PLATFORM_BUILD_ROOT\)\/..\/..\/jdk\/src\/share\/classes -jcov\/include:*/" TestCommon.gmk + fi pwd - - sed -i "s/-vmoption:-Xmx512m.*/-vmoption:-Xmx512m -xml:verify -jcov\/classes:\$\(ABS_PLATFORM_BUILD_ROOT\)\/images\/j2sdk-image\/jre\/lib\/rt.jar -jcov\/source:\$\(ABS_PLATFORM_BUILD_ROOT\)\/images\/j2sdk-image\/src.zip -jcov\/include:*/" Makefile - cd "$OPENJDK_DIR" || exit } From 8906396f74208f73d512bc6e929eb549a6acabfd Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 10 Sep 2018 11:26:58 +0100 Subject: [PATCH 004/101] Cleaner version of #320 (#393) * Use --with-freetype=bundled for JDK11 (#421) * Fix Shellcheck so it does not warn against imports * Fix timestamp pattern for consistency Signed-off-by: Keith W. Campbell * Update JDK8 build pipelines with version jdk8u181-b13 * Add AO9/J9 release pipeline (#432) * Fix odd character at end of a line and add Win32/xLinuix-LH to AO8/J9 release pipeline * Remove systemtest from AO8/J9 LinuxLH pipeline * Add release pipeline scripts in for AO9/J9 and AO10/J9 * Stop AO9/J9 and AO10/J9 running publish in parallel * Stop -dsgc passing invalid empty parameter to git * Refactor Build scripts Signed-off-by: John Oliver * Add image Signed-off-by: John Oliver * Fix refering to the wrong job in tests * Initial test of java 11 * cd to full path * Stop refering to custom repo * Try fixing gcc path * Try fixing gcc path * Try fixing gcc path * Try fixing openj9 jdk8 windows * Try removing os pinning --- .gitignore | 5 +- CHANGELOG.md | 115 ++++ README.md | 317 +++++++--- build-farm/make-adopt-build-farm.sh | 94 +++ .../platform-specific-configurations/aix.sh | 73 +++ .../platform-specific-configurations/linux.sh | 96 +++ .../platform-specific-configurations/mac.sh | 49 ++ .../windows.sh | 69 +++ .../set-platform-specific-configurations.sh | 33 ++ build-farm/sign-releases.sh | 43 ++ configureBuild.sh | 241 ++++++++ docker-build.sh | 139 +++++ docker/jdk10/x86_64/ubuntu/.gitignore | 1 - docker/jdk10/x86_64/ubuntu/Dockerfile | 12 +- .../x86_64/ubuntu/dockerConfiguration.sh | 8 + docker/jdk8/x86_64/ubuntu/.gitignore | 1 - docker/jdk8/x86_64/ubuntu/Dockerfile | 26 +- .../jdk8/x86_64/ubuntu/dockerConfiguration.sh | 8 + docker/jdk9/x86_64/ubuntu/.gitignore | 1 - docker/jdk9/x86_64/ubuntu/Dockerfile | 13 +- .../jdk9/x86_64/ubuntu/dockerConfiguration.sh | 9 + ...doptOpenJDK_Build_Script_Relationships.png | Bin 0 -> 40230 bytes makejdk-any-platform.1 | 166 ++++-- makejdk-any-platform.sh | 140 ++--- makejdk.sh | 475 --------------- .../java-tool/MercurialTracker.java | 2 +- native-build.sh | 33 ++ pipelines/build/BuildBaseFile.groovy | 296 ++++++++++ pipelines/build/createJobFromTemplate.groovy | 51 ++ pipelines/build/openjdk10_pipeline.groovy | 95 +++ pipelines/build/openjdk11_pipeline.groovy | 93 +++ pipelines/build/openjdk8_pipeline.groovy | 85 +++ pipelines/build/openjdk9_pipeline.groovy | 87 +++ pipelines/build/openjdk_build_pipeline.groovy | 28 + sbin/build.sh | 547 +++++++++--------- sbin/build.template | 49 ++ sbin/common-functions.sh | 196 ------- sbin/common/common.sh | 68 +++ sbin/common/config_init.sh | 349 +++++++++++ sbin/{colour-codes.sh => common/constants.sh} | 31 +- sbin/jtreg.sh | 170 ------ sbin/jtreg_prep.sh | 169 ------ sbin/prepareWorkspace.sh | 342 +++++++++++ sign.sh | 137 +++++ sbin/signalhandler.sh => signalhandler.sh | 15 +- 45 files changed, 3409 insertions(+), 1568 deletions(-) create mode 100644 CHANGELOG.md create mode 100755 build-farm/make-adopt-build-farm.sh create mode 100755 build-farm/platform-specific-configurations/aix.sh create mode 100755 build-farm/platform-specific-configurations/linux.sh create mode 100755 build-farm/platform-specific-configurations/mac.sh create mode 100755 build-farm/platform-specific-configurations/windows.sh create mode 100755 build-farm/set-platform-specific-configurations.sh create mode 100755 build-farm/sign-releases.sh create mode 100755 configureBuild.sh create mode 100755 docker-build.sh delete mode 100644 docker/jdk10/x86_64/ubuntu/.gitignore create mode 100644 docker/jdk10/x86_64/ubuntu/dockerConfiguration.sh delete mode 100644 docker/jdk8/x86_64/ubuntu/.gitignore create mode 100644 docker/jdk8/x86_64/ubuntu/dockerConfiguration.sh delete mode 100644 docker/jdk9/x86_64/ubuntu/.gitignore create mode 100644 docker/jdk9/x86_64/ubuntu/dockerConfiguration.sh create mode 100644 images/AdoptOpenJDK_Build_Script_Relationships.png delete mode 100755 makejdk.sh create mode 100755 native-build.sh create mode 100644 pipelines/build/BuildBaseFile.groovy create mode 100644 pipelines/build/createJobFromTemplate.groovy create mode 100644 pipelines/build/openjdk10_pipeline.groovy create mode 100644 pipelines/build/openjdk11_pipeline.groovy create mode 100644 pipelines/build/openjdk8_pipeline.groovy create mode 100644 pipelines/build/openjdk9_pipeline.groovy create mode 100644 pipelines/build/openjdk_build_pipeline.groovy create mode 100644 sbin/build.template delete mode 100644 sbin/common-functions.sh create mode 100755 sbin/common/common.sh create mode 100755 sbin/common/config_init.sh rename sbin/{colour-codes.sh => common/constants.sh} (54%) mode change 100644 => 100755 delete mode 100755 sbin/jtreg.sh delete mode 100755 sbin/jtreg_prep.sh create mode 100755 sbin/prepareWorkspace.sh create mode 100755 sign.sh rename sbin/signalhandler.sh => signalhandler.sh (67%) diff --git a/.gitignore b/.gitignore index 46882055d..37c58ce83 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,2 @@ -openjdk/ -target .idea/ -alsa-lib* -freetype* +workspace diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 000000000..1ed1cc95a --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,115 @@ +# Changelog for openjdk-build scripts + +## Version 1.0.0 (14 May 2018) + +See [Commit History](https://github.com/AdoptOpenJDK/openjdk-build/commits/master) +up until May the 14th 2018. + +## Version 2.0.0 (TBA) + +A major overhaul to split out Docker and Native builds, fix a host of small +issues and place build jobs into Groovy Pipeline scripts. + +### Core Build Changes + +1. _configureBuild.sh_ added. The pre-build configuration logic now resides in +this script. +1. _native-build.sh_ added. This script is invoked for building (Adopt) OpenJDK +binaries natively. +1. _docker-build.sh_ added. This script is invoked for building (Adopt) OpenJDK +binaries in a Docker container. +1. _sbin/prepareWorkspace.sh_ added +1. _sbin/colour-codes.sh_ removed to simplify code +1. _makejdk.sh_ removed - please use _makejdk-any-platform.sh_ or (rarely) +_sbin/build.sh_ instead. +1. _sbin/common-functions.sh_ removed and its logic split +1. _sbin/common/common.sh_ added +1. _sbin/common/config_init.sh_ added +1. _sbin/common/constants.sh_ added +1. _sbin/build.template_ added for saving off the configure configuration. +1. _sbin/signalhandler.sh_ moved to _signalhandler.sh_ +1. _sbin/build.sh_ enhanced, now requires a 'saved' build configuration to run. +This 'saved' build configuration is created by _makejdk-any-platform.sh_ but +can be generated manually as well. +1. _sign.sh_ added for code signing functionality. + + +#### _makejdk-any-platform.sh_, _build.sh_, _makejdk.sh_ usage changes + +1. More versions added, `jdk8u | jdk9 | jdk10 | jfx | amber` are now all supported +1. `-B` is now used for specifying the build number (long form `--build-number`). +1. `-bv` is removed, (long form `--variant` changes to `--build-variant`). +1. `-c` (long form `--clean-docker-build`) added to build from a clean docker container. +1. `-ca` changes to `-C`, (long form `--configure-args` stays the same). +1. `-D` (long form `--docker`) added for building in a docker container. +1. `-dsgc` is removed, (long form `--disable-shallow-git-clone` stays the same). +1. `-ftd` changes to `-f`, (long form `--freetype-dir` stays the same). +1. `-h` (long form `--help`) added. +1. `-i` (long form `--ignore-container`) added to ignore existing docker container. +1. `-j, --jtreg` and `-js, --jtreg-subsets` are removed as tests should be run +via the openjdk-tests repo / project. +1. `-J` (long form `--jdk-boot-dir` added to set JDK boot dir. +1. `-nc` changes to `-n`, (long form `--no-colour` stays the same). +1. `-p` (long form `--processors`) added to set number of processors in docker build. +1. `-sf` changes to `-F`, (long form `--skip-freetype` stays the same). +1. `--sudo` added to run the docker container as root. +1. `--tmp-space-build` (set a temporary build space if regular workspace is unavailable). +1. `-T` (long form `--target-file-name` added to specify the final name of the binary. +1. `-u` (long form `--update-version`) added to specify the update version. +1. `-V` (long form `--jvm-variant` specify the JVM variant (server or client). + +Please see _makejdk-any-platform.1_ man page for full details. + +### Test Changes + +1. _sbin/jtreg.sh_ removed (superseded by the openjdk-tests project). +1. _sbin/jtreg_prep.sh_ removed (superseded by the openjdk-tests project). + +### Docker Support + +1. `-D` (long form `--docker`) has been added for building in a docker container. +1. `-c` (long form `--clean-docker-build`) has been added to build from a clean +docker container. +1. `-i` (long form `--ignore-container`) has been added to ignore existing docker +container. +1. `-p` (long form `--processors`) added to set number of processors in docker build. +1. `--sudo` added to run the docker container as root. +1. _docker-build.sh_ added. This script is invoked for building (Adopt) OpenJDK +binaries in a Docker container. +1. _docker/jdk/x86_64/ubuntu/Dockerfile_ updated for various bug fixes. +1. _docker/jdk/x86_64/ubuntu/dockerConfiguration.sh_ files added. These +contain Docker specific environment variables that the build scripts need (as +opposed to falsely picking up the underlying native env). + +### Build Farm Support + +1. New _build-farm/make-adopt-build-farm.sh_ added for the new AdoptOpenJDK +Build Farm jenkins pipeline to build Adopt OpenJDK binaries. Sets the default +environment variables that are currently set in individual jobs. This allows +us to now track and version these variables. +1. New _build-farm/set-platform-specific-configurations.sh_ added for the new +AdoptOpenJDK Build Farm jenkins pipeline to build Adopt OpenJDK binaries. Sets +the default environment variables that are currently set in individual jobs. +This allows us to now track and version these variables. +1. New _build-farm/set-platform-specific-configurations/.sh added for +the new AdoptOpenJDK Build Farm jenkins pipeline to build Adopt OpenJDK binaries. +Sets the default environment variables for specific platforms that are currently +set in individual jobs. This allows us to now track and version these variables. +1. New _build-farm/sign-releases.sh added for the new AdoptOpenJDK Build Farm +jenkins pipeline to code sign Adopt OpenJDK binaries (Mac and Windows for now). +1. _pipelines/build/BuildBaseFile.groovy_ added. This co-ordinates the various + pipeline builds. +1. _pipelines/build/openjdk\_build\_pipeline.groovy_ added. This forms the base +pipeline code for each build. +1. _pipelines/build/openjdk\\_\\_\\_pipeline.groovy_ +files added. These will eventually replace the existing individual jobs with a +Pipeline for each version and variant. + +### Documentation and Misc + +1. _README.md_ updated to reflect new scripts +1. _images/AdoptOpenJDK_Build_Script_Relationships.png_ added to show script +relationship. +1. _.gitignore_ changed to reflect new `workspace` base directory, please check +your local .gitignore for the diff. +1. _makejdk-any-platform.1_ man page updated to reflect new script usage diff --git a/README.md b/README.md index 6ef6b214f..bb4f81a34 100644 --- a/README.md +++ b/README.md @@ -2,109 +2,278 @@ [![Build Status](https://travis-ci.org/AdoptOpenJDK/openjdk-build.svg?branch=master)](https://travis-ci.org/AdoptOpenJDK/openjdk-build) [![Slack](https://slackin-jmnmplfpdu.now.sh/badge.svg)](https://slackin-jmnmplfpdu.now.sh/) -AdoptOpenJDK makes use of these scripts to build binaries on the build farm at http://ci.adoptopenjdk.net which produces OpenJDK binaries for consumption via -https://www.adoptopenjdk.net and https://api.adoptopenjdk.net +AdoptOpenJDK makes use of these scripts to build binaries on the build farm at +http://ci.adoptopenjdk.net, which produces OpenJDK binaries for consumption via +https://www.adoptopenjdk.net and https://api.adoptopenjdk.net. + +## TLDR I want to build a JDK NOW! + +##### Build jdk natively on your system + +``` +./makejdk-any-platform.sh +i.e: +./makejdk-any-platform.sh jdk8u +``` + +##### Build jdk inside a docker container +``` +./makejdk-any-platform.sh --docker jdk8u +``` +If you need sudo to run docker on your system. +``` +./makejdk-any-platform.sh --sudo --docker jdk8u +``` + +## Build +The build has 2 modes, native and docker. + +### Native +Native builds run on whatever platform the script is invoked on, i.e +if you invoke a native build on MacOS it will build a JDK for MacOS. + +### Docker +This runs a build inside a docker container. Currently this will always +build a linux JDK. ## Repository contents -This repository contains several useful scripts in order to build OpenJDK personally or at build farm scale. +This repository contains several useful scripts in order to build OpenJDK +personally or at build farm scale. + +1. The `build-farm` folder contains shell scripts for multi configuration Jenkins +build jobs used for building Adopt OpenJDK binaries. TODO This may get removed. +2. The `docker` folder contains DockerFiles which can be used as part of building +OpenJDK inside a Docker container. +3. The `git-hg` folder contains scripts to clone an OpenJDK mercurial forest into +a GitHub repo ()and regularly update it). +4. The `images` folder contains diagrams to aid understanding. +5. The `mercurial-tags/java-tool` folder contains scripts for TODO. +6. The `pipelines` folder contains the Groovy pipeline scripts for Jenkins +(e.g. build | test | checksum | release). +7. The `sbin` folder contains the scripts that actually build (AdoptOpenJDK). +`build.sh` is the entry point which can be used stand alone but is typically +called by the `native-build.sh` or `docker-build.sh` scripts (which themselves +are typically called by `makejdk-any-platform.sh`). +8. The `security` folder contains a script and `cacerts` file that is bundled +with the JDK and used when building OpenJDK: the `cacerts` file is an important +file that's used to enable SSL connections. + +## The makejdk-any-platform.sh script + +`makejdk-any-platform.sh` is the entry point for building (Adopt) OpenJDK binaries. +Building natively or in a docker container are both supported. This script (and +its supporting scripts) have defaults, but you can override these as needed. +The scripts will auto detect the platform and architecture it is running on and +configure the OpenJDK build accordingly. The supporting scripts will also +download and locally install any required dependencies for the OpenJDK build, +e.g. The ALSA sound and Freetype font libraries. + +Many of the configuration options are passed through to the `configure` and +`make` commands that OpenJDK uses to build binaries. Please see the appropriate +_README-builds.html_ file for the OpenJDK source repository that you are building. + +**NOTE:** Usage can be found via `makejdk-any-platform.sh --help`. Here is the +man page re-formatted for convenience. + +``` +USAGE + +./makejdk-any-platform [options] version + +Please visit https://www.adoptopenjdk.net for further support. + +VERSIONS + +jdk8u - Build Java 8, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk8u +jdk9u - Build Java 9, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk9u +jdk10u - Build Java 10, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk10u +jdk11 - Build Java 10, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk11 +jfx - Build OpenJFX, defaults to https://github.com/AdoptOpenJDK/openjdk-jfx +amber - Build Project Amber, defaults to https://github.com/AdoptOpenJDK/openjdk-amber + +OPTIONS + +-b, --branch +specify a custom branch to build from, e.g. dev. +For reference, AdoptOpenJDK GitHub source repos default to the dev +branch which may contain a very small diff set to the master branch +(which is a clone from the OpenJDK mercurial forest). + +-B, --build-number +specify the OpenJDK build number to build from, e.g. b12. +For reference, OpenJDK version numbers look like 1.8.0_162-b12 (for Java 8) or +9.0.4+11 (for Java 9+) with the build number being the suffix at the end. + +-c, --clean-docker-build +removes the existing docker container and persistent volume before starting +a new docker based build. + +-C, --configure-args +specify any custom user configuration arguments. + +-d, --destination +specify the location for the built binary, e.g. /path/. +This is typically used in conjunction with -T to create a custom path +/ file name for the resulting binary. + +-D, --docker +build OpenJDK in a docker container. + +--disable-shallow-git-clone +disable the default fB--depth=1 shallow cloning of git repo(s). + +-f, --freetype-dir +specify the location of an existing FreeType library. +This is typically used in conjunction with -F. -1. The `docker` folder contains a Docker file which can be used to create a Docker container for building OpenJDK -2. The `git-hg` folder contains scripts to clone an OpenJDK mercurial forest into a GitHub repo and regularly update it -3. The `mercurial-tags/java-tool` folder contains scripts for TODO -4. The `pipelines` folder contains the Groovy pipeline scripts for Jenkins (e.g. build | test | checksum |release) -5. The `sbin` folder contains the scripts called by the main script. -6. The `security` folder contains a script and `cacerts` file that is bundled with the JDK and used when building OpenJDK: the `cacerts` file is an important -file that's used to enable SSL connections +-F, --skip-freetype +skip building Freetype automatically. +This is typically used in conjunction with -f. -The main script to build OpenJDK is `makejdk-any-platform.sh` +-i, --ignore-container +ignore the existing docker container if you have one already. + -J, --jdk-boot-dir +specify the JDK boot dir. +For reference, OpenJDK needs the previous version of a JDK in order to build +itself. You should select the path to a JDK install that is N-1 versions below +the one you are trying to build. + +-k, --keep +if using docker, keep the container after the build. + +-n, --no-colour +disable colour output. + +-p, --processors +specify the number of processors to use for the docker build. + +-r, --repository +specify the repository to clone OpenJDK source from, +e.g. https://github.com/karianna/openjdk-jdk8u. + +-s, --source +specify the location to clone the OpenJDK source (and dependencies) to. + +-S, --ssh +use ssh when cloning git. + +--sign +sign the OpenJDK binary that you build. + +--sudo +run the docker container as root. + +-t, --tag +specify the repository tag that you want to build OpenJDK from. + +-T, --target-file-name +specify the final name of the OpenJDK binary. +This is typically used in conjunction with -D to create a custom file +name for the resulting binary. + +-u, --update-version +specify the update version to build OpenJDK from, e.g. 162. +For reference, OpenJDK version numbers look like 1.8.0_162-b12 (for Java 8) or +9.0.4+11 (for Java 9+) with the update number being the number after the '_' +(162) or the 3rd position in the semVer version string (4). +This is typically used in conjunction with -b. + +-v, --build-variant +specify a OpenJDK build variant, e.g. openj9. +For reference, the default variant is hotspot and does not need to be specified. + +-V, --jvm-variant +specify the JVM variant (server or client), defaults to server. + +Example usage: + +./makejdk-any-platform --docker jdk8u +./makejdk-any-platform -T MyOpenJDK10.tar.gz jdk10 + +``` + +### Script Relationships + +![Build Variant Workflow](images/AdoptOpenJDK_Build_Script_Relationships.png) + +The main script to build OpenJDK is `makejdk-any-platform.sh`, which itself uses +and/or calls `configureBuild.sh`, `docker-build.sh` and/or `native-build.sh`. + +The structure of a build is: + + 1. Configuration phase determines what the configuration of the build is based on your current +platform and and optional arguments provided + 1. Configuration is written out to `built_config.cfg` + 1. Build is kicked off by either creating a docker container or running the native build script + 1. Build reads in configuration from `built_config.cfg` + 1. Downloads source, dependencies and prepares build workspace + 1. Configure and invoke OpenJDK build via `make` + 1. Package up built artifacts + +- Configuration phase is primarily performed by `configureBuild.sh` and `makejdk-any-platform.sh`. +- If a docker container is required it is built by `docker-build.sh`. +- In the build phase `sbin/build.sh` is invoked either natively or inside the docker container. +`sbin/build.sh` invokes `sbin/prepareWorkspace.sh` to download dependencies, source and perform +general preparation. +- Rest of the build and packaging is then handled from `sbin/build.sh` + ## Building OpenJDK ### Building on the Build Farm In order to build an OpenJDK variant on the build farm you need to follow the -[Adding-a-new-build-variant](https://github.com/AdoptOpenJDK/TSC/wiki/Adding-a-new-build-variant) instructions. +[Adding-a-new-build-variant](https://github.com/AdoptOpenJDK/TSC/wiki/Adding-a-new-build-variant) +instructions. The configuration options are often set in the Jenkins job and +passed into `makejdk-any-platform.sh` script. -### Building locally via Docker +Note that the build nodes (list of hosts on the LH side) also have configuration +where things like the BOOT_JDK environment variable is set. -**WARN: As of 23rd March 2018 these instructions do not work, there are several issues that need resolving** +### Building via Docker in your local environment -Make sure you have started your Docker Daemon first! For help with getting docker follow the instructions [here](https://docs.docker.com/engine/installation/). -Once you have Docker started you can then use the script below to build OpenJDK. +The simplest way to build OpenJDK using these scripts is to run `makejdk-any-platform.sh` +and have your user be in the Docker group on the machine (or use the `--sudo` +option to prefix all of your Docker commands with `sudo`). This script will create +a Docker container that will be configured with all of the required dependencies +and a base operating system in order to build OpenJDK. -``` -Usage: ./makejdk-any-platform.sh --version [version] [options] - -Versions: - jdk8u - https://github.com/AdoptOpenJDK/openjdk-jdk8u - jdk9u - https://github.com/AdoptOpenJDK/openjdk-jdk9u - jdk10u - https://github.com/AdoptOpenJDK/openjdk-jdk10u - jdk11 - https://github.com/AdoptOpenJDK/openjdk-jdk11 - -Options: - -s, --source specify the location for the source and dependencies to be cloned, defaults to ./openjdk. If it is specified, docker is not used - -d, --destination specify the location for the tarball (eg. /path/ or /path/here.tar.gz) - -r, --repository specify a custom repository (eg. username/openjdk-jdk8u) - -b, --branch specify a custom branch (eg. dev) - -k, --keep reuse docker container (prevents deleting) - -j, --jtreg run jtreg after building - -js, --jtreg-subsets select one or more jtreg tests to run - -S, --ssh use ssh when cloning git - -sf --skip-freetype skip building freetype - -nc --no-colour disable colour output - -ftd --freetype-dir specify the location of an existing FreeType library that can be used for the OpenJDK build process - -dsgc --disable-shallow-git-clone disable shallow cloning of git repo(s) using the --depth=1 CLI option - -bv --variant specify a build variant name, e.g. openj9 - -c --clean-docker-build clean docker data volume - -t --tag specify a custom tag - --sign specify the location for the windows p12 certificate. Used only for windows builds to sign DLL - -ca --configure-args specify a custom configuration arguments -``` - -The simplest way to build OpenJDK using our scripts is to run `makejdk-any-platform.sh` and have your user be in the Docker group on the machine -(or prefix all of your Docker commands with `sudo`). This script will create a Docker container that will be configured with all of the required -dependencies and a base operating system in order to build OpenJDK. For example: +Make sure you have started your Docker Daemon first! For help with getting +docker follow the instructions [here](https://docs.docker.com/engine/installation/). +Once you have Docker started you can then use the script below to build OpenJDK. -`./makejdk-any-platform.sh -c --ssh --version jdk8u` +Example Usage (TODO Add example of openj9): -* **NOTE:** If you don't use SSH keys (if you do then pass `-ssh`) to connect to GitHub then the script will challenge you for your GitHub username and password. -* **NOTE:** The script will clone source code into the `--source` directory (defaults to `openjdk`). -* **NOTE:** By default the docker container is removed each time and your build will be copied from the container to the host. -To override this behaviour, specify the `-k` or `--keep` option. -* **NOTE:** The entire process will take some time, especially if you have not saved the Docker image from a previous run. -* **NOTE:** If you set the `-d` option it will pass that through to `makejdk.sh`, the resulting zipped tarball will be copied to the value for -d, for example: -`makejdk.sh /target/directory` will result in the JDK being built inside of your Docker container and then copied to `/target/directory` on the host +`./makejdk-any-platform.sh --docker --sudo jdk8u` #### Configuring Docker for non sudo use -To use the Docker commands without using the sudo prefix, you will need to be in the Docker group which can be achieved with the following three commands +To use the Docker commands without using the `--sudo` option, you will need to be +in the Docker group which can be achieved with the following three commands (performed as `root`) 1. `sudo groupadd docker`: creates the Docker group if it doesn't already exist 2. `sudo gpasswd -a yourusernamehere docker`: adds a user to the Docker group 3. `sudo service docker restart`: restarts the Docker service so the above changes can take effect -### Building in your local environment - -Please note that your build host will need to have certain pre-requisites met. We provide Ansible scripts in the -[openjdk-infrastructure](https://www.github.com/AdoptOpenJDK/openjdk-infrastructure) project for setting these pre-requisites. - -You can use the `makejdk-any-platform.sh` script by providing two parameters: - -1. The _working directory_ (which is where files will be downloaded to: this includes a number of libraries used with OpenJDK itself such as FreeType and ALSA) -1. The _target directory_ which will be used to store the final _.tar.gz_ file containing the _j2sdk-image_ - -e.g `./makejdk-any-platform.sh -s /path/to/workspace -d /target/directory` +### Building natively in your local environment -**NOTE:** Usage can be found via `makejdk-any-platform.sh --help`, the exact usage is available for this script as well. +Please note that your build host will need to have certain pre-requisites met. +We provide Ansible scripts in the +[openjdk-infrastructure](https://www.github.com/AdoptOpenJDK/openjdk-infrastructure) +project for setting these pre-requisites. -### None of the above? +Example Usage (TODO Add example of openj9): -You can use the `makejdk.sh` script by providing two parameters: +`./makejdk-any-platform.sh -s /home/openjdk10/src -d /home/openjdk/target -T MyOpenJDK10.tar.gz jdk10` -1. The _working directory_ (which is where files will be downloaded to: this includes a number of libraries used with OpenJDK itself such as FreeType and ALSA) -1. The _target directory_ which will be used to store the final _.tar.gz_ file containing the _j2sdk-image_ +This would clone OpenJDK source from _https://github.com/AdoptOpenJDK/openjdk-jdk10_ +to `/home/openjdk10/src`, configure the build with sensible defaults according +to your local platform and then build (Adopt) OpenJDK and place the result in +`/home/openjdk/target/MyOpenJDK10.tar.gz`. -e.g `./makejdk.sh -s /path/to/workspace -d /target/directory` +### Building OpenJDK from a non Adopt source +These scripts default to using AdoptOpenJDK as the OpenJDK source repo to build +from, but you can override this with the `-r` flag. diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh new file mode 100755 index 000000000..124b12668 --- /dev/null +++ b/build-farm/make-adopt-build-farm.sh @@ -0,0 +1,94 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +set -e + +PLATFORM_SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +## Very very build farm specific configuration + +TIMESTAMP="$(date +'%Y-%m-%d-%H-%M')" + +export OPERATING_SYSTEM +OPERATING_SYSTEM=$(echo "${TARGET_OS}" | tr '[:upper:]' '[:lower:]') + +echo "BUILD TYPE: " +echo "VERSION: ${JAVA_TO_BUILD}" +echo "ARCHITECTURE ${ARCHITECTURE}" +echo "VARIANT: ${VARIANT}" +echo "OS: ${OPERATING_SYSTEM}" + +OPTIONS="" +EXTENSION="" +# shellcheck disable=SC2034 +CONFIGURE_ARGS_FOR_ANY_PLATFORM=${CONFIGURE_ARGS:-""} +BUILD_ARGS=${BUILD_ARGS:-""} +VARIANT_ARG="${JAVA_TO_BUILD}-" + +if [ -z "${JDK_BOOT_VERSION}" ] +then + echo "Detecting boot jdk for: ${JAVA_TO_BUILD}" + currentBuildNumber=$(echo "${JAVA_TO_BUILD}" | tr -d "[:alpha:]") + echo "Found build version: ${currentBuildNumber}" + JDK_BOOT_VERSION=$((currentBuildNumber-1)) + echo "Boot jdk version: ${JDK_BOOT_VERSION}" +fi + +case "${JDK_BOOT_VERSION}" in + "7") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK7_BOOT_DIR}";; + "8") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK8_BOOT_DIR}";; + "9") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK9_BOOT_DIR}";; + "10") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK10_BOOT_DIR}";; + "11") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK11_BOOT_DIR}";; + "home") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JAVA_HOME}";; + *) export JDK_BOOT_DIR="${JDK_BOOT_VERSION}";; +esac + +echo "Boot jdk: ${JDK_BOOT_DIR}" + + +if [ "${OPERATING_SYSTEM}" == "linux" ] ; then + EXTENSION="tar.gz" + + if [ ! -z "${TAG}" ]; then + OPTIONS="${OPTIONS} --tag $TAG" + fi +elif [ "${OPERATING_SYSTEM}" == "aix" ] ; then + EXTENSION="tar.gz" +elif [ "${OPERATING_SYSTEM}" == "mac" ] ; then + EXTENSION="tar.gz" +elif [ "${OPERATING_SYSTEM}" == "windows" ] ; then + EXTENSION=zip +fi + +# shellcheck source=build-farm/set-platform-specific-configurations.sh +source "${PLATFORM_SCRIPT_DIR}/set-platform-specific-configurations.sh" + +# Set the file name +JAVA_TO_BUILD_UPPERCASE=$(echo "${JAVA_TO_BUILD}" | tr '[:lower:]' '[:upper:]') + + +if [ ! -z "${ADDITIONAL_FILE_NAME_TAG}" ]; then + FILENAME="Open${JAVA_TO_BUILD_UPPERCASE}_${ARCHITECTURE}_${OPERATING_SYSTEM}_${VARIANT}_${ADDITIONAL_FILE_NAME_TAG}_${TIMESTAMP}.${EXTENSION}" +else + FILENAME="Open${JAVA_TO_BUILD_UPPERCASE}_${ARCHITECTURE}_${OPERATING_SYSTEM}_${VARIANT}_${TIMESTAMP}.${EXTENSION}" +fi + +echo "Filename will be: $FILENAME" + +# shellcheck disable=SC2086 +bash "$PLATFORM_SCRIPT_DIR/../makejdk-any-platform.sh" --clean-git-repo --jdk-boot-dir "${JDK_BOOT_DIR}" --configure-args "${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" --target-file-name "${FILENAME}" ${TAG_OPTION} ${OPTIONS} ${BUILD_ARGS} ${VARIANT_ARG} "${JAVA_TO_BUILD}" + diff --git a/build-farm/platform-specific-configurations/aix.sh b/build-farm/platform-specific-configurations/aix.sh new file mode 100755 index 000000000..769718811 --- /dev/null +++ b/build-farm/platform-specific-configurations/aix.sh @@ -0,0 +1,73 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/../../sbin/common/constants.sh" + +export PATH="/opt/freeware/bin:/usr/local/bin:/opt/IBM/xlC/13.1.3/bin:/opt/IBM/xlc/13.1.3/bin:$PATH" +export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-memory-size=18000 --with-cups-include=/opt/freeware/include --with-extra-ldflags=-lpthread --with-extra-cflags=-lpthread --with-extra-cxxflags=-lpthread" +export BUILD_ARGS="${BUILD_ARGS} --skip-freetype" + +if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "openj9" ]; +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} DF=/usr/sysv/bin/df" + + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/ramdisk0/build/workspace/openjdk8_openj9_build_ppc64_aix/freemarker-2.3.8/lib/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/ramdisk0/build/workspace/openjdk9_openj9_build_ppc64_aix/freemarker-2.3.8/lib/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/ramdisk0/build/workspace/openjdk10_openj9_build_ppc64_aix/freemarker-2.3.8/lib/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/ramdisk0/build/workspace/openjdk10_openj9_build_ppc64_aix/freemarker-2.3.8/lib/freemarker.jar DF=/usr/sysv/bin/df" + fi +fi + +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ]; +then + export JDK10_BOOT_DIR="$PWD/jdk-10" + if [ ! -d "$JDK10_BOOT_DIR/bin" ]; then + mkdir -p "$JDK10_BOOT_DIR" + wget -q -O - "https://api.adoptopenjdk.net/v2/binary/releases/openjdk10?os=aix&release=latest&arch=${ARCHITECTURE}" | tar xpzf - --strip-components=2 -C "$JDK10_BOOT_DIR" + fi + export JDK_BOOT_DIR=$JDK10_BOOT_DIR + + + if [ "${VARIANT}" == "hotspot" ]; then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} DF=/usr/sysv/bin/df" + fi + + if [ "${VARIANT}" == "openj9" ]; then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-warnings-as-errors" + if [ -r /usr/local/gcc/bin/gcc-7.3 ]; then + # Following line ensures the latest ccache is picked up too + export PATH=/usr/local/gcc/bin:$PATH + export CC=gcc-7.3 + export CXX=g++-7.3 + export LD_LIBRARY_PATH=/usr/local/gcc/lib64:/usr/local/gcc/lib + fi + fi + + export LANG=C + export PATH=/opt/freeware/bin:$JAVA_HOME/bin:/usr/local/bin:/opt/IBM/xlC/13.1.3/bin:/opt/IBM/xlc/13.1.3/bin:$PATH + +fi diff --git a/build-farm/platform-specific-configurations/linux.sh b/build-farm/platform-specific-configurations/linux.sh new file mode 100755 index 000000000..0b50caf78 --- /dev/null +++ b/build-farm/platform-specific-configurations/linux.sh @@ -0,0 +1,96 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/../../sbin/common/constants.sh" + +if [ "${ARCHITECTURE}" == "x64" ] +then + export PATH=/opt/rh/devtoolset-2/root/usr/bin:$PATH + + if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] + then + [ -r /usr/local/gcc/bin/gcc-7.3 ] && export CC=/usr/local/gcc/bin/gcc-7.3 + [ -r /usr/local/gcc/bin/g++-7.3 ] && export CXX=/usr/local/gcc/bin/g++-7.3 + export LD_LIBRARY_PATH=/usr/local/gcc/lib64:/usr/local/gcc/lib + fi +fi + +if [ "${ARCHITECTURE}" == "s390x" ] +then + export LANG=C + + if [ "${VARIANT}" == "openj9" ] + then + export PATH="/usr/bin:$PATH" + + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + if which g++-4.8; then + export CC=gcc-4.8 + export CXX=g++-4.8 + fi + fi + fi +fi + +if [ "${ARCHITECTURE}" == "ppc64le" ] +then + export LANG=C +fi + +if [ "${ARCHITECTURE}" == "arm" ] +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="--with-jobs=4 --with-memory-size=2000" +fi + +if [ "${ARCHITECTURE}" == "aarch64" ] +then + export BUILD_ARGS="${BUILD_ARGS} --skip-freetype" +fi + +if [ "${ARCHITECTURE}" == "s390x" ] || [ "${ARCHITECTURE}" == "ppc64le" ] +then + if [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] && [ "${VARIANT}" == "openj9" ] + then + if [ -z "$JDK9_BOOT_DIR" ]; then + export JDK9_BOOT_DIR="$PWD/jdk-9+181" + if [ ! -r "$JDK9_BOOT_DIR" ]; then + wget -O - https://github.com/AdoptOpenJDK/openjdk9-releases/releases/download/jdk-9%2B181/OpenJDK9_s390x_Linux_jdk-9.181.tar.gz | tar xpfz - + fi + fi + + export JDK_BOOT_DIR=$JDK9_BOOT_DIR + export CC=gcc-4.8 + export CXX=g++-4.8 + fi +fi + + +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] +then + export JDK10_BOOT_DIR="$PWD/jdk-10" + if [ ! -d "$JDK10_BOOT_DIR/bin" ]; then + downloadArch="${ARCHITECTURE}" + [ "$downloadArch" == "arm" ] && downloadArch="arm32" + + mkdir -p "$JDK10_BOOT_DIR" + wget -q -O - "https://api.adoptopenjdk.net/v2/binary/releases/openjdk10?os=linux&release=latest&arch=${downloadArch}" | tar xpzf - --strip-components=2 -C "$JDK10_BOOT_DIR" + fi + export JDK_BOOT_DIR=$JDK10_BOOT_DIR + +fi \ No newline at end of file diff --git a/build-farm/platform-specific-configurations/mac.sh b/build-farm/platform-specific-configurations/mac.sh new file mode 100755 index 000000000..c3420eccc --- /dev/null +++ b/build-farm/platform-specific-configurations/mac.sh @@ -0,0 +1,49 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/../../sbin/common/constants.sh" + +export MACOSX_DEPLOYMENT_TARGET=10.8 +export BUILD_ARGS="${BUILD_ARGS}" + +XCODE_SWITCH_PATH="/"; + +if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] +then + XCODE_SWITCH_PATH="/Applications/Xcode.app" +fi +sudo xcode-select --switch "${XCODE_SWITCH_PATH}" + + +if [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] +then + export PATH="/Users/jenkins/ccache-3.2.4:$PATH" +fi + + +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-extra-cxxflags=-mmacosx-version-min=10.8" + + export JDK10_BOOT_DIR="$PWD/jdk-10" + if [ ! -d "$JDK10_BOOT_DIR/bin" ]; then + mkdir -p "$JDK10_BOOT_DIR" + wget -q -O - 'https://api.adoptopenjdk.net/v2/binary/releases/openjdk10?os=mac&release=latest' | tar xpzf - --strip-components=2 -C "$JDK10_BOOT_DIR" + fi + export JDK_BOOT_DIR=$JDK10_BOOT_DIR +fi \ No newline at end of file diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh new file mode 100755 index 000000000..65839a44e --- /dev/null +++ b/build-farm/platform-specific-configurations/windows.sh @@ -0,0 +1,69 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/../../sbin/common/constants.sh" + +export ANT_HOME=/cygdrive/C/Projects/OpenJDK/apache-ant-1.10.1 +export ALLOW_DOWNLOADS=true +export LANG=C +export JAVA_HOME=$JDK_BOOT_DIR +export BUILD_ARGS="--tmp-space-build ${BUILD_ARGS}" + + + + +if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] +then + export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/C/Projects/OpenJDK/make-3.82/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} with_freetype=/cygdrive/C/openjdk/freetype --disable-ccache" +elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] +then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype=/cygdrive/C/openjdk/freetype --disable-ccache" +elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] +then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --disable-ccache" +elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] +then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013 --disable-ccache" +fi + + +if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "openj9" ] +then + export PATH="/usr/bin:$PATH" + export HAS_AUTOCONF=1 + + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;%INCLUDE%" + export PATH="$PATH:/c/cygwin64/bin" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib64 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + fi +fi diff --git a/build-farm/set-platform-specific-configurations.sh b/build-farm/set-platform-specific-configurations.sh new file mode 100755 index 000000000..cd569afd3 --- /dev/null +++ b/build-farm/set-platform-specific-configurations.sh @@ -0,0 +1,33 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/../sbin/common/constants.sh" + +if [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-warnings-as-errors" +fi + +if [ "${VARIANT}" != "hotspot" ] +then + export VARIANT_ARG="--build-variant ${VARIANT}" +fi + +# shellcheck disable=SC1091,SC1090 +source "$SCRIPT_DIR/platform-specific-configurations/${OPERATING_SYSTEM}.sh" \ No newline at end of file diff --git a/build-farm/sign-releases.sh b/build-farm/sign-releases.sh new file mode 100755 index 000000000..17be2fa82 --- /dev/null +++ b/build-farm/sign-releases.sh @@ -0,0 +1,43 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + + +BUILD_ARGS=${BUILD_ARGS:-""} +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +export OPERATING_SYSTEM + +if [ "${OPERATING_SYSTEM}" == "mac" ] ; then + EXTENSION="tar.gz" +elif [ "${OPERATING_SYSTEM}" == "windows" ] ; then + EXTENSION="zip" +else + echo "OS does not need signing ${OPERATING_SYSTEM}" + exit 0 +fi + +echo "files:" +ls -alh workspace/target/ + +echo "OpenJDK*.${EXTENSION}" + +find workspace/target/ -name "OpenJDK*.${EXTENSION}" | while read -r file; +do + echo "signing ${file}" + + # shellcheck disable=SC2086 + bash "${SCRIPT_DIR}/../sign.sh" ${CERTIFICATE} "${file}" +done \ No newline at end of file diff --git a/configureBuild.sh b/configureBuild.sh new file mode 100755 index 000000000..18086eebc --- /dev/null +++ b/configureBuild.sh @@ -0,0 +1,241 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +################################################################################ + +################################################################################ +# +# This script sets up the initial configuration for an (Adopt) OpenJDK Build. +# See the configure_build function and its child functions for details. +# It's sourced by the makejdk-any-platform.sh script. +# +################################################################################ + +set -eu + +# i.e. Where we are +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/sbin/common/constants.sh" + +# shellcheck source=sbin/common/common.sh +source "$SCRIPT_DIR/sbin/common/common.sh" + +# Bring in the source signal handler +sourceSignalHandler() +{ + #shellcheck source=signalhandler.sh + source "$SCRIPT_DIR/signalhandler.sh" +} + +# Parse the command line arguments +parseCommandLineArgs() +{ + # Defer most of the work to the shared function in common-functions.sh + parseConfigurationArguments "$@" + + # this check is to maintain backwards compatibility and allow user to use + # -v rather than the mandatory argument + if [[ "${BUILD_CONFIG[OPENJDK_FOREST_NAME]}" == "" ]] + then + if [[ $# -eq 0 ]] + then + echo "Please provide a java version to build as an argument" + exit 1 + fi + + while [[ $# -gt 1 ]] ; do + shift; + done + + # Now that we've processed the flags, grab the mandatory argument(s) + setOpenJdkVersion "$1" + fi +} + +# Extra config for OpenJDK variants such as OpenJ9, SAP et al +# shellcheck disable=SC2153 +doAnyBuildVariantOverrides() +{ + if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]]; then + # current location of Extensions for OpenJDK9 for OpenJ9 project + local repository="ibmruntimes/openj9-openjdk-${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" + local branch="openj9" + fi + if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" == "SapMachine" ]]; then + # current location of SAP variant + local repository="SAP/SapMachine" + # sapmachine10 is the current branch for OpenJDK10 mainline + # (equivalent to jdk/jdk10 on hotspot) + local branch="sapmachine10" + fi + + BUILD_CONFIG[REPOSITORY]=${repository:-${BUILD_CONFIG[REPOSITORY]}}; + BUILD_CONFIG[BRANCH]=${branch:-${BUILD_CONFIG[BRANCH]}}; +} + +# Set the working directory for this build +setWorkingDirectory() +{ + if [ -z "${BUILD_CONFIG[WORKSPACE_DIR]}" ] ; then + if [[ "${BUILD_CONFIG[USE_DOCKER]}" == "true" ]]; + then + BUILD_CONFIG[WORKSPACE_DIR]="/openjdk/"; + else + BUILD_CONFIG[WORKSPACE_DIR]="$PWD/workspace"; + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}" || exit + fi + else + echo "Workspace dir is ${BUILD_CONFIG[WORKSPACE_DIR]}" + fi + + echo "Working dir is ${BUILD_CONFIG[WORKING_DIR]}" +} + +# shellcheck disable=SC2153 +determineBuildProperties() { + local build_type=normal + local default_build_full_name=${BUILD_CONFIG[OS_KERNEL_NAME]}-${BUILD_CONFIG[OS_ARCHITECTURE]}-${build_type}-${BUILD_CONFIG[JVM_VARIANT]}-release + + BUILD_CONFIG[BUILD_FULL_NAME]=${BUILD_CONFIG[BUILD_FULL_NAME]:-"$default_build_full_name"} +} + +# Set variables that the `configure` command (which builds OpenJDK) will need +# shellcheck disable=SC2153 +setVariablesForConfigure() { + + local openjdk_core_version=${BUILD_CONFIG[OPENJDK_CORE_VERSION]} + + # TODO Regex this in the if or use cut to grab out the number and see if >= 9 + # TODO 9 should become 9u as will 10 shortly.... + if [ "$openjdk_core_version" == "${JDK9_CORE_VERSION}" ] || [ "$openjdk_core_version" == "${JDK10_CORE_VERSION}" ] || [ "$openjdk_core_version" == "${JDK11_CORE_VERSION}" ] || [ "$openjdk_core_version" == "${AMBER_CORE_VERSION}" ]; then + local jdk_path="jdk" + local jre_path="jre" + #BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]:-"--disable-warnings-as-errors"} + elif [ "$openjdk_core_version" == "${JDK8_CORE_VERSION}" ]; then + local jdk_path="j2sdk-image" + local jre_path="j2re-image" + else + echo "Please specify a version, either jdk8u, jdk9, jdk10, amber etc, with or without a 'u' suffix. e.g. $0 [options] jdk8u" + exit 1 + fi + + BUILD_CONFIG[JDK_PATH]=$jdk_path + BUILD_CONFIG[JRE_PATH]=$jre_path +} + +# Set the repository to build from +setRepository() { + local repository="${BUILD_CONFIG[REPOSITORY]:-adoptopenjdk/openjdk-${BUILD_CONFIG[OPENJDK_FOREST_NAME]}}"; + repository="$(echo "${repository}" | awk '{print tolower($0)}')"; + + BUILD_CONFIG[REPOSITORY]=$repository; +} + +# Specific platforms need to have special build settings +processArgumentsforSpecificPlatforms() { + + case "${BUILD_CONFIG[OS_KERNEL_NAME]}" in + "darwin") + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] ; then + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]="false" + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]="true" + fi + ;; + esac + +} + +# Specific architectures need to have special build settings +# shellcheck disable=SC2153 +processArgumentsforSpecificArchitectures() { + local jvm_variant=server + local build_full_name="" + local make_args_for_any_platform="" + local configure_args_for_any_platform="" + + case "${BUILD_CONFIG[OS_ARCHITECTURE]}" in + "s390x") + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] && [ "${BUILD_CONFIG[BUILD_VARIANT]}" != "openj9" ]; then + jvm_variant=zero + else + jvm_variant=server + fi + + build_full_name=linux-s390x-normal-${jvm_variant}-release + make_args_for_any_platform="CONF=${build_full_name} DEBUG_BINARIES=true images" + ;; + + "ppc64le") + jvm_variant=server + build_full_name=linux-ppc64-normal-${jvm_variant}-release + # shellcheck disable=SC1083 + BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]=${BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]:="--build=$(rpm --eval %{_host})"} + ;; + + "armv7l") + jvm_variant=zero + make_args_for_any_platform="DEBUG_BINARIES=true images" + configure_args_for_any_platform="--with-jobs=${NUM_PROCESSORS}" + ;; + + "aarch64") + BUILD_CONFIG[FREETYPE_FONT_VERSION]="2.5.2" + ;; + esac + + BUILD_CONFIG[JVM_VARIANT]=${BUILD_CONFIG[JVM_VARIANT]:-$jvm_variant} + BUILD_CONFIG[BUILD_FULL_NAME]=${BUILD_CONFIG[BUILD_FULL_NAME]:-$build_full_name} + BUILD_CONFIG[MAKE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[MAKE_ARGS_FOR_ANY_PLATFORM]:-$make_args_for_any_platform} + BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]:-$configure_args_for_any_platform} +} + +# Different platforms have different default make commands +# shellcheck disable=SC2153 +setMakeCommandForOS() { + local make_command_name + case "$OS_KERNEL_NAME" in + "aix") + make_command_name="gmake" + ;; + "SunOS") + make_command_name="gmake" + ;; + esac + + BUILD_CONFIG[MAKE_COMMAND_NAME]=${BUILD_CONFIG[MAKE_COMMAND_NAME]:-$make_command_name} +} + +################################################################################ + +configure_build() { + configDefaults + + # Parse the CL Args, see ${SCRIPT_DIR}/configureBuild.sh for details + parseCommandLineArgs "$@" + + # Update the configuration with the arguments passed in, the platform etc + setVariablesForConfigure + setRepository + processArgumentsforSpecificPlatforms + processArgumentsforSpecificArchitectures + setMakeCommandForOS + + determineBuildProperties + sourceSignalHandler + doAnyBuildVariantOverrides + setWorkingDirectory +} diff --git a/docker-build.sh b/docker-build.sh new file mode 100755 index 000000000..9ed5a9871 --- /dev/null +++ b/docker-build.sh @@ -0,0 +1,139 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +################################################################################ +# +# This script deals with the configuration to build (Adopt) OpenJDK in a docker +# container. +# It's sourced by the makejdk-any-platform.sh script. +# +################################################################################ + +set -eu + +# Create a data volume called ${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]}, +# this gets mounted at /openjdk/build inside the container and is persistent +# between builds/tests unless -c is passed to this script, in which case it is +# recreated using the source in the current ./openjdk directory on the host +# machine (outside the container) +createPersistentDockerDataVolume() +{ + set +e + ${BUILD_CONFIG[DOCKER]} volume inspect "${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]}" > /dev/null 2>&1 + local data_volume_exists=$? + set -e + + if [[ "${BUILD_CONFIG[CLEAN_DOCKER_BUILD]}" == "true" || "$data_volume_exists" != "0" ]]; then + + # shellcheck disable=SC2154 + echo "Removing old volumes and containers" + ${BUILD_CONFIG[DOCKER]} rm -f "$(${BUILD_CONFIG[DOCKER]} ps -a --no-trunc | grep "${BUILD_CONFIG[CONTAINER_NAME]}" | cut -d' ' -f1)" || true + ${BUILD_CONFIG[DOCKER]} volume rm -f "${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]}" || true + + # shellcheck disable=SC2154 + echo "Creating tmp container" + ${BUILD_CONFIG[DOCKER]} volume create --name "${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]}" + fi +} + +# Build the docker container +buildDockerContainer() +{ + echo "Building docker container" + + local dockerFile="${BUILD_CONFIG[DOCKER_FILE_PATH]}/Dockerfile" + + if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" != "" && -f "${BUILD_CONFIG[DOCKER_FILE_PATH]}/Dockerfile-${BUILD_CONFIG[BUILD_VARIANT]}" ]]; then + # TODO dont modify config in build + BUILD_CONFIG[CONTAINER_NAME]="${BUILD_CONFIG[CONTAINER_NAME]}-${BUILD_CONFIG[BUILD_VARIANT]}" + echo "Building DockerFile variant ${BUILD_CONFIG[BUILD_VARIANT]}" + dockerFile="${BUILD_CONFIG[DOCKER_FILE_PATH]}/Dockerfile-${BUILD_CONFIG[BUILD_VARIANT]}" + fi + + writeConfigToFile + + ${BUILD_CONFIG[DOCKER]} build -t "${BUILD_CONFIG[CONTAINER_NAME]}" -f "${dockerFile}" . --build-arg "OPENJDK_CORE_VERSION=${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" +} + +# Execute the (Adopt) OpenJDK build inside the Docker Container +buildOpenJDKViaDocker() +{ + + # TODO This could be extracted overridden by the user if we support more + # architectures going forwards + local container_architecture="x86_64/ubuntu" + + BUILD_CONFIG[DOCKER_FILE_PATH]="docker/${BUILD_CONFIG[OPENJDK_CORE_VERSION]}/$container_architecture" + + # shellcheck disable=SC1090 + source "${BUILD_CONFIG[DOCKER_FILE_PATH]}/dockerConfiguration.sh" + + if [ -z "$(command -v docker)" ]; then + # shellcheck disable=SC2154 + echo "Error, please install docker and ensure that it is in your path and running!" + exit + fi + + echo "Using Docker to build the JDK" + + createPersistentDockerDataVolume + + # If keep is true then use the existing container (or build a new one if we + # can't find it) + if [[ "${BUILD_CONFIG[REUSE_CONTAINER]}" == "true" ]] ; then + # shellcheck disable=SC2086 + # If we can't find the previous Docker container then build a new one + if [ "$(${BUILD_CONFIG[DOCKER]} ps -a | grep -c \"${BUILD_CONFIG[CONTAINER_NAME]}\")" == 0 ]; then + echo "No docker container for reuse was found, so creating '${BUILD_CONFIG[CONTAINER_NAME]}' " + buildDockerContainer + fi + else + # shellcheck disable=SC2154 + echo "Since you specified --ignore-container, we are removing the existing container (if it exists) and building you a new one{$good}" + # Find the previous Docker container and remove it (if it exists) + ${BUILD_CONFIG[DOCKER]} ps -a | awk '{ print $1,$2 }' | grep "${BUILD_CONFIG[CONTAINER_NAME]}" | awk '{print $1 }' | xargs -I {} "${BUILD_CONFIG[DOCKER]}" rm -f {} + + # Build a new container + buildDockerContainer + fi + + # Show the user all of the config before we build + displayParams + + local hostDir; + hostDir="$(pwd)"; + + echo "Target binary directory on host machine: ${hostDir}/target" + mkdir -p "${hostDir}/workspace/target" + + local cpuSet; + cpuSet="0-$((BUILD_CONFIG[NUM_PROCESSORS] - 1))" + + # shellcheck disable=SC2140 + # Pass in the last important variables into the Docker container and call + # the /openjdk/sbin/build.sh script inside + ${BUILD_CONFIG[DOCKER]} run -lst \ + --cpuset-cpus="${cpuSet}" \ + -v "${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]}:/openjdk/build" \ + -v "${hostDir}/workspace/target":"/${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[TARGET_DIR]}" \ + -e BUILD_VARIANT="${BUILD_CONFIG[BUILD_VARIANT]}" \ + --entrypoint /openjdk/sbin/build.sh "${BUILD_CONFIG[CONTAINER_NAME]}" + + # If we didn't specify to keep the container then remove it + if [[ -z ${BUILD_CONFIG[KEEP_CONTAINER]} ]] ; then + ${BUILD_CONFIG[DOCKER]} ps -a | awk '{ print $1,$2 }' | grep "${BUILD_CONFIG[CONTAINER_NAME]}" | awk '{print $1 }' | xargs -I {} "${BUILD_CONFIG[DOCKER]}" rm {} + fi +} \ No newline at end of file diff --git a/docker/jdk10/x86_64/ubuntu/.gitignore b/docker/jdk10/x86_64/ubuntu/.gitignore deleted file mode 100644 index c97f963b3..000000000 --- a/docker/jdk10/x86_64/ubuntu/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.sh diff --git a/docker/jdk10/x86_64/ubuntu/Dockerfile b/docker/jdk10/x86_64/ubuntu/Dockerfile index bb3b41850..29241a23a 100644 --- a/docker/jdk10/x86_64/ubuntu/Dockerfile +++ b/docker/jdk10/x86_64/ubuntu/Dockerfile @@ -23,7 +23,9 @@ RUN apt-get update \ && apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 0x219BD9C9 \ && add-apt-repository 'deb http://repos.azulsystems.com/ubuntu stable main' \ && apt-get update \ + && apt-get -y upgrade \ && apt-get install -qq -y --no-install-recommends \ + git \ cpio \ make \ gcc \ @@ -51,18 +53,20 @@ RUN apt-get update \ # Pick up build instructions RUN mkdir -p /openjdk/target -ADD sbin /openjdk/sbin +COPY sbin /openjdk/sbin +COPY workspace/config /openjdk/config +RUN mkdir -p /openjdk/build +RUN useradd -ms /bin/bash build +RUN chown -R build: /openjdk/ +USER build WORKDIR /openjdk/build/ -ENV JDK_BOOT_DIR=/usr/lib/jvm/zulu-9-amd64/ - # Default actions ENTRYPOINT ["/openjdk/sbin/build.sh"] CMD ["images"] - ARG OPENJDK_VERSION ENV OPENJDK_VERSION=$OPENJDK_VERSION ENV JDK_PATH=jdk diff --git a/docker/jdk10/x86_64/ubuntu/dockerConfiguration.sh b/docker/jdk10/x86_64/ubuntu/dockerConfiguration.sh new file mode 100644 index 000000000..947c956b2 --- /dev/null +++ b/docker/jdk10/x86_64/ubuntu/dockerConfiguration.sh @@ -0,0 +1,8 @@ +#!/bin/bash +# shellcheck disable=SC2034 +# Disable for whole file + +# This config is read in by configureBuild +BUILD_CONFIG[OS_KERNEL_NAME]="linux" +BUILD_CONFIG[OS_ARCHITECTURE]="x86_64" +BUILD_CONFIG[BUILD_FULL_NAME]="linux-x86_64-normal-server-release" diff --git a/docker/jdk8/x86_64/ubuntu/.gitignore b/docker/jdk8/x86_64/ubuntu/.gitignore deleted file mode 100644 index c97f963b3..000000000 --- a/docker/jdk8/x86_64/ubuntu/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.sh diff --git a/docker/jdk8/x86_64/ubuntu/Dockerfile b/docker/jdk8/x86_64/ubuntu/Dockerfile index 8e29c929c..3d0c14008 100644 --- a/docker/jdk8/x86_64/ubuntu/Dockerfile +++ b/docker/jdk8/x86_64/ubuntu/Dockerfile @@ -12,17 +12,27 @@ # See the License for the specific language governing permissions and # limitations under the License. -FROM ubuntu:14.04 +FROM ubuntu:16.04 MAINTAINER AdoptOpenJDK -# Install required OS tools +# Install required OS tools. Yes we have to apt-get update first in order to +# get to software-properties-common, which includes the apt-add-repository +# package, so we can add the Azul repo and then apt-get update that so we cam +# get zulu-7. This is why we can't have nice things. RUN apt-get update \ + && apt-get install -y software-properties-common \ + && apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 0x219BD9C9 \ + && apt-add-repository 'deb http://repos.azulsystems.com/ubuntu stable main' \ + && apt-get update \ + && apt-get -y upgrade \ && apt-get install -qq -y --no-install-recommends \ + git \ cpio \ make \ gcc \ g++ \ + file \ libx11-dev \ libxext-dev \ libxrender-dev \ @@ -31,24 +41,26 @@ RUN apt-get update \ libcups2-dev \ libfreetype6-dev \ libasound2-dev \ - openjdk-7-jdk \ + zulu-7 \ ccache \ zip \ wget \ git \ unzip \ - realpath \ && rm -rf /var/lib/apt/lists/* # Pick up build instructions RUN mkdir -p /openjdk/target -ADD sbin /openjdk/sbin +COPY sbin /openjdk/sbin +COPY workspace/config /openjdk/config +RUN mkdir -p /openjdk/build +RUN useradd -ms /bin/bash build +RUN chown -R build: /openjdk/ +USER build WORKDIR /openjdk/build/ -ENV JDK_BOOT_DIR=/usr/lib/jvm/java-1.7.0-openjdk-amd64 - # Default actions ENTRYPOINT ["/openjdk/sbin/build.sh"] diff --git a/docker/jdk8/x86_64/ubuntu/dockerConfiguration.sh b/docker/jdk8/x86_64/ubuntu/dockerConfiguration.sh new file mode 100644 index 000000000..947c956b2 --- /dev/null +++ b/docker/jdk8/x86_64/ubuntu/dockerConfiguration.sh @@ -0,0 +1,8 @@ +#!/bin/bash +# shellcheck disable=SC2034 +# Disable for whole file + +# This config is read in by configureBuild +BUILD_CONFIG[OS_KERNEL_NAME]="linux" +BUILD_CONFIG[OS_ARCHITECTURE]="x86_64" +BUILD_CONFIG[BUILD_FULL_NAME]="linux-x86_64-normal-server-release" diff --git a/docker/jdk9/x86_64/ubuntu/.gitignore b/docker/jdk9/x86_64/ubuntu/.gitignore deleted file mode 100644 index c97f963b3..000000000 --- a/docker/jdk9/x86_64/ubuntu/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.sh diff --git a/docker/jdk9/x86_64/ubuntu/Dockerfile b/docker/jdk9/x86_64/ubuntu/Dockerfile index e8ba11a8e..7e88f78d4 100644 --- a/docker/jdk9/x86_64/ubuntu/Dockerfile +++ b/docker/jdk9/x86_64/ubuntu/Dockerfile @@ -17,7 +17,9 @@ MAINTAINER AdoptOpenJDK # Install required OS tools RUN apt-get update \ + && apt-get -y upgrade \ && apt-get install -qq -y --no-install-recommends \ + git \ cpio \ make \ gcc \ @@ -31,6 +33,7 @@ RUN apt-get update \ libelf-dev \ libcups2-dev \ libfreetype6-dev \ + libfontconfig1-dev \ libasound2-dev \ openjdk-8-jdk \ ccache \ @@ -44,18 +47,20 @@ RUN apt-get update \ # Pick up build instructions RUN mkdir -p /openjdk/target -ADD sbin /openjdk/sbin +COPY sbin /openjdk/sbin +COPY workspace/config /openjdk/config +RUN mkdir -p /openjdk/build +RUN useradd -ms /bin/bash build +RUN chown -R build: /openjdk/ +USER build WORKDIR /openjdk/build/ -ENV JDK_BOOT_DIR=/usr/lib/jvm/java-1.8.0-openjdk-amd64 - # Default actions ENTRYPOINT ["/openjdk/sbin/build.sh"] CMD ["images"] - ARG OPENJDK_CORE_VERSION ENV OPENJDK_CORE_VERSION=$OPENJDK_CORE_VERSION ENV JDK_PATH=jdk diff --git a/docker/jdk9/x86_64/ubuntu/dockerConfiguration.sh b/docker/jdk9/x86_64/ubuntu/dockerConfiguration.sh new file mode 100644 index 000000000..1d346759e --- /dev/null +++ b/docker/jdk9/x86_64/ubuntu/dockerConfiguration.sh @@ -0,0 +1,9 @@ +#!/bin/bash +# shellcheck disable=SC2034 +# Disable for whole file + +# This config is read in by configureBuild +BUILD_CONFIG[OS_KERNEL_NAME]="linux" +BUILD_CONFIG[OS_ARCHITECTURE]="x86_64" +BUILD_CONFIG[BUILD_FULL_NAME]="linux-x86_64-normal-server-release" + diff --git a/images/AdoptOpenJDK_Build_Script_Relationships.png b/images/AdoptOpenJDK_Build_Script_Relationships.png new file mode 100644 index 0000000000000000000000000000000000000000..1c84bbe8b26099fb8054d8f21121488253ff51ed GIT binary patch literal 40230 zcmdqIcQo8z6fZhNLJ}fD5HbYOd-R?t5g|lG?<7hHQAQgr1<@lq8G;~*-pkAw(d&%f z%@BsDgTY{wHObr$=av(Mh2UC!R$iF~T7Nppqe3IG70(R!r* z3;?*$3;>W1P+cPZ#oG5)H2@F_&{BW!+;3_fP8M_fiP7?{cb8Z?F2JL2IBb`iMYmTw z^mWB3wp88wuIVs(TZ!cY+v22)`g@@ljM*J_f$zznQb@xBJ{Z4Q{O9YkjRzmE0Mbe(R17{HgG8`J2l<@Y5h>c=%t}C$k^~Kcqnl; zJvlvrG`8ajoSYOS=Keo4LbK!s%IRs$A za9|{HqfX+2PO1tniwxq$<5jlM=G_|C3A2ttm5w_q!o%)WhP3T0d=iZA^gU3TdUXq9 zuCHs5x%#GQroofw2u*@00ZTE$H0J(Hgk2L|0>LCiw_V;vZmK18tr$}3?ht-FvrqGb zZNfTFC$6W?Ljhq-l9ByeQNquNP`5Mn25*fR<*yJ#CW_;;O zst+1VVv3u@)H{}J7cF#liU0Z(V|GGvLe1;o)Y9kci4#F)+gY=0dDpI4cep=K8XmFg z1S@`$v);7mxL9}Q|9vw_8*}zk(qh_9EhJwXE`d>NC^ZPliXlLtsoEFl_F8|sj|_TM zbgp4tl~N$V3Q?Q1U)fB$MwVTGJsY~7c@fJpF}RxmKyz2#r4f6=jlf$f>t!>wHQxqh z-N+|4^$6Y)((f*i`p_v&C`&h3Xqx~YYwcw2{1-0yOVEn(lYh|fE_oUQ@y2-Ne3 zZZ%CS?Y)+UZpG`0E8-!P!l6a-kK@~RW6TN-Mali9l-4|0qsRb&0hy+4AE8O#JJM_~ zw7bOxBb%#UOBd+pxclY;GC(bduV#Y9oj)Mk7K|<4kDmFs$HpIX_{QBL)Di9!*FPM2 zyi> z0A+>{U*4#y8HubXJ;Qd2g&(@pza?UeU3L-PIMqjOm}fM}uQD%9{rUzKXJ1(JPPd)}Bx1rZ=q#6%pFaH$6!?ek81?cFcU>V-!`N)_1g z`*C0gP}H#&u&wW8qAP9n z0ibuYqhl;T=FX?oN$|TkN9D$`uV%U|ZwX+A=2^vN=LOfP%N>Ok;w!v4+v9{p{7fVP zEbaUpMAytai(?dEp|1k6%sPJV7!Lew+)1ZToB$Gic8Z@>1m;H{RDc6-;OBMt;<;4j zhi2m`5A->o$AO$4p=944-b)#8Ut}Bq{IH6)eH6o#2%gHk(L5$z^+ocHaM@4AX0YIn z{7V4?;M z$gRWEa+z&>L80o)2+?tN`y?Q=?86h>Lc4lsNy9GD_9Cs_q0++UnI{uj;5fEB(xlVR zHsr210kBN5gg`#N5 z{_upJ)EzwDDGi?(+?v83S_lz+XM)6`k|dSfHPG*ZQH>_X_l+<#pT3t#4*aNA;Nd5a zAAHhDW#vn}wtFXZ_|j&q2n{BY1J`Z(X1ZjVsMQt0YxSK!w9vBEd-Qe4mmAs*msS$L zaeR*YYJO|GUPU=`@7ugovSOlcj!V;9U)dYECb4U{l2VRf>20N}{_+0uo5O|6 zh6VF$3bL?Ne7yoHEC$e&0uzn@@@Q>jAd6;?*ZT2AqATTgAg^;DTOZ7Tqe=7!p>KKe zP{dz_B>rg)!1B6LHhFzd)?J`(D_!%clAMV?Ve}Te0_%=dy+sBJx`F1ZXYTFg9&e~x zq8M$u_D<59UjjS>#;t7xcLIzjTc7UBTluUIFa^=2iC?F|{)#K=vd{EWpC`vZ2Y|)X zn#z{%E${mWlyU+c6LCEO3`bR^2K*a$8};>5v%5!TXDQyuZ%@=Ln*RxLf!s4F)n(Z` zq$UB~%+^{kXXffg>pke@x#zH^o{z9eoxB!z?NsE}@5Q$37t0AIV7(8xL#tI!Bv;Pq zU6WK*f6mNS4%}O9)^7$mvqi5{0`v-JB*eGexWSkw%|=j;FYVU*J_Qe+!;_V7 z3D7S|TGl*6o>n(26%<~O&GcVzr8%b8I`PV-B@5(tEzN}9?j@LYMHtbl0l>8OhH`5! z8BgH)vX-blp!iCTwv&*P9J2dz@v+z63`zN#L-F z18i>YXk74^J0$?XEKj)*0nV|*MC+)PCIXr;`3X-o0fjWNhkcp>%;X0T|7=FeCf)|a z-^-pu+g(pC=R~nuPhbHDs(mPL4P9_=?7-gS-P#C7Mf7ituSu~Y2~f!wj>PND!#M#K zS#=6G-^uMs&*-u=p@}}V9q|5$Rq=d5CCO^*Mr4>keAnR9{cqla+Rpa1aRmIsY;wPl z9pyd2P$ofSb`++XYenO`jEh&DRyj=O*k|czoRTk8pYehei8=8ReLq=5iDqk3H4)H@22~*Q?>@sOCFO<8ee%fB0bLo~#Q6)}{*;3iy zMim%(dOgxi{Rl-mYVseDR-w0-_#2sa=#31e9*sjxGD^f2K7c#M-dK1a9*I3iQCKCT zzZ6b@-xZJq0vh+Q^d&5Z-Cs=)NRtda@tY~S^=n(+zW@(pnNt?9ac3G_=w!?)t@l+a zJ$SJMd88!nm{=Kb@3|=1i`u&QZ~}gSEh_M9?)>Z%0H(O!P2kvz=8#n>CHwG-e)?6F z(#DPjVJaKMhXma2){R?`wt0()K&|Yzqp8Y|TjHTJtO}b;YQ%4^Wk{@a3dmCq|Sob6@5T<7Zq%hg=UDdRKJe5yJ$MnW;#(aX>6f3C?`mI@CRE@@8q|N zBl%(efoMSRr$y(fuS%KD>TE^VYph_UFV(ku7aI&^mXW3rGw6N zLd#|di&!Bxh$g|kuLfP2OHaGRE<6W^mq~!ezi7LwkRP~eutBcvbplj8UOLD%WSgfW z(+zIo3fMyes@32Y1U6j5y?{H5sr*G5d0&~xZmaqW?M-hsG6~8zxvu2&Zr1$(WR>_y z7dBGOxmY&6HXvmVuY?c750TkiL9hsEA%0*Ym}exuaYYk=@Lq;^Vw5t$b0&Sbnnkui z$Aya-rT~E0s{;`KM(a}QYL}F9gmCCaD_A1Q@kY~>hRJI~0QhY+8*4WmeVHop!=0^Q?+IL+nqPpPD;9GT!Q?-gb$p&dDs692I8w+9p3=G(PJ-m(wH%xsJM^ab9JKC z*Gm-|f&)wa^wzW7!}}*KhCA{Jl4TvbUf!)XkH5rN=R?!5ZlBE`sC}dROY$P%gH)Yox?`dqA0pHycXFIzG(O?HPIh zG%NpbLuAi$7F;dA>@f6TzB4x<#~Tq%wR$(SP)%-~pi|(rsKEe}QC*f39J>d6hgy&m z4OI$V=XqCmjb*ko!qFOHS1OczwN@s7j=;VdVKDjYQs!mvQAq$0WWu)%6@!lJa^8fPnKB>}IxiN!i|&IOP>k4sR2UxI&&QkABb*=B=R z0Y`T<2LjgjinDx)KIiaWwdh59fi@752qL+)l^Vj*eV%N1rvl{oPr#1u`sG8ZN1&Mm zWmdEShl;ear9?v}K>CJks(%55V(U3)XsMe%K;`lV2P{GY7+h4UmV*I>#y?lAqRa{Q z+j^#9S|T9fm81CWS2QN*0$ua+y*>1b2(Hu=Pkhf=jB=j{@|Y`2KbHNf)N^`_s* z<%_#Di@wW^i}UFNO}a+`LQsRCrU;I)2MtnL?Bna+>zaVF@cr$vIyHaKqP%omjZ%?; zWElWpGm;)nBa$h%=hTJz?Z+2!(l&UaQqM zds-Som42wRpzuU(imx^>4fn~BXrBe~rqNNrHDw$s8zi`4?qrlb>Wt|vOazuT6mnV; z8!e#pl$QSKw1-U@gaW?}3k@{aF_Vyc@Q0k7nT(psX3qRPKk{ebtYzE~nk3_DzG2K0 zzrLm5@vf6hq3g0SBOwtO7){#je`nOb528^(mt(_)rPjBQJj*3;4vS2h`J>bBo+Xsz z&VC8@#k8u|bZj~rx#wljulRXXZa+4twbwv9B5V3?E^JKVs4o&p5Y0?FhKkQVLJckj z^_KR_)Wy=fuY2G=YD3XvXMKJ0aMn|cqYx3qal;UXj~1gSmxaY(%s0_%BN+3NVQ>lV zNU08iutIC+I}3u&@8imJG=XuQ2xyh215cIbDI^xlxJ$ACVGNR?_Pr-0xhcwAbq9T1dI^rVR-K7&SZvWE2MmxFvS&ac#4?J8lLc zl_;^l=C`{LGVxODfngKwNsv0LWL%NhYpA5MYW;Pb_y*lp$%achSOPrQmD=l1Hht9Z zSx`0icKYmVAT0R?{QbhmozhkG+8E|!WCgSa6~l^iw-)QGuHUhuvqD<>rJ2>+qI2lb%6W)BqW}Y zdLxET+v@(0zSw|Mt*+%#-fd#K`K>|Xy~%wWMfEq<Zm$qnuk}E4nB?2qHk^z*&{>%NF~@b$vD{rFvleCjleQn zf&}qy5^GDz7MtMR=T@;7ZjL|A!@p|X{CRP>yd8b<4aYsCBwT1}v(uI_Hdhcy^g&Es z*SCi|eS*zUE7Wdi*q^%M-Ra}Qm~cLL49Z~>JN3)qk7Gi(LhPBf@YPA9#`w-P2NoxH zCAWh7k!G`48X;auSD15!d(Ew>*;h6VU_M2RU?ySksDH`0-nBA2 ztb4{d-r!O|)6Q8|>bB%4YXAF6!p-ckP52hfp##Gozo>6q_%2Ix8@GLhteP&n7hOPE zDoYr3kT*x|zfwfCAMZtZ?*3fAAv6STk?Mh8m!HjXWsk=aeGJ|(V!?S+TrPyevDqa= zqeI3uLEnIa{icbkAQy4`vGf!sI(H^LEd;_|#P2~E!}nt*O!`{Xn9DFMqCrI#!3Mio zE2e^&=zD%YkD8*ycS_mH@dXH+DWDA7ZdHW#tMHHg0ub>(%YGi+`a59l7HVs-ztYkF zQ^42>)Z`fcyp+|`|Eq7Q;h4L0uYCY|MnVwjUvy&1u;=>h?4VJ+L}l%Y4eL++i4`IZVMt zM;Wh5^qU{wHbJ%b;iP|q3SM2ATkf7{Y`gIEO8+#U%+sP;;-xo(q5jF2554WPua*r2 zv>&<|eFJk1e~*h@)TLQaO+&jQbotvn>yfx?3*E;eun}?dI-b)_Rnl*Ns+zVLgokWPeo+@J@HIA z-_+}Wek*B3P!y-;agiizlO$)(@r;!Q`9_&5++FS$xWM+x3Y1B@^Q!p;7x5dgoi|o* zrjewYLq$d2C9gpzA2e@fLlryi8)xdd)`HHw>uP1vtOiTs{4C}b+ooT9-6XzD%6w9| zJeQl37DL-F6%#G#>Kf%0ro%|HVNA-~fV6L5j#x)=k;8TeF7IA)*`>rMhAYE>}nDfwL?REUdlYY<-G$){a_S9sQPpLvHX+A%Q3*K z?FxJ4@EOLIWkWgkC`?n&;;&fG<;O*R$`}SVhy%r@F~Cvd)YH_v07#b%RM6hwScu}; zhDf)cSPLLTOO6~HgsP)U8h}VHnN_1> zusqT!q%IKs<<4k@@Gq09_d**me#NEE&$`8$txU}&$ksgvpz*hP2f5cIO^B^JImhwd zRIMLme)Z&Cet1DYV^B9xgGVqUKAJg126v>=Ta)kF1!cQ{9c6z^7v)GAVU1;;i}gGT zw4Y&t1znx%#CoS@jR(dqpEL?NMLparao{rc-w5-Rzf`jb%+%hLyYaXroQUt=74Vh- zyB8hqg2`+pRO(>u4fpCA{o}dyX6%UKr;1l9gp@jUmbF)cOm)GX3Yi$u0?zwrN zna#RC^7Ce0eXwtbKhaX~e~jIV>CIMxtdWJpq1c$;0g#ItFQ03rDr$G0GPP_QwyT>L z&Q{4KHo6k1ocSlycqb}R9pW+-bqMJAkO0wfV&}MOJ zughz)6vI}__5IL%j#E( zE&MAE$CWj6fOQF@b_4g%p^iMlt^rak^^@sx3@bI$0=@o2y)!DVI|e|*nScsoJ& z+itaLpYe>n8+AEt*^^Ym9SC6Q5U(n}S6v4yj|X3G1}Zjl>-jtc6uukbxn4!}eu!9I z>3qu82e!JYuWzVK$rCDCt$CsP^4xMo9~@m=8D!y4~S6nZ8l@YyBmKlB- zjI!S*>;#dD70+*$`*+?u+6aVvi@1D)t!`u?eDQQht5g%5py zB+jXl00_@*09Rd0sD)X4uZYp*ysI#4e>9_jcI1H=s#XZlR+1k$!Nr@=MHuNzvb^D+ ztFaxK1r9HfU>fM<>9J2A;pR-u=sYLjb#Aitj?5GordxjSu{KM_^lPo0+RAbZNS*Rx z)3X;w{qLWOJ16Q{V@PEv8ZuaxwRCY(cIg-2pUYk=tl@yok?oedCg$q?2 zLmYi7%hF@-=P2Qj1JPT+Z5kizO4s1Y4@x3{>LU#?pDbfyk(ztFzr`y?q<`rTiHQvD zz4EfKS-H*oPz^BM*ltKJyZ1e{1+3@ehjurnf;9AB&M>QDHI}^ZW%ZTRk5frpPg})a zu?;CI00e(&=q%J~m>O@$frvm1Sl&ZtL_~(V)B3n9Su`Ox8Z}OvW_GQX9dIJ58M6oG z^qEc6`S#i7_J;P04+I&GUru^$lJeo2)whT0FzBNTIIS|l(CEuQvp`uFr>yh3>T32Y zjlZ4w5vMTuVuvp*SJvxyuz`NyAGe$05OA~ow&k6%mfkv{P!woON9B)DX!NqDbJgub zUG+?E_>0b7oTETSwD?G*V=V}Ncj`NX#!wz$AH7|zyf$+AB1vj_@3OzI+A`#aQ*1|< z%QYmkd zdQ)nCRrI~LW1TIKRDD<^&BAjNQz!31zThD!K(`&FNA4%iUSP@+{RIy|h{!mcoJ=&B z(~Ag0I}Y+;b1q?ir84*&hTLFx1HPLOdRm!bn-{wH9WJTmXoFI`SyCr5A4 zy*CMdKj1kVotsCuvb7T_8!FnkaZ|~#H2zqrGM2)?9L1HS5SbxRD4dG_2@6PYyJ($zWm23ECY6r364=cx-26dfIWmJwLoA>XVaua}aiY6iTJUU3%I z9K6Z%VQ1l2S?XGL?HYQTVNv6jmV95JJnVYvoY2niS53pY>|_=!@gZC{4%Q$f*DY^g zwWx6HnS|D3a&nw>WyO!ZruT5-N9vtT;nSFRYID;GEkj#g}^VFEW{m zQfGW(Z;q+o(SuCXY=_W2Slj#}zvu1($EQOL5wBYzwNX}@b!ICb%jcmctv zp~4kCocn9Ht=}jhpPyV?b?*hwEbFa#xp9$dl|&(+;VZ?C4!dna?-6)8pY$^hO-{0SI=MsU;bg`Mwz(7JaRc^fY!2imtkEjY`5ZVe#aoZZ==u!}V8P z@q@{A1l1kzo}V@9*a-c*nv9EfskhI)Ec|ET|Lt|&Upq9$E_aVE zxy*z5N;(_S%JfbGq0yIC*818EZqh8I_yuGMnN|t#`g``w7BOKWDN zgdxs<#{V_?=h{s)-Z5Ru6_*q*y0iP(L1(|H@?iESpmt!q z^|bbTFIIL}_>ms`UfE^St$1y6KkgNE|4C(AJ^mq%bjy0s?2K7uS*D#79QT>>!0z+X z_y$l(!-<%9oKZWu7wbYb_j502n9xuJ2p%lvo!YF328h6B*Rb;64~SYCKwkkwR`E`q zq{ObLzsbXuhT=Ls6QjkA*3CmPDl!YdehkvP$)bceRQuhDGf~|unf$+vE!W2-5VyU+=a4)?!+!34A(}F24?MG zE9h14IuXpTBtGM#-nHc zTcu!hN4K+Zym~?SBy7MOi5i$*2{@hL+yL@08=NwsIPiTZ61}rkv+oH#g@>k8Ke0P3 z`R(f3+cGQlow_f7>3J8NnKuqQE(|Z=s;JU&PdA6+m{=7`Fvk7Q-Cj1We(BKzGp2MJ zEGkLU=5d%WiWj@#x6&8oXM5q;M!SIvpdvr%@Pw)g!%1HE$Pm_{st{jeH`N9 zd;rYSDp{89th-Ix>eVwBA4}?1F=|%*R{Ha5QOyrCmY1K5ViLfNEt%PqS!;Vq(y@3V z{@a5+Yu5dSj#GCaeAiCFU7)i)ZNX8&>jZVImpLxl67A3PzKHhp5Q*JzY?Ar$J~(4p ze_TG+@%(eDP->R2j070i`nNap$fv(Y?|<~-r*Kx}dXO=8M_9God_ ze=+EGcUah4wfL(xQWW^G5)SPB*sTBCyOmASwZ=9+Z_cn0_YjIH&)jB9VfASRLK(x> z9v?$am2ZX++8|yb$Fq6$8~U(P+2zu+Z<@bo{s~P%C?H;D9V5A}(Ha-$7r@>qDKP6r zqcB|VvMJ-s;!_z7Xxi93Gi}}PK!Ez}80vW!YL&KSB=qkP*Vw;O{j2!JDDS*#dNuK| z8WGDmsF)W{v2?=6^YW@2MwKC{vWiCG&;H689=Q(hdGMGPf;#N9)h|wo;rvNT()zYd2t(EU6{|RA%KU^<-knO_pohZx5H?ez4OznS$`I_Q29)Rye zjq-YAwjHd+02qB2f+I2DoE5B8SsiRbd;56Ds8=vhJ}X4ZHIC*So}|Idw=>bxDu~+T z=N;n?E_;Z#B@xz1^O)p};|Hnzp`w!O0Zswcv56kj`Q^8o;G(Cm$Irdt|HnsYa9aEs zQp%Nm^1mW)V$Z{4!o${-?rWqH+avVhsKgm}@Gi;vA4KE$0QXgN!U`OHb8Ls*h&ka9 z*NI;u*HLX7@11BBsBC7Tik&F6 zVfwy$8vx)wz@9lAa4GGnuFN{F#i@_7d`R;sRaIhUQkFHt{;jpNYsaj8GGMG8qvnwb zL##1e{=e2(?8~fovhMTJ5T^$$IGrp$btNzF>r{8<+(I66UYW_Se_f*rKt(Ju+lHz3 zW*$qolf-zvnVHnnB=d=9PGSDSk|(K6k$vTfUYUOo_it1?c(8WC0~#hP*hjp7d^Uez z9*I?AAT9cj$#b~;FHO%Mainp6)_FCQG$_Q*Pd%Ug$H4!_w8I(@GT)7HJsw*=V}ARd z)Yd-VEVP@kS-TMF3R_dc0lOMD&jzjCEY9{q5Zh(JctUMpDeA1-nrV3m0cnE^q(DaQ1u@eX9p=9zjeovYT63>2HB_U9menlpoS9+ww?_ohP6PIMGoKn6ZU;T z-Q~V?#957S#n*u;cZH*0u{ha%OIFO$DXnW?@URTi{2*Q8lpFb^yhiwEe`-bSP_(ck_e=f0yoouq7`nw?7CREbT3fs4L4$?KN zH}{~EJ)$uEMg+TJP%S9LSee-HJ1`nU7>srI#}K12e`ZpY{k3YW$4B>Wes*#Gv-C1` z+>nru5uo*Umq*lMRUt*{>tWj2r8FL{*+7Q%}>AmDC+W) z$MGBEZ~@c_9&fuUkBHN^>hc!BChzF_=MV=mQ>8AZhpP;$n@@6QQrRt0%O#~Hc=OX0 z^d)Zef3md~2<2wE>!|}v2N=lT)sYlA&@L=6v-%x>)>ml8sYm{Zqk5{q>ydXp`Ex6E zi1xC0f^d8(so5NiBStNNL?Yi@z9V{{d;LWU8|(0okYq++nTr7AvAr%p`+MPd>SVo< zJKd;qjQR2&p-|EpgE;j)^H^9z%(F)01!$8lMT%Xv+c;x#KYo?K7Ab};+%a+;6%C1U6mL03TIWyi9NMewb&qY!>yn$Hy7#B? zhAi9`$q4znxZ#GRPM%6o5X792B7+p$E_;++jiL$pMrT4W%LkxoRT!|(%Zb^`PF{$70C1}HqfTcU)GpS?)r+7dr*`- zp>X4I6}#SA{J0c10AzXj;bpOG)NCpH=k3}?V+a?`S_xhA4x{hm5Xg0>Ug9TK4@NvCWD3b;c^zCii{;00 z4xmlZoAFc2!`~D~_mWe0o(715EfZd{nXeU>zu9%;G#cMbIW3{GxvaZ(XM3fJxR+bi zqp;DNdQ!kdRt!rbnC74u^XD$Q%U>cjDbF2G`BjQ7^CDmAV603qFI2DPS3FTNvHVlB z{z2)7_Z`oAqMgxpVM!U3Pp73y4c_&3Y`@BkWliCI*7Braz;+|mM5^NM{M|%??)(tC z$ecSAyD_}iKmFaCFZL)lzuVe6%f!+ejh#6xYH2Cj)GJu)zEFJf7_sl1sei zXmp$|3#=d}uduA(9{|qFngE?NKnmZd1Fbr3#c-XjiqklnV^zsVU;muZ*)i4%J}7uI zGuPrPJY%BzD64KRh2yg$GT4#4v!fv(@o7^1Do-f4-`$y$ibg#m;>kxWJZ5DN>K{tF z<;wne3e|3}m~+VL#j&7K$V_*%kiU#Dkj}l9Aqkne1j*MQA3W7vA+Vu$Wq5Wr2+=wHQVW2ihHGFo-U=Jwepo_*nu3`mx5WvCc3>^OYB(7!pnwh?0(kB0iXtv}0qbNRyqlSc$82QxE*Qv* z%5OXQR`ycS?Z4>DEJ3EY>fmxbbyVIU^+t2lHOQh$Ko5*uoJ^Xiv*$kL|L=(NDE(hF z{a=`FQT12rC{v?6wQ2v)MQW8E*)@V&|$m zgVn-$XnOMKvCf*w>pY^6}V;(^N3Z0cm@muqs-88?TniEwH8szc3 zOM%!sVV|f!(+n}C;*C^~{7D;f_N-gsU>b2&Sb|_e;PNKRUjaqM(>D_V#JQFTGjwm` z??D{|A=msJDq?L38M20E(q1|1qWvPU`AeujRqu2sv)%pf9qt3}M<6{PL=U?P($vo4 z@Hy}9fkIr6@fwV${@moX$t^j$_Uqbw1l}b(VBOtfYJ$%om=h%+&Kgy8dA0N>)K;-=d(7eznI$+%jg+n!5djl7U9;SUB`ilV)X+g^z-r!zh-dp}<s4icT3>%cA^%udo2EHl8IQMzhee#+O5eWqTix8mz zHtoc%9eZtVq@-L@TS}ipM2bzNR`%k}q&h>6W;&N6xcZMs=?#OA<;aqX$|FQ9YP8-I znCCTtlNvKz{;rT&p_;{g2Lz0K7=R3?Eifuu(7FQ<_IrXZA##}@g1CUU$U(3 zG{xB1wnyG5e$sE25QpI; z0L%|G97|PXLeF;HRfa*ErzMG8K#7?Dk+|x$$h2Y#d!!-Z6mF`yw1ho6Md5HrK!WwtoyiK_6EEPDO35?flf039^5zY| zS8DY%D>6MruAukAE!`&)4noAI%e5otEMHF)@#WUIi<*+&-REOEmZAs-BUNhTCDpPN z#F;|QR5WiKD@HB$-bAR3l(W)+-CG%1YKO{|+|Js=f$^N@wK~$U>FwfzPL#XQfLVIj zrmR{LvHWPJGf}d@Tw6R4DNe6}iyqjeKs5x>A(vTwWF(c^-0^AuEiF>K)_;A`E0SvV zO06q0D}(&lwKdLYJ^S&H6jN9utg|;~<;Q>J5U0ql!vSi|V7d%8YlQniv8#BAjD0|n zEOUnKWRX4NK}6^M;@&Dj)NZSKv9FqaBo;VkvN)~8cJ<%s z_H7A#p4>iVy3Hp}bj4mlw+kz}SP~U+waz+~9XAC>GV)XZ5q40sHwobiTQWs8=4EQ$ zubAB$^5+guB{Gz+-u}dEVTCo;xA>=~W-1A$u&+F&0gAcKGt4Ba^_x@0+$qTLQ>6y5 zs{cEYe(4jRbwydWt?U05tW&L2s_*Q)>dtMy!T5?*O&L+bp{UfXm{M;QY3Db#k?0u2 z-hTZr8#c1*wEd=WxDQJobSXQ8PFSY{m15W6vUhU=1eGtW`K9Ihk^d#!jaN@5IP^e& z0}YQAf0QTB9fvFDOFh^F@(8FI+coa*nEi!BGhew%HEJr5L$cM} zR*|B7Oh1_6Nn%25%g#dq6&8u&nvYW96Y;=@|CG{Pn-iH-J+|J;ryKW~JGnSU-wX>! z+@W!nO43S=ScW*D^JR$PH|F#E6b`RsMxe4yX_u0S@XQy1OUh_ zD&e!Lxb*J}QSekwT@j>zLz_`!KHsq*Sqir16W#l^E^}((X52Oh-;?E*>1#lV_4Mu>BvF>n`B|C zF6p^UIV-o=2q4H!blJ5#PgV=^Q<>HN22`<*M{}he51U}t#BMr@tdzt!^=>aNNW}Ulw5uW(;SaoFU5er)brv@Q;rSE zgx4v>Z-Ec(dvAKY;nmrfAFSF|9^KxRZ5UU3D^>Yd%e5a`GNiyIimqmvd$&#R?G%5y zc#*{#TCJ@<-^BM(YRO8XID3BkFRLT;u+CYGS*b?iuGlq?z==CouSkkm+77P562`q{ zidz-vF_>YuOtd8^iqGzS`uN}dSuCiZ(0)!yZ@5L3uMW_=hR>1vG2&+V>XG(C%P0t0 z*>C7yPg?Hs$=H|w)TPVbJE`~q07t%f+30y|p9tJnMFOhW8n5al{5w$#a(+30*cc!6 z?gJ;0GR}DBsxDog7_!Y6^(>x*XVtCk<aGTL1ROp=FP<^#`1S_Wif1Jh*YQUas*no?@ z$Wk&JFsxYZaNCw>W7{3wr*xF?x zabVpwQ+Jj7{Or)T9D02v178OBHU@Uv0L(h?YNQGl6hQIC-`jO|LPoVSJj{UNI>mnr z(Moyc4o}7wlmz=Gu29WwsHwVTxqcV^$fWAElE3CgMAt+9Fr07No~5W%c>8{^x2~(zIC{$)`w~p8J$?@SwWHvYP+Qe4x!Ox<9}U0B!t&J`8wQ z@TeojYG-~hqwcoWEv4!&S-!JC3&r6i4gR19gg(f* zCM_Jf|IL=z9hdL`Z7W>2t2m`qc`gB-$R87PgF3mvV@v7H%;L<}RBzezh9bY%IHuEJ zk(YzG7Hd++51!gIodbngLQ<|jdf+xC519@hr_L3fcT!n>?MIJlKQyy+yN4^bu*ejH zW|tEkWu7pSG~4|BjhLibI*vD%L6w$_j5@EQ!o(QDA=nYFkRw--l9kCRmd1My7a!`_ zg&>xUMAIL(k;WIVH;Y|6XndSA=*O(d8fn$f7sT)H!?b|6H#u)$Q8PSw&K=)9%jZqQ z0xX}lq25}&Y#XZ-#F#A%n(~YMoqS+z!ptm*syf-z}EHC%(`z5$gD+ zj)S8(Men%@brBv4fJ}(LfqzOAVA?7$G|}B;lS-E&*mu{S4UHmmt zvJ<-ap45WL`q2BfTs~C)tycNh>?4b;U$;mhd7+br6e+`=6jiCWrd|82M8dnd>~)o3 zf~r?~t`dn|a%P^W=!)QR1hMPd$L~UXFODAbMd-T3#UF;Il6oUN6hQ+p8RHg0fD@84 z^5EiqKaB+^-1k&x!iEEkn;*6sbiu-MY0vbu=?#&At?mf2$2NmdJ{AF zLp+zVi|)@;qY<<))e z)5Nt&hw*d-CV#Z2AOX}>5S-f#za`xnK*FE5>imGCnuO$$>?E0zSn{#;YuYDS&VQjv z{pDoyETj6ju2Z-?j@JdiL#O)HHcfRRo<>2?7Rm5MWp zAKSml3R}8+ljMnQ=!7P&Nj_ilXl1(uYbgggs0|AW;mVgd8q`xrz^|i?U}~;)U|0G< z0#K+e=Qfc~Qo)k!(%TMd;`S1+s+VE~DaZaCd#i^G;KP4JW#LAOGq+RQg&?H%TsWjf z&rLGo(Oa@qVNf;S3$q)POf$+f%Xa<`0il;QlyZZ8mXe)fevyZM8leFwM3!&|GICH) zKgBo~IMWcGO-Lfo9lN85(dN$yDV`;Yb70$ts+vgMUgqay4v*;>4|Ej;dQVHuFLyt+ z3MU!V!x$wkY{{o=?FzeWPB00meOoz$;2T`!u}((Lpq8$;U+3+LgUJA5s<9WIn1D#y z^DF(t4Jf}z^)_4|>&L)le7&W-|3Ir|@#z5dlFWs35xZW!GAg%vY7}4ZfU=IIHY>;S^^1i?;pTpd{u{g!R!@RNphR`Klu9U zxG0;h|0NYcP$U)*P^2W65Tun*x>IUtkdp2NDFrF%TuMs1y9K0?Tv{cU?gf^3FT9`o zzMuE~{64?`V6JOt=A1cm=FIndX09&gb1Rnn^dO;d#viJ6NBszMAt^w+;}W#Tz-gcp z8KV%U7$C(Lcys$ZYmrX&GlRq1z@ZwytV5ED*5w-N2;2HIzXw>Lj4ts`O^M32JtJrc z9S8vfV-YNU#15j%;k7=Js=#`6PwJ_qv1*Wfs~y4v@ZYXai0fdJ=Im-;-cc~3fcdzd zRP&$JBu)-dBxB7=b(rf13kjyfO5_yJ%IfhazdX5kKMOAVRD0KK{kSllSE0>!)*eTCf#rYkM$t}6{% zEgCC&O#phLZ`9j~W-H_41cedF`{Ts)6`@-@l#)K)?#2^P<{k2hoXs_Ztf(6y8Tm@~M+-I!IPA`y{8k7-yPr|41wpot3|6#wq{>T-WAfm~U`W_Om#j6Gwm;?uCxHY68f($V^9oID`N9}X`fnGK4#I#IDWkxDb>f(O7weTrH)JN^ z1x|p<%Reo=y6i&xsld6yP8gKiq3X&nE`yA2;}T>Gn}E0FGW^@~GyKl+JMR&I|N0^C zqYq{`EEu0A?AKibHb4~1M}WMyal;PeJ-RIoJC^w#{h@8LAU-=gLB0TSIo$o;t=VhIuxY=LD*a-GpMS{vw*~(GT32F?oO48<97;F+dMVtu z?3$L8b8*VIlb`H5LHiOoer>X3c-?r7s6>okuACtC`Vs^&wXIaw+8<%>(Oz`*PrBT1 zdx3?Q{r-;{pALfoSN|2`X=dC59pW_F>m()8tc@Gq^4{AUGIa6^?R%6E1f95|y(*6+ zryDpy)*jf&N+Y$WOE0>2t4@5cl&%_{YyMX07!)qJdU)`4<|OqVTC4TUtILC2^x`v} z^DK;2#fs()wFR=uNAN_h7an_uB+(=hKI>kC=d}5&JDQI!b4D5*m0s~-SYxVS8cvOPR2zy< zl}rJ&=|JD%SM(;MUz})~(wfFOn_fc+C|-K1a?GaytNoxu5VY06ErrXR?&90*8Ca0v zhfDkl>k|s&#>z3luX6oAKOggGw$NgH%%pPM=zF?WYxH<`7GB}kgt~fE+m5Lc7xa1!`<JiXNe96Md4o}2>=}H?aG6QI@VH{HOBhbs zfzFcSWo7IKk%WZRiR0WuWNo}JrBcz_lSl|A81N)ue_%}1;9S^{Ff*)C@qlL`Y7I&c zXk~cMbP}2^M@2*ULe!Ha6T08a9?6cJT@ou1j~!rcdh_9G_N>xj|5vsbDh?&NY0B} zT?_?9({O-c+XXTmUjl3}?fg#ZEDR5pre>viZbMLZFe+Ijf0pWXmHee&ct+5`#G5@( z*8_^m_9Z2=TgpCex->OcSB4<(e%UslESbV1U@A8#lEe=(l+V3p~oti}6DkL63v9>0hwkI^NDtLD7sv15iTCzM6_ zwjiq1;%(&%Yg~(dr)w&&D=P_|v(h8xuzBQ_uuA*oAJY8-fcw19M-zg7as$emZ>atkuFzlH4>A6xzEB{t7A z7Ul$2n%h5N^3>4g@z2Pn+Zvi$%tYL_OX{VY3;!%4FK}Wg@*d7%Yn-QeX96o&>C2#B ztAUAhbR_JVnIK|o*zFKJs8k-}>Z@YoQB!D9l$O@oRC$f5p2m!0=bNn=O4+Q%<&x5> zy!($2oHx&B@|H|x1Ei{$n`i82kI;ISb#z)=GX=G8dN!YL?~+JmRrkDR*m~IRw=V)r zq$lqzrKd%Z3B{;Z6;&w|MF9(*qt`VoZ@BxLYh7UbN zoaan#)z-hW8SH_sRe4Z0frfH=hM|%FH_pK|)#zjt4EiPO?#cOe=vPpxa2@h=YOemA zv@LBkce|cpTQH|HUV|zSq`0eVVDwpUk3Ih0Bzku~Wp?>Jo+- z`mKvPef$-Ty3(dH?gs(HK39~KO04@#zpP@6l5XkrST_dUwQjsR=}0;OTdh?<7>E3G~vBv=4LGyN1n%b-j$9zIlZH>Owk-x65 zHHFB7b-{#)<21bN)yp)Jc4g9Dz?Nug4~owHZz8NO7y@MGl{pb_&uzF9of{RFxr?UF z0MB}Ey(Vws!9aL9vr?c{HAc{XhdN$HIr(GEEj4jfOiemf0~IPQ>v-MS+3v{=_e)#3 zdgPn)*{SrjQFBzzCW~!y6iczSvz=U{z4V(mHsQ__zKmhEci*Lq%lghH|C+Ib8|ea8 zSaaN|U^9ERU0Y*bk7ui!K-~^GW&SXyy=jyps91C8$+8*1EXFdWvFHN!o|cJ4}Yeg-skgY%i_D>FQ}-ppu7E@KzHa1%cm zLl~@P@jXLJdM~C;V_MPKd|W@EF{Z$1LB=LG^P*ExNsz>-Het<9g-*i>|H!&!%udKw z8TI)7Jkm=Ob#-ws>E#zI<+UszM0PY)h;vnx+z4PJdM|%n8`XOBBpY6+mz_TbI~@7G znSq-Pi{T*8>W+_?YCR86Z<9h6c)pBN6sjF6Jj&bDnW4-SY;6(~^{KIm@$A{A6xglv zkQiOM6q&i)TzFOfOBx3!=3cx1ePjNT-2h=Bt0#i`kpgR%<;QTRGP)%ScVACJG^`KE1>dEA9=B5^AvLvUr7HTxY9J?%9K&YO&HC_&1-g~APKo5#w|y$) z%VF;(;i0)KQ{vPyW5(A+Kj=1VxH7Vb9`iwu6el0t8F+u&x7*NfD<=I%F<;3Mo@~)`;mY&OGV4e@2$+VyPui@iK8D17P z2^vie$~S7NCY|296&*?H6a^h)uA_oioM zvR>0fUxKh|sM_UxL)%Ab`=M-Mae1+Q{%AbM<^+3%Y!CCi;D;fz1$)~=s|s)FF2B(Q ztHhT;rmG0@DmJgr4L@OLMF$DS!}M7iyYQOwT9|!1yq<)Bgv_hMowjAFKgZJcxvivJ z?H=l*yt|x>q{9L`vjW2JU?E) zyo0lft(r6iyM~Xp1cq3?*uW`sFmCxdE6Hk396dQ!B=bf!GCw^k_pPrdwGYCEK=*uK z$xEqqd;M0WNK3t$ND))b>^Jrq&T@7+R^gSG8Xx1*y}KiSNl#H6iK@iyh`B)+7eRfBJA2rS~XNXC0hZe}5-^U2zZe1FBjOs#CnH+PMh%_6We zlc^Y!-wL*@P|f(zwgw2@v`@_^7@01udN#8gg*TJH`{?!P!sP<0+Jmw27`D>=`zxv+ z+phLf6HmkEp<~b5=ocatyVRI(mDoV~V;rG|UqcJa6TE%(t*=vsEZ<~3;x#{h752f& z>65XFFHoJ4eNKW21~$$%2iO6WiS^~a^|MAPO-dge5_*I;8BEvqsfJO$X&A z$e6(U{J>U|k)E+}5C#3=SAUWb83^ge`^oYvG2V=}MDH0cB4N!4589lKJBilRI7A{f@VlgywBm=jPnUzIuqLW-f_z#BlhN=vSzyz5VF zjSP@sF@5;HVuH0q`_Uf|C8bWx^fs{xiCO-bK-L+E<^m1f)J7|dFRJKHtSe5LOrm<- z=+J}~X|J8+zuYyY{aUnJa2-6&KTen6_&~5Xg;9=aTGi{tqKQx~<*18Ziu!1 zShuUEO2ML4`hkKiiFN)aSgtgEnKfWp*(AMITBywxVozuK~5K} z(AP((eP?in?quLx2X3L>UwI(u5kqxY>}og>XgOnH%P8_dYW}QxP)-vC8N;0z2llGNh!Lux-6cFDXs(bSTc7WK32KG8KV- zy>PU4e#4NFpzf=?V%e;DwN!d%bToHqJSP4G@|C+BJ+(VV0deuI5lh2)6Hb{nyJPf_ zEHKaCU!}OUrb4se+e?m7(}x3-!5SaZ1UE*5Uvq4^ARaL8eDh3MVguVyZ>B=?59B9w zB6Su?@b#-0)GyvloopXkE6nm2ZBQk{dXCY~9QStdG;~JlCzwlw1R{#HiZ9BlaxitL zs(pwR&U1BNzxNqdt3=ntQ8(o*x~A~TFIvW#=~~ep2d-rG+Z?bqx^cE_Q|MqPmZX#r zi2^HuM^=-}W|rNi_U2jQ&x(WJhkd(R3%Lg7`ooMp+&v$Bdz_0xtgNxl*YXy16H9%` zi-b=PR5ezeDnn*P^6y$oY>ZYFde8bKB?#JXJ$3(>jtR@uy7Za= zqQs*7!KSU^Ok%H$0W0OK&x#H;T{eR22K>W*H!ulOF4&2xSXhVnXQxAc#5Y;8 z^8zPUbwPdsg*c1R$#3dcvx1)uK-MY2L5M4rd+h)MxXT5=l0JC6kGKF>kTL7(YqHEU zeJ37%w0B+-YoU2tF%KVq+Rl<*X=4ew}WX(C)Cgg3ac`&mBKC5 z*3&3;k?xS7tF!J!W}P%EH0e$vmG1M~g+g3+NlCV9&ig0V`|<*YQ-A_;;11#pl0rE04^VW}PXm_%ug#GK1Eb_qO# zjFsY=&%DdI0jF)0a~lNsITHT@vTm?v$Y;_p&-E4dDCD&jXY=uTKh`e^gpH5f@qu-E zgpyQ=WTyDmvVNHZ^oqa1pmyKrKs*uE0V{PvoF^4%bwny0&N_#UBBu_js@Ys%Ib)0I z#ncTz)^OEmqG!qGm&F4mc#gs0u&`M6Z=76zOiJk(qt)s7r!091&-|>Q`=S6;4uI}> zCc5p?i9WyAU@i9&i2aGZ?i^NLzCo$}mNQl}YN1qGlbGjq2<;BhVNUCA%K5Dk&Kh*8 z15PE%i+M3o<6J=eAVW~ILmfGGeG|}qTC`3dx@-?3iYv)Ebie=fM}aM+FQd{|*b-CF z=k;-WXW#SiovBZ{^Nl;xqwt99%A>XJ;od5n5b;M&XL%;oC%G%>-&~S?$!ex;v#3~@ zUx4g)qX-{F3aF(?1>pWY`gl(?+oj*h9OamF*o=}DlO}lzGk`Do@c4EoVx<*IpQudR zKD4#FD3Yv1$*v225Zl|P?u;38;Ws1hC2mRUM)u;f$_+E^Cq>{DrNHnOozs+7(gp8a z5LC_|D1}>LLjd9efZzU}B!mt&){5gYVM%^!%;iAF%h0r~BXfw!u1B$;qWBN{LMNY) znXj9hqRT#Umho+|ZhWMT2f=RmcY(SOrFE}*1K`1YYY{3l8z0}@2xRg%ve(Bm;<&fRq;AzYV=#CWKv)% zxh-)CUwsU_iK)L9rYhz{{QAf~alXO#J|~v)%9 z*{|i_&CVntP2V6B%~+T-t~wxpd9FWFciM!@;_0V6VwvasT;N08Z}4!2GM4f7rF;*p z$-03sC;cZ&EzF`}E;8_I3B&tDPZJkHT|>3ZY2r**pVpHvI-b4CqAdh(W)52`#^&GxIBolf zCHk-NR?MH?>LG49$T**lb_-aU#0KWhU(^yX(f?m$lLzkKz!CT_UgX{lU4vSIph8P^*PQ`STl z`RTXC1$WU>&6uUuy(#Q@vxScjQHX@Vp&D2O!zQF!HGQ zUh}naK$SWtDP6;g{J6}^joiNDg1~%U+4(*sQaj|t?=LJI8byn=miPct6fCk|*6l{w z4Hkfwe7*~U$mfdlj4wN0To(opGPVkgmK_HV^4dJ(mH2`I>MkzP#Z%ybh74zm|&mgEDqW&C)(ZyedP z+)*(8u_^Eot{>M+m&szwkI+O@UW_@3K7p{u_WWDS(*W1JtY>*bK$PXCKs$-Q@7>e; z1_Z_$Ps&SjwD^7}@!d|By=dF>%p^1_2Iwy{@B4{>5z%h9Ld&o+#`)(8WI!HW820&O zE4YK77OMTdyy7L$`lD&`U~h4~i=(BOf>>Tb6v z@U>w(Q5a#E_)WfTmF5FZ6I;fQsUVSN)Z##fK$_HNXy_fGSDJ0NotlmEfZ@yUM{?au z>ACj}1o>^L2T@^#gs@587go@0`{>9yB7+5b7L})!>6N_vyX|A1#^dQX(Jw1i75CRc zP|3{#3qZ-rcERS)dE1x|VjIXo4dtAC;~e3iGseGYC;!@|p#y_nkp%rL#))?g=#@xY zO->20WCKeX+@d1^W#BF$)EIwx1-`xodJ-1z*r|pa^U-|92fQ8_{6m~NVz?I<^gYvH zHs*n=(i@m5Q6!M$1=Uq1nG~qNcy(Lg6;1SA(9fh$+Cp-pgUQ12B40|BLt+m=qkml0 z%7`Ik44Fp26B$^clFG9uWWeatC^) z9oNGOOaynr9slkG#_t{Y1fcA0b&@uBCg^BiwPuty9V-;OjxyPa6V%3Go@->`diKHN zy)kg3p1*hoZD22zKV~D9i=g%I%w|)ZEl8-dup%F5%OE0nmH44n=g!^w82(ck%LpR# zEox90CfhIuAgxgAOTIS3Y)kAc5QghhO02?aAM)N%@l%6OX&ASFllY*!amOaRSuu3`&DreB*9egIo^=6_x(jN=@^}X1R(C9ZZ16-l$}sib0f8~@ zUvNVtu^96dvhd2mAMj?w>F#Mq<3q{cL12~~IFs>PaT1x+YWWLqJE}oYCL=b8JaK=f z0LA@nVuT{^^&pU#3O)GzPDq^ZC|taI7-$6|{)iGvTsH;eKVjKCam3YsLtl3e!VXu$`i0o zO<~>(q|o@aj5M_I_u_&bg>1uZKw$e-37*)1u4OkG7s-6oxiQqA4TMPAZD%V=)&3Y-y7}_o@Y9rCE zX@*fGM1r*iFEU^66I$A$(A={i!2#(d)Qt5LQ`9^bDoQ-ZaB^*=17%n^wr|^*Tw;** zhzvr%#4hSE#yvGp*CK$3O^>c8nU{6lx9Rz8WM;pj2_?hZ8w)3O)nT*J(vb*~$+aIq`)ik*_9K+XeB@ zC!z13_L2^;{f2I`LCIwHV#2#n+7D4OZxfYWr|L4mP*-{;5U4Wlvr9d*cWD;%0a4Aq zpRwPx_x-g<)=qU>_I6=Y%gfI2U&n3EIs6DdA2$El58E6<)@7fI9Da)9$xpZQmGA~g zQ4lf{0_l0M6a1PNWDte=k15R?N-THC+B(Cf<`9S+zKEx|i7-r?VNF-L)tY6@FgO(Y zX0+0^I6OPlAKp*K&&d}M6akGTr^3%|S2xYPCt(ND)|7#zs}ivJEp^gcq-Q|`pGkqn z&awvg%DhN$L}HD8lmudx$RZOO9|c|s@?B7ih+U-NAPRJ; z@P$Y1DdG}v%5B7C*q`Wdu;L+4&>_O5SBU3z|(p>{9f@zDN<$Q{)xQX&7Vgq{7dIH?B!BSemyB2n3I2C?ru=E${ zW>p;wT`zdR_uq+gwJjUM-I#IX<;eoRYu@i6!6}odl@3N0To}L_C+Ch^GG#mBG ztcJWC#o`lpwlV3TqhEjOpup8OSRvgJXAXJ&f(TN$nz85${UGx$7`#x-yq@X&)^sLP zAfxbUHVCdpz`9jkdYM(YEoLn)PmDIWKKIr87(ew>*vGtMk!Xv?={>U21keoxF%hF; zr}Q*9wR}8{BEec`XrO_Zcl3y(ln*x`#%5v~c zGDU%`6Yl`ET*08-(HFV3+fi8=7I3>ISXUW0H=R^zE4Fs0y0l-=9tw-cErl-Q10i3) z{W}~(MZtO>D?5Y!eOgvg0U2fJDm$upv`9{ZMVTuOG~G&@n0M6T{f3i3BwC}hyt?RIVn{ND_NSE=N`!%nGsS&^JJC5CLaj6QWyG zs@!=LZ4Kuq#`#aB4FiLX8Q^u5EN25QwlukL6((ttzf?7Xg|uyCLuN2lJl8$>B$CA( zFY9@-jS7=|`IjD=W&A6-k1j*~Y3dro*C;<#vBz+p3V&w4{YQ-%N)`NNI3L$P<>^Q1 zP~Mtxm*IM(^4B1NAiyorW&CAEm7jJksQ{n;cV~_!kbH_PxUSRF=4I~0pV%d4w)g*S zE)M5o#l0UYQtcdlsXvY$&WKu+n=zw1Loq`9qd@q-c5j_;S(-dEa9!LUgfbtohZ-A+ zOm)ER%~w~;`5vR1Z8BEpu8w%pb^E$e zybB3{+VCK4p)<4H8ZzS@o-Gtd%bgbwf}C0})@ORs&z5r|(c=?ZlMB~{wD9$EnS7_Y zH2Cb7(%O&vCA5MJ7-Zs@1fMt#Pwg}awxUf2p{!{>p~k=TtW8>tp<2X2#q`F$r{uns zCre^yl@E^XTGlKUT9=&AEwoK;PpKBjVd%(7*#3`5S-+qtP$xHi8>jROgj8hh{zqeN zX7mR<$bgv2^Xt!%rA#JE(ep|zdrJKC2E;$<>(fs+edkU_Rv-E{uEkH$@Y^cU-F zx?3>hwb{~op|_wGA+`agYIda?UbK-u6UiGYO8@jnR<`$twtDf{m7jWJgQm&kfpg`qbpSpFv7dM4dU<>}W3hlboVqq-8kFI1+G=_g zth{zYkUfO{&UFZPG^9y59N-$G^4gK74wOwqX&cvejKr&7tFQ$Q%);JqK~R#9n+s*YjkVVZ zFW5#f{0EWu`V;=S>*`yrSHWnf)LS&?BMA6!yHdwA1mhX?Gj-3odWK)cUS=`w@oQ@c z0o3`v3~&F#-}Fr@O);mdi+LPAtDeypCqj!3m(M+Hk^=NRzizH21lB2D9m0H<+^(0; zhXqPidVSzLUoCWQJP+8<7PwZ@&sh8R^wsw=m(_IUFSne=A#>3PhG^wU8GU3IbsVo! zGH?~MQ+7`{C0%M}SGGZ(0b~;;wm+Rhw!bxvJ>1@FpB?%>AR4EUyOQ$P%Zt;ag8)`pIsv3FqOW1W$v(r z$FzY!V!;YC6+V0JW3lk=(tuCS&xwZW7%J!IMswzKS{->cRKsyw<~BoH+qkJpRuMt* zhYue8w*y&iQ&mZtMtjiDPM;>4Y7+!NHYsq~o!&)<%3^dpZID&kG6Y^Kya17ffl z>se7#%_T`{DUxLo!c4>L!^5@=MniMIF7!Op_=Rd8X0x@#O)2f#rLiT@^Yjdu$YLf> z_1msAFN0+~quCN>`yVbi>JB}PEcn2-=CfAkv3UG&Meh;wZuODy+L~v1p*M{g&Re`a z8{=_)7{^GU8BbcPU>4| z8|FJXGrg(+&$5PJ7NFS*U1^mG2G~_jkD1cp8<05-fm$mYf$9zsZ63}=Jw1dS5xR;~ z-sYJVZQO`L#o|fTu*s;?>|o-mfQ%nTyrw2YYd+ZXY4edGP|Guwd%^EGmUfTxX->kx z()MqOmn1#UH=kl4_m#w-*^R$^#UG3F>)<*J#9WwVaU{-xh~4gJ(A8ZLSENxB^DV@C zH^jN7A8DS?Nw#t#T+(>Q3pMwuxiuEJ%~jdbJ=h-u)>Q1FtL!wEHO9 zIsDC|Vt6aulnpI7g{KcsNPjKDAEIGb`eE014duOieE-fFzpj7WS*J|jM%QdKNxVky z;l&P_K?+0CG0l{jsF&Mqm!t*dT!wMHjaO^&Mwe#5RzUZp1-?Az$YFN9HR=T~9^Cj$ zWj4H(>eR(n@{T0zQ={mX7V2z2@ROGPXubv{bSis+rE z1<#ythz0e0T$LO52LU{L&mE^Z`p#pB1>jVwVg{i`Y>5eM9q|TGi9Phv9OMNi8-3dn z9iB}vmYbKU944I>yw!s zAIqetI~Am-(=6K&9o9L?aT^k;l*CLH)3=?;JEiVeB;%7F+n)LQZ~?0)zI`Mel$@hU zc-O6z9|&4fSm@itF`HEA_XoIM-7T2UNAb`4#dVc(OY+MYATc;TJ!7c2t9s_!{8Kf$ zklqkUJ@{M8MK!@-M7VT-Yslo#$pfa-_HA6VjN~$9b@LfM|kE?vA{=@$IINON6!!g}7pUzF&j_x#Fy5W9w|B_P#GKU?N zz}~aLU5vWF3(K9)+%yw)G{}zCM`8?lBsj5z_zrnx`wTL)jp!yNSfx;|t|0knlnI)n zR2~7t)-3Bc_K3Zcl3YMQe(CwHQ6v7cyw~+;Ahx^LXQKfZ<2R^2yP7&x(QA6B*)B5e zTfR|8$|d3CCk+LjzLA}6JF0jmS>yV7_6*tjh;uzgk!y!^nIZO-_`_L6V^)mdpeUM* zH7a}(F4}1)KBo~P)=+)7YKkK4yO%P4V`YO;9V$+b$e$jFa(}BJLMJlzN=GJPyPp?| z4Ienk(Zw&Od&7`}`bVF{fpHLVY6R#M8Xf(E|kc{k*b+(U$(_$Tbe$D_c^cDv7g!2iWn3p!|gz4JW0VhLz8k=2-f1bUX zrgfWx7(B+nb!9umpw%Q8c)1EPv>FhbS4cY9t1yN#hKR2onHfOIVtX%|Jm06#LzA=; z>V2&ck+ZLz^e|J|l@C%M3LPGAY%4jYHW{p<%)Z+jFh#O^MJPsBStMEwi5=cMyzu18 zB&Z$m>b1^${VZ~m!&7~T{d$#)0Yh;Vp8g<5k2|44apUmKv2}t#LX2dv^=hJkYu}FU z!g)jEqesw;jeubkU6rNz8mqR^V2=ZyyW8$4YwAnW@Y6oL#C#s z;I#-@zZiK}d!^B7{UDTrJ*;1WbuWIo{wO~#KV~5H^*lQHOVyTAGJKi7_1L>_;5{3j zd-Sv*#qd%ER8hu{=V5~v{P?$k$Gjd%zL&pKi*Eh8GHPp*^>%`Y>E>Dd7?_y-PgE&q z+DZo{q_J7bGDeWPv3i+DGuy;#hd20+Vn3G%Z7i%|Oxij57BQombM%wZsyLhGO~Vg5 z;IwgmVAX)e(3vYal`Jt3Di+9w9RW59@ihld9# z%e?ITwk#^u9!71wl+(S3_VD@=UK#qnfawnMX>m+p#iA7jEJd<>VT?y+JO3;{o4D#B zt3-6a@*?+IZDanJ>w}+1kSU1|PD3uG?^ahPojk|J-b4=w^R1~yBaXIcJeN{R#6-o` zv|Xan?L@v?=a(|tn>4HRC)u7~oCXMrcNXA22AhuGM|K+AbTQd}5fek9YPEcxYl)Hg zoq=Izy|4kdr?`NoO;2q-h5MuCcFI<`$NBLL+_W8Z9u`IjewO33hNsMKO66`WH%1ii zv?gOeU(+C~|LWxuk;6wPth1ql-WnSU~wud2o+JaL+if}7~S&2 z`)#+>NCw&hZ1vO$c=k=HP|W}K%6dr7Kk0L`ZPQ4%eyH)r0l&4B+do~Vf?MLwP9a+3 ztN`GROnn40d1sX$NaZ~}jro@fKLP7Gsw~Br=vb($ewhvWE)Evsf z$$?e$Kl|Y_r=D4NZ7zt}w6Ead`@5Bk^(Nsy@qce>{R-;L67y3vNaS@^lhV})gk0@; z5^IV6{xJlnmN4cgPpJRXzAS}ls?)M8#AiK8cjbS(G!{eOC7La-%kfoDlRzk1s!Q?d zKhlEE*`54Ks~p~_*vwYi&K3Nt`)O(WF_o^i*f{`T6rb&J_2L$3hdBPDh*uHS4z`JP zZM>#5n^Ik6?CJIk1%1qpb z(jIZWd$mh;2~D@HiVBh=nA% zeNgIq84Q^IB;tro0;*v zQK%3&h6}xd+blh`|3wE-fDeDkNWN4L!B>k~%ytSj%5oGe`13~esptZNCM2yN-p0wV zNwGKp;82g*fJ(^_N#>9u_DErOl$z2!mq}mAet(K~c!A}pj0iy*PK$$&TWHU8= zPvc4_kulcq@j_*8x#rg*@Mr|9w#K3QM?lYzDHog9{KD1{A7IBoIW4js?M&rGiZdI# zn0Dm6ki%>jg)awSIKW9~M}e7cxk0F<2-Y7+6Ce>_eS9)C(#DN6)=T%h-pxO-oy!I) z&q%K@L_U*&%*HM(9%0g|kZe{+jdR_o;?l)2@!La95}QL5)8Ts*bLeUpA0Z2Hcf)$E zTKI(ZbW2`xd*4W*ox**fIpZ=G-q|>j_l99Jm0TiFAT;nXw<1Nl2D#+Z$s}-o{R}%h!t0`Bt%_nEat_3$umDaHyA`qzm^LX2r z2)=;k98~Fum*S{zN+fWMo6In{3a&w5({9qKO9P3MBLBDwq6p z{Z7-AZgTr?L*QBTb76VQh^KcLV?Ej^D4Vt<)^{zJ1ZepIkrD#Ji$5ch3DONuCcf>y z)pT0127$CGh>aLOi@fI%KHVd3?HPy^+s(!W0r{Xc4FW~XiET-IC2ad>xdX5CNZu8_UiB+@H+%|E5^nU*CP1x$PdXRFFYz5)P__F}k-!}j2x-l$<{1Lbkl>jo zw!$@!){+l-YpH?q@o6AHz!e-trx^Uq#(!5bd`bBz@4MT_spFyZfrbMWJ-%4&eE4u~!U`!OSPf#v{LeP1*mtZ7g~>uHo2xD#(P;WF|1ro8Wm`3TxgTcxOp z?Zxp=j+U!^K8`cMf%xh4EfgXzc~FkkW=cP|?t0I1G+!0)(V%YpZl!BYxCKdPpU^yk zrtw0!!0)wSpe~3LniOLY3O-2Ecb`AHS|~il0Qq_JiA!sS80aSv!81p0xjx<(E%~b$ zGX5$0XWYpPDy8AGXO);okEMWlKA=XtIB*Llu2OWg+jeN1n!02S%XIhw_!0yTdZuo! z`!dwnjM@rU*L2Xt&OKPSwdgz0#yr4kg;mecSS@v=MsKr*QUT5kOF~Ucjl=&7`6(jI zaBYVVu0970Ev+z*ysEYc(~Ns2MG>JWE5+OEvMj(j&L8P6ePP=BlBF%9LweCpK_ezn zJj`qk_)*RsOI&p4LMF53P;uw*RSgBTm@Y?}@&HiTV)Rd+P$P!4x48HkgT|unHF^#D z_W`pSAx}u2CeUb^)OL6X^rYYjQ*^fbu~fM#H@UDxCXxJpqKvnpMio#)dYUb6A=cto zX3ajVLFmvOo;Zcls(k^5o2=mFrn=x?RGRDv)3oQT>k141;L4?P|g;f;CT6uoCA1Mi*e? zY;TEe6xTMs^JxR|M2)R5e>QH*pU(_vlK=~b@@nzWwdf6Q0t})yZqlrWDpygUoNem- zZ>Na6>9>GJ_Xzm|1mMv>Cr*oiDe;pG9wYLFJsHqFTlf>(ra5h-8>#R}_n5BxMZLKN zz9nE!M~cylW1&V!Gs3;Y!DaM$E(#TE3@8D5KaS@5@`3QlG+zWOH}JpC{Dh4yocYt* zXY-2sX04R=IU~S;b7JHC4S~VysYtCLcP0}Ps_9*-xV2}{1y~bXCHjNR9za4=De?x; zEI8oVariYAB?7DT+E#7YNfH~yukM_LChU0r@DPZxa~lCV?nU2EKI;5DGQrn5c+96y<3UUsUZ#?TR|UJJZk6vh_pQXhU8YW$S? zEiPaKHT$$5YDFdpYQBvg0vanaB~F$4yoKMGT5*6Hv%OgRWpSZLB{rjd8*mSP43!p% zM^|z@>vylF_^#^9iGUAzzHmyR)4n6~t`adpKBG<5?MroqHY5S6D%+^=83H1N23EOGHot^TV}Zu>nS z+}PNu7QVP3lt0HtyYMbzJl@NgN1cu&*k`7pds~Y4j-9UfU)Ldo(o}m`E zbh~Mu^nRPF&{Vg{e-~8r8*3+|v^PPw2{Y&a>3|%&7nX(B7d7C`Mu|o&U{8?$5fndP z3{0tR)dak4!oQk((I{~n_*;tprfGh&$e)c>LV>Qg{_fmg>bj{I1K^*O?_;p}5PKJg z&;IY1_ycUC|Et8$^}}CUxoNR$fe-$+rdA>PzeK`c(W4rErb|m)%<*7`+7G*Kc)yqY zs<*!2BdBirjE}>f3vaRpP!aH?bU&o^s;8RPYgkF)2=Qqc_ICRQ*=F?ILc@L~8MifPmPXDeWV*4B$gYwJZ+07~)Vd{(4&@pN8H$2VM(rgA#y%wFXDz^bBp$2;$HA2yC%~$=4Whd}- zkLz!EGg`E#4Ivv>wExrIxqq{r?R^}h(=l~=NsrNs>WnHy8#7E@>NZoC3Ps(jbxUvb zMpPmpNSK}}Rmv2ht)Qw(s-#s!ghZ#KR7grgS|q8GrbMD75lJLDUp?zNYu5Ap3(xnb zz4qEGYkj}jd%xfN_5Q52-7%;S(=ZfvK~3J2ppcO=u;kZO)0SI?E+>vTI7_J2P-4C9 z>hf;U3f^z2`jRvEWheoH&UlR7iW%6Oms?HyDH9fiREmioBn$@UH zp#$Y;Q~S9~@r5N*V|lKQ5vV>OvKr-HjE=$SeD&krJL~a^9)WQi=hk0#2JxUFo) zcbT^G=?GiMWzmCTl!G!=@x8j(Aa#{0OH@mH#y#Fp@~8Q~@?e!!eS9?8t+sz@wKHJ& zrg;NLBBWxgps^lBjZnuI^>p%4IF_?vbs9A6v6@dinI$R>9=No|x-Um7%Gg69vX)() z8M*fNG0j5I1#_nlKzsOtOh~o1zmKTJKWL+RK<8=;C2Psoj1arVr~d(6oz4jhI>d_9 zGwdvMS9RWQzJiu7@Nx95Z{>uZZ8nIQiwAo@g+%~mVV>2ea#}>asc8bR*+ac5IGC^4 zS;;duxH9hQtPSz0+iww&eFT@Jfzn{kGz1X?I)P(tzYlKst4){-GYY;8frXeK>Bh>DZ{Cc% zX?z5vV7=A8YUPA%#*rDI+6PGo&m;kCC>f!#qPcn`;yW`DS=`i%NpBqTonvm7bGa)IclxC7 z)fE*={kpT(Ld5%~EU^e;LWv7wB!#JI?}S0{)u9}xA_~o1h@9B>N_rgherSNC>Le#| zX`XBQVeE~5+*szTLgiIPqU^~>*t%oHE<72G9JyQb3V9emHCWpo4BS%DHB1IFmQQzT z6Y<+L+QZO^U4#fkY%S`zZ*e(ybh5nN6PzM=8rK;DdY$0U>OQ7^=tzsZWN-!CUe89b z6gM-}040n%V(PJQI?~{Z6-lbk4()9()=8z44BvqHHo9M%_k=#4hxaD4jHN!3%b^Bw zPGQC4(We8vPvq>0i@G0nXuBy+1wu!6Z}Vv$u^^}mr+M9)r2G5hae{h~O@zDsq-7EP zN}W<6TQu*{k-F+)(JKo!fLoL{vvxrqz^5pIV}4zW72M2F&`HbN+e7e5_rCt=;sT3b z>C8B8VveTF8!SP9D6Sr&Y?cgt(+~$nW>r=HMVzc!*E~{kcUqv3ty@eMkcNf8>*k;Y znQts0JL?9G!8YzmDcA~5wIm=20b-4SttBZm(c4tclN0B~{)(a74RwNbG}Mk4cnB4b%fT)YkO?Gd5;sEeCfwz;O=3zTf)`U6 zTaO4giON*Mp5?IxRf8rS8l_2co_T$i02u1qu?FaRF8PmQHTEVlFCi(c;iHC6UZyKS-9jr zQWc^=H9%*C#Lv3wcp<%kv&CH>V$BcOp-_h_1Ic)y8x;)zfS)4?gV{N*w~0Z ziiXA3nvlGxzaR?2oX|&OT?A2LCXVN7+t*K{$<}9KL-kg_fbN$PbY^<4u(6dt2}1Uk zNf?RF@v-)ObYR1NAAQh$t*~P1MiSRwP4#CYxLGj&y~PLyMnS^Dns1l2FlY+APt0j> zTzQsoz#<|9F$tfCkzwBvIwX@OF1O?nS@HZXx+0Nv3NU0BQjhKlkz@K67pOIiSN#uK zJ(sktp#$p{PR5?t&33uVwiOk2x`i<84#(}66q(= zZincg2dK|6S>Wh8DG?;)(wcBE|iR{{dD=qNiA6H)j`OL>7h z!l~};%E(BSo`CoX^;??vNZvLT49vxTW~hkuP`*Zt7U2CR8Z)qHEa#zr#$%YOlyNn= zyNlBjsA^Ja=na(4L|i+|$FmynQ_$qe#X#AU#^47%tBR^C388s>7Y zHaL`US*ZNkV75X3XW2)eJ$0Qg(&@gf;@)|?9Zk)l^pv{RpW0iT#yTE!!5RzZ0)|vU zN4ywoWdWv>Uf{{1ou?h<^e~vyG}u=D3McTYSY>Oh?@%#aA;F)+wgEeipLeP0MD(vi z4XQ zF}iVvX_RF*XKDh?J-+pQ2-rEWQomd=GM!^oUNg#nyZ2Xc-g%6v6&<$eN^GMkwUXH^ z{rpiI$4zmQ*HivHW8Ex0mCA!+n~7xt{F-RS>)|`kvwA-ULy}{(C|gwOR_Y;wiKAO1 zi^5Tb%12)zFA7*MT7}`Y{+LKMucBKZ{KUM$$9(Tv?D?c)~O99=piL zE=U^)d_0E}oMk0XcH=2r=N?4s84h-t`CYwZ+i!j>Pf2$rS>73VJp<=IQN`hHJ2QVP zb9R!A4%lTnLyHwza=Iinj%s1^C^RtflCpw6w0GB)UeQD+`1Zg!4e;@Pci*i zV&3NR*b>@bciyS*6XR9DLYf(o6Hth#;-@K=0=M&JA<3mpdP~%S!%;G$(c%;J#rq}c zbG^Ia-Uk;fnYNw{h^mtn&0Vz4To>e(@vU|4OkvmTP}0ey)VhD%Vl&ZY>>zSV z#&tv>GGr3N+llj(chA@p>Ld6a>w&w$l2%;jrt1!jcAq*neq$%n!8iN2JLWyll=tuO zpBiIqYS5T>6MvXydk1ABlF4FGKufsKV;txE;soT8Cw0cHljXw!khcS$h^tX!1N)5M zY~^2-#>>lp7z55M{+ep`zQk(te*T|W^7q}N`Mu<{stwG#XZHfD=uhUOUo$R$>_$a| zFeE_r-L)2YA^%L4GbzM$_CdeOgcLX2Fypk#fFp+%e@KaBA#&)2dt{3x`-`H^`%u2R z4FTl!k@U$&f5fvP4(VA0)Oq=5IX6byUydPsBRqW=bV>)PTsa$?(F^t~*iq(2H(e2l zSDp-i%jUG5YIVuJ6XCVpC-ghW-DS7I3h#yJT9dzfH&B^U$qXCyC#R z(iGxjpM+w$&RlE@htGSZqtD+;5FCt87LV_`X9fa_CbCg|#WyV~!q@xERvzf9UjF9S zFK7c;0~fwfn1<^B<R+`QPm|UuFP!Gyuf_&_~_2AWK?dPwj5s zykQGbRcx!~Y2dm5z5#rnPBvHhfsJ|9_Odtn=`V{)+pHG?HD7NH#{K{E_b>1xRJ*%P Wk~8|xZ_mxF^+UV^y_yf7y83U?w>WhG literal 0 HcmV?d00001 diff --git a/makejdk-any-platform.1 b/makejdk-any-platform.1 index c246f24f5..04675c211 100755 --- a/makejdk-any-platform.1 +++ b/makejdk-any-platform.1 @@ -1,80 +1,160 @@ .\" Manpage for makejdk-any-platform.sh -.TH "0.1.0" "Apache 2" +.TH "2.0.0" "Apache 2" .SH NAME -makejdk.sh +makejdk-any-platform.sh .SH SYNOPSIS -"./makejdk-any-platform.sh --version [version] [options]" +"./makejdk-any-platform.sh [options] version" .SH DESCRIPTION -The simplest way to build OpenJDK using our scripts is to run makejdk-any-platform.sh -c --version and have your user be in the Docker group on the machine (or prefix all of your Docker commands with sudo. This script can be used to create a Docker container that will be configured with all of the required dependencies and a base operating system in order to build OpenJDK. +This script is the entry point for building (Adopt) OpenJDK binaries. Building +natively or in a docker container are both supported. This script (and its +supporting scripts) have defaults, but you can override these as needed. + +makejdk-any-platform.sh and it's supporting scripts will auto detect the +platform and architecture it is running on and configures the OpenJDK build +accordingly. The supporting scripts will also download and locally install any +required dependencies for the OpenJDK build, e.g. The ALSA sound and Freetype +font libraries. + +Many of the configuration options are passed through to the \fBconfigure\fR and +\fBmake\fR commands that OpenJDK uses to build binaries. Please see the +appropriate \fIREADME-builds.html\fR file for the OpenJDK source repository +that you are building for further details. + +\fBExample:\fR The simplest use case to run is: + +"./makejdk-any-platform.sh --docker jdk8u" + +This will start a Docker container and build you the latest Java 8 AdoptOpenJDK +binary from the source at https://github.com/AdoptOpenJDK/openjdk-jdk8u + +Please visit https://www.adoptopenjdk.net for further support .SH VERSIONS .TP -.BR \jdk8u -https://github.com/AdoptOpenJDK/openjdk-jdk8u +.BR jdk8u +Build Java 8, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk8u +.TP +.BR jdk9 +Build Java 9, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk9 .TP -.BR \jdk9 -https://github.com/AdoptOpenJDK/openjdk-jdk9u +.BR jdk10 +Build Java 10, defaults to https://github.com/AdoptOpenJDK/openjdk-jdk10 .TP -.BR \jdk10 -https://github.com/AdoptOpenJDK/openjdk-jdk10u +.BR jfx +Build OpenJFX, defaults to https://github.com/AdoptOpenJDK/openjdk-jfx .TP .BR \jdk11 https://github.com/AdoptOpenJDK/openjdk-jdk11 .TP +.BR amber +Build Project Amber, defaults to https://github.com/AdoptOpenJDK/openjdk-amber .SH OPTIONS .TP -.BR \-s ", " \-\-source " " \fI\fR -specify the location for the source and dependencies to be cloned, defaults to ./openjdk. If it is specified, docker is not used +.BR \-b ", " \-\-branch " " \fI\fR +specify a custom branch to build from, e.g. dev. +For reference, AdoptOpenJDK GitHub source repos default to the \fI\fR +branch which may contain a very small diff set to the \fI\fR branch +(which is a clone from the OpenJDK mercurial forest). +.TP +.BR \-B ", " \-\-build-number " " \fI\fR +specify the OpenJDK build number to build from, e.g. b12. +For reference, OpenJDK version numbers look like 1.8.0_162-b12 (for Java 8) or +9.0.4+11 (for Java 9+) with the build number being the suffix at the end. +.TP +.BR \-\-build-variant " " \fI\fR +specify a OpenJDK build variant, e.g. openj9. +For reference, the default variant is hotspot and does not need to be specified. +.TP +.BR \-c ", " \-\-clean-docker-build +removes the existing docker container and persistent volume before starting +a new docker based build. +.TP +.BR \-C ", " \-\-configure-args " " \fI\fR +specify any custom user configuration arguments. .TP .BR \-d ", " \-\-destination " " \fI\fR -specify the location for the tarball (eg. /path/ or /path/here.tar.gz) +specify the location for the built binary, e.g. /path/. +This is typically used in conjunction with \fB<-T>\fR to create a custom path +/ file name for the resulting binary. .TP -.BR \-r ", " \-\-repository " " \fI\fR -specify a custom repository (eg. username/openjdk-jdk8u) +.BR \-D ", " \-\-docker +build OpenJDK in a docker container. .TP -.BR \-b ", " \-\-branch " " \fI\fR -specify a custom branch (eg. dev) +.BR \-\-disable-shallow-git-clone +disable the default \fB--depth=1\fR shallow cloning of git repo(s). .TP -.BR \-k ", " \-\-keep -reuse docker container (prevents deleting) +.BR \-f ", " \-\-freetype-dir +specify the location of an existing FreeType library. +This is typically used in conjunction with \fB<-F>\fR. .TP -.BR \-j ", " \-\-jtreg -run jtreg after building +.BR \-F ", " \-\-skip-freetype +skip building Freetype automatically. +This is typically used in conjunction with \fB<-f>\fR. .TP -.BR \-js ", " \-\-jtreg-subsets -select one or more jtreg tests to run +.BR \-h ", " \-\-help +print this help. .TP -.BR \-sf ", " \-\-skip-freetype -skip building freetype -.BR \-nc ", " \-\-no-colour -disable colour output +.BR \-i ", " \-\-ignore-container +ignore the existing docker container if you have one already. .TP -.BR \-ftd ", " \-\-freetype-dir -specify the location of an existing FreeType library that can be used for the OpenJDK build process +.BR \-J ", " \-\-jdk-boot-dir " " \fI\fR +specify the JDK boot dir. +For reference, OpenJDK needs the previous version of a JDK in order to build +itself. You should select the path to a JDK install that is N-1 versions below +the one you are trying to build. .TP -.BR \-S ", " \-\-ssh -use ssh when cloning git +.BR \-k ", " \-\-keep +if using docker, keep the container after the build. .TP -.BR \-dsgc ", " \-\-disable-shallow-git-clone -disable shallow cloning of git repo(s) using the --depth=1 CLI option +.BR \-n ", " \-\-no-colour +disable colour output. .TP -.BR \-bv ", " \-\-variant " " \fI\fR -specify a custom build variant (eg. openj9) +.BR \-p ", " \-\-processors " " \fI\fR +specify the number of processors to use for the docker build. .TP +.BR \-r ", " \-\-repository " " \fI\fR +specify the repository to clone OpenJDK source from, +e.g. https://github.com/karianna/openjdk-jdk8u. .TP -.BR \-c ", " \-\-clean-docker-build -clean docker data volume +.BR \-s ", " \-\-source " " \fI\fR +specify the location to clone the OpenJDK source (and dependencies) to. +.TP +.BR \-S ", " \-\-ssh +use ssh when cloning git. .TP +.BR \-\-sign \fI\fR +specify the location for the certificate. For windows this is the p12 +certificate to sign the DLL. .TP -.BR \-t ", " \-\-tag " " \fI\fR -specify a custom tag +.BR \-\-sudo +run the docker container as root. .TP +.BR \-t ", " \-\-tag " " \fI\fR +specify the repository tag that you want to build OpenJDK from. .TP -.BR \-\-sign " " \fI\fR -specify the location for the windows p12 certificate. Used only for windows builds to sign DLL +.BR \-\-tmp-space-build +use a temporary workspace if the main one is unavailable. .TP +.BR \-T ", " \-\-target-file-name " " \fI\fR +specify the final name of the OpenJDK binary. +This is typically used in conjunction with \fB<-D>\fR to create a custom file +name for the resulting binary. .TP -.BR \-ca ", " \-\-configure-args " " \fI\fR -specify a custom configuration arguments +.BR \-u ", " \-\-update-version " " \fI\fR +specify the update version to build OpenJDK from, e.g. 162. +For reference, OpenJDK version numbers look like 1.8.0_162-b12 (for Java 8) or +9.0.4+11 (for Java 9+) with the update number being the number after the '_' +(162) or the 3rd position in the semVer version string (4). +This is typically used in conjunction with \fB<-b>\fR. .TP +.BR \-V ", " \-\-jvm-variant " " \fI\fR +specify the JVM variant (server or client), defaults to server. + +.SH EXAMPLE USAGE + +Some common example usages: + +"./makejdk-any-platform --docker jdk8u" + +"./makejdk-any-platform -s /home/openjdk10/src -d /home/openjdk/target -T MyOpenJDK10.tar.gz jdk10" diff --git a/makejdk-any-platform.sh b/makejdk-any-platform.sh index 854c7d729..5c1f3a827 100755 --- a/makejdk-any-platform.sh +++ b/makejdk-any-platform.sh @@ -1,4 +1,6 @@ #!/bin/bash + +################################################################################ # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -12,119 +14,49 @@ # See the License for the specific language governing permissions and # limitations under the License. # +################################################################################ -# Script to clone the OpenJDK source then build it +################################################################################ +# +# Entry point to build (Adopt) OpenJDK binaries for any platform. +# +# 1. Source scripts to support configuration, docker builds and native builds. +# 2. Parse the Command Line Args +# 3. Set a host of configuration options based on args, platform etc +# 4. Display and then persist those configuration options +# 5. Build the binary in Docker or natively +# +################################################################################ -# Optionally uses Docker, otherwise you can provide two arguments: -# the area to build the JDK e.g. $HOME/mybuilddir as -s or --source -# and the target destination for the tar.gz e.g. -d or --destination $HOME/mytargetdir -# Both must be absolute paths! You can use $PWD/mytargetdir +# TODO Comment out the 'x' once script is stable. +set -eu -# A simple way to install dependencies persistently is to use our Ansible playbooks +# i.e. Where we are +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -# You can set the JDK boot directory with the JDK_BOOT_DIR environment variable +# Pull in configuration and build support +# shellcheck source=sbin/common/config_init.sh +source "${SCRIPT_DIR}/sbin/common/config_init.sh" -export OS_KERNEL_NAME="" -OS_KERNEL_NAME=$(uname | awk '{print tolower($0)}') -export OS_MACHINE_NAME="" -OS_MACHINE_NAME=$(uname -m) +# shellcheck source=docker-build.sh +source "${SCRIPT_DIR}/docker-build.sh" -# The full forest name, e.g. jdk8, jdk8u, jdk9, jdk9u, etc. -export OPENJDK_FOREST_NAME="" +# shellcheck source=native-build.sh +source "${SCRIPT_DIR}/native-build.sh" -# The abridged, core version name, e.g. jdk8, jdk9, etc. No "u"s. -export OPENJDK_CORE_VERSION="" +# shellcheck source=configureBuild.sh +source "${SCRIPT_DIR}/configureBuild.sh" -export BUILD_VARIANT="" -export REPOSITORY="" +echo "Starting $0 to configure, build (Adopt)OpenJDK binary" -counter=0 -for i in "$@"; do - (( counter++ )) - case "$i" in - "--version" | "-v") - (( counter++ )) - string="\$$counter" - OPENJDK_FOREST_NAME=$(echo "$@" | awk "{print $string}") - export OPENJDK_CORE_VERSION=${OPENJDK_FOREST_NAME} - if [[ $OPENJDK_FOREST_NAME == *u ]]; then - export OPENJDK_CORE_VERSION=${OPENJDK_FOREST_NAME%?} - fi - # Switch it back to stop it being out of sync with i - (( counter-- )) - ;; - "--variant" | "-bv") - (( counter++ )) - string="\$$counter" - BUILD_VARIANT=$(echo "$@" | awk "{print $string}") - # Switch it back to stop it being out of sync with i - (( counter-- )) - ;; - esac -done +# Configure the build, display the parameters and write the config to disk +# see ${SCRIPT_DIR}/sbin/common/config_init.sh for details +configure_build "$@" +writeConfigToFile -export COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG="false" -export COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG="false" -if [ "$OPENJDK_CORE_VERSION" == "jdk9" ] || [ "$OPENJDK_CORE_VERSION" == "jdk10" ] || [ "$OPENJDK_CORE_VERSION" == "jdk11" ] || [ "$OPENJDK_CORE_VERSION" == "amber" ]; then - export JDK_PATH="jdk" - export JRE_PATH="jre" - if [ "$OPENJDK_CORE_VERSION" == "jdk9" ]; then - export COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG="true" - export COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG="true" - fi - export CONFIGURE_ARGS_FOR_ANY_PLATFORM=${CONFIGURE_ARGS_FOR_ANY_PLATFORM:-"--disable-warnings-as-errors"} -elif [ "$OPENJDK_CORE_VERSION" == "jdk8" ]; then - export COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG="false" - export COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG="true" - export JDK_PATH="j2sdk-image" - export JRE_PATH="j2re-image" +# Let's build and test the (Adopt) OpenJDK binary in Docker or natively +if [ "${BUILD_CONFIG[USE_DOCKER]}" == "true" ] ; then + buildOpenJDKViaDocker else - echo "Please specify a version with --version or -v , either jdk9, jdk10 or jdk8, with or without a \\'u\\' suffix." - man ./makejdk-any-platform.1 - exit 1 + buildOpenJDKInNativeEnvironment fi - -REPOSITORY="${REPOSITORY:-adoptopenjdk/openjdk-$OPENJDK_FOREST_NAME}"; -REPOSITORY="$(echo "${REPOSITORY}" | awk '{print tolower($0)}')"; - -case "$OS_MACHINE_NAME" in -"s390x") - if [ "$OPENJDK_CORE_VERSION" == "jdk8" ] && [ "$BUILD_VARIANT" != "openj9" ]; then - export JVM_VARIANT=${JVM_VARIANT:-zero} - else - export JVM_VARIANT=${JVM_VARIANT:-server} - fi - - export BUILD_FULL_NAME=${BUILD_FULL_NAME:-linux-s390x-normal-${JVM_VARIANT}-release} - S390X_MAKE_ARGS="CONF=${BUILD_FULL_NAME} DEBUG_BINARIES=true images" - export MAKE_ARGS_FOR_ANY_PLATFORM=${MAKE_ARGS_FOR_ANY_PLATFORM:-$S390X_MAKE_ARGS} -;; - -"ppc64le") - export JVM_VARIANT=${JVM_VARIANT:-server} - export BUILD_FULL_NAME=${BUILD_FULL_NAME:-linux-ppc64-normal-${JVM_VARIANT}-release} - # shellcheck disable=SC1083 - export FREETYPE_FONT_BUILD_TYPE_PARAM=${FREETYPE_FONT_BUILD_TYPE_PARAM:="--build=$(rpm --eval %{_host})"} -;; - -"armv7l") - export JVM_VARIANT=${JVM_VARIANT:-zero} - export MAKE_ARGS_FOR_ANY_PLATFORM=${MAKE_ARGS_FOR_ANY_PLATFORM:-"DEBUG_BINARIES=true images"} - export CONFIGURE_ARGS_FOR_ANY_PLATFORM=${CONFIGURE_ARGS_FOR_ANY_PLATFORM:-"--with-jobs=${NUM_PROCESSORS}"} -;; - -"aarch64") - export FREETYPE_FONT_VERSION="2.5.2" -;; -esac - -case "$OS_KERNEL_NAME" in -"aix") - export MAKE_COMMAND_NAME=${MAKE_COMMAND_NAME:-"gmake"} -;; -"SunOS") - export MAKE_COMMAND_NAME=${MAKE_COMMAND_NAME:-"gmake"} -;; - -esac -./makejdk.sh "$@" diff --git a/makejdk.sh b/makejdk.sh deleted file mode 100755 index 3ad29f86c..000000000 --- a/makejdk.sh +++ /dev/null @@ -1,475 +0,0 @@ -#!/bin/bash -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -# Script to clone the OpenJDK source then build it - -# Optionally uses Docker, otherwise you can provide two arguments: -# the area to build the JDK e.g. $HOME/mybuilddir as -s or --source -# and the target destination for the tar.gz e.g. -d or --destination $HOME/mytargetdir -# Both must be absolute paths! You can use $PWD/mytargetdir - -# A simple way to install dependencies persistently is to use our Ansible playbooks - -# You can set the JDK boot directory with the JDK_BOOT_DIR environment variable - -SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -# shellcheck disable=SC1090 -source "$SCRIPT_DIR/sbin/common-functions.sh" - -OPENJDK_REPO_NAME=${OPENJDK_REPO_NAME:-openjdk} -SHALLOW_CLONE_OPTION="--depth=1" - -DOCKER_SOURCE_VOLUME_NAME="openjdk-source-volume" -CONTAINER=openjdk_container -TMP_CONTAINER_NAME=openjdk-copy-src -CLEAN_DOCKER_BUILD=false - -export COPY_TO_HOST=false -export USE_DOCKER=false - -TARGET_DIR_IN_THE_CONTAINER="/openjdk/target/" - -WORKING_DIR="" -USE_SSH=false -TARGET_DIR="" -BRANCH="" -TAG="" -KEEP=false -JTREG=false -BUILD_VARIANT=${BUILD_VARIANT-:""} -USER_SUPPLIED_CONFIGURE_ARGS="" -VERSION="" - -JVM_VARIANT=${JVM_VARIANT:-server} - -OPENJDK_UPDATE_VERSION="" -OPENJDK_BUILD_NUMBER="" - -PATH_BUILD="" - -determineBuildProperties - -sourceFileWithColourCodes() -{ - # shellcheck disable=SC1091 - source ./sbin/colour-codes.sh -} - -sourceSignalHandler() -{ - # shellcheck disable=SC1091 - source sbin/signalhandler.sh -} - -parseCommandLineArgs() -{ - while [[ $# -gt 0 ]] && [[ ."$1" = .-* ]] ; do - opt="$1"; - shift; - case "$opt" in - "--" ) break 2;; - - "--source" | "-s" ) - WORKING_DIR="$1"; shift;; - - "--ssh" | "-S" ) - USE_SSH=true;; - - "--destination" | "-d" ) - TARGET_DIR="$1"; shift;; - - "--repository" | "-r" ) - REPOSITORY="$1"; shift;; - - "--branch" | "-b" ) - BRANCH="$1"; shift;; - - "--tag" | "-t" ) - TAG="$1"; SHALLOW_CLONE_OPTION=""; shift;; - - "--keep" | "-k" ) - KEEP=true;; - - "--clean-docker-build" | "-c" ) - CLEAN_DOCKER_BUILD=true;; - - "--jtreg" | "-j" ) - JTREG=true;; - - "--jtreg-subsets" | "-js" ) - JTREG=true; JTREG_TEST_SUBSETS="$1"; shift;; - - "--no-colour" | "-nc" ) - COLOUR=false;; - - "--sign" ) - export SIGN=true; export CERTIFICATE="$1"; shift;; - - "--disable-shallow-git-clone" | "-dsgc" ) - SHALLOW_CLONE_OPTION=""; shift;; - - "--skip-freetype" | "-sf" ) - export FREETYPE=false;; - - "--version" | "-v" ) - VERSION="$1"; shift;; - - "--freetype-dir" | "-ftd" ) - export FREETYPE_DIRECTORY="$1"; shift;; - - "--variant" | "-bv" ) - export BUILD_VARIANT="$1"; shift;; - - "--configure-args" | "-ca" ) - export USER_SUPPLIED_CONFIGURE_ARGS="$1"; shift;; - - *) echo >&2 "Invalid option: ${opt}"; man ./makejdk-any-platform.1; exit 1;; - esac - done -} - -doAnyBuildVariantOverrides() -{ - if [[ "${BUILD_VARIANT}" == "openj9" ]]; then - # current location of Extensions for OpenJDK9 for OpenJ9 project - REPOSITORY="ibmruntimes/openj9-openjdk-${OPENJDK_CORE_VERSION}" - [ -z "$BRANCH" ] && BRANCH="openj9" - fi - if [[ "${BUILD_VARIANT}" == "SapMachine" ]]; then - # current location of SAP variant - REPOSITORY="SAP/SapMachine" - [ -z "$BRANCH" ] && BRANCH="sapmachine10" # sapmachine10 is the current branch for OpenJDK10 mainline (equivalent to jdk/jdk10) - fi -} - -checkIfDockerIsUsedForBuildingOrNot() -{ - # Both a working directory and a target directory provided - if [ ! -z "$WORKING_DIR" ] && [ ! -z "$TARGET_DIR" ] ; then - # This uses sbin/build.sh directly - echo "Not using Docker, working area will be ${WORKING_DIR}, target for the JDK will be ${TARGET_DIR}" - fi - - # No working directory and no target directory provided - if [ -z "${WORKING_DIR}" ] && [ -z "${TARGET_DIR}" ] ; then - echo "No parameters provided, using Docker." - USE_DOCKER=true - elif [ ! -z "$TARGET_DIR" ] && [ -z "$WORKING_DIR" ] ; then - # Target directory is defined but the working directory isn't - # Calls sbin/build.sh inside of Docker followed by a docker cp command - echo "Using Docker, target directory for the tgz on the host: ${TARGET_DIR}" - USE_DOCKER=true - fi -} - -checkIfDockerIsUsedShouldTheContainerBePreserved() -{ - if [ "${KEEP}" == "true" ] ; then - echo "We'll keep the built Docker container if you're using Docker." - else - echo "We'll remove the built Docker container if you're using Docker." - fi -} - -setDefaultIfBranchIsNotProvided() -{ - if [ -z "$BRANCH" ] ; then - echo "BRANCH is undefined so checking out dev" - BRANCH="dev" - fi -} - -setWorkingDirectoryIfProvided() -{ - if [ -z "${WORKING_DIR}" ] ; then - echo "WORKING_DIR is undefined so setting to ${PWD}." - WORKING_DIR=$PWD - else - echo "Working dir is ${WORKING_DIR}." - fi -} - -setTargetDirectoryIfProvided() -{ - if [ -z "${TARGET_DIR}" ] ; then - echo "TARGET_DIR is undefined so setting to $PWD" - TARGET_DIR=$PWD - # Only makes a difference if we're in Docker - echo "If you're using Docker the build artifact will not be copied to the host." - else - echo "Target directory is ${TARGET_DIR}" - COPY_TO_HOST=true - echo "If you're using Docker we'll copy the build artifact to the host." - fi -} - -checkOpenJDKGitRepo() -{ - if [ -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ] && { [ "$OPENJDK_CORE_VERSION" == "jdk8" ] || [ "$OPENJDK_CORE_VERSION" == "jdk9" ] || [ "$OPENJDK_CORE_VERSION" == "jdk10" ] || [ "$OPENJDK_CORE_VERSION" == "jdk11" ] ; } ; then - GIT_VERSION=$(git --git-dir "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" remote -v | grep "${OPENJDK_CORE_VERSION}") - echo "${GIT_VERSION}" - if [ "$GIT_VERSION" ]; then - # The repo is the correct JDK Version - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || return - echo "Will reset the repository at $PWD in 10 seconds..." - sleep 10 - echo "Pulling latest changes from git repo" - - showShallowCloningMessage "fetch" - git fetch --all ${SHALLOW_CLONE_OPTION} - git reset --hard origin/$BRANCH - if [ ! -z "$TAG" ]; then - git checkout "$TAG" - fi - git clean -fdx - else - # The repo is not for the correct JDK Version - echo "Incorrect Source Code for ${OPENJDK_FOREST_NAME}. Will re-clone" - rm -rf "${WORKING_DIR:?}/${OPENJDK_REPO_NAME:?}" - cloneOpenJDKGitRepo - fi - cd "${WORKING_DIR}" || return - elif [ ! -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ] ; then - # If it doesn't exist, clone it - echo "Didn't find any existing openjdk repository at WORKING_DIR (set to ${WORKING_DIR}) so cloning the source to openjdk" - cloneOpenJDKGitRepo - fi -} - -cloneOpenJDKGitRepo() -{ - if [[ "$USE_SSH" == "true" ]] ; then - GIT_REMOTE_REPO_ADDRESS="git@github.com:${REPOSITORY}.git" - else - GIT_REMOTE_REPO_ADDRESS="https://github.com/${REPOSITORY}.git" - fi - - showShallowCloningMessage "cloning" - # I want to ensure SHALLOW_CLONE_OPTION isn't explcitly wrapped in quotes PR#455 - # shellcheck disable=SC2206 - GIT_CLONE_ARGUMENTS=($SHALLOW_CLONE_OPTION "-b" "$BRANCH" "$GIT_REMOTE_REPO_ADDRESS" "${WORKING_DIR}/${OPENJDK_REPO_NAME}") - - echo "git clone ${GIT_CLONE_ARGUMENTS[*]}" - git clone "${GIT_CLONE_ARGUMENTS[@]}" - if [ ! -z "$TAG" ]; then - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || exit 1 - git checkout "$TAG" - fi - - # Building OpenJDK with OpenJ9 must run get_source.sh to clone openj9 and openj9-omr repositories - if [ "$BUILD_VARIANT" == "openj9" ]; then - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || return - bash get_source.sh - fi - -} - -# TODO This only works for jdk8u based releases. Will require refactoring when jdk9 enters an update cycle -getOpenJDKUpdateAndBuildVersion() -{ - - if [ -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ]; then - # It does exist and it's a repo other than the AdoptOpenJDK one - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || return - echo "Pulling latest tags and getting the latest update version using git fetch -q --tags ${SHALLOW_CLONE_OPTION}" - git fetch -q --tags ${SHALLOW_CLONE_OPTION} - OPENJDK_REPO_TAG=${TAG:-$(getFirstTagFromOpenJDKGitRepo)} - if [[ "${OPENJDK_REPO_TAG}" == "" ]] ; then - echo "Unable to detect git tag" - exit 1 - else - echo "OpenJDK repo tag is $OPENJDK_REPO_TAG" - fi - - OPENJDK_UPDATE_VERSION=$(echo "${OPENJDK_REPO_TAG}" | cut -d'u' -f 2 | cut -d'-' -f 1) - OPENJDK_BUILD_NUMBER=$(echo "${OPENJDK_REPO_TAG}" | cut -d'b' -f 2 | cut -d'-' -f 1) - echo "Version: ${OPENJDK_UPDATE_VERSION} ${OPENJDK_BUILD_NUMBER}" - cd "${WORKING_DIR}" || return - - fi - -} - -showShallowCloningMessage() -{ - mode=$1 - if [[ "$SHALLOW_CLONE_OPTION" == "" ]]; then - echo "Git repo ${mode} mode: deep (preserves commit history)" - else - echo "Git repo ${mode} mode: shallow (DOES NOT contain commit history)" - fi -} - -testOpenJDKViaDocker() -{ - if [[ "$JTREG" == "true" ]]; then - mkdir -p "${WORKING_DIR}/target" - docker run \ - -v "${DOCKER_SOURCE_VOLUME_NAME}:/openjdk/build" \ - -v "${WORKING_DIR}/target:${TARGET_DIR_IN_THE_CONTAINER}" \ - --entrypoint /openjdk/sbin/jtreg.sh "${CONTAINER}" - fi -} - -createPersistentDockerDataVolume() -{ - #Create a data volume called $DOCKER_SOURCE_VOLUME_NAME, - #this gets mounted at /openjdk/build inside the container and is persistent between builds/tests - #unless -c is passed to this script, in which case it is recreated using the source - #in the current ./openjdk directory on the host machine (outside the container) - docker volume inspect $DOCKER_SOURCE_VOLUME_NAME > /dev/null 2>&1 - DATA_VOLUME_EXISTS=$? - - if [[ "$CLEAN_DOCKER_BUILD" == "true" || "$DATA_VOLUME_EXISTS" != "0" ]]; then - - echo "Removing old volumes and containers" - docker rm -f "$(docker ps -a --no-trunc | grep $CONTAINER | cut -d' ' -f1)" || true - docker volume rm "${DOCKER_SOURCE_VOLUME_NAME}" || true - - echo "Creating tmp container and copying src" - docker volume create --name "${DOCKER_SOURCE_VOLUME_NAME}" - docker run -v "${DOCKER_SOURCE_VOLUME_NAME}":/openjdk/build --name "$TMP_CONTAINER_NAME" ubuntu:14.04 /bin/bash - docker cp openjdk "$TMP_CONTAINER_NAME":/openjdk/build/ - - echo "Removing tmp container" - docker rm -f "$TMP_CONTAINER_NAME" - fi -} - -buildDockerContainer() -{ - echo Building docker container - docker build -t "${CONTAINER}" "${PATH_BUILD}" --build-arg "OPENJDK_CORE_VERSION=${OPENJDK_CORE_VERSION}" - if [[ "${BUILD_VARIANT}" != "" && -f "${PATH_BUILD}/Dockerfile-${BUILD_VARIANT}" ]]; then - CONTAINER="${CONTAINER}-${BUILD_VARIANT}" - echo Building dockerfile variant "${BUILD_VARIANT}" - docker build -t "${CONTAINER}" -f "${PATH_BUILD}/Dockerfile-${BUILD_VARIANT}" "${PATH_BUILD}" --build-arg "OPENJDK_CORE_VERSION=${OPENJDK_CORE_VERSION}" - fi -} - -buildAndTestOpenJDKViaDocker() -{ - - - PATH_BUILD="docker/${OPENJDK_CORE_VERSION}/x86_64/ubuntu" - - if [ -z "$(command -v docker)" ]; then - echo "Error, please install docker and ensure that it is in your path and running!" - exit - fi - - echo "Using Docker to build the JDK" - - createPersistentDockerDataVolume - - - # Copy our scripts for usage inside of the container - rm -r "${PATH_BUILD}/sbin" - cp -r "${SCRIPT_DIR}/sbin" "${PATH_BUILD}/sbin" 2>/dev/null - - - # Keep is undefined so we'll kill the docker image - - if [[ "$KEEP" == "true" ]] ; then - # shellcheck disable=SC2086 - if [ "$(docker ps -a | grep -c \"$CONTAINER\")" == 0 ]; then - echo "No docker container found so creating '$CONTAINER' " - buildDockerContainer - fi - else - echo "Building as you've not specified -k or --keep" - docker ps -a | awk '{ print $1,$2 }' | grep "$CONTAINER" | awk '{print $1 }' | xargs -I {} docker rm -f {} - buildDockerContainer - fi - - mkdir -p "${WORKING_DIR}/target" - - docker run -t \ - -e BUILD_VARIANT="$BUILD_VARIANT" \ - -v "${DOCKER_SOURCE_VOLUME_NAME}:/openjdk/build" \ - -v "${WORKING_DIR}/target":/${TARGET_DIR_IN_THE_CONTAINER} \ - --entrypoint /openjdk/sbin/build.sh "${CONTAINER}" - - testOpenJDKViaDocker - - # Didn't specify to keep - if [[ -z ${KEEP} ]] ; then - docker ps -a | awk '{ print $1,$2 }' | grep "${CONTAINER}" | awk '{print $1 }' | xargs -I {} docker rm {} - fi -} - -testOpenJDKInNativeEnvironmentIfExpected() -{ - if [[ "$JTREG" == "true" ]]; - then - "${SCRIPT_DIR}"/sbin/jtreg.sh "${WORKING_DIR}" "${OPENJDK_REPO_NAME}" "${BUILD_FULL_NAME}" "${VERSION}" "${JTREG_TEST_SUBSETS}" - fi -} - -buildAndTestOpenJDKInNativeEnvironment() -{ - BUILD_ARGUMENTS="" - declare -a BUILD_ARGUMENT_NAMES=("--source" "--destination" "--repository" "--variant" "--update-version" "--build-number" "--repository-tag" "--configure-args") - declare -a BUILD_ARGUMENT_VALUES=("${WORKING_DIR}" "${TARGET_DIR}" "${OPENJDK_REPO_NAME}" "${JVM_VARIANT}" "${OPENJDK_UPDATE_VERSION}" "${OPENJDK_BUILD_NUMBER}" "${TAG}" "${USER_SUPPLIED_CONFIGURE_ARGS}") - - BUILD_ARGS_ARRAY_INDEX=0 - while [[ ${BUILD_ARGS_ARRAY_INDEX} < ${#BUILD_ARGUMENT_NAMES[@]} ]]; do - if [[ ${BUILD_ARGUMENT_VALUES[${BUILD_ARGS_ARRAY_INDEX}]} != "" ]]; - then - BUILD_ARGUMENTS="${BUILD_ARGUMENTS}${BUILD_ARGUMENT_NAMES[${BUILD_ARGS_ARRAY_INDEX}]} ${BUILD_ARGUMENT_VALUES[${BUILD_ARGS_ARRAY_INDEX}]} " - fi - ((BUILD_ARGS_ARRAY_INDEX++)) - done - - echo "Calling ${SCRIPT_DIR}/sbin/build.sh ${BUILD_ARGUMENTS}" - # shellcheck disable=SC2086 - "${SCRIPT_DIR}"/sbin/build.sh ${BUILD_ARGUMENTS} - - testOpenJDKInNativeEnvironmentIfExpected -} - -buildAndTestOpenJDK() -{ - if [ "$USE_DOCKER" == "true" ] ; then - buildAndTestOpenJDKViaDocker - else - buildAndTestOpenJDKInNativeEnvironment - fi -} - -################################################################## - -sourceSignalHandler -parseCommandLineArgs "$@" -doAnyBuildVariantOverrides -if [[ -z "${COLOUR}" ]] ; then - sourceFileWithColourCodes -fi -checkIfDockerIsUsedForBuildingOrNot -checkIfDockerIsUsedShouldTheContainerBePreserved -setDefaultIfBranchIsNotProvided -setWorkingDirectoryIfProvided -setTargetDirectoryIfProvided -time ( - echo "Cloning OpenJDK git repo" - checkOpenJDKGitRepo -) - -time ( - echo "Updating OpenJDK git repo" - getOpenJDKUpdateAndBuildVersion -) - -buildAndTestOpenJDK diff --git a/mercurial-tags/java-tool/MercurialTracker.java b/mercurial-tags/java-tool/MercurialTracker.java index 72d78c877..ad6f07757 100644 --- a/mercurial-tags/java-tool/MercurialTracker.java +++ b/mercurial-tags/java-tool/MercurialTracker.java @@ -84,7 +84,7 @@ public static void main(String[] args) throws IOException, InterruptedException, System.out.println(" * " + switch_buffer); System.out.println(" This one expects value. eg " + switch_buffer + "=30"); System.out.println(" it is setting the max depth of investigated path."); - System.out.println(" Note that teh compelxity is exponential in this case, sostay at 10-30. With eg 100, you can keep ti unlimited anyway."); + System.out.println(" Note that the complexity is exponential in this case, so stay at 10-30. With e.g. 100, you can keep it unlimited anyway."); System.out.println(" Without it,some high-development forests like jdk10 or jdk10-shenandoah"); System.out.println(" will run for many hours, and have unpredictable results"); System.out.println(" If yuou need to use this switch, you probably do not need this program at all."); diff --git a/native-build.sh b/native-build.sh new file mode 100755 index 000000000..fafc5bcf0 --- /dev/null +++ b/native-build.sh @@ -0,0 +1,33 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +################################################################################ +# +# This script deals with the configuration to build (Adopt) OpenJDK natively. +# It's sourced by the makejdk-any-platform.sh script. +# +################################################################################ + +set -eu + +# i.e. Where we are +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +buildOpenJDKInNativeEnvironment() +{ + displayParams + bash "${SCRIPT_DIR}"/sbin/build.sh +} diff --git a/pipelines/build/BuildBaseFile.groovy b/pipelines/build/BuildBaseFile.groovy new file mode 100644 index 000000000..c27964e45 --- /dev/null +++ b/pipelines/build/BuildBaseFile.groovy @@ -0,0 +1,296 @@ +import groovy.json.JsonSlurper + +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +def toBuildParams(params) { + + List buildParams = [] + + buildParams += [$class: 'LabelParameterValue', name: 'NODE_LABEL', label: params.get("NODE_LABEL")] + + params + .findAll { it.key != 'NODE_LABEL' } + .each { name, value -> buildParams += string(name: name, value: value) } + + return buildParams +} + +def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { + + String buildTag = "build" + + if (configuration.os == "windows" && variant == "openj9") { + buildTag = "buildj9" + } else if (configuration.arch == "s390x" && variant == "openj9") { + buildTag = "(buildj9||build)&&openj9" + } + + def additionalNodeLabels + if (configuration.containsKey("additionalNodeLabels")) { + // hack as jenkins sandbox wont allow instanceof + if ("java.util.LinkedHashMap" == configuration.additionalNodeLabels.getClass().getName()) { + additionalNodeLabels = configuration.additionalNodeLabels.get(variant) + } else { + additionalNodeLabels = configuration.additionalNodeLabels + } + + additionalNodeLabels = "${additionalNodeLabels}&&${buildTag}" + } else { + additionalNodeLabels = buildTag + } + + def buildParams = [ + JAVA_TO_BUILD: javaToBuild, + NODE_LABEL : "${additionalNodeLabels}&&${configuration.os}&&${configuration.arch}", + VARIANT : variant, + ARCHITECTURE : configuration.arch, + TARGET_OS : configuration.os + ] + + if (configuration.containsKey('bootJDK')) buildParams.put("JDK_BOOT_VERSION", configuration.bootJDK) + if (configuration.containsKey('configureArgs')) buildParams.put("CONFIGURE_ARGS", configuration.configureArgs) + if (configuration.containsKey('buildArgs')) buildParams.put("BUILD_ARGS", configuration.buildArgs) + if (configuration.containsKey('additionalFileNameTag')) buildParams.put("ADDITIONAL_FILE_NAME_TAG", configuration.additionalFileNameTag) + + if (releaseTag != null && releaseTag.length() > 0) { + buildParams.put("TAG", releaseTag) + } + + return [ + javaVersion: javaToBuild, + arch : configuration.arch, + os : configuration.os, + variant : variant, + parameters : buildParams, + test : configuration.test, + ] +} + +def getJobConfigurations(javaToBuild, buildConfigurations, String osTarget, String releaseTag) { + def jobConfigurations = [:] + + new JsonSlurper() + .parseText(osTarget) + .each { target -> + if (buildConfigurations.containsKey(target.key)) { + def configuration = buildConfigurations.get(target.key) + target.value.each { variant -> + GString name = "${configuration.os}-${configuration.arch}-${variant}" + if (configuration.containsKey('additionalFileNameTag')) { + name += "-${configuration.additionalFileNameTag}" + } + jobConfigurations[name] = buildConfiguration(javaToBuild, variant, configuration, releaseTag) + } + } + } + + return jobConfigurations +} + +static Integer getJavaVersionNumber(version) { + // version should be something like "jdk8u" + def matcher = (version =~ /(\d+)/) + return Integer.parseInt(matcher[0][1]) +} + +static def determineTestJobName(config, testType) { + + def variant + def number = getJavaVersionNumber(config.javaVersion) + + if (config.variant == "openj9") { + variant = "j9" + } else { + variant = "hs" + } + + def arch = config.arch + if (arch == "x64") { + arch = "x86-64" + } + + def os = config.os + if (os == "mac") { + os = "macos" + } + + return "openjdk${number}_${variant}_${testType}_${arch}_${os}" +} + +static def determineReleaseRepoVersion(javaToBuild) { + def number = getJavaVersionNumber(javaToBuild) + + return "jdk${number}" +} + +def getJobName(displayName, config) { + return "${config.javaVersion}-${displayName}" +} + +def getJobFolder(config) { + return "build-scripts/jobs/${config.javaVersion}" +} + +def createJob(jobName, jobFolder, config) { + + def params = config.parameters.clone() + params.put("JOB_NAME", jobName) + params.put("JOB_FOLDER", jobFolder) + + create = jobDsl targets: "pipelines/build/createJobFromTemplate.groovy", ignoreExisting: true, additionalParameters: params + + return create +} + +def doBuild(String javaToBuild, buildConfigurations, String osTarget, String enableTestsArg, String publishArg, String releaseTag) { + + if (releaseTag == null || releaseTag == "false") { + releaseTag = "" + } + + def jobConfigurations = getJobConfigurations(javaToBuild, buildConfigurations, osTarget, releaseTag) + def jobs = [:] + def buildJobs = [:] + + def enableTests = enableTestsArg == "true" + def publish = publishArg == "true" + + + echo "Java: ${javaToBuild}" + echo "OS: ${osTarget}" + echo "Enable tests: ${enableTests}" + echo "Publish: ${publish}" + echo "ReleaseTag: ${releaseTag}" + + + jobConfigurations.each { configuration -> + jobs[configuration.key] = { + def job + def config = configuration.value + def jobTopName = getJobName(configuration.key, config) + def jobFolder = getJobFolder(config) + def downstreamJob = "${jobFolder}/${jobTopName}"; + + catchError { + stage(configuration.key) { + createJob(jobTopName, jobFolder, config); + + job = build job: downstreamJob, propagate: false, parameters: toBuildParams(config.parameters) + buildJobs[configuration.key] = job + } + + if (enableTests && config.test) { + if (job.getResult() == 'SUCCESS') { + stage("test ${configuration.key}") { + def testStages = [:] + config.test.each { testType -> + testStages["${configuration.key}-${testType}"] = { + stage("test ${configuration.key} ${testType}") { + def jobName = determineTestJobName(config, testType) + catchError { + build job: jobName, + propagate: false, + parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${job.getNumber()}"), + string(name: 'UPSTREAM_JOB_NAME', value: downstreamJob)] + } + } + } + } + parallel testStages + } + } + } + + node('master') { + def downstreamJobName = downstreamJob + def jobWithReleaseArtifact = job + + if (config.os == "windows" || config.os == "mac") { + stage("sign ${configuration.key}") { + filter = "" + certificate = "" + + if (config.os == "windows") { + filter = "**/OpenJDK*_windows_*.zip" + certificate = "C:\\Users\\jenkins\\windows.p12" + + } else if (config.os == "mac") { + filter = "**/OpenJDK*_mac_*.tar.gz" + certificate = "\"Developer ID Application: London Jamocha Community CIC\"" + } + + signJob = build job: "build-scripts/release/sign_build", + propagate: false, + parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${job.getNumber()}"), + string(name: 'UPSTREAM_JOB_NAME', value: downstreamJob), + string(name: 'OPERATING_SYSTEM', value: "${config.os}"), + string(name: 'FILTER', value: "${filter}"), + string(name: 'CERTIFICATE', value: "${certificate}"), + [$class: 'LabelParameterValue', name: 'NODE_LABEL', label: "${config.os}&&build"], + ] + downstreamJobName = "build-scripts/release/sign_build" + jobWithReleaseArtifact = signJob + } + } + + + stage("archive ${configuration.key}") { + if (jobWithReleaseArtifact.getResult() == 'SUCCESS') { + currentBuild.result = 'SUCCESS' + sh "rm target/${config.os}/${config.arch}/${config.variant}/* || true" + + copyArtifacts( + projectName: downstreamJobName, + selector: specific("${jobWithReleaseArtifact.getNumber()}"), + filter: 'workspace/target/*', + fingerprintArtifacts: true, + target: "target/${config.os}/${config.arch}/${config.variant}/", + flatten: true) + + + sh 'for file in $(ls target/*/*/*/*.tar.gz target/*/*/*/*.zip); do sha256sum "$file" > $file.sha256.txt ; done' + archiveArtifacts artifacts: "target/${config.os}/${config.arch}/${config.variant}/*" + } + + } + } + } + } + } + + parallel jobs + + if (publish) { + def release = false + def tag = javaToBuild + if (releaseTag != null && releaseTag.length() > 0) { + release = true + tag = releaseTag + } + + node("master") { + stage("publish") { + build job: 'build-scripts/release/refactor_openjdk_release_tool', + parameters: [string(name: 'RELEASE', value: "${release}"), + string(name: 'TAG', value: tag), + string(name: 'UPSTREAM_JOB_NAME', value: env.JOB_NAME), + string(name: 'UPSTREAM_JOB_NUMBER', value: "${currentBuild.getNumber()}"), + string(name: 'VERSION', value: determineReleaseRepoVersion(javaToBuild))] + } + } + } +} + +return this \ No newline at end of file diff --git a/pipelines/build/createJobFromTemplate.groovy b/pipelines/build/createJobFromTemplate.groovy new file mode 100644 index 000000000..353dee836 --- /dev/null +++ b/pipelines/build/createJobFromTemplate.groovy @@ -0,0 +1,51 @@ +String buildFolder = "$JOB_FOLDER" + +if (!binding.hasVariable('JDK_BOOT_VERSION')) JDK_BOOT_VERSION = "" +if (!binding.hasVariable('CONFIGURE_ARGS')) CONFIGURE_ARGS = "" +if (!binding.hasVariable('BUILD_ARGS')) BUILD_ARGS = "" +if (!binding.hasVariable('ADDITIONAL_FILE_NAME_TAG')) ADDITIONAL_FILE_NAME_TAG = "" + +folder(buildFolder) { + description 'Automatically generated build jobs.' +} + +pipelineJob("$buildFolder/$JOB_NAME") { + description('

THIS IS AN AUTOMATICALLY GENERATED JOB DO NOT MODIFY, IT WILL BE OVERWRITTEN.

This job is defined in createJobFromTemplate.groovy in the openjdk-build repo, if you wish to change it modify that

') + definition { + cpsScm { + scm { + git { + remote { + url('https://github.com/johnoliver/openjdk-build.git') + } + branch('*/overhaul-build-scripts-6') + extensions { + cleanBeforeCheckout() + } + } + } + scriptPath('pipelines/build/openjdk_build_pipeline.groovy') + lightweight(true) + } + } + properties { + copyArtifactPermissionProperty { + projectNames('*') + } + } + logRotator { + numToKeep(5) + } + parameters { + stringParam('TAG', null, "git tag/branch/commit to bulid if not HEAD") + stringParam('NODE_LABEL', "$NODE_LABEL") + stringParam('JAVA_TO_BUILD', "$JAVA_TO_BUILD") + stringParam('JDK_BOOT_VERSION', "${JDK_BOOT_VERSION}") + stringParam('CONFIGURE_ARGS', "$CONFIGURE_ARGS", "Additional arguments to pass to ./configure") + stringParam('BUILD_ARGS', "$BUILD_ARGS", "additional args to call makejdk-any-platform.sh with") + stringParam('ARCHITECTURE', "$ARCHITECTURE") + stringParam('VARIANT', "$VARIANT") + stringParam('TARGET_OS', "$TARGET_OS") + stringParam('ADDITIONAL_FILE_NAME_TAG', "$ADDITIONAL_FILE_NAME_TAG") + } +} \ No newline at end of file diff --git a/pipelines/build/openjdk10_pipeline.groovy b/pipelines/build/openjdk10_pipeline.groovy new file mode 100644 index 000000000..bafc1f422 --- /dev/null +++ b/pipelines/build/openjdk10_pipeline.groovy @@ -0,0 +1,95 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +def buildConfigurations = [ + x64Mac : [ + os : 'mac', + arch : 'x64', + bootJDK : "/Users/jenkins/tools/hudson.model.JDK/JDK9.0.1", + additionalNodeLabels: 'build-macstadium-macos1010-1', + test : ['openjdktest', 'systemtest'] + ], + x64Linux : [ + os : 'linux', + arch : 'x64', + additionalNodeLabels: 'centos6', + test : ['openjdktest', 'systemtest', 'externaltest'] + ], + + // Currently we have to be quite specific about which windows to use as not all of them have freetype installed + x64Windows: [ + os : 'windows', + arch : 'x64', + additionalNodeLabels: 'win2012', + test : ['openjdktest'] + ], + + ppc64Aix : [ + os : 'aix', + arch : 'ppc64', + test : false + ], + + s390xLinux : [ + os : 'linux', + arch : 's390x', + additionalNodeLabels: 'ubuntu', + test : ['openjdktest', 'systemtest'] + ], + + ppc64leLinux : [ + os : 'linux', + arch : 'ppc64le', + additionalNodeLabels: 'centos7', + test : ['openjdktest', 'systemtest'] + ], + + arm32Linux : [ + os : 'linux', + arch : 'arm', + test : ['openjdktest'] + ], + + aarch64Linux : [ + os : 'linux', + arch : 'aarch64', + additionalNodeLabels: 'centos7', + test : ['openjdktest'] + ], + + /* + "x86-32Windows" : [ + os : 'windows', + arch : 'x86-32', + additionalNodeLabels: 'win2012&&x86-32', + test : false + ], + */ + "linuxXL" : [ + os : 'linux', + additionalNodeLabels: 'centos6', + arch : 'x64', + test : false, + additionalFileNameTag: "linuxXL", + configureArgs : '--with-noncompressedrefs' + ], +] + +def javaToBuild = "jdk10u" + +node ("master") { + checkout scm + def buildFile = load "${WORKSPACE}/pipelines/build/BuildBaseFile.groovy" + buildFile.doBuild(javaToBuild, buildConfigurations, osTarget, enableTests, publish, releaseTag) +} diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy new file mode 100644 index 000000000..7d8929c82 --- /dev/null +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -0,0 +1,93 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +def buildConfigurations = [ + x64Mac : [ + os : 'mac', + arch : 'x64', + additionalNodeLabels: 'build-macstadium-macos1010-1', + test : ['openjdktest', 'systemtest'] + ], + x64Linux : [ + os : 'linux', + arch : 'x64', + additionalNodeLabels: 'centos6', + test : ['openjdktest', 'systemtest', 'externaltest'] + ], + + // Currently we have to be quite specific about which windows to use as not all of them have freetype installed + x64Windows: [ + os : 'windows', + arch : 'x64', + additionalNodeLabels: 'win2012', + test : ['openjdktest'] + ], + + ppc64Aix : [ + os : 'aix', + arch : 'ppc64', + test : false + ], + + s390xLinux : [ + os : 'linux', + arch : 's390x', + test : ['openjdktest', 'systemtest'] + ], + + ppc64leLinux : [ + os : 'linux', + arch : 'ppc64le', + additionalNodeLabels: 'centos7', + test : ['openjdktest', 'systemtest'] + ], + + arm32Linux : [ + os : 'linux', + arch : 'arm32', + test : ['openjdktest'] + ], + + aarch64Linux : [ + os : 'linux', + arch : 'aarch64', + additionalNodeLabels: 'centos7', + test : ['openjdktest'] + ], + + /* + "x86-32Windows" : [ + os : 'windows', + arch : 'x86-32', + additionalNodeLabels: 'win2012&&x86-32', + test : false + ], + */ + "linuxXL" : [ + os : 'linux', + additionalNodeLabels: 'centos6', + arch : 'x64', + test : false, + additionalFileNameTag: "linuxXL", + configureArgs : '--with-noncompressedrefs' + ], +] + +def javaToBuild = "jdk11" + +node ("master") { + checkout scm + def buildFile = load "${WORKSPACE}/pipelines/build/BuildBaseFile.groovy" + buildFile.doBuild(javaToBuild, buildConfigurations, osTarget, enableTests, publish, releaseTag) +} diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy new file mode 100644 index 000000000..f13a8b55a --- /dev/null +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -0,0 +1,85 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +def buildConfigurations = [ + x64Mac : [ + os : 'mac', + arch : 'x64', + additionalNodeLabels: 'build-macstadium-macos1010-1', + test : ['openjdktest', 'systemtest'] + ], + + x64Linux : [ + os : 'linux', + arch : 'x64', + additionalNodeLabels: [ + hotspot: 'centos6', + openj9: 'build-joyent-centos69-x64-1' + ], + test : ['openjdktest', 'systemtest', 'perftest', 'externaltest'] + ], + + // Currently we have to be quite specific about which windows to use as not all of them have freetype installed + x64Windows: [ + os : 'windows', + arch : 'x64', + additionalNodeLabels: [ + hotspot: 'win2008', + openj9: 'win2012' + ], + test : ['openjdktest'] + ], + + ppc64Aix : [ + os : 'aix', + arch : 'ppc64', + test : false + ], + + s390xLinux : [ + os : 'linux', + arch : 's390x', + test : ['openjdktest', 'systemtest'] + ], + + ppc64leLinux : [ + os : 'linux', + arch : 'ppc64le', + test : ['openjdktest', 'systemtest'] + ], + + arm32Linux : [ + os : 'linux', + arch : 'arm', + test : ['openjdktest'] + ], + + "linuxXL" : [ + os : 'linux', + additionalNodeLabels: 'centos6', + arch : 'x64', + test : false, + additionalFileNameTag: "linuxXL", + configureArgs : '--with-noncompressedrefs' + ], +] + +def javaToBuild = "jdk8u" + +node ("master") { + checkout scm + def buildFile = load "${WORKSPACE}/pipelines/build/BuildBaseFile.groovy" + buildFile.doBuild(javaToBuild, buildConfigurations, osTarget, enableTests, publish, releaseTag) +} + diff --git a/pipelines/build/openjdk9_pipeline.groovy b/pipelines/build/openjdk9_pipeline.groovy new file mode 100644 index 000000000..5066c2812 --- /dev/null +++ b/pipelines/build/openjdk9_pipeline.groovy @@ -0,0 +1,87 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +def buildConfigurations = [ + x64Mac : [ + os : 'mac', + arch : 'x64', + additionalNodeLabels: 'build-macstadium-macos1010-1', + test : ['openjdktest', 'systemtest'] + ], + + x64Linux : [ + os : 'linux', + arch : 'x64', + additionalNodeLabels: 'centos6', + test : ['openjdktest', 'systemtest', 'externaltest'] + ], + + // Currently we have to be quite specific about which windows to use as not all of them have freetype installed + x64Windows: [ + os : 'windows', + arch : 'x64', + additionalNodeLabels: 'win2012', + test : ['openjdktest'] + ], + + ppc64Aix : [ + os : 'aix', + arch : 'ppc64', + test : false + ], + + s390xLinux : [ + os : 'linux', + arch : 's390x', + additionalNodeLabels: 'rhel7', + test : ['openjdktest', 'systemtest'] + ], + + ppc64leLinux : [ + os : 'linux', + arch : 'ppc64le', + additionalNodeLabels: 'centos7', + test : ['openjdktest', 'systemtest'] + ], + + arm32Linux : [ + os : 'linux', + arch : 'arm', + test : ['openjdktest'] + ], + + aarch64Linux : [ + os : 'linux', + arch : 'aarch64', + additionalNodeLabels: 'centos7', + test : ['openjdktest'] + ], + + "linuxXL" : [ + os : 'linux', + additionalNodeLabels: 'centos6', + arch : 'x64', + test : false, + additionalFileNameTag: "linuxXL", + configureArgs : '--with-noncompressedrefs' + ], +] + +def javaToBuild = "jdk9u" + +node ("master") { + checkout scm + def buildFile = load "${WORKSPACE}/pipelines/build/BuildBaseFile.groovy" + buildFile.doBuild(javaToBuild, buildConfigurations, osTarget, enableTests, publish, releaseTag) +} diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy new file mode 100644 index 000000000..4024eb0c6 --- /dev/null +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -0,0 +1,28 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +node(NODE_LABEL) { + checkout scm + + success = false + try { + sh "./build-farm/make-adopt-build-farm.sh" + archiveArtifacts artifacts: "workspace/target/*" + success = true + } catch (Exception e) { + success = false + currentBuild.result = 'FAILURE' + } +} + diff --git a/sbin/build.sh b/sbin/build.sh index 738a6503e..5d82b1955 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -1,5 +1,6 @@ #!/bin/bash -# + +################################################################################ # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at @@ -11,91 +12,79 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -# +################################################################################ -# Script to download any additional packages for building OpenJDK -# before calling ./configure (using JDK 7 as the base) +################################################################################ +# +# Build OpenJDK - can be called directly but is typically called by +# docker-build.sh or native-build.sh. +# +# See bottom of the script for the call order and each function for further +# details. +# +# Calls 'configure' then 'make' in order to build OpenJDK +# +################################################################################ +set -eu SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -# shellcheck source=sbin/common-functions.sh -source "$SCRIPT_DIR/common-functions.sh" - -WORKING_DIR="" -TARGET_DIR="" -OPENJDK_REPO_NAME="" -JVM_VARIANT="${JVM_VARIANT:-server}" -OPENJDK_UPDATE_VERSION="" -OPENJDK_BUILD_NUMBER="" -OPENJDK_REPO_TAG="" -JRE_TARGET_PATH="" -TRIMMED_TAG="" -USER_SUPPLIED_CONFIGURE_ARGS="" - -while [[ $# -gt 0 ]] && [[ ."$1" = .-* ]] ; do - opt="$1"; - shift; - case "$opt" in - "--" ) break 2;; - - "--source" | "-s" ) - WORKING_DIR="$1"; shift;; - - "--destination" | "-d" ) - TARGET_DIR="$1"; shift;; - - "--repository" | "-r" ) - OPENJDK_REPO_NAME="$1"; shift;; - "--variant" | "-jv" ) - JVM_VARIANT="$1"; shift;; +# shellcheck source=sbin/prepareWorkspace.sh +source "$SCRIPT_DIR/prepareWorkspace.sh" - "--update-version" | "-uv" ) - OPENJDK_UPDATE_VERSION="$1"; shift;; +# shellcheck source=sbin/common/config_init.sh +source "$SCRIPT_DIR/common/config_init.sh" - "--build-number" | "-bn" ) - OPENJDK_BUILD_NUMBER="$1"; shift;; +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/common/constants.sh" - "--repository-tag" | "-rt" ) - OPENJDK_REPO_TAG="$1"; shift;; +# shellcheck source=sbin/common/common.sh +source "$SCRIPT_DIR/common/common.sh" - "--configure-args" | "-ca" ) - USER_SUPPLIED_CONFIGURE_ARGS="$1"; shift;; +export OPENJDK_REPO_TAG +export OPENJDK_DIR +export JRE_TARGET_PATH +export CONFIGURE_ARGS="" +export MAKE_TEST_IMAGE="" +export GIT_CLONE_ARGUMENTS=() - *) echo >&2 "${error}Invalid build.sh option: ${opt}${normal}"; exit 1;; - esac -done +# Parse the CL arguments, defers to the shared function in common-functions.sh +function parseArguments() { + parseConfigurationArguments "$@" -OPENJDK_DIR=$WORKING_DIR/$OPENJDK_REPO_NAME + OPENJDK_DIR="${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_VERSION}" ]; then + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]="true"; + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]="true"; + fi -RUN_JTREG_TESTS_ONLY="" - + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_VERSION}" ]; then + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]="false"; + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]="true"; + fi -if [ "$JVM_VARIANT" == "--run-jtreg-tests-only" ]; then - RUN_JTREG_TESTS_ONLY="--run-jtreg-tests-only" - JVM_VARIANT="server" -fi + echo "JDK Image folder name: ${BUILD_CONFIG[JDK_PATH]}" + echo "JRE Image folder name: ${BUILD_CONFIG[JRE_PATH]}" + echo "[debug] COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG=${BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]}" + echo "[debug] COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG=${BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]}" -echo "JDK Image folder name: ${JDK_PATH}" -echo "JRE Image folder name: ${JRE_PATH}" -echo "[debug] COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG=${COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG}" -echo "[debug] COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG=${COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG}" + BUILD_CONFIG[MAKE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[MAKE_ARGS_FOR_ANY_PLATFORM]:-"images"} -MAKE_COMMAND_NAME=${MAKE_COMMAND_NAME:-"make"} -MAKE_ARGS_FOR_ANY_PLATFORM=${MAKE_ARGS_FOR_ANY_PLATFORM:-"images"} -# Defaults to not building this target, for Java 9+ we set this to test-image in order to build the native test libraries -MAKE_TEST_IMAGE="" -CONFIGURE_ARGS_FOR_ANY_PLATFORM=${CONFIGURE_ARGS_FOR_ANY_PLATFORM:-""} + BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]:-""} +} +# Add an argument to the configure call addConfigureArg() { - #Only add an arg if it is not overridden by a user-specified arg. - if [[ ${CONFIGURE_ARGS_FOR_ANY_PLATFORM} != *"$1"* ]] && [[ ${USER_SUPPLIED_CONFIGURE_ARGS} != *"$1"* ]]; then + # Only add an arg if it is not overridden by a user-specified arg. + if [[ ${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]} != *"$1"* ]] && [[ ${BUILD_CONFIG[USER_SUPPLIED_CONFIGURE_ARGS]} != *"$1"* ]]; then CONFIGURE_ARGS="${CONFIGURE_ARGS} ${1}${2}" fi } +# Add an argument to the configure call (if it's not empty) addConfigureArgIfValueIsNotEmpty() { #Only try to add an arg if the second argument is not empty. @@ -104,62 +93,70 @@ addConfigureArgIfValueIsNotEmpty() fi } -sourceFileWithColourCodes() +# Configure the boot JDK +configuringBootJDKConfigureParameter() { - # shellcheck disable=SC1090 - # shellcheck disable=SC1091 - source "$SCRIPT_DIR"/colour-codes.sh -} -checkIfDockerIsUsedForBuildingOrNot() -{ - # If on docker - - if [[ -f /.dockerenv ]] ; then - echo "Detected we're in docker" - WORKING_DIR=/openjdk/build - TARGET_DIR=/openjdk/target/ - OPENJDK_REPO_NAME=/openjdk - OPENJDK_DIR="$WORKING_DIR/$OPENJDK_REPO_NAME" - USE_DOCKER=true - fi + if [ -z "${BUILD_CONFIG[JDK_BOOT_DIR]}" ] ; then + echo "Searching for JDK_BOOT_DIR" + + # shellcheck disable=SC2046 + if [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" == "darwin" ]]; then + BUILD_CONFIG[JDK_BOOT_DIR]=$(dirname $(dirname $(readlink $(which javac)))) + else + BUILD_CONFIG[JDK_BOOT_DIR]=$(dirname $(dirname $(readlink -f $(which javac)))) + fi - # E.g. /openjdk/build if you're building in a Docker container - # otherwise ensure it's a writable area e.g. /home/youruser/myopenjdkarea - - if [ -z "$WORKING_DIR" ] || [ -z "$TARGET_DIR" ] ; then - echo "build.sh is called by makejdk.sh and requires two parameters" - echo "Are you sure you want to call it directly?" - echo "Usage: bash ./${0} " - echo "Note that you must have the OpenJDK source before using this script!" - echo "This script will try to move ./openjdk to the source directory for you, " - echo "and this will be your working area where all required files will be downloaded to." - echo "You can override the JDK boot directory by setting the environment variable JDK_BOOT_DIR" - exit; + echo "Guessing JDK_BOOT_DIR: ${BUILD_CONFIG[JDK_BOOT_DIR]}" + echo "If this is incorrect explicitly configure JDK_BOOT_DIR" + else + echo "Overriding JDK_BOOT_DIR, set to ${BUILD_CONFIG[JDK_BOOT_DIR]}" fi + + echo "Boot dir set to ${BUILD_CONFIG[JDK_BOOT_DIR]}" + + addConfigureArgIfValueIsNotEmpty "--with-boot-jdk=" "${BUILD_CONFIG[JDK_BOOT_DIR]}" } -createWorkingDirectory() +# Get the OpenJDK update version and build version +getOpenJDKUpdateAndBuildVersion() { - echo "Making the working directory to store source files and extensions: ${WORKING_DIR}" + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" - mkdir -p $WORKING_DIR + if [ -d "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" ]; then - cd $WORKING_DIR || exit -} + # It does exist and it's a repo other than the AdoptOpenJDK one + cd "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || return -configuringBootJDKConfigureParameter() -{ - if [ -z "$JDK_BOOT_DIR" ] ; then - echo "JDK_BOOT_DIR is ${JDK_BOOT_DIR}" - JDK_BOOT_DIR=/usr/lib/java-1.7.0 - else - echo "Overriding JDK_BOOT_DIR, set to ${JDK_BOOT_DIR}" - fi + if [ -f ".git/shallow.lock" ] + then + echo "Detected lock file, assuming this is an error, removing" + rm ".git/shallow.lock" + fi + + # shellcheck disable=SC2154 + echo "Pulling latest tags and getting the latest update version using git fetch -q --tags ${BUILD_CONFIG[SHALLOW_CLONE_OPTION]}" + # shellcheck disable=SC2154 + echo "NOTE: This can take quite some time! Please be patient" + git fetch -q --tags ${BUILD_CONFIG[SHALLOW_CLONE_OPTION]} + OPENJDK_REPO_TAG=${BUILD_CONFIG[TAG]:-$(getFirstTagFromOpenJDKGitRepo)} + if [[ "${OPENJDK_REPO_TAG}" == "" ]] ; then + # shellcheck disable=SC2154 + echo "Unable to detect git tag, exiting..." + exit 1 + else + echo "OpenJDK repo tag is $OPENJDK_REPO_TAG" + fi - echo "Boot dir set to ${JDK_BOOT_DIR}" + local openjdk_update_version; + openjdk_update_version=$(echo "${OPENJDK_REPO_TAG}" | cut -d'u' -f 2 | cut -d'-' -f 1) - addConfigureArgIfValueIsNotEmpty "--with-boot-jdk=" "${JDK_BOOT_DIR}" + # TODO dont modify config in build script + BUILD_CONFIG[OPENJDK_BUILD_NUMBER]=$(echo "${OPENJDK_REPO_TAG}" | cut -d'b' -f 2 | cut -d'-' -f 1) + echo "Version: ${openjdk_update_version} ${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]}" + fi + + cd "${BUILD_CONFIG[WORKSPACE_DIR]}" } # Ensure that we produce builds with versions strings something like: @@ -169,15 +166,15 @@ configuringBootJDKConfigureParameter() # OpenJDK 64-Bit Server VM (build 25.71-b00, mixed mode) configuringVersionStringParameter() { - if [ "$OPENJDK_CORE_VERSION" == "jdk8" ]; then + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ]; then # Replace the default 'internal' with our own milestone string addConfigureArg "--with-milestone=" "adoptopenjdk" # Set the update version (e.g. 131), this gets passed in from the calling script - addConfigureArgIfValueIsNotEmpty "--with-update-version=" "${OPENJDK_UPDATE_VERSION}" + addConfigureArgIfValueIsNotEmpty "--with-update-version=" "${BUILD_CONFIG[OPENJDK_UPDATE_VERSION]}" # Set the build number (e.g. b04), this gets passed in from the calling script - addConfigureArgIfValueIsNotEmpty "--with-build-number=" "${OPENJDK_BUILD_NUMBER}" + addConfigureArgIfValueIsNotEmpty "--with-build-number=" "${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]}" else if [ -z "$OPENJDK_REPO_TAG" ]; then cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || echo Cannot change to "${WORKING_DIR}/${OPENJDK_REPO_NAME}" @@ -187,211 +184,214 @@ configuringVersionStringParameter() # > JDK 8 addConfigureArg "--with-version-pre=" "adoptopenjdk" - TRIMMED_TAG=$(echo "$OPENJDK_REPO_TAG" | cut -f2 -d"-") + TRIMMED_TAG=$(echo "$OPENJDK_REPO_TAG" | cut -f2 -d"-" ) addConfigureArg "--with-version-string=" "${TRIMMED_TAG}" fi echo "Completed configuring the version string parameter, config args are now: ${CONFIGURE_ARGS}" } +# Construct all of the 'configure' parameters buildingTheRestOfTheConfigParameters() { if [ ! -z "$(which ccache)" ]; then addConfigureArg "--enable-ccache" "" fi - addConfigureArgIfValueIsNotEmpty "--with-jvm-variants=" "${JVM_VARIANT}" - addConfigureArgIfValueIsNotEmpty "--with-cacerts-file=" "${WORKING_DIR}/cacerts_area/security/cacerts" - addConfigureArg "--with-alsa=" "${WORKING_DIR}/alsa-lib-${ALSA_LIB_VERSION}" + addConfigureArgIfValueIsNotEmpty "--with-jvm-variants=" "${BUILD_CONFIG[JVM_VARIANT]}" + addConfigureArgIfValueIsNotEmpty "--with-cacerts-file=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/cacerts_area/security/cacerts" + addConfigureArg "--with-alsa=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedalsa" # Point-in-time dependency for openj9 only - if [[ "${BUILD_VARIANT}" == "openj9" ]] ; then - addConfigureArg "--with-freemarker-jar=" "${WORKING_DIR}/freemarker-${FREEMARKER_LIB_VERSION}/lib/freemarker.jar" + if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]] ; then + addConfigureArg "--with-freemarker-jar=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/freemarker-${FREEMARKER_LIB_VERSION}/freemarker.jar" fi - if [[ -z "${FREETYPE}" ]] ; then - case $OPENJDK_CORE_VERSION in - jdk8*|jdk9*|jdk10*) FREETYPE_DIRECTORY=${FREETYPE_DIRECTORY:-"${WORKING_DIR}/${OPENJDK_REPO_NAME}/installedfreetype"} ;; - *) FREETYPE_DIRECTORY=${FREETYPE_DIRECTORY:-bundled} ;; + if [[ "${BUILD_CONFIG[FREETYPE]}" == "true" ]] ; then + local freetypeDir=BUILD_CONFIG[FREETYPE_DIRECTORY] + case "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" in + jdk8*|jdk9*|jdk10*) freetypeDir=${BUILD_CONFIG[FREETYPE_DIRECTORY]:-"${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedfreetype"} ;; + *) freetypeDir=${BUILD_CONFIG[FREETYPE_DIRECTORY]:-bundled} ;; esac - addConfigureArg "--with-freetype=" "$FREETYPE_DIRECTORY" + addConfigureArg "--with-freetype=" "${freetypeDir}" fi - # These will have been installed by the package manager (see our Dockerfile) addConfigureArg "--with-x=" "/usr/include/X11" - # We don't want any extra debug symbols - ensure it's set to release, - # other options include fastdebug and slowdebug - addConfigureArg "--with-debug-level=" "release" + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] ; then + # We don't want any extra debug symbols - ensure it's set to release, + # other options include fastdebug and slowdebug + addConfigureArg "--with-debug-level=" "release" + addConfigureArg "--disable-zip-debug-info" "" + addConfigureArg "--disable-debug-symbols" "" + else + addConfigureArg "--with-debug-level=" "release" + addConfigureArg "--with-native-debug-symbols=" "none" + fi } +# Configure the command parameters configureCommandParameters() { configuringVersionStringParameter + configuringBootJDKConfigureParameter if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then echo "Windows or Windows-like environment detected, skipping configuring environment for custom Boot JDK and other 'configure' settings." else echo "Building up the configure command..." - configuringBootJDKConfigureParameter buildingTheRestOfTheConfigParameters fi - #Now we add any configure arguments the user has specified on the command line. - CONFIGURE_ARGS="${CONFIGURE_ARGS} ${USER_SUPPLIED_CONFIGURE_ARGS}" + # Now we add any configure arguments the user has specified on the command line. + CONFIGURE_ARGS="${CONFIGURE_ARGS} ${BUILD_CONFIG[USER_SUPPLIED_CONFIGURE_ARGS]}" echo "Completed configuring the version string parameter, config args are now: ${CONFIGURE_ARGS}" } -stepIntoTheWorkingDirectory() -{ - # Make sure we're in the source directory for OpenJDK now - cd "$WORKING_DIR/$OPENJDK_REPO_NAME" || exit +# Make sure we're in the source directory for OpenJDK now +stepIntoTheWorkingDirectory() { + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || exit echo "Should have the source, I'm at $PWD" } -runTheOpenJDKConfigureCommandAndUseThePrebuiltConfigParams() -{ +buildTemplatedFile() { echo "Configuring command and using the pre-built config params..." - cd "$OPENJDK_DIR" || exit + stepIntoTheWorkingDirectory echo "Currently at '${PWD}'" - CONFIGURED_OPENJDK_ALREADY=$(find . -name "config.status") - - if [[ ! -z "$CONFIGURED_OPENJDK_ALREADY" ]] ; then - echo "Not reconfiguring due to the presence of config.status in ${WORKING_DIR}" - else - CONFIGURE_ARGS="${CONFIGURE_ARGS} ${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" - - echo "Running ./configure with arguments '${CONFIGURE_ARGS}'" - # Depends upon the configure command being split for multiple args. Don't quote it. - # shellcheck disable=SC2086 - bash ./configure ${CONFIGURE_ARGS} - - # shellcheck disable=SC2181 - if [ $? -ne 0 ]; then - echo "${error}" - echo "Failed to configure the JDK, exiting" - echo "Did you set the JDK boot directory correctly? Override by exporting JDK_BOOT_DIR" - echo "For example, on RHEL you would do export JDK_BOOT_DIR=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.131-2.6.9.0.el7_3.x86_64" - echo "Current JDK_BOOT_DIR value: ${JDK_BOOT_DIR}" - exit; - else - echo "${good}Configured the JDK" - fi - echo "${normal}" - fi -} - -buildOpenJDK() -{ - cd "$OPENJDK_DIR" || exit - - #If the user has specified nobuild, we do everything short of building the JDK, and then we stop. - if [ "${RUN_JTREG_TESTS_ONLY}" == "--run-jtreg-tests-only" ]; then - rm -rf cacerts_area - echo "Nobuild option was set. Prep complete. Java not built." - exit 0 - fi + FULL_CONFIGURE="bash ./configure ${CONFIGURE_ARGS} ${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]}" + echo "Running ./configure with arguments '${FULL_CONFIGURE}'" # If it's Java 9+ then we also make test-image to build the native test libraries JDK_PREFIX="jdk" - JDK_VERSION_NUMBER="${OPENJDK_CORE_VERSION#$JDK_PREFIX}" + JDK_VERSION_NUMBER="${BUILD_CONFIG[OPENJDK_CORE_VERSION]#$JDK_PREFIX}" if [ "$JDK_VERSION_NUMBER" -gt 8 ]; then MAKE_TEST_IMAGE=" test-image" # the added white space is deliberate as it's the last arg fi - FULL_MAKE_COMMAND="${MAKE_COMMAND_NAME} ${MAKE_ARGS_FOR_ANY_PLATFORM} ${MAKE_TEST_IMAGE}" - echo "Building the JDK: calling '${FULL_MAKE_COMMAND}'" + FULL_MAKE_COMMAND="${BUILD_CONFIG[MAKE_COMMAND_NAME]} ${BUILD_CONFIG[MAKE_ARGS_FOR_ANY_PLATFORM]} ${MAKE_TEST_IMAGE}" + + # shellcheck disable=SC2002 + cat "$SCRIPT_DIR/build.template" | \ + sed -e "s|{configureArg}|${FULL_CONFIGURE}|" \ + -e "s|{makeCommandArg}|${FULL_MAKE_COMMAND}|" > "${BUILD_CONFIG[WORKSPACE_DIR]}/config/configure-and-build.sh" +} + +executeTemplatedFile() { + stepIntoTheWorkingDirectory + + echo "Currently at '${PWD}'" + bash "${BUILD_CONFIG[WORKSPACE_DIR]}/config/configure-and-build.sh" + exitCode=$? - if ! ${FULL_MAKE_COMMAND}; then - echo "${error}Failed to make the JDK, exiting" + if [ "${exitCode}" -eq 1 ]; then + echo "Failed to make the JDK, exiting" exit 1; - else - echo "${good}Built the JDK!" + elif [ "${exitCode}" -eq 2 ]; then + echo "Failed to configure the JDK, exiting" + echo "Did you set the JDK boot directory correctly? Override by exporting JDK_BOOT_DIR" + echo "For example, on RHEL you would do export JDK_BOOT_DIR=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.131-2.6.9.0.el7_3.x86_64" + echo "Current JDK_BOOT_DIR value: ${BUILD_CONFIG[JDK_BOOT_DIR]}" + exit 2; fi - echo "${normal}" + } +# Print the version string so we know what we've produced printJavaVersionString() { # shellcheck disable=SC2086 - PRODUCT_HOME=$(ls -d $OPENJDK_DIR/build/*/images/${JDK_PATH}) + PRODUCT_HOME=$(ls -d ${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/build/*/images/${BUILD_CONFIG[JDK_PATH]}) if [[ -d "$PRODUCT_HOME" ]]; then - echo "${good}'$PRODUCT_HOME' found${normal}" - # shellcheck disable=SC2154 - echo "${info}" + echo "'$PRODUCT_HOME' found" if ! "$PRODUCT_HOME"/bin/java -version; then - echo "${error} Error executing 'java' does not exist in '$PRODUCT_HOME'.${normal}" + echo " Error executing 'java' does not exist in '$PRODUCT_HOME'." exit -1 fi - echo "${normal}" - echo "" else - echo "${error}'$PRODUCT_HOME' does not exist, build might have not been successful or not produced the expected JDK image at this location.${normal}" + echo "'$PRODUCT_HOME' does not exist, build might have not been successful or not produced the expected JDK image at this location." exit -1 fi } +# Clean up removingUnnecessaryFiles() { echo "Removing unnecessary files now..." - echo "Fetching the first tag from the OpenJDK git repo..." if [ -z "$OPENJDK_REPO_TAG" ]; then + echo "Fetching the first tag from the OpenJDK git repo..." + echo "Dir=${PWD}" OPENJDK_REPO_TAG=$(getFirstTagFromOpenJDKGitRepo) fi - if [ "$USE_DOCKER" != "true" ] ; then - rm -rf cacerts_area - fi - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || return + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || return cd build/*/images || return echo "Currently at '${PWD}'" - echo "moving ${JDK_PATH} to ${OPENJDK_REPO_TAG}" + echo "moving ${BUILD_CONFIG[JDK_PATH]} to ${OPENJDK_REPO_TAG}" rm -rf "${OPENJDK_REPO_TAG}" || true - mv "$JDK_PATH" "${OPENJDK_REPO_TAG}" + mv "${BUILD_CONFIG[JDK_PATH]}" "${OPENJDK_REPO_TAG}" + + if [ -d "${BUILD_CONFIG[JRE_PATH]}" ] + then + JRE_TARGET_PATH="${OPENJDK_REPO_TAG}-jre" + [ "${JRE_TARGET_PATH}" == "${OPENJDK_REPO_TAG}" ] && JRE_TARGET_PATH="${OPENJDK_REPO_TAG}.jre" + echo "moving ${BUILD_CONFIG[JRE_PATH]} to ${JRE_TARGET_PATH}" + rm -rf "${JRE_TARGET_PATH}" || true + mv "${BUILD_CONFIG[JRE_PATH]}" "${JRE_TARGET_PATH}" + + rm -rf "${JRE_TARGET_PATH}"/demo/applets || true + rm -rf "${JRE_TARGET_PATH}"/demo/jfc/Font2DTest || true + rm -rf "${JRE_TARGET_PATH}"/demo/jfc/SwingApplet || true + fi - JRE_TARGET_PATH="${OPENJDK_REPO_TAG/jdk/jre}" - [ "${JRE_TARGET_PATH}" == "${OPENJDK_REPO_TAG}" ] && JRE_TARGET_PATH="${OPENJDK_REPO_TAG}.jre" - echo "moving ${JRE_PATH} to ${JRE_TARGET_PATH}" - rm -rf "${JRE_TARGET_PATH}" || true - mv "$JRE_PATH" "${JRE_TARGET_PATH}" # Remove files we don't need rm -rf "${OPENJDK_REPO_TAG}"/demo/applets || true rm -rf "${OPENJDK_REPO_TAG}"/demo/jfc/Font2DTest || true rm -rf "${OPENJDK_REPO_TAG}"/demo/jfc/SwingApplet || true - rm -rf "${JRE_TARGET_PATH}"/demo/applets || true - rm -rf "${JRE_TARGET_PATH}"/demo/jfc/Font2DTest || true - rm -rf "${JRE_TARGET_PATH}"/demo/jfc/SwingApplet || true + find . -name "*.diz" -type f -delete || true - echo "Finished removing unnecessary files from ${OPENJDK_REPO_TAG} and ${JRE_TARGET_PATH}" + echo "Finished removing unnecessary files from ${OPENJDK_REPO_TAG}" } +# If on a Mac, mac a copy of the font lib as required makeACopyOfLibFreeFontForMacOSX() { IMAGE_DIRECTORY=$1 PERFORM_COPYING=$2 - if [[ "$OS_KERNEL_NAME" == "darwin" ]]; then + if [ ! -d "${IMAGE_DIRECTORY}" ]; then + echo "Could not find dir: ${IMAGE_DIRECTORY}" + return + fi + + if [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" == "darwin" ]]; then echo "PERFORM_COPYING=${PERFORM_COPYING}" if [ "${PERFORM_COPYING}" == "false" ]; then - echo "${info} Skipping copying of the free font library to ${IMAGE_DIRECTORY}, does not apply for this version of the JDK. ${normal}" + echo " Skipping copying of the free font library to ${IMAGE_DIRECTORY}, does not apply for this version of the JDK. " return fi - echo "${info} Performing copying of the free font library to ${IMAGE_DIRECTORY}, applicable for this version of the JDK. ${normal}" + echo " Performing copying of the free font library to ${IMAGE_DIRECTORY}, applicable for this version of the JDK. " + SOURCE_LIB_NAME="${IMAGE_DIRECTORY}/lib/libfreetype.dylib.6" + if [ ! -f "${SOURCE_LIB_NAME}" ]; then - echo "${error}[Error] ${SOURCE_LIB_NAME} does not exist in the ${IMAGE_DIRECTORY} folder, please check if this is the right folder to refer to, aborting copy process...${normal}" + SOURCE_LIB_NAME="${IMAGE_DIRECTORY}/lib/libfreetype.dylib" + fi + + if [ ! -f "${SOURCE_LIB_NAME}" ]; then + echo "[Error] ${SOURCE_LIB_NAME} does not exist in the ${IMAGE_DIRECTORY} folder, please check if this is the right folder to refer to, aborting copy process..." exit -1 fi + TARGET_LIB_NAME="${IMAGE_DIRECTORY}/lib/libfreetype.6.dylib" INVOKED_BY_FONT_MANAGER="${IMAGE_DIRECTORY}/lib/libfontmanager.dylib" @@ -400,122 +400,97 @@ makeACopyOfLibFreeFontForMacOSX() { echo "Copying ${SOURCE_LIB_NAME} to ${TARGET_LIB_NAME}" echo " *** Workaround to fix the MacOSX issue where invocation to ${INVOKED_BY_FONT_MANAGER} fails to find ${TARGET_LIB_NAME} ***" - set -x cp "${SOURCE_LIB_NAME}" "${TARGET_LIB_NAME}" if [ -f "${INVOKED_BY_FONT_MANAGER}" ]; then otool -L "${INVOKED_BY_FONT_MANAGER}" else # shellcheck disable=SC2154 - echo "${warning}[Warning] ${INVOKED_BY_FONT_MANAGER} does not exist in the ${IMAGE_DIRECTORY} folder, please check if this is the right folder to refer to, this may cause runtime issues, please beware...${normal}" + echo "[Warning] ${INVOKED_BY_FONT_MANAGER} does not exist in the ${IMAGE_DIRECTORY} folder, please check if this is the right folder to refer to, this may cause runtime issues, please beware..." fi otool -L "${TARGET_LIB_NAME}" - set +x echo "Finished copying ${SOURCE_LIB_NAME} to ${TARGET_LIB_NAME}" fi } -signRelease() + +# Get the first tag from the git repo +getFirstTagFromOpenJDKGitRepo() { - if [ "$SIGN" ]; then - case "$OSTYPE" in - "cygwin") - echo "Signing Windows release" - signToolPath=${signToolPath:-"/cygdrive/c/Program Files/Microsoft SDKs/Windows/v7.1/Bin/signtool.exe"} - # Sign .exe files - FILES=$(find "${OPENJDK_REPO_TAG}" "${JRE_TARGET_PATH}" -type f -name '*.exe') - echo "$FILES" | while read -r f; do "$signToolPath" sign /f "$CERTIFICATE" /p "$SIGN_PASSWORD" /fd SHA256 /t http://timestamp.verisign.com/scripts/timstamp.dll "$f"; done - # Sign .dll files - FILES=$(find "${OPENJDK_REPO_TAG}" "${JRE_TARGET_PATH}" -type f -name '*.dll') - echo "$FILES" | while read -r f; do "$signToolPath" sign /f "$CERTIFICATE" /p "$SIGN_PASSWORD" /fd SHA256 /t http://timestamp.verisign.com/scripts/timstamp.dll "$f"; done - ;; - "darwin"*) - echo "Signing OSX release" - # Login to KeyChain - # shellcheck disable=SC2046 - # shellcheck disable=SC2006 - security unlock-keychain -p `cat ~/.password` - # Sign all files with the executable permission bit set. - FILES=$(find "${OPENJDK_REPO_TAG}" "${JRE_TARGET_PATH}" ! -perm +111 -type f || find "${OPENJDK_REPO_TAG}" -perm /111 -type f) - echo "$FILES" | while read -r f; do codesign -s "$CERTIFICATE" "$f"; done - ;; - *) - echo "Skipping code signing as it's not supported on $OSTYPE" - ;; - esac - fi + git fetch --tags "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" + justOneFromTheRevList=$(git rev-list --tags --max-count=1) + tagNameFromRepo=$(git describe --tags "$justOneFromTheRevList") + echo "$tagNameFromRepo" +} + +createArchive() { + repoLocation=$1 + targetName=$2 + + archiveExtension=$(getArchiveExtension) + + createOpenJDKArchive "${repoLocation}" "OpenJDK" + archive="${PWD}/OpenJDK${archiveExtension}" + + echo "Your final archive was created at ${archive}" + + echo "Moving the artifact to ${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[TARGET_DIR]}" + mv "${archive}" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[TARGET_DIR]}/${targetName}" } +# Create a Tar ball createOpenJDKTarArchive() { COMPRESS=gzip if which pigz >/dev/null 2>&1; then COMPRESS=pigz; fi echo "Archiving the build OpenJDK image and compressing with $COMPRESS" - if [ -z "$OPENJDK_REPO_TAG" ]; then + if [ -z "${OPENJDK_REPO_TAG+x}" ] || [ -z "${OPENJDK_REPO_TAG}" ]; then OPENJDK_REPO_TAG=$(getFirstTagFromOpenJDKGitRepo) fi - if [ -z "$JRE_TARGET_PATH" ]; then - JRE_TARGET_PATH="${OPENJDK_REPO_TAG/jdk/jre}" - [ "${JRE_TARGET_PATH}" == "${OPENJDK_REPO_TAG}" ] && JRE_TARGET_PATH="${OPENJDK_REPO_TAG}.jre" + if [ -z "${JRE_TARGET_PATH+x}" ] || [ -z "${JRE_TARGET_PATH}" ]; then + JRE_TARGET_PATH="${OPENJDK_REPO_TAG}-jre" fi - + echo "OpenJDK repo tag is ${OPENJDK_REPO_TAG}. JRE path will be ${JRE_TARGET_PATH}" - if [ "$USE_DOCKER" == "true" ] ; then - GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJDK.tar.gz ./"${OPENJDK_REPO_TAG}" - GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJRE.tar.gz ./"${JRE_TARGET_PATH}" - EXT=".tar.gz" + ## clean out old builds + rm -r "${BUILD_CONFIG[WORKSPACE_DIR]:?}/${BUILD_CONFIG[TARGET_DIR]}" || true + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[TARGET_DIR]}" || exit - echo "${good}Moving the artifact to ${TARGET_DIR}${normal}" - mv "OpenJDK${EXT}" "${TARGET_DIR}" - mv "OpenJRE${EXT}" "${TARGET_DIR}" - else - case "${OS_KERNEL_NAME}" in - *cygwin*) - zip -r -q OpenJDK.zip ./"${OPENJDK_REPO_TAG}" - zip -r -q OpenJRE.zip ./"${JRE_TARGET_PATH}" - EXT=".zip" ;; - aix) - GZIP=-9 tar -cf - ./"${OPENJDK_REPO_TAG}"/ | $COMPRESS -c > OpenJDK.tar.gz - GZIP=-9 tar -cf - ./"${JRE_TARGET_PATH}"/ | $COMPRESS -c > OpenJRE.tar.gz - EXT=".tar.gz" ;; - *) - GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJDK.tar.gz ./"${OPENJDK_REPO_TAG}" - GZIP=-9 tar --use-compress-program=$COMPRESS -cf OpenJRE.tar.gz ./"${JRE_TARGET_PATH}" - EXT=".tar.gz" ;; - esac - echo "${good}Your final ${EXT} was created at ${PWD}${normal}" - ls -l - echo "${good}Moving the artifacts to ${TARGET_DIR}${normal}" - # This really ought to be a separate parameter to makejdk_any_platform or - # TARGET_DIR should be a dir name as the name suggests, not a full filename - # This is currnently assuming TARGET_DIT has JDK in the filename, otherwise - # it wiill get overridden - mv "OpenJRE${EXT}" "${TARGET_DIR/JDK/JRE}" - mv "OpenJDK${EXT}" "${TARGET_DIR}" + if [ -d "${JRE_TARGET_PATH}" ]; then + createArchive "${JRE_TARGET_PATH}" "${BUILD_CONFIG[TARGET_FILE_NAME]/_/-jre_}" fi - + createArchive "${OPENJDK_REPO_TAG}" "${BUILD_CONFIG[TARGET_FILE_NAME]}" } +# Echo success showCompletionMessage() { echo "All done!" } -sourceFileWithColourCodes -checkIfDockerIsUsedForBuildingOrNot -createWorkingDirectory -downloadingRequiredDependencies # This function is in common-functions.sh +################################################################################ + +loadConfigFromFile +cd "${BUILD_CONFIG[WORKSPACE_DIR]}" + +parseArguments "$@" +configureWorkspace + +getOpenJDKUpdateAndBuildVersion configureCommandParameters -stepIntoTheWorkingDirectory -runTheOpenJDKConfigureCommandAndUseThePrebuiltConfigParams -buildOpenJDK +buildTemplatedFile +executeTemplatedFile + printJavaVersionString removingUnnecessaryFiles -makeACopyOfLibFreeFontForMacOSX "${OPENJDK_REPO_TAG}" "${COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG}" -makeACopyOfLibFreeFontForMacOSX "${JRE_PATH}" "${COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG}" -signRelease +makeACopyOfLibFreeFontForMacOSX "${OPENJDK_REPO_TAG}" "${BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]}" +makeACopyOfLibFreeFontForMacOSX "${OPENJDK_REPO_TAG}-jre" "${BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]}" createOpenJDKTarArchive showCompletionMessage + +# ccache is not detected properly TODO +# change grep to something like $GREP -e '^1.*' -e '^2.*' -e '^3\.0.*' -e '^3\.1\.[0123]$'`] +# See https://github.com/AdoptOpenJDK/openjdk-jdk8u/blob/dev/common/autoconf/build-performance.m4 diff --git a/sbin/build.template b/sbin/build.template new file mode 100644 index 000000000..16514cc60 --- /dev/null +++ b/sbin/build.template @@ -0,0 +1,49 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +################################################################################ +# +# build.template +# +# Writes the configure configuration to disk (config.status) for reuse +# +################################################################################ + +set +e + +alreadyConfigured=$(/usr/bin/find . -name "config.status") + +if [[ ! -z "$alreadyConfigured" ]] ; then + echo "Not reconfiguring due to the presence of config.status" +else + #Templated var that, gets replaced by build.sh + # shellcheck disable=SC1073,SC1054,SC1083 + {configureArg} + + exitCode=$? + if [ "${exitCode}" -ne 0 ]; then + exit 2; + fi +fi + +#Templated var that, gets replaced by build.sh +{makeCommandArg} + +exitCode=$? +# shellcheck disable=SC2181 +if [ "${exitCode}" -ne 0 ]; then + exit 3; +fi diff --git a/sbin/common-functions.sh b/sbin/common-functions.sh deleted file mode 100644 index af5eb7efe..000000000 --- a/sbin/common-functions.sh +++ /dev/null @@ -1,196 +0,0 @@ -#!/bin/bash -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -# Common functions to be used in scripts - -ALSA_LIB_VERSION=${ALSA_LIB_VERSION:-1.0.27.2} -FREETYPE_FONT_SHARED_OBJECT_FILENAME=libfreetype.so.6.5.0 -FREETYPE_FONT_VERSION=${FREETYPE_FONT_VERSION:-2.4.0} -FREEMARKER_LIB_VERSION=${FREEMARKER_LIB_VERSION:-2.3.8} - -determineBuildProperties() { - JVM_VARIANT=${JVM_VARIANT:-server} - - BUILD_TYPE=normal - DEFAULT_BUILD_FULL_NAME=${OS_KERNEL_NAME}-${OS_MACHINE_NAME}-${BUILD_TYPE}-${JVM_VARIANT}-release - export BUILD_FULL_NAME=${BUILD_FULL_NAME:-"$DEFAULT_BUILD_FULL_NAME"} -} - -# ALSA first for sound -checkingAndDownloadingAlsa() -{ - echo "Checking for ALSA" - - FOUND_ALSA=$(find "${WORKING_DIR}" -name "alsa-lib-${ALSA_LIB_VERSION}") - - if [[ ! -z "$FOUND_ALSA" ]] ; then - echo "Skipping ALSA download" - else - wget -nc ftp://ftp.alsa-project.org/pub/lib/alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 - if [[ ${OS_KERNEL_NAME} == "aix" ]] ; then - bzip2 -d alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 - tar xf alsa-lib-"${ALSA_LIB_VERSION}".tar - rm alsa-lib-"${ALSA_LIB_VERSION}".tar - else - tar xf alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 - rm alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 - fi - fi -} - -# Freemarker for OpenJ9 -checkingAndDownloadingFreemarker() -{ - echo "Checking for FREEMARKER" - - FOUND_FREEMARKER=$(find "${WORKING_DIR}" -type d -name "freemarker-${FREEMARKER_LIB_VERSION}") - - if [[ ! -z "$FOUND_FREEMARKER" ]] ; then - echo "Skipping FREEMARKER download" - else - # wget --no-check-certificate "https://sourceforge.net/projects/freemarker/files/freemarker/${FREEMARKER_LIB_VERSION}/freemarker-${FREEMARKER_LIB_VERSION}.tar.gz/download" -O "freemarker-${FREEMARKER_LIB_VERSION}.tar.gz" - # Temp fix as sourceforge is broken - wget --no-check-certificate https://ci.adoptopenjdk.net/userContent/freemarker-2.3.8.tar.gz - tar -xzf "freemarker-${FREEMARKER_LIB_VERSION}.tar.gz" - rm "freemarker-${FREEMARKER_LIB_VERSION}.tar.gz" - fi -} - -checkingAndDownloadingFreeType() -{ - echo "Checking for freetype $WORKING_DIR $OPENJDK_REPO_NAME " - - FOUND_FREETYPE=$(find "$WORKING_DIR/$OPENJDK_REPO_NAME/installedfreetype/lib" -name "${FREETYPE_FONT_SHARED_OBJECT_FILENAME}") - - if [[ ! -z "$FOUND_FREETYPE" ]] ; then - echo "Skipping FreeType download" - else - # Then FreeType for fonts: make it and use - wget -nc http://ftp.acc.umu.se/mirror/gnu.org/savannah/freetype/freetype-"$FREETYPE_FONT_VERSION".tar.gz - if [[ ${OS_KERNEL_NAME} == "aix" ]] ; then - gunzip xf freetype-"$FREETYPE_FONT_VERSION".tar.gz - tar xf freetype-"$FREETYPE_FONT_VERSION".tar - rm freetype-"$FREETYPE_FONT_VERSION".tar - MAKE=gmake - else - tar xf freetype-"$FREETYPE_FONT_VERSION".tar.gz - rm freetype-"$FREETYPE_FONT_VERSION".tar.gz - MAKE=make - fi - - cd freetype-"$FREETYPE_FONT_VERSION" || exit - - # We get the files we need at $WORKING_DIR/installedfreetype - # shellcheck disable=SC2046 - if ! (bash ./configure --prefix="${WORKING_DIR}"/"${OPENJDK_REPO_NAME}"/installedfreetype "${FREETYPE_FONT_BUILD_TYPE_PARAM}" && $MAKE all && $MAKE install); then - # shellcheck disable=SC2154 - echo "${error}Failed to configure and build libfreetype, exiting" - exit; - else - # shellcheck disable=SC2154 - echo "${good}Successfully configured OpenJDK with the FreeType library (libfreetype)!" - - if [[ ${OS_KERNEL_NAME} == "darwin" ]] ; then - TARGET_DYNAMIC_LIB_DIR="${WORKING_DIR}"/"${OPENJDK_REPO_NAME}"/installedfreetype/lib/ - TARGET_DYNAMIC_LIB="${TARGET_DYNAMIC_LIB_DIR}"/libfreetype.6.dylib - echo "" - echo "Listing the contents of ${TARGET_DYNAMIC_LIB_DIR} to see if the dynamic library 'libfreetype.6.dylib' has been created..." - ls "${TARGET_DYNAMIC_LIB_DIR}" - - echo "" - echo "Releasing the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" - set -x - install_name_tool -id @rpath/libfreetype.6.dylib "${TARGET_DYNAMIC_LIB}" - set +x - - # shellcheck disable=SC2181 - if [[ $? == 0 ]]; then - echo "Successfully released the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" - else - echo "Failed to release the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" - fi - fi - fi - # shellcheck disable=SC2154 - echo "${normal}" - fi -} - -checkingAndDownloadCaCerts() -{ - cd "$WORKING_DIR" || exit - - echo "Retrieving cacerts file" - - # Ensure it's the latest we pull in - rm -rf "${WORKING_DIR}/cacerts_area" - - git clone https://github.com/AdoptOpenJDK/openjdk-build.git cacerts_area - echo "cacerts should be here..." - - # shellcheck disable=SC2046 - if ! [ -r "${WORKING_DIR}/cacerts_area/security/cacerts" ]; then - echo "Failed to retrieve the cacerts file, exiting..." - exit; - else - echo "${good}Successfully retrieved the cacerts file!" - fi -} - -downloadingRequiredDependencies() -{ - if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then - echo "Windows or Windows-like environment detected, skipping downloading of dependencies...: Alsa, Freetype, and CaCerts." - else - echo "Downloading required dependencies...: Alsa, Freetype, Freemarker, and CaCerts." - time ( - echo "Checking and download Alsa dependency" - checkingAndDownloadingAlsa - ) - - if [[ -z "${FREETYPE}" ]] ; then - if [[ -z "$FREETYPE_DIRECTORY" ]]; then - time ( - echo "Checking and download FreeType Font dependency" - checkingAndDownloadingFreeType - ) - else - echo "" - echo "---> Skipping the process of checking and downloading the FreeType Font dependency, a pre-built version provided at $FREETYPE_DIRECTORY <---" - echo "" - fi - else - echo "Skipping Freetype" - fi - if [[ "$BUILD_VARIANT" == "openj9" ]]; then - time ( - echo "Checking and download Freemarker dependency" - checkingAndDownloadingFreemarker - ) - fi - time ( - echo "Checking and download CaCerts dependency" - checkingAndDownloadCaCerts - ) - fi -} - -getFirstTagFromOpenJDKGitRepo() -{ - git fetch --tags "${GIT_CLONE_ARGUMENTS[@]}" - justOneFromTheRevList=$(git rev-list --tags --max-count=1) - tagNameFromRepo=$(git describe --tags "$justOneFromTheRevList") - echo "$tagNameFromRepo" -} diff --git a/sbin/common/common.sh b/sbin/common/common.sh new file mode 100755 index 000000000..a630b2c37 --- /dev/null +++ b/sbin/common/common.sh @@ -0,0 +1,68 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +# shellcheck disable=SC2153 +function setOpenJdkVersion() { + local forest_name=$1 + + # Derive the openjdk_core_version from the forest name. + local openjdk_core_version=${forest_name} + if [[ ${forest_name} == *u ]]; then + openjdk_core_version=${forest_name%?} + fi + + BUILD_CONFIG[OPENJDK_CORE_VERSION]=$openjdk_core_version; + BUILD_CONFIG[OPENJDK_FOREST_NAME]=$forest_name; + + # 'u' means it's an update repo, e.g. jdk8u + if [[ ${BUILD_CONFIG[OPENJDK_FOREST_NAME]} == *u ]]; then + BUILD_CONFIG[OPENJDK_CORE_VERSION]=${BUILD_CONFIG[OPENJDK_FOREST_NAME]%?} + fi +} +# Create a Tar ball +getArchiveExtension() +{ + if [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" = *"cygwin"* ]]; then + EXT=".zip" + else + EXT=".tar.gz" + fi + + echo "${EXT}" +} + +# Create a Tar ball +createOpenJDKArchive() +{ + local repoDir="$1" + local fileName="$2" + + COMPRESS=gzip + if which pigz; then + COMPRESS=pigz; + fi + echo "Archiving the build OpenJDK image and compressing with $COMPRESS" + + EXT=$(getArchiveExtension) + + if [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" = *"cygwin"* ]]; then + zip -r -q "${fileName}.zip" ./"${repoDir}" + elif [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" == "aix" ]]; then + GZIP=-9 tar -cf - ./"${repoDir}"/ | $COMPRESS -c > $fileName.tar.gz + else + GZIP=-9 tar --use-compress-program=$COMPRESS -cf "${fileName}.tar.gz" ./"${repoDir}" + fi +} \ No newline at end of file diff --git a/sbin/common/config_init.sh b/sbin/common/config_init.sh new file mode 100755 index 000000000..fd1f13e63 --- /dev/null +++ b/sbin/common/config_init.sh @@ -0,0 +1,349 @@ +#!/bin/bash + +################################################################################ +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +################################################################################ + +################################################################################ +# +# This shell script deals with writing the AdoptOpenJDK build configuration to +# the file system so it can be picked up by further build steps, Docker +# containers etc +# +# We are deliberately writing to shell because this needs to work on some truly +# esoteric platforms to fulfil the Java Write Once Run Anywhere (WORA) promise. +# +################################################################################ + +# We can't use Bash 4.x+ associative arrays as as Apple won't support bash 4.0 +# (because of GPL3), we therefore have to name the indexes of the CONFIG_PARAMS +# map. This is why we can't have nice things. +CONFIG_PARAMS=( +BRANCH +BUILD_FULL_NAME +BUILD_VARIANT +CERTIFICATE +CLEAN_DOCKER_BUILD +CLEAN_GIT_REPO +CONFIGURE_ARGS_FOR_ANY_PLATFORM +CONTAINER_NAME +COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG +COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG +COPY_TO_HOST +DOCKER +DOCKER_FILE_PATH +DOCKER_SOURCE_VOLUME_NAME +FREETYPE +FREETYPE_DIRECTORY +FREETYPE_FONT_BUILD_TYPE_PARAM +FREETYPE_FONT_VERSION +KEEP_CONTAINER +JDK_BOOT_DIR +JDK_PATH +JRE_PATH +JVM_VARIANT +MAKE_ARGS_FOR_ANY_PLATFORM +MAKE_COMMAND_NAME +NUM_PROCESSORS +OPENJDK_BUILD_NUMBER +OPENJDK_CORE_VERSION +OPENJDK_FOREST_NAME +OPENJDK_SOURCE_DIR +OPENJDK_UPDATE_VERSION +OS_KERNEL_NAME +OS_ARCHITECTURE +REPOSITORY +REUSE_CONTAINER +SHALLOW_CLONE_OPTION +SIGN +TAG +TARGET_DIR +TARGET_FILE_NAME +TMP_CONTAINER_NAME +TMP_SPACE_BUILD +USE_DOCKER +USE_SSH +USER_SUPPLIED_CONFIGURE_ARGS +WORKING_DIR +WORKSPACE_DIR +) + +# Directory structure of build environment: +########################################################################################################################################### +# Dir Purpose Docker Default Native default +########################################################################################################################################### +# Root /openjdk/ $(pwd)/workspace/ +# /config Configuration /openjdk/config $(pwd)/workspace/config +# / Build area /openjdk/build $(pwd)/workspace/build/ +# // Source code /openjdk/build/src $(pwd)/workspace/build/src +# /target Destination of built artifacts /openjdk/target $(pwd)/workspace/target + +# Helper code to perform index lookups by name +declare -a -x PARAM_LOOKUP +numParams=$((${#CONFIG_PARAMS[@]})) + +# seq not available on aix +index=0 +# shellcheck disable=SC2086 +while [ $index -lt $numParams ]; do + paramName=${CONFIG_PARAMS[$index]}; + eval declare -r -x "$paramName=$index" + PARAM_LOOKUP[$index]=$paramName + + let index=index+1 +done + +function displayParams() { + echo "# ============================" + echo "# OPENJDK BUILD CONFIGURATION:" + echo "# ============================" + for K in "${!BUILD_CONFIG[@]}"; + do + echo "BUILD_CONFIG[${PARAM_LOOKUP[$K]}]=\"${BUILD_CONFIG[$K]}\"" + done | sort +} + +function writeConfigToFile() { + if [ ! -d "workspace/config" ] + then + mkdir -p "workspace/config" + fi + displayParams | sed 's/\r$//' > ./workspace/config/built_config.cfg +} + +function loadConfigFromFile() { + if [ -f "$SCRIPT_DIR/../config/built_config.cfg" ] + then + # shellcheck disable=SC1091,SC1090 + source "$SCRIPT_DIR/../config/built_config.cfg" + elif [ -f "config/built_config.cfg" ] + then + # shellcheck disable=SC1091 + source config/built_config.cfg + elif [ -f "workspace/config/built_config.cfg" ] + then + # shellcheck disable=SC1091 + source workspace/config/built_config.cfg + elif [ -f "built_config.cfg" ] + then + # shellcheck disable=SC1091 + source built_config.cfg + else + echo "Failed to find configuration" + exit + fi +} + +# Parse the configuration args from the CL, please keep this in alpha order +function parseConfigurationArguments() { + + while [[ $# -gt 0 ]] && [[ ."$1" = .-* ]] ; do + opt="$1"; + shift; + case "$opt" in + "--" ) break 2;; + + "--build-variant" ) + BUILD_CONFIG[BUILD_VARIANT]="$1"; shift;; + + "--branch" | "-b" ) + BUILD_CONFIG[BRANCH]="$1"; shift;; + + "--build-number" | "-B" ) + BUILD_CONFIG[OPENJDK_BUILD_NUMBER]="$1"; shift;; + + "--configure-args" | "-C" ) + BUILD_CONFIG[USER_SUPPLIED_CONFIGURE_ARGS]="$1"; shift;; + + "--clean-docker-build" | "-c" ) + BUILD_CONFIG[CLEAN_DOCKER_BUILD]=true;; + + "--clean-git-repo" ) + BUILD_CONFIG[CLEAN_GIT_REPO]=true;; + + "--destination" | "-d" ) + BUILD_CONFIG[TARGET_DIR]="$1"; shift;; + + "--docker" | "-D" ) + BUILD_CONFIG[USE_DOCKER]="true";; + + "--disable-shallow-git-clone" ) + BUILD_CONFIG[SHALLOW_CLONE_OPTION]=""; shift;; + + "--freetype-dir" | "-f" ) + BUILD_CONFIG[FREETYPE_DIRECTORY]="$1"; shift;; + + "--skip-freetype" | "-F" ) + BUILD_CONFIG[FREETYPE]=false;; + + "--help" | "-h" ) + man ./makejdk-any-platform.1;; + + "--ignore-container" | "-i" ) + BUILD_CONFIG[REUSE_CONTAINER]=false;; + + "--jdk-boot-dir" | "-J" ) + BUILD_CONFIG[JDK_BOOT_DIR]="$1";shift;; + + "--keep" | "-k" ) + BUILD_CONFIG[KEEP_CONTAINER]=true;; + + "--processors" | "-p" ) + BUILD_CONFIG[NUM_PROCESSORS]="$1"; shift;; + + "--repository" | "-r" ) + BUILD_CONFIG[REPOSITORY]="$1"; shift;; + + "--source" | "-s" ) + BUILD_CONFIG[WORKING_DIR]="$1"; shift;; + + "--ssh" | "-S" ) + BUILD_CONFIG[USE_SSH]=true;; + + "--sign" ) + BUILD_CONFIG[SIGN]=true; BUILD_CONFIG[CERTIFICATE]="$1"; shift;; + + "--sudo" ) + BUILD_CONFIG[DOCKER]="sudo docker";; + + "--tag" | "-t" ) + BUILD_CONFIG[TAG]="$1"; BUILD_CONFIG[SHALLOW_CLONE_OPTION]=""; shift;; + + "--target-file-name" | "-T" ) + BUILD_CONFIG[TARGET_FILE_NAME]="$1"; shift;; + + "--tmp-space-build") + BUILD_CONFIG[TMP_SPACE_BUILD]=true;; + + "--update-version" | "-u" ) + BUILD_CONFIG[OPENJDK_UPDATE_VERSION]="$1"; shift;; + + "--version" | "-v" ) + setOpenJdkVersion "$1"; shift;; + + "--jvm-variant" | "-V" ) + BUILD_CONFIG[JVM_VARIANT]="$1"; shift;; + + *) echo >&2 "Invalid build.sh option: ${opt}"; exit 1;; + esac + done +} + +# Set the config defaults +function configDefaults() { + # The OS kernel name, e.g. 'darwin' for Mac OS X + BUILD_CONFIG[OS_KERNEL_NAME]=$(uname | awk '{print tolower($0)}') + + # The O/S architecture, e.g. x86_64 for a modern intel / Mac OS X + BUILD_CONFIG[OS_ARCHITECTURE]=$(uname -m) + + # The full forest name, e.g. jdk8, jdk8u, jdk9, jdk9u, etc. + BUILD_CONFIG[OPENJDK_FOREST_NAME]="" + + # The abridged openjdk core version name, e.g. jdk8, jdk9, etc. + BUILD_CONFIG[OPENJDK_CORE_VERSION]="" + + # The build variant, e.g. openj9 + BUILD_CONFIG[BUILD_VARIANT]="" + + # The OpenJDK source code repository to build from, e.g. an AdoptOpenJDK repo + BUILD_CONFIG[REPOSITORY]="" + + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JDK_FLAG]="false" + BUILD_CONFIG[COPY_MACOSX_FREE_FONT_LIB_FOR_JRE_FLAG]="false" + BUILD_CONFIG[FREETYPE]=true + BUILD_CONFIG[FREETYPE_DIRECTORY]="" + BUILD_CONFIG[FREETYPE_FONT_VERSION]="2.4.0" + BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]="" + + BUILD_CONFIG[MAKE_COMMAND_NAME]="make" + BUILD_CONFIG[SIGN]="false" + BUILD_CONFIG[JDK_BOOT_DIR]="" + + BUILD_CONFIG[NUM_PROCESSORS]="1" + BUILD_CONFIG[TARGET_FILE_NAME]="OpenJDK" + + # Dir where we clone the OpenJDK source code for building, defaults to 'src' + BUILD_CONFIG[OPENJDK_SOURCE_DIR]="src" + + # By default only git clone the HEAD commit + BUILD_CONFIG[SHALLOW_CLONE_OPTION]=${BUILD_CONFIG[SHALLOW_CLONE_OPTION]:-"--depth=1"} + + # Set Docker Container names and defaults + BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]=${BUILD_CONFIG[DOCKER_SOURCE_VOLUME_NAME]:-"openjdk-source-volume"} + + BUILD_CONFIG[CONTAINER_NAME]=${BUILD_CONFIG[CONTAINER_NAME]:-openjdk_container} + + BUILD_CONFIG[TMP_CONTAINER_NAME]=${BUILD_CONFIG[TMP_CONTAINER_NAME]:-openjdk-copy-src} + BUILD_CONFIG[CLEAN_DOCKER_BUILD]=${BUILD_CONFIG[CLEAN_DOCKER_BUILD]:-false} + + # Use Docker to build (defaults to false) + BUILD_CONFIG[USE_DOCKER]=${BUILD_CONFIG[USE_DOCKER]:-false} + + # Location of DockerFile and where scripts get copied to inside the container + BUILD_CONFIG[DOCKER_FILE_PATH]=${BUILD_CONFIG[DOCKER_FILE_PATH]:-""} + + # Whether we keep the Docker container after we build it + # TODO Please note that the persistent volume is managed separately + BUILD_CONFIG[KEEP_CONTAINER]=${BUILD_CONFIG[KEEP_CONTAINER]:-false} + + # Whether we use an existing container + # TODO Please note that the persistent volume is managed separately + BUILD_CONFIG[REUSE_CONTAINER]=${BUILD_CONFIG[REUSE_CONTAINER]:-true} + + # The current working directory + BUILD_CONFIG[WORKING_DIR]=${BUILD_CONFIG[WORKING_DIR]:-"./build/"} + + # Root of the workspace + BUILD_CONFIG[WORKSPACE_DIR]=${BUILD_CONFIG[WORKSPACE_DIR]:-""} + + # Use SSH for the GitHub connection (defaults to false) + BUILD_CONFIG[USE_SSH]=${BUILD_CONFIG[USE_SSH]:-false} + + # Director where OpenJDK binary gets built to + BUILD_CONFIG[TARGET_DIR]=${BUILD_CONFIG[TARGET_DIR]:-"target/"} + + # Which repo branch to build, e.g. dev + BUILD_CONFIG[BRANCH]=${BUILD_CONFIG[BRANCH]:-"dev"} + + # Which repo tag to build, e.g. jdk8u172-b03 + BUILD_CONFIG[TAG]=${BUILD_CONFIG[TAG]:-""} + + # Update version e.g. 172 + BUILD_CONFIG[OPENJDK_UPDATE_VERSION]=${BUILD_CONFIG[OPENJDK_UPDATE_VERSION]:-""} + + # build number e.g. b03 + BUILD_CONFIG[OPENJDK_BUILD_NUMBER]=${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]:-""} + + # Build variant, e.g. openj9, defaults to "" which means hotspot + BUILD_CONFIG[BUILD_VARIANT]=${BUILD_CONFIG[BUILD_VARIANT]:-""} + + # JVM variant, e.g. client or server, defaults to server + BUILD_CONFIG[JVM_VARIANT]=${BUILD_CONFIG[JVM_VARIANT]:-""} + + # Any extra args provided by the user + BUILD_CONFIG[USER_SUPPLIED_CONFIGURE_ARGS]=${BUILD_CONFIG[USER_SUPPLIED_CONFIGURE_ARGS]:-""} + + BUILD_CONFIG[DOCKER]=${BUILD_CONFIG[DOCKER]:-"docker"} + + BUILD_CONFIG[TMP_SPACE_BUILD]=${BUILD_CONFIG[TMP_SPACE_BUILD]:-false} + + # If the wrong git repo is there allow it to be removed + BUILD_CONFIG[CLEAN_GIT_REPO]=false +} + +# Declare the map of build configuration that we're going to use +declare -ax BUILD_CONFIG +export BUILD_CONFIG diff --git a/sbin/colour-codes.sh b/sbin/common/constants.sh old mode 100644 new mode 100755 similarity index 54% rename from sbin/colour-codes.sh rename to sbin/common/constants.sh index e9a2196fc..4c8bd94c4 --- a/sbin/colour-codes.sh +++ b/sbin/common/constants.sh @@ -1,5 +1,6 @@ #!/bin/bash -# + +################################################################################ # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at @@ -11,23 +12,15 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -# -# Purpose: This script was contains colour codes that will be commonly used across multiple scripts - -# Escape code -esc=$(echo -en "\033") +################################################################################ -# Set colors -# shellcheck disable=SC2034 -error="${esc}[0;31m" -# shellcheck disable=SC2034 -warning="${esc}[0;33m" #yellow -# shellcheck disable=SC2034 -good="${esc}[0;32m" -# shellcheck disable=SC2034 -info="${esc}[0;33m" -# shellcheck disable=SC2034 -git="${esc}[0;34m" -# shellcheck disable=SC2034 -normal=$(echo -en "${esc}[m\017") +export JDK8_VERSION="jdk8u"; +export JDK9_VERSION="jdk9u"; +export JDK10_VERSION="jdk10u"; +export JDK11_VERSION="jdk11"; +export JDK8_CORE_VERSION="jdk8"; +export JDK9_CORE_VERSION="jdk9"; +export JDK10_CORE_VERSION="jdk10"; +export JDK11_CORE_VERSION="jdk11"; +export AMBER_CORE_VERSION="amber"; \ No newline at end of file diff --git a/sbin/jtreg.sh b/sbin/jtreg.sh deleted file mode 100755 index e355b7727..000000000 --- a/sbin/jtreg.sh +++ /dev/null @@ -1,170 +0,0 @@ -#!/bin/bash -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# # -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -# shellcheck source=sbin/common-functions.sh -source "$SCRIPT_DIR/common-functions.sh" - -WORKING_DIR=$1 -OPENJDK_REPO_NAME=$2 -BUILD_FULL_NAME=$3 -VERSION=$4 -# shellcheck disable=SC2001 -JTREG_TEST_SUBSETS=$(echo "$5" | sed 's/:/ /') -JTREG_VERSION=${JTREG_VERSION:-4.2.0-tip} -JTREG_TARGET_FOLDER=${JTREG_TARGET_FOLDER:-jtreg} -JOB_NAME=${JOB_NAME:-OpenJDK} -NUM_PROCESSORS=${NUM_PROCESSORS:-$(getconf _NPROCESSORS_ONLN)} -TMP_DIR=$(dirname "$(mktemp -u)") -OPENJDK_DIR="$WORKING_DIR/$OPENJDK_REPO_NAME" -TARGET_DIR="$WORKING_DIR" - -determineBuildProperties - -checkIfDockerIsUsedForBuildingOrNot() -{ - if [[ -f /.dockerenv ]] ; then - echo "Detected we're in docker" - WORKING_DIR=/openjdk/build/ - OPENJDK_REPO_NAME=openjdk/ - OPENJDK_DIR="$WORKING_DIR/$OPENJDK_REPO_NAME" - TARGET_DIR=/openjdk/target/ - # Keep as a variable for potential use later - # if we wish to copy the results to the host - # shellcheck disable=SC2034 - IN_DOCKER=true - fi -} - -downloadJtregAndSetupEnvironment() -{ - # Download then add jtreg to our path - if [[ ! -d "${WORKING_DIR}/${JTREG_TARGET_FOLDER}" ]]; then - echo "Downloading Jtreg binary" - JTREG_BINARY_FILE="jtreg-${JTREG_VERSION}.tar.gz" - - # shellcheck disable=SC2046 - if ! (cd "$TMP_DIR" && wget https://ci.adoptopenjdk.net/job/jtreg/lastSuccessfulBuild/artifact/"$JTREG_BINARY_FILE"); then - echo "Failed to retrieve the jtreg binary, exiting" - exit - fi - - - cd "$WORKING_DIR" && tar xf "$TMP_DIR/$JTREG_BINARY_FILE" - fi - - echo "List contents of the jtreg folder" - ls "$WORKING_DIR/$JTREG_TARGET_FOLDER"/* - - export JT_HOME=$WORKING_DIR/$JTREG_TARGET_FOLDER - - export PATH=$JT_HOME/bin:$PATH - - # Clean up after ourselves by removing jtreg tgz - rm -f "$JTREG_BINARY_FILE" -} - -applyingJCovSettingsToMakefileForTests() -{ - echo "Apply JCov settings to Makefile..." - if [[ $VERSION == *8* ]]; then - cd "$OPENJDK_DIR/jdk/test" || exit - sed -i "s/-vmoption:-Xmx512m.*/-vmoption:-Xmx512m -xml:verify -jcov\/classes:\$\(ABS_PLATFORM_BUILD_ROOT\)\/images\/j2sdk-image\/jre\/lib\/rt.jar -jcov\/source:\$\(ABS_PLATFORM_BUILD_ROOT\)\/images\/j2sdk-image\/src.zip -jcov\/include:*/" Makefile - else - cd "$OPENJDK_DIR/test" || exit - # TODO pass in correct jcov parameter for jdk9 and up - # sed -i "s/-vmoption:-Xmx512m.*/-vmoption:-Xmx512m -xml:verify -jcov\/classes:\$\(ABS_PLATFORM_BUILD_ROOT\)\/jdk\/classes\/ -jcov\/source:\$\(ABS_PLATFORM_BUILD_ROOT\)\/..\/..\/jdk\/src\/share\/classes -jcov\/include:*/" TestCommon.gmk - fi - pwd - cd "$OPENJDK_DIR" || exit -} - -settingUpEnvironmentVariablesForJTREG() -{ - echo "Setting up environment variables for JTREG to run" - - # This is the JDK we'll test - export PRODUCT_HOME=$OPENJDK_DIR/build/$BUILD_FULL_NAME/images/j2sdk-image - echo "$PRODUCT_HOME" - ls "$PRODUCT_HOME" - - export JTREG_DIR=$WORKING_DIR/jtreg - export JTREG_INSTALL=${JTREG_DIR} - export JT_HOME=${JTREG_INSTALL} - export JTREG_HOME=${JTREG_INSTALL} - export JPRT_JTREG_HOME=${JT_HOME} - export JPRT_JAVA_HOME=${PRODUCT_HOME} - export JTREG_TIMEOUT_FACTOR=5 - export CONCURRENCY=$NUM_PROCESSORS -} - -runJtregViaMakeCommand() -{ - echo "Running jtreg via make command (debug logs enabled)" - if [ -z "$JTREG_TEST_SUBSETS" ]; then - make test jobs=10 LOG=debug - else - make test jobs=10 LOG=debug TEST="$JTREG_TEST_SUBSETS" - fi -} - -packageTestResultsWithJCovReports() -{ - echo "Package test output into archives..." - pwd - - cd "$OPENJDK_DIR/build/$BUILD_FULL_NAME/" || exit - - artifact="${JOB_NAME}-testoutput-with-jcov-reports" - echo "Tarring and zipping the 'testoutput' folder into artefact: $artifact.tar.gz" - tar -czf "$TARGET_DIR/$artifact.tar.gz" testoutput/ - - if [ -d testoutput ]; then - rm -fr "$WORKING_DIR/$OPENJDK_REPO_NAME/testoutput" - fi - cp -fr testoutput/ "$WORKING_DIR/testoutput/" - - cd "$WORKING_DIR" || exit -} - -packageOnlyJCovReports() -{ - echo "Package jcov reports into archives..." - pwd - - cd "$OPENJDK_DIR/build/$BUILD_FULL_NAME/" || exit - pwd - - artifact="${JOB_NAME}-jcov-results-only" - echo "Tarring and zipping the 'testoutput/../jcov' folder into artefact: $artifact.tar.gz" - tar -czf "$TARGET_DIR/$artifact.tar.gz" testoutput/*/JTreport/jcov/ - - cd "$WORKING_DIR" || exit -} - -packageReports() -{ - echo "Archiving your jtreg results (includes jcov reports)" - packageTestResultsWithJCovReports - packageOnlyJCovReports -} - -checkIfDockerIsUsedForBuildingOrNot -downloadingRequiredDependencies -downloadJtregAndSetupEnvironment -applyingJCovSettingsToMakefileForTests -settingUpEnvironmentVariablesForJTREG -runJtregViaMakeCommand -packageReports diff --git a/sbin/jtreg_prep.sh b/sbin/jtreg_prep.sh deleted file mode 100755 index 7e0e3e58b..000000000 --- a/sbin/jtreg_prep.sh +++ /dev/null @@ -1,169 +0,0 @@ -#!/bin/bash -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# - -# Purpose: This script was designed to do any+all setup required by the jtreg.sh script in order to run it. -# Tasks: Retrieve Java, unpack it if need-be, and store it locally in a specific location. If the location is blank, we put it in - -set -eu - -REPOSITORY=adoptopenjdk/openjdk-jdk8u -OPENJDK_REPO_NAME=openjdk -SHALLOW_CLONE_OPTION="--depth=1" - -JAVA_SOURCE="" -JAVA_DESTINATION="" -WORKING_DIR="" -USE_SSH=false -BRANCH="" - -parseCommandLineArgs() -{ - while [[ $# -gt 0 ]] && [[ ."$1" == .-* ]] ; do - opt="$1"; - shift; - case "$opt" in - "--" ) break 2;; - - "--source" ) - JAVA_SOURCE="$1"; shift;; - - "--destination" ) - JAVA_DESTINATION="$1"; shift;; - - "--ssh" | "-S" ) - USE_SSH=true;; - - "--repository" | "-r" ) - REPOSITORY="$1"; shift;; - - "--branch" | "-b" ) - BRANCH="$1"; shift;; - - "--working_dir" ) - WORKING_DIR="$1"; shift;; - - "--disable-shallow-git-clone" | "-dsgc" ) - SHALLOW_CLONE_OPTION=""; shift;; - - *) echo >&2 "Invalid option: ${opt}"; echo "This option was unrecognised. See the script jtreg_prep.sh for a full list."; exit 1;; - esac - done - - if [ -z "${JAVA_SOURCE}" ]; then - echo >&2 "jtreg_prep.sh failed: --source must be specified"; exit 1 - fi - - if [ -z "${WORKING_DIR}" ] ; then - echo "WORKING_DIR is undefined so setting to $PWD" - WORKING_DIR=$PWD - else - echo "Working dir is $WORKING_DIR" - fi - - if [ -z "${JAVA_DESTINATION}" ]; then - JAVA_DESTINATION="$WORKING_DIR/$OPENJDK_REPO_NAME/build/java_home/images" - fi - - if [ -z "${BRANCH}" ] ; then - echo "BRANCH is undefined so checking out dev" - BRANCH="dev" - fi -} - -# Step 1: Fetch OpenJDK, as that's where the tests live. -cloneOpenJDKRepo() -{ - if [ -d "$WORKING_DIR/$OPENJDK_REPO_NAME/.git" ] && [ "$REPOSITORY" == "adoptopenjdk/openjdk-jdk8u" ] ; then - # It does exist and it's a repo other than the AdoptOpenJDK one - cd "$WORKING_DIR/$OPENJDK_REPO_NAME" || exit - echo "Will reset the repository at $PWD in 10 seconds..." - sleep 10 - echo "Pulling latest changes from git repo" - git fetch --all - git reset --hard origin/$BRANCH - cd "$WORKING_DIR" || exit - elif [ ! -d "${WORKING_DIR}/${OPENJDK_REPO_NAME}/.git" ] ; then - # If it doesn't exist, clone it - echo "Didn't find any existing openjdk repository at WORKING_DIR (set to ${WORKING_DIR}) so cloning the source to openjdk" - if [[ "$USE_SSH" == "true" ]] ; then - GIT_REMOTE_REPO_ADDRESS="git@github.com:${REPOSITORY}.git" - else - GIT_REMOTE_REPO_ADDRESS="https://github.com/${REPOSITORY}.git" - fi - - if [[ "$SHALLOW_CLONE_OPTION" == "" ]]; then - echo "Git repo cloning mode: deep (preserves commit history)" - else - echo "Git repo cloning mode: shallow (DOES NOT contain commit history)" - fi - - GIT_CLONE_ARGUMENTS=("$SHALLOW_CLONE_OPTION" '-b' "$BRANCH" "$GIT_REMOTE_REPO_ADDRESS" "${WORKING_DIR}/${OPENJDK_REPO_NAME}") - - echo "git clone ${GIT_CLONE_ARGUMENTS[*]}" - git clone "${GIT_CLONE_ARGUMENTS[@]}" - fi -} - -# Step 2: Retrieve Java -downloadJDKArtifact() -{ - if [ ! -d "${JAVA_DESTINATION}" ]; then - mkdir -p "${JAVA_DESTINATION}" - fi - cd "${JAVA_DESTINATION}" || exit - - #If it's a http location, use wget. - if [[ "${JAVA_SOURCE}" == http* ]]; then - # shellcheck disable=SC2046 - if ! (wget "${JAVA_SOURCE}"); then - echo "Failed to retrieve the JDK binary from ${JAVA_SOURCE}, exiting" - exit 1 - fi - else #Assume it's local or on a mounted drive. - if [ -f "${JAVA_SOURCE}" ] || [ -d "${JAVA_SOURCE}" ]; then - cp -r "${JAVA_SOURCE}" . - else - echo "The JDK artifact could not be found at the source location: ${JAVA_SOURCE}." - exit 1 - fi - fi -} - -# Step 3: Unpack Java if we need to. -unpackJDKTarArtifact() -{ - if [[ "$JAVA_SOURCE" == *\.tar\.gz ]]; then #If it's a tar file, unpack it. - cd "${JAVA_DESTINATION}" || exit - tar xf ./*.tar.gz - echo "The JDK artifact has been untarred at ${JAVA_DESTINATION}." - cd "${WORKING_DIR}" || exit - elif [ ! -d "${JAVA_SOURCE}" ]; then #If it's not a directory, then we don't know how to unpack it. - echo "The Java file you specified as source was copied to the destination, but this script doesn't know how to unpack it. Please add this logic to this script, or unpack it manually before running jtreg."; - fi -} - -# Step 4: Finish -printFinishedMessage() -{ - echo "$0 has finished successfully." -} - -################################################################## - -parseCommandLineArgs "$@" -cloneOpenJDKRepo -downloadJDKArtifact -unpackJDKTarArtifact -printFinishedMessage diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh new file mode 100755 index 000000000..44dfbba49 --- /dev/null +++ b/sbin/prepareWorkspace.sh @@ -0,0 +1,342 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +################################################################################ +# +# This script prepares the workspace to build (Adopt) OpenJDK. +# See the configureWorkspace function for details +# It is sourced by build.sh +# +################################################################################ + +set -eu + + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/common/constants.sh" + +# Set default versions for 3 libraries that OpenJDK relies on to build +ALSA_LIB_VERSION=${ALSA_LIB_VERSION:-1.1.6} +FREETYPE_FONT_SHARED_OBJECT_FILENAME="libfreetype.so*" +FREEMARKER_LIB_VERSION=${FREEMARKER_LIB_VERSION:-2.3.28} + +# Create a new clone or update the existing clone of the OpenJDK source repo +# TODO refactor this for SRP +checkoutAndCloneOpenJDKGitRepo() +{ + + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" + + # Check that we have a git repo of a valid openjdk version on our local file system + if [ -d "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" ] && ( [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK10_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK11_CORE_VERSION}" ]) ; then + + set +e + git --git-dir "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" remote -v + echo "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" + git --git-dir "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" remote -v | grep "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" | grep --quiet "${BUILD_CONFIG[REPOSITORY]}" + local isCorrectGitRepo=$? + set -e + + + # If the local copy of the git source repo is valid then we reset appropriately + if [ "${isCorrectGitRepo}" == "0" ]; then + cd "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || return + echo "Resetting the git openjdk source repository at $PWD in 10 seconds..." + sleep 10 + echo "Pulling latest changes from git openjdk source repository" + + git fetch --all ${BUILD_CONFIG[SHALLOW_CLONE_OPTION]} + git reset --hard "origin/${BUILD_CONFIG[BRANCH]}" + if [ ! -z "${BUILD_CONFIG[TAG]}" ]; then + git checkout "${BUILD_CONFIG[TAG]}" + fi + git clean -fdx + elif [ "${BUILD_CONFIG[CLEAN_GIT_REPO]}" == "true" ]; then + echo "Removing current git repo as it is the wrong type" + rm -rf "${BUILD_CONFIG[WORKSPACE_DIR]:?}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" + cloneOpenJDKGitRepo + else + echo "Incorrect Source Code for ${BUILD_CONFIG[OPENJDK_FOREST_NAME]}. This is an error, please check what is in $PWD and manually remove, exiting..." + echo "If this is inside a docker you can purge the existing source by passing --clean-docker-build" + exit 1 + fi + elif [ ! -d "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" ] ; then + # If it doesn't exist, clone it + echo "Didn't find any existing openjdk repository at $(pwd)/${BUILD_CONFIG[WORKING_DIR]} so cloning the source to openjdk" + cloneOpenJDKGitRepo + fi + + cd "${BUILD_CONFIG[WORKSPACE_DIR]}" +} + +# Set the git clone arguments +setGitCloneArguments() { + cd "${BUILD_CONFIG[WORKSPACE_DIR]}" + local git_remote_repo_address; + if [[ "${BUILD_CONFIG[USE_SSH]}" == "true" ]] ; then + git_remote_repo_address="git@github.com:${BUILD_CONFIG[REPOSITORY]}.git" + else + git_remote_repo_address="https://github.com/${BUILD_CONFIG[REPOSITORY]}.git" + fi + + GIT_CLONE_ARGUMENTS=(${BUILD_CONFIG[SHALLOW_CLONE_OPTION]} '-b' "${BUILD_CONFIG[BRANCH]}" "$git_remote_repo_address" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}") +} + +# Clone the git repo +cloneOpenJDKGitRepo() +{ + setGitCloneArguments + + echo "git clone ${GIT_CLONE_ARGUMENTS[*]}" + git clone "${GIT_CLONE_ARGUMENTS[@]}" + if [ ! -z "${BUILD_CONFIG[TAG]}" ]; then + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || exit 1 + git checkout "${BUILD_CONFIG[TAG]}" + fi + + # TODO extract this to its own function + # Building OpenJDK with OpenJ9 must run get_source.sh to clone openj9 and openj9-omr repositories + if [ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]; then + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || return + bash get_source.sh + fi + cd "${BUILD_CONFIG[WORKSPACE_DIR]}" +} + +# Create the workspace +createWorkspace() +{ + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}" || exit + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" || exit +} + +# ALSA first for sound +checkingAndDownloadingAlsa() +{ + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/" || exit + + echo "Checking for ALSA" + + FOUND_ALSA=$(find "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/" -name "installedalsa") + + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedalsa/" || exit + + if [[ ! -z "$FOUND_ALSA" ]] + then + echo "Skipping ALSA download" + else + # TODO Holy security problem Batman! + #wget -nc ftp://ftp.alsa-project.org/pub/lib/alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 + wget -nc https://ftp.osuosl.org/pub/blfs/conglomeration/alsa-lib/alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 + if [[ "${BUILD_CONFIG[OS_KERNEL_NAME]}" == "aix" ]] ; then + bzip2 -d alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 + tar -xf alsa-lib-"${ALSA_LIB_VERSION}".tar --strip-components=1 -C "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedalsa/" + rm alsa-lib-"${ALSA_LIB_VERSION}".tar + else + tar -xf alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 --strip-components=1 -C "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedalsa/" + rm alsa-lib-"${ALSA_LIB_VERSION}".tar.bz2 + fi + fi +} + +# Freemarker for OpenJ9 +checkingAndDownloadingFreemarker() +{ + echo "Checking for FREEMARKER" + + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/" || exit + FOUND_FREEMARKER=$(find "." -type d -name "freemarker-${FREEMARKER_LIB_VERSION}") + + if [[ ! -z "$FOUND_FREEMARKER" ]] ; then + echo "Skipping FREEMARKER download" + else + + wget -nc --no-check-certificate "http://www.mirrorservice.org/sites/ftp.apache.org/freemarker/engine/${FREEMARKER_LIB_VERSION}/binaries/apache-freemarker-${FREEMARKER_LIB_VERSION}-bin.tar.gz" + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/freemarker-${FREEMARKER_LIB_VERSION}/" || exit + tar -xzf "apache-freemarker-${FREEMARKER_LIB_VERSION}-bin.tar.gz" --strip-components=1 -C "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/freemarker-${FREEMARKER_LIB_VERSION}/" + rm "apache-freemarker-${FREEMARKER_LIB_VERSION}-bin.tar.gz" + fi +} + +# Get Freetype +checkingAndDownloadingFreeType() +{ + + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/" || exit + echo "Checking for freetype at ${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" + + FOUND_FREETYPE=$(find "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedfreetype/lib" -name "${FREETYPE_FONT_SHARED_OBJECT_FILENAME}" || true) + + if [[ ! -z "$FOUND_FREETYPE" ]] ; then + echo "Skipping FreeType download" + else + # Then FreeType for fonts: make it and use + wget -nc --no-check-certificate https://ftp.acc.umu.se/mirror/gnu.org/savannah/freetype/freetype-"${BUILD_CONFIG[FREETYPE_FONT_VERSION]}".tar.gz + + gunzip -dc freetype-"${BUILD_CONFIG[FREETYPE_FONT_VERSION]}".tar.gz | tar xf - -C . + rm freetype-"${BUILD_CONFIG[FREETYPE_FONT_VERSION]}".tar.gz + + cd freetype-"${BUILD_CONFIG[FREETYPE_FONT_VERSION]}" || exit + + # We get the files we need at $WORKING_DIR/installedfreetype + # shellcheck disable=SC2046 + if ! (bash ./configure --prefix="${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}"/installedfreetype "${BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]}" && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} all && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} install); then + # shellcheck disable=SC2154 + echo "Failed to configure and build libfreetype, exiting" + exit; + else + # shellcheck disable=SC2154 + echo "Successfully configured OpenJDK with the FreeType library (libfreetype)!" + + if [[ ${BUILD_CONFIG[OS_KERNEL_NAME]} == "darwin" ]] ; then + TARGET_DYNAMIC_LIB_DIR="${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}"/installedfreetype/lib/ + TARGET_DYNAMIC_LIB="${TARGET_DYNAMIC_LIB_DIR}"/libfreetype.6.dylib + + echo "Listing the contents of ${TARGET_DYNAMIC_LIB_DIR} to see if the dynamic library 'libfreetype.6.dylib' has been created..." + ls "${TARGET_DYNAMIC_LIB_DIR}" + + echo "Releasing the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" + install_name_tool -id @rpath/libfreetype.6.dylib "${TARGET_DYNAMIC_LIB}" + + # shellcheck disable=SC2181 + if [[ $? == 0 ]]; then + echo "Successfully released the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" + else + echo "Failed to release the runpath dependency of the dynamic library ${TARGET_DYNAMIC_LIB}" + fi + fi + fi + fi +} + +# Certificate Authority Certs (CA Certs) +checkingAndDownloadCaCerts() +{ + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" || exit + + echo "Retrieving cacerts file" + + # Ensure it's the latest we pull in + rm -rf "cacerts_area" + mkdir "cacerts_area" || exit + cd "cacerts_area" || exit + + git init + git remote add origin -f https://github.com/AdoptOpenJDK/openjdk-build.git + git config core.sparsecheckout true + echo "security/*" >> .git/info/sparse-checkout + git pull origin master + + echo "cacerts should be here..." + + # shellcheck disable=SC2046 + if ! [ -r "security/cacerts" ]; then + echo "Failed to retrieve the cacerts file, exiting..." + exit; + else + echo "Successfully retrieved the cacerts file!" + fi + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" || exit +} + +# Download all of the dependencies for OpenJDK (Alsa, FreeType, CACerts et al) +downloadingRequiredDependencies() +{ + mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/" || exit + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/" || exit + + if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then + echo "Windows or Windows-like environment detected, skipping downloading of dependencies...: Alsa, Freetype, and CaCerts." + else + echo "Downloading required dependencies...: Alsa, Freetype, Freemarker, and CaCerts." + echo "Checking and download Alsa dependency" + checkingAndDownloadingAlsa + + if [[ "${BUILD_CONFIG[FREETYPE]}" == "true" ]] ; then + if [ -z "${BUILD_CONFIG[FREETYPE_DIRECTORY]}" ]; then + echo "Checking and download FreeType Font dependency" + checkingAndDownloadingFreeType + else + echo "" + echo "---> Skipping the process of checking and downloading the FreeType Font dependency, a pre-built version provided at ${BUILD_CONFIG[FREETYPE_DIRECTORY]} <---" + echo "" + fi + else + echo "Skipping Freetype" + fi + if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]]; then + echo "Checking and download Freemarker dependency" + checkingAndDownloadingFreemarker + fi + echo "Checking and download CaCerts dependency" + checkingAndDownloadCaCerts + fi +} + +function moveTmpToWorkspaceLocation { + if [ ! -z "${TMP_WORKSPACE}" ]; then + + echo "Relocating workspace from ${TMP_WORKSPACE} to ${ORIGINAL_WORKSPACE}" + + rm -rf "${ORIGINAL_WORKSPACE}" || true + + mkdir "${ORIGINAL_WORKSPACE}" + + # shellcheck disable=SC2086 + chmod 755 ${TMP_WORKSPACE}/workspace/* || true + chmod 755 "${ORIGINAL_WORKSPACE}" || true + + # shellcheck disable=SC2086 + cp -r ${TMP_WORKSPACE}/workspace/* "${ORIGINAL_WORKSPACE}" + rm -rf "${TMP_WORKSPACE}/workspace/" + rm -r "${TMP_WORKSPACE}" + + echo "Data at: ${ORIGINAL_WORKSPACE}" + ls -alh "${ORIGINAL_WORKSPACE}" + fi +} + + +relocateToTmpIfNeeded() +{ + if [ "${BUILD_CONFIG[TMP_SPACE_BUILD]}" == "true" ] + then + local tmpdir + tmpdir=$(mktemp -d 2>/dev/null || mktemp -d -t 'tmpdir') + export TMP_WORKSPACE="${tmpdir}" + export ORIGINAL_WORKSPACE="${BUILD_CONFIG[WORKSPACE_DIR]}" + + if [ -d "${ORIGINAL_WORKSPACE}" ] + then + cp -r "${BUILD_CONFIG[WORKSPACE_DIR]}" "${TMP_WORKSPACE}/workspace" + fi + BUILD_CONFIG[WORKSPACE_DIR]="${TMP_WORKSPACE}/workspace" + + trap moveTmpToWorkspaceLocation EXIT SIGINT SIGTERM + fi +} + +################################################################## + +function configureWorkspace() { + createWorkspace + relocateToTmpIfNeeded + checkoutAndCloneOpenJDKGitRepo + downloadingRequiredDependencies +} + diff --git a/sign.sh b/sign.sh new file mode 100755 index 000000000..be1d190ed --- /dev/null +++ b/sign.sh @@ -0,0 +1,137 @@ +#!/bin/bash + +################################################################################ +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +################################################################################ + +set -eu + +SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" + +# shellcheck source=sbin/common/config_init.sh +source "$SCRIPT_DIR/sbin/common/config_init.sh" + +# shellcheck source=sbin/common/constants.sh +source "$SCRIPT_DIR/sbin/common/constants.sh" + +# shellcheck source=sbin/common/common.sh +source "$SCRIPT_DIR/sbin/common/common.sh" + +ARCHIVE="" +SIGNING_CERTIFICATE="" +WORKSPACE=$(pwd) +TMP_DIR_NAME="tmp" +TMP_DIR="${WORKSPACE}/${TMP_DIR_NAME}/" + +checkSignConfiguration() { + if [[ "${OPERATING_SYSTEM}" == "windows" ]] ; then + if [ ! -f "${SIGNING_CERTIFICATE}" ] + then + echo "Could not find certificate at: ${SIGNING_CERTIFICATE}" + exit 1 + fi + + if [ -z "${SIGN_PASSWORD+x}" ] + then + echo "If signing is enabled on window you must set SIGN_PASSWORD" + exit 1 + fi + fi +} + +# Sign the built binary +signRelease() +{ + case "$OPERATING_SYSTEM" in + "windows") + echo "Signing Windows release" + signToolPath=${signToolPath:-"/cygdrive/c/Program Files/Microsoft SDKs/Windows/v7.1/Bin/signtool.exe"} + + # Sign .exe files + FILES=$(find . -type f -name '*.exe') + echo "$FILES" | while read -r f; + do + echo "Signing ${f}" + "$signToolPath" sign /f "${SIGNING_CERTIFICATE}" /p "$SIGN_PASSWORD" /fd SHA256 /t http://timestamp.verisign.com/scripts/timstamp.dll "$f"; + done + + # Sign .dll files + FILES=$(find . -type f -name '*.dll') + echo "$FILES" | while read -r f; + do + "$signToolPath" sign /f "${SIGNING_CERTIFICATE}" /p "$SIGN_PASSWORD" /fd SHA256 /t http://timestamp.verisign.com/scripts/timstamp.dll "$f"; + done + ;; + "mac"*) + echo "Signing OSX release" + # Login to KeyChain + # shellcheck disable=SC2046 + # shellcheck disable=SC2006 + security unlock-keychain -p `cat ~/.password` + # Sign all files with the executable permission bit set. + FILES=$(find "${TMP_DIR}" -perm +111 -type f || find "${TMP_DIR}" -perm /111 -type f) + echo "$FILES" | while read -r f; do codesign -s "Developer ID Application: London Jamocha Community CIC" "$f"; done + ;; + *) + echo "Skipping code signing as it's not supported on $OPERATING_SYSTEM" + ;; + esac +} + +function parseArguments() { + parseConfigurationArguments "$@" + + while [[ $# -gt 2 ]] ; do + shift; + done + + SIGNING_CERTIFICATE="$1"; + ARCHIVE="$2"; +} + +function extractArchive { + rm -rf "${TMP_DIR}" || true + mkdir "${TMP_DIR}" + if [[ "${OPERATING_SYSTEM}" == "windows" ]]; then + unzip "${ARCHIVE}" -d "${TMP_DIR}" + elif [[ "${OPERATING_SYSTEM}" == "mac" ]]; then + gunzip -dc "${ARCHIVE}" | tar xf - -C "${TMP_DIR}" + else + echo "could not detect archive type" + exit 1 + fi +} + +if [ "${OPERATING_SYSTEM}" != "windows" ] && [ "${OPERATING_SYSTEM}" != "mac" ]; then + echo "Skipping code signing as it's not supported on ${OPERATING_SYSTEM}" + exit 0; +fi + +configDefaults +parseArguments "$@" +extractArchive + +# shellcheck disable=SC2012 +jdkDir=$(find "${TMP_DIR}" ! -path "${TMP_DIR}" -type d -exec basename {} \; | head -n1) + +cd "${TMP_DIR}/${jdkDir}" || exit 1 +signRelease + +cd "${TMP_DIR}" +createOpenJDKArchive "${jdkDir}" "OpenJDK" +archiveExtension=$(getArchiveExtension) +signedArchive="${TMP_DIR}/OpenJDK${archiveExtension}" + +cd "${WORKSPACE}" +mv "${signedArchive}" "${ARCHIVE}" +rm -rf "${TMP_DIR}" diff --git a/sbin/signalhandler.sh b/signalhandler.sh similarity index 67% rename from sbin/signalhandler.sh rename to signalhandler.sh index e8f3ecdf5..58aa144e2 100755 --- a/sbin/signalhandler.sh +++ b/signalhandler.sh @@ -1,4 +1,6 @@ #!/bin/bash + +################################################################################ # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -12,18 +14,13 @@ # See the License for the specific language governing permissions and # limitations under the License. # - -# For terminal colors -esc=$(echo -en "\033") -error="${esc}[0;31m" #red -normal=$(echo -en "${esc}[m\017") - +################################################################################ exit_script() { - if [[ -z ${KEEP} ]] ; then - docker ps -a | awk '{ print $1,$2 }' | grep "$CONTAINER" | awk '{print $1 }' | xargs -I {} docker rm -f {} + if [[ -z ${KEEP_CONTAINER} ]] ; then + docker ps -a | awk '{ print $1,$2 }' | grep "$CONTAINER_NAME" | awk '{print $1 }' | xargs -I {} docker rm -f {} fi - echo "${error}Process exited${normal}" + echo "Process exited" trap - SIGINT SIGTERM # clear the trap kill -- -$$ # Sends SIGTERM to child/sub processes } From ae70ee66d319db6809b82e1c3c962a8ed1023797 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 10 Sep 2018 11:47:30 +0100 Subject: [PATCH 005/101] Point at correct repo (#487) --- pipelines/build/createJobFromTemplate.groovy | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pipelines/build/createJobFromTemplate.groovy b/pipelines/build/createJobFromTemplate.groovy index 353dee836..8fffd2c89 100644 --- a/pipelines/build/createJobFromTemplate.groovy +++ b/pipelines/build/createJobFromTemplate.groovy @@ -16,9 +16,9 @@ pipelineJob("$buildFolder/$JOB_NAME") { scm { git { remote { - url('https://github.com/johnoliver/openjdk-build.git') + url('https://github.com/AdoptOpenJDK/openjdk-build.git') } - branch('*/overhaul-build-scripts-6') + branch('*/new_build_scripts') extensions { cleanBeforeCheckout() } From 38474d5178c82d2f7689ac223191edde817c2902 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 10 Sep 2018 17:47:36 +0100 Subject: [PATCH 006/101] Merge build and test pipeline (#488) * Point at correct repo * Test bringing build and test together --- pipelines/build/BuildBaseFile.groovy | 191 ++++++------------ pipelines/build/createJobFromTemplate.groovy | 12 ++ pipelines/build/openjdk_build_pipeline.groovy | 155 +++++++++++++- 3 files changed, 223 insertions(+), 135 deletions(-) diff --git a/pipelines/build/BuildBaseFile.groovy b/pipelines/build/BuildBaseFile.groovy index c27964e45..cd6f6e73b 100644 --- a/pipelines/build/BuildBaseFile.groovy +++ b/pipelines/build/BuildBaseFile.groovy @@ -1,3 +1,4 @@ +import groovy.json.JsonOutput import groovy.json.JsonSlurper /* @@ -14,11 +15,22 @@ See the License for the specific language governing permissions and limitations under the License. */ -def toBuildParams(params) { +/** + * This file starts a high level job, it is called from openjdk8_pipeline.groovy, openjdk9_pipeline.groovy, openjdk10_pipeline.groovy. + * + * This: + * + * 1. Generate job for each configuration based on createJobFromTemplate.groovy + * 2. Execute job + * 3. Push generated artifacts to github + */ + +def toBuildParams(enableTests, params) { List buildParams = [] buildParams += [$class: 'LabelParameterValue', name: 'NODE_LABEL', label: params.get("NODE_LABEL")] + buildParams += string(name: "ENABLE_TESTS", value: "${enableTests}") params .findAll { it.key != 'NODE_LABEL' } @@ -27,7 +39,7 @@ def toBuildParams(params) { return buildParams } -def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { +static def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { String buildTag = "build" @@ -81,9 +93,12 @@ def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { def getJobConfigurations(javaToBuild, buildConfigurations, String osTarget, String releaseTag) { def jobConfigurations = [:] + //Parse config passed to jenkins job new JsonSlurper() .parseText(osTarget) .each { target -> + + //For each requested build type, generate a configuration if (buildConfigurations.containsKey(target.key)) { def configuration = buildConfigurations.get(target.key) target.value.each { variant -> @@ -105,29 +120,6 @@ static Integer getJavaVersionNumber(version) { return Integer.parseInt(matcher[0][1]) } -static def determineTestJobName(config, testType) { - - def variant - def number = getJavaVersionNumber(config.javaVersion) - - if (config.variant == "openj9") { - variant = "j9" - } else { - variant = "hs" - } - - def arch = config.arch - if (arch == "x64") { - arch = "x86-64" - } - - def os = config.os - if (os == "mac") { - os = "macos" - } - - return "openjdk${number}_${variant}_${testType}_${arch}_${os}" -} static def determineReleaseRepoVersion(javaToBuild) { def number = getJavaVersionNumber(javaToBuild) @@ -135,25 +127,48 @@ static def determineReleaseRepoVersion(javaToBuild) { return "jdk${number}" } -def getJobName(displayName, config) { +static def getJobName(displayName, config) { return "${config.javaVersion}-${displayName}" } -def getJobFolder(config) { +static def getJobFolder(config) { return "build-scripts/jobs/${config.javaVersion}" } -def createJob(jobName, jobFolder, config) { +// Generate a job from template at `createJobFromTemplate.groovy` +def createJob(jobName, jobFolder, config, enableTests) { def params = config.parameters.clone() params.put("JOB_NAME", jobName) params.put("JOB_FOLDER", jobFolder) + params.put("TEST_CONFIG", JsonOutput.prettyPrint(JsonOutput.toJson(config))) - create = jobDsl targets: "pipelines/build/createJobFromTemplate.groovy", ignoreExisting: true, additionalParameters: params + create = jobDsl targets: "pipelines/build/createJobFromTemplate.groovy", ignoreExisting: false, additionalParameters: params return create } +// Call job to push artifacts to github +def publishRelease(javaToBuild, releaseTag) { + def release = false + def tag = javaToBuild + if (releaseTag != null && releaseTag.length() > 0) { + release = true + tag = releaseTag + } + + node("master") { + stage("publish") { + build job: 'build-scripts/release/refactor_openjdk_release_tool', + parameters: [string(name: 'RELEASE', value: "${release}"), + string(name: 'TAG', value: tag), + string(name: 'UPSTREAM_JOB_NAME', value: env.JOB_NAME), + string(name: 'UPSTREAM_JOB_NUMBER', value: "${currentBuild.getNumber()}"), + string(name: 'VERSION', value: determineReleaseRepoVersion(javaToBuild))] + } + } +} + def doBuild(String javaToBuild, buildConfigurations, String osTarget, String enableTestsArg, String publishArg, String releaseTag) { if (releaseTag == null || releaseTag == "false") { @@ -162,7 +177,6 @@ def doBuild(String javaToBuild, buildConfigurations, String osTarget, String ena def jobConfigurations = getJobConfigurations(javaToBuild, buildConfigurations, osTarget, releaseTag) def jobs = [:] - def buildJobs = [:] def enableTests = enableTestsArg == "true" def publish = publishArg == "true" @@ -177,93 +191,36 @@ def doBuild(String javaToBuild, buildConfigurations, String osTarget, String ena jobConfigurations.each { configuration -> jobs[configuration.key] = { - def job def config = configuration.value + + // jdk10u-linux-x64-hotspot def jobTopName = getJobName(configuration.key, config) def jobFolder = getJobFolder(config) - def downstreamJob = "${jobFolder}/${jobTopName}"; + + // i.e jdk10u/job/jdk10u-linux-x64-hotspot + def downstreamJobName = "${jobFolder}/${jobTopName}" catchError { + // Execute build job for configuration i.e jdk10u/job/jdk10u-linux-x64-hotspot stage(configuration.key) { - createJob(jobTopName, jobFolder, config); - - job = build job: downstreamJob, propagate: false, parameters: toBuildParams(config.parameters) - buildJobs[configuration.key] = job - } + // generate job + createJob(jobTopName, jobFolder, config, enableTests) - if (enableTests && config.test) { - if (job.getResult() == 'SUCCESS') { - stage("test ${configuration.key}") { - def testStages = [:] - config.test.each { testType -> - testStages["${configuration.key}-${testType}"] = { - stage("test ${configuration.key} ${testType}") { - def jobName = determineTestJobName(config, testType) - catchError { - build job: jobName, - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${job.getNumber()}"), - string(name: 'UPSTREAM_JOB_NAME', value: downstreamJob)] - } - } - } - } - parallel testStages - } - } - } - - node('master') { - def downstreamJobName = downstreamJob - def jobWithReleaseArtifact = job - - if (config.os == "windows" || config.os == "mac") { - stage("sign ${configuration.key}") { - filter = "" - certificate = "" - - if (config.os == "windows") { - filter = "**/OpenJDK*_windows_*.zip" - certificate = "C:\\Users\\jenkins\\windows.p12" - - } else if (config.os == "mac") { - filter = "**/OpenJDK*_mac_*.tar.gz" - certificate = "\"Developer ID Application: London Jamocha Community CIC\"" - } - - signJob = build job: "build-scripts/release/sign_build", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${job.getNumber()}"), - string(name: 'UPSTREAM_JOB_NAME', value: downstreamJob), - string(name: 'OPERATING_SYSTEM', value: "${config.os}"), - string(name: 'FILTER', value: "${filter}"), - string(name: 'CERTIFICATE', value: "${certificate}"), - [$class: 'LabelParameterValue', name: 'NODE_LABEL', label: "${config.os}&&build"], - ] - downstreamJobName = "build-scripts/release/sign_build" - jobWithReleaseArtifact = signJob - } - } + def downstreamJob = build job: downstreamJobName, propagate: false, parameters: toBuildParams(enableTests, config.parameters) + node("master") { + sh "rm target/${config.os}/${config.arch}/${config.variant}/* || true" - stage("archive ${configuration.key}") { - if (jobWithReleaseArtifact.getResult() == 'SUCCESS') { - currentBuild.result = 'SUCCESS' - sh "rm target/${config.os}/${config.arch}/${config.variant}/* || true" - - copyArtifacts( - projectName: downstreamJobName, - selector: specific("${jobWithReleaseArtifact.getNumber()}"), - filter: 'workspace/target/*', - fingerprintArtifacts: true, - target: "target/${config.os}/${config.arch}/${config.variant}/", - flatten: true) - - - sh 'for file in $(ls target/*/*/*/*.tar.gz target/*/*/*/*.zip); do sha256sum "$file" > $file.sha256.txt ; done' - archiveArtifacts artifacts: "target/${config.os}/${config.arch}/${config.variant}/*" - } + copyArtifacts( + projectName: downstreamJobName, + selector: specific("${downstreamJob.getNumber()}"), + filter: 'workspace/target/*', + fingerprintArtifacts: true, + target: "target/${config.os}/${config.arch}/${config.variant}/", + flatten: true) + sh 'for file in $(ls target/*/*/*/*.tar.gz target/*/*/*/*.zip); do sha256sum "$file" > $file.sha256.txt ; done' + archiveArtifacts artifacts: "target/${config.os}/${config.arch}/${config.variant}/*" } } } @@ -272,25 +229,11 @@ def doBuild(String javaToBuild, buildConfigurations, String osTarget, String ena parallel jobs + // publish to github if needed if (publish) { - def release = false - def tag = javaToBuild - if (releaseTag != null && releaseTag.length() > 0) { - release = true - tag = releaseTag - } - - node("master") { - stage("publish") { - build job: 'build-scripts/release/refactor_openjdk_release_tool', - parameters: [string(name: 'RELEASE', value: "${release}"), - string(name: 'TAG', value: tag), - string(name: 'UPSTREAM_JOB_NAME', value: env.JOB_NAME), - string(name: 'UPSTREAM_JOB_NUMBER', value: "${currentBuild.getNumber()}"), - string(name: 'VERSION', value: determineReleaseRepoVersion(javaToBuild))] - } - } + publishRelease(javaToBuild, releaseTag) } } + return this \ No newline at end of file diff --git a/pipelines/build/createJobFromTemplate.groovy b/pipelines/build/createJobFromTemplate.groovy index 8fffd2c89..febe95535 100644 --- a/pipelines/build/createJobFromTemplate.groovy +++ b/pipelines/build/createJobFromTemplate.groovy @@ -1,9 +1,19 @@ +/** + * A template that defines a build job. + * + * This mostly is just a wrapper to call the openjdk_build_pipeline.groovy script that defines the majority of + * what a pipeline job does + */ + String buildFolder = "$JOB_FOLDER" if (!binding.hasVariable('JDK_BOOT_VERSION')) JDK_BOOT_VERSION = "" if (!binding.hasVariable('CONFIGURE_ARGS')) CONFIGURE_ARGS = "" if (!binding.hasVariable('BUILD_ARGS')) BUILD_ARGS = "" if (!binding.hasVariable('ADDITIONAL_FILE_NAME_TAG')) ADDITIONAL_FILE_NAME_TAG = "" +if (!binding.hasVariable('TEST_CONFIG')) TEST_CONFIG = "" +if (!binding.hasVariable('ENABLE_TESTS')) ENABLE_TESTS = "false" + folder(buildFolder) { description 'Automatically generated build jobs.' @@ -47,5 +57,7 @@ pipelineJob("$buildFolder/$JOB_NAME") { stringParam('VARIANT', "$VARIANT") stringParam('TARGET_OS', "$TARGET_OS") stringParam('ADDITIONAL_FILE_NAME_TAG', "$ADDITIONAL_FILE_NAME_TAG") + stringParam('ENABLE_TESTS', "$ENABLE_TESTS") + textParam('TEST_CONFIG', "$TEST_CONFIG") } } \ No newline at end of file diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index 4024eb0c6..988228e4c 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -11,18 +11,151 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ +import groovy.json.JsonSlurper -node(NODE_LABEL) { - checkout scm - - success = false - try { - sh "./build-farm/make-adopt-build-farm.sh" - archiveArtifacts artifacts: "workspace/target/*" - success = true - } catch (Exception e) { - success = false - currentBuild.result = 'FAILURE' + +/** + * This file is a template for running a build for a given configuration + * A configuration is for example jdk10u-mac-x64-hotspot. + * + * This file is referenced by the pipeline template createJobFromTemplate.groovy + * + * A pipeline looks like: + * 1. Check out and build JDK by calling build-farm/make-adopt-build-farm.sh + * 2. Archive artifacts created by build + * 3. Run all tests defined in the configuration + * 4. Sign artifacts if needed and re-archive + * + */ + +def getJavaVersionNumber(version) { + // version should be something like "jdk8u" + def matcher = (version =~ /(\d+)/) + return Integer.parseInt(matcher[0][1]) +} + +def determineTestJobName(config, testType) { + + def variant + def number = getJavaVersionNumber(config.javaVersion) + + if (config.variant == "openj9") { + variant = "j9" + } else { + variant = "hs" + } + + def arch = config.arch + if (arch == "x64") { + arch = "x86-64" + } + + def os = config.os + if (os == "mac") { + os = "macos" + } + + return "openjdk${number}_${variant}_${testType}_${arch}_${os}" +} + +def runTests(config) { + def testStages = [:] + + config.test.each { testType -> + // For each requested test, i.e 'openjdktest', 'systemtest', 'perftest', 'externaltest', call test job + println "Running test: ${testType}}" + testStages["${testType}"] = { + stage("${testType}") { + + // example jobName: openjdk10_hs_externaltest_x86-64_linux + def jobName = determineTestJobName(config, testType) + catchError { + build job: jobName, + propagate: false, + parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${env.BUILD_NUMBER}"), + string(name: 'UPSTREAM_JOB_NAME', value: "${env.JOB_NAME}")] + } + } + } + } + return testStages +} + + +def sign(config) { + // Sign and archive jobs if needed + if (config.os == "windows" || config.os == "mac") { + node('master') { + stage("sign") { + def filter = "" + def certificate = "" + + if (config.os == "windows") { + filter = "**/OpenJDK*_windows_*.zip" + certificate = "C:\\Users\\jenkins\\windows.p12" + + } else if (config.os == "mac") { + filter = "**/OpenJDK*_mac_*.tar.gz" + certificate = "\"Developer ID Application: London Jamocha Community CIC\"" + } + + def signJob = build job: "build-scripts/release/sign_build", + propagate: true, + parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${env.BUILD_NUMBER}"), + string(name: 'UPSTREAM_JOB_NAME', value: "${env.JOB_NAME}"), + string(name: 'OPERATING_SYSTEM', value: "${config.os}"), + string(name: 'FILTER', value: "${filter}"), + string(name: 'CERTIFICATE', value: "${certificate}"), + [$class: 'LabelParameterValue', name: 'NODE_LABEL', label: "${config.os}&&build"], + ] + + //Copy signed artifact back and rearchive + sh "rm workspace/target/* || true" + + copyArtifacts( + projectName: "build-scripts/release/sign_build", + selector: specific("${signJob.getNumber()}"), + filter: 'workspace/target/*', + fingerprintArtifacts: true, + target: "workspace/target/", + flatten: true) + + sh 'for file in $(ls workspace/target/*.tar.gz workspace/target/*.zip); do sha256sum "$file" > $file.sha256.txt ; done' + archiveArtifacts artifacts: "workspace/target/*" + } + } + } +} + +try { + def config = new JsonSlurper().parseText("${TEST_CONFIG}") + println "Executing tests: ${config}" + println "Build num: ${env.BUILD_NUMBER}" + + def enableTests = ENABLE_TESTS == "true" + + stage("build") { + node(NODE_LABEL) { + checkout scm + + sh "./build-farm/make-adopt-build-farm.sh" + archiveArtifacts artifacts: "workspace/target/*" + } } + + if (enableTests) { + try { + testStages = runTests(config) + parallel testStages + } catch (Exception e) { + println "Failed test: ${e}" + } + } + + // Sign and archive jobs if needed + sign(config) + +} catch (Exception e) { + currentBuild.result = 'FAILURE' } From e9ab85f2e1099b220ef2bf84d2185e7a32221ecc Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 11 Sep 2018 11:23:43 +0100 Subject: [PATCH 007/101] Remove old builds (#490) * Point at correct repo * Test bringing build and test together * Make copying artifact conditional on build success * Remove old code --- pipelines/build/BuildBaseFile.groovy | 32 ++++++----- pipelines/openjdk10_nightly_pipeline.groovy | 55 ------------------ .../openjdk10_openj9_nightly_pipeline.groovy | 54 ------------------ .../openjdk10_openj9_release_pipeline.groovy | 56 ------------------ pipelines/openjdk10_release_pipeline.groovy | 57 ------------------- pipelines/openjdk8_nightly_pipeline.groovy | 55 ------------------ .../openjdk8_openj9_nightly_pipeline.groovy | 56 ------------------ .../openjdk8_openj9_release_pipeline.groovy | 53 ----------------- pipelines/openjdk8_release_pipeline.groovy | 56 ------------------ pipelines/openjdk9_nightly_pipeline.groovy | 56 ------------------ .../openjdk9_openj9_nightly_pipeline.groovy | 53 ----------------- .../openjdk9_openj9_release_pipeline.groovy | 55 ------------------ .../openjdk_amber_nightly_pipeline.groovy | 50 ---------------- 13 files changed, 19 insertions(+), 669 deletions(-) delete mode 100644 pipelines/openjdk10_nightly_pipeline.groovy delete mode 100644 pipelines/openjdk10_openj9_nightly_pipeline.groovy delete mode 100644 pipelines/openjdk10_openj9_release_pipeline.groovy delete mode 100644 pipelines/openjdk10_release_pipeline.groovy delete mode 100644 pipelines/openjdk8_nightly_pipeline.groovy delete mode 100644 pipelines/openjdk8_openj9_nightly_pipeline.groovy delete mode 100644 pipelines/openjdk8_openj9_release_pipeline.groovy delete mode 100644 pipelines/openjdk8_release_pipeline.groovy delete mode 100644 pipelines/openjdk9_nightly_pipeline.groovy delete mode 100644 pipelines/openjdk9_openj9_nightly_pipeline.groovy delete mode 100644 pipelines/openjdk9_openj9_release_pipeline.groovy delete mode 100644 pipelines/openjdk_amber_nightly_pipeline.groovy diff --git a/pipelines/build/BuildBaseFile.groovy b/pipelines/build/BuildBaseFile.groovy index cd6f6e73b..941fda87b 100644 --- a/pipelines/build/BuildBaseFile.groovy +++ b/pipelines/build/BuildBaseFile.groovy @@ -206,21 +206,27 @@ def doBuild(String javaToBuild, buildConfigurations, String osTarget, String ena // generate job createJob(jobTopName, jobFolder, config, enableTests) + // execute build def downstreamJob = build job: downstreamJobName, propagate: false, parameters: toBuildParams(enableTests, config.parameters) - node("master") { - sh "rm target/${config.os}/${config.arch}/${config.variant}/* || true" - - copyArtifacts( - projectName: downstreamJobName, - selector: specific("${downstreamJob.getNumber()}"), - filter: 'workspace/target/*', - fingerprintArtifacts: true, - target: "target/${config.os}/${config.arch}/${config.variant}/", - flatten: true) - - sh 'for file in $(ls target/*/*/*/*.tar.gz target/*/*/*/*.zip); do sha256sum "$file" > $file.sha256.txt ; done' - archiveArtifacts artifacts: "target/${config.os}/${config.arch}/${config.variant}/*" + if (downstreamJob.getResult() == 'SUCCESS') { + // copy artifacts from build + node("master") { + catchError { + sh "rm target/${config.os}/${config.arch}/${config.variant}/* || true" + + copyArtifacts( + projectName: downstreamJobName, + selector: specific("${downstreamJob.getNumber()}"), + filter: 'workspace/target/*', + fingerprintArtifacts: true, + target: "target/${config.os}/${config.arch}/${config.variant}/", + flatten: true) + + sh 'for file in $(ls target/*/*/*/*.tar.gz target/*/*/*/*.zip); do sha256sum "$file" > $file.sha256.txt ; done' + archiveArtifacts artifacts: "target/${config.os}/${config.arch}/${config.variant}/*" + } + } } } } diff --git a/pipelines/openjdk10_nightly_pipeline.groovy b/pipelines/openjdk10_nightly_pipeline.groovy deleted file mode 100644 index 2f62e2c61..000000000 --- a/pipelines/openjdk10_nightly_pipeline.groovy +++ /dev/null @@ -1,55 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'arm32','aarch64'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['arm32'] = [test:false, ArchOSs:'arm32_linux'] -buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk10_build_${archOS}", - parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk10_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk10_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk10'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk10_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs diff --git a/pipelines/openjdk10_openj9_nightly_pipeline.groovy b/pipelines/openjdk10_openj9_nightly_pipeline.groovy deleted file mode 100644 index d53536ed4..000000000 --- a/pipelines/openjdk10_openj9_nightly_pipeline.groovy +++ /dev/null @@ -1,54 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -def typeTests = ['openjdktest', 'systemtest'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk10_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk10_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_openj9_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk10_openj9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_openj9_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk10-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk10_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs \ No newline at end of file diff --git a/pipelines/openjdk10_openj9_release_pipeline.groovy b/pipelines/openjdk10_openj9_release_pipeline.groovy deleted file mode 100644 index 0e01bc0e2..000000000 --- a/pipelines/openjdk10_openj9_release_pipeline.groovy +++ /dev/null @@ -1,56 +0,0 @@ -println "building ${JDK_TAG}" - -def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -def typeTests = ['openjdktest', 'systemtest'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk10_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk10_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_openj9_build_${archOS}")] - } - } - } - } -} -parallel jobs - -stage('checksums') { - checksumJob = build job: 'openjdk10_openj9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_openj9_build_${archOS}"), - string(name: 'PRODUCT', value: 'releases')] -} -stage('publish releases') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'releases'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk10-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk10_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] -} diff --git a/pipelines/openjdk10_release_pipeline.groovy b/pipelines/openjdk10_release_pipeline.groovy deleted file mode 100644 index 4f0504c35..000000000 --- a/pipelines/openjdk10_release_pipeline.groovy +++ /dev/null @@ -1,57 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'arm32', 'aarch64'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['arm32'] = [test:false, ArchOSs:'arm32_linux'] -buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - stage('build') { - buildJob = build job: "openjdk10_build_${archOS}", - parameters: [string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - } - if (buildMaps[platform].test) { - buildMaps[platform].test.each { - typeTests.each { - build job:"openjdk10_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJob.getNumber()}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk10_build_${archOS}")] - } - } - } - } -} -parallel jobs - -def checksumJob -stage('checksums') { - checksumJob = build job: 'openjdk10_build_checksum', - parameters: [string(name: 'PRODUCT', value: 'releases')] -} -stage('installers') { - build job: 'openjdk10_build_installer', parameters: [string(name: 'VERSION', value: "${JDK_VERSION}")] -} -stage('publish release') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'releases'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk10'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk10_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] -} diff --git a/pipelines/openjdk8_nightly_pipeline.groovy b/pipelines/openjdk8_nightly_pipeline.groovy deleted file mode 100644 index 796af7ae8..000000000 --- a/pipelines/openjdk8_nightly_pipeline.groovy +++ /dev/null @@ -1,55 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'aarch64'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'perftest', 'externaltest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk8_build_${archOS}", - parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk8_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk8_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: 'jdk8u181-b13'), - string(name: 'VERSION', value: 'jdk8'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk8_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs - diff --git a/pipelines/openjdk8_openj9_nightly_pipeline.groovy b/pipelines/openjdk8_openj9_nightly_pipeline.groovy deleted file mode 100644 index 5be4441f5..000000000 --- a/pipelines/openjdk8_openj9_nightly_pipeline.groovy +++ /dev/null @@ -1,56 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', 'Windows', 'Windows32', 'LinuxXL'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'perftest', 'externaltest'], ArchOSs:'x86-64_linux'] -buildMaps['LinuxXL'] = [test:['openjdktest'], ArchOSs:'x86-64_linux_largeHeap'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Windows32'] = [test:['openjdktest'], ArchOSs:'x86-32_windows'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk8_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk8_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_openj9_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk8_openj9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_openj9_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: 'jdk8u181-b13'), - string(name: 'VERSION', value: 'jdk8-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk8_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs - diff --git a/pipelines/openjdk8_openj9_release_pipeline.groovy b/pipelines/openjdk8_openj9_release_pipeline.groovy deleted file mode 100644 index d0e75ba78..000000000 --- a/pipelines/openjdk8_openj9_release_pipeline.groovy +++ /dev/null @@ -1,53 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', 'Windows', 'Windows32', 'LinuxXL'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] -buildMaps['LinuxXL'] = [test:['openjdktest'], ArchOSs:'x86-64_linux_largeHeap'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Windows32'] = [test:['openjdktest'], ArchOSs:'x86-32_windows'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - stage('build') { - buildJob = build job: "openjdk8_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk8_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJob.getNumber()}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_openj9_build_${archOS}")] - } - } - } - } -} -parallel jobs - -def checksumJob -stage('checksums') { - checksumJob = build job: 'openjdk8_openj9_build_checksum', - parameters: [string(name: 'PRODUCT', value: 'releases')] -} -stage('publish release') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'releases'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk8-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk8_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] -} diff --git a/pipelines/openjdk8_release_pipeline.groovy b/pipelines/openjdk8_release_pipeline.groovy deleted file mode 100644 index efa825756..000000000 --- a/pipelines/openjdk8_release_pipeline.groovy +++ /dev/null @@ -1,56 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Mac', 'Windows', 'Linux', 'zLinux', 'ppc64le', 'AIX', 'aarch64'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Linux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - stage('build') { - buildJob = build job: "openjdk8_build_${archOS}", - parameters: [string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - } - if (buildMaps[platform].test) { - buildMaps[platform].test.each { - typeTests.each { - build job:"openjdk8_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJob.getNumber()}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk8_build_${archOS}")] - } - } - } - } -} -parallel jobs - -def checksumJob -stage('checksums') { - checksumJob = build job: 'openjdk8_build_checksum', - parameters: [string(name: 'PRODUCT', value: 'releases')] -} -stage('installers') { - build job: 'openjdk8_build_installer', parameters: [string(name: 'VERSION', value: "${JDK_VERSION}")] -} -stage('publish release') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'releases'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk8'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk8_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] -} diff --git a/pipelines/openjdk9_nightly_pipeline.groovy b/pipelines/openjdk9_nightly_pipeline.groovy deleted file mode 100644 index bdc81ff34..000000000 --- a/pipelines/openjdk9_nightly_pipeline.groovy +++ /dev/null @@ -1,56 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Mac', 'Linux', 'zLinux', 'ppc64le', 'Windows', 'AIX', 'arm32', 'aarch64'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Mac'] = [test:['openjdktest', 'systemtest'], ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['arm32'] = [test:false, ArchOSs:'arm32_linux'] -buildMaps['aarch64'] = [test:['openjdktest'], ArchOSs:'aarch64_linux'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk9_build_${archOS}", - parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk9_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: 'jdk-9+181'), - string(name: 'VERSION', value: 'jdk9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs diff --git a/pipelines/openjdk9_openj9_nightly_pipeline.groovy b/pipelines/openjdk9_openj9_nightly_pipeline.groovy deleted file mode 100644 index 00a8f9bcc..000000000 --- a/pipelines/openjdk9_openj9_nightly_pipeline.groovy +++ /dev/null @@ -1,53 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk9_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk9_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_openj9_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk9_openj9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_openj9_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: 'jdk-9.0.4+12'), - string(name: 'VERSION', value: 'jdk9-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk9_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs diff --git a/pipelines/openjdk9_openj9_release_pipeline.groovy b/pipelines/openjdk9_openj9_release_pipeline.groovy deleted file mode 100644 index 838264e23..000000000 --- a/pipelines/openjdk9_openj9_release_pipeline.groovy +++ /dev/null @@ -1,55 +0,0 @@ -println "building ${JDK_TAG}" - -def buildPlatforms = ['Linux', 'zLinux', 'ppc64le', 'AIX', "Windows"] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Linux'] = [test:['openjdktest', 'systemtest', 'externaltest'], ArchOSs:'x86-64_linux'] -buildMaps['zLinux'] = [test:['openjdktest', 'systemtest'], ArchOSs:'s390x_linux'] -buildMaps['ppc64le'] = [test:['openjdktest', 'systemtest'], ArchOSs:'ppc64le_linux'] -buildMaps['AIX'] = [test:false, ArchOSs:'ppc64_aix'] -buildMaps['Windows'] = [test:['openjdktest'], ArchOSs:'x86-64_windows'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk9_openj9_build_${archOS}", - parameters: [string(name: 'BRANCH', value: "$ALT_BRANCH"), - string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"openjdk9_j9_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_openj9_build_${archOS}")] - } - } - } - } -} -parallel jobs - -stage('checksums') { - checksumJob = build job: 'openjdk9_openj9_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk9_openj9_build_${archOS}"), - string(name: 'PRODUCT', value: 'releases')] -} -stage('publish release') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'releases'), - string(name: 'TAG', value: '${JDK_TAG}'), - string(name: 'VERSION', value: 'jdk9-openj9'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk9_openj9_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] -} diff --git a/pipelines/openjdk_amber_nightly_pipeline.groovy b/pipelines/openjdk_amber_nightly_pipeline.groovy deleted file mode 100644 index baaaf4170..000000000 --- a/pipelines/openjdk_amber_nightly_pipeline.groovy +++ /dev/null @@ -1,50 +0,0 @@ -println "building ${JDK_VERSION}" - -def buildPlatforms = ['Mac', 'Linux', 'Windows'] -def buildMaps = [:] -def PIPELINE_TIMESTAMP = new Date(currentBuild.startTimeInMillis).format("yyyyMMddHHmm") - -buildMaps['Mac'] = [test:['openjdktest'], ArchOSs:'x86-64_macos'] -buildMaps['Windows'] = [test:false, ArchOSs:'x86-64_windows'] -buildMaps['Linux'] = [test:['openjdktest'], ArchOSs:'x86-64_linux'] - -def jobs = [:] -for ( int i = 0; i < buildPlatforms.size(); i++ ) { - def index = i - def platform = buildPlatforms[index] - def archOS = buildMaps[platform].ArchOSs - jobs[platform] = { - def buildJob - def buildJobNum - def checksumJob - stage('build') { - buildJob = build job: "openjdk_amber_build_${archOS}", - parameters: [string(name: 'PIPELINE_TIMESTAMP', value: "${PIPELINE_TIMESTAMP}")] - buildJobNum = buildJob.getNumber() - } - if (buildMaps[platform].test) { - stage('test') { - buildMaps[platform].test.each { - build job:"amber-rsl_hs_${it}_${archOS}", - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk_amber_build_${archOS}")] - } - } - } - stage('checksums') { - checksumJob = build job: 'openjdk_amber_build_checksum', - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${buildJobNum}"), - string(name: 'UPSTREAM_JOB_NAME', value: "openjdk_amber_build_${archOS}")] - } - stage('publish nightly') { - build job: 'openjdk_release_tool', - parameters: [string(name: 'REPO', value: 'nightly'), - string(name: 'TAG', value: "${JDK_TAG}"), - string(name: 'VERSION', value: 'jdk-amber'), - string(name: 'CHECKSUM_JOB_NAME', value: "openjdk_amber_build_checksum"), - string(name: 'CHECKSUM_JOB_NUMBER', value: "${checksumJob.getNumber()}")] - } - } -} -parallel jobs From 6fba94b6542ca4c06520e43ec7721f02893051dc Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 11 Sep 2018 12:06:47 +0100 Subject: [PATCH 008/101] Update naming from review (#493) * Point at correct repo * Test bringing build and test together * Make copying artifact conditional on build success * Remove old code * Tidy from review * Keep consistent --- pipelines/build/BuildBaseFile.groovy | 20 ++++++++++---------- pipelines/build/openjdk10_pipeline.groovy | 2 +- pipelines/build/openjdk11_pipeline.groovy | 2 +- pipelines/build/openjdk9_pipeline.groovy | 2 +- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/pipelines/build/BuildBaseFile.groovy b/pipelines/build/BuildBaseFile.groovy index 941fda87b..f498b71b0 100644 --- a/pipelines/build/BuildBaseFile.groovy +++ b/pipelines/build/BuildBaseFile.groovy @@ -90,23 +90,23 @@ static def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { ] } -def getJobConfigurations(javaToBuild, buildConfigurations, String osTarget, String releaseTag) { +def getJobConfigurations(javaVersionToBuild, availableConfigurations, String targetConfigurations, String releaseTag) { def jobConfigurations = [:] //Parse config passed to jenkins job new JsonSlurper() - .parseText(osTarget) + .parseText(targetConfigurations) .each { target -> //For each requested build type, generate a configuration - if (buildConfigurations.containsKey(target.key)) { - def configuration = buildConfigurations.get(target.key) + if (availableConfigurations.containsKey(target.key)) { + def configuration = availableConfigurations.get(target.key) target.value.each { variant -> GString name = "${configuration.os}-${configuration.arch}-${variant}" if (configuration.containsKey('additionalFileNameTag')) { name += "-${configuration.additionalFileNameTag}" } - jobConfigurations[name] = buildConfiguration(javaToBuild, variant, configuration, releaseTag) + jobConfigurations[name] = buildConfiguration(javaVersionToBuild, variant, configuration, releaseTag) } } } @@ -169,21 +169,21 @@ def publishRelease(javaToBuild, releaseTag) { } } -def doBuild(String javaToBuild, buildConfigurations, String osTarget, String enableTestsArg, String publishArg, String releaseTag) { +def doBuild(String javaVersionToBuild, availableConfigurations, String targetConfigurations, String enableTestsArg, String publishArg, String releaseTag) { if (releaseTag == null || releaseTag == "false") { releaseTag = "" } - def jobConfigurations = getJobConfigurations(javaToBuild, buildConfigurations, osTarget, releaseTag) + def jobConfigurations = getJobConfigurations(javaVersionToBuild, availableConfigurations, targetConfigurations, releaseTag) def jobs = [:] def enableTests = enableTestsArg == "true" def publish = publishArg == "true" - echo "Java: ${javaToBuild}" - echo "OS: ${osTarget}" + echo "Java: ${javaVersionToBuild}" + echo "OS: ${targetConfigurations}" echo "Enable tests: ${enableTests}" echo "Publish: ${publish}" echo "ReleaseTag: ${releaseTag}" @@ -237,7 +237,7 @@ def doBuild(String javaToBuild, buildConfigurations, String osTarget, String ena // publish to github if needed if (publish) { - publishRelease(javaToBuild, releaseTag) + publishRelease(javaVersionToBuild, releaseTag) } } diff --git a/pipelines/build/openjdk10_pipeline.groovy b/pipelines/build/openjdk10_pipeline.groovy index bafc1f422..84e5cfc1b 100644 --- a/pipelines/build/openjdk10_pipeline.groovy +++ b/pipelines/build/openjdk10_pipeline.groovy @@ -91,5 +91,5 @@ def javaToBuild = "jdk10u" node ("master") { checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/BuildBaseFile.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, osTarget, enableTests, publish, releaseTag) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) } diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index 7d8929c82..c3d7138ab 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -89,5 +89,5 @@ def javaToBuild = "jdk11" node ("master") { checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/BuildBaseFile.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, osTarget, enableTests, publish, releaseTag) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) } diff --git a/pipelines/build/openjdk9_pipeline.groovy b/pipelines/build/openjdk9_pipeline.groovy index 5066c2812..4fedb158c 100644 --- a/pipelines/build/openjdk9_pipeline.groovy +++ b/pipelines/build/openjdk9_pipeline.groovy @@ -83,5 +83,5 @@ def javaToBuild = "jdk9u" node ("master") { checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/BuildBaseFile.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, osTarget, enableTests, publish, releaseTag) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) } From c19bbe7c06af67fb7848e6f2fe568585920a5716 Mon Sep 17 00:00:00 2001 From: Martijn Verburg Date: Tue, 11 Sep 2018 12:38:05 +0100 Subject: [PATCH 009/101] Fix PPCLE free type and C config (#494) --- build-farm/make-adopt-build-farm.sh | 2 -- build-farm/platform-specific-configurations/linux.sh | 7 +++---- pipelines/build/BuildBaseFile.groovy | 4 +++- pipelines/build/openjdk_build_pipeline.groovy | 1 - 4 files changed, 6 insertions(+), 8 deletions(-) diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index 124b12668..4a3e2e940 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -80,7 +80,6 @@ source "${PLATFORM_SCRIPT_DIR}/set-platform-specific-configurations.sh" # Set the file name JAVA_TO_BUILD_UPPERCASE=$(echo "${JAVA_TO_BUILD}" | tr '[:lower:]' '[:upper:]') - if [ ! -z "${ADDITIONAL_FILE_NAME_TAG}" ]; then FILENAME="Open${JAVA_TO_BUILD_UPPERCASE}_${ARCHITECTURE}_${OPERATING_SYSTEM}_${VARIANT}_${ADDITIONAL_FILE_NAME_TAG}_${TIMESTAMP}.${EXTENSION}" else @@ -91,4 +90,3 @@ echo "Filename will be: $FILENAME" # shellcheck disable=SC2086 bash "$PLATFORM_SCRIPT_DIR/../makejdk-any-platform.sh" --clean-git-repo --jdk-boot-dir "${JDK_BOOT_DIR}" --configure-args "${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" --target-file-name "${FILENAME}" ${TAG_OPTION} ${OPTIONS} ${BUILD_ARGS} ${VARIANT_ARG} "${JAVA_TO_BUILD}" - diff --git a/build-farm/platform-specific-configurations/linux.sh b/build-farm/platform-specific-configurations/linux.sh index 0b50caf78..f13c10da0 100755 --- a/build-farm/platform-specific-configurations/linux.sh +++ b/build-farm/platform-specific-configurations/linux.sh @@ -38,7 +38,7 @@ then then export PATH="/usr/bin:$PATH" - if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] then if which g++-4.8; then export CC=gcc-4.8 @@ -58,7 +58,8 @@ then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="--with-jobs=4 --with-memory-size=2000" fi -if [ "${ARCHITECTURE}" == "aarch64" ] +# Skip Building Freetype for certain platforms +if [ "${ARCHITECTURE}" == "aarch64" ] || [ "${ARCHITECTURE}" == "ppc64le" ] then export BUILD_ARGS="${BUILD_ARGS} --skip-freetype" fi @@ -75,8 +76,6 @@ then fi export JDK_BOOT_DIR=$JDK9_BOOT_DIR - export CC=gcc-4.8 - export CXX=g++-4.8 fi fi diff --git a/pipelines/build/BuildBaseFile.groovy b/pipelines/build/BuildBaseFile.groovy index f498b71b0..4218f3475 100644 --- a/pipelines/build/BuildBaseFile.groovy +++ b/pipelines/build/BuildBaseFile.groovy @@ -223,7 +223,10 @@ def doBuild(String javaVersionToBuild, availableConfigurations, String targetCon target: "target/${config.os}/${config.arch}/${config.variant}/", flatten: true) + // Checksum sh 'for file in $(ls target/*/*/*/*.tar.gz target/*/*/*/*.zip); do sha256sum "$file" > $file.sha256.txt ; done' + + // Archive in Jenkins archiveArtifacts artifacts: "target/${config.os}/${config.arch}/${config.variant}/*" } } @@ -241,5 +244,4 @@ def doBuild(String javaVersionToBuild, availableConfigurations, String targetCon } } - return this \ No newline at end of file diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index 988228e4c..f17f645c0 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -81,7 +81,6 @@ def runTests(config) { return testStages } - def sign(config) { // Sign and archive jobs if needed if (config.os == "windows" || config.os == "mac") { From 6f52b77f4c3f404d08951a1113fdfe220b528739 Mon Sep 17 00:00:00 2001 From: Martijn Verburg Date: Tue, 11 Sep 2018 13:07:04 +0100 Subject: [PATCH 010/101] ppcle for openj9 needs an ubuntu to run (#495) * Fix PPCLE free type and C config * ppcle for openj9 needs an ubuntu to run --- pipelines/build/openjdk10_pipeline.groovy | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/pipelines/build/openjdk10_pipeline.groovy b/pipelines/build/openjdk10_pipeline.groovy index 84e5cfc1b..f97739cc1 100644 --- a/pipelines/build/openjdk10_pipeline.groovy +++ b/pipelines/build/openjdk10_pipeline.groovy @@ -51,7 +51,10 @@ def buildConfigurations = [ ppc64leLinux : [ os : 'linux', arch : 'ppc64le', - additionalNodeLabels: 'centos7', + additionalNodeLabels: [ + hotspot: 'centos7', + openj9: 'ubuntu' + ], test : ['openjdktest', 'systemtest'] ], From 1a19c62a877edc9757d07d365531a35e6e366fc9 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 11 Sep 2018 13:24:42 +0100 Subject: [PATCH 011/101] Dont use rpm on ubuntu (#496) --- configureBuild.sh | 8 ++++++-- sbin/common/config_init.sh | 3 +++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/configureBuild.sh b/configureBuild.sh index 18086eebc..dfc9b28b7 100755 --- a/configureBuild.sh +++ b/configureBuild.sh @@ -182,8 +182,12 @@ processArgumentsforSpecificArchitectures() { "ppc64le") jvm_variant=server build_full_name=linux-ppc64-normal-${jvm_variant}-release - # shellcheck disable=SC1083 - BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]=${BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]:="--build=$(rpm --eval %{_host})"} + + if which rpm; then + # shellcheck disable=SC1083 + BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]=${BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]:="--build=$(rpm --eval %{_host})"} + fi + ;; "armv7l") diff --git a/sbin/common/config_init.sh b/sbin/common/config_init.sh index fd1f13e63..21f6cd23f 100755 --- a/sbin/common/config_init.sh +++ b/sbin/common/config_init.sh @@ -185,6 +185,9 @@ function parseConfigurationArguments() { "--freetype-dir" | "-f" ) BUILD_CONFIG[FREETYPE_DIRECTORY]="$1"; shift;; + "--freetype-build-param" ) + BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]="$1"; shift;; + "--skip-freetype" | "-F" ) BUILD_CONFIG[FREETYPE]=false;; From f829718894212704626fdd56ab4350a41ae1e0b3 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 11 Sep 2018 14:14:27 +0100 Subject: [PATCH 012/101] Fix s390x nodes (#497) --- pipelines/build/BuildBaseFile.groovy | 7 +++++-- pipelines/build/openjdk10_pipeline.groovy | 4 +++- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/pipelines/build/BuildBaseFile.groovy b/pipelines/build/BuildBaseFile.groovy index 4218f3475..b29366d84 100644 --- a/pipelines/build/BuildBaseFile.groovy +++ b/pipelines/build/BuildBaseFile.groovy @@ -49,7 +49,7 @@ static def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { buildTag = "(buildj9||build)&&openj9" } - def additionalNodeLabels + def additionalNodeLabels = null if (configuration.containsKey("additionalNodeLabels")) { // hack as jenkins sandbox wont allow instanceof if ("java.util.LinkedHashMap" == configuration.additionalNodeLabels.getClass().getName()) { @@ -58,9 +58,12 @@ static def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { additionalNodeLabels = configuration.additionalNodeLabels } + } + + if (additionalNodeLabels !== null) { additionalNodeLabels = "${additionalNodeLabels}&&${buildTag}" } else { - additionalNodeLabels = buildTag + additionalNodeLabels = "${buildTag}" } def buildParams = [ diff --git a/pipelines/build/openjdk10_pipeline.groovy b/pipelines/build/openjdk10_pipeline.groovy index f97739cc1..76213e1d8 100644 --- a/pipelines/build/openjdk10_pipeline.groovy +++ b/pipelines/build/openjdk10_pipeline.groovy @@ -44,7 +44,9 @@ def buildConfigurations = [ s390xLinux : [ os : 'linux', arch : 's390x', - additionalNodeLabels: 'ubuntu', + additionalNodeLabels: [ + hotspot: 'rhel7' + ], test : ['openjdktest', 'systemtest'] ], From 2cb1603e9d9f63d6f305e132b8cac0c7be84e8eb Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 11 Sep 2018 15:13:37 +0100 Subject: [PATCH 013/101] Fix typo (#498) --- pipelines/build/BuildBaseFile.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipelines/build/BuildBaseFile.groovy b/pipelines/build/BuildBaseFile.groovy index b29366d84..eb69fd812 100644 --- a/pipelines/build/BuildBaseFile.groovy +++ b/pipelines/build/BuildBaseFile.groovy @@ -60,7 +60,7 @@ static def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { } - if (additionalNodeLabels !== null) { + if (additionalNodeLabels != null) { additionalNodeLabels = "${additionalNodeLabels}&&${buildTag}" } else { additionalNodeLabels = "${buildTag}" From 9773a5c9e0982cb1708dca91f95a6ce4bcdd8fb5 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Thu, 13 Sep 2018 12:27:47 +0100 Subject: [PATCH 014/101] Add docs (#503) * Fix typo * Ping ppc64le hotspot * Check if tests are required * Add docs * Fix link * add build status * Update status * Tidy * Fix shellcheck * Fix links * Fix from review --- CHANGELOG.md | 2 +- README.md | 30 +- configureBuild.sh | 2 +- docs/build.md | 63 + docs/generateBuildMatrix.sh | 30 + ...doptOpenJDK_Build_Script_Relationships.png | Bin docs/images/sequence.dia | 2170 +++++++++++++++++ docs/images/sequence.svg | 256 ++ ...BaseFile.groovy => build_base_file.groovy} | 6 +- ...groovy => create_job_from_template.groovy} | 2 +- pipelines/build/openjdk10_pipeline.groovy | 15 +- pipelines/build/openjdk11_pipeline.groovy | 12 +- pipelines/build/openjdk8_pipeline.groovy | 34 +- pipelines/build/openjdk9_pipeline.groovy | 40 +- pipelines/build/openjdk_build_pipeline.groovy | 4 +- 15 files changed, 2607 insertions(+), 59 deletions(-) create mode 100644 docs/build.md create mode 100755 docs/generateBuildMatrix.sh rename {images => docs/images}/AdoptOpenJDK_Build_Script_Relationships.png (100%) create mode 100644 docs/images/sequence.dia create mode 100644 docs/images/sequence.svg rename pipelines/build/{BuildBaseFile.groovy => build_base_file.groovy} (96%) rename pipelines/build/{createJobFromTemplate.groovy => create_job_from_template.groovy} (95%) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1ed1cc95a..7ed393513 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -97,7 +97,7 @@ Sets the default environment variables for specific platforms that are currently set in individual jobs. This allows us to now track and version these variables. 1. New _build-farm/sign-releases.sh added for the new AdoptOpenJDK Build Farm jenkins pipeline to code sign Adopt OpenJDK binaries (Mac and Windows for now). -1. _pipelines/build/BuildBaseFile.groovy_ added. This co-ordinates the various +1. _pipelines/build/build_base_file.groovy_ added. This co-ordinates the various pipeline builds. 1. _pipelines/build/openjdk\_build\_pipeline.groovy_ added. This forms the base pipeline code for each build. diff --git a/README.md b/README.md index bb4f81a34..866f416e8 100644 --- a/README.md +++ b/README.md @@ -196,7 +196,7 @@ Example usage: ### Script Relationships -![Build Variant Workflow](images/AdoptOpenJDK_Build_Script_Relationships.png) +![Build Variant Workflow](docs/images/AdoptOpenJDK_Build_Script_Relationships.png) The main script to build OpenJDK is `makejdk-any-platform.sh`, which itself uses and/or calls `configureBuild.sh`, `docker-build.sh` and/or `native-build.sh`. @@ -277,3 +277,31 @@ to your local platform and then build (Adopt) OpenJDK and place the result in These scripts default to using AdoptOpenJDK as the OpenJDK source repo to build from, but you can override this with the `-r` flag. + + +# Build farm + +The documentation for files that define the build farm builds can be found at [Build Farm Definition](/docs/build.md). + +# Build status + +Table generated with `generateBuildMatrix.sh` + +| Platform | Java 8 | Java 9 | Java 10 | Java 11 | +| ------------------------- | ------ | ------ | ------- | ------- | +| aix-ppc64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-hotspot) | +| aix-ppc64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-openj9) | +| linux-aarch64-hotspot | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-aarch64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-aarch64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-aarch64-hotspot) | +| linux-arm32-hotspot | N/A | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-arm32-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-arm32-hotspot) | +| linux-arm-hotspot | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-arm-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-arm-hotspot) | N/A | +| linux-ppc64le-hotspot | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-ppc64le-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-ppc64le-hotspot) | +| linux-ppc64le-openj9 | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-ppc64le-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-ppc64le-openj9) | +| linux-s390x-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-s390x-hotspot) | +| linux-s390x-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-s390x-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-s390x-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-s390x-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-s390x-openj9) | +| linux-x64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-x64-hotspot) | +| linux-x64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-x64-openj9) | +| linux-x64-openj9-linuxXL | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-x64-openj9-linuxXL) | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-x64-openj9-linuxXL) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-x64-openj9-linuxXL) | +| mac-x64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-mac-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-mac-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-mac-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-mac-x64-hotspot) | +| windows-x64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-windows-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-windows-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-windows-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-windows-x64-hotspot) | +| windows-x64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-windows-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-windows-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-windows-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-windows-x64-openj9) | + diff --git a/configureBuild.sh b/configureBuild.sh index dfc9b28b7..cc6d60aa0 100755 --- a/configureBuild.sh +++ b/configureBuild.sh @@ -183,7 +183,7 @@ processArgumentsforSpecificArchitectures() { jvm_variant=server build_full_name=linux-ppc64-normal-${jvm_variant}-release - if which rpm; then + if [ "$(command -v rpm)" ]; then # shellcheck disable=SC1083 BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]=${BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]:="--build=$(rpm --eval %{_host})"} fi diff --git a/docs/build.md b/docs/build.md new file mode 100644 index 000000000..53036ebea --- /dev/null +++ b/docs/build.md @@ -0,0 +1,63 @@ + + +# Build + + +The build is organised as follows: + +## Top level pipeline jobs + +These live at https://ci.adoptopenjdk.net/job/build-scripts/. Currently these jobs are: + +* [openjdk8-pipeline](https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk8-pipeline/) +* [openjdk9-pipeline](https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk9-pipeline/) +* [openjdk10-pipeline](https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk10-pipeline/) +* [openjdk11-pipeline](https://ci.adoptopenjdk.net/job/build-scripts/job/openjdk11-pipeline/) + +** From here on in this documentation describes the 10 pipeline however it is the same for all Java versions. ** + +The high level sequence of each of these pipelines is shown in: + +![build sequence](images/sequence.svg "Build Sequence") + +This shows a build of `jdk10-mac-x64-hotspot` and `jdk10-linux-x64-hotspot`. The `jdk10-mac-x64-hotspot` build is expanded as +an example of how each build pipeline works, however a similar pipeline will be generated for every `jdk10---` +that was requested to be built. + +### Top level pipeline code + +The actions of the pipeline are defined by [openjdk10_pipeline.groovy](/pipelines/build/openjdk10_pipeline.groovy), which +calls [build_base_file.groovy](/pipelines/build/build_base_file.groovy). + +The actions of this script are: + +1. Read in build configuration that determines what configurations have been requested to be built. +1. For each requested configuration generate a configuration that defines such things as which nodes to run the build on, optional configuration added to [openjdk10_pipeline.groovy](/pipelines/build/openjdk10_pipeline.groovy) etc. +1. Generate a build job for each requested configuration, these live at [jdk10u](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/) +1. Execute all build jobs in parallel +1. Publish binaries to GitHub + + +#### Generated build jobs + +These are generated using the template at [create_job_from_template.groovy](/pipelines/build/create_job_from_template.groovy) +This defines a pipeline job that calls [openjdk_build_pipeline.groovy](/pipelines/build/openjdk_build_pipeline.groovy), this script defines the work of what a Build Pipeline does. + +## Build Pipelines + +Generated by [create_job_from_template.groovy](/pipelines/build/create_job_from_template.groovy), and its work defined in [openjdk_build_pipeline.groovy](/pipelines/build/openjdk_build_pipeline.groovy). + +The actions of this are: + +1. Execute build +1. Spawn requested test pipelines, these are defined in [openjdk10_pipeline.groovy](/pipelines/build/openjdk10_pipeline.groovy) +1. Call sign binaries job if needed +1. Archive file + +### Execute build + +This is kicked off by the Build Pipeline, this involves calling [make-adopt-build-farm.sh](/build-farm/make-adopt-build-farm.sh). +This script defines some platform specific configuration such as operating system and architecture specific config defined in [platform-specific-configurations](/build-farm/platform-specific-configurations). +Then it calls [make-adopt-build-farm.sh](/build-farm/make-adopt-build-farm.sh) to finally invoke the build. + + diff --git a/docs/generateBuildMatrix.sh b/docs/generateBuildMatrix.sh new file mode 100755 index 000000000..53109d41a --- /dev/null +++ b/docs/generateBuildMatrix.sh @@ -0,0 +1,30 @@ +#!/bin/bash + +# Generates markdown table of build job status + +echo "| Platform | Java 8 | Java 9 | Java 10 | Java 11 |" +echo "| ------------------------- | ------ | ------ | ------- | ------- |" + +rm "/tmp/build.txt" +for i in "8u" "9u" "10u" "11"; +do + curl -s "https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk${i}/" | egrep -o "job/jdk${i}u?-[^\/]+" >> "/tmp/build.txt" +done + +cat "/tmp/build.txt" | cut -d'/' -f2 | sed -r 's/jdk[0-9]+u?\-//g' | sort | uniq | while read buildName; +do + # buildName should be of the form: aix-ppc64-hotspot + echo -n "| ${buildName} | " + for i in "8u" "9u" "10u" "11"; + do + code=$(curl -s -o /dev/null -w "%{http_code}" "https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk${i}/jdk${i}-${buildName}") + if [ $code = 200 ]; then + echo -n "[![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk${i}/jdk${i}-${buildName})](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk${i}/job/jdk${i}-${buildName})" + else + echo -n "N/A" + fi + + echo -n " | " + done + echo "" +done \ No newline at end of file diff --git a/images/AdoptOpenJDK_Build_Script_Relationships.png b/docs/images/AdoptOpenJDK_Build_Script_Relationships.png similarity index 100% rename from images/AdoptOpenJDK_Build_Script_Relationships.png rename to docs/images/AdoptOpenJDK_Build_Script_Relationships.png diff --git a/docs/images/sequence.dia b/docs/images/sequence.dia new file mode 100644 index 000000000..a3386f3a9 --- /dev/null +++ b/docs/images/sequence.dia @@ -0,0 +1,2170 @@ + + + + + + + + + + + + + #A4# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #OpenJDK10u +Build Job# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #Start Build# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #jdk10u-mac-x64-hotspot +Build Job# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + #Generate Job# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + #Execute Build Pipeline# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #Build# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #openjdktest# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #systemtest# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #sign# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #Publish +to github# + + + + + + + + + + + + + + + + + + + + ## + + + ## + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + #jdk10-mac-x64-hotspot +Pipeline Job# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + #linux-x64-hotspot +Pipeline Job# + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + ## + + + + + + + + + + + + + + + + + + + + + + + #Generate and Execute# + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/images/sequence.svg b/docs/images/sequence.svg new file mode 100644 index 000000000..1890b7423 --- /dev/null +++ b/docs/images/sequence.svg @@ -0,0 +1,256 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + OpenJDK10u + Build Job + + + + + + Start Build + + + + + + jdk10u-mac-x64-hotspot + Build Job + + + + + + + + + + + Generate Job + + + + + + + + + + + + + + Execute Build Pipeline + + + + + + Build + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + openjdktest + + + + + + + + + + + + + + + + + + + + + + + + + + + systemtest + + + + + + + + + + + + + + + + + + + + + + + + + + + sign + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Publish + to github + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + jdk10-mac-x64-hotspot + Pipeline Job + + + linux-x64-hotspot + Pipeline Job + + + + + + + + + + + + + + + + + + + Generate and Execute + + diff --git a/pipelines/build/BuildBaseFile.groovy b/pipelines/build/build_base_file.groovy similarity index 96% rename from pipelines/build/BuildBaseFile.groovy rename to pipelines/build/build_base_file.groovy index eb69fd812..9b4d99bfb 100644 --- a/pipelines/build/BuildBaseFile.groovy +++ b/pipelines/build/build_base_file.groovy @@ -20,7 +20,7 @@ limitations under the License. * * This: * - * 1. Generate job for each configuration based on createJobFromTemplate.groovy + * 1. Generate job for each configuration based on create_job_from_template.groovy * 2. Execute job * 3. Push generated artifacts to github */ @@ -138,7 +138,7 @@ static def getJobFolder(config) { return "build-scripts/jobs/${config.javaVersion}" } -// Generate a job from template at `createJobFromTemplate.groovy` +// Generate a job from template at `create_job_from_template.groovy` def createJob(jobName, jobFolder, config, enableTests) { def params = config.parameters.clone() @@ -146,7 +146,7 @@ def createJob(jobName, jobFolder, config, enableTests) { params.put("JOB_FOLDER", jobFolder) params.put("TEST_CONFIG", JsonOutput.prettyPrint(JsonOutput.toJson(config))) - create = jobDsl targets: "pipelines/build/createJobFromTemplate.groovy", ignoreExisting: false, additionalParameters: params + create = jobDsl targets: "pipelines/build/create_job_from_template.groovy", ignoreExisting: false, additionalParameters: params return create } diff --git a/pipelines/build/createJobFromTemplate.groovy b/pipelines/build/create_job_from_template.groovy similarity index 95% rename from pipelines/build/createJobFromTemplate.groovy rename to pipelines/build/create_job_from_template.groovy index febe95535..c91830749 100644 --- a/pipelines/build/createJobFromTemplate.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -20,7 +20,7 @@ folder(buildFolder) { } pipelineJob("$buildFolder/$JOB_NAME") { - description('

THIS IS AN AUTOMATICALLY GENERATED JOB DO NOT MODIFY, IT WILL BE OVERWRITTEN.

This job is defined in createJobFromTemplate.groovy in the openjdk-build repo, if you wish to change it modify that

') + description('

THIS IS AN AUTOMATICALLY GENERATED JOB DO NOT MODIFY, IT WILL BE OVERWRITTEN.

This job is defined in create_job_from_template.groovy in the openjdk-build repo, if you wish to change it modify that

') definition { cpsScm { scm { diff --git a/pipelines/build/openjdk10_pipeline.groovy b/pipelines/build/openjdk10_pipeline.groovy index 76213e1d8..eec1f6267 100644 --- a/pipelines/build/openjdk10_pipeline.groovy +++ b/pipelines/build/openjdk10_pipeline.groovy @@ -54,7 +54,8 @@ def buildConfigurations = [ os : 'linux', arch : 'ppc64le', additionalNodeLabels: [ - hotspot: 'centos7', + // Pinned as at time of writing build-osuosl-centos74-ppc64le-2 does not have a valid boot jdk + hotspot: 'centos7&&build-osuosl-centos74-ppc64le-1', openj9: 'ubuntu' ], test : ['openjdktest', 'systemtest'] @@ -82,12 +83,12 @@ def buildConfigurations = [ ], */ "linuxXL" : [ - os : 'linux', - additionalNodeLabels: 'centos6', - arch : 'x64', - test : false, + os : 'linux', + additionalNodeLabels : 'centos6', + arch : 'x64', + test : false, additionalFileNameTag: "linuxXL", - configureArgs : '--with-noncompressedrefs' + configureArgs : '--with-noncompressedrefs' ], ] @@ -95,6 +96,6 @@ def javaToBuild = "jdk10u" node ("master") { checkout scm - def buildFile = load "${WORKSPACE}/pipelines/build/BuildBaseFile.groovy" + def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) } diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index c3d7138ab..7fb7ca303 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -75,12 +75,12 @@ def buildConfigurations = [ ], */ "linuxXL" : [ - os : 'linux', - additionalNodeLabels: 'centos6', - arch : 'x64', - test : false, + os : 'linux', + additionalNodeLabels : 'centos6', + arch : 'x64', + test : false, additionalFileNameTag: "linuxXL", - configureArgs : '--with-noncompressedrefs' + configureArgs : '--with-noncompressedrefs' ], ] @@ -88,6 +88,6 @@ def javaToBuild = "jdk11" node ("master") { checkout scm - def buildFile = load "${WORKSPACE}/pipelines/build/BuildBaseFile.groovy" + def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) } diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy index f13a8b55a..6972d4ab8 100644 --- a/pipelines/build/openjdk8_pipeline.groovy +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -21,8 +21,8 @@ def buildConfigurations = [ ], x64Linux : [ - os : 'linux', - arch : 'x64', + os : 'linux', + arch : 'x64', additionalNodeLabels: [ hotspot: 'centos6', openj9: 'build-joyent-centos69-x64-1' @@ -32,8 +32,8 @@ def buildConfigurations = [ // Currently we have to be quite specific about which windows to use as not all of them have freetype installed x64Windows: [ - os : 'windows', - arch : 'x64', + os : 'windows', + arch : 'x64', additionalNodeLabels: [ hotspot: 'win2008', openj9: 'win2012' @@ -48,30 +48,30 @@ def buildConfigurations = [ ], s390xLinux : [ - os : 'linux', - arch : 's390x', + os : 'linux', + arch : 's390x', test : ['openjdktest', 'systemtest'] ], ppc64leLinux : [ - os : 'linux', - arch : 'ppc64le', + os : 'linux', + arch : 'ppc64le', test : ['openjdktest', 'systemtest'] ], arm32Linux : [ - os : 'linux', - arch : 'arm', + os : 'linux', + arch : 'arm', test : ['openjdktest'] ], "linuxXL" : [ - os : 'linux', - additionalNodeLabels: 'centos6', - arch : 'x64', - test : false, + os : 'linux', + additionalNodeLabels : 'centos6', + arch : 'x64', + test : false, additionalFileNameTag: "linuxXL", - configureArgs : '--with-noncompressedrefs' + configureArgs : '--with-noncompressedrefs' ], ] @@ -79,7 +79,7 @@ def javaToBuild = "jdk8u" node ("master") { checkout scm - def buildFile = load "${WORKSPACE}/pipelines/build/BuildBaseFile.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, osTarget, enableTests, publish, releaseTag) + def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) } diff --git a/pipelines/build/openjdk9_pipeline.groovy b/pipelines/build/openjdk9_pipeline.groovy index 4fedb158c..5f86fa65a 100644 --- a/pipelines/build/openjdk9_pipeline.groovy +++ b/pipelines/build/openjdk9_pipeline.groovy @@ -14,15 +14,15 @@ limitations under the License. def buildConfigurations = [ x64Mac : [ - os : 'mac', - arch : 'x64', + os : 'mac', + arch : 'x64', additionalNodeLabels: 'build-macstadium-macos1010-1', test : ['openjdktest', 'systemtest'] ], x64Linux : [ - os : 'linux', - arch : 'x64', + os : 'linux', + arch : 'x64', additionalNodeLabels: 'centos6', test : ['openjdktest', 'systemtest', 'externaltest'] ], @@ -36,45 +36,45 @@ def buildConfigurations = [ ], ppc64Aix : [ - os : 'aix', - arch : 'ppc64', + os : 'aix', + arch : 'ppc64', test : false ], s390xLinux : [ - os : 'linux', - arch : 's390x', + os : 'linux', + arch : 's390x', additionalNodeLabels: 'rhel7', test : ['openjdktest', 'systemtest'] ], ppc64leLinux : [ - os : 'linux', - arch : 'ppc64le', + os : 'linux', + arch : 'ppc64le', additionalNodeLabels: 'centos7', test : ['openjdktest', 'systemtest'] ], arm32Linux : [ - os : 'linux', - arch : 'arm', + os : 'linux', + arch : 'arm', test : ['openjdktest'] ], aarch64Linux : [ - os : 'linux', - arch : 'aarch64', + os : 'linux', + arch : 'aarch64', additionalNodeLabels: 'centos7', test : ['openjdktest'] ], "linuxXL" : [ - os : 'linux', - additionalNodeLabels: 'centos6', - arch : 'x64', - test : false, + os : 'linux', + additionalNodeLabels : 'centos6', + arch : 'x64', + test : false, additionalFileNameTag: "linuxXL", - configureArgs : '--with-noncompressedrefs' + configureArgs : '--with-noncompressedrefs' ], ] @@ -82,6 +82,6 @@ def javaToBuild = "jdk9u" node ("master") { checkout scm - def buildFile = load "${WORKSPACE}/pipelines/build/BuildBaseFile.groovy" + def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) } diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index f17f645c0..a6fda2d12 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -18,7 +18,7 @@ import groovy.json.JsonSlurper * This file is a template for running a build for a given configuration * A configuration is for example jdk10u-mac-x64-hotspot. * - * This file is referenced by the pipeline template createJobFromTemplate.groovy + * This file is referenced by the pipeline template create_job_from_template.groovy * * A pipeline looks like: * 1. Check out and build JDK by calling build-farm/make-adopt-build-farm.sh @@ -142,7 +142,7 @@ try { } } - if (enableTests) { + if (enableTests && config.test != false) { try { testStages = runTests(config) parallel testStages From 31b5ee7c48fff1702ef49aa427cb3bebb39ddf2d Mon Sep 17 00:00:00 2001 From: John Oliver Date: Thu, 13 Sep 2018 13:45:20 +0100 Subject: [PATCH 015/101] Try to fix some java 11 build issues (#505) * Fix typo * Ping ppc64le hotspot * Check if tests are required * Add docs * Fix link * add build status * Update status * Tidy * Fix shellcheck * Fix links * Fix from review * Try to fix some java 11 build issues * Make arm32 less specific --- .../platform-specific-configurations/linux.sh | 12 ++++----- pipelines/build/openjdk11_pipeline.groovy | 2 +- pipelines/build/openjdk_build_pipeline.groovy | 26 +++++++++++-------- 3 files changed, 21 insertions(+), 19 deletions(-) diff --git a/build-farm/platform-specific-configurations/linux.sh b/build-farm/platform-specific-configurations/linux.sh index f13c10da0..dd770d07b 100755 --- a/build-farm/platform-specific-configurations/linux.sh +++ b/build-farm/platform-specific-configurations/linux.sh @@ -21,13 +21,6 @@ source "$SCRIPT_DIR/../../sbin/common/constants.sh" if [ "${ARCHITECTURE}" == "x64" ] then export PATH=/opt/rh/devtoolset-2/root/usr/bin:$PATH - - if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] - then - [ -r /usr/local/gcc/bin/gcc-7.3 ] && export CC=/usr/local/gcc/bin/gcc-7.3 - [ -r /usr/local/gcc/bin/g++-7.3 ] && export CXX=/usr/local/gcc/bin/g++-7.3 - export LD_LIBRARY_PATH=/usr/local/gcc/lib64:/usr/local/gcc/lib - fi fi if [ "${ARCHITECTURE}" == "s390x" ] @@ -92,4 +85,9 @@ then fi export JDK_BOOT_DIR=$JDK10_BOOT_DIR + [ -r /usr/local/gcc/bin ] && export PATH=/usr/local/gcc/bin:$PATH + [ -r /usr/local/gcc/bin/gcc-7.3 ] && export CC=/usr/local/gcc/bin/gcc-7.3 + [ -r /usr/local/gcc/bin/g++-7.3 ] && export CXX=/usr/local/gcc/bin/g++-7.3 + export LD_LIBRARY_PATH=/usr/local/gcc/lib64:/usr/local/gcc/lib + fi \ No newline at end of file diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index 7fb7ca303..bf5978fe9 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -55,7 +55,7 @@ def buildConfigurations = [ arm32Linux : [ os : 'linux', - arch : 'arm32', + arch : 'arm', test : ['openjdktest'] ], diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index a6fda2d12..5710583bf 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -63,19 +63,23 @@ def runTests(config) { config.test.each { testType -> // For each requested test, i.e 'openjdktest', 'systemtest', 'perftest', 'externaltest', call test job - println "Running test: ${testType}}" - testStages["${testType}"] = { - stage("${testType}") { - - // example jobName: openjdk10_hs_externaltest_x86-64_linux - def jobName = determineTestJobName(config, testType) - catchError { - build job: jobName, - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${env.BUILD_NUMBER}"), - string(name: 'UPSTREAM_JOB_NAME', value: "${env.JOB_NAME}")] + try { + println "Running test: ${testType}}" + testStages["${testType}"] = { + stage("${testType}") { + + // example jobName: openjdk10_hs_externaltest_x86-64_linux + def jobName = determineTestJobName(config, testType) + catchError { + build job: jobName, + propagate: false, + parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${env.BUILD_NUMBER}"), + string(name: 'UPSTREAM_JOB_NAME', value: "${env.JOB_NAME}")] + } } } + } catch (Exception e) { + println "Failed execute test: ${e.getLocalizedMessage()}" } } return testStages From 03026f4895dce26665fc694def959b3f48dff4fb Mon Sep 17 00:00:00 2001 From: John Oliver Date: Thu, 13 Sep 2018 15:03:30 +0100 Subject: [PATCH 016/101] Try to cleanup on aix (#506) * Fix typo * Ping ppc64le hotspot * Check if tests are required * Add docs * Fix link * add build status * Update status * Tidy * Fix shellcheck * Fix links * Fix from review * Try to fix some java 11 build issues * Make arm32 less specific * Try putting a cleanup for aix * Better detection --- pipelines/build/build_base_file.groovy | 4 ++++ pipelines/build/create_job_from_template.groovy | 3 +++ 2 files changed, 7 insertions(+) diff --git a/pipelines/build/build_base_file.groovy b/pipelines/build/build_base_file.groovy index 9b4d99bfb..7dcb5de6a 100644 --- a/pipelines/build/build_base_file.groovy +++ b/pipelines/build/build_base_file.groovy @@ -146,6 +146,10 @@ def createJob(jobName, jobFolder, config, enableTests) { params.put("JOB_FOLDER", jobFolder) params.put("TEST_CONFIG", JsonOutput.prettyPrint(JsonOutput.toJson(config))) + // Clean workspace on aix as they are in a ramdisk + params.put("CLEANUP", config.os.equals("aix")) + + create = jobDsl targets: "pipelines/build/create_job_from_template.groovy", ignoreExisting: false, additionalParameters: params return create diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy index c91830749..fd6a1b17b 100644 --- a/pipelines/build/create_job_from_template.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -37,6 +37,9 @@ pipelineJob("$buildFolder/$JOB_NAME") { scriptPath('pipelines/build/openjdk_build_pipeline.groovy') lightweight(true) } + if ($CLEANUP) { + cleanWs notFailBuild: true + } } properties { copyArtifactPermissionProperty { From 7b4874862457a1d702e4d70d94a7051cf0240cf0 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Thu, 13 Sep 2018 15:14:08 +0100 Subject: [PATCH 017/101] Fix typo (#507) --- pipelines/build/create_job_from_template.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy index fd6a1b17b..f76a048b2 100644 --- a/pipelines/build/create_job_from_template.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -37,7 +37,7 @@ pipelineJob("$buildFolder/$JOB_NAME") { scriptPath('pipelines/build/openjdk_build_pipeline.groovy') lightweight(true) } - if ($CLEANUP) { + if (CLEANUP) { cleanWs notFailBuild: true } } From dac3a7eaed259cb990b80870642c17bf80840f00 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Thu, 13 Sep 2018 15:41:38 +0100 Subject: [PATCH 018/101] Move cleanws onto end build (#508) * Fix typo * Update build matrix --- README.md | 11 +++++------ pipelines/build/build_base_file.groovy | 4 ---- pipelines/build/create_job_from_template.groovy | 4 ---- pipelines/build/openjdk_build_pipeline.groovy | 11 ++++++++--- 4 files changed, 13 insertions(+), 17 deletions(-) diff --git a/README.md b/README.md index 866f416e8..1c4190856 100644 --- a/README.md +++ b/README.md @@ -287,13 +287,12 @@ The documentation for files that define the build farm builds can be found at [B Table generated with `generateBuildMatrix.sh` -| Platform | Java 8 | Java 9 | Java 10 | Java 11 | -| ------------------------- | ------ | ------ | ------- | ------- | -| aix-ppc64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-hotspot) | -| aix-ppc64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-openj9) | +| Platform | Java 8 | Java 9 | Java 10 | Java 11 | +| ------------------------- | ------ | ------ | ------- | ------- | +| aix-ppc64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-hotspot) | +| aix-ppc64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-openj9) | | linux-aarch64-hotspot | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-aarch64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-aarch64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-aarch64-hotspot) | -| linux-arm32-hotspot | N/A | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-arm32-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-arm32-hotspot) | -| linux-arm-hotspot | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-arm-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-arm-hotspot) | N/A | +| linux-arm-hotspot | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-arm-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-arm-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-arm-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-arm-hotspot) | | linux-ppc64le-hotspot | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-ppc64le-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-ppc64le-hotspot) | | linux-ppc64le-openj9 | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-ppc64le-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-ppc64le-openj9) | | linux-s390x-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-s390x-hotspot) | diff --git a/pipelines/build/build_base_file.groovy b/pipelines/build/build_base_file.groovy index 7dcb5de6a..9b4d99bfb 100644 --- a/pipelines/build/build_base_file.groovy +++ b/pipelines/build/build_base_file.groovy @@ -146,10 +146,6 @@ def createJob(jobName, jobFolder, config, enableTests) { params.put("JOB_FOLDER", jobFolder) params.put("TEST_CONFIG", JsonOutput.prettyPrint(JsonOutput.toJson(config))) - // Clean workspace on aix as they are in a ramdisk - params.put("CLEANUP", config.os.equals("aix")) - - create = jobDsl targets: "pipelines/build/create_job_from_template.groovy", ignoreExisting: false, additionalParameters: params return create diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy index f76a048b2..208185223 100644 --- a/pipelines/build/create_job_from_template.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -14,7 +14,6 @@ if (!binding.hasVariable('ADDITIONAL_FILE_NAME_TAG')) ADDITIONAL_FILE_NAME_TAG = if (!binding.hasVariable('TEST_CONFIG')) TEST_CONFIG = "" if (!binding.hasVariable('ENABLE_TESTS')) ENABLE_TESTS = "false" - folder(buildFolder) { description 'Automatically generated build jobs.' } @@ -37,9 +36,6 @@ pipelineJob("$buildFolder/$JOB_NAME") { scriptPath('pipelines/build/openjdk_build_pipeline.groovy') lightweight(true) } - if (CLEANUP) { - cleanWs notFailBuild: true - } } properties { copyArtifactPermissionProperty { diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index 5710583bf..255c97691 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -140,9 +140,14 @@ try { stage("build") { node(NODE_LABEL) { checkout scm - - sh "./build-farm/make-adopt-build-farm.sh" - archiveArtifacts artifacts: "workspace/target/*" + try { + sh "./build-farm/make-adopt-build-farm.sh" + archiveArtifacts artifacts: "workspace/target/*" + } finally { + if (config.os == "aix") { + cleanWs notFailBuild: true + } + } } } From db2d51c7c793d2c69ddbc5c6b784edd1db426428 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 14 Sep 2018 11:48:36 +0100 Subject: [PATCH 019/101] Try to fix window java 8 (#510) * Fix typo * Update build matrix * Try to fix java 8 windows --- build-farm/platform-specific-configurations/windows.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 65839a44e..4dc7a17d6 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -29,8 +29,8 @@ export BUILD_ARGS="--tmp-space-build ${BUILD_ARGS}" if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] then - export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/C/Projects/OpenJDK/make-3.82/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} with_freetype=/cygdrive/C/openjdk/freetype --disable-ccache" + export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/c/openjdk/make-3.82/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib64 --disable-ccache" elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] then export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" From 0bde698c04f12380bc240af793e9bb458dbf781e Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 14 Sep 2018 14:24:23 +0100 Subject: [PATCH 020/101] Update build matrix (#511) * Fix typo * Update build matrix * Try to fix java 8 windows * Update build matrix --- README.md | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 1c4190856..e4043e405 100644 --- a/README.md +++ b/README.md @@ -287,20 +287,19 @@ The documentation for files that define the build farm builds can be found at [B Table generated with `generateBuildMatrix.sh` -| Platform | Java 8 | Java 9 | Java 10 | Java 11 | -| ------------------------- | ------ | ------ | ------- | ------- | -| aix-ppc64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-hotspot) | -| aix-ppc64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-openj9) | +| Platform | Java 8 | Java 9 | Java 10 | Java 11 | +| ------------------------- | ------ | ------ | ------- | ------- | +| aix-ppc64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-hotspot) | +| aix-ppc64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-aix-ppc64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-aix-ppc64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-aix-ppc64-openj9) | | linux-aarch64-hotspot | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-aarch64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-aarch64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-aarch64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-aarch64-hotspot) | | linux-arm-hotspot | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-arm-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-arm-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-arm-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-arm-hotspot) | -| linux-ppc64le-hotspot | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-ppc64le-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-ppc64le-hotspot) | -| linux-ppc64le-openj9 | N/A | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-ppc64le-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-ppc64le-openj9) | +| linux-ppc64le-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-ppc64le-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-ppc64le-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-ppc64le-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-ppc64le-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-ppc64le-hotspot) | +| linux-ppc64le-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-ppc64le-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-ppc64le-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-ppc64le-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-ppc64le-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-ppc64le-openj9) | | linux-s390x-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-s390x-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-s390x-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-s390x-hotspot) | | linux-s390x-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-s390x-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-s390x-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-s390x-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-s390x-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-s390x-openj9) | | linux-x64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-x64-hotspot) | | linux-x64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-x64-openj9) | -| linux-x64-openj9-linuxXL | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-x64-openj9-linuxXL) | N/A | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-x64-openj9-linuxXL) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-x64-openj9-linuxXL) | +| linux-x64-openj9-linuxXL | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-linux-x64-openj9-linuxXL) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-linux-x64-openj9-linuxXL) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-linux-x64-openj9-linuxXL) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-linux-x64-openj9-linuxXL)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-linux-x64-openj9-linuxXL) | | mac-x64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-mac-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-mac-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-mac-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-mac-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-mac-x64-hotspot) | | windows-x64-hotspot | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-windows-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-windows-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-windows-x64-hotspot) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-windows-x64-hotspot)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-windows-x64-hotspot) | | windows-x64-openj9 | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk8u/jdk8u-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/job/jdk8u-windows-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk9u/jdk9u-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk9u/job/jdk9u-windows-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk10u/jdk10u-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk10u/job/jdk10u-windows-x64-openj9) | [![Build Status](https://ci.adoptopenjdk.net/buildStatus/icon?job=build-scripts/jobs/jdk11/jdk11-windows-x64-openj9)](https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk11/job/jdk11-windows-x64-openj9) | - From bb7c8f772552c9ec34eec00c92effa29eb1e5f6f Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 14 Sep 2018 15:50:58 +0100 Subject: [PATCH 021/101] Move to dev branch (#512) * Fix typo * Update build matrix * Try to fix java 8 windows * Update build matrix * Move to dev branch --- build-farm/make-adopt-build-farm.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index 4a3e2e940..aee05dca4 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -25,11 +25,14 @@ TIMESTAMP="$(date +'%Y-%m-%d-%H-%M')" export OPERATING_SYSTEM OPERATING_SYSTEM=$(echo "${TARGET_OS}" | tr '[:upper:]' '[:lower:]') +branch="dev" + echo "BUILD TYPE: " echo "VERSION: ${JAVA_TO_BUILD}" echo "ARCHITECTURE ${ARCHITECTURE}" echo "VARIANT: ${VARIANT}" echo "OS: ${OPERATING_SYSTEM}" +echo "BRANCH: ${branch}" OPTIONS="" EXTENSION="" @@ -89,4 +92,4 @@ fi echo "Filename will be: $FILENAME" # shellcheck disable=SC2086 -bash "$PLATFORM_SCRIPT_DIR/../makejdk-any-platform.sh" --clean-git-repo --jdk-boot-dir "${JDK_BOOT_DIR}" --configure-args "${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" --target-file-name "${FILENAME}" ${TAG_OPTION} ${OPTIONS} ${BUILD_ARGS} ${VARIANT_ARG} "${JAVA_TO_BUILD}" +bash "$PLATFORM_SCRIPT_DIR/../makejdk-any-platform.sh" --clean-git-repo -b "${branch}" --jdk-boot-dir "${JDK_BOOT_DIR}" --configure-args "${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" --target-file-name "${FILENAME}" ${TAG_OPTION} ${OPTIONS} ${BUILD_ARGS} ${VARIANT_ARG} "${JAVA_TO_BUILD}" From 6d1bf0b9aaefb32607083f948a5106d128d08112 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Sun, 16 Sep 2018 17:57:17 +0100 Subject: [PATCH 022/101] Confirm test exists before running it (#513) * Fix typo * Update build matrix * Try to fix java 8 windows * Update build matrix * Move to dev branch * Confirm test exists before running it * fix typo --- pipelines/build/openjdk_build_pipeline.groovy | 20 ++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index 255c97691..2c3743832 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -70,11 +70,21 @@ def runTests(config) { // example jobName: openjdk10_hs_externaltest_x86-64_linux def jobName = determineTestJobName(config, testType) - catchError { - build job: jobName, - propagate: false, - parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${env.BUILD_NUMBER}"), - string(name: 'UPSTREAM_JOB_NAME', value: "${env.JOB_NAME}")] + + def jobExists = Jenkins.instance.getAllItems() + .findAll { job -> + job.fullName == jobName && !job.isDisabled() + }.size() > 0; + + if (jobExists) { + catchError { + build job: jobName, + propagate: false, + parameters: [string(name: 'UPSTREAM_JOB_NUMBER', value: "${env.BUILD_NUMBER}"), + string(name: 'UPSTREAM_JOB_NAME', value: "${env.JOB_NAME}")] + } + } else { + println "Requested test job that does not exist or is disabled: ${jobName}" } } } From 964acfe08b0b1aaca8942e14d9527bc46f833bc8 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 11:22:52 +0100 Subject: [PATCH 023/101] Use getInstance due to sandbox issues (#515) --- pipelines/build/openjdk_build_pipeline.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index 2c3743832..d89e8d738 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -71,7 +71,7 @@ def runTests(config) { // example jobName: openjdk10_hs_externaltest_x86-64_linux def jobName = determineTestJobName(config, testType) - def jobExists = Jenkins.instance.getAllItems() + def jobExists = Jenkins.getInstance().getAllItems() .findAll { job -> job.fullName == jobName && !job.isDisabled() }.size() > 0; From 2ab0dd5ac4be219822d36c39b8f6270cda569a90 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 12:31:03 +0100 Subject: [PATCH 024/101] Use shared library for node and job detection (#516) * Use getInstance due to sandbox issues * Use shared library for node and job detection * Fix typo --- pipelines/build/openjdk_build_pipeline.groovy | 30 ++++++++++--------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index d89e8d738..194ea57b2 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -11,6 +11,8 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ +@Library(['NodeHelper', 'JobHelper']) _ + import groovy.json.JsonSlurper @@ -71,12 +73,7 @@ def runTests(config) { // example jobName: openjdk10_hs_externaltest_x86-64_linux def jobName = determineTestJobName(config, testType) - def jobExists = Jenkins.getInstance().getAllItems() - .findAll { job -> - job.fullName == jobName && !job.isDisabled() - }.size() > 0; - - if (jobExists) { + if (JobHelper.jobIsRunnable(jobName)) { catchError { build job: jobName, propagate: false, @@ -148,16 +145,21 @@ try { def enableTests = ENABLE_TESTS == "true" stage("build") { - node(NODE_LABEL) { - checkout scm - try { - sh "./build-farm/make-adopt-build-farm.sh" - archiveArtifacts artifacts: "workspace/target/*" - } finally { - if (config.os == "aix") { - cleanWs notFailBuild: true + if(NodeHelper.nodeIsOnline(NODE_LABEL)) { + node(NODE_LABEL) { + checkout scm + try { + sh "./build-farm/make-adopt-build-farm.sh" + archiveArtifacts artifacts: "workspace/target/*" + } finally { + if (config.os == "aix") { + cleanWs notFailBuild: true + } } } + } else { + error("No node of this type exists: ${NODE_LABEL}") + return } } From 0149f040a0044dd81f46288b60dea99d837f5a4c Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 12:46:56 +0100 Subject: [PATCH 025/101] Fix import method --- pipelines/build/openjdk_build_pipeline.groovy | 3 +++ 1 file changed, 3 insertions(+) diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index d89e8d738..56e50c96b 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -11,8 +11,11 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ + import groovy.json.JsonSlurper +def JobHelper = library('openjdk-jenkins-helper').JobHelper +def NodeHelper = library('openjdk-jenkins-helper').NodeHelper /** * This file is a template for running a build for a given configuration From 3d54dd676b9071a17380f554fbd033265914f999 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 12:49:22 +0100 Subject: [PATCH 026/101] Fix import method (#517) --- pipelines/build/openjdk_build_pipeline.groovy | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index 194ea57b2..c34c9bb75 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -11,10 +11,11 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ -@Library(['NodeHelper', 'JobHelper']) _ import groovy.json.JsonSlurper +def JobHelper = library('openjdk-jenkins-helper').JobHelper +def NodeHelper = library('openjdk-jenkins-helper').NodeHelper /** * This file is a template for running a build for a given configuration From b38752f4187b88a016063ad34eb194e3cb900c76 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 13:52:19 +0100 Subject: [PATCH 027/101] Lib test --- pipelines/build/lib-test.groovy | 8 ++++++++ pipelines/build/openjdk_build_pipeline.groovy | 5 +++-- 2 files changed, 11 insertions(+), 2 deletions(-) create mode 100644 pipelines/build/lib-test.groovy diff --git a/pipelines/build/lib-test.groovy b/pipelines/build/lib-test.groovy new file mode 100644 index 000000000..165942cbf --- /dev/null +++ b/pipelines/build/lib-test.groovy @@ -0,0 +1,8 @@ +@Library('openjdk-jenkins-helper@master') +import JobHelper +import NodeHelper + +println JobHelper +println NodeHelper + +println JobHelper.jobIsRunnable("build-scripts/old-stuff/create-build-job") \ No newline at end of file diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index c34c9bb75..4d4e28921 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -14,8 +14,9 @@ limitations under the License. import groovy.json.JsonSlurper -def JobHelper = library('openjdk-jenkins-helper').JobHelper -def NodeHelper = library('openjdk-jenkins-helper').NodeHelper +def helperLib = library('openjdk-jenkins-helper') +def JobHelper = helperLib.JobHelper +def NodeHelper = helperLib.NodeHelper /** * This file is a template for running a build for a given configuration From e8fd5bba26fa92495f574efd910e906440e36dbc Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 14:03:39 +0100 Subject: [PATCH 028/101] Lib test --- pipelines/build/lib-test.groovy | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/pipelines/build/lib-test.groovy b/pipelines/build/lib-test.groovy index 165942cbf..7abe5bda0 100644 --- a/pipelines/build/lib-test.groovy +++ b/pipelines/build/lib-test.groovy @@ -5,4 +5,6 @@ import NodeHelper println JobHelper println NodeHelper -println JobHelper.jobIsRunnable("build-scripts/old-stuff/create-build-job") \ No newline at end of file +println JobHelper.jobIsRunnable("build-scripts/old-stuff/create-build-job") +println NodeHelper.nodeIsOnline("linux") +println NodeHelper.nodeIsOnline("ubuntu16&&openj9") \ No newline at end of file From 07aa6d76db7fbc50dd5800c97dee6849914be3af Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 14:04:37 +0100 Subject: [PATCH 029/101] Fix lib import yet again --- pipelines/build/lib-test.groovy | 10 ---------- pipelines/build/openjdk_build_pipeline.groovy | 7 ++++--- 2 files changed, 4 insertions(+), 13 deletions(-) delete mode 100644 pipelines/build/lib-test.groovy diff --git a/pipelines/build/lib-test.groovy b/pipelines/build/lib-test.groovy deleted file mode 100644 index 7abe5bda0..000000000 --- a/pipelines/build/lib-test.groovy +++ /dev/null @@ -1,10 +0,0 @@ -@Library('openjdk-jenkins-helper@master') -import JobHelper -import NodeHelper - -println JobHelper -println NodeHelper - -println JobHelper.jobIsRunnable("build-scripts/old-stuff/create-build-job") -println NodeHelper.nodeIsOnline("linux") -println NodeHelper.nodeIsOnline("ubuntu16&&openj9") \ No newline at end of file diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index 4d4e28921..dc59f54cd 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -14,9 +14,10 @@ limitations under the License. import groovy.json.JsonSlurper -def helperLib = library('openjdk-jenkins-helper') -def JobHelper = helperLib.JobHelper -def NodeHelper = helperLib.NodeHelper +@Library('openjdk-jenkins-helper@master') +import JobHelper +import NodeHelper + /** * This file is a template for running a build for a given configuration From ad5c692f893d2a0b98d416840f0ac5468d8bf142 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 14:08:24 +0100 Subject: [PATCH 030/101] Fix lib import yet again (#518) --- pipelines/build/openjdk_build_pipeline.groovy | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index c34c9bb75..dc59f54cd 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -14,8 +14,10 @@ limitations under the License. import groovy.json.JsonSlurper -def JobHelper = library('openjdk-jenkins-helper').JobHelper -def NodeHelper = library('openjdk-jenkins-helper').NodeHelper +@Library('openjdk-jenkins-helper@master') +import JobHelper +import NodeHelper + /** * This file is a template for running a build for a given configuration From 9e5a174647de2228992250e247b6b0de5a2aaa23 Mon Sep 17 00:00:00 2001 From: Martijn Verburg Date: Mon, 17 Sep 2018 14:34:51 +0100 Subject: [PATCH 031/101] Add JDK tip/HEAD pipeline (#519) --- .../platform-specific-configurations/aix.sh | 4 +- .../platform-specific-configurations/linux.sh | 3 +- .../platform-specific-configurations/mac.sh | 4 +- .../windows.sh | 4 +- .../set-platform-specific-configurations.sh | 2 +- pipelines/build/openjdk8_pipeline.groovy | 2 +- pipelines/build/openjdk_pipeline.groovy | 93 +++++++++++++++++++ sbin/common/constants.sh | 2 + 8 files changed, 104 insertions(+), 10 deletions(-) create mode 100644 pipelines/build/openjdk_pipeline.groovy diff --git a/build-farm/platform-specific-configurations/aix.sh b/build-farm/platform-specific-configurations/aix.sh index 769718811..d395ea441 100755 --- a/build-farm/platform-specific-configurations/aix.sh +++ b/build-farm/platform-specific-configurations/aix.sh @@ -36,13 +36,13 @@ then elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/ramdisk0/build/workspace/openjdk10_openj9_build_ppc64_aix/freemarker-2.3.8/lib/freemarker.jar" - elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/ramdisk0/build/workspace/openjdk10_openj9_build_ppc64_aix/freemarker-2.3.8/lib/freemarker.jar DF=/usr/sysv/bin/df" fi fi -if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ]; +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ]; then export JDK10_BOOT_DIR="$PWD/jdk-10" if [ ! -d "$JDK10_BOOT_DIR/bin" ]; then diff --git a/build-farm/platform-specific-configurations/linux.sh b/build-farm/platform-specific-configurations/linux.sh index dd770d07b..4af4f5ac8 100755 --- a/build-farm/platform-specific-configurations/linux.sh +++ b/build-farm/platform-specific-configurations/linux.sh @@ -72,8 +72,7 @@ then fi fi - -if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] then export JDK10_BOOT_DIR="$PWD/jdk-10" if [ ! -d "$JDK10_BOOT_DIR/bin" ]; then diff --git a/build-farm/platform-specific-configurations/mac.sh b/build-farm/platform-specific-configurations/mac.sh index c3420eccc..376a3f06f 100755 --- a/build-farm/platform-specific-configurations/mac.sh +++ b/build-farm/platform-specific-configurations/mac.sh @@ -30,13 +30,13 @@ fi sudo xcode-select --switch "${XCODE_SWITCH_PATH}" -if [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] +if [ "${JAVA_TO_BUILD}" != "${JDK8_VERSION}" ] then export PATH="/Users/jenkins/ccache-3.2.4:$PATH" fi -if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-extra-cxxflags=-mmacosx-version-min=10.8" diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 4dc7a17d6..270a00c44 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -39,7 +39,7 @@ elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] then export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --disable-ccache" -elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] +elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] then export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013 --disable-ccache" @@ -62,7 +62,7 @@ then elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" - elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" fi diff --git a/build-farm/set-platform-specific-configurations.sh b/build-farm/set-platform-specific-configurations.sh index cd569afd3..bed468b4c 100755 --- a/build-farm/set-platform-specific-configurations.sh +++ b/build-farm/set-platform-specific-configurations.sh @@ -19,7 +19,7 @@ SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" # shellcheck source=sbin/common/constants.sh source "$SCRIPT_DIR/../sbin/common/constants.sh" -if [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] +if [ "${JAVA_TO_BUILD}" != "${JDK8_VERSION}" ] then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-warnings-as-errors" fi diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy index 6972d4ab8..f2ca1b1fa 100644 --- a/pipelines/build/openjdk8_pipeline.groovy +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -27,7 +27,7 @@ def buildConfigurations = [ hotspot: 'centos6', openj9: 'build-joyent-centos69-x64-1' ], - test : ['openjdktest', 'systemtest', 'perftest', 'externaltest'] + test : ['openjdktest', 'systemtest', 'perftest', 'externaltest', 'externaltest_extended'] ], // Currently we have to be quite specific about which windows to use as not all of them have freetype installed diff --git a/pipelines/build/openjdk_pipeline.groovy b/pipelines/build/openjdk_pipeline.groovy new file mode 100644 index 000000000..4654d79a8 --- /dev/null +++ b/pipelines/build/openjdk_pipeline.groovy @@ -0,0 +1,93 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +def buildConfigurations = [ + x64Mac : [ + os : 'mac', + arch : 'x64', + additionalNodeLabels: 'build-macstadium-macos1010-1', + test : ['openjdktest', 'systemtest'] + ], + x64Linux : [ + os : 'linux', + arch : 'x64', + additionalNodeLabels: 'centos6', + test : ['openjdktest', 'systemtest', 'externaltest'] + ], + + // Currently we have to be quite specific about which windows to use as not all of them have freetype installed + x64Windows: [ + os : 'windows', + arch : 'x64', + additionalNodeLabels: 'win2012', + test : ['openjdktest'] + ], + + ppc64Aix : [ + os : 'aix', + arch : 'ppc64', + test : false + ], + + s390xLinux : [ + os : 'linux', + arch : 's390x', + test : ['openjdktest', 'systemtest'] + ], + + ppc64leLinux : [ + os : 'linux', + arch : 'ppc64le', + additionalNodeLabels: 'centos7', + test : ['openjdktest', 'systemtest'] + ], + + arm32Linux : [ + os : 'linux', + arch : 'arm', + test : ['openjdktest'] + ], + + aarch64Linux : [ + os : 'linux', + arch : 'aarch64', + additionalNodeLabels: 'centos7', + test : ['openjdktest'] + ], + + /* + "x86-32Windows" : [ + os : 'windows', + arch : 'x86-32', + additionalNodeLabels: 'win2012&&x86-32', + test : false + ], + */ + "linuxXL" : [ + os : 'linux', + additionalNodeLabels : 'centos6', + arch : 'x64', + test : false, + additionalFileNameTag: "linuxXL", + configureArgs : '--with-noncompressedrefs' + ], +] + +def javaToBuild = "jdk" + +node ("master") { + checkout scm + def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) +} diff --git a/sbin/common/constants.sh b/sbin/common/constants.sh index 4c8bd94c4..b3238d7a4 100755 --- a/sbin/common/constants.sh +++ b/sbin/common/constants.sh @@ -18,9 +18,11 @@ export JDK8_VERSION="jdk8u"; export JDK9_VERSION="jdk9u"; export JDK10_VERSION="jdk10u"; export JDK11_VERSION="jdk11"; +export JDKHEAD_VERSION="jdk"; export JDK8_CORE_VERSION="jdk8"; export JDK9_CORE_VERSION="jdk9"; export JDK10_CORE_VERSION="jdk10"; export JDK11_CORE_VERSION="jdk11"; +export JDKHEAD_CORE_VERSION="${JDKHEAD_VERSION}"; export AMBER_CORE_VERSION="amber"; \ No newline at end of file From d7f7b3ffe83c64f1402c2cadd3382133442668a8 Mon Sep 17 00:00:00 2001 From: Martijn Verburg Date: Mon, 17 Sep 2018 15:39:02 +0100 Subject: [PATCH 032/101] Re include cacerts for windows closes #478 (#520) Re include cacerts for windows closes #478 --- sbin/prepareWorkspace.sh | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index 44dfbba49..7bc8c7905 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -283,9 +283,11 @@ downloadingRequiredDependencies() echo "Checking and download Freemarker dependency" checkingAndDownloadingFreemarker fi - echo "Checking and download CaCerts dependency" - checkingAndDownloadCaCerts fi + + echo "Checking and download CaCerts dependency" + checkingAndDownloadCaCerts + } function moveTmpToWorkspaceLocation { From 4a0a744ad23c2df49ab6b032083e606a653bc021 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 16:33:26 +0100 Subject: [PATCH 033/101] Try to fix cacerts --- README.md | 4 ++++ build-farm/make-adopt-build-farm.sh | 2 ++ sbin/build.sh | 6 +++++- sbin/common/config_init.sh | 7 +++++++ sbin/prepareWorkspace.sh | 31 +++++++++++++++-------------- 5 files changed, 34 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index e4043e405..1286d7af4 100644 --- a/README.md +++ b/README.md @@ -180,6 +180,10 @@ For reference, OpenJDK version numbers look like 1.8.0_162-b12 (for Java 8) or (162) or the 3rd position in the semVer version string (4). This is typically used in conjunction with -b. +--use-jep319-certs +Use certs defined in JEP319 in Java 8/9. This will increase the volume of traffic downloaded, however will +provide an upto date ca cert list. + -v, --build-variant specify a OpenJDK build variant, e.g. openj9. For reference, the default variant is hotspot and does not need to be specified. diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index aee05dca4..34d1e9a32 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -91,5 +91,7 @@ fi echo "Filename will be: $FILENAME" +export BUILD_ARGS="${BUILD_ARGS} --use-jep319-certs" + # shellcheck disable=SC2086 bash "$PLATFORM_SCRIPT_DIR/../makejdk-any-platform.sh" --clean-git-repo -b "${branch}" --jdk-boot-dir "${JDK_BOOT_DIR}" --configure-args "${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" --target-file-name "${FILENAME}" ${TAG_OPTION} ${OPTIONS} ${BUILD_ARGS} ${VARIANT_ARG} "${JAVA_TO_BUILD}" diff --git a/sbin/build.sh b/sbin/build.sh index 5d82b1955..65f606743 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -199,7 +199,11 @@ buildingTheRestOfTheConfigParameters() fi addConfigureArgIfValueIsNotEmpty "--with-jvm-variants=" "${BUILD_CONFIG[JVM_VARIANT]}" - addConfigureArgIfValueIsNotEmpty "--with-cacerts-file=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/cacerts_area/security/cacerts" + + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ]; then + addConfigureArgIfValueIsNotEmpty "--with-cacerts-file=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/cacerts_area/security/cacerts" + fi + addConfigureArg "--with-alsa=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedalsa" # Point-in-time dependency for openj9 only diff --git a/sbin/common/config_init.sh b/sbin/common/config_init.sh index 21f6cd23f..d76b20796 100755 --- a/sbin/common/config_init.sh +++ b/sbin/common/config_init.sh @@ -74,6 +74,7 @@ TARGET_FILE_NAME TMP_CONTAINER_NAME TMP_SPACE_BUILD USE_DOCKER +USE_JEP319_CERTS USE_SSH USER_SUPPLIED_CONFIGURE_ARGS WORKING_DIR @@ -233,6 +234,9 @@ function parseConfigurationArguments() { "--update-version" | "-u" ) BUILD_CONFIG[OPENJDK_UPDATE_VERSION]="$1"; shift;; + "--use-jep319-certs" ) + BUILD_CONFIG[USE_JEP319_CERTS]=true;; + "--version" | "-v" ) setOpenJdkVersion "$1"; shift;; @@ -345,6 +349,9 @@ function configDefaults() { # If the wrong git repo is there allow it to be removed BUILD_CONFIG[CLEAN_GIT_REPO]=false + + # By default dont backport JEP318 certs to < Java 10 + BUILD_CONFIG[USE_JEP319_CERTS]=false } # Declare the map of build configuration that we're going to use diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index 7bc8c7905..113f0f273 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -229,28 +229,29 @@ checkingAndDownloadCaCerts() { cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" || exit - echo "Retrieving cacerts file" - + echo "Retrieving cacerts file" # Ensure it's the latest we pull in rm -rf "cacerts_area" mkdir "cacerts_area" || exit cd "cacerts_area" || exit - git init - git remote add origin -f https://github.com/AdoptOpenJDK/openjdk-build.git - git config core.sparsecheckout true - echo "security/*" >> .git/info/sparse-checkout - git pull origin master - - echo "cacerts should be here..." - - # shellcheck disable=SC2046 - if ! [ -r "security/cacerts" ]; then - echo "Failed to retrieve the cacerts file, exiting..." - exit; + if [ "${BUILD_CONFIG[USE_JEP319_CERTS]}" == "true" ]; + then + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ] + then + echo "Requested use of JEP319 certs" + local jreLink="https://api.adoptopenjdk.net/v2/binary/nightly/openjdk10?arch=x64&os=linux&release=latest&type=jre&openjdk_impl=hotspot"; + mkdir -p "security" + wget -O - "${jreLink}" | tar --wildcards --strip-components=4 -xpzf - "*/lib/security/cacerts" -C "./security/" + fi else - echo "Successfully retrieved the cacerts file!" + git init + git remote add origin -f https://github.com/AdoptOpenJDK/openjdk-build.git + git config core.sparsecheckout true + echo "security/*" >> .git/info/sparse-checkout + git pull origin master fi + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" || exit } From 8f01fb6e558992f47603141a07198b4256ea4c80 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 16:37:52 +0100 Subject: [PATCH 034/101] Try to fix cacerts (#521) * Fix import method * Lib test * Lib test * Fix lib import yet again * Try to fix cacerts --- README.md | 4 ++++ build-farm/make-adopt-build-farm.sh | 2 ++ sbin/build.sh | 6 +++++- sbin/common/config_init.sh | 7 +++++++ sbin/prepareWorkspace.sh | 31 +++++++++++++++-------------- 5 files changed, 34 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index e4043e405..1286d7af4 100644 --- a/README.md +++ b/README.md @@ -180,6 +180,10 @@ For reference, OpenJDK version numbers look like 1.8.0_162-b12 (for Java 8) or (162) or the 3rd position in the semVer version string (4). This is typically used in conjunction with -b. +--use-jep319-certs +Use certs defined in JEP319 in Java 8/9. This will increase the volume of traffic downloaded, however will +provide an upto date ca cert list. + -v, --build-variant specify a OpenJDK build variant, e.g. openj9. For reference, the default variant is hotspot and does not need to be specified. diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index aee05dca4..34d1e9a32 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -91,5 +91,7 @@ fi echo "Filename will be: $FILENAME" +export BUILD_ARGS="${BUILD_ARGS} --use-jep319-certs" + # shellcheck disable=SC2086 bash "$PLATFORM_SCRIPT_DIR/../makejdk-any-platform.sh" --clean-git-repo -b "${branch}" --jdk-boot-dir "${JDK_BOOT_DIR}" --configure-args "${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" --target-file-name "${FILENAME}" ${TAG_OPTION} ${OPTIONS} ${BUILD_ARGS} ${VARIANT_ARG} "${JAVA_TO_BUILD}" diff --git a/sbin/build.sh b/sbin/build.sh index 5d82b1955..65f606743 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -199,7 +199,11 @@ buildingTheRestOfTheConfigParameters() fi addConfigureArgIfValueIsNotEmpty "--with-jvm-variants=" "${BUILD_CONFIG[JVM_VARIANT]}" - addConfigureArgIfValueIsNotEmpty "--with-cacerts-file=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/cacerts_area/security/cacerts" + + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ]; then + addConfigureArgIfValueIsNotEmpty "--with-cacerts-file=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/cacerts_area/security/cacerts" + fi + addConfigureArg "--with-alsa=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedalsa" # Point-in-time dependency for openj9 only diff --git a/sbin/common/config_init.sh b/sbin/common/config_init.sh index 21f6cd23f..d76b20796 100755 --- a/sbin/common/config_init.sh +++ b/sbin/common/config_init.sh @@ -74,6 +74,7 @@ TARGET_FILE_NAME TMP_CONTAINER_NAME TMP_SPACE_BUILD USE_DOCKER +USE_JEP319_CERTS USE_SSH USER_SUPPLIED_CONFIGURE_ARGS WORKING_DIR @@ -233,6 +234,9 @@ function parseConfigurationArguments() { "--update-version" | "-u" ) BUILD_CONFIG[OPENJDK_UPDATE_VERSION]="$1"; shift;; + "--use-jep319-certs" ) + BUILD_CONFIG[USE_JEP319_CERTS]=true;; + "--version" | "-v" ) setOpenJdkVersion "$1"; shift;; @@ -345,6 +349,9 @@ function configDefaults() { # If the wrong git repo is there allow it to be removed BUILD_CONFIG[CLEAN_GIT_REPO]=false + + # By default dont backport JEP318 certs to < Java 10 + BUILD_CONFIG[USE_JEP319_CERTS]=false } # Declare the map of build configuration that we're going to use diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index 7bc8c7905..113f0f273 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -229,28 +229,29 @@ checkingAndDownloadCaCerts() { cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" || exit - echo "Retrieving cacerts file" - + echo "Retrieving cacerts file" # Ensure it's the latest we pull in rm -rf "cacerts_area" mkdir "cacerts_area" || exit cd "cacerts_area" || exit - git init - git remote add origin -f https://github.com/AdoptOpenJDK/openjdk-build.git - git config core.sparsecheckout true - echo "security/*" >> .git/info/sparse-checkout - git pull origin master - - echo "cacerts should be here..." - - # shellcheck disable=SC2046 - if ! [ -r "security/cacerts" ]; then - echo "Failed to retrieve the cacerts file, exiting..." - exit; + if [ "${BUILD_CONFIG[USE_JEP319_CERTS]}" == "true" ]; + then + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ] + then + echo "Requested use of JEP319 certs" + local jreLink="https://api.adoptopenjdk.net/v2/binary/nightly/openjdk10?arch=x64&os=linux&release=latest&type=jre&openjdk_impl=hotspot"; + mkdir -p "security" + wget -O - "${jreLink}" | tar --wildcards --strip-components=4 -xpzf - "*/lib/security/cacerts" -C "./security/" + fi else - echo "Successfully retrieved the cacerts file!" + git init + git remote add origin -f https://github.com/AdoptOpenJDK/openjdk-build.git + git config core.sparsecheckout true + echo "security/*" >> .git/info/sparse-checkout + git pull origin master fi + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" || exit } From ec329c3c28ed38f4ef4b00cc9651ede2543c01f8 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 16:46:50 +0100 Subject: [PATCH 035/101] Fix mac tar not supporting wildcards --- sbin/prepareWorkspace.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index 113f0f273..ad85c57ed 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -242,7 +242,10 @@ checkingAndDownloadCaCerts() echo "Requested use of JEP319 certs" local jreLink="https://api.adoptopenjdk.net/v2/binary/nightly/openjdk10?arch=x64&os=linux&release=latest&type=jre&openjdk_impl=hotspot"; mkdir -p "security" - wget -O - "${jreLink}" | tar --wildcards --strip-components=4 -xpzf - "*/lib/security/cacerts" -C "./security/" + + wget -O jre.tar.gz "${jreLink}" + tar --list -zf jre.tar.gz | grep cacerts | xargs tar --wildcards --strip-components=4 -xpzf jre.tar.gz -C "./security/" + rm jre.tar.gz fi else git init From 95369442434c53bedb75ec31e6034af7858b594f Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 16:49:48 +0100 Subject: [PATCH 036/101] Match specifically --- sbin/prepareWorkspace.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index 191c2f02e..388b170e4 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -243,7 +243,7 @@ checkingAndDownloadCaCerts() local jreLink="https://api.adoptopenjdk.net/v2/binary/nightly/openjdk10?arch=x64&os=linux&release=latest&type=jre&openjdk_impl=hotspot"; mkdir -p "security" wget -O jre.tar.gz "${jreLink}" - tar --list -zf jre.tar.gz | grep cacerts | xargs tar --wildcards --strip-components=4 -xpzf jre.tar.gz -C "./security/" + tar --list -zf jre.tar.gz | grep "lib/security/cacerts" | xargs tar --wildcards --strip-components=4 -xpzf jre.tar.gz -C "./security/" rm jre.tar.gz fi else From 2d2827274a9a91379b608f368c3fd32c3ce99dd5 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 16:51:17 +0100 Subject: [PATCH 037/101] Remove wildcard --- sbin/prepareWorkspace.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index 388b170e4..a8abf5897 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -243,7 +243,7 @@ checkingAndDownloadCaCerts() local jreLink="https://api.adoptopenjdk.net/v2/binary/nightly/openjdk10?arch=x64&os=linux&release=latest&type=jre&openjdk_impl=hotspot"; mkdir -p "security" wget -O jre.tar.gz "${jreLink}" - tar --list -zf jre.tar.gz | grep "lib/security/cacerts" | xargs tar --wildcards --strip-components=4 -xpzf jre.tar.gz -C "./security/" + tar --list -zf jre.tar.gz | grep "lib/security/cacerts" | xargs tar --strip-components=4 -xpzf jre.tar.gz -C "./security/" rm jre.tar.gz fi else From 8d0a9c820f7b3d4713cd6b781b709fad3bb4648a Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 16:53:27 +0100 Subject: [PATCH 038/101] Fix downloading cacerts on mac (#522) * Fix import method * Lib test * Lib test * Fix lib import yet again * Try to fix cacerts * Fix mac tar not supporting wildcards * Match specifically * Remove wildcard --- sbin/prepareWorkspace.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index 113f0f273..a8abf5897 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -242,7 +242,9 @@ checkingAndDownloadCaCerts() echo "Requested use of JEP319 certs" local jreLink="https://api.adoptopenjdk.net/v2/binary/nightly/openjdk10?arch=x64&os=linux&release=latest&type=jre&openjdk_impl=hotspot"; mkdir -p "security" - wget -O - "${jreLink}" | tar --wildcards --strip-components=4 -xpzf - "*/lib/security/cacerts" -C "./security/" + wget -O jre.tar.gz "${jreLink}" + tar --list -zf jre.tar.gz | grep "lib/security/cacerts" | xargs tar --strip-components=4 -xpzf jre.tar.gz -C "./security/" + rm jre.tar.gz fi else git init From 4335e9626b3b386ac86abedd744c846333b6af94 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 17:00:11 +0100 Subject: [PATCH 039/101] Reduce download by 40mb by not being an idiot --- sbin/prepareWorkspace.sh | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index a8abf5897..7c2cda0e6 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -240,11 +240,9 @@ checkingAndDownloadCaCerts() if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ] then echo "Requested use of JEP319 certs" - local jreLink="https://api.adoptopenjdk.net/v2/binary/nightly/openjdk10?arch=x64&os=linux&release=latest&type=jre&openjdk_impl=hotspot"; + local caLink="https://github.com/AdoptOpenJDK/openjdk-jdk10u/blob/dev/src/java.base/share/lib/security/cacerts?raw=true"; mkdir -p "security" - wget -O jre.tar.gz "${jreLink}" - tar --list -zf jre.tar.gz | grep "lib/security/cacerts" | xargs tar --strip-components=4 -xpzf jre.tar.gz -C "./security/" - rm jre.tar.gz + wget -O "./security/cacerts" "${caLink}" fi else git init From 043a4ceab0170a26b56d2841d837db63ec4b05fe Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 17 Sep 2018 19:49:26 +0100 Subject: [PATCH 040/101] Try not downloading 40mb of useless dependencies (#523) * Fix import method * Lib test * Lib test * Fix lib import yet again * Try to fix cacerts * Fix mac tar not supporting wildcards * Match specifically * Remove wildcard * Reduce download by 40mb by not being an idiot --- sbin/prepareWorkspace.sh | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index a8abf5897..7c2cda0e6 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -240,11 +240,9 @@ checkingAndDownloadCaCerts() if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ] then echo "Requested use of JEP319 certs" - local jreLink="https://api.adoptopenjdk.net/v2/binary/nightly/openjdk10?arch=x64&os=linux&release=latest&type=jre&openjdk_impl=hotspot"; + local caLink="https://github.com/AdoptOpenJDK/openjdk-jdk10u/blob/dev/src/java.base/share/lib/security/cacerts?raw=true"; mkdir -p "security" - wget -O jre.tar.gz "${jreLink}" - tar --list -zf jre.tar.gz | grep "lib/security/cacerts" | xargs tar --strip-components=4 -xpzf jre.tar.gz -C "./security/" - rm jre.tar.gz + wget -O "./security/cacerts" "${caLink}" fi else git init From b8e26ae8a4fd64e2bf967fe2a0a715da398a290d Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 18 Sep 2018 11:20:17 +0100 Subject: [PATCH 041/101] Fix jdk build --- build-farm/make-adopt-build-farm.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index 34d1e9a32..76430eac3 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -56,8 +56,7 @@ case "${JDK_BOOT_VERSION}" in "9") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK9_BOOT_DIR}";; "10") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK10_BOOT_DIR}";; "11") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK11_BOOT_DIR}";; - "home") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JAVA_HOME}";; - *) export JDK_BOOT_DIR="${JDK_BOOT_VERSION}";; + *) export JDK_BOOT_DIR="${JDK11_BOOT_DIR}";; esac echo "Boot jdk: ${JDK_BOOT_DIR}" From 70e7f7aa077be610ac0ec82ab81f614285588d70 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 18 Sep 2018 11:21:08 +0100 Subject: [PATCH 042/101] Fix jdk build --- build-farm/make-adopt-build-farm.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index 76430eac3..b65c0e390 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -56,7 +56,7 @@ case "${JDK_BOOT_VERSION}" in "9") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK9_BOOT_DIR}";; "10") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK10_BOOT_DIR}";; "11") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK11_BOOT_DIR}";; - *) export JDK_BOOT_DIR="${JDK11_BOOT_DIR}";; + *) export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK11_BOOT_DIR}";; esac echo "Boot jdk: ${JDK_BOOT_DIR}" From db7c857a276ffe4a7019fc1b08455b73e020759d Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 18 Sep 2018 12:24:27 +0100 Subject: [PATCH 043/101] Fix head jdk build (#524) * Fix import method * Lib test * Lib test * Fix lib import yet again * Try to fix cacerts * Fix mac tar not supporting wildcards * Match specifically * Remove wildcard * Reduce download by 40mb by not being an idiot * Fix jdk build * Fix jdk build --- build-farm/make-adopt-build-farm.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index 34d1e9a32..b65c0e390 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -56,8 +56,7 @@ case "${JDK_BOOT_VERSION}" in "9") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK9_BOOT_DIR}";; "10") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK10_BOOT_DIR}";; "11") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK11_BOOT_DIR}";; - "home") export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JAVA_HOME}";; - *) export JDK_BOOT_DIR="${JDK_BOOT_VERSION}";; + *) export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK11_BOOT_DIR}";; esac echo "Boot jdk: ${JDK_BOOT_DIR}" From 520c7e50cac7ebc79adcca3a5652c75b306d821f Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 18 Sep 2018 12:38:22 +0100 Subject: [PATCH 044/101] Fix head build variable detection --- build-farm/make-adopt-build-farm.sh | 6 ++++++ configureBuild.sh | 6 +++++- pipelines/build/create_job_from_template.groovy | 9 +++++++-- 3 files changed, 18 insertions(+), 3 deletions(-) diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index b65c0e390..c8aefa763 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -59,6 +59,12 @@ case "${JDK_BOOT_VERSION}" in *) export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK11_BOOT_DIR}";; esac + +if [ -d "${JDK_BOOT_DIR}" ] +then + export JDK_BOOT_DIR="${JAVA_HOME}" +fi + echo "Boot jdk: ${JDK_BOOT_DIR}" diff --git a/configureBuild.sh b/configureBuild.sh index cc6d60aa0..6440396d1 100755 --- a/configureBuild.sh +++ b/configureBuild.sh @@ -121,7 +121,11 @@ setVariablesForConfigure() { # TODO Regex this in the if or use cut to grab out the number and see if >= 9 # TODO 9 should become 9u as will 10 shortly.... - if [ "$openjdk_core_version" == "${JDK9_CORE_VERSION}" ] || [ "$openjdk_core_version" == "${JDK10_CORE_VERSION}" ] || [ "$openjdk_core_version" == "${JDK11_CORE_VERSION}" ] || [ "$openjdk_core_version" == "${AMBER_CORE_VERSION}" ]; then + if [ "$openjdk_core_version" == "${JDK9_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${JDK10_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${JDK11_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${AMBER_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${JDKHEAD_CORE_VERSION}" ]; then local jdk_path="jdk" local jre_path="jre" #BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]:-"--disable-warnings-as-errors"} diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy index 208185223..e7e02c38a 100644 --- a/pipelines/build/create_job_from_template.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -14,6 +14,11 @@ if (!binding.hasVariable('ADDITIONAL_FILE_NAME_TAG')) ADDITIONAL_FILE_NAME_TAG = if (!binding.hasVariable('TEST_CONFIG')) TEST_CONFIG = "" if (!binding.hasVariable('ENABLE_TESTS')) ENABLE_TESTS = "false" + +if (!binding.hasVariable('GIT_URI')) GIT_URI = "https://github.com/AdoptOpenJDK/openjdk-build.git" +if (!binding.hasVariable('GIT_BRANCH')) GIT_BRANCH = "new_build_scripts" + + folder(buildFolder) { description 'Automatically generated build jobs.' } @@ -25,9 +30,9 @@ pipelineJob("$buildFolder/$JOB_NAME") { scm { git { remote { - url('https://github.com/AdoptOpenJDK/openjdk-build.git') + url($GIT_URI) } - branch('*/new_build_scripts') + branch("*/${GIT_BRANCH}") extensions { cleanBeforeCheckout() } From 191e1144f9110d13bbce0e9c1edbbb758fe43956 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 18 Sep 2018 12:40:21 +0100 Subject: [PATCH 045/101] Fix dir detection --- build-farm/make-adopt-build-farm.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index c8aefa763..3f6e08b42 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -60,7 +60,7 @@ case "${JDK_BOOT_VERSION}" in esac -if [ -d "${JDK_BOOT_DIR}" ] +if [ ! -d "${JDK_BOOT_DIR}" ] then export JDK_BOOT_DIR="${JAVA_HOME}" fi From 4ecb3f80c59745e32e51b4b6c695e4248590ab97 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 18 Sep 2018 12:41:12 +0100 Subject: [PATCH 046/101] Tidy --- configureBuild.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/configureBuild.sh b/configureBuild.sh index 6440396d1..9f4978ca5 100755 --- a/configureBuild.sh +++ b/configureBuild.sh @@ -122,10 +122,10 @@ setVariablesForConfigure() { # TODO Regex this in the if or use cut to grab out the number and see if >= 9 # TODO 9 should become 9u as will 10 shortly.... if [ "$openjdk_core_version" == "${JDK9_CORE_VERSION}" ] || \ - [ "$openjdk_core_version" == "${JDK10_CORE_VERSION}" ] || \ - [ "$openjdk_core_version" == "${JDK11_CORE_VERSION}" ] || \ - [ "$openjdk_core_version" == "${AMBER_CORE_VERSION}" ] || \ - [ "$openjdk_core_version" == "${JDKHEAD_CORE_VERSION}" ]; then + [ "$openjdk_core_version" == "${JDK10_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${JDK11_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${AMBER_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${JDKHEAD_CORE_VERSION}" ]; then local jdk_path="jdk" local jre_path="jre" #BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]:-"--disable-warnings-as-errors"} From b41c5634d9dca894f3ab5050da2840ec9c67a52b Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 18 Sep 2018 13:33:22 +0100 Subject: [PATCH 047/101] Fix head build variable detection (#525) * Fix import method * Lib test * Lib test * Fix lib import yet again * Try to fix cacerts * Fix mac tar not supporting wildcards * Match specifically * Remove wildcard * Reduce download by 40mb by not being an idiot * Fix jdk build * Fix jdk build * Fix head build variable detection * Fix dir detection * Tidy --- build-farm/make-adopt-build-farm.sh | 6 ++++++ configureBuild.sh | 6 +++++- pipelines/build/create_job_from_template.groovy | 9 +++++++-- 3 files changed, 18 insertions(+), 3 deletions(-) diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index b65c0e390..3f6e08b42 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -59,6 +59,12 @@ case "${JDK_BOOT_VERSION}" in *) export JDK_BOOT_DIR="${JDK_BOOT_DIR:-$JDK11_BOOT_DIR}";; esac + +if [ ! -d "${JDK_BOOT_DIR}" ] +then + export JDK_BOOT_DIR="${JAVA_HOME}" +fi + echo "Boot jdk: ${JDK_BOOT_DIR}" diff --git a/configureBuild.sh b/configureBuild.sh index cc6d60aa0..9f4978ca5 100755 --- a/configureBuild.sh +++ b/configureBuild.sh @@ -121,7 +121,11 @@ setVariablesForConfigure() { # TODO Regex this in the if or use cut to grab out the number and see if >= 9 # TODO 9 should become 9u as will 10 shortly.... - if [ "$openjdk_core_version" == "${JDK9_CORE_VERSION}" ] || [ "$openjdk_core_version" == "${JDK10_CORE_VERSION}" ] || [ "$openjdk_core_version" == "${JDK11_CORE_VERSION}" ] || [ "$openjdk_core_version" == "${AMBER_CORE_VERSION}" ]; then + if [ "$openjdk_core_version" == "${JDK9_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${JDK10_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${JDK11_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${AMBER_CORE_VERSION}" ] || \ + [ "$openjdk_core_version" == "${JDKHEAD_CORE_VERSION}" ]; then local jdk_path="jdk" local jre_path="jre" #BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]=${BUILD_CONFIG[CONFIGURE_ARGS_FOR_ANY_PLATFORM]:-"--disable-warnings-as-errors"} diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy index 208185223..e7e02c38a 100644 --- a/pipelines/build/create_job_from_template.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -14,6 +14,11 @@ if (!binding.hasVariable('ADDITIONAL_FILE_NAME_TAG')) ADDITIONAL_FILE_NAME_TAG = if (!binding.hasVariable('TEST_CONFIG')) TEST_CONFIG = "" if (!binding.hasVariable('ENABLE_TESTS')) ENABLE_TESTS = "false" + +if (!binding.hasVariable('GIT_URI')) GIT_URI = "https://github.com/AdoptOpenJDK/openjdk-build.git" +if (!binding.hasVariable('GIT_BRANCH')) GIT_BRANCH = "new_build_scripts" + + folder(buildFolder) { description 'Automatically generated build jobs.' } @@ -25,9 +30,9 @@ pipelineJob("$buildFolder/$JOB_NAME") { scm { git { remote { - url('https://github.com/AdoptOpenJDK/openjdk-build.git') + url($GIT_URI) } - branch('*/new_build_scripts') + branch("*/${GIT_BRANCH}") extensions { cleanBeforeCheckout() } From 94c5a47a1ceb253bbbf5a0ddffad80667d4ccf6d Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 18 Sep 2018 13:40:40 +0100 Subject: [PATCH 048/101] Fix typo --- pipelines/build/create_job_from_template.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy index e7e02c38a..d2af09733 100644 --- a/pipelines/build/create_job_from_template.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -30,7 +30,7 @@ pipelineJob("$buildFolder/$JOB_NAME") { scm { git { remote { - url($GIT_URI) + url(GIT_URI) } branch("*/${GIT_BRANCH}") extensions { From ef31eb3746d5e86d7e490e2340e636191f413dfb Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 18 Sep 2018 13:41:41 +0100 Subject: [PATCH 049/101] Fix typo --- pipelines/build/create_job_from_template.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy index e7e02c38a..d2af09733 100644 --- a/pipelines/build/create_job_from_template.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -30,7 +30,7 @@ pipelineJob("$buildFolder/$JOB_NAME") { scm { git { remote { - url($GIT_URI) + url(GIT_URI) } branch("*/${GIT_BRANCH}") extensions { From f6e476ab6c9f76b0923ad1a80c3afad83e9dc949 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 19 Sep 2018 11:24:09 +0100 Subject: [PATCH 050/101] Move java 11 ppcle to ubuntu nodes --- pipelines/build/openjdk11_pipeline.groovy | 2 +- pipelines/build/openjdk_pipeline.groovy | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index bf5978fe9..a6ac51b2f 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -49,7 +49,7 @@ def buildConfigurations = [ ppc64leLinux : [ os : 'linux', arch : 'ppc64le', - additionalNodeLabels: 'centos7', + additionalNodeLabels: 'ubuntu', test : ['openjdktest', 'systemtest'] ], diff --git a/pipelines/build/openjdk_pipeline.groovy b/pipelines/build/openjdk_pipeline.groovy index 4654d79a8..09436c561 100644 --- a/pipelines/build/openjdk_pipeline.groovy +++ b/pipelines/build/openjdk_pipeline.groovy @@ -89,5 +89,6 @@ def javaToBuild = "jdk" node ("master") { checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) + //Tests off for head builds + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, false, publish, releaseTag) } From 5bcec1cb4f336d7ac91b18f3e9c07efb11e50a06 Mon Sep 17 00:00:00 2001 From: Martijn Verburg Date: Wed, 19 Sep 2018 11:26:00 +0100 Subject: [PATCH 051/101] Port PR 528 to new_build_scripts (#530) --- sbin/prepareWorkspace.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index 7c2cda0e6..b1e834f7c 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -65,7 +65,7 @@ checkoutAndCloneOpenJDKGitRepo() if [ ! -z "${BUILD_CONFIG[TAG]}" ]; then git checkout "${BUILD_CONFIG[TAG]}" fi - git clean -fdx + git clean -ffdx elif [ "${BUILD_CONFIG[CLEAN_GIT_REPO]}" == "true" ]; then echo "Removing current git repo as it is the wrong type" rm -rf "${BUILD_CONFIG[WORKSPACE_DIR]:?}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" From 680b587889a83a89034a5716f07e395f0d1d0004 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 19 Sep 2018 11:55:17 +0100 Subject: [PATCH 052/101] Fix arch detection on aix --- pipelines/build/openjdk11_pipeline.groovy | 1 + sbin/common/config_init.sh | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index a6ac51b2f..c8ca4dd25 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -43,6 +43,7 @@ def buildConfigurations = [ s390xLinux : [ os : 'linux', arch : 's390x', + additionalNodeLabels: 'build-marist-rhel74-s390x-2', test : ['openjdktest', 'systemtest'] ], diff --git a/sbin/common/config_init.sh b/sbin/common/config_init.sh index d76b20796..b045a9a91 100755 --- a/sbin/common/config_init.sh +++ b/sbin/common/config_init.sh @@ -253,8 +253,14 @@ function configDefaults() { # The OS kernel name, e.g. 'darwin' for Mac OS X BUILD_CONFIG[OS_KERNEL_NAME]=$(uname | awk '{print tolower($0)}') + local arch=$(uname -m) + + if [ "${BUILD_CONFIG[OS_KERNEL_NAME]}" == "aix" ]; then + arch=$(uname -p | sed 's/powerpc/ppc/') + fi + # The O/S architecture, e.g. x86_64 for a modern intel / Mac OS X - BUILD_CONFIG[OS_ARCHITECTURE]=$(uname -m) + BUILD_CONFIG[OS_ARCHITECTURE]=${arch} # The full forest name, e.g. jdk8, jdk8u, jdk9, jdk9u, etc. BUILD_CONFIG[OPENJDK_FOREST_NAME]="" From 2816e42e810ef812ba3d8cea2c0914d8b3f51fa0 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 19 Sep 2018 12:02:26 +0100 Subject: [PATCH 053/101] Remove pthread flags from java 11 --- build-farm/platform-specific-configurations/aix.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/build-farm/platform-specific-configurations/aix.sh b/build-farm/platform-specific-configurations/aix.sh index d395ea441..b42664f76 100755 --- a/build-farm/platform-specific-configurations/aix.sh +++ b/build-farm/platform-specific-configurations/aix.sh @@ -20,7 +20,13 @@ SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" source "$SCRIPT_DIR/../../sbin/common/constants.sh" export PATH="/opt/freeware/bin:/usr/local/bin:/opt/IBM/xlC/13.1.3/bin:/opt/IBM/xlc/13.1.3/bin:$PATH" -export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-memory-size=18000 --with-cups-include=/opt/freeware/include --with-extra-ldflags=-lpthread --with-extra-cflags=-lpthread --with-extra-cxxflags=-lpthread" +export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-memory-size=18000 --with-cups-include=/opt/freeware/include" + +if [ "${JAVA_TO_BUILD}" != "${JDK11_VERSION}" ] +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-extra-ldflags=-lpthread --with-extra-cflags=-lpthread --with-extra-cxxflags=-lpthread" +fi + export BUILD_ARGS="${BUILD_ARGS} --skip-freetype" if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "openj9" ]; From 442fd6510b89d0d4aec0f8b379e33a466e207293 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 19 Sep 2018 12:34:11 +0100 Subject: [PATCH 054/101] Move java 11 ppcle to ubuntu nodes (#531) * Fix import method * Lib test * Lib test * Fix lib import yet again * Try to fix cacerts * Fix mac tar not supporting wildcards * Match specifically * Remove wildcard * Reduce download by 40mb by not being an idiot * Fix jdk build * Fix jdk build * Fix head build variable detection * Fix dir detection * Tidy * Fix typo * Move java 11 ppcle to ubuntu nodes * Fix arch detection on aix * Remove pthread flags from java 11 --- build-farm/platform-specific-configurations/aix.sh | 8 +++++++- pipelines/build/openjdk11_pipeline.groovy | 3 ++- pipelines/build/openjdk_pipeline.groovy | 3 ++- sbin/common/config_init.sh | 8 +++++++- 4 files changed, 18 insertions(+), 4 deletions(-) diff --git a/build-farm/platform-specific-configurations/aix.sh b/build-farm/platform-specific-configurations/aix.sh index d395ea441..b42664f76 100755 --- a/build-farm/platform-specific-configurations/aix.sh +++ b/build-farm/platform-specific-configurations/aix.sh @@ -20,7 +20,13 @@ SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" source "$SCRIPT_DIR/../../sbin/common/constants.sh" export PATH="/opt/freeware/bin:/usr/local/bin:/opt/IBM/xlC/13.1.3/bin:/opt/IBM/xlc/13.1.3/bin:$PATH" -export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-memory-size=18000 --with-cups-include=/opt/freeware/include --with-extra-ldflags=-lpthread --with-extra-cflags=-lpthread --with-extra-cxxflags=-lpthread" +export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-memory-size=18000 --with-cups-include=/opt/freeware/include" + +if [ "${JAVA_TO_BUILD}" != "${JDK11_VERSION}" ] +then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-extra-ldflags=-lpthread --with-extra-cflags=-lpthread --with-extra-cxxflags=-lpthread" +fi + export BUILD_ARGS="${BUILD_ARGS} --skip-freetype" if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "openj9" ]; diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index bf5978fe9..c8ca4dd25 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -43,13 +43,14 @@ def buildConfigurations = [ s390xLinux : [ os : 'linux', arch : 's390x', + additionalNodeLabels: 'build-marist-rhel74-s390x-2', test : ['openjdktest', 'systemtest'] ], ppc64leLinux : [ os : 'linux', arch : 'ppc64le', - additionalNodeLabels: 'centos7', + additionalNodeLabels: 'ubuntu', test : ['openjdktest', 'systemtest'] ], diff --git a/pipelines/build/openjdk_pipeline.groovy b/pipelines/build/openjdk_pipeline.groovy index 4654d79a8..09436c561 100644 --- a/pipelines/build/openjdk_pipeline.groovy +++ b/pipelines/build/openjdk_pipeline.groovy @@ -89,5 +89,6 @@ def javaToBuild = "jdk" node ("master") { checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) + //Tests off for head builds + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, false, publish, releaseTag) } diff --git a/sbin/common/config_init.sh b/sbin/common/config_init.sh index d76b20796..b045a9a91 100755 --- a/sbin/common/config_init.sh +++ b/sbin/common/config_init.sh @@ -253,8 +253,14 @@ function configDefaults() { # The OS kernel name, e.g. 'darwin' for Mac OS X BUILD_CONFIG[OS_KERNEL_NAME]=$(uname | awk '{print tolower($0)}') + local arch=$(uname -m) + + if [ "${BUILD_CONFIG[OS_KERNEL_NAME]}" == "aix" ]; then + arch=$(uname -p | sed 's/powerpc/ppc/') + fi + # The O/S architecture, e.g. x86_64 for a modern intel / Mac OS X - BUILD_CONFIG[OS_ARCHITECTURE]=$(uname -m) + BUILD_CONFIG[OS_ARCHITECTURE]=${arch} # The full forest name, e.g. jdk8, jdk8u, jdk9, jdk9u, etc. BUILD_CONFIG[OPENJDK_FOREST_NAME]="" From dcd8b00825b7e9cf37c276d6e03ffe2500916e5d Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 19 Sep 2018 14:17:55 +0100 Subject: [PATCH 055/101] Call get_source.sh on openj9 always --- sbin/prepareWorkspace.sh | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index b1e834f7c..d983e8b7d 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -81,6 +81,8 @@ checkoutAndCloneOpenJDKGitRepo() cloneOpenJDKGitRepo fi + updateOpenj9Sources + cd "${BUILD_CONFIG[WORKSPACE_DIR]}" } @@ -97,6 +99,14 @@ setGitCloneArguments() { GIT_CLONE_ARGUMENTS=(${BUILD_CONFIG[SHALLOW_CLONE_OPTION]} '-b' "${BUILD_CONFIG[BRANCH]}" "$git_remote_repo_address" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}") } +updateOpenj9Sources() { + # Building OpenJDK with OpenJ9 must run get_source.sh to clone openj9 and openj9-omr repositories + if [ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]; then + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || return + bash get_source.sh + cd "${BUILD_CONFIG[WORKSPACE_DIR]}" + fi +} # Clone the git repo cloneOpenJDKGitRepo() { @@ -108,14 +118,6 @@ cloneOpenJDKGitRepo() cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || exit 1 git checkout "${BUILD_CONFIG[TAG]}" fi - - # TODO extract this to its own function - # Building OpenJDK with OpenJ9 must run get_source.sh to clone openj9 and openj9-omr repositories - if [ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]; then - cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || return - bash get_source.sh - fi - cd "${BUILD_CONFIG[WORKSPACE_DIR]}" } # Create the workspace From 9d642efadf5e26f092eca1bbe87cae33f4a8d025 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 19 Sep 2018 15:22:07 +0100 Subject: [PATCH 056/101] Add java 8 aarch64 (#533) --- pipelines/build/openjdk11_pipeline.groovy | 2 +- pipelines/build/openjdk8_pipeline.groovy | 9 ++++++++- pipelines/build/openjdk9_pipeline.groovy | 2 +- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index c8ca4dd25..f206c4c86 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -75,7 +75,7 @@ def buildConfigurations = [ test : false ], */ - "linuxXL" : [ + linuxXL : [ os : 'linux', additionalNodeLabels : 'centos6', arch : 'x64', diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy index f2ca1b1fa..634ad03a0 100644 --- a/pipelines/build/openjdk8_pipeline.groovy +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -65,7 +65,14 @@ def buildConfigurations = [ test : ['openjdktest'] ], - "linuxXL" : [ + aarch64Linux : [ + os : 'linux', + arch : 'aarch64', + additionalNodeLabels: 'centos7', + test : ['openjdktest'] + ], + + linuxXL : [ os : 'linux', additionalNodeLabels : 'centos6', arch : 'x64', diff --git a/pipelines/build/openjdk9_pipeline.groovy b/pipelines/build/openjdk9_pipeline.groovy index 5f86fa65a..fff94d21c 100644 --- a/pipelines/build/openjdk9_pipeline.groovy +++ b/pipelines/build/openjdk9_pipeline.groovy @@ -68,7 +68,7 @@ def buildConfigurations = [ test : ['openjdktest'] ], - "linuxXL" : [ + linuxXL : [ os : 'linux', additionalNodeLabels : 'centos6', arch : 'x64', From 1de41502d65db706d88e593ea0ed2243db8cce50 Mon Sep 17 00:00:00 2001 From: Stewart X Addison Date: Wed, 19 Sep 2018 10:52:19 -0500 Subject: [PATCH 057/101] Update compilers for OpenJ9 to gcc 7.3 and VS2017 (#535) --- .../platform-specific-configurations/linux.sh | 21 ++++++++++++------- .../windows.sh | 5 ++++- 2 files changed, 18 insertions(+), 8 deletions(-) diff --git a/build-farm/platform-specific-configurations/linux.sh b/build-farm/platform-specific-configurations/linux.sh index 4af4f5ac8..bd8fdd79f 100755 --- a/build-farm/platform-specific-configurations/linux.sh +++ b/build-farm/platform-specific-configurations/linux.sh @@ -83,10 +83,17 @@ then wget -q -O - "https://api.adoptopenjdk.net/v2/binary/releases/openjdk10?os=linux&release=latest&arch=${downloadArch}" | tar xpzf - --strip-components=2 -C "$JDK10_BOOT_DIR" fi export JDK_BOOT_DIR=$JDK10_BOOT_DIR - - [ -r /usr/local/gcc/bin ] && export PATH=/usr/local/gcc/bin:$PATH - [ -r /usr/local/gcc/bin/gcc-7.3 ] && export CC=/usr/local/gcc/bin/gcc-7.3 - [ -r /usr/local/gcc/bin/g++-7.3 ] && export CXX=/usr/local/gcc/bin/g++-7.3 - export LD_LIBRARY_PATH=/usr/local/gcc/lib64:/usr/local/gcc/lib - -fi \ No newline at end of file +fi +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] || [ "${VARIANT}" == "openj9" ] + # If we have the RedHat devtoolset 7 installed, use gcc 7 from there, else /usr/local/gcc/bin + if [ -r /opt/rh/devtoolset-7/root/usr/bin ]; then + export PATH=/opt/rh/devtoolset-7/root/usr/bin:$PATH + [ -r /opt/rh/devtoolset-7/root/usr/bin/gcc ] && export CC=/opt/rh/devtoolset-7/root/usr/bin/gcc + [ -r /opt/rh/devtoolset-7/root/usr/bin/g++ ] && export CC=/opt/rh/devtoolset-7/root/usr/bin/g++ + elif [ -r /usr/local/gcc/bin ]; then + export PATH=/usr/local/gcc/bin:$PATH + [ -r /usr/local/gcc/bin/gcc-7.3 ] && export CC=/usr/local/gcc/bin/gcc-7.3 + [ -r /usr/local/gcc/bin/g++-7.3 ] && export CXX=/usr/local/gcc/bin/g++-7.3 + export LD_LIBRARY_PATH=/usr/local/gcc/lib64:/usr/local/gcc/lib + fi +fi diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 270a00c44..b77bac7d6 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -42,12 +42,13 @@ then elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] then export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013 --disable-ccache" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache" fi if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "openj9" ] then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2017" export PATH="/usr/bin:$PATH" export HAS_AUTOCONF=1 @@ -66,4 +67,6 @@ then then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" fi +else + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013" fi From 026c47ffd4ce1d723e18d8e41c400f91acc47dee Mon Sep 17 00:00:00 2001 From: Stewart X Addison Date: Wed, 19 Sep 2018 11:02:59 -0500 Subject: [PATCH 058/101] Lock JDK11 to our VS2017 machine (tagged openj9 for now) (#537) --- pipelines/build/openjdk11_pipeline.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index f206c4c86..2cbd848af 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -30,7 +30,7 @@ def buildConfigurations = [ x64Windows: [ os : 'windows', arch : 'x64', - additionalNodeLabels: 'win2012', + additionalNodeLabels: 'win2012&&openj9', test : ['openjdktest'] ], From 8e4c84cce8a046d7b768c24bbbca776e4023f731 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 19 Sep 2018 17:41:59 +0100 Subject: [PATCH 059/101] add branch and additionalConfigureArgs to top level pipelines (#534) * Add java 8 aarch64 * add branch and additionalConfigureArgs to top level pipelines * Tidy * fix from review --- build-farm/make-adopt-build-farm.sh | 10 ++++-- configureBuild.sh | 1 - pipelines/build/build_base_file.groovy | 33 +++++++++++++++---- .../build/create_job_from_template.groovy | 1 + pipelines/build/openjdk10_pipeline.groovy | 2 +- pipelines/build/openjdk11_pipeline.groovy | 2 +- pipelines/build/openjdk8_pipeline.groovy | 2 +- pipelines/build/openjdk9_pipeline.groovy | 2 +- sbin/common/config_init.sh | 15 +++++++-- 9 files changed, 51 insertions(+), 17 deletions(-) diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index 3f6e08b42..83a559f1c 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -25,14 +25,13 @@ TIMESTAMP="$(date +'%Y-%m-%d-%H-%M')" export OPERATING_SYSTEM OPERATING_SYSTEM=$(echo "${TARGET_OS}" | tr '[:upper:]' '[:lower:]') -branch="dev" echo "BUILD TYPE: " echo "VERSION: ${JAVA_TO_BUILD}" echo "ARCHITECTURE ${ARCHITECTURE}" echo "VARIANT: ${VARIANT}" echo "OS: ${OPERATING_SYSTEM}" -echo "BRANCH: ${branch}" +echo "BRANCH: ${BRANCH}" OPTIONS="" EXTENSION="" @@ -82,6 +81,11 @@ elif [ "${OPERATING_SYSTEM}" == "windows" ] ; then EXTENSION=zip fi +if [ -z "${BRANCH}" ] +then + OPTIONS="${OPTIONS} -b ${BRANCH}" +fi + # shellcheck source=build-farm/set-platform-specific-configurations.sh source "${PLATFORM_SCRIPT_DIR}/set-platform-specific-configurations.sh" @@ -99,4 +103,4 @@ echo "Filename will be: $FILENAME" export BUILD_ARGS="${BUILD_ARGS} --use-jep319-certs" # shellcheck disable=SC2086 -bash "$PLATFORM_SCRIPT_DIR/../makejdk-any-platform.sh" --clean-git-repo -b "${branch}" --jdk-boot-dir "${JDK_BOOT_DIR}" --configure-args "${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" --target-file-name "${FILENAME}" ${TAG_OPTION} ${OPTIONS} ${BUILD_ARGS} ${VARIANT_ARG} "${JAVA_TO_BUILD}" +bash "$PLATFORM_SCRIPT_DIR/../makejdk-any-platform.sh" --clean-git-repo --jdk-boot-dir "${JDK_BOOT_DIR}" --configure-args "${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" --target-file-name "${FILENAME}" ${TAG_OPTION} ${OPTIONS} ${BUILD_ARGS} ${VARIANT_ARG} "${JAVA_TO_BUILD}" diff --git a/configureBuild.sh b/configureBuild.sh index 9f4978ca5..7bc49d082 100755 --- a/configureBuild.sh +++ b/configureBuild.sh @@ -73,7 +73,6 @@ doAnyBuildVariantOverrides() if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]]; then # current location of Extensions for OpenJDK9 for OpenJ9 project local repository="ibmruntimes/openj9-openjdk-${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" - local branch="openj9" fi if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" == "SapMachine" ]]; then # current location of SAP variant diff --git a/pipelines/build/build_base_file.groovy b/pipelines/build/build_base_file.groovy index 9b4d99bfb..3b2d2a3a8 100644 --- a/pipelines/build/build_base_file.groovy +++ b/pipelines/build/build_base_file.groovy @@ -39,7 +39,7 @@ def toBuildParams(enableTests, params) { return buildParams } -static def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { +static def buildConfiguration(javaToBuild, variant, configuration, releaseTag, branch, additionalConfigureArgs) { String buildTag = "build" @@ -57,7 +57,6 @@ static def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { } else { additionalNodeLabels = configuration.additionalNodeLabels } - } if (additionalNodeLabels != null) { @@ -75,10 +74,15 @@ static def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { ] if (configuration.containsKey('bootJDK')) buildParams.put("JDK_BOOT_VERSION", configuration.bootJDK) - if (configuration.containsKey('configureArgs')) buildParams.put("CONFIGURE_ARGS", configuration.configureArgs) if (configuration.containsKey('buildArgs')) buildParams.put("BUILD_ARGS", configuration.buildArgs) if (configuration.containsKey('additionalFileNameTag')) buildParams.put("ADDITIONAL_FILE_NAME_TAG", configuration.additionalFileNameTag) + buildParams.putAll(getConfigureArgs(configuration, additionalConfigureArgs)) + + if (branch != null && branch.length() > 0) { + buildParams.put("BRANCH", branch) + } + if (releaseTag != null && releaseTag.length() > 0) { buildParams.put("TAG", releaseTag) } @@ -93,7 +97,22 @@ static def buildConfiguration(javaToBuild, variant, configuration, releaseTag) { ] } -def getJobConfigurations(javaVersionToBuild, availableConfigurations, String targetConfigurations, String releaseTag) { +static def getConfigureArgs(configuration, additionalConfigureArgs) { + def buildParams = [:] + def configureArgs = ""; + + if (configuration.containsKey('configureArgs')) configureArgs += configuration.configureArgs; + if (additionalConfigureArgs != null && additionalConfigureArgs.length() > 0) { + configureArgs += " " + additionalConfigureArgs + } + + if (configureArgs.length() > 0) { + buildParams.put("CONFIGURE_ARGS", configureArgs) + } + return buildParams +} + +def getJobConfigurations(javaVersionToBuild, availableConfigurations, String targetConfigurations, String releaseTag, String branch, String additionalConfigureArgs) { def jobConfigurations = [:] //Parse config passed to jenkins job @@ -109,7 +128,7 @@ def getJobConfigurations(javaVersionToBuild, availableConfigurations, String tar if (configuration.containsKey('additionalFileNameTag')) { name += "-${configuration.additionalFileNameTag}" } - jobConfigurations[name] = buildConfiguration(javaVersionToBuild, variant, configuration, releaseTag) + jobConfigurations[name] = buildConfiguration(javaVersionToBuild, variant, configuration, releaseTag, branch, additionalConfigureArgs) } } } @@ -172,13 +191,13 @@ def publishRelease(javaToBuild, releaseTag) { } } -def doBuild(String javaVersionToBuild, availableConfigurations, String targetConfigurations, String enableTestsArg, String publishArg, String releaseTag) { +def doBuild(String javaVersionToBuild, availableConfigurations, String targetConfigurations, String enableTestsArg, String publishArg, String releaseTag, String branch, String additionalConfigureArgs) { if (releaseTag == null || releaseTag == "false") { releaseTag = "" } - def jobConfigurations = getJobConfigurations(javaVersionToBuild, availableConfigurations, targetConfigurations, releaseTag) + def jobConfigurations = getJobConfigurations(javaVersionToBuild, availableConfigurations, targetConfigurations, releaseTag, branch, additionalConfigureArgs) def jobs = [:] def enableTests = enableTestsArg == "true" diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy index d2af09733..eed6fa06c 100644 --- a/pipelines/build/create_job_from_template.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -62,6 +62,7 @@ pipelineJob("$buildFolder/$JOB_NAME") { stringParam('TARGET_OS', "$TARGET_OS") stringParam('ADDITIONAL_FILE_NAME_TAG', "$ADDITIONAL_FILE_NAME_TAG") stringParam('ENABLE_TESTS', "$ENABLE_TESTS") + stringParam('BRANCH', "$BRANCH") textParam('TEST_CONFIG', "$TEST_CONFIG") } } \ No newline at end of file diff --git a/pipelines/build/openjdk10_pipeline.groovy b/pipelines/build/openjdk10_pipeline.groovy index eec1f6267..4e8fb1e46 100644 --- a/pipelines/build/openjdk10_pipeline.groovy +++ b/pipelines/build/openjdk10_pipeline.groovy @@ -97,5 +97,5 @@ def javaToBuild = "jdk10u" node ("master") { checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs) } diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index 2cbd848af..ea02628f5 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -90,5 +90,5 @@ def javaToBuild = "jdk11" node ("master") { checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs) } diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy index 634ad03a0..87cfcc34e 100644 --- a/pipelines/build/openjdk8_pipeline.groovy +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -87,6 +87,6 @@ def javaToBuild = "jdk8u" node ("master") { checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs) } diff --git a/pipelines/build/openjdk9_pipeline.groovy b/pipelines/build/openjdk9_pipeline.groovy index fff94d21c..cc1c536eb 100644 --- a/pipelines/build/openjdk9_pipeline.groovy +++ b/pipelines/build/openjdk9_pipeline.groovy @@ -83,5 +83,5 @@ def javaToBuild = "jdk9u" node ("master") { checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs) } diff --git a/sbin/common/config_init.sh b/sbin/common/config_init.sh index b045a9a91..4b2e88af3 100755 --- a/sbin/common/config_init.sh +++ b/sbin/common/config_init.sh @@ -246,6 +246,19 @@ function parseConfigurationArguments() { *) echo >&2 "Invalid build.sh option: ${opt}"; exit 1;; esac done + + setBranch +} + +function setBranch() { + + # Which repo branch to build, e.g. dev by default for hotspot, "openj9" for openj9 + local branch="dev" + if [ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]; then + branch="openj9"; + fi + + BUILD_CONFIG[BRANCH]=${BUILD_CONFIG[BRANCH]:-$branch} } # Set the config defaults @@ -328,8 +341,6 @@ function configDefaults() { # Director where OpenJDK binary gets built to BUILD_CONFIG[TARGET_DIR]=${BUILD_CONFIG[TARGET_DIR]:-"target/"} - # Which repo branch to build, e.g. dev - BUILD_CONFIG[BRANCH]=${BUILD_CONFIG[BRANCH]:-"dev"} # Which repo tag to build, e.g. jdk8u172-b03 BUILD_CONFIG[TAG]=${BUILD_CONFIG[TAG]:-""} From 6d55c9d10bbe8a1fef82325a686c806fe907d764 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 19 Sep 2018 17:47:54 +0100 Subject: [PATCH 060/101] Fix if (#538) --- build-farm/platform-specific-configurations/linux.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build-farm/platform-specific-configurations/linux.sh b/build-farm/platform-specific-configurations/linux.sh index bd8fdd79f..3bdc9001f 100755 --- a/build-farm/platform-specific-configurations/linux.sh +++ b/build-farm/platform-specific-configurations/linux.sh @@ -84,7 +84,7 @@ then fi export JDK_BOOT_DIR=$JDK10_BOOT_DIR fi -if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] || [ "${VARIANT}" == "openj9" ] +if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] || [ "${VARIANT}" == "openj9" ]; then # If we have the RedHat devtoolset 7 installed, use gcc 7 from there, else /usr/local/gcc/bin if [ -r /opt/rh/devtoolset-7/root/usr/bin ]; then export PATH=/opt/rh/devtoolset-7/root/usr/bin:$PATH From 79abed9841e852cf5c399de86bbd05bd235d9f17 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 19 Sep 2018 18:47:39 +0100 Subject: [PATCH 061/101] Fix branch set test (#539) * Add java 8 aarch64 * add branch and additionalConfigureArgs to top level pipelines * Tidy * fix from review * fix if * fix branch test --- build-farm/make-adopt-build-farm.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index 83a559f1c..87618be9b 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -81,7 +81,7 @@ elif [ "${OPERATING_SYSTEM}" == "windows" ] ; then EXTENSION=zip fi -if [ -z "${BRANCH}" ] +if [ ! -z "${BRANCH}" ] then OPTIONS="${OPTIONS} -b ${BRANCH}" fi From b6cc029513164e96df1e4dadac00cf4507b5bb52 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Thu, 20 Sep 2018 13:00:37 +0100 Subject: [PATCH 062/101] Allow build to be run on any branch (#540) --- pipelines/build/build_base_file.groovy | 9 +++++--- .../build/create_job_from_template.groovy | 2 +- pipelines/build/openjdk10_pipeline.groovy | 4 ++-- pipelines/build/openjdk11_pipeline.groovy | 4 ++-- pipelines/build/openjdk8_pipeline.groovy | 4 ++-- pipelines/build/openjdk9_pipeline.groovy | 4 ++-- pipelines/build/openjdk_pipeline.groovy | 4 ++-- sbin/prepareWorkspace.sh | 22 ++++++++----------- 8 files changed, 26 insertions(+), 27 deletions(-) diff --git a/pipelines/build/build_base_file.groovy b/pipelines/build/build_base_file.groovy index 3b2d2a3a8..102acc47d 100644 --- a/pipelines/build/build_base_file.groovy +++ b/pipelines/build/build_base_file.groovy @@ -158,13 +158,16 @@ static def getJobFolder(config) { } // Generate a job from template at `create_job_from_template.groovy` -def createJob(jobName, jobFolder, config, enableTests) { +def createJob(jobName, jobFolder, config, enableTests, scmVars) { def params = config.parameters.clone() params.put("JOB_NAME", jobName) params.put("JOB_FOLDER", jobFolder) params.put("TEST_CONFIG", JsonOutput.prettyPrint(JsonOutput.toJson(config))) + params.put("GIT_URI", scmVars["GIT_URL"]) + params.put("GIT_BRANCH", scmVars["GIT_BRANCH"]) + create = jobDsl targets: "pipelines/build/create_job_from_template.groovy", ignoreExisting: false, additionalParameters: params return create @@ -191,7 +194,7 @@ def publishRelease(javaToBuild, releaseTag) { } } -def doBuild(String javaVersionToBuild, availableConfigurations, String targetConfigurations, String enableTestsArg, String publishArg, String releaseTag, String branch, String additionalConfigureArgs) { +def doBuild(String javaVersionToBuild, availableConfigurations, String targetConfigurations, String enableTestsArg, String publishArg, String releaseTag, String branch, String additionalConfigureArgs, scmVars) { if (releaseTag == null || releaseTag == "false") { releaseTag = "" @@ -226,7 +229,7 @@ def doBuild(String javaVersionToBuild, availableConfigurations, String targetCon // Execute build job for configuration i.e jdk10u/job/jdk10u-linux-x64-hotspot stage(configuration.key) { // generate job - createJob(jobTopName, jobFolder, config, enableTests) + createJob(jobTopName, jobFolder, config, enableTests, scmVars) // execute build def downstreamJob = build job: downstreamJobName, propagate: false, parameters: toBuildParams(enableTests, config.parameters) diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy index eed6fa06c..daba65084 100644 --- a/pipelines/build/create_job_from_template.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -32,7 +32,7 @@ pipelineJob("$buildFolder/$JOB_NAME") { remote { url(GIT_URI) } - branch("*/${GIT_BRANCH}") + branch("${GIT_BRANCH}") extensions { cleanBeforeCheckout() } diff --git a/pipelines/build/openjdk10_pipeline.groovy b/pipelines/build/openjdk10_pipeline.groovy index 4e8fb1e46..757a3c9c2 100644 --- a/pipelines/build/openjdk10_pipeline.groovy +++ b/pipelines/build/openjdk10_pipeline.groovy @@ -95,7 +95,7 @@ def buildConfigurations = [ def javaToBuild = "jdk10u" node ("master") { - checkout scm + def scmVars = checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs, scmVars) } diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index ea02628f5..cbf66ea37 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -88,7 +88,7 @@ def buildConfigurations = [ def javaToBuild = "jdk11" node ("master") { - checkout scm + def scmVars = checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs, scmVars) } diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy index 87cfcc34e..e9b1c3865 100644 --- a/pipelines/build/openjdk8_pipeline.groovy +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -85,8 +85,8 @@ def buildConfigurations = [ def javaToBuild = "jdk8u" node ("master") { - checkout scm + def scmVars = checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs, scmVars) } diff --git a/pipelines/build/openjdk9_pipeline.groovy b/pipelines/build/openjdk9_pipeline.groovy index cc1c536eb..c11e0c38d 100644 --- a/pipelines/build/openjdk9_pipeline.groovy +++ b/pipelines/build/openjdk9_pipeline.groovy @@ -81,7 +81,7 @@ def buildConfigurations = [ def javaToBuild = "jdk9u" node ("master") { - checkout scm + def scmVars = checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" - buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, enableTests, publish, releaseTag, branch, additionalConfigureArgs, scmVars) } diff --git a/pipelines/build/openjdk_pipeline.groovy b/pipelines/build/openjdk_pipeline.groovy index 09436c561..987f03442 100644 --- a/pipelines/build/openjdk_pipeline.groovy +++ b/pipelines/build/openjdk_pipeline.groovy @@ -87,8 +87,8 @@ def buildConfigurations = [ def javaToBuild = "jdk" node ("master") { - checkout scm + def scmVars = checkout scm def buildFile = load "${WORKSPACE}/pipelines/build/build_base_file.groovy" //Tests off for head builds - buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, false, publish, releaseTag) + buildFile.doBuild(javaToBuild, buildConfigurations, targetConfigurations, false, publish, releaseTag, branch, additionalConfigureArgs, scmVars) } diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index d983e8b7d..00a171b2c 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -44,7 +44,6 @@ checkoutAndCloneOpenJDKGitRepo() # Check that we have a git repo of a valid openjdk version on our local file system if [ -d "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" ] && ( [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK10_CORE_VERSION}" ] || [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK11_CORE_VERSION}" ]) ; then - set +e git --git-dir "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}/.git" remote -v echo "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" @@ -52,20 +51,12 @@ checkoutAndCloneOpenJDKGitRepo() local isCorrectGitRepo=$? set -e - # If the local copy of the git source repo is valid then we reset appropriately if [ "${isCorrectGitRepo}" == "0" ]; then cd "${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || return echo "Resetting the git openjdk source repository at $PWD in 10 seconds..." sleep 10 echo "Pulling latest changes from git openjdk source repository" - - git fetch --all ${BUILD_CONFIG[SHALLOW_CLONE_OPTION]} - git reset --hard "origin/${BUILD_CONFIG[BRANCH]}" - if [ ! -z "${BUILD_CONFIG[TAG]}" ]; then - git checkout "${BUILD_CONFIG[TAG]}" - fi - git clean -ffdx elif [ "${BUILD_CONFIG[CLEAN_GIT_REPO]}" == "true" ]; then echo "Removing current git repo as it is the wrong type" rm -rf "${BUILD_CONFIG[WORKSPACE_DIR]:?}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" @@ -81,6 +72,15 @@ checkoutAndCloneOpenJDKGitRepo() cloneOpenJDKGitRepo fi + cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" + git remote set-branches --add origin "${BUILD_CONFIG[BRANCH]}" + git fetch --all ${BUILD_CONFIG[SHALLOW_CLONE_OPTION]} + git reset --hard "origin/${BUILD_CONFIG[BRANCH]}" + if [ ! -z "${BUILD_CONFIG[TAG]}" ]; then + git checkout "${BUILD_CONFIG[TAG]}" + fi + git clean -ffdx + updateOpenj9Sources cd "${BUILD_CONFIG[WORKSPACE_DIR]}" @@ -114,10 +114,6 @@ cloneOpenJDKGitRepo() echo "git clone ${GIT_CLONE_ARGUMENTS[*]}" git clone "${GIT_CLONE_ARGUMENTS[@]}" - if [ ! -z "${BUILD_CONFIG[TAG]}" ]; then - cd "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" || exit 1 - git checkout "${BUILD_CONFIG[TAG]}" - fi } # Create the workspace From 62d54a22a2d613640d6a589d3422d01f2d494b38 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Thu, 20 Sep 2018 13:08:08 +0100 Subject: [PATCH 063/101] Default BRANCH to empty (#541) --- pipelines/build/create_job_from_template.groovy | 1 + 1 file changed, 1 insertion(+) diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy index daba65084..240615b51 100644 --- a/pipelines/build/create_job_from_template.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -13,6 +13,7 @@ if (!binding.hasVariable('BUILD_ARGS')) BUILD_ARGS = "" if (!binding.hasVariable('ADDITIONAL_FILE_NAME_TAG')) ADDITIONAL_FILE_NAME_TAG = "" if (!binding.hasVariable('TEST_CONFIG')) TEST_CONFIG = "" if (!binding.hasVariable('ENABLE_TESTS')) ENABLE_TESTS = "false" +if (!binding.hasVariable('BRANCH')) BRANCH = "" if (!binding.hasVariable('GIT_URI')) GIT_URI = "https://github.com/AdoptOpenJDK/openjdk-build.git" From b6551092aab3a434c289eba2e9f9404ab39b4ada Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 11:34:51 +0100 Subject: [PATCH 064/101] WIP- Initial test of versions fix (#542) Initial test of versions fix --- sbin/build.sh | 63 +++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 51 insertions(+), 12 deletions(-) diff --git a/sbin/build.sh b/sbin/build.sh index 65f606743..e11e94752 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -166,26 +166,65 @@ getOpenJDKUpdateAndBuildVersion() # OpenJDK 64-Bit Server VM (build 25.71-b00, mixed mode) configuringVersionStringParameter() { + stepIntoTheWorkingDirectory + + if [ -z "${BUILD_CONFIG[TAG]}" ]; then + OPENJDK_REPO_TAG=$(getFirstTagFromOpenJDKGitRepo) + echo "OpenJDK repo tag is ${OPENJDK_REPO_TAG}" + fi + + addConfigureArg "--with-milestone=" "fcs" + local dateSuffix=$(date +%Y%m%d) + if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ]; then - # Replace the default 'internal' with our own milestone string - addConfigureArg "--with-milestone=" "adoptopenjdk" + + if [ -z "${BUILD_CONFIG[TAG]}" ]; then + addConfigureArg "--with-user-release-suffix=" "${dateSuffix}" + fi + + addConfigureArg "--with-company-name=" "AdoptOpenJDK" # Set the update version (e.g. 131), this gets passed in from the calling script - addConfigureArgIfValueIsNotEmpty "--with-update-version=" "${BUILD_CONFIG[OPENJDK_UPDATE_VERSION]}" + local updateNumber=${BUILD_CONFIG[OPENJDK_UPDATE_VERSION]} + if [ -z "${updateNumber}" ]; then + updateNumber=$(echo "${OPENJDK_REPO_TAG}" | cut -f1 -d"-" | cut -f2 -d"u") + fi + addConfigureArgIfValueIsNotEmpty "--with-update-version=" "${updateNumber}" # Set the build number (e.g. b04), this gets passed in from the calling script - addConfigureArgIfValueIsNotEmpty "--with-build-number=" "${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]}" - else - if [ -z "$OPENJDK_REPO_TAG" ]; then - cd "${WORKING_DIR}/${OPENJDK_REPO_NAME}" || echo Cannot change to "${WORKING_DIR}/${OPENJDK_REPO_NAME}" - OPENJDK_REPO_TAG=$(getFirstTagFromOpenJDKGitRepo) - echo "OpenJDK repo tag is ${OPENJDK_REPO_TAG}" + local buildNumber=${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]} + if [ -z "${buildNumber}" ]; then + buildNumber=$(echo "$OPENJDK_REPO_TAG" | cut -f2 -d"-") fi - # > JDK 8 - addConfigureArg "--with-version-pre=" "adoptopenjdk" + addConfigureArgIfValueIsNotEmpty "--with-build-number=" "${buildNumber}" + elif [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ]; then - TRIMMED_TAG=$(echo "$OPENJDK_REPO_TAG" | cut -f2 -d"-" ) + TRIMMED_TAG=$(echo "${OPENJDK_REPO_TAG}" | cut -f2 -d"-" ) + + if [ -z "${BUILD_CONFIG[TAG]}" ]; then + TRIMMED_TAG="${TRIMMED_TAG}-${dateSuffix}" + fi + + addConfigureArg "--without-version-pre" "" + addConfigureArg "--without-version-opt" "" addConfigureArg "--with-version-string=" "${TRIMMED_TAG}" + else + # > JDK 9 + + # Set the build number (e.g. b04), this gets passed in from the calling script + local buildNumber=${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]} + if [ -z "${buildNumber}" ]; then + buildNumber=$(echo "${OPENJDK_REPO_TAG}" | cut -f2 -d"+") + fi + + if [ -z "${BUILD_CONFIG[TAG]}" ]; then + buildNumber="${buildNumber}-${dateSuffix}" + fi + + addConfigureArg "--without-version-pre" "" + addConfigureArg "--without-version-opt" "" + addConfigureArgIfValueIsNotEmpty "--with-version-build=" "${buildNumber}" + addConfigureArg "--with-vendor-version-string=" "AdoptOpenJDK" fi echo "Completed configuring the version string parameter, config args are now: ${CONFIGURE_ARGS}" From 53919c3ceb0d5f72b7eef47e7acc6ea946ee9f16 Mon Sep 17 00:00:00 2001 From: Stewart X Addison Date: Fri, 21 Sep 2018 05:38:10 -0500 Subject: [PATCH 065/101] fix environment variable for g++7 on Linux/x64 (#543) --- build-farm/platform-specific-configurations/linux.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build-farm/platform-specific-configurations/linux.sh b/build-farm/platform-specific-configurations/linux.sh index 3bdc9001f..61400feaf 100755 --- a/build-farm/platform-specific-configurations/linux.sh +++ b/build-farm/platform-specific-configurations/linux.sh @@ -89,7 +89,7 @@ if [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JD if [ -r /opt/rh/devtoolset-7/root/usr/bin ]; then export PATH=/opt/rh/devtoolset-7/root/usr/bin:$PATH [ -r /opt/rh/devtoolset-7/root/usr/bin/gcc ] && export CC=/opt/rh/devtoolset-7/root/usr/bin/gcc - [ -r /opt/rh/devtoolset-7/root/usr/bin/g++ ] && export CC=/opt/rh/devtoolset-7/root/usr/bin/g++ + [ -r /opt/rh/devtoolset-7/root/usr/bin/g++ ] && export CXX=/opt/rh/devtoolset-7/root/usr/bin/g++ elif [ -r /usr/local/gcc/bin ]; then export PATH=/usr/local/gcc/bin:$PATH [ -r /usr/local/gcc/bin/gcc-7.3 ] && export CC=/usr/local/gcc/bin/gcc-7.3 From 51c1e975260f67d5945309e46ef0323f8bc8c8f1 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 12:22:29 +0100 Subject: [PATCH 066/101] Fix versions for java 10+ and openj9 (#544) * Add java 8 aarch64 * add branch and additionalConfigureArgs to top level pipelines * Tidy * fix from review * fix if * fix branch test * Allow build to be run on any branch * Move to correct dir * Quick test * Quick test * Quick test * Automatically set git repo location * Fix branch * Default branch to empty * Try to fix version numbers * Fix path * Fix build number for 10+, and openj9 --- sbin/build.sh | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/sbin/build.sh b/sbin/build.sh index e11e94752..104341196 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -152,7 +152,6 @@ getOpenJDKUpdateAndBuildVersion() openjdk_update_version=$(echo "${OPENJDK_REPO_TAG}" | cut -d'u' -f 2 | cut -d'-' -f 1) # TODO dont modify config in build script - BUILD_CONFIG[OPENJDK_BUILD_NUMBER]=$(echo "${OPENJDK_REPO_TAG}" | cut -d'b' -f 2 | cut -d'-' -f 1) echo "Version: ${openjdk_update_version} ${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]}" fi @@ -174,7 +173,7 @@ configuringVersionStringParameter() fi addConfigureArg "--with-milestone=" "fcs" - local dateSuffix=$(date +%Y%m%d) + local dateSuffix=$(date -u +%Y%m%d%H%M) if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ]; then @@ -182,7 +181,9 @@ configuringVersionStringParameter() addConfigureArg "--with-user-release-suffix=" "${dateSuffix}" fi - addConfigureArg "--with-company-name=" "AdoptOpenJDK" + if [ "${BUILD_CONFIG[BUILD_VARIANT]}" == "hotspot" ]; then + addConfigureArg "--with-company-name=" "AdoptOpenJDK" + fi # Set the update version (e.g. 131), this gets passed in from the calling script local updateNumber=${BUILD_CONFIG[OPENJDK_UPDATE_VERSION]} @@ -218,7 +219,7 @@ configuringVersionStringParameter() fi if [ -z "${BUILD_CONFIG[TAG]}" ]; then - buildNumber="${buildNumber}-${dateSuffix}" + buildNumber="${dateSuffix}" fi addConfigureArg "--without-version-pre" "" From fc58e472f9c325bae42e8c5ac61033c972b0b7c0 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 12:36:15 +0100 Subject: [PATCH 067/101] Switch to curl --- sbin/prepareWorkspace.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index 00a171b2c..a9983f5e3 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -240,7 +240,7 @@ checkingAndDownloadCaCerts() echo "Requested use of JEP319 certs" local caLink="https://github.com/AdoptOpenJDK/openjdk-jdk10u/blob/dev/src/java.base/share/lib/security/cacerts?raw=true"; mkdir -p "security" - wget -O "./security/cacerts" "${caLink}" + curl -L -o "./security/cacerts" "${caLink}" fi else git init From 6e231c7b86d0b13c00c913dc335023aa9f28ebc2 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 12:38:29 +0100 Subject: [PATCH 068/101] Switch to curl (#545) --- sbin/prepareWorkspace.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index 00a171b2c..a9983f5e3 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -240,7 +240,7 @@ checkingAndDownloadCaCerts() echo "Requested use of JEP319 certs" local caLink="https://github.com/AdoptOpenJDK/openjdk-jdk10u/blob/dev/src/java.base/share/lib/security/cacerts?raw=true"; mkdir -p "security" - wget -O "./security/cacerts" "${caLink}" + curl -L -o "./security/cacerts" "${caLink}" fi else git init From 1cdb091306f11fc3fa6c51b32458d79429bc3a54 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 12:51:41 +0100 Subject: [PATCH 069/101] Take 2 at adding datestamp to version --- sbin/build.sh | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/sbin/build.sh b/sbin/build.sh index e11e94752..c86559839 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -198,16 +198,21 @@ configuringVersionStringParameter() fi addConfigureArgIfValueIsNotEmpty "--with-build-number=" "${buildNumber}" elif [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ]; then + local buildNumber=${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]} + if [ -z "${buildNumber}" ]; then + buildNumber=$(echo "${OPENJDK_REPO_TAG}" | cut -f2 -d"+") + fi TRIMMED_TAG=$(echo "${OPENJDK_REPO_TAG}" | cut -f2 -d"-" ) if [ -z "${BUILD_CONFIG[TAG]}" ]; then - TRIMMED_TAG="${TRIMMED_TAG}-${dateSuffix}" + addConfigureArg "--with-version-opt" "${dateSuffix}" + else + addConfigureArg "--without-version-opt" "" fi addConfigureArg "--without-version-pre" "" - addConfigureArg "--without-version-opt" "" - addConfigureArg "--with-version-string=" "${TRIMMED_TAG}" + addConfigureArgIfValueIsNotEmpty "--with-version-build=" "${buildNumber}" else # > JDK 9 @@ -218,11 +223,12 @@ configuringVersionStringParameter() fi if [ -z "${BUILD_CONFIG[TAG]}" ]; then - buildNumber="${buildNumber}-${dateSuffix}" + addConfigureArg "--with-version-opt" "${dateSuffix}" + else + addConfigureArg "--without-version-opt" "" fi addConfigureArg "--without-version-pre" "" - addConfigureArg "--without-version-opt" "" addConfigureArgIfValueIsNotEmpty "--with-version-build=" "${buildNumber}" addConfigureArg "--with-vendor-version-string=" "AdoptOpenJDK" From 80f08c4c5b9383e121e48bfe06ab975103656352 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 12:59:13 +0100 Subject: [PATCH 070/101] Take 2 at adding datestamp to nightly version (#546) * Switch to curl * Take 2 at adding datestamp to version --- sbin/build.sh | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/sbin/build.sh b/sbin/build.sh index 104341196..06265deae 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -199,16 +199,21 @@ configuringVersionStringParameter() fi addConfigureArgIfValueIsNotEmpty "--with-build-number=" "${buildNumber}" elif [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK9_CORE_VERSION}" ]; then + local buildNumber=${BUILD_CONFIG[OPENJDK_BUILD_NUMBER]} + if [ -z "${buildNumber}" ]; then + buildNumber=$(echo "${OPENJDK_REPO_TAG}" | cut -f2 -d"+") + fi TRIMMED_TAG=$(echo "${OPENJDK_REPO_TAG}" | cut -f2 -d"-" ) if [ -z "${BUILD_CONFIG[TAG]}" ]; then - TRIMMED_TAG="${TRIMMED_TAG}-${dateSuffix}" + addConfigureArg "--with-version-opt" "${dateSuffix}" + else + addConfigureArg "--without-version-opt" "" fi addConfigureArg "--without-version-pre" "" - addConfigureArg "--without-version-opt" "" - addConfigureArg "--with-version-string=" "${TRIMMED_TAG}" + addConfigureArgIfValueIsNotEmpty "--with-version-build=" "${buildNumber}" else # > JDK 9 @@ -219,11 +224,12 @@ configuringVersionStringParameter() fi if [ -z "${BUILD_CONFIG[TAG]}" ]; then - buildNumber="${dateSuffix}" + addConfigureArg "--with-version-opt" "${dateSuffix}" + else + addConfigureArg "--without-version-opt" "" fi addConfigureArg "--without-version-pre" "" - addConfigureArg "--without-version-opt" "" addConfigureArgIfValueIsNotEmpty "--with-version-build=" "${buildNumber}" addConfigureArg "--with-vendor-version-string=" "AdoptOpenJDK" From a7f42c9f1d3b1d8e0de8f06a8fc5b1d54ac9d18b Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 13:08:24 +0100 Subject: [PATCH 071/101] Fix typo --- sbin/build.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sbin/build.sh b/sbin/build.sh index 06265deae..18c81f00f 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -207,7 +207,7 @@ configuringVersionStringParameter() TRIMMED_TAG=$(echo "${OPENJDK_REPO_TAG}" | cut -f2 -d"-" ) if [ -z "${BUILD_CONFIG[TAG]}" ]; then - addConfigureArg "--with-version-opt" "${dateSuffix}" + addConfigureArg "--with-version-opt=" "${dateSuffix}" else addConfigureArg "--without-version-opt" "" fi @@ -224,7 +224,7 @@ configuringVersionStringParameter() fi if [ -z "${BUILD_CONFIG[TAG]}" ]; then - addConfigureArg "--with-version-opt" "${dateSuffix}" + addConfigureArg "--with-version-opt=" "${dateSuffix}" else addConfigureArg "--without-version-opt" "" fi From 8047dc1b68535c48cf22a8485e5e1bfdc9dee8cc Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 13:09:16 +0100 Subject: [PATCH 072/101] Fix typo (#547) * Switch to curl * Take 2 at adding datestamp to version * Fix typo --- sbin/build.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sbin/build.sh b/sbin/build.sh index 06265deae..18c81f00f 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -207,7 +207,7 @@ configuringVersionStringParameter() TRIMMED_TAG=$(echo "${OPENJDK_REPO_TAG}" | cut -f2 -d"-" ) if [ -z "${BUILD_CONFIG[TAG]}" ]; then - addConfigureArg "--with-version-opt" "${dateSuffix}" + addConfigureArg "--with-version-opt=" "${dateSuffix}" else addConfigureArg "--without-version-opt" "" fi @@ -224,7 +224,7 @@ configuringVersionStringParameter() fi if [ -z "${BUILD_CONFIG[TAG]}" ]; then - addConfigureArg "--with-version-opt" "${dateSuffix}" + addConfigureArg "--with-version-opt=" "${dateSuffix}" else addConfigureArg "--without-version-opt" "" fi From 7f1c7d20fa55d1bf626e02448ecb0ac341abf8da Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 16:09:46 +0100 Subject: [PATCH 073/101] Try to fix java, openj9, windows --- .../windows.sh | 34 ++++++++++--------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index b77bac7d6..89b06e88b 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -27,25 +27,27 @@ export BUILD_ARGS="--tmp-space-build ${BUILD_ARGS}" -if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] +if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "hotspot" ] then - export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/c/openjdk/make-3.82/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib64 --disable-ccache" -elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] -then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype=/cygdrive/C/openjdk/freetype --disable-ccache" -elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] -then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --disable-ccache" -elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] -then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache" + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/c/openjdk/make-3.82/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib64 --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype=/cygdrive/C/openjdk/freetype --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache" + fi fi - if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "openj9" ] then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2017" From 236dc28dde7e90781770b51344d797d4ec36c3dd Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 16:12:20 +0100 Subject: [PATCH 074/101] Try to fix java, openj9, windows (#548) * Switch to curl * Take 2 at adding datestamp to version * Fix typo * Try to fix java, openj9, windows --- .../windows.sh | 34 ++++++++++--------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index b77bac7d6..89b06e88b 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -27,25 +27,27 @@ export BUILD_ARGS="--tmp-space-build ${BUILD_ARGS}" -if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] +if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "hotspot" ] then - export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/c/openjdk/make-3.82/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib64 --disable-ccache" -elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] -then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype=/cygdrive/C/openjdk/freetype --disable-ccache" -elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] -then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --disable-ccache" -elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] -then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache" + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/c/openjdk/make-3.82/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib64 --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype=/cygdrive/C/openjdk/freetype --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache" + fi fi - if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "openj9" ] then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2017" From 1e3c41388e4392b3e277c94ef5a469bcf6b0ab67 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 16:57:02 +0100 Subject: [PATCH 075/101] pin windows java 8 openj9 node --- pipelines/build/openjdk8_pipeline.groovy | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy index e9b1c3865..1a037b093 100644 --- a/pipelines/build/openjdk8_pipeline.groovy +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -36,7 +36,9 @@ def buildConfigurations = [ arch : 'x64', additionalNodeLabels: [ hotspot: 'win2008', - openj9: 'win2012' + + //Pin to build-softlayer-win2012r2-x64-1 as build-softlayer-win2012r2-x64-2 may have freetype issues + openj9: 'win2012&&build-softlayer-win2012r2-x64-1' ], test : ['openjdktest'] ], From f0ed7c5914dfa34c13a899d154b74c129747ba7c Mon Sep 17 00:00:00 2001 From: Stewart X Addison Date: Fri, 21 Sep 2018 11:00:28 -0500 Subject: [PATCH 076/101] Revert "Lock JDK11 to our VS2017 machine (tagged openj9 for now)" (#551) --- pipelines/build/openjdk11_pipeline.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index cbf66ea37..8affab5ad 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -30,7 +30,7 @@ def buildConfigurations = [ x64Windows: [ os : 'windows', arch : 'x64', - additionalNodeLabels: 'win2012&&openj9', + additionalNodeLabels: 'win2012', test : ['openjdktest'] ], From 38969e1a54ad5c85cdb47063963bab49df968c22 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 17:04:41 +0100 Subject: [PATCH 077/101] pin windows java 8 openj9 node (#549) * Switch to curl * Take 2 at adding datestamp to version * Fix typo * Try to fix java, openj9, windows * pin windows java 8 openj9 node --- pipelines/build/openjdk8_pipeline.groovy | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy index e9b1c3865..1a037b093 100644 --- a/pipelines/build/openjdk8_pipeline.groovy +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -36,7 +36,9 @@ def buildConfigurations = [ arch : 'x64', additionalNodeLabels: [ hotspot: 'win2008', - openj9: 'win2012' + + //Pin to build-softlayer-win2012r2-x64-1 as build-softlayer-win2012r2-x64-2 may have freetype issues + openj9: 'win2012&&build-softlayer-win2012r2-x64-1' ], test : ['openjdktest'] ], From 8128f55c2c8ab6c274bbf843458a6f26f37314ae Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 17:36:13 +0100 Subject: [PATCH 078/101] add windows 32 java 8 --- build-farm/platform-specific-configurations/windows.sh | 10 +++++++++- pipelines/build/openjdk8_pipeline.groovy | 10 ++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 89b06e88b..f21749dd0 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -24,7 +24,15 @@ export LANG=C export JAVA_HOME=$JDK_BOOT_DIR export BUILD_ARGS="--tmp-space-build ${BUILD_ARGS}" - +if [ "${ARCHITECTURE}" == "x86-32" ] && [ "${VARIANT}" == "hotspot" ] +then + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export ANT_HOME=/cygdrive/C/Projects/OpenJDK/apache-ant-1.10.1 + export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/C/Projects/OpenJDK/make-3.82/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib32 --disable-ccache --with-target-bits=32 --target=x86" + fi +fi if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "hotspot" ] diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy index 1a037b093..37480e433 100644 --- a/pipelines/build/openjdk8_pipeline.groovy +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -43,6 +43,16 @@ def buildConfigurations = [ test : ['openjdktest'] ], + x32Windows: [ + os : 'windows', + arch : 'x86-32', + additionalNodeLabels: [ + hotspot: 'win2008', + openj9: 'win2012&&build-softlayer-win2012r2-x64-1' + ], + test : ['openjdktest'] + ], + ppc64Aix : [ os : 'aix', arch : 'ppc64', From 01e0c273b30f89cc2cde858eccd4c9ba265e2e3a Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 17:39:23 +0100 Subject: [PATCH 079/101] Remove ant --- build-farm/platform-specific-configurations/windows.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index f21749dd0..c47876efb 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -28,7 +28,6 @@ if [ "${ARCHITECTURE}" == "x86-32" ] && [ "${VARIANT}" == "hotspot" ] then if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] then - export ANT_HOME=/cygdrive/C/Projects/OpenJDK/apache-ant-1.10.1 export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/C/Projects/OpenJDK/make-3.82/:$PATH" export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib32 --disable-ccache --with-target-bits=32 --target=x86" fi From 67ebebc5933f3fcfe3d3769ed687b5ae50c29dbd Mon Sep 17 00:00:00 2001 From: Stewart X Addison Date: Fri, 21 Sep 2018 11:40:05 -0500 Subject: [PATCH 080/101] Include MinGW path for OpenJ9 Windows builds (#552) --- build-farm/platform-specific-configurations/windows.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 89b06e88b..f06438ecc 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -51,7 +51,7 @@ fi if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "openj9" ] then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2017" - export PATH="/usr/bin:$PATH" + export PATH="/cygdrive/c/mingw-w64\x86_64-8.1.0-win32-seh-rt_v6-rev0/mingw64/bin:/usr/bin:$PATH" export HAS_AUTOCONF=1 if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] From a57bba1b9c4841f52d2c62cbada3bced14962daf Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 17:49:46 +0100 Subject: [PATCH 081/101] add openj9 --- .../platform-specific-configurations/windows.sh | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 9d7cf293c..4dc41a0f2 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -24,6 +24,7 @@ export LANG=C export JAVA_HOME=$JDK_BOOT_DIR export BUILD_ARGS="--tmp-space-build ${BUILD_ARGS}" + if [ "${ARCHITECTURE}" == "x86-32" ] && [ "${VARIANT}" == "hotspot" ] then if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] @@ -33,6 +34,20 @@ then fi fi +if [ "${ARCHITECTURE}" == "x86-32" ] && [ "${VARIANT}" == "openj9" ] +then + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export PATH=$PATH:/c/cygwin64/bin + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --disable-ccache --with-target-bits=32 --target=x86 --with-milestone=fcs" + + # https://github.com/AdoptOpenJDK/openjdk-build/issues/243 + export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;%INCLUDE%" + export PATH="/c/cygwin64/bin:/usr/bin:$PATH" + + fi +fi + if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "hotspot" ] then From 791868d9063c6b7e6e70642c3f1484eebf7987ca Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 17:50:16 +0100 Subject: [PATCH 082/101] Tidy --- build-farm/platform-specific-configurations/windows.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 4dc41a0f2..e680e8e25 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -38,7 +38,6 @@ if [ "${ARCHITECTURE}" == "x86-32" ] && [ "${VARIANT}" == "openj9" ] then if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] then - export PATH=$PATH:/c/cygwin64/bin export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --disable-ccache --with-target-bits=32 --target=x86 --with-milestone=fcs" # https://github.com/AdoptOpenJDK/openjdk-build/issues/243 From 1cd0d222121c8284878153c57e7e3a2c4c581f5d Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 17:52:42 +0100 Subject: [PATCH 083/101] Convert to bash --- build-farm/platform-specific-configurations/windows.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index e680e8e25..7cda6eded 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -41,7 +41,7 @@ then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --disable-ccache --with-target-bits=32 --target=x86 --with-milestone=fcs" # https://github.com/AdoptOpenJDK/openjdk-build/issues/243 - export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;%INCLUDE%" + export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;$INCLUDE" export PATH="/c/cygwin64/bin:/usr/bin:$PATH" fi From fd3f0e9d6fbe4f24121d0e64c230e9b208a848a6 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 17:54:03 +0100 Subject: [PATCH 084/101] Dont pin --- pipelines/build/openjdk8_pipeline.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy index 68aece9a9..8cc0f7ffd 100644 --- a/pipelines/build/openjdk8_pipeline.groovy +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -47,7 +47,7 @@ def buildConfigurations = [ arch : 'x86-32', additionalNodeLabels: [ hotspot: 'win2008', - openj9: 'win2012&&build-softlayer-win2012r2-x64-1' + openj9: 'win2012' ], test : ['openjdktest'] ], From c64ba50429f9f81d01ea4e48d73994da65e38bb1 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 17:59:34 +0100 Subject: [PATCH 085/101] remove milestone flag --- build-farm/platform-specific-configurations/windows.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 7cda6eded..edfe16def 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -38,7 +38,7 @@ if [ "${ARCHITECTURE}" == "x86-32" ] && [ "${VARIANT}" == "openj9" ] then if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --disable-ccache --with-target-bits=32 --target=x86 --with-milestone=fcs" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --disable-ccache --with-target-bits=32 --target=x86" # https://github.com/AdoptOpenJDK/openjdk-build/issues/243 export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;$INCLUDE" From 1dbde50b47fd2fcdc1c96914b0b242f7a002b473 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Fri, 21 Sep 2018 18:24:42 +0100 Subject: [PATCH 086/101] Fix adding --with-toolchain-version=2013 to win32 --- .../windows.sh | 109 ++++++++++-------- 1 file changed, 58 insertions(+), 51 deletions(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index edfe16def..7b185c688 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -24,72 +24,79 @@ export LANG=C export JAVA_HOME=$JDK_BOOT_DIR export BUILD_ARGS="--tmp-space-build ${BUILD_ARGS}" - -if [ "${ARCHITECTURE}" == "x86-32" ] && [ "${VARIANT}" == "hotspot" ] +if [ "${ARCHITECTURE}" == "x86-32" ] then - if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache --with-target-bits=32 --target=x86" + + if [ "${VARIANT}" == "hotspot" ] then - export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/C/Projects/OpenJDK/make-3.82/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib32 --disable-ccache --with-target-bits=32 --target=x86" + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/C/Projects/OpenJDK/make-3.82/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib32" + fi fi -fi -if [ "${ARCHITECTURE}" == "x86-32" ] && [ "${VARIANT}" == "openj9" ] -then - if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + if [ "${VARIANT}" == "openj9" ] then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --disable-ccache --with-target-bits=32 --target=x86" + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" - # https://github.com/AdoptOpenJDK/openjdk-build/issues/243 - export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;$INCLUDE" - export PATH="/c/cygwin64/bin:/usr/bin:$PATH" + # https://github.com/AdoptOpenJDK/openjdk-build/issues/243 + export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;$INCLUDE" + export PATH="/c/cygwin64/bin:/usr/bin:$PATH" + fi fi fi -if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "hotspot" ] +if [ "${ARCHITECTURE}" == "x64" ] then - if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] - then - export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/c/openjdk/make-3.82/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib64 --disable-ccache" - elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] - then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype=/cygdrive/C/openjdk/freetype --disable-ccache" - elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] - then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --disable-ccache" - elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] + if [ "${VARIANT}" == "hotspot" ] then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache" + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/c/openjdk/make-3.82/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib64 --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype=/cygdrive/C/openjdk/freetype --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache" + fi fi -fi - -if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "openj9" ] -then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2017" - export PATH="/cygdrive/c/mingw-w64\x86_64-8.1.0-win32-seh-rt_v6-rev0/mingw64/bin:/usr/bin:$PATH" - export HAS_AUTOCONF=1 - if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] - then - export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;%INCLUDE%" - export PATH="$PATH:/c/cygwin64/bin" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib64 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" - elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + if [ "${VARIANT}" == "openj9" ] then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" - elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] - then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" - elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] - then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2017" + export PATH="/cygdrive/c/mingw-w64\x86_64-8.1.0-win32-seh-rt_v6-rev0/mingw64/bin:/usr/bin:$PATH" + export HAS_AUTOCONF=1 + + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;%INCLUDE%" + export PATH="$PATH:/c/cygwin64/bin" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib64 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + fi + else + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013" fi -else - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013" fi From 025e8b18df7ec06b043ca40e5d9e0f7bdfb2fa8e Mon Sep 17 00:00:00 2001 From: Martijn Verburg Date: Mon, 24 Sep 2018 11:41:31 +0100 Subject: [PATCH 087/101] move to softlayer 2 (#556) --- pipelines/build/openjdk8_pipeline.groovy | 4 ++-- sbin/prepareWorkspace.sh | 6 ++++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy index 8cc0f7ffd..ca1f12894 100644 --- a/pipelines/build/openjdk8_pipeline.groovy +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -36,8 +36,8 @@ def buildConfigurations = [ arch : 'x64', additionalNodeLabels: [ hotspot: 'win2008', - //Pin to build-softlayer-win2012r2-x64-1 as build-softlayer-win2012r2-x64-2 may have freetype issues - openj9: 'win2012&&build-softlayer-win2012r2-x64-1' + //Pin to build-softlayer-win2012r2-x64-2 as build-softlayer-win2012r2-x64-1 may have issues + openj9: 'win2012&&build-softlayer-win2012r2-x64-2' ], test : ['openjdktest'] ], diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index a9983f5e3..addbe8e81 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -57,6 +57,12 @@ checkoutAndCloneOpenJDKGitRepo() echo "Resetting the git openjdk source repository at $PWD in 10 seconds..." sleep 10 echo "Pulling latest changes from git openjdk source repository" + git fetch --all ${BUILD_CONFIG[SHALLOW_CLONE_OPTION]} + git reset --hard "origin/${BUILD_CONFIG[BRANCH]}" + if [ ! -z "${BUILD_CONFIG[TAG]}" ]; then + git checkout "${BUILD_CONFIG[TAG]}" + fi + git clean -ffdx elif [ "${BUILD_CONFIG[CLEAN_GIT_REPO]}" == "true" ]; then echo "Removing current git repo as it is the wrong type" rm -rf "${BUILD_CONFIG[WORKSPACE_DIR]:?}/${BUILD_CONFIG[WORKING_DIR]}/${BUILD_CONFIG[OPENJDK_SOURCE_DIR]}" From 8dfe91b580a74e39bcb4924687b44ddcba6e3074 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 24 Sep 2018 12:11:43 +0100 Subject: [PATCH 088/101] add toolchain arg to j8/hs/win32 --- .../platform-specific-configurations/windows.sh | 11 ++++++++++- pipelines/build/openjdk11_pipeline.groovy | 7 +++++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 7b185c688..d7f1f1ce3 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -33,7 +33,11 @@ then if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] then export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/C/Projects/OpenJDK/make-3.82/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib32" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2010 --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib32" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" fi fi @@ -46,7 +50,12 @@ then # https://github.com/AdoptOpenJDK/openjdk-build/issues/243 export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;$INCLUDE" export PATH="/c/cygwin64/bin:/usr/bin:$PATH" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + # Next line a potentially tactical fix for https://github.com/AdoptOpenJDK/openjdk-build/issues/267 + export PATH="/usr/bin:$PATH" fi fi fi diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index 8affab5ad..2ff51ee24 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -34,6 +34,13 @@ def buildConfigurations = [ test : ['openjdktest'] ], + x32Windows: [ + os : 'windows', + arch : 'x86-32', + additionalNodeLabels: 'win2012', + test : ['openjdktest'] + ], + ppc64Aix : [ os : 'aix', arch : 'ppc64', From c767fec69b6388ab7bcb0005ed51da1585958e7e Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 24 Sep 2018 12:14:40 +0100 Subject: [PATCH 089/101] Fix adding --with-toolchain-version=2013 to win32 (#554) * Fix adding --with-toolchain-version=2013 to win32 * add toolchain arg to j8/hs/win32 --- .../windows.sh | 118 ++++++++++-------- pipelines/build/openjdk11_pipeline.groovy | 7 ++ 2 files changed, 74 insertions(+), 51 deletions(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index edfe16def..d7f1f1ce3 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -24,72 +24,88 @@ export LANG=C export JAVA_HOME=$JDK_BOOT_DIR export BUILD_ARGS="--tmp-space-build ${BUILD_ARGS}" - -if [ "${ARCHITECTURE}" == "x86-32" ] && [ "${VARIANT}" == "hotspot" ] +if [ "${ARCHITECTURE}" == "x86-32" ] then - if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache --with-target-bits=32 --target=x86" + + if [ "${VARIANT}" == "hotspot" ] then - export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/C/Projects/OpenJDK/make-3.82/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib32 --disable-ccache --with-target-bits=32 --target=x86" + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/C/Projects/OpenJDK/make-3.82/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2010 --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib32" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM}" + fi fi -fi -if [ "${ARCHITECTURE}" == "x86-32" ] && [ "${VARIANT}" == "openj9" ] -then - if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + if [ "${VARIANT}" == "openj9" ] then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --disable-ccache --with-target-bits=32 --target=x86" + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" - # https://github.com/AdoptOpenJDK/openjdk-build/issues/243 - export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;$INCLUDE" - export PATH="/c/cygwin64/bin:/usr/bin:$PATH" + # https://github.com/AdoptOpenJDK/openjdk-build/issues/243 + export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;$INCLUDE" + export PATH="/c/cygwin64/bin:/usr/bin:$PATH" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + # Next line a potentially tactical fix for https://github.com/AdoptOpenJDK/openjdk-build/issues/267 + export PATH="/usr/bin:$PATH" + fi fi fi -if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "hotspot" ] +if [ "${ARCHITECTURE}" == "x64" ] then - if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] - then - export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/c/openjdk/make-3.82/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib64 --disable-ccache" - elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] - then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype=/cygdrive/C/openjdk/freetype --disable-ccache" - elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] - then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --disable-ccache" - elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] + if [ "${VARIANT}" == "hotspot" ] then - export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache" + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/c/openjdk/make-3.82/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype/include --with-freetype-lib=/cygdrive/c/openjdk/freetype/lib64 --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype=/cygdrive/C/openjdk/freetype --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --disable-ccache" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] + then + export PATH="/usr/bin:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:$PATH" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --disable-ccache" + fi fi -fi - -if [ "${ARCHITECTURE}" == "x64" ] && [ "${VARIANT}" == "openj9" ] -then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2017" - export PATH="/cygdrive/c/mingw-w64\x86_64-8.1.0-win32-seh-rt_v6-rev0/mingw64/bin:/usr/bin:$PATH" - export HAS_AUTOCONF=1 - if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] - then - export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;%INCLUDE%" - export PATH="$PATH:/c/cygwin64/bin" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib64 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" - elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + if [ "${VARIANT}" == "openj9" ] then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" - elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] - then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" - elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] - then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2017" + export PATH="/cygdrive/c/mingw-w64\x86_64-8.1.0-win32-seh-rt_v6-rev0/mingw64/bin:/usr/bin:$PATH" + export HAS_AUTOCONF=1 + + if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] + then + export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;%INCLUDE%" + export PATH="$PATH:/c/cygwin64/bin" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib64 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK10_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] + then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + fi + else + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013" fi -else - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013" fi diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index 8affab5ad..2ff51ee24 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -34,6 +34,13 @@ def buildConfigurations = [ test : ['openjdktest'] ], + x32Windows: [ + os : 'windows', + arch : 'x86-32', + additionalNodeLabels: 'win2012', + test : ['openjdktest'] + ], + ppc64Aix : [ os : 'aix', arch : 'ppc64', From 3c00b8c9dd0cfc2eea75d8cb5c696dcf39dd984b Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 24 Sep 2018 12:19:35 +0100 Subject: [PATCH 090/101] remove cleaning workspace on aix --- pipelines/build/openjdk_build_pipeline.groovy | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index dc59f54cd..7c2beadc5 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -155,9 +155,10 @@ try { sh "./build-farm/make-adopt-build-farm.sh" archiveArtifacts artifacts: "workspace/target/*" } finally { + /* if (config.os == "aix") { cleanWs notFailBuild: true - } + }*/ } } } else { From b690553d56a826761ae51e0031665d5ef6be44a2 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Mon, 24 Sep 2018 12:50:05 +0100 Subject: [PATCH 091/101] use --with-freetype-src on 8/j9/win as per configure request --- build-farm/platform-specific-configurations/windows.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index d7f1f1ce3..2f9b4c7e1 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -94,7 +94,7 @@ then then export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;%INCLUDE%" export PATH="$PATH:/c/cygwin64/bin" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib64 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" From 4458769726aaa324f07a6dc065981bb80edd5f5c Mon Sep 17 00:00:00 2001 From: Stewart X Addison Date: Mon, 24 Sep 2018 15:35:55 +0100 Subject: [PATCH 092/101] Stop building with VS2017 from JDK8/9/10+OpenJ9 (#560) --- build-farm/platform-specific-configurations/windows.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 2f9b4c7e1..107aef6d9 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -86,7 +86,6 @@ then if [ "${VARIANT}" == "openj9" ] then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2017" export PATH="/cygdrive/c/mingw-w64\x86_64-8.1.0-win32-seh-rt_v6-rev0/mingw64/bin:/usr/bin:$PATH" export HAS_AUTOCONF=1 @@ -103,7 +102,7 @@ then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" elif [ "${JAVA_TO_BUILD}" == "${JDK11_VERSION}" ] || [ "${JAVA_TO_BUILD}" == "${JDKHEAD_VERSION}" ] then - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --with-toolchain-version=2017" fi else export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013" From 74da2e5fb196ad528bcda9df57232a5805fcfdf2 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Tue, 25 Sep 2018 11:26:32 +0100 Subject: [PATCH 093/101] Brink back cleanup on aix (#562) --- pipelines/build/openjdk_build_pipeline.groovy | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index 7c2beadc5..dc59f54cd 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -155,10 +155,9 @@ try { sh "./build-farm/make-adopt-build-farm.sh" archiveArtifacts artifacts: "workspace/target/*" } finally { - /* if (config.os == "aix") { cleanWs notFailBuild: true - }*/ + } } } } else { From e9d6e40ed99ee8b999eb21d0bed521b7eb82aadb Mon Sep 17 00:00:00 2001 From: Martijn Verburg Date: Tue, 25 Sep 2018 18:34:31 +0100 Subject: [PATCH 094/101] HTTPS everywhere (#557) --- CONTRIBUTING.md | 4 ++-- LICENSE | 4 ++-- README.md | 2 +- build-farm/make-adopt-build-farm.sh | 2 +- build-farm/platform-specific-configurations/aix.sh | 2 +- .../platform-specific-configurations/linux.sh | 2 +- build-farm/platform-specific-configurations/mac.sh | 2 +- .../platform-specific-configurations/windows.sh | 2 +- build-farm/set-platform-specific-configurations.sh | 2 +- build-farm/sign-releases.sh | 2 +- configureBuild.sh | 2 +- docker-build.sh | 2 +- docker/README.md | 2 +- docker/jdk10/x86_64/ubuntu/Dockerfile | 2 +- docker/jdk10/x86_64/ubuntu/Dockerfile-openj9 | 2 +- docker/jdk8/x86_64/ubuntu/Dockerfile | 2 +- docker/jdk8/x86_64/ubuntu/Dockerfile-openj9 | 2 +- docker/jdk9/x86_64/ubuntu/Dockerfile | 2 +- docker/jdk9/x86_64/ubuntu/Dockerfile-openj9 | 2 +- docs/images/sequence.dia | 2 +- git-hg/add-branch-without-modules.sh | 4 ++-- git-hg/add-branch.sh | 6 +++--- git-hg/diff-without-getsource.sh | 4 ++-- git-hg/diff.sh | 4 ++-- git-hg/import-common.sh | 2 +- git-hg/mirrorMercurialIntoGit.sh | 8 ++++---- git-hg/setup.sh | 2 +- git-hg/setup8.sh | 2 +- git-hg/setup9.sh | 2 +- git-hg/setupAmber.sh | 2 +- git-hg/update-without-modules.sh | 6 +++--- git-hg/update.sh | 10 +++++----- git-hg/updateforesttags.sh | 8 ++++---- makejdk-any-platform.sh | 2 +- native-build.sh | 2 +- pipelines/build/build_base_file.groovy | 2 +- pipelines/build/create_job_from_template.groovy | 14 ++++++++++++++ pipelines/build/openjdk10_pipeline.groovy | 2 +- pipelines/build/openjdk11_pipeline.groovy | 2 +- pipelines/build/openjdk8_pipeline.groovy | 2 +- pipelines/build/openjdk9_pipeline.groovy | 2 +- pipelines/build/openjdk_build_pipeline.groovy | 2 +- pipelines/build/openjdk_pipeline.groovy | 2 +- sbin/build.sh | 2 +- sbin/build.template | 2 +- sbin/common/common.sh | 2 +- sbin/common/config_init.sh | 2 +- sbin/common/constants.sh | 2 +- sbin/prepareWorkspace.sh | 4 ++-- security/GenerateCertsFile.sh | 2 +- sign.sh | 2 +- signalhandler.sh | 2 +- 52 files changed, 85 insertions(+), 71 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 823823454..7e6e24347 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -28,7 +28,7 @@ We ask that you include a line similar to the following as part of your pull req ``` DCO 1.1 Signed-off-by: Random J Developer ``` -“DCO” stands for “Developer Certificate of Origin,” and refers to [the same text used in the Linux Kernel community](http://elinux.org/Developer_Certificate_Of_Origin). Of course, you should replace "Random J Developer" by your own real name. +“DCO” stands for “Developer Certificate of Origin,” and refers to [the same text used in the Linux Kernel community](https://elinux.org/Developer_Certificate_Of_Origin). Of course, you should replace "Random J Developer" by your own real name. By adding this simple comment, you are telling the community that you wrote the code you are contributing, or you have the right to pass on the code that you are contributing. @@ -39,7 +39,7 @@ All the project's source files must start with a comment, as near to the top of We don't place explicit copyright statements in the project source files. The project comprises many distinct pieces of code, spread across numerous source files, and authored by a variety of individuals. Managing copyright statements is unproductive and [can lead to confusion and contention around the edge cases](https://opensource.com/law/14/n2/copyright-statements-source-files). Rather we utilize [the NOTICE file](https://github.com/AdoptOpenJDK/openjdk-build/blob/master/NOTICE) mechanism as a way to acknowledge copyright broadly where there is a valid reason to do so. -Finally, for similar reasons to avoiding individual copyright statements, we don't maintain `@author` tags in source files. There are good arguments to suggest that [author tags discourage open contribution](http://producingoss.com/en/managing-volunteers.html#territoriality), and we depend upon Git to maintain that information for the project. +Finally, for similar reasons to avoiding individual copyright statements, we don't maintain `@author` tags in source files. There are good arguments to suggest that [author tags discourage open contribution](https://producingoss.com/en/managing-volunteers.html#territoriality), and we depend upon Git to maintain that information for the project. ### Ensuring high quality After we receive your pull request the [Travis pull request builder](https://travis-ci.org/AdoptOpenJDK/openjdk-build) will test your changes and ensure they meet the coding style guidelines. Watch for the results posted as a comment to the PR, and investigate and fix any failures. diff --git a/LICENSE b/LICENSE index 8f71f43fe..d5dd862b1 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,6 @@ Apache License Version 2.0, January 2004 - http://www.apache.org/licenses/ + https://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION @@ -192,7 +192,7 @@ you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, diff --git a/README.md b/README.md index 1286d7af4..d5a66a326 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ [![Build Status](https://travis-ci.org/AdoptOpenJDK/openjdk-build.svg?branch=master)](https://travis-ci.org/AdoptOpenJDK/openjdk-build) [![Slack](https://slackin-jmnmplfpdu.now.sh/badge.svg)](https://slackin-jmnmplfpdu.now.sh/) AdoptOpenJDK makes use of these scripts to build binaries on the build farm at -http://ci.adoptopenjdk.net, which produces OpenJDK binaries for consumption via +https://ci.adoptopenjdk.net, which produces OpenJDK binaries for consumption via https://www.adoptopenjdk.net and https://api.adoptopenjdk.net. ## TLDR I want to build a JDK NOW! diff --git a/build-farm/make-adopt-build-farm.sh b/build-farm/make-adopt-build-farm.sh index 87618be9b..c485ec0d4 100755 --- a/build-farm/make-adopt-build-farm.sh +++ b/build-farm/make-adopt-build-farm.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/build-farm/platform-specific-configurations/aix.sh b/build-farm/platform-specific-configurations/aix.sh index b42664f76..61a88a01f 100755 --- a/build-farm/platform-specific-configurations/aix.sh +++ b/build-farm/platform-specific-configurations/aix.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/build-farm/platform-specific-configurations/linux.sh b/build-farm/platform-specific-configurations/linux.sh index 61400feaf..f310d233c 100755 --- a/build-farm/platform-specific-configurations/linux.sh +++ b/build-farm/platform-specific-configurations/linux.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/build-farm/platform-specific-configurations/mac.sh b/build-farm/platform-specific-configurations/mac.sh index 376a3f06f..808e951e1 100755 --- a/build-farm/platform-specific-configurations/mac.sh +++ b/build-farm/platform-specific-configurations/mac.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 107aef6d9..4f74bacb3 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/build-farm/set-platform-specific-configurations.sh b/build-farm/set-platform-specific-configurations.sh index bed468b4c..beb62164d 100755 --- a/build-farm/set-platform-specific-configurations.sh +++ b/build-farm/set-platform-specific-configurations.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/build-farm/sign-releases.sh b/build-farm/sign-releases.sh index 17be2fa82..8ea2ab5e8 100755 --- a/build-farm/sign-releases.sh +++ b/build-farm/sign-releases.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/configureBuild.sh b/configureBuild.sh index 7bc49d082..84f492c0c 100755 --- a/configureBuild.sh +++ b/configureBuild.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/docker-build.sh b/docker-build.sh index 9ed5a9871..6612c323a 100755 --- a/docker-build.sh +++ b/docker-build.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/docker/README.md b/docker/README.md index 15ec7e29f..dba7a0770 100644 --- a/docker/README.md +++ b/docker/README.md @@ -5,7 +5,7 @@ Dockerfiles and build scripts for generating various Docker Images building Open # License The Dockerfiles and associated scripts found in this project are licensed under -the [Apache License 2.0](http://www.apache.org/licenses/LICENSE-2.0.html). +the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0.html). # Steps to build diff --git a/docker/jdk10/x86_64/ubuntu/Dockerfile b/docker/jdk10/x86_64/ubuntu/Dockerfile index 29241a23a..d9c7f72e0 100644 --- a/docker/jdk10/x86_64/ubuntu/Dockerfile +++ b/docker/jdk10/x86_64/ubuntu/Dockerfile @@ -3,7 +3,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/docker/jdk10/x86_64/ubuntu/Dockerfile-openj9 b/docker/jdk10/x86_64/ubuntu/Dockerfile-openj9 index 28f42da50..1e4b7997b 100644 --- a/docker/jdk10/x86_64/ubuntu/Dockerfile-openj9 +++ b/docker/jdk10/x86_64/ubuntu/Dockerfile-openj9 @@ -3,7 +3,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/docker/jdk8/x86_64/ubuntu/Dockerfile b/docker/jdk8/x86_64/ubuntu/Dockerfile index 3d0c14008..484721439 100644 --- a/docker/jdk8/x86_64/ubuntu/Dockerfile +++ b/docker/jdk8/x86_64/ubuntu/Dockerfile @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/docker/jdk8/x86_64/ubuntu/Dockerfile-openj9 b/docker/jdk8/x86_64/ubuntu/Dockerfile-openj9 index 831ea7367..013562822 100644 --- a/docker/jdk8/x86_64/ubuntu/Dockerfile-openj9 +++ b/docker/jdk8/x86_64/ubuntu/Dockerfile-openj9 @@ -3,7 +3,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/docker/jdk9/x86_64/ubuntu/Dockerfile b/docker/jdk9/x86_64/ubuntu/Dockerfile index 7e88f78d4..5789a1dea 100644 --- a/docker/jdk9/x86_64/ubuntu/Dockerfile +++ b/docker/jdk9/x86_64/ubuntu/Dockerfile @@ -3,7 +3,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/docker/jdk9/x86_64/ubuntu/Dockerfile-openj9 b/docker/jdk9/x86_64/ubuntu/Dockerfile-openj9 index 831ea7367..013562822 100644 --- a/docker/jdk9/x86_64/ubuntu/Dockerfile-openj9 +++ b/docker/jdk9/x86_64/ubuntu/Dockerfile-openj9 @@ -3,7 +3,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/docs/images/sequence.dia b/docs/images/sequence.dia index a3386f3a9..d3be5bd56 100644 --- a/docs/images/sequence.dia +++ b/docs/images/sequence.dia @@ -1,5 +1,5 @@ - + diff --git a/git-hg/add-branch-without-modules.sh b/git-hg/add-branch-without-modules.sh index 2841b4e56..b8fe3340b 100755 --- a/git-hg/add-branch-without-modules.sh +++ b/git-hg/add-branch-without-modules.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -39,7 +39,7 @@ echo "Create $bpath" mkdir -p "$bpath" || exit 1 echo "git hg clone $bpath (root)" -git hg clone "http://hg.openjdk.java.net/$bpath" "$bpath/root" || exit 1 +git hg clone "https://hg.openjdk.java.net/$bpath" "$bpath/root" || exit 1 echo "checkout the $branch" if [ "$branch" != "" ]; then diff --git a/git-hg/add-branch.sh b/git-hg/add-branch.sh index 053efd82f..1e3a39cfc 100755 --- a/git-hg/add-branch.sh +++ b/git-hg/add-branch.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -31,13 +31,13 @@ echo "Create $bpath" mkdir -p "$bpath" || exit 1 echo "Clone $bpath (root)" -git hg clone "http://hg.openjdk.java.net/$bpath" "$bpath/root" || exit 1 +git hg clone "https://hg.openjdk.java.net/$bpath" "$bpath/root" || exit 1 # shellcheck disable=SC2154 for module in "${modules[@]}" do echo "Clone $bpath -> $module" - git hg clone "http://hg.openjdk.java.net/$bpath/$module" "$bpath/$module" || exit 1 + git hg clone "https://hg.openjdk.java.net/$bpath/$module" "$bpath/$module" || exit 1 done echo "Exit hg" diff --git a/git-hg/diff-without-getsource.sh b/git-hg/diff-without-getsource.sh index 882b1fdde..bdbb03ed7 100755 --- a/git-hg/diff-without-getsource.sh +++ b/git-hg/diff-without-getsource.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -26,7 +26,7 @@ echo "git repo version: ${git_repo_version}" echo "hg repo version: ${hg_root_forest}/${hg_repo_version}" git clone -b master "https://github.com/AdoptOpenJDK/openjdk-${git_repo_version}.git" openjdk-git || exit 1 -hg clone "http://hg.openjdk.java.net/${hg_root_forest}/${hg_repo_version}" openjdk-hg || exit 1 +hg clone "https://hg.openjdk.java.net/${hg_root_forest}/${hg_repo_version}" openjdk-hg || exit 1 diffNum=$(diff -rq openjdk-git openjdk-hg -x '.git' -x '.hg' -x '.hgtags' | wc -l) diff --git a/git-hg/diff.sh b/git-hg/diff.sh index af198691c..85be9c2bf 100755 --- a/git-hg/diff.sh +++ b/git-hg/diff.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -26,7 +26,7 @@ echo "git repo version: ${git_repo_version}" echo "hg repo version: ${hg_root_forest}/${hg_repo_version}" git clone -b master "https://github.com/AdoptOpenJDK/openjdk-${git_repo_version}.git" openjdk-git || exit 1 -hg clone "http://hg.openjdk.java.net/${hg_root_forest}/${hg_repo_version}" openjdk-hg || exit 1 +hg clone "https://hg.openjdk.java.net/${hg_root_forest}/${hg_repo_version}" openjdk-hg || exit 1 cd openjdk-hg || exit 1 bash get_source.sh diff --git a/git-hg/import-common.sh b/git-hg/import-common.sh index bb05f0611..0cf3c26d6 100755 --- a/git-hg/import-common.sh +++ b/git-hg/import-common.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/git-hg/mirrorMercurialIntoGit.sh b/git-hg/mirrorMercurialIntoGit.sh index 91220990c..cafa03bb4 100755 --- a/git-hg/mirrorMercurialIntoGit.sh +++ b/git-hg/mirrorMercurialIntoGit.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -38,7 +38,7 @@ echo "You are in $WORKSPACE" function checkArgs() { if [ "$1" -lt 1 ]; then echo Usage: "$0" '[jdk-updates/jdk10u|jdk/jdk] (branch)' - echo "Hg Repo supplied should match a repository in http://hg.openjdk.java.net/" + echo "Hg Repo supplied should match a repository in https://hg.openjdk.java.net/" echo "For example, to get the latest jdk development repo:" echo "$0 jdk/jdk" echo "For example, to get the raw-string-literals branch from the amber repo:" @@ -72,8 +72,8 @@ function addMercurialUpstream() { # shellcheck disable=SC2143 if [ -z "$(git remote -v | grep 'hg')" ] ; then - echo "Initial setup of hg::http://hg.openjdk.java.net/$HG_REPO" - git remote add hg hg::http://hg.openjdk.java.net/"$HG_REPO" + echo "Initial setup of hg::https://hg.openjdk.java.net/$HG_REPO" + git remote add hg hg::https://hg.openjdk.java.net/"$HG_REPO" fi } diff --git a/git-hg/setup.sh b/git-hg/setup.sh index c133aa725..9fa578118 100755 --- a/git-hg/setup.sh +++ b/git-hg/setup.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/git-hg/setup8.sh b/git-hg/setup8.sh index 7d5f08e80..c024cb5ec 100755 --- a/git-hg/setup8.sh +++ b/git-hg/setup8.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/git-hg/setup9.sh b/git-hg/setup9.sh index 068a7e96e..25ea0c406 100755 --- a/git-hg/setup9.sh +++ b/git-hg/setup9.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/git-hg/setupAmber.sh b/git-hg/setupAmber.sh index 50207133b..88270e20a 100644 --- a/git-hg/setupAmber.sh +++ b/git-hg/setupAmber.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/git-hg/update-without-modules.sh b/git-hg/update-without-modules.sh index d335540e1..805e90da2 100755 --- a/git-hg/update-without-modules.sh +++ b/git-hg/update-without-modules.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -32,8 +32,8 @@ do pushd "$hg_root_forest/$hg_repo/root" echo "Update $hg_root_forest/$hg_repo -> (root)" - git hg fetch "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo" - git hg pull "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo" + git hg fetch "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo" + git hg pull "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo" popd echo "Exit hg" diff --git a/git-hg/update.sh b/git-hg/update.sh index a3c24b282..8e0000bb3 100755 --- a/git-hg/update.sh +++ b/git-hg/update.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -33,8 +33,8 @@ do pushd "$hg_root_forest/$hg_repo/root" echo "Update $hg_root_forest/$hg_repo -> (root)" - git hg fetch "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo" - git hg pull "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo" + git hg fetch "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo" + git hg pull "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo" popd # shellcheck disable=SC2154 @@ -42,8 +42,8 @@ do do pushd "$hg_root_forest/$hg_repo/$module" echo "Update $hg_root_forest/$hg_repo -> $module" - git hg fetch "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo/$module" - git hg pull "http://hg.openjdk.java.net/$hg_root_forest/$hg_repo/$module" + git hg fetch "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo/$module" + git hg pull "https://hg.openjdk.java.net/$hg_root_forest/$hg_repo/$module" popd done diff --git a/git-hg/updateforesttags.sh b/git-hg/updateforesttags.sh index 1e4daded4..1523144c7 100755 --- a/git-hg/updateforesttags.sh +++ b/git-hg/updateforesttags.sh @@ -15,7 +15,7 @@ # accompanied this code). # # You should have received a copy of the GNU General Public License version -# 2 along with this work; if not, see . +# 2 along with this work; if not, see . # # =========================================================================== @@ -36,9 +36,9 @@ GITHUB_PROJECT=git@github.com:AdoptOpenJDK GITHUB_REPO="openjdk-$OPENJDK_VERSION" case "$OPENJDK_VERSION" in - jdk8*) HG_REPO=http://hg.openjdk.java.net/jdk8u/jdk8u + jdk8*) HG_REPO=https://hg.openjdk.java.net/jdk8u/jdk8u [ -z "$TAGS" ] && TAGS="jdk8u144-b34 jdk8u151-b12 jdk8u152-b16 jdk8u161-b12 jdk8u162-b12 jdk8u172-b03 jdk8u172-b11";; - jdk9*) HG_REPO=http://hg.openjdk.java.net/jdk-updates/jdk9u + jdk9*) HG_REPO=https://hg.openjdk.java.net/jdk-updates/jdk9u [ -z "$TAGS" ] && TAGS="jdk-9+181 jdk-9.0.1+11 jdk-9.0.3+9 jdk-9.0.4+11";; *) Unknown JDK version - only jdk8u and jdk9 are supported; exit 1;; esac @@ -56,7 +56,7 @@ GIT_MINOR_VERSION=$(echo "$GIT_VERSION" | cut -d. -f2) if ! which git-remote-hg 2>/dev/null; then echo "I need git-remote-hg and could not find it" - echo "Get it from http://raw.githubusercontent.com/felipec/git-remote-hg/master/git-remote-hg" + echo "Get it from https://raw.githubusercontent.com/felipec/git-remote-hg/master/git-remote-hg" exit 1 fi diff --git a/makejdk-any-platform.sh b/makejdk-any-platform.sh index 5c1f3a827..6361c323c 100755 --- a/makejdk-any-platform.sh +++ b/makejdk-any-platform.sh @@ -6,7 +6,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/native-build.sh b/native-build.sh index fafc5bcf0..cac32550b 100755 --- a/native-build.sh +++ b/native-build.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/pipelines/build/build_base_file.groovy b/pipelines/build/build_base_file.groovy index 102acc47d..9664f3264 100644 --- a/pipelines/build/build_base_file.groovy +++ b/pipelines/build/build_base_file.groovy @@ -6,7 +6,7 @@ Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, diff --git a/pipelines/build/create_job_from_template.groovy b/pipelines/build/create_job_from_template.groovy index 240615b51..d4a7dedaf 100644 --- a/pipelines/build/create_job_from_template.groovy +++ b/pipelines/build/create_job_from_template.groovy @@ -1,3 +1,17 @@ +/* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + /** * A template that defines a build job. * diff --git a/pipelines/build/openjdk10_pipeline.groovy b/pipelines/build/openjdk10_pipeline.groovy index 757a3c9c2..d0e99a6c3 100644 --- a/pipelines/build/openjdk10_pipeline.groovy +++ b/pipelines/build/openjdk10_pipeline.groovy @@ -3,7 +3,7 @@ Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index 2ff51ee24..23b176c0f 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -3,7 +3,7 @@ Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, diff --git a/pipelines/build/openjdk8_pipeline.groovy b/pipelines/build/openjdk8_pipeline.groovy index ca1f12894..6db7fe706 100644 --- a/pipelines/build/openjdk8_pipeline.groovy +++ b/pipelines/build/openjdk8_pipeline.groovy @@ -3,7 +3,7 @@ Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, diff --git a/pipelines/build/openjdk9_pipeline.groovy b/pipelines/build/openjdk9_pipeline.groovy index c11e0c38d..7eec6b8b4 100644 --- a/pipelines/build/openjdk9_pipeline.groovy +++ b/pipelines/build/openjdk9_pipeline.groovy @@ -3,7 +3,7 @@ Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, diff --git a/pipelines/build/openjdk_build_pipeline.groovy b/pipelines/build/openjdk_build_pipeline.groovy index dc59f54cd..62cfb4144 100644 --- a/pipelines/build/openjdk_build_pipeline.groovy +++ b/pipelines/build/openjdk_build_pipeline.groovy @@ -3,7 +3,7 @@ Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, diff --git a/pipelines/build/openjdk_pipeline.groovy b/pipelines/build/openjdk_pipeline.groovy index 987f03442..babd65d11 100644 --- a/pipelines/build/openjdk_pipeline.groovy +++ b/pipelines/build/openjdk_pipeline.groovy @@ -3,7 +3,7 @@ Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at - http://www.apache.org/licenses/LICENSE-2.0 + https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, diff --git a/sbin/build.sh b/sbin/build.sh index 18c81f00f..88f642161 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/sbin/build.template b/sbin/build.template index 16514cc60..7afdb62b2 100644 --- a/sbin/build.template +++ b/sbin/build.template @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/sbin/common/common.sh b/sbin/common/common.sh index a630b2c37..eee4ababd 100755 --- a/sbin/common/common.sh +++ b/sbin/common/common.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/sbin/common/config_init.sh b/sbin/common/config_init.sh index 4b2e88af3..486873520 100755 --- a/sbin/common/config_init.sh +++ b/sbin/common/config_init.sh @@ -6,7 +6,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/sbin/common/constants.sh b/sbin/common/constants.sh index b3238d7a4..95e114850 100755 --- a/sbin/common/constants.sh +++ b/sbin/common/constants.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index addbe8e81..323fcebf8 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, @@ -170,7 +170,7 @@ checkingAndDownloadingFreemarker() echo "Skipping FREEMARKER download" else - wget -nc --no-check-certificate "http://www.mirrorservice.org/sites/ftp.apache.org/freemarker/engine/${FREEMARKER_LIB_VERSION}/binaries/apache-freemarker-${FREEMARKER_LIB_VERSION}-bin.tar.gz" + wget -nc --no-check-certificate "https://www.mirrorservice.org/sites/ftp.apache.org/freemarker/engine/${FREEMARKER_LIB_VERSION}/binaries/apache-freemarker-${FREEMARKER_LIB_VERSION}-bin.tar.gz" mkdir -p "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/freemarker-${FREEMARKER_LIB_VERSION}/" || exit tar -xzf "apache-freemarker-${FREEMARKER_LIB_VERSION}-bin.tar.gz" --strip-components=1 -C "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/freemarker-${FREEMARKER_LIB_VERSION}/" rm "apache-freemarker-${FREEMARKER_LIB_VERSION}-bin.tar.gz" diff --git a/security/GenerateCertsFile.sh b/security/GenerateCertsFile.sh index 40611070c..97f77c936 100644 --- a/security/GenerateCertsFile.sh +++ b/security/GenerateCertsFile.sh @@ -4,7 +4,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/sign.sh b/sign.sh index be1d190ed..90dfa19e9 100755 --- a/sign.sh +++ b/sign.sh @@ -5,7 +5,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, diff --git a/signalhandler.sh b/signalhandler.sh index 58aa144e2..bcb04f2e0 100755 --- a/signalhandler.sh +++ b/signalhandler.sh @@ -6,7 +6,7 @@ # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # -# http://www.apache.org/licenses/LICENSE-2.0 +# https://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, From 3c9a3b1772d0569d747f382a9ed55d7c80194778 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 26 Sep 2018 12:03:47 +0100 Subject: [PATCH 095/101] Make some freetype downloading improvements. Add release table generation script --- .../windows.sh | 8 +-- sbin/build.sh | 32 ++++++++---- sbin/common/config_init.sh | 3 ++ sbin/prepareWorkspace.sh | 51 +++++++++++-------- 4 files changed, 58 insertions(+), 36 deletions(-) diff --git a/build-farm/platform-specific-configurations/windows.sh b/build-farm/platform-specific-configurations/windows.sh index 4f74bacb3..0524afe60 100755 --- a/build-farm/platform-specific-configurations/windows.sh +++ b/build-farm/platform-specific-configurations/windows.sh @@ -65,6 +65,7 @@ if [ "${ARCHITECTURE}" == "x64" ] then if [ "${VARIANT}" == "hotspot" ] then + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013" if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] then export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 10.0/VC/bin/amd64/:/cygdrive/c/openjdk/make-3.82/:$PATH" @@ -88,12 +89,13 @@ then then export PATH="/cygdrive/c/mingw-w64\x86_64-8.1.0-win32-seh-rt_v6-rev0/mingw64/bin:/usr/bin:$PATH" export HAS_AUTOCONF=1 + export BUILD_ARGS="${BUILD_ARGS} --freetype-version 2.5.3" if [ "${JAVA_TO_BUILD}" == "${JDK8_VERSION}" ] then - export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;%INCLUDE%" + export INCLUDE="C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;$INCLUDE" export PATH="$PATH:/c/cygwin64/bin" - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" + export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-include=/cygdrive/c/openjdk/freetype-2.5.3/include --with-freetype-lib=/cygdrive/c/openjdk/freetype-2.5.3/lib64 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --disable-ccache" elif [ "${JAVA_TO_BUILD}" == "${JDK9_VERSION}" ] then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freetype-src=/cygdrive/c/openjdk/freetype-2.5.3 --with-toolchain-version=2013 --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar" @@ -104,7 +106,5 @@ then then export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-freemarker-jar=/cygdrive/c/openjdk/freemarker.jar --with-toolchain-version=2017" fi - else - export CONFIGURE_ARGS_FOR_ANY_PLATFORM="${CONFIGURE_ARGS_FOR_ANY_PLATFORM} --with-toolchain-version=2013" fi fi diff --git a/sbin/build.sh b/sbin/build.sh index 88f642161..0f80b2845 100755 --- a/sbin/build.sh +++ b/sbin/build.sh @@ -257,15 +257,6 @@ buildingTheRestOfTheConfigParameters() addConfigureArg "--with-freemarker-jar=" "${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/freemarker-${FREEMARKER_LIB_VERSION}/freemarker.jar" fi - if [[ "${BUILD_CONFIG[FREETYPE]}" == "true" ]] ; then - local freetypeDir=BUILD_CONFIG[FREETYPE_DIRECTORY] - case "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" in - jdk8*|jdk9*|jdk10*) freetypeDir=${BUILD_CONFIG[FREETYPE_DIRECTORY]:-"${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedfreetype"} ;; - *) freetypeDir=${BUILD_CONFIG[FREETYPE_DIRECTORY]:-bundled} ;; - esac - addConfigureArg "--with-freetype=" "${freetypeDir}" - fi - addConfigureArg "--with-x=" "/usr/include/X11" if [ "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" == "${JDK8_CORE_VERSION}" ] ; then @@ -280,14 +271,33 @@ buildingTheRestOfTheConfigParameters() fi } +configureFreetypeLocation() { + if [[ ! "${CONFIGURE_ARGS}" =~ "--with-freetype" ]]; then + if [[ "${BUILD_CONFIG[FREETYPE]}" == "true" ]] ; then + if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then + addConfigureArg "--with-freetype-src=" "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/freetype" + else + local freetypeDir=BUILD_CONFIG[FREETYPE_DIRECTORY] + case "${BUILD_CONFIG[OPENJDK_CORE_VERSION]}" in + jdk8*|jdk9*|jdk10*) freetypeDir=${BUILD_CONFIG[FREETYPE_DIRECTORY]:-"${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}/installedfreetype"} ;; + *) freetypeDir=${BUILD_CONFIG[FREETYPE_DIRECTORY]:-bundled} ;; + esac + + echo "setting freetype dir to ${freetypeDir}" + addConfigureArg "--with-freetype=" "${freetypeDir}" + fi + fi + fi +} + # Configure the command parameters configureCommandParameters() { configuringVersionStringParameter configuringBootJDKConfigureParameter + if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then echo "Windows or Windows-like environment detected, skipping configuring environment for custom Boot JDK and other 'configure' settings." - else echo "Building up the configure command..." buildingTheRestOfTheConfigParameters @@ -296,6 +306,8 @@ configureCommandParameters() # Now we add any configure arguments the user has specified on the command line. CONFIGURE_ARGS="${CONFIGURE_ARGS} ${BUILD_CONFIG[USER_SUPPLIED_CONFIGURE_ARGS]}" + configureFreetypeLocation + echo "Completed configuring the version string parameter, config args are now: ${CONFIGURE_ARGS}" } diff --git a/sbin/common/config_init.sh b/sbin/common/config_init.sh index 486873520..d3ac96ba9 100755 --- a/sbin/common/config_init.sh +++ b/sbin/common/config_init.sh @@ -189,6 +189,9 @@ function parseConfigurationArguments() { "--freetype-build-param" ) BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]="$1"; shift;; + "--freetype-version" ) + BUILD_CONFIG[FREETYPE_FONT_VERSION]="$1"; shift;; + "--skip-freetype" | "-F" ) BUILD_CONFIG[FREETYPE]=false;; diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index 323fcebf8..cb184add9 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -180,7 +180,6 @@ checkingAndDownloadingFreemarker() # Get Freetype checkingAndDownloadingFreeType() { - cd "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/" || exit echo "Checking for freetype at ${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}" @@ -189,17 +188,23 @@ checkingAndDownloadingFreeType() if [[ ! -z "$FOUND_FREETYPE" ]] ; then echo "Skipping FreeType download" else - # Then FreeType for fonts: make it and use - wget -nc --no-check-certificate https://ftp.acc.umu.se/mirror/gnu.org/savannah/freetype/freetype-"${BUILD_CONFIG[FREETYPE_FONT_VERSION]}".tar.gz + curl -L -o "freetype.tar.gz" "https://download.savannah.gnu.org/releases/freetype/freetype-${BUILD_CONFIG[FREETYPE_FONT_VERSION]}.tar.gz" + + rm -rf "./freetype" || true + mkdir -p "freetype" || true + tar xpzf freetype.tar.gz --strip-components=1 -C "freetype" + rm freetype.tar.gz + + if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then + return; + fi - gunzip -dc freetype-"${BUILD_CONFIG[FREETYPE_FONT_VERSION]}".tar.gz | tar xf - -C . - rm freetype-"${BUILD_CONFIG[FREETYPE_FONT_VERSION]}".tar.gz + cd freetype || exit - cd freetype-"${BUILD_CONFIG[FREETYPE_FONT_VERSION]}" || exit # We get the files we need at $WORKING_DIR/installedfreetype # shellcheck disable=SC2046 - if ! (bash ./configure --prefix="${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}"/installedfreetype "${BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]}" && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} all && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} install); then + if ! (bash ./configure "${BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]}" && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} all && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} install); then # shellcheck disable=SC2154 echo "Failed to configure and build libfreetype, exiting" exit; @@ -266,30 +271,31 @@ downloadingRequiredDependencies() cd "${BUILD_CONFIG[WORKSPACE_DIR]}/libs/" || exit if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]] ; then - echo "Windows or Windows-like environment detected, skipping downloading of dependencies...: Alsa, Freetype, and CaCerts." + echo "Windows or Windows-like environment detected, skipping downloading of dependencies...: Alsa." else echo "Downloading required dependencies...: Alsa, Freetype, Freemarker, and CaCerts." echo "Checking and download Alsa dependency" checkingAndDownloadingAlsa - if [[ "${BUILD_CONFIG[FREETYPE]}" == "true" ]] ; then - if [ -z "${BUILD_CONFIG[FREETYPE_DIRECTORY]}" ]; then - echo "Checking and download FreeType Font dependency" - checkingAndDownloadingFreeType - else - echo "" - echo "---> Skipping the process of checking and downloading the FreeType Font dependency, a pre-built version provided at ${BUILD_CONFIG[FREETYPE_DIRECTORY]} <---" - echo "" - fi - else - echo "Skipping Freetype" - fi if [[ "${BUILD_CONFIG[BUILD_VARIANT]}" == "openj9" ]]; then echo "Checking and download Freemarker dependency" checkingAndDownloadingFreemarker fi fi + if [[ "${BUILD_CONFIG[FREETYPE]}" == "true" ]] ; then + if [ -z "${BUILD_CONFIG[FREETYPE_DIRECTORY]}" ]; then + echo "Checking and download FreeType Font dependency" + checkingAndDownloadingFreeType + else + echo "" + echo "---> Skipping the process of checking and downloading the FreeType Font dependency, a pre-built version provided at ${BUILD_CONFIG[FREETYPE_DIRECTORY]} <---" + echo "" + fi + else + echo "Skipping Freetype" + fi + echo "Checking and download CaCerts dependency" checkingAndDownloadCaCerts @@ -328,13 +334,14 @@ relocateToTmpIfNeeded() export TMP_WORKSPACE="${tmpdir}" export ORIGINAL_WORKSPACE="${BUILD_CONFIG[WORKSPACE_DIR]}" + trap moveTmpToWorkspaceLocation EXIT SIGINT SIGTERM + if [ -d "${ORIGINAL_WORKSPACE}" ] then cp -r "${BUILD_CONFIG[WORKSPACE_DIR]}" "${TMP_WORKSPACE}/workspace" fi BUILD_CONFIG[WORKSPACE_DIR]="${TMP_WORKSPACE}/workspace" - trap moveTmpToWorkspaceLocation EXIT SIGINT SIGTERM fi } @@ -342,8 +349,8 @@ relocateToTmpIfNeeded() function configureWorkspace() { createWorkspace + downloadingRequiredDependencies relocateToTmpIfNeeded checkoutAndCloneOpenJDKGitRepo - downloadingRequiredDependencies } From 234fd25d86217938e8463f57b3ee4f4d9f3a3517 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 26 Sep 2018 12:07:56 +0100 Subject: [PATCH 096/101] Fix freetype build --- sbin/prepareWorkspace.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/prepareWorkspace.sh b/sbin/prepareWorkspace.sh index cb184add9..c81828d60 100755 --- a/sbin/prepareWorkspace.sh +++ b/sbin/prepareWorkspace.sh @@ -204,7 +204,7 @@ checkingAndDownloadingFreeType() # We get the files we need at $WORKING_DIR/installedfreetype # shellcheck disable=SC2046 - if ! (bash ./configure "${BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]}" && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} all && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} install); then + if ! (bash ./configure --prefix="${BUILD_CONFIG[WORKSPACE_DIR]}/${BUILD_CONFIG[WORKING_DIR]}"/installedfreetype "${BUILD_CONFIG[FREETYPE_FONT_BUILD_TYPE_PARAM]}" && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} all && ${BUILD_CONFIG[MAKE_COMMAND_NAME]} install); then # shellcheck disable=SC2154 echo "Failed to configure and build libfreetype, exiting" exit; From 8539f5fcce02a5fa7ac6c659b215befae9a7d5df Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 26 Sep 2018 13:47:08 +0100 Subject: [PATCH 097/101] Allow configuring tests based on release/nightly --- pipelines/build/build_base_file.groovy | 70 +++++++++++++++-------- pipelines/build/openjdk11_pipeline.groovy | 6 +- 2 files changed, 51 insertions(+), 25 deletions(-) diff --git a/pipelines/build/build_base_file.groovy b/pipelines/build/build_base_file.groovy index 9664f3264..aae1d2110 100644 --- a/pipelines/build/build_base_file.groovy +++ b/pipelines/build/build_base_file.groovy @@ -41,29 +41,7 @@ def toBuildParams(enableTests, params) { static def buildConfiguration(javaToBuild, variant, configuration, releaseTag, branch, additionalConfigureArgs) { - String buildTag = "build" - - if (configuration.os == "windows" && variant == "openj9") { - buildTag = "buildj9" - } else if (configuration.arch == "s390x" && variant == "openj9") { - buildTag = "(buildj9||build)&&openj9" - } - - def additionalNodeLabels = null - if (configuration.containsKey("additionalNodeLabels")) { - // hack as jenkins sandbox wont allow instanceof - if ("java.util.LinkedHashMap" == configuration.additionalNodeLabels.getClass().getName()) { - additionalNodeLabels = configuration.additionalNodeLabels.get(variant) - } else { - additionalNodeLabels = configuration.additionalNodeLabels - } - } - - if (additionalNodeLabels != null) { - additionalNodeLabels = "${additionalNodeLabels}&&${buildTag}" - } else { - additionalNodeLabels = "${buildTag}" - } + def additionalNodeLabels = formAdditionalNodeLabels(configuration, variant) def buildParams = [ JAVA_TO_BUILD: javaToBuild, @@ -83,20 +61,64 @@ static def buildConfiguration(javaToBuild, variant, configuration, releaseTag, b buildParams.put("BRANCH", branch) } + def isRelease = false if (releaseTag != null && releaseTag.length() > 0) { + isRelease = true buildParams.put("TAG", releaseTag) } + def testList = getTestList(configuration, isRelease) + return [ javaVersion: javaToBuild, arch : configuration.arch, os : configuration.os, variant : variant, parameters : buildParams, - test : configuration.test, + test : testList, ] } +static def getTestList(configuration, isRelease) { + if (configuration.containsKey("test")) { + def testJobType = isRelease ? "release" : "nightly" + + // hack as jenkins sandbox wont allow instanceof + if ("java.util.LinkedHashMap" == configuration.test.getClass().getName()) { + return configuration.test.get(testJobType) + } else { + return configuration.test + } + } + return [] +} + +static def formAdditionalNodeLabels(configuration, variant) { + def buildTag = "build" + + if (configuration.os == "windows" && variant == "openj9") { + buildTag = "buildj9" + } else if (configuration.arch == "s390x" && variant == "openj9") { + buildTag = "(buildj9||build)&&openj9" + } + + def labels = "${buildTag}" + + if (configuration.containsKey("additionalNodeLabels")) { + def additionalNodeLabels = null + + // hack as jenkins sandbox wont allow instanceof + if ("java.util.LinkedHashMap" == configuration.additionalNodeLabels.getClass().getName()) { + additionalNodeLabels = configuration.additionalNodeLabels.get(variant) + } else { + additionalNodeLabels = configuration.additionalNodeLabels + } + labels = "${additionalNodeLabels}&&${labels}" + } + + return labels +} + static def getConfigureArgs(configuration, additionalConfigureArgs) { def buildParams = [:] def configureArgs = ""; diff --git a/pipelines/build/openjdk11_pipeline.groovy b/pipelines/build/openjdk11_pipeline.groovy index 23b176c0f..c1cc1f21c 100644 --- a/pipelines/build/openjdk11_pipeline.groovy +++ b/pipelines/build/openjdk11_pipeline.groovy @@ -19,11 +19,15 @@ def buildConfigurations = [ additionalNodeLabels: 'build-macstadium-macos1010-1', test : ['openjdktest', 'systemtest'] ], + x64Linux : [ os : 'linux', arch : 'x64', additionalNodeLabels: 'centos6', - test : ['openjdktest', 'systemtest', 'externaltest'] + test : [ + nightly: ['openjdktest', 'systemtest', 'perftest', 'externaltest', 'externaltest_extended'], + release: ['openjdktest', 'systemtest', 'perftest', 'externaltest'] + ] ], // Currently we have to be quite specific about which windows to use as not all of them have freetype installed From 6a71af98922a7493c0c38e4513324f3fedc30427 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 26 Sep 2018 14:01:31 +0100 Subject: [PATCH 098/101] Remove warning --- README.md | 27 --------------------------- 1 file changed, 27 deletions(-) diff --git a/README.md b/README.md index 0da2e08c0..95e89671b 100644 --- a/README.md +++ b/README.md @@ -1,30 +1,3 @@ -# This Branch is a legacy branch!! - -Jobs are currently being migrated to the -[new_build_scripts](https://github.com/AdoptOpenJDK/openjdk-build/tree/new_build_scripts) branch. - -If you're looking to make fixes or additions to the build scripts please do so -on the [new_build_scripts](https://github.com/AdoptOpenJDK/openjdk-build/tree/new_build_scripts) branch -first and only backport to this legacy branch if required. - -**11th of Sept 2018** - -At time of writing the new_build_scripts branch are building most jdk10 jobs - -

-

-

-

-

-

-

-

-

-

-

---- - - # Repository for code and instructions for building OpenJDK [![Build Status](https://travis-ci.org/AdoptOpenJDK/openjdk-build.svg?branch=master)](https://travis-ci.org/AdoptOpenJDK/openjdk-build) [![Slack](https://slackin-jmnmplfpdu.now.sh/badge.svg)](https://slackin-jmnmplfpdu.now.sh/) From 11eec7a0324ec39c714b86d056219c928ac3e559 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 26 Sep 2018 15:01:59 +0100 Subject: [PATCH 099/101] Remove old docs --- README.md | 10 +--------- makejdk-any-platform.1 | 1 - 2 files changed, 1 insertion(+), 10 deletions(-) diff --git a/README.md b/README.md index 95e89671b..ac50c8fe1 100644 --- a/README.md +++ b/README.md @@ -235,7 +235,7 @@ passed into `makejdk-any-platform.sh` script. Note that the build nodes (list of hosts on the LH side) also have configuration where things like the BOOT_JDK environment variable is set. -**WARN: As of 23rd March 2018 these instructions do not work, there are several issues that need resolving ([see this issue as starting point](https://github.com/AdoptOpenJDK/openjdk-build/issues/194))** +### Building via Docker in your local environment The simplest way to build OpenJDK using these scripts is to run `makejdk-any-platform.sh` and have your user be in the Docker group on the machine (or use the `--sudo` @@ -249,14 +249,6 @@ Once you have Docker started you can then use the script below to build OpenJDK. Example Usage (TODO Add example of openj9): -* **NOTE:** If you don't use SSH keys (if you do then pass `-ssh`) to connect to GitHub then the script will challenge you for your GitHub username and password. (You can find more information about SSH keys for GitHub [here](https://help.github.com/articles/connecting-to-github-with-ssh/)) -* **NOTE:** The script will clone source code into the `--source` directory (defaults to `openjdk`). -* **NOTE:** By default the docker container is removed each time and your build will be copied from the container to the host. -To override this behaviour, specify the `-k` or `--keep` option. -* **NOTE:** The entire process will take some time, especially if you have not saved the Docker image from a previous run. -* **NOTE:** If you set the `-d` option it will pass that through to `makejdk.sh`, the resulting zipped tarball will be copied to the value for -d, for example: -`makejdk.sh /target/directory` will result in the JDK being built inside of your Docker container and then copied to `/target/directory` on the host - #### Configuring Docker for non sudo use To use the Docker commands without using the `--sudo` option, you will need to be diff --git a/makejdk-any-platform.1 b/makejdk-any-platform.1 index a4bfc58d7..04675c211 100755 --- a/makejdk-any-platform.1 +++ b/makejdk-any-platform.1 @@ -29,7 +29,6 @@ binary from the source at https://github.com/AdoptOpenJDK/openjdk-jdk8u Please visit https://www.adoptopenjdk.net for further support -.B WARNING: Currently the Docker based build is not working (see https://github.com/AdoptOpenJDK/openjdk-build/issues/194) .SH VERSIONS .TP .BR jdk8u From 11ede328b99f91f2a60127737edabf6a32e1b5fb Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 26 Sep 2018 15:02:41 +0100 Subject: [PATCH 100/101] Readd example --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index ac50c8fe1..d5a66a326 100644 --- a/README.md +++ b/README.md @@ -249,6 +249,8 @@ Once you have Docker started you can then use the script below to build OpenJDK. Example Usage (TODO Add example of openj9): +`./makejdk-any-platform.sh --docker --sudo jdk8u` + #### Configuring Docker for non sudo use To use the Docker commands without using the `--sudo` option, you will need to be From 325ad8ba19b9da4c2b665b99257c75430cee54a8 Mon Sep 17 00:00:00 2001 From: John Oliver Date: Wed, 26 Sep 2018 15:03:01 +0100 Subject: [PATCH 101/101] Bring inline with master (#564) * Add x86-32_windows target to openjdk8 build pipelines (#462) * Add Windows32 to buildPlatforms list (#467) * Link to issue about docker problem (#471) * Adding Docker bug info to man page (#472) * additional info in man page * link in. man page * Link to Github SHH help added (#473) * Move over most 10 jobs to the new pipeline * Add triggers for external_extended tests Signed-off-by: Mesbah * Refactor updateforesttags to use functions (#358) * Skip 8u152+b16 as it's being a PITA * Revert "Refactor updateforesttags to use functions (#358)" (#500) This reverts commit 363d56d1a70b7c75228f1d5c3611d36536d57631. * Force git clean to remove dirs with .git repos (#528) - Git clean -fxd will not remove directories that contain a Git repository (.git folder). - Adding a second force will remove these folders - Ref https://git-scm.com/docs/git-clean#git-clean---force Signed-off-by: Adam Brousseau * Remove warning * Remove old docs * Readd example