Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kie-issues#574 Initial ASF Jenkins CI Setup #1090

Merged
merged 1 commit into from
Sep 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .ci/actions/build-chain/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ inputs:
definition-file:
description: 'The `definition-file` input for the build-chain'
required: false
default: 'https://raw.githubusercontent.com/${GROUP:kiegroup}/kogito-pipelines/${BRANCH:main}/.ci/pull-request-config.yaml'
default: 'https://raw.githubusercontent.com/${GROUP:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/pull-request-config.yaml'
flow-type:
description: "the flow to execute, it can be 'cross_pr' (or 'pull-request' which is deprecated), 'full_downstream' (or 'full-downstream' which is deprecated), 'single_pr' (or 'single' which is deprecated) or 'branch'"
default: "cross_pr"
Expand Down
2 changes: 1 addition & 1 deletion .ci/actions/dsl-tests/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ runs:
- name: Checkout Seed repo
uses: actions/checkout@v3
with:
repository: kiegroup/kogito-pipelines
repository: apache/incubator-kie-kogito-pipelines
ref: main
path: seed_repo

Expand Down
2 changes: 1 addition & 1 deletion .ci/actions/os-preparation/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ inputs:
repository:
description: "The repository containing the matrix-os.json os configuration"
required: false
default: "kiegroup/kogito-pipelines"
default: "apache/incubator-kie-kogito-pipelines"
branch:
description: "The repository branch from which the matrix configuration must be taken"
required: false
Expand Down
2 changes: 1 addition & 1 deletion .ci/chain-status-info.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Kiegroup Kogito organization repositories CI Status

This project is based on [chain-status](https://github.com/kiegroup/chain-status) and information generated thanks to [build-chain-configuration-reader](https://github.com/kiegroup/build-chain-configuration-reader) using [kogito-pipelines definition file](https://github.com/kiegroup/kogito-pipelines/blob/main/.ci/pull-request-config.yaml).
This project is based on [chain-status](https://github.com/kiegroup/chain-status) and information generated thanks to [build-chain-configuration-reader](https://github.com/kiegroup/build-chain-configuration-reader) using [kogito-pipelines definition file](https://github.com/apache/incubator-kie-kogito-pipelines/blob/main/.ci/pull-request-config.yaml).

Due to the retrieved information requires a `GITHUB_TOKEN` and github API has a limitation it is better to not collect the information on every request from this webpage is made. So this webpage does not really requires an application service and all the required data is stored in github pages. This information is normally retrieved, treated and stored using [chain-status/action tool](https://github.com/kiegroup/chain-status/tree/main/packages/action) which is exposed as a Github Action tool, so we can customize the job execution frequency as we need.
97 changes: 97 additions & 0 deletions .ci/jenkins/Jenkinsfile.build-kogito-ci-image
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
import org.jenkinsci.plugins.workflow.libs.Library

@Library('jenkins-pipeline-shared-libraries')_

pipeline {
agent {
label 'ubuntu'
}

environment {
DOCKER_CONFIG = "${WORKSPACE}/.docker"

IMAGE_NAME = 'quay.io/kiegroup/kogito-ci-build'
IMAGE_TAG = "${BRANCH_NAME}-build-${BUILD_NUMBER}"
IMAGE_NAME_TAG = "${env.IMAGE_NAME}:${env.IMAGE_TAG}"
}

options {
timeout(time: 60, unit: 'MINUTES')
}

stages {
stage('Build image') {
steps {
script {
sh 'printenv'

currentBuild.displayName = env.IMAGE_TAG

cloud.loginContainerRegistry('quay.io', 'quay_kiegroup_registry_token')

dir('kogito-pipelines') {
deleteDir()

checkout scm

sh """
docker build --build-arg SDKMAN_JAVA=11.0.20-tem -t ${env.IMAGE_NAME_TAG} -f apache-nodes/Dockerfile.kogito-ci-build .
docker tag ${env.IMAGE_NAME_TAG} ${env.IMAGE_NAME}:${GIT_COMMIT}
docker tag ${env.IMAGE_NAME_TAG} ${env.IMAGE_NAME}:${BRANCH_NAME}-latest
"""

sh """
docker --config ${DOCKER_CONFIG} push ${env.IMAGE_NAME_TAG}
docker --config ${DOCKER_CONFIG} push ${env.IMAGE_NAME}:${GIT_COMMIT}
docker --config ${DOCKER_CONFIG} push ${env.IMAGE_NAME}:${BRANCH_NAME}-latest
"""
}
}
}
post {
always {
script {
sh "rm -rf ${DOCKER_CONFIG}"
sh 'docker logout quay.io'
}
}
}
}
stage('Test built image') {
agent {
docker {
image env.IMAGE_NAME_TAG
args '-v /var/run/docker.sock:/var/run/docker.sock --group-add docker --group-add input --group-add render'
}
}
steps {
echo 'Debug basics'
sh '''
locale
printenv
'''

echo 'Test tools'
sh '''
java -version
mvn --version
python --version
pip --version
node --version
npm --version
go version
s2i version
gh version
kubectl version --client
oc version --client
'''

echo 'Test docker'
sh '''
docker info
docker run hello-world
'''
}
}
}
}
16 changes: 8 additions & 8 deletions .ci/jenkins/Jenkinsfile.build-operator-node
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import org.jenkinsci.plugins.workflow.libs.Library

pipeline {
agent {
label 'kie-rhel8-priority && !built-in'
label 'ubuntu'
}

options {
Expand All @@ -20,15 +20,15 @@ pipeline {
steps {
script {
deleteDir()
checkout(githubscm.resolveRepository('kogito-pipelines', 'kiegroup', 'main', false))
checkout(githubscm.resolveRepository('incubator-kogito-pipelines', 'apache', 'main', false))
}
}
}
stage('Build image') {
steps {
script {
dir('kogito-operator-jenkins-node') {
sh 'cekit -v build podman'
sh 'cekit -v build docker'
}
}
}
Expand All @@ -43,17 +43,17 @@ pipeline {
echo "Got version for tagging -> ${version}"

sh """
podman tag kogito-operator-jenkins-node:latest quay.io/kiegroup/kogito-operator-jenkins-node:${version}
podman tag kogito-operator-jenkins-node:latest quay.io/kiegroup/kogito-operator-jenkins-node:latest
docker tag kogito-operator-jenkins-node:latest quay.io/kiegroup/kogito-operator-jenkins-node:${version}
docker tag kogito-operator-jenkins-node:latest quay.io/kiegroup/kogito-operator-jenkins-node:latest
"""

withCredentials([usernamePassword(credentialsId: 'nightly_kogito', usernameVariable: 'REGISTRY_USER', passwordVariable: 'REGISTRY_PWD')]) {
sh "podman login --tls-verify=false -u ${REGISTRY_USER} -p ${REGISTRY_PWD} quay.io"
sh "docker login -u ${REGISTRY_USER} -p ${REGISTRY_PWD} quay.io"
}

sh """
podman push quay.io/kiegroup/kogito-operator-jenkins-node:${version}
podman push quay.io/kiegroup/kogito-operator-jenkins-node:latest
docker push quay.io/kiegroup/kogito-operator-jenkins-node:${version}
docker push quay.io/kiegroup/kogito-operator-jenkins-node:latest
"""
}
}
Expand Down
4 changes: 2 additions & 2 deletions .ci/jenkins/Jenkinsfile.nightly
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ UNSTABLE_STAGES = [:]
// Should be multibranch pipeline
pipeline {
agent {
label 'kie-rhel8 && !built-in'
label 'ubuntu'
}

options {
Expand Down Expand Up @@ -118,7 +118,7 @@ pipeline {
// Update examples nightly branch
dir('kogito-examples') {
deleteDir()
checkout(githubscm.resolveRepository('kogito-examples', getGitAuthor(), getBuildBranch(), false, getGitAuthorCredsID()))
checkout(githubscm.resolveRepository('incubator-kie-kogito-examples', getGitAuthor(), getBuildBranch(), false, getGitAuthorCredsID()))
String nightlyBranch = "nightly-${getBuildBranch()}"
githubscm.createBranch(nightlyBranch)
githubscm.pushObject('origin', nightlyBranch, getGitAuthorCredsID())
Expand Down
6 changes: 3 additions & 3 deletions .ci/jenkins/Jenkinsfile.nightly.cloud
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ promoteImageParamsPrefix = 'PROMOTE_IMAGE'
// Should be multibranch pipeline
pipeline {
agent {
label 'kie-rhel8 && !built-in'
label 'ubuntu'
}

options {
Expand Down Expand Up @@ -266,11 +266,11 @@ void addSkipIntegrationTestsParam(buildParams) {

void addAppsParam(buildParams) {
addStringParam(buildParams, 'APPS_REF', "${getBuildBranch()}")
addStringParam(buildParams, 'APPS_URI', "https://github.com/${getGitAuthor()}/kogito-apps")
addStringParam(buildParams, 'APPS_URI', "https://github.com/${getGitAuthor()}/incubator-kie-kogito-apps")
}

void addExamplesParam(buildParams) {
addStringParam(buildParams, 'EXAMPLES_URI', "https://github.com/${getGitAuthor()}/kogito-examples")
addStringParam(buildParams, 'EXAMPLES_URI', "https://github.com/${getGitAuthor()}/incubator-kie-kogito-examples")
addStringParam(buildParams, 'EXAMPLES_REF', "nightly-${getBuildBranch()}")
}

Expand Down
25 changes: 12 additions & 13 deletions .ci/jenkins/Jenkinsfile.nightly.quarkus-platform
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,17 @@ import org.kie.jenkins.MavenSettingsUtils

pipeline {
agent {
label 'rhel8 && docker && !built-in'
docker {
image env.AGENT_DOCKER_BUILDER_IMAGE
args env.AGENT_DOCKER_BUILDER_ARGS
}
}

options {
timestamps()
timeout(time: 240, unit: 'MINUTES')
}

tools {
maven "${BUILD_MAVEN_TOOL}"
jdk "${BUILD_JDK_TOOL}"
}

environment {
// Static env is defined into .jenkins/dsl/jobs.groovy file
KOGITO_CI_EMAIL_TO = credentials("${JENKINS_EMAIL_CREDS_ID}")
Expand All @@ -44,27 +42,27 @@ pipeline {
script {
dir('drools') {
deleteDir()
checkout(githubscm.resolveRepository('drools', getGitAuthor(), getTargetBranch(7), false))
checkout(githubscm.resolveRepository('incubator-kie-drools', getGitAuthor(), getTargetBranch(7), false))
}
dir('kogito-runtimes') {
deleteDir()
checkout(githubscm.resolveRepository('kogito-runtimes', getGitAuthor(), getGitBranch(), false))
checkout(githubscm.resolveRepository('incubator-kie-kogito-runtimes', getGitAuthor(), getGitBranch(), false))
}
dir('kogito-apps') {
deleteDir()
checkout(githubscm.resolveRepository('kogito-apps', getGitAuthor(), getGitBranch(), false))
checkout(githubscm.resolveRepository('incubator-kie-kogito-apps', getGitAuthor(), getGitBranch(), false))
}
dir('optaplanner') {
deleteDir()
// Get current corresponding branch and if not working, latest tag
String opBranch = getTargetBranch(7)
try {
checkout(githubscm.resolveRepository('optaplanner', getGitAuthor(), opBranch, false))
checkout(githubscm.resolveRepository('incubator-kie-optaplanner', getGitAuthor(), opBranch, false))
} catch(err) {
echo "Error checking out ${opBranch}. Fallback to latest tag."
String latestTag = githubscm.getLatestTag("${opBranch.split("\\.")[0]}\\.")
echo "Got latest tag = ${latestTag}"
checkout(githubscm.resolveRepository('optaplanner', getGitAuthor(), latestTag, false))
checkout(githubscm.resolveRepository('incubator-kie-optaplanner', getGitAuthor(), latestTag, false))
}
}
}
Expand Down Expand Up @@ -156,8 +154,9 @@ pipeline {
.run('deploy')

// Upload to specific repository with credentials
String mavenDeployRepositoryZipUrl = "${QUARKUS_PLATFORM_NEXUS_URL.replaceAll('/content/', '/service/local/').replaceFirst('/*$', '')}/content-compressed"
maven.uploadLocalArtifacts("${QUARKUS_PLATFORM_NEXUS_CREDS}", getMavenDeployLocalDir(), mavenDeployRepositoryZipUrl)
// Removed deployment to reposiory as no repository defined for now ...
// String mavenDeployRepositoryZipUrl = "${QUARKUS_PLATFORM_NEXUS_URL.replaceAll('/content/', '/service/local/').replaceFirst('/*$', '')}/content-compressed"
// maven.uploadLocalArtifacts("${QUARKUS_PLATFORM_NEXUS_CREDS}", getMavenDeployLocalDir(), mavenDeployRepositoryZipUrl)
}
}
}
Expand Down
Loading