-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add config files for automated Jenkins setup #5
Changes from 4 commits
f896541
8e0c4b7
eb45fb6
a3d689d
7ee15cb
ea6e91d
747c8d9
6bf1e64
b72ec91
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,12 @@ | ||
# o3de-jenkins-pipeline | ||
# o3de-jenkins-pipeline | ||
|
||
|
||
## Jenkins Configuration | ||
|
||
The O3DE Jenkins pipeline is configured using the JCasC plugin. This allows the settings to be defined using yaml files stored in the repo and prevents users from having to manually configure Jenkins on startup. | ||
|
||
These files are located in the configs/ directory. There is a main jenkins.yaml file that contains the recommended default settings to run the pipeline. You can customize your Jenkins setup by editing this file. There are also template files that can be used for other optional settings. You can either copy the contents of these files into jenkins.yaml or copy it into a new yaml file. | ||
|
||
|
||
### Config File location | ||
The config files must be located in the configs/ directory so that they are found by the plugin. The plugin also supports having multiple yaml files and will locate all files having the .yml/.yaml extention. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
# Config for AWS EC2 plugin | ||
# AWS Clouds are configured by region. Add a new '- amazonEC2' entry under clouds to add new regions. | ||
# Add a new '- ami' entry under templates to add new build node configs. | ||
jenkins: | ||
clouds: | ||
- amazonEC2: | ||
# Example: cloud setup in us-west-2 | ||
cloudName: "<name>" | ||
instanceCapStr: "100" # A cap can also be set for each AMI config. | ||
region: "us-west-2" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe turn There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I can add a default region env var. If multiple regions are required a user can just add them there. |
||
sshKeysCredentialsId: "<credential-id>" # Use the ssh credential ID created in credentials.yaml | ||
templates: | ||
# Example: Linux build node using SSH in us-west-2a | ||
- ami: "${LINUX_AMI_ID}" | ||
amiType: | ||
unixData: | ||
sshPort: "22" | ||
connectionStrategy: PRIVATE_IP | ||
description: "<description>" | ||
ebsEncryptRootVolume: ENCRYPTED | ||
ebsOptimized: true | ||
hostKeyVerificationStrategy: ACCEPT_NEW | ||
iamInstanceProfile: "<instance-profile-arn>" | ||
idleTerminationMinutes: "30" | ||
labelString: "linux" | ||
launchTimeoutStr: "300" | ||
maxTotalUses: 1 | ||
minimumNumberOfInstances: 0 | ||
minimumNumberOfSpareInstances: 0 | ||
numExecutors: 1 | ||
remoteAdmin: "<username>" | ||
remoteFS: "<remote-home-directory>" | ||
securityGroups: "${SECURITY_GROUP_ID}" | ||
stopOnTerminate: false | ||
subnetId: "<subnet-id>" | ||
tags: | ||
- name: "Name" | ||
value: "<name-tag>" | ||
tenancy: Default | ||
type: <instance-type> # e.g. C54xlarge | ||
useEphemeralDevices: false | ||
zone: "us-west-2a" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# Config for AWS S3 artifact storage plugin | ||
aws: | ||
awsCredentials: | ||
region: "<region>>" | ||
s3: | ||
container: "<s3-bucket-name>" | ||
unclassified: | ||
artifactManager: | ||
artifactManagerFactories: | ||
- jclouds: | ||
provider: "s3" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: Newline at end of file. Couple other files are missing this as well There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sure thing, will fix. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
# Config for Build Failure Analyzer plugin | ||
# Example: Setup using MongoDB | ||
brianherrera marked this conversation as resolved.
Show resolved
Hide resolved
|
||
unclassified: | ||
buildFailureAnalyzer: | ||
gerritTriggerEnabled: true | ||
globalEnabled: true | ||
knowledgeBase: | ||
mongoDB: | ||
dbName: "<db-name>" | ||
enableStatistics: true | ||
host: "<endpoint>" | ||
password: "{<password-credential-name>}" | ||
port: 27017 | ||
successfulLogging: false | ||
userName: "<username>" | ||
maxLogSize: 0 | ||
noCausesEnabled: true | ||
noCausesMessage: "<message>" | ||
nrOfScanThreads: 3 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# Config for Jenkins credentials | ||
# Do not add credential values directly to this file. | ||
# See: https://github.com/jenkinsci/configuration-as-code-plugin/blob/master/docs/features/secrets.adoc | ||
credentials: | ||
system: | ||
domainCredentials: | ||
- credentials: | ||
# Example: SSH access for build nodes | ||
- basicSSHUserPrivateKey: | ||
id: "ssh-build-node" | ||
privateKeySource: | ||
directEntry: | ||
privateKey: "<private-credential-name>" | ||
scope: GLOBAL | ||
username: "<username>" | ||
# Example: basic username/password | ||
- usernamePassword: | ||
id: "password-access" | ||
password: "<password-credential-name>" | ||
scope: GLOBAL | ||
username: "<username>" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
# Config for Jenkins global environment variables | ||
# Add a new key/value pair under env to add new entries | ||
jenkins: | ||
globalNodeProperties: | ||
- envVars: | ||
env: | ||
- key: "<key1>" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe put in some required parameters here? For instance, I see that There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The url and email address are set in jenkins.yaml. I can look at our current env var set and see if there are values that are require to run the pipe. |
||
value: "<value1>" | ||
- key: "<key2>" | ||
value: "<value2>" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
jenkins: | ||
numExecutors: 6 # Jenkins pipeline runs some pre/post-build pipeline tasks on the controller. Increased from default (2) to prevent long queues. | ||
securityRealm: | ||
github: | ||
githubWebUri: "https://github.com" | ||
githubApiUri: "https://api.github.com" | ||
clientID: "${GITHUB_ID}" | ||
clientSecret: "${GITHUB_SECRET}" | ||
oauthScopes: "read:org,user:email" | ||
authorizationStrategy: | ||
projectMatrix: | ||
permissions: | ||
- "Overall/Administer:${ADMIN_ROLE}" # User/group assigned as admin. For additional permissions see matrix_authorization.template | ||
remotingSecurity: | ||
enabled: true | ||
unclassified: | ||
location: | ||
url: ${JENKINS_URL} | ||
adminAddress: "${JENKINS_EMAIL_ADDRESS}" | ||
gitHubConfiguration: | ||
apiRateLimitChecker: ThrottleOnOver | ||
email-ext: | ||
defaultBody: |- | ||
$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS: | ||
|
||
Check console output at $BUILD_URL to view the results. | ||
defaultContentType: "text/plain" | ||
defaultSubject: "$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS!" | ||
defaultTriggerIds: | ||
- "hudson.plugins.emailext.plugins.trigger.FailureTrigger" | ||
mailAccount: | ||
smtpHost: "${SMTP_HOST}" | ||
smtpPassword: "${SMTP_PASSWORD}" | ||
smtpUsername: "${SMTP_USERNAME}" | ||
useTls: true | ||
security: | ||
scriptApproval: | ||
approvedSignatures: | ||
- "field hudson.plugins.git.GitSCM GIT_COMMIT" | ||
- "method org.jenkinsci.plugins.workflow.steps.FlowInterruptedException getCauses" | ||
- "new java.net.URL java.lang.String java.lang.String int java.lang.String" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
# Config for Project-based Matrix Authorization Strategy | ||
# Use this file to grant additional permissions | ||
# Default admin role granted in jenkins.yaml | ||
jenkins: | ||
authorizationStrategy: | ||
projectMatrix: | ||
permissions: | ||
# Example: grant permissions to run/cancel builds | ||
- "Job/Build:<user/group>" | ||
- "Job/Cancel:<user/group>" | ||
- "Job/Read:<user/group>" | ||
- "Overall/Read:<user/group>" | ||
- "View/Read:<user/group>" | ||
# Example: grant additional admin permissions | ||
- "Overall/Administer::<user/group>" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wouldn't failing to set this properly cause the instance to be inaccessible? If so, probably want to at least set a default user There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These permissions are optional. The default admin is set in jenkins.yaml. |
||
# Example: grant public read access | ||
- "Overall/Read:anonymous" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These
<value>
entries are not pulled from environment, right? It's for the user to replace?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, these can simply be replaced.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might make it more clear to denote that in the comments, so that someone consuming this doesn't miss something
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would be great if these end up later on being some wizard that asks for the values and replaces them (generating the files)
May need some more easier to identify format for them
Should be fine for the mean time
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, I can create a script that converts these templates to .yaml files and prompts the user for the required values.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, agree, as a first pass, this is fine. End goal is to have a generator replace everything from a config file or pull from some secure parameter store (probably something like Jinja can do).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Created this issue to track: #6