Conifguration managment for sunbird deployment

CONFIGURATION MANAGEMENT:

1)Jenkins jobs to pick the environment variable from globally

• Create one global variable in jenkins RELEASE_BRANCH in manage-jenkins/configure system.
• Everyjob should contain the string parameter RELEASE_BRANCH.
• Everytime start of the release, update the global variable RELEASE_BRANCH with the current release.

2)Steps after the release is completed

• Create a tag for the completed release.
• Merge the release branch into the master.
• Delete the release branch.

3)Hotfix of previous release

• Create a hotfix branch using previous release tag.
• Just pass the value RELEASE_BRANCH=hostfix branch name while running jobs.
• After the deployment, merge the branch to master.
• Tag the new hotfix branch.
• Delete the hotfix the branch.

4)Sunbird docker images version

• Docker image tag should be changed from the random version (0.0.7 or 0.0.1) to release branch version.
• Version should have the major+minor number.
• Pick the docker image major version number from the RELEASE_BRANCH global variable.
• If it is a hotfix it picks docker image major version number from the RELEASE_BRANCH local variable of particular job.
• Minor number can be the Build number or updated manually.
• Remove the docker image version from the package.json and pom.xml. metadata.sh will pick from the Jenkins global var of RELEASE_BRANCH

5)Disadvantages of having minor number taking build number

• Docker hub will be have many bronze images.
• Minor number mismatch will happen while doing hotfix deployment.
• How can we bring the minor version to 1 when a new release starts.
• How to fix if the jenkins job is deleted.

6)Automate creating the branch

• Create a jenkins job which will create the new release branch across all the sunbird repos when release is started.
• Above job picks the new release branch name from the RELEASE_BRANCH global variable.
• We should make sure previous release branch is merged to the master before running this job.

ROLL BACK OF DEPLOYMENT

7)Roll back of production deployment

• It achieved if we followed the release branching strategy.
• Passing version number of the release branch is challenging while production roll back.

POST DEPLOYMENT VERIFICATION

8)Health check for services

• Check the service is up or not.
• Check connectivity between the services is good.
• Check the connectivity from the services to databases.

9)Check the replication

• Verify all the containers started.
• Check docker service ps
• Check for which version is deployed.

10)Check for the cpu,disk and ram

• Given limits and reservations for each service .
• Check the how much each service is consumed.