EPIC: Jenkins maintenance

[sxa]

Creating this as an umbrella to tie all the jenkins activities together:

• Ongoing - #lts updates
• Audit all plugins on our jenkins server #3214
• Improve management of jenkins "jobs" disk usage
• Evaluate job restrictions plugin to improve build/test/(trestle?) machine isolation #3379
• Jenkins: Update matrix authorization plugin #3384
• Jenkins: Look at status/process surrounding Script Approvals #3385
• Mitigate jenkins faults reported by Upguard #3400
• Incompatible JVM versions in the jenkins log #3442
• Clean up home file system on jenkins master #3491
• jenkins: Investigate whether javaposse.jobdsl.plugin.ExecuteDslScripts needs to be retained #3500
• Jenkins: audit of any retained individual build jobs to reduce disk usage #3409
• Jenkins: audit tabs at the top of the jenkins #3622
• Investigate exceptions in jenkins startup log #3645
• Move away from using jenkins as a CDN (dependency jobs) #3811

Log warnings (=Potential security issues)

• "Skipped parameter" warnings in jenkins server log #2774
• "Unable to read..." exceptions in jenkins server log #3469
• Identify source of invalid crumb starting 6fc027348 and remediate #3485
• Clean up jenkins nodes which are not contactable over ssh #3486
• jenkins: openpower-controller.osuosl.org: Name or service not known #3487
• jenkins log: java.lang.IllegalStateException: completed or broken execution #3496
• Investigate / clear up exceptions in jenkins system log #3514

[sxa]

You can run this over the logs to produce a summary of each type of entry failure from the log issues above (Number at the start of each line in the output is the issue number):
jenkinslog-process.sh.txt