Improvement suggestions for the SBoM generation process

[sxa]

Having just completed the work for #3538 I have some notes which I want to capture for discussion regarding potential improvements to the process:

• I'm not a fan of the https://ci.adoptium.net/job/build.getDependency job - it seems to run daily for no obvious reason (and was not purging old output so was constantly chewing up an increasing amount of disk space - now fixed) and explicitly "loses" the version information in the jars by storing to a filename with the version removed. Because of this, if you try to build an old version of the JDK with the old scripts for reproducibility the SBoM will not be the same since we've now updated the CycloneDX version, and indeed the SHA check on the download will surely fail.
• As an alternative to the above we could consider installing the cycloneDX jars on the build machines using the ansible playbooks. This would mean that we do it once and don't have to be relying on the network during the (critical) build process.
• We seem to have a lot of layers here - for adding a new section you need to do the following steps and I wonder if it could be optimised? Perhaps extract the SBoM code from build.sh (it's not easy to run just the sbom generation for testing changes at the moment) and just have a generateSBoM.sh instead of the separate common script for functions to reduce compexity? That could directly invoke the java code instead of the extra layer of shell functions (Since they are mostly one-liners) As a minimum if we haven't got the following steps documented somewhere we should do so - there is currently no README.md in the cyclonedx-lib directory.
  • Add a new function to sbin/common/sbom.sh
  • Call that function from sbin/build.sh
  • Add a new parameter to TemurinGenSBOM.java
  • Add a new function to TemurinGenSBOM.java to handle the parameter
  • Add tests to build.xml to verify the new functions work as expected
• Also if there is a problem with the code that means there is a runtime error the way it's run by default does not allow it to show the source line, despite the source being accessible on the machine since we've just compiled it e.g. at temurin.sbom.TemurinGenSBOM.main(Unknown Source)
• It's also somewhat inefficient to have to call into Java for each addition so I wonder if the operations can be batched in a meaningful way somehow.
• If a value isn't specified (missing/incorrect parameter) then the methods still get called with null objects. Perhaps this should be trapped with parameter validation in each of the lowest level methods so that we don't create invalid objects (or more likely generate NullPOinterExcpetions which are hard to track down due to the absence of source line numbers in the exception stacks.
• Should --create-sbom be the default instead of a parameter added by our jenkins pipelines?
  Tieing this to the issue about SBoM content too just to collate things together: Discuss Temurin SBOM format and content #3013

[sxa]

See also: #3621
We could also allow this to test a download from the mirror server ...