From b370babfe8cacc39f8c25de8907154a6ca3b77c4 Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Thu, 24 Oct 2024 15:55:07 +0200 Subject: [PATCH 01/16] Added ADR describing support of SDPX SBOM format in konflux Signed-off-by: Jindrich Luza --- ADR/0039-spdx-support.md | 144 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 144 insertions(+) create mode 100644 ADR/0039-spdx-support.md diff --git a/ADR/0039-spdx-support.md b/ADR/0039-spdx-support.md new file mode 100644 index 00000000..ff85333e --- /dev/null +++ b/ADR/0039-spdx-support.md @@ -0,0 +1,144 @@ +# 0. Record architecture decisions + +Date: 2024-09-24 + +## Status + +Proposed + +## Context + +SPDX SBOM format enables additional features not available in cyclondedx like multiple purl attributes per component. SPDX is also a widely adopted standard for software bill of materials. +This ADR describes how to enable use of SPDX SBOM format in Konflux. + +## Decision + + +### SBOM lifecycle in build pipeline + +At the start SBOMs are generated by cachi2 and syft. At later phase of the build pipeline, SBOMs from base images are extracted and merged toghether with SBOMs generated for the currently build container. When switching to spdx, newly built container image can be used as base image another container image. Therefore any tooling/task which works with sboms has to be able to work with both formats. +Due to that fact, all tools and tasks should implement sbomType attribute which will be used to determine sbom format expected on the input and produced on the output. It will also allow tools to be tested for spdx before whole ppile is switched to spdx. + +### CycloneDX -> SPDX conversion + +CycloneDX (1.4) is structured document in json format with following structure (not full specification) + +- Document Root + - Metadata + - Tools + - List + - vendor + - name + - Components + - List + - name + - version + - purl + - properties + - List + - name + - value + - formulations + - List + +SPDX (2.3) is structured document in json format with following structure(not full specification): +- Document Root + - name + - SPDXID + - creationInfo + - Creators + - List + - components + - List + - SPDXID + - name + - versionInfo + - externalRefs + - List + - referenceCategory + - referenceType + - referenceLocator + - annotations + - List + - annotationDate + - annotationType + - annotator + - Comment + - relationships + - List + - spdxElementId + - relationshipType + - relatedSpdxElement + +#### 1:1 conversions +Following CycloneDX to SPDX attributes are converted as 1:1 as they represent the same thing. + +| SPDX Attribute | CycloneDX Attribute | +|-----------------------|---------------------| +| components | packages | +| component.name | package.name | +| component.versionInfo | package.version | +|-----------------------|---------------------| + + +#### Component.purl +CycloneDX (version 1.4) supports only a single purl attribute per component. SPDX doesn’t have a direct attribute, but instead every package includes an externalRefs array which describes all external references for the package. There are defined reference categories and types. For PURL, category PACKAGE-MANAGER and type purl is used. The purl itself will be stored as referenceLocator + +|------------------------------|---------------------------------------------------------------| +| component.purl = | package.externalRefs = [{referenceCategory:”PACKAGE-MANAGER”, | +| | referenceType:purl, | +| | referenceLocator: | +| | }] | +|------------------------------|---------------------------------------------------------------| + + +#### Component.properties +CycloneDX components properties describe mapping of string:string properties for given component. SPDX component doesn’t have anything similar to cyclonedx properties. SPDX Package annotations are the only attribute where custom data can be stored and the only “customizable” field there is comment which is simple string. Due to that fact, cycloneDX property in format of {“name”: , “value”: } is encoded into json string. There can be also annotations produced by other tools. Therefore to be able to tell annotation comment is json encoded, annotator should ends with string “:jsonencoded” + +|---------------------------------|------------------------------------------------------------| +| package.properties = [ | component.annotations = [ | +| {“name”: …, “value”: …} | {..., annotator: ”:jsonencoded” | +| ] | ] | +|---------------------------------|------------------------------------------------------------| + + +#### Formulations +CycloneDX formulations describe how the container was manufactured. In SPDX, Relationship elements can be used for the same purpose. All elements in SPDX have SPDXID attribute which is an element identifier unique in the whole SBOM document. Relationship element describes relation between two elements using their SPDXID and relationship type. Relationship type BUILD_TOOL_OF can be used to express the relationship of packages which were used to build the container. + +|---------------------------------|------------------------------------------------------------| +| Formulations.components = [{}] | Relationships = [{ | +| | spdxElementId = | +| | relationshipType=BUILD_TOOL_OF | +| | relatedSpdxElement= | +| | }] | +|---------------------------------|------------------------------------------------------------| + + +#### Metadata.tools +For CycloneDX tools sub attributes, we are mostly interested in, are vendor and name elements. Information about creation of SPDX document can be stored into creationInfo. CreationInfo.creators element is basically a list of strings. There’s vague specification about how it should be structured in the standard. Strings should be formatted in the following way: “: ”. For example vendor should be stored as “Vendor: ” + +|------------------------------------------------|--------------------------------------------------| +| Metadata.tools = [{“vendor”: “X”, “name”: “Y”] |CreationInfo.creators = [“Vendor: X”, “Tool: Y”] | +|------------------------------------------------|--------------------------------------------------| + + +#### Merging SPDX +##### Packages +Packages of two SPDX documents can be merged together as a concatenation of two lists. In cycloneDX component elements can have only a single purl attribute, therefore component elements representing packages with the same name and version but with different purl have to be stored as multiple elements. CycloneDX package elements can bear multiple purls. Therefore multiple cycloneDX components can be squashed together into single SPDX package element with purls concatenated into a single list. Following rules are applied to packages merging process: +Packages with the same package name and versionInfo are squashed into single package element +Package with the same package name and versionInfo set to None (or empty) is squashed with package with the same name and non-empty versionInfo +NOTE: packages cannot be merged together based on SPDXID attribute as there’s no specification in spdx standard how SPDXID should be calculated. Individual tools can calculate it differently while still passing condition to make it unique across the whole document. +##### Relationships +SPDX relationships represent graph/tree structure of relations of elements in the document. Root element is SPDX document itself. Individual packages are in relationship CONTAINS with the root document. In the case of packages which were used to build the container. Packages are in relationship BUILD_TOOL_OF with the root document. +NOTE: Packages can be wrapped in a “virtual package”. This package can have empty name and version and on attributes, or name can be set to directory or container name which was set as source for generating SBOM document. This element has insignificant information value and can be omitted. +Relations of two documents needs to be merged together into single graph in a way which keeps the graph structure of the original graph of the main document (into which other document will be merged to). Once packages are merged together, relationships of the second document can be cleared off relations which refer to packages not included in the merged package list. SpdxElementId and relatedSpdxElement point to root document id of the second document should be replaced with root document id of the main document. If there’s “virtual package” in the second document, ids in relationships referring to it should be replaced with “virtual package” of the main document or main document id directly (if there’s no “virtual package”) + + +## Consequences +All tooling used in pipeline needs to support SPDX SBOM format + + +## References +CycloneDX specification https://cyclonedx.org/specification/overview/o +SPDX specification https://spdx.github.io/spdx-spec/v2.3/ +SPDX json schema https://github.com/spdx/spdx-spec/blob/development/v2.3/schemas/spdx-schema.json#L724 From 43df786602d8528495166ec2f7623f210393e187 Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Fri, 25 Oct 2024 09:44:23 +0200 Subject: [PATCH 02/16] - Fixed tables - Fixed title - Fixed typos and text corrections Signed-off-by: Jindrich Luza --- ADR/0039-spdx-support.md | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/ADR/0039-spdx-support.md b/ADR/0039-spdx-support.md index ff85333e..fbbc69d6 100644 --- a/ADR/0039-spdx-support.md +++ b/ADR/0039-spdx-support.md @@ -1,4 +1,4 @@ -# 0. Record architecture decisions +# 39. SPDX SBOM support Date: 2024-09-24 @@ -17,7 +17,10 @@ This ADR describes how to enable use of SPDX SBOM format in Konflux. ### SBOM lifecycle in build pipeline At the start SBOMs are generated by cachi2 and syft. At later phase of the build pipeline, SBOMs from base images are extracted and merged toghether with SBOMs generated for the currently build container. When switching to spdx, newly built container image can be used as base image another container image. Therefore any tooling/task which works with sboms has to be able to work with both formats. -Due to that fact, all tools and tasks should implement sbomType attribute which will be used to determine sbom format expected on the input and produced on the output. It will also allow tools to be tested for spdx before whole ppile is switched to spdx. +As a result, all tools and tasks should implement the sbomType attribute to specify the expected SBOM format for input and output. This will also allow tools to be tested with SPDX before the entire pipeline transitions to this format. + +Initially, SBOMs are generated by Cachi2 and Syft. In a later phase of the build pipeline, SBOMs from base images are extracted and merged with those generated for the container currently being built. When transitioning to SPDX, the newly built container image can serve as a base image for another container. Therefore, any tools or tasks that work with SBOMs must be compatible with both formats. + ### CycloneDX -> SPDX conversion @@ -78,59 +81,58 @@ Following CycloneDX to SPDX attributes are converted as 1:1 as they represent th | components | packages | | component.name | package.name | | component.versionInfo | package.version | -|-----------------------|---------------------| #### Component.purl CycloneDX (version 1.4) supports only a single purl attribute per component. SPDX doesn’t have a direct attribute, but instead every package includes an externalRefs array which describes all external references for the package. There are defined reference categories and types. For PURL, category PACKAGE-MANAGER and type purl is used. The purl itself will be stored as referenceLocator +| SPDX Attribute | CycloneDX Attribute | |------------------------------|---------------------------------------------------------------| | component.purl = | package.externalRefs = [{referenceCategory:”PACKAGE-MANAGER”, | | | referenceType:purl, | | | referenceLocator: | | | }] | -|------------------------------|---------------------------------------------------------------| #### Component.properties -CycloneDX components properties describe mapping of string:string properties for given component. SPDX component doesn’t have anything similar to cyclonedx properties. SPDX Package annotations are the only attribute where custom data can be stored and the only “customizable” field there is comment which is simple string. Due to that fact, cycloneDX property in format of {“name”: , “value”: } is encoded into json string. There can be also annotations produced by other tools. Therefore to be able to tell annotation comment is json encoded, annotator should ends with string “:jsonencoded” +CycloneDX components properties describe mapping of string:string properties for given component. SPDX component doesn’t have anything similar to cyclonedx properties. SPDX Package annotations are the only attribute where custom data can be stored and the only “customizable” field where there is comment which is a simple string. Due to that fact, cycloneDX property in format of {“name”: , “value”: } is encoded into json string. There can be also annotations produced by other tools. Therefore to be able to tell annotation comment is json encoded, annotator should ends with string “:jsonencoded” +| SPDX Attribute | CycloneDX Attribute | |---------------------------------|------------------------------------------------------------| | package.properties = [ | component.annotations = [ | | {“name”: …, “value”: …} | {..., annotator: ”:jsonencoded” | | ] | ] | -|---------------------------------|------------------------------------------------------------| #### Formulations CycloneDX formulations describe how the container was manufactured. In SPDX, Relationship elements can be used for the same purpose. All elements in SPDX have SPDXID attribute which is an element identifier unique in the whole SBOM document. Relationship element describes relation between two elements using their SPDXID and relationship type. Relationship type BUILD_TOOL_OF can be used to express the relationship of packages which were used to build the container. +| SPDX Attribute | CycloneDX Attribute | |---------------------------------|------------------------------------------------------------| | Formulations.components = [{}] | Relationships = [{ | | | spdxElementId = | | | relationshipType=BUILD_TOOL_OF | | | relatedSpdxElement= | | | }] | -|---------------------------------|------------------------------------------------------------| #### Metadata.tools -For CycloneDX tools sub attributes, we are mostly interested in, are vendor and name elements. Information about creation of SPDX document can be stored into creationInfo. CreationInfo.creators element is basically a list of strings. There’s vague specification about how it should be structured in the standard. Strings should be formatted in the following way: “: ”. For example vendor should be stored as “Vendor: ” +The CycloneDX metadata.tools sub attributes that we are mostly interested in are the vendor and name elements. Information about the creation of the SPDX document can be stored into creationInfo. CreationInfo.creators element is basically a list of strings. There’s a vague specification about how it should be structured in the standard. Strings should be formatted in the following way: “: ”. For example vendor should be stored as “Vendor: ” -|------------------------------------------------|--------------------------------------------------| -| Metadata.tools = [{“vendor”: “X”, “name”: “Y”] |CreationInfo.creators = [“Vendor: X”, “Tool: Y”] | -|------------------------------------------------|--------------------------------------------------| +| SPDX Attribute | CycloneDX Attribute | +|------------------------------------------------|---------------------------------------------------| +| Metadata.tools = [{“vendor”: “X”, “name”: “Y”] | CreationInfo.creators = [“Vendor: X”, “Tool: Y”] | #### Merging SPDX ##### Packages Packages of two SPDX documents can be merged together as a concatenation of two lists. In cycloneDX component elements can have only a single purl attribute, therefore component elements representing packages with the same name and version but with different purl have to be stored as multiple elements. CycloneDX package elements can bear multiple purls. Therefore multiple cycloneDX components can be squashed together into single SPDX package element with purls concatenated into a single list. Following rules are applied to packages merging process: -Packages with the same package name and versionInfo are squashed into single package element -Package with the same package name and versionInfo set to None (or empty) is squashed with package with the same name and non-empty versionInfo -NOTE: packages cannot be merged together based on SPDXID attribute as there’s no specification in spdx standard how SPDXID should be calculated. Individual tools can calculate it differently while still passing condition to make it unique across the whole document. +- Packages with the same package name and versionInfo are squashed into single package element +- Package with the same package name and versionInfo set to None (or empty) is squashed with package with the same name and non-empty versionInfo +NOTE: packages cannot be merged together based on SPDXID attribute as there’s no specification in the spdx standard on how SPDXID should be calculated. Individual tools can calculate it differently while still passing condition to make it unique across the whole document. ##### Relationships -SPDX relationships represent graph/tree structure of relations of elements in the document. Root element is SPDX document itself. Individual packages are in relationship CONTAINS with the root document. In the case of packages which were used to build the container. Packages are in relationship BUILD_TOOL_OF with the root document. -NOTE: Packages can be wrapped in a “virtual package”. This package can have empty name and version and on attributes, or name can be set to directory or container name which was set as source for generating SBOM document. This element has insignificant information value and can be omitted. +SPDX relationships represent graph/tree structure of relations of elements in the document. The Root element is the SPDX document itself. Individual packages are in relationship CONTAINS with the root document. In the case of packages which were used to build the container, packages are in relationship BUILD_TOOL_OF with the root document. +NOTE: Packages can be wrapped in a “virtual package”. This package can have empty name and version and no attributes, or name can be set to directory or container name which was set as source for generating SBOM document. This element has insignificant information value and can be omitted. Relations of two documents needs to be merged together into single graph in a way which keeps the graph structure of the original graph of the main document (into which other document will be merged to). Once packages are merged together, relationships of the second document can be cleared off relations which refer to packages not included in the merged package list. SpdxElementId and relatedSpdxElement point to root document id of the second document should be replaced with root document id of the main document. If there’s “virtual package” in the second document, ids in relationships referring to it should be replaced with “virtual package” of the main document or main document id directly (if there’s no “virtual package”) From f7189d5d93d0d65b14d44a43eb2425ac02d5dfde Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Tue, 29 Oct 2024 14:38:23 +0100 Subject: [PATCH 03/16] Fixed tables Signed-off-by: Jindrich Luza --- ADR/0039-spdx-support.md | 34 ++++++++++++++++------------------ 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/ADR/0039-spdx-support.md b/ADR/0039-spdx-support.md index fbbc69d6..05efcd12 100644 --- a/ADR/0039-spdx-support.md +++ b/ADR/0039-spdx-support.md @@ -19,8 +19,6 @@ This ADR describes how to enable use of SPDX SBOM format in Konflux. At the start SBOMs are generated by cachi2 and syft. At later phase of the build pipeline, SBOMs from base images are extracted and merged toghether with SBOMs generated for the currently build container. When switching to spdx, newly built container image can be used as base image another container image. Therefore any tooling/task which works with sboms has to be able to work with both formats. As a result, all tools and tasks should implement the sbomType attribute to specify the expected SBOM format for input and output. This will also allow tools to be tested with SPDX before the entire pipeline transitions to this format. -Initially, SBOMs are generated by Cachi2 and Syft. In a later phase of the build pipeline, SBOMs from base images are extracted and merged with those generated for the container currently being built. When transitioning to SPDX, the newly built container image can serve as a base image for another container. Therefore, any tools or tasks that work with SBOMs must be compatible with both formats. - ### CycloneDX -> SPDX conversion @@ -51,8 +49,8 @@ SPDX (2.3) is structured document in json format with following structure(not fu - creationInfo - Creators - List - - components - - List + - packages + - List - SPDXID - name - versionInfo @@ -76,17 +74,17 @@ SPDX (2.3) is structured document in json format with following structure(not fu #### 1:1 conversions Following CycloneDX to SPDX attributes are converted as 1:1 as they represent the same thing. -| SPDX Attribute | CycloneDX Attribute | -|-----------------------|---------------------| -| components | packages | -| component.name | package.name | -| component.versionInfo | package.version | +| CycloneDX Attribute | SPDX Attribute | +|----------------------------|---------------------| +| components | packages | +| component.name | package.name | +| component.version | package.versionInfo | #### Component.purl CycloneDX (version 1.4) supports only a single purl attribute per component. SPDX doesn’t have a direct attribute, but instead every package includes an externalRefs array which describes all external references for the package. There are defined reference categories and types. For PURL, category PACKAGE-MANAGER and type purl is used. The purl itself will be stored as referenceLocator -| SPDX Attribute | CycloneDX Attribute | +| CycloneDX Attribute | SPDX Attribute | |------------------------------|---------------------------------------------------------------| | component.purl = | package.externalRefs = [{referenceCategory:”PACKAGE-MANAGER”, | | | referenceType:purl, | @@ -97,17 +95,17 @@ CycloneDX (version 1.4) supports only a single purl attribute per component. SPD #### Component.properties CycloneDX components properties describe mapping of string:string properties for given component. SPDX component doesn’t have anything similar to cyclonedx properties. SPDX Package annotations are the only attribute where custom data can be stored and the only “customizable” field where there is comment which is a simple string. Due to that fact, cycloneDX property in format of {“name”: , “value”: } is encoded into json string. There can be also annotations produced by other tools. Therefore to be able to tell annotation comment is json encoded, annotator should ends with string “:jsonencoded” -| SPDX Attribute | CycloneDX Attribute | -|---------------------------------|------------------------------------------------------------| -| package.properties = [ | component.annotations = [ | -| {“name”: …, “value”: …} | {..., annotator: ”:jsonencoded” | -| ] | ] | +| CycloneDX Attribute | SPDX Attribute | +|-------------------------------------------|---------------------------------------------| +| components.properties = [ | package.annotations = [ | +| {“name”: …, “value”: …} | {..., annotator: ”:jsonencoded” | +| ] | ] | #### Formulations CycloneDX formulations describe how the container was manufactured. In SPDX, Relationship elements can be used for the same purpose. All elements in SPDX have SPDXID attribute which is an element identifier unique in the whole SBOM document. Relationship element describes relation between two elements using their SPDXID and relationship type. Relationship type BUILD_TOOL_OF can be used to express the relationship of packages which were used to build the container. -| SPDX Attribute | CycloneDX Attribute | +| CycloneDX Attribute | SPDX Attribute | |---------------------------------|------------------------------------------------------------| | Formulations.components = [{}] | Relationships = [{ | | | spdxElementId = | @@ -117,9 +115,9 @@ CycloneDX formulations describe how the container was manufactured. In SPDX, Rel #### Metadata.tools -The CycloneDX metadata.tools sub attributes that we are mostly interested in are the vendor and name elements. Information about the creation of the SPDX document can be stored into creationInfo. CreationInfo.creators element is basically a list of strings. There’s a vague specification about how it should be structured in the standard. Strings should be formatted in the following way: “: ”. For example vendor should be stored as “Vendor: ” +The CycloneDX metadata.tools sub attributes that we are mostly interested in are the vendor and name elements. Information about the creation of the SPDX document can be stored into creationInfo. CreationInfo.creators element is basically a list of strings. There’s a vague specification ([here](https://spdx.github.io/spdx-spec/v2.3/document-creation-information/#68-creator-field)]) about how it should be structured in the standard. Strings should be formatted in the following way: “: ”. For example vendor should be stored as “Vendor: ” -| SPDX Attribute | CycloneDX Attribute | +| CyloneDX Attribute | SPDX Attribute | |------------------------------------------------|---------------------------------------------------| | Metadata.tools = [{“vendor”: “X”, “name”: “Y”] | CreationInfo.creators = [“Vendor: X”, “Tool: Y”] | From cb666f889a7afadf3e11333195cd370661110859 Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Tue, 29 Oct 2024 14:39:31 +0100 Subject: [PATCH 04/16] Fixed typo --- ADR/0039-spdx-support.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ADR/0039-spdx-support.md b/ADR/0039-spdx-support.md index 05efcd12..bd7f6437 100644 --- a/ADR/0039-spdx-support.md +++ b/ADR/0039-spdx-support.md @@ -124,7 +124,7 @@ The CycloneDX metadata.tools sub attributes that we are mostly interested in are #### Merging SPDX ##### Packages -Packages of two SPDX documents can be merged together as a concatenation of two lists. In cycloneDX component elements can have only a single purl attribute, therefore component elements representing packages with the same name and version but with different purl have to be stored as multiple elements. CycloneDX package elements can bear multiple purls. Therefore multiple cycloneDX components can be squashed together into single SPDX package element with purls concatenated into a single list. Following rules are applied to packages merging process: +Packages of two SPDX documents can be merged together as a concatenation of two lists. In cycloneDX component elements can have only a single purl attribute, therefore component elements representing packages with the same name and version but with different purl have to be stored as multiple elements. SPDX package elements can bear multiple purls. Therefore multiple cycloneDX components can be squashed together into single SPDX package element with purls concatenated into a single list. Following rules are applied to packages merging process: - Packages with the same package name and versionInfo are squashed into single package element - Package with the same package name and versionInfo set to None (or empty) is squashed with package with the same name and non-empty versionInfo NOTE: packages cannot be merged together based on SPDXID attribute as there’s no specification in the spdx standard on how SPDXID should be calculated. Individual tools can calculate it differently while still passing condition to make it unique across the whole document. From 1a5e34a5332a233843611a5c242dee970aae9c72 Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Mon, 4 Nov 2024 16:00:32 +0100 Subject: [PATCH 05/16] - Update merging conditions - Added section about syft specific behaviour --- ADR/0039-spdx-support.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/ADR/0039-spdx-support.md b/ADR/0039-spdx-support.md index bd7f6437..52439fb0 100644 --- a/ADR/0039-spdx-support.md +++ b/ADR/0039-spdx-support.md @@ -125,15 +125,22 @@ The CycloneDX metadata.tools sub attributes that we are mostly interested in are #### Merging SPDX ##### Packages Packages of two SPDX documents can be merged together as a concatenation of two lists. In cycloneDX component elements can have only a single purl attribute, therefore component elements representing packages with the same name and version but with different purl have to be stored as multiple elements. SPDX package elements can bear multiple purls. Therefore multiple cycloneDX components can be squashed together into single SPDX package element with purls concatenated into a single list. Following rules are applied to packages merging process: -- Packages with the same package name and versionInfo are squashed into single package element -- Package with the same package name and versionInfo set to None (or empty) is squashed with package with the same name and non-empty versionInfo +- Packages with the same purl package name and version and type are squashed into single package element NOTE: packages cannot be merged together based on SPDXID attribute as there’s no specification in the spdx standard on how SPDXID should be calculated. Individual tools can calculate it differently while still passing condition to make it unique across the whole document. + ##### Relationships -SPDX relationships represent graph/tree structure of relations of elements in the document. The Root element is the SPDX document itself. Individual packages are in relationship CONTAINS with the root document. In the case of packages which were used to build the container, packages are in relationship BUILD_TOOL_OF with the root document. -NOTE: Packages can be wrapped in a “virtual package”. This package can have empty name and version and no attributes, or name can be set to directory or container name which was set as source for generating SBOM document. This element has insignificant information value and can be omitted. +SPDX relationships represent graph/tree structure of relations of elements in the document. The Root element is the SPDX document itself (with SPDXID SPDXRef-Document). Individual packages are in relationship CONTAINS with the root document. In the case of packages which were used to build the container, packages are in relationship BUILD_TOOL_OF with the root document. +NOTE: Packages can be wrapped in a “virtual package”. Details are described in syft specific sbom details section. Relations of two documents needs to be merged together into single graph in a way which keeps the graph structure of the original graph of the main document (into which other document will be merged to). Once packages are merged together, relationships of the second document can be cleared off relations which refer to packages not included in the merged package list. SpdxElementId and relatedSpdxElement point to root document id of the second document should be replaced with root document id of the main document. If there’s “virtual package” in the second document, ids in relationships referring to it should be replaced with “virtual package” of the main document or main document id directly (if there’s no “virtual package”) +## Syft specific sbom details +### Virtual packages +Syft generates "virtual package" representing source used to generate the sbom document. For example when sbom is generated by command `syft scan dir:`, the package +with SPDXID 'SPDXRef-DocumentRoot-Directory--' is generated. Such package has name se to , no versionInfo and no attributes. In relationships this package is in then in relation SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Directory-- and then all packages are in relation CONTAINS with this virtual package, e.i. openshift4----ose-cluster-update-keys CONTAINS Package-A. +Besides this, there's no other purpose of this "virtual package" + + ## Consequences All tooling used in pipeline needs to support SPDX SBOM format From 451622a530715d18f5bceb432775ca88ebaaf37d Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Thu, 14 Nov 2024 11:02:49 +0100 Subject: [PATCH 06/16] - Added glossary - Fixed sbom lifecycle - fixed tables formating - added explanation of formulations conversion to relations - added more details in relationships merging Signed-off-by: Jindrich Luza --- ...9-spdx-support.md => 0044-spdx-support.md} | 87 +++++++++++-------- 1 file changed, 49 insertions(+), 38 deletions(-) rename ADR/{0039-spdx-support.md => 0044-spdx-support.md} (55%) diff --git a/ADR/0039-spdx-support.md b/ADR/0044-spdx-support.md similarity index 55% rename from ADR/0039-spdx-support.md rename to ADR/0044-spdx-support.md index 52439fb0..2ba96b09 100644 --- a/ADR/0039-spdx-support.md +++ b/ADR/0044-spdx-support.md @@ -1,4 +1,4 @@ -# 39. SPDX SBOM support +# 44. SPDX SBOM support Date: 2024-09-24 @@ -6,6 +6,12 @@ Date: 2024-09-24 Proposed +## Glossary +* SBOM - Software Bill of Materials +* SPDX - Software Package Data Exchange +* PURL - Package URL +* Builder images - Images used in FROM instructions in the `Dockerfile` + ## Context SPDX SBOM format enables additional features not available in cyclondedx like multiple purl attributes per component. SPDX is also a widely adopted standard for software bill of materials. @@ -13,12 +19,10 @@ This ADR describes how to enable use of SPDX SBOM format in Konflux. ## Decision - ### SBOM lifecycle in build pipeline -At the start SBOMs are generated by cachi2 and syft. At later phase of the build pipeline, SBOMs from base images are extracted and merged toghether with SBOMs generated for the currently build container. When switching to spdx, newly built container image can be used as base image another container image. Therefore any tooling/task which works with sboms has to be able to work with both formats. -As a result, all tools and tasks should implement the sbomType attribute to specify the expected SBOM format for input and output. This will also allow tools to be tested with SPDX before the entire pipeline transitions to this format. - +At the start SBOMs are generated by [cachi2](#references) and [syft](#references). These two SBOM files are merged together into single SBOM document. At later phase of the build pipeline, builder images of the currently build image are added into SBOM as build dependency of the image. To switch to SPDX format, all tools producing and processing SBOMS in the pipeline has to be able to work with SPDX format. SBOMS of builder images are not processed by the pipeline, therefore builder images SBOMs doesn't have to be in SPDX format. This leads to fact that when tools generating the SBOMs are switched to SPDX format, all tools processing SBOMS can expect SPDX format only. There's no need for any tool to be able to work with mixed inputs of SPDX and CycloneDX formats +As a result, all tools and tasks should implement the sbomType attribute to specify the expected SBOM format for input and output. This will allow tools to be tested with SPDX before the entire pipeline transitions to this format. ### CycloneDX -> SPDX conversion @@ -27,20 +31,20 @@ CycloneDX (1.4) is structured document in json format with following structure ( - Document Root - Metadata - Tools - - List + - `List` - vendor - name - Components - - List + - `List` - name - version - purl - properties - - List + - `List` - name - value - formulations - - List + - `List` SPDX (2.3) is structured document in json format with following structure(not full specification): - Document Root @@ -48,25 +52,25 @@ SPDX (2.3) is structured document in json format with following structure(not fu - SPDXID - creationInfo - Creators - - List + - `List` - packages - - List + - `List` - SPDXID - name - versionInfo - externalRefs - - List + - `List` - referenceCategory - referenceType - referenceLocator - annotations - - List + - `List` - annotationDate - annotationType - annotator - Comment - relationships - - List + - `List` - spdxElementId - relationshipType - relatedSpdxElement @@ -86,9 +90,9 @@ CycloneDX (version 1.4) supports only a single purl attribute per component. SPD | CycloneDX Attribute | SPDX Attribute | |------------------------------|---------------------------------------------------------------| -| component.purl = | package.externalRefs = [{referenceCategory:”PACKAGE-MANAGER”, | +| component.purl = `` | package.externalRefs = [{referenceCategory:”PACKAGE-MANAGER”, | | | referenceType:purl, | -| | referenceLocator: | +| | referenceLocator: `` | | | }] | @@ -98,7 +102,7 @@ CycloneDX components properties describe mapping of string:string properties for | CycloneDX Attribute | SPDX Attribute | |-------------------------------------------|---------------------------------------------| | components.properties = [ | package.annotations = [ | -| {“name”: …, “value”: …} | {..., annotator: ”:jsonencoded” | +| {“name”: …, “value”: …} | {..., annotator: "``:jsonencoded” | | ] | ] | @@ -107,45 +111,52 @@ CycloneDX formulations describe how the container was manufactured. In SPDX, Rel | CycloneDX Attribute | SPDX Attribute | |---------------------------------|------------------------------------------------------------| -| Formulations.components = [{}] | Relationships = [{ | -| | spdxElementId = | -| | relationshipType=BUILD_TOOL_OF | -| | relatedSpdxElement= | -| | }] | - +| Formulations.components = [{}] | Relationships = [ | +| | { | +| | spdxElementId = ``, | +| | relationshipType=DESCRIBES, | +| | relatedSpdxElement=``, | +| | }, | +| | { | +| | spdxElementId = ``, | +| | relationshipType=BUILD_TOOL_OF, | +| | relatedSpdxElement=`` | +| | } | +| | ] | + +*Explanation: Root document `DESCRIBES` `CONTAINER-IMAGE-ID` element which represents the container itself. `BUILDER-IMAGE-ID` represents the builder image which was used to build the container. The relationship type `BUILD_TOOL_OF` is used to express that the builder image was used to build the container image.* #### Metadata.tools -The CycloneDX metadata.tools sub attributes that we are mostly interested in are the vendor and name elements. Information about the creation of the SPDX document can be stored into creationInfo. CreationInfo.creators element is basically a list of strings. There’s a vague specification ([here](https://spdx.github.io/spdx-spec/v2.3/document-creation-information/#68-creator-field)]) about how it should be structured in the standard. Strings should be formatted in the following way: “: ”. For example vendor should be stored as “Vendor: ” +The CycloneDX metadata.tools sub attributes that we are mostly interested in are the vendor and name elements. Information about the creation of the SPDX document can be stored into creationInfo. CreationInfo.creators element is basically a list of strings. There’s a vague specification ([here](https://spdx.github.io/spdx-spec/v2.3/document-creation-information/#68-creator-field)]) about how it should be structured in the standard. Strings should be formatted in the following way: `: `. For example vendor should be stored as `Vendor: ` | CyloneDX Attribute | SPDX Attribute | |------------------------------------------------|---------------------------------------------------| | Metadata.tools = [{“vendor”: “X”, “name”: “Y”] | CreationInfo.creators = [“Vendor: X”, “Tool: Y”] | - #### Merging SPDX ##### Packages Packages of two SPDX documents can be merged together as a concatenation of two lists. In cycloneDX component elements can have only a single purl attribute, therefore component elements representing packages with the same name and version but with different purl have to be stored as multiple elements. SPDX package elements can bear multiple purls. Therefore multiple cycloneDX components can be squashed together into single SPDX package element with purls concatenated into a single list. Following rules are applied to packages merging process: -- Packages with the same purl package name and version and type are squashed into single package element -NOTE: packages cannot be merged together based on SPDXID attribute as there’s no specification in the spdx standard on how SPDXID should be calculated. Individual tools can calculate it differently while still passing condition to make it unique across the whole document. +- Packages with the same purl's package name and version and type are squashed into single package element + +*NOTE: packages cannot be merged together based on SPDXID attribute as there’s no specification in the SPDX standard on how SPDXID should be calculated. Individual tools can calculate it differently while still passing condition to make it unique across the whole document.* ##### Relationships -SPDX relationships represent graph/tree structure of relations of elements in the document. The Root element is the SPDX document itself (with SPDXID SPDXRef-Document). Individual packages are in relationship CONTAINS with the root document. In the case of packages which were used to build the container, packages are in relationship BUILD_TOOL_OF with the root document. -NOTE: Packages can be wrapped in a “virtual package”. Details are described in syft specific sbom details section. -Relations of two documents needs to be merged together into single graph in a way which keeps the graph structure of the original graph of the main document (into which other document will be merged to). Once packages are merged together, relationships of the second document can be cleared off relations which refer to packages not included in the merged package list. SpdxElementId and relatedSpdxElement point to root document id of the second document should be replaced with root document id of the main document. If there’s “virtual package” in the second document, ids in relationships referring to it should be replaced with “virtual package” of the main document or main document id directly (if there’s no “virtual package”) +SPDX relationships represent graph/tree structure of relations of elements in the document. The Root element is the SPDX document itself (with SPDXID SPDXRef-Document). +SPDX Root document typically contains a package representing source used for generating the SBOM. This can be container image, directory, etc. Root document is in relationship DESCRIBES with this source package. Other packages are in specific relationships with the source package. See also [syft specific sbom details](#syft-specific-sbom-details) +Relations of two documents needs to be merged together into single graph in a way which keeps the graph structure of the original graph of the main document (into which other document will be merged to). Once packages are merged together, relationships of the second document must be cleared off relations which refer to packages not included in the merged package list. SpdxElementId and relatedSpdxElement point to root document id of the second document should be replaced with root document id of the main document. Source package element id in the second documents needs to be replaced with source package element id of the main document. ## Syft specific sbom details ### Virtual packages -Syft generates "virtual package" representing source used to generate the sbom document. For example when sbom is generated by command `syft scan dir:`, the package -with SPDXID 'SPDXRef-DocumentRoot-Directory--' is generated. Such package has name se to , no versionInfo and no attributes. In relationships this package is in then in relation SPDXRef-DOCUMENT DESCRIBES SPDXRef-DocumentRoot-Directory-- and then all packages are in relation CONTAINS with this virtual package, e.i. openshift4----ose-cluster-update-keys CONTAINS Package-A. -Besides this, there's no other purpose of this "virtual package" - +Syft generates "source package" representing source used to generate the sbom document. For example when sbom is generated by command `syft scan dir:`, the package +with SPDXID `SPDXRef-DocumentRoot-Directory--` is generated. Such package has name se to ``, no versionInfo and no attributes. In relationships this package is in then in relation `SPDXRef-DOCUMENT` `DESCRIBES` `SPDXRef-DocumentRoot-Directory--` and then all packages are in relation CONTAINS with this virtual package, e.i. `openshift4----ose-cluster-update-keys` `` `Package-A`. ## Consequences All tooling used in pipeline needs to support SPDX SBOM format - ## References -CycloneDX specification https://cyclonedx.org/specification/overview/o -SPDX specification https://spdx.github.io/spdx-spec/v2.3/ -SPDX json schema https://github.com/spdx/spdx-spec/blob/development/v2.3/schemas/spdx-schema.json#L724 +* [CycloneDX specification](https://cyclonedx.org/specification/overview/) +* [SPDX specification](https://spdx.github.io/spdx-spec/v2.3/) +* [SPDX json schema](https://github.com/spdx/spdx-spec/blob/development/v2.3/schemas/spdx-schema.json) +* [cachi2](https://github.com/containerbuildsystem/cachi2/) +* [syft](https://github.com/anchore/syft) From e3bbccf5dbbc6792597d7d3333a89dcfdca95803 Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Mon, 18 Nov 2024 12:48:52 +0100 Subject: [PATCH 07/16] - Fixed cyclonedx version - Added root package into glossary - Added CDX metadata.component conversion section - Added example of merging relationship --- ADR/0044-spdx-support.md | 159 +++++++++++++++++++++++++++++++++------ 1 file changed, 138 insertions(+), 21 deletions(-) diff --git a/ADR/0044-spdx-support.md b/ADR/0044-spdx-support.md index 2ba96b09..1ae16b34 100644 --- a/ADR/0044-spdx-support.md +++ b/ADR/0044-spdx-support.md @@ -11,6 +11,7 @@ Proposed * SPDX - Software Package Data Exchange * PURL - Package URL * Builder images - Images used in FROM instructions in the `Dockerfile` +* Root package - The package representing the source of the SBOM itself ## Context @@ -22,18 +23,19 @@ This ADR describes how to enable use of SPDX SBOM format in Konflux. ### SBOM lifecycle in build pipeline At the start SBOMs are generated by [cachi2](#references) and [syft](#references). These two SBOM files are merged together into single SBOM document. At later phase of the build pipeline, builder images of the currently build image are added into SBOM as build dependency of the image. To switch to SPDX format, all tools producing and processing SBOMS in the pipeline has to be able to work with SPDX format. SBOMS of builder images are not processed by the pipeline, therefore builder images SBOMs doesn't have to be in SPDX format. This leads to fact that when tools generating the SBOMs are switched to SPDX format, all tools processing SBOMS can expect SPDX format only. There's no need for any tool to be able to work with mixed inputs of SPDX and CycloneDX formats -As a result, all tools and tasks should implement the sbomType attribute to specify the expected SBOM format for input and output. This will allow tools to be tested with SPDX before the entire pipeline transitions to this format. +As a result, tekton tasks should implement the sbomType attribute to specify the expected SBOM format for input and output. This will allow tools to be tested with SPDX before the entire pipeline transitions to this format. ### CycloneDX -> SPDX conversion -CycloneDX (1.4) is structured document in json format with following structure (not full specification) +CycloneDX (1.5) is structured document in json format with following structure (not full specification) -- Document Root +- Document - Metadata - Tools - `List` - vendor - name + - `` (attributes same as bellow) - Components - `List` - name @@ -44,10 +46,10 @@ CycloneDX (1.4) is structured document in json format with following structure ( - name - value - formulations - - `List` + - `List` SPDX (2.3) is structured document in json format with following structure(not full specification): -- Document Root +- Document - name - SPDXID - creationInfo @@ -69,11 +71,11 @@ SPDX (2.3) is structured document in json format with following structure(not fu - annotationType - annotator - Comment - - relationships - - `List` - - spdxElementId - - relationshipType - - relatedSpdxElement + - relationships + - `List` + - spdxElementId + - relationshipType + - relatedSpdxElement #### 1:1 conversions Following CycloneDX to SPDX attributes are converted as 1:1 as they represent the same thing. @@ -86,7 +88,7 @@ Following CycloneDX to SPDX attributes are converted as 1:1 as they represent th #### Component.purl -CycloneDX (version 1.4) supports only a single purl attribute per component. SPDX doesn’t have a direct attribute, but instead every package includes an externalRefs array which describes all external references for the package. There are defined reference categories and types. For PURL, category PACKAGE-MANAGER and type purl is used. The purl itself will be stored as referenceLocator +CycloneDX (version 1.5) supports only a single purl attribute per component. SPDX doesn’t have a direct attribute, but instead every package includes an externalRefs array which describes all external references for the package. There are defined reference categories and types. For PURL, category PACKAGE-MANAGER and type purl is used. The purl itself will be stored as referenceLocator | CycloneDX Attribute | SPDX Attribute | |------------------------------|---------------------------------------------------------------| @@ -113,41 +115,156 @@ CycloneDX formulations describe how the container was manufactured. In SPDX, Rel |---------------------------------|------------------------------------------------------------| | Formulations.components = [{}] | Relationships = [ | | | { | -| | spdxElementId = ``, | +| | spdxElementId = ``, | | | relationshipType=DESCRIBES, | -| | relatedSpdxElement=``, | +| | relatedSpdxElement=``, | | | }, | | | { | | | spdxElementId = ``, | | | relationshipType=BUILD_TOOL_OF, | -| | relatedSpdxElement=`` | +| | relatedSpdxElement=`` | | | } | | | ] | -*Explanation: Root document `DESCRIBES` `CONTAINER-IMAGE-ID` element which represents the container itself. `BUILDER-IMAGE-ID` represents the builder image which was used to build the container. The relationship type `BUILD_TOOL_OF` is used to express that the builder image was used to build the container image.* +*Explanation: Root document `DESCRIBES` `ROOT-PACKAGE` element which represents the container itself. `BUILDER-IMAGE-ID` represents the builder image which was used to build the container. The relationship type `BUILD_TOOL_OF` is used to express that the builder image was used to build the container image.* #### Metadata.tools -The CycloneDX metadata.tools sub attributes that we are mostly interested in are the vendor and name elements. Information about the creation of the SPDX document can be stored into creationInfo. CreationInfo.creators element is basically a list of strings. There’s a vague specification ([here](https://spdx.github.io/spdx-spec/v2.3/document-creation-information/#68-creator-field)]) about how it should be structured in the standard. Strings should be formatted in the following way: `: `. For example vendor should be stored as `Vendor: ` +The CycloneDX metadata.tools sub attributes that we are mostly interested in are the vendor and name elements. Information about the creation of the SPDX document can be stored into creationInfo. CreationInfo.creators element is basically a list of strings. There’s a vague specification ([here](https://spdx.github.io/spdx-spec/v2.3/document-creation-information/#68-creator-field)]) about how it should be structured in the standard. Strings should be formatted in the following way: `: `. For example vendor should be stored as `Vendor: `. Redommendation is to use only Tool as vendor can be misinterpreted as vendor of the SBOM not the tool which created it. | CyloneDX Attribute | SPDX Attribute | |------------------------------------------------|---------------------------------------------------| -| Metadata.tools = [{“vendor”: “X”, “name”: “Y”] | CreationInfo.creators = [“Vendor: X”, “Tool: Y”] | +| Metadata.tools = [{“vendor”: “X”, “name”: “Y”] | CreationInfo.creators = [“Tool: Y”] | + +#### Metadata.component +Metadata component describes component which is the component which whole SBOM is related to. For example If SBOM describes internal components and dependencies of a container image, this component should represent the container image itself. In SPDX, a package which is equivalent to this component is root package (see [root-packages](#root-packages) to have idea how syft handles this package). This package is in relationship `SPDXRef-ROOT` `DESCRIBES` `SPDXRef-RootPackage`. If there's no cycloneDX metadata.component attribute, SPDX SBOM already includes this "root package" and shouldn't be changed. If there is metadata.component attribute, existing root package should be replaced with the new one representing the SBOM document. All SPDX IDs in relationships refering to the old root package should be replaced with SPDX ID of the new root package. Example: +We run syft/cachi2 on source directory of a project which should be build into container. Generated SBOM contains +```json +{ + "SPDXID": "SPDXRef-DOCUMENT", + ... + "packages": [ + { + "name": ".", + "SPDXID": "SPDXRef-DocumentRoot-Directory-.", + "supplier": "NOASSERTION", + "downloadLocation": "NOASSERTION", + "filesAnalyzed": false, + "licenseConcluded": "NOASSERTION", + "licenseDeclared": "NOASSERTION", + "primaryPackagePurpose": "FILE" + }, + { + { + "name": "attrs", + "SPDXID": "SPDXRef-Package-python-attrs-eef51168ca2a575f", + "versionInfo": "24.2.0", + ... + } + ... + ], + "relationships": [ + { + "spdxElementId": "SPDXRef-DOCUMENT", + "relatedSpdxElement": "SPDXRef-DocumentRoot-Directory-.", + "relationshipType": "DESCRIBES" + }, + { + "spdxElementId": "SPDXRef-DocumentRoot-Directory-.", + "relatedSpdxElement": "SPDXRef-Package-python-attrs-eef51168ca2a575f", + "relationshipType": "CONTAINS" + } + ... + ] +} +``` + +And we want that to express that SBOM is actually generated for a container image not for source directory. +So we remove `SPDXRef-DocumentRoot-Directory-.` package and add new virtual package representing the container image. And replace SPDX ID in relationships with ID of the new package. New SBOM should look like this: + +```json +{ + "SPDXID": "SPDXRef-DOCUMENT", + ... + "packages": [ + { + "name": "container-registry.com/my-org/my-image:latest", + "SPDXID": "SPDXRef-DocumentRoot-Image-container-registry.com/my-org/my-image:latest", + "versionInfo": "sha256:9ac75c1a392429b4a087971cdf9190ec42a854a169b6835bc9e25eecaf851258", + ... + "checksums": [ + { + "algorithm": "SHA256", + "checksumValue": "9ac75c1a392429b4a087971cdf9190ec42a854a169b6835bc9e25eecaf851258" + } + ], + ... + "externalRefs": [ + { + "referenceCategory": "PACKAGE-MANAGER", + "referenceType": "purl", + "referenceLocator": "pkg:oci/container-registry.com/my-org/my-image@sha256:9ac75c1a392429b4a087971cdf9190ec42a854a169b6835bc9e25eecaf851258" + } + ], + "primaryPackagePurpose": "CONTAINER" + }, + { + { + "name": "attrs", + "SPDXID": "SPDXRef-Package-python-attrs-eef51168ca2a575f", + "versionInfo": "24.2.0", + ... + } + ... + ], + "relationships": [ + { + "spdxElementId": "SPDXRef-DOCUMENT", + "relatedSpdxElement": "SPDXRef-DocumentRoot-Image-container-registry.com/my-org/my-image:latest", + "relationshipType": "DESCRIBES" + }, + { + "spdxElementId": "SPDXRef-DocumentRoot-Image-container-registry.com/my-org/my-image:latest", + "relatedSpdxElement": "SPDXRef-Package-python-attrs-eef51168ca2a575f", + "relationshipType": "CONTAINS" + } + ... + ] +} +``` + #### Merging SPDX ##### Packages -Packages of two SPDX documents can be merged together as a concatenation of two lists. In cycloneDX component elements can have only a single purl attribute, therefore component elements representing packages with the same name and version but with different purl have to be stored as multiple elements. SPDX package elements can bear multiple purls. Therefore multiple cycloneDX components can be squashed together into single SPDX package element with purls concatenated into a single list. Following rules are applied to packages merging process: +Packages of two SPDX documents can be merged together as a concatenation of two lists. In cycloneDX component elements can have only a single purl attribute, therefore component elements representing packages with the same name and version but with different purl have to be stored as multiple elements. SPDX package elements can bear multiple purls. Therefore multiple cycloneDX components can be squashed together into single SPDX package element with purls concatenated into a single list. Following rules are applied to generic packages merging process: - Packages with the same purl's package name and version and type are squashed into single package element *NOTE: packages cannot be merged together based on SPDXID attribute as there’s no specification in the SPDX standard on how SPDXID should be calculated. Individual tools can calculate it differently while still passing condition to make it unique across the whole document.* ##### Relationships SPDX relationships represent graph/tree structure of relations of elements in the document. The Root element is the SPDX document itself (with SPDXID SPDXRef-Document). -SPDX Root document typically contains a package representing source used for generating the SBOM. This can be container image, directory, etc. Root document is in relationship DESCRIBES with this source package. Other packages are in specific relationships with the source package. See also [syft specific sbom details](#syft-specific-sbom-details) +SPDX Root document typically contains a package representing source used for generating the SBOM. This can be container image, directory, etc. The document is in relationship DESCRIBES with this source package - called root package in this document. Other packages are in specific relationships with the root package. See also [syft specific sbom details](#syft-specific-sbom-details) + +Relations of two documents needs to be merged together into single graph in a way which keeps the graph structure of the original graph of the main document (into which other document will be merged to). Once packages are merged together, relationships of the second document must be cleared off relations which refer to packages not included in the merged package list. SpdxElementId and relatedSpdxElement point to root document id of the second document should be replaced with root document id of the main document. Root package element id in the second documents needs to be replaced with root package element id of the main document. +Example: + +-------------------------------------------------------------------------------- +| DOC 1 | DOC 2 | Merged document | +----------------------------|--------------------------|------------------------| +| Doc1 | Doc2 | Doc1 | +| Packages: | Packages: | Packages: | +| root1 | root2 | root1 | +| p1 | p2 | p1 | +| p2 | p3 | p2 | +| Relationships: | Relationships: | p3 | +| Doc1 describes root1 | Doc2 describes root2 | Relationships: | +| root1 contains p1 | root2 contains p2 | Doc1 desctibes root1 | +| root1 contains p2 | root2 contains p3 | root1 contains p1 | +| | | root1 contains p2 | +| | | root1 contains p3 | -Relations of two documents needs to be merged together into single graph in a way which keeps the graph structure of the original graph of the main document (into which other document will be merged to). Once packages are merged together, relationships of the second document must be cleared off relations which refer to packages not included in the merged package list. SpdxElementId and relatedSpdxElement point to root document id of the second document should be replaced with root document id of the main document. Source package element id in the second documents needs to be replaced with source package element id of the main document. ## Syft specific sbom details -### Virtual packages +### Root packages Syft generates "source package" representing source used to generate the sbom document. For example when sbom is generated by command `syft scan dir:`, the package with SPDXID `SPDXRef-DocumentRoot-Directory--` is generated. Such package has name se to ``, no versionInfo and no attributes. In relationships this package is in then in relation `SPDXRef-DOCUMENT` `DESCRIBES` `SPDXRef-DocumentRoot-Directory--` and then all packages are in relation CONTAINS with this virtual package, e.i. `openshift4----ose-cluster-update-keys` `` `Package-A`. From d001dc0569c30e5f0f4e16b3eb7037317ea13944 Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Wed, 20 Nov 2024 07:56:38 +0100 Subject: [PATCH 08/16] Update ADR/0044-spdx-support.md Co-authored-by: Adam Cmiel --- ADR/0044-spdx-support.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/ADR/0044-spdx-support.md b/ADR/0044-spdx-support.md index 1ae16b34..29582593 100644 --- a/ADR/0044-spdx-support.md +++ b/ADR/0044-spdx-support.md @@ -187,13 +187,13 @@ So we remove `SPDXRef-DocumentRoot-Directory-.` package and add new virtual pack ... "packages": [ { - "name": "container-registry.com/my-org/my-image:latest", - "SPDXID": "SPDXRef-DocumentRoot-Image-container-registry.com/my-org/my-image:latest", - "versionInfo": "sha256:9ac75c1a392429b4a087971cdf9190ec42a854a169b6835bc9e25eecaf851258", + "name": "my-image", + "SPDXID": "SPDXRef-image", + "versionInfo": "latest", ... "checksums": [ { - "algorithm": "SHA256", + "algorithm": "SHA-256", "checksumValue": "9ac75c1a392429b4a087971cdf9190ec42a854a169b6835bc9e25eecaf851258" } ], @@ -202,7 +202,7 @@ So we remove `SPDXRef-DocumentRoot-Directory-.` package and add new virtual pack { "referenceCategory": "PACKAGE-MANAGER", "referenceType": "purl", - "referenceLocator": "pkg:oci/container-registry.com/my-org/my-image@sha256:9ac75c1a392429b4a087971cdf9190ec42a854a169b6835bc9e25eecaf851258" + "referenceLocator": "pkg:oci/my-image@sha256:9ac75c1a392429b4a087971cdf9190ec42a854a169b6835bc9e25eecaf851258?repository_url=container-registry.com/my-org/my-image" } ], "primaryPackagePurpose": "CONTAINER" From 9203873408e6fe740e6c4400a5864f88d217a526 Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Wed, 20 Nov 2024 07:57:18 +0100 Subject: [PATCH 09/16] Update ADR/0044-spdx-support.md Co-authored-by: Adam Cmiel --- ADR/0044-spdx-support.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ADR/0044-spdx-support.md b/ADR/0044-spdx-support.md index 29582593..3545ca90 100644 --- a/ADR/0044-spdx-support.md +++ b/ADR/0044-spdx-support.md @@ -138,10 +138,10 @@ The CycloneDX metadata.tools sub attributes that we are mostly interested in are #### Metadata.component Metadata component describes component which is the component which whole SBOM is related to. For example If SBOM describes internal components and dependencies of a container image, this component should represent the container image itself. In SPDX, a package which is equivalent to this component is root package (see [root-packages](#root-packages) to have idea how syft handles this package). This package is in relationship `SPDXRef-ROOT` `DESCRIBES` `SPDXRef-RootPackage`. If there's no cycloneDX metadata.component attribute, SPDX SBOM already includes this "root package" and shouldn't be changed. If there is metadata.component attribute, existing root package should be replaced with the new one representing the SBOM document. All SPDX IDs in relationships refering to the old root package should be replaced with SPDX ID of the new root package. Example: We run syft/cachi2 on source directory of a project which should be build into container. Generated SBOM contains -```json +```jsonc { "SPDXID": "SPDXRef-DOCUMENT", - ... + // ... "packages": [ { "name": ".", From 221805ac3a4d60836775259d3c00fc00bdabcc4b Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Wed, 20 Nov 2024 09:04:52 +0100 Subject: [PATCH 10/16] - Fixed tables - Remove consufing description from metadata.component --- ADR/0044-spdx-support.md | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/ADR/0044-spdx-support.md b/ADR/0044-spdx-support.md index 3545ca90..0a7d9102 100644 --- a/ADR/0044-spdx-support.md +++ b/ADR/0044-spdx-support.md @@ -89,42 +89,45 @@ Following CycloneDX to SPDX attributes are converted as 1:1 as they represent th #### Component.purl CycloneDX (version 1.5) supports only a single purl attribute per component. SPDX doesn’t have a direct attribute, but instead every package includes an externalRefs array which describes all external references for the package. There are defined reference categories and types. For PURL, category PACKAGE-MANAGER and type purl is used. The purl itself will be stored as referenceLocator - +``` | CycloneDX Attribute | SPDX Attribute | |------------------------------|---------------------------------------------------------------| | component.purl = `` | package.externalRefs = [{referenceCategory:”PACKAGE-MANAGER”, | | | referenceType:purl, | | | referenceLocator: `` | | | }] | - +``` #### Component.properties CycloneDX components properties describe mapping of string:string properties for given component. SPDX component doesn’t have anything similar to cyclonedx properties. SPDX Package annotations are the only attribute where custom data can be stored and the only “customizable” field where there is comment which is a simple string. Due to that fact, cycloneDX property in format of {“name”: , “value”: } is encoded into json string. There can be also annotations produced by other tools. Therefore to be able to tell annotation comment is json encoded, annotator should ends with string “:jsonencoded” +``` | CycloneDX Attribute | SPDX Attribute | |-------------------------------------------|---------------------------------------------| | components.properties = [ | package.annotations = [ | -| {“name”: …, “value”: …} | {..., annotator: "``:jsonencoded” | +| {“name”: …, “value”: …} | {..., annotator: "``:jsonencoded” | | ] | ] | - +``` #### Formulations CycloneDX formulations describe how the container was manufactured. In SPDX, Relationship elements can be used for the same purpose. All elements in SPDX have SPDXID attribute which is an element identifier unique in the whole SBOM document. Relationship element describes relation between two elements using their SPDXID and relationship type. Relationship type BUILD_TOOL_OF can be used to express the relationship of packages which were used to build the container. +``` | CycloneDX Attribute | SPDX Attribute | |---------------------------------|------------------------------------------------------------| | Formulations.components = [{}] | Relationships = [ | | | { | -| | spdxElementId = ``, | +| | spdxElementId = ``, | | | relationshipType=DESCRIBES, | -| | relatedSpdxElement=``, | +| | relatedSpdxElement=``, | | | }, | | | { | | | spdxElementId = ``, | | | relationshipType=BUILD_TOOL_OF, | -| | relatedSpdxElement=`` | +| | relatedSpdxElement=`` | | | } | | | ] | +``` *Explanation: Root document `DESCRIBES` `ROOT-PACKAGE` element which represents the container itself. `BUILDER-IMAGE-ID` represents the builder image which was used to build the container. The relationship type `BUILD_TOOL_OF` is used to express that the builder image was used to build the container image.* @@ -136,7 +139,9 @@ The CycloneDX metadata.tools sub attributes that we are mostly interested in are | Metadata.tools = [{“vendor”: “X”, “name”: “Y”] | CreationInfo.creators = [“Tool: Y”] | #### Metadata.component -Metadata component describes component which is the component which whole SBOM is related to. For example If SBOM describes internal components and dependencies of a container image, this component should represent the container image itself. In SPDX, a package which is equivalent to this component is root package (see [root-packages](#root-packages) to have idea how syft handles this package). This package is in relationship `SPDXRef-ROOT` `DESCRIBES` `SPDXRef-RootPackage`. If there's no cycloneDX metadata.component attribute, SPDX SBOM already includes this "root package" and shouldn't be changed. If there is metadata.component attribute, existing root package should be replaced with the new one representing the SBOM document. All SPDX IDs in relationships refering to the old root package should be replaced with SPDX ID of the new root package. Example: +Metadata component describes component which is the component which whole SBOM is related to. For example If SBOM describes internal components and dependencies of a container image, this component should represent the container image itself. In SPDX, a package which is equivalent to this component is root package (see [root-packages](#root-packages) to have idea how syft handles this package). This package is in relationship `SPDXRef-ROOT` `DESCRIBES` `SPDXRef-RootPackage`. + +Example: We run syft/cachi2 on source directory of a project which should be build into container. Generated SBOM contains ```jsonc { @@ -181,7 +186,7 @@ We run syft/cachi2 on source directory of a project which should be build into c And we want that to express that SBOM is actually generated for a container image not for source directory. So we remove `SPDXRef-DocumentRoot-Directory-.` package and add new virtual package representing the container image. And replace SPDX ID in relationships with ID of the new package. New SBOM should look like this: -```json +```jsonc { "SPDXID": "SPDXRef-DOCUMENT", ... @@ -247,6 +252,7 @@ SPDX Root document typically contains a package representing source used for gen Relations of two documents needs to be merged together into single graph in a way which keeps the graph structure of the original graph of the main document (into which other document will be merged to). Once packages are merged together, relationships of the second document must be cleared off relations which refer to packages not included in the merged package list. SpdxElementId and relatedSpdxElement point to root document id of the second document should be replaced with root document id of the main document. Root package element id in the second documents needs to be replaced with root package element id of the main document. Example: +``` -------------------------------------------------------------------------------- | DOC 1 | DOC 2 | Merged document | ----------------------------|--------------------------|------------------------| @@ -261,7 +267,7 @@ Example: | root1 contains p2 | root2 contains p3 | root1 contains p1 | | | | root1 contains p2 | | | | root1 contains p3 | - +``` ## Syft specific sbom details ### Root packages From 682e30491fa73f08ec71bb286e9dfbe03ff5bd5f Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Thu, 21 Nov 2024 17:21:44 +0100 Subject: [PATCH 11/16] - Included document.created and package.downloadLocation fields in the description - Changed annotator format to reflex required format Signed-off-by: Jindrich Luza --- ADR/0044-spdx-support.md | 31 ++++++++++++++++++++++++------- 1 file changed, 24 insertions(+), 7 deletions(-) diff --git a/ADR/0044-spdx-support.md b/ADR/0044-spdx-support.md index 0a7d9102..eb7c6bbb 100644 --- a/ADR/0044-spdx-support.md +++ b/ADR/0044-spdx-support.md @@ -51,14 +51,17 @@ CycloneDX (1.5) is structured document in json format with following structure ( SPDX (2.3) is structured document in json format with following structure(not full specification): - Document - name + - documentNamespace - SPDXID - creationInfo - - Creators + - creators - `List` + - created - packages - `List` - SPDXID - name + - downloadLocation - versionInfo - externalRefs - `List` @@ -99,14 +102,14 @@ CycloneDX (version 1.5) supports only a single purl attribute per component. SPD ``` #### Component.properties -CycloneDX components properties describe mapping of string:string properties for given component. SPDX component doesn’t have anything similar to cyclonedx properties. SPDX Package annotations are the only attribute where custom data can be stored and the only “customizable” field where there is comment which is a simple string. Due to that fact, cycloneDX property in format of {“name”: , “value”: } is encoded into json string. There can be also annotations produced by other tools. Therefore to be able to tell annotation comment is json encoded, annotator should ends with string “:jsonencoded” +CycloneDX components properties describe mapping of string:string properties for given component. SPDX component doesn’t have anything similar to cyclonedx properties. SPDX Package annotations are the only attribute where custom data can be stored and the only “customizable” field where there is comment which is a simple string. Due to that fact, cycloneDX property in format of {“name”: , “value”: } is encoded into json string. There can be also annotations produced by other tools. Therefore to be able to tell annotation comment is json encoded, annotator should ends with string “:jsonencoded”. To indicate annotator was a tool, prefix “Tool:" has to be included in the field. ``` -| CycloneDX Attribute | SPDX Attribute | -|-------------------------------------------|---------------------------------------------| -| components.properties = [ | package.annotations = [ | -| {“name”: …, “value”: …} | {..., annotator: "``:jsonencoded” | -| ] | ] | +| CycloneDX Attribute | SPDX Attribute | +|-------------------------------------------|------------------------------------------------| +| components.properties = [ | package.annotations = [ | +| {“name”: …, “value”: …} | {..., annotator: "`Tool:`:jsonencoded” | +| ] | ] | ``` #### Formulations @@ -237,6 +240,20 @@ So we remove `SPDXRef-DocumentRoot-Directory-.` package and add new virtual pack } ``` +#### SPDX specific attributes + +There are SPDX attributes which are required to be present in the document, however there's +no cycloneDX equivalent for them. These attributes are: + +##### Document.documentNamespace + +documentNamespace is URI which provides way how locate the document or reference it other documents. When creating SPDX document locally via syft or cachi2, this attribute has no meaning +as it's not clear yet how the document will be published. Therefore it's set to `NOASSERTION`. +However when the document is copied to a container. It would make sense to provide a link to the document container + +##### Package.downloadLocation +downloadLocation is URI which provides way how to download the package. When creating SPDX document locally via syft, this information is not available. Also syft doesn't provide anything to this field. Therefore it's set to `NOASSERTION`. + #### Merging SPDX ##### Packages From 4de5505f8c5b14ea448d152d43278efdade76aea Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Mon, 25 Nov 2024 08:11:40 +0100 Subject: [PATCH 12/16] Update ADR/0044-spdx-support.md Co-authored-by: Adam Cmiel --- ADR/0044-spdx-support.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ADR/0044-spdx-support.md b/ADR/0044-spdx-support.md index eb7c6bbb..4a3d2c55 100644 --- a/ADR/0044-spdx-support.md +++ b/ADR/0044-spdx-support.md @@ -252,7 +252,7 @@ as it's not clear yet how the document will be published. Therefore it's set to However when the document is copied to a container. It would make sense to provide a link to the document container ##### Package.downloadLocation -downloadLocation is URI which provides way how to download the package. When creating SPDX document locally via syft, this information is not available. Also syft doesn't provide anything to this field. Therefore it's set to `NOASSERTION`. +downloadLocation is URI which provides way how to download the package. This is not always available, and it's not clear if it's useful. Therefore it's set to `NOASSERTION`. #### Merging SPDX From 26879fc78164f4eb37cda112ef0c12cbb3517698 Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Mon, 25 Nov 2024 14:32:03 +0100 Subject: [PATCH 13/16] Update ADR/0044-spdx-support.md Co-authored-by: Adam Cmiel --- ADR/0044-spdx-support.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ADR/0044-spdx-support.md b/ADR/0044-spdx-support.md index 4a3d2c55..7bed2cc2 100644 --- a/ADR/0044-spdx-support.md +++ b/ADR/0044-spdx-support.md @@ -108,7 +108,7 @@ CycloneDX components properties describe mapping of string:string properties for | CycloneDX Attribute | SPDX Attribute | |-------------------------------------------|------------------------------------------------| | components.properties = [ | package.annotations = [ | -| {“name”: …, “value”: …} | {..., annotator: "`Tool:`:jsonencoded” | +| {“name”: …, “value”: …} | {..., annotator: "`Tool: `:jsonencoded” | | ] | ] | ``` From 4d053a544e4262b61fe4f01eabba0a22ff1f4b27 Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Mon, 25 Nov 2024 14:35:43 +0100 Subject: [PATCH 14/16] Removed relationship from the table which could cause confusion --- ADR/0044-spdx-support.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/ADR/0044-spdx-support.md b/ADR/0044-spdx-support.md index 7bed2cc2..4f20edd6 100644 --- a/ADR/0044-spdx-support.md +++ b/ADR/0044-spdx-support.md @@ -120,11 +120,6 @@ CycloneDX formulations describe how the container was manufactured. In SPDX, Rel |---------------------------------|------------------------------------------------------------| | Formulations.components = [{}] | Relationships = [ | | | { | -| | spdxElementId = ``, | -| | relationshipType=DESCRIBES, | -| | relatedSpdxElement=``, | -| | }, | -| | { | | | spdxElementId = ``, | | | relationshipType=BUILD_TOOL_OF, | | | relatedSpdxElement=`` | From f84f1cf2e313d442a1dd8cf07d1df681abdbbf1a Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Mon, 25 Nov 2024 15:17:24 +0100 Subject: [PATCH 15/16] Fixed NOASSERTION namespace Signed-off-by: Jindrich Luza --- ADR/0044-spdx-support.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/ADR/0044-spdx-support.md b/ADR/0044-spdx-support.md index 4f20edd6..e0cecc40 100644 --- a/ADR/0044-spdx-support.md +++ b/ADR/0044-spdx-support.md @@ -243,8 +243,7 @@ no cycloneDX equivalent for them. These attributes are: ##### Document.documentNamespace documentNamespace is URI which provides way how locate the document or reference it other documents. When creating SPDX document locally via syft or cachi2, this attribute has no meaning -as it's not clear yet how the document will be published. Therefore it's set to `NOASSERTION`. -However when the document is copied to a container. It would make sense to provide a link to the document container +as it's not clear yet how the document will be published. But as stated in the SPDX specification, it should be unique. At later stages when it's clear where the sbom document will be published it would make sense to change this to a link to the container containing the sbom. URI itself doesn't need to be neccesarilly accessible ##### Package.downloadLocation downloadLocation is URI which provides way how to download the package. This is not always available, and it's not clear if it's useful. Therefore it's set to `NOASSERTION`. From 571c093b701c0467d60c75cf0f2f7355a17c49df Mon Sep 17 00:00:00 2001 From: Jindrich Luza Date: Tue, 26 Nov 2024 08:51:56 +0100 Subject: [PATCH 16/16] Changed status Signed-off-by: Jindrich Luza --- ADR/0044-spdx-support.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ADR/0044-spdx-support.md b/ADR/0044-spdx-support.md index e0cecc40..088505bb 100644 --- a/ADR/0044-spdx-support.md +++ b/ADR/0044-spdx-support.md @@ -4,7 +4,7 @@ Date: 2024-09-24 ## Status -Proposed +Accepted ## Glossary * SBOM - Software Bill of Materials