diff --git a/pipelines/docker-build-multi-platform-oci-ta/README.md b/pipelines/docker-build-multi-platform-oci-ta/README.md index ba574b2adc..97757eb1a9 100644 --- a/pipelines/docker-build-multi-platform-oci-ta/README.md +++ b/pipelines/docker-build-multi-platform-oci-ta/README.md @@ -214,6 +214,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito |name|description|used in params (taskname:taskrefversion:taskparam) |---|---|---| |IMAGES_PROCESSED| Images processed in the task.| | +|REPORTS| Mapping of image digests to report digests| | |SCAN_OUTPUT| Clair scan result.| | |TEST_OUTPUT| Tekton task test output.| | ### clamav-scan:0.1 task results diff --git a/pipelines/docker-build-oci-ta/README.md b/pipelines/docker-build-oci-ta/README.md index a5b9464514..c32e46844f 100644 --- a/pipelines/docker-build-oci-ta/README.md +++ b/pipelines/docker-build-oci-ta/README.md @@ -211,6 +211,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito |name|description|used in params (taskname:taskrefversion:taskparam) |---|---|---| |IMAGES_PROCESSED| Images processed in the task.| | +|REPORTS| Mapping of image digests to report digests| | |SCAN_OUTPUT| Clair scan result.| | |TEST_OUTPUT| Tekton task test output.| | ### clamav-scan:0.1 task results diff --git a/pipelines/docker-build/README.md b/pipelines/docker-build/README.md index 59369a9dac..8a2b87e10f 100644 --- a/pipelines/docker-build/README.md +++ b/pipelines/docker-build/README.md @@ -209,6 +209,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito |name|description|used in params (taskname:taskrefversion:taskparam) |---|---|---| |IMAGES_PROCESSED| Images processed in the task.| | +|REPORTS| Mapping of image digests to report digests| | |SCAN_OUTPUT| Clair scan result.| | |TEST_OUTPUT| Tekton task test output.| | ### clamav-scan:0.1 task results diff --git a/pipelines/java-builder/README.md b/pipelines/java-builder/README.md index 2f74a10f82..f0768f1bb7 100644 --- a/pipelines/java-builder/README.md +++ b/pipelines/java-builder/README.md @@ -181,6 +181,7 @@ |name|description|used in params (taskname:taskrefversion:taskparam) |---|---|---| |IMAGES_PROCESSED| Images processed in the task.| | +|REPORTS| Mapping of image digests to report digests| | |SCAN_OUTPUT| Clair scan result.| | |TEST_OUTPUT| Tekton task test output.| | ### clamav-scan:0.1 task results diff --git a/pipelines/nodejs-builder/README.md b/pipelines/nodejs-builder/README.md index 61839f6b81..95bac5aede 100644 --- a/pipelines/nodejs-builder/README.md +++ b/pipelines/nodejs-builder/README.md @@ -181,6 +181,7 @@ |name|description|used in params (taskname:taskrefversion:taskparam) |---|---|---| |IMAGES_PROCESSED| Images processed in the task.| | +|REPORTS| Mapping of image digests to report digests| | |SCAN_OUTPUT| Clair scan result.| | |TEST_OUTPUT| Tekton task test output.| | ### clamav-scan:0.1 task results diff --git a/pipelines/tekton-bundle-builder/README.md b/pipelines/tekton-bundle-builder/README.md index bdab87c31f..f1d7da57ff 100644 --- a/pipelines/tekton-bundle-builder/README.md +++ b/pipelines/tekton-bundle-builder/README.md @@ -153,6 +153,7 @@ |name|description|used in params (taskname:taskrefversion:taskparam) |---|---|---| |IMAGES_PROCESSED| Images processed in the task.| | +|REPORTS| Mapping of image digests to report digests| | |SCAN_OUTPUT| Clair scan result.| | |TEST_OUTPUT| Tekton task test output.| | ### clamav-scan:0.1 task results diff --git a/task/clair-scan/0.2/README.md b/task/clair-scan/0.2/README.md index 2052abf59f..c3aedabb42 100644 --- a/task/clair-scan/0.2/README.md +++ b/task/clair-scan/0.2/README.md @@ -17,10 +17,11 @@ analyzing the components of a container image and comparing them against Clair's ## Results: -| name | description | -|-------------------|--------------------------| -| TEST_OUTPUT | Tekton task test output. | -| SCAN_OUTPUT | Clair scan result. | +| name | description | +|-------------------|------------------------------------------| +| TEST_OUTPUT | Tekton task test output. | +| SCAN_OUTPUT | Clair scan result. | +| REPORTS |Mapping of image digests to report digests| ## Clair-action repository: https://github.com/quay/clair-action diff --git a/task/clair-scan/0.2/clair-scan.yaml b/task/clair-scan/0.2/clair-scan.yaml index c68c565b6b..f4999eb7f8 100644 --- a/task/clair-scan/0.2/clair-scan.yaml +++ b/task/clair-scan/0.2/clair-scan.yaml @@ -34,6 +34,8 @@ spec: description: Clair scan result. - name: IMAGES_PROCESSED description: Images processed in the task. + - name: REPORTS + description: Mapping of image digests to report digests stepTemplate: volumeMounts: - name: trusted-ca @@ -92,14 +94,28 @@ spec: value: $(params.image-url) - name: IMAGE_DIGEST value: $(params.image-digest) + workingDir: /tekton/home script: | #!/usr/bin/env bash + set -o errexit + set -o nounset + set -o pipefail + imagewithouttag=$(echo -n $IMAGE_URL | sed "s/\(.*\):.*/\1/") images_processed_template='{"image": {"pullspec": "'"$IMAGE_URL"'", "digests": [%s]}}' digests_processed=() - for sha_file in /tekton/home/image-manifest-*.sha; do + # the quay report format used by the Conftest rules in the + # conftest-vulnerabilities step doesn't contain the "issued" date which + # we require in the policy rules, so we resort to running clair-action + # twice to produce both quay and clair formatted output + clair_report() { + { clair-action report --image-ref="$1" --db-path=/tmp/matcher.db --format=quay | tee "clair-result-$2.json"; } && \ + { clair-action report --image-ref="$1" --db-path=/tmp/matcher.db --format=clair > "clair-report-$2.json"; } + } + + for sha_file in image-manifest-*.sha; do if [ -e "$sha_file" ]; then arch_sha=$(cat "$sha_file") arch=$(basename "$sha_file" | sed 's/image-manifest-//;s/.sha//') @@ -107,7 +123,7 @@ spec: echo "Running clair-action on $arch image manifest." # run the scan for each image manifest in the image index - clair-action report --image-ref=$arch_specific_digest --db-path=/tmp/matcher.db --format=quay | tee /tekton/home/clair-result-$arch.json || true + clair_report "${arch_specific_digest}" "${arch}" || true digests_processed+=("\"$arch_sha\"") fi @@ -117,7 +133,48 @@ spec: # add the image_index to the processed digests list and store the result in a file images_processed=$(echo "${images_processed_template/\[%s]/[$digests_processed_string]}") - echo "$images_processed" > /tekton/home/images-processed.json + echo "$images_processed" > images-processed.json + - name: oci-attach-report + image: quay.io/konflux-ci/oras:latest@sha256:56589c1c9132aeaccffad2fc2fd1c3b612741b961e177a91abeb81cb0d859ff2 + workingDir: /tekton/home + env: + - name: IMAGE_URL + value: $(params.image-url) + script: | + #!/usr/bin/env bash + + set -o errexit + set -o nounset + set -o pipefail + + if ! compgen -G "clair-report-*.json" > /dev/null; then + echo 'No Clair reports generated. Skipping upload.' + exit 0 + fi + + echo "Selecting auth" + select-oci-auth "$IMAGE_URL" > "$HOME/auth.json" + + repository="${IMAGE_URL/:*/}" + + arch() { + report_file="$1" + arch="${report_file/*-}" + echo "${arch/.json/}" + } + + MEDIA_TYPE='application/vnd.redhat.clair-report+json' + + reports_json="" + for f in clair-report-*.json; do + digest=$(cat "image-manifest-$(arch "$f").sha") + image_ref="${repository}@${digest}" + echo "Attaching $f to ${image_ref}" + report_digest="$(oras attach --no-tty --format go-template='{{.digest}}' --registry-config "$HOME/auth.json" --artifact-type "${MEDIA_TYPE}" "${image_ref}" "$f:${MEDIA_TYPE}")" + # shellcheck disable=SC2016 + reports_json="$(yq --output-format json --indent=0 eval-all '. as $i ireduce ({}; . * $i)' <(echo "${reports_json}") <(echo "${digest}: ${report_digest}"))" + done + echo "${reports_json}" > reports.json - name: conftest-vulnerabilities image: quay.io/redhat-appstudio/konflux-test:v1.4.7@sha256:cf6808a3bd605630a5d9f20595ff7c43f8645c00381219d32f5a11e88fe37072 # per https://kubernetes.io/docs/concepts/containers/images/#imagepullpolicy-defaulting @@ -206,6 +263,8 @@ spec: echo "$scan_result" | tee "$(results.SCAN_OUTPUT.path)" cat /tekton/home/images-processed.json | tee $(results.IMAGES_PROCESSED.path) + # shellcheck disable=SC2154 + cat /tekton/home/reports.json > "$(results.REPORTS.path)" note="Task $(context.task.name) completed: Refer to Tekton task result SCAN_OUTPUT for vulnerabilities scanned by Clair." TEST_OUTPUT=$(make_result_json -r "SUCCESS" -t "$note") diff --git a/task/clair-scan/0.2/spec/clair_scan_spec.sh b/task/clair-scan/0.2/spec/clair_scan_spec.sh new file mode 100644 index 0000000000..d205e02ae3 --- /dev/null +++ b/task/clair-scan/0.2/spec/clair_scan_spec.sh @@ -0,0 +1,177 @@ +#!/bin/env bash + +set -o errexit +set -o pipefail +set -o nounset + +eval "$(shellspec - -c) exit 1" + +task_path=clair-scan.yaml + +if [[ -f "../${task_path}" ]]; then + task_path="../${task_path}" +fi + + +extract_script() { + script="$(mktemp --tmpdir script_XXXXXXXXXX.sh)" + yq -r ".spec.steps[] | select(.name == \"$1\").script" "${task_path}" > "${script}" + chmod +x "${script}" + + echo "${script}" +} + +# array containing files/directories to remove on test exit +cleanup=() +trap 'rm -rf "${cleanup[@]}"' EXIT + +# Extract the get-vulnerabilities Step script so we can test it +get_vulnerabilities_script="$(extract_script get-vulnerabilities)" +cleanup+=("${get_vulnerabilities_script}") + +testdir() { + testdir="$(mktemp -d)" && cleanup+=("${testdir}") && cd "${testdir}" + + AfterEach 'rm -rf "$testdir"' +} + +clair_report() { + echo "report --image-ref=registry.io/repository/image@$1 --db-path=/tmp/matcher.db --format=$2" +} + + +Describe "get vulnerabilities" + BeforeEach testdir + + export IMAGE_URL=registry.io/repository/image:tag + export IMAGE_DIGEST=sha256:f0cacc1a + + It "generates reports and images-processed.json" + Mock clair-action + clair_action_args+=("$*") + %preserve clair_action_args + # expecting the --format parameter to be the last one + echo "report in ${clair_action_args[-1]#*--format=} format" + End + echo "sha256:f0cacc1a" > image-manifest-amd64.sha + echo "sha256:cc1af0ca" > image-manifest-arm64.sha + + When call "${get_vulnerabilities_script}" + The output should eq "Running clair-action on amd64 image manifest. +report in quay format +Running clair-action on arm64 image manifest. +report in quay format" + The contents of file "images-processed.json" should equal '{"image": {"pullspec": "registry.io/repository/image:tag", "digests": ["sha256:f0cacc1a","sha256:cc1af0ca"]}}' + The contents of file "clair-result-amd64.json" should equal 'report in quay format' + The contents of file "clair-report-amd64.json" should equal 'report in clair format' + The contents of file "clair-result-arm64.json" should equal 'report in quay format' + The contents of file "clair-report-arm64.json" should equal 'report in clair format' + The variable clair_action_args[@] should eq "$(clair_report sha256:f0cacc1a quay) "\ +"$(clair_report sha256:f0cacc1a clair) "\ +"$(clair_report sha256:cc1af0ca quay) "\ +"$(clair_report sha256:cc1af0ca clair)" + End + + It "fails in clair-action quay report" + Mock clair-action + clair_action_args+=("$*") + %preserve clair_action_args + [[ "$*" == *--format=quay* ]] && echo "didn't work out" && exit 1 + End + echo "sha256:f0cacc1a" > image-manifest-amd64.sha + + When call "${get_vulnerabilities_script}" + The output should eq "Running clair-action on amd64 image manifest. +didn't work out" + The contents of file "images-processed.json" should equal '{"image": {"pullspec": "registry.io/repository/image:tag", "digests": ["sha256:f0cacc1a"]}}' + The contents of file "clair-result-amd64.json" should equal "didn't work out" + The file "clair-report-amd64.json" should not exist + The variable clair_action_args[@] should eq "$(clair_report sha256:f0cacc1a quay)" + End +End + +# Extract the oci-attach-report Step script so we can test it +oci_attach_report_script="$(extract_script oci-attach-report)" +cleanup+=("${oci_attach_report_script}") + +oras_attach() { + echo "attach --no-tty --format go-template={{.digest}} --registry-config $HOME/auth.json --artifact-type application/vnd.redhat.clair-report+json registry.io/repository/image@$1 $2:application/vnd.redhat.clair-report+json" +} + +Describe "OCI attach report" + BeforeEach testdir + + export IMAGE_URL=registry.io/repository/image:tag + + It "skips attachments if no reports generated" + Mock select-oci-auth + echo select-oci-auth should not be called + End + + Mock oras + echo oras should not be called + End + + When call "${oci_attach_report_script}" + The output should eq "No Clair reports generated. Skipping upload." + End + + It "attaches for single architecture" + export HOME="${testdir}" + + Mock select-oci-auth + echo selected auth + End + + Mock oras + oras_args+=("$*") + %preserve oras_args + echo report-digest + End + + echo "sha256:f0cacc1a" > image-manifest-amd64.sha + touch clair-report-amd64.json + + When call "${oci_attach_report_script}" + The output should eq "Selecting auth +Attaching clair-report-amd64.json to registry.io/repository/image@sha256:f0cacc1a" + The contents of file "auth.json" should equal "selected auth" + The variable oras_args[@] should eq "$(oras_attach sha256:f0cacc1a clair-report-amd64.json)" + The contents of file "reports.json" should equal '{"sha256:f0cacc1a":"report-digest"}' + End + + It "attaches for multiple architecture" + export HOME="${testdir}" + + Mock select-oci-auth + echo selected auth + End + + Mock oras + oras_args+=("$*") + %preserve oras_args + for a in "$@"; do + if [[ "$a" == *@sha256:* ]]; then + echo "sha256:$(echo "${a/*@sha256:/}" | rev)" + break + fi + done + End + + echo "sha256:f0cacc1a" > image-manifest-amd64.sha + echo "sha256:cc1af0ca" > image-manifest-arm64.sha + echo "sha256:f01acacc" > image-manifest-ppc64le.sha + touch clair-report-{amd64,arm64,ppc64le}.json + + When call "${oci_attach_report_script}" + The output should eq "Selecting auth +Attaching clair-report-amd64.json to registry.io/repository/image@sha256:f0cacc1a +Attaching clair-report-arm64.json to registry.io/repository/image@sha256:cc1af0ca +Attaching clair-report-ppc64le.json to registry.io/repository/image@sha256:f01acacc" + The contents of file "auth.json" should equal "selected auth" + The variable oras_args[@] should eq "$(oras_attach sha256:f0cacc1a clair-report-amd64.json) "\ +"$(oras_attach sha256:cc1af0ca clair-report-arm64.json) "\ +"$(oras_attach sha256:f01acacc clair-report-ppc64le.json)" + The contents of file "reports.json" should equal '{"sha256:f0cacc1a":"sha256:a1ccac0f","sha256:cc1af0ca":"sha256:ac0fa1cc","sha256:f01acacc":"sha256:ccaca10f"}' + End +End