Having problems pulling trivy-checks

[gal3mu]

Description

Pulling the trivy-checks image locally, on the cluster from ghcr or on the cluster from the public AWS ECR registry fails.###

Desired Behavior

I would like to understand why I can not pull the image in no way, shape or form. Best would be if I could somehow use the one from the public AWS ECR registry, in order to not hit the TOOMANYREQUESTS error mentioned below, but I do not understand how I should configure the values.yaml, since the AWS image has no tags.

Actual Behavior

Locally:

❯ docker pull ghcr.io/aquasecurity/trivy-checks:0
0: Pulling from aquasecurity/trivy-checks
unsupported media type application/vnd.cncf.openpolicyagent.config.v1+json

From the cluster with ghcr:

{"level":"error","ts":"2024-10-02T14:52:04Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to download policies: failed to download built-in policies: download error: oci download error: failed to fetch the layer: GET https://ghcr.io/v2/aquasecurity/trivy-checks/blobs/sha256:cba49b6781cfcdeb6b063283a711ce0ddb1f36d6e2a5db69ef7d2e3f13998149: TOOMANYREQUESTS: retry-after: 421.404µs, allowed: 44000/minute","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPoliciesAndBundlePath\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.g
o:63\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Hash\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:114\ngithub.com/aquasecurity/trivy-operator/pkg/operator.(*TTLReportReconciler).applicableForDeletion\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/operator/ttl_report.go:152\ngithub.com/aquasecurity/trivy-operator/pkg/operator.(*TTLReportReconciler).DeleteReportIfExpired\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/oper
ator/ttl_report.go:105\ngithub.com/aquasecurity/trivy-operator/pkg/operator.(*TTLReportReconciler).SetupWithManager.(*TTLReportReconciler).reconcileReport.func5\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/operator/ttl_report.go:79\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/contr
oller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}
{"level":"error","ts":"2024-10-02T14:52:10Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to download policies: failed to download built-in policies: download error: oci download error: failed to fetch the layer: GET https://ghcr.io/v2/aquasecurity/trivy-checks/blobs/sha256:cba49b6781cfcdeb6b063283a711ce0ddb1f36d6e2a5db69ef7d2e3f13998149: TOOMANYREQUESTS: retry-after: 464.051µs, allowed: 44000/minute","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.(*policyLoader).GetPoliciesAndBundlePath\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.g
o:63\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Hash\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:114\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*ResourceController).SetupWithManager.(*ResourceController).reconcileResource.func2\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/resource.go:208\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go
/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/control
ler/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}

From the cluster with the public ECR registry:

{"level":"error","ts":"2024-10-02T16:07:00Z","logger":"policyLoader.Get misconfig bundle policies","msg":"failed to load policies","error":"failed to download policies: failed to download built-in policies: download error: OCI repository error: 1 error occurred:\n\t* GET https://public.ecr.aws/v2/aquasecurity/trivy-checks/manifests/latest: MANIFEST_UNKNOWN: Requested image not found\n\n","stacktrace":"github.com/aquasecurity/trivy-operator/pkg/policy.*policyLoader).GetPoliciesAndBundlePath\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/loader.go:63\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).loadPolicies\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:144\ngithub.com/aquasecurity/trivy-operator/pkg/policy.(*Policies).Hash\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/policy/policy.go:114\ngithub.com/aquasecurity/trivy-operator/pkg/configauditreport/controller.(*ResourceController).SetupWithManager.(*ResourceController).reconcileResource.func2\n\t/home/runner/work/trivy-operator/trivy-operator/pkg/configauditreport/controller/resource.go:208\nsigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/reconcile/reconcile.
go:113\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/contr
oller.(*Controller).Start.func2.2\n\t/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222"}

Reproduction Steps

Locally:
1. Run `docker pull ghcr.io/aquasecurity/trivy-checks:0`

With ghcr:
1. Deploy Trivy Operator Helm Chart with the default `policiesBundle` values.

With public AWS EKS ECR:
1. Deploy Trivy Operator Helm Chart with the AWS EKS ECR Registry configuration beneath the `policiesBundle` values, like here:

  policiesBundle:
    registry: public.ecr.aws
    repository: aquasecurity/trivy-checks
    tag: "0"

You can try also without the 0, since it seems like AWS does not tag the images.

### Target

AWS

### Scanner

Misconfiguration

### Output Format

JSON

### Mode

Standalone

### Debug Output

```bash
See Actual behavior for logs.

Operating System

EKS

Version

0.24.1

Checklist

• Run trivy clean --all
• Read the troubleshooting

[simar7]

oras pull ghcr.io/aquasecurity/trivy-checks:latest      
Downloading 16957b935ef8 bundle.tar.gz
Downloaded  16957b935ef8 bundle.tar.gz
Pulled [registry] ghcr.io/aquasecurity/trivy-checks:latest
Digest: sha256:48c53a16cc9bd4935e124993cd28d7c70df8560162b8481ac779c750874963e1

[gal3mu]

Hey @simar7

Would you maybe know from the top of your head why docker pull ghcr.io/aquasecurity/trivy-checks:latest is not working?

❯ docker pull ghcr.io/aquasecurity/trivy-checks:latest
latest: Pulling from aquasecurity/trivy-checks
unsupported media type application/vnd.cncf.openpolicyagent.config.v1+json

What about the Trivy Operator with the AWS Registy? I am getting an MANIFEST_UNKNOWN error when I try to use it.

[simar7]

I think the error is clear, docker cli doesn't support the media type.

[peetw]

@simar7 do you know if the AWS Public ECR public.ecr.aws/aquasecurity/trivy-checks repo will be populated in the future? Would be good to be able to add it as a fallback in case the GHCR repo is unavailable (e.g. TRIVY_CHECKS_BUNDLE_REPOSITORY: ghcr.io/aquasecurity/trivy-checks, public.ecr.aws/aquasecurity/trivy-checks)

[simar7]

Trivy OSS already embeds a bundle as a fallback for checks in the binary itself. At the moment we don't have any plans to host the checks bundle on ECR.

[jepp-igus]

Sadly, our logs are full with the "TOOMANYREQUESTS" error.
Are there any plans to make trivy-checks configurable to other container registries?
We already changed the dbRepository and javaDbRepository and that worked like a charm.

Only trivy-checks is making a lot of noise.

[simar7]

#7636 (reply in thread)