Action build-registry-server failing

[theganyo]

https://github.com/apigee/registry/actions/runs/5273705838/jobs/9537430024

[theganyo]

This issue is created by an incompatibility between our build and release container images.

We use golang:1.20 as our builder image as our builder - that label is tied to the latest 1.20 go build.

As of go1.20.5 (released a few days ago), this image is being built from the newly released Debian 12 image.

We use gcr.io/distroless/base:latest as our release image, which is still being built on Debian 11. Note: There is an issue to upgrade this to Debian 12.

We copy binaries from the build to the release container, but these are incompatible which causes the command chmod to fail to execute in the distroless container.

A quick fix for now would be to tie our build to go 1.20.4 - which was built on Debian 11.

Note: As distroless intentionally excludes all the binaries for security, if possible we should rethink copying them from the build container.

[timburks]

Note: As distroless intentionally excludes all the binaries for security, if possible we should rethink copying them from the build container.

Do you have a recommendation for what we might do if we didn't copy binaries from the build container?

[theganyo]

If we simply remove the extraneous binaries from our release, the build works but leaves us with a binary that gets the following errors when running in the container:

/registry-server: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /registry-server)
/registry-server: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /registry-server)
/registry-server: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /registry-server)

This is because we build with CGO_ENABLED=1, which ties us to the libraries. So, I suggest we do both for now:

1. Remove the extraneous binaries
2. Peg the builder at golang:1.20.4

Then when distroless/base is updated, we should be able to unpeg the go version.

[dlorenc]

Another option is to use the distroless images from Chainguard here: cgr.dev/chainguard/glibc-dynamic.

These work correctly with the updated glibc in bookworm, and are more flexible than original distroless images.

I'd be happy to help! I wouldn't recommend staying on go 1.20.4 - there are several severe CVEs patched in 1.20.5: https://go.dev/doc/devel/release#go1.20

[theganyo]

Thanks, @dlorenc, good point on the issues. Another option is mentioned on distroless here: GoogleContainerTools/distroless#1342. I've verified that both your suggestion and using golang:1.20-bullseye as our builder would work. I'm leaning toward switching to bullseye for now as it is the smallest delta. @timburks ?

[dlorenc]

Sounds good! Let me know if we can help at all.