Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support docker-like references #6

Merged
merged 5 commits into from
Dec 16, 2024
Merged

feat: Support docker-like references #6

merged 5 commits into from
Dec 16, 2024

Conversation

jssblck
Copy link
Member

@jssblck jssblck commented Dec 16, 2024

Overview

Adds support for docker-like references.
The following all work:

docker pull ubuntu
docker pull contribsys/faktory
docker pull docker.io/ubuntu
docker pull library/ubuntu

Previously, circe would error on these cases; now it doesn't, and works as you'd expect.
In effect, partially-qualified references are expanded to docker.io/library/{name} by default.

The choice of docker.io being the default was chosen mainly for compatibility with the behavior in FOSSA CLI. I considered adding more hosts, but decided it'd be better to push users to fully qualify their references instead of having them lean on potentially surprising fallback behavior.

Acceptance criteria

Docker-style pulls are possible.

Testing plan

Automated tests cover this mostly; did some basic manual tests below:

; cargo run -- extract contribsys/faktory scratch --overwrite
   Compiling circe_lib v0.3.3 (/Users/jess/projects/circe/lib)
   Compiling circe v0.3.3 (/Users/jess/projects/circe/bin)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.52s
     Running `target/debug/circe extract contribsys/faktory scratch --overwrite`
┐main{opts=Options { target: Target { image: "contribsys/faktory", platform: None, username: None, .. }, output_dir: "scratch", overwrite: true, layers: Squash, layer_glob: None, file_glob: None, layer_regex: None, file_regex: None }}
├─  INFO extracting image
├─  WARN expanding 'contribsys/faktory' to 'docker.io/contribsys/faktory'; fully specify the reference to avoid this behavior
├─  INFO removing existing output directory, path="/Users/jess/projects/circe/scratch"
├─  INFO enumerated 6 layers
├─  INFO applying layer 1 of 6, layer=sha256:cf04c63912e16506c4413937c7f4579018e4bb25c272d989789cfba77b12f951
├─  INFO applying layer 2 of 6, layer=sha256:c7d295e5ea131404d7daad6c665c069eaca393373338ad18a14b90556f0540f5
├─  INFO applying layer 3 of 6, layer=sha256:6fbf45066bb4f8b01361714c3110fa6963bc3b5a7b1e525802f204283a3b20b4
├─  INFO applying layer 4 of 6, layer=sha256:1704d03cf7438aa26afa5705246511247b3530b64ff596114aea2a5ac7bd7ef0
├─  INFO applying layer 5 of 6, layer=sha256:f5a817431cb3b1f5362e01b289f7989be55e37b9e9baaec262263078eedf7676
├─  INFO applying layer 6 of 6, layer=sha256:a124f83435a22b4e87a7d4e544a7f0b3273e06d196c181cc3b1b7945961640d1
├─  INFO finished applying layers
┘
; cargo run -- extract ubuntu scratch --overwrite                            
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.11s
     Running `target/debug/circe extract ubuntu scratch --overwrite`
┐main{opts=Options { target: Target { image: "ubuntu", platform: None, username: None, .. }, output_dir: "scratch", overwrite: true, layers: Squash, layer_glob: None, file_glob: None, layer_regex: None, file_regex: None }}
├─  INFO extracting image
├─  WARN expanding 'ubuntu' to 'docker.io/library/ubuntu'; fully specify the reference to avoid this behavior
├─  INFO removing existing output directory, path="/Users/jess/projects/circe/scratch"
├─  INFO enumerated 1 layer
├─  INFO applying layer 1 of 1, layer=sha256:8bb55f0677778c3027fcc4253dc452bc9c22de989a696391e739fb1cdbbdb4c2
├─  INFO finished applying layers
┘
; cargo run -- extract docker.io/ubuntu scratch --overwrite
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.11s
     Running `target/debug/circe extract docker.io/ubuntu scratch --overwrite`
┐main{opts=Options { target: Target { image: "docker.io/ubuntu", platform: None, username: None, .. }, output_dir: "scratch", overwrite: true, layers: Squash, layer_glob: None, file_glob: None, layer_regex: None, file_regex: None }}
├─  INFO extracting image
├─  WARN expanding 'docker.io/ubuntu' to 'docker.io/library/ubuntu'; fully specify the reference to avoid this behavior
├─  INFO removing existing output directory, path="/Users/jess/projects/circe/scratch"
├─  INFO enumerated 1 layer
├─  INFO applying layer 1 of 1, layer=sha256:8bb55f0677778c3027fcc4253dc452bc9c22de989a696391e739fb1cdbbdb4c2
├─  INFO finished applying layers
┘

Metrics

None

Risks

Allowing users to provide partially qualified references introduces potential for confusion into the system, but since this is the same behavior as docker (with which most container users are familiar) I think this is minimized.

Checklist

  • I added tests for this PR's change (or explained in the PR description why tests don't make sense).

@jssblck jssblck requested a review from a team as a code owner December 16, 2024 19:31
@jssblck jssblck requested a review from csasarak December 16, 2024 19:31
/// - `docker.io/library/ubuntu@sha256:1234567890` is resolved as `docker.io/library/ubuntu@sha256:1234567890`
/// - `docker.io/library/ubuntu:24.04` is resolved as `docker.io/library/ubuntu:24.04`
#[arg(verbatim_doc_comment)]
pub image: String,
Copy link

@csasarak csasarak Dec 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

[Optional, probably more relevant for the CLI and super minor] It might be worth making the "base" configurable via env var. If I'm a company who has a list of image names it may be simpler for me to set an env var specifying the base rather than have to program my CI jobs to concatenate it together with the base name. It also means that I can create a CI job template that sets the var and that makes calls to this do the right thing rather than making my engineers have to remember to use the FQN.

^^ I'm a little dubious that this is super useful, but I think it's worth explicitly rejecting.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's an incredible idea! made that quick change.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about this more over lunch, and it would be similar to also have CLI options that can do that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this is behavior that we probably don't want in flags; it's not really intended to be used most of the time, it's just an escape hatch.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks!

@jssblck jssblck enabled auto-merge December 16, 2024 20:08
@jssblck jssblck added this pull request to the merge queue Dec 16, 2024
@jssblck jssblck removed this pull request from the merge queue due to a manual request Dec 16, 2024
@jssblck jssblck merged commit c8e892c into main Dec 16, 2024
20 checks passed
@jssblck jssblck deleted the docker-integration branch December 16, 2024 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants