Implement tag versioning. #84

junaruga · 2019-08-16T00:19:55Z

Sorry, related to the PR: #83 and #74 , let me fix the versioning with minimal steps.

Because I think @meeDamian and #74 's reporter want to see the versioning will be implemented with high priority soon. Other features on #83 are not urgent for them.

Testing

Here is my Travis's result.
You see the containers are created with version tag.
https://travis-ci.org/junaruga/qemu-user-static/builds/572536735#L2217

$ docker images
REPOSITORY                   TAG                        IMAGE ID            CREATED              SIZE
multiarch/qemu-user-static   latest                     b5cb43048545        1 second ago         133MB
multiarch/qemu-user-static   v4.0                       b5cb43048545        1 second ago         133MB
multiarch/qemu-user-static   register                   1597e2d87f47        1 second ago         1.25MB
multiarch/qemu-user-static   register-v4.0              1597e2d87f47        1 second ago         1.25MB
multiarch/qemu-user-static   x86_64-xtensa              19f490eb93c7        10 seconds ago       4.65MB
multiarch/qemu-user-static   x86_64-xtensa-v4.0         19f490eb93c7        10 seconds ago       4.65MB
multiarch/qemu-user-static   xtensa                     19f490eb93c7        10 seconds ago       4.65MB
multiarch/qemu-user-static   xtensa-v4.0                19f490eb93c7        10 seconds ago       4.65MB
...

Here is my testing container repository.
https://quay.io/repository/junaruga/qemu-user-static?tab=tags

I also tested new version tag.

$ docker run --rm --privileged quay.io/junaruga/qemu-user-static --reset -p yes
$ docker run --rm --privileged quay.io/junaruga/qemu-user-static:v4.0 --reset -p yes

$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64
...
flags: F
...

$ docker run --rm --privileged quay.io/junaruga/qemu-user-static:register --reset
$ docker run --rm --privileged quay.io/junaruga/qemu-user-static:register-v4.0 --reset

$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64
...
flags:
...

$ docker run --rm -t quay.io/junaruga/qemu-user-static:x86_64-aarch64 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t quay.io/junaruga/qemu-user-static:x86_64-aarch64-v4.0 /usr/bin/qemu-aarch64-static -version

After merging this PR, I want to run with VERSION: v3.1.0-3 to create the "v3.1" containers.
Then run it with v.4.0.0-5 again.

I think it is good enough to fix #74 .

junaruga · 2019-08-16T09:05:19Z

update.sh

+docker build -t ${DOCKER_REPO}:${TAG_VER} "${out_dir}/latest"
+docker tag ${DOCKER_REPO}:${TAG_VER} ${DOCKER_REPO}:latest
+docker build -t ${DOCKER_REPO}:register-${TAG_VER} "${out_dir}/register"
+docker tag ${DOCKER_REPO}:register-${TAG_VER} ${DOCKER_REPO}:register


I think we do not need to do versioning for register image. I want to remove my modification 2 lines for the register image here.
Because actually what register image is doing is just to execute the latest qemu's qemu-binfmt-conf.sh script [1] to create binfmt_misc files. It's not related to QEMU's version. qemu-$arch-static binary is used at user's run time, the timing to run arch specific binary.

Do you agree?

[1] https://github.com/multiarch/qemu-user-static/blob/master/containers/latest/Dockerfile#L4

I removed the modification for now.

meeDamian · 2019-08-16T09:08:51Z

Please note that v4.0.0-5 does not work for me, while v4.0.0-4 does work well. How do I pull & use image build with v4.0.0-4?

junaruga · 2019-08-16T09:12:23Z

Please note that v4.0.0-5 does not work for me, while v4.0.0-4 does work well. How do I pull & use image build with v4.0.0-4?

Ah sure. I did not consider the case. That's a good rational reason to change this PR's implementation.

Below is edited.

Changing the tag from vX.Y to vX.Y.Z-R is a possible way.
But let me think about the down side to increase the number of versioning images more.

Does QEMU v3.X work on you for the case?

meeDamian · 2019-08-16T09:23:05Z

I'm afraid vX.Y.Z is also not granular enough to capture the relevant version difference :). As both v4.0.0-5, and v4.0.0-4 would be represented as v4.0.0.

That's why some popular Docker images end up using a combination of very specific, and static version tags, as well as "floating" ones that capture various levels of granularity.

Example: https://hub.docker.com/_/busybox that has:

1.31.0, 1.31, 1, latest

Where:

1.31.0 would never change,
1.31 always points to the latest version of 1.31.X branch,
1 always points to the latest version of 1.X.X branch, and
latest always points to the very latest released version .

That's why my recommendation would be to always:

use the never changing tag - so that's it's always possible to reference the exact version that existed before (I only pull them down when a serious CVE has been found)
Create as many "floating" tags as useful/convenient.

junaruga · 2019-08-16T09:32:39Z

Yes, I agree vX.Y.Z is not enough. That's the reason why we need to think about the downside of the versioning tag: vX.Y.Z-R in terms of maintainability, resource, cost and etc.
Yes, I know the container image's versioning rules.

I am maintaining a Ruby container at Red Hat. The versioning is "major version + minor version" (ruby-25).
It's not only for Ruby container, but also some other containers.
https://access.redhat.com/containers/?tab=overview#/registry.access.redhat.com/ubi8/ruby-25

There was a discussion about the versioning in the past time in the company.
That's why I tried to adapt same pattern to this project's container.
But in the case of this project, the situation is different. I admit "major version + minor version" is not enough.

junaruga · 2019-08-16T09:59:10Z

I am maintaining a Ruby container at Red Hat. The versioning is "major version + minor version" (ruby-25).
It's not only for Ruby container, but also some other containers.

NOTE: The reason of "major version + minor version" is good enough for Red Hat is, when we (Red Hat) faces an issue of specific version of the source code (in this project it is QEMU), we directly fix the issue patching the original source code by a kind of patch command in the pipeline system. We also need to commit for every release versioned container image. Reducing the number of the containers are good strategy for that.

But for multiarch/qemu-user-static project, there is no such operation, and I do not think the operation is necessary.

@meeDamian

Special thanks @meeDamian.

moul

LGTM

junaruga · 2019-08-16T11:16:46Z

Rebased and tested again.

The CI result of my repository.
https://travis-ci.org/junaruga/qemu-user-static/builds/572702098

$ docker images
REPOSITORY                          TAG                            IMAGE ID            CREATED                  SIZE
quay.io/junaruga/qemu-user-static   register                       dea10951272b        Less than a second ago   1.25MB
quay.io/junaruga/qemu-user-static   latest                         9f4e3c07ee80        Less than a second ago   133MB
quay.io/junaruga/qemu-user-static   v4.0.0-5                       9f4e3c07ee80        Less than a second ago   133MB
quay.io/junaruga/qemu-user-static   x86_64-xtensa                  9e61904d9c25        9 seconds ago            4.65MB
quay.io/junaruga/qemu-user-static   x86_64-xtensa-v4.0.0-5         9e61904d9c25        9 seconds ago            4.65MB
quay.io/junaruga/qemu-user-static   xtensa                         9e61904d9c25        9 seconds ago            4.65MB
quay.io/junaruga/qemu-user-static   xtensa-v4.0.0-5                9e61904d9c25        9 seconds ago            4.65MB
...

As the changes after the review, the version tag is "vX.Y.Z-R" (= $tag_version) to fill the requirement.
The alias of the tag is

multiarch/qemu-user-static:$tag_version images have the alias: multiarch/qemu-user-static:latest.
multiarch/qemu-user-static:$from_arch-$to_arch-$tag_version images have the aliases: multiarch/qemu-user-static:$from_arch-$to_arch, multiarch/qemu-user-static:$to_arch-$tag_version and multiarch/qemu-user-static:$to_arch.
multiarch/qemu-user-static:register DOES NOT have the alias of the versioning. Because the container image does not actually have QEMU version spesific logic.

I tested like this again.

$ docker run --rm --privileged quay.io/junaruga/qemu-user-static --reset -p yes
$ docker run --rm --privileged quay.io/junaruga/qemu-user-static:v4.0.0-5 --reset -p yes
$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64

$ docker run --rm --privileged quay.io/junaruga/qemu-user-static:register --reset
$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64

$ docker run --rm -t quay.io/junaruga/qemu-user-static:x86_64-aarch64 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t quay.io/junaruga/qemu-user-static:x86_64-aarch64-v4.0.0-5 /usr/bin/qemu-aarch64-static -version

junaruga · 2019-08-16T11:25:39Z

Thanks for the review, everyone.
Now I will start to run pipelines with 2 old versions such as v3.1.1-2 (latest version of v3.x) and v4.0.0-4 modifying the .travis.yml.
Later I will update the document with new feature too.

junaruga · 2019-08-16T11:39:47Z

NOTE: The images on our official container repository work now.

https://hub.docker.com/r/multiarch/qemu-user-static/tags

$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
$ docker run --rm --privileged multiarch/qemu-user-static:v4.0.0-5 --reset -p yes
$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64

$ docker run --rm --privileged multiarch/qemu-user-static:register --reset
$ cat /proc/sys/fs/binfmt_misc/qemu-aarch64

$ docker run --rm -t multiarch/qemu-user-static:x86_64-aarch64 /usr/bin/qemu-aarch64-static -version
$ docker run --rm -t multiarch/qemu-user-static:x86_64-aarch64-v4.0.0-5 /usr/bin/qemu-aarch64-static -version

junaruga mentioned this pull request Aug 16, 2019

Simplify & introduce versioning #83

Closed

junaruga requested a review from moul August 16, 2019 00:25

junaruga commented Aug 16, 2019

View reviewed changes

junaruga requested a review from lafin August 16, 2019 09:36

junaruga force-pushed the feature/tag-version branch from 848b614 to 07d7e66 Compare August 16, 2019 10:16

Implement tag versioning.

c7c0753

Special thanks @meeDamian.

junaruga force-pushed the feature/tag-version branch from 07d7e66 to c7c0753 Compare August 16, 2019 11:05

moul approved these changes Aug 16, 2019

View reviewed changes

lafin approved these changes Aug 16, 2019

View reviewed changes

junaruga merged commit a80f6b0 into multiarch:master Aug 16, 2019

junaruga deleted the feature/tag-version branch August 16, 2019 11:21

This was referenced Aug 16, 2019

Run pipeline with old version qemu 3.1.1-2.fc30. #85

Merged

Run pipeline with old version qemu 4.0.0-4.fc31. #87

Merged

junaruga mentioned this pull request Aug 26, 2019

Image version tag to change from "vX.Y.Z-R" to "X.Y.Z-R" #91

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement tag versioning. #84

Implement tag versioning. #84

junaruga commented Aug 16, 2019 •

edited

Loading

junaruga Aug 16, 2019 •

edited

Loading

junaruga Aug 16, 2019

meeDamian commented Aug 16, 2019

junaruga commented Aug 16, 2019 •

edited

Loading

meeDamian commented Aug 16, 2019 •

edited

Loading

junaruga commented Aug 16, 2019 •

edited

Loading

junaruga commented Aug 16, 2019 •

edited

Loading

moul left a comment

junaruga commented Aug 16, 2019

junaruga commented Aug 16, 2019 •

edited

Loading

junaruga commented Aug 16, 2019

Implement tag versioning. #84

Implement tag versioning. #84

Conversation

junaruga commented Aug 16, 2019 • edited Loading

Testing

junaruga Aug 16, 2019 • edited Loading

Choose a reason for hiding this comment

junaruga Aug 16, 2019

Choose a reason for hiding this comment

meeDamian commented Aug 16, 2019

junaruga commented Aug 16, 2019 • edited Loading

meeDamian commented Aug 16, 2019 • edited Loading

junaruga commented Aug 16, 2019 • edited Loading

junaruga commented Aug 16, 2019 • edited Loading

moul left a comment

Choose a reason for hiding this comment

junaruga commented Aug 16, 2019

junaruga commented Aug 16, 2019 • edited Loading

junaruga commented Aug 16, 2019

junaruga commented Aug 16, 2019 •

edited

Loading

junaruga Aug 16, 2019 •

edited

Loading

junaruga commented Aug 16, 2019 •

edited

Loading

meeDamian commented Aug 16, 2019 •

edited

Loading

junaruga commented Aug 16, 2019 •

edited

Loading

junaruga commented Aug 16, 2019 •

edited

Loading

junaruga commented Aug 16, 2019 •

edited

Loading