Building multi-arch Images #1103

enyachoke · 2023-02-17T16:57:08Z

This PR makes changes to the dockerfiles to compile the source code in a multi step build. This then opens up the possibility of building images for multiple architectures. I need this change since I am currently evaluating this in a K3s cluster running on arm64 nodes.

devdattakulkarni · 2023-02-17T17:46:58Z

I will go through the changes later today.

Based on a quick scan of the included files, I suggest not to include deploy/kubeplus-chart/kubeplus-components-6.yaml in the PR. We want to maintain official image tags from cloudark-kubeplus GCR repo when we publish the KubePlus Helm chart.

enyachoke · 2023-02-18T04:51:36Z

Based on a quick scan of the included files, I suggest not to include deploy/kubeplus-chart/kubeplus-components-6.yaml in the PR. We want to maintain official image tags from cloudark-kubeplus GCR repo when we publish the KubePlus Helm chart.

Yeah, will update that part later today needed to get everything at least deployed to validate. I will move that to the values.yaml

enyachoke · 2023-02-22T05:55:54Z

@devdattakulkarni I intend to switch the scripts to use https://docs.docker.com/engine/reference/commandline/buildx/ that of course will require a once time pre step

$ docker buildx create --name mybuilder
$ docker buildx use mybuilder

For now, the actions only build the latest but we will need a release process to allow us to build tagged images. I am thinking we align the chart version with the tags then we can keep the individual components versions in the versions.txt. That way we can use the tags to build the corresponding images.

devdattakulkarni · 2023-03-01T13:34:40Z

.github/workflows/build-consumerui.yaml

@@ -0,0 +1,52 @@
+name: Build consumerui


Hey @enyachoke

Can you point me to a good resource to come up to speed on Github workflows? I haven't studied them in great details. This link has some introductory material, but I was not able to find information about aspects of the syntax that you have used below, such as the release action ('release'), types of releases ('published', 'edited'), the various actions (actions/checkout@v3 - what does that mean), etc.

At a high level, I am able to understand what this file is doing but would like to precisely understand the meaning of each step.

I am running this workflow on push to master to build the latest and on release to build the releases. See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#release for details on this trigger.

actions/checkout@v3

Checks out the repository in the GitHub actions env

docker/setup-qemu-action

Install QEMU to allow us to run multi-arch builds

docker/setup-buildx-action

This does the step I mentioned above

$ docker buildx create --name mybuilder $ docker buildx use mybuilder

which is necessary for buildx

docker/login-action@v2

This logs into the registry and you will have to setup this secrets

REGISTRY=gcr.io REGISTRY_USERNAME=cloudark-kubeplus REGISTRY_PASSWORD=thepassword for the registry

docker/metadata-action@v4

This prepares the tags needed in the image build step. This step uses conditionals and will create for example

gcr.io/cloudark-kubeplus/consumerui:latest when the actions are running for the default branch which in this case is master or gcr.io/cloudark-kubeplus/consumerui:0.0.1 if the action is triggered by a release. Not we extra the version in the step Set Version

docker/build-push-action@v4

Finally this step build and pushes the images.

devdattakulkarni · 2023-03-08T11:11:45Z

.github/workflows/build-mutating-webhook-helper.yaml

@@ -0,0 +1,50 @@
+name: Build mutating-webhook-helper


mutating-webhook-helper is not used anymore.

We can exclude it from the github workflow.

devdattakulkarni · 2023-03-08T11:16:26Z

consumerui/Dockerfile

-ADD grapher.py /root/grapher.py
-ADD templates /root/templates
-ADD static /root/static
+ADD consumerui/requirements.txt /root/requirements.txt


Why is this change needed?

I see this type of change in all the Dockerfiles below. Why is that needed?

Won't this affect local builds which are needed during development?

We are building the binaries within the Dockerfile, and the context is at the base of the repository. We could have kept the context within the service folders but I think some of the go applications reference components in the base of the project and Docker does not allow us to do something like ../

What a believe we need to do is update the scripts used to build the development images and have the context set to the base of the project.

I went through the matrix strategy docs. I am still unclear on how the local builds will happen.

The current workflow to test any new code change is as follows:

make code changes

use the build scripts to create the latest tag

change the appropriate container in the kubeplus deployment yaml in deploy/kubeplus-chart/template to use the latest tag

deploy the kubeplus chart from deploy/kubeplus-chart

The changes to the Dockerfiles will break this workflow.

devdattakulkarni · 2023-03-08T11:17:21Z

deploy/Dockerfile

+RUN apt-get update && apt-get install -y openssl curl jq python3 python3-pip && pip3 install pyyaml
+ADD deploy/webhook-create-self-signed-ca-cert.sh /
+RUN arch=$(arch | sed s/aarch64/arm64/ | sed s/x86_64/amd64/) && cd /root/ && curl -LO "https://dl.k8s.io/release/v1.26.0/bin/linux/${arch}/kubectl"
+COPY deploy/kubeplus-non-pod-resources.yaml /root/.


Same comment as that for Dockerfile for consumerui.

devdattakulkarni · 2023-03-08T11:19:15Z

deploy/Dockerfile.kubeconfiggenerator

 RUN cp /root/kubectl bin/. && chmod +x /root/kubectl && chmod +x bin/kubectl && chmod +x /root/kubeconfiggenerator.sh && cp /root/helm bin/. && chmod +x /root/helm && chmod +x bin/helm

 #ENTRYPOINT ["/root/kubeconfiggenerator.sh"]

+#ENTRYPOINT ["/root/kubeconfiggenerator.sh"]


Extra. Can be deleted.

devdattakulkarni · 2023-03-08T11:21:27Z

mutating-webhook-helper/Dockerfile

@@ -1,5 +1,10 @@
-FROM ubuntu:20.04
+FROM golang:1.20.1-bullseye as builder


mutating-webhook-helper is not being used. So we can ignore this file.

devdattakulkarni · 2023-03-08T11:33:39Z

deploy/kubeplus-chart/values.yaml

+kubeconfiggenerator:
+  image:
+    registry: gcr.io
+    repository: cloudark-kubeplus/kubeconfiggenerator


is there an accidental newline after "cloudark-" ?

devdattakulkarni · 2023-03-09T11:28:06Z

deploy/kubeconfiggenerator.py

@@ -493,6 +493,8 @@ def flatten(yaml_contents, flattened, types_dict, prefix=''):
            inner_prop_dict = {}
            prop_dict = {'properties': inner_prop_dict}
            prop_dict['type'] = 'object'
+            #Allow 
+            prop_dict['additionalProperties'] = True


Is this change for the requirement that you had identified regarding supporting empty annotations?

If so, it will be better to separate it in a separate PR. That way it will be easier to merge it while we are still going through the build related changes.

devdattakulkarni · 2024-04-10T11:10:51Z

@enyachoke Hello. I hope things are going well for you.
I would like to get this PR merged. Any chance, you might have some bandwidth to take a fresh look at it and see if it is up-to-date?

devdattakulkarni · 2024-07-16T16:13:34Z

consumerui/Dockerfile

-RUN curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
-
-RUN cp /usr/bin/python3.8 /usr/bin/python
+RUN arch=$(arch | sed s/aarch64/arm64/ | sed s/x86_64/amd64/) && \


What is output of arch on a arm machine? Why do we need two sed commands?

On Linux the arch command will return aarch64 for ARM and x86_64 for AMD64 machine. Most ackage are published as either arm64 or amd64 so we need to run sed to translate.

Why do we need two sed commands?
Not the or (|)

mutating-webhook/build-artifact.sh

devdattakulkarni · 2024-07-16T16:20:43Z

deploy/values.yaml

I think you have added this file just as a way to know how the rendered values.yaml will look. Correct?

devdattakulkarni · 2024-07-16T16:23:47Z

@enyachoke Thanks for updating the PR. I went through it and have added couple of questions/comments.

Right now the CI is failing. Need to look into it.
Separately, it will be good to document/identify changes for dev/test locally.

enyachoke · 2024-07-19T16:34:58Z

@enyachoke Thanks for updating the PR. I went through it and have added couple of questions/comments.

Right now the CI is failing. Need to look into it. Separately, it will be good to document/identify changes for dev/test locally.

I see some tests are failing. I am not sure how I could fix those

devdattakulkarni · 2024-07-19T19:46:04Z

@enyachoke Thanks for updating the PR. I went through it and have added couple of questions/comments.
Right now the CI is failing. Need to look into it. Separately, it will be good to document/identify changes for dev/test locally.

I see some tests are failing. I am not sure how I could fix those

Hmm.. I want to rule out any issues with the tests themselves. Sometimes that happens. The best way to know that is to push another change and see if the tests fail or succeed. I will be working on one of the issues in next couple days. Once I submit a PR for that, we can see if the tests are failing for that PR too.

enyachoke marked this pull request as draft February 17, 2023 16:57

enyachoke mentioned this pull request Feb 17, 2023

Publish multiarch Images #1100

Open

enyachoke force-pushed the master branch 2 times, most recently from abead1b to 3c10a9f Compare February 22, 2023 05:46

enyachoke marked this pull request as ready for review February 22, 2023 19:16

devdattakulkarni reviewed Mar 1, 2023

View reviewed changes

enyachoke force-pushed the master branch from 0f8a12c to 9775adf Compare March 2, 2023 04:57

devdattakulkarni reviewed Mar 8, 2023

View reviewed changes

enyachoke force-pushed the master branch from 9775adf to 8db48d1 Compare March 9, 2023 04:44

devdattakulkarni reviewed Mar 9, 2023

View reviewed changes

enyachoke force-pushed the master branch 5 times, most recently from e830704 to 13001c2 Compare March 19, 2023 06:54

enyachoke changed the title ~~Proof of Concept PR for building multi-arch Images~~ Building multi-arch Images Mar 19, 2023

enyachoke force-pushed the master branch from 13001c2 to 702b9da Compare March 19, 2023 08:47

devdattakulkarni mentioned this pull request Jul 20, 2023

How to upgrade all the currently running application instances #1160

Closed

enyachoke added 2 commits July 10, 2024 06:54

Add github actions to build multi arch images

590ee45

Comment export GO111MODULE in helm pod

09ad3e3

enyachoke force-pushed the master branch from 3950c2d to 09ad3e3 Compare July 10, 2024 03:55

enyachoke added 4 commits July 10, 2024 07:01

Remove newline

19f4936

Fix newline

c9c646d

Fix newline

a4bbee2

Fix newline

5657e97

devdattakulkarni reviewed Jul 16, 2024

View reviewed changes

mutating-webhook/build-artifact.sh Show resolved Hide resolved

devdattakulkarni reviewed Jul 16, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Building multi-arch Images #1103

Building multi-arch Images #1103

enyachoke commented Feb 17, 2023

devdattakulkarni commented Feb 17, 2023

enyachoke commented Feb 18, 2023

enyachoke commented Feb 22, 2023

devdattakulkarni Mar 1, 2023

enyachoke Mar 2, 2023

devdattakulkarni Mar 8, 2023

devdattakulkarni Mar 8, 2023

enyachoke Mar 8, 2023

enyachoke Mar 8, 2023

devdattakulkarni Mar 9, 2023

devdattakulkarni Mar 8, 2023

devdattakulkarni Mar 8, 2023

devdattakulkarni Mar 8, 2023

devdattakulkarni Mar 8, 2023 •

edited

Loading

devdattakulkarni Mar 9, 2023

devdattakulkarni commented Apr 10, 2024

devdattakulkarni Jul 16, 2024

enyachoke Jul 19, 2024

devdattakulkarni Jul 16, 2024

devdattakulkarni commented Jul 16, 2024

enyachoke commented Jul 19, 2024

devdattakulkarni commented Jul 19, 2024

		@@ -1,5 +1,10 @@
		FROM ubuntu:20.04
		FROM golang:1.20.1-bullseye as builder

Building multi-arch Images #1103

Are you sure you want to change the base?

Building multi-arch Images #1103

Conversation

enyachoke commented Feb 17, 2023

devdattakulkarni commented Feb 17, 2023

enyachoke commented Feb 18, 2023

enyachoke commented Feb 22, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devdattakulkarni Mar 8, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devdattakulkarni commented Apr 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devdattakulkarni commented Jul 16, 2024

enyachoke commented Jul 19, 2024

devdattakulkarni commented Jul 19, 2024

devdattakulkarni Mar 8, 2023 •

edited

Loading