Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building multi-arch Images #1103

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Building multi-arch Images #1103

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
590ee45
Add github actions to build multi arch images
enyachoke Mar 19, 2023
09ad3e3
Comment export GO111MODULE in helm pod
enyachoke Mar 22, 2023
19f4936
Remove newline
enyachoke Jul 10, 2024
c9c646d
Fix newline
enyachoke Jul 10, 2024
a4bbee2
Fix newline
enyachoke Jul 10, 2024
5657e97
Fix newline
enyachoke Jul 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions .github/workflows/build-consumerui.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Build consumerui
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @enyachoke

Can you point me to a good resource to come up to speed on Github workflows? I haven't studied them in great details. This link has some introductory material, but I was not able to find information about aspects of the syntax that you have used below, such as the release action ('release'), types of releases ('published', 'edited'), the various actions (actions/checkout@v3 - what does that mean), etc.

At a high level, I am able to understand what this file is doing but would like to precisely understand the meaning of each step.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am running this workflow on push to master to build the latest and on release to build the releases. See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#release for details on this trigger.

actions/checkout@v3

Checks out the repository in the GitHub actions env

docker/setup-qemu-action

Install QEMU to allow us to run multi-arch builds

docker/setup-buildx-action

This does the step I mentioned above

$ docker buildx create --name mybuilder
$ docker buildx use mybuilder

which is necessary for buildx

docker/login-action@v2

This logs into the registry and you will have to setup this secrets

REGISTRY=gcr.io
REGISTRY_USERNAME=cloudark-kubeplus
REGISTRY_PASSWORD=thepassword for the registry

docker/metadata-action@v4

This prepares the tags needed in the image build step. This step uses conditionals and will create for example

gcr.io/cloudark-kubeplus/consumerui:latest when the actions are running for the default branch which in this case is master or gcr.io/cloudark-kubeplus/consumerui:0.0.1 if the action is triggered by a release. Not we extra the version in the step Set Version

docker/build-push-action@v4

Finally this step build and pushes the images.


on:
push:
branches:
- "master"
release:
types: [published, edited]

jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Login to Container Registry
uses: docker/login-action@v2
if: github.event_name != 'pull_request'
with:
registry: ${{ secrets.REGISTRY }}
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_PASSWORD }}
-
name: Set Version
run: echo "VERSION=$(tail -1 consumerui/versions.txt)" >> $GITHUB_ENV
-
name: Docker metadata
id: consumerui_meta
uses: docker/metadata-action@v4
with:
images: ${{ secrets.REGISTRY }}/${{ secrets.REGISTRY_USERNAME }}/consumerui
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=${{env.VERSION}},enable=${{ github.event_name == 'release' && github.event.action == 'published' }}

-
name: Build and push consumerui
uses: docker/build-push-action@v4
with:
context: ./consumerui
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.consumerui_meta.outputs.tags }}
91 changes: 91 additions & 0 deletions .github/workflows/build-deploy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
name: Build deploy utils

on:
push:
branches:
- "master"
release:
types: [published, edited]

jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Login to Container Registry
uses: docker/login-action@v2
with:
registry: ${{ secrets.REGISTRY }}
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_PASSWORD }}
-
name: Set Version
run: echo "VERSION=$(tail -1 deploy/versions.txt)" >> $GITHUB_ENV

-
name: Docker webhook metadata
id: webhook_tls_getter
uses: docker/metadata-action@v4
with:
images: ${{ secrets.REGISTRY }}/${{ secrets.REGISTRY_USERNAME }}/webhook-tls-getter
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=${{env.VERSION}},enable=${{ github.event_name == 'release' && github.event.action == 'published' }}
-
name: Build and push webhook-tls-getter
uses: docker/build-push-action@v4
with:
context: ./deploy
file: ./deploy/Dockerfile
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.webhook_tls_getter.outputs.tags }}

-
name: Docker delete-kubeplus-resources metadata
id: delete_kubeplus_resources
uses: docker/metadata-action@v4
with:
images: ${{ secrets.REGISTRY }}/${{ secrets.REGISTRY_USERNAME }}/delete-kubeplus-resources
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=${{env.VERSION}},enable=${{ github.event_name == 'release' && github.event.action == 'published' }}

-
name: Build and push delete-kubeplus-resources
uses: docker/build-push-action@v4
with:
context: ./deploy
file: ./deploy/Dockerfile.cleanup
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.delete_kubeplus_resources.outputs.tags }}

-
name: Docker kubeconfiggenerator metadata
id: kubeconfiggenerator
uses: docker/metadata-action@v4
with:
images: ${{ secrets.REGISTRY }}/${{ secrets.REGISTRY_USERNAME }}/kubeconfiggenerator
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=${{env.VERSION}},enable=${{ github.event_name == 'release' && github.event.action == 'published' }}

-
name: Build and push kubeconfiggenerator
uses: docker/build-push-action@v4
with:
context: ./deploy
file: ./deploy/Dockerfile.kubeconfiggenerator
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.kubeconfiggenerator.outputs.tags }}
51 changes: 51 additions & 0 deletions .github/workflows/build-mutating-webhook.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Build mutating-webhook

on:
push:
branches:
- "master"
release:
types: [published, edited]

jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Login to Container Registry
uses: docker/login-action@v2
with:
registry: ${{ secrets.REGISTRY }}
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_PASSWORD }}
-
name: Set Version
run: echo "VERSION=$(tail -1 mutating-webhook/versions.txt)" >> $GITHUB_ENV
-
name: Docker webhook metadata
id: pac_mutating_admission_webhook
uses: docker/metadata-action@v4
with:
images: ${{ secrets.REGISTRY }}/${{ secrets.REGISTRY_USERNAME }}/pac-mutating-admission-webhook
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=${{env.VERSION}},enable=${{ github.event_name == 'release' && github.event.action == 'published' }}

-
name: Build and push mutating-webhook
uses: docker/build-push-action@v4
with:
context: .
file: mutating-webhook/Dockerfile
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.pac_mutating_admission_webhook.outputs.tags }}
52 changes: 52 additions & 0 deletions .github/workflows/build-platform-helmer.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: Build helmer

on:
push:
branches:
- "master"
release:
types: [published, edited]

jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Login to Container Registry
uses: docker/login-action@v2
with:
registry: ${{ secrets.REGISTRY }}
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_PASSWORD }}
-
name: Set Helmer Version
run: echo "HELMER_VERSION=$(tail -1 platform-operator/helm-pod/versions.txt)" >> $GITHUB_ENV

-
name: Docker helm-pod metadata
id: helm_pod
uses: docker/metadata-action@v4
with:
images: ${{ secrets.REGISTRY }}/${{ secrets.REGISTRY_USERNAME }}/helm-pod
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=${{env.HELMER_VERSION}},enable=${{ github.event_name == 'release' && github.event.action == 'published' }}

-
name: Build and push helm pod
uses: docker/build-push-action@v4
with:
context: .
file: platform-operator/helm-pod/Dockerfile
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.helm_pod.outputs.tags }}
52 changes: 52 additions & 0 deletions .github/workflows/build-platform-operator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: Build platform-operator

on:
push:
branches:
- "master"
release:
types: [published, edited]

jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Login to Container Registry
uses: docker/login-action@v2
with:
registry: ${{ secrets.REGISTRY }}
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_PASSWORD }}

-
name: Set platform-operator Version
run: echo "OPERATOR_VERSION=$(tail -1 platform-operator/versions.txt)" >> $GITHUB_ENV
-
name: Docker platform-operator metadata
id: platform_operator
uses: docker/metadata-action@v4
with:
images: ${{ secrets.REGISTRY }}/${{ secrets.REGISTRY_USERNAME }}/platform-operator
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=raw,value=${{env.OPERATOR_VERSION}},enable=${{ github.event_name == 'release' && github.event.action == 'published' }}

-
name: Build and push platform-operator
uses: docker/build-push-action@v4
with:
context: ./platform-operator
file: platform-operator/artifacts/deployment/Dockerfile
platforms: linux/amd64,linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.platform_operator.outputs.tags }}
7 changes: 4 additions & 3 deletions consumerui/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,10 @@ FROM ubuntu:20.04

RUN apt-get update -y && DEBIAN_FRONTEND=noninteractive apt-get install -y tzdata && apt-get install -y python3-pip python-setuptools curl wget tar sudo apt-transport-https ca-certificates socat python-yaml vim graphviz

RUN curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

RUN cp /usr/bin/python3.8 /usr/bin/python
RUN arch=$(arch | sed s/aarch64/arm64/ | sed s/x86_64/amd64/) && \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is output of arch on a arm machine? Why do we need two sed commands?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Linux the arch command will return aarch64 for ARM and x86_64 for AMD64 machine. Most ackage are published as either arm64 or amd64 so we need to run sed to translate.

Why do we need two sed commands?
Not the or (|)

curl -LO https://dl.k8s.io/release/v1.26.0/bin/linux/${arch}/kubectl && \
install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl && \
cp /usr/bin/python3.8 /usr/bin/python

RUN wget https://github.com/cloud-ark/kubeplus/raw/master/kubeplus-kubectl-plugins.tar.gz && gunzip kubeplus-kubectl-plugins.tar.gz && tar -xvf kubeplus-kubectl-plugins.tar && cp -r /plugins/* bin/

Expand Down
2 changes: 1 addition & 1 deletion consumerui/build-artifact.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ fi
if [ "$artifacttype" = "versioned" ]; then
version=`tail -1 versions.txt`
echo "Building version $version"
docker build --no-cache -t gcr.io/cloudark-kubeplus/consumerui:$version .
docker build --no-cache -t gcr.io/cloudark-kubeplus/consumerui:$version .
docker push gcr.io/cloudark-kubeplus/consumerui:$version
fi

Expand Down
7 changes: 2 additions & 5 deletions deploy/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
FROM ubuntu:20.04
USER root
RUN apt-get update && apt-get upgrade && apt-get install -y curl openssl jq python3 python3-pip && pip3 install pyyaml
RUN apt-get update && apt-get install -y openssl curl jq python3 python3-pip && pip3 install pyyaml
ADD webhook-create-self-signed-ca-cert.sh /
RUN curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
RUN install -o root -g root -m 0755 kubectl bin/kubectl
RUN cp bin/kubectl /root/kubectl
RUN arch=$(arch | sed s/aarch64/arm64/ | sed s/x86_64/amd64/) && cd /root/ && curl -LO "https://dl.k8s.io/release/v1.26.0/bin/linux/${arch}/kubectl"
COPY kubeplus-non-pod-resources.yaml /root/.
COPY mutatingwebhook.yaml /root/.
#COPY webhook-patch-ca-bundle-new.sh /root/.
Expand Down
10 changes: 3 additions & 7 deletions deploy/Dockerfile.cleanup
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,7 @@
FROM ubuntu:20.04
USER root
ADD delete-kubeplus-components.sh /root/.
#COPY kubectl /root/
RUN apt-get update && apt-get install -y curl openssl jq python3 python3-pip && pip3 install pyyaml
#RUN cp /root/kubectl bin/. &&
RUN curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
RUN install -o root -g root -m 0755 kubectl bin/kubectl
RUN cp bin/kubectl /root/kubectl
RUN chmod +x /root/kubectl && chmod +x bin/kubectl
RUN apt-get update && apt-get install -y openssl curl jq python3 python3-pip && pip3 install pyyaml
RUN arch=$(arch | sed s/aarch64/arm64/ | sed s/x86_64/amd64/) && cd /root/ && curl -LO "https://dl.k8s.io/release/v1.26.0/bin/linux/${arch}/kubectl"
RUN cp /root/kubectl bin/. && chmod +x /root/kubectl && chmod +x bin/kubectl
ENTRYPOINT ["/root/delete-kubeplus-components.sh"]
27 changes: 8 additions & 19 deletions deploy/Dockerfile.kubeconfiggenerator
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,13 @@
FROM ubuntu:20.04
USER root
RUN apt-get update && apt-get upgrade && apt-get install -y curl openssl jq python3 python3-pip wget ca-certificates && pip3 install pyyaml
#COPY kubectl /root/
RUN apt-get update && apt-get install -y curl wget openssl jq python3 python3-pip && pip3 install pyyaml Flask
RUN arch=$(arch | sed s/aarch64/arm64/ | sed s/x86_64/amd64/) && \
wget "https://get.helm.sh/helm-v3.11.1-linux-${arch}.tar.gz" && \
tar xvf "helm-v3.11.1-linux-${arch}.tar.gz" && \
mv "linux-${arch}/helm" /root/ && rm "helm-v3.11.1-linux-${arch}.tar.gz" && \
rm -rf "linux-${arch}"/
RUN arch=$(arch | sed s/aarch64/arm64/ | sed s/x86_64/amd64/) && cd /root/ && curl -LO "https://dl.k8s.io/release/v1.26.0/bin/linux/${arch}/kubectl"
COPY kubeconfiggenerator.py /root/.
COPY kubeconfiggenerator.sh /root/.
ADD requirements.txt /root/requirements.txt
RUN cd /root; pip3 install -r requirements.txt
#RUN cp /root/kubectl bin/. && chmod +x /root/kubectl && chmod +x bin/kubectl &&
RUN curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
RUN install -o root -g root -m 0755 kubectl bin/kubectl
RUN cp bin/kubectl /root/kubectl
RUN chmod +x /root/kubectl && chmod +x bin/kubectl
RUN chmod +x /root/kubeconfiggenerator.sh
RUN wget https://get.helm.sh/helm-v3.12.1-linux-amd64.tar.gz \
&& gunzip helm-v3.12.1-linux-amd64.tar.gz \
&& tar -xvf helm-v3.12.1-linux-amd64.tar \
&& mv linux-amd64/helm bin/.


#ENTRYPOINT ["/root/kubeconfiggenerator.sh"]
RUN cp /root/kubectl bin/. && chmod +x /root/kubectl && chmod +x bin/kubectl && cp /root/helm bin/. && chmod +x /root/helm && chmod +x bin/helm

EXPOSE 5005
CMD ["python3", "/root/kubeconfiggenerator.py"]
Expand Down
Loading
Loading