Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[kube] use local image for kubernetes integration tests #48321

Merged
merged 7 commits into from
Nov 4, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/workflows/kube-integration-tests-non-root.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ on:
env:
TEST_KUBE: true
KUBECONFIG: /home/.kube/config
ALPINE_VERSION: 3.20.3

jobs:
changes:
Expand Down Expand Up @@ -89,6 +90,20 @@ jobs:
cp -r $HOME/.kube /home/
chown -R ci:ci /home/.kube

- name: Build Alpine image with webserver
run: |
# verify the checksum
cd ./fixtures/alpine/ && sha256sum -c alpine-minirootfs-$ALPINE_VERSION-x86_64.tar.gz.sha256 && cd -

# verify the signature
gpg --import ./fixtures/alpine/alpine-ncopa.at.alpinelinux.org.asc
gpg --verify ./fixtures/alpine/alpine-minirootfs-$ALPINE_VERSION-x86_64.tar.gz.asc ./fixtures/alpine/alpine-minirootfs-$ALPINE_VERSION-x86_64.tar.gz

CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o ./fixtures/alpine/webserver ./fixtures/alpine/webserver.go
docker build -t alpine-webserver:v1 --build-arg=ALPINE_VERSION=$ALPINE_VERSION -f ./fixtures/alpine/Dockerfile ./fixtures/alpine/
kind load docker-image alpine-webserver:v1
rm -f ./fixtures/alpine/webserver

- name: Run tests
timeout-minutes: 40
run: |
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ default.etcd

# usually release tarballs get in the way
*.gz
# ignore all tarballs except the alpine one
!fixtures/alpine/alpine-minirootfs-*.tar.gz
*.zip

# editors
Expand Down
9 changes: 9 additions & 0 deletions fixtures/alpine/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM scratch

ARG ALPINE_VERSION

ADD alpine-minirootfs-$ALPINE_VERSION-x86_64.tar.gz /

COPY webserver /webserver

CMD [ "/webserver" ]
56 changes: 56 additions & 0 deletions fixtures/alpine/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# `alpine-webserver:v1` Build Process

## Source

The `alpine-webserver:v1` image is based on `alpine` `minirootfs`, but instead of relying on Docker Hub's official Alpine image, we source the original files directly from Alpine's CDN. This approach mitigates issues with Docker Hub and GitHub Action network failures, which have been a common cause of integration test failures.

The build process is specified in the `.github/workflows/kube-integration-tests-non-root.yaml` file.

## Download

To download the new `alpine-minirootfs` image, follow the instructions:

Define the desired Alpine version:

```bash
$ export ALPINE_VERSION=3.20.3
```

Download the asset files: filesystem, signature and sha256.

```bash

tigrato marked this conversation as resolved.
Show resolved Hide resolved
# download alpine minirootfs and signature
$ curl -fSsL -o ./fixtures/alpine/alpine-minirootfs-$ALPINE_VERSION-x86_64.tar.gz https://dl-cdn.alpinelinux.org/alpine/v3.20/releases/x86_64/alpine-minirootfs-$ALPINE_VERSION-x86_64.tar.gz
$ curl -fSsL -o ./fixtures/alpine/alpine-minirootfs-$ALPINE_VERSION-x86_64.tar.gz.asc https://dl-cdn.alpinelinux.org/alpine/v3.20/releases/x86_64/alpine-minirootfs-$ALPINE_VERSION-x86_64.tar.gz.asc
$ curl -fSsL -o ./fixtures/alpine/alpine-minirootfs-$ALPINE_VERSION-x86_64.tar.gz.sha256 https://dl-cdn.alpinelinux.org/alpine/v3.20/releases/x86_64/alpine-minirootfs-$ALPINE_VERSION-x86_64.tar.gz.sha256
tigrato marked this conversation as resolved.
Show resolved Hide resolved

```

## Source Validation

The build process in `.github/workflows/kube-integration-tests-non-root.yaml` validates both the SHA-256 checksum and the GPG signature. The signature verification uses `alpine-ncopa.at.alpinelinux.org.asc`, which is the official public key used by Alpine Linux to sign its assets. This public key is available on the [Alpine Linux Downloads page](https://www.alpinelinux.org/downloads/).

## Image Build Process

The image is constructed from a scratch filesystem and incorporates only the necessary components to run the web server. Here’s the basic Dockerfile configuration:

```Dockerfile
FROM scratch

ARG ALPINE_VERSION

ADD alpine-minirootfs-$ALPINE_VERSION-x86_64.tar.gz /

COPY webserver /webserver

CMD [ "/webserver" ]
```

This minimalist configuration ensures the image remains lightweight, secure, and tailored to only the required functionalities.

## Image distribution

After sucessfull build, the image is loaded into our `kind` cluster with the tag `alpine-webserver:v1`.

Note: `:latest` can't be used otherwise Kubernetes will try loading the image from dockerhub and fail.
Binary file not shown.
16 changes: 16 additions & 0 deletions fixtures/alpine/alpine-minirootfs-3.20.3-x86_64.tar.gz.asc
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEBILYQCL1LfHE581DKTrNCQfZSVoFAmba9QIACgkQKTrNCQfZ
SVroKRAAsqQ3f8KwCvZizz53MA7pm9PWZJPYyjNIDNzwEHZFnkfrO/zTySbkPQp2
D1PHrGdhkAX8dJaKOq/d4T3Dr5UOuARZeJSsy287eW2wj07mN6YkleDL4utebrfq
IHGvaXEGQJKxreRritX4BfA1FD+BiveV+2S1O4NTbkifdj4WbT1gxp+ZKp53T9bU
UP9+8V9GRGHotdxsB5dd3YbGNkS4JGWGHp2h6ih9A1au9C7GRUFpQ2nk2cbkycDu
EA0SA8Tam8QNaBZHBWQxgmEw2iqfZPxIAqc188GzXeO8GWGjXILR2t4wOEEEmRa3
g8REzQ8xpVOs/6y2woolNZWYFi5lCc1Do+KxRbdyEP3P1scz5D2FYfA8I6i/iwk9
TR155MItfdB7nPDTvkR6p0xpI5H1V6sSKKk2EricwyvTKVoTBYa19h8pIA7fOzqQ
h32qlxW0Xfn96VZUvlsxCuPlJT8nAkxE1uotmJPb5NIuLo2SLgTB4eHKhvnryi0c
WYtrogEC5rCF+btCU3/7DcMd7BJnPW3otUQL/+iAxWlCQ4YoKx9qltC1O8qres5p
9hbF7Wv88aEuqfoi1JZDaAACeITAKLEMWaiT9pCXLJ0EeghE2y5yfC7kTh4+faS6
+cGoY1+Zmvp6o0ENG+GURkn7jx6WJ3MC9zaEgmfYbovPov4iBLg=
=+T7J
-----END PGP SIGNATURE-----
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
d4e6fd67dcf75e40c451560ac7265166c2b72a0f38ddc9aae756a7de3d1efa0c alpine-minirootfs-3.20.3-x86_64.tar.gz
52 changes: 52 additions & 0 deletions fixtures/alpine/alpine-ncopa.at.alpinelinux.org.asc
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v2

mQINBFSIEDwBEADbib88gv1dBgeEez1TIh6A5lAzRl02JrdtYkDoPr5lQGYv0qKP
lWpd3jgGe8n90krGmT9W2nooRdyZjZ6UPbhYSJ+tub6VuKcrtwROXP2gNNqJA5j3
vkXQ40725CVig7I3YCpzjsKRStwegZAelB8ZyC4zb15J7YvTVkd6qa/uuh8H21X2
h/7IZJz50CMxyz8vkdyP2niIGZ4fPi0cVtsg8l4phbNJ5PwFOLMYl0b5geKMviyR
MxxQ33iNa9X+RcWeR751IQfax6xNcbOrxNRzfzm77fY4KzBezcnqJFnrl/p8qgBq
GHKmrrcjv2MF7dCWHGAPm1/vdPPjUpOcEOH4uGvX7P4w2qQ0WLBTDDO47/BiuY9A
DIwEF1afNXiJke4fmjDYMKA+HrnhocvI48VIX5C5+C5aJOKwN2EOpdXSvmsysTSt
gIc4ffcaYugfAIEn7ZdgcYmTlbIphHmOmOgt89J+6Kf9X6mVRmumI3cZWetf2FEV
fS9v24C2c8NRw3LESoDT0iiWsCHcsixCYqqvjzJBJ0TSEIVCZepOOBp8lfMl4YEZ
BVMzOx558LzbF2eR/XEsr3AX7Ga1jDu2N5WzIOa0YvJl1xcQxc0RZumaMlZ81dV/
uu8G2+HTrJMZK933ov3pbxaZ38/CbCA90SBk5xqVqtTNAHpIkdGj90v2lwARAQAB
tCVOYXRhbmFlbCBDb3BhIDxuY29wYUBhbHBpbmVsaW51eC5vcmc+iQI2BBMBCAAg
BQJUiBA8AhsDBQsJCAcCBhUICQoLAgMWAgECHgECF4AACgkQKTrNCQfZSVrcNxAA
mEzX9PQaczzlPAlDe3m1AN0lP6E/1pYWLBGs6qGh18cWxdjyOWsO47nA1P+cTGSS
AYe4kIOIx9kp2SxObdKeZTuZCBdWfQu/cuRE12ugQQFERlpwVRNd6NYuT3WyZ7v8
ZXRw4f33FIt4CSrW1/AyM/vrA+tWNo7bbwr/CFaIcL8kINPccdFOpWh14erONd/P
Eb3gO81yXIA6c1Vl4mce2JS0hd6EFohxS5yMQJMRIS/Zg8ufT3yHJXIaSnG+KRP7
WWLR0ZaLraCykYi/EW9mmQ49LxQqvKOgjpRW9aNgDA+arKl1umjplkAFI1GZ0/qA
sgKm4agdvLGZiCZqDXcRWNolG5PeOUUpim1f59pGnupZ3Rbz4BF84U+1uL+yd0OR
5Y98AxWFyq0dqKz/zFYwQkMVnl9yW0pkJmP7r6PKj0bhWksQX+RjYPosj3wxPZ7i
SKMX7xZaqon/CHpH9/Xm8CabGcDITrS6h+h8x0FFT/MV/LKgc3q8E4mlXelew1Rt
xK4hzXFpXKl0WcQg54fj1Wqy47FlkArG50di0utCBGlmVZQA8nqE5oYkFLppiFXz
1SXCXojff/XZdNF2WdgV8aDKOYTK1WDPUSLmqY+ofOkQL49YqZ9M5FR8hMAbvL6e
4CbxVXCkWJ6Q9Lg79AzS3pvOXCJ/CUDQs7B30v026Ba5Ag0EVIgQPAEQAMHuPAv/
B0KP9SEA1PsX5+37k46lTP7lv7VFd7VaD1rAUM/ZyD2fWgrJprcCPEpdMfuszfOH
jGVQ708VQ+vlD3vFoOZE+KgeKnzDG9FzYXXPmxkWzEEqI168ameF/LQhN12VF1mq
5LbukiAKx2ytb1I8onvCvNJDvH1D/3BxSj7ThV9bP/bFufcOHFBMFwtyBmUaR5Wx
96Bq+7DEbTrxhshoQgUqILEudUyhZa05/TrpUvC4f8qc0deaqJFO1zD6guZxRWZd
SWJdcFzTadyg36P4eyFMxa1Ft7BlDKdKLAFlCGgR0jfOnKRmdRKGRNFTLQ68aBld
N4wxBuMwe0tmRw9zYwWwD43Aq9E26YtuxVR1wb3zUmi+47QH4ANAzMioimE9Mj5S
qYrgzQJ0IGwIjBt+HNzHvYX+kyMuVFK41k2Vo6oUOVHuQMu3UgLvSPMsyw69d+Iw
K/rrsQwuutrvJ8Qcda3rea1HvWBVcY/uyoRsOsCS7itS6MK6KKTKaW8iskmEb2/h
Q1ZB1QaWm2sQ8Xcmb3QZgtyBfZKuC95T/mAXPT0uET6bTpP5DdEi3wFs+qw/c9FZ
SNDZ4hfNuS24d2u3Rh8LWt/U83ieAutNntOLGhvuZm1jLYt2KvzXE8cLt3V75/ZF
O+xEV7rLuOtrHKWlzgJQzsDp1gM4Tz9ULeY7ABEBAAGJAh8EGAEIAAkFAlSIEDwC
GwwACgkQKTrNCQfZSVrIgBAArhCdo3ItpuEKWcxx22oMwDm+0dmXmzqcPnB8y9Tf
NcocToIXP47H1+XEenZdTYZJOrdqzrK6Y1PplwQv6hqFToypgbQTeknrZ8SCDyEK
cU4id2r73THTzgNSiC4QAE214i5kKd6PMQn7XYVjsxvin3ZalS2x4m8UFal2C9nj
o8HqoTsDOSRy0mzoqAqXmeAe3X9pYme/CUwA6R8hHEgX7jUhm/ArVW5wZboAinw5
BmKBjWiIwT1vxfvwgbC0EA1O24G4zQqEJ2ILmcM3RvWwtFFWasQqV7qnKdpD8EIb
oPa8Ocl7joDc5seK8BzsI7tXN4Yjw0aHCOlZ15fWHPYKgDFRQaRFffODPNbxQNiz
Yru3pbEWDLIUoQtJyKl+o2+8m4aWCYNzJ1WkEQje9RaBpHNDcyen5yC73tCEJsvT
ZuMI4Xqc4xgLt8woreKE57GRdg2fO8fO40X3R/J5YM6SqG7y2uwjVCHFBeO2Nkkr
8nOno+Rbn2b03c9MapMT4ll8jJds4xwhhpIjzPLWd2ZcX/ZGqmsnKPiroe9p1VPo
lN72Ohr9lS+OXfvOPV2N+Ar5rCObmhnYbXGgU/qyhk1qkRu+w2bBZOOQIdaCfh5A
Hbn3ZGGGQskgWZDFP4xZ3DWXFSWMPuvEjbmUn2xrh9oYsjsOGy9tyBFFySU2vyZP
Mkc=
=FcYC
-----END PGP PUBLIC KEY BLOCK-----
26 changes: 26 additions & 0 deletions fixtures/alpine/webserver.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
// Teleport
// Copyright (C) 2024 Gravitational, Inc.
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.

package main

import "net/http"

func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello, world!"))
})
http.ListenAndServe(":80", nil)
}
13 changes: 10 additions & 3 deletions integration/kube_integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1467,7 +1467,7 @@ func testKubeEphemeralContainers(t *testing.T, suite *KubeSuite) {
sessCreatorTerm := NewTerminal(250)
group := &errgroup.Group{}
group.Go(func() error {
cmd := []string{"/bin/sh", "echo", "hello from an ephemeral container"}
cmd := []string{"/bin/sh", "-c", "echo hello from an ephemeral container"}
debugPod, _, err := generateDebugContainer(contName, cmd, pod)
if err != nil {
return trace.Wrap(err)
Expand Down Expand Up @@ -1572,7 +1572,7 @@ func generateDebugContainer(name string, cmd []string, pod *v1.Pod) (*v1.Pod, *v
ec := &v1.EphemeralContainer{
EphemeralContainerCommon: v1.EphemeralContainerCommon{
Name: name,
Image: "alpine:latest",
Image: localPodImage,
Command: cmd,
ImagePullPolicy: v1.PullIfNotPresent,
Stdin: true,
Expand Down Expand Up @@ -1928,6 +1928,13 @@ func newNamespace(name string) *v1.Namespace {
}
}

// localPodImage is a container image that is used for testing
// It's a docker image that runs a simple web server
// that listens on port 80 and returns "Hello, World!" on GET /
// This image is vendored in the Teleport repository in the
// fixtures/alpine directory. Check the Dockerfile there for details.
const localPodImage = "alpine-webserver:v1"
tigrato marked this conversation as resolved.
Show resolved Hide resolved

func newPod(ns, name string) *v1.Pod {
return &v1.Pod{
ObjectMeta: metav1.ObjectMeta{
Expand All @@ -1937,7 +1944,7 @@ func newPod(ns, name string) *v1.Pod {
Spec: v1.PodSpec{
Containers: []v1.Container{{
Name: "nginx",
Image: "nginx:alpine",
Image: localPodImage,
}},
},
}
Expand Down
Loading