Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix docker build #357

Merged
merged 31 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
0b4839c
fix docker build
robkooper Sep 6, 2022
658e617
lowercase repo
robkooper Sep 6, 2022
504e593
more lowercase
robkooper Sep 6, 2022
ff348f9
no m1 builds
robkooper Sep 6, 2022
cebcba6
strip extra ed2
robkooper Sep 6, 2022
59d3de7
lower case M for model
robkooper Sep 6, 2022
ea4cf24
push to edmodel
robkooper Sep 6, 2022
15c8765
Merge branch 'master' into fix-docker
robkooper Oct 16, 2022
83b7397
Merge remote-tracking branch 'origin' into fix-docker
robkooper Mar 27, 2023
9b22a0f
compile with intel/gnu
robkooper Mar 28, 2023
98e82fb
fix spacing + ubuntu-20.04
robkooper Mar 28, 2023
0a98e73
only build on push main/master
robkooper Mar 28, 2023
4e6e37f
more build fixes
robkooper Mar 28, 2023
55867ec
typo
robkooper Mar 28, 2023
34b77a2
bedtime
robkooper Mar 28, 2023
75dcd12
update matrix
robkooper Mar 28, 2023
a1153ae
had 2 gnu
robkooper Mar 28, 2023
fcb30f7
add dev images
robkooper May 26, 2023
822f7b4
fix dev images
robkooper May 26, 2023
6afa51b
update dockerhub readme pusher
robkooper Jun 15, 2023
d2d7eb2
debug disk usage
robkooper Jun 15, 2023
e3d2e53
more DF
robkooper Jun 15, 2023
ec99fa6
use ubuntu 22.04
robkooper Mar 20, 2024
aef8777
Merge remote-tracking branch 'origin/master' into fix-docker
robkooper Apr 4, 2024
655e23f
fix out of diskspace
robkooper Apr 4, 2024
94a73d4
Merge remote-tracking branch 'origin/master' into fix-docker
robkooper Jul 25, 2024
5ac48f7
update docker
robkooper Jul 26, 2024
c75e9a9
free diskspace
robkooper Aug 15, 2024
25ba531
use intel compiler 2023.1
robkooper Aug 18, 2024
4a4a541
fix FROM
robkooper Aug 18, 2024
38cf674
update versions
robkooper Aug 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
name: CI

on: [push, pull_request]
on:
push:
branches:
- master
- main

pull_request:

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v4
- name: Install dependencies
run: sudo apt-get install -y --no-install-recommends gfortran libhdf5-openmpi-dev libopenmpi-dev
- name: Install ED2
Expand All @@ -23,7 +29,7 @@ jobs:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v4
- name: Retrieve compiled binary
uses: actions/download-artifact@v1
with:
Expand All @@ -40,7 +46,7 @@ jobs:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v4
- name: Retrieve compiled binary
uses: actions/download-artifact@v1
with:
Expand All @@ -57,7 +63,7 @@ jobs:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/checkout@v4
- name: Retrieve compiled binary
uses: actions/download-artifact@v1
with:
Expand Down
220 changes: 166 additions & 54 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: Docker

# This will run when:
# - when new code is pushed to master to push the tags latest.
# - when new code is pushed to master/main to push the tags latest.
# - when a pull request is created and updated to make sure the
# Dockerfile is still valid.

Expand All @@ -19,77 +19,189 @@ on:
push:
branches:
- master
- main

release:
types:
- published

pull_request:

# Certain actions will only run when this is the master repo.
# Certain actions will only run when this is the master/main repo.
env:
MASTER_REPO: EDModel/ED2
DOCKERHUB_ORG: pecan
MAIN_REPO: EDmodel/ED2
DOCKERHUB_ORG: edmodel
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, though we should probably acknowledge that this code is based on PEcAn somewhere.


jobs:
docker:
runs-on: ubuntu-latest
permissions:
packages: write

strategy:
fail-fast: false
matrix:
include:
- name: gnu
PLATFORM: "linux/amd64,linux/arm64"
- name: intel
PLATFORM: "linux/amd64"

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

# free up space
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@main
with:
tool-cache: true
android: true
dotnet: true
haskell: true
large-packages: true
docker-images: false
swap-storage: true

# calculate some variables that are used later
- name: github branch
- name: version information
run: |
BRANCH=${GITHUB_REF##*/}
if [ "$BRANCH" == "master" ]; then
TAGS="latest"
# find out what the BRANCH is, in case of a PR we will use the PR-<number>
if [ "${{ github.event.release.target_commitish }}" != "" ]; then
BRANCH="${{ github.event.release.target_commitish }}"
elif [[ $GITHUB_REF =~ pull ]]; then
BRANCH="$(echo $GITHUB_REF | sed 's#refs/pull/\([0-9]*\)/merge#PR-\1#')"
else
TAGS="$BRANCH"
BRANCH=${GITHUB_REF##*/}
fi
echo "TAGS=${TAGS}" >> $GITHUB_ENV
echo "GITHUB_BRANCH=${BRANCH}" >> $GITHUB_ENV

# build the docker image, this will always run to make sure
# the Dockerfile still works.
- name: Build image
uses: elgohr/[email protected]
env:
BRANCH: ${{ env.GITHUB_BRANCH }}
BUILDNUMBER: ${{ github.run_number }}
GITSHA1: ${{ github.sha }}
# calculate the version and all tags
if [ "$BRANCH" == "main" -o "$BRANCH" == "master" ]; then
VERSION="latest"
tags="${{ matrix.name }}"
if [ "${{ matrix.name }}" == "intel" ]; then
tags="${tags} latest"
fi
else
VERSION="${{ matrix.name }}-$BRANCH"
tags="${{ matrix.name }}-$BRANCH"
fi

# should we push to dockerhub, and is there a README
DOCKERHUB_PUSH="false"
DOCKERHUB_README="false"
if [ "${{ github.repository }}" == "${{ env.MAIN_REPO }}" ]; then
if [ "${{ secrets.DOCKERHUB_USERNAME }}" != "" -a "${{ secrets.DOCKERHUB_PASSWORD }}" != "" ]; then
DOCKERHUB_PUSH="true"
if [ -e "README.md" ]; then
DOCKERHUB_README="true"
fi
fi
fi

# create a list of all images to be pushed
REPO="${{ github.repository_owner }}"
REPO="${REPO,,}"
IMAGE="${{ github.event.repository.name }}"
IMAGE="${IMAGE,,}"
DEV_IMAGES=""
IMAGES=""
for tag in ${tags}; do
if [ "$DOCKERHUB_PUSH" == "true" ]; then
DEV_IMAGES="${DEV_IMAGES}${{ env.DOCKERHUB_ORG }}/${IMAGE}-dev:${tag},"
IMAGES="${IMAGES}${{ env.DOCKERHUB_ORG }}/${IMAGE}:${tag},"
fi
DEV_IMAGES="${DEV_IMAGES}ghcr.io/${REPO}/${IMAGE}-dev:${tag},"
IMAGES="${IMAGES}ghcr.io/${REPO}/${IMAGE}:${tag},"
done
IMAGES="${IMAGES%,*}"

# save the results in env
echo "BRANCH=${BRANCH}"
echo "VERSION=${VERSION}"
echo "DOCKERHUB_README=${DOCKERHUB_README}"
echo "DOCKERHUB_PUSH=${DOCKERHUB_PUSH}"
echo "IMAGES=${IMAGES}"

echo "BRANCH=${BRANCH}" >> $GITHUB_ENV
echo "VERSION=${VERSION}" >> $GITHUB_ENV
echo "DOCKERHUB_README=${DOCKERHUB_README}" >> $GITHUB_ENV
echo "DOCKERHUB_PUSH=${DOCKERHUB_PUSH}" >> $GITHUB_ENV
echo "DEV_IMAGES=${DEV_IMAGES}" >> $GITHUB_ENV
echo "IMAGES=${IMAGES}" >> $GITHUB_ENV

# setup docker build
- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2

- name: Inspect Builder
run: |
echo "Name: ${{ steps.buildx.outputs.name }}"
echo "Endpoint: ${{ steps.buildx.outputs.endpoint }}"
echo "Status: ${{ steps.buildx.outputs.status }}"
echo "Flags: ${{ steps.buildx.outputs.flags }}"
echo "Platforms: ${{ steps.buildx.outputs.platforms }}"

# login to registries
- name: Login to DockerHub
if: env.DOCKERHUB_PUSH == 'true'
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}

- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
registry: docker.pkg.github.com
name: ${{ github.repository_owner }}/${{ github.event.repository.name }}/ed
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
tags: "${{ env.TAGS }}"
buildargs: BRANCH,BUILDNUMBER,GITSHA1
no_push: true

# this will publish to github container registry
- name: Publish to GitHub
if: github.event_name == 'push' && github.repository == env.MASTER_REPO
uses: elgohr/[email protected]
env:
BRANCH: ${{ env.GITHUB_BRANCH }}
BUILDNUMBER: ${{ github.run_number }}
GITSHA1: ${{ github.sha }}

- name: DF
run: df -h .

# build the dev docker images
- name: Build and push docker
uses: docker/build-push-action@v4
with:
registry: ghcr.io
name: ${{ github.repository_owner }}/ed
username: ${{ secrets.GHCR_USERNAME }}
password: ${{ secrets.GHCR_PASSWORD }}
tags: "${{ env.TAGS }}"
buildargs: BRANCH,BUILDNUMBER,GITSHA1

# this will publish to the pecan dockerhub repo
- name: Publish to Docker Hub
if: github.event_name == 'push' && github.repository == env.MASTER_REPO
uses: elgohr/[email protected]
env:
BRANCH: ${{ env.GITHUB_BRANCH }}
BUILDNUMBER: ${{ github.run_number }}
GITSHA1: ${{ github.sha }}
push: true
platforms: ${{ matrix.PLATFORM }}
file: Dockerfile.${{ matrix.name }}
target: build
cache-from: type=gha,scope=ed-${{ matrix.name }}-build
cache-to: type=gha,scope=ed-${{ matrix.name }}-build,mode=max
tags: ${{ env.DEV_IMAGES }}
build-args: |
BRANCH: ${{ env.BRANCH }}
VERSION=${{ env.VERSION }}
BUILDNUMBER=${{ github.run_number }}
GITSHA1=${{ github.sha }}

# build the docker images
- name: Build and push docker
uses: docker/build-push-action@v4
with:
name: ${{ env.DOCKERHUB_ORG }}/ed
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
tags: "${{ env.TAGS }}"
buildargs: BRANCH,BUILDNUMBER,GITSHA1
push: true
platforms: ${{ matrix.PLATFORM }}
file: Dockerfile.${{ matrix.name }}
cache-from: type=gha,scope=ed-${{ matrix.name }}-build
cache-to: type=gha,scope=ed-${{ matrix.name }}-build,mode=max
tags: ${{ env.IMAGES }}
build-args: |
BRANCH: ${{ env.BRANCH }}
VERSION=${{ env.VERSION }}
BUILDNUMBER=${{ github.run_number }}
GITSHA1=${{ github.sha }}

# this will update the README of the dockerhub repo
- name: Docker Hub Description
if: env.DOCKERHUB_README == 'true'
uses: peter-evans/dockerhub-description@v3
env:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_PASSWORD: ${{ secrets.DOCKERHUB_PASSWORD }}
DOCKERHUB_REPOSITORY: ${{ env.DOCKERHUB_ORG }}/${{ github.event.repository.name }}
README_FILEPATH: README.md
55 changes: 0 additions & 55 deletions Dockerfile

This file was deleted.

Loading
Loading