Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: Simplify coverage dockerfile. #2679

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

chore: Simplify coverage dockerfile. #2679

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .github/workflows/post-submit.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,6 @@ jobs:
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build toxchat/c-toxcore:sources
uses: docker/build-push-action@v4
with:
file: other/docker/sources/sources.Dockerfile
tags: toxchat/c-toxcore:sources
- name: Build and push
uses: docker/build-push-action@v4
with:
Expand Down
8 changes: 3 additions & 5 deletions other/docker/coverage/coverage.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
FROM toxchat/c-toxcore:sources AS sources
FROM ubuntu:20.04 AS build

SHELL ["/bin/bash", "-o", "pipefail", "-c"]
Expand Down Expand Up @@ -41,10 +40,10 @@ ENV CC=clang-17 \
PYTHONUNBUFFERED=1 \
PATH=$PATH:/usr/lib/go-1.18/bin

COPY --from=sources /src/ /work/
COPY . /work/

WORKDIR /work
RUN git clone --depth=1 https://github.com/TokTok/toktok-fuzzer /work/testing/fuzzing/toktok-fuzzer
RUN ["git", "clone", "--depth=1", "https://github.com/TokTok/toktok-fuzzer", "/work/testing/fuzzing/toktok-fuzzer"]
RUN source .github/scripts/flags-coverage.sh \
&& go version \
&& (cd other/proxy && go get github.com/things-go/go-socks5 && go build proxy_server.go) \
Expand Down Expand Up @@ -76,8 +75,7 @@ RUN clang-17 -fuse-ld=lld -fPIC -shared -O2 -g3 -Wall -I/usr/lib/llvm-17/include
&& install mallocfail.so /usr/local/lib/mallocfail.so

WORKDIR /work/_build
COPY other/docker/coverage/run_mallocfail /usr/local/bin/
RUN ["run_mallocfail", "--ctest=1", "--jobs=8"]
RUN ["/work/other/docker/coverage/run_mallocfail", "--ctest=1", "--jobs=8"]
RUN llvm-profdata-17 merge -sparse $(find . -name "*.profraw") -o toxcore.profdata
RUN llvm-cov-17 show -format=text -instr-profile=toxcore.profdata -sources $(cmake --build . --target help | grep -o '[^:]*_test:' | grep -o '[^:]*' | xargs -n1 find . -type f -name | awk '{print "-object "$1}') > coverage.txt
RUN llvm-cov-17 show -format=html -instr-profile=toxcore.profdata -sources $(cmake --build . --target help | grep -o '[^:]*_test:' | grep -o '[^:]*' | xargs -n1 find . -type f -name | awk '{print "-object "$1}') -output-dir=html
Expand Down
25 changes: 25 additions & 0 deletions other/docker/coverage/coverage.Dockerfile.dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# ===== common =====
# Ignore everything ...
**/*
# ... except sources
!**/*.[ch]
!**/*.cc
!**/*.hh
!CHANGELOG.md
!LICENSE
!README.md
!auto_tests/data/*
!other/bootstrap_daemon/bash-completion/**
!other/bootstrap_daemon/tox-bootstrapd.*
!other/proxy/*.mod
!other/proxy/*.sum
!other/proxy/*.go
# ... and CMake build files (used by most builds).
!**/CMakeLists.txt
!.github/scripts/flags*.sh
!cmake/*.cmake
!other/pkgconfig/*
!other/rpm/*
!so.version
# ===== custom =====
!other/docker/coverage/run_mallocfail
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this file committed by a mistake? It's supposedly auto-generated by ../sources/run.sh when called by coverage/run.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, it's not a mistake. This file has to be in git because otherwise a default "build this docker file" action (github or render.com) doesn't work the way it should.

Copy link
Member

@nurupo nurupo Feb 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh. So, whenever someone changes either the master dockerignore file in other/docker/sources/, or a supplemental one in other/docker/$BUILD/, they would have to run the other/$BUILD/run script(s) to update the concatenated dockerignore file(s) before committing/making a PR?

Maybe we could add a script step to the GitHub CI before the "build this docker file" action that would generate this dockerignore file? For example, move the dockerignore concat section out of other/docker/sources/ru.sh into its own bash script, something like other/docker/source/gen-dockerignore.sh that takes $BUILD as an argument (or a path to $BUILD's directory or something like that) and concats the master dockerignore with the $BUILD's, putting it in $BUILD's directory? That script could be used in CI to generate the dockerignore file before calling the docker actions and will also be called from other/docker/sources/run.sh.

(Also, what is this render.com promotion? How is that relevant to us?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'll do that, but it'll take some time for me to find motivation for it, so I'll move this PR to the next milestone.

2 changes: 2 additions & 0 deletions other/docker/coverage/dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# ===== custom =====
!other/docker/coverage/run_mallocfail
7 changes: 2 additions & 5 deletions other/docker/coverage/run
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,5 @@ set -eux

read -a ci_env <<<"$(bash <(curl -s https://codecov.io/env))"

BUILD=coverage
other/docker/sources/build
docker build -t "toxchat/c-toxcore:$BUILD" -f "other/docker/$BUILD/$BUILD.Dockerfile" .

docker run "${ci_env[@]}" -e CI=true --name toxcore-coverage --rm -t toxchat/c-toxcore:coverage /usr/local/bin/codecov
. "$(realpath -- "$(dirname "${BASH_SOURCE[0]}")/../sources/run.sh")"
docker run "${ci_env[@]}" -e CI=true --name "toxcore-$BUILD" --rm -t "toxchat/c-toxcore:$BUILD" /usr/local/bin/codecov
8 changes: 3 additions & 5 deletions other/docker/coverage/serve
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,7 @@

set -eux

BUILD=coverage
other/docker/sources/build
docker build -t "toxchat/c-toxcore:$BUILD" -f "other/docker/$BUILD/$BUILD.Dockerfile" .
. "$(realpath -- "$(dirname "${BASH_SOURCE[0]}")/../sources/run.sh")"

docker build -t toxchat/c-toxcore:coverage-nginx -f other/docker/coverage/nginx.Dockerfile other/docker/coverage
docker run --name toxcore-coverage --rm -it -p "28192:80" toxchat/c-toxcore:coverage-nginx
docker build -t "toxchat/c-toxcore:$BUILD-nginx" -f "$DOCKERDIR/nginx.Dockerfile" "$DOCKERDIR"
docker run --name "toxcore-$BUILD" --rm -it -p "28192:80" "toxchat/c-toxcore:$BUILD-nginx"
2 changes: 1 addition & 1 deletion other/docker/doxygen/run
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/bin/env bash

. "$(cd "$(dirname "${BASH_SOURCE[0]}")/../sources" && pwd)/run.sh"
. "$(realpath -- "$(dirname "${BASH_SOURCE[0]}")/../sources/run.sh")"
docker run --name toxcore-doxygen --rm -it -p "28192:80" "toxchat/c-toxcore:$BUILD"
Loading