Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixing buildlibs.sh script + cmark reference in config.nims + prebuilt arm linux libs #113

Merged
merged 4 commits into from
Sep 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/Dockerfile.staticlibs
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
FROM ghcr.io/crashappsec/nim:ubuntu-2.0.0 as nim

# https://github.com/actions/runner/issues/2033
RUN if which git; then git config --global --add safe.directory "*"; fi

# deps for compiling static deps
RUN apt-get update -y && \
apt-get install -y \
autoconf \
cmake \
file \
g++ \
gcc \
git \
m4 \
make \
&& \
apt-get clean -y

WORKDIR /con4m
9 changes: 9 additions & 0 deletions .github/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
services:
staticlibs:
build:
context: ..
dockerfile: .github/Dockerfile.staticlibs
working_dir: /con4m
volumes:
- ..:/con4m
- $HOME/.local/c0:/root/.local/c0
51 changes: 51 additions & 0 deletions .github/workflows/staticlibs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: staticlibs

on:
workflow_dispatch:
inputs:

jobs:
buildlibs:
runs-on: ${{ matrix.builder }}

strategy:
matrix:
include:
- builder: ubuntu-latest
arch: amd64
os: linux
- builder: buildjet-2vcpu-ubuntu-2204-arm
arch: arm64
os: linux

steps:
- name: Checkout Code
uses: actions/checkout@v3

- name: Show Static Libs
run: |
set -x
ls -la ~/.local/c0/libs/* || true
ls -la files/deps/lib/* || true

- name: Clean up existing libs
run: |
rm -rf files/deps/lib/*

- name: Build Static Libs
working-directory: ./.github
run: |
docker compose run --rm staticlibs nimble build

- name: Show Static Libs
run: |
set -x
ls -la ~/.local/c0/libs/* || true
ls -la files/deps/lib/* || true

- name: Save Artifact
uses: actions/upload-artifact@v3
if: always()
with:
name: ${{ matrix.os }}-${{ matrix.arch }}
path: ~/.local/c0/libs/*
3 changes: 1 addition & 2 deletions config.nims
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,7 @@ var
default = getEnvDir("HOME") & ".local/c0"
localDir = getEnvDir("LOCAL_INSTALL_DIR", default)
libDir = localdir & "libs"
libs = ["pcre", "ssl", "crypto", "cmark-gfm", "cmark-gfm-extensions",
"gumbo"]
libs = ["pcre", "ssl", "crypto", "gumbo"]

when defined(linux):
var
Expand Down
11 changes: 10 additions & 1 deletion files/bin/buildlibs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,15 @@ function ensure_musl {
if [[ ${OS} = "macosx" ]] ; then
return
fi
if [[ ! -f ${MUSL_GCC} ]] ; then
# if musl-gcc is already installed, use it
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love this, they might not have permission to install the headers. Things can go wrong. They can delete the thing after they build it, and indeed the script does.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait this was merged???

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

vanilla apt-get install musl-tools installs correct musl and I am able to use it in chalk docker builds

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure what you mean if it was merged. PR is still open

existing_musl=$(which musl-gcc 2> /dev/null)
if [[ -n "${existing_musl}" ]]; then
Comment on lines +143 to +145
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if musl-gcc is already installed, we dont need to compile musl from scratch which requires more system deps like make. for example avoids errors like:

#14 15.44 /tmp/nimble_9/githubcom_crashappseccon4mgit_#ms/libs/files/bin/buildlibs.sh: line 156: make: command not found

mkdir -p $(dirname ${MUSL_GCC})
ln -s ${existing_musl} ${MUSL_GCC}
echo $(color GREEN Linking existing musl-gcc: ) ${existing_musl} $(color GREEN "->" ) ${MUSL_GCC}
fi
fi
if [[ ! -f ${MUSL_GCC} ]] ; then
get_src musl git://git.musl-libc.org/musl
colorln CYAN Building musl
Expand Down Expand Up @@ -220,7 +229,7 @@ function ensure_gumbo {
ensure_musl
get_src sigil-gumbo https://github.com/Sigil-Ebook/sigil-gumbo/
colorln CYAN "Watching our waistline, selecting only required gumbo ingredients..."
cat > CMakelists.txt <<EOL
cat > CMakeLists.txt <<EOL
cmake_minimum_required( VERSION 3.0 )

project(gumbo)
Expand Down
Binary file added files/deps/lib/linux-arm64/libc.a
Binary file not shown.
1 change: 1 addition & 0 deletions files/deps/lib/linux-arm64/libcrypt.a
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
!<arch>
Binary file added files/deps/lib/linux-arm64/libcrypto.a
Binary file not shown.
1 change: 1 addition & 0 deletions files/deps/lib/linux-arm64/libdl.a
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
!<arch>
Binary file added files/deps/lib/linux-arm64/libgumbo.a
Binary file not shown.
1 change: 1 addition & 0 deletions files/deps/lib/linux-arm64/libm.a
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
!<arch>
Binary file added files/deps/lib/linux-arm64/libpcre.a
Binary file not shown.
1 change: 1 addition & 0 deletions files/deps/lib/linux-arm64/libpthread.a
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
!<arch>
1 change: 1 addition & 0 deletions files/deps/lib/linux-arm64/libresolv.a
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
!<arch>
1 change: 1 addition & 0 deletions files/deps/lib/linux-arm64/librt.a
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
!<arch>
Binary file added files/deps/lib/linux-arm64/libssl.a
Binary file not shown.
1 change: 1 addition & 0 deletions files/deps/lib/linux-arm64/libutil.a
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
!<arch>
1 change: 1 addition & 0 deletions files/deps/lib/linux-arm64/libxnet.a
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
!<arch>