Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub docker testing #7571

Merged
merged 2 commits into from
Aug 21, 2024
Merged

GitHub docker testing #7571

merged 2 commits into from
Aug 21, 2024

Conversation

ShahanaFarooqui
Copy link
Collaborator

@ShahanaFarooqui ShahanaFarooqui commented Aug 14, 2024

  • Temporarily adding rc tag trigger, some input variables & more logs for testing Build and push multi-platform docker images action flow before the final release.

  • Poetry on arm32: The Docker build for linux/arm/v7 failed in recent RC releases on the Poetry installation step in the builder-python stage. This issue occurred because the builder-python stage builds on target's arch but poetry was unable to install on arm/v7 without rust >= v1.56.1.

    • Solution: Instead of installing poetry on the builder-python stage, we leveraged the existing multi-arch builder stage, which already had Poetry. Now, we export the dependencies from pyproject.toml to requirements.txt within the builder stage and then copy requirements.txt to the builder-python stage for pip installation.
  • Cryptography installation: python installations for pyln-proto started failing due to Cryptography upgrade from v41 to v42 (PR pyln-proto: update cryptography to v42 #7475). It is because now Cryptography needs cargo/rust also.

    • Solution: Installing cargo in builder-python stage also.
  • Configure Prefix: Previously, we used RUN ./configure --prefix=/tmp/lightning_install --enable-static in the builder image and then copied /tmp/lightning_install from the builder stage to /usr/local in the final stage. However, this approach is now causing errors due to missing binaries/plugins at their default locations.

    • Solution: We are now configuring the installation to use the default location (/usr/local). To prevent the local image size from increasing by up to 87MB, instead of copying the entire /usr/local/ directory, we are explicitly copying only the core lightning binaries.

Changelog-Fixed: Fixes failing Docker build for arm32 arch.

@ShahanaFarooqui ShahanaFarooqui added this to the v24.08 milestone Aug 14, 2024
@ShahanaFarooqui ShahanaFarooqui marked this pull request as draft August 14, 2024 04:13
@ShahanaFarooqui ShahanaFarooqui force-pushed the github-docker-testing branch 24 times, most recently from ed04380 to 521fb6b Compare August 19, 2024 23:59
@ShahanaFarooqui ShahanaFarooqui marked this pull request as ready for review August 19, 2024 23:59
@ShahanaFarooqui ShahanaFarooqui force-pushed the github-docker-testing branch 5 times, most recently from 2892fd6 to 167e91e Compare August 20, 2024 05:15
@cdecker
Copy link
Member

cdecker commented Aug 20, 2024

ACK 167e91e

Just a minor discrepancy between envvars and their documentation (push defaults to true, rather than false as the docs claim).

@ShahanaFarooqui ShahanaFarooqui force-pushed the github-docker-testing branch 2 times, most recently from e610300 to eec6993 Compare August 20, 2024 20:48
- Temporarily adding `rc` tag trigger for testing `Build and push multi-platform docker images` action flow before the final release.

- Added some variable inputs for testing like repo, platforms, etc.

- Added more logs for future debugging.
- **Poetry Issue:** The Docker build for `linux/arm/v7` failed in recent RC releases on the Poetry installation step in the `builder-python` stage. This issue occurred because the `builder-python` stage builds on target's arch but poetry was unable to install on arm/v7 without rust >= v1.56.1.
    - **Solution:** Instead of installing poetry on the `builder-python` stage, we leveraged the existing multi-arch `builder` stage, which already had Poetry. Now, we export the dependencies from `pyproject.toml` to `requirements.txt` within the `builder` stage and then copy `requirements.txt` to the `builder-python` stage for pip installation.

- **Cryptography installation Issue:** python installations for `pyln-proto` started failing due to Cryptography upgrade from v41 to v42 (#7475). It is because now Cryptography needs cargo/rust also.
    - **Solution:** Installing cargo in `builder-python` stage also.

- **Configure Prefix Issue:** Previously, we used `RUN ./configure --prefix=/tmp/lightning_install --enable-static` in the `builder` image and then copied `/tmp/lightning_install` from the `builder` stage to `/usr/local` in the `final` stage. However, this approach is now causing errors due to missing binaries/plugins at their default locations.
    - **Solution:** We are now configuring the installation to use the default location (`/usr/local`). To prevent the local image size from increasing by up to 87MB, instead of copying the entire `/usr/local/` directory, we are explicitly copying only the core lightning binaries.

Changelog-Fixed: Fixes failing Docker build for `arm32` arch.
@ShahanaFarooqui ShahanaFarooqui merged commit dadf9a7 into master Aug 21, 2024
38 of 39 checks passed
@ShahanaFarooqui ShahanaFarooqui deleted the github-docker-testing branch August 21, 2024 04:25
@ShahanaFarooqui ShahanaFarooqui linked an issue Sep 1, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Multiarch docker builds of pre-releases
2 participants