Cross compilation

[Ch4s3r]

Cross compilation is currently not possible.
There would be two ways to do it AFAIK.

1. Install a linker for the target platform and use that one.
2. Use cargo cross for compilation instead of cargo, maybe with a feature flag.

Both could be solved by #21.

Preference would be to have cross installed when using a target platform other than the host one installed.

Tried to go with the first approach, but that failed as there is no linker for arm64 installed.

pack build image_name \
-b docker.io/paketocommunity/rust \
-B paketobuildpacks/builder:tiny \
-e BP_RUST_TOOLCHAIN=nightly \
-e BP_RUST_PROFILE=default \
-e BP_RUST_TARGET=aarch64-unknown-linux-musl \
-e BP_CARGO_INSTALL_ARGS='--target aarch64-unknown-linux-musl' \
-e CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-linux-musl-gcc

Log

pack build image_name -b docker.io/paketocommunity/rust -B paketobuildpacks/builder:tiny -e BP_RUST_TOOLCHAIN=nightly  -e BP_RUST_PROFILE=default -e BP_RUST_TARGET=aarch64-unknown-linux-musl -e BP_CARGO_INSTALL_ARGS='--target aarch64-unknown-linux-musl' -e CARGO_TARGET_AARCH64_UNKNOWN_LINUX_MUSL_LINKER=aarch64-linux-musl-gcc
tiny: Pulling from paketobuildpacks/builder
Digest: sha256:dedb220941776b89554291527e3ca7a350fdaf13290d9e7a86c1ea66c3406d90
Status: Image is up to date for paketobuildpacks/builder:tiny
tiny-cnb: Pulling from paketobuildpacks/run
Digest: sha256:4d6d3e6cab614672bd98e6e5a664597f69136fcb575f1ed95279242d444a65a0
Status: Image is up to date for paketobuildpacks/run:tiny-cnb
latest: Pulling from paketocommunity/rust
Digest: sha256:7aa9da459048919daed07ed4617c8ccfb5ad2fd5da2426915439cbe00a78352a
Status: Image is up to date for paketocommunity/rust:latest
===> ANALYZING
===> DETECTING
3 of 4 buildpacks participating
paketo-community/rustup    1.2.0
paketo-community/rust-dist 1.5.0
paketo-community/cargo     0.3.0
===> RESTORING
Restoring metadata for "paketo-community/cargo:tini" from app image
===> BUILDING

Paketo Rustup Buildpack 1.2.0
  https://github.com/paketo-community/rustup
  Build Configuration:
    $BP_RUSTUP_ENABLED       true                        use rustup to install Rust
    $BP_RUSTUP_INIT_LIBC     gnu                         libc implementation: gnu or musl
    $BP_RUSTUP_INIT_VERSION  1                           the rustup version
    $BP_RUST_PROFILE         default                     the Rust profile to install
    $BP_RUST_TARGET          aarch64-unknown-linux-musl  an additional Rust target to install
    $BP_RUST_TOOLCHAIN       nightly                     the Rust toolchain or version number to install
  Rustup (GNU libc) 1.24.3: Contributing to layer
    Downloading from https://static.rust-lang.org/rustup/archive/1.24.3/x86_64-unknown-linux-gnu/rustup-init
    Verifying checksum
    Copying to /layers/paketo-community_rustup/rustup-gnu/bin
  Cargo: Contributing to layer
  Rustup: Contributing to layer
    Installing Rustup
  Rust: Contributing to layer
    Installing Rust
      info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
      info: latest update on 2022-02-26, rust version 1.61.0-nightly (d3ad51b48 2022-02-25)
      info: downloading component 'cargo'
      info: downloading component 'clippy'
      info: downloading component 'rust-docs'
      info: downloading component 'rust-std'
      info: downloading component 'rustc'
      info: downloading component 'rustfmt'
      info: installing component 'cargo'
      info: installing component 'clippy'
      info: installing component 'rust-docs'
      info: installing component 'rust-std'
      info: installing component 'rustc'
      info: installing component 'rustfmt'
      
        nightly-x86_64-unknown-linux-gnu installed - rustc 1.61.0-nightly (d3ad51b48 2022-02-25)
      
      info: default toolchain set to 'nightly-x86_64-unknown-linux-gnu'
      info: checking for self-updates
      info: downloading component 'rust-std' for 'aarch64-unknown-linux-musl'
      info: installing component 'rust-std' for 'aarch64-unknown-linux-musl'
Warning: the following SBoM files will be ignored for buildpack api version < 0.7 [/layers/paketo-community_rustup/rustup-gnu.sbom.syft.json]

Rust Distribution Buildpack 1.5.0
  https://github.com/paketo-community/rust-dist

Rust Cargo Build Pack 0.3.0
  https://github.com/paketo-community/cargo
  Build Configuration:
    $BP_CARGO_EXCLUDE_FOLDERS    static, templates, public, html      folders that should not be deleted and should persist to the generated image
    $BP_CARGO_INSTALL_ARGS       --target aarch64-unknown-linux-musl  additional arguments to pass to Cargo install
    $BP_CARGO_TINI_DISABLED      false                                Skip installing tini
    $BP_CARGO_WORKSPACE_MEMBERS                                       the subset of workspace members for Cargo to install
  Tini 0.19.0: Reusing cached layer
    Creating cached target directory /workspace/target
  Rust Application: Contributing to layer
    File modification times not restored
    File modification times not restored
    File modification times not restored
    cargo install --target aarch64-unknown-linux-musl --color=never --root=/layers/paketo-community_cargo/Cargo --path=.
        Installing image_name v0.1.0 (/workspace)
          Updating crates.io index
         Compiling image_name v0.1.0 (/workspace)
      error: linker `aarch64-linux-musl-gcc` not found
        |
        = note: No such file or directory (os error 2)
      
      error: failed to compile `image_name v0.1.0 (/workspace)`, intermediate artifacts can be found at `/workspace/target`
      
      Caused by:
        could not compile `image_name` due to previous error
unable to invoke layer creator
unable to contribute application layer
unable to install single
unable to build
exit status 101
ERROR: failed to build: exit status 1
ERROR: failed to build: executing lifecycle: failed with status code: 51

[dmikusa]

This is definitely something I would like to add.

At the moment, my understanding is that there's not a lot of need for this. I'd be curious if you could expand on your ideas here, in terms of how you'd use this? How you'd like to see it work? etc...

My understanding is this:

1. Buildpacks are primarily targeted to build on one architecture and run on that architecture. It should theoretically be possible to build on one architecture and produce an image for another architecture. This would require support not just in the Rust buildpack, but also in the buildpack tools though. Presently, the buildpack tools will assume you're building an image for the architecture on which you're running the tools.

2. Primarily, buildpack tools and Paketo buildpacks run on x86_64. Some of the buildpack tools have some support for running on ARM64, but it's not in a state where it's as easy to use as x86_64. Primarily because Paketo does not yet publish ARM64 images. I have some notes on this for Java here, I think you could use some of that for Rust as well. That is not really targeted for cross-compiling though, it's for running on arm64 and building arm64.

My thought for this would be to do both. I had thought about adding a Zig buildpack as I'd heard the Zig toolset can be used to make cross-compiling Rust pretty easy (see here).

Using cross would be interesting too. I do believe that #21 would at least allow you to install the cross cli tool. From there, I'm not sure. It would depend on how it's installing everything else that it needs to cross-compile. When building as buildpacks, you run as an unprivileged non-root user so you can't install packages. It would fail if it tried that. You're also running inside of a container already, so if it was trying to use Docker to run something in a container that would fail as well.

At any rate, I'm optimistic that we can make something work for cross-compilation. It just may take some time.

[Ch4s3r]

Basically my goal was to use Github Actions to use buildpack and compile on an amd64 runner but build an image which could run on my Raspberry Pi on arm64.

I was not sure if buildpacks were meant for cross compiling.
The alternative which I've done for now was to use a Dockerfile with buildkit --platform=linux/arm64 to build an image for arm.

When building as buildpacks, you run as an unprivileged non-root user so you can't install packages.
Ah didn't know that. My thought was to install the linker than, so that cargo could use it.

You're also running inside of a container already, so if it was trying to use Docker to run something in a container that would fail as well.
So cross will probably spawn more docker containers and then build the image in there.
And docker in docker in no option? Not sure how this would work, just a thought.

[dmikusa]

I was not sure if buildpacks were meant for cross compiling.

It hasn't been a possibility to date. The upstream tools for buildpacks do not allow for setting alternative platforms. There's an assumption that the platform is the one on which you're building. So if you could run a build with an ARM64 container, then the tools should work there, but you can't run on x86 and build for ARM64. It's possibly something that may be available in the future, but that's more a question for https://github.com/buildpacks.

If we can get some upstream support for it, so you can pack build --platform=linux/arm64 or something, I'd love to support it.

So cross will probably spawn more docker containers and then build the image in there.
And docker in docker in no option? Not sure how this would work, just a thought.

I don't think so, for the reason of limited access. The only possible way this would work is if you had a stack image built that already had docker installed & it was configured so that the cnb user could run docker commands. Even then, it wouldn't be my first choice for a solution. Docker in docker I believe has some security implications, requires a privileged container, and it making it work for non-Docker based builds like kpack or some CI systems like tekton could also be problematic.