Support Dockerfiles found at URL

[1Mill]

Overview

This tool only appears to work within the Docker build context which is limited to wherever you run the docker build my-dir command. It would be awesome if we could do something like

# syntax=edrevo/dockerfile-plus:0.1
FROM scratch
INCLUDE+ https://github.com/my/project/name.git#my-branch-name:my/directory/path/dockerfile

to support Dockerfiles outside of the Docker build context.

[aagrawa3-dev]

Overview

This tool only appears to work within the Docker build context which is limited to wherever you run the docker build my-dir command. It would be awesome if we could do something like

FROM scratch
INCLUDE+ https://github.com/my/project/name.git#my-branch-name:my/directory/path/dockerfile

to support Dockerfiles outside of the Docker build context.

Can't you do a wget first for above dockerfile. You can do wget even within your own dockerfile.
combining wget to INCLUDE+ : It sounds a bit odd.

[1Mill]

@aagrawa3-dev, if I am following you correctly:

Can't you do a wget first for above dockerfile.

Indeed you can fetch arbitrary content as part of your external-to-docker build script and then pass that content into the docker build command as outlined in the documentation (https://docs.docker.com/engine/reference/commandline/build/#specify-a-dockerfile--f) like below:

curl https://raw.githubusercontent.com/my/project/my-branch-name/my/directory/path/dockerfile \
| docker build --file - my-dir

However, as far as I am aware it would not let you include multiple Dockerfiles from multiple fetch requests out-of-the-box. The thing that I like about INCLUDE+ is that (1) it is part of the docker build process and not something to be managed as part of your external pre-docker-build step and (2) it automatically handles the merging of the multiple Dockerfiles together as part of that docker build process.

I have not messed around with INCLUDE+ enough yet, but I would expect the below to work if all the common-*.dockerfile were in the docker build my-dir context in the current implementation of INCLUDE+.

# syntax=edrevo/dockerfile-plus:0.1
FROM scratch

INCLUDE+ common-1.dockerfile
INCLUDE+ common-2.dockerfile
INCLUDE+ common-3.dockerfile

RUN echo "my extension on top of the common files above"

---

You can do wget even within your own dockerfile.

Not sure I know what you mean by this, can you elaborate more or provide an example of how I could (1) fetch a Dockerfile from GitHub for example and then (2) add that code from the fetched Dockerfile to INCLUDE+?

---

combining wget to INCLUDE+ : It sounds a bit odd.

Fetching Dockerfiles from online sources is supported by the docker build command and I think it would be awesome it was also included as part of the INCLUDE+ instruction. Even if INCLUDE+ could support <<EOF ... EOF syntax and we could do wget or curl requests as part of a multi-stage docker build that would be cool.

[aagrawa3-dev]

Hi @1Mill,
I was thinking something like this in your master dockerfile:

RUN wget <the url-path of common.dockerfile>
INCLUDE+ common.dockerfile

So, you fetch the dockerfile before including it.

Let me know if that works for you!!

[1Mill]

@aagrawa3-dev, I do not believe the above works because INCLUDE+ is looking for files on the host machine and not within the container itself.

I experimented with the below but always got the "could not read file" (i.e. could not find file) error:

# syntax=edrevo/dockerfile-plus:0.1

FROM alpine:3
RUN apk --no-cache add curl
RUN curl \
	--create-dirs \
	--output /tmp/common.dockerfile \
	https://raw.githubusercontent.com/1Mill/lambda-docker/main/nodejs/v14/2021-09-02/dockerfile
INCLUDE+ /tmp/common.dockerfile

failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Code(128) desc = Could not read file " /tmp/common.dockerfile". Remember that the file path is relative to the build context, not the Dockerfile path.

[aagrawa3-dev]

@1Mill Yes I agree to this. That is a real issue then.
The way I have done so far - is to have an script outside the Dockerfile grabbing the content from web and then running the docker build.
I am still not sure though that asking INCLUDE+ to do such activity is right move. (keep it simple and explicit)
What you are asking here is - WGET+ running before INCLUDE+

Do you see an issue running a script before calling docker-build?

[1Mill]

@aagrawa3-dev, there are a handful of Docker pre-processors out there that do that kind of fetching and merging without the need for INCLUDE+. Without a doubt I could configure something to do this, however, this adds another step to the build process which is undesirable for my use-case. The thing that I really like about INCLUDE+ is that it runs natively as part of the docker build command so there isn't a need for a pre-processor step.

However, as we have discovered a major limitation of INCLUDE+, and really this just seems to be Docker in general as I dig deeper into the "build context", is that you are required to structure your application in a particular way which really does not support composability from arbitrary sources (e.g. a url, another file somewhere else on your computer outside the Docker daemon / build context).

Really, the motivation for having INCLUDE+ support, wget, curl, etc. is to support composability. Even if INCLUDE+ downloads, build the Dockerfile, and then deletes those downloads internally like a traditional pre-processor the fact that it again runs as part of the docker build command natively is the biggest benefit in my mind to support composability without needed to make the build process more complex.