Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docker build tooling to help build a local set of working libraries #7

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

guss77
Copy link

@guss77 guss77 commented Feb 22, 2024

I found it useful to have a reproduceable build setup that works without needing to install development libraries (and especially the AWS C++ SDK, for which there isn't a ready package), and the GitHub Workflows CI setup didn't work for me, so I've built this.

If you think this is interesting but would rather it be changed in someway - please let me know and I will try my best to accommodate. If you do not think this would be useful in the main s3fs-fuse-awscred-lib - that's also OK 🤷 .

@ggtakec
Copy link
Owner

ggtakec commented Feb 24, 2024

@guss77 Thank you for this pull request.

Could you not build in the same build procedure as GHA CI?
I think this is a PR for creating a Docker image, but let me think a little.

I would like to incorporate the process of creating a Docker image into the GHA CI of this s3fs-fuse-awscred-lib.
At the same time, I will prepare a flow to create various packages with a script(existed or additional) under .github/workflow.
(Please give me a little time)

For that, there are some things that I have to think(as #8).
For example, whether I can request to have this package added to a general package repository(currently not), or whether there is a place to store own-built package, or whether there is a place to store Docker images(probably allowed on DockerHub).

Could you make this PR wait a little while we consider these?

@guss77
Copy link
Author

guss77 commented Feb 25, 2024

Could you not build in the same build procedure as GHA CI?

I cannot use the content of the ci.yml directly in the build (at least not without building a parser) but I can copy and use the same commands in the Dockerfile. I'll commit that change shortly.

I think this is a PR for creating a Docker image, but let me think a little.

This isn't about building a Docker image that can run s3fs inside a container (not that I think that is even possible due to containers not sharing the mount namespace) - it is about allowing users to build the library in an isolated environment without needing to install all the dependencies on their own machine. The result of running the Makefile is compiled SO files on their host machine, and the Docker image can be discarded.

I would like to incorporate the process of creating a Docker image into the GHA CI of this s3fs-fuse-awscred-lib.

As I've mentioned - I don't think that a Docker image of s3fs will be useful, as you cannot propagate the fuse mount outside the container (not even with rbinds). I've tried playing that game in the past and lost. There may be more esoteric container runtimes that allow this, but neither Docker nor Podman can.

At the same time, I will prepare a flow to create various packages with a script(existed or additional)

I think a simple script that is one command to generate a package with all dependencies is useful - but due to the non-trivial dependency on AWS SDK C++, this is not trivial - hence the docker build process. If you can supplant the entire process with a script of your own - that would be better :-)

For example, whether I can request to have this package added to a general package repository(currently not), or whether there is a place to store own-built package

Because the credlib depends on AWS SDK, I believe no distribution package repository will take it without going for the arduous process of contributing AWS SDK C++ packages (more than one) and then maintaining them - which I doubt that you'd want to do that. The other option is user-supported package collections such as AUR for Arch, PPA for Ubuntu and derivatives and COPR for Fedora and derivatives. If this is something that you are interested in - I can probably land a hand.

Could you make this PR wait a little while we consider these?

You've got it 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants