add dumb-init + running as unprivileged user

[ellerbrock]

Update rancher-cron

• add dumb-init to prevent zombie reaping problem.
• add security by running as unprivileged user

and thanks for the service!
i was looking for a way to run my cloudflare-dynamic-dns-updater and other small services in a scheduled way without the need of sleep and lots of running containers ...

[smazurov]

Any chance to use tini instead? Looks like docker 1.13 onwards will implement tini inspired solution. Cleanest is probably to apk add --update --no-cache tini and then set it as an ENTRYPOINT, instead of CMD.

💯 on alpine update and creating a new user.

[ellerbrock]

hey @smazurov,

you don't need --update when running --no-cache, you can find more about it here (done this before as well and someone mentioned it, no im the smart ass doing that haha). Dumb Init looks fine too me and do his job well. but yes there are of course lots of other tools out there too (dumb-init, tini, s6, openrc) which more or less do the same thing, prevent zombie processes. the guys from hashicorp [consul, vault, packer, vagrant comes from them ...] use dumb-init too for example in vault. worth a read for a good overview for dumb-init is this article. entrypoint is ok, i just put the stuff in cmd to prevent some breaking may cause by that change with your container ... for my stuff i put dumb-init with the default command in the entrypoint and cmd for the fine tuning which user can overwrite (you can have a look in my container stuff here and check the sources in this multi repo.

i yesterday listen to the rancher monthly talk in the background and heared them talking about that they used the interval or timeout stuff for a poor mans cron. i played with that a bit before using your container with ranger-compose.yml but was not really working. do you have experience with that.

another thing i wanted to ask is if there is a reason that you don't build your container automatically? im playing right now on a test server at home where security is not a big thing but pulling in non automatic builds feels always kind of risky.
a automatic build is very simple to setup and gives user more transparency.
the setup is super simple, just choose your repo where you put your dockerfile + binaries and let automatically build from docker hub or quay when you push new changes. saves time and makes it more transparent. you can find more about it here.

[smazurov]

yeah that stuff makes sense - just that docker chose tini for their zombie killing stuff in 1.13 is my reason for picking it. BC is not an issue - this project isn't 1.0 yet :)

. i played with that a bit before using your container with ranger-compose.yml but was not really working. do you have experience with that.

Not sure what you mean - I feel like running timeouts is such a hack job, which is why I wrote this in the first place.

another thing i wanted to ask is if there is a reason that you don't build your container automatically

I haven't seen a good example of building a go program remotely then stuffing it into a minimal image and pushing it to dockerhub (or having hub do the whole process themselves), have you? I have no problems setting it up - in fact thats preferred!

[ellerbrock]

i guess really not that big different if you use tini or dumb-init and i also not really care since both do more or less the same thing. if you prefere tini i don't mind to change the url and make a new pr, i just think having any init service on pid 1 running makes the container more "best practice like" :)

with interval or timeout i mean the following:

there is a interval setting for rancher-compose and i heard the guys from rancher in a webcast talking about that they somehow used this for a "poor mans cron" to restart container in a given interval. i will dig deeper how to do that but i thought maybe you know about that.
for example:
i run a cloudflare dynamic dns updater, execute the command and then exit. i don't see any sense to do some hacky stuff like a bash script with sleep or something and wast memory or cpu (even its not much). another example i added yesterday from the rancher catalog the letsencrypt service. this task needs to get triggered only on the need of a cert refresh but by default its running one month without any task to once a month trigger this cert build. another example from the catalog the docker-cleanup, at least there you can setup some loop=fail enviroment variable. since all this services runs this kind of sleep mode im confussed if i not understand something right. but running container when not needed seems to me like a wast of memory and performance or am i missing something? nice thing would be to have this kind of stuff directly integrated in rancher ...

about the golang stuff. i have no golang experience so far but looking forward to learn more about it (saw it a lot in docker stuff and github coming). the docker automatic build basically takes your dockerfile and execute all the commands and then deploy it automatically. your container will then be tagged as automatic build and user can see whats inside (to prevent sneaking some bad stuff into the networks). in case of go i guess u compile your source and get a binary. so u just add that binary to the repo and add it in your dockerfile via copy.

to make it fully transparent you could build the go binary with travis.
in the past i used travis natively where i got problems because you can use one language but with dockerized apps you just add docker to service and can do basically everything. i yesterday add a repo where i run link checks with travis, there you can find in the .travis.yml how to set things up: https://github.com/ellerbrock/travis-link-checker

[ellerbrock]

update:

i could not find the binary dist/rancher-cron, guess its on your local computer.
in case you can add the binary to the repo i can make this build automatic for you and make an example on my docker account ... in case you think its not a prob i could also add your script to the entrypoint and make it similar like my container stuff mentioned above.

[smazurov]

including binaries is an anti-pattern.

Even non-automatic builds show a dockerfile used to build the image - there is no difference.

Feel free to change it to tini via apk, and switch it to entry point, no need for new PR, i'll squash it anyway

[ellerbrock]

i build most of the docker stuff but i have a issue with govendor.
im not a go developer would be nice if you can help me with this issue.

what im doing so far:

• clone this repo to /tmp/build
• set ENV GOPATH to /tmp/build
• add /tmp/build/bin to PATH variable
• install govendor
• cd into directory /tmp/build

when i then run govendor sync i get:

/tmp/build # govendor sync
Error: Package "/tmp/build" not a go package or not in GOPATH.
govendor sync
	Ensures the contents of the vendor folder matches the vendor file.
	Options:
		-n           dry run, print out action only
		-insecure    allow downloading over insecure connection
		-v           verbose output

my env vars in the container:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tmp/build/bin
GOPATH=/tmp/build

some idear how to fix that?

[smazurov]

if GOPATH is /tmp/build it should have go-specific folder structure, so it it needs src and pkg folders with relevant subfolders.

The actual code needs to live in $GOPATH/src/github.com/socialengine/rancher-cron

[ellerbrock]

thanks for the tip,

govendor sync is working now, but i guess u need to update somehow your dependencies or something. after i run govendor sync and then make a govendor list it looks like this:

/tmp/build/src/github.com/socialengine/rancher-cron # govendor list
 v  github.com/Sirupsen/logrus
 v  github.com/gorilla/context
 v  github.com/gorilla/mux
 v  github.com/rancher/go-rancher-metadata/metadata
 v  github.com/rancher/go-rancher/client
 v  gopkg.in/robfig/cron.v2
pl  github.com/socialengine/rancher-cron
 l  github.com/socialengine/rancher-cron/cattle
 l  github.com/socialengine/rancher-cron/metadata
 l  github.com/socialengine/rancher-cron/model
  m github.com/gorilla/websocket
  m golang.org/x/sys/unix

i can fix this manual by running:

go get github.com/gorilla/websocket
go get golang.org/x/sys/unix

so i guess this needs to be added to the vendor.json or something (sorry im not a go programmer, just a guess).

and last thing not sure if we have to fix it or if its ok to ignore is this warning when running make build:

make build
CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o docker/dist/rancher-cron && chmod +x docker/dist/rancher-cron
# github.com/socialengine/rancher-cron
warning: unable to find runtime/cgo.a

but stuff gets build and the executable is running ...

i will add post a live example in a few minutes ...

[ellerbrock]

ok done.

i created a docker build which is fully transparent.
the go binary get automatically build inside the container from the github repo sources.
the docker container gets build from docker hub / quay so everybody can see and read whats going on inside.

you can find running container here:

https://hub.docker.com/r/ellerbrock/alpine-rancher-cron/
https://quay.io/repository/ellerbrock/alpine-rancher-cron/

The Repository for that build is here: https://github.com/ellerbrock/docker-collection/tree/master/dockerfiles/alpine-rancher-cron

In case you like how stuff is i can make a new pull request.
Would be nice if you can double check for the warning issue and that we can remove the current go get fix i made ...

Cheers

[smazurov]

cgo is needed to build a linux-compatible binary on a non-linux host (like macOs). Since you're building everything in one container, maybe you can just run

CGO_ENABLED=0 GOOS=linux go build -a -o $(OUTPUT_FILE)

Its quite clever to install all dependancies then remove them, but perhaps its better to separate build and run into different containers. Docker Hub supports build hooks in exactly the same way as Docker Cloud. So an ideal solution would be to build a "build" image, compile the source inside with mounted folder, then have dockerhub build the run image.

At any rate, your solution mostly works, but it uninstalls ca-certificates which are required.

Additionally, you can probably use go fetch instead of checking things out manually and creating the right folders.

go fetch github.com/socialengine/rancher-cron
cd $GO_PATH/src/etc
govendor sync
make build

[ellerbrock]

thanks for the response.
i added a quick fixed for the ssl cert, should hopefully work now (https://hub.docker.com/r/ellerbrock/alpine-rancher-cron/).
also i just want to mention that i did not ignore your request for tini, for your repo i will change dumb-init to tini, just want to keep my stuff with one solution.

do i have to install something for fetch?

go fetch github.com/socialengine/rancher-cron
go: unknown subcommand "fetch"
Run 'go help' for usage.

i also tried local in an empty folder to use go get:

go get github.com/socialengine/rancher-cron   ⏎
can't load package: package github.com/socialengine/rancher-cron: no buildable Go source files in /code/go/src/github.com/socialengine/rancher-cron

good that i came to this repo. yesterday night i finally took the chance and started to learn go programming :)

later on i will check your idear with the docker hooks, not really heard about that before.
with a git push i also build automatically for quay.io so not sure if this works all over, but need to read more about it. the build and remove in one step i use for images for docker hub. on my server i do build and deploy stuff in a more micro way.
i use for each task a own container which do one thing and do it hopefully right like catch git sources, build, test and at least of the chain the container which run the service.
its easy to setup with docker-compose and depends on but not a solution for one dockerfile and automatic builds. i will later read more about it.

lets try to fix the current go get hack first.
when all that stuff works i wanted to dig into the rancher catalog.
i think your container is need so would make sense to make a nice integration directly available with rancher or at least get stuff first running in a own privat catalog for my servers ...