Add jemalloc support

[marcotc]

Third time is the charm!
I had this PR open for a few weeks and realized there's work being done around jemalloc.

Should we add jemalloc only for Debian? If so, I'll just move my logic to the template file.

This PR adds jemalloc all Linux images: debian and slim.

I tried to make it generic, but I'm not sure we need this level of complexity here.

fixes #182, closes #190

[marcotc]

My initial intention was to create a new set of images, to avoid any backwards incompatibility, if any:

* 2.5.0-jemalloc
* 2.5.0-stretch-jemalloc
* 2.5.0-slim-jemalloc
* 2.5.0-slim-stretch-jemalloc

But it seems like there are talks about just adding jemalloc the default Debian images.

One option is to only use jemalloc for the newest Ruby releases, keeping users of older versions intact.
This would prevent the creation of a lot of duplicate files, but reduce risks to an extent.

[marcotc]

I had to add it as a separate set of dependencies, because buildDeps get purged after the build is finished.

[ianks]

Is there a particular reason not to do this with alpine as well?

[marcotc]

@ianks not particularly. I'll give at shot at building alpine images.

[marcotc]

@ianks I just tried to compile Alpine with jemalloc and ended up getting stuck on this Ruby/Alpine/Jemalloc bug: https://bugs.ruby-lang.org/issues/13524

It seems like we are out of luck for the time being for Alpine+jemalloc.

[n-rodriguez]

This is weird because as far as I know Redis uses jemalloc and there is a redis:alpine Docker image :/
https://github.com/sickp/docker-alpine-redis

[marcotc]

@n-rodriguez It seems like this has to do with how Ruby and Redis handle the pointer returned by getcwd differently:

• Ruby frees the returned pointer, as the Linux man page suggests. This can be seen in the code snippet provided by shyouhei in the bug report. (Direct link to snippet)
• Redis doesn't seem to free the returned pointer: https://github.com/antirez/redis/search?l=C&q=getcwd

[n-rodriguez]

@marcotc thanks for the answer!

[grosser]

maybe nice to leave this as a sanity check with RUN ruby -r rbconfig -e "abort 'jemalloc not enabled' unless RbConfig::CONFIG['LIBS'].include?('jemalloc')"

[callumj]

Thanks for working on this @marcotc, we're currently "patching" jemalloc support off the official Docker library using some custom scripts so this is a huge help by getting it official.

[ababich]

There is no issues in https://github.com/hoteltonight/ruby so may be I can comment here

As I understood this is published in Docker Hub as https://hub.docker.com/r/hoteltonight/ruby-jemalloc/ - is it possible to make relations between docker hub, github repo, etc. more transparent?

[cheerfulstoic]

I don't understand the issues that are going on, but it would be awesome to have jmalloc support built-in to our Ruby containers. Is this stalled?

[yosifkit]

I think we were still uncertain whether this is the recommended or endorsed way to build ruby according to ruby upstream. We try to keep the images as close to upstream release recommendations as possible.

See also #192 (comment).

[marcotc]

Would a PR with a new separate set of debian images using jemalloc be acceptable?
I can make those changes if that's the case.

[cheerfulstoic]

Also, would it be possible to specify an environment variable when building? There seems to be a --with-jemalloc flag which could be used (though I imagine there would need to be at least one other conditional to install jemalloc beforehand

[shleeable]

Hey @marcotc , #241 will hopefully fix the alpine blocker.

[marcotc]

Jemalloc is not "sticking" to Ruby 2.6rc as it does with older versions.
I don't see the library linked anymore, even though we built with it:

Step 11/17 : RUN ruby -r rbconfig -e "puts RbConfig::CONFIG['LIBS']"
 ---> Running in e0dc1e0e4115
-lm

Still investigating this.

[marcotc]

Now we have this scenario:
Jemalloc works fine for Ruby < 2.6 on Debian, and possibly Alpine 3.8, which would make the template file even more complex.

I'm wondering if we should just publish a separate set of images with the prefix -jemalloc, and not touch the "vanilla" images.

[tianon]

I wholeheartedly agree, and recommend that such work happen in a separate repo/image until Ruby upstream decides jemalloc is something they want to officially support and recommend for general use. 👍

[marcotc]

Just to clarify @tianon, are you saying jemalloc images should not be hosted under https://hub.docker.com/_/ruby/, not even if labelled appropriately?

[tianon]

I'm saying the docker-library team would prefer to not maintain them unless/until they're officially recommended and supported by Ruby upstream. I do see value in them existing, and recommend that someone create and maintain them elsewhere, which will serve to help make them easier for folks to consume and test, which will hopefully help convince upstream that jemalloc is something they can rely on and should officially support or even recommend outright. Hopefully that helps clarify our position (and our hesitance to do anything further with this PR at this time).

[marcotc]

I totally understand @tianon, thanks for clarifying.

[shleeable]

Is there a forked image I can use? or are we not there yet?

[giovannibonetti]

Here in my company we have been using this fork from Hotel Tonight in production for a few months and it works great. We use hoteltonight/ruby-jemalloc:2.5.1-stretch

[ababich]

@ashleyhull-versent I used hoteltonight image for some time but it is much easier to build your own based on official docker image, just take a look at 2-3 changes in Dockerfile and go ahead

[shleeable]

Does anybody know who's actually at fault with this? Is there a ticket upstream with Ruby or Alpine?

[yosifkit]

Closing PR until a recommendation from Ruby upstream. Discussion can continue on #182.

See also #182 (comment)