Problem with --as-needed with fortran and librt, at least

[minrk]

I've encountered a few hiccups with the new default -Wl,--as-needed flag for the compilers. Mostly, these have revealed a few libraries where transitive dependencies were implicit, and can be fixed by making dependencies explicit, or opting out of -Wl,--as-needed on certain packages.

The weirded issue recently is that --as-needed and --enable-new-dtags seem to combine to make entirely non-working fortran executables.

To test, create an env with gfortran:

conda create -n test-env gfortran_linux-64
conda activate test-env

And then try to compile and run a simple fortran hello world (test.f90):

program main
     print *, "ok"
end program main

Simple compilation will work:

$FC -o test test.f90 && ./test
ok

But enabling some of the flags that are present when using conda build fails:

$FC -Wl,--as-needed -Wl,--enable-new-dtags -o test test.f90 && ./test
./test: error while loading shared libraries: libquadmath.so.0: cannot open shared object file: No such file or directory

Checking ldd reveals something odd. libquadmath is still required, but not found:

ldd test
	linux-vdso.so.1 (0x00007ffd711ac000)
	libgfortran.so.4 => /opt/conda/envs/test/lib/libgfortran.so.4 (0x00007fcb98002000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcb97b76000)
	libquadmath.so.0 => not found
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fcb97872000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fcb9765b000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fcb97f15000)

Removing either as-needed or new-dtags will result in a working executable:

$FC -Wl,--enable-new-dtags -o test test.f90 && ldd ./test
	linux-vdso.so.1 (0x00007ffcaa7d3000)
	libgfortran.so.4 => /opt/conda/envs/test/lib/libgfortran.so.4 (0x00007fd081af6000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd081705000)
	libgcc_s.so.1 => /opt/conda/envs/test/lib/libgcc_s.so.1 (0x00007fd081adf000)
	libquadmath.so.0 => /opt/conda/envs/test/lib/libquadmath.so.0 (0x00007fd081aac000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd081366000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fd081a09000)

I don't know where the root of the issue is.

[minrk]

It's possible this is something specific about the MPI compilers, but it only seems to come up in the mpi + gcc7 combinations, and removing --Wl,--as-needed fixed the problem. Maybe it's mpi that's responsible for --enable-new-dtags and I need to dig into where that's coming from.

[mingwandroid]

--enable-new-dtags is always a bad idea IMHO (outside of local development workflows) .. Some people set LD_LIBRARY_PATH globally and this results in errors for conda software.

conda-build actually errors out when this is detected these days (there's a way to override it though).

[minrk]

ok, I'll dig into MPI to see if I can understand how/why they are setting new-dtags and if I can override it.

[minrk]

Found another weird one today, this time with scotch.

scotch gets clock_gettime from librt. It explicitly links -lrt. However, building with --as-needed strips librt from libscotch, even though it's needed for this symbol. Downstream packages cannot link libscotch without also linking librt, even if they don't themselves need it. I worry they might also end up stripping librt for the same reason, etc.

Hypothesis: clock_gettime is in glibc >= 2.17, so recent libc doesn't need librt for this symbol. But the host glibc is 2.12, so librt is needed. However, the libc in sysroot in the gcc_impl_linux-64 7.3.0 package is 2.31.1. I'm guessing that the more recent glibc version is being used to decide whether librt is needed or not. I don't know if there's a flag I should be adding (or excluding) to consider the host libc (or specify some baseline glibc version directly).

The relevant linker command that's producing a lib with missing librt:

$BUILD_PREFIX/bin/x86_64-conda_cos6-linux-gnu-cc -shared -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,-rpath,$PREFIX/lib -L$PREFIX/lib -lm -lz -pthread -lrt -lm -lz -pthread -lrt -lm -lz -pthread -lrt -Wl,-soname,libscotch-6.so ...lotsofthings.o... -L. -lscotcherr -lz -lpthread -lm -o libscotch-6.so

[jjhelmus]

Where are you seeing glibc 2.31.1, the latest release is 2.28? 2.12.2 is (or at least should be) present in the sysroot of the gcc_impl_linux-64 package (gcc_impl_linux-64-7.3.0-habb00fd_1).

[minrk]

Oops. I got it from ld --version which I got from googling around. I guess that's the binutils version. Sorry about that.

In any case, the linker is treating librt as 'not needed' when librt is only providing symbols present in libc for glibc >= 2.17. I don't know how it's coming to that conclusion or how to tell it to pick a different glibc base version to consider, but I was guessing that it might have to do with using a more recent gcc.

[jjhelmus]

Interesting, it seems as if the order of the libraries does have some effect. If I add -lrt to the end of the linked command so that it reads:

... -L. -lscotcherr -lz -lpthread -lrt -lm -o libscotch-6.so

librt is linked.

Adding -Wl,--no-undefined to LDFLAGS help a here so that the build fails if the symbol is undefined.

[msarahan]

I think @mingwandroid has warned me about this before. -l arguments that happen before --as-needed don't work right. They all have to come after it. That gets back to "mistakes" in makefiles - if they are written in a way that CFLAGS and such get passed after other arguments in the makefile.

[jjhelmus]

Ah, I see the issue now. Since gcc is being used as the driver the order of the arguments does matter. Specifically the comment about the importance of the order from the GCC manual covers this:

It makes a difference where in the command you write this option; the linker searches and processes libraries and object files in the order they are specified. Thus, ‘foo.o -lz bar.o’ searches library ‘z’ after file foo.o but before bar.o. If bar.o refers to functions in ‘z’, those functions may not be loaded.

Since the linker has not seen a object file with a symbol from librt when it reaches -lrt it drops it.

[jjhelmus]

The following change to the debian-dynamic.patch file fixes the issue:

diff --git a/recipe/debian-dynamic.patch b/recipe/debian-dynamic.patch
index 50e34f9..28defd0 100644
--- a/recipe/debian-dynamic.patch
+++ b/recipe/debian-dynamic.patch
@@ -131,7 +131,7 @@ Index: scotch/src/libscotch/Makefile
 -libscotch$(LIB)                       :       $(LIBSCOTCHDEPS)
 -                                      $(AR) $(ARFLAGS) $(@) $(?)
 +libscotch$(LIB)                       :       $(LIBSCOTCHDEPS) libscotcherr$(LIB)
-+                                      $(CC) -shared $(LDFLAGS) $(SONAME)libscotch-$(VERSION)$(SHLIB_EXT) $(LIBSCOTCHDEPS) -L. -lscotcherr -lz -lpthread -lm -o libscotch-$(VERSION)$(SHLIB_EXT)
++                                      $(CC) -shared $(LDFLAGS) $(SONAME)libscotch-$(VERSION)$(SHLIB_EXT) $(LIBSCOTCHDEPS) -L. -lscotcherr -lz -lpthread -lrt -lm -o libscotch-$(VERSION)$(SHLIB_EXT)
 +                                      ln -s libscotch-$(VERSION)$(SHLIB_EXT) libscotch$(SHLIB_EXT)
 +                                      $(AR) $(ARFLAGS) $(@) $(LIBSCOTCHDEPS)
                                        -$(RANLIB) $(@)