-O3 gcc libraries #613
Comments
|
(currently we use a single genericized |
|
(nevermind, just looked at an .app from sourceforge) Also @kencu of course I'm not messing with the bootstrap flags; I keep it to bootstrap-debug, just for the final build flags. I used gcc-5's stock libraries, including the gmp, mpfr and mpc versions. I haven't encountered any issues with bootstrap and so far, but still, I'm interested to know if something really breaks that I'm overlooking. |
|
Well -- Iain tells me that the optflags for the libgcc and gcc builds are intrinsic to the gcc build system, they are set to optimal settings, and they totally ignore how we set the optflags or build the the bootstrap compiler. Are you modifying the deep-in-the-guts gcc Makefiles? If so, I can't sell that to MacPorts. Perhaps you can show me what you are doing, so i understand better? |
|
@iains -- can you weigh in here for a second? We're trying to make sure that gcc builds itself with optimal settings and there is some question whether it does. Specifically the gcc is gcc48, but as you know, MacPorts uses parts of libgcc from libgcc 5,6, & 7 in the final mix. |
|
(aside -- gcc8 / libgcc8 builds on 10.4 and 10.5 PPC too -- I just haven't rolled that out to macports yet. I haven't tried gcc9 in a while, but Iain may well have done). |
|
No, really, I don't want to hack the build system, all I'm doing is setting custom CFLAGS and CXXFLAGS, which the build ignores during stage1, but uses for stage2 and 3. Even in stage2 and stage3 an -O3 flag seems to be discarded in various directories. The resulting libraries, though, get fatter with -O3, so CFLAGS/CXXFLAGS, at the very least, does something. |
|
Well if that does change the compiler, and I take your word for it for now, that is news to me as I was under the impression that gcc totally ignored all that beyond the bootstrap compiler (unless you go to some lengths kencu/TigerPorts@d26a0bf#diff-969dec99a801dec9918fbde6ddf5d442 to force gcc to do something you want it to). Live and learn, I guess. I have to stick with macports builds, as I use that infrastructure to fix hundreds of other tickets and issues with ports, so can't veer too far off the farm. |
|
One thing to know, btw, if you're comparing to the macports libgcc build -- the overall libgcc infrastructure on macports comes from (at least) three different versions of libgcc. The approach is to use the libgcc dylibs from the last version supported on the OS, so the "libgcc" that gcc48 uses on 10.4 Tiger with macports is actually mostly libgcc 7.5.0, with parts of libgcc 6, and parts from libgcc48. |
|
well, yes indeedy, you can -- this is for libc, but presumably the same for other libraries. https://www.gnu.org/software/libc/manual/html_node/Configuring-and-compiling.html. Look at that. I learn so much from you guys. Now I will have to ponder how I can use that information... |
|
I'm working on getting the gcc runtimes. I hoped I could lazily cross-compile everything from 10.5 for 10.4 but it fails to. :-( I'll just properly boot into tiger. It says default to -O2 -g . GCC (the compiler portion) might default to no -O flag specified, with runtimes libraries defaulting to -O2 -g just like glibc does.. That would make sense from my tests. |
|
I am still trying to understand how compiling a library with a different opt setting could speed up your compilation, though. The gcc48 compiler, sure -- it does a lot of this and that -- but the library is just a bunch of precompiled routines. |
|
Hi folks,
Ken <[email protected]> wrote:
I am still trying to understand how compiling a library with a different
opt setting could speed up your compilation, though.
The gcc48 compiler, sure -- it does a lot of this and that -- but the
library is just a bunch of precompiled routines.
let’s try to separate concerns.
=====
1. How to build GCC with non-standard stage 2/3 bootstrap flags.
- this is done by setting BOOT_CFLAGS= to whatever you want
[ you can specify the stage flags specifically too STAGEN_???FLAGS= … but that’s tedious ]
- it’s not necessary, useful, or wise to build the stage1 compiler with complex optimisation; one might be trying to build using c++ features that were very new when the bootstrap compiler was made….
- in the “old days” it was said by at least two major vendors that the optimum code on Darwin was -Os (smallest achieved fastest)… so for folks building on 10.4 / 10.5 that’s a consideration. Unfortunately, I am only repeating what I’ve been told by vendors and major users - no hard evidence to back this up.
- If you are using sources (unpatched) <= 6.5 then I would recommend -Os or -fdeclone-ctor-dtor (CXXFLAGS) to work around a bug with cloneing functions [I’ve fixed this in all open branches and will back port to Darwin-specific 6.5 and 5.5.].
======
2. if one is building a cross compiler without bootstrapping then
CFLAGS/CXXFLAGS take effect.
=====
3. I can bootstrap all open branches natively on Darwin8, with two
constraints
: A) the current sources need ld64-58.2.1 at least
B) patches mentioned here:
https://gcc.gnu.org/pipermail/gcc-testresults/2020-May/560310.html
On my 500MHz G4 bootstrap & test takes almost a week ….
=====
4. Non-native Darwin8 bootstrap is possible on Darwin9:
A) the current sources need ld64-58.2.1
B) patches mentioned here:
https://gcc.gnu.org/pipermail/gcc-testresults/2020-May/560310.html
C) You _must_ use —with-sysroot=/Developer/SDKs/MacOSX10.4u.sdk
D) You must ‘export MACOSX_DEVELOPMENT_TARGET=10.4
E) It’s advisable to bootstrap GMP and friends with the compiler.
======
5. The library mess ..
- we need to find a usable technical solution to cleaning this up …
The fundamental problem is that we have a bunch of work-arounds to try and allow “after market upgrades” to system libraries .. I’m not convinced that this is the right solution for these old platforms, it could be better to “make a backup and install an updated working library” — at least for libgcc_s and libstdc++.
Anyway, sorry that I don’t have many cycles free for this right now - but
hopefully once GCC 10.2 is out I’ll poke at some of these things.
Could we try to make separate threads for each problem, if possible - to
help us figure out what we decided :)
cheers
Iain
|
|
iains <[email protected]> wrote:
Hi folks,
Ken ***@***.***> wrote:
>
> I am still trying to understand how compiling a library with a different
> opt setting could speed up your compilation, though.
>
> The gcc48 compiler, sure -- it does a lot of this and that -- but the
> library is just a bunch of precompiled routines.
let’s try to separate concerns.
=====
1. How to build GCC with non-standard stage 2/3 bootstrap flags.
- this is done by setting BOOT_CFLAGS= to whatever you want
Iff you want to build the [eventual target] libraries with non-standard
optimisation this is done with CFLAGS_FOR_TARGET=
this will also apply to the target libraries that are bootstrapped with the
compiler.
fine tweaking is possible - there are settable flags for everything….
… but TBH, I’ve never found it necessary in ≈ 15 years of working with GCC
(including many native crosses etc).
I’d say “you should have a very clear idea of what you are intending to
achieve and why” before moving off the path that is well-tested.
Iain
|
|
-O3 10.4-native |
|
Wow, that's great! Thank you, that saves me a lot of time. I think we can probably ship them in the beta. |
|
You're welcome! They're already relinked the same way as in a tenfourfox bundle, so they're basically drop-in replacements. |
|
I'll integrate them into the internal build system tomorrow and give it a test run. |
|
may I ask which version of gcc7 they were built from? |
|
They're built off gcc-5.5. Cameron has a fairly outdated macports tree from what I can see; the That means someone building tenfourfox using an FSF |
|
This is just something to be solved with policy: the drop-ins are only supported with the same ABI level. Builders would be on their own for that. I'll add something to the build notes. I have intentionally not revved my tree because I'm trying to eliminate toolchain issues as a variable in problems. |
|
FYI all the gcc versions on MacPorts (gcc48, gcc5, 6, 7) use the libstdc++.dylib from gcc7. There is no compatibility issue doing that. We bump libgcc all the time without breaking things as by-and-large they are backward compatible. However, before I bump libgcc to default to libgcc8 on PPC, Iain mentioned a possibly slipped-through-the-cracks ABI error that I will need to make sure is no big deal. |
|
Yes, they're backward-compatible, but older ones are not drop in, as the dynamic linker complains about versions. If gcc48 links against libraries from gcc7, then libraries from gcc7 become the minimum. |
|
Got it. Yes, I can see that these won't work for anyone using a current MacPorts to build TFF (if there is anyone besides the three of us :> ). I guess I should have no problem building O3's with gcc7 though, for my own use at least, and then all will be well again and I can use MacPorts current infrastructure. |
|
Ken <[email protected]> wrote:
However, before I bump libgcc to default to libgcc8 on PPC, Iain
mentioned a possibly slipped-through-the-cracks ABI error that I will
need to make sure is no big deal.
This is not a Darwin-specific problem but related to zero-sized objects in
= C++17, these can/do cause abi changes for powerpc-darwin (because the
alignment of some entities depends on placement within an aggregate).. One
issue was fixed, but I think another remains (no_unique_address attribute
applied within anonymous entities / unions).
I will try to find a pointer to the page describing the issue (it’s not
“our bug” as such, but Darwin might need corrections on PPC at least).
Iain
|
|
@iains is that specific to C++17, i.e., if it's not compiling to that standard, it shouldn't be a problem? TenFourFox's compiler options don't use that. |
|
By the way -- @classilla and @iains -- I wanted to make sure that the two of you were properly introduced, if you don't already know each other. Iain is the darwin lead for gcc, and has been making it all happen for some long time now. Cameron is without a doubt the other most knowledgable PowerPC programmer I have stumbled across in the past few years. You two have things in common! |
BTW, I have a more complete patch a for |
|
So, it drops in fine. I don't notice much improvement in V8, but that largely tests the JIT. DOM seems a bit sprightlier and nothing regresses so let's ship this for the beta. I've plugged it into the autobuilder. |
|
Hi Ken, Cameron,
Ken <[email protected]> wrote:
I can bootstrap all open branches natively on Darwin8, with two
constraints : A) the current sources need ld64-58.2.1 at least B) patches
mentioned here:
https://gcc.gnu.org/pipermail/gcc-testresults/2020-May/560310.html
BTW, I have a more complete patch a for _SC_NPROCESSORS_ONLN if you like.
Thanks, that looks good - what conditions is that patch under?
* for patches to Ada, I can’t approve directly (the Ada FE and library is maintained by Adacore), but I can post for approval of course.
* in any event, for GCC, it would need to be posted to gcc-patches@ with correct attribution for the author - name & email (this is just about below the size where (c) assignment is required, so I think attribution is sufficient).
* It’s somewhat of a corner-case - affecting only Ada at present (and the
fall-back I have is safe - in that it only tries to have one process).
However - it could be/become relevant for some libstdc++ stuff .. so
clarifying the postion would be worthwhile because I want to keep 10.4
viable for as long as possible.
cheers
Iain
|
|
That's pretty much it, no magic, but helps. Small win for already cpu-optimized builds. You did the testing on the G5 quad? |
|
Yup. |
|
Marking as fixed since this is shipping. Feel free to talk about the other issues still here if you want, though. |
|
@kencu Do you still recall what was the trick to make gcc8 build on 10.5 PPC? |
|
FWIW: I don't recall needing any 'trick' to build gcc-8-5 on powerpc-apple-darwin9: here is the configure line I used to check the 8.5 release: /src-local/gcc-8/configure --prefix=/opt/iains/powerpc-apple-darwin9/gcc-8-5 --build=powerpc-apple-darwin9 --with-as=/usr/bin/as --with-ld=/usr/bin/ld --enable-languages=all nothing special and no patches. NOTE: that there would be some recommended back ports of fixes since then - but those are not build fixes. |
|
@iains Thank you for replying. For me any gcc higher than 7 fails with some Bus errors. Initially I have been trying to get gcc11 to 10.6 PPC, but then tried to build the same on 10.5.8 PPC, and that too fails. |
If you would like to open a ticket on MacPorts about building gcc8 on 10.5 PPC, we can work through it there. For a start, what I would do is just build gcc8 alone, without macports in the mix. It is not hard to do, there are four prerequisites needed, and then it will build in a folder without any outside confusion. Do it on an Intel system first, maybe, just to really get a flavour of how this is done. Once you see how to build it simply, without confusion, then build gcc8 on PowerPC 10.5 Leopard. It builds easily (for me). And then, once you know it builds, you can try to build it in MacPorts environment. That adds lots of confusion, environment variables, non-standard linker, assembler, and libtool choices, etc. Then once you get gcc8 building in a MacPorts environment, you can look at actually updating the floor gcc in MacPorts of 10.5 to gcc8. There are issues on MacPorts building gcc8 in 10.5 on the Intel side (assembler issues induced by the changes made to cctools in MacPorts) but they can be worked through (I did, on 10.6+, with Iain's help). |
There are definitely versions of
|
|
@iains I have ld64 from Macports, and apparently it works since gcc4.9, gcc6 and gcc7 have compiled and are functional. Something breaks down starting with gcc8. I have actually tried to build your xtools, but Cmake complained about the lack of tapi. Then I tried building libtapi via Macports, and it failed on 10.6 PPC and 10.5.8 alike. I opened a ticket for that: https://trac.macports.org/ticket/64187 (re failure on 10.5.8). |
|
@kencu Thank you, I will give it a try. Anyway I want to check that 10.6 PPC is indeed unable to compile for ppc64, and Macports for some reason ignores --enforce-variants +universal (which I have set in config to be ppc ppc64). I remember what you have told me, so my hopes are low here. Once ppc64 fails, I try building the same gcc for ppc32 (outside of Macports). Compilation is running now, and unless is errors soon, I will leave it compile overnight. |
what version is that?
maybe the executable gets larger and the islanding fails?
Yes, tapi needs to be built from the llvm/clang project (I have not had any chance to update from 7.1.1 - but that was working sufficiently for the ld64 being built.) note: Upstream LLVM needs patches to work on powerpc-darwin (those are on my llvm-7.1.1. branch) Then I tried building libtapi via Macports, and it failed on 10.6 PPC and 10.5.8 alike. I opened a ticket for that: https://trac.macports.org/ticket/64187 (re failure on 10.5.8). no idea on that - (but it would not be a surprise, depending on the LLVM versions, since older OS support was dropped at various points) |
|
@kencu @iains Okay, Iains branch of gcc10 failed building with this: I have seen this error a number of times, it is not related to 64-bit (I have been building for ppc previously, not ppc64). I will reconfigure now for 32-bit just to make sure. P. S. I used gcc-mp-7 and g++-mp-7 in config (which otherwise perfectly work). |
10.6 was only released on X86, and the powerpc support was via Rosetta - which only supports ppc7400. The system libraries on 10.6 do not contain a ppc64 slice (so you should not expect it to work on any standard 10.6 configuration). |
|
@iains Here is the ld64 output: So the ld64_127 have been used for everything aside of TFF build attempt. Regarding tapi and xtools: so where should I start? As @kencu suggested, I can try working completely outside of Macports set up. (Which would mean I have gcc4.2 as a system compiler.) |
|
however, compiling 32b code with -macosx-version-min=10.6 should work for recent GCC (I do not recall when, exactly, we fixed the Rosetta support). |
ld64 127 was the last version to support ppc (officially) although ISTR that the release of cctools at the same time was broken for ppc64. I have not yet back ported the re-write of branch islanding to that version.
I'd suggest you try to build an earlier version of xtools This does not need a tapi build (and 10.5 does not use .tbd libs so that's OK) - however, this does need a C++11 compiler (all later ld64 versions do) but GCC5 is fine. For the record: up to gcc-10 you should be able to bootstrap with Apple-gcc-4.2.1 (I have checked that works) - but from GCC-11 we also need a C++11 compiler (although, again, GCC-5 should be OK). |
|
JFTR: The branch does work perfectly OK on powerpc-darwin9 : the 10.3 release was configured and tested like so: https://gcc.gnu.org/pipermail/gcc-testresults/2021-April/681033.html
This looks like some kind of configuration error - i.e. it is thinks it can use a compression scheme for LTO that it fails to find later. What is the configuration line? |
|
BTW .. this "issue" seems to have drifted far from its title
Having said that, I'm not sure we're seeing a GCC issue here. |
Thank you very much for detailed reply. Is this configuration with stock linker and assembler of 10.5.8 PPC? Or your rebuilt tools? I will try building on 10.5.8 PPC exactly with your settings to make sure that works, and once successful, then will work on adapting it for 10.6 PPC. P. S. I will update on the config and result of the build once back to office. |
|
@barracuda156 -- if you would like to open a ticket on the MacPorts tracker and call it something like "libgcc - update to gcc8 (or newer) on systems less than 10.6" we will work with you on getting gcc built. For the moment, you are really going around in circles here still. Forget about 10.6 on PPC for the time being. Not only is is non-standard, but it will not work right with either gcc or cctools or ld64 without some knowledgable surgery, because what these tools expect to find on a 10.6 system (certain symbols in the system library for example) do not exist on 10.6 on PPC, and many things break / segfault / just don't work. Your best bet to get smoothly started is to use an Intel system, something between 10.6 and 10.13 if you have it, and just learn how to build gcc correctly even one time. Once you have that, you can build a few different versions of gcc. Once you know how to do that, you can try building gcc5 or something very simple on 10.5 PPC. And once you can do that, you are ready to think about gcc8, gcc9, etc. gcc11 is where it starts to get harder, because it won't build with the default toolchain on 10.5 PPC -- so don't even try that for now, until everything else is easy for you. Forget about libtapi, etc, for the time being. You will spend dozens of hours on that, and you don't need to. When it comes time to build gcc11 on 10.5 PowerPC, you will use MacPorts updated cctools and ld64, but not before then! by the end of this, you will be a big gcc expert. But right now, getting gcc5 to just build on a reasonably new Intel system would be a nice place to start. |
|
@kencu I have actually built gcc11 on Catalina (outside Macports), used XcodeLegacy to extract PPC components and install them into Xcode 12.4 and then almost built cross-compiler for PCC with gcc11. It built but some tests fail. |
|
I am very glad to hear! I hope you can build on that success! |
Based on @NapalmSauce 's comments in #607, we need new
libgcc,libatomicandlibstdc++for G3, 7400, 7450 and G5 built-O3for 10.4.The text was updated successfully, but these errors were encountered: