Optimized build fails for ARM architecture

[abhi18av]

Hi @mfikes

I really love using planck on my machine which is a traditional x86_64 but recently I was curious whether planck would build on my AArch64 / ARMv8 server and tested it out today.

I can confirm that the fast build finishes but running script/test ends up in a segmentation fault and the normal script/build fails to complete with the following message

0 error(s), 2 warning(s)
........
### Building 1st stage Planck binary
### AOT compiling macro namespaces
### Bundling ClojureScript artifacts for 2nd stage
### Optimizing bundled JavaScript with Closure Optimizations: SIMPLE
..........script/bundle: line 97: 11627 Segmentation fault      (core dumped) ../../planck-c/build/planck ../script/decode.cljs $file > $file.aot
Build Failed.

It'd be great if planck could work with this architecture as this would mean a good functionality on the Raspberry Pi as well - which I plan on using for the educational purpose.

Could you please help me out a bit?

[mfikes]

Hi @abhi18av I'm wondering if I'll be able to reproduce this. I suppose I'd need to get some ARM hardware.

Without that, one thing to try is to set things to produce a debug build. See https://github.com/planck-repl/planck/blob/master/planck-c/CMakeLists.txt#L4

Then with a debug build (using --fast which avoids using the planck binary to optimize its own bundled files), you might be able to see where the problem occurs (by running it using a debugger like gdb or some such).

[abhi18av]

Hi @mfikes, well I could give you the access to my server so that you could test things out :)

My email id is mentioned on my profile [email protected] and you could let me know the instructions about how I could enable you to access it.

I'm just relying on the scaleway armv8 server. https://www.scaleway.com/en/pricing/#virtual-instances

[pyrmont]

@abhi18av I just ran the following against the master branch on my Pi running Raspbian 9:

$ ./script/clean
$ ./script/build --fast
$ ./script/test

I had some of the tests fail for what look like unrelated reasons but no segmentation fault. My Pi has an ARMv7 chip so maybe it's something to do with ARMv8 or AArch64?

[mfikes]

@abhi18av I've created a Scaleway account, but I can't figure out how to create ARM instances via their web UI. I'm assuming that they don't have any currently to sell, unless I'm missing something obvious.

[mfikes]

I created an ARM-based Ubuntu 18.04 instance in AWS, and Planck built there without issue and all unit tests passed.

AMI description:
Canonical, Ubuntu, 18.04 LTS, ARM64 bionic image build on 2018-11-20

uname -a output:
Linux ip-172-31-90-138 4.15.0-1028-aws #29+nutmeg8-Ubuntu SMP Tue Nov 20 02:59:41 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux

I wonder if there is an AMI that more closely matches that Scaleway box...

[abhi18av]

@pyrmont and @mfikes hmm, interesting!

On the Scaleway server the uname -a output is

root@pup:~# uname -a
Linux pup 4.9.93-mainline-rev1 #1 SMP Tue Apr 10 09:54:46 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux

Well, if this turns out to be a peculiar bug then let's not spend so much time at this - I'd rather just use the compiled binary on the AWS image :)

@mfikes would it be possible for you to mail the zipped version on my email please?

[mfikes]

@abhi18av I'll make the binary available to you.

I'd like to actually get to the root of the bug. Could you tell me which choices you make in the Scaleway UI to create an ARM instance? (Or otherwise confirm that you see that this is no longer offered?)

[mfikes]

@abhi18av There is now an Ubuntu aarch64 binary on https://planck-repl.org/binaries/

[abhi18av]

Thanks @mfikes , I truly appreciate you going the extra mile here! I confirm that the new binary works just fine on the server.

Here are some screenshots, I took to document the process. This is right from where you land after logging in

Let me know if this helps :)

Also, please feel free to close the issue as it seems that this is a cloud-provider specific issue and we could only truly confirm by having a latest RPi with us.

[mfikes]

@abhi18av Thanks!

I'll follow up with Scaleway. Your pictures above are helpful. Here is what I see:

[abhi18av]

That is so strange! I don't know why that is so, maybe they are phasing out these servers or something but that seems unlikely as you don't even see the recently launched DEV servers. I'm not quite sure why this is so, Mike

…

________________________________ From: Mike Fikes <[email protected]> Sent: Friday, May 31, 2019 9:08:53 PM To: planck-repl/planck Cc: Abhinav Sharma; Mention Subject: Re: [planck-repl/planck] Optimized build fails for ARM architecture (#945) @abhi18av<https://github.com/abhi18av> Thanks! I'll follow up with Scaleway. Your pictures above are helpful. Here is what I see: [Napkin 05-31-19, 11 38 42 AM]<https://user-images.githubusercontent.com/1723464/58717185-a485a380-8398-11e9-862d-66d9b675161a.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#945?email_source=notifications&email_token=ADBU2XWSEKMC6TZUK7SLBSTPYFBA3A5CNFSM4HRGHAN2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWVSQVY#issuecomment-497756247>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ADBU2XVVQ6O5JN45REBJPW3PYFBA3ANCNFSM4HRGHANQ>.

[mfikes]

It turns out I evidently can't yet create ARM instances owing to quotas. I have a support ticket open with Scaleway.

[abhi18av]

Hmm, I see! Well, my invitation still stands if you wish to ssh into my own instance and please feel free to use it for the process.

[mfikes]

I think this repros on the 64-bit ARM boxes available for CI at Drone.io:

### AOT compiling macro namespaces
--
1269 | script/build: line 159: 17634 Segmentation fault (core dumped) planck-c/build/planck -sk planck-cljs/out/macros-tmp -e"(require 'cljs.analyzer)" -e"(do (set! cljs.analyzer/*cljs-warnings* (assoc cljs.analyzer/*cljs-warnings* :undeclared-var false)) nil)" -e "(require-macros 'planck.repl 'planck.core 'planck.shell 'planck.from.io.aviso.ansi 'clojure.template 'cljs.spec.alpha 'cljs.spec.test.alpha 'cljs.spec.gen.alpha 'cljs.test 'cljs.pprint 'cljs.analyzer.macros 'cljs.compiler.macros 'cljs.env.macros)"

[mfikes]

I got a reply from Scaleway that quotas for ARM hosts are not instantly available because they have a limited stock of our ARM instances and, for the moment we don't have plans to continue developing them.

[abhi18av]

Oh, that's really going to be inconvenient in future for sure!

The only other provider which offer ARM servers, which I know of, is packet.net but those are not economical. I'd like the students to have their own tiny inexpensive boxes in the cloud. Well, I'll figure something out, thanks for the update @mfikes !