Running docker image on Chrome OS on ARM64 host

[twilightsorcerer]

Trying to run this docker image on this host. Details are as follows:

OS details:
Google Chrome: Version 81.0.4044.94 (Official Build) (32-bit)
Platform: 12871.67.0 (Official Build) stable-channel elm
Channel: stable-channel
Firmware Version: Google_Elm.8438.184.0
ARC enabled: true
ARC: 6357842
Enterprise enrolled: false
Developer mode: false

Docker version:
Client: Docker Engine - Community
Version: 19.03.5
API version: 1.40
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:28:15 2019
OS/Arch: linux/arm64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 19.03.5
API version: 1.40 (minimum version 1.12)
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:26:47 2019
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.2.10
GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
runc:
Version: 1.0.0-rc8+dev
GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
docker-init:
Version: 0.18.0
GitCommit: fec3683

Docker image:
https://github.com/pauleve/docker-mtgo.git

[umarcor]

Hi @twilightsorcerer! I don't understand the purpose of this issue. Are you asking about how to achieve it? Are you reporting that it works/fails?

[twilightsorcerer]

Hi @umarcor ! I've been having some trouble achieving it, and was recommended here as a place to get advice, if that's possible? Thanks :)

[umarcor]

Sure! Basically, as per Usage, you need to execute docker run --rm --privileged aptman/qus -s -- -p x86_64 (or i386). Then, you can use any regular docker/podman container as you would on your PC.

For example, this was tested on a RPi:

# docker run --rm -it amd64/ubuntu:bionic bash
standard_init_linux.go:211: exec user process caused "exec format error"
...

# docker run --rm --privileged aptman/qus -s -- -p x86_64
...

# docker run --rm -it amd64/ubuntu:bionic bash
...
root@sha# apt update -qq
...
root@sha# apt install -y binutils
...
root@sha# readelf -h $(which ls)
...
Machine: Advanced Micro Devices X86-64
...

So, for your use case, I guess it should be:

# docker run --rm --privileged aptman/qus -s -- -p x86_64
# ./run-mtgo

However, I have not checked the details of run-mtgo. I don't know if qemu-user is able to handle the complexity of the tool(s) that are to be setup/executed.

BTW, you might find x11docker to be an interesting replacement for you custom X and audio settings. Unfortunately, macOS is not supported, tho.

[twilightsorcerer]

Hi, thanks for the help. When executing
docker run --rm --privileged aptman/qus -s -- -p i386
I get the error
cat ./qemu-binfmt-conf.sh | sh -s -- --path=/qus/bin -p x86_6 --suffix -static
mount: permission denied (are you root?)
even when using sudo/being logged in as superuser. Is there a fix for this?

[umarcor]

The entrypoint is register.sh and there is no command which could produce an error in there. Hence, it seems that the error is produced in qemu-binfmt-conf.sh. Likely in https://github.com/umarcor/qemu/blob/master/scripts/qemu-binfmt-conf.sh#L232, which is the only mount command.

• Does /proc/sys/fs/binfmt_misc/ exist on your host? Is it empty?
• Can you execute mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc on the host?
• Maybe you need to install binfmt-support on the host?

[twilightsorcerer]

binfmt-support is installed, and /proc/sys/fs/binfmt_misc/ does exist and is empty.
When executing mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc, this error is returned:
mount: permission denied

[umarcor]

Can you install qemu-user-static on the host? It should not be required, but it would help us debug the context.

[twilightsorcerer]

qemu-user-static has been installed.

[umarcor]

Is /proc/sys/fs/binfmt_misc/ still empty? What if you execute update-binfmts?

[twilightsorcerer]

/proc/sys/fs/binfmt_misc is still empty, update-binfmts returns:
update-binfmts: you must use one of --install, --remove, --import, --display, --enable, --disable, --find
Try 'update-binfmts --help' or 'update-binfmts --usage' for more information.

[umarcor]

Per https://wiki.debian.org/QemuUserEmulation: update-binfmts --display

[twilightsorcerer]

Whoops, sorry.

python3.5 (disabled):
package = python3.5
type = magic
offset = 0
magic = \x17\x0d\x0d\x0a
mask =
interpreter = /usr/bin/python3.5
detector =
python2.7 (disabled):
package = python2.7
type = magic
offset = 0
magic = \x03\xf3\x0d\x0a
mask =
interpreter = /usr/bin/python2.7
detector =
jar (disabled):
package = openjdk-8
type = magic
offset = 0
magic = PK\x03\x04
mask =
interpreter = /usr/bin/jexec
detector =

[umarcor]

I just checked you first message, are you running Chrome OS?

[twilightsorcerer]

Yes, will it not run on this OS?

[umarcor]

I don't know. I have never used it, neither do I have access to any device where to test it... This project has been tested on Docker for Windows and common GNU/Linux distributions (Fedora, Debian). For amd64-on-aarch64, Raspbian was used.

Anyway, before trying to use QEMU + Docker at the same time, I think you should try to test whether you can achieve it without Docker. In the end, the purpose of this project is to avoid installing QEMU locally, but we cannot do much if binfmt does not work in first place. Unfortunately, all I could find was https://www.reddit.com/r/linuxquestions/comments/5vfb2r/updatebinfmts_warning_couldnt_load_the_binfmt/

Just blind guessing, was qemu-user-binfmt installed along with qemu-user?

[twilightsorcerer]

Ah okay, oh well. Yes, I believe qemu-user-binfmt was installed along with qemu-user. Thanks for the help.

[umarcor]

You are welcome, and I'm sorry I could not help any further. Please, leave this issue open so that other users in the same context can hopefully see it. Moreover, should you get to try the docker image on any arm64 host with a different host OS, please let me know.

[pauleve]

Sorry to pop up in the conversation, but I found this http://www.courville.org/home/rk3288-arm-chromebook with potential useful tricks (there are sections related to making run wine-i386)

[umarcor]

@pauleve, merci beacoup! That's very interesting, indeed.

@twilightsorcerer, according to section "The qemu way" of @pauleve's reference, qemu-user-static might work with some tweaks:

• It seems that installing a Debian in a chroot is required. However, I'm not sure whether this is because of QEMU itself, or in order to use Wine.
• It seems that the binfmt_misc kernel module might need to be recompiled on older versions of the OS.
• The mount path for binfmt_misc is different: https://github.com/qemu/qemu/blob/master/scripts/qemu-binfmt-conf.sh#L232 and mount -t binfmt_misc none /var/binfmt_misc.
  • The qemu_register_interpreter function also needs to be updated accordingly: https://github.com/qemu/qemu/blob/master/scripts/qemu-binfmt-conf.sh#L274

You can generate the "magic" string to register a specific target manually with function qemu_generate_register: https://github.com/qemu/qemu/blob/master/scripts/qemu-binfmt-conf.sh#L269.

I would suggest to first try "Run i386 binaries on armhf architecture". If that works, I'd try to use a locally modified version of qemu-binfmt-conf.sh, as explained in #4 (comment).

[twilightsorcerer]

Hi @umarcor , and thanks, @pauleve .
I have a debian installed in a chroot (xfce4, ubuntu 16.04.6 LTS).
When executing apt install zlib1g:i386 from 'Run i386 binaries on armhf architecture', it gives the error:
Package zlib1g:i386 is not available, but is referred to by another package.

This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'zlib1g:i386' has no installation candidate

As for recompiling binfmt_misc, how would I do that, and how would I know if I need to?

[umarcor]

Did you try qemu-i386-static ./i386_binary_to_run as suggested in section "Run i386 binaries on armhf architecture"?

As for recompiling binfmt_misc, how would I do that, and how would I know if I need to?

It's explained in sections "Recompile own kernel or specific kernel modules" and "The qemu way".