Skip to content

Commit

Permalink
Merge branch 'andrew/organize-dockerfiles' into 'master'
Browse files Browse the repository at this point in the history 
Chore(NODE-1387): Clean up and organize dockerfiles

Get the Dockerfiles in a place so that the next steps are easier:
1. Extract IC-OS docker changes to a later point in the build
2. Consolidating the common Docker logic

And make the files more readable along the way.

----------

Diffing the two `rootfs-tree.tar` files (from built on master vs current commit):
        * $ sha256sum guestos/newest-rootfs-tree.tar guestos/old-cont-rootfs-tree.tar; sha256sum hostos/newest-2-rootfs-tree.tar hostos/old-cont-rootfs-tree.tar; sha256sum setupos/newest-rootfs-tree.tar setupos/old-cont-rootfs-tree.tar;
            * 9dc98fe5531f1134e6d372fc1255560565948b6977d5891691ee0ddad2bece5d  guestos/newest-rootfs-tree.tar
            * 9dc98fe5531f1134e6d372fc1255560565948b6977d5891691ee0ddad2bece5d  guestos/old-cont-rootfs-tree.tar
            * dbe50072a6f42cf18249fe828c21b2c9abf191c0d0a0b038ad24a2b5ba08c4ef  hostos/newest-2-rootfs-tree.tar
            * c56b5b5b2163729ccd2a5ec436f5e0b796eb1fe25e0dd65e28831964a588b668  hostos/old-cont-rootfs-tree.tar
            * 07778daff7e6bdb9fb06f901f143bc79dd7eb04156f17ba7141a457f15fd68c4  setupos/newest-rootfs-tree.tar
            * 07778daff7e6bdb9fb06f901f143bc79dd7eb04156f17ba7141a457f15fd68c4  setupos/old-cont-rootfs-tree.tar

HostOS changes are a result of moving updates to /etc/systemd/system-generators/* in-line with GuestOS and SetupOS 

See merge request dfinity-lab/public/ic!19374
  • Loading branch information
@andrewbattat
andrewbattat committed May 30, 2024
2 parents 6400d67 + ff4f7e3 commit e3fff91
Show file tree
Hide file tree
Showing 3 changed files with 153 additions and 137 deletions.
110 changes: 57 additions & 53 deletions ic-os/guestos/context/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
# Build steps:
# - `docker build --pull -t dfinity/guestos-main -f Dockerfile .`

# ------ NEAR-COMMON OS WORK ----------------------------------------

# The base images are defined in docker-base.prod and docker-base.dev. Update
# the references there when a new base image has been built. Note that this
# argument MUST be given by the build script, otherwise build will fail.
Expand All @@ -11,10 +13,7 @@ ARG BASE_IMAGE=
# We support prod and dev images
ARG BUILD_TYPE=

#
# First build stage:
# - Construct the actual target image (IC-OS root filesystem)
#

FROM $BASE_IMAGE as output_prod

USER root:root
Expand All @@ -24,19 +23,40 @@ RUN mkdir -p /boot/config \
/boot/grub
COPY etc /etc

# Change the permissions of /etc/{hosts,hostname,resolv.conf} in the rootfs source dir
# as they get (re)copied to the container.
# Deactivate motd, it tries creating $HOME/.cache/motd.legal-displayed,
# but we want to prohibit it from writing to user home dirs
RUN sed -e '/.*pam_motd.so.*/d' -i /etc/pam.d/login && \
sed -e '/.*pam_motd.so.*/d' -i /etc/pam.d/sshd

# Deactivate lvm backup/archive: It writes backup information to /etc/lvm,
# but this is per system (so backups are not persisted across upgrades)
# and thus not very useful, and /etc is read-only.
# So simply suppress generating backups.
RUN sed -e 's/\(backup *= *\)1/\10/' -e 's/\(archive *= *\)1/\10/' -i /etc/lvm/lvm.conf

# Deactivate systemd userdb. We don't use it.
RUN sed -e 's/ *systemd//' -i /etc/nsswitch.conf

# Clear files that may lead to indeterministic build.
RUN apt-get clean && \
find /usr/lib/python3.8 -name "*.pyc" | xargs rm && \
find /usr/lib/python3 -name "*.pyc" | xargs rm && \
find /usr/share/python3 -name "*.pyc" | xargs rm && \
truncate --size 0 /etc/machine-id

# Container engines do wacky things with these files - they bind mount them to reflect
# what's on the host. The actual rootfs files do indeed get copied to the result tar
# file. But changes to them by the RUN command don't have effect. COPY --chmod seems to
# work
# ------ NEAR-COMMON ICOS WORK ----------------------------------------

# Ensure correct permissions for /etc/{hosts, hostname, resolv.conf}.
# Container engines bind mount these files to use the host's versions,
# so changes made with RUN commands don't affect the final image.
# Using COPY --chmod ensures the files are copied with the right permissions.
COPY --chmod=644 etc/hosts /etc/hosts
COPY --chmod=644 etc/hostname /etc/hostname
COPY --chmod=644 etc/resolv.conf /etc/resolv.conf

# TODO: [NODE-1348] We overwrite all /etc files with 644 except for the specified.
# This may unintentionally drop the execute bit on some files.
# Update POSIX permissions in /etc/
# TODO: [NODE-1348] Some of the original execute bits are lost here
RUN find /etc -type d -exec chmod 0755 {} \+ && \
find /etc -type f -not -path "/etc/hostname" -not -path "/etc/hosts" -not -path "/etc/resolv.conf" -exec chmod 0644 {} \+ && \
chmod 0755 /etc/systemd/system-generators/* && \
Expand All @@ -46,37 +66,19 @@ RUN find /etc -type d -exec chmod 0755 {} \+ && \
chmod 755 /etc/initramfs-tools/scripts/init-premount/verity-root && \
chmod 755 /etc/initramfs-tools/hooks/veritysetup

# Deactivate motd, it tries creating $HOME/.cache/motd.legal-displayed,
# but we want to prohibit it from writing to user home dirs
RUN sed -e '/.*pam_motd.so.*/d' -i /etc/pam.d/login && \
sed -e '/.*pam_motd.so.*/d' -i /etc/pam.d/sshd

# Deactivate lvm backup/archive: It writes backup information to /etc/lvm, but a) this is
# per system (so backups are not persisted across upgrades) and thus not very
# useful, and b) we want to turn /etc read-only eventually. So simply suppress
# generating backups.
RUN sed -e 's/\(backup *= *\)1/\10/' -e 's/\(archive *= *\)1/\10/' -i /etc/lvm/lvm.conf
# Regenerate initramfs (config changed after copying in /etc)
RUN RESUME=none update-initramfs -c -k all

# Deactivate systemd userdb. We don't use it.
RUN sed -e 's/ *systemd//' -i /etc/nsswitch.conf
COPY prep /prep
RUN cd /prep && ./prep.sh && cd / && rm -rf /prep

# Activate the NSS IC OS Name Service Switch plugin.
# See ../../../rs/ic_os/nss_icos/README.md for context.
RUN sed -r -e 's/hosts:( *)files/hosts:\1files icos/' -i /etc/nsswitch.conf

# Divert symbolic link for dynamically generated nftables
# ruleset.
RUN ln -sf /run/ic-node/nftables-ruleset/nftables.conf /etc/nftables.conf

# Regenerate initramfs (config changed after copying in /etc)
RUN RESUME=none update-initramfs -c -k all

# Prepare for bind mount of authorized_keys
RUN mkdir -p /root/.ssh && chmod 0700 /root/.ssh

COPY prep /prep
RUN cd /prep && ./prep.sh && cd / && rm -rf /prep

# Delete generated ssh keys, otherwise every host will have the same key pair.
# They will be generated on first boot.
RUN rm /etc/ssh/ssh*key*
Expand All @@ -86,17 +88,6 @@ RUN rm /etc/ssh/ssh*key*
# strictly necessary to be explicit here.
RUN sed -e "s/.*PermitRootLogin.*/PermitRootLogin prohibit-password/" -i /etc/ssh/sshd_config

# All of the above sets up the base operating system. Everything below relates
# to node operation.

# Mount points for data storage.
RUN mkdir -p /var/lib/ic/backup \
/var/lib/ic/crypto \
/var/lib/ic/data

# Create two mount points for temporary use during setup of "var" partition
RUN mkdir -p /mnt/var_old /mnt/var_new

RUN \
for SERVICE in /etc/systemd/system/*; do \
if [ -f "$SERVICE" -a ! -L "$SERVICE" ] && grep -q '^.Install.' "$SERVICE" ; then systemctl enable "${SERVICE#/etc/systemd/system/}" ; fi ; \
Expand All @@ -120,6 +111,24 @@ RUN systemctl disable \
fstrim.service \
fstrim.timer

# Clear additional files that may lead to indeterministic build.
RUN find /usr/local/share/fonts -name .uuid | xargs rm && \
find /usr/share/fonts -name .uuid | xargs rm

# ------ GUESTOS WORK --------------------------------------------

# Divert symbolic link for dynamically generated nftables
# ruleset.
RUN ln -sf /run/ic-node/nftables-ruleset/nftables.conf /etc/nftables.conf

# Mount points for data storage.
RUN mkdir -p /var/lib/ic/backup \
/var/lib/ic/crypto \
/var/lib/ic/data

# Create two mount points for temporary use during setup of "var" partition
RUN mkdir -p /mnt/var_old /mnt/var_new

# Add user/group entries specified here: /usr/lib/sysusers.d/systemd.conf E.g., systemd-timesync/coredump.
## `systemd-sysusers` does not honor the SOURCE_DATE_EPOCH env var.
## With `podman --timestamp=0` each file gets written with timestamp of the Unix epoch.
Expand Down Expand Up @@ -219,14 +228,7 @@ RUN addgroup node_exporter && \
RUN adduser --system --disabled-password --home /var/lib/metrics-proxy --group --no-create-home metrics-proxy && \
usermod -a -G node_exporter metrics-proxy

# Clear all files that may lead to indeterministic build.
RUN apt-get clean && \
find /usr/local/share/fonts -name .uuid | xargs rm && \
find /usr/share/fonts -name .uuid | xargs rm && \
find /usr/lib/python3.8 -name "*.pyc" | xargs rm && \
find /usr/lib/python3 -name "*.pyc" | xargs rm && \
find /usr/share/python3 -name "*.pyc" | xargs rm && \
truncate --size 0 /etc/machine-id
# ------ INSTALL SCRIPTS -----------------------------------------

# Install IC binaries and other data late -- this means everything above
# will be cached when only the binaries change.
Expand All @@ -238,6 +240,8 @@ RUN find /opt -type d -exec chmod 0755 {} \+ && \
chmod 0755 /opt/ic/bin/* && \
chmod 0644 /opt/ic/share/*

# ------ DEV VARIANT ---------------------------------------------

# The following steps apply conditionally to the dev image ONLY
# https://www.docker.com/blog/advanced-dockerfiles-faster-builds-and-smaller-images-using-buildkit-and-multistage-builds/#4374
FROM output_prod as output_dev
Expand All @@ -256,7 +260,7 @@ COPY dev-certs/canister_http_test_ca.cert /usr/local/share/ca-certificates/dev-r
RUN chmod 0644 /usr/local/share/ca-certificates/dev-root-ca.crt
RUN update-ca-certificates


FROM output_${BUILD_TYPE}

USER root:root

98 changes: 52 additions & 46 deletions ic-os/hostos/context/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,15 @@
#
# Build steps:
# - `docker build --pull -t dfinity/hostos-main -f Dockerfile .`
#

# ------ NEAR-COMMON OS WORK ----------------------------------------

# The base images are defined in docker-base.prod and docker-base.dev. Update
# the references there when a new base image has been built. Note that this
# argument MUST be given by the build script, otherwise build will fail.
ARG BASE_IMAGE=

#
# First build stage:
# - Construct the actual target image (IC-OS root filesystem)
#

FROM $BASE_IMAGE

USER root:root
Expand All @@ -22,45 +20,55 @@ RUN mkdir -p /boot/config \
/boot/grub
COPY etc /etc

# Change permissions on special files as they are copied.
# See ic-os/hostos/context/Dockerfile for details
COPY --chmod=644 etc/hosts /etc/hosts
COPY --chmod=644 etc/hostname /etc/hostname
COPY --chmod=644 etc/resolv.conf /etc/resolv.conf

# Update POSIX permissions in /etc/
RUN find /etc -type d -exec chmod 0755 {} \+ && \
find /etc -type f -not -path "/etc/hostname" -not -path "/etc/hosts" -not -path "/etc/resolv.conf" -exec chmod 0644 {} \+ && \
chmod 0755 /etc/systemd/system-generators/mount-generator && \
chmod 0440 /etc/sudoers && \
chmod 755 /etc/initramfs-tools/scripts/init-bottom/set-machine-id

# Deactivate motd, it tries creating $HOME/.cache/motd.legal-displayed,
# but we want to prohibit it from writing to user home dirs
RUN sed -e '/.*pam_motd.so.*/d' -i /etc/pam.d/login && \
sed -e '/.*pam_motd.so.*/d' -i /etc/pam.d/sshd

# Deactivate lvm backup/archive: It writes backup information to /etc/lvm, but a) this is
# per system (so backups are not persisted across upgrades) and thus not very
# useful, and b) we want to turn /etc read-only eventually. So simply suppress
# generating backups.
# Deactivate lvm backup/archive: It writes backup information to /etc/lvm,
# but this is per system (so backups are not persisted across upgrades)
# and thus not very useful, and /etc is read-only.
# So simply suppress generating backups.
RUN sed -e 's/\(backup *= *\)1/\10/' -e 's/\(archive *= *\)1/\10/' -i /etc/lvm/lvm.conf

# Deactivate systemd userdb. We don't use it.
RUN sed -e 's/ *systemd//' -i /etc/nsswitch.conf

# Activate the NSS IC OS Name Service Switch plugin.
# See ../../../rs/ic_os/nss_icos/README.md for context.
RUN sed -r -e 's/hosts:( *)files/hosts:\1files icos/' -i /etc/nsswitch.conf
# Compile locale specification
RUN localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8

# Clear files that may lead to indeterministic build.
RUN apt-get clean && \
find /usr/lib/python3.8 -name "*.pyc" | xargs rm && \
find /usr/lib/python3 -name "*.pyc" | xargs rm && \
find /usr/share/python3 -name "*.pyc" | xargs rm && \
truncate --size 0 /etc/machine-id

# ------ NEAR-COMMON ICOS WORK ----------------------------------------

# Ensure correct permissions for /etc/{hosts, hostname, resolv.conf}.
# Container engines bind mount these files to use the host's versions,
# so changes made with RUN commands don't affect the final image.
# Using COPY --chmod ensures the files are copied with the right permissions.
COPY --chmod=644 etc/hosts /etc/hosts
COPY --chmod=644 etc/hostname /etc/hostname
COPY --chmod=644 etc/resolv.conf /etc/resolv.conf

# TODO: [NODE-1348] We overwrite all /etc files with 644 except for the specified.
# This may unintentionally drop the execute bit on some files.
# Update POSIX permissions in /etc/
RUN find /etc -type d -exec chmod 0755 {} \+ && \
find /etc -type f -not -path "/etc/hostname" -not -path "/etc/hosts" -not -path "/etc/resolv.conf" -exec chmod 0644 {} \+ && \
chmod 0755 /etc/systemd/system-generators/* && \
chmod 0440 /etc/sudoers && \
chmod 755 /etc/initramfs-tools/scripts/init-bottom/set-machine-id

# Regenerate initramfs (config changed after copying in /etc)
RUN RESUME=none update-initramfs -c -k all

ARG ROOT_PASSWORD=
RUN \
if [ "${ROOT_PASSWORD}" != "" ]; then \
echo "root:$(openssl passwd -6 -salt jE8zzDEHeRg/DuGq ${ROOT_PASSWORD})" | chpasswd -e ; \
fi
# Activate the NSS IC OS Name Service Switch plugin.
# See ../../../rs/ic_os/nss_icos/README.md for context.
RUN sed -r -e 's/hosts:( *)files/hosts:\1files icos/' -i /etc/nsswitch.conf

# Prepare for bind mount of authorized_keys
RUN mkdir -p /root/.ssh && chmod 0700 /root/.ssh
Expand All @@ -74,9 +82,6 @@ RUN rm /etc/ssh/ssh*key*
# strictly necessary to be explicit here.
RUN sed -e "s/.*PermitRootLogin.*/PermitRootLogin prohibit-password/" -i /etc/ssh/sshd_config

# All of the above sets up the base operating system. Everything below relates
# to node operation.

RUN \
for SERVICE in /etc/systemd/system/*; do \
if [ -f "$SERVICE" -a ! -L "$SERVICE" ] && grep -q '^.Install.' "$SERVICE" ; then systemctl enable "${SERVICE#/etc/systemd/system/}" ; fi ; \
Expand All @@ -101,13 +106,25 @@ RUN systemctl disable \
motd-news.service \
motd-news.timer

# ------ HOSTOS WORK ---------------------------------------------

ARG ROOT_PASSWORD=
RUN \
if [ "${ROOT_PASSWORD}" != "" ]; then \
echo "root:$(openssl passwd -6 -salt jE8zzDEHeRg/DuGq ${ROOT_PASSWORD})" | chpasswd -e ; \
fi

# Clear additional files that may lead to indeterministic build.
RUN rm -rf \
/usr/local/share/qemu/edk2-arm-code.fd \
/usr/local/share/qemu/edk2-arm-vars.fd

# Add user/group entries specified here: /usr/lib/sysusers.d/systemd.conf E.g., systemd-timesync/coredump
RUN systemd-sysusers && \
# Fix reproducibility issue. Notes in hostos/context/Dockerfile
usermod -p '!!' systemd-timesync && \
usermod -p '!!' systemd-coredump


# Set /bin/sh to point to /bin/bash instead of the default /bin/dash
RUN echo "set dash/sh false" | debconf-communicate && dpkg-reconfigure -fnoninteractive dash

Expand Down Expand Up @@ -171,18 +188,7 @@ RUN addgroup node_exporter && \
RUN adduser --system --disabled-password --home /var/lib/metrics-proxy --group --no-create-home metrics-proxy && \
usermod -a -G node_exporter metrics-proxy

# Compile locale specification
RUN localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8

# Clear all files that may lead to indeterministic build or are large and unnecessary
RUN apt-get clean && \
rm -rf \
/usr/local/share/qemu/edk2-arm-code.fd \
/usr/local/share/qemu/edk2-arm-vars.fd && \
find /usr/lib/python3.8 -name "*.pyc" | xargs rm && \
find /usr/lib/python3 -name "*.pyc" | xargs rm && \
find /usr/share/python3 -name "*.pyc" | xargs rm && \
truncate --size 0 /etc/machine-id
# ------ INSTALL SCRIPTS -----------------------------------------

# Install IC binaries and other data late -- this means everything above
# will be cached when only the binaries change.
Expand Down
Loading

0 comments on commit e3fff91

Please sign in to comment.