Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

UKI Upgrade fails with Extended Command Line #2992

Open
bencorrado opened this issue Nov 11, 2024 · 7 comments
Open

UKI Upgrade fails with Extended Command Line #2992

bencorrado opened this issue Nov 11, 2024 · 7 comments
Labels
bug Something isn't working waiting for reporter

Comments

@bencorrado
Copy link
Contributor

Kairos version:

nerdnode@sparkly-maroon-pigeon:~$ cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.1 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo


nerdnode@sparkly-maroon-pigeon:~$ cat /etc/kairos-release
KAIROS_BUG_REPORT_URL="https://github.com/kairos-io/kairos/issues"
KAIROS_HOME_URL="https://github.com/kairos-io/kairos"
KAIROS_ID="kairos"
KAIROS_IMAGE_REPO="quay.io/kairos/ubuntu:24.04-standard-amd64-generic-83c0aef"
KAIROS_FLAVOR_RELEASE="24.04"
KAIROS_MODEL="generic"
KAIROS_RELEASE="83c0aef"
KAIROS_PRETTY_NAME="kairos-standard-ubuntu-24.04 83c0aef"
KAIROS_IMAGE_LABEL="24.04-standard-amd64-generic-83c0aef"
KAIROS_FLAVOR="ubuntu"
KAIROS_VARIANT="standard"
KAIROS_VERSION="83c0aef"
KAIROS_ID_LIKE="kairos-standard-ubuntu-24.04"
KAIROS_VERSION_ID="83c0aef"
KAIROS_ARTIFACT="kairos-ubuntu-24.04-standard-amd64-generic-83c0aef"
KAIROS_FAMILY="ubuntu"
KAIROS_NAME="kairos-standard-ubuntu-24.04"
KAIROS_TARGETARCH="amd64"
KAIROS_REGISTRY_AND_ORG="quay.io/kairos"
KAIROS_GITHUB_REPO="kairos-io/kairos"
KAIROS_SOFTWARE_VERSION_PREFIX="k3s"

CPU architecture, OS, and Version:

Linux sparkly-maroon-pigeon 6.8.0-47-generic #47-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 27 21:40:26 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Describe the bug
When applying a UKI image using sudo kairos-agent upgrade --source oci:<SOURCE> and using --extend-cmdline "ima_appraise=fix ima_template=ima-sig ima_policy=tcb" with enki while following https://kairos.io/v3.1.3/docs/upgrade/trustedboot/ the agent fails the installer as it is looking for /efi/EFI/Kairos/norole.efi which does not exist because it is named norole_install-mode_ima_appraise_fix_ima_template_ima-sig_ima_policy_tcb.efi

To Reproduce
On the build machine:

docker run -ti --rm -v $PWD/build:/result -v $PWD/keys/:/keys -v $PWD/custom/deeep:/splash enki:local build-uki registry.corrado.farm/test-bc-nov11:latest -t uki -d /result/upgrade -k /keys --boot-branding "DeEEP Network OS" --splash /splash/deeep.bmp --secure-boot-enroll force --extend-cmdline "ima_appraise=fix ima_template=ima-sig ima_policy=tcb"
docker run -ti --rm -v $PWD/build:/result -v $PWD/keys/:/keys -v $PWD/custom/deeep:/splash enki:local build-uki registry.corrado.farm/test-bc-nov11:latest -t container -d /result/upgrade -k /keys --boot-branding "DeEEP Network OS" --splash /splash/deeep.bmp --secure-boot-enroll force --extend-cmdline "ima_appraise=fix ima_template=ima-sig ima_policy=tcb"
docker load -i build/upgrade/*.tar
docker image tag kairos_uki_83c0aef.tar:latest registry.corrado.farm/deeep-os-upgrade:nov11-test
 docker push registry.corrado.farm/deeep-os-upgrade:nov11-test

On the target:
sudo kairos-agent upgrade --source oci:registry.corrado.farm/deeep-os-upgrade:nov11-test

Expected behavior
It should upgrade with the extended command line support.

Logs

nerdnode@sparkly-maroon-pigeon:~$ sudo kairos-agent upgrade --source oci:registry.corrado.farm/deeep-os-upgrade:nov11-test
warning: skipping /etc/kairos/branding/grubmenu.cfg (extension).
warning: skipping /etc/kairos/branding/install_text (extension).
warning: skipping /etc/kairos/branding/interactive_install_text (extension).
warning: skipping /etc/kairos/branding/recovery_text (extension).
warning: skipping /etc/kairos/branding/reset_text (extension).
warning: skipping /etc/kairos/versions.yaml because it has no valid header
warning: failed to parse config:
yaml: unmarshal errors:
  line 17: mapping key "boot" already defined at line 3
warning: skipping /oem/animalname (extension).
warning: skipping /oem/ap_certs/cert.pem (extension).
warning: skipping /oem/ap_certs/key.pem (extension).
warning: skipping /oem/identity (extension).
warning: skipping /oem/tailscale/derpmap.cached.json (extension).
warning: skipping /oem/tailscale/tailscaled.state (extension).
warning: skipping /oem/vpn_dns.yaml because it has no valid header
2024-11-11T19:08:38Z INF Kairos Agent version=v2.15.3
2024-11-11T19:08:38Z INF creating a runtime
2024-11-11T19:08:38Z INF detecting boot state
2024-11-11T19:08:38Z INF Boot Mode boot_mode=active_boot
2024-11-11T19:08:38Z INF Boot in uki mode result=true
2024-11-11T19:08:38Z INF Checking if OCI image registry.corrado.farm/deeep-os-upgrade:nov11-test exists
2024-11-11T19:08:38Z INF Setting image size to 1672Mb
2024-11-11T19:08:38Z INF Running stage: kairos-uki-upgrade.pre.before

2024-11-11T19:08:39Z INF Done executing stage 'kairos-uki-upgrade.pre.before'

2024-11-11T19:08:39Z INF Running stage: kairos-uki-upgrade.pre

2024-11-11T19:08:39Z INF Done executing stage 'kairos-uki-upgrade.pre'

2024-11-11T19:08:39Z INF Running stage: kairos-uki-upgrade.pre.after

2024-11-11T19:08:39Z INF Done executing stage 'kairos-uki-upgrade.pre.after'

2024-11-11T19:08:39Z INF Running stage: kairos-uki-upgrade.pre.before

2024-11-11T19:08:39Z INF Done executing stage 'kairos-uki-upgrade.pre.before'

2024-11-11T19:08:39Z INF Running stage: kairos-uki-upgrade.pre

2024-11-11T19:08:39Z INF Done executing stage 'kairos-uki-upgrade.pre'

2024-11-11T19:08:39Z INF Running stage: kairos-uki-upgrade.pre.after

2024-11-11T19:08:39Z INF Done executing stage 'kairos-uki-upgrade.pre.after'

2024-11-11T19:08:39Z INF installing entry: active
2024-11-11T19:08:39Z INF Copying registry.corrado.farm/deeep-os-upgrade:nov11-test source to /efi
2024-11-11T19:08:44Z INF Finished copying registry.corrado.farm/deeep-os-upgrade:nov11-test into /efi
2024-11-11T19:08:44Z INF Checking artifact for valid signature what=/efi/EFI/Kairos/norole.efi
2024-11-11T19:08:44Z WRN /efi/EFI/Kairos/norole.efi does not exist
2024-11-11T19:08:44Z ERR Checking signature before upgrading error="/efi/EFI/Kairos/norole.efi does not exist"
2024-11-11T19:08:44Z WRN Upgrade artifact signature does not match, upgrading to this source would result in an unbootable active system.
Check the upgrade source and confirm that its signed with a valid key, that key is in the machine DB and it has not been blacklisted.
1 error occurred:
	* /efi/EFI/Kairos/norole.efi does not exist


nerdnode@sparkly-maroon-pigeon:~$ cat /efi/EFI/Kairos/norole.efi
cat: /efi/EFI/Kairos/norole.efi: No such file or directory
nerdnode@sparkly-maroon-pigeon:~$ ls /efi/EFI/Kairos/
active.efi.extra.d							      passive.efi.extra.d							     recovery_install-mode_ima_appraise_fix_ima_template_ima-sig_ima_policy_tcb.efi
active_install-mode_ima_appraise_fix_ima_template_ima-sig_ima_policy_tcb.efi  passive_install-mode_ima_appraise_fix_ima_template_ima-sig_ima_policy_tcb.efi  statereset_install-mode_ima_appraise_fix_ima_template_ima-sig_ima_policy_tcb.efi

Additional context
@bencorrado bencorrado added bug Something isn't working triage Add this label to issues that should be triaged and prioretized in the next planning call unconfirmed labels Nov 11, 2024
@bencorrado
Copy link
Contributor Author

This is related to #2981

@bencorrado bencorrado changed the title UKI Upgrade fails with Extended Commandline UKI Upgrade fails with Extended Command Line Nov 11, 2024
@Itxaka
Copy link
Member

Itxaka commented Nov 12, 2024

umm nice, I think this scenario is something that we never tested, upgrading to a different cmdline artifact.

I wonder how we can fix this, search for norola and then fallback to norole_* ?

@mudler mudler moved this to Under review 🔍 in 🧙Issue tracking board Nov 18, 2024
@Itxaka
Copy link
Member

Itxaka commented Nov 25, 2024

while working on a different thing I stumbled that when we generate artifacts with a extended cmdline, for some unknown reasons we output the file and config with the extended cmdline in the name of the artifacts, which is wrong.

This explains this as the upgrade artifacts are not called norole.efi but probably something like norole_ima_appraise_fix_ima_template_ima_sig_ima_policy_tcb.efi

I believe this is a leftover from before we moved to use the norole.efi as we were storing active/passive/recovery efi files and we needed to differentiate from the "normal" or "default" files.

IMHO it makes no sense anymore as we do not ship any "generic" efi files along the extended cmdline ones, thye should be called norole.efi

It may make sense in the case of the --extra-cmdline as that generates extra artifacts along, but those also ship the default efi files

@Itxaka
Copy link
Member

Itxaka commented Nov 25, 2024

probably fixed by kairos-io/AuroraBoot#122

@Itxaka
Copy link
Member

Itxaka commented Nov 27, 2024

a new auroraboot will fix it @bencorrado

Seems like we were storing the extend-cmdline efi and configs with the extended cmdline in them, which made them not upgradable.

Basically on upgrade we search for an artifact called norole.efi and that wasnt the case on upgrade artifacts with extended-cmdline

If you are using aurora from master branch or quay.io/kairos/auroraboot:latest you will get this change in like 5 minutes when its finished building. If you build and upgrade artifcat with it, it should now be possible to upgrade with it

@Itxaka Itxaka moved this from Todo 🖊 to Under review 🔍 in 🧙Issue tracking board Nov 27, 2024
@Itxaka Itxaka removed the triage Add this label to issues that should be triaged and prioretized in the next planning call label Nov 27, 2024
@Itxaka
Copy link
Member

Itxaka commented Nov 27, 2024

Lets wait until Ben can confirm that this is indeed fixed or get some time to test it ourselves

@mudler
Copy link
Member

mudler commented Dec 23, 2024

@bencorrado can you confirm this is fixed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working waiting for reporter
Projects
🧙Issue tracking board
Status: Under review 🔍
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Itxaka @mudler @bencorrado