2281 - target manually partitioned disk #235

jimmykarily · 2024-02-23T09:49:30Z

codecov · 2024-02-23T09:52:19Z

Codecov Report

Attention: Patch coverage is 5.71429% with 33 lines in your changes are missing coverage. Please review.

Project coverage is 59.03%. Comparing base (6fe65c4) to head (d3bf4eb).

Files	Patch %	Lines
pkg/config/spec.go	3.84%	24 Missing and 1 partial ⚠️
pkg/action/install.go	14.28%	5 Missing and 1 partial ⚠️
internal/agent/install.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #235      +/-   ##
==========================================
- Coverage   59.20%   59.03%   -0.17%     
==========================================
  Files          39       39              
  Lines        4233     4248      +15     
==========================================
+ Hits         2506     2508       +2     
- Misses       1451     1463      +12     
- Partials      276      277       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

jimmykarily · 2024-02-23T09:52:32Z

This is just playing around with code to find where we break it.

I created a VM in qemu with 2 disks:

/dev/vda -> 80G
/dev/vdb -> 40G

Kairos would automatically pick /dev/vda because it's the bigger disk. My goal is to point the installation to a manually partitioned /dev/vdb.

With the (not-to-be-merged) changes in this PR and a config like the one below, I can successfully install on /dev/sdb

#cloud-config

strict: true
debug: true

install:
  no-format: true
  auto: true
  poweroff: false
  reboot: false
  grub_options:
    extra_cmdline: "rd.immucore.debug"
users:
  - name: "kairos"
    passwd: "kairos"

stages:
  kairos-install.pre.before:
  - if:  '[ -e "/dev/vdb" ]'
    name: "Create partitions"
    commands:
      - |
        parted --script --machine -- "/dev/vdb" mklabel gpt
        parted --script "/dev/vdb" mkpart primary Data 0 1MB
    layout:
      add_partitions:
        - fsLabel: COS_OEM
          size: 64
          pLabel: oem
        - fsLabel: COS_RECOVERY
          size: 8500
          pLabel: recovery
        - fsLabel: COS_STATE
          size: 18000
          pLabel: state
        - fsLabel: COS_PERSISTENT
          pLabel: persistent
          size: 0
          filesystem: "ext4"
  boot:
    - systemd_firstboot:
      keymap: us

I manually create a partition of 1Mb to allow grub to install. Putting it to the add_partitions key will also format it as ext2 (and possibly other default) which make grub complain.

internal/agent/install.go

jimmykarily · 2024-02-23T09:54:45Z

internal/agent/install.go

-	installSpec.NoFormat = c.Install.NoFormat
+	fmt.Printf("!!!!!!!!!! installSpec.NoFormat = %+v\n", installSpec.NoFormat)
+	fmt.Printf("!!!!!!!!!! c.Install.NoFormat = %+v\n", c.Install.NoFormat)
+	//installSpec.NoFormat = c.Install.NoFormat


This line sets NoFormat to false although in my config I set it to true

jimmykarily · 2024-04-05T14:02:32Z

this is an alternative command for the grub partition:

        sgdisk --new=1:2048:+1M --change-name=1:'bios' --typecode=1:EF02 /dev/vdb # for grub

jimmykarily · 2024-04-05T16:11:52Z

The binary from this PR makes the added test pass: kairos-io/kairos#2291

jimmykarily · 2024-04-08T09:47:50Z

config.yaml

@@ -0,0 +1,46 @@
+#cloud-config


This sample is now part of the test in kairos: https://github.com/kairos-io/kairos/pull/2291/files#diff-1ff1699e612ac7f8c508e5f9f6a784b37441b01b8cfdebd8da3b068280385247R87

I'll remove it from here.

jimmykarily · 2024-04-08T12:59:52Z

internal/agent/install.go

@@ -213,10 +213,6 @@ func RunInstall(c *config.Config) error {
 	utils.SetEnv(c.Env)
 	utils.SetEnv(c.Install.Env)

-	if c.Install.Device == "" || c.Install.Device == "auto" {
-		c.Install.Device = detectDevice()


moved here: https://github.com/kairos-io/kairos-agent/pull/235/files#diff-53b724197bc14c3828ce6725322b892cb788860b0526184b63d7d114edb60941R571

jimmykarily · 2024-04-08T15:42:23Z

pkg/uki/install.go

-	if err != nil {
-		i.cfg.Logger.Errorf("partitioning and formating devices: %s", err.Error())
-		return err
+	if i.spec.NoFormat {


I'm trying to add a test for the UKI case too here: kairos-io/kairos#2291
to be complete.

Manually creating the disk layout that the UKI installation creates is tricky:

[kairos@fedora ~]$ sudo blkid /dev/sr0: BLOCK_SIZE="2048" UUID="2024-04-09-09-34-22-00" LABEL="UKI_ISO_INSTALL" TYPE="iso9660" /dev/loop0: UUID="B9E2-794E" BLOCK_SIZE="512" TYPE="vfat" /dev/vda2: UUID="8bfa06f9-ca4f-56dc-90c9-49cf20f4f45e" TYPE="crypto_LUKS" PARTLABEL="oem" PARTUUID="d0be3cb8-e04d-4bc9-9bde-311a516eece2" /dev/vda3: UUID="85c39d0f-4867-5227-8334-f5eec606d9eb" TYPE="crypto_LUKS" PARTLABEL="persistent" PARTUUID="bc2d3f7d-54c6-4fc9-9918-595b19129d50" /dev/vda1: LABEL_FATBOOT="COS_GRUB" LABEL="COS_GRUB" UUID="E9F3-549E" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="efi" PARTUUID="feb38d63-49d4-4b2d-a1d6-2f340a27e4c5"

We can keep the code around here but writing a test for it is rather complex (creating a layout with encrypted partitions using the tpm). I'm not sure if anyone will ever be interested in this feature in UKI mode.

Fixes kairos-io/kairos#2281 Signed-off-by: Dimitris Karakasilis <[email protected]>

Signed-off-by: Dimitris Karakasilis <[email protected]>

Itxaka

code looks good. Im a bit worried about the auto select and no format stuff, but if it passes tests then its good to me. cant break no-format anymore :D

jimmykarily · 2024-04-09T10:12:32Z

code looks good. Im a bit worried about the auto select and no format stuff, but if it passes tests then its good to me. cant break no-format anymore :D

I didn't run the full test suite, only the new test. The only thing I can say is that it fixes no-format :D .
Let's try this: https://github.com/kairos-io/kairos/pull/2291/files#diff-2705f6016a64189abd14041e2f3e8dbb829553295d9088e4eb42eccb5e61bf47R299

jimmykarily · 2024-04-10T09:04:06Z

Depends on this: mudler/yip#141

otherwise this config won't work:

#cloud-config

strict: true
debug: true

install:
  no-format: true
  auto: false
  poweroff: false
  reboot: false
  grub_options:
    extra_cmdline: "rd.immucore.debug"

users:
  - name: "kairos"
    passwd: "kairos"

stages:
  kairos-install.pre.before:
  - if:  '[ -e "/dev/vdb" ]'
    name: "Create partitions"
    commands:
      - |
        parted --script --machine -- "/dev/vdb" mklabel gpt
        #sgdisk --new=1:2048:+1M --change-name=1:'bios' --typecode=1:EF02 /dev/vdb # for grub
    layout:
      device:
        path: "/dev/vdb"
      add_partitions:
        - fsLabel: COS_GRUB
          size: 64
          pLabel: efi
          filesystem: "fat"
        - fsLabel: COS_OEM
          size: 64
          pLabel: oem
        - fsLabel: COS_RECOVERY
          size: 8500
          pLabel: recovery
        - fsLabel: COS_STATE
          size: 18000
          pLabel: state
        - fsLabel: COS_PERSISTENT
          pLabel: persistent
          size: 0
          filesystem: "ext4"

We need this from yip: mudler/yip@4ebbc75 Signed-off-by: Dimitris Karakasilis <[email protected]>

jimmykarily · 2024-04-10T10:14:55Z

code looks good. Im a bit worried about the auto select and no format stuff, but if it passes tests then its good to me. cant break no-format anymore :D

I didn't run the full test suite, only the new test. The only thing I can say is that it fixes no-format :D . Let's try this: https://github.com/kairos-io/kairos/pull/2291/files#diff-2705f6016a64189abd14041e2f3e8dbb829553295d9088e4eb42eccb5e61bf47R299

The rest of the tests pass: https://github.com/kairos-io/kairos/actions/runs/8616215034/job/23646545504?pr=2291

but I pushed some fixes here to fix the new test.

Itxaka

still looking good to me

jimmykarily commented Feb 23, 2024

View reviewed changes

internal/agent/install.go Outdated Show resolved Hide resolved

jimmykarily commented Feb 23, 2024

View reviewed changes

jimmykarily self-assigned this Feb 23, 2024

jimmykarily changed the title ~~[WIP] Try to see which code breaks it~~ 2281 - target manually partitioned disk Feb 23, 2024

This was referenced Feb 23, 2024

Custom partitioning with no-format doesn't work kairos-io/kairos#2281

Closed

Add test for custom partioning kairos-io/kairos#2291

Merged

jimmykarily force-pushed the 2281-no-format-custom-partitioning branch 3 times, most recently from 822b873 to 70ba7a0 Compare February 29, 2024 10:28

jimmykarily force-pushed the 2281-no-format-custom-partitioning branch from 70ba7a0 to 19aeef1 Compare April 5, 2024 08:14

jimmykarily commented Apr 8, 2024

View reviewed changes

jimmykarily force-pushed the 2281-no-format-custom-partitioning branch from 534eda5 to a9f3f8c Compare April 8, 2024 09:53

jimmykarily commented Apr 8, 2024

View reviewed changes

jimmykarily marked this pull request as ready for review April 9, 2024 09:51

jimmykarily added 8 commits April 9, 2024 12:52

[WIP] Try to see which code breaks it

7d9accc

Fixes kairos-io/kairos#2281 Signed-off-by: Dimitris Karakasilis <[email protected]>

WIP

804462e

Signed-off-by: Dimitris Karakasilis <[email protected]>

Detect the preconfigured device when no-format is true

9df8781

Signed-off-by: Dimitris Karakasilis <[email protected]>

Remove unecessary go.mod "replace" instruction

da9065e

Signed-off-by: Dimitris Karakasilis <[email protected]>

Fix linting error

0a3b0c0

Signed-off-by: Dimitris Karakasilis <[email protected]>

Remove development debug output

d8df60c

Signed-off-by: Dimitris Karakasilis <[email protected]>

Remove unecessary assignments and add NoFormat to UKI

571f10d

Signed-off-by: Dimitris Karakasilis <[email protected]>

Fix go.sum after rebase

b80dcdb

Signed-off-by: Dimitris Karakasilis <[email protected]>

jimmykarily force-pushed the 2281-no-format-custom-partitioning branch from 7ece12a to b80dcdb Compare April 9, 2024 09:53

jimmykarily requested a review from a team April 9, 2024 09:53

Itxaka approved these changes Apr 9, 2024

View reviewed changes

Bump yip and propagate NoFormat to the install spec

d3bf4eb

We need this from yip: mudler/yip@4ebbc75 Signed-off-by: Dimitris Karakasilis <[email protected]>

Itxaka approved these changes Apr 10, 2024

View reviewed changes

jimmykarily merged commit 9437918 into main Apr 10, 2024
10 of 11 checks passed

jimmykarily deleted the 2281-no-format-custom-partitioning branch April 10, 2024 12:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

2281 - target manually partitioned disk #235

2281 - target manually partitioned disk #235

jimmykarily commented Feb 23, 2024

codecov bot commented Feb 23, 2024 •

edited

Loading

jimmykarily commented Feb 23, 2024

jimmykarily Feb 23, 2024

jimmykarily commented Apr 5, 2024

jimmykarily commented Apr 5, 2024

jimmykarily Apr 8, 2024

jimmykarily Apr 8, 2024

jimmykarily Apr 8, 2024

jimmykarily Apr 9, 2024 •

edited

Loading

Itxaka left a comment

jimmykarily commented Apr 9, 2024

jimmykarily commented Apr 10, 2024

jimmykarily commented Apr 10, 2024

Itxaka left a comment

2281 - target manually partitioned disk #235

2281 - target manually partitioned disk #235

Conversation

jimmykarily commented Feb 23, 2024

codecov bot commented Feb 23, 2024 • edited Loading

Codecov Report

jimmykarily commented Feb 23, 2024

jimmykarily Feb 23, 2024

Choose a reason for hiding this comment

jimmykarily commented Apr 5, 2024

jimmykarily commented Apr 5, 2024

jimmykarily Apr 8, 2024

Choose a reason for hiding this comment

jimmykarily Apr 8, 2024

Choose a reason for hiding this comment

jimmykarily Apr 8, 2024

Choose a reason for hiding this comment

jimmykarily Apr 9, 2024 • edited Loading

Choose a reason for hiding this comment

Itxaka left a comment

Choose a reason for hiding this comment

jimmykarily commented Apr 9, 2024

jimmykarily commented Apr 10, 2024

jimmykarily commented Apr 10, 2024

Itxaka left a comment

Choose a reason for hiding this comment

codecov bot commented Feb 23, 2024 •

edited

Loading

jimmykarily Apr 9, 2024 •

edited

Loading