feat: integrity checks for Ubuntu Release files #106

letFunny · 2023-12-11T11:49:54Z

Chisel will download InRelease files instead of Release files, the former being the signed version of the latter. For each archive and its components/suites, the signature in the InRelease file will be verified against the public keys associated with that archive.

These public keys will be stored in the chisel-release chisel.yaml file, as a new top-level property called public-keys. Keys themselves will be stored in ASCII armored format together with their key id. Lastly, the archive definition will reference these public keys by name.

Example chisel.yaml:

    format: chisel-v1
    archives:
      ubuntu:
	version: 22.04
	components: [main, universe]
	suites: [jammy, jammy-updates, jammy-security]
	public-keys: [ubuntu]

    public-keys:
      ubuntu:
	id: C2B15A6A7FCD95FD
	armor: |
	  -----BEGIN PGP PUBLIC KEY BLOCK-----

	  mQINBFzZxGABEADSWmX0+K//0cosKPyr5m1ewmwWKjRo/KBPTyR8icHhbBWfFd8T
	  DtYggvQHPU0YnKRcWits0et8JqSgZttNa28s7SaSUTBzfgzFJZgULAi/4i8u8TUj
	  +KH2zSoUX55NKC9aozba1cR66jM6O/BHXK5YoZzTpmiY1AHlIWAJ9s6cCClhnYMR
	  ...
	  E+SWDGxtgwixyPziL56UavL/eeYJWeS/WqvGzZzsAtgSujFVLKWyUaRi0NvYW3h/
	  I50Tzj0Pkm8GtgvP2UqAWvy+iRpeUQ2ji0Nc
	  =j6+P
	  -----END PGP PUBLIC KEY BLOCK-----

This reverts commit 6bf6d46.

This commit adds the functionalities to add public keys to validate archive InRelease file signatures. Instead of downloading the "Release" files for different suites, download the signed "InRelease" file and validate the signature with the provided key(s). This commit extends the chisel release with public keys definitions. They are stored in ASCII armored format in the top-level public-keys property by name. And they are referenced by name in the public-keys list property in archive definitions. An example of the extended chisel release file is at the bottom. Example chisel.yaml: format: chisel-v1 archives: ubuntu: version: 22.04 components: [main, universe] suites: [jammy, jammy-updates, jammy-security] public-keys: [ubuntu] ubuntu-fips: version: 22.04 pro: fips components: [main] suites: [jammy] public-keys: [ubuntu-fips] ubuntu-fips-updates: version: 22.04 pro: fips-updates components: [main] suites: [jammy] public-keys: [ubuntu-fips] public-keys: ubuntu: id: C2B15A6A7FCD95FD armor: | -----BEGIN PGP PUBLIC KEY BLOCK----- mQINBFzZxGABEADSWmX0+K//0cosKPyr5m1ewmwWKjRo/KBPTyR8icHhbBWfFd8T DtYggvQHPU0YnKRcWits0et8JqSgZttNa28s7SaSUTBzfgzFJZgULAi/4i8u8TUj +KH2zSoUX55NKC9aozba1cR66jM6O/BHXK5YoZzTpmiY1AHlIWAJ9s6cCClhnYMR ... E+SWDGxtgwixyPziL56UavL/eeYJWeS/WqvGzZzsAtgSujFVLKWyUaRi0NvYW3h/ I50Tzj0Pkm8GtgvP2UqAWvy+iRpeUQ2ji0Nc =j6+P -----END PGP PUBLIC KEY BLOCK----- ubuntu-fips: id: E173597E2CAB05C1 armor: | -----BEGIN PGP PUBLIC KEY BLOCK----- mQINBE+tgXgBEADfiL1KNFHT4H4Dw0OR9LemR8ebsFl+b9E44IpGhgWYDufj0gaM /UJ1Ti3bHfRT39VVZ6cv1P4mQy0bnAKFbYz/wo+GhzjBWtn6dThYv7n+KL8bptSC Xgg1a6en8dCCIA/pwtS2Ut/g4Eu6Z467dvYNlMgCqvg+prKIrXf5ibio48j3AFvd ... mguPI1KLfnVnXnsT5JYMbG2DCLHI/OIvnpRq8v955glZ5L9aq8bNnOwC2BK6MVUs pbJRpGLQ29hbeH8jnRPOPQ+Sbwa2C8/ZSoBa/L6JGl5RDaOLQ1w= =6Bkw -----END PGP PUBLIC KEY BLOCK-----

This commit introduces a new chisel-release yaml format named "chisel-v2". The major changes in this format is the introduction of the "public-keys" field, which is used in archive definitions to verify archive InRelease files.

letFunny · 2023-12-11T12:00:45Z

spread test is failing as expected because of chisel-releases only having the v1 format at the moment. Error in CI run is (link):

error: chisel.yaml: expected format "chisel-v2", got "chisel-v1"

cjdcordeiro

Thanks for this!

Given the disruptive change, I think it would be also prepare the chisel-releases PRs already (updating #62-#66) to comply with the expected format.

letFunny · 2023-12-12T08:43:02Z

Agree, those updates are currently in progress as far as I know.

This commit adds the Ubuntu Archive Automatic Signing Key (2018) as a "public-key" in the chisel.yaml. The armored data was obtained by executing the following commands on a Ubuntu 22.04 machine: $ gpg --keyserver keyserver.ubuntu.com --receive-keys 871920D1991BC93C $ gpg --armor --export 871920D1991BC93C BREAKING CHANGE: This commit introduces the new chisel yaml format "chisel-v2", in which ``public-keys`` (top-level) and ``archive.<name>.public-keys`` fields are introduced. This changes is related to the addition of integrity checks in chisel. [1] [1] canonical/chisel#106

niemeyer

Thanks! Here is a first pass.

README.md

internal/archive/archive.go

internal/setup/testdata/ubuntu-archive-keyring.asc

internal/testutil/gpgkeys.go

niemeyer · 2023-12-13T13:21:11Z

internal/testutil/reindent_test.go

+}, {
+	raw:    "\n",
+	prefix: "\t",
+	result: "\t\n\t",


This doesn't look right. It's indenting an empty line with zero content, and may create actual issues depending on what else shows up after it. Imagine AddPrefix("foo\n") + "bar".

I agree with you. We went for a naive implementation which produces strange cases. I have changed it and made it more complex.

cjdcordeiro · 2023-12-14T09:52:46Z

Once approved and before merging, please wait for the corresponding chisel-releases PRs to be merged so that we can re-run the Spread tests

This reverts commit 0995608.

cjdcordeiro

Just to recap the plan:

v0.8.1 has just been released, including the fix for KnownFields(false). This will give existing users a longer grace period for migrating to format ~~chisel-v2~~ v1, whenever that comes
with v0.8.1 in place and communicated, we wait some time and then we can then update the chisel-releases, adding the public-keys fields to chisel-v1. While v0.8 will fail, v0.8.1 will work (let's say early Jan 2024)
this PR can then also be merged
after this PR, https://github.com/canonical/chisel/pull/108/files can also be reviewed and merged
we'll then release Chisel v0.9.0 with support for both chisel-v1 and ~~chisel-v2~~ v1 formats (at this stage, v0.8.1 still works)
after some time, we can then deprecate the chisel-v1 format from chisel-releases. v0.8.0 and v0.8.1 will no longer work, but v0.9.0 will (let's also give it some time between the v0.9.0 release and the migration to ~~chisel-v2~~ v1)

letFunny · 2023-12-14T15:30:40Z

@cjdcordeiro Sure it just caught me by surprise because that was not the plan last time we discussed. I will amend the PRs and timelines to match the new direction.

cjdcordeiro · 2023-12-14T17:02:18Z

@cjdcordeiro Sure it just caught me by surprise because that was not the plan last time we discussed. I will amend the PRs and timelines to match the new direction.

Y sry about that @letFunny . I think only #106 would need to be amended though

cjdcordeiro · 2024-01-04T11:02:58Z

FYI: note the updated comment above (#106 (review)) - we shall update the chisel-releases format to v1 instead of chisel-v2. This will still create the desired disruption while making the versioning terminology clearer between the chisel-releases "format" and chisel tool releases.

niemeyer

This is looking great, thanks Alberto.

Only final nitpicks, and then we need to sync with the Rocks team on the exact release date.

internal/openpgputil/openpgp.go

niemeyer · 2024-01-08T13:37:42Z

internal/setup/fetch_test.go

@@ -12,6 +12,8 @@ import (
 // TODO Implement local test server instead of using live repository.

 func (s *S) TestFetch(c *C) {
+	c.Skip("TODO chisel-releases need to be updated with public-keys for this test to pass")


For our own tracking, this still needs addressing.

internal/testutil/pgpkeys.go

letFunny · 2024-01-11T14:50:56Z

According to the new plan:

In this PR change the yaml key to v1-public-keys while the format remains chisel-v1.
Wait until chisel-releases has the new format.
Uncomment the tests and merge this one.
Create a new PR that supports both formats chisel-v1 and v1 where the difference is that v1-public-keys becomes public-keys.
After the latter PR is merged we can tag version v0.9.0 which supports both formats.

* feat(22.04): add ubuntu archive signing key 2018 This commit adds the Ubuntu Archive Automatic Signing Key (2018) as a "public-key" in the chisel.yaml. The armored data was obtained by executing the following commands on a Ubuntu 22.04 machine: $ gpg --keyserver keyserver.ubuntu.com --receive-keys 871920D1991BC93C $ gpg --armor --export 871920D1991BC93C BREAKING CHANGE: This commit introduces the new chisel yaml format "chisel-v2", in which ``public-keys`` (top-level) and ``archive.<name>.public-keys`` fields are introduced. This changes is related to the addition of integrity checks in chisel. [1] [1] canonical/chisel#106 * fix(22.04): revert chisel.yaml format to chisel-v1 * fix: rename public-keys to v1-public-keys

* feat(24.04): add ubuntu archive signing key 2018 This commit adds the Ubuntu Archive Automatic Signing Key (2018) as a "public-key" in the chisel.yaml. The armored data was obtained by executing the following commands on a Ubuntu 22.04 machine: $ gpg --keyserver keyserver.ubuntu.com --receive-keys 871920D1991BC93C $ gpg --armor --export 871920D1991BC93C BREAKING CHANGE: This commit introduces the new chisel yaml format "chisel-v2", in which ``public-keys`` (top-level) and ``archive.<name>.public-keys`` fields are introduced. This changes is related to the addition of integrity checks in chisel. [1] [1] canonical/chisel#106 * fix(24.04): revert chisel.yaml format to chisel-v1 * fix: rename public-keys to v1-public-keys

* feat(23.10): add ubuntu archive signing key 2018 This commit adds the Ubuntu Archive Automatic Signing Key (2018) as a "public-key" in the chisel.yaml. The armored data was obtained by executing the following commands on a Ubuntu 22.04 machine: $ gpg --keyserver keyserver.ubuntu.com --receive-keys 871920D1991BC93C $ gpg --armor --export 871920D1991BC93C BREAKING CHANGE: This commit introduces the new chisel yaml format "chisel-v2", in which ``public-keys`` (top-level) and ``archive.<name>.public-keys`` fields are introduced. This changes is related to the addition of integrity checks in chisel. [1] [1] canonical/chisel#106 * fix(23.10): revert chisel.yaml format to chisel-v1 * fix: rename public-keys to v1-public-keys

* feat(20.04): add ubuntu archive signing key 2018 This commit adds the Ubuntu Archive Automatic Signing Key (2018) as a "public-key" in the chisel.yaml. The armored data was obtained by executing the following commands on a Ubuntu 22.04 machine: $ gpg --keyserver keyserver.ubuntu.com --receive-keys 871920D1991BC93C $ gpg --armor --export 871920D1991BC93C BREAKING CHANGE: This commit introduces the new chisel yaml format "chisel-v2", in which ``public-keys`` (top-level) and ``archive.<name>.public-keys`` fields are introduced. This changes is related to the addition of integrity checks in chisel. [1] [1] canonical/chisel#106 * fix(20.04): revert chisel.yaml format to chisel-v1 * fix: rename public-keys to v1-public-keys

* feat(23.04): add ubuntu archive signing key 2018 This commit adds the Ubuntu Archive Automatic Signing Key (2018) as a "public-key" in the chisel.yaml. The armored data was obtained by executing the following commands on a Ubuntu 22.04 machine: $ gpg --keyserver keyserver.ubuntu.com --receive-keys 871920D1991BC93C $ gpg --armor --export 871920D1991BC93C BREAKING CHANGE: This commit introduces the new chisel yaml format "chisel-v2", in which ``public-keys`` (top-level) and ``archive.<name>.public-keys`` fields are introduced. This changes is related to the addition of integrity checks in chisel. [1] [1] canonical/chisel#106 * fix(23.04): revert chisel.yaml format to chisel-v1 * fix: rename public-keys to v1-public-keys

niemeyer

Thanks, I'm merging this now per our agreement, but please DO NOT RELEASE THIS before both v1-public-keys and public-keys are supported. We don't want a release in the wild that supports one and not the other as otherwise we'll have introduced the need for a new transition.

cjdcordeiro · 2024-01-12T17:50:58Z

Y agreed. We'll wait for a coming PR with those changes before releasing v0.9.0

* feat(22.04): add ubuntu archive signing key 2018 This commit adds the Ubuntu Archive Automatic Signing Key (2018) as a "public-key" in the chisel.yaml. The armored data was obtained by executing the following commands on a Ubuntu 22.04 machine: $ gpg --keyserver keyserver.ubuntu.com --receive-keys 871920D1991BC93C $ gpg --armor --export 871920D1991BC93C BREAKING CHANGE: This commit introduces the new chisel yaml format "chisel-v2", in which ``public-keys`` (top-level) and ``archive.<name>.public-keys`` fields are introduced. This changes is related to the addition of integrity checks in chisel. [1] [1] canonical/chisel#106 * fix(22.04): revert chisel.yaml format to chisel-v1 * fix: rename public-keys to v1-public-keys

rebornplusplus added 3 commits December 11, 2023 12:29

Revert "feat: support OpenPGP keyrings in release (canonical#100)"

3e8a881

This reverts commit 6bf6d46.

feat: bump chisel.yaml format version to "chisel-v2"

0995608

This commit introduces a new chisel-release yaml format named "chisel-v2". The major changes in this format is the introduction of the "public-keys" field, which is used in archive definitions to verify archive InRelease files.

cjdcordeiro reviewed Dec 12, 2023

View reviewed changes

rebornplusplus mentioned this pull request Dec 13, 2023

feat(20.04): add ubuntu archive signing key 2018 canonical/chisel-releases#77

Merged

rebornplusplus mentioned this pull request Dec 13, 2023

feat(22.04): add ubuntu archive signing key 2018 canonical/chisel-releases#78

Merged

rebornplusplus mentioned this pull request Dec 13, 2023

feat(23.04): add ubuntu archive signing key 2018 canonical/chisel-releases#79

Merged

rebornplusplus mentioned this pull request Dec 13, 2023

feat(23.10): add ubuntu archive signing key 2018 canonical/chisel-releases#80

Merged

rebornplusplus mentioned this pull request Dec 13, 2023

feat(24.04): add ubuntu archive signing key 2018 canonical/chisel-releases#81

Merged

niemeyer requested changes Dec 13, 2023

View reviewed changes

cjdcordeiro added the Priority Look at me first label Dec 14, 2023

cjdcordeiro assigned niemeyer Dec 14, 2023

letFunny added 2 commits December 14, 2023 11:28

Revert "feat: bump chisel.yaml format version to "chisel-v2""

44d6f41

This reverts commit 0995608.

Merge branch 'main' into feat/gpg/verify-release

d9ffa47

letFunny mentioned this pull request Dec 14, 2023

feat: bump chisel.yaml format version to "chisel-v2" #108

Closed

1 task

letFunny added 7 commits December 14, 2023 12:07

Reword error messages

346ae5f

simplify DecodeClearSigned by removing extra return

96a4ad6

remove wildcards from error message

4c5ef3f

move pgp logic to separate package

c42b8b0

reword DecodeClearSigned comment

7281023

rename KeyID field to ID

643d010

remove Get prefix

3fef942

cjdcordeiro reviewed Dec 14, 2023

View reviewed changes

shorten test key

986e89e

letFunny added 4 commits December 15, 2023 11:54

change key naming to be uniform

58e7b6f

change PrefixEachLine to add newline at the end

9d741b0

more name changes to make keys uniform

6c4dcc4

change PrefixEachLine to avoid prefixing empty line

cedd234

letFunny requested a review from niemeyer December 15, 2023 11:55

typo, extra space in comment

fda178a

niemeyer approved these changes Jan 8, 2024

View reviewed changes

letFunny added 3 commits January 9, 2024 11:55

rename module to pgputil

e14cb8e

rename map key

b78f108

rename yaml key to v1-public-keys

c4ef62e

cjdcordeiro requested a review from niemeyer January 11, 2024 16:43

uncomment test

b7a953c

Merge branch 'main' into feat/gpg/verify-release

2f86776

niemeyer approved these changes Jan 12, 2024

View reviewed changes

niemeyer merged commit b659240 into canonical:main Jan 12, 2024
14 checks passed

letFunny deleted the feat/gpg/verify-release branch October 17, 2024 08:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: integrity checks for Ubuntu Release files #106

feat: integrity checks for Ubuntu Release files #106

letFunny commented Dec 11, 2023 •

edited

Loading

letFunny commented Dec 11, 2023

cjdcordeiro left a comment

letFunny commented Dec 12, 2023

niemeyer left a comment

niemeyer Dec 13, 2023

letFunny Dec 15, 2023

cjdcordeiro commented Dec 14, 2023

cjdcordeiro left a comment •

edited

Loading

letFunny commented Dec 14, 2023

cjdcordeiro commented Dec 14, 2023

cjdcordeiro commented Jan 4, 2024

niemeyer left a comment

niemeyer Jan 8, 2024

letFunny commented Jan 11, 2024

niemeyer left a comment •

edited

Loading

cjdcordeiro commented Jan 12, 2024

feat: integrity checks for Ubuntu Release files #106

feat: integrity checks for Ubuntu Release files #106

Conversation

letFunny commented Dec 11, 2023 • edited Loading

letFunny commented Dec 11, 2023

cjdcordeiro left a comment

Choose a reason for hiding this comment

letFunny commented Dec 12, 2023

niemeyer left a comment

Choose a reason for hiding this comment

niemeyer Dec 13, 2023

Choose a reason for hiding this comment

letFunny Dec 15, 2023

Choose a reason for hiding this comment

cjdcordeiro commented Dec 14, 2023

cjdcordeiro left a comment • edited Loading

Choose a reason for hiding this comment

letFunny commented Dec 14, 2023

cjdcordeiro commented Dec 14, 2023

cjdcordeiro commented Jan 4, 2024

niemeyer left a comment

Choose a reason for hiding this comment

niemeyer Jan 8, 2024

Choose a reason for hiding this comment

letFunny commented Jan 11, 2024

niemeyer left a comment • edited Loading

Choose a reason for hiding this comment

cjdcordeiro commented Jan 12, 2024

letFunny commented Dec 11, 2023 •

edited

Loading

cjdcordeiro left a comment •

edited

Loading

niemeyer left a comment •

edited

Loading