Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v254 batch #371

Merged
merged 90 commits into from
Feb 28, 2024
Merged

v254 batch #371

merged 90 commits into from
Feb 28, 2024

Conversation

bluca
Copy link
Member

@bluca bluca commented Feb 27, 2024

No description provided.

mrc0mmand and others added 16 commits February 27, 2024 18:01
(cherry picked from commit 58bcbad)
(cherry picked from commit 4442012)
Since [0] delv no longer does that automagically, so we have to that
explicitly with each delv invocation.

Resolves: #30477

[0] isc-projects/bind9@c144fd2

(cherry picked from commit 438c7cb)
(cherry picked from commit d62f1bb)
In Knot 3.2 the nsec3-iterations default was changed to 0 and Knot now
issues a warning if the value is > 0. Let's just use the default value,
since it's not something that's important for our tests.

(cherry picked from commit 0652cf8)
(cherry picked from commit 12ee58c)
When using the Xen hypervisor the virtual machine UUID is exposed here.
This is useful when one needs stable IPv4 address assignment, e.g. for a
set of RAM nodes that are built from a template.

(cherry picked from commit d2f2c20)
(cherry picked from commit 38d632a)
… UUID

(cherry picked from commit 98d550b)
(cherry picked from commit 907fecc)
(cherry picked from commit 9fe1f1c)
(cherry picked from commit 81bd0e9)
knot v3.2 and later does this by default. knot v3.1 still has the default set to
10, but it also introduced a warning that the default will be changed to 0 in
later versions, so it effectively complains about its own default, which then
fails the config check. Let's just set the value explicitly to zero to avoid
that.

~# knotc --version
knotc (Knot DNS), version 3.1.6
~# grep nsec3-iterations test/knot-data/knot.conf || echo nope
nope
~# knotc -c /build/test/knot-data/knot.conf conf-check
warning: config, policy[auto_rollover_nsec3].nsec3-iterations defaults to 10, since version 3.2 the default becomes 0
Configuration is valid

Follow-up to 0652cf8.

(cherry picked from commit cb3244c)
(cherry picked from commit 3158eb9)
The original way of appending to /etc/systemd/journald.conf doesn't work
anymore, since we no longer ship the default configs in /etc/.

(cherry picked from commit 1d556e9)
(cherry picked from commit cfe745d)
Some tests have hard-coded results that need to match, and change if
the caller has a timezone set via the TZ= environment variable, as it
is the case during reproducible build tests. Unset it.

(cherry picked from commit 1e902c3)
(cherry picked from commit 85c88db)
(cherry picked from commit 431f836)
(cherry picked from commit b7fe2d4)
so that /usr/lib/systemd/tests/unit-tests/test-libsystemd-sym
builds in a reproducible way
in spite of non-deterministic filesystem readdir order

See https://reproducible-builds.org/ for why this is good.

This patch was done while working on reproducible builds for openSUSE.

(cherry picked from commit ac0054e)
(cherry picked from commit e2f8d25)
I added the filtering in 752fedb as a way
to reduce the number of items in the tables. I thought it's "obvious", but
it might not be so.

One immediate problem is that the filter is broken, because on arm64,
os.uname().machine returns "aarch64", so we incorrectly filter out the arm
syscalls (there is just one: arm_fadvise64_64). Of course we could fix the
filter, but I think it's better to nuke it altogether. The filter on applies to
1 arm syscall and 5 s390 syscalls, and we have 500+ other syscalls, so this
"optimization" doesn't really matter. OTOH, if we get the filter wrong,
the result is bad. And also, the existence of the filter at all creates
problems for cross-builds.

I wanted to get rid of 'generate-syscall-list.py', but we need to generate a
backslash in the output. mesonbuild/meson#1564 makes
this very very hard, since any attempt to put a backslash an inline argument
results in the backslash being replaces by a forward slash, which doesn't quite
have the same meaning. So let's use a standalone script until
mesonbuild/meson#1564 is resolved.

(cherry picked from commit 58fcc6b)
(cherry picked from commit ded73e6)
To get rid of the "invalid escape sequence" warning:

=============================== warnings summary ===============================
../src/ukify/test/test_ukify.py:876
  ../src/ukify/test/test_ukify.py:876: SyntaxWarning: invalid escape sequence '\s'
    assert re.search('Issuer: CN\s?=\s?SecureBoot signing key on host', out)

(cherry picked from commit a0485e0)
(cherry picked from commit 974c015)
This makes the test compatible with Python 3.9, as the follow_symlinks
keyword was introduced in Python 3.10.

(cherry picked from commit 56cdf81)
(cherry picked from commit afaa04a)
Follow-up for baa95d2.

(cherry picked from commit a140eaf)
(cherry picked from commit bd12386)
…y joined

Follow-up for 9f913d3.

Fixes #31165.

(cherry picked from commit c3e12de)
(cherry picked from commit b71ec1c)
@bluca bluca changed the base branch from v255-stable to v254-stable February 27, 2024 19:47
yuwata and others added 11 commits February 27, 2024 21:37
(cherry picked from commit 2bb1d3c)
(cherry picked from commit b2e0f71)
Fixes an issue caused by ab3aed4 (v253).

By default, all managed interface need to be configured, and at least
one interface need to be online. Hence, offline interface should be ignored.

Fixes #29506.

(cherry picked from commit 6f412c0)
(cherry picked from commit 6412633)
(cherry picked from commit 561d879)
(cherry picked from commit 0ec2d29)
…ic_full()

Fixes a bug introduced by 427d9c3.

(cherry picked from commit 738ad08)
(cherry picked from commit b5c8d1b)
(cherry picked from commit dfdcc7c)
(cherry picked from commit 0f78bd6)
Otherwise:

```
Feb 19 16:35:34 localhost systemd-gpt-auto-generator[188]: Assertion 's' failed at src/shared/image-policy.c:656, function parse_image_policy_argument(). Aborting.
```

Fixes 06e7868

(cherry picked from commit d0a0059)
(cherry picked from commit f02d013)
Skip using pidfds if we get a permission denied error.
This can happen with an old policy and a new kernel that uses the
new pidfs filesystem to back pidfds, instead of anonymous inodes,
as the existing policy denies access.

This is already the case for most uses of pidfd_open, like pidref,
but not on these two. Fix them.

(cherry picked from commit 857945c)
(cherry picked from commit 9c978a8)
With meson build --werror --buildtype=plain -Dc_args=" -O2" the build fails:

../src/boot/efi/stub.c: In function ‘load_addons.constprop’:03:06
../src/boot/efi/stub.c:475:40: error: using a dangling pointer to ‘p’ [-Werror=dangling-pointer=]03:06
  475 |                         dt_bases[n_dt] = xmemdup((uint8_t*)loaded_addon->ImageBase + addrs[UNIFIED_SECTION_DTB],03:06
      |                         ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~03:06
  476 |                                                  dt_sizes[n_dt]);03:06
      |                                                  ~~~~~~~~~~~~~~~03:06
In file included from ../src/boot/efi/stub.c:20:03:06
../src/boot/efi/util.h:33:15: note: ‘p’ declared here03:06
   33 |         void *p;03:06
      |               ^

De-inline the function and initialize p to make gcc happy.

(cherry picked from commit 6036f62)
(cherry picked from commit 4cf3445)
It has been demonstrated that tolerating an unbounded number of dnssec
signature validations is a bad idea. It is easy for a maliciously
crafted DNS reply to contain as many keytag collisions as desired,
causing us to iterate every dnskey and signature combination in vain.

The solution is to impose a maximum number of validations we will
tolerate. While collisions are not hard to craft, I still expect they
are unlikely in the wild so it should be safe to pick fairly small
values.

Here two limits are imposed: one on the maximum number of invalid
signatures encountered per rrset, and another on the total number of
validations performed per transaction.

(cherry picked from commit 67d0ce8)
(cherry picked from commit 1ebdb19)
According to RFC9267, the 2500 value is not helpful, and in fact it can
be harmful to permit a large number of iterations. Combined with limits
on the number of signature validations, I expect this will mitigate the
impact of maliciously crafted domains designed to cause excessive
cryptographic work.

(cherry picked from commit eba2911)
(cherry picked from commit 572692f)
Eric Daigle and others added 28 commits February 27, 2024 21:37
As described in #30940, systemd-firstboot currently does not perform
any validation on keymap entry, allowing nonexistent keymaps to be
written to /etc/vconsole.conf. This commit adds validation checks
based on those already performed on locale entry, preventing invalid
keymaps from being set.

Closes #30940

m

(cherry picked from commit 321a8c5)
(cherry picked from commit b4c1ec8)
So glibc exposes a close_range() syscall wrapper now, but they decided
to use "unsigned" as type for the fds. Which is a bit weird, because fds
are universally understood to be "int". The kernel internally uses
"unsigned", both for close() and for close_range(), but weirdly,
userspace didn't fix that for close_range() unlike what they did for
close()... Weird.

But anyway, let's follow suit, and make our wrapper match glibc's.

Fixes #31270

(cherry picked from commit 39d6983)
(cherry picked from commit 9b82b1f)
Needed since 87282a3.

(cherry picked from commit 03b1e10)
(cherry picked from commit 048639c)
bootctl can be installed also non uefi systems so its sole presence doesn't
mean that we can test installation of sd-boot.

(cherry picked from commit 26fff16)
(cherry picked from commit 9f7b772)
…n SUSE

Othewise test images are missing the tmpfiles snippets used to create the very
basic files at boot, which can be useful when a test wants to reuse the OS tree
(is already running in) for spawning a new container in pristine state.

(cherry picked from commit 08abfd0)
(cherry picked from commit f16f33d)
Add some extra safety checks: refuse weird open flags.

And while we are at it, also use stat_verify_regular()

(cherry picked from commit 1f47e27)
(cherry picked from commit 76cfca0)
If -Dtests=false but -Dinstall-tests=true the build will fail, as some tests will
be pulled in the build but not their prerequisites. It doesn't make sense to ask
for tests to be installed if they are disabled.

FAILED: test-acd
cc  -o test-acd test-acd.p/src_libsystemd-network_test-acd.c.o -flto -Wl,--as-needed -Wl,--no-undefined -pie -fstack-protector -Wl,-z,relro -specs=/usr/share/debhelper/dh_package_notes/debian-package-notes.specs -g -O2 -ffile-prefix-map=/tmp/s=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -ffat-lto-objects -Wdate-time -D_FORTIFY_SOURCE=2 '-Wl,-rpath,$ORIGIN/src/shared:XXXXXXXXXXXXXXX' -Wl,-rpath-link,/tmp/s/obj-x86_64-linux-gnu/src/shared -Wl,--start-group src/shared/libsystemd-shared-255.so src/libsystemd-network/libsystemd-network.a -Wl,--end-group -Wl,--fatal-warnings -Wl,-z,now -Wl,-z,relro -Wl,--warn-common -Wl,--gc-sections -Wl,--fatal-warnings -Wl,-z,now -Wl,-z,relro -Wl,--warn-common -Wl,--gc-sections
/usr/bin/ld: /tmp/cc0oYwFZ.ltrans0.ltrans.o: in function `main':
./obj-x86_64-linux-gnu/./obj-x86_64-linux-gnu/<artificial>:85:(.text.startup+0x33): undefined reference to `test_setup_logging'
collect2: error: ld returned 1 exit status

(cherry picked from commit 311efaa)
(cherry picked from commit 2dfc3b2)
…ation gracefully

Our function so far assumed that the LoaderEntries's last string is or
is not NUL terminated. But if it was, then we'd debug log about this,
claiming there was an invalid id. sd-boot actually ends the list in a
properly NUL-terminated string, hence we should just accept that. Handle
that case gracefully, and add comments explaining why we have two ways
why we exit the loop.

This is cosmetic only, just suppresses a misleading debug log message.

(cherry picked from commit 2cda44c)
(cherry picked from commit 1cd60d4)
Reword the description of the `IgnoreSIGPIPE=` service option to be more grammatical.

(cherry picked from commit 6c6ec5f)
(cherry picked from commit e16154f)
According to keyctl(2), the return value for KEYCTL_READ is:

    The amount of data that is available in the key,
    irrespective of the provided buffer size

So, we could pass in a NULL buffer to query the size, then allocate the
exact right amount of space, then call keyctl again to get the key data.
However, we must still keep the for loop to avoid TOCTOU issues: the key
might have been replaced with something bigger while we're busy
allocating the buffer to store it.

Thus, we can actually save a syscall by picking some reasonable default
buffer size and skipping the NULL call to keyctl. If our default is big
enough, we're done and have saved a syscall! If not, then the first call
behaves essentially the same as the NULL call, and we use the size it
returns to reallocate the buffer appropriately.

(cherry picked from commit d0aef63)
(cherry picked from commit 87e47aa)
From readlinkat(2):
Since Linux 2.6.39, pathname can be an empty string, in which case the
call operates on the symbolic link referred to by dirfd (which should
have been obtained using open(2) with the O_PATH and O_NOFOLLOW flags).

(cherry picked from commit e4c094c)
(cherry picked from commit 30142e7)
Follow-up for 47f0e1b.

(cherry picked from commit bec1383)
(cherry picked from commit 2e7f196)
Follow-up for cf91b91.

(cherry picked from commit 59a4e17)
(cherry picked from commit afac077)
Follow-up for 972c8db.

(cherry picked from commit 6383abd)
(cherry picked from commit 9e35c1d)
So the tests work even if the base image filesystem is not ext4.

(cherry picked from commit adafa3b)
(cherry picked from commit adcc196)
So it's created automagically with proper attributes.

(cherry picked from commit 1b0cf03)
(cherry picked from commit 06cdc05)
(cherry picked from commit 7e2bf4c)
(cherry picked from commit 6cd6d89)
If we're looking for output on stderr, let's make sure it's not
littered with debug logs if SYSTEMD_LOG_LEVEL=debug.

(cherry picked from commit 88d4b97)
(cherry picked from commit db97c3a)
Precedence for example in https://github.com/systemd/systemd/blob/ac63c8df309e37960618610d8b57ac19ac657254/rules.d/99-systemd.rules.in#L75.

Add ENV to the list of keys where string substitutions can be used.

While I'm at it, also sort the list in that paragraph alphabetically.

(cherry picked from commit 793166a)
(cherry picked from commit 75fb8e6)
TEST-23 uses run_subtests_with_signals() which I forgot about when
introducing the change.

Follow-up for 0efa27b.

(cherry picked from commit a69ec6f)
(cherry picked from commit a22452a)
This value is actually arch-specific, so this commit defines it for all
the arches that set it to some custom value

Fixes systemd/systemd#31417

(cherry picked from commit 9e3db91)
(cherry picked from commit b228a68)
Similarly to bbac11c we need to enable session lingering for the test
user, so the long-running test units are not killed prematurely:

[   18.822261] testsuite-55.sh[403]: + systemctl start --machine [email protected] --user testsuite-55-testchill.service
[   18.852775] systemd[1]: Started run-u17.service.
[   19.256431] (o-bridge)[526]: pam_unix(login:session): session opened for user testuser(uid=4711) by testuser(uid=0)
[   19.288346] systemd[1]: Started session-2.scope.
[   20.165874] systemd[392]: Created slice session.slice.
[   20.166459] systemd[392]: Starting dbus-broker.service...
[   20.220189] dbus-broker-launch[529]: Policy to allow eavesdropping in /usr/share/dbus-1/session.conf +31: Eavesdropping is deprecated and ignored
[   20.220189] dbus-broker-launch[529]: Policy to allow eavesdropping in /usr/share/dbus-1/session.conf +33: Eavesdropping is deprecated and ignored
[   20.220494] systemd[392]: Started dbus-broker.service.
[   20.224276] dbus-broker-launch[529]: Ready
[   20.231702] systemd[392]: Created slice testsuite.slice.
[   20.231976] systemd[392]: Created slice testsuite-55.slice.
[   20.232259] systemd[392]: Created slice testsuite-55-workload.slice.
[   31.065294] testsuite-55.sh[403]: + systemctl start --machine [email protected] --user testsuite-55-testbloat.service
[   31.065641] (sd-pam)[528]: pam_unix(login:session): session closed for user testuser
[   31.066103] (sd-pam)[528]: pam_systemd(login:session): Failed to release session: Access denied
[   31.066152] systemd[392]: Started testsuite-55-testchill.service.
[   31.068062] systemd[1]: run-u17.service: Deactivated successfully.
[   31.068217] dbus-broker[389]: A security policy denied :1.20 to send method call /org/freedesktop/login1:org.freedesktop.login1.Manager.ReleaseSession to org.freedesktop.login1.
[   31.075901] (o-bridge)[537]: pam_unix(login:session): session opened for user testuser(uid=4711) by testuser(uid=0)
[   31.091098] systemd[1]: Stopping session-2.scope...
[   31.092158] systemd[1]: Started run-u21.service.
[   31.092993] systemd[1]: session-2.scope: Deactivated successfully.
[   31.093287] systemd[1]: Stopped session-2.scope.
[   31.095798] systemd[1]: Stopping [email protected]...
[   31.103541] systemd[392]: Activating special unit exit.target...
[   31.108359] systemd[392]: Stopped target default.target.
[   31.109798] systemd[392]: Stopped target timers.target.
[   31.110790] systemd[392]: Stopping testsuite-55-testchill.service...
[   31.112154] systemd[392]: Stopped testsuite-55-testchill.service.
[   31.114033] systemd[392]: Removed slice testsuite-55-workload.slice.
[   31.114971] systemd[392]: Removed slice testsuite-55.slice.
[   31.115858] systemd[392]: Removed slice testsuite.slice.
...
[   31.475949] testsuite-55.sh[403]: + systemctl --machine [email protected] --user status testsuite-55-testchill.service
[   31.490464] systemd[1]: session-3.scope: Deactivated successfully.
[   31.565929] systemd[1]: Started run-u33.service.
[   31.592437] (o-bridge)[583]: pam_unix(login:session): session opened for user testuser(uid=4711) by testuser(uid=0)
[   31.610210] systemd[1]: Started session-5.scope.
[   31.616960] testsuite-55.sh[578]: ○ testsuite-55-testchill.service - No memory pressure
[   31.616960] testsuite-55.sh[578]:      Loaded: loaded (/usr/lib/systemd/tests/testdata/units/testsuite-55-testchill.service; static)
[   31.616960] testsuite-55.sh[578]:      Active: inactive (dead)
[   31.617438] (sd-pam)[586]: pam_unix(login:session): session closed for user testuser

Addresses systemd/systemd#31426 (comment).

(cherry picked from commit ff80bd2)
(cherry picked from commit 51b1a49)
…essage fields

Since we use varargs for sd_message_append() we need to make sure the
parameters we pass are actually 64bit wide, if "t" is used. Hence cast
appropriately if necessary.

I went through the whole tree, and in most cases we got it right, but
there are some cases we missed so far.

Inspired by: #31420

(cherry picked from commit 04a3af3)
(cherry picked from commit c0f501c)
I have a large initrd (built with mkosi-initrd) and the test-ukify takes 30 s.
Let's use the usual approach of skipping the slowests tests.

(pytest has marks, and it would be nicer to mark tests with pytest.mark.slow,
and then use "-m 'not slow'" in the meson test invocation. But markers must be
pre-registered, otherwise pytest emits a warning. There are a few ways to
register markers, but they all require "project configuration", but because of
how we invoke pytest, this is hard to do. So let's just use an environment
variable.)

(cherry picked from commit a181901)
(cherry picked from commit f5780a2)
The negative lookup can be quite slow. On my local network, skipping
this test saves about half of the runtime of test-nss-hosts.

(cherry picked from commit 37eba4b)
(cherry picked from commit 4b7aaef)
`__loongarch64` is no longer used for historical reasons and should be replaced by `__loongarch_lp64` in new code.

https://loongson.github.io/LoongArch-Documentation/LoongArch-toolchain-conventions-EN.html#_cc_preprocessor_built_in_macro_definitions
(cherry picked from commit f106a63)
Signed-off-by: Heinrich Schuchardt <[email protected]>
(cherry picked from commit 819874a)
(cherry picked from commit db00f6d)
SMBIOS support in QEMU for RISC-V is merged upstream.

Signed-off-by: Heinrich Schuchardt <[email protected]>
(cherry picked from commit 5041e77)
(cherry picked from commit 387a14a)
@bluca bluca merged commit 4cef472 into systemd:v254-stable Feb 28, 2024
42 of 43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.