No kABI-tracking kmod for RHEL-based Linux 9.5

[namazso]

System information

Type	Version/Name
Distribution Name	Oracle Linux
Distribution Version	9.5
Kernel Version	Linux 5.14.0-503.14.1.el9_5.x86_64
Architecture	amd64
OpenZFS Version	kmod-zfs-2.1.15-3.el9.x86_64

Describe the problem you're observing

Following the docs doesn't produce a usable ZFS install

Describe how to reproduce the problem

1. Install Oracle Linux 9.5
2. Switch to RHCK
3. Follow the instructions from the earlier linked docs

Include any warning/errors/backtraces from the system logs

# /sbin/modprobe zfs
modprobe: FATAL: Module zfs not found in directory /lib/modules/5.14.0-503.14.1.el9_5.x86_64

[namazso]

Possibly related to #16746 and #16196, however those are about DKMS and not kABI-tracking.

[clhedrick]

I'm running on RHEL 9.5 using rpm's generated according to those instructions. The only problem I had was getting it to install. If you are upgrading from 9.4 and you have the same version of ZFS already installed, installing the rpms built under 9.5 won't do anything, because they have the same name as the 9.4 ones and dnf thinks they're already installed. I had to do a reinstall, which unfortunately removed the kernel modules from any old 9.4 kernels but did give me a working 9.5 installation.

I should note that I'm using ZFS 2.2.6, while you're using 2.1.15. Version 2.1.16 is currently being prepared. The notes say that the 9.5 kernel broke the build. Take a look at #16781. You may need to apply the patches shown there or move to 2.2.6. I think 2.2 is stable now. If you're concerned, don't do a feature upgrade. But we've been running with 2.2.6 with zpool upgrade done in production on a variety of systems with no trouble. BRT is still not on by default.

[namazso]

Yes I’m running stable, and a package reinstall didn’t solve the issue. Depending on how the PR progresses I might switch to testing instead.

[paulvonbehren]

I'm seeing the same problems with ZFS ABI on AlmaLinux 9.5. All updates are done with dnf. On separate Alma 9.5 system, zfs is okay. On the 2nd system I'm holding off dnf updates until this is resolved.

[namazso]

@clhedrick I tried testing, with # dnf update --enablerepo=zfs-testing-kmod zfs but it did not improve the situation

[paulvonbehren]

As far as I can tell, there is no way to revert to AlmaLinux 9.4 from 9.5. Any attempt to undo using dnf history rollback or undo results in a list of missing older packages. I will reinstall 9.4 next...

[clhedrick]

You can't revert the whole of 9.5, but you should be able to boot a 9.4 kernel. That's probably ok. I'd expect that the update would leave old kernels. However you may have to build zfs for that kernel and install it in the old /lib/modules directory, as updating to 9.5 may have removed the 9.4 zfs. modules.`Or pull it from backups.

[albertdipp]

Thank goodness I came across this, thought I lost my marbles for a moment as to why the zfs module just isn't loading after upgrading to 9.5... Bad timing to build a new storage box it seems :D