Add a simple router template

[jonct]

I ran into a few chicken-egg surprises while setting this up, and I hope I can spare someone else the trouble.

What

This template helps organize dnsmasq and nftables to provide basic LAN infrastructure to a secondary bridge interface.

Why

My goal is to set loose a herd of Raspberry Pi on a private LAN, as K8s worker nodes.

I'm using the k3s template to anchor my control plane in another jail. (For my purposes this TrueNAS server presents a single point of failure anyway.) But I also need DHCP/DNS/TFTP/routing infrastructure for nodes on this secondary LAN/bridge.

How

Using dnsmasq here helps my headless/diskless Raspberry Pi nodes boot from the network, accessing the iSCSI and NFS services provided by TrueNAS itself.

[Jip-Hop]

Cool use case! Thanks for providing the PR. I obviously can't test this setup without the required hardware but the config looks good. Is there a reason you've left #ip link set dev eth2 master br1 in the config file? I've made some minor adjustments. I'll merge this in the develop branch and mention it in the changelog for the next release.

[jonct]

Thank you — both for considering this PR and for your leadership in charting a bona fide breakthrough for SCALE! 🚀

The #ip link breadcrumb can go — as can most if not all of the dnsmasq-example.conf file. During casual/occasional testing, I happen to have seen my physical interface fall off of the bridge on its own (twice) and I'm not yet certain what caused that. I suspect that it's a USB/sleeping issue, tied to my using an ethernet dongle on this spare 2012 Mac mini test system. But in case it were to happen to anyone else, maybe the breadcrumb would help them find and fix it too.

I go back and forth on how much to mention Raspberry Pi netbooting, here. I can imagine this becoming a crazy distraction from the support of Jailmaker itself. Whereas a router template to supply LAN infrastructure has many other potential uses. In case it would help your documentation, here are some of the sources I found useful on this external topic:

• Network Booting a Raspberry Pi 4 with an iSCSI Root via FreeNAS; the title says it all
• Raspberry Pi Network Boot Guide covers more Raspberry Pi models; written for Synology users
• pi_iscsi_netboot and prep-netboot-storage are scripts showing preparation of boot assets and iSCSI root volumes

[Jip-Hop]

Feel free to add those links to the template readme under additional resources like I did here: https://github.com/Jip-Hop/jailmaker/tree/main/templates/podman#additional-resources

[jonct]

Good thought. I've added a bit more, and I do have more confidence that this can help avoid a pile-up.

[Jip-Hop]

@jonct do you think you could lend a hand with #86? The GitHub Ubuntu VM which runs the test script of jailmaker doesn't have a bridge networking interface (with a DHCP server) so none of the config templates work out of the box.

I think it would be cool if the test script could setup networking in the VM to be more representative/compatible with the jailmaker config templates so overriding the systemd_nspawn_user_args with --network-veth is no longer required.

The config template requires a bridge interface called br1 and the jails need internet connectivity through this bridge. Something like: main network interface -> nat -> virtual network interface -> br1? And a DHCP server running behind NAT?

[Jip-Hop]

Perhaps it's as easy as creating a new br1 interface after having created a new 60-br1.network file with these contents:

[Match]
Kind=bridge
Name=br1

[Network]
# Default to using a /24 prefix, giving up to 253 addresses per virtual network.
Address=0.0.0.0/24
LinkLocalAddressing=yes
DHCPServer=yes
IPMasquerade=both
LLDP=yes
EmitLLDP=customer-bridge
IPv6AcceptRA=no
IPv6SendRA=yes

References: https://serverfault.com/questions/867055/accessing-host-services-from-systemd-nspawn-containers and https://github.com/systemd/systemd/blob/main/network/80-container-vz.network

[jonct]

"Ack." I'll follow up over there.