Using netclient at openwrt with virtualization, instead binary

[miyu4u]

Hello. Netmaker Community.

I tried to configure the Tunnel by connecting the netmaker to the custom router with the OpenWRT.

I followed the official guide for Openwrt, but it didn't work all.

The reference I refer to is:

• https://docs.netmaker.org/troubleshoot.html#netclient
• https://docs.netmaker.org/advanced-client-install.html#notes-on-openwrt
• https://github.com/sbilly/netmaker-openwrt

Among this, Sbilly's community contribution is grateful but too old. From the point of writing this article, the version of Openwrt is 22.03.

If run or install netclient-arm64 asset from release, it occured Systemd-related errors. (Even if you specify an option as --daemon off)

Perhaps it seems to be must required systemd (this is bug?)

So I gave up installing native binary on Openwrt and decided to use virtualization instead.

Openwrt 22.03 uses NFTABLES, so Docker didn't work, so i used LXC.

My machine is openwrt 22.03 MT7622 ARM64 router. (BPI-R64)

I haven't tried the MIPS target, but I think it would be enough if the LXC worked.

The reference I refer to is:

• https://openwrt.org/lxc_openwrt_host
• https://forum.openwrt.org/t/openwrt-arm64-quick-lxc-howto-guide-lms-in-debian-system-in-lxc-container/99835

Dependency installation

opkg update
opkg install xz tar gnupg
opkg install kmod-ikconfig kmod-veth
opkg install liblxc luci-app-lxc lxc lxc-attach lxc-auto lxc-autostart lxc-cgroup lxc-checkconfig lxc-common lxc-config lxc-configs lxc-console lxc-copy lxc-create lxc-destroy lxc-device lxc-execute lxc-freeze lxc-hooks lxc-info lxc-init lxc-ls lxc-monitor lxc-monitord lxc-snapshot lxc-start lxc-stop lxc-templates lxc-top lxc-unfreeze lxc-unprivileged lxc-unshare lxc-user-nic lxc-usernsexec lxc-wait rpcd-mod-lxc
opkg install gnupg2-utils gnupg2-dirmngr

if need, using lxc-checkconfig to checking.

lxc-checkconfig

I'm a beginner of LXC, so I don't know what I need to running lxc. My setting is as follows:

root@Trinity-Gakuen:/tmp# lxc-checkconfig
LXC version 5.0.1
--- Namespaces ---
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled

--- Control groups ---
Cgroups: enabled
Cgroup namespace: enabled

Cgroup v1 mount points:
/sys/fs/cgroup/cpuset
/sys/fs/cgroup/cpu
/sys/fs/cgroup/cpuacct
/sys/fs/cgroup/blkio
/sys/fs/cgroup/memory
/sys/fs/cgroup/net_cls
/sys/fs/cgroup/pids
/sys/fs/cgroup/rdma
/sys/fs/cgroup/systemd

Cgroup v2 mount points:


Cgroup v1 freezer controller: missing
Cgroup v1 clone_children flag: enabled
Cgroup device: missing
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled
Cgroup cpuset: enabled

--- Misc ---
Veth pair device: enabled, loaded
Macvlan: enabled, not loaded
Vlan: enabled, not loaded
Bridges: enabled, not loaded
Advanced netfilter: enabled, loaded
CONFIG_IP_NF_TARGET_MASQUERADE: missing
CONFIG_IP6_NF_TARGET_MASQUERADE: missing
CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled, not loaded
CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled, not loaded
FUSE (for use with lxcfs): enabled, not loaded

--- Checkpoint/Restore ---
checkpoint restore: missing
CONFIG_FHANDLE: enabled
CONFIG_EVENTFD: enabled
CONFIG_EPOLL: enabled
CONFIG_UNIX_DIAG: missing
CONFIG_INET_DIAG: enabled
CONFIG_PACKET_DIAG: missing
CONFIG_NETLINK_DIAG: enabled
File capabilities:

Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig

creating container.

export CONTAINER_NAME=foo
lxc-create --name $CONTAINER_NAME --template download -- --dist debian --release buster --arch arm64

if dependency is insufficient, error is occured.

lxc-create: $CONTAINER_NAME: tools/lxc_create.c: main: Failed to create container $CONTAINER_NAME

change container network config.

vi /srv/lxc/$CONTAINER_NAME/config

change link to br-lan, and modify hwaddr.

...
# Network configuration
lxc.net.0.type = veth
lxc.net.0.link = br-lan
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:FF:DD:BB:CC:01

container starting.

lxc-start -n $CONTAINER_NAME

if didn't change link, error occur when vert was connected.

container status checking.

lxc-ls -f

enternig the container shell.

lxc-attach -n $CONTAINER_NAME

Check the internal IP. If eth0 is designated as a band of br-lan set above, it is success.

ip address

instsalling netclient.

echo "deb http://deb.debian.org/debian/ buster-backports main " | \
sudo tee /etc/apt/sources.list.d/buster-backports.list

apt-get update -y && apt-get upgrade -y

apt-get install wireguard-tools iptables curl -y

iptables --policy FORWARD ACCEPT

curl -LO https://github.com/gravitl/netmaker/releases/download/v0.16.1/netclient-arm64

chmod +x netclient-arm64

./netclient-arm64 install

netclient join -t $TOKEN ~~

After setting egress in netmaker dashboard and then change the openwrt routing, can access to netmaker mesh inside.

My uci settings are as follows:

network.@route[0]=route
network.@route[0].interface='lan'
network.@route[0].target='10.0.0.0/24' # netmaker range
network.@route[0].gateway='10.0.0.253' # lxc container
network.@route[0].mtu='1280'

If need to set LXC automatic starting setting

uci show lxc-auto
uci add lxc-auto container
uci set lxc-auto.@container[-1].name=$CONTAINER_NAME
uci set lxc-auto.@container[-1].timeout=30
uci show lxc-auto
uci commit

if using lxc at luci.

# lxc luci
opkg install luci-app-lxc
## if it is not appered.
/etc/init.d/uhttpd stop
rm -rf /tmp/luci-*
/etc/init.d/uhttpd start

This method cant use the encryption ASIC of the Router itself, so the CPU load can be larger.

i'm not export in wireguard, netamekr, lxc, openwrt all. (literally beginner)
If there is a problem, it is faster to ask the related community.

이 게시글이 누군가에게 도움이 됬길 바랍니다. 그리고 openwrt에서 netmaker가 공식으로 지원되면 좋겠네요.

고마워요 netmaker!