Received snapshot does not contain the expected files

[dml33]

Hi.

I have just started with pyznsap.

I am taking snapshots from a remote XigmaNAS server.

My pyznap.conf is

 [ssh:22:[email protected]:pool1/www]
 frequent = 4
 hourly = 24
 daily = 7
 weekly = 4
 monthly = 6
 yearly = 1
 snap = yes
 clean = yes
 dest = poolUSB/NAS
 dest_auto_create = yes
 compress = lz4

In the NAS, I have the following content and no previous snapshots:

nas4free: ~# ls -la /mnt/pool1/www/
total 10
drwxrwxr-x   2 root  wheel   3 Jan 13 20:39 .
drwxrwxrwx  15 www   wheel  15 Dec 12 22:08 ..
-rw-r--r--   1 root  wheel   8 Jan 13 20:39 kk.txt
nas4free: ~#

The results of pyznap snap and later pyznap send:

pi@raspberrypi:/mnt $ sudo pyznap -v snap
Jan 13 20:44:51 INFO: Starting pyznap...
Jan 13 20:44:51 INFO: Taking snapshots...
Jan 13 20:44:52 DEBUG: Taking snapshots on [email protected]:pool1/www...
Jan 13 20:44:52 INFO: Taking snapshot [email protected]:pool1/www@pyznap_2021-01-13_20:44:52_yearly...
Jan 13 20:44:53 INFO: Taking snapshot [email protected]:pool1/www@pyznap_2021-01-13_20:44:53_monthly...
Jan 13 20:44:53 INFO: Taking snapshot [email protected]:pool1/www@pyznap_2021-01-13_20:44:53_weekly...
Jan 13 20:44:54 INFO: Taking snapshot [email protected]:pool1/www@pyznap_2021-01-13_20:44:54_daily...
Jan 13 20:44:54 INFO: Taking snapshot [email protected]:pool1/www@pyznap_2021-01-13_20:44:54_hourly...
Jan 13 20:44:55 INFO: Taking snapshot [email protected]:pool1/www@pyznap_2021-01-13_20:44:55_frequent...
Jan 13 20:44:55 INFO: Cleaning snapshots...
Jan 13 20:44:56 DEBUG: Cleaning snapshots on [email protected]:pool1/www...
Jan 13 20:44:56 INFO: Finished successfully...

pi@raspberrypi:/mnt $ sudo pyznap -v send
Jan 13 20:45:06 INFO: Starting pyznap...
Jan 13 20:45:06 INFO: Sending snapshots...
Jan 13 20:45:07 WARNING: lz4 does not exist on [email protected], continuing without compression...
Jan 13 20:45:07 INFO: Destination poolUSB/NAS does not exist, will create it...
Jan 13 20:45:08 INFO: Successfully created poolUSB/NAS...
Jan 13 20:45:08 DEBUG: Sending [email protected]:pool1/www to poolUSB/NAS...
Jan 13 20:45:08 INFO: No common snapshots on poolUSB/NAS, sending oldest snapshot [email protected]:pool1/www@pyznap_2021-01-13_20:44:52_yearly (~37.6K)...
Jan 13 20:45:09 INFO: Updating poolUSB/NAS with recent snapshot [email protected]:pool1/www@pyznap_2021-01-13_20:44:55_frequent (~3.0K)...
Jan 13 20:45:10 INFO: poolUSB/NAS is up to date...
Jan 13 20:45:10 INFO: Finished successfully...

The snapshots at destination are below:

pi@raspberrypi:/mnt $ zfs list -t snapshot
NAME                                              USED  AVAIL     REFER  MOUNTPOINT
poolUSB/NAS@pyznap_2021-01-13_20:44:52_yearly       0B      -       88K  -
poolUSB/NAS@pyznap_2021-01-13_20:44:53_monthly      0B      -       88K  -
poolUSB/NAS@pyznap_2021-01-13_20:44:53_weekly       0B      -       88K  -
poolUSB/NAS@pyznap_2021-01-13_20:44:54_daily        0B      -       88K  -
poolUSB/NAS@pyznap_2021-01-13_20:44:54_hourly       0B      -       88K  -
poolUSB/NAS@pyznap_2021-01-13_20:44:55_frequent     0B      -       88K  -

But, the dataset has no file inside:

pi@raspberrypi:/mnt $ ls -la /mnt/poolUSB/NAS/
total 2
drwxr-xr-x 2 root root 2 ene 13 20:16 .
drwxr-xr-x 4 root root 4 ene 13 20:16 ..
pi@raspberrypi:/mnt $

What is the problem?

BTW, f couse lz4 is available in the NAS server. In fact, the pool1/www dataset has the compression set as lz4. Why does that warning appear?

pool1/www compression lz4 local

Thanks!

[yboetz]

For compression the algorithm must be installed as a system package, e.g. through apt install lz4 on Ubuntu. pyznap cannot use the built-in lz4 from zfs.

About the missing file: Make sure the filesystem is mounted. pyznap has a problem where it unmounts filesystems for send/recv, so your dataset might just not be mounted.

[dml33]

Hi.

Thanks for the fast reply.

I have just realized that in embedded XigmaNAS the liblz4 package is not installed, so the lz4 command cannot be invoked. Is there any option to set a no-compression method for the transfer?

Thanks, the destination dataset was unmonted, indeed. Then, I mounted it manually by zfs mount pooUSB/NAS and I can see my test files :) This behaviour is a bug or it is deliberate, since, for this case, it was newly created by the pyznap send? Once mounted, I redid some pyzsnap send and it continue being mounted.

Thanks!

[yboetz]

If you don't want any compression you can set compress = none. Default is lzop, which helps quite a bit over ssh when your data is compressible, like lots of text documents. If you only have pictures/media, then compression doesn't make sense and just disable it as mentioned.

It's a mix of bug and deliberate :). For the zfs send/recv I have to set a flag in the receive command that unmounts filesystems. Sometimes they are then remounted, but often times not. So the flag is deliberate, but the datasets being mounted/unmounted after the send is a bit buggy, and I haven't figured out how to fix this. For me the datasets also stay mounted after I manually remount them once, but other people have said for them it's different.