How is a simple pyznap send supposed to work?

[recklessnl]

So I have two datasets: ssd/files and hdd/backup

I want to backup dataset ssd/files to hdd/backup. Preferably a 1-to-1 copy; a full backup. So I type:

pyznap send -s ssd/files -d hdd/backup

Once it's finished, I browse to the hdd/backup folder and I see nothing. I do see the snapshots listed when I do zfs list -t snapshot hdd/backup. Is this expected behavior? How am I supposed to progress this snapshot to an actual backup, with files and all, using pyznap?

Wouldn't it be better if pyznap could do a full incremental backup where the files appear in the dataset? Just wondering what is the recommended procedure for something like this.

[yboetz]

Most likely your dataset hdd/backup is just not mounted so it doesn't show the files. You can check if the filesystem is mounted with zfs list -r -o name,mounted,mountpoint.

pyznap does do incremental backup. Whenever you do pyznap send it searches for the most recent common snapshot and then sends data starting from that snapshots to the most recent snapshot.

[recklessnl]

I'm seeing a problem now. When I create a brand new empty dataset (zfs create hdd/test) it will immediately mount. No issues.

When I then use pyznap send -s ssd/files -d hdd/test, it will end up unmounting that dataset for some reason. Here's the output from that command:

Sep 23 02:55:47 INFO: Starting pyznap...
Sep 23 02:55:47 INFO: Sending snapshots...
Sep 23 02:55:47 INFO: No common snapshots on hdd/test, sending oldest snapshot ssd/files@pyznap_2019-09-23_02:30:03_daily (~2.8M)...
  2.77MiB 0:00:01 [2.25MiB/s] [====================================================> ] 99%
Sep 23 02:55:48 INFO: Updating hdd/test with recent snapshot ssd/files@pyznap_2019-09-23_02:45:01_frequent (~624.0B)...
Sep 23 02:55:49 INFO: hdd/test is up to date...
Sep 23 02:55:49 INFO: Finished successfully...

Then afterwards, using the command you provided, I see it has been unmounted. Which means I can't see any files when browsing to that dataset.
When I re-mount it using zfs mount hdd/test, I indeed do see the files.

Is this expected behavior? Wouldn't it be helpful if pyznap recognized this and didn't unmount the dataset?

[yboetz]

While receiving the dataset is unmounted. pyznap uses the zfs receive -u flag, meaning the filesystem is not remounted after the dataset is received. Though my backup dataset stays mounted even after receiving, so I thought pyznap behaves a bit differently... I don't know what I did different that my backup datasets stay mounted.

[recklessnl]

I'd be interested to find out how it works for you then.
I would prefer it if pyznap re-mounted the dataset after the send backup is complete. I don't know why that's not standard anyway. Saves the user doing another command.

[yboetz]

I think my idea was then that it should not try to mount datasets in the same mountpoint as the source. I had a problem at some point when I zfs send a dataset and after reboot my backup dataset was mounted instead of the source. I think I misunderstood the -u flag when I did this, I will look into this. I'll update when I know why it works differently on my system...

[yboetz]

It's a bit strange, it seems I cannot change the -u option. If I do so I get errors while sending, as certain datasets cannot be unmounted anymore for receiving. I don't know why this happens yet...

Though on all my systems my remote backups are not unmounted after every pyznap send. After the first send they get unmounted, but I can then manually mount them using zfs mount ..., and after that they will stay mounted, even though the -u option is set. So all my datasets stay mounted all the time.

[recklessnl]

That's weird... is there anything I can try to stop it from unmounting? Or perhaps Pyznap can perform a check, upon completion of the backup, to re-mount it in case it hasn't been remounted.

Would it be possible to add that?

[yboetz]

Does it still unmount even after you manually mount it at some point using zfs mount? That works for me, but for the moment there is nothing else to stop it from unmounting it seems, but I'm still thinking about how to change this.

[recklessnl]

Yes, it does mount when I manually mount it, but the point is that it would be much handier if that happened automatically.

And in case the user does NOT want an automatic re-mount (I can't think of any reason, but let's just assume there is one) perhaps add a flag to the pyznap command to disable auto remounting? I think it's better to have it remount automatically upon successful send.

[yboetz]

Obviously it would be better if datasets are remounted automatically. I only had to manually remount them once, after that the were never unmounted again. As I mentioned before, I cannot simply remove the -u flag, as I get some errors while sending then. So I'm not sure yet how to fix this, but I'll keep this open as a reminder.

[recklessnl]

Hi @yboetz, have you looked into this issue more?

[yboetz]

Not much. When I last tested it, I couldn't get it to work without the -u flag, so I haven't figured out how to fix it. It's still on my todo list, but I haven't had much time recently to work on pyznap...