container hostname gotcha leads to broken prune and compact

[hozza]

TL;DR:

When using borgmatic-collective/docker-borgmatic you should probably set the archive_name_format: 'my-pc-backup-{now:%Y-%m-%d-%H%M%S} specifically in your config, or your prunes/compacting is likely not working as you might expect.

This should be highlighted on the README imo.

Details:
borgmatic by default will use archive_name_format: '{hostname}-{now:%Y-%m-%d-%H%M%S}.

This is fine when running borgmatic natively where the hostname is likely representative of the data being backed up, but this does not work in the context of docker, especially where the host data is the sole source. I suspect this is a common use case.

Each time a container is rebuilt the hostname changes and so the default config will lead to your archives having changing names over time. These changing hostnames for the same backup data over time stops 'prune' or 'compact' working across all the archives as expected.

To prevent this issue you can specifically set the archive name in the borgmatic config file e.g. 'my-server-backup-{now:%Y-%m-%d-%H%M%S}`

To fix an existing repo with many names for the same backup due to the docker hostname changes, you can use match_archives: "*" to allow prune to work across all the named archives (however only do this to a repo that contains only one backup source, as it'll match all archives). The proper way to solve this is to specifically set a hostname as above, and then use the borgmatic borg rename command to rename all the archives that should be the same hostname/name.

FYI: Renaming archives is impossible if you're using an "append_only" ssh key from BorgBase for "untrusted" clients. You'll need to setup a full access key to rename them, then the usual prune from borgmatic using append only key and compact from borgbase server-side will work as expected across all archives.

This is not really a bug but is more a configuration gotcha when using borgmatic in docker, and it should be highlighted for those new to borg/borgmatic. I'll submit a PR shortly to update the readme, and hopefully this issue can be found by others in the future who can't get their borgmatic prune's working when using borgmatic-collective/docker-borgmatic.

I hope this will save people a LOT of backup data that should have been pruned!

Note: borg >2 will use archive_name_format: '{hostname}' only as I understand it, before this version the names must be unique.

Ref: #254 (comment)

[witten]

Great detective work, but wouldn't it be easier just to set a fixed container name so a random one isn't generated each time? For instance, with Compose you can add container_name: borgmatic or whatever.

[hozza]

Great detective work, but wouldn't it be easier just to set a fixed container name so a random one isn't generated each time? For instance, with Compose you can add container_name: borgmatic or whatever.

Unfortunately, this does not work. The container_name docker compose/docker run directive does not set the containers hostname.

[witten]

Got it. And what about setting hostname: explicitly?

[hozza]

Yeah that could work! Although I've not seen docker hostname used often in the wild as it's counter intuitive due to dockers "unique" networking. (container name vs service name vs hostname for internal networking).

Does it seem less self-exploratory to set hostname to specify the containers archive name when the user is already configuring the borgmatic conf? Also I'm not sure how hostname being set at the docker level would work in a HA/swarm style setup vs archive name configuration.

[witten]

Yeah, I think either approach could work for different use cases. I just foresee some users using the default borgmatic config and ignoring your helpful recommendation in the PR..