--keys-filter-cmd CMD

[yarikoptic]

Add support for the tool to be paired with an (external) tool to inform the tool on either any particular key should be backed up.

The immediate usecase for DANDI is to cater to use-case @satra would like to achieve that we would avoid backing up old known to be not needed huge .h5 files for 000108 (IIRC), or overall -- do not bother backing up not actively used/references assets (which I would prefer to avoid -- too much possibility for a bug). So, it would be great to be able to explicitly filter out some downloads based on some external domain-specific knowledge.

Inspired by git annex CMD --batch mode, I envision

• s3invsync passing json 1 liner record with "key" and "versionId" (if known) to the external CMD via stdin
• CMD in turn in stdout responds with a json record on the "action" which could be
  • null - no opinion, up to the s3invsync
  • "backup" - do ensure local presence
  • "skip" - do not backup
  • "kill" - do not backup and ensure that no backup of this key/version locally exists,
  • "kill-all-versions" - do not backup and ensure that no backup of this key of any version locally exists.

Last two might be important for the use cases where users demand that we destroy all copies of their data