feat(kubernetes): add S3_EXPIRE_DAYS

This adds the variable S3_EXPIRE_DAYS.

The idea of this feature is to allow the script to prune expired
snapshot files on the S3 compatible remote storage. Files are considered
expired once they exceed the threshold defined by S3_EXPIRE_DAYS.

This feature is usefull for S3 compatible storage where there exist no
lifecycle rules to clean up the storage of expired or old files, such
as:
* cloudscale object storage
* Exoscale simple object storage (SOS)

It is recommended to also configure a "Governance" lock on the files, to
ensure no files are deleted by accident before the defined
S3_EXPIRE_DAYS threshold.

kubernetes/README.md

@@ -15,5 +15,20 @@ After the snapshot is created in a temporary directory, `s3cmd` is used to sync
 * `S3_URI` - S3 URI to use to upload (s3://xxx)
 * `S3_BUCKET` - S3 bucket to point to
 * `S3_HOST` - S3 endpoint
+* `S3_EXPIRE_DAYS` - Delete files older than this threshold (expired)
 * `AWS_ACCESS_KEY_ID` - Access key to use to access S3
 * `AWS_SECRET_ACCESS_KEY` - Secret access key to use to access S3
+
+## Configuration of file retention (pruning)
+
+With AWS S3, use [lifecycle
+rules](https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-expire-general-considerations.html)
+to configure retention and automatic cleanup action (prune) for expired files.
+
+For other S3 compatible storage, ensure to set [Governance
+lock](https://community.exoscale.com/documentation/storage/versioning/#set-up-the-lock-configuration-for-a-bucket)
+to avoid any modification before `$S3_EXPIRE_DAYS`:
+
+```
+mc retention set --default GOVERNANCE "${S3_EXPIRE_DAYS}d" my-s3-remote/my-bucket
+```

kubernetes/cronjob.yaml

@@ -32,6 +32,9 @@ spec:
               value: bucketname
             - name: S3_URI
               value: s3://bucketname
+              # leave empty to retain snapshot files (default)
+            - name: S3_EXPIRE_DAYS
+              value:
             - name: VAULT_ROLE
               value: vault-snapshot
             - name: VAULT_ADDR

kubernetes/vault-snapshot.sh

@@ -7,8 +7,22 @@ VAULT_TOKEN=$(vault write -field=token  auth/kubernetes/login role="${VAULT_ROLE
 export VAULT_TOKEN
 
 # create snapshot
-
 vault operator raft snapshot save /vault-snapshots/vault_"$(date +%F-%H%M)".snapshot
 
 # upload to s3
 s3cmd put /vault-snapshots/* "${S3_URI}" --host="${S3_HOST}" --host-bucket="${S3_BUCKET}"
+
+# remove expired snapshots
+if [ "${S3_EXPIRE_DAYS}" ]; then
+    s3cmd ls "${S3_URI}" --host="${S3_HOST}" --host-bucket="${S3_BUCKET}" | while read -r line; do
+        createDate=$(echo $line | awk {'print $1" "$2'})
+        createDate=$(date -d"$createDate" +%s)
+        olderThan=$(date --date "${S3_EXPIRE_DAYS} days ago" +%s)
+        if [[ $createDate -lt $olderThan ]]; then
+            fileName=$(echo $line | awk {'print $4'})
+            if [[ $fileName != "" ]]; then
+                s3cmd del "${S3_URI}/$fileName" --host="${S3_HOST}" --host-bucket="${S3_BUCKET}"
+            fi
+        fi
+    done;
+fi