Persistent disks deleted after redeploying Memgraph

[Josipmrden]

Some helm chart functionality seems to delete the persistent disks after redeploying Memgraph (stateful set). More info coming.

[matkob]

The issue is about the PersistentVolume Reclaim Policy that is used by the charts. Current default behaviour deletes the precious volumes the moment the PersistentVolumeClaim is deleted. This is quite risky and in my setup it proved to cause some data loss.

It's not an obvious bug on memgraph side but it's definitely an area for improvement. The setup should be robust enough to survive some mistakes on user side. And this can be easily achieved by changing that default reclaim policy. I'd suggest to set it to Retain by default and allow users to change it through Helm values. Worst case scenario, users will have to delete the volumes by hand which is much better "side effect" than loosing the data.

More information on this topic:
https://kubernetes.io/docs/concepts/storage/persistent-volumes/#reclaim-policy
https://kubernetes.io/docs/tasks/administer-cluster/change-pv-reclaim-policy/

[antejavor]

Hi @matkob, thanks for providing this feedback.

This is quite reasonable, I agree with the reasoning, and we will include it in the next release of charts. 🦾