Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Make vendir lazy: don't syncing if the config has not changed #675

Merged
merged 3 commits into from
Oct 5, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions proposals/vendir/001-lazy-synching-on-stable-config/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
title: "Lazy Synching"
authors: [ "Fritz Duchardt <[email protected]>" ]
status: "In Review"
approvers: [ @kumaritanushree, @vmunishwar, @neil-hickey, @joaopapereira ]
---

# Make vendir lazy: don't syncing if the config has not changed

## Problem Statement
We have a rollout mechanism that heavily relies on vendir to pull in upstream workloads from Helm Charts, OCI Images and Git Repositories. Most of these workloads have fixed versions and don't change unless their corresponding vendir.yaml is modified. Yet, currently vendir constrains us to sync everything on every run, which slows down the process unnecessarily.

## Proposal
Add an additional config option to run sync only if the config was changed after the last sync. Users can activate this feature for all workloads they deem stable.

### Goals
Users benefit from quicker syncing process, if they use vendir to pull in stable release versions.

### Specification / Use Cases
The feature can be activated in the `vendir.yml` at `contents` level, e.g.:

```
apiVersion: vendir.k14s.io/v1alpha1
kind: Config
directories:
- path: vendor
contents:
- path: custom-repo-custom-version
lazy: true
helmChart:
name: contour
version: "7.10.1"
repository:
url: https://charts.bitnami.com/bitnami
```

Lazily `contents` are synced under two conditions:
- When their corresponding config section has changed. The mechanism works in a generic fashion for all repository types and reacts to any change to the `contents` config section.
- When the path of the parent `directory` has changed.

To track changes to the config, the vendir.lock.yml is amended with a configDigest value that contains a hash representing the state of a `contents` config section at the last sync, e.g.:
```
kind: LockConfig
apiVersion: vendir.k14s.io/v1alpha1
directories:
- path: vendor
contents:
- path: custom-repo-custom-version
configDigest: e8a5d1511f2eb22b160bb849e5c8be39da1c4ffa5fd56ded71ff095a8b24720b
helmChart:
appVersion: 1.20.1
version: 7.10.1
```
Config digests are only added, if vendir is run with the `lazy` setting.

To force a sync despite the `lazy` setting, a new option is added to the vendir binary, e.g.
```
vendir sync --lazy=false
```

### Other Approaches Considered
A simpler approach could work entirely at binary level, e.g. a lazy option to enable lazy-syncing on all `contents` of the synced vendir.yml:
```
vendir sync --lazy
```
The implementation for this feature would be much simpler, since an upfront comparison of vendir.yml and vendir.lock.yml would simply stop execution. A modification to the sync implementation that checks for changes individually for each `contents`, selectively skipping syncs while still building a valid lock file would not be required.

With this approach one loses the ability to activate `lazy` syncing separately for specific `contents`.


## Open Questions


## Answered Questions