From fb7976d8004a566f24bb6eebc1b84f47cc8b5998 Mon Sep 17 00:00:00 2001 From: Faeka Ansari Date: Tue, 19 Nov 2024 19:40:54 +0530 Subject: [PATCH 1/3] update warehouse documentation to include subscription details Signed-off-by: Faeka Ansari --- docs/docs/15-concepts.md | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/docs/docs/15-concepts.md b/docs/docs/15-concepts.md index cf83114cc..27d8634d2 100644 --- a/docs/docs/15-concepts.md +++ b/docs/docs/15-concepts.md @@ -192,9 +192,28 @@ spec: Kargo uses [semver](https://github.com/masterminds/semver#checking-version-constraints) to handle semantic versioning constraints. ::: -#### Git Subscription Path Filtering +#### Image Subscription -In some cases, it may be necessary to constrain the paths within a Git +For subscriptions to container image repositories, the `imageSelectionStrategy` field specifies the method for selecting +the desired image. The available strategies are: + +- `Digest`: Selects the image with the specified digest. +- `Lexical`: Selects the image with the lexicographically greatest tag. +- `NewestBuild`: Selects the image with the most recent build time. +- `SemVer`: Selects the image that best matches a semantic versioning constraint. + +#### Git Subscription + +In subscriptions to Git repositories, the `commitSelectionStrategy` field +specifies the method for selecting the desired commit. +The available strategies are: + +- `Lexical`: Selects the commit with the lexicographically greatest reference. +- `NewestFromBranch`: Selects the most recent commit from a specified branch. +- `NewestTag`: Selects the most recent commit associated with a tag. +- `SemVer`: Selects the commit that best matches a semantic versioning constraint. + +Additionally, in some cases, it may be necessary to constrain the paths within a Git repository that a `Warehouse` will consider as triggers for `Freight` production. This is especially useful for GitOps repositories that are "monorepos" containing configuration for multiple applications. From 1774d440c002c50597761f95d98c72c3506f19a9 Mon Sep 17 00:00:00 2001 From: Faeka Ansari Date: Tue, 19 Nov 2024 20:00:37 +0530 Subject: [PATCH 2/3] created a separate working with warehouses docs Signed-off-by: Faeka Ansari --- docs/docs/15-concepts.md | 158 +---------------- .../16-working-with-warehouses.md | 161 ++++++++++++++++++ 2 files changed, 166 insertions(+), 153 deletions(-) create mode 100644 docs/docs/30-how-to-guides/16-working-with-warehouses.md diff --git a/docs/docs/15-concepts.md b/docs/docs/15-concepts.md index 27d8634d2..172d8f262 100644 --- a/docs/docs/15-concepts.md +++ b/docs/docs/15-concepts.md @@ -81,6 +81,11 @@ A **warehouse** is a _source_ of freight. A warehouse subscribes to one or more: Anytime something new is discovered in any repository to which a warehouse subscribes, the warehouse produces a new piece of freight. +:::note +For technical details of the corresponding `Warehouse` resource type, +refer to [Working with Warehouses](./30-how-to-guides/16-working-with-warehouses.md). +::: + ### What is a Promotion? A **promotion** is a request to move a piece of freight into a specified stage. @@ -157,159 +162,6 @@ status: prod: {} ``` -### `Warehouse` Resources - -Each Kargo warehouse is represented by a Kubernetes resource of type -`Warehouse`. - -A `Warehouse` resource's most important field is its `spec.subscriptions` field, -which is used to subscribe to one or more: - -* Container image repositories - -* Git repositories - -* Helm charts repositories - -The following example shows a `Warehouse` resource that subscribes to a -container image repository and a Git repository: - -```yaml -apiVersion: kargo.akuity.io/v1alpha1 -kind: Warehouse -metadata: - name: my-warehouse - namespace: kargo-demo -spec: - subscriptions: - - image: - repoURL: public.ecr.aws/nginx/nginx - semverConstraint: ^1.26.0 - - git: - repoURL: https://github.com/example/kargo-demo.git -``` -:::info -Kargo uses [semver](https://github.com/masterminds/semver#checking-version-constraints) to handle semantic versioning constraints. -::: - -#### Image Subscription - -For subscriptions to container image repositories, the `imageSelectionStrategy` field specifies the method for selecting -the desired image. The available strategies are: - -- `Digest`: Selects the image with the specified digest. -- `Lexical`: Selects the image with the lexicographically greatest tag. -- `NewestBuild`: Selects the image with the most recent build time. -- `SemVer`: Selects the image that best matches a semantic versioning constraint. - -#### Git Subscription - -In subscriptions to Git repositories, the `commitSelectionStrategy` field -specifies the method for selecting the desired commit. -The available strategies are: - -- `Lexical`: Selects the commit with the lexicographically greatest reference. -- `NewestFromBranch`: Selects the most recent commit from a specified branch. -- `NewestTag`: Selects the most recent commit associated with a tag. -- `SemVer`: Selects the commit that best matches a semantic versioning constraint. - -Additionally, in some cases, it may be necessary to constrain the paths within a Git -repository that a `Warehouse` will consider as triggers for `Freight` -production. This is especially useful for GitOps repositories that are -"monorepos" containing configuration for multiple applications. - -The paths that may or must not trigger `Freight` production may be specified -using a combination of the `includePaths` and `excludePaths` fields of a Git -repository subscription. - -The following example demonstrates a `Warehouse` with a Git repository -subscription that will only produce new `Freight` when the latest commit -(selected by the applicable commit selection strategy) contains changes in the -`apps/guestbook` directory since the last piece of `Freight` produced by the -`Warehouse`: - -```yaml -apiVersion: kargo.akuity.io/v1alpha1 -kind: Warehouse -metadata: - name: my-warehouse - namespace: kargo-demo -spec: - subscriptions: - - git: - repoURL: https://github.com/example/kargo-demo.git - includePaths: - - apps/guestbook -``` - -The next example demonstrates the opposite: a `Warehouse` with a Git repository -subscription that will only produce new `Freight` when the latest commit -(selected by the applicable commit selection strategy) contains changes to paths -_other than_ the repository's `docs/` directory: - -```yaml -apiVersion: kargo.akuity.io/v1alpha1 -kind: Warehouse -metadata: - name: my-warehouse - namespace: kargo-demo -spec: - subscriptions: - - git: - repoURL: https://github.com/example/kargo-demo.git - excludePaths: - - docs -``` - -`includePaths` and `excludePaths` may be combined to include a broad set of -paths and then exclude a subset of those. The following example demonstrates a -`Warehouse` with a Git repository subscription that will only produce new -`Freight` when the latest commit (selected by the applicable commit selection -strategy) contains changes _within_ the `apps/guestbook` directory _other than_ -the `apps/guestbook/README.md`: - -```yaml -apiVersion: kargo.akuity.io/v1alpha1 -kind: Warehouse -metadata: - name: my-warehouse - namespace: kargo-demo -spec: - subscriptions: - - git: - repoURL: https://github.com/example/kargo-demo.git - includePaths: - - apps/guestbook - excludePaths: - - apps/guestbook/README.md -``` - -:::note -It is important to understand that new `Freight` will be produced when the -latest commit (selected by the applicable commit selection strategy) contains -_even a single change_ that is: - -1. Implicitly included via undefined `includePaths`. - -     OR - - Explicitly included via `includePaths`. - - AND - -2. Not explicitly excluded via `excludePaths`. -::: - -:::info -By default, the strings in the `includePaths` and `excludePaths` fields are -treated as exact paths to files or directories. (Selecting a directory will -implicitly select all paths within that directory.) - -Paths may _also_ be specified using glob patterns (by prefixing the string with -`glob:`) or regular expressions (by prefixing the string with `regex:` or -`regexp:`). -::: - ### `Promotion` Resources Each Kargo promotion is represented by a Kubernetes resource of type diff --git a/docs/docs/30-how-to-guides/16-working-with-warehouses.md b/docs/docs/30-how-to-guides/16-working-with-warehouses.md new file mode 100644 index 000000000..a111546b4 --- /dev/null +++ b/docs/docs/30-how-to-guides/16-working-with-warehouses.md @@ -0,0 +1,161 @@ +--- +description: Learn how to work effectively with Warehouses +sidebar_label: Working with Warehouses +--- + +# Working with Warehouses + +Each Kargo warehouse is represented by a Kubernetes resource of type +`Warehouse`. + +## The `Warehouse` Resource Type + +A `Warehouse` resource's most important field is its `spec.subscriptions` field, +which is used to subscribe to one or more: + +* Container image repositories + +* Git repositories + +* Helm charts repositories + +The following example shows a `Warehouse` resource that subscribes to a +container image repository and a Git repository: + +```yaml +apiVersion: kargo.akuity.io/v1alpha1 +kind: Warehouse +metadata: + name: my-warehouse + namespace: kargo-demo +spec: + subscriptions: + - image: + repoURL: public.ecr.aws/nginx/nginx + semverConstraint: ^1.26.0 + - git: + repoURL: https://github.com/example/kargo-demo.git +``` +:::info +Kargo uses [semver](https://github.com/masterminds/semver#checking-version-constraints) to handle semantic versioning constraints. +::: + +### Image Subscription + +For subscriptions to container image repositories, the `imageSelectionStrategy` field specifies the method for selecting +the desired image. The available strategies for subscribing to an image repository are: + +- `Digest`: Selects the image with the specified digest. +- `Lexical`: Selects the image with the lexicographically greatest tag. +- `NewestBuild`: Selects the image with the most recent build time. +- `SemVer`: Selects the image that best matches a semantic versioning constraint. + +### Git Subscription + +In subscriptions to Git repositories, the `commitSelectionStrategy` field +specifies the method for selecting the desired commit. +The available strategies for subscribing to a git repository are: + +- `Lexical`: Selects the commit with the lexicographically greatest reference. +- `NewestFromBranch`: Selects the most recent commit from a specified branch. +- `NewestTag`: Selects the most recent commit associated with a tag. +- `SemVer`: Selects the commit that best matches a semantic versioning constraint. + +#### Git Subscription Path Filtering + +In some cases, it may be necessary to constrain the paths within a Git +repository that a `Warehouse` will consider as triggers for `Freight` +production. This is especially useful for GitOps repositories that are +"monorepos" containing configuration for multiple applications. + +The paths that may or must not trigger `Freight` production may be specified +using a combination of the `includePaths` and `excludePaths` fields of a Git +repository subscription. + +The following example demonstrates a `Warehouse` with a Git repository +subscription that will only produce new `Freight` when the latest commit +(selected by the applicable commit selection strategy) contains changes in the +`apps/guestbook` directory since the last piece of `Freight` produced by the +`Warehouse`: + +```yaml +apiVersion: kargo.akuity.io/v1alpha1 +kind: Warehouse +metadata: + name: my-warehouse + namespace: kargo-demo +spec: + subscriptions: + - git: + repoURL: https://github.com/example/kargo-demo.git + includePaths: + - apps/guestbook +``` + +The next example demonstrates the opposite: a `Warehouse` with a Git repository +subscription that will only produce new `Freight` when the latest commit +(selected by the applicable commit selection strategy) contains changes to paths +_other than_ the repository's `docs/` directory: + +```yaml +apiVersion: kargo.akuity.io/v1alpha1 +kind: Warehouse +metadata: + name: my-warehouse + namespace: kargo-demo +spec: + subscriptions: + - git: + repoURL: https://github.com/example/kargo-demo.git + excludePaths: + - docs +``` + +`includePaths` and `excludePaths` may be combined to include a broad set of +paths and then exclude a subset of those. The following example demonstrates a +`Warehouse` with a Git repository subscription that will only produce new +`Freight` when the latest commit (selected by the applicable commit selection +strategy) contains changes _within_ the `apps/guestbook` directory _other than_ +the `apps/guestbook/README.md`: + +```yaml +apiVersion: kargo.akuity.io/v1alpha1 +kind: Warehouse +metadata: + name: my-warehouse + namespace: kargo-demo +spec: + subscriptions: + - git: + repoURL: https://github.com/example/kargo-demo.git + includePaths: + - apps/guestbook + excludePaths: + - apps/guestbook/README.md +``` + +:::note +It is important to understand that new `Freight` will be produced when the +latest commit (selected by the applicable commit selection strategy) contains +_even a single change_ that is: + +1. Implicitly included via undefined `includePaths`. + + OR + + Explicitly included via `includePaths`. + + AND + +2. Not explicitly excluded via `excludePaths`. +::: + +:::info +By default, the strings in the `includePaths` and `excludePaths` fields are +treated as exact paths to files or directories. (Selecting a directory will +implicitly select all paths within that directory.) + +Paths may _also_ be specified using glob patterns (by prefixing the string with +`glob:`) or regular expressions (by prefixing the string with `regex:` or +`regexp:`). +::: From d3db02744cc6f39e53983ac9cddd2a2f572ed9a5 Mon Sep 17 00:00:00 2001 From: Faeka Ansari Date: Thu, 16 Jan 2025 19:31:43 +0530 Subject: [PATCH 3/3] adjust reviews Signed-off-by: Faeka Ansari --- .../16-working-with-warehouses.md | 45 +++++++++++++++---- 1 file changed, 37 insertions(+), 8 deletions(-) diff --git a/docs/docs/30-how-to-guides/16-working-with-warehouses.md b/docs/docs/30-how-to-guides/16-working-with-warehouses.md index a111546b4..5bbb2718f 100644 --- a/docs/docs/30-how-to-guides/16-working-with-warehouses.md +++ b/docs/docs/30-how-to-guides/16-working-with-warehouses.md @@ -45,10 +45,31 @@ Kargo uses [semver](https://github.com/masterminds/semver#checking-version-const For subscriptions to container image repositories, the `imageSelectionStrategy` field specifies the method for selecting the desired image. The available strategies for subscribing to an image repository are: -- `Digest`: Selects the image with the specified digest. -- `Lexical`: Selects the image with the lexicographically greatest tag. -- `NewestBuild`: Selects the image with the most recent build time. -- `SemVer`: Selects the image that best matches a semantic versioning constraint. +- `Digest`: This strategy is used when subscribing to a specific mutable tag, such as `latest`, which is generally + discouraged due to best practices favoring immutable tags. Users must supply a value in the `constraint` field, + specifying the mutable tag they wish to track. The strategy will retrieve the latest details for the image + tagged in this way, including any new or updated digest. + +- `Lexical`: This strategy selects the image with the lexicographically greatest tag, making it suitable + for scenarios where tags incorporate date/time stamps in formats like `yyyymmdd`. When using this + strategy, it's recommended to pair it with a regular expression in the `allowTags` field to limit + eligibility to tags that match the expected format, ensuring the correct selection of tags. + +- `NewestBuild`: This strategy selects the image with the most recent build time. + + :::warning + + `NewestBuild` requires retrieving metadata for every eligible tag, which can be slow and might + exceed the registry's rate limits. It's advisable to use the `allowTags` field to limit + the number of tags for which metadata is retrieved, thereby reducing the risk of hitting rate limits. + ::: + +- **SemVer**: This strategy selects the image that best matches a semantic versioning constraint. + + :::info + Kargo uses [semver](https://github.com/masterminds/semver#checking-version-constraints) to handle these contraints, + allowing users to define and manage versions precisely. + ::: ### Git Subscription @@ -56,10 +77,18 @@ In subscriptions to Git repositories, the `commitSelectionStrategy` field specifies the method for selecting the desired commit. The available strategies for subscribing to a git repository are: -- `Lexical`: Selects the commit with the lexicographically greatest reference. -- `NewestFromBranch`: Selects the most recent commit from a specified branch. -- `NewestTag`: Selects the most recent commit associated with a tag. -- `SemVer`: Selects the commit that best matches a semantic versioning constraint. +- `Lexical`: Selects the commit referenced by the lexicographically greatest tag. + It is particularly useful in scenarios where commit references, such as tags or branches, + incorporate date/time stamps in formats like `yyyymmdd`. + To ensure the correct selection, it's advisable to use regular expressions in the +`allowTags` or `allowBranches` field, which limit the acceptable format of the references, + preventing the selection of undesired tags like `zzz-custom` over something like `nightly-20241211`. +- `NewestFromBranch`: Selects the most recent commit from a specified branch. It's useful when tracking the latest changes in a branch that receives regular updates. +- `NewestTag`: Selects the most recent commit associated with a tag. Since tags are typically immutable, + there should be only one commit per tag. + To optimize this strategy, it's recommended to constrain the eligible tags using regular expressions or specific patterns, + ensuring the selection is limited to tags that follow a consistent naming convention. +- `SemVer`: Selects the commit referenced by a *tag* that best matches the constraint. #### Git Subscription Path Filtering