Skip to content

Commit

Permalink
[TEP-0056]: Initial set of API refactors pertinent to Pipelines in Pi…
Browse files Browse the repository at this point in the history
…pelines

This is the first of many commits to begin implementing TEP-0056

- Added PipelineRef and PipelineSpec to PipelineTask
- Updated and added pipeline validation tests to reflect the above additions
- Updated and added pipeline type tests to reflect the above additions
- Added a initial set of docs and examples using the existing proposal
- The changes in this PR does not change existing behavior and merely introduces fields to existing types

Note: Pipelines in Pipelines is not yet implemented
Signed-off-by: Priti Desai <[email protected]>
  • Loading branch information
bhujangr authored and pritidesai committed Sep 5, 2023
1 parent c887347 commit bf2597e
Show file tree
Hide file tree
Showing 17 changed files with 1,819 additions and 47 deletions.
68 changes: 64 additions & 4 deletions docs/pipeline-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1797,7 +1797,7 @@ map[string]string
<h3 id="tekton.dev/v1.PipelineRef">PipelineRef
</h3>
<p>
(<em>Appears on:</em><a href="#tekton.dev/v1.PipelineRunSpec">PipelineRunSpec</a>)
(<em>Appears on:</em><a href="#tekton.dev/v1.PipelineRunSpec">PipelineRunSpec</a>, <a href="#tekton.dev/v1.PipelineTask">PipelineTask</a>)
</p>
<div>
<p>PipelineRef can be used to refer to a specific instance of a Pipeline.</p>
Expand Down Expand Up @@ -2453,7 +2453,7 @@ TaskRunStatus
<h3 id="tekton.dev/v1.PipelineSpec">PipelineSpec
</h3>
<p>
(<em>Appears on:</em><a href="#tekton.dev/v1.Pipeline">Pipeline</a>, <a href="#tekton.dev/v1.PipelineRunSpec">PipelineRunSpec</a>, <a href="#tekton.dev/v1.PipelineRunStatusFields">PipelineRunStatusFields</a>)
(<em>Appears on:</em><a href="#tekton.dev/v1.Pipeline">Pipeline</a>, <a href="#tekton.dev/v1.PipelineRunSpec">PipelineRunSpec</a>, <a href="#tekton.dev/v1.PipelineRunStatusFields">PipelineRunStatusFields</a>, <a href="#tekton.dev/v1.PipelineTask">PipelineTask</a>)
</p>
<div>
<p>PipelineSpec defines the desired state of Pipeline.</p>
Expand Down Expand Up @@ -2746,6 +2746,36 @@ Kubernetes meta/v1.Duration
Refer Go&rsquo;s ParseDuration documentation for expected format: <a href="https://golang.org/pkg/time/#ParseDuration">https://golang.org/pkg/time/#ParseDuration</a></p>
</td>
</tr>
<tr>
<td>
<code>pipelineRef</code><br/>
<em>
<a href="#tekton.dev/v1.PipelineRef">
PipelineRef
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>PipelineRef is a reference to a pipeline definition
Note: PipelineRef is in preview mode and not yet supported</p>
</td>
</tr>
<tr>
<td>
<code>pipelineSpec</code><br/>
<em>
<a href="#tekton.dev/v1.PipelineSpec">
PipelineSpec
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>PipelineSpec is a specification of a pipeline
Note: PipelineSpec is in preview mode and not yet supported</p>
</td>
</tr>
</tbody>
</table>
<h3 id="tekton.dev/v1.PipelineTaskMetadata">PipelineTaskMetadata
Expand Down Expand Up @@ -9631,7 +9661,7 @@ optional: false - the resource is considered required (default/equivalent of not
<h3 id="tekton.dev/v1beta1.PipelineRef">PipelineRef
</h3>
<p>
(<em>Appears on:</em><a href="#tekton.dev/v1beta1.PipelineRunSpec">PipelineRunSpec</a>)
(<em>Appears on:</em><a href="#tekton.dev/v1beta1.PipelineRunSpec">PipelineRunSpec</a>, <a href="#tekton.dev/v1beta1.PipelineTask">PipelineTask</a>)
</p>
<div>
<p>PipelineRef can be used to refer to a specific instance of a Pipeline.</p>
Expand Down Expand Up @@ -10445,7 +10475,7 @@ TaskRunStatus
<h3 id="tekton.dev/v1beta1.PipelineSpec">PipelineSpec
</h3>
<p>
(<em>Appears on:</em><a href="#tekton.dev/v1beta1.Pipeline">Pipeline</a>, <a href="#tekton.dev/v1beta1.PipelineRunSpec">PipelineRunSpec</a>, <a href="#tekton.dev/v1beta1.PipelineRunStatusFields">PipelineRunStatusFields</a>)
(<em>Appears on:</em><a href="#tekton.dev/v1beta1.Pipeline">Pipeline</a>, <a href="#tekton.dev/v1beta1.PipelineRunSpec">PipelineRunSpec</a>, <a href="#tekton.dev/v1beta1.PipelineRunStatusFields">PipelineRunStatusFields</a>, <a href="#tekton.dev/v1beta1.PipelineTask">PipelineTask</a>)
</p>
<div>
<p>PipelineSpec defines the desired state of Pipeline.</p>
Expand Down Expand Up @@ -10765,6 +10795,36 @@ Kubernetes meta/v1.Duration
Refer Go&rsquo;s ParseDuration documentation for expected format: <a href="https://golang.org/pkg/time/#ParseDuration">https://golang.org/pkg/time/#ParseDuration</a></p>
</td>
</tr>
<tr>
<td>
<code>pipelineRef</code><br/>
<em>
<a href="#tekton.dev/v1beta1.PipelineRef">
PipelineRef
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>PipelineRef is a reference to a pipeline definition
Note: PipelineRef is in preview mode and not yet supported</p>
</td>
</tr>
<tr>
<td>
<code>pipelineSpec</code><br/>
<em>
<a href="#tekton.dev/v1beta1.PipelineSpec">
PipelineSpec
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>PipelineSpec is a specification of a pipeline
Note: PipelineSpec is in preview mode and not yet supported</p>
</td>
</tr>
</tbody>
</table>
<h3 id="tekton.dev/v1beta1.PipelineTaskInputResource">PipelineTaskInputResource
Expand Down
115 changes: 115 additions & 0 deletions docs/pipelines-in-pipelines.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
<!--
---
linkTitle: "Pipelines in Pipelines"
weight: 406
---
-->

# Pipelines in Pipelines

- [Overview](#overview)
- [Specifying `pipelineRef` in `Tasks`](#specifying-pipelineref-in-pipelinetasks)
- [Specifying `pipelineSpec` in `Tasks`](#specifying-pipelinespec-in-pipelinetasks)
- [Specifying `Parameters`](#specifying-parameters)

## Overview

A mechanism to define and execute Pipelines in Pipelines, alongside Tasks and Custom Tasks, for a more in-depth background and inspiration, refer to the proposal [TEP-0056](https://github.com/tektoncd/community/blob/main/teps/0056-pipelines-in-pipelines.md "Proposal").

> :seedling: **Pipelines in Pipelines is an [alpha](additional-configs.md#alpha-features) feature.**
> The `enable-api-fields` feature flag must be set to `"alpha"` to specify `pipelineRef` or `pipelineSpec` in a `pipelineTask`.
> This feature is in Preview Only mode and not yet supported/implemented.
## Specifying `pipelineRef` in `pipelineTasks`

Defining Pipelines in Pipelines at authoring time, by either specifying `PipelineRef` or `PipelineSpec` fields to a `PipelineTask` alongside `TaskRef` and `TaskSpec`.

For example, a Pipeline named security-scans which is run within a Pipeline named clone-scan-notify where the PipelineRef is used:

```
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: security-scans
spec:
tasks:
- name: scorecards
taskRef:
name: scorecards
- name: codeql
taskRef:
name: codeql
---
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: clone-scan-notify
spec:
tasks:
- name: git-clone
taskRef:
name: git-clone
- name: security-scans
pipelineRef:
name: security-scans
- name: notification
taskRef:
name: notification
```

## Specifying `pipelineSpec` in `pipelineTasks`

The `pipelineRef` [example](#specifying-pipelineref-in-pipelinetasks) can be modified to use PipelineSpec instead of PipelineRef to instead embed the Pipeline specification:
```
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: clone-scan-notify
spec:
tasks:
- name: git-clone
taskRef:
name: git-clone
- name: security-scans
pipelineSpec:
tasks:
- name: scorecards
taskRef:
name: scorecards
- name: codeql
taskRef:
name: codeql
- name: notification
taskRef:
name: notification
```

## Specifying `Parameters`

Pipelines in Pipelines consume Parameters in the same way as Tasks in Pipelines
```
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: clone-scan-notify
spec:
params:
- name: repo
value: $(params.repo)
tasks:
- name: git-clone
params:
- name: repo
value: $(params.repo)
taskRef:
name: git-clone
- name: security-scans
params:
- name: repo
value: $(params.repo)
pipelineRef:
name: security-scans
- name: notification
taskRef:
name: notification
```
54 changes: 54 additions & 0 deletions docs/pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ weight: 203
- [Specifying `Parameters`](#specifying-parameters)
- [Adding `Tasks` to the `Pipeline`](#adding-tasks-to-the-pipeline)
- [Specifying Remote Tasks](#specifying-remote-tasks)
- [Specifying `Pipelines` in `PipelineTasks`](#specifying-pipelines-in-pipelinetasks)
- [Specifying `Parameters` in `PipelineTasks`](#specifying-parameters-in-pipelinetasks)
- [Specifying `Matrix` in `PipelineTasks`](#specifying-matrix-in-pipelinetasks)
- [Specifying `Workspaces` in `PipelineTasks`](#specifying-workspaces-in-pipelinetasks)
Expand Down Expand Up @@ -317,6 +318,59 @@ tasks:
value: task/golang-build/0.3/golang-build.yaml
```

### Specifying `Pipelines` in `PipelineTasks`

> :seedling: **Specifying `pipelines` in `PipelineTasks` is an [alpha](additional-configs.md#alpha-features) feature.**
> The `enable-api-fields` feature flag must be set to `"alpha"` to specify `PipelineRef` or `PipelineSpec` in a `PipelineTask`.
> This feature is in **Preview Only** mode and not yet supported/implemented.

Apart from `taskRef` and `taskSpec`, `pipelineRef` and `pipelineSpec` allows you to specify a `pipeline` in `pipelineTask`.
This allows you to generate a child `pipelineRun` which is inherited by the parent `pipelineRun`.

```
kind: Pipeline
metadata:
name: security-scans
spec:
tasks:
- name: scorecards
taskSpec:
steps:
- image: alpine
name: step-1
script: |
echo "Generating scorecard report ..."
- name: codeql
taskSpec:
steps:
- image: alpine
name: step-1
script: |
echo "Generating codeql report ..."
---
apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
name: clone-scan-notify
spec:
tasks:
- name: git-clone
taskSpec:
steps:
- image: alpine
name: step-1
script: |
echo "Cloning a repo to run security scans ..."
- name: security-scans
runAfter:
- git-clone
pipelineRef:
name: security-scans
---
```
For further information read [Pipelines in Pipelines](./pipelines-in-pipelines.md)
### Specifying `Parameters` in `PipelineTasks`
You can also provide [`Parameters`](tasks.md#specifying-parameters):
Expand Down
14 changes: 13 additions & 1 deletion pkg/apis/pipeline/v1/openapi_generated.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions pkg/apis/pipeline/v1/pipeline_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,16 @@ type PipelineTask struct {
// Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration
// +optional
Timeout *metav1.Duration `json:"timeout,omitempty"`

// PipelineRef is a reference to a pipeline definition
// Note: PipelineRef is in preview mode and not yet supported
// +optional
PipelineRef *PipelineRef `json:"pipelineRef,omitempty"`

// PipelineSpec is a specification of a pipeline
// Note: PipelineSpec is in preview mode and not yet supported
// +optional
PipelineSpec *PipelineSpec `json:"pipelineSpec,omitempty"`
}

// IsCustomTask checks whether an embedded TaskSpec is a Custom Task
Expand Down
Loading

0 comments on commit bf2597e

Please sign in to comment.