Working end to end example of tenant release pipelines

[ralphbean]

Over in QUBIP/pq-container#2, @lsm5 was trying to use the fedora instance of konflux run by the fedora + konflux sig to build their container. You can see additional discussion at QUBIP/pq-container#1.

They got the build to work in QUBIP/pq-container#4 and simple policy checks passed 🙌 , but they failed to get a tenant release pipeline set up to publish their image to a location that they care about.

They want to use the "tenant release pipelines" feature to release their image to quay.io/qubip/pq-container (an organization under their control).

Let's produce a working end to end example for them and/or a user guide how-to on how to do it.

[ralphbean]

I understand that when @scoheb tried to do this last week, he ran into issues where all of the https://github.com/konflux-ci/release-service-catalog pipelines expected a ReleasePlanAdmission to exist and so didn't work off the bat with tenant scoped pipelines.

[ralphbean]

From chatting with @rhartman93, it seems like there are two main issues:

• The existing release pipelines expect that a RPA exists.
• There's probably some additional RoleBindings that need to be applied to the default serviceaccount.

[ralphbean]

Scott was toying around with making pipelines work over in a tenant-release-pipelines branch on his fork.

[rhartman93]

* There's probably some additional RoleBindings that need to be applied to the default serviceaccount.

The issue i think was, the roles that our managed service accounts use, are cluster level, which tenant maintainers don't have access to, so they need to create their own role (with the same permissions) to bind to their SAs, solving that could be as simple as documenting that process

[ralphbean]

OK, I realized today that the version of release-service running on the fedora instance is so old (2 months) that it doesn't even have the tenants release pipeline feature.

We need to solve https://discussion.fedoraproject.org/t/mintmaker-renovate-update-infra-deployments-for-konflux/134050/2 first, and then we can make progress on this one.

[johnbieren]

I understand that when @scoheb tried to do this last week, he ran into issues where all of the https://github.com/konflux-ci/release-service-catalog pipelines expected a ReleasePlanAdmission to exist and so didn't work off the bat with tenant scoped pipelines.

That repo is all just managed pipelines so far, so that is expected. If we are going to write some widely used tenant pipelines, maybe they can also be in that repo, sure, but we should probably add some structure to differentiate

[ralphbean]

Looks like @lcarva is making some progress on this over at konflux-ci/release-service-catalog#672.

[johnbieren]

I brought up that PR to my team, and we don't really think with our current headcount we should modify our managed pipeline tasks to support tenant pipelines. I know this PR is from Lui and he did the work, but as soon as this task is in one tenant pipeline, it will likely be in many. Tenant pipelines are designed to allow a user to run their own stuff in their workspace. This is very different than the managed pipeline which we maintain so they will (hopefully) work for users and add features to them. If we start having tasks in both managed and tenant pipelines, it could get messy. Say the push-snapshot task, for example, is massaged to work for tenant pipelines. Then, many users start using the same tenant pipeline with this task in it. If the users want to change anything when they do the push, we need to modify the same managed push-snapshot task for this. The team consensus was that we aren't against this, but we don't feel we have the bandwidth right now to maintain tenant pipeline tasks in addition to all the managed pipeline tasks we already have (and will add in the future for other features).

It also isn't super clear to me why we even want to use the managed tasks in tenant pipelines. collect-data puts all of your CRs in the workspace and creates a merged data.json file. It is very specific to how the managed pipelines work. Tenant pipelines allow you to provide pipeline parameters in your ReleasePlan (impossible with managed pipelines) as that is more like normal tekton. It seems to me like we are trying to use tenant pipelines to do pretty much exactly what some managed pipeline already does. If not, why are we using the same tasks? If so, what is the hole and let's scope it out so we can enable the workflow with a managed pipeline

[lcarva]

Just want to share a working example of a Tenant Pipeline: https://github.com/enterprise-contract/ec-cli/blob/main/release/cli.yaml

That Pipeline is built with Kustomize to pick up various fixes to the Tasks in the release-service-catalog repo: https://github.com/enterprise-contract/ec-cli/tree/main/release/src/cli

[lcarva]

It also isn't super clear to me why we even want to use the managed tasks in tenant pipelines.

From my perspective, there are a wide range of features the release service provides that I want to leverage. For example, I want to use apply-mapping to process the data from my ReleasePlan, I may want to leverage single component mode, etc. It is much easier for me to just use an existing Task than to have to write my own.

I definitely understand the maintenance concern. Maybe it's best to create Task variants if it's easier to maintain. As an outsider, adapting the existing Tasks seemed like the most reasonable approach right now, but I'm happy to change my PRs as needed.

I do think that tenant releases need a little more attention and I'm happy to help in any way I can.

It seems to me like we are trying to use tenant pipelines to do pretty much exactly what some managed pipeline already does.

To me the distinction is about access. As an upstream maintainer, I want to create an upstream release of my application to the upstream container image repo. I don't want to give these credentials to another team. I want to be in full control of that workflow. This premise is widely different than, say, making a release of the application for a vendor. With that in mind, I'd be perfectly happy to have an RPA to manage my tenant release. It's just one more CR I have to create once. But I do want to have full control over this within my tenant.

[davidmogar]

I agree on what it's said here. Having some tasks to be used directly in tenant/final pipelines could be beneficial. This, though, requires time and people to work on them, things we currently lack. In addition, this opens the door to a higher number of users requiring support, as now tenant pipelines would be affected by code we own. Finally, this could potentially allow users to replace managed pipelines with their own versions of those pipelines which would be confusing and potentially problematic as we are in touch with third party services and we make sure we use them in the correct way (secrets, endpoints, config, etc). This could be not the case for user hack-and-slash pipelines.

Tenant pipelines were added as a way to do some minor tasks before the managed pipeline. I'm sure thought that Johnny will design the future of the catalog around the different release pipelines (tenant, managed and final) once we get the time and people. This obviously will require proper discussions with architects to be sure this is the direction we want to follow. We wouldn't like this to become a workaround for a small subset of users but, instead, an improvement for the catalog.

Having said this, we cannot invest time on either implementing or reviewing changes around supporting tenant pipelines in the release service catalog right now.

[lcarva]

Tenant pipelines were added as a way to do some minor tasks before the managed pipeline.

Oh! This is different than what the docs talk about.

[ralphbean]

Related: konflux-ci/konflux-ci#147