Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to push multiple flows to production using gitPush? #99

Open
gvanderweerd opened this issue Oct 11, 2024 · 4 comments
Open

How to push multiple flows to production using gitPush? #99

gvanderweerd opened this issue Oct 11, 2024 · 4 comments
Assignees
Labels
area/plugin Plugin-related issue or feature request

Comments

@gvanderweerd
Copy link

Issue description

As discussed on slack I have opened an issue. See slack chat below:
https://kestra-io.slack.com/archives/C03FQKXRK3K/p1728459250919099

@anna-geller
Copy link
Member

Could you summarize the issue here? Slack links expire quickly

@Ben8t Ben8t transferred this issue from kestra-io/kestra Oct 11, 2024
@Ben8t Ben8t added the area/plugin Plugin-related issue or feature request label Oct 11, 2024
@gvanderweerd
Copy link
Author

Thanks for your response! here is the info from the slack chat:

Giel:
At my company we have set up a dev and a prod environment of kestra. We are now looking to design a convenient system to push flows from production using git pushflows:
https://kestra.io/docs/best-practices/from-dev-to-prod
https://kestra.io/plugins/plugin-git/tasks/io.kestra.plugin.git.pushflows
Now we want to have control over which namespace files and which flows we push to production such that we do not have to alter the code of the flow that pushes to production everytime we want to. We tried to use inputs (ARRAY, MULTISELECT, and YAML) but it is not working since the ArrayList data type is not accepted by the io.kestra.plugin.git.PushFlows.
If we cast an ArrayList into string like: flows: “{{ inputs.flows | yaml }}”, the flow will execute successfully but no changes will be found.
Does anyone have experience with setting up such a system in kestra?

Benoit:
Hello Giel !
Did you try by settings a list of glob patterns (flows: "{{input.flows}}" with input.flows being an array) ?
I will try something in my end too

Giel:
We tried and found 2 working solutions:
flows: "*" # optional list of glob patterns; by default, all flows are pushed #working but dont need all
flows: '{{ inputs.string }}' #working but restricts us to 1 flow at the time
We also tried a lot of other things among which:
flows: "{{ inputs.flows }}"
flows: "{{ inputs.flows | join(',') }}"
flows: "{{ inputs.flows | join('\n') }}"
flows: "{{ inputs.dropdown_multi }}"
flows: "{{ inputs.yaml | yaml }}"
If there is no easy way to manually input a list of flows that we want we will just stick to pushing 1 flow at the time but just wanted to verify if we are overlooking something here or are using it wrong.

it accept string and the string must contain only one flow. If its multiple flows in that string it wont detect the changes

Hi Benoit were you able to replicate something that could solve my problem? Or are we using it in the wrong way? If the conclusion is that we can only do 1 flow or all flows than that is also a valid conclusion. I just want to prevent that we keep developing endlessly to try and solve a problem that cannot be solved currently.

@EverybodyGetBackToWork
Copy link

+1 for this, have tried to achieve using an Array (list of flows) such as ["test", "test1"] which works if explicitly stated in the flow but the syntax is rejected when this is passed as an input.

@anna-geller
Copy link
Member

anna-geller commented Oct 21, 2024

@Ben8t can you ask Matt to add dynamic properties for Git tasks and once that's done try to reproduce it on develop? I feel this looks like an issue that flows is not dynamic property atm, that's why it cannot be passed from Inputs this way

@Ben8t Ben8t self-assigned this Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/plugin Plugin-related issue or feature request
Projects
Status: Backlog
Development

No branches or pull requests

4 participants