Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add ability to delay runner registration #2139

Merged
merged 1 commit into from
Jul 23, 2024

Conversation

stuartwdouglas
Copy link
Collaborator

We think there may be some issues with runners registering themselves before istio is fully ready, resulting in RBAC errors.

This workaround should allow for a delay that will prevent this.

Possible fix for #2059

@stuartwdouglas stuartwdouglas requested review from a team and matt2e and removed request for a team July 23, 2024 22:43
@ftl-robot ftl-robot mentioned this pull request Jul 23, 2024
backend/runner/runner.go Outdated Show resolved Hide resolved
We think there may be some issues with runners registering themselves before istio is fully ready, resulting in RBAC errors.

This workaround should allow for a delay that will prevent this.

Possible fix for #2059
@stuartwdouglas stuartwdouglas force-pushed the stuartwdouglas/runner-start-delay branch from 16c3b0b to c046f03 Compare July 23, 2024 22:51
@stuartwdouglas stuartwdouglas enabled auto-merge (squash) July 23, 2024 22:51
Copy link
Collaborator

@matt2e matt2e left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the best place to add this 👍

@stuartwdouglas stuartwdouglas merged commit 48ea4a6 into main Jul 23, 2024
59 checks passed
@stuartwdouglas stuartwdouglas deleted the stuartwdouglas/runner-start-delay branch July 23, 2024 22:56
go func() {
// In some environments we may want a delay before registering the runner
// We have seen istio race conditions that we think this will help
startDelay := os.Getenv("FTL_RUNNER_START_DELAY")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a Kong flag on the Config struct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants