Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: pre-pull images and sideload them into the cluster #143

Merged
merged 8 commits into from
Nov 5, 2024

Conversation

abuchanan-airbyte
Copy link
Collaborator

@abuchanan-airbyte abuchanan-airbyte commented Oct 31, 2024

Pull docker images (in parallel) and sideload them into the Kind cluster, before installing the helm chart.

In my testing, when starting from a clean machine (after running docker system prune --all), the first install takes ~10 minutes, and the next install takes ~2 minutes

@abuchanan-airbyte abuchanan-airbyte requested a review from a team as a code owner October 31, 2024 20:12
internal/cmd/local/k8s/cluster.go Outdated Show resolved Hide resolved
internal/cmd/local/k8s/cluster.go Outdated Show resolved Hide resolved
internal/cmd/local/k8s/cluster.go Outdated Show resolved Hide resolved
@abuchanan-airbyte
Copy link
Collaborator Author

@colesnodgrass the latest commit adds some code to avoid reloading images that already exist on the node. Could you take a fresh look please?

internal/cmd/local/k8s/load_images.go Outdated Show resolved Hide resolved
internal/cmd/local/k8s/load_images.go Outdated Show resolved Hide resolved
internal/cmd/local/k8s/load_images.go Outdated Show resolved Hide resolved
@abuchanan-airbyte abuchanan-airbyte merged commit 985bb74 into main Nov 5, 2024
2 checks passed
@abuchanan-airbyte abuchanan-airbyte deleted the abuch/side-load branch November 5, 2024 20:14
Copy link

sentry-io bot commented Nov 6, 2024

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

  • ‼️ *fmt.wrapError: unable to install airbyte chart: unable to install helm: create: failed to create: etcdserver: re... github.com/airbytehq/abctl/internal/trace in Sp... View Issue
  • ‼️ *fmt.wrapError: unable to migrate data from previous airbyte installation: unable to update postgres user: unable... github.com/airbytehq/abctl/internal/trace in Sp... View Issue
  • ‼️ *errors.fundamental: failed to fetch https://airbytehq.github.io/helm-charts/airbyte-1.2.0.tgz : 403 Forbidden github.com/airbytehq/abctl/internal/cmd/images ... View Issue

Did you find this useful? React with a 👍 or 👎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants