Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry for big query execute #28

Merged
merged 1 commit into from
Dec 21, 2023
Merged

Conversation

psainics
Copy link
Collaborator

@psainics psainics commented Oct 31, 2023

Implement retry feature for BigQuery Execute Plugin

Jira : PLUGIN-1715

  • In general internal errors should be retried.
  • Do an exponential back-off retry on errors with reason jobBackendError , jobInternalError.
  • Backoff requirements (exponential from 1 to 32 seconds with a multiplier of 2)

Code Changes

  • Add a new set that contains all reason we should retry on.
  • A new custom exception that is raised when the error should be retried.
  • A new function to generate the exponential back- off retry policy
  • A new dependency is added dev.failsafe that handles the retry logic based on the exception raised.

Unit Tests

  • Use mocks to test the behavior of the retry.

@psainics psainics force-pushed the patch-retry-bigQuery-execute branch 2 times, most recently from c62793c to 9dd2870 Compare November 6, 2023 07:19
@psainics psainics self-assigned this Nov 8, 2023
.handle(BigQueryJobExecutionException.class)
.withBackoff(initialRetryDuration, maxRetryDuration, multiplier)
.withMaxRetries(maxRetryCount)
.onRetry(event -> LOG.debug("Retrying BigQuery Execute job. Retry count: {}", event.getAttemptCount()))
Copy link
Collaborator

@vikasrathee-cs vikasrathee-cs Nov 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of BQ execute job, mention BQ job so it can be reused in source plugin also later on and if there is any common util between BQ plugins, move retrypolicy method to that class. E.g. BigQueryUtil

@psainics psainics force-pushed the patch-retry-bigQuery-execute branch 3 times, most recently from 57ae3ae to b74a689 Compare November 16, 2023 08:34
@psainics psainics force-pushed the patch-retry-bigQuery-execute branch 11 times, most recently from 34a7113 to ddadce4 Compare December 11, 2023 20:01
@psainics psainics force-pushed the patch-retry-bigQuery-execute branch 2 times, most recently from 7a56dad to 2793ac7 Compare December 14, 2023 05:12
@vikasrathee-cs vikasrathee-cs force-pushed the patch-retry-bigQuery-execute branch from 2793ac7 to a5e3e7b Compare December 21, 2023 08:24
@psainics psainics merged commit 6ab069c into develop Dec 21, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants