Skip to content

Commit

Permalink
feat(dbt-cloud): poll the cached compile run if it is ongoing (#17061)
Browse files Browse the repository at this point in the history
## Summary & Motivation
In cases where the cached run is ongoing and a schedule has triggered a
run of a dbt Cloud job, allow the computation to proceed if the cached
run successfully terminates within 10 minutes.

## How I Tested These Changes
existing pytest
  • Loading branch information
rexledesma authored and dpeng817 committed Oct 11, 2023
1 parent 7367aed commit 765373e
Showing 1 changed file with 5 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import json
import shlex
from argparse import Namespace
from contextlib import suppress
from typing import (
Any,
Callable,
Expand Down Expand Up @@ -164,6 +165,10 @@ def get_compile_filters(parsed_args: Namespace) -> List[str]:
return dbt_compile_options

def _get_cached_compile_dbt_cloud_job_run(self, compile_run_id: int) -> Tuple[int, int]:
# If the compile run is ongoing, allow it a grace period of 10 minutes to finish.
with suppress(Exception):
self._dbt_cloud.poll_run(run_id=compile_run_id, poll_timeout=600)

compile_run = self._dbt_cloud.get_run(
run_id=compile_run_id, include_related=["trigger", "run_steps"]
)
Expand Down

0 comments on commit 765373e

Please sign in to comment.