Bump dbt-databricks from v1.8.5 to 1.9.1

[dependabot]

Bumps dbt-databricks from v1.8.5 to 1.9.1.

Release notes

Sourced from dbt-databricks's releases.

v1.9.1

What's Changed

• Remove library restrictions where possible (pandas, pydantic) by @​benc-db in databricks/dbt-databricks#874
• Extend DatabricksRelation types to help codegen by @​benc-db in databricks/dbt-databricks#875
• MERGE EXTENSION: when not matched by source - update by @​mi-volodin in databricks/dbt-databricks#866
• Fix failing concurrency test by @​benc-db in databricks/dbt-databricks#877
• More cleanup work for forward compatibility by @​benc-db in databricks/dbt-databricks#865

Full Changelog: databricks/dbt-databricks@v1.9.0...v1.9.1

v1.9.0

What's Changed

Features

• Add config for generating unique tmp table names for enabling parralel merge (thanks @​huangxingyi-git!) (854)
• Add support for serverless job clusters on python models (706)
• Add 'user_folder_for_python' behavior to switch writing python model notebooks to the user's folder (835)
• Merge capabilities are extended (739) to include the support for the following features (thanks @​mi-volodin):
  • with schema evolution clause (requires Databricks Runtime 15.2 or above);
  • when not matched by source clause, only for delete action
  • matched, not matched and not matched by source condition clauses;
  • custom aliases for source and target tables can be specified and used in condition clauses;
  • matched and not matched steps can now be skipped;
• Allow for the use of custom constraints, using the custom constraint type with an expression as the constraint (thanks @​roydobbe). (792)
• Add "use_info_schema_for_columns" behavior flag to turn on use of information_schema to get column info where possible. This may have more latency but will not truncate complex data types the way that 'describe' can. (808)
• Add support for table_format: iceberg. This uses UniForm under the hood to provide iceberg compatibility for tables or incrementals. (815)
• Add include_full_name_in_path config boolean for external locations. This writes tables to {location_root}/{catalog}/{schema}/{table} (823)
• Add a new workflow_job submission method for python, which creates a long-lived Databricks Workflow instead of a one-time run (thanks @​kdazzle!) (762)
• Allow for additional options to be passed to the Databricks Job API when using other python submission methods. For example, enable email_notifications (thanks @​kdazzle!) (762)
• Support microbatch incremental strategy using replace_where (825)

Fixes

• Replace array indexing with 'get' in split_part so as not to raise exception when indexing beyond bounds (839)
• Ensure columns that are added get backticked (859)

Under the Hood

• Significant refactoring and increased testing of python_submissions (830)
• Fix places where we were not properly closing cursors, and other test warnings (713)
• Drop support for Python 3.8 (713)
• Upgrade databricks-sql-connector dependency to 3.5.0 (833)
• Prepare for python typing deprecations (837)
• Fix behavior flag use in init of DatabricksAdapter (thanks @​VersusFacit!) (836)
• Restrict pydantic to V1 per dbt Labs' request (843)
• Switching to Ruff for formatting and linting (847)
• Refactoring location of DLT polling code (849)
• Switching to Hatch and pyproject.toml for project config (853)

New Contributors

... (truncated)

Changelog

Sourced from dbt-databricks's changelog.

dbt-databricks 1.9.1 (December 16, 2024)

Features

• Merge strategy now supports the update set ... action with the explicit list of updates for when not matched by source (866) (thanks @​mi-volodin).

Under the Hood

• Removed pins for pandas and pydantic to ease user burdens (874)
• Add more relation types to make codegen happy (875)
• add UP ruleset (865)

dbt-databricks 1.9.0 (December 9, 2024)

Features

• Add config for generating unique tmp table names for enabling parralel merge (thanks @​huangxingyi-git!) (854)
• Add support for serverless job clusters on python models (706)
• Add 'user_folder_for_python' behavior to switch writing python model notebooks to the user's folder (835)
• Merge capabilities are extended (739) to include the support for the following features (thanks @​mi-volodin):
  • with schema evolution clause (requires Databricks Runtime 15.2 or above);
  • when not matched by source clause, only for delete action
  • matched, not matched and not matched by source condition clauses;
  • custom aliases for source and target tables can be specified and used in condition clauses;
  • matched and not matched steps can now be skipped;
• Allow for the use of custom constraints, using the custom constraint type with an expression as the constraint (thanks @​roydobbe). (792)
• Add "use_info_schema_for_columns" behavior flag to turn on use of information_schema to get column info where possible. This may have more latency but will not truncate complex data types the way that 'describe' can. (808)
• Add support for table_format: iceberg. This uses UniForm under the hood to provide iceberg compatibility for tables or incrementals. (815)
• Add include_full_name_in_path config boolean for external locations. This writes tables to {location_root}/{catalog}/{schema}/{table} (823)
• Add a new workflow_job submission method for python, which creates a long-lived Databricks Workflow instead of a one-time run (thanks @​kdazzle!) (762)
• Allow for additional options to be passed to the Databricks Job API when using other python submission methods. For example, enable email_notifications (thanks @​kdazzle!) (762)
• Support microbatch incremental strategy using replace_where (825)

Fixes

• Replace array indexing with 'get' in split_part so as not to raise exception when indexing beyond bounds (839)
• Set queue enabled for Python notebook jobs (856)
• Ensure columns that are added get backticked (859)

Under the Hood

• Significant refactoring and increased testing of python_submissions (830)
• Fix places where we were not properly closing cursors, and other test warnings (713)
• Drop support for Python 3.8 (713)
• Upgrade databricks-sql-connector dependency to 3.5.0 (833)
• Prepare for python typing deprecations (837)
• Fix behavior flag use in init of DatabricksAdapter (thanks @​VersusFacit!) (836)
• Restrict pydantic to V1 per dbt Labs' request (843)
• Switching to Ruff for formatting and linting (847)
• Refactoring location of DLT polling code (849)

... (truncated)

Commits

• a59f08d Prep for 1.9.1 release (#879)
• 5e20eea More cleanup work for forward compatibility (#865)
• fe9fa10 Fix failing concurrency test (#877)
• 3fc74df MERGE EXTENSION: when not matched by source - update (#866)
• 40925aa dbt codegen only manually handles limited sets of Databricks types (#875)
• 32ec1b1 Remove library restrictions if possible (#874)
• ede83b0 Prep for 1.9.0 release (#867)
• 69aa772 Extract poll refresh pipeline from cursor (#849)
• 5066e76 Ensure column name is backticked during alter (#861)
• 6398033 Fix Python regressions in 1.9.0beta (#857)
• Additional commits viewable in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)