Request for Beta Testers: Destination Databricks

[katmarkham]

Early Access Available!

Hi Airbyte community,

Early access for Destinations Databricks is now available, bringing you:

• Per-record error handling
• Typing and deduping
• Refreshes
• Resumable Full Fresh
• All the great benefits of Destinations Version 2.0!

How to Enroll

We are actively seeking and iterating on feedback to refine these features prior to general availability.

Please note:

• You must be using Unity Catalog to use this connector.
• You must upgrade your Airbyte Platform version to at least 0.63.7 before upgrading to this connector version, or be using Airbyte Cloud.

Here's how you can try out the beta:

Cloud: Instructions

Please login to your workspace and create a new Databricks destination. It will automatically be created using the newest version.

• We recommend creating a new destination and connection for this test. If you wish to switch an existing destination configuration to the public preview version, you can go to one of its connections, click the destination, and then click the "upgrade" button. You will need to manually drop any existing tables and clear your connection after doing this.

Open-Sources: Instructions

In your Airbyte settings -> Destinations, select version 3.1.0 of the Databricks connector, then create a new Databricks destination.

• We recommend creating a new destination and connection for this test If you wish to switch an existing destination configuration to the public preview version, you can go to one of its connections, click the destination, and then click the "upgrade" button. You will need to manually drop any existing tables and clear your connection after doing this.

Request for Feedback

As you explore the new functionality, we ask that you pay special attention to table and column name changes, missing columns, incorrect datatypes or incorrect data, and the speed and cost of your syncs.

We look forward to hearing about your experience with Destinations Databricks. Please don't hesitate to share any feedback or questions in the comments below. You can also join us on Slack if you would like to chat directly with the team about this feature.

Please note, all Destination Databricks Cloud users must upgrade beyond 3.0.0 by January 31st, 2025 to experience uninterrupted syncs.