Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PLUGIN-1438: Added support for destination table write preference in BQ Execute Plugin #1473

Merged
merged 1 commit into from
Dec 12, 2024

Conversation

anshumanks
Copy link
Contributor

@anshumanks anshumanks commented Dec 11, 2024

Added support for destination table write preference in BQ Execute Plugin

Jira : PLUGIN-1438

Description

This update introduces the ability for users to select a destination table write preference when storing results in BigQuery. Users can now choose from the following options:

  • Write If Empty: Write to the table only if it is empty. If the table contains data, the operation will fail.
  • Append To Table: Append results to an existing table, preserving its schema.
  • Overwrite Table: Overwrite the existing table, replacing both the data and schema.

UI Field

  • Modified BigQueryExecute-action.json

Docs

  • Modified BigQueryExecute-action.md

Code change

  • Modified BigQueryExecute.java

Unit Tests

  • Modified BigQueryExecuteTest.java

Tested

image

@anshumanks anshumanks changed the title Added support for destination table write preference in BQ Execute Plugin PLUGIN-1438: Added support for destination table write preference in BQ Execute Plugin Dec 11, 2024
Copy link
Member

@itsankit-google itsankit-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add evidence/screenshots in PR description under Tested section.

A good PR description example: #1444 (comment)

Comment on lines 406 to 409
@Description("Specifies the write preference for the destination table. Options are: " +
"Write if Empty: Only write if the table is empty. " +
"Append to Table: Add results to existing data. " +
"Overwrite Table: Replace all existing data.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Description/Tooltip should be crisp: Specifies whether a job should overwrite or append the existing destination table if it already exists.

Please add these details in markdown: https://github.com/data-integrations/google-cloud/blob/develop/docs/BigQueryExecute-action.md

@itsankit-google itsankit-google added the build Trigger unit test build label Dec 11, 2024
Copy link
Member

@itsankit-google itsankit-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please squash commits before merge.

@anshumanks anshumanks force-pushed the bigQuery-Execute-Plugin branch 2 times, most recently from 40b5f74 to d7b5253 Compare December 12, 2024 12:25
…ugin

Addressed PR comments: updated description, added documentation for the added feature

Formatted to fix failing tests

Refactor: Replace hardcoded write preference strings with JobInfo.WriteDisposition constants

Refactor: Simplify error message for invalid write preference validation

Refactor: Centralize valid write preferences in BigQueryExecute.Config

Refactor: Dynamically generate valid write preferences from JobInfo.WriteDisposition
@anshumanks anshumanks force-pushed the bigQuery-Execute-Plugin branch from d7b5253 to a467a7a Compare December 12, 2024 12:50
@itsankit-google itsankit-google merged commit b986f30 into develop Dec 12, 2024
15 of 16 checks passed
@itsankit-google itsankit-google deleted the bigQuery-Execute-Plugin branch December 12, 2024 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Trigger unit test build
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants