Skip to content

Commit

Permalink
Docs sync 20241028 (#457)
Browse files Browse the repository at this point in the history
Co-authored-by: shibd <[email protected]>
  • Loading branch information
streamnativebot and shibd authored Oct 28, 2024
1 parent 3fce740 commit 99893c8
Show file tree
Hide file tree
Showing 33 changed files with 2,720 additions and 87 deletions.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
description: BigQuery Connector integrates Apache Pulsar with Google BigQuery.
author: StreamNative
contributors: shibd,danpi,codelipenghui,illegalnumbers
contributors: shibd,danpi,codelipenghui,nlu90
language: Java,Shell,Dockerfile
document:
source: Private source
Expand All @@ -25,7 +25,7 @@ id: "google-bigquery-sink"

The [Google Cloud BigQuery](https://cloud.google.com/bigquery) sink connector pulls data from Pulsar topics and persists data to Google Cloud BigQuery tables.

![](https://raw.githubusercontent.com/streamnative/pulsar-io-bigquery/v3.3.1.1/docs/google-bigquery-sink.png)
![](https://raw.githubusercontent.com/streamnative/pulsar-io-bigquery/v3.3.1.9/docs/google-bigquery-sink.png)

## Quick start

Expand Down Expand Up @@ -147,7 +147,7 @@ Before using the Google Cloud BigQuery sink connector, you need to configure it.
| `credentialJsonString` | String | Yes | true | "" (empty string) | The authentication JSON key. Set the environment variable `GOOGLE_APPLICATION_CREDENTIALS` to the path of the JSON file that contains your service account key when the `credentialJsonString` is set to an empty string. For details, see the [Google documentation](https://cloud.google.com/bigquery/docs/quickstarts/quickstart-client-libraries#before-you-begin). |
| `visibleModel` | String | No | false | "Committed" | The mode that controls when data written to the stream becomes visible in BigQuery for reading. For details, see the [Google documentation](https://cloud.google.com/bigquery/docs/write-api#application-created_streams). Available options are `Committed` and `Pending`. |
| `pendingMaxSize` | int | No | false | 10000 | The maximum number of messages waiting to be committed in `Pending` mode. |
| `batchMaxSize` | int | No | false | 20 | The maximum number of batch messages. |
| `batchMaxSize` | int | No | false | 20 | The maximum number of batch messages. The actual batch bytes size cannot exceed 10 MB. If it does, the batch will be flushed first. https://cloud.google.com/bigquery/quotas |
| `batchMaxTime` | long | No | false | 5000 | The maximum batch waiting time (in units of milliseconds). |
| `batchFlushIntervalTime` | long | No | false | 2000 | The batch flush interval (in units of milliseconds). |
| `failedMaxRetryNum` | int | No | false | 20 | The maximum retries when appending fails. By default, it sets 2 seconds for each retry. |
Expand Down
250 changes: 250 additions & 0 deletions connectors/google-bigquery-sink/v4.0.0.2/google-bigquery-sink.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
description: BigQuery Connector integrates Apache Pulsar with Google BigQuery.
author: StreamNative
contributors: shibd,danpi,codelipenghui,illegalnumbers
contributors: shibd,danpi,codelipenghui,nlu90
language: Java,Shell,Dockerfile
document:
source: Private source
Expand All @@ -25,7 +25,7 @@ id: "google-bigquery-source"

The [Google Cloud BigQuery](https://cloud.google.com/bigquery) Source Connector feeds data from Google Cloud BigQuery tables and writes data to Pulsar topics.

![](https://raw.githubusercontent.com/streamnative/pulsar-io-bigquery/v3.0.6.1/docs/google-bigquery-source.png)
![](https://raw.githubusercontent.com/streamnative/pulsar-io-bigquery/v3.0.7.1/docs/google-bigquery-source.png)

## Quick start

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
description: BigQuery Connector integrates Apache Pulsar with Google BigQuery.
author: StreamNative
contributors: shibd,danpi,codelipenghui,illegalnumbers
contributors: shibd,danpi,codelipenghui,nlu90
language: Java,Shell,Dockerfile
document:
source: Private source
Expand All @@ -25,7 +25,7 @@ id: "google-bigquery-source"

The [Google Cloud BigQuery](https://cloud.google.com/bigquery) Source Connector feeds data from Google Cloud BigQuery tables and writes data to Pulsar topics.

![](https://raw.githubusercontent.com/streamnative/pulsar-io-bigquery/v3.3.1.1/docs/google-bigquery-source.png)
![](https://raw.githubusercontent.com/streamnative/pulsar-io-bigquery/v3.3.1.9/docs/google-bigquery-source.png)

## Quick start

Expand Down
Loading

0 comments on commit 99893c8

Please sign in to comment.