From 50647a520a09ba51e763708f645fed7893949e99 Mon Sep 17 00:00:00 2001 From: Brandon Martin Date: Thu, 14 Sep 2023 19:31:52 -0600 Subject: [PATCH 1/3] Update README --- README.md | 105 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ readme.md | 9 ----- 2 files changed, 105 insertions(+), 9 deletions(-) create mode 100644 README.md delete mode 100644 readme.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..53243cb --- /dev/null +++ b/README.md @@ -0,0 +1,105 @@ +# Clickhouse Connector + +The Clickhouse Native Data Connector allows for connecting to a Clickhouse instance giving you an instant GraphQL API on top of your Clickhouse data. +This uses the [Rust Data Connector SDK](https://github.com/hasura/ndc-hub#rusk-sdk) from the [Data connector Hub](https://github.com/hasura/ndc-hub) and implements the [Data Connector Spec](https://github.com/hasura/ndc-spec). + +* [Clickhouse Connector information in the Hasura Connectors directory](https://hasura.io/connectors/clickhouse) +* TODO: Docs Link + +In order to use this connector you will need to: + +* Create a [Clickhouse account](https://clickhouse.cloud/signUp?loc=nav-get-started) +* Log in to A Hasura CLI Session +* Create a Pre-Shared Token for service authentication between the Hasura V3 Engine and your connector + +## Features + +TODO + +## For Hasura Users + +If you wish to use this connector with your Hasura projects, the best instructions can be found on the Hasura Hub (TODO: Link to hub page for Clickhouse Connector). + +The following steps will allow you to deploy the connector and use it in a Hasura V3 project: + +* Create a Hasura V3 Project (or use an existing project) +* Ensure that you have a metadata definition +* Create a configuration for the Clickhouse Connector referencing your credentials: + `clickhouse.connector.configuration.json` + You have 2 options for the config: + 1. The easiest option is to is to run the connector locally in config mode: + * ``` + cargo run -- configuration serve --port 5000 + ``` + * ``` + curl -X POST -d '{"connection": {"username": "your_username", "password": "your_password", "url": "your_clickhouse_url"}, "tables": []}' http://localhost:5000 > clickhouse.connector.configuration.json + ``` + 2. The other option is to manually write your config that follows this pattern: + ``` + { + "connection": { + "username": "your_username", + "password": "your_password", + "url": "your_clickhouse_url" + }, + "tables": [ + { + "name": "TableName", + "schema": "SchemaName", + "alias": "TableAlias", + "primary_key": { "name": "TableId", "columns": ["TableId"] }, + "columns": [ + { "name": "TableId", "alias": "TableId", "data_type": "Int32" }, + ] + } + ] + } + ``` +* Run the following command to deploy the connector +* Ensure you are logged in to Hasura CLI + ``` + hasura3 cloud login --pat 'YOUR-HASURA-TOKEN' + ``` +* Deploy the connector + ``` + hasura3 connector create sendgrid:v1 \ + --github-repo-url https://github.com/hasura/clickhouse_ndc/tree/main \ + --config-file ./clickhouse.connector.configuration.json + ``` +* Ensure that your deployed connector is referenced from your metadata with the service token +* Edit your metadata using the LSP support to import the defined schema, functions, procedures +* Deploy or update your Hasura cloud project + ``` + hasura3 cloud build create --project-id my-project-id \ + --metadata-file metadata.json my-build-id + ``` +* View in your cloud console, access via the graphql API + + +## For Developers + +The following instructions are for developers who wish to contribute to the Clickhouse Connector. + +### Build + +Prerequisites: + +1. Install [rustup](https://www.rust-lang.org/tools/install). + +Commands: + +``` +cargo build +cargo run -- configuration serve --port 5000 +curl -X POST -d '{"connection": {"username": "your_username", "password": "your_password", "url": "your_clickhouse_url"}, "tables": []}' http://localhost:5000 > clickhouse.connector.configuration.json +cargo run -- serve --configuration clickhouse.connector.congifuration.json +``` + +### Docker + +The `Dockerfile` is used by the `connector create` command and can be tested as follows: + +``` +docker build . --tag clickhouse_ndc +docker run -it --v ./clickhouse.connector.configuration.json:/config.json clickhouse_ndc +``` \ No newline at end of file diff --git a/readme.md b/readme.md deleted file mode 100644 index b270788..0000000 --- a/readme.md +++ /dev/null @@ -1,9 +0,0 @@ -# Clickhouse NDC - -An early stage ClickHouse Native Data Connector for Hasura V3 - -This uses the ndc_sdk from the [ndc-hub](https://github.com/hasura/ndc-hub) and implements the [ndc-spec](https://github.com/hasura/ndc-spec). - -We provide a Dockerfile to build this connector, but as we cannot currently use multistage builds, the end image is quite large because it includes the entire rust toolchain... - -See the included docker-compose file for an example of running the connector with a configuration file From bf9af3da54e5a69f9ddb4ccfa82efb9f0f318b92 Mon Sep 17 00:00:00 2001 From: Brandon Martin Date: Thu, 14 Sep 2023 19:33:51 -0600 Subject: [PATCH 2/3] Update spacing --- README.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 53243cb..e6b3bbc 100644 --- a/README.md +++ b/README.md @@ -28,12 +28,10 @@ The following steps will allow you to deploy the connector and use it in a Hasur `clickhouse.connector.configuration.json` You have 2 options for the config: 1. The easiest option is to is to run the connector locally in config mode: - * ``` - cargo run -- configuration serve --port 5000 - ``` - * ``` - curl -X POST -d '{"connection": {"username": "your_username", "password": "your_password", "url": "your_clickhouse_url"}, "tables": []}' http://localhost:5000 > clickhouse.connector.configuration.json - ``` + ``` + cargo run -- configuration serve --port 5000 + curl -X POST -d '{"connection": {"username": "your_username", "password": "your_password", "url": "your_clickhouse_url"}, "tables": []}' http://localhost:5000 > clickhouse.connector.configuration.json + ``` 2. The other option is to manually write your config that follows this pattern: ``` { From 5be3322df9162b9d60bd70c6ad695c4b11e5aeff Mon Sep 17 00:00:00 2001 From: Brandon Martin Date: Thu, 14 Sep 2023 19:58:12 -0600 Subject: [PATCH 3/3] Fix leftover from copy of sendgrid --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e6b3bbc..815ac0f 100644 --- a/README.md +++ b/README.md @@ -60,7 +60,7 @@ The following steps will allow you to deploy the connector and use it in a Hasur ``` * Deploy the connector ``` - hasura3 connector create sendgrid:v1 \ + hasura3 connector create clickhouse:v1 \ --github-repo-url https://github.com/hasura/clickhouse_ndc/tree/main \ --config-file ./clickhouse.connector.configuration.json ```