Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Timplus backends #1

Open
1 task done
Jasmine-ge opened this issue Sep 12, 2024 · 1 comment
Open
1 task done

feat: Add Timplus backends #1

Jasmine-ge opened this issue Sep 12, 2024 · 1 comment

Comments

@Jasmine-ge
Copy link
Collaborator

Jasmine-ge commented Sep 12, 2024

Is your feature request related to a problem?

No response

What is the motivation behind your request?

We would to add Timeplus backend support to ibis as we want to make streaming more convenient.

Describe the solution you'd like

We'll add some of the basic features first, then working on the details. ALL possible APIs are listed as follows, and we highlight those we plan to finish in the first PR.

ALL APIs:

  • do_connect():Create a Timeplus Backend for use with Ibis.
  • compile():Compile an ibis expression to timeplus
  • connect():Connect to the database
  • create_database():Create new database
  • create_schema()
  • create_sink():Creating a sink.
  • create_source():Creating a source.
  • create_table():Create a new stream in Timeplus.
  • create_view():Create a new view from a dataframe or stream.
  • disconnect():Close the connection to the backend.
  • drop_database():Drop a database with name
  • drop_sink():Drop a Sink.
  • drop_source():Drop a Source.
  • drop_table():Drop a stream.
  • drop_view():Drop a view.
  • execute():Execute an expression.
  • from_connection():Create a Timeplus Backend from an existing stream environment.
  • from_dataframe():Construct an ibis table from a pandas DataFrame.
  • get_schema():Return a Schema object for the indicated stream and database.
  • has_operation():Return whether the backend implements support for operation.
  • insert():Insert data into a table.
  • list_databases():List existing databases in the current connection.
  • list_schemas()
  • list_tables():Return the list of stream/view names.
  • list_views():Return the list of view names.
  • raw_sql():Execute sql
  • read_csv():Register a csv file as a table in the current database.
  • read_delta():Register a Delta Lake table in the current database.
  • read_geo():Register a GEO file as a table in the current database.
  • read_in_memory():Register an in-memory table object in the current database. Supported objects include pandas DataFrame, a Polars DataFrame/LazyFrame, or a PyArrow Table or RecordBatchReader.
  • read_json():Register a json file as a table in the current database.
  • read_parquet():Register a parquet file as a table in the current database.
  • read_(name of other database)():Register a table from Another database instance into a Timeplus table.
  • reconnect():Reconnect to the database already configured with connect.
  • register_options():Register custom backend options.
  • rename_table():Rename an existing table.
  • set_database()
  • sql()
  • table():Return a table expression from a table or view in the database.
  • to_csv():Write the results of executing the given expression to a CSV file.This method is eager and will execute the associated expression immediately.
  • to_delta():Write the results of executing the given expression to a Delta Lake table. This method is eager and will execute the associated expression immediately.
  • to_pandas():Execute an Ibis expression and return a pandas DataFrame, Series, or scalar.
  • to_pandas_batches():Execute an Ibis expression and return an iterator of pandas DataFrames.
  • to_parquet():Write the results of executing the given expression to a parquet file. This method is eager and will execute the associated expression immediately.
  • to_parquet_dir():Write the results of executing the given expression to a parquet file in a directory. This method is eager and will execute the associated expression immediately.
  • to_polars():Execute expression and return results in as a polars DataFrame. This method is eager and will execute the associated expression immediately.
  • to_pyarrow():Execute expression and return results in as a pyarrow table. This method is eager and will execute the associated expression immediately.
  • to_pyarrow_batches():Execute expression and return an iterator of pyarrow record batches. This method is eager and will execute the associated expression immediately.
  • to_torch():Execute an expression and return results as a dictionary of torch tensors.
  • truncate_table():Delete all rows from a table.

We don’t support catalogs thus we don’t need to implement APIs related to catalogs.

What version of ibis are you running?

9.3.0

What backend(s) are you using, if any?

Timeplus

Code of Conduct

  • I agree to follow this project's Code of Conduct
@Jasmine-ge
Copy link
Collaborator Author

@gangtao @jovezhong Could you please help check the APIs that need to be implemented?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant