feat: add support for sqlite connectors #680
Conversation
WalkthroughThe changes introduce support for SQLite databases in the Changes
TipsChat with CodeRabbit Bot (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (5)
- docs/connectors.md (1 hunks)
- examples/from_sql.py (2 hunks)
- pandasai/connectors/init.py (2 hunks)
- pandasai/connectors/base.py (1 hunks)
- pandasai/connectors/sql.py (2 hunks)
Files skipped from review due to trivial changes (3)
- docs/connectors.md
- pandasai/connectors/init.py
- pandasai/connectors/base.py
Additional comments: 5
examples/from_sql.py (3)
5-5: The import of
SqliteConnectoris added correctly. Ensure that theSqliteConnectorclass is correctly implemented and tested.41-51: The
SqliteConnectoris instantiated correctly. However, the "database" attribute in the config dictionary should be an absolute path to the SQLite database file. Please verify that "local_path_to_db" is a placeholder and will be replaced with the actual path in the production code.53-53: The
invoice_connectoris correctly added to theSmartDatalakeconstructor. Ensure that theSmartDatalakeclass can handle multiple connectors of different types.pandasai/connectors/sql.py (2)
8-8: The
SqliteConnectorConfigimport is added to support the newSqliteConnectorclass. Ensure that theSqliteConnectorConfigclass is defined in thebasemodule.367-450: The
SqliteConnectorclass is introduced to handle connections to SQLite databases. It extends theSQLConnectorclass and overrides the__init__,_load_connector_config,_init_connection,__del__,head, and__repr__methods. The__init__method now includes SQLite-specific environment variables. The_init_connectionmethod creates an SQLite engine and connection. Theheadmethod builds a query to fetch the first 5 rows from the SQLite database. The__repr__method returns a string representation of the SQLite connector.
There was a problem hiding this comment.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (1)
- examples/from_sql.py (2 hunks)
Additional comments: 3
examples/from_sql.py (3)
5-5: The import of
SqliteConnectoris added correctly. Ensure that theSqliteConnectorclass is implemented and exported correctly in thepandasai.connectorsmodule.41-51: The
SqliteConnectoris being used to connect to a SQLite database. Thedatabaseparameter should be a path to the SQLite database file. Please ensure that"local_path_to_db"is replaced with the actual path to the SQLite database file.53-56: The
invoice_connectoris added to the list of connectors in theSmartDatalakeconstructor. Ensure that theSmartDatalakeclass can handle multiple connectors including the newSqliteConnector.
There was a problem hiding this comment.
Review Status
Actionable comments generated: 1
Configuration used: CodeRabbit UI
Files selected for processing (3)
- pandasai/connectors/base.py (1 hunks)
- pandasai/connectors/sql.py (2 hunks)
- tests/connectors/test_sqlite.py (1 hunks)
Files skipped from review due to trivial changes (1)
- tests/connectors/test_sqlite.py
Additional comments: 5
pandasai/connectors/base.py (1)
- 42-49: The
SqliteConnectorConfigclass is introduced to handle SQLite-specific configuration parameters. It inherits fromSQLBaseConnectorConfigand adds two new attributes:tableanddatabase. Ensure that these attributes are correctly used in theSqliteConnectorclass and that they are properly documented for users. Also, consider adding default values or validation checks for these attributes if necessary.pandasai/connectors/sql.py (4)
8-8: The import statement is updated to include
SqliteConnectorConfigwhich is a new class introduced for handling SQLite-specific configuration parameters. Ensure that this class is defined and implemented correctly in thebasemodule.367-450: A new
SqliteConnectorclass is introduced, extending theSQLConnectorclass. This class includes methods to initialize the SQLite database connection and load SQLite-specific configurations. The__init__method is updated to set thedialecttosqliteand populate the configuration from environment variables if a dictionary is passed. The_load_connector_configmethod is overridden to return an instance ofSqliteConnectorConfig. The_init_connectionmethod is overridden to create a SQLite database connection. Theheadmethod is overridden to run a SQL query to get all the column names and 5 random rows from the SQLite database. The__repr__method is overridden to return the string representation of the SQLite connector. Ensure that these changes are compatible with the rest of the codebase and that the SQLite database connection is correctly initialized and used.406-407: The SQLite database connection is initialized using the
create_enginefunction fromsqlalchemywith thesqlitedialect and the database path from the configuration. Ensure that the database path is correctly formed and that the SQLite database exists at that path. Also, ensure that thecreate_enginefunction is correctly used with thesqlitedialect and that the connection is correctly initialized.432-435: A SQL query is built to get all the column names and 5 random rows from the SQLite database, and the
pd.read_sqlfunction is used to execute the query and return the result. Ensure that the_build_querymethod correctly builds the SQL query and that thepd.read_sqlfunction correctly executes the query and returns the result. Also, ensure that the SQLite database supports theRANDOM()function for random row selection.
| config["dialect"] = "sqlite" | ||
| if isinstance(config, dict): | ||
| sqlite_env_vars = {"database": "SQLITE_DB_PATH", "table": "TABLENAME"} | ||
| config = self._populate_config_from_env(config, sqlite_env_vars) |
There was a problem hiding this comment.
The dialect is hardcoded to sqlite and the configuration is populated from environment variables if a dictionary is passed. This could lead to unexpected behavior if the user passes a configuration dictionary with a different dialect or without the necessary environment variables. Consider adding error handling or validation to ensure that the configuration is correctly formed and that the necessary environment variables are set.
gventuri
changed the title
There was a problem hiding this comment.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (1)
- examples/from_sql.py (2 hunks)
Additional comments: 3
examples/from_sql.py (3)
5-5: The import of
SqliteConnectoris added correctly. Ensure that theSqliteConnectorclass is implemented and available in thepandasai.connectorsmodule.41-49: The
SqliteConnectoris instantiated with a configuration dictionary. Thedatabasekey should contain the path to the SQLite database file. Please verify that the value"local_path_to_db"is a placeholder and will be replaced with the actual path to the SQLite database file in the production code.51-53: The
invoice_connectoris added to the list of connectors passed to theSmartDatalakeconstructor. This will allow theSmartDatalaketo retrieve data from the SQLite database. Ensure that theSmartDatalakeclass can handle multiple connectors of different types.
Added support for sqlite file database. Implemented
SqliteConnectorandSqliteConnectorConfigwherefile path can be parsed as
databasekey toconfigparameter.Summary by CodeRabbit
SqliteConnector.SqliteConnector.SqliteConnectorto ensure its functionality and reliability.This update enhances the software's versatility by supporting more types of databases, making it more useful for users who work with SQLite databases. It also provides clear instructions and examples for using the new feature.