Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* refactor: clean code in smart_dataframe and smart_datalake (Sinaptik-AI#814) * refactor: extract import from file method * refactor: extract df head methods * refactor: move connector config in the relative connector file * refactor: csv and pandas files are now treated as a connector * chore: remove verbose getters and setters * refactor: remove load and save feature * refactor: create dataframe proxy * chore: simplify agent * chore: simplify datalake * refactor: simplify smart datalake * refactor: centralize context in lakes * refactor: move lake callbacks to dedicate class * fix: load connector before generating cache hex * fix: only allow direct sql to SQLConnectors * fix: check sql connector was not working * fix(connector): update connector validation at the start * fix(direct_sql): fix some leftovers * fix: merged change revert built-in shadowing --------- Co-authored-by: ArslanSaleem <[email protected]> * refactor(query_tracker): clean code and create error handling pipeling (Sinaptik-AI#875) * refactor(smart_datalake): clean chat method of smart datalake * refactor(smartlake_pipeline): minor cleanups and comments * refactor(query_tracker): refactor query tracker input and output * refactor(query_tracker): adding error pipeline * refactor(query_tracker): some rename and delete extras * refactor(query_tracker): remove comments * Merge Main Changes to v1.6 (Sinaptik-AI#878) * fix(output_type): handle errors for wrong output type (Sinaptik-AI#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (Sinaptik-AI#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (Sinaptik-AI#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (Sinaptik-AI#876) * fix: rephrase query (Sinaptik-AI#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> * refactor: remove inheritance of pandas methods in the SmartDataframe * refactor: remove shortcuts * refactor: smart df cannot chat anymore * refactor: remove unused validate method from sdf * refactor: add name and description to connectors * refactor: remove custom prompts * refactor: remove synthetic pipelines * refactor: remove custom instructions * refactor: remove starcoder and falcon * refactor: move df head to connectors * refactor: remove sdf * refactor: remove leftovers logic units * refactor: rename SmartDataLake to AgentCore * refactor: rename DataLake pipeline to Chat * refactor: remove agent core, moving the logic to agent * refactor: refactor the prompt templates using Jinja2 * feat(Agent): train agent docs or question/answers (Sinaptik-AI#895) * feat(VectorStore): adding chromadb vector store for RAG * test(agent_train): adding more test cases and error handling * refact(agent_train): add logging to chroma db * feat(RAG): use trained data from vector db in prompt (Sinaptik-AI#896) * feat(VectorStore): adding chromadb vector store for RAG * test(agent_train): adding more test cases and error handling * refact(agent_train): add logging to chroma db * feat(RAG): use trained vector in prompt * Merge v1.6 into to v2.0 (Sinaptik-AI#900) * refactor(Prompt): adding more context for dataframe and multiple types of serialization (Sinaptik-AI#880) * refactor(prompt): update dataframe serialization in prompt * tests(prompt): fix and adding new tests * fix(prompt): adding type to dataframe serialize function * refactor(prompt): add field descriptions in yml prompt * fix(prompt): direct_sql still using old dataframe table * refactor(direct_sql): add instruction and note for using relevant table only * refactor(query_tracker): clean output of query tracker format (Sinaptik-AI#888) * feat(GoogleBigQuery): adding google big query connector (Sinaptik-AI#886) * feat(VectorStore): adding chromadb vector store for RAG * test(agent_train): adding more test cases and error handling * refact(agent_train): add logging to chroma db * feat(RAG): use trained vector in prompt * Merge v1.6 to v2.0 * feat: execute_sql_query_usage creating error prompt if execute_sql_que… (Sinaptik-AI#898) * feat/execute_sql_query_usage creating error prompt if execute_sql_query is not used when direct_sql is set to true * fix/comment added before raising exception --------- Co-authored-by: Milind Lalwani <[email protected]> * fix(prompt_path): path issue if pandasai used outside of pandas env (Sinaptik-AI#909) * feat(agent): adding optional pipeline call (Sinaptik-AI#916) s * feat(system prompt): use system in llm prompts (Sinaptik-AI#925) * fix(code_manager): minor fixes in code manager * feat(system_prompt): adding system prompts to be added in llm call * fix(code_manager): minor fixes in code manager (Sinaptik-AI#923) Co-authored-by: Gabriele Venturi <[email protected]> * feat/update method in vector db added * feat/get by id method added * feat(multiturn-conv): update prompts and use api's for multiturn (Sinaptik-AI#928) * fix(code_manager): minor fixes in code manager * feat(system_prompt): adding system prompts to be added in llm call * feat(multi-turn-conv): adding support for multi turn conversation * feat(multi-turn): add missing files * feat(BambooVectorStore): adding bamboo vector to store and retrieve t… (Sinaptik-AI#935) * feat(BambooVectorStore): adding bamboo vector to store and retrieve training docs on cloud * rename test class name * feat(BambooLLM): add bamboo llm wrapper (Sinaptik-AI#940) * feat(BambooVectorStore): adding bamboo vector to store and retrieve training docs on cloud * rename test class name * feat(bamboo_llm): adding bamboo llm interface * fix: clean up few commented and leftover changes * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports * fix(plot): always pass one lib for plotting in updated prompt * Merge to release 2.0 (Sinaptik-AI#945) * fix(output_type): handle errors for wrong output type (Sinaptik-AI#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (Sinaptik-AI#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (Sinaptik-AI#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (Sinaptik-AI#876) * fix: rephrase query (Sinaptik-AI#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (Sinaptik-AI#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (Sinaptik-AI#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (Sinaptik-AI#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (Sinaptik-AI#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (Sinaptik-AI#889) (Sinaptik-AI#890) * refactor: TypeVar for IResponseParser (Sinaptik-AI#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (Sinaptik-AI#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (Sinaptik-AI#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (Sinaptik-AI#901) * fix: badge for "Open in Colab" (Sinaptik-AI#903) * feat: add support Google Gemini API in LLMs (Sinaptik-AI#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (Sinaptik-AI#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (Sinaptik-AI#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (Sinaptik-AI#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (Sinaptik-AI#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]> * fix: query tracker track code execution * fix: add skills in query tracker and rag to return one sample by default * fix(skills): add skills to query tracker and by default rag to return 1 message (Sinaptik-AI#947) * fix(output_type): handle errors for wrong output type (Sinaptik-AI#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (Sinaptik-AI#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (Sinaptik-AI#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (Sinaptik-AI#876) * fix: rephrase query (Sinaptik-AI#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (Sinaptik-AI#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (Sinaptik-AI#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (Sinaptik-AI#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (Sinaptik-AI#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (Sinaptik-AI#889) (Sinaptik-AI#890) * refactor: TypeVar for IResponseParser (Sinaptik-AI#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (Sinaptik-AI#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (Sinaptik-AI#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (Sinaptik-AI#901) * fix: badge for "Open in Colab" (Sinaptik-AI#903) * feat: add support Google Gemini API in LLMs (Sinaptik-AI#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (Sinaptik-AI#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (Sinaptik-AI#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (Sinaptik-AI#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (Sinaptik-AI#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports * fix(plot): always pass one lib for plotting in updated prompt * fix: query tracker track code execution * fix: add skills in query tracker and rag to return one sample by default --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]> * fix: function call check and query tracker tracking * fixes/in release2.0 (Sinaptik-AI#951) * fix(output_type): handle errors for wrong output type (Sinaptik-AI#866) * fix(output_type): handle errors for output type * fix: leftovers * fix: test case to mock format-response * fix: upgrade duckdb * Release v1.5.15 * fix(sql): use only added tables of connector (Sinaptik-AI#869) * fix(sql): use only added tables of connector * leftover file * chore: rephrase the error message Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix(sql): fix test cases and improve output error message --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * feat/integration_testing test cases created (Sinaptik-AI#873) * feat/integration_testing test cases created based Loan Payments data * feat/integration_testing four more datasets added * fix/moved csv datasets to integration folder * fix/changed pytest command to run only in tests folder * fix/changed pytest command to run only in tests folder --------- Co-authored-by: Milind Lalwani <[email protected]> * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned (Sinaptik-AI#876) * fix: rephrase query (Sinaptik-AI#872) * Fixed Agent rephrase_query Error * Fixed Agent rephrase_query Error * add encoding --------- Co-authored-by: Pranab Pathak <[email protected]> * Release v1.5.16 * fix(code manager): parsing of called functions (Sinaptik-AI#883) * feat(helpers): add gpt-3.5-turbo-1106 fine-tuned * fix(code manager): parsing of called functions * fmt * fix: open charts return on format plot (Sinaptik-AI#881) Co-authored-by: Long Le <[email protected]> * feat(project): add Makefile, re-lint project, restructure tests (Sinaptik-AI#884) * feat(project): add Makefile, re-lint project, restructure tests * unused imports * Release v1.5.17 * docs:pdate examples.md (Sinaptik-AI#887) Minor fix of exmaples.md * refactor: TypeVar for IResponseParser (Sinaptik-AI#889) (Sinaptik-AI#890) * refactor: TypeVar for IResponseParser (Sinaptik-AI#889) * (refactor): introduce TypeVar for IResponseParser implementation in output_logic_unit.py * (fix): add missing call of super().__init__() in ProcessOutput class * refactor: TypeVar for IResponseParser (Sinaptik-AI#889) * (style): linter fail at output_logic_unit.py * [fix] logging chart saving only if code contains chart (Sinaptik-AI#897) Co-authored-by: Lorenzobattistela <[email protected]> * fix(airtable): use personal access token instead of api key Api key has been deprecated: https://airtable.com/developers/web/api/authentication * feat: add df summarization shortcut (Sinaptik-AI#901) * fix: badge for "Open in Colab" (Sinaptik-AI#903) * feat: add support Google Gemini API in LLMs (Sinaptik-AI#902) * Updating shortcuts to include df summarization * Updating support for google gemini models * Make google-generativeai package optional * fix: upgrade google-ai --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.5.18 * docs: update Google Colab Badge (Sinaptik-AI#914) The existing badge's signature was somehow seems to be expired so just add Google Colab's officially provided svg badge. * docs: Rectify code examples by adding missing statements (Sinaptik-AI#915) Anyone who is quite new to python won't be able to simplify code errors when directly copied code demo from official website. Updated code example is taken from the root README.md and tested. * feat: add support for modin (Sinaptik-AI#907) * feat: add support for modin * fix(ci): dev deps * update docs * add some tests * upate contributing guidelines * fix helpers * fix docs example * update pandasai/smart_dataframe/__init__.py * Release v1.5.19 * feat: update OpenAI pricing * chore: add Flask openai example (Sinaptik-AI#941) * 'Refactored by Sourcery' * chore: add Flask html example * chore: add Flask openai example * sourcery refactor integration tests * fix: Flask package install set to optional --------- Co-authored-by: Sourcery AI <> * chore: restore smart dataframe and smartdatalake functionalities * fix ruff formatting * fix: yahoo connector * fix: file import sorting * fix: import sorting * fix: module import * ignore integration_tests * fix: ruff * ruff fix * remove integration folder * fix: ci workflow * fix: modin * fix: ruff imports * fix(plot): always pass one lib for plotting in updated prompt * fix: query tracker track code execution * fix: add skills in query tracker and rag to return one sample by default * fix: function call check and query tracker tracking --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]> * update comparison operator (Sinaptik-AI#953) * docs: improve readme and license * lint: fix lint in examples * docs: add deploy methods in the documentation * refactor: rename to ChromaDB * feat: make BambooVector the default vectorstore * lint: fix lint in examples * docs: add docs for training, agent description * docs: add video for training LLM --------- Co-authored-by: ArslanSaleem <[email protected]> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: milind-sinaptik <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> Co-authored-by: Pranab1011 <[email protected]> Co-authored-by: Pranab Pathak <[email protected]> Co-authored-by: Milind Lalwani <[email protected]> Co-authored-by: Lh Long <[email protected]> Co-authored-by: Long Le <[email protected]> Co-authored-by: PVA <[email protected]> Co-authored-by: Ihor <[email protected]> Co-authored-by: Lorenzo Battistela <[email protected]> Co-authored-by: Lorenzobattistela <[email protected]> Co-authored-by: Sparsh Jain <[email protected]> Co-authored-by: Devashish Datt Mamgain <[email protected]> Co-authored-by: Hemant Sachdeva <[email protected]> Co-authored-by: aloha-fim <[email protected]>
- Loading branch information
19 people
authored
Feb 29, 2024
1 parent
1eff2de
commit c0efd92