Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Whether the test results between different models will affect each other #1487

Open
Yekai97 opened this issue Dec 19, 2024 · 1 comment
Open

Comments

@Yekai97
Copy link

Yekai97 commented Dec 19, 2024

System Info

Python 3.12
pandasai 2.4.0

🐛 Describe the bug

Hello, I am using PandsAI in my project, I want to compare and test the retrieval accuracy of different models, but I found a problem, that is, when I use the same query statement to query the data of different models such as ChatGPT and local LLM on the same excel sheet, the query result of the previous model seems to have an impact on the latter result, such as the query error of the previous model, and the same error will occur when the latter model is queried, and the answer will be given quickly, like I've memorized the wrong answer and didn't do a new search process, is this normal, or how can I avoid it?

Copy link

dosubot bot commented Dec 19, 2024

PandasAI has a caching mechanism that might be causing the issue you're experiencing. The cache stores the results of previous queries, which can affect subsequent queries by different models. This caching is enabled by default and is intended to speed up query retrieval and reduce API call costs. To avoid this issue, you can disable the cache by setting the enable_cache parameter to False when creating the PandasAI object. Additionally, you can clear the cache by calling the clear_cache() method on the PandasAI object, or destroy it using the destroy() method of the Cache class, which removes all cache files [1][2][3].

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant