Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

42% of integration tests don't pass #284

Open
ProducerMatt opened this issue Jun 16, 2023 · 2 comments
Open

42% of integration tests don't pass #284

ProducerMatt opened this issue Jun 16, 2023 · 2 comments
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@ProducerMatt
Copy link
Member

ProducerMatt commented Jun 16, 2023

Admittedly this number is exaggerated. The majority of failures come from Question-related modules, which I don't have API keys for and therefore can't work.

@ProducerMatt ProducerMatt added bug Something isn't working help wanted Extra attention is needed labels Jun 17, 2023
@ProducerMatt
Copy link
Member Author

What is this? Is it supposed to be getting a DDG definition?

`TEST #6: FAILED`
The sent message was: "TEST_MESSAGE 6: what is deep learning?"
The expected message was "RegEx: Deep learning is part of a broader family of machine learning method"
The received message was "Perhaps this can answer your question? https://aisafety.info?state=8XBK_"

@ProducerMatt ProducerMatt changed the title 55% of integration tests don't pass 42% of integration tests don't pass Jun 17, 2023
@Aprillion
Copy link
Collaborator

yup, a test from duckduckgo, you can search for "what is deep learning?" in the code:

test_message="what is deep learning?",
expected_regex="Deep learning is part of a broader family of machine learning method",

not sure what would be a reliable test that wouldn't be overruled by other modules in the future - feel free to delete fragile tests for now if you won't figure out how to replace them with something more stable 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants