Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix flakt test of test_serps.py::TestSERPs::test_mojeek #26

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

lonly7star
Copy link

This PR aims to fix the flaky test on test_serps.py::TestSERPs::test_mojeek so the test could pass single run, multiple test run, and random test run. I'm doing this for a course project as a practice.

The result

The test won't pass on the single run with pytest, the following error raised
> self.assertValidSERPs(serps, use_naive_method=True)
> self.assertEqual(res.keyword, expected_keyword)
E AttributeError: 'NoneType' object has no attribute 'keyword'

Steps to reproduce the issue

  1. run the test file with pytest test_serps.py

Issue of the code

The test tried to extract keyword from mojeek.com,("https://mojeek.com", "Mojeek", ""), which gives a None from the extraction. However, the assertValidSERP function tries to use this assertion self.assertEqual(res.keyword, expected_keyword) but since the result res is None so the assertion failed by trying to access the keyword element of a None object.

Proposed solution

Although the original test has the comment this test passes because of a manual change to the generated search engines JSON file. the next time that file is regenerated, this test will break. , I do have a proposed solution to include the None return keyword situation in the test.
I adding an if statement if res is None: to check if the result is None before trying to access the element of it. if the result is None, then we compare the expect keyword to see if it is empty string self.assertEqual('', expected_keyword) and assert false for the url serp self.assertFalse(is_serp(url, **kwargs)). Using the else to include original assert statement. In this case, we could also compare the empty keyword return.
I had run the whole test suit and the rest of test still passed.
I also test it with pytest --flake-finder which is a re-run tool also detect flaky test in multiple runs. It still passed the multi-run detection. You can install it via pip install pytest-flakefinder if interested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant