fix flakt test of test_serps.py::TestSERPs::test_mojeek #26
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR aims to fix the flaky test on test_serps.py::TestSERPs::test_mojeek so the test could pass single run, multiple test run, and random test run. I'm doing this for a course project as a practice.
The result
The test won't pass on the single run with pytest, the following error raised
> self.assertValidSERPs(serps, use_naive_method=True)
> self.assertEqual(res.keyword, expected_keyword)
E AttributeError: 'NoneType' object has no attribute 'keyword'
Steps to reproduce the issue
pytest test_serps.py
Issue of the code
The test tried to extract keyword from
mojeek.com,("https://mojeek.com", "Mojeek", "")
, which gives a None from the extraction. However, theassertValidSERP
function tries to use this assertionself.assertEqual(res.keyword, expected_keyword)
but since the resultres
is None so the assertion failed by trying to access the keyword element of a None object.Proposed solution
Although the original test has the comment
this test passes because of a manual change to the generated search engines JSON file. the next time that file is regenerated, this test will break.
, I do have a proposed solution to include the None return keyword situation in the test.I adding an if statement
if res is None:
to check if the result is None before trying to access the element of it. if the result is None, then we compare the expect keyword to see if it is empty stringself.assertEqual('', expected_keyword)
and assert false for the url serpself.assertFalse(is_serp(url, **kwargs))
. Using the else to include original assert statement. In this case, we could also compare the empty keyword return.I had run the whole test suit and the rest of test still passed.
I also test it with
pytest --flake-finder
which is a re-run tool also detect flaky test in multiple runs. It still passed the multi-run detection. You can install it viapip install pytest-flakefinder
if interested.