Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix unorderable types error when sorting slow tests #1

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Fix unorderable types error when sorting slow tests
Part, where slow tests are printed, where not cevered with tests. Added tests
for that.

In rare cases, when time of two test cases are the same, sorted function lookes
to the next element of tuple, and next element is test instance which is not
orderable. And that rare case it gives following error:

    Traceback (most recent call last):
      File "CoverageTestRunner.py", line 312, in <module>
          run()
      File "CoverageTestRunner.py", line 303, in run
          result = runner.run()
      File "CoverageTestRunner.py", line 265, in run
          for secs, test in sorted(result.timings)[-10:]:
    TypeError: unorderable types: FooTests() < FooTests()

The fix was to always sort only by time, not by whole tuple.
  • Loading branch information
sirex committed Jun 14, 2017
commit cc7f26b9478382861ceba3a859228bd61cc8c571
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.pyc
6 changes: 4 additions & 2 deletions CoverageTestRunner.py
Original file line number Diff line number Diff line change
@@ -23,6 +23,7 @@
import sys
import time
import logging
import operator


__version__ = '1.11'
@@ -258,11 +259,12 @@ def run(self):
if result.missing_test_modules:
print(len(result.missing_test_modules), "missing test modules")

maxtime = int(os.environ.get('COVERAGE_TEST_RUNNER_MAX_TIME', '10'))
maxtime = float(os.environ.get('COVERAGE_TEST_RUNNER_MAX_TIME', '10'))
if end_time - start_time > maxtime:
print()
print("Slowest tests:")
for secs, test in sorted(result.timings)[-10:]:
timings = sorted(result.timings, key=operator.itemgetter(0))
for secs, test in timings[-10:]:
print(" %5.1f s %s" % (secs, str(test)[:70]))

print("Time: %.1f s" % (end_time - start_time))
4 changes: 4 additions & 0 deletions subdir/foo.py
Original file line number Diff line number Diff line change
@@ -13,4 +13,8 @@ def foo(self, a):
time.sleep(0)
return False

def slow(self, seconds):
import time
time.sleep(seconds)

foo = Foo()
12 changes: 10 additions & 2 deletions subdir/foo_tests.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,19 @@
import unittest, foo
import unittest
import foo


class FooTests(unittest.TestCase):

def testTrue(self):
f = foo.Foo()
self.failUnlessEqual(f.foo(True), True)

def testFalse(self):
f = foo.Foo()
self.failUnlessEqual(f.foo(False), False)


class SlowTests(unittest.TestCase):

def testSlow(self):
foo.Foo().slow(0.11)
2 changes: 2 additions & 0 deletions testrun
Original file line number Diff line number Diff line change
@@ -2,4 +2,6 @@

set -e

export COVERAGE_TEST_RUNNER_MAX_TIME=0.1

python CoverageTestRunner.py subdir --ignore-missing-from=test-excluded