-
Notifications
You must be signed in to change notification settings - Fork 57
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Explicitly exit with the correct status
Why: * When running tests on a directory, the exit status was 0, regardless of the actual exit status of the runner * CI builds depend on status codes for each step in the build process to determine whether the build passed or failed. Since `exit()` is being used here, test suites that experience runtime errors will still exit with a 0 status code. This change addresses the need by: * Exiting with 1 when there is a `RuntimeError` * After the rake task runs, exit with the exit status of the runner * Creates a method to assert output when the underlying process failed, instead of assuming that it will always be successful
- Loading branch information
1 parent
14f4563
commit 1783276
Showing
5 changed files
with
48 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
require_relative "../../test/test_helper.rb" | ||
|
||
class ErrorTest < MTest | ||
def test_purposeful_error | ||
raise RuntimeError | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
require "test_helper" | ||
|
||
class ExitCodesTest < MTest | ||
def test_failing_test_returns_1 | ||
m("examples/subdir_with_failures/a_test") | ||
refute $?.success?, "expected exit code to be 1 but it was #{$?.exitstatus}" | ||
end | ||
|
||
def test_test_with_error_returns_1 | ||
m("../lib/error_tests/error_test") | ||
refute $?.success?, "expected exit code to be 1 but it was #{$?.exitstatus}" | ||
end | ||
|
||
def test_dir_with_failure_returns_1 | ||
m("examples/subdir_with_failures") | ||
refute $?.success?, "expected exit code to be 1 but it was #{$?.exitstatus}" | ||
end | ||
|
||
def test_dir_with_error_returns_1 | ||
m("../lib/error_tests") | ||
refute $?.success?, "expected exit code to be 1 but it was #{$?.exitstatus}" | ||
end | ||
|
||
def test_without_errors_or_failures_returns_0 | ||
m("examples/subdir/a_test") | ||
assert $?.success?, "expected exit code to be 0 but it was #{$?.exitstatus}" | ||
end | ||
|
||
def test_dir_without_errors_or_failures_returns_0 | ||
m("examples/subdir") | ||
assert $?.success?, "expected exit code to be 0 but it was #{$?.exitstatus}" | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1783276
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@zamith this looks great to me