Fix bug where stacked decorators don't update #39
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of the bug: The
partial_credit
andleaderboard
decorators are class-based and use a class-based decorator pattern with@functools.wraps
around an inner decorator method, along with a local callback function that is passed to the tests (i.e.set_score
andset_leaderboard_value
). The bug is thatset_score
andset_leaderboard_value
update attributes of the wrong function when they are stacked. This means that only the outermost decorator with a callback function works. Example of the bug:or, with the decorators in the other order:
Existing workaround: One can in principle avoid these issues by having separate
partial_credit
andleaderboard
tests. However, this is not future-proof for adding new decorators. In fact, I discovered this bug because I added my own custom decorator fortimeout
, and I found that@timeout
followed by@partial_credit
failed while the reverse order behaved properly.Fix: This PR fixes the issue by essentially emulating the
functools.wraps
behavior after the function is called (usingupdate_wrapper
), which ensures that the new values set byset_score
andset_leaderboard_value
are copied to the outer wrappers and thus visible to the test runner. I opted to use a context manager for this because it makes it easy to drop in a 1-line fix inside the decorators.