Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle GMs without corresponding check / step name #190

Open
beatngu13 opened this issue May 2, 2019 · 0 comments
Open

Handle GMs without corresponding check / step name #190

beatngu13 opened this issue May 2, 2019 · 0 comments
Labels
enhancement New feature or request

Comments

@beatngu13
Copy link
Contributor

When we try to assign a step name to a GM or vice versa, three situations can occur:

  1. We cannot find a GM for the given step name, so we fail the test case. This is the default behavior whenever a test is being executed for the first time.
  2. The step name corresponds to a golden master, so we can proceed with the comparison of expected and actual.
  3. We have a golden master but without a step name.

AFAIK we currently don't handle no. 3? This can happen if a check is removed or a step renamed. The developer has to decide whether the checkpoint is missing, the GM can be deleted, or something else.

In any case, we should probably track the executed tests and then compare them agains the contents of the recheck folder? Afterwards, we could at least print out a warning when there is a GM but no corresponding has been executed.

@beatngu13 beatngu13 added the enhancement New feature or request label May 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

No branches or pull requests

1 participant