Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added test_id param to test types #79

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

ramji-c
Copy link
Contributor

@ramji-c ramji-c commented Feb 11, 2021

In addition to test_name and description, a unique test identifier is often assigned to quickly identify different test cases. To enable this, I have added a new optional test_id keyword argument to abstract_test as well as MFT, DIR, INV classes. Also, I created a tests directory with unit tests that checks if object creation succeeds with and without this new argument. Please merge if you feel this could be useful.

@ramji-c
Copy link
Contributor Author

ramji-c commented Feb 11, 2021

I am actively working on subtle changes to the scripts as part of implementing CheckList at Expedia. I am happy to submit more PR for these features that I am adding. If not, I will just keep them in my fork. Let me know.

@marcotcr
Copy link
Owner

I thought about having a unique identifier when I first wrote this, and decided at that time that the test name was probably going to be unique enough. Could you elaborate as to why you felt the need for an ID?

Thanks : )

@ramji-c
Copy link
Contributor Author

ramji-c commented Feb 12, 2021

Sure. In my implementation, I assign each capability a specific range of test_id values, akin to error codes. See the table below.
image
This allows users to quickly identify and associate a capability with a test_id and also provide additional differentiating factor, given that test names could become very similar as we create more and more fine grained tests.
Moreover, all tests are exported to a database on top of which we are building a web application for different teams to create and run tests. This means that test names are subject to subtle alterations by product and tech teams. Therefore, test_id could serve as the primary key, which would then allow test_name to be modified at any time without affecting the table underneath.

@marcotcr
Copy link
Owner

Did you by any chance consider using TestSuite? We group tests by capabilities there as well. I guess I didn't think about exporting tests to a database.
Let me think about this a bit more, maybe I should change how tests are saved / exported in general

@ramji-c
Copy link
Contributor Author

ramji-c commented Feb 12, 2021

Oh yeah, I am using custom test suite - inherited from TestSuite. So, we create one CustomTestSuite per model and within those test suites there are different tests.
Like I mentioned, I am actively working on implementation at Expedia and so there will be few more features that I might add along the way. I am happy to submit PR for those in the event that it could be useful.

@ramji-c
Copy link
Contributor Author

ramji-c commented Feb 12, 2021

A note on serializing test suites:
The current capability to pickle (or dill) the TestSuite is great. However, in addition to saving the TestSuite object, I also export it out as JSON file, which lends itself to a web application more readily. We could always load a TestSuite object in the backend, but having the test case saved as JSON makes it easy to be copied over for other models and provides more responsive UI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants