Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to run a specific parameter set in parameterized tests using --filter option #542

Open
chokuto opened this issue May 29, 2024 · 1 comment

Comments

@chokuto
Copy link

chokuto commented May 29, 2024

I have been using Criterion for running parameterized tests and encountered an issue when trying to run a specific parameter set using the --filter option.

I would like to run only the test with a specific parameter. I tried using the --filter option like this:

./test --filter 'suite_name/test_name/1'

However, it seems this approach does not work as expected. I need to be able to run and debug specific parameter sets in parameterized tests.

Could you please provide guidance on how to achieve this? Is there a specific way to filter and run only the tests with certain parameters in parameterized tests using Criterion? If not, is there a recommended workaround or an alternative approach to accomplish this?

Thank you for your help!

@jcsahnwaldt
Copy link

jcsahnwaldt commented Aug 18, 2024

I guess the filter .../1 is supposed to only run the test with parameter value 1. I think it would be very difficult for Criterion to implement something like this.

The problem is that C is a strongly typed language, and the parameter value can have any type (int, long, a string, a struct, ...). Criterion would have to somehow parse the value in the command line argument, convert it to the parameter's type, and compare it to the parameter values. Not easy in C...

Maybe you can modify your code in ParameterizedTestParameters and filter the list of parameter values it returns depending on the value of a special environment variable. Then you could invoke your tests like this:

FOO_BAR_PARAM_FILTER=1 ./test --filter 'foo_suite/bar_test'

Of course, this means that you'll have to implement the parsing, converting and comparing yourself. But since it's for a specific type, it's much easier than the generic case described above. Maybe half a dozen lines will do...

If the filter .../1 is supposed to select the parameter at index 1, adding that feature would be much less work, but also less useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants