Performance implications of using Verify

[arphox]

Hi!

I think it would be important to have some documentation on the performance implications of using Verify altogether, as it would help the decision process of many people who are concerned about performance.

[arphox]

I come from a background where performance is important and reading a file (or more files? help me out please) for each execution of each unit test doesn't sound that good when we are talking about unit tests which take microseconds to run.

And to back up my data about how much time unit tests need, I did some benchmarks on some of our unit tests, the results:

• the fastest ones are in the two to three digits of nanoseconds range (XX-XXX ns) (simple code and simple test, using Assert.Equals, no mock framework)
• the "fast in general" ones are in the one digit of microseconds range (2-9) μs (simple to medium code, faster FluentAssertions methods, no mock framework)
• the "medium" ones are around 100 μs (simple to medium code, includes slower FluentAssertions methods, no mock framework)
• the "slower" ones are around 1-2 ms (unit tests using Moq)

After checking how much time does it need (on a Samsung 860 QVO) to check whether a file exists (it doesn't, randomized filename postfix), and it turns out it added around 3 μs overhead.

And instead of checking for file existence, actually reading a 2.4 KB file added around 22 μs overhead but caching probably mislead me here so it should be more.

So, to sum up, in my case, reading a file from an older SATA SSD (from 2019) added about two-three digits microseconds (XX-XXX μs) overhead per test execution.

For tests that are already in the milliseconds range (i.e. using mocking frameworks or slower FluentAssertions methods), using this library probably won't be a problem (if you run your tests in parallel and you don't have many tens of thousands test cases).