Best Duration to display as a test summary

[JustinGrote]

Should I display Duration or UserDuration? I'm leaning towards UserDuration, but if something goes weird in framework duration, someone might see a test take 5 sefconds but only show 20ms and not understand. I think education could be made if it takes that long but the actual test was short that it is somethiing in the framework, but looking for thoughts.

[fflaten]

Is it a string or double/decimal ? If string, I'd consider adding the same output as Pester to keep it familiar. If only a numerical value, I'd say UserDuration is most useful.

The down-side is that you might get issues/questions regarding slow response in the extension when users feel there's a big mismatch between real and reported timing, but that will also happen if Discovery is painfully slow either way.

[JustinGrote]

@fflaten maybe I can add a warning toast if I detect if the non-User duration on a test is higher than normal, maybe make it a configurable threshold, that's a good compromise.

The tests api takes it as milliseconds. I've already filed an issue to ask them to humanize the number. There is a "description" field where I may be able to bring over the stringified test details that include the breakout.
microsoft/vscode#126717

[nohwnd]

Why not reporting the complete duration? The Framework duration is overhead of Pester, but it is still real part of the test. E.g. when many mocks are used the test can be 100x slower than if there were no mocks.

[fflaten]

Ahh, fair point. I thought that was part of userdurarion since it's invoked by the user.

[nohwnd]

Not 100% sure which property is which. I know I wanted to merge them, but I am not sure if I did, because Duration was used in many places for different purposes. If the run looks like this 100ms (90ms|10ms), then the number 100 should be reported to the user, if you choose just one number to report.

[JustinGrote]

@nohwnd fair point, that's what I'll report. I was just thinking userDuration was best from a code performance duration, e.g. if you're testing a loop and it takes 3 seconds, then you make some adjustments and it takes 2 seconds, you can see that specificially to the duration minus the scaffolding.

What I'll do is report Duration as the number in the number here, and then probably report the 100ms (90ms|10ms) into the description section of the result.