Missing Log entries & Empty results are schown as "OK"

[michael9876]

There seems to be a problem with uncaught exceptions. NUnit collects them all and reports them in their result.xml, but they are not found in the WebApp.

Here is an example for such an exception taken from the NUnit result xml:

<![CDATA[
Server stack trace:
at System.Threading.Thread.InternalCrossContextCallback(Context ctx, IntPtr ctxID, Int32 appDomainID, InternalCrossContextDelegate ftnToCall, Object[] args)
at System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessage(IMessage reqMsg)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at .When
at lambda_method(Closure , IContextManager , String )
at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(StepInstance stepInstance)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.Step(StepDefinitionKeyword stepDefinitionKeyword, String keyword, String text, String multilineTextArg, Table tableArg)
at TechTalk.SpecFlow.TestRunner.When(String text, String multilineTextArg, Table tableArg, String keyword)
at () in .feature:line <>
]]>

As a sidenode: The GlueCode and the selenium related code are in an external library.

In the WebApp the scenario result is 'OK' and when you open it, there are no Results in there.

Incase it is not possible to actually catch this exception (there are other examples as well) and write it to the result, the WebApp should definately default to:
"When a scenario doesnt contain any steps, the scenario result should definately NOT be 'OK' but 'Error'"

The following result data (generated as a result of the unlogged exception above) leads to such a dangerous behavior:
{
"given": {
"result": "OK",
"steps": [],
"start_time": "0001-01-01T00:00:00",
"end_time": "0001-01-01T00:00:00"
},
"when": {
"result": "OK",
"steps": [],
"start_time": "2014-12-08T18:55:21.0951765+01:00",
"end_time": "2014-12-08T18:55:22.7046705+01:00"
},
"then": {
"result": "OK",
"steps": [],
"start_time": "2014-12-08T18:55:22.7046705+01:00",
"end_time": "2014-12-08T18:55:23.0354289+01:00"
},
"result": "OK",
"tags": [],
"title": "my scenario title",
"start_time": "2014-12-08T18:55:21.0951765+01:00",
"end_time": "2014-12-08T18:55:23.0354289+01:00"
},

Maybe even a step without a title should be automatically marked as 'Error'

thanks alot for your support - feel free to contact me back

[jacdevos]

I get the same issue - bit of a showstopper