Validate that NUnit's ITestEventListener within Jenkins Pipeline for a netcoreapp3.1 assembly is called

[geraldbauer]

I initially posted this question on stack overflow (https://stackoverflow.com/questions/66877034/validate-that-nunits-itesteventlistener-within-jenkins-pipeline-for-a-netcoreap) and wasn't sure the question was not seen, thus the posting here.

Locally, I have successfully implemented the interface ITestEventListener within a C# netcoreapp3.1 csproj. However, when the tests are an within a Jenkins Pipeline, things appear to not be working(?).

I am using version 3.12.0 of NUnit.Engine.

By locally I am referring to using 1) Visual Studio Version 16.9.2 to run the tests and 2) command line dotnet test -c devint --test-adapter-path:. --logger:nunit to run the tests. I am getting successful test runs.

Success is my [Extension]public class ReportTestListener : ITestEventListener {...} generates an html file. I see the html file is created locally whereas from the Jenkins Pipeline the html file is not generated.

Within the Jenkins Pipeline, I am using the command sh "dotnet test -c ${env.TARGET_ENV} --test-adapter-path:. --logger:nunit" where env.TARGET_ENV resolves to devint. I know tests successfully run within the Jenkins Pipeline since the NUnit test results file is generated/published.

What I am not sure of is how to test/validate that the ReportTestListener is being called within the Jenkins Pipeline. I know that testing frameworks such as NUnit uses refection to identify test classes and methods. I am presuming that also happens with my implementation of [Extension]public class ReportTestListener : ITestEventListener {...}. Ideas/Suggestions on how to validate that ITestEventListener's method void OnTestEvent(string report) is being called besides writing out to disk?

[mikkelbu]

Hi @geraldbauer

I don't have much experience with ITestEventListener, but as far as I can tell the recommended approach is to have the ITestEventListener implementation in a separate assembly and make sure that this assembly is referred to from an addins file. (see e.g. https://stackoverflow.com/questions/38749270/how-to-use-itesteventlistener-in-nunit-3 and #487 (comment)).

If the tests were run from the NUnit console, we could use the command line argument --list-extensions to show the installed extensions, but I don't think this is available when using dotnet test ... as the tests are run using the NUnit adapter and VsTest. I've transferred the question to the NUnit console project as this is the relevant repo for the engine.

[geraldbauer]

Yes you have described how I implemented ITestEventListener. It is within another assembly as described within your reference and here https://stackoverflow.com/questions/66376104/how-to-implement-nunits-nunit-engine-itesteventlistener.

Sorry, I failed to mention in the original post that using the CLI on my developer Windows OS dotnet test -c dev --test-adapter-path:. --logger:nunit works as expected - I am getting output from my ITestEventListener implementation.

Because of your posting, I worked with my colleague who has a Mac OS. I saw a strange behavior in writing out to file. When I changed \ to / things began to work in my Jenkins Pipeline - output is being generated. Bottom-line, this is a file slash issue between Windows vs Mac OS and Unix OS.

[geraldbauer]

Changed file writing text path to use / instead of \.