Attachments not attached

[pete-thompson]

Forgive me if I've missed a configuration option for this, but I can't see any code to handle attachments so think I'm accurate.

I'm using MSTest and making use of the AddResultFile method to attach files to my test cases. I'd like those files to be included in the junit test results - it appears that the closest thing to a 'standard' is the Jenkins plugin which is also supported by GitLab: https://plugins.jenkins.io/junit-attachments/.

Both GitLab and Jenkins support text written to stdout/stderr in a particular format - [[ATTACHMENT|/absolute/path/to/some/file]]. I'm not clear what represents an 'absolute path' in the context of CI/CD given that once files upload to the CI/CD servers the real absolute path isn't meaningful - my guess is that they mean a path relative to the base path for uploading result files.

[Siphonophora]

@pete-thompson you are right this isn't something we currently are supporting. I just spent a few minutes looking at the docs, and its a little confusing. I guess GitLab is working with a different schema than we are.

I think the absolute path is probably relative to the root of the repository. Probably not too hard to experiment.

So I understand better, what is the distinction between an artifact and a attachment for you? If you were working in Gitlab CI/CD I know it would be easy to save all the files to a particular directory and then upload them all as artifacts, without relying on the attachment logic.

[pete-thompson]

Thanks @Siphonophora . I'm successfully uploading the files as artifacts, but I'd like them linked to the individual test cases in the JUnit report so that I can figure out which attachment goes with which test case.

In my particular situation I have a test case that's data driven - meaning that I'm running around 50 different tests under the same test name. I'm generating screenshots as attachments for each of the tests, which makes it hard to tell which file goes with which test instance. When I wrote all this, I was using Azure DevOps and the attachments were automatically linked in to the test reporting - but I'm now using GitLab which means I need to switch to JUnit. I'm going to update my tests to write out the filenames to the system output to make things a little easier, but it would be ideal if the JUnit report also contained the linkage.

Sadly, our on-premise GitLab instance doesn't support Junit report yet (the team that owns it were meant to upgrade it last week but delayed it) - so it's hard for me to test things yet, but I hope to write out log messages that would let me test the attachment feature as soon as our GitLab supports it.

[Siphonophora]

Was just looking at https://github.com/spekt/nunit.testlogger/issues/54 which is related. They would both depend on the core logger capturing attachment info to enable this feature.

[codito]

@Siphonophora we've added test attachments support in core testlogger with spekt/testlogger@0a35c55. Here's the NUnit support for reference: spekt/nunit.testlogger@577f536.

We should consider updating version of JUnit logger to 3.1.x with this new feature.

[codito]

Leaving this link here for examples of adding attachment in junit xml: https://github.com/testmoapp/junitxml#attachments-screenshots-and-artifacts