error.name missing when errors are serialized when running tests in parallel - Causes test results to be incorrectly reported in allure-mocha

[jamesmortensen]

Prerequisites

• Checked that your issue hasn't already been filed by cross-referencing issues with the faq label
• Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
• 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
• Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with: node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend that you not install Mocha globally.

Description

Some reporters use the type of error to determine how to treat a test that fails. Allure is one example. If a test fails due to an AssertionError, then the test is marked in red, as failed. All other errors are treated as broken tests. These results are marked in yellow, as broken.

When running mocha in parallel, the error objects are serialized and then later deserialized. During the serialization process, we are preserving the error.message but not the error.name. Because of this, it's impossible for Allure to determine the type of error by using the error.name property.

Steps to Reproduce

To Reproduce
Steps to reproduce the behavior:

1. Clone this repo: https://github.com/jamesmortensen/allure-js-reporter-issues#when-run-in-parallel-mode-with-hooks-hookerror-is-not-a-function and follow the instructions to run the tests and generate the report.

The example has us run tests in sequential mode as well as parallel mode. The README also has screenshots which show the issue.

Expected behavior

• For AssertionErrors, results should appear in the report in RED.

Actual behavior

• In parallel mode, they're reported in YELLOW.

Screenshots

• Good (shows red): https://github.com/jamesmortensen/allure-js-reporter-issues/blob/master/screenshots/sequential-mode-report.png
• Bad (shows yellow): https://github.com/jamesmortensen/allure-js-reporter-issues/blob/master/screenshots/parallel-mode-report.png

Reproduces how often:

Reproduces 100% of the time.

Versions

• The output of mocha --version and node_modules/.bin/mocha --version: 10.2.0
• The output of node --version: 18.3.0
• Your operating system
  • name and version: macOS Ventura 13.3.1
  • architecture (32 or 64-bit): arm64
• Your shell (e.g., bash, zsh, PowerShell, cmd): bash
• Your browser and version (if running browser tests): N/A
• Any third-party Mocha-related modules (and their versions): allure-mocha v2.4.0
• Any code transpiler (e.g., TypeScript, CoffeeScript, Babel) being used (and its version): N/A

Additional information:

Issue originally reported here: allure-framework/allure-js#485 but I traced the root cause to Mocha.

[JoshuaKGoldberg]

Hey thanks for such a detailed report + PR! I'm having a hard time reproducing this locally. Do you have a reproduction that doesn't involve installing the Java runtime?

~/repos/allure-js-reporter-issues $ npm run open-report 

> [email protected] open-report
> allure generate --clean && npx allure open

The operation couldn’t be completed. Unable to locate a Java Runtime.
Please visit http://www.java.com for information on installing Java.

~/repos/allure-js-reporter-issues $ npx allure-generate clean && npx allure open
npm ERR! code E404
npm ERR! 404 Not Found - GET https://registry.npmjs.org/allure-generate - Not found
npm ERR! 404 
npm ERR! 404  'allure-generate@*' is not in this registry.
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

#4995 looks like a good fix but it's hard to say without a more isolated reproduction.

[JoshuaKGoldberg]

👋 ping @jamesmortensen, is this still on your radar?

[JoshuaKGoldberg]

It's been over a month since last ping, so closing this out as stale. If you again have time for this and can provide a reproduction, feel free to reopen @jamesmortensen.

Cheers! 🤎