[Cypress] Change Log Level to Debug in parseTestPlan Function

[Gerry-Mc]

Hi! 👋

Firstly, thanks for your work on this project! 🙂

Today I used patch-package to patch [email protected] for the project I'm working on.

Description

The parseTestPlan function in allure-js-commons currently logs an error with the stack trace when it fails to parse the test plan. This can be overly verbose and may not be necessary for all use cases. We propose changing the log level from error to debug to provide cleaner log output while retaining the ability to diagnose issues when needed.

Current Behavior

When the parseTestPlan function encounters an error, it logs the following message:

could not parse test plan {path}.allure\testplan.json SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at parseTestPlan ({path}\node_modules\allure-js-commons\src\sdk\reporter\testplan.ts:15:27)
    at createRuntimeState ({path}\node_modules\allure-cypress\src\reporter.ts:502:26)
    at initializeRuntimeState ({path}\node_modules\allure-cypress\src\reporter.ts:519:30)
    at allureCypress ({path}\node_modules\allure-cypress\src\reporter.ts:560:5)
    at setupNodeEvents ({path}\cypress.config.ts:18:20)
    at {path}\AppData\Local\Cypress\Cache\13.14.2\Cypress\resources\app\node_modules\@packages\server\lib\plugins\child\run_plugins.js:122:14       
    at tryCatcher ({path}\AppData\Local\Cypress\Cache\13.14.2\Cypress\resources\app\node_modules\bluebird\js\release\util.js:16:23)
    at Function.Promise.attempt.Promise.try ({path}\AppData\Local\Cypress\Cache\13.14.2\Cypress\resources\app\node_modules\bluebird\js\release\method.js:39:29)
    at RunPlugins.load ({path}\AppData\Local\Cypress\Cache\13.14.2\Cypress\resources\app\node_modules\@packages\server\lib\plugins\child\run_plugins.js:119:9)
    at RunPlugins.runSetupNodeEvents ({path}\AppData\Local\Cypress\Cache\13.14.2\Cypress\resources\app\node_modules\@packages\server\lib\plugins\child\run_plugins.js:59:17)
    at EventEmitter.<anonymous> ({path}\AppData\Local\Cypress\Cache\13.14.2\Cypress\resources\app\node_modules\@packages\server\lib\plugins\child\run_require_async_child.js:185:22)
    at EventEmitter.emit (node:events:519:28)
    at EventEmitter.emit (node:domain:488:12)
    at process.<anonymous> ({path}\AppData\Local\Cypress\Cache\13.14.2\Cypress\resources\app\node_modules\@packages\server\lib\plugins\util.js:33:22)
    at process.emit (node:events:519:28)
    at process.emit (node:domain:488:12)
    at process.emit.sharedData.processEmitHook.installedValue [as emit] ({path}\AppData\Local\Cypress\Cache\13.14.2\Cypress\resources\app\node_modules\@cspotcode\source-map-support\source-map-support.js:745:40)
    at emit (node:internal/child_process:951:14)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

Rationale

• Cleaner Logs: Changing the log level to debug results in cleaner log output, making it easier to identify relevant information.
• Debugging Capability: The error information is still available when the debug level is enabled, allowing for effective troubleshooting.

Steps to Reproduce

1. Use the parseTestPlan function with an invalid test plan path.
2. Observe the error log output.

Suggested Solution

Update the parseTestPlan function in node_modules/allure-js-commons/dist/cjs/sdk/reporter/testplan.js as follows:

try {
// existing code
return testPlan;
} catch (e) {
// eslint-disable-next-line no-console
console.debug("could not parse test plan ".concat(testPlanPath), e);
return undefined;
}

Additional Context

This change was implemented locally and tested to ensure it suppresses the stack trace while still logging the error message. A patch file has been created using patch-package to apply this change.

Thank you for considering this improvement. Please let me know if there are any questions or if further adjustments are needed.

Here is the diff that solved my problem:

diff --git a/node_modules/allure-js-commons/dist/cjs/sdk/reporter/testplan.js b/node_modules/allure-js-commons/dist/cjs/sdk/reporter/testplan.js
index 17bf2a9..25758c0 100644
--- a/node_modules/allure-js-commons/dist/cjs/sdk/reporter/testplan.js
+++ b/node_modules/allure-js-commons/dist/cjs/sdk/reporter/testplan.js
@@ -22,7 +22,7 @@ var parseTestPlan = () => {
     return testPlan;
   } catch (e) {
     // eslint-disable-next-line no-console
-    console.error("could not parse test plan ".concat(testPlanPath), e);
+    console.debug("could not parse test plan ".concat(testPlanPath), e);
     return undefined;
   }
 };

This issue body was partially generated by patch-package.

[baev]

What is the reason for having an invalid test plan? Could you please provide steps to reproduce the problem?

[Gerry-Mc]

Hey Dmitry, Thanks for the quick response!

I'm using allurectl to handle it, rather than create the file myself, as suggested in the docs here: https://docs.qameta.io/allure-testops/ecosystem/allurectl/

Starting from allurectl release 2.10.x, the tool and Allure TestOps fully control the process of selective test run (please mind the Allure Framework part as well), and it is not required to define ALLURE_TESTPLAN_PATH and executing job-run plan command in your pipeline. You can still use the instructions described below. However, we recommend leaving this whole process to Allure TestOps and allurectl.

I see the above issue when using allurectl watch when the file does not exist at all.

[StanislavGrishaev]

I see the above issue when using allurectl watch when the file does not exist at all.

same issue here, this is the reason why i'm still on version 2.15.1

[baev]

allurectl should not provide the ALLURE_TESTPLAN_PATH variable with a path to a non-existing or invalid file.

Could you please specify the allurectl version you're using?

[StanislavGrishaev]

@baev here is part of my log on "allure-playwright": "^3.0.3":

Allure TestOps Client v2.15.1
allurectl was not able to detect CI system, watch is executed in non-ci mode.
No information about active job run...
Project [3] with name [proj]
Creating new launch with name [ui-tests]
Launch [28856] with name [ui-tests]
Session [29972] started
Report link: some-url
could not parse test plan /home/node/app/.allure/testplan.json Error: ENOENT: no such file or directory, open '/home/node/app/.allure/testplan.json'
    at readFileSync (node:fs:448:20)
    at parseTestPlan (/home/node/app/node_modules/allure-js-commons/src/sdk/reporter/testplan.ts:14:30)
    at AllureReporter.onConfigure (/home/node/app/node_modules/allure-playwright/src/index.ts:93:35)
    at /home/node/app/node_modules/playwright/lib/reporters/multiplexer.js:32:65
    at wrap (/home/node/app/node_modules/playwright/lib/reporters/multiplexer.js:86:5)
    at Multiplexer.onConfigure (/home/node/app/node_modules/playwright/lib/reporters/multiplexer.js:32:45)
    at InternalReporter.onConfigure (/home/node/app/node_modules/playwright/lib/reporters/internalReporter.js:46:20)
    at Runner.runAllTests (/home/node/app/node_modules/playwright/lib/runner/runner.js:73:14)
    at runTests (/home/node/app/node_modules/playwright/lib/program.js:218:18)
    at t.<anonymous> (/home/node/app/node_modules/playwright/lib/program.js:55:7) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/home/node/app/.allure/testplan.json'
}

[cheshi-mantu]

@baev here is part of my log:

Allure TestOps Client v2.15.1
allurectl was not able to detect CI system, watch is executed in non-ci mode.
No information about active job run...
Project [3] with name [proj]
Creating new launch with name [ui-tests]
Launch [28856] with name [ui-tests]
Session [29972] started
Report link: some-url
could not parse test plan /home/node/app/.allure/testplan.json Error: ENOENT: no such file or directory, open '/home/node/app/.allure/testplan.json'
    at readFileSync (node:fs:448:20)
    at parseTestPlan (/home/node/app/node_modules/allure-js-commons/src/sdk/reporter/testplan.ts:14:30)
    at AllureReporter.onConfigure (/home/node/app/node_modules/allure-playwright/src/index.ts:93:35)
    at /home/node/app/node_modules/playwright/lib/reporters/multiplexer.js:32:65
    at wrap (/home/node/app/node_modules/playwright/lib/reporters/multiplexer.js:86:5)
    at Multiplexer.onConfigure (/home/node/app/node_modules/playwright/lib/reporters/multiplexer.js:32:45)
    at InternalReporter.onConfigure (/home/node/app/node_modules/playwright/lib/reporters/internalReporter.js:46:20)
    at Runner.runAllTests (/home/node/app/node_modules/playwright/lib/runner/runner.js:73:14)
    at runTests (/home/node/app/node_modules/playwright/lib/program.js:218:18)
    at t.<anonymous> (/home/node/app/node_modules/playwright/lib/program.js:55:7) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/home/node/app/.allure/testplan.json'
}

hi, could you please shed a bit more light here?
How the tests exec command looks like (everything that is related to execution of allurectl and tests execution by allurectl)?
what is the used scenario?

1. CI starts the pipeline
2. Allure TestOps starts the pipeline

cheers

[StanislavGrishaev]

@cheshi-mantu hi there.

How the tests exec command looks like

./allurectl watch --results=./output/allure-results --launch-name="ui-tests" --launch-tags "local" -- npx playwright test

what is the used scenario?

just execution Playwright tests in Docker container with uploading results to Allure Test Ops by allurectl (jfyi, with "allure-playwright": "^2.15.1" there is no ENOENT error in the same scenario)

just in case, here is part of my playwright.config.ts:

    reporter: [
        [
            'allure-playwright',
            {
                detail: false,
                resultsDir: './output/allure-results',
                suiteTitle: false,
                environmentInfo: {
                    PLAYWRIGHT_VERSION: require('@playwright/test/package.json').version,
                    NODEJS_VERSION: process.version,
                    OS_TYPE: os.type,
                    OS_ARCH: os.arch(),
                    WORKERS: process.env.WORKERS,
                    TEST_SHARDS: process.env.TEST_SHARDS,
                },
            },
        ],
    ],

[Gerry-Mc]

My setup is similar, I'm testing a new framework on my local machine:
I am using the windows binary of allurectl version 2.15.1
[email protected] and [email protected]

"testops": ".\allurectl_windows_amd64.exe watch --endpoint https://{env}.testops.cloud --results cypress/reports/allure-results --token {token} --project-id {id} -- npm run cy:run",
"cy:run": "cypress run --browser chrome",

And I kick it off with 'npm run testops'

Here are my dependencies if it helps:

 "devDependencies": {
    "@types/axe-core": "^2.0.7",
    "@types/cypress": "^1.1.3",
    "@types/node": "^22.5.5",
    "allure-commandline": "^2.30.0",
    "allure-cypress": "^3.0.0",
    "cypress": "^13.14.2",
    "cypress-axe": "^1.5.0",
    "cypress-on-fix": "^1.0.3",
    "cypress-parallel": "^0.14.0",
    "cypress-split": "^1.7.0",
    "npm-run-all": "^4.1.5",
    "patch-package": "^8.0.0",
    "postinstall-postinstall": "^2.1.0",
    "rimraf": "^6.0.1",
    "typescript": "^5.6.2"
  },
  "dependencies": {
    "cypress-if": "^1.12.6",
    "dotenv": "^16.4.5"
  }

[cheshi-mantu]

@cheshi-mantu hi there.

How the tests exec command looks like

./allurectl watch --results=./output/allure-results --launch-name="ui-tests" --launch-tags "local" -- npx playwright test

what is the used scenario?

just execution Playwright tests in Docker container with uploading results to Allure Test Ops by allurectl (jfyi, with "allure-playwright": "^2.15.1" there is no ENOENT error in the same scenario)

just in case, here is part of my playwright.config.ts:

    reporter: [
        [
            'allure-playwright',
            {
                detail: false,
                resultsDir: './output/allure-results',
                suiteTitle: false,
                environmentInfo: {
                    PLAYWRIGHT_VERSION: require('@playwright/test/package.json').version,
                    NODEJS_VERSION: process.version,
                    OS_TYPE: os.type,
                    OS_ARCH: os.arch(),
                    WORKERS: process.env.WORKERS,
                    TEST_SHARDS: process.env.TEST_SHARDS,
                },
            },
        ],
    ],

just execution Playwright tests in Docker container with uploading results to Allure Test Ops by allurectl (jfyi, with "allure-playwright": "^2.15.1" there is no ENOENT error in the same scenario)

what are the ALLURE_ variables defined before allurectl start in the context of the execution of allurectl?
where docker container is starting? Is it a Ci or the local exec?
what is the user inside the container used to create the image and what is the user in the context of the execution?

allurectl uses the home directory of the current user to store the data related to its parameters like ~/.allure if there is no access rights for the user, then testplan.json won't be read by the tool.

[cheshi-mantu]

@Gerry-Mc
could not parse test plan {path}.allure\testplan.json SyntaxError: Unexpected end of JSON input

what is {path} here? Is it added manually or os it real output?

[StanislavGrishaev]

what are the ALLURE_ variables defined before allurectl start in the context of the execution of allurectl?

ALLURE_ENDPOINT=https://some-site.com
ALLURE_PROJECT_ID=3
ALLURE_TOKEN=some-token

where docker container is starting? Is it a Ci or the local exec?

local, don't check it in our CI (openshift) yet.

what is the user inside the container used to create the image and what is the user in the context of the execution?

here is part of my Dockerfile:

FROM artifactory.tutu.ru:443/docker-common/playwright:v1.47.2-focal

ENV NPM_CONFIG_CACHE=/home/node/app/.npm

WORKDIR /home/node/app

COPY package*.json ./

RUN npm install

COPY . .

RUN chmod -R 1777 /home/node/app

ARG ALLURECTL_VERSION=2.15.1
RUN ARCH=$(uname -m | sed -e 's/x86_64/linux_amd64/' -e 's/arm64/linux_arm64/' -e 's/aarch64/linux_arm64/' -e 's/i386/linux_386/' -e 's/i686/linux_386/') && \
    wget "https://github.com/allure-framework/allurectl/releases/download/${ALLURECTL_VERSION}/allurectl_$ARCH" -O ./allurectl && \
    chmod +x ./allurectl

RUN mkdir /.allure && chmod -R 1777 /.allure

ENTRYPOINT ["/home/node/app/entrypoint.sh"]

[cheshi-mantu]

RUN mkdir /.allure && chmod -R 1777 /.allure

allurectl uses ~/.allure directory
and generally manages it (creates dir and creates tetsplan.json if required)

[StanislavGrishaev]

@cheshi-mantu so... what should I try ? Some changes in Dockerfile or add some env. variables to Docker container ?

[Gerry-Mc]

@Gerry-Mc could not parse test plan {path}.allure\testplan.json SyntaxError: Unexpected end of JSON input

what is {path} here? Is it added manually or os it real output?

@cheshi-mantu Sorry for the confusion, that's manually edited, that's the project path on my local machine, all the bracketed elements are local, {id} etc.

@StanislavGrishaev You can raise the log level above error, by adding '-log fatal' before 'watch' in your command, but you may miss important messages.

If you want to add the proposed change as an interim solution you can patch it into the dockerfile with something like :

# ...

RUN npm install

COPY ..

# Copy the patch above into a .patch file - ensure it's included in the files being copied
COPY allure-js-commons.patch .

# Apply the patch to the allure-js-commons module - check the paths in the file match what you need
RUN patch -p1 < allure-js-commons.patch

RUN chmod -R 1777 /home/node/app

# ...

[StanislavGrishaev]

You can raise the log level above error, by adding '-log fatal' before 'watch' in your command, but you may miss important messages.

alright, ./allurectl watch --log fatal do the trick and it suits me as a temporary solution. thanks @Gerry-Mc !