Periodic ENOENT running in CI #86

nathancahill · 2020-06-12T14:02:21Z

This is an error that happens >50% of the time on CircleCI. No changes are made between one run and the next. One run might pass, the next will fail with this error. All versions are committed in yarn.lock.

12 06 2020 08:29:13.851:ERROR [SaucelabsLauncher]: Error: spawn /home/circleci/repo/node_modules/saucelabs/build/.sc-v4.5.4/bin/sc ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:268:19)
    at onErrorNT (internal/child_process.js:468:16)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)

{
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn /home/circleci/repo/node_modules/saucelabs/build/.sc-v4.5.4/bin/sc',
  path: '/home/circleci/repo/node_modules/saucelabs/build/.sc-v4.5.4/bin/sc',
  spawnargs: [ '--version' ],
  killed: false,
  stdout: '',
  stderr: '',
  failed: true,
  signal: null,
  cmd: '/home/circleci/repo/node_modules/saucelabs/build/.sc-v4.5.4/bin/sc --version',
  timedOut: false
}

christian-bromann · 2020-06-12T15:19:22Z

@nathancahill can you try to run the pipeline without cache? I am afraid that the cache that includes the Sauce Connect binary is used for different environments.

christian-bromann · 2020-06-16T17:20:04Z

ping @nathancahill

nathancahill · 2020-06-24T11:00:04Z

Thanks, I think removing the cache solved the issue. I'll reopen in the future if it reappears.

nathancahill · 2020-06-29T08:58:23Z

Even with the cache disabled we're still getting periodic errors.

Previously, with the cache, the error rate was probably 50%. Without the cache, it's around 10%.

Sam55555 · 2020-07-03T13:07:38Z

The error occurred too on each run after gitlab runner cached on gitlab ci using the docker executer. No error during the first run and clean cache.

enriquegh · 2020-07-13T23:40:17Z

Haven't been able to reproduce on a private Gitlab instance with a docker runner so far.

I was able to reproduce, however, if I was using the node alpine image:

 $ NODE_OPTIONS=--trace-warnings ts-node index.ts
Error: spawn /builds/enrique/sauce-sc-example/node_modules/saucelabs/build/.sc-v4.5.4/bin/sc ENOENT
    at Process.ChildProcess._handle.onexit (internal/child_process.js:268:19)
    at onErrorNT (internal/child_process.js:468:16)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn /builds/enrique/sauce-sc-example/node_modules/saucelabs/build/.sc-v4.5.4/bin/sc',
  path: '/builds/enrique/sauce-sc-example/node_modules/saucelabs/build/.sc-v4.5.4/bin/sc',
  spawnargs: [ '--version' ],
  killed: false,
  stdout: '',
  stderr: '',
  failed: true,
  signal: null,
  cmd: '/builds/enrique/sauce-sc-example/node_modules/saucelabs/build/.sc-v4.5.4/bin/sc --version',
  timedOut: false
}

This gave an error 100% of the time though and not intermittently.

enriquegh · 2020-10-07T16:19:01Z

From @joventuraz:

it looks like the node module is not available, if there happened to be more than one concurrent build and one is finishing and cleaning up, it might affect the other build if they are using the same workspace which it seems like they might from the path in the error

Adding to this I think the issue lies somewhere in bin-wrapper and how it does its checks for the file/folder.

It seems like bin-wrapper is a collection of small modules so it was a bit hard to pinpoint exactly what is missing.

Some things we could try and do:

Do more checks on whether the binary is actually there or not and retry to download if its not
Allow for users to specify a folder where sc should be looked for and the user can download this manually
use something other than bin-wrapper since it seems to not be very active

christian-bromann · 2020-10-08T17:35:27Z

I proposed a change in the Sauce Connect Launcher to retry starting Sauce Connect if it fails: karma-runner/karma-sauce-launcher#219

Let's see if we can fix this issue with that.

christian-bromann · 2020-11-16T11:18:55Z

@nathancahill have you experienced the same issue after updating the package?

Seamoo13 · 2021-02-17T14:12:26Z

@christian-bromann Are there any updates on this? I am also having this issue

markcellus · 2021-05-03T13:57:08Z

~~I'm having this issue too using Node 14 alpine (14.16.1). But is no longer an issue after downgrading node back to 12.~~ Just verified that this is not the case.

Seamoo13 · 2021-07-02T11:55:01Z

The issue went away for my project for a few months, then suddenly returned and is occurring roughly 50% of the time (Travis CI).

"sauce-connect-launcher": "1.3.2",
"@wdio/sauce-service": "7.5.7",
"@wdio/cli": "7.5.7",

LeParadoxHD · 2022-03-04T17:09:05Z

Any update on this?

enriquegh · 2022-03-30T22:28:08Z

@wswebcreation would 7.1.3 fix this as bin-wrapper is now replaced?

wswebcreation · 2022-03-31T05:16:20Z

Hi @enriquegh

I'm not 100% sure, we simplified the download now, so it's worth a try

enriquegh · 2022-04-26T16:51:09Z

I'm going to close this ticket since bin-wrapper is no longer used and no new reports have been made.
If someone still runs into the issue, we can re-open it.

Wolftousen · 2022-06-14T15:58:22Z

Not sure if a new issue should be opened up for this, but we are experiencing this issue also, here is the wdio/saucelab info from out package.json:

"@wdio/cli": "^7.19.7",
"@wdio/config": "^7.16.13",
"@wdio/cucumber-framework": "^7.19.7",
"@wdio/local-runner": "^7.19.7",
"@wdio/sauce-service": "^7.20.2",
"@wdio/selenium-standalone-service": "^7.16.13",
"@wdio/spec-reporter": "^7.19.7",
"@wdio/static-server-service": "^7.16.13",
"@wdio/types": "^7.16.13",

And here is the output we are getting with CircleCI:

Error: spawn /project_path/node_modules/saucelabs/build/sc-loader/.sc-v4.8.0/bin/sc ENOENT
at Process.ChildProcess._handle.onexit (internal/child_process.js:274:19)
at onErrorNT (internal/child_process.js:469:16)
at processTicksAndRejections (internal/process/task_queues.js:82:21) {
errno: -2,
code: 'ENOENT',
syscall: 'spawn /project_path/node_modules/saucelabs/build/sc-loader/.sc-v4.8.0/bin/sc',
path: '/project_path/node_modules/saucelabs/build/sc-loader/.sc-v4.8.0/bin/sc',
spawnargs: [
'--no-autodetect',
'--tunnel-name=team_tunnel_name',
'--user=saucelabs_user',
'--api-key=saucelabs_key',
'--region=aws-region'
]
}

I added some logging in the pipeline after looking at my local node_modules and found that while in the pipeline, node_modules/saucelabs/build/sc-loader does not exist at all. However, if you look at node_modules/saucelabs/sc-loader/.sc-v4.8.0/bin, it does contain sc in it. So not sure what the purpose of having downloaded twice is

I was able to add these commands into my test run job to trick wdio/saucelabs into using the sc that exists in saucelabs/sc-loader instead of saucelabs/build/sc-loader:

mkdir -p node_modules/saucelabs/build/sc-loader/.sc-v4.8.0/bin
ln -s node_modules/saucelabs/sc-loader/.sc-v4.8.0/bin/sc node_modules/saucelabs/build/sc-loader/.sc-v4.8.0/bin/sc

That cause the ENOENT error to go away, but now I get the error referenced in this issue: webdriverio/webdriverio#5900

enriquegh · 2022-06-16T01:00:48Z

I've re-opened this as it's essentially the same issue as before.
Looks like we replaced bin-wrapper for the download module (link).

We have something that in theory checks for the binary to be there so not sure why this is happening.

kleinbs · 2023-01-20T19:53:41Z

We are running into the same issue here running in an alpine docker container as part of a gitlab ci job. Is this still being worked on, or is there a good work around?

Execution of 2 workers started at 2023-01-20T17:59:03.464Z
2023-01-20T17:59:03.526Z DEBUG @wdio/utils:initialiseServices: initialise service "sauce" as NPM package
2023-01-20T17:59:04.939Z INFO @wdio/cli:launcher: Run onPrepare hook
2023-01-20T17:59:04.942Z INFO @wdio/sauce-service: Starting Sauce Connect Tunnel
Error: spawn /builds/consumer-identity/OneID/infra-eng/infra-source-bundles/my-account/e2e/node_modules/saucelabs/build/sc-loader/.sc-v4.8.0/bin/sc ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:283:19)
    at onErrorNT (node:internal/child_process:478:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn /builds/consumer-identity/OneID/infra-eng/infra-source-bundles/my-account/e2e/node_modules/saucelabs/build/sc-loader/.sc-v4.8.0/bin/sc',
  path: '/builds/consumer-identity/OneID/infra-eng/infra-source-bundles/my-account/e2e/node_modules/saucelabs/build/sc-loader/.sc-v4.8.0/bin/sc',
  spawnargs: [
    '--verbose',
    '--logfile=-',
    '--no-autodetect',
    '--tunnel-name=SC-tunnel-40602318395085324',
    '--no-ssl-bump-domains=127.0.0.1,localhost,1[72](https://gitlab.disney.com/consumer-identity/OneID/infra-eng/infra-source-bundles/my-account/-/jobs/18227172#L72).17.0.3',
    '--user=MyAccountTestRunner',
    '--api-key=7ee95499-d176-4eed-ace0-0c83a4ec6e[74](https://gitlab.disney.com/consumer-identity/OneID/infra-eng/infra-source-bundles/my-account/-/jobs/18227172#L74)',
    '--region=us-west-1'
  ]
}

naruaway · 2024-04-12T14:42:19Z

I think probably this is one of the root cause #241

christian-bromann added the bug label Jun 12, 2020

nathancahill closed this as completed Jun 24, 2020

nathancahill reopened this Jun 29, 2020

christian-bromann added the Hacktoberfest Curated issues which are well scoped and ready to be worked on as part of Hacktoberfest label Oct 6, 2020

christian-bromann mentioned this issue Oct 8, 2020

Retry start Sauce Connect karma-runner/karma-sauce-launcher#219

Merged

enriquegh mentioned this issue Oct 9, 2020

retry if SC fails to start webdriverio/webdriverio#5961

Merged

8 tasks

wswebcreation mentioned this issue Nov 10, 2020

Request failed with status 500 due to Error: Sauce could not start your job webdriverio/webdriverio#5900

Closed

christian-bromann removed the Hacktoberfest Curated issues which are well scoped and ready to be worked on as part of Hacktoberfest label Dec 4, 2020

christian-bromann added the Help Wanted Issues were the community can get involved label Jun 22, 2021

enriquegh closed this as completed Apr 26, 2022

enriquegh reopened this Jun 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Periodic ENOENT running in CI #86

Periodic ENOENT running in CI #86

nathancahill commented Jun 12, 2020

christian-bromann commented Jun 12, 2020

christian-bromann commented Jun 16, 2020

nathancahill commented Jun 24, 2020

nathancahill commented Jun 29, 2020 •

edited

Loading

Sam55555 commented Jul 3, 2020

enriquegh commented Jul 13, 2020

enriquegh commented Oct 7, 2020

christian-bromann commented Oct 8, 2020

christian-bromann commented Nov 16, 2020

Seamoo13 commented Feb 17, 2021 •

edited

Loading

markcellus commented May 3, 2021 •

edited

Loading

Seamoo13 commented Jul 2, 2021 •

edited

Loading

LeParadoxHD commented Mar 4, 2022

enriquegh commented Mar 30, 2022

wswebcreation commented Mar 31, 2022

enriquegh commented Apr 26, 2022

Wolftousen commented Jun 14, 2022 •

edited

Loading

enriquegh commented Jun 16, 2022

kleinbs commented Jan 20, 2023 •

edited

Loading

naruaway commented Apr 12, 2024

Periodic ENOENT running in CI #86

Periodic ENOENT running in CI #86

Comments

nathancahill commented Jun 12, 2020

christian-bromann commented Jun 12, 2020

christian-bromann commented Jun 16, 2020

nathancahill commented Jun 24, 2020

nathancahill commented Jun 29, 2020 • edited Loading

Sam55555 commented Jul 3, 2020

enriquegh commented Jul 13, 2020

enriquegh commented Oct 7, 2020

christian-bromann commented Oct 8, 2020

christian-bromann commented Nov 16, 2020

Seamoo13 commented Feb 17, 2021 • edited Loading

markcellus commented May 3, 2021 • edited Loading

Seamoo13 commented Jul 2, 2021 • edited Loading

LeParadoxHD commented Mar 4, 2022

enriquegh commented Mar 30, 2022

wswebcreation commented Mar 31, 2022

enriquegh commented Apr 26, 2022

Wolftousen commented Jun 14, 2022 • edited Loading

enriquegh commented Jun 16, 2022

kleinbs commented Jan 20, 2023 • edited Loading

naruaway commented Apr 12, 2024

nathancahill commented Jun 29, 2020 •

edited

Loading

Seamoo13 commented Feb 17, 2021 •

edited

Loading

markcellus commented May 3, 2021 •

edited

Loading

Seamoo13 commented Jul 2, 2021 •

edited

Loading

Wolftousen commented Jun 14, 2022 •

edited

Loading

kleinbs commented Jan 20, 2023 •

edited

Loading