Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When browser disconnects, remaining tests are skipped #42

Open
juanmendes opened this issue Aug 12, 2019 · 5 comments
Open

When browser disconnects, remaining tests are skipped #42

juanmendes opened this issue Aug 12, 2019 · 5 comments

Comments

@juanmendes
Copy link

juanmendes commented Aug 12, 2019 •
edited
Loading

I'm submitting a ...

  • [ X ] bug report

What is the current behavior?
When a test (shard) disconnects, that shard is restarted but it only runs 1 more test in that set of tests giving it a false SUCCESS.

More info
We had tests that had been failing on our CI but we didn't know it because the build was being reported as successful. Checking in new tests caused the shards to change revealing failures for tests that were previously being skipped

The logs below are at the time when one of the browsers disconnects and it's restarted with a single test.

[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 445 of 462 SUCCESS (13 mins 20.993 secs / 3 mins 2.351 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 488 of 496 (skipped 8) SUCCESS (12 mins 31.592 secs / 3 mins 7.926 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 374 of 375 (skipped 1) SUCCESS (9 mins 18.06 secs / 1 min 30.408 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 433 of 455 (skipped 11) SUCCESS (17 mins 5.845 secs / 4 mins 10.347 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 361 of 439 (skipped 23) SUCCESS (20 mins 55.986 secs / 3 mins 59.233 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 277 of 543 (skipped 9) SUCCESS (0 secs / 1 min 55.531 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 321 of 345 (skipped 9) SUCCESS (0 secs / 1 min 36.614 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 277 of 543 (skipped 9) SUCCESS (0 secs / 1 min 55.531 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 445 of 462 SUCCESS (13 mins 20.993 secs / 3 mins 2.351 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 488 of 496 (skipped 8) SUCCESS (12 mins 31.592 secs / 3 mins 7.926 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 374 of 375 (skipped 1) SUCCESS (9 mins 18.06 secs / 1 min 30.408 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 433 of 455 (skipped 11) SUCCESS (17 mins 5.845 secs / 4 mins 10.347 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 361 of 439 (skipped 23) SUCCESS (20 mins 55.986 secs / 3 mins 59.233 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 277 of 543 (skipped 9) SUCCESS (0 secs / 1 min 55.531 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 321 of 345 (skipped 9) SUCCESS (15 mins 51.026 secs / 1 min 36.614 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 277 of 543 (skipped 9) SUCCESS (0 secs / 1 min 55.531 secs)
[INFO] �[36m10 08 2019 10:58:57.585:DEBUG [launcher]: �[39mProcess ChromeHeadless exited with code 0
[INFO] �[36m10 08 2019 10:58:57.586:DEBUG [temp-dir]: �[39mCleaning temp dir /build/mts/release/sb-26750517/tmp/karma-85401258
[INFO] �[32m10 08 2019 10:58:57.646:INFO [launcher]: �[39mStarting browser ChromeHeadless
[INFO] �[36m10 08 2019 10:58:57.646:DEBUG [temp-dir]: �[39mCreating temp dir at /build/mts/release/sb-26750517/tmp/karma-57661985
[INFO] �[36m10 08 2019 10:58:57.647:DEBUG [launcher]: �[39m/build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/puppeteer/.local-chromium/mac-499413/chrome-mac/Chromium.app/Contents/MacOS/Chromium --user-data-dir=/build/mts/release/sb-26750517/tmp/karma-57661985 --no-default-browser-check --no-first-run --disable-default-apps --disable-popup-blocking --disable-translate --disable-background-timer-throttling --disable-renderer-backgrounding --disable-device-discovery-notifications http://localhost:9876/?id=57661985 --headless --disable-gpu --remote-debugging-port=9222
[INFO] �[36m10 08 2019 10:58:58.991:DEBUG [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/karma/static/client.html
[INFO] �[36m10 08 2019 10:58:59.098:DEBUG [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/karma/static/karma.js
[INFO] �[36m10 08 2019 10:58:59.727:DEBUG [karma]: �[39mA browser has connected on socket iPW9xSfxDhqUoxXYAAAR
[INFO] �[36m10 08 2019 10:58:59.776:DEBUG [web-server]: �[39mupgrade /socket.io/?EIO=3&transport=websocket&sid=iPW9xSfxDhqUoxXYAAAR
[INFO] �[32m10 08 2019 10:58:59.823:INFO [HeadlessChrome 63.0.3205 (Mac OS X 10.9.5)]: �[39mConnected on socket iPW9xSfxDhqUoxXYAAAR with id 57661985
[INFO] �[36m10 08 2019 10:58:59.824:DEBUG [framework:karma-parallel]: �[39mregistering browser id 57661985 with aggregated browser id 372422729345 at shard index 8
[INFO] �[36m10 08 2019 10:58:59.824:DEBUG [launcher]: �[39mChromeHeadless (id 57661985) captured in 3679.386 secs
[INFO] �[36m10 08 2019 10:58:59.928:DEBUG [middleware:karma]: �[39mcustom files null null null
[INFO] �[36m10 08 2019 10:58:59.929:DEBUG [middleware:karma]: �[39mServing static request /context.html
[INFO] �[36m10 08 2019 10:58:59.929:DEBUG [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/karma/static/context.html
...
[INFO] �[36m10 08 2019 10:58:59.961:DEBUG [middleware:parallel]: �[39minterpolating parallel shard data map in script. Data: {"1394027":{"shouldShard":true,"shardIndex":4,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"18315375":{"shouldShard":true,"shardIndex":2,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"21084253":{"shouldShard":true,"shardIndex":1,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"22596072":{"shouldShard":true,"shardIndex":7,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"57661985":{"shouldShard":true,"shardIndex":8,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"74312172":{"shouldShard":true,"shardIndex":3,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"85401258":{"shouldShard":true,"shardIndex":6,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"96928968":{"shouldShard":true,"shardIndex":0,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"}}
[INFO] �[36m10 08 2019 10:58:59.963:DEBUG [middleware:source-files]: �[39mRequesting /base/node_modules/@clr/ui/clr-ui.css?3db758150e273cb5e4fb6f6ca7d2ba31be036245 /
[INFO] �[36m10 08 2019 10:58:59.964:DEBUG [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/@clr/ui/clr-ui.css
...[INFO] �[36m10 08 2019 10:59:00.059:DEBUG [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/src/main/bootstrap.test.bundle.js

[INFO] �[36m10 08 2019 10:59:00.060:DEBUG [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/web-animations-js/web-animations.min.js
[INFO] �[36m10 08 2019 10:59:00.060:DEBUG [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/src/main/bootstrap.test.bundle.js
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 445 of 462 SUCCESS (13 mins 20.993 secs / 3 mins 2.351 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 488 of 496 (skipped 8) SUCCESS (12 mins 31.592 secs / 3 mins 7.926 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 374 of 375 (skipped 1) SUCCESS (9 mins 18.06 secs / 1 min 30.408 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 433 of 455 (skipped 11) SUCCESS (17 mins 5.845 secs / 4 mins 10.347 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 361 of 439 (skipped 23) SUCCESS (20 mins 55.986 secs / 3 mins 59.233 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 278 of 543 (skipped 9) SUCCESS (0 secs / 1 min 56.317 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 321 of 345 (skipped 9) SUCCESS (15 mins 51.026 secs / 1 min 36.614 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 278 of 543 (skipped 9) SUCCESS (0 secs / 1 min 56.317 secs)
[INFO]
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 445 of 462 SUCCESS (13 mins 20.993 secs / 3 mins 2.351 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 488 of 496 (skipped 8) SUCCESS (12 mins 31.592 secs / 3 mins 7.926 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 374 of 375 (skipped 1) SUCCESS (9 mins 18.06 secs / 1 min 30.408 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 433 of 455 (skipped 11) SUCCESS (17 mins 5.845 secs / 4 mins 10.347 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 361 of 439 (skipped 23) SUCCESS (20 mins 55.986 secs / 3 mins 59.233 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 278 of 543 (skipped 9) SUCCESS (0 secs / 1 min 56.317 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 321 of 345 (skipped 9) SUCCESS (15 mins 51.026 secs / 1 min 36.614 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 278 of 543 (skipped 9) SUCCESS (0 secs / 1 min 56.317 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 0 of 1 SUCCESS (0 secs / 0 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 445 of 462 SUCCESS (13 mins 20.993 secs / 3 mins 2.351 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 488 of 496 (skipped 8) SUCCESS (12 mins 31.592 secs / 3 mins 7.926 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 374 of 375 (skipped 1) SUCCESS (9 mins 18.06 secs / 1 min 30.408 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 433 of 455 (skipped 11) SUCCESS (17 mins 5.845 secs / 4 mins 10.347 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 361 of 439 (skipped 23) SUCCESS (20 mins 55.986 secs / 3 mins 59.233 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 278 of 543 (skipped 9) SUCCESS (0 secs / 1 min 56.317 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 321 of 345 (skipped 9) SUCCESS (15 mins 51.026 secs / 1 min 36.614 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 278 of 543 (skipped 9) SUCCESS (0 secs / 1 min 56.317 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 1 of 1 SUCCESS (0 secs / 0.021 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 445 of 462 SUCCESS (13 mins 20.993 secs / 3 mins

This is the end of a not so successful run

[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 445 of 462 SUCCESS (13 mins 20.993 secs / 3 mins 2.351 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 488 of 496 (skipped 8) SUCCESS (12 mins 31.592 secs / 3 mins 7.926 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 374 of 375 (skipped 1) SUCCESS (9 mins 18.06 secs / 1 min 30.408 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 433 of 455 (skipped 11) SUCCESS (17 mins 5.845 secs / 4 mins 10.347 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 361 of 439 (skipped 23) SUCCESS (20 mins 55.986 secs / 3 mins 59.233 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 508 of 543 (skipped 10) SUCCESS (17 mins 39.158 secs / 4 mins 51.437 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 321 of 345 (skipped 9) SUCCESS (15 mins 51.026 secs / 1 min 36.614 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 508 of 543 (skipped 10) SUCCESS (17 mins 39.158 secs / 4 mins 51.437 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 1 of 1 SUCCESS (0.082 secs / 0.021 secs)
[INFO] TOTAL: 2931 SUCCESS
[INFO] �[36m10 08 2019 11:05:37.736:DEBUG [karma]: �[39mRun complete, exiting.
[INFO] �[36m10 08 2019 11:05:37.737:DEBUG [launcher]: �[39mDisconnecting all browsers
[INFO] �[36m10 08 2019 11:05:39.813:DEBUG [launcher]: �[39mProcess ChromeHeadless exited with code 0
[INFO] �[36m10 08 2019 11:05:39.814:DEBUG [temp-dir]: �[39mCleaning temp dir /build/mts/release/sb-26750517/tmp/karma-22596072
[INFO] �[36m10 08 2019 11:05:39.821:DEBUG [launcher]: �[39mFinished all browsers

One shard that seem to be absent is the following (with 490 tests), which had been logged earlier

2019-08-10 10:37:26 gobuilds.Compile : [INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 270 of 490 (skipped 2) SUCCESS (0 secs / 53.286 secs)

That shard had not connected back to the karma server for over 7 minutes. Here's the log for that disconnect:

[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 488 of 496 (skipped 8) SUCCESS (12 mins 31.592 secs / 3 mins 7.926 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 374 of 375 (skipped 1) SUCCESS (9 mins 18.06 secs / 1 min 30.408 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 433 of 455 (skipped 11) SUCCESS (17 mins 5.845 secs / 4 mins 10.347 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 361 of 439 (skipped 23) SUCCESS (20 mins 55.986 secs / 3 mins 59.233 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 359 of 490 (skipped 2) SUCCESS (0 secs / 1 min 48.384 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 67 of 345 (skipped 4) SUCCESS (0 secs / 15.553 secs)
[INFO] �[36m10 08 2019 10:46:59.205:DEBUG [HeadlessChrome 63.0.3205 (Mac OS X 10.9.5)]: �[39mDisconnected during run, waiting 300000ms for reconnecting.
[INFO] �[33m10 08 2019 10:47:17.884:WARN [HeadlessChrome 63.0.3205 (Mac OS X 10.9.5)]: �[39mDisconnected (1 times)
[INFO] �[1A�[2K�[1A�[2K�[1A�[2K�[1A�[2K�[1A�[2K�[1A�[2K�[1A�[2KHeadlessChrome 63.0.3205 (Mac OS X 10.9.5) ERROR
[INFO]   Disconnectedundefined
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 445 of 462 SUCCESS (13 mins 20.993 secs / 3 mins 2.351 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 488 of 496 (skipped 8) SUCCESS (12 mins 31.592 secs / 3 mins 7.926 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 374 of 375 (skipped 1) SUCCESS (9 mins 18.06 secs / 1 min 30.408 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 433 of 455 (skipped 11) SUCCESS (17 mins 5.845 secs / 4 mins 10.347 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 361 of 439 (skipped 23) SUCCESS (20 mins 55.986 secs / 3 mins 59.233 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 359 of 490 (skipped 2) DISCONNECTED (16 mins 12.71 secs / 1 min 48.384 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 67 of 345 (skipped 4) SUCCESS (0 secs / 15.553 secs)
[INFO] �[1A�[2K�[1A�[2K�[1A�[2K�[1A�[2K�[1A�[2K�[1A�[2K�[1A�[2KHeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 445 of 462 SUCCESS (13 mins 20.993 secs / 3 mins 2.351 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 488 of 496 (skipped 8) SUCCESS (12 mins 31.592 secs / 3 mins 7.926 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 374 of 375 (skipped 1) SUCCESS (9 mins 18.06 secs / 1 min 30.408 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 433 of 455 (skipped 11) SUCCESS (17 mins 5.845 secs / 4 mins 10.347 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 361 of 439 (skipped 23) SUCCESS (20 mins 55.986 secs / 3 mins 59.233 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 359 of 490 (skipped 2) DISCONNECTED (16 mins 12.71 secs / 1 min 48.384 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 67 of 345 (skipped 4) SUCCESS (0 secs / 15.553 secs)
[INFO] �[karma]: �[39mRestarting HeadlessChrome 63.0.3205 (Mac OS X 10.9.5) (1 of 2 attempts)
[INFO] � [launcher]: �[39mProcess ChromeHeadless exited with code 0
[INFO] � [temp-dir]: �[39mCleaning temp dir /build/mts/release/sb-26750517/tmp/karma-22596072
[INFO] � [launcher]: �[39mRestarting ChromeHeadless
[INFO] � [temp-dir]: �[39mCreating temp dir at /build/mts/release/sb-26750517/tmp/karma-22596072
[INFO] � [launcher]: �[39m/build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/puppeteer/.local-chromium/mac-499413/chrome-mac/Chromium.app/Contents/MacOS/Chromium --user-data-dir=/build/mts/release/sb-26750517/tmp/karma-22596072 --no-default-browser-check --no-first-run --disable-default-apps --disable-popup-blocking --disable-translate --disable-background-timer-throttling --disable-renderer-backgrounding --disable-device-discovery-notifications http://localhost:9876/?id=22596072 --headless --disable-gpu --remote-debugging-port=9222
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/karma/static/client.html
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/karma/static/karma.js
[INFO] � [karma]: �[39mA browser has connected on socket m6X7yWpx9pIADeQhAAAO
[INFO] � [web-server]: �[39mupgrade /socket.io/?EIO=3&transport=websocket&sid=m6X7yWpx9pIADeQhAAAO
[INFO] �[HeadlessChrome 63.0.3205 (Mac OS X 10.9.5)]: �[39mConnected on socket m6X7yWpx9pIADeQhAAAO with id 22596072
[INFO] � [framework:karma-parallel]: �[39mregistering browser id 22596072 with aggregated browser id 372422729345 at shard index 7
[INFO] � [launcher]: �[39mChromeHeadless (id 22596072) captured in 2980.016 secs
[INFO] � [middleware:karma]: �[39mcustom files null null null
[INFO] � [middleware:karma]: �[39mServing static request /context.html
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/karma/static/context.html
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/karma/static/context.js
[INFO] � [middleware:source-files]: �[39mRequesting /base/node_modules/jasmine-core/lib/jasmine-core/jasmine.js?916005cc407925f4764668d61d04888d59258f5d /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/jasmine-core/lib/jasmine-core/jasmine.js
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/jasmine-core/lib/jasmine-core/jasmine.js
[INFO] � [middleware:parallel]: �[39minterpolating parallel shard data map in script. Data: {"1394027":{"shouldShard":true,"shardIndex":4,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"18315375":{"shouldShard":true,"shardIndex":2,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"21084253":{"shouldShard":true,"shardIndex":1,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"22596072":{"shouldShard":true,"shardIndex":7,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"74312172":{"shouldShard":true,"shardIndex":3,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"85401258":{"shouldShard":true,"shardIndex":6,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"},"96928968":{"shouldShard":true,"shardIndex":0,"executors":8,"shardStrategy":"round-robin","customShardStrategy":"ZnVuY3Rpb24gKCkgewogICAgICB0aHJvdyBuZXcgRXJyb3IoCiAgICAgICAgJ1Nob3VsZCBzcGVjaWZ5IGEgImN1c3RvbVNoYXJkU3RyYXRlZ3kiIGZ1bmN0aW9uIHdoZW4gdXNpbmcgc2hhcmRTdHJhdGVneTogImN1c3RvbSInCiAgICAgICk7CiAgICB9"}}
[INFO] � [middleware:source-files]: �[39mRequesting /base/node_modules/jasmine-ajax/lib/mock-ajax.js?df88a27d939f0dd46c1dc30c36edfe6d7e053d60 /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/jasmine-ajax/lib/mock-ajax.js
[INFO] � [middleware:source-files]: �[39mRequesting /base/node_modules/karma-jasmine/lib/boot.js?945a38bf4e45ad2770eb94868231905a04a0bd3e /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/karma-jasmine/lib/boot.js
[INFO] � [middleware:source-files]: �[39mRequesting /base/node_modules/karma-jasmine/lib/adapter.js?7a813cc290d592e664331c573a1a796192cdd1ad /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/karma-jasmine/lib/adapter.js
[INFO] � [middleware:source-files]: �[39mRequesting /base/node_modules/@clr/ui/clr-ui.css?3db758150e273cb5e4fb6f6ca7d2ba31be036245 /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/@clr/ui/clr-ui.css
[INFO] � [middleware:source-files]: �[39mRequesting /base/dist/styles/themes/theme-default.css?ac498c850ea4ef089e799d506cdbc2c4ec0e8de2 /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/dist/styles/themes/theme-default.css
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/jasmine-ajax/lib/mock-ajax.js
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/karma-jasmine/lib/boot.js
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/karma-jasmine/lib/adapter.js
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/@clr/ui/clr-ui.css
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/dist/styles/themes/theme-default.css
[INFO] � [middleware:source-files]: �[39mRequesting /base/node_modules/systemjs/dist/system.js?9b219a5749355e16d09b90e6e93e3db047c3b8be /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/systemjs/dist/system.js
[INFO] � [middleware:source-files]: �[39mRequesting /base/node_modules/core-js/client/shim.min.js?17dee087101c3d1b07880df4b15dbcfa9b208b30 /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/core-js/client/shim.min.js
[INFO] � [middleware:source-files]: �[39mRequesting /base/node_modules/mutationobserver-shim/dist/mutationobserver.min.js?1ad83990b11a736e06d4114f86941190cd115a6d /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/mutationobserver-shim/dist/mutationobserver.min.js
[INFO] � [middleware:source-files]: �[39mRequesting /base/node_modules/@webcomponents/custom-elements/custom-elements.min.js?736577d836e7526cb79c7b4eae056bd9d868948f /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/@webcomponents/custom-elements/custom-elements.min.js
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/systemjs/dist/system.js
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/core-js/client/shim.min.js
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/mutationobserver-shim/dist/mutationobserver.min.js
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/@webcomponents/custom-elements/custom-elements.min.js
[INFO] � [middleware:source-files]: �[39mRequesting /base/src/main/bootstrap.test.bundle.js?11ce64288816aff4f2c74bcde464782c0484ae5e /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/src/main/bootstrap.test.bundle.js
[INFO] � [middleware:source-files]: �[39mRequesting /base/node_modules/web-animations-js/web-animations.min.js?83042534752a6d9b0a4a086517e19230416feba4 /
[INFO] � [middleware:source-files]: �[39mFetching /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/web-animations-js/web-animations.min.js
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/src/main/bootstrap.test.bundle.js
[INFO] � [web-server]: �[39mserving (cached): /build/mts/release/sb-26750517/vcd-ui/content/core/node_modules/web-animations-js/web-animations.min.js
[INFO] 
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 445 of 462 SUCCESS (13 mins 20.993 secs / 3 mins 2.351 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 488 of 496 (skipped 8) SUCCESS (12 mins 31.592 secs / 3 mins 7.926 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 374 of 375 (skipped 1) SUCCESS (9 mins 18.06 secs / 1 min 30.408 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 433 of 455 (skipped 11) SUCCESS (17 mins 5.845 secs / 4 mins 10.347 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 361 of 439 (skipped 23) SUCCESS (20 mins 55.986 secs / 3 mins 59.233 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 0 of 543 SUCCESS (0 secs / 0 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 67 of 345 (skipped 4) SUCCESS (0 secs / 15.553 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 0 of 543 SUCCESS (0 secs / 0 secs)
[INFO] �[36m10 08 2019 10:48:19.841:DEBUG [karma]: �[39mA browser has connected on socket iUO7OrOOlA2q-QboAAAP
[INFO] �[36m10 08 2019 10:48:19.999:DEBUG [web-server]: �[39mupgrade /socket.io/?EIO=3&transport=websocket&sid=iUO7OrOOlA2q-QboAAAP
[INFO] �[36m10 08 2019 10:48:20.272:DEBUG [HeadlessChrome 63.0.3205 (Mac OS X 10.9.5)]: �[39mReconnected on iUO7OrOOlA2q-QboAAAP.
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 445 of 462 SUCCESS (13 mins 20.993 secs / 3 mins 2.351 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 488 of 496 (skipped 8) SUCCESS (12 mins 31.592 secs / 3 mins 7.926 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 374 of 375 (skipped 1) SUCCESS (9 mins 18.06 secs / 1 min 30.408 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 433 of 455 (skipped 11) SUCCESS (17 mins 5.845 secs / 4 mins 10.347 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 361 of 439 (skipped 23) SUCCESS (20 mins 55.986 secs / 3 mins 59.233 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 0 of 543 SUCCESS (0 secs / 0 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 68 of 345 (skipped 4) SUCCESS (0 secs / 16.488 secs)
[INFO] HeadlessChrome 63.0.3205 (Mac OS X 10.9.5): Executed 0 of 543 SUCCESS (0 secs / 0 secs)

And it looks like it was replaced with a duplicate of the shard with 543 (the above logs show the duplicate entry for that shard.

  • If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

We have not been able to create a minimal demo

  • What is the expected behavior?

When a shard disconnects, it either runs the entire, or restarts from the test that had a problem.

  • Please tell us about your environment:
  • version: 0.3.1
  • Browser: [Headless Chrome ]
  • Language: [TypeScript]
  • Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. stackoverflow, gitter, etc)

I can provide a full log of different runs if it helps. I can also try any suggestions. I understand this is not the best bug submission but it does look like there's a problem when restarting a shard after the browser disconnects.
@juanmendes juanmendes mentioned this issue Aug 12, 2019
Open
@paulcmason
Copy link

Hi - I have setup karma-parallel on our test suite as well. Approx 2700 tests. On the first run the sharding is working great and it has massively decreased our suite run times. However as above we are getting false positives when the runner disconnects from chrome. It will run again but not run the tests in the shard that disconnected. Unfortunately this means we can't use this day to day at the moment.

@lucasklaassen
Copy link

We are running 500 tests and it has reduced our time as well but the browser disconnect issue is preventing us from running this within our CI/CD Pipeline. :(

@nickbullock
Copy link

nickbullock commented Nov 1, 2019 •
edited
Loading

We are running 16k tests and got the same problem as well (concurrency 2-3, chunks 200-500).
Some thoughts: we used karma-parallel for last two years in ci pipeline with our own custom webpack config for huge hybrid angular application. Now we moved to ng-cli build configuration and faced this issue. Maybe ng-cli is the root of evil.
UPD: forget it. The problem was in our docker container. Now everything works as expected.

@juanmendes juanmendes mentioned this issue Jun 17, 2020
Open
3 tasks
@juanmendes
Copy link
Author

There's a PR for this problem for this at #54

@rohitkandhal
Copy link

Is it abandoned?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@juanmendes @rohitkandhal @lucasklaassen @nickbullock @paulcmason