Screenshot action hangs application

[bahlo]

Hey guys, thank you for this library, I'm really loving it (especially in comparison with the PhantomJS backend).

I'm creating screenshots of a page with multiple views with the following code.
The views are changed in the nextView function.

for (var i = 0; i < count; i++) {
  nightmare
    .wait(2000)
    .screenshot(path.resolve(__dirname, i + '.png'))
    .evaluate(nextView)
}

nightmare
  .end()
  .then(function() {
    console.log("Done")
  }).catch(function(err) {
    console.error(err);
  })

This works great on most pages, but it hangs on a specific page (I'm sorry I can't disclose the contents).
Here is a part of the log:

  nightmare:actions .evaluate() fn on the page +2ms
  nightmare:actions .wait() for 2000ms +6ms
  nightmare:actions .screenshot() +2s
  nightmare:log subscribing to browser window frames +1ms
  nightmare:log altering page to force rendering +0ms
  nightmare:log unsubscribing from browser window frames +57ms
  nightmare:actions .screenshot() captured with length 342104 +606ms

  nightmare:actions .evaluate() fn on the page +2ms
  nightmare:actions .wait() for 2000ms +392ms
  nightmare:actions .screenshot() +2s
  nightmare:log subscribing to browser window frames +0ms
  nightmare:log altering page to force rendering +0ms

As you can see, the screenshot before renders nicely, but the last one does not seem to do anything.
The requestFrame function doesn't seem to call its callback.

Do you have any tips for me to debug this? Setting a waitTimeout doesn't help since I'm not waiting.

Update: Okay, I tried again with show: yes and manually triggered a re-render and it worked. Maybe I'll try to click on something to trigger that render manually every time.

Update 2: Since it's only waiting for change after the screenshot action, calling scrollTo or click before doesn't help.

Update 3: I tested it and the callback doesn't get called in the requestFrame function for this screenshot (all the other shots are working fine).

Update 4: I'm just calling the callback directly in requestFrame which is bad, but works for me. Gonna leave this issue open though.

[Mr0grog]

At first blush, I’d guess you are likely running into this issue: #493 as a result of using a synchronous for loop, which will generally not play well with nightmare, where everything is asynchronous.

For a good primer on managing this, you might want to look into any of:
https://github.com/rosshinkley/nightmare-examples/blob/master/docs/common-pitfalls/async-operations-loops.md
#533
#522

Where there are some good explanations. If that doesn’t help, post back here. If possible, too, try and come up with a simplified test case you can share. That makes understanding the details of the problem much easier.

[Mr0grog]

@bahlo I had some time today and attempted to reproduce your issue, but could not. Here’s what I tried:

var colors = [
  '#6699cc',
  '#66cc99',
  '#cc6699'
]

nightmare.goto('about:blank')

for (var i = 0; i < colors.length; i++) {
  nightmare
    .wait(2000)
    .evaluate(function(color) {
      document.body.style.backgroundColor = color;
    }, colors[i])
    .screenshot(path.resolve(__dirname, i + '.png'))
}

nightmare
  .end()
  .then(function() {
    console.log("Done")
  }).catch(function(err) {
    console.error(err);
  })

(Also, upon looking at this again, I realize you haven’t stumbled over the looping problem at all because you don’t call then() in the loop. Sorry for my not-so-careful reading there.)

Anyway, in my very simple and generalized case, everything seems to work. The main reason screenshotting might hang in the way you describe is that when screenshot attempts to trigger a new frame render in chromium, it fails to actually change the render tree in the browser window. The triggerRender function is what does that—mainly by moving around a little 1x1 transparent div on screen. It could be that something in your page is causing that div to wind up offscreen somehow.

The first thing I’d try is setting position: fixed instead of absolute on line 78 of frame-manager.js.

If that doesn’t help, I imagine it’s going to be something particular with the page you are attempting to render (maybe something your nextView function does). If that doesn’t help, maybe you could share the code you’re working with personally via private repo, private gist, or sending a zip?

Otherwise, I’m not sure what to tell you other than to answer any question you have about how and why the screenshotting code works the way it does (which I’m happy to do; it just means I can’t really help you investigate directly).

[bahlo]

@Mr0grog Thank you so much for this. I will try and report back 👍

[bahlo]

Okay, I tried position: fixed, but that didn't fix the problem.

nextView just calls window._someObj.nextView();.

[Mr0grog]

Hmmmm, you might also try setting z-index: 2147483647 and/or transform: translate3d(0, 0, 0).

Knowing that nextView calls window._someObj.nextView(); doesn’t really help—like I said above, screenshotting mainly works by moving around a little 1x1 transparent div on screen to trigger the rendering of a new frame. So we really need to know what in your page might be preventing that div from altering the browser’s model of what’s happening on screen.

[temijun1]

I'm seeing this issue as well. It seems like .screenshot() gets stuck on waiting for "unsubscribing from browser window frames" callback from browser window frame. It unfreezes when I move the mouse around the window.

An example page:
https://www.google.com/#q=britney+spears&start=10

If I try to take the screenshot for this selector img#uid_0, it seems to be stuck the majority of the time. I also tried the style.zIndex approach with no luck.

Good DEBUG
nightmare queueing action "screenshot" +7ms
nightmare running +1ms
nightmare:actions .screenshot() +0ms
nightmare running +1ms
nightmare:log subscribing to browser window frames +1ms
nightmare:log altering page to force rendering +0ms
nightmare:log unsubscribing from browser window frames +60ms
nightmare:actions .screenshot() captured with length 19375 +65ms
Bad DEBUG
nightmare queueing action "screenshot" +8ms
nightmare running +0ms
nightmare:actions .screenshot() +1ms
nightmare running +1ms
nightmare:log subscribing to browser window frames +1ms
nightmare:log altering page to force rendering +2ms
nightmare:log unsubscribing from browser window frames +9s <<<< stuck before this line prints unless I move the mouse.

[bahlo]

I moved the complete application from Docker to a dedicated VM and it solved nearly everything. Translating or setting the z-index didn't help though.

[GautierT]

@bahlo : Hi ! What was your config for Docker ? I have the same issue and i would like to resolve it for docker. Thanks

[bahlo]

@GautierT As I said before, I moved away from Docker since it wouldn't work (I found a bug in chromium with some value not high enough in Docker, but don't know which anymore).

Works fine in the VM though 👍

[GautierT]

@bahlo i tried in VM and have the same problem that in docker. When i launch the same script 4-5 times at the same time some of them finish successfully and some of them crash or timeout or get stuck in nightmare:log altering page to force rendering...

[mackermedia]

@bahlo : I'm running into this same issue. Can you provide an example of how you manually triggered the callback for requestFrame ?

[justinmchase]

Try turning your function and loop to be asynchronous, without knowing more of your code something like this would work:

function renderFrames * (nightmare, count) {
  for (var i = 0; i < count; i++) {
    yield nightmare
      .screenshot(path.resolve(__dirname, i + '.png'))
      .evaluate(nextView)
  }

  yield nightmare.end()
}

renderFrames(nightmare, 100)
  .then(function() {
    console.log("Done")
  }).catch(function(err) {
    console.error(err);
  })

Notice the * in the function declaration, which turns the function into a generator function and the use of the keyword yield. This is es6 generator function syntax which helps you manage promises more easily. You should be able to take each screenshot serially, as opposed to the code snippet you showed above which would render them in parallel if one screenshot took longer than 2 seconds to render, and could result in something pretty bad happening.

You can use babel to down compile es6 to es5 or if you use a recent enough version of node then you can get it out of the box.

[mackermedia]

@bahlo : Nevermind. I found it in your fork ;) https://github.com/bahlo/nightmare/commit/31b17abdf53931cec87cca27b5e2d1f226b611df#diff-bc4eacf94ad1a8e7c9bea8b6b6451251R53

[mackermedia]

Update: The solution that worked for @bahlo (in his fork above) doesn't work for me. In my case, I'm wanting to take a screenshot of a particular rectangle on the page. It appears that manually calling that callback doesn't allow nightmare enough time to move the page to the specified rectangle before taking the screenshot. Still digging for a solution.

[Mr0grog]

Anyone here experiencing timeouts may want to give the #927 a try. There’s also second approach to solving this over on https://github.com/mr0grog/nightmare/tree/screenshot-with-offscreen-rendering that you might give a try.

[mackermedia]

@Mr0grog : thanks a ton for your effort!
It looks like your commit using the debugger highlighting has solved this hanging process issue for me.
I did notice that once in awhile it seems like the rectangle that I'm passing in (for size & location of screenshot to be captured) is sometimes off. Perhaps that's related to the Electron window size bug you mentioned there?

[Mr0grog]

@mackermedia Awesome!

I did notice that once in awhile it seems like the rectangle that I'm passing in (for size & location of screenshot to be captured) is sometimes off. Perhaps that's related to the Electron window size bug you mentioned there?

That sounds like a problem. It shouldn’t be related to the window size bug; that was a problem with my experimental screenshot-with-offscreen-rendering branch, not the PR I submitted. (Sorry if the explanation over there wasn’t clear. I probably shouldn’t have mentioned the other branch at all.)

If you can come up with a way to reproduce it reliably, would you please post it on that PR or as an issue (if the PR gets merged in the mean time)?

[twolfson]

As a heads up on this thread, I recently ran into an issue with #927 where a static page (i.e. one with no page updates) would hang after timing out in series due to us not properly cleaning up booleans/timeouts. I've opened #945 to fix up the issue

[entrptaher]

I fixed the problem using a timeout to move the window frame by few pixels.

nightmare
        // fix browser hanging on screenshot
        .evaluate(()=> setTimeout(()=> window.scrollBy(0,5), 1000))
        .screenshot(screenshotPath)

By the time the window hangs, it'll move and get out the stuck situation.

[Mr0grog]

@casesandberg I guess it’s not clearly linked, but #945 is the PR for this issue. It’s been solved if only someone with privileges would review and merge (or give feedback).

[Mr0grog]

(That is to say, it seems like it would make sense to keep this open since it’s tracking an actual PR)

[casesandberg]

Ah, my bad, I didn't see that it was tracking with a PR.

[JonDotsoy]

when is apply this fix?

[wojons]

I wanted to leave a little note here that I had this same issue while running nightmare in docker and was able to google enough to find that the docker shared memory is 64mb by default and is the cause of this and a few other things changing it to 128mb fixed it but you may need more or less I submitted #1570 to add more info onto the readme.