Skip to content
This repository has been archived by the owner on Oct 9, 2020. It is now read-only.

JS heap out of memory when bundling #832

Open
ZoolWay opened this issue Aug 25, 2017 · 10 comments
Open

JS heap out of memory when bundling #832

ZoolWay opened this issue Aug 25, 2017 · 10 comments

Comments

@ZoolWay
Copy link

ZoolWay commented Aug 25, 2017

Hi! I am trying to bundle the components of my application but SystemJS Builder runs quite long and then I get FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory.

I do not think my project is huge but all js, html, json and css files together make up 612 files (collected by globby).

I am using JSPM 0.17.0-beta.44 with SystemJS 0.20.18, SystemJS Builder 0.16.10, Plugin CSS 0.1.35, Plugin JSON 0.3.0 and Plugin Text 0.0.11. No transpilation by SystemJS involved (my gulp task utilizes typescript and SystemJS just reads the output js files), the JS code is ES6 in AMD format.

Complete error:

<--- Last few GCs --->

[24416:00000227427B79B0]   201807 ms: Mark-sweep 1403.9 (1450.3) -> 1403.9 (1450.3) MB, 3469.0 / 0.0 ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0000016FA9529891 <JS Object>
    1: node [D:\Working\project1\node_modules\babel-traverse\lib\index.js:~94] [pc=0000019B61D44262](this=0000031E22442839 <JS Function traverse (SharedFunctionInfo 000002341E9FB159)>,node=00000300C60A0C41 <a Node with map 000001D216CA4331>,opts=000003089A456D29 <an Object with map 000001D216CA4909>,scope=0000001354D6FC09 <a Scope with map 000001D216CA2C81>,state=...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

The security context is not always the same, here is another variant:

<--- Last few GCs --->

[10620:0000021A7F21F500]   338119 ms: Mark-sweep 1404.0 (1464.2) -> 1404.0 (1464.2) MB, 2894.8 / 0.0 ms  allocation failure GC in old space requested
[10620:0000021A7F21F500]   341011 ms: Mark-sweep 1404.0 (1464.2) -> 1404.0 (1455.7) MB, 2891.8 / 0.0 ms  last resort
[10620:0000021A7F21F500]   343887 ms: Mark-sweep 1404.0 (1455.7) -> 1404.0 (1455.7) MB, 2875.9 / 0.0 ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000002BFF49A9891 <JS Object>
    1: new constructor(aka Binding) [D:\Working\project1\node_modules\babel-traverse\lib\scope\binding.js:~12] [pc=000000451836CA58](this=00000078B2A1EAE1 <a Binding with map 000002226D865BF9>,_ref=00000078B2A1EAA1 <an Object with map 000002226D8666A1>)
    3: registerBinding [D:\Working\project1\node_modules\babel-traverse\lib\scope\i...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
@aluanhaddad
Copy link

aluanhaddad commented Aug 25, 2017

Since you're using JSPM, does this work if you run build commands from the CLI? This might help track down the issue.

$ jspm build app --minify --format iife 

Also, strongly consider configuring TypeScript to output either system, es2015, or esnext modules rather than AMD modules.

Note that when using JSPM, rather than consuming the SystemJSBuilder API directly, you should instead acquire a pre-configured builder as in

var Builder = require('jspm').Builder;

var builder = new Builder();

This results in a SystemJSBuilder instance pre-configured with JSPM config that can be further customized as needed.

@ZoolWay
Copy link
Author

ZoolWay commented Aug 28, 2017

Hm, I have not tried using JSPM CLI to build the app but in very short, it does not work.

Changing the module format to es2015 or system did not change anything, still getting JavaScript heap out of memory, same goes with obtaining the builder from jspm insead of using it directly.

I am preparting a minimum repository to share.

@ZoolWay
Copy link
Author

ZoolWay commented Aug 28, 2017

Okay, I found out a bit more. I am now trying to bundle using JSPM like this:

jspm build dist/main.js app-bundle.js

When I removed the import for Kendo UI (import 'kendo.all.min';) from my files bundling started to work without heap out of memory.

I tried to reproduce this in a minimal example using the free OpenSource variant of Kendo UI but there the issue does not seem to happen.

Then I tried to exclude Kendo UI from the main bundle but that does not work either.

jspm build dist/main.js - kendo.all.min app-bundle.js

It also produces heap out of memory. Also tried kendo.all.min.js.

Any ideas how to solve this?

@aluanhaddad
Copy link

What happens if you run

$ jspm build kendo.all.min.js

?

@ZoolWay
Copy link
Author

ZoolWay commented Aug 28, 2017

Another update: Bundling with Kendo UI works if I use --skip-source-maps...

@ZoolWay
Copy link
Author

ZoolWay commented Aug 28, 2017

Running jspm build js/kendo/kendo.all.min.js (path is provided in map parameter otherwise) I still get heap out of memory. Adding --skip-source-maps works:

     Creating the single-file build for js/kendo/kendo.all.min.js...

       js/kendo/kendo.all.min.js

ok   Built into test.js, unminified as umd.

@ZoolWay
Copy link
Author

ZoolWay commented Sep 1, 2017

I must add that deactivating the sourcemaps looks like only a temporary workaround. After the project got a little bigger I started to run into the heap out of memory issue again, especially when multiple gulp tasks are chained. I can work around it by increasing the NodeJS memory limit (i.e. --max_old_space_size=4096 gives it 4GB instead of 1.76GB) but I guess bundling 200MB if JS files should work using 1GB of RAM? Maybe there is some kind of memory leak?

@aluanhaddad
Copy link

@ZoolWay thank you for keeping this up to date and reporting your findings.

If you are running gulp tasks in a single process such that multiple bundling operations are potentially interleaved, you may indeed run into issues.

@ZoolWay
Copy link
Author

ZoolWay commented Sep 6, 2017

What is the recommended process around this (although giving it just 4GB RAM might be a normal solution too)? Can in series running gulp tasks be forked into own processes?

@aluanhaddad
Copy link

aluanhaddad commented Sep 6, 2017

Yes you can fork them, but running them sequentially should also do the trick. Consider using async/await in your gulpfile.js to orchestrate sequential bundling tasks in a simple manner or just using promises. The heapsize idea is also good

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants