Missing ndarray dependencies using plotly.js v2.5.0, npm v6 and react-plotly.js v1

[dmbarry86]

When reviewing our weekly Dependabot updates I came across an issue with our migration from Plotly 2.4.1 to 2.5.0. We now get the error shown below when trying to install dependencies. The issue was shown in our CI pipeline which runs npm ci.

I've attached our package files from 2.4.1 which work fine and the updated ones from 2.5.0 which break. You can run npm ci with the 2.5.0 files to reproduce.

error.zip

[archmoj]

What is your npm version?

[dmbarry86]

What is your npm version?

Locally it's 6.14.12. Our CI pipeline (GitHub Actions) would also be 6.x as we run on Node 14.x which I believe comes with 6.x npm as default.

[archmoj]

Upgrading to npm7 and node16 could help fix the warning. See #5922 and #5921 (comment).

[dmbarry86]

Do you no longer support node 14 then? Note that we get an error not a warning.

[archmoj]

node12 and node14 still worked for us with npm ci. The was a warning with npm i which upgrading to npm7 fixed it.
What is your package.json look like?

[dmbarry86]

What is your package.json look like?

I attached this above to the original report.

[archmoj]

Could you please try using ploly.js-dist instead?

[dmbarry86]

Could you please try using ploly.js-dist instead?

This appears to fix the issue. Are there any downsides to switching over to this package?

Are you able to investigate what is causing the original problem? I see some of our other dependencies make use of the original ndarray rather than the plotly forked version. Do you think this could be the cause?

[archmoj]

Could you please try using ploly.js-dist instead?

This appears to fix the issue. Are there any downsides to switching over to this package?

There shouldn't be any downside. It should also download and build faster. In addition you may try the minified package plotly.js-dist-min too.

[archmoj]

Are you able to investigate what is causing the original problem? I see some of our other dependencies make use of the original ndarray rather than the plotly forked version. Do you think this could be the cause?

Which other package require ndarray?
Could you try npm i ndarray --save ?

[dmbarry86]

FYI, switching to Node 16.x and NPM 7.x solves the error. We're not quite ready to switch to Node 16 yet in our project but we will keep in mind. For now maybe we just stay with Plotly 2.4.1 until we are ready to migrate.

[dmbarry86]

Which other package require ndarray?

Apologies, I was wrong there. I was manually inspecting the package-lock.json and thought another dependency relied on it. It appears to just be plotly dependencies.

Here's the output from npm ls ndarray in our working 2.4.1

Here's the output from npm ls ndarray in our broken 2.5.0

[archmoj]

npm uninstall plotly.js
npm install plotly.js-dist --save

Should fix the issue for you.

[dmbarry86]

If I uninstall plotly.js and then reinstall plot.js I now get a new error when running npm ci. The new error this time is "No matching version found for [email protected].". This is still with npm 6.x by the way as we have shown that 7.x already works fine.

After running npm ci

After running npm install

[archmoj]

Not reinstalling plotly.js,

npm install plotly.js-dist --save

is installing ploltly.js-dist

[dmbarry86]

Not reinstalling plotly.js,

npm install plotly.js-dist --save

is installing ploltly.js-dist

Yes I'm aware. We've already proved above that plotly.js-dist works just fine. I'm trying to help debug the issue with plotly.js and npm ci.

[archmoj]

Not reinstalling plotly.js,
npm install plotly.js-dist --save
is installing ploltly.js-dist

Yes I'm aware. We've already proved above that plotly.js-dist works just fine. I'm trying to help debug the issue with plotly.js and npm ci.

What's the reasoning behind not using ploly.js-dist then?

[dmbarry86]

What's the reasoning behind not using ploly.js-dist then?

We're actually using react-plotly in our React project. Switching from plotly.js to plotly.js-dist resolves the npm ci issue but we then get errors when importing from react-plotly.js. Is there a way to use react-plotly with plotly.js-dist?

[archmoj]

What's the reasoning behind not using ploly.js-dist then?

We're actually using react-plotly in our React project. Switching from plotly.js to plotly.js-dist resolves the npm ci issue but we then get errors when importing from react-plotly.js. Is there a way to use react-plotly with plotly.js-dist?

I see. Please open an issue or pull request on react-plotly.js and paste the link here.

[dmbarry86]

Do I take it from your comments so far that you don't plan to try and debug and fix the issue with plotly.js 2.5.0, npm 6.x and npm ci? If that's the case then we just stick with version 2.4.1 for now.

[archmoj]

Do I take it from your comments so far that you don't plan to try and debug and fix the issue with plotly.js 2.5.0, npm 6.x and npm ci? If that's the case then we just stick with version 2.4.1 for now.

Yes please you could continue using 2.4.1 (or bump to 2.4.2) until the react-plotly.js issue addressed or as discussed upgrade npm to v7.

[dmbarry86]

FWIW I can reproduce the error in a completely fresh project that has just plotly.js version 2.5.0 installed. Personally I would consider this a breaking change (or a bug) as you now no longer support npm 6.x and npm ci.

How to reproduce:

• Run on npm 6.x (the default with node 14)
• npm init newProject
• npm install plotly.js --save
• npm ci
• observe error regarding no matching version found for xxx dependency

[archmoj]

FWIW I can reproduce the error in a completely fresh project that has just plotly.js version 2.5.0 installed. Personally I would consider this a breaking change (or a bug) as you now no longer support npm 6.x and npm ci.

How to reproduce:

• Run on npm 6.x (the default with node 14)
• npm init newProject
• npm install plotly.js --save
• npm ci
• observe error regarding no matching version found for xxx dependency

Since v2 importing ploly.js-dist is the preferred and documented way of requiring, this may not be considered as a breaking change IMHO.

[dmbarry86]

this may not be considered as a breaking change IMHO.

Speaking as somebody who now has broken code, I beg to differ. The documentation for react-plotly clearly states to still install plotly.js. We clearly disagree here and that's fine. We can just stick to version 2.4.1 / 2.4.2 for our use case.

[alexcjohnson]

@archmoj I understand your point but I think we should still make an effort to figure out what's going on here. If it's just a matter of npm6 not recognizing the link format we're using it'll be a simple fix. If it requires us to give all these forked packages different names (@plotly/ndarray etc) then that may be a bigger lift but perhaps a cleaner representation of what we're doing anyway.

BTW one thing I notice, not sure if this has anything to do with the error, but the repository field in our ndarray fork points to the original repo, not our own.

[archmoj]

@dmbarry86 FYI - I opened plotly/react-plotly.js#260.
Any chance you could try the commit with npm=6?

[archmoj]

@dmbarry86 FYI - I opened plotly/react-plotly.js#260.
Any chance you could try the commit with npm=6?

by installing

"react-plotly.js": "git://github.com/plotly/react-plotly.js#40dfac18bd98d64b013bd726f5c212aef0d9e4d1"

in package.json.

[dmbarry86]

@dmbarry86 FYI - I opened plotly/react-plotly.js#260.
Any chance you could try the commit with npm=6?

Doesn't seem to be working for me. I get an error in our components that use react-plotly.js

Package.json is now as follows

I checked node_modules folder and it has successfully pulled down the version from your branch so I'm not sure exactly what is wrong here.

[dmbarry86]

I checked node_modules folder and it has successfully pulled down the version from your branch so I'm not sure exactly what is wrong here.

Oh actually it has only pulled down the root project files. There isn't actually any source code in the folder.

[archmoj]

@dmbarry86 thanks for testing.
Wondering what happens if plotly.js-dist is moved to dependencies like plotly/react-plotly.js@4ecac06

Could you possibly try

"react-plotly.js": "git://github.com/plotly/react-plotly.js#4ecac06b1af7152f6cca7d1d0c577b9c594c9b9b"

as well?

[dmbarry86]

@archmoj still the same problem with no source code inside node_modules\react-plotly.js folder.

[dmbarry86]

@archmoj I wonder if the problem is because the npmignore file in the repository is ignoring src folder. Maybe you could temporarily remove this and create new commit so we can test. https://github.com/plotly/react-plotly.js/blob/4ecac06b1af7152f6cca7d1d0c577b9c594c9b9b/.npmignore#L2

[archmoj]

Unfortunately this week we may not be able to allocate more time investigating this problem.
But please let us know if you found a solution for this namely at the react-plotly.js level.

[dmbarry86]

But please let us know if you found a solution for this

See my comment above regarding npmignore. I think for me to be able to npm install from GitHub and test your PR you will need to (temporarily) remove src from npmignore.

[archmoj]

But please let us know if you found a solution for this

See my comment above regarding npmignore. I think for me to be able to npm install from GitHub and test your PR you will need to (temporarily) remove src from npmignore.

Alternatively

1. Please fork https://github.com/plotly/react-plotly.js
2. Clone your repository
3. Switch to a new branch on top of master or the open PR or dev-branch
4. Commit your changes e.g. tweaking gitignore etc.
5. Push to your fork
6. Now you could use the hash of your last commit following this link react-plotly.js": "git://github.com/dmbarry86/react-plotly.js#

[dmbarry86]

@archmoj I've forked my own version of your branch and removed src from npmignore. I can now confirm that your PR is working as expected with npm version 6.

[dmbarry86]

This is a good article FWIW. It recommends the use of files in package.json rather than npmignore. https://medium.com/@jdxcode/for-the-love-of-god-dont-use-npmignore-f93c08909d8d

[archmoj]

@dmbarry86 Fantastic, please open a PR plotly/react-plotly.js@switch-to-plotly.js-dist...dmbarry86:switch-to-plotly.js-dist

[dmbarry86]

@archmoj It doesn't need a PR from me. That removal of src was just temporary to allow local testing. Your own PR plotly/react-plotly.js#260 is the one that should be merged. You should not merge the removal of src.

[archmoj]

@archmoj I've forked my own version of your branch and removed src from npmignore. I can now confirm that your PR is working as expected with npm version 6.

Thanks for testing. Please leave a comment on this pull plotly/react-plotly.js#260.