Issues with sequelize

[Tanner-Scadden]

I love your guys package and really enjoy using netlify and nextjs with an api.

I've ran into an issue where I can't build using this package when I also am using sequelize. I am wondering if I am able to get some help or possibly contribute to this package to get it working.

ModuleNotFoundError: Module not found: Error: Can't resolve 'pg-native' in '/opt/build/repo/node_modules/pg/lib/native'

I've tried altering my next.config.js to try and work around this errors. It would build locally and on netlify, but fail when creating the netlify functions with the same errors.

module.exports = { // Target must be serverless target: 'serverless', webpack: (config, { webpack }) => { // Note: we provide webpack above so you should not require it // Perform customizations to webpack config config.plugins.push(new webpack.IgnorePlugin(/\/__tests__\//)); config.externals = [ ...config.externals, 'pg', 'sqlite3', 'tedious', 'pg-hstore', ]; // Important: return the modified config return config; }, };

{ "name": "with-typescript", "version": "1.0.0", "scripts": { "dev": "next", "build": "next build", "start": "next start", "postbuild": "next-on-netlify", "type-check": "tsc" }, "dependencies": { "@material-ui/core": "^4.11.0", "@material-ui/icons": "^4.9.1", "@types/lodash": "^4.14.159", "@types/nodemailer": "^6.4.0", "@types/sequelize": "^4.28.9", "axios": "^0.19.2", "babel-plugin-react-intl": "^8.1.1", "lodash": "^4.17.19", "netlify-plugin-cache-nextjs": "^1.5.0", "next": "latest", "next-on-netlify": "^2.3.2", "nodemailer": "^6.4.11", "notistack": "^0.9.17", "pg": "^8.3.0", "pg-hstore": "^2.3.3", "react": "^16.13.1", "react-dom": "^16.12.0", "react-intl-hooks": "^1.0.11", "sequelize": "^6.3.4" }, "devDependencies": { "@types/node": "^12.12.21", "@types/pg": "^7.14.4", "@types/react": "^16.9.46", "@types/react-dom": "^16.9.8", "typescript": "^3.9.7" }, "license": "ISC" }

I've tried to create the sequelize connection multiple different ways but build is always failing. Here is what I have currently:

const sequelize = new Sequelize({ database, username, password, port, host: process.env.HOST, protocol: 'postgres', dialect: 'postgres', dialectModule: require('pg'), dialectOptions: { ssl: { rejectUnauthorized: false, }, }, logging: false, });

Any help on this issue would be incredible. Thank you!

[FinnWoelm]

Hi @Tanner-Scadden,

Happy to hear that you are enjoying Netlify and next-on-netlify! 😊

I have a suspicion that this is an issue with pg-native not getting bundled into the functions. Is there any chance I can have a look at your code on GitHub or could you create a minimal repo to reproduce the issue? Then I am more than happy to get to the bottom of this and find out how we can fix it!

Let's figure this out! 🚀 💪

[Tanner-Scadden]

Thank you for getting back so fast!

https://github.com/Tanner-Scadden/broken-sequelize-netlify

Here is a repo I made to reproduce the issue. The issue didn't happen in the build until I added the next.config.js file and the build was targeted towards serverless. I've been able to deploy this to vercel and have everything working before, so I know its possible. I'd love to get it working correctly on netlify!

Adding screenshot of build:

[laugharn]

Have you tried specifying the dialectModule in your sequelize config? Vercel suggests that in their NCC documentation: https://github.com/vercel/ncc/blob/31b1108424d460f6744045baaedcd3851e310697/package-support.md#L7 and that might be relevant here.

[Tanner-Scadden]

I have. That is how I was able to get it to work on vercel, but that method doesn't seem effective with netlify.

Pushed up dialectModule being added and still getting the build errors.

[FinnWoelm]

Hi @Tanner-Scadden,

Thank you for creating a repo to reproduce the issue. That is extremely helpful!

Let's start with the good news first: I got your repo up and running 🎉

Demo: https://next-on-netlify-sequelize.netlify.app/
Code: https://github.com/FinnWoelm/broken-sequelize-netlify/

Let's break down what's going on. First, this issue is actually related to NextJS and sequelize rather than next-on-netlify. The error(s) you shared above relate to the build phase of NextJS — all of that happens before next-on-netlify even starts doing anything. Nevertheless, I wanted to get to the bottom of this! 😊

I believe in your demo repo you forgot to run npm install pg-hstore. Once I did that, I was able to reproduce the original issue you shared:

info  - Creating an optimized production build  
Failed to compile.

ModuleNotFoundError: Module not found: Error: Can't resolve 'pg-native' in '/home/fwoelm/Projects/broken-sequelize-netlify/node_modules/pg/lib/native'


> Build error occurred

There is a long-running thread in the sequelize repo about this issue: sequelize/sequelize#3781
The thread is now five years old and has 62 comments.

It offers various workarounds for this issue. The relevant one for NextJS is the one that relates to webpack:

It requires ./native file which is depending on pg-native. So to pack it with webpack we need to ignore this import. To do it just use webpack.IgnorePlugin:

new webpack.IgnorePlugin(/.\/native/),

After this addition, everything works fine.

We can configure webpack in our next.config.js as described on this NextJS page:

// next.config.js

module.exports = {
  // Target must be serverless
  target: 'serverless',
  webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
    // Note: webpack is provided, so we do not need to `require` it

    // Do not include .native which tries to load pg-native
    // See: https://github.com/sequelize/sequelize/issues/3781#issuecomment-537979334
    config.plugins.push(new webpack.IgnorePlugin(/^pg-native$/))

    // Important: return the modified config
    return config
  },
};

Now, we push and deploy! And it works! 🎉

Of course, this requires that you do not actually need pg-native in your application. In the demo, this is not the case. Perhaps you can test it in your actual repo and see if everything works.

I looked into actually installing pg-native by running npm install pg-native. But this is currently not supported on Netlify and will fail in the build pipeline. See https://community.netlify.com/t/unable-to-publish-function-using-postgres/12293/3 and netlify/build-image#393

I opened a PR into your repo where you can see the code changes (package.json, package-lock.json, next.config.js, netlify.toml): Tanner-Scadden/broken-sequelize-netlify#1

Please let me know if this helps!

PS: Edited thanks to comment by @s-kris

[FinnWoelm]

PS: Netlify is adopting next-on-netlify (:tada:), so the repo will change ownership a few times. It should not affect you in any way, but I wanted to give you a heads up! You can read more in the README: https://github.com/FinnWoelm/next-on-netlify#-netlify-is-adopting-next-on-netlify-

[Tanner-Scadden]

I made these changes to the project that I was working and it works! Thank you guys for all your help. I am excited for the future of netlify and nextjs! Glad the package will get supported more (:

[Tanner-Scadden]

I couldn't develop on local when I changed my webpack to be like that, but I did change it to this and everything is working smoothly now! Thank you both again!

`module.exports = {
// Target must be serverless
target: 'serverless',
webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
// Note: webpack is provided, so we do not need to require it

// Do not include .native which tries to load pg-native
// See: https://github.com/sequelize/sequelize/issues/3781#issuecomment-537979334
config.plugins.push(
  new webpack.IgnorePlugin(/mariasql/, /\/knex\//),
  new webpack.IgnorePlugin(/mssql/, /\/knex\//),
  new webpack.IgnorePlugin(/mysql/, /\/knex\//),
  new webpack.IgnorePlugin(/mysql2/, /\/knex\//),
  new webpack.IgnorePlugin(/oracle/, /\/knex\//),
  new webpack.IgnorePlugin(/oracledb/, /\/knex\//),
  new webpack.IgnorePlugin(/pg-query-stream/, /\/knex\//),
  new webpack.IgnorePlugin(/sqlite3/, /\/knex\//),
  new webpack.IgnorePlugin(/strong-oracle/, /\/knex\//),
  new webpack.IgnorePlugin(/pg-native/, /\/pg\//)
);

// Important: return the modified config
return config;

},
};`

[FinnWoelm]

Hi @Tanner-Scadden,

That is great to hear! You could also consider the following to exclude pg-native in development only:

module.exports = {
  // Target must be serverless
  target: 'serverless',
  webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
    // Note: webpack is provided, so we do not need to `require` it

    // In production, do not include .native which tries to load pg-native
    // See: https://github.com/sequelize/sequelize/issues/3781#issuecomment-537979334
    if(!dev) {
      config.plugins.push(new webpack.IgnorePlugin(/^pg-native$/))
    }

    // Important: return the modified config
    return config
  },
};

In any case, I'm super glad you got it to work! 🙌 Can we close this issue for now? 😊

Happy hacking! 🔥 And please let me know if you build something awesome with next-on-netlify because we would love to feature it in the README!

PS: Edited thanks to comment by @s-kris

[FinnWoelm]

Closing this for now! Happy to reopen anytime if need be :)

[s-kris]

config.plugins.push(new webpack.IgnorePlugin(/.\/native/))

Above regex excludes 'Nativescript' module for typescript based code.

Here's a fix:

config.plugins.push(new webpack.IgnorePlugin(/^pg-native$/))