Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(cli)!: ensure env vars are set before migrating #1222

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

brenoepics
Copy link
Collaborator

fixes #1220
pg library defaults host to 'localhost' when PGHOST environment variable is missing, potentially leading to unintended database connections, To mitigate this, we now perform an explicit check to ensure the PGHOST environment variable is set.

Default Config
  user?: string, // default process.env.PGUSER || process.env.USER
  password?: string or function, //default process.env.PGPASSWORD
  host?: string, // default process.env.PGHOST
  port?: number, // default process.env.PGPORT
  database?: string, // default process.env.PGDATABASE || user
  connectionString?: string, // e.g. postgres://user:password@host:5432/database
  ssl?: any, // passed directly to node.TLSSocket, supports all tls.connect options
  types?: any, // custom type parsers
  statement_timeout?: number, // number of milliseconds before a statement in query will time out, default is no timeout
  query_timeout?: number, // number of milliseconds before a query call will timeout, default is no timeout
  lock_timeout?: number, // number of milliseconds a query is allowed to be en lock state before it's cancelled due to lock timeout
  application_name?: string, // The name of the application that created this Client instance
  connectionTimeoutMillis?: number, // number of milliseconds to wait for connection, default is no timeout
  idle_in_transaction_session_timeout?: number // number of milliseconds before terminating any session with an open idle transaction, default is no timeout

Reference:

@brenoepics brenoepics self-assigned this Jul 7, 2024
Copy link

github-actions bot commented Jul 7, 2024

Coverage Report

Status Category Percentage Covered / Total
🟢 Lines 95.59% (🎯 90%)
🟰 ±0%
6752 / 7063
🟢 Statements 95.59% (🎯 90%)
🟰 ±0%
6752 / 7063
🟢 Functions 95.47% (🎯 90%)
🟰 ±0%
253 / 265
🟢 Branches 89.96% (🎯 85%)
🟰 ±0%
807 / 897
File Coverage
File Stmts % Branch % Funcs % Lines Uncovered Lines
Unchanged Files
src/db.ts 88.02% 88.46% 87.5% 88.02% 67-69, 99, 102-114, 154-156
src/index.ts 100% 100% 100% 100%
src/migration.ts 75.4% 78.57% 61.53% 75.4% 64, 67-69, 71-83, 112-118, 123-159, 216-217, 242-245, 253-254, 271-274, 303-304
src/migrationBuilder.ts 97.56% 92.85% 88.88% 97.56% 354-358, 497-502, 526-527
src/runner.ts 74.3% 58.92% 80% 74.3% 42, 70-71, 80-81, 84-92, 130-131, 173-176, 185-189, 202, 206-219, 238, 240-246, 249, 262, 274-287, 290-293, 303-304, 313-315, 324, 326-335, 347-354
src/sqlMigration.ts 90.19% 100% 80% 90.19% 45-49
src/types.ts 100% 100% 100% 100%
src/operations/sql.ts 100% 100% 100% 100%
src/operations/casts/createCast.ts 100% 100% 100% 100%
src/operations/casts/dropCast.ts 100% 100% 100% 100%
src/operations/casts/index.ts 100% 100% 100% 100%
src/operations/domains/alterDomain.ts 100% 100% 100% 100%
src/operations/domains/createDomain.ts 100% 100% 100% 100%
src/operations/domains/dropDomain.ts 100% 100% 100% 100%
src/operations/domains/index.ts 100% 100% 100% 100%
src/operations/domains/renameDomain.ts 100% 100% 100% 100%
src/operations/domains/shared.ts 100% 100% 100% 100%
src/operations/extensions/createExtension.ts 100% 100% 100% 100%
src/operations/extensions/dropExtension.ts 100% 100% 100% 100%
src/operations/extensions/index.ts 100% 100% 100% 100%
src/operations/extensions/shared.ts 100% 100% 100% 100%
src/operations/functions/createFunction.ts 100% 100% 100% 100%
src/operations/functions/dropFunction.ts 100% 100% 100% 100%
src/operations/functions/index.ts 100% 100% 100% 100%
src/operations/functions/renameFunction.ts 100% 100% 100% 100%
src/operations/functions/shared.ts 100% 100% 100% 100%
src/operations/grants/grantOnSchemas.ts 100% 100% 100% 100%
src/operations/grants/grantOnTables.ts 100% 100% 100% 100%
src/operations/grants/grantRoles.ts 100% 100% 100% 100%
src/operations/grants/index.ts 100% 100% 100% 100%
src/operations/grants/revokeOnSchemas.ts 100% 100% 100% 100%
src/operations/grants/revokeOnTables.ts 100% 100% 100% 100%
src/operations/grants/revokeRoles.ts 100% 100% 100% 100%
src/operations/grants/shared.ts 98.63% 85.71% 100% 98.63% 71
src/operations/indexes/createIndex.ts 98.03% 95.23% 100% 98.03% 74-75
src/operations/indexes/dropIndex.ts 100% 100% 100% 100%
src/operations/indexes/index.ts 100% 100% 100% 100%
src/operations/indexes/shared.ts 91.17% 86.95% 100% 91.17% 22, 32-35, 47
src/operations/materializedViews/alterMaterializedView.ts 100% 100% 100% 100%
src/operations/materializedViews/createMaterializedView.ts 100% 100% 100% 100%
src/operations/materializedViews/dropMaterializedView.ts 100% 100% 100% 100%
src/operations/materializedViews/index.ts 100% 100% 100% 100%
src/operations/materializedViews/refreshMaterializedView.ts 100% 100% 100% 100%
src/operations/materializedViews/renameMaterializedView.ts 100% 100% 100% 100%
src/operations/materializedViews/renameMaterializedViewColumn.ts 100% 100% 100% 100%
src/operations/materializedViews/shared.ts 100% 87.5% 100% 100%
src/operations/operators/addToOperatorFamily.ts 100% 100% 100% 100%
src/operations/operators/createOperator.ts 100% 100% 100% 100%
src/operations/operators/createOperatorClass.ts 100% 83.33% 100% 100%
src/operations/operators/createOperatorFamily.ts 100% 100% 100% 100%
src/operations/operators/dropOperator.ts 100% 100% 100% 100%
src/operations/operators/dropOperatorClass.ts 100% 100% 100% 100%
src/operations/operators/dropOperatorFamily.ts 100% 100% 100% 100%
src/operations/operators/index.ts 100% 100% 100% 100%
src/operations/operators/removeFromOperatorFamily.ts 100% 100% 100% 100%
src/operations/operators/renameOperatorClass.ts 100% 100% 100% 100%
src/operations/operators/renameOperatorFamily.ts 100% 100% 100% 100%
src/operations/operators/shared.ts 90% 75% 100% 90% 24-25, 37-38
src/operations/policies/alterPolicy.ts 100% 100% 100% 100%
src/operations/policies/createPolicy.ts 100% 100% 100% 100%
src/operations/policies/dropPolicy.ts 100% 100% 100% 100%
src/operations/policies/index.ts 100% 100% 100% 100%
src/operations/policies/renamePolicy.ts 100% 100% 100% 100%
src/operations/policies/shared.ts 100% 100% 100% 100%
src/operations/roles/alterRole.ts 100% 100% 100% 100%
src/operations/roles/createRole.ts 100% 75% 100% 100%
src/operations/roles/dropRole.ts 100% 100% 100% 100%
src/operations/roles/index.ts 100% 100% 100% 100%
src/operations/roles/renameRole.ts 100% 100% 100% 100%
src/operations/roles/shared.ts 98.97% 76.92% 100% 98.97% 78
src/operations/schemas/createSchema.ts 100% 100% 100% 100%
src/operations/schemas/dropSchema.ts 100% 100% 100% 100%
src/operations/schemas/index.ts 100% 100% 100% 100%
src/operations/schemas/renameSchema.ts 100% 100% 100% 100%
src/operations/sequences/alterSequence.ts 96.87% 75% 100% 96.87% 23
src/operations/sequences/createSequence.ts 100% 100% 100% 100%
src/operations/sequences/dropSequence.ts 100% 100% 100% 100%
src/operations/sequences/index.ts 100% 100% 100% 100%
src/operations/sequences/renameSequence.ts 100% 100% 100% 100%
src/operations/sequences/shared.ts 87.67% 68.75% 100% 87.67% 41, 43-44, 49-50, 63-64, 69-70
src/operations/tables/addColumns.ts 100% 80% 100% 100%
src/operations/tables/addConstraint.ts 100% 100% 100% 100%
src/operations/tables/alterColumn.ts 92.5% 76.47% 100% 92.5% 75, 82-89
src/operations/tables/alterTable.ts 100% 100% 100% 100%
src/operations/tables/createTable.ts 90.47% 66.66% 100% 90.47% 44-48, 65, 75-76
src/operations/tables/dropColumns.ts 100% 100% 100% 100%
src/operations/tables/dropConstraint.ts 100% 100% 100% 100%
src/operations/tables/dropTable.ts 100% 100% 100% 100%
src/operations/tables/index.ts 100% 100% 100% 100%
src/operations/tables/renameColumn.ts 100% 100% 100% 100%
src/operations/tables/renameConstraint.ts 100% 100% 100% 100%
src/operations/tables/renameTable.ts 100% 100% 100% 100%
src/operations/tables/shared.ts 88.47% 78.46% 80% 88.47% 137-138, 141-142, 195-199, 224, 228-229, 248-249, 274-275, 278-285, 288-289, 426-451
src/operations/triggers/createTrigger.ts 90.83% 68.18% 100% 90.83% 53-54, 66-67, 70-71, 74-77, 113
src/operations/triggers/dropTrigger.ts 100% 100% 100% 100%
src/operations/triggers/index.ts 100% 100% 100% 100%
src/operations/triggers/renameTrigger.ts 100% 100% 100% 100%
src/operations/triggers/shared.ts 100% 100% 100% 100%
src/operations/types/addTypeAttribute.ts 100% 100% 100% 100%
src/operations/types/addTypeValue.ts 100% 100% 100% 100%
src/operations/types/createType.ts 100% 100% 100% 100%
src/operations/types/dropType.ts 100% 100% 100% 100%
src/operations/types/dropTypeAttribute.ts 100% 100% 100% 100%
src/operations/types/index.ts 100% 100% 100% 100%
src/operations/types/renameType.ts 100% 100% 100% 100%
src/operations/types/renameTypeAttribute.ts 100% 100% 100% 100%
src/operations/types/renameTypeValue.ts 100% 100% 100% 100%
src/operations/types/setTypeAttribute.ts 100% 100% 100% 100%
src/operations/views/alterView.ts 100% 100% 100% 100%
src/operations/views/alterViewColumn.ts 100% 100% 100% 100%
src/operations/views/createView.ts 100% 100% 100% 100%
src/operations/views/dropView.ts 100% 100% 100% 100%
src/operations/views/index.ts 100% 100% 100% 100%
src/operations/views/renameView.ts 100% 100% 100% 100%
src/operations/views/shared.ts 100% 66.66% 100% 100%
src/utils/PgLiteral.ts 96.49% 100% 75% 96.49% 37-38
src/utils/StringIdGenerator.ts 100% 100% 100% 100%
src/utils/createSchemalize.ts 100% 100% 100% 100%
src/utils/createTransformer.ts 100% 100% 100% 100%
src/utils/decamelize.ts 100% 100% 100% 100%
src/utils/escapeValue.ts 100% 100% 100% 100%
src/utils/formatLines.ts 100% 100% 100% 100%
src/utils/formatParams.ts 100% 100% 100% 100%
src/utils/getMigrationTableSchema.ts 100% 100% 100% 100%
src/utils/getSchemas.ts 100% 100% 100% 100%
src/utils/identity.ts 100% 100% 100% 100%
src/utils/index.ts 100% 100% 100% 100%
src/utils/intersection.ts 100% 100% 100% 100%
src/utils/makeComment.ts 100% 100% 100% 100%
src/utils/quote.ts 100% 100% 100% 100%
src/utils/toArray.ts 100% 100% 100% 100%
src/utils/types.ts 100% 100% 100% 100%
Generated in workflow #927

@Shinigami92
Copy link
Collaborator

Should this be seen as a breaking change? 🤔

@Shinigami92 Shinigami92 added c: bug Something isn't working p: 1-normal Nothing urgent labels Jul 7, 2024
@brenoepics
Copy link
Collaborator Author

Kind of yes, because it would prevent the migration from running w/o specifying a URL or env vars. The previous check is ignored since pg defaults to the PC's username if the env var is not set, same to database

@Shinigami92
Copy link
Collaborator

Kind of yes, because it would prevent the migration from running w/o specifying a URL or env vars. The previous check is ignored since pg defaults to the PC's username if the env var is not set, same to database

Okay, I will think about it
On the one side, as you described, this is a valuable addition. On the other side, I wont like to annoy the consumers by a minor breaking change, without letting them known.
I'm anyways thinking about to move forward with a v8 and some other breaking changes as well as a last major including cjs support but internally build with esm.
But that would take a bit of time from my side.

Would it be okay for you if this fix wont immediately merged because of that?
But I'm also unsure when I will find more dedicated time to address the v8 major changes.

@Shinigami92 Shinigami92 added this to the v8.0 milestone Jul 7, 2024
@Shinigami92 Shinigami92 added the breaking change Cannot be merged when next version is not a major release label Jul 7, 2024
@Shinigami92 Shinigami92 changed the title fix: ensure env vars are set before migrating fix!: ensure env vars are set before migrating Jul 7, 2024
@Shinigami92 Shinigami92 changed the title fix!: ensure env vars are set before migrating fix(cli)!: ensure env vars are set before migrating Jul 7, 2024
@brenoepics
Copy link
Collaborator Author

Alright, I'll also take a look at some other open issues

@Shinigami92 Shinigami92 added the do NOT merge yet Do not merge this PR into the target branch yet label Jul 7, 2024
if (!cp.host && !cp.port && !cp.database) {
if (
!process.env[argv[databaseUrlVarArg]] &&
(!process.env.PGHOST || !cp.user || !cp.database)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question (if-minor): you removed the check for !cp.host && !cp.port, but added !cp.user and used || (or). Can you explain a bit why you did it like that?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cp.host and cp.port are unlikely to be undefined since we read pg defaults, which set host to 'localhost' and port to '5432' if missing.
When running pg locally without any env vars, node-pg-migrate would run but give an error due to an undefined password string/function. Adding a check for the pw alone isn't enough because it would still cause a connection error if pg isn't running locally (also when pw is set but others vars aren't).
so, if there's no dburl, I check if any required variable or process.env.PGHOST is missing and stop it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change Cannot be merged when next version is not a major release c: bug Something isn't working do NOT merge yet Do not merge this PR into the target branch yet p: 1-normal Nothing urgent
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The lack of DATABASE_URL doesn't stop migrating
2 participants