Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(db): use flyway for migrations #590

Merged
merged 3 commits into from
Aug 21, 2024

Conversation

andrewazores
Copy link
Member

@andrewazores andrewazores commented Jul 31, 2024

Welcome to Cryostat! 👋

Before contributing, make sure you have:

  • Read the contributing guidelines
  • Linked a relevant issue which this PR resolves
  • Linked any other relevant issues, PR's, or documentation, if any
  • Resolved all conflicts, if any
  • Rebased your branch PR on top of the latest upstream main branch
  • Attached at least one of the following labels to the PR: [chore, ci, docs, feat, fix, test]
  • Signed all commits using a GPG signature

To recreate commits with GPG signature git fetch upstream && git rebase --force --gpg-sign upstream/main


Related to #549
Related to cryostatio/cryostat-operator#814

Description of the change:

Adds the Flyway extension for database schema versioning and migration management. The initial schema file was created using the Quarkus Flyway/Hibernate extension integration and matches the existing schema that was previously generated by Hibernate.

Motivation for the change:

This separates the database initialization from the Cryostat container's lifecycle, and provides us easy versioned hooks for data model migrations. This will allow more robust application version upgrades in the future. Combined with cryostatio/cryostat-operator/issues/814, the database's lifecycle is fully decoupled from the Cryostat server's lifecycle, so the Cryostat container can crash/restart/be upgraded without risking inadvertent data loss due to table drop-and-create.

How to manually test:

  1. Check out and build PR
  2. ./smoketest.bash -OXb
  3. Wait some time for the smoketest to come up and the database viewer tab to open in your browser.
  4. Log in to the viewer, connect to the database server, and expand out the Cryostat database schema tables. There will be a new table tracking the Flyway migrations that have been applied.

image

I also tested this in OpenShift so that I could kill the Cryostat container individually (by running a shell in its container and simply doing kill 1) to test the behaviour when Cryostat restarts and reconnects to an existing database, to simulate cryostatio/cryostat-operator#923 where Cryostat and its database are in separate Pods with separate lifecycles.

Run 1 (logs taken from a podman smoketest run, but the behaviour in OpenShift was the same - I just forgot to capture the logs):

Jul 31, 2024 8:34:58 PM org.flywaydb.core.internal.logging.apachecommons.ApacheCommonsLog info
INFO: Flyway Community Edition 9.17.0 by Redgate
Jul 31, 2024 8:34:58 PM org.flywaydb.core.internal.logging.apachecommons.ApacheCommonsLog info
INFO: See release notes here: https://rd.gt/416ObMi
Jul 31, 2024 8:34:58 PM org.flywaydb.core.internal.logging.apachecommons.ApacheCommonsLog info
INFO: 
Jul 31, 2024 8:34:58 PM org.flywaydb.core.internal.logging.apachecommons.ApacheCommonsLog info
INFO: Database: jdbc:postgresql://db:5432/cryostat (PostgreSQL 16.1)
Jul 31, 2024 8:34:58 PM org.flywaydb.core.internal.logging.apachecommons.ApacheCommonsLog warn
WARN: Flyway upgrade recommended: PostgreSQL 16.1 is newer than this version of Flyway and support has not been tested. The latest supported version of PostgreSQL is 15.
Jul 31, 2024 8:34:58 PM org.flywaydb.core.internal.logging.apachecommons.ApacheCommonsLog info
INFO: Schema history table "public"."flyway_schema_history" does not exist yet
Jul 31, 2024 8:34:58 PM org.flywaydb.core.internal.logging.apachecommons.ApacheCommonsLog info
INFO: Successfully validated 1 migration (execution time 00:00.005s)
Jul 31, 2024 8:34:58 PM org.flywaydb.core.internal.logging.apachecommons.ApacheCommonsLog info
INFO: Creating Schema History table "public"."flyway_schema_history" ...
Jul 31, 2024 8:34:58 PM org.flywaydb.core.internal.logging.apachecommons.ApacheCommonsLog info
INFO: Current version of schema "public": << Empty Schema >>
Jul 31, 2024 8:34:58 PM org.flywaydb.core.internal.logging.apachecommons.ApacheCommonsLog info
INFO: Migrating schema "public" to version "3.0.0 - cryostat"
Jul 31, 2024 8:34:58 PM org.flywaydb.core.internal.logging.apachecommons.ApacheCommonsLog info
INFO: Successfully applied 1 migration to schema "public", now at version v3.0.0 (execution time 00:00.170s)

Run 2:

2024-07-31 20:28:49,163 INFO [org.fly.cor.int.lic.VersionPrinter] (main) Flyway Community Edition 9.17.0 by Redgate
2024-07-31 20:28:49,163 INFO [org.fly.cor.int.lic.VersionPrinter] (main) See release notes here: https://rd.gt/416ObMi
2024-07-31 20:28:49,164 INFO [org.fly.cor.int.lic.VersionPrinter] (main)
2024-07-31 20:28:49,225 INFO [org.fly.cor.int.dat.bas.BaseDatabaseType] (main) Database: jdbc:postgresql://localhost:5432/cryostat (PostgreSQL 16.1)
2024-07-31 20:28:49,229 WARN [org.fly.cor.int.dat.bas.Database] (main) Flyway upgrade recommended: PostgreSQL 16.1 is newer than this version of Flyway and support has not been tested. The latest supported version of PostgreSQL is 15.
2024-07-31 20:28:49,242 INFO [org.fly.cor.int.com.DbValidate] (main) Successfully validated 1 migration (execution time 00:00.008s)
2024-07-31 20:28:49,251 INFO [org.fly.cor.int.com.DbMigrate] (main) Current version of schema "public": 3.0.0
2024-07-31 20:28:49,252 INFO [org.fly.cor.int.com.DbMigrate] (main) Schema "public" is up to date. No migration necessary.
2024-07-31 20:28:50,993 INFO [io.quarkus] (main) cryostat 4.0.0-snapshot on JVM (powered by Quarkus 3.2.12.Final) started in 2.999s. Listening on: http://0.0.0.0:8181
2024-07-31 20:28:50,994 INFO [io.quarkus] (main) Profile prod activated.

@andrewazores andrewazores added feat New feature or request safe-to-test labels Jul 31, 2024
@github-actions github-actions bot added the needs-triage Needs thorough attention from code reviewers label Jul 31, 2024
@andrewazores andrewazores removed the needs-triage Needs thorough attention from code reviewers label Jul 31, 2024
@andrewazores andrewazores marked this pull request as ready for review July 31, 2024 20:42
@andrewazores andrewazores requested a review from a team as a code owner July 31, 2024 20:42
@andrewazores andrewazores requested a review from ebaron July 31, 2024 20:42
@andrewazores andrewazores merged commit ca8c386 into cryostatio:main Aug 21, 2024
8 checks passed
@andrewazores andrewazores deleted the flyway branch August 21, 2024 14:59
@andrewazores andrewazores added the breaking change This change (potentially) breaks API compatibility and requires corresponding changes elsewhere label Aug 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change This change (potentially) breaks API compatibility and requires corresponding changes elsewhere feat New feature or request safe-to-test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants