Merge pull request #2 from vmkteam/improvements

Add docker image building, use go 1.20, update code and docs

.github/workflows/docker.yml

@@ -0,0 +1,45 @@
+name: Publish Docker image
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  push_to_registries:
+    name: Push Docker image to multiple registries
+    runs-on: ubuntu-latest
+    permissions:
+      packages: write
+      contents: read
+    steps:
+      - name: Check out the repo
+        uses: actions/checkout@v3
+
+      - name: Log in to Docker Hub
+        uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
+        with:
+          username: ${{ secrets.DOCKER_USERNAME }}
+          password: ${{ secrets.DOCKER_PASSWORD }}
+
+      - name: Log in to the Container registry
+        uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
+        with:
+          images: |
+            vmkteam/pgmigrator
+            ghcr.io/${{ github.repository }}
+
+      - name: Build and push Docker images
+        uses: docker/build-push-action@ad44023a93711e3deb337508980b4b5e9bcdc5dc
+        with:
+          context: .
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}

.github/workflows/go.yml

@@ -16,12 +16,12 @@ jobs:
   build:
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
 
     - name: Set up Go
-      uses: actions/setup-go@v2
+      uses: actions/setup-go@v4
       with:
-        go-version: 1.19
+        go-version: '1.20.x'
 
     - name: Build
       run: go build -v ./...

.github/workflows/golangci-lint.yml

@@ -16,10 +16,10 @@ jobs:
     name: lint
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/setup-go@v3
+      - uses: actions/setup-go@v4
         with:
-          go-version: 1.19
-      - uses: actions/checkout@v3
+          go-version: '1.20.x'
+      - uses: actions/checkout@v4
       - name: golangci-lint
         uses: golangci/golangci-lint-action@v3
         with:

.github/workflows/goreleaser.yml

@@ -14,12 +14,14 @@ jobs:
     steps:
       -
         name: Checkout
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           fetch-depth: 0
       -
         name: Set up Go
-        uses: actions/setup-go@v3
+        uses: actions/setup-go@v4
+        with:
+          go-version: '1.20.x'
       -
         name: Run GoReleaser
         uses: goreleaser/goreleaser-action@v4

Dockerfile

@@ -0,0 +1,7 @@
+FROM golang:1.20-alpine AS builder
+COPY . /build
+RUN cd /build && go install -mod=mod
+
+FROM alpine:latest
+
+COPY --from=builder /go/bin/pgmigrator .

README.md

@@ -12,10 +12,10 @@ Goal: to make an understandable tool for local development and stage environment
 
 Limitations
 --
-* Database: PostgeSQL
-* File format: `YYYYY-MM-DDD-<description>.sql` / `YYYYY-MM-DD-<description>-NONTR.sql`
+* Database: PostgreSQL
+* Migrations filename mask: `YYYYY-MM-DDD-<description>.sql` / `YYYYY-MM-DD-<description>-NONTR.sql`
 * Migration types: UP only
-* Algorithm: run sorted files that are in the folder and fit the file format, except for what is already in the database
+* Algorithm: applies sorted migrations files that are in the folder and fit the file mask, except for what is already applied in the database
 * The program works only with the configuration file located in the folder with migrations.
 
 
@@ -27,29 +27,29 @@ A: In development, we almost never write down migrations because it's useless in
 Q: Why only PostgreSQL? <br>
 A: There is a goal to create a highly specialized simple tool for migrations.
 
-Q: Why such a specific file format rather than the generally recognized one `V<Version>_<description>.sql`?<br>
+Q: Why such a specific file mask rather than the generally recognized one `V<Version>_<description>.sql`?<br>
 A: This is historically the case. The date in the file gives more transparency than the version number. Less development conflicts in the branch.<br>
-_It is possible to override this parameter through the config._
+_It is possible to override this parameter through the configuration file._
 
 Q: Why not as a library? Why migrations specifically as files on disk?
 A: The goal is a simple utility that works with files. Alternatives in the form of libraries have already been written here https://awesome-go.com/#database-schema-migration
 
 Migrations
 --
-Migrations are located in a folder. Subfolders are not counted. Files are sorted by name.
-All recorded migrations are written to a table (by default `public.pgMigrations`)
-File format is `YYYYY-MM-DDD-<description>.sql`.
+Migration files are located in a folder. Subfolders are not counted. Files are sorted by name.
+All recorded migrations are written to a table in database (by default `public.pgMigrations`)
+Default file mask: `YYYYY-MM-DDD-<description>.sql`.
 
-All migrations are started in a separate transaction with a specific StatementTimeout in the config. If not specified, it is not used.
-Non-transactional migrations have the following format `YYYYY-MM-DD-<description>-NONTR.sql` (e.g. for create index concurrently).
+All migrations are started in a separate transaction with a specific StatementTimeout in the configuration file. If not specified, it is not used.
+Non-transactional migrations have the following file mask: `YYYYY-MM-DD-<description>-NONTR.sql` (e.g. for create index concurrently).
 
-You can override the file mask through the config. If not specified, the default one is used.
-If there is `MANUAL` at the end of the file name, such migration is ignored.
+You can override the file mask through the configuration file. If not specified, the default one is used.
+If there is `MANUAL` at the end of the file name, this migration will be ignored.
 
         2020 // folder, not counted
-        pgmigrator.toml // mandatory config file
-        2021-04-12-create-table-commentTranslations.sql
-        2021-06-02-make-person-alias-not-null-NONTR.sql
+        pgmigrator.toml // mandatory configuration file
+        2021-04-12-create-table-commentTranslations.sql // runs inside transaction
+        2021-06-02-make-person-alias-not-null-NONTR.sql  // runs outside transaction
         2021-06-03-make-person-alias-not-null-MANUAL.sql // ignored
 
 
@@ -63,87 +63,117 @@ Configuration file
 	[Database]
 	Addr     = "localhost:5432"
 	User     = "postgres"
-	Database = "testdb"
 	Password = "tesdb"
+	Database = "testdb"
 	PoolSize = 1
 	ApplicationName = "pgmigrator"
 
 Run
 --
-	Applies PostgreSQL migrations
-
-	Usage:
-	pgmigrator [command]
-
-	Available Commands:
-	completion  Generate the autocompletion script for the specified shell
-	create      Creates default config file pgmigrator.toml in current dir
-	dryrun      Tries to apply migrations. Runs migrations inside single transaction and always rollbacks it
-	help        Help about any command
-	last        Shows recent migrations from db
-	plan        Shows migration files which can be applied
-	redo        Rerun last migration
-	run         Run to apply migrations
-	verify      Checks and shows invalid migrations
-
-	Flags:
-	--config string   config file (default "pgmigrator.toml")
-	-h, --help            help for pgmigrator
-
-	Use "pgmigrator [command] --help" for more information about a command.
+    Command-line tool for PostgreSQL migrations
+
+    Usage:
+    pgmigrator [command]
+
+    Available Commands:
+    completion  Generate the autocompletion script for the specified shell
+    dryrun      Tries to apply migrations. Runs migrations inside single transaction and always rollbacks it
+    help        Help about any command
+    init        Initialize default configuration file in current directory
+    last        Shows recent applied migrations from db
+    plan        Shows migration files which can be applied
+    redo        Rerun last applied migration from db
+    run         Applies all new migrations
+    verify      Checks and shows invalid migrations
+
+    Flags:
+    -c, --config string   configuration file (default "pgmigrator.toml")
+    -h, --help            help for pgmigrator
+    -v, --version         version for pgmigrator
+
+    Use "pgmigrator [command] --help" for more information about a command.
 
 Any command supports an argument in the form of a number. For `last` it is the number of last migrations (default is 5). For all others - the number of the file from `plan` to which to apply migrations. If no argument is passed, there are no restrictions (or default ones are used).
 
-The base directory for migrations is the one where the config file is located.
+The base directory for migrations is the one where the configuration file is located.
 That is, you can call `pgmigrator --config docs/patches/pgmigrator.toml plan` and it will take all migrations from the `docs/patches` folder.
 
 ## Commands
 
 ### Plan
 
-Algorithm:
+**Algorithm**
 
 * get a list of files sorted by name
 * connect to the database
     - check if there is a table
     - get the list of migrations from the database
-* display the list of files to be applied
+* display the list of files to be applied  (TODO: highlight DROP?)
+
+**Output**
+
+	Planning to apply Х migrations:
+		1 - 2022-07-18-movieComments.sql
+		2 - 2022-07-28-jwlinks.sql
+		3 - 2022-07-30-compilations-fix.sql 
 
 
 ### Run
 
-    create a table, if necessary
-    make a plan
-    for each file.
-        if the migration is regular
-        begin
-            perform migration
-            add a record of the completed migration
-        commit
-
-        if migration is non-transactional
-            add migration record
-            perform migration
-            update migration record
-
-        if something goes wrong - rollback and exit (except for nontr, since it is not clear what exactly happened).
+**Algorithm**
+
+* create a table, if necessary
+* make a plan
+  - for each file
+     - if the migration is regular
+        - begin
+          - perform migration
+          - add a record of the completed migration
+        - commit
+     - if migration is non-transactional
+        - add migration record
+        - perform migration
+        - update migration record
+     - if something goes wrong - rollback and exit (except for nontr, since it is not clear what exactly happened).
+
+**Output**
+
+	Planning to apply Х migrations:
+		1 - 2022-07-18-movieComments.sql
+		2 - 2022-07-28-jwlinks.sql
+		3 - 2022-07-30-compilations-fix.sql 
+
+	Applying:
+		1 - 2022-07-18-movieComments.sql ... done in 2s 
+		2 - 2022-07-28-jwlinks.sql ... done in 3m
+		3 - 2022-07-30-compilations-fix.sql ... 		
+	ERROR: <error text>
+
 ### DryRun
 
-* Like Run, but open one big transaction and use ROLLBACK.
+* Like `Run`, but open one big transaction and use ROLLBACK.
 * If there is a NONTR - do not let dryrun run (only up to a certain filename)
-* Ignore StatementTimeout
+* `StatementTimeout` setting is ignored
 
 ### Last
 
 Shows the latest database migrations from a table.
 
+**Output**
+
+		Showing last migrations in public.pgMigrations:
+		34 - 2022-08-30 22:25:03 (ERR) 	 > 2022-07-30-compilations-NONTR.sql
+		33 - 2022-08-30 22:25:03 (3s)  	 > 2022-07-30-compilations-fix.sql
+		32 - 2022-08-30 22:25:34 (1s)    > 2022-07-28-jwlinks.sql
+		31 - 2022-08-30 22:23:12 (5m 4s) > 2022-07-18-movieComments.sql
+
 ### Verify
 
 Checks patch integrity in the database and locally by md5 hash.
 
-### Create
+### Init
 
-Creates a new configuration file.
+Initializes a new configuration file with default settings.
 
 ### Redo
 
@@ -164,4 +194,15 @@ It is recommended to have a different migrations table for each schema.
         md5sum        varchar(32)               not null,
         primary key ("id"),
         unique ("filename")
-    );
+    );
+
+* id (pk) – serial
+* filename - unique migration file name
+* startedAt - timestamp of starting migration
+* finishedAt - timestamp of finishing migration
+* transactional - transactional flag (false для NONTR migrations)
+* md5sum - md5 hash of migration file 
+
+### Docker images
+- [Docker Hub](https://hub.docker.com/vmkteam/pgmigrator)
+- Packages Tab in this repo