Add kogito-db-migrator-tool and kogito-db-migrator-tool-image packages

apache · Dec 17, 2024 · 8252831 · 8252831
1 parent 37d7322
commit 8252831
Show file tree

Hide file tree

Showing 25 changed files with 1,177 additions and 2 deletions.
diff --git a/.gitignore b/.gitignore
@@ -347,6 +347,7 @@ turbo.json
 packages/*/.idea
 examples/*/.idea
 
-
+# Jar files
+**/*.jar
 # Apache RAT check excludes file
-!.rat-excludes
+!.rat-excludes
diff --git a/packages/kogito-db-migrator-tool-image/README.md b/packages/kogito-db-migrator-tool-image/README.md
@@ -0,0 +1,116 @@
+<!--
+   Licensed to the Apache Software Foundation (ASF) under one
+   or more contributor license agreements.  See the NOTICE file
+   distributed with this work for additional information
+   regarding copyright ownership.  The ASF licenses this file
+   to you under the Apache License, Version 2.0 (the
+   "License"); you may not use this file except in compliance
+   with the License.  You may obtain a copy of the License at
+     http://www.apache.org/licenses/LICENSE-2.0
+   Unless required by applicable law or agreed to in writing,
+   software distributed under the License is distributed on an
+   "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+   KIND, either express or implied.  See the License for the
+   specific language governing permissions and limitations
+   under the License.
+-->
+
+# Kogito Postgres DB Migrator Tool Image
+
+This package contains the `Containerfile/Dockerfile` and scripts to build a container image for Kogito Postgres DB Migrator Tool. Details about the Kogito Postgres DB Migrator Tool can be found [here](../kogito-db-migrator-tool/README.md)
+
+## Additional requirements
+
+- docker
+
+## Build
+
+_NOTE_: Before performing this step, be sure that the Kogito Postgres DB Migrator Tool jar has been built and available for inclusion in the image.
+
+- Enable the image to be built:
+
+  ```bash
+  export KIE_TOOLS_BUILD__buildContainerImages=true
+  ```
+
+Run the following in the root folder of the repository to build the package:
+
+```bash
+pnpm -F @kie-tools/kogito-db-migrator-tool-image... build:prod
+```
+
+- Then check if the image is correctly stored:
+
+  ```bash
+  docker images
+  ```
+
+## Run
+
+- Start up a clean container with:
+
+  ```bash
+    docker run docker.io/apache/incubator-kie-kogito-service-db-migration-postgresql:latest
+  ```
+
+## Customization
+
+1. Run a container with custom environment variables:
+
+| NAME                                    | DESCRIPTION                                                                      | DEFAULT                                   |
+| --------------------------------------- | -------------------------------------------------------------------------------- | ----------------------------------------- |
+| MIGRATE_DB_DATAINDEX                    | Set to true if you want to migrate data index database, set to false otherwise   | false                                     |
+| QUARKUS_DATASOURCE_DATAINDEX_JDBC_URL   | Data index database url e.g. jdbc:postgresql://host.docker.internal:5432/di      | jdbc:postgresql://localhost:5432/postgres |
+| QUARKUS_DATASOURCE_DATAINDEX_USERNAME   | Data index database username                                                     | postgres                                  |
+| QUARKUS_DATASOURCE_DATAINDEX_PASSWORD   | Data index database password                                                     | postgres                                  |
+| QUARKUS_FLYWAY_DATAINDEX_SCHEMAS        | Data index database schema                                                       | dataindex                                 |
+| MIGRATE_DB_JOBSSERVICE                  | Set to true if you want to migrate jobs service database, set to false otherwise | false                                     |
+| QUARKUS_DATASOURCE_JOBSSERVICE_JDBC_URL | Jobs service database url e.g. jdbc:postgresql://host.docker.internal:5432/js    | jdbc:postgresql://localhost:5432/postgres |
+| QUARKUS_DATASOURCE_JOBSSERVICE_USERNAME | Jobs service database username                                                   | postgres                                  |
+| QUARKUS_DATASOURCE_JOBSSERVICE_PASSWORD | Jobs service database password                                                   | postgres                                  |
+| QUARKUS_FLYWAY_JOBSSERVICE_SCHEMAS      | Jobs service database schema                                                     | jobsservice                               |
+
+### Example
+
+An example to use diverse environment variables
+
+```bash
+   docker run \
+   --env MIGRATE_DB_DATAINDEX=true \
+   --env QUARKUS_DATASOURCE_DATAINDEX_JDBC_URL=<data-index-db-url e.g. jdbc:postgresql://host.docker.internal:5432/di> \
+   --env QUARKUS_DATASOURCE_DATAINDEX_USERNAME=<data-index-db-user> \
+   --env QUARKUS_DATASOURCE_DATAINDEX_PASSWORD=<data-index-db-password> \
+   --env QUARKUS_FLYWAY_DATAINDEX_SCHEMAS=dataindex \
+   --env MIGRATE_DB_JOBSSERVICE=true \
+   --env QUARKUS_DATASOURCE_JOBSSERVICE_JDBC_URL=<jobs-service-db-url e.g. jdbc:postgresql://host.docker.internal:5432/js> \
+   --env QUARKUS_DATASOURCE_JOBSSERVICE_USERNAME=<jobs-service-db-user> \
+   --env QUARKUS_DATASOURCE_JOBSSERVICE_PASSWORD=<jobs-service-db-password> \
+   --env QUARKUS_FLYWAY_JOBSSERVICE_SCHEMAS=jobsservice \
+   docker.io/apache/incubator-kie-kogito-service-db-migration-postgresql:999-SNAPSHOT
+```
+
+---
+
+Apache KIE (incubating) is an effort undergoing incubation at The Apache Software
+Foundation (ASF), sponsored by the name of Apache Incubator. Incubation is
+required of all newly accepted projects until a further review indicates that
+the infrastructure, communications, and decision making process have stabilized
+in a manner consistent with other successful ASF projects. While incubation
+status is not necessarily a reflection of the completeness or stability of the
+code, it does indicate that the project has yet to be fully endorsed by the ASF.
+
+Some of the incubating project’s releases may not be fully compliant with ASF
+policy. For example, releases may have incomplete or un-reviewed licensing
+conditions. What follows is a list of known issues the project is currently
+aware of (note that this list, by definition, is likely to be incomplete):
+
+- Hibernate, an LGPL project, is being used. Hibernate is in the process of
+  relicensing to ASL v2
+- Some files, particularly test files, and those not supporting comments, may
+  be missing the ASF Licensing Header
+
+If you are planning to incorporate this work into your product/project, please
+be aware that you will need to conduct a thorough licensing review to determine
+the overall implications of including this work. For the current status of this
+project through the Apache Incubator visit:
+https://incubator.apache.org/projects/kie.html
diff --git a/packages/kogito-db-migrator-tool-image/env/index.js b/packages/kogito-db-migrator-tool-image/env/index.js
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+const { varsWithName, composeEnv, getOrDefault } = require("@kie-tools-scripts/build-env");
+
+const rootEnv = require("@kie-tools/root-env/env");
+const sonataflowImageCommonEnv = require("@kie-tools/sonataflow-image-common/env");
+
+module.exports = composeEnv([rootEnv, sonataflowImageCommonEnv], {
+  vars: varsWithName({
+    KOGITO_DB_MIGRATOR_TOOL_IMAGE__registry: {
+      default: "docker.io",
+      description: "The image registry.",
+    },
+    KOGITO_DB_MIGRATOR_TOOL_IMAGE__account: {
+      default: "apache",
+      description: "The image registry account.",
+    },
+    KOGITO_DB_MIGRATOR_TOOL_IMAGE__name: {
+      default: "incubator-kie-kogito-db-migrator-tool",
+      description: "The image name.",
+    },
+    KOGITO_DB_MIGRATOR_TOOL_IMAGE__buildTag: {
+      default: rootEnv.env.root.streamName,
+      description: "The image tag.",
+    },
+  }),
+  get env() {
+    return {
+      kogitoDbMigratorToolImage: {
+        registry: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__registry),
+        account: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__account),
+        name: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__name),
+        buildTag: getOrDefault(this.vars.KOGITO_DB_MIGRATOR_TOOL_IMAGE__buildTag),
+      },
+    };
+  },
+});
diff --git a/packages/kogito-db-migrator-tool-image/install.js b/packages/kogito-db-migrator-tool-image/install.js
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+const { execSync } = require("child_process");
+const fs = require("fs");
+
+const { env } = require("./env");
+const path = require("path");
+const pythonVenvDir = path.dirname(require.resolve("@kie-tools/python-venv/package.json"));
+const sonataflowImageCommonDir = path.dirname(require.resolve("@kie-tools/sonataflow-image-common/package.json"));
+const replaceInFile = require("replace-in-file");
+
+const activateCmd =
+  process.platform === "win32"
+    ? `${pythonVenvDir}\\venv\\Scripts\\Activate.bat`
+    : `. ${pythonVenvDir}/venv/bin/activate`;
+
+execSync(
+  `${activateCmd} && \
+  python3 ${sonataflowImageCommonDir}/resources/scripts/versions_manager.py --bump-to ${env.kogitoDbMigratorToolImage.buildTag} --source-folder ./resources`,
+  { stdio: "inherit" }
+);
+
+// Find and read the -image.yaml file
+const resourcesPath = path.resolve(__dirname, "./resources");
+const files = fs.readdirSync(resourcesPath);
+const imageYamlFiles = files.filter((fileName) => fileName.endsWith("image.yaml"));
+if (imageYamlFiles.length !== 1) {
+  throw new Error("There should only be one image.yaml file on ./resources!");
+}
+const originalYamlPath = path.join(resourcesPath, imageYamlFiles[0]);
+let imageYaml = fs.readFileSync(originalYamlPath, "utf8");
+
+const imageUrl = `${env.kogitoDbMigratorToolImage.registry}/${env.kogitoDbMigratorToolImage.account}/${env.kogitoDbMigratorToolImage.name}`;
+
+// Replace the whole string between quotes ("") with the image name
+imageYaml = imageYaml.replace(/(?<=")(.*incubator-kie-kogito-service-db-migration-postgresql.*)(?=")/gm, imageUrl);
+
+// Write file and then rename it to match the image name
+fs.writeFileSync(originalYamlPath, imageYaml);
+fs.renameSync(originalYamlPath, path.join(resourcesPath, `${env.kogitoDbMigratorToolImage.name}-image.yaml`));
+
+// Replace image URL in .feature files
+replaceInFile.sync({
+  files: ["**/*.feature"],
+  from: /@docker.io\/apache\/.*/g,
+  to: `@${imageUrl}`,
+});
diff --git a/packages/kogito-db-migrator-tool-image/package.json b/packages/kogito-db-migrator-tool-image/package.json
@@ -0,0 +1,44 @@
+{
+  "private": true,
+  "name": "@kie-tools/kogito-db-migrator-tool-image",
+  "version": "0.0.0",
+  "description": "",
+  "license": "Apache-2.0",
+  "homepage": "https://github.com/apache/incubator-kie-tools",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/apache/incubator-kie-tools.git"
+  },
+  "bugs": {
+    "url": "https://github.com/apache/incubator-kie-tools/issues"
+  },
+  "scripts": {
+    "build": ". ./node_modules/@kie-tools/python-venv/venv/bin/activate && pnpm copy-assets && pnpm image:build && rimraf build && rimraf target && rimraf dist-tests-e2e",
+    "build:dev": "run-script-os",
+    "build:dev:linux": "pnpm build",
+    "build:dev:win32:darwin": "echo \"Build skipped on Windows and MacOS\"",
+    "build:prod": "run-script-os",
+    "build:prod:linux": "pnpm build",
+    "build:prod:win32:darwin": "echo \"Build skipped on Windows and MacOS\"",
+    "copy-assets": "run-script-os",
+    "copy-assets:linux:darwin": "rimraf build && cp -R ./node_modules/@kie-tools/sonataflow-image-common/resources build && cp -R resources/* build && mkdir -p build/modules/kogito-postgres-db-migration-deps/quarkus-app && cp -R ./node_modules/@kie-tools/kogito-db-migrator-tool/target/quarkus-app/* build/modules/kogito-postgres-db-migration-deps/quarkus-app",
+    "format": "prettier --write . --ignore-path=../../.prettierignore --ignore-path=../../.gitignore",
+    "image:build": "run-script-os",
+    "image:build:linux": "pnpm setup:env make -C ./build build",
+    "image:build:win32:darwin": "echo \"Build skipped on Windows and MacOS\"",
+    "install": "node install.js && pnpm format",
+    "setup:env": ". ./node_modules/@kie-tools/python-venv/venv/bin/activate && cross-env KOGITO_IMAGE_REGISTRY=$(build-env kogitoDbMigratorToolImage.registry) KOGITO_IMAGE_REGISTRY_ACCOUNT=$(build-env kogitoDbMigratorToolImage.account) KOGITO_IMAGE_NAME=$(build-env kogitoDbMigratorToolImage.name) KOGITO_IMAGE_TAG=$(build-env kogitoDbMigratorToolImage.buildTag) QUARKUS_PLATFORM_GROUPID=$(build-env kogitoImagesCekitModules.quarkusGroupId) QUARKUS_PLATFORM_VERSION=$(build-env versions.quarkus) KOGITO_VERSION=$(build-env versions.kogito)"
+  },
+  "dependencies": {
+    "@kie-tools/kogito-db-migrator-tool": "workspace:*",
+    "@kie-tools/python-venv": "workspace:*",
+    "@kie-tools/root-env": "workspace:*",
+    "@kie-tools/sonataflow-image-common": "workspace:*"
+  },
+  "devDependencies": {
+    "cross-env": "^7.0.3",
+    "replace-in-file": "^7.1.0",
+    "rimraf": "^3.0.2",
+    "run-script-os": "^1.1.6"
+  }
+}
diff --git a/.../kogito-db-migrator-tool-image/resources/incubator-kie-kogito-db-migrator-tool-image.yaml b/.../kogito-db-migrator-tool-image/resources/incubator-kie-kogito-db-migrator-tool-image.yaml
@@ -0,0 +1,50 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+name: "docker.io/apache/incubator-kie-kogito-db-migrator-tool"
+version: "main"
+from: registry.access.redhat.com/ubi8/openjdk-17-runtime:1.20
+description: Flyway image for Data Index and Jobs Service database migration
+
+labels:
+  - name: "org.kie.kogito.version"
+    value: "### SET ME DURING BUILD PROCESS ###"
+  - name: "maintainer"
+    value: "Apache KIE <[email protected]>"
+  - name: "io.k8s.description"
+    value: "Kogito DB Migration creates schemas and tables for Data Index and Jobs Service for PostgreSQL database"
+  - name: "io.k8s.display-name"
+    value: "Kogito DB Migration for Data Index and Jobs Service - PostgreSQL"
+  - name: "io.openshift.tags"
+    value: "kogito,db-migration"
+
+modules:
+  repositories:
+    - path: modules
+  install:
+    - name: org.kie.kogito.system.user
+    - name: org.kie.kogito.project.versions
+    - name: org.kie.kogito.postgresql.db-migration-deps
+
+run:
+  workdir: "/home/kogito/bin"
+  entrypoint:
+    - "java"
+  cmd:
+    - "-jar"
+    - "./quarkus-run.jar"
diff --git a/...to-db-migrator-tool-image/resources/modules/kogito-postgres-db-migration-deps/module.yaml b/...to-db-migrator-tool-image/resources/modules/kogito-postgres-db-migration-deps/module.yaml
@@ -0,0 +1,26 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+schema_version: 1
+name: org.kie.kogito.postgresql.db-migration-deps
+version: "main"
+artifacts:
+  # The following artifact comes from the package kogito-db-migrator-tool during the pnpm build:dev phase
+  - name: sonataflow-db-migrator-quarkus-app
+    path: ./quarkus-app
+    dest: /home/kogito/bin
diff --git a/packages/kogito-db-migrator-tool/.gitignore b/packages/kogito-db-migrator-tool/.gitignore
@@ -0,0 +1,4 @@
+# kogito-db-migrator-tool
+src/main/resources/postgresql
+src/main/resources/ansi
+src/main/resources/h2