-
Notifications
You must be signed in to change notification settings - Fork 80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Issue#1785]-Create Data Index(DI)/ Jobs Service(JS) Database(DB) migration image #1791
Changes from all commits
1308719
24fb2d5
36061d2
18dd3fe
dd96e7a
32f96e4
7475600
07aa695
b96d244
d38991b
26472d2
1e0e549
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
# | ||
# Licensed to the Apache Software Foundation (ASF) under one | ||
# or more contributor license agreements. See the NOTICE file | ||
# distributed with this work for additional information | ||
# regarding copyright ownership. The ASF licenses this file | ||
# to you under the Apache License, Version 2.0 (the | ||
# "License"); you may not use this file except in compliance | ||
# with the License. You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, | ||
# software distributed under the License is distributed on an | ||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
# KIND, either express or implied. See the License for the | ||
# specific language governing permissions and limitations | ||
# under the License. | ||
# | ||
name: "docker.io/apache/incubator-kie-kogito-service-db-migration-postgresql" | ||
version: "999-SNAPSHOT" | ||
from: registry.access.redhat.com/ubi8/openjdk-17-runtime:1.19 | ||
description: Flyway image for DI/JS database migration | ||
|
||
labels: | ||
- name: "org.kie.kogito.version" | ||
value: "999-SNAPSHOT" | ||
- name: "maintainer" | ||
value: "Apache KIE <[email protected]>" | ||
- name: "io.k8s.description" | ||
value: "Kogito DB Migration creates schemas and tables for Data Index and Jobs Service for PostgreSQL database" | ||
- name: "io.k8s.display-name" | ||
value: "Kogito DB Migration for Data Index and Jobs Service - PostgreSQL" | ||
- name: "io.openshift.tags" | ||
value: "kogito,db-migration" | ||
|
||
modules: | ||
repositories: | ||
- path: modules | ||
install: | ||
- name: kogito-postgres-db-migration-deps | ||
|
||
run: | ||
workdir: "/home/default" | ||
entrypoint: | ||
- "/home/default/migration.sh" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,89 @@ | ||
#!/bin/bash | ||
# | ||
# Licensed to the Apache Software Foundation (ASF) under one | ||
# or more contributor license agreements. See the NOTICE file | ||
# distributed with this work for additional information | ||
# regarding copyright ownership. The ASF licenses this file | ||
# to you under the Apache License, Version 2.0 (the | ||
# "License"); you may not use this file except in compliance | ||
# with the License. You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, | ||
# software distributed under the License is distributed on an | ||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
# KIND, either express or implied. See the License for the | ||
# specific language governing permissions and limitations | ||
# under the License. | ||
# | ||
|
||
# DB migration function | ||
migrate () { | ||
local SERVICE_NAME=$1 # Name of service e.g. data-index or jobs-service | ||
local MIGRATE_DB=$2 # To migrate DB set to true | ||
local DB_URL=$3 | ||
local DB_USER=$4 | ||
local DB_PWD=$5 | ||
local SCHEMA_NAME=$6 | ||
|
||
if $MIGRATE_DB | ||
then | ||
echo USING ENVIRONMENT VARS: $SERVICE_NAME | ||
echo URL=$DB_URL USER=$DB_USER PWD=******** | ||
|
||
echo LISTING SQL DIR: $SERVICE_NAME | ||
ls /home/default/postgresql/$SERVICE_NAME | ||
|
||
/home/default/flyway/flyway -url="$DB_URL" -user="$DB_USER" -password="$DB_PWD" -mixed="true" -locations="filesystem:/home/default/postgresql/$SERVICE_NAME" -schemas="$SCHEMA_NAME" migrate | ||
/home/default/flyway/flyway -url="$DB_URL" -user="$DB_USER" -password="$DB_PWD" -mixed="true" -locations="filesystem:/home/default/postgresql/$SERVICE_NAME" -schemas="$SCHEMA_NAME" info | ||
fi | ||
} | ||
|
||
# DB migration flag validation | ||
function validateDBMigration() { | ||
local SERVICE_NAME=$1 | ||
local MIGRATE_DB=$2 | ||
echo Validating $SERVICE_NAME for db migration value $MIGRATE_DB | ||
|
||
if [ "${MIGRATE_DB}" = true ] || [ "${MIGRATE_DB}" = false ]; then | ||
echo "DB migration flag for service $SERVICE_NAME will be set to $MIGRATE_DB" | ||
else | ||
echo "DB migration flag for service $SERVICE_NAME, should be either true or false, but found $MIGRATE_DB, exiting" | ||
exit 1 | ||
fi | ||
} | ||
|
||
# Process data-index | ||
SERVICE_DATA_INDEX="data-index" | ||
|
||
if [ -z "$MIGRATE_DATA_INDEX" ]; then | ||
MIGRATE_DATA_INDEX=true | ||
else | ||
validateDBMigration $SERVICE_DATA_INDEX $MIGRATE_DATA_INDEX | ||
fi | ||
echo "Migrating data index: $MIGRATE_DATA_INDEX" | ||
|
||
if [ -z "$DATA_INDEX_SCHEMA" ]; then | ||
DATA_INDEX_SCHEMA=data-index-service | ||
echo "Using the data index schema: $DATA_INDEX_SCHEMA" | ||
fi | ||
|
||
migrate $SERVICE_DATA_INDEX $MIGRATE_DATA_INDEX $DI_DB_URL $DI_DB_USER $DI_DB_PWD $DATA_INDEX_SCHEMA | ||
|
||
# Process jobs-service | ||
SERVICE_JOBS_SERVICE="jobs-service" | ||
|
||
if [ -z "$MIGRATE_JOBS_SERVICE" ]; then | ||
MIGRATE_JOBS_SERVICE=true | ||
else | ||
validateDBMigration $SERVICE_JOBS_SERVICE $MIGRATE_JOBS_SERVICE | ||
fi | ||
echo "Migrating jobs service: $MIGRATE_JOBS_SERVICE" | ||
|
||
if [ -z "$JOBS_SERVICE_SCHEMA" ]; then | ||
JOBS_SERVICE_SCHEMA=jobs-service | ||
echo "Using the jobs service schema: $JOBS_SERVICE_SCHEMA" | ||
fi | ||
|
||
migrate $SERVICE_JOBS_SERVICE $MIGRATE_JOBS_SERVICE $JS_DB_URL $JS_DB_USER $JS_DB_PWD $JOBS_SERVICE_SCHEMA |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
#!/bin/sh | ||
# | ||
# Licensed to the Apache Software Foundation (ASF) under one | ||
# or more contributor license agreements. See the NOTICE file | ||
# distributed with this work for additional information | ||
# regarding copyright ownership. The ASF licenses this file | ||
# to you under the Apache License, Version 2.0 (the | ||
# "License"); you may not use this file except in compliance | ||
# with the License. You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, | ||
# software distributed under the License is distributed on an | ||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
# KIND, either express or implied. See the License for the | ||
# specific language governing permissions and limitations | ||
# under the License. | ||
# | ||
|
||
mkdir -p /home/default | ||
cd /home/default | ||
|
||
microdnf install --nodocs tar gzip wget unzip | ||
|
||
wget https://repository.apache.org/content/groups/snapshots/org/kie/kogito/kogito-ddl/10.0.999-SNAPSHOT/kogito-ddl-10.0.999-20240726.011627-10-db-scripts.zip | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This URL should be an ENV VAR that would be set during the image build. |
||
unzip kogito-ddl-10.0.999-20240726.011627-10-db-scripts.zip | ||
|
||
wget https://repo1.maven.org/maven2/org/flywaydb/flyway-commandline/10.17.0/flyway-commandline-10.17.0-linux-x64.tar.gz | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same thing here. |
||
mv flyway-commandline-10.17.0-linux-x64.tar.gz flyway.tar.gz | ||
|
||
tar -xf /home/default/flyway.tar.gz | ||
mv flyway-10.17.0 flyway | ||
|
||
chmod a+x migration.sh | ||
|
||
chgrp -R 0 /home/default | ||
chown -R 0 /home/default | ||
chmod -R g=u /home/default | ||
|
||
ls -al /home/default | ||
ls -al /home/default/flyway | ||
ls -al /home/default/db-migration-files | ||
|
||
microdnf clean all |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# | ||
# Licensed to the Apache Software Foundation (ASF) under one | ||
# or more contributor license agreements. See the NOTICE file | ||
# distributed with this work for additional information | ||
# regarding copyright ownership. The ASF licenses this file | ||
# to you under the Apache License, Version 2.0 (the | ||
# "License"); you may not use this file except in compliance | ||
# with the License. You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, | ||
# software distributed under the License is distributed on an | ||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
# KIND, either express or implied. See the License for the | ||
# specific language governing permissions and limitations | ||
# under the License. | ||
# | ||
schema_version: 1 | ||
name: kogito-postgres-db-migration-deps | ||
version: "1.0" | ||
artifacts: | ||
- name: migration.sh | ||
path: artifacts/migration.sh | ||
dest: /home/default | ||
execute: | ||
- script: install | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This module can set versions/URLs for flyway and kogito DDL scripts. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
# | ||
# Licensed to the Apache Software Foundation (ASF) under one | ||
# or more contributor license agreements. See the NOTICE file | ||
# distributed with this work for additional information | ||
# regarding copyright ownership. The ASF licenses this file | ||
# to you under the Apache License, Version 2.0 (the | ||
# "License"); you may not use this file except in compliance | ||
# with the License. You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, | ||
# software distributed under the License is distributed on an | ||
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
# KIND, either express or implied. See the License for the | ||
# specific language governing permissions and limitations | ||
# under the License. | ||
# | ||
|
||
@docker.io/apache/incubator-kie-kogito-service-db-migration-postgresql | ||
Feature: kogito-postgres-db-migration DB migration for postgresql feature. | ||
|
||
Scenario: verify if all labels are correctly set on kogito-postgres-db-migration-image image | ||
Given image is built | ||
Then the image should contain label maintainer with value Apache KIE <[email protected]> | ||
And the image should contain label io.k8s.description with value Kogito DB Migration creates schemas and tables for Data Index and Jobs Service for PostgreSQL database | ||
And the image should contain label io.k8s.display-name with value Kogito DB Migration for Data Index and Jobs Service - PostgreSQL | ||
And the image should contain label io.openshift.tags with value kogito,db-migration | ||
|
||
Scenario: Verify log entries | ||
When container is started with command bash -c '/home/default/migration.sh' | ||
Then container log should contain LISTING SQL DIR | ||
And container log should contain V1.44.0__data_index_definitions.sql | ||
And container log should contain V2.0.1__job_details_increase_job_id_size.sql | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Avoid using version names to avoid errors in the CI testing when this changes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, but this is unnecessary. You can remove this file, actually. We use GH releases now.