Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/v3' into new-branding
Browse files Browse the repository at this point in the history
Signed-off-by: Pierre-Alexandre Meyer <[email protected]>
  • Loading branch information
pierre committed Aug 22, 2023
2 parents aa7b97b + 4acbb50 commit f46edf6
Show file tree
Hide file tree
Showing 10 changed files with 453 additions and 304 deletions.
2 changes: 1 addition & 1 deletion html5/_main_toc.html.slim
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ nav.sidebar-nav
a.nav-link href="/latest/userguide_deployment.html"
| Going to Production
li
a.nav-link href="/latest/database_migrations.html"
a.nav-link href="/latest/how-to-upgrade-the-database.html"
| Database Migrations
li
a.nav-link href="/latest/PostgreSQL.html"
Expand Down
228 changes: 120 additions & 108 deletions userguide/aws/how-to-set-up-a-multi-tier-system.adoc

Large diffs are not rendered by default.

18 changes: 18 additions & 0 deletions userguide/database-migration/how-to-upgrade-the-database.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
= How to upgrade the Kill Bill Database

== Overview
Often, when you are upgrading to a higher Kill Bill version, you would also need to upgrade the Kill Bill/plugin database. This document is a step-by-step guide on how the database upgrade can be performed.

== Prerequisites

Ensure that you have https://github.com/killbill/killbill-cloud/tree/master/kpm#kpm-installation[kpm] installed.

== Upgrading Kill Bill Database

[[step_1]]
include::{sourcedir}/database-migration/includes/how-to-upgrade-the-killbill-database.adoc[]

== Upgrading Plugin Database

[[step_1]]
include::{sourcedir}/database-migration/includes/how-to-upgrade-plugin-database.adoc[]
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
[[step_1]]
=== Step 1 - Obtain the migrations to run

The `kpm migrate` command can be used to obtain database migrations.

*Command*

[source,bash]
----
kpm migrations github_repository from_tag to_tag
----

*Example*

To download the migration files to migrate the analytics plugin from `8.0.0` to `8.1.0`:

[source,bash]
----
kpm migration killbill-analytics-plugin analytics-plugin-8.0.0 analytics-plugin-8.1.0
----

This command creates a temporary folder with the migration SQL statements and displays its path.

[NOTE]
*Note:* Because the implementation relies on the GitHub API, unauthenticated requests are subject to rate limiting. To work around it, generate a token via https://github.com/settings/tokens (default public, read-only, permissions will work) and specify it to KPM via `kpm migrations killbill killbill-0.22.32 killbill-0.24.0 --token=TOKEN`

=== Step 2 - Execute the migrations

The migrations can be executed either manually or via https://flywaydb.org/[Flyway].

==== Executing migrations manually

Execute the SQL scripts obtained above directly on your database.

==== Executing migrations via the Kill Bill Flywar Jar in Dev environments

We publish a https://repo1.maven.org/maven2/org/kill-bill/billing/killbill-util/0.24.4/killbill-util-0.24.4-flyway.jar[killbill-flyway.jar] jar which is a wrapper around the Flyway utility and can be used to execute the migrations. In order to execute the migrations via this jar, you can do the following:

1. If Flyway is being used the first time, run the `baseline` command to create the `schema_history` table.
+
*Command*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/<db_name> -user=<db_user> -password=<db_password> -table=<plugin_name>_schema_history baseline
----
+
*Example*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/killbill -user=root -password=killbill -table=analytics_schema_history baseline
----

+
2. Run the `migrate` command to execute the migrations.
+
*Command*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/<db_name> -user=<db_user> -password=<db_password> -locations=filesystem:<migrations_path> -table=<plugin_name>_schema_history migrate
----
+
*Example*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/killbill -user=root -password=killbill -locations=filesystem:C:/var/migrations -table=analytics_schema_history migrate
----

==== Executing migrations via the Kill Bill Flywar Jar in Prod environments

In production environments, database access is often restricted and developers don’t necessarily have rights to execute DDL commands (i.e. CREATE, ALTER, DROP, etc. statements). In such cases, migrations can be executed as follows:

1. Create the `schema_history` table manually by running the following SQL statements:
+
[source, sql]
----
CREATE TABLE `<plugin_name>_schema_history` (
`installed_rank` int(11) NOT NULL,
`version` varchar(50) DEFAULT NULL,
`description` varchar(200) NOT NULL,
`type` varchar(20) NOT NULL,
`script` varchar(1000) NOT NULL,
`checksum` int(11) DEFAULT NULL,
`installed_by` varchar(100) NOT NULL,
`installed_on` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`execution_time` int(11) NOT NULL,
`success` tinyint(1) NOT NULL,
PRIMARY KEY (`installed_rank`),
KEY `schema_history_s_idx` (`success`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
insert into <plugin_name>_schema_history (installed_rank, version, description, type, script, installed_by, installed_on, execution_time, success) VALUES (1, 1, '<< Flyway Baseline >>', 'BASELINE', '<< Flyway Baseline >>', 'admin', NOW(), 0, 1);
----
+
Ensure that you replace `<plugin_name>` with the name of a plugin like `analytics`.
+
2. Run the `dryRunMigrate` command. This simply lists the DDL statements required for the database upgrade but does not actually run them.
+
*Command*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/<db_name> -user=<db_user> -password=<db_password> -locations=filesystem:<migrations_path> -table=<plugin_name>_schema_history dryRunMigrate
----
+
*Example*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/killbill -user=root -password=killbill -locations=filesystem:C:/var/migrations -table=analytics_schema_history dryRunMigrate
----


==== Executing migrations via the Flyway commandline tool

The migrations can also be executed via the https://documentation.red-gate.com/fd/command-line-184127404.html[Flyway commandline tool] as follows:

[source, bash]
----
flyway -url=jdbc:mysql://127.0.0.1:3306/<db_name> -user=<db_user> -password=<db_password> -table=<plugin_name>_schema_history baseline # first time only
flyway -url=jdbc:mysql://127.0.0.1:3306/<db_name> -user=<db_user> -password=<db_password> -locations=filesystem:<migrations_path> -table=<plugin_name>_schema_history migrate
----
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
[[step_1]]
=== Step 1 - Obtain the migrations to run

The `kpm migrate` command can be used to obtain database migrations.

*Command*

[source,bash]
----
kpm migrations github_repository from_tag to_tag
----

*Example*

To download the migration files to migrate from Kill Bill `0.22.32` to `0.24.0`:

[source,bash]
----
kpm migrations killbill killbill-0.22.32 killbill-0.24.0
----

This command creates a temporary folder with the migration SQL statements and displays its path.

[NOTE]
*Note:* Because the implementation relies on the GitHub API, unauthenticated requests are subject to rate limiting. To work around it, generate a token via https://github.com/settings/tokens (default public, read-only, permissions will work) and specify it to KPM via `kpm migrations killbill killbill-0.22.32 killbill-0.24.0 --token=TOKEN`

=== Step 2 - Execute the migrations

The migrations can be executed either manually or via https://flywaydb.org/[Flyway].

==== Executing migrations manually

Execute the SQL scripts obtained above directly in database.

==== Executing migrations via the Kill Bill Flywar Jar in Dev environments

We publish a https://repo1.maven.org/maven2/org/kill-bill/billing/killbill-util/0.24.4/killbill-util-0.24.4-flyway.jar[killbill-flyway.jar] jar which is a wrapper around the Flyway utility and can be used to execute the migrations. In order to execute the migrations via this jar, you can do the following:

1. If Flyway is being used the first time, run the `baseline` command to create the `schema_history` table.
+
*Command*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/<db_name> -user=<db_user> -password=<db_password> baseline
----
+
*Example*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/killbill -user=root -password=killbill baseline
----

+
2. Run the `migrate` command to execute the migrations.
+
*Command*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/<db_name> -user=<db_user> -password=<db_password> -locations=filesystem:<migrations_path> migrate
----
+
*Example*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/killbill -user=root -password=killbill -locations=filesystem:C:/var/migrations migrate
----

==== Executing migrations via the Kill Bill Flywar Jar in Prod environments

In production environments, database access is often restricted and developers don’t necessarily have rights to execute DDL commands (i.e. CREATE, ALTER, DROP, etc. statements). In such cases, migrations can be executed as follows:

1. Create the `schema_history` table manually by running the following SQL statements:
+
[source, sql]
----
CREATE TABLE `schema_history` (
`installed_rank` int(11) NOT NULL,
`version` varchar(50) DEFAULT NULL,
`description` varchar(200) NOT NULL,
`type` varchar(20) NOT NULL,
`script` varchar(1000) NOT NULL,
`checksum` int(11) DEFAULT NULL,
`installed_by` varchar(100) NOT NULL,
`installed_on` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`execution_time` int(11) NOT NULL,
`success` tinyint(1) NOT NULL,
PRIMARY KEY (`installed_rank`),
KEY `schema_history_s_idx` (`success`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
insert into schema_history (installed_rank, version, description, type, script, installed_by, installed_on, execution_time, success) VALUES (1, 1, '<< Flyway Baseline >>', 'BASELINE', '<< Flyway Baseline >>', 'admin', NOW(), 0, 1);
----
+
2. Run the `dryRunMigrate` command. This simply lists the DDL statements required for the database upgrade but does not actually run them.
+
*Command*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/<db_name> -user=<db_user> -password=<db_password> -locations=filesystem:<migrations_path> dryRunMigrate
----
+
*Example*
+
[source, bash]
----
java -jar killbill-flyway.jar -url=jdbc:mysql://127.0.0.1:3306/killbill -user=root -password=password -locations=filesystem:C:/var/migrations dryRunMigrate
----


==== Executing migrations via the Flyway commandline tool

The migrations can also be executed via the https://documentation.red-gate.com/fd/command-line-184127404.html[Flyway commandline tool] as follows:

[source, bash]
----
flyway -url=jdbc:mysql://127.0.0.1:3306/<db_name> -user=<db_user> -password=<db_password> baseline # first time only
flyway -url=jdbc:mysql://127.0.0.1:3306/<db_name> -user=<db_user> -password=<db_password> -locations=filesystem:<migrations_path> migrate
----
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,13 @@ KB_org_killbill_payment_plugin_timeout |Timeout for each payment attempt |30
KB_org_killbill_payment_plugin_threads_nb |Number of threads for plugin executor dispatcher |10 |Config File/Environment Variable
|Payment |org.killbill.payment.globalLock.retries/

KB_org_killbill_payment_globalLock_retries |Maximum number of times the system will retry to grab global lock (with a 100ms wait each time) |50 |Config File/Environment Variable
KB_org_killbill_payment_globalLock_retries |Maximum number of times the system will retry to grab global lock (with a 100ms wait each time) |50 |Config File/Environment Variable
|Payment |org.killbill.payment.method.overwrite/

KB_org_killbill_payment_method_overwrite |Ability to overwrite an existing payment method from a control plugin |false |Config File/Environment Variable
|Payment |org.killbill.payment.allow.emptyInvoice/

KB_org_killbill_payment_allow_emptyinvoices |Ability to allow payments for zero amount invoices |false |Config File/Environment Variable
|Invoice |org.killbill.invoice.maxNumberOfMonthsInFuture/

KB_org_killbill_invoice_maxNumberOfMonthsInFuture |Maximum target date to consider when generating an invoice |36 |Per-Tenant/Config File/Environment Variable
Expand Down Expand Up @@ -91,7 +97,10 @@ KB_org_killbill_invoice_parkAccountsWithUnknownUsage |Whether to park accounts
KB_org_killbill_invoice_rescheduleIntervalOnLock |Time delay to reschedule an invoice run when lock is held |0s |Per-Tenant/Config File/Environment Variable
|Invoice |org.killbill.invoice.maxInvoiceLimit/

KB_org_killbill_invoice_maxInvoiceLimit |How far back in time should invoice generation look at |P200Y |Per-Tenant/Config File/Environment Variable
KB_org_killbill_invoice_maxInvoiceLimit |How far back in time should invoice generation look at |P200Y |Per-Tenant/Config File/Environment Variable
|Invoice |org.killbill.rescheduleIntervalOnLock/

KB_org_killbill_rescheduleIntervalOnLock |Tme delay to reschedule an invoice run when lock is held |30s, 1m, 1m, 3m, 3m, 10m |Per-Tenant/Config File/Environment Variable
|Database |org.killbill.dao.url/

KB_org_killbill_dao_url |The jdbc url for the database |jdbc:h2:file:/var/tmp/killbill;MODE=MYSQL;
Expand Down Expand Up @@ -817,7 +826,14 @@ KB_org_killbill_security_skipAuthForPlugins |Specifies whether authentication
|Security |killbill.server.ldap/- |Coming Soon |false |System property
|Security |killbill.server.okta/- |Coming Soon |false |System property
|Security |killbill.server.auth0/- |Coming Soon |false |System Property
|Security |killbill.server.rbac/- |Coming Soon |true |System Property
|Security |killbill.server.rbac/- |Coming Soon |true |System Property
|Logging |killbill.server.log.obfuscate.keywords |Keywords to obfuscate |accountnumber,authenticationdata,bankaccountnumber,banknumber,bic,cardvalidationnum,cavv,ccFirstName,ccLastName,ccNumber,ccTrackData, ccVerificationValue,ccvv,cvNumber,cvc,cvv,email,iban,name,number,password, xid |System Property
|Logging |killbill.server.log.obfuscate.patterns |Key value patterns to use |\s*=\s*'([^']+)',\s*=\s*"([^"]+)" |System Property
|Logging |killbill.server.log.obfuscate.patterns.separator |Key value pattern separator |, |System Property
|Cache |org.killbill.catalog.frequentValuesCacheSize/- |Coming Soon |1000 |System property
|Cache |org.killbill.cache.disabled/
Expand Down
2 changes: 1 addition & 1 deletion userguide/tutorials/aws.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
<div class="card-body">
<h5 class="card-title">Multi-Tier single AMI with external database</h5>
<p class="card-text">Fully customizable production environment (advanced users)</p>
<a href="https://aws.amazon.com/marketplace/pp/B083LYVG9H?ref=_ptnr_doclanding_" onclick="getOutboundLink('https://aws.amazon.com/marketplace/pp/B083LYVG9H?ref=_ptnr_doclanding_'); return false;" class="btn btn-primary">Launch</a>
<a href="https://aws.amazon.com/marketplace/pp/prodview-rtgjip6tx3oea" onclick="getOutboundLink('https://aws.amazon.com/marketplace/pp/prodview-rtgjip6tx3oea'); return false;" class="btn btn-primary">Launch</a>
<p class="card-text"><a href="https://docs.killbill.io/latest/aws-multitier.html">Docs</a></p>
</div>
</div>
Expand Down
Loading

0 comments on commit f46edf6

Please sign in to comment.