Skip to content

Commit

Permalink
Merge dev into 5.x (#1929)
Browse files Browse the repository at this point in the history
Co-authored-by: NataliaIvakina <[email protected]>
Co-authored-by: Mark Dixon <[email protected]>
Co-authored-by: Bruno Buss <[email protected]>
Co-authored-by: Nick Giles <[email protected]>
Co-authored-by: gjmwoods <[email protected]>
Co-authored-by: Neil Dewhurst <[email protected]>
Co-authored-by: Lidia Zuin <[email protected]>
Co-authored-by: AlexicaWright <[email protected]>
Co-authored-by: Satia Herfert <[email protected]>
Co-authored-by: Jack Waudby <[email protected]>
Co-authored-by: Alexander Bouriakov <[email protected]>
Co-authored-by: Stefano Ottolenghi <[email protected]>
Co-authored-by: Jack Waudby <[email protected]>
Co-authored-by: Arne Fischereit <[email protected]>
Co-authored-by: Bastien Louërat <[email protected]>
Co-authored-by: Hannes Sandberg <[email protected]>
Co-authored-by: Aleksey Karasavov <[email protected]>
Co-authored-by: Lasse Heemann <[email protected]>
Co-authored-by: Love Kristofer Leifland <[email protected]>
Co-authored-by: Nathan Smith <[email protected]>
Co-authored-by: Phil Wright <[email protected]>
Co-authored-by: Balazs Lendvai <[email protected]>
Co-authored-by: Jenny <[email protected]>
Co-authored-by: Sum <[email protected]>
Co-authored-by: sumyiren <[email protected]>
Co-authored-by: David Pond <[email protected]>
Co-authored-by: Frannie-Ludmilla <[email protected]>
Co-authored-by: Fábio Botelho <[email protected]>
Co-authored-by: Stefanos Giagkiozis <[email protected]>
Co-authored-by: Nacho Cordon <[email protected]>
Co-authored-by: tselmeg <[email protected]>
Co-authored-by: Tony Butterfield <[email protected]>
Co-authored-by: Anna Sjerling <[email protected]>
Co-authored-by: Therese Magnusson <[email protected]>
Co-authored-by: Tselmeg Baasan <[email protected]>
Co-authored-by: Therese Magnusson <[email protected]>
Co-authored-by: Wilco <[email protected]>
Co-authored-by: Ragnar Wernersson <[email protected]>
Co-authored-by: Aurélien Arena <[email protected]>
Co-authored-by: Fi Quick <[email protected]>
Co-authored-by: Jens Pryce-Åklundh <[email protected]>
  • Loading branch information
Show file tree
Hide file tree
Showing 22 changed files with 441 additions and 61 deletions.
1 change: 0 additions & 1 deletion .github/workflows/docs-branch-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,3 @@ jobs:
if: ${{ inputs.lint || github.event_name == 'push' }}
name: Lint docs
uses: neo4j/docs-tools/.github/workflows/[email protected]

2 changes: 0 additions & 2 deletions .github/workflows/docs-deploy-surge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@ jobs:
- id: unzip-changelog
if: ${{ hashFiles('changelog.zip') != '' }}
run: unzip changelog.zip

- id: get-deploy-id
run: |
deployid=$(<deployid)
Expand All @@ -86,7 +85,6 @@ jobs:
run: |
deployurl=$ORG-$REPO-$DEPLOYID.surge.sh
echo "deploy-url=$deployurl" >> $GITHUB_OUTPUT
- uses: actions/setup-node@v4
with:
node-version: lts/*
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/docs-teardown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,4 +42,3 @@ jobs:
The preview documentation has now been torn down - reopening this PR will republish it.
GITHUB_TOKEN: ${{ secrets.DOCS_PR_COMMENT_TOKEN }}

6 changes: 3 additions & 3 deletions antora.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ nav:
asciidoc:
attributes:
neo4j-version: '5'
neo4j-version-minor: '5.24'
neo4j-version-exact: '5.24.1'
neo4j-buildnumber: '5.24'
neo4j-version-minor: '5.25'
neo4j-version-exact: '5.25.0'
neo4j-buildnumber: '5.25'
neo4j-debian-package-version: '1:5.22.0@'
1 change: 1 addition & 0 deletions modules/ROOT/content-nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@
** xref:backup-restore/modes.adoc[]
** xref:backup-restore/online-backup.adoc[]
** xref:backup-restore/aggregate.adoc[]
** xref:backup-restore/inspect.adoc[]
** xref:backup-restore/restore-backup.adoc[]
** xref:backup-restore/offline-backup.adoc[]
** xref:backup-restore/restore-dump.adoc[]
Expand Down
4 changes: 2 additions & 2 deletions modules/ROOT/pages/backup-restore/aggregate.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ Aggregates a chain of backup artifacts into a single artifact.
|Accepts either a path to a single artifact file or a folder containing backup artifacts.

When a file is supplied, the _<database>_ parameter should be omitted.
The option to supply a file is only available from Neo4j 5.2 onwards.
It is possible to aggregate backup artifacts from AWS S3 buckets, Google Cloud storage buckets, and Azure buckets using the appropriate URI as the path.
|

|-h, --help
Expand Down Expand Up @@ -106,7 +106,7 @@ For more information, see <<aggregate-backup-cloud-storage>>.

[NOTE]
====
Neo4j 5.24 introduces the `--temp-path` option to address potential issues related to disk space when performing backup-related commands, especially when cloud storage is involved.
Neo4j 5.24 introduces the `--temp-path` option to address potential issues related to disk space when performing backup-related commands, especially when cloud storage is involved.
If `--temp-path` is not set, a temporary directory is created inside the directory specified by the `--from-path` option.
Expand Down
1 change: 1 addition & 0 deletions modules/ROOT/pages/backup-restore/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ This chapter describes the following:
* xref:backup-restore/modes.adoc[Backup modes] -- The supported backup modes.
* xref:backup-restore/online-backup.adoc[Back up an online database] -- How to back up an online database.
* xref:backup-restore/aggregate.adoc[Aggregate a database backup chain] - How to aggregate a backup chain into a single backup.
* xref:backup-restore/inspect.adoc[Inspect the metadata of a database backup file] -- How to inspect the metadata of a database backup file.
* xref:backup-restore/restore-backup.adoc[Restore a database backup] -- How to restore a database backup in a live Neo4j deployment.
* xref:backup-restore/offline-backup.adoc[Back up an offline database] -- How to back up an offline database.
* xref:backup-restore/restore-dump.adoc[Restore a database dump] -- How to restore a database dump in a live Neo4j deployment.
Expand Down
236 changes: 236 additions & 0 deletions modules/ROOT/pages/backup-restore/inspect.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
[[inspect-backup]]
= Inspect the metadata of a backup file
:description: This section describes how to inspect the metadata of backup files. Metadata are information like the database name, the backup compression, the transaction range that the backup contains etc..
:page-role: enterprise-edition new-5.25

You can inspect the metadata of a database backup file using the `neo4j-admin backup inspect` command.

[[inspect-backup-command]]
== Command

The inspect command lists the metadata stored in the header of backup files.
This metadata primarily defines how backups are connected to form xref:backup-restore/online-backup.adoc#backup-chain[backup chains].
A backup chain is a sequence of one or more backup(s) logically connected.
The order of the sequence guarantees that when replayed (see xref:backup-restore/restore-backup.adoc[restore] or xref:backup-restore/aggregate.adoc[aggregate]), the store and the transaction data are consumed in a consistent manner.

The metadata contains the following information:

* *Database*: database name of the database fragment that the backup includes.
* *Database ID*: a unique identifier that distinguishes databases (even with the same name).
* *Time*: time the backup was taken.
* *Full*: indicates whether it is a full backup (i.e. initial backup containing the store files) or a differential backup (i.e. subsequent backup containing only the transactions to be applied to the store files).
* *Compressed*: indicates whether the backup data inside the backup file is compressed.
* *Lowest transaction ID*: when the backup is full, this value is always 1, and when it is a differential backup, the value corresponds to the first transaction ID the backup starts with.
* *Highest transaction ID*: similarly, this value indicates the last transaction ID stored in the backup file.

[[inspect-backup-syntax]]
=== Syntax

[source,role=noheader]
----
neo4j-admin backup inspect [-h] [--empty] [--expand-commands] [--latest-backup]
[--latest-chain] [--show-metadata] [--verbose]
[--additional-config=<file>] [--database=<database>]
[--format=<value>] <backup-path>
----

=== Description

Command to read the backup metadata.

[[inspect-backup-command-parameters]]
=== Parameters

.`neo4j-admin backup inspect` parameters
[options="header", cols="1m,3a"]
|===
| Parameter
| Description

|<backup-path>
|Path denoting either a directory where backups are stored or a single backup to inspect.
|===

[NOTE]
====
The `<backup-path>` parameter can also inspect backups stored in AWS S3 buckets (from Neo4j 5.19), Google Cloud storage buckets (from Neo4j 5.21), and Azure buckets (from Neo4j 5.24).
====

[[inspect-backup-command-options]]
=== Options

.`neo4j-admin backup inspect` options
[options="header", cols="5m,6a,4m"]
|===
| Option
| Description
| Default

|--additional-config=<file>
|Configuration file with additional configuration.
|

| --expand-commands
| Allow command expansion in config value evaluation.
|

|-h, --help
|Show this help message and exit.
|

| --latest-backup
| Show only the latest backup.
| false

| --latest-chain
| List the full backup chain ending with the latest downloaded backup.
| false

| --show-metadata
| Show the backup metadata.
| false

| --database=<database>
| Name of the database to inspect.
|

| --format=<value>
| Format of the output of the command. Possible values are: 'JSON, TABULAR'.
| TABULAR

| --empty
| Include empty backups.
| false

|--verbose
|Enable verbose output.
|
|===


[[aggregate-backup-example]]
== Examples

Given the folder _/backups_ containing a set of database backups:

[source,shell]
----
/backups
├── london-2024-10-07T16-03-51.backup
├── london-2024-10-07T16-04-05.backup
├── malmo-2024-10-07T16-00-07.backup
├── malmo-2024-10-07T16-00-19.backup
├── malmo-2024-10-07T16-00-34.backup
├── malmo-2024-10-07T16-00-44.backup
├── malmo-2024-10-07T16-00-50.backup
├── malmo-2024-10-07T16-01-08.backup
├── malmo-2024-10-07T16-01-24.backup
└── neo4j-2024-10-07T16-05-37.backup
----

=== Listing the metadata of the backup files

The following command lists the backup files' names along with their respective metadata:

[source,shell]
----
bin/neo4j-admin backup inspect /backups --show-metadata --empty
----

The `--empty` option is used to include the empty backups.
An empty backup is created when a database is backed up but no new data exists.
Empty backups are used to record the backup history.

.Example output
[result]
----
| FILE | DATABASE | DATABASE ID | TIME (UTC) | FULL | COMPRESSED | LOWEST TX | HIGHEST TX |
| file:///backups/neo4j-2024-10-07T16-05-37.backup | neo4j | 7dcb1d0c-4374-4476-b8ae-d3c3f124683f | 2024-10-07T16:05:37 | true | true | 1 | 3 |
| file:///backups/malmo-2024-10-07T16-01-24.backup | malmo | 62d1820c-3ac6-4b15-a0b3-bf7e7becc8d0 | 2024-10-07T16:01:24 | true | true | 1 | 8 |
| file:///backups/malmo-2024-10-07T16-01-08.backup | malmo | 62d1820c-3ac6-4b15-a0b3-bf7e7becc8d0 | 2024-10-07T16:01:08 | true | true | 1 | 7 |
| file:///backups/malmo-2024-10-07T16-00-50.backup | malmo | 62d1820c-3ac6-4b15-a0b3-bf7e7becc8d0 | 2024-10-07T16:00:50 | false | true | 0 | 0 |
| file:///backups/malmo-2024-10-07T16-00-44.backup | malmo | 62d1820c-3ac6-4b15-a0b3-bf7e7becc8d0 | 2024-10-07T16:00:44 | false | true | 7 | 7 |
| file:///backups/malmo-2024-10-07T16-00-34.backup | malmo | 62d1820c-3ac6-4b15-a0b3-bf7e7becc8d0 | 2024-10-07T16:00:34 | false | true | 6 | 6 |
| file:///backups/malmo-2024-10-07T16-00-19.backup | malmo | 62d1820c-3ac6-4b15-a0b3-bf7e7becc8d0 | 2024-10-07T16:00:19 | false | true | 0 | 0 |
| file:///backups/malmo-2024-10-07T16-00-07.backup | malmo | 62d1820c-3ac6-4b15-a0b3-bf7e7becc8d0 | 2024-10-07T16:00:07 | true | true | 1 | 5 |
| file:///backups/london-2024-10-07T16-04-05.backup | london | d4dae73c-dfef-4d28-88cd-fe6cc88ddca1 | 2024-10-07T16:04:05 | false | true | 6 | 6 |
| file:///backups/london-2024-10-07T16-03-51.backup | london | d4dae73c-dfef-4d28-88cd-fe6cc88ddca1 | 2024-10-07T16:03:51 | true | true | 1 | 5 |
----

=== Listing the latest backups

To list only the most recent backups performed for each database, use the `--latest-backup` option.

[source,shell]
----
bin/neo4j-admin backup inspect /backups --show-metadata --latest-backup
----

.Example output
[result]
----
| FILE | DATABASE | DATABASE ID | TIME (UTC) | FULL | COMPRESSED | LOWEST TX | HIGHEST TX |
| file:///backups/neo4j-2024-10-07T16-05-37.backup | neo4j | 7dcb1d0c-4374-4476-b8ae-d3c3f124683f | 2024-10-07T16:05:37 | true | true | 1 | 3 |
| file:///backups/malmo-2024-10-07T16-01-24.backup | malmo | 62d1820c-3ac6-4b15-a0b3-bf7e7becc8d0 | 2024-10-07T16:01:24 | true | true | 1 | 8 |
| file:///backups/london-2024-10-07T16-04-05.backup | london | d4dae73c-dfef-4d28-88cd-fe6cc88ddca1 | 2024-10-07T16:04:05 | false | true | 6 | 6 |
----

=== Inspecting backup chains

A backup chain corresponds to a sequence of one or more backup(s) logically connected by their transaction IDs.
To inspect the backup chains of a given database, use the `--latest-chain` option and the `--database` option with the database whose backup chain you want to inspect:

[source,shell]
----
bin/neo4j-admin backup inspect /backups --show-metadata --latest-chain --database=london
----

.Example output
[result]
----
| FILE | DATABASE | DATABASE ID | TIME (UTC) | FULL | COMPRESSED | LOWEST TX | HIGHEST TX |
| file:///backups/london-2024-10-07T16-04-05.backup | london | d4dae73c-dfef-4d28-88cd-fe6cc88ddca1 | 2024-10-07T16:04:05 | false | true | 6 | 6 |
| file:///backups/london-2024-10-07T16-03-51.backup | london | d4dae73c-dfef-4d28-88cd-fe6cc88ddca1 | 2024-10-07T16:03:51 | true | true | 1 | 5 |
----

The result returns a chain of size two:

* The first backup is a full backup containing the store files within the transaction range [1,5].
* The second backup is a differential backup containing only the subsequent modifications to the store files.
Those modifications are materialised by a sequence of transactions to apply.
Its range is [6,6].


=== Inspecting a backup chain ending with a specific backup

To inspect a backup chain ending with a specific backup, use the `--latest-chain` option as follows:

[source,shell]
----
bin/neo4j-admin backup inspect /backups/london-2024-10-07T16-04-05.backup --show-metadata --latest-chain
----

.Example output
[result]
----
| FILE | DATABASE | DATABASE ID | TIME (UTC) | FULL | COMPRESSED | LOWEST TX | HIGHEST TX |
| file:///backups/london-2024-10-07T16-04-05.backup | london | d4dae73c-dfef-4d28-88cd-fe6cc88ddca1 | 2024-10-07T16:04:05 | false | true | 6 | 6 |
| file:///backups/london-2024-10-07T16-03-51.backup | london | d4dae73c-dfef-4d28-88cd-fe6cc88ddca1 | 2024-10-07T16:03:51 | true | true | 1 | 5 |
----

[NOTE]
====
In this case, the `--database` option is unnecessary because the database identifier is part of the metadata stored in the header of the backup file _london-2024-10-07T16-04-05.backup_.
====












1 change: 1 addition & 0 deletions modules/ROOT/pages/backup-restore/offline-backup.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ The `neo4j-admin database dump` command has the following options:

|--to-path=<path>
|Destination folder of a database dump.
It is possible to dump databases into AWS S3 buckets, Google Cloud storage buckets, and Azure buckets using the appropriate URI as the path.
|

|--to-stdout
Expand Down
4 changes: 2 additions & 2 deletions modules/ROOT/pages/backup-restore/online-backup.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ Note: this is an EXPERIMENTAL option. Consult Neo4j support before use.
|

|--to-path=<path>
|Directory to place backup in (required unless `--inspect-path` is used).
|Directory to place backup in (required unless `--inspect-path` is used). It is possible to back up databases into AWS S3 buckets, Google Cloud storage buckets, and Azure using the appropriate URI as the path.
|

|--type=<type>
Expand All @@ -196,7 +196,7 @@ For more information, see <<online-backup-cloud-storage>>.

[NOTE]
====
Neo4j 5.24 introduces the `--temp-path` option to address potential issues related to disk space when performing backup-related commands, especially when cloud storage is involved.
Neo4j 5.24 introduces the `--temp-path` option to address potential issues related to disk space when performing backup-related commands, especially when cloud storage is involved.
If `--temp-path` is not set, a temporary directory is created inside the directory specified by the `--path` option.
Expand Down
3 changes: 2 additions & 1 deletion modules/ROOT/pages/backup-restore/restore-backup.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ neo4j-admin database restore [-h] [--expand-commands]
|--from-path=<path>[,<path>...]
|A single path or a comma-separated list of paths pointing to a backup artifact file.
An artifact file can be 1) a full backup, in which case it is restored directly or, 2) a differential backup, in which case the command tries first to find in the folder a backup chain ending at that specific differential backup and then restores that chain.
It is possible to restore backups from AWS S3 buckets, Google Cloud storage buckets, and Azure buckets using the appropriate URI as the path.
|

|-h, --help
Expand Down Expand Up @@ -130,7 +131,7 @@ For more information, see <<restore-cloud-storage>>.

[NOTE]
====
Neo4j 5.24 introduces the `--temp-path` option to address potential issues related to disk space when performing backup-related commands, especially when cloud storage is involved.
Neo4j 5.24 introduces the `--temp-path` option to address potential issues related to disk space when performing backup-related commands, especially when cloud storage is involved.
If `--temp-path` is not set, a temporary directory is created inside the directory specified by the `--from-path` option.
Expand Down
1 change: 1 addition & 0 deletions modules/ROOT/pages/backup-restore/restore-dump.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ If `--info` is specified, then the database is not loaded, but information (i.e.

|--from-path=<path>
|Path to directory containing archive(s).
It is possible to load databases from AWS S3 buckets, Google Cloud storage buckets, and Azure bucket using the appropriate URI as the path.
|

|--from-stdin
Expand Down
Loading

0 comments on commit b109926

Please sign in to comment.