Skip to content

Commit

Permalink
[docs](update) EN Docs Update (apache#499)
Browse files Browse the repository at this point in the history
  • Loading branch information
KassieZ authored Apr 1, 2024
1 parent 94a1a15 commit 6496d22
Show file tree
Hide file tree
Showing 22 changed files with 868 additions and 659 deletions.
2 changes: 1 addition & 1 deletion i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0.json
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@
"message": "查询缓存",
"description": "The label for category Pipeline in sidebar docs"
},
"sidebar.docs.category.Create View and Materialize View": {
"sidebar.docs.category.View and Materialize View": {
"message": "视图与物化视图",
"description": "The label for category View and Materialize View in sidebar docs"
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,36 +38,6 @@ Doris 升级请遵守**不要跨两个及以上关键节点版本升级**的原
关键节点版本:升级时必须要经历的版本,可能是单独一个版本,也可能是一个版本区间,如 `1.1.3 - 1.1.5`,则表示升级至该区间任意一版本即可继续后续升级。
:::


| 版本号 | 关键节点版本 | LTS 版本 |
| ------------------------ | ------------ | -------- |
| 0.12.x |||
| 0.13.x |||
| 0.14.x |||
| 0.15.x |||
| 1.0.0 - 1.1.2 |||
| 1.1.3 - 1.1.5 || 1.1-LTS |
| 1.2.0 - 1.2.5 || 1.2-LTS |
| 2.0.0-alpha - 2.0.0-beta || 2.0-LTS |

示例:

当前版本为 `0.12`,升级到 `2.0.0-beta` 版本的升级路线

`0.12` -> `0.13` -> `0.14` -> `0.15` -> `1.1.3 - 1.1.5` 任意版本 -> `1.2.0 - 1.2.5` 任意版本 -> `2.0.0-beta`

:::caution

- LTS 版本:Long-time Support,LTS 版本提供长期支持,会持续维护六个月以上,通常而言,**版本号第三位数越大的版本,稳定性越好**

- Alpha 版本:内部测试版本,功能还未完全确定,或许存在重大 BUG,只推荐上测试集群做测试,**不推荐上生产集群!**

- Beta 版本:公开测试版本,功能已基本确定,或许存在非重大 BUG,只推荐上测试集群做测试,**不推荐上生产集群!**

- Release 版本:公开发行版,已完成基本重要 BUG 的修复和功能性缺陷修复验证,推荐上生产集群。

:::

## 升级步骤

### 升级说明
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
{
"title": "Apache Ranger",
"title": "基于 Apache Ranger 的鉴权管理",
"language": "zh-CN"
}
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
{
"title": "SSL 密钥证书配置",
"title": "MySQL 安全传输",
"language": "zh-CN"
}
---
Expand Down Expand Up @@ -80,6 +80,6 @@ openssl pkcs12 -inkey ca-key.pem -in ca.pem -export -out ca_certificate.p12
openssl pkcs12 -inkey server-key.pem -in server.pem -export -out server_certificate.p12
```

:::note
:::info Note
[参考文档](https://www.ibm.com/docs/en/api-connect/2018.x?topic=overview-generating-self-signed-certificate-using-openssl)
:::
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
{
"title": "load balancing",
"title": "Load Balancing",
"language": "en"
}

Expand All @@ -25,7 +25,7 @@ specific language governing permissions and limitations
under the License.
-->

# load balancing


When deploying multiple FE nodes, users can deploy a load balancing layer on top of multiple FEs to achieve high availability of Doris.

Expand Down Expand Up @@ -83,7 +83,7 @@ tcp 0 0 0.0.0.0:6033 0.0.0.0:* LISTEN

ProxySQL has a configuration file `/etc/proxysql.cnf` and a configuration database file `/var/lib/proxysql/proxysql.db`. **Special attention is needed here**: If there is a `"proxysql.db"` file (under the `/var/lib/proxysql` directory), the ProxySQL service will only be read when it is started for the first time The `proxysql.cnf file` and parse it; after startup, the `proxysql.cnf` file will not be read! If you want the configuration in the proxysql.cnf file to take effect after restarting the proxysql service (that is, you want proxysql to read and parse the proxysql.cnf configuration file when it restarts), you need to delete `/var/lib/proxysql/proxysql first. db`database file, and then restart the proxysql service. This is equivalent to initializing the proxysql service, and a pure proxysql.db database file will be produced again (if proxysql related routing rules, etc. are configured before, it will be erased)

#### View and modify configuration files
**View and modify configuration files**

Here are mainly a few parameters, which have been commented out below, and you can modify them according to your needs

Expand Down Expand Up @@ -138,7 +138,7 @@ mysql_replication_hostgroups=
)
```

#### Connect to the ProxySQL management port test
**Connect to the ProxySQL management port test**

```sql
# mysql -uadmin -padmin -P6032 -hdoris01
Expand Down Expand Up @@ -187,7 +187,7 @@ MySQL [main]> show tables;
20 rows in set (0.000 sec)
```

#### ProxySQL configuration backend Doris FE
**ProxySQL configuration backend Doris FE**

Use the insert statement to add the host to the mysql_servers table, where: hostgroup_id is 10 for the write group, and 20 for the read group. We don't need to read and write the license here, and it doesn't matter which one can be set randomly.

Expand Down Expand Up @@ -261,7 +261,7 @@ MySQL [(none)]> save mysql servers to disk;
Query OK, 0 rows affected (0.348 sec)
```

#### Monitor Doris FE node configuration
**Monitor Doris FE node configuration**

After adding doris fe nodes, you also need to monitor these back-end nodes. For multiple FE high-availability load balancing environments on the backend, this is necessary because ProxySQL needs to be automatically adjusted by the read_only value of each node

Expand Down Expand Up @@ -349,7 +349,7 @@ MySQL [(none)]> select hostgroup_id,hostname,port,status,weight from mysql_serve
3 rows in set (0.000 sec)
```

#### Configure Doris users
**Configure Doris users**

All the above configurations are about the back-end Doris FE node. Now you can configure the SQL statements, including: the user who sends the SQL statement, the routing rules of the SQL statement, the cache of the SQL query, the rewriting of the SQL statement, and so on.

Expand Down Expand Up @@ -426,7 +426,7 @@ Query OK, 0 rows affected (0.123 sec)
In this way, you can use the doris username and password to connect to ProxySQL through the sql client
```

#### Connect to Doris through ProxySQL for testing
**Connect to Doris through ProxySQL for testing**

Next, use the root user and doris user to test whether they can be routed to the default hostgroup_id=10 (it is a write group) to read data. The following is connected through the forwarding port 6033, the connection is forwarded to the real back-end database!

Expand Down Expand Up @@ -543,7 +543,7 @@ cd /usr/local/nginx
/usr/local/nginx/sbin/nginx -c conf.d/default.conf
```

### verify
### Verify

```
mysql -uroot -P6030 -h172.31.7.119
Expand Down
104 changes: 39 additions & 65 deletions versioned_docs/version-2.0/admin-manual/cluster-management/upgrade.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
{
"title": "Cluster upgrade",
"title": "Cluster Upgrade",
"language": "en"
}
---
Expand All @@ -24,7 +24,6 @@ specific language governing permissions and limitations
under the License.
-->

# Cluster upgrade

## Overview

Expand All @@ -33,68 +32,45 @@ To upgrade, please use the steps recommended in this chapter to upgrade the clus
## Doris Release Notes

:::tip
When upgrading Doris, please follow the principle of **not skipping two minor versions** and upgrade sequentially.

For Doris upgrade, please follow the principle of **Do not upgrade across two or more key node versions**. If you want to upgrade across multiple key node versions, first upgrade to the nearest key node version, and then upgrade in turn. If it is not critical node version, it can be ignored and skipped.

Key node version: the version that must be experienced when upgrading, it may be a single version, or a version range, such as `1.1.3 - 1.1.5`, it means that you can continue to upgrade after upgrading to any version in this range .

For example, if you are upgrading from version 0.15.x to 2.0.x, it is recommended to first upgrade to the latest version of 1.1, then upgrade to the latest version of 1.2, and finally upgrade to the latest version of 2.0.
:::

| Version number | Key node version | LTS version |
| ------------------------ | ------------ | -------- |
| 0.12.x | Yes | No |
| 0.13.x | Yes | No |
| 0.14.x | Yes | No |
| 0.15.x | Yes | No |
| 1.0.0 - 1.1.2 | No | No |
| 1.1.3 - 1.1.5 | Yes | 1.1-LTS |
| 1.2.0 - 1.2.5 | Yes | 1.2-LTS |
| 2.0.0-alpha - 2.0.0-beta | Yes | 2.0-LTS |

Example:

The current version is `0.12`, upgrade route to `2.0.0-beta` version

`0.12` -> `0.13` -> `0.14` -> `0.15` -> `1.1.3 - 1.1.5` any version -> `1.2.0 - 1.2.5` any version -> `2.0.0 -beta`

:::tip

LTS version: Long-time Support, LTS version provides long-term support and will be maintained for more than six months. Generally speaking, the version with the larger third digit of the version number is more stable**.

Alpha version: an internal test version, the function has not been fully determined, and there may be major bugs. It is only recommended to use the test cluster for testing, ** it is not recommended to use the production cluster! **

Beta version: public test version, the function has been basically confirmed, there may be non-major bugs, it is only recommended to use the test cluster for testing, ** it is not recommended to use the production cluster! **

Release version: a public release version, which has completed the repair of basic important bugs and verification of functional defect fixes, and is recommended for production clusters.

:::

## Upgrade steps
## Upgrade Steps

### Upgrade Instructions

1. During the upgrade process, since Doris's RoutineLoad, Flink-Doris-Connector, and Spark-Doris-Connector have implemented a retry mechanism in the code, in a multi-BE node cluster, the rolling upgrade will not cause the task to fail .

2. The StreamLoad task requires you to implement a retry mechanism in your own code, otherwise the task will fail.

3. The cluster copy repair and balance function must be closed before and opened after the completion of a single upgrade task, regardless of whether all your cluster nodes have been upgraded.

### Overview of the upgrade process
### Overview of the Upgrade Process

1. Metadata backup

2. Turn off the cluster copy repair and balance function

3. Compatibility testing

4. Upgrade BE

5. Upgrade FE

6. Turn on the cluster replica repair and balance function

### Upgrade pre-work
### Upgrade Pre-work

Please perform the upgrade in sequence according to the upgrade process

#### metadata backup (important)
**01 Metadata Backup (Important)**

** Make a full backup of the `doris-meta` directory of the FE-Master node! **
**Make a full backup of the `doris-meta` directory of the FE-Master node!**

#### Turn off the cluster replica repair and balance function
**02 Turn off the cluster replica repair and balance function**

There will be node restart during the upgrade process, so unnecessary cluster balancing and replica repair logic may be triggered, first close it with the following command:

Expand All @@ -104,34 +80,32 @@ admin set frontend config("disable_colocate_balance" = "true");
admin set frontend config("disable_tablet_scheduler" = "true");
```

#### Compatibility testing
**03 Compatibility Testing**

:::tip
:::caution Warning

**Metadata compatibility is very important, if the upgrade fails due to incompatible metadata, it may lead to data loss! It is recommended to perform a metadata compatibility test before each upgrade! **
**Metadata compatibility is very important, if the upgrade fails due to incompatible metadata, it may lead to data loss! It is recommended to perform a metadata compatibility test before each upgrade!**

:::

##### FE Compatibility Test

:::tip

**important**
1. FE Compatibility Test

:::tip Important
1. It is recommended to do FE compatibility test on your local development machine or BE node.

2. It is not recommended to test on Follower or Observer nodes to avoid link exceptions
3. If it must be on the Follower or Observer node, the started FE process needs to be stopped

3. If it must be on the Follower or Observer node, the started FE process needs to be stopped
:::

1. Use the new version alone to deploy a test FE process

a. Use the new version alone to deploy a test FE process

```shell
sh ${DORIS_NEW_HOME}/bin/start_fe.sh --daemon
```

2. Modify the FE configuration file fe.conf for testing
b. Modify the FE configuration file fe.conf for testing

```shell
vi ${DORIS_NEW_HOME}/conf/fe.conf
Expand All @@ -149,9 +123,9 @@ admin set frontend config("disable_tablet_scheduler" = "true");
...
```

save and exit
Save and exit

3. Modify fe.conf
c. Modify fe.conf

- Add ClusterID configuration in fe.conf

Expand All @@ -164,50 +138,51 @@ admin set frontend config("disable_tablet_scheduler" = "true");
echo "metadata_failure_recovery=true" >> ${DORIS_NEW_HOME}/conf/fe.conf
```

4. Copy the metadata directory doris-meta of the online environment Master FE to the test environment
d. Copy the metadata directory doris-meta of the online environment Master FE to the test environment

```shell
cp ${DORIS_OLD_HOME}/fe/doris-meta/* ${DORIS_NEW_HOME}/fe/doris-meta
```

5. Change the cluster_id in the VERSION file copied to the test environment to 123456 (that is, the same as in step 3)
e. Change the cluster_id in the VERSION file copied to the test environment to 123456 (that is, the same as in step 3)

```shell
vi ${DORIS_NEW_HOME}/fe/doris-meta/image/VERSION
clusterId=123456
```

6. In the test environment, run the startup FE
f. In the test environment, run the startup FE

- If the version is greater than or equal to 2.0.2, run the following command

```shell
sh ${DORIS_NEW_HOME}/bin/start_fe.sh --daemon --metadata_failure_recovery
```

- If the version is less than 2.0.2, run the following command
```shell
sh ${DORIS_NEW_HOME}/bin/start_fe.sh --daemon
```

7. Observe whether the startup is successful through the FE log fe.log
g. Observe whether the startup is successful through the FE log fe.log

```shell
tail -f ${DORIS_NEW_HOME}/log/fe.log
```

8. If the startup is successful, it means that there is no problem with the compatibility, stop the FE process of the test environment, and prepare for the upgrade
h. If the startup is successful, it means that there is no problem with the compatibility, stop the FE process of the test environment, and prepare for the upgrade

```
sh ${DORIS_NEW_HOME}/bin/stop_fe.sh
```

##### BE Compatibility Test
2. BE Compatibility Test

You can use the grayscale upgrade scheme to upgrade a single BE first. If there is no exception or error, the compatibility is considered normal, and subsequent upgrade actions can be performed

### Upgrade process

:::tip
:::tip Tip

Upgrade BE first, then FE

Expand All @@ -223,10 +198,9 @@ However, when a major version is upgraded, new features may be added or old func

#### Upgrade BE

:::tip
:::tip Tip

In order to ensure the safety of your data, please use 3 copies to store your data to avoid data loss caused by misoperation or failure of the upgrade

:::

1. Under the premise of multiple copies, select a BE node to stop running and perform grayscale upgrade
Expand Down Expand Up @@ -265,9 +239,9 @@ In order to ensure the safety of your data, please use 3 copies to store your da

6. Complete the upgrade of other BE nodes in sequence

#### Upgrade FE
**05 Upgrade FE**

:::tip
:::tip Tip

Upgrade the non-Master nodes first, and then upgrade the Master nodes.

Expand Down Expand Up @@ -309,7 +283,7 @@ Upgrade the non-Master nodes first, and then upgrade the Master nodes.

6. Complete the upgrade of other FE nodes in turn, **finally complete the upgrade of the Master node**

#### Turn on the cluster replica repair and balance function
**06 Turn on the cluster replica repair and balance function**

After the upgrade is complete and all BE nodes become `Alive`, enable the cluster copy repair and balance function:

Expand Down
Loading

0 comments on commit 6496d22

Please sign in to comment.