forked from apache/doris-operator
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'apache:master' into disaggregated-be-decommission
- Loading branch information
Showing
3 changed files
with
65 additions
and
56 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,35 +1,29 @@ | ||
DorisCluster | [DorisDisaggregatedCluster](DISAGGREGATED-README-CN.md) | ||
中文 | [English](README.md) | ||
# doris-operator | ||
Doris-Operator 用于在 Kubernetes 上创建、配置和管理 Doris 集群,能够部署和管理 fe、be、cn、broker 所有组件。 | ||
## 特点 | ||
- 通过自定义 DorisCluster 资源创建Doris集群 | ||
- 提供定制化存储(PersistentVolumeClaim 模板) | ||
- 定制 pod 模板 | ||
- Doris 配置文件与组件解耦,灵活管理相关组件配置 | ||
- Doris 版本平滑升级 | ||
- 提供 HorizontalPodAutoscaler v1 和 v2 版本满足不同k8s环境计算节点的自动弹性扩缩容 | ||
- 通过自定义 DorisCluster 资源管控 Doris 集群。 | ||
- 提供定制化存储。 | ||
- 实现 Doris 在 Kubernetes 上无感升级。 | ||
- 提供服务在 crash 情况下,容器内 Debug 能力。 | ||
|
||
## 环境要求 | ||
- Kubernetes 1.19+ | ||
- Doris 的 FE 和 BE 组件正常启动至少需要8c和8G资源 | ||
|
||
## 安装 | ||
1. 安装 DorisCluster 资源定义: | ||
``` | ||
kubectl create -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/config/crd/bases/doris.apache.com_dorisclusters.yaml | ||
``` | ||
2. 安装 Doris-Operator 服务以及所依赖的 RBAC 权限等相关资源 | ||
2. 安装 Doris-Operator 服务以及所依赖的 RBAC 权限等相关资源: | ||
``` | ||
kubectl apply -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/config/operator/operator.yaml | ||
``` | ||
## 部署 Doris | ||
[部署范例](./doc/examples)中提供了一些使用Kubernetes特性部署 Doris 的范例。 | ||
默认的部署样例中,每个 fe 和 be 最少需要8核和16G的内存,且每个服务部署3个实例。 在使用默认部署之前,确保 K8s 集群有足够的资源能够部署成功。 | ||
部署使用容器自身存储(重启丢失数据属易失性介质)包含 fe,be 服务的 Doris 集群,命令如下: | ||
3. 在 Kubernetes 上部署 Doris 集群: | ||
``` | ||
kubectl apply -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/doc/examples/doriscluster-sample.yaml | ||
``` | ||
[doriscluster-sample-storageclass.yaml](./doc/examples/doriscluster-sample-storageclass.yaml) 展示使用 [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/) 模式提供存储卷部署 doris 的样例。 | ||
|
||
>[!WARNING] | ||
>1. 目前 Doris-operator 限定 Doris 在 Kubernetes 上必须使用 FQDN 模式启动和通信。 在使用 [DockerHub apache](https://hub.docker.com/?namespace=selectdb) 组织下的官方镜像部署时,`enable_fqdn_mode` 会被默认设置为 `true`。其他方式使用镜像时, fqdn 默认仍然是 false 。详细配置请参考文档 [example/doriscluster-sample-configmap.yaml](./doc/examples/doriscluster-sample-configmap.yaml)。 | ||
>2. 服务正常运行时,可以通过 `kubectl -n doris logs -f {pod_name}` 命令查看日志,也可以到容器内部的 `/opt/apache-doris/fe/log` 或 `/opt/apache-doris/be/log` 中查看日志。当 k8s 上没有日志处理系统时,建议为日志目录挂载存储盘,便于追溯较早的大量运行日志。为日志挂载存储盘可以参考文档 [example/doriscluster-sample-storageclass.yaml](./doc/examples/doriscluster-sample-storageclass.yaml)。 | ||
>1. 当定制化 FE 启动配置时,请设置 `enable_fqdn_mode=true`。请参考[官方文档](https://doris.apache.org/zh-CN/docs/3.0/install/cluster-deployment/k8s-deploy/compute-storage-coupled/install-quickstart)了解更详细的使用介绍。 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,41 +1,32 @@ | ||
DorisCluster | [DorisDisaggregatedCluster](DISAGGREGATED-README.md) | ||
|
||
English | [中文](README-CN.md) | ||
# doris-operator | ||
Doris-Operator for doris creates, configures and manages doris cluster running on kubernetes. Operator provide deploy and manage fe, be, cn,broker components. | ||
Users custom `DorisCluster` CRD to deploy doris as demand. | ||
|
||
## Features | ||
- Create Doris clusters by custom DorisCluster resource | ||
- Customized storage provisioning(VolumeClaim templates) | ||
- Customized pod templates | ||
- Doris configuration management | ||
- Doris version upgrades | ||
- Provided HorizontalPodAutoscaler v1 and v2 versions for compute node. | ||
- Realized Doris management by custom DorisCluster resource. | ||
- Customized storage provisioning. | ||
- Seamless upgrade Doris. | ||
- Provide the debug ability in container when the service crashed. | ||
|
||
## Requirements | ||
- Kubernetes 1.19+ | ||
- Doris's components need 8c cpu and 8G memory at least to normal start. | ||
|
||
## Installation | ||
1. Install custom resource definitions: | ||
``` | ||
```shell | ||
kubectl create -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/config/crd/bases/doris.apache.com_dorisclusters.yaml | ||
``` | ||
2. Install the operator with its RBAC rules: | ||
the default deployed namespace is doris, when deploy on specific namespace, please pull yaml and update `namespace` field. | ||
``` | ||
```shell | ||
kubectl apply -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/config/operator/operator.yaml | ||
``` | ||
## Get Started to Deploy Doris | ||
The [Quick Start Guide](./doc/examples) have some examples to deploy doris on kubernetes. they represent some mode to deploy doris on different situation. | ||
Example specify 8 cores and 16GB of memory for every fe or be, and deployed 3 fe and 3 be. Please confirm the K8s cluster have enough resources. | ||
for only deploy fe and be without persistentVolume: | ||
``` | ||
kubectl apply -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/doc/examples/doriscluster-sample.yaml | ||
3. Install Doris on Kubernetes: | ||
```shell | ||
kubectl apply -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/doc/examples/doriscluster-sample.yaml | ||
``` | ||
This [doriscluster-sample-storageclass.yaml](./doc/examples/doriscluster-sample-storageclass.yaml) displayed to deploy doris with [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/) mode to provide persistent Volume. | ||
|
||
>[!WARNING] | ||
>1. currently operator only supports the fqdn mode to deploy doris on kubernetes. when the operator uses the official image to deploy container, the relevant work service will set the `enable_fqdn_mode` as true automatically. by running the doris docker container without k8s-operator, fqdn mode is closed by default. for other configurations about deploying doris on kubernetes, refer to [example/doriscluster-sample-configmap.yaml](./doc/examples/doriscluster-sample-configmap.yaml). | ||
>2. fe and be print log by `kubectl logs -ndoris -f ${pod_name}` also in /opt/apache-doris/fe/log, /opt/apache-doris/be/log in pod. When have not log processing system on k8s, mount a volume for log directory is good idea. the config to mount volume for log can reference the doc[example/doriscluster-sample-storageclass.yaml](./doc/examples/doriscluster-sample-storageclass.yaml). | ||
>1. When custom the FE startup configuration, please set `enable_fqdn_mode=true`. Please refer to [the official doc](https://doris.apache.org/docs/3.0/install/cluster-deployment/k8s-deploy/compute-storage-coupled/install-config-cluster) for how to use. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters