Skip to content

Commit

Permalink
Merge branch 'apache:master' into disaggregated-be-decommission
Browse files Browse the repository at this point in the history
  • Loading branch information
catpineapple authored Dec 10, 2024
2 parents ffbc771 + c304420 commit 8d5c5d3
Show file tree
Hide file tree
Showing 3 changed files with 65 additions and 56 deletions.
24 changes: 9 additions & 15 deletions README-CN.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,29 @@
DorisCluster | [DorisDisaggregatedCluster](DISAGGREGATED-README-CN.md)
中文 | [English](README.md)
# doris-operator
Doris-Operator 用于在 Kubernetes 上创建、配置和管理 Doris 集群,能够部署和管理 fe、be、cn、broker 所有组件。
## 特点
- 通过自定义 DorisCluster 资源创建Doris集群
- 提供定制化存储(PersistentVolumeClaim 模板)
- 定制 pod 模板
- Doris 配置文件与组件解耦,灵活管理相关组件配置
- Doris 版本平滑升级
- 提供 HorizontalPodAutoscaler v1 和 v2 版本满足不同k8s环境计算节点的自动弹性扩缩容
- 通过自定义 DorisCluster 资源管控 Doris 集群。
- 提供定制化存储。
- 实现 Doris 在 Kubernetes 上无感升级。
- 提供服务在 crash 情况下,容器内 Debug 能力。

## 环境要求
- Kubernetes 1.19+
- Doris 的 FE 和 BE 组件正常启动至少需要8c和8G资源

## 安装
1. 安装 DorisCluster 资源定义:
```
kubectl create -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/config/crd/bases/doris.apache.com_dorisclusters.yaml
```
2. 安装 Doris-Operator 服务以及所依赖的 RBAC 权限等相关资源
2. 安装 Doris-Operator 服务以及所依赖的 RBAC 权限等相关资源
```
kubectl apply -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/config/operator/operator.yaml
```
## 部署 Doris
[部署范例](./doc/examples)中提供了一些使用Kubernetes特性部署 Doris 的范例。
默认的部署样例中,每个 fe 和 be 最少需要8核和16G的内存,且每个服务部署3个实例。 在使用默认部署之前,确保 K8s 集群有足够的资源能够部署成功。
部署使用容器自身存储(重启丢失数据属易失性介质)包含 fe,be 服务的 Doris 集群,命令如下:
3. 在 Kubernetes 上部署 Doris 集群:
```
kubectl apply -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/doc/examples/doriscluster-sample.yaml
```
[doriscluster-sample-storageclass.yaml](./doc/examples/doriscluster-sample-storageclass.yaml) 展示使用 [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/) 模式提供存储卷部署 doris 的样例。

>[!WARNING]
>1. 目前 Doris-operator 限定 Doris 在 Kubernetes 上必须使用 FQDN 模式启动和通信。 在使用 [DockerHub apache](https://hub.docker.com/?namespace=selectdb) 组织下的官方镜像部署时,`enable_fqdn_mode` 会被默认设置为 `true`。其他方式使用镜像时, fqdn 默认仍然是 false 。详细配置请参考文档 [example/doriscluster-sample-configmap.yaml](./doc/examples/doriscluster-sample-configmap.yaml)
>2. 服务正常运行时,可以通过 `kubectl -n doris logs -f {pod_name}` 命令查看日志,也可以到容器内部的 `/opt/apache-doris/fe/log``/opt/apache-doris/be/log` 中查看日志。当 k8s 上没有日志处理系统时,建议为日志目录挂载存储盘,便于追溯较早的大量运行日志。为日志挂载存储盘可以参考文档 [example/doriscluster-sample-storageclass.yaml](./doc/examples/doriscluster-sample-storageclass.yaml)
>1. 当定制化 FE 启动配置时,请设置 `enable_fqdn_mode=true`。请参考[官方文档](https://doris.apache.org/zh-CN/docs/3.0/install/cluster-deployment/k8s-deploy/compute-storage-coupled/install-quickstart)了解更详细的使用介绍。
29 changes: 10 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,32 @@
DorisCluster | [DorisDisaggregatedCluster](DISAGGREGATED-README.md)

English | [中文](README-CN.md)
# doris-operator
Doris-Operator for doris creates, configures and manages doris cluster running on kubernetes. Operator provide deploy and manage fe, be, cn,broker components.
Users custom `DorisCluster` CRD to deploy doris as demand.

## Features
- Create Doris clusters by custom DorisCluster resource
- Customized storage provisioning(VolumeClaim templates)
- Customized pod templates
- Doris configuration management
- Doris version upgrades
- Provided HorizontalPodAutoscaler v1 and v2 versions for compute node.
- Realized Doris management by custom DorisCluster resource.
- Customized storage provisioning.
- Seamless upgrade Doris.
- Provide the debug ability in container when the service crashed.

## Requirements
- Kubernetes 1.19+
- Doris's components need 8c cpu and 8G memory at least to normal start.

## Installation
1. Install custom resource definitions:
```
```shell
kubectl create -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/config/crd/bases/doris.apache.com_dorisclusters.yaml
```
2. Install the operator with its RBAC rules:
the default deployed namespace is doris, when deploy on specific namespace, please pull yaml and update `namespace` field.
```
```shell
kubectl apply -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/config/operator/operator.yaml
```
## Get Started to Deploy Doris
The [Quick Start Guide](./doc/examples) have some examples to deploy doris on kubernetes. they represent some mode to deploy doris on different situation.
Example specify 8 cores and 16GB of memory for every fe or be, and deployed 3 fe and 3 be. Please confirm the K8s cluster have enough resources.
for only deploy fe and be without persistentVolume:
```
kubectl apply -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/doc/examples/doriscluster-sample.yaml
3. Install Doris on Kubernetes:
```shell
kubectl apply -f https://raw.githubusercontent.com/apache/doris-operator/$(curl -s https://api.github.com/repos/apache/doris-operator/releases/latest | grep tag_name | cut -d '"' -f4)/doc/examples/doriscluster-sample.yaml
```
This [doriscluster-sample-storageclass.yaml](./doc/examples/doriscluster-sample-storageclass.yaml) displayed to deploy doris with [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/) mode to provide persistent Volume.

>[!WARNING]
>1. currently operator only supports the fqdn mode to deploy doris on kubernetes. when the operator uses the official image to deploy container, the relevant work service will set the `enable_fqdn_mode` as true automatically. by running the doris docker container without k8s-operator, fqdn mode is closed by default. for other configurations about deploying doris on kubernetes, refer to [example/doriscluster-sample-configmap.yaml](./doc/examples/doriscluster-sample-configmap.yaml).
>2. fe and be print log by `kubectl logs -ndoris -f ${pod_name}` also in /opt/apache-doris/fe/log, /opt/apache-doris/be/log in pod. When have not log processing system on k8s, mount a volume for log directory is good idea. the config to mount volume for log can reference the doc[example/doriscluster-sample-storageclass.yaml](./doc/examples/doriscluster-sample-storageclass.yaml).
>1. When custom the FE startup configuration, please set `enable_fqdn_mode=true`. Please refer to [the official doc](https://doris.apache.org/docs/3.0/install/cluster-deployment/k8s-deploy/compute-storage-coupled/install-config-cluster) for how to use.
68 changes: 46 additions & 22 deletions pkg/common/utils/resource/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,16 @@
package resource

import (
"strings"

dv1 "github.com/apache/doris-operator/api/disaggregated/v1"
"github.com/apache/doris-operator/api/doris/v1"
v1 "github.com/apache/doris-operator/api/doris/v1"
"github.com/apache/doris-operator/pkg/common/utils/hash"
"github.com/apache/doris-operator/pkg/common/utils/set"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/util/intstr"
"k8s.io/klog/v2"
"strings"
)

// HashService service hash components
Expand Down Expand Up @@ -254,27 +255,39 @@ func getMetaServiceContainerPorts(config map[string]interface{}) []corev1.Contai
}

func getFeContainerPorts(config map[string]interface{}) []corev1.ContainerPort {
return []corev1.ContainerPort{{
Name: GetPortKey(HTTP_PORT),
ContainerPort: GetPort(config, HTTP_PORT),
Protocol: corev1.ProtocolTCP,
}, {
Name: GetPortKey(RPC_PORT),
ContainerPort: GetPort(config, RPC_PORT),
Protocol: corev1.ProtocolTCP,
}, {
Name: GetPortKey(QUERY_PORT),
ContainerPort: GetPort(config, QUERY_PORT),
Protocol: corev1.ProtocolTCP,
}, {
Name: GetPortKey(EDIT_LOG_PORT),
ContainerPort: GetPort(config, EDIT_LOG_PORT),
Protocol: corev1.ProtocolTCP,
}}
}
ports := []corev1.ContainerPort{
{
Name: GetPortKey(HTTP_PORT),
ContainerPort: GetPort(config, HTTP_PORT),
Protocol: corev1.ProtocolTCP,
}, {
Name: GetPortKey(RPC_PORT),
ContainerPort: GetPort(config, RPC_PORT),
Protocol: corev1.ProtocolTCP,
}, {
Name: GetPortKey(QUERY_PORT),
ContainerPort: GetPort(config, QUERY_PORT),
Protocol: corev1.ProtocolTCP,
}, {
Name: GetPortKey(EDIT_LOG_PORT),
ContainerPort: GetPort(config, EDIT_LOG_PORT),
Protocol: corev1.ProtocolTCP,
},
}

arrowFlightPort := GetPort(config, ARROW_FLIGHT_SQL_PORT)
if arrowFlightPort != -1 {
ports = append(ports, corev1.ContainerPort{
Name: GetPortKey(ARROW_FLIGHT_SQL_PORT),
ContainerPort: arrowFlightPort,
Protocol: corev1.ProtocolTCP,
})
}

return ports
}
func getBeContainerPorts(config map[string]interface{}) []corev1.ContainerPort {
return []corev1.ContainerPort{
ports := []corev1.ContainerPort{
{
Name: GetPortKey(BE_PORT),
ContainerPort: GetPort(config, BE_PORT),
Expand All @@ -292,6 +305,17 @@ func getBeContainerPorts(config map[string]interface{}) []corev1.ContainerPort {
Protocol: corev1.ProtocolTCP,
},
}

arrowFlightPort := GetPort(config, ARROW_FLIGHT_SQL_PORT)
if arrowFlightPort != -1 {
ports = append(ports, corev1.ContainerPort{
Name: GetPortKey(ARROW_FLIGHT_SQL_PORT),
ContainerPort: arrowFlightPort,
Protocol: corev1.ProtocolTCP,
})
}

return ports
}

func getBrokerContainerPorts(config map[string]interface{}) []corev1.ContainerPort {
Expand Down Expand Up @@ -326,7 +350,7 @@ func GetPortKey(configKey string) string {
case BRPC_LISTEN_PORT:
return "brpc-port"
case ARROW_FLIGHT_SQL_PORT:
return "arrow-flight-port"
return "arrow-flight"
default:
return ""
}
Expand Down

0 comments on commit 8d5c5d3

Please sign in to comment.