diff --git a/README-CN.md b/README-CN.md index ae8742ae..43cf20fa 100644 --- a/README-CN.md +++ b/README-CN.md @@ -324,6 +324,7 @@ ROS 模板的示例和最佳实践。模板分类如下： | [data-migration/low-cost-link-to-business-data.yml](./solutions/data-migration/low-cost-link-to-business-data.yml) | 本实践适用于提供业务数据搬迁上云服务。业务数据量一般都比较大，迁移上云需要大量的网络带宽，BGP费用比较高。 阿里云对用户开放所需地域购买静态单线共享带宽包的权限（移动/联通/电信均可），可用为迁移数据有效降低成本。 | | [data-migration/self-built-hive-data-warehouse-migrated-to-emr.yml](./solutions/data-migration/self-built-hive-data-warehouse-migrated-to-emr.yml) | 本方案利用ROS创建自建Hadoop集群的数据迁移到阿里云自建Hadoop或者EMR。 | | [database/polardb-migration-from-rds.yml](./solutions/database/polardb-migration-from-rds.yml) | 解决RDS MySQL或自建MySQL遇到的性能或容量瓶颈，迁移上下PolarDB链路打通。 | +| [database/deploy-mysql-based-on-ebs.yml](./solutions/database/deploy-mysql-based-on-ebs.yml) | 基于EBS ESSD部署高性能的MySQL服务。 | | [devops/container-application-devops-for-ack-cluster.yml](./solutions/devops/container-application-devops-for-ack-cluster.yml) | DevOps的目的是构建一种文化和环境，使构建，测试，发布软件更加快捷，频繁和可靠。而到了容器时代，需要部署的机器不但量更大，变化更剧烈，有的甚至需要根据条件自动升缩，为了满足企业敏捷的需求，持续部署也成了必须，本方案使用云效完成容器应用（小程序后端服务）的自动化构建和持续部署。 | | [enterprise-on-cloud/create-ddh-and-deploy-cloud-server.yml](./solutions/enterprise-on-cloud/create-ddh-and-deploy-cloud-server.yml) | 介绍本地部署或托管在IDC环境的 VMware系统迁移上云至独立宿主机（DDH）的最佳实践。使用DDH在云端构建由独享物理服务器组成的资源池，同时配合ECS成熟稳定的虚拟化技术体系，充分利用云上资源弹性、按使用付费的优势，快速构建高性能、高可靠和可快速动态伸缩的虚拟化系统，满足安全、合规、自定义部署、自带许可证（BYOL）等企业级需求；此模板完成创建DDH并部署云服务器。 | | [enterprise-on-cloud/e-commerce-business-and-db-on-the-cloud.yml](./solutions/enterprise-on-cloud/e-commerce-business-and-db-on-the-cloud.yml) | 此模板可创建出电商资源建站和数据库迁移方案所需资源，属于企业上云的最佳实践之一。 | diff --git a/README.md b/README.md index a858922b..8ed0a397 100644 --- a/README.md +++ b/README.md @@ -359,6 +359,7 @@ Examples and best practices of ROS templates. The templates are categorized as f | [serviceless-compute/fc-web-file-backend-service.yml](./solutions/serviceless-compute/fc-web-file-backend-service.yml) | This template is used to build website file processing services of the serverless architecture. You can use APIs, SDKs, or Alibaba Cloud Management Console to upload files to specified OSS buckets or update files. These files can be automatically compressed or decompressed, and you can obtain information such as MD5 hashed values and file metadata. | | [studio-entertainment/game-business-zone-service-env.yml](./solutions/studio-entertainment/game-business-zone-service-env.yml) | This template is used to accelerate user access and enable partitioning and aggregation of gaming data. This helps you improve user game experience and increase retention rates and profitability. | | [studio-entertainment/light-weight-gpu-deployed-cloud-games.yml](./solutions/studio-entertainment/light-weight-gpu-deployed-cloud-games.yml) | With the widespread application of AI in scenarios such as deep learning, video processing, scientific computing, and graphic visualization, the market demand for GPU instances is growing. However, the problem of excess computing power and rising costs caused by ultra-large-scale elastic computing starting out with a single physical GPU is becoming increasingly prominent. The emergence of lightweight GPUs has broken the limitations of the traditional pass-through mode. A lightweight GPU can provide finer-grained services than a physical GPU, allowing customers to conduct business more flexibly at lower costs. | +| [database/deploy-mysql-based-on-ebs.yml](./solutions/database/deploy-mysql-based-on-ebs.yml) | Deploy high-performance MySQL services based on EBS ESSD. |
diff --git a/solutions/database/deploy-mysql-based-on-ebs.yml b/solutions/database/deploy-mysql-based-on-ebs.yml new file mode 100644 index 00000000..d59257e0 --- /dev/null +++ b/solutions/database/deploy-mysql-based-on-ebs.yml @@ -0,0 +1,458 @@ +ROSTemplateFormatVersion: '2015-09-01' +Description: + zh-cn: 基于EBS ESSD部署高性能的MySQL服务 + en: Deploy high-performance MySQL services based on EBS ESSD. +Parameters: + DataDiskType: + Default: cloud_essd + Type: String + Label: + zh-cn: 数据盘磁盘类型 + en: Data Disk Category + MySQLPassword: + Default: '' + NoEcho: true + Type: String + Label: + zh-cn: 数据库密码 + en: Mysql Password + EcsType: + Default: ecs.g6.large + AssociationProperty: 'ALIYUN::ECS::Instance::InstanceType' + AssociationPropertyMetadata: + SystemDiskCategory: cloud_essd + InstanceChargeType: PostPaid + ZoneId: '${ZoneId}' + Type: String + Label: + zh-cn: ECS 实例规格 + en: InstanceType + ZoneId: + Default: '' + AssociationProperty: 'ALIYUN::ECS::Instance::ZoneId' + Type: String + Label: + zh-cn: 可用区 + en: Availability Zone + SysDiskType: + Default: cloud_essd + Type: String + Label: + zh-cn: 系统盘磁盘类型 + en: System Disk Category + EcsPassword: + Type: String + Description: + zh-cn: >- + 服务器登录密码,长度8-30，必须包含三项（大写字母、小写字母、数字、 ()`~!@#$%^&*_-+=|{}[]:;'<>,.?/ + 中的特殊符号） + en: >- + Server login password, Length 8-30, must contain three(Capital letters, + lowercase letters, numbers, ()`~!@#$%^&*_-+=|{}[]:;'<>,.?/ Special + symbol in) + Default: null + MinLength: 8 + Label: + zh-cn: 实例密码 + en: Instance Password + AllowedPattern: '^[a-zA-Z0-9-\(\)\`\~\!\@\#\$\%\^\&\*\_\-\+\=\|\{\}\[\]\:\;\<\>\,\.\?\/]*$' + MaxLength: 30 + AssociationProperty: 'ALIYUN::ECS::Instance::Password' + ConstraintDescription: + zh-cn: '长度8-30，必须包含三项（大写字母、小写字母、数字、 ()`~!@#$%^&*_-+=|{}[]:;''<>,.?/ 中的特殊符号）' + en: >- + Length 8-30, must contain three(Capital letters, lowercase letters, + numbers, ()`~!@#$%^&*_-+=|{}[]:;'<>,.?/ Special symbol in) +Outputs: + EcsLoginAddress: + Description: + zh-cn: Ecs登陆地址。 + en: Ecs login address. + Value: + 'Fn::Sub': + - >- + https://ecs-workbench.aliyun.com/?from=EcsConsole&instanceType=ecs®ionId=${Region}&instanceId=${InstanceId} + - InstanceId: + Ref: EcsInstance + Region: + Ref: 'ALIYUN::Region' + EcsInstanceId: + Description: ECS实例ID + Value: + 'Fn::GetAtt': + - EcsInstance + - InstanceId + EcsInstanceUser: + Description: ECS实例初始用户 + Value: root + EcsInstancePublicIp: + Description: ECS实例公网IP + Value: + 'Fn::GetAtt': + - EcsInstance + - PublicIp + MysqlUser: + Description: MySQL初始用户 + Value: root + DataDisk: + Description: 数据盘ID + Value: + 'Fn::GetAtt': + - DataDisk + - DiskId +Resources: + DataDiskAttachment: + Type: 'ALIYUN::ECS::DiskAttachment' + Properties: + InstanceId: + Ref: EcsInstance + DiskId: + Ref: DataDisk + DependsOn: + - EcsInstance + - DataDisk + SecurityGroup: + Type: 'ALIYUN::ECS::SecurityGroup' + Properties: + SecurityGroupIngress: + - Priority: 1 + PortRange: '-1/-1' + NicType: intranet + SourceCidrIp: 0.0.0.0/0 + IpProtocol: all + VpcId: + Ref: VPC + SecurityGroupEgress: + - Priority: 1 + PortRange: '-1/-1' + DestCidrIp: 0.0.0.0/0 + NicType: intranet + IpProtocol: all + VPC: + Type: 'ALIYUN::ECS::VPC' + Properties: + VpcName: + 'Fn::Join': + - '-' + - - StackId + - Ref: 'ALIYUN::StackId' + CidrBlock: 192.168.0.0/16 + WriteCommand: + Type: 'ALIYUN::ECS::RunCommand' + Properties: + Type: RunShellScript + CommandContent: + 'Fn::Join': + - '' + - - > + echo ' + + # 1. Get data disk label + + DEV_LABEL=vdb + + + # 2. Check if data disk exists + + if [ ! -e /dev/${DEV_LABEL} ]; then + echo "[ERROR] /dev/${DEV_LABEL} not exist. Please check if data disk label is vdb." + exit + fi + + + # 3. Set MySQL home + + MYSQL_HOME="/root/${DEV_LABEL}" + + if [ ! -d ${MYSQL_HOME} ]; then + if mkdir -p ${MYSQL_HOME}; then + echo "[INFO] Create directory ${MYSQL_HOME} successfully." + else + echo "[ERROR] Failed to create directory ${MYSQL_HOME}." + exit + fi + fi + + echo "[INFO] Using /dev/${DEV_LABEL} as the MySQL data home." + + + # 4. Prevent virtio io split + + echo 256 >/sys/block/${DEV_LABEL}/queue/max_sectors_kb + + echo "[INFO] Set max sectors kb successfully." + + + # 5. mkfs with bigalloc enabled + + umount /dev/${DEV_LABEL} + + echo y | mkfs.ext4 -O bigalloc -C 16k /dev/${DEV_LABEL} + + echo "[INFO] mkfs with bigalloc enabled successfully." + + + # 6. Install MySQL + + cd /tmp/ + + if wget -N + http://mirrors.cloud.aliyuncs.com/mysql/MySQL-8.0/mysql-8.0.27-1.el7.x86_64.rpm-bundle.tar + -O mysql.tar ; then + echo "[INFO] Download mysql rpm bundle successfully." + else + echo "[ERROR] Failed to download mysql rpm bundle." + exit + fi + + tar -xf mysql.tar + + echo "[INFO] Extract mysql rpm bundle successfully." + + + if yum install -y + mysql-community-{server,client,common,libs,devel}-*; then + echo "[INFO] Install mysql successfully." + else + echo "[ERROR] Failed to install mysql." + exit + fi + + + # 7. Create MySQL directories + + mount /dev/${DEV_LABEL} ${MYSQL_HOME} + + mkdir -p ${MYSQL_HOME}/data/dbs + + mkdir -p ${MYSQL_HOME}/data/mysql + + mkdir -p ${MYSQL_HOME}/log/mysql + + mkdir -p ${MYSQL_HOME}/log/redo + + echo "[INFO] Create MySQL directories successfully." + + + # 8. Add mysql group and user + + systemctl stop mysqld + + groupadd mysql + + useradd -g mysql mysql + + chown -R mysql:mysql ${MYSQL_HOME} + + echo "[INFO] Add mysql group and user successfully." + + + # 9. Turn off doublewrite + + cat </etc/my.cnf + + EOF + + + # 10. Init MySQL SQL + + cat <${MYSQL_HOME}/prepare.sql + + EOF + + + # 11. Start MySQL + + killall -9 mysqld + + mysqld --defaults-file=/etc/my.cnf --initialize + + mysqld --user=root --daemonize + + mysql -uroot -p"$(cat ${MYSQL_HOME}/log/mysql/master-error.log | + grep root | grep "temporary password" | awk '\'{print \$13}\'' | + xargs -n 1 echo)" --connect-expired-password + <${MYSQL_HOME}/prepare.sql + + echo '[INFO] Prepare MySQL successfully.' + + ' > /tmp/prepare_mysql.sh + + + echo '#!/bin/bash + + # chkconfig: 345 20 80 + + # description: EBS MySQL 16K Auto Start Script + + start() { + DEV_LABEL=vdb + MYSQL_HOME=/root/${DEV_LABEL} + echo 256 >/sys/block/vdb/queue/max_sectors_kb + umount /dev/${DEV_LABEL} + mount /dev/${DEV_LABEL} ${MYSQL_HOME} + mysqld --user=root --daemonize + } + + stop() { + pgrep mysqld | xargs kill + echo "Stopping mysqld..." + } + + restart() { + stop + sleep 2 + start + } + + case "$1" in + start) + start + ;; + stop) + stop + ;; + restart) + restart + ;; + *) + echo "Usage: $0 {start|stop|restart}" + exit 1 + esac + + exit 0 + + ' > /etc/rc.d/init.d/ebs_mysql_16k_auto_start.sh + + cd /etc/rc.d/init.d + + chmod +x /etc/rc.d/init.d/ebs_mysql_16k_auto_start.sh + + sudo chkconfig --add ebs_mysql_16k_auto_start.sh + + sudo chkconfig ebs_mysql_16k_auto_start.sh on + Sync: true + InstanceIds: + - Ref: EcsInstance + DependsOn: + - DataDiskAttachment + VSwitch: + Type: 'ALIYUN::ECS::VSwitch' + Properties: + VSwitchName: + Ref: 'ALIYUN::StackName' + VpcId: + Ref: VPC + CidrBlock: 192.168.0.0/24 + ZoneId: + Ref: ZoneId + RunCommand: + Type: 'ALIYUN::ECS::RunCommand' + Properties: + Type: RunShellScript + CommandContent: + 'Fn::Sub': + - > + #!/bin/sh + + sed -i "s/#{MYSQL_PASSWORD}/${MySQLPassword}/g; + s/#{MYSQL_PORT}/3306/g;" /tmp/prepare_mysql.sh + + sh /tmp/prepare_mysql.sh | tee ~/prepare_mysql.log + - {} + Sync: true + InstanceIds: + - Ref: EcsInstance + Timeout: 600 + DependsOn: + - WriteCommand + EcsInstance: + Type: 'ALIYUN::ECS::Instance' + Properties: + SystemDiskCategory: + Ref: SysDiskType + VpcId: + Ref: VPC + InternetMaxBandwidthOut: 100 + SecurityGroupId: + Ref: SecurityGroup + SystemDiskSize: 40 + ImageId: CentOS7_9 + AllocatePublicIP: true + VSwitchId: + Ref: VSwitch + InternetChargeType: PayByTraffic + IoOptimized: optimized + Password: + Ref: EcsPassword + InstanceType: + Ref: EcsType + ZoneId: + Ref: ZoneId + DataDisk: + Type: 'ALIYUN::ECS::Disk' + Properties: + DiskCategory: + Ref: DataDiskType + ZoneId: + Ref: ZoneId + PerformanceLevel: PL0 + Size: 100 + DependsOn: + - EcsInstance +Metadata: + 'ALIYUN::ROS::Interface': + ParameterGroups: + - Parameters: + - ZoneId + - EcsType + - EcsPassword + - SysDiskType + - DataDiskType + Label: + zh-cn: 基础配置 + en: Basic Configuration + - Parameters: + - MySQLPassword + Label: + zh-cn: 数据库配置 + en: DataBase Configuration + TemplateTags: + - 'acs:solution:基于EBS部署高性能的MySQL服务' \ No newline at end of file