MySQL Master-Slave-Slave replication in Docker

README.md

@@ -1,76 +1,126 @@
 Docker MySQL master-slave replication 
 ========================
 
-MySQL master-slave replication with using Docker. 
+MySQL master-slave-slave replication with using Docker.
 
-## Run
+### Preparations
 
-To run this examples you will need to start containers with "docker-compose" 
-and after starting setup replication. See commands inside ./build.sh. 
+Start test environment: `docker-compose up -d`
 
-#### Create 2 MySQL containers with master-slave row-based replication 
+Install python requirements: `pip install -r requirements.txt`
 
-```
-./build.sh
-```
+Run master configs: `python3 master.py`
 
-#### Make changes to master
+Run generator: `python3 generator.py`, that will insert values into our table every 5 seconds.
 
 ```
-docker exec mysql_master sh -c "export MYSQL_PWD=111; mysql -u root mydb -e 'create table code(code int); insert into code values (100), (200)'"
-```
+$ mysql -h 127.0.0.1 -P 3306 -u root -p
 
-#### Read changes from slave
+mysql> SELECT count(*) FROM number;
++----------+
+| count(*) |
++----------+
+|       11 |
++----------+
+1 row in set (0.00 sec)
 
-```
-docker exec mysql_slave sh -c "export MYSQL_PWD=111; mysql -u root mydb -e 'select * from code \G'"
-```
+mysql> FLUSH TABLES WITH READ LOCK;
+Query OK, 0 rows affected (0.01 sec)
 
-## Troubleshooting
+mysql> SHOW MASTER STATUS;
++------------------+----------+--------------+------------------+-------------------+
+| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
++------------------+----------+--------------+------------------+-------------------+
+| mysql-bin.000003 |     4415 | mydb         |                  |                   |
++------------------+----------+--------------+------------------+-------------------+
+1 row in set (0.00 sec)
 
-#### Check Logs
+# mysqldump -u root -p mydb > mydb.sql
+Enter password: 
 
+$ docker cp master:mydb.sql .
+
+On master: UNLOCK TABLES;
 ```
-docker-compose logs
+
+### Configure mysql_slave_replica1
 ```
+docker cp mydb.sql mysql_slave_replica:/
 
-#### Start containers in "normal" mode
+mysql> CREATE DATABASE mydb;
 
-> Go through "build.sh" and run command step-by-step.
+# mysql -u root -p mydb < mydb.sql
+Enter password: 
 
-#### Check running containers
+mysql> use mydb
+mysql> SELECT count(*) FROM number;
++----------+
+| count(*) |
++----------+
+|       11 |
++----------+
+1 row in set (0.00 sec)
 
-```
-docker-compose ps
-```
+mysql> CHANGE MASTER TO MASTER_HOST='10.0.0.10',MASTER_USER='slave',MASTER_PASSWORD='password',MASTER_LOG_FILE = 'mysql-bin.000003',MASTER_LOG_POS=4415,GET_MASTER_PUBLIC_KEY=1;
+Query OK, 0 rows affected, 8 warnings (0.09 sec)
 
-#### Clean data dir
+mysql> START SLAVE;
+Query OK, 0 rows affected, 1 warning (0.01 sec)
 
-```
-rm -rf ./master/data/*
-rm -rf ./slave/data/*
-```
+mysql> SELECT count(*) FROM number;
++----------+
+| count(*) |
++----------+
+|      213 |
++----------+
+1 row in set (0.01 sec)
 
-#### Run command inside "mysql_master"
+```
 
+### Configure mysql_slave_replica2
+All steps the same as for mysql_slave_replica2. Replica start successfully.
 ```
-docker exec mysql_master sh -c 'mysql -u root -p111 -e "SHOW MASTER STATUS \G"'
+mysql> SELECT count(*) FROM number;
++----------+
+| count(*) |
++----------+
+|      266 |
++----------+
+1 row in set (0.00 sec)
 ```
 
-#### Run command inside "mysql_slave"
-
+### Turn off mysql_slave_replica1
 ```
-docker exec mysql_slave sh -c 'mysql -u root -p111 -e "SHOW SLAVE STATUS \G"'
+$ docker stop mysql_slave_replica1
+slave1
 ```
-
-#### Enter into "mysql_master"
-
+Check on master:
 ```
-docker exec -it mysql_master bash
+mysql> select count(*) from number;
++----------+
+| count(*) |
++----------+
+|      349 |
++----------+
+1 row in set (0.00 sec)
 ```
+Check on mysql_slave_replica2:
+```
+mysql> SELECT count(*) FROM number;
++----------+
+| count(*) |
++----------+
+|      349 |
++----------+
+1 row in set (0.00 sec)
+```
+As we can see nothing changed in our chain.
 
-#### Enter into "mysql_slave"
-
+### Try to remove column on mysql_slave_replica2 replica
 ```
-docker exec -it mysql_slave bash
+mysql> ALTER TABLE number DROP COLUMN number;
+Query OK, 0 rows affected (0.28 sec)
+Records: 0  Duplicates: 0  Warnings: 0
+
 ```
+Our `number` column was removed, but `id` column still exist and get new data from master.

build.sh

@@ -15,9 +15,15 @@ done
 priv_stmt='GRANT REPLICATION SLAVE ON *.* TO "mydb_slave_user"@"%" IDENTIFIED BY "mydb_slave_pwd"; FLUSH PRIVILEGES;'
 docker exec mysql_master sh -c "export MYSQL_PWD=111; mysql -u root -e '$priv_stmt'"
 
-until docker-compose exec mysql_slave sh -c 'export MYSQL_PWD=111; mysql -u root -e ";"'
+until docker-compose exec mysql_slave_replica1 sh -c 'export MYSQL_PWD=111; mysql -u root -e ";"'
 do
-    echo "Waiting for mysql_slave database connection..."
+    echo "Waiting for mysql_slave_replica1 database connection..."
+    sleep 4
+done
+
+until docker-compose exec mysql_slave_replica2 sh -c 'export MYSQL_PWD=111; mysql -u root -e ";"'
+do
+    echo "Waiting for mysql_slave_replica2 database connection..."
     sleep 4
 done
 
@@ -33,6 +39,10 @@ start_slave_stmt="CHANGE MASTER TO MASTER_HOST='$(docker-ip mysql_master)',MASTE
 start_slave_cmd='export MYSQL_PWD=111; mysql -u root -e "'
 start_slave_cmd+="$start_slave_stmt"
 start_slave_cmd+='"'
-docker exec mysql_slave sh -c "$start_slave_cmd"
+docker exec mysql_slave_replica1 sh -c "$start_slave_cmd"
+
+docker exec mysql_slave_replica1 sh -c "export MYSQL_PWD=111; mysql -u root -e 'SHOW SLAVE STATUS \G'"
+
+docker exec mysql_slave_replica2 sh -c "$start_slave_cmd"
 
-docker exec mysql_slave sh -c "export MYSQL_PWD=111; mysql -u root -e 'SHOW SLAVE STATUS \G'"
+docker exec mysql_slave_replica2 sh -c "export MYSQL_PWD=111; mysql -u root -e 'SHOW SLAVE STATUS \G'"

docker-compose.yml

@@ -1,11 +1,11 @@
-version: '3'
+version: '3.7'
 services:
   mysql_master:
-    image: mysql:5.7
+    image: mysql:8
     env_file:
       - ./master/mysql_master.env
     container_name: "mysql_master"
-    restart: "no"
+    restart: unless-stopped
     ports:
       - 4406:3306
     volumes:
@@ -14,19 +14,35 @@ services:
     networks:
       - overlay
 
-  mysql_slave:
-    image: mysql:5.7
+  mysql_slave1:
+    image: mysql:8
     env_file:
       - ./slave/mysql_slave.env
-    container_name: "mysql_slave"
-    restart: "no"
+    container_name: "mysql_slave_replica1"
+    restart: unless-stopped
     ports:
       - 5506:3306
     depends_on:
       - mysql_master
     volumes:
-      - ./slave/conf/mysql.conf.cnf:/etc/mysql/conf.d/mysql.conf.cnf
-      - ./slave/data:/var/lib/mysql
+      - ./slave1/conf/mysql.conf.cnf:/etc/mysql/conf.d/mysql.conf.cnf
+      - ./slave1/data:/var/lib/mysql
+    networks:
+      - overlay
+
+  mysql_slave2:
+    image: mysql:8
+    env_file:
+      - ./slave2/mysql_slave.env
+    container_name: "mysql_slave_replica2"
+    restart: unless-stopped
+    ports:
+      - 5507:3306
+    depends_on:
+      - mysql_master
+    volumes:
+      - ./slave2/conf/mysql.conf.cnf:/etc/mysql/conf.d/mysql.conf.cnf
+      - ./slave2/data:/var/lib/mysql
     networks:
       - overlay
 

generator.py

@@ -0,0 +1,19 @@
+from mysql.connector import connect, Error
+import time
+
+try:
+    with connect(
+        host="127.0.0.1",
+        user="root",
+        password="password",
+    ) as connection:
+        use_db = "USE mydb"
+        insert_query = "INSERT INTO number(number) VALUES (rand()*1024)"
+        with connection.cursor() as cursor:
+            cursor.execute(use_db)
+            for i in range(1000):
+                cursor.execute(insert_query)
+                connection.commit()
+                time.sleep(5)
+except Error as e:
+    print(e)

master.py

@@ -0,0 +1,28 @@
+from mysql.connector import connect, Error
+
+try:
+    with connect(
+        host="127.0.0.1",
+        user="root",
+        password="password",
+    ) as connection:
+        create_db = "CREATE DATABASE mydb"
+        use_db = "USE mydb"
+        create_slave_user = "CREATE USER 'slave'@'%' IDENTIFIED BY 'password'"
+        grant_permissions = "GRANT REPLICATION SLAVE ON *.* TO 'slave'@'%'"
+        flush_privileges = "FLUSH PRIVILEGES"
+        create_table_query = """
+        CREATE TABLE number(
+            id INT AUTO_INCREMENT PRIMARY KEY,
+            number INT
+        )
+        """
+        with connection.cursor() as cursor:
+            cursor.execute(create_db)
+            cursor.execute(use_db)
+            cursor.execute(create_slave_user)
+            cursor.execute(grant_permissions)
+            cursor.execute(flush_privileges)
+            cursor.execute(create_table_query)
+except Error as e:
+    print(e)

master/conf/mysql.conf.cnf

@@ -1,9 +1,15 @@
-[mysqld]
-
-skip-host-cache
-skip-name-resolve
+[client]
+port=3306
 
-server-id = 1
-log_bin = /var/log/mysql/mysql-bin.log
-binlog_format = ROW
-binlog_do_db = mydb
+[mysqld]
+server-id=1
+port=3306
+socket=/var/run/mysqld/mysqld.sock
+key_buffer_size=16M
+max_allowed_packet=128M
+pid-file=/var/run/mysqld/mysqld.pid
+datadir=/var/lib/mysql
+secure-file-priv=NULL
+log_bin=/var/run/mysqld/mysql-bin.log
+binlog_format=ROW
+binlog_do_db=mydb

requirements.txt

@@ -0,0 +1 @@
+mysql-connector-python==8.0.29

slave1/conf/mysql.conf.cnf

@@ -0,0 +1,16 @@
+[client]
+port=3306
+
+[mysqld]
+server-id=2
+port=3306
+socket=/var/run/mysqld/mysqld.sock
+key_buffer_size=16M
+max_allowed_packet=128M
+pid-file=/var/run/mysqld/mysqld.pid
+datadir=/var/lib/mysql
+secure-file-priv=NULL
+relay-log=/var/run/mysqld/mysql-relay-bin.log
+log_bin=/var/run/mysqld/mysql-bin.log
+binlog_format=ROW
+binlog_do_db=mydb

slave2/conf/mysql.conf.cnf

@@ -0,0 +1,16 @@
+[client]
+port=3306
+
+[mysqld]
+server-id=3
+port=3306
+socket=/var/run/mysqld/mysqld.sock
+key_buffer_size=16M
+max_allowed_packet=128M
+pid-file=/var/run/mysqld/mysqld.pid
+datadir=/var/lib/mysql
+secure-file-priv=NULL
+relay-log=/var/run/mysqld/mysql-relay-bin.log
+log_bin=/var/run/mysqld/mysql-bin.log
+binlog_format=ROW
+binlog_do_db=mydb

slave2/mysql_slave.env

@@ -0,0 +1,7 @@
+# Note, by default mysql root does not have a password. You need to restart a server to bring MYSQL_ROOT_PASSWORD working. Use "docker-compose restart" command.
+MYSQL_ROOT_PASSWORD=111
+MYSQL_PORT=3306
+MYSQL_USER=mydb_slave_user
+MYSQL_PASSWORD=mydb_slave_pwd
+MYSQL_DATABASE=mydb
+MYSQL_LOWER_CASE_TABLE_NAMES=0