The post MySQL 8.0 Group Replication Limitations appeared first on The WebScale Database Infrastructure Operations Experts.
]]>MySQL Group Replication can operate in both single-primary mode with automatic primary election, where only one server accepts updates at a time and multi-primary mode, where all servers can accept updates, even if they are issued concurrently. The built-in group membership service retains view of the group consistent and available for all servers at any given point in time, Servers can leave and join the group and the view is updated accordingly. If servers leave the group unexpectedly, the failure detection mechanism detects this and notifies the group that the view has changed, All this happens automatically !
For any transaction to commit, the majority of group members have to agree on the order of a given transaction in the global sequence of transactions. Decision to commit or abort a transaction is taken by respective servers, but all servers make same decision. In the case of systems built on network partition, When the members of the group are unable to reach agreement on a transaction, the system halt till the issue is resolved, This guarantees split-brain protection mechanism.
All these fault tolerance mechanisms are powered by Group Communication System (GCS) protocols, a group membership service, and completely safe ordered message delivery system. Group Replication is powered by Paxos algorithm (https://en.wikipedia.org/wiki/Paxos_(computer_science)), which acts as the group communication engine.
Till now, we have just posted about the capabilities of Group Replication, Now we have mentioned below (in bullets) about all the limitations of Group Replication:
The post MySQL 8.0 Group Replication Limitations appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post MySQL Replication simplified with GTID – Step-by-step GTID replication setup appeared first on The WebScale Database Infrastructure Operations Experts.
]]>I use MySQL replication extensively, It’s simple and robust, It’s also one main reason I love MySQL lot compared to other database systems. I basically use MySQL replication for performance, scalability, high availability, reliability, failover, fault tolerance etc. I have written blogs in the past on MySQL Master-Slave ( https://minervadb.com/index.php/2018/01/24/mysql-master-slave-replication-on-ubuntu-xenial-16-04/ & https://minervadb.com/index.php/2018/01/24/mysql-master-slave-replication-on-ubuntu-xenial-16-04-2/ ), Master-Master replication (https://minervadb.com/index.php/2018/01/24/mysql-master-slave-replication-on-ubuntu-xenial-16-04-2/) , delayed replication (https://minervadb.com/index.php/2018/01/29/how-to-lag-a-mysql-slave-behind-to-avoid-a-data-corruption/) etc. MySQL introduced global transaction identifiers (GTIDs) in MySQL 5.6.5 to make failover better and easy, GTID is a global transaction identifier which consists of two entities separated by a colon (:) , GTID looks like this – {source_id:transaction_id} / {c03c4200-0659-11e8-94ab-0800279747b5:1-2}, The source_id is server’s UUID and transaction_id is a sequence number . I have explained below step-by-step GTID based chained replication, One master and two slaves
[mysqld] !includedir /etc/mysql/conf.d/ !includedir /etc/mysql/mysql.conf.d/ server-id=900 log-bin=mysql-bin bind-address=192.168.56.15 binlog-do-db=employees gtid-mode = on enforce-gtid-consistency log-slave-updates
In this blog I have used mysqldump for backup so enabled system variable by setting “read_only” to ON
mysql> set global read_only=on; Query OK, 0 rows affected (0.00 sec)
root@VBOX110:/home/shiv# mysqldump --all-databases --single-transaction --triggers --routines --events --user=root --password=HardPassword/1947 > backup.sql
Once successfully completed back-up please disable system variable by setting “read_only” to OFF
mysql> set global read_only=off; Query OK, 0 rows affected (0.00 sec)
[mysqld] !includedir /etc/mysql/conf.d/ !includedir /etc/mysql/mysql.conf.d/ server-id=910 log-bin=mysql-bin skip-slave-start gtid-mode=on enforce-gtid-consistency log-slave-updates
!includedir /etc/mysql/conf.d/ !includedir /etc/mysql/mysql.conf.d/ server-id=930 log-bin=mysql-bin skip-slave-start gtid-mode=on enforce-gtid-consistency log-slave-updates
We need to create two pseudo-users in the master ( 192.168.56.15), This user will be used for replicating data between master and two slaves (192.168.56.16 and 192.168.56.17) .
mysql> create user 'repl_usr_16'@'192.168.56.16' identified by 'repl_usr_16' ; Query OK, 0 rows affected (0.02 sec) mysql> grant replication slave on *.* to 'repl_usr_16'@'192.168.56.16'; Query OK, 0 rows affected (0.00 sec)
mysql> create user 'repl_usr_17'@'192.168.56.17' identified by 'repl_usr_17' ; Query OK, 0 rows affected (0.02 sec) mysql> grant replication slave on *.* to 'repl_usr_17'@'192.168.56.17'; Query OK, 0 rows affected (0.00 sec)
root@VBOX125:/home/shiv# mysql -u root -p < backup.sql Enter password:
Run “change master to” command in 192.168.56.16
mysql> change master to -> master_host='192.168.56.15', -> master_user='repl_usr', -> master_password='repl_usr', -> master_auto_position=1; Query OK, 0 rows affected, 2 warnings (0.01 sec)
start slave in 192.168.56.16
mysql> start slave; Query OK, 0 rows affected (0.01 sec)
Run “change master to” command in 192.168.56.17
mysql> change master to -> master_host='192.168.56.15', -> master_user='repl_usr', -> master_password='repl_usr', -> master_auto_position=1; Query OK, 0 rows affected, 2 warnings (0.01 sec) start slave in 192.168.56.17
mysql> start slave; Query OK, 0 rows affected (0.01 sec)
Confirm replication is successful from both slaves (192.168.56.16 and 192.168.56.17)
mysql> show slave status\G; *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.56.15 Master_User: repl_usr Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql-bin.000002 Read_Master_Log_Pos: 529 Relay_Log_File: VBOX-112-relay-bin.000002 Relay_Log_Pos: 742 Relay_Master_Log_File: mysql-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 529 Relay_Log_Space: 952 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 900 Master_UUID: c03c4200-0659-11e8-94ab-0800279747b5 Master_Info_File: /var/lib/mysql/master.info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: c03c4200-0659-11e8-94ab-0800279747b5:1-2 Executed_Gtid_Set: c03c4200-0659-11e8-94ab-0800279747b5:1-2 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec) ERROR: No query specified
mysql> show slave status \G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.56.15 Master_User: repl_usr_17 Master_Port: 3306 Connect_Retry: 60 Master_Log_File: mysql-bin.000002 Read_Master_Log_Pos: 1008 Relay_Log_File: VBOX125-relay-bin.000002 Relay_Log_Pos: 893 Relay_Master_Log_File: mysql-bin.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 1008 Relay_Log_Space: 1102 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 900 Master_UUID: c03c4200-0659-11e8-94ab-0800279747b5 Master_Info_File: /var/lib/mysql/master.info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: c03c4200-0659-11e8-94ab-0800279747b5:3-4 Executed_Gtid_Set: c03c4200-0659-11e8-94ab-0800279747b5:1-4 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec)
The post MySQL Replication simplified with GTID – Step-by-step GTID replication setup appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post How to lag a MySQL slave behind to avoid data corruption ? appeared first on The WebScale Database Infrastructure Operations Experts.
]]>MySQL replication works great and indeed an great way to scale reads as well for high availability but what are pain points if you have not planned for a delayed slave ? What is delayed slave ? Before I answer this question, Have you ever thought about how an wrong transaction can corrupt MySQL master and all connected slaves ? The answer for this problem is a delayed slave replication, It’s actually you are intentionally lagging MySQL slave to a master by few minutes / hours so that you have always a MySQL instance far from damage. There are two ways to do this, 1. MySQL delayed slave replication , 2. Percona Toolkit pt-slave-delay (from Percona – https://www.percona.com/software/database-tools/percona-toolkit)
In this exercise I have used MySQL 5.7 on CentOS 7.4 with Percona Toolkit . I have written a blog on step-by-step installation of MySQL Master-Slave replication on CentOS here – https://minervadb.com/index.php/2018/01/24/step-by-step-mysql-master-slave-replication-on-centos/
MySQL delayed slave replication
Step 1: Login with MySQL root user in Slave server
[root@localhost ~]# mysql -u root -p Enter password: mysql>
Step 2: Stop the slave
mysql> stop slave; Query OK, 0 rows affected (0.01 sec) mysql>
Step 3: Now set delay time for replication in MySQL slave server. Here in this example, I am setting for 60 minutes or 3600 seconds.
mysql> CHANGE MASTER TO MASTER_DELAY = 3600;
Step 4: Start the mysql slave
mysql> start slave; Query OK, 0 rows affected (0.01 sec)
Step 5: Now check the SLAVE status
mysql> show slave status\G Master_Server_Id: 500 Master_UUID: 5ea8a1ea-f45e-11e7-bf7b-080027169869 Master_Info_File: /var/lib/mysql/master.info SQL_Delay: 3600 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
How to reset back to delay seconds Zero or default delay setting ?
You just need to stop and start MySQL slave
mysql> stop slave; Query OK, 0 rows affected (0.00 sec)
mysql> start slave; Query OK, 0 rows affected (0.01 sec)
Using pt-slave-delay for implementing delayed slave
pt-slave-delay is bundled with Percona Toolkit (https://www.percona.com/software/database-tools/percona-toolkit) . The steps here are quite direct and simple (here also we have delayed slave by 60 minutes), I have explained same below :
[root@localhost ~]# pt-slave-delay --delay 60m --user root --password PasswordComplex001 2018-01-29T16:07:28 slave running 0 seconds behind 2018-01-29T16:07:28 STOP SLAVE until 2018-01-29T17:07:28 at master position mysql-bin.000008/154
When you are building MySQL replication, please plan for delayed slaves to rollback if needed
The post How to lag a MySQL slave behind to avoid data corruption ? appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post Step-by-step MySQL Master – Slave Replication on CentOS appeared first on The WebScale Database Infrastructure Operations Experts.
]]>MySQL master-slave replication is quite simple and direct way to scale-out reads optimally. Retaining multiple copies of master across slaves also guarantee reliability and maximum availability, Replication is never an substitute or alternative for database backup and disaster recovery. In this post I am explaining simple step-by-step MySQL master-salve replication.
This blog post will be using following IP addresses:
MySQL Master – 192.168.56.11
MySQL Slave – 192.168.56.12
To configure a master to use binary log file based replication, Configure binary logging first and include a unique server ID. Once done these steps restart MySQL server. Binary logging is enabled by default (log_bin variable is set to ON). It’s always a best practice to create binary log files with a non-default base name to avoid confusion as your database infrastructure grows. To configure binary logging, first shutdown the MySQL and edit my.cnf or my.ini
[mysqld] server-id=1 bind-address=192.168.56.11 log-bin=mysql-bin binlog-do-db=sakila
To guarantee the ultimate transaction durability and consistency in the replication setup please confirm innodb_flush_log_at_trx_commit=1 and sync_binlog=1 in the master my.cnf file. You must also confirm skip-networking option is not enabled on your replication master.
MySQL slave must have a unique server ID, You must restart MySQL server after setting (this must be done in [mysqld] section of the configuration file) this in my.cnf . A slave is not required to have binary logging enabled to setup an simple MySQL master-slave replication but if you are building an relay-slave MySQL replication where server A serves as the master for slave B, and B serves as the master for slave C then slave B binary logging must be enabled. In addition to binary logging, this kind of replication topology requires the –log-slave-updates option to be enabled. You can disable binary logging on slave by configuring –skip-log-bin and –skip-log-slave-updates options in my.cnf
[mysqld] server-id=2
The slave connect to master using MySQL username and password, There must a user account on the master that slave can use to connect. This user account created in the master must be granted with REPLICATION SLAVE privilege. You may create different account for each slave or connect to master using the same account for each slave. Though it’s not compulsory to create user account specifically for replication, You must be aware that the replication user name and password is stored in plain text in the master info data dictionary table mysql.slave_master_info so from an MySQL security perspective it is recommended to have a dedicated user account for successful replication.
Please create the user below in MySQL master – 192.168.56.11
mysql> CREATE USER 'repl'@'192.168.56.12' IDENTIFIED BY 'MySQLDBA19/47'; mysql> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'192.168.56.12';
MySQL slave must be configured to start replication from the correct point, To do this task successfully you must note down master’s current coordinates with its binary log info. To collect MYSQL master binary log coordinates please FLUSH TABLES WITH READ LOCK, which blocks COMMIT operations for InnoDB tables (data consistency in replication is very important )Please run the below SQL in MySQL master – 192.168.56.11
mysql> FLUSH TABLES WITH READ LOCK;
Now from different MySQL session on the master run SHOW MASTER STATUS statement to collect current binary log file name and it’s position
mysql > SHOW MASTER STATUS; +------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +------------------+----------+--------------+------------------+ | mysql-bin.000147 | 450 | sakila | manual,mysql | +------------------+----------+--------------+------------------+
This is the position (Position 450) from which the slave database will start replication, Take a note the number as you need them while setting up slave for MySQL replication. Now you are ready to proceed with exporting your database using mysqldump from a new window ( please confirm you are not typing this command from MySQL, It’s done from bash shell )
mysqldump -u root -p --opt sakila > sakila-backup.sql
Now unlock tables in the master
UNLOCK TABLES;
mysql -u root -p sakila < sakila-backup.sql
Run the script below from MySQL terminal in the slave
CHANGE MASTER TO MASTER_HOST='192.168.56.11', MASTER_USER='repl', MASTER_PASSWORD='MySQLDBA19/47', MASTER_LOG_FILE='mysql-bin.000147', MASTER_LOG_POS= 450;
START SLAVE;
To confirm the successful slave replication please run below command on slave terminal
SHOW SLAVE STATUS\G;
Congrats, All done for an successful MySQL master-slave replication on CentOS !
The post Step-by-step MySQL Master – Slave Replication on CentOS appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post MySQL Master – Slave Replication on Ubuntu (xenial – 16.04 ) appeared first on The WebScale Database Infrastructure Operations Experts.
]]>MySQL master-slave replication is quite simple and direct way to scale-out reads optimally. Retaining multiple copies of master across slaves also guarantee reliability and maximum availability, Replication is never an substitute or alternative for database backup and disaster recovery. In this post I am explaining simple step-by-step MySQL master-salve replication.
This blog post will be using following IP addresses:
MySQL Master – 192.168.56.13
MySQL Slave – 192.168.56.23
To configure a master to use binary log file based replication, Configure binary logging first and include a unique server ID. Once done these steps restart MySQL server. Binary logging is enabled by default (log_bin variable is set to ON). It’s always a best practice to create binary log files with a non-default base name to avoid confusion as your database infrastructure grows. To configure binary logging, first shutdown the MySQL and edit my.cnf or my.ini
root@DA:/home/shiv/sakila# vi /etc/mysql/my.cnf
!includedir /etc/mysql/conf.d/ !includedir /etc/mysql/mysql.conf.d/ [mysqld] server-id = 900 log-bin = mysql-bin bind-address = 192.168.56.13 binlog-do-db = sakila
To guarantee the ultimate transaction durability and consistency in the replication setup please confirm innodb_flush_log_at_trx_commit=1 and sync_binlog=1 in the master my.cnf file. You must also confirm skip-networking option is not enabled on your replication master.
MySQL slave must have a unique server ID, You must restart MySQL server after setting (this must be done in [mysqld] section of the configuration file) this in my.cnf . A slave is not required to have binary logging enabled to setup an simple MySQL master-slave replication but if you are building an relay-slave MySQL replication where server A serves as the master for slave B, and B serves as the master for slave C then slave B binary logging must be enabled. In addition to binary logging, this kind of replication topology requires the –log-slave-updates option to be enabled. You can disable binary logging on slave by configuring –skip-log-bin and –skip-log-slave-updates options in my.cnf
root@DBA:/home/shiv# vi /etc/mysql/my.cnf
!includedir /etc/mysql/conf.d/ !includedir /etc/mysql/mysql.conf.d/ [mysqld] server-id=901
The slave connect to master using MySQL username and password, There must a user account on the master that slave can use to connect. This user account created in the master must be granted with REPLICATION SLAVE privilege. You may create different account for each slave or connect to master using the same account for each slave. Though it’s not compulsory to create user account specifically for replication, You must be aware that the replication user name and password is stored in plain text in the master info data dictionary table mysql.slave_master_info so from an MySQL security perspective it is recommended to have a dedicated user account for successful replication.
Please create the user below in MySQL master – 192.168.56.13
mysql> CREATE USER 'repl'@'192.168.56.23' IDENTIFIED BY 'repl'; mysql> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'192.168.56.23';
MySQL slave must be configured to start replication from the correct point, To do this task successfully you must note down master’s current coordinates with its binary log info. To collect MYSQL master binary log coordinates please FLUSH TABLES WITH READ LOCK, which blocks COMMIT operations for InnoDB tables (data consistency in replication is very important )Please run the below SQL in MySQL master – 192.168.56.13
mysql> FLUSH TABLES WITH READ LOCK;
Now from different MySQL session on the master run SHOW MYSQL STATUS statement to collect current binary log file name and it’s position
mysql> SHOW MASTER STATUS; +------------------+----------+--------------+------------------+-------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set | +------------------+----------+--------------+------------------+-------------------+ | mysql-bin.000001 | 771 | sakila | | | +------------------+----------+--------------+------------------+-------------------+ 1 row in set (0.00 sec)
This is the position (Position 771) from which the slave database will start replication, Take a note the number as you need them while setting up slave for MySQL replication. Now you are ready to proceed with exporting your database using mysqldump from a new window ( please confirm you are not typing this command from MySQL, It’s done from bash shell )
mysqldump -u root -p --opt sakila > sakila-backup.sql
Now unlock tables in the master
UNLOCK TABLES;
mysql -u root -p sakila < sakila-backup.sql
Run the script below from MySQL terminal in the slave
CHANGE MASTER TO MASTER_HOST='192.168.56.13', MASTER_USER='repl', MASTER_PASSWORD='repl', MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS= 771;
START SLAVE;
To confirm the successful slave replication please run below command on slave terminal
SHOW SLAVE STATUS\G;
Congrats, All done for an successful MySQL master-slave replication on Ubuntu !
The post MySQL Master – Slave Replication on Ubuntu (xenial – 16.04 ) appeared first on The WebScale Database Infrastructure Operations Experts.
]]>