The post MySQL 8.0 Delayed Replication – New Features and Benefits appeared first on The WebScale Database Infrastructure Operations Experts.
]]>Delayed Replication – You can deliberately execute transactions later than the master by a specific duration of time , Why you do that and for what ? Consider this, Accidentally someone did a wrong UPDATE / DELETE in the master and the transaction is committed, Now how can DBA rollback the database system to the last known good condition ? This is when we benefit from MySQL delayed slave replication investment. The default replication delay in MySQL is “0” seconds, To delay the slave by N seconds use the CHANGE MASTER TO MASTER_DELAY = N, The transactions received from the master is not executed until N seconds later than it’s commit on the immediate master. We have blogged here how to setup delayed slave replication in MySQL. In this blog post we have explained how MySQL 8.0 advanced Delayed Slave Replication features.
In MySQL 8.0 the delayed replication is controlled by two system variables on timestamps – orginal_commit_timestamp and immediate_commit_timestamp , They depend on GTID of each transaction (instead of each event like in MySQL 5.7) written to the binary log.These two system variables are applicable only when your entire replication infrastructure is on MySQL 8.0.1 or above , If either Master or slave is not using these timestamps, then delayed replication from MySQL 5.7 is used.
The orginal_commit_timestamp will be always same on all replication when the transaction is applied. In a typical master-slave replication, the original_commit_timestamp of a transaction in the (original) master’s binary log is always the same as its immediate_commit_timestamp. In the slave’s relay log, the original_commit_timestamp and immediate_commit_timestamp of the transaction are the same as in the master’s binary log; whereas in its own binary log, the transaction’s immediate_commit_timestamp corresponds to when the slave committed the transaction.
We strongly recommend following Performance Schema tables to monitor the replication delay (lag):
The following two matrices from the output of SHOW SLAVE STATUS is also helpful to monitor Delayed Replication:
SQL_Delay – This is measured in seconds of replication delay which configured using CHANGE MASTER TO MASTER_DELAY=N
SQL_Remaining_Delay – This shows total number seconds left of the delay configured intentionally , i.e. Slave_SQL_Running_State is Waiting for MASTER_DELAY seconds
We can never avoid the human error in database infrastructure operations. But rollback to the last known good condition from delayed Master / Slave is the best thing recommended during the entire database infrastructure corruption scenarios. We at MinervaDB strongly recommend delayed Master / Slaves for most of the customers to rollback quickly when there is an emergency, Thanks for your comments !
The post MySQL 8.0 Delayed Replication – New Features and Benefits appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post Using MySQL Shell 8.0.11 “upgrade checker” to upgrade from MySQL 5.7 to MySQL 8.0 successfully appeared first on The WebScale Database Infrastructure Operations Experts.
]]>Using MySQL Shell 8.0.11 “upgrade checker”
Typical “upgrade checker” run will look similar to this:
MySQL JS > util.checkForServerUpgrade("root@localhost:3306") Please provide the password for 'root@localhost:3306': ********** The MySQL server at localhost:3306 will now be checked for compatibility issues for upgrade to MySQL 8.0... MySQL version: 5.7.22-log - MySQL Community Server (GPL) 1) Usage of db objects with names conflicting with reserved keywords in 8.0 No issues found 2) Usage of utf8mb3 charset Warning: The following objects use the utf8mb3 character set. It is recommended to convert them to use utf8mb4 instead, for improved Unicode support. sakila.actor.first_name - column's default character set: utf8 sakila.actor.last_name - column's default character set: utf8 sakila.actor_info.first_name - column's default character set: utf8 sakila.actor_info.last_name - column's default character set: utf8 sakila.actor_info.film_info - column's default character set: utf8 sakila.address.address - column's default character set: utf8 sakila.address.address2 - column's default character set: utf8 sakila.address.district - column's default character set: utf8 sakila.address.postal_code - column's default character set: utf8 sakila.address.phone - column's default character set: utf8 sakila.category.name - column's default character set: utf8 sakila.city.city - column's default character set: utf8 sakila.country.country - column's default character set: utf8 sakila.customer.first_name - column's default character set: utf8 sakila.customer.last_name - column's default character set: utf8 sakila.customer.email - column's default character set: utf8 sakila.customer_list.name - column's default character set: utf8 sakila.customer_list.address - column's default character set: utf8 sakila.customer_list.zip code - column's default character set: utf8 sakila.customer_list.phone - column's default character set: utf8 sakila.customer_list.city - column's default character set: utf8 sakila.customer_list.country - column's default character set: utf8 sakila.customer_list.notes - column's default character set: utf8 sakila.film.title - column's default character set: utf8 sakila.film.description - column's default character set: utf8 sakila.film.rating - column's default character set: utf8 sakila.film.special_features - column's default character set: utf8 sakila.film_list.title - column's default character set: utf8 sakila.film_list.description - column's default character set: utf8 sakila.film_list.category - column's default character set: utf8 sakila.film_list.rating - column's default character set: utf8 sakila.film_list.actors - column's default character set: utf8 sakila.film_text.title - column's default character set: utf8 sakila.film_text.description - column's default character set: utf8 sakila.language.name - column's default character set: utf8 sakila.nicer_but_slower_film_list.title - column's default character set: utf8 sakila.nicer_but_slower_film_list.description - column's default character set: utf8 sakila.nicer_but_slower_film_list.category - column's default character set: utf8 sakila.nicer_but_slower_film_list.rating - column's default character set: utf8 sakila.nicer_but_slower_film_list.actors - column's default character set: utf8 sakila.sales_by_film_category.category - column's default character set: utf8 sakila.sales_by_store.store - column's default character set: utf8 sakila.sales_by_store.manager - column's default character set: utf8 sakila.staff.first_name - column's default character set: utf8 sakila.staff.last_name - column's default character set: utf8 sakila.staff.email - column's default character set: utf8 sakila.staff.username - column's default character set: utf8 sakila.staff.password - column's default character set: utf8 sakila.staff_list.name - column's default character set: utf8 sakila.staff_list.address - column's default character set: utf8 sakila.staff_list.zip code - column's default character set: utf8 sakila.staff_list.phone - column's default character set: utf8 sakila.staff_list.city - column's default character set: utf8 sakila.staff_list.country - column's default character set: utf8 3) Usage of use ZEROFILL/display length type attributes Notice: The following table columns specify a ZEROFILL/display length attributes. Please be aware that they will be ignored in MySQL 8.0 sakila.customer.active - tinyint(1) sakila.staff.active - tinyint(1) 4) Issues reported by 'check table x for upgrade' command No issues found 5) Table names in the mysql schema conflicting with new tables in 8.0 No issues found 6) Usage of old temporal type No issues found 7) Foreign key constraint names longer than 64 characters No issues found 8) Usage of obsolete MAXDB sql_mode flag No issues found 9) Usage of obsolete sql_mode flags No issues found 10) Usage of partitioned tables in shared tablespaces No issues found 11) Usage of removed functions No issues found No fatal errors were found that would prevent a MySQL 8 upgrade, but some potential issues were detected. Please ensure that the reported issues are not significant before upgrading. 1
At the end, “upgrade checker” prints a summary and returns an integer value describing he severity of the issues found:
Upgrade from MySQL 5.7 to MySQL 8.0
Step 1 – Uninstall MySQL 5.7
[root@localhost ~]# systemctl stop mysqld [root@localhost ~]# yum remove mysql-community-client.x86_64 mysql-community-common.x86_64 mysql-community-devel.x86_64 mysql-community-libs.x86_64 mysql-community-libs-compat.x86_64 mysql-community-server.x86_64 Loaded plugins: fastestmirror Resolving Dependencies --> Running transaction check ---> Package mysql-community-client.x86_64 0:5.7.22-1.el7 will be erased ---> Package mysql-community-common.x86_64 0:5.7.22-1.el7 will be erased ---> Package mysql-community-devel.x86_64 0:5.7.22-1.el7 will be erased ---> Package mysql-community-libs.x86_64 0:5.7.22-1.el7 will be erased ---> Package mysql-community-libs-compat.x86_64 0:5.7.22-1.el7 will be erased ---> Package mysql-community-server.x86_64 0:5.7.22-1.el7 will be erased --> Finished Dependency Resolution Dependencies Resolved ======================================================================================= Package Arch Version Repository Size ======================================================================================= Removing: mysql-community-client x86_64 5.7.22-1.el7 @mysql57-community 106 M mysql-community-common x86_64 5.7.22-1.el7 @mysql57-community 2.5 M mysql-community-devel x86_64 5.7.22-1.el7 @mysql57-community 21 M mysql-community-libs x86_64 5.7.22-1.el7 @mysql57-community 9.4 M mysql-community-libs-compat x86_64 5.7.22-1.el7 @mysql57-community 9.2 M mysql-community-server x86_64 5.7.22-1.el7 @mysql57-community 743 M Transaction Summary ======================================================================================= Remove 6 Packages Installed size: 892 M Is this ok [y/N]: y Downloading packages: Running transaction check Running transaction test Transaction test succeeded Running transaction Erasing : mysql-community-devel-5.7.22-1.el7.x86_64 1/6 Erasing : mysql-community-server-5.7.22-1.el7.x86_64 2/6 warning: /etc/my.cnf saved as /etc/my.cnf.rpmsave Erasing : mysql-community-client-5.7.22-1.el7.x86_64 3/6 Erasing : mysql-community-libs-compat-5.7.22-1.el7.x86_64 4/6 Erasing : mysql-community-libs-5.7.22-1.el7.x86_64 5/6 Erasing : mysql-community-common-5.7.22-1.el7.x86_64 6/6 Verifying : mysql-community-libs-compat-5.7.22-1.el7.x86_64 1/6 Verifying : mysql-community-common-5.7.22-1.el7.x86_64 2/6 Verifying : mysql-community-devel-5.7.22-1.el7.x86_64 3/6 Verifying : mysql-community-server-5.7.22-1.el7.x86_64 4/6 Verifying : mysql-community-client-5.7.22-1.el7.x86_64 5/6 Verifying : mysql-community-libs-5.7.22-1.el7.x86_64 6/6 Removed: mysql-community-client.x86_64 0:5.7.22-1.el7 mysql-community-common.x86_64 0:5.7.22-1.el7 mysql-community-devel.x86_64 0:5.7.22-1.el7 mysql-community-libs.x86_64 0:5.7.22-1.el7 mysql-community-libs-compat.x86_64 0:5.7.22-1.el7 mysql-community-server.x86_64 0:5.7.22-1.el7 Complete! [root@localhost ~]#
Step 2 – Install MySQL 8.0
[root@localhost MySQL8-Community-Edition]# rpm -ivh mysql-community-server-8.0.11-1.el7.x86_64.rpm mysql-community-client-8.0.11-1.el7.x86_64.rpm mysql-community-common-8.0.11-1.el7.x86_64.rpm mysql-community-devel-8.0.11-1.el7.x86_64.rpm mysql-community-libs-8.0.11-1.el7.x86_64.rpm mysql-community-libs-compat-8.0.11-1.el7.x86_64.rpm warning: mysql-community-server-8.0.11-1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY Preparing... ################################# [100%] Updating / installing... 1:mysql-community-common-8.0.11-1.e################################# [ 17%] 2:mysql-community-libs-8.0.11-1.el7################################# [ 33%] 3:mysql-community-client-8.0.11-1.e################################# [ 50%] 4:mysql-community-server-8.0.11-1.e################################# [ 67%] 5:mysql-community-devel-8.0.11-1.el################################# [ 83%] 6:mysql-community-libs-compat-8.0.1################################# [100%] [root@localhost MySQL8-Community-Edition]#
Step 3- Start MySQL 8.0
[root@localhost MySQL8-Community-Edition]# systemctl start mysqld [root@localhost MySQL8-Community-Edition]# mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 10 Server version: 8.0.11 MySQL Community Server - GPL Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql>
Step 4 – Run “mysql_upgrade” , mysql_upgrade checks for all tables in all databases for incompatibilities with the current version of MySQL Server, it also upgrades the system tables so that you can take advantage of new privileges or capabilities that might have been added.
[root@localhost MySQL8-Community-Edition]# mysql_upgrade -u root -p Enter password: Checking if update is needed. Checking server version. Running queries to upgrade MySQL server. Upgrading system table data. Checking system database. mysql.columns_priv OK mysql.component OK mysql.db OK mysql.default_roles OK mysql.engine_cost OK mysql.func OK mysql.general_log OK mysql.global_grants OK mysql.gtid_executed OK mysql.help_category OK mysql.help_keyword OK mysql.help_relation OK mysql.help_topic OK mysql.innodb_index_stats OK mysql.innodb_table_stats OK mysql.ndb_binlog_index OK mysql.password_history OK mysql.plugin OK mysql.procs_priv OK mysql.proxies_priv OK mysql.role_edges OK mysql.server_cost OK mysql.servers OK mysql.slave_master_info OK mysql.slave_relay_log_info OK mysql.slave_worker_info OK mysql.slow_log OK mysql.tables_priv OK mysql.time_zone OK mysql.time_zone_leap_second OK mysql.time_zone_name OK mysql.time_zone_transition OK mysql.time_zone_transition_type OK mysql.user OK Found outdated sys schema version 1.5.1. Upgrading the sys schema. Checking databases. employees.departments OK employees.dept_emp OK employees.dept_manager OK employees.employees OK employees.salaries OK employees.tab1 OK employees.titles OK sakila.actor OK sakila.address OK sakila.category OK sakila.city OK sakila.country OK sakila.customer OK sakila.film OK sakila.film_actor OK sakila.film_category OK sakila.film_text OK sakila.inventory OK sakila.language OK sakila.payment OK sakila.rental OK sakila.staff OK sakila.store OK sys.sys_config OK Upgrade process completed successfully. Checking if update is needed.
Success , You have successfully completed upgrade from MySQL 5.7 to MySQL 8.0 .
The post Using MySQL Shell 8.0.11 “upgrade checker” to upgrade from MySQL 5.7 to MySQL 8.0 successfully appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post MySQL 8.0 Group Replication Limitations appeared first on The WebScale Database Infrastructure Operations Experts.
]]>MySQL Group Replication can operate in both single-primary mode with automatic primary election, where only one server accepts updates at a time and multi-primary mode, where all servers can accept updates, even if they are issued concurrently. The built-in group membership service retains view of the group consistent and available for all servers at any given point in time, Servers can leave and join the group and the view is updated accordingly. If servers leave the group unexpectedly, the failure detection mechanism detects this and notifies the group that the view has changed, All this happens automatically !
For any transaction to commit, the majority of group members have to agree on the order of a given transaction in the global sequence of transactions. Decision to commit or abort a transaction is taken by respective servers, but all servers make same decision. In the case of systems built on network partition, When the members of the group are unable to reach agreement on a transaction, the system halt till the issue is resolved, This guarantees split-brain protection mechanism.
All these fault tolerance mechanisms are powered by Group Communication System (GCS) protocols, a group membership service, and completely safe ordered message delivery system. Group Replication is powered by Paxos algorithm (https://en.wikipedia.org/wiki/Paxos_(computer_science)), which acts as the group communication engine.
Till now, we have just posted about the capabilities of Group Replication, Now we have mentioned below (in bullets) about all the limitations of Group Replication:
The post MySQL 8.0 Group Replication Limitations appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post The first impression of MySQL 8 system variable innodb_dedicated_server appeared first on The WebScale Database Infrastructure Operations Experts.
]]>What was our first reaction, when we first read about innodb_dedicated_server ?
Wow, That will be awesome … Indeed, When we manage several hundreds of MySQL instances, This feature will really improve efficiency and DBA Ops. governance.
Now, Let us explain what we have found:
How does innodb_dedicated_server system variable in MySQL 8.0 size the following variables:
The first impression of innodb_dedicated_server system variable in MySQL 8.0 is impressive, Definitely will deliver much better performance than default value. This new feature will configure the MySQL system variable mentioned above more intuitively to improve DBA productivity. Till MySQL 5.7 it was always presumed 512M RAM with the default settings.
Are we going to follow this in our daily DBA checklists ?
Not really, We are an very conservative team about implementing the new features immediately in the critical database infrastructure of our customers, Also we are afraid about the isolated issues due to auto sizing of MySQL / InnoDB memory structures, Let’s explain why we will not be using this feature immediately for our MySQL 8.0 customers:
P.S – innodb_dedicated_server system variable is a relatively new feature, We are confident MySQL engineering team will be improving this component in coming days so our perspective will also change, We will never forget then to blog about this feature and why we are seriously thinking about implementing it for our customer production infrastructure.. Technology keeps changing for good, We are adaptive for the change !
The post The first impression of MySQL 8 system variable innodb_dedicated_server appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post MySQL 8 default character set is utf8mb4 appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The UTF-8 is a variable-length encoding. In the case of UTF-8, it means that storing one code point requires one to four bytes. But, In MySQL’s encoding called “utf8” only stores a maximum of three bytes per code point. In the modern web / mobile applications, we have to support for storing not only language characters but also symbols and emojis, Let me show you below some very weird issues faced using MySQL “utf8” :
mysql> SET NAMES utf8; # just to emphasize that the connection charset is set to `utf8` Query OK, 0 rows affected (0.00 sec) mysql> UPDATE custfeeds.reactions SET reacted = 'super like ' WHERE id = 13015; Query OK, 1 row affected, 1 warning (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 1 mysql> SELECT reactions FROM custfeeds.reactions WHERE id = 13015; +-------------+ | reactions | +-------------+ | super liked | +-------------+ 1 row in set (0.00 sec) mysql> SHOW WARNINGS; +---------+------+------------------------------------------------------------------------------+ | Level | Code | Message | +---------+------+------------------------------------------------------------------------------+ | Warning | 1366 | Incorrect string value: '\xF0\x9D\x8C\x86' for column 'reactions' at row 731 | +---------+------+------------------------------------------------------------------------------+ 1 row in set (0.00 sec)
MySQL’s utf8 charset can only store UTF-8-encoded symbols that consist of one to three bytes; encoded symbols that take up four bytes aren’t supported. MySQL 5.5.3 released utf8mb4 encoding to solve this problem (https://dev.mysql.com/doc/relnotes/mysql/5.5/en/news-5-5-3.html) so utf8mb4 charset is not a MySQL 8 new feature (yes, It’s now default charset from MySQL8)
MySQL 8 has by default utf8mb4 character set
mysql> SHOW VARIABLES WHERE Variable_name LIKE 'character\_set\_%' OR Variable_name LIKE 'collation%'; +--------------------------+--------------------+ | Variable_name | Value | +--------------------------+--------------------+ | character_set_client | utf8mb4 | | character_set_connection | utf8mb4 | | character_set_database | utf8mb4 | | character_set_filesystem | binary | | character_set_results | utf8mb4 | | character_set_server | utf8mb4 | | character_set_system | utf8 | | collation_connection | utf8mb4_0900_ai_ci | | collation_database | utf8mb4_0900_ai_ci | | collation_server | utf8mb4_0900_ai_ci | +--------------------------+--------------------+ 10 rows in set (0.01 sec)
How things changed in MySQL 8 with utf8mb4 character set ?
mysql> UPDATE custfeeds.reactions SET reacted = 'super like ' WHERE id = 13015; Query OK, 1 row affected, 0 warning (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0
mysql> SELECT reactions FROM custfeeds.reactions WHERE id = 13015; +-------------+ | reactions | +-------------+ | super liked | +-------------+ 1 row in set (0.00 sec)
Conclusion
Traditionally MySQL is built for scaling web-scale database infrastructure operations, In the modern web applications / mobile apps. , emojis and a multitude of charsets / collation needs to coexist. To address this compelling need, in MySQL 8.0 default character set has been changed from latin-1 to utf8mb4.
The post MySQL 8 default character set is utf8mb4 appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post error: Failed dependencies: pkgconfig(openssl) is needed by mysql-commercial-devel-8.0.11-1.1.el7.x86_64 appeared first on The WebScale Database Infrastructure Operations Experts.
]]>[root@localhost MySQL8-Enterprise-RPM]# rpm -ivh mysql-commercial-server-8.0.11-1.1.el7.x86_64.rpm mysql-commercial-client-8.0.11-1.1.el7.x86_64.rpm mysql-commercial-libs-8.0.11-1.1.el7.x86_64.rpm mysql-commercial-common-8.0.11-1.1.el7.x86_64.rpm mysql-commercial-devel-8.0.11-1.1.el7.x86_64.rpm warning: mysql-commercial-server-8.0.11-1.1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY error: Failed dependencies: pkgconfig(openssl) is needed by mysql-commercial-devel-8.0.11-1.1.el7.x86_64
After some research, we found “openssl-devel” installation will address the dependency issues, Actually “openssl-devel” is a prerequisite for successful installation of “mysql-commercial-devel-8.0.11-1.1.el7.x86_64.rpm”
[root@localhost MySQL8-Enterprise-RPM]# yum install openssl-devel Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirror.vbctv.in * extras: mirror.vbctv.in * updates: mirror.vbctv.in Resolving Dependencies --> Running transaction check ---> Package openssl-devel.x86_64 1:1.0.2k-12.el7 will be installed --> Processing Dependency: zlib-devel(x86-64) for package: 1:openssl-devel-1.0.2k-12.el7.x86_64 --> Processing Dependency: krb5-devel(x86-64) for package: 1:openssl-devel-1.0.2k-12.el7.x86_64 --> Running transaction check Installed: openssl-devel.x86_64 1:1.0.2k-12.el7 Dependency Installed: keyutils-libs-devel.x86_64 0:1.5.8-3.el7 krb5-devel.x86_64 0:1.15.1-19.el7 libcom_err-devel.x86_64 0:1.42.9-12.el7_5 libkadm5.x86_64 0:1.15.1-19.el7 libselinux-devel.x86_64 0:2.5-12.el7 libselinux-python.x86_64 0:2.5-12.el7 libselinux-utils.x86_64 0:2.5-12.el7 libsepol.x86_64 0:2.5-8.1.el7 libss.x86_64 0:1.42.9-12.el7_5 Complete!
[root@localhost MySQL8-Enterprise-RPM]# rpm -ivh mysql-commercial-devel-8.0.11-1.1.el7.x86_64.rpm warning: mysql-commercial-devel-8.0.11-1.1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY Preparing... ################################# [100%] Updating / installing... 1:mysql-commercial-devel-8.0.11-1.1################################# [100%]
Confirm all packages installed for an successful MySQL operations:
[root@localhost MySQL8-Enterprise-RPM]# rpm -qa | grep mysql mysql-commercial-common-8.0.11-1.1.el7.x86_64 mysql-commercial-client-8.0.11-1.1.el7.x86_64 mysql-commercial-libs-compat-8.0.11-1.1.el7.x86_64 mysql-commercial-devel-8.0.11-1.1.el7.x86_64 mysql-commercial-libs-8.0.11-1.1.el7.x86_64 mysql-commercial-server-8.0.11-1.1.el7.x86_64 [root@localhost MySQL8-Enterprise-RPM]#
The post error: Failed dependencies: pkgconfig(openssl) is needed by mysql-commercial-devel-8.0.11-1.1.el7.x86_64 appeared first on The WebScale Database Infrastructure Operations Experts.
]]>