MariaDB S3 storage engine

MariaDB S3 is a read only storage engine based on Aria code which allows you to archive MariaDB tables in Amazon S3 (on the size of s3_block_size) or any third-party public or private cloud that implements S3 API (of which there are many), but still have them accessible for reading in MariaDB. Internally, The S3 storage inherits from Aria code and hooks that change reads, So that instead if reading from local disk it reads from S3. We recommend S3 storage engine for our MariaDB 10.5 customers (if you are planning the update to MariaDB 10.5, We strongly recommend you to read our blogs MariaDB 10.5. New Features- Upgrade from MariaDB 10.4 to MariaDB 10.5 ) with very high volume tables which would become fairly inactive, but are still important so that they can not be removed. In that case, an option is to move such a table to an archiving service, which is accessible through an S3 API. So MariaDB S3 is a technically and commercially cost  efficient storage engine built for MariaDB archiving.

Installing MariaDB S3 Storage Engine

The S3 storage engine is available from MariaDB 10.5.4.

[mysqld]
plugin-maturity = alpha

Note: S3 storage engine is currently in alpha maturity   , So you cannot load by default S3 storage engine on MariaDB Server stable release ( this happens due to default value of system variable plugin_maturity ). To enable S3 storage engine you have to set plugin-maturity = alpha and restart the server.

Install plugin library:

INSTALL SONAME 'ha_s3';

How can you move data to S3 storage engine ?

If you want to move data from an existing table to S3 storage engine, Please run:

ALTER TABLE table_name ENGINE=S3

To move data back to a regular InnoDB table,  Please run:

ALTER TABLE s3_table_name ENGINE=INNODB

MariaDB S3 storage engine performance

The following are some best practices to consider for optimal performance on MariaDB S3 storage engine:

• MariaDB S3 Tables supports all the options (including ALTER TABLE) supported by Aria engine:
  • ALTER TABLE
  • DROP TABLE
  • SELECT 
  • SHOW TABLES
  • MariaDB S3 TABLES can also participate actively on Partitioning
• s3_block_size  is the default block size of the table, For most of the load the default size 4M is sufficient and we don’t recommend increasing this system variable
• Be conservative about querying information_schema tables as S3 has to check if there is new tables in S3.
• DROP statements on non existing tables are slower as S3 has to check if the table is in S3.
• MariaDB S3 Tables can use COMPRESSION_ALGORITHM=zlib to reduce the amount of data transferred from S3 to the local cache.
• If you are expecting high volume MariaDB S3 tables, Then we strongly recommend you to increase s3_pagecache_buffer_size ( sequel to innodb_buffer_pool_size of InnoDB and default value is 128M ) for optimal caching and better index handling. 
• MariaDB S3 Tables I/O performance also greatly depend on your connection speed to your S3 provider.

Conclusion

MariaDB powers database infrastructure for some of the largest planet-scale internet properties, This means handling database volume, performance, scalability, reliability, capacity planning / sizing and archiving are some of the most complex problems to solve. Thanks to MariaDB Server Engineering Team for coming up with S3 storage engine which is seamlessly addressing database archiving across multiple (vendor neutral and independent ) S3 providers.

☛ Want to engage MinervaDB for MariaDB Consulting, Enterprise-Class 24*7 Support and Remote DBA Services ?

Business Function	Contact
CONTACT GLOBAL SALES (24*7)	(844) 588-7287 (USA) (415) 212-6625 (USA) (778) 770-5251 (Canada)
TOLL FREE PHONE (24*7)	(844) 588-7287
MINERVADB FAX	+1 (209) 314-2364
MinervaDB Email - General / Sales / Consulting	contact@minervadb.com
MinervaDB Email - Support	support@minervadb.com
MinervaDB Email -Remote DBA	remotedba@minervadb.com
Shiv Iyer Email - Founder and Principal	shiv@minervadb.com
CORPORATE ADDRESS: CALIFORNIA	MinervaDB Inc., 340 S LEMON AVE #9718 WALNUT 91789 CA, US
CORPORATE ADDRESS: DELAWARE	MinervaDB Inc., PO Box 2093 PHILADELPHIA PIKE #3339 CLAYMONT, DE 19703
CORPORATE ADDRESS: HOUSTON	MinervaDB Inc., 1321 Upland Dr. PMB 19322, Houston, TX 77043, US

The post MariaDB S3 Storage Engine – MariaDB 10.5.4 New Feature appeared first on The WebScale Database Infrastructure Operations Experts.

Most often Database Systems outages happen due to user error and it is also the biggest reason for data loss / damage or corruption. In these type of failures, It is application modifying or destroying the data on its own or through a user choice. Hardware failure also contributes to database infrastructure crashes and corruption. To address these sort of data reliability issues, you must recover and restore to the point in time before the corruption occurred. Disaster Recover tools returns the data to its original state at the cost of any other changes that were being made to the data since the point the corruption took place. MinervaDB founder and Principal, hosted a webinar (Thursday, June 18, 2020 – 06:00 PM to 07:00 PM PDT) on MySQL backup strategies and tools addressing the topics below:

• Proactive MySQL DR – From strategy to execution
• Building capacity for reliable MySQL DR
• MySQL DR strategies
• MySQL Backup tools
• Managing MySQL DR Ops. for very large databases
• Testing MySQL Backups
• Biggest MySQL DR mistakes
• MySQL DR Best Practices and Checklist

You can download the PDF (slides) of webinar here

☛ MinervaDB contacts for MySQL Database Backup and Database Reliability Engineering Services

Business Function	Contact
CONTACT GLOBAL SALES (24*7)	(844) 588-7287 (USA) (415) 212-6625 (USA) (778) 770-5251 (Canada)
TOLL FREE PHONE (24*7)	(844) 588-7287
MINERVADB FAX	+1 (209) 314-2364
MinervaDB Email - General / Sales / Consulting	contact@minervadb.com
MinervaDB Email - Support	support@minervadb.com
MinervaDB Email -Remote DBA	remotedba@minervadb.com
Shiv Iyer Email - Founder and Principal	shiv@minervadb.com
CORPORATE ADDRESS: CALIFORNIA	MinervaDB Inc., 340 S LEMON AVE #9718 WALNUT 91789 CA, US
CORPORATE ADDRESS: DELAWARE	MinervaDB Inc., PO Box 2093 PHILADELPHIA PIKE #3339 CLAYMONT, DE 19703
CORPORATE ADDRESS: HOUSTON	MinervaDB Inc., 1321 Upland Dr. PMB 19322, Houston, TX 77043, US

The post MySQL Backup Strategies and Tools – MinervaDB Webinar appeared first on The WebScale Database Infrastructure Operations Experts.

(Thursday, June 18, 2020 – 06:00 PM to 07:00 PM PDT)

There can be several reasons for a MySQL database outage: hardware failure, power outage, human error, natural disaster etc. We may not be able prevent all the disaster from happening but investing on a robust disaster recovery plan is very important for building fault-tolerant database infrastructure operations on MySQL.  Every MySQL DBA is accountable for developing a disaster recovery plan addressing data sensitivity, data loss tolerance and data security. Join Shiv Iyer, Founder and Principal of MinervaDB to lean about the best practices for building highly reliable MySQL DR strategy and operations on Thursday, June 18, 2020 – 06:00 PM to 07:00 PM PDT. Building DR for a high traffic MySQL database infrastructure means deep understanding of multiple backup strategies and choosing optimal ones which are best suited for performance and reliability. Most of the data intensive MySQL infrastructure will have a combination of multiple backup methods and tools, In this webinar Shiv talks about his experiences in the past and present on building MySQL DR Ops, tools and zero tolerance data loss methods.

Join this webinar to learn more about:

• Proactive MySQL DR – From strategy to execution
• Building capacity for reliable MySQL DR
• MySQL DR strategies
• MySQL Backup tools
• Managing MySQL DR Ops. for very large databases
• Testing MySQL Backups
• Biggest MySQL DR mistakes
• MySQL DR Best Practices and Checklist

The post MySQL Backup and Disaster Recovery Webinar appeared first on The WebScale Database Infrastructure Operations Experts.

MySQL powers all the major internet properties, Which include Google, Facebook, Twitter, LinkedIn, Uber etc. So how do we plan for MySQL disaster recovery and what are the most common MySQL DR tools used today for building highly reliable database infrastructure operations ? There can be several reasons for a MySQL database outage: hardware failure, power outage, human error, natural disaster etc. We may not be able prevent all the disaster from happening but investing on a robust disaster recovery plan is very important for building fault-tolerant database infrastructure operations on MySQL.  Every MySQL DBA is accountable for developing a disaster recovery plan addressing data sensitivity, data loss tolerance and data security. Functionally you have several database backup strategies available with MySQL:

• Full backup – Full backup backs up the whole database, This also include transaction log so that the full database can be recovered after a full database backup is restored. Full database backups represent the database at the time the backup finished. Full backups are storage resource intensive and takes more time to finish. If you have for a large database, we strongly recommend to supplement a full database backup with a series of differential database backups.
• Differential backup – A differential backup is based on the most recent, previous full data backup. A differential backup captures only the data that has changed since that full backup. The full backup upon which a differential backup is based is known as the base of the differential. Full backups, except for copy-only backups, can serve as the base for a series of differential backups, including database backups, partial backups, and file backups. The base backup for a file differential backup can be contained within a full backup, a file backup, or a partial backup. The differential backups are most recommended when the subset of a database is modified more frequently than the rest of the database.
• Incremental backup – A incremental backup contains all changes to the data since the last backup. Both differential and incremental backup does only backing up changed files. But they differ significantly in how they do it, and how useful the result is.while an incremental backup only includes the data that has changed since the previous backup, a differential backup contains all of the data that has changed since the last full backup. The advantage that differential backup offers over incremental backups is a shorter restore time. Because, the backup has to be reconstituted from the last full backup and all the incremental backups since.

MySQL Backup tools

The following are the list of MySQL backup tools (logical and physical) discussed in this blog:

mysqldump

mysqldump is a MySQL client utility which can be used to perform logical backups  , The mysqldump generate output in SQL ( default and most commonly used to reproduce MySQL schema objects and data), CSV, other delimited text or XML format. We have copied below the restrictions of mysqldump:

• mysqldump does not dump performance_schema or sys schema be default. To enforce dumping or logical backup of any of these schema objects, You have to explicitly mention them –databases option or if you want to just dump performance_schema use –skip-lock-tables option.
• mysqldump does not dump the INFORMATION_SCHEMA schema.
• mysqldump does not dump the InnoDB CREATE TABLESPACE statements.
• mysqldump does not dump the NDB Cluster ndbinfo information database.
• mysqldump includes statements required to recreate the general_log and slow_query_log tables for dumps of the mysql database. But, Log table contents are not dumped

Script to dump all the databases: 

shell> mysqldump --all-databases > all_databases.sql

Script to dump the entire database:

shell> mysqldump db_name > db_name_dump.sql

Script to dump several databases with one command:

shell> mysqldump --databases db_name1 [db_name2 ...] > databases_dump.sql

mysqlpump 

The mysqlpump is another client utility for logical backup of MySQL database like mysqldump which is capable of parallel processing of databases and other schema objects with databases to perform high performance dumping process, We listed below mysqlpump most compelling features:

• MySQL dump with parallel processing capabilities for databases and other objects within databases.
• MySQL user accounts will be dumped as account-management statements (CREATE USER, GRANT) rather than as inserts into the mysql system database.
• By using mysqlpump you can create a compressed output.
• Much faster compared to mysqldump. Because, The InnoDB secondary indexes are created after rows are inserted to the table.

P.S. – We have blogged about “How to use mysqlpump for faster MySQL logical backup ? ” here

MySQL Enterprise Backup 

• Transparent page compression for InnoDB.
• Backup history available for all members of Group Replication by making sure backup_history table is updated on primary node after each mysqlbackup operation.
• Storage engine of the mysql.backup_history table on a backed-up server has switched from CSV to InnoDB.
• mysqlbackup now supports encrypted InnoDB undo logs .
• mysqlbackup now supports high performance incremental backup by setting page tracking functionality on MySQL (set –incremental=page-track).
• Much better MySQL Enterprise Backup 8.0 troubleshooting with now mysqlbackup prints a stack trace after being terminated by a signal.
• Selective restores of tables or schema from full backup for Table-Level Recovery (TLR)

Percona XtraBackup 

Percona XtraBackup is an open source MySQL hot backup solution from Percona addressing incremental, fast, compressed and secured backup for InnoDB optimally. Most of our customers users Percona XtraBackup for DR of their MySQL infrastructure, The following features makes Percona XtraBackup obvious choice for MySQL Backup and DR:

• Hot backup solution for InnoDB without blocking / locking transaction processing.
• Point-in-time recovery for InnoDB.
• MySQL incremental backup support.
• Percona XtraBackup supports incremental compressed backups.
• High performance streaming backup support for InnoDB.
• Parallel backup and copy-back support for faster backup and restoration.
• Secondary indexes defragmentation support for InnoDB.
• Percona XtraBackup support rsync to minimize locking.
• Track Percona XtraBackup history with Backup history table.
• Percona XtraBackup supports offline backup.

Conclusion

We always recommend a combination of multiple backup strategies / tools for maximum data reliability and optimal restoration. We cannot have a common backup strategy for all the customers, It depends on factors like infrastructure, MySQL distribution, database size, SLA etc. Backups are most import components in database infrastructure operations and we follow zero tolerance DR for building highly available and fault-tolerant MySQL infrastructure.

Do you want to engage MinervaDB Remote DBA for MySQL Disaster Recovery (DR) and Database Reliability Engineering (Data SRE) ?

Business Function	Contact
CONTACT GLOBAL SALES (24*7)	(844) 588-7287 (USA) (415) 212-6625 (USA) (778) 770-5251 (Canada)
TOLL FREE PHONE (24*7)	(844) 588-7287
MINERVADB FAX	+1 (209) 314-2364
MinervaDB Email - General / Sales / Consulting	contact@minervadb.com
MinervaDB Email - Support	support@minervadb.com
MinervaDB Email -Remote DBA	remotedba@minervadb.com
Shiv Iyer Email - Founder and Principal	shiv@minervadb.com
CORPORATE ADDRESS: CALIFORNIA	MinervaDB Inc., 340 S LEMON AVE #9718 WALNUT 91789 CA, US
CORPORATE ADDRESS: DELAWARE	MinervaDB Inc., PO Box 2093 PHILADELPHIA PIKE #3339 CLAYMONT, DE 19703
CORPORATE ADDRESS: HOUSTON	MinervaDB Inc., 1321 Upland Dr. PMB 19322, Houston, TX 77043, US

The post MySQL Backup Strategies – Building MySQL DR Solutions appeared first on The WebScale Database Infrastructure Operations Experts.

Recently ( on Friday, 5 June 2020 – 06:00 PM PDT to 06:45 PM PDT  ) our Founder and Principal ( Shiv Iyer ) did a webinar on building Database Systems Infrastructure Operations for Performance and Reliability. In this webinar, he discussed about capacity planning / sizing, observability & resilience, performance audit / health-check / diagnostics / forensics, performance optimization & tuning and building highly available / fault-tolerant / self-healing systems architecture. You can download the PDF of the webinar here . Thanks for joining the webinar and making it a success, Looking forward to seeing you all in the next webinar.

The post Data SRE – Building Database Systems Infrastructure for Performance and Reliability appeared first on The WebScale Database Infrastructure Operations Experts.

mysql> SET NAMES utf8; # just to emphasize that the connection charset is set to `utf8`
Query OK, 0 rows affected (0.00 sec)

mysql> UPDATE custfeeds.reactions SET reacted = 'super like ' WHERE id = 13015;
Query OK, 1 row affected, 1 warning (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 1

mysql> SELECT reactions FROM custfeeds.reactions WHERE id = 13015;
+-------------+
| reactions   |
+-------------+
| super liked |
+-------------+
1 row in set (0.00 sec)


mysql> SHOW WARNINGS;
+---------+------+------------------------------------------------------------------------------+
| Level   | Code | Message                                                                      |
+---------+------+------------------------------------------------------------------------------+
| Warning | 1366 | Incorrect string value: '\xF0\x9D\x8C\x86' for column 'reactions' at row 731 |
+---------+------+------------------------------------------------------------------------------+
1 row in set (0.00 sec)

mysql> SHOW VARIABLES WHERE Variable_name LIKE 'character\_set\_%' OR Variable_name LIKE 'collation%';
+--------------------------+--------------------+
| Variable_name            | Value              |
+--------------------------+--------------------+
| character_set_client     | utf8mb4            |
| character_set_connection | utf8mb4            |
| character_set_database   | utf8mb4            |
| character_set_filesystem | binary             |
| character_set_results    | utf8mb4            |
| character_set_server     | utf8mb4            |
| character_set_system     | utf8               |
| collation_connection     | utf8mb4_0900_ai_ci |
| collation_database       | utf8mb4_0900_ai_ci |
| collation_server         | utf8mb4_0900_ai_ci |
+--------------------------+--------------------+
10 rows in set (0.01 sec)

How things changed in MySQL 8 with utf8mb4 character set ?

mysql> UPDATE custfeeds.reactions SET reacted = 'super like ' WHERE id = 13015;
Query OK, 1 row affected, 0 warning (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> SELECT reactions FROM custfeeds.reactions WHERE id = 13015;
+-------------+
| reactions   |
+-------------+
| super liked |
+-------------+
1 row in set (0.00 sec)

Conclusion

Traditionally MySQL is built for scaling web-scale database infrastructure operations, In the modern web applications / mobile apps. , emojis and a multitude of charsets / collation needs to coexist. To address this compelling need, in MySQL 8.0 default character set has been changed from latin-1 to utf8mb4.

The post MySQL 8 default character set is utf8mb4 appeared first on The WebScale Database Infrastructure Operations Experts.

Step 1 – Check existing logs and their size:

[root@localhost ~]# lsof -c mysqld | grep ib_logfile
mysqld  1018 mysql    5uW     REG              253,0  50331648   180228 /var/lib/mysql/ib_logfile0
mysqld  1018 mysql   11uW     REG              253,0  50331648   180229 /var/lib/mysql/ib_logfile1

Step 2 – Shutdown MySQL

[root@localhost ~]# systemctl stop mysqld 
[root@localhost ~]# systemctl status mysqld 
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Sat 2018-05-12 16:33:51 IST; 5s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Process: 5848 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS (code=exited, status=0/SUCCESS)
  Process: 5831 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
 Main PID: 5848 (code=exited, status=0/SUCCESS)
   Status: "SERVER_SHUTTING_DOWN"

May 12 16:33:43 localhost.localdomain systemd[1]: Starting MySQL Server...
May 12 16:33:44 localhost.localdomain systemd[1]: Started MySQL Server.
May 12 16:33:49 localhost.localdomain systemd[1]: Stopping MySQL Server...
May 12 16:33:51 localhost.localdomain systemd[1]: Stopped MySQL Server.
[root@localhost ~]# 

Step 3 – From reliability / safety perspective, We don’t recommend you remove the existing log files. You may need these files to restore database if anything goes wrong unfortunately

[root@localhost ~]# find /var/lib/mysql -type f -name "ib_logfile?" -exec mv {} {}_OLD \;
[root@localhost ~]# ls /var/lib/mysql
auto.cnf       ca-key.pem       db1              ib_logfile1_OLD     private_key.pem  sys
binlog.000008  ca.pem           ib_buffer_pool   mysql               public_key.pem   undo_001
binlog.000009  client-cert.pem  ibdata1          mysql.ibd           server-cert.pem  undo_002
binlog.index   client-key.pem   ib_logfile0_OLD  performance_schema  server-key.pem
[root@localhost ~]#

Step 4 –  Resize innodb_log_file_size system variable in my.cnf using you favorite editor of choice, In this post we have used 64M (which actually is a good value for many mid sized systems, bigger values are always a concern under some situations)

[root@localhost ~]# grep 'innodb_log_file_size' /etc/my.cnf
innodb_log_file_size=64M

Step 5 – Restart MySQL instance

[root@localhost ~]# systemctl start mysqld 
[root@localhost ~]# systemctl status mysqld 
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2018-05-12 16:44:30 IST; 7s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Process: 5938 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
 Main PID: 5955 (mysqld)
   Status: "SERVER_OPERATING"
   CGroup: /system.slice/mysqld.service
           └─5955 /usr/sbin/mysqld

May 12 16:44:29 localhost.localdomain systemd[1]: Starting MySQL Server...
May 12 16:44:30 localhost.localdomain systemd[1]: Started MySQL Server.

You have successfully resized InnoDB log files size  

[root@localhost ~]# lsof -c mysqld | grep ib_logfile
mysqld  5955 mysql   10uW     REG              253,0  67108864   180206 /var/lib/mysql/ib_logfile0
mysqld  5955 mysql   11uW     REG              253,0  67108864   180230 /var/lib/mysql/ib_logfile1
[root@localhost ~]# 

The post How to resize InnoDB logs ? appeared first on The WebScale Database Infrastructure Operations Experts.

Last_IO_Errno: 1236
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: ‘Could not open log file’

How can we safely purge MySQL binary log files ? 

1. On each slave server, use SHOW SLAVE STATUS to check which log file it is reading.
2. Get the binary log files details on the master with SHOW BINARY LOGS.
3. Check for the earliest log file among all the slaves, This is the target file. If all the slaves are up to date, this is the last log file on the list.
4. Make a backup of all log files you are about to delete (We insist this step to our customers to avoid human errors)
5. Purge all log file up to but please do not include the target file.

The default binary log expiration is 30 days, You can get this details from MySQL system variable binlog_expire_logs_seconds . When using MySQL replication, You should specify an expiration period that no lower than maximum time your slaves might lag behind the master.

mysql> select @@binlog_expire_logs_seconds;
+------------------------------+
| @@binlog_expire_logs_seconds |
+------------------------------+
|                      2592000 |
+------------------------------+
1 row in set (0.00 sec)

List binary log files  in MySQL

mysql> SHOW BINARY LOGS;
+---------------+-----------+
| Log_name      | File_size |
+---------------+-----------+
| binlog.000001 |       657 |
| binlog.000002 |       178 |
| binlog.000003 |       178 |
| binlog.000004 |       178 |
| binlog.000005 |       178 |
| binlog.000006 |      3354 |
| binlog.000007 |       199 |
| binlog.000008 |       155 |
+---------------+-----------+
8 rows in set (0.00 sec)

mysql> 

** Both “SHOW BINARY LOGS” and “SHOW MASTER LOGS” are synonyms. **

Purging MySQL binary log files  using PURGE BINARY LOGS command 

The PURGE BINARY LOGS command deletes all the binary log files listed in the log index file prior to the specified log file name or date .

mysql> PURGE BINARY LOGS TO 'binlog.000001';
Query OK, 0 rows affected (0.01 sec)

mysql> PURGE BINARY LOGS BEFORE '2018-05-11 20:20:20';
Query OK, 0 rows affected, 1 warning (0.01 sec)

** ABOVE STATEMENTS HAS NO EFFECT IF THE MYSQL SERVER WAS NOT STARTED WITH –log-bin OPTION ENABLED

What you should be careful about before purging MySQL binary log files ? 

There are several ways to purge MySQL binary log files but none guarantee slave has already applied the binary log transactions and can be safely removed (Now, that’s the reason we have the title for this topic in red color ) , This is when “mysqlbinlogpurge” tool comes really helpful. The “mysqlbinlogpurge” tool is a part of MySQL utilities, You can download it from here . This tools ensures that any binary log files that in use or required by any of the slave in replication ecosystem will not be deleted.

How “mysqlbinlogpurge” finds when it is safe to purge binary log files ?

A MySQL slave has two entities to complete the replication successfully: the Slave_IO thread is accountable for collecting the events successfully from the master, While Slave_SQL threads(s) is responsible for executing the events locally. To monitor Slave IO / Slave SQL is running successfully and where they are at in their process, you have to run the command ” SHOW SLAVE STATUS ”

mysql > show slave statusG
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.56.3
Master_User: India
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000047
Read_Master_Log_Pos: 374
Relay_Log_File: mysql-relay.000713
Relay_Log_Pos: 45793012
Relay_Master_Log_File: mysql-bin.000038
<strong>Slave_IO_Running: Yes
Slave_SQL_Running: Yes
...
</strong>Exec_Master_Log_Pos: 45792781

The following are important matrices / parameters and their descriptions:

Master_Log_File / Read_Master_Log_Pos – This is what Slave_IO is currently fetching from the master.

Relay_Master_Log_File / Exec_Master_Log_Pos – This is what Slave_SQL thread is currently executing in terms of the Masters coordinates (master’s log file).

Relay_Log_File / Relay_Log_Pos – This is what the SQL thread is currently executing in terms of the Slave’s coordinates (relay log file)

The Master_Log_File is the latest binlog file on MySQL master that Slave_IO reads from and it is this file, and any files after this on the Master server, that we must preserve for successful completion of replication. The Relay_Log_File is the point of execution in MySQL Master’s binlog that Slave_SQL thread has successfully executed.

How to use “mysqlbinlogpurge” for purging MySQL binary log files ? 

We recommend our customers to use –dry-run option with “mysqlbinlogpurge” to check what tool is going to deliver :

** if ever slave is stopped, “mysqlbinlogpurge” tool will report error and stop purging binary log files:

$ mysqlbinlogpurge --master=root:india@192.168.56.3:3306 
>           --slaves=root:singapore@192.168.56.4:3306,root:srilanka@192.168.56.5:3306 
>           --dry-run
ERROR: Can not verify the status for slave singapore@192.168.56.4:3306. Make sure the slave are active and accessible.

You can check what the “mysqlbinlogpurge” is going to perform before executing, you can use the –dry-run option:

$ mysqlbinlogpurge --master=root:root@192.168.56.3:3306 --slaves=root:root@192.168.56.4:3306,root:root@192.168.56.5:45009 --dry-run
# Latest binlog file replicated by all slaves: mysql-bin.000481
# To manually purge purge the binary logs Execute the following query:
PURGE BINARY LOGS TO 'mysql-bin.000482'

** To remove binary log file, you just need to remove the –dry-run option.

mysqlbinlogpurge can occasionally go wrong under following circumstances :

• Not reliable when used in Multi-Source replication ecosystem.
• If slave corrupts the local relay log, We need to restart replication from Relay_Master_log_file, and mysqlbinlogpurge might already have executed the purge of required binlog .

Summary 

We caution our customers who regularly purge their binary log files on being reasonably certain that relay logs won’t be corrupted . We don’t recommend mysqlbinlogpurge multi-source replication ecosystem.

The post Purging binary logs from MySQL Master safely appeared first on The WebScale Database Infrastructure Operations Experts.

What you can expect from subscribing to our Vacation DBA services ?

• Elite-class global team of DBA available for you 24*7.
• Database infrastructure operations performance monitoring 24*7.
• Vendor neutral and independent DBA services – MySQL, MariaDB, MyRocks and Percona Server.
• MySQL installation and configuration.
• MySQL performance benchmarking.
• MySQL capacity planning and sizing.
• MySQL performance optimization and tuning.
• SQL and Index optimization.
• Disk I/O optimization.
• MySQL troubleshooting.
• MySQL database recovery services.
• MySQL high availability solutions.
• MySQL upgrades.
• Emergency DBA support and On-call DBA services – When you need us, we are available on-call (24*7).

When customers usually subscribe for vacation DBA services  ?

• Resident DBA left the company and business needs DBA coverage till they hire new DBA.
• Resident DBA is on vacation – Companies guarantee great work-life balance to the team, It’s not professional calling DBA for support when he/she is on an vacation.
• Leverage with external DBA team for 24*7 MySQL support.
• You need a expert DBA coverage for an limited period.
• Your website traffic is very high (usually for holidays / new year etc.) and you need highly responsive 24*7 DBA support.

☛ Why our customers love working with us ?

• Global team – We have expert MySQL consultants operating 24*7*365 from multiple locations worldwide.
• Vendor neutral and independent – Experts in MySQL, MariaDB, Percona Server and ClickHouse.
• We are available on a short notice for emergency DBA support services.
• No long-term contract option available – You can hire us for as-low-as 40 hours.
• We bill you only for the hours worked, Discounts available for advanced booking (only for vacation DBA services).
• Performance benchmarking – We do stress testing of the system proactively to avoid incidents which can directly impact your business.
• Capacity planning and sizing – Not more nor less, We size your systems optimally and this include recommendations on CPU, memory, storage and network.
• Performance health check, diagnostics and forensics – We will be first the one to know your performance bottleneck and fix before it become ugly.
• Performance optimization and tuning – We do full-stack optimization which include Linux, MySQL and PHP also
• Disk I/O tuning – We are experts in designing optimal storage infrastructure architecture and disk I/O distribution.
• SQL tuning – We are experts in re-writing your SQL for performance, Indeed we often see the major performance improvements by just tuning expensive SQL.
• Designing optimal schema, indexes and SQL – Design and build optimal schema and SQL for performance and scalability.
• Partitioning and archiving solutions –  Building systems horizontally scalable for web-scale platforms.
• Scale-out and replication solutions We build horizontally scalable systems which are highly available too.
• Sharding and horizontal partitioning solutions – Building systems horizontally scalable and reliable across multiple-DC for web-scale platforms.
• Clustering solutions – Grow your MySQL infrastructure horizontally for performance, scalability and reliability.
• High Availability and SRE – Architecting and building highly available and reliable database infrastructure operations  for web-scale.
• Data recovery services – Building robust disaster recovery solutions, zero data loss systems and multi-location backup retention.
• Database upgrades and migration – Seamless upgrades and migration of database infrastructure on zero downtime.

☛ Technology focus – Vendor neutral and independent

☛ MinervaDB Remote DBA plans

☛ Customer testimonials

” Shiv is a expert in MySQL performance, He can fine tune MySQL performance at instance, application and infrastructure level in a shortest duration, I would love to hire him again “

David  Hutton
Head of IT operations
National Geographic

” If it’s about MySQL performance and scalability, I will first call Shiv and He has helped us several times in building optimal, scalable and highly available MySQL infrastructure operations “

Mark Gray
  IT Manager
Nike Technologies

” If you are building an highly reliable MySQL ecosystem, Hiring Shiv and his team will simplify your project. He is a guru to build an highly available and fault tolerant web property “ 

Kevin Thomson
Business Head – Media properties
AOL

” Shiv and his team built an custom MySQL high availability solution for us across data centers which enabled 24*7*365 availability of our business services. His Soultions are non vendor biased and cost efficient “

Keshav Patel
Group Manager – IT
Lastminute.com 

” Thinking about outsourcing your DBA function ? Shiv is the guy who you should be talking to, He can build an highly reliable 24*7 remote DBA team for an fraction of cost to hiring a Sr. level resident DBA “

Brian Lewis
Lead Systems Engineer
Priceline.com  

” The shortest emergency DBA support provider I have ever worked with, They are highly responsive and professional. If we suspect something going wrong with our database systems, The immediate action item is contact Shiv and his team”

Sherly Williams
Manager -Business Continuity
GAP Inc. 

” We have contracted MinervaDB for 24*7 remote DBA services, They delivered MySQL maximum reliability and availability solutions ahead of the project schedule using 100% open source and free tools, saving considerably our IT budget. “

Simon Matthew
IT Manager – Media & Entertainment
Vodafone PLC 

The post Vacation DBA Services appeared first on The WebScale Database Infrastructure Operations Experts.

The post Why engage MinervaDB for web-scale DBA consulting and remote DBA services ? appeared first on The WebScale Database Infrastructure Operations Experts.