The default value of innodb_thread_concurrency is “0” , This mean, there is no limit on the number of concurrently executing threads. To reduce excessive context switching you can set innodb_thread_concurrency > “0”  . Now, How do you size your innodb_thread_concurrency ? There are several factors we consider before recommending the values, like your CPU, memory, Linux distribution / kernel, your application’s database architecture and other variables like innodb_adaptive hash_index (AHI) . So we recommend you to benchmark thoroughly before setting the values for innodb_thread_concurrency.

innodb_thread_sleep_delay is applicable only when innodb_thread_concurrency is greater than “0” , That’s why MySQL works faster in Servers with multi-core high performance CPUs compared to several Servers with moderately performing  CPUs.

When you have configured innodb_thread_concurrency > “0” , The SQLs that may comprise of multiple row operations are assigned with a specific number of tickets by InnoDB specified in the global system variable innodb_concurrency_tickets (default is 5000), which allows thread to be scheduled repeatedly with an minimal resource usage. If the tickets run out, the thread is evicted and innodb_thread_concurrency is observed again, Which will eventually move the thread back to FIFO (first-in-first-out) queue waiting threads.

The post Configuring InnoDB Thread Concurrency for Performance appeared first on The WebScale Database Infrastructure Operations Experts.

Step 1 – Check existing logs and their size:

[root@localhost ~]# lsof -c mysqld | grep ib_logfile
mysqld  1018 mysql    5uW     REG              253,0  50331648   180228 /var/lib/mysql/ib_logfile0
mysqld  1018 mysql   11uW     REG              253,0  50331648   180229 /var/lib/mysql/ib_logfile1

Step 2 – Shutdown MySQL

[root@localhost ~]# systemctl stop mysqld 
[root@localhost ~]# systemctl status mysqld 
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since Sat 2018-05-12 16:33:51 IST; 5s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Process: 5848 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS (code=exited, status=0/SUCCESS)
  Process: 5831 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
 Main PID: 5848 (code=exited, status=0/SUCCESS)
   Status: "SERVER_SHUTTING_DOWN"

May 12 16:33:43 localhost.localdomain systemd[1]: Starting MySQL Server...
May 12 16:33:44 localhost.localdomain systemd[1]: Started MySQL Server.
May 12 16:33:49 localhost.localdomain systemd[1]: Stopping MySQL Server...
May 12 16:33:51 localhost.localdomain systemd[1]: Stopped MySQL Server.
[root@localhost ~]# 

Step 3 – From reliability / safety perspective, We don’t recommend you remove the existing log files. You may need these files to restore database if anything goes wrong unfortunately

[root@localhost ~]# find /var/lib/mysql -type f -name "ib_logfile?" -exec mv {} {}_OLD \;
[root@localhost ~]# ls /var/lib/mysql
auto.cnf       ca-key.pem       db1              ib_logfile1_OLD     private_key.pem  sys
binlog.000008  ca.pem           ib_buffer_pool   mysql               public_key.pem   undo_001
binlog.000009  client-cert.pem  ibdata1          mysql.ibd           server-cert.pem  undo_002
binlog.index   client-key.pem   ib_logfile0_OLD  performance_schema  server-key.pem
[root@localhost ~]#

Step 4 –  Resize innodb_log_file_size system variable in my.cnf using you favorite editor of choice, In this post we have used 64M (which actually is a good value for many mid sized systems, bigger values are always a concern under some situations)

[root@localhost ~]# grep 'innodb_log_file_size' /etc/my.cnf
innodb_log_file_size=64M

Step 5 – Restart MySQL instance

[root@localhost ~]# systemctl start mysqld 
[root@localhost ~]# systemctl status mysqld 
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2018-05-12 16:44:30 IST; 7s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Process: 5938 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
 Main PID: 5955 (mysqld)
   Status: "SERVER_OPERATING"
   CGroup: /system.slice/mysqld.service
           └─5955 /usr/sbin/mysqld

May 12 16:44:29 localhost.localdomain systemd[1]: Starting MySQL Server...
May 12 16:44:30 localhost.localdomain systemd[1]: Started MySQL Server.

You have successfully resized InnoDB log files size  

[root@localhost ~]# lsof -c mysqld | grep ib_logfile
mysqld  5955 mysql   10uW     REG              253,0  67108864   180206 /var/lib/mysql/ib_logfile0
mysqld  5955 mysql   11uW     REG              253,0  67108864   180230 /var/lib/mysql/ib_logfile1
[root@localhost ~]# 

The post How to resize InnoDB logs ? appeared first on The WebScale Database Infrastructure Operations Experts.

innodb_doublewrite (enabled by default) 

InnoDB stores all data twice, first to doublewrite buffer (storage area in system tablespace to write pages that are flushed from InnoDB buffer pool, before written in data file). If ever operating system / storage / mysqld process crash during the middle of page write, InnoDB can still find a durable copy of the page from doublewrite buffer for recovery. Though data is written twice,  the doublewrite buffer does not require twice as much I/O overhead or twice as many I/O operations,  . Because, Data is written to the doublewrite buffer itself as a large sequential chunk, with a single fsync() call to the operating system. If system tablespace data file (ibdata* files) are located on Fusion-io devices that support atomic writes, doublewrite buffering is automatically disabled and Fusion-io atomic writes are used for all data files, set innodb_flush_method system variable to O_DIRECT to gain maximum advantage of this feature.

innodb_flush_log_at_trx_commit (default value is “1”)

innodb_flush_log_trx_commit guarantees ultimate durability, The default value “1” writes the contents of InnoDB log buffer to the log file with each transaction commit and the log file is flushed to disk. If value is “0”, No writes from log buffer to log file will be performed at transaction commit, The contents of the InnoDB log buffer are written to log file once per second and the log file is flushed to disk.  Because the flush to disk operation only occurs approximately once per second, you can lose up to a second of transactions with any mysqld process crash. The value “2” flushes contents of InnoDB log buffer to the log file after each transaction commit and log file is flushed to disk approximately once per second.  Once-per-second flushing is not guaranteed to happen every second due to process scheduling issues. Regardless of innodb_flush_log_at_trx_commit  setting, Transaction will be either rolled back or committed. We recommend you to retain default value “1” for “innodb_flush_log_at_trx_commit” for maximum transaction durability.

innodb_flush_log_at_timeout  (default value is “1”)

innodb_flush_log_at_timeout  system variable setting allows you to set log flushing frequency to N seconds (N is between 1 & 2700, the default value is 1 second), However, any mysqld process crash can erase up to N seconds of transactions. The DDL changes and other internal InnoDB activities flush the InnoDB log independently of the innodb_flush_log_at_trx_commit setting, DDL logs are always flushed at transaction commit.

sync_binlog (default value is “1”)

When sync_binlog=1, all transaction are synchronized to the binary log before they are committed. Therefore, even in the event of an unexpected restart, any transactions that are missing from the binary log are only in prepared state. This causes the server’s automatic recovery routine to roll back those transactions. This guarantees that no transaction is lost from the binary log, and is the safest option. However this can have a negative impact on performance because of an increased number of disk writes. Using a higher value improves performance, but with the increased risk of data loss. If sync_binlog=0 or greater than 1, transactions are committed without having been synchronized to disk. Therefore, in the event of a power failure or operating system crash, it is possible that the server has committed some transactions that have not been synchronized to the binary log. Therefore it is impossible for the recovery routine to recover these transactions and they will be lost from the binary log.

The post How InnoDB confirms maximum transaction durability in MySQL 8 ? appeared first on The WebScale Database Infrastructure Operations Experts.

We often enable MySQL slow_query_log to proactively troubleshoot the performance, There is so lot of information with-in slow_query_log which is enough to address most common (even complex ones occasionally) MySQL performance bottlenecks. What if we haven’t planned for the log lifecycle management of slow query log ? Your log will really grow huge (we keep long_query_time as-low-as 1 second), So we have to plan for slow_query_log rotating to avoid MySQL outage due to “no space left in the disk” to grow further. There are two ways you can do log rotate, I have explained below:

1. Manual log rotating (We use it only when there is an emergency)
2. Using logrotate to automatic MySQL log rotation (We love it, MinervaDB is an big fan of MySQL automation & DevOps. but we also recommend to use it very carefully)

Step-by-step manual log rotating of slow_query_log

Step 1 – Get the path of slow_query_log

mysql> show variables like '%slow_query%';
+---------------------+-----------------------------------+
| Variable_name       | Value                             |
+---------------------+-----------------------------------+
| slow_query_log      | ON                                |
| slow_query_log_file | /var/lib/mysql/localhost-slow.log |
+---------------------+-----------------------------------+
2 rows in set (0.00 sec)

Step 2 – Disable slow_query_log temporarily

mysql> set global slow_query_log=off;
Query OK, 0 rows affected (0.01 sec)

Step 3 – Flush only slow query logs

mysql> flush slow logs;
Query OK, 0 rows affected (0.00 sec)

Step 4 – Rename the old slow query log and or compress the same

[root@localhost ~]# mv /var/lib/mysql/localhost-slow.log /var/lib/mysql/localhost-slow-$(date +%Y-%m-%d).log

[root@localhost mysql]# ls *.log
localhost-slow-2018-04-18.log

[root@localhost mysql]# gzip -c /var/lib/mysql/localhost-slow-2018-04-18.log > /var/lib/mysql/localhost-slow-2018-04-18.log.gz
[root@localhost mysql]# ls *.gz
localhost-slow-2018-04-18.log.gz
[root@localhost mysql]# 

Step 5 – Re-enable slow_query_log

mysql> 
mysql> set global slow_query_log=on;
Query OK, 0 rows affected (0.00 sec)

Using logrotate  to rotate MySQL slow_query_log 

We at MinervaDB love automation, logrotate make slow_query_log rotation simple , To enable logrotate you can use logrotate config file in  /etc/logrotate.d/ mysql-slow-query:

/var/lib/mysql/localhost-slow.log {
    size 2G
    dateext
    compress
    missingok
    rotate 30
    notifempty
    delaycompress
    sharedscripts
    nocopytruncate
    create 660 mysql mysql
    postrotate
        /usr/bin/mysql -e 'select @@global.slow_query_log into @sq_log_save; set global slow_query_log=off; select sleep(5); FLUSH SLOW LOGS; select sleep(10); set global slow_query_log=@sq_log_save;'
    endscript
    rotate 250
}

We have explained below logrotate script:

• size 2G: Rotate the slow query log only if it is more that 2GB.
• dateext: archive the old slow query log files with a date extension, YYYYMMDD format.
• compress: compress slow query log files using gzip.
• delaycompress: Delay the compression of the previous slow query log file until next rotation cycle.
• missingok: Don’t issue error message, if slow query log file is missing.
• rotate30: Retain 30 slow query log files before deleting old files.
• Run prerotate & postrotate  scripts only once, no matter how many slow query logs matches to the wildcard pattern
• nocopytruncate: Don’t truncate the original slow query log file in place after creating a copy. logrotate use rename() system call to move the file to a new file name and expects its daemon to be signaled to reopen it’s slow query log file.
• copytruncate: logrotate truncates the original slow query log file in place after creating a copy. We don’t recommend you to use this option because truncation blocks MySQL due to OS serializing the access on inode. This happens more on ext3 file system compared to xfs
• create 660 mysql mysql: create new slow query log file after rotation which will be owned by mysql with permission mode 660
• postrotate: this script is executed after slow query log rotation is done

The post Rotating MySQL slow query log appeared first on The WebScale Database Infrastructure Operations Experts.

Step 1: Stop the MySQL server and confirm that it has shutdown without any errors

Step 2: Edit my.cnf to configure innodb_log_file_size (log file size) and innodb_log_files_in_group (number of log files)

Step 3: Restart MySQL server

If ever InnoDB finds innodb_log_file_size differs from the redo log files, It writes a log checkpoint, closes and removes the old log files, create again new log files at requested size and opens the new log files.

The post How to change the number or size of InnoDB Redo Log Files ? appeared first on The WebScale Database Infrastructure Operations Experts.

• Confirm it’s not CPU –  Check CPU usage, if the workload is more than 90% then bottleneck is also CPU
• Innodb_buffer_pool_size – If table data is cached in the InnoDB buffer pool, it can be accessed repeatedly by queries without requiring any disk I/O.  You can specify the size of the buffer pool with system variable innodb_buffer_pool_size. The typically recommended value for innodb_buffer_pool_size is 70 to 80 percent of available system memory but need not to be larger than the database size.
• Optimal flush method – Default InnoDB flush method is fsync() ; In some versions of GNU/Linux and Unix, This makes write performance unacceptably slow. We encourage you to benchmark the write performance by setting system variable innodb_flush_method to O_DSYNC .
• Tuning the I/O capacity for performance – Increase the innodb_io_capacity only if you find throughput is affected, Higher values causes more frequent flushing, avoiding the backlog of work that can cause dips in throughput. You can consider reducing innodb_io_capacity by monitoring InnoDB throughput from the following matrices of SHOW ENGINE INNODB STATUS command :
• • History list length low, below a few thousand  
  • Insert buffer merges close to rows inserted.
  • Modified pages in buffer pool consistently well below innodb_max_dirty_pages_pct of the buffer pool. (Measure at a time when the server is not doing bulk inserts; it is normal during bulk inserts for the modified pages percentage to rise significantly.)
  • Log sequence number – Last checkpoint is at less than 7/8 or ideally less than 6/8 of the total size of the InnoDB log files.
• Invest in non-rotational storage – Random I/O operations perform optimally in Non-rotational storage. Random I/O oriented files include general data files (both tables and indexes) , undo tablespace files and temporary tablespace files.
• • Review settings for the following configuration options when using non-rotational storage:
    • Innodb_checksum_algorithm – The crc32 option uses a faster checksum algorithm and is recommended for fast storage systems.
    • Innodb_flush_neighbors – This option optimizes I/O for rotational storage devices. Disable it for non-rotational storage or a mix of rotational and non-rotational storage.
    • Innodb_io_capacity – The default setting of 200 is generally sufficient for a lower-end non-rotational storage device. For higher-end, bus-attached devices, consider a higher setting such as 1000.
    • Innodb_io_capacity_max – The default value of 2000 is intended for workloads that use non-rotational storage. For a high-end, bus-attached non-rotational storage device, consider a higher setting such as 2500.
    • Innodb_log_compressed_pages – If redo logs are on non-rotational storage, consider disabling this option to reduce logging.
    • Innodb_log_file_size – If redo logs are on non-rotational storage, configure this option to maximize caching and write combining.
    • Innodb_page_size – Consider using a page size that matches the internal sector size of the disk. Early-generation SSD devices often have a 4k sector size. Some newer devices have a 16k sector size. The default InnoDB page size is 16k. Keeping the page size close to the storage device block size minimizes the amount of unchanged data that is rewritten to disk.
    • Binlog_row_image – If binary logs are on non-rotational storage and all tables have primary keys, consider setting this option to minimal to reduce logging.
• Store system tablespace files on Fusion-io devices – To get the advantage of doublewrite buffer-related I/O optimization, you can store system tablespace files (“ibdata files”) on Fusion-io devices that support atomic writes. In this case, doublewrite buffering (innodb_doublewrite) is automatically disabled and Fusion-io atomic writes are used for all data files. This feature is only supported on Fusion-io hardware and is only enabled for Fusion-io NVMFS on Linux. To take full advantage of this feature, an innodb_flush_method setting of O_DIRECT is recommended.
• Disable logging of compressed pages – If you are using InnoDB table compression feature, images of re-compressed pages are written to the redo log when changes are made to compressed data. This behaviour is controlled by innodb_log_controlled pages, which is enabled by default to prevent corruption that can occur if a different version of zlib compression algorithm is used during recovery. Once confirmed that zlib version will not change, you can disable innodb_log_compressed pages to reduce redo log generation for workloads that modify compressed data.  

The post InnoDB Disk I/O Performance appeared first on The WebScale Database Infrastructure Operations Experts.

InnoDB Enhancements:

• Supports online ALTER TABLE RENAME INDEX operations so we can now remake the index without table copy operations .
• We can resize innodb_buffer_pool_size dynamically, This makes buffer pool resize without restarting MySQL server possible .
• InnoDB supports Transportable Tablespace for partitioned InnoDB tables .
• More responsive Innochecksum utility , For those who don’t know about Innochecksum – This tool calculates the checksum for each InnoDB page and reports mismatches .
• MySQL 5.7 InnoDB supports “spatial indexes” and online DDL operation .
• Optimized InnoDB buffer pool dump / reload operations, A new system variable innodb_buffer_pool_dump_pct allows you to specify percentage of most recently used pages in each buffer pool to read out and dump

Performance Optmization

• MySQL 5.7 optimizer avoids the creation of temporary table for results of UNION ALL queries, This greatly benefits in reducing disk I/O and storage. Morgan Tocker blog shares more info. on this .
• MySQL 5.7 supports generated columns (virtual columns or computed columns) . This is a great feature, It’s actually adding a column + index on any logical schema without really any data redundancy .
• Faster bulk data load – InnoDB in MySQL 5.7 performance much faster bulk data loading when creating or rebuilding indexes through sorted index build, This also makes creation of FULLTEXT indexes faster .
• InnoDB in MySQL 5.7 support parallel flushing of dirty pages from the buffer pool(s). I remember using two-threaded flushing from Percona Server 5.6 . 
• Explain already running queries with EXPLAIN FOR CONNECTION , This makes proactive MySQL optimization possible .
• JSON format for EXPLAIN improved a lot from MySQL 5.6 to 5.7 – JSON format for EXPLAIN in MySQL 5.7 prints total cost of the query execution so that you can compare execution plans for both optimal and expensive SQL

Replication

• Multi-source replication – You can have several masters for a slave, This makes transactions from multiple master databases propagated towards slave database through multiple communication channels .
• MySQL 5.7 performance schema database tables have all the information about SHOW SLAVE STATUS.  Performance schema tables information on SHOW SLAVE STATUS can be used to generate more informative reports, More information on this can be found in MySQL 5.7 documentation here
• MySQL 5.7 generates faster SHOW SLAVE STATUS reports by not waiting from STOP SLAVE to finish which can be blocked by long running SQL query from replication. I remember using Percona LOCK FREE SHOW SLAVE STATUS feature from Percona Server 5.5
• Since MySQL 5.7 you can perform CHANGE MASTER TO without stopping slave via STOP SLAVE command
• MySQL 5.7 has now much improved parallel replication . We have new system variable slave_parallel_type , When using a multi-threaded slave (slave_parallel_workers is greater than 0), this option specifies the policy used to decide which transactions are allowed to execute in parallel on the slave , You can read more about it here
• More responsive Global Transaction Identifiers (GTID) mode, grid_mode system variable can be enabled / disabled dynamically, This means you can now enable / disable GTID in replication without synchronising and restarting the entire set of MySQL servers .  In Percona Server 5.6 I have used online GTID deployment feature so that you can deploy GTID on existing replication infrastructure without making MySQL master read_only and stopping all slaves .

The post Percona Server for MySQL 5.7 new features appeared first on The WebScale Database Infrastructure Operations Experts.

Please confirm data is sampled not for 0 or 1 seconds, Anything above 10 seconds helps

• Status : This shows timestamp, monitor name and the number of seconds, or the elapsed time between the current time and time the InnoDB monitor last displayed. The per-second averages are based upon this time .
• BACKGROUND THREAD : The srv_master_thread lines in BACKGROUND THREAD shows work performed by the main background thread .
• SEMAPHORES :  The threads waiting for a semaphore and stats show on how many number of times threads have needed a spin or wait on a mutex or rw-lock semaphore, If this number of threads is large then may be I/O or contention issues . You can reduce innodb_thread_concurrency system variable (this works often not always) to solve contention related to thread scheduling. Spin rounds per wait shows the total number of spinlock rounds per OS wait for a mutex. LATEST FOREIGN KEY ERROR is shown only if there has been a foreign key constraint error. LATEST DETECTED DEADLOCK will be shown only if there has been a deadlock, It displays the transaction involved in the deadlock and statements being executed, held and required locked and the transaction rolled back to .
• TRANSACTIONS : The output of this section can help to identify all the lock contention and the reasons for deadlocks .
• FILE I/O : InnoDB thread information as well as pending I/O operations and I/O performance statistics .
• LOG : InnoDB log information, including current log sequence number, how far the log has been flushed to disk, the position at which InnoDB last took a checkpoint, pending writes and write performance statistics .
• BUFFER POOL AND MEMORY : These matrices report the information on buffer pool pages read and written, Which allows you to see the number of data file I/O operations performed by your queries .
• ROW OPERATIONS : Information about the main thread, including the number and performance rate for each type of its own operation .

Example output:

mysql> show engine innodb status\G;

*************************** 1. row ***************************

  Type: InnoDB

  Name: 

Status: 

=====================================

180105 22:39:35 INNODB MONITOR OUTPUT

=====================================

Per second averages calculated from the last 17 seconds

-----------------

BACKGROUND THREAD

-----------------

srv_master_thread loops: 11 1_second, 10 sleeps, 0 10_second, 3 background, 3 flush

srv_master_thread log flush and writes: 10

----------

SEMAPHORES

----------

OS WAIT ARRAY INFO: reservation count 4457, signal count 1072

Mutex spin waits 4670, rounds 140100, OS waits 1494

RW-shared spins 2090, rounds 62850, OS waits 1169

RW-excl spins 1735, rounds 53820, OS waits 948

Spin rounds per wait: 30.00 mutex, 30.07 RW-shared, 31.02 RW-excl

------------

TRANSACTIONS

------------

Trx id counter 8D121

Purge done for trx's n:o < 862C4 undo n:o < 0

History list length 667

LIST OF TRANSACTIONS FOR EACH SESSION:

---TRANSACTION 0, not started

MySQL thread id 3, OS thread handle 0x7f965a163700, query id 25591 localhost root

show engine innodb status

---TRANSACTION 8D102, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1855, OS thread handle 0x7f9656b61700, query id 25560 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D10B, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1848, OS thread handle 0x7f9656eae700, query id 25569 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D116, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1806, OS thread handle 0x7f96570f7700, query id 25580 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D108, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1816, OS thread handle 0x7f9659fdd700, query id 25566 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D110, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1807, OS thread handle 0x7f9656ff3700, query id 25574 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D111, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1833, OS thread handle 0x7f9656c65700, query id 25575 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0EB, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1815, OS thread handle 0x7f9656e6d700, query id 25537 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D104, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1852, OS thread handle 0x7f9656f30700, query id 25562 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0FB, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1808, OS thread handle 0x7f9656ce7700, query id 25553 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11E, not started

MySQL thread id 1846, OS thread handle 0x7f9656a5d700, query id 25588 localhost root

---TRANSACTION 8D0E7, not started

MySQL thread id 1825, OS thread handle 0x7f9656fb2700, query id 25533 localhost root

---TRANSACTION 8D101, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1823, OS thread handle 0x7f9656a9e700, query id 25559 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D106, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1824, OS thread handle 0x7f9657444700, query id 25564 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F6, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1812, OS thread handle 0x7f9657403700, query id 25548 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F3, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1810, OS thread handle 0x7f96569db700, query id 25545 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F1, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1842, OS thread handle 0x7f9657340700, query id 25543 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F0, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1851, OS thread handle 0x7f96571ba700, query id 25542 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D10F, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1811, OS thread handle 0x7f9656ca6700, query id 25573 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D107, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1818, OS thread handle 0x7f9656d69700, query id 25565 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F7, not started

MySQL thread id 1817, OS thread handle 0x7f965723c700, query id 25549 localhost root

---TRANSACTION 8D0FC, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1834, OS thread handle 0x7f9656a1c700, query id 25554 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0FA, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1841, OS thread handle 0x7f96573c2700, query id 25552 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0EA, not started

MySQL thread id 1850, OS thread handle 0x7f96572be700, query id 25536 localhost root

---TRANSACTION 8D109, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1822, OS thread handle 0x7f9657075700, query id 25567 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D105, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1827, OS thread handle 0x7f965a01e700, query id 25563 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11D, not started

MySQL thread id 1809, OS thread handle 0x7f9657179700, query id 25587 localhost root

---TRANSACTION 8D100, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1839, OS thread handle 0x7f9656ba2700, query id 25558 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D10C, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1837, OS thread handle 0x7f96571fb700, query id 25570 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F4, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1819, OS thread handle 0x7f9657485700, query id 25546 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0FE, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1835, OS thread handle 0x7f9657507700, query id 25556 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0E4, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1826, OS thread handle 0x7f9656f71700, query id 25530 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D120, not started

MySQL thread id 1843, OS thread handle 0x7f965a05f700, query id 25590 localhost root

---TRANSACTION 8D0E6, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1820, OS thread handle 0x7f9656adf700, query id 25532 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11F, not started

MySQL thread id 1836, OS thread handle 0x7f9656daa700, query id 25589 localhost root

---TRANSACTION 8D0FD, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1844, OS thread handle 0x7f9656d28700, query id 25555 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0FF, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1828, OS thread handle 0x7f965727d700, query id 25557 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D10A, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1831, OS thread handle 0x7f96572ff700, query id 25568 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11A, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1853, OS thread handle 0x7f9657381700, query id 25584 localhost root query end

INSERT INTO t1 VALUES (1759592334,'3lkoxjtvgLu5xKHSTTtJuGE5F5QqmCcppCTmvFZScRZQgim93gSxwb24gKmIPEzEQStMjQiCu7WapGbkw4ilXch3xRLMhKSzgLDOovSi2qGj6rKvnuYAWDDJgaZDu2')

---TRANSACTION 8D0E9, not started

MySQL thread id 1854, OS thread handle 0x7f9657138700, query id 25535 localhost root

---TRANSACTION 8D0F5, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1813, OS thread handle 0x7f9656b20700, query id 25547 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0ED, not started

MySQL thread id 1829, OS thread handle 0x7f96574c6700, query id 25539 localhost root

---TRANSACTION 8D10D, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1821, OS thread handle 0x7f96570b6700, query id 25571 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D103, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1832, OS thread handle 0x7f9656eef700, query id 25561 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11B, not started

MySQL thread id 1830, OS thread handle 0x7f9656deb700, query id 25585 localhost root

---TRANSACTION 8D0F9, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1849, OS thread handle 0x7f965a0a0700, query id 25551 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11C, not started

MySQL thread id 1838, OS thread handle 0x7f965a0e1700, query id 25586 localhost root

---TRANSACTION 8D0EC, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1847, OS thread handle 0x7f9657034700, query id 25538 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D10E, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1814, OS thread handle 0x7f9656be3700, query id 25572 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D115, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1845, OS thread handle 0x7f9656c24700, query id 25579 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D119, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1840, OS thread handle 0x7f9656e2c700, query id 25583 localhost root query end

INSERT INTO t1 VALUES (1759592334,'3lkoxjtvgLu5xKHSTTtJuGE5F5QqmCcppCTmvFZScRZQgim93gSxwb24gKmIPEzEQStMjQiCu7WapGbkw4ilXch3xRLMhKSzgLDOovSi2qGj6rKvnuYAWDDJgaZDu2')

---TRANSACTION 8D080, not started

MySQL thread id 5, OS thread handle 0x7f965a122700, query id 25430 localhost root

--------

FILE I/O

--------

I/O thread 0 state: waiting for completed aio requests (insert buffer thread)

I/O thread 1 state: waiting for completed aio requests (log thread)

I/O thread 2 state: waiting for completed aio requests (read thread)

I/O thread 3 state: waiting for completed aio requests (read thread)

I/O thread 4 state: waiting for completed aio requests (read thread)

I/O thread 5 state: waiting for completed aio requests (read thread)

I/O thread 6 state: waiting for completed aio requests (write thread)

I/O thread 7 state: waiting for completed aio requests (write thread)

I/O thread 8 state: waiting for completed aio requests (write thread)

I/O thread 9 state: waiting for completed aio requests (write thread)

Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] ,

 ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0

Pending flushes (fsync) log: 0; buffer pool: 0

1491 OS file reads, 4495 OS file writes, 4410 OS fsyncs

0.06 reads/s, 16384 avg bytes/read, 263.98 writes/s, 258.98 fsyncs/s

-------------------------------------

INSERT BUFFER AND ADAPTIVE HASH INDEX

-------------------------------------

Ibuf: size 1, free list len 0, seg size 2, 0 merges

merged operations:

 insert 0, delete mark 0, delete 0

discarded operations:

 insert 0, delete mark 0, delete 0

Hash table size 276671, node heap has 1 buffer(s)

280.57 hash searches/s, 652.61 non-hash searches/s

---

LOG

---

Log sequence number 561353275

Log flushed up to   561352811

Last checkpoint at  556826545

0 pending log writes, 0 pending chkp writes

4396 log i/o's done, 257.98 log i/o's/second

----------------------

BUFFER POOL AND MEMORY

----------------------

Total memory allocated 137363456; in additional pool allocated 0

Dictionary memory allocated 75535

Buffer pool size   8191

Free buffers       6699

Database pages     1491

Old database pages 566

Modified db pages  256

Pending reads 0

Pending writes: LRU 0, flush list 0, single page 0

Pages made young 0, not young 0

0.00 youngs/s, 0.00 non-youngs/s

Pages read 1480, created 11, written 94

0.06 reads/s, 0.65 creates/s, 5.47 writes/s

Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000

Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s

LRU len: 1491, unzip_LRU len: 0

I/O sum[0]:cur[0], unzip sum[0]:cur[0]

--------------

ROW OPERATIONS

--------------

0 queries inside InnoDB, 0 queries in queue

1 read views open inside InnoDB

Main thread process no. 1430, id 140283695105792, state: sleeping

Number of rows inserted 14565, updated 0, deleted 0, read 2595539

856.71 inserts/s, 0.00 updates/s, 0.00 deletes/s, 135022.35 reads/s

----------------------------

END OF INNODB MONITOR OUTPUT

============================

1 row in set (0.00 sec)

ERROR: 

No query specified

The post SHOW ENGINE INNODB STATUS walk through appeared first on The WebScale Database Infrastructure Operations Experts.

The following file format configuration parameters have new default values:

• The innodb_file_format default value is now changed to Barracuda . The previous default value was Antelope.
• The innodb_large_prefix default value is changed to ON . The previous default value was OFF

The following configuration parameters are deprecated in and may be removed in the future

• innodb_file_format
• innodb_file_format_check
• innodb_file_format_max
• innodb_large_prefix 

The post How InnoDB file formatting changed from MySQL 5.6 appeared first on The WebScale Database Infrastructure Operations Experts.

How PT-ONLINE-SCHEMA-CHANGE works ?

PT-ONLINE-SCHEMA-CHANGE tool is developed by Percona to alter tables without locking them during ALTER TABLE operation. This  tool creates a new empty table like the original table with the necessary structure changes, copy the data from the original table in multiple smaller chunks to the new table, drop the original table and finally rename the new table to the original name. I have explained below PT-ONLINE-SCHEMA-CHANGE with simple example

Consider table city (I have copied below table creation script) , You want to index column “city” for some performance reason. Now what will be impact if you create index on this table while your transaction is still on ? MySQL blocks your transaction causing performance outage (may not be too long) so how can we create index with no impact on application performance and reliability ? The example below explains the step-by-step process of what I am referring to:

mysql> show create table city\G;

*************************** 1. row ***************************

       Table: city

Create Table: CREATE TABLE `city` (

  `city_id` smallint(5) unsigned NOT NULL AUTO_INCREMENT,

  `city` varchar(50) NOT NULL,

  `country_id` smallint(5) unsigned NOT NULL,

  `last_update` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,

  PRIMARY KEY (`city_id`),

  KEY `idx_fk_country_id` (`country_id`),

  CONSTRAINT `fk_city_country` FOREIGN KEY (`country_id`) REFERENCES `country` (`country_id`) ON UPDATE CASCADE

) ENGINE=InnoDB AUTO_INCREMENT=901 DEFAULT CHARSET=utf8

1 row in set (0.00 sec)

Now I am using PT-ONLINE-SCHEMA-CHANGE to create index on column “city” of table “city”:

[root@localhost ~]# pt-online-schema-change D=sakila,t=city,h=localhost,u=root --execute --alter="ADD INDEX city_idx (city)" --alter-foreign-keys-method="auto" --ask-pass

Enter MySQL password: 

No slaves found.  See --recursion-method if host localhost.localdomain has slaves.

Not checking slave lag because no slaves were found and --check-slave-lag was not specified.

Operation, tries, wait:

  analyze_table, 10, 1

  copy_rows, 10, 0.25

  create_triggers, 10, 1

  drop_triggers, 10, 1

  swap_tables, 10, 1

  update_foreign_keys, 10, 1

Child tables:

  `sakila`.`address` (approx. 603 rows)

Will automatically choose the method to update foreign keys.

Altering `sakila`.`city`...

Creating new table...

Created new table sakila._city_new OK.

Altering new table...

Altered `sakila`.`_city_new` OK.

2017-11-29T23:38:36 Creating triggers...

2017-11-29T23:38:36 Created triggers OK.

2017-11-29T23:38:36 Copying approximately 601 rows...

2017-11-29T23:38:36 Copied rows OK.

2017-11-29T23:38:36 Max rows for the rebuild_constraints method: 79360

Determining the method to update foreign keys...

2017-11-29T23:38:36   `sakila`.`address`: 603 rows; can use rebuild_constraints

2017-11-29T23:38:36 Analyzing new table...

2017-11-29T23:38:36 Swapping tables...

2017-11-29T23:38:36 Swapped original and new tables OK.

2017-11-29T23:38:36 Rebuilding foreign key constraints...

2017-11-29T23:38:36 Rebuilt foreign key constraints OK.

2017-11-29T23:38:36 Dropping old table...

2017-11-29T23:38:36 Dropped old table `sakila`.`_city_old` OK.

2017-11-29T23:38:36 Dropping triggers...

2017-11-29T23:38:36 Dropped triggers OK.

Successfully altered `sakila`.`city`.

PT-ONLINE-SCHEMA-CHANGE Limitations

The following are limitations while using PT-ONLINE-SCHEMA-CHANGE :

• The PRIMARY KEY or UNIQUE INDEX needs to be present in the table, This is because the tool creates a DELETE trigger to keep new table updated while the process is running. The exception is when a PRIMARY KEY or UNIQUE INDEX is being created from existing columns as part of the ALTER clause; In such cases it will be using these column(s) for the DELETE trigger
• The RENAME clause cannot be used to rename the table
• The columns cannot be rename by dropping and re-adding with new name. This tool will not copy the original column’s data to the new column
• The tool will fail if you add a column without a default value and make it NOT NULL
• We need to specify _constraint_name to DROP FOREIGN KEY constraint_name, i.e  –alter “DROP FOREIGN KEY _city_id_fk”
• The tool does not use LOCK IN SHARE MODE with MySQL 5.0 because it can cause a slave error which breaks replication

ONLINE DDL

MySQL 5.6 introduced online DDL for InnoDB tables, which means ALTER TABLE statements will no longer block access to database modification while running.  Typically if you want to make changes to your production servers please prefer during off hours, Especially when the server utilization is at its lowest for both CPU and disk. The Online DDL syntax is almost similar to the normal DDL statement with two additional parameters : ALGORITHM and LOCK

ALGORITHM:

• INPLACE: The table change will happen “in-place” (without rebuilding the entire table), Often copying data to a temporary table is not required
• COPY: This is actually traditional method of copying data to a temporary table, rebuilding the table and reconstructing the secondary indexes

LOCK: 

NONE: Read and write operations are allowed during the database altering process

SHARED: Only read operations are allowed during the database altering process

EXCLUSIVE: Neither SELECT nor DML are allowed, The entire  table will be locked for both reading and writing

ONLINE DDL Example

mysql> alter table city 

    -> add index city_idx(city), algorithm=inplace, lock=none;

Query OK, 0 rows affected (0.03 sec)

Records: 0  Duplicates: 0  Warnings: 0

The post Difference between ONLINE DDL and PT-ONLINE-SCHEMA-CHANGE appeared first on The WebScale Database Infrastructure Operations Experts.