Deprecated: Function Yoast\WP\SEO\Conditionals\Schema_Blocks_Conditional::get_feature_flag is deprecated since version Yoast SEO 20.5 with no alternative available. in /home1/minerho3/public_html/wp-includes/functions.php on line 6078

Deprecated: Function Yoast\WP\SEO\Conditionals\Schema_Blocks_Conditional::get_feature_flag is deprecated since version Yoast SEO 20.5 with no alternative available. in /home1/minerho3/public_html/wp-includes/functions.php on line 6078

Deprecated: Function Yoast\WP\SEO\Conditionals\Schema_Blocks_Conditional::get_feature_flag is deprecated since version Yoast SEO 20.5 with no alternative available. in /home1/minerho3/public_html/wp-includes/functions.php on line 6078

Warning: Cannot modify header information - headers already sent by (output started at /home1/minerho3/public_html/wp-includes/functions.php:6078) in /home1/minerho3/public_html/wp-includes/feed-rss2.php on line 8
MySQL - MariaDB - ClickHouse - InnoDB - Galera Cluster - MySQL Support - MariaDB Support - MySQL Consulting - MariaDB Consulting - MySQL Remote DBA - MariaDB Remote DBA - Emergency DBA Support - Remote DBA - Database Migration - PostgreSQL - PostgreSQL Consulting - PostgreSQL Support - PostgreSQL Remote DBA http://minervadb.com/index.php/category/mysql-troubleshooting/ Committed to Building Optimal, Scalable, Highly Available, Fault-Tolerant, Reliable and Secured WebScale Database Infrastructure Operations Fri, 10 Apr 2020 05:38:45 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.3 http://minervadb.com/wp-content/uploads/2017/10/cropped-LogoColorTextRight-32x32.jpeg MySQL - MariaDB - ClickHouse - InnoDB - Galera Cluster - MySQL Support - MariaDB Support - MySQL Consulting - MariaDB Consulting - MySQL Remote DBA - MariaDB Remote DBA - Emergency DBA Support - Remote DBA - Database Migration - PostgreSQL - PostgreSQL Consulting - PostgreSQL Support - PostgreSQL Remote DBA http://minervadb.com/index.php/category/mysql-troubleshooting/ 32 32 How MySQL handles connection ? Troubleshooting MySQL ERROR 1040, Too many connections! http://minervadb.com/index.php/2020/02/03/how-mysql-handles-connection-troubleshooting-mysql-error-1040-too-many-connections/ http://minervadb.com/index.php/2020/02/03/how-mysql-handles-connection-troubleshooting-mysql-error-1040-too-many-connections/#comments Mon, 03 Feb 2020 13:11:31 +0000 http://minervadb.com/?p=3054 Understanding how MySQL handles connections and Troubleshooting MySQL error 1040 – Too many connections! How to interpret “MySQL error 1040 – Too many connections ! ” ? When a client tries to log into MySQL it [...]

The post How MySQL handles connection ? Troubleshooting MySQL ERROR 1040, Too many connections! appeared first on The WebScale Database Infrastructure Operations Experts.

]]>
Understanding how MySQL handles connections and Troubleshooting MySQL error 1040 – Too many connections!

How to interpret “MySQL error 1040 – Too many connections ! ” ?

When a client tries to log into MySQL it may sometimes be rejected and receive an error message saying that there are “too many connections“. This means that the maximum number of clients that may be connected to the server has been reached. Either the client will have to wait for another client to log off, or the administrator will have to increase the maximum number of connections allowed.

Information about connections to a server can be found using the SHOW STATUS statement:

SHOW STATUS LIKE 'max_used_connections';

Prerequisite – Few points to remember before working or troubleshooting MySQL ” Too many connections ! ” error

  1. MySQL does not have it’s own thread handling mechanism / implementation and it completely relies on the thread handling implementation of the underlying operating system.
  2. MySQL system variable max_connections control the maximum number of clients the server permits to connect simultaneously,   You may have to increase max_connections if more clients attempt to connect simultaneously then the server is configured to handle (Explained more in detail –  “Too many connections”).
  3. How MySQL connection handling (both connects and disconnects) works  ?
    1. MySQL Clients connects to MySQL Server via a simple and regular TCP-IP connect message though port 3306 on the Server instance
    2. The incoming connection requests are queued and then processed by the receiver thread one by one, All that receiver thread does is create user thread.  It’s actually user thread which handles the client-server protocol for connection management, Which involves allocate and initialize the corresponding THD for user authentication based on credentials stored on THD’s security policies / directories and finally if successfully authorized in the connection phase, the user thread will enter command phase
    3. The  receiver  thread will either create a new OS thread or reuse and existing “free” OS thread if available in the  thread cache. So we strongly recommend increasing  thread cache in  cases where number of  connections fluctuates  between ver few connections  and having  many connections. But there are three things which a thread might need to wait for:  A mutex, a database lock, or IO.
    4. THD basically is a large data structure used for several purposes like connection management, authorization and even unto to query execution, So how  much THD consumes memory is directly proportionate to the query execution complexities and connection traffic.

MySQL error – Too many connections, How to fix ?

Recently one of customers ( among the top 5 largest e-commerce companies in the world ) called us to check how graceful their connection handling works during peak hours of business, They had issues in  the past with ” ERROR 1040: Too many connections “ and that clearly explains the maximum number of clients that may be connected to the server has been reached so either the client will have to wait for another client to log off, or the administrator will have to increase the maximum number of connections allowed. so wanted us to do a detailed health-check on MySQL connection management and address “Too many connections” error proactively, We have explained below on how we could successfully reproduce this issue and recommended the fix:

Goal: Manage  50,000 connections on MySQL 8.0 (Ubuntu)

The default setting for system variable max_connections is “151”and we are benchmarking 50K connections so the first step before benchmarking  is to increase max_connections to 50000. we increased max_connections to 50000 dynamically and what happened after that was not expected, We have copied the results below:

root@MDB1:~# mysql -uroot -pMDB@PassWd2020 -se "select @@max_connections"
@@max_connections
697

We got only 697 connections, Let’s interpret MySQL error log before proceeding to next steps.. We have copied the same below:

2020-01-30T19:52:35.136192Z0 [Warning] Changed limits: max_open_files: 5129 (requested 10000)
2020-01-30T19:54:13.241937Z0 [Warning] Changed limits: max_connections: 4152 (requested 10000)
2020-01-30T19:57:47.51617Z0 [Warning] Changed limits: table_open_cache: 533 (requested 15000)

This is due to open files limitations for MySQL so let’s increase now the number of allowed open files for MySQL, The following steps we did to fix this resource limit issue:

  • Option  1 – Locate the systemd configuration folder for MySQL and create file /etc/systemd/system/mysqld.service.d/override.conf (file can be called anything ending with .conf).
    • Add LimitNOFILE=55000 in the file override.conf
    • Add TasksMax=55000 in the file override.conf
    • Add LimitNPROC=55000 in the file override.conf
  • Option 2 – We can also create/modify the override file by using native systemctl command like: systemctl edit mysql
root@MDB1:~# cat /etc/systemd/system/mysql.service.d/override.conf
[Service]
LimitNOFILE=55000
TasksMax=55000
LimitNPROC=55000

** MySQL uses some files for additional work and we need to set LimitNOFILE, TasksMax and LimitMPROC higher to get 50000 connections, lets set it to 55000 and reload the systemd daemon and restart the MySQL service.

Reload the systmed daemon and restart the MySQL service:

root@MDB1:~# systemctl daemon-reload
root@MDB1:~# systemctl restart mysql

Now let’s check max_connections to confirm the change applied:

root@MDB1:~# mysql -uroot -pMDB@PassWd2020 -se "select @@max_connections"
mysql: [Warning] Using a password on the command line interface can be insecure.
@@max_connections
50000

Conclusion

We have no fixed value recommendations for system variable max_connections, It completely depends on your application load and how your application does connection handling. We advice our customers to avoid too many connections opened concurrently because each thread connected needs memory and there is also resource intensive context switching causing overall performance degradation, Thanks for reading and comments are welcome !

References

Book your appointment for 30 minutes ( absolutely free ) MinervaDB consulting 

The post How MySQL handles connection ? Troubleshooting MySQL ERROR 1040, Too many connections! appeared first on The WebScale Database Infrastructure Operations Experts.

]]>
http://minervadb.com/index.php/2020/02/03/how-mysql-handles-connection-troubleshooting-mysql-error-1040-too-many-connections/feed/ 1
error: Failed dependencies: pkgconfig(openssl) is needed by mysql-commercial-devel-8.0.11-1.1.el7.x86_64 http://minervadb.com/index.php/2018/05/20/error-failed-dependencies-pkgconfigopenssl-is-needed-by-mysql-commercial-devel-8-0-11-1-1-el7-x86_64/ Sun, 20 May 2018 08:19:08 +0000 http://minervadb.com/?p=1448 Recently we were evaluating MySQL 8 Enterprise for a customer. During our installation (our MySQL Enterprise Edition installations are always RPM based) using RPM file (CentOS Linux release 7.4.1708 (Core))  we ended up in a [...]

The post error: Failed dependencies: pkgconfig(openssl) is needed by mysql-commercial-devel-8.0.11-1.1.el7.x86_64 appeared first on The WebScale Database Infrastructure Operations Experts.

]]>
Recently we were evaluating MySQL 8 Enterprise for a customer. During our installation (our MySQL Enterprise Edition installations are always RPM based) using RPM file (CentOS Linux release 7.4.1708 (Core))  we ended up in a very unusual error, “error: Failed dependencies:pkgconfig(openssl) is needed by mysql-commercial-devel-8.0.11-1.1.el7.x86_64” , This was happening more specifically during the installation of “mysql-commercial-devel-8.0.11-1.1.el7.x86_64.rpm” . So decided to write this post about the error and how we fixed it.

[root@localhost MySQL8-Enterprise-RPM]# rpm -ivh mysql-commercial-server-8.0.11-1.1.el7.x86_64.rpm mysql-commercial-client-8.0.11-1.1.el7.x86_64.rpm mysql-commercial-libs-8.0.11-1.1.el7.x86_64.rpm mysql-commercial-common-8.0.11-1.1.el7.x86_64.rpm mysql-commercial-devel-8.0.11-1.1.el7.x86_64.rpm 
warning: mysql-commercial-server-8.0.11-1.1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
error: Failed dependencies:
	pkgconfig(openssl) is needed by mysql-commercial-devel-8.0.11-1.1.el7.x86_64

After some research, we found “openssl-devel” installation will address the dependency issues, Actually “openssl-devel” is a prerequisite for successful installation of “mysql-commercial-devel-8.0.11-1.1.el7.x86_64.rpm

[root@localhost MySQL8-Enterprise-RPM]# yum install openssl-devel

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.vbctv.in
 * extras: mirror.vbctv.in
 * updates: mirror.vbctv.in
Resolving Dependencies
--> Running transaction check
---> Package openssl-devel.x86_64 1:1.0.2k-12.el7 will be installed
--> Processing Dependency: zlib-devel(x86-64) for package: 1:openssl-devel-1.0.2k-12.el7.x86_64
--> Processing Dependency: krb5-devel(x86-64) for package: 1:openssl-devel-1.0.2k-12.el7.x86_64
--> Running transaction check

Installed:
  openssl-devel.x86_64 1:1.0.2k-12.el7                                 

Dependency Installed:
  keyutils-libs-devel.x86_64 0:1.5.8-3.el7                             
  krb5-devel.x86_64 0:1.15.1-19.el7                                    
  libcom_err-devel.x86_64 0:1.42.9-12.el7_5                            
  libkadm5.x86_64 0:1.15.1-19.el7                                      
  libselinux-devel.x86_64 0:2.5-12.el7                                 

  libselinux-python.x86_64 0:2.5-12.el7                                
  libselinux-utils.x86_64 0:2.5-12.el7                                 
  libsepol.x86_64 0:2.5-8.1.el7                                        
  libss.x86_64 0:1.42.9-12.el7_5                                       

Complete!

 

[root@localhost MySQL8-Enterprise-RPM]# rpm -ivh mysql-commercial-devel-8.0.11-1.1.el7.x86_64.rpm
warning: mysql-commercial-devel-8.0.11-1.1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:mysql-commercial-devel-8.0.11-1.1################################# [100%]

Confirm all packages installed for an successful MySQL operations:

[root@localhost MySQL8-Enterprise-RPM]# rpm -qa | grep mysql
mysql-commercial-common-8.0.11-1.1.el7.x86_64
mysql-commercial-client-8.0.11-1.1.el7.x86_64
mysql-commercial-libs-compat-8.0.11-1.1.el7.x86_64
mysql-commercial-devel-8.0.11-1.1.el7.x86_64
mysql-commercial-libs-8.0.11-1.1.el7.x86_64
mysql-commercial-server-8.0.11-1.1.el7.x86_64
[root@localhost MySQL8-Enterprise-RPM]# 

The post error: Failed dependencies: pkgconfig(openssl) is needed by mysql-commercial-devel-8.0.11-1.1.el7.x86_64 appeared first on The WebScale Database Infrastructure Operations Experts.

]]>
MySQL 8.0 : Improved error logging http://minervadb.com/index.php/2018/04/11/mysql-8-0-improved-error-logging/ Wed, 11 Apr 2018 17:53:22 +0000 http://minervadb.com/?p=1304 We extensively use MySQL error logging for proactive diagnostics, MySQL 8 has improved error logging which is capable of sending multiple format (like JSON) logs  to multiple destinations like Kibana for log processing and analysis. [...]

The post MySQL 8.0 : Improved error logging appeared first on The WebScale Database Infrastructure Operations Experts.

]]>
We extensively use MySQL error logging for proactive diagnostics, MySQL 8 has improved error logging which is capable of sending multiple format (like JSON) logs  to multiple destinations like Kibana for log processing and analysis. Let’s see how this works:

MySQL 8.0 default configuration

mysql> select * from global_variables where VARIABLE_NAME like 'log_error_%';
+---------------------+----------------------------------------+
| VARIABLE_NAME       | VARIABLE_VALUE                         |
+---------------------+----------------------------------------+
| log_error_services  | log_filter_internal; log_sink_internal |
| log_error_verbosity | 2                                      |
+---------------------+----------------------------------------+

Now that means log events will pass first through log_filter_interval (built-in filter component) and then through log_sink_interval (built-in log writer component). To configure and enable log_error_services you first have to install log component using INSTALL COMPONENT.

Currently available log components lib / plugins:

  • JSON log writer – component_log_sink_json.so
  • Built-in log filter and writer – log_filter_interval and log_sink_interval

Step 1 – Load JSON writer first if it’s not loaded:

INSTALL COMPONENT 'file://component_log_sink_json';

Step 2 – Set log_error_services to ‘ log_filter_interval ; log_sink_interval ; log_sink_json ‘  (we will set this with MySQL 8 very new feature SET PERSIST)

SET PERSIST log_error_services = 'log_filter_internal; log_sink_internal; log_sink_json';

You may also do same by setting my.cnf

[mysqld]
log_error_services='log_filter_internal; log_sink_internal; log_sink_json'

How do you order the components in log_error_services is very significant :

log_filter_internal; log_sink_1; log_sink_2

This makes log events to pass through built-in filter, then to first writer and then to second writer. Both writers receive filtered log events. Confirm that to log_error_services value / system variable:

log_sink_1; log_filter_internal; log_sink_2

So here, The log events pass through first writer, then to built-in filter, then to second writer. The second writer receives filtered events. You can as well configure error logging in way that one log contains messages for all log events and other containing messages of only the subset of log events.

The post MySQL 8.0 : Improved error logging appeared first on The WebScale Database Infrastructure Operations Experts.

]]>
Mass killing of MySQL connections / processes http://minervadb.com/index.php/2018/01/06/mass-killing-of-mysql-connections-processes/ Sat, 06 Jan 2018 10:01:27 +0000 http://minervadb.com/?p=581 Sometimes we may have to kill several MySQL connections / processes, How can we do that effortlessly ? We have to first list down the connections you want to kill with ” SHOW FULL PROCESSLIST [...]

The post Mass killing of MySQL connections / processes appeared first on The WebScale Database Infrastructure Operations Experts.

]]>
Sometimes we may have to kill several MySQL connections / processes, How can we do that effortlessly ? We have to first list down the connections you want to kill with ” SHOW FULL PROCESSLIST ” (Query 1 ), The Query 2 below will do mass killing MySQL connections / processes quickly .

Query 1

mysql> SHOW FULL PROCESSLIST\G;

*************************** 1. row ***************************

     Id: 3

   User: root

   Host: localhost

     db: NULL

Command: Query

   Time: 0

  State: NULL

   Info: SHOW FULL PROCESSLIST

*************************** 2. row ***************************

     Id: 15008

   User: root

   Host: localhost

     db: mysqlslap

Command: Sleep

   Time: 0

  State: 

   Info: NULL

*************************** 3. row ***************************

     Id: 19759

   User: root

   Host: localhost

     db: mysqlslap

Command: Query

   Time: 0

  State: query end

   Info: INSERT INTO t1 VALUES (95275444,'bNIrBDBl81tjzdvuOpQRCXgX37xGtzLKEXBIcE3k7xK7aFtqxC99jqYnpTviK83bf6lGDgsKd4R3KLmHPnI8TqnIKj1gjw7N2sXFZNS2Svyg8cpZN7atxL39w4igsp')

*************************** 4. row ***************************

     Id: 19760

   User: root

   Host: localhost

     db: mysqlslap

Command: Sleep

   Time: 0

  State: 

   Info: NULL

*************************** 5. row ***************************

     Id: 19761

   User: root

   Host: localhost

     db: mysqlslap

Command: Query

   Time: 0

  State: query end

   Info: INSERT INTO t1 VALUES (95275444,'bNIrBDBl81tjzdvuOpQRCXgX37xGtzLKEXBIcE3k7xK7aFtqxC99jqYnpTviK83bf6lGDgsKd4R3KLmHPnI8TqnIKj1gjw7N2sXFZNS2Svyg8cpZN7atxL39w4igsp')

*************************** 6. row ***************************

     Id: 19762

   User: root

   Host: localhost

     db: mysqlslap

Command: Query

   Time: 0

  State: query end

   Info: INSERT INTO t1 VALUES (95275444,'bNIrBDBl81tjzdvuOpQRCXgX37xGtzLKEXBIcE3k7xK7aFtqxC99jqYnpTviK83bf6lGDgsKd4R3KLmHPnI8TqnIKj1gjw7N2sXFZNS2Svyg8cpZN7atxL39w4igsp')

*************************** 7. row ***************************

     Id: 19763

   User: root

   Host: localhost

     db: mysqlslap

Command: Query

   Time: 0

  State: query end

   Info: INSERT INTO t1 VALUES (95275444,'bNIrBDBl81tjzdvuOpQRCXgX37xGtzLKEXBIcE3k7xK7aFtqxC99jqYnpTviK83bf6lGDgsKd4R3KLmHPnI8TqnIKj1gjw7N2sXFZNS2Svyg8cpZN7atxL39w4igsp')

The query below get you the script for mass killing of  MySQL in one key stroke , I usually use this when I am suspecting something unacceptable is happening in MySQL.

Query 2
mysql> select concat('KILL ',id,';') from information_schema.processlist where user='root';

+------------------------+

| concat('KILL ',id,';') |

+------------------------+

| KILL 21159;            |

| KILL 21158;            |

| KILL 21157;            |

| KILL 21156;            |

| KILL 21155;            |

| KILL 21154;            |

| KILL 21153;            |

| KILL 21152;            |

| KILL 21151;            |

| KILL 21150;            |

| KILL 21149;            |

| KILL 21148;            |

| KILL 21147;            |

| KILL 21146;            |

| KILL 21145;            |

| KILL 21144;            |

| KILL 21143;            |

| KILL 21142;            |

| KILL 21141;            |

| KILL 21140;            |

 

mysql> select concat('KILL ',id,';') from information_schema.processlist where user='root' into outfile '/var/lib/mysql-files/kill-mysql-conn.txt';

Query OK, 1 row affected (0.00 sec)

mysql> source /var/lib/mysql-files/kill-mysql-conn.txt

 

The post Mass killing of MySQL connections / processes appeared first on The WebScale Database Infrastructure Operations Experts.

]]>
SHOW ENGINE INNODB STATUS walk through http://minervadb.com/index.php/2018/01/06/show-engine-innodb-status-walk-through/ Sat, 06 Jan 2018 06:55:59 +0000 http://minervadb.com/?p=578 SHOW ENGINE INNODB STATUS is a very direct and simple SHOW ENGINE statement to display InnoDB Monitor output. The output of this query is quite detailed and exhaustive, It has wealth of information to troubleshoot [...]

The post SHOW ENGINE INNODB STATUS walk through appeared first on The WebScale Database Infrastructure Operations Experts.

]]>
SHOW ENGINE INNODB STATUS is a very direct and simple SHOW ENGINE statement to display InnoDB Monitor output. The output of this query is quite detailed and exhaustive, It has wealth of information to troubleshoot most common MySQL performance / operations issues real-time. Often MySQL DBAs ask me how to interpret this data so thought I will write a post on this. Please run SHOW ENGINE INNODB STATUS if you are building a pattern of InnoDB operations over a period of time.

Please confirm data is sampled not for 0 or 1 seconds, Anything above 10 seconds helps

  • Status : This shows timestamp, monitor name and the number of seconds, or the elapsed time between the current time and time the InnoDB monitor last displayed. The per-second averages are based upon this time .
  • BACKGROUND THREAD : The srv_master_thread lines in BACKGROUND THREAD shows work performed by the main background thread .
  • SEMAPHORES :  The threads waiting for a semaphore and stats show on how many number of times threads have needed a spin or wait on a mutex or rw-lock semaphore, If this number of threads is large then may be I/O or contention issues . You can reduce innodb_thread_concurrency system variable (this works often not always) to solve contention related to thread scheduling. Spin rounds per wait shows the total number of spinlock rounds per OS wait for a mutex. LATEST FOREIGN KEY ERROR is shown only if there has been a foreign key constraint error. LATEST DETECTED DEADLOCK will be shown only if there has been a deadlock, It displays the transaction involved in the deadlock and statements being executed, held and required locked and the transaction rolled back to .
  • TRANSACTIONS : The output of this section can help to identify all the lock contention and the reasons for deadlocks .
  • FILE I/O : InnoDB thread information as well as pending I/O operations and I/O performance statistics .
  • LOG : InnoDB log information, including current log sequence number, how far the log has been flushed to disk, the position at which InnoDB last took a checkpoint, pending writes and write performance statistics .
  • BUFFER POOL AND MEMORY : These matrices report the information on buffer pool pages read and written, Which allows you to see the number of data file I/O operations performed by your queries .
  • ROW OPERATIONS : Information about the main thread, including the number and performance rate for each type of its own operation .

Example output:

mysql> show engine innodb status\G;

*************************** 1. row ***************************

  Type: InnoDB

  Name: 

Status: 

=====================================

180105 22:39:35 INNODB MONITOR OUTPUT

=====================================

Per second averages calculated from the last 17 seconds

-----------------

BACKGROUND THREAD

-----------------

srv_master_thread loops: 11 1_second, 10 sleeps, 0 10_second, 3 background, 3 flush

srv_master_thread log flush and writes: 10

----------

SEMAPHORES

----------

OS WAIT ARRAY INFO: reservation count 4457, signal count 1072

Mutex spin waits 4670, rounds 140100, OS waits 1494

RW-shared spins 2090, rounds 62850, OS waits 1169

RW-excl spins 1735, rounds 53820, OS waits 948

Spin rounds per wait: 30.00 mutex, 30.07 RW-shared, 31.02 RW-excl

------------

TRANSACTIONS

------------

Trx id counter 8D121

Purge done for trx's n:o < 862C4 undo n:o < 0

History list length 667

LIST OF TRANSACTIONS FOR EACH SESSION:

---TRANSACTION 0, not started

MySQL thread id 3, OS thread handle 0x7f965a163700, query id 25591 localhost root

show engine innodb status

---TRANSACTION 8D102, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1855, OS thread handle 0x7f9656b61700, query id 25560 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D10B, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1848, OS thread handle 0x7f9656eae700, query id 25569 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D116, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1806, OS thread handle 0x7f96570f7700, query id 25580 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D108, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1816, OS thread handle 0x7f9659fdd700, query id 25566 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D110, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1807, OS thread handle 0x7f9656ff3700, query id 25574 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D111, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1833, OS thread handle 0x7f9656c65700, query id 25575 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0EB, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1815, OS thread handle 0x7f9656e6d700, query id 25537 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D104, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1852, OS thread handle 0x7f9656f30700, query id 25562 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0FB, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1808, OS thread handle 0x7f9656ce7700, query id 25553 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11E, not started

MySQL thread id 1846, OS thread handle 0x7f9656a5d700, query id 25588 localhost root

---TRANSACTION 8D0E7, not started

MySQL thread id 1825, OS thread handle 0x7f9656fb2700, query id 25533 localhost root

---TRANSACTION 8D101, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1823, OS thread handle 0x7f9656a9e700, query id 25559 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D106, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1824, OS thread handle 0x7f9657444700, query id 25564 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F6, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1812, OS thread handle 0x7f9657403700, query id 25548 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F3, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1810, OS thread handle 0x7f96569db700, query id 25545 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F1, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1842, OS thread handle 0x7f9657340700, query id 25543 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F0, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1851, OS thread handle 0x7f96571ba700, query id 25542 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D10F, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1811, OS thread handle 0x7f9656ca6700, query id 25573 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D107, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1818, OS thread handle 0x7f9656d69700, query id 25565 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F7, not started

MySQL thread id 1817, OS thread handle 0x7f965723c700, query id 25549 localhost root

---TRANSACTION 8D0FC, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1834, OS thread handle 0x7f9656a1c700, query id 25554 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0FA, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1841, OS thread handle 0x7f96573c2700, query id 25552 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0EA, not started

MySQL thread id 1850, OS thread handle 0x7f96572be700, query id 25536 localhost root

---TRANSACTION 8D109, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1822, OS thread handle 0x7f9657075700, query id 25567 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D105, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1827, OS thread handle 0x7f965a01e700, query id 25563 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11D, not started

MySQL thread id 1809, OS thread handle 0x7f9657179700, query id 25587 localhost root

---TRANSACTION 8D100, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1839, OS thread handle 0x7f9656ba2700, query id 25558 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D10C, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1837, OS thread handle 0x7f96571fb700, query id 25570 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0F4, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1819, OS thread handle 0x7f9657485700, query id 25546 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0FE, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1835, OS thread handle 0x7f9657507700, query id 25556 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0E4, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1826, OS thread handle 0x7f9656f71700, query id 25530 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D120, not started

MySQL thread id 1843, OS thread handle 0x7f965a05f700, query id 25590 localhost root

---TRANSACTION 8D0E6, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1820, OS thread handle 0x7f9656adf700, query id 25532 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11F, not started

MySQL thread id 1836, OS thread handle 0x7f9656daa700, query id 25589 localhost root

---TRANSACTION 8D0FD, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1844, OS thread handle 0x7f9656d28700, query id 25555 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0FF, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1828, OS thread handle 0x7f965727d700, query id 25557 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D10A, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1831, OS thread handle 0x7f96572ff700, query id 25568 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11A, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1853, OS thread handle 0x7f9657381700, query id 25584 localhost root query end

INSERT INTO t1 VALUES (1759592334,'3lkoxjtvgLu5xKHSTTtJuGE5F5QqmCcppCTmvFZScRZQgim93gSxwb24gKmIPEzEQStMjQiCu7WapGbkw4ilXch3xRLMhKSzgLDOovSi2qGj6rKvnuYAWDDJgaZDu2')

---TRANSACTION 8D0E9, not started

MySQL thread id 1854, OS thread handle 0x7f9657138700, query id 25535 localhost root

---TRANSACTION 8D0F5, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1813, OS thread handle 0x7f9656b20700, query id 25547 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D0ED, not started

MySQL thread id 1829, OS thread handle 0x7f96574c6700, query id 25539 localhost root

---TRANSACTION 8D10D, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1821, OS thread handle 0x7f96570b6700, query id 25571 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D103, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1832, OS thread handle 0x7f9656eef700, query id 25561 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11B, not started

MySQL thread id 1830, OS thread handle 0x7f9656deb700, query id 25585 localhost root

---TRANSACTION 8D0F9, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1849, OS thread handle 0x7f965a0a0700, query id 25551 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D11C, not started

MySQL thread id 1838, OS thread handle 0x7f965a0e1700, query id 25586 localhost root

---TRANSACTION 8D0EC, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1847, OS thread handle 0x7f9657034700, query id 25538 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D10E, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1814, OS thread handle 0x7f9656be3700, query id 25572 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D115, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1845, OS thread handle 0x7f9656c24700, query id 25579 localhost root query end

INSERT INTO t1 VALUES (73673339,'BN3152Gza4GW7atxJKACYwJqDbFynLxqc0kh30YTwgz3FktQ43XTrqJ4PQ25frn7kXhfXD8RuzN1j8Rf3y8ugKy6es3IbqPJM6ylCyD6xS7YcQCfHKZxYNvB7yTahm')

---TRANSACTION 8D119, not started flushing log

mysql tables in use 1, locked 1

MySQL thread id 1840, OS thread handle 0x7f9656e2c700, query id 25583 localhost root query end

INSERT INTO t1 VALUES (1759592334,'3lkoxjtvgLu5xKHSTTtJuGE5F5QqmCcppCTmvFZScRZQgim93gSxwb24gKmIPEzEQStMjQiCu7WapGbkw4ilXch3xRLMhKSzgLDOovSi2qGj6rKvnuYAWDDJgaZDu2')

---TRANSACTION 8D080, not started

MySQL thread id 5, OS thread handle 0x7f965a122700, query id 25430 localhost root

--------

FILE I/O

--------

I/O thread 0 state: waiting for completed aio requests (insert buffer thread)

I/O thread 1 state: waiting for completed aio requests (log thread)

I/O thread 2 state: waiting for completed aio requests (read thread)

I/O thread 3 state: waiting for completed aio requests (read thread)

I/O thread 4 state: waiting for completed aio requests (read thread)

I/O thread 5 state: waiting for completed aio requests (read thread)

I/O thread 6 state: waiting for completed aio requests (write thread)

I/O thread 7 state: waiting for completed aio requests (write thread)

I/O thread 8 state: waiting for completed aio requests (write thread)

I/O thread 9 state: waiting for completed aio requests (write thread)

Pending normal aio reads: 0 [0, 0, 0, 0] , aio writes: 0 [0, 0, 0, 0] ,

 ibuf aio reads: 0, log i/o's: 0, sync i/o's: 0

Pending flushes (fsync) log: 0; buffer pool: 0

1491 OS file reads, 4495 OS file writes, 4410 OS fsyncs

0.06 reads/s, 16384 avg bytes/read, 263.98 writes/s, 258.98 fsyncs/s

-------------------------------------

INSERT BUFFER AND ADAPTIVE HASH INDEX

-------------------------------------

Ibuf: size 1, free list len 0, seg size 2, 0 merges

merged operations:

 insert 0, delete mark 0, delete 0

discarded operations:

 insert 0, delete mark 0, delete 0

Hash table size 276671, node heap has 1 buffer(s)

280.57 hash searches/s, 652.61 non-hash searches/s

---

LOG

---

Log sequence number 561353275

Log flushed up to   561352811

Last checkpoint at  556826545

0 pending log writes, 0 pending chkp writes

4396 log i/o's done, 257.98 log i/o's/second

----------------------

BUFFER POOL AND MEMORY

----------------------

Total memory allocated 137363456; in additional pool allocated 0

Dictionary memory allocated 75535

Buffer pool size   8191

Free buffers       6699

Database pages     1491

Old database pages 566

Modified db pages  256

Pending reads 0

Pending writes: LRU 0, flush list 0, single page 0

Pages made young 0, not young 0

0.00 youngs/s, 0.00 non-youngs/s

Pages read 1480, created 11, written 94

0.06 reads/s, 0.65 creates/s, 5.47 writes/s

Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000

Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s

LRU len: 1491, unzip_LRU len: 0

I/O sum[0]:cur[0], unzip sum[0]:cur[0]

--------------

ROW OPERATIONS

--------------

0 queries inside InnoDB, 0 queries in queue

1 read views open inside InnoDB

Main thread process no. 1430, id 140283695105792, state: sleeping

Number of rows inserted 14565, updated 0, deleted 0, read 2595539

856.71 inserts/s, 0.00 updates/s, 0.00 deletes/s, 135022.35 reads/s

----------------------------

END OF INNODB MONITOR OUTPUT

============================

1 row in set (0.00 sec)

ERROR: 

No query specified

The post SHOW ENGINE INNODB STATUS walk through appeared first on The WebScale Database Infrastructure Operations Experts.

]]>