The post How MySQL handles connection ? Troubleshooting MySQL ERROR 1040, Too many connections! appeared first on The WebScale Database Infrastructure Operations Experts.
]]>When a client tries to log into MySQL it may sometimes be rejected and receive an error message saying that there are “too many connections“. This means that the maximum number of clients that may be connected to the server has been reached. Either the client will have to wait for another client to log off, or the administrator will have to increase the maximum number of connections allowed.
Information about connections to a server can be found using the SHOW STATUS statement:
SHOW STATUS LIKE 'max_used_connections';
Recently one of customers ( among the top 5 largest e-commerce companies in the world ) called us to check how graceful their connection handling works during peak hours of business, They had issues in the past with ” ERROR 1040: Too many connections “ and that clearly explains the maximum number of clients that may be connected to the server has been reached so either the client will have to wait for another client to log off, or the administrator will have to increase the maximum number of connections allowed. so wanted us to do a detailed health-check on MySQL connection management and address “Too many connections” error proactively, We have explained below on how we could successfully reproduce this issue and recommended the fix:
The default setting for system variable max_connections is “151”and we are benchmarking 50K connections so the first step before benchmarking is to increase max_connections to 50000. we increased max_connections to 50000 dynamically and what happened after that was not expected, We have copied the results below:
root@MDB1:~# mysql -uroot -pMDB@PassWd2020 -se "select @@max_connections" @@max_connections 697
We got only 697 connections, Let’s interpret MySQL error log before proceeding to next steps.. We have copied the same below:
2020-01-30T19:52:35.136192Z0 [Warning] Changed limits: max_open_files: 5129 (requested 10000) 2020-01-30T19:54:13.241937Z0 [Warning] Changed limits: max_connections: 4152 (requested 10000) 2020-01-30T19:57:47.51617Z0 [Warning] Changed limits: table_open_cache: 533 (requested 15000)
This is due to open files limitations for MySQL so let’s increase now the number of allowed open files for MySQL, The following steps we did to fix this resource limit issue:
root@MDB1:~# cat /etc/systemd/system/mysql.service.d/override.conf [Service] LimitNOFILE=55000 TasksMax=55000 LimitNPROC=55000
** MySQL uses some files for additional work and we need to set LimitNOFILE, TasksMax and LimitMPROC higher to get 50000 connections, lets set it to 55000 and reload the systemd daemon and restart the MySQL service.
root@MDB1:~# systemctl daemon-reload root@MDB1:~# systemctl restart mysql
Now let’s check max_connections to confirm the change applied:
root@MDB1:~# mysql -uroot -pMDB@PassWd2020 -se "select @@max_connections" mysql: [Warning] Using a password on the command line interface can be insecure. @@max_connections 50000
We have no fixed value recommendations for system variable max_connections, It completely depends on your application load and how your application does connection handling. We advice our customers to avoid too many connections opened concurrently because each thread connected needs memory and there is also resource intensive context switching causing overall performance degradation, Thanks for reading and comments are welcome !
References
The post How MySQL handles connection ? Troubleshooting MySQL ERROR 1040, Too many connections! appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post 2020 New Year Resolution with MinervaDB appeared first on The WebScale Database Infrastructure Operations Experts.
]]>MinervaDB wishing you all a very happy and prosperous new year – 2020 ! When we talk to our customers (both existing and prospective) and other community stakeholders during the month January, We get to know their resolutions (specifically on Business and Database Infrastructure Operations) and every company is so very passionate to share their goals for 2020, They actually spend so much energy is articulating this very well, loud and clear so that the true spirit never get diluted. In this blog we would like share with you few interesting points which really can influence all of the readers (most of our expected audiences are geeks and technology business owners) too. This post is mostly about how our resolutions are very well aligned to our customers ambitions / goals, As we often say – We are not an yet another DBA outsourcing company, We are always an extended team of customers committed to building optimal, scalable, highly reliable, fault-tolerant and secured Database Infrastructure Operations, Our accomplishments totally depend on the customer success.
Every successful business build loyalty and customer satisfaction year-on-year, They solve more customer problems faster and scale the growth ( with much deeper technology commitments ) to create an impact, so the following are our customer goals for 2020:
We are living on an Datanomy world, The data powers business and Database Infrastructure going unavailable will directly impact customers, employees and investors, We at MinervaDB is committed to building and delivering optimal, scalable, highly available, fault-tolerant, reliable and secured database infrastructure operations for our customers globally. To know more on how MinervaDB creates value for your Database Infrastructure Operations, Please download our corporate flyer PDF here.
The post 2020 New Year Resolution with MinervaDB appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post Vacation DBA Services appeared first on The WebScale Database Infrastructure Operations Experts.
]]>Technology Focus | Tools and Technologies |
---|---|
Linux | Ubuntu, Debian, CentOS, Red Hat Linux, Oracle Linux and SUSE Linux. |
MySQL | MySQL GA, MySQL Enterprise, InnoDB, MySQL Enterprise Backup, MySQL Cluster CGE, MySQL Enterprise Monitor, MySQL Utilities, MySQL Enterprise Audit, MySQL Enterprise Firewall and MySQL Router. |
Percona | Percona Server for MySQL, XtraDB, TokuDB, RocksDB, Percona Toolkit, Percona XtraBackup and PMM(Percona Monitoring & Management). |
MariaDB | MariaDB Server, RocksDB, MariaDB Galera Cluster, MariaDB Backup, MariaDB MaxScale and MariaDB ColumnStore. |
PostgreSQL | PostgreSQL Performance Benchmarking, Capacity Planning / Sizing, PostgreSQL Performance Optimization, PostgreSQL High Availability / Database Reliability Engineering, PostgreSQL Upgrades / Migration and PostgreSQL Security |
Cloud DBA Services | IaaS and DBaaS including: Oracle Cloud, Google CloudSQL, Amazon Aurora, AWS RDS®, EC2®, Microsoft Azure® and Rackspace® Cloud |
Performance Monitoring and Trending Platforms | MySQL Enterprise Monitor, Icinga, Zabbix, Prometheus and Grafana. |
High Availability, Scale-Out, Replication and Load Balancer | MySQL Group Replication, MySQL Cluster CGE, InnoDB Cluster, Galera Cluster, Percona XtraDB Cluster, MariaDB MaxScale, Continuent Tungsten Replicator, MHA (Master High Availability Manager and tools for MySQL), HAProxy, ProxySQL, MySQL Router and Vitess. |
Columnar Database Systems | ClickHouse, MariaDB ColumnStore |
DevOps. and Automation | Vagrant, Docker, Kubernetes, Jenkins, Ganglia, Chef, Puppet, Ansible, Consul, JIRA, Graylog and Grafana. |
Remote DBA Plan | Rate ( plus GST / Goods and Services Tax where relevant ) |
---|---|
On-Demand Remote DBA (8 hours Remote DBA per month) | US $1,600 / month |
Quarter DBA (40 hours of remote DBA services per month) | US $6,000 / month |
Half DBA (80 hours of remote DBA services per month) | US $10,000 / month |
Full DBA (160 hours of remote DBA services per month) | US $16,000 / month |
The Ultimate DBA (Remote DBA services for 24*7*365) | US $54,000 / month |
” Shiv is a expert in MySQL performance, He can fine tune MySQL performance at instance, application and infrastructure level in a shortest duration, I would love to hire him again “
David Hutton
Head of IT operations
National Geographic
” If it’s about MySQL performance and scalability, I will first call Shiv and He has helped us several times in building optimal, scalable and highly available MySQL infrastructure operations “
Mark Gray
IT Manager
Nike Technologies
” If you are building an highly reliable MySQL ecosystem, Hiring Shiv and his team will simplify your project. He is a guru to build an highly available and fault tolerant web property “
Kevin Thomson
Business Head – Media properties
AOL
” Shiv and his team built an custom MySQL high availability solution for us across data centers which enabled 24*7*365 availability of our business services. His Soultions are non vendor biased and cost efficient “
Keshav Patel
Group Manager – IT
Lastminute.com
” Thinking about outsourcing your DBA function ? Shiv is the guy who you should be talking to, He can build an highly reliable 24*7 remote DBA team for an fraction of cost to hiring a Sr. level resident DBA “
Brian Lewis
Lead Systems Engineer
Priceline.com
” The shortest emergency DBA support provider I have ever worked with, They are highly responsive and professional. If we suspect something going wrong with our database systems, The immediate action item is contact Shiv and his team”
Sherly Williams
Manager -Business Continuity
GAP Inc.
” We have contracted MinervaDB for 24*7 remote DBA services, They delivered MySQL maximum reliability and availability solutions ahead of the project schedule using 100% open source and free tools, saving considerably our IT budget. “
Simon Matthew
IT Manager – Media & Entertainment
Vodafone PLC
The post Vacation DBA Services appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post Why engage MinervaDB for web-scale DBA consulting and remote DBA services ? appeared first on The WebScale Database Infrastructure Operations Experts.
]]>
The post Why engage MinervaDB for web-scale DBA consulting and remote DBA services ? appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post MinervaDB Technology Partnership Program appeared first on The WebScale Database Infrastructure Operations Experts.
]]>Technology Focus | Tools and Technologies |
---|---|
Linux | Ubuntu, Debian, CentOS, Red Hat Linux, Oracle Linux and SUSE Linux. |
MySQL | MySQL GA, MySQL Enterprise, InnoDB, MySQL Enterprise Backup, MySQL Cluster CGE, MySQL Enterprise Monitor, MySQL Utilities, MySQL Enterprise Audit, MySQL Enterprise Firewall and MySQL Router. |
Percona | Percona Server for MySQL, XtraDB, TokuDB, RocksDB, Percona Toolkit, Percona XtraBackup and PMM(Percona Monitoring & Management). |
MariaDB | MariaDB Server, RocksDB, MariaDB Galera Cluster, MariaDB Backup, MariaDB MaxScale and MariaDB ColumnStore. |
PostgreSQL | PostgreSQL Performance Benchmarking, Capacity Planning / Sizing, PostgreSQL Performance Optimization, PostgreSQL High Availability / Database Reliability Engineering, PostgreSQL Upgrades / Migration and PostgreSQL Security |
Cloud DBA Services | IaaS and DBaaS including: Oracle Cloud, Google CloudSQL, Amazon Aurora, AWS RDS®, EC2®, Microsoft Azure® and Rackspace® Cloud |
Performance Monitoring and Trending Platforms | MySQL Enterprise Monitor, Icinga, Zabbix, Prometheus and Grafana. |
High Availability, Scale-Out, Replication and Load Balancer | MySQL Group Replication, MySQL Cluster CGE, InnoDB Cluster, Galera Cluster, Percona XtraDB Cluster, MariaDB MaxScale, Continuent Tungsten Replicator, MHA (Master High Availability Manager and tools for MySQL), HAProxy, ProxySQL, MySQL Router and Vitess. |
Columnar Database Systems | ClickHouse, MariaDB ColumnStore |
DevOps. and Automation | Vagrant, Docker, Kubernetes, Jenkins, Ganglia, Chef, Puppet, Ansible, Consul, JIRA, Graylog and Grafana. |
” Shiv is a expert in MySQL performance, He can fine tune MySQL performance at instance, application and infrastructure level in a shortest duration, I would love to hire him again “
David Hutton
Head of IT operations
National Geographic
” If it’s about MySQL performance and scalability, I will first call Shiv and He has helped us several times in building optimal, scalable and highly available MySQL infrastructure operations “
Mark Gray
IT Manager
Nike Technologies
” If you are building an highly reliable MySQL ecosystem, Hiring Shiv and his team will simplify your project. He is a guru to build an highly available and fault tolerant web property “
Kevin Thomson
Business Head – Media properties
AOL
” Shiv and his team built an custom MySQL high availability solution for us across data centers which enabled 24*7*365 availability of our business services. His Soultions are non vendor biased and cost efficient “
Keshav Patel
Group Manager – IT
Lastminute.com
” Thinking about outsourcing your DBA function ? Shiv is the guy who you should be talking to, He can build an highly reliable 24*7 remote DBA team for an fraction of cost to hiring a Sr. level resident DBA “
Brian Lewis
Lead Systems Engineer
Priceline.com
” The shortest notice emergency DBA support provider I have ever worked with, They are highly responsive and professional. If we suspect something going wrong with our database systems, The immediate action item is contact Shiv and his team”
Sherly Williams
Manager -Business Continuity
GAP Inc.
” We have contracted MinervaDB for 24*7 remote DBA services, They delivered MySQL maximum reliability and availability solutions ahead of the project schedule using 100% open source and free tools, saving considerably our IT budget. “
Simon Matthew
IT Manager – Media & Entertainment
Vodafone PLC
“Our database infrastructure operations was highly unreliable till we engaged Shiv Iyer and his globally distributed team of consultants, Now we have 24*7 access to expert DBA(s) for a fraction of cost to hiring a resident full-time DBA”
Kerry Jones
Head – IT Operations
The BestBuy Inc
The post MinervaDB Technology Partnership Program appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post MySQL Performance Audit, Health Check, Diagnostics and Forensics By MinervaDB appeared first on The WebScale Database Infrastructure Operations Experts.
]]>The post MySQL Performance Audit, Health Check, Diagnostics and Forensics By MinervaDB appeared first on The WebScale Database Infrastructure Operations Experts.
]]>