Sysbench is the most popular and high customizable benchmarking platform for MySQL (you can as well benchmark performance of Linux too), Sysbench pretty much helps performance benchmarking of all the following areas :
CPU – How long it takes for CPU to compute maximum primary number ? You may as well add how many threads are participating in this benchmarking process . This is very direct and easy way to benchmark the CPU performance.
File I/O – Benchmarking disk I/O performance considering all possible scenarios like direct io, sync, async etc. testing sequential reads / writes and random reads / writes , This makes an very scalable disk I/O benchmarking possible.
OLTP – Benchmarking MySQL under all possible workloads, like SELECT only queries (random / sequential SELECT) , INSERT / UPDATE / DELETE operations on multiple use case scenarios like bulk data loading, range-based updates, high volume data deletes (both random and sequential) etc. There is no MySQL benchmarking complete without Sysbench
Installing Sysbench
There are several ways to install Sysbench, It’s very well documented here – https://github.com/akopytov/sysbench#linux so we are not repeating it here again but we personally prefer installing Sysbench from source for many reasons so I have shared my method of installing Sysbench on CentOS
Git clone Sysbench :
[root@localhost Sysbench]# git clone https://github.com/akopytov/sysbench.git Cloning into 'sysbench'... remote: Counting objects: 8891, done. remote: Total 8891 (delta 0), reused 0 (delta 0), pack-reused 8891 Receiving objects: 100% (8891/8891), 3.88 MiB | 1.47 MiB/s, done. Resolving deltas: 100% (6320/6320), done.
[root@localhost sysbench]# ls COPYING Makefile.am README-WIN.txt autogen.sh configure.ac install-sh missing rpm snap tests ChangeLog README-Oracle.md README.md config debian m4 mkinstalldirs scripts src
Table of Contents
Build Requirements (we use CentOS 7.3)
[root@localhost sysbench]# yum -y install make automake libtool pkgconfig libaio-devel Verifying : glibc-2.17-157.el7.x86_64 21/23 Verifying : libgomp-4.8.5-11.el7.x86_64 22/23 Verifying : glibc-common-2.17-157.el7.x86_64 23/23 Installed: automake.noarch 0:1.13.4-3.el7 libaio-devel.x86_64 0:0.3.109-13.el7 libtool.x86_64 0:2.4.2-22.el7_3 Dependency Installed: autoconf.noarch 0:2.69-11.el7 cpp.x86_64 0:4.8.5-16.el7_4.2 gcc.x86_64 0:4.8.5-16.el7_4.2 glibc-devel.x86_64 0:2.17-196.el7_4.2 glibc-headers.x86_64 0:2.17-196.el7_4.2 kernel-headers.x86_64 0:3.10.0-693.21.1.el7 libmpc.x86_64 0:1.0.1-3.el7 m4.x86_64 0:1.4.16-10.el7 mpfr.x86_64 0:3.1.1-4.el7 perl-Data-Dumper.x86_64 0:2.145-3.el7 perl-Test-Harness.noarch 0:3.28-3.el7 perl-Thread-Queue.noarch 0:3.02-2.el7 Dependency Updated: glibc.x86_64 0:2.17-196.el7_4.2 glibc-common.x86_64 0:2.17-196.el7_4.2 libgcc.x86_64 0:4.8.5-16.el7_4.2 libgomp.x86_64 0:4.8.5-16.el7_4.2 Complete!
[root@localhost sysbench]# yum -y install mysql-devel Verifying : e2fsprogs-libs-1.42.9-9.el7.x86_64 34/35 Verifying : krb5-libs-1.14.1-26.el7.x86_64 35/35 Installed: mariadb-devel.x86_64 1:5.5.56-2.el7 Dependency Installed: keyutils-libs-devel.x86_64 0:1.5.8-3.el7 krb5-devel.x86_64 0:1.15.1-8.el7 libcom_err-devel.x86_64 0:1.42.9-10.el7 libkadm5.x86_64 0:1.15.1-8.el7 libselinux-devel.x86_64 0:2.5-11.el7 libsepol-devel.x86_64 0:2.5-6.el7 libverto-devel.x86_64 0:0.2.5-4.el7 openssl-devel.x86_64 1:1.0.2k-8.el7 pcre-devel.x86_64 0:8.32-17.el7 zlib-devel.x86_64 0:1.2.7-17.el7 Dependency Updated: e2fsprogs.x86_64 0:1.42.9-10.el7 e2fsprogs-libs.x86_64 0:1.42.9-10.el7 krb5-libs.x86_64 0:1.15.1-8.el7 libcom_err.x86_64 0:1.42.9-10.el7 libselinux.x86_64 0:2.5-11.el7 libselinux-python.x86_64 0:2.5-11.el7 libselinux-utils.x86_64 0:2.5-11.el7 libss.x86_64 0:1.42.9-10.el7 mariadb-libs.x86_64 1:5.5.56-2.el7 openssl.x86_64 1:1.0.2k-8.el7 openssl-libs.x86_64 1:1.0.2k-8.el7 pcre.x86_64 0:8.32-17.el7 Complete!
Install Sysbench from source
Step 1 – Run “autogen.sh”
[root@localhost sysbench]# ./autogen.sh perl: warning: Setting locale failed. perl: warning: Please check that your locale settings: LANGUAGE = (unset), LC_ALL = (unset), LC_CTYPE = "UTF-8", LANG = "en_US.UTF-8" configure.ac:59: installing 'config/ar-lib' configure.ac:45: installing 'config/compile' configure.ac:27: installing 'config/config.guess' configure.ac:27: installing 'config/config.sub' configure.ac:32: installing 'config/install-sh' configure.ac:32: installing 'config/missing' src/Makefile.am: installing 'config/depcomp' parallel-tests: installing 'config/test-driver' autoreconf: Leaving directory `.'
Step 2 – Run “configure.sh”
[root@localhost sysbench]# ls COPYING Makefile.in README.md autom4te.cache configure.ac m4 rpm src ChangeLog README-Oracle.md aclocal.m4 config debian missing scripts tests Makefile.am README-WIN.txt autogen.sh configure install-sh mkinstalldirs snap third_party [root@localhost sysbench]# ./configure checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking target system type... x86_64-unknown-linux-gnu =============================================================================== sysbench version : 1.1.0-651e7fd CC : gcc -std=gnu99 CFLAGS : -O3 -funroll-loops -ggdb3 -march=core2 -Wall -Wextra -Wpointer-arith -Wbad-function-cast -Wstrict-prototypes -Wnested-externs -Wno-format-zero-length -Wundef -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -Wcast-align -pthread CPPFLAGS : -D_GNU_SOURCE -I$(top_srcdir)/src -I$(abs_top_builddir)/third_party/luajit/inc -I$(abs_top_builddir)/third_party/concurrency_kit/include LDFLAGS : -L/usr/local/lib LIBS : -laio -lm prefix : /usr/local bindir : ${prefix}/bin libexecdir : ${prefix}/libexec mandir : ${prefix}/share/man datadir : ${prefix}/share MySQL support : yes Drizzle support : no AttachSQL support : no Oracle support : no PostgreSQL support : no LuaJIT : bundled LUAJIT_CFLAGS : -I$(abs_top_builddir)/third_party/luajit/inc LUAJIT_LIBS : $(abs_top_builddir)/third_party/luajit/lib/libluajit-5.1.a -ldl LUAJIT_LDFLAGS : -rdynamic Concurrency Kit : bundled CK_CFLAGS : -I$(abs_top_builddir)/third_party/concurrency_kit/include CK_LIBS : $(abs_top_builddir)/third_party/concurrency_kit/lib/libck.a configure flags : ===============================================================================
Step 3 – Make sysbench for MySQL benchmarking with Lua scripts
[root@localhost sysbench]# [root@localhost sysbench]# make -j Making all in third_party/luajit make[1]: Entering directory `/home/shiv/Sysbench/sysbench/third_party/luajit' make -C ./luajit clean make[2]: Entering directory `/home/shiv/Sysbench/sysbench/third_party/luajit/luajit' make -C src clean make[3]: Entering directory `/home/shiv/Sysbench/sysbench/third_party/luajit/luajit/src' rm -f luajit libluajit.a libluajit.so host/minilua host/buildvm lj_vm.S lj_bcdef.h lj_ffdef.h lj_libdef.h lj_recdef.h lj_folddef.h host/buildvm_arch.h jit/vmdef.lua *.o host/*.o *.obj *.lib *.exp *.dll *.exe *.manifest *.pdb *.ilk make[3]: Leaving directory `/home/shiv/Sysbench/sysbench/third_party/luajit/luajit/src' make[2]: Leaving directory `/home/shiv/Sysbench/sysbench/third_party/luajit/luajit' libtool: link: gcc -std=gnu99 -Wall -Wextra -Wpointer-arith -Wbad-function-cast -Wstrict-prototypes -Wnested-externs -Wno-format-zero-length -Wundef -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -Wcast-align -pthread -O3 -funroll-loops -ggdb3 -march=core2 -rdynamic -o sysbench sysbench.o sb_timer.o sb_options.o sb_logger.o db_driver.o sb_histogram.o sb_rand.o sb_thread.o sb_barrier.o sb_lua.o sb_util.o sb_counter.o -L/usr/local/lib tests/fileio/libsbfileio.a tests/threads/libsbthreads.a tests/memory/libsbmemory.a tests/cpu/libsbcpu.a tests/mutex/libsbmutex.a drivers/mysql/libsbmysql.a -L/usr/lib64/mysql -lmysqlclient -lpthread -lz -lssl -lcrypto /home/shiv/Sysbench/sysbench/third_party/luajit/lib/libluajit-5.1.a -ldl /home/shiv/Sysbench/sysbench/third_party/concurrency_kit/lib/libck.a -laio -lm -pthread make[2]: Leaving directory `/home/shiv/Sysbench/sysbench/src' make[1]: Leaving directory `/home/shiv/Sysbench/sysbench/src' Making all in tests make[1]: Entering directory `/home/shiv/Sysbench/sysbench/tests' make[1]: Nothing to be done for `all'. make[1]: Leaving directory `/home/shiv/Sysbench/sysbench/tests' make[1]: Entering directory `/home/shiv/Sysbench/sysbench' make[1]: Nothing to be done for `all-am'. make[1]: Leaving directory `/home/shiv/Sysbench/sysbench' [root@localhost sysbench]#
[root@localhost sysbench]# make install Making install in third_party/luajit make[1]: Entering directory `/home/shiv/Sysbench/sysbench/third_party/luajit' make[2]: Entering directory `/home/shiv/Sysbench/sysbench/third_party/luajit' make[2]: Nothing to be done for `install-exec-am'. make[2]: Nothing to be done for `install-data-am'. make[2]: Leaving directory `/home/shiv/Sysbench/sysbench/third_party/luajit' make[1]: Leaving directory `/home/shiv/Sysbench/sysbench/third_party/luajit' Making install in third_party/concurrency_kit make[1]: Entering directory `/home/shiv/Sysbench/sysbench/third_party/concurrency_kit' make[2]: Entering directory `/home/shiv/Sysbench/sysbench/third_party/concurrency_kit' make[2]: Nothing to be done for `install-exec-am'. make[2]: Nothing to be done for `install-data-am'. make[2]: Leaving directory `/home/shiv/Sysbench/sysbench/third_party/concurrency_kit' make[1]: Leaving directory `/home/shiv/Sysbench/sysbench/third_party/concurrency_kit' Making install in src make[1]: Entering directory `/home/shiv/Sysbench/sysbench/src' /usr/bin/mkdir -p '/usr/local/share/sysbench/tests' /usr/bin/install -c test_run.sh '/usr/local/share/sysbench/tests' make[2]: Leaving directory `/home/shiv/Sysbench/sysbench/tests' make[1]: Leaving directory `/home/shiv/Sysbench/sysbench/tests' make[1]: Entering directory `/home/shiv/Sysbench/sysbench' make[2]: Entering directory `/home/shiv/Sysbench/sysbench' make[2]: Nothing to be done for `install-exec-am'. make[2]: Nothing to be done for `install-data-am'. make[2]: Leaving directory `/home/shiv/Sysbench/sysbench' make[1]: Leaving directory `/home/shiv/Sysbench/sysbench' [root@localhost sysbench]#
Step 4 – Confirm successful installation of Sysbench 1.1
[root@localhost sysbench]# sysbench --version sysbench 1.1.0-651e7fd [root@localhost sysbench]#
How to use Sysbench for benchmarking ?
Benchmarking CPU
Benchmarking CPU with just 1 thread , we look for time elapsed value majorly (In this case it is 10.0457s)
[root@localhost sysbench]# sysbench --test=cpu --cpu-max-prime=300000 run sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3) Running the test with following options: Number of threads: 1 Initializing random number generator from current time Prime numbers limit: 300000 Initializing worker threads... Threads started! CPU speed: events per second: 9.85 Throughput: events/s (eps): 9.8550 time elapsed: 10.0457s total number of events: 99 Latency (ms): min: 91.73 avg: 101.46 max: 179.27 95th percentile: 110.66 sum: 10044.86 Threads fairness: events (avg/stddev): 99.0000/0.00 execution time (avg/stddev): 10.0449/0.00 [root@localhost sysbench]#
Benchmarking CPU with 60 threads (if you notice –test option is deprecated with Sysbench 1.1)
[root@localhost sysbench]# sysbench --test=cpu --cpu-max-prime=300000 --num-threads=60 run sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3) Running the test with following options: Number of threads: 60 Initializing random number generator from current time Prime numbers limit: 300000 Initializing worker threads... Threads started! CPU speed: events per second: 9.90 Throughput: events/s (eps): 9.9047 time elapsed: 12.1155s total number of events: 120 Latency (ms): min: 4802.24 avg: 5781.75 max: 6633.59 95th percentile: 6360.91 sum: 693809.67 Threads fairness: events (avg/stddev): 2.0000/0.00 execution time (avg/stddev): 11.5635/0.28 [root@localhost sysbench]#
Benchmarking RAM / Memory (both reads and writes)
[root@localhost sysbench]# sysbench --test=memory --memory-block-size=64K --memory-scope=global --memory-total-size=300G --memory-oper=read run sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3) Running the test with following options: Number of threads: 1 Initializing random number generator from current time Running memory speed test with the following options: block size: 64KiB total size: 307200MiB operation: read scope: global Initializing worker threads... Threads started! Total operations: 3210450 (321043.11 per second) 200653.12 MiB transferred (20065.19 MiB/sec) Throughput: events/s (eps): 321043.1076 time elapsed: 10.0001s total number of events: 3210450 Latency (ms): min: 0.00 avg: 0.00 max: 22.43 95th percentile: 0.00 sum: 9251.30 Threads fairness: events (avg/stddev): 3210450.0000/0.00 execution time (avg/stddev): 9.2513/0.00
[root@localhost sysbench]# sysbench --test=memory --memory-block-size=64K --memory-scope=global --memory-total-size=300G --memory-oper=write run sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3) Running the test with following options: Number of threads: 1 Initializing random number generator from current time Running memory speed test with the following options: block size: 64KiB total size: 307200MiB operation: write scope: global Initializing worker threads... Threads started! Total operations: 2378246 (237823.34 per second) 148640.38 MiB transferred (14863.96 MiB/sec) Throughput: events/s (eps): 237823.3394 time elapsed: 10.0001s total number of events: 2378246 Latency (ms): min: 0.00 avg: 0.00 max: 7.84 95th percentile: 0.00 sum: 9485.47 Threads fairness: events (avg/stddev): 2378246.0000/0.00 execution time (avg/stddev): 9.4855/0.00
Benchmarking Disk I/O
The following I/O operations can be benchmarked using Sysbench :
- Sequential write – seqwr
- Sequential rewrite – seqrewr
- Sequential read – seqrd
- Random read – rndrd
- Random write – rndwr
- Combined random read/write – rndrw
Examples
Step 1 – Create a file , define total number of threads and file test mode. In the example below, we have selected combined random read/write (rndrw)
[root@localhost sysbench]# sysbench --num-threads=16 --test=fileio --file-total-size=2G --file-test-mode=rndrw prepare sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3) 128 files, 16384Kb each, 2048Mb total Creating files for the test... Extra file open flags: (none) Creating file test_file.0 Creating file test_file.1 Creating file test_file.2 Creating file test_file.3 Creating file test_file.4 Creating file test_file.5 Creating file test_file.6 Creating file test_file.7 Creating file test_file.8 Creating file test_file.9 Creating file test_file.10 Creating file test_file.11 Creating file test_file.12 Creating file test_file.13 Creating file test_file.14 Creating file test_file.15 ......... ............ Creating file test_file.125 Creating file test_file.126 Creating file test_file.127 2147483648 bytes written in 3.21 seconds (637.65 MiB/sec).
Step 2 – Run benchmarking test on file created above
[root@localhost sysbench]# sysbench --num-threads=16 --test=fileio --file-total-size=2G --file-test-mode=rndrw run sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3) 128 files, 16MiB each 2GiB total file size Block size 16KiB Number of IO requests: 0 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Initializing worker threads... Threads started! Throughput: read: IOPS=2111.56 32.99 MiB/s (34.60 MB/s) write: IOPS=1407.37 21.99 MiB/s (23.06 MB/s) fsync: IOPS=4500.34 Latency (ms): min: 0.00 avg: 1.99 max: 621.12 95th percentile: 4.10 sum: 159899.55
Step 3 – Remove the files once completed benchmarking
[root@localhost sysbench]# sysbench --num-threads=16 --test=fileio --file-total-size=2G --file-test-mode=rndrw cleanup sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3) Removing test files...
Benchmarking MySQL using custom Lua scripts available with installation of Sysbench 1.1 (/usr/local/share/sysbench)
We have custom Lua scripts available with Sysbench 1.1 for benchmarking MySQL (I have listed them below)
[root@localhost sysbench]# ls bulk_insert.lua oltp_delete.lua oltp_point_select.lua oltp_read_write.lua oltp_update_non_index.lua select_random_points.lua tests oltp_common.lua oltp_insert.lua oltp_read_only.lua oltp_update_index.lua oltp_write_only.lua select_random_ranges.lua
Step 1 – create a MySQL database for running the benchmarking tests
mysql> create database sysbench; Query OK, 1 row affected (0.00 sec)
We have chosen “select_random_points.lua” script for benchmarking MySQL 5.7(default setting) in the example below :
Step 2 – Prepare sbtest1 table in sysbench database create above with 2M records
[root@localhost sysbench]# sysbench select_random_points.lua --table-size=2000000 --num-threads=1 --rand-type=uniform --db-driver=mysql --mysql-db=sysbench --mysql-user=root --mysql-password=SriLanka/2018 prepare sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3) Creating table 'sbtest1'... Inserting 2000000 records into 'sbtest1' Creating a secondary index on 'sbtest1'... [root@localhost sysbench]#
mysql> show table status like 'sbtest%' \G; *************************** 1. row *************************** Name: sbtest1 Engine: InnoDB Version: 10 Row_format: Dynamic Rows: 1921969 Avg_row_length: 233 Data_length: 449724416 Max_data_length: 0 Index_length: 0 Data_free: 5242880 Auto_increment: 2000001 Create_time: 2018-03-12 23:32:13 Update_time: 2018-03-12 23:32:07 Check_time: NULL Collation: latin1_swedish_ci Checksum: NULL Create_options: Comment: 1 row in set (0.00 sec) ERROR: No query specified
Step 3 – Run the benchmark test using select_random_ponts.lua script , We have selected 100 threads for this test :
[root@localhost sysbench]# sysbench select_random_points.lua --table-size=2000000 --num-threads=100 --rand-type=uniform --db-driver=mysql --mysql-db=sysbench --mysql-user=root --mysql-password=SriLanka/2018 run sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3) Running the test with following options: Number of threads: 100 Initializing random number generator from current time Initializing worker threads... Threads started! SQL statistics: queries performed: read: 28214 write: 0 other: 0 total: 28214 transactions: 28214 (2799.30 per sec.) queries: 28214 (2799.30 per sec.) ignored errors: 0 (0.00 per sec.) reconnects: 0 (0.00 per sec.) Throughput: events/s (eps): 2799.2957 time elapsed: 10.0790s total number of events: 28214 Latency (ms): min: 0.15 avg: 35.50 max: 1727.81 95th percentile: 77.19 sum: 1001542.39 Threads fairness: events (avg/stddev): 282.1400/13.24 execution time (avg/stddev): 10.0154/0.02
In the above report, time elapsed (at MinervaDB we measure performance by response time ) is most important matrix for me, Though we consider transactions / queries per second equally to measure the load in the system
Step 4 – Please don’t not forget to cleanup MySQL database created (sysbench in this example)
[root@localhost sysbench]# sysbench select_random_points.lua --table-size=2000000 --num-threads=100 --rand-type=uniform --db-driver=mysql --mysql-db=sysbench --mysql-user=root --mysql-password=SriLanka/2018 cleanup sysbench 1.1.0-651e7fd (using bundled LuaJIT 2.1.0-beta3) Dropping table 'sbtest1'... [root@localhost sysbench]#
Conclusion
The only intention of this blog is to help you learn how to install Sysbench 1.1 and benchmark MySQL, We have not tweaked MySQL for performance and the data collected with this exercise is not for publishing MySQL 5.7 performance benchmarking results.