The purpose of this tool is to test the capability of a Linux mounted filesystem being used (or being proposed) to host an MQ recovery log.
Warning!!
- If any queue manager is currently using the filesystem being tested:
- Do NOT specify existing log files to write to.
- Create a new directory on the filesystem to be tested (this must be in the same file system as the current MQ logs)
- Running this tool WILL adversely affect the performance of the existing queue manager(s), for the duration of the test, (which may, in turn, cause applications problems whose symptoms last beyond the filesystem test.
The tool is similar in purpose to fio, but has been written to test writing to a filesystem with the same options, and in the same way, that MQ writes to its recovery logs, without having to specify lots of options on the command line to get it right.
There are additionally, some problems with the use of fio, which this tool addresses (like being unable to get the open and write options exactly the same as MQ, and the pre-loading and usage pattern of the log files).
What this tool cannot do, is tell you exactly what performance you will get from your persistent MQ workload, even when log writes are the limiting factor. The workload on a machine can impact the performance of the IOPs, especially on modern multi-core machines. The best solution is always to test MQ itself.
Usage: mqldt <options> Options: --dir : Directory in which create and test files being written to. --filePrefix : prefix of test files --bsize : Blocksize(s), of writes to be used. Can be a single value, or a comma separated list. Supports k suffix. --fileSize : File size of each test file --numFiles : Number of files to create and use --duration : Number of seconds to test --csvFile : Optional csv output file --qm : Number of Queue Managers --delay : Delay to add to each IO write (microsec)
e.g. mqldt --dir=/var/san1/testdir --filePrefix=mqtestfile --bsize=5K,9K,17K,24K,49K,48K,54K,62K,77K,95K,105K --fileSize=67108864 --numFiles=16 --duration=40 --csvFile=./san_log.csv
Use the values in qm.ini (or those planned) to calculate some of the parameters to MQLDT.
| MQLDT Parm | qm.ini parm(s) |
| --dir | Any directory hosted by the same file-system as Log:LogPath |
| --fileSize | Log:LogFilePages x 4096 |
| --numFiles | Log:LogPrimaryFiles |
Options (specified or defaulted to) ===================================================================================== Write blocksize (--bsize) : 128K Directory to write to (--dir) : /var/san1/testdir Test file prefix (--filePrefix) : mqtestfile Number of files to write to (--numFiles) : 24 Size of test files (--fileSize) : 67108864 Test duration (--duration) : 20 Creating files... Executing test for write blocksize 131072 (128k). Seconds elapsed -> 20/20 Total writes to files : 46,178 Total bytes written to files : 6,052,642,816 Max bytes/sec written to files (over 1 sec interval) : 311,689,216 Min bytes/sec written to files (over 1 sec interval) : 297,795,584 Avg bytes/sec written to files : 302,732,356 Max latency of write (ns) : 4,977,438 (#24,649) Min bytes/sec (slowest write) : 26,333,226 Min latency of write (ns) : 375,889 (#42,421) Max bytes/sec (fastest write) : 348,698,688 Avg latency of write (ns) : 4,935,648
| Total writes to files | The total number of writev operations completed over the duration of the test |
| IOPS | The total number of writev operations completed per second over the duration of the test |
| Total bytes written to files | Total bytes written over the duration of the test (i.e Total writes to files x blocksize) |
| Max bytes/sec written to files (over 1 sec interval) | MQLDT calculates how much data has been written every second, this is the maximum write bandwidth measured for any one second duration. If the test is only run for one second this number will be the same as 'Avg bytes/sec written to files' |
| Min bytes/sec written to files (over 1 sec interval) | This is the minimum write bandwidth measured for any one second duration '(see Max bytes/sec written to files (over 1 sec interval)', above). If the test is only run for one second this number will be the same as 'Avg bytes/sec written to files' |
| Avg bytes/sec written to files | Average write bandwidth over duration of test. |
| Max latency of write (ns) | The longest time (in nanoseconds) to complete a write during the test. The number of the write is indicated in brackets. |
| Min bytes/sec (slowest write) | The theoretical minimum bandwidth, if every write had the maximum latency indicated above |
| Min latency of write (ns) | The shortest time (in nanoseconds) to complete a write during the test. The number of the write is indicated in brackets. |
| Max bytes/sec (fastest write) | The theoretical maximum bandwidth, if every write had the minimum latency indicated above |
| Avg latency of write (ns) | The average time (in nanoseconds) to complete a write during the test. |
This tool has now been extended to simulate multiple Queue Managers writing to the same IO device.
Use the --qm=2 option to run with 2 queue managers, each owning their own set of log files as specified by the configuration provided, noting that for qm 2 to 10, a numeric value is appended to the directory location.
i.e If --qm=3 --dir=/tmp/mqldt is provided, directories labeled /tmp/mqldt1, /tmp/mqldt2 and /tmp/mqldt3 need to be created before the test is run
Support for up to 10 QM is provided, although only a single block size can be executed per test.
There is a repo that builds a dockerized version of mqldt called mqldt-c.
The built image from that repo is available here and can be pulled with the following command:
docker pull quay.io/stmassey/mqldt