Table of Contents
Install Git and JDK21 before building Tulip from source.
Install SDKMAN from bash (https://sdkman.io)
$ curl -s "https://get.sdkman.io" | bash
$ source "$HOME/.sdkman/bin/sdkman-init.sh"
$ sdk version
Install w3m (Debian, Ubuntu) to display reports
$ sudo apt install w3m
Install SDKMAN from zsh or bash (https://sdkman.io)
$ curl -s "https://get.sdkman.io" | bash
$ source "$HOME/.sdkman/bin/sdkman-init.sh"
$ sdk version
Install w3m (MacOS) to display reports
$ brew install w3m
Open a Windows PowerShell (PS) terminal (version 5.1 or later) and from the PS C:\> prompt, run:
Step 1 - run Set-ExecutionPolicy
PS C:\> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
Step 2 - answer 'Y' when prompted to change the execution policy.
Execution Policy Change The execution policy helps protect you from scripts that you do not trust. Changing the execution policy might expose you to the security risks described in the about_Execution_Policies help topic at https:/go.microsoft.com/fwlink/?LinkID=135170. Do you want to change the execution policy? [Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "N"): Y PS C:\>
Step 3 - run Invoke-RestMethod
PS C:\> Invoke-RestMethod -Uri https://get.scoop.sh | Invoke-Expression
Step 4 - wait for the installation to finish
Initializing... Downloading... Extracting... Creating shim... Adding ~\scoop\shims to your path. Scoop was installed successfully! Type 'scoop help' for instructions. PS C:\>
Install Git and Git-Bash
scoop install gitInstall MS Java 21 (OpenJDK)
scoop bucket add java
scoop install microsoft21-jdk
Install JBang
scoop install jbang
(Optionally) Install WinGetUI (aka UniGetUI) to apply future updates to the installed applications
scoop bucket add extras
scoop install extras/wingetui
git clone https://github.com/wfouche/Tulip.git cd Tulip sdk env install ./gradlew run ./text_report.sh
git clone https://github.com/wfouche/Tulip.git cd Tulip sdk env install ./gradlew run ./text_report.sh
Tulip 2.0.1 / Micro-benchmarks / 2024-09-28 21:47:16
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Init
u:16 0 0:00:00 32 0 484.848 0.0 ms 0.294 ms 1.4 ms 0.0 ms 8.1 ms 8.1 ms 28 21:4 1 0.9 49.4 ms 1.7 ms 0.0 12.2
t:2 [0.start] 0:00:00 16 0 242.424 0.0 ms 0.538 ms 2.0 ms 0.0 ms 8.1 ms 8.1 ms 28 21:4 1 0.9 49.4 ms 1.7 ms 0.0 12.2
[8.login] 0:00:00 16 0 242.424 0.0 ms 0.050 ms 0.1 ms 0.0 ms 0.3 ms 0.3 ms 28 21:4 1 0.9 49.4 ms 1.7 ms 0.0 12.2
0:00:00 32 0 484.848 0.0 ms 0.294 ms 1.4 ms 0.0 ms 8.1 ms 8.1 ms 28 21:4 1 0.9 49.4 ms 1.7 ms 0.0 12.2
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Max TPS-a
u:16 0 0:00:30 38578634 0 1285954.467 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.3 ms 28 21:4 11 3.5 0.9 ms 0.0 ms 67.3 31.9
t:2 1 0:00:30 38263506 0 1275450.200 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.1 ms 28 21:4 11 3.5 1.4 ms 0.0 ms 64.9 31.9
2 0:00:30 39110981 0 1303699.367 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.1 ms 28 21:4 11 3.7 1.6 ms 0.0 ms 65.8 31.9
[9.noop] 0:01:30 115953121 0 1288368.011 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.3 ms 28 21:4 11 3.7 1.6 ms 0.0 ms 67.3 31.9
0:01:30 115953121 0 1288368.011 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.3 ms 28 21:4 11 3.7 1.6 ms 0.0 ms 67.3 31.9
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Max TPS-b
u:16 0 0:00:30 29999999 0 999999.967 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.1 ms 28 21:5 11 2.7 0.9 ms 0.0 ms 65.3 31.9
t:2 1 0:00:30 29999995 0 999999.833 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.5 ms 28 21:5 11 2.7 0.8 ms 0.0 ms 64.3 31.9
2 0:00:30 29999997 0 999999.900 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.1 ms 28 21:5 11 2.7 2.5 ms 0.0 ms 64.0 31.9
[9.noop] 0:01:30 89999991 0 999999.900 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.5 ms 28 21:5 11 2.7 2.5 ms 0.0 ms 65.3 31.9
0:01:30 89999991 0 999999.900 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.5 ms 28 21:5 11 2.7 2.5 ms 0.0 ms 65.3 31.9
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Fixed TPS-a
u:16 0 0:00:30 3001 0 100.033 0.0 ms 12.284 ms 8.3 ms 25.1 ms 28.2 ms 28.3 ms 28 21:5 3 1.2 63.4 ms 6.8 ms 60.4 16.6
t:2 1 0:00:30 3001 0 100.033 0.0 ms 12.374 ms 8.4 ms 25.2 ms 28.2 ms 28.2 ms 28 21:5 3 1.2 60.2 ms 6.6 ms 3.4 16.6
2 0:00:30 3001 0 100.033 0.0 ms 12.275 ms 8.4 ms 25.2 ms 28.2 ms 28.3 ms 28 21:5 3 1.2 61.6 ms 6.7 ms 4.4 16.6
3 0:00:30 3001 0 100.033 0.0 ms 12.348 ms 8.3 ms 25.1 ms 28.2 ms 28.3 ms 28 21:5 3 1.2 66.1 ms 6.1 ms 4.9 16.6
[1.DELAY-6ms] 0:02:00 2999 0 24.992 0.0 ms 6.102 ms 3.8 ms 11.2 ms 12.2 ms 12.3 ms 28 21:5 3 1.2 66.1 ms 6.8 ms 60.4 16.6
[2.DELAY-14ms] 0:02:00 9005 0 75.042 0.0 ms 14.391 ms 8.4 ms 26.2 ms 28.2 ms 28.3 ms 28 21:5 3 1.2 66.1 ms 6.8 ms 60.4 16.6
0:02:00 12004 0 100.033 0.0 ms 12.320 ms 8.3 ms 25.2 ms 28.2 ms 28.3 ms 28 21:5 3 1.2 66.1 ms 6.8 ms 60.4 16.6
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Fixed TPS-b
u:16 0 0:00:30 3001 0 100.033 10.1 ms 10.167 ms 0.0 ms 10.2 ms 10.3 ms 10.4 ms 28 21:5 1 0.9 3.3 ms 0.3 ms 6.0 13.8
t:2 1 0:00:30 3001 0 100.033 10.1 ms 10.167 ms 0.0 ms 10.2 ms 10.2 ms 10.4 ms 28 21:5 1 1.0 3.0 ms 0.3 ms 3.6 13.8
2 0:00:30 3001 0 100.033 10.1 ms 10.165 ms 0.0 ms 10.2 ms 10.2 ms 10.3 ms 28 21:5 1 1.0 2.3 ms 0.3 ms 3.5 13.8
3 0:00:30 3001 0 100.033 10.1 ms 10.167 ms 0.0 ms 10.2 ms 10.2 ms 10.5 ms 28 21:5 1 1.0 2.3 ms 0.3 ms 3.5 12.0
[10.DELAY-10ms] 0:02:00 12004 0 100.033 10.1 ms 10.167 ms 0.0 ms 10.2 ms 10.2 ms 10.5 ms 28 21:5 1 1.0 3.3 ms 0.3 ms 6.0 13.8
0:02:00 12004 0 100.033 10.1 ms 10.167 ms 0.0 ms 10.2 ms 10.2 ms 10.5 ms 28 21:5 1 1.0 3.3 ms 0.3 ms 6.0 13.8
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 HTTP-a
u:16 0 0:00:30 310864 0 10362.133 0.1 ms 0.183 ms 0.0 ms 0.2 ms 0.3 ms 4.4 ms 28 21:5 11 8.4 8.8 ms 1.6 ms 94.8 93.7
t:2 1 0:00:30 310073 0 10335.767 0.1 ms 0.183 ms 0.0 ms 0.2 ms 0.3 ms 2.2 ms 28 21:5 11 8.4 7.0 ms 1.6 ms 72.5 93.7
2 0:00:30 309076 0 10302.533 0.1 ms 0.184 ms 0.0 ms 0.2 ms 0.3 ms 1.8 ms 28 21:5 11 8.4 5.7 ms 1.6 ms 72.9 93.7
[3.HTTP-posts] 0:01:30 186002 0 2066.689 0.1 ms 0.183 ms 0.0 ms 0.2 ms 0.3 ms 2.6 ms 28 21:5 11 8.4 8.8 ms 1.6 ms 94.8 93.7
[4.HTTP-comments] 0:01:30 186006 0 2066.733 0.1 ms 0.183 ms 0.0 ms 0.2 ms 0.3 ms 1.9 ms 28 21:5 11 8.4 8.8 ms 1.6 ms 94.8 93.7
[5.HTTP-albums] 0:01:30 186002 0 2066.689 0.1 ms 0.184 ms 0.0 ms 0.2 ms 0.3 ms 2.3 ms 28 21:5 11 8.4 8.8 ms 1.6 ms 94.8 93.7
[6.HTTP-photos] 0:01:30 186002 0 2066.689 0.1 ms 0.184 ms 0.0 ms 0.2 ms 0.3 ms 3.8 ms 28 21:5 11 8.4 8.8 ms 1.6 ms 94.8 93.7
[7.HTTP-todos] 0:01:30 186001 0 2066.678 0.1 ms 0.184 ms 0.0 ms 0.2 ms 0.3 ms 4.4 ms 28 21:5 11 8.4 8.8 ms 1.6 ms 94.8 93.7
0:01:30 930013 0 10333.478 0.1 ms 0.183 ms 0.0 ms 0.2 ms 0.3 ms 4.4 ms 28 21:5 11 8.4 8.8 ms 1.6 ms 94.8 93.7
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 HTTP-b
u:16 0 0:00:30 37500 0 1250.000 0.2 ms 0.614 ms 0.1 ms 0.8 ms 0.9 ms 2.3 ms 28 21:5 3 1.1 1.9 ms 0.1 ms 69.4 93.7
t:2 1 0:00:30 37501 0 1250.033 0.2 ms 0.626 ms 0.1 ms 0.8 ms 0.9 ms 2.3 ms 28 22:0 2 1.1 2.1 ms 0.1 ms 34.7 93.7
2 0:00:30 37501 0 1250.033 0.2 ms 0.575 ms 0.2 ms 0.8 ms 0.9 ms 5.5 ms 28 22:0 4 1.1 5.3 ms 0.1 ms 35.2 93.7
[3.HTTP-posts] 0:01:30 22499 0 249.989 0.2 ms 0.602 ms 0.1 ms 0.8 ms 0.9 ms 2.4 ms 28 22:0 4 1.1 5.3 ms 0.1 ms 69.4 93.7
[4.HTTP-comments] 0:01:30 22497 0 249.967 0.2 ms 0.605 ms 0.2 ms 0.8 ms 0.9 ms 5.5 ms 28 22:0 4 1.1 5.3 ms 0.1 ms 69.4 93.7
[5.HTTP-albums] 0:01:30 22502 0 250.022 0.2 ms 0.604 ms 0.2 ms 0.8 ms 0.9 ms 2.5 ms 28 22:0 4 1.1 5.3 ms 0.1 ms 69.4 93.7
[6.HTTP-photos] 0:01:30 22503 0 250.033 0.2 ms 0.607 ms 0.2 ms 0.8 ms 0.9 ms 3.7 ms 28 22:0 4 1.1 5.3 ms 0.1 ms 69.4 93.7
[7.HTTP-todos] 0:01:30 22501 0 250.011 0.2 ms 0.606 ms 0.2 ms 0.8 ms 0.9 ms 1.9 ms 28 22:0 4 1.1 5.3 ms 0.1 ms 69.4 93.7
0:01:30 112502 0 1250.022 0.2 ms 0.605 ms 0.2 ms 0.8 ms 0.9 ms 5.5 ms 28 22:0 4 1.1 5.3 ms 0.1 ms 69.4 93.7
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Shutdown
u:16 0 0:00:01 16 0 9.950 100.2 ms 100.340 ms 0.2 ms 100.4 ms 101.3 ms 101.3 ms 28 22:0 1 1.0 0.1 ms 0.1 ms 31.7 93.7
t:2 [99] 0:00:01 16 0 9.950 100.2 ms 100.340 ms 0.2 ms 100.4 ms 101.3 ms 101.3 ms 28 22:0 1 1.0 0.1 ms 0.1 ms 31.7 93.7
0:00:01 16 0 9.950 100.2 ms 100.340 ms 0.2 ms 100.4 ms 101.3 ms 101.3 ms 28 22:0 1 1.0 0.1 ms 0.1 ms 31.7 93.7Tulip 2.0.1/ Micro-benchmarks / 2024-09-28 21:24:21
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Init
u:16 0 0:00:00 32 0 463.768 0.0 ms 0.213 ms 0.9 ms 0.1 ms 5.4 ms 5.4 ms 28 21:2 1 1.0 51.8 ms 1.8 ms 0.0 12.2
t:2 [0.start] 0:00:00 16 0 231.884 0.0 ms 0.382 ms 1.3 ms 0.1 ms 5.4 ms 5.4 ms 28 21:2 1 1.0 51.8 ms 1.8 ms 0.0 12.2
[8.login] 0:00:00 16 0 231.884 0.0 ms 0.044 ms 0.1 ms 0.0 ms 0.3 ms 0.3 ms 28 21:2 1 1.0 51.8 ms 1.8 ms 0.0 12.2
0:00:00 32 0 463.768 0.0 ms 0.213 ms 0.9 ms 0.1 ms 5.4 ms 5.4 ms 28 21:2 1 1.0 51.8 ms 1.8 ms 0.0 12.2
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Max TPS-a
u:16 0 0:00:30 39085066 0 1302835.533 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.4 ms 28 21:2 11 3.7 3.1 ms 0.0 ms 95.9 30.3
t:2 1 0:00:30 39704671 0 1323489.033 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.1 ms 28 21:2 11 3.8 1.0 ms 0.0 ms 65.3 30.3
2 0:00:30 39880431 0 1329347.700 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.1 ms 28 21:2 11 3.8 1.4 ms 0.0 ms 65.9 30.3
[9.noop] 0:01:30 118670168 0 1318557.422 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.4 ms 28 21:2 11 3.8 3.1 ms 0.0 ms 95.9 30.3
0:01:30 118670168 0 1318557.422 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.4 ms 28 21:2 11 3.8 3.1 ms 0.0 ms 95.9 30.3
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Max TPS-b
u:16 0 0:00:30 30000000 0 1000000.000 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.1 ms 28 21:2 11 2.7 2.9 ms 0.0 ms 66.2 30.3
t:2 1 0:00:30 29999999 0 999999.967 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.1 ms 28 21:2 11 2.7 1.6 ms 0.0 ms 64.3 30.3
2 0:00:30 29999998 0 999999.933 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.1 ms 28 21:2 11 2.7 1.1 ms 0.0 ms 64.1 30.3
[9.noop] 0:01:30 89999997 0 999999.967 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.1 ms 28 21:2 11 2.7 2.9 ms 0.0 ms 66.2 30.3
0:01:30 89999997 0 999999.967 0.0 ms 0.000 ms 0.0 ms 0.0 ms 0.0 ms 0.1 ms 28 21:2 11 2.7 2.9 ms 0.0 ms 66.2 30.3
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Fixed TPS-a
u:16 0 0:00:30 3001 0 100.033 0.0 ms 12.263 ms 8.3 ms 25.1 ms 28.2 ms 28.2 ms 28 21:2 3 1.2 69.3 ms 6.7 ms 59.6 22.8
t:2 1 0:00:30 3001 0 100.033 0.0 ms 12.365 ms 8.4 ms 25.1 ms 28.2 ms 28.3 ms 28 21:3 3 1.2 69.4 ms 7.4 ms 4.5 22.8
2 0:00:30 3001 0 100.033 0.0 ms 12.186 ms 8.3 ms 24.2 ms 28.2 ms 28.2 ms 28 21:3 3 1.2 75.7 ms 6.3 ms 5.1 22.8
3 0:00:30 3001 0 100.033 0.0 ms 11.993 ms 8.2 ms 24.2 ms 28.2 ms 28.2 ms 28 21:3 3 1.2 72.4 ms 6.5 ms 4.8 22.8
[1.DELAY-6ms] 0:02:00 3008 0 25.067 0.0 ms 6.124 ms 3.8 ms 11.2 ms 12.2 ms 12.3 ms 28 21:3 3 1.2 75.7 ms 7.4 ms 59.6 22.8
[2.DELAY-14ms] 0:02:00 8996 0 74.967 0.0 ms 14.234 ms 8.4 ms 26.1 ms 28.2 ms 28.3 ms 28 21:3 3 1.2 75.7 ms 7.4 ms 59.6 22.8
0:02:00 12004 0 100.033 0.0 ms 12.202 ms 8.3 ms 25.1 ms 28.2 ms 28.3 ms 28 21:3 3 1.2 75.7 ms 7.4 ms 59.6 22.8
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Fixed TPS-b
u:16 0 0:00:30 3001 0 100.033 10.1 ms 10.156 ms 0.0 ms 10.2 ms 10.2 ms 10.4 ms 28 21:3 1 1.0 3.0 ms 0.3 ms 4.6 14.0
t:2 1 0:00:30 3001 0 100.033 10.0 ms 10.154 ms 0.0 ms 10.2 ms 10.2 ms 10.6 ms 28 21:3 1 1.0 2.6 ms 0.3 ms 6.4 14.0
2 0:00:30 3001 0 100.033 10.1 ms 10.153 ms 0.0 ms 10.2 ms 10.2 ms 11.2 ms 28 21:3 1 1.0 2.3 ms 0.3 ms 17.6 13.8
3 0:00:30 3000 0 100.000 10.0 ms 10.155 ms 0.0 ms 10.2 ms 10.2 ms 10.4 ms 28 21:3 1 1.0 2.2 ms 0.3 ms 4.8 10.4
[10.DELAY-10ms] 0:02:00 12003 0 100.025 10.0 ms 10.155 ms 0.0 ms 10.2 ms 10.2 ms 11.2 ms 28 21:3 1 1.0 3.0 ms 0.3 ms 17.6 14.0
0:02:00 12003 0 100.025 10.0 ms 10.155 ms 0.0 ms 10.2 ms 10.2 ms 11.2 ms 28 21:3 1 1.0 3.0 ms 0.3 ms 17.6 14.0
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 HTTP-a
u:16 0 0:00:30 1487 0 49.567 28.7 ms 38.651 ms 24.7 ms 44.8 ms 63.2 ms 954.7 ms 28 21:3 11 8.5 1400.8 ms 334.2 ms 34.3 13.0
t:2 1 0:00:30 1560 0 52.000 29.2 ms 37.214 ms 8.6 ms 43.6 ms 61.6 ms 235.4 ms 28 21:3 11 8.8 524.8 ms 330.0 ms 12.6 13.0
2 0:00:30 1538 0 51.267 28.6 ms 37.759 ms 6.8 ms 44.9 ms 58.4 ms 160.3 ms 28 21:3 11 8.5 566.3 ms 325.7 ms 14.0 13.0
[3.HTTP-posts] 0:01:30 916 0 10.178 29.4 ms 38.936 ms 31.5 ms 44.5 ms 65.5 ms 954.7 ms 28 21:3 11 8.8 1400.8 ms 334.2 ms 34.3 13.0
[4.HTTP-comments] 0:01:30 917 0 10.189 28.8 ms 37.540 ms 6.1 ms 44.5 ms 58.2 ms 90.1 ms 28 21:3 11 8.8 1400.8 ms 334.2 ms 34.3 13.0
[5.HTTP-albums] 0:01:30 916 0 10.178 29.6 ms 37.761 ms 7.1 ms 44.9 ms 55.8 ms 160.3 ms 28 21:3 11 8.8 1400.8 ms 334.2 ms 34.3 13.0
[6.HTTP-photos] 0:01:30 918 0 10.200 28.6 ms 37.475 ms 6.0 ms 43.9 ms 57.0 ms 111.4 ms 28 21:3 11 8.8 1400.8 ms 334.2 ms 34.3 13.0
[7.HTTP-todos] 0:01:30 918 0 10.200 29.1 ms 37.603 ms 8.7 ms 44.8 ms 56.6 ms 235.4 ms 28 21:3 11 8.8 1400.8 ms 334.2 ms 34.3 13.0
0:01:30 4585 0 50.944 28.6 ms 37.863 ms 15.5 ms 44.5 ms 60.6 ms 954.7 ms 28 21:3 11 8.8 1400.8 ms 334.2 ms 34.3 13.0
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 HTTP-b
u:16 0 0:00:30 1349 0 44.967 28.6 ms 38.323 ms 6.8 ms 45.5 ms 62.0 ms 134.4 ms 28 21:3 3 1.7 178.4 ms 50.6 ms 11.0 13.5
t:2 1 0:00:30 1397 2 46.567 29.1 ms 37.787 ms 7.3 ms 45.9 ms 61.5 ms 120.4 ms 28 21:3 3 1.7 193.2 ms 48.1 ms 8.0 13.5
2 0:00:30 1379 0 45.967 29.3 ms 37.891 ms 12.1 ms 44.1 ms 60.6 ms 435.5 ms 28 21:3 3 1.7 537.6 ms 49.2 ms 18.6 13.5
[3.HTTP-posts] 0:01:30 822 0 9.133 29.1 ms 37.995 ms 7.7 ms 46.4 ms 62.0 ms 134.4 ms 28 21:3 3 1.7 537.6 ms 50.6 ms 18.6 13.5
[4.HTTP-comments] 0:01:30 825 0 9.167 30.1 ms 38.068 ms 7.0 ms 45.0 ms 62.0 ms 123.0 ms 28 21:3 3 1.7 537.6 ms 50.6 ms 18.6 13.5
[5.HTTP-albums] 0:01:30 827 2 9.189 29.5 ms 37.544 ms 5.3 ms 44.9 ms 55.2 ms 68.2 ms 28 21:3 3 1.7 537.6 ms 50.6 ms 18.6 13.5
[6.HTTP-photos] 0:01:30 827 0 9.189 28.6 ms 37.959 ms 6.0 ms 45.6 ms 61.1 ms 74.0 ms 28 21:3 3 1.7 537.6 ms 50.6 ms 18.6 13.5
[7.HTTP-todos] 0:01:30 824 0 9.156 29.3 ms 38.420 ms 15.5 ms 44.8 ms 62.7 ms 435.5 ms 28 21:3 3 1.7 537.6 ms 50.6 ms 18.6 13.5
0:01:30 4125 2 45.833 28.6 ms 37.997 ms 9.1 ms 45.2 ms 61.2 ms 435.5 ms 28 21:3 3 1.7 537.6 ms 50.6 ms 18.6 13.5
SID Name RID Duration #N #F Avg TPS Min RT Avg RT Stdev 90p RT 99p RT Max RT Max RTT MQS AQS Max WT Avg WT CPU JMM
0 Shutdown
u:16 0 0:00:01 16 0 9.944 100.2 ms 100.340 ms 0.1 ms 100.4 ms 100.9 ms 100.9 ms 28 21:3 1 1.0 0.1 ms 0.1 ms 5.3 13.5
t:2 [99] 0:00:01 16 0 9.944 100.2 ms 100.340 ms 0.1 ms 100.4 ms 100.9 ms 100.9 ms 28 21:3 1 1.0 0.1 ms 0.1 ms 5.3 13.5
0:00:01 16 0 9.944 100.2 ms 100.340 ms 0.1 ms 100.4 ms 100.9 ms 100.9 ms 28 21:3 1 1.0 0.1 ms 0.1 ms 5.3 13.5{
"actions": {
"description": "Micro-benchmarks",
"output_filename": "benchmark_output.json",
"report_filename": "benchmark_report.html",
"user_class": "org.example.user.HttpUser",
"user_params": {
"urlx": "https://jsonplaceholder.typicode.com",
"url": "http://localhost:7070",
"debug": false,
"http_port": 7070
},
"user_actions": {
"0": "onStart",
"1": "DELAY-6ms",
"2": "DELAY-14ms",
"3": "HTTP-posts",
"4": "HTTP-comments",
"5": "HTTP-albums",
"6": "HTTP-photos",
"7": "HTTP-todos",
"8": "login",
"9": "noop",
"10": "DELAY-10ms",
"11": "Pre-Auth",
"12": "Capture",
"13": "Refund",
"14": "Debit",
"99": "onStop"
}
},
"contexts": [
{
"name": "Scenario-1",
"enabled": true,
"num_users": 16,
"num_threads": 2
},
{
"name": "Scenario-2",
"enabled": false,
"num_users": 32,
"num_threads": 4
}
],
"benchmarks": [
{
"name": "Init",
"actions": [
{
"id": 0
},
{
"id": 8
}
]
},
{
"name": "Max TPS-a",
"enabled": true,
"throughput_rate": 0.0,
"worker_thread_queue_size": 0,
"time": {
"pre_warmup_duration": 15,
"warmup_duration": 30,
"benchmark_duration": 30,
"benchmark_repeat_count": 3
},
"actions": [
{
"id": 9
}
]
},
{
"name": "Max TPS-b",
"enabled": true,
"throughput_rate": 1000000.0,
"worker_thread_queue_size": 0,
"time": {
"pre_warmup_duration": 15,
"warmup_duration": 30,
"benchmark_duration": 30,
"benchmark_repeat_count": 3
},
"actions": [
{
"id": 9
}
]
},
{
"name": "Max TPS-c",
"enabled": true,
"throughput_rate": 250000.0,
"worker_thread_queue_size": 4,
"time": {
"pre_warmup_duration": 15,
"warmup_duration": 30,
"benchmark_duration": 30,
"benchmark_repeat_count": 3
},
"actions": [
{
"id": 9
}
]
},
{
"name": "Fixed TPS-a",
"enabled": true,
"throughput_rate": 50.0,
"worker_thread_queue_size": 1,
"time": {
"pre_warmup_duration": 15,
"warmup_duration": 15,
"benchmark_duration": 30,
"benchmark_repeat_count": 4
},
"actions": [
{
"id": 1,
"weight": 25
},
{
"id": 2,
"weight": 75
}
]
},
{
"name": "Fixed TPS-b",
"enabled": true,
"throughput_rate": 50.0,
"worker_thread_queue_size": 1,
"time": {
"pre_warmup_duration": 15,
"warmup_duration": 15,
"benchmark_duration": 30,
"benchmark_repeat_count": 4
},
"actions": [
{
"id": 10
}
]
},
{
"name": "HTTP-a",
"enabled": true,
"throughput_rate": 0.0,
"worker_thread_queue_size": 0,
"time": {
"pre_warmup_duration": 15,
"warmup_duration": 15,
"benchmark_duration": 30,
"benchmark_repeat_count": 3
},
"actions": [
{
"id": 3
},
{
"id": 4
},
{
"id": 5
},
{
"id": 6
},
{
"id": 7
}
]
},
{
"name": "HTTP-b",
"enabled": true,
"throughput_rate": 1250.0,
"worker_thread_queue_size": 0,
"time": {
"pre_warmup_duration": 15,
"warmup_duration": 15,
"benchmark_duration": 30,
"benchmark_repeat_count": 3
},
"actions": [
{
"id": 3
},
{
"id": 4
},
{
"id": 5
},
{
"id": 6
},
{
"id": 7
}
]
},
{
"name": "HTTP-c",
"enabled": false,
"throughput_rate": 1250.0,
"worker_thread_queue_size": 0,
"time": {
"pre_warmup_duration": 15,
"warmup_duration": 15,
"benchmark_duration": 30,
"benchmark_repeat_count": 3
},
"workflow": "rest-api1"
},
{
"name": "Shutdown",
"actions": [
{
"id": 99
}
]
}
],
"workflows": {
"payments": {
"-": {
"11": 0.5,
"14": 0.5
},
"11": {
"12": 1.0
},
"12": {
"13": 0.2,
"-": 0.8
},
"13": {
"-": 1.0
},
"14": {
"13": 0.2,
"-": 0.8
}
},
"random": {
"-": {
"1": 0.25,
"2": 0.75
},
"1": {
"-": 1.0
},
"2": {
"-": 1.0
}
}
}
}
The standalone benchmarks retrieve the tulip-runtime.jar from Maven Central.
Maven (xml)
<dependency>
<groupId>io.github.wfouche.tulip</groupId>
<artifactId>tulip-runtime</artifactId>
<version>2.0.1</version>
</dependency>
Tulip compensates for back-pressure from the system under test and adjusts the measured service times accordingly:
-
"Stop Rate Limiting! Capacity Management Done Right" by Jon Moore
Copyright 2024 Werner Fouché
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.