Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a new configuration to the model checking CI #5802

Merged
merged 9 commits into from
Oct 31, 2023

Conversation

heidihoward
Copy link
Member

Follow up from #5801

This PR as a new configuration to the exhaustive model checking CI to test atomic reconfiguration from one node to another.

@heidihoward heidihoward added the tla TLA+ specifications label Oct 31, 2023
@heidihoward heidihoward requested a review from a team October 31, 2023 15:01
@ghost
Copy link

ghost commented Oct 31, 2023

tla-a@78184 aka 20231031.32 vs main ewma over 20 builds from 77584 to 78161

Click to see table

main

build_id build_number pi_basic_mt_sgx_cft^ pi_basic_mt_sgx_cft_mem Commit latency factor tpcc_sgx_cft^ tpcc_sgx_cft_mem tpcc_virtual_cft^ pi_basic_mt_virtual_cft^ ls_sgx_cft^ ls_sgx_cft_mem pi_ls_sgx_cft^ pi_ls_sgx_cft_mem pi_basic_sgx_cft^ pi_basic_sgx_cft_mem ls_virtual_cft^ pi_ls_virtual_cft^ pi_basic_virtual_cft^ pi_basic_js_virtual_cft^ ls_jwt_virtual_cft^ pi_ls_jwt_virtual_cft^ pi_basic_js_sgx_cft^ pi_basic_js_sgx_cft_mem ls_js_virtual_cft^ ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem ls_full_js_virtual_cft^ pi_ls_jwt_sgx_cft^ pi_ls_jwt_sgx_cft_mem ls_js_jwt_virtual_cft^ ls_js_sgx_cft^ ls_js_sgx_cft_mem ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem hist_sgx_cft^ RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^
77584 20231023.15 27775.2 2.51822e+07 0.839543 5525.06 8.59996e+07 17237.3 63643 13936 1.88908e+07 13990.4 1.05021e+07 15424.6 1.46964e+07 43785.4 47940.2 54114.3 4430 17010.7 19396.4 1427 1.25993e+07 17307.2 6853.5 1.88908e+07 14810.6 6881.2 6.30784e+06 9823.55 5758.93 1.67936e+07 5762.09 1.67936e+07 3983.52 1.67936e+07 42679.4 825381 1.18095e+06 8.16893e+06 3.13159e+07
77665 20231024.10 28077.5 2.30851e+07 0.782853 5565 8.59996e+07 17332.4 71956.1 13918.6 1.88908e+07 14015.8 1.05021e+07 15439.4 1.46964e+07 43627.5 47460.4 55764.1 4480.7 16984.5 19600.5 1417.5 1.25993e+07 17210.4 6827.52 1.88908e+07 15032.4 6933.8 6.30784e+06 9934.79 5759.87 1.67936e+07 5778.73 1.67936e+07 3994.77 1.67936e+07 42277.7 837727 1.17659e+06 8.1335e+06 3.08424e+07
77722 20231024.29 28002.9 2.51822e+07 0.808583 5477.83 8.59996e+07 17482.9 68825.6 13980.7 1.88908e+07 14011.7 1.05021e+07 15103.3 1.46964e+07 43910 47389.3 55105.8 4422.3 16897.2 19100.4 1409.7 1.25993e+07 17082.1 6842.46 1.88908e+07 15051.5 6811.8 6.30784e+06 9901.25 5772.89 1.67936e+07 5489.45 1.67936e+07 4013.18 1.67936e+07 43837.6 824014 1.17985e+06 8.16421e+06 3.03628e+07
77856 20231025.44 27880.1 2.30851e+07 0.830072 5666.89 8.59996e+07 17303.7 84849.8 14118.2 1.88908e+07 14176.7 1.05021e+07 15508.7 1.46964e+07 45783 47925.3 54083.6 4428.6 17130.6 18998.8 1425.1 1.25993e+07 17531.7 7259.6 1.67936e+07 14906.9 6931.4 6.30784e+06 9965.7 5816.26 1.67936e+07 5714.88 1.67936e+07 3996.87 1.67936e+07 44821 832615 1.17692e+06 8.17447e+06 3.1229e+07
77900 20231025.59 27968.9 2.30851e+07 0.816541 5546.67 8.59996e+07 17281.4 81852.5 13948.6 1.88908e+07 14105.4 1.05021e+07 15469.3 1.46964e+07 45862.6 47692.2 54378.7 4430.3 17128.6 18994.4 1430.1 1.25993e+07 17211.3 6806.65 1.88908e+07 14724.7 6880.7 6.30784e+06 9880.77 5770.88 1.67936e+07 5488.27 1.88908e+07 3990.38 1.67936e+07 42489.7 839143 1.18218e+06 8.15475e+06 3.09642e+07
77909 20231025.62 27790.9 2.30851e+07 0.79592 5576.25 8.59996e+07 17614.6 82618.8 13953.5 1.88908e+07 13887.2 1.05021e+07 15385.2 1.25993e+07 45746.3 48128.1 53844.8 4388 17161.3 19081.4 1410.9 1.25993e+07 17522.5 6741.99 1.88908e+07 14627.1 6827.9 6.30784e+06 10342.4 5792.34 1.67936e+07 5446.53 1.67936e+07 3975.81 1.67936e+07 43740.4 830943 1.17967e+06 8.14732e+06 3.07715e+07
77921 20231026.3 27942.2 2.30851e+07 0.791384 5619.54 8.80968e+07 17373 72066.7 14116.6 1.88908e+07 14237.9 1.05021e+07 15679.6 1.25993e+07 45699.4 48314.8 54188.2 4351.5 17286.9 19246.7 1439.5 1.25993e+07 17012.1 6842.16 1.67936e+07 14641.1 7086.5 6.30784e+06 10204.5 5774.21 1.67936e+07 5775.45 1.67936e+07 3999.2 1.67936e+07 44491.1 835419 1.1779e+06 8.15017e+06 3.23452e+07
77958 20231026.14 27891.3 2.30851e+07 0.772274 5566.37 8.59996e+07 17260.9 68096.1 14074.9 1.88908e+07 14198.3 1.05021e+07 15598.1 1.46964e+07 45673.6 48273.7 54086 4383.2 17108.5 19101.5 1432.2 1.25993e+07 17286.3 6840.93 1.67936e+07 14984.8 6965.9 6.30784e+06 10377.2 5777.94 1.67936e+07 5783.74 1.67936e+07 3984.61 1.67936e+07 43732 834153 1.18014e+06 8.15037e+06 3.08165e+07
77980 20231026.23 27919.7 2.30851e+07 0.799401 5544.83 8.59996e+07 17301.4 84229.4 13945.1 1.88908e+07 14005.8 1.05021e+07 15441 1.25993e+07 45630 47959.1 54124.7 4420.8 17150.6 19602.5 1419.8 1.25993e+07 17059.4 7232.3 1.67936e+07 14690.5 6917.8 6.30784e+06 9850.2 5811.09 1.67936e+07 5487 1.67936e+07 4003.14 1.67936e+07 42472.7 827669 1.18001e+06 8.13674e+06 3.08104e+07
77994 20231026.29 27941.4 2.51822e+07 0.80113 5582.56 8.59996e+07 17290 82095.1 14017.2 1.88908e+07 14167 1.05021e+07 15610.3 1.25993e+07 43873.2 47649.7 54065 4441.8 17136.7 19419.7 1446 1.25993e+07 17733.8 6841.03 1.88908e+07 16838 6962.5 6.30784e+06 9840.75 5764.66 1.67936e+07 5729.26 1.67936e+07 4003.36 1.67936e+07 44959.6 840437 1.18274e+06 8.15494e+06 3.07517e+07
78007 20231026.34 27449.9 2.51822e+07 0.765143 5628.96 8.59996e+07 17307 81427 14025.1 1.88908e+07 14089.7 1.05021e+07 15601.5 1.46964e+07 45407.5 47735 54044.5 4441.4 17288.5 18904.7 1432.4 1.25993e+07 17015.3 6850.52 1.88908e+07 16872.1 7096.8 6.30784e+06 9816.73 5773.88 1.67936e+07 5486.07 1.67936e+07 3985.52 1.67936e+07 43179.9 826445 1.17834e+06 8.14738e+06 3.19605e+07
78033 20231027.4 28127.4 2.51822e+07 0.798824 5577.93 8.59996e+07 17273 65638.2 13972.7 1.88908e+07 14096.8 1.05021e+07 15447.2 1.25993e+07 45862.4 48024.5 53817.4 4439.6 17160.1 19602.8 1432.2 1.25993e+07 17044.2 7250.65 1.67936e+07 16883.5 6938.4 6.30784e+06 9854.32 5786.4 1.67936e+07 5743.72 1.67936e+07 4002.74 1.67936e+07 42046.2 829984 1.18448e+06 8.13813e+06 3.06775e+07
78035 20231027.5 27938.3 2.51822e+07 0.785967 5573.65 8.59996e+07 17213.8 83501.4 14007.5 1.88908e+07 14114.5 1.05021e+07 15503.3 1.46964e+07 45970.2 48037.9 54501.8 4447.9 17062.2 19333.8 1434.7 1.25993e+07 16911.6 6832.63 1.67936e+07 17015.7 6983.9 6.30784e+06 9990.67 5786.59 1.67936e+07 5775.4 1.67936e+07 3982.36 1.67936e+07 47144 831267 1.18066e+06 8.09742e+06 3.06715e+07
78049 20231027.10 28049.5 2.51822e+07 0.781085 5504.64 8.59996e+07 17300.2 85552.5 14023.9 1.88908e+07 14102.3 1.05021e+07 15421.6 1.46964e+07 43575 46933.4 55026 4431 17056.6 19111.7 1424.4 1.25993e+07 17359 6846.42 1.67936e+07 14869.3 6826.6 6.30784e+06 9805.16 5791.73 1.67936e+07 5707.49 1.67936e+07 3980.86 1.67936e+07 41017.4 840517 1.18098e+06 8.16672e+06 3.06803e+07
78065 20231030.3 27850.6 2.30851e+07 0.803529 5645.18 8.59996e+07 17287.6 80986.2 14038.5 1.88908e+07 14125 1.05021e+07 15560.2 1.46964e+07 43811.1 46782.1 54407.3 4392.3 17198.2 19724.6 1434 1.25993e+07 17634.6 7241.2 1.67936e+07 14791.7 7023.9 6.30784e+06 10345.8 5804.59 1.67936e+07 5779.74 1.67936e+07 3998.35 1.67936e+07 38753.2 831589 1.1757e+06 8.07743e+06 3.07175e+07
78110 20231031.2 28002.1 2.30851e+07 0.801968 5577.82 8.59996e+07 17706.2 90290.4 14078.8 1.88908e+07 14211.9 1.05021e+07 15500.2 1.46964e+07 43757.1 47025.6 54021.7 4412.2 17043.4 19501.2 1434.1 1.25993e+07 17190 7246.56 1.67936e+07 14923.3 7122.3 6.30784e+06 10161.5 5824.15 1.67936e+07 5724.41 1.67936e+07 3999.43 1.67936e+07 44321.1 833541 1.17964e+06 8.15397e+06 3.09702e+07
78115 20231031.5 28096.6 2.51822e+07 0.788781 5597.14 8.59996e+07 17153.5 90398.6 14046.4 1.88908e+07 14149.3 1.05021e+07 15573.1 1.46964e+07 45717.6 47743.9 55284.5 4374.5 17204.3 19257.8 1438.8 1.25993e+07 17318.7 6857.73 1.67936e+07 14893.2 6984.1 6.30784e+06 9790.65 5786.81 1.67936e+07 5770.07 1.67936e+07 3994.27 1.67936e+07 44365.6 818357 1.17717e+06 8.15176e+06 3.07813e+07
78128 20231031.10 27738.1 2.51822e+07 0.75716 5630.66 8.59996e+07 17076.3 71982.3 14058.5 1.88908e+07 14160.9 1.05021e+07 15631.8 1.46964e+07 43862 47267.7 55100.9 4371.6 17200.1 19235.2 1441.7 1.25993e+07 17122 7284.99 1.67936e+07 14762.1 7044.7 6.30784e+06 10396.2 5805.39 1.67936e+07 5776.39 1.67936e+07 3998.07 1.67936e+07 42934.8 837504 1.18196e+06 8.15251e+06 3.15981e+07
78146 20231031.17 27901 2.51822e+07 0.780638 5598.38 8.59996e+07 17229.5 76464.7 14060.8 1.88908e+07 14179 1.05021e+07 15535.7 1.46964e+07 43813 47595.9 55096.1 4355.7 17361 19744.5 1433.1 1.25993e+07 17356.3 6790.28 1.67936e+07 14955.7 6889.6 6.30784e+06 9799.34 5772.45 1.67936e+07 5488.66 1.67936e+07 3966.05 1.67936e+07 43021.9 820080 1.18077e+06 8.15047e+06 3.07762e+07
78161 20231031.23 28091.4 2.51822e+07 0.782488 5605.13 8.59996e+07 17341.2 62893.4 14058.7 1.88908e+07 14092.9 1.05021e+07 15472.1 1.46964e+07 43725.6 47396.3 54809.3 4373.5 17176.1 19492.8 1434.3 1.25993e+07 17576.4 6855.37 1.67936e+07 14802.5 7006.3 6.30784e+06 9795.27 5771.02 1.67936e+07 5735.27 1.67936e+07 3972.38 1.67936e+07 46424.3 831500 1.17237e+06 8.17199e+06 3.13663e+07

tla-a

build_id build_number Commit latency factor tpcc_sgx_cft^ tpcc_sgx_cft_mem ls_sgx_cft^ ls_sgx_cft_mem pi_ls_sgx_cft^ pi_ls_sgx_cft_mem pi_basic_mt_sgx_cft^ pi_basic_mt_sgx_cft_mem pi_basic_sgx_cft^ pi_basic_sgx_cft_mem pi_basic_js_sgx_cft^ pi_basic_js_sgx_cft_mem pi_basic_mt_virtual_cft^ ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem tpcc_virtual_cft^ pi_ls_jwt_sgx_cft^ pi_ls_jwt_sgx_cft_mem ls_js_sgx_cft^ ls_js_sgx_cft_mem ls_virtual_cft^ pi_ls_virtual_cft^ pi_basic_virtual_cft^ ls_full_js_sgx_cft^ ls_full_js_sgx_cft_mem pi_basic_js_virtual_cft^ ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem ls_jwt_virtual_cft^ pi_ls_jwt_virtual_cft^ ls_js_virtual_cft^ ls_full_js_virtual_cft^ ls_js_jwt_virtual_cft^ hist_sgx_cft^ RB put (/s)^ CHAMP put (/s)^ RB get (/s)^ CHAMP get (/s)^
78139 20231031.13 0.766606 5569.22 8.59996e+07 14011.8 1.88908e+07 14137.5 1.05021e+07 27880.5 2.30851e+07 15483.4 1.46964e+07 1438.8 1.05021e+07 65449.4 6842.57 1.88908e+07 17254 6910.3 6.30784e+06 5815.96 1.67936e+07 43853.1 47436.8 49668.8 5787.66 1.67936e+07 4380.5 3995.28 1.67936e+07 17176.2 19387.4 17409.6 14750.1 10260.5 47001.2 828189 1.17946e+06 8.14972e+06 3.06188e+07
78153 20231031.19 0.798423 5656.03 8.59996e+07 14085.2 1.88908e+07 14163.4 1.05021e+07 27799.3 2.30851e+07 15636.2 1.46964e+07 1447.3 1.25993e+07 75602.7 6800.81 1.67936e+07 17171 7085 6.30784e+06 5808.2 1.67936e+07 43826.6 47202.1 55142.4 5736.21 1.67936e+07 4323.4 4003.7 1.67936e+07 17304.4 19234.8 17396.9 14865.3 9811.84 40327.7 833030 1.17892e+06 8.1505e+06 3.08141e+07
78170 20231031.26 0.787773 5650.14 8.59996e+07 14072.1 1.88908e+07 14168 1.05021e+07 27842.3 2.30851e+07 15496.1 1.46964e+07 1437.8 1.25993e+07 84541.3 7260.65 1.67936e+07 17209.8 6948.1 6.30784e+06 5766.1 1.67936e+07 44253.7 47103.6 54626.1 5779.02 1.67936e+07 4391.8 4001.82 1.67936e+07 17649.9 17571.6 17336.5 14863.4 9929.06 43117.2 836732 1.18192e+06 8.11591e+06 3.23329e+07
78184 20231031.32 0.814171 5631.48 8.59996e+07 14057.6 1.67936e+07 14119.7 1.05021e+07 27766.3 2.30851e+07 15576.2 1.46964e+07 1435.4 1.25993e+07 62261.5 6837.3 1.88908e+07 17380.4 6973.2 6.30784e+06 5786.38 1.67936e+07 43997.6 46963.1 54857.6 5717.16 1.88908e+07 4359.7 3982.32 1.67936e+07 17047.9 19981.6 17550.2 14832 9784.71 46962.3 829227 1.17844e+06 8.14674e+06 3.08424e+07

images

@achamayou achamayou merged commit 144cbe0 into microsoft:main Oct 31, 2023
25 checks passed
@heidihoward heidihoward deleted the tla-a branch November 1, 2023 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tla TLA+ specifications
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants