Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CLI for measuring execution time and cost #1628

Merged
merged 5 commits into from
Oct 17, 2023

Conversation

iamyulong
Copy link
Member

@iamyulong iamyulong commented Oct 16, 2023

Summary

Add CLI for measuring execution time and cost.

@github-actions
Copy link

github-actions bot commented Oct 16, 2023

Benchmark for 9527f08

Click to view benchmark
Test Base PR %
costing::bench_prepare_wasm 76.3±0.52ms 76.1±0.50ms -0.26%
costing::decode_sbor 16.8±0.07µs 16.9±0.04µs +0.60%
costing::decode_sbor_bytes 46.1±0.20µs 45.8±0.29µs -0.65%
costing::deserialize_wasm 1399.6±5.67µs 1401.5±3.47µs +0.14%
costing::instantiate_flash_loan 4.2±0.55ms 5.6±0.05ms +33.33%
costing::instantiate_radiswap 5.6±0.03ms 5.6±0.05ms 0.00%
costing::spin_loop 26.2±0.05ms 26.0±0.06ms -0.76%
costing::validate_sbor_payload 30.7±0.09µs 31.1±0.07µs +1.30%
costing::validate_sbor_payload_bytes 383.4±0.79ns 379.3±1.39ns -1.07%
costing::validate_secp256k1 87.4±0.36µs 87.5±0.11µs +0.11%
costing::validate_wasm 43.8±0.11ms 43.4±0.08ms -0.91%
decimal::add/0 9.3±0.01ns 9.3±0.06ns 0.00%
decimal::add/rust-native 10.8±0.01ns 10.9±0.01ns +0.93%
decimal::add/wasmer 138.8±1.69ns 129.4±0.19ns -6.77%
decimal::add/wasmer-call-native 565.4±1.42ns 557.5±0.61ns -1.40%
decimal::add/wasmi 791.7±4.74ns 834.4±10.96ns +5.39%
decimal::add/wasmi-call-native 8.3±0.01µs 9.0±0.07µs +8.43%
decimal::div/0 201.3±0.44ns 202.0±0.36ns +0.35%
decimal::from_string/0 177.4±0.45ns 177.6±0.33ns +0.11%
decimal::mul/0 157.4±0.25ns 155.9±0.23ns -0.95%
decimal::mul/rust-native 160.9±0.18ns 153.2±0.10ns -4.79%
decimal::mul/wasmer 1688.7±0.49ns 1661.4±0.91ns -1.62%
decimal::mul/wasmer-call-native 707.4±0.53ns 714.1±1.92ns +0.95%
decimal::mul/wasmi 60.6±0.11µs 61.8±0.93µs +1.98%
decimal::mul/wasmi-call-native 8.5±0.02µs 9.3±0.02µs +9.41%
decimal::pow/0 728.9±3.06ns 724.7±0.46ns -0.58%
decimal::pow/rust-native 697.7±0.38ns 696.8±0.99ns -0.13%
decimal::pow/wasmer 7.3±0.00µs 7.3±0.00µs 0.00%
decimal::pow/wasmer-call-native 1174.4±1.83ns 1179.2±2.67ns +0.41%
decimal::pow/wasmi 282.0±0.44µs 283.9±0.29µs +0.67%
decimal::pow/wasmi-call-native 7.4±0.01µs 7.8±0.02µs +5.41%
decimal::root/0 9.4±0.01µs 9.5±0.01µs +1.06%
decimal::sub/0 9.4±0.01ns 9.4±0.01ns 0.00%
decimal::to_string/0 490.5±1.96ns 482.3±0.92ns -1.67%
precise_decimal::add/0 10.5±0.46ns 11.1±0.01ns +5.71%
precise_decimal::add/rust-native 12.7±0.04ns 12.6±0.06ns -0.79%
precise_decimal::add/wasmer 141.1±1.19ns 136.2±0.09ns -3.47%
precise_decimal::add/wasmer-call-native 577.8±0.38ns 587.8±0.61ns +1.73%
precise_decimal::add/wasmi 1025.4±12.60ns 1035.2±9.11ns +0.96%
precise_decimal::add/wasmi-call-native 8.9±0.01µs 9.5±0.03µs +6.74%
precise_decimal::div/0 325.9±0.50ns 328.6±1.11ns +0.83%
precise_decimal::from_string/0 225.3±0.34ns 224.2±0.25ns -0.49%
precise_decimal::mul/0 347.5±0.99ns 370.8±4.88ns +6.71%
precise_decimal::mul/rust-native 360.8±13.86ns 332.0±2.04ns -7.98%
precise_decimal::mul/wasmer 3.8±0.01µs 3.9±0.00µs +2.63%
precise_decimal::mul/wasmer-call-native 961.1±3.35ns 984.2±2.45ns +2.40%
precise_decimal::mul/wasmi 151.1±0.74µs 154.0±0.11µs +1.92%
precise_decimal::mul/wasmi-call-native 9.4±0.02µs 10.2±0.02µs +8.51%
precise_decimal::pow/0 1981.9±5.32ns 1998.7±8.59ns +0.85%
precise_decimal::pow/rust-native 1600.2±10.61ns 1553.3±3.06ns -2.93%
precise_decimal::pow/wasmer 18.0±0.01µs 18.5±0.01µs +2.78%
precise_decimal::pow/wasmer-call-native 2.3±0.00µs 2.3±0.00µs 0.00%
precise_decimal::pow/wasmi 731.3±0.70µs 745.3±0.65µs +1.91%
precise_decimal::pow/wasmi-call-native 16.6±0.03µs 17.4±0.04µs +4.82%
precise_decimal::root/0 64.6±0.03µs 64.6±0.01µs 0.00%
precise_decimal::sub/0 10.6±0.01ns 10.6±0.04ns 0.00%
precise_decimal::to_string/0 770.7±1.31ns 780.7±2.60ns +1.30%
schema::validate_payload 346.4±1.64µs 344.2±3.77µs -0.64%
transaction::radiswap 6.4±0.08ms 6.4±0.06ms 0.00%
transaction::transfer 2.1±0.00ms 2.1±0.02ms 0.00%
transaction_processing::prepare 3.1±0.00ms 3.0±0.00ms -3.23%
transaction_processing::prepare_and_decompile 7.0±0.01ms 6.9±0.02ms -1.43%
transaction_processing::prepare_and_decompile_and_recompile 26.5±0.28ms 26.2±0.11ms -1.13%
transaction_validation::validate_manifest 48.7±0.07µs 48.5±0.03µs -0.41%
transaction_validation::verify_ecdsa 85.0±0.25µs 84.9±0.12µs -0.12%
transaction_validation::verify_ed25519 60.5±0.27µs 60.8±0.18µs +0.50%

@iamyulong iamyulong changed the base branch from feature/improve-blueprint-definition-cache to develop October 16, 2023 15:42
@iamyulong iamyulong marked this pull request as ready for review October 17, 2023 09:05
Copy link
Contributor

@mstrug-rdx mstrug-rdx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from that comment about the network, it looks good.

impl TxnMeasure {
pub fn run(&self) -> Result<(), Error> {
let network = match &self.network {
Some(n) => NetworkDefinition::from_str(n).map_err(Error::ParseNetworkError)?,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NetworkDefinition::from_str() doesn't have stokenet as in comment above, also maybe there should be additional validation only for miannet or stokenet?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me add stokenet to NetworkDefinition since it's so commonly used.

@iamyulong iamyulong merged commit 1d776aa into develop Oct 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants