[benchmark] notebook for interactively computing benchmark config

[ehigham]

Change Description

Adds tools for analyzing benchmark variability and detecting minimal slowdowns when running benchmarks on Hail Batch. The analysis is based on methods from Laaber et al.'s paper on software microbenchmarking in the cloud.

Key additions:

• New Jupyter notebook for exploring benchmark variability
• Statistical analysis tools for computing confidence intervals and detecting performance changes
• Functions for determining minimal detectable slowdowns using different testing strategies
• Improved benchmark data import/export capabilities
• Enhanced visualization tools for benchmark results

Security Assessment

This change has no security impact

Impact Description

Analysis and visualization code that operates on benchmark data only, with no access to sensitive information or systems.

[ehigham]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• [benchmark] notebook for interactively computing benchmark config #14757 👈 (View in Graphite)
• [query] make local and remote tmp settable on backend #14748
• [query] Lowering + Optimisation with implict timing context #14731 : 1 other dependent PR (#14732 )
• [query] Lift backend state into {Service|Py4J}BackendApi #14698
• [query] Move LoweredTableReaderCoercer into ExecuteContext #14696
• [query] unify backend rpc #14693
• [query] Remove persistedIr from Backend interface #14692
• [query] Remove lookupOrCompileCachedFunction from Backend interface #14691
• [query] Remove BlockMatrix persist from Backend interface #14690
• [query] Expose references via ExecuteContext #14686
• [query] Unify CloudCredentials and Simplify BatchClient #14684 : 1 other dependent PR (#14747 )
• [query] Use sourcecode.Enclosing to handle timed blocks implicitly #14683
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.