Hook the benchmark runner and benchmark visualizations into GitHub #30
Conversation
|
I think this needs to be rebased to (or merged with) master now after #2 was merged. |
I was surprised to find out that the timeout was being ignored. I suspect that I introduced that mistake while I was benchmarking the C runtime and temporarily disabled the timeout because for C, SortedLinkedList and friends were taking so very long.
petervdonovan
force-pushed
the
automated-full-benchmark
branch
from
3b2266c
to
c210de2
Compare
runner/collect_results.py
Outdated
| @@ -10,8 +10,8 @@ | |||
| import json | |||
|
|
|||
|
|
|||
| def dir_path(string): | |||
| if os.path.isdir(string): | |||
| def src_path_type(string): | |||
There was a problem hiding this comment.
This shouldn't change anymore right? Since dir_path now is an optional argument and we don't explicitly use 'latest'
|
|
||
|
|
||
| def execute_command(command, timeout=None, stacktrace=False): |
There was a problem hiding this comment.
Could you provide some context on why you changes this and what exactly it does?
There was a problem hiding this comment.
I wanted to read the output of the process in a non-blocking way so that we could capture the stack trace of the running process before killing it. By default, the stack trace is captured because the user may not have eu-stack, but I wanted this feature so that ideally, it might be possible to debug a deadlock in CI without having to manually run the executable hundreds of times.
|
This looks like it would be super useful, but it's somehow still marked as draft. How close to being merged was this when it got stalled? |
|
I do not remember, but it was pretty close. I did not merge it partly because I do not have the time to make the behavior of the changes acceptable to everyone, but also because I do not have the time to take advantage of the outputs that the changes would give if they were merged. Performance optimization for the C target is not among my near-term goals, and if someone else does take that up as a near-term goal (I am not sure that they should), then they can do what they think is necessary in order to guard against performance regressions. |
See lf-lang/reactor-c#91 (comment) for details.