Create summary.json of results from K6 run #8
Conversation
Signed-off-by: waleedqk <[email protected]> On branch wqk_prom_scrape Changes to be committed: modified: perf_test/runHowitzer.sh new file: perf_test/scripts/scraper.py
perf_test/runHowitzer.sh
Outdated
| @@ -66,8 +66,9 @@ for TEST in render/*.js; do | |||
| emphasize "Intermediate test results for: ${TEST}" | |||
| cat $RESULT_FILE | |||
|
|
|||
| # emphasize "Generating intermediate Markdown for K6 tests..." | |||
| # python3 -m src.k6.scraper -r results -s summary -p persistent-results -c $CONFIG_FILE -k configs/kingdom.dict | |||
| emphasize "Generating intermediate Markdown for K6 tests..." | |||
There was a problem hiding this comment.
There's another python3 -m perf_test.scripts.scraper -r results -s summary -c $CONFIG_FILE down below that basically does the same command. Is there a difference here? Maybe we only need one?
There was a problem hiding this comment.
Yeah, so testing it out locally and understanding the code as well, I think we can just remove the invocation in the loop and just running it at the end will suffice
Testing it on the perf cluster to confirm no change in behaviour
Signed-off-by: waleedqk <[email protected]>
…ssed and dir exists Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
… scraping Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
waleedqk
changed the title
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
Signed-off-by: waleedqk <[email protected]>
perf_test/runHowitzer.sh
Outdated
| # emphasize "K6 TESTS ARE COMPLETE. Use the following command to copy to your CWD." | ||
| # echo "kubectl cp ${POD_NAME}:output.md ./output.md && kubectl cp ${POD_NAME}:summary.json ./summary.json" | ||
|
|
||
| # Add a sleep at the end of the job to give time to exec into pods if need be | ||
| sleep 120 |
There was a problem hiding this comment.
do you still want to keep this sleep here?
perf_test/README.md
Outdated
| @@ -16,7 +16,8 @@ The core design principle beneath Howitzer lies in "exploding" K6 parameter iter | |||
| When Howitzer is run, the bash script will create directories `render`, `results` and `summary`: | |||
| - `render` stores the "exploded" k6 tests using the k6 template specified in the test config for each k6_opts combination. | |||
| - Then the script will run each individual k6 test in this `render` directory, storing the human readable results as text in `results` directory. | |||
| - It will also store the corresponding json file of the result into `summary` directory that we can later use for MLFlow. | |||
| - It will also store the corresponding json file of the result into `summary` directory. | |||
| - The scraper script then consolidates the results from the K6 test, the input test conditions etc. into a single `summary.json` file for the tests that were run. | |||
There was a problem hiding this comment.
I think since now we have multiple scripts, let's be explicit and say the perf_test.scripts.scraper script, instead of just scraper script.
perf_test/README.md
Outdated
| @@ -16,7 +16,8 @@ The core design principle beneath Howitzer lies in "exploding" K6 parameter iter | |||
| When Howitzer is run, the bash script will create directories `render`, `results` and `summary`: | |||
There was a problem hiding this comment.
can you change this to say
the bash script `perf_test/runHowitzer.sh`
so we can be specific
…used for debugging only. Signed-off-by: waleedqk <[email protected]>
|
/lgtm |
|
We need to fix the issue with github not being able to install python in the github action. @waleedqk Please also change the .github/workflows file to run |
Signed-off-by: waleedqk <[email protected]>
|
New changes are detected. LGTM label has been removed. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: waleedqk The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@waleedqk Can you do me a favor and try with version |
When a K6 perf test is being run, individual files are created that hold the result for each test that is part of the submitted set.
The scrapper script consolidates the results from each of the output files and joins them into a single
summary.jsonfile.