Skip to content
This repository has been archived by the owner on Aug 14, 2024. It is now read-only.

Create a Command Line Application to Make Benchmarking Easier to Use #15

Open
Lementknight opened this issue Jul 6, 2022 · 0 comments · May be fixed by #27
Open

Create a Command Line Application to Make Benchmarking Easier to Use #15

Lementknight opened this issue Jul 6, 2022 · 0 comments · May be fixed by #27

Comments

@Lementknight
Copy link
Contributor

I want to create a command line application for Red Queen that would make it easier for people to perform benchmarks. At the moment, to use this benchmark it requires knowledge on Pytest.

Instead of a user having to type up:
$ pytest games/mapping/map_queko.py -m tweedledum --store

I believe that it would better if we could instead type out this
$ red-queen map_queko tweedledum
$ red-queen [file_name] [benchmark]

I believe that down the line we could also use this proposed CLI to create graphs and better visuals for the end user. Ultimately, by reducing the complexity needed to use this benchmarking suite. I hope to make Red Queen more user friendly and more robust.
mtreinish pushed a commit that referenced this issue Aug 10, 2022
@raynelfss
Added CI Job to test new benchmarks (#29)
120ef23 
* Initial commit: Added CI Job to test new benchmarks

* Added test file (copy of run_bv.py)

* Added Matthew's suggestions (#4)

* Initial commit for ci tests

* Change bash script

* Bash script does not need cat

* Added second test file to test

* Sugar is unnecessary

* Added test in mapping

* Mapping takes way longer. Will develop further.

* Delete tests.txt

Deleted old residuals, ready to merge with ci branch

* Added upload artifact action (#6)

* Another test for CI

* Organized CI actions.

* Added both the text file and the original results as artifacts

* Added more aesthetic changes suggested by Matthew (#9)

* Added more changes suggested by Matthew

* Added proper management of __init__.py files (#11)

* Another test for CI

* Organized CI actions.

* Added both the text file and the original results as artifacts

* Added more changes suggested by matthew

* Added method to handle changes to __init__.py files

* Fixed error in main.yml file

* Fixing another error in yaml file

* Fixing scope error in yaml

* Updated changed-files 24 -> 24.1

* Attempt 1

* Attempt 2

* Attempt 3

* Worked. Fixed typo... Now reordering.

* Made a change in init file, let's see if it detects

* Fixed bash code

* Checking for something suspicious

* Checking for something suspicious

* Fixing something suspicious

* Fixed?

* Fixed??

* Test with only __init__ file

* Final commit before merging

* Improved Runtime by removing Washington, Brooklyn and Rochester backends. (#13)

* Remove usage of Washington, Brooklyn and Rockester backends due to slow runtime.

* What if there are no changes?

* Handle no files in a healthy way

* No changes works! Going back to adding changes.

* Created custom action to test each specific benchmark (#15)

* Try customized action

* Added custom acrtion to workflow.

* Fixed typo in workflow.

* Modified action location

* Modified action location 2

* Modified action location 3

* Modified action location 4

* Fixed path for testing

* Fixed typo in line 22 of action.

* Housekeeping, and red-added exclusions.

* Final commit, custom action is ready to be merged.

* Final fixes before merging (#17)

* Final fixes before merging

* Added modifications to CONTRIBUTING.md

* Added last modification to CONTRIBUTING.md

* Delete test.py for final version
@kdk kdk linked a pull request Sep 21, 2023 that will close this issue
Adding Red Queen CLI #27
Open
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Adding Red Queen CLI Lementknight/red-queen
1 participant
@Lementknight