Skip to content
This repository has been archived by the owner on Jan 26, 2024. It is now read-only.

Benchmarks and non regression tests #22

Open
JoanThibault opened this issue Feb 24, 2017 · 7 comments
Open

Benchmarks and non regression tests #22

JoanThibault opened this issue Feb 24, 2017 · 7 comments

Comments

@JoanThibault
Copy link
Contributor

is there an easy way to add benchmarks to the non-regression tests. So that when we launch build.sh it does not end with "100% tests passed" but having the benchmarks' compilation which fails

@Ruyk
Copy link
Contributor

Ruyk commented Mar 1, 2017

We could add the benchmarks themselves as tests

@keryell
Copy link
Member

keryell commented May 4, 2017

Any progress on this?

@Ruyk
Copy link
Contributor

Ruyk commented May 4, 2017

The benchmarks take too much time for testing purposes. We can build them, but not automatically run them from the script, does that seems fine?

@keryell
Copy link
Member

keryell commented May 4, 2017

I do not remember it was so slow.
I have the feeling that currently the build script runs them anyway.
It is just that the results is not taken into account in the test success rate.
So it should not change the global run time, if my understanding is correct.

@Ruyk
Copy link
Contributor

Ruyk commented May 9, 2017

The CMakeList script currently builds the benchmarks but does not execute them. I am reluctant to add it to the testing, since it will probably bee too long for travis. Would adding a bash script to execute it be enough?

@keryell
Copy link
Member

keryell commented Jan 8, 2018

I have the feeling it is not explained in the documentation (the top-level README?) how to actually run the benchmarks. Only the compilation process is explained.

@Ruyk Ruyk assigned tremmelg and cjdb and unassigned tremmelg Feb 1, 2018
@keryell
Copy link
Member

keryell commented Aug 2, 2018

At least we should check that they are correctly compiled.
This might be related to #82

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants