chore(bench): improve heuristic to run throughput benchmarks

[soonum]

This changes the way we define the number of elements to load in the throughput pipeline.
Light operations needs more elements to saturate a backend. Conversely a heavy operations will require less elements to be able to run in a decent time.
That improvement will dramatically decrease benchmark total duration.

This has been tested with the following operations:

• bitand (light operation)
• add (mid operation)
• div_rem (heavy operation):

Saturation has been tested and successful on the following backends:

• CPU
• GPU

[soonum]

I need a review of the design before exporting it to all other benchmark functions handling throughput variants.

[IceTDrinker]

what's the idea of the heuristic ?

Load depending on how many PBS are in an operation ?

[soonum]

what's the idea of the heuristic ?

Load depending on how many PBS are in an operation ?

Yes this is the idea. The load is computed based on the number of threads available divided by the number of PBS needed for one operation.
This value is then used a coefficient within the previous implementation. Typically the coefficient has a value greater than 1.0 for quick operations, if the machine is big enough, and a value below 1.0 for slow operations.
That way the number of elements to pass during the throughput benchmark is dynamically generated.

[IceTDrinker]

Yes this is the idea. The load is computed based on the number of threads available divided by the number of PBS needed for one operation. This value is then used a coefficient within the previous implementation. Typically the coefficient has a value greater than 1.0 for quick operations, if the machine is big enough, and a value below 1.0 for slow operations. That way the number of elements to pass during the throughput benchmark is dynamically generated.

Did you check the measured throughput with the new loading factor is similar to the old one ?

[soonum]

Yes I've just checked and they are the same 🎉

[soonum]

get_pbs_count() do not increment the atomic counter on GPU.
Thus I cannot test this new implementation on GPU for now.

[IceTDrinker]

get_pbs_count() do not increment the atomic counter on GPU. Thus I cannot test this new implementation on GPU for now.

use the CPU stats, they should be similar I would think

[soonum]

The PBS count for GPU is fixed. Ready for review now.

[agnesLeroy]

Hey! Thanks a lot @soonum! I only have some minor fixes 🙂 do you know how long the throughput benches take now?

[soonum]

Hey! Thanks a lot @soonum! I only have some minor fixes 🙂 do you know how long the throughput benches take now?

Yes, for Cuda benchmarks, we're now down to 46 mins for de-duplicated operations in 64 bits.
I'll launch a full precision on default ops today.

[IceTDrinker]

I saw I was still supposed to review some thing here ?

Btw @soonum just thought about something but enabling the pbs stats can have an impact on CPU performance as we update a single counter via many threads, so I guess there is a need (only for CPU) to do a first pass to measure PBSes for all ops and precision and relaunch throughput loading those data from file to avoid it having an adverse effect on precision I guess

we must not launch latency pbs with the pbs-stats feature