Graph Times

[pm1113]

Hello,

I was wondering how you are timing the code.

I am consistency getting times that are about 100-1000 times greater than what my individual graph shows me.

Best,
Pranav

[Szypicyn]

To further expand on the above, In the latest series of tests (22/11/16) the reference timings are inconsistent with those obtained by myself. I have seen a case where code hasn't been touched at all, and the graph has shown a substantial speed-up

[jeremych1000]

Also @Szypicyn did your last test produce graphs? I only have the tarball pushed to my repo, but no graphs.

[m8pple]

See #25 about the lack of graphs - there should be new ones based on a fresh
run in a couple of hours.

Regarding the timing, pay careful attention to exactly what the spec says
that the execution time metric is.

I'm (hopefully) measuring that metric - are you sure that is what you are measuring,
or is there other stuff getting included as well?

[jeremych1000]

@m8pple I'm measuring puzzler::Puzzle::Execute by extracting the times from the log, but the times are still nowhere near 10^-5s as in your graphs.

Would it be possible to add in a reference line just so we can know what the speed-up is compared to the reference?

[pm1113]

I used the logs too and the times match the times show in the excel sheet. However the times dont match the times shown in the graphs. I think the axis of the graphs is off. @m8pple, @jeremych1000 could that be the case?

[jeremych1000]

@pm1113 I think that is the case too. If you look in all.log in the tarballs,

Time budget used = 10.295967, remaining = 19.704033
Time budget used = 10.953205, remaining = 19.046795
Time budget used = 11.677849, remaining = 18.322151

Doesn't match too.

[m8pple]

Sorry, hadn't realised this was still active.

I mis-understood the question, I thought this was between people seeing
their results including IO/input generation versus mine that don't (usually
the problem).

I'd only looked at the graphs as a relative measure of performance,
so I didn't realise it was a question about absolute times. I now see
you were meaning the scale, not the actual times.

I think it is simply the scale of the graph is off by 1000, so probably
I'm converting from ns to s incorrectly when plotting the actual
data. The results in the csv look correct though, and if I plot them
I get exactly the same shape as shown in the graphs.

[qc316]

Hello,

@m8pple, Is it possible to push graphs for the final implementations ? We'd like to see how much what we have done today is useful.
Thanks