Issue #625: solve aggregation issue for test.join with probs

[giuseppec]

Try to solve #625

[larskotthoff]

Could you add the example from the issue report as a test please?

[giuseppec]

Done.

Note: For repeated-CV test.join currently computes the measure for each repeated cv separately and then averages this through the mean. For example, a 3 times repeated 10-fold CV creates cross-validated predictions for each repetition, e.g. y1, y2, and y3. In this case, currently test.join computes the measure for y1 and the real y, y2 and the real y, y3 and the real y and finally aggregates the three values using the mean. (why aggregating here? especially why using the mean?).
An alternative to this (without the issue of aggregating) would be to average the cross-validated predictions accross all reps, i.e., use the mean ymean = (y1+y2+y3)/3 and compute the measure using ymean and the real y.

[larskotthoff]

Hmm, sounds to me like this should be configurable by the user (in particular how the aggregation is done). This is probably a separate PR though?

[giuseppec]

Should we first merge this PR that fixes a bug? And then open up a new issue for that as this is rather an enhancement and no bug.

[larskotthoff]

Sounds good. I'll merge this now.