Greater transparency in Ranking?

[edeutsch]

It seems to me that

1. There are many and growing nuances in our results that should affect ranking
2. It is quite difficult to see what kind of decisions the ranker is making

I think remedying item 2 would help us take better account of 1 in order to get us better ranking.

I think it would be extraordinarily useful to be able to see a "ranking report" whereby the ranker divulges all the scraps of information it used to perform its scoring for each result. I'm thinking like a table perhaps of 4 columns:

attribute, value, contributing_weight, comment

This table would then be accumulated for each result and be stored in TRAPI (in a custom field under each Result) and then viewable in the ARAX UI. And ideally also a complete report somewhere that is viewable that is just the sum of all these tabular results.

This would allow us to really see all the contributing factors contributing to the score for each result, how they are weighted.
and I suspect this would easily allow us to:

1. Spot problems where the ranking is going astray
2. Allow us to spot pieces of information that are not being used, when they should.

I feel like the Ranker is very black box that is hard to improve, but improved ranking reporting would help greatly.

What do you think?