-
Notifications
You must be signed in to change notification settings - Fork 147
Calculation of Results
For each non-text question its grade is the average of all given answers.
The total grade of a course is the weighted average of all its evaluations' total grades. The weights can be defined for each evaluation individually. The course's total grade will only be calculated if all its evaluations are finished.
Generally, the grade calculation distinguishes between grade questions and non-grade questions. The idea is that grade questions are used as "summary questions" along the lines of "how would you rate the course in total", which is why they have the same weight regardless of how many non-grade questions there are.
The overall grade for an evaluation is the average of three averages: the contributions average, the general grade questions average, and the general non-grade questions average. They are weighted by CONTRIBUTIONS_WEIGHT, GENERAL_GRADE_QUESTIONS_WEIGHT, and GENERAL_NON_GRADE_QUESTIONS_WEIGHT (default for all of them is 1), respectively.
These three grades should rate the contributors, the contents, and the overall feeling the participants had. These three sections have a constant weight, independent of how many answers were given in each of the sections, since, e.g., more contributors don't make the contents less important. (However, this might give individual answers a very high weight if one of those sections receives a very small number of answers, similar to the problem described in #1222. We simply hope this problem does not happen in practice, as these sections usually get a comparable number of answers, compared to individual questions, whose answer count might differ substantially.)
The general [non-]grade questions average is calculated from the general question's grades, weighted by the number of answers for each question. #1222 explains why this weight is important.
The contributions average is calculated from one grade for each contributor, weighted by the maximum number of answers this contributor got for any of their questions.
This weight is an approximation for "how many voters did evaluate this person" (which we cannot determine exactly), which in turn is an approximation for "how important/impactful was that person". This might not be a good approximation, but it's the best we came up with.
The grade of each contributor is the average of two averages: one for grade questions, and one for non-grade questions. They are weighted by CONTRIBUTOR_GRADE_QUESTIONS_WEIGHT, default: 4) and non-grade questions (CONTRIBUTOR_NON_GRADE_RATING_QUESTIONS_WEIGHT, default: 6), respectively, and calculated analogously to the general [non-]grade questions average.
The grade of each contributor, although calculated as an intermediate result, cannot be seen in the UI to not give grades to individuals, which felt weird to us, and to not encourage comparisons between contributors.