Calculation of Results

Calculation of the grades per question

For each non-text question its grade is the average of all given answers.

Calculation of the total grades

Generally, the grade calculation distinguishes between grade questions and non-grade questions. The idea is that grade questions are used as "summary questions" along the lines of "how would you rate the course in total", which is why they have the same weight regardless of how many non-grade questions there are.

The overall grade for a course is the average of three averages: the contributions average, the general grade questions average, and the general non-grade questions average. They are weighted by CONTRIBUTIONS_WEIGHT, GENERAL_GRADE_QUESTIONS_WEIGHT, and GENERAL_NON_GRADE_QUESTIONS_WEIGHT (default for all of them is 1), respectively.

These three grades should rate the course's contributors, the course's contents, and the overall feeling the participants had. These three sections have a constant weight, independent of how many answers were given in each of the sections, since e.g. more contributors don't make the course's contents less important. (However, this might give individual answers a very high weight if one of those sections receives a very small number of answers, similar to the problem described in #1222. We simply hope this problem does not happen in practice, as these sections usually get a comparable number of answers, compared to individual questions, whose answer count might differ substantially.)

The general [non-]grade questions average is calculated from the course question's grades, weighted by the number of answers for each question. #1222 explains why this weight is important.

The contributions average is calculated from one grade for each contributor, weighted by the maximum number of answers this contributor got for any of their questions.

This weight is an approximation for "how many voters did evaluate this person" (which we cannot determine exactly), which in turn is an approximation for "how important/impactful was that person for this course". This might not be a good approximation, but it's the best we came up with.

The grade of each contributor is the average of two averages: one for grade questions, and one for non-grade questions. They are weighted by CONTRIBUTOR_GRADE_QUESTIONS_WEIGHT, default: 4) and non-grade questions (CONTRIBUTOR_NON_GRADE_RATING_QUESTIONS_WEIGHT, default: 6), respectively, and calculated analogously to the general [non-]grade questions average.

The grade of each contributor, although calculated as an intermediate result, cannot be seen in the UI to not give grades to individuals, which felt weird to us, and to not encourage comparisons between contributors.