Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the avg_problem_grader just be the weighted grader. #1160

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

drgrice1
Copy link
Member

@drgrice1 drgrice1 commented Dec 9, 2024

The weighted grader really is just a more advanced version of the average problem grader that allows weights and "credit". If weights are not assigned then a weight of 1 is assumed for each answer. So if no weights are assigned then the weighted grader returns exactly the same thing as the previous average problem grader.

The weightedGrader.pl macro should be considered deprecated. The WEIGHTED_ANS, NAMED_WEIGHTED_ANS, and CREDIT_ANS methods should no longer be used (although they will still work even with the new average problem grader). Instead of calling WEIGHTED_ANS or NAMED_WEIGHTED_ANS, pass weight => n to the cmp method to assign a weight to an answer. Instead of calling CREDIT_ANS pass credit => $answer1 or credit => [ $answer1, $answer2, ... ] to the cmp method. That is effectively what those methods do anyway. Note that the other answers need to be assigned a name using the NEW_ANS_NAME method.

Note that if the weightedGrader.pl macro is loaded and install_weighted_grader is not called, then using the WEIGHTED_ANS, NAMED_WEIGHTED_ANS, and CREDIT_ANS methods will work with this since the implementation is completely compatible with the weighted_grader defined in the macro. Also if the macro is loaded and install_weighted_grader is called, then the macro will continue to work as before. The options can either be set using the macro method, or as described in the previous paragraph (except the credit option must be an array reference with the macro weighted grader).

There is one essential difference in this implementation from the previous weighted grader when the credit option is used. The previous weighted grader would mark optional answers correct, but not change their scores. This results in the feedback for those answers showing them as incorrect and the message shown above the problem stating that not all answers are correct, even though the overall problem score for the attempt is reported as 100%. The documentation in the weightedGrader.pl macro states that "When credit IS given, the blank answer is still marked as incorrect in the grey answer report at the top of the page, but the student gets awarded the points for that answer anyway (with no other indication). It is possible to cause the blank to be marked as correct, but this seemed confusing to the students." However, perhaps due to changes in feedback, it actually seemed much more confusing to me for the answers to be marked incorrect and have the red incorrect feedback and message above stating not all answers correct with the message way below that is often ignored by students stating that the score for the attempt is 100%. So this does what is suggested in the documentation and actually changes the scores for the optional answers. Furthermore, a message is added to those answers stating, "This answer was marked correct because the primary answer is correct." I am not sold on that wording, but this seems much clearer to me. Another option would be to not set the scores of the optional parts, but set the $problem_result{msg} informing the user what is going on. That message is displayed immediately below the problem as "Note: $problem_result{msg}". Since that is closer to the problem and not way down below all of the submit buttons that might be enough?

Both avg_problem_grader methods lib/WeBWorK/PG/Translator.pm and macros/core/PGanswermacros.pl were updated. However, only the one in macros/core/PGanswermacros.pl is actually used.

@drgrice1 drgrice1 force-pushed the weighted-avg-problem-grader branch 3 times, most recently from dfc84cc to 69e6c4b Compare December 11, 2024 13:23
@drgrice1 drgrice1 force-pushed the weighted-avg-problem-grader branch 2 times, most recently from 3709f59 to 7e7e429 Compare December 18, 2024 14:25
@drgrice1 drgrice1 force-pushed the weighted-avg-problem-grader branch from 7e7e429 to 82490d2 Compare January 14, 2025 22:18
The weighted grader really is just a more advanced version of the
average problem grader that allows weights and "credit". If weights are
not assigned then a weight of 1 is assumed for each answer. So if no
weights are assigned then the weighted grader returns exactly the same
thing as the previous average problem grader.

The `weightedGrader.pl` macro should be considered deprecated. The
`WEIGHTED_ANS`, `NAMED_WEIGHTED_ANS`, and `CREDIT_ANS` methods should no
longer be used (although they will still work even with the new average
problem grader). Instead of calling `WEIGHTED_ANS` or
`NAMED_WEIGHTED_ANS`, pass `weight => n` to the `cmp` method to assign a
weight to an answer. Instead of calling `CREDIT_ANS` pass `credit => $answer1`
or `credit => [ $answer1, $answer2, ... ]` to the `cmp` method. That is
effectively what those methods do anyway. Note that the other answers
need to be assigned a name using the `NEW_ANS_NAME` method.

Note that if the `weightedGrader.pl` macro is loaded and
`install_weighted_grader` is not called, then using the `WEIGHTED_ANS`,
`NAMED_WEIGHTED_ANS`, and `CREDIT_ANS` methods will work with this since
the implementation is completely compatible with the `weighted_grader`
defined in the macro. Also if the macro is loaded and
`install_weighted_grader` is called, then the macro will continue to
work as before. The options can either be set using the macro method, or
as described in the previous paragraph (except the `credit` option must
be an array reference with the macro weighted grader).

There is one essential difference in this implementation from the
previous weighted grader when the `credit` option is used. The previous
weighted grader would mark optional answers correct, but not change
their scores. This results in the feedback for those answers showing
them as incorrect and the message shown above the problem stating that
not all answers are correct, even though the overall problem score for
the attempt is reported as 100%. The documentation in the
`weightedGrader.pl` macro states that "When credit IS given, the blank
answer is still marked as incorrect in the grey answer report at the top
of the page, but the student gets awarded the points for that answer
anyway (with no other indication). It is possible to cause the blank to
be marked as correct, but this seemed confusing to the students."
However, perhaps due to changes in feedback, it actually seemed much
more confusing to me for the answers to be marked incorrect and have the
red incorrect feedback and message above stating not all answers correct
with the message way below that is often ignored by students stating
that the score for the attempt is 100%. So this does what is suggested
in the documentation and actually changes the scores for the optional
answers. Furthermore, a message is added to those answers stating, "This
answer was marked correct because the primary answer is correct." I am
not sold on that wording, but this seems much clearer to me. Another
option would be to not set the scores of the optional parts, but set the
`$problem_result{msg}` informing the user what is going on. That message
is displayed immediately below the problem as "Note: $problem_result{msg}".
Since that is closer to the problem and not way down below all of the
submit buttons that might be enough?
@somiaj
Copy link
Contributor

somiaj commented Jan 15, 2025

Looking this over, and wondering why credit will only apply to optional parts if the answer is blank. To me this would seem odd if a student gets the main answer right, they could then get credit for the previous answers only if the go make them all blank again. I assume this comes from how this option was used before, but my initial thought is credit should be awarded even if the answer is not blank.

@drgrice1
Copy link
Member Author

That is how the weightedGrader.pl macro works. I believe that the idea is that if a student answers a part, then it should be graded for correctness in any case. If a student answers the parts leading up to the credit answer incorrectly, then should that be ignored and the student be given full credit anyway? I don't think so.

Copy link
Contributor

@somiaj somiaj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested everything and works as advertised.

@somiaj
Copy link
Contributor

somiaj commented Jan 15, 2025

I can understand the original setup, but to me if a student can just blank out their answers to get full credit anyways, this could lead to undesirable behavior of students just blanking out answers to see if they can get credit on them. Though this really isn't a feature I use, so I have no strong opinion one way or the other.

I still think you should update the old macro to just install the new average problem grader and remove the grader from the macro.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants