Skip to content

Commit

Permalink
Performance problem deleting users from moodle system
Browse files Browse the repository at this point in the history
It was observed, that deleting users took a very long time. The
observed time ranges were 5-14 hours. The problem was brought to our
attention because it caused other adhoc task to pile up behind it.

Checking the mysql processlist consistently showed the same queries and
these could be tracked down into the mod_studentquiz dataprivacy
integration.

Running the SQLs manually against our staging system with a mostly
idleing MariaDB showed the same behavior (the manual tests were
interrupted after 1/2 hour).

To give an insight into the order of magnitude of our DB, here are the 
numbers for the affected tables:

mdl_context	1726200
mdl_studentquiz	132
mdl_question_categories	89541
mdl_question_bank_entries	690319
mdl_question_versions	748974
mdl_question	748974
mdl_question_references	168766
mdl_studentquiz_question	1227
mdl_studentquiz_rate	16953
mdl_studentquiz_comment	931
mdl_studentquiz_progress	19742
mdl_studentquiz_attempt	6468
mdl_studentquiz_comment_history	971
mdl_studentquiz_notification	335
mdl_studentquiz_state_history	2391

Looking at the SQL queries and the postprocessing code, it was observed
that there are left joins, that could directly be eliminated because
they can't yield usable results anyway.

These leafs are:

- commenthistory
- progress
- attempt
- notificationjoin
- statehistory

Instead of first doing the join and then filter for the targetuser
via where this change moves filtering for the targetuser into the
join.
  • Loading branch information
matthiasblaesing committed Oct 18, 2024
1 parent 83c1a4f commit 395a189
Showing 1 changed file with 20 additions and 4 deletions.
24 changes: 20 additions & 4 deletions classes/privacy/provider.php
Original file line number Diff line number Diff line change
Expand Up @@ -162,12 +162,15 @@ public static function get_contexts_for_userid(int $userid): contextlist {
LEFT JOIN {studentquiz_rate} rate ON rate.studentquizquestionid = question.id
LEFT JOIN {studentquiz_comment} comment ON comment.studentquizquestionid = question.id
LEFT JOIN {studentquiz_progress} progress ON progress.studentquizquestionid = question.id
AND progress.studentquizid = sq.id
AND progress.studentquizid = sq.id AND progress.userid = :progressuser2
LEFT JOIN {studentquiz_attempt} attempt ON attempt.categoryid = ca.id
AND attempt.studentquizid = sq.id
AND attempt.studentquizid = sq.id AND attempt.userid = :attemptuser2
LEFT JOIN {studentquiz_comment_history} commenthistory ON commenthistory.commentid = comment.id
AND commenthistory.userid = :commenthistoryuser2
LEFT JOIN {studentquiz_notification} notificationjoin ON notificationjoin.studentquizid = sq.id
AND notificationjoin.recipientid = :notificationuser2
LEFT JOIN {studentquiz_state_history} statehistory ON statehistory.studentquizquestionid = question.id
AND statehistory.userid = :statehistoryuser2
WHERE (
question.id IS NOT NULL
OR rate.id IS NOT NULL
Expand Down Expand Up @@ -196,10 +199,15 @@ public static function get_contexts_for_userid(int $userid): contextlist {
'rateuser' => $userid,
'commentuser' => $userid,
'progressuser' => $userid,
'progressuser2' => $userid,
'attemptuser' => $userid,
'attemptuser2' => $userid,
'commenthistoryuser' => $userid,
'commenthistoryuser2' => $userid,
'notificationuser' => $userid,
'notificationuser2' => $userid,
'statehistoryuser' => $userid,
'statehistoryuser2' => $userid,
];

$contextlist->add_from_sql($sql, $params);
Expand Down Expand Up @@ -266,12 +274,15 @@ public static function export_user_data(approved_contextlist $contextlist) {
LEFT JOIN {studentquiz_rate} rate ON rate.studentquizquestionid = question.id
LEFT JOIN {studentquiz_comment} comment ON comment.studentquizquestionid = question.id
LEFT JOIN {studentquiz_comment_history} commenthistory ON commenthistory.commentid = comment.id
AND commenthistory.userid = :commenthistoryuser2
LEFT JOIN {studentquiz_progress} progress ON progress.studentquizquestionid = question.id
AND progress.studentquizid = sq.id
AND progress.studentquizid = sq.id AND progress.userid = :progressuser2
LEFT JOIN {studentquiz_attempt} attempt ON attempt.categoryid = ca.id
AND attempt.studentquizid = sq.id
AND attempt.studentquizid = sq.id AND attempt.userid = :attemptuser2
LEFT JOIN {studentquiz_notification} notificationjoin ON notificationjoin.studentquizid = sq.id
AND notificationjoin.recipientid = :notificationuser2
LEFT JOIN {studentquiz_state_history} statehistory ON statehistory.studentquizquestionid = question.id
AND statehistory.userid = :statehistoryuser2
WHERE (
question.id IS NOT NULL
OR rate.id IS NOT NULL
Expand Down Expand Up @@ -303,10 +314,15 @@ public static function export_user_data(approved_contextlist $contextlist) {
'rateuser' => $userid,
'commentuser' => $userid,
'progressuser' => $userid,
'progressuser2' => $userid,
'attemptuser' => $userid,
'attemptuser2' => $userid,
'commenthistoryuser' => $userid,
'commenthistoryuser2' => $userid,
'notificationuser' => $userid,
'notificationuser2' => $userid,
'statehistoryuser' => $userid,
'statehistoryuser2' => $userid,
];
$params += $contextparam;

Expand Down

0 comments on commit 395a189

Please sign in to comment.