Skip to content

Commit

Permalink
docs: fixed headers + error/reset info
Browse files Browse the repository at this point in the history
  • Loading branch information
ilee2u committed Nov 6, 2023
1 parent d1455d8 commit 3698f55
Showing 1 changed file with 27 additions and 28 deletions.
55 changes: 27 additions & 28 deletions docs/test_plan/proctored_exam.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ Prerequisite steps:


Proctored Exam Flow
-------------------
===================
Expected Behavior: "A paid track learner is able to start, complete, and submit a proctored exam using Proctoring Software that is launched via LTI."

Can't Start Exam Without Proctoring Software Installed
======================================================
------------------------------------------------------
#. Enter a Proctored Exam without proctoring software installed. You should see an interstitial stating that "This exam is proctored" with a button that says "Continue to Proctored Exam"
#. Clicking "Continue to Proctored Exam" should bring you to an interstitial titled "Set up and start your proctored exam." with "Start System Check" and "Start Exam" buttons
#. Clicking the "Start Exam" button should pop up a modal that says "Cannot Start Exam"
Expand All @@ -37,15 +37,15 @@ Can't Start Exam Without Proctoring Software Installed
#. Navigating back one page and clicking "Start Exam" should still block you from starting the exam

Can't Start Exam Without Proctoring Software Running
====================================================
----------------------------------------------------
#. Install the proctoring software that you've configured with your platform. Go through the opening interstitials and click "Start System Check". This should load your respective Proctoring Software Setup page
#. Immediately navigating back and trying to click "Start Exam" should't allow you to start the exam
#. Navigating back to your proctoring software setup page and refreshing on that page should cause no issues
#. Go through your proctoring software setup for your Webcam, Desktop, etc. Upon setup completion, you should be redirected back to an exam interstitial with just the "Start Exam" button
#. Close your proctoring software and click "Start Exam". The exam should not start

Exam Flow Works With Proctoring Software Installed & Running
============================================================
------------------------------------------------------------
#. Go through the setup again, this time without closing your proctoring software. Clicking "Start Exam" should start the exam
#. Complete whatever question you put inside your exam w/ the correct answer. Clicking "End My Exam" should direct you to a "Are you sure you want to end your proctored exam?" interstitial
#. Clicking "No I'd like to continue working" should send you back to the exam
Expand All @@ -54,11 +54,11 @@ Exam Flow Works With Proctoring Software Installed & Running


Proctorio-Specific Tests
------------------------
========================
These tests are for those using Proctorio as their proctoring provider

Exam Flow Works as Expected
===========================
---------------------------
#. With Proctorio not installed, clicking "Start System Check" redirects you to https://getproctorio.com/
#. Go through the Proctorio Setup for your Webcam, Desktop, etc. Upon setup completion, your webcam should appear to the right of the screen along with some controls.
#. On the "Return to exam" page, DO NOT click on the "Sending you back to your exam" page. Wait instead, and you should be sent to the "Important"/"Start Exam" interstitial automatically
Expand All @@ -67,12 +67,12 @@ Exam Flow Works as Expected


Exams Dashboard
---------------
===============
Expected Behavior: Calls to the ACS Endpoint work, instructors can review/reset exam attempts, and expected downstream effects (e.g. w/ completion, credits, certificates, etc) of modifying an exam attempt's status occur
NOTE: These instructions currently do not account for downstream effects involving a learner's credit requirement status. We currently have no plan to include these

Dashboard UI
============
Dashboard UI Loads Exam Attempts
--------------------------------
#. Clicking on the "Special Exams" tab should load the *new* version of the exams dashboard
#. Clicking on the "Review Dashboard" tab within the exams dashboard should open a dashboard owned by your selected procotoring software (e.g. a Proctortrack or Proctorio dashboard for reviewing exam attempt details)
#. Clicking the "Select An Exam" dropdown should show the list of exams for the course
Expand All @@ -82,10 +82,10 @@ Dashboard UI
#. Filtering by username only should show exam attempts for that username
#. Clicking the "Review Dashboard" link in the modal should load the review dashboard for your chosen Proctoring Software (e.g. Proctortrack or Proctorio)


Reviewing "Second Review Required" attempts
===========================================
Can Reject/Verify "Second Review Required" attempts
---------------------------------------------------
INSTRUCTIONS: Go into another exam and be suspiscious. Block your webcam, leave for a few seconds, make strange noises, open a bunch of tabs, watch some youtube, open wikipedia, ask chatgpt for the answer to life, etc. Note you will need to do this a couple of times

#. Submit your suspiscious exam attempt, and check the exams dashboard. Assuming you were suspiscious enough, you should see a row that has a status of "Second Review Required" AND has a "Review Required" button for your attempt
#. NOTE: This exam attempt may appear with the status "Satisfied" if you are using Proctorio. This means Proctorio's system is still analyzing your attempt. If this happens, just wait a bit and refresh the page until the status changes to "Verified"
#. Clicking the "Review Required" button should spawn a modal that says "Update review status"
Expand All @@ -95,9 +95,10 @@ INSTRUCTIONS: Go into another exam and be suspiscious. Block your webcam, leave
#. Clicking "Reject" should mark attempt as reject both in the dashboard
#. Check the Gradebook (Instructor Dashboard -> Student Admin -> Gradebook) and check that the grade for the exam is 0

Reviewing Verified Attempts
===========================
Can Review Verified Attempts
----------------------------
INSTRUCTIONS: Go into another exam and don't be suspiscious. Keep your webcam on with your face in frame. Complete the exam

#. An exam attempt with a status of "Verified" should appear and have a button/link at the end of their rows titled "Manual Review"
#. NOTE: This exam attempt may appear with the status "Satisfied" if you are using Proctorio. This means Proctorio's system is still analyzing your attempt. If this happens, just wait a bit and refresh the page until the status changes to "Verified"
#. A modal should pop up upon clicking "Manual Review" that contains "Cancel" and "Reject" Buttons
Expand All @@ -107,28 +108,26 @@ INSTRUCTIONS: Go into another exam and don't be suspiscious. Keep your webcam on
#. Check the Gradebook (Instructor Dashboard -> Student Admin -> Gradebook) and check that the grade for the exam is 0
#. Check the Generated Certificates table in the LMS admin at "{your platform url}/admin/certificates/generatedcertificate/" to see if the user's certificate for the course has been marked as "invalidated"

Reviewing Rejected Attempts
===========================
Can Review Rejected Attempts
----------------------------
#. The exam attempt that you just marked as "Rejected" should have a button at the end of their rows saying "Manual Review"
#. A modal pops up upon clickin "Manual Review" that only should ha the "Cancel" and "Verify" Buttons
#. Clicking "Cancel" should close the modal
#. Clicking "Verify" should change the attempt status to "verified" in the UI
#. Upon refreshing the page, the same attempt should still be marked as "verified"
#. Check the Gradebook (Instructor Dashboard -> Student Admin -> Gradebook) and check that the grade for the exam is 100

Reviewing "Errored" attempts
============================
INSTRUCTIONS: In a separate exam, start the exam, then open devtools using Cmd+Option+I on Mac (F12 on Windows). This should error out the exam and load a "Error with proctored exam" page
#. If you are using Proctorio, this should open the "Proctorio Support" page in another tab
#. Clicking the "Review Required" button should pop up a modal that says "Update review status"
#. Clicking "Cancel" should close the modal
#. Clicking "Verify" should mark the attempt as verified both in the dashboard and in the edx-exams admin
#. Clicking "Reject" should mark the attempt as reject both in the dashboard and in the edx-exams admin

Resetting an exam attempt
=========================
#. Clicking "Reset" on an exam attempt should bring up a confirmation modal
Can Reset Errored Exam Attempts
===============================
INSTRUCTIONS: Using Proctorio, start a new exam in your learner account, then open Chrome devtools using Cmd+Option+I on Mac (F12 on Windows).

#. If you don't happen to be using Proctorio, you will need to find a way to error out your exam and test that the expected behavior occurs.
#. The exam should error out the exam and load a "Error with proctored exam" page
#. If you are using Proctorio, this should open the "Proctorio Support" page in another tab
#. In your admin account, go to the exams dashboard. The respective exam attempt should appear in the exams dashboard with the status "Error"
#. Clicking the "Reset" link/button on the exam attempt should bring up a confirmation modal
#. Clicking "No (Cancel)" should close the modal
#. Clicking "Yes I'm Sure" should remove the exam attempt from the table (effectively confirming that it's been reset)
#. Clicking "Yes I'm Sure" should remove the exam attempt from the table (confirming that it's been reset)
#. The learner's completion status for the exam should be reset (i.e. the green checkmark is now gone)
#. As a learner, try taking the exam again. You should be brought to the initial interstitials, as though this were your first time taking the exam.

0 comments on commit 3698f55

Please sign in to comment.