-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: center tests around local dev context
- added better setup
- Loading branch information
Showing
2 changed files
with
65 additions
and
24 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,27 +1,36 @@ | ||
Proctored Exams | ||
=============== | ||
|
||
Helpful ReadTheDocs Pages: | ||
|
||
Helpful ReadTheDocs Pages | ||
------------------------- | ||
- `Configuring a Proctoring Provider <https://edx.readthedocs.io/projects/edx-partner-course-staff/en/latest/proctored_exams/proctored_enabling.html#configuring-proctoring-provider>`_ | ||
- `Creating a Proctored Exam <https://edx.readthedocs.io/projects/edx-partner-course-staff/en/latest/proctored_exams/pt_create.html#creating-a-proctored-exam>`_ | ||
|
||
Prerequisite steps: | ||
#. Use Google Chrome | ||
#. Have a staff account | ||
#. Have a non-staff learner account enrolled in the verified track for this course | ||
#. As your non-staff user in your current window, and enroll in the course where you have set up your proctored exam | ||
#. In an incognito window, login to your staff account and visit your Support Tools page and search in the email for your non-staff user. You should now see the option to change the enrollment track of that user. Change the enrollment track of the user to ""verified"" (choose any reason when prompted, it doesn't matter) | ||
#. In that same incognito, still signed in as staff, navigate to the Instructor dashboard | ||
#. Find or create a test course within Stage with proctoring enabled and an LTI proctoring provider chosen, certificates are enabled, and where the exams IDA waffle flag is enabled | ||
#. The exams IDA waffle flag can be found/created in the LMS' Django Admin under "Waffle Flag Course Overrides" | ||
#. If needed, you can make a new entry using "course_apps.exams_ida" as the waffle flag, the course ID, and by setting it to "Enabled" | ||
#. Find or create two exam subsections with a type of 'Proctored exam' in your test course | ||
#. This can be done by creating a subsection and clicking the 'Configure' button (should look like a gear), going to the "Advanced Tab", and selecting "Proctored" | ||
#. You MUST also set this subsection to be Graded as a Midterm or Final exam | ||
#. This subsection should contain at least one unit with one gradeable block, e.g. a generic multiple choice question in order to test downstream effects to the gradebook | ||
#. You should also have a tab open with your event bus logs | ||
|
||
Proctoring Set Up Steps | ||
----------------------- | ||
- As of November 2023, there is a course available for testing in edX's staging environment at https://learning.stage.edx.org/course/course-v1:edx+PROC100+1T2023/home. You should be able to test using this course so long as you have access to both an instructor and learner account. | ||
- Otherwise, if you are developing on Open edX locally, then you should follow the instructions below | ||
#. Find or create a test course that has proctoring enabled and an LTI proctoring provider chosen, where a verified track exists, and where the exams IDA waffle flag is enabled | ||
- The exams IDA waffle flag can be found/created in the LMS' Django Admin under "Waffle Flag Course Overrides" | ||
- If needed, you can make a new entry using "course_apps.exams_ida" as the waffle flag, the course ID, and by setting it to "Enabled" | ||
#. Find or create two exam subsections with a type of 'Proctored exam' in your test course | ||
- This can be done by creating a subsection and clicking the 'Configure' button (should look like a gear), going to the "Advanced Tab", and selecting "Proctored" | ||
- You MUST also set this subsection to be Graded as a Midterm or Final exam | ||
- This subsection should contain at least one unit with one gradeable block, e.g. a generic multiple choice question in order to test downstream effects to the gradebook | ||
|
||
User Set Up Steps | ||
----------------- | ||
#. Use Google Chrome as your browser for these tests | ||
#. In your platform, log in with a non-staff learner account enrolled in the verified track for this course | ||
- The default credentials for local development are ([email protected] Password=edx) | ||
#. As your non-staff user in your current window, and enroll in the course where you have set up your proctored exam | ||
#. In an incognito window, login into an account with staff or instructor level permissions | ||
- The default credentials for local development are ([email protected] Password=edx) | ||
#. Visit your Support Tools page for managing enrollments. Enter the email for your non-staff user in the search bar. You should now see the option to change the enrollment track of that user. Change the enrollment track of the user to "verified" (choose any reason if prompted, it doesn't matter) | ||
- Default local URL for changing enrollments is http://localhost:18000/support/enrollment | ||
#. In that same incognito window, while still signed into your staff or instructor level account, navigate to the exams dashboard | ||
- This can be found by going to the course view for the course you are testing (the one with all the checkboxes and collapsable course sections), then navigating to the "Instructor" tab to get to the Instructor Dashboard, then navigating to the "Special Exams" tab within the Instructor dashboard. | ||
|
||
Proctored Exam Flow | ||
=================== | ||
|
@@ -46,10 +55,12 @@ Can't Start Exam Without Proctoring Software Running | |
|
||
Exam Flow Works With Proctoring Software Installed & Running | ||
------------------------------------------------------------ | ||
#. Go through the setup again, this time without closing your proctoring software. Clicking "Start Exam" should start the exam | ||
#. Go through the steps 1-4 in the "Can't Start Exam Without Proctoring Software Running" section again | ||
#. This time without closing your proctoring software, clicking "Start Exam". You should be brought to your exam, with a timer running above the exam questions. | ||
#. Complete whatever question you put inside your exam w/ the correct answer. Clicking "End My Exam" should direct you to a "Are you sure you want to end your proctored exam?" interstitial | ||
#. Clicking "No I'd like to continue working" should send you back to the exam | ||
#. Clicking "Yes end my proctored exam" should end the exam and directs you to a "You have submitted this proctored exam for review" page | ||
#. Clicking "Yes end my proctored exam" should also close your proctoring software automatically. | ||
#. In the course view, the exam you took should be marked as completed (with a green checkmark) | ||
|
||
|
||
|
@@ -73,6 +84,8 @@ NOTE: These instructions currently do not account for downstream effects involvi | |
|
||
Dashboard UI Loads Exam Attempts | ||
-------------------------------- | ||
INSTRUCTIONS: In an incognito window, log in to a staff or instructor account Go to the course view for the course you are testing (the one with all the checkboxes and collapsable course sections), then navigating to the "Instructor" tab to get to the Instructor Dashboard, then navigating to the "Special Exams" tab within the Instructor dashboard. | ||
|
||
#. Clicking on the "Special Exams" tab should load the *new* version of the exams dashboard | ||
#. Clicking on the "Review Dashboard" tab within the exams dashboard should open a dashboard owned by your selected procotoring software (e.g. a Proctortrack or Proctorio dashboard for reviewing exam attempt details) | ||
#. Clicking the "Select An Exam" dropdown should show the list of exams for the course | ||
|
@@ -83,16 +96,16 @@ Dashboard UI Loads Exam Attempts | |
#. Clicking the "Review Dashboard" link in the modal should load the review dashboard for your chosen Proctoring Software (e.g. Proctortrack or Proctorio) | ||
|
||
Can Reject/Verify "Review Required" attempts | ||
--------------------------------------------------- | ||
-------------------------------------------- | ||
INSTRUCTIONS: Go into another exam and be suspiscious. Block your webcam, leave for a few seconds, make strange noises, open a bunch of tabs, watch some youtube, open wikipedia, ask chatgpt for the answer to life, etc. Note you will need to do this a couple of times | ||
|
||
#. Submit your suspiscious exam attempt, and check the exams dashboard. Assuming you were suspiscious enough, you should see a row that has a status of "Review Required" AND has a "Review Required" button for your attempt | ||
#. NOTE: This exam attempt may appear with the status "Satisfied" if you are using Proctorio. This means Proctorio's system is still analyzing your attempt. If this happens, just wait a bit and refresh the page until the status changes to "Verified" | ||
#. Clicking the "Review Required" button should spawn a modal that says "Update review status" | ||
#. Clicking "Cancel" should close the modal | ||
#. Clicking "Verify" should mark attempt as verified both in the dashboard | ||
#. Clicking "Verify" should mark attempt as verified the dashboard | ||
#. Check the Gradebook (Instructor Dashboard -> Student Admin -> Gradebook) and check that the grade for the exam is 100 | ||
#. Clicking "Reject" should mark attempt as reject both in the dashboard | ||
#. Clicking "Reject" should mark attempt as reject the dashboard | ||
#. Check the Gradebook (Instructor Dashboard -> Student Admin -> Gradebook) and check that the grade for the exam is 0 | ||
|
||
Can Review Verified Attempts | ||
|
@@ -104,17 +117,15 @@ INSTRUCTIONS: Go into another exam and don't be suspiscious. Keep your webcam on | |
#. A modal should pop up upon clicking "Manual Review" that contains "Cancel" and "Reject" Buttons | ||
#. Clicking "Cancel" should close the modal | ||
#. Clicking "Reject" should change the attempt status to "rejected" in the UI | ||
#. Upon refreshing a filtering for this exam attempt, it is still marked as "rejected" | ||
#. Check the Gradebook (Instructor Dashboard -> Student Admin -> Gradebook) and check that the grade for the exam is 0 | ||
#. Check the Generated Certificates table in the LMS admin at "{your platform url}/admin/certificates/generatedcertificate/" to see if the user's certificate for the course has been marked as "invalidated" | ||
|
||
Can Review Rejected Attempts | ||
---------------------------- | ||
#. The exam attempt that you just marked as "Rejected" should have a button at the end of their rows saying "Manual Review" | ||
#. A modal pops up upon clickin "Manual Review" that only should ha the "Cancel" and "Verify" Buttons | ||
#. A modal pops up upon clicking "Manual Review" that should only have the "Cancel" and "Verify" Buttons | ||
#. Clicking "Cancel" should close the modal | ||
#. Clicking "Verify" should change the attempt status to "verified" in the UI | ||
#. Upon refreshing the page, the same attempt should still be marked as "verified" | ||
#. Check the Gradebook (Instructor Dashboard -> Student Admin -> Gradebook) and check that the grade for the exam is 100 | ||
|
||
|
||
|