Behavior of Deployed Test Suite in Conjunction with SCORM Adopters

[andyjohnson]

This came up as an issue on the May 2, 2018 call. Currently, when an update to the deployed LRS Conformance Test (lrstest.adlnet.gov) is made, the status of all LRSs changes from Green to Orange, indicating to vendors that they should re-run the test to ensure conformance. Orange indicates the LRS is 1.0.3 valid to SOME version of the test suite, but conformance to the current version is UNKNOWN.

The group felt uncertainty should be relayed to the Conformant LRSs section of the xAPI Adopters site ( https://adopters.adlnet.gov/products/all/0). There should be "business" processes in place that allow for both reasonable expectations among testers as well as confidence from consumers of the information on the Adopter's site. Updates to the Test Suite tests shouldn't be common (last update prior to this was 6 months before). A 2-3 month window after an update to the Test Suite of applying the "Orange" status to the adopter site as well may be appropriate. As emails were used to register, messaging of these processes and status changes can be sent following releases and with iterative warnings until a status change occurs.

We welcome discussion around the issue.