- new 'resolution-testing.xml' keeps track of unit tests
- spec is validated against this test manifest
- it can also be converted into XSpec or other test drivers
Finishing the tests by assigning them to Github Issues?
need a process for this - also Github Issues?
- validate test manifest against its spec - any broken links?
The OSCAL team welcomes feedback on the work in progress in this subdirectory, whether it be questions, points for clarification, critiques or suggestions. A rendered version of the Profile Resolution specification maintained here appears on the OSCAL web site.
Please post Issues in Github or questions to the OSCAL mailing list, or ask about them on our Gitter channel. (See https://pages.nist.gov/OSCAL/contact/ for links.)
The specifications are being edited in an ad-hoc XML back end format, a lightweight extension to HTML tagging. If you wish to edit the files directly, we have stylesheets for presentation and HTML conversion along with Schematron for validation (runtime or at check points); please contact us.
Profile resolution unit tests are maintained here and bound to the specification via markup embedded with the specifications, for convenience in alignment. Formal test scenario files can be generated dynamically from markup when needed.
In this subdirectory, the folder profile-resolution-examples includes old tests kept here for potential future use/reuse while we complete a common test set.
The comment test set we are developing can be found in folder requirement-tests.
-
XML authoring of specifications
-
HTML previews with ToC / requirements index
-
HTML rendering for Hugo / site load
-
Back end validation support for specialized semantics
-
Tagging of requirements integrates with unit tests
-
Cosmetic reformatter rewrites your document with clean tagging and whitespace
-
Unit test set (XSpec) is annotated with keys (links) back to spec document
-
Two-way indexing enables assessment of coverage by unit tests (per marked up requirements)
-
The tests themselves can be illustrated (in/out) using another rendering XSLT
Included in this folder is an oXygen XML Editor project file with settings for many of these operations.
//req shows requirements called out as statements in line.
//eg shows bindings to example documents for purposes of testing.
<eg href="requirement-tests/req-chained-deepA.xml">PENDING chained profiles</eg>
The @href is the file instance for the example. The text is a qualifying description. Prefixing the text with PENDING signals that a test should be deactivated. In this case, as soon as chained profiles are supported, the description can be changed and the test will be run.
Presently all eg are inside req so there is an implicit relation there (no examples are named apart from stated requirements).
Based on markup in the SpecML document, XSpec unit test sheets can be generated framing the unit test held in this subdirectory, to be executed in any (current) XSpec implementation.
Broken out by requirement - corresponding line by line to //req in the Spec document source.
This XSpec is generated by applying the XSLT lib/build-reqs-xspec.xsl to the SpecML source document (marked with req and eg tags).
Grouped by file instead of by requirement (since there is a many-to-many relation), this XSpec may be faster and more efficient to execute.
This XSpec is generated by applying the XSLT lib/build-examples-xspec.xsl to the same SpecML source document.
Additionally handmade XSpec works, such as profile-resolution-queryset.xspec