Skip to content

Commit

Permalink
Site checkin for project TEAM Engine
Browse files Browse the repository at this point in the history
  • Loading branch information
Jenkins committed Feb 14, 2024
1 parent b660e05 commit 7dd3c5e
Show file tree
Hide file tree
Showing 12 changed files with 1,114 additions and 1,240 deletions.
133 changes: 69 additions & 64 deletions ctl.html
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
| Generated by Apache Maven Doxia Site Renderer 1.7.4 at 2023-12-18
| Generated by Apache Maven Doxia Site Renderer 1.11.1 at 2024-02-14
| Rendered using Apache Maven Fluido Skin 1.6
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="Date-Revision-yyyymmdd" content="20231218" />
<meta name="Date-Revision-yyyymmdd" content="20240214" />
<meta http-equiv="Content-Language" content="en" />
<title>TEAM Engine &#x2013; CTL scripts</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.6.min.css" />
Expand All @@ -25,9 +25,9 @@

<div id="breadcrumbs">
<ul class="breadcrumb">
<li id="publishDate">Last Published: 2023-12-18<span class="divider">|</span>
<li id="publishDate">Last Published: 2024-02-14<span class="divider">|</span>
</li>
<li id="projectVersion">Version: 5.7</li>
<li id="projectVersion">Version: 6.0.0-RC1</li>
<li class="pull-right"><span class="divider">|</span>
<a href="https://github.com/opengeospatial/teamengine/issues" class="externalLink" title="Issues">Issues</a></li>
<li class="pull-right"><a href="https://github.com/opengeospatial/teamengine/wiki" class="externalLink" title="Wiki">Wiki</a></li>
Expand Down Expand Up @@ -65,98 +65,103 @@
</div>
<div id="bodyColumn" class="span9" >
<h1>CTL scripts</h1>
<p>The OGC test harness is capable of running tests implemented using the CTL (<a class="externalLink" href="http://portal.opengeospatial.org/files/?artifact_id=33085">Compliance Test Language</a>) scripting language developed by the OGC. The CTL grammar can be regarded as a kind of XML-based domain-specific language for defining test suites. A test suite is processed to produce XSLT 2.0 templates that are then executed by the Saxon 9.0 XSLT processor as shown in the following figure.</p>
<p>The OGC test harness is capable of running tests implemented using the CTL
(<a class="externalLink" href="http://portal.opengeospatial.org/files/?artifact_id=33085">Compliance Test Language</a>)
scripting language developed by the OGC. The CTL grammar can be regarded as a
kind of XML-based domain-specific language for defining test suites. A test
suite is processed to produce XSLT 2.0 templates that are then executed by the
Saxon 9.0 XSLT processor as shown in the following figure.</p>
<p><b>Figure 1:</b> Executing CTL scripts</p>
<p><img src="images/ctl-execution.png" alt="Executing CTL scripts" /></p>
<p>The teamengine-core-\${project.version}-base.zip archive contains a sample test suite in <tt>scripts/note/1.0/ctl/note.ctl</tt>. The OGC compliance testing program also maintains a set of test suites that are publicly available from GitHub at this location: <a class="externalLink" href="https://github.com/opengeospatial">https://github.com/opengeospatial</a>.</p>
<div class="section">
<p>The teamengine-core-${project.version}-base.zip archive contains a sample test
suite in <code>scripts/note/1.0/ctl/note.ctl</code>. The OGC compliance testing program
also maintains a set of test suites that are publicly available from GitHub at
this location: <a class="externalLink" href="https://github.com/opengeospatial">https://github.com/opengeospatial</a>.</p><section>
<h2><a name="Grouping_tests"></a>Grouping tests</h2>
<p>If one ctl:code element is used to execute multiple tests, failures in subtests can lead to confusing test results.</p>
<p>For example, this piece of ctl code:</p>

<div class="source">
<div class="source"><pre class="prettyprint">&lt;ctl:test name=&quot;Test&quot;&gt;
&lt;ctl:code&gt;
&lt;ctl:call-test name=&quot;Subtest&quot;&gt;
&lt;/ctl:call-test&gt;
&lt;xsl:if test=&quot;assertion 1&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 1.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;

&lt;ctl:call-test name=&quot;Subtest&quot;&gt;
&lt;/ctl:call-test&gt;
&lt;xsl:if test=&quot;assertion 2&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 2.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;/ctl:code&gt;
<div class="source"><pre class="prettyprint"><code>&lt;ctl:test name=&quot;Test&quot;&gt;
&lt;ctl:code&gt;
&lt;ctl:call-test name=&quot;Subtest&quot;&gt;
&lt;/ctl:call-test&gt;
&lt;xsl:if test=&quot;assertion 1&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 1.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;

&lt;ctl:call-test name=&quot;Subtest&quot;&gt;
&lt;/ctl:call-test&gt;
&lt;xsl:if test=&quot;assertion 2&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 2.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;/ctl:code&gt;
&lt;/ctl:test&gt;

&lt;ctl:test name=&quot;Subtest&quot;&gt;
&lt;ctl:code&gt;
&lt;xsl:if test=&quot;assertion 3&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 3.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;/ctl:code&gt;
&lt;ctl:code&gt;
&lt;xsl:if test=&quot;assertion 3&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 3.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;/ctl:code&gt;
&lt;/ctl:test&gt;
</pre></div></div>
</code></pre></div>
<p>The test Subtest is called twice, e.g. evaluating some common prerequisites for assertion 1 and 2.</p>
<p>If assertion 2 would evaluate to true, thus causing a ctl:fail, the following HTML report would be generated:</p>
<p>If assertion 2 would evaluate to true, thus causing a ctl:fail, the following HTML report would be generated:</p>
<p><img src="images/ctl-subtests-misleading.png" alt="TestNG results summary" /></p>
<p>No information would be given regarding the reason of failure. The ctl:message would only be found in the logs.</p>
<p>This can be confusing for users of the test suite.</p>
<p>As a good practice, we would suggest to create different Subtests that include the assertions of the xsl:if statements.</p>

<div class="source">
<div class="source"><pre class="prettyprint">&lt;ctl:test name=&quot;Test&quot;&gt;
&lt;ctl:code&gt;
&lt;ctl:call-test name=&quot;Subtest 1&quot;&gt;
&lt;/ctl:call-test&gt;

&lt;ctl:call-test name=&quot;Subtest 2&quot;&gt;
&lt;/ctl:call-test&gt;
&lt;/ctl:code&gt;
<div class="source"><pre class="prettyprint"><code>&lt;ctl:test name=&quot;Test&quot;&gt;
&lt;ctl:code&gt;
&lt;ctl:call-test name=&quot;Subtest 1&quot;&gt;
&lt;/ctl:call-test&gt;

&lt;ctl:call-test name=&quot;Subtest 2&quot;&gt;
&lt;/ctl:call-test&gt;
&lt;/ctl:code&gt;
&lt;/ctl:test&gt;

&lt;ctl:test name=&quot;Subtest 1&quot;&gt;
&lt;ctl:code&gt;
&lt;xsl:if test=&quot;assertion 3&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 3.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;xsl:if test=&quot;assertion 1&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 1.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;/ctl:code&gt;
&lt;ctl:code&gt;
&lt;xsl:if test=&quot;assertion 3&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 3.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;xsl:if test=&quot;assertion 1&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 1.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;/ctl:code&gt;
&lt;/ctl:test&gt;

&lt;ctl:test name=&quot;Subtest 2&quot;&gt;
&lt;ctl:code&gt;
&lt;xsl:if test=&quot;assertion 3&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 3.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;xsl:if test=&quot;assertion 2&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 2.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;/ctl:code&gt;
&lt;ctl:code&gt;
&lt;xsl:if test=&quot;assertion 3&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 3.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;xsl:if test=&quot;assertion 2&quot;&gt;
&lt;ctl:message&gt;[FAILURE] 2.&lt;/ctl:message&gt;
&lt;ctl:fail/&gt;
&lt;/xsl:if&gt;
&lt;/ctl:code&gt;
&lt;/ctl:test&gt;
</pre></div></div>
</code></pre></div>
<p>Now, if assertion 2 would evaluate to true, the following HTML report would be generated:</p>
<p><img src="images/ctl-subtests-good-practice.png" alt="TestNG results summary" /></p>
<p>Now the source of the error can be investigated further by clicking on the failed Subtest 2.</p></div>
<p>Now the source of the error can be investigated further by clicking on the failed Subtest 2.</p></section>
</div>
</div>
</div>
<hr/>
<footer>
<div class="container-fluid">
<div class="row-fluid">
<p>Copyright &copy;2023
<p>Copyright &copy;2024
<a href="http://www.opengeospatial.org/">Open Geospatial Consortium</a>.
All rights reserved.</p>
</div>
Expand Down
14 changes: 7 additions & 7 deletions definitions.html
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
| Generated by Apache Maven Doxia Site Renderer 1.7.4 at 2023-12-18
| Generated by Apache Maven Doxia Site Renderer 1.11.1 at 2024-02-14
| Rendered using Apache Maven Fluido Skin 1.6
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="Date-Revision-yyyymmdd" content="20231218" />
<meta name="Date-Revision-yyyymmdd" content="20240214" />
<meta http-equiv="Content-Language" content="en" />
<title>TEAM Engine &#x2013; </title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.6.min.css" />
Expand All @@ -25,9 +25,9 @@

<div id="breadcrumbs">
<ul class="breadcrumb">
<li id="publishDate">Last Published: 2023-12-18<span class="divider">|</span>
<li id="publishDate">Last Published: 2024-02-14<span class="divider">|</span>
</li>
<li id="projectVersion">Version: 5.7</li>
<li id="projectVersion">Version: 6.0.0-RC1</li>
<li class="pull-right"><span class="divider">|</span>
<a href="https://github.com/opengeospatial/teamengine/issues" class="externalLink" title="Issues">Issues</a></li>
<li class="pull-right"><a href="https://github.com/opengeospatial/teamengine/wiki" class="externalLink" title="Wiki">Wiki</a></li>
Expand Down Expand Up @@ -82,7 +82,7 @@ <h3 id="definitions-conformance-classes">Conformance Classes</h3>
conformance class in the which the requirements of a higher level contain all the requirements
of the lower levels (see <a href="#conformance-classes">Conformance classes</a>).</p>
</div>
<div id="conformance-classes" class="imageblock" style="text-align: center">
<div id="conformance-classes" class="imageblock text-center">
<div class="content">
<img src="images/conformance-classes.png" alt="Conformance classes"/>
</div>
Expand Down Expand Up @@ -133,7 +133,7 @@ <h3 id="definitions-profiles">OGC application profile (Profile)</h3>
<p>The DGIWG WMS profile is based on the OGC WMS 1.3 specification, and the NSG WMS profile is in
turn based on the DGIWG profile (<a href="#wms-profiles">WMS profile dependencies</a>).</p>
</div>
<div id="wms-profiles" class="imageblock" style="text-align: center">
<div id="wms-profiles" class="imageblock text-center">
<div class="content">
<img src="images/wms-profiles.png" alt="WMS profiles"/>
</div>
Expand All @@ -149,7 +149,7 @@ <h3 id="definitions-profiles">OGC application profile (Profile)</h3>
<footer>
<div class="container-fluid">
<div class="row-fluid">
<p>Copyright &copy;2023
<p>Copyright &copy;2024
<a href="http://www.opengeospatial.org/">Open Geospatial Consortium</a>.
All rights reserved.</p>
</div>
Expand Down
51 changes: 20 additions & 31 deletions encoding-conformance-classes.html
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
| Generated by Apache Maven Doxia Site Renderer 1.7.4 at 2023-12-18
| Generated by Apache Maven Doxia Site Renderer 1.11.1 at 2024-02-14
| Rendered using Apache Maven Fluido Skin 1.6
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="Date-Revision-yyyymmdd" content="20231218" />
<meta name="Date-Revision-yyyymmdd" content="20240214" />
<meta http-equiv="Content-Language" content="en" />
<title>TEAM Engine &#x2013; Encoding Conformance Classes</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.6.min.css" />
Expand All @@ -25,9 +25,9 @@

<div id="breadcrumbs">
<ul class="breadcrumb">
<li id="publishDate">Last Published: 2023-12-18<span class="divider">|</span>
<li id="publishDate">Last Published: 2024-02-14<span class="divider">|</span>
</li>
<li id="projectVersion">Version: 5.7</li>
<li id="projectVersion">Version: 6.0.0-RC1</li>
<li class="pull-right"><span class="divider">|</span>
<a href="https://github.com/opengeospatial/teamengine/issues" class="externalLink" title="Issues">Issues</a></li>
<li class="pull-right"><a href="https://github.com/opengeospatial/teamengine/wiki" class="externalLink" title="Wiki">Wiki</a></li>
Expand Down Expand Up @@ -66,64 +66,53 @@
<div id="bodyColumn" class="span9" >
<h1>Encoding Conformance Classes</h1>
<p>This page provides the information about how to encode conformance classes in CTL and TestNG tests to better provide information about the grouping of the conformance class and which one is the basic conformance class needed for certification.</p>
<p>The configuration in the tests to represent the conformance class (CC) and basic conformance class in tests are as follows:</p>
<div class="section">
<p>The configuration in the tests to represent the conformance class (CC) and basic conformance class in tests are as follows:</p><section>
<h2><a name="Configuration_in_CTL_test"></a>Configuration in CTL test</h2>
<p>The conformance classes must be configured on the second level of test hierarchy. Example: The starting-test of the suite references the test &#x2018;main&#x2019; (which is the test on first level). The test &#x2018;main&#x2019; executes several tests which should be marked as conformance classes, e.g. &#x2018;cc-one&#x2019; and &#x2018;cc-one&#x2019; (tests on second level). The tests assigned to a second level test may be grouped in hierarchies with undefined depth. Neither the test on the first level nor the tests below the second level can be configured as conformance classes!</p>
<p>To represent the conformance class and basic conformance class we need to add two attributes i.e. <b><tt>isConformanceClass</tt></b> and <b><tt>isBasic</tt></b>.</p>
<div class="section">
<div class="section">
<p>The conformance classes must be configured on the second level of test hierarchy.
Example: The starting-test of the suite references the test &#x2018;main&#x2019; (which is the test on first level). The test &#x2018;main&#x2019; executes several tests which should be marked as conformance classes, e.g. &#x2018;cc-one&#x2019; and &#x2018;cc-one&#x2019; (tests on second level). The tests assigned to a second level test may be grouped in hierarchies with undefined depth.
Neither the test on the first level nor the tests below the second level can be configured as conformance classes!</p>
<p>To represent the conformance class and basic conformance class we need to add two attributes i.e. <b><code>isConformanceClass</code></b> and <b><code>isBasic</code></b>.</p><section><section>
<h4><a name="isConformanceClass:"></a>isConformanceClass:</h4>
<p>By using this attribute we are making a particular test take the role of a conformance class.</p></div>
<div class="section">
<p>By using this attribute we are making a particular test take the role of a conformance class.</p></section><section>
<h4><a name="isBasic:"></a>isBasic:</h4>
<p>By using this attribute we can mark a test as basic conformance class, which is the minimum required for OGC certification.</p>

<ul>

<li>For Example:</li>
</ul>

<div class="source">
<div class="source"><pre class="prettyprint">&lt;ctl:test name=&quot;ex:basic-main&quot; isConformanceClass=&quot;true&quot; isBasic=&quot;true&quot;&gt;
<div class="source"><pre class="prettyprint"><code class="language-xml">&lt;ctl:test name=&quot;ex:basic-main&quot; isConformanceClass=&quot;true&quot; isBasic=&quot;true&quot;&gt;
......
......
&lt;/ctl:test&gt;
</pre></div></div>
</code></pre></div>
<p><b>Note:</b></p>

<ol style="list-style-type: decimal">

<li>For a CC to be basic it needs to have both attributes <i>isConformanceClass=&#x201c;true&#x201d; isBasic=&#x201c;true&#x201d;</i></li>

<li>If the test is a CC but not a basic CC then the of <i>isBasic</i> can be set to false.</li>

<li>If there is no attribute <i>isBasic</i> in a CC, then the value of <i>isBasic</i> is assumed to be false.</li>

<li>If the test is not a CC then there is no need to add any attribute.</li>
</ol></div></div></div>
<div class="section">
</ol></section></section></section><section>
<h2><a name="Configuration_in_TestNG_tests"></a>Configuration in TestNG tests</h2>
<p>TestNG tests already have the notion of conformance classes (CC). Still needs to be configured to represent the basic conformance class. The following needs to be checked and performed:</p>

<ul>
<li>Add a snippet in the config.xml of the TestNG test under the <b><tt>suite</tt></b> element that explicitly states which tests are basic. For example:</li>

<li>Add a snippet in the config.xml of the TestNG test under the <b><code>suite</code></b> element that explicitly states which tests are basic. For example:</li>
</ul>

<div class="source">
<div class="source"><pre class="prettyprint">&lt;BasicConformanceClasses&gt;
<div class="source"><pre class="prettyprint"><code class="language-xml">&lt;BasicConformanceClasses&gt;
&lt;conformanceClass&gt;Basic conformance class name&lt;/conformanceClass&gt;
&lt;/BasicConformanceClasses&gt;
</pre></div></div></div>
</code></pre></div></section>
</div>
</div>
</div>
<hr/>
<footer>
<div class="container-fluid">
<div class="row-fluid">
<p>Copyright &copy;2023
<p>Copyright &copy;2024
<a href="http://www.opengeospatial.org/">Open Geospatial Consortium</a>.
All rights reserved.</p>
</div>
Expand Down
Loading

0 comments on commit 7dd3c5e

Please sign in to comment.