Bounding box tests skipped due to no data present in dataset

[code-akki]

Describe the bug
Certain tests of type validateFeaturesWithBoundingBoxResponse get skipped since there’s no response returned using our test dataset. The dataset has no data points at certain coordinates, such as

• (-180.00, 85.00, 180.00, 90.00) [this is a polar region],
• (-80.0000000,-5.0000000,-70.0000000,5.0000000),
• (-180.0000000,-90.0000000,180.0000000,-85.0000000) [this is a polar region],
• (177.0000000,65.0000000,-177.0000000,70.0000000),
• (-1.5000000,50.0000000,1.5000000,53.0000000),
• (-180.0000000,85.0000000,180.0000000,90.0000000) [this is polar region],
• (177.0000000,65.0000000,-177.0000000,70.0000000),
• (-80.0000000,-5.0000000,-70.0000000,5.0000000),
• (177.0000000,65.0000000,-177.0000000,70.0000000)

Is there a way to better handle this case instead of skipping?

To Reproduce
Steps to reproduce the behavior:

1. Run the compliance test on the dataset OR
2. Use the URL of our test server to run the compliance test.

Expected behavior
Rather than skipping the tests, is there a better way to handle these special cases.

Screenshots

Additional context
If there are any standard datasets that we can use to run the Features compliance test? We think ldproxy uses a vineyard dataset for its compliance test, is it recommended to use the same? Here are a couple of links for reference -

• https://github.com/interactive-instruments/ldproxy/tree/master/demo/vineyards
• https://github.com/search?q=repo%3Aopengeospatial%2Fets-ogcapi-features10%20vineyards&type=code
• https://demo.ldproxy.net/vineyards/collections

[cportele]

OGC CITE decided to move away from test datasets so that the CITE tests can be executed against any API.

That said, in CQL2 we use an adapted Natural Earth dataset to specify additional conformance tests for an API that publishes that dataset and that supports Features Part 3. See https://docs.ogc.org/DRAFTS/21-065r1.html#ats.

ldproxy mainly uses the vineyards dataset as an example and for demonstration purposes (although it is also used in internal tests). The dataset does not cover all aspects specified in Features Part 1, so it would not be an ideal reference dataset, at least not without additional reference datasets.

[dstenger]

Thanks for reporting.
What do you mean with a better way of handling the special cases? Skipped tests are considered neutral and thus do not lead to a failed test result.

As @cportele already pointed out, there are no test datasets for this test suite as it should work with all datasets.

Short background: Those bounding boxes come from the ATS when the standard was still called WFS 3.0. The current ATS does not mention them anymore but I don't see a reason removing them.
I recently created this issue opengeospatial/cite#68 discussing exactly the topic if such tests are required. You all are welcome to participate in the discussion.

[code-akki]

Thank you @dstenger and @cportele for your comments. I'll be happy to join the discussion and I'll close this issue.