Specmatic Coverage reporting invalid routes when running on Latest version of Express.

[barath121]

Description
When running specmatic on express v4.20.0 random routes which are not present in app are shown in coverage with remarks as missing in spec it works for v4.19.2.

Steps to reproduce

1. Create an Express app (v4.20.0) with some routes.
2. Run Specmatic and check the coverage report.

Expected behavior
The extra routes should not be displayed

System Information:

• Specmatic version: 2.0.7(Nodejs wrapper)

[pranavgawri]

Hi, @barath121 thanks for sharing this.
We will look into this and get back asap

[barath121]

@pranavgawri Any update on this?

[pranavgawri]

Hi @barath121 ,

Thanks for your patience. We upgraded one of our sample projects to use express server 4.21.0, here is the package.json FYR and we are not able to see any additional routes in our API coverage report. Here is the link to our CI log.

Can you please take a look and let us know if we are missing anything?

[barath121]

@pranavgawri I checked the CI log and after further debugging seems like the the coverage is skipping parts of URL. I cloned your order-bff and added a small code snippet to Product router code.

const testRouter = express.Router();

testRouter.get(
    "/findAvailableProducts",
    validate({ query: productTypeSchema }),
    asyncErrorHandler(async (req, res) => {
      const pageSize = req.get("pageSize");
  
      if (!pageSize) {
        throw ApiError.BadRequest("pageSize is required in Headers");
      }
  
      if (Number.isNaN(Number.parseInt(pageSize))) {
        throw ApiError.BadRequest("pageSize must be a number");
      }
  
      if (req.query.type === "other" || pageSize > 20) {
        throw ApiError.TimeoutError("Product type was other");
      }
  
      const products = await ProductService.searchProducts(req.query.type);
      return res.status(200).json(products);
    })
  );
  
testRouter.post(
    "/products",
    validate({ body: productSchema }),
    asyncErrorHandler(async (req, res) => {
      const newProduct = await ProductService.createProduct(req.body);
      return res.status(201).json(newProduct);
    })
);

router.use("/v1.0/test/:param1/test/:param2/test",testRouter);

This should ideally add 2 routes to the coverage

• /v1.0/test/:param1/test/:param2/test/findAvailableProducts
• /v1.0/test/:param1/test/:param2/test/products

But this is the coverage table:

 |-------------------------------------------------------------------------------------------|
      | SPECMATIC API COVERAGE SUMMARY                                                            |
      |-------------------------------------------------------------------------------------------|
      | coverage | path                        | method | response | #exercised | result          |
      |----------|-----------------------------|--------|----------|------------|-----------------|
      | 100%     | /findAvailableProducts      | GET    | 200      | 2          | covered         |
      |          |                             |        | 400      | 2          | covered         |
      |          |                             |        | 503      | 1          | covered         |
      | 80%      | /orders                     | GET    | 200      | 2          | WIP             |
      |          |                             |        | 400      | 2          | WIP             |
      |          |                             | POST   | 201      | 1          | covered         |
      |          |                             |        | 400      | 6          | covered         |
      |          |                             |        | 503      | 0          | not covered     |
      | 67%      | /products                   | POST   | 201      | 12         | covered         |
      |          |                             |        | 400      | 65         | covered         |
      |          |                             |        | 503      | 0          | not covered     |
      | 0%       | /v1.0/findAvailableProducts | GET    | 0        | 0          | missing in spec |
      | 0%       | /v1.0/products              | POST   | 0        | 0          | missing in spec |
      |-------------------------------------------------------------------------------------------|
      | 69% API Coverage reported from 5 Paths                                                    |
      |-------------------------------------------------------------------------------------------|

Here as you can see the routes are

• /v1.0/findAvailableProducts
• /v1.0/products

Can you please check once again.

[harikrishnan83]

Thanks @barath121 for the details. We have been able to narrow down the issue to library which we were using to enumerate routes which has not been kept up to date with Express 4.20.0 and later approach for compiling route pattern into regular expressions.

We have tried following up with the library. However since we have not heard back, we will fix this ourselves and rollout a release of Specmatic also with the update version to fix this. We will keep you posted on the same.