How OpenAPI or any spec related can help on contract/schema testing?

[lednubr]

Context

A lot of companies uses microservices, and have HTTP or async/event communication between them. It´s a common problem when server changes a thing, that something breaks on the client microservice. An we have tools like https://docs.pact.io/ and https://www.postman.com/ that help on that, and pact is relying on OAS for some automation on contract/schema testing.

Problem

OpenAPI spec, AFAIK just describes the server part of http communications, and contract testing is all about checking the "match" between a client and a server, or a producer and a consumer. We researched and didn't find out a "contract/spec" that describes the client "behavior" or the way the client uses/sends data to the API. Pact is trying to do that with https://docs.pactflow.io/docs/workshops/bi-directional/ and using mock samples and other "client evidences" as way to extract this data to a file called PACT (their format).

Other thing to mention is that we are using the https://martinfowler.com/bliki/TolerantReader.html approach. So we don't export/generate http clients from open api spec. This approach reduces the probability of a "breaking change", because the client does not need to know/use all fields/structure that the server produces.

Proposal/discussion

What if we had a "open spec" that describes, the client part of the communication? That could be very similar to the OpenAPI spec and could be used for contract/schema checks.

One of the challenges on that is how can we generate this spec. some options:
1)from mocked samples (wiremock)
2)recording runtime behavior on integration tests or even on production/sandbox enviroments
3) support some specific stacks, and parse that info from http client or any kind of "structured" code that tell us , how the request is going to be like. That could be done through static analysis or even evaluating some "runtime" http components.

But the main thing is not about how to get this files/spec, is about having a open spec for http clients.

If this is not the right place to start a discussion on that, please, we are open to all kind of suggestions.

[hkosova]

Doesn't OpenAPI already cover the client part?

• parameters and requestBody define the data that the client is supposed to send to a specific endpoint using a specific HTTP method.
• responses define the HTTP status codes, media types, and data that the client can expect from the server.
• OAS3 links define the client-side operation traversal mechanism.

[hkosova]

For instance: server X defines a endpoint /users/, which is a POST with a User Schema as payload, and with tons of fields. Client A can use only 2 fields of them (the required ones, e.g.: username, and email), and client B can use the required 2 and others.

This is already covered, unless I'm misunderstanding your use case. You would define the User schema with the required username and email fields and other optional fields. The clients must send the required fields and may send optional fields. It's up to each client which optional fields to send.

OpenAPI also lets you define multiple examples for the request/response bodies- e.g. a minimal example with just the required fields, and a complete example with all fields.

paths:
  /users:
    post:
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/User'

            # Request body examples
            examples:
              minimal:
                summary: A minimal example with just the required fields
                value:
                  username: helen
                  email: [email protected]
              full:
                summary: All fields
                value:
                  username: helen
                  email: [email protected]
                  role: admin
                  # ...
      responses:
        '201':
          description: Created

[lednubr]

Thats nice! but, in that case, how can i know which example was generated by client A, and which of those are client B related? That information would be very important to run a "contract" check.

[lednubr]

To better understanding, what we are talking is about service relationships, and how is the "structure" of the data that its transfered from one service to another. We can infer that from examples, but the best approach is to have a request/response body spec as open api provides.

That means that client A should have "client spec"

paths:
  /users:
    post:
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/User'

And user schema for client A is a simplified version of the schema with only 2 attributes (e.g. email,name)

On the other hand, client B will have the same stuff.. but the User schema would be a richer one, or even the complete one as the server provides.

[lednubr]

Thats nice! but, in that case, how can i know which example was generated by client A, and which of those are client B related? That information would be very important to run a "contract" check.

I think the answer is already there, since we have keys (minimal and full). So client A will be the minimal, and client B the full..perhaps

[lednubr]

When we talk about CDC(Consumer driven contracts) some tools provide a way to do it like (https://www.youtube.com/watch?v=ACENQFi4dwU&t=884s). Postman approach is: write a test scenario, making a HTTP request to your server. So every time you change your API, we are going to validate if that request is still valid. This request is going to be some "client mocking", where you can have one sample to each kind of client you have. That approach, makes thinks harder, because you always need to write tests. This can be even worse if you have tons of microservices communicating with each other.

What we want to discuss and think about, is a way to do this "testing", not using samples or request examples, but a spec file that defines the client behavior. From that spec we can generate lots of samples, and do the check.

That spec could be something proposed by OAS, or other player...