The many meanings of an AsyncAPI file

[fmvilas]

As of version 2.x, an AsyncAPI file represents an application. However, as we work on #618, other proposals have arisen to:

1. Make the AsyncAPI file also become a collection of reusable objects that would be referenced from other applications.
2. Make it possible to have a single AsyncAPI file to describe multiple applications. This is especially useful when defining a client and server together.

[fmvilas]

I think option 2 would also allow for people to define their whole architecture in a single file. Which might or might not be a good thing. Definitely, something to take into account.

[jessemenning]

My feeling on this is that:

• Async encompasses a wide range of use cases, from 1:1 interactions using websockets to n:n solutions using event brokers.
• If AsyncAPI aims to be a generic async spec, we need to accommodate all async usecases
• Because n:n architectures are complex, particularly at scale, it is sometimes desirable to describe the entire architecture, in order to describe the function of a particular component of that architecture

The implications of the above for the spec would be:

• Allow--but do not mandate--flexibility on how an AsyncAPI spec is formatted . This would include "library" files, databases, event registries, etc. However, if the best representation is a single file, the spec should allow it to collapse to that.
• Strictly separate logical relationships from their physical implementations. That is, use bindings to prevent protocol specific language from entering the main spec.
• Plan for the complex (brokered) use case, collapse to the simple (1:1) use case, not the other way around (credit to @lornajane for this)

[nictownsend]

My thoughts are:

• As we achieve point 2 - we should ensure that it's bidirectional such that tooling can combine and split definitions.
• I should be able to look at the combined document and easily answer the following:
  • How many separate applications are described?
  • Which channels are used by which applications?
  • What messages are read/sent onto the channels by each application?
  • Where are those channels located?
  • If I want to send/receive a message from a specific application, which protocol/server/channel do I connect to?

Perhaps it's as simple as adding an applications stanza whereby individual application's operations are listed (I've also included the proposal in #618 for assigning messages to operations).

Including applications would mean parsers ignore top level operations.

asyncapi: 3.0.0
info:
  title: EDA
  version: 1
  description: The whole application
servers:
  kafka-server: 
    url:
    protocol:
 mq-server:
   url:
   protocol:
channels:
  kakfa-alerts:
    servers:
       - kafka-server
     messages:
       oneOf:
           - message1
           - message2
  mq-messages:
    servers:
       - mq-server
    messages:
      message2
applications:
  kafka-mq-bridge:
    info:
    operations:
      kafkaAlert:
         action: receive
         channel: kafka-alerts
         message: message2
     mqSend:
        action: send
        channel: mq-messages
  kafka-responder:
    info:
    operations:
      kafkaAlert:
         action: receive
         channel: kafka-alerts
         message: message1
components:
...

[kurtsys-howest]

I guess that's that way to go, or just a bit further:
There's a file describing the channels.
It would be good to have a file for each application using the channels, so instead of 1 file:

asyncapi-broker.yaml:

asyncapi: 3.0.0
info:
  title: EDA
  version: 1
  description: The whole application
servers:
  kafka-server: 
    url:
    protocol:
 mq-server:
   url:
   protocol:
channels:
  kakfa-alerts:
    servers:
       - kafka-server
     messages:
       oneOf:
           - message1
           - message2
  mq-messages:
    servers:
       - mq-server
    messages:
      message2

asyncapi-mq-bridge.yaml:

    $broker: 'asyncapi-broker.yaml'
    info: something
    operations:
      kafkaAlert:
         action: receive
         channel: kafka-alerts
         message: message2
     mqSend:
        action: send
        channel: mq-messages

This makes it easier for other teams to plugin to the messaging/broker system without messing with the broker/channels file describing the messages that are allowed on the system.
Since the $broker key (or any better name) references another file, it should be possible to validate if the channels exist, the right messages are sent and so forth.

[jonaslagoni]

I want to clarify some ways that you can already utilize the specification, or at least start to get them into writing so we all can get a common understanding of it. Of course, these are just the ways I can see and have heard/used, so if you use it in a different way, which is not covered, please correct me 😄 This also goes for the naming of the different types, these are just the names that made sense to me, so I would love to know if you can think of a different way we can reference them.

Because of $ref, we are able to have multiple ways to define an application. A summary of these different ways can be found below. Which one to choose depends on your situation.

NOTE: In the examples, I am assuming that $ref can be used for all JSON objects, see #650 for further information.

Defining a dictionary

One way we have seen the specification being used is in a Dictionary type of AsyncAPI document, which is a type of pattern for defining definitions for the common information, shared by each separate application.

Types

There are multiple ways to define dictionaries, these are the most common we see:

Centralized Dictionary

Using an AsyncAPI document as a dictionary and its component section to reuse definitions across all AsyncAPI documents.

##### asyncapi.yaml
asyncapi: 3.0.0
...
servers: 
  xxxx:
    $ref: dictionary.yaml#/components/servers/xxxx
...

##### dictionary.yaml
asyncapi: 3.0.0
info:  
  description: Dictionary for all definitions
channels: {}
components:  
  servers: 
    xxxx: 
      url: mqtt://test.mosquitto.org  
      protocol: mqtt
  ...

Decentralized Dictionary

For example, here we define all the servers in the same file, this could also be defined in separate files.

##### asyncapi.yaml
asyncapi: 3.0.0
...
servers: 
  xxxx:
    $ref: servers.yaml#/xxxx | $ref: servers/xxxx.yaml
...

##### servers.yaml
xxxx:
  url: mqtt://test.mosquitto.org  
  protocol: mqtt

##### servers/xxxx.yaml
url: mqtt://test.mosquitto.org  
protocol: mqtt

Mix Dictionary

You could also do a mix of the two.

##### asyncapi.yaml
asyncapi: 3.0.0
...
servers: 
  xxxx:
    $ref: servers.yaml#/xxxx
...

#### dictionary.yaml
asyncapi: 3.0.0
info:  
  description: Dictionary for all other definitions then Servers
channels: {}
components: 
  ...

##### xxxx.yaml
url: mqtt://test.mosquitto.org
protocol: mqtt

Defining the application

What the specification is built for, defining application behavior, but even here there are multiple ways to do this.

Types

These are the types of ways to define an AsyncAPI document:

Application using a Dictionary

The application uses one of the Dictionary types mentioned above.

##### asyncapi.yaml
asyncapi: 3.0.0
...
servers:  
  xxxx:    
    $ref: 'dictionary.yaml#/components/servers/xxxx'
channels:  
  xxxx:    
    $ref: 'dictionary.yaml#/components/channels/xxxx'

Application using its own definitions

An application that does not make use of any dictionary, but uses its own definition.

##### asyncapi.yaml
asyncapi: 3.0.0
...
servers:  
  xxxx:    
    ...
channels:  
  xxxx:    
    ...

Application using a mix between Dictionary and its own definitions

An application uses a mix between one of the dictionary types and its own definitions.

##### asyncapi.yaml
asyncapi: 3.0.0
...
servers:  
  xxxx: 
    ...
channels:  
  xxxx:    
    $ref: 'dictionary.yaml#/components/channels/xxxx'

[jonaslagoni]

Onwards, to follow the discussion about whether we should be able to define a collection of applications see #658

It is based on your discussion and suggestions @kurtsys-howest @nictownsend @jessemenning @fmvilas.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[fmvilas]

Reopening as there's still work happening at #811.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️