Remote and local servers

[smoya]

Context

This is part of the RFC #618 (comment).
In an effort around reducing the scope of such RFC, it has been split into several proposals so they can move forward independently.

An application can be both a server and a client, or just a server, or just a client. Therefore, servers can't be made up of that mix. Exposed server interfaces and remote servers (usually brokers) have to be separated because —even though they look the same— they're semantically different.

This fact is already being considered in The many meanings of an AsyncAPI file, and this proposal is not going through all the details that issue does regarding the semantics of the AsyncAPI file.

It focuses on solving perspectives when it comes to Servers, essential when generating code from an AsyncAPI file. For example, generating a Websocket server might be problematic if there is no distinction between those ports the application listens to and which remote servers the application connects to.

Now that Allow servers and channels to be defined inside components will be included in the 2.3.0 version of the spec; this proposal makes more sense than ever as it enhances the way we define and share servers.

The proposal

This proposal introduces remote and local servers.

• Remote servers are those our application has to connect to. They're usually brokers but can also be other kinds of servers.
• Local servers are server interfaces our application exposes. Their URL defines where clients can reach them.

There are several ways to portray this new concept into an AsyncAPI file, and the idea is that we can all find out which one fits better (or rather discard all, suggest new ones, etc.).

Proposal 1: Introduce a new kind property on the Server object

Servers get a new kind property. By default, all servers are remote servers (e.g., brokers).
However, if the kind property is set to local, that means the application actually exposes the server (e.g., WebSocket or HTTP Server-Sent Events server).

Example:

# application.yaml
asyncapi: 3.0.0

servers:
  test:
    url: ws://test.mycompany.com/ws
    protocol: ws
    kind: local
    description: The application creates a WebSocket server and listens for messages. Clients can connect on the given URL.
  mosquitto:
    url: mqtt://test.mosquitto.org
    protocol: mqtt
    kind: remote # This is the default value
    description: The application is connecting to the Mosquitto Test broker.

This option is the less intrusive one as it only requires a new property to be added to the Server object.
This is very important since improving the Developer Experience is one of the main goals of the AsyncAPI Initiative's roadmap.

However, it penalizes Server's shareability by not allowing to overwrite kind property when using $ref.
In the following example (adapted from our official social-media example), there is a backend service that connects to a remote MQTT broker, but also exposes a websocket server. Then a frontend service connects to that websocket server exposed by the backend service.

# common.yaml
asyncapi: 3.0.0

components:
  servers:
    websocketServer:
      url: ws://mycompany.com/ws
      protocol: ws
      # kind: local for some services, remote for others. 
    mosquitto:
      url: mqtt://test.mosquitto.org
      protocol: mqtt
      kind: remote # remote is the default value anyway
      bindings:
        mqtt:
          clientId: websocketServer

# backend.yaml
asyncapi: 3.0.0

info:
  title: Website Backend
  version: 1.0.0

servers:
  websiteWebSocketServer:
    $ref: 'common.yaml#/components/servers/websocketServer'
    # this server would be a `local` server. However, `kind` field can't be declared or replaced when using `$ref`.
  mosquitto:
    $ref: 'common.yaml#/components/servers/mosquitto'

# ...

# frontend.yaml
asyncapi: 3.0.0

info:
  title: Website WebSocket Client
  version: 1.0.0

servers:
  websiteWebSocketServer:
    $ref: 'common.yaml#/components/servers/websocketServer'
# ...

For the record, JSON Reference does not allow overwriting properties. More on this issue.

Proposal 2: Split current servers into remotes and servers

servers field, at root AsyncAPI document level, are split into remotes and servers based on their nature, meaning servers will be considered as local servers, and remotes (as the name indicates), remote servers.
However, servers declared inside components will all live under servers as they can't predict their nature. And here it is the benefit of this approach. Let's see it with the following example:

# common.yaml
asyncapi: 3.0.0

components:
  servers:
    websocketServer:
      url: ws://mycompany.com/ws
      protocol: ws
    mosquitto:
      url: mqtt://test.mosquitto.org
      protocol: mqtt
      bindings:
        mqtt:
          clientId: websocketServer

# backend.yaml
asyncapi: 3.0.0

info:
  title: Website Backend
  version: 1.0.0

servers:
  websiteWebSocketServer:
    $ref: 'common.yaml#/components/servers/websocketServer'
remotes:
  mosquitto:
    $ref: 'common.yaml#/components/servers/mosquitto'

# ...

# frontend.yaml
asyncapi: 3.0.0

info:
  title: Website WebSocket Client
  version: 1.0.0

remotes:
  websiteWebSocketServer:
    $ref: 'common.yaml#/components/servers/websocketServer'
# ...

As we can see, the reusability of the same server is much easier. However, it can also penalize Developer Experience:

• It introduces a new level of complexity in terms of knowledge. What are remotes? What are servers? How can we explain that to people that come from 2.x or even new users? It is not as intuitive as it might sound.
• It is a big breaking change, so jumping from 2.x to 3.x is not as trivial as with the previous solution (kind).

Conclusion

We need to keep iterating and discussing it, so feedback is more than welcome.

[fmvilas]

There was so much editing in the original issue that I think I forgot to update it but I'm all in with proposal 2, having two root objects: servers and remotes. Currently, servers is a mix of local servers (local to the app being described in the document) and remote servers. If we add the kind attribute, it will make it harder to reuse. E.g., what's local to one app will be remote for the rest. So the fact that a server is local or remote is not a property of the server, it's a property of the relationship between the app being described and the server.

[thake]

I started reading your various discussions on version 3 and I find it very refreshing that everything is discussed openly! The recordings of your meetings on youtube really help!

I would like to add my 2¢ on the naming of servers and remotes. When I first read ticket #618 the naming was quite confusing for me. IMHO both names do not express the relationship between the application that is specified with the AsyncAPI spec and the referred servers. Maybe a distinction in the way of providedServers and usedServers makes this relationship clearer: The application provides/uses servers to communicate asynchronously.

To increase reusability, the default would be that servers defined under servers are used servers. Only if a server is referenced from the providedServers property, then the application provides the server itself.

Here is an example:

# common.yaml
asyncapi: 3.0.0

components:
  servers:
    websocketServer:
      url: ws://mycompany.com/ws
      protocol: ws
    mosquitto:
      url: mqtt://test.mosquitto.org
      protocol: mqtt
      bindings:
        mqtt:
          clientId: websocketServer

# backend.yaml
asyncapi: 3.0.0

info:
  title: Website Backend
  version: 1.0.0

servers:
  websiteWebSocketServer:
    $ref: 'common.yaml#/components/servers/websocketServer'
  mosquitto:
    $ref: 'common.yaml#/components/servers/mosquitto'
providedServers:
  - websiteWebSocketServer

# ...

# frontend.yaml
asyncapi: 3.0.0

info:
  title: Website WebSocket Client
  version: 1.0.0

servers:
  websiteWebSocketServer:
    $ref: 'common.yaml#/components/servers/websocketServer'
# ...

[magicmatatjahu]

@thake Thanks for that comment! Interesting idea, because then we can easily validate duplication of servers in groups local/provided and remote/used. However I think that people will have problem with this approach that servers by default are "used" and by defining providedServers the rest becomes used. A better solution would be to have all servers assigned to the correct group:

# backend.yaml
asyncapi: 3.0.0

info:
  title: Website Backend
  version: 1.0.0

servers:
  websiteWebSocketServer:
    $ref: 'common.yaml#/components/servers/websocketServer'
  mosquitto:
    $ref: 'common.yaml#/components/servers/mosquitto'
    
usedServers:
  - mosquitto
providedServers:
  - websiteWebSocketServer
  
# frontend.yaml
asyncapi: 3.0.0

info:
  title: Website WebSocket Client
  version: 1.0.0

servers:
  websiteWebSocketServer:
    $ref: 'common.yaml#/components/servers/websocketServer'
    
usedServers:
  - websiteWebSocketServer

but that's my thought :)

[thake]

@magicmatatjahu thanks for your quick feedback!

I took a liking to @smoya's statement in the 1st proposal in which he defined every server to be remote by default because in most of the cases this is true. This is why I'm in favor of leaving out the usedServers property, as it reduces the minimum required specification.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️

[Souvikns]

Example of a use case:
Right now in glee, we are using x-kind to declare remote or local servers and by default treating servers as remote if the property is missing.

so according to our use case #689 (comment) this looks better.

[fmvilas]

Alright, taking care of this issue now. Will be moving it forward in the next weeks. Thanks a lot for the great input @smoya @thake @magicmatatjahu @Souvikns. I'll come up with a formal proposal soon.

[fmvilas]

Alright, so I got a proposal but I have some concerns. Would be awesome to get your input :) #874

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity 😴

It will be closed in 120 days if no further activity occurs. To unstale this issue, add a comment with a detailed explanation.

There can be many reasons why some specific issue has no activity. The most probable cause is lack of time, not lack of interest. AsyncAPI Initiative is a Linux Foundation project not owned by a single for-profit company. It is a community-driven initiative ruled under open governance model.

Let us figure out together how to push this issue forward. Connect with us through one of many communication channels we established here.

Thank you for your patience ❤️