Third-Party UNS/Broker Integration

[joepavitt]

Many of our larger customers and prospects already have sophisticated MQTT/UNS architectures in place. This issue to to track what can FlowFuse offer to improve their Node-RED and application development experience.

Tasks

Breakdown of the actions we've agreed upon to move forward with for UNS.

Stories

Preview Give feedback

1. Design: Third Party Broker #4965
   design +1
   
2. Third Party Broker Definition
3. Build Data Collector for 3rd Party MQTT Broker #4969
   task +1
   
4. Unified Topic/Hierarchy View
5. Schema Generation (Async API)

Ideas

Storage of future possibilities with UNS

Idea: Unified MQTT Experience

With limitations where reasonable, we want to offer a unified experience to users for external or internal brokers. The "Bring Your Own Broker" route should still offer:

• Topic Schema View
• Topic Schema Auto-Generation
• Extension: Topic Validation
• Payload Schema View
• Payload Schema Auto-Genration
• Extension: Payload Validation

Idea: Centralised Config Nodes:

Issues

• Centralised Config Nodes (View-Only) #4306
• Centralised Config Nodes (Management & Distribution) #4307

Description

TLDR for this is that users can detail their credentials in one place, and respective Node-RED instances will be aware of those configurations and make them easily available in the Node-RED Editor. This will prevent duplication of config nodes across hundreds of NR instances, and in the need for updates, save significant amounts of time.

Idea: Debugging UNS Topics

We will soon support the visualisation of a Topic Hierarchy for our in-built MQTT Broker/Clients. However, we rely on direct access to the Broker in order to populate this rich view, which, for third-party brokers, we do not have.

We could however still provide a clear "Debug" view which subscribes (whilst active) to #, and builds out the topic tree, much like MQTT Explorer does.

[hardillb]

Initial thoughts for this is to start a new container (on docker/k8s) that runs a MQTT client that is passed credentails. This will run outside the core forge app and subscribe to # (or a supplied prefix).

This client can then

• log all topics
• do payload inspection to help with schema development

Putting it in it's own resource limited container means a high volume broker will not flood the forge app or any other monitors.

[joepavitt]

Prospect brought up the need for multiple, site-based MQTT brokers. Open to FlowFuse spinning up and managing those, but didn't want them all running in the same place as FlowFuse.

There is overlap here with respect to adding more Brokers to FlowFuse, and having FF manage/assist with those, however the extra dimension here was that we could still be in "control" of those brokers.

@hardillb curious to know if we have any quick wins here? I know the Helm Chart comes packages with a single MQTT Broker alongside FlowFuse. What would your architectural proposal here be, for single FlowFuse, multiple MQTT Broker deployments, with, in theory, FlowFuse controlling it "all". Or, do we just treat this as per this epic, where they are treated as "external" brokers - the complexity there is that's infrastructure that users need to setup, manage and configure themselves.

[hardillb]

@joepavitt The whole point of a multi tenant broker was to avoid having to run more than one broker for FFC, just looks like different brokers to each team.

Now if this is running broker on different sites, that is a totally different problem.

• If outside kubernetes then we would need an agent on the site (and a framework to host containers)
• If inside remote kubernetes clusters again we would need an agent in that cluster
• If a multi site single cluster, then things get really complicated making sure all the components run in that site/region

Also what is the point of these multi site brokers, if it's protection against network outages, then the authentication source will also need to be local to the broker and sync'd to something central.

At the moment there are too many questions to answer this.

[joepavitt]

Thanks Ben, yes, their concern was latency and network problems, so having the "local" broker option means the hardware at each site can communicate to the broker still.

Appreciate it's a different direction/challenge, but needed to ask the question.

[hardillb]

Also is building a schema across multiple brokers in scope (just to add to the feature list for the MQTT/UNS agent)

[hardillb]

This also touches on multi-site/cluster FlowFuse which is a topic that has come up before (but no plan yet)

[joepavitt]

Also is building a schema across multiple brokers in scope (just to add to the feature list for the MQTT/UNS agent)

No

This also touches on multi-site/cluster FlowFuse which is a topic that has come up before (but no plan yet)

Yeah, this definitely overlaps

[joepavitt]

@hardillb I'm putting together the UI sketches together, what fields/info will we need for a third-party broker integration? My initial thinking was Broker URL, Client ID, Username/Password? But then that limits to a single client per broker, so is it as "simple" as just the Broker URL, then the client configuration is handled in a similar fashion to how we do it now?

[hardillb]

@joepavitt I think first pass should be adding a single external broker per team.

We can iterate on support for multiple brokers, just need to ensure we don't do anything in the design that limits things.

We will need the following to connect:

• broker hostname
• broker port
• TLS on/off
  • might need to includer CA certificate if not public certificate
• protocol (pure MQTT or MQTT over Websockets)
• a topic prefix (default to #) but may want to be able to subscribe to foo/#
• optional client id (default to auto generated value, max 23 chars in length e.g. flowfuse-<random>
• credentials which could be
  • username/password
  • client certificate/key (could be iteration 2)

Just to be clear, this is for data capture/schema generation, not for passing these creds into Node-RED instances.

The data collection side of this should only be a single set of credentials, we can't use the credentials creation we have for our broker because we have no control over 3rd party broker authentication.