JDBC HA Scenario

[lavocatt]

We've been chatting with @gtully about the new JDBC High Availability use case he is working on. As of right now our front-end isn't compatible with this new kind of deployments and I would like to discuss here the steps that would be required for it to be so.

Shifting from broker deployment to application deployment

Brokers are running on pods, applications are running on brokers.
The plugin would need to be able to:

• Deploy empty fully locked down brokers
• Deploy applications to available empty brokers

It's a new kind of layering we would provide.

The jolokia rbac

A fully locked down broker is heavily limiting what gets exposed. No mbean, no metrics, no nothing except a jolokia endpoint.
We will need to have a way to communicate to the jolokia endpoint who is the current user in a safe way to get appropriate answers. The idea is to rely on some tokens provided by openshift:

• The user gets a token from openshift that reflects the role they have
• The token is shared to jolokia during the login phase
• Jolokia contacts openshift with the token to get a role and to filters the operations based on that
• Some token is returned back to the client so the auth process can be skipped on successive requests

The integration within the OpenShift console

Here's a first draft of the changes to make on the plugin:

Listing applications

Nowadays the plugin is listing the deployed brokers, their statuses and provides a details page for each broker.
We would need to provide the same kind of structure but at the application level.

Deploying an application

Nowadays the plugin has a create broker button. We would need to provide a deploy an application button.

An application gets defined by it usecases

Instead of going to a form, the user would select a usecase, or a set of compatible usecases they want to achieve.

Deploying to running brokers

The user would select an available broker (or HA broker) to deploy on, with an option to spin one up if none are available.

[RoddieKieley]

In the last comment we say

user would select an available broker (or HA broker)

which alludes to the fact that there could be an available HA broker group. This likely needs some further elaboration as well, as at the moment while we can create a broker or brokers relatively seamlessly but not HA brokers of any sort.

In fact to do so for the specific JDBC based use case relies on detailed configuration that while possible is certainly verbose from the point of view of the end user. We should also concern ourselves with making this use case easy/easier to use as well as the application deployment scenario itself.

[gtully]

The split that is in my mind is based on the concept of provisioning brokers and on-boarding applications.
Possibly provisioning works for both aspects.

For an application (AppA) that wants to persistent work queue with a 10G backlog.

That is described in some new CR that maps this to messaging verbs and adverbs (choosing these is the challenge):

• app-provisioning
  name: AppA
• publisher on WQ with role Producers and Latency < 30ms
• persistent competing consumer on WQ with role Consumers and backlog 10G

Each of those verbs and adverbs has a specific intent:
eg: WQ is anycast b/c all consumers are competing
producer latency below 30ms, with pod fault, needs N>1 brokers (that federate WQ), may also mandate no paging. For low latency with very large backlog it would require an "elastic queue" provisioned broker.
broker needs disk free space > 10G

There has to be a matching and capable provisioned broker for this work queue use case.

The work flow is along the lines:

• If a provisioned broker matches we can deploy - as in make it so.
• If a provisioned broker is at capacity, it will need to be scaled if that is supported for that use case or duplicated.

I imagine an operator that supports managing these two new provisioning CRs and a UI that facilitates their creation and visualisation and understands their interrelation.

I also think a JDBC ha broker could provide the simplest route to a provisioned broker, and we can start to think on the verbs for an application provisioning CR.
Because this is essentially a new DSL, it won't work for everyone but should do exactly what is expected for some one.

[lavocatt]

Quick question about this: does it have to be a DSL? Couldn't we instead work with a set of dependencies for each application type?

[gtully]

No, only in so far as it helps.

Even dependencies would need names that are constrained by a type.

Any template/dependencies/type set will need to expand to match the use cases and then naming will be important. That is where a DSL can help, but it will depend, I will probably need to be extensible as well.

For example, an application may want to onboard 2k addresses, would it make sense to translate those into some DLS, probably not. Better to be able to import or consume directly in some way. They could be a dependency.

[lavocatt]

Any template/dependencies/type set will need to expand to match the use cases and then naming will be important. That is where a DSL can help, but it will depend, I will probably need to be extensible as well.

In my experience, a DSL can be a bit of a challenge when there's need to extend its set of operators. IMO it's easier to work with types and rely on the statistical analyzer of an established language.

Also having to handle the syntax & parsing of a DSL and to be sure it doesn't have loopholes looks like a very hard task. But maybe there are good libraries to help nowadays.

[gtully]

that is fair. I like a DSL as a means to identify the verbs/types which is I think is the real challenge.
We can investigate the tools and complexities and make an informed call on that detail.

[lavocatt]

which alludes to the fact that there could be an available HA broker group. This likely needs some further elaboration as well, as at the moment while we can create a broker or brokers relatively seamlessly but not HA brokers of any sort.

Indeed, we need a plan for that as well. I've kind of mixed two different discussions together. The split between broker provisioning and application on-boarding, and the HA brokers provisioning.

In fact to do so for the specific JDBC based use case relies on detailed configuration that while possible is certainly verbose from the point of view of the end user. We should also concern ourselves with making this use case easy/easier to use as well as the application deployment scenario itself.

Agreed. A first good goal would be to support JDBC HA creation from presets with little, if none, user input.

Then once we've got that running, we can start figuring out the jolokia bits & pieces, and after that the application onboarding.