[GOFLOW-28] abstract the queue mechanism to support other mechanisms.

[giorgiozoppi]

Goflow Queue Abstraction.

Goal

We want to support multi distributed queue support other then a Redis:

• support for RedPanda
• support for RabbitMQ

So we might want to abstract the queue mechanism of this workflow manager to allow this extensibility.

[jippi]

I would love an in memory/non-distributed mechanism :)

[s8sg]

@giorgiozoppi @jippi This has been requested by many. I think we can have the vanilla version using reds, with some customization option to support the queue.

For a quick start we definitely doesn't want users to configure a brunch of stuff

[giorgiozoppi]

Assign to me. I will work on this item.

This is an high level design.

[giorgiozoppi]

For in memory queue i'd like to use rocksdb or similar things in go.

[s8sg]

@giorgiozoppi From project POV I'll put this in low priority. I'll focus on fix the existing issue and the bugs first. Once we have adaptability we can work on additional feature. WDYT ?

[s8sg]

@giorgiozoppi Why we need a in memory queue, redis is in memory. Are we saying we don't need distributed approach ?

[giorgiozoppi]

We don't need a memory queue. My answer is was to @jippi request. So please assign me some starting bug to fix, how can i help?. Looking to the code we just need an adapter to make different stores to be used.

[s8sg]

Lets pick up from the project board https://github.com/users/s8sg/projects/1

…

On Sat, 11 Mar 2023 at 22:57, Giorgio Zoppi ***@***.***> wrote: We don't need a memory queue. My answer is was to @jippi <https://github.com/jippi> request. So please assign me some starting bug to fix, my desire is to get again working on distributed systems. — Reply to this email directly, view it on GitHub <#28 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABYIJ2WNM5CE4ZTPKQHOTKDW3SOGRANCNFSM6AAAAAASFHCM64> . You are receiving this because you commented.Message ID: ***@***.***>