Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Pipeline coroutine safety #278

Open
StarlightIbuki opened this issue Jul 5, 2024 · 6 comments
Open

[Feature Request] Pipeline coroutine safety #278

StarlightIbuki opened this issue Jul 5, 2024 · 6 comments

Comments

@StarlightIbuki
Copy link

StarlightIbuki commented Jul 5, 2024

When using a pipeline combined with a coroutine, some commands might get mixed into the pipeline. Please consider introducing pipeline objects, e.g.:

local pipe = red:start_pipeline()
pipe:set(...)
-- ...
for result in ipairs(pipe:commit()) do
  -- ...
end
@spacewander
Copy link
Member

In the current implementation, the underlay connection and the red object is 1:1. The pipeline in Redis will occupy the connection. So red:start_pipeline() is probably a syntax sugar which creates another redis connection with the current configuration of red.

@StarlightIbuki
Copy link
Author

In the current implementation, the underlay connection and the red object is 1:1. The pipeline in Redis will occupy the connection. So red:start_pipeline() is probably a syntax sugar which creates another redis connection with the current configuration of red.

or we can block the connection only when committing the pipeline, and let the user decide whether to create another connection?

@spacewander
Copy link
Member

What do you mean by "block the connection only when committing the pipeline"?

@StarlightIbuki
Copy link
Author

What do you mean by "block the connection only when committing the pipeline"?

Instead of creating a new connection, we lock the connection until the pipeline finishes

@spacewander
Copy link
Member

IMHO, it would make thing complex if we put the connection management inside the redis client.
If we ensure there is only one coroutine that can write to a cosocket, then we can implement pipeline safety in a lock-free way. This mechanism can be put outside a client so it can be used as a general solution, instead of being a complex and hard-to-maintain code inside a library.

@StarlightIbuki
Copy link
Author

IMHO, it would make thing complex if we put the connection management inside the redis client. If we ensure there is only one coroutine that can write to a cosocket, then we can implement pipeline safety in a lock-free way. This mechanism can be put outside a client so it can be used as a general solution, instead of being a complex and hard-to-maintain code inside a library.

I agree. I'm now using a strategy to clone if the connection is doing a pipeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants