Add support for existing sqs queues #24
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Looks good, thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation: Add support for existing sqs queues
The current broker implementation raises a
botocore.errorfactory.QueueNameExistsexception when the queue already exists. This PR tries to solve this replacing the call tosqs.create_queuewith a call to a new internal method_get_or_create_queuethat first tries to find the queue, and creates it when it doesn't exist. This avoids changing any logic in thedeclare_queuemethod.The part I'm most unsure of is the extra efforts I need to put in the sqs stub for the tests, but they all are green now.
TD;DR
I don't know if this could be of any help. I came accross the same situation as @dvazar in: retech-us#1 and #14. Then I find out in #19 that the PR was not accepted. I see in the comments references to too much work on various PRs to review, but I'm not able to find out if I can help with something.
Actually, currently I'm following @Bogdanp advice in #19 (comment) and subclassed SQSBroker and to override
declare_queues. We've done exactly this in one of our systems and it is working perfectly.It was then that I thought about the PR. So this are my 2¢.
Feel free to point me to any direction I can help with this and I'll try to do my best. Also, I'll understand if this is not helping. In that case, feel free to discard it.
In any case, thanks a lot for the great work with dramatiq!