Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More flexible using_solid_queue_adapter? check #489

Open
ThomasCrambert opened this issue Jan 17, 2025 · 3 comments
Open

More flexible using_solid_queue_adapter? check #489

ThomasCrambert opened this issue Jan 17, 2025 · 3 comments

Comments

@ThomasCrambert
Copy link

Hello 👋,

Currently to check if a recurring task is using solid_queue as a queue adapter or not, queue_adapter_name is used[1]. This means that by default a custom solid queue adapter will be effectively identified as such if it named SolidQueueAdapter.

At first we didn't named ours like that and had a hard time troubleshooting this issue, as the recurring jobs were successfully inserted in SolidQueue::Job, but not in SolidQueue::RecurringJob.

Would it be possible to enhance the check so that it checks the queue_adapter ancestors? I believe this is the easiest and best solution in my case.

[1]

job_class.queue_adapter_name.inquiry.solid_queue?

@rosa
Copy link
Member

rosa commented Jan 17, 2025

Oh wow! I didn't imagine someone running into this 😕 What's your use case to need a custom adapter extending solid_queue adapter?

@ThomasCrambert
Copy link
Author

ThomasCrambert commented Jan 17, 2025

What's your use case to need a custom adapter extending solid_queue adapter?

We need them to specify which database shard to use when enqueuing a job. They basically do the following:

class SolidQueueAdapter < ActiveJob::QueueAdapters::SolidQueueAdapter
    def enqueue(active_job)
      SolidQueue::Record.connected_to(shard: :some_shard) { super(active_job) }
    end

    def enqueue_at(active_job, timestamp)
      SolidQueue::Record.connected_to(shard: :some_shard) { super(active_job, timestamp) }
    end

    def enqueue_all(active_jobs)
      SolidQueue::Record.connected_to(shard: :some_shard) { super(active_jobs) }
    end
end

@rosa
Copy link
Member

rosa commented Jan 17, 2025

Oh, interesting! I'll eventually get to adding sharding support to Solid Queue. I was supposed to work on that last month but there was a change on plans 😅

Does the shard depend on the job itself? I wonder if using an around_enqueue callback would work in your case. It's not run for enqueue_all, though, but how do you decide the shard in that case, if there are multiple jobs? Do you ensure you only pass jobs that go to the same shard and fail otherwise?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants