Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support passing arguments to worker constructor #76

Open
shouichi opened this issue Nov 29, 2018 · 3 comments
Open

Support passing arguments to worker constructor #76

shouichi opened this issue Nov 29, 2018 · 3 comments

Comments

@shouichi
Copy link

Though rq.worker.Worker constructor accepts many arguments (https://github.com/rq/rq/blob/eaf598d73ce234791ee22ef82f7f095aece9f2a6/rq/worker.py#L159
), it seems that Flask-RQ2 does not support passing arguments to worker constructor. Do you guys want to support this feature?

Background

rq.worker.Worker uses hostname and pid for its name by default (https://github.com/rq/rq/blob/eaf598d73ce234791ee22ef82f7f095aece9f2a6/rq/worker.py#L239) but we run workers in container and hostname/pid are all the same. As a result, workers failed to start due to the duplicated name (https://github.com/rq/rq/blob/eaf598d73ce234791ee22ef82f7f095aece9f2a6/rq/worker.py#L275).

Thanks!

@jezdez
Copy link
Member

jezdez commented Nov 29, 2018

Assuming you mean the RQ.get_worker method: yes. But we shouldn't simply pass all **kwargs to Worker but define some options that are shared in the RQ instance instead. In other words let's try to keep Flask-RQ2 a high-level API that hides some of the low-level features.

Here are the arguments that need to be added:

  • name (add as RQ.worker_name?)
  • default_result_ttl (reuse RQ.default_result_ttl)
  • default_worker_ttl (add as RQ.default_worker_ttl)
  • job_monitoring_interval (add as RQ.job_monitoring_interval, and add as option to worker CLI callback)

@shouichi
Copy link
Author

But we shouldn't simply pass all **kwargs to Worker but define some options that are shared in the RQ instance instead. In other words let's try to keep Flask-RQ2 a high-level API that hides some of the low-level features.

Got it (I understand the RQ's goal is keeping it simple).

  • name (add as RQ.worker_name?)

Make the default value None and allow users to pass arbitrary string. That why the RQ keeps the current behavior. Correct?

  • job_monitoring_interval (add as RQ.job_monitoring_interval, and add as option to worker CLI callback)

Do you mean adding an option (@click.option) to this func?

def worker(rq, ctx, burst, logging_level, name, path, results_ttl,

Thanks!

@alvin1
Copy link

alvin1 commented Dec 6, 2018

From my use of Flask-RQ2, I'd like to add:
date_format
log_format

These 2 args are useful for the logging

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants