Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Anticipate deprecation of *args and *kwargs to BaseOperator #10

Open
lauralorenz opened this issue Jan 20, 2017 · 0 comments
Open

Anticipate deprecation of *args and *kwargs to BaseOperator #10

lauralorenz opened this issue Jan 20, 2017 · 0 comments

Comments

@lauralorenz
Copy link
Contributor

Recently running some stuff on fileflow running airflow 1.7.1.3 I was getting deprecation warnings for the following:

/Users/llorenz/Envs/fileflow/lib/python2.7/site-packages/airflow/models.py:1719: PendingDeprecationWarning: Invalid arguments were passed to DivePythonOperator. Support for passing such arguments will be dropped in Airflow 2.0. Invalid arguments were:

*args: ()

**kwargs: {'data_dependencies': {'something': 'write_a_file'}}

  category=PendingDeprecationWarning

According to the dev email list this is anticipation of only supporting explicit args:

Hi Laura,

The error is raised if an unused argument is passed to BaseOperator --
basically if there is anything in either args or kwargs. The original issue
was that in a number of cases arguments were misspelled or misused by
Operator subclasses and instead of raising an error, they were just passed
up the inheritance chain and finally (silently) absorbed by BaseOperator,
so there was no warning.

I think a workaround should be straightforward -- when you call
super().init for the BaseOperator, just pass arguments explicitly
rather than with args/kwargs, or (alternatively), pop arguments out of
kwargs when you use them ahead of calling that init.

Our case might be a little harder because we're subclassing PythonOperator not BaseOperator directly, but we can start at this point to see how we want to workaround for Airflow 2.0. That being said, Airflow 2.0 probably isn't coming out for another... year?... so we have some time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant