webserver needs restarting after scheduler restart?

[7yl4r]

The scheduler goes down occasionally (I think intentionally b/c of a timeout):

May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: [2018-05-29 20:38:55,342] {jobs.py:895} INFO - Examining DAG run <DagRun proc_myd01_to_myd0_otis_l2_gom @ 2003-11-08 17:55:00: trig__2003-11-08T17:55:00, externally triggere
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: [2018-05-29 20:38:55,398] {models.py:4584} INFO - Updating state for <DagRun proc_myd01_to_myd0_otis_l2_gom @ 2003-11-08 17:55:00: trig__2003-11-08T17:55:00, externally trig
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: [2018-05-29 20:38:55,463] {jobs.py:895} INFO - Examining DAG run <DagRun proc_myd01_to_myd0_otis_l2_gom @ 2003-11-08 19:35:00: trig__2003-11-08T19:35:00, externally triggere
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: [2018-05-29 20:38:55,548] {models.py:4584} INFO - Updating state for <DagRun proc_myd01_to_myd0_otis_l2_gom @ 2003-11-08 19:35:00: trig__2003-11-08T19:35:00, externally trig
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: [2018-05-29 20:38:55,567] {jobs.py:379} ERROR - Got an exception! Propagating...
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: Traceback (most recent call last):
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib/python3.4/site-packages/apache_airflow-1.9.0-py3.4.egg/airflow/jobs.py", line 371, in helper
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: pickle_dags)
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib/python3.4/site-packages/apache_airflow-1.9.0-py3.4.egg/airflow/utils/db.py", line 50, in wrapper
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: result = func(*args, **kwargs)
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib/python3.4/site-packages/apache_airflow-1.9.0-py3.4.egg/airflow/jobs.py", line 1792, in process_file
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: self._process_dags(dagbag, dags, ti_keys_to_schedule)
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib/python3.4/site-packages/apache_airflow-1.9.0-py3.4.egg/airflow/jobs.py", line 1391, in _process_dags
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: self._process_task_instances(dag, tis_out)
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib/python3.4/site-packages/apache_airflow-1.9.0-py3.4.egg/airflow/jobs.py", line 916, in _process_task_instances
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: run.update_state(session=session)
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib/python3.4/site-packages/apache_airflow-1.9.0-py3.4.egg/airflow/utils/db.py", line 50, in wrapper
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: result = func(*args, **kwargs)
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib/python3.4/site-packages/apache_airflow-1.9.0-py3.4.egg/airflow/models.py", line 4615, in update_state
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: if deps_met or old_state != ut.current_state(session=session):
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib/python3.4/site-packages/apache_airflow-1.9.0-py3.4.egg/airflow/utils/db.py", line 50, in wrapper
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: result = func(*args, **kwargs)
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib/python3.4/site-packages/apache_airflow-1.9.0-py3.4.egg/airflow/models.py", line 1022, in current_state
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: TI.execution_date == self.execution_date,
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib64/python3.4/site-packages/sqlalchemy/orm/query.py", line 2703, in all
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: return list(self)
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib64/python3.4/site-packages/sqlalchemy/orm/query.py", line 2851, in __iter__
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: context = self._compile_context()
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib64/python3.4/site-packages/sqlalchemy/orm/query.py", line 3365, in _compile_context
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: entity.setup_context(self, context)
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib64/python3.4/site-packages/sqlalchemy/orm/query.py", line 3727, in setup_context
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: polymorphic_discriminator=self._polymorphic_discriminator)
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib64/python3.4/site-packages/sqlalchemy/orm/loading.py", line 251, in _setup_entity_query
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: value.setup(
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: File "/usr/lib/python3.4/site-packages/apache_airflow-1.9.0-py3.4.egg/airflow/bin/cli.py", line 73, in sigint_handler
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: sys.exit(0)
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: SystemExit: 0
May 29 20:38:55 imars-airflow01.marine.usf.edu airflow[22284]: [2018-05-29 20:38:55,592] {jobs.py:1708} INFO - Deactivating DAGs that haven't been touched since 2018-05-29T20:28:55.038009
May 29 20:38:56 imars-airflow01.marine.usf.edu airflow[22284]: [2018-05-29 20:38:56,328] {jobs.py:1546} INFO - Exited execute loop
May 29 20:38:56 imars-airflow01.marine.usf.edu airflow[22284]: [2018-05-29 20:38:56,366] {jobs.py:1564} INFO - Waiting up to 5 seconds for processes to exit...
May 29 20:39:01 imars-airflow01.marine.usf.edu systemd[1]: airflow-scheduler.service holdoff time over, scheduling restart.
May 29 20:39:01 imars-airflow01.marine.usf.edu systemd[1]: Started Airflow scheduler daemon.

When this happens the webserver stops responding until it is also restarted via systemctl restart airflow-webserver.