Skip to content

Commit

Permalink
Merge pull request #3321 from tashrifbillah/master
Browse files Browse the repository at this point in the history
/history/by_task_id : a method to link each task's history uniquely to the SQL database
  • Loading branch information
dlstadther authored Nov 12, 2024
2 parents 829fc0c + a0c6604 commit 172128c
Show file tree
Hide file tree
Showing 4 changed files with 26 additions and 0 deletions.
11 changes: 11 additions & 0 deletions doc/central_scheduler.rst
Original file line number Diff line number Diff line change
Expand Up @@ -95,3 +95,14 @@ The task history has the following pages:
a listing of all runs of the task ``{name}`` restricted to runs with ``params`` matching the given history.
The ``params`` is a json blob describing the parameters,
e.g. ``data={"foo": "bar"}`` looks for a task with ``foo=bar``.
* ``/history/by_task_id/{task_id}``
the latest run of a task given the ``{task_id}``. It is different from just ``{id}``
and is a derivative of ``params``. It is available via ``{task_id}`` property of a
``luigi.Task`` instance or via `luigi.task.task_id_str
<https://luigi.readthedocs.io/en/stable/api/luigi.task.html#luigi.task.task_id_str>`_.
This kind of representation is useful for concisely recording URLs in a history tree.
Example screenshot:

.. figure:: history_by_task_id.png
:alt: By task_id screenshot

Binary file added doc/history_by_task_id.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 7 additions & 0 deletions luigi/db_task_history.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,13 @@ def find_task_by_id(self, id, session=None):
with self._session(session) as session:
return session.query(TaskRecord).get(id)

def find_task_by_task_id(self, task_id, session=None):
"""
Find task with the given task ID.
"""
with self._session(session) as session:
return session.query(TaskRecord).filter(TaskRecord.task_id == task_id).all()[-1]


class TaskParameter(Base): # type: ignore
"""
Expand Down
8 changes: 8 additions & 0 deletions luigi/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,13 @@ def get(self, id):
self.render("show.html", task=task)


class ByTaskIdHandler(BaseTaskHistoryHandler):
def get(self, task_id):
with self._scheduler.task_history._session(None) as session:
task = self._scheduler.task_history.find_task_by_task_id(task_id, session)
self.render("show.html", task=task)


class ByParamsHandler(BaseTaskHistoryHandler):
def get(self, name):
payload = self.get_argument('data', default="{}")
Expand Down Expand Up @@ -314,6 +321,7 @@ def app(scheduler):
(r'/history', RecentRunHandler, {'scheduler': scheduler}),
(r'/history/by_name/(.*?)', ByNameHandler, {'scheduler': scheduler}),
(r'/history/by_id/(.*?)', ByIdHandler, {'scheduler': scheduler}),
(r'/history/by_task_id/(.*?)', ByTaskIdHandler, {'scheduler': scheduler}),
(r'/history/by_params/(.*?)', ByParamsHandler, {'scheduler': scheduler}),
(r'/metrics', MetricsHandler, {'scheduler': scheduler})
]
Expand Down

0 comments on commit 172128c

Please sign in to comment.