forked from reanahub/reana-workflow-controller
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(opensearch): capture logs from Dask cluster pods (reanahub#616)
This commit collects logs from Dask scheduler and workers and propagates\nthem to all REANA jobs that are using the same Dask cluster. This is not\nideal, since Dask logs can become thusly duplicated for different\nworkflow steps of the workflow, which could be confusing for the user.\n\nHowever, when a user uses Dask to parallelise the workflow jobs, usually\nthe workflow steps are defined only within Dask, so this situation does\nnot occur. Hence we can afford doing this in usual real-life conditions.\n\nSeparating Dask scheduler and worker logs from regular Kubernetes job\nlogs would require a larger architectural change and is therefore\ndeferred to a future commit. Closes reanahub#610
- Loading branch information
Showing
2 changed files
with
116 additions
and
14 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters