Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send errors upstream (to client) and terminate #5

Open
tskluzac opened this issue Oct 31, 2021 · 0 comments
Open

Send errors upstream (to client) and terminate #5

tskluzac opened this issue Oct 31, 2021 · 0 comments

Comments

@tskluzac
Copy link
Contributor

We need a means of terminating (and notifying users) for the following reasons in the crawler_worker:

  • bad Globus Auth credentials
  • misspelled or incorrect file paths
  • any GlobusError that has been retried n (=10?) times without success.
  • SQS errors that have been retried n (=10?) times without success.

I think this update should happen via the crawl-worker updating the postgres entry for the crawl by having it update status to 'error' and ALSO add a descriptive free text 'message' column (that if things are running fine could just be something like 'ok') that can describe reason for error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant