-
Notifications
You must be signed in to change notification settings - Fork 566
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle null bot name correctly when trying to find region in monitoring #4592
Conversation
@@ -625,6 +624,9 @@ def metrics_store(): | |||
|
|||
def _get_region(bot_name): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not going to work for batch jobs.
THere are two ways to resolve this:
- Include the region in the batch job names.
- use some kind of GCE metadata server or something similaar to determine the region.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's already broken for batch jobs btw, so LGTMing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
@@ -625,6 +624,9 @@ def metrics_store(): | |||
|
|||
def _get_region(bot_name): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's already broken for batch jobs btw, so LGTMing
…ng (#4592) This fixes the following error in app engine crons: ``` Failed to flush metrics: expected string or bytes-like object, got 'NoneType' Traceback (most recent call last): File "/layers/google.python.runtime/python/lib/python3.11/threading.py", line 1002, in _bootstrap self._bootstrap_inner() File "/layers/google.python.runtime/python/lib/python3.11/threading.py", line 1045, in _bootstrap_inner self.run() File "/layers/google.python.runtime/python/lib/python3.11/threading.py", line 982, in run self._target(*self._args, **self._kwargs) File "/srv/clusterfuzz/_internal/metrics/monitor.py", line 171, in _flush_loop self._flush_function() File "/srv/clusterfuzz/_internal/metrics/monitor.py", line 134, in _flush_metrics logs.error(f'Failed to flush metrics: {e}') LogError: Failed to flush metrics: expected string or bytes-like object, got 'NoneType' Traceback (most recent call last): File "/srv/clusterfuzz/_internal/metrics/monitor.py", line 118, in _flush_metrics metric.monitoring_v3_time_series(series, labels, start_time, end_time, File "/srv/clusterfuzz/_internal/metrics/monitor.py", line 343, in monitoring_v3_time_series self.monitoring_v3_metric(time_series.metric, labels) File "/srv/clusterfuzz/_internal/metrics/monitor.py", line 321, in monitoring_v3_metric metric.labels['region'] = _get_region(bot_name) ^^^^^^^^^^^^^^^^^^^^^ File "/srv/clusterfuzz/_internal/metrics/monitor.py", line 637, in _get_region if re.match(pattern['pattern'], bot_name): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/layers/google.python.runtime/python/lib/python3.11/re/__init__.py", line 166, in match return _compile(pattern, flags).match(string) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: expected string or bytes-like object, got 'NoneType' ```
This fixes the following error in app engine crons: