Report user_id to New Relic for error tracking (#27)

* Report user_id to New Relic for error tracking

* Log user id reporting to NR

README.md

@@ -21,15 +21,18 @@ should subclass to implement your pure python jobs. It basically replicates
 the [delayed_job](https://github.com/collectiveidea/delayed_job) Ruby gem behavior.
 
 ## Installation
-### From Pypi:
+
+### From PyPI
 
     pip install rubydj-pyworker
 
-You can also pick up any version from [Pypi](https://pypi.org/project/rubydj-pyworker/)
+You can also pick up any version from [PyPI](https://pypi.org/project/rubydj-pyworker/)
 that supports Python 3.x (>= 1.0.0).
 
-### From Github branch:
+### From Github branch
+
     pip install git+https://github.com/rayyansys/pyworker.git@<branch_name>#egg=rubydj-pyworker
+
 ## Usage
 
 The simplest usage is creating a worker and running it:
@@ -117,7 +120,7 @@ w.run()
 
 ## Monitoring
 
-Workers can be monitored using [Newrelic](https://newrelic.com/). All you need
+Workers can be monitored using [New Relic](https://newrelic.com/). All you need
 to do is to create a free account there, then add the following to your
 environment variables:
 
@@ -129,6 +132,7 @@ NEW_RELIC_APP_NAME=<your_newrelic_app_name>
 All jobs will be reported under the `BackgroundTask` category. This includes
 standard metrics like throughput, response time (job duration), error rate, etc.
 Additional transaction custom attributes are also reported out of the box:
+
 1. `jobName`: the name of the job class
 1. `jobId`: the id of the delayed job in the database
 1. `jobQueue`: the queue name of the job
@@ -179,11 +183,20 @@ as the camel case conversion. If you wish to skip both, you can use the
 `report_raw` function instead of `report`.
 
 In all cases, two additional attributes are reported:
+
 1. `error`: a Boolean indicating whether the job has failed or not
 1. `jobFailure`: a Boolean indicating whether the job has permanently failed or not
 
 The `error` attribute is reported as is, i.e. without prefix or camel case conversion.
 
+Additionally, the end user id is reported if the custom attributes contain the id
+in any of the below formats:
+
+1. `user_id` (with and without prefix)
+1. `userId` (with and without prefix)
+
+This is useful in identifying impacted users count in case of job errors.
+
 ## Limitations
 
 - Only supports Postgres databases
@@ -213,10 +226,10 @@ Do your changes, then send a pull request.
 1. Increment the version number in `setup.py`
 1. Install `twine` and `wheel`: `pip install twine wheel`. You may need to upgrade pip first: `pip install --upgrade pip`
 1. Create the distribution files: `python setup.py sdist bdist_wheel`
-1. Optionally upload to [Test PyPi](https://test.pypi.org/) as a dry run: `twine upload -r testpypi dist/*`. You will need a separate account there
-1. Upload to [PyPi](https://pypi.org/): `twine upload dist/*`
+1. Optionally upload to [Test PyPI](https://test.pypi.org/) as a dry run: `twine upload -r testpypi dist/*`. You will need a separate account there
+1. Upload to [PyPI](https://pypi.org/): `twine upload dist/*`
 
-Enter your PyPi username and password when prompted.
+Enter your PyPI username and password when prompted.
 
 ## License
 

pyworker/reporter.py

@@ -71,6 +71,14 @@ def _convert_value(value):
     def _report_newrelic(self, attributes):
         if self._logger:
             self._logger.debug('Reporter: reporting to NewRelic: %s' % attributes)
+        # report user id if available in attributes in any form
+        possible_keys = [f'{self._prefix}userId', f'{self._prefix}user_id', 'userId', 'user_id']
+        possible_values = [attributes.get(key) for key in possible_keys if key in attributes]
+        if possible_values:
+            user_id = str(possible_values[0])
+            newrelic.agent.set_user_id(user_id)
+            if self._logger:
+                self._logger.debug('Reporter: reporting to NewRelic user_id: %s' % user_id)
         # convert attributes dict to list of tuples
         attributes = list(attributes.items())
         newrelic.agent.add_custom_attributes(attributes)

tests/test_reporter.py

@@ -187,7 +187,7 @@ def test_reporter_convert_value_converts_value_to_json_if_not_supported(self):
     #********** ._report_newrelic tests **********#
 
     @patch('pyworker.reporter.newrelic.agent')
-    def test_reporter_report_newrelic_calls_newrelic_record_exception(self, newrelic_agent):
+    def test_reporter_report_newrelic_calls_newrelic_add_custom_attributes(self, newrelic_agent):
         reporter = Reporter()
         reporter._report_newrelic({
             'test_key1': 'test_value',
@@ -198,3 +198,57 @@ def test_reporter_report_newrelic_calls_newrelic_record_exception(self, newrelic
             ('test_key1', 'test_value'),
             ('test_key2', 2)
         ])
+
+    @patch('pyworker.reporter.newrelic.agent')
+    def test_reporter_report_newrelic_does_not_call_newrelic_set_user_id_if_user_id_not_in_attributes(self, newrelic_agent):
+        reporter = Reporter()
+        reporter._report_newrelic({
+            'test_key1': 'test_value',
+            'test_key2': 2
+        })
+
+        newrelic_agent.set_user_id.assert_not_called()
+
+    @patch('pyworker.reporter.newrelic.agent')
+    def test_reporter_report_newrelic_calls_newrelic_set_user_id_if_user_id_in_attributes(self, newrelic_agent):
+        reporter = Reporter()
+        reporter._report_newrelic({
+            'test_key1': 'test_value',
+            'test_key2': 2,
+            'user_id': 123
+        })
+
+        newrelic_agent.set_user_id.assert_called_once_with('123')
+
+    @patch('pyworker.reporter.newrelic.agent')
+    def test_reporter_report_newrelic_calls_newrelic_set_user_id_if_userId_in_attributes(self, newrelic_agent):
+        reporter = Reporter()
+        reporter._report_newrelic({
+            'test_key1': 'test_value',
+            'test_key2': 2,
+            'userId': 123
+        })
+
+        newrelic_agent.set_user_id.assert_called_once_with('123')
+
+    @patch('pyworker.reporter.newrelic.agent')
+    def test_reporter_report_newrelic_calls_newrelic_set_user_id_if_prefixed_user_id_in_attributes(self, newrelic_agent):
+        reporter = Reporter(attribute_prefix='test_')
+        reporter._report_newrelic({
+            'test_key1': 'test_value',
+            'test_key2': 2,
+            'test_user_id': 123
+        })
+
+        newrelic_agent.set_user_id.assert_called_once_with('123')
+
+    @patch('pyworker.reporter.newrelic.agent')
+    def test_reporter_report_newrelic_calls_newrelic_set_user_id_if_prefixed_userId_in_attributes(self, newrelic_agent):
+        reporter = Reporter(attribute_prefix='test_')
+        reporter._report_newrelic({
+            'test_key1': 'test_value',
+            'test_key2': 2,
+            'test_userId': 123
+        })
+
+        newrelic_agent.set_user_id.assert_called_once_with('123')