Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds option for outputting a JSON log #42

Merged
merged 7 commits into from
Nov 16, 2023
Merged

Adds option for outputting a JSON log #42

merged 7 commits into from
Nov 16, 2023

Conversation

havok2063
Copy link
Contributor

@havok2063 havok2063 commented Nov 12, 2023

This PR adds a new option for outputting a JSON log instead of a human-readable log. JSON logs are easier to machine-read and parse for content, make searchable, etc, and can be used by ElasticSearch or DataDog log analyzers. To output a JSON log instead of a regular log, use log.start_file_logger(path, as_json=True). To output both a human and JSON log, use the log.start_file_logger(path, with_json=True) keyword instead.

The JSON log can be re-loaded with:

with open("my_log.json") as f:
    data = [json.loads(i) for i in f.readlines()]

Example JSON log for log.info('this is a test') is

{'message': 'this is a test',
 'timestamp': '2023-11-12T13:58:10.705419Z',
 'type': 'log',
 'level': 'INFO',
 'name': 'name',
 'msg': 'this is a test',
 'args': [],
 'levelname': 'INFO',
 'levelno': 20,
 'pathname': '<ipython-input-4-a23a84913843>',
 'filename': '<ipython-input-4-a23a84913843>',
 'module': '<ipython-input-4-a23a84913843>',
 'exc_info': None,
 'exc_text': None,
 'stack_info': None,
 'lineno': 1,
 'funcName': '<module>',
 'created': 1699815490.705419,
 'msecs': 705.4190635681152,
 'relativeCreated': 14039.128065109253,
 'thread': 4412151232,
 'threadName': 'MainThread',
 'processName': 'MainProcess',
 'process': 43655}

Example log output for log.exception(f"failed to reduce: {err}"), where err is a caught exception, is

{'message': 'failed to reduce: bad error',
 'exc_info': ["<class 'ValueError'>",
  'bad error',
  'File "<ipython-input-6-3b904e8ef568>", line 2, in <module>\n    raise ValueError(\'bad error\')'],
 'timestamp': '2023-11-12T13:58:16.490702Z',
 'type': 'log',
 'level': 'ERROR',
 'name': 'name',
 'msg': 'failed to reduce: bad error',
 'args': [],
 'levelname': 'ERROR',
 'levelno': 40,
 'pathname': '<ipython-input-6-3b904e8ef568>',
 'filename': '<ipython-input-6-3b904e8ef568>',
 'module': '<ipython-input-6-3b904e8ef568>',
 'exc_text': None,
 'stack_info': None,
 'lineno': 7,
 'funcName': '<module>',
 'created': 1699815496.4907022,
 'msecs': 490.70215225219727,
 'relativeCreated': 19824.411153793335,
 'thread': 4412151232,
 'threadName': 'MainThread',
 'processName': 'MainProcess',
 'process': 43655,
 'error': {'type': 'ValueError',
  'trace': 'Traceback (most recent call last):\n  File "<ipython-input-6-3b904e8ef568>", line 2, in <module>\n    raise ValueError(\'bad error\')\nValueError: bad error'}}

@havok2063 havok2063 added the enhancement New feature or request label Nov 12, 2023
@havok2063 havok2063 self-assigned this Nov 12, 2023
@havok2063 havok2063 requested a review from albireox as a code owner November 12, 2023 18:49
Copy link

codecov bot commented Nov 12, 2023

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (dc8a42a) 93.31% compared to head (dfec626) 92.47%.
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #42      +/-   ##
==========================================
- Coverage   93.31%   92.47%   -0.84%     
==========================================
  Files           5        5              
  Lines         508      544      +36     
  Branches      114      122       +8     
==========================================
+ Hits          474      503      +29     
- Misses         16       19       +3     
- Partials       18       22       +4     
Files Coverage Δ
src/sdsstools/logger.py 85.27% <82.93%> (-0.90%) ⬇️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@albireox albireox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks good, feel free to merge.

If you want to release a new version, you can update the changelog, change the version in pyproject.toml (remove the a0), then tag and push. The workflow will push it to PyPI. Then I normally bump the version to the next patch with a0.

@havok2063 havok2063 merged commit e048dfb into main Nov 16, 2023
6 checks passed
@havok2063 havok2063 deleted the json branch November 16, 2023 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants