Skip to content

Commit

Permalink
Merge branch 'master' into no_forward_make_packet_error
Browse files Browse the repository at this point in the history
  • Loading branch information
methane authored Feb 29, 2024
2 parents 0468c7d + 54ce654 commit 406e153
Show file tree
Hide file tree
Showing 21 changed files with 709 additions and 707 deletions.
48 changes: 48 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Run test

on:
push:
branches:
- master
pull_request:

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Ruff
run: pipx install ruff
- name: Ruff check
run: ruff check
- name: Ruff format
run: ruff format --diff

test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12", "pypy3.9", "pypy3.10"]
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: "pip"
cache-dependency-path: requirements-dev.txt
- name: Install dependencies
run: python -m pip install -r requirements-dev.txt
- name: Run tests
run: pytest --cov=fluent

build:
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: pipx run build
- uses: actions/upload-artifact@v4
with:
name: dist
path: dist/
33 changes: 0 additions & 33 deletions .travis.yml

This file was deleted.

4 changes: 0 additions & 4 deletions MANIFEST.in

This file was deleted.

36 changes: 14 additions & 22 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,20 +1,12 @@
A Python structured logger for Fluentd
======================================

.. image:: https://travis-ci.org/fluent/fluent-logger-python.svg?branch=master
:target: https://travis-ci.org/fluent/fluent-logger-python
:alt: Build Status

.. image:: https://coveralls.io/repos/fluent/fluent-logger-python/badge.svg
:target: https://coveralls.io/r/fluent/fluent-logger-python
:alt: Coverage Status
A Python structured logger for Fluentd/Fluent Bit
=================================================

Many web/mobile applications generate huge amount of event logs (c,f.
login, logout, purchase, follow, etc). To analyze these event logs could
be really valuable for improving the service. However, the challenge is
collecting these logs easily and reliably.

`Fluentd <https://github.com/fluent/fluentd>`__ solves that problem by
`Fluentd <https://github.com/fluent/fluentd>`__ and `Fluent Bit <https://fluentbit.io/>`__ solves that problem by
having: easy installation, small footprint, plugins, reliable buffering,
log forwarding, etc.

Expand All @@ -24,10 +16,11 @@ Python application.
Requirements
------------

- Python 3.5+
- Python 3.7+
- ``msgpack``
- **IMPORTANT**: Version 0.8.0 is the last version supporting Python 2.6, 3.2 and 3.3
- **IMPORTANT**: Version 0.9.6 is the last version supporting Python 2.7 and 3.4
- **IMPORTANT**: Version 0.10.0 is the last version supporting Python 3.5 and 3.6

Installation
------------
Expand Down Expand Up @@ -366,23 +359,22 @@ that this doesn't happen, or it's acceptable for your application.
Testing
-------

Testing can be done using
`nose <https://nose.readthedocs.org/en/latest/>`__.

Release
-------

Need wheel package.
Testing can be done using `pytest <https://docs.pytest.org>`__.

.. code:: sh
$ pip install wheel
$ pytest tests
After that, type following command:
Release
-------

.. code:: sh
$ python setup.py clean sdist bdist_wheel upload
$ # Download dist.zip for release from GitHub Action artifact.
$ unzip -d dist dist.zip
$ pipx twine upload dist/*
Contributors
------------
Expand Down
1 change: 1 addition & 0 deletions fluent/__about__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
__version__ = "0.10.1dev1"
8 changes: 3 additions & 5 deletions fluent/asynchandler.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
# -*- coding: utf-8 -*-

from fluent import asyncsender
from fluent import handler


class FluentHandler(handler.FluentHandler):
'''
"""
Asynchronous Logging Handler for fluent.
'''
"""

def getSenderClass(self):
return asyncsender.FluentSender
Expand All @@ -18,7 +16,7 @@ def close(self):
try:
self.sender.close()
finally:
super(FluentHandler, self).close()
super().close()
finally:
self.release()

Expand Down
63 changes: 36 additions & 27 deletions fluent/asyncsender.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
# -*- coding: utf-8 -*-

import threading
from queue import Queue, Full, Empty

Expand All @@ -17,8 +15,7 @@


def _set_global_sender(sender): # pragma: no cover
""" [For testing] Function to set global sender directly
"""
"""[For testing] Function to set global sender directly"""
global _global_sender
_global_sender = sender

Expand All @@ -37,41 +34,53 @@ def close(): # pragma: no cover


class FluentSender(sender.FluentSender):
def __init__(self,
tag,
host='localhost',
port=24224,
bufmax=1 * 1024 * 1024,
timeout=3.0,
verbose=False,
buffer_overflow_handler=None,
nanosecond_precision=False,
msgpack_kwargs=None,
queue_maxsize=DEFAULT_QUEUE_MAXSIZE,
queue_circular=DEFAULT_QUEUE_CIRCULAR,
queue_overflow_handler=None,
**kwargs):
def __init__(
self,
tag,
host="localhost",
port=24224,
bufmax=1 * 1024 * 1024,
timeout=3.0,
verbose=False,
buffer_overflow_handler=None,
nanosecond_precision=False,
msgpack_kwargs=None,
queue_maxsize=DEFAULT_QUEUE_MAXSIZE,
queue_circular=DEFAULT_QUEUE_CIRCULAR,
queue_overflow_handler=None,
**kwargs,
):
"""
:param kwargs: This kwargs argument is not used in __init__. This will be removed in the next major version.
"""
super(FluentSender, self).__init__(tag=tag, host=host, port=port, bufmax=bufmax, timeout=timeout,
verbose=verbose, buffer_overflow_handler=buffer_overflow_handler,
nanosecond_precision=nanosecond_precision,
msgpack_kwargs=msgpack_kwargs,
**kwargs)
super().__init__(
tag=tag,
host=host,
port=port,
bufmax=bufmax,
timeout=timeout,
verbose=verbose,
buffer_overflow_handler=buffer_overflow_handler,
nanosecond_precision=nanosecond_precision,
msgpack_kwargs=msgpack_kwargs,
**kwargs,
)
self._queue_maxsize = queue_maxsize
self._queue_circular = queue_circular
if queue_circular and queue_overflow_handler:
self._queue_overflow_handler = queue_overflow_handler
else:
self._queue_overflow_handler = self._queue_overflow_handler_default

self._thread_guard = threading.Event() # This ensures visibility across all variables
self._thread_guard = (
threading.Event()
) # This ensures visibility across all variables
self._closed = False

self._queue = Queue(maxsize=queue_maxsize)
self._send_thread = threading.Thread(target=self._send_loop,
name="AsyncFluentSender %d" % id(self))
self._send_thread = threading.Thread(
target=self._send_loop, name="AsyncFluentSender %d" % id(self)
)
self._send_thread.daemon = True
self._send_thread.start()

Expand Down Expand Up @@ -121,7 +130,7 @@ def _send(self, bytes_):
return True

def _send_loop(self):
send_internal = super(FluentSender, self)._send_internal
send_internal = super()._send_internal

try:
while True:
Expand Down
10 changes: 4 additions & 6 deletions fluent/event.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
# -*- coding: utf-8 -*-

import time

from fluent import sender


class Event(object):
class Event:
def __init__(self, label, data, **kwargs):
assert isinstance(data, dict), 'data must be a dict'
sender_ = kwargs.get('sender', sender.get_global_sender())
timestamp = kwargs.get('time', int(time.time()))
assert isinstance(data, dict), "data must be a dict"
sender_ = kwargs.get("sender", sender.get_global_sender())
timestamp = kwargs.get("time", int(time.time()))
sender_.emit_with_time(label, timestamp, data)
Loading

0 comments on commit 406e153

Please sign in to comment.