Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update: pytest for unit tests, bug fixes. #3

Merged
merged 3 commits into from
Dec 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[run]
omit = tests/*
23 changes: 23 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# http://editorconfig.org

root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[*.{py,rst,ini}]
indent_style = space
indent_size = 4

[*.{html,css,scss,json,yml,xml}]
indent_style = space
indent_size = 2

[*.md]
trim_trailing_whitespace = false

[Makefile]
indent_style = tab
17 changes: 11 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@ jobs:
build:
runs-on: ubuntu-latest
strategy:
max-parallel: 4
fail-fast: false
max-parallel: 3
matrix:
python-version: ['3.9', '3.10', '3.11']
python-version: ['3.9', '3.10', '3.11', '3.12', '3.13']
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
Expand All @@ -23,11 +24,15 @@ jobs:

- name: Install Dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
make init

- name: Run pre-commit
uses: pre-commit/[email protected]
- name: Lint
run: |
make lint

- name: Tests
run: |
make pytest

deploy:
needs: build
Expand Down
31 changes: 30 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,35 @@
Release History
===============

1.0.3 (2024-12-11)
-------------------
**Improvements:**

- Add unit tests.
- Improve document.

**Bugfixes:**

- Fix timeout.
- Fix missing port redirection.


1.0.3 (2024-12-05)
-------------------
**Improvements**

- improve document.

**Bugfixes**

- Fix multipart encoders, cross share auth.

1.0.2 (2024-12-05)
-------------------
**Improvements**
- Download specific TLS library versions.
- Add a document.

1.0.1 (2024-12-04)
-------------------
* First release
- First release
27 changes: 23 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
.PHONY: docs
init:
python -m pip install -r requirements.txt
python -m pip install --upgrade pip
python -m pip install -r requirements-dev.txt

test:
tox -p
rm -rf *.egg-info

test-readme:
python setup.py check --restructuredtext --strict && ([ $$? -eq 0 ] && echo "README.rst and CHANGELOG.md ok") || echo "Invalid markup in README.md or CHANGELOG.md!"
Expand All @@ -10,11 +15,25 @@ lint:
python -m isort tls_requests
python -m flake8 tls_requests

pytest:
python -m pytest tests

coverage:
python -m pytest --cov-config .coveragerc --verbose --cov-report term --cov-report xml --cov=tls_requests tests

docs:
mkdocs serve

publish-test-pypi:
python -m pip install -r requirements-dev.txt
python -m pip install 'twine>=6.0.1'
python setup.py sdist bdist_wheel
twine upload --repository testpypi dist/*
rm -rf build dist .egg wrapper_tls_requests.egg-info

publish-pypi:
python -m pip install -r requirements-dev.txt
python -m pip install 'twine>=6.0.1'
python setup.py sdist bdist_wheel
twine upload dist/*
rm -rf build dist .egg wrapper_tls_requests.egg-info

docs:
mkdocs serve
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# TLS REQUESTS
**A powerful and lightweight Python library for making secure and reliable HTTP/TLS Fingerprint requests.**
# TLS Requests
TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS fingerprinting, anti-bot bypassing, and high performance.

* * *

Expand Down
2 changes: 1 addition & 1 deletion docs/advanced/hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ client.hooks = {
Best Practices
--------------

1. **Access Content**: Use `.read()` or `await read()` in asynchronous contexts to access `response.content` before returning it.
1. **Access Content**: Use `.read()` or `await .aread()` in asynchronous contexts to access `response.content` before returning it.
2. **Always Use Lists:** Hooks must be registered as **lists of callables**, even if you are adding only one function.
3. **Combine Hooks:** You can register multiple hooks for the same event type to handle various concerns, such as logging and error handling.
4. **Order Matters:** Hooks are executed in the order they are registered.
Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
[build-system]
requires = ['setuptools>=40.8.0']
build-backend = 'setuptools.build_meta'

[tool.pytest.ini_options]
asyncio_mode = "auto"
25 changes: 25 additions & 0 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
-r requirements.txt

# Documentation
mkdocs==1.6.1
mkautodoc==0.2.0
mkdocs-material==9.5.39

# Packaging
setuptools~=75.3.0
twine~=6.0.1

# Tests & Linting
pre-commit==3.7.0
black==24.3.0
coverage[toml]==7.6.1
pre-commit==3.7.0
isort==5.13.2
flake8==7.1.1
mypy==1.11.2
pytest==8.3.3
pytest-asyncio==0.24.0
pytest-cov==6.0.0
pytest_httpserver==1.1.0
Werkzeug==3.1.3
tox==4.23.2
20 changes: 1 addition & 19 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,4 @@
chardet~=5.2.0
requests~=2.32.3
tqdm~=4.67.1

# Documentation
mkdocs==1.6.1
mkautodoc==0.2.0
mkdocs-material==9.5.39

# Packaging
setuptools~=75.3.0
twine~=6.0.1

# Tests & Linting
pre-commit==3.7.0
black==24.3.0
flake8==7.0.0
coverage[toml]==7.6.1
pre-commit==3.7.0
isort==5.13.2
mypy==1.11.2
pytest==8.3.3
idna~=3.10
3 changes: 2 additions & 1 deletion setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ classifiers =
Programming Language :: Python :: 3.10
Programming Language :: Python :: 3.11
Programming Language :: Python :: 3.12
Programming Language :: Python :: 3.13
Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: Implementation :: CPython
Programming Language :: Python :: Implementation :: PyPy
Expand All @@ -29,7 +30,7 @@ classifiers =

project_urls =
Changelog = https://github.com/thewebscraping/tls-requests/blob/main/CHANGELOG.md
Documentation = https://github.com/thewebscraping/tls-requests
Documentation = https://thewebscraping.github.io/tls-requests/
Source = https://github.com/thewebscraping/tls-requests
Homepage = https://github.com/thewebscraping/tls-requests

Expand Down
7 changes: 7 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import tls_requests

pytest_plugins = ['pytest_httpserver', 'pytest_asyncio']


def pytest_configure(config):
tls_requests.TLSLibrary.load()
Empty file added tests/files/__init__.py
Empty file.
Binary file added tests/files/coingecko.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
52 changes: 52 additions & 0 deletions tests/test_api.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
import tls_requests

RESPONSE_BYTE = b"Hello World!"
RESPONSE_TEXT = "Hello World!"


def assert_response(response):
assert response.status_code, 200
assert response.reason, "OK"
assert response.text, RESPONSE_TEXT
assert response.content, RESPONSE_BYTE


def make_request(request_fn, httpserver, is_assert_response: bool = True):
httpserver.expect_request("/api").respond_with_data(RESPONSE_BYTE)
response = request_fn(httpserver.url_for('/api'))
if is_assert_response:
assert_response(response)

return response


def test_get(httpserver):
make_request(tls_requests.get, httpserver)


def test_post(httpserver):
make_request(tls_requests.post, httpserver)


def test_put(httpserver):
make_request(tls_requests.put, httpserver)


def test_patch(httpserver):
make_request(tls_requests.patch, httpserver)


def test_delete(httpserver):
make_request(tls_requests.delete, httpserver)


def test_options(httpserver):
make_request(tls_requests.options, httpserver)


def test_head(httpserver):
response = make_request(tls_requests.head, httpserver, False)
assert response.status_code == 200
assert response.reason == "OK"
assert response.text == ""
assert response.content == b""
102 changes: 102 additions & 0 deletions tests/test_auth.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
from base64 import b64encode

import pytest

import tls_requests

auth = ("user", "pass")
AUTH_TOKEN = "Basic %s" % b64encode(b":".join([s.encode() for s in auth])).decode()
AUTH_HEADERS = {"authorization": AUTH_TOKEN}
AUTH_FUNCTION_KEY = "x-authorization"
AUTH_FUNCTION_VALUE = "123456"
AUTH_FUNCTION_HEADERS = {AUTH_FUNCTION_KEY: AUTH_FUNCTION_VALUE}


def auth_function(request):
request.headers.update(AUTH_FUNCTION_HEADERS)


@pytest.fixture
def auth_url(httpserver):
return httpserver.url_for('/auth')


@pytest.fixture
def http_auth_function(httpserver):
httpserver.expect_request("/auth", headers=AUTH_FUNCTION_HEADERS).respond_with_data()
return httpserver


@pytest.fixture
def http_auth(httpserver):
httpserver.expect_request("/auth", headers=AUTH_HEADERS).respond_with_data()
return httpserver


def test_auth(http_auth, auth_url):
response = tls_requests.get(auth_url, auth=auth)
assert response.status_code == 200
assert response.request.headers["Authorization"] == AUTH_TOKEN


def test_auth_function(http_auth_function, auth_url):
response = tls_requests.get(auth_url, auth=auth_function)
assert response.status_code == 200
assert response.request.headers[AUTH_FUNCTION_KEY] == AUTH_FUNCTION_VALUE


def test_client_auth(http_auth, auth_url):
with tls_requests.Client(auth=auth) as client:
response = client.get(auth_url)

assert response.status_code == 200
assert bool(response.closed == client.closed) is True
assert response.request.headers["Authorization"] == AUTH_TOKEN


def test_client_auth_cross_sharing(http_auth, auth_url):
with tls_requests.Client(auth=('1', '2')) as client:
response = client.get(auth_url, auth=auth)

assert response.status_code == 200
assert bool(response.closed == client.closed) is True
assert response.request.headers["Authorization"] == AUTH_TOKEN


def test_client_auth_function_cross_sharing(http_auth_function, auth_url):
with tls_requests.Client(auth=auth) as client:
response = client.get(auth_url, auth=auth_function)

assert response.status_code == 200
assert bool(response.closed == client.closed) is True
assert response.request.headers[AUTH_FUNCTION_KEY] == AUTH_FUNCTION_VALUE


@pytest.mark.asyncio
async def test_async_auth(http_auth, auth_url):
async with tls_requests.AsyncClient(auth=auth) as client:
response = await client.get(auth_url)

assert response.status_code == 200
assert bool(response.closed == client.closed) is True
assert response.request.headers["Authorization"] == AUTH_TOKEN


@pytest.mark.asyncio
async def test_async_auth_function(http_auth_function, auth_url):
async with tls_requests.AsyncClient(auth=auth_function) as client:
response = await client.get(auth_url)

assert response.status_code == 200
assert bool(response.closed == client.closed) is True
assert response.request.headers[AUTH_FUNCTION_KEY] == AUTH_FUNCTION_VALUE


@pytest.mark.asyncio
async def test_async_auth_function_cross_sharing(http_auth_function, auth_url):
async with tls_requests.AsyncClient(auth=auth) as client:
response = await client.get(auth_url, auth=auth_function)

assert response.status_code == 200
assert bool(response.closed == client.closed) is True
assert response.request.headers[AUTH_FUNCTION_KEY] == AUTH_FUNCTION_VALUE
Loading
Loading