Skip to content

Commit

Permalink
Complete run args; update REAME, and improve the test cases. (#18)
Browse files Browse the repository at this point in the history
  • Loading branch information
KenyonY authored May 6, 2023
1 parent ddc8d2f commit 73d0eb7
Show file tree
Hide file tree
Showing 10 changed files with 109 additions and 26 deletions.
3 changes: 1 addition & 2 deletions .github/workflows/run_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ on:
- main
pull_request:
branches:
- dev
- main

jobs:
Expand All @@ -26,7 +25,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install codecov pytest pytest-cov
python -m pip install codecov pytest pytest-cov psutil
python -m pip install .
- name: Run tests
run: |
Expand Down
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@
<a href="https://pypi.org/project/openai_forward/">
<img alt="pypi downloads" src="https://img.shields.io/pypi/dm/openai_forward">
</a>
<a href="https://codecov.io/gh/beidongjiedeguang/openai-forward">
<img alt="codecov" src="https://codecov.io/gh/beidongjiedeguang/openai-forward/branch/dev/graph/badge.svg">
</a>

</p>
This project is designed to solve the problem of some regions being unable to directly access OpenAI. The service is deployed on a server that can access the OpenAI API, and OpenAI requests are forwarded through the service, i.e. a reverse proxy service is set up.
Expand All @@ -44,6 +47,7 @@ Or, to put it another way, https://caloi.top/openai is equivalent to https://api
- [Deploy](#Deploy)
- [Service Usage](#Service-Usage)
- [Configuration](#Configuration)
- [Chat Log](#Chat-log)

# Features

Expand Down Expand Up @@ -190,3 +194,17 @@ refer to the `.env` file in the project root directory
|ROUTE_PREFIX| Route prefix | None |
| IP_WHITELIST | IP whitelist, separated by space. | None |
| IP_BLACKLIST | IP blacklist, separated by space. | None |

# Chat Log
The saved path is in the `Log/` directory under the current directory.
The chat log starts with `chat_` and is written to the file every 5 rounds by default.
The recording format is as follows:
```text
{'host': xxx, 'model': xxx, 'message': [{'user': xxx}, {'assistant': xxx}]}
{'assistant': xxx}
{'host': ...}
{'assistant': ...}
...
```
31 changes: 22 additions & 9 deletions README_ZH.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,6 @@

[//]: # ([![Downloads]&#40;https://static.pepy.tech/badge/openai-forward/month&#41;]&#40;https://pepy.tech/project/openai-forward&#41;)

[//]: # ( <a href="https://codecov.io/gh/beidongjiedeguang/openai-forward">)

[//]: # ( <img alt="codecov" src="https://codecov.io/gh/beidongjiedeguang/openai-forward/branch/dev/graph/badge.svg">)

[//]: # ( </a>)

<p align="center">
<a href="https://pypi.org/project/openai-forward/"><img src="https://img.shields.io/pypi/v/openai-forward?color=brightgreen" alt="PyPI version" ></a>
Expand All @@ -50,7 +45,9 @@
<a href="https://pypi.org/project/openai_forward/">
<img alt="pypi downloads" src="https://img.shields.io/pypi/dm/openai_forward">
</a>

<a href="https://codecov.io/gh/beidongjiedeguang/openai-forward">
<img alt="codecov" src="https://codecov.io/gh/beidongjiedeguang/openai-forward/branch/dev/graph/badge.svg">
</a>
</p>


Expand All @@ -67,11 +64,13 @@ api的服务器上,通过该服务转发OpenAI的请求。即搭建反向代
- [安装部署](#安装部署)
- [服务调用](#服务调用)
- [配置选项](#配置选项)
- [聊天日志](#聊天日志)

# Features

- [x] 支持转发OpenAI所有接口
- [x] 支持流式响应
- [x] 实时记录聊天记录(包括流式响应的聊天内容)
- [x] 支持默认api key(多api key 循环调用)
- [x] pip安装部署
- [x] docker部署
Expand Down Expand Up @@ -199,12 +198,26 @@ http://{ip}:{port}/v1/chat/completions
**环境变量配置项**
参考项目根目录下`.env`文件

| 环境变量 | 说明 | 默认值 |
| 环境变量 | 说明 | 默认值 |
|-----------------|--------------------------------|:------------------------:|
| OPENAI_API_KEY | 默认api key,支持多个默认api key, 以空格分割 ||
| OPENAI_BASE_URL | 转发base url | `https://api.openai.com` |
|LOG_CHAT| 是否记录聊天内容 | `true` |
|ROUTE_PREFIX| 路由前缀 ||
| LOG_CHAT | 是否记录聊天内容 | `true` |
| ROUTE_PREFIX | 路由前缀 ||
| IP_WHITELIST | ip白名单, 空格分开 ||
| IP_BLACKLIST | ip黑名单, 空格分开 ||


# 聊天日志
保存路径在当前目录下的`Log/`路径中。
聊天日志以 `chat_`开头, 默认每5轮对话写入一次文件
记录格式为
```text
{'host': xxx, 'model': xxx, 'message': [{'user': xxx}, {'assistant': xxx}]}
{'assistant': xxx}
{'host': ...}
{'assistant': ...}
...
```
2 changes: 1 addition & 1 deletion openai_forward/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
__version__ = "0.1.6"
__version__ = "0.1.7"

from dotenv import load_dotenv

Expand Down
35 changes: 34 additions & 1 deletion openai_forward/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,40 @@

class Cli:
@staticmethod
def run(port=8000, workers=1):
def run(port=8000,
workers=1,
api_key=None,
base_url='https://api.openai.com',
log_chat='true',
route_prefix=None,
ip_whitelist=None,
ip_blacklist=None,
):
""" Run forwarding serve.
Parameters
----------
port: int, default 8000
workers: int, default 1
api_key: str, default None
base_url: str, default 'https://api.openai.com'
log_chat: str, default 'true'
route_prefix: str, default None
ip_whitelist: str, default None
ip_blacklist: str, default None
"""
os.environ['OPENAI_BASE_URL'] = base_url
os.environ['LOG_CHAT'] = log_chat
if api_key:
os.environ['OPENAI_API_KEY'] = api_key
if route_prefix:
os.environ['ROUTE_PREFIX'] = route_prefix
if ip_whitelist:
os.environ['IP_WHITELIST'] = ip_whitelist
if ip_blacklist:
os.environ['IP_BLACKLIST'] = ip_blacklist

uvicorn.run(
app="openai_forward.app:app",
host="0.0.0.0",
Expand Down
1 change: 1 addition & 0 deletions openai_forward/_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ async def _reverse_proxy(cls, request: Request):
content=request.stream(),
timeout=cls.timeout,
)
logger.warning(f"{url=}")

try:
r = await client.send(req, stream=True)
Expand Down
13 changes: 2 additions & 11 deletions openai_forward/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,17 +27,8 @@ def emit(self, record):

def setting_log(log_name, multi_process=True):
# TODO 修复时区配置
time_zone = os.environ.get("TZ")
print(f"{time_zone=}")
if time_zone == "Asia/Shanghai":
import datetime
tz = pytz.timezone(time_zone)
loc_dt = tz.localize(datetime.datetime.now())
offset = loc_dt.utcoffset()
# tzname, offset = loc_dt.strftime("%Z::%z").split("::")
# offset = loc_dt.strftime("%z")
# os.environ['TZ'] = f"UTC-{offset}"
os.environ['TZ'] = f"UTC-8"
if os.environ.get("TZ") == "Asia/Shanghai":
os.environ['TZ'] = "UTC-8"
if hasattr(time, 'tzset'):
print(os.environ['TZ'])
time.tzset()
Expand Down
3 changes: 2 additions & 1 deletion tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ def openai() -> OpenaiBase:

class TestOpenai:

def teardown_method(self):
@staticmethod
def teardown_method():
OpenaiBase.IP_BLACKLIST = []
OpenaiBase.IP_WHITELIST = []
OpenaiBase._default_api_key_list = []
Expand Down
3 changes: 2 additions & 1 deletion tests/test_chat_save.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@
import pytest
from utils import rm


@pytest.fixture(scope="module")
def saver() -> ChatSaver:
return ChatSaver(save_interval=1, max_chat_size=2)


class TestChatSaver:

@classmethod
Expand All @@ -30,4 +32,3 @@ def test_add_chat(self, saver: ChatSaver):
def test_init_file(self):
saver = ChatSaver()
assert saver.chat_file.endswith("chat_3.txt")

26 changes: 26 additions & 0 deletions tests/test_http.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import os
import time
from sparrow.multiprocess import kill
import subprocess
import httpx
from utils import rm


class TestRun:
@classmethod
def setup_class(cls):
subprocess.Popen(["nohup", "openai-forward", "run", "--base_url", "https://caloi.top/openai"])
time.sleep(0.7)

@classmethod
def teardown_class(cls):
kill(8000)
rm("nohup.out")

def test_get_doc(self):
resp = httpx.get("http://localhost:8000/docs")
assert resp.is_success

def test_get_chat_completions(self):
resp = httpx.get("http://localhost:8000/v1/chat/completions")
assert resp.status_code == 401

0 comments on commit 73d0eb7

Please sign in to comment.