Support for specifying the forwarding routing prefix. (#5)

* delete push dev rule

* add workflow dispatch

* add description

* Support for specifying the forwarding routing prefix.

.env

@@ -1,3 +1,4 @@
 OPENAI_API_KEY=""
 OPENAI_BASE_URL="https://api.openai.com"
-LOG_CHAT=false
+LOG_CHAT=true
+ROUTE_PREFIX=""

.github/workflows/python-publish.yml

@@ -1,6 +1,7 @@
 name: Upload Python Package
 
 on:
+  workflow_dispatch:
   release:
     types: [published]
 

.github/workflows/run_tests.yml

@@ -3,7 +3,6 @@ name: Run tests
 on:
   push :
     branches:
-      - dev
       - main
   pull_request:
     branches:

README.md

@@ -41,7 +41,7 @@ Test access: https://caloi.top/v1/chat/completions is equivalent to https://api.
 - [Features](#Features) 
 - [Usage](#Usage) 
 - [Service Deployment](#Service-Deployment) 
-- [Service Usage](#Service Usage) 
+- [Service Usage](#Service-Usage) 
 
 # Features 
 - [x] Supports forwarding of all OpenAI interfaces 
@@ -50,6 +50,8 @@ Test access: https://caloi.top/v1/chat/completions is equivalent to https://api.
 - [x] Supports default API key 
 - [x] pip installation and deployment 
 - [x] Docker deployment 
+- [x] Support for multiple worker processes
+- [x] Support for specifying the forwarding routing prefix.
 - [ ] Chat content security: Chat content streaming filtering 
 
 # Usage 
@@ -82,7 +84,7 @@ docker run -d -p 3000:3000 -e OPENAI_API_KEY="sk-******" -e CODE="<your password
 
 ### Using in a module 
 
-**Used in JS/TS** 
+**JS/TS** 
 ```typescript 
 import { Configuration } from "openai"; 
 
@@ -92,7 +94,7 @@ const configuration = new Configuration({
 
 }); 
 ``` 
-**Used in Python** 
+**Python** 
 ```python 
 import openai 
 openai.api_base = "https://caloi.top" 
@@ -102,7 +104,7 @@ openai.api_key = "sk-******"
 # Service Deployment 
 Two service deployment methods are provided, choose one 
 
-## Method 1: pip 
+## Use `pip` 
 **Installation** 
 ```bash 
 pip install openai-forward 
@@ -121,7 +123,7 @@ OPENAI_API_KEY="sk-xxx" openai_forward run --port=9999 --workers=1
 ``` 
 Note: If both the default API key and the API key passed in the request header exist, the API key in the request header will override the default API key. 
 
-## Method 2: Docker (recommended) 
+## Use Docker (recommended) 
 ```bash 
 docker run --name="openai-forward" -d -p 9999:8000 beidongjiedeguang/openai-forward:latest 
 ``` 

README_ZH.md

@@ -64,6 +64,8 @@
 - [x] 支持默认api key
 - [x] pip安装部署
 - [x] docker部署
+- [x] 支持多进程转发
+- [x] 支持指定转发路由前缀
 - [ ] 聊天内容安全：聊天内容流式过滤
 
 # Usage
@@ -96,7 +98,7 @@ docker run -d -p 3000:3000 -e OPENAI_API_KEY="sk-xxx" -e CODE="<your password>"
 
 ### 在模块中使用
 
-**Used in JS/TS**
+**JS/TS**
 ```typescript
 import { Configuration } from "openai";
 
@@ -106,7 +108,7 @@ const configuration = new Configuration({
 
 });
 ```
-**Used in Python**  
+**Python**  
 ```python
 import openai
 openai.api_base = "https://caloi.top"
@@ -116,7 +118,7 @@ openai.api_key = "sk-******"
 # 安装部署
 提供两种服务部署方式,选择一种即可
 
-## 方式一:  pip
+## pip
 **安装**
 ```bash
 pip install openai-forward
@@ -136,7 +138,7 @@ OPENAI_API_KEY="sk-xxx" openai_forward run --port=9999 --workers=1
 注: 如果既存在默认api key又在请求头中传入了api key，则以请求头中的api key会覆盖默认api key.
 
 
-## 方式二: Docker(推荐) 
+## Docker(推荐) 
 ```bash
 docker run --name="openai-forward" -d -p 9999:8000 beidongjiedeguang/openai-forward:latest 
 ```

openai_forward/_base.py

@@ -11,6 +11,12 @@ class OpenaiBase:
     default_api_key = os.environ.get("OPENAI_API_KEY", "")
     base_url = os.environ.get("OPENAI_BASE_URL", "https://api.openai.com")
     LOG_CHAT = os.environ.get("LOG_CHAT", "False").lower() == "true"
+    ROUTE_PREFIX = os.environ.get("ROUTE_PREFIX", "")
+    if ROUTE_PREFIX:
+        if ROUTE_PREFIX.endswith('/'):
+            ROUTE_PREFIX = ROUTE_PREFIX[:-1]
+        if not ROUTE_PREFIX.startswith('/'):
+            ROUTE_PREFIX = '/' + ROUTE_PREFIX
     stream_timeout = 20
     timeout = 30
     non_stream_timeout = 30
@@ -48,7 +54,9 @@ async def aiter_bytes(cls, r: httpx.Response):
     @classmethod
     async def _reverse_proxy(cls, request: Request):
         client: httpx.AsyncClient = request.app.state.client
-        url = httpx.URL(path=request.url.path, query=request.url.query.encode('utf-8'))
+        url_path = request.url.path
+        url_path = url_path[len(cls.ROUTE_PREFIX):]
+        url = httpx.URL(path=url_path, query=request.url.query.encode('utf-8'))
         headers = dict(request.headers)
         auth = headers.pop("authorization", None)
         if auth and str(auth).startswith("Bearer sk-"):

openai_forward/app.py

@@ -18,6 +18,6 @@ async def shutdown_event():
     await app.state.client.aclose()
 
 
-app.add_route('/{api_path:path}', openai.reverse_proxy,
-              ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS', 'HEAD', 'PATCH', 'TRACE'])
+app.add_route(openai.ROUTE_PREFIX+'/{api_path:path}', openai.reverse_proxy,
+              methods=['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS', 'HEAD', 'PATCH', 'TRACE'])
 app.include_router(router_v1)

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "openai_forward"
-description = ""
+description = "🚀 Openai api forward · OpenAI 接口转发服务 · streaming forward"
 authors = [
     { name = "kunyuan", email = "[email protected]" },
 ]