Skip to content

Latest commit

 

History

History
119 lines (84 loc) · 4.56 KB

deploy_en.md

File metadata and controls

119 lines (84 loc) · 4.56 KB
Raw


Deployment Guide

Deploy with one click to Render
Deploy to Render

pip Deployment | docker Deployment | Render One-click Deployment

This document offers several deployment methods:
Local Deployment

  1. Deployment via pip
  2. Deployment with Docker

One-click Free Cloud Deployment

  1. Render One-click Deployment
  2. CloudFlare AI Gateway
  3. More Deployment: https://github.com/KenyonY/openai-forward/blob/0.5.x/deploy.md

pip Deployment

Installation

pip install openai-forward

Run the Service

aifd run

The service is now set up.
Configuration can be found here.

Service Invocation

To use, simply replace https://api.openai.com with the service port http://{ip}:{port}
For example:

# Default
https://api.openai.com/v1/chat/completions
# Replace with
http://{ip}:{port}/v1/chat/completions

Enable SSL (to access the domain via https)

First, prepare a domain, such as the one used in this project: api.openai-forward.com.

A common method is to use nginx (if you're not familiar with command-line configuration, you might consider Nginx Proxy Manager, which facilitates setting up Let's Encrypt certificates with automatic application and renewal) or Caddy to proxy forward the openai-forward service port (default 8000) to the HTTPS port.
Note that if you want to use streaming forwarding, you need to add a configuration to disable proxy caching in nginx, i.e., input in Nginx Proxy Manager's Custom Nginx Configuration:

proxy_buffering off;

Docker Deployment

docker run -d -p 8000:8000 beidongjiedeguang/openai-forward:latest

This will map the host's 8000 port. Access the service via http://{ip}:8000.
The log path inside the container is /home/openai-forward/Log/. It can be mapped when starting up.

Note: For SSL setup, refer to the above. Environment variable configuration can be found here.

Source Code Deployment

git clone https://github.com/KenyonY/openai-forward.git --depth=1
cd openai-forward

pip install -e .
aifd run

For SSL setup, refer to the above.

Render One-click Deployment

Deploy to Render

Render might be considered the easiest of all deployment methods, and the domain it generates can be directly accessed domestically!

  1. Click the one-click deployment button.
    Alternatively, first fork this repository -> Go to Render's Dashboard -> New Web Services -> Connect to the recently forked repository. Follow the default steps thereafter.
  2. Fill in the environment variables. All openai-forward configurations can be set via environment variables. Set them based on your needs.

Wait for the deployment to complete.
Render's free plan: 750 hours of free instance time each month (meaning a single instance can run continuously), 100G bandwidth, and 500 minutes of build duration.

Note: By default, Render will automatically go to sleep if there are no service requests within 15 minutes (the advantage is that once asleep, it won't count towards the 750h of free instance time). The next request after it goes to sleep will be blocked for ~15s. If you want the service not to go to sleep after 15 minutes, you can use a timer script (like every 14 minutes) to keep the Render service alive. Refer to the scripts/keep_render_alive.py for the keep-alive script.
For zero-downtime deployments, configure the Health Check Path in Render settings to /healthz.

https://render.openai-forward.com
https://openai-forward.onrender.com

This is a translated version. Make sure to review and adjust any sections as necessary to fit your specific context or technical requirements.