Skip to content

Commit

Permalink
Adding CI/CD
Browse files Browse the repository at this point in the history
  • Loading branch information
justinwlin committed Feb 18, 2024
1 parent e505a8e commit 11e98f1
Show file tree
Hide file tree
Showing 2 changed files with 56 additions and 3 deletions.
45 changes: 45 additions & 0 deletions .github/workflows/CI-Docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: CD | Docker-Build-Release

on:
push:
branches:
- "main"
release:
types: [published]
workflow_dispatch:

jobs:
docker-image:
runs-on: ubuntu-latest
strategy:
matrix:
include:
- repository: justinwlin/mistral7b_openllm
model: mistralai/Mistral-7B-Instruct-v0.2
tag: latest
- repository: justinwlin/mixtral_8x7b_openllm
model: mistralai/Mixtral-8x7B-v0.1
tag: latest
- repository: justinwlin/llama2_13b_openllm
model: NousResearch/llama-2-13b-hf
tag: latest
- repository: justinwlin/llama2_70b_openllm
model: NousResearch/llama-2-70b-hf
tag: latest
steps:
- name: Checkout repo
uses: actions/checkout@v3

- name: Log in to Docker Hub
uses: docker/login-action@f054a8b539a109f9f41c372932f1ae047eff08c9
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Set up Depot CLI
uses: depot/setup-action@v1

- name: Build and push Docker image
run: depot build -t ${{ matrix.repository }}:${{ matrix.tag }} . --build-arg MODEL=${{ matrix.model }} --push --platform linux/amd64
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
14 changes: 11 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ For those using Depot to build and deploy containers, the command structure is s

```bash
depot build -t yourusername/serverlessllm:1.0 . \
--build-arg MODE_TO_RUN=serverless \
--build-arg MODE_TO_RUN=pod \
--build-arg MODEL=mistralai/Mistral-7B-Instruct-v0.1 \
--build-arg CONCURRENCY_MODIFIER=1 \
--build-arg MAX_MODEL_LEN=25000 \
Expand All @@ -71,20 +71,28 @@ depot build -t yourusername/serverlessllm:1.0 . \
For traditional Docker builds, you can incorporate the environment variables into your build command like so:
```bash
docker build -t yourusername/serverlessllm:1.0 . \
--build-arg MODE_TO_RUN=serverless \
--build-arg MODE_TO_RUN=pod \
--build-arg MODEL=mistralai/Mistral-7B-Instruct-v0.1 \
--build-arg CONCURRENCY_MODIFIER=1 \
--build-arg MAX_MODEL_LEN=25000
```

## Common Use Case will just be changing out the model
Other variables at least if you are deploying on runpod, the ENV can be changed at runpod. The only thing that really matters at build time is the `MODEL` so it might look closer to like below:
```
depot build -t yourusername/serverlessllm:1.0 . \
--build-arg MODEL=mistralai/Mistral-7B-Instruct-v0.1 \
--push --platform linux/amd64
```

# Developer Experience

Start using the container with [GPU_POD](https://runpod.io/gsc?template=pu8uaqw765&ref=wqryvm1m
) (This is my runpod public URL to the template all ready to go for you.)

![alt text](GPU_POD.png)

If you want to deploy on serverless it's super easy! Essentially copy the template but set the environment variable for the MODE to serverless:
If you want to deploy on serverless it's super easy! Essentially copy the template but set the environment variable for the MODE to serverless. **Check to make sure the model repository names match up** as I might update template names, or you might be using a different model.:
![alt text](SERVERLESS.png)
Expand Down

0 comments on commit 11e98f1

Please sign in to comment.