Skip to content

Latest commit

 

History

History
38 lines (36 loc) · 1.54 KB

with-network-volume.md

File metadata and controls

38 lines (36 loc) · 1.54 KB
Raw

Building the Worker with a Network Volume

This will store your application on a Runpod Network Volume and build a light weight Docker image that runs everything from the Network volume without installing the application inside the Docker image.

  1. Create a RunPod Account.
  2. Create a RunPod Network Volume.
  3. Attach the Network Volume to a Secure Cloud GPU pod.
  4. Select a light-weight template such as RunPod Pytorch.
  5. Deploy the GPU Cloud pod.
  6. Once the pod is up, open a Terminal and install the required dependencies:
cd /workspace
git clone https://github.com/ashleykleynhans/runpod-worker-llava.git
cd runpod-worker-llava
python3 -m venv venv
source venv/bin/activate
pip3 install --no-cache-dir torch==2.0.1 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 && \
pip3 install --no-cache-dir xformers==0.0.22
pip3 install -r requirements.txt
pip3 install transformers==4.34.1
  1. Download the models:
export HUGGINGFACE_HUB_CACHE="/workspace/huggingface-cache/hub"
export TRANSFORMERS_CACHE="/workspace/huggingface-cache/hub"
export MODEL="liuhaotian/llava-v1.5-7b"
python3 download_models.py
  1. Sign up for a Docker hub account if you don't already have one.
  2. Build the Docker image and push to Docker hub:
docker build -t dockerhub-username/runpod-worker-llava:1.0.0 -f Dockerfile.Network_Volume .
docker login
docker push dockerhub-username/runpod-worker-llava:1.0.0