Skip to content

Commit

Permalink
Merge branch 'main' of https://github.com/NexaAI/nexaai-sdk-cpp into …
Browse files Browse the repository at this point in the history
…david/installation
  • Loading branch information
Davidqian123 committed Aug 28, 2024
2 parents 74d6761 + 86d171e commit ec815c8
Show file tree
Hide file tree
Showing 4 changed files with 34 additions and 13 deletions.
15 changes: 2 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,6 @@ Below is our differentiation from other similar tools:

## Installation

### Pre-built Wheels (Recommended)

We have released pre-built wheels for various Python versions, platforms, and backends for convenient installation on our [index page](https://nexaai.github.io/nexa-sdk/whl/).

#### CPU
Expand All @@ -75,29 +73,20 @@ pip install nexaai --index-url https://nexaai.github.io/nexa-sdk/whl/cpu --extra
For the GPU version supporting **Metal (macOS)**:

```bash
pip install nexaai --index-url https://nexaai.github.io/nexa-sdk/whl/metal --extra-index-url https://pypi.org/simple
CMAKE_ARGS="-DGGML_METAL=ON -DSD_METAL=ON" pip install nexaai --index-url https://nexaai.github.io/nexa-sdk/whl/metal --extra-index-url https://pypi.org/simple
```

#### GPU (CUDA)

For the GPU version supporting **CUDA (Linux/Windows)**:

```bash
pip install nexaai --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple
CMAKE_ARGS="-DGGML_CUDA=ON -DSD_CUBLAS=ON" pip install nexaai --index-url https://nexaai.github.io/nexa-sdk/whl/cu124 --extra-index-url https://pypi.org/simple
```

> [!NOTE]
> The CUDA wheels are built with CUDA 12.4, but should be compatible with all CUDA 12.X
### Install from source code distribution

If pre-built wheels cannot meet your requirements, you can install Nexa SDK from the source code via cmake.

#### CPU

```bash
pip install nexaai
```

#### GPU (Metal)

Expand Down
Binary file removed docs/.media/error.jpeg
Binary file not shown.
8 changes: 8 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,14 @@ pip install twine
twine upload dist/*
```

### add a tag
```
git tag
git tag -d <version>
git tag <version>
git push origin <version>
```

## Testing

### Test Inference with GGUF Files
Expand Down
24 changes: 24 additions & 0 deletions docs/cmd_macos.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash

cd "$(dirname "${BASH_SOURCE[0]}")"

if [[ "$(pwd)" =~ " " ]]; then echo This script relies on Miniconda which can not be silently installed under a path with spaces. && exit; fi

# deactivate existing conda envs as needed to avoid conflicts
{ conda deactivate && conda deactivate && conda deactivate; } 2> /dev/null

# config
CONDA_ROOT_PREFIX="$(pwd)/installer_files/conda"
INSTALL_ENV_DIR="$(pwd)/installer_files/env"

# environment isolation
export PYTHONNOUSERSITE=1
unset PYTHONPATH
unset PYTHONHOME
export CUDA_PATH="$INSTALL_ENV_DIR"
export CUDA_HOME="$CUDA_PATH"

# activate env
source $CONDA_ROOT_PREFIX/etc/profile.d/conda.sh
conda activate $INSTALL_ENV_DIR
exec bash --norc

0 comments on commit ec815c8

Please sign in to comment.