FT-with-LoRA

Setup

We recommend using the latest release of NGC's PyTorch container.

First, pull the Docker image (please replace xx.xx with the actual version number):

docker pull nvcr.io/nvidia/pytorch:xx.xx-py3
docker run --gpus all -it --rm -v /path/to/FT-with-LoRA:/workspace/FT-with-LoRA -v /path/to/models:/models nvcr.io/nvidia/pytorch:xx.xx-py3

Install Requirements

cd FT-with-LoRA
pip install -r requirements.txt

Usage

After installation, there are several possible workflows. The most comprehensive is:

1. Data download and preprocessing
2. Finetuning with LoRA
3. Downstream task evaluation or text generation
4. Plot results

Download Data and Preprocessing

Download pCLUE data:

cd data
bash downloadpClUE.sh

Preprocess the data:

cd data
bash preprocess.sh

Finetune

Run the following command to finetune:

bash run_peft_ds.sh

Evaluation

Run the following command to evaluate:

bash eval.sh

Plot results

Run the following command to plot the results:

python plotResult.py