LLM-tutorial/notebook/llama-factory.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90bf7ad9-35ea-41a9-a19c-1397780cbf50",
   "metadata": {},
   "outputs": [],
   "source": [
    "## LLaMA-Factory是一个非常流行且易于使用的大模型训练框架。它有以下的能力：\n",
    "按模型类型划分：支持纯文本和多模态模型的训练，纯文本模型包含：LLaMA、Gemma、Qwen、GLM等全系列模型，多模态模型包含：LLaVA、LLaMA-Vision、Qwen-VL等全系列模型100+。\n",
    "按训练类型划分：支持预训练、微调、人类对齐（PPO、DPO、ORPO、KTO等）全训练stage。\n",
    "按命令类型划分：支持代码训练、命令行训练、web-ui界面训练。\n",
    "按微调方式划分：支持全参数、LoRA、QLoRA、DoRA等多个轻量微调方式。\n",
    "按设备类型划分：支持单机单卡、单机多卡、多机多卡等多种微调方式。\n",
    "\n",
    "下面我们以框架自带的例子为例来讲解LLaMA-Factory的使用方式。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "76729112-7ab0-4179-a0cc-fce97a90407d",
   "metadata": {
    "ExecutionIndicator": {
     "show": true
    },
    "execution": {
     "iopub.execute_input": "2024-12-24T11:53:30.311601Z",
     "iopub.status.busy": "2024-12-24T11:53:30.311440Z",
     "iopub.status.idle": "2024-12-24T11:54:01.323917Z",
     "shell.execute_reply": "2024-12-24T11:54:01.323328Z",
     "shell.execute_reply.started": "2024-12-24T11:53:30.311580Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "正克隆到 'LLaMA-Factory'...\n",
      "remote: Enumerating objects: 19892, done.\u001b[K\n",
      "remote: Counting objects: 100% (390/390), done.\u001b[K\n",
      "remote: Compressing objects: 100% (203/203), done.\u001b[K\n",
      "remote: Total 19892 (delta 291), reused 199 (delta 187), pack-reused 19502 (from 4)\u001b[K\n",
      "接收对象中: 100% (19892/19892), 232.22 MiB | 23.35 MiB/s, 完成.\n",
      "处理 delta 中: 100% (14476/14476), 完成.\n",
      "/mnt/workspace/yzhao/LLaMA-Factory\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.10/site-packages/IPython/core/magics/osm.py:417: UserWarning: This is now an optional IPython functionality, setting dhist requires you to install the `pickleshare` library.\n",
      "  self.shell.db['dhist'] = compress_dhist(dhist)[-100:]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: https://mirrors.aliyun.com/pypi/simple\n",
      "Obtaining file:///mnt/workspace/yzhao/LLaMA-Factory\n",
      "  Installing build dependencies ... \u001b[?25ldone\n",
      "\u001b[?25h  Checking if build backend supports build_editable ... \u001b[?25ldone\n",
      "\u001b[?25h  Getting requirements to build editable ... \u001b[?25ldone\n",
      "\u001b[?25h  Preparing editable metadata (pyproject.toml) ... \u001b[?25ldone\n",
      "\u001b[?25hCollecting transformers<=4.46.1,>=4.41.2 (from llamafactory==0.9.2.dev0)\n",
      "  Downloading https://mirrors.aliyun.com/pypi/packages/75/d5/294a09a62bdd88da9a1007a341d4f8fbfc43be520c101e6afb526000e9f4/transformers-4.46.1-py3-none-any.whl (10.0 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m10.0/10.0 MB\u001b[0m \u001b[31m73.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: datasets<=3.1.0,>=2.16.0 in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (3.0.1)\n",
      "Collecting accelerate<=1.0.1,>=0.34.0 (from llamafactory==0.9.2.dev0)\n",
      "  Downloading https://mirrors.aliyun.com/pypi/packages/2c/92/48aec3736ca778ffe5fa68e19e3c18917cba4de43fa46fe6176cccafe267/accelerate-1.0.1-py3-none-any.whl (330 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m330.9/330.9 kB\u001b[0m \u001b[31m119.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: peft<=0.12.0,>=0.11.1 in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (0.12.0)\n",
      "Collecting trl<=0.9.6,>=0.8.6 (from llamafactory==0.9.2.dev0)\n",
      "  Downloading https://mirrors.aliyun.com/pypi/packages/a5/c3/6565c2c376a829f99da20d39c2912405195ec1fa6aae068dc45c46793e72/trl-0.9.6-py3-none-any.whl (245 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m245.8/245.8 kB\u001b[0m \u001b[31m101.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: tokenizers<0.20.4,>=0.19.0 in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (0.20.3)\n",
      "Collecting gradio<5.0.0,>=4.0.0 (from llamafactory==0.9.2.dev0)\n",
      "  Downloading https://mirrors.aliyun.com/pypi/packages/3f/6e/c0726e138f64cd98379a7bf95f4f3b15dd5a9f004b172540cee5653ec820/gradio-4.44.1-py3-none-any.whl (18.1 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m18.1/18.1 MB\u001b[0m \u001b[31m82.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: pandas>=2.0.0 in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (2.2.3)\n",
      "Requirement already satisfied: scipy in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (1.14.1)\n",
      "Requirement already satisfied: einops in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (0.8.0)\n",
      "Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (0.2.0)\n",
      "Requirement already satisfied: tiktoken in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (0.7.0)\n",
      "Requirement already satisfied: protobuf in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (5.29.1)\n",
      "Requirement already satisfied: uvicorn in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (0.32.1)\n",
      "Requirement already satisfied: pydantic in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (2.10.3)\n",
      "Requirement already satisfied: fastapi in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (0.115.6)\n",
      "Requirement already satisfied: sse-starlette in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (2.1.3)\n",
      "Requirement already satisfied: matplotlib>=3.7.0 in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (3.9.3)\n",
      "Requirement already satisfied: fire in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (0.7.0)\n",
      "Requirement already satisfied: packaging in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (24.2)\n",
      "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (6.0.2)\n",
      "Requirement already satisfied: numpy<2.0.0 in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (1.26.4)\n",
      "Requirement already satisfied: av in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (14.0.0)\n",
      "Collecting tyro<0.9.0 (from llamafactory==0.9.2.dev0)\n",
      "  Downloading https://mirrors.aliyun.com/pypi/packages/60/ec/e34d546cfd9c5b906d1d534bb75557be9f2b179609d60bb9e97ec07e8ead/tyro-0.8.14-py3-none-any.whl (109 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m109.8/109.8 kB\u001b[0m \u001b[31m47.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: torch>=1.13.1 in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (2.4.0)\n",
      "Requirement already satisfied: nltk in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (3.9.1)\n",
      "Requirement already satisfied: jieba in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (0.42.1)\n",
      "Requirement already satisfied: rouge-chinese in /usr/local/lib/python3.10/site-packages (from llamafactory==0.9.2.dev0) (1.0.3)\n",
      "Requirement already satisfied: psutil in /usr/local/lib/python3.10/site-packages (from accelerate<=1.0.1,>=0.34.0->llamafactory==0.9.2.dev0) (6.1.0)\n",
      "Requirement already satisfied: huggingface-hub>=0.21.0 in /usr/local/lib/python3.10/site-packages (from accelerate<=1.0.1,>=0.34.0->llamafactory==0.9.2.dev0) (0.25.2)\n",
      "Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.10/site-packages (from accelerate<=1.0.1,>=0.34.0->llamafactory==0.9.2.dev0) (0.4.5)\n",
      "Requirement already satisfied: filelock in /usr/local/lib/python3.10/site-packages (from datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (3.16.1)\n",
      "Requirement already satisfied: pyarrow>=15.0.0 in /usr/local/lib/python3.10/site-packages (from datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (18.1.0)\n",
      "Requirement already satisfied: dill<0.3.9,>=0.3.0 in /usr/local/lib/python3.10/site-packages (from datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (0.3.8)\n",
      "Requirement already satisfied: requests>=2.32.2 in /usr/local/lib/python3.10/site-packages (from datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (2.32.3)\n",
      "Requirement already satisfied: tqdm>=4.66.3 in /usr/local/lib/python3.10/site-packages (from datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (4.67.1)\n",
      "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/site-packages (from datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (3.5.0)\n",
      "Requirement already satisfied: multiprocess in /usr/local/lib/python3.10/site-packages (from datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (0.70.16)\n",
      "Requirement already satisfied: fsspec<=2024.6.1,>=2023.1.0 in /usr/local/lib/python3.10/site-packages (from fsspec[http]<=2024.6.1,>=2023.1.0->datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (2024.6.1)\n",
      "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/site-packages (from datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (3.11.10)\n",
      "Requirement already satisfied: aiofiles<24.0,>=22.0 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (23.2.1)\n",
      "Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (4.7.0)\n",
      "Requirement already satisfied: ffmpy in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (0.4.0)\n",
      "Collecting gradio-client==1.3.0 (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0)\n",
      "  Downloading https://mirrors.aliyun.com/pypi/packages/de/fe/7e9cb4d0e6aa74268fa31089189e4855882a0f2a36c45d359336946d4ae1/gradio_client-1.3.0-py3-none-any.whl (318 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m318.7/318.7 kB\u001b[0m \u001b[31m108.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: httpx>=0.24.1 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (0.27.2)\n",
      "Requirement already satisfied: importlib-resources<7.0,>=1.3 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (6.4.5)\n",
      "Requirement already satisfied: jinja2<4.0 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (3.1.4)\n",
      "Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (2.1.5)\n",
      "Requirement already satisfied: orjson~=3.0 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (3.10.12)\n",
      "Requirement already satisfied: pillow<11.0,>=8.0 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (10.4.0)\n",
      "Requirement already satisfied: pydub in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (0.25.1)\n",
      "Requirement already satisfied: python-multipart>=0.0.9 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (0.0.19)\n",
      "Requirement already satisfied: ruff>=0.2.2 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (0.8.2)\n",
      "Requirement already satisfied: semantic-version~=2.0 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (2.10.0)\n",
      "Collecting tomlkit==0.12.0 (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0)\n",
      "  Downloading https://mirrors.aliyun.com/pypi/packages/68/4f/12207897848a653d03ebbf6775a29d949408ded5f99b2d87198bc5c93508/tomlkit-0.12.0-py3-none-any.whl (37 kB)\n",
      "Requirement already satisfied: typer<1.0,>=0.12 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (0.15.1)\n",
      "Requirement already satisfied: typing-extensions~=4.0 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (4.12.2)\n",
      "Requirement already satisfied: urllib3~=2.0 in /usr/local/lib/python3.10/site-packages (from gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (2.2.3)\n",
      "Requirement already satisfied: websockets<13.0,>=10.0 in /usr/local/lib/python3.10/site-packages (from gradio-client==1.3.0->gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (11.0.3)\n",
      "Requirement already satisfied: starlette<0.42.0,>=0.40.0 in /usr/local/lib/python3.10/site-packages (from fastapi->llamafactory==0.9.2.dev0) (0.41.3)\n",
      "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/site-packages (from matplotlib>=3.7.0->llamafactory==0.9.2.dev0) (1.3.1)\n",
      "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/site-packages (from matplotlib>=3.7.0->llamafactory==0.9.2.dev0) (0.12.1)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/site-packages (from matplotlib>=3.7.0->llamafactory==0.9.2.dev0) (4.55.2)\n",
      "Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.10/site-packages (from matplotlib>=3.7.0->llamafactory==0.9.2.dev0) (1.4.7)\n",
      "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/site-packages (from matplotlib>=3.7.0->llamafactory==0.9.2.dev0) (3.2.0)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/site-packages (from matplotlib>=3.7.0->llamafactory==0.9.2.dev0) (2.9.0.post0)\n",
      "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/site-packages (from pandas>=2.0.0->llamafactory==0.9.2.dev0) (2024.2)\n",
      "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/site-packages (from pandas>=2.0.0->llamafactory==0.9.2.dev0) (2024.2)\n",
      "Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.10/site-packages (from pydantic->llamafactory==0.9.2.dev0) (0.7.0)\n",
      "Requirement already satisfied: pydantic-core==2.27.1 in /usr/local/lib/python3.10/site-packages (from pydantic->llamafactory==0.9.2.dev0) (2.27.1)\n",
      "Requirement already satisfied: sympy in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (1.13.3)\n",
      "Requirement already satisfied: networkx in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (3.4.2)\n",
      "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (9.1.0.70)\n",
      "Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (12.1.3.1)\n",
      "Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (11.0.2.54)\n",
      "Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (10.3.2.106)\n",
      "Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (11.4.5.107)\n",
      "Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (12.1.0.106)\n",
      "Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (2.20.5)\n",
      "Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (12.1.105)\n",
      "Requirement already satisfied: triton==3.0.0 in /usr/local/lib/python3.10/site-packages (from torch>=1.13.1->llamafactory==0.9.2.dev0) (3.0.0)\n",
      "Requirement already satisfied: nvidia-nvjitlink-cu12 in /usr/local/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.13.1->llamafactory==0.9.2.dev0) (12.6.85)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/site-packages (from transformers<=4.46.1,>=4.41.2->llamafactory==0.9.2.dev0) (2024.11.6)\n",
      "Requirement already satisfied: docstring-parser>=0.16 in /usr/local/lib/python3.10/site-packages (from tyro<0.9.0->llamafactory==0.9.2.dev0) (0.16)\n",
      "Requirement already satisfied: rich>=11.1.0 in /usr/local/lib/python3.10/site-packages (from tyro<0.9.0->llamafactory==0.9.2.dev0) (13.9.4)\n",
      "Requirement already satisfied: shtab>=1.5.6 in /usr/local/lib/python3.10/site-packages (from tyro<0.9.0->llamafactory==0.9.2.dev0) (1.7.1)\n",
      "Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/site-packages (from uvicorn->llamafactory==0.9.2.dev0) (8.1.7)\n",
      "Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.10/site-packages (from uvicorn->llamafactory==0.9.2.dev0) (0.14.0)\n",
      "Requirement already satisfied: termcolor in /usr/local/lib/python3.10/site-packages (from fire->llamafactory==0.9.2.dev0) (2.5.0)\n",
      "Requirement already satisfied: joblib in /usr/local/lib/python3.10/site-packages (from nltk->llamafactory==0.9.2.dev0) (1.4.2)\n",
      "Requirement already satisfied: six in /usr/local/lib/python3.10/site-packages (from rouge-chinese->llamafactory==0.9.2.dev0) (1.17.0)\n",
      "Requirement already satisfied: exceptiongroup>=1.0.2 in /usr/local/lib/python3.10/site-packages (from anyio<5.0,>=3.0->gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (1.2.2)\n",
      "Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/site-packages (from anyio<5.0,>=3.0->gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (3.10)\n",
      "Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/site-packages (from anyio<5.0,>=3.0->gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (1.3.1)\n",
      "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/site-packages (from aiohttp->datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (2.4.4)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/site-packages (from aiohttp->datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (1.3.1)\n",
      "Requirement already satisfied: async-timeout<6.0,>=4.0 in /usr/local/lib/python3.10/site-packages (from aiohttp->datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (4.0.3)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/site-packages (from aiohttp->datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (24.2.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/site-packages (from aiohttp->datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (1.5.0)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/site-packages (from aiohttp->datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (6.1.0)\n",
      "Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.10/site-packages (from aiohttp->datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (0.2.1)\n",
      "Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib/python3.10/site-packages (from aiohttp->datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (1.18.3)\n",
      "Requirement already satisfied: certifi in /usr/local/lib/python3.10/site-packages (from httpx>=0.24.1->gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (2024.8.30)\n",
      "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/site-packages (from httpx>=0.24.1->gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (1.0.7)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/site-packages (from requests>=2.32.2->datasets<=3.1.0,>=2.16.0->llamafactory==0.9.2.dev0) (3.4.0)\n",
      "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.10/site-packages (from rich>=11.1.0->tyro<0.9.0->llamafactory==0.9.2.dev0) (3.0.0)\n",
      "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.10/site-packages (from rich>=11.1.0->tyro<0.9.0->llamafactory==0.9.2.dev0) (2.18.0)\n",
      "Requirement already satisfied: shellingham>=1.3.0 in /usr/local/lib/python3.10/site-packages (from typer<1.0,>=0.12->gradio<5.0.0,>=4.0.0->llamafactory==0.9.2.dev0) (1.5.4)\n",
      "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/site-packages (from sympy->torch>=1.13.1->llamafactory==0.9.2.dev0) (1.3.0)\n",
      "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich>=11.1.0->tyro<0.9.0->llamafactory==0.9.2.dev0) (0.1.2)\n",
      "Checking if build backend supports build_editable ... \u001b[?25ldone\n",
      "\u001b[?25hBuilding wheels for collected packages: llamafactory\n",
      "  Building editable for llamafactory (pyproject.toml) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for llamafactory: filename=llamafactory-0.9.2.dev0-0.editable-py3-none-any.whl size=23812 sha256=3ae5c01cf8229e99c4117ea68f97b5b1acd8eac4d03040e37551df28c3091c10\n",
      "  Stored in directory: /tmp/pip-ephem-wheel-cache-23rivfca/wheels/3d/47/c2/cb4f8bf5699286f0c51834921a3c9f2850e56f192b3f9a9256\n",
      "Successfully built llamafactory\n",
      "Installing collected packages: tomlkit, tyro, gradio-client, transformers, gradio, accelerate, trl, llamafactory\n",
      "  Attempting uninstall: tomlkit\n",
      "    Found existing installation: tomlkit 0.13.2\n",
      "    Uninstalling tomlkit-0.13.2:\n",
      "      Successfully uninstalled tomlkit-0.13.2\n",
      "  Attempting uninstall: tyro\n",
      "    Found existing installation: tyro 0.9.2\n",
      "    Uninstalling tyro-0.9.2:\n",
      "      Successfully uninstalled tyro-0.9.2\n",
      "  Attempting uninstall: gradio-client\n",
      "    Found existing installation: gradio_client 1.5.2\n",
      "    Uninstalling gradio_client-1.5.2:\n",
      "      Successfully uninstalled gradio_client-1.5.2\n",
      "  Attempting uninstall: transformers\n",
      "    Found existing installation: transformers 4.46.3\n",
      "    Uninstalling transformers-4.46.3:\n",
      "      Successfully uninstalled transformers-4.46.3\n",
      "  Attempting uninstall: gradio\n",
      "    Found existing installation: gradio 5.9.1\n",
      "    Uninstalling gradio-5.9.1:\n",
      "      Successfully uninstalled gradio-5.9.1\n",
      "  Attempting uninstall: accelerate\n",
      "    Found existing installation: accelerate 1.1.1\n",
      "    Uninstalling accelerate-1.1.1:\n",
      "      Successfully uninstalled accelerate-1.1.1\n",
      "  Attempting uninstall: trl\n",
      "    Found existing installation: trl 0.11.4\n",
      "    Uninstalling trl-0.11.4:\n",
      "      Successfully uninstalled trl-0.11.4\n",
      "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "xtuner 0.1.23 requires lagent>=0.1.2, which is not installed.\n",
      "xtuner 0.1.23 requires mmengine>=0.10.3, which is not installed.\n",
      "xtuner 0.1.23 requires scikit-image, which is not installed.\n",
      "lmdeploy 0.6.2 requires peft<=0.11.1, but you have peft 0.12.0 which is incompatible.\n",
      "ms-swift 3.0.0.dev0 requires trl<0.12,>=0.11, but you have trl 0.9.6 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0mSuccessfully installed accelerate-1.0.1 gradio-4.44.1 gradio-client-1.3.0 llamafactory-0.9.2.dev0 tomlkit-0.12.0 transformers-4.46.1 trl-0.9.6 tyro-0.8.14\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
      "\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.3.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.3.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# clone代码\n",
    "!git clone https://github.com/hiyouga/LLaMA-Factory.git\n",
    "%cd LLaMA-Factory\n",
    "!pip install -e \".[torch,metrics]\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71106bf5-22fe-46ea-9ebc-be4080189d01",
   "metadata": {},
   "source": [
    "对于国内用户，我们建议增加如下环境变量，这样LLaMA-Factory会使用国内的模型源下载："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95619848-1bd6-4405-9e56-5d23d2568407",
   "metadata": {
    "ExecutionIndicator": {
     "show": true
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "!export USE_MODELSCOPE_HUB=1 # `set USE_MODELSCOPE_HUB=1` for Windows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8912fc10-2a4f-4aa4-a1e7-77b39d362c0a",
   "metadata": {
    "ExecutionIndicator": {
     "show": true
    },
    "execution": {
     "iopub.execute_input": "2024-12-24T12:51:33.762831Z",
     "iopub.status.busy": "2024-12-24T12:51:33.761941Z",
     "iopub.status.idle": "2024-12-24T12:52:18.421623Z",
     "shell.execute_reply": "2024-12-24T12:52:18.420782Z",
     "shell.execute_reply.started": "2024-12-24T12:51:33.762806Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2024-12-24 20:51:37,480] [INFO] [real_accelerator.py:219:get_accelerator] Setting ds_accelerator to cuda (auto detect)\n",
      "[INFO|2024-12-24 20:51:41] llamafactory.hparams.parser:355 >> Process rank: 0, device: cuda:0, n_gpu: 1, distributed training: False, compute dtype: torch.bfloat16\n",
      "Downloading Model to directory: /mnt/workspace/.cache/modelscope/hub/LLM-Research/Meta-Llama-3-8B-Instruct\n",
      "2024-12-24 20:51:41,522 - modelscope - WARNING - Using branch: master as version is unstable, use with caution\n",
      "[INFO|configuration_utils.py:677] 2024-12-24 20:51:41,769 >> loading configuration file /mnt/workspace/.cache/modelscope/hub/LLM-Research/Meta-Llama-3-8B-Instruct/config.json\n",
      "[INFO|configuration_utils.py:746] 2024-12-24 20:51:41,770 >> Model config LlamaConfig {\n",
      "  \"_name_or_path\": \"/mnt/workspace/.cache/modelscope/hub/LLM-Research/Meta-Llama-3-8B-Instruct\",\n",
      "  \"architectures\": [\n",
      "    \"LlamaForCausalLM\"\n",
      "  ],\n",
      "  \"attention_bias\": false,\n",
      "  \"attention_dropout\": 0.0,\n",
      "  \"bos_token_id\": 128000,\n",
      "  \"eos_token_id\": 128009,\n",
      "  \"head_dim\": 128,\n",
      "  \"hidden_act\": \"silu\",\n",
      "  \"hidden_size\": 4096,\n",
      "  \"initializer_range\": 0.02,\n",
      "  \"intermediate_size\": 14336,\n",
      "  \"max_position_embeddings\": 8192,\n",
      "  \"mlp_bias\": false,\n",
      "  \"model_type\": \"llama\",\n",
      "  \"num_attention_heads\": 32,\n",
      "  \"num_hidden_layers\": 32,\n",
      "  \"num_key_value_heads\": 8,\n",
      "  \"pretraining_tp\": 1,\n",
      "  \"rms_norm_eps\": 1e-05,\n",
      "  \"rope_scaling\": null,\n",
      "  \"rope_theta\": 500000.0,\n",
      "  \"tie_word_embeddings\": false,\n",
      "  \"torch_dtype\": \"bfloat16\",\n",
      "  \"transformers_version\": \"4.46.1\",\n",
      "  \"use_cache\": true,\n",
      "  \"vocab_size\": 128256\n",
      "}\n",
      "\n",
      "[INFO|tokenization_utils_base.py:2209] 2024-12-24 20:51:41,773 >> loading file tokenizer.json\n",
      "[INFO|tokenization_utils_base.py:2209] 2024-12-24 20:51:41,773 >> loading file tokenizer.model\n",
      "[INFO|tokenization_utils_base.py:2209] 2024-12-24 20:51:41,773 >> loading file added_tokens.json\n",
      "[INFO|tokenization_utils_base.py:2209] 2024-12-24 20:51:41,773 >> loading file special_tokens_map.json\n",
      "[INFO|tokenization_utils_base.py:2209] 2024-12-24 20:51:41,773 >> loading file tokenizer_config.json\n",
      "[INFO|tokenization_utils_base.py:2475] 2024-12-24 20:51:42,106 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n",
      "[INFO|configuration_utils.py:677] 2024-12-24 20:51:42,109 >> loading configuration file /mnt/workspace/.cache/modelscope/hub/LLM-Research/Meta-Llama-3-8B-Instruct/config.json\n",
      "[INFO|configuration_utils.py:746] 2024-12-24 20:51:42,110 >> Model config LlamaConfig {\n",
      "  \"_name_or_path\": \"/mnt/workspace/.cache/modelscope/hub/LLM-Research/Meta-Llama-3-8B-Instruct\",\n",
      "  \"architectures\": [\n",
      "    \"LlamaForCausalLM\"\n",
      "  ],\n",
      "  \"attention_bias\": false,\n",
      "  \"attention_dropout\": 0.0,\n",
      "  \"bos_token_id\": 128000,\n",
      "  \"eos_token_id\": 128009,\n",
      "  \"head_dim\": 128,\n",
      "  \"hidden_act\": \"silu\",\n",
      "  \"hidden_size\": 4096,\n",
      "  \"initializer_range\": 0.02,\n",
      "  \"intermediate_size\": 14336,\n",
      "  \"max_position_embeddings\": 8192,\n",
      "  \"mlp_bias\": false,\n",
      "  \"model_type\": \"llama\",\n",
      "  \"num_attention_heads\": 32,\n",
      "  \"num_hidden_layers\": 32,\n",
      "  \"num_key_value_heads\": 8,\n",
      "  \"pretraining_tp\": 1,\n",
      "  \"rms_norm_eps\": 1e-05,\n",
      "  \"rope_scaling\": null,\n",
      "  \"rope_theta\": 500000.0,\n",
      "  \"tie_word_embeddings\": false,\n",
      "  \"torch_dtype\": \"bfloat16\",\n",
      "  \"transformers_version\": \"4.46.1\",\n",
      "  \"use_cache\": true,\n",
      "  \"vocab_size\": 128256\n",
      "}\n",
      "\n",
      "[INFO|tokenization_utils_base.py:2209] 2024-12-24 20:51:42,113 >> loading file tokenizer.json\n",
      "[INFO|tokenization_utils_base.py:2209] 2024-12-24 20:51:42,113 >> loading file tokenizer.model\n",
      "[INFO|tokenization_utils_base.py:2209] 2024-12-24 20:51:42,113 >> loading file added_tokens.json\n",
      "[INFO|tokenization_utils_base.py:2209] 2024-12-24 20:51:42,113 >> loading file special_tokens_map.json\n",
      "[INFO|tokenization_utils_base.py:2209] 2024-12-24 20:51:42,113 >> loading file tokenizer_config.json\n",
      "[INFO|tokenization_utils_base.py:2475] 2024-12-24 20:51:42,444 >> Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n",
      "[INFO|2024-12-24 20:51:42] llamafactory.data.template:157 >> Add pad token: <|eot_id|>\n",
      "[INFO|2024-12-24 20:51:42] llamafactory.data.template:157 >> Add <|eot_id|>,<|eom_id|> to stop words.\n",
      "[WARNING|2024-12-24 20:51:42] llamafactory.data.template:162 >> New tokens have been added, make sure `resize_vocab` is True.\n",
      "[INFO|2024-12-24 20:51:42] llamafactory.data.loader:157 >> Loading dataset identity.json...\n",
      "Converting format of dataset (num_proc=16): 100%|█| 91/91 [00:00<00:00, 528.56 e\n",
      "[INFO|2024-12-24 20:51:43] llamafactory.data.loader:157 >> Loading dataset alpaca_en_demo.json...\n",
      "Converting format of dataset (num_proc=16): 100%|█| 1000/1000 [00:00<00:00, 6428\n",
      "Running tokenizer on dataset (num_proc=16): 100%|█| 1091/1091 [00:03<00:00, 344.\n",
      "training example:\n",
      "input_ids:\n",
      "[128000, 128006, 882, 128007, 271, 6151, 128009, 128006, 78191, 128007, 271, 9906, 0, 358, 1097, 5991, 609, 39254, 459, 15592, 18328, 8040, 555, 5991, 3170, 3500, 13, 2650, 649, 358, 7945, 499, 3432, 30, 128009]\n",
      "inputs:\n",
      "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n",
      "\n",
      "hi<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n",
      "\n",
      "Hello! I am {{name}}, an AI assistant developed by {{author}}. How can I assist you today?<|eot_id|>\n",
      "label_ids:\n",
      "[-100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 9906, 0, 358, 1097, 5991, 609, 39254, 459, 15592, 18328, 8040, 555, 5991, 3170, 3500, 13, 2650, 649, 358, 7945, 499, 3432, 30, 128009]\n",
      "labels:\n",
      "Hello! I am {{name}}, an AI assistant developed by {{author}}. How can I assist you today?<|eot_id|>\n",
      "[INFO|configuration_utils.py:677] 2024-12-24 20:51:47,617 >> loading configuration file /mnt/workspace/.cache/modelscope/hub/LLM-Research/Meta-Llama-3-8B-Instruct/config.json\n",
      "[INFO|configuration_utils.py:746] 2024-12-24 20:51:47,618 >> Model config LlamaConfig {\n",
      "  \"_name_or_path\": \"/mnt/workspace/.cache/modelscope/hub/LLM-Research/Meta-Llama-3-8B-Instruct\",\n",
      "  \"architectures\": [\n",
      "    \"LlamaForCausalLM\"\n",
      "  ],\n",
      "  \"attention_bias\": false,\n",
      "  \"attention_dropout\": 0.0,\n",
      "  \"bos_token_id\": 128000,\n",
      "  \"eos_token_id\": 128009,\n",
      "  \"head_dim\": 128,\n",
      "  \"hidden_act\": \"silu\",\n",
      "  \"hidden_size\": 4096,\n",
      "  \"initializer_range\": 0.02,\n",
      "  \"intermediate_size\": 14336,\n",
      "  \"max_position_embeddings\": 8192,\n",
      "  \"mlp_bias\": false,\n",
      "  \"model_type\": \"llama\",\n",
      "  \"num_attention_heads\": 32,\n",
      "  \"num_hidden_layers\": 32,\n",
      "  \"num_key_value_heads\": 8,\n",
      "  \"pretraining_tp\": 1,\n",
      "  \"rms_norm_eps\": 1e-05,\n",
      "  \"rope_scaling\": null,\n",
      "  \"rope_theta\": 500000.0,\n",
      "  \"tie_word_embeddings\": false,\n",
      "  \"torch_dtype\": \"bfloat16\",\n",
      "  \"transformers_version\": \"4.46.1\",\n",
      "  \"use_cache\": true,\n",
      "  \"vocab_size\": 128256\n",
      "}\n",
      "\n",
      "[INFO|modeling_utils.py:3934] 2024-12-24 20:51:47,639 >> loading weights file /mnt/workspace/.cache/modelscope/hub/LLM-Research/Meta-Llama-3-8B-Instruct/model.safetensors.index.json\n",
      "[INFO|modeling_utils.py:1670] 2024-12-24 20:51:47,641 >> Instantiating LlamaForCausalLM model under default dtype torch.bfloat16.\n",
      "[INFO|configuration_utils.py:1096] 2024-12-24 20:51:47,642 >> Generate config GenerationConfig {\n",
      "  \"bos_token_id\": 128000,\n",
      "  \"eos_token_id\": 128009\n",
      "}\n",
      "\n",
      "Loading checkpoint shards: 100%|██████████████████| 4/4 [00:08<00:00,  2.07s/it]\n",
      "[INFO|modeling_utils.py:4800] 2024-12-24 20:51:55,977 >> All model checkpoint weights were used when initializing LlamaForCausalLM.\n",
      "\n",
      "[INFO|modeling_utils.py:4808] 2024-12-24 20:51:55,977 >> All the weights of LlamaForCausalLM were initialized from the model checkpoint at /mnt/workspace/.cache/modelscope/hub/LLM-Research/Meta-Llama-3-8B-Instruct.\n",
      "If your task is similar to the task the model of the checkpoint was trained on, you can already use LlamaForCausalLM for predictions without further training.\n",
      "[INFO|configuration_utils.py:1049] 2024-12-24 20:51:55,981 >> loading configuration file /mnt/workspace/.cache/modelscope/hub/LLM-Research/Meta-Llama-3-8B-Instruct/generation_config.json\n",
      "[INFO|configuration_utils.py:1096] 2024-12-24 20:51:55,982 >> Generate config GenerationConfig {\n",
      "  \"bos_token_id\": 128000,\n",
      "  \"do_sample\": true,\n",
      "  \"eos_token_id\": [\n",
      "    128001,\n",
      "    128009\n",
      "  ],\n",
      "  \"max_length\": 4096,\n",
      "  \"temperature\": 0.6,\n",
      "  \"top_p\": 0.9\n",
      "}\n",
      "\n",
      "[INFO|2024-12-24 20:51:55] llamafactory.model.model_utils.checkpointing:157 >> Gradient checkpointing enabled.\n",
      "[INFO|2024-12-24 20:51:55] llamafactory.model.model_utils.attention:157 >> Using torch SDPA for faster training and inference.\n",
      "[INFO|2024-12-24 20:51:55] llamafactory.model.adapter:157 >> Upcasting trainable params to float32.\n",
      "[INFO|2024-12-24 20:51:55] llamafactory.model.adapter:157 >> Fine-tuning method: LoRA\n",
      "[INFO|2024-12-24 20:51:55] llamafactory.model.model_utils.misc:157 >> Found linear modules: q_proj,k_proj,gate_proj,v_proj,up_proj,o_proj,down_proj\n",
      "[INFO|2024-12-24 20:51:56] llamafactory.model.loader:157 >> trainable params: 20,971,520 || all params: 8,051,232,768 || trainable%: 0.2605\n",
      "[INFO|trainer.py:698] 2024-12-24 20:51:56,485 >> Using auto half precision backend\n",
      "[WARNING|2024-12-24 20:51:56] llamafactory.train.callbacks:168 >> Previous trainer log in this folder will be deleted.\n",
      "[INFO|trainer.py:2313] 2024-12-24 20:51:56,820 >> ***** Running training *****\n",
      "[INFO|trainer.py:2314] 2024-12-24 20:51:56,820 >>   Num examples = 981\n",
      "[INFO|trainer.py:2315] 2024-12-24 20:51:56,820 >>   Num Epochs = 3\n",
      "[INFO|trainer.py:2316] 2024-12-24 20:51:56,820 >>   Instantaneous batch size per device = 1\n",
      "[INFO|trainer.py:2319] 2024-12-24 20:51:56,820 >>   Total train batch size (w. parallel, distributed & accumulation) = 8\n",
      "[INFO|trainer.py:2320] 2024-12-24 20:51:56,820 >>   Gradient Accumulation steps = 8\n",
      "[INFO|trainer.py:2321] 2024-12-24 20:51:56,820 >>   Total optimization steps = 366\n",
      "[INFO|trainer.py:2322] 2024-12-24 20:51:56,824 >>   Number of trainable parameters = 20,971,520\n",
      "[INFO|integration_utils.py:812] 2024-12-24 20:51:56,833 >> Automatic Weights & Biases logging enabled, to disable set os.environ[\"WANDB_DISABLED\"] = \"true\"\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m The `run_name` is currently set to the same value as `TrainingArguments.output_dir`. If this was not intended, please specify a different run name by setting the `TrainingArguments.run_name` parameter.\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Using wandb-core as the SDK backend.  Please refer to https://wandb.me/wandb-core for more information.\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33myuze-zyz\u001b[0m (\u001b[33mtastelikefeet\u001b[0m). Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Tracking run with wandb version 0.19.1\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Run data is saved locally in \u001b[35m\u001b[1m/mnt/workspace/yzhao/LLaMA-Factory/wandb/run-20241224_205201-xf8b0lzr\u001b[0m\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Run \u001b[1m`wandb offline`\u001b[0m to turn off syncing.\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Syncing run \u001b[33msaves/llama3-8b/lora/sft\u001b[0m\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: ⭐️ View project at \u001b[34m\u001b[4mhttps://wandb.ai/tastelikefeet/llamafactory\u001b[0m\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: 🚀 View run at \u001b[34m\u001b[4mhttps://wandb.ai/tastelikefeet/llamafactory/runs/xf8b0lzr\u001b[0m\n",
      "  0%|                                                   | 0/366 [00:00<?, ?it/s]/usr/local/lib/python3.10/site-packages/torch/utils/checkpoint.py:295: FutureWarning: `torch.cpu.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cpu', args...)` instead.\n",
      "  with torch.enable_grad(), device_autocast_ctx, torch.cpu.amp.autocast(**ctx.cpu_autocast_kwargs):  # type: ignore[attr-defined]\n",
      "  2%|▊                                          | 7/366 [00:14<12:18,  2.06s/it]^C\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/local/bin/llamafactory-cli\", line 8, in <module>\n",
      "    sys.exit(main())\n",
      "  File \"/mnt/workspace/yzhao/LLaMA-Factory/src/llamafactory/cli.py\", line 111, in main\n",
      "    run_exp()\n",
      "  File \"/mnt/workspace/yzhao/LLaMA-Factory/src/llamafactory/train/tuner.py\", line 59, in run_exp\n",
      "    run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)\n",
      "  File \"/mnt/workspace/yzhao/LLaMA-Factory/src/llamafactory/train/sft/workflow.py\", line 101, in run_sft\n",
      "    train_result = trainer.train(resume_from_checkpoint=training_args.resume_from_checkpoint)\n",
      "  File \"/usr/local/lib/python3.10/site-packages/transformers/trainer.py\", line 2122, in train\n",
      "    return inner_training_loop(\n",
      "  File \"/usr/local/lib/python3.10/site-packages/transformers/trainer.py\", line 2474, in _inner_training_loop\n",
      "    tr_loss_step = self.training_step(model, inputs, num_items_in_batch)\n",
      "  File \"/usr/local/lib/python3.10/site-packages/transformers/trainer.py\", line 3606, in training_step\n",
      "    self.accelerator.backward(loss, **kwargs)\n",
      "  File \"/usr/local/lib/python3.10/site-packages/accelerate/accelerator.py\", line 2246, in backward\n",
      "    loss.backward(**kwargs)\n",
      "  File \"/usr/local/lib/python3.10/site-packages/torch/_tensor.py\", line 521, in backward\n",
      "    torch.autograd.backward(\n",
      "  File \"/usr/local/lib/python3.10/site-packages/torch/autograd/__init__.py\", line 289, in backward\n",
      "    _engine_run_backward(\n",
      "  File \"/usr/local/lib/python3.10/site-packages/torch/autograd/graph.py\", line 768, in _engine_run_backward\n",
      "    return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass\n",
      "KeyboardInterrupt\n",
      "Traceback (most recent call last):\n",
      "  File \"/usr/local/bin/llamafactory-cli\", line 8, in <module>\n",
      "    sys.exit(main())\n",
      "  File \"/mnt/workspace/yzhao/LLaMA-Factory/src/llamafactory/cli.py\", line 111, in main\n",
      "    run_exp()\n",
      "  File \"/mnt/workspace/yzhao/LLaMA-Factory/src/llamafactory/train/tuner.py\", line 59, in run_exp\n",
      "    run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)\n",
      "  File \"/mnt/workspace/yzhao/LLaMA-Factory/src/llamafactory/train/sft/workflow.py\", line 101, in run_sft\n",
      "    train_result = trainer.train(resume_from_checkpoint=training_args.resume_from_checkpoint)\n",
      "  File \"/usr/local/lib/python3.10/site-packages/transformers/trainer.py\", line 2122, in train\n",
      "    return inner_training_loop(\n",
      "  File \"/usr/local/lib/python3.10/site-packages/transformers/trainer.py\", line 2474, in _inner_training_loop\n",
      "    tr_loss_step = self.training_step(model, inputs, num_items_in_batch)\n",
      "  File \"/usr/local/lib/python3.10/site-packages/transformers/trainer.py\", line 3606, in training_step\n",
      "    self.accelerator.backward(loss, **kwargs)\n",
      "  File \"/usr/local/lib/python3.10/site-packages/accelerate/accelerator.py\", line 2246, in backward\n",
      "    loss.backward(**kwargs)\n",
      "  File \"/usr/local/lib/python3.10/site-packages/torch/_tensor.py\", line 521, in backward\n",
      "    torch.autograd.backward(\n",
      "  File \"/usr/local/lib/python3.10/site-packages/torch/autograd/__init__.py\", line 289, in backward\n",
      "    _engine_run_backward(\n",
      "  File \"/usr/local/lib/python3.10/site-packages/torch/autograd/graph.py\", line 768, in _engine_run_backward\n",
      "    return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass\n",
      "KeyboardInterrupt\n"
     ]
    }
   ],
   "source": [
    "# 修改examples/train_lora/llama3_lora_sft.yaml的配置：meta-llama/Meta-Llama-3-8B-Instruct -> LLM-Research/Meta-Llama-3-8B-Instruct\n",
    "!USE_MODELSCOPE_HUB=1 llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c069023-229b-45ef-86f4-a6052b0b8fec",
   "metadata": {},
   "source": [
    "下面我们讲解配置文件中的超参数的含义："
   ]
  },
  {
   "cell_type": "raw",
   "id": "541a6dab-3315-48a9-b322-6dd14df1e266",
   "metadata": {},
   "source": [
    "### model\n",
    "model_name_or_path: LLM-Research/Meta-Llama-3-8B-Instruct 模型ID或路径\n",
    "trust_remote_code: true 部分模型的代码放在了modelhub中，是否相信这部分代码并运行，一般为true即可\n",
    "\n",
    "### method\n",
    "stage: sft 微调\n",
    "do_train: true 执行训练操作\n",
    "finetuning_type: lora 使用lora\n",
    "lora_target: all lora附着在所有的Linear上，除了lm_head（该模块用于将概率映射回字典）\n",
    "\n",
    "### dataset\n",
    "dataset: identity,alpaca_en_demo 数据集\n",
    "template: llama3 模型的模板。模板是chat模型特定的输入格式，用于区分system、user、assistant，每个模型都有对应的template，如果是base模型可以任选一个使用\n",
    "cutoff_len: 2048 数据集截断长度\n",
    "max_samples: 1000 数据及采样数量\n",
    "preprocessing_num_workers: 16 数据集预处理进程数，维持16即可，如果内存或CPU比较差，可以将该值改小\n",
    "\n",
    "### output\n",
    "output_dir: saves/llama3-8b/lora/sft ckpt输出目录\n",
    "logging_steps: 10 log间隔\n",
    "save_steps: 500 保存ckpt间隔\n",
    "plot_loss: true 是否画出loss图谱\n",
    "overwrite_output_dir: true 覆盖output目录与否\n",
    "\n",
    "### train\n",
    "per_device_train_batch_size: 1 批处理大小\n",
    "gradient_accumulation_steps: 8 梯度累积步数，per_device_train_batch_size*gradient_accumulation_steps就是整体batch_size，如果是DDP，还需要乘以DDP大小\n",
    "learning_rate: 1.0e-4 学习率。如果是lora，建议设置为1e-4，如果是全参数，建议设置为1e-5\n",
    "num_train_epochs: 3.0 数据集训练轮次\n",
    "lr_scheduler_type: cosine 学习率衰减类型\n",
    "warmup_ratio: 0.1 是否使用学习率warmup。该方法用于将学习率在一定step内提升到指定学习率，在训练收敛最不稳定的初期，warmup有助于找到合适的梯度下降路线\n",
    "bf16: true 是否使用bf16，如果不是老GPU，建议维持为true\n",
    "ddp_timeout: 180000000 ddp通讯超时时间，建议维持不变\n",
    "\n",
    "### eval\n",
    "val_size: 0.1 交叉验证数据集的拆分比例，也就是按1:9拆分数据集\n",
    "per_device_eval_batch_size: 1 交叉验证批处理大小\n",
    "eval_strategy: steps 交叉验证的策略，默认为steps，也就是默认N个step进行一次交叉验证\n",
    "eval_steps: 500 交叉验证的step间隔"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5093428-12e6-491f-b65e-be6e4211300b",
   "metadata": {},
   "source": [
    "LLaMA-Factory还支持推理、合并lora和部署："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ccdb7cf9-1bcd-4458-8a75-20426011008c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 推理命令\n",
    "!USE_MODELSCOPE_HUB=1 llamafactory-cli chat examples/inference/llama3_lora_sft.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97bc4657-f3ba-41e0-90b5-7bc0be5bfa68",
   "metadata": {},
   "outputs": [],
   "source": [
    "# lora合并命令\n",
    "!USE_MODELSCOPE_HUB=1 llamafactory-cli export examples/merge_lora/llama3_lora_sft.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "59ba9eed-6f5d-4591-b0c8-12be665ef972",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 部署命令\n",
    "!USE_MODELSCOPE_HUB=1 API_PORT=8000 llamafactory-cli api examples/inference/llama3_vllm.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea37a5fa-9547-4a8d-91e1-ebbf6134dc9f",
   "metadata": {},
   "source": [
    "此外，LLamA-Factory支持界面训练，使用界面训练方式可以大大减小理解大模型训练的难度。\n",
    "使用界面训练的方式也非常简单："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c1f668f-7050-4c4b-b115-a5a1bf9fea8f",
   "metadata": {
    "ExecutionIndicator": {
     "show": true
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "!USE_MODELSCOPE_HUB=1 llamafactory-cli webui\n",
    "# 仅需要在界面中选择模型和数据集，配置好超参数点击开始即可"
   ]
  },
  {
   "cell_type": "raw",
   "id": "43b68ef4-51c4-4ffb-8c52-953c1c530698",
   "metadata": {},
   "source": [
    "在数据集配置上，LLamA-Factory支持自定义数据集，使用本地数据集的方式为：\n",
    "## SFT\n",
    "[\n",
    "  {\n",
    "    \"instruction\": \"今天的天气怎么样？\",\n",
    "    \"input\": \"\",\n",
    "    \"output\": \"今天的天气不错，是晴天。\",\n",
    "    \"history\": [\n",
    "      [\n",
    "        \"今天会下雨吗？\",\n",
    "        \"今天不会下雨，是个好天气。\"\n",
    "      ],\n",
    "      [\n",
    "        \"今天适合出去玩吗？\",\n",
    "        \"非常适合，空气质量很好。\"\n",
    "      ]\n",
    "    ]\n",
    "  }\n",
    "]\n",
    "\n",
    "同时编辑dataset_info.json文件将数据集填入：\n",
    "\"数据集名称\": {\n",
    "  \"file_name\": \"data.json\",\n",
    "  \"columns\": {\n",
    "    \"prompt\": \"instruction\",\n",
    "    \"query\": \"input\",\n",
    "    \"response\": \"output\",\n",
    "    \"system\": \"system\",\n",
    "    \"history\": \"history\"\n",
    "  }\n",
    "}\n",
    "\n",
    "## PT\n",
    "[\n",
    "  {\"text\": \"document\"},\n",
    "  {\"text\": \"document\"}\n",
    "]\n",
    "同时编辑dataset_info.json文件将数据集填入：\n",
    "\"数据集名称\": {\n",
    "  \"file_name\": \"data.json\",\n",
    "  \"columns\": {\n",
    "    \"prompt\": \"text\"\n",
    "  }\n",
    "}\n",
    "\n",
    "## RLHF\n",
    "[\n",
    "  {\n",
    "    \"instruction\": \"人类指令（必填）\",\n",
    "    \"input\": \"人类输入（选填）\",\n",
    "    \"chosen\": \"优质回答（必填）\",\n",
    "    \"rejected\": \"劣质回答（必填）\"\n",
    "  }\n",
    "]\n",
    "\n",
    "同时编辑dataset_info.json文件将数据集填入：\n",
    "\"数据集名称\": {\n",
    "  \"file_name\": \"data.json\",\n",
    "  \"ranking\": true,\n",
    "  \"columns\": {\n",
    "    \"prompt\": \"instruction\",\n",
    "    \"query\": \"input\",\n",
    "    \"chosen\": \"chosen\",\n",
    "    \"rejected\": \"rejected\"\n",
    "  }\n",
    "}\n",
    "\n",
    "KTO格式的数据集和上述有所不同，应该为：\n",
    "[\n",
    "  {\n",
    "    \"instruction\": \"人类指令（必填）\",\n",
    "    \"input\": \"人类输入（选填）\",\n",
    "    \"output\": \"模型回答（必填）\",\n",
    "    \"kto_tag\": \"人类反馈 [true/false]（必填）\"\n",
    "  }\n",
    "]\n",
    "\n",
    "同时编辑dataset_info.json文件将数据集填入：\n",
    "\"数据集名称\": {\n",
    "  \"file_name\": \"data.json\",\n",
    "  \"columns\": {\n",
    "    \"prompt\": \"instruction\",\n",
    "    \"query\": \"input\",\n",
    "    \"response\": \"output\",\n",
    "    \"kto_tag\": \"kto_tag\"\n",
    "  }\n",
    "}\n",
    "\n",
    "## 多模态数据集\n",
    "\n",
    "[\n",
    "  {\n",
    "    \"instruction\": \"人类指令（必填）\",\n",
    "    \"input\": \"人类输入（选填）\",\n",
    "    \"output\": \"模型回答（必填）\",\n",
    "    \"images\": [\n",
    "      \"图像路径（必填）\"\n",
    "    ]\n",
    "  }\n",
    "]\n",
    "\n",
    "同时编辑dataset_info.json文件将数据集填入：\n",
    "\"数据集名称\": {\n",
    "  \"file_name\": \"data.json\",\n",
    "  \"columns\": {\n",
    "    \"prompt\": \"instruction\",\n",
    "    \"query\": \"input\",\n",
    "    \"response\": \"output\",\n",
    "    \"images\": \"images\"\n",
    "  }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0585ebaf-1545-4650-b817-21597b830bc8",
   "metadata": {},
   "source": [
    "完整的数据集文档请查看：https://llamafactory.readthedocs.io/zh-cn/latest/getting_started/data_preparation.html"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}