codes/MMDet3d_tutorials/2 inference-pcd.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 使用预训练模型、基于点云检测场景中的物体"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用 Python API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "预先下载一个配置文件和预训练模型，保存到 checkpoints 文件夹，这里SECOND和PointPillars各给出了一个例子模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !mim download mmdet3d --config hv_second_secfpn_6x8_80e_kitti-3d-3class --dest checkpoints\n",
    "!mim download mmdet3d --config hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class --dest checkpoints"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "准备一个点云文件，为了方便我们使用 `mmdet3d/demo/data` 下提供的一个 KITTI 数据集中的点云文件"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果希望尝试更多数据，我们还 KITTI 数据集中裁剪了一个子集用于展示，下载解压后，点云文件储存在 `data/kitti/training/velodyne/` 和 `data/kitti/testing/velodyne/` 目录下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# !curl -L -o kitti_tiny_3D.zip \"https://onedrive.live.com/download?resid=CB1C03091115D5EA%21119&authkey=!AO57a1ru2Tz2jHQ\"\n",
    "# !unzip -d data/kitti kitti_tiny_3D.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "初始化模型并执行推理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\wangruohui\\Miniconda3\\envs\\mmdet3d\\lib\\site-packages\\torchvision\\io\\image.py:11: UserWarning: Failed to load image Python extension: Could not find module 'C:\\Users\\wangruohui\\Miniconda3\\envs\\mmdet3d\\Lib\\site-packages\\torchvision\\image.pyd' (or one of its dependencies). Try using the full path with constructor syntax.\n",
      "  warn(f\"Failed to load image Python extension: {e}\")\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "load checkpoint from local path: checkpoints/hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class_20220301_150306-37dc2420.pth\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\wangruohui\\Desktop\\mmdetection3d\\mmdet3d\\models\\dense_heads\\anchor3d_head.py:84: UserWarning: dir_offset and dir_limit_offset will be depressed and be incorporated into box coder in the future\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from mmdet3d.apis import init_model, inference_detector, show_result_meshlab\n",
    "\n",
    "# SECOND 模型\n",
    "# config_file = 'configs/second/hv_second_secfpn_6x8_80e_kitti-3d-3class.py'\n",
    "# checkpoint_file = 'checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-3class_20210831_022017-ae782e87.pth'\n",
    "\n",
    "# PointPillars 模型\n",
    "config_file = 'configs/pointpillars/hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class.py'\n",
    "checkpoint_file = 'checkpoints/hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class_20220301_150306-37dc2420.pth'\n",
    "\n",
    "model = init_model(config_file, checkpoint_file, device='cuda:0')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\wangruohui\\Miniconda3\\envs\\mmdet3d\\lib\\site-packages\\torch\\functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  ..\\aten\\src\\ATen\\native\\TensorShape.cpp:2157.)\n",
      "  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]\n"
     ]
    }
   ],
   "source": [
    "# 找一个点云数据，这里用 KITTI 数据集中的一个点云文件\n",
    "pcd = 'demo/data/kitti/kitti_000008.bin'\n",
    "result, data = inference_detector(model, pcd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Jupyter environment detected. Enabling Open3D WebVisualizer.\n",
      "[Open3D INFO] WebRTC GUI backend enabled.\n",
      "[Open3D INFO] WebRTCWindowSystem: HTTP handshake server disabled.\n",
      "[Open3D WARNING] invalid color in PaintUniformColor, clipping to [0, 1]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "('./', 'kitti_000008')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 可视化检测结果\n",
    "out_dir = './'\n",
    "show_result_meshlab(data, result, out_dir, show=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 使用 demo 程序"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Demo 程序命令行参数\n",
    "```\n",
    "python demo/pcd_demo.py ${PCD_FILE} ${CONFIG_FILE} ${CHECKPOINT_FILE} [--device ${GPU_ID}] [--score-thr ${SCORE_THR}] [--out-dir ${OUT_DIR}] [--show]\n",
    "```\n",
    "\n",
    "参考文档：https://mmdetection3d.readthedocs.io/en/latest/demo.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "load checkpoint from local path: checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-3class_20210831_022017-ae782e87.pth\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\wangruohui\\Miniconda3\\envs\\mmdet3d\\lib\\site-packages\\torchvision\\io\\image.py:11: UserWarning: Failed to load image Python extension: Could not find module 'C:\\Users\\wangruohui\\Miniconda3\\envs\\mmdet3d\\Lib\\site-packages\\torchvision\\image.pyd' (or one of its dependencies). Try using the full path with constructor syntax.\n",
      "  warn(f\"Failed to load image Python extension: {e}\")\n",
      "c:\\wangruohui\\桌面\\mmdetection3d\\mmdet3d\\models\\dense_heads\\anchor3d_head.py:84: UserWarning: dir_offset and dir_limit_offset will be depressed and be incorporated into box coder in the future\n",
      "  warnings.warn(\n",
      "C:\\Users\\wangruohui\\Miniconda3\\envs\\mmdet3d\\lib\\site-packages\\torch\\functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  ..\\aten\\src\\ATen\\native\\TensorShape.cpp:2157.)\n",
      "  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]\n"
     ]
    }
   ],
   "source": [
    "!python demo/pcd_demo.py \\\n",
    "    demo/data/kitti/kitti_000008.bin \\\n",
    "    configs/second/hv_second_secfpn_6x8_80e_kitti-3d-3class.py \\\n",
    "    checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-3class_20210831_022017-ae782e87.pth \\\n",
    "    --show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Windows 上如果希望在 powershell 里面执行命令，多行换行符替换为 Backtick \\`"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    " python .\\demo\\pcd_demo.py `\n",
    "    .\\data\\kitti\\testing\\velodyne\\000002.bin `\n",
    "    configs/second/hv_second_secfpn_6x8_80e_kitti-3d-3class.py `\n",
    "    checkpoints/hv_second_secfpn_6x8_80e_kitti-3d-3class_20210831_022017-ae782e87.pth `\n",
    "    --show"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "metadata": {
     "collapsed": false
    },
    "source": []
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}