packages/rag/llama-index/samples/LlamaIndex_Local-Models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1755d1f0-f19f-4438-a249-f8c3d487543e",
   "metadata": {},
   "source": [
    "# LlamaIndex - Starter Tutorial (Local Models)\n",
    "\n",
    "This tutorial is based on LlamaIndexofficial docs tutorial ([https://docs.llamaindex.ai/en/stable/getting_started/starter_example_local/](https://docs.llamaindex.ai/en/stable/getting_started/starter_example_local/#starter-tutorial-local-models)):"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f597b254-c829-4e34-8480-f0aba098204d",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Download data and folder setup\n",
    "\n",
    "Run the following cell to create a directory called `paul_grapham` besides this Jupyter notebook file and populate.\n",
    "\n",
    "Your directory structure should look like this:\n",
    "\n",
    "```\n",
    "├── LlamaIndex_Local-Models.ipynb\n",
    "└── paul_grapham\n",
    "    └── paul_grapham_essay.txt\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3609cf0a-b46e-4f1b-805c-4c86e7c383ea",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!mkdir -p paul_grapham\n",
    "!wget \"https://www.dropbox.com/s/f6bmb19xdg0xedm/paul_graham_essay.txt?dl=1\" -O ./paul_grapham/paul_grapham_essay.txt\n",
    "!ls -l ./paul_grapham"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e191cda6",
   "metadata": {},
   "source": [
    "## Download the Llama2 model\n",
    "\n",
    "Download the Llama2 model using `ollama` command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c627300c",
   "metadata": {},
   "outputs": [],
   "source": [
    "!ollama pull llama2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b353751b-c598-4388-9680-bcabe7ffa9b2",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Load data and build an index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2634dfa3-1263-42b9-901d-004f5166e38e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings\n",
    "from llama_index.core.embeddings import resolve_embed_model\n",
    "from llama_index.llms.ollama import Ollama\n",
    "\n",
    "reader = SimpleDirectoryReader(input_dir=\"./paul_grapham/\")\n",
    "documents = reader.load_data()\n",
    "print(f\"Loaded {len(documents)} docs\")\n",
    "\n",
    "# bge embedding model\n",
    "Settings.embed_model = resolve_embed_model(\"local:BAAI/bge-small-en-v1.5\")\n",
    "\n",
    "# ollama\n",
    "Settings.llm = Ollama(model=\"llama2\", request_timeout=60.0)\n",
    "\n",
    "index = VectorStoreIndex.from_documents(\n",
    "    documents,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6934e64e-4fa2-4a12-af1e-7a582ed63fef",
   "metadata": {},
   "source": [
    "## Query your data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6be1d7cd-7fe1-4a5f-a498-5bf486a38c58",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "query_engine = index.as_query_engine()\n",
    "\n",
    "response = query_engine.query(\"What did the author do growing up?\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5fc566dd-e2c7-41d9-8f9e-3de55cfde2d7",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}