10-3_haystack_ai.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bba43649-7cbc-4d14-bfbc-37e69709fa96",
   "metadata": {},
   "source": [
    "# Lecture 10-3\n",
    "\n",
    "# Messing around with LLMs with Haystack"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d613ee3b-3e9f-40aa-8c74-45df3785eee5",
   "metadata": {},
   "source": [
    "Step 0: Get OpenAI API Key\n",
    "\n",
    "To get an OpenAI API key, you need to sign up for an OpenAI account and subscribe to their API service. \n",
    "\n",
    "- Go to the OpenAI website.\n",
    "- Click on \"Sign Up\" to create a new account, or \"Log In\" if you already have an account.\n",
    "- Click on \"API\" at the top of the page.\n",
    "- You may need to set up a billing plan. If you stick with gpt 3.5, it is cheap. You can buy $5 of credits, which will be more than enough for experimentation.\n",
    "- Go to the API keys section of your OpenAI account.\n",
    "- Click on \"Create new secret key\" to generate a new API key.\n",
    "- Copy the generated API key.\n",
    "- Create a `.env` file in your project directory.\n",
    "- In the `.env` file, enter:\n",
    "\n",
    "```\n",
    "OPENAI_API_KEY=text_of_super_secret_api_key\n",
    "```\n",
    "\n",
    "- You won't be able to see the api key again once you close the dialog. If you don't store it and lose it, you'll need to create a new API key.  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be1ba30a-7ff2-478b-8765-6a88c54df695",
   "metadata": {},
   "source": [
    "Step 1: Create a New Conda Environment\n",
    "\n",
    "- Open your terminal (or Anaconda Prompt if you are on Windows).\n",
    "- Create a new Conda environment and activate it:\n",
    "\n",
    "```\n",
    "conda create -n haystack_env python=3.9\n",
    "conda activate haystack_env\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "521b14b1-54d0-490f-8f15-7f7eedcc8e5b",
   "metadata": {},
   "source": [
    "See: https://haystack.deepset.ai/tutorials/27_first_rag_pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b389a88-a0c0-4b57-8faf-ef8bb9924800",
   "metadata": {},
   "source": [
    "Once you've activated the haystack environment, install Jupyter, Haystack, and other necessary packages in your Conda environment:\n",
    "\n",
    "```\n",
    "conda install -c conda-forge jupyter\n",
    "pip install openai python-dotenv\n",
    "pip install torch\n",
    "pip install haystack-ai\n",
    "pip install \"datasets>=2.6.1\"\n",
    "pip install \"sentence-transformers>=2.2.0\"\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d97b7e83-7102-4dfe-85b9-1685a9f06d2f",
   "metadata": {},
   "source": [
    "Open jupyter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "5e88c5a3-388a-44e5-9f35-1fbb2603a246",
   "metadata": {},
   "outputs": [],
   "source": [
    "# cell 1: let's haystack know you are running a tutorial\n",
    "from haystack.telemetry import tutorial_running\n",
    "\n",
    "tutorial_running(27)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "99addb33-1a15-491e-9555-fef0bc709362",
   "metadata": {},
   "outputs": [],
   "source": [
    "from haystack.document_stores.in_memory import InMemoryDocumentStore\n",
    "\n",
    "document_store = InMemoryDocumentStore()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8bfdd37d-aeeb-44bf-9ba7-e3b79687be8d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from datasets import load_dataset\n",
    "from haystack import Document\n",
    "\n",
    "dataset = load_dataset(\"bilgeyucel/seven-wonders\", split=\"train\")\n",
    "docs = [Document(content=doc[\"content\"], meta=doc[\"meta\"]) for doc in dataset]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "24f91c87-9ecf-40d9-bc40-cb9df0c56803",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Miles\\anaconda3\\envs\\haystack_env\\lib\\site-packages\\huggingface_hub\\file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from haystack.components.embedders import SentenceTransformersDocumentEmbedder\n",
    "\n",
    "doc_embedder = SentenceTransformersDocumentEmbedder(model=\"sentence-transformers/all-MiniLM-L6-v2\")\n",
    "doc_embedder.warm_up()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "cabde90c-51e5-49d0-94bd-790168b13369",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ad1bac20b9aa4a8682709bd08f144397",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/5 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "151"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "docs_with_embeddings = doc_embedder.run(docs)\n",
    "document_store.write_documents(docs_with_embeddings[\"documents\"])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a3ca751d-ef20-4a7f-a74f-ffea08bdf731",
   "metadata": {},
   "outputs": [],
   "source": [
    "from haystack.components.embedders import SentenceTransformersTextEmbedder\n",
    "\n",
    "text_embedder = SentenceTransformersTextEmbedder(model=\"sentence-transformers/all-MiniLM-L6-v2\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6b9d468f-ba96-459b-9e20-85652003abb9",
   "metadata": {},
   "outputs": [],
   "source": [
    "from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever\n",
    "\n",
    "retriever = InMemoryEmbeddingRetriever(document_store)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "36489bc7-2c46-4e4c-ae3e-f6ab9dc06242",
   "metadata": {},
   "outputs": [],
   "source": [
    "from haystack.components.builders import PromptBuilder\n",
    "\n",
    "template = \"\"\"\n",
    "Given the following information, answer the question.\n",
    "\n",
    "Context:\n",
    "{% for document in documents %}\n",
    "    {{ document.content }}\n",
    "{% endfor %}\n",
    "\n",
    "Question: {{question}}\n",
    "Answer:\n",
    "\"\"\"\n",
    "\n",
    "prompt_builder = PromptBuilder(template=template)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "b333de27-ecd9-4f98-93bc-af3b73770d0e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from dotenv import load_dotenv\n",
    "import os\n",
    "\n",
    "# Load environment variables from .env file\n",
    "load_dotenv()\n",
    "\n",
    "from getpass import getpass\n",
    "from haystack.components.generators import OpenAIGenerator\n",
    "\n",
    "if \"OPENAI_API_KEY\" not in os.environ:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter OpenAI API key:\")\n",
    "generator = OpenAIGenerator(model=\"gpt-3.5-turbo\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e8556bdc-7637-469f-bbcd-04ea0ef8670d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<haystack.core.pipeline.pipeline.Pipeline object at 0x0000020F4D6ACD00>\n",
       "🚅 Components\n",
       "  - text_embedder: SentenceTransformersTextEmbedder\n",
       "  - retriever: InMemoryEmbeddingRetriever\n",
       "  - prompt_builder: PromptBuilder\n",
       "  - llm: OpenAIGenerator\n",
       "🛤️ Connections\n",
       "  - text_embedder.embedding -> retriever.query_embedding (List[float])\n",
       "  - retriever.documents -> prompt_builder.documents (List[Document])\n",
       "  - prompt_builder.prompt -> llm.prompt (str)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from haystack import Pipeline\n",
    "\n",
    "basic_rag_pipeline = Pipeline()\n",
    "# Add components to your pipeline\n",
    "basic_rag_pipeline.add_component(\"text_embedder\", text_embedder)\n",
    "basic_rag_pipeline.add_component(\"retriever\", retriever)\n",
    "basic_rag_pipeline.add_component(\"prompt_builder\", prompt_builder)\n",
    "basic_rag_pipeline.add_component(\"llm\", generator)\n",
    "\n",
    "# Now, connect the components to each other\n",
    "basic_rag_pipeline.connect(\"text_embedder.embedding\", \"retriever.query_embedding\")\n",
    "basic_rag_pipeline.connect(\"retriever\", \"prompt_builder.documents\")\n",
    "basic_rag_pipeline.connect(\"prompt_builder\", \"llm\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "f4693522-265d-4e3b-92b1-6956d81e70fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "70852c9e43f14796ad867838c555f138",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Rhodes Statue, also known as the Colossus of Rhodes, was a statue of the Greek sun-god Helios. According to scholars, while the full appearance of the statue is unknown, they do have a good idea of what the head and face looked like. The head would have had curly hair with evenly spaced spikes of bronze or silver flame radiating, similar to the images found on contemporary Rhodian coins. The statue stood approximately 70 cubits, or 33 meters (108 feet) high, making it the tallest statue in the ancient world.\n"
     ]
    }
   ],
   "source": [
    "question = \"What does Rhodes Statue look like?\"\n",
    "\n",
    "response = basic_rag_pipeline.run({\"text_embedder\": {\"text\": question}, \"prompt_builder\": {\"question\": question}})\n",
    "\n",
    "print(response[\"llm\"][\"replies\"][0])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "275fe483-68e2-4640-a557-9140e2a4853a",
   "metadata": {},
   "outputs": [],
   "source": [
    "examples = [\n",
    "    \"Where is Gardens of Babylon?\",\n",
    "    \"Why did people build Great Pyramid of Giza?\",\n",
    "    \"What does Rhodes Statue look like?\",\n",
    "    \"Why did people visit the Temple of Artemis?\",\n",
    "    \"What is the importance of Colossus of Rhodes?\",\n",
    "    \"What happened to the Tomb of Mausolus?\",\n",
    "    \"How did Colossus of Rhodes collapse?\",\n",
    "]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "272c08d5-f0c3-4739-b11d-4e512c33c730",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Where is Gardens of Babylon?\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "30d8df8a842e4a498ac26d8a6ba684b2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Gardens of Babylon are believed to have been located in the ancient city of Babylon, near present-day Hillah, Babil province, in Iraq.\n",
      "Why did people build Great Pyramid of Giza?\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "31c6c0424b62438f8f50f34d8f6b29d7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Great Pyramid of Giza was built as a tomb for the Fourth Dynasty pharaoh Khufu.\n",
      "What does Rhodes Statue look like?\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "dc99ef067bef43e39bac0b3480682bff",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Colossus of Rhodes was a statue of the Greek sun-god Helios, standing approximately 33 meters (108 feet) tall, with curly hair and spikes of bronze or silver flame radiating from its head. It was made of bronze and iron tie bars with brass plates forming the skin. The statue was built to celebrate the successful defense of Rhodes city against an attack by Demetrius Poliorcetes in 280 BC.\n",
      "Why did people visit the Temple of Artemis?\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e52d32664f0847abba4fe422389e3139",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "People visited the Temple of Artemis to pay homage to the goddess Artemis, offer jewelry and various goods, seek protection from persecution or punishment, attend festivals, and participate in games, contests, and theatrical performances held in her name. The temple was also considered an important attraction and sanctuary for those in need.\n",
      "What is the importance of Colossus of Rhodes?\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c9d2c08c374340a8bfc2eb19ebd9903b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Colossus of Rhodes was an important statue erected to celebrate the successful defense of Rhodes city against an attack by Demetrius Poliorcetes. It was considered one of the Seven Wonders of the Ancient World and was the tallest statue in the ancient world. The statue symbolized the mastery of the sea by the Rhodians, their victory over their enemies, and their autonomy. It also served as a significant votive offering and a representation of their patron god, Helios. Despite its eventual collapse, the Colossus of Rhodes remained a symbol of power and pride for the people of Rhodes.\n",
      "What happened to the Tomb of Mausolus?\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "83d04edeea0d400999256127637ccb24",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Tomb of Mausolus was destroyed by successive earthquakes from the 12th to the 15th century. Additionally, parts of the tomb were used for fortifying Bodrum Castle by the Knights of St. John of Rhodes around 1494, and later parts of the remaining tomb were broken up and used in the castle walls when it was strengthened in 1522 to defend against a potential Turkish invasion.\n",
      "How did Colossus of Rhodes collapse?\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d08bc0103ec44263aedc8245f5eb7286",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Colossus of Rhodes collapsed due to a significant earthquake in 226 BC. The earthquake caused damage to large portions of Rhodes, including the statue itself, which snapped at the knees and fell over onto the land.\n"
     ]
    }
   ],
   "source": [
    "for question in examples:\n",
    "    print(question)\n",
    "    response = basic_rag_pipeline.run({\"text_embedder\": {\"text\": question}, \"prompt_builder\": {\"question\": question}})\n",
    "    print(response[\"llm\"][\"replies\"][0])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "52c714f6-919f-4067-876e-067695a37c37",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}