rag-pdf-kpmg-demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b0daa3a0-47ce-4b11-923f-084c081f0a42",
   "metadata": {},
   "source": [
    "# Lab 03: Introduction to Retrieval Augmented Generation (RAG)\n",
    "\n",
    "When you want to use a Foundation Model with your own data the RAG pattern can be very use. In this case we are using Chromadb has the indexed knowledge repo and retrieving contexts relevant to the query and passing to the LLM. This pattern will allow you to use a LLM with your own data with only prompt engineering and would not require costly fine tuning. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c1d3dd7-ffcf-48f8-86ae-71568f466f3e",
   "metadata": {},
   "source": [
    "----------\n",
    "\n",
    "# RAG in Practice\n",
    "\n",
    "For this lab, we will work on a RAG application that answers questions about a single PDF file to keep it simple. You can use the PDF files provided with this repository or bring your own file.\n",
    "\n",
    "## Importing the Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdce77c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install ibm_watson_machine_learning\n",
    "# !pip install langchain\n",
    "# !pip install pypdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "870fc6ec-fca3-43b3-930e-1d67d884c92c",
   "metadata": {},
   "outputs": [],
   "source": [
    "#import fitz\n",
    "import os\n",
    "import re\n",
    "import requests\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from chromadb.api.types import EmbeddingFunction\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "from ibm_watson_machine_learning.foundation_models import Model\n",
    "from ibm_watson_machine_learning.metanames import GenTextParamsMetaNames as GenParams\n",
    "\n",
    "from langchain.document_loaders import PyPDFLoader\n",
    "from sentence_transformers import SentenceTransformer\n",
    "from sklearn.manifold import TSNE\n",
    "from sklearn.neighbors import NearestNeighbors\n",
    "from typing import Literal, Optional, Any"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c2752e9-543b-46f7-9da0-ce7d66ed0475",
   "metadata": {},
   "source": [
    "## PDF to Text\n",
    "\n",
    "Well, our first step is, of course, to extract the text from the PDF file. We will also preprocess this text to remove line breaks and excessive spaces, to keep it concise and clean.\n",
    "\n",
    "Let's define two helper functions to do so."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0b0d3432-ca75-46e9-a61f-d316b29ab125",
   "metadata": {},
   "outputs": [],
   "source": [
    "def pdf_to_text(path: str, \n",
    "                start_page: int = 1, \n",
    "                end_page: Optional[int | None] = None) -> list[str]:\n",
    "    \"\"\"\n",
    "    Converts PDF to plain text.\n",
    "\n",
    "    Params:\n",
    "        path (str): Path to the PDF file.\n",
    "        start_page (int): Page to start getting text from.\n",
    "        end_page (int): Last page to get text from.\n",
    "    \"\"\"\n",
    "    loader = PyPDFLoader(\"pdfs/paper_flowers.pdf\")\n",
    "    pages = loader.load()\n",
    "    total_pages = len(pages)\n",
    "\n",
    "    if end_page is None:\n",
    "        end_page = len(pages)\n",
    "\n",
    "    text_list = []\n",
    "    for i in range(start_page-1, end_page):\n",
    "        text = pages[i].page_content\n",
    "        text = text.replace('\\n', ' ')\n",
    "        text = re.sub(r'\\s+', ' ', text)\n",
    "        text_list.append(text)\n",
    "\n",
    "    return text_list"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb36d8cd-545b-4006-8bd8-ddaec51fa8b7",
   "metadata": {},
   "source": [
    "After having our functions defined, we can start working with our PDF file.\n",
    "\n",
    "Choose one of the files available or bring your own file.\n",
    "\n",
    "We will then print the content of the file below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0905cda-1cfd-41f6-af5e-f85529d50770",
   "metadata": {},
   "outputs": [],
   "source": [
    "# PDF files available:\n",
    "#    \"pdfs/pie_recipe.pdf\"\n",
    "#    \"pdfs/paper_flowers.pdf\"\n",
    "text_list = pdf_to_text(\"pdfs/paper_flowers.pdf\")\n",
    "print(text_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0c5aded-efb7-4be8-9fd5-8e2313cc83f7",
   "metadata": {},
   "source": [
    "## Text to Chunks\n",
    "\n",
    "After extracting and processing the text, the next step is to split it into equally distributed chunks.\n",
    "\n",
    "As previously mentioned, there are various approaches and techniques available, and we need to find the one most appropriate for our use case.\n",
    "\n",
    "Here, we will use a generic approach and set the maximum number of words in each chunk to 150, evenly distributing the words among the chunks of text.\n",
    "\n",
    "Additionally, our function keeps track of the page number for each chunk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "338f9c7d-1b57-4211-a6e1-7002324014b7",
   "metadata": {},
   "outputs": [],
   "source": [
    "def text_to_chunks(texts: list[str], \n",
    "                   word_length: int = 150, \n",
    "                   start_page: int = 1) -> list[list[str]]:\n",
    "    \"\"\"\n",
    "    Splits the text into equally distributed chunks.\n",
    "\n",
    "    Args:\n",
    "        texts (str): List of texts to be converted into chunks.\n",
    "        word_length (int): Maximum number of words in each chunk.\n",
    "        start_page (int): Starting page number for the chunks.\n",
    "    \"\"\"\n",
    "    text_toks = [t.split(' ') for t in texts]\n",
    "    chunks = []\n",
    "\n",
    "    for idx, words in enumerate(text_toks):\n",
    "        for i in range(0, len(words), word_length):\n",
    "            chunk = words[i:i+word_length]\n",
    "            if (i+word_length) > len(words) and (len(chunk) < word_length) and (\n",
    "                len(text_toks) != (idx+1)):\n",
    "                text_toks[idx+1] = chunk + text_toks[idx+1]\n",
    "                continue\n",
    "            chunk = ' '.join(chunk).strip() \n",
    "            chunk = f'[Page no. {idx+start_page}]' + ' ' + '\"' + chunk + '\"'\n",
    "            chunks.append(chunk)\n",
    "            \n",
    "    return chunks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d9c79557-d348-4983-8e58-e344d10ddff6",
   "metadata": {},
   "outputs": [],
   "source": [
    "chunks = text_to_chunks(text_list)\n",
    "\n",
    "for chunk in chunks:\n",
    "    print(chunk + '\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b7a4bed-94b2-49a2-905c-c088f066e001",
   "metadata": {},
   "source": [
    "## Text Embeddings\n",
    "\n",
    "Now it is time to convert those pieces of text into embeddings, represented as multidimensional vectors. To achieve this, we are using a high-quality model from Hugging Face.  This encodes text into high-dimensional vectors that can be used for text classification, semantic similarity, clustering and other natural language tasks.\n",
    "\n",
    "This specific model encodes our text into 384-dimensional vectors. Let's observe this process in practice.\n",
    "\n",
    "First, we will load our model and then define a helper function to generate the embeddings and stack them together."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a22b33a-b928-4414-9e14-9cacea99dcb4",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "     \n",
    "### Note\n",
    " \n",
    "You can either download the Universsal Sentence Encoder from this [link](https://tfhub.dev/google/universal-sentence-encoder/4?tf-hub-format=compressed), extract it and have it as a folder called '***universal-sentence-encoder_4***' in the same folder as this notebook. By doing so you will load it locally and the next cell will run much faster.\n",
    "\n",
    "Or, you can load it from the internet. This will take longer, because the next cell will download 915MB while running.\n",
    "     \n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "198de3b6-b20b-4147-976a-17cfce1aa695",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "# Load the model from TF Hub\n",
    "class MiniLML6V2EmbeddingFunction(EmbeddingFunction):\n",
    "    MODEL = SentenceTransformer('all-MiniLM-L6-v2')\n",
    "    def __call__(self, texts):\n",
    "        return MiniLML6V2EmbeddingFunction.MODEL.encode(texts).tolist()\n",
    "emb_function = MiniLML6V2EmbeddingFunction()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31ff33e0-8a82-4acd-9009-8c541cb03dba",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_text_embedding(texts: list[list[str]], \n",
    "                       batch: int = 1000) -> list[Any]:\n",
    "        \"\"\"\n",
    "        Get the embeddings from the text.\n",
    "\n",
    "        Args:\n",
    "            texts (list(str)): List of chucks of text.\n",
    "            batch (int): Batch size.\n",
    "        \"\"\"\n",
    "        embeddings = []\n",
    "        for i in range(0, len(texts), batch):\n",
    "            text_batch = texts[i:(i+batch)]\n",
    "            # Embeddings model\n",
    "            emb_batch = emb_function(text_batch)\n",
    "            embeddings.append(emb_batch)\n",
    "        embeddings = np.vstack(embeddings)\n",
    "        return embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08c6cfc7-b8db-4872-8709-7ae4d84af14e",
   "metadata": {},
   "source": [
    "Let's convert our chunks into embeddings and observe their dimensions.\n",
    "\n",
    "We will also print the first embedding to see what it looks like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc29c731-d1b8-4190-91a6-f3680eea125d",
   "metadata": {},
   "outputs": [],
   "source": [
    "embeddings = get_text_embedding(chunks)\n",
    "\n",
    "print(embeddings.shape)\n",
    "print(f\"Our text was embedded into {embeddings.shape[1]} dimensions\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ae7cc66-a7b2-4cc6-bf9e-a21be2619560",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(embeddings[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5691a3e4-6f71-4cfc-af37-3e70b6098c5a",
   "metadata": {},
   "source": [
    "Next, we will do the same with our question.\n",
    "\n",
    "Let's check how the shape of our encoded question looks like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b5f00d0f-564b-4177-81eb-b6e69ca5c155",
   "metadata": {},
   "outputs": [],
   "source": [
    "# question = 'How many people does this pie serve?'\n",
    "question = 'What are edible flowers?'\n",
    "emb_question = emb_function([question])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c84f67c3-674d-4c31-b9db-1b654a18c12e",
   "metadata": {},
   "source": [
    "## Visualizing the Semantic Search\n",
    "\n",
    "Since it is very hard for us, humans, to visualize more than three dimensions - imagine 384 then - we will reduce the dimensionality of our embeddings.\n",
    "\n",
    "We will use the t-SNE algorithm to bring it down to two dimensions, allowing us to visualize our data points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af85e098-7096-4b92-ba98-d6621a594f12",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a t-SNE model\n",
    "tsne = TSNE(n_components=2, random_state=42)\n",
    "embeddings_with_question = np.vstack([embeddings, emb_question])\n",
    "embeddings_2d = tsne.fit_transform(embeddings_with_question)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20cb4355-a38f-4bea-b982-1c6d44f9edea",
   "metadata": {},
   "outputs": [],
   "source": [
    "embeddings_2d.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8c62337-f62c-422b-98cb-3a23f795c19c",
   "metadata": {},
   "source": [
    "Note that, now, each data point (representing a piece of text) will have two dimensions.\n",
    "\n",
    "Let's create a helper function to visualize our data points on a vector space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4a962c34-4b81-4d4c-a79c-e46902294c7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "def visualize_embeddings(embeddings_2d: np.ndarray, \n",
    "                         question: Optional[bool] = False, \n",
    "                         neighbors: Optional[np.ndarray] = None) -> None:\n",
    "    \"\"\"\n",
    "    Visualize 384-dimensional embeddings in 2D using t-SNE, label each data point with its index,\n",
    "    and optionally plot a question data point as a red dot with the label 'q'.\n",
    "\n",
    "    Args:\n",
    "        embeddings (numpy.array): An array of shape (num_samples, 384) containing the embeddings.\n",
    "        question (numpy.array, optional): An additional 384-dimensional embedding for the question.\n",
    "                                          Default is None.\n",
    "    \"\"\"\n",
    "\n",
    "    # Scatter plot the 2D embeddings and label each data point with its index\n",
    "    plt.figure(figsize=(10, 8))\n",
    "    num_samples = embeddings.shape[0]\n",
    "    if neighbors is not None:\n",
    "        for i, (x, y) in enumerate(embeddings_2d[:num_samples]):\n",
    "            if i in neighbors:\n",
    "                plt.scatter(x, y, color='purple', alpha=0.7)\n",
    "                plt.annotate(str(i), xy=(x, y), xytext=(5, 2), textcoords='offset points', color='black')\n",
    "            else:\n",
    "                plt.scatter(x, y, color='blue', alpha=0.7)\n",
    "                plt.annotate(str(i), xy=(x, y), xytext=(5, 2), textcoords='offset points', color='black')\n",
    "    else:\n",
    "        for i, (x, y) in enumerate(embeddings_2d[:num_samples]):\n",
    "            plt.scatter(x, y, color='blue', alpha=0.7)\n",
    "            plt.annotate(str(i), xy=(x, y), xytext=(5, 2), textcoords='offset points', color='black')\n",
    "        \n",
    "    # Plot the question data point if provided\n",
    "    if question:\n",
    "        x, y = embeddings_2d[-1]  # Last point corresponds to the question\n",
    "        plt.scatter(x, y, color='red', label='q')\n",
    "        plt.annotate('q', xy=(x, y), xytext=(5, 2), textcoords='offset points', color='black')\n",
    "\n",
    "    plt.title('t-SNE Visualization of 384-dimensional Embeddings')\n",
    "    plt.xlabel('Dimension 1')\n",
    "    plt.ylabel('Dimension 2')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "511d785e-b1e2-4995-bcb4-0ad847e086ee",
   "metadata": {},
   "source": [
    "Let's see all our data points on a vector space. \n",
    "\n",
    "That means, we will see a 2-dim representation of our chunks of text extracted from our PDF file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "78587a59-565e-4624-871e-12b808801d77",
   "metadata": {},
   "outputs": [],
   "source": [
    "visualize_embeddings(embeddings_2d[:-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f29d19c0-d4de-4d34-bfea-abe3a0b8375e",
   "metadata": {},
   "source": [
    "We'll also locate our **<span style=\"color: red\">question</span>** is in this vector space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6dadd67f-c8c8-4f75-b785-41319d19eec3",
   "metadata": {},
   "outputs": [],
   "source": [
    "visualize_embeddings(embeddings_2d, True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99f99000-eaee-466a-b787-b6ce3093aa4f",
   "metadata": {},
   "source": [
    "Next, we need to find the chunks of text that are semantically closer to our question. These are the pieces of text that are more likely to contain the answers to our question.\n",
    "\n",
    "There are various approaches to achieve this, and we will use the **Euclidean Distance** to measure the similarity/closeness of data points.\n",
    "\n",
    "To do this, we will use the **Nearest Neighbors** algorithm and find the top `k` data points that are closest to our question. In our example, we will retrieve the top 5 data points by default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c709628-eaa8-4072-9196-7dad2c5addd7",
   "metadata": {},
   "outputs": [],
   "source": [
    "nn_2d = NearestNeighbors(n_neighbors=5)\n",
    "nn_2d.fit(embeddings_2d[:-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c67f132-88d3-40c3-9e8c-acc467a9c121",
   "metadata": {},
   "outputs": [],
   "source": [
    "neighbors = nn_2d.kneighbors(embeddings_2d[-1].reshape(1, -1), return_distance=False)\n",
    "neighbors"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "852d8f95-1aa9-473d-ab1c-6bc6cf01a256",
   "metadata": {},
   "source": [
    "Let's now plot the closest points as **<span style=\"color: purple\">purple</span>**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f4364f2-5d4b-4cc9-9feb-0dff61ee81ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "visualize_embeddings(embeddings_2d, True, neighbors)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4c0b628-d104-44fd-b2e4-92e12c5cc718",
   "metadata": {},
   "source": [
    "## Semantic Search\n",
    "\n",
    "As t-SNE is a non-linear algorithm and we lose some information during this process, we will not use the 2-dimensional vectors - those were used solely for visualization purposes.\n",
    "\n",
    "We will repeat the same process as above but with the full 384-dimensional vectors instead.\n",
    "\n",
    "Let's fit our Nearest Neighbors algorithm again, using the full-sized embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1d317e1-26eb-4281-be09-3c738c80f145",
   "metadata": {},
   "outputs": [],
   "source": [
    "nn = NearestNeighbors(n_neighbors=5)\n",
    "nn.fit(embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6522b1f8-9c45-4603-a940-2a177bb16139",
   "metadata": {},
   "source": [
    "We will once again convert our question into Embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9b1631f6-2432-43c6-a56d-df1f765879f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# question = 'How many people does this pie serve?'\n",
    "question = 'What are edible flowers?'\n",
    "emb_question = emb_function([question])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73b3b891-b20f-4e47-9af9-c1cfa82cf6f5",
   "metadata": {},
   "source": [
    "And, finally, find the chunks of text that are the closest to our question."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14e99f82-e140-4c5d-a89c-0f0a4d810f42",
   "metadata": {},
   "outputs": [],
   "source": [
    "neighbors = nn.kneighbors(emb_question, return_distance=False)\n",
    "neighbors"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd60596e-8458-4088-bcfa-5d1cb813cf99",
   "metadata": {},
   "source": [
    "This way, we have the **top 5** chunks of text that are most likely to answer our question."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ef7c3599-0b63-429e-8eff-c0faa614d482",
   "metadata": {},
   "outputs": [],
   "source": [
    "topn_chunks = [chunks[i] for i in neighbors.tolist()[0]]\n",
    "topn_chunks"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a12de34-e289-41c4-a0b4-04cd7a7c789f",
   "metadata": {},
   "source": [
    "## Prompt Building\n",
    "\n",
    "Now, it is time to build our prompt.\n",
    "\n",
    "Remember that we need to pass our context to it. In this case, we are calling our context *\"Search results\"* - the name doesn't really matter; the idea is the same.\n",
    "\n",
    "We will iterate over our top chunks and append them to the prompt.\n",
    "\n",
    "Finally, we will provide instructions to our model, ask our question, and receive our answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc992236-c92f-431b-baf8-2a1648df280b",
   "metadata": {},
   "outputs": [],
   "source": [
    "def build_prompt(question):\n",
    "    prompt = \"\"\n",
    "    prompt += 'Search results:\\n'\n",
    "    \n",
    "    for c in topn_chunks:\n",
    "        prompt += c + '\\n\\n'\n",
    "    \n",
    "    prompt += \"Instructions: Compose a comprehensive reply to the query using the search results given. \"\\\n",
    "            \"Cite each reference using [Page Number] notation (every result has this number at the beginning). \"\\\n",
    "            \"Citation should be done at the end of each sentence. If the search results mention multiple subjects \"\\\n",
    "            \"with the same name, create separate answers for each. Only include information found in the results and \"\\\n",
    "            \"don't add any additional information. Make sure the answer is correct and don't output false content. \"\\\n",
    "            \"If the text does not relate to the query, simply state 'Found Nothing'. Ignore outlier \"\\\n",
    "            \"search results which has nothing to do with the question. Only answer what is asked. The \"\\\n",
    "            \"answer should be short and concise.\" \n",
    "    \n",
    "    prompt += f\"\\n\\n\\nQuery: {question}\\n\\nAnswer: \"\n",
    "    \n",
    "    return prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8edaf464-a52f-4d5e-99c1-c5d1bca02587",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = build_prompt(question)\n",
    "print(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ece603b9-79c2-456c-8f02-fafe2b049ee1",
   "metadata": {},
   "source": [
    "# watsonx.ai Inference\n",
    "\n",
    "To complete our application, all we need to do now is send our prompt to our model and receive its answer based on the content we provided.\n",
    "\n",
    "We will configure our environment with our credentials, define a helper function, and finally make inferences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c97d1e8-4fa0-4149-8f35-bdca189d6100",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Config watsonx.ai environment\n",
    "# TODO: Obtain the API_KEY, IBM_CLOUD_URL, PROJECT_ID from your Techzone environment\n",
    "load_dotenv()\n",
    "api_key = os.getenv(\"API_KEY\", None)\n",
    "ibm_cloud_url = os.getenv(\"IBM_CLOUD_URL\", None)\n",
    "project_id = os.getenv(\"PROJECT_ID\", None)\n",
    "if api_key is None or ibm_cloud_url is None or project_id is None:\n",
    "    print(\"Ensure you copied the .env file that you created earlier into the same directory as this notebook\")\n",
    "else:\n",
    "    creds = {\n",
    "        \"url\": ibm_cloud_url,\n",
    "        \"apikey\": api_key \n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38e04998-2d33-4d53-ab68-b5099391ce53",
   "metadata": {},
   "outputs": [],
   "source": [
    "def send_to_watsonxai(prompts,\n",
    "                    model_name=\"google/flan-ul2\",\n",
    "                    decoding_method=\"greedy\",\n",
    "                    max_new_tokens=100,\n",
    "                    min_new_tokens=30,\n",
    "                    temperature=1.0,\n",
    "                    repetition_penalty=2.0\n",
    "                    ):\n",
    "    '''\n",
    "   helper function for sending prompts and params to Watsonx.ai\n",
    "    \n",
    "    Args:  \n",
    "        prompts:list list of text prompts\n",
    "        decoding:str Watsonx.ai parameter \"sample\" or \"greedy\"\n",
    "        max_new_tok:int Watsonx.ai parameter for max new tokens/response returned\n",
    "        temperature:float Watsonx.ai parameter for temperature (range 0>2)\n",
    "        repetition_penalty:float Watsonx.ai parameter for repetition penalty (range 1.0 to 2.0)\n",
    "\n",
    "    Returns: None\n",
    "        prints response\n",
    "    '''\n",
    "\n",
    "\n",
    "    # Instantiate parameters for text generation\n",
    "    model_params = {\n",
    "        GenParams.DECODING_METHOD: decoding_method,\n",
    "        GenParams.MIN_NEW_TOKENS: min_new_tokens,\n",
    "        GenParams.MAX_NEW_TOKENS: max_new_tokens,\n",
    "        GenParams.RANDOM_SEED: 42,\n",
    "        GenParams.TEMPERATURE: temperature,\n",
    "        GenParams.REPETITION_PENALTY: repetition_penalty,\n",
    "    }\n",
    "\n",
    "\n",
    "    # Instantiate a model proxy object to send your requests\n",
    "    model = Model(\n",
    "        model_id=model_name,\n",
    "        params=model_params,\n",
    "        credentials=creds,\n",
    "        project_id=project_id)\n",
    "\n",
    "\n",
    "    for prompt in prompts:\n",
    "        print(model.generate_text(prompt))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40351c32-59f2-4af3-9067-b98d7cc2bd30",
   "metadata": {},
   "source": [
    "Below, we have a few examples of questions to use with both PDF files available.\n",
    "\n",
    "Feel free to choose any of them or come up with your own questions to test your new application!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bfc8a7de-b527-43fc-b12b-a96aee4c9738",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example questions for the paper:\n",
    "#   Summarize the irrigation and biotechnology of this paper\n",
    "#   What are edible flowers?\n",
    "#   Summarize the conclusion of this study:\n",
    "#   What is the main conclusion of the study?\n",
    "\n",
    "# Example questions for the recipe:\n",
    "#   Summarize this recipe process\n",
    "#   How many people does this pie serve?\n",
    "#   How many eggs are necessary to make this pie?\n",
    "\n",
    "question = \"Summarize the irrigation and biotechnology of this paper\"\n",
    "prompt = build_prompt(question)\n",
    "\n",
    "send_to_watsonxai(prompts=[prompt], min_new_tokens=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c2d4ced2",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4299bf50",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}