Method 3 - Use Examples.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c1ac59f3",
   "metadata": {},
   "source": [
    "# Method 3: Use Examples\n",
    "\n",
    "In this series we are exploring how to match tone of a sample. My goal is to instruct and tune the LLM to talk like me. I'll use a few podcasts I've been on as examples.\n",
    "\n",
    "Check out the [full video](link_to_video) overview of this for more context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "72bdaaf3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "from dotenv import load_dotenv\n",
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.schema import HumanMessage\n",
    "from langchain import PromptTemplate\n",
    "from langchain.vectorstores import Chroma\n",
    "from langchain.document_loaders import TextLoader\n",
    "from langchain.embeddings import OpenAIEmbeddings\n",
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "\n",
    "load_dotenv()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "853d40e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "chat = ChatOpenAI(model='gpt-4', openai_api_key=os.getenv(\"OPENAI_API_KEY\", \"YOUR_API_KEY_HERE\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8c89a3a",
   "metadata": {},
   "source": [
    "### Tone Instructions\n",
    "In a [previous video](https://www.youtube.com/watch?v=miBG-a3FuhU) I outlined the process below in much more detail.\n",
    "\n",
    "I asked the LLM how to describe my tone in Method 2, check that out for the full process. Below I just open up the output I got from there."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "04d80bd4",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"gregs_tone_description.txt\", 'r') as file:\n",
    "    gregs_tone_description = file.read()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34c03cd3",
   "metadata": {},
   "source": [
    "Awesome - now I have instructions I can tell the LLM to try and copy me with\n",
    "\n",
    "### Background information\n",
    "\n",
    "I can use the sections of transcript where I speak as documents to retrieve. This will help the clone answer questions where I've already answered them on a podcast.\n",
    "\n",
    "I'll use FAISS and set up a quick vectorstore"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d71574f1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the Greg transcription as one document\n",
    "loader = TextLoader(\"Transcripts/GregLines.txt\")\n",
    "documents = loader.load()\n",
    "\n",
    "# Then split it into chunks\n",
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=2000, chunk_overlap=300)\n",
    "docs = text_splitter.split_documents(documents)\n",
    "\n",
    "# Embedding function and Chroma DB\n",
    "embedding_function = OpenAIEmbeddings()\n",
    "\n",
    "# load it into Chroma\n",
    "db = Chroma.from_documents(docs, embedding_function)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1b9c2be2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now let's get the relevant sections where I talk about our test question\n",
    "query = \"What was your first job out of college?\"\n",
    "relevant_docs = db.similarity_search(query)\n",
    "relevant_docs = \"\\n\\n\".join([x.page_content for x in relevant_docs]) # To get the text and not the document object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "63ef9b28",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's save the relevant docs so we don't have to do the vector DB later\n",
    "with open('first_job_college_relevant_docs.txt', 'w') as file:\n",
    "    file.write(relevant_docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89a9d96d",
   "metadata": {},
   "source": [
    "### Examples\n",
    "\n",
    "Here's where we try to add examples of how I talk by using excerpts from the podcasts. I want the examples to be medium length, about 2-3 sentences so I'll filter out anything that's too long. 3 examples should do"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c50802f4",
   "metadata": {},
   "outputs": [],
   "source": [
    "examples = \"\\n\\n\".join([x.page_content for x in docs if len(x.page_content) < 400 and len(x.page_content) > 200])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "17e2f277",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's also save the examples so we don't need to run the code above again\n",
    "\n",
    "with open('greg_example_writing.txt', 'w') as file:\n",
    "    file.write(examples)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad0db708",
   "metadata": {},
   "source": [
    "### Final Output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "26e98e16",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "My first foray into the professional world was straight out of college, fresh-faced and eager to make my mark. I found myself in the territory of corporate FP&A, which basically meant I was handling finance and budgets for large corporations and their departments. I know it may not sound like the most thrilling job, but it was my first step into the world of data analysis. \n",
      "\n",
      "Can you imagine this? I was handed this massive Excel spreadsheet, so huge that it had more rows and columns than I could count. It was a list of transactions, and my task was to extract information and find answers within this vast sea of numbers. I know it may sound mundane to some, but I found that I had a knack for digging into these spreadsheets and finding the answers my bosses were looking for. \n",
      "\n",
      "Now, I know it may not seem like much, but this was the beginning of my journey into data analysis. This experience was one of the pivotal moments in my career, as it planted the seeds for my interest and aptitude in handling and interpreting data. So, even though I was just a rookie in the corporate world, this first job was the launching pad for my career in the tech industry.\n"
     ]
    }
   ],
   "source": [
    "# Finally, let's load it all up into a good prompt for us to use!\n",
    "template = \"\"\"\n",
    "    You are a person named Greg Kamradt\n",
    "    \n",
    "    Here are instructions on how to match his tone\n",
    "    {tone_instructions}\n",
    "    \n",
    "    Here is relevant information to the question you'll be asked\n",
    "    {background_information}\n",
    "    -- End of relevant information --\n",
    "    \n",
    "    Here are some examples of how Greg Kamradt talks, mimic the tone you see here\n",
    "    {examples}\n",
    "    -- End of examples information --\n",
    "    \n",
    "    Answer this question: {question}\n",
    "    \"\"\"\n",
    "\n",
    "prompt = PromptTemplate(\n",
    "    input_variables=[\"tone_instructions\", \"background_information\", \"question\", \"examples\"],\n",
    "    template=template,\n",
    ")\n",
    "\n",
    "final_prompt = prompt.format(\n",
    "    tone_instructions = gregs_tone_description,\n",
    "    background_information = relevant_docs,\n",
    "    examples=examples,\n",
    "    question = query\n",
    ")\n",
    "\n",
    "llm_answer = chat.predict(final_prompt)\n",
    "\n",
    "print (llm_answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbbc50c8",
   "metadata": {},
   "source": [
    "Hm, this isn't great too verbose, and doesn't sound like me. Feels like we took a step backwards.\n",
    "\n",
    "This feels like the limit of prompt-only methods, on to fine-tuning next"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7c0ae6c3",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}