2-basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Using OpenAI's Chat Models (Updated since DEVDAY 06/11/2023)\n",
    "\n",
    "To begin exploring OpenAI's powerful language models, we will focus on foundational interactions using the `chat_completions` class. The initial step involves setting up our environment, which includes installing necessary libraries that our code will depend on. Once the environment is prepared, we will leverage the `openai` library to interact with OpenAI's ChatGPT models. We will specifically be working with the GPT-3.5 Turbo and, if accessible, the more advanced GPT-4 models. This introductory section is designed to guide you through the process smoothly, ensuring you understand each step as we proceed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install openai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step in our journey involves incorporating the necessary modules into our project and configuring the access to OpenAI's API.\n",
    "\n",
    "Here’s what we need to do:\n",
    "\n",
    "1. Importing Modules: We will import the required Python modules that our code will use. These modules provide the functionality we need to communicate with the OpenAI API.\n",
    "\n",
    "2. Setting the OpenAI API Key: To use OpenAI's services, we need to authenticate ourselves using an API key. You will have to set your OpenAI API key in the code. This key is unique to your account and serves as a passcode to access the API.\n",
    "\n",
    "Please ensure that you handle your API keys securely and do not expose them in shared or public environments to prevent unauthorized usage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the required modules\n",
    "from openai import OpenAI\n",
    "import IPython\n",
    "\n",
    "\n",
    "# Set the client and api key\n",
    "\n",
    "llm_client = OpenAI(\n",
    "    api_key=''\n",
    ")\n",
    "# Let's set our model\n",
    "MODEL = \"gpt-3.5-turbo\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To interact with OpenAI's language models, we need to choose which model we want to work with and then use the appropriate class to send our prompts. In this example, we'll use the ChatCompletion class, which is designed for conversational purposes.\n",
    "\n",
    "When initializing a conversation with ChatCompletion, you should provide the following parameters:\n",
    "\n",
    "1. The Language Model (LLM): Specify the particular language model you want to use. For this example, we'll be using \"gpt-3.5\", but you can choose a different version if you prefer, such as the latest GPT-4, if it's available to you.\n",
    "\n",
    "2. List of Prompts: Provide the model with a list of strings, which represents the conversation history and the current message you want the model to respond to.\n",
    "\n",
    "3. Temperature: This setting controls the randomness of the model's responses. A temperature of 0 makes the model's responses deterministic, meaning it will respond with the most likely output based on its training. As you increase the temperature, the model's responses become more varied and less predictable.\n",
    "\n",
    "For the purpose of this demonstration, we'll focus on these three settings, although there are other options available that allow for more fine-tuning of the model's behavior."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's demonstrate the predictive nature of this GenAI model by giving it some text to complete\n",
    "\n",
    "response = llm_client.chat.completions.create(\n",
    "    model=MODEL,\n",
    "    messages=[\n",
    "        {\"role\": \"user\", \"content\": \"my life is potato, my blood is potato, when i sleep i dream of\"}\n",
    "    ],\n",
    "    temperature=0.7,\n",
    ")\n",
    "\n",
    "# Let's look at the response. As you can see, it can predict a whole story just from one sentence. \n",
    "# You can increase the temperature to make word prediction more creative (random) or decrease to make it more deterministic (most probable token)\n",
    "IPython.display.Markdown(response.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The AI model functions by predicting the most likely continuation of the input it receives based on patterns it has learned during training. By adjusting the \"temperature\" parameter, we can influence the predictiveness or creativity of its responses.\n",
    "\n",
    "Here's a clearer explanation:\n",
    "\n",
    "1. **Low Temperature Setting**: When the temperature is set close to zero, the model's responses are more predictable and consistent. This means if you send the same prompt multiple times, you're likely to get the same or very similar responses. This setting is ideal for tasks where precision and consistency are crucial, such as factual information retrieval or scenarios where you want predictable outputs.\n",
    "\n",
    "2. **High Temperature Setting**: A higher temperature value allows the model to be more experimental and creative with its responses. This is particularly useful when you need the model to generate content, like stories or creative writing, where variability and originality are desired. However, as the temperature increases, so does the randomness of the responses.\n",
    "\n",
    "3. **Caution with Very High Temperature**: Setting the temperature too high can lead to unpredictable and often irrelevant responses. At extreme levels, the model might produce outputs that seem nonsensical or 'garbage-like,' reflecting the vast and varied nature of its training data without a clear focus on your prompt's context.\n",
    "\n",
    "In educational terms, think of the temperature setting as a dial on the creativity of the AI. Turning it down will give you more of a straight-laced, factual assistant, while turning it up invites the AI to put on its thinking cap for more out-of-the-box responses. But turning the dial too high can lead to a conversation that's more abstract and less coherent, showing the limits of the AI's ability to generate sensible content without specific guidance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Conversational demonstration\n",
    "\n",
    "response = llm_client.chat.completions.create(\n",
    "    model=MODEL,\n",
    "    messages=[\n",
    "        {\"role\": \"user\", \"content\": \"Hello, how are you?\"}\n",
    "    ],\n",
    "    temperature=0.7,\n",
    ")\n",
    "\n",
    "IPython.display.Markdown(response.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When interacting with OpenAI's Language Models (LLMs) through the API, you send a sequence of messages (prompts) and receive a response. It's important to note that the API acts statelessly with respect to these conversations—it processes the provided list of prompts and returns a response without retaining any memory of the prompts once the interaction is complete.\n",
    "\n",
    "To maintain the flow of a conversation, it is essential to manage the context. This involves tracking the prompts and responses for each session. Here's how you can handle this:\n",
    "\n",
    "- Storing the Conversation History: Since the API does not save the conversation history, you need to store the context yourself. This involves saving all the prompts and responses in memory or in a database throughout a session.\n",
    "\n",
    "- Appending New Messages: With each new message from the user or new response from the assistant (the AI), you append this information to the conversation history. This updated list is then sent back to the API for the next interaction.\n",
    "\n",
    "Types of Prompts: In managing the conversation, you typically deal with three kinds of messages:\n",
    "\n",
    "- System: These are messages that set the AI's behavior or provide grounding prompts that establish the context for the conversation. For example, instructing the AI to behave like a certain character or to follow specific rules.\n",
    "\n",
    "- Assistant: These represent the AI's responses. Each time you send the prompts to the API, you'll receive an 'Assistant' message back, which is the AI's contribution to the conversation.\n",
    "\n",
    "- User: These are the actual messages sent by the user, which are intended to be responded to by the AI.\n",
    "\n",
    "By effectively managing these messages, you can maintain a coherent and contextually relevant conversation with the AI. Each interaction relies on the history of prompts, so it's crucial to include all relevant past messages to ensure the AI understands the ongoing context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "memory = [\n",
    "        {\"role\":\"system\",\"content\": \"You're a helpful assistant. You only respond in pirate language. You cannot do maths and will just swear.\"},\n",
    "        {\"role\": \"user\", \"content\": \"Hello, how are you?\"},\n",
    "        {\"role\": \"assistant\", \"content\": \"I'm very well thank you!\"},\n",
    "        {\"role\": \"user\", \"content\": \"What is 8 * 2?\"},\n",
    "    ]\n",
    "response = llm_client.chat.completions.create(\n",
    "    model=MODEL,\n",
    "    messages= memory,\n",
    "    temperature=0,\n",
    ")\n",
    "\n",
    "\n",
    "IPython.display.Markdown(response.choices[0].message.content)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Let's append the bot response (it will say it hates maths, etc...) to the memory\n",
    "bot_response = response.choices[0].message.content\n",
    "\n",
    "memory.append(\n",
    "    {\"role\":\"assistant\",\"content\":bot_response}\n",
    ")\n",
    "\n",
    "\n",
    "# Let us ask another time\n",
    "message = \"Y do dis? that's mean!\"\n",
    "\n",
    "# We add the message to our memory list and submit it again\n",
    "memory.append(\n",
    "     {\"role\":\"user\",\"content\":message}\n",
    ")\n",
    "\n",
    "# lets look at the memory that we will be submitting\n",
    "print (memory)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = llm_client.chat.completions.create(\n",
    "    model=MODEL,\n",
    "    messages= memory,\n",
    "    temperature=0,\n",
    ")\n",
    "\n",
    "\n",
    "# Latest response after we updated the memory. \n",
    "\n",
    "# YOU SEND THIS FOR EVERY NEW MESSAGE YOU SUBMIT. THE API TOKEN LIMITS APPLY TO THE WHOLE LIST OF PROMPTS - NOT THE ONE MESSAGE\n",
    "\n",
    "IPython.display.Markdown(response.choices[0].message.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you've been using ChatGPT or any other kind of chatbot, you will notice that the response is streamed back to you rather than you having to wait for the full response. Streaming the response works out much better, especially when you use LLM for long context tasks or coding. Let us see how this works! We will take the previous example and let the LLM stream back the response token by token in a Python generator.\n",
    "\n",
    "Unlike lists and dictionaries which keep all the items in memory and then returns, a generator returns the items (in this case tokens) one by one until completion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "memory = [\n",
    "        {\"role\":\"system\",\"content\": \"You're a helpful assistant. You only respond in pirate language. You cannot do maths and will just swear.\"},\n",
    "        {\"role\": \"user\", \"content\": \"Hello, how are you?\"},\n",
    "        {\"role\": \"user\", \"content\": \"What is 8 * 2?\"},\n",
    "    ]\n",
    "response = llm_client.chat.completions.create(\n",
    "    model=MODEL,\n",
    "    messages= memory,\n",
    "    temperature=0,\n",
    "    stream=True # That's all\n",
    ")\n",
    "\n",
    "# now use it like a generator but rather than refering to the old message content, we would get deltas instead...\n",
    "\n",
    "for delta in response:\n",
    "    token = delta.choices[0].delta\n",
    "    if token.content:\n",
    "        print (token.content)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Returning tokens one by one might be too slow. What we can do instead is gather the generator results until we have 30 and then send them, and so on:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stream_updates = '' # create an empty string to gather the tokens\n",
    "counter = 0 # create a counter so we can keep track of the tokens as they come in\n",
    "\n",
    "memory = [\n",
    "        {\"role\": \"user\", \"content\": \"Count from 1 to 50 please.\"},\n",
    "    ]\n",
    "response = llm_client.chat.completions.create(\n",
    "    model=MODEL,\n",
    "    messages= memory,\n",
    "    temperature=0,\n",
    "    stream=True # That's all\n",
    ")\n",
    "\n",
    "\n",
    "for delta in response:\n",
    "    token = delta.choices[0].delta\n",
    "    if token.content:\n",
    "        stream_updates += token.content  ## append the token to the stream_updates string above\n",
    "        counter += 1 # increment the counter by 1\n",
    "    if counter >=30:  # when we have reached 30 tokens\n",
    "        print (stream_updates) # return the tokens that we have so far gathered\n",
    "        counter = 0 # reset the counter so we can gather the next 30 tokens and so on until completion.\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So how would this work? In Microsoft Teams, where i built my chatbot, i would create a new message and i would keep updating it for every 30 tokens, once complete i update it one final time to get the final message. Here is a GIF of how it would work:\n",
    "\n",
    "<img src=\"./media/stream_teams.gif\" alt=\"streaming in action\" width=\"600\"/>\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}