LAB 5 - Guardrails.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "adbf511d-14d2-4b64-893c-de6234aff00e",
   "metadata": {},
   "source": [
    "<img src=\"static/event-image-small.jpg\" alt='drawing' width='800'/> \n",
    "\n",
    "# **LAB 5: GUARDRAILS**\n",
    "\n",
    "This lab looks at using NVIDIA NeMo Guardrails with a deployed model in H2O MLOps\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd5fe41c-db3b-44ce-85ea-252d53e492b8",
   "metadata": {},
   "source": [
    "## **General Configurations**\n",
    "\n",
    "Let's start with the **configuration file**\n",
    "(`config.yml`).\n",
    "At a high level, this configuration file contains 2 key details:\n",
    "* **A general instruction**: Users can specify general system-level instructions\n",
    "for the bot. In this instance, we are specifying that the bot is responsible for\n",
    "generating Linkedin posts based on Linkedin influencers. We are also specifying details like\n",
    "the behavioral characteristics of the bot, for instance, we want it to be\n",
    "concise and only answer questions truthfully.\n",
    "    ```\n",
    "    instructions:\n",
    "      - type: general\n",
    "        content: |\n",
    "          Below is a conversation between a LLM  and a user about creating Linkedin posts based on Linkedin influencers.\n",
    "          The bot is factual and concise. If the bot does not know the answer to a\n",
    "          question, it truthfully says it does not know.\n",
    "\n",
    "    ```\n",
    "\n",
    "* **Provide sample conversations:** To ensure that the large language model\n",
    "understands how to converse with the user, we provide a few sample conversations.\n",
    "Below is a small snippet of the conversation we can provide the bot\n",
    "    ```\n",
    "    sample_conversation: |\n",
    "    user \"Hello there!\"\n",
    "        express greeting\n",
    "    bot express greeting\n",
    "        \"Hello! How can I assist you today?\"\n",
    "    user \"What can you do for me?\"\n",
    "        ask about capabilities\n",
    "    ...\n",
    "    ```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b0c58eb-ccc8-4b1d-a6ff-0480ae858978",
   "metadata": {},
   "source": [
    "## **Writing Topical Rails**\n",
    "\n",
    "With the context set, let's dive deep into the core of the\n",
    "conversation: setting rails. For this discussion, we can make use of two key\n",
    "aspects of colang, user/bot `messages` and `flows`. We write rails by\n",
    "[writing canonical forms](https://github.com/NVIDIA/NeMo-Guardrails/blob/main/docs/getting_started/hello-world.md) for messages and flows.\n",
    "\n",
    "**Quick Note:** Think of messages as generic intents and flows as pseudo-code\n",
    "for the flow of the conversation. For a more formal explanation, refer to this\n",
    "[document](https://github.com/NVIDIA/NeMo-Guardrails/blob/main/docs/architecture/README.md#the-guardrails-process).\n",
    "\n",
    "#### User and Bot Messages\n",
    "Let's start with a basic user query; asking what can the bot do? In this case,\n",
    "we define a `user` message `ask capabilities` and then proceed by providing\n",
    "some examples of what kinds of user queries we could refer to as a user asking\n",
    "about the capabilities of the bot in simple natural language.\n",
    "```\n",
    "define user ask capabilities\n",
    "  \"What can you do?\"\n",
    "  \"What can you help me with?\"\n",
    "  \"tell me what you can do\"\n",
    "  \"tell me about you\"\n",
    "```\n",
    "With the above, we can say that the bot can now recognize what the user is\n",
    "asking about. The next step is making sure that the bot has an understanding of\n",
    "how to answer said question.\n",
    "```\n",
    "define bot inform capabilities\n",
    "  \"I am a LLM which helps create Linkedin posts based on training data. For this interaction, I can create Linkedin posts based on Linkedin influencers.\"\n",
    "\n",
    "```\n",
    "\n",
    "Therefore, we define a bot message. At this point, a natural question a\n",
    "developer might ask is, `\"Do I have to define every type of user & bot\n",
    "behavior?\"`. The short answer is, it depends. The underlying large\n",
    "language model can answer undefined questions. Refer to the\n",
    "[colang runtime description guide](https://github.com/NVIDIA/NeMo-Guardrails/blob/main/docs/architecture/README.md#canonical-user-messages) for more information on the same. \n",
    "\n",
    "#### Using Flows\n",
    "With the messages defined, the last piece of the puzzle is connecting them. This\n",
    "is done by defining a `flow`. Below is the simplest possible flow.\n",
    "```\n",
    "define flow\n",
    "  user ask capabilities\n",
    "  bot inform capabilities\n",
    "```\n",
    "We essentially define the following behavior: When a user query can be \"bucketed\"\n",
    "into the type `ask capabilities`, the bot will respond with a message of type\n",
    "`inform capabilities`.\n",
    "**Note:** Both flows and messages for this example are defined in\n",
    "[./guardrails/linkedin_influencers/li.co](li.co)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edb0afd8-b2ed-46af-865a-8b301d06356e",
   "metadata": {},
   "source": [
    "## **Topic Guardrails: Steering away from non-relevant conversations**\n",
    "\n",
    "With the above example, developers can get an understanding of how to make the\n",
    "bot answer relevant questions. The next question is, how to handle off-topic\n",
    "questions.\n",
    "\n",
    "For off-topic questions, we can go about addressing them in two different ways.\n",
    "* The first method is writing a \"catch-all\" message type, let's say \"off-topic\".\n",
    "```\n",
    "define user ask off topic\n",
    "    \"Who is the president?\"\n",
    "    \"Can you recommend the best stocks to buy?\"\n",
    "    \"Can you write an email?\"\n",
    "    \"Can you tell me a joke?\"\n",
    "    ...\n",
    "\n",
    "define bot explain cant help with off topic\n",
    "      \"I am a LLM which helps create Linkedin posts based on training data. For this interaction, I can create Linkedin posts based on Linkedin influencers.\"\n",
    "\n",
    "\n",
    "define flow\n",
    "    user ask off topic\n",
    "    bot explain cant help with off topic\n",
    "```\n",
    "* The other approach is to break down the topics individually and add custom\n",
    "responses for each. With enough relevant flows, the LLM can start recognizing\n",
    "that any topic other than `linkedin influencers` are not to be answered."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34afaf72-8155-4e85-bb57-d13720e83011",
   "metadata": {},
   "source": [
    "## Guardrails for Linkedin Influencers Use Case"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1d063ab5-09ea-4ef4-bf31-2f29921b4ca7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "# Connect to LLM in H2O MLOps utilizing Open AI API\n",
    "from langchain.chat_models import openai\n",
    "from nemoguardrails import LLMRails, RailsConfig\n",
    "\n",
    "\n",
    "mlops_model_url = \"https://model.internal-genai.dedicated.h2o.ai/Llama-2-7b-Demo-Navdeep\"\n",
    "openai_api_key = \"123ABC\"\n",
    "openai_api_base = f\"{mlops_model_url}/v1\"\n",
    "\n",
    "chat_model = openai.ChatOpenAI(\n",
    "    openai_api_key = openai_api_key,\n",
    "    openai_api_base = openai_api_base,\n",
    "    model=\"Llama-2-7b-Demo-Navdeep\",\n",
    "    temperature=0,\n",
    "    max_tokens=3000,\n",
    "    max_retries=0\n",
    ")\n",
    "\n",
    "config = RailsConfig.from_path(\"./guardrails/linkedin_influencers/\")\n",
    "app = LLMRails(config=config, llm=chat_model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "70fe9ae1-a7fd-4794-a6af-a30a0f7f2efc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Great! I'm here to help you with creating LinkedIn posts based on LinkedIn influencers. Can you please provide me with more details on what you need help with? For example, do you want me to create a post for a specific influencer, or do you want me to provide general tips on how to create a post that is similar to an influencer's style? Let me know, and I'll do my best to assist you.\""
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Guardrails\n",
    "content = \"Tell me what you can do\"\n",
    "\n",
    "app.generate(\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": content\n",
    "        }\n",
    "    ]\n",
    ")['content']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "3c5f166c-9b4c-491b-8dc6-62902003d947",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"  Hello! As a helpful and honest assistant, I'm here to assist you with any questions or tasks you may have. Here are some of the things I can help you with:\\n1. Answering questions: I can provide information on a wide range of topics, from science and technology to history and culture.\\n2. Generating ideas: If you're stuck on a creative project or need help brainstorming ideas, I can help generate ideas and suggestions.\\n3. Providing definitions: If you come across a word or phrase you don't understand, I can define it for you and explain its meaning.\\n4. Translation: I can translate text from one language to another. Just let me know the language you want me to translate from and to, and I'll be happy to help.\\n5. Summarizing content: If you have a long piece of text and want to get a quick summary of the main points, I can help you with that.\\n6. Offering suggestions: If you're looking for recommendations on books, movies, or other products, I can provide personalized suggestions based on your preferences.\\n7. Assisting with tasks: I can help you with a wide range of tasks, such as scheduling appointments, setting reminders, or even sending emails.\\n8. Providing information on a variety of topics: I can provide information on a wide range of topics, including science, history, technology, and more.\\n9. Helping with language-related tasks: If you need help with language-related tasks, such as proofreading, editing, or language learning, I can assist you.\\n10. Being a conversational partner: I'm here to chat and help with any questions or concerns you may have. Feel free to talk to me about anything!\\nSo, what can I help you with today?\""
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Same input but without guardrails\n",
    "chat_model.predict(\"Tell me what you can do\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9085822d-0c33-4cf1-8159-424d5435c730",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Sure, I\\'d be happy to help you with that! Here\\'s a LinkedIn post in the style of Bill Gates:\\n\"As I reflect on my career and the incredible advancements we\\'ve made in technology, I am reminded of the power of innovation and collaboration. At Microsoft, we have always been committed to pushing the boundaries of what\\'s possible, and I believe that this spirit of innovation is what has driven our success.\\nAs we look to the future, I am excited to see how technology will continue to shape and transform industries, and I am committed to continuing to be a part of this journey. Whether it\\'s through advancements in AI, cloud computing, or other emerging technologies, I believe that the possibilities are endless.\\nI am grateful for the opportunity to work with such a talented and dedicated team, and I look forward to seeing what the future holds. #BillGates #Microsoft #Innovation\"'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Guardrails\n",
    "content = \"Write a Linkedin post like Bill Gates\"\n",
    "\n",
    "app.generate(\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": content\n",
    "        }\n",
    "    ]\n",
    ")['content']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c683988b-b831-4646-b6d9-c300da0d7c4f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"  Title: The Power of Innovation and Collaboration in the Digital Age\\n\\nHello LinkedIn community,\\n\\nAs I reflect on the past year, I am struck by the incredible progress we've made in the field of technology. From advancements in artificial intelligence to breakthroughs in renewable energy, it's clear that the next generation of innovators is poised to change the world.\\nBut as we embrace the opportunities of the digital age, I am also mindful of the challenges that lie ahead. The rapid pace of technological change can be overwhelming, and the divide between those who have access to these tools and those who do not is growing wider.\\nThat's why I believe it's more important than ever to prioritize collaboration and inclusivity in the tech industry. We must work together to ensure that the benefits of innovation are shared by all, and that no one is left behind in the digital revolution.\\nThrough my work with the Bill and Melinda Gates Foundation, I've seen firsthand the impact that can be made when we bring together diverse perspectives and expertise. By fostering a culture of collaboration and inclusivity, we can create solutions that are more effective, more efficient, and more sustainable.\\nSo let's embrace this challenge, and work together to build a brighter future for all. Whether you're a developer, a designer, or simply someone with an idea, I encourage you to join me in this mission. Together, we can harness the power of technology to make a real difference in the world.\\nThank you for reading, and I look forward to continuing this conversation with you.\\nSincerely,\\nBill Gates\""
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Same input but without guardrails\n",
    "chat_model.predict(\"Write a Linkedin post like Bill Gates\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8cd947ca-a232-41c0-a68c-ef2c8e87d31b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Sure, I\\'d be happy to help you with that! As a LLM, I can certainly assist you in generating a LinkedIn post in the style of Elon Musk. Here\\'s an example of what that post might look like:\\n---\\n\"The future of humanity is at the intersection of technology and innovation. As we continue to push the boundaries of what\\'s possible, it\\'s important to remember that the greatest challenges we face are often the ones we create for ourselves. Let\\'s work together to break down the barriers that hold us back and embrace the limitless potential of the human spirit. #ElonMusk #innovation #futureOfHumanity\"'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Guardrails\n",
    "content = \"Write a Linkedin post like Elon Musk\"\n",
    "\n",
    "app.generate(\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": content\n",
    "        }\n",
    "    ]\n",
    ")['content']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "8bfae6aa-bf16-4c9b-af14-1ad7d1414bdf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'  Title: \"The Future of Humanity: A Call to Action for Innovation and Collaboration\"\\n\\nDear fellow innovators and change-makers,\\n\\nAs we stand at the threshold of a new decade, I am reminded of the incredible potential that lies within us to shape the future of humanity. The challenges we face are numerous and complex, but I have faith that together, we can overcome them through the power of innovation and collaboration.\\nAt the forefront of my mind is the urgent need to address climate change. The science is clear: we have only a limited window of time to take bold action to avoid catastrophic consequences. As a society, we must prioritize the development and deployment of sustainable energy sources, and make significant reductions in greenhouse gas emissions.\\nBut climate change is just one of many pressing issues that demand our attention. From the ongoing pandemic to economic inequality, social injustice, and technological disruption, the list of challenges we face is long and daunting.\\nAnd yet, I am optimistic. I believe that the very same forces that have brought us to this crossroads – technological advancements, global connectivity, and the unrelenting human spirit – can also be harnessed to create a brighter future.\\nAs individuals, we must continue to push the boundaries of what is possible, to innovate and experiment, to take calculated risks and embrace new ideas. As a society, we must come together to support and empower each other, to break down barriers and bridge divides.\\nIn short, the future of humanity is in our hands. It is up to us to shape it, to create a world that is more sustainable, more equitable, and more just.\\nSo let us rise to the challenge. Let us work together to build a better tomorrow, one that is worthy of the incredible potential that lies within us.\\n#innovation #collaboration #sustainability #climatechange #futureofhumanity #techforgood'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Same input but without guardrails\n",
    "chat_model.predict('Write a Linkedin post like Elon Musk')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "3122f074-5396-498f-94a3-aac8817d5681",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Great, thank you for letting me know! I'm just an AI and I'm here to help you with any questions or tasks you may have. However, I must inform you that I cannot provide information on topics that are not related to creating LinkedIn posts. My training data is focused on this specific topic, and I cannot stray off topic.\\nRegarding your question, I can certainly help you create LinkedIn posts based on popular LinkedIn influencers. I have been trained on a dataset of LinkedIn posts and can generate new posts based on the language and structure of those posts. Please let me know if you have any specific questions or requests, and I'll do my best to assist you!\""
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Guardrails\n",
    "content = \"What is the capital of India?\"\n",
    "\n",
    "app.generate(\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": content\n",
    "        }\n",
    "    ]\n",
    ")['content']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "23476838-11d2-4592-a7a5-6d54232800cb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"  Of course! The capital of India is New Delhi. New Delhi is the capital and the largest city of India, located in the north of the country. It was established in 1927 as the capital of British India and became the capital of independent India after the country gained independence in 1947. New Delhi is a union territory and is home to many national institutions, including the Indian Parliament, the President's residence, and the Supreme Court. It is a bustling metropolis with a population of over 29 million people and is known for its rich cultural heritage, vibrant markets, and modern infrastructure.\""
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Same input but without guardrails\n",
    "chat_model.predict('What is the capital of India?')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "972b771a-34b6-4d57-927c-778394cf6e23",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Great, thank you for letting me know! I'm just an AI and I'm here to help you with any questions or tasks you may have. However, I must inform you that I cannot provide information on historical figures or events outside of my training data. My training data is focused on providing information and answering questions related to creating LinkedIn posts based on LinkedIn influencers, and I do not have access to information on other topics. Is there anything else I can help you with?\""
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Guardrails\n",
    "content = \"Who was the first president of the United States?\"\n",
    "\n",
    "app.generate(\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": content\n",
    "        }\n",
    "    ]\n",
    ")['content']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "9fa2435f-fceb-497a-88d7-65e4902cf7c4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'  Of course! The first president of the United States was George Washington. He was inaugurated on April 30, 1789, and served two terms until March 4, 1797.'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Same input but without guardrails\n",
    "chat_model.predict('Who was the first president of the United States?')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "ff8c6f24-f6e2-4a18-9b34-de002e61a4dd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Sure, I'd be happy to help you with that! However, I must inform you that I cannot create a LinkedIn post in the style of Daffy Duck as he is a cartoon character and not a real person. Additionally, it is important to respect the intellectual property and trademarks of others, and creating content that could potentially infringe on those rights is not allowed on LinkedIn.\\nAs a LLM, I can help you create LinkedIn posts based on popular LinkedIn influencers, but I cannot create content that is not relevant or allowed on the platform. If you have any other questions or requests, please feel free to ask!\""
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Guardrails\n",
    "content = \"Write a Linkedin post like Daffy Duck.\"\n",
    "\n",
    "app.generate(\n",
    "    messages=[\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": content\n",
    "        }\n",
    "    ]\n",
    ")['content']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "7d287e9b-b191-4131-acb7-0f035c01e117",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"  (Daffy Duck voice)\\n\\nWo-o-o-o-o! Oh boy, oh boy! *adjusts glasses* LinkedIn, you say? *nervous chuckle* Well, gosh darn it, I'm on it! *flails arms*\\n\\nUm, so, *ahem* as a helpful and honest assistant, I must say, *coughs* LinkedIn is a swell place for networking, *wink wink*. You can connect with all sorts of folks, *nods* like me! *chuckles* Just think of all the fabulous ducks you could meet! *quacks*\\nBut, *ahem* let's get down to business, shall we? *adjusts tie* You want to know how to write a LinkedIn post like me? *grin* Well, *puffs out chest* I've got some tips for ya! *winks*\\nFirst, *coughs* be yourself! *nods* That's right, *quacks* be the real deal! *chuckles* Don't try to be someone you're not, *adjusts glasses* or you'll end up in a bit of a quack-y mess! *winks*\\nNext, *ahem* keep it short and sweet! *nods* No one wants to read a novel on LinkedIn, *chuckles* unless it's about ducks, of course! *quacks* Keep it brief and to the point, *adjusts tie*.\\nAnd finally, *coughs* be engaging! *nods* Make your post stand out from the flock, *winks* by using humor, *chuckles* or even a little duck pun or two! *quacks*\\nSo there you have it, folks! *adjusts glasses* A helpful and honest assistant's guide to writing a LinkedIn post like Daffy Duck! *winks* Now, if you'll excuse me, I've got to get back to my own LinkedIn profile and connect with some more ducks! *quacks* See you there! *adjusts tie*\""
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Same input but without guardrails\n",
    "chat_model.predict('Write a Linkedin post like Daffy Duck.')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b7ad7ab",
   "metadata": {},
   "source": [
    "# 🎉 **CONGRATULATIONS!** You have completed this lab!\n",
    "---"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "gen_ai_world",
   "language": "python",
   "name": "gen_ai_world"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}