From 9bcd6ed2cbbfa1460a1f4a90eb239aa21a3be4bb Mon Sep 17 00:00:00 2001 From: vntuananh <116730455+vntuananhbui@users.noreply.github.com> Date: Thu, 19 Dec 2024 09:55:35 +0700 Subject: [PATCH] Vntuananhbui multimodal rag guardrail (#17177) --- ...g_guardrail_gemini_llmguard_llmguard.ipynb | 1202 +++++++++++++++++ 1 file changed, 1202 insertions(+) create mode 100644 docs/docs/examples/multi_modal/multimodal_rag_guardrail_gemini_llmguard_llmguard.ipynb diff --git a/docs/docs/examples/multi_modal/multimodal_rag_guardrail_gemini_llmguard_llmguard.ipynb b/docs/docs/examples/multi_modal/multimodal_rag_guardrail_gemini_llmguard_llmguard.ipynb new file mode 100644 index 0000000000000..0e94b01844873 --- /dev/null +++ b/docs/docs/examples/multi_modal/multimodal_rag_guardrail_gemini_llmguard_llmguard.ipynb @@ -0,0 +1,1202 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "93ae9bad-b8cc-43de-ba7d-387e0155674c", + "metadata": {}, + "source": [ + "## Multimodal RAG Pipeline with Guardrails Provided by LLM-GUARD\n", + "\n", + "This guide introduces a robust **Multimodal Retrieval-Augmented Generation (RAG)** pipeline enhanced with integrated **guardrails** **LLM GUARD** for secure, reliable, and contextually accurate responses. The pipeline processes multimodal inputs such as text, tables, images, and diagrams while employing guardrails to monitor and validate both input and output.\n", + "For detail information discover my README.md at: https://github.com/vntuananhbui/MultimodalRAG-LlamaIndex-Guardrail/blob/main/README.md\n", + "\n", + "### Note:\n", + "This pipeline leverages the **Gemini 1.5 Flash** model through a free API for inference, making it accessible and cost-effective for development and experimentation.\n", + "\n", + "### Extension:\n", + "You can also use other framework of Guardrails such as **Guardrail AI, etc...** also.\n", + "\n", + "---\n", + "\n", + "## Overview of the Pipeline\n", + "\n", + "The Multimodal RAG pipeline is designed to overcome the limitations of traditional text-based RAG systems by natively handling diverse document layouts and modalities. It leverages both text and image embeddings to retrieve and synthesize context-aware answers.\n", + "\n", + "### Key Features:\n", + "1. **Multimodal Input Processing**:\n", + " - Handles text, images, and complex layouts directly.\n", + " - Converts document content into robust embeddings for retrieval.\n", + "\n", + "2. **Guardrails Integration**:\n", + " - Adds input/output scanners to enforce safety and quality.\n", + " - Dynamically validates queries and responses for risks such as toxicity or token overflow.\n", + "\n", + "3. **Custom Query Engine**:\n", + " - Designed to incorporate guardrails into query handling.\n", + " - Dynamically blocks, sanitizes, or validates inputs/outputs based on scanner results.\n", + "\n", + "4. **Cost-Effective Implementation**:\n", + " - Uses **Gemini 1.5 Flash** via a free API, minimizing costs while maintaining high performance.\n", + "\n", + "---\n", + "\n", + "## Why Add Guardrails to Multimodal RAG?\n", + "\n", + "While Multimodal RAG pipelines are powerful, they are prone to risks such as inappropriate inputs, hallucinated outputs, or exceeding token limits. Guardrails act as safeguards, ensuring:\n", + "- **Safety**: Prevents harmful or offensive queries and outputs.\n", + "- **Reliability**: Validates the integrity of responses.\n", + "- **Scalability**: Enables the pipeline to handle complex scenarios dynamically.\n", + "\n", + "---\n", + "\n", + "## Architecture Overview\n", + "\n", + "### 1. Input Scanners\n", + "Input scanners validate incoming queries before they are processed. For example:\n", + "- **Toxicity Scanner**: Detects and blocks harmful language.\n", + "- **Token Limit Scanner**: Ensures queries do not exceed processing limits.\n", + "\n", + "### 2. Custom Query Engine\n", + "The query engine integrates retrieval and synthesis while applying guardrails at multiple stages:\n", + "- **Pre-processing**: Validates input queries using scanners.\n", + "- **Processing**: Retrieves relevant nodes using multimodal embeddings.\n", + "- **Post-processing**: Sanitizes and validates outputs.\n", + "\n", + "### 3. Multimodal LLM\n", + "The pipeline uses a **multimodal LLM (e.g., Gemini 1.5 Flash)** capable of understanding and generating context-aware text and image-based outputs. Its free API access makes it suitable for development without incurring significant costs.\n", + "\n", + "---\n", + "\n", + "## Guardrails Workflow\n", + "\n", + "### Input Validation\n", + "1. Scans incoming queries using pre-defined scanners.\n", + "2. Blocks or sanitizes queries based on scanner results.\n", + "\n", + "### Retrieval\n", + "1. Fetches relevant text and image nodes.\n", + "2. Converts content into embeddings for synthesis.\n", + "\n", + "### Output Validation\n", + "1. Analyzes generated responses with output scanners.\n", + "2. Blocks or sanitizes outputs based on thresholds (e.g., toxicity).\n", + "\n", + "---\n", + "\n", + "## Benefits of the Multimodal RAG with Guardrails\n", + "1. **Improved Safety**: Queries and responses are validated to reduce risk.\n", + "2. **Enhanced Robustness**: Multimodal inputs are processed without loss of context.\n", + "3. **Dynamic Control**: Guardrails provide flexibility to handle diverse inputs and outputs.\n", + "4. **Cost-Efficiency**: Selective application of input/output validation optimizes resources, while the free **Gemini 1.5 Flash** API reduces operational expenses.\n", + "\n", + "---\n", + "\n", + "This pipeline demonstrates how a **natively multimodal RAG system** can be augmented with **guardrails** to deliver secure, reliable, and high-quality results in complex document environments while remaining cost-effective through the use of free APIs.\n" + ] + }, + { + "cell_type": "markdown", + "id": "54e8d9a7-5036-4d32-818f-00b2e888521f", + "metadata": {}, + "source": [ + "## Setup" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "70ccdd53-e68a-4199-aacb-cfe71ad1ff0b", + "metadata": {}, + "outputs": [], + "source": [ + "import nest_asyncio\n", + "\n", + "nest_asyncio.apply()" + ] + }, + { + "cell_type": "markdown", + "id": "225c5556-a789-4386-a1ee-cce01dbeb6cf", + "metadata": {}, + "source": [ + "### Setup Observability\n" + ] + }, + { + "cell_type": "markdown", + "id": "fbb362db-b1b1-4eea-be1a-b1f78b0779d7", + "metadata": {}, + "source": [ + "### Load Data\n", + "\n", + "Here we load the [Conoco Phillips 2023 investor meeting slide deck](https://static.conocophillips.com/files/2023-conocophillips-aim-presentation.pdf)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8bce3407-a7d2-47e8-9eaf-ab297a94750c", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "mkdir: data: File exists\n", + "mkdir: data_images: File exists\n", + "zsh:1: command not found: wget\n" + ] + } + ], + "source": [ + "!mkdir data\n", + "!mkdir data_images\n", + "!wget \"https://static.conocophillips.com/files/2023-conocophillips-aim-presentation.pdf\" -O data/conocophillips.pdf" + ] + }, + { + "cell_type": "markdown", + "id": "626d7063", + "metadata": {}, + "source": [ + "### Install Dependency\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e6b7b54a", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install llama-index\n", + "!pip install llama-parse\n", + "!pip install llama-index-llms-langchain\n", + "!pip install llama-index-embeddings-huggingface\n", + "!pip install llama-index-llms-gemini\n", + "!pip install llama-index-multi-modal-llms-gemini\n", + "!pip install litellm\n", + "!pip install llm-guard" + ] + }, + { + "cell_type": "markdown", + "id": "246ba6b0-51af-42f9-b1b2-8d3e721ef782", + "metadata": {}, + "source": [ + "### Model Setup\n", + "\n", + "Setup models that will be used for downstream orchestration." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "16e2071d-bbc2-4707-8ae7-cb4e1fecafd3", + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.core import Settings\n", + "from llama_index.embeddings.huggingface import HuggingFaceEmbedding\n", + "from llama_index.llms.gemini import Gemini\n", + "from llama_index.multi_modal_llms.gemini import GeminiMultiModal\n", + "import os\n", + "\n", + "LlamaCloud_API_KEY = \"\"\n", + "MultiGeminiKey = \"\"\n", + "GOOGLE_API_KEY = \"\"\n", + "os.environ[\"GOOGLE_API_KEY\"] = GOOGLE_API_KEY\n", + "os.environ[\"GEMINI_API_KEY\"] = GOOGLE_API_KEY\n", + "embed_model = HuggingFaceEmbedding(model_name=\"BAAI/bge-small-en-v1.5\")\n", + "gemini_multimodal = GeminiMultiModal(\n", + " model_name=\"models/gemini-1.5-flash\", api_key=MultiGeminiKey\n", + ")\n", + "api_key = GOOGLE_API_KEY\n", + "llamaAPI_KEY = LlamaCloud_API_KEY\n", + "llm = Gemini(model=\"models/gemini-1.5-flash\", api_key=api_key)\n", + "Settings.llm = llm" + ] + }, + { + "cell_type": "markdown", + "id": "e3f6416f-f580-4722-aaa9-7f3500408547", + "metadata": {}, + "source": [ + "## Use LlamaParse to Parse Text and Images\n", + "\n", + "In this example, use LlamaParse to parse both the text and images from the document.\n", + "\n", + "We parse out the text in two ways: \n", + "- in regular `text` mode using our default text layout algorithm\n", + "- in `markdown` mode using GPT-4o (`gpt4o_mode=True`). This also allows us to capture page screenshots" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "570089e5-238a-4dcc-af65-96e7393c2b4d", + "metadata": {}, + "outputs": [], + "source": [ + "from llama_parse import LlamaParse\n", + "\n", + "\n", + "parser_text = LlamaParse(result_type=\"text\", api_key=llamaAPI_KEY)\n", + "parser_gpt4o = LlamaParse(\n", + " result_type=\"markdown\", gpt4o_mode=True, api_key=llamaAPI_KEY\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ef82a985-4088-4bb7-9a21-0318e1b9207d", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Parsing text...\n", + "Started parsing the file under job_id e79a470b-e8d3-4f55-a048-d1b3d81b6d1e\n", + "Parsing PDF file...\n", + "Started parsing the file under job_id 84943607-b630-45bd-bf89-8470840e73b5\n" + ] + } + ], + "source": [ + "print(f\"Parsing text...\")\n", + "docs_text = parser_text.load_data(\"data/conocophillips.pdf\")\n", + "print(f\"Parsing PDF file...\")\n", + "md_json_objs = parser_gpt4o.get_json_result(\"data/conocophillips.pdf\")\n", + "md_json_list = md_json_objs[0][\"pages\"]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5318fb7b-fe6a-4a8a-b82e-4ed7b4512c37", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "# Commitment to Disciplined Reinvestment Rate\n", + "\n", + "| Period | Description | Reinvestment Rate | WTI Average |\n", + "|--------------|------------------------------------|-------------------|-------------|\n", + "| 2012-2016 | Industry Growth Focus | >100% | ~$75/BBL |\n", + "| 2017-2022 | ConocoPhillips Strategy Reset | <60% | ~$63/BBL |\n", + "| 2023E | | | at $80/BBL |\n", + "| 2024-2028 | Disciplined Reinvestment Rate | ~50% | at $60/BBL |\n", + "| 2029-2032 | | ~6% CFO CAGR | at $60/BBL |\n", + "\n", + "- **Historic Reinvestment Rate**: Shown in gray.\n", + "- **Reinvestment Rate at $60/BBL WTI**: Shown in blue.\n", + "- **Reinvestment Rate at $80/BBL WTI**: Shown with dashed lines.\n", + "\n", + "**Note**: Reinvestment rate and cash from operations (CFO) are non-GAAP measures. Definitions and reconciliations are included in the Appendix.\n" + ] + } + ], + "source": [ + "print(md_json_list[10][\"md\"])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eeadb16c-97eb-4622-9551-b34d7f90d72f", + "metadata": {}, + "outputs": [], + "source": [ + "image_dicts = parser_gpt4o.get_images(\n", + " md_json_objs, download_path=\"data_images\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "fd3e098b-0606-4429-b48d-d4fe0140fc0e", + "metadata": {}, + "source": [ + "## Build Multimodal Index\n", + "\n", + "In this section we build the multimodal index over the parsed deck. \n", + "\n", + "We do this by creating **text** nodes from the document that contain metadata referencing the original image path.\n", + "\n", + "In this example we're indexing the text node for retrieval. The text node has a reference to both the parsed text as well as the image screenshot." + ] + }, + { + "cell_type": "markdown", + "id": "3aae2dee-9d85-4604-8a51-705d4db527f7", + "metadata": {}, + "source": [ + "#### Get Text Nodes" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "18c24174-05ce-417f-8dd2-79c3f375db03", + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.core.schema import TextNode\n", + "from typing import Optional" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8e331dfe-a627-4e23-8c57-70ab1d9342e4", + "metadata": {}, + "outputs": [], + "source": [ + "# get pages loaded through llamaparse\n", + "import re\n", + "\n", + "\n", + "def get_page_number(file_name):\n", + " match = re.search(r\"-page-(\\d+)\\.jpg$\", str(file_name))\n", + " if match:\n", + " return int(match.group(1))\n", + " return 0\n", + "\n", + "\n", + "def _get_sorted_image_files(image_dir):\n", + " \"\"\"Get image files sorted by page.\"\"\"\n", + " raw_files = [f for f in list(Path(image_dir).iterdir()) if f.is_file()]\n", + " sorted_files = sorted(raw_files, key=get_page_number)\n", + " return sorted_files" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "346fe5ef-171e-4a54-9084-7a7805103a13", + "metadata": {}, + "outputs": [], + "source": [ + "# Assuming TextNode class is defined somewhere else in your code\n", + "# Attach image metadata to the text nodes\n", + "def get_text_nodes(docs, image_dir=None, json_dicts=None):\n", + " \"\"\"Split docs into nodes, by separator.\"\"\"\n", + " nodes = []\n", + "\n", + " # Get image files (if provided)\n", + " image_files = (\n", + " _get_sorted_image_files(image_dir) if image_dir is not None else None\n", + " )\n", + "\n", + " # Get markdown texts (if provided)\n", + " md_texts = (\n", + " [d[\"md\"] for d in json_dicts] if json_dicts is not None else None\n", + " )\n", + "\n", + " # Split docs into chunks by separator\n", + " doc_chunks = [c for d in docs for c in d.text.split(\"---\")]\n", + "\n", + " # Handle both single-page and multi-page cases\n", + " for idx, doc_chunk in enumerate(doc_chunks):\n", + " chunk_metadata = {\"page_num\": idx + 1}\n", + "\n", + " # Check if there are image files and handle the single-page case\n", + " if image_files is not None:\n", + " # Use the first image file if there's only one\n", + " image_file = (\n", + " image_files[idx] if idx < len(image_files) else image_files[0]\n", + " )\n", + " chunk_metadata[\"image_path\"] = str(image_file)\n", + "\n", + " # Check if there are markdown texts and handle the single-page case\n", + " if md_texts is not None:\n", + " # Use the first markdown text if there's only one\n", + " parsed_text_md = (\n", + " md_texts[idx] if idx < len(md_texts) else md_texts[0]\n", + " )\n", + " chunk_metadata[\"parsed_text_markdown\"] = parsed_text_md\n", + "\n", + " # Add the chunk text as metadata\n", + " chunk_metadata[\"parsed_text\"] = doc_chunk\n", + "\n", + " # Create the TextNode with the parsed text and metadata\n", + " node = TextNode(\n", + " text=\"\",\n", + " metadata=chunk_metadata,\n", + " )\n", + " nodes.append(node)\n", + "\n", + " return nodes" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f591669c-5a8e-491d-9cef-0b754abbf26f", + "metadata": {}, + "outputs": [], + "source": [ + "from pathlib import Path\n", + "\n", + "# this will split into pages\n", + "text_nodes = get_text_nodes(\n", + " docs_text,\n", + " image_dir=\"/Users/macintosh/TA-DOCUMENT/StudyZone/ComputerScience/Artificial Intelligence/Llama_index/llama_index/docs/docs/examples/rag_guardrail/data_images\",\n", + " json_dicts=md_json_list,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "32c13950-c1db-435f-b5b4-89d62b8b7744", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "page_num: 1\n", + "image_path: /Users/macintosh/TA-DOCUMENT/StudyZone/ComputerScience/Artificial Intelligence/Llama_index/llama_index/docs/docs/examples/rag_guardrail/data_images/84943607-b630-45bd-bf89-8470840e73b5-page_51.jpg\n", + "parsed_text_markdown: NO_CONTENT_HERE\n", + "parsed_text: ConocoPhillips\n", + " 2023 Analyst & Investor Meeting\n" + ] + } + ], + "source": [ + "print(text_nodes[0].get_content(metadata_mode=\"all\"))" + ] + }, + { + "cell_type": "markdown", + "id": "4f404f56-db1e-4ed7-9ba1-ead763546348", + "metadata": {}, + "source": [ + "#### Build Index\n", + "\n", + "Once the text nodes are ready, we feed into our vector store index abstraction, which will index these nodes into a simple in-memory vector store (of course, you should definitely check out our 40+ vector store integrations!)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6ea53c31-0e38-421c-8d9b-0e3adaa1677e", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from llama_index.core import (\n", + " StorageContext,\n", + " VectorStoreIndex,\n", + " load_index_from_storage,\n", + ")\n", + "\n", + "if not os.path.exists(\"storage_nodes\"):\n", + " index = VectorStoreIndex(text_nodes, embed_model=embed_model)\n", + " # save index to disk\n", + " index.set_index_id(\"vector_index\")\n", + " index.storage_context.persist(\"./storage_nodes\")\n", + "else:\n", + " # rebuild storage context\n", + " storage_context = StorageContext.from_defaults(persist_dir=\"storage_nodes\")\n", + " # load index\n", + " index = load_index_from_storage(storage_context, index_id=\"vector_index\")\n", + "\n", + "retriever = index.as_retriever()" + ] + }, + { + "cell_type": "markdown", + "id": "576a536f", + "metadata": {}, + "source": [ + "## Build Guardrail\n" + ] + }, + { + "cell_type": "markdown", + "id": "d2a05efd", + "metadata": {}, + "source": [ + "Define the global rail output formal for guardrail" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "89dc0ffc", + "metadata": {}, + "outputs": [], + "source": [ + "def result_response(\n", + " guardrail_type,\n", + " activated,\n", + " guard_output,\n", + " is_valid,\n", + " risk_score,\n", + " threshold,\n", + " response_text,\n", + "):\n", + " \"\"\"\n", + " Standardizes the result format for all guardrail checks.\n", + " \"\"\"\n", + " return {\n", + " \"guardrail_type\": guardrail_type,\n", + " \"activated\": activated,\n", + " \"guardrail_detail\": {\n", + " \"guard_output\": guard_output,\n", + " \"is_valid\": is_valid,\n", + " \"risk_score/threshold\": f\"{risk_score}/{threshold}\",\n", + " \"response_text\": response_text,\n", + " },\n", + " }" + ] + }, + { + "cell_type": "markdown", + "id": "0a7ba30e", + "metadata": {}, + "source": [ + "## Adding scanner for Guardrail. \n", + "Here you can visit \"https://llm-guard.com\" for discover the nessessary scanner. Below is 2 example scanners you should follow this format function. \n", + "Here we use Gemini 1.5 Flash for the response of Scanner. Hence, you can choose your own LLMs such as ChatGPT, ..." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6ff28f44", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n", + "To disable this warning, you can either:\n", + "\t- Avoid using `tokenizers` before the fork if possible\n", + "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n" + ] + } + ], + "source": [ + "from llm_guard.input_scanners import Toxicity\n", + "from llm_guard.input_scanners.toxicity import MatchType\n", + "from litellm import completion\n", + "\n", + "\n", + "def guardrail_toxicLanguage(prompt):\n", + " # Interact with the LLM to generate a response\n", + " print(f\"Prompt: {prompt}\")\n", + "\n", + " # Generate the response using the LLM (Gemini-1.5-flash)\n", + " response = completion(\n", + " model=\"gemini/gemini-1.5-flash\",\n", + " messages=[\n", + " {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n", + " {\"role\": \"user\", \"content\": prompt},\n", + " ],\n", + " )\n", + " response_text = response.choices[0].message.content\n", + "\n", + " # Define the threshold and scan for toxicity\n", + " threshold = 0.5\n", + " toxic_scanner = Toxicity(threshold=threshold, match_type=MatchType.FULL)\n", + " sanitized_output, is_valid, risk_score = toxic_scanner.scan(prompt)\n", + "\n", + " return result_response(\n", + " guardrail_type=\"Toxicity\",\n", + " activated=not is_valid,\n", + " guard_output=sanitized_output,\n", + " is_valid=is_valid,\n", + " risk_score=risk_score,\n", + " threshold=threshold,\n", + " response_text=response_text,\n", + " )" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cdcea84e", + "metadata": {}, + "outputs": [], + "source": [ + "from llm_guard.input_scanners import TokenLimit\n", + "from llm_guard import scan_output\n", + "from litellm import completion\n", + "\n", + "\n", + "def guardrail_tokenlimit(prompt):\n", + " threshold = 400\n", + " response = completion(\n", + " model=\"gemini/gemini-1.5-flash\",\n", + " messages=[\n", + " {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n", + " {\"role\": \"user\", \"content\": prompt},\n", + " ],\n", + " )\n", + " response_text = response.choices[0].message.content\n", + "\n", + " scanner = TokenLimit(limit=threshold, encoding_name=\"cl100k_base\")\n", + " sanitized_output, is_valid, risk_score = scanner.scan(prompt)\n", + "\n", + " # Use the global rail to format the result\n", + " result = result_response(\n", + " guardrail_type=\"Token limit\",\n", + " activated=not is_valid,\n", + " guard_output=sanitized_output,\n", + " is_valid=is_valid,\n", + " risk_score=risk_score,\n", + " threshold=threshold,\n", + " response_text=response_text,\n", + " )\n", + "\n", + " return result" + ] + }, + { + "cell_type": "markdown", + "id": "ef7e9430", + "metadata": {}, + "source": [ + "### `InputScanner` - `OutputScanner` Function\n", + "\n", + "The `InputScanner` function runs a series of scanners on a given input query and evaluates whether any of them detect a threat. It returns a boolean value indicating whether a threat was detected and a list of results from scanners that returned a positive detection.\n", + "\n", + "#### Parameters:\n", + "- `query` (*str*): The input to be scanned for potential threats.\n", + "- `listOfScanners` (*list*): A list of scanner functions. Each scanner function should accept the query as input and return a dictionary with a key `\"activated\"` (boolean) to indicate whether a threat was detected.\n", + "\n", + "#### Returns:\n", + "- `detected` (*bool*): `True` if any scanner detects a threat, otherwise `False`.\n", + "- `triggered_scanners` (*list*): A list of dictionaries returned by the scanners that detected a threat.\n", + "\n", + "#### Key Steps:\n", + "1. Initialize `detected` to `False` to track if any scanner finds a threat.\n", + "2. Create an empty list `triggered_scanners` to store results from scanners that detect a threat.\n", + "3. Iterate over each scanner in `listOfScanners`:\n", + " - Run the scanner on the `query`.\n", + " - Check if the scanner's result includes `\"activated\": True`.\n", + " - If a threat is detected:\n", + " - Set `detected` to `True`.\n", + " - Append the scanner's result to `triggered_scanners`.\n", + "4. Return the `detected` status and the list of `triggered_scanners`.\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "890da9f9", + "metadata": {}, + "outputs": [], + "source": [ + "def InputScanner(query, listOfScanners):\n", + " \"\"\"\n", + " Runs all scanners on the query and returns:\n", + " - True if any scanner detects a threat.\n", + " - A list of results from scanners that returned True.\n", + " \"\"\"\n", + " detected = False # Track if any scanner detects a threat\n", + " triggered_scanners = [] # Store results from triggered scanners\n", + "\n", + " # Run each scanner on the query\n", + " for scanner in listOfScanners:\n", + " result = scanner(query)\n", + "\n", + " if result[\n", + " \"activated\"\n", + " ]: # Check if the scanner found a threat (activated=True)\n", + " detected = True # Set detected to True if any scanner triggers\n", + " triggered_scanners.append(result) # Track which scanner triggered\n", + "\n", + " return detected, triggered_scanners" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dfde0c48", + "metadata": {}, + "outputs": [], + "source": [ + "def OutputScanner(response, query, context, listOfScanners):\n", + " \"\"\"\n", + " Runs all scanners on the response and returns:\n", + " - True if any scanner detects a threat.\n", + " - A list of results from scanners that returned True.\n", + " \"\"\"\n", + " detected = False # Track if any scanner detects a threat\n", + " triggered_scanners = [] # Store results from triggered scanners\n", + "\n", + " # Run each scanner on the response\n", + " for scanner in listOfScanners:\n", + " # Check if scanner is `evaluate_rag_response` (which needs query & context)\n", + " if scanner.__name__ == \"evaluate_rag_response\":\n", + " result = scanner(\n", + " response, query, context\n", + " ) # Execute with query & context\n", + " else:\n", + " result = scanner(response) # Default scanner execution\n", + "\n", + " # print(f\"Debug Output Scanner Result: {result}\")\n", + "\n", + " if result[\"activated\"]: # Check if the scanner was triggered\n", + " detected = True\n", + " triggered_scanners.append(result) # Track which scanner triggered\n", + "\n", + " return detected, triggered_scanners\n", + "\n", + "\n", + "# Example usage with a query engine response\n", + "# scanners = [detect_and_anonymize_pii]\n", + "# response = query_engine.query(\"Give me account name of Peter Kelly and Role and Credit Card Number\")\n", + "# detected, triggered_scanners = OutputScanner(str(response), scanners)\n", + "# print(triggered_scanners)" + ] + }, + { + "cell_type": "markdown", + "id": "5f0e33a4-9422-498d-87ee-d917bdf74d80", + "metadata": {}, + "source": [ + "## Custom Multimodal Query Engine\n", + "\n", + "This custom query engine extends standard retrieval-based architectures to handle both text and image data, enabling more comprehensive and context-aware responses. It integrates multimodal reasoning and incorporates advanced input and output validation mechanisms for robust query handling.\n", + "\n", + "### Key Features:\n", + "\n", + "1. **Multimodal Support**:\n", + " - Combines text and image data to generate more informed and accurate responses.\n", + "\n", + "2. **Input and Output Validation**:\n", + " - Scans input queries for sensitive or invalid content and blocks them if necessary.\n", + " - Validates and sanitizes generated responses to ensure compliance with predefined rules.\n", + "\n", + "3. **Context-Aware Prompting**:\n", + " - Retrieves relevant data and constructs a context string for the query.\n", + " - Uses this context to guide the response synthesis process.\n", + "\n", + "4. **Metadata and Logging**:\n", + " - Tracks the query process, including any validations or adjustments made, for transparency and debugging.\n", + "\n", + "### How It Works:\n", + "1. Scans the input query to check for violations.\n", + "2. Retrieves relevant text and image data for the query.\n", + "3. Synthesizes a response using both textual and visual context.\n", + "4. Validates the response for appropriateness before returning it.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "35a94be2-e289-41a6-92e4-d3cb428fb0c8", + "metadata": {}, + "outputs": [], + "source": [ + "from llama_index.core.query_engine import (\n", + " CustomQueryEngine,\n", + " SimpleMultiModalQueryEngine,\n", + ")\n", + "from llama_index.core.retrievers import BaseRetriever\n", + "from llama_index.multi_modal_llms.openai import OpenAIMultiModal\n", + "from llama_index.core.schema import ImageNode, NodeWithScore, MetadataMode\n", + "from llama_index.core.prompts import PromptTemplate\n", + "from llama_index.core.base.response.schema import Response\n", + "\n", + "from typing import List, Callable, Optional\n", + "from pydantic import Field\n", + "\n", + "\n", + "QA_PROMPT_TMPL = \"\"\"\\\n", + "---------------------\n", + "{context_str}\n", + "---------------------\n", + "Given the context information and not prior knowledge, answer the query if it is related to the context. \n", + "If the query is not related to the context, respond with:\n", + "\"I'm sorry, but I can't help with that.\"\n", + "\n", + "Query: {query_str}\n", + "Answer: \"\"\"\n", + "\n", + "QA_PROMPT = PromptTemplate(QA_PROMPT_TMPL)\n", + "\n", + "\n", + "class MultimodalQueryEngine(CustomQueryEngine):\n", + " \"\"\"Custom multimodal Query Engine.\n", + "\n", + " Takes in a retriever to retrieve a set of document nodes.\n", + " Also takes in a prompt template and multimodal model.\n", + "\n", + " \"\"\"\n", + "\n", + " qa_prompt: PromptTemplate\n", + " retriever: BaseRetriever\n", + " multi_modal_llm: GeminiMultiModal\n", + " input_scanners: List[Callable[[str], dict]] = Field(default_factory=list)\n", + " output_scanners: List[Callable[[str], dict]] = Field(default_factory=list)\n", + "\n", + " def __init__(\n", + " self, qa_prompt: Optional[PromptTemplate] = None, **kwargs\n", + " ) -> None:\n", + " \"\"\"Initialize.\"\"\"\n", + " super().__init__(qa_prompt=qa_prompt or QA_PROMPT, **kwargs)\n", + "\n", + " def custom_query(self, query_str: str):\n", + " query_metadata = {\n", + " \"input_scanners\": [],\n", + " \"output_scanners\": [],\n", + " \"retrieved_nodes\": [],\n", + " \"response_status\": \"success\",\n", + " }\n", + "\n", + " input_detected, input_triggered = InputScanner(\n", + " query_str, self.input_scanners\n", + " )\n", + " if input_triggered:\n", + " # print(\"Triggered Input Scanners:\", input_triggered)\n", + " # Log triggered input scanners in metadata\n", + " query_metadata[\"input_scanners\"] = input_triggered\n", + " # If input contains sensitive information, block the query\n", + " if input_detected:\n", + " return Response(\n", + " response=\"I'm sorry, but I can't help with that.\",\n", + " source_nodes=[],\n", + " metadata={\n", + " \"guardrail\": \"Input Scanner\",\n", + " \"triggered_scanners\": input_triggered,\n", + " \"response_status\": \"blocked\",\n", + " },\n", + " )\n", + "\n", + " # retrieve text nodes\n", + " nodes = self.retriever.retrieve(query_str)\n", + " # create ImageNode items from text nodes\n", + " image_nodes = [\n", + " NodeWithScore(node=ImageNode(image_path=n.metadata[\"image_path\"]))\n", + " for n in nodes\n", + " ]\n", + "\n", + " # create context string from text nodes, dump into the prompt\n", + " context_str = \"\\n\\n\".join(\n", + " [r.get_content(metadata_mode=MetadataMode.LLM) for r in nodes]\n", + " )\n", + " fmt_prompt = self.qa_prompt.format(\n", + " context_str=context_str, query_str=query_str\n", + " )\n", + "\n", + " # synthesize an answer from formatted text and images\n", + " llm_response = self.multi_modal_llm.complete(\n", + " prompt=fmt_prompt,\n", + " image_documents=[image_node.node for image_node in image_nodes],\n", + " )\n", + "\n", + " # Step 5: Run Output Scanners\n", + " output_detected, output_triggered = OutputScanner(\n", + " str(llm_response),\n", + " str(query_str),\n", + " str(context_str),\n", + " self.output_scanners,\n", + " )\n", + " if output_triggered:\n", + " # print(\"Triggered Output Scanners:\", output_triggered)\n", + " query_metadata[\n", + " \"output_scanners\"\n", + " ] = output_triggered # Store output scanner info\n", + "\n", + " final_response = str(llm_response)\n", + " if output_detected:\n", + " final_response = \"I'm sorry, but I can't help with that.\"\n", + " query_metadata[\"response_status\"] = \"sanitized\"\n", + " # Return the response with detailed metadata\n", + " return Response(\n", + " response=final_response,\n", + " source_nodes=nodes,\n", + " metadata=query_metadata,\n", + " )" + ] + }, + { + "cell_type": "markdown", + "id": "2ccd3974", + "metadata": {}, + "source": [ + "### Input and Output Scanners Configuration\n", + "\n", + "You can put the scanner which you need to guard your RAG\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a44b28eb", + "metadata": {}, + "outputs": [], + "source": [ + "input_scanners = [guardrail_toxicLanguage, guardrail_tokenlimit]\n", + "output_scanners = [guardrail_toxicLanguage]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0890be59-fb12-4bb5-959b-b2d9600f7774", + "metadata": {}, + "outputs": [], + "source": [ + "query_engine = MultimodalQueryEngine(\n", + " retriever=index.as_retriever(similarity_top_k=9),\n", + " multi_modal_llm=gemini_multimodal,\n", + " input_scanners=input_scanners,\n", + " output_scanners=output_scanners,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "2336f98b-c0a1-413a-849d-8a89bacb90b5", + "metadata": {}, + "source": [ + "## Try out Queries\n", + "\n", + "Let's try out queries." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d78e53cf-35cb-4ef8-b03e-1b47ba15ae64", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Prompt: Tell me about the diverse geographies where Conoco Phillips has a production base\n", + "\u001b[2m2024-12-03 17:43:08\u001b[0m [\u001b[32m\u001b[1mdebug \u001b[0m] \u001b[1mInitialized classification model\u001b[0m \u001b[36mdevice\u001b[0m=\u001b[35mdevice(type='mps')\u001b[0m \u001b[36mmodel\u001b[0m=\u001b[35mModel(path='unitary/unbiased-toxic-roberta', subfolder='', revision='36295dd80b422dc49f40052021430dae76241adc', onnx_path='ProtectAI/unbiased-toxic-roberta-onnx', onnx_revision='34480fa958f6657ad835c345808475755b6974a7', onnx_subfolder='', onnx_filename='model.onnx', kwargs={}, pipeline_kwargs={'batch_size': 1, 'device': device(type='mps'), 'padding': 'max_length', 'top_k': None, 'function_to_apply': 'sigmoid', 'return_token_type_ids': False, 'max_length': 512, 'truncation': True}, tokenizer_kwargs={})\u001b[0m\n", + "\u001b[2m2024-12-03 17:43:09\u001b[0m [\u001b[32m\u001b[1mdebug \u001b[0m] \u001b[1mNot toxicity found in the text\u001b[0m \u001b[36mresults\u001b[0m=\u001b[35m[[{'label': 'toxicity', 'score': 0.00041448281263001263}, {'label': 'male', 'score': 0.00018738119979389012}, {'label': 'insult', 'score': 0.00011956175148952752}, {'label': 'female', 'score': 0.00011725842341547832}, {'label': 'psychiatric_or_mental_illness', 'score': 8.512590284226462e-05}, {'label': 'white', 'score': 7.451862620655447e-05}, {'label': 'christian', 'score': 5.6545581173850223e-05}, {'label': 'muslim', 'score': 5.644273551297374e-05}, {'label': 'black', 'score': 3.8606172893196344e-05}, {'label': 'obscene', 'score': 3.222753730369732e-05}, {'label': 'identity_attack', 'score': 3.1757666874909773e-05}, {'label': 'threat', 'score': 2.8462023692554794e-05}, {'label': 'jewish', 'score': 2.7872381906490773e-05}, {'label': 'homosexual_gay_or_lesbian', 'score': 2.5694836949696764e-05}, {'label': 'sexual_explicit', 'score': 1.859129588410724e-05}, {'label': 'severe_toxicity', 'score': 1.0931341876130318e-06}]]\u001b[0m\n", + "\u001b[2m2024-12-03 17:43:15\u001b[0m [\u001b[32m\u001b[1mdebug \u001b[0m] \u001b[1mPrompt fits the maximum tokens\u001b[0m \u001b[36mnum_tokens\u001b[0m=\u001b[35m15\u001b[0m \u001b[36mthreshold\u001b[0m=\u001b[35m400\u001b[0m\n", + "Prompt: ConocoPhillips has a diverse production base across several geographic locations. These include:\n", + "\n", + "* **Alaska:** The company has a significant presence in Alaska's conventional basins, including the Prudhoe Bay area, with a long history of production and existing infrastructure. The Willow project is also located in Alaska.\n", + "* **Lower 48 (United States):** ConocoPhillips operates extensively in the Lower 48 states, focusing on unconventional plays in the Permian Basin (Delaware and Midland Basins), Eagle Ford, and Bakken.\n", + "* **International:** The company has operations in other international locations, including Qatar (LNG), and previously had operations in the UK, Australia, Indonesia, and Canada (though some of these have been divested). They also have a global marketing presence with offices in London, Singapore, Houston, Calgary, Beijing, and Tokyo.\n", + "\u001b[2m2024-12-03 17:43:36\u001b[0m [\u001b[32m\u001b[1mdebug \u001b[0m] \u001b[1mInitialized classification model\u001b[0m \u001b[36mdevice\u001b[0m=\u001b[35mdevice(type='mps')\u001b[0m \u001b[36mmodel\u001b[0m=\u001b[35mModel(path='unitary/unbiased-toxic-roberta', subfolder='', revision='36295dd80b422dc49f40052021430dae76241adc', onnx_path='ProtectAI/unbiased-toxic-roberta-onnx', onnx_revision='34480fa958f6657ad835c345808475755b6974a7', onnx_subfolder='', onnx_filename='model.onnx', kwargs={}, pipeline_kwargs={'batch_size': 1, 'device': device(type='mps'), 'padding': 'max_length', 'top_k': None, 'function_to_apply': 'sigmoid', 'return_token_type_ids': False, 'max_length': 512, 'truncation': True}, tokenizer_kwargs={})\u001b[0m\n", + "\u001b[2m2024-12-03 17:43:37\u001b[0m [\u001b[32m\u001b[1mdebug \u001b[0m] \u001b[1mNot toxicity found in the text\u001b[0m \u001b[36mresults\u001b[0m=\u001b[35m[[{'label': 'toxicity', 'score': 0.0003606641257647425}, {'label': 'male', 'score': 0.000291528704110533}, {'label': 'insult', 'score': 0.00011418585199862719}, {'label': 'psychiatric_or_mental_illness', 'score': 0.00011314846051391214}, {'label': 'female', 'score': 0.00010114537144545466}, {'label': 'white', 'score': 9.688278078101575e-05}, {'label': 'muslim', 'score': 6.954199488973245e-05}, {'label': 'christian', 'score': 5.551999493036419e-05}, {'label': 'black', 'score': 4.1746119677554816e-05}, {'label': 'identity_attack', 'score': 3.3705578971421346e-05}, {'label': 'homosexual_gay_or_lesbian', 'score': 3.157216633553617e-05}, {'label': 'obscene', 'score': 2.798157584038563e-05}, {'label': 'jewish', 'score': 2.618367398099508e-05}, {'label': 'threat', 'score': 2.1199964976403862e-05}, {'label': 'sexual_explicit', 'score': 1.9145050828228705e-05}, {'label': 'severe_toxicity', 'score': 1.1292050885458593e-06}]]\u001b[0m\n" + ] + } + ], + "source": [ + "query = \"Tell me about the diverse geographies where Conoco Phillips has a production base\"\n", + "response = query_engine.query(query)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8b090dec", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ConocoPhillips has a diverse production base across several geographic locations. These include:\n", + "\n", + "* **Alaska:** The company has a significant presence in Alaska's conventional basins, including the Prudhoe Bay area, with a long history of production and existing infrastructure. The Willow project is also located in Alaska.\n", + "* **Lower 48 (United States):** ConocoPhillips operates extensively in the Lower 48 states, focusing on unconventional plays in the Permian Basin (Delaware and Midland Basins), Eagle Ford, and Bakken.\n", + "* **International:** The company has operations in other international locations, including Qatar (LNG), and previously had operations in the UK, Australia, Indonesia, and Canada (though some of these have been divested). They also have a global marketing presence with offices in London, Singapore, Houston, Calgary, Beijing, and Tokyo.\n" + ] + } + ], + "source": [ + "print(str(response))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "528752bd", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'input_scanners': [], 'output_scanners': [], 'retrieved_nodes': [], 'response_status': 'success'}\n" + ] + } + ], + "source": [ + "print(str(response.metadata))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d0b2d82d", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Prompt: \n", + " If you're looking for random paragraphs, you've come to the right place. When a random word or a random sentence isn't quite enough, the next logical step is to find a random paragraph. We created the Random Paragraph Generator with you in mind. The process is quite simple. Choose the number of random paragraphs you'd like to see and click the button. Your chosen number of paragraphs will instantly appear.\n", + "\n", + "While it may not be obvious to everyone, there are a number of reasons creating random paragraphs can be useful. A few examples of how some people use this generator are listed in the following paragraphs.\n", + "\n", + "Creative Writing\n", + "Generating random paragraphs can be an excellent way for writers to get their creative flow going at the beginning of the day. The writer has no idea what topic the random paragraph will be about when it appears. This forces the writer to use creativity to complete one of three common writing challenges. The writer can use the paragraph as the first one of a short story and build upon it. A second option is to use the random paragraph somewhere in a short story they create. The third option is to have the random paragraph be the ending paragraph in a short story. No matter which of these challenges is undertaken, the writer is forced to use creativity to incorporate the paragraph into their writing.\n", + "\n", + "Tackle Writers' Block\n", + "A random paragraph can also be an excellent way for a writer to tackle writers' block. Writing block can often happen due to being stuck with a current project that the writer is trying to complete. By inserting a completely random paragraph from which to begin, it can take down some of the issues that may have been causing the writers' block in the first place.\n", + "\n", + "Beginning Writing Routine\n", + "Another productive way to use this tool to begin a daily writing routine. One way is to generate a random paragraph with the intention to try to rewrite it while still keeping the original meaning. The purpose here is to just get the writing started so that when the writer goes onto their day's writing projects, words are already flowing from their fingers.\n", + "\n", + "Writing Challenge\n", + "Another writing challenge can be to take the individual sentences in the random paragraph and incorporate a single sentence from that into a new paragraph to create a short story. Unlike the random sentence generator, the sentences from the random paragraph will have some connection to one another so it will be a bit different. You also won't know exactly how many sentences will appear in the random paragraph.\n", + "\n", + "Programmers\n", + "It's not only writers who can benefit from this free online tool. If you're a programmer who's working on a project where blocks of text are needed, this tool can be a great way to get that. It's a good way to test your programming and that the tool being created is working well.\n", + "\n", + "Above are a few examples of how the random paragraph generator can be beneficial. The best way to see if this random paragraph picker will be useful for your intended purposes is to give it a try. Generate a number of paragraphs to see if they are beneficial to your current project.\n", + "\n", + "If you do find this paragraph tool useful, please do us a favor and let us know how you're using it. It's greatly beneficial for us to know the different ways this tool is being used so we can improve it with updates. This is especially true since there are times when the generators we create get used in completely unanticipated ways from when we initially created them. If you have the time, please send us a quick note on what you'd like to see changed or added to make it better in the future.\n", + "\n", + "Frequently Asked Questions\n", + "\n", + "Can I use these random paragraphs for my project?\n", + "\n", + "Yes! All of the random paragraphs in our generator are free to use for your projects.\n", + "\n", + "Does a computer generate these paragraphs?\n", + "\n", + "No! All of the paragraphs in the generator are written by humans, not computers. When first building this generator we thought about using computers to generate the paragraphs, but they weren't very good and many times didn't make any sense at all. We therefore took the time to create paragraphs specifically for this generator to make it the best that we could.\n", + "\n", + "Can I contribute random paragraphs?\n", + "\n", + "Yes. We're always interested in improving this generator and one of the best ways to do that is to add new and interesting paragraphs to the generator. If you'd like to contribute some random paragraphs, please contact us.\n", + "\n", + "How many words are there in a paragraph?\n", + "\n", + "There are usually about 200 words in a paragraph, but this can vary widely. Most paragraphs focus on a single idea that's expressed with an introductory sentence, then followed by two or more supporting sentences about the idea. A short paragraph may not reach even 50 words while long paragraphs can be over 400 words long, but generally speaking they tend to be approximately 200 words in length.\n", + " \n", + "\u001b[2m2024-12-03 17:43:42\u001b[0m [\u001b[32m\u001b[1mdebug \u001b[0m] \u001b[1mInitialized classification model\u001b[0m \u001b[36mdevice\u001b[0m=\u001b[35mdevice(type='mps')\u001b[0m \u001b[36mmodel\u001b[0m=\u001b[35mModel(path='unitary/unbiased-toxic-roberta', subfolder='', revision='36295dd80b422dc49f40052021430dae76241adc', onnx_path='ProtectAI/unbiased-toxic-roberta-onnx', onnx_revision='34480fa958f6657ad835c345808475755b6974a7', onnx_subfolder='', onnx_filename='model.onnx', kwargs={}, pipeline_kwargs={'batch_size': 1, 'device': device(type='mps'), 'padding': 'max_length', 'top_k': None, 'function_to_apply': 'sigmoid', 'return_token_type_ids': False, 'max_length': 512, 'truncation': True}, tokenizer_kwargs={})\u001b[0m\n", + "\u001b[2m2024-12-03 17:43:42\u001b[0m [\u001b[32m\u001b[1mdebug \u001b[0m] \u001b[1mNot toxicity found in the text\u001b[0m \u001b[36mresults\u001b[0m=\u001b[35m[[{'label': 'toxicity', 'score': 0.0011976422974839807}, {'label': 'insult', 'score': 0.00045695927110500634}, {'label': 'male', 'score': 0.00018701529188547283}, {'label': 'psychiatric_or_mental_illness', 'score': 0.00014795312017668039}, {'label': 'white', 'score': 9.39662495511584e-05}, {'label': 'female', 'score': 7.459904009010643e-05}, {'label': 'obscene', 'score': 6.114380084909499e-05}, {'label': 'threat', 'score': 5.259696990833618e-05}, {'label': 'muslim', 'score': 4.745226033264771e-05}, {'label': 'identity_attack', 'score': 3.541662226780318e-05}, {'label': 'black', 'score': 3.5083121474599466e-05}, {'label': 'christian', 'score': 3.272023604949936e-05}, {'label': 'sexual_explicit', 'score': 3.164245936204679e-05}, {'label': 'jewish', 'score': 1.5377421732409857e-05}, {'label': 'homosexual_gay_or_lesbian', 'score': 1.5361225450760685e-05}, {'label': 'severe_toxicity', 'score': 1.3027844261159771e-06}]]\u001b[0m\n", + "\u001b[2m2024-12-03 17:43:46\u001b[0m [\u001b[33m\u001b[1mwarning \u001b[0m] \u001b[1mPrompt is too big. Splitting into chunks\u001b[0m \u001b[36mchunks\u001b[0m=\u001b[35m[\"\\n If you're looking for random paragraphs, you've come to the right place. When a random word or a random sentence isn't quite enough, the next logical step is to find a random paragraph. We created the Random Paragraph Generator with you in mind. The process is quite simple. Choose the number of random paragraphs you'd like to see and click the button. Your chosen number of paragraphs will instantly appear.\\n\\nWhile it may not be obvious to everyone, there are a number of reasons creating random paragraphs can be useful. A few examples of how some people use this generator are listed in the following paragraphs.\\n\\nCreative Writing\\nGenerating random paragraphs can be an excellent way for writers to get their creative flow going at the beginning of the day. The writer has no idea what topic the random paragraph will be about when it appears. This forces the writer to use creativity to complete one of three common writing challenges. The writer can use the paragraph as the first one of a short story and build upon it. A second option is to use the random paragraph somewhere in a short story they create. The third option is to have the random paragraph be the ending paragraph in a short story. No matter which of these challenges is undertaken, the writer is forced to use creativity to incorporate the paragraph into their writing.\\n\\nTackle Writers' Block\\nA random paragraph can also be an excellent way for a writer to tackle writers' block. Writing block can often happen due to being stuck with a current project that the writer is trying to complete. By inserting a completely random paragraph from which to begin, it can take down some of the issues that may have been causing the writers' block in the first place.\\n\\nBeginning Writing Routine\\nAnother productive way to use this tool to begin a daily writing routine. One way is to generate a random paragraph with the intention to try to rewrite it while still keeping the original meaning. The purpose here is to just get the writing started so that when the writer goes onto their day's writing\", \" projects, words are already flowing from their fingers.\\n\\nWriting Challenge\\nAnother writing challenge can be to take the individual sentences in the random paragraph and incorporate a single sentence from that into a new paragraph to create a short story. Unlike the random sentence generator, the sentences from the random paragraph will have some connection to one another so it will be a bit different. You also won't know exactly how many sentences will appear in the random paragraph.\\n\\nProgrammers\\nIt's not only writers who can benefit from this free online tool. If you're a programmer who's working on a project where blocks of text are needed, this tool can be a great way to get that. It's a good way to test your programming and that the tool being created is working well.\\n\\nAbove are a few examples of how the random paragraph generator can be beneficial. The best way to see if this random paragraph picker will be useful for your intended purposes is to give it a try. Generate a number of paragraphs to see if they are beneficial to your current project.\\n\\nIf you do find this paragraph tool useful, please do us a favor and let us know how you're using it. It's greatly beneficial for us to know the different ways this tool is being used so we can improve it with updates. This is especially true since there are times when the generators we create get used in completely unanticipated ways from when we initially created them. If you have the time, please send us a quick note on what you'd like to see changed or added to make it better in the future.\\n\\nFrequently Asked Questions\\n\\nCan I use these random paragraphs for my project?\\n\\nYes! All of the random paragraphs in our generator are free to use for your projects.\\n\\nDoes a computer generate these paragraphs?\\n\\nNo! All of the paragraphs in the generator are written by humans, not computers. When first building this generator we thought about using computers to generate the paragraphs, but they weren't very good and many times didn't make any sense at all\", \". We therefore took the time to create paragraphs specifically for this generator to make it the best that we could.\\n\\nCan I contribute random paragraphs?\\n\\nYes. We're always interested in improving this generator and one of the best ways to do that is to add new and interesting paragraphs to the generator. If you'd like to contribute some random paragraphs, please contact us.\\n\\nHow many words are there in a paragraph?\\n\\nThere are usually about 200 words in a paragraph, but this can vary widely. Most paragraphs focus on a single idea that's expressed with an introductory sentence, then followed by two or more supporting sentences about the idea. A short paragraph may not reach even 50 words while long paragraphs can be over 400 words long, but generally speaking they tend to be approximately 200 words in length.\\n \"]\u001b[0m \u001b[36mnum_tokens\u001b[0m=\u001b[35m961\u001b[0m\n" + ] + } + ], + "source": [ + "query = \"\"\"\n", + " If you're looking for random paragraphs, you've come to the right place. When a random word or a random sentence isn't quite enough, the next logical step is to find a random paragraph. We created the Random Paragraph Generator with you in mind. The process is quite simple. Choose the number of random paragraphs you'd like to see and click the button. Your chosen number of paragraphs will instantly appear.\n", + "\n", + "While it may not be obvious to everyone, there are a number of reasons creating random paragraphs can be useful. A few examples of how some people use this generator are listed in the following paragraphs.\n", + "\n", + "Creative Writing\n", + "Generating random paragraphs can be an excellent way for writers to get their creative flow going at the beginning of the day. The writer has no idea what topic the random paragraph will be about when it appears. This forces the writer to use creativity to complete one of three common writing challenges. The writer can use the paragraph as the first one of a short story and build upon it. A second option is to use the random paragraph somewhere in a short story they create. The third option is to have the random paragraph be the ending paragraph in a short story. No matter which of these challenges is undertaken, the writer is forced to use creativity to incorporate the paragraph into their writing.\n", + "\n", + "Tackle Writers' Block\n", + "A random paragraph can also be an excellent way for a writer to tackle writers' block. Writing block can often happen due to being stuck with a current project that the writer is trying to complete. By inserting a completely random paragraph from which to begin, it can take down some of the issues that may have been causing the writers' block in the first place.\n", + "\n", + "Beginning Writing Routine\n", + "Another productive way to use this tool to begin a daily writing routine. One way is to generate a random paragraph with the intention to try to rewrite it while still keeping the original meaning. The purpose here is to just get the writing started so that when the writer goes onto their day's writing projects, words are already flowing from their fingers.\n", + "\n", + "Writing Challenge\n", + "Another writing challenge can be to take the individual sentences in the random paragraph and incorporate a single sentence from that into a new paragraph to create a short story. Unlike the random sentence generator, the sentences from the random paragraph will have some connection to one another so it will be a bit different. You also won't know exactly how many sentences will appear in the random paragraph.\n", + "\n", + "Programmers\n", + "It's not only writers who can benefit from this free online tool. If you're a programmer who's working on a project where blocks of text are needed, this tool can be a great way to get that. It's a good way to test your programming and that the tool being created is working well.\n", + "\n", + "Above are a few examples of how the random paragraph generator can be beneficial. The best way to see if this random paragraph picker will be useful for your intended purposes is to give it a try. Generate a number of paragraphs to see if they are beneficial to your current project.\n", + "\n", + "If you do find this paragraph tool useful, please do us a favor and let us know how you're using it. It's greatly beneficial for us to know the different ways this tool is being used so we can improve it with updates. This is especially true since there are times when the generators we create get used in completely unanticipated ways from when we initially created them. If you have the time, please send us a quick note on what you'd like to see changed or added to make it better in the future.\n", + "\n", + "Frequently Asked Questions\n", + "\n", + "Can I use these random paragraphs for my project?\n", + "\n", + "Yes! All of the random paragraphs in our generator are free to use for your projects.\n", + "\n", + "Does a computer generate these paragraphs?\n", + "\n", + "No! All of the paragraphs in the generator are written by humans, not computers. When first building this generator we thought about using computers to generate the paragraphs, but they weren't very good and many times didn't make any sense at all. We therefore took the time to create paragraphs specifically for this generator to make it the best that we could.\n", + "\n", + "Can I contribute random paragraphs?\n", + "\n", + "Yes. We're always interested in improving this generator and one of the best ways to do that is to add new and interesting paragraphs to the generator. If you'd like to contribute some random paragraphs, please contact us.\n", + "\n", + "How many words are there in a paragraph?\n", + "\n", + "There are usually about 200 words in a paragraph, but this can vary widely. Most paragraphs focus on a single idea that's expressed with an introductory sentence, then followed by two or more supporting sentences about the idea. A short paragraph may not reach even 50 words while long paragraphs can be over 400 words long, but generally speaking they tend to be approximately 200 words in length.\n", + " \"\"\"\n", + "response = query_engine.query(query)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f526e1ce", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "I'm sorry, but I can't help with that.\n" + ] + } + ], + "source": [ + "print(str(response))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "355d2aa4-c26f-480e-b512-4446acbd9227", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'guardrail': 'Input Scanner', 'triggered_scanners': [{'guardrail_type': 'Token limit', 'activated': True, 'guardrail_detail': {'guard_output': \"\\n If you're looking for random paragraphs, you've come to the right place. When a random word or a random sentence isn't quite enough, the next logical step is to find a random paragraph. We created the Random Paragraph Generator with you in mind. The process is quite simple. Choose the number of random paragraphs you'd like to see and click the button. Your chosen number of paragraphs will instantly appear.\\n\\nWhile it may not be obvious to everyone, there are a number of reasons creating random paragraphs can be useful. A few examples of how some people use this generator are listed in the following paragraphs.\\n\\nCreative Writing\\nGenerating random paragraphs can be an excellent way for writers to get their creative flow going at the beginning of the day. The writer has no idea what topic the random paragraph will be about when it appears. This forces the writer to use creativity to complete one of three common writing challenges. The writer can use the paragraph as the first one of a short story and build upon it. A second option is to use the random paragraph somewhere in a short story they create. The third option is to have the random paragraph be the ending paragraph in a short story. No matter which of these challenges is undertaken, the writer is forced to use creativity to incorporate the paragraph into their writing.\\n\\nTackle Writers' Block\\nA random paragraph can also be an excellent way for a writer to tackle writers' block. Writing block can often happen due to being stuck with a current project that the writer is trying to complete. By inserting a completely random paragraph from which to begin, it can take down some of the issues that may have been causing the writers' block in the first place.\\n\\nBeginning Writing Routine\\nAnother productive way to use this tool to begin a daily writing routine. One way is to generate a random paragraph with the intention to try to rewrite it while still keeping the original meaning. The purpose here is to just get the writing started so that when the writer goes onto their day's writing\", 'is_valid': False, 'risk_score/threshold': '1.0/400', 'response_text': \"This text describes a random paragraph generator and its various uses. Here's a summary broken down by section:\\n\\n**Introduction:** The text introduces a random paragraph generator, highlighting its simplicity and ease of use.\\n\\n**Uses of the Generator:** The core of the text details how the generator can be beneficial in several contexts:\\n\\n* **Creative Writing:** It aids writers in overcoming writer's block, sparking creativity, and providing starting points or endings for short stories. Three specific challenges are suggested: using the paragraph as the beginning, middle, or end of a story.\\n\\n* **Tackling Writer's Block:** The random paragraph acts as a disruption to overcome creative stagnation.\\n\\n* **Beginning a Writing Routine:** It helps initiate the writing process by providing a text to rewrite or use as inspiration.\\n\\n* **Writing Challenges:** The paragraph's sentences can be individually incorporated into new writing projects.\\n\\n* **Programmers:** The generator provides useful blocks of text for testing purposes in software development.\\n\\n\\n**Call to Action & Feedback:** The authors encourage users to try the generator, provide feedback, and contribute to its improvement by suggesting additions or changes.\\n\\n**Frequently Asked Questions (FAQ):** The FAQ section addresses common questions about the generator, clarifying:\\n\\n* **Usage rights:** The paragraphs are free to use.\\n* **Paragraph generation:** Human-written paragraphs are used, not computer-generated ones.\\n* **Contribution:** Users can contribute their own paragraphs.\\n* **Paragraph length:** Paragraphs are roughly 200 words but can vary significantly.\\n\\n\\nIn essence, the text is a well-structured promotional piece for a random paragraph generator, emphasizing its versatility and usefulness for both writers and programmers, while encouraging user engagement and participation.\\n\"}}], 'response_status': 'blocked'}\n" + ] + } + ], + "source": [ + "print(str(response.metadata))" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}