12_bedrock/14_llamaindex_reranker_bedrock_claude.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f1b23ffc-22d8-4414-b2e4-66d45a03523d",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Using LLM-powered retreival and reranking - (Claude LLM + Bedrock Titan embedding)\n",
    "> *If you see errors, you may need to be allow-listed for the Bedrock models used by this notebook*\n",
    "\n",
    "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio*\n",
    "\n",
    "\n",
    "### Context\n",
    "\n",
    "Utilizing LLM-driven retrieval has the potential to yield more pertinent documents compared to retrieval based on embeddings. However, this advantage comes at the expense of increased latency and expenses. We will demonstrate that employing embedding-based retrieval initially, followed by a secondary retrieval stage for reevaluation, can offer a balanced solution.\n",
    "\n",
    "A recent surge in applications involving \"Develop a chatbot using your data\" has emerged in the past several months. This trend has been facilitated by frameworks such as LlamaIndex and LangChain. Many of these applications rely on a standard approach known as retrieval augmented generation (RAG):\n",
    "\n",
    "1) A vector store is employed to store unstructured documents (knowledge corpus).\n",
    "2) When presented with a query, a retrieval model is utilized to fetch relevant documents from the corpus, followed by a synthesis model that generates a response.\n",
    "3) The retrieval model retrieves the top-k documents based on the similarity of their embeddings to the query. It's important to note that the concept of top-k embedding-based semantic search has existed for over a decade and doesn't involve the use of LLM.\n",
    "\n",
    "The utilization of embedding-based retrieval offers numerous advantages:\n",
    "\n",
    "* Dot product calculations are swift and don't necessitate model invocations during query processing.\n",
    "* Although not flawless, embeddings can effectively capture the semantics of documents and queries. There's a subset of queries for which embedding-based retrieval yields highly relevant outcomes.\n",
    "\n",
    "However, embedding-based retrieval can exhibit imprecision and return irrelevant context for the query due to various factors. This subsequently diminishes the quality of the overall RAG system, irrespective of the LLM's quality.\n",
    "\n",
    "Addressing this challenge is not novel; existing information retrieval and recommendation systems have adopted a two-stage approach. The initial stage employs embedding-based retrieval with a higher top-k value to maximize recall while accepting a lower precision. Subsequently, the second stage utilizes a somewhat more computationally intensive process characterized by higher precision and lower recall (such as BM25) to \"rerank\" the initially retrieved candidates.\n",
    "\n",
    "Delving into the shortcomings of embedding-based retrieval would require an entire series of blog posts. This current post serves as an initial exploration of an alternative retrieval technique and its potential to enhance embedding-based retrieval methodologies.\n",
    " \n",
    "![LLM retrival works](./images/rag-overview.png)\n",
    "\n",
    "### LLM Retrieval and Reranking\n",
    "\n",
    "LLM Retrieval and reranking strategy employs the LLM to determine the document(s) or sections of text that align with the provided query. The input prompt comprises a collection of potential documents, and the LLM is entrusted with choosing the pertinent group of documents while also assigning a score to gauge their relevance using an internal measurement.\n",
    "\n",
    "\n",
    "In this notebook we explain how to approach the retriever pattern of LLM-powered retrieval and reranking using Amazon Bedrock LLM and LlamaIndex\n",
    "\n",
    "#### LlamaIndex\n",
    "LlamaIndex is a data framework for your LLM application. It provides the following tools:\n",
    "\n",
    "* Offers data connectors to ingest your existing data sources and data formats (APIs, PDFs, docs, SQL, etc.)\n",
    "* Provides ways to structure your data (indices, graphs) so that this data can be easily used with LLMs.\n",
    "* Provides an advanced retrieval/query interface over your data: Feed in any LLM input prompt, get back retrieved context and knowledge-augmented output.\n",
    "* Allows easy integrations with your outer application framework (e.g. with LangChain, Flask, Docker, anything else).\n",
    "* LlamaIndex provides tools for both beginner users and advanced users. Our high-level API allows beginner users to use LlamaIndex to ingest and query their data in 5 lines of code. Our lower-level APIs allow advanced users to customize and extend any module (data connectors, indices, retrievers, query engines, reranking modules), to fit their needs.\n",
    "\n",
    "### LLM Used:\n",
    "We will be leveraging Bedrock - Anthropic Claude LLM and Bedrock Embedding (Titan model) for demonstration.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f320367c-42b6-46e3-b367-de7327d6e665",
   "metadata": {},
   "source": [
    "### Setup\n",
    "\n",
    "We will first install the necessary libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b5d72382-652c-4108-bef4-b3c86fb76c18",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mWARNING: Ignoring invalid distribution -wscli (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mWARNING: Ignoring invalid distribution -wscli (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "daal4py 2021.6.0 requires daal==2021.4.0, which is not installed.\n",
      "spyder 5.3.3 requires pyqt5<5.16, which is not installed.\n",
      "spyder 5.3.3 requires pyqtwebengine<5.16, which is not installed.\n",
      "distributed 2022.7.0 requires tornado<6.2,>=6.0.3, but you have tornado 6.3.3 which is incompatible.\n",
      "hdijupyterutils 0.20.5 requires pandas<2.0.0,>=0.17.1, but you have pandas 2.1.1 which is incompatible.\n",
      "jupyterlab 3.4.4 requires jupyter-server~=1.16, but you have jupyter-server 2.7.3 which is incompatible.\n",
      "jupyterlab-server 2.10.3 requires jupyter-server~=1.4, but you have jupyter-server 2.7.3 which is incompatible.\n",
      "llama-index 0.8.37 requires urllib3<2, but you have urllib3 2.0.6 which is incompatible.\n",
      "numba 0.55.1 requires numpy<1.22,>=1.18, but you have numpy 1.26.0 which is incompatible.\n",
      "panel 0.13.1 requires bokeh<2.5.0,>=2.4.0, but you have bokeh 3.2.2 which is incompatible.\n",
      "sagemaker-datawrangler 0.4.3 requires sagemaker-data-insights==0.4.0, but you have sagemaker-data-insights 0.3.3 which is incompatible.\n",
      "sparkmagic 0.20.4 requires nest-asyncio==1.5.5, but you have nest-asyncio 1.5.8 which is incompatible.\n",
      "spyder 5.3.3 requires ipython<8.0.0,>=7.31.1, but you have ipython 8.15.0 which is incompatible.\n",
      "spyder 5.3.3 requires pylint<3.0,>=2.5.0, but you have pylint 3.0.0a7 which is incompatible.\n",
      "spyder-kernels 2.3.3 requires ipython<8,>=7.31.1; python_version >= \"3\", but you have ipython 8.15.0 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0m\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
      "\u001b[0mNote: you may need to restart the kernel to use updated packages.\n",
      "\u001b[33mWARNING: Ignoring invalid distribution -qlalchemy (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mWARNING: Ignoring invalid distribution -wscli (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mWARNING: Ignoring invalid distribution -qlalchemy (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mWARNING: Ignoring invalid distribution -wscli (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
      "\u001b[0mNote: you may need to restart the kernel to use updated packages.\n",
      "\u001b[33mWARNING: Ignoring invalid distribution -andas (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mWARNING: Ignoring invalid distribution -wscli (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mWARNING: Error parsing requirements for langchain: [Errno 2] No such file or directory: '/opt/conda/lib/python3.10/site-packages/langchain-0.0.309.dist-info/METADATA'\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mWARNING: Ignoring invalid distribution -andas (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mWARNING: Ignoring invalid distribution -wscli (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33m    WARNING: No metadata found in /opt/conda/lib/python3.10/site-packages\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33m    WARNING: No metadata found in /opt/conda/lib/python3.10/site-packages\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "daal4py 2021.6.0 requires daal==2021.4.0, which is not installed.\n",
      "spyder 5.3.3 requires pyqt5<5.16, which is not installed.\n",
      "spyder 5.3.3 requires pyqtwebengine<5.16, which is not installed.\n",
      "autovizwidget 0.20.5 requires pandas<2.0.0,>=0.20.1, but you have pandas 2.1.1 which is incompatible.\n",
      "distributed 2022.7.0 requires tornado<6.2,>=6.0.3, but you have tornado 6.3.3 which is incompatible.\n",
      "hdijupyterutils 0.20.5 requires pandas<2.0.0,>=0.17.1, but you have pandas 2.1.1 which is incompatible.\n",
      "jupyterlab 3.4.4 requires jupyter-server~=1.16, but you have jupyter-server 2.7.3 which is incompatible.\n",
      "jupyterlab-server 2.10.3 requires jupyter-server~=1.4, but you have jupyter-server 2.7.3 which is incompatible.\n",
      "numba 0.55.1 requires numpy<1.22,>=1.18, but you have numpy 1.26.0 which is incompatible.\n",
      "panel 0.13.1 requires bokeh<2.5.0,>=2.4.0, but you have bokeh 3.2.2 which is incompatible.\n",
      "sagemaker-datawrangler 0.4.3 requires sagemaker-data-insights==0.4.0, but you have sagemaker-data-insights 0.3.3 which is incompatible.\n",
      "sparkmagic 0.20.4 requires nest-asyncio==1.5.5, but you have nest-asyncio 1.5.8 which is incompatible.\n",
      "spyder 5.3.3 requires ipython<8.0.0,>=7.31.1, but you have ipython 8.15.0 which is incompatible.\n",
      "spyder 5.3.3 requires pylint<3.0,>=2.5.0, but you have pylint 3.0.0a7 which is incompatible.\n",
      "spyder-kernels 2.3.3 requires ipython<8,>=7.31.1; python_version >= \"3\", but you have ipython 8.15.0 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0m\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
      "\u001b[0mNote: you may need to restart the kernel to use updated packages.\n",
      "\u001b[33mWARNING: Ignoring invalid distribution -wscli (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0mNote: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install langchain==0.0.309 --force-reinstall --quiet\n",
    "%pip install pypdf==3.15.2 --force-reinstall --quiet\n",
    "%pip install llama-index==0.8.37 --force-reinstall --quiet\n",
    "%pip install sentence_transformers==2.2.2 --force-reinstall --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5aeb9b17-92c3-4e58-a12a-b35e629595de",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mWARNING: Ignoring invalid distribution -wscli (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mWARNING: Ignoring invalid distribution -wscli (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "spyder 5.3.3 requires pyqt5<5.16, which is not installed.\n",
      "spyder 5.3.3 requires pyqtwebengine<5.16, which is not installed.\n",
      "panel 0.13.1 requires bokeh<2.5.0,>=2.4.0, but you have bokeh 3.2.2 which is incompatible.\n",
      "spyder 5.3.3 requires ipython<8.0.0,>=7.31.1, but you have ipython 8.15.0 which is incompatible.\n",
      "spyder 5.3.3 requires pylint<3.0,>=2.5.0, but you have pylint 3.0.0a7 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0m\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
      "\u001b[0mNote: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install pydantic==1.10.13 --force-reinstall --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "91a1c19e-e102-462e-bbac-63fac3978940",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mWARNING: Ignoring invalid distribution -wscli (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[33mWARNING: Ignoring invalid distribution -wscli (/opt/conda/lib/python3.10/site-packages)\u001b[0m\u001b[33m\n",
      "\u001b[0m\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "spyder 5.3.3 requires pyqt5<5.16, which is not installed.\n",
      "spyder 5.3.3 requires pyqtwebengine<5.16, which is not installed.\n",
      "panel 0.13.1 requires bokeh<2.5.0,>=2.4.0, but you have bokeh 3.2.2 which is incompatible.\n",
      "spyder 5.3.3 requires ipython<8.0.0,>=7.31.1, but you have ipython 8.15.0 which is incompatible.\n",
      "spyder 5.3.3 requires pylint<3.0,>=2.5.0, but you have pylint 3.0.0a7 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0m\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
      "\u001b[0mNote: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install sqlalchemy==2.0.21 --force-reinstall --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "093fc356-f39e-49e3-99d1-0128db6824c0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# needed for llamaindex\n",
    "%pip install openai==0.28 --force-reinstall --quiet "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ab68a3ee-f4c1-4830-856a-9c37f7a42364",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import nest_asyncio\n",
    "\n",
    "nest_asyncio.apply()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8c16404b-db81-4ee6-9c0f-cc4cdf3cb147",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "from llama_index import (\n",
    "    VectorStoreIndex,\n",
    "    SimpleDirectoryReader,\n",
    "    ServiceContext,\n",
    "    LLMPredictor,\n",
    "    get_response_synthesizer,\n",
    "    set_global_service_context,\n",
    "    StorageContext,\n",
    "    ListIndex\n",
    ")\n",
    "from llama_index.indices.postprocessor import LLMRerank\n",
    "from IPython.display import Markdown, display"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ce8809c-b174-4970-8758-f867cf968d81",
   "metadata": {},
   "source": [
    "### Setup langchain and llama index\n",
    "\n",
    "In this step we will be creating of instance for LLM and embedding models. We will be using Claude and Titan models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "9222da53-3112-47ad-950d-832e020098ae",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#### Un comment the following lines to run from your local environment outside of the AWS account with Bedrock access\n",
    "\n",
    "#import os\n",
    "#os.environ['BEDROCK_ASSUME_ROLE'] = '<YOUR_VALUES>'\n",
    "#os.environ['AWS_PROFILE'] = 'bedrock-user'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "83932140-8e5b-457d-84ff-9cc3040c0576",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Create new client\n",
      "  Using region: us-west-2\n",
      "boto3 Bedrock client successfully created!\n",
      "bedrock-runtime(https://bedrock-runtime.us-west-2.amazonaws.com)\n"
     ]
    }
   ],
   "source": [
    "import boto3\n",
    "import json \n",
    "\n",
    "bedrock = boto3.client(service_name=\"bedrock\")\n",
    "bedrock_runtime = boto3.client(service_name=\"bedrock-runtime\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "c0514e54-36cd-4faf-b824-4cce34177bbc",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index import LangchainEmbedding\n",
    "from langchain.llms.bedrock import Bedrock \n",
    "from langchain.embeddings.bedrock import BedrockEmbeddings\n",
    "\n",
    "model_kwargs_claude = {\n",
    "    \"temperature\":0,\n",
    "    \"top_k\":10, \n",
    "    \"max_tokens_to_sample\":512\n",
    "}\n",
    "\n",
    "llm = Bedrock(model_id=\"anthropic.claude-v2\",\n",
    "              model_kwargs=model_kwargs_claude,\n",
    "              client=bedrock_runtime)\n",
    "\n",
    "embed_model = LangchainEmbedding(\n",
    "    BedrockEmbeddings(model_id=\"amazon.titan-embed-text-v1\",\n",
    "                      client=bedrock_runtime)\n",
    ")\n",
    "\n",
    "service_context = ServiceContext.from_defaults(llm=llm, \n",
    "                                               embed_model=embed_model, \n",
    "                                               chunk_size=512)\n",
    "set_global_service_context(service_context)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23a8a8a2-0e95-4c56-bf0a-c558009d38d9",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Load Datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "3491600b-32fa-4559-8d25-c7d07a57b84b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!mkdir -p ./data\n",
    "\n",
    "from urllib.request import urlretrieve\n",
    "urls = [\n",
    "    'https://s2.q4cdn.com/299287126/files/doc_financials/2023/ar/2022-Shareholder-Letter.pdf',\n",
    "    'https://s2.q4cdn.com/299287126/files/doc_financials/2022/ar/2021-Shareholder-Letter.pdf',\n",
    "    'https://s2.q4cdn.com/299287126/files/doc_financials/2021/ar/Amazon-2020-Shareholder-Letter-and-1997-Shareholder-Letter.pdf',\n",
    "    'https://s2.q4cdn.com/299287126/files/doc_financials/2020/ar/2019-Shareholder-Letter.pdf'\n",
    "]\n",
    "\n",
    "filenames = [\n",
    "    'AMZN-2022-Shareholder-Letter.pdf',\n",
    "    'AMZN-2021-Shareholder-Letter.pdf',\n",
    "    'AMZN-2020-Shareholder-Letter.pdf',\n",
    "    'AMZN-2019-Shareholder-Letter.pdf'\n",
    "]\n",
    "\n",
    "metadata = [\n",
    "    dict(year=2022, source=filenames[0]),\n",
    "    dict(year=2021, source=filenames[1]),\n",
    "    dict(year=2020, source=filenames[2]),\n",
    "    dict(year=2019, source=filenames[3])]\n",
    "\n",
    "data_root = \"./data/\"\n",
    "\n",
    "for idx, url in enumerate(urls):\n",
    "    file_path = data_root + filenames[idx]\n",
    "    urlretrieve(url, file_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fa549fa-41ff-4712-91f1-c9c9cd8e03b6",
   "metadata": {},
   "source": [
    "As part of Amazon's culture, the CEO always includes a copy of the 1997 Letter to Shareholders with every new release. This will cause repetition, take longer to generate embeddings, and may skew your results. In the next section you will take the downloaded data, trim the 1997 letter (last 3 pages) and overwrite them as processed files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "91a0b300-0c6c-43be-a2ea-ae16fa0912f5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import glob\n",
    "from pypdf import PdfReader, PdfWriter\n",
    "\n",
    "local_pdfs = glob.glob(data_root + '*.pdf')\n",
    "\n",
    "for local_pdf in local_pdfs:\n",
    "    pdf_reader = PdfReader(local_pdf)\n",
    "    pdf_writer = PdfWriter()\n",
    "    for pagenum in range(len(pdf_reader.pages)-3):\n",
    "        page = pdf_reader.pages[pagenum]\n",
    "        pdf_writer.add_page(page)\n",
    "\n",
    "    with open(local_pdf, 'wb') as new_file:\n",
    "        new_file.seek(0)\n",
    "        pdf_writer.write(new_file)\n",
    "        new_file.truncate()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee9eb1db-5209-493c-8d6b-25bc700fc96e",
   "metadata": {},
   "source": [
    "Now that you have clean PDFs to work with, you will enrich your documents with metadata, then use a process called \"chunking\" to break up a larger document into small pieces. These small pieces will allow you to generate embeddings without surpassing the input limit of the embedding model.\n",
    "\n",
    "In this example you will break the document into 1000 character chunks, with a 100 character overlap. This will allow your embeddings to maintain some of its context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4429b839-e244-42b8-9b24-5368c1bb0a21",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "docs = []\n",
    "for filename in filenames:\n",
    "    doc = SimpleDirectoryReader(input_files=[f\"data/{filename}\"]).load_data()\n",
    "    doc[0].doc_id = filename.replace(\".pdf\", \"\")\n",
    "    docs.extend(doc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9ca7948-128b-4a86-a32a-33dbaf07ca4d",
   "metadata": {},
   "source": [
    "### Build Document Summary Index\n",
    "\n",
    "We show two ways of building the index:\n",
    "- default mode of building the document summary index\n",
    "- customizing the summary query\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "d8555b6d-5289-4803-bced-6b2f85dea3da",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "index = VectorStoreIndex.from_documents(docs,\n",
    "    service_context=service_context)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "d9ba68d6-3a91-4b31-a8b9-b7cc6d33b5f8",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "nodes = service_context.node_parser.get_nodes_from_documents(docs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "a413a570-686c-4865-a424-7f69b822b640",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# initialize storage context (by default it's in-memory)\n",
    "storage_context = StorageContext.from_defaults()\n",
    "storage_context.docstore.add_documents(nodes)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0bb7247-7ecc-411a-b3db-4161ff36b975",
   "metadata": {},
   "source": [
    "## Retrieval"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "1f234932-6362-42a9-9d21-056b89aac40d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.prompts.base import Prompt\n",
    "from llama_index.prompts.prompt_type import PromptType\n",
    "\n",
    "CLAUDE_CHOICE_SELECT_PROMPT_TMPL = (\n",
    "    \"\"\"\n",
    "    \n",
    "    Human: A list of documents is shown below. Each document has a number next to it along  with a summary of the document. A question is also provided. \n",
    "    Respond with the respective document number. You should consult to answer the question, in order of relevance, as well as the relevance score. \n",
    "    The relevance score is a number from 1-10 based on how relevant you think the document is to the question.\\n\"\n",
    "    Do not include any documents that are not relevant to the question. \n",
    "    \n",
    "    Example format: \n",
    "    Document 1:\\n<summary of document 1>\n",
    "    Document 2:\\n<summary of document 2>\n",
    "    ...\\n\\n\n",
    "    Document 10:\\n<summary of document 10>\n",
    "    \n",
    "    Question: <question>\n",
    "    Answer:\n",
    "    Doc: 9, Relevance: 7\n",
    "    Doc: 3, Relevance: 4\n",
    "    Doc: 7, Relevance: 3\n",
    "\n",
    "    Let's try this now: \n",
    "    {context_str}\n",
    "\n",
    "    Question: {query_str}\n",
    "    \n",
    "    Assistant: Answer:\"\"\"\n",
    ")\n",
    "\n",
    "claude_choice_select_prompt = Prompt(\n",
    "    CLAUDE_CHOICE_SELECT_PROMPT_TMPL, prompt_type=PromptType.CHOICE_SELECT\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "805847d9-a2c1-4930-98a9-98126e730000",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Retrieval"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "ace86835-832e-4964-a025-428891aa2c8b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from llama_index.retrievers import VectorIndexRetriever\n",
    "from llama_index.indices.query.schema import QueryBundle\n",
    "import pandas as pd\n",
    "from IPython.display import display, HTML\n",
    "\n",
    "pd.set_option(\"display.max_colwidth\", None)\n",
    "\n",
    "def get_retrieved_nodes(\n",
    "    query_str, vector_top_k=10, reranker_top_n=3, with_reranker=False\n",
    "):\n",
    "    query_bundle = QueryBundle(query_str)\n",
    "    # configure retriever\n",
    "    retriever = VectorIndexRetriever(\n",
    "        index=index,\n",
    "        similarity_top_k=vector_top_k,\n",
    "        choice_select_prompt=claude_choice_select_prompt\n",
    "\n",
    "    )\n",
    "    retrieved_nodes = retriever.retrieve(query_bundle)\n",
    "\n",
    "    if with_reranker:\n",
    "        # configure reranker\n",
    "        reranker = LLMRerank(\n",
    "            choice_batch_size=5, \n",
    "            top_n=reranker_top_n, \n",
    "            service_context=service_context, \n",
    "            choice_select_prompt=claude_choice_select_prompt\n",
    "\n",
    "        )\n",
    "        retrieved_nodes = reranker.postprocess_nodes(retrieved_nodes, query_bundle)\n",
    "\n",
    "    return retrieved_nodes\n",
    "\n",
    "\n",
    "def pretty_print(df):\n",
    "    return display(HTML(df.to_html().replace(\"\\\\n\", \"<br>\")))\n",
    "\n",
    "\n",
    "def visualize_retrieved_nodes(nodes) -> None:\n",
    "    result_dicts = []\n",
    "    for node in nodes:\n",
    "        result_dict = {\"Score\": node.score, \"Text\": node.node.get_text()}\n",
    "        result_dicts.append(result_dict)\n",
    "\n",
    "    pretty_print(pd.DataFrame(result_dicts))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57492945-9ac6-44d9-9588-d6f97a460b0c",
   "metadata": {},
   "source": [
    "Now, we will showcase how to do a two-stage pass for retrieval. Use embedding-based retrieval with a high top-k value in order to maximize recall and get a large set of candidate items. Then, use LLM-based retrieval to dynamically select the nodes that are actually relevant to the query."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "20993200-b725-403e-8a79-e2571dab2ebc",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "retrieved_nodes1 = get_retrieved_nodes(\n",
    "    \"How has AWS evolved?\", \n",
    "    vector_top_k=3, \n",
    "    with_reranker=False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "c65220b0-ce3d-4bfb-a409-35d7cc035a61",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(retrieved_nodes1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "9dd3bc25-6ef1-46b7-869e-247c83af9d4d",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6576742602806154\n",
      "This type of iterative innovation is never finished and has periodic peaks in investment years, but leadsto better long-term customer experiences, customer loyalty, and returns for our shareholders.\n",
      "AWS : As we were defining AWS and working backwards on the services we thought customers wanted, we\n",
      "kept triggering one of the biggest tensions in product development—where to draw the line on functionality inV1. One early meeting in particular—for our core compute service called Elastic Compute Cloud (“EC2”)—was scheduled for an hour, and took three, as we animatedly debated whether we could launch a computeservice without an accompanying persistent block storage companion (a form of network attached storage).\n",
      "-----------------------------------------------------------------------------------------------------------------------------------\n",
      "0.6324904710991853\n",
      "Think about howmuch of an impact 40% improvement on compute is. Compute is used for every bit of technology. That’s ahuge deal for customers. And, while Graviton2 has been a significant success thus far (48 of the top 50 AWSEC2 customers have already adopted it), the AWS Chips team was already learning from what customerssaid could be better, and announced Graviton3 this past December (offering a 25% improvement on top ofGraviton2’s relative gains). The list of what we’ve invented and delivered for customers in EC2 (and AWS ingeneral) is pretty mind-boggling, and this iterative approach to innovation has not only given customersmuch more functionality in AWS than they can find anywhere else (which is a significant differentiator), butalso allowed us to arrive at the much more game-changing offering that AWS is today.\n",
      "Devices : Our first foray into devices was the Kindle, released in 2007. It was not the most sophisticated\n",
      "industrial design (it was creamy white in color and the corners were uncomfortable for some people to hold),but revolutionary because it offered customers the ability to download any of over 90,000 books (nowmillions) in 60 seconds—and we got better and faster at building attractive designs. Shortly thereafter, welaunched a tablet, and then a phone (with the distinguishing feature of having front-facing cameras and agyroscope to give customers a dynamic perspective along with varied 3D experiences). The phone wasunsuccessful, and though we determined we were probably too late to this party and directed these resourceselsewhere, we hired some fantastic long-term builders and learned valuable lessons from this failure thathave served us well in devices like Echo and FireTV .\n",
      "When I think of the first Echo device—and what Alexa could do for customers at that point—it was\n",
      "noteworthy, yet so much less capable than what’s possible today.\n",
      "-----------------------------------------------------------------------------------------------------------------------------------\n",
      "0.6098452374935144\n",
      "Everybody agreed that having a persistent block store was important to a complete compute service;\n",
      "however, to have one ready would take an extra year. The question became could we offer customers auseful service where they could get meaningful value before we had all the features we thought they wanted?We decided that the initial launch of EC2 could be feature-poor if we also organized ourselves to listen tocustomers and iterate quickly. This approach works well if you indeed iterate quickly; but, is disastrous if youcan’t. We launched EC2 in 2006 with one instance size, in one data center, in one region of the world, withLinux operating system instances only (no Windows), without monitoring, load balancing, auto-scaling, oryes, persistent storage. EC2 was an initial success, but nowhere near the multi-billion-dollar service it’sbecome until we added the missing capabilities listed above, and then some.\n",
      "In the early days of AWS, people sometimes asked us why compute wouldn’t just be an undifferentiated\n",
      "commodity. But, there’s a lot more to compute than just a server. Customers want various flavors of compute(e.g. server configurations optimized for storage, memory, high-performance compute, graphics rendering,machine learning), multiple form factors (e.g. fixed instance sizes, portable containers, serverless functions),various sizes and optimizations of persistent storage, and a slew of networking capabilities. Then, there’sthe CPU chip that runs in your compute. For many years, the industry had used Intel or AMD x86 processors.We have important partnerships with these companies, but realized that if we wanted to push price andperformance further (as customers requested), we’d have to develop our own chips, too. Our first generalizedchip was Graviton, which we announced in 2018. This helped a subset of customer workloads run morecost-effectively than prior options. But, it wasn’t until 2020, after taking the learnings from Graviton and\n",
      "innovating on a new chip, that we had something remarkable with our Graviton2 chip, which provides up to40% better price-performance than the comparable latest generation x86 processors.\n",
      "-----------------------------------------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "for i, node in enumerate(retrieved_nodes1):\n",
    "    print(node.score)\n",
    "    print(node.node.get_text())\n",
    "    print(\"-----------------------------------------------------------------------------------------------------------------------------------\")\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "0ad7d62b-dba9-45d5-896f-4baa9a40edba",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "retrieved_nodes1_withreranker = get_retrieved_nodes(\n",
    "    \"How has AWS evolved?\",\n",
    "    vector_top_k=1,\n",
    "    reranker_top_n=1,\n",
    "    with_reranker=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "8d031c16-16ac-425e-ad56-3b9ef0dd0fea",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(retrieved_nodes1_withreranker)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "ae22d297-dd3b-4a71-a890-faceafab4e1a",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10.0\n",
      "This type of iterative innovation is never finished and has periodic peaks in investment years, but leadsto better long-term customer experiences, customer loyalty, and returns for our shareholders.\n",
      "AWS : As we were defining AWS and working backwards on the services we thought customers wanted, we\n",
      "kept triggering one of the biggest tensions in product development—where to draw the line on functionality inV1. One early meeting in particular—for our core compute service called Elastic Compute Cloud (“EC2”)—was scheduled for an hour, and took three, as we animatedly debated whether we could launch a computeservice without an accompanying persistent block storage companion (a form of network attached storage).\n",
      "-----------------------------------------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "for i, node in enumerate(retrieved_nodes1_withreranker):\n",
    "    print(node.score)\n",
    "    print(node.node.get_text())\n",
    "    print(\"-----------------------------------------------------------------------------------------------------------------------------------\")\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "7033d487-3b31-4c95-9a63-afec75e3bc37",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "retrieved_nodes2 = get_retrieved_nodes(\n",
    "    \"Why is Amazon successful?\", vector_top_k=3, with_reranker=False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "444a3658-740c-4efa-b73d-d102528ddf8a",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(retrieved_nodes2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "104633ac-d7ee-496d-8c4b-44aa462098f5",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.622700345059641\n",
      "through the pandemic the same way without the dedication and extraordinary efforts shown by our teams\n",
      "during this period, and I’m eternally grateful.\n",
      "It’s not normal for a company of any size to be able to respond to something as discontinuous and\n",
      "unpredictable as this pandemic turned out to be. What is it about Amazon that made it possible for us to doso? It’s because we weren’t starting from a standing start. We had been iterating on and remaking ourfulfillment capabilities for nearly two decades. In every business we pursue, we’re constantly experimentingand inventing. We’re divinely discontented with customer experiences, whether they’re our own or not. Webelieve these customer experiences can always be better, and we strive to make customers’ lives better andeasier every day. The beauty of this mission is that you never run out of runway; customers always want better,and our job is both to listen to their feedback and to imagine what else is possible and invent on theirbehalf.\n",
      "People often assume that the game-changing inventions they admire just pop out of somebody’s head, a\n",
      "light bulb goes off, a team executes to that idea, and presto—you have a new invention that’s a breakawaysuccess for a long time. That’s rarely, if ever, how it happens. One of the lesser known facts about innovativecompanies like Amazon is that they are relentlessly debating, re-defining, tinkering, iterating, andexperimenting to take the seed of a big idea and make it into something that resonates with customers andmeaningfully changes their customer experience over a long period of time.\n",
      "Let me give you some Amazon examples.Our Fulfillment Network : Going back to the pandemic, there’s no way we could have started working on\n",
      "our fulfillment network in March 2020 and satisfied anything close to what our customers needed. We’d beeninnovating in our fulfillment network for 20 years, constantly trying to shorten the time to get items to\n",
      "customers. In the early 2000s, it took us an average of 18 hours to get an item through our fulfillment centersand on the right truck for shipment.\n",
      "-----------------------------------------------------------------------------------------------------------------------------------\n",
      "0.5915859220789728\n",
      "AWS benefits from multi-tenant usage patterns and operates at far higher server utilization rates. In addition, AWS has been successful inincreasing the energy efficiency of its facilities and equipment, for instance by using more efficient evaporativecooling in certain data centers instead of traditional air conditioning. A study by 451 Research found that AWS’sinfrastructure is 3.6 times more energy efficient than the median U.S. enterprise data center surveyed. Along withour use of renewable energy, these factors enable AWS to do the same tasks as traditional data centers with an88% lower carbon footprint. And don’t think we’re not going to get those last 12 points—we’ll make AWS 100%carbon free through more investments in renewable energy projects.\n",
      "Leveraging scale for good\n",
      "Over the last decade, no company has created more jobs than Amazon. Amazon directly employs 840,000\n",
      "workers worldwide, including over 590,000 in the U.S., 115,000 in Europe, and 95,000 in Asia. In total, Amazondirectly and indirectly supports 2 million jobs in the U.S., including 680,000-plus jobs created by Amazon’sinvestments in areas like construction, logistics, and professional services, plus another 830,000 jobs created bysmall and medium-sized businesses selling on Amazon. Globally, we support nearly 4 million jobs. We areespecially proud of the fact that many of these are entry-level jobs that give people their first opportunity toparticipate in the workforce.\n",
      "And Amazon’s jobs come with an industry-leading $15 minimum wage and comprehensive benefits. More than\n",
      "40 million Americans—many making the federal minimum wage of $7.25 an hour—earn less than the lowest-paid Amazon associate. When we raised our starting minimum wage to $15 an hour in 2018, it had an immediateand meaningful impact on the hundreds of thousands of people working in our fulfillment centers. We want otherbig employers to join us by raising their own minimum pay rates, and we continue to lobby for a $15 federalminimum wage.\n",
      "-----------------------------------------------------------------------------------------------------------------------------------\n",
      "0.5791360704486281\n",
      "Now, it takes us two. To deliver as reliably and cost-effectively as wedesire, and to serve Amazon Prime members expecting shipments in a couple of days, we spent years buildingout an expansive set of fulfillment centers, a substantial logistics and transportation capability, andreconfigured how we did virtually everything in our facilities. For perspective, in 2004, we had sevenfulfillment centers in the U.S. and four in other parts of the world, and we hadn’t yet added delivery stations,which connect our fulfillment and sortation centers to the last-mile delivery vans you see driving aroundyour neighborhood. Fast forward to the end of 2021, we had 253 fulfillment centers, 110 sortation centers,and 467 delivery stations in North America, with an additional 157 fulfillment centers, 58 sortation centers,and 588 delivery stations across the globe. Our delivery network grew to more than 260,000 driversworldwide, and our Amazon Air cargo fleet has more than 100 aircraft. This has represented a capitalinvestment of over $100 billion and countless iterations and small process improvements by over a millionAmazonians in the last decade and a half.\n",
      "Ironically, just before COVID started, we’d made the decision to invest billions of incremental dollars over\n",
      "several years to deliver an increasing number of Prime shipments in one day. This initiative was slowed by thechallenges of the pandemic, but we’ve since resumed our focus here. Delivering a substantial amount ofshipments in one day is hard (especially across the millions of items that we offer) and initially expensive aswe build out the infrastructure to scale this efficiently. But, we believe our over 200 million Prime customers,who will tell you very clearly that faster is almost always better, will love this. So, this capability to shipmillions of items within a couple days (and increasingly one day) was not from one aha moment and notdeveloped in a year or two. It’s been hard-earned by putting ourselves in the shoes of our customers, knowingwhat they wanted, organizing Amazonians to work together to invent better solutions, and investing alarge amount of financial and people resources over 20 years (often well in advance of when it would payout).\n",
      "-----------------------------------------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "for i, node in enumerate(retrieved_nodes2):\n",
    "    print(node.score)\n",
    "    print(node.node.get_text())\n",
    "    print(\"-----------------------------------------------------------------------------------------------------------------------------------\")\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "ab0acde8-cb0a-4fa8-b30f-4fc965d8e7f3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "retrieved_nodes2_withreranker = get_retrieved_nodes(\n",
    "    \"Why is Amazon successful?\",\n",
    "    vector_top_k=3,\n",
    "    reranker_top_n=2,\n",
    "    with_reranker=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "e8ba718e-4ae9-4e15-8739-4f68f2d046b0",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(retrieved_nodes2_withreranker)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "04678820-d492-4c4a-b246-04a10e7d6b6c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.0\n",
      "Now, it takes us two. To deliver as reliably and cost-effectively as wedesire, and to serve Amazon Prime members expecting shipments in a couple of days, we spent years buildingout an expansive set of fulfillment centers, a substantial logistics and transportation capability, andreconfigured how we did virtually everything in our facilities. For perspective, in 2004, we had sevenfulfillment centers in the U.S. and four in other parts of the world, and we hadn’t yet added delivery stations,which connect our fulfillment and sortation centers to the last-mile delivery vans you see driving aroundyour neighborhood. Fast forward to the end of 2021, we had 253 fulfillment centers, 110 sortation centers,and 467 delivery stations in North America, with an additional 157 fulfillment centers, 58 sortation centers,and 588 delivery stations across the globe. Our delivery network grew to more than 260,000 driversworldwide, and our Amazon Air cargo fleet has more than 100 aircraft. This has represented a capitalinvestment of over $100 billion and countless iterations and small process improvements by over a millionAmazonians in the last decade and a half.\n",
      "Ironically, just before COVID started, we’d made the decision to invest billions of incremental dollars over\n",
      "several years to deliver an increasing number of Prime shipments in one day. This initiative was slowed by thechallenges of the pandemic, but we’ve since resumed our focus here. Delivering a substantial amount ofshipments in one day is hard (especially across the millions of items that we offer) and initially expensive aswe build out the infrastructure to scale this efficiently. But, we believe our over 200 million Prime customers,who will tell you very clearly that faster is almost always better, will love this. So, this capability to shipmillions of items within a couple days (and increasingly one day) was not from one aha moment and notdeveloped in a year or two. It’s been hard-earned by putting ourselves in the shoes of our customers, knowingwhat they wanted, organizing Amazonians to work together to invent better solutions, and investing alarge amount of financial and people resources over 20 years (often well in advance of when it would payout).\n",
      "-----------------------------------------------------------------------------------------------------------------------------------\n",
      "8.0\n",
      "through the pandemic the same way without the dedication and extraordinary efforts shown by our teams\n",
      "during this period, and I’m eternally grateful.\n",
      "It’s not normal for a company of any size to be able to respond to something as discontinuous and\n",
      "unpredictable as this pandemic turned out to be. What is it about Amazon that made it possible for us to doso? It’s because we weren’t starting from a standing start. We had been iterating on and remaking ourfulfillment capabilities for nearly two decades. In every business we pursue, we’re constantly experimentingand inventing. We’re divinely discontented with customer experiences, whether they’re our own or not. Webelieve these customer experiences can always be better, and we strive to make customers’ lives better andeasier every day. The beauty of this mission is that you never run out of runway; customers always want better,and our job is both to listen to their feedback and to imagine what else is possible and invent on theirbehalf.\n",
      "People often assume that the game-changing inventions they admire just pop out of somebody’s head, a\n",
      "light bulb goes off, a team executes to that idea, and presto—you have a new invention that’s a breakawaysuccess for a long time. That’s rarely, if ever, how it happens. One of the lesser known facts about innovativecompanies like Amazon is that they are relentlessly debating, re-defining, tinkering, iterating, andexperimenting to take the seed of a big idea and make it into something that resonates with customers andmeaningfully changes their customer experience over a long period of time.\n",
      "Let me give you some Amazon examples.Our Fulfillment Network : Going back to the pandemic, there’s no way we could have started working on\n",
      "our fulfillment network in March 2020 and satisfied anything close to what our customers needed. We’d beeninnovating in our fulfillment network for 20 years, constantly trying to shorten the time to get items to\n",
      "customers. In the early 2000s, it took us an average of 18 hours to get an item through our fulfillment centersand on the right truck for shipment.\n",
      "-----------------------------------------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "for i, node in enumerate(retrieved_nodes2_withreranker):\n",
    "    print(node.score)\n",
    "    print(node.node.get_text())\n",
    "    print(\"-----------------------------------------------------------------------------------------------------------------------------------\")\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "870ea771-7116-41e6-8032-89f4624cd6ac",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}