12_bedrock/29_finetune_cohere_summarization.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "308a0e04-6066-4405-8a13-8b060e552a8a",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Introduction to Bedrock - Fine-Tuning\n",
    "\n",
    "> *If you see errors, you may need to be allow-listed for the Bedrock models used by this notebook*\n",
    "\n",
    "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio*\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03f9a3e6-4d64-4da4-8a85-d5c12ae70978",
   "metadata": {},
   "source": [
    "In this demo notebook, we demonstrate how to use the Bedrock Python SDK for fine-tuning Bedrock models with your own data. If you have text samples to train and want to adapt the Bedrock models to your domain, you can further fine-tune the Bedrock foundation models by providing your own training datasets. You can upload your datasets to Amazon S3, and provide the S3 bucket path while configuring a Bedrock fine-tuning job. You can also adjust hyper parameters (learning rate, epoch, and batch size) for fine-tuning. After the fine-tuning job of the model with your dataset has completed, you can start using the model for inference in the Bedrock playground application. You can select the fine-tuned model and submit a prompt to the fine-tuned model along with a set of model parameters. The fine-tuned model should generate texts to be more alike your text samples. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca26768f-3775-436d-8b77-dee584420aff",
   "metadata": {},
   "source": [
    "-----------"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1a0821c-2d18-4698-8efc-df81030bcdc8",
   "metadata": {},
   "source": [
    "1. Setup\n",
    "2. Fine-tuning\n",
    "3. Testing the fine-tuned model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b889b27f-7d55-4c07-81fc-3f00d55e5544",
   "metadata": {},
   "source": [
    " Note: This notebook was tested in Amazon SageMaker Studio with Python 3 (Data Science 2.0) kernel."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb8bb89f-2b60-41e4-8ea7-8cace718bed5",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "335c5e24-b7c8-4932-b8d1-729ff52c6fb1",
   "metadata": {},
   "source": [
    "## 1. Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "fd9051ac-179b-4d7b-bcea-86beb17efad3",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com\n",
      "Collecting pandas==2.1.2\n",
      "  Obtaining dependency information for pandas==2.1.2 from https://files.pythonhosted.org/packages/02/52/815f643ed3afb3365354548b3c8b557dbf926a65c40ad5b6d9e455147c7e/pandas-2.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata\n",
      "  Downloading pandas-2.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)\n",
      "Collecting datasets==2.15.0\n",
      "  Obtaining dependency information for datasets==2.15.0 from https://files.pythonhosted.org/packages/e2/cf/db41e572d7ed958e8679018f8190438ef700aeb501b62da9e1eed9e4d69a/datasets-2.15.0-py3-none-any.whl.metadata\n",
      "  Downloading datasets-2.15.0-py3-none-any.whl.metadata (20 kB)\n",
      "Collecting numpy<2,>=1.22.4 (from pandas==2.1.2)\n",
      "  Obtaining dependency information for numpy<2,>=1.22.4 from https://files.pythonhosted.org/packages/64/41/284783f1014685201e447ea976e85fed0e351f5debbaf3ee6d7645521f1d/numpy-1.26.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata\n",
      "  Downloading numpy-1.26.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m61.2/61.2 kB\u001b[0m \u001b[31m296.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting python-dateutil>=2.8.2 (from pandas==2.1.2)\n",
      "  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m247.7/247.7 kB\u001b[0m \u001b[31m200.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting pytz>=2020.1 (from pandas==2.1.2)\n",
      "  Obtaining dependency information for pytz>=2020.1 from https://files.pythonhosted.org/packages/32/4d/aaf7eff5deb402fd9a24a1449a8119f00d74ae9c2efa79f8ef9994261fc2/pytz-2023.3.post1-py2.py3-none-any.whl.metadata\n",
      "  Downloading pytz-2023.3.post1-py2.py3-none-any.whl.metadata (22 kB)\n",
      "Collecting tzdata>=2022.1 (from pandas==2.1.2)\n",
      "  Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m341.8/341.8 kB\u001b[0m \u001b[31m412.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting pyarrow>=8.0.0 (from datasets==2.15.0)\n",
      "  Obtaining dependency information for pyarrow>=8.0.0 from https://files.pythonhosted.org/packages/34/65/204f7c0d507056c37b56dddb3bd60f55744f2609c0f96a5e4ca91c67c42a/pyarrow-14.0.1-cp310-cp310-manylinux_2_28_x86_64.whl.metadata\n",
      "  Downloading pyarrow-14.0.1-cp310-cp310-manylinux_2_28_x86_64.whl.metadata (3.0 kB)\n",
      "Collecting pyarrow-hotfix (from datasets==2.15.0)\n",
      "  Obtaining dependency information for pyarrow-hotfix from https://files.pythonhosted.org/packages/e4/f4/9ec2222f5f5f8ea04f66f184caafd991a39c8782e31f5b0266f101cb68ca/pyarrow_hotfix-0.6-py3-none-any.whl.metadata\n",
      "  Downloading pyarrow_hotfix-0.6-py3-none-any.whl.metadata (3.6 kB)\n",
      "Collecting dill<0.3.8,>=0.3.0 (from datasets==2.15.0)\n",
      "  Obtaining dependency information for dill<0.3.8,>=0.3.0 from https://files.pythonhosted.org/packages/f5/3a/74a29b11cf2cdfcd6ba89c0cecd70b37cd1ba7b77978ce611eb7a146a832/dill-0.3.7-py3-none-any.whl.metadata\n",
      "  Downloading dill-0.3.7-py3-none-any.whl.metadata (9.9 kB)\n",
      "Collecting requests>=2.19.0 (from datasets==2.15.0)\n",
      "  Obtaining dependency information for requests>=2.19.0 from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata\n",
      "  Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)\n",
      "Collecting tqdm>=4.62.1 (from datasets==2.15.0)\n",
      "  Obtaining dependency information for tqdm>=4.62.1 from https://files.pythonhosted.org/packages/00/e5/f12a80907d0884e6dff9c16d0c0114d81b8cd07dc3ae54c5e962cc83037e/tqdm-4.66.1-py3-none-any.whl.metadata\n",
      "  Downloading tqdm-4.66.1-py3-none-any.whl.metadata (57 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.6/57.6 kB\u001b[0m \u001b[31m288.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting xxhash (from datasets==2.15.0)\n",
      "  Obtaining dependency information for xxhash from https://files.pythonhosted.org/packages/80/8a/1dd41557883b6196f8f092011a5c1f72d4d44cf36d7b67d4a5efe3127949/xxhash-3.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata\n",
      "  Downloading xxhash-3.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (12 kB)\n",
      "Collecting multiprocess (from datasets==2.15.0)\n",
      "  Obtaining dependency information for multiprocess from https://files.pythonhosted.org/packages/35/a8/36d8d7b3e46b377800d8dec47891cdf05842d1a2366909ae4a0c89fbc5e6/multiprocess-0.70.15-py310-none-any.whl.metadata\n",
      "  Downloading multiprocess-0.70.15-py310-none-any.whl.metadata (7.2 kB)\n",
      "Collecting fsspec[http]<=2023.10.0,>=2023.1.0 (from datasets==2.15.0)\n",
      "  Obtaining dependency information for fsspec[http]<=2023.10.0,>=2023.1.0 from https://files.pythonhosted.org/packages/e8/f6/3eccfb530aac90ad1301c582da228e4763f19e719ac8200752a4841b0b2d/fsspec-2023.10.0-py3-none-any.whl.metadata\n",
      "  Downloading fsspec-2023.10.0-py3-none-any.whl.metadata (6.8 kB)\n",
      "Collecting aiohttp (from datasets==2.15.0)\n",
      "  Obtaining dependency information for aiohttp from https://files.pythonhosted.org/packages/b0/36/c7bd200871e7351ab8396e8edcbb91e1198e0ded67a0824c93110c4c5df2/aiohttp-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata\n",
      "  Downloading aiohttp-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.4 kB)\n",
      "Collecting huggingface-hub>=0.18.0 (from datasets==2.15.0)\n",
      "  Obtaining dependency information for huggingface-hub>=0.18.0 from https://files.pythonhosted.org/packages/05/09/1945ca6ba3ad8ad6e2872ba682ce8d68c5e63c8e55458ed8ab4885709f1d/huggingface_hub-0.19.4-py3-none-any.whl.metadata\n",
      "  Downloading huggingface_hub-0.19.4-py3-none-any.whl.metadata (14 kB)\n",
      "Collecting packaging (from datasets==2.15.0)\n",
      "  Obtaining dependency information for packaging from https://files.pythonhosted.org/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl.metadata\n",
      "  Downloading packaging-23.2-py3-none-any.whl.metadata (3.2 kB)\n",
      "Collecting pyyaml>=5.1 (from datasets==2.15.0)\n",
      "  Obtaining dependency information for pyyaml>=5.1 from https://files.pythonhosted.org/packages/29/61/bf33c6c85c55bc45a29eee3195848ff2d518d84735eb0e2d8cb42e0d285e/PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata\n",
      "  Downloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)\n",
      "Collecting attrs>=17.3.0 (from aiohttp->datasets==2.15.0)\n",
      "  Downloading attrs-23.1.0-py3-none-any.whl (61 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m61.2/61.2 kB\u001b[0m \u001b[31m313.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting multidict<7.0,>=4.5 (from aiohttp->datasets==2.15.0)\n",
      "  Downloading multidict-6.0.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m114.5/114.5 kB\u001b[0m \u001b[31m335.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting yarl<2.0,>=1.0 (from aiohttp->datasets==2.15.0)\n",
      "  Obtaining dependency information for yarl<2.0,>=1.0 from https://files.pythonhosted.org/packages/b6/b2/44b31699e27f82c577143d062a2b58cbe0c6e7a0828d13c0ffd10891ad40/yarl-1.9.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata\n",
      "  Downloading yarl-1.9.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (28 kB)\n",
      "Collecting frozenlist>=1.1.1 (from aiohttp->datasets==2.15.0)\n",
      "  Obtaining dependency information for frozenlist>=1.1.1 from https://files.pythonhosted.org/packages/1e/28/74b8b6451c89c070d34e753d8b65a1e4ce508a6808b18529f36e8c0e2184/frozenlist-1.4.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata\n",
      "  Downloading frozenlist-1.4.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.2 kB)\n",
      "Collecting aiosignal>=1.1.2 (from aiohttp->datasets==2.15.0)\n",
      "  Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)\n",
      "Collecting async-timeout<5.0,>=4.0 (from aiohttp->datasets==2.15.0)\n",
      "  Obtaining dependency information for async-timeout<5.0,>=4.0 from https://files.pythonhosted.org/packages/a7/fa/e01228c2938de91d47b307831c62ab9e4001e747789d0b05baf779a6488c/async_timeout-4.0.3-py3-none-any.whl.metadata\n",
      "  Downloading async_timeout-4.0.3-py3-none-any.whl.metadata (4.2 kB)\n",
      "Collecting filelock (from huggingface-hub>=0.18.0->datasets==2.15.0)\n",
      "  Obtaining dependency information for filelock from https://files.pythonhosted.org/packages/81/54/84d42a0bee35edba99dee7b59a8d4970eccdd44b99fe728ed912106fc781/filelock-3.13.1-py3-none-any.whl.metadata\n",
      "  Downloading filelock-3.13.1-py3-none-any.whl.metadata (2.8 kB)\n",
      "Collecting typing-extensions>=3.7.4.3 (from huggingface-hub>=0.18.0->datasets==2.15.0)\n",
      "  Obtaining dependency information for typing-extensions>=3.7.4.3 from https://files.pythonhosted.org/packages/24/21/7d397a4b7934ff4028987914ac1044d3b7d52712f30e2ac7a2ae5bc86dd0/typing_extensions-4.8.0-py3-none-any.whl.metadata\n",
      "  Downloading typing_extensions-4.8.0-py3-none-any.whl.metadata (3.0 kB)\n",
      "Collecting six>=1.5 (from python-dateutil>=2.8.2->pandas==2.1.2)\n",
      "  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)\n",
      "Collecting charset-normalizer<4,>=2 (from requests>=2.19.0->datasets==2.15.0)\n",
      "  Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/da/f1/3702ba2a7470666a62fd81c58a4c40be00670e5006a67f4d626e57f013ae/charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata\n",
      "  Downloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)\n",
      "Collecting idna<4,>=2.5 (from requests>=2.19.0->datasets==2.15.0)\n",
      "  Obtaining dependency information for idna<4,>=2.5 from https://files.pythonhosted.org/packages/ea/65/9c7a31be86861d43da3d4f8661f677b38120320540773a04979ad6fa9ecd/idna-3.5-py3-none-any.whl.metadata\n",
      "  Downloading idna-3.5-py3-none-any.whl.metadata (9.9 kB)\n",
      "Collecting urllib3<3,>=1.21.1 (from requests>=2.19.0->datasets==2.15.0)\n",
      "  Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/96/94/c31f58c7a7f470d5665935262ebd7455c7e4c7782eb525658d3dbf4b9403/urllib3-2.1.0-py3-none-any.whl.metadata\n",
      "  Downloading urllib3-2.1.0-py3-none-any.whl.metadata (6.4 kB)\n",
      "Collecting certifi>=2017.4.17 (from requests>=2.19.0->datasets==2.15.0)\n",
      "  Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/64/62/428ef076be88fa93716b576e4a01f919d25968913e817077a386fcbe4f42/certifi-2023.11.17-py3-none-any.whl.metadata\n",
      "  Downloading certifi-2023.11.17-py3-none-any.whl.metadata (2.2 kB)\n",
      "Downloading pandas-2.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.3 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m12.3/12.3 MB\u001b[0m \u001b[31m229.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading datasets-2.15.0-py3-none-any.whl (521 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m521.2/521.2 kB\u001b[0m \u001b[31m443.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading dill-0.3.7-py3-none-any.whl (115 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m115.3/115.3 kB\u001b[0m \u001b[31m358.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading aiohttp-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m436.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading huggingface_hub-0.19.4-py3-none-any.whl (311 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m311.7/311.7 kB\u001b[0m \u001b[31m417.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading numpy-1.26.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m18.2/18.2 MB\u001b[0m \u001b[31m281.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading packaging-23.2-py3-none-any.whl (53 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.0/53.0 kB\u001b[0m \u001b[31m289.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading pyarrow-14.0.1-cp310-cp310-manylinux_2_28_x86_64.whl (38.0 MB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m38.0/38.0 MB\u001b[0m \u001b[31m357.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hDownloading pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m502.5/502.5 kB\u001b[0m \u001b[31m375.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m705.5/705.5 kB\u001b[0m \u001b[31m453.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading requests-2.31.0-py3-none-any.whl (62 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.6/62.6 kB\u001b[0m \u001b[31m300.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading tqdm-4.66.1-py3-none-any.whl (78 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m78.3/78.3 kB\u001b[0m \u001b[31m315.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading multiprocess-0.70.15-py310-none-any.whl (134 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m357.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading pyarrow_hotfix-0.6-py3-none-any.whl (7.9 kB)\n",
      "Downloading xxhash-3.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (194 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.1/194.1 kB\u001b[0m \u001b[31m398.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading async_timeout-4.0.3-py3-none-any.whl (5.7 kB)\n",
      "Downloading certifi-2023.11.17-py3-none-any.whl (162 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m162.5/162.5 kB\u001b[0m \u001b[31m390.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m142.1/142.1 kB\u001b[0m \u001b[31m349.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading frozenlist-1.4.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (225 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m225.7/225.7 kB\u001b[0m \u001b[31m386.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading fsspec-2023.10.0-py3-none-any.whl (166 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m166.4/166.4 kB\u001b[0m \u001b[31m375.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading idna-3.5-py3-none-any.whl (61 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m61.6/61.6 kB\u001b[0m \u001b[31m300.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading typing_extensions-4.8.0-py3-none-any.whl (31 kB)\n",
      "Downloading urllib3-2.1.0-py3-none-any.whl (104 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m104.6/104.6 kB\u001b[0m \u001b[31m357.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading yarl-1.9.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (300 kB)\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m300.7/300.7 kB\u001b[0m \u001b[31m424.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hDownloading filelock-3.13.1-py3-none-any.whl (11 kB)\n",
      "Installing collected packages: pytz, xxhash, urllib3, tzdata, typing-extensions, tqdm, six, pyyaml, pyarrow-hotfix, packaging, numpy, multidict, idna, fsspec, frozenlist, filelock, dill, charset-normalizer, certifi, attrs, async-timeout, yarl, requests, python-dateutil, pyarrow, multiprocess, aiosignal, pandas, huggingface-hub, aiohttp, datasets\n",
      "  Attempting uninstall: pytz\n",
      "    Found existing installation: pytz 2023.3.post1\n",
      "    Uninstalling pytz-2023.3.post1:\n",
      "      Successfully uninstalled pytz-2023.3.post1\n",
      "  Attempting uninstall: xxhash\n",
      "    Found existing installation: xxhash 3.4.1\n",
      "    Uninstalling xxhash-3.4.1:\n",
      "      Successfully uninstalled xxhash-3.4.1\n",
      "  Attempting uninstall: urllib3\n",
      "    Found existing installation: urllib3 2.1.0\n",
      "    Uninstalling urllib3-2.1.0:\n",
      "      Successfully uninstalled urllib3-2.1.0\n",
      "  Attempting uninstall: tzdata\n",
      "    Found existing installation: tzdata 2023.3\n",
      "    Uninstalling tzdata-2023.3:\n",
      "      Successfully uninstalled tzdata-2023.3\n",
      "  Attempting uninstall: typing-extensions\n",
      "    Found existing installation: typing_extensions 4.8.0\n",
      "    Uninstalling typing_extensions-4.8.0:\n",
      "      Successfully uninstalled typing_extensions-4.8.0\n",
      "  Attempting uninstall: tqdm\n",
      "    Found existing installation: tqdm 4.66.1\n",
      "    Uninstalling tqdm-4.66.1:\n",
      "      Successfully uninstalled tqdm-4.66.1\n",
      "  Attempting uninstall: six\n",
      "    Found existing installation: six 1.16.0\n",
      "    Uninstalling six-1.16.0:\n",
      "      Successfully uninstalled six-1.16.0\n",
      "  Attempting uninstall: pyyaml\n",
      "    Found existing installation: PyYAML 6.0.1\n",
      "    Uninstalling PyYAML-6.0.1:\n",
      "      Successfully uninstalled PyYAML-6.0.1\n",
      "  Attempting uninstall: pyarrow-hotfix\n",
      "    Found existing installation: pyarrow-hotfix 0.6\n",
      "    Uninstalling pyarrow-hotfix-0.6:\n",
      "      Successfully uninstalled pyarrow-hotfix-0.6\n",
      "  Attempting uninstall: packaging\n",
      "    Found existing installation: packaging 23.2\n",
      "    Uninstalling packaging-23.2:\n",
      "      Successfully uninstalled packaging-23.2\n",
      "  Attempting uninstall: numpy\n",
      "    Found existing installation: numpy 1.26.2\n",
      "    Uninstalling numpy-1.26.2:\n",
      "      Successfully uninstalled numpy-1.26.2\n",
      "  Attempting uninstall: multidict\n",
      "    Found existing installation: multidict 6.0.4\n",
      "    Uninstalling multidict-6.0.4:\n",
      "      Successfully uninstalled multidict-6.0.4\n",
      "  Attempting uninstall: idna\n",
      "    Found existing installation: idna 3.5\n",
      "    Uninstalling idna-3.5:\n",
      "      Successfully uninstalled idna-3.5\n",
      "  Attempting uninstall: fsspec\n",
      "    Found existing installation: fsspec 2023.10.0\n",
      "    Uninstalling fsspec-2023.10.0:\n",
      "      Successfully uninstalled fsspec-2023.10.0\n",
      "  Attempting uninstall: frozenlist\n",
      "    Found existing installation: frozenlist 1.4.0\n",
      "    Uninstalling frozenlist-1.4.0:\n",
      "      Successfully uninstalled frozenlist-1.4.0\n",
      "  Attempting uninstall: filelock\n",
      "    Found existing installation: filelock 3.13.1\n",
      "    Uninstalling filelock-3.13.1:\n",
      "      Successfully uninstalled filelock-3.13.1\n",
      "  Attempting uninstall: dill\n",
      "    Found existing installation: dill 0.3.7\n",
      "    Uninstalling dill-0.3.7:\n",
      "      Successfully uninstalled dill-0.3.7\n",
      "  Attempting uninstall: charset-normalizer\n",
      "    Found existing installation: charset-normalizer 3.3.2\n",
      "    Uninstalling charset-normalizer-3.3.2:\n",
      "      Successfully uninstalled charset-normalizer-3.3.2\n",
      "  Attempting uninstall: certifi\n",
      "    Found existing installation: certifi 2023.11.17\n",
      "    Uninstalling certifi-2023.11.17:\n",
      "      Successfully uninstalled certifi-2023.11.17\n",
      "  Attempting uninstall: attrs\n",
      "    Found existing installation: attrs 23.1.0\n",
      "    Uninstalling attrs-23.1.0:\n",
      "      Successfully uninstalled attrs-23.1.0\n",
      "  Attempting uninstall: async-timeout\n",
      "    Found existing installation: async-timeout 4.0.3\n",
      "    Uninstalling async-timeout-4.0.3:\n",
      "      Successfully uninstalled async-timeout-4.0.3\n",
      "  Attempting uninstall: yarl\n",
      "    Found existing installation: yarl 1.9.3\n",
      "    Uninstalling yarl-1.9.3:\n",
      "      Successfully uninstalled yarl-1.9.3\n",
      "  Attempting uninstall: requests\n",
      "    Found existing installation: requests 2.31.0\n",
      "    Uninstalling requests-2.31.0:\n",
      "      Successfully uninstalled requests-2.31.0\n",
      "  Attempting uninstall: python-dateutil\n",
      "    Found existing installation: python-dateutil 2.8.2\n",
      "    Uninstalling python-dateutil-2.8.2:\n",
      "      Successfully uninstalled python-dateutil-2.8.2\n",
      "  Attempting uninstall: pyarrow\n",
      "    Found existing installation: pyarrow 14.0.1\n",
      "    Uninstalling pyarrow-14.0.1:\n",
      "      Successfully uninstalled pyarrow-14.0.1\n",
      "  Attempting uninstall: multiprocess\n",
      "    Found existing installation: multiprocess 0.70.15\n",
      "    Uninstalling multiprocess-0.70.15:\n",
      "      Successfully uninstalled multiprocess-0.70.15\n",
      "  Attempting uninstall: aiosignal\n",
      "    Found existing installation: aiosignal 1.3.1\n",
      "    Uninstalling aiosignal-1.3.1:\n",
      "      Successfully uninstalled aiosignal-1.3.1\n",
      "  Attempting uninstall: pandas\n",
      "    Found existing installation: pandas 2.1.2\n",
      "    Uninstalling pandas-2.1.2:\n",
      "      Successfully uninstalled pandas-2.1.2\n",
      "  Attempting uninstall: huggingface-hub\n",
      "    Found existing installation: huggingface-hub 0.19.4\n",
      "    Uninstalling huggingface-hub-0.19.4:\n",
      "      Successfully uninstalled huggingface-hub-0.19.4\n",
      "  Attempting uninstall: aiohttp\n",
      "    Found existing installation: aiohttp 3.9.0\n",
      "    Uninstalling aiohttp-3.9.0:\n",
      "      Successfully uninstalled aiohttp-3.9.0\n",
      "  Attempting uninstall: datasets\n",
      "    Found existing installation: datasets 2.15.0\n",
      "    Uninstalling datasets-2.15.0:\n",
      "      Successfully uninstalled datasets-2.15.0\n",
      "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
      "daal4py 2021.6.0 requires daal==2021.4.0, which is not installed.\n",
      "spyder 5.3.3 requires pyqt5<5.16, which is not installed.\n",
      "spyder 5.3.3 requires pyqtwebengine<5.16, which is not installed.\n",
      "autovizwidget 0.21.0 requires pandas<2.0.0,>=0.20.1, but you have pandas 2.1.2 which is incompatible.\n",
      "botocore 1.31.63 requires urllib3<2.1,>=1.25.4; python_version >= \"3.10\", but you have urllib3 2.1.0 which is incompatible.\n",
      "distributed 2022.7.0 requires tornado<6.2,>=6.0.3, but you have tornado 6.3.3 which is incompatible.\n",
      "hdijupyterutils 0.21.0 requires pandas<2.0.0,>=0.17.1, but you have pandas 2.1.2 which is incompatible.\n",
      "jupyterlab 3.4.4 requires jupyter-server~=1.16, but you have jupyter-server 2.7.3 which is incompatible.\n",
      "jupyterlab-server 2.10.3 requires jupyter-server~=1.4, but you have jupyter-server 2.7.3 which is incompatible.\n",
      "notebook 6.5.6 requires jupyter-client<8,>=5.3.4, but you have jupyter-client 8.4.0 which is incompatible.\n",
      "notebook 6.5.6 requires pyzmq<25,>=17, but you have pyzmq 25.1.1 which is incompatible.\n",
      "numba 0.55.1 requires numpy<1.22,>=1.18, but you have numpy 1.26.2 which is incompatible.\n",
      "panel 0.13.1 requires bokeh<2.5.0,>=2.4.0, but you have bokeh 3.3.0 which is incompatible.\n",
      "sagemaker-datawrangler 0.4.3 requires sagemaker-data-insights==0.4.0, but you have sagemaker-data-insights 0.3.3 which is incompatible.\n",
      "spyder 5.3.3 requires ipython<8.0.0,>=7.31.1, but you have ipython 8.16.1 which is incompatible.\n",
      "spyder 5.3.3 requires pylint<3.0,>=2.5.0, but you have pylint 3.0.1 which is incompatible.\n",
      "spyder-kernels 2.3.3 requires ipython<8,>=7.31.1; python_version >= \"3\", but you have ipython 8.16.1 which is incompatible.\n",
      "spyder-kernels 2.3.3 requires jupyter-client<8,>=7.3.4; python_version >= \"3\", but you have jupyter-client 8.4.0 which is incompatible.\u001b[0m\u001b[31m\n",
      "\u001b[0mSuccessfully installed aiohttp-3.9.0 aiosignal-1.3.1 async-timeout-4.0.3 attrs-23.1.0 certifi-2023.11.17 charset-normalizer-3.3.2 datasets-2.15.0 dill-0.3.7 filelock-3.13.1 frozenlist-1.4.0 fsspec-2023.10.0 huggingface-hub-0.19.4 idna-3.5 multidict-6.0.4 multiprocess-0.70.15 numpy-1.26.2 packaging-23.2 pandas-2.1.2 pyarrow-14.0.1 pyarrow-hotfix-0.6 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 requests-2.31.0 six-1.16.0 tqdm-4.66.1 typing-extensions-4.8.0 tzdata-2023.3 urllib3-2.1.0 xxhash-3.4.1 yarl-1.9.3\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\n",
      "\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install -U --force-reinstall pandas==2.1.2 datasets==2.15.0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d966521-f63a-4c96-8494-68d6d2367938",
   "metadata": {},
   "source": [
    "#### Now let's set up our connection to the Amazon Bedrock SDK using Boto3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "48703d82-bb32-44fa-916e-6cc5040a67cb",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sagemaker.config INFO - Not applying SDK defaults from location: /etc/xdg/sagemaker/config.yaml\n",
      "sagemaker.config INFO - Not applying SDK defaults from location: /root/.config/sagemaker/config.yaml\n",
      "sagemaker.config INFO - Not applying SDK defaults from location: /etc/xdg/sagemaker/config.yaml\n",
      "sagemaker.config INFO - Not applying SDK defaults from location: /root/.config/sagemaker/config.yaml\n",
      "sagemaker.config INFO - Not applying SDK defaults from location: /etc/xdg/sagemaker/config.yaml\n",
      "sagemaker.config INFO - Not applying SDK defaults from location: /root/.config/sagemaker/config.yaml\n"
     ]
    }
   ],
   "source": [
    "import sagemaker\n",
    "\n",
    "sess = sagemaker.Session()\n",
    "sagemaker_session_bucket = sess.default_bucket()\n",
    "role = sagemaker.get_execution_role()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "91ad566f-2a15-46ac-aeeb-346f9db0a10e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "import json \n",
    "\n",
    "bedrock = boto3.client(service_name=\"bedrock\")\n",
    "bedrock_runtime = boto3.client(service_name=\"bedrock-runtime\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a24ced70-f97f-4cce-84f8-fff24017f644",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "modelArn : arn:aws:bedrock:us-east-1::foundation-model/cohere.command-text-v14:7:4k\n",
      "modelId : cohere.command-text-v14:7:4k\n",
      "modelName : Command\n",
      "providerName : Cohere\n",
      "inputModalities : ['TEXT']\n",
      "outputModalities : ['TEXT']\n",
      "responseStreamingSupported : True\n",
      "customizationsSupported : ['FINE_TUNING']\n",
      "inferenceTypesSupported : ['PROVISIONED']\n",
      "-----\n",
      "\n",
      "modelArn : arn:aws:bedrock:us-east-1::foundation-model/cohere.command-light-text-v14:7:4k\n",
      "modelId : cohere.command-light-text-v14:7:4k\n",
      "modelName : Command Light\n",
      "providerName : Cohere\n",
      "inputModalities : ['TEXT']\n",
      "outputModalities : ['TEXT']\n",
      "responseStreamingSupported : True\n",
      "customizationsSupported : ['FINE_TUNING']\n",
      "inferenceTypesSupported : ['PROVISIONED']\n",
      "-----\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for model in bedrock.list_foundation_models(byProvider=\"cohere\", byCustomizationType=\"FINE_TUNING\")[\"modelSummaries\"]:\n",
    "    for key, value in model.items():\n",
    "        print(key, \":\", value)\n",
    "    print(\"-----\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a8153b92-f383-4750-8e14-b21e5fd8d06e",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-----\n",
      "modelArn: arn:aws:bedrock:us-east-1::foundation-model/cohere.command-text-v14\n",
      "modelId: cohere.command-text-v14\n",
      "modelName: Command\n",
      "customizationsSupported: \n",
      "-----\n",
      "modelArn: arn:aws:bedrock:us-east-1::foundation-model/cohere.command-light-text-v14\n",
      "modelId: cohere.command-light-text-v14\n",
      "modelName: Command Light\n",
      "customizationsSupported: \n",
      "-----\n",
      "modelArn: arn:aws:bedrock:us-east-1::foundation-model/cohere.embed-english-v3\n",
      "modelId: cohere.embed-english-v3\n",
      "modelName: Embed English\n",
      "customizationsSupported: \n",
      "-----\n",
      "modelArn: arn:aws:bedrock:us-east-1::foundation-model/cohere.embed-multilingual-v3\n",
      "modelId: cohere.embed-multilingual-v3\n",
      "modelName: Embed Multilingual\n",
      "customizationsSupported: \n"
     ]
    }
   ],
   "source": [
    "for model in bedrock.list_foundation_models(byProvider=\"cohere\")[\"modelSummaries\"]:\n",
    "    print(\"-----\\n\" + \"modelArn: \" + model[\"modelArn\"] + \"\\nmodelId: \" + model[\"modelId\"] + \"\\nmodelName: \" + model[\"modelName\"] + \"\\ncustomizationsSupported: \" + ','.join(model[\"customizationsSupported\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "367c1fa8-657c-4e45-94e8-e907857d145a",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Invoke Model before Fine-Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4f675dec-d75c-4ecc-8177-73d390366302",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "command_base_model_id = \"cohere.command-light-text-v14\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "ad1684e7-8b0f-436d-a8a0-2a2b45ffe387",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "base_model_id = \"cohere.command-light-text-v14:7:4k\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3dfa6614-7080-4dbb-af08-cd67ce6a8706",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Convert the dataset into jsonlines format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6f96f947-e885-48a1-8530-b70b8fefc073",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatasetDict({\n",
       "    train: Dataset({\n",
       "        features: ['id', 'dialogue', 'summary', 'topic'],\n",
       "        num_rows: 12460\n",
       "    })\n",
       "    validation: Dataset({\n",
       "        features: ['id', 'dialogue', 'summary', 'topic'],\n",
       "        num_rows: 500\n",
       "    })\n",
       "    test: Dataset({\n",
       "        features: ['id', 'dialogue', 'summary', 'topic'],\n",
       "        num_rows: 1500\n",
       "    })\n",
       "})"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from datasets import load_dataset\n",
    "dataset = load_dataset(\"knkarthick/dialogsum\")\n",
    "dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "8cef0311-b45f-481b-bdbe-a14d4baa1359",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def wrap_instruction_fn(example):\n",
    "    prompt = 'Summarize the simplest and most interesting part of the following conversation.\\n\\n'\n",
    "    end_prompt = '\\n\\nSummary: '\n",
    "    example[\"instruction\"] = prompt + example[\"dialogue\"] + end_prompt\n",
    "    return example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e94a8473-2226-4f0e-9890-03635534156b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1d92995111d8490eaee3d6ac351a1584",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Creating json from Arrow format:   0%|          | 0/1 [00:00<?, ?ba/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9f10d87a43f4434283ed534df50e0931",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Creating json from Arrow format:   0%|          | 0/1 [00:00<?, ?ba/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fc2c9c13fb1746d5bb1b8b961235b57b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Creating json from Arrow format:   0%|          | 0/2 [00:00<?, ?ba/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "1509839"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset['train']\\\n",
    "  .select(range(1000))\\\n",
    "  .select_columns(['dialogue', 'summary'])\\\n",
    "  .map(wrap_instruction_fn)\\\n",
    "  .remove_columns(['dialogue'])\\\n",
    "  .rename_column('instruction', 'prompt')\\\n",
    "  .rename_column('summary', 'completion')\\\n",
    "  .to_json('./train-summarization2.jsonl', index=False)\n",
    "\n",
    "dataset['validation']\\\n",
    "  .select_columns(['dialogue', 'summary'])\\\n",
    "  .map(wrap_instruction_fn)\\\n",
    "  .remove_columns(['dialogue'])\\\n",
    "  .rename_column('instruction', 'prompt')\\\n",
    "  .rename_column('summary', 'completion')\\\n",
    "  .to_json('./validation-summarization2.jsonl', index=False)\n",
    "\n",
    "dataset['test']\\\n",
    "  .select_columns(['dialogue', 'summary'])\\\n",
    "  .map(wrap_instruction_fn)\\\n",
    "  .remove_columns(['dialogue'])\\\n",
    "  .rename_column('instruction', 'prompt')\\\n",
    "  .rename_column('summary', 'completion')\\\n",
    "  .to_json('./test-summarization2.jsonl', index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "cb359664-b18a-4f32-a688-d8ccfaf5f942",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pandas                               2.1.2\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m23.3.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip list | grep pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "8c60adba-f766-47ac-8bac-95f6fed26028",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>completion</th>\n",
       "      <th>prompt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Mr. Smith's getting a check-up, and Doctor Haw...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Mrs Parker takes Ricky for his vaccines. Dr. P...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>#Person1#'s looking for a set of keys and asks...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>#Person1#'s angry because #Person2# didn't tel...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Malik invites Nikki to dance. Nikki agrees if ...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>995</th>\n",
       "      <td>In terms of the sandstorm, #Person1# prefers t...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>996</th>\n",
       "      <td>#Person2# invites two people that fill the abs...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>997</th>\n",
       "      <td>Claire is pretty stressed and the stress serio...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>998</th>\n",
       "      <td>Marian and Jeanine are shopping, but Marian is...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>999</th>\n",
       "      <td>#Person1# and #Person2# are sharing their diff...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1000 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            completion  \\\n",
       "0    Mr. Smith's getting a check-up, and Doctor Haw...   \n",
       "1    Mrs Parker takes Ricky for his vaccines. Dr. P...   \n",
       "2    #Person1#'s looking for a set of keys and asks...   \n",
       "3    #Person1#'s angry because #Person2# didn't tel...   \n",
       "4    Malik invites Nikki to dance. Nikki agrees if ...   \n",
       "..                                                 ...   \n",
       "995  In terms of the sandstorm, #Person1# prefers t...   \n",
       "996  #Person2# invites two people that fill the abs...   \n",
       "997  Claire is pretty stressed and the stress serio...   \n",
       "998  Marian and Jeanine are shopping, but Marian is...   \n",
       "999  #Person1# and #Person2# are sharing their diff...   \n",
       "\n",
       "                                                prompt  \n",
       "0    Summarize the simplest and most interesting pa...  \n",
       "1    Summarize the simplest and most interesting pa...  \n",
       "2    Summarize the simplest and most interesting pa...  \n",
       "3    Summarize the simplest and most interesting pa...  \n",
       "4    Summarize the simplest and most interesting pa...  \n",
       "..                                                 ...  \n",
       "995  Summarize the simplest and most interesting pa...  \n",
       "996  Summarize the simplest and most interesting pa...  \n",
       "997  Summarize the simplest and most interesting pa...  \n",
       "998  Summarize the simplest and most interesting pa...  \n",
       "999  Summarize the simplest and most interesting pa...  \n",
       "\n",
       "[1000 rows x 2 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_json(\"./train-summarization2.jsonl\", lines=True)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "215843b6-9c4e-4c82-b3c1-c8cc99d01488",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "data = \"./train-summarization2.jsonl\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8de721f4-3f65-4aa4-84ce-19f7a7c47778",
   "metadata": {},
   "source": [
    "Read the JSON line file into an object like any normal file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "bfa423f6-7827-45fc-bd49-39c1b6f9759a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "with open(data) as f:\n",
    "    lines = f.read().splitlines()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46c2ea85-a95b-4cfb-9463-692b9a642897",
   "metadata": {},
   "source": [
    "#### Load the ‘lines’ object into a pandas Data Frame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "7da61e9b-ac1f-4f69-8cb6-b525a737cba2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "df_inter = pd.DataFrame(lines)\n",
    "df_inter.columns = ['json_element']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efbf4879-a796-4359-ba80-f7a54dc05cd6",
   "metadata": {},
   "source": [
    "This intermediate data frame will have only one column with each json object in a row. A sample output is given below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "75a8528c-d478-48a9-acef-ac135b3090d1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      {'completion': 'Mr. Smith's getting a check-up...\n",
       "1      {'completion': 'Mrs Parker takes Ricky for his...\n",
       "2      {'completion': '#Person1#'s looking for a set ...\n",
       "3      {'completion': '#Person1#'s angry because #Per...\n",
       "4      {'completion': 'Malik invites Nikki to dance. ...\n",
       "                             ...                        \n",
       "995    {'completion': 'In terms of the sandstorm, #Pe...\n",
       "996    {'completion': '#Person2# invites two people t...\n",
       "997    {'completion': 'Claire is pretty stressed and ...\n",
       "998    {'completion': 'Marian and Jeanine are shoppin...\n",
       "999    {'completion': '#Person1# and #Person2# are sh...\n",
       "Name: json_element, Length: 1000, dtype: object"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_inter['json_element'].apply(json.loads)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fc711d5-6522-47be-b4b6-fe292d70e6e8",
   "metadata": {},
   "source": [
    "Now we will apply json loads function on each row of the ‘json_element’ column. ‘json.loads’ is a decoder function in python which is used to decode a json object into a dictionary. ‘apply’ is a popular function in pandas that takes any function and applies to each row of the pandas dataframe or series."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "600eac79-5830-4422-a39a-19b6770827a1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_final = pd.json_normalize(df_inter['json_element'].apply(json.loads))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10f2430c-c36f-43a7-8ce6-e17b0efdbd20",
   "metadata": {},
   "source": [
    "Once decoding is done we will apply the json normalize function to the above result. json normalize will convert any semi-structured json data into a flat table. Here it converts the JSON ‘keys’ to columns and its corresponding values to row elements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "e2061f7a-eb1a-4089-96c1-7bfc8dd3d741",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>completion</th>\n",
       "      <th>prompt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Mr. Smith's getting a check-up, and Doctor Haw...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Mrs Parker takes Ricky for his vaccines. Dr. P...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>#Person1#'s looking for a set of keys and asks...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>#Person1#'s angry because #Person2# didn't tel...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Malik invites Nikki to dance. Nikki agrees if ...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>995</th>\n",
       "      <td>In terms of the sandstorm, #Person1# prefers t...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>996</th>\n",
       "      <td>#Person2# invites two people that fill the abs...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>997</th>\n",
       "      <td>Claire is pretty stressed and the stress serio...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>998</th>\n",
       "      <td>Marian and Jeanine are shopping, but Marian is...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>999</th>\n",
       "      <td>#Person1# and #Person2# are sharing their diff...</td>\n",
       "      <td>Summarize the simplest and most interesting pa...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1000 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            completion  \\\n",
       "0    Mr. Smith's getting a check-up, and Doctor Haw...   \n",
       "1    Mrs Parker takes Ricky for his vaccines. Dr. P...   \n",
       "2    #Person1#'s looking for a set of keys and asks...   \n",
       "3    #Person1#'s angry because #Person2# didn't tel...   \n",
       "4    Malik invites Nikki to dance. Nikki agrees if ...   \n",
       "..                                                 ...   \n",
       "995  In terms of the sandstorm, #Person1# prefers t...   \n",
       "996  #Person2# invites two people that fill the abs...   \n",
       "997  Claire is pretty stressed and the stress serio...   \n",
       "998  Marian and Jeanine are shopping, but Marian is...   \n",
       "999  #Person1# and #Person2# are sharing their diff...   \n",
       "\n",
       "                                                prompt  \n",
       "0    Summarize the simplest and most interesting pa...  \n",
       "1    Summarize the simplest and most interesting pa...  \n",
       "2    Summarize the simplest and most interesting pa...  \n",
       "3    Summarize the simplest and most interesting pa...  \n",
       "4    Summarize the simplest and most interesting pa...  \n",
       "..                                                 ...  \n",
       "995  Summarize the simplest and most interesting pa...  \n",
       "996  Summarize the simplest and most interesting pa...  \n",
       "997  Summarize the simplest and most interesting pa...  \n",
       "998  Summarize the simplest and most interesting pa...  \n",
       "999  Summarize the simplest and most interesting pa...  \n",
       "\n",
       "[1000 rows x 2 columns]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_final"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6546f2e9-000b-4c87-807e-2a63554bcc8c",
   "metadata": {},
   "source": [
    "### Uploading data to S3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe91c8bc-cb24-4b04-8de8-25c03c3490df",
   "metadata": {},
   "source": [
    "Next, we need to upload our training dataset to S3:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "49e9547f-127d-444e-89ed-6403781c83a9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "s3_location = f\"s3://{sagemaker_session_bucket}/bedrock/finetuning/train-summarization2.jsonl\"\n",
    "s3_output = f\"s3://{sagemaker_session_bucket}/bedrock/finetuning/output\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "1fb873b6-c50b-46fb-bd1f-a050bf523221",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "upload: ./train-summarization2.jsonl to s3://sagemaker-us-east-1-079002598131/bedrock/finetuning/train-summarization2.jsonl\n"
     ]
    }
   ],
   "source": [
    "!aws s3 cp ./train-summarization2.jsonl $s3_location"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbb85e28-aec1-46bc-aa6c-ba9b04cbc106",
   "metadata": {
    "tags": []
   },
   "source": [
    "Now we can create the fine-tuning job. \n",
    "\n",
    "### ^^ **Note:** Make sure the IAM role you're using has these [IAM policies](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-iam-role.html) attached that allow Amazon Bedrock access to the specified S3 buckets ^^"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c04f82b-a918-4226-bf88-3ce567e50bcc",
   "metadata": {},
   "source": [
    "## 2. Fine-tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "093e54cf-f0f3-403f-8ef5-84e0cbb40d1b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import time\n",
    "timestamp = int(time.time())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "cdc3af82-7cd6-44bd-bf14-e3e455c15e3a",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'command-1700891352'"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "job_name = \"command-{}\".format(timestamp)\n",
    "job_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "adfbd806-cb7e-4414-97a6-9239e98e5604",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'custom-command-1700891352'"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "custom_model_name = \"custom-{}\".format(job_name)\n",
    "custom_model_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "e4f0ead8-31d2-4146-82c8-2d4d5fa07397",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ResponseMetadata': {'RequestId': '981f79c6-9e13-42d9-8a87-3d5e7e161f6f',\n",
       "  'HTTPStatusCode': 201,\n",
       "  'HTTPHeaders': {'date': 'Sat, 25 Nov 2023 05:49:12 GMT',\n",
       "   'content-type': 'application/json',\n",
       "   'content-length': '123',\n",
       "   'connection': 'keep-alive',\n",
       "   'x-amzn-requestid': '981f79c6-9e13-42d9-8a87-3d5e7e161f6f'},\n",
       "  'RetryAttempts': 0},\n",
       " 'jobArn': 'arn:aws:bedrock:us-east-1:079002598131:model-customization-job/cohere.command-light-text-v14:7:4k/zx11jc2mn2hu'}"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bedrock.create_model_customization_job(\n",
    "    jobName=job_name,\n",
    "    customModelName=custom_model_name,\n",
    "    roleArn=role,\n",
    "    baseModelIdentifier=base_model_id,\n",
    "    hyperParameters = {\n",
    "        \"epochCount\": \"1\",\n",
    "        \"batchSize\": \"8\",\n",
    "        \"learningRate\": \"0.000005\",\n",
    "    },\n",
    "    trainingDataConfig={\"s3Uri\": s3_location},\n",
    "    outputDataConfig={\"s3Uri\": s3_output},\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "2b1a6b37-6a22-4f63-8729-185642acc1f1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'InProgress'"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "status = bedrock.get_model_customization_job(jobIdentifier=job_name)[\"status\"]\n",
    "status"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "988e0489-c5e3-4cc7-981b-6706497c6527",
   "metadata": {},
   "source": [
    "# Let's periodically check in on the progress.\n",
    "### The next cell might run for ~40min"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "6f87f790-8ee0-4485-b090-1be7af0f0554",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "InProgress\n",
      "Completed\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "status = bedrock.get_model_customization_job(jobIdentifier=job_name)[\"status\"]\n",
    "\n",
    "while status == \"InProgress\":\n",
    "    print(status)\n",
    "    time.sleep(30)\n",
    "    status = bedrock.get_model_customization_job(jobIdentifier=job_name)[\"status\"]\n",
    "    \n",
    "print(status)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "9813589e-901f-49d5-9a1b-82609a405da9",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ResponseMetadata': {'RequestId': 'aa18f242-4972-4710-8127-5e84a1bd5bab',\n",
       "  'HTTPStatusCode': 200,\n",
       "  'HTTPHeaders': {'date': 'Sat, 25 Nov 2023 06:23:24 GMT',\n",
       "   'content-type': 'application/json',\n",
       "   'content-length': '1297',\n",
       "   'connection': 'keep-alive',\n",
       "   'x-amzn-requestid': 'aa18f242-4972-4710-8127-5e84a1bd5bab'},\n",
       "  'RetryAttempts': 0},\n",
       " 'jobArn': 'arn:aws:bedrock:us-east-1:079002598131:model-customization-job/cohere.command-light-text-v14:7:4k/zx11jc2mn2hu',\n",
       " 'jobName': 'command-1700891352',\n",
       " 'outputModelName': 'custom-command-1700891352',\n",
       " 'outputModelArn': 'arn:aws:bedrock:us-east-1:079002598131:custom-model/cohere.command-light-text-v14:7:4k/4lhrdnpzkqzc',\n",
       " 'clientRequestToken': 'b9265ea3-de40-4646-b0d9-5189967c9a9c',\n",
       " 'roleArn': 'arn:aws:iam::079002598131:role/service-role/AmazonSageMaker-ExecutionRole-20220804T150518',\n",
       " 'status': 'Completed',\n",
       " 'creationTime': datetime.datetime(2023, 11, 25, 5, 49, 12, 797000, tzinfo=tzlocal()),\n",
       " 'lastModifiedTime': datetime.datetime(2023, 11, 25, 6, 23, 21, 570000, tzinfo=tzlocal()),\n",
       " 'endTime': datetime.datetime(2023, 11, 25, 6, 23, 20, 738000, tzinfo=tzlocal()),\n",
       " 'baseModelArn': 'arn:aws:bedrock:us-east-1::foundation-model/cohere.command-light-text-v14:7:4k',\n",
       " 'hyperParameters': {'batchSize': '8',\n",
       "  'earlyStoppingPatience': '6',\n",
       "  'earlyStoppingThreshold': '0.01',\n",
       "  'epochCount': '1',\n",
       "  'evalPercentage': '20.0',\n",
       "  'learningRate': '0.000005'},\n",
       " 'trainingDataConfig': {'s3Uri': 's3://sagemaker-us-east-1-079002598131/bedrock/finetuning/train-summarization2.jsonl'},\n",
       " 'outputDataConfig': {'s3Uri': 's3://sagemaker-us-east-1-079002598131/bedrock/finetuning/output'},\n",
       " 'validationMetrics': []}"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "completed_job = bedrock.get_model_customization_job(jobIdentifier=job_name)\n",
    "completed_job"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15b47ec7-9933-45d1-9ad6-67be8fb318f9",
   "metadata": {},
   "source": [
    "## 3. Testing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b13364c0-17a8-4a30-b354-100c7b43a81b",
   "metadata": {},
   "source": [
    "Now we can test the fine-tuned model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "42f5bdf2-29a9-4041-8a62-b1f69ddbf129",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ResponseMetadata': {'RequestId': '81654d0d-f457-4a98-9aef-8cc79522dccb',\n",
       "  'HTTPStatusCode': 200,\n",
       "  'HTTPHeaders': {'date': 'Sat, 25 Nov 2023 06:23:24 GMT',\n",
       "   'content-type': 'application/json',\n",
       "   'content-length': '383',\n",
       "   'connection': 'keep-alive',\n",
       "   'x-amzn-requestid': '81654d0d-f457-4a98-9aef-8cc79522dccb'},\n",
       "  'RetryAttempts': 0},\n",
       " 'modelSummaries': [{'modelArn': 'arn:aws:bedrock:us-east-1:079002598131:custom-model/cohere.command-light-text-v14:7:4k/4lhrdnpzkqzc',\n",
       "   'modelName': 'custom-command-1700891352',\n",
       "   'creationTime': datetime.datetime(2023, 11, 25, 5, 49, 12, 797000, tzinfo=tzlocal()),\n",
       "   'baseModelArn': 'arn:aws:bedrock:us-east-1::foundation-model/cohere.command-light-text-v14:7:4k',\n",
       "   'baseModelName': ''}]}"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bedrock.list_custom_models()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "c7be0a01-1a4a-41e9-8c39-89f4d36942f1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-----\n",
      "jobArn: arn:aws:bedrock:us-east-1:079002598131:model-customization-job/cohere.command-light-text-v14:7:4k/zx11jc2mn2hu\n",
      "jobName: command-1700891352\n",
      "status: Completed\n",
      "customModelName: custom-command-1700891352\n",
      "-----\n",
      "jobArn: arn:aws:bedrock:us-east-1:079002598131:model-customization-job/meta.llama2-13b-v1:0:4k/xds19wndokym\n",
      "jobName: llama2-1700891164\n",
      "status: InProgress\n",
      "customModelName: custom-llama2-1700891164\n",
      "-----\n",
      "jobArn: arn:aws:bedrock:us-east-1:079002598131:model-customization-job/cohere.command-light-text-v14:7:4k/tzgn6nrk6vug\n",
      "jobName: command-1700867835\n",
      "status: Completed\n",
      "customModelName: custom-command-1700867835\n",
      "-----\n",
      "jobArn: arn:aws:bedrock:us-east-1:079002598131:model-customization-job/meta.llama2-13b-v1:0:4k/nhdtyu1j1p2l\n",
      "jobName: titan-1700863680\n",
      "status: Completed\n",
      "customModelName: custom-titan-1700863680\n",
      "-----\n",
      "jobArn: arn:aws:bedrock:us-east-1:079002598131:model-customization-job/meta.llama2-13b-v1:0:4k/dk583xowsxt2\n",
      "jobName: llama2-1700862197\n",
      "status: Completed\n",
      "customModelName: custom-llama2-1700862197\n"
     ]
    }
   ],
   "source": [
    "for job in bedrock.list_model_customization_jobs()[\"modelCustomizationJobSummaries\"]:\n",
    "    print(\"-----\\n\" + \"jobArn: \" + job[\"jobArn\"] + \"\\njobName: \" + job[\"jobName\"] + \"\\nstatus: \" + job[\"status\"] + \"\\ncustomModelName: \" + job[\"customModelName\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adcf92c4-e2d3-48d4-83ab-592770e6aa61",
   "metadata": {},
   "source": [
    "## GetCustomModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "b0456a56-8bc3-47d2-8d69-4205ad0b1cb3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ResponseMetadata': {'RequestId': '1cf07e91-b825-4527-804b-7ec0f6bdc086',\n",
       "  'HTTPStatusCode': 200,\n",
       "  'HTTPHeaders': {'date': 'Sat, 25 Nov 2023 06:23:24 GMT',\n",
       "   'content-type': 'application/json',\n",
       "   'content-length': '964',\n",
       "   'connection': 'keep-alive',\n",
       "   'x-amzn-requestid': '1cf07e91-b825-4527-804b-7ec0f6bdc086'},\n",
       "  'RetryAttempts': 0},\n",
       " 'modelArn': 'arn:aws:bedrock:us-east-1:079002598131:custom-model/cohere.command-light-text-v14:7:4k/4lhrdnpzkqzc',\n",
       " 'modelName': 'custom-command-1700891352',\n",
       " 'jobArn': 'arn:aws:bedrock:us-east-1:079002598131:model-customization-job/cohere.command-light-text-v14:7:4k/zx11jc2mn2hu',\n",
       " 'baseModelArn': 'arn:aws:bedrock:us-east-1::foundation-model/cohere.command-light-text-v14:7:4k',\n",
       " 'hyperParameters': {'batchSize': '8',\n",
       "  'earlyStoppingPatience': '6',\n",
       "  'earlyStoppingThreshold': '0.01',\n",
       "  'epochCount': '1',\n",
       "  'evalPercentage': '0.2',\n",
       "  'learningRate': '0.000005'},\n",
       " 'trainingDataConfig': {'s3Uri': 's3://sagemaker-us-east-1-079002598131/bedrock/finetuning/train-summarization2.jsonl'},\n",
       " 'outputDataConfig': {'s3Uri': 's3://sagemaker-us-east-1-079002598131/bedrock/finetuning/output'},\n",
       " 'trainingMetrics': {'trainingLoss': 0.9875775575637817},\n",
       " 'validationMetrics': [],\n",
       " 'creationTime': datetime.datetime(2023, 11, 25, 5, 49, 12, 797000, tzinfo=tzlocal())}"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bedrock.get_custom_model(modelIdentifier=custom_model_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "03bb02cd-0624-43f1-b3bc-2369333640d3",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'arn:aws:bedrock:us-east-1:079002598131:custom-model/cohere.command-light-text-v14:7:4k/4lhrdnpzkqzc'"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "custom_model_arn = bedrock.get_custom_model(modelIdentifier=custom_model_name)['modelArn']\n",
    "custom_model_arn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "6a6a7e0b-2ee9-4c58-9db7-9d504b35d67e",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'arn:aws:bedrock:us-east-1::foundation-model/cohere.command-light-text-v14:7:4k'"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "base_model_arn = bedrock.get_custom_model(modelIdentifier=custom_model_name)['baseModelArn']\n",
    "base_model_arn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f80e23e8-c4ee-4598-95ec-65620abb3631",
   "metadata": {},
   "source": [
    "## **Note:** To invoke custom models, you need to first create a provisioned throughput resource and make requests using that resource."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "bd028a6c-f13b-4d58-9b4f-ce8a18aecd66",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'custom-command-1700891352-provisioned'"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "provisioned_model_name = \"{}-provisioned\".format(custom_model_name)\n",
    "provisioned_model_name"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "301d7cbf-f391-4436-953e-70eaa987e3aa",
   "metadata": {
    "tags": []
   },
   "source": [
    "## !! **Note:** SDK currently only supports 1 month and 6 months commitment terms. Go to Bedrock console to manually purchase no commitment term option for testing !!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "531afd39-5bde-4ead-a342-bea7c972c39c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# bedrock.create_provisioned_model_throughput(\n",
    "#     modelUnits = 1,\n",
    "#     commitmentDuration = \"OneMonth\", ## Note: SDK is currently missing No Commitment option\n",
    "#     provisionedModelName = provisioned_model_name,\n",
    "#     modelId = base_model_arn\n",
    "# ) "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3828ec02-d7ae-43d3-afcb-060c95d4f4fd",
   "metadata": {},
   "source": [
    "## ListProvisionedModelThroughputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "269b30af-8167-4c70-be70-ec0050fca906",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'provisionedModelName': 'custom-command-1700891352-provisioned',\n",
       "  'provisionedModelArn': 'arn:aws:bedrock:us-east-1:079002598131:provisioned-model/iv3md8agh95i',\n",
       "  'modelArn': 'arn:aws:bedrock:us-east-1:079002598131:custom-model/cohere.command-light-text-v14:7:4k/4lhrdnpzkqzc',\n",
       "  'desiredModelArn': 'arn:aws:bedrock:us-east-1:079002598131:custom-model/cohere.command-light-text-v14:7:4k/4lhrdnpzkqzc',\n",
       "  'foundationModelArn': 'arn:aws:bedrock:us-east-1::foundation-model/cohere.command-light-text-v14:7:4k',\n",
       "  'modelUnits': 0,\n",
       "  'desiredModelUnits': 1,\n",
       "  'status': 'Creating',\n",
       "  'creationTime': datetime.datetime(2023, 11, 25, 7, 12, 30, 477000, tzinfo=tzlocal()),\n",
       "  'lastModifiedTime': datetime.datetime(2023, 11, 25, 7, 12, 40, 559000, tzinfo=tzlocal())}]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bedrock.list_provisioned_model_throughputs()[\"provisionedModelSummaries\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d22d9e7b-68f8-4162-97d6-fe8f43e3b421",
   "metadata": {
    "tags": []
   },
   "source": [
    "## GetProvisionedModelThroughput"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "c1308145-1e5a-474e-ad82-138de065f16e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#provisioned_model_name = \"<YOUR_PROVISIONED_MODEL_NAME>\" # e.g. custom-titan-1698257909-provisioned\n",
    "provisioned_model_name = \"custom-command-1700891352-provisioned\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "61669df7-a58d-4f86-bb8c-a9319ca52c46",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'arn:aws:bedrock:us-east-1:079002598131:provisioned-model/iv3md8agh95i'"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "provisioned_model_arn = bedrock.get_provisioned_model_throughput(\n",
    "     provisionedModelId=provisioned_model_name)[\"provisionedModelArn\"]\n",
    "provisioned_model_arn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "8b15e071-9f84-4a0b-af8c-2f94875fbefe",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Creating'"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "deployment_status = bedrock.get_provisioned_model_throughput(\n",
    "    provisionedModelId=provisioned_model_name)[\"status\"]\n",
    "deployment_status"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93817fcb-22d3-4c85-9ab5-c78fd67b5a16",
   "metadata": {},
   "source": [
    "## The next cell might run for ~10min"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "b22987e2-82fe-4550-9ecd-d13004f0919f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating\n",
      "Creating\n",
      "Creating\n",
      "Creating\n",
      "Creating\n",
      "Creating\n",
      "Creating\n",
      "Creating\n",
      "Creating\n",
      "Creating\n",
      "Creating\n",
      "Creating\n",
      "Creating\n",
      "InService\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "deployment_status = bedrock.get_provisioned_model_throughput(\n",
    "    provisionedModelId=provisioned_model_name)[\"status\"]\n",
    "\n",
    "while deployment_status == \"Creating\":\n",
    "    \n",
    "    print(deployment_status)\n",
    "    time.sleep(30)\n",
    "    deployment_status = bedrock.get_provisioned_model_throughput(\n",
    "        provisionedModelId=provisioned_model_name)[\"status\"]  \n",
    "    \n",
    "print(deployment_status)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5364ddd6-8ad2-47c8-90b3-e518be36a329",
   "metadata": {},
   "source": [
    "# Qualitative Results with Zero Shot Inference AFTER Fine-Tuning\n",
    "\n",
    "As with many GenAI applications, a qualitative approach where you ask yourself the question \"is my model behaving the way it is supposed to?\" is usually a good starting point. In the example below (the same one we started this notebook with), you can see how the fine-tuned model is able to create a reasonable summary of the dialogue compared to the original inability to understand what is being asked of the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "a1a1abfd-f4c5-4b0f-a17b-a21a31d59f05",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sagemaker.config INFO - Not applying SDK defaults from location: /etc/xdg/sagemaker/config.yaml\n",
      "sagemaker.config INFO - Not applying SDK defaults from location: /root/.config/sagemaker/config.yaml\n",
      "sagemaker.config INFO - Not applying SDK defaults from location: /etc/xdg/sagemaker/config.yaml\n",
      "sagemaker.config INFO - Not applying SDK defaults from location: /root/.config/sagemaker/config.yaml\n"
     ]
    }
   ],
   "source": [
    "import sagemaker\n",
    "\n",
    "sess = sagemaker.Session()\n",
    "sagemaker_session_bucket = sess.default_bucket()\n",
    "role = sagemaker.get_execution_role()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "baabd635-771a-4cb1-ba8e-bc2689ad8b45",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "import json \n",
    "\n",
    "bedrock = boto3.client(service_name=\"bedrock\")\n",
    "bedrock_runtime = boto3.client(service_name=\"bedrock-runtime\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "f012cdf5-98fe-49eb-b0e8-ee305a6e68ea",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "command_base_model_id = \"cohere.command-light-text-v14\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "2152c240-ad34-4615-afc9-6105ed47e090",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "base_model_id = \"cohere.command-light-text-v14:7:4k\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "430411f7-d297-459e-828e-ea52c2c877a4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "prompt = \"Summarize the simplest and most interesting part of the following conversation.\\\\n\\\\n#Person1#: Hello. My name is John Sandals, and I've got a reservation.\\\\n#Person2#: May I see some identification, sir, please?\\\\n#Person1#: Sure. Here you are.\\\\n#Person2#: Thank you so much. Have you got a credit card, Mr. Sandals?\\\\n#Person1#: I sure do. How about American Express?\\\\n#Person2#: Unfortunately, at the present time we take only MasterCard or VISA.\\\\n#Person1#: No American Express? Okay, here's my VISA.\\\\n#Person2#: Thank you, sir. You'll be in room 507, nonsmoking, with a queen-size bed. Do you approve, sir?\\\\n#Person1#: Yeah, that'll be fine.\\\\n#Person2#: That's great. This is your key, sir. If you need anything at all, anytime, just dial zero.\\\\n\\\\nSummary: \""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a8e4853-3e57-48c2-a762-47dd8c710a07",
   "metadata": {},
   "source": [
    "# Command Base Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "087f128b-caf4-41e5-81c2-716ba15fa424",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "John Sandals made a reservation and provided #Person2# with his VISA credit card. #Person2# helped Sandals to check in and gave him the key to his room 507.\n"
     ]
    }
   ],
   "source": [
    "body = {\n",
    "    \"prompt\": prompt,\n",
    "    \"temperature\": 0.5,\n",
    "    \"p\": 0.9,\n",
    "    \"max_tokens\": 512,\n",
    "}\n",
    "\n",
    "response = bedrock_runtime.invoke_model(\n",
    "    modelId=command_base_model_id, \n",
    "    body=json.dumps(body)\n",
    ")\n",
    "\n",
    "response_body = response[\"body\"].read().decode('utf8')\n",
    "print(json.loads(response_body)[\"generations\"][0][\"text\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9a3ff08-cb25-4ea1-9926-4114e34cd275",
   "metadata": {},
   "source": [
    "# Fine-Tuned Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "ad2509c7-9513-4c6b-a516-4707e5ba3a65",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#Person2# helps John Sandals check in and tells him he'll be in room 507.\n"
     ]
    }
   ],
   "source": [
    "body = {\n",
    "    \"prompt\": prompt,\n",
    "    \"temperature\": 0.5,\n",
    "    \"p\": 0.9,\n",
    "    \"max_tokens\": 512,\n",
    "}\n",
    "\n",
    "response = bedrock_runtime.invoke_model(\n",
    "    modelId=provisioned_model_arn, \n",
    "    body=json.dumps(body)\n",
    ")\n",
    "\n",
    "response_body = response[\"body\"].read().decode('utf8')\n",
    "print(json.loads(response_body)[\"generations\"][0][\"text\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36f62c91-c793-439d-b3b4-66f6eb0b0c1b",
   "metadata": {},
   "source": [
    "## Delete Provisioned Throughput"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72cdbf87-b454-410a-9f11-fb42c17ed159",
   "metadata": {},
   "source": [
    "When you're done testing, you can delete Provisioned Throughput to stop charges"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3ad633b8-db9c-43a9-b2ab-b39bc7b3ee97",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# bedrock.delete_provisioned_model_throughput(\n",
    "#     provisionedModelId = provisioned_model_name\n",
    "# )"
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "forced_instance_type": "ml.t3.medium",
  "forced_lcc_arn": "",
  "instance_type": "ml.m5.2xlarge",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}