training_a_BERT_sentiment_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### In this project, we aim to perform sentiment analysis on IMDb movie reviews using the powerful BERT (Bidirectional Encoder Representations from Transformers) model. Sentiment analysis involves determining the sentiment expressed in a piece of text, and in this case, we're focused on classifying movie reviews as either positive or negative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from transformers import BertTokenizer, TFBertForSequenceClassification\n",
    "import pandas as pd\n",
    "from bs4 import BeautifulSoup\n",
    "import re\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import classification_report"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### We will be working with the IMDb review dataset, a widely used benchmark for sentiment analysis. This dataset contains a collection of movie reviews along with their corresponding sentiments.\n",
    "\n",
    "### We start by loading in the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(50000, 2)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>review</th>\n",
       "      <th>sentiment</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>One of the other reviewers has mentioned that ...</td>\n",
       "      <td>positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A wonderful little production. &lt;br /&gt;&lt;br /&gt;The...</td>\n",
       "      <td>positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>I thought this was a wonderful way to spend ti...</td>\n",
       "      <td>positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Basically there's a family where a little boy ...</td>\n",
       "      <td>negative</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Petter Mattei's \"Love in the Time of Money\" is...</td>\n",
       "      <td>positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Probably my all-time favorite movie, a story o...</td>\n",
       "      <td>positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>I sure would like to see a resurrection of a u...</td>\n",
       "      <td>positive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>This show was an amazing, fresh &amp; innovative i...</td>\n",
       "      <td>negative</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Encouraged by the positive comments about this...</td>\n",
       "      <td>negative</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>If you like original gut wrenching laughter yo...</td>\n",
       "      <td>positive</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                              review sentiment\n",
       "0  One of the other reviewers has mentioned that ...  positive\n",
       "1  A wonderful little production. <br /><br />The...  positive\n",
       "2  I thought this was a wonderful way to spend ti...  positive\n",
       "3  Basically there's a family where a little boy ...  negative\n",
       "4  Petter Mattei's \"Love in the Time of Money\" is...  positive\n",
       "5  Probably my all-time favorite movie, a story o...  positive\n",
       "6  I sure would like to see a resurrection of a u...  positive\n",
       "7  This show was an amazing, fresh & innovative i...  negative\n",
       "8  Encouraged by the positive comments about this...  negative\n",
       "9  If you like original gut wrenching laughter yo...  positive"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imdb_data=pd.read_csv('./dataset/IMDB_Dataset.csv')\n",
    "print(imdb_data.shape)\n",
    "imdb_data.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Given the extended training time of the model, we decide to reduce the dataset size while maintaining an even distribution of sentiment labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "samples_per_label = 5000\n",
    "\n",
    "imdb_data_subset = imdb_data.groupby('sentiment', group_keys=False).apply(lambda x: x.head(samples_per_label))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Before proceeding with model training, let's verify that the dataset has the same amount of labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "sentiment\n",
       "negative    5000\n",
       "positive    5000\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "imdb_data_subset['sentiment'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnIAAAHWCAYAAADzS2TwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA74klEQVR4nO3deVRV9f7/8dcRBJnOQQ1BE4c0B0othxRv2jVRLOpqYmaZoTmkoeWUXm7l9K1L2WCTZvfbumLd/JZ2y0qcB7SSSvGaQzmVhoWAExyHBITP749+nOsRVETwuPX5WOus5fl8Pvuz3/tYe73co80YYwQAAADLqeLpAgAAAFA+BDkAAACLIsgBAABYFEEOAADAoghyAAAAFkWQAwAAsCiCHAAAgEUR5AAAACyKIAcAAGBRBDkAljdw4EA1aNDA02UAwGVHkANwUbZu3ao+ffqofv36qlatmq6//np169ZNb775ZqWuNyMjQ1OmTNHmzZsrdT2V5eTJk5oyZYpSUlIqfO7Nmzfr4YcfVnh4uHx9fVWjRg1FRUVpzpw5KiwsrPD1lcff//53LVy40NNlAFcdG+9aBVBW69evV5cuXVSvXj3FxcUpLCxM+/fv1zfffKOffvpJe/bsqbR1b9y4Ue3atdOcOXM0cOBAt76CggIVFRXJ19e30tZ/qQ4dOqSQkBBNnjxZU6ZMqbB53333XQ0fPlyhoaEaMGCAbrzxRh07dkyrVq1ScnKynnvuOf3tb3+rsPWVV2BgoPr06aOkpCRPlwJcVbw9XQAA63j++eflcDi0YcMGBQcHu/VlZ2d7pihJVatW9di6Pembb77R8OHDFRkZqcWLFysoKMjVN3r0aG3cuFHbtm3zYIUAKhunVgGU2U8//aSbbrqpRIiTpFq1apVo+9e//qU2bdrIz89PNWrUUL9+/bR//363MX/+8591880364cfflCXLl3k7++v66+/XtOnT3eNSUlJUbt27SRJgwYNks1mk81mcx3dOfsauX379slms+nll1/WzJkzdcMNN8jf31/du3fX/v37ZYzR//zP/6hu3bry8/NTz549deTIkRL1L1myRJ06dVJAQICCgoIUExOj7du3u40ZOHCgAgMD9dtvv6lXr14KDAxUSEiIxo8f7zqtuW/fPoWEhEiSpk6d6qq/+MhcZmamBg0apLp168rX11e1a9dWz549tW/fvvP+fRTP9cEHH7iFuGJt27Z1O3p54sQJjRs3znUKtmnTpnr55Zd15omZ4t+utCNnZ9YsSVOmTJHNZtOePXs0cOBABQcHy+FwaNCgQTp58qTbcidOnNDcuXNd215c17FjxzR69Gg1aNBAvr6+qlWrlrp166ZNmzadd9sB/IEjcgDKrH79+kpNTdW2bdt08803n3fs888/r2effVZ9+/bVkCFDdPDgQb355pvq3Lmz/vOf/7iFwaNHj6pHjx7q3bu3+vbtq48//lgTJ05UixYtdNddd6l58+aaNm2aJk2apGHDhqlTp06SpI4dO563hg8++ED5+fkaNWqUjhw5ounTp6tv37668847lZKSookTJ2rPnj168803NX78eP3zn/90Lfv+++8rLi5O0dHRevHFF3Xy5Em9/fbbuv322/Wf//zHLTgWFhYqOjpa7du318svv6yVK1fqlVdeUaNGjTRixAiFhITo7bff1ogRI3Tfffepd+/ekqSWLVtKkmJjY7V9+3aNGjVKDRo0UHZ2tlasWKH09PRz3sRx8uRJrVq1Sp07d1a9evXO+ztIkjFGf/nLX7RmzRoNHjxYt9xyi5YtW6annnpKv/32m2bMmHHBOc6lb9++atiwoRITE7Vp0ya9++67qlWrll588UXXbzlkyBDddtttGjZsmCSpUaNGkqThw4fr448/1siRIxUREaHDhw/rq6++0o8//qjWrVuXuybgmmEAoIyWL19uvLy8jJeXl4mMjDQTJkwwy5YtM/n5+W7j9u3bZ7y8vMzzzz/v1r5161bj7e3t1n7HHXcYSea9995zteXl5ZmwsDATGxvratuwYYORZObMmVOirri4OFO/fn3X97179xpJJiQkxOTk5LjaExISjCTTqlUrU1BQ4Gp/8MEHjY+Pjzl16pQxxphjx46Z4OBgM3ToULf1ZGZmGofD4dYeFxdnJJlp06a5jb311ltNmzZtXN8PHjxoJJnJkye7jTt69KiRZF566aUS23U+33//vZFknnzyyTKNX7hwoZFknnvuObf2Pn36GJvNZvbs2WOM+e9vV9rvfHb9kydPNpLMo48+6jbuvvvuMzVr1nRrCwgIMHFxcSXmdDgcJj4+vkzbAKAkTq0CKLNu3bopNTVVf/nLX/T9999r+vTpio6O1vXXX6/PP//cNe6TTz5RUVGR+vbtq0OHDrk+YWFhuvHGG7VmzRq3eQMDA/Xwww+7vvv4+Oi2227Tzz//fEn13n///XI4HK7v7du3lyQ9/PDD8vb2dmvPz8/Xb7/9JklasWKFcnJy9OCDD7rV7+Xlpfbt25eoX/rjyNKZOnXqVKb6/fz85OPjo5SUFB09erTM2+Z0OiWp1FOqpVm8eLG8vLz0xBNPuLWPGzdOxhgtWbKkzOs+W2nbfvjwYVeN5xMcHKxvv/1WGRkZ5V4/cC0jyAG4KO3atdMnn3yio0eP6rvvvlNCQoKOHTumPn366IcffpAk7d69W8YY3XjjjQoJCXH7/PjjjyVujKhbt65sNptbW/Xq1S8q2JTm7FOOxaEuPDy81Pbi9e3evVuSdOedd5aof/ny5SXqr1atmusauIut39fXVy+++KKWLFmi0NBQde7cWdOnT1dmZuZ5l7Pb7ZL+uMasLH755RfVqVOnRPBr3ry5q7+8zv6dq1evLkll2v7p06dr27ZtCg8P12233aYpU6ZccoAHriVcIwegXHx8fNSuXTu1a9dOTZo00aBBg7RgwQJNnjxZRUVFstlsWrJkiby8vEosGxgY6Pa9tDGS3C7CL49zzXuh9RUVFUn649qusLCwEuPOPJp3vvnKavTo0br33nu1cOFCLVu2TM8++6wSExO1evVq3XrrraUu07hxY3l7e2vr1q2XtO6znR2oi53veXSX8vfXt29fderUSZ9++qmWL1+ul156SS+++KI++eQT3XXXXWUrGriGEeQAXLK2bdtKkg4cOCDpjwvZjTFq2LChmjRpUiHrOFfAqAzFF+LXqlVLUVFRFTLnhepv1KiRxo0bp3Hjxmn37t265ZZb9Morr+hf//pXqeP9/f115513avXq1dq/f3+Jo4xnq1+/vlauXKljx465HZXbsWOHq1/679G0nJwct+Uv5YiddP7tr127th5//HE9/vjjys7OVuvWrfX8888T5IAy4NQqgDJbs2ZNqUdZFi9eLElq2rSpJKl3797y8vLS1KlTS4w3xujw4cMXve6AgABJJQNGZYiOjpbdbtff//53FRQUlOg/ePDgRc/p7+8vqWT9J0+e1KlTp9zaGjVqpKCgIOXl5Z13zsmTJ8sYowEDBuj48eMl+tPS0jR37lxJ0t13363CwkK99dZbbmNmzJghm83mCk12u13XXXed1q1b5zZu1qxZF97I8wgICCix7YWFhcrNzXVrq1WrlurUqXPBbQfwB47IASizUaNG6eTJk7rvvvvUrFkz5efna/369froo4/UoEEDDRo0SNIfQeS5555TQkKC9u3bp169eikoKEh79+7Vp59+qmHDhmn8+PEXte5GjRopODhYs2fPVlBQkAICAtS+fXs1bNiwwrfTbrfr7bff1oABA9S6dWv169dPISEhSk9PV3Jysv70pz+VCEQX4ufnp4iICH300Udq0qSJatSooZtvvlmnT59W165d1bdvX0VERMjb21uffvqpsrKy1K9fv/PO2bFjR82cOVOPP/64mjVr5vZmh5SUFH3++ed67rnnJEn33nuvunTpoqefflr79u1Tq1attHz5cn322WcaPXq06yikJA0ZMkQvvPCChgwZorZt22rdunXatWvXxf+QZ2jTpo1WrlypV199VXXq1FHDhg3VtGlT1a1bV3369FGrVq0UGBiolStXasOGDXrllVcuaX3ANcNTt8sCsJ4lS5aYRx991DRr1swEBgYaHx8f07hxYzNq1CiTlZVVYvy///1vc/vtt5uAgAATEBBgmjVrZuLj483OnTtdY+644w5z0003lVj27EeKGGPMZ599ZiIiIoy3t7fbIzLO9fiRsx/psWbNGiPJLFiwwK19zpw5RpLZsGFDifHR0dHG4XCYatWqmUaNGpmBAweajRs3utUZEBBQov7iR3Ocaf369aZNmzbGx8fH9SiPQ4cOmfj4eNOsWTMTEBBgHA6Had++vZk/f36JOc8lLS3NPPTQQ6ZOnTqmatWqpnr16qZr165m7ty5prCw0DXu2LFjZsyYMa5xN954o3nppZdMUVGR23wnT540gwcPNg6HwwQFBZm+ffua7Ozscz5+5ODBg6X+nnv37nW17dixw3Tu3Nn4+fkZSSYuLs7k5eWZp556yrRq1coEBQWZgIAA06pVKzNr1qwybztwreNdqwAAABbFNXIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIvigcBlUFRUpIyMDAUFBV3W1wQBAIBrkzFGx44dU506dVSlyrmPuxHkyiAjI+OC7zEEAACoaPv371fdunXP2U+QK4PiF0zv379fdrvdw9UAAICrndPpVHh4uCuDnAtBrgyKT6fa7XaCHAAAuGwudEkXNzsAAABYFEEOAADAoghyAAAAFkWQAwAAsCiCHAAAgEUR5AAAACyKIAcAAGBRBDkAAACLIsgBAABYFEEOAADAoghyAAAAFuXRIDdlyhTZbDa3T7NmzVz9p06dUnx8vGrWrKnAwEDFxsYqKyvLbY709HTFxMTI399ftWrV0lNPPaXTp0+7jUlJSVHr1q3l6+urxo0bKykp6XJsHgAAQKXy+BG5m266SQcOHHB9vvrqK1ffmDFj9MUXX2jBggVau3atMjIy1Lt3b1d/YWGhYmJilJ+fr/Xr12vu3LlKSkrSpEmTXGP27t2rmJgYdenSRZs3b9bo0aM1ZMgQLVu27LJuJwAAQEWzGWOMp1Y+ZcoULVy4UJs3by7Rl5ubq5CQEM2bN099+vSRJO3YsUPNmzdXamqqOnTooCVLluiee+5RRkaGQkNDJUmzZ8/WxIkTdfDgQfn4+GjixIlKTk7Wtm3bXHP369dPOTk5Wrp0aZnqdDqdcjgcys3Nld1uv/QNBwAAOI+yZg+PH5HbvXu36tSpoxtuuEH9+/dXenq6JCktLU0FBQWKiopyjW3WrJnq1aun1NRUSVJqaqpatGjhCnGSFB0dLafTqe3bt7vGnDlH8ZjiOUqTl5cnp9Pp9gEAALjSeHty5e3bt1dSUpKaNm2qAwcOaOrUqerUqZO2bdumzMxM+fj4KDg42G2Z0NBQZWZmSpIyMzPdQlxxf3Hf+cY4nU79/vvv8vPzK1FXYmKipk6dWlGbWXY22+VfJ3At8tyJiEpnm8p+BLgczOQrYz/i0SB31113uf7csmVLtW/fXvXr19f8+fNLDViXS0JCgsaOHev67nQ6FR4e7rF6AAAASuPxU6tnCg4OVpMmTbRnzx6FhYUpPz9fOTk5bmOysrIUFhYmSQoLCytxF2vx9wuNsdvt5wyLvr6+stvtbh8AAIArzRUV5I4fP66ffvpJtWvXVps2bVS1alWtWrXK1b9z506lp6crMjJSkhQZGamtW7cqOzvbNWbFihWy2+2KiIhwjTlzjuIxxXMAAABYlUeD3Pjx47V27Vrt27dP69ev13333ScvLy89+OCDcjgcGjx4sMaOHas1a9YoLS1NgwYNUmRkpDp06CBJ6t69uyIiIjRgwAB9//33WrZsmZ555hnFx8fL19dXkjR8+HD9/PPPmjBhgnbs2KFZs2Zp/vz5GjNmjCc3HQAA4JJ59Bq5X3/9VQ8++KAOHz6skJAQ3X777frmm28UEhIiSZoxY4aqVKmi2NhY5eXlKTo6WrNmzXIt7+XlpUWLFmnEiBGKjIxUQECA4uLiNG3aNNeYhg0bKjk5WWPGjNHrr7+uunXr6t1331V0dPRl314AAICK5NHnyFnFZXuOHHetApfHVbzb465V4PKo7LtWLfMcOQAAAJQPQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALOqKCXIvvPCCbDabRo8e7Wo7deqU4uPjVbNmTQUGBio2NlZZWVluy6WnpysmJkb+/v6qVauWnnrqKZ0+fdptTEpKilq3bi1fX181btxYSUlJl2GLAAAAKtcVEeQ2bNigd955Ry1btnRrHzNmjL744gstWLBAa9euVUZGhnr37u3qLywsVExMjPLz87V+/XrNnTtXSUlJmjRpkmvM3r17FRMToy5dumjz5s0aPXq0hgwZomXLll227QMAAKgMNmOM8WQBx48fV+vWrTVr1iw999xzuuWWW/Taa68pNzdXISEhmjdvnvr06SNJ2rFjh5o3b67U1FR16NBBS5Ys0T333KOMjAyFhoZKkmbPnq2JEyfq4MGD8vHx0cSJE5WcnKxt27a51tmvXz/l5ORo6dKlZarR6XTK4XAoNzdXdru94n+EYjZb5c0N4L88u9urVLap7EeAy8FMrtz9SFmzh8ePyMXHxysmJkZRUVFu7WlpaSooKHBrb9asmerVq6fU1FRJUmpqqlq0aOEKcZIUHR0tp9Op7du3u8acPXd0dLRrjtLk5eXJ6XS6fQAAAK403p5c+YcffqhNmzZpw4YNJfoyMzPl4+Oj4OBgt/bQ0FBlZma6xpwZ4or7i/vON8bpdOr333+Xn59fiXUnJiZq6tSp5d4uAACAy8FjR+T279+vJ598Uh988IGqVavmqTJKlZCQoNzcXNdn//79ni4JAACgBI8FubS0NGVnZ6t169by9vaWt7e31q5dqzfeeEPe3t4KDQ1Vfn6+cnJy3JbLyspSWFiYJCksLKzEXazF3y80xm63l3o0TpJ8fX1lt9vdPgAAAFcajwW5rl27auvWrdq8ebPr07ZtW/Xv39/156pVq2rVqlWuZXbu3Kn09HRFRkZKkiIjI7V161ZlZ2e7xqxYsUJ2u10RERGuMWfOUTymeA4AAACr8tg1ckFBQbr55pvd2gICAlSzZk1X++DBgzV27FjVqFFDdrtdo0aNUmRkpDp06CBJ6t69uyIiIjRgwABNnz5dmZmZeuaZZxQfHy9fX19J0vDhw/XWW29pwoQJevTRR7V69WrNnz9fycnJl3eDAQAAKphHb3a4kBkzZqhKlSqKjY1VXl6eoqOjNWvWLFe/l5eXFi1apBEjRigyMlIBAQGKi4vTtGnTXGMaNmyo5ORkjRkzRq+//rrq1q2rd999V9HR0Z7YJAAAgArj8efIWQHPkQOuMlfxbo/nyAGXB8+RAwAAwCUhyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALMqjQe7tt99Wy5YtZbfbZbfbFRkZqSVLlrj6T506pfj4eNWsWVOBgYGKjY1VVlaW2xzp6emKiYmRv7+/atWqpaeeekqnT592G5OSkqLWrVvL19dXjRs3VlJS0uXYPAAAgErl0SBXt25dvfDCC0pLS9PGjRt15513qmfPntq+fbskacyYMfriiy+0YMECrV27VhkZGerdu7dr+cLCQsXExCg/P1/r16/X3LlzlZSUpEmTJrnG7N27VzExMerSpYs2b96s0aNHa8iQIVq2bNll314AAICKZDPGGE8XcaYaNWropZdeUp8+fRQSEqJ58+apT58+kqQdO3aoefPmSk1NVYcOHbRkyRLdc889ysjIUGhoqCRp9uzZmjhxog4ePCgfHx9NnDhRycnJ2rZtm2sd/fr1U05OjpYuXVqmmpxOpxwOh3Jzc2W32yt+o4vZbJU3N4D/urJ2exXKNpX9CHA5mMmVux8pa/a4Yq6RKyws1IcffqgTJ04oMjJSaWlpKigoUFRUlGtMs2bNVK9ePaWmpkqSUlNT1aJFC1eIk6To6Gg5nU7XUb3U1FS3OYrHFM9Rmry8PDmdTrcPAADAlcbjQW7r1q0KDAyUr6+vhg8frk8//VQRERHKzMyUj4+PgoOD3caHhoYqMzNTkpSZmekW4or7i/vON8bpdOr3338vtabExEQ5HA7XJzw8vCI2FQAAoEJ5PMg1bdpUmzdv1rfffqsRI0YoLi5OP/zwg0drSkhIUG5uruuzf/9+j9YDAABQGm9PF+Dj46PGjRtLktq0aaMNGzbo9ddf1wMPPKD8/Hzl5OS4HZXLyspSWFiYJCksLEzfffed23zFd7WeOebsO12zsrJkt9vl5+dXak2+vr7y9fWtkO0DAACoLB4/Ine2oqIi5eXlqU2bNqpatapWrVrl6tu5c6fS09MVGRkpSYqMjNTWrVuVnZ3tGrNixQrZ7XZFRES4xpw5R/GY4jkAAACsyqNH5BISEnTXXXepXr16OnbsmObNm6eUlBQtW7ZMDodDgwcP1tixY1WjRg3Z7XaNGjVKkZGR6tChgySpe/fuioiI0IABAzR9+nRlZmbqmWeeUXx8vOuI2vDhw/XWW29pwoQJevTRR7V69WrNnz9fycnJntx0AACAS+bRIJedna1HHnlEBw4ckMPhUMuWLbVs2TJ169ZNkjRjxgxVqVJFsbGxysvLU3R0tGbNmuVa3svLS4sWLdKIESMUGRmpgIAAxcXFadq0aa4xDRs2VHJyssaMGaPXX39ddevW1bvvvqvo6OjLvr0AAAAVqVzPkdu0aZOqVq2qFi1aSJI+++wzzZkzRxEREZoyZYp8fHwqvFBP4jlywFWG58gBuESWfo7cY489pl27dkmSfv75Z/Xr10/+/v5asGCBJkyYUL6KAQAAcFHKFeR27dqlW265RZK0YMECde7cWfPmzVNSUpL+/e9/V2R9AAAAOIdyBTljjIqKiiRJK1eu1N133y1JCg8P16FDhyquOgAAAJxTuYJc27Zt9dxzz+n999/X2rVrFRMTI+mPF9Sf/RYFAAAAVI5yBbkZM2Zo06ZNGjlypJ5++mnXA30//vhjdezYsUILBAAAQOnK9fiRVq1aaevWrSXaX3rpJXl7e/xlEQAAANeEch2Ru+GGG3T48OES7adOnVKTJk0uuSgAAABcWLmC3L59+1RYWFiiPS8vT7/++uslFwUAAIALu6jzoJ9//rnrz8Wv0SpWWFioVatWqWHDhhVXHQAAAM7pooJcr169JEk2m01xcXFufVWrVlWDBg30yiuvVFhxAAAAOLeLCnLFz45r2LChNmzYoOuuu65SigIAAMCFlesW071791Z0HQAAALhI5X5WyKpVq7Rq1SplZ2e7jtQV++c//3nJhQEAAOD8yhXkpk6dqmnTpqlt27aqXbu2bDZbRdcFAACACyhXkJs9e7aSkpI0YMCAiq4HAAAAZVSu58jl5+fzKi4AAAAPK1eQGzJkiObNm1fRtQAAAOAilOvU6qlTp/SPf/xDK1euVMuWLVW1alW3/ldffbVCigMAAMC5lSvIbdmyRbfccoskadu2bW593PgAAABweZQryK1Zs6ai6wAAAMBFKtc1cgAAAPC8ch2R69Kly3lPoa5evbrcBQEAAKBsyhXkiq+PK1ZQUKDNmzdr27ZtiouLq4i6AAAAcAHlCnIzZswotX3KlCk6fvz4JRUEAACAsqnQa+Qefvhh3rMKAABwmVRokEtNTVW1atUqckoAAACcQ7lOrfbu3dvtuzFGBw4c0MaNG/Xss89WSGEAAAA4v3IFOYfD4fa9SpUqatq0qaZNm6bu3btXSGEAAAA4v3IFuTlz5lR0HQAAALhI5QpyxdLS0vTjjz9Kkm666SbdeuutFVIUAAAALqxcQS47O1v9+vVTSkqKgoODJUk5OTnq0qWLPvzwQ4WEhFRkjQAAAChFue5aHTVqlI4dO6bt27fryJEjOnLkiLZt2yan06knnniiomsEAABAKcp1RG7p0qVauXKlmjdv7mqLiIjQzJkzudkBAADgMinXEbmioiJVrVq1RHvVqlVVVFR0yUUBAADgwsoV5O688049+eSTysjIcLX99ttvGjNmjLp27VphxQEAAODcyhXk3nrrLTmdTjVo0ECNGjVSo0aN1LBhQzmdTr355psVXSMAAABKUa5r5MLDw7Vp0yatXLlSO3bskCQ1b95cUVFRFVocAAAAzu2ijsitXr1aERERcjqdstls6tatm0aNGqVRo0apXbt2uummm/Tll19WVq0AAAA4w0UFuddee01Dhw6V3W4v0edwOPTYY4/p1VdfrbDiAAAAcG4XFeS+//579ejR45z93bt3V1pa2iUXBQAAgAu7qCCXlZVV6mNHinl7e+vgwYOXXBQAAAAu7KKC3PXXX69t27ads3/Lli2qXbv2JRcFAACAC7uoIHf33Xfr2Wef1alTp0r0/f7775o8ebLuueeeCisOAAAA52YzxpiyDs7KylLr1q3l5eWlkSNHqmnTppKkHTt2aObMmSosLNSmTZsUGhpaaQV7gtPplMPhUG5ubqk3elQYm63y5gbwX2Xf7VmObSr7EeByMJMrdz9S1uxxUc+RCw0N1fr16zVixAglJCSoOAPabDZFR0dr5syZV12IAwAAuFJd9AOB69evr8WLF+vo0aPas2ePjDG68cYbVb169cqoDwAAAOdQrjc7SFL16tXVrl27iqwFAAAAF6Fc71oFAACA5xHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAW5dEgl5iYqHbt2ikoKEi1atVSr169tHPnTrcxp06dUnx8vGrWrKnAwEDFxsYqKyvLbUx6erpiYmLk7++vWrVq6amnntLp06fdxqSkpKh169by9fVV48aNlZSUVNmbBwAAUKk8GuTWrl2r+Ph4ffPNN1qxYoUKCgrUvXt3nThxwjVmzJgx+uKLL7RgwQKtXbtWGRkZ6t27t6u/sLBQMTExys/P1/r16zV37lwlJSVp0qRJrjF79+5VTEyMunTpos2bN2v06NEaMmSIli1bdlm3FwAAoCLZjDHG00UUO3jwoGrVqqW1a9eqc+fOys3NVUhIiObNm6c+ffpIknbs2KHmzZsrNTVVHTp00JIlS3TPPfcoIyNDoaGhkqTZs2dr4sSJOnjwoHx8fDRx4kQlJydr27ZtrnX169dPOTk5Wrp06QXrcjqdcjgcys3Nld1ur5yNlySbrfLmBvBfV85ur8LZprIfAS4HM7ly9yNlzR5X1DVyubm5kqQaNWpIktLS0lRQUKCoqCjXmGbNmqlevXpKTU2VJKWmpqpFixauECdJ0dHRcjqd2r59u2vMmXMUjyme42x5eXlyOp1uHwAAgCvNFRPkioqKNHr0aP3pT3/SzTffLEnKzMyUj4+PgoOD3caGhoYqMzPTNebMEFfcX9x3vjFOp1O///57iVoSExPlcDhcn/Dw8ArZRgAAgIp0xQS5+Ph4bdu2TR9++KGnS1FCQoJyc3Ndn/3793u6JAAAgBK8PV2AJI0cOVKLFi3SunXrVLduXVd7WFiY8vPzlZOT43ZULisrS2FhYa4x3333ndt8xXe1njnm7Dtds7KyZLfb5efnV6IeX19f+fr6Vsi2AQAAVBaPHpEzxmjkyJH69NNPtXr1ajVs2NCtv02bNqpatapWrVrlatu5c6fS09MVGRkpSYqMjNTWrVuVnZ3tGrNixQrZ7XZFRES4xpw5R/GY4jkAAACsyKNH5OLj4zVv3jx99tlnCgoKcl3T5nA45OfnJ4fDocGDB2vs2LGqUaOG7Ha7Ro0apcjISHXo0EGS1L17d0VERGjAgAGaPn26MjMz9cwzzyg+Pt51VG348OF66623NGHCBD366KNavXq15s+fr+TkZI9tOwAAwKXy6ONHbOd43MacOXM0cOBASX88EHjcuHH6v//7P+Xl5Sk6OlqzZs1ynTaVpF9++UUjRoxQSkqKAgICFBcXpxdeeEHe3v/NqSkpKRozZox++OEH1a1bV88++6xrHRfC40eAqwyPHwFwia6Ux49cUc+Ru1IR5ICrzFW82yPIAZfHlRLkrpi7VgEAAHBxCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEeDXLr1q3Tvffeqzp16shms2nhwoVu/cYYTZo0SbVr15afn5+ioqK0e/dutzFHjhxR//79ZbfbFRwcrMGDB+v48eNuY7Zs2aJOnTqpWrVqCg8P1/Tp0yt70wAAACqdR4PciRMn1KpVK82cObPU/unTp+uNN97Q7Nmz9e233yogIEDR0dE6deqUa0z//v21fft2rVixQosWLdK6des0bNgwV7/T6VT37t1Vv359paWl6aWXXtKUKVP0j3/8o9K3DwAAoDLZjDHG00VIks1m06effqpevXpJ+uNoXJ06dTRu3DiNHz9ekpSbm6vQ0FAlJSWpX79++vHHHxUREaENGzaobdu2kqSlS5fq7rvv1q+//qo6dero7bff1tNPP63MzEz5+PhIkv76179q4cKF2rFjR5lqczqdcjgcys3Nld1ur/iNL2azVd7cAP7rytjtVQrbVPYjwOVgJlfufqSs2eOKvUZu7969yszMVFRUlKvN4XCoffv2Sk1NlSSlpqYqODjYFeIkKSoqSlWqVNG3337rGtO5c2dXiJOk6Oho7dy5U0ePHi113Xl5eXI6nW4fAACAK80VG+QyMzMlSaGhoW7toaGhrr7MzEzVqlXLrd/b21s1atRwG1PaHGeu42yJiYlyOByuT3h4+KVvEAAAQAW7YoOcJyUkJCg3N9f12b9/v6dLAgAAKOGKDXJhYWGSpKysLLf2rKwsV19YWJiys7Pd+k+fPq0jR464jSltjjPXcTZfX1/Z7Xa3DwAAwJXmig1yDRs2VFhYmFatWuVqczqd+vbbbxUZGSlJioyMVE5OjtLS0lxjVq9eraKiIrVv3941Zt26dSooKHCNWbFihZo2barq1atfpq0BAACoeB4NcsePH9fmzZu1efNmSX/c4LB582alp6fLZrNp9OjReu655/T5559r69ateuSRR1SnTh3Xna3NmzdXjx49NHToUH333Xf6+uuvNXLkSPXr10916tSRJD300EPy8fHR4MGDtX37dn300Ud6/fXXNXbsWA9tNQAAQMXw9uTKN27cqC5duri+F4eruLg4JSUlacKECTpx4oSGDRumnJwc3X777Vq6dKmqVavmWuaDDz7QyJEj1bVrV1WpUkWxsbF64403XP0Oh0PLly9XfHy82rRpo+uuu06TJk1ye9YcAACAFV0xz5G7kvEcOeAqcxXv9niOHHB58Bw5AAAAXBKCHAAAgEUR5AAAACyKIAcAAGBRBDkAAACLIsgBAABYFEEOAADAoghyAAAAFkWQAwAAsCiCHAAAgEUR5AAAACyKIAcAAGBRBDkAAACLIsgBAABYFEEOAADAoghyAAAAFkWQAwAAsCiCHAAAgEUR5AAAACyKIAcAAGBRBDkAAACLIsgBAABYFEEOAADAoghyAAAAFkWQAwAAsCiCHAAAgEUR5AAAACyKIAcAAGBRBDkAAACLIsgBAABYFEEOAADAoghyAAAAFkWQAwAAsCiCHAAAgEUR5AAAACyKIAcAAGBRBDkAAACLIsgBAABYFEEOAADAoghyAAAAFkWQAwAAsCiCHAAAgEUR5AAAACyKIAcAAGBRBDkAAACLIsgBAABYFEEOAADAoghyAAAAFkWQAwAAsCiCHAAAgEUR5AAAACyKIAcAAGBRBDkAAACLIsgBAABYFEEOAADAoghyAAAAFkWQAwAAsCiCHAAAgEUR5AAAACyKIAcAAGBRBDkAAACLIsgBAABYFEEOAADAoq6pIDdz5kw1aNBA1apVU/v27fXdd995uiQAAIByu2aC3EcffaSxY8dq8uTJ2rRpk1q1aqXo6GhlZ2d7ujQAAIByuWaC3KuvvqqhQ4dq0KBBioiI0OzZs+Xv769//vOfni4NAACgXLw9XcDlkJ+fr7S0NCUkJLjaqlSpoqioKKWmppYYn5eXp7y8PNf33NxcSZLT6az8YgFUvqv5/+VTni4AuDZUdiYont8Yc95x10SQO3TokAoLCxUaGurWHhoaqh07dpQYn5iYqKlTp5ZoDw8Pr7QaAVxGDoenKwBgcY4XLs9+5NixY3KcZ591TQS5i5WQkKCxY8e6vhcVFenIkSOqWbOmbDabByvDlcbpdCo8PFz79++X3W73dDkALIj9CEpjjNGxY8dUp06d8467JoLcddddJy8vL2VlZbm1Z2VlKSwsrMR4X19f+fr6urUFBwdXZomwOLvdzg4YwCVhP4Kzne9IXLFr4mYHHx8ftWnTRqtWrXK1FRUVadWqVYqMjPRgZQAAAOV3TRyRk6SxY8cqLi5Obdu21W233abXXntNJ06c0KBBgzxdGgAAQLlcM0HugQce0MGDBzVp0iRlZmbqlltu0dKlS0vcAAFcDF9fX02ePLnEqXgAKCv2I7gUNnOh+1oBAABwRbomrpEDAAC4GhHkAAAALIogBwAAYFEEOeAyatCggV577TVPlwHAw1JSUmSz2ZSTk3PecewzcCEEOVw1Bg4cKJvNphdeeMGtfeHChZf9jRxJSUmlPkR6w4YNGjZs2GWtBUD5Fe9XbDabfHx81LhxY02bNk2nT5++pHk7duyoAwcOuB74yj4D5UWQw1WlWrVqevHFF3X06FFPl1KqkJAQ+fv7e7oMABehR48eOnDggHbv3q1x48ZpypQpeumlly5pTh8fH4WFhV3wH5nsM3AhBDlcVaKiohQWFqbExMRzjvnqq6/UqVMn+fn5KTw8XE888YROnDjh6j9w4IBiYmLk5+enhg0bat68eSVOb7z66qtq0aKFAgICFB4erscff1zHjx+X9Mcpk0GDBik3N9f1L/kpU6ZIcj9N8tBDD+mBBx5wq62goEDXXXed3nvvPUl/vIEkMTFRDRs2lJ+fn1q1aqWPP/64An4pAGXl6+ursLAw1a9fXyNGjFBUVJQ+//xzHT16VI888oiqV68uf39/3XXXXdq9e7druV9++UX33nuvqlevroCAAN10001avHixJPdTq+wzcCkIcriqeHl56e9//7vefPNN/frrryX6f/rpJ/Xo0UOxsbHasmWLPvroI3311VcaOXKka8wjjzyijIwMpaSk6N///rf+8Y9/KDs7222eKlWq6I033tD27ds1d+5crV69WhMmTJD0xymT1157TXa7XQcOHNCBAwc0fvz4ErX0799fX3zxhSsAStKyZct08uRJ3XfffZKkxMREvffee5o9e7a2b9+uMWPG6OGHH9batWsr5PcCcPH8/PyUn5+vgQMHauPGjfr888+VmpoqY4zuvvtuFRQUSJLi4+OVl5endevWaevWrXrxxRcVGBhYYj72GbgkBrhKxMXFmZ49expjjOnQoYN59NFHjTHGfPrpp6b4P/XBgwebYcOGuS335ZdfmipVqpjff//d/Pjjj0aS2bBhg6t/9+7dRpKZMWPGOde9YMECU7NmTdf3OXPmGIfDUWJc/fr1XfMUFBSY6667zrz33nuu/gcffNA88MADxhhjTp06Zfz9/c369evd5hg8eLB58MEHz/9jAKgQZ+5XioqKzIoVK4yvr6/p1auXkWS+/vpr19hDhw4ZPz8/M3/+fGOMMS1atDBTpkwpdd41a9YYSebo0aPGGPYZKL9r5hVduLa8+OKLuvPOO0v8q/b777/Xli1b9MEHH7jajDEqKirS3r17tWvXLnl7e6t169au/saNG6t69epu86xcuVKJiYnasWOHnE6nTp8+rVOnTunkyZNlvp7F29tbffv21QcffKABAwboxIkT+uyzz/Thhx9Kkvbs2aOTJ0+qW7dubsvl5+fr1ltvvajfA0D5LVq0SIGBgSooKFBRUZEeeugh9e7dW4sWLVL79u1d42rWrKmmTZvqxx9/lCQ98cQTGjFihJYvX66oqCjFxsaqZcuW5a6DfQZKQ5DDValz586Kjo5WQkKCBg4c6Go/fvy4HnvsMT3xxBMllqlXr5527dp1wbn37dune+65RyNGjNDzzz+vGjVq6KuvvtLgwYOVn59/URcm9+/fX3fccYeys7O1YsUK+fn5qUePHq5aJSk5OVnXX3+923K8kxG4fLp06aK3335bPj4+qlOnjry9vfX5559fcLkhQ4YoOjpaycnJWr58uRITE/XKK69o1KhR5a6FfQbORpDDVeuFF17QLbfcoqZNm7raWrdurR9++EGNGzcudZmmTZvq9OnT+s9//qM2bdpI+uNfuWfeBZuWlqaioiK98sorqlLlj8tM58+f7zaPj4+PCgsLL1hjx44dFR4ero8++khLlizR/fffr6pVq0qSIiIi5Ovrq/T0dN1xxx0Xt/EAKkxAQECJfUbz5s11+vRpffvtt+rYsaMk6fDhw9q5c6ciIiJc48LDwzV8+HANHz5cCQkJ+t///d9Sgxz7DJQXQQ5XrRYtWqh///564403XG0TJ05Uhw4dNHLkSA0ZMkQBAQH64YcftGLFCr311ltq1qyZoqKiNGzYML399tuqWrWqxo0bJz8/P9djAho3bqyCggK9+eabuvfee/X1119r9uzZbutu0KCBjh8/rlWrVqlVq1by9/c/55G6hx56SLNnz9auXbu0Zs0aV3tQUJDGjx+vMWPGqKioSLfffrtyc3P19ddfy263Ky4urhJ+NQBlceONN6pnz54aOnSo3nnnHQUFBemvf/2rrr/+evXs2VOSNHr0aN11111q0qSJjh49qjVr1qh58+alzsc+A+Xm6Yv0gIpy5kXJxfbu3Wt8fHzMmf+pf/fdd6Zbt24mMDDQBAQEmJYtW5rnn3/e1Z+RkWHuuusu4+vra+rXr2/mzZtnatWqZWbPnu0a8+qrr5ratWsbPz8/Ex0dbd577z23C5eNMWb48OGmZs2aRpKZPHmyMcb9wuViP/zwg5Fk6tevb4qKitz6ioqKzGuvvWaaNm1qqlatakJCQkx0dLRZu3btpf1YAMqktP1KsSNHjpgBAwYYh8Ph2hfs2rXL1T9y5EjTqFEj4+vra0JCQsyAAQPMoUOHjDElb3Ywhn0GysdmjDEezJHAFe/XX39VeHi4Vq5cqa5du3q6HAAAXAhywFlWr16t48ePq0WLFjpw4IAmTJig3377Tbt27XJdiwIAwJWAa+SAsxQUFOhvf/ubfv75ZwUFBaljx4764IMPCHEAgCsOR+QAAAAsild0AQAAWBRBDgAAwKIIcgAAABZFkAMAALAoghwAAIBFEeQAoIKlpKTIZrMpJyfH06UAuMoR5ABctQ4ePKgRI0aoXr168vX1VVhYmKKjo/X1119X2Dr+/Oc/a/To0W5tHTt21IEDB+RwOCpsPeU1cOBA9erVy9NlAKgkPBAYwFUrNjZW+fn5mjt3rm644QZlZWVp1apVOnz4cKWu18fHR2FhYZW6DgCQdMabxAHgKnL06FEjyaSkpJx3zODBg811111ngoKCTJcuXczmzZtd/ZMnTzatWrUy7733nqlfv76x2+3mgQceME6n0xjzxwvVJbl99u7dW+KF6HPmzDEOh8N88cUXpkmTJsbPz8/ExsaaEydOmKSkJFO/fn0THBxsRo0aZU6fPu1a/6lTp8y4ceNMnTp1jL+/v7ntttvMmjVrXP3F8y5dutQ0a9bMBAQEmOjoaJORkeGq/+z6zlwegPVxahXAVSkwMFCBgYFauHCh8vLySh1z//33Kzs7W0uWLFFaWppat26trl276siRI64xP/30kxYuXKhFixZp0aJFWrt2rV544QVJ0uuvv67IyEgNHTpUBw4c0IEDBxQeHl7quk6ePKk33nhDH374oZYuXaqUlBTdd999Wrx4sRYvXqz3339f77zzjj7++GPXMiNHjlRqaqo+/PBDbdmyRffff7969Oih3bt3u8378ssv6/3339e6deuUnp6u8ePHS5LGjx+vvn37qkePHq76OnbseMm/LYAriKeTJABUlo8//thUr17dVKtWzXTs2NEkJCSY77//3hhjzJdffmnsdrs5deqU2zKNGjUy77zzjjHmjyNa/v7+riNwxhjz1FNPmfbt27u+33HHHebJJ590m6O0I3KSzJ49e1xjHnvsMePv72+OHTvmaouOjjaPPfaYMcaYX375xXh5eZnffvvNbe6uXbuahISEc847c+ZMExoa6voeFxdnevbsWabfC4D1cI0cgKtWbGysYmJi9OWXX+qbb77RkiVLNH36dL377rs6ceKEjh8/rpo1a7ot8/vvv+unn35yfW/QoIGCgoJc32vXrq3s7OyLrsXf31+NGjVyfQ8NDVWDBg0UGBjo1lY899atW1VYWKgmTZq4zZOXl+dW89nzlrc+ANZEkANwVatWrZq6deumbt266dlnn9WQIUM0efJkPf7446pdu7ZSUlJKLBMcHOz6c9WqVd36bDabioqKLrqO0uY539zHjx+Xl5eX0tLS5OXl5TbuzPBX2hzGmIuuD4A1EeQAXFMiIiK0cOFCtW7dWpmZmfL29laDBg3KPZ+Pj48KCwsrrsD/79Zbb1VhYaGys7PVqVOncs9TWfUBuDJwswOAq9Lhw4d155136l//+pe2bNmivXv3asGCBZo+fbp69uypqKgoRUZGqlevXlq+fLn27dun9evX6+mnn9bGjRvLvJ4GDRro22+/1b59+3To0KFyHa0rTZMmTdS/f3898sgj+uSTT7R371599913SkxMVHJy8kXVt2XLFu3cuVOHDh1SQUFBhdQH4MpAkANwVQoMDFT79u01Y8YMde7cWTfffLOeffZZDR06VG+99ZZsNpsWL16szp07a9CgQWrSpIn69eunX375RaGhoWVez/jx4+Xl5aWIiAiFhIQoPT29wrZhzpw5euSRRzRu3Dg1bdpUvXr10oYNG1SvXr0yzzF06FA1bdpUbdu2VUhISIU+DBmA59kMF1MAAABYEkfkAAAALIogBwAAYFEEOQAAAIsiyAEAAFgUQQ4AAMCiCHIAAAAWRZADAACwKIIcAACARRHkAAAALIogBwAAYFEEOQAAAIv6f/1ISdUUiUgFAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 700x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sentiment_counts = imdb_data_subset['sentiment'].value_counts()\n",
    "\n",
    "# Skapa stapeldiagram med pyplot\n",
    "plt.bar(['Negative', 'Positive'], sentiment_counts.values, color=['red', 'green'])\n",
    "plt.title('Sentiments Counts')\n",
    "plt.xlabel('Sentiment')\n",
    "plt.ylabel('Counts')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To assess the performance of our sentiment analysis model, we split the dataset into training and test sets. For this specific configuration, we aim to have 4000 samples for each sentiment label in the training set and 1000 samples for each label in the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_samples_per_label = 4000\n",
    "test_samples_per_label = 1000\n",
    "train_df, test_df = train_test_split(imdb_data_subset, stratify=imdb_data_subset['sentiment'], \n",
    "                                       train_size=train_samples_per_label*2, test_size=test_samples_per_label*2, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Before feeding the text data into the model, it's essential to preprocess it. The following function, `text_cleaning`, utilizes BeautifulSoup and regular expressions to clean the text from HTML tags and special characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def text_cleaning(text):\n",
    "    soup = BeautifulSoup(text, \"html.parser\")\n",
    "    text = re.sub(r'\\[[^]]*\\]', '', soup.get_text())\n",
    "    pattern = r\"[^a-zA-Z0-9\\s,']\"\n",
    "    text = re.sub(pattern, '', text)\n",
    "    return text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now that we have defined the `text_cleaning` function, let's apply it to the review text in both the training and test datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Nicla\\AppData\\Local\\Temp\\ipykernel_8440\\3524044076.py:2: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.\n",
      "  soup = BeautifulSoup(text, \"html.parser\")\n",
      "C:\\Users\\Nicla\\AppData\\Local\\Temp\\ipykernel_8440\\3757904458.py:2: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  train_df['Cleaned_sentence'] = train_df['review'].apply(text_cleaning).tolist()\n",
      "C:\\Users\\Nicla\\AppData\\Local\\Temp\\ipykernel_8440\\3524044076.py:2: MarkupResemblesLocatorWarning: The input looks more like a filename than markup. You may want to open this file and pass the filehandle into Beautiful Soup.\n",
      "  soup = BeautifulSoup(text, \"html.parser\")\n",
      "C:\\Users\\Nicla\\AppData\\Local\\Temp\\ipykernel_8440\\3757904458.py:4: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  test_df['Cleaned_sentence'] = test_df['review'].apply(text_cleaning)\n"
     ]
    }
   ],
   "source": [
    "# Train dataset\n",
    "train_df['Cleaned_sentence'] = train_df['review'].apply(text_cleaning).tolist()\n",
    "# Test dataset\n",
    "test_df['Cleaned_sentence'] = test_df['review'].apply(text_cleaning)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To gain insights into the distribution of review lengths in the training dataset, a bar chart is created. Each bar represents the number of characters in a respective review."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Text(0.5, 1.0, 'No. of characters in each review'),\n",
       " Text(0.5, 0, 'Number of review'),\n",
       " Text(0, 0.5, 'Number of Characters in each review')]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnsAAAHWCAYAAAAPXk86AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABf3UlEQVR4nO3dd1gUV9sG8HtBdmnSlCIWxBJUxEqiWNBEIirxtb2xhMRuGkZRY9QklhTFkpior8YYE/X9YmyJGkssBFQsiBUrYg9GBYwIa0Vgz/eHH/OxgrgDs7C73L/r2kv3zNkzz5mZnX2YckYlhBAgIiIiIotkVd4BEBEREZHxMNkjIiIismBM9oiIiIgsGJM9IiIiIgvGZI+IiIjIgjHZIyIiIrJgTPaIiIiILBiTPSIiIiILxmSPiIiIyIIx2SOiYh0+fBht2rSBg4MDVCoVEhMTS9zW7t27oVKp8OuvvyoXYAXXsWNHdOzYsbzDKDWVSoWRI0eWdxjPNW3aNKhUqvIOg0gWJntEZWz58uVQqVSwtbXF9evXC03v2LEjGjduXA6RFZaTk4PXX38dGRkZ+Oabb/A///M/8PHxKe+wysWBAwcwbdo0ZGZmlncoRESyMNkjKifZ2dmYOXNmeYdRrEuXLuGvv/7Chx9+iLfffhtvvvkmXF1dyzuscnHgwAF89tlnJpfs7dy5Ezt37izvMCqMTz/9FA8fPizvMIhkYbJHVE6aNWuGH374ATdu3CjvUJ4pPT0dAODi4lK+gch0//798g7BYA8ePCjV59VqNdRqtULRWI5Hjx5Bp9Mp3m6lSpVga2ureLtExsRkj6icfPzxx8jLyzPo6F5ubi6++OIL1K1bFxqNBrVr18bHH3+M7OzsEs8/NjYW7du3h4ODA1xcXNCjRw8kJSVJ0wcPHowOHToAAF5//XWoVKrnXhuWmZmJMWPGoHbt2tBoNKhRowYGDhyIf/75R6+eTqfD9OnTUaNGDdja2qJTp064ePGiXp29e/fi9ddfR61ataDRaFCzZk2MGTOm0FGVwYMHw9HREZcuXUK3bt1QuXJlhIeHy2oDAM6dO4e+ffvC3d0ddnZ28PPzwyeffALgyXVa48ePBwD4+vpCpVJBpVLh6tWr0ud//vlntGzZEnZ2dnBzc0P//v1x7do1vXnkn6I/evQogoODYW9vj48//hgAcOTIEYSGhqJq1aqws7ODr68vhg4dWuzyzm+z4HrJvy5y7dq1z13Gz3L9+nUMHToUnp6e0Gg08Pf3x08//aRX5/Hjx5gyZQpatmwJZ2dnODg4oH379ti1a1eh9nQ6HebNm4eAgADY2trC3d0dXbp0wZEjRwrV3bhxIxo3bizNd/v27c+NN7/Pq1evxqefforq1avD3t4eWq0WAJCQkIAuXbrA2dkZ9vb26NChA/bv3y99/tdff4VKpcKePXsKtf39999DpVLh9OnTAJ59zd7z1v/8+fNhbW2td2T466+/hkqlwtixY6WyvLw8VK5cGRMmTHhuv4kMVam8AyCqqHx9fTFw4ED88MMPmDhxIry9vZ9Zd/jw4VixYgX+/e9/Y9y4cUhISEBUVBSSkpKwYcMG2fP+888/0bVrV9SpUwfTpk3Dw4cPsWDBArRt2xbHjh1D7dq18c4776B69eqYMWMGRo0ahRdffBGenp7PbPPevXto3749kpKSMHToULRo0QL//PMPNm3ahL///htVq1aV6s6cORNWVlb48MMPkZWVhdmzZyM8PBwJCQlSnXXr1uHBgwd47733UKVKFRw6dAgLFizA33//jXXr1unNOzc3F6GhoWjXrh2++uor2Nvby2rj5MmTaN++PWxsbPD222+jdu3auHTpEjZv3ozp06ejd+/eOH/+PFatWoVvvvlG6ou7uzsAYPr06Zg8eTL69u2L4cOH49atW1iwYAGCg4Nx/PhxvSOjt2/fRteuXdG/f3+8+eab8PT0RHp6Ojp37gx3d3dMnDgRLi4uuHr1KtavXy973cpZxkVJS0tD69atpRsm3N3dsW3bNgwbNgxarRaRkZEAAK1Wi6VLl2LAgAEYMWIE7t69ix9//BGhoaE4dOgQmjVrJrU5bNgwLF++HF27dsXw4cORm5uLvXv34uDBgwgMDJTq7du3D+vXr8f777+PypUrY/78+ejTpw9SUlJQpUqV5/b5iy++gFqtxocffojs7Gyo1WrExsaia9euaNmyJaZOnQorKyssW7YMr7zyCvbu3YuXXnoJYWFhcHR0xNq1a6U/cPKtWbMG/v7+xV5Ha8j6b9++PXQ6Hfbt24fXXnsNwJM/RqysrLB3716prePHj+PevXsIDg5+bn+JDCaIqEwtW7ZMABCHDx8Wly5dEpUqVRKjRo2Spnfo0EH4+/tL7xMTEwUAMXz4cL12PvzwQwFAxMbGyo6hWbNmwsPDQ9y+fVsqO3HihLCyshIDBw6Uynbt2iUAiHXr1j23zSlTpggAYv369YWm6XQ6vfYaNmwosrOzpenz5s0TAMSpU6eksgcPHhRqJyoqSqhUKvHXX39JZYMGDRIAxMSJEwvVN7SN4OBgUblyZb2ygnELIcScOXMEAHHlyhW9OlevXhXW1tZi+vTpeuWnTp0SlSpV0ivv0KGDACAWL16sV3fDhg3SNiFXhw4dRIcOHaT3cpZxUYYNGyaqVasm/vnnH73y/v37C2dnZ2mZ5ubm6rUvhBB37twRnp6eYujQoVJZbGysAKC3jecruHwBCLVaLS5evCiVnThxQgAQCxYsKDbm/D7XqVNHb53rdDpRv359ERoaqjevBw8eCF9fX/Hqq69KZQMGDBAeHh4iNzdXKrt586awsrISn3/+uVQ2depUUfCn09D1n5eXJ5ycnMRHH30kxValShXx+uuvC2tra3H37l0hhBBz584VVlZW4s6dO8X2mUgOnsYlKkd16tTBW2+9hSVLluDmzZtF1vnjjz8AQO9UDwCMGzcOALB161ZZ87x58yYSExMxePBguLm5SeVNmjTBq6++Ks1Prt9++w1NmzZFr169Ck17+rTXkCFD9K4za9++PQDg8uXLUpmdnZ30//v37+Off/5BmzZtIITA8ePHC83jvffeK1RmSBu3bt1CXFwchg4dilq1ahUbd1HWr18PnU6Hvn374p9//pFeXl5eqF+/fqHTmhqNBkOGDNEryz/yt2XLFuTk5Dx3noYwZBk/TQiB3377Dd27d4cQQq8/oaGhyMrKwrFjxwAA1tbWUvs6nQ4ZGRnIzc1FYGCgVAd4sl2oVCpMnTq10PyeXr4hISGoW7eu9L5JkyZwcnIqNuaCBg0apLfOExMTceHCBbzxxhu4ffu21Jf79++jU6dOiIuLk67r69evH9LT07F7927p87/++it0Oh369ev3zHkauv6trKzQpk0bxMXFAQCSkpJw+/ZtTJw4EUIIxMfHA3hytK9x48Zmd50smTYme0Tl7NNPP0Vubu4zr93766+/YGVlhXr16umVe3l5wcXFBX/99Zes+eXX9/PzKzStYcOG0o+hXJcuXTJ4yJink6r8O3zv3LkjlaWkpEgJqaOjI9zd3aVTbFlZWXqfr1SpEmrUqFFoPoa0kZ9IlHS4mwsXLkAIgfr168Pd3V3vlZSUJN3kkq969eqFbqjo0KED+vTpg88++wxVq1ZFjx49sGzZslJdk2nIMn7arVu3kJmZiSVLlhTqS36CWrA/K1asQJMmTWBra4sqVarA3d0dW7du1Vs/ly5dgre3t94fFobGnB93cTEX5Ovrq/f+woULAJ4kgU/3Z+nSpcjOzpZizb+mb82aNdLn16xZg2bNmuGFF1545jzlrP/27dvj6NGjePjwIfbu3Ytq1aqhRYsWaNq0qXQqd9++fVJiTqQUXrNHVM7q1KmDN998E0uWLMHEiROfWc+SBnK1trYuslwIAeDJReqvvvoqMjIyMGHCBDRo0AAODg64fv06Bg8eXOguS41GAysr/b9d5bZRUjqdDiqVCtu2bSuyX46OjnrvCx55ypc/0PTBgwexefNm7NixA0OHDsXXX3+NgwcPFmrDEM9bxkXJXyZvvvkmBg0aVGSdJk2aAHhyQ8LgwYPRs2dPjB8/Hh4eHrC2tkZUVBQuXbokO96SxlzQ08s2vz9z5szRu4awoPxlq9Fo0LNnT2zYsAGLFi1CWloa9u/fjxkzZhQ7Tznrv127dsjJyUF8fDz27t0rJXXt27fH3r17ce7cOdy6dYvJHimOyR6RCfj000/x888/Y9asWYWm+fj4QKfT4cKFC2jYsKFUnpaWhszMTNmDHOfXT05OLjTt3LlzqFq1KhwcHGT2AKhbt650x2JpnTp1CufPn8eKFSswcOBAqTw6OlrxNurUqQMAz439Wcl23bp1IYSAr69vsUeADNG6dWu0bt0a06dPxy+//ILw8HCsXr0aw4cPL1W7hnJ3d0flypWRl5eHkJCQYuv++uuvqFOnDtavX6+3bJ4+XVu3bl3s2LEDGRkZBh3dU1L+KWEnJ6fn9gd4cip3xYoViImJQVJSEoQQxZ7CzZ+Hoev/pZdeglqtxt69e7F3717pDu/g4GD88MMPiImJkd4TKYmncYlMQN26dfHmm2/i+++/R2pqqt60bt26AQC+/fZbvfK5c+cCAMLCwqSyS5cuPfeoSrVq1dCsWTOsWLFCbxiI06dPY+fOndL85OrTpw9OnDhR5N3Bhh6ZyZd/hKTg54QQmDdvnuJtuLu7Izg4GD/99BNSUlKeGXd+Avz0oMq9e/eGtbU1Pvvss0L9FELg9u3bz431zp07hT6bfySqNKdy5bK2tkafPn3w22+/FZn83rp1S68uoL+MEhISpGvP8vXp0wdCCHz22WeF2pO7XcjVsmVL1K1bF1999RXu3btXaHrB/gBPrhl0c3PDmjVrsGbNGrz00kuFTg0/Tc76t7W1xYsvvohVq1YhJSVF78jew4cPMX/+fNStWxfVqlUraZeJisQje0Qm4pNPPsH//M//IDk5Gf7+/lJ506ZNMWjQICxZsgSZmZno0KEDDh06hBUrVqBnz554+eWXpbqdOnUCAL3x34oyZ84cdO3aFUFBQRg2bJg09IqzszOmTZtWovjHjx+PX3/9Fa+//jqGDh2Kli1bIiMjA5s2bcLixYvRtGlTg9tq0KAB6tatiw8//BDXr1+Hk5MTfvvtN4Ov3ZLbxvz589GuXTu0aNECb7/9Nnx9fXH16lVs3bpVehZwy5YtATxZT/3794eNjQ26d++OunXr4ssvv8SkSZNw9epV9OzZE5UrV8aVK1ewYcMGvP322/jwww+LjXXFihVYtGgRevXqhbp16+Lu3bv44Ycf4OTkVOLku6RmzpyJXbt2oVWrVhgxYgQaNWqEjIwMHDt2DH/++ScyMjIAAK+99hrWr1+PXr16ISwsDFeuXMHixYvRqFEjvcTq5ZdfxltvvYX58+fjwoUL6NKlC3Q6Hfbu3YuXX37ZqM/DtbKywtKlS9G1a1f4+/tjyJAhqF69Oq5fv45du3bByckJmzdvlurb2Nigd+/eWL16Ne7fv4+vvvrqufOQu/7bt2+PmTNnwtnZGQEBAQAADw8P+Pn5ITk5GYMHD1Z8ORBx6BWiMlZw6JWn5Q8jUnDoFSGEyMnJEZ999pnw9fUVNjY2ombNmmLSpEni0aNHevV8fHyEj4+PQXH8+eefom3btsLOzk44OTmJ7t27i7Nnz+rVkTP0ihBC3L59W4wcOVJUr15dqNVqUaNGDTFo0CBpGI9ntXflyhUBQCxbtkwqO3v2rAgJCRGOjo6iatWqYsSIEdJQHAXrDRo0SDg4OBQZj6FtCCHE6dOnRa9evYSLi4uwtbUVfn5+YvLkyXp1vvjiC1G9enVhZWVVaBiW3377TbRr1044ODgIBwcH0aBBAxERESGSk5OlOk8Pq5Pv2LFjYsCAAaJWrVpCo9EIDw8P8dprr4kjR44Ut7ilNosaesWQZfwsaWlpIiIiQtSsWVPY2NgILy8v0alTJ7FkyRKpjk6nEzNmzBA+Pj5Co9GI5s2biy1btohBgwYV2gZzc3PFnDlzRIMGDYRarRbu7u6ia9eu4ujRo1IdACIiIqJQLD4+PmLQoEHFxvu87fT48eOid+/eokqVKkKj0QgfHx/Rt29fERMTU6hudHS0ACBUKpW4du1aoelPD72Sz5D1L4QQW7duFQBE165d9cqHDx8uAIgff/yx2L4SlYRKCCMfRyciIiKicsNr9oiIiIgsGJM9IiIiIgvGZI+IiIjIgjHZIyIiIrJgTPaIiIiILBiTPSIiIiILxkGVFaLT6XDjxg1UrlzZop5hSkRERKZJCIG7d+/C29u70PPBC2Kyp5AbN26gZs2a5R0GERERVTDXrl1DjRo1njmdyZ5CKleuDODJAndycirnaIiIiMjSabVa1KxZU8pBnoXJnkLyT906OTkx2SMiIqIy87zLx3iDBhEREZEFY7JHREREZMGY7BERERFZMCZ7RERERBaMyR4RERGRBWOyR0RERGTBmOwRERERWTAme0REREQWjMkeERERkQVjskdERERkwZjsEREREVkwJntEREREFozJHhEREZEFY7JHREREZMGY7BERERFZMCZ7RERERBaMyR4RERGRBWOyR0RERGTBmOwRERERWTAme0REREQWjMkeERERkQVjskdERERkwZjsEREREVkwJntEREREFqxck724uDh0794d3t7eUKlU2Lhxo950IQSmTJmCatWqwc7ODiEhIbhw4YJenYyMDISHh8PJyQkuLi4YNmwY7t27p1fn5MmTaN++PWxtbVGzZk3Mnj27UCzr1q1DgwYNYGtri4CAAPzxxx+K95eIiIiorJVrsnf//n00bdoUCxcuLHL67NmzMX/+fCxevBgJCQlwcHBAaGgoHj16JNUJDw/HmTNnEB0djS1btiAuLg5vv/22NF2r1aJz587w8fHB0aNHMWfOHEybNg1LliyR6hw4cAADBgzAsGHDcPz4cfTs2RM9e/bE6dOnjdd5IiIiorIgTAQAsWHDBum9TqcTXl5eYs6cOVJZZmam0Gg0YtWqVUIIIc6ePSsAiMOHD0t1tm3bJlQqlbh+/boQQohFixYJV1dXkZ2dLdWZMGGC8PPzk9737dtXhIWF6cXTqlUr8c477xgcf1ZWlgAgsrKyDP4MERERUUkZmnuY7DV7V65cQWpqKkJCQqQyZ2dntGrVCvHx8QCA+Ph4uLi4IDAwUKoTEhICKysrJCQkSHWCg4OhVqulOqGhoUhOTsadO3ekOgXnk18nfz5Fyc7Ohlar1XsRERERmRqTTfZSU1MBAJ6ennrlnp6e0rTU1FR4eHjoTa9UqRLc3Nz06hTVRsF5PKtO/vSiREVFwdnZWXrVrFlTbheJiIiIjM5kkz1TN2nSJGRlZUmva9eulXdIRERERIWYbLLn5eUFAEhLS9MrT0tLk6Z5eXkhPT1db3pubi4yMjL06hTVRsF5PKtO/vSiaDQaODk56b2IiIhIBpWqvCOoEEw22fP19YWXlxdiYmKkMq1Wi4SEBAQFBQEAgoKCkJmZiaNHj0p1YmNjodPp0KpVK6lOXFwccnJypDrR0dHw8/ODq6urVKfgfPLr5M+HiIiIyFyVa7J37949JCYmIjExEcCTmzISExORkpIClUqFyMhIfPnll9i0aRNOnTqFgQMHwtvbGz179gQANGzYEF26dMGIESNw6NAh7N+/HyNHjkT//v3h7e0NAHjjjTegVqsxbNgwnDlzBmvWrMG8efMwduxYKY7Ro0dj+/bt+Prrr3Hu3DlMmzYNR44cwciRI8t6kRAREREpq4zuDi7Srl27BIBCr0GDBgkhngy/MnnyZOHp6Sk0Go3o1KmTSE5O1mvj9u3bYsCAAcLR0VE4OTmJIUOGiLt37+rVOXHihGjXrp3QaDSievXqYubMmYViWbt2rXjhhReEWq0W/v7+YuvWrbL6wqFXiIiIZDKdEeDMkqG5h0oIIcox17QYWq0Wzs7OyMrK4vV7REREhlCpAKYhJWZo7mGy1+wRERERUekx2SMiIiKyYEz2iIiIiCwYkz0iIiIiC8Zkj4iIiMiCMdkjIiIismBM9oiIiIgsGJM9IiIiIgvGZI+IiIjIgjHZIyIiIrJgTPaIiIiILBiTPSIiIiILxmSPiIiIyIIx2SMiIiKyYEz2iIiIiCwYkz0iIiIiC8Zkj4iIiMiCMdkjIiIismBM9oiIiIgsGJM9IiIiIgvGZI+IiIjIgjHZIyIiIrJgTPaIiIiILBiTPSIiIiILxmSPiIiIyIIx2SMiIiKyYLKTvSlTpmDXrl149OiRMeIhIiIiIgXJTvbi4+PRvXt3uLi4oH379vj000/x559/4uHDh8aIj4iIiIhKQXayFx0djczMTMTExKBbt244cuQIevfuDRcXF7Rr184YMRIRERFRCVUq0YcqVULbtm3h7u4ONzc3VK5cGRs3bsS5c+eUjo+IiIiISkH2kb0lS5bgjTfeQPXq1dGmTRts374d7dq1w5EjR3Dr1i1jxEhEREREJST7yN67774Ld3d3jBs3Du+//z4cHR2NERcRERERKUD2kb3169cjPDwcq1evhru7O9q0aYOPP/4YO3fuxIMHD4wRIxERERGVkEoIIUr64aysLOzduxfr1q3DqlWrYGVlVWGHZNFqtXB2dkZWVhacnJzKOxwiIiLTp1IBJU9DKjxDc48S3aBx+/Zt7NmzB7t378bu3btx5swZuLq6on379iUOmIiIiIiUJzvZCwgIQFJSElxdXREcHIwRI0agQ4cOaNKkiTHiIyIiIqJSKNENGh06dEDjxo2NEQ8RERERKUh2shcREQEAePz4Ma5cuYK6deuiUqUSnQ0mIiIiIiOTfTfuw4cPMWzYMNjb28Pf3x8pKSkAgA8++AAzZ85UPEAiIiIiKjnZyd7EiRNx4sQJ7N69G7a2tlJ5SEgI1qxZo2hwRERERFQ6ss+/bty4EWvWrEHr1q2hUqmkcn9/f1y6dEnR4IiIiIiodGQf2bt16xY8PDwKld+/f18v+SMiIiKi8ic72QsMDMTWrVul9/kJ3tKlSxEUFKRcZERERERUarJP486YMQNdu3bF2bNnkZubi3nz5uHs2bM4cOAA9uzZY4wYiYiIiKiEZB/Za9euHRITE5Gbm4uAgADs3LkTHh4eiI+PR8uWLY0RIxERERGVUKmejUv/j8/GJSIikonPxi0VRZ+Nq9VqpUa0Wm2xdZnoEBEREZkOg5I9V1dX3Lx5Ex4eHnBxcSnyrlshBFQqFfLy8hQPkoiIiIhKxqBkLzY2Fm5ubtL/OcQKERERkXngNXsK4TV7REREMvGavVIxNPeQfTdu/fr1MW3aNFy4cKFUARIRERGR8clO9t5//31s3boVDRo0wIsvvoh58+YhNTXVGLERERERUSnJTvbGjBmDw4cPIykpCd26dcPChQtRs2ZNdO7cGf/973+NESMRERERlZAi1+wdPHgQ7733Hk6ePFlh78blNXtEREQy8Zq9UlF0nL1nOXToEH755ResWbMGWq0Wr7/+emmaIyIiIiKFyU72zp8/j5UrV2LVqlW4cuUKXnnlFcyaNQu9e/eGo6OjMWIkIiIiohKSnezl35gRERGB/v37w9PT0xhxEREREZECZCd7ycnJqF+/vjFiISIiIiKFlWicvczMTCxduhSTJk1CRkYGAODYsWO4fv264gESERERUcnJPrJ38uRJdOrUCS4uLrh69SpGjBgBNzc3rF+/HikpKRx+hYiIiMiElGicvSFDhuDChQuwtbWVyrt164a4uDhFg8vLy8PkyZPh6+sLOzs71K1bF1988QUKjhYjhMCUKVNQrVo12NnZISQkpNDTPTIyMhAeHg4nJye4uLhg2LBhuHfvnl6dkydPon379rC1tUXNmjUxe/ZsRftCREREVB5kJ3tHjhzBO++8U6i8evXqij9JY9asWfjuu+/wn//8B0lJSZg1axZmz56NBQsWSHVmz56N+fPnY/HixUhISICDgwNCQ0Px6NEjqU54eDjOnDmD6OhobNmyBXFxcXj77bel6VqtFp07d4aPjw+OHj2KOXPmYNq0aViyZImi/SEiIiIqa7JP42o0Gmi12kLl58+fh7u7uyJB5Ttw4AB69OiBsLAwAEDt2rWxatUqHDp0CMCTo3rffvstPv30U/To0QMA8N///heenp7YuHEj+vfvj6SkJGzfvh2HDx9GYGAgAGDBggXo1q0bvvrqK3h7e2PlypV4/PgxfvrpJ6jVavj7+yMxMRFz587VSwqJiIiIzI3sI3v/+te/8PnnnyMnJwcAoFKpkJKSggkTJqBPnz6KBtemTRvExMTg/PnzAIATJ05g37596Nq1KwDgypUrSE1NRUhIiPQZZ2dntGrVCvHx8QCA+Ph4uLi4SIkeAISEhMDKygoJCQlSneDgYKjVaqlOaGgokpOTcefOnSJjy87Ohlar1XsRERERmRrZyd7XX3+Ne/fuwcPDAw8fPkSHDh1Qr149VK5cGdOnT1c0uIkTJ6J///5o0KABbGxs0Lx5c0RGRiI8PBwApNPGT4/15+npKU1LTU2Fh4eH3vRKlSrBzc1Nr05RbRScx9OioqLg7OwsvWrWrFnK3hIREREpT/ZpXGdnZ0RHR2P//v04ceIE7t27hxYtWugdXVPK2rVrsXLlSvzyyy/SqdXIyEh4e3tj0KBBis9PjkmTJmHs2LHSe61Wy4SPiIiITI6sZC8nJwd2dnZITExE27Zt0bZtW2PFBQAYP368dHQPAAICAvDXX38hKioKgwYNgpeXFwAgLS0N1apVkz6XlpaGZs2aAQC8vLyQnp6u125ubi4yMjKkz3t5eSEtLU2vTv77/DpP02g00Gg0pe8kERERkRHJOo1rY2ODWrVqIS8vz1jx6Hnw4AGsrPRDtLa2hk6nAwD4+vrCy8sLMTEx0nStVouEhAQEBQUBAIKCgpCZmYmjR49KdWJjY6HT6dCqVSupTlxcnHQdIgBER0fDz88Prq6uRusfERERkbHJvmbvk08+wccffyw9OcOYunfvjunTp2Pr1q24evUqNmzYgLlz56JXr14AntwcEhkZiS+//BKbNm3CqVOnMHDgQHh7e6Nnz54AgIYNG6JLly4YMWIEDh06hP3792PkyJHo378/vL29AQBvvPEG1Go1hg0bhjNnzmDNmjWYN2+e3mlaIiIiInOkEgVHKDZA8+bNcfHiReTk5MDHxwcODg56048dO6ZYcHfv3sXkyZOxYcMGpKenw9vbGwMGDMCUKVOkO2eFEJg6dSqWLFmCzMxMtGvXDosWLcILL7wgtZORkYGRI0di8+bNsLKyQp8+fTB//nw4OjpKdU6ePImIiAgcPnwYVatWxQcffIAJEyYYHKtWq4WzszOysrLg5OSk2DIgIiKyWCoVIC8NoQIMzT1kJ3ufffZZsdOnTp0qpzmLwWSPiIhIJiZ7pWK0ZI+KxmSPiIhIJiZ7pWJo7iH7mj0iIiIiMh9M9oiIiIgsGJM9IiIiIgvGZI+IiIjIgjHZIyIiIrJgsp+Nm5eXh+XLlyMmJgbp6enS0yzyxcbGKhYcEREREZWO7GRv9OjRWL58OcLCwtC4cWOoVCpjxEVERERECpCd7K1evRpr165Ft27djBEPERERESlI9jV7arUa9erVM0YsRERERKQw2cneuHHjMG/ePPDBG0RERESmz6DTuL1799Z7Hxsbi23btsHf3x82NjZ609avX69cdERERERUKgYle87Oznrve/XqZZRgiIiIiEhZBiV7y5YtM3YcRERERGQEsq/Zu3LlCi5cuFCo/MKFC7h69aoSMRERERGRQmQne4MHD8aBAwcKlSckJGDw4MFKxERERERECpGd7B0/fhxt27YtVN66dWskJiYqERMRERERKUR2sqdSqXD37t1C5VlZWcjLy1MkKCIiIiJShuxkLzg4GFFRUXqJXV5eHqKiotCuXTtFgyMiIiKi0pH9uLRZs2YhODgYfn5+aN++PQBg79690Gq1iI2NVTxAIiIiIio52Uf2GjVqhJMnT6Jv375IT0/H3bt3MXDgQJw7dw6NGzc2RoxEREREVEIqweeeKUKr1cLZ2RlZWVlwcnIq73CIiIhMn0oFMA0pMUNzD9mncfM9ePAAKSkpePz4sV55kyZNStokERERESlMdrJ369YtDBkyBNu2bStyOu/IJSIiIjIdsq/Zi4yMRGZmJhISEmBnZ4ft27djxYoVqF+/PjZt2mSMGImIiIiohGQf2YuNjcXvv/+OwMBAWFlZwcfHB6+++iqcnJwQFRWFsLAwY8RJRERERCUg+8je/fv34eHhAQBwdXXFrVu3AAABAQE4duyYstERERERUanITvb8/PyQnJwMAGjatCm+//57XL9+HYsXL0a1atUUD5CIiIiISk72adzRo0fj5s2bAICpU6eiS5cuWLlyJdRqNZYvX650fERERERUCqUeZ+/Bgwc4d+4catWqhapVqyoVl9nhOHtEREQycZy9UjE095B9Gjff48ePkZycDLVajRYtWlToRI+IiIjIVMlO9h48eIBhw4bB3t4e/v7+SElJAQB88MEHmDlzpuIBEhEREVHJyU72Jk2ahBMnTmD37t2wtbWVykNCQrBmzRpFgyMiIiKi0pF9g8bGjRuxZs0atG7dGiqVSir39/fHpUuXFA2OiIiIiEpH9pG9W7duSePsFXT//n295I+IiIiIyp/sZC8wMBBbt26V3ucneEuXLkVQUJBykRERERFRqck+jTtjxgx07doVZ8+eRW5uLubNm4ezZ8/iwIED2LNnjzFiJCIiIqISkn1kr127dkhMTERubi4CAgKwc+dOeHh4ID4+Hi1btjRGjERERERUQqUeVJme4KDKREREMnFQ5VIx+qDKRERERGT6mOwRERERWTAme0REREQWjMkeERERkQVjskdERERkwWSPs3f//n3MnDkTMTExSE9Ph06n05t++fJlxYIjIiIiotKRnewNHz4ce/bswVtvvYVq1arxEWlEREREJkx2srdt2zZs3boVbdu2NUY8RERERKQg2dfsubq6ws3NzRixEBEREZHCZCd7X3zxBaZMmYIHDx4YIx4iIiIiUpDs07hff/01Ll26BE9PT9SuXRs2NjZ6048dO6ZYcERERERUOrKTvZ49exohDCIiIiIyBpUQfAKxEgx9GDERERH9H5UKYBpSYobmHhxUmYiIiMiCGXQa183NDefPn0fVqlXh6upa7Nh6GRkZigVHRERERKVjULL3zTffoHLlygCAb7/91pjxEBEREZGCeM2eQnjNHhERkUy8Zq9UeM0eERERETHZIyIiIrJkTPaIiIiILBiTPSIiIiILVupkT6vVYuPGjUhKSlIinkKuX7+ON998E1WqVIGdnR0CAgJw5MgRaboQAlOmTEG1atVgZ2eHkJAQXLhwQa+NjIwMhIeHw8nJCS4uLhg2bBju3bunV+fkyZNo3749bG1tUbNmTcyePdso/SEiIiIqS7KTvb59++I///kPAODhw4cIDAxE37590aRJE/z222+KBnfnzh20bdsWNjY22LZtG86ePYuvv/4arq6uUp3Zs2dj/vz5WLx4MRISEuDg4IDQ0FA8evRIqhMeHo4zZ84gOjoaW7ZsQVxcHN5++21pularRefOneHj44OjR49izpw5mDZtGpYsWaJof4iIiIjKnJDJ09NTJCYmCiGEWLlypahXr564f/++WLRokWjWrJnc5oo1YcIE0a5du2dO1+l0wsvLS8yZM0cqy8zMFBqNRqxatUoIIcTZs2cFAHH48GGpzrZt24RKpRLXr18XQgixaNEi4erqKrKzs/Xm7efn98x5P3r0SGRlZUmva9euCQAiKyurxP0lIiKqUOSnIVRAVlaWQbmH7CN7WVlZcHNzAwBs374dffr0gb29PcLCwgqdPi2tTZs2ITAwEK+//jo8PDzQvHlz/PDDD9L0K1euIDU1FSEhIVKZs7MzWrVqhfj4eABAfHw8XFxcEBgYKNUJCQmBlZUVEhISpDrBwcFQq9VSndDQUCQnJ+POnTtFxhYVFQVnZ2fpVbNmTUX7TkRERKQE2clezZo1ER8fj/v372P79u3o3LkzgCenXG1tbRUN7vLly/juu+9Qv3597NixA++99x5GjRqFFStWAABSU1MBAJ6ennqf8/T0lKalpqbCw8NDb3qlSpXg5uamV6eoNgrO42mTJk1CVlaW9Lp27Vope0tERESkPIMel1ZQZGQkwsPD4ejoCB8fH3Ts2BEAEBcXh4CAAEWD0+l0CAwMxIwZMwAAzZs3x+nTp7F48WIMGjRI0XnJpdFooNFoyjUGIiKTxqcjEJkE2Uf23n//fRw8eBA//fQT9u3bByurJ03UqVMHX375paLBVatWDY0aNdIra9iwIVJSUgAAXl5eAIC0tDS9OmlpadI0Ly8vpKen603Pzc1FRkaGXp2i2ig4DyIiIiJzJCvZy8nJQd26dWFvb49evXrB0dFRmhYWFoa2bdsqGlzbtm2RnJysV3b+/Hn4+PgAAHx9feHl5YWYmBhpularRUJCAoKCggAAQUFByMzMxNGjR6U6sbGx0Ol0aNWqlVQnLi4OOTk5Up3o6Gj4+fnp3flLREREZG5kJXs2NjZ6Q5oY25gxY3Dw4EHMmDEDFy9exC+//IIlS5YgIiICAKBSqRAZGYkvv/wSmzZtwqlTpzBw4EB4e3ujZ8+eAJ4cCezSpQtGjBiBQ4cOYf/+/Rg5ciT69+8Pb29vAMAbb7wBtVqNYcOG4cyZM1izZg3mzZuHsWPHlllfiYiIiIxC7m2+06dPF4MGDRI5OTklvlVYjs2bN4vGjRsLjUYjGjRoIJYsWaI3XafTicmTJwtPT0+h0WhEp06dRHJysl6d27dviwEDBghHR0fh5OQkhgwZIu7evatX58SJE6Jdu3ZCo9GI6tWri5kzZ8qK09Dbn4mIKgwOq0HPw22kVAzNPVRCyLt6tlevXoiJiYGjoyMCAgLg4OCgN339+vUKpqLmQ6vVwtnZGVlZWXBycirvcIiIyh9v0KDn4TZSKobmHrLvxnVxcUGfPn1KFRwRERERlQ3Zyd6yZcuMEQcRERERGYHsoVeAJ0OX/Pnnn/j+++9x9+5dAMCNGzdw7949RYMjIiIiotKRfWTvr7/+QpcuXZCSkoLs7Gy8+uqrqFy5MmbNmoXs7GwsXrzYGHESERERUQnIPrI3evRoBAYG4s6dO7Czs5PK82/cICIiIiLTIfvI3t69e3HgwAGo1Wq98tq1a+P69euKBUZEREREpSf7yJ5Op0NeXl6h8r///huVK1dWJCgiIiIiUobsZK9z58749ttvpfcqlQr37t3D1KlT0a1bNyVjIyIiIqJSkj2o8t9//43Q0FAIIXDhwgUEBgbiwoULqFq1KuLi4uDh4WGsWE0aB1UmInoKB8yl5+E2UiqG5h6ykz3gydAra9aswYkTJ3Dv3j20aNEC4eHhejdsVDRM9oiInsIfcnoebiOlYrRkLy4uDm3atEGlSvr3duTm5uLAgQMIDg4uWcRmjskeGYw7N6oouK3T83AbKRVDcw/Z1+y9/PLLyMjIKFSelZWFl19+WW5zRERERGREspM9IQRUKlWh8tu3b8PBwUGRoIiIiIhIGQaPs9e7d28AT+6+HTx4MDQajTQtLy8PJ0+eRJs2bZSPkIiIiIhKzOBkz9nZGcCTI3uVK1fWuxlDrVajdevWGDFihPIREhEREVGJGZzsLVu2DMCTJ2WMHz8e9vb2RguKiIiILAxvxig3sq/ZGzhwYJGPRbtw4QKuXr2qRExEREREpBDZyd7gwYNx4MCBQuUJCQkYPHiwEjERERERkUJkJ3vHjx9H27ZtC5W3bt0aiYmJSsRERERERAqRneypVCrcvXu3UHlWVhby8vIUCYoUVsRQOURERFQxyE72goODERUVpZfY5eXlISoqCu3atVM0OCIiIiIqHYPvxs03a9YsBAcHw8/PD+3btwcA7N27F1qtFrGxsYoHSEREREQlJ/vIXqNGjXDy5En07dsX6enpuHv3LgYOHIhz586hcePGxoiRiIiIiEpIJQQHvVGCoQ8jLhcc28i0cH1QRVFW2zq/U+ahqPXEdVcqhuYesk/j5nvw4AFSUlLw+PFjvfImTZqUtEkiIiIiUpjsZO/WrVsYMmQItm3bVuR03pFLREREZDpkX7MXGRmJzMxMJCQkwM7ODtu3b8eKFStQv359bNq0yRgxEhEREVEJyT6yFxsbi99//x2BgYGwsrKCj48PXn31VTg5OSEqKgphYWHGiJOIiIiISkD2kb379+/Dw8MDAODq6opbt24BAAICAnDs2DFloyMiIiKiUpGd7Pn5+SE5ORkA0LRpU3z//fe4fv06Fi9ejGrVqikeIBERERGVnOzTuKNHj8bNmzcBAFOnTkWXLl2wcuVKqNVqLF++XOn4iIiIiKgUSj3O3oMHD3Du3DnUqlULVatWVSous8Nx9shgXB9UUXCcPSqI4+wpztDcQ9Zp3JycHNStWxdJSUlSmb29PVq0aFGhEz0iIiIiUyUr2bOxscGjR4+MFQuVN5WqvCMgIiIihcm+QSMiIgKzZs1Cbm6uMeIhIiIiIgXJvkHj8OHDiImJwc6dOxEQEAAHBwe96evXr1csOCIiIiIqHdnJnouLC/r06WOMWIiIiIhIYbKTvWXLlhkjDiIiIiIyAtnX7BERERGR+ZB9ZA8Afv31V6xduxYpKSl4/Pix3jQ+Mo2IiIjIdMg+sjd//nwMGTIEnp6eOH78OF566SVUqVIFly9fRteuXY0RIxERERGVkOxkb9GiRViyZAkWLFgAtVqNjz76CNHR0Rg1ahSysrKMESMRERERlZDsZC8lJQVt2rQBANjZ2eHu3bsAgLfeegurVq1SNjoiIiIiKhXZyZ6XlxcyMjIAALVq1cLBgwcBAFeuXEEpH7NLRERERAqTney98sor2LRpEwBgyJAhGDNmDF599VX069cPvXr1UjxAIiIiIio5lZB5OE6n00Gn06FSpSc38q5evRoHDhxA/fr18c4770CtVhslUFOn1Wrh7OyMrKwsODk5lXc4+lQqwJDVbGg9Kh0uZ6ooympb53fKPBS1nrjuSsXQ3EN2skdFY7JnQky9H6YeH5FSmOxRQUz2FGdo7lGicfYyMzNx6NAhpKenQ6fT6U0bOHBgSZokIiIiIiOQnext3rwZ4eHhuHfvHpycnKBSqaRpKpWKyV5Fxr/QiIiITI7sGzTGjRuHoUOH4t69e8jMzMSdO3ekV/5dukRERERkGmQne9evX8eoUaNgb29vjHiIiIiISEGyk73Q0FAcOXLEGLEQERERkcIMumYvf1w9AAgLC8P48eNx9uxZBAQEwMbGRq/uv/71L2UjJCIiIqISM2joFSsrww4AqlQq5OXllTooc8ShVxT4vFJMJY5nMfX4KgquB+Pj0CtUEIdeUZyiQ688PbwKERERKYDJDpUB2dfsEREREZH5MDjZi42NRaNGjaDVagtNy8rKgr+/P+Li4hQNjoiIiIhKx+Bk79tvv8WIESOKPCfs7OyMd955B998842iwREREZGCCjwIgSoOg5O9EydOoEuXLs+c3rlzZxw9elSRoIiI6Cn8kSaiEjI42UtLSys0zEpBlSpVwq1btxQJ6llmzpwJlUqFyMhIqezRo0eIiIhAlSpV4OjoiD59+iAtLU3vcykpKQgLC4O9vT08PDwwfvx45Obm6tXZvXs3WrRoAY1Gg3r16mH58uVG7QsRERFRWTA42atevTpOnz79zOknT55EtWrVFAmqKIcPH8b333+PJk2a6JWPGTMGmzdvxrp167Bnzx7cuHEDvXv3lqbn5eUhLCwMjx8/xoEDB7BixQosX74cU6ZMkepcuXIFYWFhePnll5GYmIjIyEgMHz4cO3bsMFp/iIiIiMqEMNDIkSNF48aNxcOHDwtNe/DggWjcuLH44IMPDG1Olrt374r69euL6Oho0aFDBzF69GghhBCZmZnCxsZGrFu3TqqblJQkAIj4+HghhBB//PGHsLKyEqmpqVKd7777Tjg5OYns7GwhhBAfffSR8Pf315tnv379RGhoqMExZmVlCQAiKyurpN00HkNXs+Gbg3E+r5Sn4zCVuPKZWjwVlbmtB3OLV4iyi9kcl02+so69PJdVUfM253VnAgzNPQw+svfpp58iIyMDL7zwAmbPno3ff/8dv//+O2bNmgU/Pz9kZGTgk08+MUpCGhERgbCwMISEhOiVHz16FDk5OXrlDRo0QK1atRAfHw8AiI+PR0BAADw9PaU6oaGh0Gq1OHPmjFTn6bZDQ0OlNoqSnZ0NrVar9yIiIiIyNQYNqgwAnp6eOHDgAN577z1MmjQJ4v8GgVSpVAgNDcXChQv1EiqlrF69GseOHcPhw4cLTUtNTYVarYaLi0uhWFNTU6U6T8eV//55dbRaLR4+fAg7O7tC846KisJnn31W4n4RERERlQWDkz0A8PHxwR9//IE7d+7g4sWLEEKgfv36cHV1NUpw165dw+jRoxEdHQ1bW1ujzKOkJk2ahLFjx0rvtVotatasWY4RERERERUmK9nL5+rqihdffFHpWAo5evQo0tPT0aJFC6ksLy8PcXFx+M9//oMdO3bg8ePHyMzM1Du6l5aWBi8vLwCAl5cXDh06pNdu/t26Bes8fQdvWloanJycijyqBwAajQYajabUfSQiIiIyJpN+XFqnTp1w6tQpJCYmSq/AwECEh4dL/7exsUFMTIz0meTkZKSkpCAoKAgAEBQUhFOnTiE9PV2qEx0dDScnJzRq1EiqU7CN/Dr5bRARERGZqxId2SsrlStXRuPGjfXKHBwcUKVKFal82LBhGDt2LNzc3ODk5IQPPvgAQUFBaN26NYAngz03atQIb731FmbPno3U1FR8+umniIiIkI7Mvfvuu/jPf/6Djz76CEOHDkVsbCzWrl2LrVu3lm2HiYiIiBRm0smeIb755htYWVmhT58+yM7ORmhoKBYtWiRNt7a2xpYtW/Dee+8hKCgIDg4OGDRoED7//HOpjq+vL7Zu3YoxY8Zg3rx5qFGjBpYuXYrQ0NDy6BIRERGRYlQi/7baYrRo0QIxMTFwdXXF559/jg8//BD29vZlEZ/Z0Gq1cHZ2RlZWVpHPDy5XKhXw/NVseD1jfV4pT8dhKnHlM7V4KqqC68Ec1ok5xPi0sorZHJdNvrKOvTyXVVHzNud1ZwIMzT0MumYvKSkJ9+/fBwB89tlnuHfvnjJREhEREZFRGXQat1mzZhgyZAjatWsHIQS++uorODo6Flm34GPIiIiIiKh8GXQaNzk5GVOnTsWlS5dw7NgxNGrUCJUqFc4TVSoVjh07ZpRATR1P4yrweaXwNC4ZgqdxjY+ncZ+Pp3HNd92ZAENzD4OSvYKsrKyQmpoKDw+PUgdpSZjsKfB5pTDZI0Mw2TM+JnvPx2TPfNedCTA095B9N65OpytVYERERERUdko09MqlS5fw7bffIikpCQDQqFEjjB49GnXr1lU0OCIqY/wrm4jI4sh+gsaOHTvQqFEjHDp0CE2aNEGTJk2QkJAAf39/REdHGyNGIiIiIioh2dfsNW/eHKGhoZg5c6Ze+cSJE7Fz507eoMFr9kr+eaXwmr2SM+XYlMZr9oyP1+w9H6/ZMzwec17PRqLoOHsFJSUlYdiwYYXKhw4dirNnz8ptjoiIqGJTqco7ArJwspM9d3d3JCYmFipPTEzkHbpEREREJkb2DRojRozA22+/jcuXL6NNmzYAgP3792PWrFkYO3as4gESERERUcnJTvYmT56MypUr4+uvv8akSZMAAN7e3pg2bRpGjRqleIBEREREVHKyb9Ao6O7duwCAypUrKxaQueINGgp8Xim8QaPkTDk2pfEGDePjDRrPl3+9XlnFzxs0LIrRBlUuiEkeERERkWmTfYMGERGZGd7tSVShMdkjIiIismBM9oiIiJ6FR0XJAshK9nJyctCpUydcuHDBWPGQqTKlHZ4pxUJERGTiZCV7NjY2OHnypLFiISIiKn/8g5IsjOzTuG+++SZ+/PFHY8RCRERERAqTPfRKbm4ufvrpJ/z5559o2bIlHBwc9KbPnTtXseCIiIiIqHRkJ3unT59GixYtAADnz5/Xm6bioW8iIiIikyI72du1a5cx4iAiIiIiIyjx0CsXL17Ejh078PDhQwBAKZ66RkREpoZnaogshuxk7/bt2+jUqRNeeOEFdOvWDTdv3gQADBs2DOPGjVM8QCL6P/zxJSKiEpCd7I0ZMwY2NjZISUmBvb29VN6vXz9s375d0eCoCPzBLxkuNyIiqqBkX7O3c+dO7NixAzVq1NArr1+/Pv766y/FAqMyoFIBPP1ORMZkqfsZS+0XWSTZR/bu37+vd0QvX0ZGBjQajSJBERGZBB4RJiILIDvZa9++Pf773/9K71UqFXQ6HWbPno2XX35Z0eCIiIiIqHRkn8adPXs2OnXqhCNHjuDx48f46KOPcObMGWRkZGD//v3GiJGIiIiISkj2kb3GjRvj/PnzaNeuHXr06IH79++jd+/eOH78OOrWrWuMGImIiIiohGQf2QMAZ2dnfPLJJ0rHQkREREQKK1Gyd+fOHfz4449ISkoCADRq1AhDhgyBm5ubosERERERlSsLuPNa9mncuLg41K5dG/Pnz8edO3dw584dzJ8/H76+voiLizNGjERERERUQrKP7EVERKBfv3747rvvYG1tDQDIy8vD+++/j4iICJw6dUrxIImIiIioZGQf2bt48SLGjRsnJXoAYG1tjbFjx+LixYuKBkdEREREpSM72WvRooV0rV5BSUlJaNq0qSJBEREREZEyDDqNe/LkSen/o0aNwujRo3Hx4kW0bt0aAHDw4EEsXLgQM2fONE6UREREZJos4AYGS6cS4vlryMrKCiqVCs+rqlKpkJeXp1hw5kSr1cLZ2RlZWVlwcnIy3oxK8qV61meeLi+ubUPmW9rPG+p5bRXVL8B0dkYlXRaGfq40y7oi7bQL9tXQ70h5Kuv1qkTfy+q7p/R6Kstto6z3T8bYpkuzb5ITT3l9H01pP/AUQ3MPg47sXblyRbHAiMhE8TmwZG5M+EeYyJQYlOz5+PgYOw4iIiIiMoISDap848YN7Nu3D+np6dDpdHrTRo0apUhgRERUTniUl8iiyE72li9fjnfeeQdqtRpVqlSBqsBOQaVSMdkjIiIiMiGyk73JkydjypQpmDRpEqysZI/cQkRkHLx+i4ioSLKztQcPHqB///5M9IiIiIjMgOyMbdiwYVi3bp0xYiFLwmt+iMoWv3NE9AwGjbNXUF5eHl577TU8fPgQAQEBsLGx0Zs+d+5cRQM0FxxnD8WPTcVx9v6fqY6zZ2rLSS65/ba0cfaUHOdSqW3B2NtUfr84zp68+XGcPXlMaT/wFEXH2SsoKioKO3bsgJ+fHwAUukGDiIiIiEyH7GTv66+/xk8//YTBgwcbIRwyKSb81wxRhcI/pImoFGRfs6fRaNC2bVtjxEJERERECpOd7I0ePRoLFiwwRixERFQeeOSQyKLJPo176NAhxMbGYsuWLfD39y90g8b69esVC46KwVOsRFTWuN8hMkuykz0XFxf07t3bGLEQERFVDBUtcebR43IlO9lbtmyZMeIgIlJORfshJaLSs+D9Bh+DQURkKB6dICIzJPvInq+vb7Hj6V2+fLlUAVE5MfcfMQv+i4yIiKg0ZCd7kZGReu9zcnJw/PhxbN++HePHj1cqLiIyF8Z6igERESlCdrI3evToIssXLlyII0eOlDogsgCmcJSQiQcREcllob8dil2z17VrV/z2229KNUdEREREClAs2fv111/h5uamVHNEVJGYwtFgIqpYKtB+R3ay17x5c7Ro0UJ6NW/eHNWqVcPHH3+Mjz/+WNHgoqKi8OKLL6Jy5crw8PBAz549kZycrFfn0aNHiIiIQJUqVeDo6Ig+ffogLS1Nr05KSgrCwsJgb28PDw8PjB8/Hrm5uXp1du/ejRYtWkCj0aBevXpYvny5on0hIiITUoF+6IlkX7PXs2dPvfdWVlZwd3dHx44d0aBBA6XiAgDs2bMHERERePHFF5Gbm4uPP/4YnTt3xtmzZ+Hg4AAAGDNmDLZu3Yp169bB2dkZI0eORO/evbF//34AQF5eHsLCwuDl5YUDBw7g5s2bGDhwIGxsbDBjxgwAwJUrVxAWFoZ3330XK1euRExMDIYPH45q1aohNDRU0T4RlTsLvSaFjIjbDJFZUwlhPt/gW7duwcPDA3v27EFwcDCysrLg7u6OX375Bf/+978BAOfOnUPDhg0RHx+P1q1bY9u2bXjttddw48YNeHp6AgAWL16MCRMm4NatW1Cr1ZgwYQK2bt2K06dPS/Pq378/MjMzsX37doNi02q1cHZ2RlZWFpycnJTvfL6n73w0ZCf8rDoFy/P/yi1Y7+nphs7neW3J9fRni2qruL4UFU95KumyMGQ9GtJ+ce0A8mMzZFssaUwlicPQ9gypL3fZKuFZ2zdQ8mUod/kX9R0qyR3Xxv7uleZOcEOXl6HbhlzF7S+NyRjbsKHfN6D4fbcS85Hjedu9searIENzD7MaVDkrKwsApGsDjx49ipycHISEhEh1GjRogFq1aiE+Ph4AEB8fj4CAACnRA4DQ0FBotVqcOXNGqlOwjfw6+W0UJTs7G1qtVu9FZLJ4yoqIqGyZ0H7X4GTPysoK1tbWxb4qVZJ9VthgOp0OkZGRaNu2LRo3bgwASE1NhVqthouLi15dT09PpKamSnUKJnr50/OnFVdHq9Xi4cOHRcYTFRUFZ2dn6VWzZs1S95GIiIhIaQZnZxs2bHjmtPj4eMyfPx86nU6RoIoSERGB06dPY9++fUabhxyTJk3C2LFjpfdarZYJHxGRUkz41BmRuTE42evRo0ehsuTkZEycOBGbN29GeHg4Pv/8c0WDyzdy5Ehs2bIFcXFxqFGjhlTu5eWFx48fIzMzU+/oXlpaGry8vKQ6hw4d0msv/27dgnWevoM3LS0NTk5OsLOzKzImjUYDjUZT6r4RERERGVOJrtm7ceMGRowYgYCAAOTm5iIxMRErVqyAj4+PosEJITBy5Ehs2LABsbGx8PX11ZvesmVL2NjYICYmRipLTk5GSkoKgoKCAABBQUE4deoU0tPTpTrR0dFwcnJCo0aNpDoF28ivk98GERERmQkTulbOVMhK9rKysjBhwgTUq1cPZ86cQUxMDDZv3ixdQ6e0iIgI/Pzzz/jll19QuXJlpKamIjU1VbqOztnZGcOGDcPYsWOxa9cuHD16FEOGDEFQUBBat24NAOjcuTMaNWqEt956CydOnMCOHTvw6aefIiIiQjoy9+677+Ly5cv46KOPcO7cOSxatAhr167FmDFjjNIvIiIiojIjDDRr1izh5uYmGjVqJDZu3Gjox0oFQJGvZcuWSXUePnwo3n//feHq6irs7e1Fr169xM2bN/XauXr1qujatauws7MTVatWFePGjRM5OTl6dXbt2iWaNWsm1Gq1qFOnjt48DJGVlSUAiKysrJJ21zD5q+zpfw35THHlT66OKX66nNiKa0suQ9oqri9FxVOeShqLIevxWfUMWZclXU5FbYty178S66ek22tx9ZXcjg31rPVXmmUo97NFfYcMaaeodpXe3ouqU5rt9nnTDN02Sjr/st4/GWNehq6r5+2blJhPaeopEaMSn5PB0NzD4HH2rKysYGdnh5CQEFhbWz+z3vr160udgJojjrOH4seNsrRx9pTsT2k/V5Ll86x2gJKPV1bcPDjOnuE4zl7RMRVXxxzG2XvWtsRx9gyPx9C6Ja2n9Dh7ZbC/MDT3MPgGjYEDB0LF8+BERETmh3c3K8cMl6XByR6fFWthmLgTERFVCGb1BA0iIiIikofJHhFVbDzKTWTe+B1+LiZ7RMbAnY9l4/olU2RK26UpxUJM9oioCNxRE1k2fscrFCZ7ZJ64oyIiMg5j7l+57y4XTPYsRUX9Allyv/P7Zsl9rIi4PomojDHZI8vCH1IiMmXcR5kXC1lfTPaIyHgsZEdZoXCdEVkcJntERGT5mMRSBcZkj+ThDpOIiMisMNmjssVksWxwOZOpKrhtcjslKhNM9kifKe18TSkWKjtc70REimKyR0RERIYxpz/GzClWI2OyR8Xjl4WIzAX3V0RFYrJHpccdLJHp4feSyhu3QZPBZI+IzAd/PP6fSsXlURJcbqQkM9mWmOyZIzPZuMgEcFshKnv83pW/otZBBV4vTPbo+SrwF8Tkcd0QlS0lv3P8/lIZYbJHROZNzg8mf1zJmLh9kYliskdkCvgjYTmUXpfcNoiolJjsEX9MiCyJuX2fzS3e8sLlRKXAZI/IlCmxg+ePBBFRhcZkjwzHpEFZXJ5UHEtM9BmPfJa4HZS1it5/MNmjssAvmunhOlEGlyMB3A6UwuVoNEz2qGIy9b+WudMzjLktJ3OLVw5L7huRmWOyR1QeTOmH0ZRieRZziNFcWeqyNbRfltp/c8X1YRRM9shymdtOw9jxKtV+eS1Xc1uf5qYiL9+K3Pfn4bKxCEz2qHQq2o6A/aV8PHJUNvjECvOh1ADnXE+KY7JHFZcxdijmcnSuvBnSj/Lua3nPvyimGFNFZm7rw9ziNVcmuJyZ7JF5McEvkaJM9RSpuS93c4+/NCpy38uSqS1nS3mSi6ktVzPFZI+IylZpd97muPM3x5jLQ2mWE4+qEz0Tkz0iY6mIPw5F9dnSjxpS6RlrG+C2RQSAyR5R6VTkHxNLOU1UFHO6c7msltvT8zGl9fU0U47tefjHkTym9oeCia4fJntUdgp+CUzxTiwT/ZKalLK6scLUki1Tu/PW1LZVU4unpEy1H6YaV0VjxuuByR5RUUpyOrK4dsx4J1Hh8I8N08Tl8/9MfVmYenwVEJM9Mg3PSorMZadh6FHL4j5XEVjymGmmFo+l4nJ+Pi6jsmfiy5zJHpUfE/9ykAmQu41wmzIdlpzYmwJD/jA21vV/lnC5gildblIGmOxZgmdtcKZ24SoZX1mvG24L5kXpaxgt+Sh2ecZrqpd/mFo8chjzd9IMlguTvYrGDDZKoynNNXekPFNbtmUZjykkEmRcSl33W1bk3jRnqkmSqSxjU4nj/zDZs1Tmeu1bSZT1TscUl6UxEllT7KdSTD3xt+RlT89m6ttlcSriYOlmhMleRccvGJFxlNUwNaVpxxS//+UdU3nPn4rH9VMiTPaoZMryh4yIyByZ2z6Q13nrM9e4i8BkjwqzoA3conE9EVFpcT9SITDZI2Vwh0FEZcnUnmpiibjsLAaTPUvCL2b54HI3LRx+hojksvDvMZM9IrJspr4TN/X4TBGXGZEsTPbIdFjqDtxS+0VERGaByR4REZkWS/oDyZL6QmaLyR5RWeFO3zRwPZCl4TYtTwVcXkz2iAoyh52AOcRYUpbcNyIyHu47isVkj4iIiJ5g0mSRmOxVJPwSPyF3OXC5PcHlQM/DbYTKCrc1WZjsWSJ+CQzHZUVEz8P9hGkpz/VhptsCkz0yH2b6JSMqM8b4jphLm2S5uL2UGpM9Kl+m/CU25djIsljStmZJfXmWitBHQ3FZmAUme0REFQ1/oIkqFCZ7RERERBaMyR4RERGRBWOy95SFCxeidu3asLW1RatWrXDo0KHyDomIiIioxJjsFbBmzRqMHTsWU6dOxbFjx9C0aVOEhoYiPT29vEMjIiIiKhEmewXMnTsXI0aMwJAhQ9CoUSMsXrwY9vb2+Omnn8o7NCIiIqISqVTeAZiKx48f4+jRo5g0aZJUZmVlhZCQEMTHxxeqn52djezsbOl9VlYWAECr1Ro/2HxFzevpMkPqGNq2seZvjnWKwmVW+jpFMbUYTa1OUUwtRlOrUxRTi9HU6hTF1GI0tTrPKlNQfs4hhCi+oiAhhBDXr18XAMSBAwf0ysePHy9eeumlQvWnTp0qAPDFF1988cUXX3yV6+vatWvF5jg8sldCkyZNwtixY6X3Op0OGRkZqFKlClRGGsNKq9WiZs2auHbtGpycnIwyD1NTEfsMsN8Vqd8Vsc8A+12R+l0R+wyUTb+FELh79y68vb2Lrcdk7/9UrVoV1tbWSEtL0ytPS0uDl5dXofoajQYajUavzMXFxZghSpycnCrUFwaomH0G2O+KpCL2GWC/K5KK2GfA+P12dnZ+bh3eoPF/1Go1WrZsiZiYGKlMp9MhJiYGQUFB5RgZERERUcnxyF4BY8eOxaBBgxAYGIiXXnoJ3377Le7fv48hQ4aUd2hEREREJcJkr4B+/frh1q1bmDJlClJTU9GsWTNs374dnp6e5R0agCenjqdOnVro9LElq4h9BtjvitTvithngP2uSP2uiH0GTKvfKiGed78uEREREZkrXrNHREREZMGY7BERERFZMCZ7RERERBaMyR4RERGRBWOyZyYWLlyI2rVrw9bWFq1atcKhQ4fKOySDxcXFoXv37vD29oZKpcLGjRv1pgshMGXKFFSrVg12dnYICQnBhQsX9OpkZGQgPDwcTk5OcHFxwbBhw3Dv3j29OidPnkT79u1ha2uLmjVrYvbs2cbuWrGioqLw4osvonLlyvDw8EDPnj2RnJysV+fRo0eIiIhAlSpV4OjoiD59+hQa2DslJQVhYWGwt7eHh4cHxo8fj9zcXL06u3fvRosWLaDRaFCvXj0sX77c2N0r0nfffYcmTZpIg4gGBQVh27Zt0nRL6++zzJw5EyqVCpGRkVKZJfZ92rRpUKlUeq8GDRpI0y2xzwBw/fp1vPnmm6hSpQrs7OwQEBCAI0eOSNMtcZ9Wu3btQutapVIhIiICgGWu67y8PEyePBm+vr6ws7ND3bp18cUXX+g9h9Zs1rUyT5YlY1q9erVQq9Xip59+EmfOnBEjRowQLi4uIi0trbxDM8gff/whPvnkE7F+/XoBQGzYsEFv+syZM4Wzs7PYuHGjOHHihPjXv/4lfH19xcOHD6U6Xbp0EU2bNhUHDx4Ue/fuFfXq1RMDBgyQpmdlZQlPT08RHh4uTp8+LVatWiXs7OzE999/X1bdLCQ0NFQsW7ZMnD59WiQmJopu3bqJWrVqiXv37kl13n33XVGzZk0RExMjjhw5Ilq3bi3atGkjTc/NzRWNGzcWISEh4vjx4+KPP/4QVatWFZMmTZLqXL58Wdjb24uxY8eKs2fPigULFghra2uxffv2Mu2vEEJs2rRJbN26VZw/f14kJyeLjz/+WNjY2IjTp09bZH+LcujQIVG7dm3RpEkTMXr0aKncEvs+depU4e/vL27evCm9bt26JU23xD5nZGQIHx8fMXjwYJGQkCAuX74sduzYIS5evCjVscR9Wnp6ut56jo6OFgDErl27hBCWua6nT58uqlSpIrZs2SKuXLki1q1bJxwdHcW8efOkOuayrpnsmYGXXnpJRERESO/z8vKEt7e3iIqKKseoSubpZE+n0wkvLy8xZ84cqSwzM1NoNBqxatUqIYQQZ8+eFQDE4cOHpTrbtm0TKpVKXL9+XQghxKJFi4Srq6vIzs6W6kyYMEH4+fkZuUeGS09PFwDEnj17hBBP+mljYyPWrVsn1UlKShIARHx8vBDiSaJsZWUlUlNTpTrfffedcHJykvr60UcfCX9/f7159evXT4SGhhq7SwZxdXUVS5curRD9vXv3rqhfv76Ijo4WHTp0kJI9S+371KlTRdOmTYucZql9njBhgmjXrt0zp1eUfdro0aNF3bp1hU6ns9h1HRYWJoYOHapX1rt3bxEeHi6EMK91zdO4Ju7x48c4evQoQkJCpDIrKyuEhIQgPj6+HCNTxpUrV5CamqrXP2dnZ7Rq1UrqX3x8PFxcXBAYGCjVCQkJgZWVFRISEqQ6wcHBUKvVUp3Q0FAkJyfjzp07ZdSb4mVlZQEA3NzcAABHjx5FTk6OXt8bNGiAWrVq6fU9ICBAb2Dv0NBQaLVanDlzRqpTsI38OuW9feTl5WH16tW4f/8+goKCLL6/ABAREYGwsLBC8Vly3y9cuABvb2/UqVMH4eHhSElJAWC5fd60aRMCAwPx+uuvw8PDA82bN8cPP/wgTa8I+7THjx/j559/xtChQ6FSqSx2Xbdp0wYxMTE4f/48AODEiRPYt28funbtCsC81jWTPRP3zz//IC8vr9BTPDw9PZGamlpOUSknvw/F9S81NRUeHh560ytVqgQ3Nze9OkW1UXAe5Umn0yEyMhJt27ZF48aNATyJS61Ww8XFRa/u031/Xr+eVUer1eLhw4fG6E6xTp06BUdHR2g0Grz77rvYsGEDGjVqZLH9zbd69WocO3YMUVFRhaZZat9btWqF5cuXY/v27fjuu+9w5coVtG/fHnfv3rXYPl++fBnfffcd6tevjx07duC9997DqFGjsGLFCr24LXmftnHjRmRmZmLw4MFSPJa4ridOnIj+/fujQYMGsLGxQfPmzREZGYnw8HC9uM1hXfNxaURlICIiAqdPn8a+ffvKOxSj8/PzQ2JiIrKysvDrr79i0KBB2LNnT3mHZVTXrl3D6NGjER0dDVtb2/IOp8zkH+EAgCZNmqBVq1bw8fHB2rVrYWdnV46RGY9Op0NgYCBmzJgBAGjevDlOnz6NxYsXY9CgQeUcXdn48ccf0bVrV3h7e5d3KEa1du1arFy5Er/88gv8/f2RmJiIyMhIeHt7m9265pE9E1e1alVYW1sXuqspLS0NXl5e5RSVcvL7UFz/vLy8kJ6erjc9NzcXGRkZenWKaqPgPMrLyJEjsWXLFuzatQs1atSQyr28vPD48WNkZmbq1X+678/r17PqODk5lcsPrlqtRr169dCyZUtERUWhadOmmDdvnsX2F3hyyjI9PR0tWrRApUqVUKlSJezZswfz589HpUqV4OnpabF9L8jFxQUvvPACLl68aLHru1q1amjUqJFeWcOGDaXT15a+T/vrr7/w559/Yvjw4VKZpa7r8ePHS0f3AgIC8NZbb2HMmDHS0XtzWtdM9kycWq1Gy5YtERMTI5XpdDrExMQgKCioHCNThq+vL7y8vPT6p9VqkZCQIPUvKCgImZmZOHr0qFQnNjYWOp0OrVq1kurExcUhJydHqhMdHQ0/Pz+4urqWUW/0CSEwcuRIbNiwAbGxsfD19dWb3rJlS9jY2Oj1PTk5GSkpKXp9P3XqlN7OIjo6Gk5OTtIPTlBQkF4b+XVMZfvQ6XTIzs626P526tQJp06dQmJiovQKDAxEeHi49H9L7XtB9+7dw6VLl1CtWjWLXd9t27YtNITS+fPn4ePjA8Cy92kAsGzZMnh4eCAsLEwqs9R1/eDBA1hZ6adJ1tbW0Ol0AMxsXSt2qwcZzerVq4VGoxHLly8XZ8+eFW+//bZwcXHRu6vJlN29e1ccP35cHD9+XAAQc+fOFcePHxd//fWXEOLJresuLi7i999/FydPnhQ9evQo8tb15s2bi4SEBLFv3z5Rv359vVvXMzMzhaenp3jrrbfE6dOnxerVq4W9vX25Dr3y3nvvCWdnZ7F79269IQsePHgg1Xn33XdFrVq1RGxsrDhy5IgICgoSQUFB0vT84Qo6d+4sEhMTxfbt24W7u3uRwxWMHz9eJCUliYULF5bbcAUTJ04Ue/bsEVeuXBEnT54UEydOFCqVSuzcudMi+1ucgnfjCmGZfR83bpzYvXu3uHLliti/f78ICQkRVatWFenp6UIIy+zzoUOHRKVKlcT06dPFhQsXxMqVK4W9vb34+eefpTqWuk/Ly8sTtWrVEhMmTCg0zRLX9aBBg0T16tWloVfWr18vqlatKj766COpjrmsayZ7ZmLBggWiVq1aQq1Wi5deekkcPHiwvEMy2K5duwSAQq9BgwYJIZ7cvj558mTh6ekpNBqN6NSpk0hOTtZr4/bt22LAgAHC0dFRODk5iSFDhoi7d+/q1Tlx4oRo166d0Gg0onr16mLmzJll1cUiFdVnAGLZsmVSnYcPH4r3339fuLq6Cnt7e9GrVy9x8+ZNvXauXr0qunbtKuzs7ETVqlXFuHHjRE5Ojl6dXbt2iWbNmgm1Wi3q1KmjN4+yNHToUOHj4yPUarVwd3cXnTp1khI9ISyvv8V5OtmzxL7369dPVKtWTajValG9enXRr18/vfHmLLHPQgixefNm0bhxY6HRaESDBg3EkiVL9KZb6j5tx44dAkChvghhmetaq9WK0aNHi1q1aglbW1tRp04d8cknn+gNkWIu61olRIGhoImIiIjIovCaPSIiIiILxmSPiIiIyIIx2SMiIiKyYEz2iIiIiCwYkz0iIiIiC8Zkj4iIiMiCMdkjIiIismBM9oiIiIgsGJM9IqICrl69CpVKhcTExPIORXLu3Dm0bt0atra2aNasWZnOe9q0aWU+TyJSFpM9IjIpgwcPhkqlwsyZM/XKN27cCJVKVU5Rla+pU6fCwcEBycnJhR4Ub2wffvhhmc+TiJTFZI+ITI6trS1mzZqFO3fulHcoinn8+HGJP3vp0iW0a9cOPj4+qFKlitHnV5Cjo6PB8yQi08Rkj4hMTkhICLy8vBAVFfXMOkWdXvz2229Ru3Zt6f3gwYPRs2dPzJgxA56ennBxccHnn3+O3NxcjB8/Hm5ubqhRowaWLVtWqP1z586hTZs2sLW1RePGjbFnzx696adPn0bXrl3h6OgIT09PvPXWW/jnn3+k6R07dsTIkSMRGRmJqlWrIjQ0tMh+6HQ6fP7556hRowY0Gg2aNWuG7du3S9NVKhWOHj2Kzz//HCqVCtOmTSuynWfNr7g4lyxZAm9vb+h0Or22evTogaFDhz5zOS9duhQNGzaEra0tGjRogEWLFknT/v3vf2PkyJHS+8jISKhUKpw7dw7AkyTUwcEBf/75Z5H9ICLlMdkjIpNjbW2NGTNmYMGCBfj7779L1VZsbCxu3LiBuLg4zJ07F1OnTsVrr70GV1dXJCQk4N1338U777xTaD7jx4/HuHHjcPz4cQQFBaF79+64ffs2ACAzMxOvvPIKmjdvjiNHjmD79u1IS0tD37599dpYsWIF1Go19u/fj8WLFxcZ37x58/D111/jq6++wsmTJxEaGop//etfuHDhAgDg5s2b8Pf3x7hx43Dz5k18+OGHz+zr0/N7Xpyvv/46bt++jV27dkltZGRkYPv27QgPDy9yHitXrsSUKVMwffp0JCUlYcaMGZg8eTJWrFgBAOjQoQN2794t1d+zZw+qVq0qlR0+fBg5OTlo06bNM/tBRAoTREQmZNCgQaJHjx5CCCFat24thg4dKoQQYsOGDaLgLmvq1KmiadOmep/95ptvhI+Pj15bPj4+Ii8vTyrz8/MT7du3l97n5uYKBwcHsWrVKiGEEFeuXBEAxMyZM6U6OTk5okaNGmLWrFlCCCG++OIL0blzZ715X7t2TQAQycnJQgghOnToIJo3b/7c/np7e4vp06frlb344ovi/fffl943bdpUTJ06tdh2ipqfIXH26NFDWsZCCPH9998Lb29vaZk9vZzr1q0rfvnll0LzCQoKEkIIcfLkSaFSqUR6errIyMgQarVafPHFF6Jfv35CCCG+/PJL0aZNm2L7QkTK4pE9IjJZs2bNwooVK5CUlFTiNvz9/WFl9f+7Ok9PTwQEBEjvra2tUaVKFaSnp+t9LigoSPp/pUqVEBgYKMVx4sQJ7Nq1C46OjtKrQYMGAJ5cX5evZcuWxcam1Wpx48YNtG3bVq+8bdu2Jerz0/MzJM7w8HD89ttvyM7OBvDkyF3//v31llm++/fv49KlSxg2bJhem19++aXUXuPGjeHm5oY9e/Zg7969aN68OV577TXpNPiePXvQsWNH2X0jopKrVN4BEBE9S3BwMEJDQzFp0iQMHjxYb5qVlRWEEHplOTk5hdqwsbHRe69SqYose/q6teLcu3cP3bt3x6xZswpNq1atmvR/BwcHg9tUwtPzMyTO7t27QwiBrVu34sUXX8TevXvxzTffFNn+vXv3AAA//PADWrVqpTfN2toawJNlGRwcjN27d0Oj0aBjx45o0qQJsrOzcfr0aRw4cKDYU9FEpDwme0Rk0mbOnIlmzZrBz89Pr9zd3R2pqakQQkhDsig5Nt7BgwcRHBwMAMjNzcXRo0elGw9atGiB3377DbVr10alSiXfjTo5OcHb2xv79+9Hhw4dpPL9+/fjpZdeKl0HDIzT1tYWvXv3xsqVK3Hx4kX4+fmhRYsWRdb19PSEt7c3Ll++/Mxr+oAn1+398MMP0Gg0mD59OqysrBAcHIw5c+YgOzu70JFMIjIunsYlIpMWEBCA8PBwzJ8/X6+8Y8eOuHXrFmbPno1Lly5h4cKF2LZtm2LzXbhwITZs2IBz584hIiICd+7cke5QjYiIQEZGBgYMGIDDhw/j0qVL2LFjB4YMGYK8vDxZ8xk/fjxmzZqFNWvWIDk5GRMnTkRiYiJGjx5d6j4YGmd4eDi2bt2Kn376qdgkDgA+++wzREVFYf78+Th//jxOnTqFZcuWYe7cuVKdjh074uzZszhz5gzatWsnla1cuRKBgYFlfsSTqKJjskdEJu/zzz8vdJq1YcOGWLRoERYuXIimTZvi0KFDip4enDlzJmbOnImmTZti37592LRpE6pWrQoA0tG4vLw8dO7cGQEBAYiMjISLi0uR17oVZ9SoURg7dizGjRuHgIAAbN++HZs2bUL9+vVL3QdD43zllVfg5uaG5ORkvPHGG8W2OXz4cCxduhTLli1DQEAAOnTogOXLl8PX11eqExAQABcXFzRr1gyOjo4AniR7eXl5vF6PqByoxNMXvRARERGRxeCRPSIiIiILxmSPiIiIyIIx2SMiIiKyYEz2iIiIiCwYkz0iIiIiC8Zkj4iIiMiCMdkjIiIismBM9oiIiIgsGJM9IiIiIgvGZI+IiIjIgjHZIyIiIrJg/wuqX4RePrXV9wAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 700x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sentences = [len(sent) for sent in train_df['review']]\n",
    "\n",
    "plt.rcParams.update({'figure.figsize':(7,5), 'figure.dpi':100})\n",
    "plt.bar(range(1,8001), sentences, color = ['red'])\n",
    "plt.gca().set(title='No. of characters in each review', xlabel='Number of review', ylabel='Number of Characters in each review')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now that the text data has been cleaned, we can prepare the reviews and their corresponding sentiment labels for model training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "reviews = train_df['Cleaned_sentence']\n",
    "target = train_df['sentiment']\n",
    "\n",
    "test_reviews = test_df['Cleaned_sentence']\n",
    "test_targets = test_df['sentiment']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To facilitate binary classification, the sentiment labels are converted into boolean values. The target labels for both the training and test datasets are generated based on the condition of being 'positive' or not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((6000,), (1000,), 0.49533333333333335, 0.501)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_y = np.array(target) == 'positive'\n",
    "test_y = np.array(test_targets) == 'positive'\n",
    "train_y.shape, test_y.shape, np.mean(train_y), np.mean(test_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To assess the model's performance on unseen data, we split the test dataset into a validation set (`x_val`, `y_val`) and a final test set (`x_test`, `y_test`). The split is done while ensuring a balanced distribution of sentiment labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_val, x_test, y_val, y_test = train_test_split(test_reviews,\n",
    "                                                    test_y,\n",
    "                                                    test_size=0.5, \n",
    "                                                    stratify = test_targets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### For tokenizing the text data using the BERT model, we initialize the tokenizer from the 'bert-base-uncased' pre-trained model with lowercase tokenization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', do_lower_case=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To prepare the text data for input to the BERT model, we tokenize and encode the sentences using the BERT tokenizer. The maximum length of the sequences is set to 300."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "max_len= 300\n",
    "# Tokenize and encode the sentences\n",
    "X_train_encoded = tokenizer.batch_encode_plus(reviews.tolist(),\n",
    "                                              padding=True, \n",
    "                                              truncation=True,\n",
    "                                              max_length = max_len,\n",
    "                                              return_tensors='tf')\n",
    " \n",
    "X_val_encoded = tokenizer.batch_encode_plus(x_val.tolist(), \n",
    "                                              padding=True, \n",
    "                                              truncation=True,\n",
    "                                              max_length = max_len,\n",
    "                                              return_tensors='tf')\n",
    " \n",
    "X_test_encoded = tokenizer.batch_encode_plus(x_test.tolist(), \n",
    "                                              padding=True, \n",
    "                                              truncation=True,\n",
    "                                              max_length = max_len,\n",
    "                                              return_tensors='tf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now, let's initialize the BERT model for sequence classification. We use the 'bert-base-uncased' pre-trained model with two output labels for binary classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Kodprojekt\\Skolarbete\\env\\lib\\site-packages\\keras\\src\\backend.py:873: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "All PyTorch model weights were used when initializing TFBertForSequenceClassification.\n",
      "\n",
      "Some weights or buffers of the TF 2.0 model TFBertForSequenceClassification were not initialized from the PyTorch model and are newly initialized: ['classifier.weight', 'classifier.bias']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    }
   ],
   "source": [
    "model = TFBertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To prepare the BERT model for training, we compile it with the specified optimizer, loss function, and metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizer = tf.keras.optimizers.Adam(learning_rate=2e-5)\n",
    "loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)\n",
    "metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')\n",
    "model.compile(optimizer=optimizer, loss=loss, metrics=[metric])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's train the BERT model on the provided training data. The training history is stored in the `history` variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n",
      "WARNING:tensorflow:From c:\\Kodprojekt\\Skolarbete\\env\\lib\\site-packages\\keras\\src\\utils\\tf_utils.py:492: The name tf.ragged.RaggedTensorValue is deprecated. Please use tf.compat.v1.ragged.RaggedTensorValue instead.\n",
      "\n",
      "188/188 [==============================] - 2003s 11s/step - loss: 0.4227 - accuracy: 0.7797 - val_loss: 0.1207 - val_accuracy: 0.9580\n",
      "Epoch 2/3\n",
      "188/188 [==============================] - 1986s 11s/step - loss: 0.1778 - accuracy: 0.9363 - val_loss: 0.0549 - val_accuracy: 0.9860\n",
      "Epoch 3/3\n",
      "188/188 [==============================] - 1988s 11s/step - loss: 0.0922 - accuracy: 0.9732 - val_loss: 0.0261 - val_accuracy: 0.9960\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(\n",
    "    [X_train_encoded['input_ids'], X_train_encoded['token_type_ids'], X_train_encoded['attention_mask']],\n",
    "    train_y,\n",
    "    validation_data=(\n",
    "      [X_val_encoded['input_ids'], X_val_encoded['token_type_ids'], X_val_encoded['attention_mask']],y_val),\n",
    "    batch_size=32,\n",
    "    epochs=3\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Now, let's evaluate the trained BERT model on the test data to assess its performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16/16 [==============================] - 47s 3s/step - loss: 0.0454 - accuracy: 0.9860\n",
      "Test loss: 0.045367445796728134, Test accuracy: 0.9860000014305115\n"
     ]
    }
   ],
   "source": [
    "test_loss, test_accuracy = model.evaluate(\n",
    "    [X_test_encoded['input_ids'], X_test_encoded['token_type_ids'], X_test_encoded['attention_mask']],\n",
    "    y_test\n",
    ")\n",
    "print(f'Test loss: {test_loss}, Test accuracy: {test_accuracy}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To preserve the trained BERT model and the associated tokenizer for future use, let's save them to the specified path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = './model/'\n",
    "# Save tokenizer\n",
    "tokenizer.save_pretrained(path +'/Tokenizer')\n",
    " \n",
    "# Save model\n",
    "model.save_pretrained(path +'/Model')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To use the saved BERT tokenizer and model for inference, let's load them from the specified path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some layers from the model checkpoint at ./model//Model were not used when initializing TFBertForSequenceClassification: ['dropout_37']\n",
      "- This IS expected if you are initializing TFBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
      "- This IS NOT expected if you are initializing TFBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
      "All the layers of TFBertForSequenceClassification were initialized from the model checkpoint at ./model//Model.\n",
      "If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertForSequenceClassification for predictions without further training.\n"
     ]
    }
   ],
   "source": [
    "bert_tokenizer = BertTokenizer.from_pretrained(path +'/Tokenizer')\n",
    " \n",
    "# Load model\n",
    "bert_model = TFBertForSequenceClassification.from_pretrained(path +'/Model')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's use the loaded BERT model to make predictions on the test data and compare them with the actual labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16/16 [==============================] - 49s 3s/step\n",
      "Predicted Label : ['Negative', 'Negative', 'positive', 'Negative', 'Negative', 'Negative', 'Negative', 'positive', 'positive', 'Negative']\n",
      "Actual Label    : ['Negative', 'Negative', 'positive', 'Negative', 'Negative', 'Negative', 'Negative', 'positive', 'positive', 'Negative']\n"
     ]
    }
   ],
   "source": [
    "pred = bert_model.predict(\n",
    "    [X_test_encoded['input_ids'], X_test_encoded['token_type_ids'], X_test_encoded['attention_mask']])\n",
    " \n",
    "# pred is of type TFSequenceClassifierOutput\n",
    "logits = pred.logits\n",
    " \n",
    "# Use argmax along the appropriate axis to get the predicted labels\n",
    "pred_labels = tf.argmax(logits, axis=1)\n",
    " \n",
    "# Convert the predicted labels to a NumPy array\n",
    "pred_labels = pred_labels.numpy()\n",
    " \n",
    "label = {\n",
    "    1: 'positive',\n",
    "    0: 'Negative'\n",
    "}\n",
    " \n",
    "# Map the predicted labels to their corresponding strings using the label dictionary\n",
    "pred_labels = [label[i] for i in pred_labels]\n",
    "Actual = [label[i] for i in y_test]\n",
    " \n",
    "print('Predicted Label :', pred_labels[:10])\n",
    "print('Actual Label    :', Actual[:10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To provide a comprehensive evaluation of the model's performance, let's generate a classification report based on the predicted and actual labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "    Negative       0.98      1.00      0.99       249\n",
      "    positive       1.00      0.98      0.99       251\n",
      "\n",
      "    accuracy                           0.99       500\n",
      "   macro avg       0.99      0.99      0.99       500\n",
      "weighted avg       0.99      0.99      0.99       500\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"Classification Report: \\n\", classification_report(Actual, pred_labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Here is a Python function, `Get_sentiment`, that takes a review or a list of reviews as input and returns the predicted sentiment labels using the loaded BERT tokenizer and model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def Get_sentiment(Review, Tokenizer=bert_tokenizer, Model=bert_model):\n",
    "    # Convert Review to a list if it's not already a list\n",
    "    if not isinstance(Review, list):\n",
    "        Review = [Review]\n",
    "    label = {\n",
    "        1: 'positive',\n",
    "        0: 'Negative'\n",
    "    }\n",
    "    Input_ids, Token_type_ids, Attention_mask = Tokenizer.batch_encode_plus(Review,\n",
    "                                                                             padding=True,\n",
    "                                                                             truncation=True,\n",
    "                                                                             max_length=300,\n",
    "                                                                             return_tensors='tf').values()\n",
    "\n",
    "    prediction = Model.predict([Input_ids, Token_type_ids, Attention_mask])\n",
    "\n",
    "    # Use argmax along the appropriate axis to get the predicted labels\n",
    "    pred_labels = tf.argmax(prediction.logits, axis=1)\n",
    "\n",
    "    # Convert the TensorFlow tensor to a NumPy array and then to a list to get the predicted sentiment labels\n",
    "    pred_labels = [label[i] for i in pred_labels.numpy().tolist()]\n",
    "    return pred_labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's use the `Get_sentiment` function to predict the sentiment for the provided review."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1/1 [==============================] - 1s 1s/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['Negative']"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Review =\"\"\"Too much cgi, too much lenght, too boring characters, (ecept my precious thing )\n",
    "so many better fantasties out there.\n",
    "Zzz too overated. The first one was better. Oh, and the rings are boring too. And the walking trees. Ok, they are ok for a bit. Cant really thionk of anything positive to say. Think ill watch willow insted.\"\"\"\n",
    "\n",
    "Get_sentiment(Review)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The BERT sentiment analysis model has completed training, and now it's time to incorporate it into the `app.py` project. In this project, users can input an IMDb movie link to fetch the latest 25 reviews and receive a sentiment label (positive/negative) for each review. Additionally, users can input a single review for sentiment analysis."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}