3 Driverless AI: AutoML.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Driverless AI for AutoML\n",
    "\n",
    "This notebook is intended to help you get started with AutoML in the H2O AI Cloud using python.\n",
    "\n",
    "* **Product Documentation:** https://docs.h2o.ai/driverless-ai/1-10-lts/docs/userguide/index.html\n",
    "* **Python Documentation:** https://docs.h2o.ai/driverless-ai/pyclient/docs/html/index.html\n",
    "* **Additional Tutorials:** https://github.com/h2oai/driverlessai-tutorials/tree/master/dai_python_client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequistes\n",
    "\n",
    "Update the `h2o_ai_cloud.py` file with the connection parameters for your H2O AI Cloud environemnt:\n",
    "1. Login to your H2O AI Cloud environment\n",
    "1. Click your username or avatar in the H2O AI Cloud navigation bar\n",
    "1. Navigate to `CLI & API Access`\n",
    "1. Use the variables from the `Accessing H2O AI Cloud APIs` section to populate the parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from getpass import getpass\n",
    "\n",
    "from h2o_ai_cloud import steam_client\n",
    "from h2osteam.clients import DriverlessClient\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Securely connect \n",
    "We first connect to the H2O AI Cloud using our platform token to create a token provider object. We can then use this object to log into Steam and other APIs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Visit https://cloud.h2o.ai/auth/get-platform-token to get your platform token\n",
      "Enter your platform token: ········\n"
     ]
    }
   ],
   "source": [
    "steam = steam_client()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect to Driverless AI\n",
    "We'll create a connection object called dai that we will use to interact with the platform."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['visitor-driverless-kubernetes']\n"
     ]
    }
   ],
   "source": [
    "dai_profiles = [profile[\"name\"] for profile in steam.get_profiles() if profile[\"profile_type\"] == \"driverless_kubernetes\"]\n",
    "\n",
    "print(dai_profiles)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Driverless AI instance is submitted, please wait...\n",
      "Driverless AI instance is running\n"
     ]
    }
   ],
   "source": [
    "dai_machine = DriverlessClient(steam).launch_instance(\n",
    "    name=\"my-test-instance\",\n",
    "    version=steam.get_driverless_engines()[-1][\"version\"],\n",
    "    profile_name=dai_profiles[0],\n",
    "    max_uptime_h=1\n",
    ")\n",
    "\n",
    "dai = dai_machine.connect()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visit the DAI URL\n",
    "You must click the link below if you would like to use the DAI UI anywhere else in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'https://steam.cloud.h2o.ai:443/oidc-login-start?forward=/proxy/driverless/3286/openid/callback'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dai_machine.openid_login_url()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.10.3'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dai.server.version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Documentation\n",
    "You can get links to the home page and search the documentation from the python client any time you want to know more about a specific DAI functionality. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre><a href='https://steam.cloud.h2o.ai:443/proxy/driverless/3286/docs/userguide/search.html?q=autoviz' rel='noopener noreferrer' target='_blank'>https://steam.cloud.h2o.ai:443/proxy/driverless/3286/docs/userguide/search.html?q=autoviz</a></pre>"
      ],
      "text/plain": [
       "'https://steam.cloud.h2o.ai:443/proxy/driverless/3286/docs/userguide/search.html?q=autoviz'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dai.server.docs(\"autoviz\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data\n",
    "Various methods for loading, interacting with, and modifiying data wihthin Driverless AI."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### List Available Connectors\n",
    "View all ways you are allowed to add data to DAI on this instace - to enable more connectors update the `config.toml` file <br/><br/>\n",
    "**Note:** Interactions with Data Recipes is not yet available so these connectors are not shown"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['upload', 'file', 's3', 'recipe_file', 'recipe_url']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dai.connectors.list()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### List Existing Datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "    | Type   | Key   | Name\n",
      "----+--------+-------+--------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Get a link to view all datasets in the UI\n",
    "dai.datasets.gui()\n",
    "\n",
    "print()\n",
    "\n",
    "# List all datasets    \n",
    "print(dai.datasets.list(start_index=0, count=4))\n",
    "\n",
    "print()\n",
    "\n",
    "# pretty print\n",
    "for d in dai.datasets.list(start_index=0, count=4):\n",
    "    print(type(d), d.key, d.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Upload and Download Data\n",
    "You can upload data using any method that is enabled on your system. Here we will show:\n",
    "* Add data from a public s3 link\n",
    "* Download a dataset\n",
    "* Upload data from your local machine\n",
    "* Rename a dataset\n",
    "* Upload using the JDBC connector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Complete 100.00% - [4/4] Computed stats for column Churn?\n"
     ]
    }
   ],
   "source": [
    "telco_churn = dai.datasets.create(data=\"https://h2o-internal-release.s3-us-west-2.amazonaws.com/data/Splunk/churn.csv\", \n",
    "                                  data_source=\"s3\", \n",
    "                                  name=\"Telco_Churn\",\n",
    "                                  force=True\n",
    "                                 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloaded 'churn.csv.1657067059.647558.csv'\n"
     ]
    }
   ],
   "source": [
    "local_file_path = telco_churn.download(\"./\", overwrite=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Complete 100.00% - [4/4] Computed stats for column Churn?\n"
     ]
    }
   ],
   "source": [
    "telco_churn2 = dai.datasets.create(local_file_path, name=\"Telco_Churn_Duplicate\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Old Name: Telco_Churn_Duplicate\n",
      "New Name: Fancy New Name\n"
     ]
    }
   ],
   "source": [
    "print(\"Old Name:\", telco_churn2.name)\n",
    "\n",
    "telco_churn2.rename(\"Fancy New Name\")\n",
    "\n",
    "print(\"New Name:\", telco_churn2.name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Upload using the JDBC connector\n",
    "# Note 1: You must have JDBC in the list of available connectors\n",
    "# Note 2: Appropriate JARS must be on the DAI server\n",
    "\n",
    "jdbc_config = {\n",
    "    'jdbc_jar': '/data/postgresql-42.2.9.jar',\n",
    "    'jdbc_driver': 'org.postgresql.Driver',\n",
    "    'jdbc_url': 'jdbc:postgresql://mr-dl2:5432/h2oaidev',\n",
    "    'jdbc_username': 'h2oaitester',\n",
    "    'jdbc_password': 'h2oaitesterreadonly'\n",
    "}\n",
    "\n",
    "# jdbc_ds = dai.datasets.create(   \n",
    "#     data='SELECT * FROM creditcardtrain',\n",
    "#     data_source='jdbc',\n",
    "#     data_source_config=jdbc_config,\n",
    "#     name='beta_jdbc_test'\n",
    "# )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explore the Dataset\n",
    "* View the column names\n",
    "* View the data shape\n",
    "* View the first and last rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f956fdfa-fcc1-11ec-88b7-66b02643127c | Telco_Churn\n",
      "\n",
      "Columns: ['State', 'Account Length', 'Area Code', 'Phone', \"Int'l Plan\", 'VMail Plan', 'VMail Message', 'Day Mins', 'Day Calls', 'Day Charge', 'Eve Mins', 'Eve Calls', 'Eve Charge', 'Night Mins', 'Night Calls', 'Night Charge', 'Intl Mins', 'Intl Calls', 'Intl Charge', 'CustServ Calls', 'Churn?']\n",
      "\n",
      "Shape: (3333, 21)\n"
     ]
    }
   ],
   "source": [
    "print(telco_churn.key, \"|\", telco_churn.name)\n",
    "print(\"\\nColumns:\", telco_churn.columns)\n",
    "print('\\nShape:', telco_churn.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead>\n",
       "<tr><th>State  </th><th style=\"text-align: right;\">  Account Length</th><th style=\"text-align: right;\">  Area Code</th><th>Phone   </th><th>Int&#x27;l Plan  </th><th>VMail Plan  </th><th style=\"text-align: right;\">  VMail Message</th><th style=\"text-align: right;\">  Day Mins</th><th style=\"text-align: right;\">  Day Calls</th><th style=\"text-align: right;\">  Day Charge</th><th style=\"text-align: right;\">  Eve Mins</th><th style=\"text-align: right;\">  Eve Calls</th><th style=\"text-align: right;\">  Eve Charge</th><th style=\"text-align: right;\">  Night Mins</th><th style=\"text-align: right;\">  Night Calls</th><th style=\"text-align: right;\">  Night Charge</th><th style=\"text-align: right;\">  Intl Mins</th><th style=\"text-align: right;\">  Intl Calls</th><th style=\"text-align: right;\">  Intl Charge</th><th style=\"text-align: right;\">  CustServ Calls</th><th>Churn?  </th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr><td>KS     </td><td style=\"text-align: right;\">             128</td><td style=\"text-align: right;\">        415</td><td>382-4657</td><td>no          </td><td>yes         </td><td style=\"text-align: right;\">             25</td><td style=\"text-align: right;\">     265.1</td><td style=\"text-align: right;\">        110</td><td style=\"text-align: right;\">       45.07</td><td style=\"text-align: right;\">     197.4</td><td style=\"text-align: right;\">         99</td><td style=\"text-align: right;\">       16.78</td><td style=\"text-align: right;\">       244.7</td><td style=\"text-align: right;\">           91</td><td style=\"text-align: right;\">         11.01</td><td style=\"text-align: right;\">       10  </td><td style=\"text-align: right;\">           3</td><td style=\"text-align: right;\">         2.7 </td><td style=\"text-align: right;\">               1</td><td>False.  </td></tr>\n",
       "<tr><td>OH     </td><td style=\"text-align: right;\">             107</td><td style=\"text-align: right;\">        415</td><td>371-7191</td><td>no          </td><td>yes         </td><td style=\"text-align: right;\">             26</td><td style=\"text-align: right;\">     161.6</td><td style=\"text-align: right;\">        123</td><td style=\"text-align: right;\">       27.47</td><td style=\"text-align: right;\">     195.5</td><td style=\"text-align: right;\">        103</td><td style=\"text-align: right;\">       16.62</td><td style=\"text-align: right;\">       254.4</td><td style=\"text-align: right;\">          103</td><td style=\"text-align: right;\">         11.45</td><td style=\"text-align: right;\">       13.7</td><td style=\"text-align: right;\">           3</td><td style=\"text-align: right;\">         3.7 </td><td style=\"text-align: right;\">               1</td><td>False.  </td></tr>\n",
       "<tr><td>NJ     </td><td style=\"text-align: right;\">             137</td><td style=\"text-align: right;\">        415</td><td>358-1921</td><td>no          </td><td>no          </td><td style=\"text-align: right;\">              0</td><td style=\"text-align: right;\">     243.4</td><td style=\"text-align: right;\">        114</td><td style=\"text-align: right;\">       41.38</td><td style=\"text-align: right;\">     121.2</td><td style=\"text-align: right;\">        110</td><td style=\"text-align: right;\">       10.3 </td><td style=\"text-align: right;\">       162.6</td><td style=\"text-align: right;\">          104</td><td style=\"text-align: right;\">          7.32</td><td style=\"text-align: right;\">       12.2</td><td style=\"text-align: right;\">           5</td><td style=\"text-align: right;\">         3.29</td><td style=\"text-align: right;\">               0</td><td>False.  </td></tr>\n",
       "<tr><td>OH     </td><td style=\"text-align: right;\">              84</td><td style=\"text-align: right;\">        408</td><td>375-9999</td><td>yes         </td><td>no          </td><td style=\"text-align: right;\">              0</td><td style=\"text-align: right;\">     299.4</td><td style=\"text-align: right;\">         71</td><td style=\"text-align: right;\">       50.9 </td><td style=\"text-align: right;\">      61.9</td><td style=\"text-align: right;\">         88</td><td style=\"text-align: right;\">        5.26</td><td style=\"text-align: right;\">       196.9</td><td style=\"text-align: right;\">           89</td><td style=\"text-align: right;\">          8.86</td><td style=\"text-align: right;\">        6.6</td><td style=\"text-align: right;\">           7</td><td style=\"text-align: right;\">         1.78</td><td style=\"text-align: right;\">               2</td><td>False.  </td></tr>\n",
       "<tr><td>OK     </td><td style=\"text-align: right;\">              75</td><td style=\"text-align: right;\">        415</td><td>330-6626</td><td>yes         </td><td>no          </td><td style=\"text-align: right;\">              0</td><td style=\"text-align: right;\">     166.7</td><td style=\"text-align: right;\">        113</td><td style=\"text-align: right;\">       28.34</td><td style=\"text-align: right;\">     148.3</td><td style=\"text-align: right;\">        122</td><td style=\"text-align: right;\">       12.61</td><td style=\"text-align: right;\">       186.9</td><td style=\"text-align: right;\">          121</td><td style=\"text-align: right;\">          8.41</td><td style=\"text-align: right;\">       10.1</td><td style=\"text-align: right;\">           3</td><td style=\"text-align: right;\">         2.73</td><td style=\"text-align: right;\">               3</td><td>False.  </td></tr>\n",
       "</tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<driverlessai._utils.Table at 0x120b8ac40>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "telco_churn.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead>\n",
       "<tr><th>State  </th><th style=\"text-align: right;\">  Account Length</th><th style=\"text-align: right;\">  Area Code</th><th>Phone   </th><th>Int&#x27;l Plan  </th><th>VMail Plan  </th><th style=\"text-align: right;\">  VMail Message</th><th style=\"text-align: right;\">  Day Mins</th><th style=\"text-align: right;\">  Day Calls</th><th style=\"text-align: right;\">  Day Charge</th><th style=\"text-align: right;\">  Eve Mins</th><th style=\"text-align: right;\">  Eve Calls</th><th style=\"text-align: right;\">  Eve Charge</th><th style=\"text-align: right;\">  Night Mins</th><th style=\"text-align: right;\">  Night Calls</th><th style=\"text-align: right;\">  Night Charge</th><th style=\"text-align: right;\">  Intl Mins</th><th style=\"text-align: right;\">  Intl Calls</th><th style=\"text-align: right;\">  Intl Charge</th><th style=\"text-align: right;\">  CustServ Calls</th><th>Churn?  </th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr><td>AZ     </td><td style=\"text-align: right;\">             192</td><td style=\"text-align: right;\">        415</td><td>414-4276</td><td>no          </td><td>yes         </td><td style=\"text-align: right;\">             36</td><td style=\"text-align: right;\">     156.2</td><td style=\"text-align: right;\">         77</td><td style=\"text-align: right;\">       26.55</td><td style=\"text-align: right;\">     215.5</td><td style=\"text-align: right;\">        126</td><td style=\"text-align: right;\">       18.32</td><td style=\"text-align: right;\">       279.1</td><td style=\"text-align: right;\">           83</td><td style=\"text-align: right;\">         12.56</td><td style=\"text-align: right;\">        9.9</td><td style=\"text-align: right;\">           6</td><td style=\"text-align: right;\">         2.67</td><td style=\"text-align: right;\">               2</td><td>False.  </td></tr>\n",
       "<tr><td>WV     </td><td style=\"text-align: right;\">              68</td><td style=\"text-align: right;\">        415</td><td>370-3271</td><td>no          </td><td>no          </td><td style=\"text-align: right;\">              0</td><td style=\"text-align: right;\">     231.1</td><td style=\"text-align: right;\">         57</td><td style=\"text-align: right;\">       39.29</td><td style=\"text-align: right;\">     153.4</td><td style=\"text-align: right;\">         55</td><td style=\"text-align: right;\">       13.04</td><td style=\"text-align: right;\">       191.3</td><td style=\"text-align: right;\">          123</td><td style=\"text-align: right;\">          8.61</td><td style=\"text-align: right;\">        9.6</td><td style=\"text-align: right;\">           4</td><td style=\"text-align: right;\">         2.59</td><td style=\"text-align: right;\">               3</td><td>False.  </td></tr>\n",
       "<tr><td>RI     </td><td style=\"text-align: right;\">              28</td><td style=\"text-align: right;\">        510</td><td>328-8230</td><td>no          </td><td>no          </td><td style=\"text-align: right;\">              0</td><td style=\"text-align: right;\">     180.8</td><td style=\"text-align: right;\">        109</td><td style=\"text-align: right;\">       30.74</td><td style=\"text-align: right;\">     288.8</td><td style=\"text-align: right;\">         58</td><td style=\"text-align: right;\">       24.55</td><td style=\"text-align: right;\">       191.9</td><td style=\"text-align: right;\">           91</td><td style=\"text-align: right;\">          8.64</td><td style=\"text-align: right;\">       14.1</td><td style=\"text-align: right;\">           6</td><td style=\"text-align: right;\">         3.81</td><td style=\"text-align: right;\">               2</td><td>False.  </td></tr>\n",
       "<tr><td>CT     </td><td style=\"text-align: right;\">             184</td><td style=\"text-align: right;\">        510</td><td>364-6381</td><td>yes         </td><td>no          </td><td style=\"text-align: right;\">              0</td><td style=\"text-align: right;\">     213.8</td><td style=\"text-align: right;\">        105</td><td style=\"text-align: right;\">       36.35</td><td style=\"text-align: right;\">     159.6</td><td style=\"text-align: right;\">         84</td><td style=\"text-align: right;\">       13.57</td><td style=\"text-align: right;\">       139.2</td><td style=\"text-align: right;\">          137</td><td style=\"text-align: right;\">          6.26</td><td style=\"text-align: right;\">        5  </td><td style=\"text-align: right;\">          10</td><td style=\"text-align: right;\">         1.35</td><td style=\"text-align: right;\">               2</td><td>False.  </td></tr>\n",
       "<tr><td>TN     </td><td style=\"text-align: right;\">              74</td><td style=\"text-align: right;\">        415</td><td>400-4344</td><td>no          </td><td>yes         </td><td style=\"text-align: right;\">             25</td><td style=\"text-align: right;\">     234.4</td><td style=\"text-align: right;\">        113</td><td style=\"text-align: right;\">       39.85</td><td style=\"text-align: right;\">     265.9</td><td style=\"text-align: right;\">         82</td><td style=\"text-align: right;\">       22.6 </td><td style=\"text-align: right;\">       241.4</td><td style=\"text-align: right;\">           77</td><td style=\"text-align: right;\">         10.86</td><td style=\"text-align: right;\">       13.7</td><td style=\"text-align: right;\">           4</td><td style=\"text-align: right;\">         3.7 </td><td style=\"text-align: right;\">               0</td><td>False.  </td></tr>\n",
       "</tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<driverlessai._utils.Table at 0x1215cdd60>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "telco_churn.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explore the Columns\n",
    "* View column summaries\n",
    "* Update the datatype to be used in modeling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# View all column summaries\n",
    "# print(telco_churn.column_summaries())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- Area Code ---\n",
      "\n",
      " 408|██████████\n",
      "    |████████████████████\n",
      " 510|██████████\n",
      "\n",
      "Data Type: int\n",
      "Logical Types: []\n",
      "Datetime Format: \n",
      "Count: 3333\n",
      "Missing: 0\n",
      "Mean: 437\n",
      "SD: 42.4\n",
      "Min: 408\n",
      "Max: 510\n",
      "Unique: 3\n",
      "Freq: 1655\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(telco_churn.column_summaries([\"Area Code\"]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- Area Code ---\n",
      "\n",
      " 408|██████████\n",
      "    |████████████████████\n",
      " 510|██████████\n",
      "\n",
      "Data Type: int\n",
      "Logical Types: ['categorical']\n",
      "Datetime Format: \n",
      "Count: 3333\n",
      "Missing: 0\n",
      "Mean: 437\n",
      "SD: 42.4\n",
      "Min: 408\n",
      "Max: 510\n",
      "Unique: 3\n",
      "Freq: 1655\n"
     ]
    }
   ],
   "source": [
    "# Force a numeric column to only be used as a category\n",
    "telco_churn.set_logical_types({'Area Code': ['categorical']})\n",
    "print(telco_churn.column_summaries()[\"Area Code\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Split a Dataset\n",
    "The split function returns a dictionary of two datasets so you can easily pass them to the experiments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Complete\n"
     ]
    }
   ],
   "source": [
    "telco_churn_split = telco_churn.split_to_train_test(\n",
    "    train_size=0.8, \n",
    "    train_name='telco_churn_train', \n",
    "    test_name='telco_churn_test', \n",
    "    target_column= \"Churn?\", # Beta users with client from before March 15th use target_col\n",
    "    seed=42\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'train_dataset': <class 'Dataset'> 0cb68884-fcc2-11ec-88b7-66b02643127c telco_churn_train,\n",
       " 'test_dataset': <class 'Dataset'> 0cb6a9c2-fcc2-11ec-88b7-66b02643127c telco_churn_test}"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "telco_churn_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_dataset \t 0cb68884-fcc2-11ec-88b7-66b02643127c telco_churn_train\n",
      "test_dataset \t 0cb6a9c2-fcc2-11ec-88b7-66b02643127c telco_churn_test\n"
     ]
    }
   ],
   "source": [
    "for k, v in telco_churn_split.items():\n",
    "    print(k, \"\\t\" ,v.key, v.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Recipes\n",
    "Recipes are components of the ML pipeline such as algorithms, feature transformers, and scores. You can view all availabe recipes and upload new ones to your instance of DAI.\n",
    "\n",
    "* List all available models\n",
    "* List any custom transformers\n",
    "* List all scorers that can be used for binomial classification\n",
    "* Upload a custom recipe and save it to be tested later"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Constant',\n",
       " 'DecisionTree',\n",
       " 'FTRL',\n",
       " 'GLM',\n",
       " 'ImageAuto',\n",
       " 'ImbalancedLightGBM',\n",
       " 'ImbalancedXGBoostGBM',\n",
       " 'IsolationForestAnomaly',\n",
       " 'KMeans',\n",
       " 'KMeansFreq',\n",
       " 'KMeansOHE',\n",
       " 'LightGBM',\n",
       " 'LightGBMDask',\n",
       " 'RuleFit',\n",
       " 'TensorFlow',\n",
       " 'TextALBERT',\n",
       " 'TextBERT',\n",
       " 'TextCamemBERT',\n",
       " 'TextDistilBERT',\n",
       " 'TextMultilingualBERT',\n",
       " 'TextRoBERTa',\n",
       " 'TextXLM',\n",
       " 'TextXLMRoberta',\n",
       " 'TextXLNET',\n",
       " 'TorchGrowNet',\n",
       " 'TruncSVD',\n",
       " 'Unsupervised',\n",
       " 'XGBoostDart',\n",
       " 'XGBoostDartDask',\n",
       " 'XGBoostGBM',\n",
       " 'XGBoostGBMDask',\n",
       " 'XGBoostRF',\n",
       " 'XGBoostRFDask',\n",
       " 'ZeroInflatedLightGBM',\n",
       " 'ZeroInflatedXGBoost']"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[m.name for m in dai.recipes.models.list()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[t.name for t in dai.recipes.transformers.list() if t.is_custom ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['ACCURACY',\n",
       " 'AUC',\n",
       " 'AUCPR',\n",
       " 'F05',\n",
       " 'F1',\n",
       " 'F2',\n",
       " 'GINI',\n",
       " 'LOGLOSS',\n",
       " 'MACROAUC',\n",
       " 'MACROF1',\n",
       " 'MACROMCC',\n",
       " 'MCC']"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[s.name for s in dai.recipes.scorers.list() if s.for_binomial]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Complete 100.00%\n"
     ]
    }
   ],
   "source": [
    "dai.recipes.create(\"https://github.com/h2oai/driverlessai-recipes/blob/rel-1.8.6/transformers/numeric/sum.py\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['SumTransformer|sum_ef3e7726_content.py']"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[t.name for t in dai.recipes.transformers.list() if t.is_custom ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "transformers_to_use = [t for t in dai.recipes.transformers.list() if not t.is_custom or 'Sum' in t.name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modeling\n",
    "**Notes:** Dictionaries allow you to easily use common settings in your experiments <br/>\n",
    "**Notes:** Experiments will be `sync` by default meaning they will lock the notebook until they are complete. You can also use `async` versions of the fucntions. With the `async` functions you can use included code below to monthior and experiment as it runs, see logs in real time, and stop it when it is \"good enough\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### List Existing Experiments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[e.name for e in dai.experiments.list()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dictionary for a Use Case\n",
    "We might want to run several experiments with different dial and expert settings. All of these will likely have some things in common, namely details about this specific dataset. We will create a dictionary to use in many experiments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "telco_settings = {\n",
    "    **telco_churn_split,\n",
    "    'task': 'classification',\n",
    "    'target_column': \"Churn?\", # Beta users with client from before March 15th use target_col\n",
    "    'scorer': 'F1'\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dictionary for Fast Experiments\n",
    "\n",
    "There may be several common types of experiments you want to run, and H2O.ai will be creating common experiment settings in dictionaries for easy use. The one below turns off all extra settings such as building pipelines or checking for leakage. It also uses the fastest experiment settings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "fast_settings = {\n",
    "    'accuracy': 1,\n",
    "    'time': 1,\n",
    "    'interpretability': 6,\n",
    "    'make_python_scoring_pipeline': 'off',\n",
    "    'make_mojo_scoring_pipeline': 'off',\n",
    "    'benchmark_mojo_latency': 'off',\n",
    "    'make_autoreport': False,\n",
    "    'check_leakage': 'off',\n",
    "    'check_distribution_shift': 'off'\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Search for Settings\n",
    "There are many expert settings available, you can use the serach functionality to look for names or keywords for settings you may want to use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "enable_lightgbm_multiclass_balancing | default_value: auto\n",
      "force_stratified_splits_for_imbalanced_threshold_binary | default_value: 0.01\n",
      "imbalance_sampling_method | default_value: off\n",
      "imbalance_sampling_threshold_min_rows_original | default_value: 100000\n",
      "imbalance_ratio_sampling_threshold | default_value: 5\n",
      "heavy_imbalance_ratio_sampling_threshold | default_value: 25\n",
      "imbalance_sampling_number_of_bags | default_value: -1\n",
      "imbalance_sampling_max_number_of_bags | default_value: 10\n",
      "imbalance_sampling_max_number_of_bags_feature_evolution | default_value: 3\n",
      "imbalance_sampling_max_multiple_data_size | default_value: 1.0\n",
      "imbalance_sampling_target_minority_fraction | default_value: -1.0\n",
      "imbalance_ratio_notification_threshold | default_value: 2.0\n"
     ]
    }
   ],
   "source": [
    "dai.experiments.search_expert_settings('imbalanced')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "enable_lightgbm_multiclass_balancing | default_value: auto | LightGBM multiclass balancing\n",
      "  Whether to enable automatic class weighting for imbalanced multiclass problems. Can make worse probabilities, but improve confusion-matrix based scorers for rare classes without the need to manually calibrate probabilities or fine-tune the label creation process.\n",
      "\n",
      "force_stratified_splits_for_imbalanced_threshold_binary | default_value: 0.01 | Perform stratified sampling for binary classification if the target is more imbalanced than this.\n",
      "  Ratio of minority to majority class of the target column beyond which stratified sampling is done for binary classification. Otherwise perform random sampling. Set to 0 to always do random sampling. Set to 1 to always do stratified sampling.\n",
      "\n",
      "imbalance_sampling_method | default_value: off | Sampling method for imbalanced binary classification problems\n",
      "  Sampling method for imbalanced binary classification problems. Choices are: \"auto\": sample both classes as needed, depending on data \"over_under_sampling\": over-sample the minority class and under-sample the majority class, depending on data \"under_sampling\": under-sample the majority class to reach class balance \"off\": do not perform any sampling\n",
      "\n",
      "imbalance_sampling_threshold_min_rows_original | default_value: 100000 | Threshold for minimum number of rows in original training data to allow imbalanced sampling techniques. For smaller data, will disable imbalanced sampling, no matter what imbalance_sampling_method is set to.\n",
      "  For smaller data, there's no generally no benefit in using imbalanced sampling methods.\n",
      "\n",
      "imbalance_ratio_sampling_threshold | default_value: 5 | Ratio of majority to minority class for imbalanced binary classification to trigger special sampling techniques if enabled\n",
      "  For imbalanced binary classification: ratio of majority to minority class equal and above which to enable special imbalanced models with sampling techniques (specified by imbalance_sampling_method) to attempt to improve model performance.\n",
      "\n",
      "heavy_imbalance_ratio_sampling_threshold | default_value: 25 | Ratio of majority to minority class for heavily imbalanced binary classification to only enable special sampling techniques if enabled\n",
      "  For heavily imbalanced binary classification: ratio of majority to minority class equal and above which to enable only special imbalanced models on full original data, without upfront sampling.\n",
      "\n",
      "imbalance_sampling_number_of_bags | default_value: -1 | Number of bags for sampling methods for imbalanced binary classification (if enabled). -1 for automatic.\n",
      "  -1: automatic\n",
      "\n",
      "imbalance_sampling_max_number_of_bags | default_value: 10 | Hard limit on number of bags for sampling methods for imbalanced binary classification.\n",
      "  -1: automatic\n",
      "\n",
      "imbalance_sampling_max_number_of_bags_feature_evolution | default_value: 3 | Hard limit on number of bags for sampling methods for imbalanced binary classification during feature evolution phase.\n",
      "  Only for shift/leakage/tuning/feature evolution models. Not used for final models. Final models can be limited by imbalance_sampling_max_number_of_bags.\n",
      "\n",
      "imbalance_sampling_max_multiple_data_size | default_value: 1.0 | Max. size of data sampled during imbalanced sampling (in terms of dataset size)\n",
      "  Max. size of data sampled during imbalanced sampling (in terms of dataset size), controls number of bags (approximately). Only for imbalance_sampling_number_of_bags == -1.\n",
      "\n",
      "imbalance_sampling_target_minority_fraction | default_value: -1.0 | Target fraction of minority class after applying under/over-sampling techniques. -1.0 for automatic\n",
      "  A value of 0.5 means that models/algorithms will be presented a balanced target class distribution after applying under/over-sampling techniques on the training data. Sometimes it makes sense to choose a smaller value like 0.1 or 0.01 when starting from an extremely imbalanced original target distribution. -1.0: automatic\n",
      "\n",
      "imbalance_ratio_notification_threshold | default_value: 2.0 | \n",
      "  For binary classification: ratio of majority to minority class equal and above which to notify of imbalance in GUI to say slightly imbalanced. More than ``imbalance_ratio_sampling_threshold`` will say problem is imbalanced.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "dai.experiments.search_expert_settings('imbalanced', show_description=True )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get Reccomended Dial Settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ACCURACY [7/10]:\n",
      "- Training data size: *2,666 rows, 20 cols*\n",
      "- Feature evolution: *[Constant, LightGBM, XGBoostGBM]*, *3-fold CV**, 2 reps*\n",
      "- Final pipeline: *Blend of up to 2 [Constant, LightGBM, XGBoostGBM] models, each averaged across 3-fold CV splits*\n",
      "\n",
      "TIME [2/10]:\n",
      "- Feature evolution: *8 individuals*, up to *10 iterations*\n",
      "- Early stopping: After *5* iterations of no improvement\n",
      "\n",
      "INTERPRETABILITY [8/10]:\n",
      "- Feature pre-pruning strategy: Permutation Importance FS\n",
      "- Monotonicity constraints: enabled\n",
      "- Feature engineering search space: [CVCatNumEncode, CVTargetEncode, Frequent, Interactions, NumCatTE, NumToCatWoEMonotonic, NumToCatWoE, Original, Text, WeightOfEvidence]\n",
      "- Pre-trained PyTorch NLP transformers (without fine-tuning): ['disabled']\n",
      "- Pre-trained PyTorch NLP models (with fine-tuning): ['disabled']\n",
      "\n",
      "[Constant, LightGBM, XGBoostGBM] models to train:\n",
      "- Model and feature tuning: *72*\n",
      "- Feature evolution: *288*\n",
      "- Final pipeline: *6*\n",
      "\n",
      "Estimated runtime: *10 minutes*\n",
      "Estimated mojo_size: *3.6MB*\n",
      "Auto-click Finish/Abort if not done in: *1 day*/*7 days*\n"
     ]
    }
   ],
   "source": [
    "# Get experiment preview with our settings\n",
    "dai.experiments.preview(\n",
    "    **telco_settings\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preview Using Our Custom Recipe\n",
    "Notice that `Sum` has been added to the `Feature engineering search space`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ACCURACY [7/10]:\n",
      "- Training data size: *2,666 rows, 20 cols*\n",
      "- Feature evolution: *[Constant, LightGBM, XGBoostGBM]*, *3-fold CV**, 2 reps*\n",
      "- Final pipeline: *Blend of up to 2 [Constant, LightGBM, XGBoostGBM] models, each averaged across 3-fold CV splits*\n",
      "\n",
      "TIME [2/10]:\n",
      "- Feature evolution: *8 individuals*, up to *10 iterations*\n",
      "- Early stopping: After *5* iterations of no improvement\n",
      "\n",
      "INTERPRETABILITY [8/10]:\n",
      "- Feature pre-pruning strategy: Permutation Importance FS\n",
      "- Monotonicity constraints: enabled\n",
      "- Feature engineering search space: [CVCatNumEncode, CVTargetEncode, Frequent, Interactions, NumCatTE, NumToCatWoEMonotonic, NumToCatWoE, Original, Sum, Text, WeightOfEvidence]\n",
      "- Pre-trained PyTorch NLP transformers (without fine-tuning): ['disabled']\n",
      "- Pre-trained PyTorch NLP models (with fine-tuning): ['disabled']\n",
      "\n",
      "[Constant, LightGBM, XGBoostGBM] models to train:\n",
      "- Model and feature tuning: *72*\n",
      "- Feature evolution: *288*\n",
      "- Final pipeline: *6*\n",
      "- Transformers without MOJO support: *['Sum']*\n",
      "\n",
      "Estimated runtime: *10 minutes*\n",
      "Estimated mojo_size: *3.6MB*\n",
      "Auto-click Finish/Abort if not done in: *1 day*/*7 days*\n"
     ]
    }
   ],
   "source": [
    "# Get experiment preview with our settings\n",
    "dai.experiments.preview(\n",
    "    **telco_settings\n",
    "    ,transformers=transformers_to_use\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Launch an Experiment\n",
    "We will start by running an async experiment which will immeadiatly free our notebook to run additional commands"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Experiment launched at: https://steam.cloud.h2o.ai:443/proxy/driverless/3286/#/experiment?key=271c24a4-fcc2-11ec-88b7-66b02643127c\n"
     ]
    }
   ],
   "source": [
    "default_baseline = dai.experiments.create_async(\n",
    "    **telco_settings, \n",
    "    name='Fastest Settings', **fast_settings,\n",
    "    force=True\n",
    "    # name='Default Baseline', accuracy=7, time=2, interpretability=8\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Information on an Experiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name: Fastest Settings\n",
      "Datasets: {'train_dataset': <class 'Dataset'> 0cb68884-fcc2-11ec-88b7-66b02643127c telco_churn_train, 'validation_dataset': None, 'test_dataset': <class 'Dataset'> 0cb6a9c2-fcc2-11ec-88b7-66b02643127c telco_churn_test}\n",
      "Target: Churn?\n",
      "Scorer: F1\n",
      "Task: classification\n",
      "Status: Running 0.00% - Selecting modeling backend...\n",
      "Web Page: "
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre><a href='https://steam.cloud.h2o.ai:443/proxy/driverless/3286/#/experiment?key=271c24a4-fcc2-11ec-88b7-66b02643127c' rel='noopener noreferrer' target='_blank'>https://steam.cloud.h2o.ai:443/proxy/driverless/3286/#/experiment?key=271c24a4-fcc2-11ec-88b7-66b02643127c</a></pre>"
      ],
      "text/plain": [
       "'https://steam.cloud.h2o.ai:443/proxy/driverless/3286/#/experiment?key=271c24a4-fcc2-11ec-88b7-66b02643127c'"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"Name:\", default_baseline.name)\n",
    "print(\"Datasets:\", default_baseline.datasets)\n",
    "print(\"Target:\", default_baseline.settings['target_column']) # beta users from before March 15th use target_col\n",
    "print(\"Scorer:\", default_baseline.metrics()['scorer'])\n",
    "print(\"Task:\", default_baseline.settings['task'])\n",
    "print(\"Status:\", default_baseline.status(verbose=2))\n",
    "print(\"Web Page: \", end='')\n",
    "default_baseline.gui()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Monitor and Finish the Model Early\n",
    "Example of how you may want to monitor a running experiment, this will print the currently logs and accuracy metrics. You can also finish a model early if it reaches a certain accuracy metric or run time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running 95.00% - Building pipeline visualization... - Validation F1: 0.708 +/- 0.0142\n",
      "\n",
      "2022-07-06 00:26:22,488 C:  2% D:62.9GB  M:28.2GB  NODE:LOCAL1      541    DATA   | total 0\n",
      "2022-07-06 00:26:22,488 C:  2% D:62.9GB  M:28.2GB  NODE:LOCAL1      541    DATA   |\n",
      "2022-07-06 00:26:22,488 C:  2% D:62.9GB  M:28.2GB  NODE:LOCAL1      541    INFO   | Returning from experiment 271c24a4-fcc2-11ec-88b7-66b02643127c after 45.5 seconds.\n",
      "\n",
      "Test F1: 0.6497\n"
     ]
    }
   ],
   "source": [
    "# Monitor the experiment and stop at a nice model\n",
    "import time\n",
    "from IPython.display import clear_output\n",
    "\n",
    "while default_baseline.is_running():\n",
    "    time.sleep(1)\n",
    "    \n",
    "    # grab experiment status\n",
    "    status = default_baseline.status(verbose=2)\n",
    "    \n",
    "    # grab current metrics\n",
    "    metrics = default_baseline.metrics()   \n",
    "    \n",
    "    # pretty print info\n",
    "    clear_output(wait=True)\n",
    "    print(status, \" - Validation \", metrics['scorer'], \": \", sep='', end='')\n",
    "    \n",
    "    if metrics['val_score'] is not None:\n",
    "        print(round(metrics['val_score'], 4), '+/-', round(metrics['val_score_sd'], 4))\n",
    "        if metrics['val_score'] > 0.9:\n",
    "            default_baseline.finish()\n",
    "    else:\n",
    "        print()\n",
    "    \n",
    "    print()\n",
    "    default_baseline.log.tail(3)\n",
    "    time.sleep(1)\n",
    "    \n",
    "print(\"\\nTest \", default_baseline.metrics()['scorer'], \": \", \n",
    "      round(default_baseline.metrics()['test_score'], 4), sep='')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View the Experiment Summary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Status: Complete\n",
      "Experiment: Fastest Settings (271c24a4-fcc2-11ec-88b7-66b02643127c)\n",
      "  Version: 1.10.3, 2022-07-06 00:26\n",
      "  Settings: 1/1/6, seed=776249329, GPUs disabled\n",
      "  Train data: telco_churn_train (2666, 21)\n",
      "  Validation data: N/A\n",
      "  Test data: [Test] (667, 20)\n",
      "  Target column: Churn? (binary, 14.479% target class)\n",
      "System specs: Docker/Linux, 28 GB, 64 CPU cores, 0/0 GPU\n",
      "  Max memory usage: 0.605 GB, 0 GB GPU\n",
      "Recipe: AutoDL (3 iterations, 2 individuals)\n",
      "  Validation scheme: stratified, 1 internal holdout\n",
      "  Feature engineering: 24 features scored (17 selected)\n",
      "Timing:\n",
      "  Data preparation: 6.82 secs\n",
      "  Shift/Leakage detection: 0.00 secs\n",
      "  Model and feature tuning: 10.34 secs (6 models trained)\n",
      "  Feature evolution: 1.24 secs (0 of 3 model trained)\n",
      "  Final pipeline training: 20.04 secs (7 models trained)\n",
      "  Python / MOJO scorer building: 0.00 secs / 5.45 secs\n",
      "Validation score: F1 = 0.2529489 (constant preds of -1.776)\n",
      "Validation score: F1 = 0.6604651 +/- 0.0332954 (baseline)\n",
      "Validation score: F1 = 0.7080182 +/- 0.01416975 (final pipeline)\n",
      "Test score:       F1 = 0.6496815 +/- 0.0350438 (final pipeline)\n"
     ]
    }
   ],
   "source": [
    "default_baseline.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interact with Model Artifacts\n",
    "* See which are available\n",
    "* Create the AutoReport\n",
    "* Download the AutoReport\n",
    "* Open the AutoReport"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Available artifacts: ['logs', 'summary', 'test_predictions', 'train_predictions']\n"
     ]
    }
   ],
   "source": [
    "print(\"Available artifacts:\", default_baseline.artifacts.list())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generating autodoc...\n"
     ]
    }
   ],
   "source": [
    "default_baseline.artifacts.create('autoreport')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloaded 'report.docx'\n"
     ]
    }
   ],
   "source": [
    "artifacts = default_baseline.artifacts.download(['autoreport'], \"./\", overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### View Final Model Performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'scorer': 'F1',\n",
       " 'val_score': 0.708018154311649,\n",
       " 'val_score_sd': 0.014169753774657252,\n",
       " 'val_roc_auc': 0.8737807926552131,\n",
       " 'val_pr_auc': 0.7533498607545223,\n",
       " 'test_score': 0.6496815286624203,\n",
       " 'test_score_sd': 0.03504380042337759,\n",
       " 'test_roc_auc': 0.8358292638813528,\n",
       " 'test_pr_auc': 0.6776371014771233}"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "default_baseline.metrics()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation F1 :\t 0.708\n",
      "Test F1 :\t 0.65\n"
     ]
    }
   ],
   "source": [
    "print(\"Validation\", default_baseline.metrics()[\"scorer\"], \":\\t\",round(default_baseline.metrics()['val_score'], 3))\n",
    "print(\"Test\", default_baseline.metrics()[\"scorer\"], \":\\t\",round(default_baseline.metrics()['test_score'], 3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the Test Set Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloaded 'test_preds.csv'\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Churn?.False.</th>\n",
       "      <th>Churn?.True.</th>\n",
       "      <th>Churn?.predicted(th=0.40483)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.859335</td>\n",
       "      <td>0.140665</td>\n",
       "      <td>False.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.980480</td>\n",
       "      <td>0.019520</td>\n",
       "      <td>False.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.973192</td>\n",
       "      <td>0.026808</td>\n",
       "      <td>False.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.961689</td>\n",
       "      <td>0.038311</td>\n",
       "      <td>False.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.864147</td>\n",
       "      <td>0.135853</td>\n",
       "      <td>False.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Churn?.False.  Churn?.True. Churn?.predicted(th=0.40483)\n",
       "0       0.859335      0.140665                       False.\n",
       "1       0.980480      0.019520                       False.\n",
       "2       0.973192      0.026808                       False.\n",
       "3       0.961689      0.038311                       False.\n",
       "4       0.864147      0.135853                       False."
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Download predictions from test dataset\n",
    "artifacts = default_baseline.artifacts.download(['test_predictions'], \"./\", overwrite=True)\n",
    "local_predictions = pd.read_csv(artifacts['test_predictions'])\n",
    "\n",
    "local_predictions.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot ROC curve\n",
    "* Download the predictions with the Actual column\n",
    "* Use sklearn to calculate ROC curve\n",
    "* Plot AUC & ROC Curve by a categorical column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Complete\n",
      "Downloaded 'test_predictions.csv'\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Area Code</th>\n",
       "      <th>Churn?</th>\n",
       "      <th>Churn?.False.</th>\n",
       "      <th>Churn?.True.</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>415</td>\n",
       "      <td>False.</td>\n",
       "      <td>0.859335</td>\n",
       "      <td>0.140665</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>408</td>\n",
       "      <td>False.</td>\n",
       "      <td>0.980480</td>\n",
       "      <td>0.019520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>415</td>\n",
       "      <td>False.</td>\n",
       "      <td>0.973192</td>\n",
       "      <td>0.026808</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>415</td>\n",
       "      <td>False.</td>\n",
       "      <td>0.961689</td>\n",
       "      <td>0.038311</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>408</td>\n",
       "      <td>False.</td>\n",
       "      <td>0.864147</td>\n",
       "      <td>0.135853</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Area Code  Churn?  Churn?.False.  Churn?.True.\n",
       "0        415  False.       0.859335      0.140665\n",
       "1        408  False.       0.980480      0.019520\n",
       "2        415  False.       0.973192      0.026808\n",
       "3        415  False.       0.961689      0.038311\n",
       "4        408  False.       0.864147      0.135853"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "preds = default_baseline.predict(telco_churn_split['test_dataset'], \n",
    "                         include_columns=[\"Churn?\", \"Area Code\"])\n",
    "\n",
    "test_predictions = pd.read_csv(preds.download(\"./\", dst_file=\"test_predictions.csv\"))\n",
    "\n",
    "test_predictions.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA0PklEQVR4nO3dd3hUZfbA8e9JAoQSSuglIfQeigFERBBQQBDWimIB1hVd64qrYFnXtqu7Fqw/V1AEXeuCJQrKrgVBRQEbHaVJQhEIEEoIpJzfH3eCk5AyIXNnMjPn8zx5mFtm5lzQnPve933PK6qKMcaYyBUV7ACMMcYElyUCY4yJcJYIjDEmwlkiMMaYCGeJwBhjIlxMsAMorwYNGmhSUlKwwzDGmJDy7bff7lHVhsUdC7lEkJSUxPLly4MdhjHGhBQR+aWkY/ZoyBhjIpwlAmOMiXCWCIwxJsKFXB9BcXJyckhPTyc7OzvYoVRasbGxtGjRgipVqgQ7FGNMJRMWiSA9PZ24uDiSkpIQkWCHU+moKhkZGaSnp9OqVatgh2OMqWRcezQkIjNFZJeIrCrhuIjIUyKyQURWiEivk/2u7Oxs6tevb0mgBCJC/fr1rcVkjCmWm30Es4DhpRwfAbTz/EwCnqvIl1kSKJ39/RhTiaUthcWPOX9W5JyT5NqjIVVdJCJJpZwyBnhZnTrYX4tIXRFpqqo73IrJGGMqnbSl8NI5kJ8DEgWNu0K12oVOycvOJGrXakQVYmJhfCok9PFbCMEcNdQcSPPaTvfsO4GITBKR5SKyfPfu3QEJrrxEhFtvvfX49qOPPsq9997r8/t//fVXRo0aRffu3encuTPnnHMOAAsXLmTUqFEnnJ+amsrDDz8MwL333sujjz4KwIQJE5gzZ04FrsQYE1A/vu4kAQDNh+zMQoczj+Tw665fnWMo5B2DLYv9GkJIDB9V1emqmqKqKQ0bFjtDOuiqVavG22+/zZ49e07q/ffccw9nnXUWP/74I2vWrDn+S74ko0ePZurUqSf1XcaYSiJtKSyfVXjf6ZNh4jwyL3mXqbUfovvWm3mw2mTyo2NBoiG6KiQN8GsYwUwE24AEr+0Wnn0hKSYmhkmTJjFt2rQTjm3ZsoXBgweTnJzMkCFD2Lp16wnn7NixgxYtWhzfTk5OPuGcZcuW0bNnTzZu3MisWbO44YYb/HsRxpjyqehz+y2LgfzC+45kkJevXPDcV7y1PI1rBrbm8VsnET3hfRh8l98fC0Fwh4+mAjeIyBtAXyDTX/0DY59fcsK+UclNuaJfEkeO5THhpRP/0S48pQUXpSSw9/Ax/vjvbwsde/Oafj597/XXX09ycjK33357of033ngj48ePZ/z48cycOZObbrqJd99994T3jh07lmeeeYahQ4cyceJEmjVrdvz4V199xY033sh7771HYmIiixf7t2lojCknH57tl+lg4V95GlUFWp5OdJTw57M70KxuLMkt6joHE/r4PQEUcHP46OvAEqCDiKSLyFUicq2IXOs5ZT6wCdgAzACucyuWQKlduzZXXnklTz31VKH9S5YsYdy4cQBcccUVfPHFFye8d9iwYWzatImrr76adevW0bNnTwr6Q9auXcukSZN4//33SUxMdP9CjAlX/hx5s2Vxqc/2fZKf+9tLhP/kD+KNnU0BGN61yW9JwGVujhq6tIzjClzvxneXdgdfvWp0qcfja1b1uQVQnD/96U/06tWLiRMnlvu98fHxjBs3jnHjxjFq1CgWLVpE/fr1adq0KdnZ2Xz//feFWgnGmHLwxx28tyJ385w+GVImlDum/NnnornHOKYxfBc/nKta1jv5mE5SSHQWh5L4+HguvvhiXnzxxeP7TjvtNN544w0AXn31VQYMOLGj59NPPyUrKwuAgwcPsnHjxuN3/3Xr1mXevHnccccdLFy40P2LMCZUlXbH7487eG9ed/MQBUcyyv0R72U054qcu3ky/2I+7j2Dv914Fe0ax1UsrpMQFiUmKptbb72VZ5555vj2008/zcSJE3nkkUdo2LAhL7300gnv+fbbb7nhhhuIiYkhPz+fP/zhD/Tu3fv4L/7GjRvzwQcfMGLECGbOnBmoSzEmdJR1x++PO/ii3zd7tDOc8yRH8tSpXgVJ6MNF519FQnyNk4+lgsR5QhM6UlJStOjCNGvXrqVTp05Biih02N+TCVtpS2HhQ7Dx09/21W0JdbwGJmamwf6CtVmiYMjdMOBWKiRtqdPSSBrgU0dubl4+L36xmZy8fG4Y3A5waoEFYua/iHyrqinFHbMWgTEmtHm3BLwVveP3wx38CcoxkmfN9gNMmbuCldsyGZnc9HgCqAzlXywRGBOJynknW6l5P/s/rphn9gl9nDH4Ab7uo7l5PPPpBp5buJG6Narwf5f1YkTXJpUiARSwRGBMpPH36JlgK/rsnyiIqVb8Hb+LY/FLsmVPFv/6fCOjezTjLyM7U69m1YB+vy8sERgTaYobPRPKiaDQ6B2BNoNg0B1BbekcPprL/9b8yu96NqdDkzg+mTyIxPrB6wwuiyUCY0JRRR7tJA0ABFCIrgYXvBDaj4eKPvsPchJY/PNu7nh7Jdv2H6Fr89q0bRRXqZMAWCIwJvRU9NHO0QNAwWjB0Bo1WKwgPfsvKjMrh7/NX8Nby9Np3aAmb07qR9tGgZ8TcDJsQpmf1KpVq8KfsXz5cm666aYSj2/ZsoXXXnvN5/NNiCqrDEJFJ0Z5n5+f5/eSxkGR0McZChqkJJCXr1zwr6+Y+902rhvUhvk3D6BPq/igxHIyrEVQiaSkpJCSUuwwX+C3RFBQt6is800I8uVuv6ITo9wYRhmh9h4+Rt3qVYiOEm4b1oHmdavTtXmdYIdVbpHbInBx2bcCP/zwA6eeeirJycmcd9557Nu3D3DKSScnJ9OjRw9uu+02unbtChRehObzzz+nR48e9OjRg549e3Lw4EGmTp3K4sWL6dGjB9OmTSt0/qFDh5g4cSLdunUjOTmZuXPnunZdxiUFk6LKutuvaGmDgkcpLpU0jgSqytxv0znz0YW8scxZX2tYlyYhmQQgHFsEH06FnStLP+foAfh1lfM/mi/PWJt0gxGlLxRTnCuvvJKnn36agQMHcs8993DffffxxBNPMHHiRGbMmEG/fv1KXFzm0Ucf5dlnn6V///4cOnSI2NhYHn74YR599FE++OADgEJ1hx544AHq1KnDypXOtRckHRMifJ0UVXBuRe/ogzCMMlyk78vizndWsein3ZzSsl5IPQIqSWS2CLIzPcu+4Z/iU8XIzMxk//79DBw4EIDx48ezaNEi9u/fz8GDB+nXz6lwWvCYp6j+/fszefJknnrqKfbv309MTOk5++OPP+b6638r5lqvXuArGEYkf7UsfZ0UBXZHH0TvfJ/OsGmLWL5lL/eN7sJ/rulH20YV7x8MtvBrEfhy5170jqoSDp+bOnUqI0eOZP78+fTv358FCxYEOyRTlD8nZpVnUhTYHX2QxNesxilJ8fz9vK60qFe5h4SWR2S2CAJwR1WnTh3q1at3fCWxV155hYEDB1K3bl3i4uL45ptvAI6Xpy5q48aNdOvWjSlTptC7d2/WrVtHXFwcBw8eLPb8s846i2efffb4tj0aqiBf7vT9Wda4uElRdrcfdDl5+fzfwg089cnPAAxs35DZE3uHVRKAcGwR+MrPd1RZWVmF1hyePHkys2fP5tprryUrK4vWrVsfLz/94osvcvXVVxMVFcXAgQOpU+fEDqYnnniCzz77jKioKLp06cKIESOIiooiOjqa7t27M2HCBHr27Hn8/Lvvvpvrr7+erl27Eh0dzV//+lfOP/98v11fRPH1Tt+fZY0r2aQoA6u2ZTJl7gpWbz/Aud2bVaoicf5mZaiD4NChQ8fnHTz88MPs2LGDJ5980vXvDbW/p6BZ/Bh8cv9v20XLGRfwd1njcCoEF8Kyc/J46pOfeX7RJurVqMqDv+vC8K5Ngx1WhVkZ6kpm3rx5PPTQQ+Tm5tKyZUtmzZoV7JCMt+wDhbe7nAdn3Xfief4ej2/P/SuFXzKymLF4E+f3bM7dIztTp0aVYIfkOksEQTB27FjGjh0b7DCMN++78Z0rCh8rul2gkpQ2MBV3+GguC1bv5PxeLejQJI5Pbx0U1BXDAi1sEkGgVvkJVaH2CDCgivYJVC8yLrxJcsnvtbv4kPf5T7u58+2VbM88QnKLOrRtFBdRSQDCZNRQbGwsGRkZ9suuBKpKRkYGsbGxwQ4luEoaCVR09E/RETyxIVyi2ZRo3+FjTH7rB8bPXEpslSjPnIDQKBLnb2HRImjRogXp6ens3r072KFUWrGxsYVGNUWc0kYCFR39c8oE+OZ5q8UTxgqKxP2SkcUNZ7blhsFtia0SHeywgiYsEkGVKlVo1apVsMMwFeXmqJnSFmMpWrsntrY9+w9TGYeOUq9GVaKjhKnDO9K8XnW6NAvN+kD+FBaJwIQBt5dPLG3Mf3Gjf+zZf1hRVf7zbToPfrCGKSM6clnflpzdpUmww6o0LBGY4Cup6qY/E0FpFTtt9E9YS9ubxZ3vrGTxz3vokxRPv9b1gx1SpWOJwARXeapuVvR7Shvzby2AsPT2d+nc/e4qBHjgd125rE8iUVE2urAoSwQmuH583feqmxVhd/0RqUGtavRpFc/fzutG87rVgx1OpWWJwARP2lJYPqvIzjKqblaE3fWHvZy8fJ7/fCN5+XDz0Hac0b4hZ7RvGOywKj1LBCY4CvoFyC+8v80gK7hmTsqqbZncNmcFa3ccYEyPZjbJtBwsEZjAK6lfIKqKJQFTbtk5eTzx8c/MWLyJ+JpVef6KUxhmI4LKxdWZxSIyXETWi8gGETlhTUYRSRSRz0TkexFZISLnuBmPcZmvq3UVuxqXQK/LLQmYctu6N4sXv9jEhb1a8PEtAy0JnATXWgQiEg08C5wFpAPLRCRVVdd4nXY38JaqPicinYH5QJJbMRkXlWceQEmrcXUvftlOY4o6mJ3DR6t2clFKAu0bx/HZnweF3WIxgeTmo6E+wAZV3QQgIm8AYwDvRKBAwW+LOsB2F+MxbinvPIDiVuOyR0LGR5+t28Vd76xk54FseibWpW2jOEsCFeRmImgOpHltpwN9i5xzL/BfEbkRqAkMLe6DRGQSMAkgMTHR74GaCjiZeQC2Gpc5CXsPH+OBD9bwzvfbaNeoFnP+eFrEFonzt2B3Fl8KzFLVx0SkH/CKiHRV1UJDSVR1OjAdnBXKghCnKUmxz/vLmAdgY/pNOeXlKxc+9xVb92Zx05B2XH9mG6rFRG6ROH9zMxFsA7zX92vh2eftKmA4gKouEZFYoAGwy8W4jD8lDXD6BApyt0RBtA/zAGxMv/HB7oNHqV/TKRJ35zmdaF6vOp2aWllwf3Nz1NAyoJ2ItBKRqsAlQGqRc7YCQwBEpBMQC1gt6cqquFFBCX2gXiuIrQv9/wSD73bu9u2XvKkAVeXNZVsZ/NhCXlu6FYChnRtbEnCJay0CVc0VkRuABUA0MFNVV4vI/cByVU0FbgVmiMgtOB3HE9RWl6mcShoVdPQA7N3onPPN85YETIVtzchi6tsr+GpjBn1bxXN62wbBDinsudpHoKrzcYaEeu+7x+v1GqC/mzEYP/GuCeQ9Kig787dz8o45z/0tEZiTNOfbdP7y7iqio4S/ndeVS3tbkbhACHZnsQkFxdUEKhgVVFZVT2PKoXHtapzWpj4PnteVpnWsSFygWCIwZfvxdU6oCVQwKshGAJkKOJabz3MLN5Kvyi1ntWdAu4YMaGdF4gLNEoEpXXGtgagqhe/8bQSQOQk/pu3n9jkrWP/rQc7v2dyKxAWRJQJTui2LOaE1YDWBTAUcOZbH4/9bz4tfbKZRXCwvXJnC0M6Ngx1WRLNEYEqXNAAQnEFdOHMErCaQqYC0fVnM/uoXLumTyNQRHakdWyXYIUU8SwSmbNXrQc4RaDsE+t9srQFTbgc8ReIu9hSJW3jbIJrZimGVhqtlqE2IS1sKM4fDkb2QewR+/l+wIzIh6NN1v3L244uYOncFG3YdArAkUMlYIohUvqwdsGUxaN5v2wXzBIzxQcaho9z8xvf8ftZy6lSvwtvX9adto1rBDssUwx4NRSJf1w4oum5AVIzNEzA+yctXLvrXEtL2ZXHL0Pb8cVAbqsbYfWdlZYkg0pRn7YCi6wbYaCFThl0Hs2lQsxrRUcJdIzvRol4NOjSxUtGVnc+JQERqqGqWm8EYl5V37YCis4ZttJApQX6+8vqyrTw0fx1TRnTkilNbMqSTDQkNFWUmAhE5DXgBqAUkikh34BpVvc7t4IyflXftAJs1bHywZc9hpr69gq837eW0NvUZaDODQ44vLYJpwDA8JaRV9UcROcPVqIw7TmbtAJs1bErx1vI0/vLuKqpGR/Hw+d0Y2zvBZgeHIJ8eDalqWpF/3LySzjWVSNrSwnfzBWsHZGXAKRMgtrbd6ZsKaV63Ome0b8gDY7rSpE5ssMMxJ8mXRJDmeTykIlIFuBlY625YpsKKGxkEtnaAqZCjuXn832cbUVUmn92B/m0b0N/WCwh5voznuha4Hmcx+m1AD8D6Byo77/6AgpFBxa0dYIyPvt+6j3Of/oInP/mZbfuzsTWkwocvLYIOqnqZ9w4R6Q986U5Ixi+yDxTe7nIedBxpaweYcss6lstj//2JmV9upkntWGZOSGFwRxsRFE58SQRPA7182GcCpeiz/+LsXHHi9ln32SggU27b9h3hla9/4bK+iUwZ3pE4KxIXdkpMBCLSDzgNaCgik70O1cZZg9gEg6+zgvduLrzdJNn500YBGR9kHsnhw5U7uKRPIu0ax/H5bYNsxbAwVlqLoCrO3IEYwHtq4AHgQjeDMqUo7tl/cYkg2vufVpwRQsb44L+rd3L3u6vIOHyMlKR42jaqZUkgzJWYCFT1c+BzEZmlqr8EMCZTGu+5ADHV4YIXir/Dt7WETTntOXSUe1NX88GKHXRsEscL41OsSFyE8KWPIEtEHgG6AMcHCqvqYNeiMiVL6OM8DsrOLDkJFJxn/QHGR3n5yoXPfcX2/dn8+ez2XDOwDVWirUhcpPAlEbwKvAmMwhlKOh7Y7WZQEceXzt+TYf0Bpgy/HsimYS2nSNxfz+1Ci3rVadfYisRFGl8SQX1VfVFEbvZ6XLTM7cAihq+dvwWOHvhtRNDs0TYpzJyU/Hzl1aVb+ceH65gyvANX9EvizI6Ngh2WCRJfEkFBlbIdIjIS2A7EuxdShPG187dAcZPCLBGYcti0+xBT317J0s17Ob1tAwZ1sAQQ6XxJBA+KSB3gVpz5A7WBP7kZVETxtfO3gHUCmwp4c9lW7nlvNdViovjnhclcdEoLKxJnyk4EqvqB52UmcCYcn1ls/CGhD9Rq6pSC7ntN2Xf31glsKqBFvRoM6uAUiWtU24rEGUdpE8qigYtxagx9pKqrRGQUcCdQHegZmBDD3PJZcHCb8/rLJ5zqoMUtEuPNOoGNj47m5vH0JxsA+PMwKxJnilfa+LAXgT8A9YGnROTfwKPAP1XVksDJKrpo/Nr3Ch8vum3MSfr2l72c8+RinvlsA7sOWpE4U7LSHg2lAMmqmi8iscBOoI2qlrCclSlTcSOEDhf56ywoBWHMSTp8NJdHFqxn9pItNKtTndm/78PA9rZqmClZaS2CY6rOUlaqmg1sKm8SEJHhIrJeRDaIyNQSzrlYRNaIyGoRea08n1/pFb37L26EkJWCMH62ff8RXlu6lStPbcmCW86wJGDKVFqLoKOIFJSwFKCNZ1sAVdVSb109fQzPAmcB6cAyEUlV1TVe57QD7gD6q+o+EQmfcWzF3f0fO1z4nNMnQ+PONgrIVFhmVg7zVu5gXF+nSNzi28+ksXUGGx+Vlgg6VfCz+wAbVHUTgIi8AYwB1nidczXwrKruA1DVXRX8zuDyniFc3N1/IZ5F420UkKmgj1bt5C/vrWLv4WP0bR1Pm4a1LAmYcimt6FxFC801B9K8ttOBvkXOaQ8gIl/ilLa+V1U/KvpBIjIJmASQmJhYwbBcUrQFUKtp4eOl3f3bKCBzEnYdzObe1NXMX7mTzk1r89KE3rRpaEXiTPn5tHi9y9/fDhgEtAAWiUg3Vd3vfZKqTgemA6SkpFSuoQ8FrYDM9MItgBzvx0B292/8Ky9fufhfS9iemc1twzow6YzWViTOnDQ3E8E2IMFru4Vnn7d04BtVzQE2i8hPOIkhNGoZebcCijplgrNAvN39Gz/akXmExnGxTpG40V1IqFfDSkWbCvPpFkJEqotIh3J+9jKgnYi0EpGqwCVAapFz3sVpDSAiDXAeFW0q5/cETmmjgAqJckb/jE+FwXdZYThTYfn5yqwvNzPksc/59zfOU9szOzSyJGD8oswWgYicizORrCrQSkR6APer6ujS3qequSJyA7AA5/n/TFVdLSL3A8tVNdVz7GwRWQPkAbdV2nkKvowCiqriPBYqaAHY3b/xgw27DjF17gqW/7KPM9o3ZLBVCTV+5sujoXtxRgAtBFDVH0SklS8frqrzgflF9t3j9VqByZ6fyqXoGgG+jALqdTnUSbDn/8Zv3li6lXtSV1O9SjSPXdSd83s1tyJxxu98KkOtqplF/uOrXB22/naycwC6j7MEYPwqsX4NhnZqxH2ju9IwrlqwwzFhypdEsFpExgHRnglgNwFfuRtWEKUthYUP2RwAExTZOXk89cnPANw+vCOntWnAaW2sSJxxly+J4EbgLuAo8BrOc/0H3QwqaEoaBWRzAEwALN+yl9vnrmDT7sNc0jsBVbXHQCYgfEkEHVX1LpxkEL6KtgSOs7t/465DR3N55KN1vPz1LzSvW52Xf9+HM6w+kAkgXxLBYyLSBJgDvKmqq1yOKfBKnA8QBTHV7O7fuGpn5hHeWJbG+H5J3DasAzWrBXuep4k0Zc4jUNUzcVYm2w08LyIrReRu1yMLpB9fLyYJCLQZZHMAjCv2HT7GK1878wHaNnKKxN07uoslARMUPk0oU9WdqvoUcC3wA3BP6e8IIWlLnVXCComCmFgYdIclAeNXqsr8lTs4a9rn3Je6mo27DwHYspEmqHyZUNYJGAtcAGQAb+IsZB/6CvoFyC+8v80gSwLG73YdyOYv761iwepf6da8Di//vq8ViTOVgi/t0Jk4v/yHqep2l+MJnJL6BaKqWBIwfpeXr1z0/BJ2ZmZzx4iOXHV6K2KsSJypJMpMBKraLxCBBFyxdYLEmR1sScD4yfb9R2hS2ykSd/+YriTUq05rawWYSqbEWxIRecvz50oRWeH1s9Jr5bLQlX2gyA5x+gW6jwtKOCa85OUrLxUpEjewfUNLAqZSKq1FcLPnz1GBCCSg0pbC2iKFUONbwXnPW2vAVNiGXQe5fc4Kvtu6n0EdGjKkU+Ngh2RMqUpboWyH5+V1qjrF+5iI/AOYcuK7QkBJfQOdRlsSMBX22jdbuTd1NTWrRTNtbHd+18OKxJnKz5feqrOK2TfC34EETEl9A7G1gxKOCS9JDWpwdpfG/G/yQM7r2cKSgAkJJbYIROSPwHVA6yJ9AnHAl24H5pqkAU5FUfUMGZUoiPaaPWxMOWTn5DHt458QhKkjrEicCU2l9RG8BnwIPARM9dp/UFX3uhqVmxL6OGWlszOdYnJHMqx2kDkp32zKYOrbK9m85zCX9U20InEmZJWWCFRVt4jI9UUPiEh8SCeDarWdn5QJwY7EhKCD2Tn846N1/PvrrSTG1+C1P/TltLbWCjChq6wWwSjgW5yFaLxvdRRo7WJcxlRavx44ypxv0/nD6a2YfHZ7alS1+kAmtJU2amiU50+flqU0JpztPXyMeSu2c0W/JNo2qsXi2wfbimEmbPhSa6g/8IOqHhaRy4FewBOqutX16IwJMlXlgxU7uDd1NQeyc+jftgGtG9ayJGDCii/DR58DskSkO06xuY3AK65GZUwl8OuBbK5++VtufP17mterzvs3nm4zg01Y8uXhZq6qqoiMAZ5R1RdF5Cq3AzMmmPLylYs9ReLuOqcTE/snWZE4E7Z8SQQHReQO4ApggIhEAVXcDcuY4Ejfl0XTOtWJjhIeGNOVxPgaJDWoGeywjHGVL7c4Y3EWrv+9qu4EWgCPuBqVMQGWl6+8sHgTQx//nH97Vg47o31DSwImIvhShnqniLwK9BaRUcBSVX3Z/dCMCYz1Ow9y+9wV/Ji2nyEdG3F2FysSZyKLL6OGLsZpASzEmUvwtIjcpqpzXI7NGNf9++tfuO/91cTFVuHJS3owunszmx1sIo4vfQR3Ab1VdReAiDQEPgYsEZiQVVAOom2jWpzTrSn3jOpM/Vo2JNREJl8SQVRBEvDIwMdF7yultKWQseG311ZjKKIcOZbH4/9bT1SUcMeITpzauj6ntq4f7LCMCSpffqF/JCILRGSCiEwA5gHz3Q3LJWlLYeZwOLTT+Zk1ytlnIsKSjRkMf3IRMxZvJutoHqoa7JCMqRR86Sy+TUTOB0737Jququ+4G5ZLtiwGzfttO++Ys89aBWHtQHYOD81fx+tLt9Kyfg1eu7qvlYo2xktp6xG0Ax4F2gArgT+r6rZABeaKousUR0XbOgQRYNeBo7z7/TYmndGaW4a2p3rV6GCHZEylUtqjoZnAB8AFOBVIny7vh4vIcBFZLyIbRGRqKeddICIqIinl/Y5y2bmi8HbT7tYaCFMZh44y68vNALRtVIsvppzJned0siRgTDFKezQUp6ozPK/Xi8h35flgEYkGnsVZ6jIdWCYiqaq6psh5ccDNwDfl+fyT0mkMbPz0t+2eV7r+lSawVJXUH7dzb+pqDh3N5Yz2DWndsJaNCDKmFKUlglgR6clv6xBU995W1bISQx9gg6puAhCRN4AxwJoi5z0A/AO4rZyxl1/jzlA1DvKOwqnX2cI0YWb7/iPc/e4qPl23ix4JdfnnhclWJM4YH5SWCHYAj3tt7/TaVmBwGZ/dHEjz2k4H+nqfICK9gARVnSciJSYCEZkETAJITEws42tLUDBiqKCz+OvnoONIezQUJnLz8rlk+tfsPniUv4zqzITTkoiOsolhxviitIVpznTziz3F6x4HJpR1rqpOB6YDpKSknNyYPxsxFJbS9mbRrG51YqKj+Pt53UiMr0Fi/RrBDsuYkOLmxLBtQILXdgvPvgJxQFdgoYhsAU4FUl3rME4aQKHVNqOr2oihEJabl8/0RRsZ+vjnvLJkCwCnt2tgScCYk+DmYqvLgHYi0gonAVwCjCs4qKqZwPHB3CKyEGeI6nJXoknoA026waFd0PEc6H6ptQZC1NodB5gydwUr0jM5q3NjRnRrGuyQjAlpriUCVc0VkRuABUA0MFNVV4vI/cByVU1167tLVK228zNqWsC/2vjHK0u2cN/7a6hTvQrPjOvJyG5NrUicMRXkS/VRAS4DWqvq/SKSCDRR1TJrM6jqfIqUo1DVe0o4d5BPEZuIVFAkrn3jOM7t3oy/jOpMfM2qwQ7LmLDgS4vg/4B8nFFC9wMHgblAbxfjMgaArGO5PLrgJ2KihTvP6UTf1vXpa0XijPErXzqL+6rq9UA2gKruA+xWzLjuyw17GPbEImZ+uZljuflWJM4Yl/jSIsjxzBJWOL4eQb6rUZmIlnkkh7/PW8uby9No1aAmb13Tjz6t4oMdljFhy5dE8BTwDtBIRP4GXAjc7WpUJqLtOXSU91ds59qBbfjT0HbEVrH6QMa4yZcy1K+KyLfAEJyB+L9T1bWuR2Yiyu6DR3n/x+38/vRWtGlYiy+mDLbOYGMCxJdRQ4lAFvC+9z5V3epmYCYyqCrv/rCN+95fQ9bRPM7s2IhWDWpaEjAmgHx5NDQPp39AgFigFbAe6OJiXCYCbNt/hLveWcnC9bvplegUiWvVoGawwzIm4vjyaKib97anUNx1rkVkIoJTJG4JGYeOce+5nbminxWJMyZYyj2zWFW/E5G+ZZ9pzIm2ZmTRvJ5TJO7h85NJjK9BQrzVBzImmHzpI5jstRkF9AK2uxaRCUu5efnMWLyZaR//xB0jOjKxfyv6t7V1g42pDHxpEcR5vc7F6TOY6044Jhyt3p7JlLkrWLXtAMO6NGakFYkzplIpNRF4JpLFqeqfAxSPCTOzv9rCAx+soW6Nqjx3WS+rFGpMJVRiIhCRGE8F0f6BDMg1aUshY8Nvr60EtasKisR1bBLHmB7N+cuoTtStYUNCjamMSmsRLMXpD/hBRFKB/wCHCw6q6tsux+Y/RZepnDUKJnxgycAFh4/m8siC9VSJFu4a2dmKxBkTAnzpI4gFMnCqjxbMJ1AgdBKBLVMZEIt+2s0db69ke+YRxvdLOt4qMMZUbqUlgkaeEUOr+C0BFAitMpDHl6n0hG3LVPpVZlYOD8xbw5xv02nd0CkS1zvJisQZEypKSwTRQC0KJ4ACoZUIbJlKV+05fJQPV+7gukFtuGmIFYkzJtSUlgh2qOr9AYvEbbZMpV/tOphN6g/b+cOA1seLxNWz+kDGhKTSEoE93DUnUFXmfreNBz5Yw5GcPIZ0akyrBjUtCRgTwkpLBEMCFoUJCWl7s7jznZUs/nkPKS3r8fAFViTOmHBQYiJQ1b2BDMRUbrl5+Vw642v2HT7GA2O6cFnflkRZkThjwkK5i86ZyLJlz2ES4msQEx3FPy90isS1qGdF4owJJ74sXm8iUE5ePs9+toGzpy3i5SVbADitTQNLAsaEIWsRmBOs2pbJ7XNWsGbHAUZ2a8qo5GbBDskY4yJLBKaQl77czIPz1hJfsyr/uvwUhndtEuyQjDEus0RggN+KxHVpVofzezbn7pGdqVOjSrDDMsYEgCWCCHfoaC7//GgdVaOjuHtUZ/q0iqdPKysPYUwksc7iCLZw/S6GTVvEK1//guK0CowxkcdaBBFo3+FjPDBvDW9/t422jWox59rTOKVlvWCHZYwJEksEEWhf1jH+u/pXbhrclusHt6VajBWJMyaSufpoSESGi8h6EdkgIlOLOT5ZRNaIyAoR+UREWroZTyTbdSCb6Ys2oqq0bliLL6cMZvLZHSwJGGPcSwSe9Y6fBUYAnYFLRaRzkdO+B1JUNRmYA/zTrXgilary1rI0hjz+OY/99ye2ZGQB2IggY8xxbj4a6gNsUNVNACLyBjAGWFNwgqp+5nX+18DlLsYTcdL2ZnHH2yv5YsMe+rSK5+Hzu1mROGPMCdxMBM2BNK/tdKBvKedfBXxY3AERmQRMAkhMTPRXfGGtoEjc/qwcHvxdV8b1SbQiccaYYlWKzmIRuRxIAQYWd1xVpwPTAVJSUmyMYyk27zlMoqdI3CMXdqdl/Ro0q1s92GEZYyoxNzuLtwEJXtstPPsKEZGhwF3AaFU96mI8YS0nL5+nP/mZYdMWMfurLQD0a1PfkoAxpkxutgiWAe1EpBVOArgEGOd9goj0BJ4HhqvqLhdjCWsr0vdz+5wVrNt5kHO7N2N0DysSZ4zxnWuJQFVzReQGYAEQDcxU1dUicj+wXFVTgUeAWsB/RARgq6qOdiumcDTzi808OG8NDeOqMePKFM7q3DjYIRljQoyrfQSqOh+YX2TfPV6vh7r5/eGsoEhccos6jO2dwNQRnahT3YaEGmPKr1J0FhvfHczO4eEP11EtJpp7zu1MSlI8KUlWJM4Yc/Ks6FwI+WzdLs6etojXl24lJlqsSJwxxi+sRRAC9h4+xv3vr+bdH7bTvnEt/u+y0+iZaEXijDH+YYkgBGQeyeGTtbu4eUg7rj+zLVVjrCFnjPEfSwSV1M7MbN79YRvXnNGaVg1q8sXUwdYZbIxxhSWCSkZVeWNZGn+ft5ac/HyGd2lCUoOalgSMMa6xRFCJ/JJxmKlzV7JkUwanto7n4fOTSbIiccYYl1kiqCRy8/IZN+MbMo/k8PfzunFJ7wQrEmeMCQhLBEG2cfchWnqKxD12sVMkrmkdqw9kjAkcG34SJMdy83ni458Y/sQiXl7yCwCntq5vScAYE3DWIgiCH9L2M2XOCtb/epAxPZrxu57Ngx2SMSaCWSIIsBe/2Mzf5q2hUVwsL45PYUgnKxJnjAkuSwQBUlAkrkdCHS7pk8jUER2pHWtDQo0xwWeJwGUHsnN4aP46YqtE8ddzu3BKy3hOaWlF4owxlYd1Frvo4zW/ctbjn/Pmsq1UjYmyInHGmErJWgQuyDh0lPveX0Pqj9vp2CSO6Vek0D2hbrDDMsaYYlkicMHB7Fw+W7+LW4a254+D2liROGNMpWaJwE+27z/CO99v47pBbUhqUJMvpw62zmBjTEiwRFBB+fnKa0u38vCH68jLV0Z2a0pSg5qWBIwxIcMSQQVs3nOYqXNX8M3mvfRvW5+HzksmsX6NYIdljDHlYongJOXm5XP5C99wIDuHf16QzEUpLRCxInHGmNBjiaCcNuw6SFL9msRERzFtbA9a1q9B49qxwQ7LGGNOmg1n8dHR3Dwe/99PDH9iMbM9ReL6tIq3JGCMCXnWIvDBd1v3MWXOCn7edYjzezbnfCsSZ4wJI5YIyjBj0Sb+/uFamtaO5aWJvTmzQ6Ngh2SMMX5liaAE+flKVJTQq2VdLuubyJThHYmzIaHGmDBkiaCIzCM5/G3eGqpXiea+MV2tSJwxJuxZZ7GXBat3ctbjnzP3u23UrBZjReKMMRHBWgTAnkNH+et7q5m3cgedm9Zm5oTedG1eJ9hhGWNMQFgiAA5l57L4593cNqwDk85oTZVoaygZYyJH5PzGO3oAMtMgbSkA2/Yf4ZlPf0ZVSWpQk6/uGML1Z7a1JGCMiTiu/tYTkeEisl5ENojI1GKOVxORNz3HvxGRJFcCSVsKv66C/b+gs0fz4Yfvcfbjn/PsZxv5JSMLgFrVrHFkjIlMriUCEYkGngVGAJ2BS0Wkc5HTrgL2qWpbYBrwD1eC2bIYNB+A/NxsVn4xj14t6/HfW84gqUFNV77SGGNChZstgj7ABlXdpKrHgDeAMUXOGQPM9ryeAwwRNyq3Va+PAgpEoZzZqyMv/74PCfFWKdQYY9xMBM2BNK/tdM++Ys9R1VwgE6hf9INEZJKILBeR5bt37y5/JEcyEMDJMFH0bqRWKdQYYzxComdUVaeraoqqpjRs2LD8H5A0AGKqg0QjMdWcbWOMMYC7w0e3AQle2y08+4o7J11EYoA6QIbfI0noA+NTnb6CpAHOtjHGGMDdRLAMaCcirXB+4V8CjCtyTiowHlgCXAh8qm5N503oYwnAGGOK4VoiUNVcEbkBWABEAzNVdbWI3A8sV9VU4EXgFRHZAOzFSRbGGGMCyNXB86o6H5hfZN89Xq+zgYvcjMEYY0zpQqKz2BhjjHssERhjTISzRGCMMRHOEoExxkQ4CbXFV0RkN/DLSb69AbDHj+GEArvmyGDXHBkqcs0tVbXYGbkhlwgqQkSWq2pKsOMIJLvmyGDXHBncumZ7NGSMMRHOEoExxkS4SEsE04MdQBDYNUcGu+bI4Mo1R1QfgTHGmBNFWovAGGNMEZYIjDEmwoVlIhCR4SKyXkQ2iMjUYo5XE5E3Pce/EZGkIITpVz5c82QRWSMiK0TkExFpGYw4/amsa/Y67wIRUREJ+aGGvlyziFzs+bdeLSKvBTpGf/Phv+1EEflMRL73/Pd9TjDi9BcRmSkiu0RkVQnHRUSe8vx9rBCRXhX+UlUNqx+cktcbgdZAVeBHoHORc64D/uV5fQnwZrDjDsA1nwnU8Lz+YyRcs+e8OGAR8DWQEuy4A/Dv3A74Hqjn2W4U7LgDcM3TgT96XncGtgQ77gpe8xlAL2BVCcfPAT7EWX33VOCbin5nOLYI+gAbVHWTqh4D3gDGFDlnDDDb83oOMERCexHjMq9ZVT9T1SzP5tc4K8aFMl/+nQEeAP4BZAcyOJf4cs1XA8+q6j4AVd0V4Bj9zZdrVqC253UdYHsA4/M7VV2Esz5LScYAL6vja6CuiDStyHeGYyJoDqR5bad79hV7jqrmAplA/YBE5w5frtnbVTh3FKGszGv2NJkTVHVeIANzkS//zu2B9iLypYh8LSLDAxadO3y55nuBy0UkHWf9kxsDE1rQlPf/9zK5ujCNqXxE5HIgBRgY7FjcJCJRwOPAhCCHEmgxOI+HBuG0+haJSDdV3R/MoFx2KTBLVR8TkX44qx52VdX8YAcWKsKxRbANSPDabuHZV+w5IhKD05zMCEh07vDlmhGRocBdwGhVPRqg2NxS1jXHAV2BhSKyBedZamqIdxj78u+cDqSqao6qbgZ+wkkMocqXa74KeAtAVZcAsTjF2cKVT/+/l0c4JoJlQDsRaSUiVXE6g1OLnJMKjPe8vhD4VD29MCGqzGsWkZ7A8zhJINSfG0MZ16yqmaraQFWTVDUJp19ktKouD064fuHLf9vv4rQGEJEGOI+KNgUwRn/z5Zq3AkMARKQTTiLYHdAoAysVuNIzeuhUIFNVd1TkA8Pu0ZCq5orIDcACnBEHM1V1tYjcDyxX1VTgRZzm4wacTplLghdxxfl4zY8AtYD/ePrFt6rq6KAFXUE+XnNY8fGaFwBni8gaIA+4TVVDtrXr4zXfCswQkVtwOo4nhPKNnYi8jpPMG3j6Pf4KVAFQ1X/h9IOcA2wAsoCJFf7OEP77MsYY4wfh+GjIGGNMOVgiMMaYCGeJwBhjIpwlAmOMiXCWCIwxJsJZIjCVkojkicgPXj9JpZx7yA/fN0tENnu+6zvPDNXyfsYLItLZ8/rOIse+qmiMns8p+HtZJSLvi0jdMs7vEerVOI37bPioqZRE5JCq1vL3uaV8xizgA1WdIyJnA4+qanIFPq/CMZX1uSIyG/hJVf9WyvkTcKqu3uDvWEz4sBaBCQkiUsuzjsJ3IrJSRE6oNCoiTUVkkdcd8wDP/rNFZInnvf8RkbJ+QS8C2nreO9nzWatE5E+efTVFZJ6I/OjZP9azf6GIpIjIw0B1Txyveo4d8vz5hoiM9Ip5lohcKCLRIvKIiCzz1Ji/xoe/liV4io2JSB/PNX4vIl+JSAfPTNz7gbGeWMZ6Yp8pIks95xZXsdVEmmDX3rYf+ynuB2dW7A+en3dwZsHX9hxrgDOrsqBFe8jz563AXZ7X0Tj1hhrg/GKv6dk/BbinmO+bBVzoeX0R8A1wCrASqIkzK3s10BO4AJjh9d46nj8X4lnzoCAmr3MKYjwPmO15XRWnimR1YBJwt2d/NWA50KqYOA95Xd9/gOGe7dpAjOf1UGCu5/UE4Bmv9/8duNzzui5OLaKawf73tp/g/oRdiQkTNo6oao+CDRGpAvxdRM4A8nHuhBsDO73eswyY6Tn3XVX9QUQG4ixW8qWntEZVnDvp4jwiInfj1Km5Cqd+zTuqetgTw9vAAOAj4DER+QfO46TF5biuD4EnRaQaMBxYpKpHPI+jkkXkQs95dXCKxW0u8v7qIvKD5/rXAv/zOn+2iLTDKbNQpYTvPxsYLSJ/9mzHAomezzIRyhKBCRWXAQ2BU1Q1R5yKorHeJ6jqIk+iGAnMEpHHgX3A/1T1Uh++4zZVnVOwISJDijtJVX8SZ62Dc4AHReQTVb3fl4tQ1WwRWQgMA8biLLQCzmpTN6rqgjI+4oiq9hCRGjj1d64HnsJZgOczVT3P07G+sIT3C3CBqq73JV4TGayPwISKOsAuTxI4EzhhzWVx1mH+VVVnAC/gLPf3NdBfRAqe+dcUkfY+fudi4HciUkNEauI81lksIs2ALFX9N04xv+LWjM3xtEyK8yZOobCC1gU4v9T/WPAeEWnv+c5iqbPa3E3ArfJbKfWCUsQTvE49iPOIrMAC4EbxNI/EqUprIpwlAhMqXgVSRGQlcCWwrphzBgE/isj3OHfbT6rqbpxfjK+LyAqcx0IdfflCVf0Op+9gKU6fwQuq+j3QDVjqeUTzV+DBYt4+HVhR0FlcxH9xFgb6WJ3lF8FJXGuA78RZtPx5ymixe2JZgbMwyz+BhzzX7v2+z4DOBZ3FOC2HKp7YVnu2TYSz4aPGGBPhrEVgjDERzhKBMcZEOEsExhgT4SwRGGNMhLNEYIwxEc4SgTHGRDhLBMYYE+H+H3COfGFhckF/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import roc_curve\n",
    "from sklearn.metrics import auc\n",
    "from matplotlib import pyplot\n",
    "%matplotlib inline\n",
    "\n",
    "test_predictions[\"Actual\"] = np.where(test_predictions[\"Churn?\"] == \"True.\", 1, 0)\n",
    "\n",
    "# calculate roc curves\n",
    "ns_fpr, ns_tpr, _ = roc_curve(test_predictions[\"Actual\"], [0 for _ in range(len(test_predictions[\"Churn?\"]))])\n",
    "lr_fpr, lr_tpr, _ = roc_curve(test_predictions[\"Actual\"], test_predictions[\"Churn?.True.\"])\n",
    "\n",
    "# plot the roc curve for the model\n",
    "pyplot.plot(ns_fpr, ns_tpr, linestyle='--', label='No Skill')\n",
    "pyplot.plot(lr_fpr, lr_tpr, marker='.', label='Logistic')\n",
    "\n",
    "# axis labels\n",
    "pyplot.xlabel('False Positive Rate')\n",
    "pyplot.ylabel('True Positive Rate')\n",
    "# show the legend\n",
    "pyplot.legend()\n",
    "# show the plot\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA5ZElEQVR4nO3dd3hUZfbA8e9JAoTeO4RQpTdDExGkKCji2hUXAXVZ17riKthYLLu6a8G+NhD1p6suKiKguLoiqCDFQkfpCZ0AgRACKef3x73BMUySCZk7k8mcz/PMw9wyc89Nwpx57/ve84qqYowxJnrFhDsAY4wx4WWJwBhjopwlAmOMiXKWCIwxJspZIjDGmChnicAYY6KcJQITduJ4TUQOiMiScMfjNRGZLyLXhzsOY/JYIohi7gfSARGp4Gf99fnWDRCRFJ9lEZFbRWSViBwRkRQR+Y+IdDqFUM4EhgBNVLVnIfEOEBEVkQmncIwSEZGeIjJXRA6KyH4RWSIiY0Mdh08817g/i+t91p0tIl+KSJqIbPHzmi0iclRE0t3HZ8U4nojIP0Qk1X38Q0SkkP1vEZHNInJIRJaJyJk+224XkU3uth0iMkVE4txt9UTk3+76NBH5RkR6BfyDMafEEkGUEpFEoB+gwIhTeIungduAW4FaQBtgJnD+KbxXM2CLqh4pYr/RwH7gmsJ2yvtQCRYR6QP8D/gKaAXUBv4EDAvmcYoRT03gHmB1vk1HgGnAnYW8/AJVreI+zinGYccBvwO6AJ2BC4A/FhBfL+BR4FKgOjAV+FBEYt1dZgHdVbUa0NF9z1vdbVWApcDpOH9XrwNzRKRKMWI1xaWq9ojCBzAJ+AZ4Epidb9t84Pp86wYAKe7z1kAO0LMYx2uE8wGwH9gA/MFdfx2Q6b5fOvBAAa+vDBwGrgSOA0k+2xJxEtp1wDZggbv+WmAtcACYBzTzec3TQDJwCFgO9Csk9q+B54s4vz+457XfPc9GPtuGAOuANOA5nIRyvc/2AuMs4FgvAjf6+z252wfjJNb867cAg0/x7+VbYJzP8nXA4gL2vQJYku93p0BDP/vWBj4HXijk2IeA08P9f6YsP8IegD3C9It3PrRuxPnmlQXU99lWVCK4AdhazOMtAF4A4oGuwF5goLttDPB1Ea8fBewEYoGPgWd9tuUlgjfcD52KwIXuObYD4oD7gG99XvN790MoDrgD2AXE+zluJZwkdXYhsQ0E9gHdgQrAs/yajOrgJLBLgXLA7UB23s+3qDj9HKsnsAynNX8qiWC3+7P/DOjis+1M4GAhx00DevksJwGHC9i3Gk5y7eX+vm4BfgDEZ5+R7ge8uvF0KeC9uuJ8Uage7v8zZfkR9gDsEYZfuvOfPguo4y6vA2732V5UIriXAr4NFnC8pu6HaVWfdY8A093ngSSCz4Gn3OdXuR8e5dzlvETQwmf/T4DrfJZjgAwK+LaN8228i5/1jd33bltIbFOBf/osV3F/vok4l7EW+2wTIMUnEQQcp/uhugzoXdDvyV1fUCLoi5MkKwF34yS/GgH+DnN8fwY4rUL1/XDPd473uD+DbJwk2aOA920NPAQ08LOtGrASuDvc/2fK+sP6CKLTaOAzVd3nLr/trsuTjfPt1Vc5nP/YAKlAw2IcrxGwX1UP+6zbivMhWyQRaQqcDbzlrvoIp2WRvz8i2ed5M+Bpt3P3IM4lG8k7poj8RUTWuh2SB3GuZdfxc/gDQC6Fn28j93wAUNV0nJ9RY3dbss82LU6c+dwIrFDVxYXEUiBV/UZVj6pqhqo+AhzE6ScKRDrOB3OeakC6ez75XQeMBToA5XFaX7NFpJGfmH7B6et4wXe9iFTEafktdmM1HrJEEGXc/2CXA/1FZJeI7MK5XNFFRLq4u23D+Tbrqzm/fth9ATQRkaQAD7sDqCUiVX3WJQDbA3z9KJy/1Y/deDfhJILR+fbz/VBKBv6oqjV8HhVV9VsR6QfchfNzqKmqNXAufZw0CkZVM4BFwCVFnF+zvAURqYxz2Wk7zuWspj7bxHe5sDj9HGcQcJHP7+0M4AkRea6Q2Aqj+DnnAqzG6dTN04WTO6vzdMXpd/pZVXNV9VOcn8MZBewfB7TMW3BHsc3EaTn57ZA2QRbuJok9QvvAuayyH+eDuIHPYwHwhLvPucAenOvRgjMiaC1wg8/7PAv8gnPJqDzOB/OVwMQCjrsQp6M0HmfUyW7cjkuKuDQErAcm54t3BHAM5wM3EedDLc7nNRcBq4AO7nJ14DL3+Xk4H94N3Ngn4Vz68NuRivMBlo4zGqe2u64L8I77fDDOpaquOH0ET+edD7/2EVyM84F3G7/tIygwTj9x1Mj3M/gWGI97/RwnWcbjjGba6j4v725LwLk0lPe7utONuXaAfzc3uH8Dea2c1b5/D/n2HQ38DLRw/36G4Fzuautuvx6o5z5v777Xk+5yOZyWwEzf36c9PP5cCHcA9gjxLxw+xf3Az7f+cpxrxnHu8rXuf9BDOJ2ZE4EYn/3F/VBb7f4n3w68m/eB5uf9mwCzcZLQRn6bVMZQQCIAeuN0Ftb1s201cDN+EoG7fRTONeZDON+8p7nrY3GGWR7C+aZ6F0WMqMFJip/gtBz2A98B1/hsv8E9r/3ueTbx2TbU/WAsaNSQ3zgD+F3Oz/c+A9yfg+9jvrutA7ACZ4hpKk6rznfkVT+cSz0FHUuAf7rnt9997tv5m4478srd90GcluVhnAQyymff13C+CBxxf+6P4XbUA/3duDPc90z3fW97ePMQ94dvjDEmSlkfgTHGRDlLBMYYE+UsERhjTJSzRGCMMVEuqMW5QqFOnTqamJgY7jCMMSaiLF++fJ+q1vW3LeISQWJiIsuWLQt3GMYYE1FEZGtB2+zSkDHGRDlLBMYYE+UsERhjTJSLuD4Cf7KyskhJSSEzMzPcoZRa8fHxNGnShHLl8hcVNcZEuzKRCFJSUqhatSqJiYkUMo1q1FJVUlNTSUlJoXnz5uEOxxhTynh2aUhEponIHhFZVcB2EZFnRGSDiKwQke6neqzMzExq165tSaAAIkLt2rWtxWSM8cvLPoLpOFUXCzIMZ3ai1jgTY/+rJAezJFA4+/kYU8olL4GFTzj/nsr2EvDs0pCqLhCRxEJ2uRB4Q53yp4tFpIaINFTVnV7FZIwxpVLyEnjtPMjNAomB+h2hwq8TwuVkphGzZzWiCnHxMHoWNO0ZtMOHc9RQY347ZV8KBUxdKCLjRGSZiCzbu3dvSIIrLhHhjjvuOLH8+OOPM3ny5IBfv3v3boYPH06XLl1o37495513HgDz589n+PDhJ+0/a9YsHn30UQAmT57M448/DsCYMWOYMWNGCc7EGBNyWxY6SQBAcyEz7cSmtKNZ7N6z21mPQs5xZ/8giojho6r6sqomqWpS3bp+75AOuwoVKvDBBx+wb9++onf2Y9KkSQwZMoSffvqJNWvWnPiQL8iIESOYOHHiKR3LGFPKJPZzWgIAcRXhkldJu3ImE6s9Qpdtt/FwhfHkxsaDxEJseWf/IApnItjOb+dubULgc9iWOnFxcYwbN44pU6actG3Lli0MHDiQzp07M2jQILZt23bSPjt37qRJkyYnljt37nzSPkuXLqVbt25s3LiR6dOnc/PNNwf3JIwx4dG0p3M5qEYzGD2LnMY9uORf3/LesmT+2L8FT94xjtgxH8PAe4N+WQjCO3x0FnCziLwD9ALSgtU/cMVLi05aN7xzQ0b1SeTo8RzGvHZyZ8ulpzfhsqSm7D9ynD/93/LfbHv3j30COu5NN91E586dueuuu36z/pZbbmH06NGMHj2aadOmceuttzJz5syTXnvFFVfw3HPPMXjwYMaOHUujRo1ObP/222+55ZZb+Oijj0hISGDhwuA2DY0pk5KXOJdREvsF/cPTCzmqxKgSGyP85ZzTaFQjns5Najgbm/b07Bw8SwQi8m+cOVTriEgK8FecialR1ReBuTiTiG/AmZ90rFexhEq1atW45ppreOaZZ6hYseKJ9YsWLeKDDz4AYNSoUSclCoBzzz2XTZs28emnn/LJJ5/QrVs3Vq1yRt6uXbuWcePG8dlnn/0mORhjClFEB2xposfSYNdKYoCc6SOIG/sxQzuGLnF5OWroqiK2K3CTF8cu7Bt8xfKxhW6vVbl8wC0Af/785z/TvXt3xo4tfl6rVasWI0eOZOTIkQwfPpwFCxZQu3ZtGjZsSGZmJj/88IMlAmMC5a8DthQmgmPZOaTt3UtdBRGI1Swn9hC2YMrEncWlSa1atbj88suZOnUq1157LQBnnHEG77zzDqNGjeKtt96iX7+TO3r+97//0bt3bypVqsThw4fZuHEjCQkJHDlyhBo1ajB16lSGDBlC5cqVGTBgQIjPypgIlNcBq7knOmBL2+Whj37czr0frqJT7nreLPcwsZqNeNAZXBRLBB644447eO65504sP/vss4wdO5bHHnuMunXr8tprr530muXLl3PzzTcTFxdHbm4u119/PT169GD+/PkA1K9fn9mzZzNs2DCmTZsWqlMxJnLldcBmppXKJABQvWI5ujatwSMXX0/ckd5h688Q5wpN5EhKStL8E9OsXbuWdu3ahSmiyGE/JxN1Xjvf+XfsnPDG4crOyWXq15vJysnl5oGtAacWWCju/BeR5aqa5G+btQiMMWWDvxFCxw45LYLkJWFvEazZcYgJ769g5fY0zu/c8EQCKA3lXywRGGMin78RQgC7Vjj/vj7Ck/H3gTiWncNz/9vAv+ZvpEalcrxwdXeGdWxQKhJAHksExpjIV0iJBuDXsgxhSARb9mXw4lcbGdG1Efef356alcuHPIaiWCIwxkQ+fyOEwGkJ5Bz3pCxDYY4cy+a/a3bzu26NOa1BVb4YP4CE2pVCdvziskRgjIl8BY0QGj0r5CNxFv6yl7s/WMn2g0fp2LgarepVLdVJACwRGGMimW8HsT8elmXILy0ji7/NXcN7y1JoUacy747rQ6t6VUNy7JKKiOqjkaBKlSolfo9ly5Zx6623Frh9y5YtvP322wHvb0yZltdB/MWDMPUcp2P44FbncpAHk7cUJidXueTFb3n/++3cOKAlc2/rR8/mtUIaQ0lYi6AUSUpKIinJ7zBf4NdEMHLkyID2N6ZM8+0gxud+qBB2DO8/cpwaFcsRGyPcee5pNK5RkY6Nq3t+3GCL3haBh9O+5fnxxx/p3bs3nTt35qKLLuLAgQOAU066c+fOdO3alTvvvJOOHZ2hbr6T0Hz11Vd07dqVrl270q1bNw4fPszEiRNZuHAhXbt2ZcqUKb/ZPz09nbFjx9KpUyc6d+7M+++/79l5GVMq+Nbwjy0PsRU8q9efn6ry/vIUzn58Pu8sdebXOrdDg4hMAlAWWwSfTIRdKwvf59gh2L3KGWEQSFXCBp1gWOETxfhzzTXX8Oyzz9K/f38mTZrEAw88wFNPPcXYsWN55ZVX6NOnT4GTyzz++OM8//zz9O3bl/T0dOLj43n00Ud5/PHHmT17NsCJ8hMADz30ENWrV2flSufc85KOMWVW/g5iCEnHcMqBDO75cBULft7L6c1qRtQloIJEZ4sgM82d9g3/Y46DIC0tjYMHD9K/f38ARo8ezYIFCzh48CCHDx+mTx+nwmneZZ78+vbty/jx43nmmWc4ePAgcXGF5+zPP/+cm276tZhrzZo1g3Qmxq8QtChNMTXtCf3u8DQJfPhDCudOWcCyLft5YEQH/vPHPrSqV/L+wXArey2CQL65Jy/57fjiUliQauLEiZx//vnMnTuXvn37Mm/evHCHZPJEUJ37Mu3YoZDfOVyrcgVOT6zF3y/qSJOapXtIaHFEZ4ugaU/nj8ajad8AqlevTs2aNU/MJPbmm2/Sv39/atSoQdWqVfnuu+8AeOedd/y+fuPGjXTq1IkJEybQo0cP1q1bR9WqVTl8+LDf/YcMGcLzzz9/YtkuDXmoqLtYTWj4/tw9mNAdICsnlxfmb+CZL34BoH+burw+tkeZSgJQFlsEgQry+OKMjIzfzDk8fvx4Xn/9dW644QYyMjJo0aLFifLTU6dO5Q9/+AMxMTH079+f6tVP7mB66qmn+PLLL4mJiaFDhw4MGzaMmJgYYmNj6dKlC2PGjKFbt24n9r/vvvu46aab6NixI7Gxsfz1r3/l4osvDtr5GR8RUOc+KuRv2Qe5g3jV9jQmvL+C1TsOcUGXRqWqSFywWRnqMEhPTz9x38Gjjz7Kzp07efrppz0/bqT9nEq1F/uV6jr3UcODOYkzs3J45otfeGnBJmpWKs/Dv+vA0I4Ng/Le4WRlqEuZOXPm8Mgjj5CdnU2zZs2YPn16uEMKnwibXNyUMh7cObw1NYNXFm7i4m6Nue/89lSvVC6o718aWYsgipS6n1Okdrr6dlLGVQxbeWMTPEeOZTNv9S4u7u5c3k3en0HTWmWrH6CwFkGZ6SyOtIQWaqXy5xOpna4h6KQ0ofPVz3s5Z8oC7vjPT2zY4wzGKGtJoChl4tJQfHw8qamp1K5du0x25JSUqpKamkp8fHy4Q/mtSO109biT0oTGgSPHeWjOGj74fjst61Z27wmIjCJxwVYmEkGTJk1ISUlh79694Q6l1IqPj//NqKZSIQImF/crb/ix9W1ErLwicVtTM7j57FbcPLAV8eViwx1W2JSJRFCuXDmaN28e7jBMNHX8hrC8sQme1PRj1KxUntgYYeLQtjSuWZEOjSKzPlAwlYlEYEqBU+n4DcOdoSY6qSr/WZ7Cw7PXMGFYW67u1YxzOjQId1ilRpnpLDZhdiodv9bpakIgeX8G10xbwl0zVtC2QTX6tKgd7pBKHWsRmOA4lY5f63Q1Hvvg+xTum7kKAR76XUeu7plATIwNKMnPEoEJjlPp+LVOV+OxOlUq0LN5Lf52USca16gY7nBKLUsEJngqVHMexflAt05XE0RZObm89NVGcnLhtsGtOatNXc5qUzfcYZV61kdgAhNI/f1jhyAt2Wr0m7BYtT2NEc99w+Of/cymfeml8ybKUspaBKZogYwIshFAJkwys3J46vNfeGXhJmpVLs9Lo07nXBsRVCyetghEZKiIrBeRDSJy0pyMIpIgIl+KyA8iskJEzvMyHnOKAhkRZCOATJhs25/B1K83cWn3Jnx+e39LAqfAsxaBiMQCzwNDgBRgqYjMUtU1PrvdB7ynqv8SkfbAXCDRq5jMKQpkRJCNADIhdDgzi09X7eKypKa0qV+VL/8yoMxNFhNKXl4a6glsUNVNACLyDnAh4JsIFMi7xlAd2OFhPOZUBTIiyEYAmRD5ct0e7v1wJbsOZdItoQat6lW1JFBCXiaCxkCyz3IK0CvfPpOBz0TkFqAyMNjfG4nIOGAcQEJCQtADNQEIZESQjQAyHtp/5DgPzV7Dhz9sp3W9Ksz40xlRWyQu2MLdWXwVMF1VnxCRPsCbItJRVXN9d1LVl4GXwZmPIAxxGmPCKCdXufRf37Jtfwa3DmrNTWe3pEJc9BaJCzYvE8F2oKnPchN3na/rgKEAqrpIROKBOsAeD+MyxkSIvYePUbuyUyTunvPa0bhmRdo1jIDJiyKMl6OGlgKtRaS5iJQHrgRm5dtnGzAIQETaAfGA1ZI2JsqpKu8u3cbAJ+bz9pJtAAxuX9+SgEc8axGoaraI3AzMA2KBaaq6WkQeBJap6izgDuAVEbkdp+N4jNpdIMZEtW2pGUz8YAXfbkylV/NanNmqTrhDKvM87SNQ1bk4Q0J9103yeb4G6OtlDCYAgcwjcOyQM2ooeYl1CBvPzFiewv0zVxEbI/ztoo5c1cOKxIVCuDuLTbjZXcOmFKlfrQJntKzNwxd1pGF1KxIXKpYIop2/u4bzJwJ/dw1bIjBBcDw7l3/N30iuKrcPaUO/1nXp19qKxIWaJYJoZ3cNmzD5Kfkgd81Ywfrdh7m4W2NUFRG7DBQOlgiind01bELs6PEcnvzveqZ+vZl6VeN59ZokBrevH+6wopolgmiW10l8/EjR+9pdwyZIkg9k8Pq3W7myZwITh7WlWny5cIcU9STSRmsmJSXpsmXLwh1G5PPtJM4TV9E6go0nDrlF4i5Pcu4x3XHwKI1sxrCQEpHlqprkb5u1CKKVbydxHusINh7437rd3PPBKvYczqR7Qk1a1atiSaCUsUQQrXw7icF5bh3BJohS04/x4Ow1fPTjDk6rX5UXR51Oq3pVwh2W8cMSQbTy7SQ+czwcTbWOYBM0ObnKZS8uIvlABrcPbsOfBrSkfJzNjFtaWSIoqwK5WzhP/faWAExQ7DmcSZ3KFYiNEe49vx1NalbitAZWKrq0CzgRiEglVc3wMhgTJHa3sAmx3Fzl30u38cjcdUwY1pZRvZsxqJ0NCY0URbbVROQMEVkDrHOXu4jIC55HZk6dzTFsQmjLviOMfHUx9364is5NqtPf7gyOOIG0CKYA5+KWkFbVn0TkLE+jMiVjdwubEHlvWTL3z1xF+dgYHr24E1f0aGp3B0eggC4NqWpyvl9ujjfhmKCwu4VNiDSuUZGz2tTloQs70qB6fLjDMacokESQLCJnACoi5YDbgLXehmVKzOYYNh44lp3DC19uRFUZf85p9G1Vh742X0DEC2Q81w3ATTiT0W8HugI3ehiTKankJZC6wXkkLwl3NKaM+GHbAS549mue/uIXth/MJNKqEpiCBdIiOE1Vr/ZdISJ9gW+8CcmUSPISmDYU1L16N304jJlt3/zNKcs4ns0Tn/3MtG8206BaPNPGJDGwrY0IKksCaRE8G+A6UxpsWfhrEgAbEWRKbPuBo7y5eCtX90rgs9vPsiRQBhXYIhCRPsAZQF0RGe+zqRrOHMSmNErsBwjOFNDYiCBzStKOZvHJyp1c2TOB1vWr8tWdA2zGsDKssEtD5YEq7j6+twYeAi71MihTAk17QoNOkL4H2p4HXa6yy0KmWD5bvYv7Zq4i9chxkhJr0apeFUsCZVyBiUBVvwK+EpHpqro1hDGZwgRaOiKugiUBUyz70o8xedZqZq/YSdsGVXl1dJIViYsSgXQWZ4jIY0AH4MRAYVUd6FlUxj8rHWE8kpOrXPqvb9lxMJO/nNOGP/ZvSblYKxIXLQJJBG8B7wLDcYaSjgb2ehmUKYBNNG+CbPehTOpWcYrE/fWCDjSpWZHW9a1IXLQJJBHUVtWpInKbz+WipV4HZvyw0hEmSHJzlbeWbOMfn6xjwtDTGNUnkbPb1gt3WCZMAkkEedNY7RSR84EdQC3vQjIFstIRJgg27U1n4gcrWbJ5P2e2qsOA0ywBRLtAEsHDIlIduAPn/oFqwJ+9DKpMK848AafKSkeYAry7dBuTPlpNhbgY/nlpZy47vYkViTNFJwJVne0+TQPOhhN3FpviCqSztzDWEWxKqEnNSgw4zSkSV6+aFYkzjsJuKIsFLsepMfSpqq4SkeHAPUBFoFtoQixDAunsLYx1BJtiOpadw7NfbADgL+dakTjjX2EtgqlAU2AJ8IyI7ACSgImqOjMEsZU9gXT2FsY6gk0xLN+6n7tmrGDj3iNcntQEVbXLQMavwhJBEtBZVXNFJB7YBbRU1dTQhFYGBdLZW9TrrSPYFOHIsWwem7ee1xdtoVH1irx+bU/6t7FZw0zBCksEx1U1F0BVM0VkU3GTgIgMBZ7GqU30qqo+6mefy4HJOMVxflLVkcU5RqliHcGmFNhx8ChvL9nGNb2bcefQtlSpEPDU5CZKSUE1xUUkA9iQtwi0dJcFUFXtXOgbO30MPwNDgBRgKXCVqq7x2ac18B4wUFUPiEg9Vd1T2PsmJSXpsmXLAjm30CruXb9xFa2z1wRNWkYWc1buZGSvBMC5Uay+dQYbHyKyXFWT/G0r7KtCuxIetyewQVU3uUG8A1wIrPHZ5w/A86p6AKCoJFCq2V2/Jkw+XbWL+z9axf4jx+nVohYt61axJGCKpbCicyUtNNcYSPZZTgF65dunDYCIfINz+Wiyqn6a/41EZBwwDiAhIaGEYXnE7vo1IbbncCaTZ61m7spdtG9YjdfG9KBlXSsSZ4ov3BcP44DWwACgCbBARDqp6kHfnVT1ZeBlcC4NhTjGwNhdvyaEcnKVy19cxI60TO489zTGndXCisSZU+ZlItiOM/w0TxN3na8U4DtVzQI2i8jPOIkhcmoZ+XYQB8I6e00J7Ew7Sv2q8U6RuBEdaFqzkpWKNiUW0FcIEakoIqcV872XAq1FpLmIlAeuBGbl22cmTmsAEamDc6loUzGPEz55HcRfPAhTz3E6gg9udS7/2KTxJohyc5Xp32xm0BNf8X/fOVdtzz6tniUBExRFJgIRuQD4EfjUXe4qIvk/0E+iqtnAzcA8YC3wnqquFpEHRWSEu9s8IFVE1gBfAndG1H0Kvh3E+FyxsnmCTRBt2JPO5S8tYvLHa0hKrMVAqxJqgiyQS0OTcUYAzQdQ1R9FpHkgb66qc4G5+dZN8nmuwHj3EXl8O4hjywMCudnWEWyC5p0l25g0azUVy8XyxGVduLh7Y7s72ARdQGWoVTUt3x9f6eywDbX8HcRgHcEmqBJqV2Jwu3o8MKIjdatWCHc4powKJBGsFpGRQKx7A9itwLfehhVBKlRzHnkf/JYATAlkZuXwzBe/AHDX0Lac0bIOZ7S0InHGW4F0Ft+CM1/xMeBtnHLUf/YwptBKXgILnzj1zt1jhyAt2TqHTYkt27Kf855ZyAvzN7L/yHEKuuvfmGALpEXQVlXvBe71OpiQs/kBTCmQfiybxz5dxxuLt9K4RkXeuLYnZ1mROBNCgbQInhCRtSLykIh09DyiUPJXFqI4/JWMMKaYdqUd5Z2lyYzuk8i8P59lScCEXCAzlJ0tIg1wJql5SUSqAe+q6sOeR+c1mx/AhMmBI8eZvXIno3o3o1W9qiy862ybMcyETUB3FqvqLpzJab4E7gImAZGfCGx+ABNiqsonq3Yx6aNVHMzI4oyWtWlZt4olARNWRSYCEWkHXAFcAqQC7+JMZB+Zgj1ngJWMMAHacyiT+z9axbzVu+nUuDpvXNvLisSZUiGQFsE0nA//c1V1h8fxeCt/53DN5rB/o7PNOnuNh3JylcteWsSutEzuHtaW685sTpwViTOlRCB9BH1CEUhI5O8czvCpZmHzAxgP7Dh4lAbVnCJxD17YkaY1K9LCWgGmlCnwK4mIvOf+u1JEVvg8VorIitCFGER5ncPgdA4PfsD5V2Kts9cEVU6u8lq+InH929S1JGBKpcJaBLe5/w4PRSAh4a9zuH576+w1QbVhz2HumrGC77cdZMBpdRnUrn64QzKmUIXNULbTfXqjqk7w3SYi/wAmnPyqUi55CaTnmw3TOntNEL393TYmz1pN5QqxTLmiC7/rakXiTOkXSG/VED/rhgU7EM8lL4FpQyF9l/OYPtzKQpigS6xTiXM61Oe/4/tzUbcmlgRMRCiwRSAifwJuBFrk6xOoCnzjdWBBt2UhaM6vy9Y5bIIgMyuHKZ//jCBMHGZF4kxkKqyP4G3gE+ARYKLP+sOqut/TqLyQ2A8QTlTQts5hU0LfbUpl4gcr2bzvCFf3SkBVrQVgIlJhiUBVdYuI3JR/g4jUirhk0LQnNOjk9BG0PQ+6XGWtAXNKDmdm8Y9P1/F/i7eRUKsSb1/fizNaWSvARK6iWgTDgeU4X6N9v+oo0MLDuLyRN3fA8CnhjsREsN2HjjFjeQrXn9mc8ee0oVL5gCq1GFNqFTZqaLj7b0DTUhpTlu0/cpw5K3Ywqk8irepVYeFdA23GMFNmBFJrqC/wo6oeEZHfA92Bp1R1m+fRGRNmqsrsFTuZPGs1hzKz6NuqDi3qVrEkYMqUQIaP/gvIEJEuOMXmNgJvehqVMaXA7kOZ/OGN5dzy7x9oXLMiH99ypt0ZbMqkQC5uZquqisiFwHOqOlVErvM6MGPCKSdXudwtEnfvee0Y2zfRisSZMiuQRHBYRO4GRgH9RCQGKOdtWMaER8qBDBpWr0hsjPDQhR1JqFWJxDqVwx2WMZ4K5CvOFTgT11/rTlDTBHjM06iMCbGcXOXVhZsY/ORX/N9ip0jcWW3qWhIwUSGQMtS7ROQtoIeIDAeWqOob3odmTGis33WYu95fwU/JBxnUth7ndLAicSa6BDJq6HKcFsB8nHsJnhWRO1V1hsexGeO5/1u8lQc+Xk3V+HI8fWVXRnRpZHcHm6gTSB/BvUAPVd0DICJ1gc8BSwQmYuWVg2hVrwrndWrIpOHtqV3FhoSa6BRIIojJSwKuVALrWzCm1Dl6PIcn/7uemBjh7mHt6N2iNr1b1A53WMaEVSCJ4FMRmQf8212+ApjrXUgeSV4CqRt+fW51hqLOoo2pTPxgBVtTMxjVu5kViTPGFUhn8Z0icjFwprvqZVX90NuwgixvLoK8MtTTh8OY2ZYMosShzCwembuOfy/ZRrPalXj7D72sVLQxPgqbj6A18DjQElgJ/EVVt4cqsKCyuQii2p5Dx5j5w3bGndWC2we3oWL52HCHZEypUti1/mnAbOASnAqkzxb3zUVkqIisF5ENIjKxkP0uEREVkaTiHiMgJ+YicNlcBGVeavoxpn+zGYBW9arw9YSzuee8dpYEjPGjsEtDVVX1Fff5ehH5vjhvLCKxwPM4U12mAEtFZJaqrsm3X1XgNuC74rx/sdhcBFFDVZn10w4mz1pN+rFszmpTlxZ1q9iIIGMKUVgiiBeRbvz6Vbqi77KqFpUYegIbVHUTgIi8A1wIrMm330PAP4A7ixl78dhcBGXejoNHuW/mKv63bg9dm9bgn5d2tiJxxgSgsESwE3jSZ3mXz7ICA4t478ZAss9yCtDLdwcR6Q40VdU5IlJgIhCRccA4gISEhCIOa6JRdk4uV768mL2Hj3H/8PaMOSOR2BgbEWRMIAqbmOZsLw/sFq97EhhT1L6q+jLwMkBSUpJ6GZeJLMn7M2hUoyJxsTH8/aJOJNSqRELtSuEOy5iI4uWNYduBpj7LTdx1eaoCHYH5IrIF6A3M8qzD2JQp2Tm5vLxgI4Of/Io3F20B4MzWdSwJGHMKvJxsdSnQWkSa4ySAK4GReRtVNQ04MZhbRObjDFFd5mFMpgxYu/MQE95fwYqUNIa0r8+wTg3DHZIxEc2zRKCq2SJyMzAPiAWmqepqEXkQWKaqs7w6tim73ly0hQc+XkP1iuV4bmQ3zu/U0O4ONqaEAqk+KsDVQAtVfVBEEoAGqrqkqNeq6lzylaNQ1UkF7DsgoIhNVMorB9GmflUu6NKI+4e3p1bl8uEOy5gyIZAWwQtALs4ooQeBw8D7QA8P4zIGgIzj2Tw+72fiYoV7zmtHrxa16WVF4owJqkA6i3up6k1AJoCqHgDsq5jx3Dcb9nHuUwuY9s1mjmfnomoDxozxQiAtgiz3LmGFE/MR5HoalYlqaUez+Puctby7LJnmdSrz3h/70LN5rXCHZUyZFUgieAb4EKgnIn8DLgXu8zQqE9X2pR/j4xU7uKF/S/48uDXx5aw+kDFeCqQM9VsishwYhFNe4nequtbzyExU2Xv4GB//tINrz2xOy7pV+HrCQOsMNiZEAhk1lABkAB/7rlPVbV4GZqKDqjLzx+088PEaMo7lcHbbejSvU9mSgDEhFMiloTk4/QMCxAPNgfVABw/jMlFg+8Gj3PvhSuav30v3BKdIXPM6lcMdljFRJ5BLQ518l91CcTd6FpGJCk6RuEWkph9n8gXtGdXHisQZEy7FvrNYVb8XkV5F72nMybalZtC4plMk7tGLO5NQqxJNa1l9IGPCKZA+gvE+izFAd2CHZxGZMik7J5dXFm5myuc/c/ewtozt25y+rWzeYGNKg0BaBFV9nmfj9Bm87004pixavSONCe+vYNX2Q5zboT7nW5E4Y0qVQhOBeyNZVVX9S4jiMWXM699u4aHZa6hRqTz/urq7VQo1phQqMBGISJxbQbRvKAMyZUNekbi2DapyYdfG3D+8HTUq2ZBQY0qjwloES3D6A34UkVnAf4AjeRtV9QOPYzMR6MixbB6bt55yscK957e3InHGRIBA+gjigVSc6qN59xMoYInA/MaCn/dy9wcr2ZF2lNF9Ek+0CowxpVthiaCeO2JoFb8mgDxWBtKckJaRxUNz1jBjeQot6jpF4nokWpE4YyJFYYkgFqjCbxNAHksE5oR9R47xycqd3DigJbcOsiJxxkSawhLBTlV9MGSRmIiy53Ams37cwfX9WpwoElfT6gMZE5EKSwR2cdecRFV5//vtPDR7DUezchjUrj7N61S2JGBMBCssEQwKWRQmIiTvz+CeD1ey8Jd9JDWryaOXWJE4Y8qCAhOBqu4PZSCmdMvOyeWqVxZz4MhxHrqwA1f3akaMFYkzpkwodtE5E1227DtC01qViIuN4Z+XOkXimtS0InHGlCWBTF5volBWTi7Pf7mBc6Ys4I1FWwA4o2UdSwLGlEHWIjAnWbU9jbtmrGDNzkOc36khwzs3CndIxhgPWSIwv/HaN5t5eM5aalUuz4u/P52hHRuEOyRjjMcsERjg1yJxHRpV5+Jujbnv/PZUr1Qu3GEZY0LAEkGUSz+WzT8/XUf52BjuG96ens1r0bO5lYcwJppYZ3EUm79+D+dOWcCbi7eiOK0CY0z0sRZBFDpw5DgPzVnDB99vp1W9Ksy44QxOb1Yz3GEZY8LEEkEUOpBxnM9W7+bWga24aWArKsRZkThjopmnl4ZEZKiIrBeRDSIy0c/28SKyRkRWiMgXItLMy3ii2Z5Dmby8YCOqSou6VfhmwkDGn3OaJQFjjHeJwJ3v+HlgGNAeuEpE2ufb7QcgSVU7AzOAf3oVT7RSVd5bmsygJ7/iic9+ZktqBoCNCDLGnODlpaGewAZV3QQgIu8AFwJr8nZQ1S999l8M/N7DeKJO8v4M7v5gJV9v2EfP5rV49OJOViTOGHMSLxNBYyDZZzkF6FXI/tcBn/jbICLjgHEACQkJwYqvTMsrEncwI4uHf9eRkT0TrEicMcavUtFZLCK/B5KA/v62q+rLwMsASUlJNsaxEJv3HSHBLRL32KVdaFa7Eo1qVAx3WMaYUszLzuLtQFOf5Sbuut8QkcHAvcAIVT3mYTxlWlZOLs9+8QvnTlnA699uAaBPy9qWBIwxRfKyRbAUaC0izXESwJXASN8dRKQb8BIwVFX3eBhLmbYi5SB3zVjBul2HuaBLI0Z0tSJxxpjAeZYIVDVbRG4G5gGxwDRVXS0iDwLLVHUW8BhQBfiPiABsU9URXsVUFk37ejMPz1lD3aoVeOWaJIa0rx/ukIwxEcbTPgJVnQvMzbduks/zwV4evyzLKxLXuUl1rujRlInD2lG9og0JNcYUX6noLDaBO5yZxaOfrKNCXCyTLmhPUmItkhKtSJwx5tRZ0bkI8uW6PZwzZQH/XrKNuFixInHGmKCwFkEE2H/kOA9+vJqZP+6gTf0qvHD1GXRLsCJxxpjgsEQQAdKOZvHF2j3cNqg1N53divJx1pAzxgSPJYJSaldaJjN/3M4fz2pB8zqV+XriQOsMNsZ4whJBKaOqvLM0mb/PWUtWbi5DOzQgsU5lSwLGGM9YIihFtqYeYeL7K1m0KZXeLWrx6MWdSbQiccYYj1kiKCWyc3IZ+cp3pB3N4u8XdeLKHk2tSJwxJiQsEYTZxr3pNHOLxD1xuVMkrmF1qw9kjAkdG34SJsezc3nq858Z+tQC3li0FYDeLWpbEjDGhJy1CMLgx+SDTJixgvW7D3Nh10b8rlvjcIdkjIlilghCbOrXm/nbnDXUqxrP1NFJDGpnReKMMeFliSBE8orEdW1anSt7JjBxWFuqxduQUGNM+Fki8NihzCwembuO+HIx/PWCDpzerBanN7MiccaY0sM6iz30+ZrdDHnyK95duo3ycTFWJM4YUypZi8ADqenHeODjNcz6aQdtG1Tl5VFJdGlaI9xhGWOMX5YIPHA4M5sv1+/h9sFt+NOAllYkzhhTqlkiCJIdB4/y4Q/buXFASxLrVOabiQOtM9gYExEsEZRQbq7y9pJtPPrJOnJylfM7NSSxTmVLAsaYiGGJoAQ27zvCxPdX8N3m/fRtVZtHLupMQu1K4Q7LGGOKxRLBKcrOyeX3r37Hocws/nlJZy5LaoKIFYkzxkQeSwTFtGHPYRJrVyYuNoYpV3SlWe1K1K8WH+6wjDHmlNlwlgAdy87hyf/+zNCnFvK6WySuZ/NalgSMMRHPWgQB+H7bASbMWMEve9K5uFtjLrYiccaYMsQSQRFeWbCJv3+ylobV4nltbA/OPq1euEMyxpigskRQgNxcJSZG6N6sBlf3SmDC0LZUtSGhxpgyyBJBPmlHs/jbnDVULBfLAxd2tCJxxpgyL3o6i48dgrRkSF5S4C7zVu9iyJNf8f7326lcIc6KxBljokJ0tAiSl8DuVaC58PoIGD0LmvY8sXlf+jH++tFq5qzcSfuG1Zg2pgcdG1cPY8DGGBM60ZEItix0kgBAznFn2ScRpGdms/CXvdx57mmMO6sF5WKjp6FkjDHRkQgS+4HEOMkgtjwk9mP7waN8+H0KN53disQ6lfn27kFUqRAdPw5jjPHl6VdfERkqIutFZIOITPSzvYKIvOtu/05EEj0JpGlPqN8RajQjd9RHvJlSj3Oe/Irnv9zI1tQMAEsCxpio5VkiEJFY4HlgGNAeuEpE2ufb7TrggKq2AqYA//AqHoCsHOX+j1Zx/0er6d6sJp/dfhaJdSp7eUhjjCn1vGwR9AQ2qOomVT0OvANcmG+fC4HX3eczgEHiReW25CXo7lXEHd7GffsnMnVQLm9c25OmtaxSqDHGeJkIGgPJPssp7jq/+6hqNpAG1M7/RiIyTkSWiciyvXv3Fj+SLQsRVQSIlxwGxf9slUKNMcYVEcNjVPVlVU1S1aS6desW/w0S+0FcPEgs4nYWG2OMcXjZQ7odaOqz3MRd52+fFBGJA6oDqUGPpGlP596BLQudJOAzdNQYY6Kdl4lgKdBaRJrjfOBfCYzMt88sYDSwCLgU+J96dTtv056WAIwxxg/PEoGqZovIzcA8IBaYpqqrReRBYJmqzgKmAm+KyAZgP06yMMYYE0KeDp5X1bnA3HzrJvk8zwQu8zIGY4wxhYuIzmJjjDHesURgjDFRzhKBMcZEOUsExhgT5STSJl8Rkb3A1lN8eR1gXxDDiQR2ztHBzjk6lOScm6mq3ztyIy4RlISILFPVpHDHEUp2ztHBzjk6eHXOdmnIGGOinCUCY4yJctGWCF4OdwBhYOccHeyco4Mn5xxVfQTGGGNOFm0tAmOMMflYIjDGmChXJhOBiAwVkfUiskFEJvrZXkFE3nW3fyciiWEIM6gCOOfxIrJGRFaIyBci0iwccQZTUefss98lIqIiEvFDDQM5ZxG53P1drxaRt0MdY7AF8LedICJfisgP7t/3eeGIM1hEZJqI7BGRVQVsFxF5xv15rBCR7iU+qKqWqQdOyeuNQAugPPAT0D7fPjcCL7rPrwTeDXfcITjns4FK7vM/RcM5u/tVBRYAi4GkcMcdgt9za+AHoKa7XC/ccYfgnF8G/uQ+bw9sCXfcJTzns4DuwKoCtp8HfAII0Bv4rqTHLIstgp7ABlXdpKrHgXeAC/PtcyHwuvt8BjBIInsS4yLPWVW/VNUMd3ExzoxxkSyQ3zPAQ8A/gMxQBueRQM75D8DzqnoAQFX3hDjGYAvknBWo5j6vDuwIYXxBp6oLcOZnKciFwBvqWAzUEJGGJTlmWUwEjYFkn+UUd53ffVQ1G0gDaockOm8Ecs6+rsP5RhHJijxnt8ncVFXnhDIwDwXye24DtBGRb0RksYgMDVl03gjknCcDvxeRFJz5T24JTWhhU9z/70XydGIaU/qIyO+BJKB/uGPxkojEAE8CY8IcSqjF4VweGoDT6lsgIp1U9WA4g/LYVcB0VX1CRPrgzHrYUVVzwx1YpCiLLYLtQFOf5SbuOr/7iEgcTnMyNSTReSOQc0ZEBgP3AiNU9ViIYvNKUedcFegIzBeRLTjXUmdFeIdxIL/nFGCWqmap6mbgZ5zEEKkCOefrgPcAVHUREI9TnK2sCuj/e3GUxUSwFGgtIs1FpDxOZ/CsfPvMAka7zy8F/qduL0yEKvKcRaQb8BJOEoj068ZQxDmrapqq1lHVRFVNxOkXGaGqy8ITblAE8rc9E6c1gIjUwblUtCmEMQZbIOe8DRgEICLtcBLB3pBGGVqzgGvc0UO9gTRV3VmSNyxzl4ZUNVtEbgbm4Yw4mKaqq0XkQWCZqs4CpuI0HzfgdMpcGb6ISy7Ac34MqAL8x+0X36aqI8IWdAkFeM5lSoDnPA84R0TWADnAnaoasa3dAM/5DuAVEbkdp+N4TCR/sRORf+Mk8zpuv8dfgXIAqvoiTj/IecAGIAMYW+JjRvDPyxhjTBCUxUtDxhhjisESgTHGRDlLBMYYE+UsERhjTJSzRGCMMVHOEoEplUQkR0R+9HkkFrJvehCON11ENrvH+t69Q7W47/GqiLR3n9+Tb9u3JY3RfZ+8n8sqEflYRGoUsX/XSK/Gabxnw0dNqSQi6apaJdj7FvIe04HZqjpDRM4BHlfVziV4vxLHVNT7isjrwM+q+rdC9h+DU3X15mDHYsoOaxGYiCAiVdx5FL4XkZUiclKlURFpKCILfL4x93PXnyMii9zX/kdEivqAXgC0cl873n2vVSLyZ3ddZRGZIyI/ueuvcNfPF5EkEXkUqOjG8Za7Ld399x0ROd8n5ukicqmIxIrIYyKy1K0x/8cAfiyLcIuNiUhP9xx/EJFvReQ0907cB4Er3FiucGOfJiJL3H39VWw10SbctbftYQ9/D5y7Yn90Hx/i3AVfzd1WB+euyrwWbbr77x3Ave7zWJx6Q3VwPtgru+snAJP8HG86cKn7/DLgO+B0YCVQGeeu7NVAN+AS4BWf11Z3/52PO+dBXkw+++TFeBHwuvu8PE4VyYrAOOA+d30FYBnQ3E+c6T7n9x9gqLtcDYhznw8G3nefjwGe83n934Hfu89r4NQiqhzu37c9wvsocyUmTJlxVFW75i2ISDng7yJyFpCL8024PrDL5zVLgWnuvjNV9UcR6Y8zWck3bmmN8jjfpP15TETuw6lTcx1O/ZoPVfWIG8MHQD/gU+AJEfkHzuWkhcU4r0+Ap0WkAjAUWKCqR93LUZ1F5FJ3v+o4xeI253t9RRH50T3/tcB/ffZ/XURa45RZKFfA8c8BRojIX9zleCDBfS8TpSwRmEhxNVAXOF1Vs8SpKBrvu4OqLnATxfnAdBF5EjgA/FdVrwrgGHeq6oy8BREZ5G8nVf1ZnLkOzgMeFpEvVPXBQE5CVTNFZD5wLnAFzkQr4Mw2dYuqziviLY6qalcRqYRTf+cm4BmcCXi+VNWL3I71+QW8XoBLVHV9IPGa6GB9BCZSVAf2uEngbOCkOZfFmYd5t6q+AryKM93fYqCviORd868sIm0CPOZC4HciUklEKuNc1lkoIo2ADFX9P5xifv7mjM1yWyb+vItTKCyvdQHOh/qf8l4jIm3cY/qlzmxztwJ3yK+l1PNKEY/x2fUwziWyPPOAW8RtHolTldZEOUsEJlK8BSSJyErgGmCdn30GAD+JyA8437afVtW9OB+M/xaRFTiXhdoGckBV/R6n72AJTp/Bq6r6A9AJWOJeovkr8LCfl78MrMjrLM7nM5yJgT5XZ/pFcBLXGuB7cSYtf4kiWuxuLCtwJmb5J/CIe+6+r/sSaJ/XWYzTcijnxrbaXTZRzoaPGmNMlLMWgTHGRDlLBMYYE+UsERhjTJSzRGCMMVHOEoExxkQ5SwTGGBPlLBEYY0yU+3+R8HItEWG06QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA4xUlEQVR4nO3dd3hUZfbA8e9JKKEGQugkhN4DaAQBEQQUUATFguIqYGFd64qrYlnEsoq9/3YtIOqqqFgWBWXXgqAixUZHaZLQawiEQMr5/XFvcAgpE8idycycz/PMw9wy9547E+bce993ziuqijHGmMgVFewAjDHGBJclAmOMiXCWCIwxJsJZIjDGmAhnicAYYyKcJQJjjIlwlghM0InjVRHZIyILgx2P10RkjohcHew4jMlniSCCuV9Ie0SkciHzry4wr6+IpPlMi4jcJCLLROSAiKSJyHsi0uk4QjkNOBNooqrdiom3r4ioiNxxHPs4ISLSTURmicheEdktIgtFZEyg4/CJ5wr3vbjaZ56IyCMisst9PCIi4rO8n4j8KCL7RGSdiIwt5T5vEZGt7uunFPy7KbDuxSKyUkQyRGSFiJzn77ZEpKf7/maIyBIROa00cZrSs0QQoUQkCegNKDD0ODbxDHAzcBMQB7QGPgLOOY5tNQU2qOqBEtYbBewGrihuJRGpcBwxFLe9HsCXwNdAS6AO8BdgcFnupxTx1AbuApYXWDQWOA/oDCQD5wJ/dl9TEfgQeBGIBUYAT4pIZz/3ORAYD/TH+byaA/cVsW5j4N/AOKAmcBvwlojUK2lbIhIHfAw8BtQCHgU+do/ZeEVV7RGBD2AC8C3wJPBJgWVzgKsLzOsLpLnPWwG5QLdS7K8RMAPni3wNcI07/yogy93efuC+Il5fDcgALgEOAyk+y5JwEtpVwEZgrjv/SmAlsAeYDTT1ec0zQCqwD/gB6F1M7N8AL5RwfNe4x7XbPc5GPsvOBFYB6cDzOAnlap/lRcZZxL7+BVxX8HMCvgPG+kxfBXzvPq/vvkdVfZYvAi718/N7C3jIZ7o/sLWIdbsD2wvM2wH0KGlbwBBgeYHX/gpcFez/M+H8sCuCyHUF8Kb7GCgi9Uvx2v44SaE09/OnAWk4CeFC4CER6aeqk4FrgfmqWl1V7y3i9cNxEsV7OF+WowpZpw/QDud4huGcNQ8H6gLzgLd91l0EdMG5mnkLeE9EYgpuUESqAj2A6UUdmIj0Ax4GLgYaAr+7x4uIxAMfAPcA8cBaoJfPa0uKs+C+ugEpOMmgoA7ALz7Tv7jzUNVt7nbHiEi0e5XTFCfJISKJ7m2vxCJ2Xdi264tInULWXQysFJGh7r7OAw4BS/zclnA0AToWEZcpC8HORPYI/APnnnw2EO9OrwJu8Vk+h+KvCO7GPdP0c38JOGf8NXzmPQxMdZ+PBr4pYRufA0+7zy/FOcOs6E4n4ZztNvdZ/1N8ziJxboNmUsTZNs7ZeOdC5jd2t922mNgmA4/6TFd3398knIT7vc8ywUmIV5c2TiAa50v21MI+J/c9busz3cqNXdzpc4FtQI77uKYUn+FaYJDPdEV320lFrH8VTuLOcY/nHH+2hXPbba/7GVfESfh5wIvB/n8Tzg+7IohMo4D/qupOd/otjj7DzsH5T+irIs6XG8AunDNffzUCdqtqhs+833G+ZEskIgnAGThXLwD/AWI4tj0i1ed5U+AZ9yx3L84tG8nfp4j8zW3MTHeXx+KcsRe0B+eLqLjjbeQeDwCquh/nPWrsLkv1WaalibOA64Alqvp9EXHsx7knn68msF9VVUTa4lylXAFUwjkrv11E/G3TKWzb4NyuO4qIDMC5t9/X3Vcf4BUR6VLStlR1FzAMp31hGzAI5yQgDeMZSwQRRkSq4NzC6OP22tgK3AJ09mk43IhzduarGX982X0BNBGRFD93uxmIE5EaPvMSgU1+vv5ynL/Vj9141+EkgoK3h3xL6aYCf1bVWj6PKqr6nYj0Bm7HeR9qq2otnPv3BW9JoKqZwHzgghKOr2n+hIhUwzmz3QRswbkiyl8mvtPFxVnIfvoD5/t8bj2BJ0TkeXf5cpyG4nyd+aNBuSPwq6rOVtU8VV0NzMT/Bu/Ctr3N/eIuqAtOO81id1+LgAXAAH+2papfq+opqhqH89m3BcK+W3FQBfuSxB6BfeBccu/G+SJu4POYCzzhrjMQ2A50w/lybI3TmHmtz3aeA37jj7O+GJyG3PFF7HceTkNpDE6Plm3AAHfZaIq5NQSsBiYWiHcozn3nOvxxa6iCz2vOB5YBHdzpWOAi9/nZOF/eDdzYJ+DcVhlQxP574pzF3gbUced1Bqa5zwfg3KrqAlTGaYj+xl0Wj3PWPByogNPTKoc/bg0VGWchcdQq8B58h3PmHOsuv9b9nPKvRJbnf2ZAC/cY+rmfaQucxu2xRb3vBfY9CNgKtHfj+BKYVMS6fYCdQBd3uivOFdJZ/mzLXb8izpXC08C3wf5/E+6PoAdgjwB/4PAZ7hd+gfkXu/85K7jTV7pfJPvcL4zxQJTP+uJ+qS3HuQe8CXgn/wutkO03AT7BSUJrOTqpjKaIRACcitOrqG4hy5YDN1BIInCXXw4sdY8hFZjizo8Gprjzt+BcHWygiETgvqYbzv38dPcYFgBX+Cy/1j2u3e5xNvFZNgin50tRvYYKjdOPz3JOge0Izi2Z3e7jUdz2AZ/PeBlOYkoDHsn/THFODPYDicXsL/92zT7gVaBygc/iMp/pG9y/mwycK7hbS7Gtt933Kt39m6oX7P834f7Ib0QyxhgToayNwBhjIpwlAmOMiXCWCIwxJsJZIjDGmAhXpsW5AiE+Pl6TkpKCHYYxxoSUH374Yaeq1i1sWcglgqSkJBYvXhzsMIwxJqSIyO9FLbNbQ8YYE+EsERhjTISzRGCMMREu5NoICpOdnU1aWhpZWVnBDqXciomJoUmTJlSsWLCoqDEm0oVFIkhLS6NGjRokJSXhM0Srcakqu3btIi0tjWbNmgU7HGNMOePZrSF3QOrtIrKsiOUiIs+KyBp3gOqTjndfWVlZ1KlTx5JAEUSEOnXq2BWTMaZQXrYRTMWpuliUwTgjKLXCGXT7nyeyM0sCxbP3xxiPpC6EeU84/4bofjy7NaSqc0UkqZhVhgGvq1P+9HsRqSUiDVV1i1cxGWNMmUpdCK+eDXnZIFFQvyNUrlny60opNyudqO3LEVWoEAOjZkBCtzLbfjB7DTXm6CH70ihi6EIRGSsii0Vk8Y4dOwISXGmJCLfeeuuR6ccff5yJEyf6/fpt27YxZMgQOnfuTPv27Tn77LMBmDNnDkOGDDlm/RkzZjBp0iQAJk6cyOOPPw7A6NGjmT69yHHWjTFlacM8JwkAaB5kpZf5LtIPZrNt+zZn+yjkHnb2W4ZCorFYVV8CXgJISUkplwMoVK5cmQ8++IA777yT+PjChr4t3oQJEzjzzDO5+eabAViyZEmx6w8dOpShQ4ceV6zGmDKS1Nu5EtA8qFAFLnilzM7U0w9m8/CslUxbncrZtTbyXPZEovOyIbqSs98yFMwrgk0cPXZrE/wfw7bcqVChAmPHjuWpp546ZtmGDRvo168fycnJ9O/fn40bNx6zzpYtW2jSpMmR6eTk5GPWWbRoEV27dmXt2rVMnTqVG264oWwPwhhTOgndnNtBtZqW6e2a3Dzlgn9+x7uLU/lzn+Y8eetYokd/DP3uLvPbQhDcK4IZwA0iMg3oDqSXVfvAiBfnHzNvSHJDLu+RxMHDuYx+9djGlgtPbsJFKQnsPnCYv/z7h6OWvfPnHn7t9/rrryc5OZnbb7/9qPk33ngjo0aNYtSoUUyZMoWbbrqJjz766JjXjhgxgueff54BAwYwZswYGjVqdGT5d999x4033sh//vMfEhMTmTevbC8NjTHHqXJN51EGX857DhymVtWKREcJfzurDY1qxZDcpJazMKFbmSeAfF52H30bmA+0EZE0EblKRK4VkWvdVWbhjGW6BngZuM6rWAKlZs2aXHHFFTz77LNHzZ8/fz4jR44E4PLLL+ebb7455rUDBw5k3bp1XHPNNaxatYquXbuS3x6ycuVKxo4dy8cff0xiYqL3B2KMCShV5cOf0jjjiTlMW+Q0nQ7q2OCPJOAxL3sNXVrCcgWu92LfxZ3BV6kUXezyuGqV/L4CKMxf//pXTjrpJMaMGVPq18bFxTFy5EhGjhzJkCFDmDt3LnXq1KFhw4ZkZWXx008/HXWVYIznUhc6DZNJvT07Gw15h/Y5jcSpC4/rPdq89yB3f7iUr1bvoGtiLVKa1vYgyOKFRGNxKImLi+Piiy9m8uTJXHnllQD07NmTadOmcfnll/Pmm2/Su/exDT1ffvklp556KlWrViUjI4O1a9eSmJjIgQMHqFWrFpMnT+bMM8+kWrVq9O3bN8BHZSJSgLpGhrRD+2Cr27HjtaGlvn//n583cfeHy8jNUyYMac+onklERwX+Nz9WdM4Dt956Kzt37jwy/dxzz/Hqq6+SnJzMG2+8wTPPPHPMa3744QdSUlJITk6mR48eXH311ZxyyilHltevX59PPvmE66+/ngULFgTkOEyEC0DXyJDn+54cR7fO2CoV6ZJQi//ecjpXntYsKEkAQJw7NKEjJSVFCw5Ms3LlStq1axekiEKHvU+mVFIXwpSBf3SN9KC3SshLXehcCeQedrp1lvAe5eTmMfmb9WTn5nFDv1aA0z4QiF/+i8gPqppS2DK7NWSMKVx+18is9DLtHx9WEro5X/5+tKOs2LyPO95fwtJN6ZyT3PBIAigP5V8sERhjilaGXSPDVgndOg/l5PL8l2v455y11Kpakf+77CQGd2xQLhJAPksExoQyr3v1nGCPGAMbdmbyr6/XMrRLI/5+TntqV6sU7JCOYYnAmFDlda+eE+wRE8kOHMrhfyu2cV7XxrRpUIMvxvUlsU7VYIdVJEsExoSqwnr1lGUiKKxHjCWCEs37bQd3frCUTXsP0rFxTVrWq1GukwBYIjAmdHlY8Aw4tkdMGRc6Czfpmdn8Y9YK3l2cRvP4arwztgct69UIdlh+sd8RlJHq1auf8DYWL17MTTfdVOTyDRs28NZbb/m9vglzHhU8O2r7o2Z4VugsnOTmKRf86zve/3ET1/Vtwaybe9OtWVyww/KbXRGUIykpKaSkFNrNF/gjEeTXLSppfRMBvO7V42Ghs3Cw+8BhalVxisTdNrANjWtVoWPj2GCHVWqRe0UQgOHlfv75Z0499VSSk5M5//zz2bNnD+CUk05OTqZLly7cdtttdOzYETh6EJqvv/6aLl260KVLF7p27UpGRgbjx49n3rx5dOnShaeeeuqo9ffv38+YMWPo1KkTycnJvP/++54dlzGRTlV5/4c0znj8jyJxAzs0CMkkAOF4RfDpeNi6tPh1Du2Dbcuce6v+9LZo0AkGTyp1KFdccQXPPfccffr0YcKECdx33308/fTTjBkzhpdffpkePXowfvz4Ql/7+OOP88ILL9CrVy/2799PTEwMkyZN4vHHH+eTTz4BnMSR74EHHiA2NpalS51jz086JgSVpkuode8MuLQ9mdz14TLm/rqDk5vWDqlbQEUJv0Tgj6x0d9g3vOltAaSnp7N371769OkDwKhRo7jooovYu3cvGRkZ9OjhVDgdOXLkkS92X7169WLcuHFcdtllDB8+/KhBawrz+eefM23atCPTtWsHvoKhKQOl6RJq3TsD7sOf0rjnw2UocN/QDlx+alOiglQfqCyFXyLw58y9YG+Icvjz+fHjx3POOecwa9YsevXqxezZs4MdkgmE0nQJte6dARdXrTInJ8Xx0PkdaVK7fHcJLY3wSwT+KEV9kOMVGxtL7dq1mTdvHr179+aNN96gT58+1KpVixo1arBgwQK6d+9+1Fm8r7Vr19KpUyc6derEokWLWLVqFQkJCWRkZBS6/plnnskLL7zA008/DTi3huyqIASVpkuode/0XHZuHi/PW0dOrnJT/1b0aV2X01vFl6vyEGUhMhMBlHlviMzMzKNu34wbN47XXnuNa6+9lszMTJo3b86rr74KwOTJk7nmmmuIioqiT58+xMYe28D09NNP89VXXxEVFUWHDh0YPHgwUVFRREdH07lzZ0aPHk3Xrl2PrH/PPfdw/fXX07FjR6Kjo7n33nsZPnx4mR2fCZDSFHoLwAlNJFu2KZ073l/C8s37OLdzo3JVJK6sWRnqINi/f/+R3x1MmjSJLVu2FDpGQVkLtfcpYr16jvPvmJnBjSNCZWXn8uwXv/Hi3HXUrlqJB8/rwKCODYMd1gmzMtTlzMyZM3n44YfJycmhadOmTJ06NdghGWNcv+/K5OV56xjetTH3nNOe2KoVgx2S5ywRBMGIESMYMWJEsMMwvsrT2LzWJTTgDhzKYfbyrQw/qQltGtTgy1v7khAXPo3BJQmbRBCoUX5CVajdAgyo8jQ2r3UJDbivf93BXR8sZXP6QZKbxNKyXo2ISgIQJr8sjomJYdeuXfZlVwRVZdeuXcTExAQ7lPKpPI3Ne4Jj4Br/7TlwmHHv/syoKQuJqRjFe38OnSJxZS0srgiaNGlCWloaO3bsCHYo5VZMTEyJP0qLWF5X8SwN6xIaEPlF4n7flckNZ7Tkhn4tiakYHeywgiYsEkHFihVp1qxZsMMwoao8jc1rXUI9tWv/IWpXrUR0lDB+UFsa165Ch0ahWR+oLIVFIjDmhJWnsXmt4meZU1Xe+yGNBz9ZwR2D23JZ96ac1aFBsMMqNywRGGPCWuruTO76cCnzfttJt6Q4ejSvE+yQyh1LBMaYsPXBj2nc89EyBHjgvI5c1i0xLIrElTVLBMaYsBVfvTLdmsXxj/M70bhWlWCHU25ZIjDGhI3s3Dxe/HotuXlw84BWnN66Lqe3rhvssMo9SwTGmLCwbFM6t01fwsot+xjWpZH9yLQULBEYY0JaVnYuT3/+Gy/PW0dctUq8ePnJDLQeQaXi6S+LRWSQiKwWkTUicsyYjCKSKCJfichPIrJERM72Mh5jTPjZuDuTyd+s48KTmvD5LX0sCRwHzxKBiEQDLwCDgfbApSLSvsBq9wDvqmpX4BLg/7yKx5RzqQth3hPOv8FwaB+kpwZv/6ZUMrKyeW+xM2h86/o1+OpvfXnkwuSIqBTqBS9vDXUD1qjqOgARmQYMA1b4rKNAfnWvWGCzh/GY8irYRd+s0FtI+WrVdu7+cClb92XRNbEWLevVCKthI4PBy1tDjYFUn+k0d56vicCfRCQNmAXcWNiGRGSsiCwWkcVWTygMBbvomxV6Cwm7Dxzmlnd+ZszURVSrXIHpf+kZsUXiylqwG4svBaaq6hMi0gN4Q0Q6qmqe70qq+hLwEjgjlAUhTuOlYBd9s0Jv5V5unnLhP79j4+5MburfiuvPaEHlCpFbJK6seZkINgEJPtNN3Hm+rgIGAajqfBGJAeKB7R7GZcqbYBd9s0Jv5daOjEPUqeYUibvr7HY0rl2Fdg2DNFZEGPPy1tAioJWINBORSjiNwTMKrLMR6A8gIu2AGMDu/USiyjUhNiF4X8IJ3aD3rZYEyglV5Z1FG+n3xBzeWrgRgAHt61sS8IhnVwSqmiMiNwCzgWhgiqouF5H7gcWqOgO4FXhZRG7BaTgerTa6jDERbeOuTMZ/sITv1u6ie7M4TmsZH+yQwp6nbQSqOgunEdh33gSf5yuAXl7GYIwJHdN/SOPvHy0jOkr4x/kdufQUKxIXCMFuLDbGmCPq16xMzxZ1ePD8jjSMtSJxgWKJwBgTNIdz8vjnnLXkqXLLma3p3aouvVtZkbhAs0RgjAmKX1L3cvv0JazelsHwro2tSFwQWSIwxgTUwcO5PPm/1Uz+Zj31asTwyhUpDGhfP9hhRTRLBMaYgErdk8lr3/3OJd0SGT+4LTVjrD5QsFkiMMGVutD5IVfGFsjLcaatL3/Y2ZeVzWfLtnJxSgKt69dgzm19aWQjhpUblghM8PgWm8tnRd/CzpertnHXB8vYnpHFSYm1aVmvuiWBcsbT8QiMKZZvsbl8VvQtbOzaf4ibp/3ElVMXE1ulIh9c14uW9aoHOyxTCLsiMMHjW2wOnOdW9C0s5OYpF/1rPql7MrllQGv+0rcFlSrYeWd5ZYnABI9vsbnTxsHBXVb0LcRtz8givlploqOEu89pR5PaVWnTwEpFl3d+JwIRqaqqmV4GYyJQ5ZrOI2V0sCMxJyAvT3l70UYenrWKOwa35fJTm9K/nXUJDRUlXquJSE8RWQGscqc7i4gNKWmMAWDDzgOMfOV77v5wGclNYuljvwwOOf5cETwFDMQtIa2qv4jI6Z5GZUJPfjfQ0t7aObTPuTVk3UZD0ruLU/n7R8uoFB3FpOGdGHFKgv06OAT5dWtIVVMLfLi53oRjQtLxjjlsYwWHvMa1qnB667o8MKwjDWJjgh2OOU7+JIJUEekJqIhUBG4GVnoblgkphY057E8iKGysYEsE5dqhnFz+76u1qCrjzmpDr5bx9LLxAkKeP4ngWuAZnIHnNwH/Ba7zMigTYo53zGEbKzik/LRxD3e8v4Rft+3ngpOaWJG4MOJPImijqpf5zhCRXsC33oRkQs7xjjlsYwWHhMzDOTzx31+Z8u16GtSMYcroFPq1tR5B4cSfRPAccJIf80wky+8GWtov84RulgDKuU17DvLG979zWfdE7hjUlhpWJC7sFJkIRKQH0BOoKyLjfBbVxBmD2BgrGhem0g9m8+nSLVzSLZFW9Wvw9W19bcSwMFbcFUEloLq7ju9PA/cBF3oZlAkRVjQuLP13+Vbu+WgZuw4cJiUpjpb1qlsSCHNFJgJV/Rr4WkSmqurvAYzJhIriisZZIgg5O/cfYuKM5XyyZAttG9TglVEpViQuQvjTRpApIo8BHYAjHYVVtZ9nUZnQYEXjwkZunnLhP79j894s/nZWa/7cpwUVo61IXKTwJxG8CbwDDMHpSjoK2OFlUCZEWNG4kLdtXxZ1qztF4u49twNNalehVX0rEhdp/EkEdVR1sojc7HO7aJHXgZkQYUXjQlJenvLmwo088ukq7hjUhst7JHFG23rBDssEiT+JIP8m8BYROQfYDMR5F5Ixxkvrduxn/AdLWbh+N6e1jKdvG0sAkc6fRPCgiMQCt+L8fqAm8FcvgzLH4XiLvp0oKxoXUt5ZtJEJ/1lO5QpRPHphMhed3MR+HWxKTgSq+on7NB04A478stiUF8db9O1EWdG4kNOkdlX6tnGKxNWraUXijKO4H5RFAxfj1Bj6TFWXicgQ4C6gCtA1MCGaEh1v0bcTZUXjyr1DObk898UaAP420IrEmcIVd0UwGUgAFgLPishmIAUYr6ofBSA246/jLfp2oqxoXLn2w++7uX36EtbuOMDFKVYkzhStuESQAiSrap6IxABbgRaquiswoRm/HW/Rt7LYrxWNK3cOHMrhsdmreW3+BhrFVuG1K7vRp7WNGmaKVlwiOKzq/FJIVbNEZF1pk4CIDMIpYR0NvKKqkwpZ52JgIqDAL6o6sjT7MK7jLfp2oqxoXLmzee9B3lq4kStObcptg9pSvbLfQ5ObCFXcX0hbEXFbAhGghTstgKpqcnEbdtsYXgDOBNKARSIyQ1VX+KzTCrgT6KWqe0TE+rHZkI/mOKRnZjNz6RZGdneKxM27/QzqW2Ow8VNxiaDdCW67G7BGVdcBiMg0YBiwwmeda4AXVHUPgKpuP8F9hjYb8tEch8+WbeXv/1nG7gOH6d48jhZ1q1sSMKVSXNG5Ey001xhI9ZlOA7oXWKc1gIh8i3P7aKKqflZwQyIyFhgLkJiYeIJhlWM25KMphe0ZWUycsZxZS7fSvmFNXh19Ci3qWpE4U3rBvnlYAWgF9AWaAHNFpJOq7vVdSVVfAl4CSElJ0QDHGDg25KPxU26ecvG/5rM5PYvbBrZh7OnNrUicOW5eJoJNON1P8zVx5/lKAxaoajawXkR+xUkMkVnLyIZ8NCXYkn6Q+jVinCJxQzuQULuqlYo2J8yvUwgRqSIibUq57UVAKxFpJiKVgEuAGQXW+QjnagARice5VbSulPsJL5VrQmzC8Q352PtWSwJhKi9Pmfrtevo/8TX/XuDctT2jTT1LAqZMlJgIRORc4GfgM3e6i4gU/EI/hqrmADcAs4GVwLuqulxE7heRoe5qs4FdIrIC+Aq4zX6nYMzR1mzfz8UvzmfixytISYqjn1UJNWXMn1tDE3F6AM0BUNWfRaSZPxtX1VnArALzJvg8V2Cc+zDGFDBt4UYmzFhOlYrRPHFRZ4af1Nh+HWzKnF9lqFU1vcAfX/g22BpTjiTWqcqAdvW4b2hH6taoHOxwTJjyJxEsF5GRQLT7A7CbgO+8DcuYyJSVncuzX/wGwO2D2tKzRTw9W1iROOMtfxqLb8QZr/gQ8BZOOeq/ehiTMRFp8YbdnP3sPP5vzlp2HziMc+fUGO/5c0XQVlXvBu72OhhjItH+Qzk89tkqXv/+dxrXqsLrV3bjdCsSZwLIn0TwhIg0AKYD76jqMo9jMiaibE0/yLRFqYzqkcRtA9tQzYrEmQAr8daQqp6BMzLZDuBFEVkqIvd4HlkgpC6EeU84/5YXh/ZBemr5ismUuT0HDvPG987vAVrWc4rETRzawZKACQq//upUdSvO4DRfAbcDE4AHvQzMc8Ea3rE4Vjwu7Kkqny7byoT/LGNvZjY9W9ShRd3qNmykCSp/flDWTkQmishSnMHrv8MpFxHaCivwFmyFFY8zYWP7viyu/fcPXPfmjzSMrcKMG06zInGmXPDnimAK8A4wUFU3exxP4ARreMfiWPG4sJWbp1z04ny2pmdx5+C2XHVaMypYkThTTpSYCFS1RyACCbhgDe9YUkxWPC6sbN57kAY1nSJx9w/rSELtKjS3qwBTzhSZCETkXVW92L0l5Nuh2a8RykJCsIZ3LI4N/RgWcvOU1+dv4NHPVnPn2W25okeSjRtsyq3irghudv8dEohAjAkXa7ZncPv0Jfy4cS9929Slf7v6wQ7JmGIVN0LZFvfpdap6h+8yEXkEuOPYVxkT2d5asJGJM5ZTrXI0T43ozHldrEicKf/8aa06s5B5g8s6EGPCQVJ8Vc7qUJ//jevD+V2bWBIwIaG4NoK/ANcBzUVkic+iGsC3XgdmTCjIys7lqc9/RRDGD7YicSY0FddG8BbwKfAwMN5nfoaq7vY0KmNCwIJ1uxj/wVLW7zzAZd0TUVW7AjAhqbhEoKq6QUSuL7hAROIsGZhIlZGVzSOfreLf328kMa4qb13dnZ4t7SrAhK6SrgiGAD/gdB/1PdVRoLmHcRlTbm3bd4jpP6Rx9WnNGHdWa6pWsvpAJrQV12toiPuvX8NSGhPOdh84zMwlm7m8RxIt61Vn3u39bMQwEzZKPJURkV7Az6p6QET+BJwEPK2qGz2PzpggU1U+WbKFiTOWsy8rm14t42let7olARNW/Ok++k8gU0Q6A7cCa4E3PI3KmHJg274srnn9B258+yca167CxzeeZuUhTFjy5+ZmjqqqiAwDnlfVySJyldeBGRNMuXnKxW6RuLvPbseYXklWJM6ELX8SQYaI3AlcDvQWkSigordhGRMcaXsyaRhbhego4YFhHUmMq0pSfLVgh2WMp/w5xRmBM3D9le4ANU2AxzyNypgAy81TXpm3jgFPfs2/3ZHDTm9d15KAiQj+lKHeKiJvAqeIyBBgoaq+7n1oxgTG6q0Z3P7+En5J3Uv/tvU4q4MViTORxZ9eQxfjXAHMwfktwXMicpuqTvc4Nu+kLnRq/mdsgbwcZ9pKP0ekf3//O/d9vJwaMRV55pIuDO3cyH4dbCKOP20EdwOnqOp2ABGpC3wOhGYi8B2rOJ+NDxxx8stBtKxXnbM7NWTCkPbUqW5dQk1k8icRROUnAdcu/GtbKJ98xyrOlz8+sCWCsHfwcC5P/m81UVHCnYPbcWrzOpzavE6wwzImqPxJBJ+JyGzgbXd6BDDLu5A85jtWMTjPbXzgiDB/7S7Gf7CE33dlcvmpTa1InDEufxqLbxOR4cBp7qyXVPVDb8PykO9YxaeNg4O7bHzgMLcvK5uHZ63i7YUbaVqnKm9d091KRRvjo7jxCFoBjwMtgKXA31R1U6AC81T+WMUpo4MdiQmA7fsO8dFPmxh7enNuGdCaKpWigx2SMeVKcff6pwCfABfgVCB9rrQbF5FBIrJaRNaIyPhi1rtARFREUkq7D2MKs2v/IaZ+ux6AlvWq880dZ3DX2e0sCRhTiOJuDdVQ1Zfd56tF5MfSbFhEooEXcIa6TAMWicgMVV1RYL0awM3AgtJsv9Tyu4wm9YZD+5xbQ9ZtNOyoKjN+2czEGcvZfyiH01vXpXnd6tYjyJhiFJcIYkSkK3+MQ1DFd1pVS0oM3YA1qroOQESmAcOAFQXWewB4BLitlLH776guo4IznALWbTTMbN57kHs+WsaXq7bTJaEWj16YbEXijPFDcYlgC/Ckz/RWn2kF+pWw7cZAqs90GtDddwUROQlIUNWZIlJkIhCRscBYgMTExBJ2W4ijuozqH/Ot22jYyMnN45KXvmdHxiH+PqQ9o3smER1lPYKM8UdxA9Oc4eWO3eJ1TwKjS1pXVV8CXgJISUnRElY/lm+X0ehKgDi/KLZuoyEvdXcmjWpVoUJ0FA+d34nEuKok1qka7LCMCSlejrG3CUjwmW7izstXA+gIzHH7cjcAZojIUFVdXKaR+HYZveAVZ15+e4FdDYSknNw8pny7nif++yt3Dm7L6F7NOK2VdQk15nh4mQgWAa1EpBlOArgEGJm/UFXTgSP/c0VkDk4X1bJNAvnyu4zmf/FbAghZK7fs4473l7AkLZ0z29dncKeGwQ7JmJDmWSJQ1RwRuQGYDUQDU1R1uYjcDyxW1Rle7duErzfmb+C+j1cQW6Uiz4/syjmdGtqvg405Qf5UHxXgMqC5qt4vIolAA1VdWNJrVXUWBcpRqOqEItbt61fEJiLll4NoXb8G53ZuxN+HtCeuWqVgh2VMWPDniuD/gDycXkL3AxnA+8ApHsZlDACZh3N4fPavVIgW7jq7Hd2b16G7FYkzpkz5U0W0u6peD2QBqOoewE7FjOe+XbOTgU/PZcq36zmck4dq6TuMGWNK5s8VQbb7K2GFI+MR5HkalYlo6QezeWjmSt5ZnEqz+Gq8++cedGsWF+ywjAlb/iSCZ4EPgXoi8g/gQuAeT6MyEW3n/kN8vGQz1/ZpwV8HtCKmotUHMsZL/pShflNEfgD649RnOE9VV3oemYkoOzIO8fEvm7nytGa0qFudb+7oZ43BxgSIP72GEoFM4GPfeaq60cvATGRQVT76eRP3fbyCzEO5nNG2Hs3iq1kSMCaA/Lk1NBOnfUCAGKAZsBro4GFcJgJs2nuQuz9cypzVOzgp0SkS1yy+WrDDMibi+HNrqJPvtFso7jrPIjIRwSkSN59d+w8z8dz2XN7DisQZEyyl/mWxqv4oIt1LXtOYY23clUnj2k6RuEnDk0mMq0pCnBWJMyaY/GkjGOczGQWcBGz2LCITlnJy83h53nqe+twpEjemVzN6tbQiccaUB/5cEdTweZ6D02bwvjfhmHC0fHM6d7y/hGWb9jGwQ33OsSJxxpQrxSYC94dkNVT1bwGKx4SZ177bwAOfrKBW1Ur887KTrFKoMeVQkYlARCq4FUR7BTIgEx7yi8S1bVCDYV0a8/ch7ahV1bqEGlMeFXdFsBCnPeBnEZkBvAccyF+oqh94HJsJQQcO5fDY7NVUjBbuPqe9FYkzJgT400YQA+zCqT6a/3sCBSwRmKPM/XUHd36wlM3pBxnVI+nIVYExpnwrLhHUc3sMLeOPBJDPykCaI9Izs3lg5gqm/5BG87pOkbhTkqxInDGhorhEEA1U5+gEkM8SgTli54FDfLp0C9f1bcFN/a1InDGhprhEsEVV7w9YJCakbM/IYsbPm7m6d/MjReJqW30gY0JScYnAbu6aY6gq7/+4iQc+WcHB7Fz6t6tPs/hqlgSMCWHFJYL+AYvChITU3Znc9eFS5v22k5SmtZl0gRWJMyYcFJkIVHV3IAMx5VtObh6Xvvw9ew4c5oFhHbise1OirEicMWGh1EXnTGTZsPMACXFVqRAdxaMXOkXimtS2InHGhBN/Bq83ESg7N48XvlrDWU/N5fX5GwDo2SLekoAxYciuCMwxlm1K5/bpS1ixZR/ndGrIkORGwQ7JGOMhSwTmKK9+u54HZ64krlol/vWnkxnUsUGwQzLGeMwSgQH+KBLXoVEsw7s25p5z2hNbtWKwwzLGBIAlggi3/1AOj362ikrRUdwzpD3dmsXRrZmVhzAmklhjcQSbs3o7A5+ayxvf/47iXBUYYyKPXRFEoD0HDvPAzBV88OMmWtarzvRre3Jy09rBDssYEySWCCLQnszD/Hf5Nm7q15Lr+7WkcgUrEmdMJPP01pCIDBKR1SKyRkTGF7J8nIisEJElIvKFiDT1Mp5Itn1fFi/NXYuq0rxudb69ox/jzmpjScAY410icMc7fgEYDLQHLhWR9gVW+wlIUdVkYDrwqFfxRCpV5d1FqfR/8mue+O+vbNiVCWA9gowxR3h5a6gbsEZV1wGIyDRgGLAifwVV/cpn/e+BP3kYT8RJ3Z3JnR8s5Zs1O+nWLI5JwztZkThjzDG8TASNgVSf6TSgezHrXwV8WtgCERkLjAVITEwsq/jCWn6RuL2Z2Tx4XkdGdku0InHGmEKVi8ZiEfkTkAL0KWy5qr4EvASQkpJifRyLsX7nARLdInGPXdiZpnWq0qhWlWCHZYwpx7xsLN4EJPhMN3HnHUVEBgB3A0NV9ZCH8YS17Nw8nvviNwY+NZfXvtsAQI8WdSwJGGNK5OUVwSKglYg0w0kAlwAjfVcQka7Ai8AgVd3uYSxhbUnaXm6fvoRVWzM4t3MjhnaxInHGGP95lghUNUdEbgBmA9HAFFVdLiL3A4tVdQbwGFAdeE9EADaq6lCvYgpHU75Zz4MzV1C3RmVeviKFM9vXD3ZIxpgQ42kbgarOAmYVmDfB5/kAL/cfzvKLxCU3iWXEKQmMH9yO2CrWJdQYU3rlorHY+C8jK5tJn66icoVoJpzbnpSkOFKSrEicMeb4WdG5EPLVqu2c9dRc3l64kQrRYkXijDFlwq4IQsDuA4e5/+PlfPTzZlrXr87/XdaTrolWJM4YUzYsEYSA9IPZfLFyOzf3b8X1Z7SkUgW7kDPGlB1LBOXU1vQsPvp5E38+vTnN4qvxzfh+1hhsjPGEJYJyRlWZtiiVh2auJDsvj0EdGpAUX82SgDHGM5YIypHfdx1g/PtLmb9uF6c2j2PS8GSSrEicMcZjlgjKiZzcPEa+vID0g9k8dH4nLjklwYrEGWMCwhJBkK3dsZ+mbpG4Jy52isQ1jLX6QMaYwLHuJ0FyOCePpz//lUFPz+X1+b8DcGrzOpYEjDEBZ1cEQfBz6l7umL6E1dsyGNalEed1bRzskIwxEcwSQYBN/mY9/5i5gno1Ypg8KoX+7axInDEmuCwRBEh+kbguCbFc0i2R8YPbUjPGuoQaY4LPEoHH9mVl8/CsVcRUjOLecztwctM4Tm5qReKMMeWHNRZ76PMV2zjzya95Z9FGKlWIsiJxxphyya4IPLBr/yHu+3gFM37ZTNsGNXjp8hQ6J9QKdljGGFMoSwQeyMjK4avV27llQGv+0reFFYkzxpRrlgjKyOa9B/nwp01c17cFSfHV+HZ8P2sMNsaEBEsEJygvT3lr4UYmfbqK3DzlnE4NSYqvZknAGBMyLBGcgPU7DzD+/SUsWL+bXi3r8PD5ySTWqRrssIwxplQsERynnNw8/vTKAvZlZfPoBclclNIEESsSZ4wJPZYISmnN9gyS6lSjQnQUT43oQtM6ValfMybYYRljzHGz7ix+OpSTy5P/+5VBT8/jNbdIXLdmcZYEjDEhz64I/PDjxj3cMX0Jv23fz/CujRluReKMMWHEEkEJXp67joc+XUnDmjG8OuYUzmhTL9ghGWNMmYqcRHBoH2SlQ+pCSOhW4up5eUpUlHBS01pc1j2ROwa1pYZ1CTXGhKHISASpC2HbMtA8eG0ojJpRZDJIP5jNP2auoErFaO4b1tGKxBljwl5kNBZvmOckAYDcw850IWYv38qZT37N+z9uolrlClYkzhgTESLjiiCpN0iUkwyiKznTPnbuP8S9/1nOzKVbaN+wJlNGn0LHxrFBCtYYYwIrMhJBQjeo39FpI7jglWNuC+3PymHebzu4bWAbxp7enIrRkXGhZIwxECmJAKByTefhJoFNew/y4Y9pXH9GS5Liq/Hdnf2pXjly3g5jjMnn6amviAwSkdUiskZExheyvLKIvOMuXyAiSV7GA05voDfmb+CsJ7/mha/W8vuuTABLAsaYiOXZt5+IRAMvAGcCacAiEZmhqit8VrsK2KOqLUXkEuARYIQnAR3aR/aBvUx8fgpvbm5A71bxPHR+JxLirEicMSayeXlF0A1Yo6rrVPUwMA0YVmCdYcBr7vPpQH/xonJb6kJ02zIqZGzknt3jmdw/j9ev7GZJwBhj8DYRNAZSfabT3HmFrqOqOUA6UKfghkRkrIgsFpHFO3bsKH0kG+YhqggQI7n0j/nVKoUaY4wrJLrHqOpLqpqiqil169Yt/QaSekOFGJBopJDuo8YYE8m8bCHdBCT4TDdx5xW2TpqIVABigV1lHklCN+fXxBvmOUnAjxITxhgTKbxMBIuAViLSDOcL/xJgZIF1ZgCjgPnAhcCX6tXPeRO6WQIwxphCeJYIVDVHRG4AZgPRwBRVXS4i9wOLVXUGMBl4Q0TWALtxkoUxxpgA8rTzvKrOAmYVmDfB53kWcJGXMRhjjCleSDQWG2OM8Y4lAmOMiXCWCIwxJsJZIjDGmAgnoTb4iojsAH4/zpfHAzvLMJxQYMccGeyYI8OJHHNTVS30F7khlwhOhIgsVtWUYMcRSHbMkcGOOTJ4dcx2a8gYYyKcJQJjjIlwkZYIXgp2AEFgxxwZ7JgjgyfHHFFtBMYYY44VaVcExhhjCrBEYIwxES4sE4GIDBKR1SKyRkTGF7K8soi84y5fICJJQQizTPlxzONEZIWILBGRL0SkaTDiLEslHbPPeheIiIpIyHc19OeYReRi97NeLiJvBTrGsubH33aiiHwlIj+5f99nByPOsiIiU0Rku4gsK2K5iMiz7vuxREROOuGdqmpYPXBKXq8FmgOVgF+A9gXWuQ74l/v8EuCdYMcdgGM+A6jqPv9LJByzu14NYC7wPZAS7LgD8Dm3An4CarvT9YIddwCO+SXgL+7z9sCGYMd9gsd8OnASsKyI5WcDnwICnAosONF9huMVQTdgjaquU9XDwDRgWIF1hgGvuc+nA/0ltAcxLvGYVfUrVc10J7/HGTEulPnzOQM8ADwCZAUyOI/4c8zXAC+o6h4AVd0e4BjLmj/HrEBN93kssDmA8ZU5VZ2LMz5LUYYBr6vje6CWiDQ8kX2GYyJoDKT6TKe58wpdR1VzgHSgTkCi84Y/x+zrKpwzilBW4jG7l8wJqjozkIF5yJ/PuTXQWkS+FZHvRWRQwKLzhj/HPBH4k4ik4Yx/cmNgQgua0v5/L5GnA9OY8kdE/gSkAH2CHYuXRCQKeBIYHeRQAq0Czu2hvjhXfXNFpJOq7g1mUB67FJiqqk+ISA+cUQ87qmpesAMLFeF4RbAJSPCZbuLOK3QdEamAczm5KyDRecOfY0ZEBgB3A0NV9VCAYvNKScdcA+gIzBGRDTj3UmeEeIOxP59zGjBDVbNVdT3wK05iCFX+HPNVwLsAqjofiMEpzhau/Pr/XhrhmAgWAa1EpJmIVMJpDJ5RYJ0ZwCj3+YXAl+q2woSoEo9ZRLoCL+IkgVC/bwwlHLOqpqtqvKomqWoSTrvIUFVdHJxwy4Q/f9sf4VwNICLxOLeK1gUwxrLmzzFvBPoDiEg7nESwI6BRBtYM4Aq399CpQLqqbjmRDYbdrSFVzRGRG4DZOD0OpqjqchG5H1isqjOAyTiXj2twGmUuCV7EJ87PY34MqA6857aLb1TVoUEL+gT5ecxhxc9jng2cJSIrgFzgNlUN2atdP4/5VuBlEbkFp+F4dCif2InI2zjJPN5t97gXqAigqv/CaQc5G1gDZAJjTnifIfx+GWOMKQPheGvIGGNMKVgiMMaYCGeJwBhjIpwlAmOMiXCWCIwxJsJZIjDlkojkisjPPo+kYtbdXwb7myoi6919/ej+QrW023hFRNq7z+8qsOy7E43R3U7++7JMRD4WkVolrN8l1KtxGu9Z91FTLonIflWtXtbrFrONqcAnqjpdRM4CHlfV5BPY3gnHVNJ2ReQ14FdV/Ucx64/Gqbp6Q1nHYsKHXRGYkCAi1d1xFH4UkaUickylURFpKCJzfc6Ye7vzzxKR+e5r3xORkr6g5wIt3deOc7e1TET+6s6rJiIzReQXd/4Id/4cEUkRkUlAFTeON91l+91/p4nIOT4xTxWRC0UkWkQeE5FFbo35P/vxtszHLTYmIt3cY/xJRL4TkTbuL3HvB0a4sYxwY58iIgvddQur2GoiTbBrb9vDHoU9cH4V+7P7+BDnV/A13WXxOL+qzL+i3e/+eytwt/s8GqfeUDzOF3s1d/4dwIRC9jcVuNB9fhGwADgZWApUw/lV9nKgK3AB8LLPa2Pdf+fgjnmQH5PPOvkxng+85j6vhFNFsgowFrjHnV8ZWAw0KyTO/T7H9x4wyJ2uCVRwnw8A3nefjwae93n9Q8Cf3Oe1cGoRVQv2522P4D7CrsSECRsHVbVL/oSIVAQeEpHTgTycM+H6wFaf1ywCprjrfqSqP4tIH5zBSr51S2tUwjmTLsxjInIPTp2aq3Dq13yoqgfcGD4AegOfAU+IyCM4t5PmleK4PgWeEZHKwCBgrqoedG9HJYvIhe56sTjF4tYXeH0VEfnZPf6VwP981n9NRFrhlFmoWMT+zwKGisjf3OkYINHdlolQlghMqLgMqAucrKrZ4lQUjfFdQVXnuoniHGCqiDwJ7AH+p6qX+rGP21R1ev6EiPQvbCVV/VWcsQ7OBh4UkS9U9X5/DkJVs0RkDjAQGIEz0Ao4o03dqKqzS9jEQVXtIiJVcervXA88izMAz1eqer7bsD6niNcLcIGqrvYnXhMZrI3AhIpYYLubBM4AjhlzWZxxmLep6svAKzjD/X0P9BKR/Hv+1USktZ/7nAecJyJVRaQazm2deSLSCMhU1X/jFPMrbMzYbPfKpDDv4BQKy7+6AOdL/S/5rxGR1u4+C6XOaHM3AbfKH6XU80sRj/ZZNQPnFlm+2cCN4l4eiVOV1kQ4SwQmVLwJpIjIUuAKYFUh6/QFfhGRn3DOtp9R1R04X4xvi8gSnNtCbf3Zoar+iNN2sBCnzeAVVf0J6AQsdG/R3As8WMjLXwKW5DcWF/BfnIGBPldn+EVwEtcK4EdxBi1/kRKu2N1YluAMzPIo8LB77L6v+wpon99YjHPlUNGNbbk7bSKcdR81xpgIZ1cExhgT4SwRGGNMhLNEYIwxEc4SgTHGRDhLBMYYE+EsERhjTISzRGCMMRHu/wG8plgmEL0CegAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA5kUlEQVR4nO3dd3hUZfbA8e9JKKGEltAhhN5DMYKACFKkiLBWXF0EG3bd1VWwIZa1d9d1LSDqqrgiKgrK/iwICgqoSEdpktAJEBJCQsr5/XFvcAgpE5LJtPN5nnmY2+aem4Q58973nfOKqmKMMSZ8Rfg7AGOMMf5licAYY8KcJQJjjAlzlgiMMSbMWSIwxpgwZ4nAGGPCnCUC43fieF1EDojIUn/H42siskBErvJ3HMbks0QQxtw3pAMiUrWQ9VcVWDdQRJI9lkVEbhaR1SJyWESSReR9Eel6EqGcDgwFmqlqr2LiHSgiKiKTTuIcZSIivURknogcFJH9IrJURC73Qxzq/rzT3cdrHtvOFJGvRSRVRLYWcmy8uz1DRNaLyJBSnFdE5DERSXEfj4mIFLP/TSKyRUQOichyETm9wPaeIrLQvYbdInJLIa8xwL3eh7yN05wcSwRhSkTigf6AAqNP4iWeA24BbgbqAe2Aj4CzT+K1WgBbVfVwCfuNB/YDlxW3k4hUOokYinu9PsBXwDdAGyAGuA4YUZ7nKYVuqlrTfXgm7MPAdOD2Io57F/gZJ/67gVkiUt/Lc04E/gR0AxKAc4BrCttRRHoDjwIXALWBacCHIhLpbo8FPgdedmNpA/yvwGtUxvkb+8HL+ExZqKo9wvABTAG+A54GPi2wbQFwVYF1A4Fk93lbIBfoVYrzNQHm4LyRbwSudtdfCWS6r5cO3F/E8TWANOBi4CiQ6LEtHiehXQlsAxa6668A1gEHgPlAC49jngOSgEPAj0D/YmL/FnixhOu72r2u/e51NvHYNhRYD6QC/8RJKFd5bC8yzkLOo0CbEmIZgpNYPde1A7KAaI91i4Brvfz9LQYmeixfCXxfxL5jgaUFfncKNHaXHwbeKuF8k4HHgRnAQ/7+/xLqD2sRhK/LgLfdxzARaViKYwfjJIXS3M+fCSTjJIQLgIdFZJCqTgOuBZao8wn3viKOPw8nUbyP82Y5vpB9BgAdca5nDHCXe1x9nDe9dz32XQZ0x2nNvAO8LyJRBV9QRKoDfYBZRV2YiAwCHgEuAhoDv7vXm//pdzZwDxALbAL6eRxbUpyFWSgiu0Rkttuy80ZnYLOqpnms+8Vdj4icLiIHSzj+l8KOLcRnQKSI9HZbAVcAK4Bd7vbTgP0islhE9ojIJyISl3+wiLRwj3nAy2szZWSJIAy592tbAP9V1R9x3pwuKcVLxAA7S3G+5jhvfpNUNVNVVwCvUcItngLGA++pai7OG/fF7u0DT1NV9bCqHsFJLo+o6jpVzcH5FNrdfZNBVf+jqimqmqOqTwFVgfaFnLcuzv+T4q73UmC6qv6kqlnAnUAf9016JLBGVWepajbwLH+8IVJSnIUYgNMC6gDsAD718lZYTZwWiadUIBpAVb9V1TqlOD4VqFlEP0Ea8AFOSyoLuA+nNZFf2KwZzu/zFiAO2MLxye954F5VTS/5skx5sEQQnsYD/1PVfe7yOxz/CTsHKPgmWxnIdp+n4Hzy9VYTYH+BT6O/A029OdhNJGfitF4APgaiOLE/IsnjeQvgObdz9yDOLRvJP6eI/F1E1rkdqwdx7mXHFnL6A0AexV9vE/d6AHDfwFLcczXxjMt9M/Q6zoJUdaGqHlXVgzhvpC1xWkElSQdqFVhXC+dN2xsFj68FpHu8uXu6Ergcp8VQBfgLTsJq4m4/AnyoqstUNRO4H+grIrVF5Byc21fveRmXKQeWCMKMiFTDuYUxwL29sAv4G9BNRLq5u23D+dTpqSV/vNl9CTQTkUQvT7sDqCci0R7r4oDtXh4/Dudv9RM33s04iaDg7SHPN6Uk4BpVrePxqKaqi0WkP3AHzs+hrvtJOBXnDfj4F1TNAJYA55dwfcc+wYtIDZxW03aclkRzj23iuVxcnMWcr+A1Fzl6x8MaoFWB30E3d7031rj7e3Nsd5x+p19VNU9VP8f5OfR1t6/k+N+V5/PBQKLH3+ZY4K8i8rGXcZqT4e9OCntU7AP4M86nzjigkcdjIfCUu88wYA/QC+dNph1OZ+a1Hq/zAvAbTidyFZw35ouByUWcdxFOR2kUzqiT3cAQd9sE4NtiYt4ATC0Q72ic2w4x/NFZXMnjmHOB1UBnd7k2cKH7fCTOm3cjN/YpOJ3VQ4o4f1+cT8S3AzHuum7ATPf5EGAvzhtgVZyO6G/dbbE4n7rPAyrhfIrPwe0sLi7OQuLo7J4jEudWzbPuz6ayuz3C/fmOwEnaUUAVj+O/B550158LHATqe/l3c637N5DfyllDER3NOAn6V6CV+/czFMgAOrjbB+G0tLrjtDSfARa526IL/J7fc7fX8/f/nVB++D0Ae1TwL9wZtvdUIesvwrl3XcldvsL9z34IZzTMZCDCY39x39TWuP/Jt7v/aTsXcd5mwKc4SWgTxyeVCRSRCHA6FjMLe8Nyz30jhSQCd/s4YJV7DUk49/Fx30inu+t34rQOtlJEInCP6YXTCZrqXsMPwGUe2691r2u/e53NPLYNd98Yixo1VGichcQwCOeN/zBOov4IaOuxfaD7c/B8LPDYHo8zIuyI+zpDPLb1x7nVU9T1C84onv3u43FAPLan4468cvd9AKdlmYaTQMYVeL3r3L+ZA8AnQPMizjsDGzXk84e4P2xjjDFhyvoIjDEmzFkiMMaYMGeJwBhjwpwlAmOMCXPlWpyrIsTGxmp8fLy/wzDGmKDy448/7lPVQosMBl0iiI+PZ/ny5f4OwxhjgoqI/F7UNrs1ZIwxYc4SgTHGhDlLBMYYE+aCro+gMNnZ2SQnJ5OZmenvUAJWVFQUzZo1o3LlgkVFjTHhLiQSQXJyMtHR0cTHx1PMNKphS1VJSUkhOTmZli1b+jscY0yA8dmtIRGZ7s4+tLqI7SIiz4vIRhFZKSI9T/ZcmZmZxMTEWBIogogQExNjLSZjTKF82UcwA6fqYlFG4Mx92xZnYuyXynIySwLFs5+PMQEmaSksesr51xf7l4LPbg2p6sIS5lMdA7ypTvnT70Wkjog0VlWvp0A0xpiglLQUXh8JedkgEdCwC1QtOIHcH3IzU4nYswZRhUpRMH4ONO9VbuH4c9RQU46fsi+ZIqbnE5GJIrJcRJbv3bu3QoIrLRHhtttuO7b85JNPMnXqVK+P3717N6NGjaJbt2506tSJkSNHArBgwQJGjRp1wv5z5szh0UcfBWDq1Kk8+eSTAEyYMIFZs4qcZ90YEwi2LnKSAIDmQWbB6aT/kHokm917djv7oZB71Dm+HAVFZ7GqvgK8ApCYmBiQEyhUrVqV2bNnc+eddxIbW9jUt8WbMmUKQ4cO5ZZbbgFg5cqVxe4/evRoRo8efVKxGmP8LL6/0xLQPKhUDc5/7YRP+KlHsnlk3jpmbkhiZJ1tvJA9lci8bIis4hxfjvzZItjO8XO3NsP7OWwDTqVKlZg4cSLPPPPMCdu2bt3KoEGDSEhIYPDgwWzbtu2EfXbu3EmzZs2OLSckJJywz7Jly+jRowebNm1ixowZ3HjjjeV7EcaYitG8l3M7qE6LQm/z5OYp57+0mP8uT+KaAa14+raJRE74BAbdXe63hcC/LYI5wI0iMhPoDaSWV//A2JeXnLBuVEJjxvWJ58jRXCa8fmJnywWnNOPCxObsP3yU6/7z43Hb3rumj1fnveGGG0hISOCOO+44bv1NN93E+PHjGT9+PNOnT+fmm2/mo48+OuHYsWPH8s9//pMhQ4Zw+eWX06RJk2PbFy9ezE033cTHH39MXFwcixaVb9PQGFPBqtZyHh5v6gcOH6VO9cpERgh/P6s9TepEkdCsjrOxea9yTwD5fDl89F1gCdBeRJJF5EoRuVZErnV3mQdsxpkP91Xgel/FUlFq1arFZZddxvPPP3/c+iVLlnDJJZcAMG7cOL799tsTjh02bBibN2/m6quvZv369fTo0YP8/pB169YxceJEPvnkE+Li4nx/IcaYCqWqfPhzMmc+tYCZy5yu0+FdGv2RBHzMl6OG/lzCdgVu8MW5i/sEX61KZLHb69Wo4nULoDB//etf6dmzJ5dffnmpj61Xrx6XXHIJl1xyCaNGjWLhwoXExMTQuHFjMjMz+fnnn49rJZgwkbTU6RyM7++zT4TGD7IOQWYqe9cu4o4fqvL1hr30iKtDYou6FR5KUHQWB5N69epx0UUXMW3aNK644goA+vbty8yZMxk3bhxvv/02/fuf2NHz1Vdfcdppp1G9enXS0tLYtGkTcXFxHD58mDp16jBt2jSGDh1KjRo1GDhwYAVflfGbUg4zNEEi6xDsWokCNd87j8y8e5ky6hzG940nMqLiv/NjRed84LbbbmPfvn3Hll944QVef/11EhISeOutt3juuedOOObHH38kMTGRhIQE+vTpw1VXXcWpp556bHvDhg359NNPueGGG/jhhx8q5DpMACjFMEMTRNzfowBVJIeXTs/gitNb+iUJAIhzhyZ4JCYmasGJadatW0fHjh39FFHwsJ9TEEpaCtOH/THM0AcjRkzFycnNY9q3W4g5sIILVt8AuUfRyCpIBfxeReRHVU0sbJvdGjImkOUPM8xMLXSsuQkea3ccYtIHK1m1PZWzE1py/mUfI79/iwRA348lAmMCXSHDDE3wyMrJ5Z9fbeSlBZuoU70y/7q0JyO6NHLqf8X19nd4gCUCY4zxqa37Mvj3N5sY3b0J957dibo1qvg7pBNYIjCmOIEwdNMdZkjSUmsVBInDWTn839rd/KlHU9o3iubLWwcSF1Pd32EVyRKBMUUJhKGb7jBDAN4YbZ3FQWDRb3u5c/Yqth88QpemtWjTIDqgkwDY8FFjihYIQzc9z+mDqpOm/KRmZHPHrF8YN20pVSIjeG9iH9o0iPZ3WF6xFkE5qVmzJunp6WV6jeXLl/Pmm2+eUKIi39atW1m8ePGxchUl7W/KyIsKkT6XtNRpCeQe9UnVSVM+cvOU8/+9mC37DnP9wNbcPLgtUZUj/R2W1ywRBJDExEQSEwsd5gs4ieCdd945lghK2t+UUSAM3Wzey7kd5O9+ClOo/YePUqeaUyTu9mHtaVqnGl2a1vZ3WKUWvreGfDjtW74VK1Zw2mmnkZCQwLnnnsuBAwcAp5x0QkIC3bt35/bbb6dLly7A8ZPQfPPNN3Tv3p3u3bvTo0cP0tLSmDx5MosWLaJ79+4888wzx+2fnp7O5ZdfTteuXUlISOCDDz7w2XWFlaq1oHZz/74BN+8F/W+zJBBAVJUPfkzmzCf/KBI3rHOjoEwCEIotgs8mw65Vxe+TdQh2r3aa/N50AjbqCiMeLXUol112GS+88AIDBgxgypQp3H///Tz77LNcfvnlvPrqq/Tp04fJkycXeuyTTz7Jiy++SL9+/UhPTycqKopHH32UJ598kk8//RRwEke+Bx98kNq1a7NqlXPt+UnHGFO+kg9kcNeHq1n4615OaVGXXi3r+TukMgvPFkFmqjvtGz7rBExNTeXgwYMMGDAAgPHjx7Nw4UIOHjxIWloaffo4FU7zb/MU1K9fP2699Vaef/55Dh48SKVKxefsL774ghtu+KOYa926FV/BMKh42yLMOgSpST5tOZrg8eHPyQx7ZiHLt+7n/tGdef+aPrRpUNPfYZVZ6LUIvPnkXrADLgC/uj958mTOPvts5s2bR79+/Zg/f76/Qwod3g4LtaGbpoB6NapySnw9Hj63C83qBvaQ0NIIzxZBfgecj6Z9A6hduzZ169Y9NpPYW2+9xYABA6hTpw7R0dHHKojOnDmz0OM3bdpE165dmTRpEqeeeirr168nOjqatLS0QvcfOnQoL7744rFluzVUDG+HhdrQzbCXnZvHvxZs5PkvfwNgQLv6vHH5qSGVBCAUWwTeKudp3zIyMo6bc/jWW2/ljTfe4NprryUjI4NWrVrx+uuvAzBt2jSuvvpqIiIiGDBgALVrn9jB9Oyzz/L1118TERFB586dGTFiBBEREURGRtKtWzcmTJhAjx49ju1/zz33cMMNN9ClSxciIyO57777OO+888rt+kKKt8NCbehmWFu9PZVJH6xkzY5DnNOtCaqKiDg1gkKMlaH2g/T0dGrWdO4rPvroo+zcubPQOQrKW7D9nHzq3/29GxYaCCUmTIXKzM7l+S9/4+WFm6lbvQoP/akzw7s09ndYZWZlqAPM3LlzeeSRR8jJyaFFixbMmDHD3yGFH28revpwwnATmH5PyeDVRZs5r0dT7jm7E7WrV/Z3SD5nicAPxo4dy9ixY/0dhjHGdTgrh/lrdnFez2a0bxTNV7cNpHm90OoHKE7IJIL8+3emcOV+CzDYb5lYRU/j+ubXvdw1exU7Uo+Q0Kw2bRpEh1USgBBJBFFRUaSkpBATE2PJoBCqSkpKClFRUeXzgoFQlbMsbFioAQ4cPsqDc9cy+6fttK5fw/1OQHAUiStvIZEImjVrRnJyMnv37vV3KAErKirquFFNZVLY8MtgSgSFDQu1RBBW8ovE/Z6SwY1ntuHGQW2CqkhceQuJRFC5cmVatmzp7zDCRyBU5SwLGxYatlLSs6hbvQqREcLk4R1oWrcanZsEZ32g8hQSicBUsECoylkWVtEz7Kgq7/+YzEOfrmXSiA5c2rsFZ3Vu5O+wAoYlAnNygn1CdRsWGjaS9mdw14erWPTbPnrF16NPqxh/hxRwLBGYwpU0KshG3ZggMPunZO75aDUCPPinLlzaK46ICBtQUpAlAnOikkYF2agbEyRia1alV8t6/OPcrjStU83f4QQsSwTmRCWNCrJRNyZAZefm8fI3m8jNg1uGtOWMdvU5o119f4cV8CwRmBOVNCrIRt2YALR6eyq3z1rJup2HGNO9iX3JtBQsEZgTlTQqyEbdmACSmZ3Ls1/8xquLNlOvRhVeHncKw2xEUKn4NBGIyHDgOSASeE1VHy2wPQ54A6jj7jNZVef5MibjpZJGBdmoGxMgtu3PYNq3m7mgZzPuGtkxLIrElTefJQIRiQReBIYCycAyEZmjqms9drsH+K+qviQinYB5QLyvYjLGhIa0zGw+X72LCxOb065hNF//fWDITRZTkXzZIugFbFTVzQAiMhMYA3gmAgXyeyFrAzt8GI/xRv6w0bSdkJdjw0NNwPl6/R7u/nAVuw5l0iOuDm0aRFsSKCNfJoKmQJLHcjLQu8A+U4H/ichNQA1gSGEvJCITgYkAcXFx5R6ocXkOG81nw0NNgNh/+CgPfrqWD3/eTtsGNZl1Xd+wLRJX3vw9Z/GfgRmq2gwYCbwlIifEpKqvqGqiqibWr29DwXzGc9hoPpur1wSA3DzlgpcW88kvO7h5cFs+vfl0esbV9XdYIcOXLYLtQHOP5WbuOk9XAsMBVHWJiEQBscAeH8ZliuI5bBSc5zY81PjR3rQsYmo4ReLuGtmRpnWr0bFxEFW6DRK+TATLgLYi0hInAVwMXFJgn23AYGCGiHQEogCrJe0vnsNGT78VjqTY8FDjF6rKf5cn8dDcdUwa3oG/nNaCIZ0a+juskOWzRKCqOSJyIzAfZ2jodFVdIyIPAMtVdQ5wG/CqiPwNp+N4gpb7VFqmVPKHjSZO8HckJkxtS8lg8uyVLN6UQu+W9Ti9Tay/Qwp5Pv0egfudgHkF1k3xeL4W6OfLGIwxwWPWj8nc+9FqIiOEf5zbhT+fakXiKoJ9szjUlHUuYasqavyoYa2q9G0dw0PndqFxbSsSV1EsEYSSss4lbFVFTQU7mpPHSws2kafK34a2o3/b+vRvayMDK5q/h4+a8lRY1dDSKKyqqDE+8kvSQc554Vue+eJXkvZnYN2D/mMtglBS1rmEraqoqQBHjuby9P9tYNq3W2gQHcVrlyXaiCA/s0QQSso6l7BVFTUVIOlABm8s/p2Le8UxeUQHakVZkTh/s0QQaso6l7BVFTU+cMgtEneRWyRuwe0DaWIzhgUMSwTGGJ/6av1u7pq9mj1pmfSMq0ubBjUtCQQY6ywOBElLYdFTzr9llXUIUpPK57WMKYOU9CxumfkzV8xYTu1qlZl9fT/aNKjp77BMIaxF4G9lHfLpyYZ/mgCRm6dc+O8lJB3I4G9D2nHdwNZUqWSfOwOVJQJ/K2mi+NKwSeWNn+1JyyS2RlUiI4S7z+5Is7rVad/ISkUHOq8TgYhUV9UMXwYTlso65NOTDf80fpKXp7y7bBuPzFvPpBEdGHdaCwZ3tCGhwaLERCAifYHXgJpAnIh0A65R1et9HVxYKOuQz4KvZcM/TQXbuu8wk2ev5PvN++nbOoYB9s3goONNi+AZYBgwB0BVfxGRM3waVbgp65BPTzb801Sg/y5P4t6PVlMlMoJHz+vK2FObI2JF4oKNV7eGVDWpwC831zfhGGOCSdM61TijXX0eHNOFRrWj/B2OOUneJIIk9/aQikhl4BZgnW/DClFFVQa1ip8mSGTl5PKvrzehqtx6Vnv6tYmln80XEPS8SQTXAs/hTEa/HfgfYP0DpVXUMFEb8mmCxM/bDjDpg5X8ujud83s2Q1XtNlCI8CYRtFfVSz1XiEg/4DvfhBSiihomakM+TYDLOJrDU//7lenfbaFRrSimT0hkUAcbERRKvEkELwA9vVhnilPUMFEb8mkC3PYDR3jr+9+5tHcck4Z3INqKxIWcIhOBiPQB+gL1ReRWj021cOYgNqVR1DBRG/JpAlDqkWw+W7WTi3vF0bZhNN/cPtBmDAthxbUIquB8d6AS4PnVwEPABb4MKmQVNUzUhnyaAPK/Nbu456PVpBw+SmJ8Pdo0qGlJIMQVmQhU9RvgGxGZoaq/V2BMga0scwLb6CATwPalZzF1zho+XbmTDo2ieW18ohWJCxPe9BFkiMgTQGfg2EBhVR3ks6gCVVkKxNnoIBPAcvOUC15azI6Dmfz9rHZcM6A1lSOtSFy48CYRvA28B4zCGUo6Htjry6ACVlkKxNnoIBOAdh/KpH5Np0jcfed0plndarRtaEXiwo03iSBGVaeJyC0et4uW+TqwgFSWAnE2OsgEkLw85e2l23jss/VMGt6ecX3iObNDA3+HZfzEm0TgfgRmp4icDewA6vkupABWlgJxNjrIBIjNe9OZPHsVS7fs5/Q2sQxsbwkg3HmTCB4SkdrAbTjfH6gF/NWXQQW0shSIs9FBxs/eW7aNKR+voWqlCB6/IIELT2lm3w42JScCVf3UfZoKnAnHvllsjAkyzepWZ2B7p0hcg1pWJM44ivtCWSRwEU6Noc9VdbWIjALuAqoBPSomRD8rOFzUhoCaIJKVk8sLX24E4O/DrEicKVxxLYJpQHNgKfC8iOwAEoHJqvpRBcTmfwWHi9ZtCfs3OdtsCKgJcD/+vp87Zq1k097DXJRoReJM0YpLBIlAgqrmiUgUsAtoraopFRNaACg4XDTD49JtCKgJUIezcnhi/gbeWLKVJrWr8cYVvRjQzmYNM0Ur7hsjR1U1D0BVM4HNpU0CIjJcRDaIyEYRmVzEPheJyFoRWSMi75Tm9X0uf7goOMNFh9zv/CuRNgTUBKwdB4/wztJtXHZaC+b/7QxLAqZExbUIOoiI+1VYBGjtLgugqppQ3Au7fQwvAkOBZGCZiMxR1bUe+7QF7gT6qeoBEQmscWyFDRdt2MmGgJqAk5qRzdxVO7mkt1MkbtEdZ9LQOoONl4pLBB3L+Nq9gI2quhlARGYCY4C1HvtcDbyoqgcAVHVPGc9Z/goOF7UhoCbAfL56F/d+vJr9h4/Su1U9WtevaUnAlEpxRefKWmiuKZDksZwM9C6wTzsAEfkOp7T1VFX9vOALichEYCJAXFxcGcMyJjTsSctk6pw1zFu1i06Na/H6hFNpXd+KxJnS82ryeh+fvy0wEGgGLBSRrqp60HMnVX0FeAUgMTFRKzhGYwJObp5y0b+XsCM1k9uHtWfiGa2sSJw5ab5MBNtxhp/ma+au85QM/KCq2cAWEfkVJzGEZy0jY0qwM/UIDaOjnCJxozvTvG51KxVtysyrjxAiUk1E2pfytZcBbUWkpYhUAS4G5hTY5yOc1gAiEotzq2hzKc9jTMjLy1NmfLeFwU99w39+cO7antm+gSUBUy5KTAQicg6wAvjcXe4uIgXf0E+gqjnAjcB8YB3wX1VdIyIPiMhod7f5QIqIrAW+Bm4Pq+8pGOOFjXvSuejlJUz9ZC2J8fUYZFVCTTnz5tbQVJwRQAsAVHWFiLT05sVVdR4wr8C6KR7PFbjVfRhjCpi5dBtT5qyhWuVInrqwG+f1bGrfDjblzqsy1KqaWuCPzzpsjakAcTHVGdKxAfeP7kL96Kr+DseEKG8SwRoRuQSIdL8AdjOw2LdhGROeMrNzef7L3wC4Y3gH+raOpW9rKxJnfMubzuKbcOYrzgLewSlH/VcfxhQYkpbCoqcgbSekJjnLxvjQ8q37Gfn8Iv61YBP7Dx/FuXNqjO950yLooKp3A3f7OpiA4Vl1NJ9VGzU+kp6VwxOfr+fN73+naZ1qvHlFL86w+kCmAnmTCJ4SkUbALOA9VV3t45j8z7PqaD6rNmp8ZFfqEWYuS2J8n3huH9aeGlX9/T1PE25KvDWkqmfizEy2F3hZRFaJyD0+j8yfPKuOgvPcqo2acnTg8FHe+t75PkCbBk6RuKmjO1sSMH7h1V+dqu7CmZzma+AOYArwkC8D8yvPqqOn3wpHUqzaqCkXqspnq3cx5ePVHMzIpm/rGFrXr2nTRhq/KjERiEhHYCxwPpACvIczkX1oy686mjjB35GYELHnUCb3frya+Wt207Vpbd68orcViTMBwZsWwXScN/9hqrrDx/EYE5Jy85QLX17CrtRM7hzRgStPb0klKxJnAkSJiUBV+1REIH5TcHL6fDZJvSkHOw4eoVEtp0jcA2O60LxuNVpZK8AEmCITgYj8V1UvEpFVHP9NYq9mKAsKBSenb9jFuR2UdQh2uZOz2bBRcxJy85Q3l2zl8c83cOfIDlzWJ96mjDQBq7gWwS3uv6MqIhC/KDg5fWaqkwgyU//Yx4aNmlLauCeNO2at5KdtBxnYvj6DOzb0d0jGFKu4Gcp2uk+vV9VJnttE5DFg0olHBZn8YaKa50xKnz8vcdJSpyWQe9SGjZpSeeeHbUyds4YaVSN5Zmw3/tTdisSZwOdNZ/FQTnzTH1HIuuBT2OT0+evHz7FJ6k2pxcdW56zODZk6ujOxNa1InAkOxfURXAdcD7QSkZUem6KB73wdWIUpODl9Ppuk3nghMzuXZ774FUGYPMKKxJngVFyL4B3gM+ARYLLH+jRV3e/TqIwJAj9sTmHy7FVs2XeYS3vHoap2G8gEpeISgarqVhG5oeAGEalnycCEq7TMbB77fD3/+X4bcfWq885VvenbxloBJniV1CIYBfyIM3zU86OOAq18GJcxAWv3oSxm/ZjMVae35Naz2lG9itUHMsGtuFFDo9x/vZqW0phQtv/wUeau3MG4PvG0aVCTRXcMshnDTMjwptZQP2CFqh4Wkb8APYFnVXWbz6Mzxs9UlU9X7mTqnDUcysymX5tYWtWvaUnAhBRvip28BGSISDecYnObgLd8GpUxAWD3oUyufvNHbnr3Z5rWrcYnN51u5SFMSPLm5maOqqqIjAH+qarTRORKXwdmjD/l5ikXuUXi7h7Zkcv7xVuROBOyvEkEaSJyJzAO6C8iEUBl34ZljH8kH8igce1qREYID47pQly96sTH1vB3WMb4lDcfccbiTFx/hTtBTTPgCZ9GZUwFy81TXlu0mSFPf8N/3JnDzmhX35KACQvelKHeJSJvA6eKyChgqaq+6fvQjKkYG3alcccHK/kl6SCDOzTgrM5WJM6EF29GDV2E0wJYgPNdghdE5HZVneXj2Izxuf98/zv3f7KG6KjKPHdxd0Z3a2LfDjZhx5s+gruBU1V1D4CI1Ae+ACwRmKCVXw6iTYOajOzamCmjOhFjReJMmPImEUTkJwFXCt71LRgTcI4czeXp/9tARIRw54iOnNYqhtNaxfg7LGP8yptE8LmIzAfedZfHAvN8F5IxvrFkUwqTZ6/k95QMxp3WworEGePyprP4dhE5DzjdXfWKqn7o27AqQP5cxWk7IS/H5iYOYYcys3lk3nreXbqNFjHVeefq3lYq2hgPxc1H0BZ4EmgNrAL+rqrbKyown/KcqzifzU0csvYcyuKjn7cz8YxW/G1IO6pVifR3SMYElOLu9U8HPgXOx6lA+kJpX1xEhovIBhHZKCKTi9nvfBFREUks7TlOiudcxfny5yY2ISElPYsZ320BoE2Dmnw76UzuGtnRkoAxhSju1lC0qr7qPt8gIj+V5oVFJBJ4EWeqy2RgmYjMUdW1BfaLBm4BfijN65eJ51zF4Dy3uYlDgqoy55cdTJ2zhvSsHM5oV59W9WvaiCBjilFcIogSkR78MQ9BNc9lVS0pMfQCNqrqZgARmQmMAdYW2O9B4DHg9lLGfvI85yo+/VY4kmJzE4eAHQePcM9Hq/lq/R66N6/D4xckWJE4Y7xQXCLYCTztsbzLY1mBQSW8dlMgyWM5GejtuYOI9ASaq+pcESkyEYjIRGAiQFxcXAmn9VL+XMWJE8rn9Yxf5eTmcfEr37M3LYt7R3ViQt94IiNsRJAx3ihuYpozfXlit3jd08CEkvZV1VeAVwASExPVl3GZ4JK0P4MmdapRKTKCh8/tSly96sTFVPd3WMYEFV9+MWw70NxjuZm7Ll800AVYICJbgdOAORXWYWyCWk5uHq8s3MSQp7/hrSVbATi9bawlAWNOgi8nW10GtBWRljgJ4GLgkvyNqpoKHBvMLSILcIaoLvdhTCYErNt5iEkfrGRlcipDOzVkRNfG/g7JmKDms0SgqjkiciMwH4gEpqvqGhF5AFiuqnN8dW4Tut5aspX7P1lL7WqV+eclPTi7a2P7drAxZeRN9VEBLgVaqeoDIhIHNFLVpSUdq6rzKFCOQlWnFLHvQK8iNmEpvxxEu4bRnNOtCfeO6kS9GlX8HZYxIcGbFsG/gDycUUIPAGnAB8CpPozLGAAyjubw5PxfqRQp3DWyI71bxdDbisQZU6686Szurao3AJkAqnoAsI9ixue+27iPYc8uZPp3Wziak4eqDRgzxhe8aRFku98SVjg2H0GeT6MyYS31SDYPz13He8uTaBlbg/9e04deLev5OyxjQpY3ieB54EOggYj8A7gAuMenUZmwti89i09W7uDaAa3565C2RFW2+kDG+JI3ZajfFpEfgcE45SX+pKrrfB6ZCSt707L45JcdXHF6S1rXr8m3kwZZZ7AxFcSbUUNxQAbwiec6Vd3my8BMeFBVPlqxnfs/WUtGVi5ndmhAy9galgSMqUDe3Bqai9M/IEAU0BLYAHT2YVwmDGw/eIS7P1zFgg176RnnFIlrGVvD32EZE3a8uTXU1XPZLRR3vc8iMmHBKRK3hJT0o0w9pxPj+liROGP8pdTfLFbVn0Skd8l7GnOibSkZNK3rFIl79LwE4upVp3k9qw9kjD9500dwq8diBNAT2OGziExIysnN49VFW3jmi1+5c0QHLu/Xkn5tbN5gYwKBNy2CaI/nOTh9Bh/4JhwTitbsSGXSBytZvf0Qwzo35GwrEmdMQCk2EbhfJItW1b9XUDwmxLyxeCsPfrqWOtWr8NKlPa1SqDEBqMhEICKV3Aqi/SoyIBMa8ovEdWgUzZjuTbl3VEfqVLchocYEouJaBEtx+gNWiMgc4H3gcP5GVZ3t49hMEDqclcMT8zdQOVK4++xOViTOmCDgTR9BFJCCU300//sEClgiMMdZ+Ote7py9ih2pRxjfJ/5Yq8AYE9iKSwQN3BFDq/kjAeSzMpDmmNSMbB6cu5ZZPybTqr5TJO7UeCsSZ0ywKC4RRAI1OT4B5LNEYI7ZdziLz1bt5PqBrbl5sBWJMybYFJcIdqrqAxUWiQkqe9IymbNiB1f1b3WsSFxdqw9kTFAqLhHYzV1zAlXlg5+28+CnazmSncvgjg1pGVvDkoAxQay4RDC4wqIwQSFpfwZ3fbiKRb/tI7FFXR4934rEGRMKikwEqrq/IgMxgS0nN48/v/o9Bw4f5cExnbm0dwsirEicMSGh1EXnTHjZuu8wzetVp1JkBI9f4BSJa1bXisQZE0q8mbzehKHs3Dxe/HojZz2zkDeXbAWgb+tYSwLGhCBrEZgTrN6eyh2zVrJ25yHO7tqYUQlN/B2SMcaHLBGY47z+3RYemruOejWq8O+/nMLwLo38HZIxxscsERjgjyJxnZvU5rweTbnn7E7Url7Z32EZYyqAJYIwl56Vw+Ofr6dKZAT3jOpEr5b16NXSykMYE06ssziMLdiwh2HPLOSt739HcVoFxpjwYy2CMHTg8FEenLuW2T9tp02Dmsy6ti+ntKjr77CMMX5iiSAMHcg4yv/W7ObmQW24YVAbqlayInHGhDOf3hoSkeEiskFENorI5EK23yoia0VkpYh8KSItfBZM0lJY9JTzL0DWIUhN+mM5xO05lMkrCzehqrSqX5PvJg3i1rPaWxIwxviuReDOd/wiMBRIBpaJyBxVXeux289AoqpmiMh1wOPA2HIPJmkpvD4S8rJBIqBuS9i/ydn2xmgYPwea9yr30wYCVeX95ck8OHctR3PyGNqpES1ja9iIIGPMMb5sEfQCNqrqZlU9CswExnjuoKpfq2qGu/g90MwnkWxd5CQBAM2DjJQ/tuUedbaHoKT9GYybtpQ7PlhJx8a1+OyW/lYkzhhzAl8mgqZAksdysruuKFcCnxW2QUQmishyEVm+d+/e0kcS399pCQBUqgZD7nf+lUiIrOJsDzH5ReJWJB3koT91YebVp9Gqfk1/h2WMCUAB0VksIn8BEoEBhW1X1VeAVwASExNLP8axeS9o2AUyU+H819zlTk5LIL5/SN0W2rLvMHFukbgnLuhGi5jqNKlTzd9hGWMCmC8TwXagucdyM3fdcURkCHA3MEBVs3wWTdVaziP/Tb95r5BKANm5efx7wSZe+Gojk0d04IrTW9KndYy/wzLGBAFfJoJlQFsRaYmTAC4GLvHcQUR6AC8Dw1V1jw9jCWkrkw9yx6yVrN+VxjndmjC6uxWJM8Z4z2eJQFVzRORGYD4QCUxX1TUi8gCwXFXnAE8ANYH3RQRgm6qO9lVMoWj6t1t4aO5a6kdX5dXLEhnaqaG/QzLGBBmf9hGo6jxgXoF1UzyeD/Hl+UNZfpG4hGa1GXtqcyaP6EjtajYk1BhTegHRWWy8l5aZzaOfradqpUimnNOJxPh6JMZbkThjzMmzonNB5Ov1ezjrmYW8u3QblSLFisQZY8qFtQiCwP7DR3ngkzV8tGIH7RrW5F+X9qVHnBWJM8aUD0sEQSD1SDZfrtvDLYPbcsOZbahSyRpyxpjyY4kgQO1KzeSjFdu55oxWtIytwbeTB1lnsDHGJywRBBhVZeayJB6eu47svDyGd25EfGwNSwLGGJ+xRBBAfk85zOQPVrFkcwqntarHo+clEG9F4owxPmaJIEDk5OZxyas/kHokm4fP7crFpzYnIkL8HZYxJgxYIvCzTXvTaeEWiXvqIqdIXOPaViTOGFNxbPiJnxzNyePZL35l+LMLeXPJ7wCc1irGkoAxpsJZi8APViQdZNKslWzYncaY7k34U4/ipmkwxhjfskRQwaZ9u4V/zF1Lg+gopo1PZHBHKxJnjPEvSwQVJL9IXPfmtbm4VxyTR3SgVpQNCTXG+J8lAh87lJnNI/PWE1U5gvvO6cwpLepxSgsrEmeMCRzWWexDX6zdzdCnv+G9ZduoUinCisQZYwKStQh8ICU9i/s/WcucX3bQoVE0r4xLpFvzOv4OyxhjCmWJwAfSMnP4esMe/jakHdcNbG1F4owxAc0SQTnZcfAIH/68nesHtiY+tgbfTR5kncHGmKBgiaCM8vKUd5Zu49HP1pObp5zdtTHxsTUsCRhjgoYlgjLYsu8wkz9YyQ9b9tOvTQyPnJtAXEx1f4dljDGlYongJOXk5vGX137gUGY2j5+fwIWJzRCxInHGmOBjiaCUNu5JIz6mBpUiI3hmbHdaxFSnYa0of4dljDEnzYazeCkrJ5en/+9Xhj+7iDfcInG9WtazJGCMCXrWIvDCT9sOMGnWSn7bk855PZpynhWJM8aEEEsEJXh14WYe/mwdjWtF8frlp3Jm+wb+DskYY8pV+CSCrEOQmQpJS6F5rxJ3z8tTIiKEni3qcGnvOCYN70C0DQk1xoSg8EgESUth92rQPHhjNIyfU2QySD2SzT/mrqVa5UjuH9PFisQZY0JeeHQWb13kJAGA3KPOciHmr9nF0Ke/4YOftlOjaiUrEmeMCQvh0SKI7w8S4SSDyCrOsod96Vnc9/Ea5q7aSafGtZg+4VS6NK3tp2CNMaZihUciaN4LGnZx+gjOf+2E20LpmTks+m0vtw9rz8QzWlE5MjwaSsYYA+GSCACq1nIebhLYfvAIH/6UzA1ntiE+tgaL7xxMzarh8+Mwxph8Pv3oKyLDRWSDiGwUkcmFbK8qIu+5238QkXhfxgPOaKC3lmzlrKe/4cWvN/F7SgaAJQFjTNjy2bufiEQCLwJDgWRgmYjMUdW1HrtdCRxQ1TYicjHwGDDWJwFlHSL78EGm/nM6b+9oRP+2sTx8blea17MiccaY8ObLFkEvYKOqblbVo8BMYEyBfcYAb7jPZwGDxReV25KWortXUyltG/fsn8y0wXm8eUUvSwLGGINvE0FTIMljOdldV+g+qpoDpAIxBV9IRCaKyHIRWb53797SR7J1EaKKAFGSy+CoX61SqDHGuIJieIyqvqKqiaqaWL9+/dK/QHx/qBQFEokUMnzUGGPCmS97SLcDzT2Wm7nrCtsnWUQqAbWBlHKPpHkv59vEWxc5ScCLEhPGGBMufJkIlgFtRaQlzhv+xcAlBfaZA4wHlgAXAF+pr77O27yXJQBjjCmEzxKBquaIyI3AfCASmK6qa0TkAWC5qs4BpgFvichGYD9OsjDGGFOBfDp4XlXnAfMKrJvi8TwTuNCXMRhjjCleUHQWG2OM8R1LBMYYE+YsERhjTJizRGCMMWFOgm3yFRHZC/x+kofHAvvKMZxgYNccHuyaw0NZrrmFqhb6jdygSwRlISLLVTXR33FUJLvm8GDXHB58dc12a8gYY8KcJQJjjAlz4ZYIXvF3AH5g1xwe7JrDg0+uOaz6CIwxxpwo3FoExhhjCrBEYIwxYS4kE4GIDBeRDSKyUUQmF7K9qoi8527/QUTi/RBmufLimm8VkbUislJEvhSRFv6IszyVdM0e+50vIioiQT/U0JtrFpGL3N/1GhF5p6JjLG9e/G3HicjXIvKz+/c90h9xlhcRmS4ie0RkdRHbRUSed38eK0WkZ5lPqqoh9cApeb0JaAVUAX4BOhXY53rg3+7zi4H3/B13BVzzmUB19/l14XDN7n7RwELgeyDR33FXwO+5LfAzUNddbuDvuCvgml8BrnOfdwK2+jvuMl7zGUBPYHUR20cCnwECnAb8UNZzhmKLoBewUVU3q+pRYCYwpsA+Y4A33OezgMES3JMYl3jNqvq1qma4i9/jzBgXzLz5PQM8CDwGZFZkcD7izTVfDbyoqgcAVHVPBcdY3ry5ZgVquc9rAzsqML5yp6oLceZnKcoY4E11fA/UEZHGZTlnKCaCpkCSx3Kyu67QfVQ1B0gFYiokOt/w5po9XYnziSKYlXjNbpO5uarOrcjAfMib33M7oJ2IfCci34vI8AqLzje8ueapwF9EJBln/pObKiY0vynt//cS+XRiGhN4ROQvQCIwwN+x+JKIRABPAxP8HEpFq4Rze2ggTqtvoYh0VdWD/gzKx/4MzFDVp0SkD86sh11UNc/fgQWLUGwRbAeaeyw3c9cVuo+IVMJpTqZUSHS+4c01IyJDgLuB0aqaVUGx+UpJ1xwNdAEWiMhWnHupc4K8w9ib33MyMEdVs1V1C/ArTmIIVt5c85XAfwFUdQkQhVOcLVR59f+9NEIxESwD2opISxGpgtMZPKfAPnOA8e7zC4Cv1O2FCVIlXrOI9ABexkkCwX7fGEq4ZlVNVdVYVY1X1XicfpHRqrrcP+GWC2/+tj/CaQ0gIrE4t4o2V2CM5c2ba94GDAYQkY44iWBvhUZZseYAl7mjh04DUlV1Z1leMORuDalqjojcCMzHGXEwXVXXiMgDwHJVnQNMw2k+bsTplLnYfxGXnZfX/ARQE3jf7Rffpqqj/RZ0GXl5zSHFy2ueD5wlImuBXOB2VQ3a1q6X13wb8KqI/A2n43hCMH+wE5F3cZJ5rNvvcR9QGUBV/43TDzIS2AhkAJeX+ZxB/PMyxhhTDkLx1pAxxphSsERgjDFhzhKBMcaEOUsExhgT5iwRGGNMmLNEYAKSiOSKyAqPR3wx+6aXw/lmiMgW91w/ud9QLe1rvCYindzndxXYtrisMbqvk/9zWS0in4hInRL27x7s1TiN79nwUROQRCRdVWuW977FvMYM4FNVnSUiZwFPqmpCGV6vzDGV9Loi8gbwq6r+o5j9J+BUXb2xvGMxocNaBCYoiEhNdx6Fn0RklYicUGlURBqLyEKPT8z93fVnicgS99j3RaSkN+iFQBv32Fvd11otIn9119UQkbki8ou7fqy7foGIJIrIo0A1N4633W3p7r8zReRsj5hniMgFIhIpIk+IyDK3xvw1XvxYluAWGxORXu41/iwii0WkvftN3AeAsW4sY93Yp4vIUnffwiq2mnDj79rb9rBHYQ+cb8WucB8f4nwLvpa7LRbnW5X5Ldp099/bgLvd55E49YZicd7Ya7jrJwFTCjnfDOAC9/mFwA/AKcAqoAbOt7LXAD2A84FXPY6t7f67AHfOg/yYPPbJj/Fc4A33eRWcKpLVgInAPe76qsByoGUhcaZ7XN/7wHB3uRZQyX0+BPjAfT4B+KfH8Q8Df3Gf18GpRVTD379ve/j3EXIlJkzIOKKq3fMXRKQy8LCInAHk4XwSbgjs8jhmGTDd3fcjVV0hIgNwJiv5zi2tUQXnk3RhnhCRe3Dq1FyJU7/mQ1U97MYwG+gPfA48JSKP4dxOWlSK6/oMeE5EqgLDgYWqesS9HZUgIhe4+9XGKRa3pcDx1URkhXv964D/89j/DRFpi1NmoXIR5z8LGC0if3eXo4A497VMmLJEYILFpUB94BRVzRanomiU5w6qutBNFGcDM0TkaeAA8H+q+mcvznG7qs7KXxCRwYXtpKq/ijPXwUjgIRH5UlUf8OYiVDVTRBYAw4CxOBOtgDPb1E2qOr+Elziiqt1FpDpO/Z0bgOdxJuD5WlXPdTvWFxRxvADnq+oGb+I14cH6CEywqA3scZPAmcAJcy6LMw/zblV9FXgNZ7q/74F+IpJ/z7+GiLTz8pyLgD+JSHURqYFzW2eRiDQBMlT1PzjF/AqbMzbbbZkU5j2cQmH5rQtw3tSvyz9GRNq55yyUOrPN3QzcJn+UUs8vRTzBY9c0nFtk+eYDN4nbPBKnKq0Jc5YITLB4G0gUkVXAZcD6QvYZCPwiIj/jfNp+TlX34rwxvisiK3FuC3Xw5oSq+hNO38FSnD6D11T1Z6ArsNS9RXMf8FAhh78CrMzvLC7gfzgTA32hzvSL4CSutcBP4kxa/jIltNjdWFbiTMzyOPCIe+2ex30NdMrvLMZpOVR2Y1vjLpswZ8NHjTEmzFmLwBhjwpwlAmOMCXOWCIwxJsxZIjDGmDBnicAYY8KcJQJjjAlzlgiMMSbM/T8XtnsX2ZZ+VgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "for ac in test_predictions[\"Area Code\"].unique():\n",
    "    grp = test_predictions[test_predictions[\"Area Code\"] == ac]\n",
    "\n",
    "    # calculate roc curves\n",
    "    ns_fpr, ns_tpr, _ = roc_curve(grp[\"Actual\"], [0 for _ in range(len(grp[\"Churn?\"]))])\n",
    "    lr_fpr, lr_tpr, _ = roc_curve(grp[\"Actual\"], grp[\"Churn?.True.\"])\n",
    "\n",
    "    # plot the roc curve for the model\n",
    "    pyplot.plot(ns_fpr, ns_tpr, linestyle='--', label='No Skill')\n",
    "    pyplot.plot(lr_fpr, lr_tpr, marker='.', label='Logistic')\n",
    "\n",
    "    pyplot.title('AUC of Area Code ' + str(ac) + ': ' + str(round(auc(lr_fpr, lr_tpr), 3)))\n",
    "    # axis labels\n",
    "    pyplot.xlabel('False Positive Rate')\n",
    "    pyplot.ylabel('True Positive Rate')\n",
    "    # show the legend\n",
    "    pyplot.legend()\n",
    "    # show the plot\n",
    "    pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variable Importance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead>\n",
       "<tr><th style=\"text-align: right;\">    gain</th><th>interaction             </th><th>description                                                                                         </th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr><td style=\"text-align: right;\">1       </td><td>17_CVTE:CustServ Calls.0</td><td>Out-of-fold mean of the response grouped by: [&#x27;CustServ Calls&#x27;] using 5 folds [internal:(10, 3, 10)]</td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.900773</td><td>1_CustServ Calls        </td><td>CustServ Calls (Original)                                                                           </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.873801</td><td>3_Day Charge            </td><td>Day Charge (Original)                                                                               </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.834801</td><td>14_VMail Message        </td><td>VMail Message (Original)                                                                            </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.777393</td><td>4_Day Mins              </td><td>Day Mins (Original)                                                                                 </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.501504</td><td>7_Eve Mins              </td><td>Eve Mins (Original)                                                                                 </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.455562</td><td>6_Eve Charge            </td><td>Eve Charge (Original)                                                                               </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.344162</td><td>8_Intl Calls            </td><td>Intl Calls (Original)                                                                               </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.32054 </td><td>13_Night Mins           </td><td>Night Mins (Original)                                                                               </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.309224</td><td>12_Night Charge         </td><td>Night Charge (Original)                                                                             </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.298618</td><td>18_CVTE:Day Calls.0     </td><td>Out-of-fold mean of the response grouped by: [&#x27;Day Calls&#x27;] using 5 folds [internal:(10, 3, 10)]     </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.295002</td><td>5_Eve Calls             </td><td>Eve Calls (Original)                                                                                </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.29082 </td><td>10_Intl Mins            </td><td>Intl Mins (Original)                                                                                </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.278274</td><td>11_Night Calls          </td><td>Night Calls (Original)                                                                              </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.276101</td><td>2_Day Calls             </td><td>Day Calls (Original)                                                                                </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.254629</td><td>0_Account Length        </td><td>Account Length (Original)                                                                           </td></tr>\n",
       "<tr><td style=\"text-align: right;\">0.237415</td><td>9_Intl Charge           </td><td>Intl Charge (Original)                                                                              </td></tr>\n",
       "</tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<driverlessai._utils.Table at 0x133a1bca0>"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "default_baseline.variable_importance()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Retrain a Model for Production\n",
    "* Retrain the final model with the full dataset\n",
    "* Print the model metrics\n",
    "* Create and Download the MOJO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "make_mojo_scoring_pipeline | default_value: auto\n",
      "mojo_for_predictions | default_value: auto\n",
      "mojo_for_predictions_max_rows | default_value: 10000\n",
      "mojo_for_predictions_batch_size | default_value: 100\n",
      "mojo_acceptance_test_rtol | default_value: 0.0\n",
      "mojo_acceptance_test_atol | default_value: 0.0\n",
      "reduce_mojo_size | default_value: False\n",
      "make_pipeline_visualization | default_value: auto\n",
      "make_python_pipeline_visualization | default_value: auto\n",
      "benchmark_mojo_latency | default_value: auto\n",
      "benchmark_mojo_latency_auto_size_limit | default_value: 500\n",
      "mojo_building_timeout | default_value: 1800.0\n",
      "mojo_building_parallelism | default_value: -1\n",
      "mojo_building_parallelism_base_model_size_limit | default_value: 100000000\n",
      "pytorch_nlp_pretrained_models | default_value: ['bert-base-uncased', 'distilbert-base-uncased', 'bert-base-multilingual-cased']\n",
      "included_pretransformers | default_value: []\n",
      "enable_rulefit | default_value: auto\n",
      "enable_rapids_cuml_models | default_value: False\n",
      "enable_rapids_models_dask | default_value: False\n",
      "enable_lightgbm_cat_support | default_value: False\n",
      "enable_lightgbm_linear_tree | default_value: False\n",
      "cross_validate_meta_learner | default_value: False\n",
      "enable_rapids_transformers | default_value: False\n",
      "enable_rapids_transformers_dask | default_value: False\n",
      "pred_labels | default_value: True\n",
      "enable_early_stopping_threshold | default_value: auto\n",
      "mli_enable_mojo_scorer | default_value: True\n",
      "make_mojo_scoring_pipeline_for_features_only | default_value: False\n",
      "mojo_replace_target_encoding_with_grouped_input_cols | default_value: False\n"
     ]
    }
   ],
   "source": [
    "dai.experiments.search_expert_settings(\"mojo\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Experiment launched at: https://steam.cloud.h2o.ai:443/proxy/driverless/3286/#/experiment?key=726e5012-fcc2-11ec-88b7-66b02643127c\n",
      "Complete 100.00% - Status: Complete                            \n"
     ]
    }
   ],
   "source": [
    "# Retrain on all data for productionalizing\n",
    "full_model = default_baseline.retrain(final_pipeline_only=True, \n",
    "                                      train_dataset=telco_churn,\n",
    "                                      test_dataset=\"\",\n",
    "                                      make_mojo_scoring_pipeline=\"on\")\n",
    "\n",
    "full_model = dai.experiments.get(full_model.key)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'scorer': 'F1',\n",
       " 'val_score': 0.7113276492082826,\n",
       " 'val_score_sd': 0.016703770755948092,\n",
       " 'val_roc_auc': 0.8727579819113,\n",
       " 'val_pr_auc': 0.7525046158060812,\n",
       " 'test_score': None,\n",
       " 'test_score_sd': None,\n",
       " 'test_roc_auc': None,\n",
       " 'test_pr_auc': None}"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "full_model.metrics()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloaded 'mojo.zip'\n"
     ]
    }
   ],
   "source": [
    "artifacts = full_model.artifacts.download(\"mojo_pipeline\", './', overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clean up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "!rm $local_file_path\n",
    "!rm test_predictions.csv\n",
    "!rm report.docx\n",
    "!rm test_preds.csv\n",
    "!rm mojo.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "dai_machine.terminate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "336px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}