gender_by_transaction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center> Predicting Client Gender </center>\n",
    "\n",
    "### It is necessary to identify the gender of the client based on their transactional historical data. The quality metric is [ROC AUC](https://dyakonov.org/2017/07/28/auc-roc-%D0%BF%D0%BB%D0%BE%D1%89%D0%B0%D0%B4%D1%8C-%D0%BF%D0%BE%D0%B4-%D0%BA%D1%80%D0%B8%D0%B2%D0%BE%D0%B9-%D0%BE%D1%88%D0%B8%D0%B1%D0%BE%D0%BA/), which needs to be maximized."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-23T13:11:26.066752Z",
     "iopub.status.busy": "2021-08-23T13:11:26.065958Z",
     "iopub.status.idle": "2021-08-23T13:11:26.073084Z",
     "shell.execute_reply": "2021-08-23T13:11:26.072336Z",
     "shell.execute_reply.started": "2021-08-23T13:11:26.066698Z"
    }
   },
   "source": [
    "## File Descriptions\n",
    "- transactions.csv - historical transactions of bank clients\n",
    "- gender.csv - gender information for some clients (null for test clients)\n",
    "- tr_mcc_codes.csv - mcc codes of transactions\n",
    "- tr_types.csv - types of transactions\n",
    "\n",
    "## Field Descriptions\n",
    "### transactions.csv\n",
    "- client_id - client identifier\n",
    "- tr_datetime - date and time of the transaction (days are numbered from the start of the data)\n",
    "- mcc_code - mcc code of the transaction\n",
    "- tr_type - type of transaction\n",
    "- amount - transaction amount in conditional units; with a \"+\" sign — funds credited to the client, \"-\" — funds debited\n",
    "- term_id - terminal identifier\n",
    "\n",
    "### gender.csv\n",
    "- client_id - client identifier\n",
    "- gender - client gender (empty values for test clients)\n",
    "\n",
    "### tr_mcc_codes.csv\n",
    "- mcc_code - mcc code of the transaction\n",
    "- mcc_description - description of the mcc code\n",
    "\n",
    "### tr_types.csv\n",
    "- tr_type - type of transaction\n",
    "- tr_description - description of the transaction type"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Tasks:\n",
    "- Develop a binary classification model to determine the client's gender. There are no restrictions on the model - it can be anything from KNN to transformers. The main goal is to achieve an ROC AUC above 77.5% on the hold-out test set.\n",
    "- Interpret the model results: the importance of the variables included in it, demonstrating on several examples why the corresponding prediction was made. This will help understand which gender corresponds to which target (0/1). Again, there is complete freedom of choice of approaches! Useful keywords: gain, permutation importance, SHAP.\n",
    "- Convert the results into a report without code (ideally - directly into [html](https://stackoverflow.com/questions/49907455/hide-code-when-exporting-jupyter-notebook-to-html))\n",
    "\n",
    "#### P.S. Don't forget about [PEP8](https://www.python.org/dev/peps/pep-0008/)!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "%load_ext autoreload\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from catboost import CatBoostClassifier\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.model_selection import cross_validate\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn.ensemble import HistGradientBoostingClassifier, \\\n",
    "                             GradientBoostingClassifier, \\\n",
    "                             RandomForestClassifier \\\n",
    "                            #  BaggingClassifier,\\\n",
    "                            #  StackingClassifier, \\\n",
    "                            #  VotingClassifier\n",
    "# from xgboost import XGBClassifier\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "# from sklearn.linear_model import LogisticRegressionCV\n",
    "# from lightgbm import LGBMClassifier\n",
    "# from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from operator import itemgetter\n",
    "import warnings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "warnings.filterwarnings(action = 'ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Объединение датасетов"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "tr_mcc_codes = pd.read_csv(\"data/mcc_codes.csv\", sep=\";\", index_col=\"mcc_code\")\n",
    "tr_types = pd.read_csv(\"data/trans_types.csv\", sep=\";\", index_col=\"trans_type\")\n",
    "\n",
    "transactions = pd.read_csv(\"data/transactions.csv\", index_col=\"client_id\")\n",
    "test_gender = pd.read_csv(\"data/test.csv\", index_col=\"client_id\")\n",
    "train_gender = pd.read_csv(\"data/train.csv\", index_col=\"client_id\")\n",
    "gender = pd.concat([train_gender, test_gender]).drop('Unnamed: 0', axis=1)\n",
    "\n",
    "del test_gender, train_gender"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mcc_description</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mcc_code</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>742</th>\n",
       "      <td>Ветеринарные услуги</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1711</th>\n",
       "      <td>Генеральные подрядчики по вентиляции, теплосна...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1731</th>\n",
       "      <td>Подрядчики по электричеству</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            mcc_description\n",
       "mcc_code                                                   \n",
       "742                                     Ветеринарные услуги\n",
       "1711      Генеральные подрядчики по вентиляции, теплосна...\n",
       "1731                            Подрядчики по электричеству"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tr_mcc_codes.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>trans_description</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>trans_type</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3200</th>\n",
       "      <td>Плата за предоставление услуг посредством моби...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3210</th>\n",
       "      <td>Плата за предоставление отчета по счету карты ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3800</th>\n",
       "      <td>Плата за обслуживание банковской карты (за пер...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            trans_description\n",
       "trans_type                                                   \n",
       "3200        Плата за предоставление услуг посредством моби...\n",
       "3210        Плата за предоставление отчета по счету карты ...\n",
       "3800        Плата за обслуживание банковской карты (за пер..."
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tr_types.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>trans_time</th>\n",
       "      <th>mcc_code</th>\n",
       "      <th>trans_type</th>\n",
       "      <th>amount</th>\n",
       "      <th>term_id</th>\n",
       "      <th>trans_city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>d1bbbc9a0e0410d3cf12a3d2f44f3450</th>\n",
       "      <td>35 08:24:41</td>\n",
       "      <td>4829</td>\n",
       "      <td>2370</td>\n",
       "      <td>-1808.56</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Tver</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81c7349f7bdcb661bc7d5f5f0fdac6aa</th>\n",
       "      <td>351 13:27:15</td>\n",
       "      <td>6011</td>\n",
       "      <td>7010</td>\n",
       "      <td>7954.51</td>\n",
       "      <td>442207</td>\n",
       "      <td>Kazan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fc0c7c937f139bd6d1c5b5d0a0023dd3</th>\n",
       "      <td>150 16:03:59</td>\n",
       "      <td>6011</td>\n",
       "      <td>2010</td>\n",
       "      <td>-12293.56</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Khabarovsk</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    trans_time  mcc_code  trans_type  \\\n",
       "client_id                                                              \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450   35 08:24:41      4829        2370   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa  351 13:27:15      6011        7010   \n",
       "fc0c7c937f139bd6d1c5b5d0a0023dd3  150 16:03:59      6011        2010   \n",
       "\n",
       "                                    amount term_id  trans_city  \n",
       "client_id                                                       \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450  -1808.56     NaN        Tver  \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa   7954.51  442207       Kazan  \n",
       "fc0c7c937f139bd6d1c5b5d0a0023dd3 -12293.56     NaN  Khabarovsk  "
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transactions.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gender</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>c5f18a1d1d83fda62e51e6ae1d7b8911</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4b6861803556de90249e13d7cf66f0a9</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7d3f908e1cd1e3624a1445df5e04c4fa</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  gender\n",
       "client_id                               \n",
       "c5f18a1d1d83fda62e51e6ae1d7b8911     1.0\n",
       "4b6861803556de90249e13d7cf66f0a9     0.0\n",
       "7d3f908e1cd1e3624a1445df5e04c4fa     1.0"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gender.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>trans_time</th>\n",
       "      <th>mcc_code</th>\n",
       "      <th>trans_type</th>\n",
       "      <th>amount</th>\n",
       "      <th>term_id</th>\n",
       "      <th>trans_city</th>\n",
       "      <th>gender</th>\n",
       "      <th>trans_description</th>\n",
       "      <th>mcc_description</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>d1bbbc9a0e0410d3cf12a3d2f44f3450</th>\n",
       "      <td>35 08:24:41</td>\n",
       "      <td>4829</td>\n",
       "      <td>2370</td>\n",
       "      <td>-1808.56</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Tver</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Списание с карты на карту по операции &lt;перевод...</td>\n",
       "      <td>Денежные переводы</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81c7349f7bdcb661bc7d5f5f0fdac6aa</th>\n",
       "      <td>351 13:27:15</td>\n",
       "      <td>6011</td>\n",
       "      <td>7010</td>\n",
       "      <td>7954.51</td>\n",
       "      <td>442207</td>\n",
       "      <td>Kazan</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Взнос наличных через АТМ (в своем тер.банке)</td>\n",
       "      <td>Финансовые институты — снятие наличности автом...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fc0c7c937f139bd6d1c5b5d0a0023dd3</th>\n",
       "      <td>150 16:03:59</td>\n",
       "      <td>6011</td>\n",
       "      <td>2010</td>\n",
       "      <td>-12293.56</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Khabarovsk</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Выдача наличных в АТМ Сбербанк России</td>\n",
       "      <td>Финансовые институты — снятие наличности автом...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3495487d5beaa5caaa0682fddc38379a</th>\n",
       "      <td>373 11:54:12</td>\n",
       "      <td>6010</td>\n",
       "      <td>7070</td>\n",
       "      <td>49.52</td>\n",
       "      <td>888987</td>\n",
       "      <td>Kaliningrad</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>07e279042b1da23710fa2f65034d30b9</th>\n",
       "      <td>340 13:11:45</td>\n",
       "      <td>6010</td>\n",
       "      <td>7070</td>\n",
       "      <td>45.29</td>\n",
       "      <td>888918</td>\n",
       "      <td>Saint Petersburg</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e91858185ae43cf3970220748893a01b</th>\n",
       "      <td>328 13:45:38</td>\n",
       "      <td>6010</td>\n",
       "      <td>7070</td>\n",
       "      <td>227.40</td>\n",
       "      <td>888907</td>\n",
       "      <td>Penza</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>429b6c8255f13bdb506fdbd6e342ccb1</th>\n",
       "      <td>10 19:22:35</td>\n",
       "      <td>4814</td>\n",
       "      <td>1030</td>\n",
       "      <td>-1084.44</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Khabarovsk</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Оплата услуги. Банкоматы СБ РФ</td>\n",
       "      <td>Звонки с использованием телефонов, считывающих...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9443b6a0d948cf368d8d72eb295bdfdf</th>\n",
       "      <td>167 23:59:15</td>\n",
       "      <td>5411</td>\n",
       "      <td>1110</td>\n",
       "      <td>-569.20</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Tver</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Покупка. POS ТУ Россия</td>\n",
       "      <td>Бакалейные магазины, супермаркеты</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1be75d38ad89c0f2b4d85b95f0fd8069</th>\n",
       "      <td>101 23:59:03</td>\n",
       "      <td>5814</td>\n",
       "      <td>1110</td>\n",
       "      <td>-71.43</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Vladivostok</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Покупка. POS ТУ Россия</td>\n",
       "      <td>Рестораны, закусочные</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8d69e4ecae4b509d528b9a0168f749d0</th>\n",
       "      <td>148 19:12:04</td>\n",
       "      <td>6010</td>\n",
       "      <td>7070</td>\n",
       "      <td>50.92</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Kazan</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3562083 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    trans_time  mcc_code  trans_type  \\\n",
       "client_id                                                              \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450   35 08:24:41      4829        2370   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa  351 13:27:15      6011        7010   \n",
       "fc0c7c937f139bd6d1c5b5d0a0023dd3  150 16:03:59      6011        2010   \n",
       "3495487d5beaa5caaa0682fddc38379a  373 11:54:12      6010        7070   \n",
       "07e279042b1da23710fa2f65034d30b9  340 13:11:45      6010        7070   \n",
       "...                                        ...       ...         ...   \n",
       "e91858185ae43cf3970220748893a01b  328 13:45:38      6010        7070   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1   10 19:22:35      4814        1030   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf  167 23:59:15      5411        1110   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069  101 23:59:03      5814        1110   \n",
       "8d69e4ecae4b509d528b9a0168f749d0  148 19:12:04      6010        7070   \n",
       "\n",
       "                                    amount term_id        trans_city  gender  \\\n",
       "client_id                                                                      \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450  -1808.56     NaN              Tver     0.0   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa   7954.51  442207             Kazan     0.0   \n",
       "fc0c7c937f139bd6d1c5b5d0a0023dd3 -12293.56     NaN        Khabarovsk     NaN   \n",
       "3495487d5beaa5caaa0682fddc38379a     49.52  888987       Kaliningrad     0.0   \n",
       "07e279042b1da23710fa2f65034d30b9     45.29  888918  Saint Petersburg     0.0   \n",
       "...                                    ...     ...               ...     ...   \n",
       "e91858185ae43cf3970220748893a01b    227.40  888907             Penza     0.0   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1  -1084.44     NaN        Khabarovsk     0.0   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf   -569.20     NaN              Tver     0.0   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069    -71.43     NaN       Vladivostok     1.0   \n",
       "8d69e4ecae4b509d528b9a0168f749d0     50.92     NaN             Kazan     0.0   \n",
       "\n",
       "                                                                  trans_description  \\\n",
       "client_id                                                                             \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450  Списание с карты на карту по операции <перевод...   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa       Взнос наличных через АТМ (в своем тер.банке)   \n",
       "fc0c7c937f139bd6d1c5b5d0a0023dd3              Выдача наличных в АТМ Сбербанк России   \n",
       "3495487d5beaa5caaa0682fddc38379a  Перевод на карту (с карты) через Мобильный бан...   \n",
       "07e279042b1da23710fa2f65034d30b9  Перевод на карту (с карты) через Мобильный бан...   \n",
       "...                                                                             ...   \n",
       "e91858185ae43cf3970220748893a01b  Перевод на карту (с карты) через Мобильный бан...   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1                     Оплата услуги. Банкоматы СБ РФ   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf                             Покупка. POS ТУ Россия   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069                             Покупка. POS ТУ Россия   \n",
       "8d69e4ecae4b509d528b9a0168f749d0  Перевод на карту (с карты) через Мобильный бан...   \n",
       "\n",
       "                                                                    mcc_description  \n",
       "client_id                                                                            \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450                                  Денежные переводы  \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa  Финансовые институты — снятие наличности автом...  \n",
       "fc0c7c937f139bd6d1c5b5d0a0023dd3  Финансовые институты — снятие наличности автом...  \n",
       "3495487d5beaa5caaa0682fddc38379a   Финансовые институты — снятие наличности вручную  \n",
       "07e279042b1da23710fa2f65034d30b9   Финансовые институты — снятие наличности вручную  \n",
       "...                                                                             ...  \n",
       "e91858185ae43cf3970220748893a01b   Финансовые институты — снятие наличности вручную  \n",
       "429b6c8255f13bdb506fdbd6e342ccb1  Звонки с использованием телефонов, считывающих...  \n",
       "9443b6a0d948cf368d8d72eb295bdfdf                  Бакалейные магазины, супермаркеты  \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069                              Рестораны, закусочные  \n",
       "8d69e4ecae4b509d528b9a0168f749d0   Финансовые институты — снятие наличности вручную  \n",
       "\n",
       "[3562083 rows x 9 columns]"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = transactions\n",
    "df = df.join(gender)\n",
    "df = pd.merge(df, tr_types, left_on=\"trans_type\", right_index=True)\n",
    "df = pd.merge(df, tr_mcc_codes, left_on=\"mcc_code\", right_index=True)\n",
    "del transactions, tr_types, tr_mcc_codes, gender\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.loc[df['gender'].notna()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>trans_time</th>\n",
       "      <th>mcc_code</th>\n",
       "      <th>trans_type</th>\n",
       "      <th>amount</th>\n",
       "      <th>term_id</th>\n",
       "      <th>trans_city</th>\n",
       "      <th>gender</th>\n",
       "      <th>trans_description</th>\n",
       "      <th>mcc_description</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>d1bbbc9a0e0410d3cf12a3d2f44f3450</th>\n",
       "      <td>35 08:24:41</td>\n",
       "      <td>4829</td>\n",
       "      <td>2370</td>\n",
       "      <td>-1808.56</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Tver</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Списание с карты на карту по операции &lt;перевод...</td>\n",
       "      <td>Денежные переводы</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81c7349f7bdcb661bc7d5f5f0fdac6aa</th>\n",
       "      <td>351 13:27:15</td>\n",
       "      <td>6011</td>\n",
       "      <td>7010</td>\n",
       "      <td>7954.51</td>\n",
       "      <td>442207</td>\n",
       "      <td>Kazan</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Взнос наличных через АТМ (в своем тер.банке)</td>\n",
       "      <td>Финансовые институты — снятие наличности автом...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3495487d5beaa5caaa0682fddc38379a</th>\n",
       "      <td>373 11:54:12</td>\n",
       "      <td>6010</td>\n",
       "      <td>7070</td>\n",
       "      <td>49.52</td>\n",
       "      <td>888987</td>\n",
       "      <td>Kaliningrad</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    trans_time  mcc_code  trans_type   amount  \\\n",
       "client_id                                                                       \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450   35 08:24:41      4829        2370 -1808.56   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa  351 13:27:15      6011        7010  7954.51   \n",
       "3495487d5beaa5caaa0682fddc38379a  373 11:54:12      6010        7070    49.52   \n",
       "\n",
       "                                 term_id   trans_city  gender  \\\n",
       "client_id                                                       \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450     NaN         Tver     0.0   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa  442207        Kazan     0.0   \n",
       "3495487d5beaa5caaa0682fddc38379a  888987  Kaliningrad     0.0   \n",
       "\n",
       "                                                                  trans_description  \\\n",
       "client_id                                                                             \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450  Списание с карты на карту по операции <перевод...   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa       Взнос наличных через АТМ (в своем тер.банке)   \n",
       "3495487d5beaa5caaa0682fddc38379a  Перевод на карту (с карты) через Мобильный бан...   \n",
       "\n",
       "                                                                    mcc_description  \n",
       "client_id                                                                            \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450                                  Денежные переводы  \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa  Финансовые институты — снятие наличности автом...  \n",
       "3495487d5beaa5caaa0682fddc38379a   Финансовые институты — снятие наличности вручную  "
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Index: 3237373 entries, d1bbbc9a0e0410d3cf12a3d2f44f3450 to 8d69e4ecae4b509d528b9a0168f749d0\n",
      "Data columns (total 9 columns):\n",
      " #   Column             Dtype  \n",
      "---  ------             -----  \n",
      " 0   trans_time         object \n",
      " 1   mcc_code           int64  \n",
      " 2   trans_type         int64  \n",
      " 3   amount             float64\n",
      " 4   term_id            object \n",
      " 5   trans_city         object \n",
      " 6   gender             float64\n",
      " 7   trans_description  object \n",
      " 8   mcc_description    object \n",
      "dtypes: float64(2), int64(2), object(5)\n",
      "memory usage: 247.0+ MB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "term_id 1310923 (40.49%)\n"
     ]
    }
   ],
   "source": [
    "def missing_features(data, column_set):\n",
    "    incomplete_features = {feature: data.shape[0]-sum(data[feature].value_counts())\n",
    "                                   for feature in column_set\n",
    "                                   if not sum(data[feature].value_counts()) == data.shape[0]}\n",
    "    incomplete_features_sorted = sorted(incomplete_features, key=lambda feature: incomplete_features[feature], reverse=True)\n",
    "    incompleteness = [round((incomplete_features[feature]/data.shape[0])*100, 2) for feature in incomplete_features_sorted]\n",
    "    \n",
    "    for feature, percentage in zip(incomplete_features_sorted, incompleteness):\n",
    "        print(f'{feature} {incomplete_features[feature]} ({percentage}%)')\n",
    "missing_features(df, df.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='amount', ylabel='Density'>"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHACAYAAABeV0mSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAAArlElEQVR4nO3de3RU5b3G8WcScoGSBLnlAgEiCIJAEu4XUWhTA3LQSIse9RwuCsueAy12pEqsC4q1jawShFoKUgspKkJRhHo3Jwq0GLlHwJb7JRQSCAIJiTgJyZw/QkZCMslMmGTv7Hw/a81azjvv7P0bRpKH9333u21Op9MpAAAAi/AzugAAAABfItwAAABLIdwAAABLIdwAAABLIdwAAABLIdwAAABLIdwAAABLIdwAAABLIdwAAABLIdwAAABLadLhZsuWLRo3bpyioqJks9m0YcOGej/n6dOn9V//9V9q06aNmjdvrj59+mjnzp31fl4AAJqKJh1uioqKFBsbqyVLljTI+S5evKjhw4crICBAH374of75z38qNTVVt9xyS4OcHwCApsDGjTPL2Ww2vfPOO0pKSnK1ORwO/fKXv9Sbb76pS5cuqXfv3po/f75GjhxZp3PMnj1bW7du1d///nffFA0AAKpo0iM3tZkxY4YyMzO1Zs0a7d27VxMmTNDo0aN1+PDhOh3vb3/7mwYMGKAJEyaoffv2io+P15/+9CcfVw0AQNPGyM01N47cZGdn69Zbb1V2draioqJc/RISEjRo0CD99re/9focwcHBkiS73a4JEyZox44dmjlzppYtW6ZJkyb55HMAANDUNTO6ALPat2+fSktL1b1790rtDodDbdq0kSQdOHBAPXv2rPE4zzzzjF588UVJUllZmQYMGOAKRvHx8dq/fz/hBgAAHyLcuFFYWCh/f3/t2rVL/v7+lV5r2bKlJOnWW2/Vv/71rxqPUxGEJCkyMlK9evWq9HrPnj319ttv+6hqAABAuHEjPj5epaWlOnfunEaMGFFtn8DAQN1+++0eH3P48OE6ePBgpbZDhw6pc+fON1UrAAD4TpMON4WFhTpy5Ijr+fHjx5WVlaXWrVure/fuevTRRzVx4kSlpqYqPj5eeXl5ysjIUN++fTV27Fivz/fzn/9cw4YN029/+1s9+OCD2r59u5YvX67ly5f78mMBANCkNekFxZs2bdKoUaOqtE+aNElpaWkqKSnRCy+8oFWrVun06dNq27athgwZonnz5qlPnz51Oud7772n5ORkHT58WDExMbLb7Zo2bdrNfhQAAHBNkw43AADAetjnBgAAWArhBgAAWEqTW1BcVlamM2fOKCQkRDabzehyAACAB5xOpy5fvqyoqCj5+dU8NtPkws2ZM2cUHR1tdBkAAKAOTp06pY4dO9bYp8mFm5CQEEnlfzihoaEGVwMAADxRUFCg6Oho1+/xmjS5cFMxFRUaGkq4AQCgkfFkSQkLigEAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKUQbgAAgKU0M7oAAE3T6m3Z9XLcRwZ3qpfjAmg8GLkBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWQrgBAACWYmi4SUlJ0cCBAxUSEqL27dsrKSlJBw8erPV969at0+23367g4GD16dNHH3zwQQNUCwAAGgNDw83mzZs1ffp0ffHFF0pPT1dJSYnuueceFRUVuX3P559/rocffliPP/649uzZo6SkJCUlJWn//v0NWDkAADArm9PpdBpdRIW8vDy1b99emzdv1l133VVtn4ceekhFRUV67733XG1DhgxRXFycli1bVus5CgoKFBYWpvz8fIWGhvqsdgDeWb0tu16O+8jgTvVyXADG8ub3t6nW3OTn50uSWrdu7bZPZmamEhISKrUlJiYqMzOz2v4Oh0MFBQWVHgAAwLpME27Kysr05JNPavjw4erdu7fbfrm5uQoPD6/UFh4ertzc3Gr7p6SkKCwszPWIjo72ad0AAMBcTBNupk+frv3792vNmjU+PW5ycrLy8/Ndj1OnTvn0+AAAwFyaGV2AJM2YMUPvvfeetmzZoo4dO9bYNyIiQmfPnq3UdvbsWUVERFTbPygoSEFBQT6rFQAAmJuhIzdOp1MzZszQO++8o08//VQxMTG1vmfo0KHKyMio1Jaenq6hQ4fWV5kAAKARMXTkZvr06Vq9erU2btyokJAQ17qZsLAwNW/eXJI0ceJEdejQQSkpKZKkmTNn6u6771ZqaqrGjh2rNWvWaOfOnVq+fLlhnwMAAJiHoSM3S5cuVX5+vkaOHKnIyEjXY+3ata4+2dnZysnJcT0fNmyYVq9ereXLlys2NlZvvfWWNmzYUOMiZAAA0HSYap+bhsA+N4A5sM8NAG802n1uAAAAbhbhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWArhBgAAWIqh4WbLli0aN26coqKiZLPZtGHDhhr7b9q0STabrcojNze3YQoGAACmZ2i4KSoqUmxsrJYsWeLV+w4ePKicnBzXo3379vVUIQAAaGyaGXnyMWPGaMyYMV6/r3379mrVqpXvCwIAAI1eo1xzExcXp8jISP3whz/U1q1bjS4HAACYiKEjN96KjIzUsmXLNGDAADkcDr366qsaOXKktm3bpn79+lX7HofDIYfD4XpeUFDQUOUCAAADNKpw06NHD/Xo0cP1fNiwYTp69Kheeuklvfbaa9W+JyUlRfPmzWuoEgEAgMEa5bTU9QYNGqQjR464fT05OVn5+fmux6lTpxqwOgAA0NAa1chNdbKyshQZGen29aCgIAUFBTVgRQAAwEiGhpvCwsJKoy7Hjx9XVlaWWrdurU6dOik5OVmnT5/WqlWrJEmLFi1STEyM7rjjDn377bd69dVX9emnn+qTTz4x6iMAAACTMTTc7Ny5U6NGjXI9t9vtkqRJkyYpLS1NOTk5ys7Odr1eXFysp556SqdPn1aLFi3Ut29f/d///V+lYwAAgKbN5nQ6nUYX0ZAKCgoUFham/Px8hYaGGl0O0GSt3pZde6c6eGRwp3o5LgBjefP7u9EvKAYAALge4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFgK4QYAAFhKncLNsWPHfF0HAACAT9Qp3HTr1k2jRo3S66+/rm+//dbXNQEAANRZncLN7t271bdvX9ntdkVEROiJJ57Q9u3bfV0bAACA1+oUbuLi4rR48WKdOXNGK1asUE5Oju6880717t1bCxcuVF5enq/rBAAA8MhNLShu1qyZxo8fr3Xr1mn+/Pk6cuSIZs2apejoaE2cOFE5OTm+qhMAAMAjNxVudu7cqf/93/9VZGSkFi5cqFmzZuno0aNKT0/XmTNndP/99/uqTgAAAI80q8ubFi5cqJUrV+rgwYO69957tWrVKt17773y8yvPSjExMUpLS1OXLl18WSsAAECt6hRuli5dqscee0yTJ09WZGRktX3at2+vP//5zzdVHAAAgLfqFG7S09PVqVMn10hNBafTqVOnTqlTp04KDAzUpEmTfFIkAACAp+q05qZr1646f/58lfYLFy4oJibmposCAACoqzqFG6fTWW17YWGhgoODb6ogAACAm+HVtJTdbpck2Ww2zZkzRy1atHC9Vlpaqm3btikuLs6nBQIAAHjDq3CzZ88eSeUjN/v27VNgYKDrtcDAQMXGxmrWrFm+rRAAAMALXoWbzz77TJI0ZcoULV68WKGhofVSFAAAQF3V6WqplStX+roOAAAAn/A43IwfP15paWkKDQ3V+PHja+y7fv36my4MAACgLjwON2FhYbLZbK7/BgAAMCOPw831U1FMSwEAALOq0z43V65c0TfffON6fvLkSS1atEiffPKJzwoDAACoizqFm/vvv1+rVq2SJF26dEmDBg1Samqq7r//fi1dutSnBQIAAHijTuFm9+7dGjFihCTprbfeUkREhE6ePKlVq1bp97//vU8LBAAA8Eadws0333yjkJAQSdInn3yi8ePHy8/PT0OGDNHJkyd9WiAAAIA36hRuunXrpg0bNujUqVP6+OOPdc8990iSzp07x8Z+AADAUHUKN3PmzNGsWbPUpUsXDR48WEOHDpVUPooTHx/v0wIBAAC8Uacdin/84x/rzjvvVE5OjmJjY13tP/jBD/TAAw/4rDgAAABv1SncSFJERIQiIiIqtQ0aNOimCwIAALgZdQo3RUVFevHFF5WRkaFz586prKys0uvHjh3zSXEAAADeqlO4mTp1qjZv3qz//u//VmRkpOu2DAAAAEarU7j58MMP9f7772v48OG+rgcAAOCm1OlqqVtuuUWtW7f2dS0AAAA3rU7h5te//rXmzJlT6f5SAAAAZlCnaanU1FQdPXpU4eHh6tKliwICAiq9vnv3bp8UBwAA4K06hZukpCQflwEAAOAbdQo3c+fO9XUdAAAAPlGnNTeSdOnSJb366qtKTk7WhQsXJJVPR50+fdpnxQEAAHirTiM3e/fuVUJCgsLCwnTixAlNmzZNrVu31vr165Wdna1Vq1b5uk4AAACP1Gnkxm63a/LkyTp8+LCCg4Nd7ffee6+2bNnis+IAAAC8Vadws2PHDj3xxBNV2jt06KDc3NybLgoAAKCu6hRugoKCVFBQUKX90KFDateu3U0XBQAAUFd1Cjf33Xefnn/+eZWUlEiSbDabsrOz9cwzz+hHP/qRTwsEAADwRp3CTWpqqgoLC9WuXTtduXJFd999t7p166aQkBD95je/8XWNAAAAHqvT1VJhYWFKT0/X1q1b9eWXX6qwsFD9+vVTQkKCr+sDAADwitfhpqysTGlpaVq/fr1OnDghm82mmJgYRUREyOl0ymaz1UedAAAAHvFqWsrpdOq+++7T1KlTdfr0afXp00d33HGHTp48qcmTJ+uBBx6orzoBAAA84tXITVpamrZs2aKMjAyNGjWq0muffvqpkpKStGrVKk2cONGnRQIAAHjKq5GbN998U88++2yVYCNJ3//+9zV79my98cYbHh9vy5YtGjdunKKiomSz2bRhw4Za37Np0yb169dPQUFB6tatm9LS0rz4BAAAwOq8Cjd79+7V6NGj3b4+ZswYffnllx4fr6ioSLGxsVqyZIlH/Y8fP66xY8dq1KhRysrK0pNPPqmpU6fq448/9vicAADA2ryalrpw4YLCw8Pdvh4eHq6LFy96fLwxY8ZozJgxHvdftmyZYmJilJqaKknq2bOn/vGPf+ill15SYmKix8cBAADW5dXITWlpqZo1c5+H/P39dfXq1Zsuyp3MzMwql5snJiYqMzOz3s4JAAAaF69GbpxOpyZPnqygoKBqX3c4HD4pyp3c3NwqI0fh4eEqKCjQlStX1Lx582prur6u6m4bAQAArMOrcDNp0qRa+5jtSqmUlBTNmzfP6DIAAEAD8SrcrFy5sr7q8EhERITOnj1bqe3s2bMKDQ2tdtRGkpKTk2W3213PCwoKFB0dXa91AgAA49Tp9gtGGTp0qD744INKbenp6Ro6dKjb9wQFBbmdRgMAANZTpxtn+kphYaGysrKUlZUlqfxS76ysLGVnZ0sqH3W5fprrJz/5iY4dO6ann35aBw4c0B//+Ef99a9/1c9//nMjygcAACZkaLjZuXOn4uPjFR8fL0my2+2Kj4/XnDlzJEk5OTmuoCNJMTExev/995Wenq7Y2Filpqbq1Vdf5TJwAADgYnM6nU6ji2hIBQUFCgsLU35+vkJDQ40uB2iyVm/Lrr1THTwyuFO9HBeAsbz5/W3oyA0AAICvEW4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4AAIClEG4ANBpfFzr0+dHzKiktM7oUACbWzOgCAMBT6f86q73/zlfLoGbq27GV0eUAMClGbgA0GoXfXpUkXSkpNbgSAGZGuAHQaDiulk9HlZY5Da4EgJkRbgA0GsWEGwAeINwAaDSKry0kLiPcAKgB4QZAo+G4Wr7WptRJuAHgHuEGQKPgdDqZlgLgEcINgEbhaplTFZmGbW4A1IRwA6BRqLhSSpLKmJYCUAPCDYBGofi6cMO0FICaEG4ANAqEGwCeItwAaBSKr363KzFXSwGoCeEGQKNQac0NIzcAakC4AdAoXB9uGLkBUBPCDYBGobiUNTcAPEO4AdAoOFhQDMBDhBsAjUIx+9wA8BDhBkCjcP3VUlcZuQFQA8INgEaBq6UAeIpwA6BRYBM/AJ4i3ABoFCrfW8rAQgCYHuEGQKPAyA0ATxFuADQKXAoOwFOEGwCNQnEp95YC4BnCDYBGoZirpQB4iHADoFFgWgqApwg3ABqFYm6cCcBDhBsApud0OrlaCoDHCDcATK+k1Knr4wz3lgJQE8INANNzXHdfKYmRGwA1I9wAML3rp6Sk8h2KnYzeAHDDFOFmyZIl6tKli4KDgzV48GBt377dbd+0tDTZbLZKj+Dg4AasFkBDq7hSKsDf5mpjUTEAdwwPN2vXrpXdbtfcuXO1e/duxcbGKjExUefOnXP7ntDQUOXk5LgeJ0+ebMCKATS0ipGb5gH+rjampgC4Y3i4WbhwoaZNm6YpU6aoV69eWrZsmVq0aKEVK1a4fY/NZlNERITrER4e3oAVA2hoxaXXwk3gd+GmrMxdbwBNnaHhpri4WLt27VJCQoKrzc/PTwkJCcrMzHT7vsLCQnXu3FnR0dG6//779dVXX7nt63A4VFBQUOkBoHFxVDdyw7QUADcMDTfnz59XaWlplZGX8PBw5ebmVvueHj16aMWKFdq4caNef/11lZWVadiwYfr3v/9dbf+UlBSFhYW5HtHR0T7/HADqV/G1q6WCmvnL79qyG6alALhj+LSUt4YOHaqJEycqLi5Od999t9avX6927drplVdeqbZ/cnKy8vPzXY9Tp041cMUAblbFyE1gMz/5X0s3hBsA7jQz8uRt27aVv7+/zp49W6n97NmzioiI8OgYAQEBio+P15EjR6p9PSgoSEFBQTddKwDjFN8QbkpKndw8E4Bbho7cBAYGqn///srIyHC1lZWVKSMjQ0OHDvXoGKWlpdq3b58iIyPrq0wABrt+5MbPdm3khjU3ANwwdORGkux2uyZNmqQBAwZo0KBBWrRokYqKijRlyhRJ0sSJE9WhQwelpKRIkp5//nkNGTJE3bp106VLl/S73/1OJ0+e1NSpU438GADqUcXITRDTUgA8YHi4eeihh5SXl6c5c+YoNzdXcXFx+uijj1yLjLOzs+Xn990A08WLFzVt2jTl5ubqlltuUf/+/fX555+rV69eRn0EAPXMFW78/eRvI9wAqJnN2cT2MC8oKFBYWJjy8/MVGhpqdDlAk7V6W7YXfU9q/5kCjesbqc+Pfq2vi4r1xF23qnOb71Xp+8jgTr4sE4BJePP7u9FdLQWg6anYxC+omb/8mJYCUAvCDQDTq3QpOAuKAdSCcAPA9G68FFxi5AaAe4QbAKbnqOZqKfa5AeAO4QaA6RVXu8+NkRUBMDPCDQDTc4Ubfz/5X/upVcptwQG4QbgBYGpOp1MlpdWtuTGyKgBmRrgBYGpXy5yqmIEKuG4TP9bcAHCHcAPA1EquG6IJ8Pf7bp8bLgUH4AbhBoCplVxbOexnk/z9bFwKDqBWhBsAplYxchNwbSUx95YCUBvCDQBTqxJuKva5YVoKgBuEGwCmVjEtFeBfHmq4txSA2hBuAJga01IAvEW4AWBq7qaluFoKgDuEGwCmduO0FPeWAlAbwg0AU7tx5Oa7e0sRbgBUj3ADwNSqTkuVt7PmBoA7hBsAplYxLdXMNS1V/mOLe0sBcIdwA8DUrpZ+d0dwSbqWcdjnBoBbhBsAplZ8Ldw0q1hzc21B8VWmpQC4QbgBYGpXr01LBXK1FAAPEW4AmNqNIzds4gegNoQbAKZWZc0N95YCUAvCDQBTK3ZzbynW3ABwh3ADwNSuurm3FGtuALhDuAFgasXcWwqAlwg3AEztKveWAuAlwg0AU3N3bynW3ABwh3ADwNRKbrwUnKulANSCcAPA1Epcm/jdsOaGkRsAbhBuAJjadyM319bcVFwtRbYB4AbhBoCpldywid+1m4Kz5gaAW4QbAKbldDpd01JVRm4INwDcINwAMK3rR2eqrLlhQTEANwg3AEyrYkpKqnq1FAuKAbhDuAFgWhVTUv42myvU+LGJH4BaEG4AmFbJ1cpXSknfrblxir1uAFSPcAPAtErKKl8pJX03LSUxNQWgeoQbAKZV7cgN4QZALQg3AEyrpKzippnf/aiquLeUxLobANUj3AAwrYqRm8rhRqqIN1wODqA6hBsAplXdyI3NZnNdMcW0FIDqEG4AmNZ3Ize2Su0VV0wRbgBUh3ADwLQqrpa6fuRG+u7+UkxLAagO4QaAabkdubmWbsrKqrwFAAg3AMyrujU3klSRdZiWAlAdwg0A03KN3DS7Idxw80wANSDcADCtihtnBvjdsOaGBcUAakC4AWBaFTfODGh245qbazfPZOQGQDUINwBMy93IjT/73ACoAeEGgGm5wo27NTeEGwDVINwAMC3XtJRf5Wkp1twAqAnhBoBp1TZyw5obANUh3AAwLdbcAKgLwg0A03J7tRTTUgBqQLgBYFpu97lhEz8ANSDcADCtWtfcMHIDoBqEGwCm5e5qKe4tBaAmhBsAplXrPjdkGwDVINwAMKUyp1NX3dwVnH1uANSEcAPAlK5eNywT4F/9vaUINwCqQ7gBYEpXr01JSVVHbtjED0BNCDcATKn4Wrjx97O5pqEqsM8NgJoQbgCYkutKqRumpCT2uQFQM8INAFNyXSnlX/XHFGtuANSEcAPAlAq+LZEkBdYQbtjED0B1CDcATGn78QuSpG7tW1Z5jTU3AGpiinCzZMkSdenSRcHBwRo8eLC2b99eY/9169bp9ttvV3BwsPr06aMPPviggSoF0BDOFzp0IPeyJGl417ZVXq8YuckrdBBwAFRheLhZu3at7Ha75s6dq927dys2NlaJiYk6d+5ctf0///xzPfzww3r88ce1Z88eJSUlKSkpSfv372/gygHUl8+Pfi1J6hEeorYhQVVe7xERogB/m/598Yre/fKMnCwsBnAdw8PNwoULNW3aNE2ZMkW9evXSsmXL1KJFC61YsaLa/osXL9bo0aP1i1/8Qj179tSvf/1r9evXT3/4wx8auHIA9eFKcal2n7woSRrereqojSS1bRmkhwZ0kk3S9hMX9N7eHJ25dIWQA0CS1MzIkxcXF2vXrl1KTk52tfn5+SkhIUGZmZnVviczM1N2u71SW2JiojZs2FCfpdbqYlGxvjj2da39PP3R6+nPaKeHR/T8eJ4cy7e/QAz7rD78Myk/Xu09Pf6TM/H3793x3HesWFMjScVXy1TkKNW5y9/qWF6RikvLFB4apK7tvuf2/b2iQnVvn0i9vy9Hmce+VuaxrxUc4KeVn59Q+5AgdW7TQl3afE+tWgQoOMBfNptNxVfLVFpWJn8/PzXzs8n/2sOv6tXmAG5CWPNADe3axrDzGxpuzp8/r9LSUoWHh1dqDw8P14EDB6p9T25ubrX9c3Nzq+3vcDjkcDhcz/Pz8yVJBQUFN1N6FftOXdQTK2peKwTAMy2D/HXPbe115ZvCGvvFRQQqoOwWfXU6X8fOF+kbh1OHCgt1SNI/GqZUANWI7RimN6YN8ekxK35ve/IPSUPDTUNISUnRvHnzqrRHR0cbUA0AT71gdAEA6uyUpLBZ9XPsy5cvKywsrMY+hoabtm3byt/fX2fPnq3UfvbsWUVERFT7noiICK/6JycnV5rGKisr04ULF9SmTRvZbOYbiy4oKFB0dLROnTql0NBQo8tp8vg+zIPvwlz4PsylKXwfTqdTly9fVlRUVK19DQ03gYGB6t+/vzIyMpSUlCSpPHxkZGRoxowZ1b5n6NChysjI0JNPPulqS09P19ChQ6vtHxQUpKCgyldbtGrVyhfl16vQ0FDL/g/aGPF9mAffhbnwfZiL1b+P2kZsKhg+LWW32zVp0iQNGDBAgwYN0qJFi1RUVKQpU6ZIkiZOnKgOHTooJSVFkjRz5kzdfffdSk1N1dixY7VmzRrt3LlTy5cvN/JjAAAAkzA83Dz00EPKy8vTnDlzlJubq7i4OH300UeuRcPZ2dny8/vuivVhw4Zp9erVeu655/Tss8/qtttu04YNG9S7d2+jPgIAADARw8ONJM2YMcPtNNSmTZuqtE2YMEETJkyo56qMERQUpLlz51aZSoMx+D7Mg+/CXPg+zIXvozKbk12vAACAhRi+QzEAAIAvEW4AAIClEG4AAIClEG4aAYfDobi4ONlsNmVlZRldTpN04sQJPf7444qJiVHz5s3VtWtXzZ07V8XFxUaX1mQsWbJEXbp0UXBwsAYPHqzt27ndiRFSUlI0cOBAhYSEqH379kpKStLBgweNLguSXnzxRdlstkr7wDVVhJtG4Omnn/ZoR0bUnwMHDqisrEyvvPKKvvrqK7300ktatmyZnn32WaNLaxLWrl0ru92uuXPnavfu3YqNjVViYqLOnTtndGlNzubNmzV9+nR98cUXSk9PV0lJie655x4VFRUZXVqTtmPHDr3yyivq27ev0aWYAldLmdyHH34ou92ut99+W3fccYf27NmjuLg4o8uCpN/97ndaunSpjh07ZnQpljd48GANHDhQf/jDHySV72QeHR2tn/70p5o9e7bB1TVteXl5at++vTZv3qy77rrL6HKapMLCQvXr109//OMf9cILLyguLk6LFi0yuixDMXJjYmfPntW0adP02muvqUWLFkaXgxvk5+erdevWRpdhecXFxdq1a5cSEhJcbX5+fkpISFBmZqaBlUEq/3sgib8LBpo+fbrGjh1b6e9IU2eKTfxQldPp1OTJk/WTn/xEAwYM0IkTJ4wuCdc5cuSIXn75ZS1YsMDoUizv/PnzKi0tde1aXiE8PFwHDhwwqCpI5SNoTz75pIYPH84u8QZZs2aNdu/erR07dhhdiqkwctPAZs+eLZvNVuPjwIEDevnll3X58mUlJycbXbKlefp9XO/06dMaPXq0JkyYoGnTphlUOWC86dOna//+/VqzZo3RpTRJp06d0syZM/XGG28oODjY6HJMhTU3DSwvL09ff/11jX1uvfVWPfjgg3r33Xdls9lc7aWlpfL399ejjz6qv/zlL/VdapPg6fcRGBgoSTpz5oxGjhypIUOGKC0trdJ9z1A/iouL1aJFC7311ltKSkpytU+aNEmXLl3Sxo0bjSuuCZsxY4Y2btyoLVu2KCYmxuhymqQNGzbogQcekL+/v6uttLRUNptNfn5+cjgclV5rSgg3JpWdna2CggLX8zNnzigxMVFvvfWWBg8erI4dOxpYXdN0+vRpjRo1Sv3799frr7/eZH9oGGHw4MEaNGiQXn75ZUnl0yGdOnXSjBkzWFDcwJxOp37605/qnXfe0aZNm3TbbbcZXVKTdfnyZZ08ebJS25QpU3T77bfrmWeeadJThay5MalOnTpVet6yZUtJUteuXQk2Bjh9+rRGjhypzp07a8GCBcrLy3O9FhERYWBlTYPdbtekSZM0YMAADRo0SIsWLVJRUZGmTJlidGlNzvTp07V69Wpt3LhRISEhys3NlSSFhYWpefPmBlfXtISEhFQJMN/73vfUpk2bJh1sJMIN4JH09HQdOXJER44cqRIuGfysfw899JDy8vI0Z84c5ebmKi4uTh999FGVRcaof0uXLpUkjRw5slL7ypUrNXny5IYvCKgG01IAAMBSWA0JAAAshXADAAAshXADAAAshXADAAAshXADAAAshXADAAAshXADAAAshXADAAAshXADAAB8YsuWLRo3bpyioqJks9m0YcMGr4/hdDq1YMECde/eXUFBQerQoYN+85vfeHUMbr8AAB6YPHmyLl26VKcf1kBTUVRUpNjYWD322GMaP358nY4xc+ZMffLJJ1qwYIH69OmjCxcu6MKFC14dg3ADAAB8YsyYMRozZozb1x0Oh375y1/qzTff1KVLl9S7d2/Nnz/fda+yf/3rX1q6dKn279+vHj16SJJiYmK8roNpKQAN5qOPPtKdd96pVq1aqU2bNvqP//gPHT16VJJ04sQJ2Ww2/fWvf9WIESPUvHlzDRw4UIcOHdKOHTs0YMAAtWzZUmPGjKl0V/aysjI9//zz6tixo4KCglw31aywadMm2Ww2Xbp0ydWWlZUlm82mEydOSJLS0tLUqlUrffzxx+rZs6datmyp0aNHKycnR5L0q1/9Sn/5y1+0ceNG2Ww22Ww2bdq0qd7/vACrmTFjhjIzM7VmzRrt3btXEyZM0OjRo3X48GFJ0rvvvqtbb71V7733nmJiYtSlSxdNnTrV65Ebwg2ABlNUVCS73a6dO3cqIyNDfn5+euCBB1RWVubqM3fuXD333HPavXu3mjVrpkceeURPP/20Fi9erL///e86cuSI5syZ4+q/ePFipaamasGCBdq7d68SExN13333uX5Yeuqbb77RggUL9Nprr2nLli3Kzs7WrFmzJEmzZs3Sgw8+6Ao8OTk5GjZsmG/+UIAmIjs7WytXrtS6des0YsQIde3aVbNmzdKdd96plStXSpKOHTumkydPat26dVq1apXS0tK0a9cu/fjHP/bqXExLAWgwP/rRjyo9X7Fihdq1a6d//vOfatmypaTyIJGYmCipfO794YcfVkZGhoYPHy5Jevzxx5WWluY6xoIFC/TMM8/oP//zPyVJ8+fP12effaZFixZpyZIlHtdWUlKiZcuWqWvXrpLK/4X5/PPPS5Jatmyp5s2by+FwKCIiom4fHmji9u3bp9LSUnXv3r1Su8PhUJs2bSSVj8Q6HA6tWrXK1e/Pf/6z+vfvr4MHD7qmqmpDuAHQYA4fPqw5c+Zo27ZtOn/+vGvEJjs7W7169ZIk9e3b19U/PDxcktSnT59KbefOnZMkFRQU6MyZM67gU2H48OH68ssvvaqtRYsWrmAjSZGRka7zALh5hYWF8vf3165du+Tv71/ptYp/3ERGRqpZs2aVAlDPnj0llf+cINwAMJ1x48apc+fO+tOf/qSoqCiVlZWpd+/eKi4udvUJCAhw/bfNZqu27fpprNr4+ZXPvjudTldbSUlJlX7Xn6PiPNe/B8DNiY+PV2lpqc6dO6cRI0ZU22f48OG6evWqjh496vrHxqFDhyRJnTt39vhcrLkB0CC+/vprHTx4UM8995x+8IMfqGfPnrp48eJNHTM0NFRRUVHaunVrpfatW7e6RoLatWsnSa7FwVL5gmJvBQYGqrS0tO7FAk1AYWGhsrKyXH/Hjh8/rqysLGVnZ6t79+569NFHNXHiRK1fv17Hjx/X9u3blZKSovfff1+SlJCQoH79+umxxx7Tnj17tGvXLj3xxBP64Q9/WGU6qyaEGwAN4pZbblGbNm20fPlyHTlyRJ9++qnsdvtNH/cXv/iF5s+fr7Vr1+rgwYOaPXu2srKyNHPmTElSt27dFB0drV/96lc6fPiw3n//faWmpnp9ni5dumjv3r06ePCgzp8/X+3oD9DU7dy5U/Hx8YqPj5ck2e12xcfHuy4CWLlypSZOnKinnnpKPXr0UFJSknbs2KFOnTpJKh9pfffdd9W2bVvdddddGjt2rHr27Kk1a9Z4VQfTUgAahJ+fn9asWaOf/exn6t27t3r06KHf//73rv0t6upnP/uZ8vPz9dRTT+ncuXPq1auX/va3v+m2226TVD7d9Oabb+p//ud/1LdvXw0cOFAvvPCCJkyY4NV5pk2bpk2bNmnAgAEqLCzUZ599dtO1A1YzcuTIGqdzAwICNG/ePM2bN89tn6ioKL399ts3VYfNyaQyAACwEKalAACApRBuAACApRBuAACApRBuAACApRBuAACApRBuAACApRBuAACApRBuAACApRBuAACApRBuAACApRBuAACApRBuAACApfw/q0UzdN0vwiIAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.distplot(df['amount'], bins=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Corelation Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['trans_time', 'mcc_code', 'trans_type', 'amount', 'term_id',\n",
       "       'trans_city', 'gender', 'trans_description', 'mcc_description'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABZcAAAMzCAYAAADarSxeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAACDyUlEQVR4nOzdeZid8/k/8PdkEYklsgdfbRCiRIJYYq82+GqJpUVRsbfVWoOSltgFJai19qqvUmur9saaIkgkaMQSS2rJZBOxZplzfn/4dZgmqZwzk5yZzuvV67muzOd8nue8H3odyZ373E9VsVgsBgAAAAAAStCi0gEAAAAAAGh6FJcBAAAAACiZ4jIAAAAAACVTXAYAAAAAoGSKywAAAAAAlExxGQAAAACAkikuAwAAAABQMsVlAAAAAABKprgMAAAAAEDJFJcBAAAAACiZ4jIAAAAAQBP2+OOPZ6eddspKK62Uqqqq3HXXXV97zqOPPpoNNtggbdq0Sc+ePXP99deX/L6KywAAAAAATdgnn3ySvn375tJLL12k/W+++Wa+//3vZ5tttsnYsWNz1FFH5eCDD84DDzxQ0vtWFYvFYjmBAQAAAABoXKqqqnLnnXdml112Weie448/Pvfcc09eeuml2rUf/ehHmTlzZu6///5Ffi+dywAAAAAAjczs2bMza9asOsfs2bMb5NpPPfVUBgwYUGdt++23z1NPPVXSdVo1SJoGMHfaG5WOANCgzu53UqUjADSoS2aOrnQEgAb1ec3cSkcAaFAffjyx0hGanMZckxx2yQ059dRT66ydfPLJOeWUU+p97cmTJ6dbt2511rp165ZZs2bls88+S9u2bRfpOo2muAwAAAAAwBeGDBmSwYMH11lr06ZNhdIsmOIyAAAAAEAj06ZNm8VWTO7evXuqq6vrrFVXV2f55Zdf5K7lRHEZAAAAAGiuCjWVTlARm266ae699946aw899FA23XTTkq7jgX4AAAAAAE3Yxx9/nLFjx2bs2LFJkjfffDNjx47NpEmTknwxYmPQoEG1+3/2s5/ljTfeyC9/+ctMmDAhl112Wf70pz/l6KOPLul9FZcBAAAAAJqw5557Luuvv37WX3/9JMngwYOz/vrrZ+jQoUmS999/v7bQnCSrrrpq7rnnnjz00EPp27dvzj///Fx99dXZfvvtS3rfqmKxWGy42yhfY34yI0A5zu53UqUjADSoS2aOrnQEgAb1ec3cSkcAaFAffjyx0hGanLnVr1Q6wkK17tar0hG+ls5lAAAAAABKprgMAAAAAEDJWlU6AAAAAABARRQKlU7QpOlcBgAAAACgZIrLAAAAAACUzFgMAAAAAKBZKhaNxagPncsAAAAAAJRMcRkAAAAAgJIZiwEAAAAANE8FYzHqQ+cyAAAAAAAlU1wGAAAAAKBkxmIAAAAAAM1T0ViM+tC5DAAAAABAyRSXAQAAAAAombEYAAAAAEDzVKipdIImTecyAAAAAAAlU1wGAAAAAKBkxmIAAAAAAM1TsVDpBE2azmUAAAAAAEqmuAwAAAAAQMmMxQAAAAAAmqeCsRj1oXMZAAAAAICSKS4DAAAAAFAyYzEAAAAAgGapWDQWoz50LgMAAAAAUDLFZQAAAAAASmYsBgAAAADQPBWMxagPncsAAAAAAJRMcRkAAAAAgJIZiwEAAAAANE9FYzHqQ+cyAAAAAAAlU1wGAAAAAKBkxmIAAAAAAM1ToabSCZo0ncsAAAAAAJRMcRkAAAAAgJIZiwEAAAAANE/FQqUTNGk6lwEAAAAAKJniMgAAAAAAJTMWAwAAAABongrGYtSHzmUAAAAAAEqmuAwAAAAAQMmMxQAAAAAAmqeisRj1oXMZAAAAAICSKS4DAAAAAFAyYzEAAAAAgOapYCxGfehcBgAAAACgZIrLAAAAAACUzFgMAAAAAKBZKhZrKh2hSdO5DAAAAABAyRSXAQAAAAAombEYAAAAAEDzVCxUOkGTpnMZAAAAAICSKS4DAAAAAFAyYzEAAAAAgOapYCxGfehcBgAAAACgZIrLAAAAAACUzFgMAAAAAKB5KhqLUR86lwEAAAAAKFnZxeWZM2fm6quvzpAhQzJjxowkyZgxY/Luu+82WDgAAAAAABqnssZivPDCCxkwYEDat2+ft956K4ccckg6duyYO+64I5MmTcoNN9zQ0DkBAAAAABpWoabSCZq0sjqXBw8enP333z+vvfZall566dr1733ve3n88ccbLBwAAAAAAI1TWcXlZ599Nj/96U/nW1955ZUzefLkeocCAAAAAKBxK2ssRps2bTJr1qz51l999dV06dKl3qEAAAAAABa7YqHSCZq0sjqXBw4cmNNOOy1z585NklRVVWXSpEk5/vjj84Mf/KBBAwIAAAAA0PiUVVw+//zz8/HHH6dr16757LPPsvXWW6dnz55ZbrnlcuaZZzZ0RgAAAAAAGpmyxmK0b98+Dz30UEaOHJkXXnghH3/8cTbYYIMMGDCgofMBAAAAACweBWMx6qOs4vK/bLHFFtliiy0aKgsAAAAAAE3EIheXf/vb3y7yRY844oiywgAAAAAA0DQscnH5ggsuqPPz1KlT8+mnn2aFFVZIksycOTPt2rVL165dFZcBAAAAgMavaCxGfSzyA/3efPPN2uPMM8/Meuutl5dffjkzZszIjBkz8vLLL2eDDTbI6aefvjjzAgAAAADQCCxycfmrTjrppFx88cXp1atX7VqvXr1ywQUX5MQTT2ywcAAAAAAANE5lPdDv/fffz7x58+Zbr6mpSXV1db1DAQAAAAAsdgVjMeqjrM7l7373u/npT3+aMWPG1K6NHj06hx56aAYMGNBg4QAAAAAAaJzKKi5fe+216d69ezbccMO0adMmbdq0ycYbb5xu3brl6quvbuiMAAAAAAA0MmWNxejSpUvuvffevPrqq3n55ZdTVVWVtdZaK2uuuWZD5wMAAAAAWDyMxaiXsorL/7LmmmtmjTXWSJJUVVU1SCAAAAAAABq/ssZiJMkNN9yQddddN23btk3btm3Tp0+f/OEPf2jIbAAAAAAANFJldS4PHz48J510Ug477LBsvvnmSZKRI0fmZz/7WaZNm5ajjz66QUMCAAAAADS0YrGm0hGatLKKyxdffHEuv/zyDBo0qHZt4MCBWWeddXLKKacoLgMAAAAA/JcrayzG+++/n80222y+9c022yzvv/9+vUMBAAAAANC4lVVc7tmzZ/70pz/Nt37LLbfUPuAPAAAAAKBRKxQa79EElDUW49RTT82ee+6Zxx9/vHbm8t///veMGDFigUVnAAAAAAD+u5TVufyDH/wgo0aNSufOnXPXXXflrrvuSufOnfPMM89k1113beiMAAAAAAA0MmV1LidJv379cuONNzZkFgAAAACAJafYNMZPNFZldS7fe++9eeCBB+Zbf+CBB3LffffVOxQsiufGvphf/PLkbDNwn/TefIeMePzJrz3nmTEvZPcDDsv6394pO+xxYO6656H59vzx9ruz3Q/2ywbbDMxehxyVF8e/sjjiAyzQhoO2zeEjL8yQV67LgXedmpX6rrbQvWv974Y56O7Tc9wLV+b4l6/JIfeelXV33aL29RatWua7J/woP33g7Bz/8jU56plLsvPwn2XZrissgTsB+MIBB++dZ18Ykberx+W+Ebdk/Q3W/Y/7d9pl+4x89t68XT0ujz75l3x3263qvH7RZcNS/eGEOscfb79qcd4CQB0H/+THeeEfj6V62viMeOT2bNCvz3/cv8uuO+TZMQ+metr4PDnq3my73bfrvH7Cr47Is2MezHvVL+btf47Jn+++If027LsY7wCg4ZRVXD7hhBNSU1Mz33qxWMwJJ5xQ71CwKD777PP06rlafn3Mzxdp/zvvTc4vjhuajTfom9uuvzT77rFLTj7nwvx91OjaPff97bGce/GVOfTAfXLrtRenV89V89PBJ2b6BzMX010AfGntHftn2xP3yeMX3ZGrdjwx1S9Pyt5/OCHtOi2/wP2fzfwkIy/5c67b7ZRcuf2QjLv1sQw87ydZbasvCjet2y6V7r175Inf3pmrv39ibv3phem02orZ85pjluBdAc3ZzrvtkFPPOiHnn3Nptt1qt/zjpVdy851Xp3Pnjgvcv+HG6+eKa87PTX+4LQO23DX33fO3XH/TJVnrW3UfGj7iocfTe40tao+fHeRzDVgydvvB93PWsF/lnGG/zVZbDMxLL03InXddn85dOi1w/8abbJBrrrswf/j9rdly851yz18fyk03X55vrb1m7Z7XX3szxw0+JZtt8r1sv92emfT2O7nzz79Pp4V8VgI0JlXFYrFY6klt27bNyy+/nB49etRZf+utt7LOOuvkk08+KTnI3GlvlHwO/EvvzXfIRcNOyne32myhe4Zfdk0ef/LZ3HXjFbVrxw4dlo8+/iS/G35GkmSvQ45K77XWrC1YFwqFDNh1UPb+4cAcvO8ei/cm+K9zdr+TKh2BJubAu07Ney+8kfuH/v6LhaqqHPn0b/Ps9Q/mycvvXqRrHHzPGXn94bF59PzbFvj6in1Wy8F3n56LNj0is96b3lDRaSYumTn66zfBV9w34pY8P+al/Oq405MkVVVVeX78o7nmyhtz8QXzdxtfed3wtGvXLj/e82e1a/f+7ea89OKE/PLoU5J80bncvv1y2X+fw5bIPfDf7fOauZWOQBMz4pHbM2bMCznumFOTfPG5Nv6VkbnyihtywfDfzbf/ut//Nu3atc2eux9Su/a3h2/Liy++nKOPXPCfF5Zbbtm88/64DNxx3zz26Nd/Qxe+6sOPJ1Y6QpPz2YgrKx1hodp+9yeVjvC1yupcbt++fd54Y/5i8Ouvv55lllmm3qFgcRj30oT033C9Omubb9Iv4156OUkyd+7cjH/ltfTf6Ms9LVq0SP8N16vdA7C4tGjdMiuuu2reHPnSl4vFYt4c+VL+Z4M1Fn7iV/TYfJ10Wm3FvD1qwkL3LL1c2xQLhXw+69P6Rgb4j1q3bp0+662TJ75SGCkWi3n80aey4Vd+v/VV/TZaL4//WyHlkRF/n2//ZltsnH+8/vf8/bn7cs7wk9OhwwoNnB5gfq1bt8566/fOo4/U/Vx79JEns9HG6y/wnI02Xj+PPvL3OmsjRjyx0P2tW7fO/gf8KDNnzsqLL/pzKND4lfVAv5133jlHHXVU7rzzzqy++upJvigsH3PMMRk4cODXnj979uzMnj27zlqL2bPTpk2bcuLAIpk244N06tihzlqnDivk408+zeezZ2fWrI9TU1OYf0/HDnlz0jtLMirQDLXrsFxatGqZj6d9WGf9k2mz0nn1lRZ6Xpvl2uaoUZek5VKtUqwp5N6Trq9boP6Klm1a57tD9spLf3kqcz7+rEHzA/y7jp06pFWrVpk6pe63JKZOnZY11lx1ged07dZ5gfu7dutc+/MjI57IvXc/mElvv5seq66SIUOPzh9vvzLfG/CjFAoeyAMsPp3+/+falCnT6qxPnTIta6654OdkdOvWOVOmTp9vf7duXeqsbf+/2+Ta6y9Ku3ZtM3nylOw6cFBmTP+gYW8AYDEoq3P53HPPzTLLLJO11lorq666alZdddV861vfSqdOnXLeeed97fnDhg1L+/bt6xznXHTF154HANQ1++PPc+UOv8o1A4fmkfNuzXYn7pNv9v/WfPtatGqZH156eFKV3Pvr6yqQFKBh3HX7vXngvkfy8vhXc989I7Lvnj/L+v36ZPMtN650NICyPfH409lys52y7Xd3z4iHHs/1N1y80DnOQAMrFhrv0QSU1bncvn37PPnkk3nooYcybty4tG3bNn369MlWW2319ScnGTJkSAYPHlxnrcVH75YTBRZZ544dMn1G3b/5nf7BzCy7TLss3aZNWq7QIi1btph/z4wP0vnfupkBGtqnH3yUwryaLNu5fZ31ZTovn4+nfriQs5IUi/ng7eokSfX4t9O550rZ/OcD8/bTX36NskWrlvnBpYen/cqd84e9ztK1DCwRM6Z/kHnz5qVL17rFkS5dOmdK9bQFnjOlelpJ+5Pk7bfeybRpM9JjtW/miceern9wgIWY/v8/17p27VxnvUvXzqmunrrAc6qrp6XrvxWJF7T/008/yxtvvJ033ng7zz07NmPGjsigQbtn+Pka8YDGrazO5eSLofXbbbddjjvuuBx22GELLCyvu+66+ec//znfeps2bbL88svXOYzEYHHr23utjBo9rs7aU88+n769v+jwa926ddbutUZGPTe29vVCoZBRo8fW7gFYXApza/L+i2+mx+brfLlYVZVVN++dd8a8tsjXqWpRlZZLffl3x/8qLHdctXtu3GdYPpv5cUPGBliouXPn5oWx/8iWW29au1ZVVZUtt+6f554du8BzRj87ts7+JNl6m80Wuj9JVlypWzp2XCFTJk9piNgACzV37tyMff6lbP3tLx8kX1VVla2/vWmefeb5BZ7z7DPP19mfJNtss8VC9/9LixZVWarNUvUPDbCYlV1cXhRvvfVW5s719F0Wj08//SwTXp2YCa9+8STUd9+rzoRXJ+b9//8Hiwsuvy5DTv9yTMseu3w/77z3fs6/9Jq88fY/c/Mdf80DDz+eQXvuWrtn0J675ra778+f730oE9+alNPPuySffT47u3x/2yV7c0Cz9PTV92WDH22TPj/YMp17rpTvnXlAWrdrk3G3PpYk2Xn4z/KdX+5Zu3/znw/Mqlv0zgqrdEnnniul/yHfy7q7bpEX7/rioTEtWrXMDy8/Miv2WS13HXlZqlq2yDJd2meZLu3TonXLitwj0Lxccen12We/3bPHXrtkjTVXy7kXnJJ2y7TNzTfekSS5+Iqz8+uTv/xG45WX/yHbDNgiPzvsgPRcY9Uce8Jh6bv+Orn2yv9LkrRbpl2Gnn5c+m3YN6t8Y+VsuXX//P6Pl+XNNyblkREjK3KPQPNy6SXXZr/998xee++WNXutngsuOj3LtGuXG2+8LUlyxZXn5eRTjq3df/ll12fAtlvlsMMPyhprrpYTfnVE1t+gd6783R+SJO3atc3Qk4/Jhhutl1VWWSnrrdc7l1x2dlZcqXvuuvO+itwjNDuFQuM9moCyxmJAY/DShNdy4OHH1/587sVXJkl23mFAzjzxmEybPiPvV3/ZwfI/K3XPpb85Lef+9ne58da70q1L55x6/FHZfJN+tXt2GLB1Ppj5YS65+sZMmzEja62xeq44/3RjMYAlYvxfn067Tstl68E/zLJd2qd6/Nu5adA5+WTarCTJ8it1SrFQrN3ful2b7HDGAVl+xY6Z9/mcTJv4Xu466vKM/+sXXwtfrnuH9Nrui8+4n9w/rM573bDnGXVGZwAsDn++47506tQxv/zV4enarUv+8eLL2Wu3QzL1/z/cauX/WSmFr3yuPffM8zn04GNzwolH5VdDj86bE9/K/nsflgkvf/ENjkJNTdZep1f23GuXLN9+uUx+f2oee+TvOeeMizJnjqYWYPG74/Z70qlzx/zqxKPSrVvnvPjCy9lt1wNqH0b6P6usWOfhos+MGpODDzw6J540OENPOSYTJ76dvX90aF4e/2qSpKamJmv2Wj177bNbOnXqkBkzZmbM6Beyw3Z71n72ATRmVcVisfj128qz3HLLZdy4cVlttQU/NfWr5k57Y3HFAKiIs/udVOkIAA3qkpmjKx0BoEF9XuMvJYD/Lh9+PLHSEZqczx68rNIRFqrtdj+vdISvpXMZAAAAAGieik1j/ERjtVhnLgMAAAAA8N9JcRkAAAAAgJIt1rEYv/vd79KtW7fF+RYAAAAAAOUpGItRH2V1Lh9xxBH57W9/O9/6JZdckqOOOqr257333jvLLLNM2eEAAAAAAGicyiou33777dl8883nW99ss81y22231TsUAAAAAACNW1ljMaZPn5727dvPt7788stn2rRp9Q4FAAAAALDYGYtRL2V1Lvfs2TP333//fOv33XdfVltttXqHAgAAAACgcSurc3nw4ME57LDDMnXq1HznO99JkowYMSLnn39+LrzwwobMBwAAAABAI1RWcfnAAw/M7Nmzc+aZZ+b0009PkvTo0SOXX355Bg0a1KABAQAAAAAWi6KxGPVRVnE5SQ499NAceuihmTp1atq2bZtll122IXMBAAAAANCIlVVcfvPNNzNv3rysscYa6dKlS+36a6+9ltatW6dHjx4NlQ8AAAAAgEaorAf67b///nnyySfnWx81alT233//+mYCAAAAAFj8CoXGezQBZRWXn3/++Wy++ebzrffv3z9jx46tbyYAAAAAABq5sorLVVVV+eijj+Zb//DDD1NTU1PvUAAAAAAANG5lFZe32mqrDBs2rE4huaamJsOGDcsWW2zRYOEAAAAAABabYqHxHk1AWQ/0O+ecc7LVVlulV69e2XLLLZMkTzzxRD788MM88sgjDRoQAAAAAIDGp6zO5bXXXjvjxo3LHnvskSlTpuSjjz7KoEGD8sorr6R3794NnREAAAAAgEamrM7lJOnUqVN22WWXbLbZZin8/6cXjhw5MkkycODAhkkHAAAAALC4FJrG+InGqqzi8v33359BgwZl+vTpKRaLdV6rqqryUD8AAAAAgP9yZY3FOPzww7P77rvnvffeS6FQqHMoLAMAAAAA/Pcrq3O5uro6gwcPTrdu3Ro6DwAAAADAklE0FqM+yupc/uEPf5hHH320gaMAAAAAANBUlNW5fMkll2T33XfPE088kXXXXTetW7eu8/oRRxzRIOEAAAAAAGicyiou//GPf8yDDz6YpZdeOo8++miqqqpqX6uqqlJcBgAAAAAav4KxGPVRVnH517/+dU499dSccMIJadGirMkaAAAAAAA0YWVVhufMmZM999xTYRkAAAAAoJkqqzq833775ZZbbmnoLAAAAAAAS06h0HiPJqCssRg1NTU599xz88ADD6RPnz7zPdBv+PDhDRIOAAAAAIDGqazi8osvvpj1118/SfLSSy/Vee2rD/cDAAAAAOC/U1nF5UceeaShcwAAAAAALFnFYqUTNGmeyAcAAAAAQMkUlwEAAAAAKFlZYzEAAAAAAJq8QqHSCZo0ncsAAAAAAJRMcRkAAAAAgJIZiwEAAAAANE/GYtSLzmUAAAAAAEqmuAwAAAAAQMkUlwEAAAAAKJmZywAAAABA81Q0c7k+dC4DAAAAAFAyxWUAAAAAAEpmLAYAAAAA0DwVjMWoD53LAAAAAACUTHEZAAAAAICSGYsBAAAAADRPxWKlEzRpOpcBAAAAAJq4Sy+9ND169MjSSy+dTTbZJM8888x/3H/hhRemV69eadu2bVZZZZUcffTR+fzzz0t6T8VlAAAAAIAm7JZbbsngwYNz8sknZ8yYMenbt2+23377TJkyZYH7b7rpppxwwgk5+eST8/LLL+eaa67JLbfckl/96lclva/iMgAAAADQPBUKjfcowfDhw3PIIYfkgAMOyNprr50rrrgi7dq1y7XXXrvA/U8++WQ233zz7L333unRo0e222677LXXXl/b7fzvFJcBAAAAABqZ2bNnZ9asWXWO2bNnz7dvzpw5GT16dAYMGFC71qJFiwwYMCBPPfXUAq+92WabZfTo0bXF5DfeeCP33ntvvve975WUUXEZAAAAAKCRGTZsWNq3b1/nGDZs2Hz7pk2blpqamnTr1q3Oerdu3TJ58uQFXnvvvffOaaedli222CKtW7fO6quvnm9/+9vGYgAAAAAALJJKj774D8eQIUPy4Ycf1jmGDBnSILf96KOP5qyzzspll12WMWPG5I477sg999yT008/vaTrtGqQNAAAAAAANJg2bdqkTZs2X7uvc+fOadmyZaqrq+usV1dXp3v37gs856STTsq+++6bgw8+OEmy7rrr5pNPPslPfvKT/PrXv06LFovWk6xzGQAAAACgiVpqqaXSr1+/jBgxonatUChkxIgR2XTTTRd4zqeffjpfAblly5ZJkmKxuMjvrXMZAAAAAGieioVKJ2gQgwcPzn777ZcNN9wwG2+8cS688MJ88sknOeCAA5IkgwYNysorr1w7s3mnnXbK8OHDs/7662eTTTbJ66+/npNOOik77bRTbZF5USguAwAAAAA0YXvuuWemTp2aoUOHZvLkyVlvvfVy//331z7kb9KkSXU6lU888cRUVVXlxBNPzLvvvpsuXbpkp512yplnnlnS+1YVS+lzXozmTnuj0hEAGtTZ/U6qdASABnXJzNGVjgDQoD6vmVvpCAAN6sOPJ1Y6QpPz2dWDKx1hodoePLzSEb6WzmUAAAAAoFkqFhpF322T5YF+AAAAAACUTHEZAAAAAICSGYsBAAAAADRPhUKlEzRpOpcBAAAAACiZ4jIAAAAAACUzFgMAAAAAaJ6KxmLUh85lAAAAAABKprgMAAAAAEDJjMUAAAAAAJqnQrHSCZo0ncsAAAAAAJRMcRkAAAAAgJIZiwEAAAAANE+FQqUTNGk6lwEAAAAAKJniMgAAAAAAJTMWAwAAAABonozFqBedywAAAAAAlExxGQAAAACAkhmLAQAAAAA0T8VipRM0aTqXAQAAAAAomeIyAAAAAAAlMxYDAAAAAGieCoVKJ2jSdC4DAAAAAFAyxWUAAAAAAEpmLAYAAAAA0DwVipVO0KTpXAYAAAAAoGSKywAAAAAAlMxYDAAAAACgeSoWKp2gSdO5DAAAAABAyRSXAQAAAAAombEYAAAAAEDzVChWOkGTpnMZAAAAAICSKS4DAAAAAFCyRjMW4+x+J1U6AkCDOmH06ZWOANCg7ulzYKUjADSo52dMrHQEACqsWChUOkKTpnMZAAAAAICSKS4DAAAAAFCyRjMWAwAAAABgiSoUK52gSdO5DAAAAABAyRSXAQAAAAAombEYAAAAAEDzVCxUOkGTpnMZAAAAAICSKS4DAAAAAFAyYzEAAAAAgOapUKx0giZN5zIAAAAAACVTXAYAAAAAoGTGYgAAAAAAzVOhUOkETZrOZQAAAAAASqa4DAAAAABAyYzFAAAAAACap0Kx0gmaNJ3LAAAAAACUTHEZAAAAAICSGYsBAAAAADRPxUKlEzRpOpcBAAAAACiZ4jIAAAAAACUzFgMAAAAAaJ4KxUonaNJ0LgMAAAAAUDLFZQAAAAAASmYsBgAAAADQLBULhUpHaNJ0LgMAAAAAUDLFZQAAAAAASmYsBgAAAADQPBWKlU7QpOlcBgAAAACgZIrLAAAAAACUzFgMAAAAAKB5MhajXnQuAwAAAABQMsVlAAAAAABKZiwGAAAAANA8FQuVTtCk6VwGAAAAAKBkissAAAAAAJTMWAwAAAAAoHkqFCudoEnTuQwAAAAAQMkUlwEAAAAAKJmxGAAAAABAs1Q0FqNedC4DAAAAAFAyxWUAAAAAAEpmLAYAAAAA0DwZi1EvOpcBAAAAACiZ4jIAAAAAACUzFgMAAAAAaJ4KhUonaNJ0LgMAAAAAUDLFZQAAAAAASmYsBgAAAADQPBWKlU7QpOlcBgAAAACgZIrLAAAAAACUzFgMAAAAAKB5MhajXnQuAwAAAABQMsVlAAAAAABKZiwGAAAAANAsFYvGYtSHzmUAAAAAAEqmuAwAAAAAQMmMxQAAAAAAmqeCsRj1oXMZAAAAAICSKS4DAAAAAFAyYzEAAAAAgObJWIx6qVfn8pw5c/LKK69k3rx5DZUHAAAAAIAmoKzi8qeffpqDDjoo7dq1yzrrrJNJkyYlSQ4//PCcffbZDRoQAAAAAIDGp6zi8pAhQzJu3Lg8+uijWXrppWvXBwwYkFtuuaXBwgEAAAAALC7FQrHRHk1BWTOX77rrrtxyyy3p379/qqqqatfXWWedTJw4scHCAQAAAADQOJXVuTx16tR07dp1vvVPPvmkTrEZAAAAAID/TmUVlzfccMPcc889tT//q6B89dVXZ9NNN22YZAAAAAAAi1Oh2HiPJqCssRhnnXVWdthhh4wfPz7z5s3LRRddlPHjx+fJJ5/MY4891tAZAQAAAABoZMrqXN5iiy0yduzYzJs3L+uuu24efPDBdO3aNU899VT69evX0BkBAAAAAGhkyupcTpLVV189V111VUNmAQAAAABYcgqVDtC0lV1crqmpyZ133pmXX345SbL22mtn5513TqtWZV8SAAAAAIAmoqxK8D/+8Y8MHDgwkydPTq9evZIk55xzTrp06ZK77747vXv3btCQAAAAAAA0LmUVlw8++OCss846ee6559KhQ4ckyQcffJD9998/P/nJT/Lkk082aEgAAAAAgIZWLBQrHaFJK6u4PHbs2DqF5STp0KFDzjzzzGy00UYNFg4AAAAAgMapRTknrbnmmqmurp5vfcqUKenZs2e9QwEAAAAA0LiV1bk8bNiwHHHEETnllFPSv3//JMnTTz+d0047Leecc05mzZpVu3f55ZdvmKQAAAAAAA3JWIx6Kau4vOOOOyZJ9thjj1RVVSVJisUv/kXstNNOtT9XVVWlpqamIXICAAAAANCIlFVcfvjhh2uLygAAAAAAND9lFZe//e1vN3AMAAAAAIAlrFDpAE1bWQ/0W3XVVXPaaadl0qRJDZ0HAAAAAIAmoKzi8pFHHpk77rgjq622WrbddtvcfPPNmT17dkNnAwAAAACgkSqruHzUUUdl7NixeeaZZ/Ktb30rhx9+eFZcccUcdthhGTNmTENnBAAAAABocMVCsdEeTUFZxeV/2WCDDfLb3/427733Xk4++eRcffXV2WijjbLeeuvl2muvTbHYNP4hAAAAAABQmrIe6Pcvc+fOzZ133pnrrrsuDz30UPr375+DDjoo77zzTn71q1/lb3/7W2666aaGygoAAAAAQCNRVnF5zJgxue666/LHP/4xLVq0yKBBg3LBBRdkrbXWqt2z6667ZqONNmqwoAAAAAAADapQ6QBNW1nF5Y022ijbbrttLr/88uyyyy5p3br1fHtWXXXV/OhHP6p3QAAAAAAAGp+yistvvPFGvvnNb/7HPcsss0yuu+66skIBAAAAANC4lfVAv2222SbTp0+fb33mzJlZbbXV6h0KAAAAAGBxKxaKjfZoCsoqLr/11lupqamZb3327Nl599136x0KAAAAAIDGraSxGH/5y19qf/3AAw+kffv2tT/X1NRkxIgR6dGjR4OFAwAAAACgcSqpuLzLLrskSaqqqrLffvvVea1169bp0aNHzj///AYLB19nw0HbZtOffD/Ldmmf6pcn5f6Tf5/3xr2xwL1r/e+G2fwXO6fjN7ulReuWmfFmdZ6+6t68eOfIJEmLVi2zzbG7p+c262WFb3TJ7I8+y5sjX8qIs2/Ox1NmLsG7Apqr58a+mOtuui3jJ7yeqdNn5KJhJ+W7W232H895ZswL+c3FV+b1N99O965d8tP99sou39+2zp4/3n53rrvptkyb8UF69Vwtvzr60Ky7dq/FeSsAtX6w/y758aE/SscuHfP6+Ndz/om/zfixExa4d9U1e+Qnxx2Qtfr0yoqrdM8FQy/JLVffttBr73vY3vnFr36Sm6+6LReefMniugWAOn720/1y9OCfpnu3LnnhhZdz9OChee65sQvdv9tu388pJx+bb37zf/L662/l178+K/c/8Ejt6zvv/L/5ySH7Zv31102nTh2y0cbb54UXxi+BOwGSJIVKB2jaShqLUSgUUigU8o1vfCNTpkyp/blQKGT27Nl55ZVXsuOOOy6urFDH2jv2z7Yn7pPHL7ojV+14YqpfnpS9/3BC2nVafoH7P5v5SUZe8udct9spuXL7IRl362MZeN5PstpW6yZJWrddKt1798gTv70zV3//xNz60wvTabUVs+c1xyzBuwKas88++zy9eq6WXx/z80Xa/857k/OL44Zm4w365rbrL82+e+ySk8+5MH8fNbp2z31/eyznXnxlDj1wn9x67cXp1XPV/HTwiZn+wczFdBcAXxowcJscefLPc/Xw67Pf9ofktfETc+FNv0mHTisscP/Sbdvk3Unv59Kzrsy06vmf8fJV3+rbK7v+eKe89o/XF0NygAX74Q93yrnnnpQzz7wwm/T/Xl58cXz+evcf0qVLpwXu79+/X/5wwyW5/vqbs8kmO+Qvdz+QW2+9Omt/5S/6l1mmXf7+5DP59YlnLanbAGgwZc1cfvPNN9O5c+ev3bfuuuvmn//8ZzlvAV+r/8E75PmbH8m4Wx/PtNfezT2/ujZzP5ud9fbYeoH733765bzywHOZ9vp7+WDSlDxz3QOpnjAp39joi/+oz/7os/zfj8/O+HtGZfob7+fd51/PfUN/n5X6rJblV1rwbxQAGtKWm26UI36yXwZsvfki7f/TXfdk5RW757jDD8nqPb6RvX84MNt+e4vccMudtXtuuOXO/HCnHbLr97fL6qt+M0OPOzxLt2mTO//64OK6DYBae/1k9/z5pntyzy33563X3s45xw/P5599nh33+t4C97887pVccvoV+dufH87cOXMXet227drm1EtOzLDjzstHH368uOIDzOfIIw7Jtdf+MTfc8KdMmPBafnHYkHz66efZb789F7j/sF8clAcffDTDL/hdJrzyek499bw8//xL+fmhX34b/Kab7shZZ12Uhx8euaRuA6DBlFVcXlRvvfVW5s5d+G8KoVwtWrfMiuuumjdHvvTlYrGYN0e+lP/ZYI1FukaPzddJp9VWzNujFvy1zCRZerm2KRYK+XzWp/WNDNDgxr00If03XK/O2uab9Mu4l15OksydOzfjX3kt/Tf6ck+LFi3Sf8P1avcALC6tWrdKrz698uwTX36bolgs5tknRmfdfmvX69rHnnVk/j7i6TrXBljcWrdunQ02WLdOEbhYLObhR55I/036LfCcTfpvMF/R+KG/PZZNFrIfWPKKhcZ7NAUlzVyGxqJdh+XSolXLfDztwzrrn0yblc6rr7TQ89os1zZHjbokLZdqlWJNIfeedH3dAvVXtGzTOt8dslde+stTmfPxZw2aH6AhTJvxQTp17FBnrVOHFfLxJ5/m89mzM2vWx6mpKcy/p2OHvDnpnSUZFWiGVujYPq1atcyMqTPqrH8w7YP06PmNsq87YOfvpNe6a+bA7/2svhEBStK5c8e0atUq1VOm1lmfUj0tvdbsucBzunfrkuop0+bb361bl8WWE2BJqkhxefbs2Zk9e3adtXnFmrSqalmJODQjsz/+PFfu8KsstczSWXXzdbLdiftk5qQpefvpuh18LVq1zA8vPTypSu799XUVSgsAwFd1XalLBp92WI740bGZM3tOpeMAADR7FSkuDxs2LKeeemqdtW8v3zvfWaFPJeLQBH36wUcpzKvJsp3b11lfpvPy+Xjqhws5K0mxmA/erk6SVI9/O517rpTNfz6wTnG5RauW+cGlh6f9yp3zh73O0rUMNFqdO3bI9Bkf1Fmb/sHMLLtMuyzdpk1artAiLVu2mH/PjA/S+d+6mQEa2swZH2bevJp07NKxznqHzh0y/d+6mRfVWn16pWOXjrn+gatq11q1apn1+vfJDw/YNVv12DaFQhP5DinQ5EybNiPz5s1Lt651u467duuc6uqpCzxncvXUdOvaeZH3AxXgtw71slhnLi/MkCFD8uGHH9Y5tmq/TiWi0EQV5tbk/RffTI/Nv/L/m6qqrLp577wz5rVFvk5Vi6q0XOrLv2P5V2G546rdc+M+w/LZTA+IARqvvr3XyqjR4+qsPfXs8+nb+1tJvpgLuHavNTLqubG1rxcKhYwaPbZ2D8DiMm/uvLzywivZaIsNateqqqqy0Rb98uLo8WVd87knRmfvbQ7IoG0Prj3Gj52QB+74WwZte7DCMrBYzZ07N2PGvJhttvny4ctVVVXZ5ttb5OlRC54BP+rpMXX2J8l3v7NlRi1kP0BTU5HO5TZt2qRNmzZ1gxiJQYmevvq+7Hz+T/P+C2/mvXETs/GB/5vW7dpk3K2PJUl2Hv6zfDT5gzx87i1Jks1/PjDvvfBGPni7Oq3atE7PbdbLurtukXtP/GLsRYtWLfPDy49M9949csuB56WqZYss0+WLzujPZn6cwtyaytwo0Gx8+ulnmfTOe7U/v/tedSa8OjHtl18uK3bvmgsuvy5Tpk3PsJOOTZLsscv388fb7875l16TXXfcLs+MHpcHHn48l/3mtNprDNpz1/z6zPOzzlprpPfavXLjn+7KZ5/Pzi7f33aJ3x/Q/Pzxyltz0oVD8vK4VzL++Zez5yE/zNLtls49N9+XJBl60ZBMnTwtlw/7ohO5VetWWXXNHrW/7rJi56yxTs989slneeetd/PpJ5/ljVferPMen3/6eT78YNZ86wCLw0W/vSrXXD08o8e8kOeeHZvDDz8oyyzTNjfc8KckyTXXXJD33puck046J0lyyaXX5G8P3ZqjjvxJ7rtvRHbfY2D69euTn//ihNprduiwQlZZZaWstGK3JMmaa66eJKmunqrDGWj0Gqy4PHPmzKywwgp11n73u9+lW7duDfUWUMf4vz6ddp2Wy9aDf5hlu7RP9fi3c9Ogc/LJtFlJkuVX6pRioVi7v3W7NtnhjAOy/IodM+/zOZk28b3cddTlGf/Xp5Mky3XvkF7bffHE3p/cP6zOe92w5xnzzWUGaGgvTXgtBx5+fO3P5158ZZJk5x0G5MwTj8m06TPyfvWU2tf/Z6XuufQ3p+Xc3/4uN956V7p16ZxTjz8qm3/l6eM7DNg6H8z8MJdcfWOmzZiRtdZYPVecf7qxGMAS8be/PJIVOq2QQ447IJ26dMxr/3g9R+/zy8yY9sW4nu4rd6vz+7Uu3TrnDw9dXfvzjw/9UX586I8y5smx+fkPj1rS8QHmc9ttd6dL544ZOvSYdO/WJePGjc9OA/fNlP//0L5VVlk5ha98rj399OgM2u/wnHrKcTnttF/m9dffyu67H5zx41+p3bPjjtvm6quG1/78fzdeliQ5/YzhOeOMC5bQnUHzVfTFp3qpKhaLxa/fVtc555yTHj16ZM8990yS7LHHHrn99tvTvXv33Hvvvenbt2/JQU7/5j4lnwPQmJ0w+vRKRwBoUFv2ObDSEQAa1PMzJlY6AkCDmv35PysdocmZtsPWlY6wUJ3ve6zSEb5WWTOXr7jiiqyyyipJkoceeigPPfRQ7rvvvuywww457rjjGjQgAAAAAACNT1ljMSZPnlxbXP7rX/+aPfbYI9ttt1169OiRTTbZpEEDAgAAAAAsFsZi1EtZncsdOnTIP//5RZv9/fffnwEDBiRJisViamo89AwAAAAA4L9dWZ3Lu+22W/bee++sscYamT59enbYYYckyfPPP5+ePXs2aEAAAAAAABqfsorLF1xwQXr06JF//vOfOffcc7PssssmSd5///38/Oc/b9CAAAAAAACLQ9FYjHopq7jcunXrHHvssfOtH3300fUOBAAAAABA41dWcTlJXnvttTzyyCOZMmVKCoW6Jf6hQ4fWOxgAAAAAAI1XWcXlq666Koceemg6d+6c7t27p6qqqva1qqoqxWUAAAAAoNEzFqN+WpRz0hlnnJEzzzwzkydPztixY/P888/XHmPGjGnojAAAAAAA/AeXXnppevTokaWXXjqbbLJJnnnmmf+4f+bMmfnFL36RFVdcMW3atMmaa66Ze++9t6T3LKtz+YMPPsjuu+9ezqkAAAAAADSgW265JYMHD84VV1yRTTbZJBdeeGG23377vPLKK+natet8++fMmZNtt902Xbt2zW233ZaVV145b7/9dlZYYYWS3reszuXdd989Dz74YDmnAgAAAAA0CsVC4z1KMXz48BxyyCE54IADsvbaa+eKK65Iu3btcu211y5w/7XXXpsZM2bkrrvuyuabb54ePXpk6623Tt++fUt637I6l3v27JmTTjopTz/9dNZdd920bt26zutHHHFEOZcFAAAAACDJ7NmzM3v27Dprbdq0SZs2beqszZkzJ6NHj86QIUNq11q0aJEBAwbkqaeeWuC1//KXv2TTTTfNL37xi/z5z39Oly5dsvfee+f4449Py5YtFzljWcXlK6+8Mssuu2wee+yxPPbYY3Veq6qqUlwGAAAAAKiHYcOG5dRTT62zdvLJJ+eUU06pszZt2rTU1NSkW7dudda7deuWCRMmLPDab7zxRh5++OHss88+uffee/P666/n5z//eebOnZuTTz55kTOWVVx+8803yzkNAAAAAKDxKFZVOsFCDRkyJIMHD66z9u9dy+UqFArp2rVrrrzyyrRs2TL9+vXLu+++m9/85jeLv7gMAAAAAMDis6ARGAvSuXPntGzZMtXV1XXWq6ur07179wWes+KKK6Z169Z1RmB861vfyuTJkzNnzpwstdRSi5Sx7OLyO++8k7/85S+ZNGlS5syZU+e14cOHl3tZAAAAAAAW0VJLLZV+/fplxIgR2WWXXZJ80Zk8YsSIHHbYYQs8Z/PNN89NN92UQqGQFi1aJEleffXVrLjiiotcWE7KLC6PGDEiAwcOzGqrrZYJEyakd+/eeeutt1IsFrPBBhuUc0kAAAAAgCWqWKh0goYxePDg7Lffftlwww2z8cYb58ILL8wnn3ySAw44IEkyaNCgrLzyyhk2bFiS5NBDD80ll1ySI488Mocffnhee+21nHXWWSU/S6+s4vKQIUNy7LHH5tRTT81yyy2X22+/PV27ds0+++yT//3f/y3nkgAAAAAAlGHPPffM1KlTM3To0EyePDnrrbde7r///tqH/E2aNKm2QzlJVllllTzwwAM5+uij06dPn6y88so58sgjc/zxx5f0vlXFYrFYatjlllsuY8eOzeqrr54OHTpk5MiRWWeddTJu3LjsvPPOeeutt0q9ZE7/5j4lnwPQmJ0w+vRKRwBoUFv2ObDSEQAa1PMzJlY6AkCDmv35PysdocmZvNW3Kx1hobo//milI3ytsjqXl1lmmdo5yyuuuGImTpyYddZZJ0kybdq0hksHAAAAALCYFAtVlY7QpJVVXO7fv39GjhyZb33rW/ne976XY445Ji+++GLuuOOO9O/fv6EzAgAAAADQyJRVXB4+fHg+/vjjJMmpp56ajz/+OLfcckvWWGONDB8+vEEDAgAAAADQ+JRcXK6pqck777yTPn36JPliRMYVV1zR4MEAAAAAABanYqHSCZq2Fl+/pa6WLVtmu+22ywcffLA48gAAAAAA0ASUXFxOkt69e+eNN95o6CwAAAAAADQRZRWXzzjjjBx77LH561//mvfffz+zZs2qcwAAAAAANHbFYlWjPZqCsh7o973vfS9JMnDgwFRVfXmjxWIxVVVVqampaZh0AAAAAAA0SmUVl6+77rqsssoqadmyZZ31QqGQSZMmNUgwAAAAAAAar7KKywceeGDef//9dO3atc769OnTM2DAgOy3334NEg4AAAAAYHEpFiqdoGkra+byv8Zf/LuPP/44Sy+9dL1DAQAAAADQuJXUuTx48OAkSVVVVU466aS0a9eu9rWampqMGjUq6623XoMGBAAAAACg8SmpuPz8888n+aJz+cUXX8xSSy1V+9pSSy2Vvn375thjj23YhAAAAAAAi0GxMP90BhZdScXlRx55JElywAEH5KKLLsryyy+/WEIBAAAAANC4lfVAv+uuu66hcwAAAAAA0ISUVVwGAAAAAGjqisVKJ2jaWlQ6AAAAAAAATY/iMgAAAAAAJTMWAwAAAABoloqFqkpHaNJ0LgMAAAAAUDLFZQAAAAAASmYsBgAAAADQLBmLUT86lwEAAAAAKJniMgAAAAAAJTMWAwAAAABolorFSido2nQuAwAAAABQMsVlAAAAAABKZiwGAAAAANAsFQtVlY7QpOlcBgAAAACgZIrLAAAAAACUzFgMAAAAAKBZKhaNxagPncsAAAAAAJRMcRkAAAAAgJIZiwEAAAAANEvFQqUTNG06lwEAAAAAKJniMgAAAAAAJTMWAwAAAABolgrFqkpHaNJ0LgMAAAAAUDLFZQAAAAAASmYsBgAAAADQLBWNxagXncsAAAAAAJRMcRkAAAAAgJIZiwEAAAAANEvFgrEY9aFzGQAAAACAkikuAwAAAABQMmMxAAAAAIBmqVisdIKmTecyAAAAAAAlU1wGAAAAAKBkxmIAAAAAAM1SsVBV6QhNms5lAAAAAABKprgMAAAAAEDJjMUAAAAAAJqlQtFYjPrQuQwAAAAAQMkUlwEAAAAAKJmxGAAAAABAs1Q0FqNedC4DAAAAAFAyxWUAAAAAAEpmLAYAAAAA0CwVi5VO0LTpXAYAAAAAoGSKywAAAAAAlMxYDAAAAACgWSoUqyodoUnTuQwAAAAAQMkUlwEAAAAAKJmxGAAAAABAs1Q0FqNedC4DAAAAAFAyxWUAAAAAAEpmLAYAAAAA0CwVi5VO0LTpXAYAAAAAoGSKywAAAAAAlMxYDAAAAACgWSoUqyodoUnTuQwAAAAAQMkUlwEAAAAAKFmjGYtxyczRlY4A0KDu6XNgpSMANKgnXri20hEAGlTblbasdAQAKqxoLEa96FwGAAAAAKBkissAAAAAAJSs0YzFAAAAAABYkgrGYtSLzmUAAAAAAEqmuAwAAAAAQMmMxQAAAAAAmqVipQM0cTqXAQAAAAAomeIyAAAAAAAlMxYDAAAAAGiWCsWqSkdo0nQuAwAAAABQMsVlAAAAAABKZiwGAAAAANAsFY3FqBedywAAAAAAlExxGQAAAACAkhmLAQAAAAA0S4VKB2jidC4DAAAAAFAyxWUAAAAAAEpmLAYAAAAA0CwVU1XpCE2azmUAAAAAAEqmuAwAAAAAQMmMxQAAAAAAmqVCsdIJmjadywAAAAAAlExxGQAAAACAkhmLAQAAAAA0S4VUVTpCk6ZzGQAAAACAkikuAwAAAABQMmMxAAAAAIBmqWgsRr3oXAYAAAAAoGSKywAAAAAAlMxYDAAAAACgWSpUOkATp3MZAAAAAICSKS4DAAAAAFAyYzEAAAAAgGapmKpKR2jSdC4DAAAAAFAyxWUAAAAAAEpmLAYAAAAA0CwVKh2gidO5DAAAAABAyRSXAQAAAAAombEYAAAAAECzZCxG/ehcBgAAAACgZIrLAAAAAACUzFgMAAAAAKBZKqaq0hGaNJ3LAAAAAACUTHEZAAAAAICSGYsBAAAAADRLBVMx6kXnMgAAAAAAJVNcBgAAAACgZMZiAAAAAADNUiHmYtSHzmUAAAAAAEqmuAwAAAAAQMmMxQAAAAAAmqVipQM0cTqXAQAAAAAomeIyAAAAAAAlMxYDAAAAAGiWCpUO0MTpXAYAAAAAoGSKywAAAAAAlMxYDAAAAACgWSpUVVU6QpOmcxkAAAAAgJIpLgMAAAAAUDJjMQAAAACAZqlY6QBNnM5lAAAAAABKprgMAAAAAEDJjMUAAAAAAJqlQqUDNHE6lwEAAAAAKJniMgAAAAAAJTMWAwAAAABolgpVlU7QtOlcBgAAAACgZIrLAAAAAACUTHEZAAAAAGiWCqlqtEepLr300vTo0SNLL710NtlkkzzzzDOLdN7NN9+cqqqq7LLLLiW/p+IyAAAAAEATdsstt2Tw4ME5+eSTM2bMmPTt2zfbb799pkyZ8h/Pe+utt3Lsscdmyy23LOt9FZcBAAAAAJqw4cOH55BDDskBBxyQtddeO1dccUXatWuXa6+9dqHn1NTUZJ999smpp56a1VZbraz3Lau4/Pjjj2fevHnzrc+bNy+PP/54WUEAAAAAAJakYiM+FtWcOXMyevToDBgwoHatRYsWGTBgQJ566qmFnnfaaaela9euOeigg0p4t7palXPSNttsk/fffz9du3ats/7hhx9mm222SU1NTdmBAAAAAACau9mzZ2f27Nl11tq0aZM2bdrUWZs2bVpqamrSrVu3OuvdunXLhAkTFnjtkSNH5pprrsnYsWPrlbGszuVisZiqqvmHSk+fPj3LLLNMvQIBAAAAADR3w4YNS/v27escw4YNq/d1P/roo+y777656qqr0rlz53pdq6TO5d122y1JUlVVlf33379OlbympiYvvPBCNttss3oFAgAAAABYEgrz9882GkOGDMngwYPrrP1713KSdO7cOS1btkx1dXWd9erq6nTv3n2+/RMnTsxbb72VnXbaqXatUCgkSVq1apVXXnklq6+++iJlLKm43L59+yRfdC4vt9xyadu2be1rSy21VPr3759DDjmklEsCAAAAAPBvFjQCY0GWWmqp9OvXLyNGjMguu+yS5Iti8YgRI3LYYYfNt3+ttdbKiy++WGftxBNPzEcffZSLLrooq6yyyiJnLKm4fN111yVJevTokWOPPdYIDAAAAACAChs8eHD222+/bLjhhtl4441z4YUX5pNPPskBBxyQJBk0aFBWXnnlDBs2LEsvvXR69+5d5/wVVlghSeZb/zplPdDv5JNPLuc0AAAAAIBGo1DpAA1kzz33zNSpUzN06NBMnjw56623Xu6///7ah/xNmjQpLVqU9fi9/6iqWCwWSz2puro6xx57bEaMGJEpU6bk3y9RU1NTcpBu7dcq+RyAxmzVZeafawTQlD3xwrWVjgDQoNqutGWlIwA0qHlz3q10hCbn+pV/XOkIC7X/uzdWOsLXKqtzef/998+kSZNy0kknZcUVV0xVVSOefA0AAAAAQIMrq7g8cuTIPPHEE1lvvfUaOA4AAAAAwJJR8kgH6ihr0MYqq6wy3ygMAAAAAACaj7KKyxdeeGFOOOGEvPXWWw0cBwAAAACApqCssRh77rlnPv3006y++upp165dWrduXef1GTNmNEg4AAAAAIDFpeBRcvVSVnH5wgsvbOAYAAAAAAA0JWUVl/fbb7+GzgEAAAAAQBNSVnF50qRJ//H1b3zjG2WFAQAAAABYUgqVDtDElVVc7tGjR6qqFj6QpKampuxAAAAAAAA0fmUVl59//vk6P8+dOzfPP/98hg8fnjPPPLNBggEAAAAA0HiVVVzu27fvfGsbbrhhVlpppfzmN7/JbrvtVu9gAAAAAACLk7EY9dOiIS/Wq1evPPvssw15SfiPDjh47zz7woi8XT0u9424JetvsO5/3L/TLttn5LP35u3qcXn0yb/ku9tuVef1iy4bluoPJ9Q5/nj7VYvzFgDq+MH+u+TOUTfnsTcezDV/vSxrr7fWQveuumaPDLvq1Nw56uY8/d6j2fPgH/7Ha+972N55+r1Hc9SphzV0bIAFem7si/nFL0/ONgP3Se/Nd8iIx5/82nOeGfNCdj/gsKz/7Z2ywx4H5q57Hppvzx9vvzvb/WC/bLDNwOx1yFF5cfwriyM+wH90ysnH5p9vj8lHH76eB+67OT17rvq15xz6s/3y+qtP5+NZE/PkyLuz0Ybr1Xn94IP2yYiHbs2MaRMyb867ad9++cWUHqBhlFVcnjVrVp3jww8/zIQJE3LiiSdmjTXWaOiMsEA777ZDTj3rhJx/zqXZdqvd8o+XXsnNd16dzp07LnD/hhuvnyuuOT83/eG2DNhy19x3z99y/U2XZK1v1f3/7IiHHk/vNbaoPX520DFL4nYAMmDgNjny5J/n6uHXZ7/tD8lr4yfmwpt+kw6dVljg/qXbtsm7k97PpWddmWnV0//jtb/Vt1d2/fFOee0fry+G5AAL9tlnn6dXz9Xy62N+vkj733lvcn5x3NBsvEHf3Hb9pdl3j11y8jkX5u+jRtfuue9vj+Xci6/MoQfuk1uvvTi9eq6anw4+MdM/mLmY7gJgfscd+/Mc9osD8/PDTshmW+yUTz79NPf+9f/Spk2bhZ6z++4Dc95vTs7pZwzPRpv8b8a9MD733vN/6dKlU+2edu3a5oEHH83Z51y8JG4DoN7KKi6vsMIK6dChQ+3RsWPHrL322nnqqady+eWXN3RGWKCf/WL/3Pj7W3Pz/92RV1+ZmOOOOjmfffp59tr3Bwvc/5ND980jfxuZy357bV579Y2cc+Zv8+K48TnwJ/vU2Tdn9pxMnTKt9vhw5qwlcTsA2esnu+fPN92Te265P2+99nbOOX54Pv/s8+y41/cWuP/lca/kktOvyN/+/HDmzpm70Ou2bdc2p15yYoYdd14++vDjxRUfYD5bbrpRjvjJfhmw9eaLtP9Pd92TlVfsnuMOPySr9/hG9v7hwGz77S1ywy131u654ZY788Oddsiu398uq6/6zQw97vAs3aZN7vzrg4vrNgDmc8ThB+esYRfl7rsfzIsvvpz9DzgyK63ULTvvvP1Czzn6yENy9TU35fc3/Ckvv/xafv6LE/Lpp5/lgP1/VLvntxdfnXN/c2lGjRqzJG4DSFKsarxHU1BWcfmRRx7Jww8/XHs8+uijGT9+fCZOnJhNN920oTPCfFq3bp0+662TJx798quVxWIxjz/6VDbcaL0FntNvo/Xy+KN1v4r5yIi/z7d/sy02zj9e/3v+/tx9OWf4yenQYYUGTg8wv1atW6VXn1559okvu/OKxWKefWJ01u23dr2ufexZR+bvI56uc22AxmjcSxPS/9++Ir75Jv0y7qWXk3zxIPHxr7yW/l/5/VuLFi3Sf8P1avcALG6rrvqNrLhit4x4eGTt2qxZH+WZZ55P/036LfCc1q1bZ4MN+mTEw0/UrhWLxYx4eGT691/wOQBNQVkP9Nt6663r9aazZ8/O7Nmz66wVi4VUVTXoCGj+i3Xs1CGtWrXK1Cl1vwY+deq0rLHmgudcde3WeYH7u3brXPvzIyOeyL13P5hJb7+bHquukiFDj84fb78y3xvwoxQKRrwDi88KHdunVauWmTF1Rp31D6Z9kB49v1H2dQfs/J30WnfNHPi9n9U3IsBiN23GB+nUsUOdtU4dVsjHn3yaz2fPzqxZH6empjD/no4d8uakd5ZkVKAZ696ta5KkunpqnfXqKdPSvXvXBZ7TuXPHtGrVKlOqp9VZnzJlatbqtfriCQqwBJRVXE6SiRMn5sILL8zLL3/RIbD22mvnyCOPzOqrf/2H4rBhw3LqqafWWWu3VKcsu3TnhZwBS8Zdt99b++uXx7+a8f94Jc+M+1s233LjPPHY0xVMBlC6rit1yeDTDssRPzo2c2bPqXQcAIAmaa+9ds3ll55T+/PAnQdVMA3Q0LQS1k9ZrcIPPPBA1l577TzzzDPp06dP+vTpk1GjRmWdddbJQw/N/zTnfzdkyJB8+OGHdY5l2iz4IWywIDOmf5B58+alS9dOdda7dOk8398E/8uU6mkl7U+St996J9OmzUiP1b5Z/9AA/8HMGR9m3ryadOxS97+HHTp3yPR/62ZeVGv16ZWOXTrm+geuyshJIzJy0ohssNl62eOg3TJy0oi0aOEbQ0Dj0rljh0yf8UGdtekfzMyyy7TL0m3apMMKy6dlyxbz75nxQTr/WzczQEO5++4H02+j7WqPadO/+L1Zt25d6uzr1rVzJk+essBrTJs2I/Pmzavzzdkk6dq1Syb/Wwc0QFNS1p8qTzjhhBx99NEZNWpUhg8fnuHDh2fUqFE56qijcvzxx3/t+W3atMnyyy9f5zASg1LMnTs3L4z9R7bc+ssZ31VVVdly6/557tmxCzxn9LNj6+xPkq232Wyh+5NkxZW6pWPHFTJlIb9BAGgo8+bOyysvvJKNttigdq2qqiobbdEvL44eX9Y1n3tidPbe5oAM2vbg2mP82Al54I6/ZdC2Bxv3AzQ6fXuvlVGjx9VZe+rZ59O397eSfDGzdO1ea2TUc2NrXy8UChk1emztHoCG9vHHn2TixLdqj/HjX83771fnO9tsUbtnueWWzcYbr5+nRy34GRdz587NmDEv1Dmnqqoq39lmizz9tOdiAE1XWRXdl19+OQcddNB86wceeGDGjy/vD8BQqisuvT777Ld79thrl6yx5mo594JT0m6Ztrn5xjuSJBdfcXZ+ffLg2v1XXv6HbDNgi/zssAPSc41Vc+wJh6Xv+uvk2iv/L0nSbpl2GXr6cem3Yd+s8o2Vs+XW/fP7P16WN9+YlEdGjFxgBoCG9Mcrb83AvXfM93bfPj16fiO/PPvoLN1u6dxz831JkqEXDcmhQw6p3d+qdaussU7PrLFOz7Rq3SpdVuycNdbpmf/psXKS5NNPPssbr7xZ5/j808/z4Qez8sYrb1bkHoHm5dNPP8uEVydmwqsTkyTvvledCa9OzPv//y/uL7j8ugw5/bza/Xvs8v288977Of/Sa/LG2//MzXf8NQ88/HgG7blr7Z5Be+6a2+6+P3++96FMfGtSTj/vknz2+ezs8v1tl+zNAc3aby++Or8ackR23HHb9O69Vq6/7qK89151/vznB2r3PHj/Lfn5ofvX/nzBRVfl4IP2zr777p611uqZSy85O8ss0zbX//6W2j3dunVJ377rZPXVeyRJ1u29Vvr2XceD5mExKjTioykoa+Zyly5dMnbs2Kyxxhp11seOHZuuXRc8vB4a2p/vuC+dOnXML391eLp265J/vPhy9trtkEyd+sVD+1b+n5VSKBRr9z/3zPM59OBjc8KJR+VXQ4/OmxPfyv57H5YJL7+WJCnU1GTtdXplz712yfLtl8vk96fmsUf+nnPOuChz5sytyD0Czcvf/vJIVui0Qg457oB06tIxr/3j9Ry9zy8zY9oXX//uvnK3FL/yudalW+f84aGra3/+8aE/yo8P/VHGPDk2P//hUUs6PsB8XprwWg48/MtvNp578ZVJkp13GJAzTzwm06bPyPvVX35D7H9W6p5Lf3Nazv3t73LjrXelW5fOOfX4o7L5Jv1q9+wwYOt8MPPDXHL1jZk2Y0bWWmP1XHH+6cZiAEvUb867LMss0y5XXHZuVlhh+fz978/m+zv9OLNnz67ds9pq30znzl+OPLv11r+kS+eOOWXosenevUvGjftHvr/jjzNlypejGn/6k30z9KRjan9+9JE7kyQHHnR0bvjDn5bAnQGUpqpYLBa/fltdp512Wi644IKccMIJ2WyzzZIkf//733POOedk8ODBOemkk0oO0q39WiWfA9CYrbpM90pHAGhQT7xwbaUjADSotittWekIAA1q3px3Kx2hyblklR9XOsJCHfbPGysd4WuV1bl80kknZbnllsv555+fIUOGJElWWmmlnHLKKTniiCMaNCAAAAAAwOJQctctdZRVXK6qqsrRRx+do48+Oh999FGSZLnllmvQYAAAAAAANF5lFZe/SlEZAAAAAKD5Kau4PH369AwdOjSPPPJIpkyZkkKh7vMLZ8yY0SDhAAAAAAAWl0JVpRM0bWUVl/fdd9+8/vrrOeigg9KtW7dUVfm3AAAAAADQnJRVXH7iiScycuTI9O3bt6HzAAAAAADQBJRVXF5rrbXy2WefNXQWAAAAAIAlpvD1W/gPWpRz0mWXXZZf//rXeeyxxzJ9+vTMmjWrzgEAAAAAwH+3sjqXV1hhhcyaNSvf+c536qwXi8VUVVWlpqamQcIBAAAAANA4lVVc3meffdK6devcdNNNHugHAAAAADRJxmLUT1nF5ZdeeinPP/98evXq1dB5AAAAAABoAsqaubzhhhvmn//8Z0NnAQAAAACgiSirc/nwww/PkUcemeOOOy7rrrtuWrduXef1Pn36NEg4AAAAAIDFpVjpAE1cWcXlPffcM0ly4IEHzveaB/oBAAAAAPz3K6u4/OabbzZ0DgAAAAAAmpCyisvf/OY3kyTjx4/PpEmTMmfOnNrXqqqqal8HAAAAAGisClWVTtC0lVVcfuONN7LrrrvmxRdfTFVVVYrFL6aTVFV98W/DWAwAAAAAgP9uLco56cgjj8yqq66aKVOmpF27dnnppZfy+OOPZ8MNN8yjjz7awBEBAAAAAGhsyupcfuqpp/Lwww+nc+fOadGiRVq2bJktttgiw4YNyxFHHJHnn3++oXMCAAAAADSoQqUDNHFldS7X1NRkueWWS5J07tw57733XpIvZjG/8sorDZcOAAAAAIBGqazO5d69e2fcuHFZddVVs8kmm+Tcc8/NUkstlSuvvDKrrbZaQ2cEAAAAAKCRKau4fOKJJ+aTTz5Jkpx22mnZcccds+WWW6ZTp0655ZZbGjQgAAAAAMDiUKx0gCaurOLy9ttvX/vrnj17ZsKECZkxY0Y6dOiQqqqqBgsHAAAAAEDjVFZxeUE6duzYUJcCAAAAAKCRa7DiMgAAAABAU1IwGKNeWlQ6AAAAAAAATY/iMgAAAAAAJTMWAwAAAABolgqVDtDE6VwGAAAAAKBkissAAAAAAJTMWAwAAAAAoFkqVjpAE6dzGQAAAACAkikuAwAAAABQMmMxAAAAAIBmqVDpAE2czmUAAAAAAEqmuAwAAAAAQMmMxQAAAAAAmqVCVaUTNG06lwEAAAAAKJniMgAAAAAAJVNcBgAAAACgZGYuAwAAAADNUiHFSkdo0nQuAwAAAABQMsVlAAAAAABKZiwGAAAAANAsGYpRPzqXAQAAAAAomeIyAAAAAAAlMxYDAAAAAGiWCpUO0MTpXAYAAAAAoGSKywAAAAAAlMxYDAAAAACgWSqkWOkITZrOZQAAAAAASqa4DAAAAABAyYzFAAAAAACaJUMx6kfnMgAAAAAAJVNcBgAAAACgZMZiAAAAAADNUqHSAZo4ncsAAAAAAJRMcRkAAAAAgJIZiwEAAAAANEuFFCsdoUnTuQwAAAAAQMkUlwEAAAAAKJmxGAAAAABAs2QoRv3oXAYAAAAAoGSKywAAAAAAlMxYDAAAAACgWSpUOkATp3MZAAAAAICSKS4DAAAAAFAyYzEAAAAAgGapmGKlIzRpOpcBAAAAACiZ4jIAAAAAACUzFgMAAAAAaJYKlQ7QxOlcBgAAAACgZIrLAAAAAACUzFgMAAAAAKBZKqRY6QhNms5lAAAAAABKprgMAAAAAEDJjMUAAAAAAJolQzHqR+cyAAAAAAAlU1wGAAAAAKBkxmIAAAAAAM1SwWCMetG5DAAAAABAyRSXAQAAAAAombEYAAAAAECzVKh0gCZO5zIAAAAAACVTXAYAAAAAoGTGYgAAAAAAzVIxxUpHaNJ0LgMAAAAAUDLFZQAAAAAASmYsBgAAAADQLBUqHaCJ07kMAAAAAEDJFJcBAAAAAChZoxmL8XnN3EpHAGhQz8+YWOkIAA2q7UpbVjoCQIP67L0nKh0BgAorpljpCE2azmUAAAAAAEqmuAwAAAAAQMkazVgMAAAAAIAlqVDpAE2czmUAAAAAAEqmuAwAAAAAQMmMxQAAAAAAmqVCsVjpCE2azmUAAAAAAEqmuAwAAAAAQMmMxQAAAAAAmiVDMepH5zIAAAAAACVTXAYAAAAAoGTGYgAAAAAAzVLBYIx60bkMAAAAAEDJFJcBAAAAACiZsRgAAAAAQLNUNBajXnQuAwAAAABQMsVlAAAAAABKZiwGAAAAANAsFSodoInTuQwAAAAAQMkUlwEAAAAAKJmxGAAAAABAs1RIsdIRmjSdywAAAAAAlExxGQAAAACAkhmLAQAAAAA0S0VjMepF5zIAAAAAACVTXAYAAAAAoGTGYgAAAAAAzVKh0gGaOJ3LAAAAAACUTHEZAAAAAKCJu/TSS9OjR48svfTS2WSTTfLMM88sdO9VV12VLbfcMh06dEiHDh0yYMCA/7h/YRSXAQAAAIBmqVgsNtqjFLfccksGDx6ck08+OWPGjEnfvn2z/fbbZ8qUKQvc/+ijj2avvfbKI488kqeeeiqrrLJKtttuu7z77rslvW9VsdSki0n7ZVevdASABvX5vDmVjgDQoGoKJtIB/10+e++JSkcAaFCtO69W6QhNzq7f2KnSERbqzkl3L/LeTTbZJBtttFEuueSSJEmhUMgqq6ySww8/PCeccMLXnl9TU5MOHTrkkksuyaBBgxb5fXUuAwAAAAA0MrNnz86sWbPqHLNnz55v35w5czJ69OgMGDCgdq1FixYZMGBAnnrqqUV6r08//TRz585Nx44dS8qouAwAAAAANEuFFBvtMWzYsLRv377OMWzYsPnuYdq0aampqUm3bt3qrHfr1i2TJ09epH8Oxx9/fFZaaaU6BepF0aqk3QAAAAAALHZDhgzJ4MGD66y1adOmwd/n7LPPzs0335xHH300Sy+9dEnnKi4DAAAAADQybdq0WaRicufOndOyZctUV1fXWa+urk737t3/47nnnXdezj777Pztb39Lnz59Ss5oLAYAAAAA0CwVGvGxqJZaaqn069cvI0aM+PK+CoWMGDEim2666ULPO/fcc3P66afn/vvvz4YbbljCO35J5zIAAAAAQBM2ePDg7Lffftlwww2z8cYb58ILL8wnn3ySAw44IEkyaNCgrLzyyrUzm88555wMHTo0N910U3r06FE7m3nZZZfNsssuu8jvq7gMAAAAANCE7bnnnpk6dWqGDh2ayZMnZ7311sv9999f+5C/SZMmpUWLL4dYXH755ZkzZ05++MMf1rnOySefnFNOOWWR37eqWCwWG+QO6qn9sqtXOgJAg/p83pxKRwBoUDWFUr6cB9D4ffbeE5WOANCgWnderdIRmpwdv/H9SkdYqL9OuqfSEb6WmcsAAAAAAJRMcRkAAAAAgJKZuQwAAAAANEuFNIqJwU2WzmUAAAAAAEqmuAwAAAAAQMmMxQAAAAAAmqVi0ViM+tC5DAAAAABAyRSXAQAAAAAombEYAAAAAECzVKh0gCZO5zIAAAAAACVTXAYAAAAAoGTGYgAAAAAAzVIxxUpHaNJ0LgMAAAAAUDLFZQAAAAAASmYsBgAAAADQLBWMxagXncsAAAAAAJRMcRkAAAAAgJIZiwEAAAAANEvForEY9aFzGQAAAACAkikuAwAAAABQMmMxAAAAAIBmqRBjMepD5zIAAAAAACVTXAYAAAAAoGTGYgAAAAAAzVLRWIx60bkMAAAAAEDJFJcBAAAAACiZsRgAAAAAQLNUKBqLUR86lwEAAAAAKJniMgAAAAAAJTMWAwAAAABolgzFqB+dywAAAAAAlExxGQAAAACAkhmLAQAAAAA0SwWDMeql5M7lmpqaPP7445k5c+ZiiAMAAAAAQFNQcnG5ZcuW2W677fLBBx8sjjwAAAAAADQBZY3F6N27d954442suuqqDZ0HAAAAAGCJMBajfsp6oN8ZZ5yRY489Nn/961/z/vvvZ9asWXUOAAAAAAD+u5XVufy9730vSTJw4MBUVVXVrheLxVRVVaWmpqZh0gEAAAAA0CiVVVx+5JFHGjoHAAAAAMASVSwai1EfZRWXt95664bOAQAAAABAE1LWzOUkeeKJJ/LjH/84m222Wd59990kyR/+8IeMHDmywcIBAAAAANA4lVVcvv3227P99tunbdu2GTNmTGbPnp0k+fDDD3PWWWc1aEAAAAAAgMWhkGKjPZqCsorLZ5xxRq644opcddVVad26de365ptvnjFjxjRYOAAAAAAAGqeyisuvvPJKttpqq/nW27dvn5kzZ9Y3EwAAAAAAjVxZxeXu3bvn9ddfn2995MiRWW211eodCgAAAABgcSs24v81BWUVlw855JAceeSRGTVqVKqqqvLee+/l//7v/3Lsscfm0EMPbeiMAAAAAAA0Mq3KOemEE05IoVDId7/73Xz66afZaqut0qZNmxx77LE5/PDDGzojAAAAAACNTFWxWCy7x3rOnDl5/fXX8/HHH2fttdfOsssuW3aQ9suuXva5AI3R5/PmVDoCQIOqKRQqHQGgQX323hOVjgDQoFp3Nq62VBuuuGWlIyzUc+83/v9OldW5/C9LLbVU1l577YbKAgAAAABAE7HIxeXddtttkS96xx13lBUGAAAAAICmYZEf6Ne+ffvaY/nll8+IESPy3HPP1b4+evTojBgxIu3bt18sQWFBDv7Jj/PCPx5L9bTxGfHI7dmgX5//uH+XXXfIs2MeTPW08Xly1L3Zdrtv13n9hF8dkWfHPJj3ql/M2/8ckz/ffUP6bdh3Md4BQF0/++l+eeWVJ/PhzNfyxON/yYYbrvcf9++22/fzwrhH8uHM1zL6uYfyv9tvU+f1nXf+39zz1//Le+++kNmf/zN9+vjGEVAZp5x8bP759ph89OHreeC+m9Oz56pfe86hP9svr7/6dD6eNTFPjrw7G/3bZ+LBB+2TEQ/dmhnTJmTenHfTvv3yiyk9wBeeG/tifvHLk7PNwH3Se/MdMuLxJ7/2nGfGvJDdDzgs6397p+ywx4G5656H5tvzx9vvznY/2C8bbDMwex1yVF4c/8riiA8sQCHFRns0BYtcXL7uuutqj27dumWPPfbIm2++mTvuuCN33HFH3njjjfzoRz9K586dF2deqLXbD76fs4b9KucM+2222mJgXnppQu686/p07tJpgfs33mSDXHPdhfnD72/NlpvvlHv++lBuuvnyfGvtNWv3vP7amzlu8CnZbJPvZfvt9sykt9/JnX/+fTp17riE7gpozn74w51y7rkn5cwzL8wm/b+XF18cn7/e/Yd0WcjnWv/+/fKHGy7J9dffnE022SF/ufuB3Hrr1Vl77V61e5ZZpl3+/uQz+fWJZy2p2wCYz3HH/jyH/eLA/PywE7LZFjvlk08/zb1//b+0adNmoefsvvvAnPebk3P6GcOz0Sb/m3EvjM+99/xfnc/Edu3a5oEHH83Z51y8JG4DIJ999nl69Vwtvz7m54u0/533JucXxw3Nxhv0zW3XX5p999glJ59zYf4+anTtnvv+9ljOvfjKHHrgPrn12ovTq+eq+engEzP9g5mL6S4AGk5ZD/Tr0qVLRo4cmV69etVZf+WVV7LZZptl+vTpJQfxQD9KNeKR2zNmzAs57phTkyRVVVUZ/8rIXHnFDblg+O/m23/d73+bdu3aZs/dD6ld+9vDt+XFF1/O0UeetMD3WG65ZfPO++MycMd989ijX/830vBVHuhHqZ54/C8ZPXpcjjr6i8+kqqqqTHz9mVx2+XU577zL5tt/4x8uyzLLtM2uux1Qu/b4Y3/OCy/8I4cd/qs6e7/5zf/Jq688lY023j4vvDB+8d4I/7U80I9y/fPtMbngwt9l+AVf/B5t+eWXy3vvjM2BBx+dP/3pLws858mRd+fZ58blyKNOTPLFZ+JbbzybSy+7Luf+5tI6e7featOM+Ntt6dTlW/nww1mL92b4r+KBftRH7813yEXDTsp3t9psoXuGX3ZNHn/y2dx14xW1a8cOHZaPPv4kvxt+RpJkr0OOSu+11qwtWBcKhQzYdVD2/uHAHLzvHov3Jviv44F+pdtgxS0qHWGhxrw/stIRvtYidy5/1bx58zJhwoT51idMmJCCP3SwBLRu3Trrrd87jz7yZcG3WCzm0UeezEYbr7/AczbaeP08+sjf66yNGPHEQve3bt06+x/wo8ycOSsvvvhyw4UHWIDWrVtngw3WzcMPf/mbh2KxmIcfeSL9N+m3wHM26b9Bnf1J8tDfHssmC9kPUAmrrvqNrLhit4z4yufVrFkf5Zlnnl/o59sXn4l9MuLhLwt/xWIxIx4emf79fcYBTce4lyak/7+N9Nl8k34Z99IXf8acO3duxr/yWvpv9OWeFi1apP+G69XuARavYrHYaI+mYJEf6PdVBxxwQA466KBMnDgxG2+8cZJk1KhROfvss3PAAQd8zdlQf506dUirVq0yZcq0OutTp0zLmmsu+G/punXrnClTp8+3v1u3LnXWtv/fbXLt9RelXbu2mTx5SnYdOCgzpn/QsDcA8G86d+6YVq1apXrK1DrrU6qnpdeaPRd4TvduXVL9b5+DU6rn/1wDqKTu3bomSaqr636+VU+Zlu7duy7wnH99Jk6p/rfPuClTs1Yv33gEmo5pMz5Ip44d6qx16rBCPv7k03w+e3Zmzfo4NTWF+fd07JA3J72zJKMClKWs4vJ5552X7t275/zzz8/777+fJFlxxRVz3HHH5Zhjjvna82fPnp3Zs2fXWSsWi6mqqionDjSoJx5/OltutlM6duqQ/fffM9ffcHG+s80PMm1q6eNeAACam7322jWXX3pO7c8Ddx5UwTQAACxOZRWXW7RokV/+8pf55S9/mVmzvphptvzyi/5k5mHDhuXUU0+ts7ZU6xWy9FIemsaimT79g8ybNy9du9Z9gGSXrp3n64r5l+rqaen6bw/FWtD+Tz/9LG+88XbeeOPtPPfs2IwZOyKDBu2e4edfEYDFZdq0GZk3b166da3bddy128I/1yZXT023f/sc/E/7AZaEu+9+MM8883ztz23aLJUk6datSyZPnlK73q1r54wd948FXuNfn4ldu/3bZ1zXLpnsMw5oQjp37JDpM+p+E3b6BzOz7DLtsnSbNmm5Qou0bNli/j0zPkjnf+tmBhaPQprG+InGqqyZy1+1/PLLl1RYTpIhQ4bkww8/rHO0ae1Dk0U3d+7cjH3+pWz97S8fnFBVVZWtv71pnv3KH2a+6tlnnq+zP0m22WaLhe7/lxYtqrLU//9DEcDiMnfu3IwZ82K22Wbz2rWqqqps8+0t8vRXnib+VaOeHlNnf5J89ztbZtRC9gMsCR9//EkmTnyr9hg//tW8/351vrPNlw/LWW65ZbPxxusv9PPti8/EF+qcU1VVle9ss0WeftpnHNB09O29VkaNHldn7alnn0/f3t9K8sWM+bV7rZFRz42tfb1QKGTU6LG1ewAas7KKy9XV1dl3332z0korpVWrVmnZsmWd4+u0adOmtij9r8NIDEp16SXXZr/998xee++WNXutngsuOj3LtGuXG2+8LUlyxZXn5eRTjq3df/ll12fAtlvlsMMPyhprrpYTfnVE1t+gd6783R+SJO3atc3Qk4/Jhhutl1VWWSnrrdc7l1x2dlZcqXvuuvO+itwj0Lxc9NurcuCBe+XHP/5h1urVM5dcfFaWWaZtbrjhT0mSa665IKeffnzt/ksuvSbbbfftHHXkT9JrzdVz4olHp1+/Prns8t/X7unQYYX06bN2vrXWGkmSNddcPX36rG0uM7BE/fbiq/OrIUdkxx23Te/ea+X66y7Ke+9V589/fqB2z4P335KfH7p/7c8XXHRVDj5o7+y77+5Za62eufSSs7PMMm1z/e9vqd3TrVuX9O27TlZfvUeSZN3ea6Vv33XSocMKS+jOgObm008/y4RXJ2bCqxOTJO++V50Jr07M+///mxkXXH5dhpx+Xu3+PXb5ft557/2cf+k1eePtf+bmO/6aBx5+PIP23LV2z6A9d81td9+fP9/7UCa+NSmnn3dJPvt8dnb5/rZL9uYAylDWWIz9998/kyZNykknnZQVV1xRYZiKuOP2e9Kpc8f86sSj0q1b57z4wsvZbdcDMnXKF7OR/2eVFVMoFGr3PzNqTA4+8OiceNLgDD3lmEyc+Hb2/tGheXn8q0mSmpqarNlr9ey1z27p1KlDZsyYmTGjX8gO2+2ZCS+/VpF7BJqX2267O106d8zQoceke7cuGTdufHYauG/tw0tXWWXlFApffmXr6adHZ9B+h+fUU47Laaf9Mq+//lZ23/3gjB//Su2eHXfcNldfNbz25/+78bIkyelnDM8ZZ1ywhO4MaO5+c95lWWaZdrnisnOzwgrL5+9/fzbf3+nHdZ7Dstpq30znzl+Oybv11r+kS+eOOWXosenevUvGjftHvr/jj+s80PmnP9k3Q0/68pkvjz5yZ5LkwIOOzg1/+NMSuDOguXlpwms58PAv/7L/3IuvTJLsvMOAnHniMZk2fUber/5yBND/rNQ9l/7mtJz729/lxlvvSrcunXPq8Udl80361e7ZYcDW+WDmh7nk6hszbcaMrLXG6rni/NONxYAlpGgsRr1UFYvFkv8JLrfccnniiSfy/9q787Cq6jyO45/LIqgsKiqikrixuOACqLiRSkmTjmiL2iKkLVPjkoilk4qPVjiNmE5miz1hzVgulY2TaUOUjpKmgqaZgplmKoTmCqYo98wfPt7xJiYHqMvyfvXc5/H8zu+c8733j1/nfvme7+3cuXOFBeLtwa8+A6heLlwucnQIAFChiq/5oy0AVAc/H9vo6BAAoEK5Nmzl6BCqnNAmkY4O4YZ25W12dAg3Vaa2GP7+/ipDThoAAAAAAAAAUE2UKbk8f/58TZkyRYcOHargcAAAAAAAAADg92E1jEr7qgrK1HN5+PDhOn/+vFq3bq06derI1dXVbv/JkycrJDgAAAAAAAAAQOVUpuTy/PnzKzgMAAAAAAAAAEBVUqbkclxcXEXHAQAAAAAAAAC/K0NVo/1EZVWmnsuSdODAAU2bNk0jR45Ufn6+JGnt2rXas2dPhQUHAAAAAAAAAKicypRc3rBhgzp27Kgvv/xSH3zwgQoKCiRJX331lZKSkio0QAAAAAAAAABA5VOm5PKUKVP07LPPKi0tTbVq1bKN9+/fX1u2bKmw4AAAAAAAAADgt2I1jEr7qgrKlFzevXu3hg4det1448aNdeLEiXIHBQAAAAAAAACo3MqUXK5Xr55yc3OvG9+xY4eaNWtW7qAAAAAAAAAAAJVbmZLLI0aM0NNPP628vDxZLBZZrVZlZGQoMTFRo0aNqugYAQAAAAAAAKDCGZX4v6qgTMnl559/XsHBwfL391dBQYHatWunPn36qGfPnpo2bVpFxwgAAAAAAAAAqGQshlH27tA//PCDdu/erYKCAnXp0kVt27YtcyDeHq3LfCwAVEYXLhc5OgQAqFDFVqujQwCACvXzsY2ODgEAKpRrw1aODqHKCW4c4egQbmhf/jZHh3BTLmU5KCEh4bqxLVu2yGKxyN3dXW3atNGQIUPUoEGDcgcIAAAAAAAAAL8Fa9nrbqEyVi7369dPWVlZKi4uVlBQkCQpJydHzs7OCg4OVnZ2tiwWizZt2qR27dqV6pxULgOobqhcBlDdULkMoLqhchlAdUPlsnmBjcIdHcIN5Rzf7ugQbqpMPZeHDBmi6OhoHTt2TJmZmcrMzNSRI0d02223aeTIkTp69Kj69u2riRMnVnS8AAAAAAAAAIBKoEyVy82aNVNaWtp1Vcl79uzR7bffrqNHjyorK0u33367Tpw4UapzUrkMoLqhchlAdUPlMoDqhsplANUNlcvmtW0U5ugQbmj/8UxHh3BTZapcPnPmjPLz868bP378uM6ePStJqlevnoqKSKwAAAAAAAAAQHVU5rYYo0eP1qpVq3TkyBEdOXJEq1at0pgxYxQbGytJ2rp1qwIDAysyVgAAAAAAAABAJeFSloNee+01TZw4USNGjNDly5evnMjFRXFxcXrxxRclScHBwXrjjTcqLlIAAAAAAAAAqEBW8x2DcY0y9Vy+qqCgQN99950kqVWrVvLw8ChzIPRcBlDd0HMZQHVDz2UA1Q09lwFUN/RcNq91w66ODuGGDpzIcnQIN1WmyuWrPDw8FBoaWlGxAAAAAAAAAACqiHIllwEAAAAAAACgqjJEW4zyKNMP+gEAAAAAAAAAajaSywAAAAAAAAAA02iLAQAAAAAAAKBGMgx+tLo8qFwGAAAAAAAAAJhGchkAAAAAAAAAYBptMQAAAAAAAADUSFYZjg6hSqNyGQAAAAAAAABgGsllAAAAAAAAAIBptMUAAAAAAAAAUCMZBm0xyoPKZQAAAAAAAACAaSSXAQAAAAAAAACm0RYDAAAAAAAAQI1kFW0xyoPKZQAAAAAAAACAaSSXAQAAAAAAAACm0RYDAAAAAAAAQI1kGLTFKA8qlwEAAAAAAAAAppFcBgAAAAAAAACYRlsMAAAAAAAAADWSlbYY5ULlMgAAAAAAAADANJLLAAAAAAAAAADTaIsBAAAAAAAAoEYyRFuM8qByGQAAAAAAAABgGsllAAAAAAAAAIBptMUAAAAAAAAAUCMZBm0xyoPKZQAAAAAAAACAaSSXAQAAAAAAAACm0RYDAAAAAAAAQI1kFW0xyoPKZQAAAAAAAACAaSSXAQAAAAAAAACm0RYDAAAAAAAAQI1kGLTFKA8qlwEAAAAAAAAAppFcBgAAAAAAAACYRlsMAAAAAAAAADWSlbYY5ULlMgAAAAAAAADANJLLAAAAAAAAAADTaIsBAAAAAAAAoEYyaItRLlQuAwAAAAAAAABMI7kMAAAAAAAAADCNthgAAAAAAAAAaiSraItRHlQuAwAAAAAAAABMI7kMAAAAAAAAADCNthgAAAAAAAAAaiTDoC1GeVC5DAAAAAAAAAAwjeQyAAAAAAAAAMA02mIAAAAAAAAAqJGstMUoFyqXAQAAAAAAAACmkVwGAAAAAAAAAJhGWwwAAAAAAAAANZIh2mKUB5XLAAAAAAAAAADTSC4DAAAAAAAAAEyjLQYAAAAAAACAGslq0BajPKhcBgAAAAAAAACYRnIZAAAAAAAAAGAabTEAAAAAAAAA1EgGbTHKhcplAAAAAAAAAIBpJJcBAAAAAAAAAKbRFgMAAAAAAABAjWSIthjlQeUyAAAAAAAAAMA0kssAAAAAAAAAUMW9/PLLCggIkLu7u7p3766tW7f+6vyVK1cqODhY7u7u6tixoz7++GPT1yS5DAAAAAAAAKBGMgyj0r7MWL58uRISEpSUlKSsrCx16tRJAwcOVH5+fonzv/jiC40cOVJjxozRjh07FBsbq9jYWH399demrmsxzEb6G/H2aO3oEACgQl24XOToEACgQhVbrY4OAQAq1M/HNjo6BACoUK4NWzk6hCqnlltzR4dwQ0UXj5R6bvfu3RUREaGFCxdKkqxWq/z9/TVu3DhNmTLluvnDhw9XYWGhPvroI9tYjx491LlzZ7366qulvi6VywAAAAAAAABQyVy8eFFnz561e128ePG6eUVFRcrMzFR0dLRtzMnJSdHR0dq8eXOJ5968ebPdfEkaOHDgDeffCMllAAAAAAAAADWSo1tf/NorOTlZ3t7edq/k5OTr3sOJEydUXFwsX19fu3FfX1/l5eWV+L7z8vJMzb8RF1OzAQAAAAAAAAC/ualTpyohIcFuzM3NzUHRlIzkMgAAAAAAAABUMm5ubqVKJjds2FDOzs768ccf7cZ//PFHNWnSpMRjmjRpYmr+jdAWAwAAAAAAAECNZFTiV2nVqlVLYWFhSk9Pt41ZrValp6crMjKyxGMiIyPt5ktSWlraDeffCJXLAAAAAAAAAFCFJSQkKC4uTuHh4erWrZvmz5+vwsJCPfTQQ5KkUaNGqVmzZraezRMmTFBUVJRSUlJ05513atmyZdq+fbtef/11U9cluQwAAAAAAAAAVdjw4cN1/PhxzZgxQ3l5eercubPWrVtn+9G+w4cPy8np/00sevbsqXfeeUfTpk3TX/7yF7Vt21YffvihOnToYOq6FsMwzFRZA1XaxYsXlZycrKlTp1a6BugAUBasawCqG9Y1ANUN6xqA6ozkMmqUs2fPytvbW2fOnJGXl5ejwwGAcmNdA1DdsK4BqG5Y1wBUZ/ygHwAAAAAAAADANJLLAAAAAAAAAADTSC4DAAAAAAAAAEwjuYwaxc3NTUlJSfyIAoBqg3UNQHXDugagumFdA1Cd8YN+AAAAAAAAAADTqFwGAAAAAAAAAJhGchkAAAAAAAAAYBrJZQAAAAAAAACAaSSXgQoSEBCg+fPnOzoMAAAAAEAVEh8fr9jYWEeHAQBlQnIZAACTbr31Vj355JOODuOmLBaLPvzwQ0eHAQCVBgkcAACAikVyGQCACmYYhi5fvuzoMAAAAFADcO8JwJFILuN3c+utt2rcuHF68sknVb9+ffn6+mrx4sUqLCzUQw89JE9PT7Vp00Zr1661HbNnzx4NGjRIXl5e8vT0VJ8+fXTgwAHb/jfffFPt27eXm5ub/Pz8NHbs2FLFcvr0aT322GPy9fWVu7u7OnTooI8++si2//3337edNyAgQCkpKXbH5+fna/Dgwapdu7ZatmyppUuXlniNhx9+WI0aNZKXl5f69++vr776yuzHBqCSiY+P14YNG7RgwQJZLBZZLBYtWbJEFotFa9euVVhYmNzc3LRp0yYdOHBAQ4YMka+vrzw8PBQREaFPP/3U7nwBAQF6/vnnNXr0aHl6euqWW27R66+/bttfVFSksWPHys/PT+7u7mrRooWSk5NvGmdAQIAkaejQobJYLAoICNChQ4fk5OSk7du3282dP3++WrRoIavVqvXr18tisWjNmjUKDQ2Vu7u7evTooa+//trumE2bNqlPnz6qXbu2/P39NX78eBUWFpbxUwVQlaxbt069e/dWvXr15OPjo0GDBtnuzw4dOiSLxaIVK1bY1oiIiAjl5ORo27ZtCg8Pl4eHh+644w4dP37cdk6r1apZs2apefPmcnNzU+fOnbVu3Trb/qtr0+nTp21jO3fulMVi0aFDhyRJS5YsUb169fTJJ58oJCREHh4eiomJUW5uriRp5syZeuutt/Svf/3Ltn6vX7/+N/+8AFQd586d0/3336+6devKz89PL774ot0TaxcvXlRiYqKaNWumunXrqnv37nbryM3WIUkqLi5WQkKCbQ196qmnZBiGXRxWq1XJyclq2bKlateurU6dOum9996z7b+6Jv7y3hMAHIHkMn5Xb731lho2bKitW7dq3Lhxevzxx3XPPfeoZ8+eysrK0u23364HH3xQ58+f19GjR9W3b1+5ubnps88+U2ZmpkaPHm37i+wrr7yiP//5z3r00Ue1e/durV69Wm3atLlpDFarVXfccYcyMjL0z3/+U998843mzJkjZ2dnSVJmZqbuvfdejRgxQrt379bMmTM1ffp0LVmyxHaO+Ph4/fDDD/r888/13nvvadGiRcrPz7e7zj333KP8/HytXbtWmZmZ6tq1qwYMGKCTJ09W3AcK4He3YMECRUZG6pFHHlFubq5yc3Pl7+8vSZoyZYrmzJmjvXv3KjQ0VAUFBfrDH/6g9PR07dixQzExMRo8eLAOHz5sd86UlBSFh4drx44deuKJJ/T4448rOztbkvT3v/9dq1ev1ooVK5Sdna2lS5faEse/Ztu2bZKk1NRU5ebmatu2bQoICFB0dLRSU1Pt5qampio+Pl5OTv+/LZg8ebJSUlK0bds2NWrUSIMHD9alS5ckSQcOHFBMTIzuuusu7dq1S8uXL9emTZtK/Qc+AFVbYWGhEhIStH37dqWnp8vJyUlDhw6V1Wq1zUlKStK0adOUlZUlFxcX3XfffXrqqae0YMECbdy4Ud9++61mzJhhm79gwQKlpKRo7ty52rVrlwYOHKg//vGP2r9/v6nYzp8/r7lz5+of//iH/vvf/+rw4cNKTEyUJCUmJuree++1JXpyc3PVs2fPivlQAFQLCQkJysjI0OrVq5WWlqaNGzcqKyvLtn/s2LHavHmzli1bpl27dumee+5RTEyM3Vr1a+uQdOW+b8mSJXrzzTe1adMmnTx5UqtWrbKLIzk5WW+//bZeffVV7dmzRxMnTtQDDzygDRs22M375b0nADiEAfxOoqKijN69e9u2L1++bNStW9d48MEHbWO5ubmGJGPz5s3G1KlTjZYtWxpFRUUlnq9p06bGM888YzqOTz75xHBycjKys7NL3H/fffcZt912m93Y5MmTjXbt2hmGYRjZ2dmGJGPr1q22/Xv37jUkGS+++KJhGIaxceNGw8vLy7hw4YLdeVq3bm289tprpmMGULlERUUZEyZMsG1//vnnhiTjww8/vOmx7du3N1566SXbdosWLYwHHnjAtm21Wo3GjRsbr7zyimEYhjFu3Dijf//+htVqNR2nJGPVqlV2Y8uXLzfq169vW58yMzMNi8ViHDx40O69LFu2zHbMTz/9ZNSuXdtYvny5YRiGMWbMGOPRRx+1O+/GjRsNJycn4+effzYdJ4Cq7fjx44YkY/fu3cbBgwcNScYbb7xh2//uu+8akoz09HTbWHJyshEUFGTbbtq0qfHcc8/ZnTciIsJ44oknDMP4/9p06tQp2/4dO3YYkmzrV2pqqiHJ+Pbbb21zXn75ZcPX19e2HRcXZwwZMqQi3jaAaubs2bOGq6ursXLlStvY6dOnjTp16hgTJkwwvv/+e8PZ2dk4evSo3XEDBgwwpk6dahhG6dYhPz8/44UXXrBtX7p0yWjevLltbbpw4YJRp04d44svvrC7zpgxY4yRI0cahmHu3hMAfmtULuN3de1fU52dneXj46OOHTvaxnx9fSVdaTuxc+dO9enTR66urtedJz8/X8eOHdOAAQNMx7Bz5041b95cgYGBJe7fu3evevXqZTfWq1cv7d+/X8XFxdq7d69cXFwUFhZm2x8cHKx69erZtr/66isVFBTIx8dHHh4ettfBgwft2noAqF7Cw8PttgsKCpSYmKiQkBDVq1dPHh4e2rt373WVy9eujRaLRU2aNLE9DREfH6+dO3cqKChI48eP13/+859yxRgbGytnZ2dbhcySJUvUr1+/66qhIyMjbf9u0KCBgoKCtHfvXklX1rglS5bYrW8DBw6U1WrVwYMHyxUfgMpv//79GjlypFq1aiUvLy/b+nHt2nbtunb1/u6X93xX17mzZ8/q2LFjJd5/XV13SqtOnTpq3bq1bdvPz++6p8sAoCTfffedLl26pG7dutnGvL29FRQUJEnavXu3iouLFRgYaHcPtGHDBrvveL+2Dp05c0a5ubnq3r27bb+Li4vdPeS3336r8+fP67bbbrO7zttvv33dd8lf3nsCgCO4ODoA1Cy/TBRbLBa7MYvFIulK64ratWvf8Dy/tu9mynNsaRUUFMjPz6/EPn7XJqEBVC9169a1205MTFRaWprmzp2rNm3aqHbt2rr77rtVVFRkN6+ktfHq4+Vdu3bVwYMHtXbtWn366ae69957FR0dbdd3z4xatWpp1KhRSk1N1bBhw/TOO+9owYIFps5RUFCgxx57TOPHj79u3y233FKmuABUHYMHD1aLFi20ePFiNW3aVFarVR06dLBb20q6v/vl2LVtNG7matse45q+pFdb9VyrpPXU+EUvUwAoi4KCAjk7OyszM9PWUvEqDw8P27/Luw4VFBRIktasWaNmzZrZ7XNzc7Pb/uW9JwA4AsllVFqhoaF66623dOnSpev+B+3p6amAgAClp6erX79+ps975MgR5eTklFi9HBISooyMDLuxjIwMBQYGytnZWcHBwbp8+bIyMzMVEREhScrOzrb7gZmuXbsqLy9PLi4upeqNCqBqqVWrloqLi286LyMjQ/Hx8Ro6dKikK18Wrv7wlBleXl4aPny4hg8frrvvvlsxMTE6efKkGjRo8KvHubq6lhjnww8/rA4dOmjRokW6fPmyhg0bdt2cLVu22BLFp06dUk5OjkJCQiRdWeO++eabUvW5B1C9/PTTT8rOztbixYvVp08fSSr3j0h5eXmpadOmysjIUFRUlG08IyPDVkHYqFEjSVJubq7q168v6crTaGaVdv0GUPO0atVKrq6u2rZtm+0e6MyZM8rJyVHfvn3VpUsXFRcXKz8/37b+meXt7S0/Pz99+eWX6tu3ryTZvlt27dpVktSuXTu5ubnp8OHDdmsiAFRWJJdRaY0dO1YvvfSSRowYoalTp8rb21tbtmxRt27dFBQUpJkzZ+pPf/qTGjdurDvuuEPnzp1TRkaGxo0b96vnjYqKUt++fXXXXXdp3rx5atOmjfbt2yeLxaKYmBhNmjRJERERmj17toYPH67Nmzdr4cKFWrRokSQpKChIMTExeuyxx/TKK6/IxcVFTz75pF1FdHR0tCIjIxUbG6sXXnhBgYGBOnbsmNasWaOhQ4fy+BJQxQUEBOjLL7/UoUOH5OHhccPqu7Zt2+qDDz7Q4MGDZbFYNH36dFOVepI0b948+fn5qUuXLnJyctLKlSvVpEmTUj0FcfWPcL169ZKbm5stIRMSEqIePXro6aef1ujRo0t8omPWrFny8fGRr6+vnnnmGTVs2FCxsbGSpKefflo9evTQ2LFj9fDDD6tu3br65ptvlJaWpoULF5p6fwCqlvr168vHx0evv/66/Pz8dPjwYU2ZMqXc5508ebKSkpLUunVrde7cWampqdq5c6eWLl0qSWrTpo38/f01c+ZMPffcc8rJyVFKSorp6wQEBOiTTz5Rdna2fHx85O3tXWILNgA1j6enp+Li4jR58mQ1aNBAjRs3VlJSkpycnGSxWBQYGKj7779fo0aNUkpKirp06aLjx48rPT1doaGhuvPOO0t1nQkTJmjOnDlq27atgoODNW/ePLtCJU9PTyUmJmrixImyWq3q3bu3zpw5o4yMDHl5eSkuLu43+gQAoGzouYxKy8fHR5999pkKCgoUFRWlsLAwLV682PYFIC4uTvPnz9eiRYvUvn17DRo0qNS/KP7+++8rIiJCI0eOVLt27fTUU0/Zqli6du2qFStWaNmyZerQoYNmzJihWbNmKT4+3nZ8amqqmjZtqqioKA0bNkyPPvqoGjdubNtvsVj08ccfq2/fvnrooYcUGBioESNG6Pvvv7f1HQRQdSUmJsrZ2Vnt2rVTo0aNruuhfNW8efNUv3599ezZU4MHD9bAgQNtVSml5enpqRdeeEHh4eGKiIjQoUOH9PHHH9seEf81KSkpSktLk7+/v7p06WK3b8yYMSoqKtLo0aNLPHbOnDmaMGGCwsLClJeXp3//+9+qVauWpCtPgGzYsEE5OTnq06ePunTpohkzZqhp06am3huAqsfJyUnLli1TZmamOnTooIkTJ+pvf/tbuc87fvx4JSQkaNKkSerYsaPWrVun1atXq23btpKuPInx7rvvat++fQoNDdVf//pXPfvss6av88gjjygoKEjh4eFq1KjRdU+rAajZ5s2bp8jISA0aNEjR0dHq1auXQkJC5O7uLunK98BRo0Zp0qRJCgoKUmxsrF2lc2lMmjRJDz74oOLi4hQZGSlPT0/bU25XzZ49W9OnT1dycrJCQkIUExOjNWvWqGXLlhX6fgGgIlgMmpABAFDjzJ49WytXrtSuXbvsxtevX69+/frp1KlT9IgHAAA1WmFhoZo1a6aUlBSNGTPG0eEAQKVEWwwAAGqQq32fFy5cWKaqPwAAgOpqx44d2rdvn7p166YzZ85o1qxZkqQhQ4Y4ODIAqLxoi4FqZ+nSpfLw8Cjx1b59e0eHBwAVoqxr3dixYxUWFqZbb731hi0xAAAAaqq5c+eqU6dOio6OVmFhoTZu3KiGDRs6OiwAqLRoi4Fq59y5c/rxxx9L3Ofq6qoWLVr8zhEBQMVjrQMAAAAAOBrJZQAAAAAAAACAabTFAAAAAAAAAACYRnIZAAAAAAAAAGAayWUAAAAAAAAAgGkklwEAAAAAAAAAppFcBgAAAAAAAACYRnIZAAAAAAAAAGAayWUAAAAAAAAAgGkklwEAAAAAAAAApv0P9zjHKEWbLKUAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 2000x1000 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(20, 10))\n",
    "sns.heatmap(\n",
    "    df.select_dtypes(include=[np.number]).corr(),  # Select only numerical columns\n",
    "    annot=True,\n",
    "    fmt='.2f',\n",
    "    # cmap='coolwarm'\n",
    ")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature Engineering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>trans_time</th>\n",
       "      <th>mcc_code</th>\n",
       "      <th>trans_type</th>\n",
       "      <th>amount</th>\n",
       "      <th>term_id</th>\n",
       "      <th>trans_city</th>\n",
       "      <th>gender</th>\n",
       "      <th>trans_description</th>\n",
       "      <th>mcc_description</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>d1bbbc9a0e0410d3cf12a3d2f44f3450</th>\n",
       "      <td>35 08:24:41</td>\n",
       "      <td>4829</td>\n",
       "      <td>2370</td>\n",
       "      <td>-1808.56</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Tver</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Списание с карты на карту по операции &lt;перевод...</td>\n",
       "      <td>Денежные переводы</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81c7349f7bdcb661bc7d5f5f0fdac6aa</th>\n",
       "      <td>351 13:27:15</td>\n",
       "      <td>6011</td>\n",
       "      <td>7010</td>\n",
       "      <td>7954.51</td>\n",
       "      <td>442207</td>\n",
       "      <td>Kazan</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Взнос наличных через АТМ (в своем тер.банке)</td>\n",
       "      <td>Финансовые институты — снятие наличности автом...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3495487d5beaa5caaa0682fddc38379a</th>\n",
       "      <td>373 11:54:12</td>\n",
       "      <td>6010</td>\n",
       "      <td>7070</td>\n",
       "      <td>49.52</td>\n",
       "      <td>888987</td>\n",
       "      <td>Kaliningrad</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>07e279042b1da23710fa2f65034d30b9</th>\n",
       "      <td>340 13:11:45</td>\n",
       "      <td>6010</td>\n",
       "      <td>7070</td>\n",
       "      <td>45.29</td>\n",
       "      <td>888918</td>\n",
       "      <td>Saint Petersburg</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ec41f598a99167b68cb8e232f524017f</th>\n",
       "      <td>117 11:28:27</td>\n",
       "      <td>6011</td>\n",
       "      <td>2010</td>\n",
       "      <td>-28924.22</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Vladivostok</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Выдача наличных в АТМ Сбербанк России</td>\n",
       "      <td>Финансовые институты — снятие наличности автом...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e91858185ae43cf3970220748893a01b</th>\n",
       "      <td>328 13:45:38</td>\n",
       "      <td>6010</td>\n",
       "      <td>7070</td>\n",
       "      <td>227.40</td>\n",
       "      <td>888907</td>\n",
       "      <td>Penza</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>429b6c8255f13bdb506fdbd6e342ccb1</th>\n",
       "      <td>10 19:22:35</td>\n",
       "      <td>4814</td>\n",
       "      <td>1030</td>\n",
       "      <td>-1084.44</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Khabarovsk</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Оплата услуги. Банкоматы СБ РФ</td>\n",
       "      <td>Звонки с использованием телефонов, считывающих...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9443b6a0d948cf368d8d72eb295bdfdf</th>\n",
       "      <td>167 23:59:15</td>\n",
       "      <td>5411</td>\n",
       "      <td>1110</td>\n",
       "      <td>-569.20</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Tver</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Покупка. POS ТУ Россия</td>\n",
       "      <td>Бакалейные магазины, супермаркеты</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1be75d38ad89c0f2b4d85b95f0fd8069</th>\n",
       "      <td>101 23:59:03</td>\n",
       "      <td>5814</td>\n",
       "      <td>1110</td>\n",
       "      <td>-71.43</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Vladivostok</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Покупка. POS ТУ Россия</td>\n",
       "      <td>Рестораны, закусочные</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8d69e4ecae4b509d528b9a0168f749d0</th>\n",
       "      <td>148 19:12:04</td>\n",
       "      <td>6010</td>\n",
       "      <td>7070</td>\n",
       "      <td>50.92</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Kazan</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3237373 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    trans_time  mcc_code  trans_type  \\\n",
       "client_id                                                              \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450   35 08:24:41      4829        2370   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa  351 13:27:15      6011        7010   \n",
       "3495487d5beaa5caaa0682fddc38379a  373 11:54:12      6010        7070   \n",
       "07e279042b1da23710fa2f65034d30b9  340 13:11:45      6010        7070   \n",
       "ec41f598a99167b68cb8e232f524017f  117 11:28:27      6011        2010   \n",
       "...                                        ...       ...         ...   \n",
       "e91858185ae43cf3970220748893a01b  328 13:45:38      6010        7070   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1   10 19:22:35      4814        1030   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf  167 23:59:15      5411        1110   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069  101 23:59:03      5814        1110   \n",
       "8d69e4ecae4b509d528b9a0168f749d0  148 19:12:04      6010        7070   \n",
       "\n",
       "                                    amount term_id        trans_city  gender  \\\n",
       "client_id                                                                      \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450  -1808.56     NaN              Tver     0.0   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa   7954.51  442207             Kazan     0.0   \n",
       "3495487d5beaa5caaa0682fddc38379a     49.52  888987       Kaliningrad     0.0   \n",
       "07e279042b1da23710fa2f65034d30b9     45.29  888918  Saint Petersburg     0.0   \n",
       "ec41f598a99167b68cb8e232f524017f -28924.22     NaN       Vladivostok     0.0   \n",
       "...                                    ...     ...               ...     ...   \n",
       "e91858185ae43cf3970220748893a01b    227.40  888907             Penza     0.0   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1  -1084.44     NaN        Khabarovsk     0.0   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf   -569.20     NaN              Tver     0.0   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069    -71.43     NaN       Vladivostok     1.0   \n",
       "8d69e4ecae4b509d528b9a0168f749d0     50.92     NaN             Kazan     0.0   \n",
       "\n",
       "                                                                  trans_description  \\\n",
       "client_id                                                                             \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450  Списание с карты на карту по операции <перевод...   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa       Взнос наличных через АТМ (в своем тер.банке)   \n",
       "3495487d5beaa5caaa0682fddc38379a  Перевод на карту (с карты) через Мобильный бан...   \n",
       "07e279042b1da23710fa2f65034d30b9  Перевод на карту (с карты) через Мобильный бан...   \n",
       "ec41f598a99167b68cb8e232f524017f              Выдача наличных в АТМ Сбербанк России   \n",
       "...                                                                             ...   \n",
       "e91858185ae43cf3970220748893a01b  Перевод на карту (с карты) через Мобильный бан...   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1                     Оплата услуги. Банкоматы СБ РФ   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf                             Покупка. POS ТУ Россия   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069                             Покупка. POS ТУ Россия   \n",
       "8d69e4ecae4b509d528b9a0168f749d0  Перевод на карту (с карты) через Мобильный бан...   \n",
       "\n",
       "                                                                    mcc_description  \n",
       "client_id                                                                            \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450                                  Денежные переводы  \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa  Финансовые институты — снятие наличности автом...  \n",
       "3495487d5beaa5caaa0682fddc38379a   Финансовые институты — снятие наличности вручную  \n",
       "07e279042b1da23710fa2f65034d30b9   Финансовые институты — снятие наличности вручную  \n",
       "ec41f598a99167b68cb8e232f524017f  Финансовые институты — снятие наличности автом...  \n",
       "...                                                                             ...  \n",
       "e91858185ae43cf3970220748893a01b   Финансовые институты — снятие наличности вручную  \n",
       "429b6c8255f13bdb506fdbd6e342ccb1  Звонки с использованием телефонов, считывающих...  \n",
       "9443b6a0d948cf368d8d72eb295bdfdf                  Бакалейные магазины, супермаркеты  \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069                              Рестораны, закусочные  \n",
       "8d69e4ecae4b509d528b9a0168f749d0   Финансовые институты — снятие наличности вручную  \n",
       "\n",
       "[3237373 rows x 9 columns]"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['trans_time',\n",
       " 'mcc_code',\n",
       " 'trans_type',\n",
       " 'amount',\n",
       " 'term_id',\n",
       " 'trans_city',\n",
       " 'gender',\n",
       " 'trans_description',\n",
       " 'mcc_description']"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns.tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['earned'] = [0 if i <0 else 1 for i in df['amount']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['day'] = pd.DataFrame(df[\"trans_time\"].str.split(' ', expand = True)[0]).astype(\"int64\").fillna(0)\n",
    "df['time'] = pd.DataFrame(df[\"trans_time\"].str.split(' ', expand = True)[1].str.split(':', expand = True)[0]).astype(\"int64\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['total'] = df.groupby(['client_id'])[\"amount\"].sum().fillna(0)\n",
    "df['total_spent'] = df.loc[df['earned'] == 0].groupby(['client_id'])[\"amount\"].sum().fillna(0)\n",
    "df['total_earned'] = df.loc[df['earned'] == 1].groupby(['client_id'])[\"amount\"].sum().fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"avg_amount_per_day\"] = df.groupby(['client_id','day'])[\"amount\"].transform('mean').fillna(0)\n",
    "df[\"var_amount_per_day\"] = df.groupby(['client_id','day'])[\"amount\"].transform('std').fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['avg_sum_per_transaction'] = df.groupby(['client_id'])['amount'].mean().fillna(0)\n",
    "df['avg_sum_spent_per_transaction'] = df.loc[df['earned'] == 0].groupby(['client_id'])['amount'].mean().fillna(0)\n",
    "df['avg_sum_earned_per_transaction'] = df.loc[df['earned'] == 1].groupby(['client_id'])['amount'].mean().fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['var_sum_per_trans'] = df.groupby(['client_id'])['amount'].std().fillna(0)\n",
    "df['var_sum_spent_per_trans'] = df.loc[df['earned'] == 0].groupby(['client_id'])['amount'].std().fillna(0)\n",
    "df['var_sum_earned_per_trans'] = df.loc[df['earned'] == 1].groupby(['client_id'])['amount'].std().fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['transactions_per_day'] = df.groupby([df.index, 'day'])[\"day\"].transform(\"count\").fillna(0)\n",
    "df['transactions_per_day_spent'] = df.groupby(['client_id'])[\"day\"].transform(lambda x: x[x < 0].count()).fillna(0)\n",
    "df['transactions_per_day_earned'] = df.groupby(['client_id'])[\"day\"].transform(lambda x: x[x > 0].count()).fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"var_amount_per_day\"] = df.groupby(['client_id','day'])['amount'].transform('std').fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['sum_per_day'] = df.groupby(['client_id', 'day'])['amount'].transform('sum').fillna(0)\n",
    "df['sum_per_day_spent'] = df.loc[df['earned'] == 0].groupby(['client_id', 'day'])['amount'].sum().fillna(0)\n",
    "df['sum_per_day_earned'] = df.loc[df['earned'] == 1].groupby(['client_id', 'day'])['amount'].sum().fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['terminal_unique'] = df.groupby(['client_id','term_id'])['term_id'].transform('count').fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"var_time_transaction\"] = df.groupby(['client_id'])['time'].transform('std')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "df['tr_unique_count'] = df.groupby(['client_id',\"trans_type\"])[\"trans_type\"].transform('count').fillna(0)\n",
    "df['tr_unique_sum'] = df.groupby(['client_id',\"trans_type\"])[\"trans_type\"].transform('sum').fillna(0)\n",
    "df['tr_unique_std'] = df.groupby(['client_id',\"trans_type\"])[\"trans_type\"].transform('std').fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['mcc_unique_count'] = df.groupby(['client_id','mcc_code'])['mcc_code'].transform('count').fillna(0)\n",
    "df['mcc_unique_sum'] = df.groupby(['client_id','mcc_code'])['mcc_code'].transform('sum').fillna(0)\n",
    "df['mcc_unique_std'] = df.groupby(['client_id','mcc_code'])['mcc_code'].transform('std').fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['tr_unique_count'] = df.groupby(['client_id','trans_type'])['trans_type'].transform('count').fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"total_amount_spend_to_earn\"] = np.divide(df[\"total_spent\"].abs(), df[\"total_earned\"].abs())\n",
    "df[\"total_amount_spend_to_earn\"].replace(np.inf, 1000, inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame(pd.get_dummies(data = df, columns = ['trans_type', 'mcc_code']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Grouping by client_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>trans_time</th>\n",
       "      <th>amount</th>\n",
       "      <th>term_id</th>\n",
       "      <th>trans_city</th>\n",
       "      <th>gender</th>\n",
       "      <th>trans_description</th>\n",
       "      <th>mcc_description</th>\n",
       "      <th>earned</th>\n",
       "      <th>day</th>\n",
       "      <th>time</th>\n",
       "      <th>...</th>\n",
       "      <th>mcc_code_8299</th>\n",
       "      <th>mcc_code_8398</th>\n",
       "      <th>mcc_code_8641</th>\n",
       "      <th>mcc_code_8699</th>\n",
       "      <th>mcc_code_8999</th>\n",
       "      <th>mcc_code_9211</th>\n",
       "      <th>mcc_code_9222</th>\n",
       "      <th>mcc_code_9311</th>\n",
       "      <th>mcc_code_9399</th>\n",
       "      <th>mcc_code_9402</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>client_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>d1bbbc9a0e0410d3cf12a3d2f44f3450</th>\n",
       "      <td>35 08:24:41</td>\n",
       "      <td>-1808.56</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Tver</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Списание с карты на карту по операции &lt;перевод...</td>\n",
       "      <td>Денежные переводы</td>\n",
       "      <td>0</td>\n",
       "      <td>35</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81c7349f7bdcb661bc7d5f5f0fdac6aa</th>\n",
       "      <td>351 13:27:15</td>\n",
       "      <td>7954.51</td>\n",
       "      <td>442207</td>\n",
       "      <td>Kazan</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Взнос наличных через АТМ (в своем тер.банке)</td>\n",
       "      <td>Финансовые институты — снятие наличности автом...</td>\n",
       "      <td>1</td>\n",
       "      <td>351</td>\n",
       "      <td>13</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3495487d5beaa5caaa0682fddc38379a</th>\n",
       "      <td>373 11:54:12</td>\n",
       "      <td>49.52</td>\n",
       "      <td>888987</td>\n",
       "      <td>Kaliningrad</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "      <td>1</td>\n",
       "      <td>373</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>07e279042b1da23710fa2f65034d30b9</th>\n",
       "      <td>340 13:11:45</td>\n",
       "      <td>45.29</td>\n",
       "      <td>888918</td>\n",
       "      <td>Saint Petersburg</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "      <td>1</td>\n",
       "      <td>340</td>\n",
       "      <td>13</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ec41f598a99167b68cb8e232f524017f</th>\n",
       "      <td>117 11:28:27</td>\n",
       "      <td>-28924.22</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Vladivostok</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Выдача наличных в АТМ Сбербанк России</td>\n",
       "      <td>Финансовые институты — снятие наличности автом...</td>\n",
       "      <td>0</td>\n",
       "      <td>117</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e91858185ae43cf3970220748893a01b</th>\n",
       "      <td>328 13:45:38</td>\n",
       "      <td>227.40</td>\n",
       "      <td>888907</td>\n",
       "      <td>Penza</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "      <td>1</td>\n",
       "      <td>328</td>\n",
       "      <td>13</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>429b6c8255f13bdb506fdbd6e342ccb1</th>\n",
       "      <td>10 19:22:35</td>\n",
       "      <td>-1084.44</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Khabarovsk</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Оплата услуги. Банкоматы СБ РФ</td>\n",
       "      <td>Звонки с использованием телефонов, считывающих...</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>19</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9443b6a0d948cf368d8d72eb295bdfdf</th>\n",
       "      <td>167 23:59:15</td>\n",
       "      <td>-569.20</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Tver</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Покупка. POS ТУ Россия</td>\n",
       "      <td>Бакалейные магазины, супермаркеты</td>\n",
       "      <td>0</td>\n",
       "      <td>167</td>\n",
       "      <td>23</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1be75d38ad89c0f2b4d85b95f0fd8069</th>\n",
       "      <td>101 23:59:03</td>\n",
       "      <td>-71.43</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Vladivostok</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Покупка. POS ТУ Россия</td>\n",
       "      <td>Рестораны, закусочные</td>\n",
       "      <td>0</td>\n",
       "      <td>101</td>\n",
       "      <td>23</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8d69e4ecae4b509d528b9a0168f749d0</th>\n",
       "      <td>148 19:12:04</td>\n",
       "      <td>50.92</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Kazan</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Перевод на карту (с карты) через Мобильный бан...</td>\n",
       "      <td>Финансовые институты — снятие наличности вручную</td>\n",
       "      <td>1</td>\n",
       "      <td>148</td>\n",
       "      <td>19</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3237373 rows × 290 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    trans_time    amount term_id  \\\n",
       "client_id                                                          \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450   35 08:24:41  -1808.56     NaN   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa  351 13:27:15   7954.51  442207   \n",
       "3495487d5beaa5caaa0682fddc38379a  373 11:54:12     49.52  888987   \n",
       "07e279042b1da23710fa2f65034d30b9  340 13:11:45     45.29  888918   \n",
       "ec41f598a99167b68cb8e232f524017f  117 11:28:27 -28924.22     NaN   \n",
       "...                                        ...       ...     ...   \n",
       "e91858185ae43cf3970220748893a01b  328 13:45:38    227.40  888907   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1   10 19:22:35  -1084.44     NaN   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf  167 23:59:15   -569.20     NaN   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069  101 23:59:03    -71.43     NaN   \n",
       "8d69e4ecae4b509d528b9a0168f749d0  148 19:12:04     50.92     NaN   \n",
       "\n",
       "                                        trans_city  gender  \\\n",
       "client_id                                                    \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450              Tver     0.0   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa             Kazan     0.0   \n",
       "3495487d5beaa5caaa0682fddc38379a       Kaliningrad     0.0   \n",
       "07e279042b1da23710fa2f65034d30b9  Saint Petersburg     0.0   \n",
       "ec41f598a99167b68cb8e232f524017f       Vladivostok     0.0   \n",
       "...                                            ...     ...   \n",
       "e91858185ae43cf3970220748893a01b             Penza     0.0   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1        Khabarovsk     0.0   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf              Tver     0.0   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069       Vladivostok     1.0   \n",
       "8d69e4ecae4b509d528b9a0168f749d0             Kazan     0.0   \n",
       "\n",
       "                                                                  trans_description  \\\n",
       "client_id                                                                             \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450  Списание с карты на карту по операции <перевод...   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa       Взнос наличных через АТМ (в своем тер.банке)   \n",
       "3495487d5beaa5caaa0682fddc38379a  Перевод на карту (с карты) через Мобильный бан...   \n",
       "07e279042b1da23710fa2f65034d30b9  Перевод на карту (с карты) через Мобильный бан...   \n",
       "ec41f598a99167b68cb8e232f524017f              Выдача наличных в АТМ Сбербанк России   \n",
       "...                                                                             ...   \n",
       "e91858185ae43cf3970220748893a01b  Перевод на карту (с карты) через Мобильный бан...   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1                     Оплата услуги. Банкоматы СБ РФ   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf                             Покупка. POS ТУ Россия   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069                             Покупка. POS ТУ Россия   \n",
       "8d69e4ecae4b509d528b9a0168f749d0  Перевод на карту (с карты) через Мобильный бан...   \n",
       "\n",
       "                                                                    mcc_description  \\\n",
       "client_id                                                                             \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450                                  Денежные переводы   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa  Финансовые институты — снятие наличности автом...   \n",
       "3495487d5beaa5caaa0682fddc38379a   Финансовые институты — снятие наличности вручную   \n",
       "07e279042b1da23710fa2f65034d30b9   Финансовые институты — снятие наличности вручную   \n",
       "ec41f598a99167b68cb8e232f524017f  Финансовые институты — снятие наличности автом...   \n",
       "...                                                                             ...   \n",
       "e91858185ae43cf3970220748893a01b   Финансовые институты — снятие наличности вручную   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1  Звонки с использованием телефонов, считывающих...   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf                  Бакалейные магазины, супермаркеты   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069                              Рестораны, закусочные   \n",
       "8d69e4ecae4b509d528b9a0168f749d0   Финансовые институты — снятие наличности вручную   \n",
       "\n",
       "                                  earned  day  time  ...  mcc_code_8299  \\\n",
       "client_id                                            ...                  \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450       0   35     8  ...          False   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa       1  351    13  ...          False   \n",
       "3495487d5beaa5caaa0682fddc38379a       1  373    11  ...          False   \n",
       "07e279042b1da23710fa2f65034d30b9       1  340    13  ...          False   \n",
       "ec41f598a99167b68cb8e232f524017f       0  117    11  ...          False   \n",
       "...                                  ...  ...   ...  ...            ...   \n",
       "e91858185ae43cf3970220748893a01b       1  328    13  ...          False   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1       0   10    19  ...          False   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf       0  167    23  ...          False   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069       0  101    23  ...          False   \n",
       "8d69e4ecae4b509d528b9a0168f749d0       1  148    19  ...          False   \n",
       "\n",
       "                                  mcc_code_8398  mcc_code_8641  mcc_code_8699  \\\n",
       "client_id                                                                       \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450          False          False          False   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa          False          False          False   \n",
       "3495487d5beaa5caaa0682fddc38379a          False          False          False   \n",
       "07e279042b1da23710fa2f65034d30b9          False          False          False   \n",
       "ec41f598a99167b68cb8e232f524017f          False          False          False   \n",
       "...                                         ...            ...            ...   \n",
       "e91858185ae43cf3970220748893a01b          False          False          False   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1          False          False          False   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf          False          False          False   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069          False          False          False   \n",
       "8d69e4ecae4b509d528b9a0168f749d0          False          False          False   \n",
       "\n",
       "                                  mcc_code_8999  mcc_code_9211  mcc_code_9222  \\\n",
       "client_id                                                                       \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450          False          False          False   \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa          False          False          False   \n",
       "3495487d5beaa5caaa0682fddc38379a          False          False          False   \n",
       "07e279042b1da23710fa2f65034d30b9          False          False          False   \n",
       "ec41f598a99167b68cb8e232f524017f          False          False          False   \n",
       "...                                         ...            ...            ...   \n",
       "e91858185ae43cf3970220748893a01b          False          False          False   \n",
       "429b6c8255f13bdb506fdbd6e342ccb1          False          False          False   \n",
       "9443b6a0d948cf368d8d72eb295bdfdf          False          False          False   \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069          False          False          False   \n",
       "8d69e4ecae4b509d528b9a0168f749d0          False          False          False   \n",
       "\n",
       "                                  mcc_code_9311  mcc_code_9399  mcc_code_9402  \n",
       "client_id                                                                      \n",
       "d1bbbc9a0e0410d3cf12a3d2f44f3450          False          False          False  \n",
       "81c7349f7bdcb661bc7d5f5f0fdac6aa          False          False          False  \n",
       "3495487d5beaa5caaa0682fddc38379a          False          False          False  \n",
       "07e279042b1da23710fa2f65034d30b9          False          False          False  \n",
       "ec41f598a99167b68cb8e232f524017f          False          False          False  \n",
       "...                                         ...            ...            ...  \n",
       "e91858185ae43cf3970220748893a01b          False          False          False  \n",
       "429b6c8255f13bdb506fdbd6e342ccb1          False          False          False  \n",
       "9443b6a0d948cf368d8d72eb295bdfdf          False          False          False  \n",
       "1be75d38ad89c0f2b4d85b95f0fd8069          False          False          False  \n",
       "8d69e4ecae4b509d528b9a0168f749d0          False          False          False  \n",
       "\n",
       "[3237373 rows x 290 columns]"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "numeric_cols = df.select_dtypes(include=[np.number, np.bool_]).columns\n",
    "df_customers = df.groupby(['client_id'])[numeric_cols].max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = df_customers['gender']\n",
    "X = df_customers.drop(['gender', 'amount', 'earned'], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "X['total_earned'] = X['total_earned'].fillna(0)\n",
    "X['avg_sum_earned_per_transaction'] = X['avg_sum_earned_per_transaction'].fillna(0)\n",
    "X['total_spent'] = X['total_spent'].fillna(0)\n",
    "X['avg_sum_spent_per_transaction'] = X['avg_sum_spent_per_transaction'].fillna(0)\n",
    "X['var_sum_earned_per_trans'] = X['var_sum_earned_per_trans'].fillna(0)\n",
    "X['var_sum_spent_per_trans'] = X['var_sum_spent_per_trans'].fillna(0)\n",
    "X['var_time_transaction'] = X['var_time_transaction'].fillna(0)\n",
    "X['total_amount_spend_to_earn'] = X['total_amount_spend_to_earn'].fillna(0)\n",
    "# X['amount'] =  X['amount'].fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sum_per_day_spent 7560 (100.0%)\n",
      "sum_per_day_earned 7560 (100.0%)\n"
     ]
    }
   ],
   "source": [
    "missing_features(X, X.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "# drop the columns with missing values\n",
    "X = X.drop(['sum_per_day_spent', 'sum_per_day_earned'], axis = 1)\n",
    "missing_features(X, X.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Catboost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "cb = CatBoostClassifier(depth= 4, iterations= 25, l2_leaf_reg= 0, learning_rate= 0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:\tlearn: 0.6088873\ttotal: 5.06ms\tremaining: 121ms\n",
      "1:\tlearn: 0.5651690\ttotal: 7.93ms\tremaining: 91.2ms\n",
      "2:\tlearn: 0.5448952\ttotal: 11ms\tremaining: 81ms\n",
      "3:\tlearn: 0.5286176\ttotal: 14.6ms\tremaining: 76.5ms\n",
      "4:\tlearn: 0.5186193\ttotal: 17.5ms\tremaining: 70.1ms\n",
      "5:\tlearn: 0.5082568\ttotal: 20.4ms\tremaining: 64.4ms\n",
      "6:\tlearn: 0.5024343\ttotal: 24ms\tremaining: 61.6ms\n",
      "7:\tlearn: 0.4973212\ttotal: 26.7ms\tremaining: 56.6ms\n",
      "8:\tlearn: 0.4921810\ttotal: 29.1ms\tremaining: 51.7ms\n",
      "9:\tlearn: 0.4863886\ttotal: 31.3ms\tremaining: 47ms\n",
      "10:\tlearn: 0.4836113\ttotal: 33.5ms\tremaining: 42.6ms\n",
      "11:\tlearn: 0.4794957\ttotal: 36.4ms\tremaining: 39.5ms\n",
      "12:\tlearn: 0.4764210\ttotal: 39.2ms\tremaining: 36.2ms\n",
      "13:\tlearn: 0.4724502\ttotal: 41.4ms\tremaining: 32.6ms\n",
      "14:\tlearn: 0.4697401\ttotal: 43.8ms\tremaining: 29.2ms\n",
      "15:\tlearn: 0.4667888\ttotal: 46.3ms\tremaining: 26ms\n",
      "16:\tlearn: 0.4640025\ttotal: 48.6ms\tremaining: 22.9ms\n",
      "17:\tlearn: 0.4612791\ttotal: 51ms\tremaining: 19.8ms\n",
      "18:\tlearn: 0.4588061\ttotal: 53.3ms\tremaining: 16.8ms\n",
      "19:\tlearn: 0.4558982\ttotal: 55.6ms\tremaining: 13.9ms\n",
      "20:\tlearn: 0.4520908\ttotal: 58ms\tremaining: 11.1ms\n",
      "21:\tlearn: 0.4488772\ttotal: 60.8ms\tremaining: 8.29ms\n",
      "22:\tlearn: 0.4463045\ttotal: 63.4ms\tremaining: 5.51ms\n",
      "23:\tlearn: 0.4440520\ttotal: 65.7ms\tremaining: 2.74ms\n",
      "24:\tlearn: 0.4410981\ttotal: 67.9ms\tremaining: 0us\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<catboost.core.CatBoostClassifier at 0x16661a810>"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cb.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "cv_split = KFold(n_splits = 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:\tlearn: 0.6070579\ttotal: 4.22ms\tremaining: 101ms\n",
      "1:\tlearn: 0.5625855\ttotal: 6.28ms\tremaining: 72.2ms\n",
      "2:\tlearn: 0.5434949\ttotal: 8.24ms\tremaining: 60.5ms\n",
      "3:\tlearn: 0.5309186\ttotal: 11.4ms\tremaining: 59.6ms\n",
      "4:\tlearn: 0.5200495\ttotal: 13.4ms\tremaining: 53.7ms\n",
      "5:\tlearn: 0.5102590\ttotal: 15.5ms\tremaining: 49.1ms\n",
      "6:\tlearn: 0.5050187\ttotal: 17.6ms\tremaining: 45.1ms\n",
      "7:\tlearn: 0.4973843\ttotal: 19.5ms\tremaining: 41.5ms\n",
      "8:\tlearn: 0.4920119\ttotal: 21.7ms\tremaining: 38.6ms\n",
      "9:\tlearn: 0.4883333\ttotal: 24ms\tremaining: 36ms\n",
      "10:\tlearn: 0.4824356\ttotal: 26.1ms\tremaining: 33.2ms\n",
      "11:\tlearn: 0.4772078\ttotal: 28.4ms\tremaining: 30.8ms\n",
      "12:\tlearn: 0.4744371\ttotal: 30.5ms\tremaining: 28.1ms\n",
      "13:\tlearn: 0.4703884\ttotal: 32.8ms\tremaining: 25.8ms\n",
      "14:\tlearn: 0.4653196\ttotal: 34.9ms\tremaining: 23.3ms\n",
      "15:\tlearn: 0.4607757\ttotal: 36.9ms\tremaining: 20.8ms\n",
      "16:\tlearn: 0.4581277\ttotal: 39ms\tremaining: 18.3ms\n",
      "17:\tlearn: 0.4554511\ttotal: 41.3ms\tremaining: 16.1ms\n",
      "18:\tlearn: 0.4525035\ttotal: 43.5ms\tremaining: 13.7ms\n",
      "19:\tlearn: 0.4488797\ttotal: 45.5ms\tremaining: 11.4ms\n",
      "20:\tlearn: 0.4450705\ttotal: 48.1ms\tremaining: 9.17ms\n",
      "21:\tlearn: 0.4413403\ttotal: 50.4ms\tremaining: 6.87ms\n",
      "22:\tlearn: 0.4383347\ttotal: 52.3ms\tremaining: 4.55ms\n",
      "23:\tlearn: 0.4344191\ttotal: 54.3ms\tremaining: 2.26ms\n",
      "24:\tlearn: 0.4309427\ttotal: 56.3ms\tremaining: 0us\n",
      "0:\tlearn: 0.6072530\ttotal: 2.65ms\tremaining: 63.6ms\n",
      "1:\tlearn: 0.5689521\ttotal: 4.6ms\tremaining: 53ms\n",
      "2:\tlearn: 0.5479969\ttotal: 6.39ms\tremaining: 46.9ms\n",
      "3:\tlearn: 0.5372499\ttotal: 8.49ms\tremaining: 44.6ms\n",
      "4:\tlearn: 0.5259051\ttotal: 11.3ms\tremaining: 45.1ms\n",
      "5:\tlearn: 0.5170251\ttotal: 13.2ms\tremaining: 41.8ms\n",
      "6:\tlearn: 0.5080184\ttotal: 15.4ms\tremaining: 39.7ms\n",
      "7:\tlearn: 0.5041857\ttotal: 17.4ms\tremaining: 37ms\n",
      "8:\tlearn: 0.4969437\ttotal: 20ms\tremaining: 35.5ms\n",
      "9:\tlearn: 0.4908805\ttotal: 22.7ms\tremaining: 34.1ms\n",
      "10:\tlearn: 0.4865866\ttotal: 24.8ms\tremaining: 31.6ms\n",
      "11:\tlearn: 0.4801992\ttotal: 27ms\tremaining: 29.3ms\n",
      "12:\tlearn: 0.4753303\ttotal: 29.4ms\tremaining: 27.1ms\n",
      "13:\tlearn: 0.4712161\ttotal: 31.6ms\tremaining: 24.8ms\n",
      "14:\tlearn: 0.4673342\ttotal: 33.9ms\tremaining: 22.6ms\n",
      "15:\tlearn: 0.4624457\ttotal: 36ms\tremaining: 20.3ms\n",
      "16:\tlearn: 0.4574008\ttotal: 38.5ms\tremaining: 18.1ms\n",
      "17:\tlearn: 0.4535480\ttotal: 41.1ms\tremaining: 16ms\n",
      "18:\tlearn: 0.4496205\ttotal: 43.6ms\tremaining: 13.8ms\n",
      "19:\tlearn: 0.4464348\ttotal: 45.8ms\tremaining: 11.4ms\n",
      "20:\tlearn: 0.4427797\ttotal: 48.2ms\tremaining: 9.19ms\n",
      "21:\tlearn: 0.4396595\ttotal: 50.3ms\tremaining: 6.86ms\n",
      "22:\tlearn: 0.4364098\ttotal: 52.4ms\tremaining: 4.56ms\n",
      "23:\tlearn: 0.4316292\ttotal: 54.5ms\tremaining: 2.27ms\n",
      "24:\tlearn: 0.4291688\ttotal: 56.7ms\tremaining: 0us\n",
      "0:\tlearn: 0.6139741\ttotal: 2.12ms\tremaining: 50.8ms\n",
      "1:\tlearn: 0.5721110\ttotal: 4.07ms\tremaining: 46.8ms\n",
      "2:\tlearn: 0.5460087\ttotal: 5.88ms\tremaining: 43.1ms\n",
      "3:\tlearn: 0.5330424\ttotal: 8.05ms\tremaining: 42.3ms\n",
      "4:\tlearn: 0.5229117\ttotal: 10ms\tremaining: 40.2ms\n",
      "5:\tlearn: 0.5143490\ttotal: 12ms\tremaining: 38ms\n",
      "6:\tlearn: 0.5072341\ttotal: 14.2ms\tremaining: 36.6ms\n",
      "7:\tlearn: 0.5022855\ttotal: 16.3ms\tremaining: 34.7ms\n",
      "8:\tlearn: 0.4972045\ttotal: 18.5ms\tremaining: 32.9ms\n",
      "9:\tlearn: 0.4909470\ttotal: 20.5ms\tremaining: 30.7ms\n",
      "10:\tlearn: 0.4849808\ttotal: 22.5ms\tremaining: 28.6ms\n",
      "11:\tlearn: 0.4790643\ttotal: 24.7ms\tremaining: 26.8ms\n",
      "12:\tlearn: 0.4728142\ttotal: 26.8ms\tremaining: 24.7ms\n",
      "13:\tlearn: 0.4663630\ttotal: 28.9ms\tremaining: 22.7ms\n",
      "14:\tlearn: 0.4631054\ttotal: 31.2ms\tremaining: 20.8ms\n",
      "15:\tlearn: 0.4595076\ttotal: 33.4ms\tremaining: 18.8ms\n",
      "16:\tlearn: 0.4567104\ttotal: 35.6ms\tremaining: 16.8ms\n",
      "17:\tlearn: 0.4533419\ttotal: 37.5ms\tremaining: 14.6ms\n",
      "18:\tlearn: 0.4502388\ttotal: 40.1ms\tremaining: 12.7ms\n",
      "19:\tlearn: 0.4465696\ttotal: 42.6ms\tremaining: 10.6ms\n",
      "20:\tlearn: 0.4434011\ttotal: 45.3ms\tremaining: 8.62ms\n",
      "21:\tlearn: 0.4402958\ttotal: 47.3ms\tremaining: 6.45ms\n",
      "22:\tlearn: 0.4372036\ttotal: 49.6ms\tremaining: 4.32ms\n",
      "23:\tlearn: 0.4337120\ttotal: 52.3ms\tremaining: 2.18ms\n",
      "24:\tlearn: 0.4310421\ttotal: 54.2ms\tremaining: 0us\n",
      "0:\tlearn: 0.6111970\ttotal: 2.57ms\tremaining: 61.8ms\n",
      "1:\tlearn: 0.5751030\ttotal: 4.79ms\tremaining: 55.1ms\n",
      "2:\tlearn: 0.5485869\ttotal: 7.04ms\tremaining: 51.6ms\n",
      "3:\tlearn: 0.5333677\ttotal: 9.12ms\tremaining: 47.9ms\n",
      "4:\tlearn: 0.5228205\ttotal: 11.2ms\tremaining: 45ms\n",
      "5:\tlearn: 0.5132558\ttotal: 13.2ms\tremaining: 42ms\n",
      "6:\tlearn: 0.5057527\ttotal: 16.7ms\tremaining: 42.9ms\n",
      "7:\tlearn: 0.4986727\ttotal: 18.9ms\tremaining: 40.2ms\n",
      "8:\tlearn: 0.4919105\ttotal: 21.1ms\tremaining: 37.5ms\n",
      "9:\tlearn: 0.4863854\ttotal: 23.7ms\tremaining: 35.6ms\n",
      "10:\tlearn: 0.4805417\ttotal: 25.8ms\tremaining: 32.8ms\n",
      "11:\tlearn: 0.4754737\ttotal: 27.9ms\tremaining: 30.2ms\n",
      "12:\tlearn: 0.4697715\ttotal: 30ms\tremaining: 27.7ms\n",
      "13:\tlearn: 0.4664508\ttotal: 32.4ms\tremaining: 25.4ms\n",
      "14:\tlearn: 0.4625976\ttotal: 34.4ms\tremaining: 22.9ms\n",
      "15:\tlearn: 0.4587109\ttotal: 36.4ms\tremaining: 20.4ms\n",
      "16:\tlearn: 0.4557828\ttotal: 38.6ms\tremaining: 18.2ms\n",
      "17:\tlearn: 0.4517387\ttotal: 41.1ms\tremaining: 16ms\n",
      "18:\tlearn: 0.4488045\ttotal: 43.2ms\tremaining: 13.6ms\n",
      "19:\tlearn: 0.4469078\ttotal: 45.2ms\tremaining: 11.3ms\n",
      "20:\tlearn: 0.4419467\ttotal: 47.3ms\tremaining: 9ms\n",
      "21:\tlearn: 0.4388632\ttotal: 49.5ms\tremaining: 6.75ms\n",
      "22:\tlearn: 0.4361161\ttotal: 51.6ms\tremaining: 4.49ms\n",
      "23:\tlearn: 0.4322327\ttotal: 53.9ms\tremaining: 2.25ms\n",
      "24:\tlearn: 0.4282540\ttotal: 56.2ms\tremaining: 0us\n"
     ]
    }
   ],
   "source": [
    "cv_results = cross_validate(cb, X, y, scoring='roc_auc', cv=cv_split, return_train_score=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'fit_time': array([0.07458711, 0.07164717, 0.07024884, 0.07219625]),\n",
       " 'score_time': array([0.00770807, 0.00687504, 0.00667977, 0.00659776]),\n",
       " 'test_score': array([0.84256833, 0.84267361, 0.83362673, 0.84181719]),\n",
       " 'train_score': array([0.8841668 , 0.88611471, 0.88528546, 0.88673717])}"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.840171463632342"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv_results['test_score'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Grid Search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "# params = {\n",
    "#     'iterations': [5,10,15,20,25,30],\n",
    "#     'learning_rate': [0.5, 0.1, 0.05, 0.01],\n",
    "#     'l2_leaf_reg': [0.5, 0.1, 0.05, 0.01],\n",
    "#     'depth': [None,1,2,3,4,5],\n",
    "#     'l2_leaf_reg': [0,0.1,0.01]\n",
    "# }\n",
    "\n",
    "# cb_cv = GridSearchCV(cb, param_grid=params, scoring='roc_auc', cv=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "# cb_cv.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [],
   "source": [
    "# cb_cv.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature importance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('time', 0.0),\n",
       " ('var_amount_per_day', 0.0),\n",
       " ('transactions_per_day', 0.0),\n",
       " ('transactions_per_day_spent', 0.0),\n",
       " ('tr_unique_count', 0.0),\n",
       " ('tr_unique_sum', 0.0),\n",
       " ('tr_unique_std', 0.0),\n",
       " ('mcc_unique_count', 0.0),\n",
       " ('mcc_unique_std', 0.0),\n",
       " ('trans_type_1000', 0.0),\n",
       " ('trans_type_1010', 0.0),\n",
       " ('trans_type_1030', 0.0),\n",
       " ('trans_type_1110', 0.0),\n",
       " ('trans_type_1200', 0.0),\n",
       " ('trans_type_1210', 0.0),\n",
       " ('trans_type_1310', 0.0),\n",
       " ('trans_type_1510', 0.0),\n",
       " ('trans_type_2010', 0.0),\n",
       " ('trans_type_2011', 0.0),\n",
       " ('trans_type_2020', 0.0),\n",
       " ('trans_type_2021', 0.0),\n",
       " ('trans_type_2100', 0.0),\n",
       " ('trans_type_2110', 0.0),\n",
       " ('trans_type_2200', 0.0),\n",
       " ('trans_type_2210', 0.0),\n",
       " ('trans_type_2320', 0.0),\n",
       " ('trans_type_2330', 0.0),\n",
       " ('trans_type_2331', 0.0),\n",
       " ('trans_type_2340', 0.0),\n",
       " ('trans_type_2341', 0.0),\n",
       " ('trans_type_2371', 0.0),\n",
       " ('trans_type_4010', 0.0),\n",
       " ('trans_type_4011', 0.0),\n",
       " ('trans_type_4021', 0.0),\n",
       " ('trans_type_4031', 0.0),\n",
       " ('trans_type_4035', 0.0),\n",
       " ('trans_type_4041', 0.0),\n",
       " ('trans_type_4045', 0.0),\n",
       " ('trans_type_4051', 0.0),\n",
       " ('trans_type_4061', 0.0),\n",
       " ('trans_type_4071', 0.0),\n",
       " ('trans_type_4090', 0.0),\n",
       " ('trans_type_4100', 0.0),\n",
       " ('trans_type_4200', 0.0),\n",
       " ('trans_type_4210', 0.0),\n",
       " ('trans_type_4500', 0.0),\n",
       " ('trans_type_6000', 0.0),\n",
       " ('trans_type_6010', 0.0),\n",
       " ('trans_type_6100', 0.0),\n",
       " ('trans_type_6110', 0.0),\n",
       " ('trans_type_6210', 0.0),\n",
       " ('trans_type_7010', 0.0),\n",
       " ('trans_type_7011', 0.0),\n",
       " ('trans_type_7014', 0.0),\n",
       " ('trans_type_7015', 0.0),\n",
       " ('trans_type_7020', 0.0),\n",
       " ('trans_type_7021', 0.0),\n",
       " ('trans_type_7024', 0.0),\n",
       " ('trans_type_7025', 0.0),\n",
       " ('trans_type_7031', 0.0),\n",
       " ('trans_type_7034', 0.0),\n",
       " ('trans_type_7035', 0.0),\n",
       " ('trans_type_7041', 0.0),\n",
       " ('trans_type_7074', 0.0),\n",
       " ('trans_type_7075', 0.0),\n",
       " ('trans_type_8100', 0.0),\n",
       " ('trans_type_8145', 0.0),\n",
       " ('trans_type_8146', 0.0),\n",
       " ('mcc_code_742', 0.0),\n",
       " ('mcc_code_1711', 0.0),\n",
       " ('mcc_code_1731', 0.0),\n",
       " ('mcc_code_1799', 0.0),\n",
       " ('mcc_code_2741', 0.0),\n",
       " ('mcc_code_3000', 0.0),\n",
       " ('mcc_code_3351', 0.0),\n",
       " ('mcc_code_3501', 0.0),\n",
       " ('mcc_code_4111', 0.0),\n",
       " ('mcc_code_4112', 0.0),\n",
       " ('mcc_code_4121', 0.0),\n",
       " ('mcc_code_4131', 0.0),\n",
       " ('mcc_code_4214', 0.0),\n",
       " ('mcc_code_4215', 0.0),\n",
       " ('mcc_code_4411', 0.0),\n",
       " ('mcc_code_4511', 0.0),\n",
       " ('mcc_code_4722', 0.0),\n",
       " ('mcc_code_4784', 0.0),\n",
       " ('mcc_code_4789', 0.0),\n",
       " ('mcc_code_4814', 0.0),\n",
       " ('mcc_code_4816', 0.0),\n",
       " ('mcc_code_4829', 0.0),\n",
       " ('mcc_code_4899', 0.0),\n",
       " ('mcc_code_4900', 0.0),\n",
       " ('mcc_code_5013', 0.0),\n",
       " ('mcc_code_5039', 0.0),\n",
       " ('mcc_code_5044', 0.0),\n",
       " ('mcc_code_5045', 0.0),\n",
       " ('mcc_code_5047', 0.0),\n",
       " ('mcc_code_5072', 0.0),\n",
       " ('mcc_code_5074', 0.0),\n",
       " ('mcc_code_5085', 0.0),\n",
       " ('mcc_code_5094', 0.0),\n",
       " ('mcc_code_5099', 0.0),\n",
       " ('mcc_code_5111', 0.0),\n",
       " ('mcc_code_5122', 0.0),\n",
       " ('mcc_code_5137', 0.0),\n",
       " ('mcc_code_5169', 0.0),\n",
       " ('mcc_code_5172', 0.0),\n",
       " ('mcc_code_5192', 0.0),\n",
       " ('mcc_code_5193', 0.0),\n",
       " ('mcc_code_5199', 0.0),\n",
       " ('mcc_code_5200', 0.0),\n",
       " ('mcc_code_5231', 0.0),\n",
       " ('mcc_code_5261', 0.0),\n",
       " ('mcc_code_5300', 0.0),\n",
       " ('mcc_code_5309', 0.0),\n",
       " ('mcc_code_5310', 0.0),\n",
       " ('mcc_code_5311', 0.0),\n",
       " ('mcc_code_5331', 0.0),\n",
       " ('mcc_code_5399', 0.0),\n",
       " ('mcc_code_5411', 0.0),\n",
       " ('mcc_code_5422', 0.0),\n",
       " ('mcc_code_5441', 0.0),\n",
       " ('mcc_code_5451', 0.0),\n",
       " ('mcc_code_5462', 0.0),\n",
       " ('mcc_code_5532', 0.0),\n",
       " ('mcc_code_5542', 0.0),\n",
       " ('mcc_code_5571', 0.0),\n",
       " ('mcc_code_5599', 0.0),\n",
       " ('mcc_code_5681', 0.0),\n",
       " ('mcc_code_5697', 0.0),\n",
       " ('mcc_code_5712', 0.0),\n",
       " ('mcc_code_5713', 0.0),\n",
       " ('mcc_code_5719', 0.0),\n",
       " ('mcc_code_5733', 0.0),\n",
       " ('mcc_code_5735', 0.0),\n",
       " ('mcc_code_5811', 0.0),\n",
       " ('mcc_code_5816', 0.0),\n",
       " ('mcc_code_5931', 0.0),\n",
       " ('mcc_code_5940', 0.0),\n",
       " ('mcc_code_5944', 0.0),\n",
       " ('mcc_code_5945', 0.0),\n",
       " ('mcc_code_5946', 0.0),\n",
       " ('mcc_code_5947', 0.0),\n",
       " ('mcc_code_5950', 0.0),\n",
       " ('mcc_code_5965', 0.0),\n",
       " ('mcc_code_5967', 0.0),\n",
       " ('mcc_code_5968', 0.0),\n",
       " ('mcc_code_5969', 0.0),\n",
       " ('mcc_code_5970', 0.0),\n",
       " ('mcc_code_5971', 0.0),\n",
       " ('mcc_code_5976', 0.0),\n",
       " ('mcc_code_5994', 0.0),\n",
       " ('mcc_code_5995', 0.0),\n",
       " ('mcc_code_5999', 0.0),\n",
       " ('mcc_code_6010', 0.0),\n",
       " ('mcc_code_6011', 0.0),\n",
       " ('mcc_code_6012', 0.0),\n",
       " ('mcc_code_6211', 0.0),\n",
       " ('mcc_code_6300', 0.0),\n",
       " ('mcc_code_6513', 0.0),\n",
       " ('mcc_code_6536', 0.0),\n",
       " ('mcc_code_7210', 0.0),\n",
       " ('mcc_code_7216', 0.0),\n",
       " ('mcc_code_7221', 0.0),\n",
       " ('mcc_code_7273', 0.0),\n",
       " ('mcc_code_7278', 0.0),\n",
       " ('mcc_code_7298', 0.0),\n",
       " ('mcc_code_7299', 0.0),\n",
       " ('mcc_code_7338', 0.0),\n",
       " ('mcc_code_7375', 0.0),\n",
       " ('mcc_code_7395', 0.0),\n",
       " ('mcc_code_7399', 0.0),\n",
       " ('mcc_code_7512', 0.0),\n",
       " ('mcc_code_7523', 0.0),\n",
       " ('mcc_code_7531', 0.0),\n",
       " ('mcc_code_7542', 0.0),\n",
       " ('mcc_code_7629', 0.0),\n",
       " ('mcc_code_7699', 0.0),\n",
       " ('mcc_code_7829', 0.0),\n",
       " ('mcc_code_7841', 0.0),\n",
       " ('mcc_code_7922', 0.0),\n",
       " ('mcc_code_7932', 0.0),\n",
       " ('mcc_code_7933', 0.0),\n",
       " ('mcc_code_7991', 0.0),\n",
       " ('mcc_code_7996', 0.0),\n",
       " ('mcc_code_7997', 0.0),\n",
       " ('mcc_code_7999', 0.0),\n",
       " ('mcc_code_8011', 0.0),\n",
       " ('mcc_code_8043', 0.0),\n",
       " ('mcc_code_8062', 0.0),\n",
       " ('mcc_code_8071', 0.0),\n",
       " ('mcc_code_8099', 0.0),\n",
       " ('mcc_code_8220', 0.0),\n",
       " ('mcc_code_8244', 0.0),\n",
       " ('mcc_code_8299', 0.0),\n",
       " ('mcc_code_8398', 0.0),\n",
       " ('mcc_code_8641', 0.0),\n",
       " ('mcc_code_8699', 0.0),\n",
       " ('mcc_code_8999', 0.0),\n",
       " ('mcc_code_9211', 0.0),\n",
       " ('mcc_code_9222', 0.0),\n",
       " ('mcc_code_9311', 0.0),\n",
       " ('mcc_code_9399', 0.0),\n",
       " ('mcc_code_9402', 0.0),\n",
       " ('mcc_code_5499', 0.06319917147446137),\n",
       " ('mcc_code_5065', 0.11255526696109533),\n",
       " ('trans_type_6200', 0.11356569997337534),\n",
       " ('mcc_code_5964', 0.1652921377148675),\n",
       " ('trans_type_1100', 0.19891945073946302),\n",
       " ('transactions_per_day_earned', 0.2385179111871083),\n",
       " ('var_sum_earned_per_trans', 0.25736597939255856),\n",
       " ('total_amount_spend_to_earn', 0.2640140425281379),\n",
       " ('mcc_code_7011', 0.3067790667805393),\n",
       " ('trans_type_2440', 0.32646839792594534),\n",
       " ('mcc_code_8021', 0.3366774611180189),\n",
       " ('mcc_code_5983', 0.3467206114812134),\n",
       " ('mcc_code_5812', 0.35124011846615477),\n",
       " ('var_sum_per_trans', 0.36404677359540655),\n",
       " ('mcc_code_5921', 0.3769004282921796),\n",
       " ('trans_type_7070', 0.377629956744949),\n",
       " ('mcc_code_5993', 0.39057669612428647),\n",
       " ('mcc_code_5251', 0.3937463979393287),\n",
       " ('mcc_code_7995', 0.4052104842365709),\n",
       " ('mcc_code_5722', 0.44681661554052593),\n",
       " ('avg_amount_per_day', 0.4502360503661314),\n",
       " ('avg_sum_per_transaction', 0.4696223470254558),\n",
       " ('mcc_code_5948', 0.4860336320371329),\n",
       " ('mcc_code_7311', 0.5245843519133667),\n",
       " ('mcc_code_4812', 0.5618075434022315),\n",
       " ('mcc_code_7832', 0.5670749622198563),\n",
       " ('mcc_code_6051', 0.5689885007884817),\n",
       " ('mcc_code_5641', 0.573577226035987),\n",
       " ('mcc_code_7993', 0.5870399726442183),\n",
       " ('mcc_code_5211', 0.5878093544845793),\n",
       " ('mcc_code_5992', 0.6017599496192494),\n",
       " ('mcc_code_7230', 0.6027582498626496),\n",
       " ('mcc_code_5655', 0.6159613491754404),\n",
       " ('terminal_unique', 0.6193852726590374),\n",
       " ('trans_type_4110', 0.6242978407953682),\n",
       " ('total_earned', 0.6665691263828396),\n",
       " ('mcc_code_5942', 0.6744724624487232),\n",
       " ('trans_type_7040', 0.7225648948534457),\n",
       " ('mcc_code_5699', 0.7354243270261357),\n",
       " ('mcc_unique_sum', 0.7853320431945597),\n",
       " ('mcc_code_5943', 0.8031773516181298),\n",
       " ('mcc_code_5714', 0.8131752414337506),\n",
       " ('mcc_code_5734', 0.8174719698460007),\n",
       " ('mcc_code_5611', 0.8955817396021176),\n",
       " ('sum_per_day', 0.9012629564186337),\n",
       " ('mcc_code_5941', 0.9399222781485316),\n",
       " ('mcc_code_5131', 0.9943089269285926),\n",
       " ('avg_sum_earned_per_transaction', 1.017237569238564),\n",
       " ('mcc_code_5813', 1.059778531572201),\n",
       " ('trans_type_2370', 1.1319704978095535),\n",
       " ('mcc_code_5949', 1.1780060344526557),\n",
       " ('trans_type_7071', 1.1969757080914634),\n",
       " ('mcc_code_5651', 1.1991797205965022),\n",
       " ('mcc_code_5511', 1.2621262441088708),\n",
       " ('trans_type_1410', 1.375348476007627),\n",
       " ('mcc_code_5814', 1.4501659660234487),\n",
       " ('mcc_code_7538', 1.5371844110478392),\n",
       " ('mcc_code_5912', 1.6330831575952116),\n",
       " ('mcc_code_5631', 1.6969451869471217),\n",
       " ('trans_type_7030', 1.7514519093774708),\n",
       " ('mcc_code_5732', 1.759319054523612),\n",
       " ('avg_sum_spent_per_transaction', 1.8262506356479833),\n",
       " ('total', 2.235652218991576),\n",
       " ('mcc_code_7994', 2.2866060090844584),\n",
       " ('mcc_code_5691', 2.5280077606234443),\n",
       " ('mcc_code_7372', 2.5686875962233624),\n",
       " ('mcc_code_5661', 2.6657439258399505),\n",
       " ('day', 2.680257096333392),\n",
       " ('var_time_transaction', 2.947416282837944),\n",
       " ('trans_type_4020', 3.836826927580763),\n",
       " ('var_sum_spent_per_trans', 4.101914867434216),\n",
       " ('total_spent', 4.129548352892742),\n",
       " ('mcc_code_5621', 4.315517076778837),\n",
       " ('mcc_code_5541', 4.793716878983239),\n",
       " ('mcc_code_5977', 7.838646207746995),\n",
       " ('mcc_code_5533', 7.969993106462129)]"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_importance = sorted(list(zip(X.columns,cb.feature_importances_.tolist())), key=itemgetter(1))\n",
    "feature_importance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "# remove all features with importance <= 0.01\n",
    "important_features = [feature for feature, importance in feature_importance if importance > 0.01]\n",
    "X = X[important_features]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [],
   "source": [
    "# params = {\n",
    "#     'iterations': [5,10,15,20,25,30],\n",
    "#     'learning_rate': [0.5, 0.1, 0.05, 0.01],\n",
    "#     'l2_leaf_reg': [0.5, 0.1, 0.05, 0.01],\n",
    "#     'depth': [None,1,2,3,4,5],\n",
    "#     'l2_leaf_reg': [0,0.1,0.01]\n",
    "# }\n",
    "\n",
    "# cb_cv = GridSearchCV(cb, param_grid=params, scoring='roc_auc', cv=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [],
   "source": [
    "# cb_cv.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "# cb_cv.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use best parameters\n",
    "cb = CatBoostClassifier(depth= 4, iterations= 30, l2_leaf_reg= 0.1, learning_rate= 0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:\tlearn: 0.6091104\ttotal: 1.28ms\tremaining: 37.2ms\n",
      "1:\tlearn: 0.5736331\ttotal: 2.81ms\tremaining: 39.4ms\n",
      "2:\tlearn: 0.5514127\ttotal: 4.37ms\tremaining: 39.3ms\n",
      "3:\tlearn: 0.5363634\ttotal: 5.66ms\tremaining: 36.8ms\n",
      "4:\tlearn: 0.5234070\ttotal: 7.06ms\tremaining: 35.3ms\n",
      "5:\tlearn: 0.5125624\ttotal: 8.35ms\tremaining: 33.4ms\n",
      "6:\tlearn: 0.5059650\ttotal: 9.68ms\tremaining: 31.8ms\n",
      "7:\tlearn: 0.5021888\ttotal: 11ms\tremaining: 30.2ms\n",
      "8:\tlearn: 0.4963361\ttotal: 12.7ms\tremaining: 29.5ms\n",
      "9:\tlearn: 0.4908059\ttotal: 14.1ms\tremaining: 28.2ms\n",
      "10:\tlearn: 0.4845095\ttotal: 15.5ms\tremaining: 26.8ms\n",
      "11:\tlearn: 0.4809031\ttotal: 16.9ms\tremaining: 25.3ms\n",
      "12:\tlearn: 0.4777873\ttotal: 18.5ms\tremaining: 24.2ms\n",
      "13:\tlearn: 0.4741133\ttotal: 19.8ms\tremaining: 22.6ms\n",
      "14:\tlearn: 0.4705449\ttotal: 21.2ms\tremaining: 21.2ms\n",
      "15:\tlearn: 0.4666503\ttotal: 22.5ms\tremaining: 19.7ms\n",
      "16:\tlearn: 0.4630552\ttotal: 23.8ms\tremaining: 18.2ms\n",
      "17:\tlearn: 0.4600403\ttotal: 25.4ms\tremaining: 16.9ms\n",
      "18:\tlearn: 0.4562847\ttotal: 26.9ms\tremaining: 15.6ms\n",
      "19:\tlearn: 0.4533599\ttotal: 28.5ms\tremaining: 14.3ms\n",
      "20:\tlearn: 0.4490914\ttotal: 30.1ms\tremaining: 12.9ms\n",
      "21:\tlearn: 0.4450867\ttotal: 31.8ms\tremaining: 11.6ms\n",
      "22:\tlearn: 0.4415902\ttotal: 33.4ms\tremaining: 10.2ms\n",
      "23:\tlearn: 0.4384496\ttotal: 35ms\tremaining: 8.76ms\n",
      "24:\tlearn: 0.4352236\ttotal: 36.9ms\tremaining: 7.37ms\n",
      "25:\tlearn: 0.4332164\ttotal: 38.4ms\tremaining: 5.9ms\n",
      "26:\tlearn: 0.4303484\ttotal: 39.7ms\tremaining: 4.41ms\n",
      "27:\tlearn: 0.4282374\ttotal: 41.1ms\tremaining: 2.94ms\n",
      "28:\tlearn: 0.4253679\ttotal: 42.6ms\tremaining: 1.47ms\n",
      "29:\tlearn: 0.4220845\ttotal: 44ms\tremaining: 0us\n",
      "0:\tlearn: 0.6173286\ttotal: 1.74ms\tremaining: 50.3ms\n",
      "1:\tlearn: 0.5704289\ttotal: 3.34ms\tremaining: 46.7ms\n",
      "2:\tlearn: 0.5448841\ttotal: 4.66ms\tremaining: 41.9ms\n",
      "3:\tlearn: 0.5317393\ttotal: 5.97ms\tremaining: 38.8ms\n",
      "4:\tlearn: 0.5165210\ttotal: 7.21ms\tremaining: 36ms\n",
      "5:\tlearn: 0.5082984\ttotal: 8.6ms\tremaining: 34.4ms\n",
      "6:\tlearn: 0.5000234\ttotal: 10ms\tremaining: 33ms\n",
      "7:\tlearn: 0.4932209\ttotal: 11.4ms\tremaining: 31.3ms\n",
      "8:\tlearn: 0.4872868\ttotal: 12.7ms\tremaining: 29.6ms\n",
      "9:\tlearn: 0.4815171\ttotal: 14.1ms\tremaining: 28.1ms\n",
      "10:\tlearn: 0.4764682\ttotal: 15.4ms\tremaining: 26.5ms\n",
      "11:\tlearn: 0.4734251\ttotal: 16.9ms\tremaining: 25.4ms\n",
      "12:\tlearn: 0.4691651\ttotal: 18.5ms\tremaining: 24.1ms\n",
      "13:\tlearn: 0.4652436\ttotal: 20.1ms\tremaining: 23ms\n",
      "14:\tlearn: 0.4606984\ttotal: 21.6ms\tremaining: 21.6ms\n",
      "15:\tlearn: 0.4567075\ttotal: 22.9ms\tremaining: 20.1ms\n",
      "16:\tlearn: 0.4526792\ttotal: 24.5ms\tremaining: 18.7ms\n",
      "17:\tlearn: 0.4498080\ttotal: 26.1ms\tremaining: 17.4ms\n",
      "18:\tlearn: 0.4460855\ttotal: 27.9ms\tremaining: 16.1ms\n",
      "19:\tlearn: 0.4438180\ttotal: 29.3ms\tremaining: 14.7ms\n",
      "20:\tlearn: 0.4410364\ttotal: 30.9ms\tremaining: 13.2ms\n",
      "21:\tlearn: 0.4378747\ttotal: 32.4ms\tremaining: 11.8ms\n",
      "22:\tlearn: 0.4350591\ttotal: 33.8ms\tremaining: 10.3ms\n",
      "23:\tlearn: 0.4316198\ttotal: 35.5ms\tremaining: 8.86ms\n",
      "24:\tlearn: 0.4281254\ttotal: 37.3ms\tremaining: 7.45ms\n",
      "25:\tlearn: 0.4258907\ttotal: 38.6ms\tremaining: 5.95ms\n",
      "26:\tlearn: 0.4234415\ttotal: 40.1ms\tremaining: 4.45ms\n",
      "27:\tlearn: 0.4208075\ttotal: 41.6ms\tremaining: 2.97ms\n",
      "28:\tlearn: 0.4187562\ttotal: 43.4ms\tremaining: 1.5ms\n",
      "29:\tlearn: 0.4154115\ttotal: 45.1ms\tremaining: 0us\n",
      "0:\tlearn: 0.6124943\ttotal: 1.3ms\tremaining: 37.9ms\n",
      "1:\tlearn: 0.5728303\ttotal: 2.61ms\tremaining: 36.5ms\n",
      "2:\tlearn: 0.5490086\ttotal: 3.85ms\tremaining: 34.7ms\n",
      "3:\tlearn: 0.5332721\ttotal: 5.22ms\tremaining: 33.9ms\n",
      "4:\tlearn: 0.5230945\ttotal: 6.62ms\tremaining: 33.1ms\n",
      "5:\tlearn: 0.5127542\ttotal: 8.09ms\tremaining: 32.4ms\n",
      "6:\tlearn: 0.5038814\ttotal: 9.45ms\tremaining: 31ms\n",
      "7:\tlearn: 0.4964355\ttotal: 10.7ms\tremaining: 29.5ms\n",
      "8:\tlearn: 0.4916129\ttotal: 12.2ms\tremaining: 28.5ms\n",
      "9:\tlearn: 0.4863990\ttotal: 13.6ms\tremaining: 27.2ms\n",
      "10:\tlearn: 0.4822473\ttotal: 15.1ms\tremaining: 26.1ms\n",
      "11:\tlearn: 0.4781778\ttotal: 16.5ms\tremaining: 24.8ms\n",
      "12:\tlearn: 0.4730763\ttotal: 18.1ms\tremaining: 23.7ms\n",
      "13:\tlearn: 0.4678501\ttotal: 19.8ms\tremaining: 22.7ms\n",
      "14:\tlearn: 0.4636562\ttotal: 21.6ms\tremaining: 21.6ms\n",
      "15:\tlearn: 0.4607462\ttotal: 22.9ms\tremaining: 20.1ms\n",
      "16:\tlearn: 0.4556798\ttotal: 24.3ms\tremaining: 18.6ms\n",
      "17:\tlearn: 0.4527589\ttotal: 25.8ms\tremaining: 17.2ms\n",
      "18:\tlearn: 0.4489248\ttotal: 27.3ms\tremaining: 15.8ms\n",
      "19:\tlearn: 0.4455821\ttotal: 28.9ms\tremaining: 14.4ms\n",
      "20:\tlearn: 0.4427835\ttotal: 30.8ms\tremaining: 13.2ms\n",
      "21:\tlearn: 0.4400821\ttotal: 32.3ms\tremaining: 11.7ms\n",
      "22:\tlearn: 0.4373877\ttotal: 34.1ms\tremaining: 10.4ms\n",
      "23:\tlearn: 0.4352760\ttotal: 35.9ms\tremaining: 8.98ms\n",
      "24:\tlearn: 0.4318135\ttotal: 37.3ms\tremaining: 7.46ms\n",
      "25:\tlearn: 0.4292922\ttotal: 39.1ms\tremaining: 6.01ms\n",
      "26:\tlearn: 0.4265616\ttotal: 40.9ms\tremaining: 4.54ms\n",
      "27:\tlearn: 0.4236158\ttotal: 42.5ms\tremaining: 3.04ms\n",
      "28:\tlearn: 0.4213542\ttotal: 43.9ms\tremaining: 1.51ms\n",
      "29:\tlearn: 0.4182284\ttotal: 45.5ms\tremaining: 0us\n",
      "0:\tlearn: 0.6080957\ttotal: 1.56ms\tremaining: 45.2ms\n",
      "1:\tlearn: 0.5663567\ttotal: 2.85ms\tremaining: 39.9ms\n",
      "2:\tlearn: 0.5435618\ttotal: 4.11ms\tremaining: 37ms\n",
      "3:\tlearn: 0.5289980\ttotal: 5.42ms\tremaining: 35.2ms\n",
      "4:\tlearn: 0.5201303\ttotal: 6.64ms\tremaining: 33.2ms\n",
      "5:\tlearn: 0.5116896\ttotal: 7.95ms\tremaining: 31.8ms\n",
      "6:\tlearn: 0.5044693\ttotal: 9.26ms\tremaining: 30.4ms\n",
      "7:\tlearn: 0.5005854\ttotal: 10.7ms\tremaining: 29.5ms\n",
      "8:\tlearn: 0.4920401\ttotal: 12.2ms\tremaining: 28.4ms\n",
      "9:\tlearn: 0.4858899\ttotal: 13.6ms\tremaining: 27.2ms\n",
      "10:\tlearn: 0.4798969\ttotal: 15ms\tremaining: 25.9ms\n",
      "11:\tlearn: 0.4745466\ttotal: 16.7ms\tremaining: 25ms\n",
      "12:\tlearn: 0.4723197\ttotal: 18.2ms\tremaining: 23.8ms\n",
      "13:\tlearn: 0.4674036\ttotal: 19.9ms\tremaining: 22.8ms\n",
      "14:\tlearn: 0.4643204\ttotal: 21.5ms\tremaining: 21.5ms\n",
      "15:\tlearn: 0.4619446\ttotal: 23ms\tremaining: 20.1ms\n",
      "16:\tlearn: 0.4583718\ttotal: 24.4ms\tremaining: 18.6ms\n",
      "17:\tlearn: 0.4552154\ttotal: 25.8ms\tremaining: 17.2ms\n",
      "18:\tlearn: 0.4522823\ttotal: 27.1ms\tremaining: 15.7ms\n",
      "19:\tlearn: 0.4486903\ttotal: 28.7ms\tremaining: 14.3ms\n",
      "20:\tlearn: 0.4465834\ttotal: 30.1ms\tremaining: 12.9ms\n",
      "21:\tlearn: 0.4435514\ttotal: 31.7ms\tremaining: 11.5ms\n",
      "22:\tlearn: 0.4399046\ttotal: 33ms\tremaining: 10ms\n",
      "23:\tlearn: 0.4369216\ttotal: 34.5ms\tremaining: 8.63ms\n",
      "24:\tlearn: 0.4337884\ttotal: 36.1ms\tremaining: 7.22ms\n",
      "25:\tlearn: 0.4305574\ttotal: 37.6ms\tremaining: 5.79ms\n",
      "26:\tlearn: 0.4278570\ttotal: 39.1ms\tremaining: 4.34ms\n",
      "27:\tlearn: 0.4253770\ttotal: 40.8ms\tremaining: 2.91ms\n",
      "28:\tlearn: 0.4229094\ttotal: 42.2ms\tremaining: 1.46ms\n",
      "29:\tlearn: 0.4203186\ttotal: 43.5ms\tremaining: 0us\n"
     ]
    }
   ],
   "source": [
    "cv_results = cross_validate(cb, X, y, scoring='roc_auc', cv=cv_split, return_train_score=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'fit_time': array([0.05416393, 0.05241323, 0.05259705, 0.05024004]),\n",
       " 'score_time': array([0.00280595, 0.00268483, 0.00264287, 0.00287199]),\n",
       " 'test_score': array([0.84246976, 0.8387333 , 0.84902274, 0.84163397]),\n",
       " 'train_score': array([0.88953708, 0.89542985, 0.89180808, 0.89116679])}"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8429649408679287"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv_results['test_score'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Random Forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "mm_scaler = MinMaxScaler()\n",
    "scaled_X = mm_scaler.fit_transform(X)\n",
    "scaled_X = pd.DataFrame(scaled_X, columns = X.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [],
   "source": [
    "rf = RandomForestClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [],
   "source": [
    "# params = {\n",
    "#     'n_estimators': [5,10,15,20,25,30,100],\n",
    "# #     'learning_rate': [0.5, 0.1, 0.05, 0.01],\n",
    "# #     'l2_leaf_reg': [0.5, 0.1, 0.05, 0.01],\n",
    "#     'max_depth': [None,1,2,3,4,5],\n",
    "#     'min_samples_split': [2,3,5,10],\n",
    "#     'min_samples_leaf': [1,3,5,10],\n",
    "# #     'l2_leaf_reg': [0,0.1,0.01]\n",
    "# #     n_estimators=100,\n",
    "# #     *,\n",
    "# #     criterion='gini',\n",
    "# }\n",
    "\n",
    "# rf_cv = GridSearchCV(rf, param_grid=params, scoring='roc_auc', cv=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [],
   "source": [
    "# rf_cv.fit(scaled_X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [],
   "source": [
    "# rf_cv.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use best parameters\n",
    "rf = RandomForestClassifier(max_depth= None, min_samples_leaf= 5, min_samples_split= 10, n_estimators= 100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [],
   "source": [
    "rf_results = cross_validate(rf, X, y, scoring='roc_auc', cv=cv_split, return_train_score=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'fit_time': array([0.66046214, 0.67192507, 0.64102006, 0.63527298]),\n",
       " 'score_time': array([0.01722908, 0.01693106, 0.01677799, 0.01695108]),\n",
       " 'test_score': array([0.8228968 , 0.82620064, 0.82574311, 0.82761491]),\n",
       " 'train_score': array([0.98632146, 0.98528191, 0.98487667, 0.98546825])}"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8256138653476028"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_results['test_score'].mean()"
   ]
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}