dota 2 prediction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Игра Dota 2\n",
    "\n",
    "[Dota 2](https://ru.wikipedia.org/wiki/Dota_2) — многопользовательская компьютерная игра жанра [MOBA](https://ru.wikipedia.org/wiki/MOBA). Игроки играют между собой матчи. В каждом матче участвует две команды, 5 человек в каждой. Одна команда играет за светлую сторону (The Radiant), другая — за тёмную (The Dire). Цель каждой команды — уничтожить главное здание базы противника (трон).\n",
    "\n",
    "Существуют [разные режимы игры](http://dota2.gamepedia.com/Game_modes/ru), мы будем рассматривать режим [Captain's Mode](http://dota2.gamepedia.com/Game_modes/ru#Captain.27s_Mode), в формате которого происходит большая часть киберспортивных мероприятий по Dota 2.\n",
    "\n",
    "### Как проходит матч\n",
    "\n",
    "#### 1. Игроки выбирают героев\n",
    "\n",
    "Всего в игре чуть более 100 различных героев (персонажей). В начале игры, команды в определенном порядке выбирают героев себе и запрещают выбирать определенных героев противнику (баны). Каждый игрок будет управлять одним героем, в рамках одного матча не может быть несколько одинаковых героев.  Герои различаются между собой своими характеристиками и способностями. От комбинации выбранных героев во многом зависит успех команды.\n",
    "\n",
    "\n",
    "\n",
    "#### 2. Основная часть\n",
    "\n",
    "Игроки могут получать золото и опыт за убийство чужих героев или прочих юнитов. Накопленный опыт влияет на уровень героя, который в свою очередь позволяет улучшать способности. За накопленное золото игроки покупают предметы, которые улучшают характеристики героев или дают им новые способности.\n",
    "\n",
    "После смерти герой отправляется в \"таверну\" и возрождается только по прошествии некоторого времени, таким образом команда на некоторое время теряет игрока, однако игрок может досрочно выкупить героя из таверны за определенную сумму золота.\n",
    "\n",
    "В течение игры команды развивают своих героев, обороняют свою часть поля и нападают на вражескую.\n",
    "\n",
    "\n",
    "\n",
    "#### 3. Конец игры\n",
    "\n",
    "Игра заканчивается, когда одна из команд разрушет определенное число \"башен\" противника и уничтожает трон.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Задача: предсказание победы по данным о первых 5 минутах игры\n",
    "\n",
    "По первым 5 минутам игры предсказать, какая из команд победит: Radiant или Dire?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Данные\n",
    "\n",
    "Данные `matches.jsonlines.bz2`.\n",
    "В  `dictionaries` приведены расшифровки идентификаторов,  присутствующих в записях матчей."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import bz2\n",
    "\n",
    "with bz2.BZ2File('./matches.jsonlines.bz2') as matches_file:\n",
    "    for line in matches_file:\n",
    "        match = json.loads(line)\n",
    "        \n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Признаки\n",
    "\n",
    "\n",
    "Признаки данны в `features.csv`. Признаки в файле `features.csv`  информацию, известную про матч за первые 5 игровых минут. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Пример чтения файла с признаками"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>start_time</th>\n",
       "      <th>lobby_type</th>\n",
       "      <th>r1_hero</th>\n",
       "      <th>r1_level</th>\n",
       "      <th>r1_xp</th>\n",
       "      <th>r1_gold</th>\n",
       "      <th>r1_lh</th>\n",
       "      <th>r1_kills</th>\n",
       "      <th>r1_deaths</th>\n",
       "      <th>r1_items</th>\n",
       "      <th>...</th>\n",
       "      <th>dire_boots_count</th>\n",
       "      <th>dire_ward_observer_count</th>\n",
       "      <th>dire_ward_sentry_count</th>\n",
       "      <th>dire_first_ward_time</th>\n",
       "      <th>duration</th>\n",
       "      <th>radiant_win</th>\n",
       "      <th>tower_status_radiant</th>\n",
       "      <th>tower_status_dire</th>\n",
       "      <th>barracks_status_radiant</th>\n",
       "      <th>barracks_status_dire</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>match_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1430198770</td>\n",
       "      <td>7</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>2098</td>\n",
       "      <td>1489</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>-52.0</td>\n",
       "      <td>2874</td>\n",
       "      <td>1</td>\n",
       "      <td>1796</td>\n",
       "      <td>0</td>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1430220345</td>\n",
       "      <td>0</td>\n",
       "      <td>42</td>\n",
       "      <td>4</td>\n",
       "      <td>1188</td>\n",
       "      <td>1033</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>-5.0</td>\n",
       "      <td>2463</td>\n",
       "      <td>1</td>\n",
       "      <td>1974</td>\n",
       "      <td>0</td>\n",
       "      <td>63</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1430227081</td>\n",
       "      <td>7</td>\n",
       "      <td>33</td>\n",
       "      <td>4</td>\n",
       "      <td>1319</td>\n",
       "      <td>1270</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>13.0</td>\n",
       "      <td>2130</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1830</td>\n",
       "      <td>0</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1430263531</td>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4</td>\n",
       "      <td>1779</td>\n",
       "      <td>1056</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>1459</td>\n",
       "      <td>0</td>\n",
       "      <td>1920</td>\n",
       "      <td>2047</td>\n",
       "      <td>50</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1430282290</td>\n",
       "      <td>7</td>\n",
       "      <td>13</td>\n",
       "      <td>4</td>\n",
       "      <td>1431</td>\n",
       "      <td>1090</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>-16.0</td>\n",
       "      <td>2449</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1974</td>\n",
       "      <td>3</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 108 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          start_time  lobby_type  r1_hero  r1_level  r1_xp  r1_gold  r1_lh  \\\n",
       "match_id                                                                     \n",
       "0         1430198770           7       11         5   2098     1489     20   \n",
       "1         1430220345           0       42         4   1188     1033      9   \n",
       "2         1430227081           7       33         4   1319     1270     22   \n",
       "3         1430263531           1       29         4   1779     1056     14   \n",
       "4         1430282290           7       13         4   1431     1090      8   \n",
       "\n",
       "          r1_kills  r1_deaths  r1_items          ...           \\\n",
       "match_id                                         ...            \n",
       "0                0          0         7          ...            \n",
       "1                0          1        12          ...            \n",
       "2                0          0        12          ...            \n",
       "3                0          0         5          ...            \n",
       "4                1          0         8          ...            \n",
       "\n",
       "          dire_boots_count  dire_ward_observer_count  dire_ward_sentry_count  \\\n",
       "match_id                                                                       \n",
       "0                        4                         2                       2   \n",
       "1                        4                         3                       1   \n",
       "2                        4                         3                       1   \n",
       "3                        4                         2                       0   \n",
       "4                        3                         3                       0   \n",
       "\n",
       "          dire_first_ward_time  duration  radiant_win  tower_status_radiant  \\\n",
       "match_id                                                                      \n",
       "0                        -52.0      2874            1                  1796   \n",
       "1                         -5.0      2463            1                  1974   \n",
       "2                         13.0      2130            0                     0   \n",
       "3                         27.0      1459            0                  1920   \n",
       "4                        -16.0      2449            0                     4   \n",
       "\n",
       "          tower_status_dire  barracks_status_radiant  barracks_status_dire  \n",
       "match_id                                                                    \n",
       "0                         0                       51                     0  \n",
       "1                         0                       63                     1  \n",
       "2                      1830                        0                    63  \n",
       "3                      2047                       50                    63  \n",
       "4                      1974                        3                    63  \n",
       "\n",
       "[5 rows x 108 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas\n",
    "features = pandas.read_csv('./features.csv', index_col='match_id')\n",
    "\n",
    "features.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Описание признаков в таблице\n",
    "\n",
    "- `match_id`: идентификатор матча в наборе данных\n",
    "- `start_time`: время начала матча (unixtime)\n",
    "- `lobby_type`: тип комнаты, в которой собираются игроки (расшифровка в `dictionaries/lobbies.csv`)\n",
    "- Наборы признаков для каждого игрока (игроки команды Radiant — префикс `rN`, Dire — `dN`):\n",
    "    - `r1_hero`: герой игрока (расшифровка в dictionaries/heroes.csv)\n",
    "    - `r1_level`: максимальный достигнутый уровень героя (за первые 5 игровых минут)\n",
    "    - `r1_xp`: максимальный полученный опыт\n",
    "    - `r1_gold`: достигнутая ценность героя\n",
    "    - `r1_lh`: число убитых юнитов\n",
    "    - `r1_kills`: число убитых игроков\n",
    "    - `r1_deaths`: число смертей героя\n",
    "    - `r1_items`: число купленных предметов\n",
    "- Признаки события \"первая кровь\" (first blood). Если событие \"первая кровь\" не успело произойти за первые 5 минут, то признаки принимают пропущенное значение\n",
    "    - `first_blood_time`: игровое время первой крови\n",
    "    - `first_blood_team`: команда, совершившая первую кровь (0 — Radiant, 1 — Dire)\n",
    "    - `first_blood_player1`: игрок, причастный к событию\n",
    "    - `first_blood_player2`: второй игрок, причастный к событию\n",
    "- Признаки для каждой команды (префиксы `radiant_` и `dire_`)\n",
    "    - `radiant_bottle_time`: время первого приобретения командой предмета \"bottle\"\n",
    "    - `radiant_courier_time`: время приобретения предмета \"courier\" \n",
    "    - `radiant_flying_courier_time`: время приобретения предмета \"flying_courier\" \n",
    "    - `radiant_tpscroll_count`: число предметов \"tpscroll\" за первые 5 минут\n",
    "    - `radiant_boots_count`: число предметов \"boots\"\n",
    "    - `radiant_ward_observer_count`: число предметов \"ward_observer\"\n",
    "    - `radiant_ward_sentry_count`: число предметов \"ward_sentry\"\n",
    "    - `radiant_first_ward_time`: время установки командой первого \"наблюдателя\", т.е. предмета, который позволяет видеть часть игрового поля\n",
    "- Итог матча (данные поля отсутствуют в тестовой выборке, поскольку содержат информацию, выходящую за пределы первых 5 минут матча)\n",
    "    - `duration`: длительность\n",
    "    - `radiant_win`: 1, если победила команда Radiant, 0 — иначе\n",
    "    - Состояние башен и барраков к концу матча (см. описание полей набора данных)\n",
    "        - `tower_status_radiant`\n",
    "        - `tower_status_dire`\n",
    "        - `barracks_status_radiant`\n",
    "        - `barracks_status_dire`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "import datetime\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.linear_model import LogisticRegression, LogisticRegressionCV\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>start_time</th>\n",
       "      <th>lobby_type</th>\n",
       "      <th>r1_hero</th>\n",
       "      <th>r1_level</th>\n",
       "      <th>r1_xp</th>\n",
       "      <th>r1_gold</th>\n",
       "      <th>r1_lh</th>\n",
       "      <th>r1_kills</th>\n",
       "      <th>r1_deaths</th>\n",
       "      <th>r1_items</th>\n",
       "      <th>...</th>\n",
       "      <th>dire_boots_count</th>\n",
       "      <th>dire_ward_observer_count</th>\n",
       "      <th>dire_ward_sentry_count</th>\n",
       "      <th>dire_first_ward_time</th>\n",
       "      <th>duration</th>\n",
       "      <th>radiant_win</th>\n",
       "      <th>tower_status_radiant</th>\n",
       "      <th>tower_status_dire</th>\n",
       "      <th>barracks_status_radiant</th>\n",
       "      <th>barracks_status_dire</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>match_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1430198770</td>\n",
       "      <td>7</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>2098</td>\n",
       "      <td>1489</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>-52.0</td>\n",
       "      <td>2874</td>\n",
       "      <td>1</td>\n",
       "      <td>1796</td>\n",
       "      <td>0</td>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1430220345</td>\n",
       "      <td>0</td>\n",
       "      <td>42</td>\n",
       "      <td>4</td>\n",
       "      <td>1188</td>\n",
       "      <td>1033</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>-5.0</td>\n",
       "      <td>2463</td>\n",
       "      <td>1</td>\n",
       "      <td>1974</td>\n",
       "      <td>0</td>\n",
       "      <td>63</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1430227081</td>\n",
       "      <td>7</td>\n",
       "      <td>33</td>\n",
       "      <td>4</td>\n",
       "      <td>1319</td>\n",
       "      <td>1270</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>13.0</td>\n",
       "      <td>2130</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1830</td>\n",
       "      <td>0</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1430263531</td>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4</td>\n",
       "      <td>1779</td>\n",
       "      <td>1056</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>1459</td>\n",
       "      <td>0</td>\n",
       "      <td>1920</td>\n",
       "      <td>2047</td>\n",
       "      <td>50</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1430282290</td>\n",
       "      <td>7</td>\n",
       "      <td>13</td>\n",
       "      <td>4</td>\n",
       "      <td>1431</td>\n",
       "      <td>1090</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>-16.0</td>\n",
       "      <td>2449</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1974</td>\n",
       "      <td>3</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 108 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          start_time  lobby_type  r1_hero  r1_level  r1_xp  r1_gold  r1_lh  \\\n",
       "match_id                                                                     \n",
       "0         1430198770           7       11         5   2098     1489     20   \n",
       "1         1430220345           0       42         4   1188     1033      9   \n",
       "2         1430227081           7       33         4   1319     1270     22   \n",
       "3         1430263531           1       29         4   1779     1056     14   \n",
       "4         1430282290           7       13         4   1431     1090      8   \n",
       "\n",
       "          r1_kills  r1_deaths  r1_items          ...           \\\n",
       "match_id                                         ...            \n",
       "0                0          0         7          ...            \n",
       "1                0          1        12          ...            \n",
       "2                0          0        12          ...            \n",
       "3                0          0         5          ...            \n",
       "4                1          0         8          ...            \n",
       "\n",
       "          dire_boots_count  dire_ward_observer_count  dire_ward_sentry_count  \\\n",
       "match_id                                                                       \n",
       "0                        4                         2                       2   \n",
       "1                        4                         3                       1   \n",
       "2                        4                         3                       1   \n",
       "3                        4                         2                       0   \n",
       "4                        3                         3                       0   \n",
       "\n",
       "          dire_first_ward_time  duration  radiant_win  tower_status_radiant  \\\n",
       "match_id                                                                      \n",
       "0                        -52.0      2874            1                  1796   \n",
       "1                         -5.0      2463            1                  1974   \n",
       "2                         13.0      2130            0                     0   \n",
       "3                         27.0      1459            0                  1920   \n",
       "4                        -16.0      2449            0                     4   \n",
       "\n",
       "          tower_status_dire  barracks_status_radiant  barracks_status_dire  \n",
       "match_id                                                                    \n",
       "0                         0                       51                     0  \n",
       "1                         0                       63                     1  \n",
       "2                      1830                        0                    63  \n",
       "3                      2047                       50                    63  \n",
       "4                      1974                        3                    63  \n",
       "\n",
       "[5 rows x 108 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features = pd.read_csv('./features.csv', index_col='match_id')\n",
    "features.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>start_time</th>\n",
       "      <th>lobby_type</th>\n",
       "      <th>r1_hero</th>\n",
       "      <th>r1_level</th>\n",
       "      <th>r1_xp</th>\n",
       "      <th>r1_gold</th>\n",
       "      <th>r1_lh</th>\n",
       "      <th>r1_kills</th>\n",
       "      <th>r1_deaths</th>\n",
       "      <th>r1_items</th>\n",
       "      <th>...</th>\n",
       "      <th>radiant_ward_sentry_count</th>\n",
       "      <th>radiant_first_ward_time</th>\n",
       "      <th>dire_bottle_time</th>\n",
       "      <th>dire_courier_time</th>\n",
       "      <th>dire_flying_courier_time</th>\n",
       "      <th>dire_tpscroll_count</th>\n",
       "      <th>dire_boots_count</th>\n",
       "      <th>dire_ward_observer_count</th>\n",
       "      <th>dire_ward_sentry_count</th>\n",
       "      <th>dire_first_ward_time</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>match_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1430198770</td>\n",
       "      <td>7</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>2098</td>\n",
       "      <td>1489</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>35.0</td>\n",
       "      <td>103.0</td>\n",
       "      <td>-84.0</td>\n",
       "      <td>221.0</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>-52.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1430220345</td>\n",
       "      <td>0</td>\n",
       "      <td>42</td>\n",
       "      <td>4</td>\n",
       "      <td>1188</td>\n",
       "      <td>1033</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-20.0</td>\n",
       "      <td>149.0</td>\n",
       "      <td>-84.0</td>\n",
       "      <td>195.0</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>-5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1430227081</td>\n",
       "      <td>7</td>\n",
       "      <td>33</td>\n",
       "      <td>4</td>\n",
       "      <td>1319</td>\n",
       "      <td>1270</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>-39.0</td>\n",
       "      <td>45.0</td>\n",
       "      <td>-77.0</td>\n",
       "      <td>221.0</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>13.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1430263531</td>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4</td>\n",
       "      <td>1779</td>\n",
       "      <td>1056</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>-30.0</td>\n",
       "      <td>124.0</td>\n",
       "      <td>-80.0</td>\n",
       "      <td>184.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>27.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1430282290</td>\n",
       "      <td>7</td>\n",
       "      <td>13</td>\n",
       "      <td>4</td>\n",
       "      <td>1431</td>\n",
       "      <td>1090</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>182.0</td>\n",
       "      <td>-80.0</td>\n",
       "      <td>225.0</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>-16.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 102 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          start_time  lobby_type  r1_hero  r1_level  r1_xp  r1_gold  r1_lh  \\\n",
       "match_id                                                                     \n",
       "0         1430198770           7       11         5   2098     1489     20   \n",
       "1         1430220345           0       42         4   1188     1033      9   \n",
       "2         1430227081           7       33         4   1319     1270     22   \n",
       "3         1430263531           1       29         4   1779     1056     14   \n",
       "4         1430282290           7       13         4   1431     1090      8   \n",
       "\n",
       "          r1_kills  r1_deaths  r1_items          ...           \\\n",
       "match_id                                         ...            \n",
       "0                0          0         7          ...            \n",
       "1                0          1        12          ...            \n",
       "2                0          0        12          ...            \n",
       "3                0          0         5          ...            \n",
       "4                1          0         8          ...            \n",
       "\n",
       "          radiant_ward_sentry_count  radiant_first_ward_time  \\\n",
       "match_id                                                       \n",
       "0                                 0                     35.0   \n",
       "1                                 0                    -20.0   \n",
       "2                                 1                    -39.0   \n",
       "3                                 0                    -30.0   \n",
       "4                                 0                     46.0   \n",
       "\n",
       "          dire_bottle_time  dire_courier_time  dire_flying_courier_time  \\\n",
       "match_id                                                                  \n",
       "0                    103.0              -84.0                     221.0   \n",
       "1                    149.0              -84.0                     195.0   \n",
       "2                     45.0              -77.0                     221.0   \n",
       "3                    124.0              -80.0                     184.0   \n",
       "4                    182.0              -80.0                     225.0   \n",
       "\n",
       "          dire_tpscroll_count  dire_boots_count  dire_ward_observer_count  \\\n",
       "match_id                                                                    \n",
       "0                           3                 4                         2   \n",
       "1                           5                 4                         3   \n",
       "2                           3                 4                         3   \n",
       "3                           0                 4                         2   \n",
       "4                           6                 3                         3   \n",
       "\n",
       "          dire_ward_sentry_count  dire_first_ward_time  \n",
       "match_id                                                \n",
       "0                              2                 -52.0  \n",
       "1                              1                  -5.0  \n",
       "2                              1                  13.0  \n",
       "3                              0                  27.0  \n",
       "4                              0                 -16.0  \n",
       "\n",
       "[5 rows x 102 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features_to_drop = ['duration', 'radiant_win', 'tower_status_radiant', 'tower_status_dire', \\\n",
    "                  'barracks_status_radiant', 'barracks_status_dire']\n",
    "\n",
    "y = features['radiant_win'].values\n",
    "\n",
    "features = features.drop(features_to_drop, axis = 1)\n",
    "features.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "first_blood_time\n",
      "first_blood_team\n",
      "first_blood_player1\n",
      "first_blood_player2\n",
      "radiant_bottle_time\n",
      "radiant_courier_time\n",
      "radiant_flying_courier_time\n",
      "radiant_first_ward_time\n",
      "dire_bottle_time\n",
      "dire_courier_time\n",
      "dire_flying_courier_time\n",
      "dire_first_ward_time\n"
     ]
    }
   ],
   "source": [
    "count_rows = features.shape[0]\n",
    "for (i , count) in enumerate(features.count()):\n",
    "    if count != count_rows:\n",
    "        print(features.columns[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = features.fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Count trees: 10\n",
      "0.660797865361\n",
      "Time elapsed: 0:00:42.687378\n",
      "\n",
      "# Count trees: 20\n",
      "0.681792174816\n",
      "Time elapsed: 0:01:13.919352\n",
      "\n",
      "# Count trees: 30\n",
      "0.6870462783\n",
      "Time elapsed: 0:01:44.433548\n",
      "\n"
     ]
    }
   ],
   "source": [
    "kf = KFold(n_splits = 5, shuffle = True)\n",
    "\n",
    "for count_trees in [10, 20, 30]: \n",
    "    print('# Count trees:', count_trees)\n",
    "    start_time = datetime.datetime.now()\n",
    "    \n",
    "    clf = GradientBoostingClassifier(n_estimators=count_trees)\n",
    "    scores = []\n",
    "    X = features.values\n",
    "    \n",
    "    for train_index, test_index in kf.split(X):\n",
    "        X_train, X_test = X[train_index], X[test_index]\n",
    "        y_train, y_test = y[train_index], y[test_index]\n",
    "        \n",
    "        clf  = clf.fit(X_train, y_train)\n",
    "        pred = clf.predict_proba(X_test)[:, 1]\n",
    "        scores.append(roc_auc_score(y_test, pred))   \n",
    "        \n",
    "    print(np.median(scores))\n",
    "    print('Time elapsed:', datetime.datetime.now() - start_time)\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_test = pd.read_csv('./features_test.csv', index_col='match_id')\n",
    "X_train = StandardScaler().fit_transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_test = X_test.fillna(0)\n",
    "\n",
    "X_pick = np.zeros((X_test.shape[0], 112))\n",
    "for i, match_id in enumerate(X_test.index):\n",
    "    for p in range(5):\n",
    "        X_pick[i, X_test.loc[match_id, 'r%d_hero' % (p+1)]-1] = 1\n",
    "        X_pick[i, X_test.loc[match_id, 'd%d_hero' % (p+1)]-1] = -1\n",
    "\n",
    "X_test = X_test.drop(cat_features_to_delete, axis=1)\n",
    "X_test = np.hstack((X_test.values, X_pick))\n",
    "X_test = StandardScaler().fit_transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.82499721,  0.75855499,  0.18698925, ...,  0.23114303,\n",
       "        0.63195917,  0.42597692])"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf = LogisticRegression(C=best_c)\n",
    "clf = clf.fit(X_train, y)\n",
    "pred = clf.predict_proba(X_test)[:, 1]\n",
    "pred"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}