Day_024_CountEncoder_and_FeatureHash_Ans.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業 : (Kaggle)鐵達尼生存預測\n",
    "https://www.kaggle.com/c/titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [作業目標]\n",
    "- 試著模仿範例寫法, 在鐵達尼生存預測中, 觀察計數編碼與特徵雜湊的效果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [作業重點]\n",
    "- 仿造範例, 完成計數編碼以及搭配邏輯斯迴歸的預測 (In[4], Out[4], In[5], Out[5]) \n",
    "- 仿造範例, 完成雜湊編碼, 以及計數編碼+雜湊編碼 搭配邏輯斯迴歸的預測 (In[6], Out[6], In[7], Out[7]) \n",
    "- 試著回答上述執行結果的觀察"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業1\n",
    "* 參考範例，將鐵達尼的艙位代碼( 'Cabin' )欄位使用特徵雜湊 / 標籤編碼 / 目標均值編碼三種轉換後，  \n",
    "與其他數值型欄位一起預估生存機率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Pclass                                               Name     Sex   Age  \\\n",
       "0       3                            Braund, Mr. Owen Harris    male  22.0   \n",
       "1       1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0   \n",
       "2       3                             Heikkinen, Miss. Laina  female  26.0   \n",
       "3       1       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0   \n",
       "4       3                           Allen, Mr. William Henry    male  35.0   \n",
       "\n",
       "   SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "0      1      0         A/5 21171   7.2500   NaN        S  \n",
       "1      1      0          PC 17599  71.2833   C85        C  \n",
       "2      0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      1      0            113803  53.1000  C123        S  \n",
       "4      0      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 做完特徵工程前的所有準備 (與前範例相同)\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import copy\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "data_path = 'data/'\n",
    "df_train = pd.read_csv(data_path + 'titanic_train.csv')\n",
    "df_test = pd.read_csv(data_path + 'titanic_test.csv')\n",
    "\n",
    "train_Y = df_train['Survived']\n",
    "ids = df_test['PassengerId']\n",
    "df_train = df_train.drop(['PassengerId', 'Survived'] , axis=1)\n",
    "df_test = df_test.drop(['PassengerId'] , axis=1)\n",
    "df = pd.concat([df_train,df_test])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5 Numeric Features : ['Name', 'Sex', 'Ticket', 'Cabin', 'Embarked']\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>None</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>None</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>113803</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>373450</td>\n",
       "      <td>None</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Name     Sex  \\\n",
       "0                            Braund, Mr. Owen Harris    male   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female   \n",
       "2                             Heikkinen, Miss. Laina  female   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female   \n",
       "4                           Allen, Mr. William Henry    male   \n",
       "\n",
       "             Ticket Cabin Embarked  \n",
       "0         A/5 21171  None        S  \n",
       "1          PC 17599   C85        C  \n",
       "2  STON/O2. 3101282  None        S  \n",
       "3            113803  C123        S  \n",
       "4            373450  None        S  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#只取類別值 (object) 型欄位, 存於 object_features 中\n",
    "object_features = []\n",
    "for dtype, feature in zip(df.dtypes, df.columns):\n",
    "    if dtype == 'object':\n",
    "        object_features.append(feature)\n",
    "print(f'{len(object_features)} Numeric Features : {object_features}\\n')\n",
    "\n",
    "# 只留類別型欄位\n",
    "df = df[object_features]\n",
    "df = df.fillna('None')\n",
    "train_num = train_Y.shape[0]\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業2\n",
    "* 承上題，三者比較效果何者最好?\n",
    "> 估計效果 : 計數編碼 > 計數編碼 + 特徵雜湊 > 特徵雜湊 > 標籤編碼  \n",
    "所以可以看出, 計數編碼是相對比較有效的特徵  \n",
    "特徵雜湊依照不同的Hash空間, 效果未必理想, 只是資訊比較密集, 計算上較快  \n",
    "混合使用也未必比只有計數編碼的效果好"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.780004837244799\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>155</td>\n",
       "      <td>1</td>\n",
       "      <td>720</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>286</td>\n",
       "      <td>0</td>\n",
       "      <td>816</td>\n",
       "      <td>106</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>523</td>\n",
       "      <td>0</td>\n",
       "      <td>914</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>422</td>\n",
       "      <td>0</td>\n",
       "      <td>65</td>\n",
       "      <td>70</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>649</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Name  Sex  Ticket  Cabin  Embarked\n",
       "0   155    1     720    185         3\n",
       "1   286    0     816    106         0\n",
       "2   523    0     914    185         3\n",
       "3   422    0      65     70         3\n",
       "4    22    1     649    185         3"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 對照組 : 標籤編碼 + 邏輯斯迴歸\n",
    "df_temp = pd.DataFrame()\n",
    "for c in df.columns:\n",
    "    df_temp[c] = LabelEncoder().fit_transform(df[c])\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = LogisticRegression()\n",
    "print(cross_val_score(estimator, train_X, train_Y, cv=5).mean())\n",
    "df_temp.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Python\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:2: FutureWarning: using a dict on a Series for aggregation\n",
      "is deprecated and will be removed in a future version\n",
      "  \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Cabin_Count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>185</th>\n",
       "      <td>None</td>\n",
       "      <td>1014</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>80</th>\n",
       "      <td>C23 C25 C27</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>184</th>\n",
       "      <td>G6</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>B57 B59 B63 B66</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>B96 B98</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>180</th>\n",
       "      <td>F2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>117</th>\n",
       "      <td>D</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>C22 C26</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>181</th>\n",
       "      <td>F33</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>183</th>\n",
       "      <td>F4</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Cabin  Cabin_Count\n",
       "185             None         1014\n",
       "80       C23 C25 C27            6\n",
       "184               G6            5\n",
       "47   B57 B59 B63 B66            5\n",
       "60           B96 B98            4\n",
       "180               F2            4\n",
       "117                D            4\n",
       "79           C22 C26            4\n",
       "181              F33            4\n",
       "183               F4            4"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 加上 'Cabin' 欄位的計數編碼\n",
    "count_df = df.groupby(['Cabin'])['Name'].agg({'Cabin_Count':'size'}).reset_index()\n",
    "df = pd.merge(df, count_df, on=['Cabin'], how='left')\n",
    "count_df.sort_values(by=['Cabin_Count'], ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7856230275549181\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Cabin_Count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>155</td>\n",
       "      <td>1</td>\n",
       "      <td>720</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "      <td>1014</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>286</td>\n",
       "      <td>0</td>\n",
       "      <td>816</td>\n",
       "      <td>106</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>523</td>\n",
       "      <td>0</td>\n",
       "      <td>914</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "      <td>1014</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>422</td>\n",
       "      <td>0</td>\n",
       "      <td>65</td>\n",
       "      <td>70</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>649</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "      <td>1014</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Name  Sex  Ticket  Cabin  Embarked  Cabin_Count\n",
       "0   155    1     720    185         3         1014\n",
       "1   286    0     816    106         0            2\n",
       "2   523    0     914    185         3         1014\n",
       "3   422    0      65     70         3            2\n",
       "4    22    1     649    185         3         1014"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 計數編碼 + 邏輯斯迴歸\n",
    "df_temp = pd.DataFrame()\n",
    "for c in object_features:\n",
    "    df_temp[c] = LabelEncoder().fit_transform(df[c])\n",
    "df_temp['Cabin_Count'] = df['Cabin_Count']\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = LogisticRegression()\n",
    "print(cross_val_score(estimator, train_X, train_Y, cv=5).mean())\n",
    "df_temp.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7766277027307382\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Cabin_Hash</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>155</td>\n",
       "      <td>1</td>\n",
       "      <td>720</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>286</td>\n",
       "      <td>0</td>\n",
       "      <td>816</td>\n",
       "      <td>106</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>523</td>\n",
       "      <td>0</td>\n",
       "      <td>914</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>422</td>\n",
       "      <td>0</td>\n",
       "      <td>65</td>\n",
       "      <td>70</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>649</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Name  Sex  Ticket  Cabin  Embarked  Cabin_Hash\n",
       "0   155    1     720    185         3           3\n",
       "1   286    0     816    106         0           1\n",
       "2   523    0     914    185         3           3\n",
       "3   422    0      65     70         3           3\n",
       "4    22    1     649    185         3           3"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 特徵雜湊 + 邏輯斯迴歸\n",
    "df_temp = pd.DataFrame()\n",
    "for c in object_features:\n",
    "    df_temp[c] = LabelEncoder().fit_transform(df[c])\n",
    "df_temp['Cabin_Hash'] = df['Cabin'].map(lambda x:hash(x) % 5)\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = LogisticRegression()\n",
    "print(cross_val_score(estimator, train_X, train_Y, cv=5).mean())\n",
    "df_temp.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7878576644264266\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Cabin_Hash</th>\n",
       "      <th>Cabin_Count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>155</td>\n",
       "      <td>1</td>\n",
       "      <td>720</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1014</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>286</td>\n",
       "      <td>0</td>\n",
       "      <td>816</td>\n",
       "      <td>106</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>523</td>\n",
       "      <td>0</td>\n",
       "      <td>914</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1014</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>422</td>\n",
       "      <td>0</td>\n",
       "      <td>65</td>\n",
       "      <td>70</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>22</td>\n",
       "      <td>1</td>\n",
       "      <td>649</td>\n",
       "      <td>185</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>1014</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Name  Sex  Ticket  Cabin  Embarked  Cabin_Hash  Cabin_Count\n",
       "0   155    1     720    185         3           3         1014\n",
       "1   286    0     816    106         0           1            2\n",
       "2   523    0     914    185         3           3         1014\n",
       "3   422    0      65     70         3           3            2\n",
       "4    22    1     649    185         3           3         1014"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 計數編碼 + 特徵雜湊 + 邏輯斯迴歸\n",
    "df_temp = pd.DataFrame()\n",
    "for c in object_features:\n",
    "    df_temp[c] = LabelEncoder().fit_transform(df[c])\n",
    "df_temp['Cabin_Hash'] = df['Cabin'].map(lambda x:hash(x) % 5)\n",
    "df_temp['Cabin_Count'] = df['Cabin_Count']\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = LogisticRegression()\n",
    "print(cross_val_score(estimator, train_X, train_Y, cv=5).mean())\n",
    "df_temp.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}