Day_027_HW.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業 : (Kaggle)鐵達尼生存預測\n",
    "***\n",
    "https://www.kaggle.com/c/titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [作業目標]\n",
    "- 試著模仿範例寫法, 在鐵達尼生存預測中, 觀察計數編碼與特徵雜湊的效果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [作業重點]\n",
    "- 仿造範例, 完成自己挑選特徵的群聚編碼 (In[2], Out[2])\n",
    "- 觀察群聚編碼, 搭配邏輯斯回歸, 看看有什麼影響 (In[5], Out[5], In[6], Out[6]) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業1\n",
    "* 試著使用鐵達尼號的例子，創立兩種以上的群聚編碼特徵( mean、median、mode、max、min、count 均可 )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Pclass                                               Name     Sex   Age  \\\n",
       "0       3                            Braund, Mr. Owen Harris    male  22.0   \n",
       "1       1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0   \n",
       "2       3                             Heikkinen, Miss. Laina  female  26.0   \n",
       "3       1       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0   \n",
       "4       3                           Allen, Mr. William Henry    male  35.0   \n",
       "\n",
       "   SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "0      1      0         A/5 21171   7.2500   NaN        S  \n",
       "1      1      0          PC 17599  71.2833   C85        C  \n",
       "2      0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3      1      0            113803  53.1000  C123        S  \n",
       "4      0      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 做完特徵工程前的所有準備 (與前範例相同)\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import copy\n",
    "from sklearn.preprocessing import LabelEncoder, MinMaxScaler\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "data_path = 'data/'\n",
    "df = pd.read_csv(data_path + 'titanic_train.csv')\n",
    "\n",
    "train_Y = df['Survived']\n",
    "df = df.drop(['PassengerId', 'Survived'] , axis=1)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Fare_Mean</th>\n",
       "      <th>Fare_Mode</th>\n",
       "      <th>Fare_Median</th>\n",
       "      <th>Fare_Max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>84.154687</td>\n",
       "      <td>26.55</td>\n",
       "      <td>60.2875</td>\n",
       "      <td>512.3292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>20.662183</td>\n",
       "      <td>13.00</td>\n",
       "      <td>14.2500</td>\n",
       "      <td>73.5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>13.675550</td>\n",
       "      <td>8.05</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>69.5500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Pclass  Fare_Mean  Fare_Mode  Fare_Median  Fare_Max\n",
       "0       1  84.154687      26.55      60.2875  512.3292\n",
       "1       2  20.662183      13.00      14.2500   73.5000\n",
       "2       3  13.675550       8.05       8.0500   69.5500"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "      <th>Fare_Mean</th>\n",
       "      <th>Fare_Mode</th>\n",
       "      <th>Fare_Median</th>\n",
       "      <th>Fare_Max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>13.675550</td>\n",
       "      <td>8.05</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>69.5500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "      <td>84.154687</td>\n",
       "      <td>26.55</td>\n",
       "      <td>60.2875</td>\n",
       "      <td>512.3292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>13.675550</td>\n",
       "      <td>8.05</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>69.5500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "      <td>84.154687</td>\n",
       "      <td>26.55</td>\n",
       "      <td>60.2875</td>\n",
       "      <td>512.3292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>13.675550</td>\n",
       "      <td>8.05</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>69.5500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Name     Sex   Age  SibSp  \\\n",
       "0                            Braund, Mr. Owen Harris    male  22.0      1   \n",
       "1  Cumings, Mrs. John Bradley (Florence Briggs Th...  female  38.0      1   \n",
       "2                             Heikkinen, Miss. Laina  female  26.0      0   \n",
       "3       Futrelle, Mrs. Jacques Heath (Lily May Peel)  female  35.0      1   \n",
       "4                           Allen, Mr. William Henry    male  35.0      0   \n",
       "\n",
       "   Parch            Ticket     Fare Cabin Embarked  Fare_Mean  Fare_Mode  \\\n",
       "0      0         A/5 21171   7.2500   NaN        S  13.675550       8.05   \n",
       "1      0          PC 17599  71.2833   C85        C  84.154687      26.55   \n",
       "2      0  STON/O2. 3101282   7.9250   NaN        S  13.675550       8.05   \n",
       "3      0            113803  53.1000  C123        S  84.154687      26.55   \n",
       "4      0            373450   8.0500   NaN        S  13.675550       8.05   \n",
       "\n",
       "   Fare_Median  Fare_Max  \n",
       "0       8.0500   69.5500  \n",
       "1      60.2875  512.3292  \n",
       "2       8.0500   69.5500  \n",
       "3      60.2875  512.3292  \n",
       "4       8.0500   69.5500  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 取一個類別型欄位, 與一個數值型欄位, 做群聚編碼\n",
    "\n",
    "# 這邊取艙等對票價做群聚編碼\n",
    "# df['Pclass'] = df['Pclass'].fillna('None')\n",
    "mean_df = df.groupby(['Pclass'])['Fare'].mean().reset_index()\n",
    "mode_df = df.groupby(['Pclass'])['Fare'].apply(lambda x: x.mode()[0]).reset_index()\n",
    "median_df = df.groupby(['Pclass'])['Fare'].median().reset_index()\n",
    "max_df = df.groupby(['Pclass'])['Fare'].max().reset_index()\n",
    "\n",
    "temp = pd.merge(mean_df, mode_df, how='left', on=['Pclass'])\n",
    "temp = pd.merge(temp, median_df, how='left', on=['Pclass'])\n",
    "temp = pd.merge(temp, max_df, how='left', on=['Pclass'])\n",
    "temp.columns = ['Pclass', 'Fare_Mean', 'Fare_Mode', 'Fare_Median', 'Fare_Max']\n",
    "display(temp)\n",
    "\n",
    "df = pd.merge(df, temp, how='left', on=['Pclass'])\n",
    "df = df.drop(['Pclass'] , axis=1)\n",
    "display(df.head())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8 Numeric Features : ['Age', 'SibSp', 'Parch', 'Fare', 'Fare_Mean', 'Fare_Mode', 'Fare_Median', 'Fare_Max']\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Fare_Mean</th>\n",
       "      <th>Fare_Mode</th>\n",
       "      <th>Fare_Median</th>\n",
       "      <th>Fare_Max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>22.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>13.675550</td>\n",
       "      <td>8.05</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>69.5500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>38.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>84.154687</td>\n",
       "      <td>26.55</td>\n",
       "      <td>60.2875</td>\n",
       "      <td>512.3292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>26.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>13.675550</td>\n",
       "      <td>8.05</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>69.5500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>35.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>84.154687</td>\n",
       "      <td>26.55</td>\n",
       "      <td>60.2875</td>\n",
       "      <td>512.3292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>13.675550</td>\n",
       "      <td>8.05</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>69.5500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Age  SibSp  Parch     Fare  Fare_Mean  Fare_Mode  Fare_Median  Fare_Max\n",
       "0  22.0    1.0    0.0   7.2500  13.675550       8.05       8.0500   69.5500\n",
       "1  38.0    1.0    0.0  71.2833  84.154687      26.55      60.2875  512.3292\n",
       "2  26.0    0.0    0.0   7.9250  13.675550       8.05       8.0500   69.5500\n",
       "3  35.0    1.0    0.0  53.1000  84.154687      26.55      60.2875  512.3292\n",
       "4  35.0    0.0    0.0   8.0500  13.675550       8.05       8.0500   69.5500"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#只取 int64, float64 兩種數值型欄位, 存於 num_features 中\n",
    "num_features = []\n",
    "for dtype, feature in zip(df.dtypes, df.columns):\n",
    "    if dtype == 'float64' or dtype == 'int64':\n",
    "        num_features.append(feature)\n",
    "print(f'{len(num_features)} Numeric Features : {num_features}\\n')\n",
    "\n",
    "# 削減文字型欄位, 只剩數值型欄位\n",
    "df = df[num_features]\n",
    "# df = df.fillna(-1)\n",
    "df = df.fillna(-1)\n",
    "\n",
    "# 將所有int換成float\n",
    "for c in df:\n",
    "    if df[c].dtype == \"int64\":\n",
    "        df[c] = df[c].astype(\"float64\")\n",
    "\n",
    "MMEncoder = MinMaxScaler()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業2\n",
    "* 將上述的新特徵，合併原有的欄位做生存率預估，結果是否有改善?\n",
    "> 加上新特徵後 準確率提升了0.04"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6408846483794697"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 原始特徵 + 邏輯斯迴歸\n",
    "\n",
    "# 沒有這四個新特徵的 dataframe 稱為 df_minus\n",
    "df_minus = df.drop(['Fare_Mean', 'Fare_Mode', 'Fare_Median','Fare_Max'] , axis=1)\n",
    "train_X = MMEncoder.fit_transform(df_minus)\n",
    "estimator = LogisticRegression(solver=\"lbfgs\")\n",
    "cross_val_score(estimator, train_X, train_Y, cv=5).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6881330752996592"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 新特徵 + 邏輯斯迴歸\n",
    "\n",
    "train_X = MMEncoder.fit_transform(df)\n",
    "estimator = LogisticRegression(solver=\"lbfgs\")\n",
    "\n",
    "cross_val_score(estimator, train_X, train_Y, cv=5).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}