Day_022_LabelEncoder_and_OneHotEncoder.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 範例 : (Kaggle)房價預測\n",
    "***\n",
    "- 以下用房價預測資料, 觀查標籤編碼與獨編碼熱的影響"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [教學目標]\n",
    "- 以下用房價預測資料, 觀查標籤編碼與獨編碼熱的影響"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [範例重點]\n",
    "- 觀察標籤編碼與獨熱編碼, 在特徵數量 / 線性迴歸分數 / 線性迴歸時間上, 分別有什麼影響 (In[3], Out[3], In[4], Out[4]\n",
    "- 觀察標籤編碼與獨熱編碼, 在特徵數量 / 梯度提升樹分數 / 梯度提升樹時間上, 分別有什麼影響 (In[5], Out[5], In[6], Out[6]) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MSSubClass</th>\n",
       "      <th>MSZoning</th>\n",
       "      <th>LotFrontage</th>\n",
       "      <th>LotArea</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>LotShape</th>\n",
       "      <th>LandContour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>LotConfig</th>\n",
       "      <th>...</th>\n",
       "      <th>ScreenPorch</th>\n",
       "      <th>PoolArea</th>\n",
       "      <th>PoolQC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>MiscFeature</th>\n",
       "      <th>MiscVal</th>\n",
       "      <th>MoSold</th>\n",
       "      <th>YrSold</th>\n",
       "      <th>SaleType</th>\n",
       "      <th>SaleCondition</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>65.0</td>\n",
       "      <td>8450</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>20</td>\n",
       "      <td>RL</td>\n",
       "      <td>80.0</td>\n",
       "      <td>9600</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>2007</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>68.0</td>\n",
       "      <td>11250</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>70</td>\n",
       "      <td>RL</td>\n",
       "      <td>60.0</td>\n",
       "      <td>9550</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>2006</td>\n",
       "      <td>WD</td>\n",
       "      <td>Abnorml</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>60</td>\n",
       "      <td>RL</td>\n",
       "      <td>84.0</td>\n",
       "      <td>14260</td>\n",
       "      <td>Pave</td>\n",
       "      <td>NaN</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>2008</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 79 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   MSSubClass MSZoning  LotFrontage  LotArea Street Alley LotShape  \\\n",
       "0          60       RL         65.0     8450   Pave   NaN      Reg   \n",
       "1          20       RL         80.0     9600   Pave   NaN      Reg   \n",
       "2          60       RL         68.0    11250   Pave   NaN      IR1   \n",
       "3          70       RL         60.0     9550   Pave   NaN      IR1   \n",
       "4          60       RL         84.0    14260   Pave   NaN      IR1   \n",
       "\n",
       "  LandContour Utilities LotConfig      ...       ScreenPorch PoolArea PoolQC  \\\n",
       "0         Lvl    AllPub    Inside      ...                 0        0    NaN   \n",
       "1         Lvl    AllPub       FR2      ...                 0        0    NaN   \n",
       "2         Lvl    AllPub    Inside      ...                 0        0    NaN   \n",
       "3         Lvl    AllPub    Corner      ...                 0        0    NaN   \n",
       "4         Lvl    AllPub       FR2      ...                 0        0    NaN   \n",
       "\n",
       "  Fence MiscFeature MiscVal  MoSold  YrSold  SaleType  SaleCondition  \n",
       "0   NaN         NaN       0       2    2008        WD         Normal  \n",
       "1   NaN         NaN       0       5    2007        WD         Normal  \n",
       "2   NaN         NaN       0       9    2008        WD         Normal  \n",
       "3   NaN         NaN       0       2    2006        WD        Abnorml  \n",
       "4   NaN         NaN       0      12    2008        WD         Normal  \n",
       "\n",
       "[5 rows x 79 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 做完特徵工程前的所有準備\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import copy, time\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.ensemble import GradientBoostingRegressor\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "data_path = 'data/'\n",
    "df_train = pd.read_csv(data_path + 'house_train.csv.gz')\n",
    "df_test = pd.read_csv(data_path + 'house_test.csv.gz')\n",
    "\n",
    "train_Y = np.log1p(df_train['SalePrice'])\n",
    "ids = df_test['Id']\n",
    "df_train = df_train.drop(['Id', 'SalePrice'] , axis=1)\n",
    "df_test = df_test.drop(['Id'] , axis=1)\n",
    "df = pd.concat([df_train,df_test])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "43 Numeric Features : ['MSZoning', 'Street', 'Alley', 'LotShape', 'LandContour', 'Utilities', 'LotConfig', 'LandSlope', 'Neighborhood', 'Condition1', 'Condition2', 'BldgType', 'HouseStyle', 'RoofStyle', 'RoofMatl', 'Exterior1st', 'Exterior2nd', 'MasVnrType', 'ExterQual', 'ExterCond', 'Foundation', 'BsmtQual', 'BsmtCond', 'BsmtExposure', 'BsmtFinType1', 'BsmtFinType2', 'Heating', 'HeatingQC', 'CentralAir', 'Electrical', 'KitchenQual', 'Functional', 'FireplaceQu', 'GarageType', 'GarageFinish', 'GarageQual', 'GarageCond', 'PavedDrive', 'PoolQC', 'Fence', 'MiscFeature', 'SaleType', 'SaleCondition']\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MSZoning</th>\n",
       "      <th>Street</th>\n",
       "      <th>Alley</th>\n",
       "      <th>LotShape</th>\n",
       "      <th>LandContour</th>\n",
       "      <th>Utilities</th>\n",
       "      <th>LotConfig</th>\n",
       "      <th>LandSlope</th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Condition1</th>\n",
       "      <th>...</th>\n",
       "      <th>GarageType</th>\n",
       "      <th>GarageFinish</th>\n",
       "      <th>GarageQual</th>\n",
       "      <th>GarageCond</th>\n",
       "      <th>PavedDrive</th>\n",
       "      <th>PoolQC</th>\n",
       "      <th>Fence</th>\n",
       "      <th>MiscFeature</th>\n",
       "      <th>SaleType</th>\n",
       "      <th>SaleCondition</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>RL</td>\n",
       "      <td>Pave</td>\n",
       "      <td>None</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>CollgCr</td>\n",
       "      <td>Norm</td>\n",
       "      <td>...</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>RFn</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>RL</td>\n",
       "      <td>Pave</td>\n",
       "      <td>None</td>\n",
       "      <td>Reg</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>Veenker</td>\n",
       "      <td>Feedr</td>\n",
       "      <td>...</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>RFn</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>RL</td>\n",
       "      <td>Pave</td>\n",
       "      <td>None</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Inside</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>CollgCr</td>\n",
       "      <td>Norm</td>\n",
       "      <td>...</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>RFn</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>RL</td>\n",
       "      <td>Pave</td>\n",
       "      <td>None</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>Corner</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>Crawfor</td>\n",
       "      <td>Norm</td>\n",
       "      <td>...</td>\n",
       "      <td>Detchd</td>\n",
       "      <td>Unf</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>WD</td>\n",
       "      <td>Abnorml</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>RL</td>\n",
       "      <td>Pave</td>\n",
       "      <td>None</td>\n",
       "      <td>IR1</td>\n",
       "      <td>Lvl</td>\n",
       "      <td>AllPub</td>\n",
       "      <td>FR2</td>\n",
       "      <td>Gtl</td>\n",
       "      <td>NoRidge</td>\n",
       "      <td>Norm</td>\n",
       "      <td>...</td>\n",
       "      <td>Attchd</td>\n",
       "      <td>RFn</td>\n",
       "      <td>TA</td>\n",
       "      <td>TA</td>\n",
       "      <td>Y</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>WD</td>\n",
       "      <td>Normal</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 43 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  MSZoning Street Alley LotShape LandContour Utilities LotConfig LandSlope  \\\n",
       "0       RL   Pave  None      Reg         Lvl    AllPub    Inside       Gtl   \n",
       "1       RL   Pave  None      Reg         Lvl    AllPub       FR2       Gtl   \n",
       "2       RL   Pave  None      IR1         Lvl    AllPub    Inside       Gtl   \n",
       "3       RL   Pave  None      IR1         Lvl    AllPub    Corner       Gtl   \n",
       "4       RL   Pave  None      IR1         Lvl    AllPub       FR2       Gtl   \n",
       "\n",
       "  Neighborhood Condition1      ...      GarageType GarageFinish GarageQual  \\\n",
       "0      CollgCr       Norm      ...          Attchd          RFn         TA   \n",
       "1      Veenker      Feedr      ...          Attchd          RFn         TA   \n",
       "2      CollgCr       Norm      ...          Attchd          RFn         TA   \n",
       "3      Crawfor       Norm      ...          Detchd          Unf         TA   \n",
       "4      NoRidge       Norm      ...          Attchd          RFn         TA   \n",
       "\n",
       "  GarageCond PavedDrive PoolQC Fence MiscFeature SaleType SaleCondition  \n",
       "0         TA          Y   None  None        None       WD        Normal  \n",
       "1         TA          Y   None  None        None       WD        Normal  \n",
       "2         TA          Y   None  None        None       WD        Normal  \n",
       "3         TA          Y   None  None        None       WD       Abnorml  \n",
       "4         TA          Y   None  None        None       WD        Normal  \n",
       "\n",
       "[5 rows x 43 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#只取類別值 (object) 型欄位, 存於 object_features 中\n",
    "object_features = []\n",
    "for dtype, feature in zip(df.dtypes, df.columns):\n",
    "    if dtype == 'object':\n",
    "        object_features.append(feature)\n",
    "print(f'{len(object_features)} Object Features : {object_features}\\n')\n",
    "\n",
    "# 只留類別型欄位\n",
    "df = df[object_features]\n",
    "df = df.fillna('None')\n",
    "train_num = train_Y.shape[0]\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "shape : (1460, 43)\n",
      "score : 0.66156068668513\n",
      "time : 0.016023874282836914 sec\n"
     ]
    }
   ],
   "source": [
    "# 標籤編碼 + 線性迴歸\n",
    "df_temp = pd.DataFrame()\n",
    "for c in df.columns:\n",
    "    df_temp[c] = LabelEncoder().fit_transform(df[c])\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = LinearRegression()\n",
    "start = time.time()\n",
    "print(f'shape : {train_X.shape}')\n",
    "print(f'score : {cross_val_score(estimator, train_X, train_Y, cv=5).mean()}')\n",
    "print(f'time : {time.time() - start} sec')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "shape : (1460, 274)\n",
      "score : -5.719847316893547e+22\n",
      "time : 0.2896397113800049 sec\n"
     ]
    }
   ],
   "source": [
    "# 獨熱編碼 + 線性迴歸\n",
    "df_temp = pd.get_dummies(df)\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = LinearRegression()\n",
    "start = time.time()\n",
    "print(f'shape : {train_X.shape}')\n",
    "print(f'score : {cross_val_score(estimator, train_X, train_Y, cv=5).mean()}')\n",
    "print(f'time : {time.time() - start} sec')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "shape : (1460, 43)\n",
      "score : 0.7773106525946885\n",
      "time : 0.5833423137664795 sec\n"
     ]
    }
   ],
   "source": [
    "# 標籤編碼 + 梯度提升樹\n",
    "df_temp = pd.DataFrame()\n",
    "for c in df.columns:\n",
    "    df_temp[c] = LabelEncoder().fit_transform(df[c])\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = GradientBoostingRegressor()\n",
    "start = time.time()\n",
    "print(f'shape : {train_X.shape}')\n",
    "print(f'score : {cross_val_score(estimator, train_X, train_Y, cv=5).mean()}')\n",
    "print(f'time : {time.time() - start} sec')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "shape : (1460, 274)\n",
      "score : 0.7926834220226007\n",
      "time : 2.1887552738189697 sec\n"
     ]
    }
   ],
   "source": [
    "# 獨熱編碼 + 梯度提升樹\n",
    "df_temp = pd.get_dummies(df)\n",
    "train_X = df_temp[:train_num]\n",
    "estimator = GradientBoostingRegressor()\n",
    "start = time.time()\n",
    "print(f'shape : {train_X.shape}')\n",
    "print(f'score : {cross_val_score(estimator, train_X, train_Y, cv=5).mean()}')\n",
    "print(f'time : {time.time() - start} sec')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 作業1\n",
    "* 觀察範例，在房價預測中調整標籤編碼(Label Encoder) / 獨熱編碼 (One Hot Encoder) 方式，  \n",
    "對於線性迴歸以及梯度提升樹兩種模型，何者影響比較大?\n",
    "\n",
    "# 作業2\n",
    "* 鐵達尼號例題中，標籤編碼 / 獨熱編碼又分別對預測結果有何影響?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}