Final Report.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting House Prices of Washington Using Machine Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hanna Seyoum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem Statement"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "House prices generally tend to go up with time, with some probability of crashing. With price increase, or decrease, many people (either looking to buy or sell) are affected. The goal is to predict current values of the houses so that buyers and sellers can make informed decisions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data contains houses from the state of Washington with 18 attributes such as number of bedrooms and bathrooms, square footage of d/t parts of house, house location, etc. The dataset was acquired from Kaggle."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Methodology"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I have treated this as a supervised learning regression problem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Acquired dataset from Kaggle  \n",
    "* Applied data wrangling & cleaning for feautre engineering and selection, and to handle missing values and outliers \n",
    "* Exploratory Data Analysis and Visualizations to find patterns and insights w.r.t various features in housing data  \n",
    "* Hypothesis testing leveraging inferential statistics  \n",
    "* Predictive modeling for house prices leveraging linear regression, ridge regression, and random forest regression. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "pandas for:  \n",
    "    \n",
    "    data loading, wrangling, cleaning, and manipulation  \n",
    "    feature selection and engineering  \n",
    "    descriptive statistics\n",
    "\n",
    "\n",
    "numpy for:\n",
    "\n",
    "    generate an array of values\n",
    "    array sorting and manipulation\n",
    "    \n",
    "\n",
    "matplotlib and seaborn for:\n",
    "\n",
    "    data visualization\n",
    "    \n",
    "\n",
    "scikit-learn for:\n",
    "\n",
    "    data preprocessing\n",
    "    regression and ensemble models\n",
    "    cross-validation\n",
    "    model selection\n",
    "    model performance / metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Wrangling & Cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data is a CSV file which I uploaded onto pandas as a data frame. A combination of feature engineering and selection was used. There were no NaNs, but instead there were 0s that indicated missing values in some entries."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Cleaning steps**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Used a combination of **feature engineering** & **feature selection**. The more the features the better, to a certain extent (i.e., curse of dimensionality), therefore I kept all features except for `statezip` which I deleted after splitting it into `state` and `zipcode` features. \n",
    "\n",
    "* Created a `month` feature by extracting the months from the `date` column to factor in how price is affected over time.\n",
    "\n",
    "* Created a `total_sqft` feature by summing `sqft_living` and `sqft_lot`.\n",
    "\n",
    "* Changed the data types of `waterfront` & `condition` features from `int` to `category` because they both have values `0` & `1` where 1 means yes & 0 means no.\n",
    "\n",
    "* Split `statezip` feature into two features, `state` & `zipcode` and deleted the `statezip` after split."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Handling missing/zero values**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Found 2 houses with 0 bedrooms & 0 bathrooms that were priced over \\$1 million. It was clear that they were erroneous data. Therefore I replaced the bedroom & bathroom values of both houses with the mean bedroom & bathroom values.\n",
    "\n",
    "* The `yr_renovated` column is a numeric column with years of when the houses were last renovated, & 59.5\\% of the rows have 0s as their value. I was not sure if the 0 meant that a house was never renovated, or if the renovation date was missing. I considered removing the column since 59.5\\% of the rows are zeros, but instead of deleting the column, I added a boolean array column with 1s for all the houses that have a 0, and 0s for all the houses that have a renovation year listed.\n",
    "\n",
    "* 1.1\\% of the `Price` column has houses with 0 values. It is unlikely that the houses were worth \\$0, so I did the same as with the `yr_renovated` column and added a boolean array column. I also created a new dataframe with these houses removed to compare it's models to the ones with \\$0 house price included. There was a slight improvement in the models with the 0s removed. However, I don't think this improvement in model performance is worth removing 1.1\\% of the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Outliers**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Found two 3-bedroom houses with prices over \\$10 million. To check if they were outliers or possibly erroneous, I plotted a linear regression line of the `price` column & `sqft_living` column to check if the prices are high because of their square footage.\n",
    "\n",
    "Result: They seem to be erroneous entries, so I created a new dataframe with both houses removed and compared it's model performance with the original dataframe's model. There was significant improvement in model performance. Therefore I've conducted remaining analysis with the dataframe exlcuding these 2 houses."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exploratory Data Analysis & Statistical Analysis Plots"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Feature Observation**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some assumptions:\n",
    "\n",
    "    Houses with more bedrooms will be worth more\n",
    "    Houses with more bathrooms will be worth more\n",
    "    Houses in bad condition will be worth less\n",
    "    Houses with higher square footage will be worth more\n",
    "    Houses with a waterfront will be worth more"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Plotted a heatmap of all the features and target variable to get an idea of their relationships.\n",
    "* Plotted scatter plots of `price` against # of bedrooms, house size, and house condition. These plots showed a positive linear relationship between `sqft_living` & `price`, a non linear (polynomial) relationship between `bedrooms` & `price`, and the `condition` plot showed that houses in poor condition are worth much less than houses in moderate and good condition. Also, majority of the houses are in moderate condition, and the plot has a slightly parabolic shape.\n",
    "* Plotted a barplot of `waterfront` vs `price` which showed that houses with waterfronts are worth more.\n",
    "* Plotted a line plot of `yr_built` and `price`. This plot shows that house prices are higher with houses that are very old and also new. Old houses could be priced higher due to their historical value. \n",
    "* Plot of mean & median prices per bedrooms. Noticed a slight difference between the two plots, with mean prices being slightly larger than median prices. This could be due to some price outliers.\n",
    "* Plot of mean & median house sizes per number of bedrooms. We see a strong positive linear relationship between `bedrooms` and `price`, and the mean & median plots are very similar.\n",
    "* An ecdf plot of house prices by each month (may, june, & july) shows that house prices remained about the same in each month."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Feature Importance**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The heatmap gives us a visual representation of the features correlation with each other and with the target variable `price`. In addition to the heatmap, I wrote a function that calculates the importance of each feature in regards to its predictability of the target variable, i.e., the relationship of each feature with the target variable. House size (`sqft_living`) holds the most importance in prediction. \n",
    "\n",
    "We can see that `sqft_living` has the highest correlation with `price`, with `sqft_above`, and `bathrooms` following. \n",
    "It's interesting that the `condition` of the house has a small effect on the price. It could be due to people's willingness to buy houses in poorer conditions at lower prices to fix them up themselves."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Fitting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3 algorithms were used: Linear Regression, Ridge Regression & Random Forrest.\n",
    "\n",
    "**Choosing hyperparameters**\n",
    "\n",
    "*Ridge Regression*   \n",
    "The hyperparameter that I tuned for Ridge is alpha. I tried values 0.0001, 0.001, 0.01, 0.1, 1, 10, and found alpha=0.0001 to be the best parameter.\n",
    "\n",
    "*Random Forest*    \n",
    "The hyperparameters that I tuned for Random Forest are n_estimators & max_depth, with best values of None & 200 respectively. \n",
    "\n",
    "With the above hyperparameters, I proceceded to fit several models using the 3 algorithims, using data with outlier prices removed, data with price outliers & zero prices removed, and using data with both included. Also fitted some models with parabolic transformations of the `bedrooms` feature since it had a parabolic shape. Overall, the best model was the Random Forest Regressor that was fitted with data excluding houses with erroneous prices."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our initial assumptions of the features are confirmed: \n",
    "\n",
    "`bedrooms` and `sqft_living` having a linear relationship with `price`. In contrast, `waterfront` has an inversely proportional relationship with `price`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$R^2$ represents the proportion of variance of the target variable `price` that has been explained by the features in the model. The higher the $R^2$ value the better. Random Forest resulted in the highest $R^2$ value of 65\\%, which is better than average, but needs to be improved."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'predicted prices')"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaYAAAEeCAYAAADB6LEbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOydeZgU1dW4X3oAUQREQEB2RE9ENnEDjQoqqJ+yKKAhuIDyi0Yjfu6JJn5Ro3FJMMaoSZQoCpq4sYhJZBGMRMGFzRE5gsgqyCKCMLJNz++PWz309FT1VM90T/fMnPd5eKb7bnWquqhT59xzz61VVFSEYRiGYeQKkWwLYBiGYRjxmGIyDMMwcgpTTIZhGEZOYYrJMAzDyClMMRmGYRg5hSkmwzAMI6cwxWRUGiLSLu7zISLSNJvyxMgxWdqV3cooCxFpKyK1si2HUT5MMRmlEJGjRWSKiHwrIt+JyHwR+Z8KjjkAeCGu6B2gR4UELfuYq0TknBBNyyWLiPxYRL4Rka0iUj91CUuNdz3wmwqOMVJE5lZUljKOsU5E+mTyGAnHO0dEVnmfR4jI9DLaNweWAnnlONZcERlZDjHjx2grIjtFJOXjG47a2RbAyC1EJAL8C/gzcAmwHxgEvCIivVV1STmHbkLJF6FmFRI0vZRXllHAI6r62zTKYW/5SVDVicDEMpodDFT4RaG8qOoa4NBsHb86YIrJSKQZcBTwoqru8cpeF5Ef4JQLItIEeAI4H9gJPKyqj3lWwx+Bs4DYW+sob4w/A3VE5FNgCdAWeENERqvqRBG5CbgB9x/6DeBGVd0pIr8GTgA6A7uBrqoajQkrInOA94DhQGNgAvC/qro//qRE5FRgrDfOSuAOVX1LRF5KlCVkv0neeZ4uIqKqIxP6nQw84vWrC0wCrlbVQhFp712PU4EtwC+AXcCdQC3v+j4ETFDV1t54nYDlqlrL+347MNKTfYsn1z9Igmf1/hroBESBcap6h1dXBNwI3A4cBLyiqtd5db2Av+Dui/EEWCLeb9XRG78b8F/gSlXd6Pc7Amd716g98BFwjap+4Y11F3CT1/bluGOMBEar6g89i+TXwP8D6gCTgZ/i7geAb0WkM/AN7jccCOwB/qSqj3jjHQM858n7L+CQgHMbCQzFvVydibuHR6nqMq9uJHAYcATu/8UioI6q7heRHwH3AEcCHwBXqepqEekIPAWcDKwDblXVt7zj3eKd/0HAQuBaVV3pJ1t1xFx5RglU9Wvcf+w5IvJLEfmhiBykqg+o6myv2V9x904r4AzgTu8BfhvQAuiCUxIK3KWqnwDXAu+r6nGqOhxYAwzwlNKPcQ+UfrgHWwPcAytGH+A8oHe8Uorjx17fH+AeGmPiKz3Xzlu4h+vhwM+BV0XkmERZUuh3EfAu8LNEpeTxD+BZVW2GcxNe4MkI8BrwGe4l4FLgGdyD7gHgH6p6vs948XL1wSnxczlwrR4ro8+hwN+Bn6vq4UB/4CbvhSPGDwEBzgGuEJEzRORg3AP/z7jf9CvcbxzEcOBeoCmwGXevxOiD9zsC7YBXgVtw1+GfwBQRiYjIRcD1Xrsenlx+XAcMAU7BKbejvPFO9eoP86yXsd4xjsHdH1eJyKVem1dwrtzDganA8UnO7QLcb9cYmIP7jWOc7sn8A2B7rFBEeuB+35/gFNcC4K8iUhv3AjYHp8zGAC95bsBOwC+982oBfIl7eakxmMVk+NEf9+C7GPdGultExgM341xNA4AuqroT2CkifYGNOEVUBOwFOgDf4t6My2Ik8PuEt+XFInKdV/+hqi5P0v8xVV3h9X0UGI17GMUYAHymqs963/8tIv/CuSqTzemUtx+4h/tKEWmEsx6/AY4UkaOA7sAPPYv0QxE53asPywdAL5ySaAV8DzT33LBBfA/0UNWVnkXWAGftHgks89o8qqq7cNd+Mc7yqQPsV9WnAETkQdzDP4h/quq/vbZ3AyoiMSuk+HcUkZ95bWd5dWNF5GbgJNx992xc2wcp+XvGuAT4g6qu9tpdRsIzzbsml3vnvgPYISJjgZEi8hFwHNBLVfcCL4jIbUnObYmqjvPGvQe4xbO4AFar6n+9usPj+gwBpqrqO3H9OuGspMZxbuDZ3r01HHgeZ7n9P5y1+NOAF7JqiykmoxTew+lB4EERaYB70/8D8B3uzbwOsD6u/VIodjf9Fffg/RTnhgljlbcBfi8iD8WVRXBvkgAbyugf7+JYj1ME8TQDVieUrQFalzFuefsBnAbMwinyj3Aumdg5bVPV72MNVXURgIiEGBZwyv9+nGtqNc5lCkmutedCHOI9/HfilFuthD6b4z7v8+qa4xRg/DjrCSbxt6iN5wKm5O/YBrhIRL6NK6uLc002B96PK0/8DWIcQcn7cB2A5yqN0QyoB8yPu74RnBXSHPgm/rdIciyIOzdV3S0iWzlwrwXdo4ky7gQWeRbbEQnnX9uTZ4OIXADcgbPS14nIDar6rySyVSvMlWeUQER+JCKxN2hU9TtVfR3nyukKbMI9tI6M63OZ5156EpgPNFPVM3A++zBsAH6iqoepasxP3807FrgHcTJaxn1uS9yDwGMdznUUT/u48YMoVz8RaY1T0INUtZ2qDuGAe2c90NhzkcXa3yAi3ROGiVLyxbFx3OebPLnaqOrxwO/KOI/YXNltOOvgaOCysvp4bARaixd67f1NVPzxJP4Wezmg8OJ/xw3A87Hf3Pvde+DcWxu9vn5jxrOekvdhLxEZndBmC+5+7Rx3nA7A/3jHOVxKRlQGHatEnff7NeXAvRZ0jybKeLj3ArYRWJlw/scCvxK3dGGHqvbDKfW/4NywNQZTTEYiM4FmInK/iLTyfP4/AIYBb6hqIW5u4B5x63864eY49uJ86AWqGvUmna/HWVfgrKf4SKX47xOAWz3/eh3cXMvrKcg8RkSOFJFWuIf2hIT6fwKdRGSUiNQWkfNwD6bXAmQL2y+IBt7fAhHJE5H/h1O0dbw5jw9w16+OFyRxL05xxcuxEmgqImeKSD1cUEKMw7y2+72H2L1eeR2COQwoxLllDwL+zytL1gdcAMN+nNuqDu76JotiHCQip3oP7nuB11R1t0+7fwBDROQ0EaklIhcDi3EP+xdx80BdROQwnOXgx9+BG737tCHuvmmGuzYAh3r369+B34rIoSLSGPf73ekFE3wEPCAidUVkKC5AI4hTROQiEakL3IdzTZYVkPAqMMC7JnneufTAWYQREfmpd28dC3yI8060A94Ska6e9+JbUnP1VnlMMRklUNUtuMnmY3EPil3AFJzP/89es595f9cAbwP/p6rv4eYeRojId7hJ5eeBo7yJ3v8A9ePcQC8AE715pGdxvvR3cG+4xwMXq2rYzcIWeH0X4x4ETyec01bcxPU1wDbg98BwVV3sI0sq/XxR1c+8tvOBr3Eut5dxgQUAP8LNbWzCKdGRqroKeBP38HvPc0vdDryEU1Jvxx3iUdwcxBbv3D/w5EvmC/w3zoL9HPe7HeWNmdR/6M2DDfBk/hY3B7Q0SZd5wMPeeefhAhT8xlXgCpyVvR0XtTZEVdd5c1T341yh6o3pxzickpmHu0af4V6SvvbOdZ1nid6AU8orvH+rcNYjuHmqbrjrdw0wm2A+Aa7CWYDdcYErSfHc3KM8Wb/BeR2u8ua0LsTNp20CZuDm+F5R1Y+Bu4E3RWSXJ1eZx6pO1LKNAo2qjLhw8Qmq+ky2ZanpiAsJ76SqYd2EVQaJC1PPtiw1AbOYDMMwjJzCFJNhGIaRU5grzzAMw8gpbB1TBfn444/34yzPHdmWxTAMowrREIiecMIJpfSQKaaKEwFq5eXlNfKrLCwsBCAvL3cTDZuM6cFkTA8mY3rIdRk9+Xynk0wxVZwdeXl5jXr08N81wUXFprSqv9IxGdODyZgeTMb0kOsyLlq0iMLCQl9PkwU/GIZhGDmFKSbDMAwjpzDFZBiGYeQUppgMwzCMnMIUk2EYhpFTWFSeYRgVJhqNMiV/Nl9sWcNRTdsyqEtfIhF77zXKhykmwzAqRDQaZcQLd/Da4hnFZUO792PC5Q+ZcjLKhd01hmFUiCn5s0soJYBXF89gav6c7AhkVHlMMRmGUSG+2LLGt3zFlmS7lBtGMKaYDMOoEEc1betb3qlp4q70hhEOU0yGYVSIQV36MrR7vxJlw3r0Z2CXPtkRyKjyWPCDYRgVIhKJMOHyh7jk+PNZsWU1nZq2Y2CXPhb4YJQbU0yGYVSYSCTC4G5nZVsMo5pgrzSGYRhGTmGKyTAMw8gpTDEZhmEYOYUpJsMwDCOnMMVkGIZh5BSmmAzDMIycolLCxUXkRuBaIAp85H1uB3wIfBHXtLeqfi8iNwDXA/uAl1X1Pm+cLsA44BBgHXClqm4SkTrAU8BpwB7gl6o6zeszDLgHKATmADep6n4RaQVMAJoB24BRqroic1fBMAzDCEPGLSYR6QbcCJyiqscB9YCfAr2AF1S1R9y/70WkBzAGOAU4CThPRGLLyicA96pqV+BfwKNe+fVAQ6AzMAh4SkQai0gL4I9AP6A70AIY5fV5AnhVVbsAjwDjM3cVDMMwjLBkXDGp6hJAVHWHiBwKHAFsxSmmLiKyUET+KyKne10GAJNVdbuq7gYmAsNFpA3QSlXf9NqNA4Z41tIAYLyqFqnqauB9r6wfMFdV16tqFHjWG6sOcC6eMlLVqUAnEfFP+mUYhmFUGpXiylPVfSJyGfA4zgU3FWfBTASeBk4EpnnWVStgWVz3r4DWXvn6uDF3ichunCuuRF1cn2hAeRNgj6rujKvb4NX5p0pOQmFhIarqW1dQUBCTN9VhKw2TMT2YjOnBZEwPuS5jYWFhYF2lpSRS1QkiMhF4GHhWVQfHVX8gIu8DZ+GsuKK4ulq4+aHE8mR1sfK8coxlGIZhZJGMKyYR6Qi0UNX3VLVIRMbjrKM7gcdV9bs4WfYBa4Ej44ZoibN6SpSLSH2gDvBNXN3iuD4f4pRNd5+xNgH1RKS+qu5KqEuZvLw8RMS3Lva2ElSfC5iM6cFkTA8mY3rIdRkXLVoUaDVVRrj4EcCLItLY+z4CFx13DvATABHpDpwAzASmARd7wQv1vPbTVHUtsFFEBnrjXA1MV9V9Xp+rRCTizROd7o01AzhDRNqJSAQX+DBNVfcD070x8Mb8SlXXZfJCGIZhGGWTcYtJVeeJyMPAXBEpBJYAPwMaA8+IyCicC224qm4DtonIWGAuUBd4RVUne8MNB54WkQdxVs8Ir/wJQIB8nJU0RlU3AojIdTjFdRDwrtcW4DrgbyJyDbALuCxT18AwDMMIT2UFPzwJPJlQ/C1wdkD7p3DrkhLLl+LWKiWW78WtjfIbaxIwyad8PS4yzzAMw8ghLPODYRiGkVOYYjIMwzByClNMhmEYRk5hiskwDMPIKUwxGYZhGDmFKSbDMAwjpzDFZBiGYeQUppgMwzCMnMIUk2EYhpFTmGIyDMMwcgpTTIZhGEZOYYrJMAzDyClMMRmGYRg5hSkmwzAMI6cwxWQYhmHkFKaYDMMwjJzCFJNhGIaRU5hiMgzDMHKKStlaXURuxG19HgU+8j7XBl4AjgYKgOtV9QOv/Q3A9cA+4GVVvc8r7wKMAw4B1gFXquomEamD24r9NGAP8EtVneb1GQbcAxQCc4CbVHW/iLQCJgDNgG3AKFVdkeFLYRiGYZRBxi0mEekG3AicoqrHAfWAnwL3Acu9smuAl0Sktoj0AMYApwAnAeeJSD9vuAnAvaraFfgX8KhXfj3QEOgMDAKeEpHGItIC+CPQD+gOtABGeX2eAF5V1S7AI8D4TF0DwzAMIzwZV0yqugQQVd0hIocCRwBbgQHAc16bBV7ZqV75ZFXdrqq7gYnAcBFpA7RS1Te9occBQzxraQAwXlWLVHU18L5X1g+Yq6rrVTUKPOuNVQc4F08ZqepUoJOItM309TAMwzCSUymuPFXdJyKXAY/jXHBTgb8A6+OafQW0BloBywLKi9ur6i4R2Y1zxZWoi+sTDShvAuxR1Z1xdRu8ujWpnl9hYSGq6ltXUFAQkzfVYSsNkzE9mIzpwWRMD7kuY2FhYWBdpQU/qOoE4HDg3zjLJQIUxTWphZsHCltenj5hxjIMwzCySMYtJhHpCLRQ1fdUtUhExgPTgLXAkcB2r2lLnHUTKydZuYjUB+oA38TVLY7r8yFO2XT3GWsTUE9E6qvqroS6lMnLy0NEfOtibytB9bmAyZgeTMb0YDKmh1yXcdGiRYFWU2VYTEcAL4pIY+/7CFx03DRgNICI9MQplg+98ou94IV6XvtpqroW2CgiA71xrgamq+o+r89VIhLx5olOB2YCM4AzRKSdiERwgQ/TVHU/MN0bA2/Mr1R1XSYvhGEYhlE2GbeYVHWeiDwMzBWRQmAJ8DOcK+1pEfkU50Iboap7gIUiMhaYC9QFXlHVyd5ww70+D+KsnhFe+ROAAPk4K2mMqm4EEJHrcIrrIOBdry3AdcDfROQaYBdwWaaugWEYhhGeygp+eBJ40qfqkoD2T+HWJSWWL8WtVUos34tbG+U31iRgkk/5elxknmEYhpFDWOYHwzAMI6cwxWQYhmHkFKaYDMMwjJzCFJNhGIaRU5hiMgzDMHIKU0yGYRhGTmGKyTAMw8gpTDEZhmEYOYUpJsMwDCOnCK2YvLx1ePnoBonI6ZkTyzAMw6iphFJM3l5KG7yvDwF/Bv4hIrdlSjDDMAyjZhLWYrodGCgidXHboA/AbX1+Y6YEMwzDMGomYRVTa1V9FzgT2KmqH+F2om2QMckMwzAqmWg0yqQls/jd288yacksotFotkWqkYTNLv6FiPwUGAS84VlOt+C2mTAMw6jyRKNRRrxwB68tnlFcNrR7PyZc/hCRiMWJVSZhr/ZPcHsh7QfuAk71vvtuNWEYhlHVmJI/u4RSAnh18Qym5s/JjkA1mFAWk6ouBM6IK5oDdMuEQIZhGNngiy1rfMtXbFldyZIYoRSTiOQBN+O2Jm8O9AT+BIxS1S2ZE88wDKNyOKppW9/yTk3bVbIkRlhX3gPABcAYr88WYCfwlwzJZRiGUakM6tKXod37lSgb1qM/A7v0yY5ANZiwwQ8jgG6q+o2IFKnqLhG5GlgbprOI3IqztqLASmA0cAIwMW6Mbara12t/PzAU2A08rqrPeOV9gMdwyjEfGO3J0gB4ATgaKACuV9UPvD43ANcD+4CXVfU+r7wLMA44BBdheKWqbgp5PQzDqGZEIhEmXP4Qlxx/Piu2rKZT03YM7NLHAh+yQFjFFKW0dXUwTgkkRUTOwCmlU1R1p4j8FvgtThk8pKoPJ7QfDPQBugL1gfki8i6wCpgAXKCqi0VkLHAnLhjjPmC5qg4WkZ7AKyIiQBeclXcisAeYJSLzVHWGN9ZdqvqmiIwBHsUpYMMwaiiRSITB3c7Kthg1nrCvAhOAySLSD4iISC+ctfNSiL6bgetUdaf3fQHQDugF9BeRxSIyw7NgwC3efVFV96rqNmAycClwMrBBVRd77Z7BRQbG+jwHoKoLgK24yMEBwGRV3a6quz2Zh4tIG6CVqr7p9R8HDBGROiGvh2EYhpEhwlpMd+OyP/wJqItTVBOB+8vqqKqfAZ8BiEhD4FfAU8DpONfaZBEZBEwTkWOBVsAbcUN8BXT2ytcnlLf2PgfVtQKWBZQXt/fcgbuBZl6blCgsLERVfesKCgpix0h12ErDZEwPJmN6MBnTQ67LWFhYGFgXymJS1f3A80BPVT0EuBgYp6p7wwohIkcCbwPvqepTqvpjVZ3sjT8F2A4c78lUFNe1FlCYpJxy9EksTxzPMAzDyBJhw8UvAZ7GWTlLcGua7hGRK1V1Woj+3YBpwNOqep+I1Af+F3hAVWMKojYuQGEtcGRc95Y46yaonLi67SH7lCj35KkDfFPWufiRl5eHm9IqTextJag+FzAZ04PJmB5MxvSQ6zIuWrQo0GpKJVz8bFVdAqCqfwLOB35XVkcRaQHMBH4ei4hT1V3AVTjLCxE5H+ciXIhTYJeJSD0RaQxcBLwJzAdae8EN4CL7Ykpxmvcdr/5I4EOv/GIRaext2zECmKaqa4GNIjLQ6381MF1V94W8HoZhGEaGCDvH1AxYlFC2ADgiRN/bgUOB20Xkdq9MgWHAkyJyL25N1BDPZThJRI73xq8NPOZlnkBELgWeEZGDgc+By73x7gaeFpFPce64Eaq6B1joRe/NxSm+V2LuQ1zgxNMi8iCwCYvIMwzDyAnCKqb3gAdF5E5V3SsiBwH3eOVJUdWbcVkj/OgV0OdunLJJLJ+LyzqRWL4duCRgrKdwwRaJ5UuB0wIFNwzDMLJCWMV0LfA6sFNEtgGNgQ84EK5tGIZhGGkhbBLX1cAJItIR575b783TGIZhGEZaSaqYRORyVX1BRK5KqOoci/RQ1b9lSjjDMAyj5lGWxTQcl4Pu8oD6IsAUk2EYhpE2kiomVf0f7+PfgNdUtczceIZhGIZREcKuY/oDEDrLg2EYhmGUl7BReS8Dj4rIP4ANxKXzUdWVmRDMMIzcIhqNMiV/Nl9sWcNRTdsyqEtf2xLCyAhhFdM13t/rE8qLgLz0iWMYRi4SjUYZ8cIdvLZ4RnHZ0O79mHD5Q6acjLQTNlzc7jzDqMFMyZ9dQikBvLp4Bpccf77tX2SknbAWE16aoCFAc1wi1Jc0V/OpG4aRVr7Yssa3fMWW1ZUsiVETCGUJicj/A/4DtMFt/NcJt7PsxRmUzTCMHOGopm19yzs1bVfJkhg1gVQ2CjxbVT+IFYjIqbg9ml7PhGCGYeQOg7r0ZWj3frwa584b1qM/A7v0yZ5QRrUlrGI6GMhPKFsA1E+vOIZh5CKRSIQJlz/EJcefz4otq+nUtB0Du/SxwAcjI4RVTA8Br4jIL4AvcRvu/dor6xhrZKHjhlF9iUQiFuhgVAqpKCZwmwMm8jPvr4WOG4ZhGBXGwsUNwzBSIBqNMnPlfNbs2EivPSfYQuMMEDpc3DAMo6ZTaqHx/BdtoXEGsCtpGIYRkqCFxlPz52RHoGpKpVhMInIrMAqIAiuB0UBdYALQDNgGjFLVFV77+4GhwG7gcVV9xivvAzyGU6j5wGhV3SUiDXDbcxwNFADXx0LbReQGXCqlfcDLqnqfV94FGAccAqwDrlTVTZm9EoZhVGVsoXHlkHGLSUTOwCmlU1S1K7AU+C3wBPCqqnYBHgHGe+0HA32Art7f28VxEE6RXeGNswG40zvMfcByVT0Ol9fvJRGpLSI9gDHAKcBJwHki0s/rMwG41xvrX8CjmbsKhmFUB2yhsSMajTJpySx+9/azTFoyi2g0mtbxy9rBdjZxmcT9UNWy4kc3A9ep6k7v+wLgOqAXcJk3xlQReVpE2gIDgBdVdS+wV0QmA5cCs4ENqrrYG+cZYBpwl9dnoDfWAhHZCpwKnAlMVtXt3vlMBIaLyDKglaq+6Y01DnhYREaq6r4yzscwjBqKLTSunIS+ZbnyJnh/TwL6AX8CVuPWMf0MmF7WAVT1M+AzABFpCPwKeAk4Pk5ZgbOAWgOtgDfiyr8COnvl6xPKW3ufg+paAcsCyovbe+7A3Ti34ldlnVMihYWFBKUNLCgoiB0j1WErDZMxPZiM6SHXZfzVKVdx0mHC2u++plPTtvRtfyLLly8P3T9aFOXtLz9kzY6NtG3YgrM6nESkVvqdV5m6jjNXzvedZ/th826c3eHk0OMUFhYG1pW1g+04ABG5A+irqsUOVs+S+S9wYxghRORIYCrwHs5td3tCk1pAIc69WJRCOeXok1ieOJ5hGIYvkVoR+rQ9AYBDDjkkpb7Roii3zXyM6SvnFZf179iLR865MSPKKROs2bHRt3z19g1pO0bY4IdmwPcJZbWABmE6i0g3nNvtaVW9T0RqA/VEpL6q7vKatcRZMWuBI+O6l1VOXN328owlIvWBOsA3Yc4nkby8PETEty72thJUnwuYjOnBZEwP1VnGSUtmlVBKANNXzmP03kvSnlUjU9ex154TYP6Lpcp7H3tiSsdatGhRoNUUVkU/D8wUkatF5DwR+QkwA/hzWR1FpAUwE/h5LCJOVffj3IBXe20GAl+p6jqcArtMROqJSGPgIuBNYD7QWkR6ekOP9tri/R3tjdUTp3Q+9MovFpHGIlIPGAFMU9W1wEbvuHhyTLf5JcMwMkl1iOqLzbPFk+55trAW08246LYrcfsxbQT+APw1RN/bgUNx0XUx953iAiD+JiLXALs4EAgxydv7aYEn32OquhBARC4FnhGRg4HPgcu98e4GnhaRT3HuuBGqugdYKCJjgbm48PRXVHWy12e41+dBYBNOaRmGYWSM6hDVVxkJfcOmJCoEHhWRCUA7YCFQR1WTRux5fW/GKTY/zg3oczdO2SSWzwV6+pRvBy4JGOsp4Cmf8qXAaYGCG4ZhpJnqEtWX6YS+oRSTiDTDBSz0A/bglMO7InKBqn6UMekMwzCqEbZ9SDjCuvKewoV8D8HNBX0uIg8AjwO9MyWcYRhGdcO2DymbsIqpD3Ckqu4VkZj77nHgnoxIZRiGUQ6i0ShT8mfzxZY1HNW0rWX+rqKEVUybgR8AS+LKfoALgjAMw8g6lZGRwKgcwv5a9wEzvOSqdUXk57gQ7t9mTDLDKCeZzuNl5CaW+bv6EDYq70URWYULF/8PLov3Vao6O3OiGUbq2FtzzaU6rBEyHGGj8n6uqg/i0gnFl9+vqndlRDLDKAdBb82XHH++TThXc6rDGiHDEaiYRKQlcL739W4R+RqXhijGobhErqaYjJzB3pprLtVljZCR3GLaBJyHy5NXF7gioX43cEOG5DKMcmFvzTUXWyNUfQhUTF62h0sARORBVf15rE5EGqjqd5Ugn2GkhL0112xsjVD1IGy4+Isi8hlu+/EPcK69C4CLNFc3TTFqJGW9Nds6F8PIfcIqpr/iNgn80Pt+O7AKeBo4I/1iGUb5CXprtog9w6gahP3f2Bl4Mpa01fv7JNA1U4IZRrqxdS6GUTUIq5i+wO2LFM8AYGV6xTGMzGERe4ZRNQjryrsVmCQiN+N2gG2Fs6IGZUoww6gIfnNJFrFnGFWDsJkfZonIMcD/AEfgdob9l6puyaRwhlEeguaSnh/xW4vYMw8uHbAAACAASURBVIwqQFLFJCLtVXWViHT0iubEVTcUkYaqau48I634WTupkCz7g61zSR8W4WhkirIspiVAQ2AFUETJzA94ZXkZkMuooQRZO7865SoitcI99JLNJdk6l/RgEY5GJkmqmFS1ofe3wneaiNQF/gn8QVWnich5wERgrddkm6r29dreDwzFZZd4XFWf8cr7AI/hgjbygdGquktEGgAv4JLLFgDXe+utEJEbgOuBfcDLqnqfV94FGAccAqzDrdHaVNHzNMpPNBrlzjcf87V2fti8G2d3ODnUOFVtLikajTLpk1m84UUHDuzSh8Fdz87pB7zlJDQySVmuvDLXKKnqf8pqIyI9gL/gwsv/4BX3Ah5S1YcT2g7GbUzYFagPzBeRd3HrpiYAF6jqYhEZC9yJy9V3H7BcVQeLSE/gFRERoAswBjgRtyX8LBGZp6ozvLHuUtU3RWQM8CgwoqxzMTKD3xt4PKu3bwg9Vi5kfwjr5opGo/z4+dt5fcnM4rIXP36TId36MfGK3LU+klml5uIzKkpZrryJ3t8I0AL4FheV1xxoCiwl3Fqma4G7gV/ElfUCaovICFxevptUNR8Xhv6iqu4F9orIZOBSYDawQVUXe/2fwQVh3OX1GQigqgtEZCtwKnAmMFlVtwOIyERguIgsA1qp6pveWOOAh0VkpKruC3E+JSgsLCQoAUZBQQGeXKkOWylEi6L8S+ey7rtNHLWyDWd1OCm0yyydzFw5P1ApAbSodzgFBQWhr+OvTrmKHzbvxurtG2jXqCV925/I8uXL0yWuL7Hf+rNln3HbzMeYvnJecV3/jr145JwbS13bmSvnl1BKMV5bMoPT3upKv46nZETGit6PB+/x9+DX25PHwCevC3XumZYxk5iMFaewsDCwLumdoqptVLUNMAn4FdBMVbupanPgNkruaJtsnGtV9a2E4m+AP6lqd1xWiWkicjAuFH19XLuvgNZJyilHnxLlqroL5zZsFuZ8qgvRoii3zXyMn7/zBH9a8Ao3zRjLbTMfI1pU+RvrrdkRvBnyuR17c0ab41MaL1IrwtkdTuaqHoM4u8PJlaps3/7ywxIPZoDpK+cxe9VHpdomO++nPn41K79FGM7qcBL9O/YqUXZux94UFRWFPnfDCCLsOqYrgMaqGv+/5I/AtvIeWFV/HPd5iojcCxyPU5ZFcU1rAYVJyilHn8TyxPFSIi8vD+c5LE3sbSWoPptMWjLL9yEyeu8llT5P0GvPCTD/xVLlA7v05e9XPMIXX3wBlLyOueYyiv3W3x/kfxsV1N1f6j4IOm+A5d+sQfduYHC3s9J2rum8H6ce8yRT8+eUiHAcO2e8b1u/c68MGTOFyVhxFi1aFGg1hVVMK4GrcTnzYowBlpVHIBGpD/wv8EAszZEnyz5cMMSRcc1b4qyboHLi6raH7FOi3JOnDs6KqzHkUiYEv3khgKn5s7li4i9KReVVRlRYeZVBKsEXg7r0ZUi3c3jNx50HB+ZsshEBV9b5+0U4VrXAEyM3CXtXXwvcIyJrRGSeiKzDKaarynNQz3V2FXAxgIicj9vzaSFu3ugyEaknIo1xqZDeBOYDrb3gBoDRXlu8v6O9sXrilM6HXvnFItJYROrhghumqepaYKOIDPT6Xw1ML8/8UlUmlx4isazgt/QdWaru1cUzSrmCMp33LqYMLn3uFu6c9hiXPncLl71wB9Fo2a61mJKNJyj4IhKJMPGKh7m5z5W+Y3Vq2i4rOf7Ke/6pnLthBBE288M8EemACyhoDmwE/usFKJSXYcCTngtvJzBEVffjUh8dDyzw5HtMVRcCiMilwDPeXNTnwOXeWHcDT4vIpzh33AhV3QMs9KL35uIU3yuqOtnrM9zr8yAu+KLGReTlQvRaPJFIhCb1G/nWJUblZdraq0g4dKob1kUiER648EbWbPvK97cIco9l0rKtyPkP6dGfg2rXpQh3jw3qepZF5RkpEdaVBy76bijOGrkauEFExsa54spEVfvEfV6Ai8zza3c3Ttkkls8FevqUb8fb1NCn7ingKZ/ypcBpIUWvlsQeoLHotd7Hnpj1TAhBVly7Ri1DtUuXtVdRxZfqQt54ZbZ88yq+KdhB40MaMiV/Nh2btPbtk0nLtjzn7+dy3Lt/L4O62romIzVCKSYRuQx4CHgWZ1nkAaNw1tPtGZPOyDiRSKR44WouTJIGWXF9258Yql26rL1suDkjkQgDu/Qp9XAf0u0chnTrx2tLKs+yLc/5B1lZ7Q5vRZP6jXIiQMWoGoS1mH4FnKeqn4jI9aq6SUTOBT7CFJORRoLcYIlrkJJZGMkWs/rl4JuSP5vlm1fzTcF2mhzSiE7N2jGg85lpV3xhgin8Hu6vLZnJ36/8HZf2rLwcf+VR/EFW1u9nP1f82dIWGWEIq5gOBz7zPsdcd5tT6G9UUbIRkh3WDRZkYfg9/PzcTEO6nQPgGxEXy0YeryAv7HxGua9F2Mi6oIf7yq1rufWsUaGOlQ5SnSeDYCsrHktbZIQhrGKZDYwVkTviyv4PeCf9IhnZJF4RdWjSmlcXvlXiwZ2uN950Kbywk/RBlkgQiWNUNGQ7rJy5FimZigIJCvlPxDZmNMoirGK6HpdbbgcQEZGdODfejzIlmFH5lJWvDg7MGdx/wZiUrIV4JTSg85lcPvEXaVmXE3aSPqhdMuLHCKtYokVRJi2ZVUrhhpUz1yIlUyHRytq6a3sJN14MW9NklEVYxXQccCEuP14bXM66tcm7GFUNv4evH7+f/Ryrv1kfSpH4Kbte7bszb9XiEu3K6+IJa2GEcTMlGyOMYomleIrPphFTuEHH79ikTYnvqbrQci37RbyVFY1GWf3N+iqpZI3sElYxvQa0VNUNQPg0z0ZOE3uozfvsY9o2bEFB3f2h+4ZRJEHbWCQqpRjlcfGEtTD82g3t3o9lm74kf8OKUuN2bXl0iTHCKEC/HHmx6xSU4eHROc8z8Lg+1K594L9iWBdaru+JVJ55KsOA8IppJjBGRP6OU0zFa5cS8ucZVQS/h1rvdt1SGiPVNS1lEdbFk2glJAYpJD78Yu2Pb30s7Q5vReNDGnJMs/YM7NKHSZ/MYvj420od467+1xQr1qUbVnBsi6O4uOs5vP7JAcWSqACDtuaIbVA47PhzSymm+auXcPaTVzP7Z88mfWD7WUZVYU8k25jRKA9hFVMfXKaGhzmglGphO9hWWfweau+vXuLrZgsi1TUt8fRu3533444T1sWTqpUQ1P7WviOJRCJc1PVsX4vrwmPP4Oj7L2D99q8B+Pey/9Kq0RG8dMXDfPnNes8FV8TYOeOL581mr/bPoL1113ai0Sgrt67zrX9/1WKm5s8JfIAHncPxrY/1bW/BBUZVJ6xiCrd1qFFlCJozufC4M6kTqc27Kz9O2r93u25c2Dl4H8lkwQbDevRn/I8fYNrS/6Ts4knVSiirfZC76c43HytWSjHWb9/Ex2uX8psLxvjOmy3+2n+/p9ic3JAe/QPPK9kGe8kWrvphwQVGVSdsrrzVItICFwDRHJeh+w1V3ZpJ4YzMETRnckyz9nRq2qZMxfT+6iVcMfEXgZZK0Pi39h3Jb7yIvvK4eFZs9rcGgqyEMEELfrIs9Zl3ApftPEpR6HmzGK8unsGwHucGWqQdm7QJtASDzqHRwQ2qbASfYSQj1Cykl4V7BS5p6nG4TN7LReT0DMpmZJBkWaAHdz2bId36BfQ8QLIM10Hj/yYuzDwadaHVv3v7WSYtmVVm5upoUTTweEFWQllBC4ky7N+/n0lLZrFr727ffp9vXs3Y2f5JVcvii61refu6cfRu371E+bAe/cFH2cWub9A5/HPpf3h+xG95eeRYHrjwRl4eOZYXLnvQgguMKk9YV97vgR+pamybCUTkIuAJILUZcyMniHdhvf/ZR7Rr1JJr+48ofqhNvOIhLs0/n8mfzOLFj98MHCfIUikrIqs8EWXTv5jP/NWlN03u1b57oJXgF43Xu313lm9exWuLZ5RaQNywXn127N4VeL7J6N78GBZ//Xlgfaem7ahduzazf/Zs6A32VmxZzc19rqR3u268n3Du81YtZtrS/1Q4uCDXQs4NI6xiOgKYnlD2BvB0esUxKpOYC+vYg1oVf0+sG9ilD3v37w1czZ9sPiM2RuzBFwsUSBZRFrR4N1oUZez8Cb7H6Xh4q6RbSsQU5OebVzEt/x3eX7W4ROBFPOVVSsN69OfOk67kndUL2FlnH9M+faeEyy7exZbKBnufbljBlPzZXHDcmaUUE1Q80CHXQ85TwRRs9SGsYnoS+JOI3Kaq271N9+4FxotILVyEnoWOV3GC/mNPuPwhhvY4l99M/wv5Gw5M8IeZzwgKS/+f4870bR+0ePftLz9kw84tvn1q1Ur+8IkpgklLZjFvdbiIwzDc2nckh9dvVCLR7NkdTkZEuLXvyFJWUbKH5IDOZ9KqUfNSARcTP36TiR+/GRjKX9FAh6oQch6G6qRgjfCKaSQu6OFqEfkWaMiBMPH/xULHqzzRouT/sS/ufg6Du56V0sMWgsPStxbsCOwTCxSgVq1iJRm0RggIPdlfnrREQfRu36PEfFk8iQo+zHV6Y+k7pZRSPH6h/OkIdMj0houVRXVRsIYjrGLy3dDPqJokPjh/UPdI3v7ywzL/Y5cVSedncQU9+D7fvIq2jVuyZpu/wrl1yu9Z++2Buu7Nj/Ft17t991Ib0QVZfuVJS+THqe27M/O6Z0rMl8UyaLRu0Jxfv/900sS3qVyneC487kxu7nNlWrMo5FLS2IpQXRSs4QgdLl7RA4lIXeCfwB9UdZqItMIlhm0GbANGqeoKr+39uN1ydwOPq+ozXnkf4DFcNGE+MFpVd4lIA+AF4GigALheVT/w+tyAS0K7D3hZVe/zyrsA44BDgHXAlaq6qaLnmev4uTz6d+xF56YdfNv/bd5rFFFUpr8+yJUyNMnanSClBJRQSgCLv/6cxvUasG33d8Vlvdv3YFacgkgmx4TLHwpMSzSke38emPFXPtngvw5paPd+DOtxLl9sXRsqiCOReAUfJN/F3c8J7B/jmGbt0/72X5WTxsZTXRSs4aiU/ZREpAfwF9z27H/wip8AXlXVJ7xw9PHAaSIyGJdpoitQH5gvIu8Cq3CK7AJVXSwiY4E7gbuA+4DlqjpYRHoCr4jbjrULMAY4EdgDzBKReao6wxvrLlV9U0TGAI/iduet1vi5PKavnMdX3232bf/vZf/l38v+y5Bu/Zh4RbC/PsiVsmf/PhrVO5Ttu3dWWPaYUjqueSfuPu9aBnU9K9RGe/GKIShS8KJuZxe7KTs2aUMtKFZEF3Y+gzeWuh1eig5k4wo8nh+xN/cg+dZ+G+zGA6eEM6Esqks+u+qiYA1HZW30dy1wN/ALABGpA5wLXAagqlNF5GkRaQsMAF5U1b3AXhGZDFyK2xNqg6rGnOzPANNwimkAMNAba4GIbAVOBc4EJqvqdu+4E4HhIrIMaKWqsTjoccDDIjJSVfdl8kJkmyCXR/7mL5L2e23JDC755FwuCnizDxr3jU/npCRfGD79egXzV39SyoUHwQtwl29elXTuJ8hNWdaketh5q6OatGHSklk8O+9133q/MPgYia7DdFMd8tlVFwVrOCpFManqtQAi8guvqAmwR1XjX6M3AK2BVrhQ9BhfAZ298vUJ5a29z0F1rYBlAeXF7T134G6cW/GrVM+vsLAQVfWtKygoiB0j1WEzwsF7yh+fMuG9KXSuV3KbhmhRlLe//JD3Vi2oqGgpMXbOeN5YMptrew7hnI4nE/Ei81as/dK3/UsfvMmz773Oim0H8tV1b34Mzw38P2pHgv8bzFw539fC+WHzbpzd4eRQ17N/x148859XmPHl/DCnVswP2/Rg6LFn07f9iXzxRfIXhxix32PNjo20bdiCszqcVHxtIPfuRz8qIuOxB7Xi2FZu+cPy5f6u2XRQ3a9jZVBYWBhYl62t0SOQ4BNxkX2FPnVllfuNl+pYieNVW87qcBL9O/YqtT1DGJZ/s5aZK+cXP+z89h9KJ80Oaczmgm1J5FnDLTMfpX/HXjxyzo1EakVoVO9Q37Z+FuHirz9n1NR7GD/onhIP73jW7NjoW77qW/f+4nc9+3foxXmdTmXtjo20a9SSwmiUW2Y+GngePZofwyKfhblDjz2bszuET1Pp93vEXxvDqCpkSzFtAuqJSH1Vja1obImzYtYCR8a1LaucuLrt5RlLROoDdYBvynMyeXl5uCmt0sTeVoLq00GqCwunHvMkd735R9/dRZOxbOsqbpoxttiVNSV/tq9S+vEJF/DGJ3P4bm/5FqvGSKaU4pm+ch6j917C4G5n0XvPifzhg5dCH2PR15+jezeU2NxuSv5sVmxezdaC7Wzc961vv/mblvLg0bcRiUSYesyTTM2f45tBA+B3bz/rO8Z5PziNq3oN4cLOZ3DFxF+Umh9JHKcsJi2ZVer3iL82UPb9mAuLVCvj/0xFMRkrzqJFiwKtpqwoJlXdLyLTgauBP3rBD1+p6joRmQbcKiLPAgcDFwGXAJ8CrUWkp6ouwOXri6VImuZ9v8ULfjgS+BDYBbwsIg8C3+OCGx5R1bUislFEBqrqVE+O6VVxfqk8CwsjkQj3XzCm1O6iYYmtM5qSP9u3/r0vF1VYKaVKLLjAbxI8bN9U9pCK36oiWQYNCI4Yu6rXkGKFkY75kYqGTNsiVSNXyObddh1wgYh8CvySA4EQk4BZwAJgPvCYqi70giEuBZ4Rkc+Ao3ABFXh/23hjPQeMUNU9qroQGAvMBT4B3lHVyV6f4cAdIrIUuBgXUl7lCIryCkp2GiM2WfyPK39Hp8Ztkrb1477pfwnMobfqm/W+5ZkkFhYcO69b+o5MuW/YCLsYYR/4yRLmxogpt1vPGlWs7FKloiHT5b2XDCPdVKrFpKp94j6vx0Xm+bW7mwNKJ758LtDTp3w7zqryG+sp4Cmf8qXAaSFFz1nK+5Zc7LLZupafnjiUT75eznNLpiXtE09+wJqfbDC0ez+iRYX87u1ni91P918whv9+ubDM7Si6tjyawqJCotFoypkhgh74fu6wyogYq2jItC1SNXKFbM0xGWmiPG/J/otsT2FIt3NKbf3tR73aB7F7/57Uhc0AJ7ftSrSoiB/FbY8ecz+9fd04zn7y6hIJW49p2o6ebTrz3y8XsvbbjXyyYTnDx9/GkG7nMOx43/ckX4Ie+MlSO6USkl2euZ6KhkzbIlUjVzDFVMUZ1KUvQ7r147UlJR+Eyd6S/RfZzuelKx5hz/59TPMWkwaRK0oJ4IM1n/DBmk9KlMUvqJ113Th+9a8/kb9hOV9uXc/nm1fxuY8F8NqSmQzp0T9wfipZ9od4wqR2KouKzPVUZE2SLVI1cgVTTNWAxGwERUUHvie+eQ/ofCZTPnnbd5zbp/yu1FhVlclLZrJ88yreyJ/DvCSLV+O5760/M/yEC2h3eCsaH9yAb7//jsMOacgxzdqHtjyCwstTcYdlKyGpLVI1cgVTTFWcKfmzeT3B/fbakplcmj+HgV36lHrzbt2oOesCslivTZLduqrx4oJ/ptzns69Xcvc/Hwf8E69OWjKrTNda24YtfMdOxR1WnrmedIV5V4csEEbVxxRTFSYajQZaPyu2rPZ98w5SSlWBlvWb0K5p67TuqRREmMSrfq61szqcVGF3WKpzPRbmbVQ37K6tosQeRkEh21t3bQ/My1ZVufS4/rx9/Thu7nMF9WrXzfjxykq86hdGHanl3GEvjxzLAxfeyMsjx/LCZQ+mpCDChJfHY2HeRnXDLKYsUVHXS7I1N60bNU85q0NVYNvu7zj50eFpCVW/6YzL+fuif7Nhh39WdThgoaTqWquoOyzVuR4L8zaqG6aYskB5XS/xyiz/K/+Hc5P6h1Vpd10yxqewzqosXl0yM6lSirdQshFGnYpyszBvo7phiikLpBJ1FVNGyzevZlr+O2XOr2zd5Z/bzShJ4kaEMc6VU7m699ASFkquh1HnunyGkSqmmLJAWNdLKrnbajqN6h3KVadcTNNDD6NT07YUASu3rqVjkza8svCtEuu82h7WkjUBiunq3kNLvRzkehh1rstnGKliiikLhHW9pJq7rSazffdOHn3neYZ278ctfUeWeCgP7noWl+afz+ebVyW1OpPtEpvrYdS5Lp9hpIK9UmWBsFFXqeZuM/yj0WIP7aObtQtUSqe2786sDO4SaxhGeMxiygJhXS8dmrQOGMEAaNGgCRu/21qqPCgaLUjRjzjhQsYNv9eUkmHkCKaYskRZrpdoNMqrC9+qRImqHn5KCYKj0YJcqIO6ut8hTGYHwzAyjymmHGVK/uxQmb5rOse16MSnG1cUf0+WwDYoeu3CzmdY5gTDyCFMMeUoNW1+qVWjI1i/fVPK/RocdEiJ70VJctAGuVCzlTTVMAx/TDHlKEFup+rK+u2baHxwQ7Z9vyN0n97tu5fYawngtSUzuDQ/WKH4uVAtc4Jh5BammHIUP7dTdacspdSj+THc1v8q3lz6LrWAKEWlFBOkrlAsc4Jh5BZZV0wi8nfgBGCXVzQeeBmYADQDtgGjVHWF1/5+YCiwG3hcVZ/xyvsAj+FC4POB0aq6S0QaAC8ARwMFwPWq+oHX5wbgemAf8LKq3pfxEw5BLNtDj9bHsrdwP1PzZ2dbpKwhzdpz2UkDaLC/Lme268lvPniuzLVdqSoUy5xgGLlF1hUTcArQS1WLE7yJyGTgVVV9QkQG4pTVaSIyGOgDdAXqA/NF5F1gFU6RXaCqi0VkLHAncBdwH7BcVQeLSE/gFRERoAswBjgR2APMEpF5qppVE8Uv20MtqCbb96XGrX1H8psLxhCJRFBVZq6cX6ZSKo9CscwJhpFbZFUxiUhz4HDgORFpBcwGfgmcC1wGoKpTReRpEWkLDABeVNW9wF5PgV3q9dugqjG/zjPANJxiGgAM9MZaICJbgVOBM4HJqrrdk2UiMBzIqmLym4iviUoJ4PD6jUooh6DdYds2bsk1pw7j6BR2mk3EMicYRu6QbYupJTATuAHYAjwPjAX2qOrOuHYbgNZAK+CNuPKvgM5e+fqE8tjq1KC6VsCygD4pUVhYiKr61hUUFAAE1icy77OPyyNCteSQvbWLr1tBQQHN6zX2bbdm2wbq76vDsQe1Yvnyim+JUV5S/a2zgcmYHkzGilNYWBhYl1XFpKqLgCGx7yLyEPAPShsJtYBC3PxRUQrllLNP1mjdoHm2RcgKhx10KN/uOfAucm7H3vRtf2KJNme26cnRh7dl+Telo+hWb/dPymoYRtUj26683kAzVZ0aJ88+oJ6I1FfVWEBES5zVsxY4Mm6IssqJq9ueQp+UyMvLw01blSb2thJUHyMajTL5k1k8m/9G0nbVlb/86NfUqhUJnOOJXcf7BtzAj8bfVqp/72NPLPMaZ5qwv3U2MRnTg8lYcRYtWhRoNWXblVcXeNwLYNgO3AS8AhwPXA380Qt++EpV14nINOBWEXkWOBi4CLgE+BRoLSI9VXUBMBo3x4T3dzRwixf8cCTwIS4K8GUReRD4HhgBPFIZJ51ITd/eYliP/gzqelaouaHBXc+2CLoMUtGdlQ0jHWTblfeOiPwBeM+TZQ7wAC5M/G8icg1OgcQCISaJyPHAAq/9Y6q6EEBELgWeEZGDgc+By73D3A08LSKf4lx1I1R1D7DQi96bi1OQr6jq5Eo47VLUxO0tbu07ksPrN0o5As4i6DJHeXdWNox0k22LCVV9FHg0oXg9LjLPr/3dOGWTWD4X6OlTvh1nVfmN9RTwVIoip52aln5oWI/+xWHg5cEi6DKDpWYycoWsKyajZqQfKq+FZFQelprJyBVMMeUAAzqfScN6h7Jj986yG1dBhnbvVyELqSLYnEl4LDWTkSuYYsoB3lj6TrVTSqd36Enbw4/kwuPOoFatWoydM77SFYPNmaSGpWYycgVTTFkk9jb/7LzXsy1KWhnWoz8vXPYgQFYVg82ZpIYFlhi5gimmLFEdQ8R/cEQHdu/fy4btW3h9yQwitSK+iqHd4a1oUr9Rxi0omzNJHQssMXIBU0xZorqFiNeulceyTV8CsOqb9cz9cgFtDmvh2/b3s58r/pxJC8rmTAyjamI2epaobiHi+4tKr+Be+61/0tV4Xl08g6n5czIg0YE5k3hszsQwch+zmLJExyblyhebE9TNq83ewv3F31s0aMLG77aWe7xMudZszsQwqiammLJEVdvKovHBDTm5bRe6HnkMvz73Ov65bG7xw/79VYt4dM7zZY5xescTeHdl6ezpHZu0yYTIgM2ZGEZVxBRTlvhy67psi5ASt/S9ktvPubr4e/zD/sLOZ/Dywn+zfvumpGP87PThHHFoY15bMrNE+SsL32JwyFx5hmFUf+xJkCWqWraHY47oEFhXu3Ztlt/1T27ucyU9Wv2A0zuewCltu5ZoE0vUOuz40pmmXluSuXkmwzCqHmYxZYlBXfrSpcXR5G/M3sZ2YendvkeZAQO1a9fmwYE3FX+PRqNMzZ9Tam5nZYClaCHchmHEMMWUJSKRCL889ye+ewvlEr3bd2fWdc+k7GYLmtuxEG7DMMrCFFMW8dtbKFtcePTpHNOqAxu2b2Hd9q9p07gFg7r0Db1PUlgs7Y1hGGVhiimLxIczf77pSx6cOY6dewsqXY7+HXqxZ/9exsZF1g3t3i/tSgkshNswjLIxxZRl4l1e/3vG5fT8/TA+35ze+ZbG9Rowps8VTF82l/dXLS4u79ryaO7qfw3rv1rPLTNKbomVyZxyFsJtGEYyTDHlEHXr1mXJHZOY8snb/Ondl3zX/MRzYeczeWvZXPZFS2ddiOfWs0dx29lX8YtzrvYNSLhj2cO+/SwgwTCMbGD+kxwjEolwUfdzmHHd06XS6cQzrEd/Xr3qUe4+77oyxzy6WfvisQd3O4tbzxrF4G4H3HRtG/rntLOA5grqTAAAC6BJREFUBMMwskGNtphEZBhwD1AIzAFuUtX9STtVEolzMUc1aUMRsHLr2hLWzjFHtE86TpjAgrM6nET/jr2YvnJeSv0MwzAyQY1VTCLSAvgjcCKwAfgHMAp4OptyxRNmLsYvyu3U9t254LgzObpZ+1CBBZFaER4550ZG773EAhIMw8g6NVYxAf2Auaq6HkBEngVuJYcUUxjSFeUWqWUBCYZh5AY1WTG1AtbHff8KKFfK78LCQlTVt66gwIV/B9Wni2MPasWxrVoBsHx5atkkKkvGimAypgeTMT2YjBWnsDA4aKsm+2oilEzyXQs312QYhmFkkZpsMa0Fusd9b0lJCyo0eXl5iIhvXextJag+FzAZ04PJmB5MxvSQ6zIuWrQo0GqqyYppBvCwiLTDKalRwLTsimQYhmHUWFeeqm4ErsMpo2XADuCJrAplGIZhUKuoqKrtpZpbfPzxx1GgVl5enm99zFQNqs8FTMb0YDKmB5MxPeS6jJ58RSeccEIpA6kmu/LSRRSIFBYW7kjWKFkESq5gMqYHkzE9mIzpIYdlbIh7fpbCLCbDMAwjp6ixc0yGYRhGbmKKyTAMw8gpTDEZhmEYOYUpJsMwDCOnMMVkGIZh5BSmmAzDMIycwhSTYRiGkVOYYjIMwzByClNMhmEYRk5hiskwDMPIKUwxGYZhGDmFJXHNICIyDLgHtzPuHOAmVd2f5mP8HTgB2OUVjQdeBiYAzYBtwChVXeG1vx8YCuwGHlfVZ7zyPsBjuJeVfGC0qu4SkQbAC8DRQAFwvap+4PW5Abge2Ae8rKr3JchWF/gn8AdVnSYirbIpl4h0AcYBhwDrgCuBbxNkPA+YiNujC2CbqvbNoozzgWG4ZJcrgdFA3Ry7jn4ynpBj13EucJkn40fAtbjnX4XvIVXdJCJ1gKeA04A9wC9VdZrXx/c54PP/YY53XeJlbAd8CHzBAXqr6vdZkrH4XsskZjFlCBFpAfwR6IfbKbcFbjPCdHMK8ENV7eH9exS3r9SrqtoFeASnrBCRwcD/b+/cg62q6jj+8cZTUbkhIA+zEZ3fjBdGgkpqkqjEHtMNH02pYBZgwKAOlpKjE9NwK8IhG4GBmopAmJwiHXWuGT4mG00bx0GBsvmSM6V0gQEfaCCIAv3xWwf2OZ1zz+VyDmdP/D7/3Lsfa63vXmft9dtr7bV/vwnAqPR3rjm98cb3NUmjgG3AbSn/NuAfklqAGcA9ZtbDzEYDN6byPwJ8zswmZq5/NPAk8PGM1kbrWgPMT3k9DKwso3EcsDBTn4XOtBEaNwNzgAvT9ovAgpzVYyWNearHDcCtSWML0AeYVYN8HwZ+kvbPxr1lnw9MApabWXOVfiD7O/4WuKWMxnHA6kw9jk5GqREaD7e1ehOGqX5MBJ6S1CHpIPAr4KpaFmBmg4H3AyvNbKOZ3ZWeLj9LakCSHgTONbMPAK3AryXtl/QGcD/wVeCjwDZJG1LWv8hobcU7cCStB17DO/JW4H5Jb0rahz8dZ69vJjAPKDyB9mykLjM7Cxgm6aGU/pfAJfhT4rMZ3eOAS8xsg5k9mp4+C+Udb42rgL740y3AemBEzuqxnMazc1aPbUBPYK+Z9QMGpTxq0YauSG27FVgl6ZCkl4Fn0r6y/UCZ+2EJ8CbQv0TjOGCkmT1vZn82s4sydXK8NWbbWl0Jw1Q/hgEdme2twPAalzEEeAyYBnwYGAzcCbwjaXfmvG2p7EqaOtPanTRImilpXeb4gAbrKtovaQ+wG3iBYl4Hlkq6AFgKtJtZ3wZpXA/sBQaa2WnAd4FHyFc9ltN4H/mqxz34tOEMfGrxDODBGuY7sBt5VbofppVofBs3OmOBm4B70winkRpr3Y/9D/GOqX40AdlgVyfh87c1Q9ILwBWFbTNbCPympNxs2ZU0daa1O2nKUXr+8dbVWfmHkXR15v8HzGw+8KEGayx0Uk/jT69zK1xHLjRKWo6/ywByVY9r8ampO/BRQS3bUKU07zvKvH4PfK+gUdKlmePPmtkzwKcbrLHukQdjxFQ/tgBDM9tDKH4qOWbM7GNm9qXMrh74i9A+ZnZKmbIraepMa3fSlGNHg3UV7U86euJP9of3mdntZnZSJn2hThulsRfQDjwgaSb5rMcijTmsxxagN/C6pEO4cR9do2svtKGjzavodzSzc4CzgI6sRjO7LU3PFziqeqylxpJjdSUMU/14FBhvZmebWRP+MrG9xmX0ApakF5hN+FB/LT7dMw0gGa6tkv6dyp9iZn3MrBm4DHgIX1U13MzGpHynZ7S2p23S8aH4KqF24PJUdh9gcmfXJ1+N2DBdkrYA2zOGfBrwiKR3Mxr3AFOBy1O+n091/HyDNM7B79FblVZc5bAey2nMWz1OxVe69Uvbk/GVZ7VsQ+3AVDNrSu9gLsKn2cv2A6W/Y6qrfhxZXVvQeDHwzaTxAnxK77FGaCxpa3UlQqvXETO7DJiPP609CczMdoQ1KuMmvOH2wBvybHw+eQU+F7wHmCrpr+n8+fiS1B7AYklL0/5P4Ctz+uIrra6R9JaZnQ78HGjBh/A3SPpTSjMLuB7vdNZKKqyiyup7AlikI8vFG6bLzM5PaZrxp8HJkjpKNI4BlgGn4u+grpO0sUEaT8WnyJSpUgHfylE9VtK4MEf1uANYhy8XPwBsTGkO1SDfQhvqlbSPx6e75klam9KU7QfK3A/rcAOV1diMLwQZnPbPkfTHBmo83NbqSRimIAiCIFfEVF4QBEGQK8IwBUEQBLkiDFMQBEGQK8IwBUEQBLkiDFMQ5ARzP21Dq58ZBP/fhGEKgvxwD/CFY8nAzFaa2fePVYiZ7U4ffQbBcSdcEgVBfhjYaAEFJPWrflYQ1IcwTEFQB8zsWvxD2A/ijjh/JOmudOwzwI9xT+F/B64Drsa/xB9nZoPwjybPlTQlpZkOTJE0IbmIWYz7TBuMh5r4hqRNneiZACwC/ob7V/wnMEPS0+nYYtxtzUjcw7aA8yS9VE6vpA1J59Kk4w081MLqVN4U3HN7c9I3R9Jz3a7Q4IQipvKCoMaY2Qi8o58i6XTc8NxhZqeZ2RA81MMPgP7A3XiQt+/gX9xfL+mHVYq4BY+ZMxLv+AXc3gVpY4F/4aFSlgP3mXv8Bo+P9FPgHEmbM9dSVm86vAbYiXun/nK6xgvN7GTcW8Ek3EP1H/AYTUHQJcIwBUHt2QKMkrTJPETBe/gI6Az8HdImSWslHcANxLVW7PC0GkuAa4D9eOyjXRQ74azEa0CbPBbSMjyGUiG+z7u4W5u3StJU0jsMHynNlfSOPI7SCuDrKa+9uKusscACSRMJgi4ShikIas97wI1m9irwOB52HPx+G0RxvJwDkv6SPEp3lWbgd8B2fGRyHl27l7ckx5wFtuJTgQA7k+Eppaxe3HdaE9BhZrvMbBceUXVY8gd5MW40nwBeSVObQdAlwjAFQe25Eh9pmDxM9s2ZYx0UhyVoMrNF5kH2shyk+B1wc+b/Zbg37oGSxuPhs7vCmYWRWfo7nCNGp5JhLKsXDxi3Dxggqb+k/riBnGYepuFkSZPwacObgRVmNqCLOoMTnDBMQVB7+uPTbPvNw2QvTPt74kakxcwuTSEGZgFfTFNo+zgSmuEl4JNmNjB92zS9JP+3JR1MnqRnp7yrcSY+kusJ3IB7q36qSppKel8BngPazKx3mtp7HJ/KOwVYZ2afkrQfeBWf2tvXBY1BEIYpCOrAKuBlfLSxGV+V9yI+gtqJLwqYh69ku4oUtwj/jqnNzBYA9+KLBoTHzFmdyf/bwGQz+w8ef+tuYISZVVtluwMYgy9YmAy0JsNRkSp6rwQsXef6pPdOSdvxmD4/M7M9+EKQr6Q4TUFQlQh7EQQnAGlJ+BpJwxutJQiqESOmIAiCIFeEYQqCIAhyRUzlBUEQBLkiRkxBEARBrgjDFARBEOSKMExBEARBrgjDFARBEOSKMExBEARBrgjDFARBEOSK/wKZVUAkU5gxCgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# scatter plot of actual price and predicted price of the linear model\n",
    "plt.scatter(y_test_transformed, predictions_dict['lr_pred3'], s=20)\n",
    "plt.title('Scatter plot of actual and predicted prices')\n",
    "plt.xlabel('actual prices')\n",
    "plt.ylabel('predicted prices')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see in the above plot that the predicted prices are somewhat close to the actual prices, but not very close. The best case scenario would be for the scatter plots to follow a perfectly straight diagonal line, indicating that our model perfectly predicted the house prices. This is one visual indication that a linear regression model does not best fit this data. \n",
    "\n",
    "Let's take a look at a histogram of the residuals of the best performing linear regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAewAAAENCAYAAADNMlZHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAbW0lEQVR4nO3de5hddX3v8XcyEUIABRWFJKNQxa+KaCBYqNRC+xRFLRQVpEi0InjAUOB4w2otXhAv+AgiAseCF45c5HKOEYOtYJ/SA7VVCwQt1q9SFUMSpCL3AQKTOX/81sjOOJlLsnZmfrPfr+eZJ7N/e63f/n332pPPXr+19tqzhoaGkCRJ09vsqR6AJEkan4EtSVIFDGxJkipgYEuSVAEDW5KkChjYkiRVwMCWNoOImBMR86d6HJLqNWeqByBNZxExBOyambd1tO0M/Bx4EjAf+BHwlMwcHKOrS4FvARd0b7TdFxHvAf4WWJ2Zz+9C/0cCf5mZrxjlvo8CCzPzLZvQ/y+AYzLz2xvbhzRVDGxpE2TmL4FtJrDoDt0ey2byP4BjM/PSbnSemRcDF3ejb6l2Bra0CUbsbc+m7EG/BhgArgWWAh8GXg7sExHPAD4OvLe5bx7wT8BfZeavImIW8FHgOOB+4O8oAblzRLwFeAuwHfAM4AXAIcA7gZ2bx/xEZp7VjOsG4H819w9Qwvb1wBuAXwCHZmaOUtNBwGnAs4FbgRMz898j4kbgOcAXI+L5mfnBEesNAZ8HDgf+J/B/gDOAg4FHgc9l5qeaZV8BnAksAP4LeH9mfqup8ZjM/MOI2Bo4H/izZpkfN/0QEdcBF2XmBc3ti4DbMvNDEbEr8FngJcBTgG8DSzLzgRHjfRfwDmBL4GbguMz82cjnQ5ouPIYtje+miLh3+Af4wQaWexMl5BYCL6IExmGZ+V7gekoofwx4O/A24E+BfuBeypQ5wNHAocCewO9Twq7Ty4HjgecDT6cE05LMfErT5+kR8eRm2QWUNwQ7ABcCV1FC/GnAfwAnjywgIl4MXNbc9zRKYP5DRDw1MxcDvwQOGhnWHQaBHYErKWG9A/A8YD/grRFxeLPcF4CTM3M74DPA50bp62PAMynP51HAgRt4zJHOB/61WW8XYFfgyBF1Phf4ALB3M96fA++bYP/SlDCwpfHtmZnbDf8AL97AcvdTgnQJJShfmplfGWW5NwKfzsyfZObDlL28/SJiYXPfGZl5e2b+N/ChEevenpn/kpn3AyuB3TPzhxGxI/A4sAUlyIed1Rxb/3/AA5n5pcxcS9mr7x9lbG8AlmfmP2Tm45n5JeBnwKvHfIaecFlmPgo8THkD89eZeX9m/oIS4G/peK6WRMTLKW9WnjdKX6+jzBjcn5krgNGey9G8GfgksBXlTcvdlHMNOj1E2UZvAwJ4e2a+bYL9S1PCwJZakplXAB+h7EHfAVzX7MmNtANwe8d6DwL3UPYIFzTrDrt9/VVZ0/H748CJEfFr4B+Bw5r2zr/r3zT/DgL3dbSvY/S///XG1vhlM7aJGB7fDsBc4LsdMxOf5ongPBgYAr4O/IpR9vYpe9erOm6PHNeG7EaZ4v4pcCplWny9WjNzDeXQxb7ATcBPIuJVE+xfmhIGttSS5tjptZm5JyWYVlNCaqQ7KFPnw+s9GdgeuKu5rzMcRwZl59fr/QVlzzcyczfg3aM81mS/jm+9sTV2bsY2EcOP92vgMeCFHTMTuwCvjogtgJ0z842UYP8L4EMRsceIvu4EntVxe6eO39ex/jk42wM0fV8OvDsz52fmn1Gmu9cTEU8H7s/MAyhT/58HvjrBGqUpYWBL7TkY+EoTBr8BHuGJPdxHeOJs8ouAd0XE8yJiK0qo39ic8HQRcFJE9EfEUykfodqQ7YC1wNqI2IYyDQzlBLiNdRnwmog4sPns+FGU6epvTqaTZhr+q8DHI2KbiNiechLa+ymhfnlELKEE751N270jurkE+EBEPDUiXkiZ6h52G3BIRDypmVb/46Z9S8pU+IMRMas5ge5Afvc5eTbwrYjYPTMfah77N0jTmIEttedsyglpP6YcN30q5WxwKMdpT42IjwNfppxNfg1lOngH4LXNchcCfw/8ELiRMl27dgOPdyFlmngV8BPKmeA/ohyT3SiZ+RPKWd6nU0JsKfCqzLxzI7o7gTIVf1vz8wvgPZn5GGX6/j2UY9nfAJZm5sg94Q9SnsufA/8XWN5x30cpU+53N8td3Iz/AeAk4ApKAL8L+BIjnpPMvBE4Bbg6Ih4Cjm3qlqatWUNDk50xk9QtEbEIWJOZv2puvwr4YGbuM7UjkzTV/By2NL0cBOwdEW+g/H2eSPk8t6Qe55S4NL2cQTmbeyVlKvgXlIuYSOpxTolLklSBCU2JR8RXgcWUiw1AOdnlcsoZrTtQPkN61PAXJETEaZSrNT0CnD18+cDJuPHGGx+nzADcP9l1JUmq0JOBdYsXLx41myd6DHtvYJ/hE2EAImIZcGVmnhMRB1NCfN+IOATYH9gd2Jpy4YTrR7tm8ThmA7P6+vqeMsn1RjU4WL5Iqa+vr43uqtCLNUNv1t2LNUNv1t2LNUNv1N3UuMFD1eMGdkQ8k/LxlC9HxALKJQ0/ALyScglGMvOqiDg/Ip5FOWnmkubyh2ubYD+ccgWoybgfeMpWW201ydVGNzAwAEBb/dWgF2uG3qy7F2uG3qy7F2uG3qj7wQcfhDFmlSdy0tlOlG+7ORrYi3K5wDOAR5tLKg5bwxOXVuy8nOBqJn5ZQ0mSNIpx97Cbi+6/fvh2RHyScjWkkWerzaJcJGH2iPuG2yetr6+PiI2+BsR6hmfk2+qvBr1YM/Rm3b1YM/Rm3b1YM/RG3StWrPjt1P9oxt3Djog/aI5RD5tDuUbw3Ob7aoftRNmzXsn634wz3C5JkjbSRKbEtwDOjojtI2I25asAr6BcVvFogCbQV2fmHZTLBy6JiLnN9YNfC1zdldFLktQjxg3szPxnyhfMfwdIyke7Pka5xvBrIuJWyklowyegfY3yVX83Ad+lfB/vzV0ZvSRJPWJCH+vKzDOBM0c0r6KcKT7a8qdQLqwvSZJa4KVJJUmqgF/+Ic0A/f39Uz0ESV1mYEszwNpZg9yyarIXExzdSxYE2221bSt9SWqPgS3NALesSg4495hW+rp26QXs99y9WulLUns8hi1JUgUMbEmSKmBgS5JUAQNbkqQKGNiSJFXAwJYkqQIGtiRJFTCwJUmqgIEtSVIFDGxJkipgYEuSVAEDW5KkChjYkiRVwMCWJKkCBrYkSRUwsCVJqoCBLUlSBQxsSZIqYGBLklQBA1uSpAoY2JIkVcDAliSpAga2JEkVMLAlSaqAgS1JUgUMbEmSKmBgS5JUgTlTPQCpF9378APcsipb6WvfXRa10o+k6c3AlqbALauSA849ppW+7jrt+lb6kTS9OSUuSVIFJryHHREnAW/KzL0iYlvgK8CuwABwfGZ+r1nuBOB44DHg8sw8tf1hS5LUWya0hx0RewAndzSdCvw0M3cDjgUujYg5EbEIOBHYG3gpcGBEHNDymCVJ6jnj7mFHxDbA54H3Ayc0zQcBBwNk5k0RcTfwMmA/YFlm3tesezFwBHDtxgxucHCQzHZOzBkYGKAZbyv91aAXa4bpX3d/f/9UD2FcAwMDrFy5cqqHMa7pvq27oRdrht6oe3BwcMz7J7KHfR7waeD2jrYFwKqO26uBhWO0S5KkTTDmHnZEvAVYm5mXRcT+HXfNBoY6bs8CBsdo3yh9fX1ExMauvp7hd2Vt9VeDXqwZerfuNs2bN6+K568Xt3Uv1gy9UfeKFSvG3Mseb0r8SGB+RKwAtml+/zawEpgP3NcstxNlz3q4nRHtkiRpE4w5JZ6ZB2Tmbpm5CDgG+FFm/imwvLlNROxJCenvN+2vi4jtI2IuJfCXd7MASZJ6wcZeOOUU4PyIuJUy5X1kZj4K3BwRZwA3AFsAV2TmsnaGKklS75pwYGfmdcBeze/3AW/YwHLnUU5UkyRJLfFKZ5IkVcDAliSpAga2JEkVMLAlSaqAgS1JUgUMbEmSKmBgS5JUAQNbkqQKGNiSJFXAwJYkqQIGtiRJFTCwJUmqgIEtSVIFDGxJkipgYEuSVAEDW5KkChjYkiRVwMCWJKkCBrYkSRUwsCVJqoCBLUlSBQxsSZIqYGBLklQBA1uSpAoY2JIkVcDAliSpAga2JEkVMLAlSaqAgS1JUgUMbEmSKmBgS5JUAQNbkqQKGNiSJFXAwJYkqQJzJrJQRJwEHAesA/69+X0O8BVgV2AAOD4zv9csfwJwPPAYcHlmntr+0CVJ6h3j7mFHxIuBk4C9M3M3YC7wduBU4KdN27HApRExJyIWAScCewMvBQ6MiAO6VYAkSb1g3D3szPxBRERmPhYR2wDPAO6m7EEf3CxzU0TcDbwM2A9Ylpn3AUTExcARwLWTHdzg4CCZOdnVRjUwMEAz1lb6q0Ev1gzTv+7+/v6pHsK4BgYGWLly5VQPY1zTfVt3Qy/WDL1R9+Dg4Jj3T+gYdhPWS4CVwNOBq4AFwKqOxVYDC8dolyRJG2lCx7ABMvOiZm/5dOBLlLAf6lhkFjA4Rvuk9fX1EREbs+rvGH5X1lZ/NejFmqF3627TvHnzqnj+enFb92LN0Bt1r1ixYsy97Ikcw/69iHgZQGYOARcCiyh72/M7Ft2Jsme9oXZJkrSRJjIl/gzgkojYvrl9JHAdsBw4BiAi9qSE9Peb9tdFxPYRMbdZfnnL45YkqadM5KSzf4uI04EbImIQ+AHwV5Rp7/Mj4lbKlPeRmfkocHNEnAHcAGwBXJGZy7pWgSRJPWBCx7Az81zg3FHuesMGlj8POG8TxiVJkjp4pTNJkipgYEuSVAEDW5KkChjYkiRVwMCWJKkCBrYkSRUwsCVJqoCBLUlSBQxsSZIqYGBLklQBA1uSpAoY2JIkVcDAliSpAga2JEkVMLAlSaqAgS1JUgUMbEmSKmBgS5JUAQNbkqQKGNiSJFXAwJYkqQIGtiRJFTCwJUmqgIEtSVIFDGxJkipgYEuSVAEDW5KkChjYkiRVwMCWJKkCBrYkSRUwsCVJqoCBLUlSBQxsSZIqYGBLklSBORNZKCLeDRwFrAN+BhwDbAFcBOwA3AMclZm3NcufBhwKPAKcnZkXtD90SZJ6x7h72BHxR5Sw3jszdwd+BHwcOAe4MjNfBHwKuLBZ/hBgf2D35t+TIyK6MXhJknrFRPaw/xtYmpkPNrdvApYC+wBLADLzqog4PyKeBRwEXJKZa4G1EbEMOBz4yGQHNzg4SGZOdrVRDQwM0Iy1lf5q0Is1w/Svu7+/f6qHMK6BgQFWrlw51cMY13Tf1t3QizVDb9Q9ODg45v3jBnZm/ifwnwAR8WTgb4FLgT06QhxgDbAQWAB8o6N9NfDCSY1akiStZ0LHsAEiYj5wFfAdyvT3ySMWmQUMUqbZh0Zpn7S+vj7amk0fflfWS7PzvVgz9G7dbZo3b14Vz18vbuterBl6o+4VK1aMuZc9obPEI+LFwL8BX8/M44C7gLkRsXXHYjsBq4CVwPxR2iVJ0kaayElnOwLfBv46M08FyMzHgWuAo5tlDgZWZ+YdwHJgSUTMjYjtgdcCV3dp/JIk9YSJTImfDGxDOdt7eBo8KSeefTEijgUe4okT0L4WEXtQTk6bA5yVmTe3PnJJknrIRE46eyfwzg3c/coNrHMKcMomjEuSJHXwSmeSJFXAwJYkqQIGtiRJFTCwJUmqgIEtSVIFDGxJkiow4UuTSr3q3ocf4JZV7X3hwL67LGqtL0m9w8CWxnHLquSAc49prb+7Tru+tb4k9Q6nxCVJqoCBLUlSBQxsSZIqYGBLWs9znt4/1UOQNApPOpO0nm22nNfqmfEvWRBst9W2rfQl9TIDW9LvaPPM+GuXXsB+z92rlb6kXuaUuCRJFTCwJUmqgIEtSVIFDGxJkipgYEuSVAEDW5KkChjYkiRVwMCWJKkCBrYkSRUwsCVJqoCBLUlSBQxsSZIqYGBLklQBA1uSpAoY2JIkVcDAliSpAga2JEkVMLAlSaqAgS1JUgUMbEmSKjBnogtGxBbAN4HPZObyiFgAXATsANwDHJWZtzXLngYcCjwCnJ2ZF7Q+ckmSesiE9rAjYhFwPfCyjuZzgCsz80XAp4ALm2UPAfYHdm/+PTkior0hS5LUeya6h30ccArwPoCIeBLwSmAJQGZeFRHnR8SzgIOASzJzLbA2IpYBhwMfmezgBgcHyczJrjaqgYEBmrG20l8NerFmaLfu/v7+Te5DZZusXLmyK/1Cb73Ge7Fm6I26BwcHx7x/QnvYmXlcZn6ro+lpwKOZ+WBH2xpgIbAAWNXRvrpplyRJG2nCx7BHmA0MjWibBQyOct9w+6T19fXR1mz68LuyXpqd78WaoXfrns7mzZvXle3Ri9u6F2uG3qh7xYoVY+5lb+xZ4ncBcyNi6462nSh71iuB+aO0S5KkjbRRgZ2ZjwPXAEcDRMTBwOrMvANYDiyJiLkRsT3wWuDqlsYrSVJP2pTPYS8FXhMRtwIf4IkT0L4G/CNwE/Bd4KzMvHlTBypJUi+b1DHszNy/4/dVlDPFR1vuFMpZ5ZIkqQVe6UySpAoY2JIkVcDAliSpAga2JEkVMLAlSaqAgS1JUgUMbEmSKmBgS5JUAQNbkqQKGNiSJFXAwJYkqQIGtiRJFTCwJUmqgIEtSVIFDGxJkipgYEuSVAEDW5KkChjYkiRVwMCW1FXPeXr/VA9BmhHmTPUAJM1s22w5j3sffoBbVmVrfb5kQbDdVtu21p9UAwNbUtfdsio54NxjWuvv2qUXsN9z92qtP6kGBrZmnP5+p2AlzTwGtmactbMGW5t+3XeXRa30I0mbysDWjNPm9Otdp13fSj+StKk8S1ySpAoY2JIkVcDAliSpAga2JEkVMLAlSaqAZ4lLqs7w5U79zL16iYEtqTq/vdzp6nY+b++lTlUDA1tSldr8vL2XOlUNDGxNuTa/GMIrk0maqQxsTTmvTKap5leAqgZdC+yIOAz4MDAIXAe8IzMf79bjSdLGavsrQPfdZREPrn3YrxRVq7oS2BGxI/BZYC9gDXAZcBRwfjceT5I2VdszPW1/pejPTvkW2221rWfG97BZQ0NDrXcaEW8CDs7Mw5rbrwbenZl/MtE+brzxxnXArDbHNXu2HzvfVLNmtbpJABgaGmJw3WArffXN7gOYtv11o89e668bfU73/ob77MbfX5u6kSdta/s5HBoaYt26da12uXjx4lHDqltT4guAVR23VwMLJ9nHOsqFXe5va1AtP6lq0ayW3psNb+Pp2l83+uy1/rrR53Tvr7NPzWhPpmTfqLoV2LOBzrdasyjHsids8eLFnhAnSVKjW3PEK4H5Hbd3Yv09bkmSNAnd2ou9Fjg9Ip5NCe+jgOVdeixJkma8ruxhZ+adwFJKSP+Ychz6nG48liRJvaArZ4lLkqR2+TknSZIqYGBLklQBA1uSpAoY2JIkVcDAliSpAga2JEkVMLAlSaqAgS1JUgWq+YKNiFgIfA94XmY+2LR9AjgCuKdZ7NrMfE9EbAt8BdgVGACOz8zvNeucABwPPAZcnpmnNu0vAr4AzAPuAP4yM++KiCcB5wH7Ao8CH8jM5c06hwEfpnyxyXXAOzLz8S7XvD9wFuXN1n8Ax2TmQzOl5hH1rwF+1dF0QmZeP9XPwXS0ObdLWyLiq8Bi4KGm6ULgcuAiYAfK3/VRmXlbs/xpwKHAI8DZmXlB074/Lb0euljrFsA3gc9k5vKIWDCVdW7odb8Z6j4QuJhyyWqAezLzj2da3d1SxR52RBxK+U9opxF37QMclpmLmp/3NO2nAj/NzN2AY4FLI2JORCwCTgT2Bl4KHBgRBzTrXAR8JDN3B/4eOLNpP57ylWcvBP4cOC8ito+IHYHPAgcALwF2pFwzvWs1R8SWzTjf3IxzDfD+mVJzp4jYmfLHvKjj5/qpfg66Ueum2pzbpWV7A3/YsX3PpFzC+MrMfBHwKUqIExGHAPsDuzf/nhxF26+H1jWPdz3wso7mqa5zQ6/7bte9D/DJjm0+HNYzpu5umvaB3WywNwOvHtHeB+wJ/E1E/DAiLuz4D/Ug4MsAmXkTcDflRXMQsCwz78vMRyjv9I6IiH5gQWZe3az/BeD1zV7WQcCFmTmUmbcD/9q0HQDckJmrMnMd8CXK3n7XagZ+H1iTmbc0ty/oeMyqax7FPgAR8S8RsSIi3t60T/VzMB1tzu3Sioh4JvBU4MsR8YOIOKvZY3olTXhl5lXAcyPiWZTn/pLMXJuZ9wDLgMNp8fXQxXKPA06hzJbRvL6mrM5xXvddq7uxD/CKiLglIq5t9niHa5gpdXfNtJkSj4g3AueOctfvZebBzTKd7c8AbgDeC/wEOB34O+AwYAHrf53namBh0/7jDbT/dvlmuuURynTVhvpat4H2CduImjc0lrHum1Y1jzTGc3AScA3wPsqe7j9FxC+BbccYw+Z4DqajmsY6bCfg28AJwK+B/w2cATw6fPinsYYnttU3OtpXU2Y/2vyb6IrMPA4gIt7XND2Nqa1zrNf96slXOLpR6gb4DWWKellE/DmwPCJewAyqu5umTWBn5iXAJZNYfg0de6DN8Y/VETGbMnPQ+a0msyjH9ibaPpF1+jbQPmGTrXmMsYx137SqeaRxnoMLm38fjojzKVPT140xhs3xHExHNY0VgMxcAbx++HZEfBK4jMlvjzb/JjaXjXndba6//a7KzDd2/P71iPgIsMcoY5pRdbdl2k+Jb0hEvCAi3tzRNAcYbKYEVwLzO+7bifLOakLtEbE18CTKu8HJ9tVNYz3mjKo5Io6IiBd3NM2hnEAy1c/BdFTTWAGIiD+IiIM7moa379xmOwyb1DZk014Pm8tdTG2dY73uuyYito6Iv4mIWR3N4/1dV193m6oNbGAtcGZzchLAu4Arm9+XA8cARMSelI30/ab9dc0JVHOBI4HlmbkSuLPjP5CjgWsy87FmnbdGxOzmGNPLKVN51wJ/FBHPbvbqj2qW7abvAgubmmhqHH7MmVbzrsCHI6KvOTfhrcAV0+A5mI6m4rW4qbYAzm62yWzgHZTtew1lO9Bsl9WZeQelniURMbd5PbwWuJoWXw/dLnhYc/b+lNU5zuu+m3U/RPk7fl0z1ldRXgc3M4PrbtO0mRKfrMz8ryin7l8dEXOAWykvBignOpwfEbdSpjuOzMxHgZsj4gzKse8tgCsyc1mzzhHNOp+gvAM+smk/BwjKxwlmASdm5p0AEbGU8uLYknI25DldrnltRBwOXBARW1GO3b9phtZ8OuXY9g8pr9PPZeZ1zRim9DmYbjLzzs39WtxUmfnPEfEZ4DuU7Xsd8DHK8cQvRsSxlI97LWmW/1pE7AHc1Cx/VmbeDK2/HjaXpUxtnRt63XfbYcC5zVT4g8DrmzcwM73uVswaGho5pS9JkqabmqfEJUnqGQa2JEkVMLAlSaqAgS1JUgUMbEmSKmBgS5JUAQNbkqQK/H9tzNQPdza0ygAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.subplots(figsize=(8,4))\n",
    "plt.hist(y_test_transformed - predictions_dict['lr_pred3'], bins=20)\n",
    "plt.title('Histogram of residuals')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ideally a residual histogram would follow a normal distribution, and that would tell us that we've chosen an appropriate model type. However, the above plot does not closely follow a normal distribution, which is another indication that a linear regression model is not best for my dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Accuracy Score Table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is an accuracy score table of all the models tested. We can see that Random Forest performed the best. \n",
    "The train & test scores are $R^2$ values and the train & test errors are the RMSE values. I chose RMSE instead of MSE since RMSE gives us the price value, while MSE gives us the price squared."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>model</th>\n",
       "      <th>test_score</th>\n",
       "      <th>train_score</th>\n",
       "      <th>test_error</th>\n",
       "      <th>train_error</th>\n",
       "      <th>price outs removed</th>\n",
       "      <th>description</th>\n",
       "      <th>features</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>random forest</td>\n",
       "      <td>0.651</td>\n",
       "      <td>0.944</td>\n",
       "      <td>217673.139</td>\n",
       "      <td>88051.867</td>\n",
       "      <td>yes</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except bedrooms, date, price, street, city...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>random forest</td>\n",
       "      <td>0.642</td>\n",
       "      <td>0.946</td>\n",
       "      <td>222715.763</td>\n",
       "      <td>85393.498</td>\n",
       "      <td>yes</td>\n",
       "      <td>parabolic transformation</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>random forest</td>\n",
       "      <td>0.635</td>\n",
       "      <td>0.947</td>\n",
       "      <td>224897.897</td>\n",
       "      <td>84269.506</td>\n",
       "      <td>yes</td>\n",
       "      <td>zero prices removed</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>linear regression</td>\n",
       "      <td>0.595</td>\n",
       "      <td>0.572</td>\n",
       "      <td>236687.279</td>\n",
       "      <td>239639.587</td>\n",
       "      <td>yes</td>\n",
       "      <td>zero prices removed</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>linear regression</td>\n",
       "      <td>0.595</td>\n",
       "      <td>0.572</td>\n",
       "      <td>236648.326</td>\n",
       "      <td>239620.430</td>\n",
       "      <td>yes</td>\n",
       "      <td>parabolic transformation</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>ridge regression</td>\n",
       "      <td>0.595</td>\n",
       "      <td>0.572</td>\n",
       "      <td>236687.280</td>\n",
       "      <td>239639.587</td>\n",
       "      <td>yes</td>\n",
       "      <td>zero prices removed</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>ridge regression</td>\n",
       "      <td>0.595</td>\n",
       "      <td>0.572</td>\n",
       "      <td>236648.328</td>\n",
       "      <td>239620.430</td>\n",
       "      <td>yes</td>\n",
       "      <td>parabolic transformation</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>linear regression</td>\n",
       "      <td>0.565</td>\n",
       "      <td>0.577</td>\n",
       "      <td>242953.056</td>\n",
       "      <td>241694.255</td>\n",
       "      <td>yes</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>ridge regression</td>\n",
       "      <td>0.565</td>\n",
       "      <td>0.577</td>\n",
       "      <td>242953.051</td>\n",
       "      <td>241694.255</td>\n",
       "      <td>yes</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>random forest</td>\n",
       "      <td>0.079</td>\n",
       "      <td>0.952</td>\n",
       "      <td>815920.713</td>\n",
       "      <td>83409.410</td>\n",
       "      <td>no</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>linear regression</td>\n",
       "      <td>0.068</td>\n",
       "      <td>0.594</td>\n",
       "      <td>820661.051</td>\n",
       "      <td>241949.171</td>\n",
       "      <td>no</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>ridge regression</td>\n",
       "      <td>0.068</td>\n",
       "      <td>0.594</td>\n",
       "      <td>820661.054</td>\n",
       "      <td>241949.171</td>\n",
       "      <td>no</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except bedrooms, date, price, street, city...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                model  test_score  train_score  test_error  train_error  \\\n",
       "10      random forest       0.651        0.944  217673.139    88051.867   \n",
       "12      random forest       0.642        0.946  222715.763    85393.498   \n",
       "11      random forest       0.635        0.947  224897.897    84269.506   \n",
       "3   linear regression       0.595        0.572  236687.279   239639.587   \n",
       "4   linear regression       0.595        0.572  236648.326   239620.430   \n",
       "7    ridge regression       0.595        0.572  236687.280   239639.587   \n",
       "8    ridge regression       0.595        0.572  236648.328   239620.430   \n",
       "2   linear regression       0.565        0.577  242953.056   241694.255   \n",
       "6    ridge regression       0.565        0.577  242953.051   241694.255   \n",
       "9       random forest       0.079        0.952  815920.713    83409.410   \n",
       "1   linear regression       0.068        0.594  820661.051   241949.171   \n",
       "5    ridge regression       0.068        0.594  820661.054   241949.171   \n",
       "\n",
       "   price outs removed               description  \\\n",
       "10                yes        no transformations   \n",
       "12                yes  parabolic transformation   \n",
       "11                yes       zero prices removed   \n",
       "3                 yes       zero prices removed   \n",
       "4                 yes  parabolic transformation   \n",
       "7                 yes       zero prices removed   \n",
       "8                 yes  parabolic transformation   \n",
       "2                 yes        no transformations   \n",
       "6                 yes        no transformations   \n",
       "9                  no        no transformations   \n",
       "1                  no        no transformations   \n",
       "5                  no        no transformations   \n",
       "\n",
       "                                             features  \n",
       "10  all except bedrooms, date, price, street, city...  \n",
       "12        all except date, price, street, city, state  \n",
       "11        all except date, price, street, city, state  \n",
       "3         all except date, price, street, city, state  \n",
       "4         all except date, price, street, city, state  \n",
       "7         all except date, price, street, city, state  \n",
       "8         all except date, price, street, city, state  \n",
       "2         all except date, price, street, city, state  \n",
       "6         all except date, price, street, city, state  \n",
       "9         all except date, price, street, city, state  \n",
       "1         all except date, price, street, city, state  \n",
       "5   all except bedrooms, date, price, street, city...  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table.sort_values('test_score', ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Model description:\n",
    "* Models described as having no transformations are those which did not have their `bedrooms` feature undergo parabolic transformation.\n",
    "* Models with zero prices removed are those with zero values in the `price` feature and price outliers removed.\n",
    "* Models with parabolic transformation are based on data with price outliers and zero prices removed, and have their `bedrooms` feature transformed.\n",
    "\n",
    "From the table above we can see that the best model is the Random Forest with 66% accuracy. That model only has the price outliers removed but not the zero prices. There were also no transformations on the features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Analysing the coefficients**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's take a look at the coefficients of the best performing ridge regression model that has its `bedrooms` feature transformed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>bedrooms</th>\n",
       "      <td>-73983.937888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bathrooms</th>\n",
       "      <td>41692.401968</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sqft_living</th>\n",
       "      <td>180.235738</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sqft_lot</th>\n",
       "      <td>-4.171500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>floors</th>\n",
       "      <td>62571.711411</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>waterfront</th>\n",
       "      <td>468650.386551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>view</th>\n",
       "      <td>54503.817213</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>condition</th>\n",
       "      <td>29156.616047</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sqft_above</th>\n",
       "      <td>102.644951</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sqft_basement</th>\n",
       "      <td>77.368144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>yr_built</th>\n",
       "      <td>-2684.743900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>yr_renovated</th>\n",
       "      <td>12.473432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>zipcode</th>\n",
       "      <td>-140.794058</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total_sqft</th>\n",
       "      <td>3.477844</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>month</th>\n",
       "      <td>-2095.817430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>price_is_zero</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>renov_date_is_zero</th>\n",
       "      <td>12714.779384</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bedrooms_squared</th>\n",
       "      <td>2154.343263</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             Coef\n",
       "bedrooms            -73983.937888\n",
       "bathrooms            41692.401968\n",
       "sqft_living            180.235738\n",
       "sqft_lot                -4.171500\n",
       "floors               62571.711411\n",
       "waterfront          468650.386551\n",
       "view                 54503.817213\n",
       "condition            29156.616047\n",
       "sqft_above             102.644951\n",
       "sqft_basement           77.368144\n",
       "yr_built             -2684.743900\n",
       "yr_renovated            12.473432\n",
       "zipcode               -140.794058\n",
       "total_sqft               3.477844\n",
       "month                -2095.817430\n",
       "price_is_zero            0.000000\n",
       "renov_date_is_zero   12714.779384\n",
       "bedrooms_squared      2154.343263"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coefs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The coefficient of a term represents the change in the mean response for one unit of change in that term. If the coefficient is positive, as the term increases, the mean value of the response increases. If the coefficient is negative, as the term increases, the mean value of the response decreases. So for example we see than the coefficient of `sqft_living` is 180. This means that for every 1 sqft increase on a house, the price would increase by \\\\$180.\n",
    "\n",
    "I was surprised to see `bedrooms` have a negative coeficient, since my assumption was that when the number of bedrooms increase, so does the price. However, we saw previously with a `bedrooms` and `price` scatterplot that price increases with number of bedrooms only for homes with up to 5 bedrooms. The price then starts to decrease for homes with 6 bedrooms or more.  \n",
    "Notice also `bedrooms_squared` has a positive coeficient. This feature is `bedrooms` with a parabolic transformation. Since this feature has adjusted for `bedrooms` shape, it makes sense that it is positive.\n",
    "\n",
    "Not only is the coefficient of `sqft_lot` negative, but it's also a small number, indicating that its affect on  price is miniscule."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**In-depth Analysis**\n",
    "\n",
    "The following analysis is based on my 95% confidence that the error in the home price predictions is between -429250 and -429250:\n",
    "\n",
    "If I've predicted a house to be $\\$$500,000 and someone were to buy the house at that price, the worst loss would be \\\\$429250 and the highest gain would be \\\\$429250. That is, the least the house would be worth is \\\\$70,750 (500,000-429250), and the most the house would be worth is \\\\$929,250 (500,000+429250).\n",
    "\n",
    "Let's say my boss wants to buy a house at below my estimate in the hopes of flipping it, i.e., selling the house at around double the price at which he acquired it. With a house that I have estimated to be \\\\$500,000, if he purchases it at \\\\$300,000 for example, 3 things can happen:\n",
    "\n",
    "1. My estimation is correct and he would make a profit of \\\\$200,000 by selling it for \\\\$500,000.  \n",
    "2. The house is actually worth \\\\$70,750 and he ends up buying at a higher price than what the house is worth, losing \\\\$229,250 on the purchase, and resulting in a 24\\% loss.\n",
    "3. The house is actually worth \\\\$929,250 and he ends up buying \\\\$629,250 below the house value, resulting in a 309\\%  profit if he were to sell it at \\\\$929,250.\n",
    "\n",
    "**What happens if my boss purchases 100 homes at my predicted price?**\n",
    "\n",
    "If my boss was to buy 100 homes at my predicted price and sell them at their true price, I can be 95\\% confident that the most she would lose is \\\\$4,358,500 and the most she would gain is \\\\$4,979,500.\n",
    "\n",
    "**What happens if my boss purchases 100 homes \\\\$1000 below my predicted prices?**\n",
    "\n",
    "If my boss was to buy 100 homes at \\\\$1000 less than my predicted price and sell them at their true price, I can be 95% confident that the most she would lose is \\\\$4,309,300 and the most she would gain is \\\\$4,734,600.\n",
    "\n",
    "So if she purchases 100 homes at $1000 below my predicted price, the most she would lose decreases by \\\\$49,200.\n",
    "This is clearly a better option, and since finding homes that are selling \\\\$1000 below their true value is practical, I would recommend this plan to my boss."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is a histogram of the distribution of purchasing 100 homes at my predicted prices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-20872 25984\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD/CAYAAADytG0IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAYQElEQVR4nO3dfZBddZ3n8XceoDtNeJAxtUJoFMH6VvFggomjBTMSKWNmd5b4MMwgA8waSC1UKJ2aEbPCOmytyGINO4pYwFhElBGhVHbJMPEJmKlxcV11BuiMD/BdcZy16YBQIyBwgWAn+8c5DU3MTXffh+6T/N6vf5J77nn49O+e+72/+zvnnjNv586dSJL2ffPnOoAkaXZY8CWpEBZ8SSqEBV+SCmHBl6RCWPAlqRALpzNTROwPfBW4KjO3TJr+amAE+O3M/EE97X3AhcALwJcy87JOgt1zzz2/ovpA+mUny0tSoQ4CdqxYseLX6vuUBT8ilgOfBk4Arpo0fSHwOWC/XeZ9P7ASeB7424j4Tmbe2UHo+cA84OAFCxZ0sHj/jY+PA9DEfE3OBs3O1+Rs0Ox8Tc4Gzc7Xq2z1enY7ejOdHv4FwKXAxbtMvwzYDBw1adppwObMfBIgIr4AnAl0UvB/CRy8ePFiIqKDxfsvMwEama/J2aDZ+ZqcDZqdr8nZoNn5epVtZGSE8fHx3Y6MTFnwM/OCOsSLBT8iVgPHA5cAfzJp9qXAA5MebwOO6CDzi1qt1osN0TStVgugkfmanA2ana/J2aDZ+ZqcDZqdr1fZJr4p7M6MD9pGxL8BrgTOzcxdr8swH5g8bR7QfuuSpFkzrYO2uziN6qDAnfVXj8OBWyPiXGC0fjzhMGCsm4BDQ0ON/PoFZXw97Jcm52tyNmh2viZng2bn6/GQzm6fm3HBz8xNwKaJxxHxL8DpmfmDiHgW+FJEfAx4FjiL6tuAJGmO9fQ8/My8D/g48C3g+8A3M3NzL7chSerMtHv4mbmqzfTX7PL4OuC6rlJJknrOX9pKUiE6OWgr7dOGh4fnOoLUFxZ8FeuJZ59i69jMz3k++ajlPL392Y6WBVi2NDhk0YEdLSt1w4KvYm0dS1Zfu37Gyz16+d0dLwtw54ZNnHLMyo6WlbrhGL4kFcKCL0mFsOBLUiEs+JJUCAu+JBXCgi9JhbDgS1IhLPiSVAgLviQVwoIvSYWw4EtSISz4klQIC74kFcKCL0mFsOBLUiEs+JJUCAu+NMuOfmV3t1BcsmRJj5KoNN7xSppliweGOr694rKlweDgYB9SqQTTLvgRsT/wVeCqzNwSEW8ErgYWATuBD2XmN+p5LwdOB54DPpWZm3qeXNqLdXqLxDs3bOKNhx/bh0QqwbSGdCJiOXA3cNKkyV8E/nNmLgfOAW6JiEUR8U5gFXBC/e/GiIhehpYkzdx0e/gXAJcCF8OLvf3LM/Pv6ufvBxYAhwKnATdn5nZge0RsBs4APtJJwFarRebMv/rOhlarBdDIfE3OBnOfb3i4u3H0uTQ+Pt7Y98Vcv65TaXK+XmUbHx9v+9y0eviZecHEcE39eHtmfmbSLJcCD2TmGLAUGJv03DbgiBklliT1XFcHbSNiHvBR4A+At9aT51ON6U+YB7T/yJnC0NAQTR0RmvgkbmK+JmeD5udrsgULFjT2fdH017XJ+XqVbWRkpG0vv+OCHxEDwE1UPfqTMvOx+qlR4PBJsx7Gy3v8kqQ50E0P/wZgP+Ctmfn8pOlbgIsi4rNUZ/C8i+obgCRpDnVU8CPi9cAfAgl8d9JXkDMy87aIOBG4t17/JzPzvl6ElSR1bkYFPzNXTXo4bw/zXUp1IFeS1BBeWkGSCmHBl6RCWPAlqRAWfEkqhAVfkgphwZekQljwJakQFnxJKoQFX5IKYcGXpEJY8CWpEBZ8SSqEBV+SCtHVHa+kufbEs0+xdWzm9wA9+ajlfUgjNZsFX3u1rWPJ6mvXz3i5Ry+/uw9ppGZzSEeSCmHBl6RCWPAlqRAWfEkqhAVfkgphwZekQljwJakQ0z4PPyL2B74KXJWZWyJiKXATsAR4HFiXmQ/W814OnA48B3wqMzf1PLkkaUam1cOPiOXA3cBJkyZfA9yamccDVwI31vO+E1gFnFD/uzEioneRJUmdmG4P/wLgUuBigIjYD1gDnA2QmbdHxPURcSRwGnBzZm4HtkfEZuAM4COdBGy1WmTO/Kfzs6HVagE0Ml+Ts0Fv8g0PD/cqzl5lfHy8se+LEva7fulVtvHx8bbPTavgZ+YFABFxcT3pN4DnM/PpSbM9DBwBLAX+ZtL0bcCxM8irgixZsoTBwcGOlh0YGOhxGmnf1um1dOYDO3eZNg8Y381zE9M7MjQ0RFNHhCY+iZuYr8nZ4KV8hx56qBdAm6EFCxY09n2xt+x3TczXq2wjIyNte/mdFvxHgcGIOCAzn6mnHQaMAaPA4ZPmnZgu7ZYXQJNmR0enZWbmr4A7gPMAImItsC0zHwK2AGdHxGBEvAJ4F/CVHuWVJHWom/PwNwC/GxE/BD7MSwdwbwP+FrgX+C7wycy8r9ugkqTuzGhIJzNXTfr/GNWZOrub71Kqs3okSQ3hL20lqRAWfEkqhAVfkgphwZekQljwJakQFnxJKoQFX5IKYcGXpEJY8CWpEBZ8SSqEBV+SCmHBl6RCWPAlqRAWfEkqhAVfkgphwZekQljwJakQFnxpL3L0K4cZGBhgeHh4rqNoLzSjWxxKmluLB4Z4anuLrdtyxssuWxocsujAPqTS3sKCL+1lto4lq69dP+Pl7tywiVOOWdmHRNpbOKQjSYXoqocfEWcCl9QPHwDOBQ4CbgKWAI8D6zLzwW62I0nqXsc9/Ih4BXAt8LbMPAF4EvggcA1wa2YeD1wJ3NiLoJKk7nQzpLMQ2A84OCIWAEPAC8Aa6iKfmbcDx0TEkd0GlSR1p+Mhncx8LCI+DHyfqnf/c6oe/gcy8+lJsz4MHAH8rJPttFotMmd+RsJsaLVaAI3M1+Rs8FK+iX81O1qtFqOjo31dPzR/v2tivl5lGx8fb/tcxwU/IpYB5wOvBR4BrgI+C+zcZdZ5QPsE2qstWbKEwcHBGS83sVMODAz0OpKkNro5aPt24JuZOQYQEZ8GvgEMRsQBmflMPd9hwFinGxkaGiIiuojZPxOfxE3MN5vZnnj2KbaOddYrOfmo5T1Ooz3p9/upye8JaHa+XmUbGRlp28vvpuDfB5wfEYdm5i+AtcC3gf2B84CrI2ItsC0zH+piO2q4Ts8LB3j08rt7nEZSO92M4d8VEdcC34mI54CfABfU67whIs4HngHO7klSSVJXujoPPzM/Dnx8N0+t6Wa9kqTe85e2klQIC74kFcKCL0mFsOBLUiEs+JJUCAu+JBXCgi9JhbDgS1IhLPiSVAgLviQVwoIvSYWw4EtSISz4klQIC74kFcKCL0mFsOBLUiEs+JJUCAu+VIijXzk81xE0x7q6xaGkvcfigSGeePYpto7ljJddtjQ4ZNGBfUil2WTBlwqydSxZfe36GS9354ZNnHLMyj4k0mxySEeSCmHBl6RCdDWkExFrgI8Ci4CfAmcBBwI3AUuAx4F1mflglzklSV3quIcfEa+hKuxnZ+bxwChwMXANcGs97Urgxh7klCR1qZse/ruB2zJz4pD/JcDBwAPA2QCZeXtEXB8RR2bmzzrZSKvV4qVNNEur1QJoZL7ZyjY87Kl+pWi1WoyOjk45DzTzPQHNzterbOPj422f66bgHwO8EBFfBl4H3EfVw38+M5+eNN/DwBFARwVfktQb3RT8/YC3A78FjAGfAK4Adu4y3zyg/UfOFIaGhoiIThfvq4lP4ibma3I27Z2m815s+n7X5Hy9yjYyMtK2l9/NWToPA9/MzNHM3AHcDPwmMBgRB0ya7zCqDwRJ0hzqpuDfDpwaEUvrx+8A7gHuAM4DiIi1wLbMfKirlJKkrnU8pJOZ/xgRHwS+HhELgQeBdcAAcENEnA88Q30AV5I0t7o6Dz8zbwFu2c1Ta7pZrySp9/ylrSQVwoIvSYWw4EtSISz4klQIC74kFcKCL0mFsOBLUiEs+JJUCAu+JBXCgi9JhbDgS1IhLPiSVAgLviQVwoIvSYWw4EtSISz4klQIC74kFcKCL0mFsOBLUiEs+JJUCAu+JBXCgi9JhVjYi5VExB8D52Tmyog4EPg88DqgBVyYmd/rxXYkSZ3ruocfEScCGydNugz4cWYeB5wP3BIRPflgkSR1rqtCHBGLgU8DlwDvqyefBqwFyMx7I+JfgZOA/9XJNlqtFpnZTcy+abVaAI3MN1vZhoeH+7p+NUer1WJ0dHTKeaCZ7wlodr5eZRsfH2/7XLc97+uAvwB+PmnaUmBs0uNtwBFdbkd9tGTJEgYHBztadmBgoMdpJPVLxwU/It4LbM/ML0bEqklPzQd2Tno8D2j/kTOFoaEhIqLTxftq4pO4iflmmu2JZ59i69jMexYnH7V8xsto7zSd92KT3xPQ7Hy9yjYyMtK2l99ND/8s4PCIGAEW1/+/CxgFDgeerOc7jJf3+NVAW8eS1deun/Fyj15+dx/SSOqHjg/aZubqzDwuM5cD64EfZebbgC31YyLiDVTF/x96EVaS1Ll+nD1zKXB9RPyQaijnrMx8vg/bkSTNQE8Kfmb+PbCy/v+TwB/0Yr2SpN7xl7aSVAgLviQVwoIvSYWw4EtSISz4klQIC74kFcKCL0mFsOBLUiG8Tr2kvuv04nwAy5YGhyw6sMeJymTBl9R3nV6cD+DODZs45ZiVPU5UJod0JKkQFnxJKoQFX5IKYcGXpEJ40FbSlI5+5fRuVu9N7ZvNgi9pSosHhrzv8T7Agi9pWrzv8d7PMXxJKoQFX5IKYcGXpEJY8CWpEF0dtI2Ii4B1wA7gn4H1wP7ATcAS4HFgXWY+2GVOSVKXOu7hR8RbqIr9mzLzBOBHwBXANcCtmXk8cCVwYy+CamaGh4c9J1rSy3TTw38M2JCZT9eP7wU2AG8GzgbIzNsj4vqIODIzf9ZdVE3F86Ql7UnHBT8z7wfuB4iIg4A/A24BTpz0IQDwMHAE0FHBb7VaZHZ2He1+a7VaAI3INzw8zNZtnietfVOr1WJ0dLQn64FmvGd31ats4+PjbZ/r+qBtRBwO/B3wbarhm527zDIPaJ9AkjQruj1o+3pgC3B9Zl4WEQuBwYg4IDOfqWc7DBjrdBtDQ0NERDcx+2bik7ip+aR9Ra/qQJPfs73KNjIy0raX381B21cBdwEfyszLADLzV8AdwHn1PGuBbZn5UKfbkST1Rjc9/I3AYmBjRGyspyXVgdsbIuJ84BnqA7iSpLnVzUHbPwX+tM3TazpdrySpP/ylrSQVwoIvSYXwevgN44+nJPWLBb9hvMmEpH5xSEdSo033frqamj18SY3Wzf10ly0NDll0YB9S7Z0s+JIar9Ohzjs3bOKUY1b2IdHeySEdSSqEBV+SCmHBl6RCOIYvaZ+16xk+pd8FzoIvaZ/lGT4vZ8GXtE/zDJ+XOIYvSYWw4EtSISz4klQIx/D7wCteSmoiC34X2p3i5RUvJTWRBX83Ou2hg710aV+xL16l04K/G5320MFeurSv2BfP4bfgS1Ib+9o5/BZ8SeqxToaDZuOyD30r+BHx+8B/BcaBvwf+JDN/1a/tSVJTdDMcBP0bEupLwY+IVwFXAyuBh4EvAuuA6/uxPUlqmm6OBfZrSGjezp07e77SiDgHWJuZv18//nfARZl56nTXcc899+wA5nWy/fnz5zNvXkeLArBz507Gd4x3tOyC+QsAOlq+tGXnctv+zXvHsnO57Sb8zTt27OhoeWDnihUrfu2Htf0a0lkKjE16vA04Yobr2EH1S+BfznTjXTTSi+Z19lnz4rY7Wb60Zedy2/7Ne8eyc7ntJvzNHTqIqn7+mn4V/PnA5K8O86jG8qdtxYoVHlCWpB7q17V0RoHDJz0+jJf3+CVJs6xfveg7gT+PiFdTFf91wJY+bUuSNA196eFn5iPABqoi/wDVOPw1/diWJGl6+nKWjiSpebweviQVwoIvSYWw4EtSISz4klQIC74kFcKCL0mFsOBLUiHm5Ho1EfHnwO9SXWPny5n5X+rpq4BPUn0Q/QBYn5nPRMSBwOeB1wEt4MLM/F69zPuAC4EXgC9l5mX19OOBzwBDwEPAf8jMR6eRbSNwZr3c5sz8T03JtkvO/wH8NDMv2tM6I2I/4DrgZOB54MOZuaVeZrf3LIiIpcBNwBLgcWBdZj44RZ5B4NPAirqNvkZ1hdSdTWu7qczWvRwi4iKqX6HvAP4ZWA/sT5u2j4jLgdOB54BPZeamevoqZti+M8z5x8A5mbmyl6/ZnvbNaWRaA3wUWAT8FDgLOLApbRcRZwKX1A8fAM6luqjZnOab9R5+RPx74C3AMuANwOkRcWpEDFA1xh9l5glU19GfaLDLgB9n5nHA+cAtEbEwIpYD7wfeBLwR+J2IWF0vcxPwkXpdXwM+MY1sZwLvAX67zrc6ItY0IdsuOS8ETtllcrt1Xki1ox0LvAO4LiJeMemeBavrv/VVVMUHql9F35qZxwNXAjdOI9ZGqgvmnQAspyr8ZzWt7aYyRbv0cjtvqdf7pvpv+RFwBW3aPiLeCayiat9VwMaozLh9Z5jzRKrXdkIvX7Pd7pvTyPSaep1n1+00ClxMQ9qu/huuBd5Wr/dJ4INNyDcXQzr7UX3SDwIDVN8yngN+E3g4M7fW822i6mkDnAZ8DiAz7wX+FTipnr45M5/MzOeALwBnRsQwsDQzv1Iv/xng9+oexZ6cCfz3zHy6Xt9a4B8akg2AiFgGnEHVm56Ytqd1ngbcmJk7M/P/Af+nnrYa+FZmjmXmDuCzdb79gDXUO2Nm3g4cExFHThHtfwOX19vZDvwQeHWT2m6adtsuPVz/hMeADZn5dP34XuBo2rf9acDNmbk9Mx8HNlPtB52077RExGKq/eySSZN7+Zq12zen8m7gtsycuJ3UJcBf0py2W0hV5w6OiAVU9e6FJuTr5y0O/5DqU25XRwMPUn21W0g1pPPtiHgP7a+h3+76+kupvi7tbvqL89dfgZ6j+iq1bQ/ZtgFHRsQ3qHp2t2bmZfUQx6xkgz223XFUL/p7gHOoPjB/LcMu62yXb0eb6b8BPD+pEEHVszgC+Nkesr02M39R5z+BasdcRdV7m7W264Fe3MthSpl5P3A/QEQcBPwZcAtwYpu2Xwr8zS65jp0ib7d/y3XAXwA/nzStl69Zp/mOAV6IiC9TDWncR9XDb7ffzmrbZeZjEfFh4PtUvfufU/XwPzDX+fpW8DPzZuDmXadHxH+st/sqql7+VyPivcB22l9Dv9319ac7/WXr20O2HwO/Bbyrzvj1iPiXerlZyTZFvs8BV2fmTyJi8lN7Wme7HAs6ydcu26SMJwO3Au/PzH+qx29nre16oOt7OcxERBwO3A58m6r3t3GXWWbaJtNp3+nkei+wPTO/WI8jT+jla9Zpvv2At1O9V8eohoiu6GA7/Wq7ZVTDLK8FHgGuovqmOOf55mJI5x1UX1+ey8wnqIrHW9nzNfTbPTet6RFxANVO8ospsj0M/M/MbGXmL4G/Bt7chGz1QZo1wAciYgS4APijiPjYFOucab5HgcF6Hbv7e/eU8Uyqr6PnZuZN9eQ5b7sZmrV7OUTE64HvAH+dmRew57afaVsxxXNTOQt4c72vbQKOjYi7OsjRyb45lYeBb2bmaD3sdjPV8EdT2u7tdb6xzBynGhY7rgn55qLg30d1oHZ+PY73b4HvAd8FjoiIN9Tzreela+hvqR9TP3841dj6FuDd9UHIQaqddEtmjgKPRMTaevnzgDsy84Upsm2mHseOiP2B3wHuaUK2zHwqMw/LzOWZuZxqzPKvMvNDU6xzC3Bu3d5HUh2QvovqngVviYhXR8R86nsWZHU2yh31OqjXuS0zH9pTvqjOmvgEcGpmfm3SU3PedjO023bp4fqBFw8O3wV8KOuzWaZo+y3A2RExWB8UfBfwFTpr3yll5urMPK7e19YDP8rMt+1hnZ28Zu32zancDpxaD7VC1Ym8pyltR1Xj3hYRh9aP11J9g5vzfHNxWuZ/Az5FNd63Hfg68JeZOR4RZwCbImIR8H+pxqkBLgWuj4gfUn1tOSsznwfui4iPA9+iOp3ty5m5uV7mzHqZj1H1nM6aRrarqcYW/4lqyGML8LnM3NGAbFNpt85rgKA6pWse1VDLIwARMXHPggHgbl66Z8EG4IaIOB94Bjh7Gtu/gqoD8flJw01fyMwr94K2e1FmPrKHdumljcBiqjMyJoZxkjZtn5m3RXXGzL1U79tPZuZ9AB20bzd6+Zq13Tf3JDP/MSI+SDXkupDqmOA6qtdrztsuM++KiGuB79THK35C9Y184Vzn83r4klQIf2krSYWw4EtSISz4klQIC74kFcKCL0mFsOBLUiEs+JJUCAu+JBXi/wMklLxtnqJs2QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(error_means, bins=20)\n",
    "plt.plot()\n",
    "print(min_error, max_error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It shows that the mode is shifted to the positive, which is good because it means we're profiting. I conducted this experiment a few more times to make sure the shape is real."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other experiments:\n",
    "* experiments with other models\n",
    "* experiemnts with predicted prices reduced by \\\\$5,000, \\\\$10,000, \\\\$50,000\n",
    "\n",
    "When running experiments using my other models, the Random Forest model resulted in the most profit, which was expected since the model had the least error in its prediction.\n",
    "\n",
    "I ran experiments with \\\\$5,000, \\\\$10,000, \\\\$50,000 deducted from the predicted prices. \n",
    "This means that my boss would purchase 100 homes with \\\\$5,000, \\\\$10,000, and \\\\$50,000 below my predicted prices. \n",
    "The result was that these 3 experiments produced better profits and less losses than the scenario where my boss would purchase homes at \\\\$1000 less than my predicted prices. However, finding 100 houses at those reduced prices would be difficult. Therefore, I would reccomend focusing on finding homes with the \\\\$1000 reductions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Limitations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Linear regression: assumes a straight line relationship between the features and target variable. My data did not have a linear relationship. House prices do increase as square footage and number of bedrooms increase, but once the square footage reaches a certain point, and once the houses get up to 6 bedrooms, the house prices start to decrease. Also, there aren't many houses with over 6 bedrooms, so the decrease in average price for houses with over 6 bedrooms could be due to lack of sufficient data.\n",
    "\n",
    "Ridge regression: same as linear regression in that it assumes a linear relationship between the feature and target variables. \n",
    "\n",
    "Random forest regression:\n",
    "Random Forest can’t extrapolate. It can only make a prediction that is an average of previously observed labels. Therefore, the range of predictions a Random Forest can make is bound by the highest and lowest labels in the training data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Out of all the models, Random Forest yielded the best prediction accuracy of around 65%. Linear Regression had the poorest performance. This is due to the data having many features that are not linearly correlated to the target variable `price`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next Steps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To improve the prediction accuracy of my model I will do the following:\n",
    "* Scrape Washington state data such as city population, salary, and schools to gain more insight\n",
    "* Transform more features when modeling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import necessary modules/libraries\n",
    "\n",
    "import numpy as  np\n",
    "import pandas as pd\n",
    "\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "from scipy import stats\n",
    "from scipy.stats import pearsonr\n",
    "\n",
    "from matplotlib import rcParams\n",
    "sns.set_style(\"whitegrid\")\n",
    "sns.set_context(\"talk\", font_scale=0.7)\n",
    "sns.set_palette(\"Greens_r\")\n",
    "#set_palette(\"Set1\", 8, .75) # makes plot lines red\n",
    "\n",
    "# Ignore  the warnings\n",
    "import warnings\n",
    "warnings.filterwarnings('always')\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "from sklearn.preprocessing import Imputer # to impute missing data\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "import sklearn\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "from sklearn.linear_model import Lasso\n",
    "from sklearn.linear_model import Ridge\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.model_selection import cross_validate\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "# read the data\n",
    "df1 = pd.read_csv('house_price_prediction.csv')\n",
    "df2 = df1.copy()\n",
    "\n",
    "# change date col to datetime\n",
    "df2['date'] = pd.to_datetime(df2['date'])\n",
    "\n",
    "# change data types of 'waterfront' & 'condition' to categorical\n",
    "df2[['waterfront', 'condition']] = df1[['waterfront', 'condition']].astype('category')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Engineering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# splittig statezip into state & zipcode features & removing statezip\n",
    "df2['state'] = df2['statezip'].apply(lambda x: x.split(' ')[0])\n",
    "df2['zipcode'] = df2['statezip'].apply(lambda x: int(x.split(' ')[1]))\n",
    "df2 = df2.drop('statezip', axis='columns')\n",
    "\n",
    "# creating total_sqft feature: sqft_above + sqft_lot\n",
    "df2['total_sqft'] = df2.sqft_living + df2.sqft_lot\n",
    "\n",
    "# creating month feature\n",
    "df2['month'] = pd.DatetimeIndex(df2['date']).month\n",
    "\n",
    "df3 = df2.copy()\n",
    "\n",
    "# creating a column of boolean arrays with 1s for when price is $0 & 0s for when price is not $0\n",
    "df3['price_is_zero'] = (df3['price'] == 0).astype(int)\n",
    "\n",
    "# creating a column of boolean arrays with 1s for when yr_renov is 0 & 0s for others\n",
    "df3['renov_date_is_zero'] = (df3['yr_renovated'] == 0).astype(int)\n",
    "\n",
    "# replacing 0s with the mean bedroom & bathroom values\n",
    "df3[\"bedrooms\"].replace({0: round(df3[\"bedrooms\"].mean(), 0)}, inplace=True)\n",
    "df3[\"bathrooms\"].replace({0: round(df3[\"bathrooms\"].mean(), 0)}, inplace=True)\n",
    "\n",
    "# removing price outliers\n",
    "df_no_outs = df3[df3.price < 10000000].copy()\n",
    "\n",
    "# creating a new dataframe with $0 prices removed to compare later\n",
    "df_no_zeros = df_no_outs[df_no_outs[\"price\"] != 0].copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Statistical Data Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "meanprice_bedrooms = df_no_zeros.groupby('bedrooms')['price'].mean()\n",
    "medianprice_bedrooms = df_no_zeros.groupby('bedrooms')['price'].median()\n",
    "\n",
    "meansqft_bedroom = df_no_zeros.groupby(\"bedrooms\")[\"sqft_living\"].mean()\n",
    "mediansqft_bedroom = df_no_zeros.groupby(\"bedrooms\")[\"sqft_living\"].median()\n",
    "\n",
    "may_price = df_no_zeros[df_no_zeros.month==5]['price']\n",
    "june_price = df_no_zeros[df_no_zeros.month==6]['price']\n",
    "july_price = df_no_zeros[df_no_zeros.month==7]['price']\n",
    "\n",
    "def ecdf(data):\n",
    "    \"\"\"Compute ECDF for a one-dimensional array of measurements.\"\"\"\n",
    "    n = len(data)\n",
    "\n",
    "    x = np.sort(data)\n",
    "\n",
    "    y = np.arange(1, n+1) / n\n",
    "\n",
    "    return x, y\n",
    "\n",
    "# Compute ECDFs\n",
    "x_5, y_5 = ecdf(may_price)\n",
    "x_6, y_6 = ecdf(june_price)\n",
    "x_7, y_7 = ecdf(july_price)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feature Importance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_selection import f_regression\n",
    "from sklearn.feature_selection import SelectKBest\n",
    "\n",
    "def feature_importance(X, y, model='reg'):\n",
    "    score_func = {'reg': f_regression}\n",
    "\n",
    "    # Score each of the features\n",
    "    bestfeatures = SelectKBest(score_func=score_func[model], k='all')\n",
    "    fit = bestfeatures.fit(X, y)\n",
    "\n",
    "    # Organize and return the scores\n",
    "    featureScores = pd.DataFrame([X.columns, fit.scores_]).T\n",
    "    featureScores.columns = ['Feature', 'Score']\n",
    "    return featureScores.sort_values('Score', ascending=False).set_index('Feature') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Modeling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_no_outs = df3[df3.price < 10000000]\n",
    "df_no_zeros = df_no_outs[df_no_outs[\"price\"] != 0]\n",
    "\n",
    "# dataframe with both zero prices and outliers.\n",
    "X = df3.drop(['price', 'date', 'street', 'city', 'state', 'country'], axis='columns')\n",
    "y = df3[\"price\"]\n",
    "# splitting data into training and test data\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state=42)\n",
    "\n",
    "# dataframe without price outliers\n",
    "X_no_outs = df_no_outs.drop(['price', 'date', 'street', 'city', 'state', 'country'], axis='columns')\n",
    "y_no_outs = df_no_outs[\"price\"]\n",
    "X_train_no_outs, X_test_no_outs, y_train_no_outs, y_test_no_outs = train_test_split(X_no_outs, y_no_outs, test_size = 0.3, random_state=42)\n",
    "\n",
    "# dataframe without $0 price values\n",
    "X_no_zeros = df_no_zeros.drop(['price', 'date', 'street', 'city', 'state', 'country'], axis='columns')\n",
    "y_no_zeros = df_no_zeros[\"price\"]\n",
    "X_train_no_zeros, X_test_no_zeros, y_train_no_zeros, y_test_no_zeros = train_test_split(X_no_zeros, y_no_zeros, test_size = 0.3, random_state=42)\n",
    "\n",
    "# dataframe with a parabolic transformation on the `bedrooms` feature\n",
    "X_transformed = df_no_zeros.drop(['price', 'date', 'street', 'city', 'state', 'country'], axis='columns')\n",
    "X_transformed['bedrooms_squared'] = X['bedrooms']**2  # parabolic transformation of bedrooms\n",
    "y_transformed = df_no_zeros['price']\n",
    "X_train_transformed, X_test_transformed, y_train_transformed, y_test_transformed = train_test_split(X_transformed, y_transformed, test_size = 0.3, random_state=42)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 4 training sets and test sets\n",
    "# 3 algorithms\n",
    "\n",
    "train_test_sets = [\n",
    "    [X_train, X_test, y_train, y_test],\n",
    "    [X_train_no_outs, X_test_no_outs, y_train_no_outs, y_test_no_outs],\n",
    "    [X_train_no_zeros, X_test_no_zeros, y_train_no_zeros, y_test_no_zeros],\n",
    "    [X_train_transformed, X_test_transformed, y_train_transformed, y_test_transformed],\n",
    "]\n",
    "\n",
    "train_test_dicts = []\n",
    "for i, sets in enumerate(train_test_sets):\n",
    "    d = dict(zip('X_train X_test y_train y_test'.split(), sets))\n",
    "    train_test_dicts.append(d)\n",
    "\n",
    "algorithms = [LinearRegression, Ridge, RandomForestRegressor]\n",
    "names = ['linear regression', 'ridge regression ', 'random forest    ']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def calc_train_error(X_train, y_train, model):\n",
    "    '''returns in-sample error for already fit model.'''\n",
    "    y_pred = model.predict(X_train)\n",
    "    mse = mean_squared_error(y_train, y_pred)\n",
    "    rmse = np.sqrt(mse)\n",
    "    return rmse\n",
    "    \n",
    "def calc_test_error(X_test, y_test, model):\n",
    "    '''returns out-of-sample error for already fit model.'''\n",
    "    y_pred = model.predict(X_test)\n",
    "    mse = mean_squared_error(y_test, y_pred)\n",
    "    rmse = np.sqrt(mse)\n",
    "    return rmse\n",
    "    \n",
    "def calc_metrics(X_train, y_train, X_test, y_test, model):\n",
    "    '''fits model and returns the RMSE for in-sample error and out-of-sample error, and their accuracy score'''\n",
    "    \n",
    "    model.fit(X_train, y_train)\n",
    "    \n",
    "    train_error = calc_train_error(X_train, y_train, model)\n",
    "    test_error = calc_test_error(X_test, y_test, model)\n",
    "    \n",
    "    train_score = model.score(X_train, y_train)\n",
    "    test_score = model.score(X_test, y_test)\n",
    "    \n",
    "    y_pred = model.predict(X_test)\n",
    "    return train_error, test_error, train_score, test_score, y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr_train_score = []\n",
    "lr_test_score = []\n",
    "lr_train_error = []\n",
    "lr_test_error = []\n",
    "\n",
    "ridge_train_score = []\n",
    "ridge_test_score = []\n",
    "ridge_train_error = []\n",
    "ridge_test_error = []\n",
    "\n",
    "rf_train_score = []\n",
    "rf_test_score = []\n",
    "rf_train_error = []\n",
    "rf_test_error = []\n",
    "\n",
    "predictions_dict = dict()\n",
    "\n",
    "for sets in train_test_dicts:\n",
    "    \n",
    "    # linear regression\n",
    "    \n",
    "    #model = algorithms[0]()\n",
    "    lr = LinearRegression()\n",
    "    train_error, test_error, train_score, test_score, y_pred = calc_metrics(model=lr, **sets)\n",
    "    train_error, test_error, train_score, test_score = round(train_error, 3), round(test_error, 3), round(train_score, 3), round(test_score, 3)\n",
    "    #print('train error: {} | test error: {}'.format(train_score, test_score))\n",
    "    \n",
    "    lr_train_error.append(train_error)\n",
    "    lr_test_error.append(test_error)\n",
    "    lr_train_score.append(train_score)\n",
    "    lr_test_score.append(test_score)\n",
    "    \n",
    "    for i in range(4):\n",
    "        predictions_dict['lr_pred{}'.format(i)] = y_pred\n",
    "    \n",
    "    # ridge regression\n",
    "    \n",
    "    #model = algorithms[1](alpha=0.0001)\n",
    "    ridge = Ridge(alpha=0.0001)\n",
    "    train_error, test_error, train_score, test_score, y_pred = calc_metrics(model=ridge, **sets)\n",
    "    train_error, test_error, train_score, test_score = round(train_error, 3), round(test_error, 3), round(train_score, 3), round(test_score, 3)\n",
    "    #print('train error: {} | test error: {}'.format(train_score, test_score))\n",
    "    \n",
    "    ridge_train_error.append(train_error)\n",
    "    ridge_test_error.append(test_error)\n",
    "    ridge_train_score.append(train_score)\n",
    "    ridge_test_score.append(test_score)\n",
    "    \n",
    "    for i in range(4):\n",
    "        predictions_dict['ridge_pred{}'.format(i)] = y_pred\n",
    "    \n",
    "    # random forest\n",
    "    \n",
    "    #model = algorithms[2](max_depth=None, n_estimators=200)\n",
    "    rf = RandomForestRegressor(max_depth=None, n_estimators=200)\n",
    "    train_error, test_error, train_score, test_score, y_pred = calc_metrics(model=rf, **sets)\n",
    "    train_error, test_error, train_score, test_score = round(train_error, 3), round(test_error, 3), round(train_score, 3), round(test_score, 3)\n",
    "    #print('train error: {} | test error: {}'.format(train_score, test_score))\n",
    "    \n",
    "    rf_train_error.append(train_error)\n",
    "    rf_test_error.append(test_error)\n",
    "    rf_train_score.append(train_score)\n",
    "    rf_test_score.append(test_score)\n",
    "    \n",
    "    for i in range(4):\n",
    "        predictions_dict['rf_pred{}'.format(i)] = y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>model</th>\n",
       "      <th>test_score</th>\n",
       "      <th>train_score</th>\n",
       "      <th>test_error</th>\n",
       "      <th>train_error</th>\n",
       "      <th>price outs removed</th>\n",
       "      <th>description</th>\n",
       "      <th>features</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>random forest</td>\n",
       "      <td>0.651</td>\n",
       "      <td>0.944</td>\n",
       "      <td>217673.139</td>\n",
       "      <td>88051.867</td>\n",
       "      <td>yes</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except bedrooms, date, price, street, city...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>random forest</td>\n",
       "      <td>0.642</td>\n",
       "      <td>0.946</td>\n",
       "      <td>222715.763</td>\n",
       "      <td>85393.498</td>\n",
       "      <td>yes</td>\n",
       "      <td>parabolic transformation</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>random forest</td>\n",
       "      <td>0.635</td>\n",
       "      <td>0.947</td>\n",
       "      <td>224897.897</td>\n",
       "      <td>84269.506</td>\n",
       "      <td>yes</td>\n",
       "      <td>zero prices removed</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>linear regression</td>\n",
       "      <td>0.595</td>\n",
       "      <td>0.572</td>\n",
       "      <td>236687.279</td>\n",
       "      <td>239639.587</td>\n",
       "      <td>yes</td>\n",
       "      <td>zero prices removed</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>linear regression</td>\n",
       "      <td>0.595</td>\n",
       "      <td>0.572</td>\n",
       "      <td>236648.326</td>\n",
       "      <td>239620.430</td>\n",
       "      <td>yes</td>\n",
       "      <td>parabolic transformation</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>ridge regression</td>\n",
       "      <td>0.595</td>\n",
       "      <td>0.572</td>\n",
       "      <td>236687.280</td>\n",
       "      <td>239639.587</td>\n",
       "      <td>yes</td>\n",
       "      <td>zero prices removed</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>ridge regression</td>\n",
       "      <td>0.595</td>\n",
       "      <td>0.572</td>\n",
       "      <td>236648.328</td>\n",
       "      <td>239620.430</td>\n",
       "      <td>yes</td>\n",
       "      <td>parabolic transformation</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>linear regression</td>\n",
       "      <td>0.565</td>\n",
       "      <td>0.577</td>\n",
       "      <td>242953.056</td>\n",
       "      <td>241694.255</td>\n",
       "      <td>yes</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>ridge regression</td>\n",
       "      <td>0.565</td>\n",
       "      <td>0.577</td>\n",
       "      <td>242953.051</td>\n",
       "      <td>241694.255</td>\n",
       "      <td>yes</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>random forest</td>\n",
       "      <td>0.079</td>\n",
       "      <td>0.952</td>\n",
       "      <td>815920.713</td>\n",
       "      <td>83409.410</td>\n",
       "      <td>no</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>linear regression</td>\n",
       "      <td>0.068</td>\n",
       "      <td>0.594</td>\n",
       "      <td>820661.051</td>\n",
       "      <td>241949.171</td>\n",
       "      <td>no</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except date, price, street, city, state</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>ridge regression</td>\n",
       "      <td>0.068</td>\n",
       "      <td>0.594</td>\n",
       "      <td>820661.054</td>\n",
       "      <td>241949.171</td>\n",
       "      <td>no</td>\n",
       "      <td>no transformations</td>\n",
       "      <td>all except bedrooms, date, price, street, city...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                model  test_score  train_score  test_error  train_error  \\\n",
       "10      random forest       0.651        0.944  217673.139    88051.867   \n",
       "12      random forest       0.642        0.946  222715.763    85393.498   \n",
       "11      random forest       0.635        0.947  224897.897    84269.506   \n",
       "3   linear regression       0.595        0.572  236687.279   239639.587   \n",
       "4   linear regression       0.595        0.572  236648.326   239620.430   \n",
       "7    ridge regression       0.595        0.572  236687.280   239639.587   \n",
       "8    ridge regression       0.595        0.572  236648.328   239620.430   \n",
       "2   linear regression       0.565        0.577  242953.056   241694.255   \n",
       "6    ridge regression       0.565        0.577  242953.051   241694.255   \n",
       "9       random forest       0.079        0.952  815920.713    83409.410   \n",
       "1   linear regression       0.068        0.594  820661.051   241949.171   \n",
       "5    ridge regression       0.068        0.594  820661.054   241949.171   \n",
       "\n",
       "   price outs removed               description  \\\n",
       "10                yes        no transformations   \n",
       "12                yes  parabolic transformation   \n",
       "11                yes       zero prices removed   \n",
       "3                 yes       zero prices removed   \n",
       "4                 yes  parabolic transformation   \n",
       "7                 yes       zero prices removed   \n",
       "8                 yes  parabolic transformation   \n",
       "2                 yes        no transformations   \n",
       "6                 yes        no transformations   \n",
       "9                  no        no transformations   \n",
       "1                  no        no transformations   \n",
       "5                  no        no transformations   \n",
       "\n",
       "                                             features  \n",
       "10  all except bedrooms, date, price, street, city...  \n",
       "12        all except date, price, street, city, state  \n",
       "11        all except date, price, street, city, state  \n",
       "3         all except date, price, street, city, state  \n",
       "4         all except date, price, street, city, state  \n",
       "7         all except date, price, street, city, state  \n",
       "8         all except date, price, street, city, state  \n",
       "2         all except date, price, street, city, state  \n",
       "6         all except date, price, street, city, state  \n",
       "9         all except date, price, street, city, state  \n",
       "1         all except date, price, street, city, state  \n",
       "5   all except bedrooms, date, price, street, city...  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "features1 = 'all except date, price, street, city, state'\n",
    "features2 = 'all except date, price, street, city, state'\n",
    "features3 = 'all except date, price, street, city, state'\n",
    "features4 = 'all except date, price, street, city, state'\n",
    "features5 = 'all except bedrooms, date, price, street, city, state'\n",
    "features6 = 'all except date, price, street, city, state'\n",
    "features7 = 'all except date, price, street, city, state'\n",
    "features8 = 'all except date, price, street, city, state'\n",
    "features9 = 'all except date, price, street, city, state'\n",
    "features10 = 'all except bedrooms, date, price, street, city, state'\n",
    "features11 = 'all except date, price, street, city, state'\n",
    "features12 = 'all except date, price, street, city, state'\n",
    "\n",
    "\n",
    "content = {'features': [features1, features2, features3, features4, features5, features6, features7, features8, \n",
    "                        features9, features10, features11, features12], \n",
    "           'model': ['linear regression', 'linear regression', 'linear regression', 'linear regression',\n",
    "                     'ridge regression', 'ridge regression', 'ridge regression', 'ridge regression', \n",
    "                     'random forest', 'random forest', 'random forest', 'random forest'], \n",
    "           'test_error': [lr_test_error[0], lr_test_error[1], lr_test_error[2], lr_test_error[3], \n",
    "                    ridge_test_error[0], ridge_test_error[1], ridge_test_error[2], ridge_test_error[3], \n",
    "                    rf_test_error[0], rf_test_error[1], rf_test_error[2], rf_test_error[3]],\n",
    "           'train_error': [lr_train_error[0], lr_train_error[1], lr_train_error[2], lr_train_error[3], \n",
    "                    ridge_train_error[0], ridge_train_error[1], ridge_train_error[2], ridge_train_error[3], \n",
    "                    rf_train_error[0], rf_train_error[1], rf_train_error[2], rf_train_error[3]],\n",
    "           'test_score': [lr_test_score[0], lr_test_score[1], lr_test_score[2], lr_test_score[3], \n",
    "                     ridge_test_score[0], ridge_test_score[1], ridge_test_score[2], ridge_test_score[3],  \n",
    "                     rf_test_score[0], rf_test_score[1], rf_test_score[2], rf_test_score[3]],\n",
    "           'train_score': [lr_train_score[0], lr_train_score[1], lr_train_score[2], lr_train_score[3],\n",
    "                             ridge_train_score[0], ridge_train_score[1], ridge_train_score[2], ridge_train_score[3],\n",
    "                             rf_train_score[0], rf_train_score[1], rf_train_score[2], rf_train_score[3]],\n",
    "           'description': ['no transformations', 'no transformations', 'zero prices removed', 'parabolic transformation', \n",
    "                           'no transformations', 'no transformations', 'zero prices removed', 'parabolic transformation', \n",
    "                           'no transformations', 'no transformations', 'zero prices removed', 'parabolic transformation', \n",
    "                            ],\n",
    "           'price outs removed': ['no', 'yes', 'yes', 'yes', 'no', 'yes', 'yes', 'yes', 'no', 'yes', 'yes', 'yes']}\n",
    "\n",
    "table = pd.DataFrame(content, columns=['model', 'test_score', 'train_score', 'test_error', 'train_error', 'price outs removed', 'description', 'features'], index=[1,2,3,4,5,6,7,8,9,10,11,12])\n",
    "table.sort_values('test_score', ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Coefficients of the best performing ridge regression model that has its `bedrooms` feature transformed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Coef</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>bedrooms</th>\n",
       "      <td>-73983.937888</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bathrooms</th>\n",
       "      <td>41692.401968</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sqft_living</th>\n",
       "      <td>180.235738</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sqft_lot</th>\n",
       "      <td>-4.171500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>floors</th>\n",
       "      <td>62571.711411</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>waterfront</th>\n",
       "      <td>468650.386551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>view</th>\n",
       "      <td>54503.817213</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>condition</th>\n",
       "      <td>29156.616047</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sqft_above</th>\n",
       "      <td>102.644951</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sqft_basement</th>\n",
       "      <td>77.368144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>yr_built</th>\n",
       "      <td>-2684.743900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>yr_renovated</th>\n",
       "      <td>12.473432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>zipcode</th>\n",
       "      <td>-140.794058</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total_sqft</th>\n",
       "      <td>3.477844</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>month</th>\n",
       "      <td>-2095.817430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>price_is_zero</th>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>renov_date_is_zero</th>\n",
       "      <td>12714.779384</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bedrooms_squared</th>\n",
       "      <td>2154.343263</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             Coef\n",
       "bedrooms            -73983.937888\n",
       "bathrooms            41692.401968\n",
       "sqft_living            180.235738\n",
       "sqft_lot                -4.171500\n",
       "floors               62571.711411\n",
       "waterfront          468650.386551\n",
       "view                 54503.817213\n",
       "condition            29156.616047\n",
       "sqft_above             102.644951\n",
       "sqft_basement           77.368144\n",
       "yr_built             -2684.743900\n",
       "yr_renovated            12.473432\n",
       "zipcode               -140.794058\n",
       "total_sqft               3.477844\n",
       "month                -2095.817430\n",
       "price_is_zero            0.000000\n",
       "renov_date_is_zero   12714.779384\n",
       "bedrooms_squared      2154.343263"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coefs = pd.DataFrame(ridge.coef_, index=list(X_transformed), columns=['Coef'])\n",
    "coefs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CI around my prediction accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[815920.713, 217673.139, 224897.897, 222715.763]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# RMSE == standard deviation of errors\n",
    "rf_test_error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "one_sd = min(rf_test_error)\n",
    "two_sd = 2*one_sd\n",
    "three_sd = 3*one_sd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 68% Confidence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I can be 68% confident that the error in the home price predictions will be between -217673.139 and 217673.139.\n"
     ]
    }
   ],
   "source": [
    "min_error = 0 - one_sd\n",
    "max_error = 0 + one_sd\n",
    "\n",
    "print('I can be 68% confident that the error in the home price predictions will be '\n",
    "      'between {} and {}.'.format(min_error, max_error))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 95% Confidence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I can be 95% confident that the error in the home price predictions will be between -435346.278 and 435346.278.\n"
     ]
    }
   ],
   "source": [
    "min_error = 0 - two_sd\n",
    "max_error = 0 + two_sd\n",
    "\n",
    "print('I can be 95% confident that the error in the home price predictions will be '\n",
    "      'between {} and {}.'.format(min_error, max_error))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What happens if my boss purchases 100 homes?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from math import sqrt\n",
    "\n",
    "np.random.seed(47)\n",
    "n = 100 # length of sample\n",
    "\n",
    "# function to sample the actual and predicted prices of 100 homes at a time.\n",
    "def draw_bs_reps(y_test, y_pred):\n",
    "    bs_error = np.random.choice(y_test - y_pred, n) # price sold - price bought == profit (gain or loss)\n",
    "    \n",
    "    error_mean = np.sum(bs_error) / len(bs_error) # avg gain or loss on n transactions\n",
    "    return error_mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mean of means: 2872\n",
      "std of means: 22766\n",
      "\n",
      "The above result means that if I calculate the error between the true/historical price and my predicted price of 100 homes, I can be 95% confident that the error of the mean of average error in the home price predictions will be between -4,266,000 and 4,840,600.\n",
      "\n",
      "In other words, if my boss was to buy 100 homes at my predicted price and sell them at their true price, I can be 95% confident that the most she would lose is $4,266,000 and the most she would gain is $4,840,600.\n"
     ]
    }
   ],
   "source": [
    "error_means = np.array([draw_bs_reps(y_test_transformed, predictions_dict['rf_pred3']) for i in range(1000)]) # Random Forest model\n",
    "error_means = np.round(error_means, 2)\n",
    "error_means_mean = np.mean(error_means) # mean of means\n",
    "print('mean of means: {}'.format(int(error_means_mean)))\n",
    "\n",
    "error_means_std = sqrt(np.sum((error_means-error_means_mean)**2) / (1000-1)) # standard deviation of means\n",
    "print('std of means: {}'.format(int(error_means_std)))\n",
    "\n",
    "# confidence interval\n",
    "min_error = int(error_means_mean - 2*error_means_std)\n",
    "max_error = int(error_means_mean + 2*error_means_std)\n",
    "\n",
    "print('')\n",
    "print('The above result means that if I calculate the error between the true/historical price and my predicted price ' \n",
    "      'of 100 homes, I can be 95% confident that the error of the mean of average error in the home price ' \n",
    "      'predictions will be between {:,} and {:,}.'.format(min_error*100, max_error*100))\n",
    "print('')\n",
    "print('In other words, if my boss was to buy 100 homes at my predicted price and sell them at their true price, ' \n",
    "      'I can be 95% confident that the most she would lose is ${:,} ' \n",
    "      'and the most she would gain is ${:,}.'.format(-1*min_error*100, max_error*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Histogram of the distribution of purchasing 100 homes at my predicted prices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-20872 25984\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD/CAYAAAAddgY2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAYXklEQVR4nO3dfbRddX3n8XdyQx4uj1qzRpJcFMH1XUtAgqHVBa1EFjGd6RAfSosU6BjIGlhhaVcrZoSxzBqRwVWmiriAuogoFWGpzJDS+AS0qw6Oo7bATVXgO2LteLkBYVVA4ADBk8wfe994SXPPvefsc+7dyX6//knOPvvhc39nn+/5nd/eZ+95u3btQpLUDPPnOoAkafZY9CWpQSz6ktQgFn1JahCLviQ1iEVfkhpkwUxmioiFwFeBqzNz66TprwFGgd/KzB+U094HXAS8BHwpMy/vJdi99977S4oPpV/0srwkNdQhwM5Vq1bttb5PW/QjYiXwaeA44OpJ0xcAnwMO2GPe9wMnAi8CfxMR38nMu3oIPh+YNzQ0dGi73QZgaGioh9UMXp3z1Tkb1DtfnbNBvfPVORvUO1/VbOXyU47izKSnfyFwGXDJHtMvB7YAR06adjqwJTOfBoiILwBnAb0U/V8MDQ0dunLlSjKTcn09rGbw6pyvztmg3vnqnA3qna/O2aDe+apmGx0dpd1uTzlCMm3Rz8wLywC7i35ErAGOBS4F/njS7MuBhyY93g6s6DLzbu12m8yk1WpNZOl1VQNV53x1zgb1zlfnbFDvfHXOBvXOVzXbxDeFqXR9IDci/g1wFXBeZu55DYf5wORp84DOCSRJs2ZGB3L3cDrFgYK7yq8fy4DbIuI8YKx8POFwYLzXcENDQ0RErb+Kwf79VXHQ6pyvztmg3vnqnA3qna9PwztTPt910c/MzcDmiccR8c/AGZn5g4h4HvhSRHwMeB44m+JbgSSpBvp6nn5m3g98HPgW8H3gm5m5pZ/bkCT1bsY9/cxcPcX01+7x+Hrg+kqpJEkD4S9yJalBejmQKzXCyMjIXEeQ+s6ir8Z66vln2Dbe/bnQJx+5kmd3PN/TsgDHLw8OW3JwT8tKVVn01VjbxpM1123oernHr7in52UB7tq4mVOOPrGnZaWqHNOXpAax6EtSg1j0JalBLPqS1CAWfUlqEIu+JDWIRV+SGsSiL0kNYtGXpAax6EtSg1j0JalBLPqS1CAWfUlqEIu+JDWIRV+SGsSiL0kNYtGXZtlRr/I2jJo73jlLmmUHLRru+VaN3mpRVc246EfEQuCrwNWZuTUifh24BlgC7AI+lJnfKOe9AjgDeAH4VGZu7ntyaR/W6+0WvdWiqprR8E5ErATuAU6aNPmLwH/OzJXAucCtEbEkIt4JrAaOK//dFBHRz9CSpN7MtKd/IXAZcAns7vVfkZl/Wz7/IDAEvBI4HbglM3cAOyJiC3Am8JFuw7XbbTKTVqsFQGb3X4dnQ53z1TkbzF2+kZF9d1y91WoxNjZW69e2ztmg3vmqZmu32x2fn1FPPzMvnBi6KR/vyMzPTJrlMuChzBwHlgPjk57bDqyYcWJJ0sBUOpAbEfOAjwK/D7ytnDyfYox/wjyg80fPFIaGhoiI3Z94dR0lqnO+OmeD+uero+Hh4dq/L+qcDeqdr2q20dHRjr39not+RCwCbqbo2Z+UmU+UT40ByybNejgv7/lLkuZIlZ7+jcABwNsy88VJ07cCF0fEZynO7HkXxTcBSdIc66noR8QbgT8AEvjupK8hZ2bm7RFxAnBfuf5PZub9/QgrSaqmq6KfmasnPZzXYb7LKA7uSpJqxMswSFKDWPQlqUEs+pLUIBZ9SWoQi74kNYhFX5IaxKIvSQ1i0ZekBrHoS1KDWPQlqUEs+pLUIBZ9SWoQi74kNUilO2dJc+2p559h23j39xI9+ciVA0gj1Z9FX/u0bePJmus2dL3c41fcM4A0Uv05vCNJDWLRl6QGsehLUoNY9CWpQSz6ktQgFn1JahCLviQ1yIzP04+IhcBXgaszc2tELAduBpYCTwLrM/Phct4rgDOAF4BPZebmvieXJHVtRj39iFgJ3AOcNGnytcBtmXkscBVwUznvO4HVwHHlv5siIvoXWZLUq5n29C8ELgMuAYiIA4C1wDkAmXlHRNwQEUcApwO3ZOYOYEdEbAHOBD7Sbbh2u01m0mq1KLfT7SpmRZ3z1TkbVMs3MjLS7zj7hFarxdjYWK1f2zpng3rnq5qt3W53fH5GRT8zLwSIiEvKSb8GvJiZz06a7VFgBbAc+OtJ07cDb5hhXjXM8uXLWbJkCUNDQ10vu2jRogEkkvZvvV57Zz6wa49p84D2Xp6bmN61oaEhImL3J15dR4nqnK/O2aDo1eyY1+a+8Qe6XrapF00bHh6u/fuiztmg3vmqZhsdHe3Y2++16D8OLI6IAzPzuXLa4cA4MAYsmzTvxHRpr7xomjR7ejplMzN/CdwJnA8QEeuA7Zn5CLAVOCciFkfEK4B3AV/pU15JUgVVztPfCPxORPwQ+DC/Oqh7O/A3wH3Ad4FPZub9VYNKkqrrangnM1dP+v84xRk8e5vvMoqzfSRJNeIvciWpQSz6ktQgFn1JahCLviQ1iEVfkhrEoi9JDWLRl6QGsehLUoNY9CWpQSz6ktQgFn1JahCLviQ1iEVfkhrEoi9JDWLRl6QGsehLUoNY9CWpQXq9MbqkOXDUq0Z2/39kZKTDnNLeWfSlfchBi4Z56vln2DaeXS97/PLgsCUHDyCV9iUWfWkfs208WXPdhq6Xu2vjZk45+sQBJNK+xDF9SWqQSj39iDgLuLR8+BBwHnAIcDOwFHgSWJ+ZD1fZjiSpP3ru6UfEK4DrgNMy8zjgaeCDwLXAbZl5LHAVcFM/gkqSqqsyvLMAOAA4NCKGgGHgJWAtZaHPzDuAoyPiiKpBJUnV9Ty8k5lPRMSHge9T9PJ/RtHT/0BmPjtp1keBFcBPu91Gu90mM2m1WhPb7DXuQNU5X52zASxbtmyuIzRKq9VibGxsVrYD9d3v6pyvarZ2u93x+Z6LfkQcD1wAvA54DLga+Cywa49Z5wGdU2iftXTpUhYvXtzz8gsXLuxjGknTqXIg9+3ANzNzHCAiPg18A1gcEQdm5nPlfIcD471sYGhoiIjY/YkXERXiDk6d881Gtl7PGwc4+ciVfU6jToaHh2dlP63zewLqna9qttHR0Y69/SpF/37ggoh4ZWb+HFgHfBtYCJwPXBMR64DtmflIhe2o5no9bxzg8Svu6XMaSZ1UGdO/OyKuA74TES8APwYuLNd5Y0RcADwHnNOXpJKkyiqdp5+ZHwc+vpen1lZZryRpMPxFriQ1iEVfkhrEoi9JDWLRl6QGsehLUoNY9CWpQSz6ktQgFn1JahCLviQ1iEVfkhrEoi9JDWLRl6QGsehLUoNY9CWpQSz6ktQgFn1JahCLviQ1iEVfaoijXjUy1xFUA5Vulyhp33HQomGeev4Zto1n18sevzw4bMnBA0il2WbRlxpk23iy5roNXS9318bNnHL0iQNIpNnm8I4kNYhFX5IapNLwTkSsBT4KLAF+ApwNHAzcDCwFngTWZ+bDFXNKkvqg555+RLyWorifk5nHAmPAJcC1wG3ltKuAm/qQU5LUB1V6+u8Gbs/MiVMBLgUOBR4CzgHIzDsi4oaIOCIzf9rtBtrtNplJq9WiXF+FuINT53yDzjYy4mmATdFqtRgbG5vxvFDP9wTUO1/VbO12u+PzVYr+0cBLEfFl4PXA/RQ9/Rcz89lJ8z0KrAC6LvqSpP6qUvQPAN4O/CYwDnwCuBLYtcd884DOHz1TGBoaIiJ2f+JFRM9hB6nO+eqcTfuW4eHhGe9Hdd/v6pyvarbR0dGOvf0qZ+88CnwzM8cycydwC/AbwOKIOHDSfIdTfChIkuZYlaJ/B3BqRCwvH78DuBe4EzgfICLWAdsz85FKKSVJfdHz8E5m/kNEfBD4ekQsAB4G1gOLgBsj4gLgOcqDupKkuVfpPP3MvBW4dS9Pra2yXknSYPiLXElqEIu+JDWIRV+SGsSiL0kNYtGXpAax6EtSg1j0JalBLPqS1CAWfUlqEIu+JDWIRV+SGsSiL0kNYtGXpAax6EtSg1j0JalBLPqS1CAWfUlqEIu+JDWIRV+SGsSiL0kNYtGXpAax6EtSgyzox0oi4o+AczPzxIg4GPg88HqgBVyUmd/rx3YkSdVU7ulHxAnApkmTLgd+lJnHABcAt0ZEXz5cJEnVVCrGEXEQ8GngUuB95eTTgXUAmXlfRPwLcBLwv7pdf7vdJjNptVqU66sSd2DqnG/Q2UZGRgayXtVPq9VibGxsxvNCPd8TUO98VbO12+2Oz1ftgV8P/Dnws0nTlgPjkx5vB1ZU3I4GaOnSpSxevLinZRctWtTnNJIGqeeiHxHvBXZk5hcjYvWkp+YDuyY9ngd0/uiZwtDQEBGx+xMvInoLO2B1zjfTbE89/wzbxrvvWZx85MqecmnfMzw8PON9vM7vCah3vqrZRkdHO/b2q/T0zwaWRcQocFD5/7uBMWAZ8HQ53+G8vOevGto2nqy5bkPXyz1+xT0DSCNpUHo+kJuZazLzmMxcCWwAHsjM04Ct5WMi4k0UHwB/34+wkqRqBnFWzWXADRHxQ4phnbMz88UBbEeS1KW+FP3M/DvgxPL/TwO/34/1SpL6y1/kSlKDWPQlqUEs+pLUIBZ9SWoQi74kNYhFX5IaxKIvSQ1i0ZekBvE695IGrtcL+gEcvzw4bMnBfU7UXBZ9SQPX6wX9AO7auJlTjj6xz4may+EdSWoQi74kNYhFX5IaxKIvSQ3igVxJ0zrqVSNdzT8y0t38mj0WfUnTOmjRsPdR3k9Y9CXNiPdR3j84pi9JDWLRl6QGsehLUoNY9CWpQSodyI2Ii4H1wE7gn4ANwELgZmAp8CSwPjMfrphTktQHPff0I+KtFAX/zZl5HPAAcCVwLXBbZh4LXAXc1I+g6s3IyIjnTEvarUpP/wlgY2Y+Wz6+D9gIvAU4ByAz74iIGyLiiMz8abWomo7nUUuaTs9FPzMfBB4EiIhDgD8FbgVOmPRBAPAosALouui3220yk1arNbHNXuMOVB3yjYyMsG2751Fr/9RqtRgbG+vr+qCeNaVqtna73fH5ygdyI2IZ8LfAtymGcnbtMcs8oHMKSdKsqHog943AVuCGzLw8IhYAiyPiwMx8rpztcGC8l/UPDQ0REbs/8SKiStyBqXs+aV83PDzc1/dXnd+zVbONjo527O1XOZD7auBu4EOZeTlAZv4SuBM4v5xnHbA9Mx/pdTuSpP6p0tPfBBwEbIqITeW0pDiYe2NEXAA8R3lQV5I096ocyP0T4E+meHptr+uVJA2Ov8iVpAax6EtSg3g9/ZrxB1aSBsmiXzPeqELSIDm8I6nWur0/rzqzpy+p1qrcn/f45cFhSw4eQKp9l0VfUu31Oux518bNnHL0iQNItO9yeEeSGsSiL0kNYtGXpAZxTF/SfmuqM3+afDc5i76k/ZZn/vxrFn1J+zXP/Hk5x/QlqUEs+pLUIBZ9SWoQx/QHwCtlSqori34f7Hn6l1fKlFRXFv296LWnDvbWpf3F/np1T4v+XvTaUwd769L+Yn89x9+iL0lT2B/P8bfoS1KfVRkaGvQlIgZW9CPi94D/CrSBvwP+ODN/OajtSVJdVBkagmJ4aLjPmSYMpOhHxKuBa4ATgUeBLwLrgRsGsT1JqpsqxwYHOTw0b9euXX1faUScC6zLzN8rH/874OLMPHWm67j33nt3AvN62f78+fOZN6+nRQHYtWsX7Z3tnpYdmj8E0NPyTVt2Lrft37xvLDuX267D37xz586elgd2rVq1aq8/vh3U8M5yYHzS4+3Aii7XsZPiF8O/6HbjFRpqt3m9fd7s3nYvyzdt2bnctn/zvrHsXG67Dn9zjw6hqJ97NaiiPx+Y/BViHsXY/oytWrXKg8yS1GeDuvbOGLBs0uPDeXnPX5I0BwbVm74L+LOIeA3FB8B6YOuAtiVJmqGB9PQz8zFgI0Whf4hiXP7aQWxLkjRzAzl7R5JUT15PX5IaxKIvSQ1i0ZekBrHoS1KDWPQlqUEs+pLUIBZ9SWqQOb2+TUT8GfA7FNfm+XJm/pdy+mrgkxQfSj8ANmTmcxFxMPB54PVAC7goM79XLvM+4CLgJeBLmXl5Of1Y4DPAMPAI8B8y8/EZZNsEnFUutyUz/1Ndsu2R838AP8nMizutMyIOAK4HTgZeBD6cmVvLZfZ674OIWA7cDCwFngTWZ+bD0+RZDHwaWFW20dcorrC6q25tN53ZuidERFxM8av1ncA/ARuAhUzR9hFxBXAG8ALwqczcXE5fTZft22XOPwLOzcwT+/maddo3Z5BpLfBRYAnwE+Bs4OC6tF1EnAVcWj58CDiP4oJoc5Zvznr6EfHvgbcCxwNvAs6IiFMjYhFFg/xhZh5HcT3+iUa7HPhRZh4DXADcGhELImIl8H7gzcCvA78dEWvKZW4GPlKu62vAJ2aQ7SzgPcBvlfnWRMTaOmTbI+dFwCl7TJ5qnRdR7GxvAN4BXB8Rr5h074M15d/6aooCBMWvqG/LzGOBq4CbZhBrE8XF9o4DVlIU/7Pr1nbTmaZd+rmdt5brfXP5tzwAXMkUbR8R7wRWU7TvamBTFLpu3y5znkDx2k7o52u2131zBpleW67znLKdxoBLqEnblX/DdcBp5XqfBj441/nmcnjnAIpP/MXAIopvHS8AvwE8mpnbyvk2U/S4AU4HPgeQmfcB/wKcVE7fkplPZ+YLwBeAsyJiBFiemV8pl/8M8Ltlz6KTs4D/npnPlutbB/x9TbIBEBHHA2dS9KonpnVa5+nATZm5KzP/H/B/ymlrgG9l5nhm7gQ+W+Y7AFhLuUNm5h3A0RFxxDTR/jdwRbmdHcAPgdfUqe1maK/t0sf1T3gC2JiZz5aP7wOOYuq2Px24JTN3ZOaTwBaK/aCX9p2RiDiIYj+7dNLkfr5mU+2b03k3cHtmTtye6lLgL6hP2y2gqHOHRsQQRb17aa7zDXx4JyL+gOLTbk9HAQ9TfM1bQDG88+2IeA9TX4t/quv0L6f46rS36bvnL78OvUDxtWp7h2zbgSMi4hsUPbzbMvPycrhjVrJBx7Y7huKFfw9wLsWH5r/KsMc6p8q3c4rpvwa8OKkYQdHDWAH8tEO212Xmz8v8x1HsnKspenGz1nZ90I97QkwrMx8EHgSIiEOAPwVuBU6You2XA3+9R643TJO36t9yPfDnwM8mTevna9ZrvqOBlyLiyxTDG/dT9PSn2m9nte0y84mI+DDwfYpe/s8oevofmMt8Ay/6mXkLcMue0yPiP5bbfzVFb/+rEfFeYAdTX4t/quv0z3T6y9bXIduPgN8E3lVm/HpE/HO53Kxkmybf54BrMvPHETH5qU7rnCrHUC/5pso2KePJwG3A+zPzH8vx3Flruz6ofE+IbkTEMuAO4NsUvcBNe8zSbZvMpH1nkuu9wI7M/GI5rjyhn69Zr/kOAN5O8V4dpxguurKH7Qyq7Y6nGHJ5HfAYcDXFN8Y5zTeXwzvvoPgq80JmPkVRQN5G52vxT/XcjKZHxIEUO8rPp8n2KPA/M7OVmb8A/gp4Sx2ylQdu1gIfiIhR4ELgDyPiY9Oss9t8jwOLy3Xs7e/tlPEsiq+m52XmzeXkOW+7Ls3aPSEi4o3Ad4C/yswL6dz23bYV0zw3nbOBt5T72mbgDRFxdw85etk3p/Mo8M3MHCuH4G6hGAqpS9u9vcw3npltiiGyY+Y631wW/fspDt7OL8f1/i3wPeC7wIqIeFM53wZ+dS3+reVjyueXUYy1bwXeXR6YXEyxo27NzDHgsYhYVy5/PnBnZr40TbYtlOPaEbEQ+G3g3jpky8xnMvPwzFyZmSspxjD/MjM/NM06twLnle19BMVB6rsp7n3w1oh4TUTMp7z3QRZnqdxZroNyndsz85FO+aI4m+ITwKmZ+bVJT81523Vpr+3Sx/UDuw8Y3w18KMuzXKZp+63AORGxuDxQ+C7gK/TWvtPKzDWZeUy5r20AHsjM0zqss5fXbKp9czp3AKeWw65QdCTvrUvbUdS40yLileXjdRTf5OY031yesvnfgE9RjP/tAL4O/EVmtiPiTGBzRCwB/i/FuDXAZcANEfFDiq8wZ2fmi8D9EfFx4FsUp7p9OTO3lMucVS7zMYoe1NkzyHYNxVjjP1IMf2wFPpeZO2uQbTpTrfNaIChO95pHMezyGEBETNz7YBFwD7+698FG4MaIuAB4DjhnBtu/kqIz8flJQ09fyMyr9oG22y0zH+vQLv20CTiI4kyNiSGdZIq2z8zboziT5j6K9+8nM/N+gB7at4p+vmZT7pudZOY/RMQHKYZfF1AcI1xP8XrNedtl5t0RcR3wnfL4xY8pvpkvmMt8Xk9fkhrEX+RKUoNY9CWpQSz6ktQgFn1JahCLviQ1iEVfkhrEoi9JDWLRl6QG+f9d1/8ZzJb+vQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(error_means, bins=20)\n",
    "plt.plot()\n",
    "print(min_error, max_error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What happens if my boss purchases homes $1000 below my predicted prices?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# running the same experiment as before, with $1000 deducted from my predicted prices\n",
    "\n",
    "np.random.seed(47)\n",
    "n = 100 # length of sample\n",
    "N_rep = 1000 # number of times to bootstrap/resample\n",
    "\n",
    "def draw_bs_reps(y_test, y_pred):\n",
    "    bs_error = np.random.choice(y_test - (y_pred-1000), n) # deduct 1000 from each predicted price\n",
    "    \n",
    "    error_mean = np.sum(bs_error) / len(bs_error) # avg gain or loss on n transactions\n",
    "    return error_mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mean of means: 3280\n",
      "std of means: 22203\n",
      "\n",
      "If my boss was to buy 100 homes at $1000 less than my predicted price and sell them at their true price, I can be 95% confident that the most she would lose is $4,112,500 and the most she would gain is $4,768,700.\n"
     ]
    }
   ],
   "source": [
    "error_means = np.array([draw_bs_reps(y_test_transformed, predictions_dict['rf_pred3']) for i in range(1000)]) # Random Forest model\n",
    "error_means = np.round(error_means, 2)\n",
    "error_means_mean = np.mean(error_means) # mean of means\n",
    "print('mean of means: {}'.format(int(error_means_mean)))\n",
    "\n",
    "error_means_std = sqrt(np.sum((error_means-error_means_mean)**2) / (1000-1)) # standard deviation of means\n",
    "print('std of means: {}'.format(int(error_means_std)))\n",
    "\n",
    "# confidence interval\n",
    "min_error_1000 = int(error_means_mean - 2*error_means_std)\n",
    "max_error_1000 = int(error_means_mean + 2*error_means_std)\n",
    "\n",
    "print('')\n",
    "print('If my boss was to buy 100 homes at $1000 less than my predicted price and sell them at their true price, ' \n",
    "      'I can be 95% confident that the most she would lose is ${:,} ' \n",
    "      'and the most she would gain is ${:,}.'.format(-1*min_error_1000*100, max_error_1000*100))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}