2_Feature_Selection_and_Test_Subsets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mutual Fund Price Prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mary Donovan Martello"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The goal of this project was to design models to predict mutual fund return values and to identify the features that best predict those return values. This notebook includes feature selection and tests different subsets of input features to see which subset(s) may produce the best results in the predictive models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 2:  Feature Selection and Test Subsets of Input Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import libraries\n",
    "from __future__ import print_function, division\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "# import libraries\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import random\n",
    "import yellowbrick\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython.display import Markdown, display\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.decomposition import PCA\n",
    "\n",
    "from sklearn.feature_selection import RFECV\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.svm import SVR\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import Lasso\n",
    "\n",
    "\n",
    "from sklearn import metrics\n",
    "from sklearn.model_selection import StratifiedKFold, KFold, cross_val_score\n",
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "#stop unnecessary warnings from printing to the screen\n",
    "import warnings\n",
    "warnings.simplefilter('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### This data source contains 25,000+ mutual funds with 125 features including general aspects (as total net assets, management company and size), portfolio indicators (as cash, stocks, bonds, and sectors), returns (as year-to-date, 2018-10) and financial ratios (as price/earnings, Treynor and Sharpe ratios, alpha, and beta).  The target variable it year-to-date return (ytd_return) of the mutual funds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load data into a dataframe\n",
    "# import file\n",
    "mf = pd.read_csv('Mutual Funds.csv')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fund_name</th>\n",
       "      <th>fund_extended_name</th>\n",
       "      <th>category</th>\n",
       "      <th>fund_family</th>\n",
       "      <th>net_assets</th>\n",
       "      <th>ytd_return</th>\n",
       "      <th>fund_yield</th>\n",
       "      <th>morningstar_rating</th>\n",
       "      <th>inception_date</th>\n",
       "      <th>investment</th>\n",
       "      <th>...</th>\n",
       "      <th>fund_sharpe_ratio_5years</th>\n",
       "      <th>category_sharpe_ratio_5years</th>\n",
       "      <th>fund_sharpe_ratio_10years</th>\n",
       "      <th>category_sharpe_ratio_10years</th>\n",
       "      <th>fund_treynor_ratio_3years</th>\n",
       "      <th>category_treynor_ratio_3years</th>\n",
       "      <th>fund_treynor_ratio_5years</th>\n",
       "      <th>category_treynor_ratio_5years</th>\n",
       "      <th>fund_treynor_ratio_10years</th>\n",
       "      <th>category_treynor_ratio_10years</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AAAAX</td>\n",
       "      <td>DWS RREEF Real Assets Fund - Class A</td>\n",
       "      <td>World Allocation</td>\n",
       "      <td>DWS</td>\n",
       "      <td>530280000.0</td>\n",
       "      <td>12.79</td>\n",
       "      <td>1.43</td>\n",
       "      <td>3</td>\n",
       "      <td>2007-07-29</td>\n",
       "      <td>Blend</td>\n",
       "      <td>...</td>\n",
       "      <td>0.31</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.73</td>\n",
       "      <td>0.01</td>\n",
       "      <td>5.05</td>\n",
       "      <td>0.05</td>\n",
       "      <td>2.26</td>\n",
       "      <td>0.02</td>\n",
       "      <td>6.71</td>\n",
       "      <td>0.07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AAADX</td>\n",
       "      <td>Aberdeen Income Builder Fund Class A</td>\n",
       "      <td>Large Blend</td>\n",
       "      <td>Aberdeen</td>\n",
       "      <td>88840000.0</td>\n",
       "      <td>9.21</td>\n",
       "      <td>2.62</td>\n",
       "      <td>2</td>\n",
       "      <td>2011-12-29</td>\n",
       "      <td>Blend</td>\n",
       "      <td>...</td>\n",
       "      <td>0.62</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.97</td>\n",
       "      <td>0.01</td>\n",
       "      <td>11.55</td>\n",
       "      <td>0.11</td>\n",
       "      <td>6.91</td>\n",
       "      <td>0.08</td>\n",
       "      <td>12.77</td>\n",
       "      <td>0.14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2 rows × 125 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  fund_name                    fund_extended_name          category  \\\n",
       "0     AAAAX  DWS RREEF Real Assets Fund - Class A  World Allocation   \n",
       "1     AAADX  Aberdeen Income Builder Fund Class A       Large Blend   \n",
       "\n",
       "  fund_family   net_assets  ytd_return  fund_yield  morningstar_rating  \\\n",
       "0         DWS  530280000.0       12.79        1.43                   3   \n",
       "1    Aberdeen   88840000.0        9.21        2.62                   2   \n",
       "\n",
       "  inception_date investment  ... fund_sharpe_ratio_5years  \\\n",
       "0     2007-07-29      Blend  ...                     0.31   \n",
       "1     2011-12-29      Blend  ...                     0.62   \n",
       "\n",
       "  category_sharpe_ratio_5years  fund_sharpe_ratio_10years  \\\n",
       "0                         0.00                       0.73   \n",
       "1                         0.01                       0.97   \n",
       "\n",
       "   category_sharpe_ratio_10years  fund_treynor_ratio_3years  \\\n",
       "0                           0.01                       5.05   \n",
       "1                           0.01                      11.55   \n",
       "\n",
       "   category_treynor_ratio_3years  fund_treynor_ratio_5years  \\\n",
       "0                           0.05                       2.26   \n",
       "1                           0.11                       6.91   \n",
       "\n",
       "   category_treynor_ratio_5years  fund_treynor_ratio_10years  \\\n",
       "0                           0.02                        6.71   \n",
       "1                           0.08                       12.77   \n",
       "\n",
       "   category_treynor_ratio_10years  \n",
       "0                            0.07  \n",
       "1                            0.14  \n",
       "\n",
       "[2 rows x 125 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mf.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### This notebook uses two datasets that were cleaned and prepared in the 1_EDA_Prep_Dimension notebook.  One of the datasets included PCA Components.  The other dataset includes the original features before PCA transformation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "pcaDf = pd.read_csv('pcaMFdf4.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('regMFdf2.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Best PCA Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "# function to show best features from the PCA analysis (and the original feature names)\n",
    "model = PCA(n_components=57, whiten=True).fit(qPredictors)\n",
    "X_pc = model.transform(qPredictors)\n",
    "# number of components\n",
    "n_pcs= model.components_.shape[0]\n",
    "# get the index of the most important feature on EACH component\n",
    "# LIST COMPREHENSION HERE\n",
    "most_important = [np.abs(model.components_[i]).argmax() for i in range(n_pcs)]\n",
    "initial_feature_names = ['fund_yield', 'morningstar_rating', 'net_annual_expense_ratio_fund',\n",
    "       'net_annual_expense_ratio_category', 'portfolio_cash',\n",
    "       'portfolio_stocks', 'portfolio_bonds', 'portfolio_others',\n",
    "       'portfolio_preferred', 'portfolio_convertable', 'price_book',\n",
    "       'median_market_cap', 'basic_materials', 'consumer_cyclical',\n",
    "       'financial_services', 'real_estate', 'consumer_defensive', 'healthcare',\n",
    "       'utilities', 'communication_services', 'energy', 'industrials',\n",
    "       'technology', 'rating_aaa', 'rating_aa', 'rating_a', 'rating_bbb',\n",
    "       'rating_bb', 'rating_b', 'rating_below_b', 'rating_others',\n",
    "       'morningstar_return_rating', 'category_return_ytd',\n",
    "       'fund_return_1month', 'category_return_1month', 'fund_return_3months',\n",
    "       'fund_return_1year', 'category_return_1year', 'fund_return_3years',\n",
    "       'category_return_3years', 'fund_return_5years',\n",
    "       'category_return_5years', 'fund_return_10years',\n",
    "       'category_return_10years', 'fund_return_2018', 'category_return_2018',\n",
    "       'fund_return_2017', 'category_return_2017', 'fund_return_2016',\n",
    "       'category_return_2016', 'fund_return_2015', 'category_return_2015',\n",
    "       'category_return_2011', 'morningstar_risk_rating', 'years_up',\n",
    "       'years_down', 'fund_alpha_3years', 'category_alpha_3years',\n",
    "       'fund_alpha_5years', 'category_alpha_5years', 'category_alpha_10years',\n",
    "       'fund_beta_3years', 'category_beta_3years', 'fund_beta_5years',\n",
    "       'category_beta_5years', 'category_beta_10years',\n",
    "       'fund_mean_annual_return_3years', 'category_mean_annual_return_3years',\n",
    "       'fund_mean_annual_return_5years', 'category_mean_annual_return_5years',\n",
    "       'category_mean_annual_return_10years', 'fund_r_squared_3years',\n",
    "       'category_r_squared_3years', 'fund_r_squared_5years',\n",
    "       'category_r_squared_5years', 'category_r_squared_10years',\n",
    "       'fund_standard_deviation_3years', 'category_standard_deviation_3years',\n",
    "       'fund_standard_deviation_5years', 'category_standard_deviation_5years',\n",
    "       'category_standard_deviation_10years', 'fund_sharpe_ratio_3years',\n",
    "       'category_sharpe_ratio_3years', 'fund_sharpe_ratio_5years',\n",
    "       'category_sharpe_ratio_5years', 'category_sharpe_ratio_10years',\n",
    "       'category_treynor_ratio_3years', 'category_treynor_ratio_5years',\n",
    "       'category_treynor_ratio_10years', 'net_assets']\n",
    "# get the names\n",
    "most_important_names = [initial_feature_names[most_important[i]] for i in range(n_pcs)]\n",
    "# LIST COMPREHENSION HERE AGAIN\n",
    "dic = {'PC{}'.format(i+1): most_important_names[i] for i in range(n_pcs)}\n",
    "# build the dataframe\n",
    "dfpca = pd.DataFrame(sorted(dic.items()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "       0                               1\n",
      "0    PC1                      net_assets\n",
      "1   PC10                fund_return_2015\n",
      "2   PC11                      price_book\n",
      "3   PC12                      price_book\n",
      "4   PC13            category_return_2016\n",
      "5   PC14            category_return_2011\n",
      "6   PC15                fund_return_2018\n",
      "7   PC16                      rating_aaa\n",
      "8   PC17            category_return_2015\n",
      "9   PC18            category_return_2015\n",
      "10  PC19                        rating_a\n",
      "11   PC2               median_market_cap\n",
      "12  PC20                      fund_yield\n",
      "13  PC21         morningstar_risk_rating\n",
      "14  PC22                     real_estate\n",
      "15  PC23                      rating_aaa\n",
      "16  PC24                          energy\n",
      "17  PC25            category_return_2018\n",
      "18  PC26              fund_return_1month\n",
      "19  PC27              fund_return_1month\n",
      "20  PC28             fund_return_10years\n",
      "21  PC29                      fund_yield\n",
      "22   PC3                portfolio_stocks\n",
      "23  PC30                  portfolio_cash\n",
      "24  PC31                     real_estate\n",
      "25  PC32                   rating_others\n",
      "26  PC33                      healthcare\n",
      "27  PC34                     industrials\n",
      "28  PC35          communication_services\n",
      "29  PC36                 basic_materials\n",
      "30  PC37                     industrials\n",
      "31  PC38                 basic_materials\n",
      "32  PC39              consumer_defensive\n",
      "33   PC4           fund_r_squared_5years\n",
      "34  PC40              financial_services\n",
      "35  PC41  category_treynor_ratio_10years\n",
      "36  PC42                portfolio_others\n",
      "37  PC43                        years_up\n",
      "38  PC44                      healthcare\n",
      "39  PC45   net_annual_expense_ratio_fund\n",
      "40  PC46  category_treynor_ratio_10years\n",
      "41  PC47            category_return_2017\n",
      "42  PC48             portfolio_preferred\n",
      "43  PC49                  rating_below_b\n",
      "44   PC5               fund_return_1year\n",
      "45  PC50              fund_return_5years\n",
      "46  PC51             portfolio_preferred\n",
      "47  PC52                        rating_a\n",
      "48  PC53       morningstar_return_rating\n",
      "49  PC54                        rating_b\n",
      "50  PC55                        rating_b\n",
      "51  PC56                fund_beta_5years\n",
      "52  PC57          category_return_1month\n",
      "53   PC6                fund_return_2017\n",
      "54   PC7                 portfolio_bonds\n",
      "55   PC8                fund_return_2017\n",
      "56   PC9                fund_return_2015\n"
     ]
    }
   ],
   "source": [
    "# print the most important PCA components with the labels from original dataframe\n",
    "print(dfpca)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Recursive Feature Selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> See if RFECV will narrow down important features for the models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "scikit-learn's RFECV to conduct recursive feature elimination using cross-validation\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 317,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "from sklearn.feature_selection import RFECV\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 318,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a linear regression\n",
    "ols = LinearRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 319,
   "metadata": {},
   "outputs": [],
   "source": [
    "# separate into input and output columns (X variable for features and y for target variables)\n",
    "\n",
    "X = df.drop(['ytd_return', 'investment', 'size'], axis = 1) \n",
    "y = df['ytd_return']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 320,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(11796, 90)"
      ]
     },
     "execution_count": 320,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check shape\n",
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 321,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RFECV(estimator=LinearRegression(), scoring='neg_mean_squared_error')"
      ]
     },
     "execution_count": 321,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# recursively eliminate features\n",
    "rfecv = RFECV(estimator=ols, step=1, \n",
    "              scoring='neg_mean_squared_error')\n",
    "rfecv.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 322,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.43,  3.  ,  1.22, ...,  0.01,  0.05,  0.02],\n",
       "       [ 0.  ,  3.  ,  1.15, ...,  0.01,  0.13,  0.1 ],\n",
       "       [ 1.38,  4.  ,  0.91, ...,  0.01,  0.05,  0.03],\n",
       "       ...,\n",
       "       [ 1.78,  5.  ,  0.76, ...,  0.01,  0.09,  0.07],\n",
       "       [ 1.36,  4.  ,  1.27, ...,  0.01,  0.11,  0.08],\n",
       "       [ 0.  ,  2.  ,  3.19, ...,  0.  ,  0.05, -0.09]])"
      ]
     },
     "execution_count": 322,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rfecv.transform(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 323,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "86"
      ]
     },
     "execution_count": 323,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# of best features\n",
    "rfecv.n_features_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 324,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ True,  True,  True,  True,  True, False,  True,  True,  True,\n",
       "        True,  True, False,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True,  True,  True,\n",
       "        True,  True,  True,  True,  True,  True,  True, False, False])"
      ]
     },
     "execution_count": 324,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# which categories are best\n",
    "rfecv.support_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 326,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 4, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,\n",
       "       3, 5])"
      ]
     },
     "execution_count": 326,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Rank features best (1) to worst\n",
    "rfecv.ranking_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test Subsets of Input Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Create feature dataframes for modeling.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   investment_Blend  investment_Growth  investment_Value  size_Large  \\\n",
      "0                 1                  0                 0           1   \n",
      "1                 0                  1                 0           1   \n",
      "2                 0                  1                 0           1   \n",
      "3                 1                  0                 0           1   \n",
      "4                 1                  0                 0           1   \n",
      "5                 0                  1                 0           1   \n",
      "6                 1                  0                 0           1   \n",
      "7                 1                  0                 0           1   \n",
      "\n",
      "   size_Medium  size_Small  \n",
      "0            0           0  \n",
      "1            0           0  \n",
      "2            0           0  \n",
      "3            0           0  \n",
      "4            0           0  \n",
      "5            0           0  \n",
      "6            0           0  \n",
      "7            0           0  \n"
     ]
    }
   ],
   "source": [
    "# convert categorical data to numbers\n",
    "\n",
    "#get the categorical data\n",
    "cat_features = ['investment', 'size']\n",
    "df_cat = pcaDf[cat_features]\n",
    "\n",
    "# One Hot Encoding \n",
    "dfDumm = pd.get_dummies(df_cat)\n",
    "\n",
    "# check the data\n",
    "print(dfDumm.head(8))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Create separate dataframes/variables for the feature variables and the target variable "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a whole features dataset that can be used for train and validation data splitting\n",
    "# here we will combine the numerical features and the dummie features together\n",
    "dfNum = pcaDf.drop(['ytd_return', 'investment', 'size'], axis = 1)\n",
    "X = pd.concat([dfNum, dfDumm], axis=1)\n",
    "# create a whole target dataset that can be used for train and validation data splitting\n",
    "y =  pcaDf['ytd_return']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implement a baseline model for comparison to enhanced models and subsets of features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Before using k-fold cross-validation for model evaluation, use holdout method to create a baseline linear regression model for comparison. Data was scaled before PCA analysis and therefore no further scaling of the training dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# separate data into training and validation \n",
    "XTrain, XTest, yTrain, yTest = train_test_split(X, y, test_size =0.3, random_state=11)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Baseline linear regression model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9615529409719875\n"
     ]
    }
   ],
   "source": [
    "# to compare a k-fold cross-validation data split, train a model and evaluate the performance score of the holdout data method\n",
    "#train simple linear regression model\n",
    "\n",
    "# Instantiate the linear regression model using default parameters\n",
    "ols11_2=LinearRegression()\n",
    "\n",
    "# Fit the model with training data\n",
    "ols11_2.fit(XTrain, yTrain)\n",
    "\n",
    "# evaluate the baseline with the r-squared score\n",
    "print(ols11_2.score(XTest, yTest))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Predict the target using the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# predicting on XTest\n",
    "\n",
    "pred = ols11_2.predict(XTest)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show residuals from the predictions using a residual plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Residual plot')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAFXCAYAAABz8D0iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXhTZdo/8G+WpkvSNhVQCm3ZFFmKFErBERDxHUGrLVWsOzo/HX3dBgFBxA1UNkWGUXAbRx3UcVRQS3EQdV4d2RErYMvILrSFUim0tFnaNMn5/VESmjYnSdvT5OTk+7muucbkJKfPQ5Jzn2e7H5UgCAKIiIgopNShLgARERExIBMREckCAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMlGQXXzxxcjJycGkSZOQl5eHiRMnYvLkySguLm73OZ988kls2bKl1fPFxcW48sor233e8vJyDBs2rE3vWb58OZ577jm/r7v77rtx+vTp9haNSHG0oS4AUSRauXIlzjvvPPfjt99+G/Pnz8fHH3/crvMtWLBAqqIFzebNm0NdBCJZYQuZKMTsdjsqKiqQmJjofu7111/H9ddfj0mTJuHBBx9EZWUlAODrr7/G9ddfjxtuuAH5+fnYsWMHAGDKlClYv349AODDDz90t7o//PBD9zlbtlybP961axduv/125Ofn44orrsATTzzhs8zl5eUYP348nnnmGUyaNAm5ubn48ccfW73uwIEDmDJlCnJycpCbm4uCggIAwJw5cwAAd911FyoqKtr8b0akRGwhE4XAXXfdBQCorq5GdHQ0xo8fj0WLFgEACgoKsH//fqxatQparRYff/wxnnrqKbz11lt48cUX8dJLLyEjIwObNm3C9u3bkZWV5T7vL7/8ghUrVmDNmjXo1q0bnnnmmYDK895772Hq1KkYNWoUzGYz/ud//gclJSUwGo2i7zl+/DiysrLw3HPP4fvvv8e0adPw3XffuY/b7XY88MADeOyxxzBhwgRUVlYiPz8fvXr1wqJFi/DZZ5+16ikgimQMyEQh4ApEe/bswX333YdRo0ahS5cuAIDvvvsOxcXFmDx5MgDA6XTCarUCAK699lo8/PDDGDduHEaPHo17773X47xbt27F6NGj0a1bNwDAzTffjE2bNvktz+LFi7Fhwwa88cYbOHz4MBoaGmCxWHwG5MTEROTk5AAAxo0bB41Gg3379rmPHzlyBA0NDZgwYQIA4IILLsCECROwcePGNo9LE0UCBmSiEBo8eDDmzJmDxx9/HAMHDkRKSgqcTif++Mc/4rbbbgMA2Gw2nDlzBgAwffp0TJ48GZs3b8Znn32Gd955B6tXr/Y4Z/P09BqNxv3fKpXK41hjY6P7v++44w5cfPHFGDt2LK655hrs3r0b/tLcNz830HTj0Pw5h8MBlUrVqmx2u93neYkiFceQiULsuuuuwyWXXOLush4zZgxWr14Nk8kEAHj55Zfx2GOPwW6348orr4TVasWtt96KuXPnYt++fbDZbO5zjR49Gps3b8aJEycAAJ9//rn7WFJSEvbs2QNBEGAymdzdy7W1tSguLsbMmTMxYcIEnDhxAqWlpXA6nT7Lffr0aWzYsAEA8O233yIqKgr9+/d3H+/bty+0Wi2+/vprAEBlZSW++uorXHbZZQCaAjqDM9E5bCETycDTTz+N3NxcbNy4Efn5+aisrMRNN90ElUqF5ORkLF68GFqtFk888QRmzpwJrVYLlUqFhQsXQqfTuc9z8cUXY9asWbjrrrug1+txySWXuI+5zj9hwgRccMEFGDlyJARBQEJCAu677z5cf/31iIuLwwUXXIDhw4fj6NGjSE1NFS1zdHQ01qxZg5deegkxMTF49dVXPVrIUVFReO211zB//nwsX74cDocDDz30EC699FIAwNVXX40pU6Zg+fLlHoGcKFKpuP0iEbVVeXk5cnJysHPnzlAXhUgx2GVNREQkA2whExERyQBbyERERDLAgExERCQDIZtl7XQ6YTabERUV1WqtIhERkRIJgoDGxkbo9Xqo1Z5t4pAFZLPZjP3794fqzxMREYVM//79ER8f7/FcyAJyVFQUgKZCNV9H2RElJSVIT0+X5FxyxnoqC+upLKynskhdT5vNhv3797tjYHMhC8iubmqdTofo6GjJzivlueSM9VQW1lNZWE9l6Yx6ehuq5aQuIiIiGWBAJiIikgEGZCIiIhlgQCYiIpIBBmQiIiIZYEAmIiKSAQZkIiIiGWBAJlIYi82OQ1V1sNjsoS4KEbVByBKDEJG07A4nZq0tQmFJGUprzEgz6pGbnoolOZnQanjvTSR3DMhECjFrbRFe2bjX/fhItdn9eFleVqiKRUQB4m0zkQJYbHasKSnzeqywpJzd10RhgAGZSAEqaq0oqzF7PVZWY0JFrTXIJSKitmJAJlKA5IRYpBn1Xo+lGg1ITogNcomIqK0YkIkUIE6nRW56qtdjuekpiNNxugiR3PFXSqQQS3IyATSNGZfVmJBqNCA3PcX9PBHJGwMykUJoNWosy8vCguxhqKi1Ijkhli1jojDCXyuRwsTptOjXNT7UxSCiNuIYMhERkQwwIBMREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMhERkQwwIBMREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMhERkQwwIBMREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMhERkQwwIBMREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMhERkQwwIBMREckAAzIREZEMMCATERHJQIcC8u7duzFlyhQAwNGjR3Hrrbfitttuw9y5c+F0OiUpIBERUSRod0B+66238NRTT6GhoQEAsGjRIkybNg0ffvghBEHA//3f/0lWSCIiIqVrd0BOS0vD8uXL3Y/37NmDkSNHAgAuv/xybNmypeOlIyIiihDtDsgTJ06EVqt1PxYEASqVCgCg1+tRV1fX8dIRERFFCK3/lwRGrT4X281mMxISEgJ6X0lJiVRFAAAUFRVJej65Yj2VhfVUFtZTWYJVT8kC8qBBg7B9+3aMGjUKGzZswKWXXhrQ+9LT0xEdHS1JGYqKipCZmSnJueSM9VQW1lNZWE9lkbqeDQ0Nog1RyZY9zZ49G8uXL8fNN9+MxsZGTJw4UapTExERKV6HWsgpKSn45JNPAAB9+vTBBx98IEmhiIiIIg0TgxAREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMhERkQwwIBMREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMhERkQwwIBMREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMhERkQwwIBMREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMhERkQwwIBMREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMhERkQwwIBMREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMhERkQwwIBMREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADMlEIWWx2lNfZYLHZQ12UgFlsdhyqqgurMhOFA22oC0AUiewOJ2atLUJhSRlKq81I21SB3PRULMnJhFYjz/tku8OJaQU7UFhShuO1VvRK0su+zEThhAGZKARmrS3CKxv3uh8fqTa7Hy/Ly3I/b7HZUVFrRXJCLOJ0ofu52h1OjPrLOuw6Xu1+TqzMRNQ+vK0lCjKLzY41JWVejxWWlMNis8PucOJPn27HgEUFGLC4AENeLMT0gh2wO5xBLm2T6QU7PIJxc64yE1HHsIVMFGQVtVaU1Zi9HiurMaGsxozb3t8om9aorxsIACitNqGi1op+XeODWCoi5WELmSjIkhNikWbUez2WajTgz//5r6xaoxW1VlTUWUWPJyfEITkhNoglIlImBmSiIIvTaZGbnur1WPbAHlj3yzHR95bVNLVGg8nXDQQA5KanhHR8m0gpJP8V5eXlIT6+qesqJSUFixYtkvpPEIW9JTmZAJpavKXVJqQlGZCbnoL7L+uP17fsF31f94TYoLdGXTcQzSehuWT0SMJfOKGLSBKSBuSGhgYAwPvvvy/laYkUR6tRY1leFhZkD8M3W3bgqsuyEKfTwmKzo1eSHkeqvY8xTxqcGpLWaPMbiLIaE7onxGLS4FQsy8vikiciiUj6y967dy+sVivuvvtu2O12zJgxAxkZGVL+CSJFidNpkRKvcwdZf63RUC0van4DIYdlWERKpBIEQZDqZPv27cPu3buRn5+PI0eO4N5778X69euh1bb+4TY0NKCkpESqP02kGHangFd+qsT3x2pxwmxH11gNxvVMwIwR3aFVq0JdPCKSQHp6OqKjoz2ek/QWt0+fPujVqxdUKhX69OkDo9GIkydPIjk5uU2Faq+ioiJkZmZKci45Yz2VpaioCAOHDPVoef4jSz5JQaQSSZ8n66kcUtfTV2NU0l/56tWrsX//fsybNw+VlZUwmUzo1q2blH+CSFHsDif+/OMJbPvyKEprzEgznktHGafTcm0vUQSRNCDfeOONmDNnDm699VaoVCosXLjQa3c1ETWZtbYIH+0/7X7MdJREkUvSaKnT6bB06VIpT0mkWP5SaC7IHqaIrmoiCgzXKxCFiL8UmsFOAEJEocWATNTJxPYP9pdCk+koiSIL+8OIOonHnsctJmxpNWqfa47llo5SaTO+ieSIvyyidvIXpALZ83hJTiZ+q/wN207aUFZjQqqxKYWmKzNWqPm7qSAi6TAgE7WRtyB1zcCe+NPYAUg16t0pMAOZsKXVqDFjRPdW65DlIpCbCiKSBm9xidrIFaSOVJvhFJqC1Otb9mPQC4UY8mIhphfsQFmNuU0TtlxrjlsGY7Hx52Dwd1MRijIRKZl8bsWJwoCvIAWca0E2OpxIM3rfJCKQCVu1VhseKdiB/xw8gfIzlpB0FQcyC5yJS4ikw4BM1Aa+glRzX/5yHNkDe+I1L1spGmOjoBMJqq7u8Hd/OIi6hnMt0FB0Fbtmgbf3poKI2oZd1kRt4GupUnNlNSY8PHYAMnoktTq263g1Zq0t8vo+V3d482DcnL+uYim7uF2zwL2R2yxwIiVgQCZqA19BqrlUowFd4qJRbbV5Pf73Hw6itsUxf93hQFOgP3yqrlXQtTucmF6wA0NeLMSAxQXusWy7wxlArcQtyh6GjB5J0JzdZUqjViGjRxIWZQ/r0HmJqDUGZKI2WpKTialjB6B3kkH0NbnpKThT3yjavV3bYMcjBTs8ngukO1yv0yLnb9+2CrreJpq9snGvaEs8UHPW7cSu49VwOJt2aXU4Bew6Xo0563Z26LxE1BoDMlEbaTVqLMvLQvFjOfjv7Fw8eFl/9E4yQKMCeicZMHXsACzJyURyQixSjHGi5/nPwUqPVm4g3eG1DXaU1lg8gu70gh2dMhu61mrDuz8clPy8ROQdB4GIvAgkM1WcTouLz0/E8smj8EKz1wPA0WozkhNicUW/7njvx8Ne33/sjNlj+ZOvzF0AEKVWofFsS7W5z0tKcaK23ut72jsb2mKz456Pt4qOZXOWNZH0GJCJmrE7nHj40+0o3FOG30wN6JUU2HKjOJ0W+igN7vzHJuwoq8LxWivSjHpcPaAH4nVa1HlpTXaPj0NiTBRqmj3nytD19x8OorZFMPQWjAGgorYeWjVg9zJc3NbZ0K7u74LiUpTWWERf1zNRz1nWRBJjQCY6q95mR6/5n6HK3OB+LpDlRvU2O0YvX4/dx6vRPGQeqTbjja0HkNEjCbuOV7d637FaC0b9ZR0u7RaNlRlOaDVqaDVqLMgehs+LS1sFZF+8BWOg7bOhW2bmEnPFhRdwljWRxDiGTHTWpa986RGMm/M1Zjp6+XrsahGMm6u22vDA2XHmlo5Um/HR/tMek68qaq04dka8dRqohGgtnp041O/rXEulqkz1fmd5u877MtNmEkmOAZkIQJWpHntO1IgeL632vj9xlakexSdat36bK68xY/q4Qdg+7Rr0FOnmbR7wfU3uiosK/CdrttlxUuQGA2i9VGr40i9w1EsSkJb+MPJCJMTqAi4HEQWGAZkinsVmx1d7j0NkiBYA0M0Q3WrM1O5w4v5V2+Bvqa9rHPdMfSMq6loHdQA4Um3Cr6frAPhe69yva4LvP+bl74ppuVTqmJcbjuZ6JendM8iJSHocBKKI1XzXJn8tw9zBqa3GTGetLcLnAXTxusZxfaWiBIDbP9iEXTNzAJyb3FVYUu7elvGagT2w7pdjgVTN4++6ZownxkThTH2jO0gH0j3tcueIvnh18iiOGxN1Iv66KGIFOoGpS5wO068YBIvN7g5ItVYb3hFZo+sSpVbhgdEXu4NrnE6Lq/on463t3t/338ozqDLVo6shxr3WeUH2MBw+Ved+zZtbW+fGbsmg0+LuURdiUfYw9xrlo9VmaNSAwwmkJMZi0AVGlPq4CVGrAKcAaFTAkOQkvD55FGIYjIk6FX9hFJECSVOpAnBenA76aC3SXyxEN30MctNT8JdJWbjslS9h8jELul8XA7ZNvQYxOq17TXKcTovJQ3uJBmSHU0BxRQ3GX9QdQFML/sl1O937LqcY4xAXpYHJ5vBZbmujHY0OJ2b/6yes2LSv2fmb/r/8jBXlZ3x3T7u67x1CU+7t2f/6CS9fP9Lne4ioYxiQKSL5SlOpBvD3W0dj46+VeGvbQZyyNOWcrjTV461tB/H+j4dRL7bOCE2B/Pf9k/HsNz/jiz3lKK0xu7dPnD1+sLul6u19A88/N0bcsgVfWh3YzGuHALy+ZT/io6X7ea/ccQiLrh3OLmuiTsRfF0UkX+O5aUkGTBzQA0+v3+X1vb6CMQAIAN7cesDjuebrmYd0974uWQAw/9/FmD5uEOwOBz7a+WtglREhlmWrvefaf/IMMnp2keycROSJs6wpIvnbWtDXxhAdUVhSjjV3X4Gos7sntfTXbfvRf1EBBr24Fr+ZxJcshcKS7/aEughEisYWMkUsbzOZc9NTsCQnEzaHE930Mag0ec8R3V5lNSYUn6iBQ/C+xqqDuyV2qn/995jHxDYikhZ/WRSxms9kbrmRhFajRm56Ct7a5nsmdVs5BODmld9Dr9NK2qUcDHUNdhw+VYf05CS/rw1kcw4i8sRfCkWclsEiTqf1umvRihtG4fPiMtF0mu1lbnQCkHFTuA1a/ls2X9vdfDIbk4kQ+ceATBHDV7DwtpOTVqPG0aduwLA/f4H9J+u8nLGJe82uWgWnU0Bakh45g1Ngdwh4w8e6YX2UBvUOh6y6qfVaNax2p9fbhYRoLfp2abpxEfu3dAqCx1Kr5pPZ7kjllBUiX/gLoYjRMlWkK1i4NnaoMtXjy1/KsfFQJUoqqmGx2RGj06JoxnVISRRPQeles+sUIADIHtgTL18/Eg+M7u+zPOZGBwQZBWMA6NctQbTtfmdWP3f3s9i/5codh7y+t7CkHDX1dpRUVLv/bYnIE1vIFBF8JQJZU1KGbw9UoOTEGY/nDdFa/CGrH56/OgNXXpSM9348HNDfWlNShhdzMmF3+o620Wrg/IQ4lPnYdzjYiiu8b7ARpVZh7lWXAPD9byk2Ln6k2oTcNQdQ72jqMYiP1uKurH5YmjvC5z7TRJGEAZkigq9EIGJ5rE0NdqzYtA8rdxyC2WZHfLQWgiDAZHPgvNgonLY2en3f8Vorej67CnmDvS+rcmlwAtZG31m3gk1sf41Gp4BH1xbh3VtH4/CpOpQFsCtUS/WOc2evO/tvq1apRPeZJoo0vDUlRagy1eO7AydQ1WyZksV2ros0MSZKdEtDf+oa7HAKTf/vSlspFoxdauvteK/If2KPKnMDNO0qVfD959AJPPzpduT87VvRbu22ZgcrKC5j9zXRWWwhU1irt9kxevl6FJ+ohsMJaNRA+gVGZKV2xT93HYH57MU+TqcR3Ys41OTVRhZXVm3B61t8b25xV1Y/qFUq94YW/pTXmFFRa/U6yx3g8imKLPyGU1gbvXy9RxpKhxPYXVGD3S3GQi02Bw5UmYJdPEVRq1VwiGwa3StJj0nNZqzfM+pCZLz0hWgXuEuKUe91z+a2zognUgIGZApbVaZ6FJ9onROaOodYMFYDKLxnvEfCkL5d4tHTGIvyGt+7SuUNObfPdPPW8JPrdnpsrNF8+RTHnEmpGJAp7Lgu3PtPnpHVGl6lU8N7OpPu8bHoHn+ulVtvs2Ps8vU+g7FrlvWSnMxWreGUxDhUW21e31dYUo4F2cPYfU2KxG81hY2WF+4eCbFQQXxmMElL7N7neJ0Vo/6yzt2lfNkrX7YaMgAADYCbh/fGo1cMQv9uie6gOr1gh+c2kz6WgZXVmHyOOROFMwZkChst9wcuP+O7O5SCx9WlbLXZvQZjoGny2rJJWehqiHE/52tNszc9E/WwNtq5yQUpEmdHUFjwdeHW8lssG2tKSn0e31F6yuOxr/Xh3lRbGzBs6RcY8mIhphfsgJ1jFqQgvJRRWPB14bbzmiwbv5m9j/26dDNEez7WRyMuyvtKbL1Og1RjHDSqc+ubXWvCW6Y9JVICBmQKC8kJsdCzizLs2R2COxGI3eHE+Ne+didbaclsc0ClAm7O6A1jrM7rawpLyplYhBSDVzgiCprRK9a7Z1jbHU6PNeTelFZb8GH1EdHjnORFSsKATGGhotYKk8jGBRReXHms9brAk4ZqRJKSpBoNXhOLEIUjdllTWEhOiEWqjy0QKfyYRbqqvRFLSpKbnsLZ1qQYDMgUFuJ0Wlg4eyti9TLq8eBl/dE7yQCNCuidZMDUsQOwJCcz1EUjkgxvLUmWLDY7yutsGHh2venPx6pQZW4IdbFIQvE6DeoCbCVPGpKKZXlZeKGNm01UmepRXFGDIclGj/XPRHLEgEyy4pGNq9qMlA3HYLE7GYwV6PbMftBqVPj7Dwc9ZlpHqVWIidLAYrMj1WhAbnqKuyUcp9MGNIHL2y5gQ7onYfOfrkYMu7hJpiTtsnY6nXjmmWdw8803Y8qUKTh69KiUp6cI4MrGdaTaDCeA0jNWBmOFmnr5AKhVqlbLnhqdAuoa7Lg9sy+KH8vBsrysNu/w5NoFzJU3xOEEdh2vxujl66UqPpHkJA3I//73v2Gz2fDxxx/j0UcfxeLFi6U8PSlcW9MoUvjqnWRAl7hofF4sntnr+0OV7Tq3r13Aik/UoMpU367zEnU2SftuioqKMHbsWABARkYGSkpKpDy9X3fccQd0Ou8JBJTEZrMpsp52p4DTZyzgilLls0VHYdzHc1Hj4/M+DSDr43nQqlVtOne93YG4OvGgO3rdYsRoA19y1VZK/X22FCn1fOedd4L2tyQNyCaTCQaDwf1Yo9HAbrdDqxX/M1IHbZvNd+o+pVBiPQUB0KgAB7dvUjStGojTCKg2+26palSAo7ERzrbF46Yvki8OOxocDjgEARqVCqq2nj8ASvx9ehMp9SwqCk6KVkkDssFggNl8Lt+w0+n0GYwBID09HdHR0T5fE6gPPvgAmZnKXwZRVFSk2Hq23IqPlCk12YiDIrtCuUwdOwDL8rLadf7MpV94zQJ2SQ8jrujX3b2FZ5pR7942sq3j1GKU/PtsjvVsn4aGBtGGqKRjyMOHD8eGDRsAALt27UL//v2lPD1FgCU5mcjokRTqYlAn+9lHMI6P1uLhMRd3aI3x5j9djYweSdCc7e7WqFXI6JGEMX3OPzdpkJtUkMxI2kK+6qqrsHnzZtxyyy0QBAELFy6U8vQUAWqsNlTWcZ/jSKUCsOlPVyM9uWM3ZTE6LYoevc5jHXKcTov0Fwu9vr6wpBwLsocx6xeFlKTfPrVajeeee07KU1KEcK0//nT3UVT4mJBDytYryYC+XaSb1tfVEIPxF3UHAByqqhPdwpObVJAc8HaQQs5is+OhT7fjvR8Ph7ooFGLXDe7Zaa3U5IRYpBn1OFLdOihzkwqSA+ayppCxO5yYXrADg19Yw2BMnS5Op0VueqrXY9ykguSAAZlCxpWVq7TGEuqikEx8secYLLbO22ZzSU4mpo4dwE0qSJZ4S0ghwaxc5E1nj+VqNWosy8vCguxhbdqkgigY+E2kkKiotaLUy1geRbZgjeUGukkFUTCxy5qCzu5w4i/f/xdqfvuoBY7lUiTjJZGCbtbaIry2Zb97Jx4ioCkhyF1ZfTt1DJlIzngrSkEV6rHjOK0KFjuTZctRXYMdI/68Dr2SpE9nSRQO+G2noKqotYomZ+hs+UNTcfelTOcqZwKYzpIiFwMyBVViTBSS40OTgCFKo8HS3BEey156JekRF8WfgRwVlpSz+5oiCrusKShcqTELS8pwrDY0uao3HT4Jm8PZatnL7LNj2iQvTGdJkYZNAwoKVxIQb2kLg+XYGTMqzt4MuJa9xOm0WJaXhaHJxpCVi7xjOkuKNAzI1Ok6cyKXQacJ+LXdE2K9XuC1GjV+mH4tHrisP5LjY6EC0D0+GundjYhSe+5ePyQ5EUO6J3a02BSAy/udH+oiEAUVu6yp03XWRK4otRq3Z/ZFbX0j/rnziN/XTxqcKrrGVatRY8XkUZg3cah7u76uhhhYbHb890QNTpoakJXWBV0NMbA7nJhWsAPv/XgIZptD4lpFHhWAwd0TUVdvR/kZM2KjtIAKeP/Hw9hwqNLrjGuLzc5MW6Q4/CZTp/O1y05HNDqdeHPrAUzJ7OP3tRk9krAsL8vrsSpTPXYdO43VPx/FN/sqUFpjRprx3NKbEWldPV7vCt4Ls4fhkYId+P5QJcprzOieEItjZ7iXc1sJAEpOnEF6dyNiLBqYmk3kcs24BoBleVkecxFafk5cIkXhjgGZOp1rlx3XhTVQMWqgPoDkIZ8Xl0GjhtdEI2oVcPfIC/Hq5FGtLtj1NjtGL1+P4hPVrd7bMhB4kxCrw7u3jna31hJjojDqL+v83ngYorUwNXD2cEslJ2pEjxWWlGNB9jA8uW6nx/cokM+JKFzwlpKCovkuO4G6Mi3B/R5fX1STzS6a9ev+3/XHmzf9zmvrafTy9dh1vHUwbk5s6Y3FZsehqjpYbHb3BLGuhhjR7aumc60AABeKSURBVP2App2F7hzBTFTtUVZjwuFTdaJzEbhEipSALWQKCq1GjSU5mbA7nFi9+yh+Mzf4fH1CtBYzRyRj3O+aligdPlWHnLe/RWl1YFs1atQq3HfpRT67qYtPVPs9T8ulN/66TF3b+BWWlKOsxoRUowHZA3vg4bEDkGrUw+5w4vPiUtT5aCHHR2t9Ho9EqcamGzmxuQhcIkVKwIBMQTOrDet9/zDyQhh0Ta3aOJ0W6clJyEtPC7zbWxAwfdwg0XHF4oqagHJp63Vaj5nZruVbLi27TMW297PY7Dh8qg4vfFviN9g6nfJN7ema1W4K8mS23PQU9O0SLzoXgUukSAnYZU1BEejSJxWABy/r73XD+Jaby6cZ9YiP9n5P6e8CPSTZiEDmADUPjb7q0LLL1NWNrdOo8cjnP6DHvFUY+tIX+PCnI37/prnRAX0blnN1NjWaMprdOaIvNjx0NSyN3oOxRgXcOaIveicZoPL6ivbJ6JGEJTmZ7rkI3nCXKFICfoMpKAJd+nTf7y7C8smjvB7z1voUy7Ll7wLd1RCDwecb8bOPiUQAYG6wu7tCfdVBrMt01toirNi0z+ff8KatAU0Fz5sHqfRK0qPwnvHo2yXe3dL31Up9dfIoWGx2DFv6BY4HmJEtzahH9sCe+KDokNeWd421ETaHU3RIIDc9xesNHFG4YQuZgsJfDutUYxymjh2AV64f6fdccToteiXp8eS6nfjXf8sBNI0Zq9AUQKaOHRDQBfofd4zx+5q0pHMtbdfyLe/lb90it9js+Ly41O/f8MZsc2DA+QnQBBiZh/QwYurYAaI9BgnRWsRp2/5zn5SeivTkJPfNTSCt1DP1jThR5z8Ya1TA7cN7Y/fM6zDjikGiLW/XhK5DVXXu1KfFj+Xgl8fzUPxYjnuoIBDNJ+MRyQ1byNSpAslhfeeIvnh18ih3C+xotdnveGDLsVzH2XHX7IE9A17+0rtLPHon+V4f3byl7Wv5lrcWeUWtFeU1gU1Ca8mg02Lvb7V+X6dRqzCkuxGb/3Q1YnRaPDtxaKu10ZMGp2JZXhZsDicOn6rDgapa3Pj3DX7PfUmyUXToABBvpQa67twhAP/46Qi66GOwIHuY6HvidFrkvP0tymssHpPo2jKBi+uXKRwwIFOnahk4m+ud5Hkhn16ww+OCeWm3aKzMcLa6YPoay/3yl+PupUj++Aqw8dFa/L+RF7YKSG3pMk1OiEVPYyzKa9qeLKTe7nvG2XlxOqy85XcY2et8dDXEuJ9vuTa6eSYrrUaN9OQk9A3gRgQATlka3F3FzYlNXHNp67pz1xpjsffUNdjdE+Hau+7Y32Q8IjngrSF1Gl+Bs2diLLZPuwYLsofhaLUZ0wt2uDefcApNF8yP9p/2uiduIGO5gWo5Ucw1ean06cleu0Jdwah5l6mrDs27Qe0OJ55ctxNnrI2if/umjDTRnoBGp++AfNpiwzcHKj2CcXPNN8/wdszXemmXE7VWHD5VJ3rc19+YM35wwBcX12fWlkl7bVl33JbJeEShxBYydRpfgfNErRWz1hZhw6FKHK02Qy1y9Xa1nppf9H11ibZ1+Yu/1p4Y1zi2WDeor56BKLUK9156ER4eOwDpLxYGXNaWvP3bBMrVon93+wHUiSxhcghAzt++Rd6QNDw7cShOmhsC/vcZ/8Y3CGBVGYBzn1nLz8La2DQ5zJu2rDtuz2Q8olBgQKZO4ytw6nVavPfjYfdjsTXB3i6YbR3LDYSrtdcWYt2gjQ4n1v1yTPR9jU4BWo0aqUa96L9PIMlBOhJMXDOWG+wOvL39AMR6yEtrLHhl4168+8NBmM/OsBYbe3Wtta6x2HDgpP/xb5eWn5nrs/A3ozvQGy8pb+CIOhO7rKnT+OoaDXSJjtgFs2X3Zu8kQ8Czq6XgrxvU3xIv13vF/n3uyuqHqWMHoFeS91ndQMeDyay1RXhzq3gwbq6uwe4eSnhl416PoQS7w+mx1nrca1/DEcAH3Mvoe0a8VOuOuX6ZwgW/idSpvE2Curzf+Xi/WevYF7ELZnu7mqXiqxu0otaCboYYVJrqRd9fXmN2j5v+Vvkbtp20tZokptWosSB7GB76dLtHb4JLR4JJR/eobt5d3pa11ioAP8+8FtFRUQF9Zm1ddyy2LSPXL1M4YECmTuUtcALAhkOVXrsQNWoVIAhINRpwaTcdZo8fjO8OnHDvT9xSe7qapeCrGzQtyYCrLu6Ot7YdFH1/ilHvHjedMaI7Bg4ZKjpj+a2bfgdjrE7SYNLRPapd3eWJMVH4eOevAb9vaI8kDEo+L+DXB3rj5W9ZU6hv4IgCwW8kBUXLwCk2BnzfpRdh+rhBiNaoMO6VfyFt/qdwOAGNGhjSPcm93jbU/I1jL8nJxI7SU9h13PsGFpPSU72Om3rTGcGko3tUpyTq8ef/7MH7RYdhDiCvtUatQr8EHTb/6ep2/T1/N16BLmsK1Q0cUSA4hkwhsSh7GDJ6JDW1iNF0wc7okYTF2cOwYtNeXLiwAEdqbe7JXg4nsOt4NUYvXx/CUnvyNY6t1aixfVo2/vd3F3nkpU6I1uLhMRe3q3Xra5lRe84lNq6a0SPJXSexZUdJcTq8sfWA32DcMzEWX/xxPI7PvREfZPfrlJspLmsipQh9U4Mi0px1Oz1ajw6ngF3HqzHuta9FW5UAUHyiBlWmetH1t8Hkr+Wq1ajx2o2X4qXcEe71vK6c0HKwJCcTTkHAyh2H3DO646O1GNP3fDx/dQZOmhvQTR+NuV/t9uguv2ZgD3xxNmWpP5Mv6YVrBqYAAI76eJ3Y2G8guKyJlEIeVwaKKL5aNMV+NntwOAUUV9Rg/EXdO6No7eKvG9S1faTcaDVqqFUqj+VVdQ12rNi0D2qVyt3V2/Kmo6LWijf8bKNp0GnwBy+ZzlqSIqUllzWRUrDLmoLu8Kk6lImMXTr87AWsUaswJNnYGcWKOO3ZTjLu7P7QqcY40fMmJ8Ti0JM34OXrR/oNqq6x3+YZ2louq/KHy5pIKRiQqdO03FnH7nBiesEO5PztW9EsTq4xZTFDunufbU1t194UpHE6LfKGpImeN39or4A+IynHfkO9Lp1ICrx1JMmJdUM6BcHvetUh3Y1ex5BVaFoy095ZutRaR7p6vY0/J0RrcWdWv4CDoJRjv1zWRErAbyxJTmwJitiMXaBpU4dJ6alYlD0Mc9btRGFJOUqrTeiRGIes1C5YccNIdE8U7yaltutIClKtRo2Xrx+JRdcOb/eEtc4Y++WyJgpnDMgkqSpTPT7d7X0+rVhuZjWAwnvGuyc+uVo632zZgasuy2JLpxN1NINVRyasdUZOcqJwxm88ScLVTf3p7qM41obtD4GmzFZ9u3i2auJ0WqTE63hR7mSh7uplSkuic3i1I0n42m7QJSFai1ovrWS2hoLLteY3MSYKZ+ob3UE4FF29ob4hIJITfvOpwwLdqODOrH5Qq1RsDYWIqxdjTXEpjtZYoFE3ZUBzjd+3Ze2v1Dj2S8SATBLwt1FBz8RYTL6kl8cORmwNBV/LXgxXWtKjInmfvelIRi0i8o2/KOowX7NleybE4acZ13qsS2VrKPhqrTa8+4P47lOA55aKLUmRUYuIfOMviTrMV6akyUPTmMhDBh4p2CE6y93FVzIQKTJqEZFvDMgkCWZKki+LzY7vDp7w+zqxtb/cTYkoONhlTZLgbFn5qqi14tgZi9/Xic12525KRMEh2RVTEARcfvnl6N27NwAgIyMDjz76qFSnpzDB8WH58TXGDwBpSXHIS08T7c3gbkpEwSFZQC4tLcXgwYPxxhtvSHVKCmOcjSsfvjJi3TmiL16dPMrnZ9SWjFr83InaT7JfzJ49e1BZWYkpU6YgJiYGc+bMQd++faU6PYUJzsaVJ18ZsQL5XPxl1OLnTtRxKkEQfG9A68WqVauwcuVKj+eeeeYZnDp1Ctdccw1+/PFHLFq0CJ9++qnoORoaGlBSUtL2EpOs/fnHE/ho/+lWz9/S/zzMGNE9BCWi5urtTlRZ7egaq0WMtu2BUuz9/NyJ2iY9PR3R0dEez7WrhZyfn4/8/HyP56xWKzQaDQBgxIgRqKyshCAIUKl872/rrVDtVVRUhMxM5c/qlWs9LTY7tn7pfWOJbSdtGDhkaJu6MeVaT6mFez0D/dzDvZ6BYj2VRep6+mqMStaXtGLFCneree/evejRo4ffYEzK0t4N7ym88XMnkoZkY8j33XcfZs2ahe+//x4ajQaLFi2S6tQUJjgbNzLxcyeShmQBOTExEX/961+lOh2FIe5vG5n4uRNJg78UkpQc9rfl0pvgk8PnThTueLUiSYUyYxeX3oQOM7URdRx/MdQpQpGxq+X2gkfasK0gSYOZ2ojaj80GUgRugEBE4Y4BmRSBS2+IKNwxIJMiuJbeeMOlN0QUDhiQSRFcS2+84dIbIgoHvEqRYnDpDRGFMwZkUgwuvSGicMarFSkOl94QUTjiGDIREZEMMCATERHJAAMyERGRDDAgExERyQADMhERkQwwIBMREckAAzIREZEMMCATERHJAAMyERGRDDAgK5TFZsehqjruAxwB+FkTKQNTZyqM3eHErLVFKCwpQ2mNGWlGPXLTU7EkJxNaDe+/lISfNZGyMCArzKy1RXhl41734yPVZvfjZXlZoSoWdQJ+1kTKwttoBbHY7FhTUub1WGFJObs0FYSfNZHyMCArSEWtFWU1Zq/HympMqKi1BrlE1Fn4WRMpDwOygiQnxCLNqPd6LNVoQHJCbJBLRJ2FnzWR8jAgK0icTovc9FSvx3LTUxCn45QBpeBnTaQ8/NUqzJKcTABN44hlNSakGg3ITU9xP0/Kwc+aSFkYkBVGq1FjWV4WFmQPQ0WtFckJsWwtKRQ/ayJl4a9XoeJ0WvTrGh/qYlAQ8LMmUgaOIRMREckAAzIREZEMMCATERHJAAMyERGRDDAgExERyQADchiqtzu53R4RkcJw2VMYcW23t+qng6i07OV2e0RECsKAHEa43R4RkXKxWRUmuN0eEZGyMSCHCW63R0SkbAzIYYLb7RERKRsDcpjgdntERMrGq3gYcW2rt+qng/jNYud2e0RECsKAHEZc2+3dmAx07zeA2+0RESkIr+ZhKEar5nZ7REQKwzFkIiIiGWBAJiIikgEGZCIiIhnoUED+5ptv8Oijj7of79q1C/n5+bjllluwYsWKDheOiIgoUrQ7IM+fPx9Lly6F0+l0Pzd37lwsXboU//znP7F7927s2bNHkkISEREpXbsD8vDhwzFv3jz3Y5PJBJvNhrS0NKhUKowZMwZbt26VooxERESK53fZ06pVq7By5UqP5xYuXIjs7Gxs377d/ZzJZILBYHA/1uv1KCvzvhlCcyUlJW0pr19FRUWSnk+uWE9lYT2VhfVUlmDV029Azs/PR35+vt8TGQwGmM3nNj8wm81ISEjw+7709HRER0f7fV0gioqKkJmp/KxVrKeysJ7Kwnoqi9T1bGhoEG2ISjbL2mAwICoqCqWlpRAEAZs2bcKIESOkOj0REZGiSZqp69lnn8XMmTPhcDgwZswYDB06VMrTExERKVaHAvKoUaMwatQo9+OMjAx88sknHS4UERFRpGFiECIiIhlgQCYiIpIBBmQiIiIZYEAmIiKSAQZkIgIAWGx2HKqqg8VmD3VRiCKSpMueiCj82B1OzFpbhMKSMpTWmJFm1CM3PRVLcjKh1fCenShYGJCJItystUV4ZeNe9+Mj1Wb342V5WaEqFlHE4e0vUQSz2OxYU+I953xhSTm7r4mCiAGZKIJV1FpRVmP2eqysxoSKWmuQS0QUuRiQiSJYckIs0ox6r8dSjQYkJ8QGuUREkYsBmSiCxem0yE1P9XosNz0FcTpOMyEKFv7aiCLckpymreUKS8pRVmNCqtGA3PQU9/NEFBwMyEQRTqtRY1leFhZkD0NFrRXJCbFsGROFAH91RASgqfu6X9f4UBeDKGJxDJmIiEgGGJCJiIhkgAGZiIhIBhiQiYiIZIABmYiISAYYkImIiGSAAZmIiEgGQrYOWRAEAIDNZpP0vA0NDZKeT65YT2VhPZWF9VQWKevpinmuGNicSvD2bBDU1dVh//79ofjTREREIdW/f3/Ex3sm4glZQHY6nTCbzYiKioJKpQpFEYiIiIJKEAQ0NjZCr9dDrfYcNQ5ZQCYiIqJzOKmLiIhIBhiQiYiIZIABmYiISAYYkImIiGRAEfsh19XVYdasWTCZTGhsbMTjjz+OYcOGYdeuXViwYAE0Gg3GjBmDhx9+ONRF7bBvvvkG69evx9KlSwFAkXV0Op2YN28e9u3bB51Oh/nz56NXr16hLpakdu/ejZdeegnvv/8+jh49iscffxwqlQoXXXQR5s6d22r2ZbhpbGzEE088gWPHjsFms+GBBx7AhRdeqLh6OhwOPPXUU/j111+h0WiwaNEiCIKguHq6nDp1CjfccAPeeecdaLVaRdYzLy/PvRwpJSUFN998c/CusYICvPzyy8K7774rCIIgHDp0SMjLyxMEQRByc3OFo0ePCk6nU/jjH/8olJSUhLCUHff8888LEydOFKZNm+Z+Tml1FARB+Oqrr4TZs2cLgiAIO3fuFO6///4Ql0haf/3rX4XrrrtOyM/PFwRBEP73f/9X2LZtmyAIgvD0008LX3/9dSiLJ4nVq1cL8+fPFwRBEE6fPi2MGzdOkfX85ptvhMcff1wQBEHYtm2bcP/99yuynoIgCDabTXjwwQeFCRMmCAcPHlRkPevr64VJkyZ5PBfMa2z4384A+MMf/oBbbrkFQNMda3R0NEwmE2w2G9LS0qBSqTBmzBhs3bo1xCXtmOHDh2PevHnux0qsIwAUFRVh7NixAICMjAyUlJSEuETSSktLw/Lly92P9+zZg5EjRwIALr/8cmzZsiVURZPM1VdfjUceecT9WKPRKLKev//97/H8888DAI4fP46uXbsqsp4A8MILL+CWW27B+eefD0CZ39u9e/fCarXi7rvvxp133okdO3YE9RobdgF51apVuO666zz+d+TIEcTExODkyZOYNWsWZsyYAZPJBIPB4H6fXq9HXV1dCEseOG91/Pnnn5Gdne2RRCWc6+hLy3ppNBrY7fYQlkhaEydOhFZ7brRIEAT356qUz1Cv18NgMMBkMmHq1KmYNm2aIusJAFqtFrNnz8bzzz+PiRMnKrKen332Gc477zz3jTKgzO9tTEwM7rnnHrz99tt49tlnMWfOHMTGxrqPd3Y9w24MOT8/H/n5+a2e37dvH2bMmIHHHnsMI0eOhMlkgtlsdh83m81ISEgIZlHbTayOLRkMhrCtoy8t6+V0Oj0CmNI0H3dTymcIABUVFXjooYdw2223IScnB0uWLHEfU1I9gabW48yZM3HTTTd55D1WSj0//fRTqFQqbN26Fb/88gtmz56N06dPu48rpZ59+vRBr169oFKp0KdPH8THx6OmpsZ9vLPrGXYtZG8OHjyIRx55BEuXLsW4ceMANF3Uo6KiUFpaCkEQsGnTJowYMSLEJZWWUus4fPhwbNiwAUDTpLX+/fuHuESda9CgQdi+fTsAYMOGDYr4DKuqqnD33Xdj1qxZuPHGGwEos54FBQV48803AQCxsbFQqVRIT09XXD3/8Y9/4IMPPsD777+PgQMH4oUXXsDll1+uuHquXr0aixcvBgBUVlbCarUiLi4uaNdYRaTOfOCBB7Bv3z707NkTQFOgev3117Fr1y4sXLgQDocDY8aMwfTp00Nc0o7bvn07PvroIyxbtgwAFFlH1yzr/fv3QxAELFy4EP369Qt1sSRVXl6OGTNm4JNPPsGvv/6Kp59+Go2Njejbty/mz58PjUYT6iJ2yPz58/Hll1+ib9++7ueefPJJzJ8/X1H1tFgsmDNnDqqqqmC323HvvfeiX79+ivs8m5syZQrmzZsHtVqtuHrabDbMmTMHx48fh0qlwsyZM6FWq4N2jVVEQCYiIgp3iuiyJiIiCncMyERERDLAgExERCQDDMhEREQywIBMREQkAwzIREREMsCATEREJAMMyERERDLw/wEQKI/X6WpR9AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x396 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# residual plot\n",
    "Xplot = plt.scatter(pred, (pred - yTest), c='b')\n",
    "\n",
    "plt.hlines(y=0, xmin= -20, xmax=50)\n",
    "\n",
    "plt.title('Residual plot')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test different subsets of input features to see which subset(s) may produce the best results in the predictive models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, I ran a function to see which of the PCA components are the best features.  I ran various basic regression models using those variables to see how well they did compared to the baseline linear regression model with all variables.  The top PCA components did not result in good baseline regression models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, I created subsets of the features and ran baseline linear regression models with these subsets to try to identify a smaller set of variables to use model evaluation instead of using all the features.  Below are the best performing of those subsets in baseline linear regression models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7129487498094862\n"
     ]
    }
   ],
   "source": [
    "# create a subset of the df that can be used for train and validation data splitting\n",
    "\n",
    "subset1 = df.loc[:, ['portfolio_cash',\n",
    " 'portfolio_stocks',\n",
    " 'portfolio_bonds',\n",
    " 'portfolio_others',\n",
    " 'portfolio_preferred',\n",
    " 'portfolio_convertable']]\n",
    "\n",
    "# separate data into training and validation \n",
    "S1Train, S1Test, yTrain_S1, yTest_S1 = train_test_split(subset1, y, test_size =0.3, random_state=11)\n",
    "\n",
    "# Instantiate the linear regression model using default parameters\n",
    "olsS1 = LinearRegression()\n",
    "\n",
    "# Fit the model with training data\n",
    "olsS1.fit(S1Train, yTrain_S1)\n",
    "\n",
    "# evaluate the baseline with the r-squared score\n",
    "print(olsS1.score(S1Test, yTest_S1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.4788733496452161\n"
     ]
    }
   ],
   "source": [
    "# create a subset of the df that can be used for train and validation data splitting\n",
    "\n",
    "subset2 = df.loc[:, [\n",
    " 'rating_aaa',\n",
    " 'rating_aa',\n",
    " 'rating_a',\n",
    " 'rating_bbb',\n",
    " 'rating_bb',\n",
    " 'rating_b',\n",
    " 'rating_below_b',\n",
    " 'rating_others']]\n",
    "\n",
    "# separate data into training and validation \n",
    "S2Train, S2Test, yTrain_S2, yTest_S2 = train_test_split(subset2, y, test_size =0.3, random_state=11)\n",
    "\n",
    "# Instantiate the linear regression model using default parameters\n",
    "olsS2 = LinearRegression()\n",
    "\n",
    "# Fit the model with training data\n",
    "olsS2.fit(S2Train, yTrain_S2)\n",
    "\n",
    "# evaluate the baseline with the r-squared score\n",
    "print(olsS2.score(S2Test, yTest_S2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9603314014662799\n"
     ]
    }
   ],
   "source": [
    "# create a subset of the df that can be used for train and validation data splitting\n",
    "\n",
    "subset3 = df.loc[:, [\n",
    " 'fund_return_1month',\n",
    " 'category_return_1month',\n",
    " 'fund_return_3months',\n",
    " 'fund_return_1year',\n",
    " 'category_return_1year',\n",
    " 'fund_return_3years',\n",
    " 'category_return_3years',\n",
    " 'fund_return_5years',\n",
    " 'category_return_5years',\n",
    " 'fund_return_10years',\n",
    " 'category_return_10years',\n",
    " 'fund_return_2018',\n",
    " 'category_return_2018',\n",
    " 'fund_return_2017',\n",
    " 'category_return_2017',\n",
    " 'fund_return_2016',\n",
    " 'category_return_2016',\n",
    " 'fund_return_2015',\n",
    " 'category_return_2015']]\n",
    "\n",
    "# separate data into training and validation \n",
    "S3Train, S3Test, yTrain_S3, yTest_S3 = train_test_split(subset3, y, test_size =0.3, random_state=11)\n",
    "\n",
    "# Instantiate the linear regression model using default parameters\n",
    "olsS3 = LinearRegression()\n",
    "\n",
    "# Fit the model with training data\n",
    "olsS3.fit(S3Train, yTrain_S3)\n",
    "\n",
    "# evaluate the baseline with the r-squared score\n",
    "print(olsS3.score(S3Test, yTest_S3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.838066638842735\n"
     ]
    }
   ],
   "source": [
    "# create a subset of the df that can be used for train and validation data splitting\n",
    "\n",
    "subset4 = df.loc[:, [\n",
    " 'category_return_1month',\n",
    " 'category_return_1year',\n",
    " 'category_return_3years',\n",
    " 'category_return_5years',\n",
    " 'category_return_10years',\n",
    " 'category_return_2018',\n",
    " 'category_return_2017',\n",
    " 'category_return_2016',\n",
    " 'category_return_2015']]\n",
    "\n",
    "# separate data into training and validation \n",
    "S4Train, S4Test, yTrain_S4, yTest_S4 = train_test_split(subset4, y, test_size =0.3, random_state=11)\n",
    "\n",
    "# Instantiate the linear regression model using default parameters\n",
    "olsS4 = LinearRegression()\n",
    "\n",
    "# Fit the model with training data\n",
    "olsS4.fit(S4Train, yTrain_S4)\n",
    "\n",
    "# evaluate the baseline with the r-squared score\n",
    "print(olsS4.score(S4Test, yTest_S4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9562559679065663\n"
     ]
    }
   ],
   "source": [
    "# create a subset of the df that can be used for train and validation data splitting\n",
    "\n",
    "subset5 = df.loc[:, [\n",
    " 'fund_return_1month',\n",
    " 'fund_return_3months',\n",
    " 'fund_return_1year',\n",
    " 'fund_return_3years',\n",
    " 'fund_return_5years',\n",
    " 'fund_return_10years',\n",
    " 'fund_return_2018',\n",
    " 'fund_return_2017',\n",
    " 'fund_return_2016',\n",
    " 'fund_return_2015']]\n",
    "\n",
    "# separate data into training and validation \n",
    "S5Train, S5Test, yTrain_S5, yTest_S5 = train_test_split(subset5, y, test_size =0.3, random_state=11)\n",
    "\n",
    "# Instantiate the linear regression model using default parameters\n",
    "olsS5 = LinearRegression()\n",
    "\n",
    "# Fit the model with training data\n",
    "olsS5.fit(S5Train, yTrain_S5)\n",
    "\n",
    "# evaluate the baseline with the r-squared score\n",
    "print(olsS5.score(S5Test, yTest_S5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8108853227418291\n"
     ]
    }
   ],
   "source": [
    "# create a subset of the df that can be used for train and validation data splitting\n",
    "\n",
    "subset6 = df.loc[:, [\n",
    " 'fund_mean_annual_return_3years',\n",
    " 'category_mean_annual_return_3years',\n",
    " 'fund_mean_annual_return_5years',\n",
    " 'category_mean_annual_return_5years',\n",
    " 'category_mean_annual_return_10years']]\n",
    "\n",
    "# separate data into training and validation \n",
    "S6Train, S6Test, yTrain_S6, yTest_S6 = train_test_split(subset6, y, test_size =0.3, random_state=11)\n",
    "\n",
    "# Instantiate the linear regression model using default parameters\n",
    "olsS6 = LinearRegression()\n",
    "\n",
    "# Fit the model with training data\n",
    "olsS6.fit(S6Train, yTrain_S6)\n",
    "\n",
    "# evaluate the baseline with the r-squared score\n",
    "print(olsS6.score(S6Test, yTest_S6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7895946634427098\n"
     ]
    }
   ],
   "source": [
    "# create a subset of the df that can be used for train and validation data splitting\n",
    "\n",
    "subset7 = df.loc[:, [\n",
    " 'fund_standard_deviation_3years',\n",
    " 'category_standard_deviation_3years',\n",
    " 'fund_standard_deviation_5years',\n",
    " 'category_standard_deviation_5years',\n",
    " 'category_standard_deviation_10years']]\n",
    "\n",
    "# separate data into training and validation \n",
    "S7Train, S7Test, yTrain_S7, yTest_S7 = train_test_split(subset7, y, test_size =0.3, random_state=11)\n",
    "\n",
    "# Instantiate the linear regression model using default parameters\n",
    "olsS7 = LinearRegression()\n",
    "\n",
    "# Fit the model with training data\n",
    "olsS7.fit(S7Train, yTrain_S7)\n",
    "\n",
    "# evaluate the baseline with the r-squared score\n",
    "print(olsS7.score(S7Test, yTest_S7))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.23751658729364955\n"
     ]
    }
   ],
   "source": [
    "# create a subset of the df that can be used for train and validation data splitting\n",
    "\n",
    "subset8 = dfDumm.loc[:, [\n",
    " 'investment_Blend',\n",
    " 'investment_Growth',\n",
    " 'investment_Value']]\n",
    "\n",
    "# separate data into training and validation \n",
    "S8Train, S8Test, yTrain_S8, yTest_S8 = train_test_split(subset8, y, test_size =0.3, random_state=11)\n",
    "\n",
    "# Instantiate the linear regression model using default parameters\n",
    "olsS8 = LinearRegression()\n",
    "\n",
    "# Fit the model with training data\n",
    "olsS8.fit(S8Train, yTrain_S8)\n",
    "\n",
    "# evaluate the baseline with the r-squared score\n",
    "print(olsS8.score(S8Test, yTest_S8))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.10695349234587892\n"
     ]
    }
   ],
   "source": [
    "# create a subset of the df that can be used for train and validation data splitting\n",
    "\n",
    "subset9 = dfDumm.loc[:, [\n",
    " 'size_Large',\n",
    " 'size_Medium',\n",
    " 'size_Small']]\n",
    "\n",
    "# separate data into training and validation \n",
    "S9Train, S9Test, yTrain_S9, yTest_S9 = train_test_split(subset9, y, test_size =0.3, random_state=11)\n",
    "\n",
    "# Instantiate the linear regression model using default parameters\n",
    "olsS9 = LinearRegression()\n",
    "\n",
    "# Fit the model with training data\n",
    "olsS9.fit(S9Train, yTrain_S9)\n",
    "\n",
    "# evaluate the baseline with the r-squared score\n",
    "print(olsS9.score(S9Test, yTest_S9))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 190,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.21552238336802754\n"
     ]
    }
   ],
   "source": [
    "# create a subset of the df that can be used for train and validation data splitting\n",
    "\n",
    "subset10 = dfDumm.loc[:, [\n",
    " 'median_market_cap',\n",
    " 'fund_return_1month']]\n",
    "\n",
    "# separate data into training and validation \n",
    "S10Train, S10Test, yTrain_S10, yTest_S10 = train_test_split(subset10, y, test_size =0.3, random_state=11)\n",
    "\n",
    "# Instantiate the linear regression model using default parameters\n",
    "olsS10 = LinearRegression()\n",
    "\n",
    "# Fit the model with training data\n",
    "olsS10.fit(S10Train, yTrain_S10)\n",
    "\n",
    "# evaluate the baseline with the r-squared score\n",
    "print(olsS10.score(S10Test, yTest_S10))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}