3.02_Regression_Algorithms.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"id":"zeD8WGmObs_I"},"source":["### <b>**Regression** </b>"]},{"cell_type":"markdown","metadata":{"id":"H9j_qmmEbzuy"},"source":["### **Agenda**"]},{"cell_type":"markdown","metadata":{"id":"vOoy8vQqb-f9"},"source":["In this lesson, we will cover the following concepts with the help of a business use case:\n","* Use Case: Regression\n","* Regression Algorithms\n","* Types of Model Evaluation Metrics\n","* Gradient Descent\n","* Types of Gradient Descents\n","* Use Case: Stochastic Gradient Descent (SGD)\n"]},{"cell_type":"markdown","metadata":{"id":"Po9OHdbTohLr"},"source":["### **Use Case: Regression**\n","\n","Note: At first, with the help of a use case, we are going to perform all the basic steps to reach the model training and predicition part.\n","  "]},{"cell_type":"markdown","metadata":{"id":"bQKjQddGolUL"},"source":["#### <b> **Problem Statement:** </b>"]},{"cell_type":"markdown","metadata":{"id":"2nbWDeSHonx7"},"source":["Google Play Store team is about to launch a new feature wherein, certain apps that are promising are boosted in visibility. The boost will manifest in multiple ways including higher priority in recommendations sections (“Similar apps”, “You might also like”, “New and updated games”). These will also get a boost in search results visibility.  This feature will help bring more attention to newer apps that have the potential."]},{"cell_type":"markdown","metadata":{"id":"-JlzR6UYbmkg"},"source":["#### <b> **Analysis to be done:** </b>\n","\n","The problem is to identify the apps that are going to be good for Google to promote. App ratings, which are provided by the customers, are always great indicators of the goodness of the app. The problem reduces to: predict which apps will have high ratings."]},{"cell_type":"markdown","metadata":{"id":"oNYZuDx9op8z"},"source":["#### <b>**Dataset**</b>"]},{"cell_type":"markdown","metadata":{"id":"8ExUfIJNotUZ"},"source":["Google Play Store data (**googleplaystore.csv**)\n","\n","Link: https://www.dropbox.com/sh/i06ohrau3ucfgbm/AACeYXumL56543KnDNQFlj8ma?dl=0\n"]},{"cell_type":"markdown","metadata":{"id":"iMeGdaAwotkY"},"source":["#### <b> **Data Dictionary:**</b>"]},{"cell_type":"markdown","metadata":{"id":"4YUU25rPowJf"},"source":["|Variables|Description|\n","|:-|:-|\n","|App| Application name|\n","|Category|Category to which the app belongs|\n","|Rating|Overall user rating of the app|\n","|Reviews|Number of user reviews for the app|\n","|Size|Size of the app|\n","|Installs|Number of user downloads/installs for the app|\n","|Type|Paid or Free|\n","|Price|Price of the app|\n","|Content Rating|Age group the app is targeted at - Children / Mature 21+ / Adult|\n","|Genres|An app can belong to multiple genres (apart from its main category)<br>For example, a musical family game will belong to Music, Game, Family genres|\n","|Last Updated|Date when the app was last updated on Play Store|\n","|Current Ver|Current version of the app available on Play Store|\n","|Android Ver|Minimum required Android version|"]},{"cell_type":"markdown","metadata":{"id":"Kr90ro50ozGI"},"source":["#### <b> **Solution:**</b>"]},{"cell_type":"markdown","metadata":{"id":"W14jX_v34LoZ"},"source":["#### <b> **Import Libraries**</b>"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"6MAzISPa4L97"},"outputs":[],"source":["#Importing libraries\n","import pandas as pd\n","import numpy as np\n","import matplotlib.pyplot as plt, seaborn as sns\n","%matplotlib inline"]},{"cell_type":"markdown","metadata":{"id":"kTrPpBFR4YZm"},"source":["#### <b> **Import and Check Dataset**</b>"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"kcE49r1x4fL8"},"outputs":[],"source":["inp0 = pd.read_csv(\"./googleplaystore.csv\")"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"4Aq-fRZv4fg5"},"outputs":[],"source":["# Check first five rows\n","inp0.head(2)"]},{"cell_type":"markdown","metadata":{"id":"xW16LHo8z8ii"},"source":["#### <b> Observations </b>\n","The data will be displayed on the screen."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"lHP7mep35QZl"},"outputs":[],"source":["#Check number of columns and rows, and data types\n","inp0.info()"]},{"cell_type":"markdown","metadata":{"id":"2Kh77c_4iYZe"},"source":["#### <b> **Check Data Types**</b>"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"GQ9Xdj6NiYZf"},"outputs":[],"source":["#checking datatypes\n","inp0.dtypes"]},{"cell_type":"markdown","metadata":{"id":"RdC9Cp5A59Zs"},"source":["#### <b> **Finding and Treating Null Values**</b>"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"PUPGM_kN5S5P"},"outputs":[],"source":["#Finding count of null values\n","inp0.isnull().sum(axis=0)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"x63VpabT6DPm"},"outputs":[],"source":["#Dropping the records with null ratings\n","#This is done because ratings is our target variable\n","inp0.dropna(how ='any', inplace = True)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"x8J8PBEd6aOG"},"outputs":[],"source":["inp0.isnull().sum(axis=0)"]},{"cell_type":"markdown","metadata":{"id":"bWju26fEULXA"},"source":["#### <b> Handling the Variables </b>"]},{"cell_type":"markdown","metadata":{"id":"UWKyYsF7iYZj"},"source":["**1. Clean the price column**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"z0Km-RkWiYZk"},"outputs":[],"source":["#Cleaning the price column\n","inp0.Price.value_counts()[:5]"]},{"cell_type":"markdown","metadata":{"id":"GsFTB2GkiYZm"},"source":["#### <b> Observations </b>\n","Some have dollars, some have 0\n","* We need to conditionally handle this.\n","* First, let's modify the column to take 0 if value is 0, else take the first letter onwards."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"39s073WbiYZm"},"outputs":[],"source":["#Modifying the column\n","inp0['Price'] = inp0.Price.map(lambda x: 0 if x=='0' else float(x[1:]))"]},{"cell_type":"markdown","metadata":{"id":"jA16xDwviYZp"},"source":["**The other columns with numeric data are:<br>**\n","1. Reviews\n","2. Installs\n","3. Size"]},{"cell_type":"markdown","metadata":{"id":"OeoT3_56iYZp"},"source":["**2. Convert reviews to numeric**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"UK1BOY4iiYZp"},"outputs":[],"source":["#Converting reviews to numeric\n","inp0.Reviews = inp0.Reviews.astype(\"int32\")"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"oyFJKcmniYZr"},"outputs":[],"source":["inp0.Reviews.describe()"]},{"cell_type":"markdown","metadata":{"id":"ai6T8UdaiYZu"},"source":["**3. Handle the installs column**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"DE_b23nRiYZu"},"outputs":[],"source":["#Handling the installs column\n","inp0.Installs.value_counts()"]},{"cell_type":"markdown","metadata":{"id":"2xTkBcWtiYZw"},"source":["We'll need to remove the commas and the plus signs."]},{"cell_type":"markdown","metadata":{"id":"y8SDpiyAiYZw"},"source":["<b> Defining function for the same </b>"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"ILohI9zHiYZx"},"outputs":[],"source":["def clean_installs(val):\n","    return int(val.replace(\",\",\"\").replace(\"+\",\"\"))"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"j4B6bJfFiYZz"},"outputs":[],"source":["inp0.Installs = inp0.Installs.map(clean_installs)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"X_mgD2ONiYZ1"},"outputs":[],"source":["inp0.Installs.describe()"]},{"cell_type":"markdown","metadata":{"id":"J1lIXPfviYZ3"},"source":["**4. Handle the app size field**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"lENw90neiYZ4"},"outputs":[],"source":["#Handling the app size field\n","def change_size(size):\n","    if 'M' in size:\n","        x = size[:-1]\n","        x = float(x)*1000\n","        return(x)\n","    elif 'k' == size[-1:]:\n","        x = size[:-1]\n","        x = float(x)\n","        return(x)\n","    else:\n","        return None"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"eG9fjJn8iYZ6"},"outputs":[],"source":["inp0[\"Size\"] = inp0[\"Size\"].map(change_size)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"232kUvlliYZ8"},"outputs":[],"source":["inp0.Size.describe()"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"BMJQPESkiYZ-"},"outputs":[],"source":["#Filling Size which had NA\n","inp0.Size.fillna(method = 'ffill', inplace = True)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"GNUrivb0iYaA"},"outputs":[],"source":["#Checking datatypes\n","inp0.dtypes"]},{"cell_type":"markdown","metadata":{"id":"yF-L5Vs-iYaC"},"source":["#### **Sanity checks**"]},{"cell_type":"markdown","metadata":{"id":"sRUvEtMpiYaD"},"source":["1. Average rating should be between 1 and 5, as only these values are allowed on Play Store. Drop any rows that have a value outside this range."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"wWqz6ARViYaD","scrolled":true},"outputs":[],"source":["#Checking the rating\n","inp0.Rating.describe()"]},{"cell_type":"markdown","metadata":{"id":"6Ze0cfTyZgB4"},"source":["#### <b> Observations </b>"]},{"cell_type":"markdown","metadata":{"id":"HNY_wVsziYaF"},"source":["Min is 1 and max is 5. None of the values have rating outside the range."]},{"cell_type":"markdown","metadata":{"id":"L2o5-NgkiYaG"},"source":["2. Reviews should not be more than installs as only those who installed can review the app."]},{"cell_type":"markdown","metadata":{"id":"Q6qjc2SciYaG"},"source":["Checking if reviews are more than installs. Counting total rows like this."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"pCCYfKDfiYaG"},"outputs":[],"source":["#Checking and counting the rows\n","len(inp0[inp0.Reviews > inp0.Installs])"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"trhR-Sq-iYaJ"},"outputs":[],"source":["inp0[inp0.Reviews > inp0.Installs]"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"anu7J6FEiYaL"},"outputs":[],"source":["inp0 = inp0[inp0.Reviews <= inp0.Installs].copy()"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"wbkBvVLOiYaN"},"outputs":[],"source":["inp0.shape"]},{"cell_type":"markdown","metadata":{"id":"o7mV1Ug0iYaT"},"source":["3. For free apps **(Type == “Free”)**, the price should not be **> 0**. Drop any such rows."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"PCWSrL1jiYaT"},"outputs":[],"source":["len(inp0[(inp0.Type == \"Free\") & (inp0.Price>0)])"]},{"cell_type":"markdown","metadata":{"id":"MytS2idhiYaV"},"source":["#### **EDA**"]},{"cell_type":"markdown","metadata":{"id":"1IUocNLqiYaW"},"source":["#### <b> Box Plot: Price</b>"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"L2joMoc-iYaW"},"outputs":[],"source":["#Are there any outliers? Think about the price of usual apps on the Play Store.\n","sns.boxplot(inp0.Price)\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"LBUZ1sI9iYaY"},"source":["#### <b> Box Plot: Reviews</b>"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Y4W5mSiviYaY"},"outputs":[],"source":["#Are there any apps with very high number of reviews? Do the values seem right?\n","sns.boxplot(inp0.Reviews)\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"vW9AgNBVa1Wc"},"source":["#### **Checking Distribution and Skewness:**\n","\n","How are the ratings distributed? Is it more toward higher ratings?"]},{"cell_type":"markdown","metadata":{"id":"ndvWSN3RiYaa"},"source":["##### **Distribution of Ratings**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Xw7CH8UliYab"},"outputs":[],"source":["#Distributing the ratings\n","inp0.Rating.plot.hist()\n","#Show plot\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"4wZHlXf-iYad"},"source":["##### **Histogram: Size**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"ib13K1wSiYad"},"outputs":[],"source":["inp0['Size'].plot.hist()\n","#Show plot\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"f7vcIm-pZgB8"},"source":["#### <b> Observations </b>\n","A histogram is plotted with ratings on the x-axis and frequency on the y-axis, and the ratings are distributed."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"18slR9triYaf"},"outputs":[],"source":["#Pair plot\n","sns.pairplot(data=inp0)"]},{"cell_type":"markdown","metadata":{"id":"PXV4d2U6iYah"},"source":["##### **Outlier Treatment:**\n"]},{"cell_type":"markdown","metadata":{"id":"LCxDNFiDdwln"},"source":["##### **1. Price:** \n","\n","From the box plot, it seems like there are some apps with very high prices. A price of $200 for an application on the Play Store is very high and suspicious.\n","Check the records that have very high price:\n","Is 200 a high price?"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Cx1sY6HCiYah","scrolled":true},"outputs":[],"source":["#Checking the records\n","len(inp0[inp0.Price > 200])"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"dNN02OKgiYaj"},"outputs":[],"source":["inp0[inp0.Price > 200]"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"CpqkCkmeiYal"},"outputs":[],"source":["inp0 = inp0[inp0.Price <= 200].copy()\n","\n","inp0.shape"]},{"cell_type":"markdown","metadata":{"id":"UvbcViJ2d8-Y"},"source":["##### **2. Reviews:**\n","\n","Very few apps have very high number of reviews. These are all star apps that don’t help with the analysis and, in fact, will skew it. Drop records having more than 2 million reviews."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"U8uOf23NiYao"},"outputs":[],"source":["#Dropping the records with more than 2 million reviews\n","inp0 = inp0[inp0.Reviews <= 2000000]\n","inp0.shape"]},{"cell_type":"markdown","metadata":{"id":"HXtw_jNiiYar"},"source":["##### **3. Installs:**\n","\n","There seem to be some outliers in this field too. Apps having a very high number of installs should be dropped from the analysis.\n","Find out the different percentiles – 10, 25, 50, 70, 90, 95, 99.\n","\n","Decide a threshold as the cutoff for outliers and drop records having values more than the threshold.\n","\n","\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"bHCK_AwdiYar"},"outputs":[],"source":["#Dropping the apps that have a very high number of installs\n","inp0.Installs.quantile([0.1, 0.25, 0.5, 0.70, 0.9, 0.95, 0.99])"]},{"cell_type":"markdown","metadata":{"id":"i2uTG0sNZgB9"},"source":["#### <b> Observations </b>"]},{"cell_type":"markdown","metadata":{"id":"ak3aontjiYau"},"source":["Looks like there are just 1% of apps having more than 100M installs. These apps might be genuine, but will definitely skew our analysis.  \n","We need to drop these.\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"h36NFnV8iYau"},"outputs":[],"source":["#Dropping the apps with more than 100M installs\n","len(inp0[inp0.Installs >= 1000000000])"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"cfD2ZFf9iYax"},"outputs":[],"source":["inp0 = inp0[inp0.Installs < 1000000000].copy()\n","inp0.shape"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"hrZkGODGiYa0"},"outputs":[],"source":["#Importing warnings\n","import warnings\n","warnings.filterwarnings(\"ignore\")"]},{"cell_type":"markdown","metadata":{"id":"0p7aaAfYiYa2"},"source":["#### **Bi-variate Analysis:**"]},{"cell_type":"markdown","metadata":{"id":"eWEdZlW4fo2U"},"source":["Let’s look at how the available predictors relate to the variable of interest, i.e., our target variable rating. Make scatter plots (for numeric features) and box plots (for character features) to assess the relationhips between rating and the other features."]},{"cell_type":"markdown","metadata":{"id":"z3a_GTWViYa2"},"source":["##### **1.\tMake scatter plot/join plot for Rating vs. Price**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"SEtsOwwIiYa3"},"outputs":[],"source":["#What pattern do you observe? Does rating increase with price?\n","sns.jointplot(inp0.Price, inp0.Rating)"]},{"cell_type":"markdown","metadata":{"id":"jTntjoDiiYa6"},"source":["##### **2.\tMake scatter plot/joinplot for Rating vs Size**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"jYfuh8YRiYa6"},"outputs":[],"source":["#Are heavier apps rated better?\n","sns.jointplot(inp0.Size, inp0.Rating)"]},{"cell_type":"markdown","metadata":{"id":"d5YUE0-yiYa8"},"source":["##### **3.\tMake scatter plot/joinplot for Rating vs Reviews**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"sAnz2OOxiYa9"},"outputs":[],"source":["# Does more review mean a better rating always?\n","sns.jointplot(inp0.Reviews, inp0.Rating)"]},{"cell_type":"markdown","metadata":{"id":"TUNw3cfbiYbA"},"source":["##### **4.\tMake boxplot for Rating vs Content Rating**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"ByNz7YM6iYbB"},"outputs":[],"source":["#Is there any difference in the ratings? Are some types liked better?\n","plt.figure(figsize=[8,6])\n","sns.boxplot(inp0['Content_Rating'], inp0.Rating)"]},{"cell_type":"markdown","metadata":{"id":"cTsKCO1MgKNv"},"source":["##### **5. Make boxplot for Ratings vs. Category**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"uW4cbujciYbC"},"outputs":[],"source":["#Which genre has the best ratings?\n","plt.figure(figsize=[18,6])\n","g = sns.boxplot(inp0.Category, inp0.Rating)\n","plt.xticks(rotation=90)"]},{"cell_type":"markdown","metadata":{"id":"K84OIeq5iYbE"},"source":["#### **Pre-processing the Dataset**"]},{"cell_type":"markdown","metadata":{"id":"1QwhHCpPiYbF"},"source":["##### **1. Make a copy of the dataset**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"vi9bn-ypiYbF"},"outputs":[],"source":["#Making a copy\n","inp1 = inp0.copy()"]},{"cell_type":"markdown","metadata":{"id":"7wdlD-_EkusJ"},"source":["##### **2. Apply log transformation (np.log1p) to Reviews and Installs**\n","\n","Reviews and Installs have some values that are still relatively very high.\n","Before building a linear regression model, you need to reduce the skew."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Psvvu5pwiYbI"},"outputs":[],"source":["#Reducing the skew\n","inp0.Installs.describe()"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"wqUaLLD0iYbL"},"outputs":[],"source":["inp1.Installs = inp1.Installs.apply(np.log1p)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"UM2pmr5ziYbN"},"outputs":[],"source":["inp1.Reviews = inp1.Reviews.apply(np.log1p)"]},{"cell_type":"markdown","metadata":{"id":"xsZEsfMKiYbQ"},"source":["##### **3. Drop columns App, Last Updated, Current Ver, and Android Ver**\n","\n"," These variables are not useful for our task."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"YtobZfveiYbQ"},"outputs":[],"source":["inp1.dtypes"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"HGezO1I6iYbS"},"outputs":[],"source":["#Dropping the variables that are not useful for our task\n","inp1.drop([\"App\", \"Last Updated\", \"Current Ver\", \"Android Ver\"], axis=1, inplace=True)\n","inp1.shape"]},{"cell_type":"markdown","metadata":{"id":"srbCi7VwlT2-"},"source":["##### **4. Dummy Columns:**\n","\n","\n","Get dummy columns for Category, Genres, and Content Rating. This needs to be done as the models do not understand categorical data, and all data should be numeric. Dummy encoding is one way to convert character fields to numeric fields. Name of the dataframe should be **inp2**."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"mHqSQ_NuiYbV"},"outputs":[],"source":["inp2 = pd.get_dummies(inp1, drop_first=True)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"w7OQaLh4iYbX"},"outputs":[],"source":["inp2.columns"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"gKS6RUcU3gww"},"outputs":[],"source":["inp2.shape"]},{"cell_type":"markdown","metadata":{"id":"Ko2AVCZriYbY"},"source":["#### **Train-test split**"]},{"cell_type":"markdown","metadata":{"id":"cxAnZejbz8jP"},"source":["Let us distribute the data into **training** and **test** datasets using the **train_test_split()** function."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"JBbvq6Z8iYbZ"},"outputs":[],"source":["from sklearn.model_selection import train_test_split\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"gaBkZtKsiYbb"},"outputs":[],"source":["#?train_test_split"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"GhBFCj3jiYbc"},"outputs":[],"source":["df_train, df_test = train_test_split(inp2, train_size = 0.7, random_state = 100)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"OiPz2HP2iYbe"},"outputs":[],"source":["df_train.shape, df_test.shape"]},{"cell_type":"markdown","metadata":{"id":"0K8Dc4E9iYbf"},"source":["Let us separate the dataframes into **X_train, y_train, X_test, y_test.**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"LZqLkOrliYbf"},"outputs":[],"source":["y_train = df_train.pop(\"Rating\")\n","X_train = df_train"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"rIooKn-21WB8"},"outputs":[],"source":["X_train.head(1)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"ZmlT5RUwiYbh"},"outputs":[],"source":["y_test = df_test.pop(\"Rating\")\n","X_test = df_test"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"H7FoNKI4_XCj"},"outputs":[],"source":["X_test.head(1)"]},{"cell_type":"markdown","metadata":{"id":"J6SlMiJrgoZj"},"source":["### <b> **Regression Algorithms:**</b>\n","\n","Note: Let us take a look at the theory part before moving on to the training and prediction."]},{"cell_type":"markdown","metadata":{"id":"BmvPniXBhEDw"},"source":["#### <b>Types of Regression Algorithms:</b>"]},{"cell_type":"markdown","metadata":{"id":"DSn9GMFhhPFM"},"source":["* Linear regression\n","* Multiple linear regression\n","* Polynomial regression\n","* Ridge regression\n","* Lasso regression\n","* ElasticNet regression"]},{"cell_type":"markdown","metadata":{"id":"ZUsnK-BxjXUH"},"source":["#### **When to use regression?**"]},{"cell_type":"markdown","metadata":{"id":"-hZzR-UsjuMB"},"source":["If target variable is a continuous numeric variable (100–2000), then use a regression algorithm.<br><br>\n","\n","![Use of Reg](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/Use%20of%20Reg.JPG)\n","\n","<br><br>\n","Note: Let us take a look at the basics of linear regression and then move on to the model builiding part where we are going to use all the concepts that we saw in previous sessions."]},{"cell_type":"markdown","metadata":{"id":"yjP1JLXdhn68"},"source":["#### **1. Linear Regression:**"]},{"cell_type":"markdown","metadata":{"id":"qIGWoFPKh9EO"},"source":["Linear Regression is a statistical model used to predict the relationship between independent and dependent variables denoted by x and y respectively.\n"]},{"cell_type":"markdown","metadata":{"id":"ZHjjpch3ibIb"},"source":["![Lin Reg](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/Lin%20Reg.JPG)"]},{"cell_type":"markdown","metadata":{"id":"x7yOduLSlQuq"},"source":["#### **2. Muliple Linear Regression:**"]},{"cell_type":"markdown","metadata":{"id":"dXg3x1S1lT44"},"source":["Multiple linear regression is a statistical technique used to predict the outcome of a response variable through several explanatory variables and model the relationships between them."]},{"cell_type":"markdown","metadata":{"id":"tg1VjZnGlYmU"},"source":["![Multiple LR](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/Multiple%20LR.JPG)"]},{"cell_type":"markdown","metadata":{"id":"d9TqAOj7l6Gt"},"source":["#### **3. Polynomial Regression:**"]},{"cell_type":"markdown","metadata":{"id":"FZoN6CFBl8Ia"},"source":["Polynomial regression is applied when data is not formed in a straight line. \n","It is used to fit a linear model to non-linear data by creating new features from powers of non-linear features.\n","\n","Example: Quadratic features \n"]},{"cell_type":"markdown","metadata":{"id":"SsoRmWzOnlUV"},"source":["![Pol Reg](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/Pol%20Reg.JPG)"]},{"cell_type":"markdown","metadata":{"id":"O1Axf7SUQ8ir"},"source":["### <b> Types of Model Evaluation Metrics:</b>"]},{"cell_type":"markdown","metadata":{"id":"NghG9e0g4EIn"},"source":["#### **Assumption** <br>\n","\n","Let us consider the following:\n","\n","$y_i$ – the observed value\n","\n","$\\bar{y}$ – the mean value of a sample\n","\n","$\\hat{y}_i$ – the value estimated by the regression line\n"]},{"cell_type":"markdown","metadata":{"id":"w37a9h7kJToC"},"source":["#### **Sum of Squares Total (SST)** <br>\n","\n","  The squared variations between the measured dependent variable and its mean are referred to as the **Sum of Squares Total (SST)** or **Total Sum of Squares (TSS)**.\n","\n","  It's similar to the variation of descriptive statistics in that it's the dispersion of measured variables around the mean.\n","\n","  It is a measure of the dataset's overall variability.\n","\n","  $$\n","  SST = SSR + SSE = \\sum_{i=1}^{n}(\\hat{y}_i - \\bar{y}_i)^2 + \\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2\n","  $$\n","\n","  ![SST](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/SST.png)\n","\n","<br/>\n","\n","#### **Sum of Squares due to Regression (SSR)** <br>\n","\n","  The difference between the predicted value and the dependent variable's mean are referred to as the **Sum of Squares due to Regression (SSR)** or **Explained Sum of Squares (ESS)**.\n","\n","  It can be considered as a metric for describing how well our line fits the data.\n","\n","  If the SSR (or ESS) is equal to the SST (or TSS), the regression model is flawless and captures all observed variability.\n","\n","  $$\n","  SSR = \\sum_{i=1}^{n}(y_i - \\bar{y})^2\n","  $$\n","\n","  ![SSR](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/SSR.png)\n","\n","<br/>\n","\n","#### **Sum of Squares Error (SSE)** <br>\n","\n","  The difference between the observed and predicted values are referred to as the **Sum of Squares Error (SSE)** or **Residual Sum of Squares (RSS)**, where `residual` stands for `remaining` or `unexplained`.\n","\n","  This error must be reduced since the smaller it is, the better the regression's estimation power.\n","\n","  $$\n","  SSE = \\sum_{i=1}^{n}(y_i - \\hat{y}_i)^2\n","  $$\n","\n","  $$\n","  Or\n","  $$\n","\n","  $$\n","  SSE = \\sum_{i=1}^{n}e_i^2\n","  $$\n","\n","  $$\n","  where,\n","  $$\n","  \n","  $$\n","  e_i = (y_i - \\hat{y}_i)\n","  $$\n","\n","  ![SSE](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/SSE.png)\n","\n","<br/>\n","\n","#### **Relation Among SST, SSR, and SSE** <br>\n","\n","  Since certain people use these abbreviations in various ways, it can be very confusing.\n","\n","  We use one of two sets of notations for these abbreviations: SST, SSR, and SSE or TSS, ESS, and RSS.\n","\n","  These equations are related in the following ways:\n","\n","  $$\n","  SST = SSR + SSE\n","  $$\n","\n","  $$Or$$\n","\n","  $$\n","  TSS = ESS + RSS\n","  $$\n","\n","  This is because the overall variability of the dataset is equivalent to the variability described by the regression line and the unknown variability (also known as error).\n","\n","  For a constant total variability, a lower error would result in a better regression. A higher error, on the other hand, would result in a weaker regression. This should always be remembered regardless of the notation set used.\n","\n","  ![Connection](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/Connection.png)\n","\n","<br/>\n","\n","#### **R-Square Matrix** <br>\n","\n","  The determination coefficient also known as **R2 (R-squared) score** is used for the performance evaluation of a linear regression model.\n","\n","  R2 displays the proportion of data points inside the regression equation line.\n","\n","  A higher R2 value means improved results.\n","\n","  It is calculated as follows:\n","\n","  $$\n","  R^2 = 1 - \\frac{SSE}{SSR}\n","  $$\n","\n","  $$\n","  Or\n","  $$\n","\n","  $$\n","  R^2 = 1 - \\frac{RSS}{ESS}\n","  $$\n","\n","  The highest possible score is 1, which is achieved when the predicted and actual values are the same.\n","\n","  The R2 score is 0 for a baseline model.\n","\n","  In the worst-case scenario, the R2 score can also be negative.\n","\n","<br/>\n"]},{"cell_type":"markdown","metadata":{"id":"JcMO9bJoQtZ_"},"source":["#### **Import *statsmodels* Library for Linear Regression**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"dxQrGQ4nQt0p"},"outputs":[],"source":["#Importing the statsmodel library\n","import statsmodels.api as sm"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"ZsNHSNlYQ631"},"outputs":[],"source":["#Applying linear regression\n","model1 = sm.OLS(y_train, X_train)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"cNn50TMfQ7FZ"},"outputs":[],"source":["model1 = model1.fit()"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"4lFH1DRYQ7MW"},"outputs":[],"source":["#Finding the summary\n","model1.summary()"]},{"cell_type":"markdown","metadata":{"id":"N084Cy8E7o6M"},"source":["#### **Identified values for *model1*:**"]},{"cell_type":"markdown","metadata":{"id":"kOBM-LSgYvsS"},"source":["* **$R^2$ (uncentered): 0.987**\n","\n","  **Overview of $R^2$ (R-square):**\n","  ![R-square](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/R-square.JPG)"]},{"cell_type":"markdown","metadata":{"id":"SMeoNzybJW2u"},"source":["* **Adjusted $R^2$ (uncentered): 0.987**\n","\n","  **Overview of adjusted $R^2$:**\n","\n","  The disadvantage with R-squared is that it assumes every independent variable in the model and explains variations in the dependent variable.\n","Use adjusted R-squared when working on a multiple linear regression problem.\n","\n","  ![Adj R-square](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/Adj%20R-square.JPG)\n"]},{"cell_type":"markdown","metadata":{"id":"jOPn8cahJ79r"},"source":["* **F-statistic: 3717**\n","  \n","  **Overview of F-statistic:**\n","\n","  The F-test indicates whether the linear regression model provides a better fit to the data in comparison to a model that contains no independent variables.\n","  \n","    Note: In general, the models with high F-statistic are considered as optimum.\n","\n","  An F-test provides 2 values:\n","  * F-critical: This value is also called as F statistic\n","  * F-value: This value is without the “critical” part\n","  \n","    Note: The F-value is always observed along with the p-value. In general, the higher the F-value of a variable, the lesser the p-value.\n"]},{"cell_type":"markdown","metadata":{"id":"skY89YvRP1Iw"},"source":["* **AIC: 8811**\n","\n","* **BIC: 9646**\n","\n","  **Overview of AIC and BIC:**\n","\n","  * AIC and BIC are penalized-likelihood criteria.\n","\n","  * They are used for selecting the best predictor subsets in regression and comparing the nonnested models.\n","\n","  * AIC: \"It is an estimate of a constant plus the relative distance between the unknown true likelihood function of the data and the fitted likelihood function of the model.\" \n","\n","  * BIC: \"It is an estimate of a function of the posterior probability of a model being true, under a certain Bayesian setup.\"\n","\n","\n","    Source: https://www.methodology.psu.edu/resources/AIC-vs-BIC/\n","\n","\n","\n","* **Formula:**\n","\n","  The AIC or BIC for a model is usually written in the form:\n","  \n","  **$[-2logL + kp]$**\n","  \n","  Where: \n","    * L = likelihood function\n","    * p = number of parameters in the model\n","    * k = 2 for AIC and log(n) for BIC\n","\n","\n","* In popular opinion:\n","  * Lower AIC tells that the model is closer to the truth\n","  * Lower BIC tells that the model is more likely to be the true model"]},{"cell_type":"markdown","metadata":{"id":"Dkq-ZXdaHSse"},"source":["#### **Prediction and evaluation using \"model1\"**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Fzq7EnrcC1Ds"},"outputs":[],"source":["#Prediction and evaluation\n","import sklearn.metrics as metrics\n","y_test_pred= model1.predict(X_test)"]},{"cell_type":"markdown","metadata":{"id":"SPi02JkAMRfr"},"source":["**Metrics used for regression models:**\n","<br><br>\n","\n","![RMSE1](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/RMSE1.JPG)\n","\n","\n","Lower values of RMSE indicate better fit."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"-JkJCMsBHWi6"},"outputs":[],"source":["#Assessing the model using RMSE\n","print(np.sqrt(metrics.mean_squared_error(y_test, y_test_pred)))\n","\n","#RMSE of \"model1\" = 0.49203326281981113"]},{"cell_type":"markdown","metadata":{"id":"Ddo_2E0Ja9iO"},"source":["#### **MPE:**\n","\n","Mean Percentage Error measures the errors that indicate whether or not the prediction is biased. \n","\n","Bias is one component of the mean squared error and is measured as the variance of the errors plus the square of the mean error. \n","\n","$$MSE = VAR(E) + (MPE)^2$$\n","\n","Therefore, minimizing the MSE will implicitly minimize the bias and the variance of the errors.\n","\n","\n","MPE is the computed average of percentage errors by which prediction of a model differ from actual values of the quantity being predicted.\n","\n","Formula:\n","\n","$$MPE=\\ \\frac{100\\%}{n}\\ \\sum^{n}_{i=1}\\ \\frac{a_i - p_i}{a_i}$$\n","\n","\n","Where:\n","* $a_i$ = Actual value of the quantity being predicted\n","* $p_i$ = Predicted value\n","* $n$ = Number of different times for which the variable is predicted"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"LImD-epui9UM"},"outputs":[],"source":["#Calculating MPE\n","def MPE(y_test_pred,y_test):\n","    return ( ((y_test - y_test_pred) / y_test).mean()) * 100\n","\n","print ('MPE: ' + str(MPE(y_test_pred,y_test)) )"]},{"cell_type":"markdown","metadata":{"id":"lMITDPOra_sG"},"source":["#### **MAPE:**\n","\n","Mean Absolute Percentage Error measures the prediction accuracy of a model.\n","\n","Formula:\n","\n","$$MAPE\\ =\\ \\frac{1}{n}\\ \\sum^{n}_{i=1}\\ \\frac{|A_i-P_i|}{|A_i|}$$\n","\n","Where:\n","* $A_i$ = Actual value of the quantity being predicted\n","* $P_i$ = Predicted value\n","* $n$ = Number of different times for which the variable is predicted\n","\n","\n","<br><br>\n","Mean Absolute Percentage Error is similar to Mean Absolute Error but is normalized by true observation.\n","\n","The mean absolute percentage error is useful for reporting purpose and is expressed in generic percentage terms.\n","\n","<br><br>\n","Note: In general, the desirable MAPE value is less than 20.\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"SXnEMol7GHBy"},"outputs":[],"source":["#Calculating MAPE\n","def MAPE(y_test_pred,y_test):\n","    return np.mean(np.abs((y_test - y_test_pred) / y_test)) * 100\n","    \n","print ('MAPE: ' + str(MAPE(y_test_pred,y_test)) )"]},{"cell_type":"markdown","metadata":{"id":"u4vAfEDV1fiC"},"source":["#### **Reiterating the Model**"]},{"cell_type":"markdown","metadata":{"id":"U860p5LSXKc9"},"source":["**1. Feature selection using p-value:**"]},{"cell_type":"markdown","metadata":{"id":"apZf_-JaIWd_"},"source":["In regression analysis, p-values and coefficients together indicate which relationships in the model are statistically significant and the nature of those relationships.\n","\n","\n","Coefficients describe the mathematical relationship between each independent variable and the dependent variable.\n","\n","p-values for the coefficients indicate whether these relationships are statistically significant.\n"]},{"cell_type":"markdown","metadata":{"id":"tz7pyGTuXM1Z"},"source":["In general, the threshold for the p-value is taken as $\\leq$ 0.05.\n","\n","\n","Let's find the variable with value $\\leq$ 0.05."]},{"cell_type":"markdown","metadata":{"id":"zMigGqNa2DNR"},"source":["**2. List:**\n","\n","Identified 99 variables that have p-value $\\leq$ 0.05."]},{"cell_type":"markdown","metadata":{"id":"hagJwcIm2LMJ"},"source":["'Rating','Reviews','Installs','Category_AUTO_AND_VEHICLES','Category_BEAUTY','Category_BOOKS_AND_REFERENCE','Category_BUSINESS','Category_COMICS','Category_COMMUNICATION','Category_DATING','Category_EDUCATION','Category_EDUCATION','Category_ENTERTAINMENT','Category_EVENTS','Category_FAMILY','Category_FINANCE','Category_FOOD_AND_DRINK','Category_GAME','Category_HEALTH_AND_FITNESS','Category_HOUSE_AND_HOME','Category_LIBRARIES_AND_DEMO','Category_LIFESTYLE','Category_MAPS_AND_NAVIGATION','Category_MEDICAL','Category_NEWS_AND_MAGAZINES','Category_PARENTING','Category_PERSONALIZATION','Category_PHOTOGRAPHY','Category_PRODUCTIVITY','Category_SHOPPING','Category_SOCIAL','Category_SPORTS','Category_TOOLS','Category_TRAVEL_AND_LOCAL','Category_VIDEO_PLAYERS','Category_WEATHER','Type_Paid','Content_Rating_Everyone','Content_Rating_Everyone_10+','Content_Rating_Mature_17+','Content_Rating_Teen','Content_Rating_Unrated','Genres_Art_&_Design','Genres_Art_&_Design;Creativity','Genres_Art_&_Design;Pretend_Play','Genres_Auto_&_Vehicles','Genres_Beauty','Genres_Board;Brain_Games','Genres_Books_&_Reference','Genres_Business','Genres_Casual','Genres_Casual;Action_&_Adventure','Genres_Casual;Brain_Games','Genres_Casual;Pretend_Play','Genres_Comics','Genres_Comics;Creativity','Genres_Communication','Genres_Dating','Genres_Education','Genres_Education;Action_&_Adventure','Genres_Education;Creativity','Genres_Education;Education','Genres_Education;Pretend_Play','Genres_Educational;Education','Genres_Educational;Pretend_Play','Genres_Entertainment','Genres_Entertainment;Brain_Games','Genres_Entertainment;Creativity','Genres_Entertainment;Music_&_Video','Genres_Events','Genres_Finance','Genres_Food_&_Drink','Genres_Health_&_Fitness','Genres_House_&_Home','Genres_Libraries_&_Demo','Genres_Lifestyle','Genres_Maps_&_Navigation','Genres_Medical','Genres_Music;Music_&_Video','Genres_News_&_Magazines','Genres_Parenting','Genres_Parenting;Education','Genres_Parenting;Music_&_Video','Genres_Personalization','Genres_Photography','Genres_Productivity','Genres_Puzzle','Genres_Puzzle;Brain_Games','Genres_Racing;Action_&_Adventure','Genres_Role_Playing','Genres_Shopping','Genres_Simulation','Genres_Simulation;Action_&_Adventure','Genres_Social','Genres_Strategy','Genres_Tools','Genres_Tools;Education','Genres_Travel_&_Local','Genres_Travel_&_Local;Action_&_Adventure','Genres_Weather'"]},{"cell_type":"markdown","metadata":{"id":"6fO0v8HT2ZGA"},"source":["**3. Create a new dataset with chosen variables and split it into training and testing dataset**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"g6POnvM_bt9n"},"outputs":[],"source":["#Create inp3\n","inp3 = inp2[['Rating','Reviews','Installs','Category_AUTO_AND_VEHICLES','Category_BEAUTY',\n","             'Category_BOOKS_AND_REFERENCE','Category_BUSINESS','Category_COMICS',\n","             'Category_COMMUNICATION','Category_DATING','Category_EDUCATION','Category_EDUCATION',\n","             'Category_ENTERTAINMENT','Category_EVENTS','Category_FAMILY','Category_FINANCE',\n","             'Category_FOOD_AND_DRINK','Category_GAME','Category_HEALTH_AND_FITNESS',\n","             'Category_HOUSE_AND_HOME','Category_LIBRARIES_AND_DEMO','Category_LIFESTYLE',\n","             'Category_MAPS_AND_NAVIGATION','Category_MEDICAL','Category_NEWS_AND_MAGAZINES',\n","             'Category_PARENTING','Category_PERSONALIZATION','Category_PHOTOGRAPHY','Category_PRODUCTIVITY',\n","             'Category_SHOPPING','Category_SOCIAL','Category_SPORTS','Category_TOOLS','Category_TRAVEL_AND_LOCAL',\n","             'Category_VIDEO_PLAYERS','Category_WEATHER','Type_Paid','Content_Rating_Everyone',\n","             'Content_Rating_Everyone_10+','Content_Rating_Mature_17+','Content_Rating_Teen','Content_Rating_Unrated',\n","             'Genres_Art_&_Design','Genres_Art_&_Design_Creativity','Genres_Art_&_Design_Pretend_Play',\n","             'Genres_Auto_&_Vehicles','Genres_Beauty','Genres_Board_Brain_Games','Genres_Books_&_Reference',\n","             'Genres_Business','Genres_Casual','Genres_Casual_Action_&_Adventure','Genres_Casual_Brain_Games',\n","             'Genres_Casual_Pretend_Play','Genres_Comics','Genres_Comics_Creativity','Genres_Communication',\n","             'Genres_Dating','Genres_Education','Genres_Education_Action_&_Adventure','Genres_Education_Creativity',\n","             'Genres_Education_Education','Genres_Education_Pretend_Play','Genres_Educational_Education',\n","             'Genres_Educational_Pretend_Play','Genres_Entertainment','Genres_Entertainment_Brain_Games',\n","             'Genres_Entertainment_Creativity','Genres_Entertainment_Music_&_Video','Genres_Events',\n","             'Genres_Finance','Genres_Food_&_Drink','Genres_Health_&_Fitness','Genres_House_&_Home',\n","             'Genres_Libraries_&_Demo','Genres_Lifestyle','Genres_Maps_&_Navigation','Genres_Medical',\n","             'Genres_Music_Music_&_Video','Genres_News_&_Magazines','Genres_Parenting','Genres_Parenting_Education',\n","             'Genres_Parenting_Music_&_Video','Genres_Personalization','Genres_Photography','Genres_Productivity',\n","             'Genres_Puzzle','Genres_Puzzle_Brain_Games','Genres_Racing_Action_&_Adventure','Genres_Role_Playing',\n","             'Genres_Shopping','Genres_Simulation','Genres_Simulation_Action_&_Adventure','Genres_Social',\n","             'Genres_Strategy','Genres_Tools','Genres_Tools_Education','Genres_Travel_&_Local',\n","             'Genres_Travel_&_Local_Action_&_Adventure','Genres_Weather']]"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"KZl6k3LY2jBF"},"outputs":[],"source":["#Checking datatypes\n","inp3.dtypes"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"hgjEdGCA3oZ5"},"outputs":[],"source":["inp3.shape"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"N1l_0deV30QO"},"outputs":[],"source":["#split imp3\n","df_train, df_test = train_test_split(inp3, train_size = 0.7, random_state = 100)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"mpmHgEDM3-us"},"outputs":[],"source":["df_train.shape, df_test.shape"]},{"cell_type":"markdown","metadata":{"id":"agsTeU4O4FJE"},"source":["**4. Separate the new dataframes into X_train, y_train, X_test, y_test**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"cdW26yXq4FJF"},"outputs":[],"source":["#Separating the new dataframes\n","y_train1 = df_train.pop(\"Rating\")\n","X_train1 = df_train"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"qUjEfkCd4FJH"},"outputs":[],"source":["X_train1.head(1)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"jngnU_1Q4FJI"},"outputs":[],"source":["y_test1 = df_test.pop(\"Rating\")\n","X_test1 = df_test"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"e_yWUBB4cMnA"},"outputs":[],"source":["X_test1.head(1)"]},{"cell_type":"markdown","metadata":{"id":"hafT1CQW21Oh"},"source":["**5. Create the second model**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Obzr0hwT2-Cj"},"outputs":[],"source":["#Apply linear regression\n","model2 = sm.OLS(y_train1, X_train1)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"byvg8JvV3Bx8"},"outputs":[],"source":["model2 = model2.fit()"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"0BlbsGi83FrF"},"outputs":[],"source":["#Find the summary\n","model2.summary()"]},{"cell_type":"markdown","metadata":{"id":"Sf9FDt5V29YS"},"source":["* Rating\tR-squared (uncentered):\t0.987\n","\n","* Adj. R-squared (uncentered): 0.987\n","\n","* F-statistic: 6412\n","\n","* AIC: 8749\n","\n","* BIC: 9234"]},{"cell_type":"markdown","metadata":{"id":"Y469JsvQ93Bv"},"source":["#### **How Well Does the Model Fit the Data?**\n"," ***model2* performs better:**\n","\n","\n","* R-squared value:<br>\n","\n","  The most common way to evaluate the overall fit of a linear model is by the **R-squared** value.<br>\n","\n","  R-squared is between 0 and 1 (or between 0 to 100%), and higher is better because it means that more variance is explained by the model.<br>\n","\n","  In case of our models, model1 and model2 have same R-squared values.\n","\n","* F-statistics:<br>\n","\n","  In case of dataset that only have numerical values, it is said higher the F-statistic better the model.\n","\n","  In case of our models, model2 has more F-statistic value than model1.\n","\n","* AIC and BIC: <br>\n","\n","  In case of models, lesser the AIC and BIC, better the model.\n","\n","  In case of models, model2 has lesser AIC and BIC values than the model1."]},{"cell_type":"markdown","metadata":{"id":"Mzwl9WxZ7Qs2"},"source":["**Prediction using \"model2\"**"]},{"cell_type":"markdown","metadata":{"id":"4dFq4Lgaz8j2"},"source":["Let us perform prediction using model2 by importing **sklearn.metrics**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"tek2QIiiX-UF"},"outputs":[],"source":["#Importing sklearn.metrics\n","import sklearn.metrics as metrics\n","y_test_pred1= model2.predict(X_test1)\n","print(np.sqrt(metrics.mean_squared_error(y_test, y_test_pred1)))\n","\n","\n","#RMSE value of model1 = 0.49203326281981113 "]},{"cell_type":"code","execution_count":null,"metadata":{"id":"msSs5VEjGuBO"},"outputs":[],"source":["#Calculating MAPE\n","def MAPE(y_test_pred1,y_test1):\n","    return np.mean(np.abs((y_test1 - y_test_pred1) / y_test1)) * 100\n","print ('MAPE: ' + str(MAPE(y_test_pred1,y_test1)) )"]},{"cell_type":"markdown","metadata":{"id":"YZI2fnVkLqJ1"},"source":["**model2** shows slight improvement as the RMSE and MAPE value of this model is lesser than that of **model1**."]},{"cell_type":"markdown","metadata":{"id":"OeedFsJUkJpe"},"source":["#### **4. Ridge Regression:**"]},{"cell_type":"markdown","metadata":{"id":"TCe3lNvgkOJm"},"source":["* Ridge Regression (L2) is used when there is a problem of multicollinearity.\n","\n","* By adding a degree of bias to the regression estimates, ridge regression reduces the standard errors.\n","\n","<br>\n","\n","![Ridge Reg](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/Ridge%20Reg.JPG)"]},{"cell_type":"markdown","metadata":{"id":"vLtahPask1A3"},"source":["#### **5. Lasso Regression:**"]},{"cell_type":"markdown","metadata":{"id":"ADAsum1Mk_9c"},"source":["* Lasso Regression (L1) is similar to ridge, but it also performs feature selection.\n","\n","\n","* It will set the coefficient value for features that do not help in decision making very low, potentially zero.\n","\n","  ![Lasso Reg](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/Lasso%20Reg.JPG)\n","\n","* Lasso regression tends to exclude variables that are not required from the equation, whereas ridge tends to do better when all variables are present\n"]},{"cell_type":"markdown","metadata":{"id":"pySCTk23lima"},"source":["#### **6. ElasticNet Regression:**"]},{"cell_type":"markdown","metadata":{"id":"cwBx313Jl0A6"},"source":["![ElasticNet Reg](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/ElasticNet%20Reg.JPG)"]},{"cell_type":"markdown","metadata":{"id":"btWzKyP7mAoK"},"source":["#### **Use Case: Ridge, Lasso, ElasticNet Regression for Training and Prediction:**"]},{"cell_type":"markdown","metadata":{"id":"BUP0dPU-mFYb"},"source":["We are going to use the same dataset that we used in the previous use case at the time of training of \"model2\" i.e. inp3"]},{"cell_type":"markdown","metadata":{"id":"Ht0pkliww9f2"},"source":["**1. Ridge Regression:**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"6xpgqzPAxAzx"},"outputs":[],"source":["#Importing Ridge\n","import sklearn\n","from sklearn.linear_model import Ridge\n","ridgeReg = Ridge(alpha=0.001, normalize=True)\n","ridgeReg.fit(X_train1,y_train1)\n"]},{"cell_type":"markdown","metadata":{"id":"UlQbpZ2Qk9RP"},"source":["**Evaluating using RMSE:**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Cp6hlHIklAWc"},"outputs":[],"source":["print(np.sqrt(sklearn.metrics.mean_squared_error(y_train1, ridgeReg.predict(X_train1))))\n","print(np.sqrt(sklearn.metrics.mean_squared_error(y_test1, ridgeReg.predict(X_test1))))\n","print('R2 Value/Coefficient of Determination: {}'.format(ridgeReg.score(X_test1, y_test1)))\n"]},{"cell_type":"markdown","metadata":{"id":"phjbBvo538Mq"},"source":["**2. Lasso Regression:**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"AMyu1Lk-394b"},"outputs":[],"source":["#Importing Lasso\n","from sklearn.linear_model import Lasso\n","lassoreg = Lasso(alpha=0.001, normalize=True)\n","lassoreg.fit(X_train1,y_train1)\n"]},{"cell_type":"markdown","metadata":{"id":"G7ew0yN_lC9R"},"source":["**Evaluating using RMSE:**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"gqO61XjtlEm2"},"outputs":[],"source":["print(np.sqrt(sklearn.metrics.mean_squared_error(y_train1, lassoreg.predict(X_train1))))\n","print(np.sqrt(sklearn.metrics.mean_squared_error(y_test1, lassoreg.predict(X_test1))))\n","print('R2 Value/Coefficient of Determination: {}'.format(lassoreg.score(X_test1, y_test1)))\n"]},{"cell_type":"markdown","metadata":{"id":"qnjUflT44p_M"},"source":["**3. ElasticNet Regression:**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"ODkIycTW4tMX"},"outputs":[],"source":["#Importing ElasticNet\n","from sklearn.linear_model import ElasticNet\n","Elastic = ElasticNet(alpha=0.001, normalize=True)\n","Elastic.fit(X_train1,y_train1)\n"]},{"cell_type":"markdown","metadata":{"id":"RCCNPFnplH-9"},"source":["**Evaluating using RMSE:**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"ofNarEA0lIir"},"outputs":[],"source":["print(np.sqrt(sklearn.metrics.mean_squared_error(y_train1, Elastic.predict(X_train1))))\n","print(np.sqrt(sklearn.metrics.mean_squared_error(y_test1, Elastic.predict(X_test1))))\n","print('R2 Value/Coefficient of Determination: {}'.format(Elastic.score(X_test1, y_test1)))\n"]},{"cell_type":"markdown","metadata":{"id":"uzlv97mx49Oz"},"source":["### **Exercise:**"]},{"cell_type":"markdown","metadata":{"id":"aTSZPu7c4_OW"},"source":["* Perform the iteration of the model with Lasso, Ridge, and ElasticNet Regression by using the original dataset i.e., **inp0** as done in the case of Linear Regression.\n","\n","* Use the following metrics to evaluate the model:\n","  * RMSE\n","  * MAPE\n","  * R$^2$ error"]},{"cell_type":"markdown","metadata":{"id":"p2Ra1XbqK9bc"},"source":["#### **Cost Function:**\n","- A cost function is a function that evaluates a model's performance for a given dataset.\n","\n","- It evaluates and expresses the error between predicted values and expected values as a single, real number."]},{"cell_type":"markdown","metadata":{"id":"oq0faxmKL_LV"},"source":["### **Gradient:**\n","\n","- A gradient is a measurement of how much a function's output varies as its inputs are changed."]},{"cell_type":"markdown","metadata":{"id":"bXOr6kXwMDtH"},"source":["### **Gradient Descent:**\n","\n","- Gradient descent is an optimization algorithm that is used to find the values of the parameters (coefficients) of a function that minimizes the cost function by iteratively moving in the direction of steepest descent as determined by the gradient's negative.\n","\n","- It's an optimization algorithm to discover the local minimum of a differentiable function or feature.\n","\n","- We use gradient descent to update the parameters of our model. In linear regression, parameters correspond to coefficients, and in neural networks, parameters correspond to weights.\n","\n","- The gradient descent equation is as follows:\n","\n","![GD](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Applied_Machine_Learning/Images/0.7_Supervised_Learning_-_Regression_and_Classification/Trainer_PPT_and_IPYNB/0.2_Linear_Regression/GD.png)\n","\n","### **Types of Gradient Descents:**\n","\n","Gradient descents are divided into three categories, which vary primarily in the amount of data they use. These categories are:\n","\n","1. Batch Gradient Descent (BGD)\n","2. Stochastic Gradient Descent (SGD) \n","3. Mini-Batch Gradient Descent (Mini BGD)\n","\n","\n","#### **1. Batch Gradient Descent (BGD):**\n","\n","- Batch gradient descent (BGD), also known as vanilla gradient descent, measures the error for each example in the training dataset, but the model is updated only after all of the training examples have been evaluated.\n","\n","- This whole process is referred to as a training epoch because it resembles a loop.\n","\n","- For each gradient descent iteration, it processes all the training samples. However, batch gradient descent is computationally very expensive when the number of training examples is high.\n","\n","- Thus, if the number of training examples is high, we tend to use stochastic gradient descent (SGD) or mini-batch gradient descent (Mini BGD) instead.\n","\n","#### **2. Stochastic Gradient Descent (SGD):**\n","\n","- Stochastic gradient descent (SGD) is a form of gradient descent that processes one training example per iteration.\n","\n","- It estimates the error for each example in the training dataset and updates the parameters one by one.\n","\n","- Based on the problem statement, batch gradient descent (BGD) can be much faster because the parameters are modified even after an iteration in which only a single example has been processed.\n","\n","- One advantage is that the frequent updates allow us to track our progress in great detail.\n","\n","- However, even if the number of training examples is high, it can only process one of them, which will add to the system's overhead and the number of iterations needed.\n","\n","#### **3. Mini-Batch Gradient Descent (Mini BGD):**\n","\n","- Mini-batch gradient descent (Mini BGD) combines the principles of stochastic gradient descent (SGD) and batch gradient descent (BGD) and is faster than both.\n","\n","- It divides the training dataset into small batches and updates each of those batches. This establishes a balance between the robustness of stochastic gradient descent (SGD) and the efficiency of batch gradient descent (BGD).\n","\n","- It is compatible with both larger and smaller training examples.\n"]},{"cell_type":"markdown","metadata":{"id":"7Xt9x-WFMKx8"},"source":["### <b>Use Case: Stochastic Gradient Descent (SGD):</b>"]},{"cell_type":"markdown","metadata":{"id":"0w-_L7okMPCA"},"source":["#### <b> Importing Required Libraries </b>"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"kmg_RHS6MMJY"},"outputs":[],"source":["#Importing libraries\n","import numpy as np\n","import pandas as pd\n","\n","from tabulate import tabulate\n","\n","from sklearn.datasets import load_boston\n","from sklearn.preprocessing import StandardScaler\n","from sklearn.model_selection import cross_val_score, train_test_split\n","from sklearn.linear_model import LinearRegression, SGDRegressor\n","from sklearn.metrics import mean_squared_error, mean_absolute_error, explained_variance_score\n","\n","import warnings\n","warnings.filterwarnings(\"ignore\")\n","\n","import matplotlib.pyplot as plt\n","%matplotlib inline"]},{"cell_type":"markdown","metadata":{"id":"8Nigq9_SMUuG"},"source":["#### <b> Load the Data</b>"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"EPc44DKDMbvL"},"outputs":[],"source":["#Load the data dictionary\n","boston = load_boston()\n","\n","#Find the dict keys\n","print(boston.keys())"]},{"cell_type":"markdown","metadata":{"id":"PfA07X1-MgQN"},"source":["#### <b> Dataset Description</b>"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"OzmUllK2MkaT"},"outputs":[],"source":["#To print the boston dataset description\n","print(boston.DESCR)"]},{"cell_type":"markdown","metadata":{"id":"69wj19BtMoRZ"},"source":["#### <b> Find Features and Target</b>"]},{"cell_type":"markdown","metadata":{"id":"g3tSIErbz8kK"},"source":["Let us split the data into **test** and **train.**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"LZ_sG4lqMtUI"},"outputs":[],"source":["X = boston.data\n","Y = boston.target\n","\n","#Splitting the data to test and train\n","x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size=0.3)"]},{"cell_type":"markdown","metadata":{"id":"WPbJFsVoMxMl"},"source":["#### <b>Find Feature's Name</b>"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Tw0lvsyzM4FC"},"outputs":[],"source":["columns = boston.feature_names\n","columns"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"DbgfOT6IM9h7"},"outputs":[],"source":["boston_df = pd.DataFrame(boston.data)\n","boston_df.columns = columns\n","boston_df[\"MEDV\"] = Y"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"iAv9Es_LM_hU"},"outputs":[],"source":["boston_df.head()"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"xNDXo4AjNB4G"},"outputs":[],"source":["print(boston_df.describe())"]},{"cell_type":"markdown","metadata":{"id":"TtfLKygJM6_f"},"source":["#### <b> Standardizing Data</b>"]},{"cell_type":"markdown","metadata":{"id":"ueEJJaynz8kM"},"source":["Let us standardize the **test** and **train** data using the **StandardScaler()** function."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"QY9bk5gqNH-P"},"outputs":[],"source":["scaler = StandardScaler().fit(x_train)\n","x_train = scaler.transform(x_train)\n","x_test = scaler.transform(x_test)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Y85tX7CKNKyq"},"outputs":[],"source":["train_data=pd.DataFrame(x_train)\n","train_data['price']=y_train\n","train_data.head(3)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"M-OtJpnhNNWl"},"outputs":[],"source":["x_test = np.array(x_test)\n","y_test = np.array(y_test)"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"7JyGVKFGNRoi"},"outputs":[],"source":["#Shape of test and train data metrics\n","print(x_train.shape)\n","print(y_train.shape)\n","\n","print(x_test.shape)\n","print(y_test.shape)"]},{"cell_type":"markdown","metadata":{"id":"ZXO5iaEHNUPe"},"source":["#### <b> Linear Regression: Boston Housing Prediction</b>"]},{"cell_type":"markdown","metadata":{"id":"4bZKJusiz8kO"},"source":["Let us calculate **R-Squared**, **Linear Regressor Model Accuracy**, **MAE**, **MSE**, and **RMSE**."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"5jSg_01tNVue"},"outputs":[],"source":["print(\"Linear Regression: Boston Housing Prediction\")\n","lin_reg = LinearRegression()\n","lin_reg.fit(x_train, y_train)\n","lin_score = lin_reg.score(x_train, y_train)\n","print(\"R-squared:\", lin_score)\n","\n","lin_y_pred = lin_reg.predict(x_test)\n","lin_accuracy = explained_variance_score(y_test, lin_y_pred)\n","lin_accuracy = round(lin_accuracy*100, 6)\n","print(\"Linear Regressor Model Accuracy:\", lin_accuracy, \"%\")\n","print()\n","\n","lin_mae = mean_absolute_error(y_test, lin_y_pred)\n","lin_mse = mean_squared_error(y_test, lin_y_pred)\n","lin_rmse = lin_mse*(1/2.0)\n","\n","print(\"MAE:\", lin_mae)\n","print(\"MSE:\", lin_mae)\n","print(\"RMSE:\", lin_rmse)\n","print()"]},{"cell_type":"markdown","metadata":{"id":"WDNKc9zdz8kO"},"source":["Let us plot a graph of the **Actual vs. Predicted Target**."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"lxLV3uEWNVXR"},"outputs":[],"source":["plt.scatter(y_test, lin_y_pred)\n","plt.grid()\n","plt.xlabel('Actual Y')\n","plt.ylabel('Predicted Y')\n","plt.title('Actual vs. Predicted Target')\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"nHLvonh8z8kP"},"source":["Let us plot a graph of the **Test vs. Predicted Data**."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"_FYH7LKaNcHt"},"outputs":[],"source":["x_ax = range(len(y_test))\n","plt.plot(x_ax, y_test, label=\"original\")\n","plt.plot(x_ax, lin_y_pred, label=\"predicted\")\n","plt.title(\"Test vs. Predicted Data\")\n","plt.xlabel('X-axis')\n","plt.ylabel('Y-axis')\n","plt.legend(loc='best',fancybox=True, shadow=True)\n","plt.grid(True)\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"wSXlb3HANhud"},"source":["#### <b> SGD Regression: Boston Housing Prediction</b>"]},{"cell_type":"markdown","metadata":{"id":"j9EOqTdiz8kQ"},"source":["Let us calculate **R-squared**, **SGD Regressor Model Accuracy**, **MAE**, **MSE**, and **RMSE**."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"Udml5S5yNcCk"},"outputs":[],"source":["print(\"SGD Regression: Boston Housing Prediction\")\n","sgd_reg = SGDRegressor()\n","sgd_reg.fit(x_train, y_train)\n","sgd_score = sgd_reg.score(x_train, y_train)\n","print(\"R-squared:\", sgd_score)\n","\n","sgd_y_pred = sgd_reg.predict(x_test)\n","sgd_accuracy = explained_variance_score(y_test, sgd_y_pred)\n","sgd_accuracy = round(sgd_accuracy*100, 6)\n","print(\"SGD Regressor Model Accuracy:\", sgd_accuracy, \"%\")\n","print()\n","\n","sgd_mae = mean_absolute_error(y_test, sgd_y_pred)\n","sgd_mse = mean_squared_error(y_test, sgd_y_pred)\n","sgd_rmse = sgd_mse*(1/2.0)\n","\n","print(\"MAE:\", sgd_mae)\n","print(\"MSE:\", sgd_mse)\n","print(\"RMSE:\", sgd_rmse)\n","print()"]},{"cell_type":"markdown","metadata":{"id":"BL40Dijpz8kU"},"source":["Let us plot a graph of the **Actual vs. Predicted Target**."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"oVBJm8YENb3t"},"outputs":[],"source":["plt.scatter(y_test, sgd_y_pred)\n","plt.grid()\n","plt.xlabel('Actual Y')\n","plt.ylabel('Predicted Y')\n","plt.title('Actual vs. Predicted Target')\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"ppS9W-S8z8kV"},"source":["Let us plot a graph of the **Test vs. Predicted Data**."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"K4C6JNxwNqN4"},"outputs":[],"source":["x_ax = range(len(y_test))\n","plt.plot(x_ax, y_test, label=\"original\")\n","plt.plot(x_ax, sgd_y_pred, label=\"predicted\")\n","plt.title(\"Test vs. Predicted Data\")\n","plt.xlabel('X-axis')\n","plt.ylabel('Y-axis')\n","plt.legend(loc='best',fancybox=True, shadow=True)\n","plt.grid(True)\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"Ys3AYiTsN4s3"},"source":["#### <b> Model Comparison</b>"]},{"cell_type":"markdown","metadata":{"id":"LoRNgEp0N76q"},"source":["**1. Evaluation Matrix Comparison**"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"CxJC83crNqG9"},"outputs":[],"source":["#Performing evaluation matrix comparison\n","model_table = pd.DataFrame(columns = [\"Comparison Matrix\", \"LR Model\", \"SGD Model\"])\n","model_table[\"Comparison Matrix\"] = [\"Accuracy (Variance Score)\", \"R2 Score\", \"MAE\", \"MSE\", \"RMSE\"]\n","model_table[\"LR Model\"] = [lin_accuracy, lin_score, lin_mae, lin_mse, lin_rmse]\n","model_table[\"SGD Model\"] = [sgd_accuracy, sgd_score, sgd_mae, sgd_mse, sgd_rmse]\n","\n","print(tabulate(model_table, headers = 'keys', tablefmt = 'psql', numalign=\"left\"))"]},{"cell_type":"markdown","metadata":{"id":"TJfBD4P_OBXo"},"source":["**2. Prediction Comparison - A: Scatter Plot**"]},{"cell_type":"markdown","metadata":{"id":"2W_seA-wz8kW"},"source":["Let us perform a comparison of the **Actual vs. Predicted Target** for the **Scatter Plot**."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"07tmOmViNp8r"},"outputs":[],"source":["plt.scatter(y_test, lin_y_pred, c=\"b\", marker=\"d\", label='LR')\n","plt.scatter(y_test, sgd_y_pred, c=\"r\", marker=\".\", label='SGD')\n","plt.xlabel('Actual Y')\n","plt.ylabel('Predicted Y')\n","plt.title('Actual Vs. Predicted Target - LR Vs. SGD')\n","plt.legend(loc='best',fancybox=True, shadow=True)\n","plt.grid()\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"ySqRXaJ3OFa_"},"source":["**3. Prediction Comparison - B: Line Graph**"]},{"cell_type":"markdown","metadata":{"id":"T4JNp0Miz8kX"},"source":["Let us perform a comparison of **LR vs. SGD Prediction** for the **Line Graph**."]},{"cell_type":"code","execution_count":null,"metadata":{"id":"xM0XolC4OKgz"},"outputs":[],"source":["x_ax = range(len(sgd_y_pred))\n","plt.plot(x_ax, lin_y_pred, c=\"C0\", linestyle=\"-\", linewidth = 2, label=\"LR Prediction\")\n","plt.plot(x_ax, sgd_y_pred, c=\"C2\", linestyle=\":\", linewidth = 4, label=\"SGD Prediction\")\n","plt.title(\"LR Vs. SGD Prediction\")\n","plt.xlabel('X-axis')\n","plt.ylabel('Y-axis')\n","plt.legend(loc='best',fancybox=True, shadow=True)\n","plt.grid(True)\n","plt.show()"]},{"cell_type":"markdown","metadata":{"id":"JtbFaaGOOOeU"},"source":["**4. Prediction Comparison - C: Target Prediction Table**"]},{"cell_type":"markdown","metadata":{"id":"tdIIxseez8kY"},"source":["Let us compare the **LR Predicted value** and **SGD Predicted value**.\n"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"v-2xvfpgOPfK"},"outputs":[],"source":["prediction_table = pd.DataFrame(columns=[\"LR Predicted Value\", \"SGD Predicted Value\"])\n","prediction_table[\"LR Predicted Value\"] = lin_y_pred\n","prediction_table[\"SGD Predicted Value\"] = sgd_y_pred\n","\n","print(tabulate(prediction_table, headers = 'keys', tablefmt = 'psql', numalign=\"left\"))"]},{"cell_type":"markdown","metadata":{"id":"M58DVrEtZhNw"},"source":["**Note: In this topic, we saw the use of the linear regression methods, but in the next topic we will be working on \"Logistic Regression\".**"]},{"cell_type":"markdown","metadata":{"id":"wWnaEd6tqhcn"},"source":["![Simplilearn_Logo](https://labcontent.simplicdn.net/data-content/content-assets/Data_and_AI/Logo_Powered_By_Simplilearn/SL_Logo_1.png)"]}],"metadata":{"colab":{"collapsed_sections":[],"name":"3.02_Regression_Algorithms.ipynb","private_outputs":true,"provenance":[]},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.6"}},"nbformat":4,"nbformat_minor":0}