07. Advanced SVMs - Python.ipynb

{
  "cells": [
    {
      "metadata": {
        "collapsed": true
      },
      "cell_type": "markdown",
      "source": "Exercise 7 - Advanced Support Vector Machines\n=====\n\nSupport vector machines let us predict catergories. In this example we will be looking at practically using SVMs by formatting data correctly, visualising the SVM model and then evaluating the SVM model.\n\nWe will be looking at __prions__ - misfolded proteins that are associated with several fatal neurodegenerative diseases (kind of like Daleks, if you have seen Doctor Who). Looking at examples of proteins mass and weight, we will build a predictive model to detect prions in blood samples.\n\n#### Run the code below to set up the graphing features for this notebook."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this code!\n# It sets up the graphing configuration\nimport warnings\nwarnings.filterwarnings(\"ignore\")\nimport matplotlib.pyplot as graph\n%matplotlib inline\ngraph.rcParams['figure.figsize'] = (15,5)\ngraph.rcParams[\"font.family\"] = 'DejaVu Sans'\ngraph.rcParams[\"font.size\"] = '12'\ngraph.rcParams['image.cmap'] = 'rainbow'",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 1\n-----\n\nLets load up the data first, and save it temporarily as rawData. Our dataset is called \"PrionData.csv\".\n\n#### Replace `<addPathToData>`  with `'Data/PrionData.csv'` and then __Run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import pandas as pd\nimport numpy as np\n\n###\n# REPLACE <addPathToData> BELOW WITH 'Data/PrionData.csv' (INCLUDING THE QUOTES) TO LOAD THE DATA FROM THAT FILE \n###\nrawData = pd.read_csv(<addPathToData>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 2\n-----\n\nLets take a look at the data.\n\n#### In the cell below replace the text `<printDataHere>` with `print(rawData.head())` and then __Run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <printDataHere> with print(rawData.head()) TO VIEW THE TOP 5 DATA POINTS OF THE DATA SET\n###\n<printDataHere>\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Looks like we have an extra column, this happens regularly when exporting data sets from a program like Excel and then importing them into a dataframe.\n\nStep 3\n-----\n\nLets get rid of that extra column, and then check that it's gone.\n\n#### __Run the code__ below."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this box to remove the extra column.\ndataset = rawData.drop(['Unnamed: 0'], axis = 1)\nprint(dataset.head())",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "All gone!\n\nStep 4\n-----\n\nLet's graph the data set to better understand what we're working with.\n\nLooking at the output of the last step we can see the 'categories' we're looking at is called __prion_status__ (the label).\n\n### In the cell below replace:\n#### 1. `<addMass>` with `'mass'`\n#### 2. `<addWeight>` with `'weight'`\n#### then __run the code__.\n"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE THE <addMass> BELOW WITH 'mass' (INCLUDING THE QUOTES)\n###\nX = dataset[<addMass>]\n###\n\n##\n# REPLACE THE <addWeight> BELOW WITH 'weight' (INCLUDING THE QUOTES)\n###\nY = dataset[<addWeight>]\n###\n\n# This makes a list that says which items are prions and which are not\ntarget = dataset['prion_status'] == 'prion'\n\ngraph.scatter(X, Y, c = target, zorder = 10, s = 40)\n\ngraph.title(\"Classification plot for prion data\")\ngraph.ylabel(\"Mass\")\ngraph.xlabel(\"Weight\")\n\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 5\n-------\n\nLet's split up our data into test and training sets. We'll start by checking the total number of instances in our dataset by using the DataFrame attribute *shape*. The first number is the one we want.\n\n#### In the cell below replace `<addShape>` with `shape` and then __Run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE THE <addShape> BELOW WITH THE NAME OF THE ATTRIBUTE WE WANT TO LOOK AT - shape\n###\ndataset.<addShape>\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 6\n-----\n\nStep 5 has told us that we have nearly 500 data points. We'll use 400 examples for our training set, and the remainder for our test set.\n\n#### Replace the `<add400>` below with `400` and run the cell."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# This makes our training set out of the first 400 examples\ntrain_X = dataset.drop(['prion_status'], 1).truncate(after = 399)\ntrain_Y = dataset['prion_status'].truncate(after = 399)\n\n###\n# REPLACE THE <add400> BELOW WITH 400 TO MAKE THE TEST SET OUT OF THE REMAINING EXAMPLES\n###\ntest_X = dataset.drop(['prion_status'], 1).truncate(before = <add400>).reset_index(drop = True)\ntest_Y = dataset['prion_status'].truncate(before = <add400>).reset_index(drop = True)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 7\n-----\n\nWell done! Lets look at a summary of our training data.\n\n#### In the cell below replace `<addDescribe>` with `describe()` then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE THE <addDescribe> BELOW WITH 'describe()'\n###\nprint(train_X.<addDescribe>)\nprint(train_Y.<addDescribe>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "314 non-prions out of 400, which means there's 86 prions in there. That looks about right if we refer to the graph we made in Step 4.\n\nLet's take a look at our test set too.\n\n#### Use the `describe()` function again, this time looking at __test__ instead of train."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE THE <addDescribe> BELOW WITH describe()\n###\nprint(test_X.<addDescribe>)\nprint(test_Y.<addDescribe>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Looks good to me! Alright, enough of that - lets make an SVM.\n\nStep 8\n-----\n\nBelow we will make an SVM, similar to the previous exercise.\n\nRemember, the syntax for SVM's is:\n\n`SVM_Model = svm.SVC().fit(features, labels)`\n\n### In the cell below replace:\n#### 1. `<addFeatures>` with `train_X`\n#### 2. `<addLabels>` with `train_Y`\n#### and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from sklearn import svm\n\n###\n# REPLACE <addFeatures> WITH train_X and <addLabels> WITH train_Y\n###\nSVM_Model = svm.SVC(gamma = 'auto').fit(<addFeatures>, <addLabels>)\n###\nprint(\"done!\")",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Well done! We've made a SVM Model from our training set.\n\nStep 9\n-----\n\nLets use our model to make some predictions. __Run the code__ in the cell below."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Don't edit this! Just hit run to plot the graph\n\n\n#This makes a plot of our SVM\ndef plot_SVM(clf, data, target):\n    #Make a list of which are prions\n    is_prion = target == 'prion'\n\n    graph.scatter(data['mass'], data['weight'], c = is_prion, zorder = 10, edgecolor = 'k', s = 40)\n    \n    # Put the result into a colour plot\n    XX, YY = np.mgrid[0:1:255j, 0:1:255j]\n    Z = clf.decision_function(np.c_[XX.ravel(), YY.ravel()]).reshape(XX.shape)\n    graph.pcolormesh(XX, YY, Z > 0)\n    graph.contour(XX, YY, Z, colors = ['k', 'k', 'k'], linestyles = ['--', '-', '--'], levels = [-.5, 0, .5])\n    \n    graph.ylim(0, 1)\n    graph.xlim(0, 1)\n    \n    graph.show()\n\n#Call the code to plot our SVM\nplot_SVM(SVM_Model, train_X, train_Y)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 10\n-------\n\nThe SVM has done a reasonable job of separating our test dataset into two. Now lets take a look at our test set.\n\nRemember our syntax for plotting SVM's is: `plot_SVM(SVM_Model, features, labels)`\n\nAdd our __test__ set below to see how it looks.\n\n### In the cell below replace:\n#### 1. `<addTestX>` with `test_X`\n#### 2. `<addTestY>` with `test_Y`\n#### and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <addTestX> WITH test_X AND <addTestY> WITH test_Y\n###\nplot_SVM(SVM_Model, <addTestX>, <addTestY>)\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 11\n-----\n\nGraphing is a good way to see how our model has done, but sometimes numbers can be better. Lets calculate the accuracy of our SVM in each dataset.\n\n### In the cell below replace:\n#### 1. `<addTrainX>` with `train_X`\n#### 2. `<addTestX>` with `test_X`\n#### 3. `<addTrainY>` with `train_Y`\n#### 4. `<addTestY>` with `test_Y`\n#### and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <addTrainX> WITH train_X AND <addTestX> with test_X FEATURE SETS TO GENERATE THE PREDICTIONS\n###\ntrain_P = SVM_Model.predict(<addTrainX>.values)\ntest_P = SVM_Model.predict(<addTestX>.values)\n###\n\n# This function evaluates the SVM's accuracy\ndef evaluate_SVM(pred, real, name):\n    matches = pred == real #see where predicted and real are the same\n    accuracy = sum(matches)/len(matches)*100 #convert to percent\n    print(name, \"Set Accuracy:\", accuracy, \"%\") \n\n\n###\n# REPLACE <addTrainY> WITH train_Y AND <addTestY> with test_Y\n###\nevaluate_SVM(train_P, <addTrainY>, 'Train')\nevaluate_SVM(test_P, <addTestY>, 'Test')\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "That's a good result. \n\nConclusion\n------\n\nWell done! We've taken a data set, cleaned and prepared it, made a SVM, and then evaluated it. Well done!\n\nYou can go back to the course now, or you can try using different kernels with your SVM below.\n\nOPTIONAL: Step 12\n-----\n\nWant to have a play around with different kernels for your SVM models? It's really easy!\n\nThe standard kernel is a Radial Basis Function kernel. But there's a few more you can choose from - linear (`linear`), polynomial (`poly`), and sigmoid (`sigmoid`). Lets try them out.\n\nIf you wanted to use a linear kernel, all you need to do is add `kernel='linear'` to your model. Like this:\n\n`SVM_Model = svm.SVC(kernel='linear')`\n\nGive it a go with all the different kernels below. The first one is done for you\n\n#### Run the cell below"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "def assess_SVM(SVM_Model):\n    # Plot the new linear SVM model\n    plot_SVM(SVM_Model, train_X, train_Y)\n    plot_SVM(SVM_Model, test_X, test_Y)\n\n    # Use the model to predict the training and test sets.\n    train_P = SVM_Model.predict(train_X.values)\n    test_P = SVM_Model.predict(test_X.values)\n\n    # Evaluate the model using the training and test sets\n    evaluate_SVM(train_P, train_Y, 'Train')\n    evaluate_SVM(test_P, test_Y, 'Test')\n\n# Make a new linear SVM model\nSVM_Model = svm.SVC(kernel = 'linear').fit(train_X, train_Y)\n\nassess_SVM(SVM_Model)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "You can see the hyperplane is a linear line!\n\nNow lets try a sigmoid kernel.\n\n#### Replace `<replaceThis>` with `'sigmoid'` then run the cell."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Make a new sigmoid SVM model\n\n###\n# REPLACE THE <replaceThis> BELOW WITH 'sigmoid' (INCLUDING THE QUOTES)\n###\nSVM_Model = svm.SVC(kernel = <replaceThis>, gamma = 4, coef0 = 0).fit(train_X, train_Y)\n###\nassess_SVM(SVM_Model)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Perhaps a sigmoid kernel isn't a good idea for this data set....\n\nLets try a polynomial kernel\n\n#### Replace `<replaceWithPoly>` with `'polynomial'` then run the cell."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Make a new polynomial SVM model\n\n###\n# REPLACE THE <replaceWithPoly> BELOW WITH 'poly' (INCLUDING THE QUOTES)\n###\nSVM_Model = svm.SVC(kernel = <replaceWithPoly>, gamma = 10, degree = 3, coef0 = 0).fit(train_X, train_Y)\n###\n\nassess_SVM(SVM_Model)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "If we were to carry on analyzing prions like this, polynomial looks like a good choice. If the data set was more complicated we could try different degrees for the polynomial to see which one was the most accurate. This is part of __`tuning`__ a model.\n\nWell done!"
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "python36",
      "display_name": "Python 3.6",
      "language": "python"
    },
    "language_info": {
      "mimetype": "text/x-python",
      "nbconvert_exporter": "python",
      "name": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6",
      "file_extension": ".py",
      "codemirror_mode": {
        "version": 3,
        "name": "ipython"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}