06. Support Vector Machines - Python.ipynb

{
  "cells": [
    {
      "metadata": {
        "collapsed": true
      },
      "cell_type": "markdown",
      "source": "Exercise 6 - Support Vector Machines\n=====\n\nSupport vector machines (SVMs) let us predict categories. This exercise will demonstrate a simple support vector machine that can predict a category from a small number of features. \n\nOur problem is that we want to be able to categorise which type of tree an new specimen belongs to. To do this, we will use features of three different types of trees to train an SVM. \n\n__Run the code__ in the cell below."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this code!\n# It sets up the graphing configuration.\nimport warnings\nwarnings.filterwarnings(\"ignore\")\nimport matplotlib.pyplot as graph\n%matplotlib inline\ngraph.rcParams['figure.figsize'] = (15,5)\ngraph.rcParams[\"font.family\"] = 'DejaVu Sans'\ngraph.rcParams[\"font.size\"] = '12'\ngraph.rcParams['image.cmap'] = 'rainbow'",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 1\n-----\n\nFirst, we will take a look at the raw data first to see what features we have.\n\n#### Replace `<printDataHere>` with `print(dataset.head())` and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "import pandas as pd\nimport numpy as np\n\n# Loads the SVM library\nfrom sklearn import svm\n\n# Loads the dataset\ndataset = pd.read_csv('Data/trees.csv')\n\n###\n# REPLACE <printDataHere> with print(dataset.head()) TO PREVIEW THE DATASET\n###\n<printDataHere>\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "It looks like we have _four features_ (leaf_width, leaf_length, trunk_girth, trunk_height) and _one label_ (tree_type).\n\nLet's plot it.\n\n__Run the code__ in the cell below."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this code to plot the leaf features\n\n# This extracts the features. drop() deletes the column we state (tree_type), leaving on the features\nallFeatures = dataset.drop(['tree_type'], axis = 1)\n\n# This keeps only the column we state (tree_type), leaving only our label\nlabels = np.array(dataset['tree_type'])\n\n#Plots the graph\nX = allFeatures['leaf_width']\nY = allFeatures['leaf_length']\ncolor=labels\ngraph.scatter(X, Y, c = color)\ngraph.title('classification plot for leaf features')\ngraph.xlabel('leaf width')\ngraph.ylabel('leaf length')\ngraph.legend()\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "__Run the code__ in the cell below to plot the trunk features"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this code to plot the trunk features\ngraph.scatter(allFeatures['trunk_girth'], allFeatures['trunk_height'], c = labels)\ngraph.title('Classification plot for trunk features')\ngraph.xlabel('trunk girth')\ngraph.ylabel('trunk height')\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 2\n-----\n\nLets make a support vector machine.\n\nThe syntax for a support vector machine is as follows:\n\n__`model = svm.SVC().fit(features, labels)`__\n\nYour features set will be called __`train_X`__ and your labels set will be called __`train_Y`__\n\n#### Let's first run the SVM in the cell below using the first two features, the leaf features."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Sets up the feature and target sets for leaf features\n\n# Feature 1\nfeature_one = allFeatures['leaf_width'].values\n\n# Feature 2\nfeature_two = allFeatures['leaf_length'].values\n\n# Features\ntrain_X = np.asarray([feature_one, feature_two]).transpose()\n\n# Labels\ntrain_Y = labels \n\n# Fits the SVM model\n###\n# REPLACE THE <makeSVM> WITH THE CODE TO MAKE A SVM MODEL AS ABOVE\n###\nmodel = <makeSVM>\n###\nprint(\"Model ready. Now plot it to see the result.\")",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "#### Let's plot it! Run the cell below to visualise the SVM with our dataset."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this to plots the SVM model\nX_min, X_max = train_X[:, 0].min() - 1, train_X[:, 0].max() + 1\nY_min, Y_max = train_X[:, 1].min() - 1, train_X[:, 1].max() + 1\n\nXX, YY = np.meshgrid(np.arange(X_min, X_max, .02), np.arange(Y_min, Y_max, .02))\nZ = model.predict(np.c_[XX.ravel(), YY.ravel()]).reshape(XX.shape)\n\ngraph.scatter(feature_one, feature_two, c = train_Y, cmap = graph.cm.rainbow, zorder = 10, edgecolor = 'k', s = 40)\ngraph.contourf(XX, YY, Z, cmap = graph.cm.rainbow, alpha = 1.0)\ngraph.contour(XX, YY, Z, colors = 'k', linestyles = '--', alpha=0.5)\n\ngraph.title('SVM plot for leaf features')\ngraph.xlabel('leaf width')\ngraph.ylabel('leaf length')\n\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "The graph shows three colored zones that the SVM has chosen to group the datapoints in. Color, here, means type of tree. As we can see, the zones correspond reasonably well with the actual tree types of our training data. This means that the SVM can group, for its training data, quite well calculate tree type based on leaf features.\n\n\nNow let's do the same using trunk features.\n\n### In the cell below replace:\n#### 1. `<addTrunkGirth>` with `'trunk_girth'`\n#### 2. `<addTrunkHeight>` with `'trunk_height'`\n#### Then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Feature 1\n###--- REPLACE THE <addTrunkGirth> BELOW WITH 'trunk_girth' (INCLUDING THE QUOTES) ---###\n###\ntrunk_girth = allFeatures[<addTrunkGirth>].values\n###\n\n# Feature 2\n###--- REPLACE THE <addTrunkHeight> BELOW WITH 'trunk_height' (INCLUDING THE QUOTES) ---###\ntrunk_height = allFeatures[<addTrunkHeight>].values\n###\n\n# Features\ntrunk_features = np.asarray([trunk_girth, trunk_height]).transpose()\n\n# Fits the SVM model\nmodel = svm.SVC().fit(trunk_features, train_Y)\n\n# Plots the SVM model\nX_min, X_max = trunk_features[:, 0].min() - 1, trunk_features[:, 0].max() + 1\nY_min, Y_max = trunk_features[:, 1].min() - 1, trunk_features[:, 1].max() + 1\n\nXX, YY = np.meshgrid(np.arange(X_min, X_max, .02), np.arange(Y_min, Y_max, .02))\nZ = model.predict(np.c_[XX.ravel(), YY.ravel()]).reshape(XX.shape)\n\ngraph.scatter(trunk_girth, trunk_height, c = train_Y, cmap = graph.cm.rainbow, zorder = 10, edgecolor = 'k', s = 40)\ngraph.contourf(XX, YY, Z, cmap = graph.cm.rainbow, alpha = 1.0)\ngraph.contour(XX, YY, Z, colors = 'k', linestyles = '--', alpha = 0.5)\n\ngraph.title('SVM plot for leaf features')\ngraph.xlabel('trunk girth')\ngraph.ylabel('trunk height')\n\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Conclusion\n-------\n\nAnd that's it! You've made a simple support vector machine that can predict the type of tree based on the leaf and trunk measurements!\n\nYou can go back to the course now and click __'Next Step'__ to move onto how we can test AI models."
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "python36",
      "display_name": "Python 3.6",
      "language": "python"
    },
    "language_info": {
      "mimetype": "text/x-python",
      "nbconvert_exporter": "python",
      "name": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6",
      "file_extension": ".py",
      "codemirror_mode": {
        "version": 3,
        "name": "ipython"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}