08. Neural Networks Introduction - Python.ipynb

{
  "cells": [
    {
      "metadata": {
        "collapsed": true
      },
      "cell_type": "markdown",
      "source": "Exercise 8 - Introduction to Neural Networks\n=======\n\nOriginally hypothesised in the 1940s, neural networks are now one of the main tools used in modern AI. Neural networks can be used for both regression and categorisation applications. Recent advances with storage, processing power, and open-source tools have allowed many successful applications of neural networks in medical diagnosis, filtering explicit content, speech recognition and machine translation.\n\nIn this exercise we will compare three dog breeds, using their age, weight, and height. We will make a neural network model to classify the breeds of the dogs based on these features.\n\nNote: It's extremely common for AI practitioners to use a template such as the one below for making neural networks quickly. After you are done, feel free to play around with the template to get a feel of how you can easily adjust a neural network to your problems using Keras.\n\n__Run the code__ in the cell below."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this!\nimport warnings\nwarnings.filterwarnings(\"ignore\")\nimport tensorflow as tf\nimport keras\nprint('keras using %s backend'%keras.backend.backend())\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as graph\n%matplotlib inline\ngraph.rcParams['figure.figsize'] = (15,5)\ngraph.rcParams[\"font.family\"] = 'DejaVu Sans'\ngraph.rcParams[\"font.size\"] = '12'\ngraph.rcParams['image.cmap'] = 'rainbow'",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 1\n------\n\nLet's start by opening up our data and having a look at it.\n\n#### In the cell below replace the `<printDataHere>` with `print(dataset.head())` and then __Run the code__ in the cell below."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# This loads the dataset\ndataset = pd.read_csv('Data/dog_data.csv')\n\n###\n# REPLACE <printDataHere> with print(dataset.head()) TO PREVIEW OUR DATASET\n###\n<printDataHere>\n###\n\n# This tells us the shape of the data set\nprint(\"Shape of data set:\", dataset.shape)\n\n# Defines the feature dataframe\nfeatures = dataset.drop(['breed'], axis = 1)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "So we've got data from __200 dogs__. The features are their __age__, __weight__, and __height__.\n\nStep 2\n------\n\nOur labels are three breeds of dogs, represented as numbers in our dataset, as `0`, `1`, and `2`. \n\nFor a neural network these numbers are misleading, as they might imply that breed `1` is closer to breed `2` than breed `0` is, in some way. But that is not the case here.\n\nTo allow the neural network to predict categories properly we represent categories as one-hot vectors. The labels (dog breeds) will go from being represented as `0`, `1`, and `2` to this:\n\n| breed 0 | breed 1 | breed 2 |\n|:------- |:------- |:------- |\n| `1 0 0` | `0 1 0` | `0 0 1` |\n\nSo the if the 1 is in the first position, the neural network knows that it's breed 0.\n\nIf the 1 is in the second position, the neural network knows that it's breed 1, and so on.\n\n#### Replace `<addLabels>` with `labels` and __Run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "from sklearn.preprocessing import OneHotEncoder\n\n# This sets the  labels (numerical)\nlabels = np.array(dataset['breed'])\n\n###\n# REPLACE THE <addLabels> BELOW WITH labels\n###\nonehot = OneHotEncoder(sparse = False).fit_transform(np.transpose([<addLabels>]))\n###\n\nprint(onehot[:5])",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "There we go!\n\nStep 3\n-------\n\nBefore we make our model, let's get our test set and training set ready.\n\nWe've got data on 200 dogs, so we'll use 160 for a training set, and 40 for our test set.\n\n#### Run the code below to set up our training at test sets."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this! This sets up our training and test sets.\n\n# This takes the first 160 examples for our training set\ntrain_X = features.values[:160]\ntrain_Y = onehot[:160]\n\n# This takes the last 40 examples of the 200 for our test set\ntest_X = features.values[160:]\ntest_Y = onehot[160:]",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "## Step 4\n\nThat's our data ready. Now it's time to make your first neural network model!\n\nThis is the standard syntax for a model in Keras. You can always play around with adding in extra hidden layers and changing their size and activation functions later.\n\nOur **first layer** is our **input layer**, with **3 nodes** because we have three features.\n\nOur __second layer__ is our 1st hidden layer, so let's try **4 nodes** for it.\n\nOur __third layer__ is our second hidden layer, let's try **2 nodes** for it.\n\nOur **final layer** will be the **output layer**, in which we have **3 nodes**, one for each of the dog breeds.\n\n### In the cell below replace:\n#### 1. `<addSequential>` with `Sequential()`\n#### 2. `<inputNodes>` with `3`\n#### 3. `<hiddenNotes1>` with `4`\n#### 4. `<hiddenNotes2>` with `2`\n#### 5. `<outputNotes>` with `3`\n\n#### and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Set a randomisation seed for replicatability.\nnp.random.seed(6)\n\n\n# This creates our base model for us to add to\n###\n# REPLACE THE <addSequential> BELOW WITH Sequential()\n###\nmodel = keras.models.<addSequential>\n###\n\n###\n# REPLACE THE <inputNodes>, <hiddenNotes1>, <hiddenNotes2>, <outputNotes> BELOW WITH THE APPROPRIATE NUMBERS OF NODES AS DESCRIBED ABOVE\n###\nstructure = [<inputNodes>, <hiddenNotes1>, <hiddenNotes2>, <outputNotes>]\n###\n\n# Input layer + hidden layer 1\nmodel.add(keras.layers.Dense(units=structure[1], input_dim = structure[0], activation = 'relu'))\n\n# Hidden layer 2\nmodel.add(keras.layers.Dense(units=structure[2], activation = 'relu'))\n\n# Output layer - note that the activation function is softmax\n# Softmax will predict a category and provide a value for how likely this is the correct prediction.\nmodel.add(keras.layers.Dense(units=structure[3], activation = tf.nn.softmax))\n\nprint(\"Layer structure:\", structure)",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Expected output:  \n`Layer structure: [3, 4, 2, 3]`\n\nAlright, that's your first model ready.\n\n('tanh' is another common activation function if you want to try it instead of relu, but it doesn't perform very well here)\n\nStep 5\n-------\n\nNext up we'll compile it and see how it runs.\n\nThere's a few parameters you can chose that change how the model trains, and end up changing how the model performs.\n\nWe will use some standard parameters for now. Feel free to experiment with some different parameters later on.\n\nIf this doesn't work, check your input the correct size for the input and output layers in step 4 (3 nodes each).\n\n### In the cell below replace:\n#### 1. `<addLoss>` with `'categorical_crossentropy'`\n#### 2. `<addOptimizer>` with `sgd'`\n#### 3. `<addMetric>` with `'accuracy'`\n#### 4. `<addTrainX>` with `train_X`\n#### 5. `<addTrainY>` with `train_Y`\n\n#### and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Let's compile the model\n\n###\n# REPLACE THE <addLoss> WITH 'categorical_crossentropy', <addOptimizer> WITH 'sgd', AND <addMetric> with 'accuracy' (INCLUDING THE QUOTES)\n###\nmodel.compile(loss = <addLoss>, optimizer = <addOptimizer>, metrics = [<addMetric>])\n###\n\n# Time to fit the model\nprint('Starting training')\n\n###\n# REPLACE THE <addTrainX> WITH train_X AND <addTrainY> WITH train_Y\n###\ntraining_stats = model.fit(<addTrainX>, <addTrainY>, batch_size = 1, epochs = 24, verbose = 0)\n###\n\nprint('Training finished')\nprint('Training Evaluation: loss = %0.3f, accuracy = %0.2f%%'\n      %(training_stats.history['loss'][-1], 100 * training_stats.history['accuracy'][-1]))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "See? Neural networks aren't too hard.\n\n`'adam'` is another popular optimizer if you want to try it instead of `'sgd'`\n\n#### Lets plot it! Run the cell below."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Run this!\n\naccuracy, = graph.plot(training_stats.history['acc'],label = 'Accuracy')\ntraining_loss, = graph.plot(training_stats.history['loss'],label = 'Training Loss')\n\n\ngraph.legend(handles = [accuracy,training_loss])\nloss = np.array(training_stats.history['loss'])\nxp = np.linspace(0, loss.shape[0], 10 * loss.shape[0])\ngraph.plot(xp, np.full(xp.shape, 1), c = 'k', linestyle = ':', alpha = 0.5)\ngraph.plot(xp, np.full(xp.shape, 0), c = 'k', linestyle = ':', alpha = 0.5)\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Step 6\n------\n\nNow that our model is trained and ready, let's see how it performs on our test data!\n\nIt's important to test a model on data that it has never seen before, to make sure it doesn't overfit. Now let's evaluate it against the test set.\n\n### In the cell below replace:\n#### 1. `<addTestX>` with `test_X`\n#### 2. `<addTestY>` with `test_Y`\n#### and then __run the code__."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE <addTestX> WITH test_X AND <addTestY> with test_Y\n###\nevaluation = model.evaluate(<addTestX>, <addTestY>, verbose=0)\n###\n\nprint('Test Set Evaluation: loss = %0.6f, accuracy = %0.2f' %(evaluation[0], 100*evaluation[1]))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "It seems to be very accurate with the random seed that we set, but let's see how it predicts something completely new and unclassified!\n\nCome up with a brand new sample of the format `[age, weight, height]` to test it with.\n\n#### Replace the `<addNumber>`'s below with any numbers you want."
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE THE <addNumber> BELOW WITH A WHATEVER NUMBERS YOU WANT, e.g. [9, 7, 7]\n###\n# [age, weight, height]\nnew_sample = [<addNumber>, <addNumber>, <addNumber>]\n###",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Let's have a look at where our new sample sits in comparison to our dataset.\n\n#### Replace the `<addNewSample>`'s below with `new_sample`"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "# Plots out the age-weight relationship\n\n###\n# REPLACE THE <addNewSample> BELOW WITH new_sample\n###\ngraph.plot(<addNewSample>[0], <addNewSample>[1], 'ko', marker='x')\n###\n\ngraph.scatter(train_X[:,0], train_X[:,1], c = labels[:160])\ngraph.title('samples by age and weight')\ngraph.xlabel('age')\ngraph.ylabel('weight')\ngraph.show()\n\n# Plot out the age-height relationship\n\n###\n# REPLACE THE <addNewSample> BELOW WITH new_sample\n###\ngraph.plot(<addNewSample>[0], <addNewSample>[2], 'ko', marker='x')\n###\n\ngraph.scatter(train_X[:,0], train_X[:,2], c = labels[:160])\ngraph.title('samples by age and height')\ngraph.xlabel('age')\ngraph.ylabel('height')\ngraph.show()",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Looks alright? Now let's see what breed of dog the model says it is!\n\n#### Replace `<addNewSample>` with `new_sample`"
    },
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "###\n# REPLACE THE <addNewSample> BELOW WITH new_sample\n###\npredicted = model.predict(np.array([<addNewSample>]))\nprint('Breed prediction for %s:' %(<addNewSample>))\n###\n\nprint(np.around(predicted[0],2))\nprint('Breed %s, with %i%% certainty.' %(np.argmax(predicted), np.round(100 * predicted[:, np.argmax(predicted)][0])))",
      "execution_count": null,
      "outputs": []
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Breed `0` should be purple, breed `1` should be green, and breed `2` should be red."
    },
    {
      "metadata": {},
      "cell_type": "markdown",
      "source": "Conclusion\n------\n\nWe've built a simple neural network to help us predict dog breeds! In the next exercise we'll look into neural networks with a bit more depth, and at the factors that influence how well it learns.\n\nIf you want to play around with this neural network and a new data set, just remember to set your input and output sizes correctly."
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "python36",
      "display_name": "Python 3.6",
      "language": "python"
    },
    "language_info": {
      "mimetype": "text/x-python",
      "nbconvert_exporter": "python",
      "name": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.6",
      "file_extension": ".py",
      "codemirror_mode": {
        "version": 3,
        "name": "ipython"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}