Crash_Course_in_Neural_Networks.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Crash Course in Neural Networks.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ZackAkil/crash-course-in-neural-networks/blob/master/Crash_Course_in_Neural_Networks.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "hTco7Spw0nnl",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Crash Course in Neural Networks 🧠\n",
        "\n",
        "Neural network are all the rage in machine learning. Although there are a few things you need to know to use them without spiraling down a pit of fustration.\n",
        "\n",
        "\n",
        "## 1. They can beat Linear Regression"
      ]
    },
    {
      "metadata": {
        "id": "v83noMdi0md1",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# lets import teh library that will create a dataset for use to use\n",
        "from sklearn.datasets import make_moons"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "xk6UR02p9lsw",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "call the make moons function to to create 400 points of data with the following\n",
        "```python\n",
        "X, y = make_moons(n_samples=400, noise=.1, random_state = 42)\n",
        "```"
      ]
    },
    {
      "metadata": {
        "id": "zQ7QLG6Dgb-4",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "IJTLU3KQhGfx",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# lets import a library for plotting our data \n",
        "\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "FriFvIG1-HgK",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "plot out our data using: \n",
        "```python\n",
        "plt.scatter(X[:,0],X[:,1], c=y)\n",
        "```"
      ]
    },
    {
      "metadata": {
        "id": "rP64pdsdhKg4",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "R_mLFBMagnBh",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# lets import a library to split our data into training data and test data\n",
        "\n",
        "from sklearn.model_selection import train_test_split"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1t5k5p-t_ZLb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "plot out our data using: \n",
        "```python\n",
        "X_train, X_test, y_train, y_test = train_test_split(\n",
        "    X, y, test_size=0.33, random_state=42)\n",
        "```\n",
        "https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html"
      ]
    },
    {
      "metadata": {
        "id": "3fcQog-pgqNS",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Wq4t9aqkgrUD",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# import a library that will build a linear classification model\n",
        "\n",
        "from sklearn.linear_model import LogisticRegression"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "qjzxGW21_vxM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "we create a new LogisticRegression model with the following code:\n",
        "```python\n",
        "LogisticRegression()\n",
        "```\n",
        "give it a varible name like `my_model`."
      ]
    },
    {
      "metadata": {
        "id": "kBI8cPw-hldH",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "6xUNmdG4ApDh",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "then train it to our data by calling `.fit`:\n",
        "```python\n",
        "my_model.fit(X_train, y_train)\n",
        "```\n"
      ]
    },
    {
      "metadata": {
        "id": "U5fUAqkDAem2",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "kd2kKsQrArxC",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "then we can check its performance using our test data with the `.score` function:\n",
        "\n",
        "```python\n",
        "my_model.score(X_test, y_test)\n",
        "```\n",
        "\n",
        "the score is from 0.0 to 1.0, 1 meaning all correct, 0 meaning none are correct."
      ]
    },
    {
      "metadata": {
        "id": "ZP8_QzOPAhUf",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "4dWGVSK-A3Zw",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "we can also visualise how good our model is by plotting out the data and setting `c` (the colour of the points) to be the prediction from our model. We get predictions from models by calling `.predict`:\n",
        "\n",
        "```python\n",
        "plt.scatter(X[:,0], X[:,1], c= YOUR_MODEL.predict(X) )\n",
        "\n",
        "```\n"
      ]
    },
    {
      "metadata": {
        "id": "VhcgwXWhhym6",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "AyNw8IIOBT3W",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Now enters the Neural Network 🧠"
      ]
    },
    {
      "metadata": {
        "id": "0ucq-uqhiFX-",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# we'll import the library\n",
        "\n",
        "from sklearn.neural_network import MLPClassifier"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "P_XKQrLQBlko",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Similar to before with create a new model with 3 layers of 50 nodes by calling:\n",
        "```python\n",
        "MLPClassifier(hidden_layer_sizes=(50,50,50), random_state=42)\n",
        "```\n",
        "and assign it to a varible name like `nn_model`"
      ]
    },
    {
      "metadata": {
        "id": "q_IYf4YeiJit",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "NHCDRT_HCdz0",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now fit your model to the training data like with the logistic regression model (using `.fit`)"
      ]
    },
    {
      "metadata": {
        "id": "xRjjAn6jCcVZ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "33qBTYCbB_qO",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "if you get a message like \n",
        "```\n",
        "ConvergenceWarning: Stochastic Optimizer:  reached and the optimization hasn't converged yet.\n",
        "\n",
        "```\n",
        "\n",
        "then you can add to your model an `max_iter` to train for longer:\n",
        "\n",
        "```python\n",
        "nn_model = MLPClassifier(max_iter=2000)\n",
        "\n",
        "```\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "PjWnfgTMCcDt",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Same agin with looking at the performance using `.score`"
      ]
    },
    {
      "metadata": {
        "id": "K1wjSVEHCxG7",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "yOePr36SC6Hb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now visualise your model by plotting out its predictions using:\n",
        "\n",
        "```python\n",
        "plt.scatter(X[:,0], X[:,1], c= YOUR_MODEL.predict(X) )\n",
        "```"
      ]
    },
    {
      "metadata": {
        "id": "jN_qlTBkiZRu",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "JODWKp0ZDKDx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Congratulations!!!! 🎉🎉🎉 You've trained and neural network to correctly classify data that a normal linear model failed with!!! \n",
        "### On to the next thing to know!"
      ]
    },
    {
      "metadata": {
        "id": "Caqlpd2_MF-D",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 2 Be VERY cautious of scale\n",
        "Neural networks are very good when they are in their comfort zone. That zone being between 0 and 1!"
      ]
    },
    {
      "metadata": {
        "id": "CuC1Xbugi-Fr",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# let's create some data and mess with its scale\n",
        "\n",
        "X2, y2 = make_moons(n_samples=200, noise=.1)\n",
        "X2[:,1] *= 7000"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "6QfCDRk2D0cX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "If we plot out our data with the following code:\n",
        "\n",
        "```python\n",
        "plt.scatter(X2[:,0],X2[:,1], c=y2)\n",
        "```\n",
        "\n",
        "Its looks the same as before, but **look carefully** at the axis scale: "
      ]
    },
    {
      "metadata": {
        "id": "BwSV28kGjKTp",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "lssFA4w7EHJG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Let's throw it at our neural network anyway and see what happens.\n",
        "\n",
        "Same as before we should split our data:\n",
        "\n",
        "```python\n",
        "\n",
        "X_train2, X_test2, y_train2, y_test2 = train_test_split(\n",
        "    X2, y2, test_size=0.33, random_state=42)\n",
        "```"
      ]
    },
    {
      "metadata": {
        "id": "p6cvV0Svm1FS",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "jsY-cCVrEgeq",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "in the next cell you should create a new `MLPClassifier()`, \n",
        "assign it to some varible name like `nn_model2`. \n",
        "\n",
        "Then `.fit` it to your new `train2` data,\n",
        "\n",
        "Then look at the accuracy score of the data using `.score` and your `test` data.  "
      ]
    },
    {
      "metadata": {
        "id": "WcEYLuNjFWKv",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "SyG5HwwFFEuv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Visualise your network prediction using the scatter plot like before.\n",
        "**Be careful** to use your new model and data"
      ]
    },
    {
      "metadata": {
        "id": "s5BwKhzWkV4Z",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "twEj3fYvkdTD",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "You should see that your model isn't performing that well on data :S\n",
        "\n",
        "Thats becuase the range of value is way way beyond 0 and 1 (neural networks comfort zone). \n",
        "\n",
        "## Lets fix that!"
      ]
    },
    {
      "metadata": {
        "id": "jGqbKUfck5Qw",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Lets import a library that will scale our data\n",
        "\n",
        "from sklearn.preprocessing import MinMaxScaler"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "c4FuiuwvHNRl",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Create a transformer that will scale our data back to normal with the following:\n",
        "\n",
        "```python\n",
        "scaler = MinMaxScaler()\n",
        "\n",
        "```"
      ]
    },
    {
      "metadata": {
        "id": "kdLGTH3jFYuq",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "GIaUz72nH4gX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Fit the scaler to our data by calling :\n",
        "```python\n",
        "scaler.fit(X2)\n",
        "```\n",
        "the library will default to scaling our data between 0 and 1\n",
        "\n",
        "**Note how we only need to scale the `X` data**"
      ]
    },
    {
      "metadata": {
        "id": "qY9AOpWlk827",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "NTHUQyY_IcYn",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "use the scaler on our poorly scaled data and save the output with the following:\n",
        "```python\n",
        "\n",
        "scaled_X_train = scaler.transform(X_train2)\n",
        "scaled_X_test = scaler.transform(X_test2)\n",
        "```"
      ]
    },
    {
      "metadata": {
        "id": "UYIdG7tVoilA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "D1eJpjcYJ169",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Lets look at our transformed data and **look closely at the axis scale**"
      ]
    },
    {
      "metadata": {
        "id": "WxnqXMx9IryT",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# let view our re-scaled training data\n",
        "\n",
        "plt.scatter(scaled_X_train[:,0], scaled_X_train[:,1], c=y_train2)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "VkPGPTZyJDCK",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# let view our re-scaled test data\n",
        "\n",
        "plt.scatter(scaled_X_test[:,0], scaled_X_test[:,1], c=y_test2)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "phB6oFQNKLt4",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "in the next cell you should create a new MLPClassifier(), assign it to some varible name like nn_model3.\n",
        "\n",
        "Then .fit it to your new re-scaled data as such:\n",
        "```python\n",
        "nn_model3.fit(scaled_X_train, y_train2)\n",
        "```\n",
        "\n",
        "Then look at the accuracy score of the data using .score and your scaled test data."
      ]
    },
    {
      "metadata": {
        "id": "2OXNjElmlDRp",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "VXL29AieKln8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Visualise your network prediction using the scatter plot like before. **Be careful** to use your new model and data:\n",
        "\n",
        "\n",
        "the code should look similar to this:\n",
        "```python\n",
        "plt.scatter(X2[:,0], X2[:,1], c= nn_model3.predict( scaler.transform(X2) ) )\n",
        "```"
      ]
    },
    {
      "metadata": {
        "id": "7DMueaeOloBU",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# WRITE YOUR CODE HERE\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "PnkkBR20K4u7",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Congradulations!!!!  🎉🎉🎉 you have brought and neural newtork back into its comfort zone by scaling it data!!!!\n",
        "\n",
        "## Onwards to experimenting using training curves"
      ]
    },
    {
      "metadata": {
        "id": "1HJAkbHWGDn9",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# 3. Know your Training curves\n",
        "\n",
        "How do you actually work out what size and shape of neural newtrok you should use??\n",
        "\n",
        "Well its trail and error, mostly error, and your guide is the training and testing curves!!!"
      ]
    },
    {
      "metadata": {
        "id": "ckf9utEb0wKQ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# let's get some data\n",
        "\n",
        "X3, y3 = make_moons(n_samples=100, noise=.2, random_state=42)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "SfX3vd-9qFJ5",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# view it\n",
        "\n",
        "plt.scatter(X3[:,0], X3[:,1], c=y3)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "syrp7eL2qBCZ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# split it\n",
        "\n",
        "X_train3, X_test3, y_train3, y_test3 = train_test_split(\n",
        "    X3, y3, test_size=0.33, random_state=42)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "_iZkO_GJ2G3m",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Now the fun part\n",
        "\n",
        "You want **Both** curves to go up \n",
        "\n",
        "You can play with the `hidden_layer_sizes` and  number of `epocs`\n"
      ]
    },
    {
      "metadata": {
        "id": "r_c6w6WcqPqo",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "nn_model3 = MLPClassifier(max_iter=1, \n",
        "                          warm_start=True, \n",
        "                          hidden_layer_sizes=(1,1,1)\n",
        "                         )\n",
        "\n",
        "# number of training cycles\n",
        "epocs = 10\n",
        "\n",
        "# --------------------------DON'T CHANGE FOLLOWING CODE-------------------------\n",
        "\n",
        "train_score = []\n",
        "test_score = []\n",
        "\n",
        "for i in range(epocs):\n",
        "  \n",
        "  # do one epoc of training    \n",
        "  nn_model3.fit(X_train3, y_train3)\n",
        "  \n",
        "  #    \n",
        "  train_score.append(nn_model3.score(X_train3, y_train3))\n",
        "  test_score.append(nn_model3.score(X_test3, y_test3))\n",
        "  \n",
        "plt.plot(range(epocs), train_score, c='b')\n",
        "plt.plot(range(epocs), test_score, c='r')\n",
        "plt.ylim(0, 1)\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "f3omjIlTMfDj",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Hopefully you have gained some intuition aroudn how to use neural networks. \n",
        "\n",
        "To learn more advance topic i recommend Andrw Ng's deepleanring courses:\n",
        "\n",
        "https://www.coursera.org/specializations/deep-learning"
      ]
    }
  ]
}