Binary+Classification+with+IMDB+data+set..ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Binary classification of IMDB dataset in Keras\n",
    "\n",
    "This notebook is for the binary classification of **IMDB dataset** in Keras. Since IMDB dataset comes packaged with the Keras so you don't have to download it from any website. \n",
    "\n",
    "The IMDB dataset a set of 50,000 reviews from the Internet Movie Database. The dataset split into 25,000 reviews for training and 25,000 reviews for testing, each set consisting in 50% negative and 50% positive reviews.\n",
    "\n",
    "The IMDB dataset comes packaged with Keras and it is also preprocessed. The reviews (sequences of words) have been turned into sequences of integers, where each integer stands for a specific word in a dectionary. *We can see the sequences of integers once we will load the dataset*.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# import the libraries \n",
    "import numpy as np\n",
    "from keras import models\n",
    "from keras import layers\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Loading the dataset\n",
    "from keras.datasets import imdb\n",
    "(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=15000) #The argument 15,000 means we are \n",
    "# only keeping most frequent 15,000 words. \n",
    "#The variables train_data and text_data \n",
    "# are lists of reviews, train_labels and test_labels are lists of 0s and 1s for negative and positive reviews respectively.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1,\n",
       " 194,\n",
       " 1153,\n",
       " 194,\n",
       " 8255,\n",
       " 78,\n",
       " 228,\n",
       " 5,\n",
       " 6,\n",
       " 1463,\n",
       " 4369,\n",
       " 5012,\n",
       " 134,\n",
       " 26,\n",
       " 4,\n",
       " 715,\n",
       " 8,\n",
       " 118,\n",
       " 1634,\n",
       " 14,\n",
       " 394,\n",
       " 20,\n",
       " 13,\n",
       " 119,\n",
       " 954,\n",
       " 189,\n",
       " 102,\n",
       " 5,\n",
       " 207,\n",
       " 110,\n",
       " 3103,\n",
       " 21,\n",
       " 14,\n",
       " 69,\n",
       " 188,\n",
       " 8,\n",
       " 30,\n",
       " 23,\n",
       " 7,\n",
       " 4,\n",
       " 249,\n",
       " 126,\n",
       " 93,\n",
       " 4,\n",
       " 114,\n",
       " 9,\n",
       " 2300,\n",
       " 1523,\n",
       " 5,\n",
       " 647,\n",
       " 4,\n",
       " 116,\n",
       " 9,\n",
       " 35,\n",
       " 8163,\n",
       " 4,\n",
       " 229,\n",
       " 9,\n",
       " 340,\n",
       " 1322,\n",
       " 4,\n",
       " 118,\n",
       " 9,\n",
       " 4,\n",
       " 130,\n",
       " 4901,\n",
       " 19,\n",
       " 4,\n",
       " 1002,\n",
       " 5,\n",
       " 89,\n",
       " 29,\n",
       " 952,\n",
       " 46,\n",
       " 37,\n",
       " 4,\n",
       " 455,\n",
       " 9,\n",
       " 45,\n",
       " 43,\n",
       " 38,\n",
       " 1543,\n",
       " 1905,\n",
       " 398,\n",
       " 4,\n",
       " 1649,\n",
       " 26,\n",
       " 6853,\n",
       " 5,\n",
       " 163,\n",
       " 11,\n",
       " 3215,\n",
       " 10156,\n",
       " 4,\n",
       " 1153,\n",
       " 9,\n",
       " 194,\n",
       " 775,\n",
       " 7,\n",
       " 8255,\n",
       " 11596,\n",
       " 349,\n",
       " 2637,\n",
       " 148,\n",
       " 605,\n",
       " 2,\n",
       " 8003,\n",
       " 15,\n",
       " 123,\n",
       " 125,\n",
       " 68,\n",
       " 2,\n",
       " 6853,\n",
       " 15,\n",
       " 349,\n",
       " 165,\n",
       " 4362,\n",
       " 98,\n",
       " 5,\n",
       " 4,\n",
       " 228,\n",
       " 9,\n",
       " 43,\n",
       " 2,\n",
       " 1157,\n",
       " 15,\n",
       " 299,\n",
       " 120,\n",
       " 5,\n",
       " 120,\n",
       " 174,\n",
       " 11,\n",
       " 220,\n",
       " 175,\n",
       " 136,\n",
       " 50,\n",
       " 9,\n",
       " 4373,\n",
       " 228,\n",
       " 8255,\n",
       " 5,\n",
       " 2,\n",
       " 656,\n",
       " 245,\n",
       " 2350,\n",
       " 5,\n",
       " 4,\n",
       " 9837,\n",
       " 131,\n",
       " 152,\n",
       " 491,\n",
       " 18,\n",
       " 2,\n",
       " 32,\n",
       " 7464,\n",
       " 1212,\n",
       " 14,\n",
       " 9,\n",
       " 6,\n",
       " 371,\n",
       " 78,\n",
       " 22,\n",
       " 625,\n",
       " 64,\n",
       " 1382,\n",
       " 9,\n",
       " 8,\n",
       " 168,\n",
       " 145,\n",
       " 23,\n",
       " 4,\n",
       " 1690,\n",
       " 15,\n",
       " 16,\n",
       " 4,\n",
       " 1355,\n",
       " 5,\n",
       " 28,\n",
       " 6,\n",
       " 52,\n",
       " 154,\n",
       " 462,\n",
       " 33,\n",
       " 89,\n",
       " 78,\n",
       " 285,\n",
       " 16,\n",
       " 145,\n",
       " 95]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's check train_data \n",
    "train_data[1] #We can see sequence of integer (Every integer represents some specific word in a dectionary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's check label_data\n",
    "train_labels[1] #The movie review of train_data is negative "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Data Prepration \n",
    "#The current data set is in sequence of integer so we have to convert into vectorize form.\n",
    "#let's vectorize our data\n",
    "#Let's vectorize movie sequence of integer\n",
    "\n",
    "def vectorize_sequences(sequences, dimension=15000):\n",
    "    results = np.zeros((len(sequences), dimension)) #We are creating an all-zero matrix of shape (len(sequences), dimension)\n",
    "    for i, sequence in enumerate(sequences):\n",
    "        results[i, sequence] = 1\n",
    "    return results\n",
    "\n",
    "#Traning data after vectoriziation\n",
    "x_train = vectorize_sequences(train_data)\n",
    "#Test data after vectorization\n",
    "x_test = vectorize_sequences(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.,  1.,  1., ...,  0.,  0.,  0.])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Let's check \n",
    "x_train[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Let's vectorize labels\n",
    "y_train = np.asarray(train_labels).astype('float32')\n",
    "y_test = np.asarray(test_labels).astype('float32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Building the network \n",
    "#We are building 3 layer network, hidden unit in intermediate layers (layer 1 and layer 2) is 16 and \n",
    "#third layer which will output the prediction. \n",
    "#We fully connected dense layer because it's a 2D tensor and input id vector and output is scaler. \n",
    "#Activation function relu for hidden layer and sigmoid for output layer\n",
    "\n",
    "model = models.Sequential()\n",
    "model.add(layers.Dense(16, activation='relu', input_shape=(15000,)))\n",
    "model.add(layers.Dense(16, activation='relu'))\n",
    "model.add(layers.Dense(1, activation='sigmoid'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Compiling ou model\n",
    "# We are using optimizer(optimizer optmize the weight based on feedback from accuracy) as rmsprop, \n",
    "# loss function(the distance between prediction and ground truth) as binary_crossentropy.\n",
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Creating validation set of 10000\n",
    "x_val = x_train[:10000]\n",
    "validation_x_train = x_train[10000:]\n",
    "\n",
    "y_val = y_train[:10000]\n",
    "validation_y_train = y_train[10000:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 15000 samples, validate on 10000 samples\n",
      "Epoch 1/10\n",
      "15000/15000 [==============================] - 2s - loss: 8.1632e-04 - acc: 1.0000 - val_loss: 0.8098 - val_acc: 0.8670\n",
      "Epoch 2/10\n",
      "15000/15000 [==============================] - 2s - loss: 3.6247e-04 - acc: 1.0000 - val_loss: 0.8139 - val_acc: 0.8675\n",
      "Epoch 3/10\n",
      "15000/15000 [==============================] - 2s - loss: 2.5568e-04 - acc: 1.0000 - val_loss: 0.8835 - val_acc: 0.8638\n",
      "Epoch 4/10\n",
      "15000/15000 [==============================] - 2s - loss: 7.7088e-04 - acc: 0.9998 - val_loss: 0.9099 - val_acc: 0.8660\n",
      "Epoch 5/10\n",
      "15000/15000 [==============================] - 2s - loss: 1.1088e-04 - acc: 1.0000 - val_loss: 0.9197 - val_acc: 0.8675\n",
      "Epoch 6/10\n",
      "15000/15000 [==============================] - 2s - loss: 8.2922e-05 - acc: 1.0000 - val_loss: 0.9407 - val_acc: 0.8669\n",
      "Epoch 7/10\n",
      "15000/15000 [==============================] - 2s - loss: 5.9840e-05 - acc: 1.0000 - val_loss: 1.0008 - val_acc: 0.8647\n",
      "Epoch 8/10\n",
      "15000/15000 [==============================] - 2s - loss: 2.5335e-04 - acc: 1.0000 - val_loss: 1.0339 - val_acc: 0.8661\n",
      "Epoch 9/10\n",
      "15000/15000 [==============================] - 2s - loss: 2.2388e-05 - acc: 1.0000 - val_loss: 1.0382 - val_acc: 0.8654\n",
      "Epoch 10/10\n",
      "15000/15000 [==============================] - 2s - loss: 1.8128e-05 - acc: 1.0000 - val_loss: 1.0552 - val_acc: 0.8654\n"
     ]
    }
   ],
   "source": [
    "# Traning our model\n",
    "# We have used 25 epochs with batch_size=512. We have used history so our model could remember the traning\n",
    "history = model.fit(validation_x_train, validation_y_train, epochs=10, batch_size=512, validation_data=(x_val, y_val))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Actually accuray for above model is **100%** but validation accuray is **86.54%** **which means model is not performing on unseen data the way it is performing on test data**. This is called overfitting. There are various methods which can be used to reduce overfitting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['val_loss', 'acc', 'val_acc', 'loss'])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# history dictionary\n",
    "history_dict = history.history\n",
    "#find entries in history dictionary\n",
    "history_dict.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "History dictionary conatins 4 entries as we had earlier. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAE+dJREFUeJzt3XvUZXV93/H3ByYQb3i/DgyJoNHQWIQsmFxKjwvaGZRK\nEzVxhBaNpW0iwVxMIbZZPKxeVklW6w3bhoRQb3RcSDQmC3Uk9nGVVC4VRmGcEXTkJog3qsWsuHD4\n9o+zwbOfnOc+e/Z+nnm/1nrWnL3P7+z5zFnz7M/Zv33OPqkqJEl61CF9B5AkDYvFIElqsRgkSS0W\ngySpxWKQJLVYDJKklk6LIcnlSR5I8vl57n9dks8l2ZnkuiQ/1WUeSdLiuj5iuALYssD9e4FTqup4\n4N8Bf9RxHknSIjZ0ufGqui7J0Qvcf/3E4vXAxi7zSJIWN6RzDP8M+FjfISTpYNfpEcNSJXkZ8Abg\n5/vOIkkHu96LIclLgMuArVX14ALjvKiTJK1AVWU54w/EVFKan799R7IJuBr4J1X15cU2VFWD+rno\noot6z7BWcpnJTAdDriFmWolOjxiSXAmMgKcnuRu4CDgMqKq6DPg94GnAf0kS4OGqOqnLTJKkhXX9\nrqTXLXL/ucC5XWaQJC3PkN6VtOaMRqO+I0w1xFxmWhozLd0Qcw0x00pkpXNQB1qSWitZJWkoklAD\nPPksSVpDLAZJUovFIElqsRgkSS0WgySpxWKQJLVYDJKkFotBktRiMUiSWiwGSVKLxSBJarEYJEkt\nFoMkqcVikCS1WAySpBaLQZLUYjFIklosBklSi8UgSWqxGCRJLRaDJKllQ98BJGm9mJ2F0ajvFOMc\ns7Mrf3yqan9l6VSSWitZJXVvKDvhSTMz458hSUJVZTmPcSpJ0pq0mlfEWphTSZK0CpPTNhdf/MP1\no9HwjmiWymKQtGYMcSc89+8e2lTSSnRaDEkuB84AHqiql8wz5p3A6cD3gNdX1c4uM0lau9bjTniI\nuj7HcAWwZb47k5wOHFNVLwD+BfDfOs4jSZ1Zq1NHc3VaDFV1HfDgAkPOBN7bjL0BeHKSZ3eZSdLy\nDPUk7xB3wkPMtBJ9vytpI3DPxPJXm3WSBsJiOPj0XQzT3lvrhxUkqUd9vyvpXuCoieUjgfvmGzwz\ncaZpNBox8iWD1IkhvvtHSzM7O8vsKg/zOv/kc5IfA/68qn5qyn0vB95UVa9Ishl4e1Vtnmc7fvJZ\n6sEQP82rpVvJJ5+7frvqlcAIeHqSu4GLgMOAqqrLquqaJC9P8iXGb1d9Q5d5JEmL81pJkhY0xGsS\naelWcsRgMUjSOuZF9CRJq2YxSJJaLAZJUovFIA3IUD9lrIOLxSANiMWgIbAYJEktfV8SQzroefkJ\nDY3FoIPWUD645ZfPaGicStJBy/l8aTqLQQeEO+GlGcIRjORUkg6IoUzbDH0+fwgZJItBBxXn86XF\nWQzqzNBfnUuazmJYh4YybTP0V+dDeI6kIfLk8zrkid6lsRik6SwGHRDuhKW1w6mkdWLo8/lDyCBp\naSyGdWLo8/mS1g6nkiRJLRbDOuS0jaTVSFX1nWFJktRaySpJQ5GEqspyHuMRgySpxWKQJLVYDJKk\nFotBktRiMUiSWiyGVfCaRJLWI4thFSwGSetR58WQZGuSPUluT3LBlPuPSvKpJDcn2Znk9K4zSZLm\n1+m1kpIcAlwKnArcB9yU5M+qas/EsH8DfLCq/jDJi4FrgB/vMtdqDP1idZK0Wl1fRO8k4I6qugsg\nyXbgTGCyGB4BjmhuPwX46nwbe/TCcH3uhL1YnaT1ruti2AjcM7F8L+OymHQxsCPJ+cDjgdPm25g7\nYUnqXtfFMO36HHMveLQNuKKq3pZkM/B+4LhpG5uZaIbRaMSo57kbp44kDc3s7Cyzq3xnTKcX0Wt2\n9DNVtbVZvhCoqrpkYsxtwJaq+mqz/GXg5Kr65pxteRE9SVqmIV5E7ybg2CRHJzkMeC3w0Tlj7qKZ\nPmpOPh8+txQkSQdOp8VQVfuA84AdwC5ge1XtTnJxkjOaYW8Bzk2yE/gAcE6XmSRJC/P7GCRpHRvi\nVJIkaY2xGCRJLRaDJKnFYpAktVgMkqQWi0GS1GIxSJJaLAZJUovFIElqsRgkSS0WgySpxWKQJLVY\nDJKkFotBktRiMUiSWiwGSVKLxSBJarEYJEktFoMkqcVikCS1WAySpBaLQZLUYjFIklosBklSy5KK\nIckxSQ5vbo+SnJ/kKd1GkyT1YalHDFcD+5IcC1wGHAVc2VkqSVJvlloMj1TVD4BfAN5VVb8DPLe7\nWJKkviy1GB5Osg04B/iLZt2PdBNJktSnpRbDG4CfAf59VX0lyY8D71/KA5NsTbInye1JLphnzC8l\n2ZXk1iRL2q4kqRupquU9IHkqcFRVfX4JYw8BbgdOBe4DbgJeW1V7JsYcC3wQeFlVfTfJM6rqm1O2\nVcvNKkkHuyRUVZbzmKW+K2k2yRFJngbcDPxRkv+8hIeeBNxRVXdV1cPAduDMOWPOBd5dVd8FmFYK\nkqQDZ6lTSU9udty/CLy3qk4GTlvC4zYC90ws39usm/RC4CeSXJfkfyfZssRMkqQObFjquCTPBX4J\n+NfL2P60w5e580EbgGOBU4BNwP9KctyjRxCTZmZmHrs9Go0YjUbLiCJJ69/s7Cyzs7Or2saSzjEk\neQ3we8BfVdWvJnk+8AdV9apFHrcZmKmqrc3yhUBV1SUTY/4r8Jmqem+zfC1wQVV9ds62PMcgScu0\nknMMyz75vKyNJ4cCX2R88vl+4EZgW1XtnhizpVn3+iTPAD4LHF9VD87ZlsUgScvU5cnnI5N8OMnX\nkzyQ5OokRy72uKraB5wH7AB2AduraneSi5Oc0Yz5BPCtJLuAvwTeMrcUJEkHzlKnkj7J+BIY72tW\nnQ2cVVX/oMNsczN4xCBJy9TZVFKSnVV1/GLrumQxSNLydTaVBHwzydlJDm1+zga+tfyIkqShW2ox\n/Arjt6p+jfFJ5FczvkyGJGmdWfG7kpL8RlW9fT/nWejvcypJkpbpgL5dNcndVbVpRQ9e2d9nMUjS\nMnV5jmHq37eKx0qSBmo1xeDLd0lahxa8VlKS/8f0AgjwuE4SSZJ6tWAxVNWTDlQQSdIwrGYqSZK0\nDlkMkqQWi0GS1GIxSJJaLAZJUovFIElqsRgkSS0WgySpxWKQJLVYDJKkFotBktRiMUiSWiwGSVKL\nxSBJarEYJEktFoMkqcVikCS1WAySpBaLQZLU0nkxJNmaZE+S25NcsMC4Vyd5JMkJXWeSJM2v02JI\ncghwKbAFOA7YluRFU8Y9Efh14Pou80iSFtf1EcNJwB1VdVdVPQxsB86cMu7fApcA3+84jyRpEV0X\nw0bgnonle5t1j0lyPHBkVV3TcRZJ0hJs6Hj7mbKuHrszCfA24JxFHgPAzMzMY7dHoxGj0WjVASVp\nPZmdnWV2dnZV20hVLT5qpRtPNgMzVbW1Wb4QqKq6pFk+AvgS8BDjQngO8C3glVV185xtVZdZJWk9\nSkJVzfuCe+pjOi6GQ4EvAqcC9wM3Atuqavc84/8n8FtVdcuU+ywGSVqmlRRDp+cYqmofcB6wA9gF\nbK+q3UkuTnLGtIewwFSSJKl7nR4x7E8eMUjS8g3uiEGStPZYDJKkFotBktRiMUiSWiwGSVKLxSBJ\narEYJEktFoMkqcVikCS1WAySpBaLQZLUYjFIklosBklSi8UgSWqxGCRJLRaDJKnFYpAktVgMkqQW\ni0GS1GIxSJJaLAZJUovFIElqsRgkSS0WgySpxWKQJLVYDJKkFotBktRiMUiSWjovhiRbk+xJcnuS\nC6bc/5tJdiXZmeSTSY7qOpMkaX6dFkOSQ4BLgS3AccC2JC+aM+xm4MSqOh64GviDLjNJkhbW9RHD\nScAdVXVXVT0MbAfOnBxQVZ+uqr9pFq8HNnacSZK0gK6LYSNwz8TyvSy8438j8LFOE0mSFrSh4+1n\nyrqaOjA5GzgR+PudJpIkLajrYrgX2DSxfCRw39xBSU4Dfhc4pZlymmpmZuax26PRiNFotL9yStK6\nMDs7y+zs7Kq2kaqpL+D3iySHAl8ETgXuB24EtlXV7okxLwWuArZU1ZcX2FZ1mVWS1qMkVNW02Zt5\ndXqOoar2AecBO4BdwPaq2p3k4iRnNMN+H3gCcFWSW5J8pMtMkqSFdXrEsD95xCBJyze4IwZJ0tpj\nMUiSWiwGSVKLxSBJarEYJEktFoMkqcVikCS1WAySpBaLQZLUYjFIklosBklSi8UgSWqxGCRJLRaD\nJKnFYpAktVgMkqQWi0GS1GIxSJJaLAZJUovFIElqsRgkSS0WgySpxWKQJLVYDJKkFotBktRiMUiS\nWiwGSVKLxSBJaum8GJJsTbInye1JLphy/2FJtie5I8lnkmzqOpMkaX6dFkOSQ4BLgS3AccC2JC+a\nM+yNwLer6gXA24Hf7zLT/jQ7O9t3hKmGmMtMS2OmpRtiriFmWpGq6uwH2Ax8bGL5QuCCOWM+Dpzc\n3D4U+MY826qzzpqpvXvvrL7t3XtnnXXWTB199CmDyVQ1zFxmMtP+NsRcQ8403s0vc9+93Acsa+Pw\nKuCyieWzgXfOGXMr8LyJ5TuAp03ZVsFDdcwxv93rk7537511zDG/XfBQwUWDyDTUXGYy08GQa/iZ\nhlcMr55SDO+YM+a2OcXwJeCpU7ZVUAUP1VlnzXTwVC7NuIEfarJcNIhMQ81lJjMdDLmGn2n5xZAa\n73Q7kWQzMFNVW5vlC5uQl0yM+Vgz5oYkhwL3V9Wzpmyru6CStI5VVZYzfkNXQRo3AccmORq4H3gt\nsG3OmD8HzgFuAF4DfGrahpb7D5MkrUynxVBV+5KcB+xg/A6oy6tqd5KLgZuq6i+Ay4H3JbkD+Bbj\n8pAk9aTTqSRJ0toz+E8+J7k8yQNJPt93lkclOTLJp5J8IcmtSc4fQKbDk9yQ5JYm00V9Z3pUkkOS\n3Jzko31neVSSO5N8rnm+buw7D0CSJye5KsnuJLuSnNxznhc2z8/NzZ/fGcj/9d9McluSzyf5QJLD\nBpDpzc3vXa/7g2n7yyRPTbIjyReTfCLJkxfbzuCLAbiC8QfkhuQHwG9V1U8CPwO8acoH9w6oqvo+\n8LKqeilwPHB6kpP6zDThzcAX+g4xxyPAqKpeWlVDeZ7eAVxTVS8G/i6wu88wVXV78/ycAJwIfA/4\ncJ+ZkjwP+HXghKp6CePp8F6nn5Mcx/iDuj/N+HfvHyU5pqc40/aXFwLXVtVPMD6H+7uLbWTwxVBV\n1wEP9p1jUlV9rap2NrcfYvwLvLHfVFBVf93cPJzxL0zv84RJjgReDvxx31nmCAP6/5/kScDfq6or\nAKrqB1X13Z5jTToN+HJV3dN3EMYfhH1Ckg3A44H7es7zYuD6qvp+Ve0DPg38Qh9B5tlfngm8p7n9\nHuAfL7adwfxirFVJfozxq4Qb+k3y2JTNLcDXgE9W1U19ZwLeBvwOAyipOQr4RJKbkpzbdxjg+cA3\nk1zRTN1cluRxfYea8MvA/+g7RFXdB/wn4G7gq8D/rapr+03FbcApzZTN4xm/EDqq50yTnlVVD8D4\nRS3wzMUeYDGsQpInAh8C3twcOfSqqh5pppKOBE5O8pN95knyCuCB5ugqzc9Q/GxV/TTjX+I3Jfn5\nnvNsAE4A3t1M3fw14ymA3iX5EeCVwFUDyPIUxq+AjwaeBzwxyev6zFRVe4BLgGuBa4CdjKeb1yyL\nYYWaw9gPAe+rqj/rO8+kZgpiFtjac5SfA16ZZC/jV5svS/LenjMBj71yoqq+wXjevO/zDPcC91TV\n/2mWP8S4KIbgdOCzzXPVt9OAvVX17Wba5k+Bn+05E1V1RVWdWFUjxlM5d/QcadIDSZ4NkOQ5wNcX\ne8BaKYahvdoE+BPgC1X1jr6DACR5xqPvNmimIE4D9vSZqareWlWbqur5jE8Qfqqq/mmfmQCSPL45\n2iPJE4B/yHg6oDfNof49SV7YrDqV4Zyw38YAppEadwObk/xokjB+nno9SQ+Q5JnNn5sYn1/o8/ma\nu7/8KPD65vY5wKIvZLv+5POqJbkSGAFPT3I3cNGjJ+h6zPRzwFnArc2cfgFvraqP9xjrucB7mkud\nHwJ8sKqu6THPkD0b+HBzmZUNwAeqakfPmQDOBz7QTN3sBd7Qc57JFxn/vO8sAFV1Y5IPAbcADzd/\nXtZvKgCuTvI0xpl+raq+00eIaftL4D8CVyX5FcbF+ppFt+MH3CRJk9bKVJIk6QCxGCRJLRaDJKnF\nYpAktVgMkqQWi0GS1GIx6KCXZN/EpaVvTvKv9uO2j05y6/7annQgDP4DbtIB8L3m+kRd8cNCWlM8\nYpDmudxKkq8kuaT5Qpjrkzy/Wb8pybVJdib5ZHNpcZI8K8mfNutvSbK52dSG5mqptyX5eJLDm/Hn\nN1/Is7P5xKo0CBaDBI+bM5U0ecmAB5svhHk34y/SAbgU+O9VdTxwJfCuZv07gdlm/QnArmb9C4B3\nVdXfAb4DvKpZfwFwfDP+X3b1j5OWy0ti6KCX5LtVdcSU9V9h/K14dzZX072/qp6Z5BvAc6pqX7P+\nvqp6VpKvAxur6uGJbRwN7Gi+PYvm/MWGqvoPSa5h/K1oHwE+UlXf6/5fKy3OIwZpYTXP7fnGTPP9\nidv7+OG5vVcwPvo4AbipuQCi1Dv/I0oLX9L9l5s/Xwt8prn9V4wvRQ1wNnBdc/ta4NfgsW/Te9Ii\n299UVZ9m/IU8RwBPXH50af/zXUkS/GiSmxnvwAv4eFW9tbnvqUk+B/wNPyyDNwN/kuQtwDf44eWx\nfwO4LMkbGX+D168y/prVv3VE0UxBvT/JEc3f+46BfcezDmKeY5Dm0ZxjOLGqvt13FulAcipJmp+v\nmnRQ8ohBktTiEYMkqcVikCS1WAySpBaLQZLUYjFIklosBklSy/8HqyqrM3IfkrwAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1ecaa445f8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Accuracy visuliziation\n",
    "#Plotting Traning loss and validation loss\n",
    "\n",
    "loss_values = history_dict['loss']\n",
    "val_loss_values = history_dict['val_loss']\n",
    "epochs = range(1, len(loss_values) + 1)\n",
    "plt.plot(epochs, loss_values, 'bo') # \"bo\" is for \"blue dot\"\n",
    "plt.plot(epochs, val_loss_values, 'b+') # b+ is for \"blue crosses\"\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Graph shows the loss_valus was always zero from epochs 1 to 10 but validation loss value was increasing with every passing epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEPCAYAAABRHfM8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGbNJREFUeJzt3X2UHXWd5/H3B8KDAiLDigoaWEEX0KOIChl19Cq4xJGV\nGTmOYRkB8Yy6DuLODgh4nLXRwRWP86CDrkcH8AHdgIy6rDqS4YTrw+6IUQgIJhCNQAKIugKKjgyE\n7/5xK3D30p1Uh1TuTef9OqdPV/3q6du3u+tz61e3qlJVSJK0MduNuwBJ0tbBwJAktWJgSJJaMTAk\nSa0YGJKkVgwMSVIrnQZGkvOT3Jnkug3M8+Ekq5IsT3LIUPuJSW5KcmOSE7qsU5K0cV0fYVwIHDXT\nxCSvBPavqqcDbwY+1rTvAfxX4AXA4cC7k+zeca2SpA3oNDCq6lvAXRuY5Rjg0828VwG7J3kig5BZ\nUlX3VNXdwBJgYZe1SpI2bNznMPYB1gyNr23aRttva9okSWMy7sDINOM1TTtNuyRpTOaNeftrgacO\njT8FuL1p7420XzndCpIYJJK0CapqujfnM9oSRxhh+iMGgMuAEwCSLADurqo7gcuBVyTZvTkB/oqm\nbQYF3Mvxx09RVWP7Ov74KeDepp53T0Rd1mRN20Jd1rQpNW2CLosDPsfgiOE+4FbgDQw+DfWmoXnO\nA34IXAscOtR+ErAKuAk4YQPbKLi39t//z2v16ptrnFavvrn23//PC+4tePdE1GVN1rQt1GVNm1IT\nVbPdp892gUn7Aur446fG/k+03urVN9fxx0/Vfvu9ZGLqsiZr2twmsS5rml1N22xgTKIrr7xy3CU8\ngjW1Y03tTWJd1tTOpgRGBsttvZLU1v4zSNKWloSawJPekqQ5wMCQJLViYEiSWjEwJEmtGBiSpFYM\nDElSKwaGJKkVA0OS1IqBIUlqxcCQJLViYEiSWjEwJEmtGBiSpFYMDElSKwaGJKkVA0OS1IqBIUlq\nxcCQJLViYEiSWjEwJEmtGBiSpFY6D4wkC5OsTHJTkjOmmT4/yRVJrk2yNMneQ9POTXJ9khuS/G3X\ntUqSZtZpYCTZDjgPOAp4JnBckgNHZvsg8Mmqeg7wHuD9zbK/C7ywqp4FPAs4LMlLuqxXkjSzro8w\nDgNWVdUtVXU/sBg4ZmSeg4GlAFXVH5pewM5JdgYeA8wD7uy4XknSDLoOjH2ANUPja5u2YcuBYwGS\nvAbYNckeVfVtoA/cAdwGXF5VN3ZcryRpBvM6Xn+maauR8dOB85KcBHyDQTg8kGR/4EBg72Y9VyS5\nvKq+NbrCqamph4Z7vR69Xm9z1C5Jc0a/36ff7z+qdaRqdP+9+SRZAExV1cJm/EygqurcGebfBVhR\nVfOTnAbsVFXnNNP+AviXqvrgyDLV5c8gSXNREqpqujf1M+q6S2oZcECSfZPsCCwCLhueIcmeSdYX\nfRZwQTN8K/DSJNsn2QF4KbCi43olSTPoNDCqah1wCrAEuAFYXFUrkpyd5Ohmth5wY5KVwF7AOU37\npcBq4PvANcA1VfWVLuuVJM2s0y6pLcEuKUmavUnskpIkzREGhiSpFQNDktSKgSFJasXAkCS1YmBI\nkloxMCRJrRgYkqRWDAxJUisGhiSpFQNDktSKgSFJasXAkCS1YmBIkloxMCRJrRgYkqRWDAxJUisG\nhiSpFQNDktSKgSFJasXAkCS1YmBIkloxMCRJrXQeGEkWJlmZ5KYkZ0wzfX6SK5Jcm2Rpkr2Hpj01\nyeVJfpDk+iTzu65XkjS9VFV3K0+2A24CjgBuB5YBi6pq5dA8lwCXVdVFSXrAyVV1QjPtSuC9VbU0\nyWOBB6vqtyPbqC5/Bkmai5JQVZnNMl0fYRwGrKqqW6rqfmAxcMzIPAcDSwGqqr9+epKDgO2rav20\n34yGhSRpy+k6MPYB1gyNr23ahi0HjgVI8hpg1yR7AM8A7knyD0m+l+TcJLNKQ0nS5jOv4/VPt4Mf\n7T86HTgvyUnAN4DbgAcY1PZi4BAGoXMJcBJw4egKp6amHhru9Xr0er1HW7ckzSn9fp9+v/+o1tH1\nOYwFwFRVLWzGzwSqqs6dYf5dgBVVNT/J4cB/q6qXN9P+GDi8qt42soznMCRplibxHMYy4IAk+ybZ\nEVgEXDY8Q5I9h7qazgIuGFp2jyR7NuMvB37Qcb2SpBl0GhhVtQ44BVgC3AAsrqoVSc5OcnQzWw+4\nMclKYC/gnGbZB4HTgKVJrm3m/USX9UqSZtZpl9SWYJeUJM3eJHZJSZLmCANDktSKgSFJasXAkCS1\nYmBIkloxMCRJrRgYkqRWDAxJUisGhiSpFQNDktSKgSFJasXAkCS1YmBIkloxMCRJrRgYkqRWDAxJ\nUisGhiSpFQNDktSKgSFJasXAkCS1YmBIkloxMCRJrRgYkqRWNhoYSU5JssembiDJwiQrk9yU5Ixp\nps9PckWSa5MsTbL3yPTdkqxN8uFNrUGS9Oi1OcJ4ErAsySXNzj9tV55kO+A84CjgmcBxSQ4cme2D\nwCer6jnAe4D3j0x/L9Bvu01JUjc2GhhV9S7g6cD5wEnAqiTvS7J/i/UfBqyqqluq6n5gMXDMyDwH\nA0ubbfWHpyd5HrAXsKTFtiRJHWp1DqOqCvhJ8/UAsAdwaZIPbGTRfYA1Q+Nrm7Zhy4FjAZK8Btg1\nyR7NkcwHgdOB1kc1kqRuzNvYDElOBU4Efg78PXB6Vd3fdDetAt6xocWnaauR8dOB85KcBHwDuI1B\nKL0V+EpV3db0gs0YGlNTUw8N93o9er3eBn8mSdrW9Pt9+v3+o1pHBgcPG5gheQ9wflXdMs20g6pq\nxQaWXQBMVdXCZvxMBgcs584w/y7Aiqqan+Qi4MXAg8BuwA7AR6vqnSPL1MZ+BknS/y8JVTWr3ps2\ngbEAuKGqftWM7wYcXFVXtShoe+BG4AjgDuA7wHHDIZNkT+AXVVVJ/hJ4oKqmRtZzIvC8qjp1mm0Y\nGJI0S5sSGG3OYfx34N6h8V83bRtVVeuAUxictL4BWFxVK5KcneToZrYecGOSlQxOcJ/TsnZJ0hbU\n5ghjeVUdMtJ2XVU9u9PKWvIIQ5Jmr6sjjNVJTk2yQ/P1dmD1ppUoSdpatQmMtwAvZPDppbXA4cCb\nuixKkjR5NtolNenskpKk2duULqk212HsDLyRwa09dl7fXlUnz7pCSdJWq02X1GcY3E/qKODrwFOA\nX3VZlCRp8rT5lNQ1VfXc9Z+MSrID8M2qWrBlStwwu6Qkafa6+pTU/c33u5M8C9idwfUSkqRtyEbP\nYQAfb56H8S7gMmBX4C86rUqSNHE2GBjNDQZ/WVV3Mbgx4NO2SFWSpImzwS6pqnqQDd+NVpK0jWhz\n0vv9DG5tfjGD+0gBUFW/6La0djzpLUmz19Xdan88TXNV1UR0TxkYkjR7nQTGpDMwJGn2urrS+4Tp\n2qvq07PZkCRp69bmY7UvGBremcHDkK4GDAxJ2obMuksqye7AxesfuzpudklJ0ux1daX3qN8A/3YT\nlpMkbcXanMP4X8D6t/DbAQcDl3RZlCRp8rT5WO1Lh0YfAG6pqrWdVjULdklJ0ux18ikp4Fbgjqr6\nbbORxyTZr6pu3oQaJUlbqTbnMD4PPDg0vq5pkyRtQ9oExryq+tf1I83wjt2VJEmaRG0C42dJXr1+\nJMkxDO4tJUnahrQJjLcA70xya5JbgTOAN7fdQJKFSVYmuSnJGdNMn5/kiiTXJlmaZO+m/TlJ/k+S\n7ydZnuSP2m5TkrT5tb5wL8muzfytn+fdPE/jJgZXh98OLAMWVdXKoXkuAS6rqouS9ICTq+qEJE8H\nHqyqHyV5MvA94MCq+uXINvyUlCTNUicX7iV5X5LHV9W9VfWrJHsk+cuW6z8MWFVVt1TV/cBi4JiR\neQ4GlgJUVX/99KpaVVU/aobvAH4KPKHldiVJm1mbLqlXVtXd60eap+/9fsv17wOsGRpf27QNWw4c\nC5DkNcCuzSNhH5LkMGCH9QEiSdry2lyHsX2SnarqPhhchwHs1HL90x3ujPYfnQ6cl+QkBo+BvY3B\nBYI023sygxsdvn6mjUxNTT003Ov16PV6LcuTpG1Dv9+n3+8/qnW0udL7HcCrgQubpjcwOOfwgY2u\nPFkATK2/UWGSMxk8fOncGebfBVhRVfOb8d2APnBOVX1hhmU8hyFJs9TJld5V9YEk1wFHMjhi+Bqw\nb8v1LwMOSLIvcAewCDhupOg9gV80e/2zgAua9h2ALwGfmiksJElbTtu71f6EwdXexzL4xNOKNgtV\n1TrgFGAJcAOwuKpWJDk7ydHNbD3gxiQrgb2Ac5r2PwJeDJyU5JokVyd5dst6JUmb2YxdUkmewcNH\nBP8XuBg4raraHl1sEXZJSdLsbdZneid5EPgm8Maq+mHTtrqqnvaoK92MDAxJmr3NfR3GsQy6oq5M\n8okkRzD9p54kSduANp+S2gX4AwZdUy8HPgV8saqWdF/exnmEIUmzt1m7pGbYwO8ArwVeV1Uvn2V9\nnTAwJGn2Og+MSWRgSNLsdXIvKUmSwMCQJLVkYEiSWjEwJEmtGBiSpFYMDElSKwaGJKkVA0OS1IqB\nIUlqxcCQJLViYEiSWjEwJEmtGBiSpFYMDElSKwaGJKkVA0OS1IqBIUlqxcCQJLXSeWAkWZhkZZKb\nkpwxzfT5Sa5Icm2SpUn2Hpp2YrPcjUlO6LpWSdLMOn2md5LtgJuAI4DbgWXAoqpaOTTPJcBlVXVR\nkh5wclWdkGQP4LvAoUCA7wGHVtU9I9vwmd6SNEuT+Ezvw4BVVXVLVd0PLAaOGZnnYGApQFX1h6Yf\nBSypqnuq6m5gCbCw43olSTPoOjD2AdYMja9t2oYtB44FSPIaYNfm6GJ02dumWVaStIXM63j90x3u\njPYfnQ6cl+Qk4BsMguGBlssCMDU19dBwr9ej1+vNvlJJmsP6/T79fv9RraPrcxgLgKmqWtiMnwlU\nVZ07w/y7ACuqan6SRUCvqt7STPsYcGVVXTyyjOcwJGmWJvEcxjLggCT7JtkRWARcNjxDkj2TrC/6\nLOCCZvhy4BVJdm+6qF7RtEmSxqDTwKiqdcApDE5Y3wAsrqoVSc5OcnQzWw+4MclKYC/gnGbZu4D3\nMvik1FXA2c3Jb0nSGHTaJbUl2CUlSbM3iV1SkqQ5wsCQJLViYEiSWjEwJEmtGBiSpFYMDElSKwaG\nJKkVA0OS1IqBIUlqxcCQJLViYEiSWjEwJEmtGBiSpFYMDElSKwaGJKkVA0OS1IqBIUlqxcCQJLVi\nYEiSWjEwJEmtGBiSpFYMDElSKwaGJKmVzgMjycIkK5PclOSMaaY/NcnSJFcnWZ7klU37vCSfTHJd\nkhuSnNl1rZKkmXUaGEm2A84DjgKeCRyX5MCR2d4FXFxVhwLHAR9t2l8L7FhVzwaeD7w5yfwu65Uk\nzazrI4zDgFVVdUtV3Q8sBo4ZmedB4HHN8OOB25rhAnZJsj3wWOA+4Jcd1ytJmkHXgbEPsGZofG3T\nNuxs4PVJ1gBfBt7WtF8K/Aa4A7gZ+GBV3d1ptZKkGc3reP2Zpq1Gxo8DLqyqv0myALiIQffV4cAD\nwJOAPYFvJrmiqm4eXeHU1NRDw71ej16vtzlql6Q5o9/v0+/3H9U6UjW6/958mgCYqqqFzfiZQFXV\nuUPzXA8cVVW3NeM/BBYAU8A/V9Vnm/bzgX+sqktHtlFd/gySNBcloaqme1M/o667pJYBByTZN8mO\nwCLgspF5bgGOBEhyELBzVf0cuBV4edO+C4MQWdlxvZKkGXQaGFW1DjgFWALcACyuqhVJzk5ydDPb\nacCfJFkOfBY4sWn/CLBbcwRyFXB+VV3fZb2SpJl12iW1JdglJUmzN4ldUpKkOcLAkCS1YmBIklox\nMCRJrRgYkqRWDAxJUisGhiSpFQNDktSKgSFJasXAkCS1YmBIkloxMCRJrRgYHXmUzynRGPm7k6Zn\nYHTEnc7WaxJ/d5NYk7Y9BobGyh1hO75O7fladafrZ3pvU/r9h/9Yzz774fZeb/ClR+r3J+O18Xc3\nd0zK39SwSaxpU8yJwJiaGnwf9z/36PbX16XJN4m/u60hxObKjrBrc+V1mlOBoa3D1rAjnASTGGKj\nJmVH6N/UljEnAmMSTeIf6aT8c0/6jnASXiPNziT+TU1iiA3XtCkMjI5M4k5nUgJj0k3iazRJNU3i\njnASTWKIDdc0/Ltry8DQWLmDaWeSXqdJ3BEOm6TXaq4xMOa4SX83OAk1aG6ZxL+pSaxpU3QeGEkW\nAn/L4JqP86vq3JHpTwU+BTy+meesqvrHZtqzgY8BjwPWAS+oqn/tuua5ZNLfDWrrNld2hF2bK69T\npxfuJdkOOA84CngmcFySA0dmexdwcVUdChwHfLRZdnvgM8CbqupZQA+4v8t6N6f+BF49dPPN/XGX\n8AiT+DpZ02z0x13AI0ziazWJNW2Krq/0PgxYVVW3VNX9wGLgmJF5HmRwBAGDo4zbmuF/D1xbVdcD\nVNVdVVUd17vZTOYfSH/cBTzCJL5O1tTeJNZlTd3pOjD2AdYMja9t2oadDbw+yRrgy8DbmvZnACT5\nWpLvJjm941rnvP32G3cFkrZmXQdGpmkbPUo4Driwqp4KvAq4qGmfB7yomf57wB8meVlXhUqSNixd\n9vIkWQBMVdXCZvxMoIZPfCe5Hjiqqm5rxn8EHA4c0bSf3LS/C/iXqvqrkW1sNd1UkjRJqmq6N/Uz\n6vpTUsuAA5LsC9wBLGJwxDDsFuBI4FNJDgJ2qqqfJ7kcOD3JzsADwEuBvx7dwGx/YEnSpuk0MKpq\nXZJTgCU8/LHaFUnOBpZV1ZeB04BPJPkzBifAT2yWvTvJXwPfbdq/sv7jtpKkLa/TLilJ0tyx1T5A\nKcn5Se5Mct24a1kvyVOSLE3ygyTfT3LqBNS0U5KrklzT1PTucde0XpLtklyd5LJx17JekpuTXNu8\nXt8Zdz0ASXZP8vkkK5LckOTwMdfzjOb1ubr5fs+E/K3/WZLrk1yX5LNJdpyAmt7e/N+NdX8w3f4y\nyR5JliS5McnlSXbf2Hq22sAALmRwQeAkeQD4L1V1MPC7wJ9Oc6HiFlVV9wEvq6rnAocAr0xy2Dhr\nGvJ24AfjLmLEg0Cvqp5bVZPyOn0I+GpVHQQ8B1gxzmKq6qbm9TkUeB7wa+CL46wpyd4MPpJ/aFU9\nm0F3+6Ix1/RM4I3A8xn87/2HJPuPqZzp9pdnAldU1b8DlgJnbWwlW21gVNW3gLvGXcewqvpJVS1v\nhu9l8I89et3JFldVv2kGd2LwjzT2fsgkTwF+H/j7cdcyIkzQ/0WS3YDfq6oLAarqgar65ZjLGnYk\n8KOqWrPRObu3PbBLknnAY4Hbx1zPQcC3q+q+qloHfB34w3EUMsP+8hgGt2Wi+f4HG1vPxPxjzDVJ\n9mPwruKq8VbyUNfPNcBPgH+qqmXjrgn4G+B0JiC8RhRweZJlSf5k3MUATwN+nuTCpgvo40keM+6i\nhrwO+B/jLqKqbgf+CriVwd0i7q6qK8ZbFdcDL2m6fh7L4A3SU8dc07C9qupOGLzZBZ6wsQUMjA4k\n2RW4FHh7c6QxVlX1YNMl9RTg8CQHj7OeJK8C7myOxsL0F3iOywur6vkM/rn/NMmLx1zPPOBQ4CNN\nF9BvGHQljF2SHYBXA5+fgFoez+Ad877A3sCuSf7jOGuqqpXAucAVwFeB5Qy6rbdaBsZm1hwOXwp8\npqr+57jrGdZ0ZfSBhWMu5UXAq5OsZvDu9GVJPj3mmoCH3mlRVT9j0C8/7vMYa4E1VfXdZvxSBgEy\nCV4JfK95rcbtSGB1Vf2i6f75AvDCMddEVV1YVc+rqh6DLqFVYy5p2J1JngiQ5EnATze2wNYeGJP2\n7hTgAuAHVfWhcRcCkOTfrP/0Q9OVcSSwcpw1VdU7q2p+VT2NwYnJpVV1wjhrAkjy2ObokCS7MLgB\n5vXjrKnpMliT5BlN0xFMzgcFjmMCuqMatwILkuycJAxep7F+OAAgyROa7/MZnL8Y5+s1ur+8DDip\nGT4R2Ogb3K32AUpJPsfglud7JrkVePf6E4NjrOlFwPHA95tzBgW8s6q+NsaynszgKvrtGLxBuLiq\nvjrGeibZE4EvNrebmQd8tqqWjLkmgFOBzzZdQKuBN4y5nuE3H28ady0AVfWdJJcC1zB4DMI1wMfH\nWxUA/5DkdxjU9NaqumccRUy3vwTeD3w+yckMAve1G12PF+5JktrY2rukJElbiIEhSWrFwJAktWJg\nSJJaMTAkSa0YGJKkVgwMaQZJ1g3dwvvqJO/YjOveN8n3N9f6pC1hq71wT9oCft3cv6krXgSlrYpH\nGNLMpr3tTJIfJzm3eVDPt5M8rWmfn+SKJMuT/FNzC3eS7JXkC037NUkWNKua19x99vokX0uyUzP/\nqc2DkpY3V+hKE8HAkGb2mJEuqeFbJ9zVPKjnIwwecARwHvDJqjoE+Bzwd037h4F+034ocEPT/nTg\n76rqWcA9wLFN+xnAIc38b+nqh5Nmy1uDSDNI8suqetw07T9m8BTDm5u7E99RVU9I8jPgSVW1rmm/\nvar2SvJTYJ+qun9oHfsCS5qnndGcH5lXVe9L8lUGT7H7EvClqvp19z+ttHEeYUibpmYYnmme6dw3\nNLyOh88pvorB0cqhwLLmxpHS2PmHKM1sQ7fOf13zfRHwz83w/2Zwy2+APwa+1QxfAbwVHnr64W4b\nWf/8qvo6gwclPQ7YdfalS5ufn5KSZrZzkqsZ7NgL+FpVvbOZtkeSa4Hf8nBIvB24IMlpwM94+Dbk\n/xn4eJI3Mnji2n9i8LjcRxyBNF1ZFyV5XLPdD03YM7y1DfMchjRLzTmM51XVL8Zdi7Ql2SUlzZ7v\nsrRN8ghDktSKRxiSpFYMDElSKwaGJKkVA0OS1IqBIUlqxcCQJLXy/wBGvHBMBKCM+QAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f1eb8cbf358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Plotting traning and validation accuracy\n",
    "\n",
    "plt.clf() # clear figure\n",
    "acc_values = history_dict['acc']\n",
    "val_acc_values = history_dict['val_acc']\n",
    "\n",
    "plt.plot(epochs, acc_values, 'bo')\n",
    "plt.plot(epochs, val_acc_values, 'b+')\n",
    "plt.xlabel('Epochs')\n",
    "plt.ylabel('Accuracy')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above graph shows traning accuracy was always 100% but validation accuracy was some what linear and close to 87%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  1.00000000e+00],\n",
       "       [  9.43055212e-01],\n",
       "       [  1.00000000e+00],\n",
       "       ..., \n",
       "       [  9.99999046e-01],\n",
       "       [  4.03508671e-10],\n",
       "       [  9.90481555e-01]], dtype=float32)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's test our model on test data\n",
    "model.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can do some experiments for increasing the accuracy. \n",
    "1. Try to use more than 2 hidden layers \n",
    "2. Increase and decrease the size of epochs\n",
    "3. Try different activalition function, optimizer and loss function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}