Keras - StarCraft Unit Classification from disk.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Star Craft classification from disk \n",
    "The objective of this guide is to show how to train a model from disk using keras. This is useful if your training or testing set is numerous and you just can not have a numpy array with all the values.  \n",
    "\n",
    "In addition you will see:  \n",
    "* Convolutional Neural Networks\n",
    "* Save a model \n",
    "\n",
    "The origin of the images can be found here **[1]**. \n",
    "  \n",
    "* [1]http://starcraft.wikia.com/wiki/List_of_StarCraft_II_units  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import zipfile\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import keras\n",
    "from PIL import Image\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Flatten\n",
    "from keras.layers import BatchNormalization\n",
    "from keras.layers import Conv2D, MaxPooling2D\n",
    "from keras.optimizers import SGD\n",
    "from keras.models import load_model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "show the files in the current location"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "${f}\t\t\t\t\t\t\tSC2Units\r\n",
      "get_images_from_sc2_html\t\t\t\tSC2UnitsByRace.zip\r\n",
      "Keras - StarCraft Unit Classification from disk.ipynb\tsc2_units_files\r\n",
      "Keras - StarCraft unit prediction from disk - FC.ipynb\tsc2_units.html\r\n",
      "notes\t\t\t\t\t\t\tSC2Units.zip\r\n",
      "README.md\r\n"
     ]
    }
   ],
   "source": [
    "#First get the data\n",
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will work with the zip file **SC2UnitsByRace.zip**. This zip File contains a folder with the following structure:  \n",
    "* SC2Units/\n",
    "* SC2Units/Protoss/\n",
    "* SC2Units/Zerg/\n",
    "* SC2Units/Terran/  \n",
    "Inside each of them there are images that correspond to those classes. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File Name                                             Modified             Size\n",
      "SC2Units/                                      2018-04-19 09:27:32            0\n",
      "SC2Units/Protoss/                              2018-04-19 09:26:54            0\n",
      "SC2Units/Protoss/Icon_Protoss_High_Templar.jpg 2018-04-18 17:10:20         5513\n",
      "SC2Units/Protoss/Icon_Protoss_Cybernetics_Core.jpg 2018-04-18 17:10:28         5002\n",
      "SC2Units/Protoss/Icon_Protoss_Sentry.jpg       2018-04-18 17:10:24         5574\n",
      "SC2Units/Protoss/Icon_Protoss_Templar_Archives.jpg 2018-04-18 17:10:34         4544\n",
      "SC2Units/Protoss/Icon_Protoss_Phoenix.jpg      2018-04-18 17:10:22         4471\n",
      "SC2Units/Protoss/Icon_Protoss_Oracle.jpg       2018-04-18 17:10:36         6254\n",
      "SC2Units/Protoss/Icon_Protoss_Photon_Cannon.jpg 2018-04-18 17:10:30         5094\n",
      "SC2Units/Protoss/Icon_Protoss_Dark_Shrine.jpg  2018-04-18 17:10:28         3521\n",
      "SC2Units/Protoss/Icon_Protoss_Mothership.jpg   2018-04-18 17:10:22         7162\n",
      "SC2Units/Protoss/Icon_Protoss_Robotics_Facility.jpg 2018-04-18 17:10:32         5560\n",
      "SC2Units/Protoss/Icon_Protoss_Gateway.jpg      2018-04-18 17:10:30         6217\n",
      "SC2Units/Protoss/Icon_Protoss_Observer.jpg     2018-04-18 17:10:22         3976\n",
      "SC2Units/Protoss/Icon_Protoss_Warp_Prism.jpg   2018-04-18 17:10:26         6252\n",
      "SC2Units/Protoss/Icon_Protoss_Tempest.jpg      2018-04-18 17:10:36         5841\n",
      "SC2Units/Protoss/Icon_Protoss_Immortal.jpg     2018-04-18 17:10:22         5891\n",
      "SC2Units/Protoss/Icon_Protoss_Adept.jpg        2018-04-18 17:10:38         4032\n",
      "SC2Units/Protoss/Icon_Protoss_Probe.jpg        2018-04-18 17:10:24         3983\n",
      "SC2Units/Protoss/Icon_Protoss_Carrier.jpg      2018-04-18 17:10:18         5183\n",
      "SC2Units/Protoss/Icon_Protoss_Stargate.jpg     2018-04-18 17:10:34         4726\n",
      "SC2Units/Protoss/Icon_Protoss_Zealot.jpg       2018-04-18 17:10:26         5263\n",
      "SC2Units/Protoss/Icon_Protoss_Stalker.jpg      2018-04-18 17:10:24         3970\n",
      "SC2Units/Protoss/Icon_Protoss_Nexus.jpg        2018-04-18 17:10:30         6425\n",
      "SC2Units/Protoss/Icon_Protoss_Disruptor.jpg    2018-04-18 17:10:38         5803\n",
      "SC2Units/Protoss/Icon_Protoss_Mothership_Core.jpg 2018-04-18 17:10:38         6545\n",
      "SC2Units/Protoss/Icon_Protoss_Pylon.jpg        2018-04-18 17:10:32         5528\n",
      "SC2Units/Protoss/Icon_Protoss_Colossus.jpg     2018-04-18 17:10:20         5077\n",
      "SC2Units/Protoss/Icon_Protoss_Fleet_Beacon.jpg 2018-04-18 17:10:28         5583\n",
      "SC2Units/Protoss/Icon_Protoss_Archon.jpg       2018-04-18 17:10:18         6894\n",
      "SC2Units/Protoss/Icon_Protoss_Dark_Templar.jpg 2018-04-18 17:10:20         5368\n",
      "SC2Units/Protoss/Icon_Protoss_Assimilator.jpg  2018-04-18 17:10:26         5411\n",
      "SC2Units/Protoss/Icon_Protoss_Warp_Gate.jpg    2018-04-18 17:10:34         5539\n",
      "SC2Units/Protoss/Icon_Protoss_Twilight_Council.jpg 2018-04-18 17:10:34         5139\n",
      "SC2Units/Protoss/Icon_Protoss_Robotics_Bay.jpg 2018-04-18 17:10:32         5655\n",
      "SC2Units/Protoss/Icon_Protoss_Forge.jpg        2018-04-18 17:10:30         5336\n",
      "SC2Units/Zerg/                                 2018-04-19 09:27:16            0\n",
      "SC2Units/Zerg/Icon_Zerg_Drone.jpg              2018-04-18 17:11:02         4348\n",
      "SC2Units/Zerg/Icon_Zerg_Hydralisk.jpg          2018-04-18 17:11:04         4249\n",
      "SC2Units/Zerg/Icon_Zerg_Baneling_Nest.jpg      2018-04-18 17:11:12         5629\n",
      "SC2Units/Zerg/Icon_Zerg_Overlord.jpg           2018-04-18 17:11:06         4907\n",
      "SC2Units/Zerg/Icon_Zerg_Queen.jpg              2018-04-18 17:11:06         5162\n",
      "SC2Units/Zerg/Icon_Zerg_Nydus_Network.jpg      2018-04-18 17:11:16         5795\n",
      "SC2Units/Zerg/Icon_Zerg_Hatchery.jpg           2018-04-18 17:11:14         6550\n",
      "SC2Units/Zerg/Icon_Zerg_Creep_Tumor.jpg        2018-04-18 17:11:12         5247\n",
      "SC2Units/Zerg/Icon_Zerg_Roach_Warren.jpg       2018-04-18 17:11:18         5335\n",
      "SC2Units/Zerg/Icon_Zerg_Nydus_Worm.jpg         2018-04-18 17:11:18         4497\n",
      "SC2Units/Zerg/Icon_Zerg_Spire.jpg              2018-04-18 17:11:20         5463\n",
      "SC2Units/Zerg/Icon_Zerg_Corruptor.jpg          2018-04-18 17:11:02         5442\n",
      "SC2Units/Zerg/Icon_Zerg_Lair.jpg               2018-04-18 17:11:14         5272\n",
      "SC2Units/Zerg/Icon_Zerg_Viper.jpg              2018-04-18 17:11:22         5437\n",
      "SC2Units/Zerg/Icon_Zerg_Infested_Terran.jpg    2018-04-18 17:11:10         5309\n",
      "SC2Units/Zerg/Icon_Zerg_Broodling.jpg          2018-04-18 17:11:10         4847\n",
      "SC2Units/Zerg/Icon_Zerg_Zergling.jpg           2018-04-18 17:11:08         4182\n",
      "SC2Units/Zerg/Icon_Zerg_Baneling.jpg           2018-04-18 17:11:08         4675\n",
      "SC2Units/Zerg/Icon_Zerg_Infestor.jpg           2018-04-18 17:11:04         4581\n",
      "SC2Units/Zerg/Icon_Zerg_Larva.jpg              2018-04-18 17:11:04         3665\n",
      "SC2Units/Zerg/Icon_Zerg_Spawning_Pool.jpg      2018-04-18 17:11:18         5661\n",
      "SC2Units/Zerg/Icon_Zerg_Ultralisk.jpg          2018-04-18 17:11:08         6464\n",
      "SC2Units/Zerg/Icon_Zerg_Ravager.jpg            2018-04-18 17:11:24         4238\n",
      "SC2Units/Zerg/Icon_Zerg_Infestation_Pit.jpg    2018-04-18 17:11:16         5194\n",
      "SC2Units/Zerg/Icon_Zerg_Swarm_Host.jpg         2018-04-18 17:11:22         5045\n",
      "SC2Units/Zerg/Icon_Zerg_Roach.jpg              2018-04-18 17:11:08         5079\n",
      "SC2Units/Zerg/Icon_Zerg_Brood_Lord.jpg         2018-04-18 17:11:02         5526\n",
      "SC2Units/Zerg/Icon_Zerg_Changeling.jpg         2018-04-18 17:11:10         4047\n",
      "SC2Units/Zerg/Icon_Zerg_Lurker.jpg             2018-04-18 17:11:24         4831\n",
      "SC2Units/Zerg/Icon_Zerg_Spine_Crawler.jpg      2018-04-18 17:11:20         4716\n",
      "SC2Units/Zerg/Icon_Zerg_Greater_Spire.jpg      2018-04-18 17:11:20         5499\n",
      "SC2Units/Zerg/LurkerDen_SC2_Icon1.png          2018-04-18 17:11:26         7297\n",
      "SC2Units/Zerg/Icon_Zerg_Overseer.jpg           2018-04-18 17:11:06         4583\n",
      "SC2Units/Zerg/Icon_Zerg_Hive.jpg               2018-04-18 17:11:14         5386\n",
      "SC2Units/Zerg/Icon_Zerg_Spore_Crawler.jpg      2018-04-18 17:11:22         4683\n",
      "SC2Units/Zerg/Icon_Zerg_Hydralisk_Den.jpg      2018-04-18 17:11:16         5829\n",
      "SC2Units/Zerg/Icon_Zerg_Ultralisk_Cavern.jpg   2018-04-18 17:11:22         5783\n",
      "SC2Units/Zerg/Icon_Zerg_Extractor.jpg          2018-04-18 17:11:12         5697\n",
      "SC2Units/Zerg/Icon_Zerg_Evolution_Chamber.jpg  2018-04-18 17:11:12         5509\n",
      "SC2Units/Zerg/Icon_Zerg_Mutalisk.jpg           2018-04-18 17:11:04         4877\n",
      "SC2Units/Terran/                               2018-04-19 09:27:06            0\n",
      "SC2Units/Terran/Banshee_SC2_Icon1.jpg          2018-04-18 17:10:38         5094\n",
      "SC2Units/Terran/CommandCenter_SC2_Icon1.jpg    2018-04-18 17:10:52         6187\n",
      "SC2Units/Terran/Viking_SC2_Icon1.jpg           2018-04-18 17:10:48         4993\n",
      "SC2Units/Terran/Hellbat_SC2-HotS_Icon1.jpg     2018-04-18 17:10:58         5897\n",
      "SC2Units/Terran/MULE_SC2_Icon1.jpg             2018-04-18 17:10:44         5766\n",
      "SC2Units/Terran/Hellion_SC2_Icon1.jpg          2018-04-18 17:10:40         3949\n",
      "SC2Units/Terran/Liberator_SC2-LotV_Icon1.jpg   2018-04-18 17:11:00         4927\n",
      "SC2Units/Terran/Armory_SC2_Icon1.jpg           2018-04-18 17:10:50         5320\n",
      "SC2Units/Terran/Raven_SC2_Icon1.jpg            2018-04-18 17:10:44         4157\n",
      "SC2Units/Terran/Reaper_SC2_Icon1.jpg           2018-04-18 17:10:44         4077\n",
      "SC2Units/Terran/Cyclone_SC2-LotV_Icon1.jpg     2018-04-18 17:11:00         5158\n",
      "SC2Units/Terran/SupplyDepot_SC2_Icon1.jpg      2018-04-18 17:10:58         4069\n",
      "SC2Units/Terran/Thor_SC2_Icon1.jpg             2018-04-18 17:10:46         6340\n",
      "SC2Units/Terran/SiegeTank_SC2_Icon1.jpg        2018-04-18 17:10:46         5415\n",
      "SC2Units/Terran/Reactor_SC2_Icon1.jpg          2018-04-18 17:10:58         4603\n",
      "SC2Units/Terran/SCV_SC2_Icon1.jpg              2018-04-18 17:10:46         4905\n",
      "SC2Units/Terran/TechLab_SC2_Icon1.jpg          2018-04-18 17:10:58         4517\n",
      "SC2Units/Terran/Bunker_SC2_Icon1.jpg           2018-04-18 17:10:50         4240\n",
      "SC2Units/Terran/Point_defense_drone_SC2_Icon1.jpg 2018-04-18 17:10:48         4207\n",
      "SC2Units/Terran/PlanetaryFortress_SC2_Icon1.jpg 2018-04-18 17:10:52         5027\n",
      "SC2Units/Terran/Barracks_SC2_Icon1.jpg         2018-04-18 17:10:50         4522\n",
      "SC2Units/Terran/GhostAcademy_SC2_Icon1.jpg     2018-04-18 17:10:54         4513\n",
      "SC2Units/Terran/Starport_SC2_Icon1.jpg         2018-04-18 17:10:56         5228\n",
      "SC2Units/Terran/Medivac_SC2_Icon1.jpg          2018-04-18 17:10:44         5010\n",
      "SC2Units/Terran/Auto-turret_SC2_Icon1.jpg      2018-04-18 17:10:48         4669\n",
      "SC2Units/Terran/FusionCore_SC2_Icon1.jpg       2018-04-18 17:10:54         4914\n",
      "SC2Units/Terran/MissileTurret_SC2_Icon1.jpg    2018-04-18 17:10:54         4673\n",
      "SC2Units/Terran/OrbitalCommand_SC2_Icon1.jpg   2018-04-18 17:10:52         5357\n",
      "SC2Units/Terran/Ghost_SC2_Icon1.jpg            2018-04-18 17:10:40         4344\n",
      "SC2Units/Terran/Marauder_SC2_Icon1.jpg         2018-04-18 17:10:42         4549\n",
      "SC2Units/Terran/Battlecruiser_SC2_Icon1.jpg    2018-04-18 17:10:40         6112\n",
      "SC2Units/Terran/WidowMine_SC2-HotS_Icon1.jpg   2018-04-18 17:11:00         4283\n",
      "SC2Units/Terran/Marine_SC2_Icon1.jpg           2018-04-18 17:10:42         4457\n",
      "SC2Units/Terran/Refinery_SC2_Icon1.jpg         2018-04-18 17:10:56         4838\n",
      "SC2Units/Terran/SensorTower_SC2_Icon1.jpg      2018-04-18 17:10:56         3549\n",
      "SC2Units/Terran/EngineeringBay_SC2_Icon1.jpg   2018-04-18 17:10:52         4709\n",
      "SC2Units/Terran/Factory_SC2_Icon1.jpg          2018-04-18 17:10:54         5679\n"
     ]
    }
   ],
   "source": [
    "zf = 'SC2UnitsByRace.zip'\n",
    "\n",
    "#To see the content of the zip file we list its content with the printdir() function.\n",
    "tz = zipfile.ZipFile(zf)\n",
    "tz.printdir()\n",
    "tz.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will create a Dataframe that will have in one column the location to the images and in another a vector, **hot encoding**, for the three races.  \n",
    "For this purposes the last folder to the right in each file name will represent the class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New DataFrame:\n",
      "                                                 file   target\n",
      "0     SC2Units/Protoss/Icon_Protoss_High_Templar.jpg  Protoss\n",
      "1  SC2Units/Protoss/Icon_Protoss_Cybernetics_Core...  Protoss\n",
      "2           SC2Units/Protoss/Icon_Protoss_Sentry.jpg  Protoss\n"
     ]
    }
   ],
   "source": [
    "target_col = 'target' #target_class_column_name\n",
    "file_col = 'file' #file_location_column_name\n",
    "hot_key_col = 'y_col' #Column that will hold the hot key encoding\n",
    "\n",
    "list_to_df = []\n",
    "tz = zipfile.ZipFile(zf)\n",
    "for file in tz.namelist():\n",
    "    if \"jpg\" in file:\n",
    "        list_to_df.append({file_col:file,target_col:file.split('/')[-2]})\n",
    "tz.close()\n",
    "location_df = pd.DataFrame(list_to_df)\n",
    "print(\"New DataFrame:\\n\",location_df.head(3))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we define the hot enconding manually. We could use pandas to_dummies function too.  \n",
    "Nonetheless, I like to keep track of the conversion.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hot key dictionary:\n",
      " {'Zerg': array([[0., 1., 0.]]), 'Protoss': array([[1., 0., 0.]]), 'Terran': array([[0., 0., 1.]])} \n",
      "\n",
      "DataFrame with hot key encoding:\n",
      "                                                 file   target  \\\n",
      "0     SC2Units/Protoss/Icon_Protoss_High_Templar.jpg  Protoss   \n",
      "1  SC2Units/Protoss/Icon_Protoss_Cybernetics_Core...  Protoss   \n",
      "2           SC2Units/Protoss/Icon_Protoss_Sentry.jpg  Protoss   \n",
      "\n",
      "               y_col  \n",
      "0  [[1.0, 0.0, 0.0]]  \n",
      "1  [[1.0, 0.0, 0.0]]  \n",
      "2  [[1.0, 0.0, 0.0]]  \n"
     ]
    }
   ],
   "source": [
    "\n",
    "def get_hot_key_dic(idf,class_column_name):\n",
    "    unique_classes = list(idf[class_column_name].unique())\n",
    "    hot_key_len = len(unique_classes)\n",
    "    hot_key_dic = {}\n",
    "    for _,class_name in enumerate(unique_classes):\n",
    "        hot_key_vec = np.zeros([1,hot_key_len])\n",
    "        hot_key_vec[0,_]=1\n",
    "        hot_key_dic[class_name]=hot_key_vec\n",
    "    return hot_key_dic.copy()\n",
    "\n",
    "hkd = get_hot_key_dic(location_df,target_col)\n",
    "print(\"Hot key dictionary:\\n\",hkd,\"\\n\")\n",
    "\n",
    "#Now create a column with the hot enconding\n",
    "location_df[hot_key_col] = location_df[target_col].apply(lambda x: hkd[x])\n",
    "\n",
    "print(\"DataFrame with hot key encoding:\\n\",location_df.head(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Split the dataframe into training and testing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(88, 4) (22, 4)\n",
      "Train 2 samples:\n",
      "    index                                          file  target  \\\n",
      "0     76    SC2Units/Terran/Hellbat_SC2-HotS_Icon1.jpg  Terran   \n",
      "1    108  SC2Units/Terran/EngineeringBay_SC2_Icon1.jpg  Terran   \n",
      "\n",
      "               y_col  \n",
      "0  [[0.0, 0.0, 1.0]]  \n",
      "1  [[0.0, 0.0, 1.0]]   \n",
      "\n",
      "Test 2 samples:\n",
      "     index                                       file target              y_col\n",
      "88     72       SC2Units/Zerg/Icon_Zerg_Mutalisk.jpg   Zerg  [[0.0, 1.0, 0.0]]\n",
      "89     64  SC2Units/Zerg/Icon_Zerg_Greater_Spire.jpg   Zerg  [[0.0, 1.0, 0.0]]\n"
     ]
    }
   ],
   "source": [
    "def split_df(idf,train_percentage=0.8,shuffle=True,v=False):\n",
    "    \"\"\"\n",
    "    idf: input Dataframe \n",
    "    train_percentage: Perentage of the dataframe that the training set will have.\n",
    "    shuffle: Tell whether shuffle before the split\n",
    "    v: Verbosity, if True print results log.\n",
    "    \"\"\"\n",
    "    tdf = idf.sample(frac=1).reset_index()\n",
    "    rows = tdf.shape[0]\n",
    "    train_rows = int(rows*train_percentage)\n",
    "    test_rows = rows - train_rows\n",
    "    _train_df = tdf.head(train_rows)\n",
    "    _test_df = tdf.tail(test_rows)\n",
    "    if v==True:\n",
    "        print(_train_df.shape,_test_df.shape)\n",
    "    return _train_df,_test_df\n",
    "\n",
    "train_df,test_df = split_df(location_df,train_percentage=0.8,v=True)\n",
    "\n",
    "#Validate if it is working:\n",
    "show_samples=2\n",
    "print(\"Train \"+str(show_samples)+\" samples:\\n\",train_df.head(show_samples),\"\\n\")\n",
    "print(\"Test \"+str(show_samples)+\" samples:\\n\",test_df.head(show_samples))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create a function to load an image from a file location and in addition we resize it to a shape **(64,64)**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(32, 32, 3)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD8CAYAAABXXhlaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXl8XFeV53+nSqV9XyzJlmV5X+I1cZzFJBhnIQk0a0gT\naDrdHT5umMCEhpkmGWgGaGiSnpkOzYce6JCkoXsg7CHE0EkcJyYL8RYv8b7JirWvJam013LnD5Xf\nuedJVS7LUklOne/n449P6d5679ard+udc89yyRgDRVFSC890D0BRlOSjE19RUhCd+IqSgujEV5QU\nRCe+oqQgOvEVJQXRia8oKcglTXwiuo2IThDRaSJ6YLIGpSjK1EITDeAhIi+AkwBuAdAAYA+Au40x\nRydveIqiTAVpl/DeDQBOG2NqAYCIfgrg/QBiTnwi0jBBC4rTphdKmSjGmHi3FoBLm/hzANRbrxsA\nXHMJx5txuK/ehCYjyaN4LA0rzdVG1hlHTITP6xqI/fM5ZkyUYL8k8nb6gXu7fJZLmfjjXYOx9yHR\nFgBbLuE8iqJMMpcy8RsAzLVeVwFocncyxjwK4FHg8lP13YOlOG2xOpJrDcWT5nPkEfdBwkHu57W+\nmkhYjivOo1w22Wu3EUyMBI8RRz1K9FqNYQbeLTNwSBPiUlb19wBYTETziSgdwEcB/HZyhqUoylQy\n4Se+MSZERJ8B8BwAL4AnjDFHJm1kiqJMGRN2503oZJeZqu9mYqq+bPJaqn4oQVXfuFX9OHq0HOPM\nUPXjcpmp+pcDU72qn9LE/RGw/uDxSmsqFLImt0e21SxY4Mi1tbWxz215A0zcmR8Z/++uMcZngrMv\n4V9JZTrQkF1FSUF04itKCqI2fhzGaMcxLCePq8F+6b6+9235pCMX9gdE20u/e8aRMyorHPlwU5vo\n197Tx+cmr2iLwFoPuAz8aDNjFJfCzLNpErHx9YmvKCmITnxFSUF04itKCqLuvLgkZoF606SdHQyF\nHPnuP/u4aCvOzXbknb/7vWh7/F++4ciFs0sd+bHfvCT6ffvH/L7ODmn/p1kuwpCJ7Xcna/wm3udM\n1ISNe6liP1/MhOMLpot4z8qZYeMngj7xFSUF0YmvKCmIqvpxia262dFztmoPAPOqZjuyv12q4t/f\n+jtH/so9HxRtyzZf58idZ0868q233Cj6DRTMc+Rvff2bcsQRjgxM3NM0BSpqTPXepdpfdv48t2ly\neT47L89RK4pySejEV5QURCP34jA2r4X/QmT9ZhqZPVdVWuzIS5YuEW1XrVrBL/paRVtpaYEjb7j1\nNke+5oabRb9XntvuyA8/9Ihoe+0wlzwcGRlx5Ehk+lbPLztt/jJHI/cURRkXnfiKkoLoxFeUFETd\neXGIa5tajR6XoTo4NOTI/q520fb7p37uyCXZ6aKtcNlaR666o9yR/9vnHxT9fIdfd+QPrl4p2o6d\nPOXILZaNTxT700x8UxX7IGNaEzzIDKkBniDu65jMNbLJRJ/4ipKC6MRXlBREVf04jHWAsVpHVgJM\nxKXVDoTYvZeWJi/xoqWLHLm3s1O0rV2z2pFf3cpFOc6+KpN0Pnkluwg760+LtqtyWH6ml8fo/qLt\nzzZWWR1fhx9rLsRL9PFifNxXleK0zTwuV9XejT7xFSUF0YmvKCmITnxFSUE0ZDceCdeid/9+sq2a\n7jrGyvmcuVdYVCDb1q1z5OP79vHfi3NFv02L5jtyeYa03snLJ/zWm82O/NQLcp1AbtghbWtZpIOJ\nd6+Mdd7Zu4qw7HF1tLcWEPsFAggGOetxoiHHsZyKk3UjzrxSm5MUsktETxBRGxEdtv5WTETbiOhU\n9P+iSx2soijJIxFV/4cAbnP97QEA240xiwFsj75WFOUy4YLuPGPMy0RU4/rz+wFsiso/ArADwBcn\ncVwzg4T3f3OpoVYRihFX3bsDtbyTeHaGLNKRWcDRem81+x3Z69pkb9UqVrCuWrNUtAX9HCkY/gOb\nCz6XKy4YlsVDxPBj/D1e9J+7xd4ePGzJ6T5XRy+7/XLzpEkzu3KOIw8ODzty/Vv1ot/wCLdFwvJ6\n25/F3rLMbd4kytgrMBn7EyafiS7ulRtjmgEg+v+syRuSoihTzZQH8BDRFgBbpvo8iqIkTkKr+lFV\nf6sxZmX09QkAm4wxzURUCWCHMWZpnEOcP85MWfhMiLiL+vZKtXH3s1bFya1U2Ukp8o3FRazqRzys\nExdlyyg4087mwpc+/eeibf/uXY6cWcnHy8rOEf32vMk1/ZqHpYqaW1bF47C27t712iuin8/a8jsY\ncqu5nICUW1joyEtWXiF65RWw2XL0wOuiLTLCW4xFiJ9Rc6vmin5dHRwB6fXKxKfGhnOOHAryZ5ms\nZBuylOaZUip8Kgtx/BbAPVH5HgBPT/A4iqJMA4m4854E8DqApUTUQET3AngIwC1EdArALdHXiqJc\nJiSyqn93jKabJnksiqIkCc3OmyiWSRgv34zi+ATdLQMDbNPWzGU7u/Wts6LfdWs5O++1Q8dE2+mm\nLkdeWlDiyHfMl8rd/Z95jyPvxTzRNrCYf9NbG2oduf1cnehXW89utfTi5aItc/61jnz9zbc78gdu\n3yT6HTjwpiOPeOQYqwq4oMmp03yujhZZ3KS1lV+vXnulaCuZxQ6nxrd4/K0tzYjFxdj/M8Wuv1g0\nVl9RUhCd+IqSgmiSTjzi+PMoTkfp9ovzkV1eF1vFzMnMcuTSwjzRryQ305HLiotFW3YO9/3jTnaP\nXb+oVPT70w/x9l2+eatF2zNvsKvv/e99tyO3nzwk+v3mOU788WfViDYqWePIhVlljpzn8n2ePM3n\nGu56WbStXJjhyM0NHOW4bMky0e+NfQcdub9/ULR1dnY7ck4uRwYO9PaKfm2trPqP/don97ZNsBph\n/F2Mx+ntSFpXX1GU8dCJrygpiE58RUlB1MafBC5mb7h4fcnD3tXcIg5zLcrNEP1yDdfLHxqQtmpB\nIdv8fmvH7HnLpB3/oTs/6sgHd/1BtM2v5GN89O67HLlvULqu+sMcHvu0lQkIAPsOdzhyVxMPJCss\nr0hPN7sL+4cPi7a8PHbnzV+wwJFbW2RWY1kJhyZ3d/WItqo5nOG3Z88BPkaTdAkOBNiVGo4MY2ag\nNr6iKJOITnxFSUFU1Z8EJkvVtzP+quZxBlomjYhu6YOszma6Yi9nzeEoPFPMxwgPSzdXupd/830+\nmdF2+3vYhTcywmp6ZZWM8Ft33UZHfv6Pe0XbszvYTTcQ4AjC+lPHRb/G2hcdubh4QLS1dnC0XqmV\naZibJWsV5mewC7O4UGYhVs7hyL3s7HxH3v68zDQ8ceSEIxsThOTyum1V1VcUZVx04itKCqJJOpPA\nWEXQE6d1/NLVAEAeXr0PjFgr/CWyiHFOIauvXW1yddrfzCr9DUs5Yq7TtV3Xrl17HHndOpnYcuAI\n77hrDG8Hlpsl1egzOznS7p3z54u2XF6Qx3d/8J+O3HLmpOg3rzzbkQuL80VbRT5/zp4ONm/yfdmi\nX3MLF9uYPVt+lrQ0jnIMBPq4gWTNQWPsnYVdhU9M7A3H7GjLRM3mhKu2TyH6xFeUFEQnvqKkIDrx\nFSUFURt/wsTePEm0uAw6E/MF4E3n3+GiYnZZNbXLaLTiPLZxM0rmiDZ/L9uxB/bvd+TNmzeLfq1W\nMYvde2TUXWZxhSP39nF223CH3IbrzzZtcOT9e2Vm3YkG3hfg2hqOQhyo7xD9rqhZ7MgFvizRhl5e\nXzh8jsdxw81Xi27b9rEbcP9++VlmV/D1KSvhNY/aM7WIxUW5uGUqZqyGuJD1vsnOBIyFPvEVJQXR\nia8oKYhG7l0UNK5McO82y7iLKdgXIEMGzMGTxq05Baxup2dKN9dgHyfmzCopFG3ZOZZ7rIDfl58t\nE30iYSvRZ0SO8cDZFj7XMLu9Ni6sEP1WWJr52UO7RdsPnnySz13D9fg+/dnPi35DVq371QUyMjBj\ngK/r2WE2TV48JtX56nUrHNmXLq3XxrpGRz51nNX7wICMZIyEw4hJwhr8RPfOndw9dzVyT1GUcdGJ\nrygpiE58RUlB1MaPS+ySmvFNPev31Ljrrlt2fL7cFrp/wMoK83Co6ac+9wXRb7/lptu17XeiLTOL\nbfxli2Y78vorrxL9aus4zPXMW42iraiCs/oCAbaFPQNdol+o9S1HzvPIENhla9Y68tWbP+TIN26+\nXfSrPcP7AvhPnBZtjWcaHPl0ExfDPFB/VPRbeCUX6fD3+kVbRRm7837/zHOOPDwsi23EzbC8mPTL\nGCReYHOCnD+BmSQbn4jmEtFLRHSMiI4Q0f3RvxcT0TYiOhX9v+hCx1IUZWaQiKofAvAFY8xyANcC\nuI+IVgB4AMB2Y8xiANujrxVFuQxIZO+8ZgDNUTlARMcAzAHwfgCbot1+BGAHgC9OyShnCpYCRZaJ\nZMg3TudoP4q9wdbgkHSx5VfyFtKbbr7FkWs7pea28rb/4sh5S/5EtJ068Bq/b+e/8bl65Tg6ejl9\nLq9AmhyZlqtveIRdhxnZsr5/dyEX2MgqlW2/38WFLd5oeNWRW0fkbuqZbNFg3d3vE205bexiO/jA\n/3Pk4jSpEO/eyUVAFq5ZINrOnmUTISytERfju2pHMTHkONtkuyM2Y+7JIDP8bDkSmbrtuS5qcY+I\nagCsA7ALQHn0R+H8j8Os2O9UFGUmkXCsPhHlAvgVgM8ZY3rdGwvGed8WAFsmNjxFUaaChJ74ROTD\n6KT/sTHm19E/txJRZbS9EkDbeO81xjxqjFlvjFk/GQNWFOXSueATn0Yf7Y8DOGaM+Ser6bcA7gHw\nUPT/p6dkhNOKtOdieT49kMaj13LnRVxvChvLJhyRRTQ3bb7GkWdXcW37H3z/F6JfTRvb05vvule0\n0Sy2oUNedtO9+aasWb/xWt7GuqOlTrTVneSQ2FCYx5uVXyn6BQ2vUZysbxFtucXsSlx+BWcG/u6F\nI6LfZ/7mA46cv07W/s8cZht34ybOOnx5269kPy+7FQ/uPSjazCDHRQuTOU7W5ISdznH9vVajR55c\n2PLW/eJxadWRmJmAGMdtHJ9EVP2NAD4B4BARnd+R4H9gdML/nIjuBXAOwEcu6syKokwbiazqv4rY\n8Qc3Te5wFEVJBlqIIw5jnDq26mUVZEwzUtUfidiZXrIwpDefo+LWXLNWtN1y83WOXFrOmXB5hTIr\n7p+//qgjH58v3VcLrn2nIzcc54IVVxdViX4nDnBd+UgwINoystm9l5XGqnKaz+V+zOTX+YVlou3U\nSS7uefBNjjT83Jf/p+i3ajWbNGcbG0RbzRwe87pbedvt3zz7mOhnvPxdpLtcjr3drY5MsNyuYx5l\nth4tl76E63bMu6y/WGacO2NTvMfVNGcWX7s5pewi3XtU7kFAlgnpNjutwL2E0Fh9RUlBdOIrSgqi\nqr47EcdS4ceoa7bKZ+lrI5AVNao33ObIkdIrRFvlwmWO/CcbrxNt+09zzXnfOV7FLs2QX9OCRWw+\nnN27XbRlla105MFsPldPtzRH2kLWCr2nRLRVlLAKP9DOxSsG+1pFv/YeroMXisjj19QsceS8PE4+\n+s4//b3oV/gEX9OFGxaKtg989JOOvPs13um2cK6szVcW4oIjbxw6J9oCrhr5DnFW3ceqy/EUaH6f\nx+oXcZ/Xul9K0uR99eW7OHFp7VVsnt15/4OiX3Of3GJMHv/ilH194itKCqITX1FSEJ34ipKCaCGO\nMb99iUVAedLYNfT++x4WbfNuv8+Rm7tlwYdwK9vJLWfkvndhD7uvQn0cCdd79AnRL1C/w5FrNnxS\ntA2VciRcDp21jidr84cjbHdnZcpCk51nOcPv8EuPO3KaK0Ixv6zUkb2uyqHtDeyamzuPi4AsWiJr\n4ge6ubhHe5ssgEkF6/j4meymW+CThTg+82EutvH0Cy+Ktid+/0c+htdaGwjLc9lFUSOupS8CXyt3\nioqxXHhpVluQpOszG3y+f/jQNaJt1SxeY/nDGe63u0de72d3sgvW65H3bThy/uRhLbapKMr46MRX\nlBTkbezOG1PuwJIjMWTgyqvYHdbZ1S/aWjq5KMW738N15K55x22i39YXuTBEKCxV4PQhdoEFB6Xl\n0xfiBMeMCLvz5lRKd9upFv69DgalOphhbQuNYU4CMlkysm6wl7eyGmmXW1fXHeZIu8JKVrfXXLlG\n9KtZyKr+wX2vi7bMTK7EVn+aa+l1++V23YVFvC9AeflG0VZaw+7OkOWmSu+UNQKffZ7V+5aGM6LN\nji7sHWKzK64u7LZITdzsGwevddSgke68Eus2uHGR/D6DPWxKnGrn76zTVyr6+TJ4m/LIsLw3iUan\ncqKWuz7xFSUF0YmvKCmITnxFSUHeZjZ+vIKJbMvn5HLI67tuulH0WruKw1zdrrht23h/uMOn2c48\n99j3RL9AN9tiIRSItkiEwy6zMlyuvjC/bmzh4hLF6TJU1gzxWsNgtwxRLZzNbjSvhwtqDoakeynP\ny+Nor5XbXw90sU1euuJmR6658hbRj/o4eyzD5V5auoKLasyyatufOrpf9GtsqOcxRmShj0Xr2GXq\n7+LP1eyXRT8iXnaB1cxfLNpG0nlN5Xgjvy8wOCT69Vjbi8OVbSlKbbq3NrfWA7xeazqFgqLf6is4\nhLkkP1O0nbWSI/1ZfA06QjKz0+Ph79BA2viOn/HCnrzRYyXUS1GUtxU68RUlBUm6qp9IDlG8qubx\nsEuZGde7Zs+vduRP3PuXjpzmkZeg9gyrzkUFOaJt6RJ2ie3cuceR5xdLtW4owMcYHJHjGOxmd9bq\nMlmP/5pcduVULmF3WG/5fNGvt/y9jvyLrTJSrbqdVdjmAnY5mhGpXubm8PVo97rNEdY9y7LYzGjc\n8xPRL3T6DUdeUipdT28dYzMgQHzueVfUiH79vVxkxBOShTh2PPNXjrx5I9cInFcjr3drHUceHjl0\nVrRRJkfrLavkc3V3y+3A2jxsCrYFpBlgvPw9+bzSTdc/yCZTJGx/18WiX1oxu0K7c1xuuoV8/Ydb\nWL2P9MhxRKyNAdw7OYSdDNE4231b6BNfUVIQnfiKkoIkXdVPRG2faCaPXH2VR6koYzWvsoxXTnv6\nZXGD9k5OnDFGrrr39rEKnGHVnwuOyOi/CPFlzc2TKnZlKUeqzfLJc79jebkjz87mUK/XA9L48Zax\nmj5/iazbd/Qoq7reDRyFGDJyJ9qKEvYM+LNkmW8Cf7acPDY5+tvrRb+vP/gZR16+dIlo+9SXuBL7\nwcOswmePyPp++Xn8OQuLZL28ikou0b3nBH8uX0BG/1UX8PWeM09GKHb6OTmpv5u/2+CQNBcKCln9\nbnV5Skoy+Ppff53cHuJkHXtfWlr5+JFhqXLXNbJ34d92SxMys5Bfl+azKdHtbxb9Gq0kKXlnAjDn\nP48W4lAUJQY68RUlBdGJrygpyIx05yV8LFdVBNuuL5slbb3mFo7aevwxLmzR0e0X/dauYbdLmk+6\nbnyWW8eOkPJ3ymOQ5Vf0d0t7tKiK7dbuQWmpHa5le9HM5vGPeOU21p1tbDPf9M4Nou0HB37Ixxhk\nOz4/XRb9KMnmSLX8fNEEY/i2aGnmjME733WV6JddwG6o227/U9E2mM5utE0br+exd8iou4wstusz\nSD6Hikp4feHsOf7MuT7pzDpVz1GUnjoZdZedxx+uOI/PleuVhTj6BzkSbvNVciuvinz+LDke+b78\nas60q53D6y09HfK7vX7zexz5SIc8RuNpXuupqeTvOuxay4hY25cb973vRKZOko1PRJlEtJuIDhLR\nESL6WvTv84loFxGdIqKfEVH6hY6lKMrMIBFVfxjAZmPMGgBrAdxGRNcCeBjAI8aYxQD8AO6NcwxF\nUWYQieydZwCc1wt90X8GwGYAH4v+/UcAvgrge+73TyUej1TFw1Zk0+q10s01ZCdeWHXSsrJlIkT1\nXFbF68/VyWMMsIo2MsKqnMcn1a7qCk5KCTfKohGDA6xSDqVLV98ev+Vis9TvXtMr+rW27nTk0KDc\nndwKUMT+up86cvmCWaJfczOPv2bZMtFW8OabjtzeyuNv8c8T/b7w5W85clO/dI/df+/HHfmvPvku\nR25olEU//uMnXBOv0CvNs+o5/F2cPcm7/da7avNl5HGtwuF+2TY0xKpvUw8XH8knGRVXbSXO5AzJ\n+oSLlnPk5HtulGbAdx75jiO3F9/hyHfeLYuK+K1xRQbl83b5SjbXAl1sknUP7hH97Mg9r2vH3fDF\nbZab2OIeEXmjO+W2AdgG4AyAbmOc2dQAYE6s9yuKMrNIaOIbY8LGmLUAqgBsALB8vG7jvZeIthDR\nXiLaO167oijJ56LcecaYbgA7AFwLoJDICVGrAtAU4z2PGmPWG2PWj9euKEryuaCNT0RlAILGmG4i\nygJwM0YX9l4CcCeAnwK4B8DTiZxwMgvrx9sTwN/dLV5XVPP21BmZHCLZ1iqLXHS2W26vkDSchofZ\njrXP3dMjz1VkheVmZ8t93vKsUN+8XOkIaW3h47c38lpATrbsl+XlfgNdMqMtP4/HNdTEYbqNOfI3\nPq+AQ5iL8+TnzM7n85Xn8md59bXXRL/lC3lB4Y717xBt2/ZxwY3rj/KCxc2bZKGMyoIbHPm7Dz8j\n2ura6hz5qgXsNuvsliHGJpOtzNCwzLrLSmdbfsSqjx9w7UO3fj27Kvs65fe5bds2R773zhtE21e+\n+DeO/LHf8JblR+vkMbqs0OGhDHkNzjXwOtPcal5DWHHdnaJfd9MOR+51fe8ULYRiIokV4kjEj18J\n4Ec0upukB8DPjTFbiegogJ8S0TcA7AfweLyDKIoyc0hkVf9NAOvG+XstRu19RVEuMy6Lmnuxqpob\nV038zCxWqysrKkRb7Wne7tmXw6p+RXm56JeXz5FTIwOyrllHJ0dSZeTwudZeKX8X+wc4Ki7Q1yfa\nRqwiF6EOGWXW3cZmR1kFu7L6/PIYGRF2DZXkykwvGFbTDViFHBqR2WKhML8uLpJ13qWniK940HWM\nhrN1jpxdKI+Rk8/uw8ef4GIhBS6/09W3sivxLz8po/++8aV/ceSVG9g9O9B9QvQrX8bnbm+RGYQ9\nbVxnPy+XvzOTJk2w5/cdceRSn5wW+bns6nvJtUXXjTducuR7Psj7ALz8yk7RLxjhaMOIkfUPAx1s\njpwOsDmS3i8zGcMmxpbfQOIF9aNorL6ipCA68RUlBbksVH1bifFYiRzuVf2yMo788rtW2rMs1bzF\nUqlXr5BRa6Egq1rBsFRtwxE+X0YGj+PMGRmNFrHG5XFtx1RZxUVAVs6rEW0vPv+8I586x4Un3n39\ndaJf8xFWdX//jFQpfUWs6vdZkY0+I5N0PBWszx87JKPYcnK5XlxeLifitLfIpJG2Lo4azDv4hmhb\nv2KpIx9uY/X+ge/vEP2+ankXTh2SCTzdYVaPS8o5Oq+/71nRr/HcMUcO9MtIRm+Ev8O+fjaZPJmy\n6MexBo7qu2aZjFBcXcPnLiySEZBHrO/i5zv+0ZEzy1aJful57HkYHpEehUwv36vpxPdpr196yPt7\n+Tt0Be4hkmBZbef9F9VbUZS3BTrxFSUF0YmvKCnINNr48WyS2K6JiGF70Zcmh19/joskXnuDzI5a\nvYEjs15/5VVH7umUNqHt5upyRf91+7nvFSt5baCsWNZQ77Ky7DJcbiNfOtvdf9i9S7S1W9tmly/n\n8f7vH/yr6Pejr37ZkdvaZCbZEPH1Of0KR9plDdSIfvmGo8e6+2UkXGX1Qkc2QT5eQUWV6Oev58/5\nzQf+q2jb88J2R+6czesaBzplpuH9X/qxI5emyQIbH/vzv3DkkGWrF5VK1+GZxiPWK3lP2A7TETtB\ns0+6yjJ8vDZyuqlDtJUXsov3bIe8J4KDfO0CIV7X8BXKtaMhD0dAhkPy3AjyMSmdB2mG5bkQ5rWB\nNJdnbySsNr6iKBdAJ76ipCDTWHPvIiKNYmgxkYiMArNr8J06LevI51j124oK2UXV0SxdJm0drOb1\n9EiVrKSY1TVjmQQDARnh5/ez2yunqFC0jYywuzArS0bdtbVxdOGad3FRh/s+/yXRryzA/f7u/vtE\n27FmHvNzp9lt6Q3KRJ+XXv69I+eVyy2d2vx8DQisUy5fvkj0M92cfPPAt2WqxsZrOJr7j69udeSq\nBQtFvz1H+Hv68hfuF237rIIgs6u4GEb5HFn6obbFrj8v7ytjFV1xR3raBIOssjf7penz3B52F+7Y\ne0i0pXvYlNv4Z3/tyD05MnK0vZPvFx9kkcMs66vxEZuT/QG5HZhgTDLOxVXi0Ce+oqQgOvEVJQXR\nia8oKciM3Dsv0TdFxthz/PrQ/oOirXwW21w5VpZWe7sMQ/VaYa6Dg66ijhns1glZtSUHILPshqx9\n2eYUSFdfOMR9T584I9q8ZO2N1nbUkRcurhH9rrvyJkd+8smnRNupVv48hV52v81bKrMQ97/5uiP3\n9ch1Do+x929jm/P4m/JaXbGIQ1sbW6UL7ImnfufI5YUcHtt1Tp5r/XouPHHgrLRpX/rVLx35sX/8\npiOvrpaf5Y977ftAhllTDNvXuJ559r3kDocNWH5AlyMOHiuT78xJri6XVSbdxHNLeW0jJ1euqezb\n+YIjDzZzgc2ut16SJ7PC1YPG/bnOf27dO09RlBjoxFeUFOSyyM6LRTyXoHHVy9v5Kkfr5VvbMVdV\nu1xDJ2utV/LyFFawipbu49/Mc3Wy/lme5S5sb5c14Pr8HH3VH5DbLK26gs2RW6/nMX5wgyxq/Mz/\n5dp0w7KcPQJdXAd/VjqP0dslI+bmlbHZcuicHKO9X0FWJvuaBkLyZPuPsZvrz2+StegqrDqBv3ie\nIwhzFlwh+nV0sstx54mXRVuptefBwgzOIKzLld97TSV/h4UuV9+Bvbsd2eexVWX3M49NBHfWJwnd\n3/W+EJuDbcd/5siz2qU63wOOSuwYkKZhi71/Q8T+LmTWpI1x+bjPmzSJmtL6xFeUFEQnvqKkIJe1\nqj9Wr2H1xx3V1+PnHW3DIVahWpubRb+gtSJfOUcWZOgLcCGHrnZ+X3Z2gehnR+QFw1Kd77IiA4dG\npCr35lFivEPzAAAZsUlEQVQ2GTZezar+i9vlyn3dWU5GqnapzhHD44+MWAVCAnK7roEevlb9AzLj\nIzOfow19lrngyXB5UUY4ieTIPlkQ5KYP/4kjV/31Xzjyd7f+QfRDD6vK87JlJGOFtVvxkX2c0NTd\nJROTTIjV9I1Xy/qvVVZ58+detk0J9+o/WbL8nHZhFbeXwGv4OCvncNum6xeIfkdPciLRc6++Kdpg\nJSBRrAKTY18m3DYe+sRXlBREJ76ipCA68RUlBUm+jc/peRfsciHGHsL+i/so/LovMIhEaGp4K2Yb\niZ9MaSP7T/F6Qn+vX7TZWVUekvXVI5at+toujhFb/n5Z1KEpyC5HT6+0d7sCnFkWyuMIt86gLPBI\n2Wzz5wdlBmF22Uo+vhWV2NN1XPQbtApFHHFd0k8/9mtH3rSGj/e//urDot+8HHbZHTggC5Mgg6/P\n7iO8XrG/QbofS4r4GJk9smDnhzdd68gj/by+8tqRY6LfwDDb5+7ClXa9VLcL2ZfJ17G4hK+3u7hJ\ne7d1HYdc7kJiV58Raw+u6LyJ1a4Zl4Sf+NGtsvcT0dbo6/lEtIuIThHRz4go/ULHUBRlZnAxqv79\nAOyfyYcBPGKMWQzAD+DeyRyYoihTR0KqPhFVAXgPgG8C+DyNVrzYDOBj0S4/AvBVAN+bgjHG1mIo\ntu4z1u1iu2uYsakOfEm8rpr4ZLlu7J2gutplQob9ayqdaLIGXMjISDjbYFhi1dzvMzIarauE3YyB\nHBkhdhKs+q8qYxOhs01uLdVYX8fnLZDKmi+bx9XRxe7IiEt9LctiFbu2VZocn97yKUfuOcWurL/7\nu6+Lfptv3ezIxWnyORTu5yjEviB/a4VV0vTxebnIyut1crsxP/F3tnrjXY78yqH/I/oZw5GNY+8q\niiHLW5ACbEqcbZD9dh7j6z8UiX2MRE3gmJ7sBFX+RJ/43wbwt+B5UgKg2xhz/j5uADBnvDcqijLz\nuODEJ6L3AmgzxthbpYz3qB33t4aIthDRXiLaO167oijJJxFVfyOA9xHRHRjVXPMxqgEUElFa9Klf\nBaBpvDcbYx4F8CgAENHFBhgpijIFXHDiG2MeBPAgABDRJgD/zRjzcSL6BYA7AfwUwD0Ank7ojAlM\nfSN9ZdLCsgoQuH9GbC+M2yUT8bIFbYdnRozMlEKEX4ddx/d4rXFZIcFE8jKSld0Wjsjj21lV5JVu\nQBO09nk7zkU6fIUytHe4nt15585I2z00wG7A7ja2Oftdut2wVVwit1iOMTPMLrHMES4k2huUrkmf\ntX10pWu77pef5jDjq1ZzWPFIkVyTePI1LpiyrEIW2KjJ5uszZK2AHDgq3azBubyleNlcuWV5b661\nR16QfY4FpbJwaLCN161DocTcvQAQCfJ3U9/AaxINwzIU3B+w1hDI9b271npiEXcn7Isrq39JATxf\nxOhC32mM2vyPX6C/oigzhIsK4DHG7ACwIyrXAtgQr7+iKDOTacvOc2smorDAmKyk8eumecbU3LOO\n71KnIrY6Hopd4GC2FdEW9kk3V2tPr7v76LlcKnvYKgISdn8Ye1xBqeLlWUO8/SpWURcYGY32lZtZ\nTR0ckefOSWNzYcjayuvXRxpFv20tfO5wWG5dNWRF/1UWs8vODMl+Zxo5m7AgV247fbqBx9zUwsU2\nsrPlNS3Is7aWcpln5wYs0yfEn7M3Is2KwgweY0vTOdG2uHSJI3sjrMLnZMjr1mNt3zVW1Y+tYw9a\nVtLOJjaLxkT/WT47Y8Jx2pKzDKax+oqSgujEV5QUZNpU/XgJNu5CCKKXpUGFvVJt9Fgr/pGwXKnO\nsRJgFtXMZbm6UvS79xMfd+QNa1eJtv/+tW858n9s3cbjDcliG3Ny2FzwReRnGbHGP+TLFW1XLuPi\nDZVr3sHHCLSKfqXCuyCaxMvIkjJH3nKtVG0LXuawjMdekqXIBxZySfA+a3ffvFkyRqvTiljMzJC7\nAnuK+dYaHubrE5BaLvq62AvRH5LXo3Iel6RuG+DIwKISqep/4S8+6MiP/ujfRdve3Vx8o8raZbe+\nUdZJJMjvUBLv+chXfCQSL7Yuzj09Ger9VCXpKIry9kEnvqKkIDrxFSUFmQYb//xvjWuL6zivbAPG\nY7k+Iq4677brbNUcGQX2kavYrXPd1Vc6cn2vXAv495/y9tGNe18Tbd+6m7euKiS2CXtd23B94gYu\nPJE5KItG9IBtYX+OLOa5t5XdSA/95BVHLsqS18r0ss3v80rbegBcvKKigG3hdy6R57p7M68hzF0o\no9ge+c1/OnJXL7u5BnzSnTdvMb+vp1u6Ov0BLtIxdy5vcd3bJwuCZNi1/7OqRNvSDbc78p5fcHzY\nQlk3BAPnuHhl9pDcyutUJ7smaxZd7cjzl10r+p08xIGnrsBRmDFbUlvY2XnCVne5mmMfQR4uSa49\nfeIrSgqiE19RUpBpq7nn/sURkU4e17CsRJdcyz22ZK5U50sqeAuqpXNnibYuS8V8+Ps/dOTGgIzi\nm73iGkf+1XOyBlx7He9kuuoK7udZWCH6BfpZvTdDUtUvrGQX1XGXifD01mcd+Yq1axx5ELI2X38l\nq6m9aXI33gwr4aN8hGvYv7zrj6Jf7ZlTjvzZz31KtOUMc4LQq/sPOXKgeIno9+IxjgZcvVomx1RU\nsKof8PM1jsDlgrVuhKArXypsbXNVNZtdsB1NMgrx8Z/8ypE7XfUUPdnsIi2eze7ZhnMyuclYOwR7\nPPHU9EQL38UpEuNqslV6jdxTFGXK0ImvKCmITnxFSUGmwZ03asO4TRk7TNdEpJsuy/p5unZZjSO7\nbfz6ZraZd+87Itr8Vp3IyoXXO/LsDJlV5u/grLJZ1atF245WLgDRd4JdXiGPDDX1Wv6g9FBAtGXl\nse1bPyTt3avescmRfVbmXtdIjejX5Hu3Iw9AuvNKQ1z7fiTCbq6M6nzRb38zf87/3HtWjt/HbrVb\nNrKNnL5kreiXNZeLV2x99jnRVpJjjStiXY9M+ZnTLLeoxydvil4/F7OoKOBr3PSWDD8eyeH7YNUi\n+TlP1fJ6QGsthyavWXOl6Dfcwq/P1sq1HY+1EBFxhWDbywHS6xe7Skyy7Ph46BNfUVIQnfiKkoJQ\nMtUOImKFJ45XZI2VPQcAc0q50EJbJ6vz/oCs876whiPEhoelW6c7zBlzg1mLHXkoJDO9TB+rl2nZ\nsir+rDmsAuel8bkHIVX99AIeRzAsP2hOJltXZPpFW7CX1e+uTlaBR3LfIfq1ENewSyN5jEJL1S+z\n9j+pLJZpcfV1bArNWXKdaGvuZPOnNI+vz4feKaP/qkp4jF996CHR9soezv4rnFvjyMMB6cIM9nPE\nX9lcqX5XzePreLaWP1fYqqMPALnWVlvz82QEYV4WX/+MUnbntXTIaMhVVkbi7p3PiLb9R/laedwZ\noWFry7IE6+PHJc4xEj28MSaezxGAPvEVJSXRia8oKUjSV/WdJARX+NICq0TywjmzRVtbM0dZDRhW\n64aypIrdOsTq24LZMpou2MYqcdgqjW28MvEkK59Xhft8shT02R6OBizI499MSpdZIz3tXAADGfKz\npFvlrzNDrtV04kIXI1a56jSvVOfnGvYoZEFuGRVJ4+vam2aZMb11op+x+vkhVeemEla5e6zEnB07\nd4h+f3Mzvy97WEYoklUO2y6JPtQrVX2rCYFuGZF3oJnHXLWQvQte11Zb3kEu+13kKvPd3cMJTXuP\nsQpf5fLY9Hbz9QgMSLMo0/JQDPXHKb09GVZznGNMplGuT3xFSUF04itKCqITX1FSkKS7887LWRky\n4yzPsqN8YWljZafbW15ZDenSnotYnyXiqp0/yyoUWVTGGXIh10bWmcQ2s98ji0vWDbAtT4bdOBk5\nZaJfViFnsfWNZIu2TMte9w7LraC8Xv7cXmt754qMOnn8Qd5/lDzy+IEwu8D8Pe18vDRXjpkVbUg5\nK0Vb+UouXllQwJ/53IuPiH63L+Powqdf2Cba/niMM/xAvE7gcW8XZUU5Rlz3ojedP1vEWq9YPH++\n6FeUw5+lq0sevzfIiwihCN8TeZny/ivK5885FJQuQW8ar8u0NMqsvo52jsRExFVJdJpIxJ2X0OIe\nEdUBCAAIAwgZY9YTUTGAnwGoAVAH4C5jjD/WMRRFmTlcjKr/LmPMWmPM+ujrBwBsN8YsBrA9+lpR\nlMuAhFT96BN/vTGmw/rbCQCbjDHNRFQJYIcxZukFjmPOu/PIFboXsfw6N1wv69kvXsBuo8ggu1O8\nIRlFFQ5ZrjmvVGZ6A6xiFxZwlFYkJCO4hno4ei57rhzHQBqPI4O4akRvUCbKNAb4mBlZ0l0YDnBN\nuHmzZJJRZTGbDD4vq6WtHbWi34B17frD0uU4HOKou7wQ71w+4C0Q/QKGa8yXl0vX1qyiakduaeHx\ntrTKcQz1HnbkM/ufFG19/jpHFnUS42ih7i3R7G+GiFXzcteuuob4+ofTpXmWlsd987P53H3tp+V4\ne1hZLZklE32q5/H9kumTz8ptz77E4w3Gq80/tZz/ZAaTG7lnADxPRG8Q0Zbo38qNMc0YPVEzgFkx\n360oyowi0QCejcaYJiKaBWAbER2/4DuiRH8otlywo6IoSSOhJ74xpin6fxuApzC6PXZrVMVH9P+2\nGO991Biz3lobUBRlmrngE5+IcgB4jDGBqHwrgK8D+C2AewA8FP3/6dhHYZw1BZKntjx98JIMQ21o\nZNtpaJhta8+IPMZID7tyvK41BI+139xgJ7trqstluG15BVssB4/tFG1NDRz+uWIJ25I33PJh0e9r\n3/lnR77rzo2irXohu54CXdL9036WbfLeIV7L6EhfKPr1l77HkTMKq0VbQTp/ts4OdqkVFUrXZ0kp\n18Tv6pfXYGct27vhYd5b0Fcm1wJG8ngNJHx0t2jzos6S+bsYcRUO9cCqsEmuapvCVOV7oKXZ5Qa1\n9iDMLna7Lfk6kpfXP/Kz5TOvs5HXdvp9MkS6t5DXaeo6XSHHIZkhOpVMbGe+8UlE1S8H8FR0US4N\nwE+MMc8S0R4APyeiewGcA/CRizy3oijTxAUnvjGmFsCacf7eCeCmse9QFGWmM22FOAy8rjZrmywj\nXWzXXMXq5oHDXCiDXFrWnAKOwvOG5THsehgZ6ewGJNc204WWS+2mWzeLtu3PsjWTmcbutvZQiejX\nNMDq5b9+917RdnzfPkf+h0e2i7bi6lscuaGNl0y8uTLDL5TGKrbPJ11PaWnsfiusZDNguFHWkVu/\n4WZH9syTpsqJ06xWF2SxidAblKZJRharxI0vfFa09db+gsdk1awbjshIQ4/YnlpG3UUs5da+P+JV\ncTGutrQ067u2i/hHXPeH5Yrz+uS9mVvE32+PX2YhRoKuSMQZgBbiUBRlXHTiK0oKohNfUVKQpFfg\nYUst7Po7WS3SROkbYJvrlht5q+M2ywUDABS2avOTtNNClk3nsX7v3Gsc7b3syqr3S7dOboUVDjrC\nbrOi7MWiX8NZzuB644B0PX3wfZz59pM/yCywjm4Oq80s4AUMb/ik6FeaxW7FsJFViOzrOCvMx88s\nklllrzz1DUe+4S65PfWszGV8/GHOPquQHkF0dHIRyt6uc6LNY41DXuKLWVMy40hjDhiXUHDowp0A\nUREq4trEz9/G1zu+8Ww/RyMxe8XeBP4CHRMhwUujT3xFSUF04itKCjINhTjO/9bEUYXGbCPMsp3s\nNnuOdKPlFVmFG1yq/lAXu9jSLVcfudw68PAJgq6ijnNms1utu4PdOpm50kW19nqO1vv5L2WN9rVX\nsBp913s3ibZ8K/Pr+EmOujthRQwCQEs/q6It3TJDsZ29ecgg3hp8xapFot8Ba4ux6qUfEG0FCz7k\nyN50NiWqMw6Kfide/HtHfmXvK6LN4+EIvYjh8ZJxF6uwTIIxum3se0SJjbrzFEUZF534ipKCTFvN\nvXgrm25V3260m9yKoNfS7t3KTsRaqC0t5GSNomyZNJJmFfXrH5TJQuWlXIgj00r66RqQK8ct/RwF\nVuIqttHWxGp7xJWTUlPNq+tFxXyueUvk1lLhLPYutLZLz0BbU4Mj+3u5VlxWpgxzNBGOciyuvF60\nZZVwTbvmc7zjbtaA9FDU5PMFf/0NuVtuWw8nHNlfmonIZ40dkWfG7DALZQKoqq8oyrjoxFeUFEQn\nvqKkINO3TbaLiYyCSP5uGZHVRzH7GsullJ/lqq+ezYUbPSRdT54w2/JDAc7K8mRJl1p+FWfFnWuQ\nEW0mzMGSeYVyf79QiI85MGBFDabJuv2efN5HrqhARu6V5PKYM/It92N3g+iXlcnFN4pLZeRht1UQ\ntN9/lPt55Ld0xw23O/LvXviNaHv9wB94vNZ6SCTstvGtfQzdqzZq408ItfEVRRkXnfiKkoJMmztv\nko7oep3Y4T0eW+2X6mVmOqv+86plAYzsdCts0DqVJ1PW1R8Ksuvs6BFZkDhiuQvT0+X7Fi9ilbuy\nkl12XQE5xvoONgPSvPK3OzvN8hGmcT14b7rsFw5xvzSP3EYsO5evQVkJj7GvSdbVD7Sxy66uVW5x\n3TfMkZL2NR5zB9iuPleTqvoTQ1V9RVHGRSe+oqQgOvEVJQW5LGx8IruoQ2KHIFfcr3xf7N87+22+\nDOmmi1XAw21RhazCjZFw4hlmGdYaQqZl/2flyrr3ldU1jhwMy+vR3mLt/ZfL2Yth12cmy+7OSHcV\nLbEqkzY1c4ixJ9gh+oUHuGjJ8JivxT4mN9KYQGsap1fsvygXRm18RVHGRSe+oqQgM7IQxyWcwZLj\nfa54v3fJLP4wMXdkcbG1MbErejES4ci9Fau5/n55uXRN9vWyS7ClVdYuPH6Ei3QERwatFhnJKDLr\nxgw9Xh5lrH5uVNWfCJOm6hNRIRH9koiOE9ExIrqOiIqJaBsRnYr+X3ThIymKMhNIVNX/ZwDPGmOW\nYXQ7rWMAHgCw3RizGMD26GtFUS4DLqjqE1E+gIMAFhirMxGdALDJGNMc3SZ7hzFm6QWOZVi1i33e\niSnA473z4o8ykYrGF4OsK5dojbmLGRV/zuVreHfbJQsWiF5nrJp+Z+pkItFgf48je+JE1hljPzfc\nY3TX1lOSxWSp+gsAtAP4NyLaT0SPRbfLLjfGNEdP1AxgVryDKIoyc0hk4qcBuBLA94wx6wD04yLU\neiLaQkR7iWjvBMeoKMokk8jEbwDQYIw5v93qLzH6Q9AaVfER/b9tvDcbYx41xqw3xqyfjAErinLp\nJOTOI6JXAHzSGHOCiL4K4PxmSp3GmIeI6AEAxcaYv73AcSYWuWfJM8fBk+io3OZWom6uSz93Zh4X\nFc30yt3Surv97u4W9vMg3lqMXShzYusr6sybfBKx8RPdO++zAH5MROkAagH8JUbvjp8T0b0AzgH4\nyEQHqihKcrk8YvUteeY8BfSJr0/8mUkiT/zLYuLPTKzJQXEmcNyItum7HHYxkoh7G7GYLscxDr04\nZ0jsc86Mq/H2QpN0FEUZF534ipKC6MRXlBQk0VX9y58JFPSPHzocidVwASa0g0Cc15GYLbJBtth2\nfbyiJRRnvBMLP3YfQ5kO9ImvKCmITnxFSUGSrep3AHgLQGlUTh5jdcoLjiFJamgC1yJxN1rMljhu\n26hqP+444l8DE0O+JJJ/b4zP5TqOeYl0Sqof3zkp0d7pjt2fCWPQceg4pmscquorSgqiE19RUpDp\nmviPTtN5bWbCGAAdhxsdh2RKxjEtNr6iKNOLqvqKkoIkdeIT0W1EdIKITkeLdyTrvE8QURsRHbb+\nlvTy4EQ0l4heipYoP0JE90/HWIgok4h2E9HB6Di+Fv37fCLaFR3Hz6L1F6YcIvJG6zluna5xEFEd\nER0iogPny8RN0z2SlFL2SZv4ROQF8C8AbgewAsDdRLQiSaf/IYDbXH+bjvLgIQBfMMYsB3AtgPui\n1yDZYxkGsNkYswbAWgC3EdG1AB4G8Eh0HH4A907xOM5zP0ZLtp9nusbxLmPMWst9Nh33SHJK2Rtj\nkvIPwHUAnrNePwjgwSSevwbAYev1CQCVUbkSwIlkjcUaw9MAbpnOsQDIBrAPwDUYDRRJG+/7msLz\nV0Vv5s0AtmI06H86xlEHoNT1t6R+LwDyAZxFdO1tKseRTFV/DoB663VD9G/TxbSWByeiGgDrAOya\njrFE1esDGC2Sug3AGQDdxphQtEuyvp9vA/hbcFZPyTSNwwB4nojeIKIt0b8l+3tJWin7ZE788RLH\nUtKlQES5AH4F4HPGmN7pGIMxJmyMWYvRJ+4GAMvH6zaVYyCi9wJoM8a8Yf852eOIstEYcyVGTdH7\niOjGJJzTzSWVsr8YkjnxGwDMtV5XAWhK4vndJFQefLIhIh9GJ/2PjTG/ns6xAIAxphvADoyuORQS\n0fn8jWR8PxsBvI+I6gD8FKPq/renYRwwxjRF/28D8BRGfwyT/b1cUin7iyGZE38PgMXRFdt0AB8F\n8Nsknt/NbwHcE5Xvwai9PaXQaOL74wCOGWP+abrGQkRlRFQYlbMA3IzRRaSXANyZrHEYYx40xlQZ\nY2owej+8aIz5eLLHQUQ5RJR3XgZwK4DDSPL3YoxpAVBPROe3orsJwNEpGcdUL5q4FinuAHASo/bk\nl5J43icBNAMIYvRX9V6M2pLbAZyK/l+chHG8A6Nq65sADkT/3ZHssQBYDWB/dByHAXwl+vcFAHYD\nOA3gFwAykvgdbQKwdTrGET3fwei/I+fvzWm6R9YC2Bv9bn4DoGgqxqGRe4qSgmjknqKkIDrxFSUF\n0YmvKCmITnxFSUF04itKCqITX1FSEJ34ipKC6MRXlBTk/wPkza7DtofoWAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f38daabc6d8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def zip_image_file_to_np(file_name,image_func,zip_file,image_func_kargs):\n",
    "    with zipfile.ZipFile(zip_file) as tz:\n",
    "        with tz.open(file_name) as f:\n",
    "            return image_func(f,**image_func_kargs)\n",
    "\n",
    "def image_file_to_np(ifile,resize=(64,64),batch_shape=False):\n",
    "    \"\"\"\n",
    "    Will return a numpy array.\n",
    "    resize: specify the resize of the image so you could fix a shape for your images.\n",
    "    batch_shape: Normal output will be a numpy array of shape (width, Height , 3).\n",
    "     if batch_shape is set to True, the shape will be (1, width, Height , 3). Useful for batch processing.\n",
    "    \"\"\"\n",
    "    pil_img = Image.open(ifile)\n",
    "    pil_img = pil_img.resize(resize,Image.ANTIALIAS)\n",
    "    np_img = np.asarray(pil_img)\n",
    "    pil_img.close()\n",
    "    if batch_shape==True:\n",
    "        nps = list(np_img.shape)\n",
    "        nps.insert(0,1)\n",
    "        np_img = np_img.reshape(nps)\n",
    "    return np_img.copy()\n",
    "\n",
    "#zf is the variable that holds the zip file.\n",
    "test_img = train_df['file'][0]\n",
    "test_img = zip_image_file_to_np(test_img,image_file_to_np,zf,image_func_kargs={'resize':(64,64),'batch_shape':False})\n",
    "print(np_img.shape)\n",
    "\n",
    "plt.imshow(test_img)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we define a function that will use the numpy image and extract some features. In this case a histogram for the three channels, and will concatenate one after the other. The number of bars is represented by the variable **bins**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a function to create batches from disk  \n",
    "We need to load the content according to the mini-batch size, so the previous functions of loading a numpy array from disk and extracting a histogram must be called only for a batch. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(4, 64, 64, 3) (4, 3)\n"
     ]
    }
   ],
   "source": [
    "def batch_proc_from_df(idf,x_func,x_func_kargs,x_col='file',y_col='target',batch_size=4,offset=4):\n",
    "    \"\"\"\n",
    "    This function will extract a batch of size *batch_size* starting at the poing *offset*\n",
    "    of the dataframe *idf*. Then the values from the column *x_col* will be processed by transformed \n",
    "    using the function *x_func*. The setup parameters of the *x_func* function are specified in \n",
    "    *x_func_kargs*. \n",
    "    The values of the *y_col* column of the *idf* will be stacked vertically and returned.\n",
    "    \"\"\"\n",
    "    start_index = offset\n",
    "    rows = idf.shape[0]\n",
    "    if start_index >= rows:\n",
    "        start_index = start_index%rows\n",
    "    end_index = start_index + batch_size\n",
    "    \n",
    "    if end_index>rows:\n",
    "        end_index = rows\n",
    "    \n",
    "    tdf = idf.iloc[start_index:end_index]\n",
    "    _y = np.vstack(tdf[y_col])\n",
    "    \n",
    "    arrays = list(map(lambda x: x_func(x,**x_func_kargs),tdf[x_col]))\n",
    "    \n",
    "    _x = np.vstack(arrays)\n",
    "    return _x.copy(),_y.copy()\n",
    "\n",
    "func_params={\n",
    "    'image_func':image_file_to_np,\n",
    "    'zip_file':zf,\n",
    "    'image_func_kargs':{'resize':(64,64),'batch_shape':True}\n",
    "}\n",
    "\n",
    "xb,yb = batch_proc_from_df(train_df,zip_image_file_to_np,func_params,\n",
    "                           x_col=file_col,y_col=hot_key_col,batch_size=4,offset=4)\n",
    "\n",
    "print(xb.shape,yb.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The keras model\n",
    "In the keras website we find a model example [2].  \n",
    "\n",
    "* [2]: https://keras.io/getting-started/sequential-model-guide/  \n",
    "\n",
    "### Changing our model  \n",
    "Here we have defined how many bins we want for the histogram, number of iterations (epochs), learning rate, batch-size, the resizing shape for each image.  \n",
    "We use batch normalization after each fully connected layer for a better performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter 0 / 128 Batch 0 / 11 Loss: 2.080329\n",
      "Iter 0 / 128 Batch 1 / 11 Loss: 1.7729788\n",
      "Iter 0 / 128 Batch 2 / 11 Loss: 1.3560543\n",
      "Iter 0 / 128 Batch 3 / 11 Loss: 1.4980118\n",
      "Iter 0 / 128 Batch 4 / 11 Loss: 1.0883739\n",
      "Iter 0 / 128 Batch 5 / 11 Loss: 2.0776749\n",
      "Iter 0 / 128 Batch 6 / 11 Loss: 1.5172718\n",
      "Iter 0 / 128 Batch 7 / 11 Loss: 1.9306554\n",
      "Iter 0 / 128 Batch 8 / 11 Loss: 1.5628343\n",
      "Iter 0 / 128 Batch 9 / 11 Loss: 1.2856245\n",
      "Iter 0 / 128 Batch 10 / 11 Loss: 1.42596\n",
      "Iter 1 / 128 Batch 0 / 11 Loss: 1.4723729\n",
      "Iter 1 / 128 Batch 1 / 11 Loss: 1.1237562\n",
      "Iter 1 / 128 Batch 2 / 11 Loss: 2.0861871\n",
      "Iter 1 / 128 Batch 3 / 11 Loss: 0.75791764\n",
      "Iter 1 / 128 Batch 4 / 11 Loss: 1.2064581\n",
      "Iter 1 / 128 Batch 5 / 11 Loss: 0.9402886\n",
      "Iter 1 / 128 Batch 6 / 11 Loss: 1.6349444\n",
      "Iter 1 / 128 Batch 7 / 11 Loss: 0.7002928\n",
      "Iter 1 / 128 Batch 8 / 11 Loss: 0.6238731\n",
      "Iter 1 / 128 Batch 9 / 11 Loss: 0.860409\n",
      "Iter 1 / 128 Batch 10 / 11 Loss: 1.0814195\n",
      "Iter 2 / 128 Batch 0 / 11 Loss: 1.3730533\n",
      "Iter 2 / 128 Batch 1 / 11 Loss: 0.78191185\n",
      "Iter 2 / 128 Batch 2 / 11 Loss: 1.3119752\n",
      "Iter 2 / 128 Batch 3 / 11 Loss: 0.82754683\n",
      "Iter 2 / 128 Batch 4 / 11 Loss: 1.3817322\n",
      "Iter 2 / 128 Batch 5 / 11 Loss: 1.3056452\n",
      "Iter 2 / 128 Batch 6 / 11 Loss: 1.0555837\n",
      "Iter 2 / 128 Batch 7 / 11 Loss: 0.46475485\n",
      "Iter 2 / 128 Batch 8 / 11 Loss: 1.2477485\n",
      "Iter 2 / 128 Batch 9 / 11 Loss: 0.477997\n",
      "Iter 2 / 128 Batch 10 / 11 Loss: 1.2274702\n",
      "Iter 3 / 128 Batch 0 / 11 Loss: 0.8226558\n",
      "Iter 3 / 128 Batch 1 / 11 Loss: 0.49550498\n",
      "Iter 3 / 128 Batch 2 / 11 Loss: 0.80939245\n",
      "Iter 3 / 128 Batch 3 / 11 Loss: 0.34900403\n",
      "Iter 3 / 128 Batch 4 / 11 Loss: 0.7144946\n",
      "Iter 3 / 128 Batch 5 / 11 Loss: 1.648543\n",
      "Iter 3 / 128 Batch 6 / 11 Loss: 0.2762018\n",
      "Iter 3 / 128 Batch 7 / 11 Loss: 0.42520684\n",
      "Iter 3 / 128 Batch 8 / 11 Loss: 0.4234287\n",
      "Iter 3 / 128 Batch 9 / 11 Loss: 0.5583647\n",
      "Iter 3 / 128 Batch 10 / 11 Loss: 0.5712143\n",
      "Iter 4 / 128 Batch 0 / 11 Loss: 0.22586718\n",
      "Iter 4 / 128 Batch 1 / 11 Loss: 0.44047245\n",
      "Iter 4 / 128 Batch 2 / 11 Loss: 0.9529208\n",
      "Iter 4 / 128 Batch 3 / 11 Loss: 0.59556276\n",
      "Iter 4 / 128 Batch 4 / 11 Loss: 0.37338918\n",
      "Iter 4 / 128 Batch 5 / 11 Loss: 0.76329875\n",
      "Iter 4 / 128 Batch 6 / 11 Loss: 0.8810539\n",
      "Iter 4 / 128 Batch 7 / 11 Loss: 0.17800991\n",
      "Iter 4 / 128 Batch 8 / 11 Loss: 0.6002138\n",
      "Iter 4 / 128 Batch 9 / 11 Loss: 0.52074236\n",
      "Iter 4 / 128 Batch 10 / 11 Loss: 0.5560422\n",
      "Iter 5 / 128 Batch 0 / 11 Loss: 0.48988327\n",
      "Iter 5 / 128 Batch 1 / 11 Loss: 0.37044537\n",
      "Iter 5 / 128 Batch 2 / 11 Loss: 0.96710676\n",
      "Iter 5 / 128 Batch 3 / 11 Loss: 0.6577828\n",
      "Iter 5 / 128 Batch 4 / 11 Loss: 0.43603894\n",
      "Iter 5 / 128 Batch 5 / 11 Loss: 1.0406276\n",
      "Iter 5 / 128 Batch 6 / 11 Loss: 0.46454176\n",
      "Iter 5 / 128 Batch 7 / 11 Loss: 0.42642254\n",
      "Iter 5 / 128 Batch 8 / 11 Loss: 0.36688304\n",
      "Iter 5 / 128 Batch 9 / 11 Loss: 0.8595978\n",
      "Iter 5 / 128 Batch 10 / 11 Loss: 0.69236654\n",
      "Iter 6 / 128 Batch 0 / 11 Loss: 0.29292625\n",
      "Iter 6 / 128 Batch 1 / 11 Loss: 0.5735792\n",
      "Iter 6 / 128 Batch 2 / 11 Loss: 1.0093867\n",
      "Iter 6 / 128 Batch 3 / 11 Loss: 0.27042997\n",
      "Iter 6 / 128 Batch 4 / 11 Loss: 0.9508461\n",
      "Iter 6 / 128 Batch 5 / 11 Loss: 0.30588946\n",
      "Iter 6 / 128 Batch 6 / 11 Loss: 0.94507396\n",
      "Iter 6 / 128 Batch 7 / 11 Loss: 0.02722767\n",
      "Iter 6 / 128 Batch 8 / 11 Loss: 0.2522038\n",
      "Iter 6 / 128 Batch 9 / 11 Loss: 0.55698925\n",
      "Iter 6 / 128 Batch 10 / 11 Loss: 0.23661688\n",
      "Iter 7 / 128 Batch 0 / 11 Loss: 0.16140744\n",
      "Iter 7 / 128 Batch 1 / 11 Loss: 0.14859816\n",
      "Iter 7 / 128 Batch 2 / 11 Loss: 0.71736676\n",
      "Iter 7 / 128 Batch 3 / 11 Loss: 0.26348767\n",
      "Iter 7 / 128 Batch 4 / 11 Loss: 0.28327012\n",
      "Iter 7 / 128 Batch 5 / 11 Loss: 0.19725198\n",
      "Iter 7 / 128 Batch 6 / 11 Loss: 0.28178206\n",
      "Iter 7 / 128 Batch 7 / 11 Loss: 0.14292228\n",
      "Iter 7 / 128 Batch 8 / 11 Loss: 0.3278368\n",
      "Iter 7 / 128 Batch 9 / 11 Loss: 0.22353196\n",
      "Iter 7 / 128 Batch 10 / 11 Loss: 0.35010552\n",
      "Iter 8 / 128 Batch 0 / 11 Loss: 0.25313112\n",
      "Iter 8 / 128 Batch 1 / 11 Loss: 0.34771007\n",
      "Iter 8 / 128 Batch 2 / 11 Loss: 0.6501298\n",
      "Iter 8 / 128 Batch 3 / 11 Loss: 0.18617274\n",
      "Iter 8 / 128 Batch 4 / 11 Loss: 0.25026745\n",
      "Iter 8 / 128 Batch 5 / 11 Loss: 0.9150642\n",
      "Iter 8 / 128 Batch 6 / 11 Loss: 0.13784371\n",
      "Iter 8 / 128 Batch 7 / 11 Loss: 0.13596404\n",
      "Iter 8 / 128 Batch 8 / 11 Loss: 0.17264538\n",
      "Iter 8 / 128 Batch 9 / 11 Loss: 0.5334607\n",
      "Iter 8 / 128 Batch 10 / 11 Loss: 0.9313655\n",
      "Iter 9 / 128 Batch 0 / 11 Loss: 0.13750526\n",
      "Iter 9 / 128 Batch 1 / 11 Loss: 0.5911062\n",
      "Iter 9 / 128 Batch 2 / 11 Loss: 0.52294046\n",
      "Iter 9 / 128 Batch 3 / 11 Loss: 0.34599227\n",
      "Iter 9 / 128 Batch 4 / 11 Loss: 0.28994334\n",
      "Iter 9 / 128 Batch 5 / 11 Loss: 0.42714816\n",
      "Iter 9 / 128 Batch 6 / 11 Loss: 0.687917\n",
      "Iter 9 / 128 Batch 7 / 11 Loss: 0.026777778\n",
      "Iter 9 / 128 Batch 8 / 11 Loss: 0.21170056\n",
      "Iter 9 / 128 Batch 9 / 11 Loss: 0.17702559\n",
      "Iter 9 / 128 Batch 10 / 11 Loss: 0.1859039\n",
      "Iter 10 / 128 Batch 0 / 11 Loss: 0.22668293\n",
      "Iter 10 / 128 Batch 1 / 11 Loss: 0.12878092\n",
      "Iter 10 / 128 Batch 2 / 11 Loss: 0.47518647\n",
      "Iter 10 / 128 Batch 3 / 11 Loss: 0.33215368\n",
      "Iter 10 / 128 Batch 4 / 11 Loss: 0.3529011\n",
      "Iter 10 / 128 Batch 5 / 11 Loss: 0.7261195\n",
      "Iter 10 / 128 Batch 6 / 11 Loss: 0.3538066\n",
      "Iter 10 / 128 Batch 7 / 11 Loss: 0.08109644\n",
      "Iter 10 / 128 Batch 8 / 11 Loss: 0.53143775\n",
      "Iter 10 / 128 Batch 9 / 11 Loss: 0.19199717\n",
      "Iter 10 / 128 Batch 10 / 11 Loss: 0.18061137\n",
      "Iter 11 / 128 Batch 0 / 11 Loss: 0.10207307\n",
      "Iter 11 / 128 Batch 1 / 11 Loss: 0.29936102\n",
      "Iter 11 / 128 Batch 2 / 11 Loss: 0.49457932\n",
      "Iter 11 / 128 Batch 3 / 11 Loss: 0.21507302\n",
      "Iter 11 / 128 Batch 4 / 11 Loss: 0.18844947\n",
      "Iter 11 / 128 Batch 5 / 11 Loss: 0.38885486\n",
      "Iter 11 / 128 Batch 6 / 11 Loss: 0.9656564\n",
      "Iter 11 / 128 Batch 7 / 11 Loss: 0.04617452\n",
      "Iter 11 / 128 Batch 8 / 11 Loss: 0.1848766\n",
      "Iter 11 / 128 Batch 9 / 11 Loss: 0.18915662\n",
      "Iter 11 / 128 Batch 10 / 11 Loss: 0.37066028\n",
      "Iter 12 / 128 Batch 0 / 11 Loss: 0.26231393\n",
      "Iter 12 / 128 Batch 1 / 11 Loss: 0.10571212\n",
      "Iter 12 / 128 Batch 2 / 11 Loss: 0.74583924\n",
      "Iter 12 / 128 Batch 3 / 11 Loss: 0.083289824\n",
      "Iter 12 / 128 Batch 4 / 11 Loss: 0.31967008\n",
      "Iter 12 / 128 Batch 5 / 11 Loss: 0.2626158\n",
      "Iter 12 / 128 Batch 6 / 11 Loss: 0.18992546\n",
      "Iter 12 / 128 Batch 7 / 11 Loss: 0.024851697\n",
      "Iter 12 / 128 Batch 8 / 11 Loss: 0.09438065\n",
      "Iter 12 / 128 Batch 9 / 11 Loss: 0.13285956\n",
      "Iter 12 / 128 Batch 10 / 11 Loss: 0.4391972\n",
      "Iter 13 / 128 Batch 0 / 11 Loss: 1.0205514\n",
      "Iter 13 / 128 Batch 1 / 11 Loss: 0.13468157\n",
      "Iter 13 / 128 Batch 2 / 11 Loss: 0.3130641\n",
      "Iter 13 / 128 Batch 3 / 11 Loss: 0.25172788\n",
      "Iter 13 / 128 Batch 4 / 11 Loss: 0.06867793\n",
      "Iter 13 / 128 Batch 5 / 11 Loss: 0.52514863\n",
      "Iter 13 / 128 Batch 6 / 11 Loss: 0.18107638\n",
      "Iter 13 / 128 Batch 7 / 11 Loss: 0.04517567\n",
      "Iter 13 / 128 Batch 8 / 11 Loss: 0.2401835\n",
      "Iter 13 / 128 Batch 9 / 11 Loss: 0.17821147\n",
      "Iter 13 / 128 Batch 10 / 11 Loss: 0.17944713\n",
      "Iter 14 / 128 Batch 0 / 11 Loss: 0.4599129\n",
      "Iter 14 / 128 Batch 1 / 11 Loss: 0.12300417\n",
      "Iter 14 / 128 Batch 2 / 11 Loss: 0.5728\n",
      "Iter 14 / 128 Batch 3 / 11 Loss: 0.14854787\n",
      "Iter 14 / 128 Batch 4 / 11 Loss: 0.086808234\n",
      "Iter 14 / 128 Batch 5 / 11 Loss: 0.20542674\n",
      "Iter 14 / 128 Batch 6 / 11 Loss: 0.20626251\n",
      "Iter 14 / 128 Batch 7 / 11 Loss: 0.124378264\n",
      "Iter 14 / 128 Batch 8 / 11 Loss: 0.06824565\n",
      "Iter 14 / 128 Batch 9 / 11 Loss: 0.25571567\n",
      "Iter 14 / 128 Batch 10 / 11 Loss: 0.26304135\n",
      "Iter 15 / 128 Batch 0 / 11 Loss: 0.057049237\n",
      "Iter 15 / 128 Batch 1 / 11 Loss: 0.09697364\n",
      "Iter 15 / 128 Batch 2 / 11 Loss: 0.64498055\n",
      "Iter 15 / 128 Batch 3 / 11 Loss: 0.124350235\n",
      "Iter 15 / 128 Batch 4 / 11 Loss: 0.28185266\n",
      "Iter 15 / 128 Batch 5 / 11 Loss: 0.4308934\n",
      "Iter 15 / 128 Batch 6 / 11 Loss: 0.28667128\n",
      "Iter 15 / 128 Batch 7 / 11 Loss: 0.040962454\n",
      "Iter 15 / 128 Batch 8 / 11 Loss: 0.2816774\n",
      "Iter 15 / 128 Batch 9 / 11 Loss: 0.19049296\n",
      "Iter 15 / 128 Batch 10 / 11 Loss: 0.17188105\n",
      "Iter 16 / 128 Batch 0 / 11 Loss: 0.043851867\n",
      "Iter 16 / 128 Batch 1 / 11 Loss: 0.09079999\n",
      "Iter 16 / 128 Batch 2 / 11 Loss: 0.3792519\n",
      "Iter 16 / 128 Batch 3 / 11 Loss: 0.08809961\n",
      "Iter 16 / 128 Batch 4 / 11 Loss: 0.3741386\n",
      "Iter 16 / 128 Batch 5 / 11 Loss: 0.6634772\n",
      "Iter 16 / 128 Batch 6 / 11 Loss: 0.23168819\n",
      "Iter 16 / 128 Batch 7 / 11 Loss: 0.07854084\n",
      "Iter 16 / 128 Batch 8 / 11 Loss: 0.26362887\n",
      "Iter 16 / 128 Batch 9 / 11 Loss: 0.24440919\n",
      "Iter 16 / 128 Batch 10 / 11 Loss: 0.114207834\n",
      "Iter 17 / 128 Batch 0 / 11 Loss: 0.28166378\n",
      "Iter 17 / 128 Batch 1 / 11 Loss: 0.12681109\n",
      "Iter 17 / 128 Batch 2 / 11 Loss: 0.81167436\n",
      "Iter 17 / 128 Batch 3 / 11 Loss: 0.10415029\n",
      "Iter 17 / 128 Batch 4 / 11 Loss: 0.12110856\n",
      "Iter 17 / 128 Batch 5 / 11 Loss: 0.13757442\n",
      "Iter 17 / 128 Batch 6 / 11 Loss: 0.14757383\n",
      "Iter 17 / 128 Batch 7 / 11 Loss: 0.02866874\n",
      "Iter 17 / 128 Batch 8 / 11 Loss: 0.1136851\n",
      "Iter 17 / 128 Batch 9 / 11 Loss: 0.19046253\n",
      "Iter 17 / 128 Batch 10 / 11 Loss: 0.1062787\n",
      "Iter 18 / 128 Batch 0 / 11 Loss: 0.0928096\n",
      "Iter 18 / 128 Batch 1 / 11 Loss: 0.048523\n",
      "Iter 18 / 128 Batch 2 / 11 Loss: 0.3634532\n",
      "Iter 18 / 128 Batch 3 / 11 Loss: 0.077285565\n",
      "Iter 18 / 128 Batch 4 / 11 Loss: 0.18244588\n",
      "Iter 18 / 128 Batch 5 / 11 Loss: 0.14993593\n",
      "Iter 18 / 128 Batch 6 / 11 Loss: 0.16596764\n",
      "Iter 18 / 128 Batch 7 / 11 Loss: 0.044302613\n",
      "Iter 18 / 128 Batch 8 / 11 Loss: 0.1258779\n",
      "Iter 18 / 128 Batch 9 / 11 Loss: 0.114526965\n",
      "Iter 18 / 128 Batch 10 / 11 Loss: 0.17029391\n",
      "Iter 19 / 128 Batch 0 / 11 Loss: 0.09018202\n",
      "Iter 19 / 128 Batch 1 / 11 Loss: 0.07692583\n",
      "Iter 19 / 128 Batch 2 / 11 Loss: 0.23541887\n",
      "Iter 19 / 128 Batch 3 / 11 Loss: 0.064326905\n",
      "Iter 19 / 128 Batch 4 / 11 Loss: 0.27030918\n",
      "Iter 19 / 128 Batch 5 / 11 Loss: 0.25527847\n",
      "Iter 19 / 128 Batch 6 / 11 Loss: 0.24313074\n",
      "Iter 19 / 128 Batch 7 / 11 Loss: 0.018955264\n",
      "Iter 19 / 128 Batch 8 / 11 Loss: 0.1300079\n",
      "Iter 19 / 128 Batch 9 / 11 Loss: 0.13804446\n",
      "Iter 19 / 128 Batch 10 / 11 Loss: 0.120190516\n",
      "Iter 20 / 128 Batch 0 / 11 Loss: 0.09398614\n",
      "Iter 20 / 128 Batch 1 / 11 Loss: 0.17170815\n",
      "Iter 20 / 128 Batch 2 / 11 Loss: 0.52613986\n",
      "Iter 20 / 128 Batch 3 / 11 Loss: 0.05288582\n",
      "Iter 20 / 128 Batch 4 / 11 Loss: 0.109924264\n",
      "Iter 20 / 128 Batch 5 / 11 Loss: 0.23085952\n",
      "Iter 20 / 128 Batch 6 / 11 Loss: 0.18683156\n",
      "Iter 20 / 128 Batch 7 / 11 Loss: 0.036969\n",
      "Iter 20 / 128 Batch 8 / 11 Loss: 0.07504697\n",
      "Iter 20 / 128 Batch 9 / 11 Loss: 0.22517313\n",
      "Iter 20 / 128 Batch 10 / 11 Loss: 0.11048798\n",
      "Iter 21 / 128 Batch 0 / 11 Loss: 0.060331896\n",
      "Iter 21 / 128 Batch 1 / 11 Loss: 0.024459295\n",
      "Iter 21 / 128 Batch 2 / 11 Loss: 0.3829191\n",
      "Iter 21 / 128 Batch 3 / 11 Loss: 0.041500617\n",
      "Iter 21 / 128 Batch 4 / 11 Loss: 0.16965035\n",
      "Iter 21 / 128 Batch 5 / 11 Loss: 0.10088946\n",
      "Iter 21 / 128 Batch 6 / 11 Loss: 0.057552144\n",
      "Iter 21 / 128 Batch 7 / 11 Loss: 0.040424343\n",
      "Iter 21 / 128 Batch 8 / 11 Loss: 0.21334806\n",
      "Iter 21 / 128 Batch 9 / 11 Loss: 0.35792398\n",
      "Iter 21 / 128 Batch 10 / 11 Loss: 0.21011421\n",
      "Iter 22 / 128 Batch 0 / 11 Loss: 0.05889948\n",
      "Iter 22 / 128 Batch 1 / 11 Loss: 0.049201306\n",
      "Iter 22 / 128 Batch 2 / 11 Loss: 0.7122481\n",
      "Iter 22 / 128 Batch 3 / 11 Loss: 0.33225352\n",
      "Iter 22 / 128 Batch 4 / 11 Loss: 0.064142674\n",
      "Iter 22 / 128 Batch 5 / 11 Loss: 0.099067606\n",
      "Iter 22 / 128 Batch 6 / 11 Loss: 0.09938012\n",
      "Iter 22 / 128 Batch 7 / 11 Loss: 0.019574521\n",
      "Iter 22 / 128 Batch 8 / 11 Loss: 0.028174382\n",
      "Iter 22 / 128 Batch 9 / 11 Loss: 0.27589598\n",
      "Iter 22 / 128 Batch 10 / 11 Loss: 0.15047103\n",
      "Iter 23 / 128 Batch 0 / 11 Loss: 0.17882651\n",
      "Iter 23 / 128 Batch 1 / 11 Loss: 0.056553893\n",
      "Iter 23 / 128 Batch 2 / 11 Loss: 0.35619438\n",
      "Iter 23 / 128 Batch 3 / 11 Loss: 0.13579546\n",
      "Iter 23 / 128 Batch 4 / 11 Loss: 0.043517202\n",
      "Iter 23 / 128 Batch 5 / 11 Loss: 0.46018735\n",
      "Iter 23 / 128 Batch 6 / 11 Loss: 0.15423441\n",
      "Iter 23 / 128 Batch 7 / 11 Loss: 0.080434024\n",
      "Iter 23 / 128 Batch 8 / 11 Loss: 0.10332666\n",
      "Iter 23 / 128 Batch 9 / 11 Loss: 0.079970434\n",
      "Iter 23 / 128 Batch 10 / 11 Loss: 0.19148177\n",
      "Iter 24 / 128 Batch 0 / 11 Loss: 0.083222866\n",
      "Iter 24 / 128 Batch 1 / 11 Loss: 0.038747385\n",
      "Iter 24 / 128 Batch 2 / 11 Loss: 0.39721355\n",
      "Iter 24 / 128 Batch 3 / 11 Loss: 0.24947388\n",
      "Iter 24 / 128 Batch 4 / 11 Loss: 0.15274265\n",
      "Iter 24 / 128 Batch 5 / 11 Loss: 0.31097546\n",
      "Iter 24 / 128 Batch 6 / 11 Loss: 0.06960927\n",
      "Iter 24 / 128 Batch 7 / 11 Loss: 0.00981639\n",
      "Iter 24 / 128 Batch 8 / 11 Loss: 0.12928289\n",
      "Iter 24 / 128 Batch 9 / 11 Loss: 0.04084598\n",
      "Iter 24 / 128 Batch 10 / 11 Loss: 0.21153682\n",
      "Iter 25 / 128 Batch 0 / 11 Loss: 0.075540036\n",
      "Iter 25 / 128 Batch 1 / 11 Loss: 0.10730955\n",
      "Iter 25 / 128 Batch 2 / 11 Loss: 0.31751192\n",
      "Iter 25 / 128 Batch 3 / 11 Loss: 0.07644499\n",
      "Iter 25 / 128 Batch 4 / 11 Loss: 0.12091769\n",
      "Iter 25 / 128 Batch 5 / 11 Loss: 0.24819688\n",
      "Iter 25 / 128 Batch 6 / 11 Loss: 0.13897718\n",
      "Iter 25 / 128 Batch 7 / 11 Loss: 0.011320414\n",
      "Iter 25 / 128 Batch 8 / 11 Loss: 0.14303944\n",
      "Iter 25 / 128 Batch 9 / 11 Loss: 0.108584054\n",
      "Iter 25 / 128 Batch 10 / 11 Loss: 0.15092382\n",
      "Iter 26 / 128 Batch 0 / 11 Loss: 0.034825444\n",
      "Iter 26 / 128 Batch 1 / 11 Loss: 0.047099497\n",
      "Iter 26 / 128 Batch 2 / 11 Loss: 0.18116993\n",
      "Iter 26 / 128 Batch 3 / 11 Loss: 0.086288385\n",
      "Iter 26 / 128 Batch 4 / 11 Loss: 0.13436027\n",
      "Iter 26 / 128 Batch 5 / 11 Loss: 0.084705226\n",
      "Iter 26 / 128 Batch 6 / 11 Loss: 0.06284806\n",
      "Iter 26 / 128 Batch 7 / 11 Loss: 0.12560916\n",
      "Iter 26 / 128 Batch 8 / 11 Loss: 0.053187057\n",
      "Iter 26 / 128 Batch 9 / 11 Loss: 0.04218381\n",
      "Iter 26 / 128 Batch 10 / 11 Loss: 0.062133156\n",
      "Iter 27 / 128 Batch 0 / 11 Loss: 0.03294339\n",
      "Iter 27 / 128 Batch 1 / 11 Loss: 0.021584928\n",
      "Iter 27 / 128 Batch 2 / 11 Loss: 0.19928046\n",
      "Iter 27 / 128 Batch 3 / 11 Loss: 0.030055411\n",
      "Iter 27 / 128 Batch 4 / 11 Loss: 0.096605964\n",
      "Iter 27 / 128 Batch 5 / 11 Loss: 0.2844913\n",
      "Iter 27 / 128 Batch 6 / 11 Loss: 0.105805606\n",
      "Iter 27 / 128 Batch 7 / 11 Loss: 0.020870365\n",
      "Iter 27 / 128 Batch 8 / 11 Loss: 0.2157476\n",
      "Iter 27 / 128 Batch 9 / 11 Loss: 0.11425382\n",
      "Iter 27 / 128 Batch 10 / 11 Loss: 0.026891362\n",
      "Iter 28 / 128 Batch 0 / 11 Loss: 0.018316872\n",
      "Iter 28 / 128 Batch 1 / 11 Loss: 0.01914298\n",
      "Iter 28 / 128 Batch 2 / 11 Loss: 0.42496777\n",
      "Iter 28 / 128 Batch 3 / 11 Loss: 0.044063713\n",
      "Iter 28 / 128 Batch 4 / 11 Loss: 0.1691888\n",
      "Iter 28 / 128 Batch 5 / 11 Loss: 0.17018121\n",
      "Iter 28 / 128 Batch 6 / 11 Loss: 0.07530262\n",
      "Iter 28 / 128 Batch 7 / 11 Loss: 0.009516824\n",
      "Iter 28 / 128 Batch 8 / 11 Loss: 0.24674651\n",
      "Iter 28 / 128 Batch 9 / 11 Loss: 0.05777426\n",
      "Iter 28 / 128 Batch 10 / 11 Loss: 0.055414565\n",
      "Iter 29 / 128 Batch 0 / 11 Loss: 0.055804815\n",
      "Iter 29 / 128 Batch 1 / 11 Loss: 0.05591975\n",
      "Iter 29 / 128 Batch 2 / 11 Loss: 0.22734633\n",
      "Iter 29 / 128 Batch 3 / 11 Loss: 0.018801937\n",
      "Iter 29 / 128 Batch 4 / 11 Loss: 0.059623487\n",
      "Iter 29 / 128 Batch 5 / 11 Loss: 0.081975974\n",
      "Iter 29 / 128 Batch 6 / 11 Loss: 0.033586305\n",
      "Iter 29 / 128 Batch 7 / 11 Loss: 0.01784885\n",
      "Iter 29 / 128 Batch 8 / 11 Loss: 0.16220473\n",
      "Iter 29 / 128 Batch 9 / 11 Loss: 0.08082881\n",
      "Iter 29 / 128 Batch 10 / 11 Loss: 0.17760822\n",
      "Iter 30 / 128 Batch 0 / 11 Loss: 0.021650909\n",
      "Iter 30 / 128 Batch 1 / 11 Loss: 0.012628628\n",
      "Iter 30 / 128 Batch 2 / 11 Loss: 0.19551045\n",
      "Iter 30 / 128 Batch 3 / 11 Loss: 0.020216595\n",
      "Iter 30 / 128 Batch 4 / 11 Loss: 0.0843385\n",
      "Iter 30 / 128 Batch 5 / 11 Loss: 0.026298936\n",
      "Iter 30 / 128 Batch 6 / 11 Loss: 0.05227173\n",
      "Iter 30 / 128 Batch 7 / 11 Loss: 0.028935233\n",
      "Iter 30 / 128 Batch 8 / 11 Loss: 0.038554683\n",
      "Iter 30 / 128 Batch 9 / 11 Loss: 0.12836827\n",
      "Iter 30 / 128 Batch 10 / 11 Loss: 0.026725598\n",
      "Iter 31 / 128 Batch 0 / 11 Loss: 0.022460325\n",
      "Iter 31 / 128 Batch 1 / 11 Loss: 0.010989532\n",
      "Iter 31 / 128 Batch 2 / 11 Loss: 0.15545473\n",
      "Iter 31 / 128 Batch 3 / 11 Loss: 0.04149177\n",
      "Iter 31 / 128 Batch 4 / 11 Loss: 0.03077784\n",
      "Iter 31 / 128 Batch 5 / 11 Loss: 0.07478749\n",
      "Iter 31 / 128 Batch 6 / 11 Loss: 0.057222806\n",
      "Iter 31 / 128 Batch 7 / 11 Loss: 0.011349773\n",
      "Iter 31 / 128 Batch 8 / 11 Loss: 0.08531279\n",
      "Iter 31 / 128 Batch 9 / 11 Loss: 0.082317814\n",
      "Iter 31 / 128 Batch 10 / 11 Loss: 0.15059575\n",
      "Iter 32 / 128 Batch 0 / 11 Loss: 0.016305031\n",
      "Iter 32 / 128 Batch 1 / 11 Loss: 0.054868154\n",
      "Iter 32 / 128 Batch 2 / 11 Loss: 0.1860406\n",
      "Iter 32 / 128 Batch 3 / 11 Loss: 0.049289715\n",
      "Iter 32 / 128 Batch 4 / 11 Loss: 0.02708599\n",
      "Iter 32 / 128 Batch 5 / 11 Loss: 0.07436945\n",
      "Iter 32 / 128 Batch 6 / 11 Loss: 0.12490725\n",
      "Iter 32 / 128 Batch 7 / 11 Loss: 0.010899695\n",
      "Iter 32 / 128 Batch 8 / 11 Loss: 0.089295626\n",
      "Iter 32 / 128 Batch 9 / 11 Loss: 0.07036326\n",
      "Iter 32 / 128 Batch 10 / 11 Loss: 0.04230719\n",
      "Iter 33 / 128 Batch 0 / 11 Loss: 0.059348825\n",
      "Iter 33 / 128 Batch 1 / 11 Loss: 0.038309205\n",
      "Iter 33 / 128 Batch 2 / 11 Loss: 0.17949423\n",
      "Iter 33 / 128 Batch 3 / 11 Loss: 0.022073017\n",
      "Iter 33 / 128 Batch 4 / 11 Loss: 0.023356937\n",
      "Iter 33 / 128 Batch 5 / 11 Loss: 0.11434092\n",
      "Iter 33 / 128 Batch 6 / 11 Loss: 0.08980728\n",
      "Iter 33 / 128 Batch 7 / 11 Loss: 0.019384887\n",
      "Iter 33 / 128 Batch 8 / 11 Loss: 0.038452737\n",
      "Iter 33 / 128 Batch 9 / 11 Loss: 0.058515694\n",
      "Iter 33 / 128 Batch 10 / 11 Loss: 0.052022435\n",
      "Iter 34 / 128 Batch 0 / 11 Loss: 0.015391576\n",
      "Iter 34 / 128 Batch 1 / 11 Loss: 0.056549367\n",
      "Iter 34 / 128 Batch 2 / 11 Loss: 0.112697735\n",
      "Iter 34 / 128 Batch 3 / 11 Loss: 0.024866661\n",
      "Iter 34 / 128 Batch 4 / 11 Loss: 0.22034337\n",
      "Iter 34 / 128 Batch 5 / 11 Loss: 0.06708038\n",
      "Iter 34 / 128 Batch 6 / 11 Loss: 0.01419238\n",
      "Iter 34 / 128 Batch 7 / 11 Loss: 0.0017146481\n",
      "Iter 34 / 128 Batch 8 / 11 Loss: 0.02059704\n",
      "Iter 34 / 128 Batch 9 / 11 Loss: 0.042439297\n",
      "Iter 34 / 128 Batch 10 / 11 Loss: 0.018899012\n",
      "Iter 35 / 128 Batch 0 / 11 Loss: 0.016294802\n",
      "Iter 35 / 128 Batch 1 / 11 Loss: 0.19170761\n",
      "Iter 35 / 128 Batch 2 / 11 Loss: 0.1899158\n",
      "Iter 35 / 128 Batch 3 / 11 Loss: 0.021003228\n",
      "Iter 35 / 128 Batch 4 / 11 Loss: 0.07176879\n",
      "Iter 35 / 128 Batch 5 / 11 Loss: 0.18078744\n",
      "Iter 35 / 128 Batch 6 / 11 Loss: 0.038423352\n",
      "Iter 35 / 128 Batch 7 / 11 Loss: 0.017145459\n",
      "Iter 35 / 128 Batch 8 / 11 Loss: 0.01593643\n",
      "Iter 35 / 128 Batch 9 / 11 Loss: 0.052280173\n",
      "Iter 35 / 128 Batch 10 / 11 Loss: 0.028352644\n",
      "Iter 36 / 128 Batch 0 / 11 Loss: 0.01495001\n",
      "Iter 36 / 128 Batch 1 / 11 Loss: 0.04201981\n",
      "Iter 36 / 128 Batch 2 / 11 Loss: 0.05031797\n",
      "Iter 36 / 128 Batch 3 / 11 Loss: 0.08367922\n",
      "Iter 36 / 128 Batch 4 / 11 Loss: 0.062259804\n",
      "Iter 36 / 128 Batch 5 / 11 Loss: 0.23608297\n",
      "Iter 36 / 128 Batch 6 / 11 Loss: 0.08392352\n",
      "Iter 36 / 128 Batch 7 / 11 Loss: 0.013229959\n",
      "Iter 36 / 128 Batch 8 / 11 Loss: 0.047521338\n",
      "Iter 36 / 128 Batch 9 / 11 Loss: 0.024301771\n",
      "Iter 36 / 128 Batch 10 / 11 Loss: 0.02560226\n",
      "Iter 37 / 128 Batch 0 / 11 Loss: 0.0153074395\n",
      "Iter 37 / 128 Batch 1 / 11 Loss: 0.045499664\n",
      "Iter 37 / 128 Batch 2 / 11 Loss: 0.2094239\n",
      "Iter 37 / 128 Batch 3 / 11 Loss: 0.021589452\n",
      "Iter 37 / 128 Batch 4 / 11 Loss: 0.027876657\n",
      "Iter 37 / 128 Batch 5 / 11 Loss: 0.050753996\n",
      "Iter 37 / 128 Batch 6 / 11 Loss: 0.046072695\n",
      "Iter 37 / 128 Batch 7 / 11 Loss: 0.02138099\n",
      "Iter 37 / 128 Batch 8 / 11 Loss: 0.03817475\n",
      "Iter 37 / 128 Batch 9 / 11 Loss: 0.14964148\n",
      "Iter 37 / 128 Batch 10 / 11 Loss: 0.035917312\n",
      "Iter 38 / 128 Batch 0 / 11 Loss: 0.030002177\n",
      "Iter 38 / 128 Batch 1 / 11 Loss: 0.025364382\n",
      "Iter 38 / 128 Batch 2 / 11 Loss: 0.19992262\n",
      "Iter 38 / 128 Batch 3 / 11 Loss: 0.0047436506\n",
      "Iter 38 / 128 Batch 4 / 11 Loss: 0.023150885\n",
      "Iter 38 / 128 Batch 5 / 11 Loss: 0.14184111\n",
      "Iter 38 / 128 Batch 6 / 11 Loss: 0.03193439\n",
      "Iter 38 / 128 Batch 7 / 11 Loss: 0.005816579\n",
      "Iter 38 / 128 Batch 8 / 11 Loss: 0.020301685\n",
      "Iter 38 / 128 Batch 9 / 11 Loss: 0.014998761\n",
      "Iter 38 / 128 Batch 10 / 11 Loss: 0.06281553\n",
      "Iter 39 / 128 Batch 0 / 11 Loss: 0.035142645\n",
      "Iter 39 / 128 Batch 1 / 11 Loss: 0.013832318\n",
      "Iter 39 / 128 Batch 2 / 11 Loss: 0.18614191\n",
      "Iter 39 / 128 Batch 3 / 11 Loss: 0.022014597\n",
      "Iter 39 / 128 Batch 4 / 11 Loss: 0.026110297\n",
      "Iter 39 / 128 Batch 5 / 11 Loss: 0.11681316\n",
      "Iter 39 / 128 Batch 6 / 11 Loss: 0.08595492\n",
      "Iter 39 / 128 Batch 7 / 11 Loss: 0.008589599\n",
      "Iter 39 / 128 Batch 8 / 11 Loss: 0.07281538\n",
      "Iter 39 / 128 Batch 9 / 11 Loss: 0.08566867\n",
      "Iter 39 / 128 Batch 10 / 11 Loss: 0.04460907\n",
      "Iter 40 / 128 Batch 0 / 11 Loss: 0.016077552\n",
      "Iter 40 / 128 Batch 1 / 11 Loss: 0.036444295\n",
      "Iter 40 / 128 Batch 2 / 11 Loss: 0.12000573\n",
      "Iter 40 / 128 Batch 3 / 11 Loss: 0.025616478\n",
      "Iter 40 / 128 Batch 4 / 11 Loss: 0.011649818\n",
      "Iter 40 / 128 Batch 5 / 11 Loss: 0.02145862\n",
      "Iter 40 / 128 Batch 6 / 11 Loss: 0.020551916\n",
      "Iter 40 / 128 Batch 7 / 11 Loss: 0.024979368\n",
      "Iter 40 / 128 Batch 8 / 11 Loss: 0.048984323\n",
      "Iter 40 / 128 Batch 9 / 11 Loss: 0.030840803\n",
      "Iter 40 / 128 Batch 10 / 11 Loss: 0.019328248\n",
      "Iter 41 / 128 Batch 0 / 11 Loss: 0.006061429\n",
      "Iter 41 / 128 Batch 1 / 11 Loss: 0.036629915\n",
      "Iter 41 / 128 Batch 2 / 11 Loss: 0.08197418\n",
      "Iter 41 / 128 Batch 3 / 11 Loss: 0.033424374\n",
      "Iter 41 / 128 Batch 4 / 11 Loss: 0.018481556\n",
      "Iter 41 / 128 Batch 5 / 11 Loss: 0.022945752\n",
      "Iter 41 / 128 Batch 6 / 11 Loss: 0.17958306\n",
      "Iter 41 / 128 Batch 7 / 11 Loss: 0.0128583545\n",
      "Iter 41 / 128 Batch 8 / 11 Loss: 0.008687226\n",
      "Iter 41 / 128 Batch 9 / 11 Loss: 0.01679213\n",
      "Iter 41 / 128 Batch 10 / 11 Loss: 0.013190598\n",
      "Iter 42 / 128 Batch 0 / 11 Loss: 0.012559008\n",
      "Iter 42 / 128 Batch 1 / 11 Loss: 0.012940666\n",
      "Iter 42 / 128 Batch 2 / 11 Loss: 0.23846328\n",
      "Iter 42 / 128 Batch 3 / 11 Loss: 0.015988378\n",
      "Iter 42 / 128 Batch 4 / 11 Loss: 0.014380671\n",
      "Iter 42 / 128 Batch 5 / 11 Loss: 0.10623576\n",
      "Iter 42 / 128 Batch 6 / 11 Loss: 0.18168312\n",
      "Iter 42 / 128 Batch 7 / 11 Loss: 0.013171043\n",
      "Iter 42 / 128 Batch 8 / 11 Loss: 0.025333967\n",
      "Iter 42 / 128 Batch 9 / 11 Loss: 0.049972333\n",
      "Iter 42 / 128 Batch 10 / 11 Loss: 0.046564188\n",
      "Iter 43 / 128 Batch 0 / 11 Loss: 0.042279992\n",
      "Iter 43 / 128 Batch 1 / 11 Loss: 0.018715233\n",
      "Iter 43 / 128 Batch 2 / 11 Loss: 0.08286558\n",
      "Iter 43 / 128 Batch 3 / 11 Loss: 0.010955455\n",
      "Iter 43 / 128 Batch 4 / 11 Loss: 0.17497468\n",
      "Iter 43 / 128 Batch 5 / 11 Loss: 0.037679054\n",
      "Iter 43 / 128 Batch 6 / 11 Loss: 0.07057391\n",
      "Iter 43 / 128 Batch 7 / 11 Loss: 0.012347134\n",
      "Iter 43 / 128 Batch 8 / 11 Loss: 0.00879887\n",
      "Iter 43 / 128 Batch 9 / 11 Loss: 0.06957291\n",
      "Iter 43 / 128 Batch 10 / 11 Loss: 0.02337145\n",
      "Iter 44 / 128 Batch 0 / 11 Loss: 0.011053291\n",
      "Iter 44 / 128 Batch 1 / 11 Loss: 0.0061081806\n",
      "Iter 44 / 128 Batch 2 / 11 Loss: 0.15102255\n",
      "Iter 44 / 128 Batch 3 / 11 Loss: 0.030776896\n",
      "Iter 44 / 128 Batch 4 / 11 Loss: 0.04201518\n",
      "Iter 44 / 128 Batch 5 / 11 Loss: 0.06420427\n",
      "Iter 44 / 128 Batch 6 / 11 Loss: 0.015181591\n",
      "Iter 44 / 128 Batch 7 / 11 Loss: 0.007422193\n",
      "Iter 44 / 128 Batch 8 / 11 Loss: 0.042013876\n",
      "Iter 44 / 128 Batch 9 / 11 Loss: 0.03559456\n",
      "Iter 44 / 128 Batch 10 / 11 Loss: 0.011129635\n",
      "Iter 45 / 128 Batch 0 / 11 Loss: 0.027101303\n",
      "Iter 45 / 128 Batch 1 / 11 Loss: 0.011402661\n",
      "Iter 45 / 128 Batch 2 / 11 Loss: 0.26568198\n",
      "Iter 45 / 128 Batch 3 / 11 Loss: 0.014930646\n",
      "Iter 45 / 128 Batch 4 / 11 Loss: 0.048998445\n",
      "Iter 45 / 128 Batch 5 / 11 Loss: 0.023276929\n",
      "Iter 45 / 128 Batch 6 / 11 Loss: 0.029994257\n",
      "Iter 45 / 128 Batch 7 / 11 Loss: 0.026699236\n",
      "Iter 45 / 128 Batch 8 / 11 Loss: 0.103872985\n",
      "Iter 45 / 128 Batch 9 / 11 Loss: 0.048557736\n",
      "Iter 45 / 128 Batch 10 / 11 Loss: 0.08536284\n",
      "Iter 46 / 128 Batch 0 / 11 Loss: 0.014222323\n",
      "Iter 46 / 128 Batch 1 / 11 Loss: 0.05431336\n",
      "Iter 46 / 128 Batch 2 / 11 Loss: 0.13702388\n",
      "Iter 46 / 128 Batch 3 / 11 Loss: 0.017232168\n",
      "Iter 46 / 128 Batch 4 / 11 Loss: 0.057916567\n",
      "Iter 46 / 128 Batch 5 / 11 Loss: 0.119778246\n",
      "Iter 46 / 128 Batch 6 / 11 Loss: 0.03946685\n",
      "Iter 46 / 128 Batch 7 / 11 Loss: 0.0039817244\n",
      "Iter 46 / 128 Batch 8 / 11 Loss: 0.11664543\n",
      "Iter 46 / 128 Batch 9 / 11 Loss: 0.026529841\n",
      "Iter 46 / 128 Batch 10 / 11 Loss: 0.04779116\n",
      "Iter 47 / 128 Batch 0 / 11 Loss: 0.0051629464\n",
      "Iter 47 / 128 Batch 1 / 11 Loss: 0.021935957\n",
      "Iter 47 / 128 Batch 2 / 11 Loss: 0.23453587\n",
      "Iter 47 / 128 Batch 3 / 11 Loss: 0.02587806\n",
      "Iter 47 / 128 Batch 4 / 11 Loss: 0.028730078\n",
      "Iter 47 / 128 Batch 5 / 11 Loss: 0.05949757\n",
      "Iter 47 / 128 Batch 6 / 11 Loss: 0.029330106\n",
      "Iter 47 / 128 Batch 7 / 11 Loss: 0.0071527446\n",
      "Iter 47 / 128 Batch 8 / 11 Loss: 0.31788802\n",
      "Iter 47 / 128 Batch 9 / 11 Loss: 0.11116483\n",
      "Iter 47 / 128 Batch 10 / 11 Loss: 0.010053523\n",
      "Iter 48 / 128 Batch 0 / 11 Loss: 0.020205824\n",
      "Iter 48 / 128 Batch 1 / 11 Loss: 0.053075477\n",
      "Iter 48 / 128 Batch 2 / 11 Loss: 0.076986045\n",
      "Iter 48 / 128 Batch 3 / 11 Loss: 0.049120042\n",
      "Iter 48 / 128 Batch 4 / 11 Loss: 0.03357481\n",
      "Iter 48 / 128 Batch 5 / 11 Loss: 0.042889416\n",
      "Iter 48 / 128 Batch 6 / 11 Loss: 0.045328047\n",
      "Iter 48 / 128 Batch 7 / 11 Loss: 0.0067708194\n",
      "Iter 48 / 128 Batch 8 / 11 Loss: 0.023750447\n",
      "Iter 48 / 128 Batch 9 / 11 Loss: 0.038980514\n",
      "Iter 48 / 128 Batch 10 / 11 Loss: 0.0123243425\n",
      "Iter 49 / 128 Batch 0 / 11 Loss: 0.032811955\n",
      "Iter 49 / 128 Batch 1 / 11 Loss: 0.006229263\n",
      "Iter 49 / 128 Batch 2 / 11 Loss: 0.08320813\n",
      "Iter 49 / 128 Batch 3 / 11 Loss: 0.009017981\n",
      "Iter 49 / 128 Batch 4 / 11 Loss: 0.036568016\n",
      "Iter 49 / 128 Batch 5 / 11 Loss: 0.09303637\n",
      "Iter 49 / 128 Batch 6 / 11 Loss: 0.052913465\n",
      "Iter 49 / 128 Batch 7 / 11 Loss: 0.007623808\n",
      "Iter 49 / 128 Batch 8 / 11 Loss: 0.0607936\n",
      "Iter 49 / 128 Batch 9 / 11 Loss: 0.010845557\n",
      "Iter 49 / 128 Batch 10 / 11 Loss: 0.020248469\n",
      "Iter 50 / 128 Batch 0 / 11 Loss: 0.022910094\n",
      "Iter 50 / 128 Batch 1 / 11 Loss: 0.0039974656\n",
      "Iter 50 / 128 Batch 2 / 11 Loss: 0.15008967\n",
      "Iter 50 / 128 Batch 3 / 11 Loss: 0.042757593\n",
      "Iter 50 / 128 Batch 4 / 11 Loss: 0.021754649\n",
      "Iter 50 / 128 Batch 5 / 11 Loss: 0.15986408\n",
      "Iter 50 / 128 Batch 6 / 11 Loss: 0.0420246\n",
      "Iter 50 / 128 Batch 7 / 11 Loss: 0.006385115\n",
      "Iter 50 / 128 Batch 8 / 11 Loss: 0.044171676\n",
      "Iter 50 / 128 Batch 9 / 11 Loss: 0.021811852\n",
      "Iter 50 / 128 Batch 10 / 11 Loss: 0.00505864\n",
      "Iter 51 / 128 Batch 0 / 11 Loss: 0.02097459\n",
      "Iter 51 / 128 Batch 1 / 11 Loss: 0.021664605\n",
      "Iter 51 / 128 Batch 2 / 11 Loss: 0.059007756\n",
      "Iter 51 / 128 Batch 3 / 11 Loss: 0.13688377\n",
      "Iter 51 / 128 Batch 4 / 11 Loss: 0.008117648\n",
      "Iter 51 / 128 Batch 5 / 11 Loss: 0.046010807\n",
      "Iter 51 / 128 Batch 6 / 11 Loss: 0.033206634\n",
      "Iter 51 / 128 Batch 7 / 11 Loss: 0.0010384065\n",
      "Iter 51 / 128 Batch 8 / 11 Loss: 0.016477356\n",
      "Iter 51 / 128 Batch 9 / 11 Loss: 0.035340883\n",
      "Iter 51 / 128 Batch 10 / 11 Loss: 0.010313756\n",
      "Iter 52 / 128 Batch 0 / 11 Loss: 0.008550999\n",
      "Iter 52 / 128 Batch 1 / 11 Loss: 0.008851955\n",
      "Iter 52 / 128 Batch 2 / 11 Loss: 0.048903473\n",
      "Iter 52 / 128 Batch 3 / 11 Loss: 0.013368695\n",
      "Iter 52 / 128 Batch 4 / 11 Loss: 0.012235565\n",
      "Iter 52 / 128 Batch 5 / 11 Loss: 0.051551174\n",
      "Iter 52 / 128 Batch 6 / 11 Loss: 0.03669766\n",
      "Iter 52 / 128 Batch 7 / 11 Loss: 0.0031509968\n",
      "Iter 52 / 128 Batch 8 / 11 Loss: 0.017043393\n",
      "Iter 52 / 128 Batch 9 / 11 Loss: 0.022695508\n",
      "Iter 52 / 128 Batch 10 / 11 Loss: 0.0353636\n",
      "Iter 53 / 128 Batch 0 / 11 Loss: 0.011822192\n",
      "Iter 53 / 128 Batch 1 / 11 Loss: 0.03410164\n",
      "Iter 53 / 128 Batch 2 / 11 Loss: 0.20275551\n",
      "Iter 53 / 128 Batch 3 / 11 Loss: 0.0076730293\n",
      "Iter 53 / 128 Batch 4 / 11 Loss: 0.083183825\n",
      "Iter 53 / 128 Batch 5 / 11 Loss: 0.15626603\n",
      "Iter 53 / 128 Batch 6 / 11 Loss: 0.07012781\n",
      "Iter 53 / 128 Batch 7 / 11 Loss: 0.0019420268\n",
      "Iter 53 / 128 Batch 8 / 11 Loss: 0.036822602\n",
      "Iter 53 / 128 Batch 9 / 11 Loss: 0.058664404\n",
      "Iter 53 / 128 Batch 10 / 11 Loss: 0.009600959\n",
      "Iter 54 / 128 Batch 0 / 11 Loss: 0.01664327\n",
      "Iter 54 / 128 Batch 1 / 11 Loss: 0.029850744\n",
      "Iter 54 / 128 Batch 2 / 11 Loss: 0.08558309\n",
      "Iter 54 / 128 Batch 3 / 11 Loss: 0.026908774\n",
      "Iter 54 / 128 Batch 4 / 11 Loss: 0.016171187\n",
      "Iter 54 / 128 Batch 5 / 11 Loss: 0.045255758\n",
      "Iter 54 / 128 Batch 6 / 11 Loss: 0.0062426305\n",
      "Iter 54 / 128 Batch 7 / 11 Loss: 0.0077902777\n",
      "Iter 54 / 128 Batch 8 / 11 Loss: 0.027468003\n",
      "Iter 54 / 128 Batch 9 / 11 Loss: 0.07040473\n",
      "Iter 54 / 128 Batch 10 / 11 Loss: 0.03541459\n",
      "Iter 55 / 128 Batch 0 / 11 Loss: 0.028192889\n",
      "Iter 55 / 128 Batch 1 / 11 Loss: 0.009766971\n",
      "Iter 55 / 128 Batch 2 / 11 Loss: 0.10090838\n",
      "Iter 55 / 128 Batch 3 / 11 Loss: 0.010018393\n",
      "Iter 55 / 128 Batch 4 / 11 Loss: 0.010852914\n",
      "Iter 55 / 128 Batch 5 / 11 Loss: 0.033230335\n",
      "Iter 55 / 128 Batch 6 / 11 Loss: 0.045337163\n",
      "Iter 55 / 128 Batch 7 / 11 Loss: 0.009648563\n",
      "Iter 55 / 128 Batch 8 / 11 Loss: 0.022648558\n",
      "Iter 55 / 128 Batch 9 / 11 Loss: 0.046319082\n",
      "Iter 55 / 128 Batch 10 / 11 Loss: 0.01256861\n",
      "Iter 56 / 128 Batch 0 / 11 Loss: 0.003949756\n",
      "Iter 56 / 128 Batch 1 / 11 Loss: 0.016828166\n",
      "Iter 56 / 128 Batch 2 / 11 Loss: 0.10062907\n",
      "Iter 56 / 128 Batch 3 / 11 Loss: 0.020397082\n",
      "Iter 56 / 128 Batch 4 / 11 Loss: 0.05008498\n",
      "Iter 56 / 128 Batch 5 / 11 Loss: 0.05823364\n",
      "Iter 56 / 128 Batch 6 / 11 Loss: 0.009164294\n",
      "Iter 56 / 128 Batch 7 / 11 Loss: 0.0027378926\n",
      "Iter 56 / 128 Batch 8 / 11 Loss: 0.015690839\n",
      "Iter 56 / 128 Batch 9 / 11 Loss: 0.008755341\n",
      "Iter 56 / 128 Batch 10 / 11 Loss: 0.020939047\n",
      "Iter 57 / 128 Batch 0 / 11 Loss: 0.056175116\n",
      "Iter 57 / 128 Batch 1 / 11 Loss: 0.013701908\n",
      "Iter 57 / 128 Batch 2 / 11 Loss: 0.19929919\n",
      "Iter 57 / 128 Batch 3 / 11 Loss: 0.1277839\n",
      "Iter 57 / 128 Batch 4 / 11 Loss: 0.04001759\n",
      "Iter 57 / 128 Batch 5 / 11 Loss: 0.039305642\n",
      "Iter 57 / 128 Batch 6 / 11 Loss: 0.027558614\n",
      "Iter 57 / 128 Batch 7 / 11 Loss: 0.0029822239\n",
      "Iter 57 / 128 Batch 8 / 11 Loss: 0.03824999\n",
      "Iter 57 / 128 Batch 9 / 11 Loss: 0.15322053\n",
      "Iter 57 / 128 Batch 10 / 11 Loss: 0.015007222\n",
      "Iter 58 / 128 Batch 0 / 11 Loss: 0.02325985\n",
      "Iter 58 / 128 Batch 1 / 11 Loss: 0.17378542\n",
      "Iter 58 / 128 Batch 2 / 11 Loss: 0.11993193\n",
      "Iter 58 / 128 Batch 3 / 11 Loss: 0.06116846\n",
      "Iter 58 / 128 Batch 4 / 11 Loss: 0.16917779\n",
      "Iter 58 / 128 Batch 5 / 11 Loss: 0.019930083\n",
      "Iter 58 / 128 Batch 6 / 11 Loss: 0.16723728\n",
      "Iter 58 / 128 Batch 7 / 11 Loss: 0.014144473\n",
      "Iter 58 / 128 Batch 8 / 11 Loss: 0.079150856\n",
      "Iter 58 / 128 Batch 9 / 11 Loss: 0.03776226\n",
      "Iter 58 / 128 Batch 10 / 11 Loss: 0.023405252\n",
      "Iter 59 / 128 Batch 0 / 11 Loss: 0.0102898115\n",
      "Iter 59 / 128 Batch 1 / 11 Loss: 0.017340379\n",
      "Iter 59 / 128 Batch 2 / 11 Loss: 0.17693853\n",
      "Iter 59 / 128 Batch 3 / 11 Loss: 0.004595433\n",
      "Iter 59 / 128 Batch 4 / 11 Loss: 0.018422674\n",
      "Iter 59 / 128 Batch 5 / 11 Loss: 0.107209116\n",
      "Iter 59 / 128 Batch 6 / 11 Loss: 0.07571882\n",
      "Iter 59 / 128 Batch 7 / 11 Loss: 0.002760837\n",
      "Iter 59 / 128 Batch 8 / 11 Loss: 0.015747946\n",
      "Iter 59 / 128 Batch 9 / 11 Loss: 0.037099835\n",
      "Iter 59 / 128 Batch 10 / 11 Loss: 0.027140915\n",
      "Iter 60 / 128 Batch 0 / 11 Loss: 0.052966774\n",
      "Iter 60 / 128 Batch 1 / 11 Loss: 0.045018222\n",
      "Iter 60 / 128 Batch 2 / 11 Loss: 0.11146322\n",
      "Iter 60 / 128 Batch 3 / 11 Loss: 0.008532358\n",
      "Iter 60 / 128 Batch 4 / 11 Loss: 0.0065807044\n",
      "Iter 60 / 128 Batch 5 / 11 Loss: 0.040599093\n",
      "Iter 60 / 128 Batch 6 / 11 Loss: 0.033259455\n",
      "Iter 60 / 128 Batch 7 / 11 Loss: 0.0032125348\n",
      "Iter 60 / 128 Batch 8 / 11 Loss: 0.010263417\n",
      "Iter 60 / 128 Batch 9 / 11 Loss: 0.016600475\n",
      "Iter 60 / 128 Batch 10 / 11 Loss: 0.020023415\n",
      "Iter 61 / 128 Batch 0 / 11 Loss: 0.100677624\n",
      "Iter 61 / 128 Batch 1 / 11 Loss: 0.010464918\n",
      "Iter 61 / 128 Batch 2 / 11 Loss: 0.14871147\n",
      "Iter 61 / 128 Batch 3 / 11 Loss: 0.015848026\n",
      "Iter 61 / 128 Batch 4 / 11 Loss: 0.014868129\n",
      "Iter 61 / 128 Batch 5 / 11 Loss: 0.032370932\n",
      "Iter 61 / 128 Batch 6 / 11 Loss: 0.030472644\n",
      "Iter 61 / 128 Batch 7 / 11 Loss: 0.008578548\n",
      "Iter 61 / 128 Batch 8 / 11 Loss: 0.02675423\n",
      "Iter 61 / 128 Batch 9 / 11 Loss: 0.11317131\n",
      "Iter 61 / 128 Batch 10 / 11 Loss: 0.08759174\n",
      "Iter 62 / 128 Batch 0 / 11 Loss: 0.0089871045\n",
      "Iter 62 / 128 Batch 1 / 11 Loss: 0.010973025\n",
      "Iter 62 / 128 Batch 2 / 11 Loss: 0.18673934\n",
      "Iter 62 / 128 Batch 3 / 11 Loss: 0.014904243\n",
      "Iter 62 / 128 Batch 4 / 11 Loss: 0.030931871\n",
      "Iter 62 / 128 Batch 5 / 11 Loss: 0.035237923\n",
      "Iter 62 / 128 Batch 6 / 11 Loss: 0.011821667\n",
      "Iter 62 / 128 Batch 7 / 11 Loss: 0.011074239\n",
      "Iter 62 / 128 Batch 8 / 11 Loss: 0.021247065\n",
      "Iter 62 / 128 Batch 9 / 11 Loss: 0.06480234\n",
      "Iter 62 / 128 Batch 10 / 11 Loss: 0.007506095\n",
      "Iter 63 / 128 Batch 0 / 11 Loss: 0.008878451\n",
      "Iter 63 / 128 Batch 1 / 11 Loss: 0.16149138\n",
      "Iter 63 / 128 Batch 2 / 11 Loss: 0.18353508\n",
      "Iter 63 / 128 Batch 3 / 11 Loss: 0.003846681\n",
      "Iter 63 / 128 Batch 4 / 11 Loss: 0.0617177\n",
      "Iter 63 / 128 Batch 5 / 11 Loss: 0.038244493\n",
      "Iter 63 / 128 Batch 6 / 11 Loss: 0.042503178\n",
      "Iter 63 / 128 Batch 7 / 11 Loss: 0.012939043\n",
      "Iter 63 / 128 Batch 8 / 11 Loss: 0.022685979\n",
      "Iter 63 / 128 Batch 9 / 11 Loss: 0.038827717\n",
      "Iter 63 / 128 Batch 10 / 11 Loss: 0.009841093\n",
      "Iter 64 / 128 Batch 0 / 11 Loss: 0.0052250056\n",
      "Iter 64 / 128 Batch 1 / 11 Loss: 0.021629088\n",
      "Iter 64 / 128 Batch 2 / 11 Loss: 0.049878757\n",
      "Iter 64 / 128 Batch 3 / 11 Loss: 0.012316833\n",
      "Iter 64 / 128 Batch 4 / 11 Loss: 0.026404701\n",
      "Iter 64 / 128 Batch 5 / 11 Loss: 0.027804235\n",
      "Iter 64 / 128 Batch 6 / 11 Loss: 0.034168728\n",
      "Iter 64 / 128 Batch 7 / 11 Loss: 0.00066165277\n",
      "Iter 64 / 128 Batch 8 / 11 Loss: 0.012162146\n",
      "Iter 64 / 128 Batch 9 / 11 Loss: 0.042533837\n",
      "Iter 64 / 128 Batch 10 / 11 Loss: 0.016605083\n",
      "Iter 65 / 128 Batch 0 / 11 Loss: 0.0054870187\n",
      "Iter 65 / 128 Batch 1 / 11 Loss: 0.00607182\n",
      "Iter 65 / 128 Batch 2 / 11 Loss: 0.12841462\n",
      "Iter 65 / 128 Batch 3 / 11 Loss: 0.018081985\n",
      "Iter 65 / 128 Batch 4 / 11 Loss: 0.013611674\n",
      "Iter 65 / 128 Batch 5 / 11 Loss: 0.024200542\n",
      "Iter 65 / 128 Batch 6 / 11 Loss: 0.013753768\n",
      "Iter 65 / 128 Batch 7 / 11 Loss: 0.019833867\n",
      "Iter 65 / 128 Batch 8 / 11 Loss: 0.014756943\n",
      "Iter 65 / 128 Batch 9 / 11 Loss: 0.01775206\n",
      "Iter 65 / 128 Batch 10 / 11 Loss: 0.015275685\n",
      "Iter 66 / 128 Batch 0 / 11 Loss: 0.00534222\n",
      "Iter 66 / 128 Batch 1 / 11 Loss: 0.07820683\n",
      "Iter 66 / 128 Batch 2 / 11 Loss: 0.03551101\n",
      "Iter 66 / 128 Batch 3 / 11 Loss: 0.009421328\n",
      "Iter 66 / 128 Batch 4 / 11 Loss: 0.008143439\n",
      "Iter 66 / 128 Batch 5 / 11 Loss: 0.13482685\n",
      "Iter 66 / 128 Batch 6 / 11 Loss: 0.012974275\n",
      "Iter 66 / 128 Batch 7 / 11 Loss: 0.004378447\n",
      "Iter 66 / 128 Batch 8 / 11 Loss: 0.024020584\n",
      "Iter 66 / 128 Batch 9 / 11 Loss: 0.012073785\n",
      "Iter 66 / 128 Batch 10 / 11 Loss: 0.04642205\n",
      "Iter 67 / 128 Batch 0 / 11 Loss: 0.02587707\n",
      "Iter 67 / 128 Batch 1 / 11 Loss: 0.027779542\n",
      "Iter 67 / 128 Batch 2 / 11 Loss: 0.15711747\n",
      "Iter 67 / 128 Batch 3 / 11 Loss: 0.022196995\n",
      "Iter 67 / 128 Batch 4 / 11 Loss: 0.06558067\n",
      "Iter 67 / 128 Batch 5 / 11 Loss: 0.11254522\n",
      "Iter 67 / 128 Batch 6 / 11 Loss: 0.020663965\n",
      "Iter 67 / 128 Batch 7 / 11 Loss: 0.0028926826\n",
      "Iter 67 / 128 Batch 8 / 11 Loss: 0.008838549\n",
      "Iter 67 / 128 Batch 9 / 11 Loss: 0.055864897\n",
      "Iter 67 / 128 Batch 10 / 11 Loss: 0.0530869\n",
      "Iter 68 / 128 Batch 0 / 11 Loss: 0.017968232\n",
      "Iter 68 / 128 Batch 1 / 11 Loss: 0.014364269\n",
      "Iter 68 / 128 Batch 2 / 11 Loss: 0.19994739\n",
      "Iter 68 / 128 Batch 3 / 11 Loss: 0.094179325\n",
      "Iter 68 / 128 Batch 4 / 11 Loss: 0.12745564\n",
      "Iter 68 / 128 Batch 5 / 11 Loss: 0.030498417\n",
      "Iter 68 / 128 Batch 6 / 11 Loss: 0.01438039\n",
      "Iter 68 / 128 Batch 7 / 11 Loss: 0.005173101\n",
      "Iter 68 / 128 Batch 8 / 11 Loss: 0.05653522\n",
      "Iter 68 / 128 Batch 9 / 11 Loss: 0.039943926\n",
      "Iter 68 / 128 Batch 10 / 11 Loss: 0.03985489\n",
      "Iter 69 / 128 Batch 0 / 11 Loss: 0.0133579\n",
      "Iter 69 / 128 Batch 1 / 11 Loss: 0.048047297\n",
      "Iter 69 / 128 Batch 2 / 11 Loss: 0.09609181\n",
      "Iter 69 / 128 Batch 3 / 11 Loss: 0.06890376\n",
      "Iter 69 / 128 Batch 4 / 11 Loss: 0.038323283\n",
      "Iter 69 / 128 Batch 5 / 11 Loss: 0.025854087\n",
      "Iter 69 / 128 Batch 6 / 11 Loss: 0.007815968\n",
      "Iter 69 / 128 Batch 7 / 11 Loss: 0.005697839\n",
      "Iter 69 / 128 Batch 8 / 11 Loss: 0.031346668\n",
      "Iter 69 / 128 Batch 9 / 11 Loss: 0.020915875\n",
      "Iter 69 / 128 Batch 10 / 11 Loss: 0.011413202\n",
      "Iter 70 / 128 Batch 0 / 11 Loss: 0.019548252\n",
      "Iter 70 / 128 Batch 1 / 11 Loss: 0.0064601265\n",
      "Iter 70 / 128 Batch 2 / 11 Loss: 0.06548979\n",
      "Iter 70 / 128 Batch 3 / 11 Loss: 0.0042384034\n",
      "Iter 70 / 128 Batch 4 / 11 Loss: 0.018286463\n",
      "Iter 70 / 128 Batch 5 / 11 Loss: 0.0551197\n",
      "Iter 70 / 128 Batch 6 / 11 Loss: 0.023876961\n",
      "Iter 70 / 128 Batch 7 / 11 Loss: 0.0028197132\n",
      "Iter 70 / 128 Batch 8 / 11 Loss: 0.013679974\n",
      "Iter 70 / 128 Batch 9 / 11 Loss: 0.014783223\n",
      "Iter 70 / 128 Batch 10 / 11 Loss: 0.022904959\n",
      "Iter 71 / 128 Batch 0 / 11 Loss: 0.057356227\n",
      "Iter 71 / 128 Batch 1 / 11 Loss: 0.010177134\n",
      "Iter 71 / 128 Batch 2 / 11 Loss: 0.052117668\n",
      "Iter 71 / 128 Batch 3 / 11 Loss: 0.011785627\n",
      "Iter 71 / 128 Batch 4 / 11 Loss: 0.013030997\n",
      "Iter 71 / 128 Batch 5 / 11 Loss: 0.06662958\n",
      "Iter 71 / 128 Batch 6 / 11 Loss: 0.022010358\n",
      "Iter 71 / 128 Batch 7 / 11 Loss: 0.010669636\n",
      "Iter 71 / 128 Batch 8 / 11 Loss: 0.004198051\n",
      "Iter 71 / 128 Batch 9 / 11 Loss: 0.065006\n",
      "Iter 71 / 128 Batch 10 / 11 Loss: 0.02621107\n",
      "Iter 72 / 128 Batch 0 / 11 Loss: 0.030483982\n",
      "Iter 72 / 128 Batch 1 / 11 Loss: 0.01503284\n",
      "Iter 72 / 128 Batch 2 / 11 Loss: 0.12317914\n",
      "Iter 72 / 128 Batch 3 / 11 Loss: 0.036434103\n",
      "Iter 72 / 128 Batch 4 / 11 Loss: 0.011659803\n",
      "Iter 72 / 128 Batch 5 / 11 Loss: 0.036544897\n",
      "Iter 72 / 128 Batch 6 / 11 Loss: 0.032288887\n",
      "Iter 72 / 128 Batch 7 / 11 Loss: 0.011183953\n",
      "Iter 72 / 128 Batch 8 / 11 Loss: 0.21826164\n",
      "Iter 72 / 128 Batch 9 / 11 Loss: 0.014381324\n",
      "Iter 72 / 128 Batch 10 / 11 Loss: 0.008248047\n",
      "Iter 73 / 128 Batch 0 / 11 Loss: 0.0056753466\n",
      "Iter 73 / 128 Batch 1 / 11 Loss: 0.015528421\n",
      "Iter 73 / 128 Batch 2 / 11 Loss: 0.14253943\n",
      "Iter 73 / 128 Batch 3 / 11 Loss: 0.0019757706\n",
      "Iter 73 / 128 Batch 4 / 11 Loss: 0.018571168\n",
      "Iter 73 / 128 Batch 5 / 11 Loss: 0.08297322\n",
      "Iter 73 / 128 Batch 6 / 11 Loss: 0.016518638\n",
      "Iter 73 / 128 Batch 7 / 11 Loss: 0.0028913952\n",
      "Iter 73 / 128 Batch 8 / 11 Loss: 0.005828598\n",
      "Iter 73 / 128 Batch 9 / 11 Loss: 0.022653135\n",
      "Iter 73 / 128 Batch 10 / 11 Loss: 0.006341825\n",
      "Iter 74 / 128 Batch 0 / 11 Loss: 0.007840601\n",
      "Iter 74 / 128 Batch 1 / 11 Loss: 0.004998048\n",
      "Iter 74 / 128 Batch 2 / 11 Loss: 0.121381335\n",
      "Iter 74 / 128 Batch 3 / 11 Loss: 0.001705637\n",
      "Iter 74 / 128 Batch 4 / 11 Loss: 0.010828232\n",
      "Iter 74 / 128 Batch 5 / 11 Loss: 0.058450423\n",
      "Iter 74 / 128 Batch 6 / 11 Loss: 0.025289083\n",
      "Iter 74 / 128 Batch 7 / 11 Loss: 0.0012598392\n",
      "Iter 74 / 128 Batch 8 / 11 Loss: 0.007194382\n",
      "Iter 74 / 128 Batch 9 / 11 Loss: 0.0121897645\n",
      "Iter 74 / 128 Batch 10 / 11 Loss: 0.017924085\n",
      "Iter 75 / 128 Batch 0 / 11 Loss: 0.008312095\n",
      "Iter 75 / 128 Batch 1 / 11 Loss: 0.05559893\n",
      "Iter 75 / 128 Batch 2 / 11 Loss: 0.051802233\n",
      "Iter 75 / 128 Batch 3 / 11 Loss: 0.002668727\n",
      "Iter 75 / 128 Batch 4 / 11 Loss: 0.058702048\n",
      "Iter 75 / 128 Batch 5 / 11 Loss: 0.039647035\n",
      "Iter 75 / 128 Batch 6 / 11 Loss: 0.017463928\n",
      "Iter 75 / 128 Batch 7 / 11 Loss: 0.0034090737\n",
      "Iter 75 / 128 Batch 8 / 11 Loss: 0.067010134\n",
      "Iter 75 / 128 Batch 9 / 11 Loss: 0.013295314\n",
      "Iter 75 / 128 Batch 10 / 11 Loss: 0.0062593017\n",
      "Iter 76 / 128 Batch 0 / 11 Loss: 0.0048980457\n",
      "Iter 76 / 128 Batch 1 / 11 Loss: 0.060927667\n",
      "Iter 76 / 128 Batch 2 / 11 Loss: 0.22409366\n",
      "Iter 76 / 128 Batch 3 / 11 Loss: 0.00813934\n",
      "Iter 76 / 128 Batch 4 / 11 Loss: 0.021715134\n",
      "Iter 76 / 128 Batch 5 / 11 Loss: 0.25195578\n",
      "Iter 76 / 128 Batch 6 / 11 Loss: 0.008655364\n",
      "Iter 76 / 128 Batch 7 / 11 Loss: 0.004866628\n",
      "Iter 76 / 128 Batch 8 / 11 Loss: 0.008966233\n",
      "Iter 76 / 128 Batch 9 / 11 Loss: 0.014082175\n",
      "Iter 76 / 128 Batch 10 / 11 Loss: 0.13791089\n",
      "Iter 77 / 128 Batch 0 / 11 Loss: 0.005416582\n",
      "Iter 77 / 128 Batch 1 / 11 Loss: 0.049406633\n",
      "Iter 77 / 128 Batch 2 / 11 Loss: 0.42853317\n",
      "Iter 77 / 128 Batch 3 / 11 Loss: 0.0077444846\n",
      "Iter 77 / 128 Batch 4 / 11 Loss: 0.049922973\n",
      "Iter 77 / 128 Batch 5 / 11 Loss: 0.03417512\n",
      "Iter 77 / 128 Batch 6 / 11 Loss: 0.020805366\n",
      "Iter 77 / 128 Batch 7 / 11 Loss: 0.0064399117\n",
      "Iter 77 / 128 Batch 8 / 11 Loss: 0.011001313\n",
      "Iter 77 / 128 Batch 9 / 11 Loss: 0.01655071\n",
      "Iter 77 / 128 Batch 10 / 11 Loss: 0.0073478916\n",
      "Iter 78 / 128 Batch 0 / 11 Loss: 0.035310354\n",
      "Iter 78 / 128 Batch 1 / 11 Loss: 0.0075496156\n",
      "Iter 78 / 128 Batch 2 / 11 Loss: 0.039392717\n",
      "Iter 78 / 128 Batch 3 / 11 Loss: 0.015844371\n",
      "Iter 78 / 128 Batch 4 / 11 Loss: 0.01199845\n",
      "Iter 78 / 128 Batch 5 / 11 Loss: 0.02848735\n",
      "Iter 78 / 128 Batch 6 / 11 Loss: 0.037147403\n",
      "Iter 78 / 128 Batch 7 / 11 Loss: 0.004288665\n",
      "Iter 78 / 128 Batch 8 / 11 Loss: 0.01471598\n",
      "Iter 78 / 128 Batch 9 / 11 Loss: 0.010658021\n",
      "Iter 78 / 128 Batch 10 / 11 Loss: 0.018247742\n",
      "Iter 79 / 128 Batch 0 / 11 Loss: 0.008659998\n",
      "Iter 79 / 128 Batch 1 / 11 Loss: 0.0034192386\n",
      "Iter 79 / 128 Batch 2 / 11 Loss: 0.058922675\n",
      "Iter 79 / 128 Batch 3 / 11 Loss: 0.0068399017\n",
      "Iter 79 / 128 Batch 4 / 11 Loss: 0.01579287\n",
      "Iter 79 / 128 Batch 5 / 11 Loss: 0.026353138\n",
      "Iter 79 / 128 Batch 6 / 11 Loss: 0.009834601\n",
      "Iter 79 / 128 Batch 7 / 11 Loss: 0.001584034\n",
      "Iter 79 / 128 Batch 8 / 11 Loss: 0.020855088\n",
      "Iter 79 / 128 Batch 9 / 11 Loss: 0.055579033\n",
      "Iter 79 / 128 Batch 10 / 11 Loss: 0.02028319\n",
      "Iter 80 / 128 Batch 0 / 11 Loss: 0.015526836\n",
      "Iter 80 / 128 Batch 1 / 11 Loss: 0.014039835\n",
      "Iter 80 / 128 Batch 2 / 11 Loss: 0.10486817\n",
      "Iter 80 / 128 Batch 3 / 11 Loss: 0.020075627\n",
      "Iter 80 / 128 Batch 4 / 11 Loss: 0.014143521\n",
      "Iter 80 / 128 Batch 5 / 11 Loss: 0.068750404\n",
      "Iter 80 / 128 Batch 6 / 11 Loss: 0.00856846\n",
      "Iter 80 / 128 Batch 7 / 11 Loss: 0.0009825097\n",
      "Iter 80 / 128 Batch 8 / 11 Loss: 0.01219\n",
      "Iter 80 / 128 Batch 9 / 11 Loss: 0.066634625\n",
      "Iter 80 / 128 Batch 10 / 11 Loss: 0.014590931\n",
      "Iter 81 / 128 Batch 0 / 11 Loss: 0.0026475322\n",
      "Iter 81 / 128 Batch 1 / 11 Loss: 0.006357869\n",
      "Iter 81 / 128 Batch 2 / 11 Loss: 0.07314046\n",
      "Iter 81 / 128 Batch 3 / 11 Loss: 0.002974874\n",
      "Iter 81 / 128 Batch 4 / 11 Loss: 0.011800919\n",
      "Iter 81 / 128 Batch 5 / 11 Loss: 0.017502094\n",
      "Iter 81 / 128 Batch 6 / 11 Loss: 0.009030606\n",
      "Iter 81 / 128 Batch 7 / 11 Loss: 0.0026811387\n",
      "Iter 81 / 128 Batch 8 / 11 Loss: 0.003997273\n",
      "Iter 81 / 128 Batch 9 / 11 Loss: 0.037020903\n",
      "Iter 81 / 128 Batch 10 / 11 Loss: 0.020000387\n",
      "Iter 82 / 128 Batch 0 / 11 Loss: 0.016628254\n",
      "Iter 82 / 128 Batch 1 / 11 Loss: 0.005682236\n",
      "Iter 82 / 128 Batch 2 / 11 Loss: 0.081014596\n",
      "Iter 82 / 128 Batch 3 / 11 Loss: 0.011868246\n",
      "Iter 82 / 128 Batch 4 / 11 Loss: 0.00831487\n",
      "Iter 82 / 128 Batch 5 / 11 Loss: 0.043988075\n",
      "Iter 82 / 128 Batch 6 / 11 Loss: 0.01716771\n",
      "Iter 82 / 128 Batch 7 / 11 Loss: 0.0039344314\n",
      "Iter 82 / 128 Batch 8 / 11 Loss: 0.021997975\n",
      "Iter 82 / 128 Batch 9 / 11 Loss: 0.038453124\n",
      "Iter 82 / 128 Batch 10 / 11 Loss: 0.001595257\n",
      "Iter 83 / 128 Batch 0 / 11 Loss: 0.0037490039\n",
      "Iter 83 / 128 Batch 1 / 11 Loss: 0.004282506\n",
      "Iter 83 / 128 Batch 2 / 11 Loss: 0.04563434\n",
      "Iter 83 / 128 Batch 3 / 11 Loss: 0.0028618323\n",
      "Iter 83 / 128 Batch 4 / 11 Loss: 0.012091257\n",
      "Iter 83 / 128 Batch 5 / 11 Loss: 0.03538301\n",
      "Iter 83 / 128 Batch 6 / 11 Loss: 0.010705047\n",
      "Iter 83 / 128 Batch 7 / 11 Loss: 0.005349583\n",
      "Iter 83 / 128 Batch 8 / 11 Loss: 0.009565688\n",
      "Iter 83 / 128 Batch 9 / 11 Loss: 0.017109947\n",
      "Iter 83 / 128 Batch 10 / 11 Loss: 0.0099497\n",
      "Iter 84 / 128 Batch 0 / 11 Loss: 0.008861022\n",
      "Iter 84 / 128 Batch 1 / 11 Loss: 0.031726174\n",
      "Iter 84 / 128 Batch 2 / 11 Loss: 0.109707266\n",
      "Iter 84 / 128 Batch 3 / 11 Loss: 0.030765034\n",
      "Iter 84 / 128 Batch 4 / 11 Loss: 0.009768889\n",
      "Iter 84 / 128 Batch 5 / 11 Loss: 0.022686196\n",
      "Iter 84 / 128 Batch 6 / 11 Loss: 0.04837071\n",
      "Iter 84 / 128 Batch 7 / 11 Loss: 0.0017848035\n",
      "Iter 84 / 128 Batch 8 / 11 Loss: 0.006528208\n",
      "Iter 84 / 128 Batch 9 / 11 Loss: 0.045419283\n",
      "Iter 84 / 128 Batch 10 / 11 Loss: 0.0057584075\n",
      "Iter 85 / 128 Batch 0 / 11 Loss: 0.013127537\n",
      "Iter 85 / 128 Batch 1 / 11 Loss: 0.0014655124\n",
      "Iter 85 / 128 Batch 2 / 11 Loss: 0.12251169\n",
      "Iter 85 / 128 Batch 3 / 11 Loss: 0.00452841\n",
      "Iter 85 / 128 Batch 4 / 11 Loss: 0.018936843\n",
      "Iter 85 / 128 Batch 5 / 11 Loss: 0.0139698\n",
      "Iter 85 / 128 Batch 6 / 11 Loss: 0.005929753\n",
      "Iter 85 / 128 Batch 7 / 11 Loss: 0.006338313\n",
      "Iter 85 / 128 Batch 8 / 11 Loss: 0.006895454\n",
      "Iter 85 / 128 Batch 9 / 11 Loss: 0.011372693\n",
      "Iter 85 / 128 Batch 10 / 11 Loss: 0.016603976\n",
      "Iter 86 / 128 Batch 0 / 11 Loss: 0.0022512337\n",
      "Iter 86 / 128 Batch 1 / 11 Loss: 0.0038955796\n",
      "Iter 86 / 128 Batch 2 / 11 Loss: 0.22774336\n",
      "Iter 86 / 128 Batch 3 / 11 Loss: 0.005374206\n",
      "Iter 86 / 128 Batch 4 / 11 Loss: 0.09932595\n",
      "Iter 86 / 128 Batch 5 / 11 Loss: 0.013607614\n",
      "Iter 86 / 128 Batch 6 / 11 Loss: 0.041194323\n",
      "Iter 86 / 128 Batch 7 / 11 Loss: 0.016146643\n",
      "Iter 86 / 128 Batch 8 / 11 Loss: 0.008350333\n",
      "Iter 86 / 128 Batch 9 / 11 Loss: 0.012396229\n",
      "Iter 86 / 128 Batch 10 / 11 Loss: 0.005377926\n",
      "Iter 87 / 128 Batch 0 / 11 Loss: 0.0027746216\n",
      "Iter 87 / 128 Batch 1 / 11 Loss: 0.106728755\n",
      "Iter 87 / 128 Batch 2 / 11 Loss: 0.038321093\n",
      "Iter 87 / 128 Batch 3 / 11 Loss: 0.007267261\n",
      "Iter 87 / 128 Batch 4 / 11 Loss: 0.0042384565\n",
      "Iter 87 / 128 Batch 5 / 11 Loss: 0.041669898\n",
      "Iter 87 / 128 Batch 6 / 11 Loss: 0.007713765\n",
      "Iter 87 / 128 Batch 7 / 11 Loss: 0.0083531\n",
      "Iter 87 / 128 Batch 8 / 11 Loss: 0.0064873314\n",
      "Iter 87 / 128 Batch 9 / 11 Loss: 0.013113235\n",
      "Iter 87 / 128 Batch 10 / 11 Loss: 0.010787563\n",
      "Iter 88 / 128 Batch 0 / 11 Loss: 0.0021992833\n",
      "Iter 88 / 128 Batch 1 / 11 Loss: 0.011397404\n",
      "Iter 88 / 128 Batch 2 / 11 Loss: 0.038285095\n",
      "Iter 88 / 128 Batch 3 / 11 Loss: 0.008479724\n",
      "Iter 88 / 128 Batch 4 / 11 Loss: 0.003657754\n",
      "Iter 88 / 128 Batch 5 / 11 Loss: 0.030254178\n",
      "Iter 88 / 128 Batch 6 / 11 Loss: 0.01792477\n",
      "Iter 88 / 128 Batch 7 / 11 Loss: 0.0054421406\n",
      "Iter 88 / 128 Batch 8 / 11 Loss: 0.00670249\n",
      "Iter 88 / 128 Batch 9 / 11 Loss: 0.011454211\n",
      "Iter 88 / 128 Batch 10 / 11 Loss: 0.010866739\n",
      "Iter 89 / 128 Batch 0 / 11 Loss: 0.0041553797\n",
      "Iter 89 / 128 Batch 1 / 11 Loss: 0.20580724\n",
      "Iter 89 / 128 Batch 2 / 11 Loss: 0.027468143\n",
      "Iter 89 / 128 Batch 3 / 11 Loss: 0.01733768\n",
      "Iter 89 / 128 Batch 4 / 11 Loss: 0.14624786\n",
      "Iter 89 / 128 Batch 5 / 11 Loss: 0.0097262\n",
      "Iter 89 / 128 Batch 6 / 11 Loss: 0.010387338\n",
      "Iter 89 / 128 Batch 7 / 11 Loss: 0.0039432924\n",
      "Iter 89 / 128 Batch 8 / 11 Loss: 0.008827047\n",
      "Iter 89 / 128 Batch 9 / 11 Loss: 0.003499533\n",
      "Iter 89 / 128 Batch 10 / 11 Loss: 0.0054393145\n",
      "Iter 90 / 128 Batch 0 / 11 Loss: 0.0039802273\n",
      "Iter 90 / 128 Batch 1 / 11 Loss: 0.03868023\n",
      "Iter 90 / 128 Batch 2 / 11 Loss: 0.18864863\n",
      "Iter 90 / 128 Batch 3 / 11 Loss: 0.012472864\n",
      "Iter 90 / 128 Batch 4 / 11 Loss: 0.019270612\n",
      "Iter 90 / 128 Batch 5 / 11 Loss: 0.030359052\n",
      "Iter 90 / 128 Batch 6 / 11 Loss: 0.022512015\n",
      "Iter 90 / 128 Batch 7 / 11 Loss: 0.0030265884\n",
      "Iter 90 / 128 Batch 8 / 11 Loss: 0.0046189367\n",
      "Iter 90 / 128 Batch 9 / 11 Loss: 0.013412323\n",
      "Iter 90 / 128 Batch 10 / 11 Loss: 0.0047749667\n",
      "Iter 91 / 128 Batch 0 / 11 Loss: 0.0018705141\n",
      "Iter 91 / 128 Batch 1 / 11 Loss: 0.06991014\n",
      "Iter 91 / 128 Batch 2 / 11 Loss: 0.052294377\n",
      "Iter 91 / 128 Batch 3 / 11 Loss: 0.0034374855\n",
      "Iter 91 / 128 Batch 4 / 11 Loss: 0.0073367646\n",
      "Iter 91 / 128 Batch 5 / 11 Loss: 0.020215722\n",
      "Iter 91 / 128 Batch 6 / 11 Loss: 0.029376198\n",
      "Iter 91 / 128 Batch 7 / 11 Loss: 0.000619729\n",
      "Iter 91 / 128 Batch 8 / 11 Loss: 0.016341563\n",
      "Iter 91 / 128 Batch 9 / 11 Loss: 0.020362228\n",
      "Iter 91 / 128 Batch 10 / 11 Loss: 0.022680627\n",
      "Iter 92 / 128 Batch 0 / 11 Loss: 0.028840039\n",
      "Iter 92 / 128 Batch 1 / 11 Loss: 0.010329071\n",
      "Iter 92 / 128 Batch 2 / 11 Loss: 0.029049072\n",
      "Iter 92 / 128 Batch 3 / 11 Loss: 0.004563311\n",
      "Iter 92 / 128 Batch 4 / 11 Loss: 0.031999957\n",
      "Iter 92 / 128 Batch 5 / 11 Loss: 0.037988633\n",
      "Iter 92 / 128 Batch 6 / 11 Loss: 0.00955287\n",
      "Iter 92 / 128 Batch 7 / 11 Loss: 0.002984358\n",
      "Iter 92 / 128 Batch 8 / 11 Loss: 0.008214518\n",
      "Iter 92 / 128 Batch 9 / 11 Loss: 0.012251644\n",
      "Iter 92 / 128 Batch 10 / 11 Loss: 0.007195856\n",
      "Iter 93 / 128 Batch 0 / 11 Loss: 0.011139577\n",
      "Iter 93 / 128 Batch 1 / 11 Loss: 0.005458447\n",
      "Iter 93 / 128 Batch 2 / 11 Loss: 0.06798137\n",
      "Iter 93 / 128 Batch 3 / 11 Loss: 0.0017821433\n",
      "Iter 93 / 128 Batch 4 / 11 Loss: 0.0030409447\n",
      "Iter 93 / 128 Batch 5 / 11 Loss: 0.013112614\n",
      "Iter 93 / 128 Batch 6 / 11 Loss: 0.008788275\n",
      "Iter 93 / 128 Batch 7 / 11 Loss: 0.001982587\n",
      "Iter 93 / 128 Batch 8 / 11 Loss: 0.0035026018\n",
      "Iter 93 / 128 Batch 9 / 11 Loss: 0.0056742355\n",
      "Iter 93 / 128 Batch 10 / 11 Loss: 0.015118048\n",
      "Iter 94 / 128 Batch 0 / 11 Loss: 0.0035439737\n",
      "Iter 94 / 128 Batch 1 / 11 Loss: 0.0062425476\n",
      "Iter 94 / 128 Batch 2 / 11 Loss: 0.073160425\n",
      "Iter 94 / 128 Batch 3 / 11 Loss: 0.0054892935\n",
      "Iter 94 / 128 Batch 4 / 11 Loss: 0.009975987\n",
      "Iter 94 / 128 Batch 5 / 11 Loss: 0.03362537\n",
      "Iter 94 / 128 Batch 6 / 11 Loss: 0.016361393\n",
      "Iter 94 / 128 Batch 7 / 11 Loss: 0.015822167\n",
      "Iter 94 / 128 Batch 8 / 11 Loss: 0.006442708\n",
      "Iter 94 / 128 Batch 9 / 11 Loss: 0.060487665\n",
      "Iter 94 / 128 Batch 10 / 11 Loss: 0.016150327\n",
      "Iter 95 / 128 Batch 0 / 11 Loss: 0.021155009\n",
      "Iter 95 / 128 Batch 1 / 11 Loss: 0.0033418238\n",
      "Iter 95 / 128 Batch 2 / 11 Loss: 0.038456947\n",
      "Iter 95 / 128 Batch 3 / 11 Loss: 0.0055814357\n",
      "Iter 95 / 128 Batch 4 / 11 Loss: 0.0078121386\n",
      "Iter 95 / 128 Batch 5 / 11 Loss: 0.007139857\n",
      "Iter 95 / 128 Batch 6 / 11 Loss: 0.01425053\n",
      "Iter 95 / 128 Batch 7 / 11 Loss: 0.004764229\n",
      "Iter 95 / 128 Batch 8 / 11 Loss: 0.007911089\n",
      "Iter 95 / 128 Batch 9 / 11 Loss: 0.008640132\n",
      "Iter 95 / 128 Batch 10 / 11 Loss: 0.005991838\n",
      "Iter 96 / 128 Batch 0 / 11 Loss: 0.004938681\n",
      "Iter 96 / 128 Batch 1 / 11 Loss: 0.011733071\n",
      "Iter 96 / 128 Batch 2 / 11 Loss: 0.035388004\n",
      "Iter 96 / 128 Batch 3 / 11 Loss: 0.005354422\n",
      "Iter 96 / 128 Batch 4 / 11 Loss: 0.0056772307\n",
      "Iter 96 / 128 Batch 5 / 11 Loss: 0.012887208\n",
      "Iter 96 / 128 Batch 6 / 11 Loss: 0.034208316\n",
      "Iter 96 / 128 Batch 7 / 11 Loss: 0.00062509556\n",
      "Iter 96 / 128 Batch 8 / 11 Loss: 0.010040309\n",
      "Iter 96 / 128 Batch 9 / 11 Loss: 0.008482223\n",
      "Iter 96 / 128 Batch 10 / 11 Loss: 0.023775598\n",
      "Iter 97 / 128 Batch 0 / 11 Loss: 0.010541378\n",
      "Iter 97 / 128 Batch 1 / 11 Loss: 0.00613019\n",
      "Iter 97 / 128 Batch 2 / 11 Loss: 0.10844099\n",
      "Iter 97 / 128 Batch 3 / 11 Loss: 0.00921266\n",
      "Iter 97 / 128 Batch 4 / 11 Loss: 0.00843074\n",
      "Iter 97 / 128 Batch 5 / 11 Loss: 0.009999288\n",
      "Iter 97 / 128 Batch 6 / 11 Loss: 0.00229477\n",
      "Iter 97 / 128 Batch 7 / 11 Loss: 0.007935749\n",
      "Iter 97 / 128 Batch 8 / 11 Loss: 0.0054224916\n",
      "Iter 97 / 128 Batch 9 / 11 Loss: 0.004281028\n",
      "Iter 97 / 128 Batch 10 / 11 Loss: 0.008040127\n",
      "Iter 98 / 128 Batch 0 / 11 Loss: 0.0016421848\n",
      "Iter 98 / 128 Batch 1 / 11 Loss: 0.09731769\n",
      "Iter 98 / 128 Batch 2 / 11 Loss: 0.05726113\n",
      "Iter 98 / 128 Batch 3 / 11 Loss: 0.0053299656\n",
      "Iter 98 / 128 Batch 4 / 11 Loss: 0.014185274\n",
      "Iter 98 / 128 Batch 5 / 11 Loss: 0.06291301\n",
      "Iter 98 / 128 Batch 6 / 11 Loss: 0.019147184\n",
      "Iter 98 / 128 Batch 7 / 11 Loss: 0.0038965128\n",
      "Iter 98 / 128 Batch 8 / 11 Loss: 0.057823136\n",
      "Iter 98 / 128 Batch 9 / 11 Loss: 0.022382338\n",
      "Iter 98 / 128 Batch 10 / 11 Loss: 0.026780106\n",
      "Iter 99 / 128 Batch 0 / 11 Loss: 0.007277614\n",
      "Iter 99 / 128 Batch 1 / 11 Loss: 0.0021192196\n",
      "Iter 99 / 128 Batch 2 / 11 Loss: 0.042475298\n",
      "Iter 99 / 128 Batch 3 / 11 Loss: 0.003596493\n",
      "Iter 99 / 128 Batch 4 / 11 Loss: 0.003821384\n",
      "Iter 99 / 128 Batch 5 / 11 Loss: 0.018862763\n",
      "Iter 99 / 128 Batch 6 / 11 Loss: 0.006290034\n",
      "Iter 99 / 128 Batch 7 / 11 Loss: 0.0037250305\n",
      "Iter 99 / 128 Batch 8 / 11 Loss: 0.020335453\n",
      "Iter 99 / 128 Batch 9 / 11 Loss: 0.111193635\n",
      "Iter 99 / 128 Batch 10 / 11 Loss: 0.010499019\n",
      "Iter 100 / 128 Batch 0 / 11 Loss: 0.0083757825\n",
      "Iter 100 / 128 Batch 1 / 11 Loss: 0.021446254\n",
      "Iter 100 / 128 Batch 2 / 11 Loss: 0.04463145\n",
      "Iter 100 / 128 Batch 3 / 11 Loss: 0.02805463\n",
      "Iter 100 / 128 Batch 4 / 11 Loss: 0.008928141\n",
      "Iter 100 / 128 Batch 5 / 11 Loss: 0.031560577\n",
      "Iter 100 / 128 Batch 6 / 11 Loss: 0.0080888895\n",
      "Iter 100 / 128 Batch 7 / 11 Loss: 0.0010634239\n",
      "Iter 100 / 128 Batch 8 / 11 Loss: 0.029632438\n",
      "Iter 100 / 128 Batch 9 / 11 Loss: 0.019653913\n",
      "Iter 100 / 128 Batch 10 / 11 Loss: 0.009119138\n",
      "Iter 101 / 128 Batch 0 / 11 Loss: 0.0067238077\n",
      "Iter 101 / 128 Batch 1 / 11 Loss: 0.005888483\n",
      "Iter 101 / 128 Batch 2 / 11 Loss: 0.043939114\n",
      "Iter 101 / 128 Batch 3 / 11 Loss: 0.012055537\n",
      "Iter 101 / 128 Batch 4 / 11 Loss: 0.0032324633\n",
      "Iter 101 / 128 Batch 5 / 11 Loss: 0.017965272\n",
      "Iter 101 / 128 Batch 6 / 11 Loss: 0.008013576\n",
      "Iter 101 / 128 Batch 7 / 11 Loss: 0.000743664\n",
      "Iter 101 / 128 Batch 8 / 11 Loss: 0.053354815\n",
      "Iter 101 / 128 Batch 9 / 11 Loss: 0.03320891\n",
      "Iter 101 / 128 Batch 10 / 11 Loss: 0.015987689\n",
      "Iter 102 / 128 Batch 0 / 11 Loss: 0.0031668316\n",
      "Iter 102 / 128 Batch 1 / 11 Loss: 0.0027106577\n",
      "Iter 102 / 128 Batch 2 / 11 Loss: 0.052854437\n",
      "Iter 102 / 128 Batch 3 / 11 Loss: 0.002195797\n",
      "Iter 102 / 128 Batch 4 / 11 Loss: 0.011222206\n",
      "Iter 102 / 128 Batch 5 / 11 Loss: 0.01356368\n",
      "Iter 102 / 128 Batch 6 / 11 Loss: 0.066563666\n",
      "Iter 102 / 128 Batch 7 / 11 Loss: 0.009712528\n",
      "Iter 102 / 128 Batch 8 / 11 Loss: 0.003037429\n",
      "Iter 102 / 128 Batch 9 / 11 Loss: 0.011260388\n",
      "Iter 102 / 128 Batch 10 / 11 Loss: 0.0033332105\n",
      "Iter 103 / 128 Batch 0 / 11 Loss: 0.0015151721\n",
      "Iter 103 / 128 Batch 1 / 11 Loss: 0.005809077\n",
      "Iter 103 / 128 Batch 2 / 11 Loss: 0.027855646\n",
      "Iter 103 / 128 Batch 3 / 11 Loss: 0.0021244208\n",
      "Iter 103 / 128 Batch 4 / 11 Loss: 0.0040568537\n",
      "Iter 103 / 128 Batch 5 / 11 Loss: 0.01122403\n",
      "Iter 103 / 128 Batch 6 / 11 Loss: 0.0025994577\n",
      "Iter 103 / 128 Batch 7 / 11 Loss: 0.003685019\n",
      "Iter 103 / 128 Batch 8 / 11 Loss: 0.010952812\n",
      "Iter 103 / 128 Batch 9 / 11 Loss: 0.008357607\n",
      "Iter 103 / 128 Batch 10 / 11 Loss: 0.02599512\n",
      "Iter 104 / 128 Batch 0 / 11 Loss: 0.0057282066\n",
      "Iter 104 / 128 Batch 1 / 11 Loss: 0.0052483324\n",
      "Iter 104 / 128 Batch 2 / 11 Loss: 0.056696516\n",
      "Iter 104 / 128 Batch 3 / 11 Loss: 0.0012553114\n",
      "Iter 104 / 128 Batch 4 / 11 Loss: 0.0050351173\n",
      "Iter 104 / 128 Batch 5 / 11 Loss: 0.009284781\n",
      "Iter 104 / 128 Batch 6 / 11 Loss: 0.006962404\n",
      "Iter 104 / 128 Batch 7 / 11 Loss: 0.007244573\n",
      "Iter 104 / 128 Batch 8 / 11 Loss: 0.020798208\n",
      "Iter 104 / 128 Batch 9 / 11 Loss: 0.016899765\n",
      "Iter 104 / 128 Batch 10 / 11 Loss: 0.07492624\n",
      "Iter 105 / 128 Batch 0 / 11 Loss: 0.0010366688\n",
      "Iter 105 / 128 Batch 1 / 11 Loss: 0.005711875\n",
      "Iter 105 / 128 Batch 2 / 11 Loss: 0.13719475\n",
      "Iter 105 / 128 Batch 3 / 11 Loss: 0.003966217\n",
      "Iter 105 / 128 Batch 4 / 11 Loss: 0.012322229\n",
      "Iter 105 / 128 Batch 5 / 11 Loss: 0.012288736\n",
      "Iter 105 / 128 Batch 6 / 11 Loss: 0.003856359\n",
      "Iter 105 / 128 Batch 7 / 11 Loss: 0.0017911362\n",
      "Iter 105 / 128 Batch 8 / 11 Loss: 0.0073953257\n",
      "Iter 105 / 128 Batch 9 / 11 Loss: 0.0033731614\n",
      "Iter 105 / 128 Batch 10 / 11 Loss: 0.0067088758\n",
      "Iter 106 / 128 Batch 0 / 11 Loss: 0.004324766\n",
      "Iter 106 / 128 Batch 1 / 11 Loss: 0.027773295\n",
      "Iter 106 / 128 Batch 2 / 11 Loss: 0.15307572\n",
      "Iter 106 / 128 Batch 3 / 11 Loss: 0.0031602576\n",
      "Iter 106 / 128 Batch 4 / 11 Loss: 0.014412812\n",
      "Iter 106 / 128 Batch 5 / 11 Loss: 0.019660367\n",
      "Iter 106 / 128 Batch 6 / 11 Loss: 0.017416313\n",
      "Iter 106 / 128 Batch 7 / 11 Loss: 0.00038865441\n",
      "Iter 106 / 128 Batch 8 / 11 Loss: 0.06532082\n",
      "Iter 106 / 128 Batch 9 / 11 Loss: 0.010061422\n",
      "Iter 106 / 128 Batch 10 / 11 Loss: 0.0032343932\n",
      "Iter 107 / 128 Batch 0 / 11 Loss: 0.0030616485\n",
      "Iter 107 / 128 Batch 1 / 11 Loss: 0.005314146\n",
      "Iter 107 / 128 Batch 2 / 11 Loss: 0.05272664\n",
      "Iter 107 / 128 Batch 3 / 11 Loss: 0.0026744907\n",
      "Iter 107 / 128 Batch 4 / 11 Loss: 0.050492257\n",
      "Iter 107 / 128 Batch 5 / 11 Loss: 0.018391646\n",
      "Iter 107 / 128 Batch 6 / 11 Loss: 0.044557255\n",
      "Iter 107 / 128 Batch 7 / 11 Loss: 0.0029615518\n",
      "Iter 107 / 128 Batch 8 / 11 Loss: 0.0035076523\n",
      "Iter 107 / 128 Batch 9 / 11 Loss: 0.011979327\n",
      "Iter 107 / 128 Batch 10 / 11 Loss: 0.0010419162\n",
      "Iter 108 / 128 Batch 0 / 11 Loss: 0.011732534\n",
      "Iter 108 / 128 Batch 1 / 11 Loss: 0.010468576\n",
      "Iter 108 / 128 Batch 2 / 11 Loss: 0.034332562\n",
      "Iter 108 / 128 Batch 3 / 11 Loss: 0.002103681\n",
      "Iter 108 / 128 Batch 4 / 11 Loss: 0.001190269\n",
      "Iter 108 / 128 Batch 5 / 11 Loss: 0.07863802\n",
      "Iter 108 / 128 Batch 6 / 11 Loss: 0.0038829795\n",
      "Iter 108 / 128 Batch 7 / 11 Loss: 0.000435427\n",
      "Iter 108 / 128 Batch 8 / 11 Loss: 0.0013892483\n",
      "Iter 108 / 128 Batch 9 / 11 Loss: 0.0052199513\n",
      "Iter 108 / 128 Batch 10 / 11 Loss: 0.004091447\n",
      "Iter 109 / 128 Batch 0 / 11 Loss: 0.0038762898\n",
      "Iter 109 / 128 Batch 1 / 11 Loss: 0.04556167\n",
      "Iter 109 / 128 Batch 2 / 11 Loss: 0.04516636\n",
      "Iter 109 / 128 Batch 3 / 11 Loss: 0.008186951\n",
      "Iter 109 / 128 Batch 4 / 11 Loss: 0.009166372\n",
      "Iter 109 / 128 Batch 5 / 11 Loss: 0.016139718\n",
      "Iter 109 / 128 Batch 6 / 11 Loss: 0.006479673\n",
      "Iter 109 / 128 Batch 7 / 11 Loss: 0.0015718116\n",
      "Iter 109 / 128 Batch 8 / 11 Loss: 0.041719712\n",
      "Iter 109 / 128 Batch 9 / 11 Loss: 0.009233026\n",
      "Iter 109 / 128 Batch 10 / 11 Loss: 0.013241099\n",
      "Iter 110 / 128 Batch 0 / 11 Loss: 0.004889302\n",
      "Iter 110 / 128 Batch 1 / 11 Loss: 0.031213574\n",
      "Iter 110 / 128 Batch 2 / 11 Loss: 0.010767114\n",
      "Iter 110 / 128 Batch 3 / 11 Loss: 0.002833976\n",
      "Iter 110 / 128 Batch 4 / 11 Loss: 0.0072518755\n",
      "Iter 110 / 128 Batch 5 / 11 Loss: 0.014536437\n",
      "Iter 110 / 128 Batch 6 / 11 Loss: 0.010970151\n",
      "Iter 110 / 128 Batch 7 / 11 Loss: 0.0017829849\n",
      "Iter 110 / 128 Batch 8 / 11 Loss: 0.0026844416\n",
      "Iter 110 / 128 Batch 9 / 11 Loss: 0.011510038\n",
      "Iter 110 / 128 Batch 10 / 11 Loss: 0.0040410645\n",
      "Iter 111 / 128 Batch 0 / 11 Loss: 0.024676008\n",
      "Iter 111 / 128 Batch 1 / 11 Loss: 0.0014489833\n",
      "Iter 111 / 128 Batch 2 / 11 Loss: 0.03410055\n",
      "Iter 111 / 128 Batch 3 / 11 Loss: 0.027472995\n",
      "Iter 111 / 128 Batch 4 / 11 Loss: 0.0086239\n",
      "Iter 111 / 128 Batch 5 / 11 Loss: 0.008306488\n",
      "Iter 111 / 128 Batch 6 / 11 Loss: 0.005975291\n",
      "Iter 111 / 128 Batch 7 / 11 Loss: 0.00074745837\n",
      "Iter 111 / 128 Batch 8 / 11 Loss: 0.003285239\n",
      "Iter 111 / 128 Batch 9 / 11 Loss: 0.0076373676\n",
      "Iter 111 / 128 Batch 10 / 11 Loss: 0.0042757345\n",
      "Iter 112 / 128 Batch 0 / 11 Loss: 0.0018100846\n",
      "Iter 112 / 128 Batch 1 / 11 Loss: 0.008728321\n",
      "Iter 112 / 128 Batch 2 / 11 Loss: 0.2536215\n",
      "Iter 112 / 128 Batch 3 / 11 Loss: 0.09662236\n",
      "Iter 112 / 128 Batch 4 / 11 Loss: 0.004217826\n",
      "Iter 112 / 128 Batch 5 / 11 Loss: 0.011841401\n",
      "Iter 112 / 128 Batch 6 / 11 Loss: 0.056176912\n",
      "Iter 112 / 128 Batch 7 / 11 Loss: 0.004613881\n",
      "Iter 112 / 128 Batch 8 / 11 Loss: 0.009245831\n",
      "Iter 112 / 128 Batch 9 / 11 Loss: 0.0113120265\n",
      "Iter 112 / 128 Batch 10 / 11 Loss: 0.010001968\n",
      "Iter 113 / 128 Batch 0 / 11 Loss: 0.0063748765\n",
      "Iter 113 / 128 Batch 1 / 11 Loss: 0.0062170187\n",
      "Iter 113 / 128 Batch 2 / 11 Loss: 0.054304093\n",
      "Iter 113 / 128 Batch 3 / 11 Loss: 0.025215432\n",
      "Iter 113 / 128 Batch 4 / 11 Loss: 0.011844021\n",
      "Iter 113 / 128 Batch 5 / 11 Loss: 0.011988678\n",
      "Iter 113 / 128 Batch 6 / 11 Loss: 0.0032282579\n",
      "Iter 113 / 128 Batch 7 / 11 Loss: 0.0016688446\n",
      "Iter 113 / 128 Batch 8 / 11 Loss: 0.016286358\n",
      "Iter 113 / 128 Batch 9 / 11 Loss: 0.022466933\n",
      "Iter 113 / 128 Batch 10 / 11 Loss: 0.0038768866\n",
      "Iter 114 / 128 Batch 0 / 11 Loss: 0.0039950004\n",
      "Iter 114 / 128 Batch 1 / 11 Loss: 0.0010404675\n",
      "Iter 114 / 128 Batch 2 / 11 Loss: 0.07904345\n",
      "Iter 114 / 128 Batch 3 / 11 Loss: 0.015764568\n",
      "Iter 114 / 128 Batch 4 / 11 Loss: 0.0058103153\n",
      "Iter 114 / 128 Batch 5 / 11 Loss: 0.021762699\n",
      "Iter 114 / 128 Batch 6 / 11 Loss: 0.020842858\n",
      "Iter 114 / 128 Batch 7 / 11 Loss: 0.0012633634\n",
      "Iter 114 / 128 Batch 8 / 11 Loss: 0.043196484\n",
      "Iter 114 / 128 Batch 9 / 11 Loss: 0.011320742\n",
      "Iter 114 / 128 Batch 10 / 11 Loss: 0.07918576\n",
      "Iter 115 / 128 Batch 0 / 11 Loss: 0.0011798355\n",
      "Iter 115 / 128 Batch 1 / 11 Loss: 0.012385452\n",
      "Iter 115 / 128 Batch 2 / 11 Loss: 0.07516119\n",
      "Iter 115 / 128 Batch 3 / 11 Loss: 0.0032613603\n",
      "Iter 115 / 128 Batch 4 / 11 Loss: 0.0035365936\n",
      "Iter 115 / 128 Batch 5 / 11 Loss: 0.009379916\n",
      "Iter 115 / 128 Batch 6 / 11 Loss: 0.025395762\n",
      "Iter 115 / 128 Batch 7 / 11 Loss: 0.0067430334\n",
      "Iter 115 / 128 Batch 8 / 11 Loss: 0.007130298\n",
      "Iter 115 / 128 Batch 9 / 11 Loss: 0.053454414\n",
      "Iter 115 / 128 Batch 10 / 11 Loss: 0.014101192\n",
      "Iter 116 / 128 Batch 0 / 11 Loss: 0.005476855\n",
      "Iter 116 / 128 Batch 1 / 11 Loss: 0.0061762873\n",
      "Iter 116 / 128 Batch 2 / 11 Loss: 0.05174338\n",
      "Iter 116 / 128 Batch 3 / 11 Loss: 0.007357399\n",
      "Iter 116 / 128 Batch 4 / 11 Loss: 0.0033999982\n",
      "Iter 116 / 128 Batch 5 / 11 Loss: 0.012596625\n",
      "Iter 116 / 128 Batch 6 / 11 Loss: 0.011317038\n",
      "Iter 116 / 128 Batch 7 / 11 Loss: 0.0022196462\n",
      "Iter 116 / 128 Batch 8 / 11 Loss: 0.027619513\n",
      "Iter 116 / 128 Batch 9 / 11 Loss: 0.005622736\n",
      "Iter 116 / 128 Batch 10 / 11 Loss: 0.004203738\n",
      "Iter 117 / 128 Batch 0 / 11 Loss: 0.001481855\n",
      "Iter 117 / 128 Batch 1 / 11 Loss: 0.010532798\n",
      "Iter 117 / 128 Batch 2 / 11 Loss: 0.07462712\n",
      "Iter 117 / 128 Batch 3 / 11 Loss: 0.012687252\n",
      "Iter 117 / 128 Batch 4 / 11 Loss: 0.00905857\n",
      "Iter 117 / 128 Batch 5 / 11 Loss: 0.049192917\n",
      "Iter 117 / 128 Batch 6 / 11 Loss: 0.01078871\n",
      "Iter 117 / 128 Batch 7 / 11 Loss: 0.006316674\n",
      "Iter 117 / 128 Batch 8 / 11 Loss: 0.006816442\n",
      "Iter 117 / 128 Batch 9 / 11 Loss: 0.03101837\n",
      "Iter 117 / 128 Batch 10 / 11 Loss: 0.0022551469\n",
      "Iter 118 / 128 Batch 0 / 11 Loss: 0.0032527638\n",
      "Iter 118 / 128 Batch 1 / 11 Loss: 0.0058603706\n",
      "Iter 118 / 128 Batch 2 / 11 Loss: 0.036828358\n",
      "Iter 118 / 128 Batch 3 / 11 Loss: 0.005013882\n",
      "Iter 118 / 128 Batch 4 / 11 Loss: 0.003000869\n",
      "Iter 118 / 128 Batch 5 / 11 Loss: 0.011746724\n",
      "Iter 118 / 128 Batch 6 / 11 Loss: 0.007311576\n",
      "Iter 118 / 128 Batch 7 / 11 Loss: 0.00097023696\n",
      "Iter 118 / 128 Batch 8 / 11 Loss: 0.030194122\n",
      "Iter 118 / 128 Batch 9 / 11 Loss: 0.0061252075\n",
      "Iter 118 / 128 Batch 10 / 11 Loss: 0.0050423155\n",
      "Iter 119 / 128 Batch 0 / 11 Loss: 0.0068288133\n",
      "Iter 119 / 128 Batch 1 / 11 Loss: 0.0037308082\n",
      "Iter 119 / 128 Batch 2 / 11 Loss: 0.033032503\n",
      "Iter 119 / 128 Batch 3 / 11 Loss: 0.0030453028\n",
      "Iter 119 / 128 Batch 4 / 11 Loss: 0.023609944\n",
      "Iter 119 / 128 Batch 5 / 11 Loss: 0.015361892\n",
      "Iter 119 / 128 Batch 6 / 11 Loss: 0.0040260046\n",
      "Iter 119 / 128 Batch 7 / 11 Loss: 0.0038161413\n",
      "Iter 119 / 128 Batch 8 / 11 Loss: 0.002982209\n",
      "Iter 119 / 128 Batch 9 / 11 Loss: 0.013213188\n",
      "Iter 119 / 128 Batch 10 / 11 Loss: 0.013435415\n",
      "Iter 120 / 128 Batch 0 / 11 Loss: 0.0029316451\n",
      "Iter 120 / 128 Batch 1 / 11 Loss: 0.008665474\n",
      "Iter 120 / 128 Batch 2 / 11 Loss: 0.031959217\n",
      "Iter 120 / 128 Batch 3 / 11 Loss: 0.001130268\n",
      "Iter 120 / 128 Batch 4 / 11 Loss: 0.013936248\n",
      "Iter 120 / 128 Batch 5 / 11 Loss: 0.01802665\n",
      "Iter 120 / 128 Batch 6 / 11 Loss: 0.008083979\n",
      "Iter 120 / 128 Batch 7 / 11 Loss: 0.0033875844\n",
      "Iter 120 / 128 Batch 8 / 11 Loss: 0.06511446\n",
      "Iter 120 / 128 Batch 9 / 11 Loss: 0.039604664\n",
      "Iter 120 / 128 Batch 10 / 11 Loss: 0.0050748168\n",
      "Iter 121 / 128 Batch 0 / 11 Loss: 0.0017379969\n",
      "Iter 121 / 128 Batch 1 / 11 Loss: 0.0015987237\n",
      "Iter 121 / 128 Batch 2 / 11 Loss: 0.032028545\n",
      "Iter 121 / 128 Batch 3 / 11 Loss: 0.014392915\n",
      "Iter 121 / 128 Batch 4 / 11 Loss: 0.031626634\n",
      "Iter 121 / 128 Batch 5 / 11 Loss: 0.008056046\n",
      "Iter 121 / 128 Batch 6 / 11 Loss: 0.011404221\n",
      "Iter 121 / 128 Batch 7 / 11 Loss: 0.0028127183\n",
      "Iter 121 / 128 Batch 8 / 11 Loss: 0.00473286\n",
      "Iter 121 / 128 Batch 9 / 11 Loss: 0.019283459\n",
      "Iter 121 / 128 Batch 10 / 11 Loss: 0.0027110681\n",
      "Iter 122 / 128 Batch 0 / 11 Loss: 0.009427983\n",
      "Iter 122 / 128 Batch 1 / 11 Loss: 0.0022091982\n",
      "Iter 122 / 128 Batch 2 / 11 Loss: 0.015303941\n",
      "Iter 122 / 128 Batch 3 / 11 Loss: 0.0036838364\n",
      "Iter 122 / 128 Batch 4 / 11 Loss: 0.005424865\n",
      "Iter 122 / 128 Batch 5 / 11 Loss: 0.035178233\n",
      "Iter 122 / 128 Batch 6 / 11 Loss: 0.0070954524\n",
      "Iter 122 / 128 Batch 7 / 11 Loss: 0.00080067595\n",
      "Iter 122 / 128 Batch 8 / 11 Loss: 0.00634654\n",
      "Iter 122 / 128 Batch 9 / 11 Loss: 0.038619656\n",
      "Iter 122 / 128 Batch 10 / 11 Loss: 0.049477827\n",
      "Iter 123 / 128 Batch 0 / 11 Loss: 0.0016827508\n",
      "Iter 123 / 128 Batch 1 / 11 Loss: 0.0034400485\n",
      "Iter 123 / 128 Batch 2 / 11 Loss: 0.04404418\n",
      "Iter 123 / 128 Batch 3 / 11 Loss: 0.002063546\n",
      "Iter 123 / 128 Batch 4 / 11 Loss: 0.008378985\n",
      "Iter 123 / 128 Batch 5 / 11 Loss: 0.06287889\n",
      "Iter 123 / 128 Batch 6 / 11 Loss: 0.017461402\n",
      "Iter 123 / 128 Batch 7 / 11 Loss: 0.0012940468\n",
      "Iter 123 / 128 Batch 8 / 11 Loss: 0.010569137\n",
      "Iter 123 / 128 Batch 9 / 11 Loss: 0.023667596\n",
      "Iter 123 / 128 Batch 10 / 11 Loss: 0.023909764\n",
      "Iter 124 / 128 Batch 0 / 11 Loss: 0.002079795\n",
      "Iter 124 / 128 Batch 1 / 11 Loss: 0.0020138805\n",
      "Iter 124 / 128 Batch 2 / 11 Loss: 0.12266191\n",
      "Iter 124 / 128 Batch 3 / 11 Loss: 0.0051247217\n",
      "Iter 124 / 128 Batch 4 / 11 Loss: 0.011833701\n",
      "Iter 124 / 128 Batch 5 / 11 Loss: 0.013004356\n",
      "Iter 124 / 128 Batch 6 / 11 Loss: 0.014501839\n",
      "Iter 124 / 128 Batch 7 / 11 Loss: 0.0012266285\n",
      "Iter 124 / 128 Batch 8 / 11 Loss: 0.0047215614\n",
      "Iter 124 / 128 Batch 9 / 11 Loss: 0.007977078\n",
      "Iter 124 / 128 Batch 10 / 11 Loss: 0.098067336\n",
      "Iter 125 / 128 Batch 0 / 11 Loss: 0.0048334114\n",
      "Iter 125 / 128 Batch 1 / 11 Loss: 0.029626912\n",
      "Iter 125 / 128 Batch 2 / 11 Loss: 0.037482344\n",
      "Iter 125 / 128 Batch 3 / 11 Loss: 0.0018318511\n",
      "Iter 125 / 128 Batch 4 / 11 Loss: 0.0077369837\n",
      "Iter 125 / 128 Batch 5 / 11 Loss: 0.0051943213\n",
      "Iter 125 / 128 Batch 6 / 11 Loss: 0.009036242\n",
      "Iter 125 / 128 Batch 7 / 11 Loss: 0.0037466527\n",
      "Iter 125 / 128 Batch 8 / 11 Loss: 0.012397115\n",
      "Iter 125 / 128 Batch 9 / 11 Loss: 0.018854318\n",
      "Iter 125 / 128 Batch 10 / 11 Loss: 0.011309043\n",
      "Iter 126 / 128 Batch 0 / 11 Loss: 0.005465159\n",
      "Iter 126 / 128 Batch 1 / 11 Loss: 0.0025174755\n",
      "Iter 126 / 128 Batch 2 / 11 Loss: 0.05687248\n",
      "Iter 126 / 128 Batch 3 / 11 Loss: 0.000738314\n",
      "Iter 126 / 128 Batch 4 / 11 Loss: 0.01932218\n",
      "Iter 126 / 128 Batch 5 / 11 Loss: 0.01944587\n",
      "Iter 126 / 128 Batch 6 / 11 Loss: 0.0112268645\n",
      "Iter 126 / 128 Batch 7 / 11 Loss: 0.00221693\n",
      "Iter 126 / 128 Batch 8 / 11 Loss: 0.0428811\n",
      "Iter 126 / 128 Batch 9 / 11 Loss: 0.031156698\n",
      "Iter 126 / 128 Batch 10 / 11 Loss: 0.011217221\n",
      "Iter 127 / 128 Batch 0 / 11 Loss: 0.0021280057\n",
      "Iter 127 / 128 Batch 1 / 11 Loss: 0.002585235\n",
      "Iter 127 / 128 Batch 2 / 11 Loss: 0.016813595\n",
      "Iter 127 / 128 Batch 3 / 11 Loss: 0.00086064206\n",
      "Iter 127 / 128 Batch 4 / 11 Loss: 0.013654385\n",
      "Iter 127 / 128 Batch 5 / 11 Loss: 0.009788636\n",
      "Iter 127 / 128 Batch 6 / 11 Loss: 0.008673697\n",
      "Iter 127 / 128 Batch 7 / 11 Loss: 0.0009274685\n",
      "Iter 127 / 128 Batch 8 / 11 Loss: 0.0035823763\n",
      "Iter 127 / 128 Batch 9 / 11 Loss: 0.042452633\n",
      "Iter 127 / 128 Batch 10 / 11 Loss: 0.023436967\n"
     ]
    }
   ],
   "source": [
    "resize_shape=[64,64]\n",
    "func_params={\n",
    "    'image_func':image_file_to_np,\n",
    "    'zip_file':zf,\n",
    "    'image_func_kargs':{'resize':resize_shape,'batch_shape':True}\n",
    "}\n",
    "\n",
    "#Set up parameters\n",
    "input_df = train_df\n",
    "batch_x_func = zip_image_file_to_np\n",
    "batch_x_func_params = func_params\n",
    "df_x_col=file_col\n",
    "df_y_col=hot_key_col\n",
    "\n",
    "input_dims = [resize_shape[0],resize_shape[1],3]\n",
    "output_classes = 3 #Terran, Protoss, Zerg\n",
    "iterations = 128\n",
    "learning_rate=1e-3\n",
    "batch_size=8\n",
    "\n",
    "#Model\n",
    "model = Sequential()\n",
    "\n",
    "model.add(Conv2D(32, (3, 3), activation='relu', input_shape=input_dims))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Conv2D(32, (3, 3), activation='relu'))\n",
    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "model.add(Dropout(0.25))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Conv2D(64, (3, 3), activation='relu'))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Conv2D(64, (3, 3), activation='relu'))\n",
    "model.add(MaxPooling2D(pool_size=(2, 2)))\n",
    "model.add(Dropout(0.25))\n",
    "model.add(Flatten())\n",
    "model.add(BatchNormalization())\n",
    "model.add(Dense(256, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Dense(output_classes, activation='softmax'))\n",
    "\n",
    "model.compile(loss=keras.losses.categorical_crossentropy,\n",
    "              optimizer=keras.optimizers.SGD(lr=learning_rate, momentum=0.9, nesterov=True))\n",
    "\n",
    "# Iterations\n",
    "\n",
    "rows = input_df.shape[0]\n",
    "batches = int(np.ceil(rows/batch_size))\n",
    "\n",
    "for iterx in range(iterations):\n",
    "    for batch in range(batches):\n",
    "        offset = batch*batch_size\n",
    "        xb,yb = batch_proc_from_df(input_df,batch_x_func,batch_x_func_params,\n",
    "                           x_col=df_x_col,y_col=df_y_col,batch_size=batch_size,offset=offset)\n",
    "        loss = model.train_on_batch(xb, yb)\n",
    "        print(\"Iter\",iterx,\"/\",iterations,\"Batch\",batch,\"/\",batches,\"Loss:\",loss)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test our model in the testing set  \n",
    "Now, we define our batch testing using the testing set. Since the **predict** or **evaluate ** functions of the model keras doesn't have the metrics parameter, we calculate the accuracy manually."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def test_model(idf,keras_model):\n",
    "    test_rows = idf.shape[0]\n",
    "    tbatches = int(np.ceil(test_rows/batch_size))\n",
    "    general_accuracy = []\n",
    "    for batch in range(tbatches):\n",
    "        offset = batch*batch_size\n",
    "        xb,yb = batch_proc_from_df(idf,batch_x_func,batch_x_func_params,\n",
    "                           x_col=df_x_col,y_col=df_y_col,batch_size=batch_size,offset=offset)\n",
    "    \n",
    "        classes = keras_model.predict(xb)\n",
    "        loss = keras_model.evaluate(xb, yb,verbose=0)\n",
    "    \n",
    "        class_pred = np.argmax(classes,1)\n",
    "        \n",
    "        print(\"Batch\",batch,\"/\",tbatches,\"Loss:\",loss)\n",
    "        ground_truth = np.argmax(yb,1)\n",
    "\n",
    "        accuracy = sum(np.equal(class_pred,ground_truth))/batch_size\n",
    "        general_accuracy.append(accuracy)    \n",
    "\n",
    "    print(\"Total accuracy\",np.mean(general_accuracy))\n",
    "\n",
    "test_model(test_df,model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Save the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model_name='sc2_cnn.h5'\n",
    "\n",
    "model.save(model_name)  # creates a HDF5 file 'my_model.h5'\n",
    "#del model  # deletes the existing model\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Batch 0 / 3 Loss: 0.2925415635108948\n",
      "Batch 1 / 3 Loss: 0.9142528772354126\n",
      "Batch 2 / 3 Loss: 0.3205617368221283\n",
      "Total accuracy 0.7083333333333334\n"
     ]
    }
   ],
   "source": [
    "modelx = load_model(model_name)\n",
    "test_model(test_df,keras_model=modelx)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter 0 / 64 Batch 0 / 11 Loss: 0.0032971296\n",
      "Iter 0 / 64 Batch 1 / 11 Loss: 0.0074045938\n",
      "Iter 0 / 64 Batch 2 / 11 Loss: 0.04290335\n",
      "Iter 0 / 64 Batch 3 / 11 Loss: 0.0030955374\n",
      "Iter 0 / 64 Batch 4 / 11 Loss: 0.010244563\n",
      "Iter 0 / 64 Batch 5 / 11 Loss: 0.0247713\n",
      "Iter 0 / 64 Batch 6 / 11 Loss: 0.008359117\n",
      "Iter 0 / 64 Batch 7 / 11 Loss: 0.002094245\n",
      "Iter 0 / 64 Batch 8 / 11 Loss: 0.000978267\n",
      "Iter 0 / 64 Batch 9 / 11 Loss: 0.010149945\n",
      "Iter 0 / 64 Batch 10 / 11 Loss: 0.0041710446\n",
      "Iter 1 / 64 Batch 0 / 11 Loss: 0.0013087601\n",
      "Iter 1 / 64 Batch 1 / 11 Loss: 0.0031152186\n",
      "Iter 1 / 64 Batch 2 / 11 Loss: 0.015789017\n",
      "Iter 1 / 64 Batch 3 / 11 Loss: 0.005446627\n",
      "Iter 1 / 64 Batch 4 / 11 Loss: 0.0038260755\n",
      "Iter 1 / 64 Batch 5 / 11 Loss: 0.005698045\n",
      "Iter 1 / 64 Batch 6 / 11 Loss: 0.00369098\n",
      "Iter 1 / 64 Batch 7 / 11 Loss: 0.002388537\n",
      "Iter 1 / 64 Batch 8 / 11 Loss: 0.0045649563\n",
      "Iter 1 / 64 Batch 9 / 11 Loss: 0.003764973\n",
      "Iter 1 / 64 Batch 10 / 11 Loss: 0.010052489\n",
      "Iter 2 / 64 Batch 0 / 11 Loss: 0.00096925325\n",
      "Iter 2 / 64 Batch 1 / 11 Loss: 0.0019156449\n",
      "Iter 2 / 64 Batch 2 / 11 Loss: 0.05678006\n",
      "Iter 2 / 64 Batch 3 / 11 Loss: 0.0045129494\n",
      "Iter 2 / 64 Batch 4 / 11 Loss: 0.0035420803\n",
      "Iter 2 / 64 Batch 5 / 11 Loss: 0.0042100423\n",
      "Iter 2 / 64 Batch 6 / 11 Loss: 0.016513636\n",
      "Iter 2 / 64 Batch 7 / 11 Loss: 0.0010482115\n",
      "Iter 2 / 64 Batch 8 / 11 Loss: 0.001118751\n",
      "Iter 2 / 64 Batch 9 / 11 Loss: 0.0062341276\n",
      "Iter 2 / 64 Batch 10 / 11 Loss: 0.0018023185\n",
      "Iter 3 / 64 Batch 0 / 11 Loss: 0.0038335046\n",
      "Iter 3 / 64 Batch 1 / 11 Loss: 0.0043793814\n",
      "Iter 3 / 64 Batch 2 / 11 Loss: 0.10989762\n",
      "Iter 3 / 64 Batch 3 / 11 Loss: 0.005346791\n",
      "Iter 3 / 64 Batch 4 / 11 Loss: 0.015706616\n",
      "Iter 3 / 64 Batch 5 / 11 Loss: 0.007538869\n",
      "Iter 3 / 64 Batch 6 / 11 Loss: 0.003481419\n",
      "Iter 3 / 64 Batch 7 / 11 Loss: 0.004642613\n",
      "Iter 3 / 64 Batch 8 / 11 Loss: 0.00593182\n",
      "Iter 3 / 64 Batch 9 / 11 Loss: 0.015007629\n",
      "Iter 3 / 64 Batch 10 / 11 Loss: 0.00096742116\n",
      "Iter 4 / 64 Batch 0 / 11 Loss: 0.0022525988\n",
      "Iter 4 / 64 Batch 1 / 11 Loss: 0.008759218\n",
      "Iter 4 / 64 Batch 2 / 11 Loss: 0.05038898\n",
      "Iter 4 / 64 Batch 3 / 11 Loss: 0.0006760219\n",
      "Iter 4 / 64 Batch 4 / 11 Loss: 0.004126097\n",
      "Iter 4 / 64 Batch 5 / 11 Loss: 0.0038827306\n",
      "Iter 4 / 64 Batch 6 / 11 Loss: 0.051242676\n",
      "Iter 4 / 64 Batch 7 / 11 Loss: 0.0011744245\n",
      "Iter 4 / 64 Batch 8 / 11 Loss: 0.0042912327\n",
      "Iter 4 / 64 Batch 9 / 11 Loss: 0.0027140728\n",
      "Iter 4 / 64 Batch 10 / 11 Loss: 0.010905158\n",
      "Iter 5 / 64 Batch 0 / 11 Loss: 0.0036805163\n",
      "Iter 5 / 64 Batch 1 / 11 Loss: 0.04360299\n",
      "Iter 5 / 64 Batch 2 / 11 Loss: 0.056452904\n",
      "Iter 5 / 64 Batch 3 / 11 Loss: 0.0030252023\n",
      "Iter 5 / 64 Batch 4 / 11 Loss: 0.008454744\n",
      "Iter 5 / 64 Batch 5 / 11 Loss: 0.0037508588\n",
      "Iter 5 / 64 Batch 6 / 11 Loss: 0.025771176\n",
      "Iter 5 / 64 Batch 7 / 11 Loss: 0.002084381\n",
      "Iter 5 / 64 Batch 8 / 11 Loss: 0.012327354\n",
      "Iter 5 / 64 Batch 9 / 11 Loss: 0.01563853\n",
      "Iter 5 / 64 Batch 10 / 11 Loss: 0.008239465\n",
      "Iter 6 / 64 Batch 0 / 11 Loss: 0.0020765455\n",
      "Iter 6 / 64 Batch 1 / 11 Loss: 0.03690393\n",
      "Iter 6 / 64 Batch 2 / 11 Loss: 0.028434591\n",
      "Iter 6 / 64 Batch 3 / 11 Loss: 0.0024485318\n",
      "Iter 6 / 64 Batch 4 / 11 Loss: 0.0029255971\n",
      "Iter 6 / 64 Batch 5 / 11 Loss: 0.013122769\n",
      "Iter 6 / 64 Batch 6 / 11 Loss: 0.002240804\n",
      "Iter 6 / 64 Batch 7 / 11 Loss: 0.005527063\n",
      "Iter 6 / 64 Batch 8 / 11 Loss: 0.0053192796\n",
      "Iter 6 / 64 Batch 9 / 11 Loss: 0.005230642\n",
      "Iter 6 / 64 Batch 10 / 11 Loss: 0.004398332\n",
      "Iter 7 / 64 Batch 0 / 11 Loss: 0.0019054818\n",
      "Iter 7 / 64 Batch 1 / 11 Loss: 0.05510514\n",
      "Iter 7 / 64 Batch 2 / 11 Loss: 0.036933698\n",
      "Iter 7 / 64 Batch 3 / 11 Loss: 0.0012452538\n",
      "Iter 7 / 64 Batch 4 / 11 Loss: 0.0064202165\n",
      "Iter 7 / 64 Batch 5 / 11 Loss: 0.012416193\n",
      "Iter 7 / 64 Batch 6 / 11 Loss: 0.009620659\n",
      "Iter 7 / 64 Batch 7 / 11 Loss: 0.02088565\n",
      "Iter 7 / 64 Batch 8 / 11 Loss: 0.010866765\n",
      "Iter 7 / 64 Batch 9 / 11 Loss: 0.004966951\n",
      "Iter 7 / 64 Batch 10 / 11 Loss: 0.0012050015\n",
      "Iter 8 / 64 Batch 0 / 11 Loss: 0.005983578\n",
      "Iter 8 / 64 Batch 1 / 11 Loss: 0.0023777813\n",
      "Iter 8 / 64 Batch 2 / 11 Loss: 0.010030933\n",
      "Iter 8 / 64 Batch 3 / 11 Loss: 0.007865808\n",
      "Iter 8 / 64 Batch 4 / 11 Loss: 0.0069105607\n",
      "Iter 8 / 64 Batch 5 / 11 Loss: 0.018507391\n",
      "Iter 8 / 64 Batch 6 / 11 Loss: 0.010342255\n",
      "Iter 8 / 64 Batch 7 / 11 Loss: 0.0020034716\n",
      "Iter 8 / 64 Batch 8 / 11 Loss: 0.006274003\n",
      "Iter 8 / 64 Batch 9 / 11 Loss: 0.004817902\n",
      "Iter 8 / 64 Batch 10 / 11 Loss: 0.008130881\n",
      "Iter 9 / 64 Batch 0 / 11 Loss: 0.0013456956\n",
      "Iter 9 / 64 Batch 1 / 11 Loss: 0.00577418\n",
      "Iter 9 / 64 Batch 2 / 11 Loss: 0.13242209\n",
      "Iter 9 / 64 Batch 3 / 11 Loss: 0.018797956\n",
      "Iter 9 / 64 Batch 4 / 11 Loss: 0.001966389\n",
      "Iter 9 / 64 Batch 5 / 11 Loss: 0.003832819\n",
      "Iter 9 / 64 Batch 6 / 11 Loss: 0.003172104\n",
      "Iter 9 / 64 Batch 7 / 11 Loss: 0.0011919561\n",
      "Iter 9 / 64 Batch 8 / 11 Loss: 0.0012751569\n",
      "Iter 9 / 64 Batch 9 / 11 Loss: 0.015017876\n",
      "Iter 9 / 64 Batch 10 / 11 Loss: 0.004189435\n",
      "Iter 10 / 64 Batch 0 / 11 Loss: 0.0014470039\n",
      "Iter 10 / 64 Batch 1 / 11 Loss: 0.0016587486\n",
      "Iter 10 / 64 Batch 2 / 11 Loss: 0.036513627\n",
      "Iter 10 / 64 Batch 3 / 11 Loss: 0.0015720008\n",
      "Iter 10 / 64 Batch 4 / 11 Loss: 0.005905707\n",
      "Iter 10 / 64 Batch 5 / 11 Loss: 0.010903917\n",
      "Iter 10 / 64 Batch 6 / 11 Loss: 0.0059534563\n",
      "Iter 10 / 64 Batch 7 / 11 Loss: 0.012344059\n",
      "Iter 10 / 64 Batch 8 / 11 Loss: 0.0013169902\n",
      "Iter 10 / 64 Batch 9 / 11 Loss: 0.0233213\n",
      "Iter 10 / 64 Batch 10 / 11 Loss: 0.0027043922\n",
      "Iter 11 / 64 Batch 0 / 11 Loss: 0.0022272302\n",
      "Iter 11 / 64 Batch 1 / 11 Loss: 0.00087159185\n",
      "Iter 11 / 64 Batch 2 / 11 Loss: 0.026653023\n",
      "Iter 11 / 64 Batch 3 / 11 Loss: 0.0013679309\n",
      "Iter 11 / 64 Batch 4 / 11 Loss: 0.0023656785\n",
      "Iter 11 / 64 Batch 5 / 11 Loss: 0.004612607\n",
      "Iter 11 / 64 Batch 6 / 11 Loss: 0.013708389\n",
      "Iter 11 / 64 Batch 7 / 11 Loss: 0.0019919178\n",
      "Iter 11 / 64 Batch 8 / 11 Loss: 0.007962058\n",
      "Iter 11 / 64 Batch 9 / 11 Loss: 0.00925723\n",
      "Iter 11 / 64 Batch 10 / 11 Loss: 0.010147628\n",
      "Iter 12 / 64 Batch 0 / 11 Loss: 0.0026034154\n",
      "Iter 12 / 64 Batch 1 / 11 Loss: 0.002791733\n",
      "Iter 12 / 64 Batch 2 / 11 Loss: 0.026527671\n",
      "Iter 12 / 64 Batch 3 / 11 Loss: 0.00648262\n",
      "Iter 12 / 64 Batch 4 / 11 Loss: 0.0026559692\n",
      "Iter 12 / 64 Batch 5 / 11 Loss: 0.023780392\n",
      "Iter 12 / 64 Batch 6 / 11 Loss: 0.017432913\n",
      "Iter 12 / 64 Batch 7 / 11 Loss: 0.002647788\n",
      "Iter 12 / 64 Batch 8 / 11 Loss: 0.005010308\n",
      "Iter 12 / 64 Batch 9 / 11 Loss: 0.0057933666\n",
      "Iter 12 / 64 Batch 10 / 11 Loss: 0.00094828964\n",
      "Iter 13 / 64 Batch 0 / 11 Loss: 0.0014897321\n",
      "Iter 13 / 64 Batch 1 / 11 Loss: 0.0017483758\n",
      "Iter 13 / 64 Batch 2 / 11 Loss: 0.015183276\n",
      "Iter 13 / 64 Batch 3 / 11 Loss: 0.0016471974\n",
      "Iter 13 / 64 Batch 4 / 11 Loss: 0.0050882716\n",
      "Iter 13 / 64 Batch 5 / 11 Loss: 0.0069355885\n",
      "Iter 13 / 64 Batch 6 / 11 Loss: 0.0025259345\n",
      "Iter 13 / 64 Batch 7 / 11 Loss: 0.002782974\n",
      "Iter 13 / 64 Batch 8 / 11 Loss: 0.0043829833\n",
      "Iter 13 / 64 Batch 9 / 11 Loss: 0.026361395\n",
      "Iter 13 / 64 Batch 10 / 11 Loss: 0.0043249107\n",
      "Iter 14 / 64 Batch 0 / 11 Loss: 0.0019986155\n",
      "Iter 14 / 64 Batch 1 / 11 Loss: 0.0014924943\n",
      "Iter 14 / 64 Batch 2 / 11 Loss: 0.011927114\n",
      "Iter 14 / 64 Batch 3 / 11 Loss: 0.0027341621\n",
      "Iter 14 / 64 Batch 4 / 11 Loss: 0.0032164503\n",
      "Iter 14 / 64 Batch 5 / 11 Loss: 0.0063809203\n",
      "Iter 14 / 64 Batch 6 / 11 Loss: 0.0043752086\n",
      "Iter 14 / 64 Batch 7 / 11 Loss: 0.0040910724\n",
      "Iter 14 / 64 Batch 8 / 11 Loss: 0.002352827\n",
      "Iter 14 / 64 Batch 9 / 11 Loss: 0.006415689\n",
      "Iter 14 / 64 Batch 10 / 11 Loss: 0.0020076758\n",
      "Iter 15 / 64 Batch 0 / 11 Loss: 0.0017703611\n",
      "Iter 15 / 64 Batch 1 / 11 Loss: 0.0013711022\n",
      "Iter 15 / 64 Batch 2 / 11 Loss: 0.01924024\n",
      "Iter 15 / 64 Batch 3 / 11 Loss: 0.003288458\n",
      "Iter 15 / 64 Batch 4 / 11 Loss: 0.0032038493\n",
      "Iter 15 / 64 Batch 5 / 11 Loss: 0.035190996\n",
      "Iter 15 / 64 Batch 6 / 11 Loss: 0.004415673\n",
      "Iter 15 / 64 Batch 7 / 11 Loss: 0.00072993914\n",
      "Iter 15 / 64 Batch 8 / 11 Loss: 0.0026528384\n",
      "Iter 15 / 64 Batch 9 / 11 Loss: 0.008773364\n",
      "Iter 15 / 64 Batch 10 / 11 Loss: 0.00047093723\n",
      "Iter 16 / 64 Batch 0 / 11 Loss: 0.0013987437\n",
      "Iter 16 / 64 Batch 1 / 11 Loss: 0.0031650322\n",
      "Iter 16 / 64 Batch 2 / 11 Loss: 0.018698368\n",
      "Iter 16 / 64 Batch 3 / 11 Loss: 0.0015117878\n",
      "Iter 16 / 64 Batch 4 / 11 Loss: 0.007733051\n",
      "Iter 16 / 64 Batch 5 / 11 Loss: 0.010468669\n",
      "Iter 16 / 64 Batch 6 / 11 Loss: 0.023947824\n",
      "Iter 16 / 64 Batch 7 / 11 Loss: 0.0014928103\n",
      "Iter 16 / 64 Batch 8 / 11 Loss: 0.019288866\n",
      "Iter 16 / 64 Batch 9 / 11 Loss: 0.0060186293\n",
      "Iter 16 / 64 Batch 10 / 11 Loss: 0.0019751596\n",
      "Iter 17 / 64 Batch 0 / 11 Loss: 0.003432387\n",
      "Iter 17 / 64 Batch 1 / 11 Loss: 0.0012895791\n",
      "Iter 17 / 64 Batch 2 / 11 Loss: 0.057597794\n",
      "Iter 17 / 64 Batch 3 / 11 Loss: 0.0018907634\n",
      "Iter 17 / 64 Batch 4 / 11 Loss: 0.004409025\n",
      "Iter 17 / 64 Batch 5 / 11 Loss: 0.029087218\n",
      "Iter 17 / 64 Batch 6 / 11 Loss: 0.012264663\n",
      "Iter 17 / 64 Batch 7 / 11 Loss: 0.00067873707\n",
      "Iter 17 / 64 Batch 8 / 11 Loss: 0.0009121428\n",
      "Iter 17 / 64 Batch 9 / 11 Loss: 0.0040120576\n",
      "Iter 17 / 64 Batch 10 / 11 Loss: 0.0019278055\n",
      "Iter 18 / 64 Batch 0 / 11 Loss: 0.002537278\n",
      "Iter 18 / 64 Batch 1 / 11 Loss: 0.0012563233\n",
      "Iter 18 / 64 Batch 2 / 11 Loss: 0.046255562\n",
      "Iter 18 / 64 Batch 3 / 11 Loss: 0.014492637\n",
      "Iter 18 / 64 Batch 4 / 11 Loss: 0.004473831\n",
      "Iter 18 / 64 Batch 5 / 11 Loss: 0.027263772\n",
      "Iter 18 / 64 Batch 6 / 11 Loss: 0.002117186\n",
      "Iter 18 / 64 Batch 7 / 11 Loss: 0.0003646593\n",
      "Iter 18 / 64 Batch 8 / 11 Loss: 0.022090452\n",
      "Iter 18 / 64 Batch 9 / 11 Loss: 0.0034629756\n",
      "Iter 18 / 64 Batch 10 / 11 Loss: 0.004449692\n",
      "Iter 19 / 64 Batch 0 / 11 Loss: 0.0005824557\n",
      "Iter 19 / 64 Batch 1 / 11 Loss: 0.0029279885\n",
      "Iter 19 / 64 Batch 2 / 11 Loss: 0.028720085\n",
      "Iter 19 / 64 Batch 3 / 11 Loss: 0.032548703\n",
      "Iter 19 / 64 Batch 4 / 11 Loss: 0.0107079325\n",
      "Iter 19 / 64 Batch 5 / 11 Loss: 0.0064145965\n",
      "Iter 19 / 64 Batch 6 / 11 Loss: 0.010927377\n",
      "Iter 19 / 64 Batch 7 / 11 Loss: 0.0034246002\n",
      "Iter 19 / 64 Batch 8 / 11 Loss: 0.009458215\n",
      "Iter 19 / 64 Batch 9 / 11 Loss: 0.009138123\n",
      "Iter 19 / 64 Batch 10 / 11 Loss: 0.0020148428\n",
      "Iter 20 / 64 Batch 0 / 11 Loss: 0.0016074756\n",
      "Iter 20 / 64 Batch 1 / 11 Loss: 0.0013070246\n",
      "Iter 20 / 64 Batch 2 / 11 Loss: 0.046602096\n",
      "Iter 20 / 64 Batch 3 / 11 Loss: 0.003485854\n",
      "Iter 20 / 64 Batch 4 / 11 Loss: 0.008747709\n",
      "Iter 20 / 64 Batch 5 / 11 Loss: 0.0039825495\n",
      "Iter 20 / 64 Batch 6 / 11 Loss: 0.0025693423\n",
      "Iter 20 / 64 Batch 7 / 11 Loss: 0.00025106868\n",
      "Iter 20 / 64 Batch 8 / 11 Loss: 0.0014944859\n",
      "Iter 20 / 64 Batch 9 / 11 Loss: 0.049653735\n",
      "Iter 20 / 64 Batch 10 / 11 Loss: 0.00190895\n",
      "Iter 21 / 64 Batch 0 / 11 Loss: 0.0005838923\n",
      "Iter 21 / 64 Batch 1 / 11 Loss: 0.0011229049\n",
      "Iter 21 / 64 Batch 2 / 11 Loss: 0.015547862\n",
      "Iter 21 / 64 Batch 3 / 11 Loss: 0.014637202\n",
      "Iter 21 / 64 Batch 4 / 11 Loss: 0.0068641426\n",
      "Iter 21 / 64 Batch 5 / 11 Loss: 0.017935473\n",
      "Iter 21 / 64 Batch 6 / 11 Loss: 0.004805686\n",
      "Iter 21 / 64 Batch 7 / 11 Loss: 0.0038392553\n",
      "Iter 21 / 64 Batch 8 / 11 Loss: 0.0019477566\n",
      "Iter 21 / 64 Batch 9 / 11 Loss: 0.011196156\n",
      "Iter 21 / 64 Batch 10 / 11 Loss: 0.0020688963\n",
      "Iter 22 / 64 Batch 0 / 11 Loss: 0.004641004\n",
      "Iter 22 / 64 Batch 1 / 11 Loss: 0.0035834056\n",
      "Iter 22 / 64 Batch 2 / 11 Loss: 0.013492168\n",
      "Iter 22 / 64 Batch 3 / 11 Loss: 0.002248073\n",
      "Iter 22 / 64 Batch 4 / 11 Loss: 0.01090695\n",
      "Iter 22 / 64 Batch 5 / 11 Loss: 0.06503112\n",
      "Iter 22 / 64 Batch 6 / 11 Loss: 0.0047250753\n",
      "Iter 22 / 64 Batch 7 / 11 Loss: 0.0014799748\n",
      "Iter 22 / 64 Batch 8 / 11 Loss: 0.0019081932\n",
      "Iter 22 / 64 Batch 9 / 11 Loss: 0.0035183937\n",
      "Iter 22 / 64 Batch 10 / 11 Loss: 0.008764341\n",
      "Iter 23 / 64 Batch 0 / 11 Loss: 0.0019309388\n",
      "Iter 23 / 64 Batch 1 / 11 Loss: 0.0027839541\n",
      "Iter 23 / 64 Batch 2 / 11 Loss: 0.03840247\n",
      "Iter 23 / 64 Batch 3 / 11 Loss: 0.00022544432\n",
      "Iter 23 / 64 Batch 4 / 11 Loss: 0.0031233032\n",
      "Iter 23 / 64 Batch 5 / 11 Loss: 0.01866537\n",
      "Iter 23 / 64 Batch 6 / 11 Loss: 0.026339047\n",
      "Iter 23 / 64 Batch 7 / 11 Loss: 0.002307919\n",
      "Iter 23 / 64 Batch 8 / 11 Loss: 0.008090072\n",
      "Iter 23 / 64 Batch 9 / 11 Loss: 0.01241324\n",
      "Iter 23 / 64 Batch 10 / 11 Loss: 0.0014778159\n",
      "Iter 24 / 64 Batch 0 / 11 Loss: 0.0017676086\n",
      "Iter 24 / 64 Batch 1 / 11 Loss: 0.0023806568\n",
      "Iter 24 / 64 Batch 2 / 11 Loss: 0.018844662\n",
      "Iter 24 / 64 Batch 3 / 11 Loss: 0.0018803752\n",
      "Iter 24 / 64 Batch 4 / 11 Loss: 0.0025171132\n",
      "Iter 24 / 64 Batch 5 / 11 Loss: 0.004419707\n",
      "Iter 24 / 64 Batch 6 / 11 Loss: 0.0032882954\n",
      "Iter 24 / 64 Batch 7 / 11 Loss: 0.00038316002\n",
      "Iter 24 / 64 Batch 8 / 11 Loss: 0.027197205\n",
      "Iter 24 / 64 Batch 9 / 11 Loss: 0.01133574\n",
      "Iter 24 / 64 Batch 10 / 11 Loss: 0.0013336944\n",
      "Iter 25 / 64 Batch 0 / 11 Loss: 0.002741668\n",
      "Iter 25 / 64 Batch 1 / 11 Loss: 0.033152595\n",
      "Iter 25 / 64 Batch 2 / 11 Loss: 0.029255657\n",
      "Iter 25 / 64 Batch 3 / 11 Loss: 0.0005402853\n",
      "Iter 25 / 64 Batch 4 / 11 Loss: 0.0019529072\n",
      "Iter 25 / 64 Batch 5 / 11 Loss: 0.013826609\n",
      "Iter 25 / 64 Batch 6 / 11 Loss: 0.0024174238\n",
      "Iter 25 / 64 Batch 7 / 11 Loss: 0.0019611777\n",
      "Iter 25 / 64 Batch 8 / 11 Loss: 0.034313895\n",
      "Iter 25 / 64 Batch 9 / 11 Loss: 0.0073798825\n",
      "Iter 25 / 64 Batch 10 / 11 Loss: 0.0057979096\n",
      "Iter 26 / 64 Batch 0 / 11 Loss: 0.004043749\n",
      "Iter 26 / 64 Batch 1 / 11 Loss: 0.0016248397\n",
      "Iter 26 / 64 Batch 2 / 11 Loss: 0.14065751\n",
      "Iter 26 / 64 Batch 3 / 11 Loss: 0.0014483903\n",
      "Iter 26 / 64 Batch 4 / 11 Loss: 0.03466462\n",
      "Iter 26 / 64 Batch 5 / 11 Loss: 0.004490672\n",
      "Iter 26 / 64 Batch 6 / 11 Loss: 0.0126030715\n",
      "Iter 26 / 64 Batch 7 / 11 Loss: 0.0018452223\n",
      "Iter 26 / 64 Batch 8 / 11 Loss: 0.0010676344\n",
      "Iter 26 / 64 Batch 9 / 11 Loss: 0.0051319627\n",
      "Iter 26 / 64 Batch 10 / 11 Loss: 0.0012968926\n",
      "Iter 27 / 64 Batch 0 / 11 Loss: 0.0006564879\n",
      "Iter 27 / 64 Batch 1 / 11 Loss: 0.0012623263\n",
      "Iter 27 / 64 Batch 2 / 11 Loss: 0.059151933\n",
      "Iter 27 / 64 Batch 3 / 11 Loss: 0.004526332\n",
      "Iter 27 / 64 Batch 4 / 11 Loss: 0.00658302\n",
      "Iter 27 / 64 Batch 5 / 11 Loss: 0.043087337\n",
      "Iter 27 / 64 Batch 6 / 11 Loss: 0.010659565\n",
      "Iter 27 / 64 Batch 7 / 11 Loss: 0.0002856179\n",
      "Iter 27 / 64 Batch 8 / 11 Loss: 0.013500993\n",
      "Iter 27 / 64 Batch 9 / 11 Loss: 0.0058449106\n",
      "Iter 27 / 64 Batch 10 / 11 Loss: 0.015205288\n",
      "Iter 28 / 64 Batch 0 / 11 Loss: 0.0033161493\n",
      "Iter 28 / 64 Batch 1 / 11 Loss: 0.010563951\n",
      "Iter 28 / 64 Batch 2 / 11 Loss: 0.08434234\n",
      "Iter 28 / 64 Batch 3 / 11 Loss: 0.0013815295\n",
      "Iter 28 / 64 Batch 4 / 11 Loss: 0.0011325204\n",
      "Iter 28 / 64 Batch 5 / 11 Loss: 0.004604471\n",
      "Iter 28 / 64 Batch 6 / 11 Loss: 0.035072222\n",
      "Iter 28 / 64 Batch 7 / 11 Loss: 0.0010849616\n",
      "Iter 28 / 64 Batch 8 / 11 Loss: 0.02162595\n",
      "Iter 28 / 64 Batch 9 / 11 Loss: 0.012285633\n",
      "Iter 28 / 64 Batch 10 / 11 Loss: 0.0033588253\n",
      "Iter 29 / 64 Batch 0 / 11 Loss: 0.0021630363\n",
      "Iter 29 / 64 Batch 1 / 11 Loss: 0.004215671\n",
      "Iter 29 / 64 Batch 2 / 11 Loss: 0.015743\n",
      "Iter 29 / 64 Batch 3 / 11 Loss: 0.00177332\n",
      "Iter 29 / 64 Batch 4 / 11 Loss: 0.0026452055\n",
      "Iter 29 / 64 Batch 5 / 11 Loss: 0.013646839\n",
      "Iter 29 / 64 Batch 6 / 11 Loss: 0.0036925822\n",
      "Iter 29 / 64 Batch 7 / 11 Loss: 0.0021507097\n",
      "Iter 29 / 64 Batch 8 / 11 Loss: 0.008594563\n",
      "Iter 29 / 64 Batch 9 / 11 Loss: 0.033804562\n",
      "Iter 29 / 64 Batch 10 / 11 Loss: 0.0020031258\n",
      "Iter 30 / 64 Batch 0 / 11 Loss: 0.0045361673\n",
      "Iter 30 / 64 Batch 1 / 11 Loss: 0.005279099\n",
      "Iter 30 / 64 Batch 2 / 11 Loss: 0.02610842\n",
      "Iter 30 / 64 Batch 3 / 11 Loss: 0.0062146336\n",
      "Iter 30 / 64 Batch 4 / 11 Loss: 0.0336755\n",
      "Iter 30 / 64 Batch 5 / 11 Loss: 0.047644295\n",
      "Iter 30 / 64 Batch 6 / 11 Loss: 0.0047988105\n",
      "Iter 30 / 64 Batch 7 / 11 Loss: 0.0009514958\n",
      "Iter 30 / 64 Batch 8 / 11 Loss: 0.0035769194\n",
      "Iter 30 / 64 Batch 9 / 11 Loss: 0.0063883155\n",
      "Iter 30 / 64 Batch 10 / 11 Loss: 0.0023518237\n",
      "Iter 31 / 64 Batch 0 / 11 Loss: 0.0020991233\n",
      "Iter 31 / 64 Batch 1 / 11 Loss: 0.004256789\n",
      "Iter 31 / 64 Batch 2 / 11 Loss: 0.019869436\n",
      "Iter 31 / 64 Batch 3 / 11 Loss: 0.027105918\n",
      "Iter 31 / 64 Batch 4 / 11 Loss: 0.0068759136\n",
      "Iter 31 / 64 Batch 5 / 11 Loss: 0.026844872\n",
      "Iter 31 / 64 Batch 6 / 11 Loss: 0.005438981\n",
      "Iter 31 / 64 Batch 7 / 11 Loss: 0.0011979961\n",
      "Iter 31 / 64 Batch 8 / 11 Loss: 0.0021336407\n",
      "Iter 31 / 64 Batch 9 / 11 Loss: 0.0038538757\n",
      "Iter 31 / 64 Batch 10 / 11 Loss: 0.016132047\n",
      "Iter 32 / 64 Batch 0 / 11 Loss: 0.005799587\n",
      "Iter 32 / 64 Batch 1 / 11 Loss: 0.037457976\n",
      "Iter 32 / 64 Batch 2 / 11 Loss: 0.025711741\n",
      "Iter 32 / 64 Batch 3 / 11 Loss: 0.0008972622\n",
      "Iter 32 / 64 Batch 4 / 11 Loss: 0.006286945\n",
      "Iter 32 / 64 Batch 5 / 11 Loss: 0.0035611042\n",
      "Iter 32 / 64 Batch 6 / 11 Loss: 0.0028924579\n",
      "Iter 32 / 64 Batch 7 / 11 Loss: 0.00054753653\n",
      "Iter 32 / 64 Batch 8 / 11 Loss: 0.0077440706\n",
      "Iter 32 / 64 Batch 9 / 11 Loss: 0.014788183\n",
      "Iter 32 / 64 Batch 10 / 11 Loss: 0.003284811\n",
      "Iter 33 / 64 Batch 0 / 11 Loss: 0.0016025399\n",
      "Iter 33 / 64 Batch 1 / 11 Loss: 0.006239647\n",
      "Iter 33 / 64 Batch 2 / 11 Loss: 0.03402818\n",
      "Iter 33 / 64 Batch 3 / 11 Loss: 0.0028300202\n",
      "Iter 33 / 64 Batch 4 / 11 Loss: 0.0007731266\n",
      "Iter 33 / 64 Batch 5 / 11 Loss: 0.002905719\n",
      "Iter 33 / 64 Batch 6 / 11 Loss: 0.0012198113\n",
      "Iter 33 / 64 Batch 7 / 11 Loss: 0.0017700958\n",
      "Iter 33 / 64 Batch 8 / 11 Loss: 0.0068201534\n",
      "Iter 33 / 64 Batch 9 / 11 Loss: 0.008826843\n",
      "Iter 33 / 64 Batch 10 / 11 Loss: 0.019078832\n",
      "Iter 34 / 64 Batch 0 / 11 Loss: 0.0013647956\n",
      "Iter 34 / 64 Batch 1 / 11 Loss: 0.0025464236\n",
      "Iter 34 / 64 Batch 2 / 11 Loss: 0.13078795\n",
      "Iter 34 / 64 Batch 3 / 11 Loss: 0.001221685\n",
      "Iter 34 / 64 Batch 4 / 11 Loss: 0.008761709\n",
      "Iter 34 / 64 Batch 5 / 11 Loss: 0.003406188\n",
      "Iter 34 / 64 Batch 6 / 11 Loss: 0.0037178244\n",
      "Iter 34 / 64 Batch 7 / 11 Loss: 0.00021713544\n",
      "Iter 34 / 64 Batch 8 / 11 Loss: 0.0009122085\n",
      "Iter 34 / 64 Batch 9 / 11 Loss: 0.008657265\n",
      "Iter 34 / 64 Batch 10 / 11 Loss: 0.0065506077\n",
      "Iter 35 / 64 Batch 0 / 11 Loss: 0.0008780706\n",
      "Iter 35 / 64 Batch 1 / 11 Loss: 0.024357617\n",
      "Iter 35 / 64 Batch 2 / 11 Loss: 0.014106126\n",
      "Iter 35 / 64 Batch 3 / 11 Loss: 0.00064774486\n",
      "Iter 35 / 64 Batch 4 / 11 Loss: 0.014103818\n",
      "Iter 35 / 64 Batch 5 / 11 Loss: 0.0088666305\n",
      "Iter 35 / 64 Batch 6 / 11 Loss: 0.0034702243\n",
      "Iter 35 / 64 Batch 7 / 11 Loss: 0.00150991\n",
      "Iter 35 / 64 Batch 8 / 11 Loss: 0.009729006\n",
      "Iter 35 / 64 Batch 9 / 11 Loss: 0.0034440141\n",
      "Iter 35 / 64 Batch 10 / 11 Loss: 0.0016906739\n",
      "Iter 36 / 64 Batch 0 / 11 Loss: 0.00025913925\n",
      "Iter 36 / 64 Batch 1 / 11 Loss: 0.009874815\n",
      "Iter 36 / 64 Batch 2 / 11 Loss: 0.027077314\n",
      "Iter 36 / 64 Batch 3 / 11 Loss: 0.0032881107\n",
      "Iter 36 / 64 Batch 4 / 11 Loss: 0.007116221\n",
      "Iter 36 / 64 Batch 5 / 11 Loss: 0.0048932447\n",
      "Iter 36 / 64 Batch 6 / 11 Loss: 0.006715283\n",
      "Iter 36 / 64 Batch 7 / 11 Loss: 0.007899553\n",
      "Iter 36 / 64 Batch 8 / 11 Loss: 0.0070914617\n",
      "Iter 36 / 64 Batch 9 / 11 Loss: 0.0037524854\n",
      "Iter 36 / 64 Batch 10 / 11 Loss: 0.0005286217\n",
      "Iter 37 / 64 Batch 0 / 11 Loss: 0.0013603116\n",
      "Iter 37 / 64 Batch 1 / 11 Loss: 0.0012289906\n",
      "Iter 37 / 64 Batch 2 / 11 Loss: 0.033955105\n",
      "Iter 37 / 64 Batch 3 / 11 Loss: 0.0042789164\n",
      "Iter 37 / 64 Batch 4 / 11 Loss: 0.0021661196\n",
      "Iter 37 / 64 Batch 5 / 11 Loss: 0.027531106\n",
      "Iter 37 / 64 Batch 6 / 11 Loss: 0.007444714\n",
      "Iter 37 / 64 Batch 7 / 11 Loss: 0.0015155\n",
      "Iter 37 / 64 Batch 8 / 11 Loss: 0.006609492\n",
      "Iter 37 / 64 Batch 9 / 11 Loss: 0.0032411893\n",
      "Iter 37 / 64 Batch 10 / 11 Loss: 0.007898883\n",
      "Iter 38 / 64 Batch 0 / 11 Loss: 0.007875197\n",
      "Iter 38 / 64 Batch 1 / 11 Loss: 0.0015970137\n",
      "Iter 38 / 64 Batch 2 / 11 Loss: 0.09611543\n",
      "Iter 38 / 64 Batch 3 / 11 Loss: 0.0022281087\n",
      "Iter 38 / 64 Batch 4 / 11 Loss: 0.014475401\n",
      "Iter 38 / 64 Batch 5 / 11 Loss: 0.01214402\n",
      "Iter 38 / 64 Batch 6 / 11 Loss: 0.0050299466\n",
      "Iter 38 / 64 Batch 7 / 11 Loss: 0.0003839009\n",
      "Iter 38 / 64 Batch 8 / 11 Loss: 0.0013218437\n",
      "Iter 38 / 64 Batch 9 / 11 Loss: 0.0045671333\n",
      "Iter 38 / 64 Batch 10 / 11 Loss: 0.0016629746\n",
      "Iter 39 / 64 Batch 0 / 11 Loss: 0.002062828\n",
      "Iter 39 / 64 Batch 1 / 11 Loss: 0.006302264\n",
      "Iter 39 / 64 Batch 2 / 11 Loss: 0.019670319\n",
      "Iter 39 / 64 Batch 3 / 11 Loss: 0.0018382305\n",
      "Iter 39 / 64 Batch 4 / 11 Loss: 0.0037950454\n",
      "Iter 39 / 64 Batch 5 / 11 Loss: 0.0034620054\n",
      "Iter 39 / 64 Batch 6 / 11 Loss: 0.0037618643\n",
      "Iter 39 / 64 Batch 7 / 11 Loss: 0.0018529143\n",
      "Iter 39 / 64 Batch 8 / 11 Loss: 0.0012157685\n",
      "Iter 39 / 64 Batch 9 / 11 Loss: 0.008742597\n",
      "Iter 39 / 64 Batch 10 / 11 Loss: 0.004657327\n",
      "Iter 40 / 64 Batch 0 / 11 Loss: 0.0013823485\n",
      "Iter 40 / 64 Batch 1 / 11 Loss: 0.0030385295\n",
      "Iter 40 / 64 Batch 2 / 11 Loss: 0.038181633\n",
      "Iter 40 / 64 Batch 3 / 11 Loss: 0.0058672284\n",
      "Iter 40 / 64 Batch 4 / 11 Loss: 0.0063469433\n",
      "Iter 40 / 64 Batch 5 / 11 Loss: 0.0043521803\n",
      "Iter 40 / 64 Batch 6 / 11 Loss: 0.0059974287\n",
      "Iter 40 / 64 Batch 7 / 11 Loss: 0.0022738455\n",
      "Iter 40 / 64 Batch 8 / 11 Loss: 0.0038070611\n",
      "Iter 40 / 64 Batch 9 / 11 Loss: 0.009144979\n",
      "Iter 40 / 64 Batch 10 / 11 Loss: 0.0014561\n",
      "Iter 41 / 64 Batch 0 / 11 Loss: 0.00040619748\n",
      "Iter 41 / 64 Batch 1 / 11 Loss: 0.0016312695\n",
      "Iter 41 / 64 Batch 2 / 11 Loss: 0.06571631\n",
      "Iter 41 / 64 Batch 3 / 11 Loss: 0.0005279987\n",
      "Iter 41 / 64 Batch 4 / 11 Loss: 0.0047423188\n",
      "Iter 41 / 64 Batch 5 / 11 Loss: 0.022055583\n",
      "Iter 41 / 64 Batch 6 / 11 Loss: 0.005409865\n",
      "Iter 41 / 64 Batch 7 / 11 Loss: 0.00520636\n",
      "Iter 41 / 64 Batch 8 / 11 Loss: 0.00094813295\n",
      "Iter 41 / 64 Batch 9 / 11 Loss: 0.005571594\n",
      "Iter 41 / 64 Batch 10 / 11 Loss: 0.0015800295\n",
      "Iter 42 / 64 Batch 0 / 11 Loss: 0.0002907567\n",
      "Iter 42 / 64 Batch 1 / 11 Loss: 0.007559041\n",
      "Iter 42 / 64 Batch 2 / 11 Loss: 0.011004286\n",
      "Iter 42 / 64 Batch 3 / 11 Loss: 0.004328272\n",
      "Iter 42 / 64 Batch 4 / 11 Loss: 0.0011596424\n",
      "Iter 42 / 64 Batch 5 / 11 Loss: 0.0034276352\n",
      "Iter 42 / 64 Batch 6 / 11 Loss: 0.029076306\n",
      "Iter 42 / 64 Batch 7 / 11 Loss: 0.0023900662\n",
      "Iter 42 / 64 Batch 8 / 11 Loss: 0.0013538986\n",
      "Iter 42 / 64 Batch 9 / 11 Loss: 0.008857878\n",
      "Iter 42 / 64 Batch 10 / 11 Loss: 0.0028835528\n",
      "Iter 43 / 64 Batch 0 / 11 Loss: 0.0007489201\n",
      "Iter 43 / 64 Batch 1 / 11 Loss: 0.0013901187\n",
      "Iter 43 / 64 Batch 2 / 11 Loss: 0.01810945\n",
      "Iter 43 / 64 Batch 3 / 11 Loss: 0.0022488234\n",
      "Iter 43 / 64 Batch 4 / 11 Loss: 0.03402062\n",
      "Iter 43 / 64 Batch 5 / 11 Loss: 0.014040219\n",
      "Iter 43 / 64 Batch 6 / 11 Loss: 0.017477812\n",
      "Iter 43 / 64 Batch 7 / 11 Loss: 0.0007040083\n",
      "Iter 43 / 64 Batch 8 / 11 Loss: 0.0085369665\n",
      "Iter 43 / 64 Batch 9 / 11 Loss: 0.014787723\n",
      "Iter 43 / 64 Batch 10 / 11 Loss: 0.0010499492\n",
      "Iter 44 / 64 Batch 0 / 11 Loss: 0.0023442493\n",
      "Iter 44 / 64 Batch 1 / 11 Loss: 0.00086029316\n",
      "Iter 44 / 64 Batch 2 / 11 Loss: 0.022018762\n",
      "Iter 44 / 64 Batch 3 / 11 Loss: 0.0008701193\n",
      "Iter 44 / 64 Batch 4 / 11 Loss: 0.0017620851\n",
      "Iter 44 / 64 Batch 5 / 11 Loss: 0.006279829\n",
      "Iter 44 / 64 Batch 6 / 11 Loss: 0.0033782888\n",
      "Iter 44 / 64 Batch 7 / 11 Loss: 0.00066716044\n",
      "Iter 44 / 64 Batch 8 / 11 Loss: 0.01758974\n",
      "Iter 44 / 64 Batch 9 / 11 Loss: 0.00963243\n",
      "Iter 44 / 64 Batch 10 / 11 Loss: 0.02263306\n",
      "Iter 45 / 64 Batch 0 / 11 Loss: 0.0036029618\n",
      "Iter 45 / 64 Batch 1 / 11 Loss: 0.015842518\n",
      "Iter 45 / 64 Batch 2 / 11 Loss: 0.013662376\n",
      "Iter 45 / 64 Batch 3 / 11 Loss: 0.0007909352\n",
      "Iter 45 / 64 Batch 4 / 11 Loss: 0.0030574626\n",
      "Iter 45 / 64 Batch 5 / 11 Loss: 0.014643964\n",
      "Iter 45 / 64 Batch 6 / 11 Loss: 0.003816359\n",
      "Iter 45 / 64 Batch 7 / 11 Loss: 0.00087276276\n",
      "Iter 45 / 64 Batch 8 / 11 Loss: 0.0009371297\n",
      "Iter 45 / 64 Batch 9 / 11 Loss: 0.0073789977\n",
      "Iter 45 / 64 Batch 10 / 11 Loss: 0.0022025637\n",
      "Iter 46 / 64 Batch 0 / 11 Loss: 0.0019357493\n",
      "Iter 46 / 64 Batch 1 / 11 Loss: 0.00893084\n",
      "Iter 46 / 64 Batch 2 / 11 Loss: 0.025252819\n",
      "Iter 46 / 64 Batch 3 / 11 Loss: 0.004348172\n",
      "Iter 46 / 64 Batch 4 / 11 Loss: 0.0029102229\n",
      "Iter 46 / 64 Batch 5 / 11 Loss: 0.0053050593\n",
      "Iter 46 / 64 Batch 6 / 11 Loss: 0.0022051993\n",
      "Iter 46 / 64 Batch 7 / 11 Loss: 0.0044383593\n",
      "Iter 46 / 64 Batch 8 / 11 Loss: 0.0009239029\n",
      "Iter 46 / 64 Batch 9 / 11 Loss: 0.0034851735\n",
      "Iter 46 / 64 Batch 10 / 11 Loss: 0.0011514549\n",
      "Iter 47 / 64 Batch 0 / 11 Loss: 0.0030283474\n",
      "Iter 47 / 64 Batch 1 / 11 Loss: 0.002606572\n",
      "Iter 47 / 64 Batch 2 / 11 Loss: 0.017726593\n",
      "Iter 47 / 64 Batch 3 / 11 Loss: 0.021648489\n",
      "Iter 47 / 64 Batch 4 / 11 Loss: 0.0040288614\n",
      "Iter 47 / 64 Batch 5 / 11 Loss: 0.016661027\n",
      "Iter 47 / 64 Batch 6 / 11 Loss: 0.011971214\n",
      "Iter 47 / 64 Batch 7 / 11 Loss: 0.0005168126\n",
      "Iter 47 / 64 Batch 8 / 11 Loss: 0.0019918918\n",
      "Iter 47 / 64 Batch 9 / 11 Loss: 0.0028635738\n",
      "Iter 47 / 64 Batch 10 / 11 Loss: 0.001016951\n",
      "Iter 48 / 64 Batch 0 / 11 Loss: 0.0032754608\n",
      "Iter 48 / 64 Batch 1 / 11 Loss: 0.0008468566\n",
      "Iter 48 / 64 Batch 2 / 11 Loss: 0.056758966\n",
      "Iter 48 / 64 Batch 3 / 11 Loss: 0.0007823365\n",
      "Iter 48 / 64 Batch 4 / 11 Loss: 0.0039533377\n",
      "Iter 48 / 64 Batch 5 / 11 Loss: 0.0041224654\n",
      "Iter 48 / 64 Batch 6 / 11 Loss: 0.0030321798\n",
      "Iter 48 / 64 Batch 7 / 11 Loss: 0.0167697\n",
      "Iter 48 / 64 Batch 8 / 11 Loss: 0.008090189\n",
      "Iter 48 / 64 Batch 9 / 11 Loss: 0.012449002\n",
      "Iter 48 / 64 Batch 10 / 11 Loss: 0.0038545995\n",
      "Iter 49 / 64 Batch 0 / 11 Loss: 0.0010807079\n",
      "Iter 49 / 64 Batch 1 / 11 Loss: 0.0037191038\n",
      "Iter 49 / 64 Batch 2 / 11 Loss: 0.005298635\n",
      "Iter 49 / 64 Batch 3 / 11 Loss: 0.0019948613\n",
      "Iter 49 / 64 Batch 4 / 11 Loss: 0.00070652063\n",
      "Iter 49 / 64 Batch 5 / 11 Loss: 0.026797539\n",
      "Iter 49 / 64 Batch 6 / 11 Loss: 0.03451316\n",
      "Iter 49 / 64 Batch 7 / 11 Loss: 0.0010520663\n",
      "Iter 49 / 64 Batch 8 / 11 Loss: 0.08148769\n",
      "Iter 49 / 64 Batch 9 / 11 Loss: 0.01549216\n",
      "Iter 49 / 64 Batch 10 / 11 Loss: 0.006845666\n",
      "Iter 50 / 64 Batch 0 / 11 Loss: 0.0024861635\n",
      "Iter 50 / 64 Batch 1 / 11 Loss: 0.0006608432\n",
      "Iter 50 / 64 Batch 2 / 11 Loss: 0.049050026\n",
      "Iter 50 / 64 Batch 3 / 11 Loss: 0.0010189776\n",
      "Iter 50 / 64 Batch 4 / 11 Loss: 0.0045069815\n",
      "Iter 50 / 64 Batch 5 / 11 Loss: 0.0038995026\n",
      "Iter 50 / 64 Batch 6 / 11 Loss: 0.009811198\n",
      "Iter 50 / 64 Batch 7 / 11 Loss: 0.0006274566\n",
      "Iter 50 / 64 Batch 8 / 11 Loss: 0.0058910036\n",
      "Iter 50 / 64 Batch 9 / 11 Loss: 0.14695728\n",
      "Iter 50 / 64 Batch 10 / 11 Loss: 0.0027167131\n",
      "Iter 51 / 64 Batch 0 / 11 Loss: 0.0011677097\n",
      "Iter 51 / 64 Batch 1 / 11 Loss: 0.03846879\n",
      "Iter 51 / 64 Batch 2 / 11 Loss: 0.021025222\n",
      "Iter 51 / 64 Batch 3 / 11 Loss: 0.0005789068\n",
      "Iter 51 / 64 Batch 4 / 11 Loss: 0.0015437673\n",
      "Iter 51 / 64 Batch 5 / 11 Loss: 0.0035737986\n",
      "Iter 51 / 64 Batch 6 / 11 Loss: 0.0027727308\n",
      "Iter 51 / 64 Batch 7 / 11 Loss: 0.0013184901\n",
      "Iter 51 / 64 Batch 8 / 11 Loss: 0.0025913827\n",
      "Iter 51 / 64 Batch 9 / 11 Loss: 0.0040953737\n",
      "Iter 51 / 64 Batch 10 / 11 Loss: 0.0016691474\n",
      "Iter 52 / 64 Batch 0 / 11 Loss: 0.0012205477\n",
      "Iter 52 / 64 Batch 1 / 11 Loss: 0.004164524\n",
      "Iter 52 / 64 Batch 2 / 11 Loss: 0.068656646\n",
      "Iter 52 / 64 Batch 3 / 11 Loss: 0.008856733\n",
      "Iter 52 / 64 Batch 4 / 11 Loss: 0.0043939883\n",
      "Iter 52 / 64 Batch 5 / 11 Loss: 0.0058889617\n",
      "Iter 52 / 64 Batch 6 / 11 Loss: 0.004794322\n",
      "Iter 52 / 64 Batch 7 / 11 Loss: 0.00016773163\n",
      "Iter 52 / 64 Batch 8 / 11 Loss: 0.0059275003\n",
      "Iter 52 / 64 Batch 9 / 11 Loss: 0.037382077\n",
      "Iter 52 / 64 Batch 10 / 11 Loss: 0.004454814\n",
      "Iter 53 / 64 Batch 0 / 11 Loss: 0.0029018014\n",
      "Iter 53 / 64 Batch 1 / 11 Loss: 0.0074337865\n",
      "Iter 53 / 64 Batch 2 / 11 Loss: 0.02004101\n",
      "Iter 53 / 64 Batch 3 / 11 Loss: 0.0073243775\n",
      "Iter 53 / 64 Batch 4 / 11 Loss: 0.0025323755\n",
      "Iter 53 / 64 Batch 5 / 11 Loss: 0.042269304\n",
      "Iter 53 / 64 Batch 6 / 11 Loss: 0.0033080322\n",
      "Iter 53 / 64 Batch 7 / 11 Loss: 0.0007521464\n",
      "Iter 53 / 64 Batch 8 / 11 Loss: 0.0034699384\n",
      "Iter 53 / 64 Batch 9 / 11 Loss: 0.004474721\n",
      "Iter 53 / 64 Batch 10 / 11 Loss: 0.0009354329\n",
      "Iter 54 / 64 Batch 0 / 11 Loss: 0.0008384642\n",
      "Iter 54 / 64 Batch 1 / 11 Loss: 0.025748627\n",
      "Iter 54 / 64 Batch 2 / 11 Loss: 0.014347524\n",
      "Iter 54 / 64 Batch 3 / 11 Loss: 0.0012773525\n",
      "Iter 54 / 64 Batch 4 / 11 Loss: 0.0059369495\n",
      "Iter 54 / 64 Batch 5 / 11 Loss: 0.008057589\n",
      "Iter 54 / 64 Batch 6 / 11 Loss: 0.008872206\n",
      "Iter 54 / 64 Batch 7 / 11 Loss: 0.0015351363\n",
      "Iter 54 / 64 Batch 8 / 11 Loss: 0.0017241042\n",
      "Iter 54 / 64 Batch 9 / 11 Loss: 0.0044354126\n",
      "Iter 54 / 64 Batch 10 / 11 Loss: 0.0009168519\n",
      "Iter 55 / 64 Batch 0 / 11 Loss: 0.0016091252\n",
      "Iter 55 / 64 Batch 1 / 11 Loss: 0.0038360616\n",
      "Iter 55 / 64 Batch 2 / 11 Loss: 0.025311444\n",
      "Iter 55 / 64 Batch 3 / 11 Loss: 0.0099102\n",
      "Iter 55 / 64 Batch 4 / 11 Loss: 0.013239969\n",
      "Iter 55 / 64 Batch 5 / 11 Loss: 0.007780972\n",
      "Iter 55 / 64 Batch 6 / 11 Loss: 0.006100402\n",
      "Iter 55 / 64 Batch 7 / 11 Loss: 0.0005127437\n",
      "Iter 55 / 64 Batch 8 / 11 Loss: 0.0035400526\n",
      "Iter 55 / 64 Batch 9 / 11 Loss: 0.006125911\n",
      "Iter 55 / 64 Batch 10 / 11 Loss: 0.0015726613\n",
      "Iter 56 / 64 Batch 0 / 11 Loss: 0.002007761\n",
      "Iter 56 / 64 Batch 1 / 11 Loss: 0.004772638\n",
      "Iter 56 / 64 Batch 2 / 11 Loss: 0.029505704\n",
      "Iter 56 / 64 Batch 3 / 11 Loss: 0.011107805\n",
      "Iter 56 / 64 Batch 4 / 11 Loss: 0.0013317\n",
      "Iter 56 / 64 Batch 5 / 11 Loss: 0.029893432\n",
      "Iter 56 / 64 Batch 6 / 11 Loss: 0.0049717883\n",
      "Iter 56 / 64 Batch 7 / 11 Loss: 0.0017191225\n",
      "Iter 56 / 64 Batch 8 / 11 Loss: 0.0006051605\n",
      "Iter 56 / 64 Batch 9 / 11 Loss: 0.028197207\n",
      "Iter 56 / 64 Batch 10 / 11 Loss: 0.003612142\n",
      "Iter 57 / 64 Batch 0 / 11 Loss: 0.0014182825\n",
      "Iter 57 / 64 Batch 1 / 11 Loss: 0.0052988054\n",
      "Iter 57 / 64 Batch 2 / 11 Loss: 0.033239584\n",
      "Iter 57 / 64 Batch 3 / 11 Loss: 0.0032064866\n",
      "Iter 57 / 64 Batch 4 / 11 Loss: 0.00031608477\n",
      "Iter 57 / 64 Batch 5 / 11 Loss: 0.048879325\n",
      "Iter 57 / 64 Batch 6 / 11 Loss: 0.0036811687\n",
      "Iter 57 / 64 Batch 7 / 11 Loss: 0.00078044715\n",
      "Iter 57 / 64 Batch 8 / 11 Loss: 0.0018476844\n",
      "Iter 57 / 64 Batch 9 / 11 Loss: 0.006592127\n",
      "Iter 57 / 64 Batch 10 / 11 Loss: 0.0021427595\n",
      "Iter 58 / 64 Batch 0 / 11 Loss: 0.0015867781\n",
      "Iter 58 / 64 Batch 1 / 11 Loss: 0.0027335132\n",
      "Iter 58 / 64 Batch 2 / 11 Loss: 0.019466957\n",
      "Iter 58 / 64 Batch 3 / 11 Loss: 0.002302488\n",
      "Iter 58 / 64 Batch 4 / 11 Loss: 0.0012416863\n",
      "Iter 58 / 64 Batch 5 / 11 Loss: 0.02945411\n",
      "Iter 58 / 64 Batch 6 / 11 Loss: 0.0028108095\n",
      "Iter 58 / 64 Batch 7 / 11 Loss: 0.00042312077\n",
      "Iter 58 / 64 Batch 8 / 11 Loss: 0.0039000749\n",
      "Iter 58 / 64 Batch 9 / 11 Loss: 0.007125162\n",
      "Iter 58 / 64 Batch 10 / 11 Loss: 0.014130996\n",
      "Iter 59 / 64 Batch 0 / 11 Loss: 0.0011202871\n",
      "Iter 59 / 64 Batch 1 / 11 Loss: 0.0011445654\n",
      "Iter 59 / 64 Batch 2 / 11 Loss: 0.034542475\n",
      "Iter 59 / 64 Batch 3 / 11 Loss: 0.0012302678\n",
      "Iter 59 / 64 Batch 4 / 11 Loss: 0.00054906076\n",
      "Iter 59 / 64 Batch 5 / 11 Loss: 0.013899608\n",
      "Iter 59 / 64 Batch 6 / 11 Loss: 0.011834465\n",
      "Iter 59 / 64 Batch 7 / 11 Loss: 0.00078636734\n",
      "Iter 59 / 64 Batch 8 / 11 Loss: 0.007228379\n",
      "Iter 59 / 64 Batch 9 / 11 Loss: 0.005339581\n",
      "Iter 59 / 64 Batch 10 / 11 Loss: 0.005131996\n",
      "Iter 60 / 64 Batch 0 / 11 Loss: 0.014125115\n",
      "Iter 60 / 64 Batch 1 / 11 Loss: 0.0025091837\n",
      "Iter 60 / 64 Batch 2 / 11 Loss: 0.019404968\n",
      "Iter 60 / 64 Batch 3 / 11 Loss: 0.0020562855\n",
      "Iter 60 / 64 Batch 4 / 11 Loss: 0.015632\n",
      "Iter 60 / 64 Batch 5 / 11 Loss: 0.002105907\n",
      "Iter 60 / 64 Batch 6 / 11 Loss: 0.001980634\n",
      "Iter 60 / 64 Batch 7 / 11 Loss: 0.0043608914\n",
      "Iter 60 / 64 Batch 8 / 11 Loss: 0.0030752148\n",
      "Iter 60 / 64 Batch 9 / 11 Loss: 0.0023448367\n",
      "Iter 60 / 64 Batch 10 / 11 Loss: 0.0058957697\n",
      "Iter 61 / 64 Batch 0 / 11 Loss: 0.0019121276\n",
      "Iter 61 / 64 Batch 1 / 11 Loss: 0.0017792755\n",
      "Iter 61 / 64 Batch 2 / 11 Loss: 0.050602894\n",
      "Iter 61 / 64 Batch 3 / 11 Loss: 0.0013925025\n",
      "Iter 61 / 64 Batch 4 / 11 Loss: 0.0065174154\n",
      "Iter 61 / 64 Batch 5 / 11 Loss: 0.017379712\n",
      "Iter 61 / 64 Batch 6 / 11 Loss: 0.0017035007\n",
      "Iter 61 / 64 Batch 7 / 11 Loss: 0.0019565208\n",
      "Iter 61 / 64 Batch 8 / 11 Loss: 0.0009640821\n",
      "Iter 61 / 64 Batch 9 / 11 Loss: 0.061811354\n",
      "Iter 61 / 64 Batch 10 / 11 Loss: 0.0017653774\n",
      "Iter 62 / 64 Batch 0 / 11 Loss: 0.01502688\n",
      "Iter 62 / 64 Batch 1 / 11 Loss: 0.0012507772\n",
      "Iter 62 / 64 Batch 2 / 11 Loss: 0.05674372\n",
      "Iter 62 / 64 Batch 3 / 11 Loss: 0.002817112\n",
      "Iter 62 / 64 Batch 4 / 11 Loss: 0.025876975\n",
      "Iter 62 / 64 Batch 5 / 11 Loss: 0.015776668\n",
      "Iter 62 / 64 Batch 6 / 11 Loss: 0.00895705\n",
      "Iter 62 / 64 Batch 7 / 11 Loss: 0.0062050386\n",
      "Iter 62 / 64 Batch 8 / 11 Loss: 0.0020703194\n",
      "Iter 62 / 64 Batch 9 / 11 Loss: 0.00935754\n",
      "Iter 62 / 64 Batch 10 / 11 Loss: 0.004891853\n",
      "Iter 63 / 64 Batch 0 / 11 Loss: 0.05237192\n",
      "Iter 63 / 64 Batch 1 / 11 Loss: 0.0016201248\n",
      "Iter 63 / 64 Batch 2 / 11 Loss: 0.015234127\n",
      "Iter 63 / 64 Batch 3 / 11 Loss: 0.005357415\n",
      "Iter 63 / 64 Batch 4 / 11 Loss: 0.003913563\n",
      "Iter 63 / 64 Batch 5 / 11 Loss: 0.021372966\n",
      "Iter 63 / 64 Batch 6 / 11 Loss: 0.003856706\n",
      "Iter 63 / 64 Batch 7 / 11 Loss: 0.00048725327\n",
      "Iter 63 / 64 Batch 8 / 11 Loss: 0.0023703664\n",
      "Iter 63 / 64 Batch 9 / 11 Loss: 0.019982582\n",
      "Iter 63 / 64 Batch 10 / 11 Loss: 0.004214909\n"
     ]
    }
   ],
   "source": [
    "resize_shape=[64,64]\n",
    "func_params={\n",
    "    'image_func':image_file_to_np,\n",
    "    'zip_file':zf,\n",
    "    'image_func_kargs':{'resize':resize_shape,'batch_shape':True}\n",
    "}\n",
    "\n",
    "#Set up parameters\n",
    "input_df = train_df\n",
    "batch_x_func = zip_image_file_to_np\n",
    "batch_x_func_params = func_params\n",
    "df_x_col=file_col\n",
    "df_y_col=hot_key_col\n",
    "\n",
    "iterations = 64\n",
    "batch_size=8\n",
    "\n",
    "rows = input_df.shape[0]\n",
    "batches = int(np.ceil(rows/batch_size))\n",
    "\n",
    "for iterx in range(iterations):\n",
    "    for batch in range(batches):\n",
    "        offset = batch*batch_size\n",
    "        xb,yb = batch_proc_from_df(input_df,batch_x_func,batch_x_func_params,\n",
    "                           x_col=df_x_col,y_col=df_y_col,batch_size=batch_size,offset=offset)\n",
    "        loss = modelx.train_on_batch(xb, yb)\n",
    "        print(\"Iter\",iterx,\"/\",iterations,\"Batch\",batch,\"/\",batches,\"Loss:\",loss)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test training set\n",
      "Batch 0 / 11 Loss: 0.02864028885960579\n",
      "Batch 1 / 11 Loss: 0.01135550532490015\n",
      "Batch 2 / 11 Loss: 0.00039284140802919865\n",
      "Batch 3 / 11 Loss: 0.051368847489356995\n",
      "Batch 4 / 11 Loss: 0.0008814703905954957\n",
      "Batch 5 / 11 Loss: 0.000958493328653276\n",
      "Batch 6 / 11 Loss: 0.0004208873142488301\n",
      "Batch 7 / 11 Loss: 0.009767487645149231\n",
      "Batch 8 / 11 Loss: 0.002210453152656555\n",
      "Batch 9 / 11 Loss: 0.002058537909761071\n",
      "Batch 10 / 11 Loss: 0.05804501101374626\n",
      "Total accuracy 1.0\n",
      "test testing set\n",
      "Batch 0 / 3 Loss: 0.21175697445869446\n",
      "Batch 1 / 3 Loss: 0.7520488500595093\n",
      "Batch 2 / 3 Loss: 0.3832610547542572\n",
      "Total accuracy 0.7083333333333334\n"
     ]
    }
   ],
   "source": [
    "print(\"test training set\")\n",
    "test_model(train_df,keras_model=modelx)\n",
    "\n",
    "print(\"test testing set\")\n",
    "test_model(test_df,keras_model=modelx)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}