08_Markov_Relational Representation_Neural_Embeddings_20171128.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Driven Modeling\n",
    "<hr style=\"height:3px;border:none;color:stillblue;background-color:black;\" />\n",
    "### PhD seminar series at Chair for Computer Aided Architectural Design (CAAD), ETH Zurich\n",
    "\n",
    "\n",
    "[Vahid Moosavi](href=https://vahidmoosavi.com/>)\n",
    "<hr style=\"height:1px;border:none;color:stillblue;background-color:black;\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a></a>\n",
    "\n",
    "# Session 08 \n",
    "<hr style=\"height:1px;border:none;color:stillblue;background-color:steelblue;\" />\n",
    "<span style=\"color:steelblue;font-size: 1em;\"> 28 November 2017</span>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Topics to be discussed \n",
    "\n",
    "* **Text generation with Markov Chains**\n",
    "* **Markov Chains from the point of view of relational representation**\n",
    "* **Neuro-probablistic Models of the language**\n",
    "* ** Natural Language Modeling problems**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "pd.__version__\n",
    "import sys\n",
    "from scipy import stats\n",
    "import time\n",
    "from scipy.linalg import norm\n",
    "import sompylib.sompy as SOM\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generative examples of Markov"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example of sequence of characters in English texts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "25000"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with open('./Data/IMDB_data/pos.txt','r') as infile:\n",
    "    reviews = infile.readlines()\n",
    "len(reviews)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Bromwell High is a cartoon comedy. It ran at the same time as some other programs about school life, such as \"Teachers\". My 35 years in the teaching profession lead me to believe that Bromwell High\\'s satire is much closer to reality than is \"Teachers\". The scramble to survive financially, the insightful students who can see right through their pathetic teachers\\' pomp, the pettiness of the whole situation, all remind me of the schools I knew and their students. When I saw the episode in which a student repeatedly tried to burn down the school, I immediately recalled ......... at .......... High. A classic line: INSPECTOR: I\\'m here to sack one of your teachers. STUDENT: Welcome to Bromwell High. I expect that many adults of my age think that Bromwell High is far fetched. What a pity that it isn\\'t!\\n'"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reviews[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cleanText(corpus):\n",
    "    import string\n",
    "    validchars = string.ascii_letters + string.digits + ' '\n",
    "    punctuation = \"\"\".,:;?!@(){}[]$1234567890\"\"\"\n",
    "    corpus = [z.lower().replace('\\n','') for z in corpus]\n",
    "    corpus = [z.replace('<br />', ' ') for z in corpus]\n",
    "    \n",
    "    for c in punctuation:\n",
    "        corpus =[z.replace(c, '') for z in corpus]\n",
    "    \n",
    "\n",
    "    corpus = [''.join(ch for ch in z if ch in validchars) for z in corpus]\n",
    "    \n",
    "    #treat punctuation as individual words\n",
    "    for c in punctuation:\n",
    "        corpus = [z.replace(c, ' %s '%c) for z in corpus]\n",
    "#     corpus = [z.split() for z in corpus]\n",
    "    corpus = [z.replace(' ', '_') for z in corpus]\n",
    "    return corpus\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "texts  = cleanText(reviews)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'bromwell_high_is_a_cartoon_comedy_it_ran_at_the_same_time_as_some_other_programs_about_school_life_such_as_teachers_my__years_in_the_teaching_profession_lead_me_to_believe_that_bromwell_highs_satire_is_much_closer_to_reality_than_is_teachers_the_scramble_to_survive_financially_the_insightful_students_who_can_see_right_through_their_pathetic_teachers_pomp_the_pettiness_of_the_whole_situation_all_remind_me_of_the_schools_i_knew_and_their_students_when_i_saw_the_episode_in_which_a_student_repeatedly_tried_to_burn_down_the_school_i_immediately_recalled__at__high_a_classic_line_inspector_im_here_to_sack_one_of_your_teachers_student_welcome_to_bromwell_high_i_expect_that_many_adults_of_my_age_think_that_bromwell_high_is_far_fetched_what_a_pity_that_it_isnt'"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "texts[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'_': 0,\n",
       " 'a': 1,\n",
       " 'b': 2,\n",
       " 'c': 3,\n",
       " 'd': 4,\n",
       " 'e': 5,\n",
       " 'f': 6,\n",
       " 'g': 7,\n",
       " 'h': 8,\n",
       " 'i': 9,\n",
       " 'j': 10,\n",
       " 'k': 11,\n",
       " 'l': 12,\n",
       " 'm': 13,\n",
       " 'n': 14,\n",
       " 'o': 15,\n",
       " 'p': 16,\n",
       " 'q': 17,\n",
       " 'r': 18,\n",
       " 's': 19,\n",
       " 't': 20,\n",
       " 'u': 21,\n",
       " 'v': 22,\n",
       " 'w': 23,\n",
       " 'x': 24,\n",
       " 'y': 25,\n",
       " 'z': 26}"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_chars = '_abcdefghijklmnopqrstuvwxyz'\n",
    "dictionary = {}\n",
    "for i in range(len(all_chars)):\n",
    "    dictionary[all_chars[i]] = i\n",
    "dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25000\n"
     ]
    }
   ],
   "source": [
    "# building data with the format of sequence\n",
    "data = []\n",
    "for text in texts[:]:\n",
    "    d = []\n",
    "    for c in text:\n",
    "        d.append(dictionary[c])\n",
    "    data.append(d)\n",
    "print len(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def buildTM_from_sequential_data(data,states,irreducible=True):\n",
    "    # each row is a sequence of observation\n",
    "    n = len(states)\n",
    "    M = np.zeros((n,n))\n",
    "    for d in data:\n",
    "        for k in range(1,len(d)):\n",
    "            i = d[k-1]\n",
    "            \n",
    "            j = d[k]\n",
    "            M[i,j]= M[i,j] + 1\n",
    "    \n",
    "    eps = .001\n",
    "    for i in range(M.shape[0]):\n",
    "        s= sum(M[i])\n",
    "        \n",
    "        if s==0:\n",
    "            if irreducible==True:\n",
    "                M[i]=eps\n",
    "                M[i,i]=1.\n",
    "                s= sum(M[i])\n",
    "                M[i]=np.divide(M[i],s)\n",
    "            else:\n",
    "                M[i,i]=1.\n",
    "        else:\n",
    "            M[i]=np.divide(M[i],s)    \n",
    "    return M\n",
    "\n",
    "\n",
    "# Power iteration Method\n",
    "def simulate_markov(TM,verbose='on'):\n",
    "    e1 = time.time()\n",
    "    states_n = TM.shape[0]\n",
    "    pi = np.ones(states_n);  pi1 = np.zeros(states_n);\n",
    "    pi = np.random.rand(states_n)\n",
    "   \n",
    "    pi = pi/pi.sum()\n",
    "    n = norm(pi - pi1); i = 0;\n",
    "    diff = []\n",
    "    while n > 1e-6 and i <1*1e4 :\n",
    "        pi1 = TM.T.dot(pi).copy()\n",
    "        n = norm(pi - pi1); i += 1\n",
    "        diff.append(n)\n",
    "        pi = pi1.copy()\n",
    "    if verbose=='on':\n",
    "        print \"Iterating {} times in {}\".format(i, time.time() - e1)\n",
    "    \n",
    "    mixing_ = i\n",
    "    return pi1,mixing_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "states = np.unique(dictionary.values())\n",
    "M_char = buildTM_from_sequential_data(data,states,irreducible=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "chars = np.asarray([c for c in all_chars])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## steady state probabilites : equal to the frequencies of characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iterating 13 times in 0.000731945037842\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl4nGXV+PHvyb42bZo0KWmbdKN7\nKaQtVAQpKBZkfS0KIoKKdaHuKK0L+vKCiqC8+lNf2RSQpewCpVC2gshSuu9ruqdLkiZNOtkzc35/\nzDN1miaZmWS2NOdzXXN15pnz3Lknmc6Z515FVTHGGGMSYl0BY4wx8cESgjHGGMASgjHGGIclBGOM\nMYAlBGOMMQ5LCMYYYwBLCMYYYxyWEIwxxgCWEIwxxjiSYl2BUOTl5WlJSUm3zq2vryczMzMi8ZEs\nO9LxVpfwxFtd4r8uocb35rq0t2LFiipVzQ8YqKq95lZaWqrdtWTJkojFR7LsSMdbXcITb3WJ/7qE\nGt+b69IesFyD+Iy1JiNjjDGA9SEYY4xxWEIwxhgDWEIwxhjjsIRgjDEGsIRgjDHGYQnBGGMM0EcS\nwvOr9vHWntZYV8MYY+Jan0gIi9YdtIRgjDEB9ImEkJeVSm2LxroaxhgT1/pEQsjPTsXVAm1uT6yr\nYowxcavPJAQFqutbYl0VY4yJW30jIWSlAFBxtDnGNTHGmPjVNxJCdioAlS5LCMYY05mgEoKIzBKR\nLSKyXUTmdfD8uSKyUkTaRGS23/GZIrLa79YkIlc4zz0kIjv9npsSvpd1vPysNACq7ArBGGM6FXCD\nHBFJBP4MfArYBywTkRdVdaNf2B7gBuBm/3NVdQkwxSknF9gOvOYX8iNVfaYnLyAYedneJiO7QjDG\nmM4Fs2PadGC7qu4AEJEFwOXAsYSgqruc57oaxjMbeEVVG7pd227KSEkiLREq7QrBGGM6FUyTURGw\n1+/xPudYqK4Gnmh37A4RWSsi94hIajfKDFpOqlhCMMaYLoh3d7UuArx9ArNU9Ubn8XXAmao6t4PY\nh4CF7ZuBRGQwsBY4RVVb/Y4dBFKA+4AyVb2tgzLnAHMACgoKShcsWBDqawTgf953kZyUyLzp6UHF\nu1wusrKywh4bb/FWl/DEW13ivy6hxvfmurQ3c+bMFao6NWBgoD02gRnAYr/H84H5ncQ+BMzu4Ph3\ngfu6+Bnn4U0kEdtT+bO/f0XPv3tJ0PG9de/VUOOtLuGJt7rEf11Cje/NdWmPMO6pvAwYLSLDRSQF\nb9PPiyGlJ7iGds1FzhUCIiLAFcD6EMsMiTUZGWNM1wImBFVtA+YCi4FNwFOqukFEbhORywBEZJqI\n7AOuAu4VkQ2+80WkBBgKvNOu6MdEZB2wDsgDbu/5y+lcTqpQ19RGc5s7kj/GGGN6rWBGGaGqi4BF\n7Y7d6nd/GTCkk3N30UEntKqeH0pFeyonRQCocrVQ1D+4fgRjjOlL+sRMZYB+qd6EYM1GxhjTsT6T\nEHIsIRhjTJf6TkI41mRkCcEYYzrSZxKCNRkZY0zX+kxCSE4QctKTLSEYY0wn+kxCAO8y2NZkZIwx\nHetbCSEr1a4QjDGmE30qIeRlp9oS2MYY04k+lRDsCsEYYzrXtxJCdioNLW7qm9tiXRVjjIk7fS4h\ngM1FMMaYjvSphJCX5Wylac1Gxhhzgj6VEOwKwRhjOtcnE4JdIRhjzIn6VELIzUhBxBKCMcZ0pE8l\nhKTEBAZmplDpaol1VYwxJu70qYQAkGdzEYwxpkN9LiHk22xlY4zpUFAJQURmicgWEdkuIvM6eP5c\nEVkpIm0iMrvdc24RWe3cXvQ7PlxEljplPikiKT1/OYHlZ6VSZVcIxhhzgoAJQUQSgT8DFwHjgWtE\nZHy7sD3ADcDjHRTRqKpTnNtlfsfvBO5R1VFADfDVbtQ/ZL4rBFWNxo8zxpheI5grhOnAdlXdoaot\nwALgcv8AVd2lqmsBTzA/VEQEOB94xjn0MHBF0LXugfzsVFraPNQ12fIVxhjjL5iEUATs9Xu8zzkW\nrDQRWS4iH4qI70N/IHBEVX2fyqGW2W15WTYXwRhjOiKBmk6cPoFZqnqj8/g64ExVndtB7EPAQlV9\nxu9YkaqWi8gI4C3gAqAW+NBpLkJEhgKvqOrEDsqcA8wBKCgoKF2wYEG3XqjL5SIrK4uNh938dlkT\n86anMTY3MWB8KGWHWpd4iLe6hCfe6hL/dQk1vjfXpb2ZM2euUNWpAQNVtcsbMANY7Pd4PjC/k9iH\ngNldlPUQMBsQoApI6uhndHYrLS3V7lqyZImqqm45WKfFtyzUF1eXBxUfStm9Md7qEp54q0v81yXU\n+N5cl/aA5Rrg81VVg2oyWgaMdkYFpQBXAy8GOAcAERkgIqnO/TzgbGCjU8ElTnIAuB54IZgyeyrf\nmoyMMaZDAROCetv55wKLgU3AU6q6QURuE5HLAERkmojsA64C7hWRDc7p44DlIrIGbwL4japudJ67\nBfiBiGzH26fwYDhfWGdy0pNJShCbi2CMMe0kBROkqouARe2O3ep3fxkwpIPz3gcmdVLmDrwjmKIq\nIUHIs7kIxhhzgj43UxlstrIxxnSk7yYEu0Iwxpjj9MmEkJeVYpvkGGNMO30yIeRnp1LlasHjseUr\njDHGp28mhKxU3B6lpsH2RTDGGJ8+mRDyfFtpWrORMcYc0ycTgm9yWtVRu0IwxhifvpkQjl0hNMW4\nJsYYEz/6dkKwoafGGHNMn0wIWalJpCYlUOWyJiNjjPHpkwlBRGxymjHGtNMnEwLYbGVjjGmvzyaE\nvCxLCMYY46/PJgTvbGVLCMYY49N3E0JWKtUNLbS6PbGuijHGxIW+mxCyU1GF6nobaWSMMdCHE0Ke\nbaVpjDHH6bMJId/WMzLGmOMElRBEZJaIbBGR7SIyr4PnzxWRlSLSJiKz/Y5PEZEPRGSDiKwVkc/7\nPfeQiOwUkdXObUp4XlJwBtlsZWOMOU7APZVFJBH4M/ApYB+wTEReVNWNfmF7gBuAm9ud3gB8SVW3\nicgpwAoRWayqR5znf6Sqz/T0RXSHNRkZY8zxAiYEYDqwXVV3AIjIAuBy4FhCUNVdznPHDdlR1a1+\n9/eLSAWQDxwhxtJTEslKTbKhp8YY4wimyagI2Ov3eJ9zLCQiMh1IAcr8Dt/hNCXdIyKpoZbZUzZb\n2Rhj/kNUu95G0ukTmKWqNzqPrwPOVNW5HcQ+BCxs3wwkIoOBt4HrVfVDv2MH8SaJ+4AyVb2tgzLn\nAHMACgoKShcsWBDiS/RyuVxkZWUdd+xXSxsRYP6Z6UHFh1J2b4m3uoQn3uoS/3UJNb4316W9mTNn\nrlDVqQEDVbXLGzADWOz3eD4wv5PYh4DZ7Y71A1a2P94u5jy8iaTLupSWlmp3LVmy5IRj33x0uZ5/\n94nHO4sPpezeEm91CU+81SX+6xJqfG+uS3vAcg3w+aqqQTUZLQNGi8hwEUkBrgZeDCYrOfHPA49o\nx1cNiIgAVwDrgykznPJtPSNjjDkmYEJQ1TZgLrAY2AQ8paobROQ2EbkMQESmicg+4CrgXhHZ4Jz+\nOeBc4IYOhpc+JiLrgHVAHnB7WF9ZEPKzU6lraqOp1R3tH22MMXEnmFFGqOoiYFG7Y7f63V8GDOng\nvEeBRzsp8/yQahoBvslpVa5mhgzIiHFtjDEmtvrsTGX4z1wE2znNGGP6eEKwvZWNMeY/LCFgCcEY\nY6CPJ4SBmf/pQzDGmL6uTyeElKQE+mck2xWCMcbQxxMC2FwEY4zxsYSQnWp7IhhjDJYQyMtKtT4E\nY4zBEoKteGqMMQ5LCNmpNLS4qW9ui3VVjDEmpiwhZNnQU2OMAUsI5NnkNGOMASwhHLtCsIRgjOnr\nLCH4rhCsycgY08f1+YSQm5lCgkCVXSEYY/q4Pp8QEhOE3EybnGaMMX0+IYDNRTDGGLCEAPiWr7BN\ncowxfVtQCUFEZonIFhHZLiLzOnj+XBFZKSJtIjK73XPXi8g253a93/FSEVnnlPlHEZGev5zuyctK\nsT4EY0yfFzAhiEgi8GfgImA8cI2IjG8Xtge4AXi83bm5wC+AM4HpwC9EZIDz9P8BXwNGO7dZ3X4V\nPeRrMlLVWFXBGGNiLpgrhOnAdlXdoaotwALgcv8AVd2lqmsBT7tzPw28rqrVqloDvA7MEpHBQD9V\n/VC9n8KPAFf09MV0V35WKi1uD3WNtnyFMabvCiYhFAF7/R7vc44Fo7Nzi5z73Skz7GwugjHGgARq\nJnH6BGap6o3O4+uAM1V1bgexDwELVfUZ5/HNQJqq3u48/jnQCLwN/EZVP+kcPwe4RVUv6aDMOcAc\ngIKCgtIFCxZ064W6XC6ysrI6fG7jYTe/XdbELdPSGDcwMWB8KGVHO76myUNSawPZ2b2v7vFUl1Dj\nrS7xX5dQ43tzXdqbOXPmClWdGjBQVbu8ATOAxX6P5wPzO4l9CJjt9/ga4F6/x/c6xwYDmzuL6+xW\nWlqq3bVkyZJOn9t6sE6Lb1moL6wuDyo+lLKjGb+j0qXD5y3U//f06zGvS3fi46kuocZbXeK/LqHG\n9+a6tAcs1wCfr6oaVJPRMmC0iAwXkRTgauDFIBPTYuBCERngdCZf6CSXA0CdiJzljC76EvBCkGWG\nna/JqLePNFpfXotHYU9d+64cY4wJLGBCUNU2YC7eD/dNwFOqukFEbhORywBEZJqI7AOuAu4VkQ3O\nudXA/+BNKsuA25xjAN8CHgC2A2XAK2F9ZSHISU8mOVF6fR/Cjsp6ACoabLSUMSZ0ScEEqeoiYFG7\nY7f63V8GDOnk3L8Bf+vg+HJgYiiVjRQRIS+r989WLqt0AVDRaFcIxpjQ2Uxlx8mwfMWxhGBXCMaY\nbrCE4MjPSu3Vu6Z5PMqOynpE4HCj0tJmVwnGmNBYQnD09iajA3VNNLa6mTK0PwqUH2mMdZWMMb2M\nJQRHfnYqh+tb8Hh6Z3PLDqe56PwxgwDYU90Qy+oYY3ohSwiO/OxU3B6lpqF3rnpaVuFNCDPHOgnh\ncH0sq2OM6YUsITjysnr38hVllfVkpyUx4ZR+pCTA7sN2hWCMCY0lBMex9Yx6aT9CWaWLEflZiAj5\nGcJuazIyxoTIEoLjZEgII/MzARiUkcAeu0IwxoTIEoLj2PIVvbDJyNXcxqG6Zkbmexe/GpQu7Klu\nsP0djDEhsYTgyExJJC05oVdeIfhGGPkSQn5GAo2t7l75WowxsWMJwSEivXa2sm+G8qhBviYj726k\n1o9gjAmFJQQ/3tnKvW/YaVlFPYkJwrDc//QhANaPYIwJiSUEP711tnJZpYthuRmkJHn/nHnpQoLY\nFYIxJjSWEPzkZ6f2ynkI/iOMAJIShFP6p9vkNGNMSCwh+MnPTqW6voVWd+9ZGM7tUXZVNRzrUPYZ\nlpthVwjGmJBYQvDjG3paXd97+hH21TTQ4vackBCKB2ZYH4IxJiSWEPwcW76iF/Uj+EYYjRyUedzx\nYbmZHK5vwdXcFotqGWN6IUsIfnrjbOWyCm8/wYi8E68QAHZbP4IxJkhBJQQRmSUiW0Rku4jM6+D5\nVBF50nl+qYiUOMevFZHVfjePiExxnnvbKdP33KBwvrDuyO+FC9yVVbrIzUxhQGbKcceH5XoTgjUb\nGWOCFTAhiEgi8GfgImA8cI2IjG8X9lWgRlVHAfcAdwKo6mOqOkVVpwDXATtVdbXfedf6nlfVijC8\nnh7pjVcIOyrrjxth5OO7QrB9EYwxwQrmCmE6sF1Vd6hqC7AAuLxdzOXAw879Z4ALRETaxVzjnBu3\n0pITyU5N6lUJwTvkNOuE49lpyeRmpthII9NrPL18Lw+u6z3/905GwSSEImCv3+N9zrEOY1S1DagF\nBraL+TzwRLtjf3eai37eQQKJid40F6GmvoXD9S0dJgTwNhtZk5HpDQ67mrntpY28W95GXVNrrKvT\nZ0mgFTFFZDYwS1VvdB5fB5ypqnP9YtY7Mfucx2VOTJXz+EzgAVWd5HdOkaqWi0g28CzwqKo+0sHP\nnwPMASgoKChdsKB7Fxkul4usrI4/OP39eql3L+JvT3AHFR9K2eGO31bj5o6lTXzvjFSmDEo6If6v\na5ooO+Lhrk9kxF3d470uocZbXXoW++jGZt7Y4x0Rd8u0NMYNTAx7XUKN742/x87MnDlzhapODRio\nql3egBnAYr/H84H57WIWAzOc+0lAFU6ycY7dA/yki59xA/CnQHUpLS3V7lqyZElQcd96dIXOvHtJ\n0PGhlB3u+Cc/2qPFtyzUnZWuDuPvXrxZR8x/WVva3BGvSzji46kuocZbXbofu6PSpSPnv6xff2S5\nFt+yUO99Z3tE6hJqfG/7PXYFWK4BPl9VNagmo2XAaBEZLiIpwNXAi+1iXgSud+7PBt5yKoGIJACf\nw6//QESSRCTPuZ8MXAKsD6IuEdebVjwtq3KRkpjAkAHpHT4/LDcDt0cpr2mMcs2MCd5vX91MSlIC\nt10xgYFpwtp9tbGuUp8VMCGot09gLt6rgE3AU6q6QURuE5HLnLAHgYEish34AeA/NPVcYK+q7vA7\nlgosFpG1wGqgHLi/x68mDPKzUzna1EaLO/43lymrqKckL4OkxI7/jMUDvaOPrGPZxKsVu2t4Zf1B\n5pw7gkHZaZTkJLC+3BJCrCQFDgFVXQQsanfsVr/7TcBVnZz7NnBWu2P1QGmIdY0K31yEupb4Twg7\nKl2MKczu9PljQ08P1wP5UaqVMcFRVX61aBP52al87ZwRABT3S2DFtgbqmlrpl5Yc4xr2PTZTuZ28\nbO8Er9rm+E4ILW0edlefuKidv0HZqaQlJ7DbRhqZOLR4w0FW7K7hB586lcxU73fTkn7ejyS7SogN\nSwjt5GelAfF/hbCnuh63RxnRwaQ0HxHxDj21JiMTZ1rdHu58dQujB2VxVemQY8dLcryjiywhxIYl\nhHZ8s5Xj/Qphu7OGUVdXCOBd5M4Sgok3jy/dw86qeuZdNPa4PrB+KUJR/3TrWI4RSwjtDMxKQQSq\nm+I7Ieyo8q5y2tUVAnDsCkEDzDcxJlrqmlr5w5vbOGtELuePPXEJs4lF/ewKIUYsIbSTnJjAyPws\n9tTF9yY5ZRX1FPRLJTtAx1vxwAwaWty9Zva1Ofn99e0yqutb+OnF4+logYJJRTnsOtxgM5ZjwBJC\nByYX5bAr3hNCJ2sYtTdsoK16auLH/iONPPjvnVw+5RQmDcnpMGZikfe4XSVEnyWEDkwsyuFIs1JR\n1xTrqnRIVYNOCMW5vn0R4jshtLk9bKtxx7oaJsJ+99pWVOHmC8d0GjPJEkLMWELogO+by7o4fUNW\nupo52tQWsP8AYMiADBIk/ien3fPGVu5Y2sT2iqOxroqJkI3763hu1T5uOLuEobmdr681MCvVOpZj\nxBJCB8YP7ocQvwlhR2VwI4wAUpISGJyT7kxOi097qxu4/92dAGzYXxfj2phI+fUrm+iXlsxN540K\nGGsdy7FhCaEDmalJDM4S1sXpN5T/7KMc3OqHxQPjey7Cr1/ZRKIIiQKbD9oVQntuT+8fIfavrZW8\nu62Kb58/ipyMwDOQrWM5NiwhdKKkX2LcXiGUVdSTnpzI4H5pQcXHc0L4cMdhFq07yDfPG8ngTGHz\nAbtC8Nl/pJH5z61lzM9eYVVFW6yr021uj3eJiqG56Vw3ozioc6xjOTYsIXRieL8EKo42cygOO5bL\nKl2MyM8kISG4PYWG5mZQ5WrB1RxfHypuj3LbSxsp6p/OnHNHMCQ7gS12hUDl0Wb++6UNnHfX2zyz\nYh8JCcLqit7b4f7cyn1sPniUH396LKlJwe1zYB3LsWEJoRMlOd5fTTw2GwU7wsinONfb+RxvQ0+f\nXr6XjQfqmHfRWNKSExmancD+2iZqG/tmM8GRhhbufHUz5/52CY98sJsrTy9iyc3n8fFReb12BFaz\nW/nda1s5bUgOl0weHPR51rEcG5YQOjEsO4EEib+O5aZWN+VHGoMaYeRzbNXT6vjpWD7a1Mrdr21h\nWsmAYx8UQ7K9b8eth/rWVcLRplb+8MY2zrlzCX99p4wLJxTwxg8+wZ2zJzNkQAZTSwawv16prm+J\ndVVD9tquVg7WNfGTi8d1OAmtK9axHH1BLX/dF6UmCaMGZcVdQthZVY9qcCOMfHyT0+JpLsKf3trO\n4foW/n7D9GMfFEOyvAlh84E6ppXkxrJ6UdHY4uaRD3bx13fKqGlo5dMTCvjBp8acsKS573exYncN\nnxpfEIOads/+I428vKOVT44r4MwR7bdYD2xSUQ6LNxyypbCjyBJCFyYW5fDutipUNeRvN5FybIRR\nCAmhX1oyAzKS42Yuwq6qev723k5mnzHkuNmquWlCdlrSST/SqLnNzRu7W/nRe0uoPNrMJ07N54cX\nnsrkIf07jJ9UlEOSwPJd1b0mIawvr+WrDy8DYP7FY7tVxiTn97G+vJaPjcwLW91M56zJqAuTi3Ko\nPNrMobr4WQeorKIeERieF3yTEcCwgZlx04dwx6JNpCQm8KNZx89WFRHGFfY76TuWb3x4OY9uamF4\nXiZPfX0GD39leqfJACAtOZHhOQks21UdxVp23+INB7nqrx+QKMJPz0oP6cuLP+tYjj5LCF2IxxnL\nZZUuivqnk54S3GgNn+I42Rfhve1VvL7xEDedP4pB2ScOmx1TmM2Wg0dP2tVZ9xxu4N1tVVw2Mpkn\n55zF9OHBNY2NHuAdBt3UGr+dy6rKve+U8Y1HV3BqYTb/nHs2Q7O7/xGTm5liHctRFtRfS0RmicgW\nEdkuIvM6eD5VRJ50nl8qIiXO8RIRaRSR1c7tr37nlIrIOuecP0q8tMn4GT84J+46lr1DTkP/xlU8\nMIPyI420umO3aF+b28NtL21kaG46Xzl7eIcxYwqzOdrcRvmRxijXLjoWrtsPwLlDkkJqhjx1QAKt\nbmXN3iORqlqPtLR5mPfsOn79ymYunjiYJ+ec1WHCD5V1LEdXwIQgIonAn4GLgPHANSIyvl3YV4Ea\nVR0F3APc6fdcmapOcW7f8Dv+f8DXgNHObVb3X0ZkpKckMnpQNuv2xcd/Qo9H2VFZz8gQRhj5DM3N\nwO1R9sfwg/aJZXvZcugoP714HGnJHV/hjBvs7VA9WZuNFq45wOnD+pOXHto351H9vb+v5btrIlGt\nHqltaOX6v33Ek8v38u3zR/H/rjm9079vqGzGcnQF866cDmxX1R2q2gIsAC5vF3M58LBz/xnggq6+\n8YvIYKCfqn6o3raBR4ArQq59FEwsymFdeV1cNGEcrGuisdXdrTbZWK96WtvQyu9f28JZI3L59ITC\nTuNOLfAmhJOxY7ms0sXGA3VcOvmUkM/NShFOLciKu36EXVX1XPmX91ixu4bff+40fnjhmKAnTAbD\nv2PZRF4wCaEI2Ov3eJ9zrMMYVW0DagHfOLPhIrJKRN4RkXP84vcFKDMuTB6SQ5WrmYNxMGO5OyOM\nfIoHeq8qYjXS6A9vbqO2sZVbL5nQZVNJdloyRf3TT8qEsHDNAUTgMyFM0PI3tSSXFbtr4mZto6U7\nDnPFX96jpqGFR288k/86Y0jgk0JkHcvRJYG++YrIbGCWqt7oPL4OOFNV5/rFrHdi9jmPy4AzgaNA\nlqoeFpFS4J/ABOBU4Deq+kkn/hzgFlW9pIOfPweYA1BQUFC6YMGCbr1Ql8tFVlbwH6S++O01bm5f\n2sR3Tk/ljIKOR+l2t+xQ41/f3cpjm1r435np9E/tPJd3VL5Hla+/3sAFw5K4emxqVOteRwY/f6+R\nc4qSuGFiapexWVlZ3LOiiapGD3d8vPMlkqNV92Djg4n9yb8byE4W5p+Z3q26rK1L4761zfzP2eld\ndtZG4/eypjaVv61vYVCG8P3SNAZldFyfcNTlh283MLJ/At+acmKfRKz/pvFal/Zmzpy5QlWnBgxU\n1S5vwAxgsd/j+cD8djGLgRnO/SSgCifZtIt7G5gKDAY2+x2/Brg3UF1KS0u1u5YsWdKt+IbmNh0+\nb6H+bvHmsJcdavzPnl+nE3/xqno8nm6V/8nfva1fe3hZWOoSSvwNf1uqE299VSuPNgVV9m9f3aQj\n57+sTa1tYa9LpOIDxW4+UKfFtyzUR97f2e267Dlcf1wZ3a1LT+Ldbo/edO9iLb5loX7h/g/0SH1L\nxOsy55Fl+onfvhWx8sMRG291aQ9YrgE+X1U1qCajZcBoERkuIinA1cCL7WJeBK537s8G3lJVFZF8\np1MaERmBt/N4h6oeAOpE5Cynr+FLwAtB1CXq0lMSObUgm7VxcMm6o8o7wqi7A7Jiserp2so2lmyp\n5DsXjCYvq/OrA39jCvvR5lHKKuJnqY2eWrh2PwkCsyZ2r7kIYMiAdAr7pcW0Y/mHT69h4Y5Wrpk+\njIe+PD2opax7avKQ/taxHCUBE4J6+wTm4r0K2AQ8paobROQ2EbnMCXsQGCgi24EfAL6hqecCa0Vk\nNd7O5m+oqq9X7FvAA8B2oAx4JUyvKewmFuWwvrw25h3LZRXdG2HkMyw3kz3VDVF7Ha1uD09s9k7A\nuv5jJUGfN9ZZumHLoZNjKWxV5aU1+/nYyDzys4NLih0REUpLBrB8V2wSwv4jjTy/qpxPFyfxqysn\nkpwYnWlMthR29AS1dIWqLgIWtTt2q9/9JuCqDs57Fni2kzKXAxNDqWysTB6SwzMr9nGgtolT+qfH\npA6u5jYO1jV1e9YneK8QGlrcVLlaevTBFKxHP9zNgXrlgc+OIyUp+A+P4XmZJCfKSdOxvGF/HbsO\nN/CNT4zscVnTigfw8toDlB9ppCjK70XflcmMU0KbQ9FT/h3LtoRFZNlM5SD4vqHEcoLajh6MMPIZ\nlhu9VU9dzW387xvbmDAwgQvGDQrp3OTEBEYNyj5p5iK8tHY/SQnCrImdD7cN1lRnobvlMRh+umxn\nNZkpiT2afdwdNmM5eiwhBGH84H4kJkhML1l9Q05HDepBk1EUVz19cfV+ahtbuXJUSre+TY4tzGbz\ngd6fEFSVl9ce4OOj8+ifkdLj8sYWZpOVmhSTZqNlu6o5o3gAiWGcZxCsSU6zrYksSwhBSEtOZPSg\nrJh+Q9lRWU9igjAst/sJYciAdESikxAWLNvDmIJsRvbv3ltsTGE2B+uaqG3o3R2Jq/ceYV9NI5d0\nYzJaR5ISEzh9WP+oT1CrbWzvl+44AAAgAElEQVRly6GjMVuWfNIQm7EcDZYQgjQpxh3LZZUuhuVm\nhNQW315qUiKn5KRHfKTR+vJa1u6r5erpQ7vd1uzrWN58sHd3LL+05gApiQlcOCF8y1ZPK8lly6Gj\nUd1ZbuWeGlRhasmAqP1Mf9axHB2WEII0aUgOh+tb2F8bmxnLPR1h5DMsN4PdhyPbh7Bg2R5SkxK4\n8vTuTz4fW9gPgC29ePc0j0dZtO4AnxiTH9YNXqaWDEDV+yEdLct3VZOUIEwZ2vky3ZFkM5ajwxJC\nkHxvyFjssexRZWdVfY86lH0iPRehoaWNF1bt5+JJg3vUZl7QL5Wc9GQ29eJ+hOW7azhY1xTSXsLB\nmDK0P0kJEtWO5WW7aphQlENGSmz21LKO5eiwhBCkcU7H8rry6K98WtWotLg9YUkIwwZmUOVqwdXc\nFoaanejltQc42tzGNdOH9agcEXH2Rui9TUYL1+4nLTmBT44L7y5nGSlJTCjKYVmUOpab29ys3nuE\nacWxaS7ysY7lyLOEEKS0ZO+M5XXl0f+A2u/y7mEwsgcjjHyKnU7pvRG6Snjioz2MzM9kWhjamscV\nZrP1kAtPnCzmFoo2t4dF6w5wwdgCMlPD/616WvEA1uw9QnNb5DfMWV9eS0ub59iQ11ixjuXIs4QQ\ngknOZh3R7lg+WO/9eSPywnCFEMFlsLccPMrKPUe4etqwsExcGlPYD1cMNstpbHHz13fKONrS/b/z\n0p3VVLlawt5c5DO1JJfmNg/ro/AFxXclEqsOZR/rWI48SwghmFSUQ3V9S9Q/oA7Ue8jNTGFAZs/H\nsfvmIkRictoTH+0hJTGBz5aGZxnkMYWx2RvhmRV7+c0rm/nL6ibaurnD3MK1+8lMSWTm2NAm5QXL\n9+EcjX6E5buqGZGfGfRaVJFiHcuRZwkhBLHarONAvScsI4wActKT6Z+RHPYrhKZWN8+vKufCCQXk\nhiFxwX8SQrT7EZ5bVU52ahKbqj3c/drWkM9vdXt4Zf1BPjm+IGw7h7WXl5XKiLzMiPcjeDzK8t01\nTCuObXMRWMdyNFhCCMHYwmySEiTqb0hvQuh5c5FPcW74Rxq9uv4gtY2tPe5M9peVmsTQ3OhulrOz\nqp5Ve45w0/mjOG9oEn99p4xX1x8IqYz3tldxpKE1bJPROlNaPIAVu6sj2sdSVuniSENrzJuLfKxj\nObIsIYQgLTmR0QXZUV3T6EhDC0dberaGUXvDBmaG/Qrh8Y/2UDwwgxkjBgYODsGYgn5RTQjPr9yH\nCFwxpYhrx6Vw2tD+3Pz0WrZXuIIu46U1B8hOS+LcUyO7ENu0klxqGlrZURV83ULluwKJ1Qzl9qxj\nObIsIYRocpRnLJdVetv6wzHCyKc4N4PyI420drN9vL2yShcf7azm89OGhnU/XfBele2sqo/KaBpV\n5fnV5Zw9Mo/CnDSSE4T/u/YMUpMS+MajK4Iaqtvc5ua1jQf59IRCUpMi01zk4/vWHslmo2W7qsnL\nSqV4YNe710WLdSxHliWEEE0ckkNNQyv7aqLTsexb1C4cI4x8hg3MwO1R9oepc/zJZXtJShBmh6kz\n2d/Ywdm4PRrSN/TuWr67hr3VjcfNsD6lfzr/75rT2VHp4pZn1gb8IvCvrVUcbWqL2Ogif8PzMhmY\nmRLRdY2W7apmWsmAqC533RXrWI4sSwghmhzlN2RZpYsk8S5MFy7Fx5bB7nmzUXObm2dW7OOT4woY\nlH3inrc9dWxNoyjMWH5uZTnpyYknLFP9sVF5/HjWWF5ed4AH3t3ZZRkL1+5nQEYyZ4+K/Lr9IsLU\nCG6Yc6C2kX01jTGff+DPOpYjyxJCiMb4OpajlRAq6inIFJLCuDtVOJfBfn3jIarrW7h6+tAel9WR\nkoGZpCQlRHxNo6ZWNwvX7mfWxMIOJ5J9/dwRzJpQyG9e3cwHZYc7LKOxxc3rGw8xa2Jh1HYTm1aS\ny57qBirqwr/Gli/RTI+jhADWsRxJlhBC5JuxHOk3ZJvbw7Mr9rFsVzWDM8P7ZyrITiMlKSEsVwgL\nPtpLUf90zhmdH4aanSgpMYFR+VkR71h+a3MFR5vaOl2QT0S466rJlAzM4NtPrORA7YnNbUu2VNDQ\n4ubSCI8u8ndsw5wI7LO8fFc1GSmJjBucHfaye8I6liMnqE8aEZklIltEZLuIzOvg+VQRedJ5fqmI\nlDjHPyUiK0RknfPv+X7nvO2Uudq5RWYGTwRMHpLDugh1LLe6PTy1fC8X/P4dfvj0GgbnpHHpyPBu\nZJ6QIGFZ9XT34Xr+vb2Kz08bGtFNU8YOjvyaRs+tLGdQdmqXTT3Zacnce10pjS1uvvXYyhM6uheu\n3U9eVipnhnmkVVcmnNKPtOSEiPQjLNtVwxnDBoT16jQcrGM5cgL+pUUkEfgzcBEwHrhGRMa3C/sq\nUKOqo4B7gDud41XApao6Cbge+Ee7865V1SnOraIHryOqJhblcCTMHcstbR4WfLSH83/3Nj9+Zi3Z\naUncd10pi75zDsX9wj9apTg3o8dNRk8u20uCwOemRqa5yGdsYTaH6pqpqW+JSPmHXc28vaWCK04v\nCpjYRg3K5rezT2PVniPcvnDTseNNbcpbmyu4eFJhVHcUS05M4PSh4e9HqGtqZfPBuriZf+DPOpYj\nJ5jUPx3Yrqo7VLUFWABc3i7mcuBh5/4zwAUiIqq6SlX3O8c3AOkiEtv572EweUj49lhubnPz6Ie7\nmXn328x7bh0DMlJ48PqpvDT341w4oTDswzh9hjnLYHf3Ksd7JbOP88cOojAn/J3J/sY4eyNEqtlo\n4doDtHk06P0bPjN5MHPOHcE/PtzNsyv2AbC6wk1Tqyfik9E6Mq1kABv214Z1BduVu2vwaPzMP/Bn\nHcuRE0xCKAL2+j3e5xzrMEZV24BaoP1182eBlara7Hfs705z0c8lXsa1BWFMYTbJiT2bsdzU6uaR\nD3Zx3l1v87N/ric/O5W/f3kaL9x0NheMK4j4ML/i3AwaWtzUdfNL95ubKqhyNXP1tPDNTO7MuAgv\nYfHcqnLGFmYzbnC/oM/58afHcNaIXH7y/Do27K9l6cE2CvulMTUGS0RPLcnFo7B6T/iWZl++q4bE\nGG6IE4h1LEeGBPqGKCKzgVmqeqPz+DrgTFWd6xez3onZ5zwuc2KqnMcTgBeBC1W1zDlWpKrlIpIN\nPAs8qqqPdPDz5wBzAAoKCkoXLFjQrRfqcrnIygp+LH+g+F+830hWMvxoWnpIZbe4ldfK6nmjPIEj\nzcro/glcPiqFCQMTOk0C4a47wJrKNu5Z0cwPJiuTTwm97N8vb2LvUQ93fyK9yyaSUOreWayq8u23\nGjijIImvTEwNGB9K+QdcHub/u5HPj0nhouHJAeP91TUrv3i/kcQEONLk4YJhyVwzLrgL4HD+TRvb\nlG+90cBlI5O5cnRKWMr+9dJGWtzwi4+dONw5HH/Tnsa/VNbCs9tauessJb9/+MsPNTbS8aGW3d7M\nmTNXqOrUgIGq2uUNmAEs9ns8H5jfLmYxMMO5n4S378CXbIYAW4Gzu/gZNwB/ClSX0tJS7a4lS5aE\nNX7es2t08i8Xq8fjCbrsIw0tesWf/63FtyzUq/76vr63rVI9Hk+P69Kd+O0VR7X4loV6+6OvhVz2\n3up6LZm3UH+3eHNY6hJM7OfvfV8v/9O/u112Z/F3vbpZh89bqIdqG0Oqj8+K3dU66icva/EtC3Xl\n7uoe1aUn8Rf/4V/6hfs/CEvZza1uPfWni/S2lzZ0qy7djQ0l/u0tFVp8y0L9yzNvRKT8UGMjHR9q\n2e0ByzXA56uqBtVktAwYLSLDRSQFuBrvt31/L+LtNAaYDbylqioi/YGXgXmq+p4vWESSRCTPuZ8M\nXAKsD6IucWNiUQ61ja3srQ6uY7mmvoVrH/iQ9eW1fGtKKk99fQYfG5UXsxmgQwakIwKVjaH3ITy1\n3Ntu/rlpke1M9je2sB9bDx0N60JuHo/y/Kpyzh6Vx6B+3esHOWPYAO6+6jQ+XpQU0+aVqcUDWLXn\nSFiWI1m/v5bmNk9YNjmKFF/H8q668Cy/YrwCJgT19gnMxXsVsAl4SlU3iMhtInKZE/YgMFBEtgM/\nAHxDU+cCo4Bb2w0vTQUWi8haYDVQDtwfzhcWacf2WA6iHbPyaDPX3P8hWw+5uO+6qUwvjM2+tP5S\nkxIZ3C+NiobQPmDdHuXp5Xs5d3Q+QwZEb32bsYXZNLS4wzqya9muasqPNPLZM3q25MblU4q4cVJq\nTJd3mFqSS0OLm00Het7P4ttjoTQOlrzuTG5mCiPyM3l9dyv7aiK3R3hfE9QAY1VdpKqnqupIVb3D\nOXarqr7o3G9S1atUdZSqTlfVHc7x21U1U/8ztHSKqlaoar2qlqrqZFWdoKrfVdXIr14WRsc6lgPs\nsXyoromr7/uAXYfr+fsN0yK2YUp3DBuYQUVDaN+w1lW5OVDbxDURmpncGd/eCJvC2LH83MpyMlIS\nuXBCePc8joVwLnS3bFcNw/Myyc+O7wGBf7z6dJralC/cv5SDteGfqd0XxdeMk14kNSmRMYVdz1gu\nP9LI5+79gIO1TTzylTOjsr5NKIpzMznU4OFQXVPQw0/f2ddGXlYqF4R54/hATi3wjTQKz9DTplY3\ni9YdYNbEQjJSYn/F1lODc9IZMiC9xzuoeTzK8l3VMRktFaqJRTncPDWN6voWvvDAh1QebQ58kumS\nJYQemFTUn3X7Op6xvPtwPZ/76wdU17fwjxvPZPrw+Lv8PrUwm6MtcOav3mTyf7/GlX95jx89vYZ7\n3ynjrc2H2HO44bg2+4O1TaypdHPV1CFRW6vHJzM1ieKBGWFLCG9sOsTR5rYeNxfFk2kluSzbVdOj\nGfQ7qlzUNLTG5fyDjozon8jfvzyNA0ea+OIDS6mO0OTFvqL3fzWKoUlFOTzx0R4qG48frlhW6eLa\n+5fS1Obmia+ddWyqfby5fkYxLYd2kHnKSLYdcrG9wsWSLZU87Uy2AkhLTmBEXhajBmVxtKkVj8LV\nUexM9jemIJvNYWoyen5lOYX90jgristMRNrUkgE8v6qciobur4zra3KKxxnKnZlWksuD10/lyw8t\n47oHl/L4jWeRkxHe5V7CbcXuGpraorOnSigsIfTAsZEOtf9ph99y8CjXPrAUUBbMOYuxhcFPdoq2\npMQExg1M5LwZJccdP9LQwvYK17HbtgoXK3bXUH6kkcn5iRQPDN9mPaEYW5jNG5sO0dTq7tFexVWu\nZt7eWsmN5wyP6jITkeb7Vr+1pvvdcd4NcVIYnhebv3F3fWxUHvdeV8qcR1Zw/d8/4h9fnU52WveT\nQnObm8ONkRnB9PrGQ3ztkeWcPiiRT1+gcbPXBFhC6JFTC7NISUxgpzP0bX15Ldc9uJTkxAQe/9oM\nRg0K36Y20dQ/I4WpJbknrIPf0NLGB/9+N0a18i5h4VHYXuHq0VXXS2v24/Yo/3X6ydNcBDAqP4uc\n9GS2Hen+B9nyXTVMLc6Nqw+pYJ03ZhB/+sLpfOuxlXzloWU8/JXpIfcPtbR5eHrFXv701nYq6poo\nGV8d1tFWlUebmffsWrJTk1hV0cY/V5dzZRy9D60PoQd8Hcu7at2s3nuEL9z/IRkpSTz19d6bDLqS\nkZIU02/UY51lmHu6ptHzq8qZcEq/YyOXThYJCcLU4gHdvkI4VNfEnuqGXtVc1N6FEwr5w9Wns2J3\nDTc+vJym1uB+F23OKsPn/+5tfvr8egbnpJGbJnz78VVhW1RRVbnl2bW4mtt45psfY1T/BH7xwgYO\nRWAvi+6yhNBDk4bkUFbr4YsPLKV/RgpPfv0sSnrZ5XZvUTIwk9SkBDb3YKz99oqjrN1XG/RCdr3N\ntOG5HKxXVu4Jffipbwnt3tKh3JnPTB7M7z53Gh/sOMw3Hl3R5X7cbo/yz1XlfOqef/HjZ9aSm5nC\n3788jWe/+TFumpJKlauFHz69JiwTIh//aA9vba5g3kVjGVOYzY2TUmlu8/CT59ZFbY/2QCwh9NCk\nohxa3DAo2zv7OJqTtfqaxARhdEFWj3ZPe25lOQkCl02J/qqk0XD1tKHkpwtf/8eKkMfmL99VQ3py\nIuNPid9+r2BdefoQfn3lJN7eUsncx1edMIPb41FeXnuAWf/7L7735GpSkxK477pSXrjpbGaOGYSI\nUJKTyE8/M463NlfwwL939Kg+Oypd3L5wE+eMzuN6p8+uMDOBH88ay5ubK3h2ZXmPyg8XSwg99JnJ\ng7lkRDILvn5WxJeBNjCmoF+3m4w8qrywej/nnpofkf2f40H/jBS+d0YaDc1tzPlH8E0m4L1COH1Y\n/6gPKY6Uq6cP47bLJ/D6xkN878nVtLk9qCqvbTjIxX98l5seX4kCf7n2DBZ95xwunFB4Qt/Jl2YU\nc9HEQu58dQsrdndvjker28P3n1pDSlICd80+7bgl7b/8sRKml+Ty3y9t6HAXvmg7Of7yMdQvLZnZ\np6actB8w8Wbc4GwqjzZz2BX6JKQt1R7KjzSetM1FPkXZCfzh6tNZV17Lj59ZG1RzxNGmVjYdqDth\nIEFv96UZJfz04nG8vPYA33xsJbd90MScf6yguc3DH66ewuLvncvFkwZ3uu+IiHDn7MkU9U9nbjf7\nE/701nbW7D3CHVdOPOFLY0KC8NvZk2lzK/OejX3TkSUE06uMKez+jOX397eRlZrEheMLw12tuPPJ\n8QXcfOEYXlyzn/97pyxg/Ko9R/AoTD/JEgLA184dwc0XnsrrGw/halXumj2Z179/LpdPCbxDHni/\n9P35C2dwuBv9Cav21PCnJdu58vSiTjdPKsnLZN5FY3lnayVPLd/bYUy0WEIwvYovIYTabNTY4mbZ\nwTYumlhIekr4tySNR986bySXnnYKdy3ewhsbD3UZu3xXtXdDnGHxuSFOT809fzSvfPccfn1OOldN\nHRryPtGThuTws0u8/Qn3vRtcf0J9cxvff3I1hf3S+O/LJ3QZe91ZxZw1Ipf/WbiJ8iOxazqyhGB6\nlfysVAZmpoR8hfD6pkM0ueHKM07u5iJ/IsJvPzuZiafk8N0Fq9jaRWf8sl01jB/cj6zUk3dq0rjB\n/UjqwbDp684q5uJJhdy1eEtQa0bd/vImdlc38LvPnUa/AJPkEhKEu2afhkeVec8G18wXCZYQTK8i\nIowpDG0Ji8YWN08s3UNumnDW8JNnqYpgpKckct+XSslITeLGh5d32Abe5lFW7a3p1fMPokFE+M1n\nvf0J335iVZfrJr2x8RBPfLSHOeeMCHp5lKG5Gcy/eBzvbqviiY9i03RkCcH0OmMKs9l6yIUnwLeo\nKlczv399Kx/7zZt8sOMwFwxL6rTz8GQ2OCede68r5WBtEzc9vvKEIZi76zw0tXp6/fyDaOiXlsxf\nrnX6E55a3WF/QpWrmXnPrWXc4H784MJTQyr/2unDOHvUQO54eSN7q6O/z4MlBNPrjC3MprHVTWUn\nm/vsqHTxk+fXcfZv3uKPb26jtDiXp78xg4uHx/eCZ5F0xrAB/Oq/JvF+2WHueHnTcc9trfEmiN6w\n5HU8mFiUw88vGceSLZXc+6/j+xPUafKpa2rjfz8/hdSk0PqrEhKEOz87GYBbnl0b1h0Cg3HyNhia\nk5ZvwcB9ruO/6a7YXc297+zg9U2HSE5M4LNnFPHVj484tozI27v63tWBv9mlQ9hysI77393JmMJs\nrpk+DIBtNW6KB2Z0exvRvuiLZxXz4Y5q7n5tC1NLBhy7ulqwbC9vbKrgZ58Z1+2lUYYMyOBnl4xn\n/nPreGzpbq5rt/hkJFlCML3OqQXZiMDeox7cHuX1jYe4719lrNxzhJz0ZObOHMWXZpTE/Y5fsTDv\nonFsPeTi5/9cz4i8TKYPz2VbjZtPT7bmolCICL/+7CTW76/l24+vYtF3z+FQvYf/eWsjZ48ayFfO\nHt6j8q+eNpRF6w7wq0Wb+cSp0dtlMagmIxGZJSJbRGS7iMzr4PlUEXnSeX6piJT4PTffOb5FRD4d\nbJnGdCY9JZHi3Aw+OtjGJ3//Dt94dAUVR5v55aXj+WD++fzwwjGWDDqRmCD88ZrTGZabwTcfW8m7\n26o42grTrEM5ZL75CdX1LXz/ydXcu7aZpATh7qtO63FflYi36SgpQbj5mTUB+8vCJWBCEJFE4M/A\nRcB44BoRGd8u7KtAjaqOAu4B7nTOHQ9cDUwAZgF/EZHEIMs0plMTi3LY71Ky05L40xdO5+2bz+OG\ns4efFNthRlpOejL3Xz+VVreHrz2yHOCkm6EcLROLcvj5peN5Z2slO2o93H7lJAbndH+DIn+n9E/n\n55eO56Od1by5py0sZQYSzBXCdGC7qu5Q1RZgAXB5u5jLgYed+88AF4h3UZDLgQWq2qyqO4HtTnnB\nlGlMp269dDy3zkjjhZvO5pLJp4Q80aivG5mfxZ++cAatbg/ZyTDCVujtti+eOYyvnD2cWSXJXHZa\neBdNvKp0CDPH5PP0lhZ2VdWHteyOBPO/qAjwHxS7zznWYYyqtgG1wMAuzg2mTGM6NSg7jRE5ib1y\nI5d48YlT87nn81P43JgU+z32gIhw66XjuXpsSkTK/vV/TebMwUn0S4/8KDkJNCNORGYDs1T1Rufx\ndcCZqjrXL2a9E7PPeVwGnAn8EvhQVR91jj8IvOKc1mWZfmXPAeYAFBQUlC5YsKBbL9TlcpGVFfym\nNaHER7LsSMdbXcITb3WJ/7qEGt+b69LezJkzV6jq1ICBqtrlDZgBLPZ7PB+Y3y5mMTDDuZ8EVAHS\nPtYXF0yZHd1KS0u1u5YsWRKx+EiWHel4q0t44q0u8V+XUON7c13aA5ZrgM9XVQ2qyWgZMFpEhotI\nCt5O4hfbxbwIXO/cnw285VTiReBqZxTScGA08FGQZRpjjImigEMyVLVNRObi/XafCPxNVTeIyG14\ns86LwIPAP0RkO1CN9wMeJ+4pYCPQBtykqm6AjsoM/8szxhgTrKDG6KnqImBRu2O3+t1vAq7q5Nw7\ngDuCKdMYY0zs2Fg9Y4wxgCUEY4wxDksIxhhjAEsIxhhjHAEnpsUTEakEdnfz9Dy88yMiER/JsiMd\nb3UJT7zVJf7rEmp8b65Le8Wqmh8wKpjJCifDjSAnZnQnPpJl96W6x1NdenPdrS4nX91DLbu7N2sy\nMsYYA1gfgjHGGEdfSgj3RTA+kmVHOt7qEp54q0v0y450fG+uS7f0qk5lY4wxkdOXrhCMMcZ0wRJC\nD4hIibMXRDR+1i9F5OYIlPsdEdkkIo+Fscxu/V5E5P1InNOd+oiIK9S6mO4Tkf4i8q1Y16Ovs4Rg\nvgV8SlWvjXVFVPVj0TjHhId4heszpD/e96KJIUsI7YjIP0VkhYhscHZrCyRJRB5zvmU/IyIZAcr/\nkoisFZE1IvKPALE/FZGtIvJvYEwQdf+iiHwkIqtF5F4RSQwQ/1dgBPCKiHw/QOzPRWSLiPxbRJ4I\n4molUUTud36Pr4lIwJ3Hu/OtPNRzRGSEiKwSkWmh/qx25ZSIyGYRecj5Gz0mIp8UkfdEZJuITO/i\nvE3B/m5E5Acist65fS/IOgX1fvR/fwX5N/X9jC0i8giwHhjaSVymiLzsvM/Xi8jnAxT9G2Ck8969\nK8h6rPd7fLOI/LKT2N+IyE1+jzu92haRH4nId5z794jIW8798zu6ihaRac7/5zTnNW8QkYld1Ps2\n/7+jiNwhIt/tIv4bzu9ktYjsFJElncWGRTQmO/SmG5Dr/JuO9w0/sIvYEkCBs53HfwNu7iJ+ArAV\nyPP/WZ3ElgLrgAygH7A9QNnjgJeAZOfxX4AvBfF6d/nq00XMNGA1kAZkA9sC1KUE7/4XU5zHTwFf\nDKIurm78vQKe49RnPd6kugo4rafl+r3GSXi/WK1w/v4CXA78s6e/G7/3QCaQBWwATg/H+zHU91e7\nn+EBzgoQ91ngfr/HOcH8jUL4ux8XD9wM/LKT2NOBd/webwSGdhJ7FvC0c/9dvBt6JQO/AL7eyTm3\nA3cDfybAzo9OvVc69xOAMrr4jPE7L9mpz6Wh/h8J5WZXCCf6joisAT7E++1ndID4var6nnP/UeDj\nXcSej/fNVgWgqtVdxJ4DPK+qDapaR+Ad5S7A+598mYisdh6PCHBOsM4GXlDVJlU9ijfxBLJTVVc7\n91fg/Y8QS/nAC8C1qromTGXuVNV1qurB+2H9pnr/966j69cb7O/m43jfA/Wq6gKew/u+6Eqw78dQ\n31/+dqvqhwFi1gGfEpE7ReQcVa0NofywUtVVwCAROUVETgNqVHVvJ+ErgFIR6Qc0Ax8AU/H+vt7t\n5JzbgE85cb8NUJddwGEROR24EFilqoeDeBl/wLsTZTD/97otqA1y+goROQ/4JN79oRtE5G2834q7\n0n7cbqzG8QrwsKrOj9HPb6/Z774b7xVXLNUCe/B+QG4MU5n+r9Hj99hD1/+3Ivm7icb7sT5gJVS3\nisgZwMXA7SLypqreFsY6tHF8k3eg/6dP493etxB4srMgVW0VkZ3ADcD7wFpgJjAK2NTJaQPxXsEl\nO/UI9Pt5wCm/EO9VXJdE5AagGJgbKLan7ArheDl4vz00iMhYvJePgQwTkRnO/S8A/+4i9i3gKhEZ\nCCAiuV3E/gu4QkTSRSQbuDRAPd4EZovIIF/ZIlIcRP2D8R5wqdNOmgVcEqZyo6kFuBL4koh8IdaV\nCdK7eN8DGSKSibf+nX1L9Qn2/Rjq+yskInIK0KCqjwJ3AWcEOOUo3ubIYB3C+61/oIikEvg9+STe\nrX1n400OXXkXbxPUv5z738D7Tb6z5Hov8HPgMeDOIOr+PDALb1Ps4q4CRaTUqcsXnSvRiLIrhOO9\nCnxDRDYBW/A2GwWyBbhJRP6G95vn/3UWqN49pu8A3hERN9727Bs6iV0pIk8Ca4AKYFlXlVDVjSLy\nM+A18Y78aAVuovurw/qXvUxEXsT7bekQ3uaAmDUBtBP0N2BVrReRS4DXRcSl3v3A45bzHngIbzs2\nwANO80dXgno/hvr+6sdzBXIAAAEMSURBVIZJwF0i4sH7XvxmV8GqetjpkF8PvKKqPwoQ3yrefd0/\nAsqBzQHiNziJr1xVDwSo+7vAT4EPnPdME50kYhH5EtCqqo+LdxDH+yJyvqq+1UVdWpzO4SPq7DHf\nhblALrBERMC7yN2NAc7pNpupbIIiIlmq6nJGrfwLmKOqK2Ncp4F4O+jCdSXUq4lICbBQVTsd5dLF\nub/E25F+d5irZdpxvrCtBK5S1W2xro+/k/oKwfnAeLODpy4IsiPH/Md9IjIebxvpw3GQDE4B3sY7\nusOYXsH5P7QQb4d+XCUDsCsEY4wxDutUNsYYA1hCMMYY47CEYIwxBrCEYIwxxmEJwRhjDGAJwRhj\njOP/A1J/JRS7K+1nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10f9c90d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pi,mixing_ = simulate_markov(M_char,verbose='on')\n",
    "plt.plot(pi);\n",
    "plt.xticks(range(27),chars);\n",
    "plt.grid()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlAAAAJCCAYAAAAP/PnVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XuQpWd9H/jvr7vnqhmN7pExgkFc\nBcKANTiAYRNjO3Y2sGuv7diLKZbspnoxuIiTlSvrtb1LUaYKr4lTSZVvY9ZrCHKKxQ42dtYGB7C5\ng+53JIEldAV0YYRGM5qZ7n72j26xgzyGebqfQ5+DPp+qKZ0zes+3f93nnPd8+3nfc6ZaawEA4OTN\nbfYAAACzRoECAOikQAEAdFKgAAA6KVAAAJ0UKACATgoUAEAnBQoAoJMCBQDQaWHSX2Dr/I62Y8ue\nYXltfn5Y1tccOjw+cwYsn3nK0Lz5+x8emjczdu8cn/nQofGZg1XV8MxZ+JcRasuW4Znt2LHhmYzx\neH2cT0ItTKJyjP9ZfnXpvvtaa2d/s+0mXqB2bNmTl5z3mmF5y6fvGpb1qHbZtcMzZ8GDr3jx0Lw9\n7/zk0LxJqcElfHnf84bmJcncX10xPHO0uW3bh2euHD06PDNtZWjcwrnfOTQvSZbuvGt45iwY/VxM\nkra8PDRvbuvWoXnJhB7nM2DhjLPGh66ML1B/cd/vfOFktnMIDwCgkwIFANBJgQIA6KRAAQB0UqAA\nADopUAAAnRQoAIBOChQAQCcFCgCgkwIFANBJgQIA6KRAAQB0UqAAADopUAAAnRQoAIBOChQAQCcF\nCgCgU3eBqqrXVdVVa39uraoPT2IwAIBp1V2gWmu/3Vp7fpIXJrkzya8/dpuqWqyqy6rqsqPLhweM\nCQAwPRY2cNt/l+RDrbU/fez/aK3tT7I/SU571jlt69sPbeDLfL3D33f7sKyvqRk4ktlWhkee8Wef\nHZq3PDRtctry2EkXPnHd0LwkGX9vj9eWjm32CCdn9PN7+7axeUlqfn545ujH+ST2k3O7dw/PXDn4\n8Ni8o0eH5k3K6MfQ8MdPkqV77xueuZnWVaCq6rVJnpzkZ4dOAwAwA7oLVFVdlOTiJC9rbQLLIgAA\nU249a7I/m+SMJB9eO5H87YNnAgCYat0rUK21fzaJQQAAZsUMnD0NADBdFCgAgE4KFABAJwUKAKCT\nAgUA0EmBAgDopEABAHRSoAAAOilQAACdFCgAgE4KFABAJwUKAKCTAgUA0KmrQFXVm6vq5467/paq\n+hfjxwIAmF69K1C/l+Q1SVJVc0l+Ksm7HrtRVS1W1WVVddnRA4c3PiUAwBRZ6Nm4tXZbVd1fVS9I\n8veSXNlau/8E2+1Psj9JTt31ne3Qvz53yLBJsvCdW4ZlPWrp9jvHBr7oeWPzkuRTV4/PXGnjM2dB\njT1yPXfmGUPzkmTlni8OzxytLS9v9gibYvnWLwzPPPyKfcMzt//Jp8cGtpWxeUmWH/zq8My5F1ww\nNu+W24fmJcnyQw8Nz5yF52MtjH/9bkvHhmeerK4CtebtSV6b5NysrkgBADyurOdX8fcm+eEkL0zy\n/rHjAABMv+4VqNba0ar6cJIDrbXpXzMEABisu0CtnTz+oiQ/MX4cAIDp1/sxBs9O8rkkH2yt3TKZ\nkQAAplvvu/BuSHL+hGYBAJgJPokcAKCTAgUA0EmBAgDopEABAHRSoAAAOilQAACdFCgAgE4KFABA\nJwUKAKCTAgUA0EmBAgDo1PuPCb+wqq6pqu1VdUpVXV9VF05qOACAadT7jwlfWlXvS/IrSXYkeVdr\n7bqJTAYAMKW6CtSaNye5NMkjSd54og2qajHJYpJsXzg1C/c+tO4BH+vwBd8xLOtRW26/c2zgp64e\nmzcpTxr8szxwYGzehKz8V88fmrd0bGVoXpLkni+OzxytJnAGQBv/s5zftWtoXj1x/D5o+598enjm\naHNbt44PncBjaOWK64dnMkZbOrbZIwy1nkfvmUl2JdmdZPuJNmit7W+t7Wut7ds6v2Mj8wEATJ31\nFKjfSfLLSS5J8qtjxwEAmH5dh/Cq6jVJjrXW/qCq5pN8oqpe3lr70GTGAwCYPr0nkb8zyTvXLi8n\n+fuTGAoAYJr5HCgAgE4KFABAJwUKAKCTAgUA0EmBAgDopEABAHRSoAAAOilQAACdFCgAgE4KFABA\nJwUKAKCTAgUA0EmBAgDodFIFqqp+vqreuHb531bVh9Yuv7yqLpnkgAAA0+ZkV6A+muRla5f3JdlV\nVVvW/u4jkxgMAGBaLZzkdpcnuaiqTk1yJMkVWS1SL0vyxsduXFWLSRaTZHt2Zvm2O8ZMm2T7A18Z\nlvWo5eGJ49X8/PDMW35h29C88//7oXETM/fXVwzNe/A1Lx6alyR7PjE8cry2stkTnJTlgweH5s3f\n8+WhebNi5ejRzR4BpspJrUC11o4luTXJa5N8IqsrUt+X5GlJbjzB9vtba/taa/u21PZx0wIATIGe\nk8g/muTirB6y+2iS1yW5srXWJjEYAMC06i1Q35Hkk621LyV5ZO3vAAAeV072HKi01j6YZMtx158x\nkYkAAKacz4ECAOikQAEAdFKgAAA6KVAAAJ0UKACATgoUAEAnBQoAoJMCBQDQSYECAOikQAEAdFKg\nAAA6KVAAAJ0UKACATl0FqqreWlVvOO76m6rq4vFjAQBMr94VqHcn+afHXf+na38HAPC4sdCzcWvt\nyqo6p6qekOTsJF9prd3x2O2qajHJYpJsn9uVuVN3DRk2SdojR4ZlPWpu69ahebVz59C8JGlHxn/f\nT1/8/NC85Rp/RHjhrDOGZy7de9/QvNPfc9XQvCRZGZ443sLTzh+euXL3F4dnzp33hKF5Szd9bmhe\nkszv2TM8c/nBB4fmzV/wjKF5SdK2d70EnZSV7VvGBn7y6rF5j2cTeI1I27y95Xoeve9J8uNJzs3f\nsfrUWtufZH+S7Fk4u617OgCAKbSeAvXuJL+b5Kwk/2DsOAAA0697Pa21dn2S3Unuaq3dM34kAIDp\ntq4D0K21544eBABgVvgcKACATgoUAEAnBQoAoJMCBQDQSYECAOikQAEAdFKgAAA6KVAAAJ0UKACA\nTgoUAEAnBQoAoJMCBQDQSYECAOikQAEAdOoqUFW1t6quO+76xVX1puFTAQBMsYVJhFbVYpLFJNm6\n8/Q89H3PHJa9+wPXD8t61PKhQ0Pzanl5aF6StAlkLjz5vLGBDz00Ni/J0r33Dc984H96ydC8c/7i\ntqF5SbJy1+HhmaMt3/qF4ZmTeJyv3PS5oXkL5+8dmpckS39z2/DM0doX7hyeWXPjD4LMHT06NnDr\n1rF5SVZGz5gkNfhn2VbG5iWp+fnhmW1p/JwnayKH8Fpr+1tr+1pr+7ZsO2USXwIAYNP0Fqilx9xm\n+8BZAABmQm+B+lKSc6rqzKraluQVE5gJAGCqdZ0D1Vo7VlVvTvKZJHcl+exEpgIAmGLdJ5G31v59\nkn8/gVkAAGaCz4ECAOikQAEAdFKgAAA6KVAAAJ0UKACATgoUAEAnBQoAoJMCBQDQSYECAOikQAEA\ndFKgAAA6ratAVdVpVfX60cMAAMyC9a5AnZZEgQIAHpfWW6DemuSpVXVVVf3ayIEAAKbdwjpv978m\nubC19vwT/c+qWkyymCRbd56+zi8BADCd1lugvqHW2v4k+5Nk23nntS++qIZl7/p/l4ZlTUpt3To8\nsx0+PDwz8/PjM2fA2X90w9C8Njfu8T1L2kobH1oz8L6Whw4Oj5zbtn145sqRR8bmHTo0NC9J5nft\nGp5Zzzx/aF77/O1D85IkR4+Oz2wr4zMHa8vLmz3CUDOwtwIAmC7rLVAPJdk9chAAgFmxrgLVWrs/\nycer6jonkQMAjzfrPgeqtfaqkYMAAMwK50ABAHRSoAAAOilQAACdFCgAgE4KFABAJwUKAKCTAgUA\n0EmBAgDopEABAHRSoAAAOilQAACdFCgAgE4KFABAp+4CVVWnVNV/rqqrq+q6qvrJSQwGADCtFtZx\nmx9Ocndr7Z8kSVXteewGVbWYZDFJFk4/fUMDAgBMm/UUqGuT/Juq+tUkf9Za++hjN2it7U+yP0l2\nn/rE9qT3H9vYlMeZO/vMYVmPWrnj7rF5jxwZmpckc9u2D898+DnnDM3bfvtdQ/OSJG1leOTygQND\n8xbO3zs0L0nywFeGR9bClrF5czU0b1LaShsbuGP8c3HlvgeGZ442t3Xr8MzauWN4Zrvji2MDl5bG\n5iVJTeDsmQnsK0ebP/1vrbds2PIE9pUnq/tebK3dnOS7s1qkfqWq/vfhUwEATLHuFaiqekKSB1pr\n76qqA0n++fixAACm13oO4T03ya9V1UqSY0l+ZuxIAADTrbtAtdben+T9E5gFAGAm+BwoAIBOChQA\nQCcFCgCgkwIFANBJgQIA6KRAAQB0UqAAADopUAAAnRQoAIBOChQAQCcFCgCgkwIFANBpSIGqVcoY\nAPC4sO7SU1V7q+qmqnpnkuuSnDduLACA6bWwwds/Pcn/0Fr71PF/WVWLSRaTZNv20zb4JQAApku1\n1tZ3w6q9ST7cWnvKN9ru1Lkz24sWfmhdX+NE2tKxYVkA8I28/+6rh2f+0BOeNzyTcf5L+8PLW2v7\nvtl2Gz1v6eEN3h4AYOY48RsAoJMCBQDQad0nkbfWbkty4bhRAABmgxUoAIBOChQAQCcFCgCgkwIF\nANBJgQIA6KRAAQB0UqAAADopUAAAnRQoAIBOChQAQCcFCgCgkwIFANBJgQIA6NRdoKrqF6vq5qr6\nWFX9x6q6eBKDAQBMq4WejavqoiQ/leT5a7e9IsnlE5gLAGBqdRWoJC9L8t7W2qEkqar3nWijqlpM\nspgk2+d3Z/7cczY05NfZvm1c1pqlz/3N8MxZsPSD+4bmLfzlZUPzJmV+166hebXn1KF5SbJ0193D\nM0eb/64LhmcuX3Pj8MzRFp76lPGhX31oeOTSvfcNzZt/zjOG5iVJu/XO4Zk1N/bMlH/8tBcPzVt1\naAKZ06/m54dntuXl4ZknayLnQLXW9rfW9rXW9m2d2zGJLwEAsGl6C9RHkvxIVe2oqt1JXjmBmQAA\nplrXIbzW2hVV9e4kVyf5cpJLJzIVAMAU6z6E11p7S2vtGa21lya5eQIzAQBMNZ8DBQDQqfddeF+n\ntfamQXMAAMwMK1AAAJ0UKACATgoUAEAnBQoAoJMCBQDQSYECAOikQAEAdFKgAAA6KVAAAJ0UKACA\nTgoUAECnrgJVVXur6rNVdUlV3VhVf1hVOyc1HADANFrPCtQzk/xma+2CJF9N8vqxIwEATLeFddzm\njtbax9cuvyvJG5O87fgNqmoxyWKSbN9yatrpuzc05PGWr7t5WNajan5+eOZotXXr8MwtH7pyaN78\nk88bmpcky3fePT7z4MGheXX48NC8SRn+OP+bO8bmJVk45+zhmStPPGdoXrvl9qF5SdKOHhueOVzV\nZk9wUmrbtrF5Z5w2NC9J6t77h2emtaFxyw8+ODQvSdrK2Bk323pWoB77E/hbP5HW2v7W2r7W2r6t\nC6esbzIAgCm1ngL1pKp68drlVyX52MB5AACm3noK1E1J3lBVNyY5PclvjR0JAGC6reccqKXW2quH\nTwIAMCN8DhQAQKeuFajW2m1JLpzMKAAAs8EKFABAJwUKAKCTAgUA0EmBAgDopEABAHRSoAAAOilQ\nAACdFCgAgE4KFABAJwUKAKCTAgUA0Km7QFXVv6qq69b+/NwkhgIAmGZd/5hwVV2U5J8l+ftJKsmn\nq+qvW2tXTmI4AIBp1FWgkrw0yXtbaw8nSVX9pyQvS/J1BaqqFpMsJsm2Hafl4FP3DBh11a5bdw7L\netTywYND8+Z37RqalyTLFz51eObCPQ8MzVu6/a6heUky913PGp7Zrr5haN7cjh1D85Lxj8lJWDl8\neHhmm8D3XQ8cGJo3t+fUoXlJsvzQQ8MzR1u+7qbhmfN7xr02fM3pYzOXPnfr0LyJaSubPcE3Nbel\nt3J8cytHjw7PPFkTOQeqtba/tbavtbZvYdv4MgEAsJl6C9RHk/xIVe2sqlOS/Oja3wEAPG50rae1\n1q6oqt9P8pm1v3q7858AgMeb7gOSrbVfT/LrE5gFAGAm+BwoAIBOChQAQCcFCgCgkwIFANBJgQIA\n6KRAAQB0UqAAADopUAAAnRQoAIBOChQAQCcFCgCgkwIFANBJgQIA6NRVoKpqb1XdWFW/W1XXV9UH\nqmrHpIYDAJhG61mBenqS32itPSfJgSQ/NnYkAIDptrCO29zaWrtq7fLlSfY+doOqWkyymCTbt+7J\nzrsPrXvAxzp20TOGZT1q4ZPXDc1bPnhwaF6S5FNXD4889E++Z2je1i/cMTQvSVauvmF45mh12p7x\noRN4DLXl5eGZo81t2z48c+XII0Pzlp7+xKF5STI/gftm+cCB4ZnDTeD7Xvrc3wzNm9+9e2hekiw/\n9NDwzJkwP7/ZEwy1nhWoI8ddXs4JSlhrbX9rbV9rbd+WLaesezgAgGnkJHIAgE4KFABAp65zoFpr\ntyW58Ljrbxs9EADAtLMCBQDQSYECAOikQAEAdFKgAAA6KVAAAJ0UKACATgoUAEAnBQoAoJMCBQDQ\nSYECAOikQAEAdFKgAAA6KVAAAJ0UKACATt+0QFXV3qr6bFX9flXdXFWXVNUPVNXHq+qWqvqeb8Wg\nAADT4mRXoJ6W5N8kedban1cleWmSi5P8b4/duKoWq+qyqrrs2LGHR80KADAVFk5yu1tba9cmSVVd\nn+SDrbVWVdcm2fvYjVtr+5PsT5I9pzyh1XIbNG6y9fNfGpb1qKWjR4dnjjZ/2mnDM7d/8dDQvDY/\nPzQvSdry8vDM0druUzZ7hJNTY4/Yz73ggqF5STL3hS8Oz1w58sjQvIU77xualyRLBw4Mz5wFk3h+\n18KWoXntyJGheUmGPxcnoq0Mj5x70ncOz1y56XPDM0/Wyd6Lxz+CVo67vpKTL2EAAN8WZqAGAwBM\nFwUKAKDTNz381lq7LcmFx11/7d/1/wAAHg+sQAEAdFKgAAA6KVAAAJ0UKACATgoUAEAnBQoAoJMC\nBQDQSYECAOikQAEAdFKgAAA6KVAAAJ02VKCq6uCoQQAAZoUVKACATgoUAECniRSoqlqsqsuq6rKj\nS4cm8SUAADbNwiRCW2v7k+xPkl1nntcefObuYdmn3XTbsKxZsnzgwPDM+1747KF5Z12+PDQvSVLT\nv0i6cur2zR7h5LSVsXHX3Dw0L0mWlo4Nz1w45+yhectf/PLQvJkxgefi3Gl7hmcu3fPFsYHz82Pz\nkuHPxVmxcuvtmz3CUNP/6gQAMGUUKACATgoUAECnDRWo1tquUYMAAMwKK1AAAJ0UKACATgoUAEAn\nBQoAoJMCBQDQSYECAOikQAEAdFKgAAA6KVAAAJ0UKACATgoUAECn7gJVVXur6rpJDAMAMAusQAEA\ndNpQgaqq86vqyqp64aiBAACm3boLVFU9M8kfJXlta+3Sx/y/xaq6rKouO/bIwxudEQBgqlRrre8G\nVXuTfDrJV5L8d621G77R9qfWGe1FC/9ovfP9bRc9e1zWmvaZa4dnzoL55zxjaN7y9TcPzZsVczt2\nDM9cOXx4eCZj3PuGlwzPPPs3PjE8c7iawBkfbWV85mD3/cz4+/us35qB+/tx7L+0P7y8tbbvm223\n3mfEg0luT/LSdd4eAGBmLazzdkeT/GiS91fVwdbaHwycCQBgqq23QKW19nBVvSLJX66VqPcNnAsA\nYGp1F6jW2m1JLly7fCCJd+ABAI8rPgcKAKCTAgUA0EmBAgDopEABAHRSoAAAOilQAACdFCgAgE4K\nFABAJwUKAKCTAgUA0EmBAgDotOECVVWfGDEIAMCs2HCBaq29ZMQgAACzYsQK1MERgwAAzIqFSYRW\n1WKSxSTZnp2T+BIAAJtmIgWqtbY/yf4k2bNwdpvbtWtc9tW3DMv6WmaNPZd+4awzhuYlycrBh4dn\n1r1fGZo3t2370LzV0BoeuXL48FTnTUotbBmaN3/OWUPzkqQ9NH5B++GXP3to3rmX3DA0L0mWB++D\nkiRtZWjc/CnjfxleedZThmfOHTk2NO+s3xp/mu8k9pUrRx4ZnjkTJvLcObnNvAsPAKCTAgUA0GlE\ngTrJxS4AgG8PGypQVXVmkgcGzQIAMBPWXaCq6glJPpnkbePGAQCYfut+F15r7e4kzxg4CwDATHAS\nOQBAJwUKAKCTAgUA0EmBAgDopEABAHRSoAAAOilQAACdFCgAgE4KFABAp43+W3ifGDUIAMCs2FCB\naq29ZNQgAACzYqMrUAdHDQIAMCucAwUA0GlhEqFVtZhkMUm2L5yaOuesYdkrZ5wyLOtrPn3N0Lil\ne+8bmjcpt/3rFwzNe9KbHp+nxNW+5w7PbJddOz5z6djQvKW77xmaNynb33fp0Lx6ypOG5iVJDhwY\nnznYyiNHhmdO5HG+bfvQvJqfH5qXJCtHHhmeOQsm8bNsy8vDM0/WRFagWmv7W2v7Wmv7ts7vnMSX\nAADYNA7hAQB0UqAAADpt9GMMdo0aBABgVliBAgDopEABAHRSoAAAOilQAACdFCgAgE4KFABAJwUK\nAKCTAgUA0EmBAgDopEABAHRSoAAAOnUVqKraW1XXTWoYAIBZYAUKAKDTegrUfFX9blVdX1UfqKod\nw6cCAJhi6ylQT0/yG6215yQ5kOTHxo4EADDdFtZxm1tba1etXb48yd7HblBVi0kWk2TL7tNzx3/z\n99Y94GM98f+6YVjWo5aHJ86Gve89MDRvZWjaqpqfH545/7SnjA2840tj85IsDU8cb27H+MXnlcOH\nh2fWXI0NXJqFe2e8+XPOGp658pWx+6Bk/GNofteuoXlJsnzw4PDMWdBW2maPMNR6VqCOHHd5OSco\nYa21/a21fa21ffM7T1n3cAAA08hJ5AAAnRQoAIBOXedAtdZuS3LhcdffNnogAIBpZwUKAKCTAgUA\n0EmBAgDopEABAHRSoAAAOilQAACdFCgAgE4KFABAJwUKAKCTAgUA0EmBAgDopEABAHRSoAAAOi30\n3qCqfjnJq5Pcm+SOJJe31t42ejAAgGnVVaCq6oVJfizJ85JsSXJFkssnMBcAwNTqXYH63iR/0lp7\nJMkjVfWnJ9qoqhaTLCbJ1l2nZ9uDGxvy67J37hgX9qgDB8ZnzoC5Aw8NzavTThualyTLE7hvlm76\n3NjAlzx/bF6SfOnL4zNHW2mbPcFJacvLQ/NWztozNC9Jcvud4zNHa+Pv75XDh4dnzu0Y+xpR27YN\nzUuSHDw4PnMWtJXNnmCoiZwD1Vrb31rb11rbt7D9lEl8CQCATdNboD6e5JVVtb2qdiV5xQRmAgCY\nal2H8Fprl1bV+5Jck+RLSa5NMvAAHQDA9FvPIby3tdaekeSHkjw5TiIHAB5nuj/GIMn+qnp2ku1J\n3tFau2LwTAAAU627QLXWXjWJQQAAZoVPIgcA6KRAAQB0UqAAADopUAAAnRQoAIBOChQAQCcFCgCg\nkwIFANBJgQIA6KRAAQB0WneBqqo3VtWNVXXJyIEAAKbdev4x4Ue9PskPtNbuHDUMAMAsWNcKVFX9\ndpLzk/x5Vf3LsSMBAEy3da1AtdZeV1U/nOT7Wmv3Pfb/V9ViksUk2bL79Czt3NiQX+fI0YFhkzG3\nY8f4zF2nDM9cuff+sXmHHxmalySpCZym11aGxtWnrx2alyRteOJ4tX3b8MxJnJQ5d+7ZQ/OWrrxx\naF6S1Pz88My2vDw27+zTh+YlycKWjRwEObGV+x4Ymrd0/9j9ZJLMbds+PLMtHRueOdwE9ueb+X1P\n5CTy1tr+1tq+1tq+hR3jX/gBADaTd+EBAHRSoAAAOilQAACd1n0GX2tt78A5AABmhhUoAIBOChQA\nQCcFCgCgkwIFANBJgQIA6KRAAQB0UqAAADopUAAAnRQoAIBOChQAQCcFCgCgkwIFANBJgQIA6NRd\noKrq1VX1maq6qqp+p6rmJzEYAMC06ipQVXVBkp9M8r2ttecnWU7y0yfYbrGqLquqy5YOPzxmUgCA\nKbHQuf33J7koyaVVlSQ7knz5sRu11vYn2Z8k2/Y+sR34rmMbHPP/9x3/99FhWZOycvjw8Mx2dPz3\nPfecZ4wNvObGsXlJUtN/lHn+yecNz1z6m9uGZ462cvDg8My2vDw+8867h+YtnH3m0LwkWfryvcMz\nR2s33To+c37883vl0KGheXNbtw7NS5KVCezP01bGZw42t3X8Aas2PPHk9RaoSvKO1tovTGIYAIBZ\n0Fv/P5jkx6vqnCSpqjOq6snjxwIAmF5dBaq1dkOSX0rygaq6JslfJvmOSQwGADCteg/hpbX27iTv\nnsAsAAAzYfrP0AUAmDIKFABAJwUKAKCTAgUA0EmBAgDopEABAHRSoAAAOilQAACdFCgAgE4KFABA\nJwUKAKDThgpUVb2pqi4eNQwAwCywAgUA0Km7QFXVL1bVzVX1sSTPnMBMAABTbaFn46q6KMlPJXn+\n2m2vSHL5CbZbTLKYJPNnnrbxKQEApkhXgUrysiTvba0dSpKqet+JNmqt7U+yP0n2LJzdLvhfbt7Q\nkF/nKU8cl/Woa24cGlcLW4bmJUlbOjY+c8v88Mzh2sr4zBc/b2zezXeOzZuQhbPPGprXDj8yNC9J\nlg8eHJ7ZlpeH5i0/6dyheUmSL987PnOw2rF9eObyg18dnjn/3GcNzWufv31oXpLk6NHxmTNg5djS\nZo8wlHOgAAA69RaojyT5karaUVW7k7xyAjMBAEy1rkN4rbUrqurdSa5O8uUkl05kKgCAKdZ7DlRa\na29J8pYJzAIAMBOcAwUA0EmBAgDopEABAHRSoAAAOilQAACdFCgAgE4KFABAJwUKAKCTAgUA0EmB\nAgDopEABAHRSoAAAOilQAACd1lWgquo1VXVNVV1dVf9h9FAAANNsofcGVfWcJL+U5CWttfuq6owT\nbLOYZDFJttcpGx4SAGCadBeoJC9P8p7W2n1J0lp74LEbtNb2J9mfJHvmzmztkSMbGvJ4c189NCzr\nUTU/PzSvLR0bmjcpX3nO7qF5ey4fGjc5n7x6bN7ZZ43NS5Iaf3R96d77huYtPP2pQ/OSJLccHJ85\n2NzhcfuzRy0PTxyvtm0dnjkmQPGSAAALcElEQVS3ZT0vQd/Y8nU3D82be8EFQ/OSJFfeOD6zrYzP\nHGzh3HOGZy7d88XhmSfLOVAAAJ3WU6A+lOQnqurMJDnRITwAgG9n3eunrbXrq+otSf66qpaTXJnk\ntaMHAwCYVus6AN1ae0eSdwyeBQBgJjgHCgCgkwIFANBJgQIA6KRAAQB0UqAAADopUAAAnRQoAIBO\nChQAQCcFCgCgkwIFANBJgQIA6NRVoKpqb1VdN6lhAABmgRUoAIBO6ylQC1V1SVXdWFV/WFU7h08F\nADDF1lOgnpnkN1trFyT5apLXjx0JAGC6LazjNne01j6+dvldSd6Y5G3Hb1BVi0kWk2T7wqmZe+J3\nbGjI4y19/tZhWY+a27Z9bN6eU4fmJUmOHhseueednxwbWOOPCM9tWc9D9BtbOXp0bN7Bh4fmJUna\nyvDIua1bxwauTGDGneMXtI+9+IKxgR+8fGxeJnDfZPzjPLt3jc1LMrdjx/DM2jV2f96uu2VoXjIj\n9/cE9kErD351eOZmWs8rXvsm19Na299a29da27d1fvwTBABgM62nQD2pql68dvlVST42cB4AgKm3\nngJ1U5I3VNWNSU5P8ltjRwIAmG5dJ5i01m5L8qzJjAIAMBt8DhQAQCcFCgCgkwIFANBJgQIA6KRA\nAQB0UqAAADopUAAAnRQoAIBOChQAQCcFCgCgkwIFANBJgQIA6KRAAQB06i5QVfXHVXV5VV1fVYuT\nGAoAYJotrOM2/2Nr7YGq2pHk0qr6o9ba/aMHAwCYVuspUG+sqh9du3xekqcn+boCtbYytZgk27Mz\nK3fctaEhjzf/nGcMy/qae+4dm7e8MjYvSVsZnzm3Y8fQvNq2bWhekrTDjwzPnD/j9KF5teuUoXlJ\n0u66Z3jmyrGlsYF3f3FsXpLasX145paPXjs0b+7ss4bmJcnKw4eGZ+bo0aFxKxN4TM6dftrwzJXt\np44NXF4em7caOjxx4ewzh+YtfXnw62KSduTI8MzN1FWgquofJvmBJC9urR2qqr9K8rf2eK21/Un2\nJ8meuTPbxscEAJgevedA7UnylbXy9KwkL5rATAAAU623QP1FkoWqujHJW5N8avxIAADTresQXmvt\nSJJ/PKFZAABmgs+BAgDopEABAHRSoAAAOilQAACdFCgAgE4KFABAJwUKAKCTAgUA0EmBAgDopEAB\nAHRSoAAAOilQAACdFCgAgE4KFABAJwUKAKBTtdbGh1YtJllcu3phkusGxp+V5L6BebOSOQszTiJz\nFmacROYszDiJzFmYcRKZszDjJDJnYcZJZM7CjJPInIUZk+TJrbWzv9lGEylQX/cFqi5rre2b1rxZ\nyZyFGSeROQszTiJzFmacROYszDiJzFmYcRKZszDjJDJnYcZJZM7CjD0cwgMA6KRAAQB0+lYUqP1T\nnjcrmbMw4yQyZ2HGSWTOwoyTyJyFGSeROQszTiJzFmacROYszDiJzFmY8aRN/BwoAIBvNwu9N6iq\nM5N88AT/6/tba/dvfCQAgOnWfQivtXZ/a+35J/gzc+WpqvZW1ciPWJi4qnpTVV282XOcSFW9sapu\nrKpLNnuWE5nU/V1VnxidOTJ3gt/3wdGZsF5VdVpVvX6z5+Dxw0nkjPT6JD/YWvvpzR7kW6m19pJZ\nyoWNqlXT9vpxWlb3QfAtMW1PgG+oqv64qi6vquvXPqxzhIWqumRt5eQPq2rnRgOr6jVVdU1VXV1V\n/2FA3i9W1c1V9bEkz9xo3lrmq6vqM1V1VVX9TlXNbzDvt5Ocn+TPq+pfDprxl6vqpqr6WFX9x0Er\nb/NV9btrj6EPVNWOAXNOZCVmErlVdX5VXVlVLxydvV5rK2SfrarfX3ucX1JVP1BVH6+qW6rqezaQ\ne+PI+7uq/lVVXbf25+c2knXcjJ8duQ86fn8x8Hnz6Kw3VdU7s/rhyOdtIOuUqvrPa/vI66rqJweM\n+NYkT13bp/3agLy/tXpbVRdX1Zs2kPfWqnrDcdc3dEShqn6+qt64dvnfVtWH1i6/fL1HAqrqhWuv\nX9vX7qfrq+rC9c64lvnm458vVfWWqvoXG8x83dp9fVVV3VpVH95I3rq01mbmT5Iz1v67I6tP4DM3\nmLc3SUvyvWvXfy/JxRvMfE6Sm5OcdfzMG8i7KMm1SXYmOTXJ5wbMeEGSP02yZe36byZ5zYD757ZH\nv+8BWS9MclWS7Ul2J7llwPe9N8lSkuevXf9/krx6wKwHR3zPk8pd+76vy2r5vjLJ86ZwvqUkz83q\nL3WXrz0XK8l/m+SPp+H+Pu65eEqSXUmuT/KCAd/7sH3QJPYXj5l1JcmLBmT9WJLfPe76nkHzXTfi\ne/27MpNcnORNG8h7QZK/Pu76DUnO20Dei5K8Z+3yR5N8JsmWJP9Hkv95A7m/kuRtSX4jyS8M+jle\nsXZ5Lsnns8HX7+Oyt6x9768ced+fzJ+ZWoFK8saqujrJp7L628/TB2Te0Vr7+NrldyV56QbzXp7V\nB/R9SdJae2CDeS9L8t7W2qHW2leTvG+DeUny/Vnd0V5aVVetXT9/QO5I35vkT1prj7TWHspq4Rvh\n1tbaVWuXL8/qE/vx4Owkf5Lkp1trV2/2MCdwa2vt2tbaSlaLyQfb6t7x2mzsPhp5f780q8/Fh1tr\nB5P8p6w+Pzdq5D5oEvuL432htfapATnXJvnBqvrVqnpZa+3BAZlTr7V2ZZJzquoJVfW8JF9prd2x\ngcjLk1xUVacmOZLkk0n2ZfVx8NEN5L45yQ+uZf2fG8hJkrTWbktyf1W9IMk/SnJlG3fe9L9L8qHW\n2qjXiJPW/S68zVJV/zDJDyR5cWvtUFX9VVZXJzbqsZ/j8Hj4XIdK8o7W2i9s9iCb4Mhxl5ezupr5\nePBgktuz+uJ8wybPciLH3y8rx11fycb2U7Nwf8/SPujhESGttZur6ruT/NdJfqWqPthae/OI7MGW\n8vWnuox4zXlPkh9Pcm6Sd28kqLV2rKpuTfLaJJ9Ick2S70vytCQ3biD6zKyusm7J6vc84n5/e1bn\nPDerK60bVlWvTfLkJD87Iq/XLK1A7clqWz9UVc/K6tLlCE+qqhevXX5Vko9tMO9DSX6iVj/uIVV1\nxgbzPpLkR6pqR1XtTvLKDeYlqx9D8eNVdU6yOmNVPXlA7kgfT/LKtePwu5K8YrMHmnFHk/xoktdU\n1as2e5gZ9dGsPhd3VtUpWf15buS3/EeN3AdNYn8xXFU9Icmh1tq7kvxaku8eEPtQVg/3j/SlrK4Y\nnVlV2zJmP/TuJD+V1RL1ngF5H83qocWPrF1+XVZXeDZSxH8nyS8nuSTJr254wlXvTfLDWT094/0b\nDauqi7L6fb96beX6W25mVqCS/EWS11XVjUluyuphvBFuSvKGqvq9rP5m/lsbCWutXV9Vb0ny11W1\nnNVzTl67gbwrqurdSa5O8uUkl25kvrXMG6rql5J8oFbfSXMsyRuSfGGj2aO01i6tqvdl9TeqL2V1\nyf9xscx/nKErEa21h6vqFUn+sqoOttZGH975trb2XPz9rJ5nkiRvXzsks1HD9kGT2F9MyHOT/FpV\nrWR1//MzGw1srd2/9saD65L8eWvt5wdkHquqN2f1Pr8ryWcHZF6/Vm7vaq3ds9G8rJamX0zyybXn\n+CPZQLGvqtckOdZa+4NafXPRJ6rq5a21D21kyNba0bUTvQ+01pY3krXmZ5OckeTDVZUkl7XW/vmA\n3JPmk8iZWlW1q7V2cO1dSR9Jsthau2Kz5/pWWFvBvKK1Nm0rgwxUVXuT/FlrbUPvcvoG+W/K6sn+\nb5tEPpystV/Wr0jyE621WzZ7nhEmsgJVPq2cMfZX1bOzegz+HY+j8vSEJH+V1XfBAMy0tf34n2X1\nDQ7fFuUpsQIFANBtlk4iBwCYCgoUAEAnBQoAoJMCBQDQSYECAOikQAEAdPr/AKaFhfSGCEcVAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x115a8d590>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure(figsize=(10,10))\n",
    "plt.pcolor(M_char);\n",
    "plt.xticks(range(27),chars);\n",
    "# with graph maybe\n",
    "plt.yticks(range(27),chars);\n",
    "# with graph maybe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the selected char: u\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAD8CAYAAAB3u9PLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzsvXmYXGWV+P85tfae9JJ0kk7SnZAQ\nkhC2zoqoBBXRUUABBRkWfyLjKI8zP8eZwIw6Dl+d0VHH+Y46bqAooAFh1EwME1ACw5aQNISshHSW\nTrqTdKf3vdb3+8e9t1OpVFfdqq6Vfj/PU0/XvXXuW29V3b7nnnPec44opdBoNBqNxpHrCWg0Go0m\nP9AKQaPRaDSAVggajUajMdEKQaPRaDSAVggajUajMdEKQaPRaDSAVggajUajMdEKQaPRaDSAVgga\njUajMXHlegLJUFNToxoaGlI6dmhoiNLS0ozIZ3LsTMvruaRHXs8l/+eSrHwhzyWapqamTqXUtISC\nSqmCeTQ2NqpU2bJlS8bkMzl2puX1XNIjr+eS/3NJVr6Q5xINsEPZuMZql5FGo9FoAB1D0Gg0Go2J\nVggajUajAbRC0Gg0Go2JVggajUajAbRC0Gg0Go2JVggajSYvaGrpYeMhP00tPbmeyqRFKwSNRpNz\nmlp6+MRPt/LkwQC3PrBVK4UcoRWCRqPJOVsPd+ELhlFAIBhm6+GuXE9pUqIVgkajyTkLppeNPXe7\nHKyeX53D2UxetELQaDQ5p7ljcOz5T29fTmN9ZQ5nM3nRCkGj0eQUpRRPNLWObZ83rSyOtCaTaIWg\n0WhyymvHejnSOcS7zzeKcQ6MBnM8o8mLVggajSanPNHUSrHbycdXzAFgYDSQ4xlNXrRC0Gg0OWM0\nEGLjrhN84MIZzJxSBEC/Vgg5QysEjUaTM57Z187AaJAbGmdTXuQGtMsol2iFoNFocsYTTa3MmlLE\nmvnVVBQZDRz7tULIGVohaDSanNDeP8oLB0/z0ctm43BIhIWgXUa5QisEjUaTE377ehthBR+9rA6A\nIrcDp2iXUS7RCkGj0WQdpRRPNrXSWF/JfDPvQEQodmkLIZdohaDRaLLOrtY+DnYMcsNls8/aX+IW\n+ke0hZArtELQaDRZ58nXWvG6HPzZRTPP2l/sEm0h5BBbCkFErhGRAyLSLCL3xnj9XSLymogEReTG\nqNdCIrLTfGyI2D9PRLaZYz4mIp6JfxyNRpPv+IIhNrxxgquXzmBKsfus10pcOoaQSxIqBBFxAj8A\nPgAsAW4RkSVRYseAO4FfxRhiRCl1ifm4NmL/N4HvKqUWAD3Ap1KYv0ajKTCe3d9B73CAGxtnn/Oa\nYSFohZAr7FgIK4FmpdRhpZQfWA9cFymglDqqlNoFhO28qYgIcBXwhLnrF8D1tmet0WgKlidfa6W2\nwssVC2rOeU27jHKLHYVQBxyP2G4199mlSER2iMhWEbEu+tVAr1LKuhVIdkyNRlOAnB7wseXAaT5y\n6WycDjnn9RK3dhnlElFKxRcwYgLXKKXuMrdvA1Yppe6JIfsQsFEp9UTEvjqlVJuIzAeeBd4D9AFb\nTXcRIjIHeEopdWGMMe8G7gaora1tXL9+fUofdHBwkLIy+2V1k5HP5NiZltdzSY+8nos92c1HA/z6\nTT//fEUxs8rOvR99bO8gTx0Xfvb+EhxyrsLI5dwLaS7RrF27tkkptTyhoFIq7gNYA2yO2L4PuG8c\n2YeAG+OM9RBwIyBAJ+CK9R7jPRobG1WqbNmyJWPymRw70/J6LumR13OxJ/v+7z6vrv3+i+PKr/vZ\n06p+3UbVO+xP+1ySlc/n7zFZgB0qwfVVKWXLZbQdWGiuCvIANwMbEhwDgIhUiojXfF4DvAPYZ05w\ni6kcAO4Afm9nTI1GU5jsPdHHm6cGuPGy8b3D1qIjHUfIDQkVgjL8/PcAm4H9wONKqb0icr+IXAsg\nIitEpBW4CfixiOw1D18M7BCRNzAUwDeUUvvM19YBXxCRZoyYwoPp/GAajSa/eKKpFY/TwYcvnjWu\nTInLcBPpOEJucNkRUkptAjZF7ftKxPPtwDlryJRSLwPLxhnzMMYKJo1G8zYnEAqzYecJ3rtkOlNL\nxk85KtYKIafoTGWNRpNxnjtwmq4hf8zcg0hKzFtU7TLKDVohaDSajPNE03Fqyry8a+G0uHIlbsNC\n0F3TcoNWCBqNJqN0D/l59s0OPnLpLFzO+Jcc7TLKLVohaDSajLJhZxuBkOKGBO4igOIxl5FWCLlA\nKwSNRpNRnnytjaWzKrhgRkVCWY9T8Dgd2mWUI7RC0Gg0GaN1IMzutr6EweRIyotc2kLIEVohaDSa\njPFiWxCXQ7g2Tu5BNFoh5A6tEDQaTUZ49UgXW44HuGzuVKrLvLaPqyh20z+iXUa5QCsEjUaTdppa\nerj1gW34QvD68V6aWnpsH2tYCFoh5AKtEDQaTdrZeriLQMiopBwOK7Ye7rJ9bLnXrV1GOUIrBI1G\nk3ZqKwwXkQBul4PV86ttH6tjCLnDVi0jjUajSYaXD3XhdTl4f72TO65eQWN9pe1jy4vc2mWUI7RC\n0Gg0aaWjf5T/fuMEt66q58qK00kpAzAshCF/iGAonDCzWZNe9Let0WjSysNbWwiGFZ98R0NKx1eY\nTREGfdptlG20QtBoNGljNBDika0tvG9xLfXVpSmNUV5kOC50HCH7aIWg0WjSxn+91kbPcIBPXTEv\n5TEqTIWgy1dkH60QNBpNWgiHFT976QgX1lWwcl5VyuOUFxkuI20hZB+tEDQaTVp4/uBpmjsGueuK\n+YhIyuNol1Hu0ApBo9GkhZ+9eITaCi8fXDZzQuNUjFkI2mWUbbRC0Gg0E+bAqQFeONjJ7Wsa8Lgm\ndlmxLARdzyj7aIWg0WgmzIMvHqbY7eTWVXMnPJaOIeQOWwpBRK4RkQMi0iwi98Z4/V0i8pqIBEXk\nxoj9l4jIKyKyV0R2icjHI157SESOiMhO83FJej6SRqPJJp2DPn638wQ3NNYxtcQz4fE8Lgdel4MB\nnYeQdRJmKouIE/gB8D6gFdguIhuUUvsixI4BdwJfjDp8GLhdKXVQRGYBTSKyWSnVa77+t0qpJyb6\nITQaTe54ZGsL/mCYT74j9aWm0ejyFbnBTumKlUCzUuowgIisB64DxhSCUuqo+Vo48kCl1FsRz0+I\nSAcwDehFo9EUPKOBEA+/0sJ7LpjOedPK0jZuRZGLfu0yyjp2XEZ1wPGI7VZzX1KIyErAAxyK2P11\n05X0XRGx30FDo9HkBRt2nqBryD+hRLRYlBfrEti5QJRS8QWMmMA1Sqm7zO3bgFVKqXtiyD4EbIx2\nA4nITOA54A6l1NaIfacwlMRPgENKqftjjHk3cDdAbW1t4/r165P8iAaDg4OUldm/g0lGPpNjZ1pe\nzyU98pNxLkopvvzSCCLC/ZcXxcw9SHXsb20fYSQIX1lTnJG5p1s23+YSzdq1a5uUUssTCiql4j6A\nNcDmiO37gPvGkX0IuDFqXwXwWvT+KJkrMRRJ3Lk0NjaqVNmyZUvG5DM5dqbl9VzSIz8Z5/K/b3Wo\n+nUb1ePbj6V97L98ZIe66tuJj307fI+ZGDsaYIdKcH1VStlyGW0HForIPBHxADcDG+xoJVP+t8Av\nVWyrATFuK64H9tgZU6PR5AcPvniEmjIv114yK+1j665puSGhQlBKBYF7gM3AfuBxpdReEblfRK4F\nEJEVItIK3AT8WET2mod/DHgXcGeM5aWPishuYDdQA3wtrZ9Mo9FkjOaOAZ47cJrb19TjdTnTPr7u\nmpYbbDXIUUptAjZF7ftKxPPtwOwYxz0CPDLOmFclNVONRpM3PPjiUTwuR1oS0WJRXuRmJBAiEArj\n1k1ysob+pjUaTVIM+BX/9VorN1xWR3VZZhYHVhTrAne5QCsEjUaTFFuOB/AFw/x/aUxEi6ZcF7jL\nCVohaDQa2/iCIf50LMi7zp/GwtryjL2PLoGdG7RC0Gg0tvn+s830+RTvPr8mo+9Trrum5QStEDQa\njS2aWnr4/rPNAHxr8wGaWnoy9l4VuuJpTtAKQaPR2OLFg6ex6hoEgmG2Hu7K2Htpl1Fu0ApBo9HY\nYtEMI2YggNvlYPX86oy9l2Uh6CY52UUrBI1GY4tZU426QmtmOXn0rtU01ldm7L3KtIWQE7RC0Gg0\ntuga8gNw1Rx3RpUBgNvpoNjt1MtOs4xWCBqNxhZdg4ZCqPCeW9U0E+jyFdlHKwSNJss0tfSw8ZA/\no6t0MkH3kA+Ack8WFYJPWwjZRCsEjSaLNLX08ImfbuXJgwFufWBrQSmFrkE/HpeDovTXsouJ0UZT\nWwjZRCsEjSaLbD3chS8YRpH5pZvppmvIT3WpJ2YjnExQUezWq4yyjFYIGk0WWT2vaux5ppduppuu\nQR/VZZ6svZ+OIWQfrRA0mixy3nSjDWKpi4wv3Uw33UN+qkqz1/q8oshFv1YIWUUrBI0mi7T2jAAQ\nUBSUMgDDZVRTmk0Lwa2XnWYZrRA0mizS1msoBH8IhnyFdffbNeinKpsKwevCFwzjD4az9p6THa0Q\nNJoscsJUCACdg74cziQ5hv1BRgIhqrIcQwDdEyGbaIWg0WSRtp7CVAhWUlpNNmMIxWY9Ix1HyBpa\nIWg0WaStdwSHuWrz9IA/t5NJgm6zbEVWXUa6a1rW0QpBo8kibb0jLJxuVA0tKAvBzFLO9rJT0AXu\nsokthSAi14jIARFpFpF7Y7z+LhF5TUSCInJj1Gt3iMhB83FHxP5GEdltjvkfkq1sF40mh5zoHWHZ\n7ClAgSkE02VUnUWXkY4hZJ+ECkFEnMAPgA8AS4BbRGRJlNgx4E7gV1HHVgH/CKwCVgL/KCLWWrsf\nAp8GFpqPa1L+FBpNATAaCNE56KehuoRSd2EpBMtllE0LYawngrYQsoYdC2El0KyUOqyU8gPrgesi\nBZRSR5VSu4Do9WHvB55RSnUrpXqAZ4BrRGQmUKGU2qqUUsAvgesn+mE0mnSRiQJ01pLTWVOLmeIR\nOgsohtA15MfrclDiyVIhI3QbzVxgRyHUAccjtlvNfXYY79g683kqY2o0GWXLgQ5u+tHLaS9AZ60w\nqptaTIVXCspC6BrMbh0jONMkR9czyh5i3KDHETBiAtcope4yt28DViml7okh+xCwUSn1hLn9RaBI\nKfU1c/vLwAjwHPANpdR7zf3vBNYppT4UY8y7gbsBamtrG9evX5/SBx0cHKSsrCwj8pkcO9Pyei7n\n8pNdo7x8IgQYd0wfXejmQ+fFd5XYGfv54wF+vtfPt99dzK/2DtM27OCb7ypJ69yTlbUr/287Run3\nK756eXFW5/KZZ4Z412wXn1gcO3YxGc7HVMaOZu3atU1KqeUJBZVScR/AGmBzxPZ9wH3jyD4E3Bix\nfQvw44jtH5v7ZgJvjic33qOxsVGlypYtWzImn8mxMy2v53Iu/7Jpn6pft1HVr9uoFn1pk9pxtDst\nY39785tq3r0bVSAYUnf95/+opV/5H1vzsTt+KrJ25T/8vRfU7Q9uy/pcVn39j+qLj+/M2Pjpks23\nuUQD7FAJrq9KKVsuo+3AQhGZJyIe4GZgg03FtBm4WkQqzWDy1aZyOQn0i8hqc3XR7cDvbY6p0WSU\nYrfhqphb7khrAbq2nhFmVBThcjqo8AiDviCjgVBaxs40XYP+rAaULXTF0+ySUCEopYLAPRgX9/3A\n40qpvSJyv4hcCyAiK0SkFbgJ+LGI7DWP7Qb+D4ZS2Q7cb+4D+CzwANAMHAKeSusn02hSpH1gFACP\nM70F6Fp7R6irNBrVW20oTw/kfxxBKUXXkI/qLCalWeiuadnFZUdIKbUJ2BS17ysRz7cDs8c59mfA\nz2Ls3wFcmMxkNZps0NFvXKQ7huPH15LlRO8Iy00FM8VsQ9k56GNOlb04Qq4Y9ocYDYSpLsteDoJF\nRbF7bMmrJvPoTGWNJooO00Lo96u0VSQNhRWn+kbPsRA6B/P/YpeLshUW5UW6a1o20QpBo4mio99H\nudcwnlu6htMyZnv/KMGwom6qYQ1EWgj5TpeVlJYrl5GOIWQNrRA0mghCYcXpQR+NDYZr51j3UFrG\nPZOUVgRAuaUQCiCG0DVo1THKvstIK4TsohWCRhNB15CPUFixosHofZwuC8HqgzDbdBl5nEJ5kUtb\nCAmoKHLjD4ULZjVWoaMVgkYTgRVQPm9aGWVuaOlOj0KwWmfOmlo8tm9ambcgYghjhe1ytOwUdPmK\nbKEVgkYTgRVQrq3wMr3EQUtX+lxGVaUeSjxnFvbVlHk5XQAWQveQjyK346y5Z4sK3RMhq2iFoNFE\nYFkI0yuKmF4iaXMZtfWMjMUPLGrKPYXhMhr0Z7XsdSSWhaArnmYHrRA0mgjaTYUwrczLtBIHJ3pH\n0tLk/UTvCHUR7iIwLISCCCoP5SZLGXTXtGyjFYJGE0HHwChVpR48Lge1JUJYQWvPxKwEpRRtvSNj\nS04tasq89I8G8QXzO2DaPeTPSUAZdAwh22iFoNFE0N7vY3q54R6ZXmL8e0w0sNw7HGDYHxpLSrOo\nMZdxduV5YLlr0EdVjl1G2kLIDlohaDQRnB4YZXqF4eufXmzkChybYBzBykGoi44hmG6YfI4jGHWM\n8sFlpC2EbKAVgkYTQXu/j1rTQpjiFYrdzgkHls8ohCiXkfk++awQhvwhfMFw7lxGXhciOqicLbRC\n0GhMwmaW8vQK40ItItRXl0x46elYp7Qol9E002WUz600uwdzV8cIwOEQyjwuXc8oS2iFoNGYdA35\nCYUVtRVnXDtzq0omHENo6x2h2O2kssR91n4rhpDPuQidQ8bcanJQtsJCl6/IHlohaDQmVlKaFVQG\nqK8u4Vj3MOFw6qWwrRyE6H7ExR4npR5nXruMcm0hgBFH0EHl7KAVgkZjEpmUZjG3uhR/MDzWNCcV\nTvSNUFcZu+dBTXl+l6+wSl/nKqgM2kLIJlohaDQmsSyEhmrjQn60M3W3UVvPuUlpFvmenGa5jHKV\nqQy6a1o20QpBozEZy1KOdBlVlQKpl8Ee8YfoGvKPVTmNpqYsv8tXdA/6KXY7KfY4czaHimI3/SPa\nQsgGWiFoNCYdA6NUlrjxus5c/GZNLcLlSL2mUXQfhGhqyrx5rRBymYNgYbiMtIWQDbRC0GhM2vt9\nZ60wAnA5HdRVFqe80ujEODkIFjVlXnqGAwRCE6+XlAm6cli2wsIIKgdRKr09rjXnohWCRmPSMeA7\ny11kUV9dmnIuwlhS2nguI/P98rWRfNegLyed0iIpL3IRDCtGA/mpNN9O2FIIInKNiBwQkWYRuTfG\n614Recx8fZuINJj7bxWRnRGPsIhcYr72nDmm9dr0dH4wjSZZOvpHz7EQAOqrSmjpGk7pDrWtZwSn\nQ8ayn6OZZrpjTudpYLl7yJ/TJaegK55mk4QKQUScwA+ADwBLgFtEZEmU2KeAHqXUAuC7wDcBlFKP\nKqUuUUpdAtwGHFFK7Yw47lbrdaVURxo+j0aTEuGw4vSA76wVRhb11SUMjAbpHU7+gtTWO8KMiiJc\nztj/albCVz7GEXJdx8iiQvdEyBp2LISVQLNS6rBSyg+sB66LkrkO+IX5/AngPRKdhQO3mMdqNHlH\n97CfYFSWskV9tbHSKJU4QluMPgiRnFEI+ecyGvQF8eewjpGF1TWtX1sIGceOQqgDjkdst5r7Ysoo\npYJAH1AdJfNx4NdR+35uuou+HEOBaDRZYywpbRwLAUgpjtDWMzJu/ADyu8CdFdfIVelrC90TIXtI\nIr+oiNwIXKOUusvcvg1YpZS6J0JmjynTam4fMmU6ze1VwANKqWURx9QppdpEpBx4EnhEKfXLGO9/\nN3A3QG1tbeP69akZGYODg5SVlWVEPpNjZ1pez8Vg1+kg/9bk40urilhQ6TxL3h9S3P3MMB9Z4Oa6\nBbHvlmONHQorPv3MMH82z80N53tiyiul+Itnhlk718UtF4x/4c3F99jcE+Jr20b5QqOXi6a5Espn\nai6tA2G+9NIIn73Yy8qZroTy6ZpPIf9vRLN27dompdTyhIJKqbgPYA2wOWL7PuC+KJnNwBrzuQvo\nxFQ25r7vAn8f5z3uBL6faC6NjY0qVbZs2ZIx+UyOnWl5PReDx149purXbVTHuoZiyq/8+jPqC4/t\nTGrs1p5hVb9uo3p0a0tc+Xd840/qr379Wspzn4hsPPmn955S9es2qjeO9+R0Lm3m9/irbfG/x3TP\np5D/N6IBdqgE11ellC2X0XZgoYjMExEPcDOwIUpmA3CH+fxG4FlzEoiIA/gYEfEDEXGJSI353A18\nCNhjYy4aTUZo7zfLVlTEvkuvry5NOlv5RIIlpxZGclr+xRC6TDdWPiw7Bb3KKBskVAjKiAncg2EF\n7AceV0rtFZH7ReRaU+xBoFpEmoEvAJFLU98FHFdKHY7Y5wU2i8guYCfQBvx0wp9Go0mRjgEfU6Oy\nlCOpryrhaJLZymN9EOIElSF/s5W7rMJ2OQ4ql3pcOETHELKBK7EIKKU2AZui9n0l4vkocNM4xz4H\nrI7aNwQ0JjlXjSZjtPePUlseu7wEGIHl0wM+hv1BSjy2/m0iOqXFVwjTyj3sPN5jf7JZonvIT6nH\nSZE7d3WMwGyS49VNcrKBzlTWaDAshPHcRWCUwQY4lsTS09aeEapKPQkLw9WUeek2m/PkE12DPqpy\nnINgYZWv0GQWrRA0Gows5elxLISGsaWn9hXCiQQ5CBY1ZV7CKv/KVxh1jHIbP7AoL3LpxLQsoBWC\nZtIT3Us5FlYZ7GRyERIlpVnka7Zy12DuC9tZVOiuaVlBKwTNpKdn2E8gpMatNwQwpcTNlGK3bQtB\nKZUwKc2ixnTL5JtCyIc6Rha6a1p20ApBM+npGDi3dWYsrP7KdugZDjASCNmzEPIwW1kpRddQ7iud\nWlQUu3XXtCygFYJm0mPlINTGcRmBkYtw1KbL6MRYY5wkXEYD+RNDGPAFCYRU3riMyotcumtaFtAK\nQTPpGbMQ4gSVwchFONE7aquZTauZgzBe68xIKopceJyOvLIQus1EuVxXOrUoL3Ix6NNNcjKNVgia\nSU+HaSHEao4TydzqEkJhNZZwFg+7OQgAIkJNmYfTeaQQuoaMueRPDMFNKKwY9odyPZW3NVohaCY9\nHQM+phS7EyZg1VeZS09txBHaekYo8TiZWuK2NYea8vwqX9FlzqUmT2IIuuJpdtAKQTPpae8fTRg/\nAGiosb/09ETvCLOmFmO3qntNmZfOPOqa1jVW+jp/LATQ9YwyjVYImklPx4AvYfwAjF4JRW6HraWn\ndnMQLGrKPPkVQ8gzhaC7pmUHrRA0k56O/vhJaRYiwlyzv3Ii2nrt5SBY1JR56RryE86T8hWdgz7K\nvK6c1zGyKNdd07KCVgiaSY1Sio6B+GUrIplblbgM9rA/SPeQP0kLwUsorOjNkwJu+ZSUBmcsBB1D\nyCxaIWgmNT3DASNL2YaFAEZNo5au4bh38id6jVVLSSmEPEtO6x7y582SU9AxhGyhFYJmUtMxYDbG\nsWkh1FeX4AuGx3IXYtFmszFOJGPlK/IksNyZR3WMQK8yyhZaIWgmNe39xgXYroVglcGOt9LIbmOc\nSKaZyzvzJRehe8iXN5VOAUo8TpwO0RZChtEKQTOpsZLSbFsINnIR2nqHcTqE2gS1kSI5U/E097kI\nSikjhpBHLiMR0QXusoBWCJpJzZnCdvbuhusqi3E6JK6FcKJ3lBkVRTgd9nIQAKYUu3E5JC9iCP2j\n+VXHyMKoZ6QthEyiFYJmUtPRP0pFkf3llW6ng7qpxXGXntotex2JwyFUl3nyIobQZSqlfAoqA5R7\ndde0TKMVgsY2TS09bDzkp6kl//r/pkp7vy8p1w4kLoPd1jvC7CTiBxY1Zd68sBDOJKXlTwwBdE+E\nbKAVgsYWTS093PSjl3niYIBbH9j6tlEKHQOjtt1FFvGS04KhMKf6R5O2EMBSCLmPIVhlK/LPZeTW\niWkZxpZCEJFrROSAiDSLyL0xXveKyGPm69tEpMHc3yAiIyKy03z8KOKYRhHZbR7zH2K36IsmJ7xy\nqBNr6X0gGGbr4a7cTihNtPf7qLUZULZoqC6lbyRA7/C5F+/2AR+hsLLVByGafLEQuvKs9LVFhbYQ\nMk5ChSAiTuAHwAeAJcAtIrIkSuxTQI9SagHwXeCbEa8dUkpdYj4+E7H/h8CngYXm45rUP4Ym01iF\n3QBcTger51fncDbpQSnF6QEf05K1EKrNlUYxrIRUlpxa1JR76Br057zmf3eelb62qCjWfZUzjR0L\nYSXQrJQ6rJTyA+uB66JkrgN+YT5/AnhPvDt+EZkJVCiltirj7P8lcH3Ss9dkjcjE3HXXLKKxvjJ3\nk0kTvcMB/KFw0hZCffX4S0/beo19qbiMppV58YfCOe8M1jnop9zrwuvKjzpGFuVFLgZ8wbyp9/R2\nxI5CqAOOR2y3mvtiyiilgkAfYN1CzhOR10XkeRF5Z4R8a4IxNXnE7tZeXOYyyuDb5B8y2SWnFnPN\nXIRjMZaeTshCyJPktHwrW2FRXuRCKRjya7dRppBE5qmI3Ahco5S6y9y+DVillLonQmaPKdNqbh8C\nVgEDQJlSqktEGoHfAUuB84FvKKXea8q/E1inlPpQjPe/G7gboLa2tnH9+vUpfdDBwUHKysoyIp/J\nsTMtb1f2G6+O4A9B10iIJTUu/uIie3fV+fw97ukM8u0dPv5+VRHnV557Nxxv/L/eMsyFNU7uWuY9\nS/ahPT6aOoJ876rSmMfFG3tvZ4hv7Rjl3pVFXFDlTCifzNjJyP/rduO3/tLq2EotV7/pc8cDPLTX\nz3feXUx1sSPt409ENt/mEs3atWublFLLEwoqpeI+gDXA5ojt+4D7omQ2A2vM5y6gE1PZRMk9BywH\nZgJvRuy/Bfhxork0NjaqVNh2uFP95Y/+R+042m37mC1btmRENt/k7ciGQmG19Cv/o770293qw996\nSr3/u8/nbC7plP/NjuOqft1GdbRzMOnxb/zhS+qmH758juxtD25TH/7eC0nPRSml9p/sU/XrNqr/\nfqMtqblMRDaW/Pu/+7z61EPb0zJ+On/T/36jTdWv26jePNmfkfEnIptvc4kG2KESXF+VUrZcRtuB\nhSIyT0Q8wM3AhiiZDcAd5vP9TeaoAAAgAElEQVQbgWeVUkpEpplBaURkPkbw+LBS6iTQLyKrzVjD\n7cDvbcwlaZpaerj5J1vZdCT4tloumU2OdA0x6AuybPYU5pQ7aO4YxBcs/N627UmWrYhkblUpLTHK\nYLf1DKfkLoKI8hU5Tk7rHvKPFdvLJyp0xdOMk1AhKCMmcA+GFbAfeFwptVdE7heRa02xB4FqEWkG\nvgBYS1PfBewSkZ0YwebPKKW6zdc+CzwANAOHgKfS9JnOYuvhLiyvmD8PlksWYnLXnrY+AC6aPYU5\nFQ6CYcXB9sEcz2rinB7wUV7kotiTfPC0obqE9n4fIxFN35VSSXdKi6SyxINDclvPSFl1jPJshRGc\nqXiqcxEyh8uOkFJqE7Apat9XIp6PAjfFOO5J4MlxxtwBXJjMZFNh9fxqvG4Ho4EwSkFj/dRMv+W4\nNLX08ImfbsUfDLPx6FYevWt1QazW2dXaR5HbwYJpZcwtN+4h9p/s58K6KTme2cQweiknbx3AmaWn\nx7qHWTSjHDB6K4wGwinlIAA4HUJVaW5zEfpHggTDiuqy/MpShsieCDqonCne9pnKjfWVPHrXaq6Y\n5UQB+04M5GwuWw934QuGUeSHtWKX3W19LJlZgcvpYEapUOR2sP9k7r7HdGH0Uk7twlcfowz22Aqj\nFJacWuS6t3KnmYOQb1nKoPsqZ4O3vUIAQyl8apmXKxbU8L1nD+bM5Fw6q2LsuYgURHJXOKzY29bH\nMtMacIiwqLac/Sf7czyziTMRC8Eqgx1Z02gsByFFCwFgWrmX0zl0GVl1jPJz2amOIWSaSaEQwLgA\n3/uBC+gZDvCj5w7lZA5WELOqSBBgzgTuJLPF4c4hhvwhls0+42pbPLOC/af6c55ROxGUUhOyEKaW\nuKkocnE0wkJoNS2E2ROyELw5DSpblU7zMYZQ5Hbgcoh2GWWQSaMQAC6sm8L1l8ziwRePcLJvJOvv\n/4fdp5hTVcy6FUWEleKBF49kfQ7JsrutF2DMQgBDIfQOBzhlKrhCpG8kgD8YZnqKFoKIUF9delb5\nihO9o5R6nEwpdqc8L8tllKqyneiihTOF7fIvhiAiunxFhplUCgHgb65ehFLwb0+/ldX37R3283Jz\nJx9cNpPaUgcfvngWj2xtiVkgLZ/Y3dpPsdvJedPOJFotnmm4vgrZbTSWpZyihQBGYDnaZTRrajET\nqdNYU+bFFwwz6Ev+LthatDCRirRWYbt8tBDAapKjLYRMMekUwpyqEu64vJ4nXmvlzVPZu6A9vbed\nYFjxZ8tmAvDZKxcw7A/x0MtHszaHVNjd1suSWUZA2eKCmcaqmn0nClchnMlBSF0h1FeV0NYzQiAU\nBow+CBMJKMPEWmluPdyFP2jMJdVFC91DfsqLXHhc+XlpMHoiaAshU+Tnr55hPrd2AeVeF9946s2s\nvefG3SeZU1U85npZNKOc9y2p5ecvHU3pbjAbhMKKvSf6z3IXgZEgNKequKBXGnX0GxZCqkFlMMpg\nB8OKE72G+7GtJ/UcBIuackshJB9HiFyk4Ehx0ULXkH9MKeUjumtaZpmUCmFqiYd7rlrAcwdO83Jz\nZ8bfL9JdFOlO+OyV59E3EuDX245lfA6pcPj0IMP+0DkKAWDxjIqCdhm1D5gWQpKF7SKJLIPtCyp6\nhgNpsBAMV00qgeUF08uwIg8XzZ6SUo5L16Avb91FoLumZZpJqRAAbl/TQN3UYv7lqTczXk432l1k\ncencSt6xoJqfvnCY0UD+lYLY1XomQzmaxTMrONI1xHCBVp7s6PdR7nVR4rGVmxmTyDLYXaPGOTRR\nC2FaWeoWQlOLUQRgRqlwsGMwpfO6e8iflzkIFuVFOqicSSatQihyO/mbq89nd1sf/73rREbf6w+7\nTzK7sjjmnfbnrlxAx4CPJ19rjXFk8qSzNMbutj5KPE7mTzu3yuLimRUoBQdOFabbqGNgNOnGONHU\nlhfhcTk41jVE54jhu5+oQqgq9SBCSrkI2w5343E6eH+94VZ5qyP536ZzMD9LX1tUFGsLIZNMWoUA\ncP0ldSyeWcG3Nh/IWLG23mE/LzV38mcXzYy5+mTNedVcMmcqP3r+EEEzOJkq2492c8tPtvJkmvoe\n727rY+msCpyOc+e9ZGylUYEqhBRaZ0bjcAj1VSUc7Rqma8S0ECboMnI5HVSWpJatvPVINxfPmcLS\nGqM20/ajyf3+4bCiZzg/6xhZlBe5GfAFCb1NenLkG5NaITgcwt9/8AJae0Z4+JWWjLzHeO4iCxHh\nc2sXcLx7hI27Tk7ovb751Jv4Q0ZpjIn2PQ6Gwuw70c+yuti1n2ZXFlPmdRVsHKF9YHRC8QOL+uoS\njnUZLiOXQ1KqnBpNTZkn6RjCkC/InrY+Vs2rZlqxML3cy46j3YkPjKBvJEAorPIyB8HCKl+Rrwsx\nCp1JrRAA3rlwGu9cWMP3tzTTN5J+32Q8d5HFey6YzqLacv7zueaU4xkPvXSEHREWgds1sb7Hh04P\nMRIIsWx2RczXHQ5h8czCLGGhlDIshAmsMLKYW1XKse5hOkfCzJhSFNOaSpaasuQL3DW19BAKK1bO\nq0JEWNFQxY4kLYSuPC5bYWFVPNVxhMww6RUCwL0fuIC+kQA/THNJizF30bLY7iILh0P47NrzeKt9\nkD/ub0/6fZ7ee4p/2riP9y2p5UMXzUSAn96+fEKVVHebJa/HsxDALGFxsr/getz2jwTxBcMTykGw\nqK8uYSQQ4nBfeMLxAwtDISQXQ9h2pAunQ8Z+8+UNlbT1jtDWaz8jvzuPs5QtdMXTzKIVArB01hQ+\nckkdP3vpSFL/QIl4ep/pLrootrsokj9bNpO5VSX84LlDSZUteON4L59f/zoX1U3hP26+lDsvb0Bh\nNJCfCLtbeyn1OJlfM34ryMUzKxjyhzjec26z+XymY2zJ6cQtBGulUcewmnD8wCIVC2Hb4W6W1U2h\n1GvcQa9oqAJIym2Uz3WMLM5YCFohZAKtEEy+cPX5QHpLWvxhV2J3kYXL6eAz7z6PN4738vIhe77/\n493DfOoX25lW7uWBO1ZQ7HFy6dxKKjyGMpoIu9v6WFo3BUccF0ihlrBo75942QoLqww2wOx0WQjl\nHob9IdtLekcDId5o7WXV/KqxfRfMKKfM60rKbWS5jPKxW5qF7pqWWbRCMJldWcInL2/gv15vTUtJ\nBrvuokhuaKxjermXH2xpTijbNxzgzp+/ij8Y5ud3rmSaeXFzOoRLprvY8mZHyiungqEw+06em6Ec\nzaLachwC+wpspZFlIaQjhlA3tRhLZ6baGCeaM6007bmNXjvWQyCkWDXvjEJwOR1cOncq25OyEIz3\nqywAC0F3TcsMWiFE8NkrF1BR5Obvf7t7wmv5LXfRB8dZXRQLr8vJ3e+az8uHunjt2Pjv7QuGuPvh\nHRzvHuEnty9nwfSz8wQum+5k0Bdk6+HkVplYNJ8eZDQQjpmQFkmxx0lDTemkthA8LseYIkiXy8hK\nTjtt02207XA3DoHlDVVn7V/RUMWB9gHbiyW6h3xMKXbjdubvZUHHEDJL/v7yOWBKiZvrL5nFzuO9\nE17Lv8lcXZToohrNLSvnMrXEzX9uiR3gVkqx7oldbDvSzbduuijmSqIl1U5KPE6e3nsqpblbGcp2\nWmRageVComNglDKva8zfPlGqSow76nRVrq1JMlv51SPdLJlVMeZOsVjeUIlSxL25iKQzz7OUQccQ\nMo1WCFFYAbWJrOXvHfbz4sHk3EUWpV4Xn7x8Hn/c3x6zGuu/PfMWv9t5gr99/yKuu6Qu5hgep3Dl\nomk8s689pRVAe9r6KPO6mFc9fkDZYsnMClp7RgrKhO/oT70xTjRNLT3sNRXi3/5mV1oyxGvKzXpG\nNhSCLxjitWM9rGw498bgkjlTcTnEdmC5ezC/k9LAqDDgcToK6nwrJLRCiOKKhdPGSv8qYPW8qvgH\nxCAVd1Ekd1xeT6nHec4y2Me3H+d7zzZz84o5fPbK8+KOcfWSGXQM+HijtTfp99/VamQoxwsoW1gZ\ny28WUByhI01JaWCUnLZWhQVC6emTbS37tBND2NXahy8YPiugbFHicbG0bortjOXuofwuW2GhC9xl\nDlsKQUSuEZEDItIsIvfGeN0rIo+Zr28TkQZz//tEpElEdpt/r4o45jlzzJ3mY3q6PtREaKyv5Nef\nXs2FNQ7CykjQSpZU3UUWU0s8/Pnqev77jRNjTdxfOHiav//tbt51/jT+z/UXJrQ81i6ajsshSa82\nCpgBZbtzt1Ya7TvRl9T75JL2fl9aMorBKDntcTlwMPFkQAuPy8GUYrctC2GbqYBWNsS+cVlRX8kb\nx3ttLTDoGvJRlcc5CBZG1zStEDJBQoUgIk7gB8AHgCXALSKyJErsU0CPUmoB8F3gm+b+TuDDSqll\nwB3Aw1HH3aqUusR8dEzgc6SVxvpKvtBYxOXnVfOPG/Zy6PSg7WP7hgMpu4si+dQV83A5Hfzo+cMc\nHwjzl4+8xoLpZfzgE5faCvpNKXGzen510nGEg+2D+IPhs3oox6O2wktlibtgahoZvZRHqU2ThdBY\nX8mjd63mowvdPHrX6gklA0ZitdJMxLYj3SyqLR93ZdDyhip8wTB72uLHecJhRfeQP6+XnFoYXdO0\nyygT2LEQVgLNSqnDSik/sB64LkrmOuAX5vMngPeIiCilXldKWaVE9wLFIpL/tyAYDUb+7WOX4HU7\n+Kv1r491okrE5n2nJuQuspheUcTHls/mNzuO8/WtI3icDn7+yRVjqyzscPXSWg6dHqK5w75Ci9VD\nOR4iYgSWs9h9biL0jwYZDYTTZiGAoRQ+dJ4nbcoA7CWnBUJhmlp6YrqLLJY3GHNKFEfoHQkQVvmd\nlGahu6ZlDjsKoQ44HrHdau6LKaOUCgJ9QLTtfAPwmlIq8iz/ueku+rJM5HY6Q8yYUsQ3b7iIPW39\nfOfpA7aOmai7KJJ3nFdDMKwYDcGgP8iJ3uSa2r93cS0AzyThNtrd1kd5kYv6qhLbxyyeWcGBUwMT\nqtaazrLd8TidhsY42aCmPHH5ij1tfQz7Q6yaN76bqqbMy/ya0oRxhO4h49+yOo+7pVnormmZQxKV\nSRCRG4FrlFJ3mdu3AauUUvdEyOwxZVrN7UOmTKe5vRTYAFytlDpk7qtTSrWJSDnwJPCIUuqXMd7/\nbuBugNra2sb169en9EEHBwcpKzu3rr8d+V/s9bHleJC/XV40Vlo4luxQQPH5Z4d5X72bmy8Y/07L\n7lw2HvLz5MEACkNzf3Shmw+dl/gOLnL8f3p5BIfAl9fEXiMfPZd/emWEIiesW2lPHuDFtgAP7Pbz\nz1cUM6vMkVA+muaeEN/cPkowrHA7hL9bUcSCynO/ZztzSSR/zFfMv24fZd2KIhZXx3+PZMafyPkV\ni0f2+XjpRJAfvrd0XPlNR/w8fiDAv68tZqr3zPceLfvgbh+vdwT5j6tKcMS47xocHKTVX8w3Xh3l\n71YUsSSPvxcwPs+ezhDfXVuSkfFTkc20fLJjR7N27dompdTyhIJKqbgPYA2wOWL7PuC+KJnNwBrz\nuQsjdmApm9nAW8A74rzHncD3E82lsbFRpcqWLVtSlh/2BdV7v/OcWvG1Z1TnwOi4so9vP6bq121U\nO4/1pGUuO452q0Vf2qTmrduoFn1pk9pxtDvpuX/vT2+p+nUbVXvfSEJZXyCkFv7DJvXPf9hna2yL\nvW19qn7dRvX7nW225KP5/rMHVcO6jap+3UY1796N6vvPHkx4jN2xo+X/67Xjqn7dRnWoYyCt40/k\n/IqF9buN+IPjyn/y56+qtd8+d3+07GPmefnWqf5x5/KHXSdU/bqNat+JvgnPPVVZu/L/tGGvWvLl\npzI2fiqy+TaXaIAdKsH1VSlly2W0HVgoIvNExAPcjHG3H8kGjKAxwI3As0opJSJTgT8A9yqlXrKE\nRcQlIjXmczfwIWCPjbnkhGKPk/+45VJ6RwKse3LXuMXn/pBGdxGkJ2B59dIZADxjo4rqW+0D+INh\nWwlpkSyYXobbKSknqK1oqCTyG03HSp3xGMtSTkPZikySKDktFFZsP9Id111kYRW6i+c2KoTS1xYV\nxS6G/KEJN5TSnEtChaCMmMA9GFbAfuBxpdReEblfRK41xR4EqkWkGfgCYC1NvQdYAHwlanmpF9gs\nIruAnUAb8NN0frB0s3hmBfd94AL+uL+DR7ae20ynbzjAS82dfHCCq4uimWjAcuH0MhqqS3h6b2KF\nsKdt/B7K8fC4HJw3rSzlGlA9ZmXWKR7je5s1NXMX645+H6UeJ2VpylLOFGcUQuw4wv6T/Qz4gmfV\nLxqPhuoSaso8cQPLVqXTypL8VwjWwgrdJCf92PqvUEptAjZF7ftKxPNR4KYYx30N+No4wzban2Z+\ncOflDTz/1mm+9of9rJxXzaIZ5WOvPb3vFIHQxFcXpRsR4eqlM/j5S0cYGA3EXaW0q62PiiIXc5MI\nKFssmVnBi82dKc3x4VdamDmliL+6SLjvxRF+/epxvvC+81MaKxFGp7T8tg7ACCoD43ZO23bEuLjH\nW2FkISIsr69ie8v4CqF7yM/UkvyuY2Shy1dkjvz/9fMIEeHbN11MeZGbz//6dUYDZ5J9Nu0+Sd3U\nYi5Ok7sonVy9pJZASPHcgdNx5fa09bFs9pSULJzFMyvoGPCN3WnapbljkBebO7l11VxmlDq48vxp\n/PrVYwQy5A44ncayFZnEygcYz2W07XAXc6tKmDnFXkG95Q2VHO8e4VRf7JVqXQVQtsKiQlc8zRha\nISRJTZmXb990EQfaB/iXTfsBGAooXmzu5M8uSq+7KF1cOreSmjJP3KxlXzDE/pP9SccPLM70Rkgu\nQe2RrS14nA5uXjkXgNvW1HN6wGfLxZUKBWMhxIkhhMOKV492szKJsipjDXPGsRK6hnzUFECWMuiK\np5lEK4QUuHLRdD51xTx+8UoLf9rfzmvtwbx0F1k4HcJ7F9fG7ZHw1qlBAiHFRXFaZsZj8UzDfZZM\nYHnQF+TJplY+uGzG2AXw3edPZ3Zlccw4zURRVi/lArAQitxOyr2umDGEgx2D9A4HbMUPLJbMqqDY\n7Ry3YU5hWQhaIWQKrRBS5O+uWcTimRX89WM7+W1zgJoyT166iyyuXlobt0fCmR7KqX2G6jIvtRXe\npBTCb19vY8AX5PbLG8b2OR3CJ1bN5ZXDXTR3pLccxkgQRgKhvE9Ks6gp98bsibDtiFG/KJnVWO4E\nDXO6h/xUFcAKI4iMIWiXUbrRCiFFvC4nf/nu+QyMBukeVfQMB3jtWPKVRbPF5efVxO2RsLutlynF\nbuZUpd7kZfHMCvbZVAhKKR5+5SgX1lVw6ZyzrZKPLZ+Dx+ngka3HUp5LLHp9xuLWdHRKywY1ZZ6Y\nQeVth7uZOaWI2Uk25FneUGWsToq6kIaVomfYT02BWAhjXdN0PaO0oxXCBDjeM4IVMVBKpaX0caYo\ncjt59/nj90jY3dbHRSkGlC0Wz6zg0OlBW3Wfth7u5q32QW5f3XDOe9aUefngshk82dTKUBqXFloK\nYVoBuIwgdj0jpRTbjnSxal5V0r/VioZKwgpej7pxGQxQMHWMQMcQMolWCBNg9fxqvG6j9LEnTaWP\nM8nVS2tj9kjwBUMcODWQckDZYvHMCgIhxUEbrp6Htx5laombay+ZFfP129bUM+AL8vudJ2K+ngqF\nZyGcW8/ocOcQnYN+VqVwrl06txKHnFvobsBvfC+FUMcIjP81r8vBgM5DSDtaIUyATJU+zhRXLarF\nGaNHwoFTA2ZAeWIKYclYYDm+QjjVN8rmve18bPkcityx6+ZcNreSC2aU88jWlnEzw5Ol12dYLoWw\n7BQMhdA3EjjL4tpmxoCSCShblHldLJlVcU7Gcr+pKPO9fWYk5UVuHUPIAFohTJBMlD7OFEaPhKpz\n4gjJ9FCOR0N1KV6XI2Fg+VfbWggrxZ+vqh9XRkS4bU09+072py020zeqKCmALGULq5Vm19AZt9G2\nI13UlHmZV5O4vWksltdX8frxnrPyPAYChWUhgFG+ol+7jNKOVgiTjKuXzDinR8Ketj4qS9xJBymj\ncTkdLJpRHlch+INhfvXqcdYums7c6vgZ0ddfUkeZ15W2Jag9PsX0cm9e5orEYiwXwWylqZRi2+Fu\nVs1PPn5gsaKhitFAmL0RZUYsC6FQYghgWQhaIaQbrRAmGe9bcm6PhF2tfVxYN7GAssWSmRXsP9k/\nrpvnqT0n6Rz0cdua8a0Di1Kvixsuq+MPu04mnQEdiz6fKoikNIvo5LTj3SOc6h9Nqc+3RayGOQN+\nhQhUlthvvpRrKnTXtIygFcIkY9bUYpbVTeHpfYbbyB9SvNU+kLYKrYtnVtAzHBirKhrNw6+0UF9d\nwrsXTrM13p+vrscfCvObptYJz63XtBAKhWmmQrByEbaa+QcrbVQ4HY/aiiLmVpWclY8wEFBMLXbj\nKoA6Rha6a1pmKJwzQJM2rl5Sy+vHeunoH6V1IEwwrFJOSIvmTAmLc91Ge0/0saOlh9tW1+Nw2LNG\nFtaWs2peFY9uayEUY7msXZRS9PpUwawwgjMxBMtCePVIN5UlbhZOT71RChhuox1He8asuH6fKih3\nEeiuaZlCK4RJSGSPhCP9RnBx2ezUSlZEc4G50ihWgtrDr7RQ5HZwU+OcpMa8bU09x7tH+N+34hfn\ni8egL4gvVDgrjABKPC5KPM6xGMK2I12snFdlW5mOx4qGSrqG/BzpHAIMl1EhBZTBshC0Qkg3WiFM\nQs6vLaPe7JFwtC9MVamHWVPSc+dcUWQEp6MVQt9wgN/tbOP6S+qYkqSv+uolM5hW7uXhCQSXO8yM\n30KyEOBMclrXSJjj3SMTchdZLLcK3ZnLTwf8qqCWnAJUFLsZCYQITsBq1JyLVgiTEBHh6iW1vHyo\nkwM9IZalKaBssdgMLEfym6bjjAbCtoLJ0XhcDm5ZMYctBzo43j2c0pza+42yz4VkIYBZvmLQx4Ee\nw5JLJf8gmvOmlVJZ4h6LIxgWQmEpBKt8xUiBGglNLT1sPOSnqWX8Lna5QCuEScrVS2cQCCk6htMX\nP7BYMrOCo51DjPiNyqrhsOLhrS0sr69k6azU3uvmlXMR4Fevplbf6PRAYbTOjMayEA50hygvco3F\naCaCiLC8oYodLT2EworBAFQVSOlrC6t8xUiwsCwEpRTfefoAN/zwZZ44GOATP92aV0pBK4RJymVz\nK8cajZR4Y2cLp8rimRWEFRxoNzKW//fgaVq6hlOyDixmTS3mvYtreWz78XFLeMdjzEIokEqnFjXl\nRvmKA90hVjRU4Zxg/MBiRUMlRzqHONgxgOJMQ55CwbIQhgOFoxCOdw9zx8+3871nm8f2+YJhvvP0\ngbGbp1yjFcIkZefxXoZ8xkn4f/94MK13KUuiVhr98pUWasq8fODCifWLuG1NPd1Dfp7aHbtiazw6\n+n14nFBeIFnKFjVlXrqH/JwaVmlxF1lYcQSrEVHBrTIqIJdRKKz42YtHeP+//y9NR7u564p5FLkd\nCOAQePlQF+/5znNseONE2sq0pIpWCJOUrYe7UBgnXzAUTmul1tmVxZR5Xew/2U/HcJgtBzr4xMo5\neFwTO93ecV4N82pKUwoutw/4mOqVgslStpgWceeeSkG78bhw1hS8Lgf/s8dQroWmEKwmOcN57jI6\n2D7AjT96mfs37mPlvCqe/sK7+dKHlvDoXau5YaGb33zmch7/izVUlnr4/K9f56YfvcJus5RMLtAK\nYZKyen41HpdRqdWd5kqtDodwgVnC4tljQRwifCJO3aJkxr111VyaWnrYd8J+Ix6AQx0DhMPklb/W\nDla2stcJS2dNPH5g4XE5uGTO1LHVYDUFtuy0Is9jCP5gmH//41t88D9e4GjnEN/9+MX8/M4V1E01\nysNE1kBbOa+KDfdcwTdvWMbRriGu/cGL/N0Tb9AxELv/dSaxpRBE5BoROSAizSJyb4zXvSLymPn6\nNhFpiHjtPnP/ARF5v90xNZkl05VaF8+sYN+Jfl5oC/D+pbXMSNOy1hsbZ+N1OXhkm30r4bkDHew/\nOUDnqOLWB/IriJeIGnNV1FSvjBUhTBdWn2UoPAvhTAzB/jHZWtnz+rEePvy9F/n3Px7kmgtn8swX\n3s1HLp0d1zp1OoSPr5jLs1+8kk+/cz6/fb2Nq779PD96/lBKMbNUSehQFREn8APgfUArsF1ENiil\n9kWIfQroUUotEJGbgW8CHxeRJcDNwFJgFvBHETnfPCbRmJoM01hfyUCGKrUunlnBkBkoW5WGtfMW\nU0s8XHvxLH73ehtXvPPsu1qlFO39Pva09bH3RD97Txh/23pHxmQCQcM9VgjVaQE6zGB4+7ChzNKp\nvK26RgJUlhSWQigbiyHEthB6h/0c7BikuWOQg+2DNLV0s6u1DwX89tAr3P3OeVy9dAbn15ZTmoa4\nUlNLD7876OeJttf4w56T1JYX8eAdy3nP4tqkxqkocvP3H1zMLSvn8vU/7OMbT73JQy8dYV5pkPKG\nbhob0hdHioWdb2Il0KyUOgwgIuuB64DIi/d1wFfN508A3xdDHV4HrFdK+YAjItJsjoeNMTUFjCPC\n9vyXp/ZzYd2UtF3IbltTz2+aWvne66M0Ow4yEgixp62PfSf66RoysnpFYF5NKZfVV3LVBdN5bPtx\ngqFw2t1jmeZo1xACKNKvzC4zx3E7jEUGhaIkwegR7XE6eL0jxEMvH8EhwsF2UwF0DJ7Vaa7Y7aSi\n2IWlOkJhxQ+fP8wPnz8MwNyqEhbNKOeCGeVjf7uH/Gw85Kd8Xg+N9ZWEw4ohf5AhX4hBX5BBX5Ah\n8+/etj7+87lDZpLcSd6/tJZv33Tx2NLYVJhXU8oDd6zgwRcP87WN+znVD7c8sI1ffzqzfVfsKIQ6\n4HjEdiuwajwZpVRQRPqAanP/1qhj68znicbUFDDtfWf8n+m+kAVCRnXO/d1h9j/zFk6BRTMquOqC\n6VxYN4WlsypYPLPirNpD25wAAAmvSURBVDu/6y+t49d/3M4t711RUBe+1fNr8Lqb8QfSr8wOtg8i\ngD9M2q2PTNPU0oM/FOZoP3x1g3EfWe51saC2jKsumMbC6eUsmF7Ggull1E0t5vXjvdz6wFb8gTAe\nt4Pv3HQxLqeDA6cGOHBqgDdP9fOn/e1EJz4/cfBlvC4HPhttYcFYNXTR7KkTUgaRjAbCiIBSEApl\n3rqVRMucRORG4Bql1F3m9m3AKqXUPREye0yZVnP7EMYF/qvAVqXUI+b+B4GnzMPijhkx9t3A3QC1\ntbWN69evT+mDDg4OUlZmvyhYMvKZHDvT8pkau7knxL9uHyUYVrgcwt+tKGJBZfx8B7tjbzzk58mD\nARSGu+P6BW6uW5DY5VGI3yMY3+Ubp0a4eEZxwu8wmbEjv0cH8NGFbj50Xnq/x0x9L9HnwAfnubjx\nfE9cP32i79EfUpwcCrPxUIDt7Wf89gunOlhS7aTIJRS7oMgpFLmg2GX8bR8K89PdfkJJnOt2P2sq\n/0exWLt2bZNSanlCQaVU3AewBtgcsX0fcF+UzGZgjfncBXRi/E5nyVpydsaM9WhsbFSpsmXLlozJ\nZ3LsTMtncuwdR7vV3zywWe042p3WsXcc7VaLvrRJzVu3US360qa0j5+KfCH+ptn4HjP1OTM591TG\nTvZctzuXVMeOBtihElxflVK2VhltBxaKyDwR8WAEiTdEyWwA7jCf3wg8a05iA3CzuQppHrAQeNXm\nmJoCJ1PtRQutl3W+UsjfYybnnsrYmWylm802vQljCMqICdyDcXfvBH6mlNorIvdjaJ0NwIPAw2bQ\nuBvjAo8p9zhGsDgIfE4pFQKINWb6P57m7UomV0hNJgr5e8zk3Av5e5kIttZbKaU2AZui9n0l4vko\ncNM4x34d+LqdMTUajUaTO3Smskaj0WgArRA0Go1GY6IVgkaj0WgArRA0Go1GY6IVgkaj0WgAG5nK\n+YSInAZS7bReg5Ewlwn5TI6daXk9l/TI67nk/1ySlS/kuURTr5SallDKTvba2+GBzUy9VOQzOfZk\nmns+zaWQ567n8vabe7Jjp/rQLiONRqPRADqGoNFoNBqTyaQQfpJB+UyOnWl5PZf0yOu5ZH/sTMsX\n8lxSoqCCyhqNRqPJHJPJQtBoNBpNHLRCmAAi0mA2B8rGe31VRL6YgXE/LyL7ReTRNI6Z0vciIi9n\n4phU5iMig8nORZM6IjJVRD6b63lMdrRC0HwWeJ9S6tZcT0QpdXk2jtGkBzFI1zVkKsa5qMkhWiFE\nISK/E5EmEdlrtu9MhEtEHjXvsp8QkZIE498uIrtE5A0ReTiB7D+IyFsi8iKwyMbc/1xEXhWRnSLy\nYxGJ22tPRH4EzAeeEpH/P4Hsl0XkgIi8KCK/tmGtOEXkp+b3+LSIFNuYf9J35ckeIyLzReR1EVmR\n7HtFjdMgIm+KyEPmb/SoiLxXRF4SkYMisjLOcfvtfjci8gUR2WM+/trmnGydj5Hnl83f1HqPAyLy\nS2APMGccuVIR+YN5nu8RkY8nGPobwHnmufstm/PYE7H9RRH56jiy3xCRz0Vsj2tti8jfisjnzeff\nFZFnzedXxbKiRWSF+f9cZH7mvSJyYZx53x/5O4rI10Xkr+LIf8b8TnaKyBER2TKebFrIRrJDIT2A\nKvNvMcYJXx1HtgFQwDvM7Z8BX4wjvxR4C6iJfK9xZBuB3UAJUAE0Jxh7MfDfgNvc/k/gdhuf96g1\nnzgyK4CdQBFQDhxMMJcGjIZIl5jbjwN/bmMugyn8XgmPMeezB0Opvg5cPNFxIz7jMowbqybz9xfg\nOuB3E/1uIs6BUqAM2Atcmo7zMdnzK+o9wsDqBHI3AD+N2J5i5zdK4nc/Sx74IvDVcWQvBZ6P2N4H\nzBlHdjXwG/P5CxgdHt3APwJ/Mc4xXwO+DfyABK2AzXm/Zj53AIeIc42JOM5tzufDyf6PJPPQFsK5\nfF5E3gC2Ytz9LEwgf1wp9ZL5/BHgijiyV2GcbJ0ASqnuOLLvBH6rlBpWSvWTuMXoezD+ybeLyE5z\ne36CY+zyDuD3SqlRpdQAhuJJxBGl1E7zeRPGP0IumQb8HrhVKfVGmsY8opTarZQKY1ys/6SM/97d\nxP+8dr+bKzDOgSGl1CDwXxjnRTzsno/Jnl+RtCiltiaQ2Q28T0S+KSLvVEr1JTF+WlFKvQ5MF5FZ\nInIx0KOUOj6OeBPQKCIVgA94BViO8X29MM4x9wPvM+X+NcFcjgJdInIpcDXwulKqy8bH+L8YrYnt\n/O+ljK2OaZMFEbkSeC+wRik1LCLPYdwVxyN63W6u1vEK8Aul1H05ev9ofBHPQxgWVy7pA45hXCD3\npWnMyM8YjtgOE/9/K5PfTTbOx6GEk1DqLRG5DPgg8DUR+ZNS6v40ziHI2S7vRP+nv8Ho9z4DeGw8\nIaVUQESOAHcCLwO7gLXAAmD/OIdVY1hwbnMeib6fB8zxZ2BYcXERkTuBeuCeRLITRVsIZzMF4+5h\nWEQuwDAfEzFXRNaYzz8BvBhH9lngJhGpBhCRqjiy/wtcLyLFIlIOfDjBPP4E3Cgi062xRaTexvzt\n8BLwYdNPWgZ8KE3jZhM/8BHgdhH5RK4nY5MXMM6BEhEpxZj/eHepFnbPx2TPr6QQkVnAsFLqEeBb\nwGUJDhnAcEfapR3jrr9aRLwkPicfw+j1fiOGcojHCxguqP81n38G405+POX6Y+DLwKPAN23M/bfA\nNRiu2M3xBEWk0ZzLn5uWaEbRFsLZ/A/wGRHZDxzAcBsl4gDwORH5Gcad5w/HE1RK7RWRrwPPi0gI\nw5995ziyr4nIY8AbQAewPd4klFL7RORLwNNirPwIAJ8j9eqwkWNvF5ENGHdL7RjugJy5AKKwfQes\nlBoSkQ8Bz4jIoFIqGTdJ1jHPgYcw/NgAD5juj3jYOh+TPb9SYBnwLREJY5yLfxlPWCnVZQbk9wBP\nKaX+NoF8QETux/hu2oA3E8jvNRVfm1LqZIK5vwD8A/CKec6MMo4iFpHbgYBS6ldiLOJ4Wf5fO3ds\n00AQRGH4n5wIMiJiyoAa3AEioggKoAJn9OAUsCjADdAEARESegQ7khEC20KWbKP/S08rTXL3dm72\nruoiyeOKWt57OPya5GNNLTfAMfBUVTB+cne1Zs2f+aWyNlJVR0ne+tTKM3CdZLHjmk4YA7ptdUIH\nrarOgFmSX0+5rFh7yxik3225LH3TG7YFMEnysut6vvrXHUI/MB5+uHS54SBHS9OqOme8I73fgzA4\nBeaM0x3SQeh7aMYY6O9VGIAdgiSpOVSWJAEGgiSpGQiSJMBAkCQ1A0GSBBgIkqT2CToU5435yHrq\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x116077e10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# For each char\n",
    "ind_initial = np.random.randint(0,n_state,size=1)[0]\n",
    "\n",
    "print 'the selected char: {}'.format(chars[ind_initial])\n",
    "plt.plot(range(M_char.shape[0]),M_char[ind_initial],'.-');\n",
    "\n",
    "plt.xticks(range(M_char.shape[0]),chars);\n",
    "plt.grid();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Markov chain as a generative model?!!\n",
    "## What is the next probable characters for a give charachter\n",
    "\n",
    "\n",
    "# As we will see soon this is not the proper way of talking about Markov Chains"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "h\n",
      "e\n",
      "_\n",
      "t\n",
      "h\n",
      "e\n",
      "_\n",
      "t\n",
      "h\n",
      "e\n",
      "_\n",
      "t\n",
      "h\n",
      "e\n",
      "_\n",
      "t\n",
      "h\n",
      "e\n",
      "_\n",
      "t\n",
      "h\n"
     ]
    }
   ],
   "source": [
    "# To see if we can generate something\n",
    "n_state = M_char.shape[0]\n",
    "ind_initial = np.random.randint(0,n_state,size=1)\n",
    "print chars[ind_initial[0]]\n",
    "ind = ind_initial[0]\n",
    "for i in range(20):\n",
    "\n",
    "    \n",
    "    # If we take the most likely next chars, it quickly falls in a loop?!!\n",
    "    ind = np.argmax(M_char[ind])\n",
    "    \n",
    "    \n",
    "    # If we take the next char based on a random choice based on the probabilites \n",
    "#     ind = np.random.choice(range(M_char.shape[0]),size=1,p=M_char[ind])[0]\n",
    "    \n",
    "    print chars[ind]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f\n",
      "_\n",
      "f\n",
      "i\n",
      "s\n",
      "u\n",
      "t\n",
      "h\n",
      "e\n",
      "_\n",
      "a\n",
      "s\n",
      "t\n",
      "i\n",
      "n\n",
      "d\n",
      "_\n",
      "r\n",
      "i\n",
      "t\n",
      "h\n"
     ]
    }
   ],
   "source": [
    "# To see if we can generate something\n",
    "n_state = M_char.shape[0]\n",
    "ind_initial = np.random.randint(0,n_state,size=1)\n",
    "print chars[ind_initial[0]]\n",
    "ind = ind_initial[0]\n",
    "for i in range(20):\n",
    "\n",
    "    \n",
    "#     If we take the most likely next chars, it quickly falls in a loop?!!\n",
    "#     ind = np.argmax(M_char[ind])\n",
    "    \n",
    "    \n",
    "    # If we take the next char based on a random choice based on the probabilites \n",
    "    ind = np.random.choice(range(M_char.shape[0]),size=1,p=M_char[ind])[0]\n",
    "    \n",
    "    print chars[ind]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## at the char level, it is highly unlikely to expect something interesting\n",
    "\n",
    "## As we will see later in this session and later on RNNs it is a matter of long term memory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# World level Markov chain\n",
    "## with more depth: higher order Markov chains"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "\n",
      "Three horror stories based loosely on the door\n",
      "1\n",
      "\n",
      "But his acting hits the radio in \"Good Earth\" she shows a side note, when Poncelet apologizes to a style it will creep up\n",
      "2\n",
      "\n",
      "I really enjoyed this film, like me more story! As I said that a TV series 'the Sweeney and as funny nearly 60+ years with very little to be a vicious schemer, just a one-joke moment but by standing there\n",
      "3\n",
      "\n",
      "There are elements of humor as the maestro, his wife while dying; the bad guy to play Aragorn; I love Romy Schneider shines as Binodini, a young girl, who in the year it was a perfect fit for the unique footage about the characters get an overview of apartheid's cruelty in just about the lives of a soldier's mouth how war becomes a masterpiece\n"
     ]
    }
   ],
   "source": [
    "# codes from https://github.com/codebox/markov-text\n",
    "import sys\n",
    "\n",
    "sys.path.insert(0, './markovtext')\n",
    "\n",
    "from db import Db\n",
    "from gen import Generator\n",
    "from parse import Parser\n",
    "from sql import Sql\n",
    "from rnd import Rnd\n",
    "import sys\n",
    "import sqlite3\n",
    "import codecs\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "SENTENCE_SEPARATOR = '.'\n",
    "WORD_SEPARATOR = ' '\n",
    "\n",
    "# args = ['','parse','IMDB2',2]\n",
    "\n",
    "# if (len(args) < 3):\n",
    "#     raise ValueError(usage)\n",
    "mode  = 'gen'\n",
    "name  = './markovtext/IMDB_N2'\n",
    "depth = 3\n",
    "\n",
    "name  = './markovtext/IMDB'+ str(depth)\n",
    "count = 4\n",
    "\n",
    "\n",
    "if mode == 'parse':\n",
    "    file_name = './Data/IMDB_data/pos.txt'\n",
    "\n",
    "    db = Db(sqlite3.connect(name + '.db'), Sql())\n",
    "    db.setup(depth)\n",
    "\n",
    "    txt = codecs.open(file_name, 'r', 'utf-8').read()\n",
    "    Parser(name, db, SENTENCE_SEPARATOR, WORD_SEPARATOR).parse(txt)\n",
    "\n",
    "elif mode == 'gen':    \n",
    "    db = Db(sqlite3.connect(name + '.db'), Sql())\n",
    "    generator = Generator(name, db, Rnd())\n",
    "    for i in range(0, count):\n",
    "        print \"{}\\n\".format(i)\n",
    "        print generator.generate(WORD_SEPARATOR)\n",
    "        \n",
    "\n",
    "else:\n",
    "\traise ValueError(usage)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Note: BE AWARE OF STATE SPACE EXPLOSION!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<hr style=\"height:1px;border:none;color:stillblue;background-color:red;\" />\n",
    "# A quick conclusion on this part!\n",
    "# As it can be seen the generated results are far from being reasonable!\n",
    "\n",
    "# However, they are good for generating fake and confusing news!\n",
    "\n",
    "\n",
    "<hr style=\"height:1px;border:none;color:stillblue;background-color:red;\" />\n",
    "### In fact, the problem of \"generative models\" is still an open question, unlike \"discriminative models\"\n",
    "\n",
    "\n",
    "\n",
    "# <a></a>\n",
    "# <a></a>\n",
    "\n",
    "\n",
    "<hr style=\"height:1px;border:none;color:stillblue;background-color:red;\" />\n",
    "# The second view two Markov chains as a representational body\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Two types of representation of objects\n",
    "\n",
    "## 1- Representation of objects based on a priori given features \n",
    "### Based on the idea of \"Abstract Universals\"\n",
    "![](Images/FeatureBasedRepresentation.jpg)\n",
    "\n",
    "## Limits:\n",
    "* ** features might be arbitrary**\n",
    "* ** if few features, the space become small with size of the data**\n",
    "* ** if many features, observation might become expensive**\n",
    "\n",
    "## Important assumption: Independence of observations! \n",
    "## Fits very well with rule based and mechanical societies\n",
    "\n",
    "\n",
    "\n",
    "<hr style=\"height:3px;border:none;color:stillblue;background-color:black;\" />\n",
    "\n",
    "## 2- Representation of objects based on the context: Relational Representation\n",
    "### Based on the idea of Concrete Universals\n",
    "![](Images/RelationalRepresentation.jpg)\n",
    "\n",
    "\n",
    "## Important assumption: No independence in observations! \n",
    "## Fits well with quantum societies\n",
    "\n",
    "\n",
    "## Benefits:\n",
    "* ** No need to external abstract definitions**\n",
    "* ** State space expands with the size of data **\n",
    "\n",
    "## Limits:\n",
    "* ** Data intensive**\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Relational Representation\n",
    "# Markov chains as the early pioneering work in this direction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](Images/8_Markov_in_time.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### SOME MORE HISTORICAL MILESTONES: In Linguistic this is aslso called Distributional Semantic Models with references to (de Saussure, 1966; Harris, 1951 and Wittgenstein, 1963; Firth, 1957) \n",
    "sources from https://www.inf.uni-hamburg.de/en/inst/ab/lt/publications/cogalex-invited-biemann.pdf\n",
    "\n",
    "\n",
    "## Relational Representation is the case in many applications:\n",
    "* pixels in an image and their neighboring cells\n",
    "* A house and its neigborhood\n",
    "* A person and his friends\n",
    "* combination of different elements in a floor plan or building system\n",
    "* Combination of different elements in a chemical compound\n",
    "* an ingridient in a food recepie\n",
    "* ...\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Important concept: co-occurance matrix\n",
    "### Now if we take each row as the representation of each char, we can assume that we have a proper representation. \n",
    "### In fact, we have a matrix form for our data, where each object (e.g. char or a word) is being represented based on its normalized co-occurance matrix \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Char markov chain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the selected char: w\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnXl4W+d1p9+DHSTBRRQpUV6oxYod\nO6vpRXaWqbI6M4ndSezWjrPNxHXSxpN2WjuOp9nGTTptkmmnmaSTxXWzuZUTJ01cxYnj2HLixJYj\ny/umSJQlS6IkUhTBFTu++ePeS0IUSWwXC4HzPg8eAuDBxQFw8bsH557vHDHGoCiKojQHnlo7oCiK\nolQPFX1FUZQmQkVfURSliVDRVxRFaSJU9BVFUZoIFX1FUZQmQkVfURSliVDRVxRFaSJU9BVFUZoI\nX60dmM/KlSvN2rVrS3789PQ0ra2trttW2r6efCnWXn1xx159cce+mXzJZefOnceMMT15DY0xdXUZ\nGBgw5bBt27aK2Fbavp58KdZefXHHXn1xx76ZfMkFeMQUoLGa3lEURWkiVPQVRVGaCBV9RVGUJkJF\nX1EUpYlQ0VcURWkiVPQVRVGaiKYV/Z37x9g6mGTn/rFau6IoilI1mlL0d+4f4+pvbOcHu1Ncfct2\nFX5FUZqGphT97XtHSaSzGCCZzrJ972itXVIURakKTSn6m9Z34/EIAD6vh03ru2vskaIoSnVoStEf\n6O/i4g2W0P+Pt53FQH9XjT1SFEWpDk0p+gDxVAaA1R2hGnuiKIpSPZpW9IeicQAmYukae6IoilI9\nmlL0M1nDkQlb9OOpGnujKIpSPZpS9Icn42SyBoDJuEb6iqI0DwWJvohcIiK7RGSPiHx8gf//uYg8\nKyJPisi9ItKf87/3i8hu+/J+N50vlaFobPa6ir6iKM1EXtEXES/wFeBtwNnAVSJy9jyzx4DzjDGv\nAO4APm8/dgXwaeBC4ALg0yJS81IZJ58Pmt5RFKW5KCTSvwDYY4zZa4xJAluAy3INjDHbjDEz9s3t\nwKn29bcC9xhjjhtjxoB7gEvccb10nEi/OyRMqugritJEiDVlawkDkcuBS4wx19i33wtcaIy5bhH7\nLwNHjDGfFZHrgZAx5rP2/z4JxIwxX5z3mGuBawFWrVo1sGXLlpJf0NTUFG1tbUvafOfZBA8Opelr\nMQR9Xm68IOzatku1r+S2K22vvrhjr764Y99MvuSyefPmncaY8/Ia5punCFwO3JJz+73AlxexfQ9W\npB+0b18PfCLn/58Erl/q+aoxI/eD39xh3vJ3vzTv+OJPzX/60q9c3Xap9vU017NYe/XFHXv1xR37\nZvIlF1yckXsIOC3n9qn2fScgIm8C/hK41BiTKOax1ebweIw1nSFafHoiV1GU5qIQ0d8BbBSRdSIS\nAK4E7sw1EJFXA1/DEvzhnH/dDbxFRLrsE7hvse+rKUPRGH2dYVp8oqKvKEpT4ctnYIxJi8h1WGLt\nBW41xjwjIjdj/Zy4E/gC0AZ8X0QAXjTGXGqMOS4if4V14AC42RhzvCKvpEBiyQxjMylO6QwzPmyd\nyDXGYPutKIrS0OQVfQBjzF3AXfPu+1TO9Tct8dhbgVtLddBthsatyp01nSFe8EMqY4insoQD3hp7\npiiKUnmabkWuU665psNK7wBatqkoStPQvKLfOSf6E5rXVxSlSWg60T8UjSMCq9pDhP3WfboqV1GU\nZqHpRP9wNEZvJEjA58lJ72ikryhKc9B0oj80HqOvw1qBqzl9RVGajeYT/WicUzot0XfSOxrpK4rS\nLDSV6BtjGIpaq3EBws6J3JhG+qWyc/8YWweT7Nw/VmtXFEUpgKYS/ePTSRLpLGvsSD/kBY9opF8q\nO/ePcdXXt3PH7hTv/sZ2FX5FWQY0leg7ffSdnL6IEAn5NadfItv3jpLKZAFIpLP8ctdwnkcoilJr\nmkv07dW4Tk4fIBLyaZ1+iWxa343HM9e+4t7nj5JIZ2rokaIo+Wgu0Y/OtWBw0Ei/dAb6u/gPG1cS\n8sKfvWkjzwxN8rE7nnTaaCuKUocU1HunURiKxgj6PKxoDcze166Rfln4vB5WhoU/e9NL8Hs9fOHu\nXZzaFeaGt55Va9cURVmAJhP9OGs6wyd01IyE/BzKGZSuFMd4LEWL33o//+T3NnBwLMZXtg1ySmcL\n777w9Bp7pyjKfJpL9MdjJ6R2wIr0n9OSzZIZj6VotUVfRPiry87hyHiMT/74afo6Qmw+q7fGHiqK\nkkvT5fSdyh2H9rDm9MshV/TBSvd8+d3n8tK+CB/5l0d5+tB4Db1TFGU+TSP6yXSW4cnEbI2+QyTk\nYyqR1pOPJTIeS9E67/dia9DHre8/n66WAP/lmzs4ODZTG+cURTmJphH9oxNxjIFT5qV3IiEfWQPT\nSS01LJZUJstMMjOb08+ltz3EN//L+cRTGT7wzzsYn9FfU4pSDzSN6Of20c8lErIa8GgrhuIZt9+z\n1gVEH2Djqghff+957B+d5kPffURr+BWlDmge0bcXZp2U07dFX1sxFE8+0Qe4aEM3X7zilWzfe1xr\n+BWlDmia6h2nBcP86p1IyHoL9GRu8URnHNFf2u6yV53CwbEYX7h7Fz6PIJNJIuvGGOjvqoKXiqLk\n0jyRfjRGV4uflsCJxzlH9HV6VvE4KbGFcvrz+ZPf28CbX7qKHzx6iB/sTnH1LdqgTVFqQVOJ/vzU\nDlglm6DpnVJw0jttBYi+iPCK0zoAMEAqnWX73tFKuqcoygI0kejHTzqJC7mRvop+sYwXEekDXLxh\nJV57NbTf52HT+u6K+aYoysI0j+iPx04q14TcE7ma3ikWJ6ffUuCZoYH+Li552Sp8Hrjtmk2a01eU\nGtAUoj8RTzEZTy8Y6Qd9HvxeYSKmkX6xjMdStAa8+DyFRfoAZ65uJ52FV5zaUUHPFEVZjKYQ/cPO\n8JQFRF9EaNf2yiUxHkvREc5TujOPzhbLPqqLtRSlJjSF6M8NTzk5vQNWXl9P5BbPeCxFR0sgv2EO\nnbb9eCxZCZcURclDc4j+IqtxHSIhv5ZslsB4LElHuLilHl12pD+mkb6i1ISmEX2vR+iNaKTvJiWl\nd8JWpD82rZG+otSCJhH9OKvbQ3gXOeGoOf3SKCunr72OFKUmNInonzw8JReN9EtjPJaazdEXSpc9\nqjI6o5G+otSC5hD98dii+Xywc/oaeRZFPJUhnsoWHem3Brx4RXP6ilIrGl70s1nDkfH4gi0YHNrD\nPqaTGTJZ7QBZKM5Bsr1I0RcRWv2iJZuKUiMaXvSPTSVIZcyi5Zow11N/SlM8BeO0YCg20gdoC2h6\nR1FqRcOL/qE85ZqgnTZLwRH9zlJE3y+MqegrSk1oeNF3+ugvmd5R0S+aciJ9Te8oSu1oAtF3VuMu\nJfraXrlYHNEuKb2joq8oNaMg0ReRS0Rkl4jsEZGPL/D/14vIoyKSFpHL5/0vIyKP25c73XK8UIbG\nY7QGvLQvsXI0oqJfNOVG+preUZTakHcNvYh4ga8AbwYOAjtE5E5jzLM5Zi8CHwCuX2ATMWPMq1zw\ntSSGojH6OsOILN4Jcjanr2WbBTNeYvUOWCdyE+ks8VSGkN/rtmuKoixBIZH+BcAeY8xeY0wS2AJc\nlmtgjNlnjHkSyFbAx7JYbHhKLnPTs1T0C2U8liIS8i26ynkpnElbGu0rSvUpRPRPAQ7k3D5o31co\nIRF5RES2i8jvF+WdCxxeZHhKLnPD0TW9UygTJbRgcGi1RV/z+opSfcSYpRck2Tn6S4wx19i33wtc\naIy5bgHbbwJbjTF35Nx3ijHmkIisB+4D3miMGZz3uGuBawFWrVo1sGXLlpJf0NTUFG1tbQAkM4Zr\n75nhnRv9XLrh5HYBubbX/nyaN5zu48qzggVtu1hf3LStB/u/3xknmjD8z4vDRW/70YNTfOlp4cbz\nQ7y0O396p5HfR/XFfftm8iWXzZs37zTGnJfX0Biz5AW4CLg75/ZNwE2L2H4TuHyJbS35f2MMAwMD\nphy2bds2e33vyJTpv3Gr+f4jB/LanvfZe8yNdzxR8LaL9cVN23qwf+c//sZc9fWHStr2t++81/Tf\nuNX85MkhV3wp1bbe7NUXd+ybyZdcgEdMHj03xhSU3tkBbBSRdSISAK4ECqrCEZEuEQna11cCrwGe\nXfpR7nF4dmHW0ukdsGr1Nb1TOFaztdLSO232wzS9oyjVJ6/oG2PSwHXA3cBzwPeMMc+IyM0icimA\niJwvIgeBK4Cvicgz9sNfCjwiIk8A24C/MSdW/VSUQwXU6DvoIJXiKKWtskOrnshVlJpR0NgjY8xd\nwF3z7vtUzvUdwKkLPO5B4OVl+lgyzmrc1R35I/1IyMeERvoFYYxhfCZVUrkmQMArhPwe7b+jKDWg\noVfkDkVjrGwLEvTlP1nYHtZBKoUST2VJZopvq5xLV0tA0zuKUgMaW/QLKNd00Jx+4ZSzGtehsyWg\nPfUVpQY0tuhHlx6ekktERyYWzFyHzeKmZuXSGfZrekdRakDDir4xhqHo0sNTcokEfVbaIl13i4rr\nDkesy0rvtPp1Tq6i1ICGFf3xWIpYKlNQuSZoK4ZicCu9o5G+olSfhhX9Yso1QVsxFIMroh/2E51J\nOYv2FEWpEg0r+rPDU4rI6YMOUimEWdEvcXEWWNU76axhKqEHWUWpJg0s+oWvxoW56Vka6ednIpZC\nxDoPUirOal4t21SU6tK4oj8eI+D1sLJ18QZqucwNUlERykc0lqI95MdTQltlh84Wq/JHV+UqSnVp\nXNGPxlndESpYmOaGo2ukn49yWjA4dGmkryg1oYFFP1Zwagfm5uTq9Kz8lNNszUEjfUWpDQ0r+oeL\nWJgF0LaMcvo794+xdTDJzv1jNXl+NyJ956AxrgdZRakqDSn66UyWIxPxgss1AbweoS1Y/60Ydu4f\n48qvP8Qdu1Ncfcv2mgh/Oc3WHDrtx49Nq+grSjVpSNE/Opkgayh4Na6D1WmzvkVo+95RUhmrtj2V\nzrJ972jVfXAj0vd5PUSCPk3vKEqVaUjRL2Z4Si7ty6D/zqb13Tjnpn1eD5vWd1f1+Y0xrog+QGer\nX9M7ilJlGlL0i12N6xBZBp02B/q7OK2rBYDPXHoOA/1dVX3+mWSGdNbMpmfKoasloJG+olSZhhT9\nYlfjOiwH0QdmV7F2t5be5bJU3GjB4NAR9mt7ZUWpMg0q+jHaQz7ailwxuhxGJqYyWUanreh4eDJR\n9ed36urdEP2ulgDjGukrSlVpSNE/PF5cuaZDe7j+I/1jU3NCP1ID0Xcz0u9q0UhfUapNQ4r+oWhx\n5ZoOziCVeu78mCv0tYj03Wi25tDREmAiniKTrd/3W1EajYYU/aFojL4iK3fAyumnMoZ4qn4HqQxP\nWEIv1CbSn3A50jdGV0ErSjVpONGPp62SwpLSO8ug6dqInd7paxNGJuNVf/5orPypWQ5d2opBUapO\nw4n+8biVKigtvVP/TdecSP/0iKdmOX1n9XK5OCkizesrSvVoONEfjVmpmWJX48LyiPSHJ+N0tfjp\nDnkYmUpU/fzDeCxFe8iHSOltlR2cSH88ppG+olSLhhN9J9IvdjUuLI9If2QyQW8kRGdQSGVM1VsT\nj8fSsx0yy8Vpr6z9dxSlejSc6I/GDR6BVe3Fi/5yGI4+PJmgJxKkIyizt6tJdCZZdrM1h86w5vQV\npdo0nujHDL2REH5v8S9tOQxHtyL9XNGv7sncCZf67oD1fntE2ysrSjVpONE/Hs+WlNqBnOHodSpC\nxhhGJhP0tM+JfrVP5rrVbA3A4xE6tf+OolSVBhR9U1K5JkBrwItH6jfSH4+lSGay9LQF6axRemc8\nlnKl2ZpDZ9ivIxOp/WAcpXloKNHPZg2jcVNSuSaAiMyuyq1HnKi+tz1EyCe0BLxVjfSzWffaKjt0\ntqjo79w/xlVf317TwThK89BQoj86nSSdhb6O0tI7UN+dNp2ovjcSnP1bzUh/Kpkma9xZmOWg6R1r\nME4yY5Ua12owjtI8NJToHx53hqeUFulDfXfadE7a9tii3xMJVnVV7riLHTYdNNI/cTCOtwaDcZTm\noqFEfyhavui3h3x1W6c/clKkH6pqpO9mszWHrpYA0SaP9Af6u2YPpH/x5pdUfTCO0lw0lOgfsoen\nlBvp1216ZyJByO+ZbYFgRfrVE303m605dIb9TCczJNP12+Su0kzGU7OtKFpcaG+hKEvRUKI/FI0R\n8Myt9CyF9pCvbks2h+3VuE4LhJ5IkMl4mngqU5Xnj1ZC9O3pX9EmbsWwd2R69voRO0WpKJWioUT/\n8HiMFWEpqy9Me7i+q3ec1A7MpXmcJmyVxs0BKg7OAbqZ8/qDI1MAeAUOR6vfOVVpLhpK9A9F43SH\nymsEFgn5mEqk63KQyvBkfPYkLsyd0B2Zqo5QVEL0Z1sxTDdvpD84MoXPI/S3ezg8rqKvVJaGEv39\no9NMpyirzjkS8pE1MJ2sTsqkGE6O9K3S1GpG+n6vtT7ALTqdSL9OU2rVYHB4mv7uFlaGhSMTKvpK\nZSlI9EXkEhHZJSJ7ROTjC/z/9SLyqIikReTyef97v4jsti/vd8vx+Wzfe4zoTIp9E9myFrjUayuG\neCrDRDxNb04jublIv3qi3xH2u9JW2aHLyek3cQXP4MgUG3raWBEShqKxuvyVqTQOeUVfRLzAV4C3\nAWcDV4nI2fPMXgQ+APzLvMeuAD4NXAhcAHxaRCpSj/ar3x2bvV7OApe5nvr1VcHjVOn0tM1F+ita\nA3g9Ur1IfyblWodNB6elQ7MOUklnsuwbnWZDbxsrQh4S6WxTn99QKk8hkf4FwB5jzF5jTBLYAlyW\na2CM2WeMeRKYX3f3VuAeY8xxY8wYcA9wiQt+n8QbX7qKkN+DB/D7Sl/gMtdps76+eE49fk/7nOh7\nPUJ3a6BqZZtut2AAaAl4CXg9TSt0B8ZipDKGDT1tdNnnozSvr1SSQkT/FOBAzu2D9n2FUM5ji2Kg\nv4vbrtnEOzf6ue2aTSUvcKnX9srOytvcSB+gtz1YtfbKbjdbA6vfkbUqtznTO4PDVuXOhp5WVsyK\nvpZtKpVD8uUP7Rz9JcaYa+zb7wUuNMZct4DtN4Gtxpg77NvXAyFjzGft258EYsaYL8573LXAtQCr\nVq0a2LJlS8kvaGpqira2tpJth6ay/I9fx/jQK4JctMaX176SvuRy74spvvNskv+zOUxn0DNr/3c7\n44wnDP/z4qUXpLnh+w2/nOGMTg8femUor20x2/7LX8+wutXDf3v1wj2T3Hwf683+rheSfG9Xiq+8\nsYWxiWk+sUN439kB3nB6/oNrLffHerZvJl9y2bx5805jzHl5DY0xS16Ai4C7c27fBNy0iO03gctz\nbl8FfC3n9teAq5Z6voGBAVMO27ZtK8v26ETM9N+41Xz7oX1lbdsNX3L54t3Pm3Uf32rSmewJ9h/7\n/hPm/M/e46ovi9m//NM/M5/60VOub/uKrz5o/uCrDxblixu29WB/w/cfN+fZn9+9991n1t/0E/OF\nnz1fE1+qte1K2zeTL7kAj5g8em6MKSi9swPYKCLrRCQAXAncWeDB527gLSLSZZ/AfYt9X91Sr8PR\nhycSdLcF8XpOrJzpiQQZnU6SyVa24iObNUwm0q7n9MFaoNWsOf3BkWk29LQC4BFhVSTIkKZ3lAqS\nV/SNMWngOiyxfg74njHmGRG5WUQuBRCR80XkIHAF8DURecZ+7HHgr7AOHDuAm+376pagz4PfK0zE\n6iynP3Vijb5Db3uQTNZwvMKLmybjaYzB9eodsBZoNWMbBmMMe4atck2Hvs4wR/RErlJBCuruZIy5\nC7hr3n2fyrm+Azh1kcfeCtxaho9VRURor8NBKsOT8YVF32nFMG+1rts4q3E7WwKub7uz1c/YTApj\njKtrAOqd49NJxmOpE0R/dUeIZ4cmauiV0ug01Ipct6jHQSrDE4kFRX12gVaFyzadSLwy6Z0AyXSW\nWJUax9ULg3ajtfV2egegrz3E4XFdoKVUDhX9Bai3kYmZrGF0OjnbdiGX2VYMFRb9SvTdcXDKQJst\nr+80Wpuf3omnsrPvt6K4jYr+AkTqbJDKcftEbS0j/YqKvp0yaraxiYPDUwR9nhNmOjujPoe026ZS\nIVT0F6DecvrzJ2blEvJ7iYR8VRP9ThenZjl0Nml75cGRKdb3tOHJqchabYv+kQmt4FEqg4r+AtRb\nTt9ZcdvbvvCJ2mpM0KpkpN/V4jRdazbRnyvXdFjTYUX92opBqRQq+gsQCfnrqsvmbN+dtoVXrPZG\nKt+KYXwmRcDnIeR3r62ygzNIpZnSO/FUhgNjMyfk88E6gHs9osNUlIqhor8A7WEf08lMxRc8Fcps\nemfRSD9UlUi/ElE+zA1ab6b+O/tGpzEGNvSeKPpej9AbCWqkr1QMFf0FcHrqT9VJimdkMkEk5Fs0\nyrYi/cqLvtvN1hyCPi8tAW9TpXecubjz0ztgnczVnL5SKVT0F8DptDlRJydz8y286okEmUlmmE5U\n7iBVyUgfrLx+M/XUd7prrl95cnOtvo6wpneUiqGivwDtdSb688ckzmduVW7lov3oTGVFvyPcXO2V\nB0emOKUzTHiB0ZN9HSEOj8d1gZZSEVT0F6DepmcNTyboWWBhloPzK2C4gvNVKx7pt/qbak7u4Mj0\nSfl8h9UdIWKpTN31f1IaAxX9BYjUmejnj/StA0IlZ+VOxNwflZhLZ0ugaap3jDH2XNyT8/lgpXcA\n7bapVAQV/QWYzenXQeQ5lUgzk8wUlt6p0KzcdCbLZCJdkYVZDp3h5mmvfGQizkwyc1K5pkNfp71A\nSyt4lAqgor8ATkRbD6tynZTNUidyO1v8+L1SsUjfaUlR6RO50Zkk2Topk60kg8NO5c4iom+vytWy\nTaUSqOgvQD3NyZ1rwbB4Tl9E6GkLVizSr+RqXIfOFj9ZA5MVrECqF2YbrfUunN7paQviEZ2Vq1QG\nFf0F8Hs9hPyeuhCg4TwLsxx6IsGKRfrVEX2rFcN4E6R4BkemiIR8Jw25d/B5PaxqD2mkr1QEFf1F\nqJdWDCOzLRjyiX6oYtU7lWy25tBMrRisk7htSw6MWd0R0py+UhFU9BehvU6arg1PJvB7Ja/g9kSC\nHGuASL8pRH94etF8vkNfR0ird5SKoKK/CJGQvy4WZw1PxulpC+YdI9hrD0hPZ7Ku+zBuC3FlSzat\nbTf68JCpRJojE/ETpmUtRF+HNStXF2gpbqOivwj1MkhlZDJBT/viJ3EdeiJBjIFjU+5HytWI9J32\nymMVHvBea/YuMC1rIfo6QswkM3WxDyqNhYr+IrSH62OQSr6FWQ69FZygNR5LEfZ7Cfrcb6vs4LS+\naPT+O07lzhmLVO44rJ4t29QUj+IuKvqLUE85/aVq9B1mWzFUoK9+pVswgFWx0h7yNXx6Z3B4Gq9H\nOH1F/vQOaK2+4j4q+otQD8PRU5ksx6eThUX6dgqoEpF+pZutOXS1Nn4rhsGRKfpXtBDwLf3Vm12g\npd02FZdR0V+ESNBHPJUlmXb/xGihONU4Sy3McljZZuXEK9FpsxqRPlitGJohvbM+Tz4frHSdR+CI\npncUl1HRX4R6aMUwW6NfQKQf9HnpbPFXLKdfycodh86WwGylUCOSzmTZd2xm0ZW4ufi8HnojukBL\ncR8V/UWoh1YMTluFQtI7jl0lcvoTsVRFF2Y5dLU0dqR/cCxGMpPNW7njsLpDRV9xHxX9RXDaK9ey\nVn+4iEjfsatUpF+V9E6Dt1fee6ywck0Ha5iKpncUd1HRX4T2Ooj0HQFfmacFg0NvJOR6Tj+VyTKd\nzFRJ9P1MxtMVWWBWD8x118yf3gF7bKIu0FJcRkV/EeYGqdQy0o+zojWQt9LDwYn03RSJaizMcnAW\naDVq2ebgyBQr2wKzLSfyoQu0lEqgor8Ic8PRa5jTn0zkbbSWS28kSCKdddXnajRbc+icbbrWuKJf\nSOWOg7NASxuvKW6ior8IzpzcWnbaHJlM5G2pnEvP7Kpc90TCEf1qVe9Yz9mYef3BkfyN1nJZ06mr\nchX3UdFfhLY6yekXehIXclflupfXd/rbVye9Y0f6040X6R+fTnJ8OllwPh9gta7KVSqAiv4ieD1C\nW7B2rRiMMUWLfiX671Qzp98ZtiL9aAPm9GcbrfUWHun3RoKIqOgr7qKivwSRkK9mJ3LHYymSmWxB\nq3EdeiLut2Koqui3Ws8RbcCyzdlGa0Wkd/xeD72RoK7KVVxFRX8J2mvYU7/YGn2wykyDPo+76Z0q\nin4k6MPrkYas1R8cmSbo87CmM1zU41bbZZuK4hYq+ksQqWGnzbmB6IWLvoi4vkArOpOiNeDF7638\nriIidIb9RBuwemdweIp1K1vxepYehjOfNboqV3EZFf0lqKXoO+0UihF9x97NVgzVWo3r0NnSoKI/\nMlVUPt9BZ+UqbqOivwS1HJlYTLO1XNyO9KvVbM2hqwFbMSTSGV48PsOGlYVX7jj0dYSYSqTrYnSn\n0hgUJPoicomI7BKRPSLy8QX+HxSR2+3/Pywia+3714pITEQety9fddf9ytIermGkP5Eg7PfSFvQV\n9Ti3WzFUq9maQyNG+vtHZ8ia4ip3HJxhKhrtK26RV/RFxAt8BXgbcDZwlYicPc/sg8CYMeYM4O+B\nv83536Ax5lX25cMu+V0VnEEqteh94kzMyjcQfT49kSDRmRSJdMYVP6qf3gk0XPXO4HBxjdZymR2m\noqKvuEQhkf4FwB5jzF5jTBLYAlw2z+Yy4Fv29TuAN0qxalWHREI+UhlDogaDVAqdjTsf5zFuDUiP\nxpLVFf0GHKTilGuuKyG9MzsrN6plm4o7FCL6pwAHcm4ftO9b0MYYkwbGgW77f+tE5DER+aWIvK5M\nf6tKLVsxDE/Gi2rB4DC7KnfCnciw2pF+V2uAWCpDPOXOL5V6YHBkmjUdIVqLTNUBrGoP6QItxVUk\nX+pCRC4HLjHGXGPffi9woTHmuhybp22bg/btQeBCYBJoM8aMisgA8CPgHGPMxLznuBa4FmDVqlUD\nW7ZsKfkFTU1N0dZW2M/ofLbbh9J89ckEf/3aMGvaPEVtu1xf/vgX01y8xsd7z15Y+Bfb9r7xDJ95\nKM5HXx3k3FW+vPZL+RNsaeU4nP7rAAAcO0lEQVSPfj7Duzb6eceGxTtDuvm+bHsxxbeeTfL3vxem\nK+QpevuV/IxKtf/fT3pp8cMN5y9do7/Ytv9s2wyv6PHyX18WLMi+2O2Xa1tv9s3kSy6bN2/eaYw5\nL6+hMWbJC3ARcHfO7ZuAm+bZ3A1cZF/3AcewDyjz7O4Hzlvq+QYGBkw5bNu2zTXb+547avpv3Goe\n3X+86G2X40ssmTb9N241X75vd9HbPhyNmf4bt5rvPLSvZF8c+6MT1ra+PW9bhfpSiv3WJ4ZM/41b\nzXOHx0vafiU/o1Ls77vvPnP2J39qPv3jp0ve9qX/9wHznlu2l+3Lcn4fl6vvlfYlF+ARk0fPjTEF\npXd2ABtFZJ2IBIArgTvn2dwJvN++fjlwnzHGiEiPfSIYEVkPbAT2FvCcdUGt2iuXWq4J1oB0EXda\nMVSz2ZqD03StUSp4ognDdDJTVKO1+fR1hLV6R3GNvElGY0xaRK7Diua9wK3GmGdE5GasI8udwD8B\n3xGRPcBxrAMDwOuBm0UkBWSBDxtjjlfihVSCWg1HL6UFg4PP66G7NeBK2WY1WzA4OO2VG6WC5/C0\nlT4tpXLHYXVHiF/vOeaWS0qTU9CZJWPMXcBd8+77VM71OHDFAo/7AfCDMn2sGbUajj5S4mpch5Vt\n7izQqo3oN9YglcPTVuVXKTX6Ds4Crcl4anaim6KUiq7IXYJajUwsJ9IH6G0PuTJIZXZqVpVX5ELj\npHcOT2VpC/pKPoAD9HXqAi3FPVT0l6A14MUjMBGrfk7fI9DdWppQ9CzjSD8c8BL0eRoovZNlQ09r\n0YvscnEWaA2p6CsuoKK/BCIyuyq3mgxPJFjZFiy6I6NDb3uQkanyB6Q70XY1e+9AY7ViODxtysrn\nA6xud2bl6gItpXxU9PNQi06bw5PxklM7YEX6qYwpOy8+HkvN9rivJo3SdG06keZ43JSVzwddoKW4\ni4p+HmrRaXNkqrQWDA7OSt5yUzwTsRQdVWy25tAokf4Lx6YByirXBAj4PKxsC3I4qqKvlI+Kfh7a\nQ76q1+kPTySKGpM4n542Z0B6eSJR7RYMDp3hANHY8o/0nZ475aZ3wMrrH3aptYbS3Kjo58HK6VdP\n9DNZw+h0sqz0Tm+7O7NyozUS/a7Wxmi6Njg8hUfg9O6WsrfV1xHSnL7iCir6eWiv8nD049NJMllT\nUrM1h9mma2WKfs0ifbu9crknomvN4Mg0PWEh6POWva2+jrCmdxRXUNHPQyTkq2qXTScl46RoSqEt\n6KMl4C070q9desdPKmOYSS7vTpuDI1OsbnXnK7a6I8SkvUBLUcpBRT8P7WE/U4l01aLO2YHoZUT6\n4MzKLV30jTGW6NfgRK6zQGs5V/BksoY9I1PMpAw794+VvT2nVv+o5vWVMlHRz0Mk5CNrYLpKUacj\n1OWcyAVnVm7pApHMQjKdrUmk39EATdd+9vRh0hnDnmiWq2/ZXrbwO2MThzTFo5SJin4eIlUepFJO\nh81cyp2VO5OyftnU5ERuA7Ri+N4jBwEwQCqdZfve0bK250T62opBKRcV/Ty0z/bfqU4Fz8hkgkjI\nR8hf3sk/K9IvXfSnbb2tjeg7TdeWZ3rHGMPuo5OIWF8wv8/DpvXdeR+3FKvadVau4g7Fz29rMuY6\nbVYn6hyejJe1MMuhJxJkMp4mnsqUdACZtiP9zvDiE7MqxVx6Z3mK/iP7xxgaj/ORzRs4evBFrnrT\n+Qz0d5W1zdkFWlq2qZSJRvp5qHZ75ZHJRNmpHcidlVtatD+Trl16xznQLNf0zu07DtAW9PGRzWfw\n9g2BsgXfYU1nSCN9pWxU9PMwm9OvWqRf3mpcB+fXwshUaSIxlayd6Ad8HtqCvmW5QGsynuInTx7m\nHa/soyXg7g/p1e0hzekrZaOin4f2cPVGJhpjGJ6ol0jf+lsL0Xeedzmmd7Y+eZhYKsMfnHea69vu\n6wgxpOkdpUxU9PPQXsVBKtPJDLFUxpWcvvNrYWSqNNGfThlE5tJb1aar1U+0iovi3GLLjgO8ZFUb\nrzqt0/Vt93WGmYynmUpUtxeU0lio6Och6PPg90pVBqkM2wtvyl2YBbCiNYBHSo/0p1OG9pAfT5Xb\nKjssx/bKu45M8sSBKH9w3mllDU1ZDC3bVNxART8PIkJ7lQapzNbot5Wf0/d6pKxZudMpU7PUDljp\nnfFlltO/fccB/F7hneeeWpHtr54t29QUj1I6KvoFUK1BKsMutWBw6G0PltxeeSZVu3w+LL9IP5HO\n8G+PHeTNZ69iRWtlylzX2LNytYJHKQcV/QKo1sjEuRYM7oh+T1uwrJx+bUXfz3gsRTa7PDpt/uLZ\nYcZmUhU5gevgBAPabVMpBxX9AohUaZDKyGSCgNfjmtj2RkKl5/TTpibN1hw6WgJkTfXWR5TL7Y8c\nYE1HiNdt7KnYcwR9Xla2BTgyUV/pnYf3jvLjPUlXGssplUdX5BZAe8jP3mNTFX8eZzauWycBeyJB\njk0lyJQQLc/UQaQPy6MVw6FojAd2j/Df3rCx4vOE+zrCNU/vZLKGZ4bG+c2eUX729GGeODgOwE/3\nbee2P9rk2mI0pTKo6BfAXE6//GEYSzEymWClS6kdsNIBWWMNZikGYwzTdZDTh+Uh+nfYzdWuGKjM\nCdxcVneEeHF0puLPk4sxhsGRKX6zZ5QHB4/x0ODo7C/flW1z5y8SdmM5Ff36RkW/AOZGJlZe9E9b\nUf5oPYdSZ+XOJDNkTG1Ff7b/TixFbYpGCyObNXzvkQO8ZsNKVz+7xejrCPFwmR0787Fz/xjf25Vg\nR+J5hqJxHhw8xlE7TXhqV5i3vayPi8/o5qIN3Rw4HuPqW7YTT2UxQEeodvuMUhgq+gXQHvYxlUiT\nNZVtPjY8mXA1SnJO/BVbtjluL4rqrINIPzqTpJ7jxt8MHuNQNMaNbzurKs/X1xFmIp5mukILtHbu\nH+PKrz1EKmvghUE6wj5e/5JeXrOhm9eccfKBrTcS4rZrNvGdn+/gwWHhll/v5V0DpxIOVDZAUkpH\nT+QWgNN/p5Lrs9JZw/EyB6LPx6n3L7avviP6dZHTn67vWv3bdxygI+znLWevqsrzOQu0KpXX/+Gj\nBy3BBzwC175+Pf/3qldz5QWnL/pLZqC/i9/fGOAfrjyXfaMzfOHuXRXxTXEHFf0CcFoROINFKsGE\n3eDMjWZrDs4BpNhI3+luWUvRj4T8iFDXrRjGppP8/Jmj/OdXn1L2/INCWV3BVbnHp5Pc/fQRBEsY\nAj4Pm9avLPjxF23o5v0X9fPPD77Ab1847rp/ijuo6BdAuyP66cqJfjThiL57kX444CUS9JWc3mmv\noeh7PVL3Tdd+9PghkplsRWvz57Omw1mg5W7ZZiZr+NMtjzERT/P5d72Cd270c9s1xVfifOySszi1\nK8zH7niC2DIfbN+oqOgXQHsV0jvjtui7md4B6GkvvhXDRB2kd8A6p1Cv7ZWNMdy+4wAvP6WDs9e0\nV+15ZxdouRzpf+ne3Tyw+xifufQcrjj/tJLnALQGfXz+Xa/UNE8do6JfAHM5/cpF+o7ou9WCwaGn\nrfhWDLMncmu4OMt6/kDdRvpPHRrn+SOT/MH51YvyAUJ+L92tAVdF//5dw3zpvt2889xTuOqC8l/P\nRRu6eZ+meeoWFf0CqEZO3xH97lZ3Rb+3PVR8Tj+WxCPQFqxtcVdXi79up2dt2XGAkN/Dpa9cU/Xn\n7usMccSl9M6haIz/fvvjnLkqwud+/+WuLQy8UdM8dYuKfgE4ue2ZCqZ3ognDitYAAZ+7H0lvJFhS\n9U6Lj4q0By6GzjptuhZLZvj3x4f4jy/rq0kKbHW7O6tyk+ksH7ntUVIZwz9efa6rZZaa5qlfVPQL\nwIn0K5neiSaMqydxHXoiQWaSmaJ8H4+lafXXfklUZ0t9tle+66nDTCbSVU/tOPR1uDMr93M/eZbH\nD0T54hWvYH1PmwuenYimeeoTFf0C8Hs9hPweKqk/4wnj+klcmKsGctJHBfkSS9WF6He1BJhMpEnX\nWafN2x85wNruFi5ct6Imz9/XGWI8liJRRhBy5xNDfOuh/Vzz2nVc8rI+F707EU3z1B8q+gUSCfkr\nWrJZKdHvKVH0W+pA9J0TyZU62O7cP8bWweK6Qx6ZzvLbF45zRYWmYxWCs0DreLy0/XH30Uk+/oMn\nOX9tV8VXEmuap/7QNgwF0h7yEUtXJqlvjGE8YVxdmOXgbLMo0Z9JsqoOWqh02q0Yplw8gX5wbIaH\n9x7nJ08OsW3XCAb4tz0P8adv2sj7L16bN0f/wME0HoHLq9BcbTH67Fr9sSI+U4fpRJo/vu1RWgJe\nvvzuc/F7Kx/35aZ53vby1Zy/tja/kBSLgkRfRC4B/gGr49gtxpi/mff/IPBtYAAYBf7QGLPP/t9N\nwAeBDPBRY8zdrnlfRTwivDiRZef+Mde7CP7qdyOkjTV9yW2cSD9aZKS/vqX2kb7TiqEc0T9wfIaH\nXzjO9r2jbN87ysExq+ol6PPgbDVjDH93z+/4h3t3c15/F284q5c3vrSXDT1tJ0Tz6UyW3wyl2Xxm\nL6va3T9AF8pcpJ8t6nHGGD7+w6fYOzLFdz94YVVfw42XnMW2XcPc8P0n+Omfvr5qz1tLnF+SkXXu\na0Y55BV9EfECXwHeDBwEdojIncaYZ3PMPgiMGWPOEJErgb8F/lBEzgauBM4B1gC/EJGXGGOWVXJv\n5/4xBkemyBq48usPccNbz2Sgv4uOcICOsJ+OsP+kqpulPvBs1jCTyjCdSPPwC6P8xfeeAOC72/fz\n9lescXUH6Qz78Xpg59F0QQcsYwwT8TSt/tr/COwMW5H+dIGiv2PfcbY8b3WHPDKeYPveUQ5FLZHv\nbPFz4boVfPC167hwXTczyTTv+aeHSaayBPwePvX2szk4FuO+54f5Xz99nv/10+c5bUWYN5zZy+az\netm0vptvPbiPaMIwsLa2X2BHrItN79z7Ypp/f26IG956JhefUXh7BTdw0jxXfWM7N3z/CULx2oqh\ns5/f+9xRfvZcgsCpx1x5TzJZw1A0xj3PHuGv73qedNbw470PcfNlL+Ot56ymq8Vf86q4Qr7ZFwB7\njDF7AURkC3AZkCv6lwGfsa/fAXxZrFd2GbDFGJMAXhCRPfb2HnLH/eqwfe8oxv5+pTKGv77r+ZNs\nWgLe2QOAR+D5I5NkDfxwz4Os7W7BIEwlrO6IM4uc0Mpkjev9yB87ECWThV1jWa746oOcv7aL9T0R\nVrT6WdEaZEWrn66WAN2tQbparYNXJmvYV6FfNcXg5PR/cyjNhXuO0dcZ5vB4jKMTcY6MJ+y/cQ5P\nxDkwOsNxp7xz3yCRkI/XnrGSP3rdOjZt6OYlvRE88wac3HbNJv71Fzu46k3nz77Oj11yFkPRGNt2\nDXPfc8Pc/sgBvvXQfgJeD8mMFVl/6d7dXLiuu2bvTcjvpT3kY+eRDPfvGmagvwu/10PA6znpNTr8\n629f5LbnkpzX38Uf/4cNVfbY4qIN3bztZavZ+tRhAH40+BB/eP5prFvZStDnIeBcvF4CPs/sfS8c\nm+YXzyaY6DrEuad3EfR5Cfk9BH1e/F45QUSdYKulf5RTu1o4FI0xFI1xcMz6OxSN2ffFmcrpVPrz\nWx6mLeilv7uVNZ1hTrEvazrDnNIVZk1niJWtQR47EOXfB5Mkeo7QFvSxb3SafcemeeHYDPtGp3lx\ndGZ2P3FIZQw3/fApbvrhU0RCPtatbKW/u5W13S30d7eybqX1d/+x6ar8MihE9E8BDuTcPghcuJiN\nMSYtIuNAt33/9nmPPaVkb2vEpvXdBP0ekqksfp+Hz1x6Dn0dVgXFeCzF+Iz1N2rffu7wBE7BSdYA\nIpzT105b0EerfWkLemm1++J8Zdse0hljN7jqdtX37XtHEcDYvgyOTDM4Ms3YTGrJiVpPH8tw9S3b\nS+q/4hb7j1vDQh45muHdtzx80v8jQR+rOkKsbg/R1xlibCaJweoO+aHXr+e6N2xccvsD/V1MLtBu\nYE1nmKsv7OfqC/uJpzI8tHeUL927m8dejAKQqvGwkJ37x5hMpJmIwwf+eccJ//N6BL9XZg8Cfq+H\nrMkyPGkdEJ86NM5jB6I18/3MVRF++vQRwOose9vDLxb82HtffPyk+zxijZEM+j0IVrNAA9yxe/tJ\ntl0tftZ0hlnb3crFG1ay79g0v/yddV5HgDN62+hqCfDi6AwPDY6ecFAA8HmFTMZggB/s3jl7f9Dn\nob+7hfUrW3njWb2sXdlKMp3lr+96jlTa0ow/f/NL8Hk97B+dZt/oDE8ciPKTJ4dY6Cu4dV9lv3e1\n/w0PiMi1wLUAq1at4v777y95W1NTUwU/vhjb688N8MSRGK9cHaBvZi/MQAfWBS/QZl+APRHD53dY\nO7XPI7x7fYYzuiZO3GAWiMEpPrjxvODstidfeIL7X3DP92A0g98z58uHz/FwRpeXrPExk7Ly5ZPJ\nucuOo2mePmYNxEimsvzrL3YwuWHpOQLFvI/F2G8dPHFh1vmrvGw+3U9XUOgKCSGfcziLsWcsw+4j\n9usUITT+Ivfff8gVXwT4T30Znjlobd8rQjC6n/vvP+jK9ou13TqYxDkhIcDLVno5s8tD2kA6a10y\nxpDJZkmbLPvG5yLPVLq2n2nbVIaAvT96PcKfDwQ5vd1r+21I2f6nsoZ01jpx/sCh9Kwwn7vKy9nd\nXlIZSNr2qYxlPxjNkluHdW6vl82n+egOe1gxu79kgClgij2ZDA/mfDfevibBGV2O0AeZTgU4Hjcc\ni2UZjRl+eyTN78asN16A153i47Iz/HSFBI+Y2e1iDza7YWBOM84wByANGxzRWC+ksy0cixmOzmS5\n78U0T4xYGYBCv3clY4xZ8gJcBNydc/sm4KZ5NncDF9nXfcAx+305wTbXbrHLwMCAKYdt27ZVxLZY\n+0f2HTd/ccvd5pF9x5eVL4/sO27O/MRdZt2NW82Zn7iroMdUyvdifSn2PS/Gl2psv5LvS718po4/\nldgfS3md9fLdKGXb8wEeMXn03BhTUJ3+DmCjiKwTkQDWidk759ncCbzfvn45cJ/txJ3AlSISFJF1\nwEbgtyUcm5YdA/1dJXcqrKUvA/1d3HbNppJb67pJsb5U+j2vl8+0lPelXj5Tx59K7I+lvM56+W5U\n8zPKm94xVo7+Oqwo3Qvcaox5RkRuxjqy3An8E/Ad+0TtcawDA7bd97BO+qaBj5hlVrnTjCyW6252\nX+qJYt+X5fw+FuN7pV9nJbdfrc+ooJy+MeYu4K55930q53ocuGKRx34O+FwZPiqKoiguoW0YFEVR\nmggVfUVRlCZCRV9RFKWJUNFXFEVpIlT0FUVRmggxprimTZVGREaA/WVsYiXW4jC3bSttX0++FGuv\nvrhjr764Y99MvuTSb4zpyWtVyAqu5XShwFVpxdpW2r6efFnOvteTL8vZ93ryZTn7XmlfSrloekdR\nFKWJUNFXFEVpIhpR9L9eIdtK29eTL8Xaqy/u2Ksv7tg3ky9FU3cnchVFUZTK0YiRvqIoirIIKvoF\nICJrReTpKj3XZ0Tkepe3+VEReU5EbnN5uyW9LyLyoNv2ZfgyVexjlNIRkU4R+ZNa+9HMqOg3B38C\nvNkYc3WtHQEwxlxcSXvFPcTCTZ3oxNoflRrRtKIvIj8SkZ0i8ow9rjEfPhG5zY6Y7xCRljzbf5+I\nPCkiT4jId/LY/qWI/E5Efg2cWYDv7xGR34rI4yLyNRHxLmH7VWA98FMR+e8FbPuTIrJLRH4tIv9a\nwK8Or4h8w34ffy4i4QKeo6jougT79SLymIicX8zjFtnWWhF5XkS+aX9Gt4nIm0TkNyKyW0QuWOQx\nzxXzvojIn4vI0/blzwr0qaD9MXf/KuQztbe/S0S+DTwNnLaEbauI/MTez58WkT9catvA3wAb7H33\nCwX48XTO7etF5DNL2P+NiHwk5/aCv5pF5AYR+ah9/e9F5D77+hsW+zUsIufb3+eQ/ZqfEZGXLeHL\nzbmfo4h8TkT+dAn7D9vvyeMi8oKIbFvMtmwqvRCgXi/ACvtvGGvH7l7Cdi3WVNLX2LdvBa5fwv4c\n4HfAytznWsR2AHgKaAHagT15tv1S4N8Bv337H4H35Xmt+xxf8tidDzwOhIAIsDuPL2uxhuO8yr79\nPeA9BTzPVJGfVV5725ensQ6ajwGvdGPbOa/x5VhB0k778xfgMuBH5b4vOftAK9ak5WeAV7uxPxa7\nf+VsPwtsKuD9exfwjZzbHYV8TgV+NifYAtcDn1nC/tXAL3NuPwuctoDdJuD79vUHsKb5+YFPAx9a\nYvufBb4IfIV5I2MX8f1R+7oHGGQJjcl5nN/26R3FfEeKuTRtpA98VESeALZjRTIb89gfMMb8xr7+\nXeC1S9i+AWunOgZgjDm+hO3rgH8zxswYYyY4eRTlfN6I9UXeISKP27fX53lMobwG+LExJm6MmcQ6\nuOTjBWPM4/b1nVg7e63oAX4MXG2MecLF7b5gjHnKGJPFEuR7jfUNfYrFX28x78trsfaBaWPMFPBD\nrP1iKQrdH4vdvxz2G2O2F2D3FPBmEflbEXmdMWa8wO27jjHmMaBXRNaIyCuBMWPMgQVMdwIDItIO\nJICHgPOw3qsHlniKm4E327afz+PLPmBURF4NvAV4zBgzWsDL+AescbOFfPdKoqDJWY2GiPwe8Cas\nIe0zInI/VnS7FPNrW2tV6yrAt4wxN9Xo+eeTyLmewfrlVCvGgRexBPBZF7eb+xqzObezLP4dqvT7\nUun9cbogJ4z5nYicC/xH4LMicq8x5maXfEhzYgo633cU4PtYc7pXA7cvZGCMSYnIC8AHgAeBJ4HN\nwBnAc0tsuxvrl5jf9iXfe3SL/RyrsX6NLYmIfADoB67LZ1sOzRrpd2BFATMichbWz718nC4iF9nX\n3w38egnb+4ArRKQbQERWLGH7K+D3RSQsIhHgHXn8uBe4XER6nW2LSH8B/hfCb4B32HnLNuDtLm23\nWiSB/wy8T0TeXWtniuABrH2gRURasV7DUhEnFL4/Frt/FYWIrAFmjDHfBb4AnJvnIZNYqcNCOIoV\nuXeLSJDC9sfbsWZ0X451AFiMB7DSRb+yr38YKxpf6uD5NeCTwG3A3xbgy78Bl2ClTe9eylBEBmx/\n3mP/oqwYTRnpAz8DPiwizwG7sFI8+dgFfEREbsWKIv/fYobGGgj/OeCXIpLByjF/YBHbR0XkduAJ\nYBjYsZQTxphnReQTwM/FqqpIAR+hvM6kzrZ3iMidWJHPUayf7jX7uZ5DwVGsMWZaRN4O3CMiU8aY\nQtMZNcPeB76JlVsGuMVOVSxFQftjsftXCbwc+IKIZLH2xT9eytgYM2qfBH8a+Kkx5oYlbFMicjPW\n+3IIeD6fM/Z3LwIcMsYcXsL0AeAvgYfsfSbOEgdaEXkfkDLG/ItYhRMPisgbjDH3LeFL0j4hGzXG\nZPK4fh2wAtgmImA1Xrsmz2NKQlfkKicgIm3GmCm7GuRXwLXGmEdr6E831gkxt37NLHtEZC2w1Riz\naPXIEo/9DNbJ6y+67JYyDzsoexS4whizu9b+ODREpG8Lw70L/OuNBZ48Ueb4uoicjZWz/FaNBX8N\ncD9WxYSiLBvs79BWrJPodSP4oJG+oihKU9GsJ3IVRVGaEhV9RVGUJkJFX1EUpYlQ0VcURWkiVPQV\nRVGaCBV9RVGUJuL/AzVWNht2VX2WAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10f8eae50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# For each char\n",
    "ind_initial = np.random.randint(0,n_state,size=1)[0]\n",
    "\n",
    "print 'the selected char: {}'.format(chars[ind_initial])\n",
    "plt.plot(range(M_char.shape[0]),M_char[ind_initial],'.-');\n",
    "\n",
    "plt.xticks(range(M_char.shape[0]),chars);\n",
    "plt.grid();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's train a SOM with this matrix and see how it works"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/SVM/anaconda/envs/tensorflow/lib/python2.7/site-packages/sklearn/utils/deprecation.py:52: DeprecationWarning: Class RandomizedPCA is deprecated; RandomizedPCA was deprecated in 0.18 and will be removed in 0.20. Use PCA(svd_solver='randomized') instead. The new implementation DOES NOT store whiten ``components_``. Apply transform to get them.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n",
      "/Users/SVM/anaconda/envs/tensorflow/lib/python2.7/site-packages/sklearn/utils/deprecation.py:52: DeprecationWarning: Class RandomizedPCA is deprecated; RandomizedPCA was deprecated in 0.18 and will be removed in 0.20. Use PCA(svd_solver='randomized') instead. The new implementation DOES NOT store whiten ``components_``. Apply transform to get them.\n",
      "  warnings.warn(msg, category=DeprecationWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Total time elapsed: 3.248000 secodns\n",
      "final quantization error: 0.003545\n"
     ]
    }
   ],
   "source": [
    "import sompylib.sompy as SOM\n",
    "\n",
    "msz11 =20\n",
    "msz10 = 20\n",
    "\n",
    "X = M_char\n",
    "\n",
    "som_char = SOM.SOM('', X, mapsize = [msz10, msz11],norm_method = 'var',initmethod='pca')\n",
    "# som1 = SOM1.SOM('', X, mapsize = [msz10, msz11],norm_method = 'var',initmethod='pca')\n",
    "som_char.init_map()\n",
    "som_char.train(n_job = 1, shared_memory = 'no',verbose='final')\n",
    "codebook_char = som_char.codebook[:]\n",
    "codebook_char_n = SOM.denormalize_by(som_char.data_raw, codebook_char, n_method = 'var')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we projects all the vectors in SOM and visualize it \n",
    "xy = som_char.ind_to_xy(som_char.project_data(X))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([], <a list of 0 Text yticklabel objects>)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWQAAAD0CAYAAACsClzXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAHYFJREFUeJzt3XtUVWXi//H35l4aCF4BL6iNY2qZ\nevKSjmJYRqtMRxuczGSy1JQKJ7+VdrGLl8rVxUuZ5vKSmWm/yTFNyzQxy9QAm9FGRAskU9MBIS0F\nxOf3B8pIeAHPgb0Pfl5rnYU+z+bsjygfN8/e+xzLGIOIiNjPx+4AIiJSTIUsIuIQKmQREYdQIYuI\nOIQKWUTEIVTIIiIOoUIWEXEIFbKIiEOokEVEHMKvIhvXqVPHREVFVVIUEZHqKSUl5b/GmLoX265C\nhRwVFUVycvKlpxIRuQxZlrW3PNtpyUI8IjMzE8uyiI+PtzuKiNdSIYuIOESFlixEzicyMpKdO3cS\nEhJidxQRr6VCFo/w9/enZcuWdscQ8WqXvGSxdOlSunfvTkhICFdccQXXXnstkydPJj8/35P5xEto\nDVkuB1u2bGHAgAE0aNCAgIAAGjVqxPDhw9m/f79Hnv+SCnncuHHExcWxc+dO7r77bhISEjDGMG7c\nOHr37k1BQYFHwomIOMXcuXPp2rUrq1evpmfPniQmJuJyuZgzZw4ul4usrCz3d2KMKfejQ4cOZtOm\nTQYwjRo1MgcOHDBnFBYWmttvv90AZuLEiUYuLxkZGQYwQ4YMsTuKiMft2rXL+Pv7m+bNm5t9+/aV\nmlu7dq3x8fExffv2Pe/nA8mmHB1b4SPkuXPnAvDUU0/RoEGDknE/Pz9eeeUVfHx8mDNnjvv/U4iI\nOMTMmTMpLCxk6tSpREZGlpqLiYmhT58+rFixgqNHj7q1nwqf1EtNTQXgpptuKjPXokULGjZsSEZG\nBnl5eTrjLiLVwtdffw3Ahg0b+Oabb8rMHzp0iKKiItLT0+nQocMl76fChZyXlwdAeHj4OefDw8PJ\nysoiNzdXhSwi1UJ2djYAU6ZMueB2x44dc2s/FS7kMyV78OBBmjdvXmb+wIEDpbYTEfF2Z/osLy+P\n4ODgSttPhdeQ27VrB0BSUlKZuT179rBv3z6aNm1KrVq13A4nIuIEnTt3BmDjxo2Vup8KF/J9990H\nwIQJEzh8+HDJeFFREWPGjOHUqVMMHTrUcwlFRGyWkJCAv78/o0ePJj09vcx8QUGBR8q6wksWN954\nI4899hgvv/wybdq0YcCAAdSoUYPVq1ezY8cOunXrxv/93/+5HUxExClatmzJ3Llzue+++2jdujW3\n3norLVq0oLCwkKysLDZu3EjdunVJS0tzaz+XdOv0Sy+9RLt27ZgxYwbvvPMOhYWFNG/enAkTJvDo\no48SEBDgVigREae55557aNu2La+88grr169nzZo11KhRg4iICAYMGEBcXJzb+7CKr1kuH5fLZfR6\nyCIiFWNZVooxxnWx7fTymyIiDqFXexNxkNzcXF5//fVybRsfH4/eUq16USGLOEhubi7PPfdcubaN\njo5WIVczKmQRB4mKiqIi53WketEasoiIQ6iQRUQcQoUsIuIQKmQREYdQIYuIOIQKWUTEIVTIbpo/\nfz79+/enWbNmXHHFFQQHB9O1a1feffddu6OJiJfRdchuevDBB2ndujXdu3cnPDyc7OxsVq1axeDB\ng9m1axcvvPCC3RFFxEuokN20Y8eOMu+cUlBQQGxsLC+++CIjRowo86aIIiLnoiULN53rbawCAgIY\nNWoUJ0+eZN26dTakEhFvpCNkN2VlZfHSSy+xbt06srKyOH78eKn5n376yaZkIuJtVMhu+OGHH+jY\nsSNHjhzhT3/6E7fccgshISH4+vqSmZnJggULyM/PtzumiHgJFbIbXn31VbKzs5k3bx7x8fGl5hYv\nXsyCBQvsCSYiXklryG7Ys2cPAP379y8zt2HDhqqOIyJeToXshjOvRZuUlFRq/NNPP2XOnDlVH0hE\nvJoK2Q0jR44kICCAu+66i3vuuYfHHnuM2267jdjYWAYMGGB3PBHxMlpDdsN1113H+vXreeqpp/j4\n4485efIkbdu25cMPP6RWrVosWbLE7ogi4kVUyG668cYb+fzzz885p3d+EJGK0JKFiIhDqJBFRBxC\nhSwi4hAqZBERh1Ahi4g4hApZRMQhVMgiIg6hQhYRcQgVsoiIQ6iQRUQcQoUsHpGUlIRlWTz77LPn\nnI+Kiip5dTwROTcVsoiIQ6iQRUQcQoUsIuIQKmQREYdQIYuIOMQlFXJmZiaWZZV5p2W5fPn4FP9T\nOnny5Dnnc3NzqzKOiFfSEbJ4RGhoKAA//vhjmbk9e/aQl5dX1ZFEvI4KWTyiZcuWBAcHs3z5cg4d\nOlQyfvz4cR5++GEbk4l4DxWyeIS/vz+PPPIIeXl5tGvXjoSEBEaMGEGbNm04evQoERERdkcUcTwV\nsnjMc889x+TJkwkKCmL27NmsWrWK/v378+mnn+Lv7293vMvO2ed6MjMzGThwIHXq1CEoKAiXy8XK\nlSvtjug1+vbti2VZTJs2rczc008/jWVZDB061P0dGWPK/ejQoYMxxpiMjAwDmCFDhhgRcaYz36fR\n0dGmbt26plOnTiYxMdHce++9JjAw0Pj4+JjPP//c7pheITs72zRu3NgEBgaa1NTUkvG1a9caHx8f\n06pVK/Prr7+e9/OBZFOOjtURskg1l5SUxKhRo9i8eTOvvfYaCxYsYPny5Zw6dYopU6bYHc8rhIWF\nsXjxYoqKioiLi+PYsWP8/PPP3HPPPQQGBrJ06VKuvPJKt/ejQhap5po0acJTTz1Vaqx37940btyY\nrVu32pTK+9x444288MIL7N69m+HDhzN48GAOHjzItGnTaN26tUf24eeRZxERx7r++uvx9fUtM96o\nUSO+/vprGxJ5r8cff5z169fz3nvvAfDXv/6V+++/32PPryNkkWquVq1a5xz38/Pj1KlTVZzGu1mW\nxZ///OeS3ycmJnr0+VXIIiLltHv3bsaMGUNoaCg+Pj7cf//9nDhxwmPPr0IWESmH/Px84uLi+PXX\nX1myZAljx45l+/btHj1KViGLiJTDmDFj2LZtG4899hg333wzzz33HF27dmXWrFl88MEHHtmHCllE\n5CKWLVvGjBkz6NSpExMmTADA19eXxYsXExYWxv33388PP/zg9n5UyCIiF5CVlcXQoUMJCQnh/fff\nx8/vfxenNWrUiLlz5/LLL78wcOBACgoK3NqXVXwTSfm4XC6TnJzs1g5FRC43lmWlGGNcF9tOR8gi\nIg6hQhYRcQgVsoiIQ6iQRUQcQoUsIuIQKmQREYdQIYuIOIQKWUTEIVTIIiIOoUIWEXEIFbKbzn5n\n3/T0dOLi4qhXrx4+Pj4kJSXZHU9EvIjewslDvv/+ezp16kSLFi0YNGgQx48fJzg42O5YIuJFVMge\n8uWXXzJ27FgmTZpkdxQR8VJasvCQ+vXrM378eLtjiIgXUyF7SNu2bQkMDLQ7hoh4MRWyhzRo0MDu\nCCLi5VTIHmJZlt0RRMTLqZBFRBxChSwi4hAqZBERh1Ahi4g4hArZTVFRURhjmD9/vt1RLntpaWlY\nlkXPnj3Pu821116Lv78/Bw4cqMJkIuWjQpZqo2XLlvTs2ZOkpCTS09PLzG/atIkdO3Zw5513Eh4e\nbkNCkQtTIUu1MnLkSABmz55dZu7M2PDhw6s0k0h5WcaYcm/scrlMcnJyJcYRcc/Jkydp3LgxBQUF\n/PTTTyV3T+bm5hIREUFERAS7d+/WdeNSpSzLSjHGuC62nY6QpVrx8/PjgQceIDs7m3/84x8l4wsX\nLuT48eMMGzZMZSyOpUKWamfYsGH4+voya9askrHZs2cTEBDA3/72NxuTiVyYXn5Tqp3IyEj69OnD\nsmXLSEtLIycnhx07dhAXF0fdunXtjidyXipkqZZGjhzJsmXLmDVrFkeOHAF0Mk+cTyf1pFoyxtCy\nZUsOHz7MiRMnaNy4MWlpaXbHksuUTurJZc2yLEaMGMGRI0dKTuaJOJ0KWaqt+Ph4fHx8CAoKYsiQ\nIXbHEbkoFbJUW//61784deoUAwYMoHbt2nbHqXLGGGbMmEHr1q0JCgoiMjKShIQE8vLyiIqKIioq\nyu6I8js6qSfV1ssvvwxAQkKCzUnskZiYyLRp0wgPD2fYsGH4+/uzfPlytmzZQkFBAQEBAXZHlN9R\nIUu1sn37dlauXElKSgqrV6/m9ttvp1OnTnbHqnKbNm1i2rRpNG/enK1btxIWFgbAxIkT6dmzJwcO\nHKBJkyY2p5Tf05KFVCspKSmMGzeOzz77jLvuuot58+bZHckWZ/7cTz75ZEkZAwQFBTF58mS7YslF\n6AhZqpX4+Hji4+PtjmG71NRUAHr06FFmrlu3bvj6+lZ1JCkHHSGLVEN5eXkA1K9fv8ycn58fderU\nqepIUg4qZJFqKCQkBICff/65zNzJkyf573//W9WRpBxUyHJRukTK+7Rv3x6ADRs2lJn78ssvKSoq\nqupIUg4qZJFq6Mw6+sSJE8nJySkZP3HiBGPHjrUplVyMTuqJVENdu3bloYceYvr06bRp04YBAwaU\nXIccGhqqt7ByKB0hi1RTU6dOZfr06YSEhDBr1iwWL15M7969Wbt2rW4KcahLKuTMzEwsyyI+Pp60\ntDT69u1LWFgYNWrUoFu3bqxZs8bTOaWSXeg2W/FOlmWRkJDAzp07yc/PZ//+/bzxxhslJ/ykYj76\n6CNiYmIIDw8nMDCQiIgIevTowZtvvumxfbh1hJyRkUGXLl3Iyclh+PDh3HXXXaSkpBAbG8uSJUs8\nlVGqQGJiIg899BBHjhxh2LBhDBw4kE8++YRevXpRUFBgdzwRW82ePZs777yT//znP9xxxx08+uij\n3HbbbRw/ftyzNx8ZY8r96NChgzHGmIyMDAMYwIwZM8ac7ZtvvjF+fn6mVq1aJi8vz4jzffXVVwYw\nzZs3N9nZ2SXjx48fN507dzaAadKkiX0BxeOaNGmiv9MKaN++vQkICDA///xzmbnDhw9f9POBZFOO\njnXrCDkkJIRnnnmm1JjL5WLQoEHk5uaybNkyd55eqohusxW5OD8/P/z9/cuMe/ImG7cKuX379lx1\n1VVlxqOjowHYtm2bO08vVUS32V5+MjMzyczMtDuG1xg0aBC//fYbrVq1YvTo0fzzn//k8OHDHt+P\nW4V8rtsyARo0aACgE0JeQrfZilzY3//+dxYsWECTJk2YNm0a/fr1o379+vTs2RNPvq2dW4V8rtsy\nAQ4ePAigs7leQrfZilzcvffey+bNm8nOzubjjz9m6NChfPHFF/Tu3dtjR8tuFXJqaipHjx4tM56U\nlARAu3bt3Hl6r3D2JYDeSrfZipRfrVq1uO2223j77beJj48nJyeHL774wiPP7VYh5+Xl8fzzz5ca\nS05OZtGiRYSEhNCvXz+3wknV0G22Ihe2fv16ii+WKO3QoUMAXHnllR7Zj1u3Tnfv3p05c+awZcsW\nunbtyoEDB1iyZAmnTp1i1qxZBAcHeySkVC7dZityYf369aNmzZp07tyZqKgojDFs3LiRb775hg4d\nOtCrVy+P7MetI+SmTZuyadMmQkNDeeutt1i6dCnt27dn1apVxMXFeSSgVA3dZityfi+++CI33HAD\nqampvPnmm8ybN4/CwkJeeukl1q9ff87L4S5JeS5WNue5MWTIkCGXfqV1NXG+r0VRUZF5+OGHDWD6\n9etnfvvtN3sCiojtKOeNIXq1t0pw4sQJBg0axIcffsioUaOYNm0aPj56HScRuTAVsofl5OTQp08f\nNm3axIsvvsjjjz9udyQR8RIqZA/au3cvt956K99//z0LFy5k0KBBdkcSES9ySYV85iyj/M+uXbvo\n0qULv/76K6tXryYmJsbuSCLiZbSw6SHp6ekcOHCAZs2aldxoISJSESpkD7njjjuYNGkS3377LTEx\nMWRnZ9sdSUS8jArZg8aOHctrr73Gtm3biI6OPu9rfYiInIsK2cMSExOZOXMm3333HT169GD//v12\nRxIRL6FCrgQjRoxg7ty57N69m+7du5OVlWV3JBHxAirkShIfH8+7777L3r176d69Oz/88IPdkUTE\n4ayKXL7mcrmMJ1+MWUTkcmBZVooxxnWx7XSELCLiECpkERGHUCGLiDiECllExCFUyCIiDqFCFhFx\nCBWyiIhDqJBFRBxChSwi4hAqZBGpVMYYpk6dSqtWrQgKCiIyMpKEhATy8vKIiooiKirK7oiOobdw\nEpFKNWrUKGbOnElERATDhg0jICCAjz76iK1bt1JYWIi/v7/dER1DhSwilWbjxo3MnDmTFi1asGXL\nFmrVqgXApEmT6NWrF/v376dJkyY2p3QOLVmISKVZsGABAE8++WRJGQMEBAQwefJku2I5lgpZRCrN\ntm3bAOjWrVuZuc6dO+Pnpx/Sz6ZCFpFKk5eXB0D9+vXLzPn6+lK7du2qjuRoKmQRqTTBwcEA53x/\nyaKiIr0Z8O9cUiFnZmZiWRbx8fEejiMC06ZNo1WrVlxxxRVYlsXrr79udyS5RO3atQPgyy+/LDO3\nefNmTp48WdWRHE1HyOIo77//Po888ghBQUEkJiYyfvx4OnfubHcsuUT33nsvABMnTixZvgAoKChg\n3LhxdsVyLK2oi6OsXLmy5GNERITNacRdPXr0YNiwYcyePZvWrVvTv39//P39WbFiBSEhIURERODj\no+PCM/SVEEfZv38/gMq4Gpk5cyavvvoqNWvW5K233uK9996jV69efPbZZ/zyyy8l68zigUJOS0uj\nb9++hIWFUaNGDbp168aaNWs8kU0uI88++yyWZbF+/XoALMsqeYh38/HxYfTo0aSlpZGfn8/+/ft5\n4403OHToEMeOHeOaa66xO6JjuLVkkZGRQZcuXbj22msZPnw4Bw4cYMmSJcTGxvLee+8RFxfnqZxS\nzUVHRwMwf/589u7dy/jx4+0NJB5z8OBB6tWrV2pp4rfffiMxMRGAfv362RXNcSxjTLk3drlcJjk5\nmczMTJo2bQrAmDFjmDJlSsk2ycnJdOnShZo1a7J37179OCIVEh0dzYYNG6jIv0txtieeeILFixcT\nHR1NeHg4Bw8eZN26dezbt4/Y2Fg+/vjjav+TkGVZKcYY18W2c2vJIiQkhGeeeabUmMvlYtCgQeTm\n5rJs2TJ3nl5EqoGbb76ZVq1asWbNGl577TU++OAD6tSpw8svv8zy5curfRlXhFtLFu3bt+eqq64q\nMx4dHc2CBQvYtm0bQ4YMcWcXIuLlYmJiiImJsTuGV3DrCPlct0MCNGjQAKDUdYciInJhbhXyuW6H\nhOJFfChe0hARkfJxq5BTU1M5evRomfGkpCTgf7dNikjF6OUJLk9uFXJeXh7PP/98qbHk5GQWLVpE\nSEiILmcRkWqhqv6DdOukXvfu3ZkzZw5btmyha9euJdchnzp1ilmzZumSN5FLFBkZyc6dO7Xsd5lx\n6wi5adOmbNq0idDQUN566y2WLl1K+/btWbVqlW4KEXGDv78/LVu2JDw83O4oUpWMMeV+dOjQwUhp\nGRkZBjBDhgwxe/bsMf379zdhYWGmZs2a5uabbzbbt283xhhz6NAh88ADD5gGDRqYwMBA43K5zOef\nf25zenGqs/9dib3Gjx9vgHM+5s2bV67nAJJNOTpWr/bmIZmZmXTq1IlrrrmG+Ph4MjMzWbZsGdHR\n0Xz99dfceuutBAcHExcXR05ODu+//z6xsbGkp6fTuHFju+OLyHlER0eTm5vL1KlTadu2LX379i2Z\nu/766z27s/K0ttER8nmdOZIBzIQJE0rNPf/88wYwoaGhZvjw4aaoqKhk7p133jGASUxMrOrI4gV0\nhOws7v59UM4jZL38podERUXxxBNPlBo7c5difn4+U6ZMKfXiKnfffTd+fn58++23VZpTRJxLhewh\n119/Pb6+vqXGzrymb4sWLcrcYu7r60v9+vXZt29flWUUEWdTIXvIuS5POvMW5+e7dMnPz4/CwsJK\nzSUi3kOFLCLiECpkEZGLOLMcWVRUVKn7USGLiFxEaGgolmWRlZVVqfvRdcgiIhdRs2ZNOnXqxMaN\nGxk0aBAtWrTA19eXPn36cN1113lsPypkEZFyWLhwIaNHj+aTTz5h8eLFGGNo2LChCtlJoqKiLvj+\nbxeay8zMrIREUh2cOHECgMDAQJuTyBlXX301K1asqNR9aA1ZxIHS09MBaNiwoc1JpCrpCFnEQf79\n73+zaNEiFi1ahI+Pj15T/DKjI2QRB0lNTWX69OnUq1eP5cuX06ZNG7sjSRWyLrTG+Xsul8skJydX\nYhwRkerHsqwUY4zrYtvpCFlExCFUyCIiDqFCFhFxCBWyiIhDqJBFRBxChSwi4hAqZBERh1Ahi4g4\nhApZRMQhVMgiIg6hQhYRcQgVsoiIQ6iQRUQcQoUsIuIQKmQREYdQIYuIOIQKWUTEIVTIIiIOoUIW\nEXEIFbKIiEOokEVEHEKFLCLiECpkERGHUCGLiDiEClkcIzMzE8uyiI+PtzuKiC1UyCIiDqFCFhFx\nCBWyiIhDXFIhR0VFYVnWeR9aAyyfzZs3Y1kW/fr1O+8211xzDYGBgeTk5FRhMvtlZmYycOBA6tSp\nQ1BQEC6Xi5UrV9ody2scO3aMgIAAunbtWmr8+PHjBAUFYVkWCxcuLDU3c+ZMLMti7ty5VRnVq2zd\nupW4uDgiIyMJDAwkPDycW265haVLl3rk+f0u5ZMSExPJzc0tM75ixQpSU1O58sor3Q52OejcuTN/\n/OMfWbVqFdnZ2dSuXbvU/NatW0lLS6N///6EhYXZlLLq7d27l44dO9KsWTMGDx5MTk4OS5Ys4c47\n72Tt2rX07NnT7oiOV7NmTTp27MiWLVs4evQoV111FQBfffUV+fn5AKxbt47BgweXfM66desAiImJ\nqfrAXuDtt9/mwQcfxNfXlz59+vCHP/yBQ4cOkZyczJtvvslf/vIX93dijCn3o0OHDuZ81qxZY/z8\n/MzVV19tDh8+fN7tpLRJkyYZwEyfPr3M3MiRIw1gPvroIxuSVb2MjAwDGMA8++yzpeY++eQTA5jY\n2Fib0nmfp59+2gBm5cqVJWNPPPGE8fX1NTfddJNp2LBhyXhRUZEJCwszzZo1syOq43333XfGz8/P\nhIaGmh07dpSZ//HHHy/4+UCyKUfHeqSQt2/fboKDg03t2rVNenp6hf+wl7Mff/zR+Pj4GJfLVWo8\nPz/fhIWFmXr16pnCwkKb0lWtM4XcpEkTc/LkyTLzjRs3NrVr17YhmXdKSkoygBk9enTJ2A033GA6\nduxoZsyYYQCza9cuY4wxKSkpBjAPPPCAXXEdLSEhwQDm1VdfvaTPL28hW8Xblo9lWYeBvb8b9gda\nnv6YDhxz54j9MvUHIBj4DjhxeqwW0Bz4GdhnU66qFgBcC+QC359j/o9ATSClKkN5MQu4HsgH/gP4\nnv79QSAbaA1kAYeB+kBD4AfgiB1hHe4a4EpKf49WRBNjTN2LblWe1j7fA6hB8TfHKeCv7jzX5fwA\n7qb4R/WXzhr76PRYW7vzVeHXIer0n3n+eeaTiv/J2p/VWx7AmtPfn3WBO09/fWNOz+0H/t/pX68+\nvV09uzM78QHsPv21u6oy93PJl71ZluULvA+0B54yxiy+1OcSlgG/APdYluVrWVY9IBb4lzHmX/ZG\nEy/3OcVHyjGnHyeAr86a62lZViDwJ+A7Y8whW1I635mrGCIrcyfuXIf8OnA7MNcYM8lDeS5Lxpjj\nwFIgAuhF8RGzH7DAzlxSLaw7/TEGuAnYZIw5cdZcGPAgxT/triv76XLa5tMfYytzJ5dUyJZlJQIJ\nwFpghEcTXb7mn/547+nHSWCRbWmkukgF8ihermhN6dL9/PTHsb/7vZQ1k+Lvyacty2r1+0nLshp6\nYicVvg7ZsqwGwCsUr6fsAJ60LOv3m31rjPmn+/EuH8aYryzL2gPcRfEJ0hX68VHcZYwpsiwrieJC\nhrMK2Riz17Ks7yk+eVwEbKj6hN7BGPMfy7JGAm8B2yzLWk7xunJt4AaKlxzdvkD+Um4MCeJ/R9aJ\n59lmAaBCrrgFwAtn/VrEE9ZRXMi/AMnnmGsOpBhj8qo6mDcxxrxtWdYOYAwQDfQF/gv8G5jjiX1U\n6LI3ERGpPHpxIRERh1Ahi4g4hApZRMQhVMgiIg6hQhYRcQgVsoiIQ6iQRUQcQoUsIuIQKmQREYf4\n/9aqodnoOUlQAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x116109c10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax = plt.subplot(1,1,1)\n",
    "for i in range(len(X)):\n",
    "    plt.annotate(chars[i], (xy[i,1],xy[i,0]),size=20, va=\"center\")\n",
    "    plt.xlim((0,som_char.mapsize[0]))\n",
    "    plt.ylim((0,som_char.mapsize[0]))\n",
    "plt.xticks([])\n",
    "plt.yticks([])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## In the char model, we can't make sure if it makes sense, maybe at the world level, it works\n",
    "### Nevertheless, we only represent each chars based on its relation with the next possible chars\n",
    "### And this means to loose lots of valuable data\n",
    "![](Images/RelationalMarkov.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## At the word level, defnitely we get better results, but..\n",
    "\n",
    "# We will have \"state space explosion\" with few amount of texts\n",
    "# Therefore, we need to perform some types of dimensionality reduction. \n",
    "### For example, to use PCA or  similar methods (LSA,LDA,SVD,...) to reduce the dimensionality.\n",
    "# Problems: Scalability and speed, memory,...!\n",
    "\n",
    "# <a></a>\n",
    "# <a></a>\n",
    "# <a></a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<span style=\"color:red;font-size: 3em;\"> Neural embeddings!</span>\n",
    "<hr style=\"height:1px;border:none;color:stillblue;background-color:steelblue;\" />\n",
    "# A game changer idea: do not build the co-occurence matrix explicitly!\n",
    "### Instead learn a classifier that predicts a word given a context around the word or the other way around.\n",
    "### (Bengio et al. 2003) and Mikolov et al., 2013 known as Word2vec\n",
    "\n",
    "### In word2Vec there are two main models\n",
    "### <center> CBOW </center>\n",
    "![](Images/CBOW.png)\n",
    "\n",
    "\n",
    "\n",
    "### <center> Skipgram </center>\n",
    "![](Images/Skip-gram.png)\n",
    "\n",
    "\n",
    "* ** We have v unique words**\n",
    "* **each word is a one-hot v dimensional vector**\n",
    "* **we have Two matrices: W1(vxn) ,W2(nxv)**\n",
    "* **n is the dimension we choose (50-1000)**\n",
    "* **read arrows as dot product**\n",
    "* ** there is an objective function that implies: given a word the network should predict the contexts and vice versa**\n",
    "* ** if one can write a parametric objective function, there are usually several methods to find an optimum value for it**\n",
    "* ** In majorty of machine learning methods they use Stochastic Gradient Descent (SGD)**\n",
    "* **After training the W1 has amazing features**\n",
    "\n",
    "\n",
    "* Think about of SGD as an extension to Least Square Method in Session 3!\n",
    "\n",
    "A visulization of the process: https://ronxin.github.io/wevi/\n",
    "\n",
    "** Details about training: http://www-personal.umich.edu/~ronxin/pdf/w2vexp.pdf**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Just a hint to Gradient Descent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.97610697152e-08\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAHCZJREFUeJzt3X2UHVWZ7/HvzyRGR+S9yWSSSAAj\niA7dhDbg4MUYRHlx3cAERnRG4wxrokNUFNY44PwBzr2sq0sxXlAyRlEiS2W4RCWDwRECDOIsCB3p\nzguI9BBYJOSlRV6HMZL0c/+oSro66ZfT3ae6TtX5fdY661TtqtNnV5/up3c/e9feigjMzKy6XlN0\nBczMLF8O9GZmFedAb2ZWcQ70ZmYV50BvZlZxDvRmZhXnQG9mVnEO9GZmFedAb2ZWcROLrgDA4Ycf\nHjNnziy6GmZmpbJ27drfRkTLcOc1RKCfOXMmHR0dRVfDzKxUJD1Vy3lO3ZiZVZwDvZlZxTnQm5lV\nnAO9mVnFOdCbmVVczYFe0gRJD0u6Pd0/StKDkrol/Yuk16blk9P97vT4zHyqbmZmtRhJi/4S4NHM\n/peAJRHxZuA54KK0/CLgubR8SXqemZkVpKZAL2k6cA7w7XRfwDzg1vSU5cC56fb8dJ/0+Onp+WZm\nlhEBnZ3Jc55qbdF/Dfgc0JvuHwY8HxG70v3NwLR0exrwNEB6/IX0/H4kLZLUIamjp6dnlNU3Myuv\nri5YsCB5ztOwgV7SB4AdEbG2nm8cEcsioj0i2ltahr2D18ysclpbYcWK5DlPtUyBcCrwPyWdDbwO\nOBD4v8DBkiamrfbpwJb0/C3ADGCzpInAQcCzda+5mVnJSdDWlv/7DNuij4grImJ6RMwELgTujoi/\nBO4Bzk9PWwjclm6vTPdJj98dkXcGyszMBjOWcfT/AFwqqZskB39DWn4DcFhafilw+diqaGZmYzGi\n2Ssj4l7g3nT7CWDOAOf8HrigDnUzM6uciKTztbU1Sd2MB98Za2Y2jsZrpE2WA72Z2Tgar5E2WQ2x\n8IiZWbMYr5E2WW7Rm5lVnAO9mVnFOdCbmeVsvOa0GYwDvZlZzooYaZPlQG9mlrMiRtpkedSNmVnO\nihhpk+UWvZlZxTnQm5lVnAO9mVkOih5pk+VAb2aWg6JH2mQ50JuZ5aDokTZZHnVjZpaDokfaZNWy\nZuzrJK2R1CVpo6QvpOU3StokqTN9tKXlknStpG5J6yTNzvsizMxscLW06HcC8yLiZUmTgPsl3ZEe\n+/uIuHWf888CZqWPk4Gl6bOZWWUVsaBIrWpZMzYi4uV0d1L6GKofeT7wvfR1D5AsIj517FU1M2tc\njdT5uq+aOmMlTZDUCewA7oyIB9NDV6fpmSWSJqdl04CnMy/fnJaZmVVWI3W+7qumQB8RuyOiDZgO\nzJH0duAK4DjgHcChJIuF10zSIkkdkjp6enpGWG0zs8ayp/O10dI2MMLhlRHxPHAPcGZEbE3TMzuB\n79K3UPgWYEbmZdPTsn2/1rKIaI+I9paWltHV3szMhlXLqJsWSQen268HzgB+vSfvLknAucCG9CUr\ngY+mo29OAV6IiK251N7MzIZVy6ibqcBySRNI/jDcEhG3S7pbUgsgoBP4RHr+KuBsoBt4Bfjr+lfb\nzKx4jTzSJmvYQB8R64ATByifN8j5ASwee9XMzBrbnpE2K1Y0zs1RA/EUCGZmo9TII22yPAWCmdko\nNdI0B0Nxi97MrOIc6M3MKs6B3sxsBBppQZFaOdCbmY1AI89pMxgHejOzESjLSJssj7oxMxuBsoy0\nyXKL3sys4hzozcyGUcYO2CwHejOzYZSxAzbLgd7MbBhl7IDNcmesmdkwytgBm+UWvZlZxTnQm5kN\noOwdsFkO9GZmAyh7B2xWLUsJvk7SGkldkjZK+kJafpSkByV1S/oXSa9Nyyen+93p8Zn5XoKZWf2V\nvQM2q5YW/U5gXkS0Am3AmelasF8ClkTEm4HngIvS8y8CnkvLl6TnmZmVyp4O2EZeIrBWwwb6SLyc\n7k5KHwHMA25Ny5eTLBAOMD/dJz1+erqAuJmZFaCmHL2kCZI6gR3AncB/As9HxK70lM3AtHR7GvA0\nQHr8BeCwelbazCwPVeqAzaop0EfE7ohoA6YDc4DjxvrGkhZJ6pDU0dPTM9YvZ2Y2ZlXqgM0a0aib\niHgeuAd4J3CwpD03XE0HtqTbW4AZAOnxg4BnB/hayyKiPSLaW1paRll9M7P6qVIHbFYto25aJB2c\nbr8eOAN4lCTgn5+ethC4Ld1eme6THr87omr/CJlZFVWpAzarlikQpgLLJU0g+cNwS0TcLukR4GZJ\n/xt4GLghPf8G4CZJ3cDvgAtzqLeZmdVo2EAfEeuAEwcof4IkX79v+e+BC+pSOzMzGzPfGWtmTa2q\nI22yHOjNrKlVdaRNlgO9mTW1qo60yfJ89GbW1Mo+13wt3KI3M6s4B3ozazrN0AGb5UBvZk2nGTpg\nsxzozazpNEMHbJY7Y82s6TRDB2yWW/RmZhXnQG9mTaHZOmCzHOjNrCk0WwdslgO9mTWFZuuAzXJn\nrJk1hWbrgM1yi97MKquZ8/JZDvRmVlnNnJfPqmUpwRmS7pH0iKSNki5Jy6+StEVSZ/o4O/OaKyR1\nS3pM0vvzvAAzs8E0c14+q5Yc/S7gsoj4laQ3Amsl3ZkeWxIRX8meLOl4kuUD3wb8CXCXpLdExO56\nVtzMbDjNnJfPGrZFHxFbI+JX6fZLJAuDTxviJfOBmyNiZ0RsAroZYMlBMzMbHyPK0UuaSbJ+7INp\n0SclrZP0HUmHpGXTgKczL9vMAH8YJC2S1CGpo6enZ8QVNzMbiDtg91dzoJd0ALAC+ExEvAgsBY4B\n2oCtwDUjeeOIWBYR7RHR3tLSMpKXmpkNyh2w+6sp0EuaRBLkvx8RPwKIiO0RsTsieoFv0Zee2QLM\nyLx8elpmZpY7d8Dur5ZRNwJuAB6NiK9myqdmTjsP2JBurwQulDRZ0lHALGBN/apsZja4PR2wUtE1\naRy1jLo5FfgIsF5SZ1r2eeBDktqAAJ4EPg4QERsl3QI8QjJiZ7FH3JiZFWfYQB8R9wMD/W1cNcRr\nrgauHkO9zMxqFpHk5Ftb3ZIfiO+MNbPScwfs0Bzozaz03AE7NM9eaWal5ztgh+YWvZmVkm+Mqp0D\nvZmVkvPytXOgN7NScl6+ds7Rm1kpOS9fO7fozcwqzoHezErDHbCj40BvZqXhDtjRcaA3s9JwB+zo\nuDPWzErDHbCj4xa9mTUs5+Trw4HezBqWc/L14UBvZg3LOfn6qGWFqRmS7pH0iKSNki5Jyw+VdKek\nx9PnQ9JySbpWUne6cPjsvC/CzKrJq0XVRy0t+l3AZRFxPHAKsFjS8cDlwOqImAWsTvcBziJZPnAW\nsIhkEXEzMyvIsIE+IrZGxK/S7ZeAR4FpwHxgeXracuDcdHs+8L1IPAAcvM/6smZmg3IHbP2NKEcv\naSZwIvAgMCUitqaHtgFT0u1pwNOZl21Oy8zMhuUO2PqrOdBLOgBYAXwmIl7MHouIIFkkvGaSFknq\nkNTR09MzkpeaWYW5A7b+agr0kiaRBPnvR8SP0uLte1Iy6fOOtHwLMCPz8ulpWT8RsSwi2iOivaWl\nZbT1N7OKcQds/dUy6kbADcCjEfHVzKGVwMJ0eyFwW6b8o+nom1OAFzIpHjOz/Tgvn69aWvSnAh8B\n5knqTB9nA18EzpD0OPDedB9gFfAE0A18C7i4/tU2sypxXj5fw851ExH3A4P9E3X6AOcHsHiM9TKz\nJuK8fL48qZmZFc6TleXLUyCYWSGclx8/DvRmVgjn5cePA72ZFcJ5+fHjHL2ZFcJ5+fHjFr2ZjRvn\n5YvhQG9m48Z5+WI40JvZuHFevhjO0ZvZuHFevhhu0ZtZrpyXL54DvZnlynn54jnQm1munJcvnnP0\nZpYr5+WL5xa9mdWd8/KNxYHezOrOefnG4kBvZnXnvHxjqWUpwe9I2iFpQ6bsKklb9llxas+xKyR1\nS3pM0vvzqriZNS6v+9pYamnR3wicOUD5kohoSx+rACQdD1wIvC19zfWSJtSrsmbWuJyXb1zDBvqI\nuA/4XY1fbz5wc0TsjIhNJOvGzhlD/cysJJyXb1xjydF/UtK6NLVzSFo2DXg6c87mtMzMKs55+cY1\n2kC/FDgGaAO2AteM9AtIWiSpQ1JHT0/PKKthZkXKpmucl29cowr0EbE9InZHRC/wLfrSM1uAGZlT\np6dlA32NZRHRHhHtLS0to6mGmRXM6ZpyGFWglzQ1s3sesGdEzkrgQkmTJR0FzALWjK2KZtaonK4p\nh2GnQJD0Q2AucLikzcCVwFxJbUAATwIfB4iIjZJuAR4BdgGLI2J3PlU3s6J5eoNyUDTAWKj29vbo\n6OgouhpmVoOIJFXT2up8fNEkrY2I9uHO852xZjYizsuXjwO9mY2I8/Ll40BvZsPyMMpyc6A3s2E5\nXVNuDvRmNiyna8rNK0yZ2bA8jLLc3KI3swF5NsrqcKA3swE5L18dDvRmNiDn5avDgd7MEp/7HPG6\n19O5cImHUVaMA72ZJa67jq6dx7LgpvlO11SMA72ZwWWXwauv0vqaDaz4yG1O11SMA71ZE4uAzr/8\nMvHVr8Lu3WjSRNqWf9bpmopxoDdrYl0f/hILfvDndJE24RcvLrZClgvfMGXWrC6+mNabl7KCH9JK\nF1xyCVwz4lVBrQTcojdrMhHQueB/EUuXIqCNLvSpT8HXvlZ01SwnwwZ6Sd+RtEPShkzZoZLulPR4\n+nxIWi5J10rqlrRO0uw8K29mI9d13lUs+NGH+9I1f/d3cO21xVbKclVLi/5G4Mx9yi4HVkfELGB1\nug9wFsk6sbOARcDS+lTTzOpi4UJab/sCK1iQpGsWLYLrry+6VpazYQN9RNwH/G6f4vnA8nR7OXBu\npvx7kXgAOHifhcTrynNxmNUmAjrP+Ufie9/rS9dcdBF885tFV83GwWhz9FMiYmu6vQ2Ykm5PA57O\nnLc5LduPpEWSOiR19PT0jKoSnovDrDZdZ13OglV/05euWbgQvv3tYitl42bMnbGRrC4+4jZ1RCyL\niPaIaG9paRnVe3suDrMaXHABrf/2pb50zV/9Fdx4Y9G1snE02kC/fU9KJn3ekZZvAWZkzpueluUi\nOxeH0zhm/UVA57xLiVtv7UvXfOhDcNNNRVfNxtloA/1KYGG6vRC4LVP+0XT0zSnAC5kUT66cxjHL\niKDr3Z9mwT2L+9I1F1wAP/hBsfWyQgx7w5SkHwJzgcMlbQauBL4I3CLpIuAp4C/S01cBZwPdwCvA\nX+dQ5wE5jWOW6u2FD3yA1l/cwQruS9I1550Ht9xSdM2sIIoGyHW0t7dHR0dHXb9mRNK6b231NKvW\nPGJ3L13vupjWB77J3h/7D3wA/vVfi6yW5UTS2ohoH+68yt4Z61SONZ3eXrre+QkWPPD3femas85y\nkLfqznXjVI41jQjo6YHzz6f1oV+wgjVJuuaMM+CnPy26dtYAKtui94gcawq9vcT/OI3OKe8jfvGL\nvtE18+bBz37mvKUBFQ70WU7jWCXt3g0nn0zXL19iASv60jWnnQZ33gmvaYpfb6tBZVM3WU7jWOXs\n2gWnnAJr19IKfTdDzZ0Lq1c7yFs/TfHT4DSOVcbu3bBpEzH7JDrX7iIgSde0T0I9PXD33Q7ytp+m\n+4lwGsdK6w9/gNmz4eij6VqvvnTNSSfBAw/A4Yc7J28DarpAn03juHVvpbFzJ3FCK53rRACtdCXp\nmhNIgvyECUXX0BpY0wX6bBrHrXsrhZdegmOPpeuxyXtb8QLaZk9AaztgYlN0tdkYNPVPiDtpreG9\n+GKSknn1VVp5KmnFH7sT7ngCjjzS+XirSVP/lLiT1hrWzp3w0EPEscfR+erxfZ2uRz6PujrhqKMc\n5K1m/klJOY1jDePFF+G442DOHLq2HdHX6TppEqxfD5MnF11DKxkH+pQ7aa0hbNtGHPNmOp88qH+n\n69Qe+O1v4Y1vLLqGVkIO9Cl30lqhXnkFVq2CadPo+u2f9O90PeZl9Phv4MADi66llZQD/QDcSWvj\n5uWX4Y47YOZMOOcc6O3ta8Uf/gysWQOPPAJveEPRNbUSa+pRN4PZ07oHz2tvOXr5ZTj2WHjmGQLo\nopVWupJW/IQNsH4z/PEfF11Lq4AxteglPSlpvaROSR1p2aGS7pT0ePp8SH2qWgyncSwXL7wA110H\nzzwDJEF+b6drSws89ZSDvNVNPVI374mItswqJ5cDqyNiFrA63S8td9Ja3a1bB9OnE5//PJ209nW6\nHvAxWn/+FXjySZg2rehaWoXkkaOfDyxPt5cD5+bwHuPGnbRWN88+C0uXJq2Gl1/u14rXwQfT9vj/\nQ2e8F/7oj4quqVXMWAN9AD+XtFbSorRsSkRsTbe3AVMGeqGkRZI6JHX09PSMsRrjw617G7XHHoOZ\nM4mLL+7fimcBrYduhg0bnKqx3Iw10L8rImYDZwGLJZ2WPRjJyuMDhsKIWBYR7RHR3tLSMsZqjA+3\n7m3E1q+Hj38cTjhh/1Y80HbjZ9GmJ5yqsVyNKdBHxJb0eQfwY2AOsF3SVID0ecdYK9mI3Lq3IW3Z\nAldeCSecQCxbRucf3tq/Ff/aXyethIULPT7ecjfqQC/pDZLeuGcbeB+wAVgJLExPWwjcNtZKNiK3\n7m1QDz0ExxwD//RPAPu34t/QjTZuSFr5ZuNgLC36KcD9krqANcBPI+JnwBeBMyQ9Drw33a80t+4N\ngE2b4NOfJk4+hc6dx+3NWe5txb/uN/DP/5yMqnnzm4usqTWZUd8wFRFPAPvdOxoRzwKnj6VSZZO9\nwaqzM2ndr1jRV2YV95vfwFe+AjfcAL29e1vwK1hAG11o4kTarroA/vZv4Ygjiq6tNSHfGVtn+7bu\nfVdthT3yCFx/PSxdSqQBvpWuvhY8XclMk7/8ZbLcn1lBPNdNnTl33ySWLIG3vx2+8Y1+rfi9eXi6\n0GWXQXe3g7wVzoE+R87dV9BNN8Hxx8OllxIR+4+JpwtOPx3uvz9J50yfXnSNzRzo8zRU696Bv2T+\n4z+gvZ346EfpfPS1eych69eKf8169N3vwl13wamnFl1js70c6MfJvlMfO61TEvfcAyefnATutWv7\nBfe9rXith8WLk5z9xz5WdI3N9uNAP06yrXtwWqfh3XEH/Omfwrx5xJo1A6ZoBLSdexRa1wVf/3oy\n5bBZA3KgL8hgaR0H/YJ97nNw4IHE2WfTuWHCwCkautCcOfDww/DjHyedsmYNzIG+AWRb907pjLMI\nuPHG5AamCRPgy1+Gl14aOEVD+qFcfjk8+KBvlLDSUDRA07G9vT06OjqKrkZDyI69B4/Dz01vL3zw\ng/CTn8CuXQD9VnmC/is+AUmu/ppr3NFqDUPS2sxaIINyi77BOKWTo927k/lnDj00ab3feiuxa9fe\n/PuAKRpIcvWrV8MDDzjIWyk50DewwVI6Dvoj9F//BSeeCBMnwpVXEs89N2Bw75eimTgR3vEOuP32\nZEWoefOKvgqzUXOgb2DZ1n1pgv6OHUlQPOyw5HlHQbNUb9oEBxyQfPMOOIDo7Bw2uAtom/xr9KlP\nwauvwpo1cM45xdTfrJ4iovDHSSedFFa73t6Ihx/uez766P77vb0FVOrd745I/t4M/Jg7N9/37+mJ\nmD2733v2QjxM697no+nut9+brd9BB0UsW1bQN89sdICOqCHGujO25LKdt3ta+tmW/7h05I7kDcby\n8/bqq0lL/eGHoaUlucBLL+3/5enrRM3OIrlnv1/nKiT/eVxxRfJ13ONtJVNrZ2zhrflwi75uCmnp\nD9WKH+xRi1WrIubMSVrahx0WceSRg369EbXc9zxOPTXi97/P4RtiNn6osUWfW45e0pmSHpPULeny\nvN7H+ox7Tn/u3NG9bupUmDQpqehgj7PPTnLkL7wAzz4LTz219+UBe/PtMEzOPduCP/jgpM8gIpl0\nbPLkUV+6WZnkEuglTQC+QbJo+PHAhyQdn8d72cBGGvRH9Qfg3/99dJXbtm3v2PWhZAN6djsb2GGA\naQmywX3KFHjmmeTCnnsuSfmYNZm8WvRzgO6IeCIi/gDcDMzP6b1sGLUE/aFm1xzwj0AdR9PUEtCH\nulN1v+D++tfDxo1JhbdtS/6DMGtieQX6acDTmf3NaZkVbLCgP9TsmgP+F/DBCwcN0CPdriWgD9lq\nB7j66r6/Sq+8kswZb2aJWhL5I30A5wPfzux/BPj6PucsAjqAjje96U25dljYyGU7cAfs5D3wtH4d\nn2PZznaaDtmBmn382Z9FPPFExH//d9HfKrPCUOTwSknvBK6KiPen+1ekf1T+z0Dne3hleewdzvnZ\neXDvPQPODTPS7WEHNUrw1rfCW96StNQ/8QmYMSOX6zMrk1qHV+YV6CcCvwFOB7YADwEfjoiNA53v\nQF9CO3YkHZ15OuigZG6aT3863/cxK6laA/3EPN48InZJ+iTwb8AE4DuDBXkrqSOOqN/XOvPMZKEP\nM8tFLoEeICJWAavy+vrWAN797tENsWxrS+5uNbNx4UnNbPTuvXd0r3OQNxtXDvQ2NiPt48mhT8jM\nhuZAb2MXAe95z9DnnH66g7xZQRzorT7uvjsJ5Nu3J0H/0EOT5+3bk/K77iq6hmZNK7fOWGtSRxyR\nBH0zaxgNMR+9pB7gqWFPHNjhwG/rWJ0i+VoaU1WupSrXAb6WPY6MiGFn6muIQD8WkjpquWGgDHwt\njakq11KV6wBfy0g5R29mVnEO9GZmFVeFQL+s6ArUka+lMVXlWqpyHeBrGZHS5+jNzGxoVWjRm5nZ\nEEob6CVdIGmjpF5J7fscuyJdlPwxSe8vqo6jIekqSVskdaaPs4uu00hUaVF4SU9KWp9+DqWaR1vS\ndyTtkLQhU3aopDslPZ4+H1JkHWs1yLWU7vdE0gxJ90h6JI1dl6TluX8upQ30wAbgz4H7soXpIuQX\nAm8DzgSuTxcrL5MlEdGWPkozA2hFF4V/T/o5lG0o340kP/9ZlwOrI2IWsDrdL4Mb2f9aoHy/J7uA\nyyLieOAUYHH6+5H751LaQB8Rj0bEYwMcmg/cHBE7I2IT0E2yWLnlz4vCN4iIuA/43T7F84Hl6fZy\n4NxxrdQoDXItpRMRWyPiV+n2S8CjJGtp5/65lDbQD6EKC5N/UtK69F/WUvx7narC9z4rgJ9LWitp\nUdGVqYMpEbE13d4G5LxEWO7K+nuCpJnAicCDjMPn0tCBXtJdkjYM8Ch1K3GY61oKHAO0AVuBawqt\nbHN7V0TMJklFLZZ0WtEVqpd0YekyD7kr7e+JpAOAFcBnIuLF7LG8PpeGntQsIt47ipdtAbIrR09P\nyxpGrdcl6VvA7TlXp54a/ns/EhGxJX3eIenHJKmp+4Z+VUPbLmlqRGyVNBXYUXSFRisitu/ZLtPv\niaRJJEH++xHxo7Q498+loVv0o7QSuFDSZElHAbOANQXXqWbpB73HeSSdzmXxEDBL0lGSXkvSKb6y\n4DqNiqQ3SHrjnm3gfZTrsxjISmBhur0QuK3AuoxJGX9PJAm4AXg0Ir6aOZT751LaG6YknQdcB7QA\nzwOdEfH+9Ng/An9D0sv9mYgozcrTkm4i+Xc0gCeBj2fydw0vHeb2NfoWhb+64CqNiqSjgR+nuxOB\nH5TpWiT9EJhLMjPiduBK4CfALcCbSGaL/YuIaPhOzkGuZS4l+z2R9C7gF8B6oDct/jxJnj7Xz6W0\ngd7MzGpTxdSNmZllONCbmVWcA72ZWcU50JuZVZwDvZlZxTnQm5lVnAO9mVnFOdCbmVXc/wfCfAor\nZ7jMCwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11607c210>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_old = 10\n",
    "x_new = np.random.randint(-10,high=20)\n",
    "\n",
    "eps = .0051\n",
    "precision = .0000001\n",
    "\n",
    "# y = x^2\n",
    "def f(x):\n",
    "    return np.power(x,2)\n",
    "#     return np.power(x,2) -3*np.power(x,3)  + 2\n",
    "\n",
    "def f_deriv(x):\n",
    "    return 2*np.power(x,1)\n",
    "#     return 2*np.power(x,1) - 9*np.power(x,2)\n",
    "\n",
    "\n",
    "\n",
    "counter = 0\n",
    "while abs(f(x_old)-f(x_new))>precision:\n",
    "    x_old = x_new\n",
    "    x_new = x_old - eps*f_deriv(x_old)\n",
    "    plt.plot(x_new,f(x_new),'or',markersize=.015*counter+1.94)\n",
    "    counter = counter + 1\n",
    "print f(x_old)-f(x_new) \n",
    "\n",
    "for x in np.linspace(-10,20,100):\n",
    "    plt.plot(x,f(x),'.b',markersize=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### in principle, in most of machine learning methods, we have a loss function which is differetiable to all the parameters.\n",
    "### Therefore, using the training data we gradually update the parameters toward a direction that minimizes the loss function\n",
    "\n",
    "### we will discuss it in more details later on neural networks\n",
    "### For now let's go back to Word2vec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# After traning the model now we have a dense vector based representation for each world in our corpus. \n",
    "<hr style=\"height:3px;border:none;color:stillblue;background-color:red;\" />\n",
    "\n",
    "## Think about it in terms of data compression!\n",
    "## Example: a small corpus may have around 50K unique words. Now each world can be represented easily with a 50-100 dimensional vector. In a way now we are going back to the first type of object representation!\n",
    "<hr style=\"height:3px;border:none;color:stillblue;background-color:red;\" />\n",
    "\n",
    "\n",
    "## As expected this dense vectors keep a lot of interesting features about the data\n",
    "\n",
    "# Demo: https://rare-technologies.com/word2vec-tutorial/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "#### We use an amazing library called, gensim\n",
    "import gensim\n",
    "from gensim import corpora, models, similarities\n",
    "import logging\n",
    "logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# from gensim.models import word2vec\n",
    "# # get the pretrained vector from https://code.google.com/archive/p/word2vec/\n",
    "# Google_w2v = word2vec.Word2Vec.load_word2vec_format('/Users/SVM/Downloads/GoogleNews-vectors-negative300.bin', binary=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You can load pretrained vectors or train it based on your data set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# print Google_w2v.most_similar(['girl', 'father'], ['boy'], topn=1)\n",
    "# print Google_w2v.most_similar(positive=['woman', 'king'], negative=['man'], topn=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What can be done with Word2vec?\n",
    "## In general the learned densed vectors can be used in any other tasks that require fixed lenght vectors. \n",
    "### Some of interesting applications:\n",
    "* **In Fashion industry:**\n",
    "    * http://multithreaded.stitchfix.com/blog/2015/03/11/word-is-worth-a-thousand-vectors/\n",
    "    * http://developers.lyst.com/2014/11/11/word-embeddings-for-fashion/\n",
    " \n",
    "* ** Any recommendation system**\n",
    "    * **music selection considering user's playlists**\n",
    "    * **Shopping baskets**\n",
    "* **Graphical data: Deep Walk:**\n",
    "    * https://sites.google.com/site/bryanperozzi/projects/deepwalk\n",
    "* **Sentiment Analysis**\n",
    "\n",
    "* ** Consider the flow of people in a city: words could be buildings, street, neighborhoods, regions, etc.**\n",
    "* ** The example of GPS traces of cars in the previous session**\n",
    "* ** A person and his friends**\n",
    "* ** combination of different elements in a floor plan or building system**\n",
    "* ** Combination of different elements in a chemical compound**\n",
    "* ** an ingridient in a food recepie depending on what you have at home**\n",
    "* ** Authors/papers networks**\n",
    "* ** Stock Market with several symbols?**\n",
    "* ...\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Movie Reviews Sentiment Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from sklearn.cross_validation import train_test_split\n",
    "from gensim.models.word2vec import Word2Vec\n",
    "import numpy as np\n",
    "with open('/All_Files/Files/Data/gensim/sample_Data/IMDB_data/pos.txt', 'r') as infile:\n",
    "    pos_tweets = infile.readlines()\n",
    "\n",
    "with open('/All_Files/Files/Data/gensim/sample_Data/IMDB_data/neg.txt', 'r') as infile:\n",
    "    neg_tweets = infile.readlines()\n",
    "    \n",
    "with open('/All_Files/Files/Data/gensim/sample_Data/IMDB_data/unsup.txt','r') as infile:\n",
    "    unsup_reviews = infile.readlines()\n",
    "\n",
    "#use 1 for positive sentiment, 0 for negative\n",
    "y = np.concatenate((np.ones(len(pos_tweets)), np.zeros(len(neg_tweets))))\n",
    "\n",
    "x_train, x_test, y_train, y_test = train_test_split(np.concatenate((pos_tweets, neg_tweets)), y, test_size=0.5)\n",
    "\n",
    "#Do some very minor text preprocessing\n",
    "\n",
    "\n",
    "\n",
    "def cleanText(corpus):\n",
    "    import string\n",
    "    validchars = string.ascii_letters + string.digits + ' '\n",
    "    punctuation = \"\"\".,:;@(){}[]$1234567890\"\"\"\n",
    "    corpus = [z.lower().replace('\\n','') for z in corpus]\n",
    "    corpus = [z.replace('<br />', ' ') for z in corpus]\n",
    "    \n",
    "    for c in punctuation:\n",
    "        corpus =[z.replace(c, '') for z in corpus]\n",
    "    \n",
    "\n",
    "    corpus = [''.join(ch for ch in z if ch in validchars) for z in corpus]\n",
    "    \n",
    "    #treat punctuation as individual words\n",
    "    for c in punctuation:\n",
    "        corpus = [z.replace(c, ' %s '%c) for z in corpus]\n",
    "    corpus = [z.split() for z in corpus]\n",
    "#     corpus = [z.replace(' ', '_') for z in corpus]\n",
    "    return corpus\n",
    "\n",
    "\n",
    "x_train_c = cleanText(x_train)\n",
    "x_test_c = cleanText(x_test)\n",
    "unsup_  = cleanText(unsup_reviews)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train Word2vec model here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2017-11-28 11:26:04,613 : WARNING : consider setting layer size to a multiple of 4 for greater performance\n",
      "2017-11-28 11:26:04,629 : INFO : collecting all words and their counts\n",
      "2017-11-28 11:26:04,630 : INFO : PROGRESS: at sentence #0, processed 0 words, keeping 0 word types\n",
      "2017-11-28 11:26:05,017 : INFO : PROGRESS: at sentence #10000, processed 2276370 words, keeping 67769 word types\n",
      "2017-11-28 11:26:05,384 : INFO : PROGRESS: at sentence #20000, processed 4552346 words, keeping 97737 word types\n",
      "2017-11-28 11:26:05,622 : INFO : collected 110167 word types from a corpus of 5682885 raw words and 25000 sentences\n",
      "2017-11-28 11:26:05,623 : INFO : Loading a fresh vocabulary\n",
      "2017-11-28 11:26:05,721 : INFO : min_count=5 retains 30401 unique words (27% of original 110167, drops 79766)\n",
      "2017-11-28 11:26:05,740 : INFO : min_count=5 leaves 5565097 word corpus (97% of original 5682885, drops 117788)\n",
      "2017-11-28 11:26:05,815 : INFO : deleting the raw counts dictionary of 110167 items\n",
      "2017-11-28 11:26:05,819 : INFO : sample=0.001 downsamples 47 most-common words\n",
      "2017-11-28 11:26:05,820 : INFO : downsampling leaves estimated 4172504 word corpus (75.0% of prior 5565097)\n",
      "2017-11-28 11:26:05,821 : INFO : estimated required memory for 30401 words and 150 dimensions: 51681700 bytes\n",
      "2017-11-28 11:26:05,904 : INFO : resetting layer weights\n",
      "2017-11-28 11:26:06,217 : INFO : training model with 3 workers on 30401 vocabulary and 150 features, using sg=0 hs=0 sample=0.001 negative=5 window=5\n",
      "2017-11-28 11:26:07,221 : INFO : PROGRESS: at 7.16% examples, 1496396 words/s, in_qsize 5, out_qsize 0\n",
      "2017-11-28 11:26:08,226 : INFO : PROGRESS: at 14.10% examples, 1463361 words/s, in_qsize 6, out_qsize 0\n",
      "2017-11-28 11:26:09,229 : INFO : PROGRESS: at 21.25% examples, 1475155 words/s, in_qsize 4, out_qsize 0\n",
      "2017-11-28 11:26:10,231 : INFO : PROGRESS: at 28.42% examples, 1477354 words/s, in_qsize 4, out_qsize 1\n",
      "2017-11-28 11:26:11,231 : INFO : PROGRESS: at 35.39% examples, 1474334 words/s, in_qsize 5, out_qsize 0\n",
      "2017-11-28 11:26:12,231 : INFO : PROGRESS: at 42.42% examples, 1472612 words/s, in_qsize 6, out_qsize 0\n",
      "2017-11-28 11:26:13,232 : INFO : PROGRESS: at 49.44% examples, 1470636 words/s, in_qsize 5, out_qsize 0\n",
      "2017-11-28 11:26:14,233 : INFO : PROGRESS: at 55.92% examples, 1456142 words/s, in_qsize 6, out_qsize 0\n",
      "2017-11-28 11:26:15,239 : INFO : PROGRESS: at 62.09% examples, 1436029 words/s, in_qsize 6, out_qsize 0\n",
      "2017-11-28 11:26:16,245 : INFO : PROGRESS: at 68.94% examples, 1435193 words/s, in_qsize 6, out_qsize 1\n",
      "2017-11-28 11:26:17,248 : INFO : PROGRESS: at 75.85% examples, 1435093 words/s, in_qsize 5, out_qsize 0\n",
      "2017-11-28 11:26:18,250 : INFO : PROGRESS: at 82.15% examples, 1424716 words/s, in_qsize 5, out_qsize 0\n",
      "2017-11-28 11:26:19,254 : INFO : PROGRESS: at 88.39% examples, 1414285 words/s, in_qsize 6, out_qsize 0\n",
      "2017-11-28 11:26:20,258 : INFO : PROGRESS: at 94.80% examples, 1408634 words/s, in_qsize 6, out_qsize 0\n",
      "2017-11-28 11:26:21,033 : INFO : worker thread finished; awaiting finish of 2 more threads\n",
      "2017-11-28 11:26:21,034 : INFO : worker thread finished; awaiting finish of 1 more threads\n",
      "2017-11-28 11:26:21,037 : INFO : worker thread finished; awaiting finish of 0 more threads\n",
      "2017-11-28 11:26:21,037 : INFO : training on 28414425 raw words (20860997 effective words) took 14.8s, 1407770 effective words/s\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "20860997"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n_dim = 150\n",
    "#Initialize model and build vocab\n",
    "imdb_w2v = Word2Vec(size=n_dim)\n",
    "\n",
    "\n",
    "# imdb_w2v.build_vocab(np.concatenate((unsup_,x_train)))\n",
    "imdb_w2v.build_vocab(x_train_c)\n",
    "\n",
    "# Train the model over train_reviews (this may take several minutes)\n",
    "# imdb_w2v.train(np.concatenate((unsup_,x_train)))\n",
    "imdb_w2v.train(x_train_c,total_examples=imdb_w2v.corpus_count,epochs=imdb_w2v.iter)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('book', 0.7908188104629517), ('adaptation', 0.7196881771087646), ('dickens', 0.6851674318313599), ('author', 0.6722975373268127), ('adapted', 0.6581281423568726)]\n"
     ]
    }
   ],
   "source": [
    "print imdb_w2v.most_similar(['novel'], topn=5)\n",
    "# print imdb_w2v.most_similar(positive=['woman', 'king'], negative=['man'], topn=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# To make DocVec based on SOMind\n",
    "### Now the question is how to make prediction at the document level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1- The simple way that works well and it is very fast is to sum all the vectors in a document and train a predictor\n",
    "### 2- Further, you can cluster the vectors and use the cluster vectors as representation of the documents\n",
    "### 3- Train a SOM by word-vectors and then use the activation of this SOM as the representation of the documents (Similar to the idea of WEBSOM)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DEMO of an application for news clustering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Some extensions\n",
    "* **Doc2vec**\n",
    "    * https://arxiv.org/pdf/1405.4053v2.pdf\n",
    "* **Thought vector: In the same direction in other compression applications**\n"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}