Clase 12 - Redes Neuronales Artificiales.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Redes Neuronales Artificiales"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Julián D. Arias Londoño\n",
    "\n",
    "Profesor Titular  \n",
    "Departamento de Ingeniería de Sistemas  \n",
    "Universidad de Antioquia, Medellín, Colombia  \n",
    "julian.ariasl@udea.edu.co"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Una de las principales razones que inspiró la aparición de las redes neuronales artificiales (en inglés Artificial Neural Networks - ANN), es el hecho de que las tareas cognitivas complejas realizadas por el ser humano, requieren un nivel de paralelismo el cual es llevado a cabo a través de una compleja red de neuronas interconectadas."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![alt text](./Images/NN.png \"Neuronas\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Algunas estadísticas interesantes que nos ayudan a entender el poder de la red de neuronas en el ser humano:\n",
    "<li>Se estima que el cerebro humano contiene una densa red de neuronas interconectadas de alrededor de $10^{11}$ neuronas.  </li>\n",
    "<li>Cada neurona se conecta en promedio con otras $10^{4}$ neuronas.</li>\n",
    "<li>La actividad de las neuronas es típicamente exitada o inhibida a través de las conexiones con otras neuronas.</li>\n",
    "<li>La velocidad de cambio de estado de una neurona en el mejor de los casos es del orden de $10^{−3}$ seg. Comparado con los computadores actuales es estramadamente lento. Sin embargo los humanos pueden tomar decisiones sorprendentemente complejas en muy poco tiempo. Por ejemplo Ud requiere aproximadamente $10^{−1}$ segs reconocer visualmente a su madre.</li>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## El perceptrón"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Uno de los tipos más usados de RNA está basado en una unidad llamada Perceptrón. Un perceptrón toma un vector de valores reales como entrada, calcula una combinación lineal de dichas entradas y produce una salida 1 si el resultado es mayor a algún umbral y -1 en otro caso."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![alt text](./Images/Perceptron01.jpg \"Neuronas\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Si observamos con detenimiento, el perceptrón es equivalente a la regresión logística vista en las primeras clases del curso. El umbral que determina si la salida es 1 o -1 es igual al negativo del término independiente en la regresión logística. Formalmente, dada una muestra ${\\bf{x}} =  \\left\\lbrace x_1, x_2,..., x_d \\right\\rbrace \\;\\;$,              la salida $O({\\bf{x}}) = O(x_1,x_2,...,x_d) \\;\\;\\;\\;\\;$     computada por el perceptrón es:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$O(x_1,x_2,...,x_d) = \\left\\{\n",
    "                \\begin{array}{ll}\n",
    "                  1\\;\\;\\;{\\rm{si}}\\;\\;\\;w_0  + w_1 x_1  + w_2 x_2  +  \\cdots  + w_d x_d  > 0\\\\\n",
    "                  - 1\\;\\;{\\rm{\\text{en otro caso}}}\n",
    "                \\end{array}\n",
    "              \\right. $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "donde $w_i$ es una constante real o peso que determina la contribución de la entrada $x_i$ a la salida del perceptrón.\n",
    "<b>Note</b> que $−w_ 0$ es el umbral que la combinación sopesada de entradas $w_1x_1 + w_2x_2 + \\cdots + w_d x_d \\;\\;\\;\\;$ debe sobrepasar para que la salida del perceptrón sea 1."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Una forma alternativa de pensar el perceptrón, y que para nosotros es familiar, es asumir que existe una entrada adicional con valor constante 1, $x_0 = 1$, de tal forma que la inecuación anterior se puede escribir como:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\sum_{i=0}^{d} w_ix_i > 0$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "o en forma matricial, simplemente ${\\bf{w}}^T{\\bf{x}} > 0$. La determinación del valor a la salida del perceptrón se representa normalmente como la función signo (<i>sgn</i>), entonces la función de salida del perceptrón se puede reescribir como:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$O({\\bf{x}}) = sgn({\\bf{w}}^T{\\bf{x}})$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "donde"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$sgn(u) = \\left\\{\n",
    "                \\begin{array}{ll}\n",
    "                  1\\;\\;\\;{\\rm{si}}\\;\\;\\;u  > 0\\\\\n",
    "                  - 1\\;\\;{\\rm{\\text{en otro caso}}}\n",
    "                \\end{array}\n",
    "              \\right. $$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El perceptrón puede se visto como la representación de una superficie de decisión en el espacio $d$-dimensional en el cual se ubican las muestras. El perceptrón asigna valores 1 para las muestras (puntos) que se ubican a un lado del hiperplano y −1 para las que se ubican al otro lado."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![alt text](./Images/Perceptron2.png \"Neuronas\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Entrenamiento del perceptrón"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Una forma de entrenar un vector de pesos es comenzar con pesos aleatorios, aplicar iterativamente el perceptrón a cada muestra de entrenamiento modificando los pesos cada vez que una muestra sea mal clasificada y repitiendo el procedimiento tantas veces como sea posible hasta que todas las muestras sean bien clasificadas. Es decir aplicar la regla:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$w_i \\leftarrow w_i + \\Delta w_i$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "donde"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\\Delta w_i = \\eta (y_j - O({\\bf{x}}_j))x _{ji}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "donde $y_j$ es la salida deceseada para la muestra ${\\bf{x}}_j$. Sin embargo este procedimiento solo funciona cuando las muestras son linealmente separales."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Una forma alternativa que puede ser utilizada aún cuando las muestras no sean linealmente separables, es definir una medida de error e intentar minizarla a través de una regla de gradiente. Es decir, si asumimos:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "podemos utilizar la regla del gradiente descedente para encontrar el conjunto de pesos que hagan el error mínimo (no tiene que ser cero). Esta definición ya fue estudiada cuando vimos el modelo de regresión logística. El único inconveniente es que la salida del percetrón depende de la función <i>sgn</i> y la regla del gradiente necesita calcular la derivada. Sin embargo podemos asumir que si ${\\bf{w}}^T{\\bf{x}}\\;\\;$ tiende a 1 (o −1 según sea del caso), la aplicación de la función sgn obtendrá por consiguiente un buen resultado."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consideremos ahora el tipo de problemas que podremos resolver utilizando un solo perceptron."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![alt text](./Images/ORANDXOR.png \"ORANDXOR\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La gráfica anterior nos muestra 3 problemas de clasifiación haciendo una analogía a las funciones Booleanas OR, AND y XOR. Es posible observar que los dos primeros problemas de clasificación se pueden resolver a partir de una frontera de decisión lineal, sin embargo, en la última gráfica no es posible separa las círculos negros de los círculos blancos utilizando únicamente una función lineal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Como sabemos, la función XOR se puede construir a partir de funciones AND y funciones OR. De la misma forma podemos resolver el problema de la gráfica 3 utilizando la combinación de dos fronteras de decisión lineales:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![alt text](./Images/XOR.png \"XOR\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Si utilizamos la combinación de perceptrones mostrada en la figura, podremos obtener una salida del modelo $a=1\\;\\;$ para la zona sombreada y una salida $a= 0\\;\\;$ para el resto del espacio, obteniendo de esa manera una clasificación perfecta. La red de perceptrones interconectados de la figura anterior coresponde entonces a una <b>Red Neuronal Artificial</b>, en la que cada neurona es un perceptrón y la combinación de los perceptrones, que son modelos simples, permitió la solución de un problema más complejo."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El problema que surge en este momento, es que el algoritmo de entrenamiento que fue descrito para un único perceptrón no puede ser aplicado en este caso, debido a que ahora en la salida de toda la red intervienen varios perceptrones, razón por la cual es necesario aplicar la función <i>sgn</i> antes de poder obtener la salida final de la red y por consiguiente el método de gradiente no puede ser aplicado porque la función de salida de la red nuevamente es NO derivable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para poder resolver el problema, debemos entonces encontrar una función equivalente a la función <i>sgn</i>, pero que pueda ser derivable para poder usar el error cuadrático medio como criterio de entrenamiento. De manera similar al a regresión logística, una solución es usar la función sigmoide:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "f(u) = \\frac{\\exp(u)}{1 + \\exp(u)}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x3134710>]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXUAAAD9CAYAAABDaefJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAG5JJREFUeJzt3Xt0lPWdx/F3IBERFVAgkJloaBKSyCW4JiBF2ohgWE4J\nWrFGux6KSLPZUtddt9qjp23oFkpEz+4paVfkoniLysUE2iRg0PGCkriA4gHEoKBJhCzhXqOGDM/+\n8UgkkNuQzPxmnvm8znnOZDLPTL7OGT/+/M7v93siLMuyEBERR+hhugAREek+CnUREQdRqIuIOIhC\nXUTEQRTqIiIOolAXEXGQDkP9nnvuITo6mpEjR7Z5zn333UdiYiKpqals3769WwsUEZHO6zDUZ82a\nRVlZWZuPl5SUsHfvXqqqqnjyySfJzc3t1gJFRKTzOgz1CRMm0L9//zYfX7duHTNnzgRg7NixHDt2\njLq6uu6rUEREOq3LPfXa2lpiY2Ob77vdbmpqarr6siIicgEiu+NFzt1pICIi4rxzWvudiIh0zJfd\nXLoc6i6Xi+rq6ub7NTU1uFyuLhcm7cvLyyMvL890GY6g97J9lgU1NfDRR7BnD3z8Mezf/91x+jS4\nXPYxeDB8+mkeWVl5DBwIV14JV1wB/ftDv37Qty9cdhn07Gn4HyqE+Dog7nKoZ2VlUVBQQHZ2Nlu2\nbKFfv35ER0d39WVFxADLskP73Xdh2zZ4/3344AO45BJISrKPYcPgxhth6FC4+mo7sM+WlwcPP2yk\nfKEToX7nnXfyxhtvUF9fT2xsLPPmzePUqVMA5OTkMHXqVEpKSkhISKBPnz489dRTfi9aRLqHZcGu\nXbBxI7z2mh3ml10G48ZBWhpMnw6pqTBggOlKpbMiArX1bkREhNov3cjj8ZCRkWG6DEcIt/eyqQk8\nHli1Cv72N4iKgsxMuOkmGD8eYmK69vrh9n76m6/ZqVAXCRP/+7+wfDmsWQNxcXD77fZIPDERNI8h\nePmand0y+0VEglNDAzz/PDzxBBw5AnPmQEWF3Q8XZ9JIXcSBTp6EP/8Z/vu/YexYyM2Fm2+GHtrt\nKeRopC4Sxr76Cv7rv+xj8mTYtAmGDzddlQSSQl3EASwLiorg3/8drrsONm+2px5K+FGoi4S4ffsg\nJwdqa2HZMnsWi4QvddhEQpRlwTPPwJgxMGmSvVBIgS4aqYuEoCNH4J//2V44VF5uLxASAY3URULO\nRx/Zo/PBg+G99xTo0pJCXSSEbNwIP/gBPPII/OlP0Lu36Yok2Kj9IhIi/ud/YN48WL3aDnaR1ijU\nRULAY4/Zq0I3b4b4eNPVSDBTqIsEuUWLYMkSexMut9t0NRLs1FMXCWKLFsGTTyrQpfMU6iJB6okn\n7EB//XUFunSeNvQSCUIbNsDMmeqhizb0Egl5O3fC3Xfb+54r0MVXar+IBJG6OvjRj+Dxx2HCBNPV\nSChS+0UkSHi9MHGiHeZ/+IPpaiRY+JqdGqmLBIn58yEy0l5gJHKh1FMXCQKbN8Nf/gLbtkHPnqar\nkVCmkbqIYUePwk9/CkuXQkyM6Wok1KmnLmKQZcEdd9g7Lv7pT6arkWCkKY0iIWTNGnsK4zPPmK5E\nnEIjdRFDjh+3Lwr94otwww2mq5Fg5Wt2KtRFDPnlL+Hrr+1eukhb1H4RCQGVlfa+6Dt3mq5EnEaz\nX0QCrKkJcnLsPdKvuMJ0NeI0CnWRAPvLX+DKK+Guu0xXIk6knrpIAB0/DsOGQXk5jBxpuhoJBdom\nQCSILVoEU6cq0MV/NFIXCZAvvrDDfPt2uOoq09VIqNCURpEg9fOfQ79+8OijpiuRUKIpjSJBaPdu\nKCqCPXtMVyJOp566SAA88gg8+CD072+6EnE6tV9E/OzDD+Hmm+HTT6F3b9PVSKjp9tkvZWVlJCcn\nk5iYSH5+/nmP19fXM2XKFEaPHs2IESN4+umnfSpYxOkWLoT771egS2C0O1L3er0kJSVRXl6Oy+Ui\nPT2dwsJCUlJSms/Jy8vjm2++4Y9//CP19fUkJSVRV1dHZGTLdr1G6hKOPvkExo61R+mXX266GglF\n3TpSr6ysJCEhgbi4OKKiosjOzqa4uLjFOUOGDOHEiRMAnDhxgiuvvPK8QBcJV48+Crm5CnQJnHbT\nt7a2ltjY2Ob7brebioqKFufMmTOHiRMnEhMTw8mTJ3n55ZfbfL28vLzmnzMyMsjIyLiwqkVCQG0t\nrFoFH39suhIJJR6PB4/Hc8HPbzfUIyIiOnyBBQsWMHr0aDweD5988gmTJ0/mgw8+4LLLLjvv3LND\nXcTpHn8cZs6EAQNMVyKh5NwB7zwfr0TebvvF5XJRXV3dfL+6uhq3293inHfeeYfbb78dgPj4eIYO\nHcoeTcaVMHf4MDz9NDzwgOlKJNy0G+ppaWlUVVWxf/9+Ghsbeemll8jKympxTnJyMuXl5QDU1dWx\nZ88evve97/mvYpEQsGwZTJsG54yBRPyu3fZLZGQkBQUFZGZm4vV6mT17NikpKSxZsgSAnJwcHn74\nYWbNmkVqaiqnT5/m0Ucf5QptEi1hrKnJ3l53zRrTlUg40uIjkW72yiv2rJd33zVdiTiBtt4VMayg\nwL7+qIgJGqmLdKOdO2HSJPjsM7joItPViBNopC5iUEGBvcWuAl1M0UhdpJscOwZDh9qj9ZgY09WI\nU2ikLmLIypUwZYoCXcxSqIt0A8uCJ5+EnBzTlUi4U6iLdIP33oNvvoEf/tB0JRLuFOoi3WDFCpg1\nCzqxXZKIX+mLUpEuamiwtwPYsUPbAkj30xelIgG2di1cf70CXYKDQl2ki5Yvh3vuMV2FiE3tF5Eu\n+OQTe5ReUwO9epmuRpxI7ReRAHr6afjpTxXoEjw0Uhe5QKdPQ1wcrF8PqammqxGn0khdJEDeegv6\n91egS3BRqItcoBdegLvuMl2FSEtqv4hcgMZGe4+XrVvh6qtNVyNOpvaLSABs2AApKQp0CT4KdZEL\noNaLBCu1X0R89Pe/26tH9+6FAQNMVyNOp/aLiJ8VF8MNNyjQJTgp1EV8pNaLBDO1X0R8cOgQJCba\n2wJceqnpaiQcqP0i4kdr19qXrFOgS7BSqIv4YPVq+MlPTFch0ja1X0Q6qb4e4uPhwAG45BLT1Ui4\nUPtFxE+KiiAzU4EuwU2hLtJJq1fDjBmmqxBpn9ovIp1w5AgMHQq1tfqSVAJL7RcRP1i3DiZNUqBL\n8FOoi3SCWi8SKtR+EenAsWNw1VX2gqPLLzddjYQbtV9Eutn69XDjjQp0CQ0KdZEOrF4Nt91mugqR\nzlH7RaQdX34JQ4bAZ5/Z1yMVCbRub7+UlZWRnJxMYmIi+fn5rZ7j8Xi49tprGTFiBBkZGZ3+4yLB\nbuNGGDNGgS6ho92RutfrJSkpifLyclwuF+np6RQWFpKSktJ8zrFjxxg/fjwbNmzA7XZTX1/PgFY2\nmtZIXULRzJmQng5z55quRMJVt47UKysrSUhIIC4ujqioKLKzsykuLm5xzgsvvMBtt92G2+0GaDXQ\nRUJRUxP89a8wfbrpSkQ6r91Qr62tJTY2tvm+2+2mtra2xTlVVVUcOXKEG2+8kbS0NJ599ln/VCoS\nYG+9Za8iPetfAZGgF9negxERER2+wKlTp9i2bRubNm2ioaGBcePGcf3115OYmHjeuXl5ec0/Z2Rk\nqP8uQa2oCG65xXQVEm48Hg8ej+eCn99uqLtcLqqrq5vvV1dXN7dZzoiNjWXAgAH07t2b3r1784Mf\n/IAPPvigw1AXCWaWZYf63/5muhIJN+cOeOfNm+fT89ttv6SlpVFVVcX+/ftpbGzkpZdeIisrq8U5\n06dP5+2338br9dLQ0EBFRQXXXHONT0WIBJv334eoKBg+3HQlIr5pd6QeGRlJQUEBmZmZeL1eZs+e\nTUpKCkuWLAEgJyeH5ORkpkyZwqhRo+jRowdz5sxRqEvIO9N66UQHUiSoaPGRSCtSU+HPf4YbbjBd\niYQ77f0i0kX79tmXrBs3znQlIr5TqIucY/16+NGPoGdP05WI+E6hLnKOdevgnPkAIiFDPXWRsxw/\nbi82OnAA+vQxXY2IeuoiXVJWBhMmKNAldCnURc6i1ouEOrVfRL516hQMHgw7doDLZboaEZvaLyIX\naPNmewMvBbqEMoW6yLfWr1frRUKfQl0EewOv4mKFuoQ+hboI8NFH0Nhobw8gEsoU6iJ8t4pUG3hJ\nqFOoi6CpjOIcmtIoYa++HuLjoa4OLr7YdDUiLWlKo4iPSkth4kQFujiDQl3C3vr1MG2a6SpEuofa\nLxLWGhshOtqe/RIdbboakfOp/SLigzffhKQkBbo4h0JdwppaL+I0CnUJW5alUBfnUahL2Nq1C7xe\nGDnSdCUi3UehLmHrzChdq0jFSRTqErbObA0g4iSa0ihh6dAhSEiA//s/6NXLdDUibdOURpFOKCmB\nyZMV6OI8CnUJS+vWadaLOJPaLxJ2vv7aXmy0dy8MHGi6GpH2qf0i0gGPB0aNUqCLMynUJeyo9SJO\nFmm6AJFAOrOK9NVXTVci4h8aqUtYef996N3b3sRLxIkU6hJWzrRetIpUnEqhLmFl/Xpdi1ScTVMa\nJWzU1tqzXurqIFLfJkmI0JRGkTasWwf/+I8KdHE2hbqEjaIiuPVW01WI+FeHoV5WVkZycjKJiYnk\n5+e3ed57771HZGQka9eu7dYCRbrDsWPw7ruQmWm6EhH/ajfUvV4vc+fOpaysjF27dlFYWMju3btb\nPe+hhx5iypQp6ptLUCothR/+EC691HQlIv7VbqhXVlaSkJBAXFwcUVFRZGdnU1xcfN55ixcvZsaM\nGQzUumsJUkVFcMstpqsQ8b92Q722tpbY2Njm+263m9ra2vPOKS4uJjc3F7C/qRUJJl9/DRs2aGsA\nCQ/tzgPoTEDff//9LFy4sHnaTXvtl7y8vOafMzIyyMjI6HShIhfqtdfs65AOGmS6EpGOeTwePB7P\nBT+/3VB3uVxUV1c336+ursbtdrc4Z+vWrWRnZwNQX19PaWkpUVFRZLWywuPsUBcJFLVeJJScO+Cd\nN2+eT89vd/FRU1MTSUlJbNq0iZiYGMaMGUNhYSEpKSmtnj9r1iymTZvGj3/84/P/kBYfiQFeL7hc\n8Pbb9uXrREKNr9nZ7kg9MjKSgoICMjMz8Xq9zJ49m5SUFJYsWQJATk5O16oV8bOKCnvfdAW6hAtt\nEyCO9uCD9nVI//M/TVcicmG0TYDItywL1qyB224zXYlI4CjUxbG2b7e32E1NNV2JSOAo1MWxVq+G\nGTO0d7qEF4W6OJJlwapVdqiLhBOFujjShx/CqVNw3XWmKxEJLIW6OJJaLxKuFOriSGdCXSTcKNTF\ncXbtgpMnYcwY05WIBJ5CXRxn9Wp7bnoPfbolDOljL46j1ouEM4W6OMrOnXD4MHz/+6YrETFDoS6O\nUlgId96p1ouEr3Z3aRQJJZYFL7xgt19EwpXGM+IYFRVw0UVw7bWmKxExR6EujvHCC3DXXVpwJOFN\n+6mLIzQ12Vc42rxZF8QQZ9F+6hKWXnsN4uIU6CIKdXGEM60XkXCn9ouEvK++gpgYe3uAIUNMVyPS\nvdR+kbDz17/aW+wq0EUU6uIATz0FP/uZ6SpEgoPaLxLSampg1Cj79pJLTFcj0v3UfpGwsnIl/OQn\nCnSRMzRSl5B1+jQkJsKLL0J6uulqRPxDI3UJG2+9ZY/Q09JMVyISPBTqErJWrIB77tG2ACJnU/tF\nQtKJE3DVVVBVBQMHmq5GxH/UfpGw8OKLcNNNCnSRcynUJSQtW2a3XkSkJYW6hJzKSjh0CKZMMV2J\nSPBRqEvIWbwYfvEL6NnTdCUiwUdflEpIqauD5GT45BO44grT1Yj4n74oFUdbuhRmzFCgi7RFI3UJ\nGadOwdChUFJi7/ciEg40UhfHKiqC+HgFukh7FOoSMhYvhrlzTVchEtw6FeplZWUkJyeTmJhIfn7+\neY8///zzpKamMmrUKMaPH8+OHTu6vVAJb9u2wb59cMstpisRCW4d9tS9Xi9JSUmUl5fjcrlIT0+n\nsLCQlJSU5nPeffddrrnmGvr27UtZWRl5eXls2bKl5R9ST1264PbbYfx4uP9+05WIBFa399QrKytJ\nSEggLi6OqKgosrOzKS4ubnHOuHHj6Nu3LwBjx46lpqbGx7JF2vbRR/DmmzBnjulKRIJfZEcn1NbW\nEhsb23zf7XZTUVHR5vnLly9n6tSprT6Wl5fX/HNGRgYZGRmdr1TCVn4+/PKX0KeP6UpE/M/j8eDx\neC74+R2GeoQP+5q+/vrrrFixgs2bN7f6+NmhLtIZn30G69bB3r2mKxEJjHMHvPPmzfPp+R2Gusvl\norq6uvl+dXU1brf7vPN27NjBnDlzKCsro3///j4VIdKWxx6De+8FfaREOqfDL0qbmppISkpi06ZN\nxMTEMGbMmPO+KP3888+ZOHEizz33HNdff33rf0hflIqPzmwJsHs3DB5suhoRM3zNzg5H6pGRkRQU\nFJCZmYnX62X27NmkpKSwZMkSAHJycvj973/P0aNHyc3NBSAqKorKysoL/EcQsT32GNx1lwJdxBfa\nJkCC0uefw7XXwocfQkyM6WpEzPE1OxXqEpR+9jNwuWD+fNOViJjV7e0XkUDbsQNKS+Hjj01XIhJ6\ntPeLBJ1f/xoefhi+Xc8mIj7QSF2Cyuuv2ytIX3nFdCUioUkjdQkap0/Dgw/affRevUxXIxKaFOoS\nNJYsscP8jjtMVyISujT7RYLCgQP2xS88Hhg+3HQ1IsFDUxolJN1xByQkaAqjyLk0pVFCTmkpbN0K\nTz9tuhKR0KdQF6MaGuAXv4AnnoDevU1XIxL61H4Ro/7lX+DkSXj2WdOViAQntV8kZBQVQVkZbN9u\nuhIR59BIXYyoqYHrroPiYmhjt2YRwQ/XKBXpbl4v/NM/wb/+qwJdpLsp1CXg5s+HHj3goYdMVyLi\nPOqpS0CtXg3LlsGWLdCzp+lqRJxHoS4BU1kJubmwcaMufCHiL2q/SEB89hnceiusWGFf0UhE/EOh\nLn535AhMmwYPPGDfioj/KNTFr44cgcmT7ePf/s10NSLOp1AXvzlyBCZNghtvhMceg4gI0xWJOJ9C\nXfziTKDfdBMsWqRAFwkUhbp0u6oqGDcObr4ZHn1UgS4SSAp16Vavvw433AD/8R+wcKECXSTQFOrS\nLSzLvhxddjYUFsKcOaYrEglPWnwkXXb4MPz853bb5a23YNgw0xWJhC+N1KVLNm6E1FQYOtReMapA\nFzFLI3W5IAcPwq9/DZs2wcqV9iwXETFPI3XxSWOjPed8xAgYPBh27VKgiwQTjdSlU775xh6RL1wI\nKSnwzjtqtYgEI4W6tOvECTvMFy2C4cPhuefg+983XZWItEWhLq3autWeorhqld1eWbMG0tNNVyUi\nHVGoS7Ndu+wQX7UK/v53e675rl0wZIjpykSks3Th6TDW0ABvvGFPSywrg5MnYcYMuP12e5l/D32N\nLmKcr9mpUA8TlgVffGHPJX/nHfvYsQP+4R/sPVpuvhmuu05BLhJsfM3ODv8VLisrIzk5mcTERPLz\n81s957777iMxMZHU1FS2b9/e+Wrlgnk8nlZ/b1lQVwdvvglLl9oXppg0CQYOtK84tHQp9O0Lf/gD\nHDhgj9QfecTul4droLf1XsqF0ftpVrs9da/Xy9y5cykvL8flcpGenk5WVhYpKSnN55SUlLB3716q\nqqqoqKggNzeXLVu2+L3wcPTNN3DokH0sW+Zh374MamvtEfjnn8P+/fZx8cWQlPTd8cADMHq0Pa9c\nG2ydz+PxkJGRYboMx9D7aVa7oV5ZWUlCQgJxcXEAZGdnU1xc3CLU161bx8yZMwEYO3Ysx44do66u\njujoaP9VHYQsC5qa4NSp747GRjuIz9x+/fV3R0MDfPWVffvll/YXk2eO48ftqYTHj8PRo/be5EeP\n2s8bMAAGDbL73z17gstlzxufMgXi4uzj8stNvxsiYkq7oV5bW0tsbGzzfbfbTUVFRYfn1NTUtBrq\nU6fat2e3h85tFbX12Jmfz75t7XdnP3bucfp02787fbr1w+tteZw+bYe312vfnjm8XjtkL7oIoqLs\no1cv+/6Zo3dvexTdqxdccol99O4NffrApZfaR3S03R65/HL7tn9/uOIK+7jssu9G2nl59iEicrZ2\nQz2ik/+vfm4Tv63nlZY6+//9vV579P3VV4H5e/PmzQvMHwoDei+7l95Pc9oNdZfLRXV1dfP96upq\n3G53u+fU1NTgcrnOey3NfBER8b925zukpaVRVVXF/v37aWxs5KWXXiIrK6vFOVlZWTzzzDMAbNmy\nhX79+oVdP11EJFi0O1KPjIykoKCAzMxMvF4vs2fPJiUlhSVLlgCQk5PD1KlTKSkpISEhgT59+vDU\nU08FpHAREWmF5Ucvv/yydc0111g9evSwtm7d2uKxBQsWWAkJCVZSUpK1YcMGf5bhSL/73e8sl8tl\njR492ho9erRVWlpquqSQVFpaaiUlJVkJCQnWwoULTZcT8q6++mpr5MiR1ujRo6309HTT5YSUWbNm\nWYMGDbJGjBjR/LvDhw9bkyZNshITE63JkydbR48e7fB1/Brqu3fvtvbs2WNlZGS0CPWdO3daqamp\nVmNjo7Vv3z4rPj7e8nq9/izFcfLy8qzHH3/cdBkhrampyYqPj7f27dtnNTY2WqmpqdauXbtMlxXS\n4uLirMOHD5suIyS9+eab1rZt21qE+q9+9SsrPz/fsizLWrhwofXQQw91+Dp+XUOYnJzMsFY23S4u\nLubOO+8kKiqKuLg4EhISqKys9GcpjmTpy+cuOXsdRlRUVPM6DOkafS4vzIQJE+jfv3+L3529Dmjm\nzJkUFRV1+DpGFoZ/8cUXLWbRuN1uamtrTZQS0hYvXkxqaiqzZ8/m2LFjpssJOa2tsdDnsGsiIiKY\nNGkSaWlpLF261HQ5Ie/shZzR0dHU1dV1+Jwub707efJkDh48eN7vFyxYwLRp0zr9Op2dEx9O2npv\n58+fT25uLr/97W8B+M1vfsMDDzzA8uXLA11iSNNnrvtt3ryZIUOGcOjQISZPnkxycjITJkwwXZYj\nREREdOoz2+VQf/XVV31+Tmfntoe7zr639957r0//ARVbZ9ZhiG+GfLv5/sCBA7n11luprKxUqHdB\ndHQ0Bw8eZPDgwRw4cIBBgwZ1+JyAtV/O7rNlZWXx4osv0tjYyL59+6iqqmLMmDGBKsURDhw40Pzz\nK6+8wsiRIw1WE5o6sw5DOq+hoYGTJ08C8OWXX7Jx40Z9LrsoKyuLlStXArBy5UpuueWWjp/kn+9x\nbWvXrrXcbrd18cUXW9HR0daUKVOaH5s/f74VHx9vJSUlWWVlZf4sw5Huvvtua+TIkdaoUaOs6dOn\nWwcPHjRdUkgqKSmxhg0bZsXHx1sLFiwwXU5I+/TTT63U1FQrNTXVGj58uN5PH2VnZ1tDhgyxoqKi\nLLfbba1YscI6fPiwddNNN/k0pTFgF8kQERH/C9PLIoiIOJNCXUTEQRTqIiIOolAXEXEQhbqIiIMo\n1EVEHOT/ASuDD+blwe6SAAAAAElFTkSuQmCC\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "u=linspace(-10,10,100)\n",
    "g = np.exp(u)/(1 + exp(u))\n",
    "plt.plot(u,g)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La función sigmoide es derivable: $$\\frac{\\partial f(u)}{\\partial w_i} = f(u)(1-f(u))\\frac{\\partial u}{\\partial w_i}$$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "También es posible usar la función tangente hiperbólica:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$f(u) = \\frac{\\exp(u) - \\exp(-u)}{\\exp(u) + \\exp(-u)}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x33d3b90>]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD9CAYAAABQvqc9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFoFJREFUeJzt3XtwVOX9x/HPkkQpiAQENrgbTWUDCRgClUvbmWgqRETa\nFaYXw0ydDMWW4iCdynTGP9oapsKE1v7RH5lpsaMWmJaLrUDshJR02qhtJesljtgACVUwFxK5S4Ga\nkJzfHwsYICHZW54957xfMztkN8/u+bqzfHj8nuc867EsyxIAwBWGmC4AADB4CH0AcBFCHwBchNAH\nABch9AHARQh9AHCRmEP/O9/5jrxer/Ly8vocs3LlSmVnZys/P191dXWxHhIAEKWYQ3/JkiWqqqrq\n8/eVlZU6dOiQGhsb9dxzz2n58uWxHhIAEKWYQ7+goECjRo3q8/cVFRUqKSmRJM2ePVunT59We3t7\nrIcFAEQh4T39lpYWZWZmXrnv9/vV3Nyc6MMCAHqROhgHuXanB4/Hc92Y3h4DAPQvkt10Eh76Pp9P\nTU1NV+43NzfL5/P1OpZtgOKjtLRUpaWlpstwjGR7P8+ckXbvlioqpFdflf73P2nmTOmee6SJE6VA\nQJowQRo3ThqShOvzku39tLtIJ8wJD/1gMKjy8nIVFxdr7969Sk9Pl9frTfRhAUexLGnPHulXv5L+\n8Q+poEAKBqVnnpE+/3mJ/1HGQMUc+osXL9arr76q48ePKzMzU6tXr1ZnZ6ckadmyZXrooYdUWVmp\nQCCg4cOH68UXX4y5aMAtLl6U/vAH6dlnw/dXrZK2b5duucVsXbCvmEN/y5Yt/Y4pLy+P9TCIQGFh\noekSHMXU+1lfL5WUSEOHSj//uTRvnjNm9Hw+zfIky376Ho+Hnj6g8Oz+l78Mz+6feUb63vecEfZI\njEizc1BW7wAYmHPnpIcflrq7pTfflLKyTFcEp0nCc/uAO509K82fL2VmStXVBD4Sg9AHksCZM+Ge\nfU6O9PzzUkqK6YrgVIQ+YNiFC+HAnz5d+s1vknNtPZyDE7mAYU88IX38sbR1KydsETlO5AI2UlkZ\nvrL23XcJfAwOQh8w5OOPpccek7ZskW6wUS0QV7R3AAMsK7yNwpQpUlmZ6WpgZ7R3ABvYtElqbZX+\n9CfTlcBtmOkDg6yzM7wb5qZN4Y3TgFhEmp0sDgMG2aZN4a2PCXyYwEwfGESdndKkSdLGjYQ+4oOZ\nPpDENm+W7rqLwIc5zPSBQcIsH4nATB9IUszykQyY6QODwLKk3FxpwwbpvvtMVwMnYaYPJKG33pK6\nuqR77zVdCdyO0AcGwebN0re/zf46MI/2DpBgnZ2Szye98UZ4fT4QT7R3gCSzZ48UCBD4SA6EPpBg\nmzdLjz5qugogjPYOkEBnzkh33CF98IF0222mq4ET0d4BksjLL0tf+QqBj+RB6AMJRGsHyYb2DpAg\nLS1SXl543/yhQ01XA6eivQMkiaoq6YEHCHwkF0IfSJDq6nDoA8mE9g6QAN3dktcrvfOOlJlpuho4\nGe0dIAnU1UljxhD4SD6EPpAA1dVSUZHpKoDrEfpAAuzZQ+gjOdHTB+Ls/PlwP7+1VRoxwnQ1cDp6\n+oBhr70mTZ9O4CM5EfpAnO3Zw1JNJC9CH4gzTuIimdHTB+Lo6FFpyhTp2DEpJcV0NXADevqAQdXV\n0v33E/hIXoQ+EEevvhoOfSBZEfpAHNXWSl/8oukqgL7R0wfi5OxZKSNDOn1aSkszXQ3cgp4+YMjb\nb0v5+QQ+khuhD8RJba00e7bpKoAbI/SBOAmFpFmzTFcB3BihD8QJoQ87IPSBOGhtlS5ckO66y3Ql\nwI0R+kAcXJ7lezymKwFuLObQr6qqUk5OjrKzs7Vu3brrfl9TU6ORI0dq+vTpmj59up555plYDwkk\nndpaWjuwh9RYntzV1aUVK1bor3/9q3w+n2bOnKlgMKjc3Nyrxt13332qqKiIqVAgmYVC0qpVpqsA\n+hfTTD8UCikQCCgrK0tpaWkqLi7Wrl27rhvHRVdwsu5u6a23mOnDHmKa6be0tCizxzc/+/1+1dbW\nXjXG4/HoX//6l/Lz8+Xz+fTss89q8uTJvb5eaWnplZ8LCwtVWFgYS3nAoDhwIPwl6GPGmK4EblBT\nU6Oampqonx9T6HsGcNbqC1/4gpqamjRs2DDt3r1bCxcuVENDQ69je4Y+YBcs1cRgunZCvHr16oie\nH1N7x+fzqamp6cr9pqYm+f3+q8aMGDFCw4YNkyTNnz9fnZ2dOnnyZCyHBZJKKMSVuLCPmEJ/xowZ\namxs1OHDh9XR0aFt27YpGAxeNaa9vf1KTz8UCsmyLI0ePTqWwwJJhZU7sJOY2jupqakqLy/XvHnz\n1NXVpaVLlyo3N1cbNmyQJC1btkx//OMf9etf/1qpqakaNmyYtm7dGpfCgWTQ0SHV10vTppmuBBgY\ntlYGYrBvn/SNb0gHD5quBG7F1srAINq3T8rLM10FMHCEPhADQh92Q+gDMdi3T5o61XQVwMAR+kAM\nmOnDbjiRC0TpzBnJ55M++UQawvQJhnAiFxgk778vTZ5M4MNe+LgCUaK1Azsi9IEoEfqwI0IfiBKh\nDzviRC4QBcuSRo+WGhqksWNNVwM340QuMAiam6WbbybwYT+EPhAFWjuwK0IfiAKhD7si9IEoEPqw\nK0IfiAKhD7ti9Q4Qoc5OaeRI6fhx6dI3gQLGsHoHSLCGBikzk8CHPRH6QITef1+6+27TVQDRIfSB\nCB04IOXmmq4CiA6hD0TowAEpJ8d0FUB0CH0gQoQ+7IzVO0AEurulESOktrbwn4BprN4BEqipSRo1\nisCHfRH6QAQ4iQu7I/SBCNDPh90R+kAE9u8n9GFvhD4QAWb6sDtCH4gAoQ+7I/SBATp1Sjp3Trr9\ndtOVANEj9IEBOngwPMv3eExXAkSP0AcGiNYOnIDQBwaI0IcTEPrAABH6cAJCHxggQh9OwIZrwAB0\ndob32zlzRrr5ZtPVAJ9hwzUgAQ4dCn9FIoEPuyP0gQFgozU4BaEPDAD9fDgFoQ8MAKEPpyD0gQE4\neFCaONF0FUDsCH2gH5YVDv1Jk0xXAsSO0Af6cfx4eL+dMWNMVwLEjtAH+nG5tcNGa3ACQh/oR0MD\nrR04B6EP9IN+PpyE0Af6wcodOAmhD/SD9g6cJObQr6qqUk5OjrKzs7Vu3bpex6xcuVLZ2dnKz89X\nXV1drIcEBs3Fi9IHH0iBgOlKgPiIKfS7urq0YsUKVVVVqb6+Xlu2bNH+/fuvGlNZWalDhw6psbFR\nzz33nJYvXx5TwcBgOnJEysiQPvc505UA8RFT6IdCIQUCAWVlZSktLU3FxcXatWvXVWMqKipUUlIi\nSZo9e7ZOnz6t9vb2WA4LDBr6+XCa1Fie3NLSoszMzCv3/X6/amtr+x3T3Nwsr9d73euVlpZe+bmw\nsFCFhYWxlAfEjH4+kk1NTY1qamqifn5Moe8Z4NUq127w39fzeoY+kAwOHpQmTzZdBfCZayfEq1ev\njuj5MbV3fD6fmpqartxvamqS3++/4Zjm5mb5fL5YDgsMGmb6cJqYQn/GjBlqbGzU4cOH1dHRoW3b\ntikYDF41JhgMatOmTZKkvXv3Kj09vdfWDpCM6OnDaWJq76Smpqq8vFzz5s1TV1eXli5dqtzcXG3Y\nsEGStGzZMj300EOqrKxUIBDQ8OHD9eKLL8alcCDR/vtf6eRJ6Y47TFcCxA9fjA70oa5OKimR3nvP\ndCVA3/hidCBOGhpo7cB5CH2gD2y0Bici9IE+cBIXTkToA31guSaciBO5QC8sSxo5Ujp8WBo92nQ1\nQN84kQvEQXu7dPPNBD6ch9AHenHggJSTY7oKIP4IfaAXhD6citAHekHow6kIfaAX+/cT+nAmQh/o\nxYEDUm6u6SqA+GPJJnCNc+ekMWPCG66lpJiuBrgxlmwCMWpokLKzCXw4E6EPXIOTuHAyQh+4BqEP\nJyP0gWsQ+nAyQh+4BqEPJ2P1DtBDV5c0YoT08cfSLbeYrgboH6t3gBh89FF4uSaBD6ci9IEeuCgL\nTkfoAz3Qz4fTEfpAD+y5A6cj9IEemOnD6Qh9oAdCH05H6AOXnDghffqplJFhuhIgcQh94JKDB8Oz\nfI/HdCVA4hD6wCW0duAGhD5wyb//LU2ebLoKILEIfeCSffukvDzTVQCJRegDlxD6cANCH5B07Jh0\n4YLk95uuBEgsQh/QZ7N8Vu7A6Qh9QLR24B6EPqBw6E+daroKIPEIfUDM9OEefHMWXK+7W7r1Vqml\nRRo50nQ1QGT45iwgQh9+KN12G4EPdyD04Xq0duAmhD5cj9CHmxD6cD1CH25C6MP1CH24Cat34GoX\nLkijR0tnzkg33WS6GiByrN4BIrB/vxQIEPhwD0IfrkZrB25D6MPVCH24DaEPV2PPHbgNoQ/Xsizp\n3Xel/HzTlQCDJzXaJ548eVKPPPKIjhw5oqysLG3fvl3p6enXjcvKytKtt96qlJQUpaWlKRQKxVQw\nEC9HjkgpKZLPZ7oSYPBEPdMvKytTUVGRGhoaNGfOHJWVlfU6zuPxqKamRnV1dQQ+kkooJM2axRen\nwF2iDv2KigqVlJRIkkpKSrRz584+x7L+HskoFJJmzzZdBTC4om7vtLe3y+v1SpK8Xq/a29t7Hefx\neDR37lylpKRo2bJl+u53v9vna5aWll75ubCwUIWFhdGWB/Srtlbq8ZEDbKGmpkY1NTVRP/+GV+QW\nFRWpra3tusfXrFmjkpISnTp16spjo0eP1smTJ68be/ToUY0fP17Hjh1TUVGR1q9fr4KCgusL4Ypc\nDKKLF6X0dPbQh/1Fmp03nOlXV1f3+Tuv16u2tjZlZGTo6NGjGjduXK/jxo8fL0kaO3asFi1apFAo\n1GvoA4Pp/felO+4g8OE+Uff0g8GgNm7cKEnauHGjFi5ceN2Y8+fP6+zZs5Kkc+fOac+ePcrjShgk\ngcsncQG3iTr0n3rqKVVXV2vixIn629/+pqeeekqS1NraqgULFkiS2traVFBQoGnTpmn27Nn66le/\nqgceeCA+lQMx4CQu3IpdNuFKeXnS734n3XOP6UqA2ESanYQ+XOfsWSkjQzp1it01YX9srQz04513\nwlsvEPhwI0IfrlNby0lcuBehD9dh5Q7cjNCH69TWsnIH7kXow1VaW6Xz56W77jJdCWAGoQ9Xef11\n6ctfZmdNuBehD1eprpaKikxXAZhD6MM1LIvQBwh9uEZDg9TdLeXkmK4EMIfQh2tcnuXTz4ebEfpw\nDVo7AHvvwCU6O6WxY6XGxvCfgFOw9w7Qi9ra8Np8Ah9uR+jDFWjtAGGEPlyhulri+3sAevpwgdOn\npcxM6dgxaehQ09UA8UVPH7jG3/8e3nqBwAcIfbhAVRWtHeAy2jtwtI4O6fbbpbfflu6803Q1QPzR\n3gF6qKyUpkwh8IHLCH042ubN0qOPmq4CSB60d+BYp05JWVnSkSNSerrpaoDEoL0DXPLSS+ETuAQ+\n8BlCH45Fawe4Hu0dONKHH0qzZkktLdJNN5muBkgc2juApN//XvrWtwh84FqEPhzHsmjtAH0h9OE4\nr7wiDRsmzZ5tuhIg+RD6cBTLkkpLwze+FhG4HqEPR3nllXDwB4OmKwGSE6EPx7g8y3/6aWb5QF8I\nfTjG5Vn+ww+brgRIXoQ+HIFZPjAwhD4c4aWXpO5uZvlAf1JNFwDEqrVVWrlSevllZvlAf5jpw9a6\nu6UlS6Tvfz/8lYgAbozQh62tXy998on04x+brgSwBzZcg23t2yfdf7+0d680YYLpagAz2HANrtDS\nIn3969KzzxL4QCQIfdjORx9J990nLV0qlZSYrgawF1bvwFYOHw63dJ54QvrhD01XA9gPM33Yxptv\nSoWF0pNPEvhAtJjpI+l9+mn4atsXXpDKy6VvftN0RYB9EfpIWpYlvf669PjjUna29N57ktdruirA\n3gh9JJ2uLmnnTukXv5COH5d+9jOpuJirbYF4iLqn/9JLL2nKlClKSUnRO++80+e4qqoq5eTkKDs7\nW+vWrYv2cIhATU2N6RIi1tUlvfaatGqVFAiEA/9HP5IOHpQWLzYb+HZ8P5MZ76dZUYd+Xl6eduzY\noXvvvbfPMV1dXVqxYoWqqqpUX1+vLVu2aP/+/dEeEgOU7H+pLEs6dkyqrpbWrpUWLZIyMqQf/EAa\nMSK8h84bb4TX4aekmK42+d9Pu+H9NCvq9k5OTk6/Y0KhkAKBgLKysiRJxcXF2rVrl3Jzc6M9LJKQ\nZUkXLkjnz0vnzklnzkinT4dvx49L7e1SW1t4Y7T//Cd8GzJEmjpVmjkz3Lr5v/+TMjNN/5cAzpfQ\nnn5LS4sye/xN9vv9qq2t7XP8176WyGqSV7S7T/R8Xs+fGxulnm+zZX12u/b+5Vt392d/Xr51dX12\nu3gxfOvsDN86OsKrai7fbrpJGj48fBs5UkpPD99Gjw7P4rOypC99KXz17IQJ4ccBDL4bhn5RUZHa\n2tque3zt2rX62gAS2hNhI/bPf+ZMXbwcOrR6UI93OfxPnpSamgb10INi9erBfT+djvfTnBuGfnV1\ndUwv7vP51NQjAZqamuT3+3sdy2ZrAJB4cbkit6/AnjFjhhobG3X48GF1dHRo27ZtCgaD8TgkACAK\nUYf+jh07lJmZqb1792rBggWaP3++JKm1tVULFiyQJKWmpqq8vFzz5s3T5MmT9cgjj3ASFwBMsgza\nvn27NXnyZGvIkCHW22+/fdXv1q5dawUCAWvSpEnWX/7yF0MV2tfTTz9t+Xw+a9q0ada0adOs3bt3\nmy7Jlnbv3m1NmjTJCgQCVllZmelybO3OO++08vLyrGnTplkzZ840XY7tLFmyxBo3bpx19913X3ns\nxIkT1ty5c63s7GyrqKjIOnXqVL+vY3TDtb7W+tfX12vbtm2qr69XVVWVHn/8cXV3dxuq0p48Ho+e\nfPJJ1dXVqa6uTg8++KDpkmyH60ziy+PxqKamRnV1dQqFQqbLsZ0lS5aoqqrqqsfKyspUVFSkhoYG\nzZkzR2VlZf2+jtHQz8nJ0cSJE697fNeuXVq8eLHS0tKUlZWlQCDAhyQKFifHY9LzOpO0tLQr15kg\nenwmo1dQUKBRo0Zd9VhFRYVKLn2pRElJiXbu3Nnv6yTl1sqtra1XrfLx+/1qaWkxWJE9rV+/Xvn5\n+Vq6dKlOnz5tuhzb6e06Ez6H0fN4PJo7d65mzJih3/72t6bLcYT29nZ5L+1C6PV61d7e3u9zEr7h\nWqxr/S+LdM2/G/T13q5Zs0bLly/XT3/6U0nST37yE61atUrPP//8YJdoa3zm4uuf//ynxo8fr2PH\njqmoqEg5OTkqKCgwXZZjeDyeAX1mEx760az1v3Z9f3Nzs3w+XzzLcoSBvrePPfZYRP/AIiyS60zQ\nv/Hjx0uSxo4dq0WLFikUChH6MfJ6vWpra1NGRoaOHj2qcePG9fucpGnv9Oz1BYNBbd26VR0dHfrw\nww/V2NioWbNmGazOfo4ePXrl5x07digvL89gNfbEdSbxc/78eZ09e1aSdO7cOe3Zs4fPZBwEg0Ft\n3LhRkrRx40YtXLiw/yclbH3RALz88suW3++3hg4danm9XuvBBx+88rs1a9ZYEyZMsCZNmmRVVVUZ\nrNKeHn30USsvL8+aOnWq9fDDD1ttbW2mS7KlyspKa+LEidaECROstWvXmi7Htj744AMrPz/fys/P\nt6ZMmcJ7GYXi4mJr/PjxVlpamuX3+60XXnjBOnHihDVnzpyIlmx6LIvT6QDgFknT3gEAJB6hDwAu\nQugDgIsQ+gDgIoQ+ALgIoQ8ALvL/iVCly6GrW3UAAAAASUVORK5CYII=\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "g = (np.exp(u) - np.exp(-u))/(np.exp(u) + np.exp(-u))\n",
    "plt.plot(u,g)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Si se observa con detenimiento veremos que el rango de valores que toma la función sigmoide está en el intervalo $[0,1]$, mientras que la tangente hiperbólica está entre $[-1,1]\\;$. A las funciones de salida de los perceptrones se les conoce como <b> funciones de activación</b>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Utilizar funciones de activación nolineales, también permite que las fronteras de decisión no estén restringidas a rectas, permitiendo la definición de fronteras cada vez más complejas. Una red neuronal artificial es entonces un modelo que utiliza varias neuronas interconectadas y que puede a partir de ellas resolver problemas complejos como el mostrado en la siguiente figura."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![alt text](./Images/RNA.png \"RNA\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Como podemos observar en la figura, una Red Neuronal Artificial (RNA) tiene tres tipos de capas:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<li>La capa de entrada la cual recibe las características o variables de las muestras a ser evaluadas.  </li>\n",
    "<li>La o las capas ocultas son capas de perceptrones que permiten llevar a cabo la definición de fronteras complejas, como en el caso de la función XOR, en la cual se tenía una capa oculta con 2 perceptrones que llevaban a cabo funciones AND .</li>\n",
    "<li>La capa de salida en la que se encuentran los perceptrones que proporcionan la salida final de la red. Una RNA puede tener varias salidas, que pueden ser usadas en problemas de clasificación multi-clase, o en problemas de regresión en los que existen varias variables a predecir.</li>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----------------------------------------------------------------------------------------------------------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Algoritmo Backpropagation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El algoritmo Backpropagation ajusta (aprende) los pesos de una red multicapa, dada una estructura de red con un conjunto fijo de unidades e interconexiones.\n",
    "\n",
    "El criterio de entrenamiento puede ser ajustado al tipo de problema e incluso pueden usarse diferentes criterios para problemas similares. Para explicar el principio de funcionamiento del algoritmo usaremos como criterio la minimización del error cuadrático medio entre las salidas de la red y los valores deseados (objetivos) para dichas salidas. El algoritmo utiliza un técnica de gradiente descendente para llevar a cabo el entrenamiento de la red."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La principal diferencia en el caso de redes multicapa es que la función de error, utilizada como criterio para el entrenamiento, puede tener múltiples mínimos locales a diferencia de un solo perceptrón en el cual existe un sólo mínimo. Esto significa que desafortunadamente no se puede garantizar la convergencia a un mínimo global."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Los perceptrones multicapa (<b>MultiLayer Perceptrón - MLP</b>) son por definición redes neuronales de propagación hacia adelante ya que la activación de las neuronas se hace desde la entrada (lugar donde se conectan las variables) hacia las neuronas de salida las cuales entregan la predicción deseada."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![alt text](./Images/RNA2.png \"RNA\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para poder describir el algoritmo Backpropagation es necesario ponernos de acuerdo en la notación utilizada:\n",
    "\n",
    "<li> La capa de entrada tendrá tantos nodos como variables (atributos o características), que notaremos $d$.</li>\n",
    "<li> Un nodo es una entrada a la red o una salida de alguna unidad en la red.</li>\n",
    "<li> Cada nodo tendrá un sub-índice que indicará la posición del nodo en la capa y un super-índice que indicará la capa a la cual pertence el nodo. $x_{ji}$ denota la entrada a partir del nodo $i$ a la unidad $j$, y $w_{ji}^{(1)}$ denota el correspondiente peso, el cual corresponde a la capa 1.</li>\n",
    "<li> El número de neuronas o unidades en cada capa es diferente y se denotará por $M_k$, donde $k$ hace referencia a la capa.</li>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "De acuerdo a la notación anterior, en la primera capa oculta se construyen $M_1$ combinaciones lineales de las variables de entrada ${\\bf{x}}=\\{x_1,x_2,...,x_d\\;\\;\\;\\}$ de la forma:\n",
    " \n",
    "$$\n",
    " a_j = \\sum_{i=0}^{d} w_{ji}^{(1)} x_i\n",
    "$$\n",
    "  \n",
    "donde $j=1,...,M_1\\;\\;$ y el superíndice $(1)$ indica que los parámetros corresponden a la primera \"capa\" oculta de la red. Las activaciones $a_j$ luego se transforman usando una función de activación $h(\\cdot)\\;\\;$ no lineal para dar:\n",
    "  \n",
    "$$\n",
    " z_j = h(a_j)\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Estos valores se combinan linealmente de nuevo para dar unidades de activación en otras capas ocultas o de salida\n",
    "\n",
    "$$\n",
    "a_k = \\sum_{j=0}^{M_1} w_{kj}^{(2)} z_j\n",
    "$$\n",
    "\n",
    "donde $k=1,...,M_2\\;\\;\\;$. Si la red sólo tiene una capa oculta, $M_2\\;\\;$ será el número total de salidas."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La diferencia fundamental entre una RNA entrenada para resolver un problema de regresión o un problema de clasificación, está en la función de activación de la capa de salida:\n",
    "\n",
    "<li> Regresión: $y_k = a_k$ </li>\n",
    "<li> Clasificación: $y_k = \\sigma(a_k)$ En problemas de clasificación de más de dos clases se utiliza comúnmente la función de activación softmax. <b>Consultar en qué consiste.</b></li>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Combinando todas estas etapas se obtiene\n",
    "\n",
    "$$\n",
    "y_k({\\bf{x}},{\\bf{w}}) = \\sigma\\left( \\sum_{j=1}^{M_1} w_{kj}^{(2)} h \\left( \\sum_{i=1}^{d} w_{ji}^{(1)} x_i + w_{j0}^{(1)} \\right) + w_{k0}^{(2)}\\right) \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "En la ecuación anterior, todos los parámetros se agrupan en el vector ${\\bf{w}}$. El modelo de red neuronal es una función no lineal de un conjunto de variables de entrada $\\{x_i\\}$ a un conjunto de variables de salida $\\{y_k\\}$ controlado por el vector de parámetros ajustables  ${\\bf{w}}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El problema principal consiste entonces en encontrar valores adecuados para los parámetros de la red dado un conjunto de datos de entrenamiento. Es en este punto en el que necesitamos definir el algoritmo <b>Backpropagation</b>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Como ya sabemos, la regla de actualización a partir del algoritmo de gradiente descendente está dada por:\n",
    "\n",
    "$$\n",
    "{\\bf{w}}{(\\tau + 1)} = {\\bf{w}}{(\\tau)} - \\eta \\nabla E({\\bf{w}}{(\\tau)})\n",
    "$$\n",
    "\n",
    "$\\tau$ denota la iteración del algoritmo."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La función de error está dada por:\n",
    "\n",
    "$$\n",
    "E({\\bf{w}})=\\frac{1}{N}\\sum_{n=1}^{N}E_n({\\bf{w}})\n",
    "$$\n",
    "\n",
    "Esta aproximación hace parte de las técnicas que usan todo el conjunto de entrenamiento simultánemente es decir, calculan el error como la suma de los errores individuales por cada muestra, y se conocen como métodos <mark>Batch</mark> (lote).\n",
    "\n",
    "Si por el contrario la medida de error se calcula para cada muestra de entrenamiento y posteriormente se actualizan los pesos de la red, el algoritmo se conoce como <mark>On-Line</mark> y la regla de actualización está dada por:\n",
    "\n",
    "$$\n",
    "{\\bf{w}}{(\\tau + 1)} = {\\bf{w}}{(\\tau)} - \\eta \\nabla E_n({\\bf{w}}{(\\tau)})\n",
    "$$\n",
    "\n",
    "Cuando se usa la regla de actualización on-line, se suele hablar de <i>gradiente descendente secuencial</i> o <i>gradiente descendente estocástico</i>, ya que las actualizaciones se realizan con una sola muestra a la vez, y las muestras son evaluadas de manera aleatoria por varias razones. En primer lugar para evitar que el algoritmo presente problemas de convergencia si por ejemplo estamos resolviendo un problema de clasificación y las muestras están ordenadas por clase; podría suceder que el algoritmo ajuste la frontera de decisión para reducir el error en la primera clase sacrificando muchas muestras de la segunda clase. Posteriormente cuando comience a evaluar las muestras de la segunda clase moverá la frontera para reducir el error en éstas sin tener en cuenta el error en la primera clase. De esta manera podría permanecer en un círculo vicioso que evitaría la convergencia del algoritmo.\n",
    "\n",
    "Otra razón para el uso de muestras aleatorias en el gradiente descendente estocástico, es debido al hecho de que en problemas en los que el número de muestras es muy grande, no se utilizan todas las muestras en cada época de la red neuronal, sino que se usa un subconjunto de ellas.\n",
    "\n",
    "Actualmente se usa el concepto de <b>minibatch</b> para definir un entrenamiento en el que se el error es acumulado en subconjunto de muestras y con base en él se actualizan los pesos de la red. Es por lo tanto un punto intermedio entre el entrenamiento tipo batch y el on-line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El principal problema con el entrenamiento de una red neuronal es que para calcular el error en cada capa es necesario conocer la salida deseada, y en el caso de las capas ocultas la salida deseada es desconocida, precisamente por esa razón reciben su nombre.\n",
    "\n",
    "El algoritmo Backpropagation es entonces una forma eficiente de evaluar el gradiente de la función de error $E({\\bf{w}})$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "El algoritmo se realiza en dos etapas:\n",
    "\n",
    "<li> Primera etapa: Evaluación de las derivadas de la función de error con respecto a los pesos. </li>\n",
    "<li> Segunda etapa: Las derivadas se emplean para realizar los ajustes de los pesos. </li>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Primera etapa"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lo primero que debe hacer para poder llevar a cabo el entrenamiento es presentarle a la red el vector de entrada (muestra) y calcular las activaciones de las correspondientes unidades ocultas y de salida. Este proceso se conoce como propagación hacia adelante. En general en la red hacia adelante cada unidad calcula una suma ponderada de sus entradas\n",
    "\n",
    "$$\n",
    "a_j = \\sum_i w_{ji}z_i\n",
    "$$\n",
    "\n",
    "que se transforma mediante una función de activación\n",
    "\n",
    "$$\n",
    "z_j = h(a_j)\n",
    "$$\n",
    "\n",
    "Con los valores de salida obtenidos a partir de propagar la red hacia adelante, se puede calcular el error.\n",
    "\n",
    "\n",
    "Teniendo en cuenta la función de error \n",
    "\n",
    "$$\n",
    " E({\\bf{w}})=\\sum_{n=1}^{N}E_n({\\bf{w}})\n",
    "$$\n",
    " \n",
    "la derivada de la función de error requiere resolver el problema de evaluar  $\\nabla E_n({\\bf{w}})$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consideremos el modelo lineal simple en el que las salidas $y_k$ son combinaciones lineales de las variables de entrada\n",
    "\n",
    "$$\n",
    "y_k=\\sum_i w_{ki}x_i\n",
    "$$\n",
    "\n",
    "con una función de error\n",
    "\n",
    "$$\n",
    "E_n = \\frac{1}{2}\\sum_k(y_{nk} - t_{nk})^2\n",
    "$$\n",
    "\n",
    "donde $y _{nk} = y_k({\\bf{x}}_n,{\\bf{w}})\\;\\;\\;$ y $t _{nk}\\;\\;\\;$ es la salida deseada. <b>Note</b> que estamos considerando un modelo que tiene $k$ salidas, razón por la cual el error de una muestra ${\\bf{x}}_n$, corresponde a la suma de los errores proporcionados por cada salida de la red para dicha muestra.  El gradiente de esta función con respecto a $w _{ji}$ está dado como: \n",
    "\n",
    "$$\n",
    "\\frac{\\partial E_n}{\\partial w_{ij}} = \\left( y_{nj} - t_{nj} \\right) x_{ni} \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "que puede interpretarse como un cálculo local que incluye el producto del error $y_{nj} - t_{nj}\\;\\;\\;$ asociada a la salida del enlace $w_{ji}$ y la variable $x_{ni}$ asociada con la entrada de ese enlace."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Si se considera la evaluación de $E_n$ con respecto a un peso $w_{ji}$\n",
    "\n",
    "$$\n",
    "\\frac{\\partial E_n}{\\partial w_{ji}} = \\frac{\\partial E_n}{\\partial a_j} \\frac{\\partial a_j}{\\partial w_{ji}}  \n",
    "$$\n",
    "\n",
    "Se utiliza la notación \n",
    "\n",
    "$$\n",
    "\\delta_j = \\frac{\\partial E_n}{\\partial a_j} \n",
    "$$\n",
    "\n",
    "donde los $\\delta$'s se conocen como errores. Igualmente\n",
    "\n",
    "$$\n",
    "\\frac{\\partial a_j}{\\partial w_{ji}} = z_i \n",
    "$$\n",
    "\n",
    "Haciendo las sustituciones anteriores se obtiene\n",
    "\n",
    "$$\n",
    "\\frac{\\partial E_n}{\\partial w_{ji}} = \\delta_j z_i\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "La ecuación anterior indica que la derivada requerida se obtiene multiplicando el valor de $\\delta$ de la unidad en el lado de salida del peso, por el valor de $z$ para la unidad en el lado de la entrada del peso.\n",
    "\n",
    "Como se ha visto, para las unidades de salida se tiene\n",
    "\n",
    "$$\n",
    "\\delta_k = y_k - t_k\n",
    "$$\n",
    "\n",
    "mientras que para evaluar los $\\delta$'s para los nodos ocultos, se emplea la regla de la cadena para derivadas parciales\n",
    "\n",
    "$$\n",
    "\\delta_j \\equiv \\frac{\\partial E_n}{\\partial a_j}  = \\sum_k \\frac{\\partial E_n}{\\partial a_k} \\frac{\\partial a_k}{\\partial a_j}  \n",
    "$$\n",
    "\n",
    "la cual evalúa todos los nodos $k$ a los cuales la unidad $j$ envía una conexión."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Haciendo las sustituciones adecuadas\n",
    "\n",
    "$$\n",
    "\\delta_j = \\dot{h}(a_j)\\sum_k w_{kj}\\delta_k\n",
    "$$\n",
    "\n",
    "que expresa que el valor de $\\delta$ para una unidad escondida en particular puede obtenerse propagando los $\\delta$'s hacia atrás desde unidades más altas (más cercanas a la salida) en la red. El procedimiento de propagación hacia atrás se puede resumir como:\n",
    "\n",
    "<li> Aplicar un vector de entrada ${\\bf{x}}_n$ a la red y propagarlo hacia adelante. </li>\n",
    "<li> Evaluar todos los $\\delta_k$ para las unidades de salida. </li>\n",
    "<li> Propagar los $\\delta$'s del paso anterior para obtener los $\\delta_j$ de cada nodo oculto en la red. </li>\n",
    "<li> Usar $\\partial E_n/ w_{ji} = \\delta_j z_i \\;\\;$ para evaluar las derivadas requeridas. </li>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Para los métodos basados en entrenamiento por lotes, la derivada del error se calcula como:\n",
    "\n",
    "$$\n",
    "\\frac{\\partial E}{\\partial w_{ji}} = \\sum_n \\frac{\\partial E_n}{\\partial w_{ji}}\n",
    "$$\n",
    "\n",
    "Todo el algoritmo de entrenamiento puede ser usado para múltiples capas ocultas y múltiples salidas. Tanto en problemas de regresión como en problemas de clasificación, cuya única diferencia desde el punto de vista del entrenamiento será el cálculo de los $\\delta$'s para la capa de salida."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ej: Uso de la librería neurolab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1500, 2)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEXCAYAAACDChKsAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOydZ5QU1RKAv5q8mZyzoIhKFkVUzIo5oJgT5vdUDM+MKCqYxaei8hSzgglzwoQJlSRRkJwks7BpdlLX+9EDbpjdnd3p2djfOX12p/veujU7s119b9WtElXFxsbGxsamsjhqWgEbGxsbm7qJbUBsbGxsbKqEbUBsbGxsbKqEbUBsbGxsbKqEbUBsbGxsbKqEbUBsbGxsbKqEbUBs6iwioiIyNNE2tQkR+UREXq5pPUoiIi+LyCc1rYdN7cI2IDZJJXrj0egREpEVIvKoiKTVtG71FRG5XURmiEiOiGwRkY9FZN+a1sum/mEbEJvq4GugNdAFuAu4Bni0rMYi4q4mveorhwHjgYOAI4Aw8LWINKlJpWzqH7YBsakOAqq6UVXXquqbwBvAqQAiclh0dnK8iPwuIkHg2Oi1K0VkmYgEoz8vjyG7lYh8KiIFIrJaRM4vTxERaSsik0QkO3p8KiLdily/R0QWiMhFIrJKRPJF5CUR8YjINSKyVkS2icjjIuIo0u/86FN/rohsFpF3RKRtBbqkRmdoeSKySUTuiNHGIyIPici66HucISLHlidXVY9V1ZdUdYGqzgcuAJoDgyrQ50AR+Tb6nndGf29TRtvjROTH6N9wu4h8KSJ7l2hzd/QzCYjIRhF5tcg1EZFbRGS5iPhFZH7Jz668/ja1A9uA2NQEfqDkLOMhzNlJd+A3ETkNeBoYB+wLPAmMF5GTSvS7F/gI6A1MAF4Vkf6xBhWRVOA7oBAYDAwENmA+nacWadoJOAU4ETgdODM6xv7AMcBlwLXAaUX6eIBRQK9ov2bAWxX8HR4FjgbOAI4E+gCHlmjzUlTXc6N/h1eAj0WkVwWyi5KB+b+eXVaDqLzvgGWYhuZAYDLgKqNLGuZnMwBzxrMzqpcnKu8M4GbM2WY3zL/J70X63w8MB/4F9ADGAs+LyAlx9repDaiqfdhH0g7gZeCTIq8HAFuBydHXhwEKnFGi38/AxBiyfiryWoH/lWjzNfB6iTZDo79fCiwFpMh1J7ANOCv6+h5MA5dVpM27wBbAU+Tc98DT5bzv7tGx25VxPR0IAOeVOLcDeDn6eg/AADqU6PsBML4Sn8HbwBzAWU6bN4Dp8X6OMa6nARHg4OjrG4ElgLuMtn7gkBLnxwGfVdTfPmrPYc9AbKqD46LLNIXAdOAHzCf4osws8XpvTCNSlJ8wn1aLMj3G65JtdtEP6AzkRvXJw3xybox5s97FGlXdWeT1JuAvVQ2WONdi1wsR6SsiH0aXXHKLvJ8OZeiyB+asZbf+qpoHzC/Spi8gwKJd+kZ1PqGEvmUiIo8DB2Ma6Eg5TfsA38YjMyp3DxF5M7oElYP593Dwz/t9B/ABK0XkRRE5U0S80Ws9ote+KPG+ri7yvsrrb1NLKGt6amNjJT8AVwAh4G9VDcVokx+nrETSRzuAP4CzY1zbXuT3kvppGeecAGJGlH2JOfu5ANiMuYT1I6aRSERfxVw6Kzm+v6LOIvIE5ns9XFVXJKBHLD4B1gFXAusxHfWLiL5fVV0rInthLs0dBTwGjBKRA/hn6fwkYE0JuaGK+qtqvN8VmyRjGxCb6qBAVZdVss+fmGvxLxY5dzDmTaooBwITS7z+swyZs4FzgK2quqOS+pRHd0yDcYeqrgQQkdMr6LMc82Z5ILAi2icN08+xPNpmDuYMpJWqflcZhUTkSWAYpvFYHEeXOZgRW/HIbor5nq/ZpZeI9KXE/URVC4FPgU9F5EFgI+ZnOh1z+a6jqpY56ymn/1fx6GmTfGwDYlNbeQR4R0RmYd4wjgPOw3RqF+V0EZmB6ZMYivnEekAZMt/AdMx+KCJ3Yz79tsd0mD+nqkurqOsazBviv0XkGczlt/vK66CqeSLyIvCQiGwB/gbuJjqribb5S0TeAF4WkZswDWATTL/RClV9P5bsqA4XYEa6ZYtIq+ilvOgyWSweAX4VkQnAM5iBBocAX6lqyVlCNqYf63IRWQu0jfYPF9HhYsz7y29AHqYxCwFLVTVXRB4FHhURwZyhpmMaU0NVJ5TXvwz9bWqCmnbC2Ef9PqjY+XoY5jJNsxjXrsKMCgpFf15e4roC/wa+wFzSWQNcFKPN0CKvW2JGNm3GvOmvxJzBNItevwdYUELG08D3Jc5NAt4t8noY5syhEDNa6Njo2IeV897TgFcxb5CbgZGYS0MvF2njjuq0AghiPoV/BPQrR66WcdxTwWd1MObN3I/pzP8aaB3rc8ScrSyIvt8F0febB1wcvX4q5kxjB+by5AzgxCL9BdMPtij6OWwBpgJHx9PfPmrHIdEPy8bGxsbGplLYUVg2NjY2NlXCNiA2NjY2NlXCNiA2NjY2NlXCNiA2NjY2NlWiQYXxNmvWTDt16lTTatjY2NjUKWbNmrVVVZuXPN+gDEinTp2YObNkxgwbGxsbm/IQkdWxzttLWDY2NjY2VcI2IDY2NjY2VcI2IDY2NjY2VcI2IDY2NjY2VcI2IDY2NjY2VcI2IDY2NjY2VaJBhfHa2CSbpbNX8O7jH7NhxWb6HLEvp153PI1bZNW0WjY2ScE2IDY2FvHTlN948IL/EiwMoYaybM4KPvvf1zw7+2GatW1a0+rZ2FiOvYRlY2MBkUiEcVdNIFAQRA2zREIoECY3O5/X73uvhrWzsUkO9gzEplaQvWkHM7+cy8oFa1gyczn+XD+HDh3IKf86lpT0lN3tNq/dyqQHpzD3+0W07NiMYbeeSq/B+9Sg5lG9Vm8lkB8odT4SjjDjizk1oJGNTfKxDYhNjfP+k5/w4u1vEg5FMCLG7vMr569h6qvTeGbGg/hSvWxctZmr+92CP6+QSCjCmj/XMe+HRYx47gqOOn9wXGP58/wsnb2SzKYZdNqnvWXvIS0rlUgkEvNaZtN0y8axsalN2AbEpkZZ9sdKJt7xFsHCUKlr4WCYjas28c3rP3DCFUfz2r3vUJDjL2ZkAgVBHrlkPL999gdDbzgBVSWreSatO7csJe+jZ79kws2v4nQ7iYQN2uzRkuFjziUlI4XuA7ri8Xni1jtnWy7Zm3fSuktLPF43mU0z6H3Efsz5Zj7h4O7S4PhSvZxxw0mV/KvY2NQNGlRJ2/79+6udTDF5LJm5nF8/nok31cthww6iVacWpdpEwhFWL1pHSrqP1l1aMn7ES3zw9Oe7/QaxGHhyf0Z/cCvndbqazWu2lquDN9WDGkrXPp25Z8otuyOg5v2wiDuOf4BAQbBYexHBl+4FhRsmXMXhZw8qV35hQYBHLn6a6R/PwuV2ggjDx5zDKf8aQm52Hnef8hBLZ63A5XERCoQ4/foTuHTMuYhIuXJtbGozIjJLVfuXPG/PQGwSRlV56t8v8NUr3xMsDOF0OXnt3re5/rkrOObCw3a3+/WTWTxyyTOEAiEiEYP2e7WhbddW5RoPgLSsNAAat2xUoQHZZSCWzFzOqFMf4r+/jAFgyn8/K2U8dunuzy0E4LHh4+nSswMde5S9tPXo8PH8+sksQoEQoYA5a3rh1jdo0aE5A0/qzxM/3Me6pRvYum4bXXp2JLNpRrn62tjUZewoLJuEmTdtEVNfnbY7AikcDBMsDDHuyufZuHozhmGwdsl67j/7cXK25eLPKyToD7Ji7ip+fP+3CuV/N+knfvtsNsNuPRVfmjcunSKhCCvmrubv5RsB00lfEcFAkBfueIP8nIKY13Oz8/jlgxmlltsKCwLcfepDDPGdw33DHictK5Xeh+9rGw+beo+9hGVTihlfzOHluyezYcUmOnRvy5DLjmT/43rTpFXjmO2fvHoCn06YSqyvksPpIC0rlS49OzD/x8XF/BeVwe11cdfkm1g5fzVvjZ1C0B+IOV6xPj43LTs2Y9v6bCIRg6C/9AwkFi63k6sev4hT/jUEgJ1bc1g+dzVGJMLIkx4kHIrtLAdwupw0b9+UlxY/idPl5M9f/2Lhz0to3KoRg04bQEqaL+73bGNTWyhrCcs2IDbFmPbOdB655OlSyz0ut5O+R/fkjjdHkJaZWuzaU/9+gY+f+6rCpahEcXlciEO46tEL+fmDGcz+el7SxnL73Dw89W6mfzyTKU9+hsfnJuAPFnOQl0VKho+bXriar175nnnTFhEKhvF43TjdTh799h726NUpaXrb2CQD24BgG5CKUFXO63Q1W9Zui3nd7XXT75he3PfhrcXOL5q+hFuOvo9AQel9EMlCnKBlTwQsoWOPdmxctTmm76RcBA48oS9zvl1Qqm+bPVry8l9P2U51mzpFWQbE9oHY7CZYGGTb39llXg8FQsz6ai47tuwsdn7vA/ek0z7tkq1eMZJtPABWL1pXeeMBoLBkxvKYfbdt2MH6pRss0M7Gpuap0SgsEZkInAhsVtV9Y1wX4EngeKAAuFhVZ0evXQTcFW16v6q+Uj1a1188Pg8p6T7yd8Z2IoO5lLVzay6rFq7l42e/ImdbLk1bN2b53Jglkxss2Zt2xjwvApEq+oFsbGobNR3G+zLwNPBqGdeHAN2ixwHAs8ABItIEGAX0BxSYJSIfqWrZj882FSIinHnzyUwaO4XCspajRBh95mOsWbSuepWrJ2Q0yaBD97aoKgt/XszmtdvYs/8etOvWuqZVs7GpNDVqQFT1BxHpVE6TU4BX1XTU/CoijUSkNXAYMFVVtwOIyFTgOOCt5Gpc/znn9tMIBUO8+9gnpXwanhQPqRk+23gkwEX3nkX2ph3cfMS9bF23DcTccd//2N7cPPEaMpvYob82dYfa7gNpC6wt8npd9FxZ50shIleIyEwRmblly5akKVpfcDgcXHzv2UzZ/hJjvriLrn0640vz0rxdU5q3b1quj8SmfBxOYeu67Yw590n+XrYBf14h/txCQoEw0z+ayVmtLmP0WY9RkOsvV04kEmHJzOUsm7MSw7CXw2xqjppewko6qjoBmABmFFYNq1Nn+Hv5Jh4Y9jjBwiChQJjCGJlmbSqJCA6nsPCXJUTCpW/8kbDBrx/P5P5hjzPmsztjipj7/ULuO+sxgoEwqJKWlco9U25hr/57JFt7G5tS1PYZyHqgaF6JdtFzZZ23sYDCggB3Hv8A+TsLCAUq3vdgEx9GxGD/4/rgcJQdwhsKhJn7/UI2ry2dsiV7807uOmksO7fm4s/1488rZOv67dx69Gj8+YXJVN3GJia13YB8BFwoJgcCO1V1A/AlcIyINBaRxsAx0XM2CWAYBgV5fm46fBSbVpefc8qmCig8cM4TpGamlNvMMJSNKzeXOv/tmz/G3MlvRAx+nvK7ZWra2MRLTYfxvoXpEG8mIuswI6vcAKr6HPAZZgjvMsww3kui17aLyH3AjKio0bsc6jaVxzAM3nzgPd557GP8eYU0pM2l1c26vzbg9rpwe12EgmEzhrAE4WCYSQ9PoeehPYqdz964o8y09zu35CRLZRubMqnpKKxzKriuwL/KuDYRmJgMvRoCfy/fyJo/19NurzZMfW0a7z1eOurKJjmEAmFS0n2EQ5EyjfWMz/7gzhPGcMdbZuqYv5dvpFHLRvjSvKX8UQ6Xk16H1XxVRpuGR713otsUJxgI8cDZTzDzyz9wedyEgiEiJSoB2iQff17FPotZX8/j7lMeIhKKsHTOSlxuJwF/EJfbuTuhoy/NywEn9KNrn87JVtnGphS2AWlgvHj76/z26SwiYSPmcohN7SESijD/xz8BUEMJ7orudUL77m1p1DyTIcOP5MjzD6k5JW0aNLYBaUCs/nMt74/7LO72ZiYZrTBtekPD6XLgdDmrxQDHynAcDkVo3CKLx76/N+nj29iUR22PwrKxiPycAv494PZK9TnqgkPxpMRXwCkR2nRtRaOWWQnJcLgcSDnhsZYhcMGoM7nysYtwupzJH68MsjfHzrVlY1Od2AakgfDN6z8SiLOgEpi1NwIFgaQ71kWEa8ZdwvNzHim72mA5dsHpdpLVPJOjzj8UVxw39JR0H8defDjN2zc1a5pXFoWTrj6Wk646htZ7tKx8f4sYMKRPjY1tY7MLewmrgbBi3uq4Cz6JQ+japzPt9mqTZK3Am+olUBCgSavGjP38Tsac+yTZm3cWL9xUhtpn33oKw8eev/v1khnLWb1wbezGUT7Y8QoOhwNVZcmMZXz87Fd89cr3cevbpHXj3fmqzrr5ZB6//Lm4+1qF0+Vg2C2nVPu4NjYlsWcgDYSufTrH/cTtdDno1rcz3fp1SbJWYEQi9D7czOS/78F788bqZ2nerkmF/cQhZG8uvvfhoa9GktEkvcw+Wc0zcTjMr7yI0H1ANw48sR/eFE9cunpTvVx83zDALL714h1vxNXPShwuB+fffSZzvl3A6LMeY9xVz/PXrOWl2q3+cx0PXvgUw/cZwb1DH2Xp7BXVrqtN/ceegdQjtv69nZlfzsXjc3PACX0JBUK8ft+7fP36DxTk+OOegYSDET557is+nfB10nR1OAS3183wseeR2fSfDLQighGpWE81lDnfzi92rmnrxryz6QXuGPIAs78ufs2b6uXs206NJSmupb3MZhlcdM9ZDLn0SAAWTf+L/B1l101JBr40L3sfuCezvpzL5D8+oDA/gMMhfP36D1zx8AWcfM1xACydvYIbB99NsDCEETFYu/hvZnwxh9Ef3kbfI/erVp1t6je2AaknvPv4x0y88y2cUWdyJGTgdDvxV5DZtSxUQZOwNyQ1M4UDTuhHWlYKQ4YfyZ79SicBHHTaAD4a/2WF9cebtm5c6pzT6eTBL0cy6aEPmDR2CuFwBKfTXPI5Y8SJpdrnVWAE3D431z1zGcdefHixMrSbVm3G4XZAqBpKIwJN2jShYGc+i6b/RTgYJhI2xzUMJVAQZPyIl+l7dC/adWvNcze+UmyzoarZ5ulrX2DioifJzc7jhdveYNrbvwBwyNADufyh8+1U8jaVxjYg9YDlc1fx8shJhAIhQkV93jWQX0+cgpYxg/D43Jx09TFcVsRvEYvz7jyDXz6YwY4tOynMDyAipXZs+1K9DLsl1ozCnMWcc9tpnHnTSezcmktm03TcHnfMtju35pary3XjL+e4iw8vdb5r3y5QjXsvd2zMxihnBhkJR7iy9808NX0Mi39fGrPN+r824C8IMOLgkfy9fONuA/31q9NY8OOf/G/+47jc9i3BJn5sH0g9YOpr0wgFasemwMFDB9KoRRaIWf7Wm+LB43PjS/OyZ/89uODuMyuUkdk0gwnzH+OKRy5k8JkDOePGE+lz5H64vW5SM1PwpXm5aPQwBp06oFw5LreLpq0bl2k8APY5aC8cztj/Bm26toxpPAA6dG/L/kP6VE/oMJRrPHYR9Ad5dPj4Mv1AnhQPc76ex5Z1W4vN7sKhCNs2ZPPrJ7Ms09emYWA/btQDAgXBuG4w1cGML/7gnU0v4Pa4iUQizPpqHhtXbmaP3p3oMXDPYstA5ZGS5uOkq47hpKuO2X1u+8Zstm/cQbs92+BLtWZ/yr4Hd6f7gG4smr6k2HlxCLe9dl25fe+adAMT73iTd5/4JG7/UrJZMXcVF40exhv3v18sBNub4uHEK49h1YK1BApK+3z8uYWsnLeGg087oDrVtanj2DOQOkbejnxmfjWXpbNXoKoEC4MMPKlf2XsoqhnDMFi/dCNg+iMGDOnDydccyz4H7RW38SiLJq0a07V3Z8uMB5jLXY9+N4qL7j2LrOaZ+NK87Hfo3ry4aBx7H7BnuX1dbhdXPHIhU7a/zLVPX8Y+g/ayTK+qEgkbNGndmOOHH4HHZ87Y3D43g886iOFjz6Vt11Z4U0tHnaWk+2jTtVUNaGxTl5GGlLq7f//+OnPmzJpWo8pMfuRDXh01GZfHRSQUATEzu4pARpMMCnIKajy/ldvr5o1V42ncslGN6lFTHJ9ybo0vJ3pTPHTr14VR793MlrXbaNGhGVnNMgEIBUNc2PVatm/I3p1A0+F00KhFFq8tfxqPL76QZpuGhYjMUtX+Jc/bM5A6wuyv5/Have8QLAxRkOMn4A+aS1cRg0jYYMeWnQQLQ+aafPUsy5fC7XHR+4h9G6zxADNQoKYJ+IMsm7OSGV/8Qbe+XXYbDzBnTf/95QH6HdUTh8uBw+WgzxH78t9fHrCNh02lsX0gdYQp//2s/LQi0YlkTa3Fiwj7De7BHW9cXyPj1xaOPO8QPn/xm+KlgM2clNVKYX6AV+6ezFtjp9C8XVNadmzGzx/8Ts62PNrt2Zqrn7iEez+8BaDcIAMbm/KwDUgdYefW2lFxLiXTR2FuoFhYrcfnZviY8zh9xAk1qFnt4NIx57Jo+l+sX7qBcDCMy+siq1lmzBK1yWbT6i0ArF28vtj5dX9tYPTQR3ngszvoNdguRGVTdWwDUkfof2xvFv++rMajfQL5QTKbphMKhDEMAyNicNjZgzj1uiE1qldtIS0zlWdmPMgf3y1k1fw1tO3Wir7H9OKktPN3b/6rDQT8QV4Z9TaP2ynhbRLANiC1HMMwmP/Dn3z31k81bjxMlCPOO4R+R/Uke9NO9hm0F+33alvTStUqHA4HfY/cb3fakHk/LMLhFCLlb6yvdkrOTGxsKkuNGhAROQ54EnACL6jqgyWuPwHs2smVCrRQ1UbRaxFgV8KjNap6cvVoXX1kb97JzYePYvPqrRRWU71yb4oHcQjhcIRwoPQdz4goOVtzOeCEftWiT31gyYzlZdY+jwen22lG3VlMx33aWS7TpmFRY1FYIuIEngGGAD2Ac0SkR9E2qnqDqvZW1d7AU8D7RS77d12rj8YD4PHLnmX90o3VZjxcXhdn/ecU3lr7PGM/vyvmvo1dNbht4qdlx2a4vaUd1SKCy+0kNSOFtKxUeh2+T6kIOodT8ESNupV4UzxcPPpsS2XaNDxqcgYyAFimqisARGQScAqwqIz25wCjqkm3GidYGGTml39U67q5y+Xk7NtPxeP10PuwfRh2yylMeeqz3TuXvaleOu/XgUPOsHcrV4aBJ/cnJc1HYX6g2DJkWlYqj3w7CjWUzvt1IDc7n3M7XFUszYgRUfw5VUuIWRKny0EkYtC8fTPadGnJl698R1azDHsJ0qbK1OQ+kLZA0eo/66LnSiEiHYHOwLdFTvtEZKaI/CoisbPqmX2viLabuWXLFiv0rhYi4Ui11yIPhyI8cPa43cstw8eex93v3MyBJ/Wj12H7cNVjF/Hod/faCfcqidvjZtxP97PX/l1xeVy4vS669OzI4z+MpmvvznTr2wWX28XCnxcndR/Jrq/TljVbmfv9Qr544Vsu3XsE466ekLQxbeo3deVOcDbwrqoWfRzvqKrrRaQL8K2IzFfVUpV1VHUCMAHMnejVo27ipKSn0L57G1YtKL/CnpWEg2Fmfz2fP75bQJ8jTAfwgCF97PKpFtC6S0uemj6GnG25RCIGjVuUrgGf0SQ9IV9JRRjh2OmDP50wlUNOP4B+R/dK2tg29ZOanIGsB9oXed0uei4WZwNvFT2hquujP1cA3wP17i7Xeb+O1T5mYX4hv306u9rHbShkNs2IaTwA9jtkb9KyUhPOGVZp1EyTY2NTWWrSgMwAuolIZxHxYBqJj0o2EpHuQGNgepFzjUXEG/29GTCIsn0ndZacbdW/edDldpLeKK3ax7Uxw38fnno3rTo3JyXdR2pmCt5UD70P3yfusrtVJXvTjqTKt6mf1NgSlqqGReTfwJeYYbwTVXWhiIwGZqrqLmNyNjBJi8/t9waeFxED0wg+qKr1xoD48/y8OeZ9/vh2YbWP7XQ5OeqCQ6t9XBuT9nu15ZWlT7NszkoKcvzsNaArvlQvK+at5unrXmTBj4uTssy1df12VsxbzZtj3mPF3NV07tmR8+48gy49q38WbFN3sLPx1jIKcv1c0/+W3SnRLaGMXExurwsRwelyIg7BCBvc/NK/GHzmQOvGtrGM7E07uGjPa/HnJqfUpNvnJhwMo4YiDsHr8zD2y7vYd1D3pIxnU3coKxtvXXGiNxg+ef4rNq2yNlrM5XbiEAfBImnGvalern/2cg4bdhDzf/iTUDBMz8E9SEnzWTq2jXU0btmIZ2c/wuX73piUlPGhIqUA1FAKCwI8c/1Enp35sOVj2dQP7HTutYyfp/xO2OJdx+FghFDon70FWc0zuWbcxRx9wWDcHjd9j+rJAcf3tY1HHaDtHq14ZsaDdNi7LeJMvrN9+R+rkj6GTd3FNiC1DKfbmRS5uzaweVI8HHRyf46/7KikjGOTfDrv24EXF47j1aVPc/7dQy2p/1JW5FdaVmriwm3qLbYBqWWkpid3FhD0B/nmjR/x51mzu9mm5mjVqQUX3TOMO94YgTfVW2VD4kvzcsq/ji1V6tbjc3P69XaKfpuysQ1ILcNbDctIDoeDnG15SR/Hpno4dOiB7H9sb9weN26vC6ercrPYg07dnyseu5C+R/Usdj4citR4eV6b2o1tQGoZvQ8rnVDPapweJ03bNE7uIDbVxgdPf86ML+cQCoQIBcJEwhEcTkfc36Nv3/iJc9tfxZxv5hc7b0QM3n/yM+b9YEbI52zPZeX81fjzkxMFZlP3sKOwahkDT9mf8SNeJhxKTvEIT4qH4WPOtfNZ1SM+fX7q7oSXuzAisdOWlMWOzbE3rQb9AT574Rs+n/gtP7z9Cy63i0g4wrBbT+X8kUOrf9e8Ta3CvovUEvJ35rP492WMOW9c0oxHRpN0bp54DQedvH9S5NvUDMEkLjOpwpxv5pO3I59gYYhgNNR38sMf0qJ9M4695PAKJNjUZ+wlrFrAGw+8x9AWw7nt2PvJ2Wqtb8LtdeHxuTnu0iOYtO5523jUQw4dOhC3N3nPgts3ZBP0F5/hBAoCTHpoStLGtKkb2DOQGmDN4vUsnb0CI2KQuz2PN8e8b/neD4DUjBRGvnMj+x68N75Ur+XybWoH595xOr98OIO/l22o1hIA2Zt3Vt9gNrUS24BUI7nZedx14lgW/76s0mvUVcEwDNt4NADSG6UxYe6jnNrk4j0TKOIAACAASURBVGK7yZPNPgP3qraxbGon9hJWNfLQhU9Vm/Hwpnq55IFzbOPRQPD4PPQYuGe1jOVwOvCl+7jswfOqZTyb2ottQKqJ3Ow8Zn89L6nGw+Vx0qZrK3oO7sHIt2/k9OvsTWANicsfPN/cUJhMBPYd1J3xMx6skXo1NiaqmtTiY/FiL2FVEwU5fhwO6+210+3E7XGR2SyDe977D936drF8jNqOGtvRgncgvBhc+yGpZyCO2EWb6jN77d+VcT/ex0sj32LJ78vI2Z5XrAa7JSj0HNzDrqNejWjoTwj8AJKCegZB/ngo/BKIoJ5DkKxRiLPtboNSnaHVZRoQEWkPPIJZp/xz4BFVDUWvfaCqZdYhtylN8/ZNSW+URqBENEsiHHhSPy6+72ycTicde7RrkDH5Gl6GbjsbNAgUAt+g+c9D0/cQV7uaVq/a6dqnMw98cgeqyhU9b2LN4vWWznp96T5admphmTybslFVNGc0+N8DQphlk0KYC0fRoJvgD+jW01BHW4j8CbjRlJORjNsRR3rSdSzvkXgiZqnYa4HWwDQRaRq9Zs9dK4nD4eCG/12Fx+e2RN7Zt53KvVNuYY+enei0T/sGaTwAdOfdoLmYxgPzp+5Ac+6pQa1qHhHhgU9vp92ebfClWbespYZBoCDAa6PfYfbX82rFMkq9JTgdCt/H/G5HgCBmYZ+iEZsG6A6ILDR/JwD+D9Hsy6pFxfIMSHNVfU5V/1DVa4HxwA8isgcxyxPZVMQBx/flqV/Hcvg5g2jRsRkOV9WWtJxuJ116dkrKklhdwjDCEJpJ6a+jQvAHjC0noKH5sbo2CFp0aM79n9xG5/06WCYzUBDkuRtf5tV732bU6Y9w02GjkrqRsSGjhR+CViXpaRDCf6Kh5Fc0Le8O5BaR3Zn9VPV14HrMErStk61YfaVLz47c8cYIzhhxQpWXFhwOYe8Du1msWd1CA9Nh6xHlN4osRbdfiEY2V49StQx/fiHXDbyTJb8vs1RuOBQBhcK8Qv6auZwPnvrcUvk2URJ6TBcIL7dKkzIpz4C8ABxQ9ISqfg2cCSywYnAROU5ElojIMhG5Lcb1i0Vki4j8ET0uK3LtIhFZGj0uskKf6kJVef2+96r8Bem0Xweat2taccN6iobXoNlXgRFH2V8Nof63k69ULWTa5F8ozC/EsNqRXoSAP8jH479MmvyGjKScDJJStc4aAVdXaxWKQZkGRFWfUNVpMc7PUdWjEx1YRJzAM8AQoAdwjoj0iNF0sqr2jh4vRPs2AUZhGrgBwCgRqRXpZZfMXM47j33M1FenlVlzY+X81eRur3rKkjWL1vHo8PFV7l/X0YI3gXjzhQUhvDKZ6tRa1ixeR2F+IOnjbFq9mSUzk/+02+DwHAS+UwEfpgPdg3nLjmPp2r0f4o51O7WWmlxEHwAsU9UVqhoEJgGnxNn3WGCqqm5X1WxgKnBckvSMi0gkwugzH+Omw0Yx8c43+O+/X+Cc9leV+sdau2Q91w68I6GxAgVBpr09na3rtyUkp84SWYUZjRIPKeDul0Rlai9denYiJckFysBMuPjqqMlJH6ehISI4su5Fmk5C0q+DlHMwjUhFS98O8PSvBg1r1oC0BdYWeb0ueq4kZ4jIPBF5NxpaXJm+1cY3r//IjC/mECgIEA5GKMwrJH9nAfec9jCG8c8HPnroYwT9iTsdPT43qxetS1hOXUKNfIzs6yDwfSV6uZCUeJ9L6heHDj2QjKbplS4wVRWWz12d9DEaKqohNLIVQnOAeGaUBuQ/h7F9OBpalFTdansYz8dAJ1XtiTnLeKWyAkTkChGZKSIzt2zZYrmCu/j8hW9iLhfk7yxgxTzzn2vT6i2sWri2VJuqEAqEaLNHK0tk1RV0xwgIfEPFT2BFcKQlTZ/ajsfn4elfxzL4rIGlytVaTbs97bgaq1ENYmw9E7afCf7XIDyPSjlOgz+h285BQ/OSpmNcO9FF5ARgH8zFOABUdXSCY68H2hd53S56bjeqWnSN5gXg4SJ9DyvR9/tYg6jqBGACQP/+/ZPmTYyUEVGlKF+98j3PXD+RVQusMR4en5veR+xL6y4tLZFXF9DIRgj+RPEY+DgwNqKb90e9g5GssYijUVL0q600btmI21+/HoCFvyxh9NBH8ecVYkQMyza1elM9XHjPWZbIsvkHzXkAwnMTkQD40dxHkCavWaVWMSqcgYjIc8AwzA2FghmFZcVGwhlANxHpLCIe4GzgoxJjF32sORn4M/r7l8AxItI46jw/Jnquxjj6wsEx8xAFCoJ8/OxXLPhxMXnZ+QmPIyIcdcGhjHz7poRl1SW08FsqbTx2E4bANHR79Wyuqq3sc9BevLXueR797h7SGqVaIjM1M4W7Jt1Iz0OT77Ctz6hG0OAMNPAzqn7U2AH+t6wRnsT9IPHMQA5S1Z4iMk9V7xWRxzBTmySEqoZF5N+YN34nMFFVF4rIaGCmqn4EXCciJ2OG3GwHLo723S4i92EaIYDRqro9UZ0SYcjwI/jx/V/589elFOYV4knxEA6GMSIG4aA1FQbdXhfnjxzKuXecYYm8uoKGlkLuAwlKCUP4LzS0qFqiU2orDocDf14h/rzE65r70rz8b95jtOjQ3ALNGi4aWoBmXw5aiPmMHgHPUdYN4GhinawSxGNAdsWiFohIG2AbFm0kVNXPgM9KnLu7yO+3A7eX0XciZrqVWoHL7eLBL+7ij28X8Mf3C2jULIvn//OqZfLdXheNWmRx8jU1GmxWI2jeo8QfdVUeQYishwZsQAB2bNqJkFjqm7SsVJ785QHbeCSIahDdfgloieJcgc9id6gKkb8xCt7HkXq6dTKjxONE/0REGmEmVpwNrAIsmlvVLxwOB32P6sml95/LnvvvYVmeIBEYcumRPDfnEdIbNUCncHCWRYIM1NnJIll1lx4H7UUowVlxYUGAWVMTWZ+3ASAwjdh7mhTrYpzCkHMPGrY2IwHEoaGq3qeqO1T1PUzfR3dVHWm5JvWMwvwALrc12fLdPg+d9m1PZpMMS+TVOaycgjua7DbsauSggWlo8I8GlRSwebumnHjl0TjdVQ/vjYQiPH/jKyz69S8LNWuAGLnEjqwyMPd8WEUI9b9voTyTeJzo/4rOQFDVAOAQkWss16Sesc9BexIJW+P7cDodpGRUMaVBfSB1uEWCBLYchG7qjrH5KHTzQeiOG9DsS9AtR6DhVRaNU/sZdmvi1RgMQ7l36KMWaNOA8R5gph0phQvTNVyZjaDlZfqOgJFTKdXiIZ450uWqumPXi+jO78st16SesWNzDumNrcnHr6oMPLl6dpbWSty9sKb2mf5zGGuAIGgeaD4Yf6PbL2kwM5HZU+fh8SZeWmD739lMfa1UxiObOBFnW0i9MEbOqzCQzz9lCuKhPD9hKuI9stL6VUQ8/5VOERGN/mdFc1gld1dSHaQg18/PH/xO7vY8Nq7azKfPTyVYmJjjNyXDh4hwz/v/IS3TmrDLuog40lGcxJ//qiooaDaE5oGnVxLHqR14Uz2W1ZCZeMdbHH3BYEtkNUQcmf9BvQPRgsnm98/YYP0g7u7gtf4ziseAfAFMFpHno6+vjJ6zibLwlyXcdux9hIJhIuGIJdVSmrZtwqUPnMNhZx2Ex9ew7bW42qGubhBeSHJL0ThArZ/m10YGHN/XstnW1vXbCIfClvn8GiLiPRjxHoyx6YCKG1eFjLsRsT7xSDwSbwW+A66OHt8At1iuSR0lEo5w5wljKMwPEAlZYzwAsjfu4K2xU3BbsMxQH5DG44HM5A6iIXD3Se4YtQRfqpf7ProNlyfxm35qZkq15NtqEGgyinNlIO69kyA3vigsQ1WfVdWh0eN51ZhenwbJoul/UZBTlaph5WNEDLat386fvy21XHZdRJytwHtYsqQDKZBxS7XUka4t9Bzcw5ISy32O3K/BllS2HO8hFgt0QJMXkvb5lPn4ISJvq+pZIjKfGM/V0QSHDZ6c7blJc7yKQ9i+ITspsusi4h2IBj6mUskUy8RjOi/Di8DRBEk9H/H0tUBu3aGwIGDJjvQ/p/+FqtpGxAIk42Y0MA0oSFCSF1y9TH/K9rMxHC0g/d9IylmWfk7lzV+vj/480bLR6iH7HZKcqSFAKBim+4DkVxWrM6ScADn3A1UvxoVzDyTjevAegYgHNbZD4BcwtqLqR6paAa4O4k3xkJLuS3gGvX3TDjas2NTgskMnA3G1h+afoVsvAK1q8lUvZNwOuQ+yO4rL2AQ5Y1ANImkXWKVuuRUJd4UCXKOqq4segL0PJEpmkwza7dXGcrm+NC8nXH4Uzdo23NK1JRHxQtr5kEgaDld3xHccIh6M/NfRzYPRnSPRnbea+0ICv1qmb23H4XBw8GkWOG0VRhwysswKnDaVQ4NzQROJxApA3qOUDgH2Q95TqFoxgzeJx4keq3ztEMs0qOOoKvk7E8+yWxS3z80Nz1/JNeMusVRuXUUDv2FsG4qxqRcUfkZCe0K8B5syQ4sh92HMAj355l4QzUd3XIUaiS4f1A1UlTnfzbdEVkFOAV+//qMlshoyqhqdOSQYsq5lzNI1D9S673eZBkREro76P7pHKwLuOlYCyatQUsfYsSUnofrmsQgHwhx4cv8Gvaas4VUYuU9gZP8Lzb7UjI9XP0TWYM5Aqvi3yX8BVUX9U4BY9TAEgg1jY5w/r5Dtf++ouGEcBAqC/GmnNYkL1SAa+RszsUdJAuZyU7KQdBDr9pSV9yj3Jmba9rHAbUXO59Z06vTaRFpmChbOCAHzKeSxS8c3uJofuzAKPoKcuzCfwmI9iQWBLExHYyXDHo0NEF4QfQqL9cGpaagaAE6XA3FY95CyflkSNsDVI1QVzf8f5D/LrpuGpp6HZNyEuT8bwAuSBpprvQKSYjrSLdwPUp4PZKeqrgLuAjZGfR+dgfN35caywazqJtZHYf322WyW/bHScrm1HTXyosajkPKn8QWQeh6VnoloAbrjTnD3i/0kpmHwHFw5mXWQwoIANxwy0tIIwkW//MXi3+2w87JQ/7uQ94y5XIrfPAreQPOe2d1GRCD1Usw8WBbiaAHptyOp1jnQIT4fyHtARES6YpaGbY85O7EBvn3zRxwO6zdRqaEs+HGx5XJrPcEZIHH4OMQDKlRp52ZkMeTcXsKIOAAfpI9AnC0qL7OO8f6Tn7L6z3Xm5lcLueHQu9m5LQlPz/WB/PH8U15pF34oeKm4Y1tSqXr1zVg4kCav4Ug72/Jl8XgMiKGqYeB04ClV/Q8WFZSqD8yaOo9QwPrdo063k8atGuBET+JJ25ICKeeCf1ICA0XAKEAaPQkpZ0DKuUjTN3GkN4yyt9+++RNBv/Xf23AwzMiTHiTgj7W+38CJbI19Xv3sWoo1Aj9B3liLB1YoTE72qXgMSEhEzgEuBD6JnrPzawDhUJg531gTxVISl9vFwJP6JUV2rcazP2V/LR2mEzDtUih4j9JPc5UkPA/xDsaRNRZH1t2Ie9/E5NUhysrE63AIbbu2Ii2BwmV//voXQ5sP5/0nP6m4cUPC3T32eUcrwINqGHZcm4SBJWmbneMxIJcAA4EHVHWliHQGXkuKNnWMFfNWW+qE3IXD6eCi0cMaZBJFEQ/S+PlotEgakAJ4IfVypMUspMVMcGQBOyuQFA/hpFRpqwuccMVReFO9xc6JQLu92vDSkv+y/3G9E5JfWBBg4p2TmP7xzITk1Cck41ZK1/fwQcad5tJSaH4cubB8VD4C0YWkHFPJPvERTy6sRap6naq+FX29UlUfsmJwETlORJaIyDIRuS3G9RtFZFE0fPgbEelY5FpERP6IHh9ZoU9lSUn3YUQsDsHCzIP1v1teZ+X81ZbLrguIpx/S/Cck834k8w6k2ec4Mv+DONLMCJLAd1izRqzo9gvNJ78GxnHDj+CAE/riTfWYO9IzUshqnsWo9/7DVy9/z68W3PgDBQEmP/SBBdrWD8TTH2nyGngGgaMZuPshjZ/HkbJrq51QvvNcQOJNKCqYt3cvpF+DuJKT0aLGcmFF64o8g7lRcR0wQ0Q+UtVFRZrNAfqraoGIXA08DAyLXvOramKPSQnSbs82tOrUgjWL16OGtVPEUCDEe098ws0T/2Wp3LqCOFLN1CWxcDSzbiAjD4LTk5DErnbjdDoZOflGVsxbzcJfltC0dWP2H9Ibh8PByJMfpDDfGh/GNjuXWzHE0wtp8lLsi+59QXzlhJEr6OaKB3F0QFLPBAzwHZ004wE1mwtrALBMVVcAiMgk4BRgtwFR1e+KtP8VOD9JulSaVQvX8sFTn5PRJJ3UzBTCwTCBglgb06qGETFYv2yjZfLqE5J6IVr4OdYUmAqD0XBvcl16dqRLT3NiP/W1aYwf8RJ52dZkVnC6HPQ5ouH4lRJBjZ3mzDp1KOS/jvndrmKQg7EG0i5B4gpISYwyDYiqbojOEl5W1cOTMHZboGi2sHVAeYl5hmNubNyFT0RmYv6lH1TVmHNlEbkCuAKgQ4cOCSm8i98+ncV9wx4nFAhjRAw8PrflZY7cXje9Drf/+WIhnl6oa38IT7dAmgGeBhisUIK53y/kyasnWPoQ5E31cv7IoZbJq68Y/qmw8ybMIBEFIuA+CEK/UDUjUn0ZLMoNuFfViIgYIpKlqlZ4LauEiJwP9AeK1mTsqKrrRaQL8K2IzFfV5SX7quoEzP0r9O/fP+H7vGEYPDr82WL/aImWri2J0+UgNcPHqf8+zlK59Qr92xo53iPNutQNnEkPTbHUeIgIV4+7hBYdmlsmsz6iRnbUeEQTH+66Q4V+xfSHVOHe4h5ULbMPiC8rXR4wX0SmYlZ5B0BVr0tw7PWYmxJ30S56rhgichRwJzBYiySPUdX10Z8rROR7oA9QyoBYzd/LN1GYn3gNhVikN07Dm+Jh/+N6c+E9w2jUPCsp49QLyoqprxQZkPWUBXLqPptWl/H33PUwW8lHL4fTwd4HdEtIpwZB4ddm+Fupv28QSKd0Rt2KaIQ0snofSdnEE8b7PjAS+AGYVeRIlBlANxHpLKa5PBsoFk0lIn2A54GTVf/xHolIYxHxRn9vBgyiiO8kmaRm+IiErY+8AsjLzkccQq/D9qF5OzuNe7k4rUihL4hR6pmlQdLz0B4xy9K6XC4cjsrnToqEI7w8chKGkZz/lfpDCGLu0VCgKjv6/RBZl6BO8RNPGO8rwNvAr6r6yq4j0YGju9v/DXwJ/Am8raoLRWS0iJwcbfYIphl+p0S47t7ATBGZi1mv/cES0VtJo0mrxux9QLek1YDeum474676H99N+jkp8usNGSNIfK03B912ullUqoFz7h2n4Uv34nD+c0vwpno5+qLBVQ5V/+n93/jkua+sUrFeos6uWJu2JIDmPWmhvPKRinYoishJwKOAR1U7i0hvYLSqnlxux1pI//79debMxOPbszft4LZj72fd0g0E/datGxel9R4teXXp00mRXR8wjBBkXwyhGYkLc3ZGmr6DOOKNsa+fbFi5idfufYc/vltAk1aNGXbrqaycv4rX7n23yjKzmmXy0NSRdOnZsUGXJyiJhteg2VdDZC2mAbHQj+pohqPFL9bJA0Rklqr2L3U+DgMyCzgC+F5V+0TPLVDVOhciZJUBATM1831nPc6P7yWngp3T7eSLQCK5nuonZkrsCZD/HGgQa/7xBNz9cDS1c4SW5MreN7NiXmIbWn1pXlp3acmYz++kWZsmFmlWd1GNoFsOB2MzsUsKJIizG47mn1oqsiwDElcurBgRWA16YVNVeXnUZH56/7ekjdG6S8ukya7LaMHLkDc+mhK7qPFIZElRIbQADa9ITLl6RiQcYdXCNQnLKcwPsHrROu457WELtKoHBH+L1vtI0m3UtWdy5MYgHgOyUETOBZwi0k1EngKsnR/VMSY/8iGTH/ogaQnKvCkeLht7XlJk13nyniN2EsUE15HFBRG7IFJRAv4ghkUZFoyIwaoFa9mwMonV9uoKxrbkyg9MQyvMqWUN8RiQa4F9MItHv4mZxe76cnvUY7ZtyGbi7W9aXkchNTMFp8tJ2z1bc/sb1zPo1AGWyq8PqBqgSdo1rnlo9pUYmwZhbDoQY/PhGHnjUU2Oj6sukJLuo0lr60oKOF1Oy3a512ncfeJImpgIYQj+nkT5/xCPATlBVe9U1f2jx11AnXOgW8Wjlz5j+czDm+rhoa9G8kVwEi8v/q9tPMpAxAFOa7IJxCYIugV0OxjrIe85NPuKJI5XuxERrnz4IuvkOYRO+7avuGE9R1ztIGUo8W3DK4+ylm0L0eDsBGXHRzwG5PY4zzUIZn9tXf0Pp9uJx+fmvDvPoPsAe9NVXKTHSontITklagohNAcNzUuC7LrB1NenJS5EzIeka5+5DLfHLiUEIJmjICPR26hBmaHshR8nKDs+ysvGOwQ4HmgrIv8tcikTa7LY1Rlys/P449sFeFO95cbEd9qvA6vmx+d0FIdw1WMXMejUAfamwUrgSDkadYxHc5+AyCpw7YGk34iGZkEy4t9VzToN7oSST9dJtm3IZuYXfyQkw+EQ+h7dk0vuP4c9++1hkWa1Fw3OQQteg8gW8B6OpJ6FONJLtRMRJO0CjNBSKKxqtGU5KyGRVahq0kOny5tD/Q3MxFyuKrrzPBe4IZlK1SY+ef4rnr3hZVye8qebDpeDpq0bx2VA3B4XY7+8i16D97FKzQaFeA9GvAcXO6fuPpD3DJY/24gTHFbseq97fP7iNwnL8KR4OOnqYxuE8TAKJkPOA5juYoXQXNT/FjSdEtOIAOaG2MLJVDpXTEVI42rZd1NeNt65wFwRmQLkq2oEdtfx8JbVrz6xYt5qnrvxFYKFoXITJooI5488g9adW7Hw58Xl1lJIyfDx2vKnyWpm57mylII3Sc7E2IF6BlRjftPaw7q/Ek9YGQ5F2PvA6gsrrSlU/ZA7huK5qwohshEteANJvzJmP4ezCYbvTCj8ADP/lRWkQNrlFskqn3h8IF9h1hXdRQrwdXLUqV188dK3hIKlb0purwu3143T7cSb6uHCe87k/LvO5LBhB1U4U/HnFvL2wzVSQLF+U/hFcuRqALKHJy1kuzaz76C98aYmltX1rFtOZtv67dx54liGtbmcEYfcxaypcy3SsBYRWkhsp3bATJhYBqpqFjMTCzdYpl2KpA23Tl45xGNAfKqat+tF9PfU5KlUeyjYWRDT5+Fyuxjx3BW8teY57v/kdlbMW825Ha7ixLTz4gpTfPeJTwiHrQ0DbvA4SjrWrSII4cVm1cIGxpHnH0JWs0yc7qpt0nS4HPQ8pAcjDhnJjM9ns33jDhb+vIRRpz3Md5PrWa43yQIt43/a0bjMbprzMOy8HtSi4nGOTjgyrq+2tDHxGJB8Eem764WI9CP2Tq56x0GnDsCXXvrGFA5F6H9sL94f9yl3HD+GH9/7ja3rt8edpdeIGAxrczm/fmJFUmMbAEk9h8R2o5eDFkKoHj41V0BKmo9nZjzI0RcMrrhxDHypPp676RUCBYFiCWcDBUGeu/Hl+jWrc3UFZztK31JTkLTYodBG6C/wv4ilO9JTqreAVzwGZARmNtwfReQnYDJmFt16z4En9mO/Q/bGl2a6fERk95JVQY6f95/8lFAVi0nlbM3l/rMf569ZSS9h0iAwb0bJuiH5wNkwU8s0ap7FgSf2IzUzpeLGJTAiETasjF3DO2dbHvk7CxJVr9YgIkiT/4GzE0gqSDrghfRrEe+g2J1y77dej9QzLJdZHhXuZFHVGSLSHdgremqJVtc++RrG4XBw30e38suHM5n29i+kpPsYMvwIegzci/efTDxZWdAf4u2HP+SuyTdaoG0DJ/9pKv8kF0+1JAFxg7fhVof0+NyVTunucDpo3r4ZRsRg/dLSKWJcLufuB7P6gjjbQLPPIbwIjB3g3g9xZKJaCIHvwcgBz0DEFd1MGbR4VuvogDird0tAvFsh9wJ6YO7g6isiqOqryVOr9uB0Ojnk9AM45PTi5dp9qd6E1xlVlXUx/rlsqkCV8lg5iR255WL3cpizA9JoHOJoEG6/mLTZo1W5kYVFcbgcuFxO2ndvy+gPb2XetEWMu2oCgYJ/+ntTvZx0zbG43InuxK59iAi4/wnP1+BcNPtSwAA1AANNPQ/JuJXEd6KXwL2ftfLioMIlLBEZBTwVPQ4HHqYBpzLZReNWjQhYUAtkv0P2tkAbG1zdq9CprJlHGDJHIc2n4mj+KeJu2FkCJj04Je62RsSg9xH7csaIE/ngqc8JBUIcfcGheFO9eFM9eFM8HH/ZkQwfc24SNa4dGIYfzR5uZt7VfEzXcQD8b0HwJ0g5Gev8dg5wd7VIVvzEYwKHAr2AOap6iYi0BF5Prlq1l4A/wA/v/MpT175gibzDzzm44kY2FSIZN6PbzyP+rLxSftu8xyHldAs0q/v88d3C+Bsr/P7ZHH7/Ys4/K4pizth9aT5Gvfcf9h1UFWNft9DCqbDjP0AMP4/60YK3Ie1q8L+HNRUJPUhK9fo/ID4D4ldVQ0TCIpIJbAYaZEa0Leu2ce2Bt5O3o6DYlLyqeFI8ZDYpY4eqTaUQT19UMiuRrbcCh7uRC8ZGcLZBNQLGVnBkIZKscOHaS9M2jdm4KrYzvEyKukzUrAlSmB9g5EkPMmn983hT6of/QwPfo3kvmN8P7yAk7UrQHHTHTRTfVFiCwAwIfGmNEpKFNH4GcbayRl4liCcKa6aINAL+h5nSZDbQ8ILigf9e8z+yN+20xHgAZDROp03X6v/Q6y1qZVRPCCQDo+B9dPNB6Jaj0E37Y+y8u8GleB9266kVbpCNl0gkwvSPrKkKWtMY+S+h2ddD6HeIrICCt9CtJ6N5E6m4UuZ2CzRwg3sA0uI3xFMzGbzLNSBieonHquoOVX0OOBq4SFUvsWJwETlORJaIyDIRuS3Gda+ITI5e/01EOhW5dnv0/BIROdYKfcpDVZnxxZxKR6PswuX5S7jupgAAIABJREFUJ+rE7XHhS/Ny22vX4nDEY8NtKkIj66k4lUll1psV3X4p5NwWndUEzMP/AZozusp61kUGntSfS8ecY8nmtEg4Qs62vIob1nLUKIDccRTfEhc2/R2BX7BmWaoCnJ2g8QtmmYMaotyR1Qyu/6zI61Wqaklu62hOrWeAIZgRXueISI8SzYYD2araFXgCeCjatwdwNmahq+OA8VF5SaUq/0DiELOfCplNM+h7VE+G3XYqLy4aR+/D61xZ+dpLZCul07yXpHIGhHCsMMtC8H+IGnX/JlgZzrzxZN5YPZ79h/TB4az6DUsQeh1eD5KIhpeaiTZLEQKtpsjKyDrEqmWwKhLPN2G2iOyfhLEHAMtUdYWaawKTgFNKtDkFeCX6+7vAkdFZ0SnAJFUNqOpKYFlUXtIQEQaesj9OV/EvjTjLNyriEFSVcCjM5jVbWfjzYnoe2oMW7ZslU92Gh6sr5T/1uSB1mEWDOcCwYgmibhEORlj0y5Iq7yD3pXk58vxD6Lh3O4s1qwGczcqpKlhdO+z9aMEb1TRWbOIxIAcA00VkuYjME5H5ImLFLKQtsLbI63XRczHbqGoYs5xu0zj7AiAiV4jITBGZuWXLloQUvvbpy2jZsTkpGT5cbicpGT7adWuDx1d2kRyjRHqTgD/IC7fV7IdeHxFHGqRfXU4LJwRnWDSYE2rAYVnTvH7/u/jzCtFK1kkXh7DPoL34z0v/YsRzsbPS1jXE2RbcvSldyKya8zYbNVsiOB7PWNL9C8lEVScAEwD69++f0KNB4xZZTPxzHL9/Poe1i9fTcZ/27DVgD5657iV+fG86kdA/xsLpdpZZN33t4vWJqGFTBpJ6IZr3JLF3pHvMpIgJ44C0qxFJLEttbUdV+X7yL7z9yAfs2JxD7yP2Zd60RVXyATqcDgadMoBDhw5MgqY1hzR+Ct0xAoKzQKK3UskCI/E0+MVwD4LQDEqne/dCyhBrx6ok8aQyWQ0gIi2oeJG5MqyneDhwu+i5WG3WiYgLyAK2xdk3KThdTgae1J8DT+zH/9s78zi5qjLvf3+3qrqq13S6O8QYQFZZxSCJsgWJMgqILAMEXhUBUWREHeOwjSgivqAsgjjMyLCIwIsCsgZlGSDgxogEZQkosomCIQkh6fTeVXWf949zO+mlqru6u5ZezvfzqU/fuvfcc566XXWfe855zu+54Vu3ct5RlxCvimNZl/0rWVtFT0cPYZj/hzZ7q1nlMHXaoaAWi+8CmRUMHEaIu6fF9K+L0EoIHddg8S0g+U+UYeqtItx0/u3ccuFdG1egL/vJb0Z8uI7Fg5yCotl0lkyeh6nJjIJG1PRjLLvKDWnGt4XuB7HWr1E8vVmhmkMhOAFb9yXcMG0aSELQBKnBo/7lpZCV6IdKehF4Ffgl8FfgviK0/QSwvaSt5R7njgUGJ8pYCvRJWR4FLIsm9pcCx0ZRWlsD2wO/L4JNBfPLWx/jtu/dQ293ms4NXYShEcQD172XsGzuzk6ypooTzju2nKZOK9R4Iaiejc86qnHDTTVHUDTpCFuHrT8DW/dZLJ+E9ySms62Lm79z5wD5kjAbgjFkDjCIiZqGahYctDuLzzg8b53ZcOpdpz4Um40SO7leaepgqPs8kIoEFccbIWWYGlHyg6jl51BzNKgBJ43SBm8dRNh2acWUjQv5RX0b2BN4yMx2l7QI+NR4GzazjKQvAg/gwmN+ZGbPSToPWG5mS4FrgRslvYQLnD42Ovc5SbcCz+NiN0+1Mv+Sb7v0niH6QNl0Nu+wFUDznJl87uLj2OfwysRsTwcU3w5mLcO67obMK6hqN/ejtgxGnOJlLeyG9B+h5yFITepR3iG89vzrLgfIoIfoMBvSPLeJ7o5u913PhOx9+AJOu/YLpGqS3HPl/+Qdun3r72vLZH1lkYTqvoDVfBoyL2OdP4PuW8dXafdSSC1C8S2x9AvReqfMpkn8zushsROkyj+cVYgDSZvZWkmBpMDMHpH0/WI0bmb30i9MONp3Tr/tbuDoPOeeD5xfDDvGQutbbaMqv+WOc7nmucvKluhlOqOgAdUeN2hnEptxPrSeTtHyL1gn1v0AmmIOpGVuE5kcmTgl2GH+tnzj1q+y6rU1NDTXUz9zk5LC1rtuQSKZGOJAUrVJtp23dcntnkgoqCOMvTNKVTtOupcBYNk3Ib2CIQ9B1oV1XIcq4EAK6V+tl1QH/Aq4SdLlQGWn/icACw6cN6Q7D+QcJ04k4yw8ak/vPCpMUP1xCIoZQhpEwwlTi1mbN/PeRbuSSA58vqyqrmLx6YcRT8SZu92cAc4DYJd9dmTLneYOOC+IBVTXVfPhTy4si+3lxrKrneJuuGHTPusmXH86rNmf4uQ578K673fyOvnm3ML1RWhn9BTiQA7DKYItAe4HXgY+XkqjJgOfOPtI6mfWkki6ML6+ZFNHLjmEZE3VxsVWyZoqmt/ZxJFLDqmkuZ4+lD/kevRUoZqcHWQALP0C1nUX1vvkpMu+9/Wbl/CBj+1BIhknWZNkRks9p//oVHbZe4e850jiogfP4SPHL6K6PkUimWCvQ+dzxe+/Q0396BNSTWTMugnXfRFb8yFs3YnY6n0IN1yMmWGtZ0P3fYwsZzKK9tquhPjWDA0bxu1LfrhobY0G5ftiS9oOmG1mvx20f19gpZlNulR68+fPt+XLi6fDs251K3f94F7+8PCzzN5qFkd/9ePssGA7Xl3xN+6+4j5WvfYWCw6cx4Gf+dCU+wFNVsL266D9O8WpTM1QfyZBzcDJY7NebN2p0Ps49MlMxDZHTTegoKk4bZeJ9vUdtK1rZ7MtW4jFpmbE2VgIW8+Crl/gJG76qIa6JdB+EcWba4vQTILZjxN23QetZ+J6NiEuGqsRtdxd0u+WpCfNbP6Q/cM4kJ8D/25mzw7a/x7gAjObdL2QYjsQz+QjDHtg9b64NanFoBrqvkxQd9KmNtp+AB3XMFCNNQ7J/QhmXlmkdj2VwqwHW7UHOYen9E6wIq8DAUh8gKD5Rtd++nms48eQ/YdTAK75BApmFL/NfuRzIMNNos8e7DwAzOzZ/qKGHs+kov1yiuc8ALqg4wqs9rhNiwu7bmWolHcGen6FWfe0lISfUlgHeeVKSuE8ABrO3ripxM6o8aLStDNKhpsDaRzmmB+P8UxOOq8rfp2Whuyb/d4PI/dvRR7a8JSVsOt/sLWLKeb8xvDEoO50gsTETMI1nANZLulzg3dK+iwuL4jHM6kIwx5KI7Pdi7Wes2miPLmInMq/8e1Q4BOITVas+2FoXQLZv5WtTTXfQVA35DY8YRhuCOsrwJ2SPskmhzEfqAKOKLVhHk+xkapKp5OafgzrugMSu0DivdDzS7Bu3Gq8JCiOZhRp8t5TEWzDtylfzyNqs+NmmHHuhF0CkNeBmNkqYO9o5Xlf4opfmNmysljm8RQZSVhib0g/luNowLgXGG74lnNQioFlIWiBxK6Q2BFVH41iXgNtUhOWKc9Hf7rvguQ8qJ6Yz+yFiCk+AjxSBls8npKjpiuxtf8HMs+zaSI0SXEWfEUT533VhqshqCOo+0IR6vZUHlG+XB99dGEdN6LJ6kA8nqmAZV6FnkdASTTzaqcnlH0NC9tgw9egJAv9eqHrdsKakwgS25agfk9ZiW0D2ZfK365N3OyXPiG3Z8oTtl2OvXUo1nYptuFCbM0iLL0CJRciet1wU8kwWHsklpl06249g9CM8yj/M3fVhBbr9A7EM6Wx9DPQcS1uxXAvbpipB1rPxMJWN0dRLHHFvHRhbUXRH/VUkrCV8mYcjENsM1R70shFK4R3IJ4pjXXdw0C5iT5i0POok3+v2q/UVkDaR75PVsy6CLuWYev/jdJHYQVAAmLbQf1ZqPkeFAy3JK+y+DkQz9TGhutdRMdmXAhrhqg0FBe1EHZcD+lnIP5uVHP0pNPFmo5Y9zJs/RJcz7XUE+gpaLwYJT+Eiir6WTp8D8QzpVH1xyCndEgWkh90ZYJ6iBVT5n0wMQjfgLbvQfc90H4ZtnqhE8bzTFgsu9rlPKeL0juPJNSdSpD66KRxHuAdiGeKo6r3QfViXIrbaHiAJDScu7EHIAnVf4PcUtnFIBvpJ/XpY4VAGlq/Qth5e4na9Iyb7p9T+vkxQDPQrEcJ6j5f+raKjB/C8kx5goazseojsO6HnZBh6iAUH9jjUGoRVvMp6PwxpXnazHUjMrf4sPpgJC8vN9GwsJ3Sz3kkoeY4FGsucTuloSI9EElNkh6U9GL0d2aOMvMk/a+k5yQ9I+mYfsd+LOlVSU9Fr3nl/QSeyYYSOxPUfwnVfW6I89hYpvowXE+lnMSg95kytzm9MTPMukdM8qXkQkrzfaiK6q2GqvehulNK0EZ5qNQQ1lnAw2a2PfBw9H4wncCnzWwX4EDg+5L6hyOcbmbzotdTpTfZM9VRYmdI7kNZnYgMvMBiWTAzwo7/h63eE1s1D1uzF2HnLflPSMyD1AE4pYKiWgL1p6HmGwmart+UBmASUqkhrMOA/aPt64FHgTP7FzCzv/Tb/oek1cAsoDLJfz3TAjX+AOu8CTpvgbAT7E1KOg6uaqzncbcyPjF/wormTQWs82Zovxisy+0I34YNFxCSQFW7Qfo5iL0Ti78Xum6HrrsoTeaKKhTfGiV2K0Hd5aVSDmS2mfUpk70JzB6usKT34/p9/Zfzni/pHKIejNlwSRg8nkKJuRtMuDJaoS7cz6R/Ho8kudeWjIao8x+2Q/slmKogvhM0XVdQwimzbpeDJNgMBTXjtGWa0PEfm5zHRrpgwzcw+v7XMdz/upQPDYCmxv+sZENYkh6StCLH67D+5cwNROYdjJQ0B7gRONFsY1D/vwM7AguAJgb1Xgadf7Kk5ZKWr1mzZrwfyzPFcU+pP+wXNZXFfT3j7hXMgcSexWgJd5PqATKuB5J+Fmu/avizzAjbr8BWfQBbezi2+gOEG87HSirHMvkxMwjfynM0jftfZNmUa7yEqBYSu5e2jTJRsh6ImR2Q75ikVZLmmNnKyEGszlOuAfgFcLaZ/a5f3X29lx5J1wGnDWPHVcBV4HKij/6TeKYVHf+Fi/vvTxaoh1mPoqAOW7NvERrK9VXshc6fQv2X85/V+RNov9rZ2FdF5y2YalH9V4pg19REEhbMdetxKkIMSEBQi2ZeizQ1VlBU6lMsBY6Pto8H7h5cQG5m6U7gBjO7bdCxOdFfAYcDK0pqrWf6EL6d50A7CqqjOYoSTnraWsINF+c/3vHfDHVw3dB5/YhRRdOe+tMpf5QdENsWNV6KZv4Qzfo1SuxQfhtKRKXmQL4L3CrpJOA1YDGApPnAKWb22WjffkCzpBOi806IIq5ukjQLN5r4FDB54+A8E4v4dpD509D9sS2Qop9LzdFumGvc8yB56LyG0DohqAfVoupDUGyuOxauy32OdQFpzNxiSD8ZPxCzLAoasNTHoPsOypfXI4XqTkGpg8rUXnnRdHpqmT9/vi1fvrzSZngmMNbzOLbuc2xaNQ6QQo2Xo9QiV8Z6sXX/Ar2/pfQrlWNAHGZcQFD9ccK1iyGdI2o92ByqFkD3L4AsVO2DGs7Nu+ZlOmGZv2JvH+fyaliGkjn+IcSg9mRU95VJ79AlPWlmQwTjpsZAnMdTJJT8AGq6Hqr2gqAZEnugmVdvdB7gcqur8dIyWZTFyc9/jTC7DuK7MlRSPAlSJL3R687p/Q229qhoNfX0xcywdae47JDWQdmch2bBrN8Q1C+Z9M5jOLyUicczCFXt7pzIMFj7DymLTtJG0vD2cZD9GwOGX1QHtV+BjksZKLsRgnVhXXej2k+W0c4JRvYVyP6DsqeitQ0o82eI7VPedsuMdyAez1jovr/MDYaQ/StDc7dH+3PK1ndB5s8lt2yiYZnXsA3/F3ofw/XWSq1nlYserP1ylPQOxOPxDEbFlrcohMHOA7d+JGwHBTkesqvd4sRphIVvY2uPBGuj7L2OwWReq2z7ZcDPgXg8Y6HmWHKHhIqy/6yUgthWDJSjD0DVqPrQ8tqCu4mH688gXDWPcNXuhK1nu/TBY6nLQizzdyxf9BnRPEf6z1jPb7CO68B6qLjzAIhvX2kLSo7vgXg8Y0A1x2G9T0LPrwC5HgAGlmbgkIko+c0s+xJquhHbcH40kZ6Fqn3RjHNRmYUazXqxtUdDdiUb5V+67sJ6/wgt9yDFCq4r7LwH2r4J1g0Iq9oTNV6Cgk3i3ZZdg739mWh4L0Z5kj8VQgrVL6m0ESXHOxCPZwxIcTTzCiz9FydBkl3ptJaG3LzGezOrZpPERh7SK7Cex6D3l6AEkHA52DMvQt/6kT5rsm9iXXdCuBZV7QPJ/UZ1Ux+Rnochu5aB2mFppy3W+2tI7o9lXsXaLoaex6OhtzTQDfHtUP3XUXJPwvZrof3CgXX3/gZ7+0Qs9VHIrkLJvbHWC8H+Xjz7x4QgtjPYOieXEt8e1Z+FqvaosF2lxzsQj2ccKPFuLFwFG86hNE++g1ed56IbWr80ZK+tOxlLHokavopis7CeX2HrvoiLHuvFOm+DxK7Q9KOCJcXN0m7eRQ05w1Mt/QIuE8PgA92QfgGLvzuao+jA9dj6lcn8BVt3IhbMgfD1XK1D5nlofxFIY123U741HfmIu6HCmZeh+FYVtqX8+DkQj2ccmBnW+jXGFOmTPLzo9gyh53Zszb6EHTdh67+KWyDZ15vphMyzWOedG4ub9WCZV7Bww4BqzNJOtHHVHtjqvbE1Cwm7hkaiKb41uVMDBxDfCmu/doQ5imwe59GfvmtdKudRqFSNoGoBalk6LZ0HeAfi8YyPcBWEY0lRI+hZWnRzcmPQ9q0cUua4fZ23E4YdhB3XYas/gK39Z2z13oTrz8DMORtrPccJPdKNG5JaDa2nuVwm/atL7EZuZ5rG4u92Q2sVCastENVB6pACC5uTmhk0TDid8ENYHs94UC1jW1CoMZ43HvLcuLNPweoc8uLd92PEsaABum/PcWIv1nYhSt6xcY96HsVI5G5r/RKIzYnWppT7sxeIpSM5mEIIIGgpqTkTHe9APJ5xoKAeSy6Enl8zuifrCXoDHUA3dN+Ji27Kw5CFisN8rsyfolXhcYYNCqgkSubuqeUkjmqOLak5Ex0/hOXxjBPNuAgSuwEpUD2QgNg25P95TabnthEiwMgMTGaVPID8n9vAWhmq5TWBsICCHwRSB6HEziU1Z6Izmb7JHs+EREEDav4plnnZPWHHd0KxFsKwDdYcDLaaTZPGcedkbAPu5jzZEdb2H1D7KRRrcWHEqo8mynNh5J78FsTnQeaPJbS1EPoc3AgRdapFqYPLYdCExvdAPJ4iofi2KLnQ3UiBIKhHs+6OJmWTQBUkPwLNd0+hsXODzquwtw4kTL+AvXUIWL7UsSPUk3m66NaNnlwRZIMJQA2QXFhyayY6vgfi8ZQQBU2o8XtD9ofNd8KahUyNXkjGaU+tP8Pl3BgzE2FeqBd3W8zkOR6HxG6o8XubEoxNY3wPxOOpAEGsBWZcgtPTKuJK8IphkM2RyXFSks95AMRQ3Remdehuf7wD8XgqRFD9MdRyB1R/Ev9TLDcjXe98x3uwDecV25hJi//WejwVRPHtUP2pjNwLSTKho5cmGzWnDD8PFducvNc7+zqWN0hgelERByKpSdKDkl6M/s7MUy4r6anotbTf/q0lPS7pJUm3qFAhH49nIqJGCBqHKRCHxv+EqoX4acsi0XkNNN8L1OQ4mIKaT0MwO/e5SlHYZPvUp1I9kLOAh81se+Dh6H0uusxsXvTqn9jgQuAyM9sOWAecVFpzPZ7SIQnqv0ZeDabGywhS+6HGi6Kbl2f8CGVfhhkX4+ah+hxzjVPTrTkGaj8Hqh50XgqqP4HkB28AZFZ+7XxJLwD7m9lKSXOAR81shxzl2s2sbtA+AWuAd5hZRtJewLlm9tGR2p0/f74tX768SJ/C4yku1vNbrO0yJ8OuOMTe7aQ1sn8GqqFqPiQPgq4fO1XaCRG1NImZ8X2C6oOxzF+xrp9Bdg1K7gepjyIlnFBm+2XQ8WNQDCwD1UeghnOmXQSWpCfNbP6Q/RVyIOvNrDHaFrCu7/2gchngKVxYxHfN7C5JLcDvot4HkrYA7jOzXfO0dTJwMsCWW265x2uvTf00k57Jj2VexdYe4aTTBxNsDk0/hfRT0HoGhUm+T0eSDKvYqxlos9+OKGVvYSdk34DYO1BQX1wTJwn5HEjJ3Kikh4B35Dh0dv83ZmaS8nmxd5nZG5K2AZZJeha3VLRgzOwq4CpwPZDRnOvxVArruCr/au7wdWhdghq+jpUh4eGkRYkoWVW+nloGeh6F1EeGryaogWDqp6cdCyVzIGZ2QL5jklZJmtNvCGt1njreiP6+IulRYHfgdqBRUtzMMsDmwBtF/wAeTyVJr2DYRYbp5Vh2A1NjDUkx6Fv8l3JZDtUEaPiFjRaOUYrf00elZoKWAsdH28cDdw8uIGmmpGS03QLsAzxvbsztEeCo4c73eCY18e0ZMWx3/cnjXPldLhK4iepSEIfUYqj7N2j5JWq6ETXdjGY9DOGaEc4NoWrPEtk1PaiUA/ku8E+SXgQOiN4jab6ka6IyOwHLJT2NcxjfNbPno2NnAl+V9BLQDFxbVus9nhKj2s/jxvCHo7scpoyTFDR8m8JsHUNorKpR9cEEdScRxOegqveixI4usi2+9bDnUXMMim85+jY9G6lIKIGZrQU+nGP/cuCz0fZjwHvynP8K8P5S2ujxVBIldoCma7F1XwZbO8qzg36Kv329mBjlzwQYQM2JUShsjBF1v1IHQ/d95JaPzzPZYz0Q3y5ndao/DVt3KgOdl1yYbt0SSH6okA/hGQYfzOzxTFBUtYBg9v9Cw0XkXvCWjwRquQ+13Isa/xO1PEDwjufQZo9D7b+WytwchNB9P7SeSUGikd0PQPNtwKC1F8mPOQXjIT2yFFR/HMU2y1mdkgvRzCsgvgNQBbEtoeESgpafo9SHXS/FMy6mVzCzxzMJCWoOJ+y5H3qWFXhCPQRNSC0Q33bjbgUzUf2phCSg4zLKogQcvjqawgSJHbHZT2G9f4BwJSQXIUXOs+VurO1C6H3c9bBqjkO1w68hVnI/t7bDUxK8A/F4JgOJBdDzGCPPJaSg/pxhV0oH9SdjyfdjHddB5iXIvoILdR1FPLCaozUqRVyDktzXVS2h5B5Y+Da24Wys+0FnW/KDqOFbKJZrdYCnEngH4vFMAlSzGOu8DsIMm+TGq51jUeAcQXwrVPcvqGrByPVVzUNVlwPRosWOqyD9vMuoaCMttUpAcj/o+Z8irkFJoBkXbHxnlsXWHgvZ19n4eXsexdaugFkPEQVoeiqMdyAezyRAQT0034W1XwE9D4Fq3BBOzSeQxrcWRPGt0YzvABB23Q+tXx6mdADxrSC1CHoeHFe7m0hAwzdR0LRpV8+vojDc/rk5si5xVfd9UH14kdr2jAfvQDyeSYJiLWjGucC5BZW38G0I2yC2ecFORsl9sbw5wQUNF6DqQ8B6MRsuqiugMK2uAIIGVP3xgbuzr+ReiW+dWOYlL2w/QfBRWB7PFMPC9YRvfwZbvR/21qHY6r0Jux4o7GTVQjAn97GqvQhq/hmpCgV10PANcisIBxA0MzRqKgbVi6Fqb7dNDBLzUdOtaLDqbXxbyDVMpRoU97IiEwXfA/F4phi27l8g/Qwb131YF7SejsXnokROzdGNSIKGb2Lr/5VNE/YBKIXqzxxQNqhZjCV2wzZcAOkncAsB5UQHZ14N4Sqs9euQ/Wtf7WA9aOYP6Xt2zTuXUbUQgs0g+3c2DWPFQA2QOnAUV8NTSnwPxOOZQljmVUg/x9BFg71Yx48KqkOpRajpeqj6IMTeBamDUfNtKLHT0LKJHQmab0Cz/4iarkXNt6CW+6MV3lWQXYkbDjMgA933Y+u/ipQcdiJciqHmm93iQpK4ifsDUPPP/AT6BML3QDyeqUR2daRCOzjcN4wimgpDVbujpqsLL68UDIr+so6rGCqn3gM9v8Gyq1AsT8a/vjqDmajxEuCSgu3wlBffA/F4phKJHcBySYFUQdVe5bUl+xo5J+OVgOyb5bXFUxK8A/F4phAKGqH2RAbKgcRBdaj2+HynlYbEHuQc5LA0xLcpry2ekuAdiMczxVDdEjTjfIjvAsFcqF6MWu4euM6iLHacHAkp9r/NVEPtidM2s99Uw8+BeDxTDElQfYhbr1FJO2Jzofl2rO1Sp18VzITaz6LqIytql6d4eAfi8XhKhuJboZk/qLQZnhLhh7A8Ho/HMya8A/F4PB7PmPAOxOPxeDxjwjsQj8fj8YwJ70A8Ho/HMya8A/F4PB7PmPAOxOPxeDxjQmZFy0k54ZG0BnitTM21AG+Vqa1i422vDJPZdpjc9nvbh+ddZjZr8M5p5UDKiaTlZja/0naMBW97ZZjMtsPktt/bPjb8EJbH4/F4xoR3IB6Px+MZE96BlI6rKm3AOPC2V4bJbDtMbvu97WPAz4F4PB6PZ0z4HojH4/F4xoR3IB6Px+MZE96BFAlJR0t6TlIoKW9InaQDJb0g6SVJZ5XTxnxIapL0oKQXo78z85TLSnoqei0tt52DbBn2OkpKSrolOv64pK3Kb2VuCrD9BElr+l3rz1bCzlxI+pGk1ZJW5DkuST+IPtszkt5XbhvzUYDt+0tq7Xfdzym3jbmQtIWkRyQ9H91j/jVHmcpcdzPzryK8gJ2AHYBHgfl5ysSAl4FtgCrgaWDnCWD7RcBZ0fZZwIV5yrVX2tZCryPwBeDKaPtY4JZK2z0K208Arqi0rXns3w94H7Aiz/GDgfsAAXsCj1fa5lHYvj/w80rbmcOuOcD7ou164C85vjMVue6+B1IkzOxPZvbCCMXeD7xkZq+YWS9wM3BY6a0bkcOA66Pt64HDK2hLIRRyHft/ptuAD0tSGW3Mx0T9DhSEmf0KeHuYIocBN5jjd0CjpDnlsW7cDlQtAAAF90lEQVR4CrB9QmJmK83sD9F2G/AnYO6gYhW57t6BlJe5wN/7vX+doV+ESjDbzFZG228Cs/OUS0laLul3kirpZAq5jhvLmFkGaAWay2Ld8BT6HTgyGoq4TdIW5TGtKEzU73ih7CXpaUn3Sdql0sYMJhqK3R14fNChilx3nxN9FEh6CHhHjkNnm9nd5bZnNAxne/83ZmaS8sV2v8vM3pC0DbBM0rNm9nKxbfVwD/BTM+uR9HlcT+pDFbZpOvAH3He8XdLBwF3A9hW2aSOS6oDbga+Y2YZK2wPegYwKMztgnFW8AfR/mtw82ldyhrNd0ipJc8xsZdTtXZ2njjeiv69IehT3JFQJB1LIdewr87qkODADWFse84ZlRNvNrL+d1+DmqCYLFfuOj5f+N2Uzu1fSf0lqMbOKiyxKSuCcx01mdkeOIhW57n4Iq7w8AWwvaWtJVbjJ3YpGM0UsBY6Pto8HhvSmJM2UlIy2W4B9gOfLZuFACrmO/T/TUcAyi2YbK8yItg8auz4UN+Y9WVgKfDqKCtoTaO03PDqhkfSOvnkySe/H3R8r/tAR2XQt8CczuzRPscpc90pHGEyVF3AEbtyxB1gFPBDtfydwb79yB+OiKF7GDX1NBNubgYeBF4GHgKZo/3zgmmh7b+BZXNTQs8BJFbZ5yHUEzgMOjbZTwM+Al4DfA9tU+jqPwvbvAM9F1/oRYMdK29zP9p8CK4F09H0/CTgFOCU6LuA/o8/2LHkiEieo7V/sd91/B+xdaZsju/YFDHgGeCp6HTwRrruXMvF4PB7PmPBDWB6Px+MZE96BeDwej2dMeAfi8Xg8njHhHYjH4/F4xoR3IB6Px+MZE96BeKYVUaz/zZJelvSkpHslvbuE7c2LVjWP9rx3SrptmOONkr5QaPlRtLufpD9Iykg6arz1eaY23oF4pg3Rgqw7gUfNbFsz2wP4d/Jrfw05X9JofzPzcDH7o7Ezbmb/MLPhbuCNOMVhAAooXyh/w6kB/6QIdXmmON6BeKYTi4C0mV3Zt8PMnjazX0uqk/Rw9PT9rKTDwInXyeXuuAFYAWwh6YeRqORzkr7VV5ekBZIei8T4fi9pBm6B4DFRfoljJNVGeSl+L+mP/do5QdJSScuAh6N2V0THdonKPxUJLG4PfBfYNtp38aDyMUmXSFoRlf9StP8cSU9E+6/qW3XdHzP7q5k9A4Ql+Q94phReC8szndgVeDLPsW7gCDPbEEm1/E6bkmZtDxxvTiYbSWeb2duSYrib/W7An4FbgGPM7AlJDUAncA5uVfAXo3MvwMmqfEZSI/D7SOgSXK6K3aK6t+pn2ynA5WZ2UyR/EsPlbdnVzOZF9fYvfzKwFTDPzDKSmqL9V5jZeVH5G4FDcMKNHs+Y8A7E43EIuEDSfrin77lsGtp6rc95RCyWdDLu9zMH2BknNbHSzJ6ATcJ8OR7yPwIcKum06H0K2DLaftDMcuWr+F/gbEmbA3eY2Ys56u3PAbhkWpnIlr46F0k6A6gBmnCyHd6BeMaMH8LyTCeeA/bIc+yTwCxgj+ipfhXu5g7Q0VdI0tbAacCHzWw34Bf9yhWCgCPNbF702tLM+sQSO3KdYGY/wYkqdgH3Shq1tLukFPBfwFFm9h7g6lHa7fEMwTsQz3RiGZCMeg8ASNpN0kKc3PtqM0tLWgS8K08dDbgbfauk2cBB0f4XgDmSFkT11svJyLfh0pD28QDwpX6qr7uPZLRc/pVXzOwHOKXk3XLU258Hgc9H7RMNYfU5i7fk8kr4CCvPuPEOxDNtMKccegRwQBTG+xxO+fZN4CZgvqRngU/j5jRy1fE08Mfo+E+A30b7e4FjgP+Q9DTuJp7Cqenu3DeJDnwbSADPRO1/uwDTFwMrJD2Fm8e5wVzOkN9GE+IXDyp/DS6a6pnIlk+Y2Xpcr2MFzok9kauhKBDgdeBo4L8jGz2enHg1Xo/H4/GMCd8D8Xg8Hs+Y8A7E4/F4PGPCOxCPx+PxjAnvQDwej8czJrwD8Xg8Hs+Y8A7E4/F4PGPCOxCPx+PxjIn/Dy7qhcpxQiUlAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from sklearn import cluster, datasets\n",
    "n_samples = 1500\n",
    "X, y = datasets.make_moons(n_samples=n_samples, noise=.05)\n",
    "print(X.shape)\n",
    "plt.title('Problema de 2 clases', fontsize=14)\n",
    "plt.xlabel('Caracteristica 1')\n",
    "plt.ylabel('Caracteristica 2')\n",
    "plt.scatter(X[:,0], X[:,1], c=y);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 15; Error: 0.48249993074701175;\n",
      "The goal of learning is reached\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEXCAYAAACzhgONAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nOydZ3gc1dWA3zNb1OUmyd24V2zcaaEFA6YTOoFUAoRAeiGEhJAOgRTyEUIIEEhCJxTTew3NNja4415lualYVlnt7vl+zMpeSburlXa26r7Ps89qZu7ce3Z3NGfuPU1UFYPBYDAYomGlWwCDwWAwZDZGURgMBoMhJkZRGAwGgyEmRlEYDAaDISZGURgMBoMhJkZRGAwGgyEmRlEYMh4RURE5N9E2mYSIPCMi96ZbjvaIyL0i8ky65TBkFkZRGBwhdIPR0KtFRNaJyC0iUpRu2XIVEblWROaLSJ2I7BSRp0Xk4HTLZcg9jKIwOMkrwEBgJPBT4BvALdEai4gnRXLlKscCtwNHAJ8F/MArItI3nUIZcg+jKAxO0qyq21V1s6o+ANwPnAUgIseGZhuniMiHIuIDTgodu0JE1oiIL/R+WYS+B4jIsyLSICIbReSSWIKIyGAReUhEqkOvZ0VkTNjxG0RkqYh8SUQ2iMg+EfmniHhF5BsisllEdovIH0XECjvvktBT/F4R2SEij4rI4E5kKQzNuOpFpEpEfhKhjVdEbhKRLaHPOF9ETorVr6qepKr/VNWlqroE+AJQDhzZiTyHichroc9cG/p7UJS2c0Xk7dB3uEdEXhSRCe3aXB/6TZpFZLuI/CvsmIjIj0RkrYg0isiS9r9drPMNmYFRFIZk0gi0nzXchD3bGA98ICKfA24D/gwcDNwK3C4ip7c77xfAPGAqcCfwLxGZGWlQESkEXgeagGOAw4FK7KftwrCmw4EzgdOAs4HzQmPMAk4EvgZ8E/hc2Dle4OfAIaHzyoAHO/kebgFOAM4BjgemAUe3a/PPkKyfD30P9wFPi8ghnfQdTgn2/3R1tAah/l4H1mArlMOAhwF3lFOKsH+b2dgzmNqQXN5Qf+cAP8CePY7B/k4+DDv/18ClwFXAROB3wN9F5NQ4zzdkAqpqXuaV8Au4F3gmbHs2sAt4OLR9LKDAOe3O+x9wT4S+3gnbVuAf7dq8AvynXZtzQ39/FVgNSNhxF7AbOD+0fQO2IusV1uYxYCfgDdv3BnBbjM89PjT2kCjHi4Fm4OJ2+2qAe0Pbo4AgMKzduU8Ct3fhN3gEWAS4YrS5H3gv3t8xwvEiIAB8JrT9PWAV4InSthE4qt3+PwPPdXa+eWXOy8woDE4yN7S80gS8B7yF/UQezoJ22xOwlUU472A/fYbzXoTt9m1amQGMAPaG5KnHfhLug31TbmWTqtaGbVcBn6qqr92+itYNEZkuIk+Flkr2hn2eYVFkGYU9C9kvv6rWA0vC2kwHBFjeKm9I5lPbyRsVEfkj8BlsRRyI0XQa8Fo8fYb6HSUiD4SWjuqwvw+LA5/3USAfWC8id4vIeSKSFzo2MXTshXaf68qwzxXrfEOGEG26aTB0h7eAy4EWYJuqtkRosy/OvhJJa2wBi4ELIxzbE/Z3e/k0yj4XgNgeXC9iz2a+AOzAXnp6G1sZJCKvYi95tR+/sbOTReRP2J/1OFVdl4AckXgG2AJcAWzFNpgvJ/R5VXWziIzDXlKbA/wB+LmIHMqBpe3TgU3t+m3p7HxVjfdaMSQZoygMTtKgqmu6eM4K7LXyu8P2fQb7ZhTOYcA97bZXROnzI+AiYJeq1nRRnliMx1YMP1HV9QAicnYn56zFvikeBqwLnVOEbYdYG2qzCHtGMUBVX++KQCJyK3ABtpJYGccpi7A9pOLpux/2Z/5Gq1wiMp129w1VbQKeBZ4VkRuB7di/6XvYy24HqWrUWUyM81+KR05D8jGKwpBubgYeFZGF2DeGucDF2MblcM4WkfnYNoNzsZ9AD43S5/3YBtKnROR67KfZodiG6ztUdXU3Zd2EfeO7WkT+ir1s9qtYJ6hqvYjcDdwkIjuBbcD1hGYpoTafisj9wL0i8n1sRdcX266zTlUfj9R3SIYvYHuWVYvIgNCh+tDyViRuBt4XkTuBv2Ib/I8CXlLV9k/91dh2pstEZDMwOHS+P0yGL2PfRz4A6rGVVguwWlX3isgtwC0iItgzzmJspRlU1TtjnR9FfkM6SLeRxLxy40XnRtBjsZdXyiIc+zq2F05L6P2ydscVuBp4AXspZhPwpQhtzg3b7o/tSbQD++a+HntGUhY6fgOwtF0ftwFvtNv3EPBY2PYF2DOBJmzvnJNCYx8b47MXAf/CvhHuAH6GvaRzb1gbT0imdYAP+6l6HjAjRr8a5XVDJ7/VZ7Bv2o3YRvVXgIGRfkfs2cfS0OddGvq89cCXQ8fPwp451GAvK84HTgs7X7DtVMtDv8NO4GXghHjON6/MeEnoxzIYDAaDISLG68lgMBgMMTGKwmAwGAwxMYrCYDAYDDExisJgMBgMMck599jS0jItLx+ebjEMBoMhq1i3buEuVS2PdCznFEV5+XBuvLF9lgiDwWAwxOL882VjtGNm6clgMBgMMTGKwmAwGAwxMYrCYDAYDDExisJgMBgMMUmrohCRe0LlJJdGOX5sqFTj4tDr+lTLaDAYDD2ddHs93YudiC1Wjdy3VfW01IhjMBgMhvakdUahqm/RtpCMwWAwGDKMbLBRHC4iH4vI8yIyKVIDEblcRBaIyIK6up2pls9gMBhymkxXFB9hV8c6BPg/7GLzHVDVO1V1pqrOLC2NGFhoMBgMhm6S0YpCVes0VKlLVZ8DPCJSlmaxDAaDoUeRbmN2TEKlHatUVUVkNrZi251msQwGgyHrOY9H426bVkUhIg9il8gsE5EtwM+xS0Kiqndg10a+UkT82GUbL1RTks9gMBhSSloVhape1Mnx27DdZw0Gg8GQJjLaRmEwGAyG9JPRNgqDwWAwOEdX7BLhmBmFwWAwGGJiZhQGg8GQw3R3FhGOmVEYDAaDISZGURgMBoMhJkZRGAwGgyEmRlEYDAaDISbGmG0wGAw5hBPG6/aYGYXBYDAYYmIUhcFgMBhiYhSFwWAwGGJibBQGg8GQ5STDLhGOmVEYDAaDISZGURgMBoMhJkZRGAwGgyEmRlEYDAaDISbGmG0wGAxZRrKN1+0xMwqDwWAwxMTMKAwGgyELSPUsIhwzozAYDAZDTMyMwmDoJi1NjSx/+WXWf/A+eUVFTJp7MsOmTU+3WAaD4xhFYTB0A39zM09e9xPqduwg4PMBULliBYecfgYzzjs/zdIZDM5iFIUh4wgGA2xbupSda9dSs20rGgwyfNZshs+ajeVyAVC7vZLlL73E3h07GHTwwYw79lg8+QUpk/HTt95k786d+5UE2Mpj8VNPMvGkkygo7ZUyWQy5SzrtEuEYRWHIKGq2beWZX/6Cxro6NBDYv3/te+8xcPwETrnup2xbtoyXbvk9Ab8fDQTY8vFiFj/5BCf/+Fr6DR8Rve+tW1n9ztsEfD6Gzz6U/mPHIiLdknPjwgX4m5s77Lfcbqo+/ZThM2d1q1+DIRMxisKQdOp2VLFn40ZKyivoN3x4h+MtTU0E/C3kFRXzwo030lBd3aGNBgJUrlzB2vfe5YP//LvNTdrv8+H3+fjvj6/BW1CIpyCfshEjmXHeeZSFFMfyl17kvX//i2AggAYCLHn+OVweDwW9ejFhzglMOeVULHfkfwd/czNLX3yBNe+8jdvjZcIJJ1DYuw8igqq2E1TJLynt/pdlMGQgRlEYkkYwEOCN229j/QcfYLndaDBI36HDOPnan5BXXExT/V7e/NvtbF60GICivn3ZV70nan8aCLDy1Vfw7dsXpYHia9iHr2Ef+/bsYesnH3Pqz66npKI/7/3rPgItLW368gcC7G1qYuGjj1C5fBkn//gnHT+D38+8G66nesuW/ctMuzdtpGzESMTlQv3+A41FyC8tpf/Ysd34tgyGzMUoCkOXaWlqYuOC+TTW1TFo4qSIswSAT555mg0ffkigpWX/TXrnurU8fu01TDzxJFa/9RY1W7cQDC0x7d25o9OxK5cvj/rk3wZV/D4fb/7tb5QO6E8wGIzaNODzsXnRIp751S856rLL6DVgYKgLZf2HH7ZREq3tq1at3C+Hy+tFRCjs25ejv3Y5LY2NeAsLO5fRYMgSpMPUOcsZNWqm3njjgnSLkbPsXLeWZ3/1SwKBgH3zVEUsi2EzZnDEl75CSXn5/rYPXHUl9bt2RezHcrn2K4hMwuXxMOe732fFq6+wedFHaAwF04rb62XiSSez8tWX7aWtYJCRhx/OUZddgdvrTYHUhlwhncZrOf/8hao6M9IxM6MwxI0Gg7x08+/xNTR02L9xwQKqVq7kgj//hbziYsCeeUQjE5UEQKClhRdvvsneiPMhKhgIsPS5Z9p8pnXvv48Glc9+81vJENNgSClGURjiZvfGjR2UxH5UaWluZsXrr1IxajQ716yhqF8ZzfX1qRXSCbo4y46k9AI+H+s/eB/fpZfiLSxySjJDDpIpLrCxSKuiEJF7gNOAHap6cITjAtwKnAI0AF9W1Y9SK6WhlWAgADHcSQM+H/MfeCCu5ZqegLhc1G7fzs6162isrWHg+AkMnDSp2y65BkO6SPeM4l7gNuBfUY6fDIwJvQ4F/hZ6N6SBshEjOjUkGyVxgGAgwDO/uAFVxd/cjLhc5BcXM+uCCxl91NHGfmHIGtKaFFBV3wKi+0PCmcC/1OZ9oLeIDEyNdIb2WC4Xc779XVzmBhcfqrQ0Ne2P+dBAgMbaWt65526euPYaWpoaO5yy6aOFPPydb3Pnhefz78svY+kLz3eM1TAYUkymZ48dDGwO294S2tcGEblcRBaIyIK6up0pE64nMnjyZD5z6WXpFiMriGawD/r91FZVsfT559vs3/LJJ7z8pz9SW7kNVGmsreHDB+7n43lPpUJcQwo5j0f3v7KBTFcUcaGqd6rqTFWdWVpa3vkJhi7TVFfHgkce5rEf/YA377g93eJkBzFmAsGWFpY892ybfQsefqhNvAbYUeGLnng8Y73EDD2DdNsoOmMrMDRse0honyHJBIMBtixezO6Nm8gvLWHBww/ha2hoE91sSIymujqe/fUvmfOd77Fz3Tp2b9oYsV3Q76d5X71JNGhIG5muKOYBV4vIQ9hG7FpVrUyzTDlP8759PPWz69i7YweB1hQVZp08KWxdupT/XHkFAh1mE61Ybjd5RcWpFcxgCCPd7rEPAscCZSKyBfg54AFQ1TuA57BdY9dgu8d+JT2S9hxUlad+9lNqtpqJW0pQjaogANx5eUz73Nn706sbDOkgrYpCVS/q5LgCV6VIHAPw1t/voGbrlnSLYQAKevdm+tnnMPHEk9ItiiFBssVoHY1MX3oypJDG2lo+ffONdIthAMSyuOSOO01wniEjMIrCANhLTluWLCETLBElFRXsq64m2BXDeesNtTNbSrzt2jHqyCOpXL48Yq2MZJBXXMyTP/0Jgw+ezORTTqWgl23IrquqYuFjj1C5fDmFffoy9ayzTJGkDCXbZxHhGEXRA9m9cSMLHnmInWvXUlJeTsWYsSx/6cWM8Giy3G4mnHAiQ6ZM4aWbfx81+2w43uJiTvrBj/AWFfHEtdcQDK8REd63y8XZN96Et7CI5S+/xOInn4hPJpeLGeeezyPf/26XPksiNO3dS1NdHbvXr2f5yy9x3i1/JOBv4fEf/4iWpiY0GKR+1y5e+8utzLro80w++RT2bN7M+g8/QEQYeehh9B7cIeTIYOgWRlH0MHZtWM+863+2P1q4obqaqk8/TbNUIUTIKypiwmePJ6+4mJN/ch2PX/OjThWYy+Vi4IQJAJz1698y7+c/61Cm1HK7GTp1Gn2HHQTA1LM+F5eicHk8HP7FL+Fr2IdlWTHrWjhKaMYTDATw7dvHI9/7DkOmTNmvJFrxNzcz/6EHad67l4+fnmcrSREWPfE4M8+/gENOPyM18hpympwIuDPEz4cPPBCx1nM6EZeLvOISxh51NGffeNP+NOV9Bg+hdMCAmOv0YlkMnnLI/u2yESP4yn3/ZvycOVguF+68fFweDwMnTOS4q67e3655797YMlkWk046mfP/dCsTTzyJgtJeaV2Wa2lsZMP8BZFzaQWDfDzvKQI+HxoMoqFaIQsefoi6qqrUC2vIOcyMIsfxNTRQu72S4rIy8otL2LFmdbpFaoPb6+W062+gYsyYiMfnXnMtz/76lzTW1BAMBtu4klpuN578fGZdcGGbc0SEoy+7gsMuvoTqLVso7NO3TUElsMuuujyeqLOVudf8mKFTp+3fLqmooGLUKKpWreruR00YDUaOzg5EWWpTVTbM/5App52eTLEMIXLJJtEeoyhyFFXlwwfuZ+nzzyEul529VCRt2V0tt5sB4yeQV1RE9dYtNNbW0u+gg5h90eepGB1ZSQCUlJdzwZ//wo7Vq2msrcVbWMCqN96gtrKSQZMmcvDJp1DYu0/Ec72FRfQfOy6qPLMuvIj3/90xcfH4449voyRaOekHP+LpX95A9ebNHY6lC5fHQ2n/AdRWbusw4xERxDKLBobEMYoiR1n+0osse/EF+4k59NScziykLo+HyaecwkEzIlZajImI0H/s2P3bgyZ1KF3SLaacdjoFpb14//5/01RXh7ewkGmfO4fJp54asX1+aSnn3fJHtn7yCUtffIGNCxekLWJdQgF45aNGUdinDzXbIgdIjphtsvIbEscoihzlk6efzihbhOVyMeSQQzpvmGLGHH00Y44+ukvnDJ4yhcFTpvDOPXez/KUX06IsWmeG21eubLNfXC6s0CziiC9/heKyspTLZsg9jKLIURpqa9Itwn7yS0o45Sc/xeX2pFsURzl47lw+ff01/DFScCSNaMpJlelnn8OYY46luF+/1MpkyFmMosgxdqxZzTt33xUzf1CqsDweBowbzynXXYdl5V6uot6DBjPnez/gzb/9db/bajAQSGuVP5fHQ0GfPkZJpIhcNmCHYxRFDlGzbRvP/PIXaV9yKuzbF29+PhNPmsvEE0/MSSXRyrBp07j4jr9TV7kdV56XB6/6RlrlERHcntyauRnSj1EUOcTHT89LSXS15XZHjX4u7d+fC/9yW9JlyCQsy0XvwYNRVdz5+fibmtImS0tTEwsfe4x+w0fQZ8iQtMlhyC2M71wOsWfjhpQsexw0cxaf+dpl0M710uX1MvviLyR9/ExFRJh04okdaoqLZeEtLKS4vJz+4yK76zpJbeU2Hvvh99m7Y0fSxzL0DMyMIofoNXAQO9euTfo4mxYu4KhLv0ZxWTnzH3yA2u2VlA4YwOwLL2LY9BlJHz+TmXXBRTTW1rL23Xf3B/SNO/Y4jvjqV7EsF0uff45d69YlfeanwSDv3XcvJ/7wR0kdpyfSU+wS4RhFkSP4fT4a62pTMpbldlO/ZzfDpk1j2LSOgWk9Gcvt5thvXM2hl3yBvTt2UDpgAPnFJfuPD581mw/u/09KZNmy5BNWvv4aVStXUjpwIOOOPY7C3r1TMrYhtzCKIgdoqKnmvz/6EY0pcokNBgKU9h+QkrESJdbT36Ocl7RxC0p7RaxxXVxWxhFf+Srv/vMeO+W5KgG/H8vlimr36S7+5mbeufsugi0tuDweFj/5BKdd/3PKR45ydJxcpyfOINoTVVGIyFDgZmAw8Dxws6q2hI49qapnpUZEQ2f87557UqYk3Hl5HDz3ZLwFBSkZLxeZcPwchk2bxob581FVDpoxk7qq7bx159/Z63ASv9aaHoGWFgItLbzx19s47w9/cnQMQ+4Ta0ZxD/Bf4H3gUuBNETldVXcDB6VCOEN8bFywIGl9W243vQcPoW77dgpKSznkjDOYcMKJSRsvUbry9BfeNpmzi0gU9e3HpJPm7t8uKS/n3N/fzKPf/15cNTi6S+327TTW1u4vhGQwxEMsRVGuqneE/v6miFwCvCUiZ0BGFEIzhAhGySqaKJ78fM789W/pO3RoUvo3tMWTX8CFt/4fL//pD2xcuDApqUFUFcttVpwNXSPWFeMRkXxVbQJQ1f+IyHbgRaAoJdIZ4kJIjuZWMEoixVhuNyf98BoaaqrZ9NEi3v7H3x11ebZcLlxuN76Gfezbs4fisnI8+fmO9W/ITWIpiruAQ4E3W3eo6isich7w+2QLZogfb1ERzfX1zvebJTcQJ4yN6VyGikRh7z6MO+44Fjz6CA17djva97O//TW71q7FcrnRYIDJp53OzPMviFkgqqdhDNhtiaooVDWixUtVFwEnJE0iQ1wEgwFWvvoqHz3+36QoCcvtZmLYGroh9TTV1dFcV+donwGfj6pQxtnWWI4lzz5DYe/ebWwmBkM4ZrEyi9i1bh0bFsynpamJHWvXsGPVqqTVmBhx6GFMPdM4tqUTT34+mgJzoL+5mY/nPWUUhSEqRlFkAX6fjxdu+h2Vy5YltfiQ5XbTa8BAjr3qKuNrnwG48/IYedjhrH33f0lPzdLk8MzFkFsYRZEFfPT4Y1SuWJFUJVE2ahQn/eCHFPXNjvTUyVxDbt93Om0WR112ObvWr6dm65akjlM2cmRS+890ErmeFAWrBYJuJEfT58WlKETkVGASsN+6qaq/TJZQhrasev11NJAcF1hEcHu9HPW1y7JGSURCSzahQ96Cgl1QdxCy5RikqW+6xUoYT34+J/3oRzzyve8m7Rpw5+Vx2Be+lJS+cxHttQ4d+AG4mqChHPqthLxaUBdaOQu2HYGUbIGWIqgdnhPKo1NFISJ3AIXAcdieUOcCHyZZLkMYjieQE8GyLAp696H/mDFMP/e8jHeDjfXEp32Xo+MfsZ/qBCjYjZYvgcVXIo3lqRMySfQaMJAZ557LR48/vj/S2jFEOPIrl9JveM+Loe3OLCI49HUY+uaBa63PGvvdPgqD34XB76IqoC5bWSz5KtKU3SVp45lRHKGqU0TkE1X9hYj8ATulhyFFDJ8xk0/ffivhACxxuSjq05chhxzC9LPPyYl6ykECMOZxcIXdQK0gSDM6/f/Q9XORbYcjZLfr5/Szz2XYtBmsfvstlr30onMKQ5V37/0n7/3rXj77rW8zbNp0Z/rNQdRTD8PeACssJ1f7y6p1WxTwg1WLTvo3LPxOVl+D8SiKxtB7g4gMAnYDA5MnkqE9sy++mC2ffEJDTXW3+ygZMIDzbr4FtzfPQcmSS2dPfFqwAybfA57GjgcFkAAMfwl1NSKbj3dEjnTaK8pGjKBsxAgGTpjAS7fc7Fi/LU329/fyH//ABX/+iymjGo1eGyDoaqsoOkOwl6UKd0BD/2RJlnTiWTx7RkR6YycI/AjYADzoxOAiMldEVonIGhH5cYTjXxaRnSKyOPT6mhPjZirBQIBNiz5i5WuvUrN16/79hb37cMGtt3LIGWd2u+8hUw7JKiXRGUoQPfhe8O7t+FQXjqsFhr6DirOZWdPJhgXz7cyzDhPw+Xj/P/92vN+cwd/dAFQBV3rLEydKpzMKVf1V6M//isgzQL6qJlz4QERcwF+xg/e2APNFZJ6qLm/X9GFVvTrR8TKd2u2VPH3Dz2lpakKDQVSVEbMP5birrkYsC3+zj2Uvvdjt/j9943UOvegivIW5kX1F+6wEb31sJXGgtd22OTdqMexcszYpeaAA1r33Hk1fvZT8kpLOG/c0akZA0APaHOd1F0ICaP3ALF54imNGISJXhWYUqGozYImIExXkZwNrVHWdqvqAh4DuPzJnOS/dcjMNNTW0NDbib24m4POxYf6HrHrzDQCe+91vEqrFbFkWmz76yCFpk8d5PLr/FY1g//kw4UF7aSkuFHy5oSAB+gwZnJQZBYDldrFxYfKyEaebeK6vaAguWHGhvfykxJ9gTQIw889o4fYuj5kpxLP0dJmq7i92oKrVwGUOjD0Y2By2vSW0rz3niMgnIvJYqEZGB0TkchFZICIL6up2OiBaaqmrqqKuqqrDU6K/uZkPH7ifB791NbvXr09oDAWCgeTX00426q2B0U+BKxjfU50C4kfHPIW6ItgyspCpZ52N2+NJSt8igubAdZIMgpYPDr4XrEDIBhbniQLk1aBT7s7aJdB4jNkuERENRXuFloy8nZzjFE8DD6pqs4hcAdwHfLZ9I1W9E7gTYNSomVmXAj3Q4kOsyDq7qa7OkahZDQRyomypjnk85FESJ63/0OWL0KJKWHR1Qt4nmRCMVzZiBCf+8Br+d/dd1G6vdLZzVYbNMJ5PAFqwCx38DhRWwd4h9rvl79qyUyutzhV9VsOeCU6LmnTiURQvAA+LyN9D21eE9iXKViB8hjAktG8/oSJJrdxFjmat7T1oMJ78/ISWlmJhud0c+ZVLyS8tTUr/qUL7Loc+a7v3j2ph/6P32gC1IxyWLPUMmTKFC279Cy2NjSx76QUWPfEELY2JzZgsl4vDv/glCnv3cUjK7EVLNqKT/wniB0uhZKt9o09kxc/VjPZaj+SoorgGWzlcGdp+GfumnSjzgTEiMgJbQVwIfD68gYgMVNXWR6YzgBUOjJtxiGVx/De/zQu/v5FgIOBo7WR3fj7n3HgTvQYOcqxPJ+nKWrEe9Gpi/6iiaMF2JAcURSueggKmnvk5GqprWPr8c93vyLI46ze/o2xE7nw30P3UHDrmqXaxOYHEi74IMPADdOcUpH5Igp2llni8noLA30Ivx1BVv4hcjV0IyQXco6rLROSXwAJVnQd8K1RRzw/sAb7spAyZxKCDD+b8P/6ZVa+/xqaPF7Nz9eqE+3Tl5XHqdT/LWCXRZfIdqAs+5G2CJVttt9riSmjuhWw+Ftk9KfG+00jF6NG48vIINHfTDTMYJOB3OOo7S1GrxY57aI8T/gOWHx32GrL8iw50ljqiKgoReURVzxeRJUTQpao6JdHBVfU54Ll2+64P+/ta4NpEx8kWisvKmHHe+eSVlCSsKFweD8d+/Ur6jx3rkHTO0e0EbA1lUJpgcryCWshfZP8tgHcfOu5RdP1erMrDutxdpgTjjTj0MN65+y4SyQb17C9/wZf/eV/Wl0pNOGFk0LI9m1wJzOyVyIpFiKyEMpxYV8S3Q++npUKQnozf52PJM/uDzFwAACAASURBVE+z/JWX8TU2OmarGHzwZEf6yRh2ToKSLd1/spN27624WuwI7u2zEHUlIGD6cHk8lFRUsHvDhm734ff5WPn6a0w84UTnBMtCBBe6YypULO6+soh2jSpQn30z/FgV7lptA99Q1WvCj4nITdi2C0OCqCrzfv4zdq1b51if7rw8pp51VsYYr51KCS61o9HgK+BKQhZVCdqpFrI442xJeWKKAmD122/1eEUBIOtORb17offa7ns6Ret783HOdZYi4omjiFT29GSnBempbFu6xFElgQgzz7uA6Wef61yfGYLsG2Tny0mGA7Sonekzi5ly+hlYrsSWjUzdbBsJerGWfxEWX9l543hRoHoU7BvgXJ8pIpaN4krgG8AoEfkk7FAJ8L9kC9ZTWPvee852qEpT/V5n++wGySosJMu+jB76W2c7DbhhxyFIILFcWOm2VwwYN46KMaPZHqqJ3R08BYUOSpQ6kna9NfRHA15w+xLrqPXhptcmdNpfYcmliL8gYflSRawZxQPA6cBToffW1wxVvSQFsvUI3HnOJuqz3G68hdn5zx4X/jyoG5LYrELbvawAuJtQyX6vn9IBiSV23rzoI3Zv3OiQNNmPILDm9O5fb63XWGvgp8v2qNLRTzomYyqIqihUtVZVNwA/Bbar6kZgBHBJa+4nQ+I4vR4slsXoI490tM90oxJAi7cQHPI6ethvoLiq+50F3LB7gp3crfWfVxT6rkJHPeuUyGmj96DEDaVv3uGoJ3zWIzVjQbtZpS5Sqg8rAP1WoAn5qKWWeBY0/wvMFJHR2GkynsKebZySTMF6CnVVVXaCN4eygR5z5VUUl2V/VbdWgv2WwdjH7ajY1qpirURzQYyGAoE8O37C1W724PJDxSJ07elZ6/kEODIb2LVuLf7mZsdnu5mM5u9CR7wAvdaDvwC2HolUHmqXMe29FoJusBJcfgpHgvYrS661eNRkUFX9wNnA/6nqDzGFixzDt2+fY0pCLCt5tbXTgBbshHGPgrvJvrFHqibWla9OAM8+8EbJki9BWxllKbWVlWz48ANH+lryfPbPruJFvTXotL/Zta89TVBQDSNePDDDDDocV6LA3iGIJiexYzKI5xtoEZGLgC9i2ygAsucTZjjbP13lWF8aDLLsxRcYc9RRjvXZFZw2KOqAD+NLJd6VmYWEToh0TnNve8bhAOlIHvj6X28j0OJM+pf5Dz5Icb/ytF1LneHktaZD/hearYY9dbhaYMACdNNn0b0DwOXQbCIoEMxDVn/Omf5SRDwziq8AhwO/UdX1odxMpgyWQ6x+601H+/M3JyexYFrIq7PrX3dGVz06W9u3dh0UCHiQNWdmbV1jX0MDu9atxUnf4ddv+wtP/vQ6e3k0lyndaNsN2hN0owVVMPUfzo2lLmTBd5EsK4saT66n5cC3wrbXAzclU6ieQs22bfi7m5snEiIMnzXbuf46IVkuia1I9Vi076qO9oRWumqjaI+/EHyl0FiGbD7GjtNIEkl3nU1S/MPONat56vqf8vnbbseVpBoY8ZK0662hHIq22VliwxG/nRPME28lxTgIeJCWYoc6Sx1RZxQi8kjofUmocFCbV+pEzF327d6Ny+tgaQ9V9u7IvjwyUdkxxY6UDoQ9zwQtaMmHfRX2391FAFcTsn4u1sqLkqokUoG3oICKMWM6BswlqEBUFX9Tk12nO0eRLUeBtntmDrihZiTijvEgp+3eOyMosCvhFHlpIdZ/Wniup9MjvAwJUFtZyZLnnnG8BsW6999Dk1RPOdWIepDFX4eNx8PeQVAzAll5AfL+T5GNJxDfymkMXEF0QO6U/Tzuqm+S36sXnvx8xLIcS+4XaGmhfmf2VY6MF2kYgCy/BBr72skAgy7YNRk2noD2+TR6oaxoucPCaT014IbmvsjGOQ5Knjpi5noKVbO7V1WzLzlJBlO/aydP/OTH+BIsNBOJgN9ve1HlSCoGCXqRrUfD1qPb7NfqUbaXUsIDZGdpykiUVFTw+dtuZ8P8D6lcsYKVr73qiEeduN2UjxzlgISZi9SMhgXfsz3sAh4o3YQecqd9fbR613X3X2rbDKRuNOyaiLSfuWQJMaVW1YCIBEWkl6pG8Sk0dJWP582jpbnZMbfYcAaMHx+1rKoTJNsuES8SzEPrB4Yqj3WzEwXZmfrysMm0V7g8HkYdcSQbFiwg6JCrdK/+/Rk4KfX1OlJ9rQkC/gIURcc80dY21l1loYKsPw0Jpqp6dHKIR73VA0tE5GVgX+tOVf1W9FMMsdi+amVS4h3c+fl85tLLHO83Y+lutCzY//R+D7prfJb6OcVm357djj2IDJgwsWclC3Q32pmE29NlJYFt58hyJQHxLfI+DvwMeAtYGPYydJNeAwcmZWnIcrnoNTD7MlN2m7qDun+uAO4WmPAAigNLWBnGsOkzHHOUWP7iC6x+523HZiiZjObVoGMfjS9+pzMEyK9OvJ8MoFNFoar3AY8A76vqfa2v5IuWu0w94yzcSXA11GCQzYsXO95vJqKF26FsWWijm50I0G8VOvZxp8TKGCbOOYHCXr3burQmsCT55t9u5/4rr0i43kUmo1YzOvVv0He1c+6wBT1EUYjI6cBi4IXQ9lQRmZdswXKZfiNG0H/CBMf7VVWa6+sd7zfTUMuHTrnLflqLlHStKwhQvgSNltYjS/EWFnL2Tb9n6lmfo9/w4QyeMoW8BLIKB/1+Gmtree63vyYYzNGZRfkndj6naF5O3SGR5dEMIh4bxQ3AbOANAFVdLCIjkyhTzrPoiSfY+vHHjvergQCDJh3saJ+ZYrxuQ7/l9tJAewXRXc8U8aOzbkHVZafwqJqKbDo+JWvLyUz1kVdUxIxzz2PGuXafd38x8eoAfp+P7ctXMOhgZ6+zVtJ5vWnR9sTrTnQggHprEV8vh/tNLfGou5YIHk+5t6ibInyNjSx4+EHH+3Xn5TFp7smUlOdO5tioeOsjp1xIpJa2FbS9XLz1MOh9dMpdOWe7KK2ocKQfX5Pzbt2ZgOwb0Da405FOQznLspx4vpVlIvJ5wCUiY7DTebybXLFyD39zM5sWL2LN2+841qfldtNr0CBKKyqYMOcEhk5NvatnWqgbFkrPnKQlEJcfCnZC73VQMzo5Y6QBJ1JwBP1+Bk6Y6IA0qUelBSo+ttPC+ErsNOLhOZd2ToHhL3e/RnakGW1oaZNNkSpKZw/xKIpvAtcBzdh1KF4EfpVMoXKN7StX8PyNvwNwLLeTy+tl9JFHcvQVV/Ys10WAvUOhZoR9I2/1dQ+6Ii9HdRfLD0WVOaMomuvrE67N7vZ6mX3xJeQVZV9tcbV86CF3QMEe+5pRQft/hH56NlYorYYE82DxN9CD77bbOXUt5dU51FH6iEdRnKqq12ErCwBE5DzIxMXrzCPQ0sILN91Ii4NR2OOOO47Jp5xG32HDHOszmxAEll+MDpwP/RfYxseqqTD8JXA5tFwkQVQ05TEWyQrGq1yxHBHpdnoXl8fD8NmHsuPTT1ldVMzIww/D5XbGcy8Vdgkd8CEU7LZni2BfM64WGPMkuvtAxLQ090YWfp/g4TfY7tNdIdrF4ivtttyZQjw2imvj3GeIwNalSxzNvdRn6FCOvvzrPVZJtCK4sCoPw1p8NdaibyJ7JjrrrSLAkDfRHEnx4ckvSCj3UzAQYO1777Lmf+/w9j/+zlM/+yl+n4OZj5NN+ZIDSqI9xVvbbKrVbBcrcuJyCrhh42cd6Ci9RL1yRORk7HKng0XkL2GHSoHc+O9JAQGfsxXTqjdv5uHvfIvTfv4Livv1c7TvrEZdoF0tedcJotB7DVSPd67PNDFw4kQ8BQUEWrp3PWrwwEzN39xM9ZYtrHjlFSafcmqX+0qLZ1MgP/J+CXYoVqWT7gVPY+JLTwpsnIO1c2qCHaWfWDOKbcACoIm2EdnzgJOSL1puMGjSJIL+jnrV5fViubpXL7euqopXb/1ToqLlFs2lzq0p70chVprpLMJyuTjlJz/Fkx/lhtlFAj4fa9/NHp8W2XaYnewvHBXwFaO91qMD5qOefainHnptcuZa8hVibc3MCoFdJaqiUNWPQxHYk4H/hEVkP4Vt2DbEQV5xMUd85au4vN79yfrc+fn0HTosoaWAnWvX0libW0FiCdFrg/OF6q0g1OROyFDZiBFMO+dcx/rzFDijdFLCnvGw9XB7ScmfF3p5bUPziBfQkc+is36PDnGo4qQCG3LneTqeO9VLwBzs5IAABaF9RyRLqFxjwvFz6D92HJ++8Tq+hgaGTp/BlsWL2Ll2TUL9+n1OBwdlMZava1GwkVwZW/cpEPTA5mOQlhLHROwqyQjGq96yJeE+wI7bmXjCiY70lQoEQTaehFYejvZeA+WLoM+6jtfAoA/sKDEnnjl2pT7jbrKIR1Hkq+r+vBCqWi8i3c8F0EPpO3Qoh33hi+xYs5rnfvPrhGtRiMtFcVmZQ9LlAHXDu1ZbItLSgmDfJPYOg81HI24fWrYEqscg0da4s4zty5cn3okIE46fk9Kyu91F+65Eh71qZ4OtH2Qbloe+CQW7olwDDrlY+0qRYG5cMxCfotgnItNV9SMAEZkBOOLrKSJzgVux9fddqnpju+N5wL+AGcBu4AJV3eDE2OkgGAjwwo2/w9fQkHBfFaNGpyR+ov1TbEam9AAkkI/WjoA+axL7R7cA1z6YdD+qVig+Q2HlBcge5/NzpZq8kmL27kysXK7L7Wb88XO6dP2l47oJVnwEo+cdiLXps9qOveks1Uui/1ZBC9acbrtx5wjxzNW/AzwqIm+LyDvAw8DViQ4cqp73V+BkYCJwkYi0D/m8FKhW1dHAn4CbEh03nWxfscKuQJcgbm8ek085xQGJcgzLgaU4BYp228rBCti5f1wt6PiHUXfiCj7dTDntjIRT3FsuN3s2bXRIouSgBGHk8x2LD0kAXDEi+hP5ahTwFSBLLsXak53R69HodEahqvNFZDwwLrRrlao64fM5G1ijqusAROQh4EwgfG58JnZSQoDHgNtERDRLi0L7W1pI9HHFnZfHkEOmctCMmc4IlUtYDiwbRDtfxU5GWJW+792JYLxRRxzBjjWrWfrcs92WQzVISUX/zhumE08DuCI8OCTzIV8Fth2G1A1P4iDpIV63m3HYT/35wPRQhOe/Ehx7MLA5bHsLcGi0NqrqF5FaoB+wK7yRiFwOXA5QVpa5gWgDJ4xHE0zRPHTqVOZ89/s9L21HPNSOil4atfXRotuJAwNgORsTk0oC/hbWvfc+G+Z/SH5pCZ6Cgm5lC7DcbnoPGkz5qAyvoe3PD8XVpBBLoffGtne1HKFTRSEiPweOxVYUz2EvFb2DbTvICFT1TuBOgFGjZmbsbMOTX8BRl13OW3f+nWAg0K1yqHVVVWlVEuFPsplmr5CtR6KD344eoZ2Q7SKAempBWhB1vuhUV+nK7CLQ0sLTN/ycPZs32bnGRLpdJvWgGTM5+vIr4roG03l9iLrRylkwcH7b5afupqIPJ1ofQcCXfXmw4iEeG8W5wPHAdlX9CnAI4ERy9a3A0LDtIaF9EduIiDs07m4Hxk4bY446mnNu+j3jjjl2f1xFV8j4KX8akZZi2D7T0eDsA50DQ99FJ9+TdenHV7/11gElAd1WEpbHwwnf+z55xcUOSpc8ZP1cqJxlB9oF3HbchFOxNgGr43UmQNlyghW5Vyk6nqWnRlUNiohfREqBHbS9wXeX+cAYERmBrRAuBD7frs084EvAe9gK67VstU+E03vQYLxFhV02Krq9XqaecWaSpMoNZNMctGw5uBvtgDmwbxLSzdTR4VgBKN4OfT+1A7iyhLXvvxt31mJxuaLOdMPTeEQi42aYuJD1p6IbTrRTcviK0JHPwYCFB5YRu3tNNPWzg/ZKtxzoo9VYPnoeumcC4s+dKIJ4HmkXiEhv4B/YKTw+wr5xJ4Sq+rG9p14EVgCPqOoyEfmliJwRanY30E9E1gDfA36c6LiZwr7du7u09JRXVMwx37iaijFjkihV9iMtxchHV0PlbGjoBzXDkZUX2UZoJx4xXD60V+R03UoATcp0JjG8BZFvWGJZWG43noICLI+H8Z89nrN+/duodSvyS9IXfNgd1PIRHPABOvYxdNC7kFeLrD8Z6hJ8zhWgcCc0RYljUgv6rkpsjAwj5oxC7IXI36lqDXCHiLwAlKrqJ04MrqrPYds9wvddH/Z3EziYazmDGHLIVDYuXBjXk15B7958/va/4XI5XH0rR5GWUmTd6W13Vo9CyxeDO0H35KDYhtIwtNd6dNQ8++YR9KDbDkU2noA4nVKkm0w84UQ2L17U4Vor7N2bM3/1Gxpqquk9eAjeggIAxh9/PCtfe41AWOS/Oy+PKae1+04zGHU3olNvB+9e20YRdKGD3odd46HXeme8nworbYN5JJtYjtTKbiXmnUdVVUSew873RDYHu2Uao444kk+eeZq6qqr9/5Aur3f/9D7o9yMiuDxejvn6NzJSSWRLMB6AqAetHWkvGyVykxCFQR+g22cjLcVo0TZ00n0HDKYun11K1bMPWX2OI7J3RmepPgZPnszUs85i0eOP24koRXB7vZx87XUUl5V1iPA/7JIv0lS3lw3zP8Tl8RBoaWHsMccw5dTTkv5ZnEKHvGlHY7eWzLUCQAAqljijJAT7WlA30M4bThT2jIt0VtYSz93nIxGZparzky5ND8Lt9XLWr3/DshdeZO17/8OTn8/EE+cycMIElj7/HJXLl9Nr4ECmnHY6/YYPT7e4uYETHikCuPehQ95A1p+GDn3TroYXjqsFKj5B189F/JnhBTP97HMZ/9k5VK5YTl5REYMmHRw1e7HL4+H4b3+Hhppq9u7YQa+Bg7Jq2Unzd8GA+ZHrqjs2CFA/FHZPhCGt5Y1DicJWnZczKV9aiUdRHApcLCIbgX2Evg1VnZJUyXoAnvwCpp51FlPPOqvN/kMvviRNEiVGJrvOAtD/E2eeJi2FQe8T3DgHCqsiLz0EXZBfA/WZoSjAXmoadXj8uTwLe/ehsHefmG0y7XcO9l0GEx6y60xEwknP8o3HY/l6oTum2TaJoAd2T0JaMuc3d4p4FEXu5Mo19GzEwSdMUZj+f/ZsIpJfvQSgqa9z4xk6RcUP4x8+4O2WlEFC76vOxvLZUQLSVAbbcjtBZzwpPDYCiEgFdmS2wdApGTm7UCv6k2ZXEewZQ6Qn1IAHts9E/AXOjNVFklV3O9PR0o3JXW4C+/cOSNfraWc5nZrmReQMEVkNrAfeBDYAzydZLoPBMdRbS3D4i7aNov0qUSLerJHqWQRcsOk4ZJ1J2phyXE0pGkdtG0gPIp6lp18BhwGvqOo0ETkOyM5FdEOPQwt3oIfcYQfcuQL2zTxcOTi5Zr0/O2kzEleIUvJxenaRMbPDSNSNSN1YyZ65ZBjxXM0tqrobsETEUtXXAZO61JAV6MhnwNV8ILV0q2KoH5AcX3cLGPwuaplqwalELR/0XWHHNSQ75jHggqqpSR4ks4hnRlEjIsXAW8D9IrID2/vJYIiLtMZb9NoQwdAMFFfB7nHQb6XzqaclgB56kx2lXTccVp+J5evt8CBdp7ulVTN6FgFo8VZ08t32AwEkN5V4wIKG/si2nlUJOp5HqjOBBuC7wAvAWiB7QjQNPZtglEyvQTes+VxyxpQguJvtokd9PoXZNxPstzQ5Y/VwlCA68d/29y0kR0ko4CuE6lGw6gJk8deRoDcJA2UuUWcUIjIa6K+q/wvtCgL3ichngN5keRZXQw+hchYMeg9cYUFxATdUTcdqKSa4fYadJM6pG0x7V9nWv8c/gi4chBiXWWcp3nZgJuE0ih0Ps/Y0rKrMrw+eTGItPf0ZuDbC/trQMTOrMHSLVLrOysY5aMEuu16yumxjc+0IOzkcIBvmomXLwN2U3CULCaD9FyIbT0jiIF0j091o1dWE9l9g/3bNvZFthyMNA9o2kiBJ++H8BfDRVVi+2EGHPYFYiqK/qi5pv1NVl4jI8KRJZDA4iKgbWXEJmr/bTtrXWIY0HgiOEn8huvJCmPiftrMOxwUB+q5Aq8dA3UFIUrVS9qPuBnTaX8GzL5TUT9CKj9F1JyOWHySI1g6DAe+B044DQQvUhaz4PGKUBBBbUcSyvqUnkshg6CbS1M+uIRDpWM1otCYsYWAiVdCi1twGiqrQg++18wOtOi+qstDizWj/j8BqQXZNhuqxSVcsmWaw1sHv2Jlf9yf1U6DFrvXQvnGiv1mbgbGDJbceZZYJw4ilKBaIyGWq+o/wnSLyNey6FAZDTiAI6m5i/x0nWs3thLLOht5dLVD2CbpjMlI9oUOz4NA3YOgbdtyHpWj5MtgzFlZe2LNmIf1WRI5ViPYVOPXVBN3IulMyotxtJhFLUXwHeEJELuaAYpgJeIEkuYsYehqZkKpcvXVQsi1KXQEJ1RxwMH+QpTDhQYILv4W09LJvTgjqrYWhr7ddAnP57JlO77VQM9o5GTIdfxqyBQVcUDXdKIkIRFUUqloFHBGKxD44tPtZVX0tJZIZDKnC1Rw9+M5XApUzYeibB4L2nMAKwKw/2csoAS+65WjwFRPRY93yoUNft6u0+UqRytnIvkHOyZJhaF51qNYDyXUwgFCkvgAW7JqCrDs1yQNmJ/EkBXwdeD0FshgM6Ukm2NjPdpl1+druD7hgxxRk+yx06NuAk9lnw/52+2DYq7BnjL3kFInSTXZW1KCgFYvR1Wdg7ZzunDwZgm3Evh3cDalREs29YOkXkZZeaUvimA1kRkIagyGNCBay+mw762sw9C8R8EBLMbLlaKSlFDYdl9zUEJZCv0+jL3G1ps621LZzjH4atXIng6l6a9DB76DjHrCT+6VCSTT2RpZcitU4wCiJTsi8+poGQxqQPRNg8ZXooPcgrxpqRiPbZx2oVJa/J0WCtNuOuvwiULzVThGSRFT8YAWQQF7SxgiWL4YxT9gxLhLFmSARIn2HasGuQ2xvOEOnGEVhMISQhv7ImrMiH3Q1J/8ptyv9i0ISb95qNaOj50H5EkDRpn7I6rOQOBWTuhuhbAl4GqBmJOwd2sZrS/Oq0f4LbQVcviS5xYYifa9W0Fa0hrgwisJgiAPZNRntu8q2J6SaoITiCEIotpF934EoZS3eCgU7oaHCEUO3TrwfwgsBFe60Y0AWXQ1NfaDfMvv7cDWD5YNAAVI1A6rH2Eb3kc8d6GyYBdWj0arpiIBaTTDmqQOzh2Rne41E0AX7BqZh4OzEKApDxpJRVfJ2T7CXeXptsI3eqfDIaSWSB5BrH9pvGdSMQSfdC8WVITdeResHI0u/FDNxnaJQstk2Gu8dhvgLDxzL32UrifaR6uJHB70NxduhqMq2lYTJpf2W2zdgV0u7fFfBkIvvOlQt+/uLlA8rmYR/f6EcTrLtsBQMnBsYRWEwxIFgwbIv2Dfn8Y90banE6UhvAbxNMOFBNOCxn/rD5Snego54AdbPRSsWQd/V0NwLqTwUaahA8/egB98NnvpQ/4pu+izWlmPt8/OrQy6j7bDUjufIq7eVQXv5rGD070VIboqUzlDLdlSwAlA3FFl7BpIBqd+zBaMoDFlBJgTmCRYEvWjQ43x+oe4J1PHpHewb8sAP0IEf2Dd8S2232v4L0Q3Hw/CX7Rtm+HkHvUKwfhBWzVi0sd8BRRCOAt46Z+NJko1i521a+mWoHQFoxlQfzCbMN2YwdIVI0duxSPYSVayUFsIB20arW+3IFzoqCbA/V8iuIHl1kQMQhcxSEvH8FApUzkRqRyKIURLdxHxrBkNXqBnZtXQemZieKdpyVsFu1N1gF3VSV/Rz02F8bk/cCthCdvassqXJwCgKg6ELSNALq86zI7mDrtDSRtgr04l1c5UgOvsmtM9yiBXMFy1pYiqJR0koUD8I2Tss2dLkPMZGYchK0ukRZe2ehC74HpQvQV2NUD0W3I0w7mHbo8ci5Flj2cbdTJxVRKLV4HzQm12X2clU3/HS2XhBN7LmzFRJk9MYRWEwdAPx9YKtn2lzn9L516AD37dTZPtKYNvhMORtu0JbtiiLREj1Z4xV+wNg26E5nTwxlRhFYTA4hATykS3HQqubKaC1I2xX1F6bkht97CTZrtQE8LuR6vHpliRnSIuNQkT6isjLIrI69B6x3qCIBERkceg1L9VyGgyJIljI0kuRFZ+PHJvQSjbYN9JFLPtP1O/Ngrza5MjTA0mXMfvHwKuqOgZ4NbQdiUZVnRp6nZE68QwG5xAsO+ng7vGRb2za7t3Qllb7R/vvRwF/3oGMv23OCcLewcmXrYeQLkVxJnBf6O/7gCiZ2AyG3EHWnQr+graKodUgK+32Gdoi2Df+gMf2NvN77Sp4y75k15QIhLnzBjywZzzSWJE2cXONdNko+qtqZejv7UD/KO3yRWQB4AduVNUnIzUSkcuBywHKyowrnCEzkeY+sOD7aP8PoWIxFO5o+6jWxjIeYV9PIJonk2CnO//ka3a+LV8J7J6ABL3o4m+gQ1+D8mUQ8ELlbJPHyWGSpihE5BVgQIRD14VvqKqKRA13PUhVt4rISOA1EVmiqmvbN1LVO4E7AUaNmmmex3oYGZU8sBPEX4BsPQZtrEDHPZoZqUAyhc7cXb31SP0QqB/SZrf4C5H1p8H605IqXk8maYpCVedEOyYiVSIyUFUrRWQgsCNKH1tD7+tE5A1gGtBBURgMWUdjWecR3pk0m3A6RiKUwXW/J1jAYwf5RXtmDArUDndQAENXSNfS0zzgS8CNofen2jcIeUI1qGqziJQBRwK/T6mUBkOSkMZytG6YnY012lJLphBvTiWIv0KfWrDsy1B7ECJBVF0w849QUB2576AX2XhCF4Q2OEm6jNk3AieIyGpgTmgbEZkpIneF2kwAFojIx8Dr2DaK5WmR1mBIArL8EtgxpWsG7PYpQ1KVPqQzxRVujKfd3+1lCwrUjMSqHYmFC1EPVqS65Rpqu2sSOYhtsAAADZlJREFUsugqpLEs0U9h6CZpmVGo6m7g+Aj7FwBfC/39LjA5xaIZDClDgl7k0wsIrj0dZt0Mnjiq5/kLYf53kMLdtrePr9QuNDT5HsirS85MJN4+fSWwr6LtLEmw40dUQ8kGLfCVIp+e23GY2pGw6Cq7Ql7BLqgdYdfQ8Bc59UkM3cREZhsMacYKFBLccgwMe71tcZ/w5RzFvtGuPhMrUAR7D9w8pakcnf8D9KBXYcCH4Gk8cF48OGF/UGDTZ7G2zyY45U4o3XTA3iBqK4v6gcjGE20FEGVAaSxH1pr8TJmGURQGQwYgW49E+6yGkm0g/gNP37UH2TOFhgpk25HgaiY4+S7I3w31g5GNc5CGAQgu+ya88USUIJQtRSsWQ+kGcDdHVwRBFxDsep2NcBTY1x/ZPguVFijd3LE/USiqsmcNhqzDKAqDIQMQ9cCSr0Gv9VCyxV5W2j0RCXr2twn2WwrjHjtQfS5vL9pnDXx8GbLvQBSyYMGuKciuKQQrFsDYJzoOGDZDYejbdkxHtPThsQzUCgS8yNIvIwiKxDBgR6lxYch4TD0KgyFDEMSuxLblaGTnIW2UhKIw6pm2JUpFwWqx62NH63PnIXaai/YEXchH38baOd3OQ9WS39EoHgjNaiKlzlCxX3uHIh9fgbSUhkRyw55xHdNqBNxQNS2u78GQeRhFYTBkA64m8DR03C9Aydaop4l6kKWXgq84lPYiz35fdQHS1Ndu01iOfHgtrDsZ9vW32+wrhzVngq+w4+zAAvz5yP9+gfXx15GGtnG1suZz0NTPHifgsV/1g4x7axZjlp4Mhmwg6A3VsY5Qs9pXEvNUqR8MH1xjL2lZfqgbai91hbdRN7LtM7DtM22HHRMxa45dqCmKWUNaimDht6D3etuWsm+APfPIqOAQQ1cwisJgyAJEXWjlLBg4v+3yU8ADm4/p/Hws6E5J0IYyKK7quL+pD0J0m4NgQc0oYFTXxzRkHGbpyWDIEmTDXHudP+AOLet4YeNxyI7krf3L+lNsZRROwAPrT07amIbMw8woDIYsQdSFrD0T3TAXPPXQXNphCcnxMWtGw7Ivoge9bHtGNfVDNpyA1IxJ6riGzMIoCoMhy5BAHgQieDIla7zakcgnV6RsPEPmYZaeDAaDwRAToygMBoPBEBOz9GTIejK9WJHBkO2YGYXBYDAYYmIUhcFgMBhiYhSFwWAwGGJiFIXBYDAYYmIUhcFgMBhiYhSFwWAwGGJi3GMNWYlxiTUYUoeZURgMBoMhJmZGYcgKzAzCYEgfZkZhMBgMhpgYRWEwGAyGmBhFYTAYDIaYGEVhMBgMhpgYY7YhYzEGbIMhMzAzCoPBYDDExCgKg8FgMMTEKAqDwWAwxCQtNgoROQ+4AZgAzFbVBVHazQVuBVzAXap6Y8qENKQFY5cwGDKPdM0olgJnA29FayAiLuCvwMnAROAiEZmYGvEMBoPB0EpaZhSqugJARGI1mw2sUdV1obYPAWcCy5MuoMFgMBj2k8k2isHA5rDtLaF9HRCRy0VkgYgsqKvbmRLhDAaDoaeQtBmFiLwCDIhw6DpVfcrJsVT1TuBOgFGjZqqTfRv+v717j5GqvMM4/n2CImlFEaHITRFDL2gJChprLdFKbEsaLRHFtImY2igxmPYP09iQmFQTe7FpUrXWWkoiBpSUSksrhqJo2tgioLIseKlgtAURtKZ4aatSf/3jfadOlpmzM7s7F2afTzLZM+e8c86Td2bnnTnnzO80lo9JmLW/hg0UETG7n6vYA0wsuz8hzzMzsyZq511Pm4Epkk6WNBS4HFjT4kxmZoNOq06PnQvcDowGHpS0NSK+IGkc6TTYORFxUNIiYB3p9NilEbGjFXltYHl3k9nhpVVnPa0GVleY/wowp+z+WmBtE6OZmVkP7bzryczM2oAHCjMzK+Qy49ZwPiZhdnjzNwozMyvkgcLMzAp5oDAzs0KK6KyKF5JeA16uoeko4PUGx+kP5+u7ds4Gztcf7ZwNDu98J0XE6EoLOm6gqJWkLRExs9U5qnG+vmvnbOB8/dHO2aBz83nXk5mZFfJAYWZmhQbzQHF3qwP0wvn6rp2zgfP1Rztngw7NN2iPUZiZWW0G8zcKMzOrgQcKMzMrNGgGCkmXStoh6QNJVU8Pk/SSpG5JWyVtacN8X5T0vKSdkm5oYr6RktZLeiH/Pa5Ku//mvtsqqaEXmuqtLyQdJWllXv6EpEmNzNOHfFdKeq2sv77RxGxLJe2XtL3Kckm6LWffJumMNsp2nqQDZf12Y7Oy5e1PlPSopGfy/+w3K7RpSf/VmK3+/ouIQXEDPgV8AngMmFnQ7iVgVDvmI13AaRcwGRgKdAFTm5Tvh8ANefoG4AdV2r3dpDy99gVwLXBXnr4cWNnE57OWfFcCdzT7tZa3PQs4A9heZfkc4CFAwNnAE22U7Tzg963ot7z9scAZeXo48NcKz21L+q/GbHX336D5RhERz0bE863OUU2N+c4CdkbEixHxHnA/cHHj00Hezj15+h7gK03abjW19EV55lXABZLURvlaJiL+CLxR0ORiYFkkG4ERksa2SbaWioi9EfFUnn4LeBYY36NZS/qvxmx1GzQDRR0C+IOkJyVd3eowPYwH/l52fzcD8CKo0ZiI2JunXwXGVGk3TNIWSRslNXIwqaUv/t8mIg4CB4DjG5ip4razas/VJXnXxCpJE5sTrSatfK3V4jOSuiQ9JOnUVoXIuzNPB57osajl/VeQDersv466HoWkh4ETKixaHBG/rXE150bEHkkfA9ZLei5/wmmXfA1TlK/8TkSEpGrnVZ+U+28ysEFSd0TsGuisHeJ3wH0R8a6ka0jffj7f4kyHg6dIr7O3Jc0BfgNMaXYISUcDvwa+FRFvNnv7RXrJVnf/ddRAERGzB2Ade/Lf/ZJWk3YhDMhAMQD59gDlnzon5HkDoiifpH2SxkbE3vwVen+VdZT670VJj5E+0TRioKilL0ptdks6AjgW+EcDslTSa76IKM+yhHQcqF009LXWH+VvfBGxVtKdkkZFRNOK8Uk6kvRGvDwiHqjQpGX911u2vvSfdz2VkfRRScNL08CFQMUzL1pkMzBF0smShpIO0Db0zKIya4AFeXoBcMg3IEnHSToqT48CPgs806A8tfRFeeZ5wIbIR/OaoNd8PfZZX0Tan9wu1gBX5LN3zgYOlO16bClJJ5SONUk6i/Q+1qwPAORt/xJ4NiJ+XKVZS/qvlmx96r9mHIlvhxswl7Sf8F1gH7Auzx8HrM3Tk0lnp3QBO0i7hNomX74/h3Qmw64m5zseeAR4AXgYGJnnzwSW5OlzgO7cf93AVQ3OdEhfADcBF+XpYcCvgJ3AJmByk19zveX7Xn6ddQGPAp9sYrb7gL3A+/l1dxWwEFiYlwv4ac7eTcGZgi3Itqis3zYC5zT5eT2XdCxzG7A13+a0Q//VmK3u/nMJDzMzK+RdT2ZmVsgDhZmZFfJAYWZmhTxQmJlZIQ8UZmZWyAOFdZx8nvj9knblUixrJX28gdubnn/hWu/jxklaVbB8hKRra21fx3ZnSXpK0kFJ8/q7Put8Hiiso+QfEq0GHouIUyJiBvAdqtemOuTxkur9v5hOOle9npxHRMQrEVH0Rj2CVAEXgBra1+pvpMq1KwZgXTYIeKCwTnM+8H5E3FWaERFdEfEnSUdLeiR/mu6WdDGk4mlK141YRvol/kRJP8vFDXdI+m5pXZLOlPTnXFBtk6RjST+im59r+8/Pv/Bfmpc/XbadKyWtkbQBeCRvd3tedmpuv1WpSOAU4PvAKXnerT3aD5H0I0nbc/vr8vwbJW3O8+8u/QK3XES8FBHbgA8a8gxYx+moWk9mwGnAk1WW/QeYGxFv5hIjG/XhxZWmAAsilYRG0uKIeEPSENKb+jTgOWAlMD8iNks6BvgXcCPpl7eL8mNvIZUL+bqkEcAmpYKLkK6zMC2ve1JZtoXATyJieS75MYR03Y/TImJ6Xm95+6uBScD0iDgoaWSef0dE3JTb3wt8mVR80KzPPFDYYCLgFkmzSJ+mx/PhLqmXS4NEdplSmfkjSBeDmUoqjbA3IjbDh8XVKnxovxC4SNL1+f4w4MQ8vT4iKl1r4S/AYkkTgAci4oUK6y03m3RRpoM5S2md50v6NvARYCSpVIMHCusX73qyTrMDmFFl2deA0cCM/Cl9H+lNHOCdUiNJJwPXAxdExDTgwbJ2tRBwSURMz7cTI6JU8O+dSg+IiBWkwoD/BtZKqrvcuKRhwJ3AvIj4NPCLOnObVeSBwjrNBuAolV10StI0SZ8jlRnfHxHvSzofOKnKOo4hvaEfkDQG+FKe/zwwVtKZeb3DlcqXv0W67GTJOuC6sgqdp/cWWun6HS9GxG2kyrzTKqy33Hrgmrx98q6n0qDwutL1CHxGkw0IDxTWUSJVuZwLzM6nx+4gVWl9FVgOzJTUDVxBOuZQaR1dwNN5+Qrg8Tz/PWA+cLukLtKb9TBS5deppYPZwM3AkcC2vP2ba4h+GbBd0lbScZZlka5X8Xg+MH1rj/ZLSGcvbctZvhoR/yR9i9hOGqw2V9pQPiC/G7gU+HnOaFaVq8eamVkhf6MwM7NCHijMzKyQBwozMyvkgcLMzAp5oDAzs0IeKMzMrJAHCjMzK/Q/fL9gzIUttgEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import neurolab as nl\n",
    "from matplotlib.colors import ListedColormap\n",
    "# Create train samples\n",
    "input = X\n",
    "target = np.array(y)[np.newaxis]\n",
    "target = target.T\n",
    "# Create network with 2 inputs, 5 neurons in input layer and 1 in output layer\n",
    "net = nl.net.newff([[X.min(), X.max()], [y.min(), y.max()]], [5, 1],  [nl.trans.LogSig(),nl.trans.LogSig()])\n",
    "# Train process\n",
    "err = net.train(input, target, show=15)\n",
    "# Test\n",
    "cmap_light = ListedColormap(['#AAAAFF','#AAFFAA','#FFAAAA',])\n",
    "x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5\n",
    "y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5\n",
    "xx, yy = np.meshgrid(np.linspace(x_min, x_max, 100),\n",
    "                         np.linspace(y_min, y_max, 100))\n",
    "Z = np.zeros((100,100))\n",
    "for i in range(100):\n",
    "    for j in range(100):\n",
    "        #print([xx[1,i],yy[j,1]])\n",
    "        Z[i,j]=net.sim([[xx[1,i],yy[j,1]]])\n",
    "Z = np.round(Z)\n",
    "plt.title('Problema de 2 clases', fontsize=14)\n",
    "plt.xlabel('Caracteristica 1')\n",
    "plt.ylabel('Caracteristica 2')\n",
    "plt.pcolormesh(xx, yy, Z.T, cmap=cmap_light)\n",
    "plt.scatter(X[:,0], X[:,1], c=y);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Batch, Minibatch and online learning\n",
    "\n",
    "Durante el entrenamiento de un MLP las siguientes cantidades deben ser estimadas en cada iteración:\n",
    "\n",
    "<img src=\"Images/ForwardBackward.png\" alt=\"FBI\" width=\"500\"/>\n",
    "\n",
    "El entrenamiento que acumula todos los errores cometidos a partir de todas las meutras de entrenamiento antes de llevar a cabo la actualización de los pesos se conoce como **Batch training**. Desafortunadamente si el número de muestras es muy grande, el algoritmo presenta problemas de computo no sólo por la cantidad de cálculos sino de memoria necesaria para realizar la multiplicación de matrices. Pero el problema más significativo es que la trayectoria del gradiente Batch tiende a estancarse en puntos silla (**saddle point**) de la función de costo.\n",
    "\n",
    "Como alternativa el entrenamiento se puede realizar propagando una sola muestra hacia adelante y realizando la actualización de los pesos a partir del error que se comete con esa muestra. Los dos pasos son repetidos para todas las muestras de entrenamiento. Esta estrategia es llamada **on-line learning** y el algoritmo resultante es llamado **Stochastic Gradient Descent (SGD)**. Debido a que el algoritmo usa una muestra aleatoria a la vez, la convergencia al óptimo es más ruidosa, pero eso ayuda al algoritmo a escapar de óptimos locales y puntos silla.\n",
    "\n",
    "<img src=\"Images/SGD_MB.png\" alt=\"SGD\" width=\"600\"/>\n",
    "\n",
    "Uno de los problemas del algoritmo SGD es que requiere muchas iteraciones para converger. Por consiguiente una solución intermedia, llamada **Mini-batch gradient descent**, parte las muestras de entrenamiento en mini-batchs, y se realiza los pasos forward y backward por cada mini-batch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 ok\n",
      "2 ok\n",
      "3 ok\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f6ccc4f9150>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3deZwU9Z3/8ddnuufgGEBgvBguCWKQSxgQNHHZHzECq2gMRlBXMYcag+huNEHdeBDjEd3EzYqJrovHb38egZVIdIwoYrwiMIYBAUEQUQdBRlQOGebo+fz++HbP9Aw9Mz09fVXxeT4e/eiq6uqqTw/Ne77zrapviapijDHG+3IyXYAxxpjksEA3xhifsEA3xhifsEA3xhifsEA3xhifCGZqx71799YBAwZkavfGGONJb7/99meqWhTrtYwF+oABAygrK8vU7o0xxpNE5MOWXoury0VEJovIJhHZIiJzY7z+WxEpDz/eE5EvO1KwMcaY9muzhS4iAWA+cDpQAawSkSWquiGyjqr+S9T6VwEnpaBWY4wxrYinhT4O2KKqW1W1BngSOLuV9WcCTySjOGOMMfGLpw+9D/Bx1HwFcHKsFUWkPzAQeLnjpRljslltbS0VFRUcPHgw06X4UkFBAcXFxeTm5sb9nmQfFJ0BLFLVUKwXReQy4DKAfv36JXnXxph0qqiooLCwkAEDBiAimS7HV1SV3bt3U1FRwcCBA+N+XzxdLtuBvlHzxeFlscygle4WVX1QVUtUtaSoKOZZN8YYjzh48CC9evWyME8BEaFXr17t/usnnkBfBQwWkYEikocL7SUxCjgBOAL4W7sqMMZ4loV56iTys20z0FW1DpgNvAC8C/xRVdeLyDwRmRa16gzgSU3xeLwbN8IvfgHWbWeMMU3FdR66qpaq6vGqOkhVfxVedpOqLola5xZVPeQc9WT785/htttg3DiorEz13owx2SwQCDBq1ChGjhzJ6NGjefPNN1td/8svv+T+++9vc7sTJ0705IWPnhvL5brr4LnnYPNmOOssqK3NdEXGmEzp1KkT5eXlrFmzhjvuuIPrr7++1fXjDXSv8lygA0ydCo89BitWwK9+lelqjDHZYO/evRxxxBEA7N+/n0mTJjF69GiGDx/OM888A8DcuXN5//33GTVqFNdddx0Ad911F8OHD2fkyJHMndvYybBw4ULGjRvH8ccfz2uvvZb+D5SAjI3l0lHnnQczZsBdd8EPfgB9+7b9HmNMalxzDZSXJ3ebo0bBvfe2vk5VVRWjRo3i4MGD7Nixg5dfdpfAFBQUsHjxYrp168Znn33G+PHjmTZtGnfeeSfr1q2jPFzs888/zzPPPMOKFSvo3Lkzn3/+ecO26+rqWLlyJaWlpdx666289NJLyf2AKeDJFnrEnXeCKtx4Y6YrMcZkQqTLZePGjfzlL3/h4osvRlVRVW644QZGjBjBt771LbZv386nn356yPtfeuklLr30Ujp37gxAz549G14799xzARgzZgzbtm1Ly+fpKM+20AH694erroLf/AbmzQMbjdeYzGirJZ0OEyZM4LPPPqOyspLS0lIqKyt5++23yc3NZcCAAe0+pzs/Px9wB17r6upSUXLSebqFDnD11SAC992X6UqMMZm0ceNGQqEQvXr1Ys+ePRx55JHk5uayfPlyPvzQjThbWFjIvn37Gt5z+umn8/DDD3PgwAGAJl0uXuTpFjpAcbHrT3/oIbjlFujaNdMVGWPSJdKHDu5y+UcffZRAIMCFF17IWWedxfDhwykpKeGEE04AoFevXpx66qkMGzaMKVOmcPfdd1NeXk5JSQl5eXlMnTqV22+/PZMfqUMkxdcBtaikpESTdZ7nG2/AN74Bjz4KF1+clE0aY9rw7rvv8vWvfz3TZfharJ+xiLytqiWx1vd8lwvAKae4/vPHH890JcYYkzm+CHQRuOACeOkliHEg2xhjDgu+CHRwgR4KwcKFma7EGGMywzeBfuKJMGQIPPtspisxxpjM8F6gv/KKG9ClpuaQl6ZOdS9/9VXaqzLGmIzzXqCvWgX33BNzVK6pU6G6GpYvz0BdxhiTYd4L9JxwyfX1h7z0zW9Cly5QWprmmowxGSEiXHTRRQ3zdXV1FBUVceaZZwKwZMkS7rzzzla38cknnzB9+vSYr82aNYuBAwcyatQoTjjhBG699dY2a3rkkUf45JNP2lxn9uzZbW6rvXwV6Pn5MGkSLF2a5pqMMRnRpUsX1q1bR1VVFQAvvvgiffr0aXh92rRpTUZQjOXYY49l0aJFLb4eufiovLycRx99lA8++KDV7cUT6Kniq0AHmDgR3n8fMvTzNMak2dSpU3nuuecAeOKJJ5g5c2bDa9Et4VmzZjFnzhxOOeUUjjvuuIYQ37ZtG8OGDWtzP5GxYLp06QLAvHnzGDt2LMOGDeOyyy5DVVm0aBFlZWVceOGFjBo1iqqqKlatWsUpp5zCyJEjGTduXMPQA5988gmTJ09m8ODB/OxnP0vKz8J7l/63EeinneaeX33VDa9rjEmDTI2fC8yYMYN58+Zx5plnsnbtWr7//e+3OH75jh07eP3119m4cSPTpk1rsasl2nXXXcdtt93Gli1bmDNnDkceeSQAs2fP5qabbgLgn//5n3n22WeZPn069913H/fccw8lJSXU1NRw/vnn89RTTzF27Fj27t1Lp06dACgvL2f16tXk5+czZMgQrrrqKvp2cBxw77XQIzdObSHQR46EwkIX6MYY/xsxYgTbtm3jiSeeYOrUqa2ue84555CTk8PQoUNjDqcbS6TLZefOnSxbtqzhNnfLly/n5JNPZvjw4bz88susX7/+kPdu2rSJY445hrFjxwLQrVs3gkHXjp40aRLdu3enoKCAoUOHNgwg1hHebaG3MAZNMAinnmqBbkxaZXj83GnTpnHttdfyyiuvsHv37hbXiwyJC24wr+YuvfRSVq9ezbHHHktps7MrunbtysSJE3n99dcZPXo0V155JWVlZfTt25dbbrkl4eF5IXlD9MbVQheRySKySUS2iEjMIwwi8j0R2SAi60UkdaOqtNHlAu5sl/XroZV/V2OMj3z/+9/n5ptvZvjw4R3azsMPP0x5efkhYQ7uDJoVK1YwaNCghvDu3bs3+/fvb3JQNXqI3iFDhrBjxw5WrVoFwL59+1I6tnqbLXQRCQDzgdOBCmCViCxR1Q1R6wwGrgdOVdUvROTIVBUcT6CPH++eV62CyZNTVokxJksUFxczZ86clGw70odeU1PDpEmTOPfccxERfvSjHzFs2DCOPvrohi4VcAdfr7jiCjp16sTf/vY3nnrqKa666iqqqqro1KlTSm9l1+bwuSIyAbhFVc8Iz18PoKp3RK3za+A9VX0o3h0nPHzuQw/Bj34EH3/sBkOPYc8e6NHD3cXoF79o/y6MMW2z4XNTLxXD5/YBPo6arwgvi3Y8cLyIvCEib4lI6trFcbTQu3eHE05wLXRjjDlcJOsslyAwGJgIzAT+S0R6NF9JRC4TkTIRKausrExsT22c5RIxdqwL9Azdv8MYY9IunkDfDkSfHFkcXhatAliiqrWq+gHwHi7gm1DVB1W1RFVLioqKEqy49bNcIsaOhZ07YXvzSo0xxqfiCfRVwGARGSgiecAMYEmzdf6Ea50jIr1xXTBbk1hnozi6XADGjXPP1u1ijDlctBnoqloHzAZeAN4F/qiq60VknohMC6/2ArBbRDYAy4HrVDU1Jw3GGegjRrhVk33xmjHGZKu4LixS1VKgtNmym6KmFfjX8CO14gz0Tp3g+ONh7dqUV2SMMVnBd5f+RxsxwgLdGD+rqKjg7LPPZvDgwQwaNIirr76amhg3v2nJK6+80q6hdrOd9wI9zoOi4AJ961YIX7RljPERVeXcc8/lnHPOYfPmzbz33nvs37+fG2+8MaHtxTPUbrbzbqDH2UIHeOedFNZjjMmIl19+mYKCAi699FLAjYfy29/+lgULFnD//fdz7rnntmt42niG2gU3WNfYsWMZMWIEN998c2o+XIK8OzhXHIE+cqR7XrsWTjklhTUZc5i75i/XUL4zuWcgjDp6FPdObnnQr/Xr1zNmzJgmy7p160a/fv2oq6vr8PC0sYbaXbp0KZs3b2blypWoKtOmTePVV1/ltMi43Rnm6xZ6377uqlHrRzfm8NPR4WljDbW7dOlSli5dykknncTo0aPZuHEjmzdvTkX5CfF1C13EDowakw6ttaRTZejQoYfcOm7v3r189NFHBIPBmMPTLl68uOG+oA891PrQU7GG2lVVrr/+ei6//PJkfYyk8l4LvR1nuUBjoNsQAMb4y6RJkzhw4ACPPfYYAKFQiJ/+9KfMmjWLzp07x3zPd77znYb7g5aUxBzfqlVnnHEGCxYsYP/+/QBs376dXbt2Jf4hksx7gd6Os1zABfq+fbBtW+pKMsakn4iwePFiFi5cyODBgzn++OMpKCjg9ttvT9k+v/3tb3PBBRcwYcIEhg8fzvTp0xvGPs8GbQ6fmyoJD5/7/PMwdSq89RacfHKbq7/5pruD0Z//DOHTTY0xSWDD56ZeKobPzS7t6EMHOPFE9xzjdn/GGOMrvg/07t3dfTDWrUthTcYYkwV8H+gAw4ZZC92YVMhUl+3hIJGfrfcCvZ1nuYDrdtmwAUKhFNVkzGGooKCA3bt3W6ingKqye/duCgoK2vU+756H3o4v0bBhUF0N77/vRmA0xnRccXExFRUVJHz3MdOqgoICilu4b3JLvBvo7Wyhg+t2sUA3Jjlyc3MZOHBgpsswUbzX5ZJAoA8d6p7twKgxxs8Oi0Dv0gWOO84OjBpj/M17gZ7AQVFw3S7WQjfG+Jn3Aj2Bg6LgDoxu2gTtuJmJMcZ4incDPYEWel0dZNFIl8YYk1RxBbqITBaRTSKyRUQOuUeTiMwSkUoRKQ8/fpj8UsMSDPRhw9yzdbsYY/yqzdMWRSQAzAdOByqAVSKyRFU3NFv1KVWdnYIam0ow0IcMcW+1A6PGGL+Kp4U+DtiiqltVtQZ4Ejg7tWW1IsFALyiAwYOthW6M8a94Ar0P8HHUfEV4WXPfFZG1IrJIROK/cV97JXiWC7h+dGuhG2P8KlkHRf8MDFDVEcCLwKOxVhKRy0SkTETKEr5cOMGzXMD1o2/ZAgcPJrZrY4zJZvEE+nYgusVdHF7WQFV3q2p1ePYhoOmtuBvXe1BVS1S1pKioKJF6E+5yARfo9fWwcWNiuzbGmGwWT6CvAgaLyEARyQNmAEuiVxCRY6JmpwHvJq/EZjoQ6JExXawf3RjjR22e5aKqdSIyG3gBCAALVHW9iMwDylR1CTBHRKYBdcDnwKyUVdyBQB88GHJzLdCNMf4U12iLqloKlDZbdlPU9PXA9cktrQUdOCiam+tOX7QDo8YYPzpsrhSNGDbMWujGGH/ybqAneJeUE0+Ebdtg//7klWSMMdnAu4HegRY6uFvSGWOMnxy2gW796MYYvznsAn3gQDcMgPWjG2P8xnuB3oGzXAACAXdLOgt0Y4zfeC/QO3hQFGxMF2OMP3k30BNsoYPrR9++Hb78Mkk1GWNMFjgsAz0yBIC10o0xfnJYBrrdvcgY40eHZaD36wddu1oL3RjjL94L9A6e5RLZxIknWgvdGOMv3gv0JJzlAjamizHGf7wb6B1ooYNroVdWwq5dSajJGGOywGEb6KNHu+dVqzpYjzHGZAnvBnoo1KHNjBnjNrViRRJqMsaYLOC9QA8E3HMHA71rV9ePboFujPGLwzbQAcaNg5UrO9x7Y4wxWcF7gS7iHkkI9JNPdpf/b96chLqMMSbDvBfoAMFg0gId4K23OrwpY4zJuLgCXUQmi8gmEdkiInNbWe+7IqIiUpK8EmMIBKCursObGToUevSA115LQk3GGJNhbQa6iASA+cAUYCgwU0SGxlivELgaSP1hxkAgKS30QABOOw3++tck1GSMMRkWTwt9HLBFVbeqag3wJHB2jPV+CdwFHExifbElKdAB/uEfYMsWN5yuMcZ4WTyB3gf4OGq+IrysgYiMBvqq6nNJrK1lSepDB5g40T1bK90Y43UdPigqIjnAb4CfxrHuZSJSJiJllZWVie80SX3oACNHQvfu8MorSdmcMcZkTDyBvh3oGzVfHF4WUQgMA14RkW3AeGBJrAOjqvqgqpaoaklRUVHiVSexyyUQgH/8R1i6tMPjfRljTEbFE+irgMEiMlBE8oAZwJLIi6q6R1V7q+oAVR0AvAVMU9WylFQMSe1yAZg6FT78EDZsSNomjTEm7doMdFWtA2YDLwDvAn9U1fUiMk9EpqW6wJiS2EIHmDLFPZeWJm2TxhiTdsF4VlLVUqC02bKbWlh3YsfLakMS+9ABiothxAh47jm47rqkbdYYY9LKm1eKJrmFDvBP/wSvvw67dyd1s8YYkzbeDPQk96EDTJ/uNvn000ndrDHGpI03Az0FLfSTToKvfQ2eeiqpmzXGmLSxQA8TgfPPh+XL4dNPk7ppY4xJC+8GehIPikbMnOnGRn/88aRv2hhjUs6bgZ6CPnRwN44ePx4eeMAuMjLGeI83Az0FXS4Rl18OmzbZkLrGGO+xQG/me99zY7s88EBKNm+MMSnjzUAPBlPShw7QuTNcfDEsWgQ7dqRkF8YYkxLeDPQUttABrr7a/b64996U7cIYY5LOAj2GQYNc18vvf+9uIm2MMV5ggd6CuXNh3z6YPz+luzHGmKTxZqCnsA89YuRIN77LPffA55+ndFfGGJMU3gz0NLTQAW6/HfbsgTvuSPmujDGmwyzQWzFiBFxyCfzud+4GGMYYk80s0Nswbx7k5MC//VtadmeMMQnzZqCnoQ89om9f+Jd/gf/5H3j11bTs0hhjEuLNQE9jCx3gxhthwAA3LEB1ddp2a4wx7WKBHocuXeD++2HjRrj77rTt1hhj2sWbgZ6i0RZbM2UKnHce3HYbrF+f1l0bY0xc4gp0EZksIptEZIuIzI3x+hUi8o6IlIvI6yIyNPmlRgkGoaYmpbuI5T//E7p1gwsvtK4XY0z2aTPQRSQAzAemAEOBmTEC+3FVHa6qo4BfA79JeqVhD69+mGFFCzlYW5WqXbToqKNgwQJYswZ+8Yu0794YY1oVTwt9HLBFVbeqag3wJHB29AqqujdqtguQsttDfHHwC9YHdlNdcyBVu2jVmWfCFVe4K0iffz4jJRhjTEzxBHof4OOo+YrwsiZE5Cci8j6uhT4nOeUdKj+QD0C11ENtbap206p//3c3NMDMmbB5c0ZKMMaYQyTtoKiqzlfVQcDPgZiX4YjIZSJSJiJllZWVCe0nP+gC/WAQOJCZVnrnzrB4sevKP+ccN4iXMcZkWjyBvh3oGzVfHF7WkieBc2K9oKoPqmqJqpYUFRXFX2WUhhZ6AKhKfz96xIAB8NRT7nZ13/mOHSQ1xmRePIG+ChgsIgNFJA+YASyJXkFEBkfN/hOQso6ISAu9OoMt9IhJk9xB0mXL4IIL0nbxqjHGxBRsawVVrROR2cALQABYoKrrRWQeUKaqS4DZIvItoBb4ArgkVQVnSws94uKL4Ysv4Jpr4KKL4LHHIC8v01UZYw5HbQY6gKqWAqXNlt0UNX11kutqUZMWehYEOrhb1tXWwnXXubHT//d/obAw01UZYw43nrtStCBYAIRb6Bnucol27bXwyCPw8stwyimub90YY9LJc4He0OWSRS30iEsugdJS2LkTxoxx3S+asjPyjTGmKe8FejCqDz2LWugR3/42rF4No0a5gD/jDNiyJdNVGWMOB94L9OgW+hdfZLaYFhQXw1//CvfdBytWwNCh8OMfQ0VFpiszxviZ9wI9uoX+6aeZLaYVgQD85CduyN0f/hD++7/huONgxgwX9tYVY4xJNu8FeqSF3jU/qwM94phj3Fjqmze7gH/hBZg4Eb72NfjZz+Ctt6C+PtNVGmP8wHuBHmmhdy/0RKBH9O8Pv/0tbN8ODz8Mxx8P994LEya4URy/+113M+rVqzMyMrAxxgfiOg89mzS00Ht0hQ+2ZbaYBHTuDLNmuceXX8Kzz8JLL7lumKefduvk5sLXvw4jRsCwYa6rZsAAGDgQevUCkQx+AGNM1vJeoEda6EOHwBMvwK9/DYMGwbHHwsknQ06zPzpUD03A+fNh5UrXF9KlS+PyDz5wd4UOpufH0qOHu7r0oovc/EcfwRtvwNq1bsz15cvdzamjde3qDroeeWTjo6jIPbp3dzfgKCx0j+jpzp0P/dEYY/zFc4EekAAA9RPGw1Hl8POfN13hiCPc2S/jxrnQjujfH3r2dMMj/uEPsGMHvPmmS7pTT4WtW10H949/7JrIe/e6k8mXLXPX9x91lPul8d57rvlcXu7uHj1njkvTM890R0Krq91rI0a4wV3acclov37uMXNm47I9e2DbNvf44AP3+OQT2LXL3Qpv+XLYvTu+7QeDUFAA+fmNj+j5ggI3bEEg4B7BYON0PPORZSKNj5ycpvMtLWvv8njXhY49J2Mb2bxPP3++ZC/zwl/Hohk63aKkpETLysra/b5QfYjgL4PMmziPX5z4YxeoPXu6lOvc2R1h3LDBJd6OHY1vPPpod8UPuP/1c+a4kbX27o29o0RE7wNck3nGDFfbl1+6xIzcISOJA77U1blQ37vXPfbta3yOTFdVwcGD7vdN5BFrvqbG3a418qira/+8atOHMX5QVeX+C2eaiLytqiWxXvNcCz1HXL9BSEPQuzc8/njrb/joIxf0vXu7YN2xwzWDi4rg1ltd07euzjVRa2rcaSfjx7t0+tOf4Fe/cv+KV13l3te/vxs3d+pU11yO/BWwd6/r//jgA1i1yi3bs8fVFz1g+osvusFf7rgD5h5ye9aEBIPuD4ijjkrK5pKuecCrut+7HV3e1rqRfSf6nIxtZPM+/fz5UrEsTT2xHeKBEpsSEQShXuM8169fv8bpY491j4hu3VzXSLSTTmqcLilxoR/5Wz7irLPaX/iyZe6S0R074MEH4YYb3OWkZ5yR/X/HdVD0n9nGmNTx5GGyHMkhVB9Kz84CgeQcTZw0CS6/HG65xY3c1acPTJniBnwxxpgk8GSgB3IC8bfQs1FhISwJ3yPEAt0YkySeDPQcyfF2oIPr2rntNjfe7po1ma7GGOMDng30kKapyyWVrrjCdecsXpzpSowxPuDJQA+Ix7tcInr1cufLP/98pisxxviAJwPdF10uEVOmuNMcKyszXYkxxuM8G+hpO8sl1U4/3Z3s+uqrma7EGONxcQW6iEwWkU0iskVEDrkaRkT+VUQ2iMhaEVkmIv2TX2ojz5/lEq2kxF34ZIFujOmgNgNdRALAfGAKMBSYKSJDm622GihR1RHAIuDXyS40mm8OioIbN2bkSDf+izHGdEA8LfRxwBZV3aqqNcCTwNnRK6jqclWN3ODzLaA4uWU25ZuDohEjRrhTF23gE2NMB8QT6H2Aj6PmK8LLWvIDIOZpGyJymYiUiUhZZQcOAvrqoCi42xft2dN0zBdjjGmnpB4UFZGLgBLg7livq+qDqlqiqiVFRUUJ78dXXS7gRmmEpiM1GmNMO8UT6NuBvlHzxeFlTYjIt4AbgWmqWp2c8mLz1UFRsEA3xiRFPIG+ChgsIgNFJA+YASyJXkFETgIewIX5ruSX2ZTvulws0I0xSdBmoKtqHTAbeAF4F/ijqq4XkXkiMi282t1AV2ChiJSLyJIWNpcUvjoPHSzQjTFJEdd46KpaCpQ2W3ZT1PS3klxXq3x3lkvPnu70RQt0Y0wHePdKUT8dFM3JcbcbskA3xnSAZwPdVy10OPR+pMYY006eDHTfneUCLtCjb2ptjDHt5MlA991BUXBdLrtSfoKQMcbHPBnovjsoClBU5IbQtcv/jTEJ8mSg+7IPvagIamvdEADGGJMAzwa6r85yARfoYDe6MMYkzJOB7suDohboxpgO8mSg+/KgqAW6MaaDPBvo1kI3xpimPBnovj3LBSzQjTEJ82Sg+/KgaKdO0KWLBboxJmGeDHRfHhQFOPJIC3RjTMI8Gei+7EMH1+1iV4saYxLk2UD33Vku0Hi1qDHGJMCTge7Lg6JggW6M6RBPBrovD4qCjedijOkQzwa6b1vo1dWwf3+mKzHGeJAnA93XZ7mAHRg1xiTEk4Hu24Oiffq45+3bM1uHMcaT4gp0EZksIptEZIuIzI3x+mki8ncRqROR6ckvsynfHhTt1889f/RRZuswxnhSm4EuIgFgPjAFGArMFJGhzVb7CJgFPJ7sAmPxbR96377u2QLdGJOAYBzrjAO2qOpWABF5Ejgb2BBZQVW3hV9LS8r69iyXTp3cgVELdGNMAuLpcukDfBw1XxFe1m4icpmIlIlIWWUHzrf27UFRcN0uFujGmASk9aCoqj6oqiWqWlIUGV0wATn49KAoWKAbYxIWT6BvB/pGzReHl2WMb7tcwAX6hx/axUXGmHaLJ9BXAYNFZKCI5AEzgCWpLat1eYE8akO1mSwhdQYNchcW7dyZ6UqMMR7TZqCrah0wG3gBeBf4o6quF5F5IjINQETGikgFcB7wgIisT2XR+cF8qkPVqdxF5gwb5p7XrctsHcYYz4nnLBdUtRQobbbspqjpVbiumLTID+RTXefzQH/nHTj99MzWYozxFE9eKZofzKe2vtafZ7oUFcFRR7lAN8aYdvBmoAfyAagJ1WS4khQZNswC3RjTbp4M9IJgAQAH6w5muJIUGTMG1q6FqqpMV2KM8RBPBnp+0LXQfduP/o1vQG0trFyZ6UqMMR7izUAPd7n49kyXU091z6+9ltk6jDGe4s1A93sLvWdP14++fHmmKzHGeIg3A93vLXSAs86Cv/4VPvss05UYYzzCm4Hu9xY6wPe+B6EQPP10pisxxniEJwPd92e5AIwcCSecAAsWZLoSY4xHeDLQD4suFxG48kpYscLOdjHGxMWbgX44dLkAXHIJHHEE3HCDjb5ojGmTNwM93EL3dZcLQLdu8MtfwrJl1pdujGmTJwP96K5HE5AAd75xJ0+/+zQ79+/07w0vLr/c9adfeSVsz+gw9MaYLCeaoT/lS0pKtKysLOH3P1L+CD9/6efs+moXAILQs1NPOud2RlG65Xeja15XVLXhZhjBnNj5+uoAAAoPSURBVCCCNGwjpCFUlWBOkEBOgIAECOQEyJEc9lbvRVUbunfqtf6Q9ytK5OcXmW5tmRJeroqI0Dm3MwEJuPpFEARFCdWHCGmIeq1326qqgnXrqO/ciX3H9aFLfiGdczuTIzlUh6o5UHuAgATomteV/TX7KQgWoCj1Wk+oPkRuIJeueV35quarhuMOglBbX0t+IJ+CYAG5gVwO1h0kIAH21+xH0YbPGqkt8ryneg8FwQK653entr6W6rpqAjkB8gP55EgOilIbqm34uUdvJ3o+1rKW5iPLqkPVTcbDzwvkkRvIZW/1XoI5wYYD5jWhGurq62J+d1r6zkf+fVK1fkhD5EgOdfV1ru4c9zPvlt+Nuvo66rWe6lA1ofoQBcGChu+yqlJXX0dIQ+QF8ho+V47kkCPtb5O19/98S5+zNc/OfJYjOh3R7veZtonI26paEvM1rwY6QG2olrcq3mLNp2uo/KqSygOVVNW58U/2Ve9jf81+RKQhNGvraxGEqroqAhIgP5jf8B8sEqKR58K8QkSEmlANgjSsFxEJvEjIwaHBF2tZJKDqtZ6q2ioX2lG/BIAmv1zAhVPwsy8IrFlLYUF3DkwYw4EcF/iC0Cm3E8GcIPuq99EptxN19XUN/9lzJIc9B/dQV19H17yuDYEHLhAO1h2ktr6WA7UH6JLbhXqtp2te14agiPXLqCBYQG19Lftr9pObk0teII+QhqgNNY6AmRvIbfi5R7YTeX9Ly1qaj16WG8ilJlRDbk4uIkJtqJbqUDWFeYUoSlWt+/fPD+YTkECTXwjRon+pNFmewvVFhKraKvICedRrPXX1dQ3fscjnyc3JbfjleqD2QMN3puHfI9wAERFC9aGGf5+W6mhJS5+nxfXbuf2F5y2kR0GPdr3HxKe1QI9rPPRslRvI5Zv9v8k3+38z06WkR2kpTJ8OpRvhv/4LpkzJdEXGmCziyT70w9bUqfD669Cjh5s++2x4++1MV2WMyRIW6F4zerQL8dtuc4N3lZS40Rl/9zs7aGrMYc4C3Yvy8+HGG2HbNrjrLti7F66+GoqLYfx4uPZaWLgQPv7Yzl835jDi6YOiJsqmTbBoETz3HPz971AdvuiqqAiGDIHBgxsfxxwDRx7pHt26uatSjTGe0OGzXERkMvAfQAB4SFXvbPZ6PvAYMAbYDZyvqtta26YFegrV1MCaNW7YgPJy2LwZ3nsPdu48dN38/MZwP+oo91xU5PrpCwtd4Hfr1jjdpQt06gQFBY3PBQX2S8GYNOnQWS4iEgDmA6cDFcAqEVmiqhuiVvsB8IWqfk1EZgB3Aed3vHSTkLw8GDvWPaLt2wdbt8Knn8KuXY3PkelPP3X3Mt21q7GFH69IsDcP+/x8yM11j2CwcTrWfGvr5ORAIBDfc3vWbf4sEvvR2mvZ9h5z2IrntMVxwBZV3QogIk8CZwPRgX42cEt4ehFwn4iIZqo/x8RWWOiuOo1HdbX7BbB3r3vs2wd79sCBA+5Cp4MHYz83X1Zd7W6nV1UFdXVuOvJoa96+PolrHvrRy6OfW5pOxeuZ2GemX2/pPTffDOcnv80bT6D3AT6Omq8ATm5pHVWtE5E9QC+gyd0ZROQy4DKAfv36JViySYv8fPfo3TtzNdTXN4Z7fb17hELxPbdn3ehn1ZYf9fWtv56t74mITMdalurXM7HPTL/e0nvADbqXAmm9sEhVHwQeBNeHns59Gw/KyWn8xWKMaVM8py1uB/pGzReHl8VcR0SCQHfcwVFjjDFpEk+grwIGi8hAEckDZgBLmq2zBLgkPD0deNn6z40xJr3a7HIJ94nPBl7Anba4QFXXi8g8oExVlwD/DfxfEdkCfI4LfWOMMWkUVx+6qpYCpc2W3RQ1fRA4L7mlGWOMaQ+79N8YY3zCAt0YY3zCAt0YY3zCAt0YY3wiY6Mtikgl8GGCb+9Ns6tQs5jVmhpWa+p4qd7Dsdb+qloU64WMBXpHiEhZS6ONZRurNTWs1tTxUr1Wa1PW5WKMMT5hgW6MMT7h1UB/MNMFtIPVmhpWa+p4qV6rNYon+9CNMcYcyqstdGOMMc1YoBtjjE94LtBFZLKIbBKRLSIyNwvqWSAiu0RkXdSyniLyoohsDj8fEV4uIvK7cO1rRWR0mmvtKyLLRWSDiKwXkauztV4RKRCRlSKyJlzrreHlA0VkRbimp8JDOiMi+eH5LeHXB6Sr1qiaAyKyWkSezeZaRWSbiLwjIuUiUhZelnXfgfD+e4jIIhHZKCLvisiEbKxVRIaEf56Rx14RuSbttaqqZx644XvfB44D8oA1wNAM13QaMBpYF7Xs18Dc8PRc4K7w9FTgeUCA8cCKNNd6DDA6PF0IvAcMzcZ6w/vsGp7OBVaEa/gjMCO8/A/Aj8PTVwJ/CE/PAJ7KwHfhX4HHgWfD81lZK7AN6N1sWdZ9B8L7fxT4YXg6D+iRrbVG1RwAdgL9011r2j9sB39QE4AXouavB67PgroGNAv0TcAx4eljgE3h6QeAmbHWy1DdzwCnZ3u9QGfg77h72X4GBJt/H3Dj9U8ITwfD60kaaywGlgH/B3g2/B81W2uNFehZ9x3A3fnsg+Y/m2ystVl93wbeyEStXutyiXXD6j4ZqqU1R6nqjvD0TuCo8HTW1B/+M/8kXMs3K+sNd2GUA7uAF3F/nX2pqnUx6mlyo3IgcqPydLkX+BlQH57vRfbWqsBSEXlb3I3bITu/AwOBSuDhcFfWQyLSJUtrjTYDeCI8ndZavRbonqPu129WnRsqIl2B/wWuUdW90a9lU72qGlLVUbjW7zjghAyXFJOInAnsUtW3M11LnL6hqqOBKcBPROS06Bez6DsQxHVn/l5VTwK+wnVbNMiiWgEIHyeZBixs/lo6avVaoMdzw+ps8KmIHAMQft4VXp7x+kUkFxfm/09Vnw4vztp6AVT1S2A5rtuih7gbkTevJ5M3Kj8VmCYi24Ancd0u/5GltaKq28PPu4DFuF+W2fgdqAAqVHVFeH4RLuCzsdaIKcDfVfXT8Hxaa/VaoMdzw+psEH3T7EtwfdWR5ReHj3CPB/ZE/TmWciIiuPu/vquqv8nmekWkSER6hKc74fr638UF+/QWas3IjcpV9XpVLVbVAbjv5MuqemE21ioiXUSkMDKN6+9dRxZ+B1R1J/CxiAwJL5oEbMjGWqPMpLG7JVJT+mpN9wGDJBxwmIo7O+N94MYsqOcJYAdQi2tR/ADXH7oM2Ay8BPQMryvA/HDt7wAlaa71G7g/+dYC5eHH1GysFxgBrA7Xug64Kbz8OGAlsAX3Z21+eHlBeH5L+PXjMvR9mEjjWS5ZV2u4pjXhx/rI/6Fs/A6E9z8KKAt/D/4EHJHFtXbB/aXVPWpZWmu1S/+NMcYnvNblYowxpgUW6MYY4xMW6MYY4xMW6MYY4xMW6MYY4xMW6MYY4xMW6MYY4xP/H8OR59VqXzLpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.neural_network import MLPClassifier\n",
    "clf1 = MLPClassifier(activation='logistic',solver='sgd', learning_rate_init = 0.1, learning_rate = 'constant', batch_size=1500, hidden_layer_sizes=(5,), random_state=1,n_iter_no_change = 500, max_iter=2000)\n",
    "clf1.fit(X, y)\n",
    "print (\"1 ok\")\n",
    "clf2 = MLPClassifier(activation='logistic',solver='sgd', learning_rate_init = 0.1, learning_rate = 'constant', batch_size=50, hidden_layer_sizes=(5,), random_state=1,n_iter_no_change =500, max_iter=2000)\n",
    "clf2.fit(X, y)\n",
    "print (\"2 ok\")\n",
    "clf3 = MLPClassifier(activation='logistic',solver='sgd', learning_rate_init = 0.1, learning_rate = 'constant', batch_size=1, hidden_layer_sizes=(5,), random_state=1,n_iter_no_change =500, max_iter=2000)\n",
    "clf3.fit(X, y)\n",
    "print (\"3 ok\")\n",
    "plt.plot(clf1.loss_curve_,color='b',label='Batch')\n",
    "plt.plot(clf2.loss_curve_,color='r',label='Mini-Batch')\n",
    "plt.plot(clf3.loss_curve_,color='g',label='On-line')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 ok\n",
      "2 ok\n",
      "3 ok\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x7f6cccbd37d0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8li6FKAAAgAElEQVR4nO3deXwURdrA8d+Ti0C4E5CbBOSQ5YgY8QAVBARdBUFfBRUFUVYU8dhVQVdRdL1vUdSXRdB3AcUTEZV78USCooLciBpAgXDGhJCj3j96ejKTOTKTTJikeb6fz3xmpvqY6kzm6eqq6ioxxqCUUsq5YqKdAaWUUpVLA71SSjmcBnqllHI4DfRKKeVwGuiVUsrh4qKdAX9SUlJMampqtLOhlFLVxurVq/caYxr5W1YlA31qaiqZmZnRzoZSSlUbIvJLoGVadaOUUg6ngV4ppRxOA71SSjlclayjV0pVXwUFBWRlZXHkyJFoZ8WREhMTadGiBfHx8SFvo4FeKRVRWVlZ1KlTh9TUVEQk2tlxFGMM2dnZZGVlkZaWFvJ2WnWjlIqoI0eOkJycrEG+EogIycnJYV8taaBXSkWcBvnKU56/rQZ6l4KiAqZ/N51iUxztrCilVERpoHd58ssnGT1vNDPXzIx2VqqF/Xn72XFoR7SzoZRfsbGxpKen061bN7p3786XX34ZdP0DBw7w0ksvlbnf3r17V8ubOR0Z6AuLC9mftz+sbfbm7gUgOy+73J97/bzrWbJtift9fmE+uQW55d5fpPxy4BciPcFMm+fb0OKZFhHdZ3nsy9tHztGcaGdDVTE1a9ZkzZo1fP/99zzyyCNMnDgx6PqhBvrqypGBfvS80TR8vGFY1TB2vVdFAuK076bR741+7vcdpnQg6eGkcu8vEr7d9S2pz6Xy0qrI/hMfOHIgovsrr+THk2n7fNtoZ0NVYYcOHaJBgwYA5OTk0LdvX7p3706XLl344IMPAJgwYQJbt24lPT2dO+64A4DHHnuMLl260K1bNyZMmODe39y5c+nRowft27fns88+O/YHVA6O7F75+vevA1bJPiE2IaRtYsQ65xnKF+j9nVR+ORhw6IljZsPeDQB88dsX3NTjpijnpnLs/nN3tLNQ6bbs20KLui1IjEuMdlbCcuutsGZNZPeZng7PPht8nby8PNLT0zly5Ai7du1i6dKlgNUH/b333qNu3brs3buX008/nUGDBvHoo4+ydu1a1rgy+/HHH/PBBx+wcuVKatWqxb59+9z7Liws5JtvvmHBggU88MADLF68OLIHWAkcWaK3FRYXhryuYJXoy9sYG85nRdr7G94nvzDf77Ki4iKg5ERmW7JtCb8d/K3S86YqLrcgl3YvtGPk+yOjnZVqw6662bBhA5988glXX301xhiMMdx999107dqVfv36sWPHDv744w+f7RcvXsyoUaOoVasWAA0bNnQvGzp0KACnnHIK27dvPybHU1GOLNELgsFQUFQAId48VtGqm2gF+uXblzPkzSHcfvrtPDXgKZ/lRcZ/oO/3Rj/q1ajHgQlVowomXP/d/l9OanRShfaRdSiLls+05N3L3mXISUMilLPIO1Jo9ZleuHVhlHMSvrJK3sfCGWecwd69e9mzZw8LFixgz549rF69mvj4eFJTU8Puk16jRg3AavAtLIxeAS8cZZboRWS6iOwWkbUBlt8hImtcj7UiUiQiDV3LtovIj65lx6yp2g7aBcUFIW9jB8LylujtkvOxlp1rNR7/fOBnv8vt44mNiXWn2YHjYP7BgPstNsVc+e6VfPHrF0E/P9KNvKHqPbM3Z712VoX28d2u7wCYvmZ6JLLkY8m2Jfx59M8K78f+DrVvevls2LCBoqIikpOTOXjwII0bNyY+Pp5ly5bxyy9W9WqdOnU4fPiwe5v+/fvz2muvkZtrdabwrLqpjkKpupkBDAy00BjzhDEm3RiTDkwE/muM8fyr9HEtz6hYVkNnV8MUFB27QO9Zor9h/g08sPyBcu0nkEVbF/Hc188FXB6obcE+AcVKSaBPfjy5zM87cOQAs36cxYWzL/RK/3Djh5z92tnu9/lF/quMKkNRcRF3L7nb3a1zU/amY/bZ4dq2fxv93ujH9R9eX+F92f/H+/L2hfU/fTyz6+jT09O5/PLLmTlzJrGxsVx55ZVkZmbSpUsXXn/9dTp27AhAcnIyPXv2pHPnztxxxx0MHDiQQYMGkZGRQXp6Ok8++WSUj6hiyqy6McasEJHUEPc3HJhdkQxFUukSvTGGnw/8TJsGbXzWtU8O5W2M9Qz0r6x+pcz1l/68lLW71zL+tPEh7f+8/zsPgFtOvyWsfLlL9K5A/9kvn4XU5dMOKPExJXVfU1dN5cYFN3qtl1+Yf8waCD/79TMe+fwRvs76utz72H5gO63rtfYqHVfGVYn9N/7+j+8rvC/P/+O7Ft/F0wOervA+na6oyP8VdkpKCl999ZXfZbNmzfJ6P2HCBK/eNgDLly/32ld1qaOPWGOsiNTCKvm/45FsgIUislpExpSx/RgRyRSRzD179lQ0L4BvvfmTXz5J2+fbsna3by2UvxJ9xykdeTnzZfd7Ywz3LbuPzdmbfba368JD1ff1vtzySXhB25+yLudL19GX7gm0ZNsSvvrN9x/frt6Jjy0J9M+u9K1wPVJ4hGJTfEyqcOzP2H/E9x6J738vO6D+8McPpD2XxvMrn4943kqzT5AFRQUYY7hr0V3uqqJwHS066n79xW/Bq9KU8ieSvW4uAr4oVW3TyxjTHTgfuElEzva/KRhjXjXGZBhjMho18jvtYcjs0vnwd4ZzOL+k3m3Z9mUA/HrwV99tSjXGGmPYmL2RsR+NZfQHowHYeXgnD654kPP/c77P9tFqjLWP1dPnv37urtawq25eXv0yX2d97VM11e+Nfpw5/UyffeQV5gEQF1Ny0eevHSK/KJ/YybEMnjO43MfQYUoH7lp0l0/63ty97PnTOulPXTWVu5febeWtIM9n3fRX0ssM9j/88QMAX++wrgjc33k5r+JsuQW57rYSm90mUlBcwKH8Qzz+5eP0f6N/ufbvWV0TrTYRVb1FMtAPo1S1jTFmh+t5N/Ae0COCn+ej2BQz6oNR7kvdb3Z8w5j5Y7h4zsUs376cLfu2AL49UDzT7B+9ZynKbqyzl9lB0FM4gd6zgTNSjbjGGL749QsWbl3IWa+dRYcpHQDvK5SPN38c8ufZVQ+eVTf+rlrskv+Hmz4Mur/Pf/2cOo/UYV+eb6PWpuxNPP7l4z7pjZ5oxEkvWj1rblxwo7vKxv7M0nq91itoHg7lHwKgbkLdoOvZfvzjR+5adFeZwbX7K91JeSLFK83+uxcWF7rv3PX3fxcKz6obHYtJlUdEAr2I1APOAT7wSEsSkTr2a+A8wG/PnUjZdXgXM9bM8Eqbs3YOH2z8gD4z+7B5n1Xl4tkw6c6vq2S8L28fCzYv8NvIaP/gPX9s2bnZJD6UyOJtwW+aeOend7hjoXXHnWdA+rOg4r0ywDoJ9XqtFwP+b4BXumdwnrxiMr/n/B7S/uxSs2fVjb+ThL/StT/3L7+fnKM5ZO4s6XxljPH6Wz6w/AHuX34/gPtKzN+QFP5OtAA5R3OY/l3gHjTuQF/Df6AvLC50D4UBcMGsC3j8y8eZmjnVK58z18z0unrYmL3R777AKo3bvZsS4xI5eOQgI98fyc7DO93dJYuKi2j/Qnuun3c9B48c5Lp517nzau/DVtGrD3V8KrMxVkRmA72BFBHJAibh6p1ujLErsIcAC40xnlHrBOA91+VxHDDLGPNJ5LLuK1h3QU+eXQ1t9mX8i6te5MVVL/L9Dd7VALX+VYt6ifUA+D3ndy6cdSEj00dSO6E2+UX5PLjiwYCfZ4zh0rmXAnBO6jley3KO5gQMPPvz9vPO+ncYffLogPsuq23g5/3e3S7LKnnb7GD6e87v/J7zO01qN/F71RJqzxf7BFMzrqY77emvnuYfi/7hfn//f++3nnvf73Wite90ductyMll9LzRjEofhYiwOXszreu3JiE2gXW717F8+3IAnvzqSWon1KZ70+5e247/eDxTM6eSe3cuNeNrugsENy24iaLiIm4+7WYARn4wEgAzKXDQtU+KBcUFHDxSEuif+uopZn4/k5nfW4Pnrb9pPc3rNGfzvs1s3reZZnWa8e/v/k1q/VT+efY/Ae+rS626UeURSq+b4SGsMwOrG6Zn2jagW3kzVh6epbFg/JXoS19Wlx7LJa8wj7yckgDz0eaPqJ1Qm6u7XQ0Ev6T2DMYXzb7Ia5l9WZ9fmM/RoqPUqVHHvey6D6/j3fXvckrTUwLu2w4C/j5/x6EdTFk1xe/nBXO06Ch/5Fh3Cx44coCmTzVl47iNfk8ql719WZn7A1i3Zx2A15AU//vt/wZcf8Likt4O17x/jdeyw0cPl17dS8zkGOZcModh7wzj2vRrGXjiQJ983v/f+3nnMqvfgB087RNKflE+NeNrUiu+lnv93w5ZdxGv+b3kfv7C4kKvNgxP9t+qoKjAXTqvGV/Tp0Cw49AOUmqVVPvY/4eedzpr1Y2qKEcNgRBqoJ/14yw6TOngVToqHegD1QN7enPdm9y77F4geF17sPr7Q/mHOFp0lMR/JVL3Ue+S/a7Du3zysnjbYvdNOJOWTWL0PKu0X7oxEKw7P0sLpapo0OxBXPXeVV5pHaZ0KLMdYvzH4316luw6vIuPNn3kfm/v40jhEb9VHrZ/f/fvMvMZzKy1Vle56WumBzwZrdqxyuu9fRVRWFzIy5kvs+NwyTDMxaaYgqICTp92ujtt7PyxXtvnFuQydv5Ydh7e6VWit6uf7MZgT3/8+YdXULfbMOzgfvDIQUZ9MMq9XKtuQiMiXHVVyf9wYWEhjRo14sILrftC5s2bx6OPPhp0Hzt37uTSSy/1u2zkyJGkpaWRnp5Ox44deeCBsu+bmTFjBjt37ixznXHjxpW5r3Adl4H+1W9fZVP2JneDY1FxkU91QKhD336761sgeDAPtiw7N5v//PAf9/vN2Zt9grZnr5j+b/RnxHsjAKvO3T4J+Avq/uqy7WqE0jyrBz7d+qnfdcoK9C988wLdX+3O3HVz3Wl9ZvbxuunKDmA1/1XTZ3tbJEqtnscTyKNfWD/09XvXAyXH9/6G9xn70VivevLD+Ydp8FgDryqlad9NY9aPJX2vkx5O4uXVL/PCyhfcJfrC4sKgYwodOHLA60T+/DdW10+7Xn5q5lS27d/mXq5VN6FJSkpi7dq15OVZv4FFixbRvHlz9/JBgwb59JEvrVmzZrz99tsBlz/xxBOsWbOGNWvWMHPmTH7+2f/d6bZQAn1lcVSg99ejI5jsvGxu//R24h6MY/KKyV7Lbph/Q1j72pMbuO9/sLsZ9+bupUZcDff79lPak/JECm+ufZOvsvzf2OEv3a5a8OTvqiTQ36jTi53IL8zn3JnnBsyrZ+AL5rK3LyOvII86j9TxKbUfLTpaZrDyd59CuMIZemD7ge1eJxd/d7O++u2rfq+Grnz3Sp+0ujXquk8ahcWFfrvz2o4UHuHEF070SbdPVHUS6nila9VN6C644AI++si6mpw9ezbDh5fUQnuWnEeOHMn48eM588wzadOmjTu4b9++nc6dO5f5OfZYOUlJ1pDkkydP5tRTT6Vz586MGTMGYwxvv/02mZmZXHnllaSnp5OXl8eqVas488wz6datGz169HAPwbBz504GDhxIu3btuPPOOyPyt3DUoGbhBvrN2Zt55utn/C4LFrjD9dzKwEMX7M3dS9uGvuOpD3tnWMBtQr3asIco9hTo0n/r/q08uOJB970GFfXu+nf95vPyty/n81GfB92244sdy/WZ5594Ph9v+Riw7qINRySHOj5w5IBXVV6w/6VAV6FTVk3hnrPv4Y0f3vBKr3aBPlrjFAPDhg1j8uTJXHjhhfzwww9ce+21AceP37VrF59//jkbNmxg0KBBAatsPN1xxx089NBDbNmyhfHjx9O4cWMAxo0bx3333QfAiBEjmD9/PpdeeilTpkzhySefJCMjg6NHj3L55Zfz5ptvcuqpp3Lo0CFq1rSuctesWcN3331HjRo16NChAzfffDMtW7YM9a/jl6NK9P7qqYMJNBBYpD3w38D1d9l52WGPXxJqoL/t09vC2u+/PvtXWOsHU7qO33bgyAG/peBI8GzIDlfpbrkVsTt3t1fDdbArIbvLrz/Pr3yelTtWeqWt37u+So/xU5V07dqV7du3M3v2bC644IKg61588cXExMTQqVMnv8MW+2NX3fz+++8sWbLEPV3hsmXLOO200+jSpQtLly5l3bp1Pttu3LiRpk2bcuqppwJQt25d4uKscnffvn2pV68eiYmJdOrUyT3wWkU4q0R/JLwS/ZRvppS5TqzEhj28QTgeWvEQN50a/oQg9y69txJyE5o2Ddp41RuHK1hVRkV4dt0MV0WOp7QZa2Z43bEcrJfQxr2BG6Qf+fwRv+kdpnQI2rWzSonyOMWDBg3iH//4B8uXLyc7O3BB0B56GPy3g4waNYrvvvuOZs2asWDBAq9ltWvXpnfv3nz++ed0796dG2+8kczMTFq2bMn9999f7mGQIXJDIR/XJfpQBpwK1H0uUgzGpwtkKB767KGAyy456ZKKZKlME3oGb8Qqi7+xaiLB8y7ecAXr6umpf5vQhjF4bc1r7tfBOgn8uPvHkPanyufaa69l0qRJdOnSpUL7ee2111izZo1PkAerR8/KlStp27atO6inpKSQk5Pj1ZjrORRyhw4d2LVrF6tWWT2/Dh8+XKlj2zsq0HvW0fdr0y/ImqGrSKC/qP1FZa8UQTdmWCNLNq3dtNI+I6VWSsg3ph1rlX1SBrz6vJc29a9T6dzYt/Eu0lUtZ7Wq2Dj8x5MWLVowfnxoI8SG64477iA9PZ2uXbvSpUsXhg4dSv369bn++uvp3LkzAwYMcFfNgNXoe8MNN5Cenk5RURFvvvkmN998M926daN///5hl/zDIVWxu1ZGRobJzAx/npKmTzV134G54/YdNH+6eRlblG3giQP5ZMsnfD7qc6+hC5rWbsrRoqN+b9G3XfaXy3hr3VsVzkOo9t+1n6NFR5n27TTuWXpPwPXsGbjKo0FiA54d+CzXvH8NvVN7u+82jYS2Ddqydf/WsLapk1CHw0cPc1XXq0iKTwppiOgasTXKPY7+jRk38lKm70TrvVr14rNRn3HPknt4+POHy7XvYG4//XamfTeNTo06ERcTx2ejqu6k1OvXr+ekkyo2+5cKzt/fWERWB5r3wzElemMM+/L2cd3J1zHz4pk0q9MsIvudc8kcVl63kp6tenqld27cmb+2/2vQbUtXJcwYPCMieQqkdkJtGic15i+N/hJ0vb13elclBCullvZYv8cY0XUEi0csZunVS0PeblT6qDLXyWiWwZ1nht6dbP7w+ZzewrqB6aL2F4XcqF2R8dwbJzX2m27PcVCRWaDq1agXcNnkPpM5cNcBUmqlhNwYr5TNMYEeYN+d+3h6wNPuYQn+2i54ILY9P7BkfPLSP+R6ifXo0dwadLP4vmIK7y3kvrPv4/Uhr5MYG3zCDc8BwUafPJr6ifVDyo8trX5awHFw/LGrLgZ3HOzOsz8Nazb0en9mS99hiv05s+WZXH/K9YgIfdv0RUR457J3WDxiMan1U4Nu++zAshvl4mPjOaVZ4OEeAK/j+mv7v7q7G8bHxLtvxuqYErx7pucwDOEKFMif7G/NQBRohMoasTX8pnv6cvSXXNrpUsZ09526ISE2AREhKT4pItMTquOLYwK9iJCUkOTVxe7x/iVD33475tuA2zap3cT9+rfbfqPwXv+NIiJCbEwsD/R5gCa1m3Ag3//E2l0aWw0/niX6aYOmufMWqFRYWkFxgc+4PP4Cf3xMPM8M8L4foENyh5A+A3DPEFUnoQ5T/zrVnd4ntY/Xes3r+FaFDT1pKH3b9KVVvVYB9z+i6wifG3/8iY+J9xoOoLQlVy/h/cvf90qzA31cTJz7JqMrOl8BBA7o5W20XX/TencgH9RhEHf1tMbQH9xhMI2SrDkUAgX6O3uWfaXSqVEn5v7PXO49x7dHlX0ST4pPitiIp+r44ZhA70+nRp3cr09uejI/ji3p4XDv2SU/Js9BoxJiE4iNiWXLzVtYeNXCcn1uzXirm1/pxkF7kKy0+mmYSYa5/zPXa/mQjkO83tuTbngqPeIiWNVIt55+q1dai7otAGhdrzUTe00Mml87n1d3u5obMm5wB/TS+T+v7Xll7sOfu8+6GxEp84omPiY+6BhDPZr3cI8gauud2huwTtZ2f/XW9VsD1h3Jy69ZzusXl4x+ufTqpV5XWp4N5i9d4Fv3DlYh4Z3L3qFjSkfOaW2NPjq+x3huyLDunh6ZPtK97t9O+ZvffXh2tyx9Eu7cuLPX1ae/qRntK4mkBC3Rq/A5OtCX5lkX7Xnnor+63bYN29K/bfCudIHGUzmzhVUV0qtVLxZetZDFI6yx6u1Ab5fILu1UcvddvRr1eOhcq8vkv861blyKkRge7lvSsDfu1HE80te3b/XYjLE+aZPOmcTsS2bz8y0/83Dfh/nvyP9yY8aNLLvG985Xu4RrB2s7gNknLFuw4ZI9A/3JTU5myvklXUbtq5Jfb/2VfXd63+uQUiuFe866x72PPmneVxGeEuMSfapAJvSaQNZtWZza/FR3w7hdjdQhpQPnpJ7jru5JrplMn7Q+7t5Z13e/nnnD57n3dX4735nD7j/nfk5uejJDTxoKwFmtzyL37lz6tulLav1UzCTDxR0vdq/ftE7TMtsully9hFcufIXVY1Zzz1n3sOyaZcy/Yr57ebBqnofOfYifbzk2N/op53DUDVP+bBq3yT3uiGc1iOdNUOWdBjBQ49/EsyZy+xm307Ke923LdrfHvml9fbY5MMGqBrJvhGlZtyXdmnSj6wldGfuRFcifGvCU112Wm8Ztol1yO795qBFXg2GdS4ZROLv12Zzd2v9MjnYPHPvvc9vpt7Evbx8Xtr+QeRtLAmGwhkbPQP9/Q/+PTo068fTXT3vdiGRXXZlJhryCPGo9XIu7e93tvqIqKC7gxIYncnHHi3l/g3cVTenPsCXEJtC8rnUFYgfwxkmN+eTKT+jWxBol2z7B2idme2A3u4rNllo/lb137OVfn/3LPTTGpN6TfD6z9AmwtNoJtX3Szkk9B1ZYr5vXbc6YU6x6eH9XaEkJSfRo3oO7et7FK6tfcU9QAoEnTVEqGMeX6Nslt+OkRlY3JM9AYQf3x/o9xhVdrijXvv11UTxyzxEaJzX2CfIAjZIa8cutv/BE/yfK3PeIbiPoekJXr7S4mDiSaya73wcK8uFynwhdE7IkJSTx9ICn6dmyZ7DNvHRvUhKw7DtU7bYPfwG6ZnxNzCTDbWfc5g5edrWNZ31+5vWhd7N9rN9jNE5qTFr9NAacOMD9+UkJ1mBTdpfKW06/hQf7POi+cunVqqTbbHKtZB7s8yBP9n8yYFtNWfwF+nPTAg8WV1qMxLDyupUMPWko84fP5+CEqnnfQlWXlZXF4MGDadeuHW3btuWWW27h6NGyRzW1LV++PKxhjasyxwd6T54zS13X/Trq1qjLZX+5zB10wr2t/KULXuK6k68j/58lDYie9b/+tKrXqsx1AomRGHepOpI3B9n7Kr3PGnE1OP9E3+oMfyb1nsRpzU8DSkqd71z2DlP/OpW0BmlBt7VPNHaVxanNrJtMGtZsWGYvHE9DTxrKH//4w2s0UPAt0ddOqM0/z/6n+3tYevVS/ry7pN47KSGJv5/5d78zkYXCX6Avr/jYeC3Fl4MxhqFDh3LxxRezefNmNm3aRE5ODvfcE/j+kmBCGda4KnN81Y2nOgl1uDb9Wq7qehUdUzpWuKTUun5r/neQ963z5Z0AOhxrx64Nu6tmILOGzgo6l+7c/5lL7UfKDlxxMXF8ce0X7MrZRXIt66qjSe0m7lJzMHbPGfsKZlyPcXRv2t1978KsobNY8csKr23CufvXPoGM7+H/Dsn42Phyn3z9sa8gSptzyZyAy1RkLV26lMTEREaNsu7fiI2N5ZlnniEtLY20tDQWL15Mbm4uW7duZciQITz+uO/k9J5mzJhBZmYmU6ZMYeTIkdStW5fMzEx+//13Hn/8cfdol0888QRvvfUW+fn5DBkyJKQJSY6FUOaMnQ5cCOw2xvjc3y0ivbEmBbdbiN41xkx2LRsIPAfEAtOMMVG99hER/j24YjMXRdqmcZvCLjn+pXHwG6LCMbzLcB5Ybv0z+gt24QSm2JhYd2+fcPztlL8RHxPP9adY48CLiNcNasO7DGd4l5KxxDeO2+hVhVUWEaHgXt+uqpXFLtFf3PFi/sj5w90z6PLOlx+Tz69Kbv3kVq/pFyMhvUl6mfdlrFu3jlNO8b4arFu3Lq1ataKwsLDCQwH7G9Z44cKFbN68mW+++QZjDIMGDWLFihWcfbb/trFjKZQS/QxgCvB6kHU+M8Zc6JkgIrHAi0B/IAtYJSLzjDE/lTOvjhRqPbu/PuyRcuvpt/LLwV98umja3hjyBm0b+I6ZHyk14mow9lTfnkOBtE9uH/ZnHItxcGwJsQlsHLeRlnVbltlwq6LDHgoYcA8FHE6g9zes8cKFC1m4cCEnn3wyADk5OWzevLl6BHpjzAoRSS3HvnsAW1yThCMic4DBgCMDfUazDDJ3hj8+Tyj237W/Qndz+rNoxCK27rPGlamXWI/pg6cHXPeqrv7HlleBledk5ESh3BFdGTp16uQzDeChQ4f49ddfiYuL8zsU8Hvvveeuapk2bVrQ/fsb1tgYw8SJE/nb3/zfSxFNkapQPkNEvheRj0XErldoDnjOb5flSvNLRMaISKaIZO7ZE7nZnY6VFSNXkH1neMMkh6p+Yn13g2Kk9GvTj79lVL1/SKUioW/fvuTm5vL661ZFRFFREX//+98ZOXIktWr5/y0NGTLEPQdsRobfscGCGjBgANOnTycnxxqLaMeOHezeHbmZyyoiEoH+W6C1MaYb8ALg2wE6BMaYV40xGcaYjEaNGkUgW8dWzfiaPmPIKKWiQ0R47733mDt3Lu3ataN9+/YkJiby8MORH1nUdoUMYAYAABY+SURBVN5553HFFVdwxhln0KVLFy699FL3+PPRFtIwxa6qm/n+GmP9rLsdyADaAfcbYwa40icCGGP8T5vjobzDFCulok+HKa58x3yYYhFpIq7O3SLSw7XPbGAV0E5E0kQkARgGzAu8J6WUUpUhlO6Vs4HeQIqIZAGTgHgAY8zLwKXAWBEpBPKAYca6TCgUkXHAp1jdK6cbY3xnyVVKKVWpQul1M7yM5VOwul/6W7YA8J1kUSnlaMaYCk3CogIrz6yAx9UQCEqpypeYmEh2dna5ApIKzhhDdnY2iYnBJz0q7bgaAkEpVflatGhBVlYW1bGbdHWQmJhIixbh3YGugV4pFVHx8fGkpQUfyE4dW1p1o5RSDqeBXimlHE4DvVJKOZwGeqWUcjgN9Eop5XAa6JVSyuE00CullMNpoFdKKYfTQK+UUg6ngV4ppRxOA71SSjmcBnqllHI4DfRKKeVwGuiVUsrhNNArpZTDlRnoRWS6iOwWkbUBll8pIj+IyI8i8qWIdPNYtt2VvkZEMiOZcaWUUqEJpUQ/AxgYZPnPwDnGmC7Ag8CrpZb3McakG2MyypdFpZRSFRHK5OArRCQ1yPIvPd5+DYQ3x5VSSqlKFek6+tHAxx7vDbBQRFaLyJhgG4rIGBHJFJFMnWtSKaUiJ2JzxopIH6xA38sjuZcxZoeINAYWicgGY8wKf9sbY17FVe2TkZGh08crpVSERKRELyJdgWnAYGNMtp1ujNnhet4NvAf0iMTnKaWUCl2FA72ItALeBUYYYzZ5pCeJSB37NXAe4LfnjlJKqcpTZtWNiMwGegMpIpIFTALiAYwxLwP3AcnASyICUOjqYXMC8J4rLQ6YZYz5pBKOQSmlVBCh9LoZXsby64Dr/KRvA7r5bqGUUupY0jtjlVLK4TTQK6WUw2mgV0oph9NAr5RSDqeBXimlHE4DvVJKOZwGeqWUcjgN9Eop5XAa6JVSyuE00CullMNpoFdKKYfTQK+UUg6ngV4ppRxOA71SSjmcBnqllHI4DfRKKeVwGuiVUsrhNNArpZTDhRToRWS6iOwWEb+Te4vleRHZIiI/iEh3j2XXiMhm1+OaSGVcKaVUaEIt0c8ABgZZfj7QzvUYA0wFEJGGWJOJnwb0ACaJSIPyZlYppVT4Qgr0xpgVwL4gqwwGXjeWr4H6ItIUGAAsMsbsM8bsBxYR/IShlFIqwiJVR98c+M3jfZYrLVC6DxEZIyKZIpK5Z8+eCGVLKaVUlWmMNca8aozJMMZkNGrUKNrZUUopx4hUoN8BtPR438KVFihdKaXUMRKpQD8PuNrV++Z04KAxZhfwKXCeiDRwNcKe50pTSil1jMSFspKIzAZ6AykikoXVkyYewBjzMrAAuADYAuQCo1zL9onIg8Aq164mG2OCNeoqpZSKsJACvTFmeBnLDXBTgGXTgenhZ00ppVQkVJnGWKWUUpVDA71SSjmcBnqllHI4DfRKKeVwGuiVUsrhNNArpZTDaaBXSimH00CvlFIOp4FeKaUcTgO9Uko5nAZ6pZRyOA30SinlcBrolVLK4TTQK6WUw2mgV0oph9NAr5RSDqeBXimlHE4DvVJKOVxIgV5EBorIRhHZIiIT/Cx/RkTWuB6bROSAx7Iij2XzIpl5pZRSZStzzlgRiQVeBPoDWcAqEZlnjPnJXscYc5vH+jcDJ3vsIs8Ykx65LCullApHKCX6HsAWY8w2Y8xRYA4wOMj6w4HZkcicUkqpigsl0DcHfvN4n+VK8yEirYE0YKlHcqKIZIrI1yJycaAPEZExrvUy9+zZE0K2lFJKhSLSjbHDgLeNMUUeaa2NMRnAFcCzItLW34bGmFeNMRnGmIxGjRpFOFtKKXX8CiXQ7wBaerxv4UrzZxilqm2MMTtcz9uA5XjX3yullKpkoQT6VUA7EUkTkQSsYO7Te0ZEOgINgK880hqISA3X6xSgJ/BT6W2VUkpVnjJ73RhjCkVkHPApEAtMN8asE5HJQKYxxg76w4A5xhjjsflJwCsiUox1UnnUs7eOUkqpyifecblqyMjIMJmZmdHOhlJKVRsistrVHurDMXfG5ubCxIkwf360c6KUUlWLYwJ9YiLMmgUvvRTtnCilVNXimEAfEwOXXAJLl0J+frRzo5RSVYdjAj1Az55WkP/++2jnRCmlqg5HBfquXa3ndeuimw+llKpKHBXo27SxqnC2bYt2TpRSqupwVKCPjYUmTWDnzmjnRCmlqg5HBXqAZs000CullCcN9Eop5XAa6JVSyuEcGej37tW+9EopZXNcoD/hBOt59+7o5kMppaoKxwX65GTred++6OZDKaWqCscF+oYNrWcN9EopZXFcoLdL9NnZ0c2HUkpVFRrolVLK4RwX6LXqRimlvDku0NesaT20RK+UUpaQAr2IDBSRjSKyRUQm+Fk+UkT2iMga1+M6j2XXiMhm1+OaSGY+kIYNtUSvlFK2MicHF5FY4EWgP5AFrBKReX4m+X7TGDOu1LYNgUlABmCA1a5t90ck9wEkJ2uJXimlbKGU6HsAW4wx24wxR4E5wOAQ9z8AWGSM2ecK7ouAgeXLauiSk7VEr5RStlACfXPgN4/3Wa600i4RkR9E5G0RaRnmtojIGBHJFJHMPXv2hJCtwBo21BK9UkrZItUY+yGQaozpilVqnxnuDowxrxpjMowxGY0aNapQZrTqRimlSoQS6HcALT3et3CluRljso0x9jBi04BTQt22MtiNscZU9icppVTVF0qgXwW0E5E0EUkAhgHzPFcQkaYebwcB612vPwXOE5EGItIAOM+VVqmSk6GwEA4fruxPUkqpqq/MXjfGmEIRGYcVoGOB6caYdSIyGcg0xswDxovIIKAQ2AeMdG27T0QexDpZAEw2xlR6M6nnwGZ161b2pymlVNVWZqAHMMYsABaUSrvP4/VEYGKAbacD0yuQx7DZd8dmZ0Nq6rH8ZKWUqnocd2cs6Hg3SinlydGBXvvSK6WUQwO9Z9WNUkod7xwd6LVEr5RSDg308fFQp46W6JVSCpwW6Nu0gcmTAb07VimlbM4K9NnZcOAAoAObKaWUzVmBPiEBjh4FdGAzpZSyOTbQa4leKaUsjg30jRvDrl1QXBzlPCmlVJQ5NtB36wY5ObBlS5TzpJRSUea8QJ9vjZZ86qlW0jffRDE/SilVBTgv0LtK9J06QVISfPVVlPOklFJR5thAHxsLZ50Fy5ZFOU9KKRVljg30AH37wvr1sHNnFPOklFJR5uhAf+651rOW6pVSxzNHB/r0dGjQAJYujWKelFIqyhwd6GNioHdvWL48ajlSSqmoCynQi8hAEdkoIltEZIKf5beLyE8i8oOILBGR1h7LikRkjesxr/S2EZWQAHl5XklnnQXbtmk9vVLq+FVmoBeRWOBF4HygEzBcRDqVWu07IMMY0xV4G3jcY1meMSbd9RgUoXz716YNbN4MGze6k3r1sp6XL7eqcPQGKqXU8SaUycF7AFuMMdsARGQOMBj4yV7BGOPZ3Pk1cFUkMxmy886DRx+Fjh3h9ddhxAi6d4dGjeDKK0tWa9LEqtZp0gRat/Z+1KkDcXHWIz6+5HXpR7BlMc6qEFNKVXOhBPrmwG8e77OA04KsPxr42ON9oohkAoXAo8aY9/1tJCJjgDEArVq1CiFbfvTpA3feCY8/DrffDsOGERsfz333wR13wD/+YY2B8+23VjDetQs2bIBPP4Xc3PJ9pD8xMYFPAmWdJGJjrYf9OljasV5WsybUrm09kpKs54QEEInc304pFXmhBPqQichVQAZwjkdya2PMDhFpAywVkR+NMVtLb2uMeRV4FSAjI8OUOxOPPQY9e8LgwbB4MZx/PuPGwY03Bi5pG2MNafzrr/Dnn1BY6P9RUBB4WUWXFxRAUVHJ6yNHrNd2muezvzR/y47FgG6xsZCSAi1bQosW0KED9OhhfQUnnFD5n6+UKlsogX4H0NLjfQtXmhcR6QfcA5xjjMm3040xO1zP20RkOXAy4BPoI8oe6Gb7dndSsOoUEStYpaRUaq6OOWPKd4Lwt8w++eTkWCfDnJySx5498NtvsGkTfPSRta4InHMOXHstDB9uXRUopaIjlJ/fKqCdiKRhBfhhwBWeK4jIycArwEBjzG6P9AZArjEmX0RSgJ54N9RWjsaNraLmDp/z0XFFpKRK6FjJz4c1a+CTT2DWLLj6amt2x4cfhksv1WoepaKhzGZDY0whMA74FFgPvGWMWScik0XE7kXzBFAbmFuqG+VJQKaIfA8sw6qj/4nKFhtr1Rvs2lXpH6W81agBp50GkyZZ7R8ffGDV5192GQwZAn/8Ee0cKnX8EWPKXx1eWTIyMkxmZmbFdnLSSdClC7z1VmQypcqtsBCefRbuvdeqHnv/fTjllGjnSilnEZHVxpgMf8uc2xGwdm2rMllFXVyc1ePpq6+stpJevWDBgmjnSqnjh7MDfU5OtHOhPKSnw6pV8Je/WNU4H39c9jZKqYrTQK+OqcaNYdEi6NwZLrnECvxKqcrl3ECflKSBvopq0MAqzZ9wAgwaZN2/oJSqPM4N9Fqir9IaN7b63OflwYUXwqFD0c6RUs6lgV5FTadO8Pbb8NNPcMUV1o1ZSqnIc3ag//NP6/ZQVWX16wcvvGCV7if4DICtlIoE596YXru2VUTMz4fExGjnRgUxdqxVqn/ySauUP2pUtHOklLM4t0SflGQ9a/VNtfDMM9C/P/ztb/DZZ9HOjVLO4txAX7u29ayBvlqIi4M334S0NBg6FH7+Odo5Uso5NNCrKqNBA/jwQ2vIhIsu0p44SkWKBnpVpbRvD3PnWgOiXXGFNeSxUqpiNNCrKqdfP5gyxeqJc9FFcPhwtHOkVPXm3EBvN8bqwGbV0g03wLRp1iRhp51mjXGvlCof5wZ6LdFXe6NHW/P5HjhgTU/497/Dvn3RzpVS1Y8GelWl9e0LP/wAI0ZYXTDT0uCWW2DtWr0XTqlQaaBXVV5KCvz73/D999a4OFOnWnPKdOxojXP/wQewe3fZ+1HqeOXcO2P1hinH6dIF/vMfq2T/zjvWTFUvvABPPWUtb9XKmljspJOsk0BqKjRvDi1aQL16Ol+tOn6FFOhFZCDwHBALTDPGPFpqeQ3gdeAUIBu43Biz3bVsIjAaKALGG2M+jVjug4mNhbp1ISvrmHycOnYaN7aGTRg7Fo4cgdWr4csvrQbbDRusO2tzc723qVULmjWD5GRo2ND3UaeOVTbw96hd23quUUNPFqp6KjPQi0gs8CLQH8gCVonIvFKTfI8G9htjThSRYcBjwOUi0gkYBvwFaAYsFpH2xphjM07hBRdYRcBu3eDMM60B0OvXt371+ot1hMRE6NnTetiKi63z+6+/wo4d1iMry5orft8+2LMHNm60Xh84EN7nJSRYAT/Yc+m02Fjrzl9/z8GWBVsnJsZ6iPi+9pcWyuuKrAvWc7DXoa5X1bepjkIp0fcAthhjtgGIyBxgMOAZ6AcD97tevw1MERFxpc8xxuQDP4vIFtf+vopM9svw3HOwdSvcfLN3ugjEx1u/Gs9fUFyc73+uv+dgy8JdpyqrDvn0k8cYoJXrEVANoCmYptbYd8XF1sMUQ7Epee9Oc6Ub+1EMJhfMnx5ppZcD17Razh5pTGGh9TlFRbhf+3suLKyUv5KKsGAnBM/ncNNOOMEKWZEWSqBvDvzm8T4LOC3QOsaYQhE5CCS70r8utW1zfx8iImOAMQCtWgX9iYaucWNYuRI2bbK6aWRnW0W4w4dLflWeD/uXaB2I/+dgy8JdpyqrDvmMQB6Fym2o+uzlOGgY3jbFxWWfEIzHycjf67KWR2I7Y7x/Lp7/4qVfB1vmpG08n8uTZvchibQq0xhrjHkVeBUgIyMjclFGBDp0sB5KVQN2lUh8fLRzopwilO6VO4CWHu9buNL8riMicUA9rEbZULZVSilViUIJ9KuAdiKSJiIJWI2r80qtMw+4xvX6UmCpMca40oeJSA0RSQPaAd9EJutKKaVCUWbVjavOfRzwKVb3yunGmHUiMhnINMbMA/4NvOFqbN2HdTLAtd5bWA23hcBNx6zHjVJKKQDEVMFGt4yMDJOZmRntbCilVLUhIquNMRn+ljl3CASllFKABnqllHI8DfRKKeVwGuiVUsrhqmRjrIjsAX4p5+YpwN4IZqc6OB6PGY7P4z4ejxmOz+MO95hbG2Ma+VtQJQN9RYhIZqCWZ6c6Ho8Zjs/jPh6PGY7P447kMWvVjVJKOZwGeqWUcjgnBvpXo52BKDgejxmOz+M+Ho8Zjs/jjtgxO66OXimllDcnluiVUkp50ECvlFIO55hALyIDRWSjiGwRkQnRzk8kiUhLEVkmIj+JyDoRucWV3lBEFonIZtdzA1e6iMjzrr/FDyLSPbpHUH4iEisi34nIfNf7NBFZ6Tq2N11DZ+MaCvtNV/pKEUmNZr4rQkTqi8jbIrJBRNaLyBlO/65F5DbX//ZaEZktIolO/K5FZLqI7BaRtR5pYX+3InKNa/3NInKNv8/y5IhA7zGB+flAJ2C4a2JypygE/m6M6QScDtzkOr4JwBJjTDtgies9WH+Hdq7HGGDqsc9yxNwCrPd4/xjwjDHmRGA/1sT04DFBPfCMa73q6jngE2NMR6Ab1vE79rsWkebAeCDDGNMZazj0YTjzu54BDCyVFtZ3KyINgUlYU7r2ACbZJ4eAjDHV/gGcAXzq8X4iMDHa+arE4/0A6A9sBJq60poCG12vXwGGe6zvXq86PbBmJFsCnAvMx5ridS8QV/p7x5ov4QzX6zjXehLtYyjHMdcDfi6ddyd/15TMOd3Q9d3NBwY49bsGUoG15f1ugeHAKx7pXuv5eziiRI//Ccz9TkJe3bkuU08GVgInGGN2uRb9Dpzgeu2Uv8ezwJ1Aset9MnDAGFPoeu95XF4T1AP2BPXVTRqwB3jNVWU1TUSScPB3bYzZATwJ/ArswvruVuP879oW7ncb9nfulEB/XBCR2sA7wK3GmEOey4x1andMX1kRuRDYbYxZHe28HGNxQHdgqjHmZOBPSi7lAUd+1w2AwVgnuWZAEr7VG8eFyvpunRLoHT8JuYjEYwX5/xhj3nUl/yEiTV3LmwK7XelO+Hv0BAaJyHZgDlb1zXNAfdcE9OB9XIEmqK9usoAsY8xK1/u3sQK/k7/rfsDPxpg9xpgC4F2s79/p37Ut3O827O/cKYE+lAnMqy0REax5edcbY572WOQ5Kfs1WHX3dvrVrlb704GDHpeG1YIxZqIxpoUxJhXr+1xqjLkSWIY1AT34HrO/CeqrFWPM78BvItLBldQXa85lx37XWFU2p4tILdf/un3Mjv6uPYT73X4KnCciDVxXQ+e50gKLdsNEBBs4LgA2AVuBe6KdnwgfWy+sy7kfgDWuxwVY9ZJLgM3AYqCha33B6oW0FfgRqzdD1I+jAsffG5jvet0G+AbYAswFarjSE13vt7iWt4l2vitwvOlApuv7fh9o4PTvGngA2ACsBd4AajjxuwZmY7VDFGBdvY0uz3cLXOs6/i3AqLI+V4dAUEoph3NK1Y1SSqkANNArpZTDaaBXSimH00CvlFIOp4FeKaUcTgO9Uko5nAZ6pZRyuP8HVJ3GvM92v0AAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "clf1 = MLPClassifier(activation='logistic',solver='sgd', learning_rate_init = 1, learning_rate = 'constant', batch_size=1500, hidden_layer_sizes=(5,), random_state=1,n_iter_no_change = 500, max_iter=2000)\n",
    "clf1.fit(X, y)\n",
    "print (\"1 ok\")\n",
    "clf2 = MLPClassifier(activation='logistic',solver='sgd', learning_rate_init = 1, learning_rate = 'constant', batch_size=50, hidden_layer_sizes=(5,), random_state=1,n_iter_no_change =500, max_iter=2000)\n",
    "clf2.fit(X, y)\n",
    "print (\"2 ok\")\n",
    "clf3 = MLPClassifier(activation='logistic',solver='sgd', learning_rate_init = 1, learning_rate = 'constant', batch_size=1, hidden_layer_sizes=(5,), random_state=1,n_iter_no_change =500, max_iter=2000)\n",
    "clf3.fit(X, y)\n",
    "print (\"3 ok\")\n",
    "plt.plot(clf1.loss_curve_,color='b',label='Batch')\n",
    "plt.plot(clf2.loss_curve_,color='r',label='Mini-Batch')\n",
    "plt.plot(clf3.loss_curve_,color='g',label='On-line')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Desventajas de las aproximaciones clásicas\n",
    "\n",
    "**Falta de flexibilidad**\n",
    "\n",
    "* Las aproximaciones clásicas requieren una formulación completa si se desea cambiar la función de costo o evaluar una arquitectura ligeramente diferente.\n",
    "* Una nueva arquitectura requiere el cálculo de todas las formulas de re-estimación de los parámetros y no saca ventaja de herramientas de cálculo simbólico.\n",
    "* Algunos frameworks clasicos soportan regularización, pero no incluyen los avances más recientes a este respecto.\n",
    "* Hay nuevas funciones de activación que evitan problemas con el desvanecimiento del gradiente cuando el número de capas es grande.\n",
    "* Los frameworks clasicos no usan paralelismo.\n",
    "* Los frameworks clásicos no permiten el uso de estrategias más avanzadas de entrenamiento como el Transfer learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[1] Simon Haykin, Neural Networks and Learning Machines, 3ra Edición, Prentice Hall, USA, 2009."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}