Assignment 2. Ethics and Sentiment Classification.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Assignment 2. Ethics and Sentiment Classification.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/DataScienceUB/DeepLearningMaster2019/blob/master/Assignment%202.%20Ethics%20and%20Sentiment%20Classification.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "metadata": {
        "id": "R5RudM9wBzd9",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Assignment 2\n",
        "\n",
        "(Part of this exercise is based on https://blog.conceptnet.io/posts/2017/how-to-make-a-racist-ai-without-really-trying/)\n",
        "\n",
        "## Sentiment Classification\n",
        "\n",
        "This assignment approaches the sentiment classification problem by using deep learning techniques and also exposes some ethical problem related to these methods.\n",
        "\n",
        "This is what we’re going to do:\n",
        "\n",
        "+ Acquire some pre-computed word embeddings to represent the meanings of words\n",
        "+ Acquire training and test data, with gold-standard examples of positive and negative words\n",
        "+ Train a simple classifier to recognize other positive and negative words based on their word embeddings\n",
        "+ Compute sentiment scores for sentences of text using this classifier\n",
        "+ Analyze the results to look for unwanted bias. "
      ]
    },
    {
      "metadata": {
        "id": "RsS5s_tEBzd_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Word embeddings\n",
        "\n",
        "There are several datasets of pre-trained English word embeddings such as `word2vec`, pretrained on Google News data, and `GloVe`, pretrained on the Common Crawl of web pages. We will use `GloVe`.\n",
        "\n",
        "GloVe comes in three sizes: 6B, 42B, and 840B. The 42B version is pretty good and is also neatly trimmed to a vocabulary of 1 million words. We will just use the 42B version.\n",
        "\n",
        "> **GloVe.42B** data: 42B tokens, 1.9M vocab, uncased, 300d vectors, 1.75 GB download"
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T09:08:49.211876Z",
          "start_time": "2019-03-29T09:01:34.100348Z"
        },
        "id": "cEqMSox2BzeA",
        "colab_type": "code",
        "outputId": "dea1030b-6cf6-48c6-c99a-c14154317bb3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 277
        }
      },
      "cell_type": "code",
      "source": [
        "!wget http://nlp.stanford.edu/data/glove.42B.300d.zip"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2019-04-02 09:11:45--  http://nlp.stanford.edu/data/glove.42B.300d.zip\n",
            "Resolving nlp.stanford.edu (nlp.stanford.edu)... 171.64.67.140\n",
            "Connecting to nlp.stanford.edu (nlp.stanford.edu)|171.64.67.140|:80... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://nlp.stanford.edu/data/glove.42B.300d.zip [following]\n",
            "--2019-04-02 09:11:45--  https://nlp.stanford.edu/data/glove.42B.300d.zip\n",
            "Connecting to nlp.stanford.edu (nlp.stanford.edu)|171.64.67.140|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 1877800501 (1.7G) [application/zip]\n",
            "Saving to: ‘glove.42B.300d.zip’\n",
            "\n",
            "glove.42B.300d.zip  100%[===================>]   1.75G  3.88MB/s    in 10m 14s \n",
            "\n",
            "2019-04-02 09:22:00 (2.92 MB/s) - ‘glove.42B.300d.zip’ saved [1877800501/1877800501]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T09:10:02.327170Z",
          "start_time": "2019-03-29T09:09:24.646719Z"
        },
        "id": "aLvMgbhYBzeE",
        "colab_type": "code",
        "outputId": "611af88b-c53d-4d57-8877-e3162a4ae0f9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "cell_type": "code",
      "source": [
        "!unzip glove.42B.300d.zip\n",
        "!rm glove.42B.300d.zip"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Archive:  glove.42B.300d.zip\n",
            "  inflating: glove.42B.300d.txt      \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:04:47.628009Z",
          "start_time": "2019-03-29T15:01:58.108263Z"
        },
        "id": "G_FFD2bbBzeH",
        "colab_type": "code",
        "outputId": "7f38b44f-b29f-410f-a2bc-da292286c555",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "from tqdm import tqdm\n",
        "\n",
        "def load_embeddings(filename):\n",
        "    labels = []\n",
        "    rows = []\n",
        "    with open(filename, encoding='utf-8') as infile:\n",
        "        for i, line in tqdm(enumerate(infile)):\n",
        "            items = line.rstrip().split(' ')\n",
        "            if len(items) == 2:\n",
        "                # This is a header row giving the shape of the matrix\n",
        "                continue\n",
        "            labels.append(items[0])\n",
        "            values = np.array([float(x) for x in items[1:]], 'f')\n",
        "            rows.append(values)\n",
        "    \n",
        "    arr = np.vstack(rows)\n",
        "    return pd.DataFrame(arr, index=labels, dtype='f')\n",
        "\n",
        "embeddings = load_embeddings('glove.42B.300d.txt')\n",
        "embeddings.shape"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1917494it [03:34, 8920.24it/s]\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(1917494, 300)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "metadata": {
        "id": "fR6RR0h9BzeJ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Positive and Negative Words\n",
        "\n",
        "We need some input about which words are positive and which words are negative. There are many sentiment lexicons you could use, but we’re going to go with a very straightforward lexicon from https://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html  \n",
        "\n",
        "There is a copy of these files in the GitHub repository of the course."
      ]
    },
    {
      "metadata": {
        "id": "nOVI_gAAE9OB",
        "colab_type": "code",
        "outputId": "941716d0-2fe0-428a-865f-7d8e2984f565",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 399
        }
      },
      "cell_type": "code",
      "source": [
        "!wget https://raw.githubusercontent.com/DataScienceUB/DeepLearningMaster2019/master/data/positive-words.txt\n",
        "!wget https://raw.githubusercontent.com/DataScienceUB/DeepLearningMaster2019/master/data/negative-words.txt\n"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--2019-04-02 09:28:00--  https://raw.githubusercontent.com/DataScienceUB/DeepLearningMaster2019/master/data/positive-words.txt\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 22408 (22K) [text/plain]\n",
            "Saving to: ‘positive-words.txt’\n",
            "\n",
            "\rpositive-words.txt    0%[                    ]       0  --.-KB/s               \rpositive-words.txt  100%[===================>]  21.88K  --.-KB/s    in 0.01s   \n",
            "\n",
            "2019-04-02 09:28:00 (1.71 MB/s) - ‘positive-words.txt’ saved [22408/22408]\n",
            "\n",
            "--2019-04-02 09:28:15--  https://raw.githubusercontent.com/DataScienceUB/DeepLearningMaster2019/master/data/negative-words.txt\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.0.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 50857 (50K) [text/plain]\n",
            "Saving to: ‘negative-words.txt’\n",
            "\n",
            "negative-words.txt  100%[===================>]  49.67K  --.-KB/s    in 0.03s   \n",
            "\n",
            "2019-04-02 09:28:15 (1.84 MB/s) - ‘negative-words.txt’ saved [50857/50857]\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:04:56.665741Z",
          "start_time": "2019-03-29T15:04:56.619197Z"
        },
        "id": "u6kfpHJaBzeJ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def load_lexicon(filename):\n",
        "    lexicon = []\n",
        "    with open(filename, encoding='latin-1') as infile:\n",
        "        for line in infile:\n",
        "            line = line.rstrip()\n",
        "            if line and not line.startswith(';'):\n",
        "                lexicon.append(line)\n",
        "    return lexicon\n",
        "\n",
        "pos_words = load_lexicon('positive-words.txt')\n",
        "neg_words = load_lexicon('negative-words.txt')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "ud0fhRxyBzeM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Some of these words are not in the GloVe vocabulary. Those words end up with rows full of NaN to indicate their missing embeddings, so we will use Pandas to clean the dataset."
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:04:59.940580Z",
          "start_time": "2019-03-29T15:04:58.168200Z"
        },
        "id": "mW5w55RiBzeM",
        "colab_type": "code",
        "outputId": "36e44eb5-40e0-47c2-ae58-49b45ffd5685",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 260
        }
      },
      "cell_type": "code",
      "source": [
        "pos_vectors = embeddings.loc[pos_words].dropna()\n",
        "neg_vectors = embeddings.loc[neg_words].dropna()"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:1: FutureWarning: \n",
            "Passing list-likes to .loc or [] with any missing label will raise\n",
            "KeyError in the future, you can use .reindex() as an alternative.\n",
            "\n",
            "See the documentation here:\n",
            "http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike\n",
            "  \"\"\"Entry point for launching an IPython kernel.\n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:2: FutureWarning: \n",
            "Passing list-likes to .loc or [] with any missing label will raise\n",
            "KeyError in the future, you can use .reindex() as an alternative.\n",
            "\n",
            "See the documentation here:\n",
            "http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike\n",
            "  \n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "metadata": {
        "id": "aDd0yYBBBzeO",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now we make arrays of the desired inputs and outputs. The inputs are the embeddings, and the outputs are `1` for positive words and `-1` for negative words. We also make sure to keep track of the words they’re labeled with, so we can interpret the results."
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:05:03.485323Z",
          "start_time": "2019-03-29T15:05:03.466320Z"
        },
        "id": "TPjdI-oKBzeP",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "vectors = pd.concat([pos_vectors, neg_vectors])\n",
        "targets = np.array([1 for entry in pos_vectors.index] + [-1 for entry in neg_vectors.index])\n",
        "labels = list(pos_vectors.index) + list(neg_vectors.index)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:05:05.256478Z",
          "start_time": "2019-03-29T15:05:05.210348Z"
        },
        "id": "nBW36XlcBzeR",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "train_vectors, test_vectors, train_targets, test_targets, train_labels, test_labels = \\\n",
        "    train_test_split(vectors, targets, labels, test_size=0.1, random_state=0)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "qmOaXxGpBzeU",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now it is time to make your classifier, and train it by running the training vectors through it for 100 iterations. You can use a logistic function as the loss, so that the resulting classifier can output the probability that a word is positive or negative."
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:16:50.546163Z",
          "start_time": "2019-03-29T15:16:50.542669Z"
        },
        "id": "LreSnh1RBzeU",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 176
        },
        "outputId": "6774d9ba-e529-44b4-c02b-2684b8e12861"
      },
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import SGDClassifier\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "model = SGDClassifier(loss='log', random_state=0, n_iter=100)\n",
        "model.fit(train_vectors, train_targets)\n",
        "SGDClassifier(alpha=0.0001, average=False, class_weight=None, epsilon=0.1,\n",
        "       eta0=0.0, fit_intercept=True, l1_ratio=0.15,\n",
        "       learning_rate='optimal', loss='log', n_iter=100, n_jobs=1,\n",
        "       penalty='l2', power_t=0.5, random_state=0, shuffle=True, verbose=0,\n",
        "       warm_start=False)\n"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/stochastic_gradient.py:152: DeprecationWarning: n_iter parameter is deprecated in 0.19 and will be removed in 0.21. Use max_iter and tol instead.\n",
            "  DeprecationWarning)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SGDClassifier(alpha=0.0001, average=False, class_weight=None,\n",
              "       early_stopping=False, epsilon=0.1, eta0=0.0, fit_intercept=True,\n",
              "       l1_ratio=0.15, learning_rate='optimal', loss='log', max_iter=None,\n",
              "       n_iter=100, n_iter_no_change=5, n_jobs=1, penalty='l2', power_t=0.5,\n",
              "       random_state=0, shuffle=True, tol=None, validation_fraction=0.1,\n",
              "       verbose=0, warm_start=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "metadata": {
        "id": "4UlxWk8NBzeX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We can evaluate the classifier on the test vectors. \n"
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:16:54.627721Z",
          "start_time": "2019-03-29T15:16:54.625191Z"
        },
        "id": "gh0kOsUdBzeX",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "ad472c45-7051-45a0-aa82-99b90b7b7332"
      },
      "cell_type": "code",
      "source": [
        "accuracy_score(model.predict(test_vectors), test_targets)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9502262443438914"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "metadata": {
        "id": "983emjCeBzeZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Let’s define a function that we can use to see the sentiment that this classifier predicts for particular words, then use it to see some examples of its predictions on the test data."
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:06:07.705034Z",
          "start_time": "2019-03-29T15:06:07.670604Z"
        },
        "id": "QAtTmjPkBzea",
        "colab_type": "code",
        "outputId": "b72efa4a-e10c-4717-e76a-5cebf599a07e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 783
        }
      },
      "cell_type": "code",
      "source": [
        "def vecs_to_sentiment(vecs):\n",
        "    # predict_log_proba gives the log probability for each class\n",
        "    predictions = model.predict_log_proba(vecs)\n",
        "\n",
        "    # To see an overall positive vs. negative classification in one number,\n",
        "    # we take the log probability of positive sentiment minus the log\n",
        "    # probability of negative sentiment.\n",
        "    return predictions[:, 1] - predictions[:, 0]\n",
        "\n",
        "\n",
        "def words_to_sentiment(words):\n",
        "    vecs = embeddings.loc[words].dropna()\n",
        "    log_odds = vecs_to_sentiment(vecs)\n",
        "    return pd.DataFrame({'sentiment': log_odds}, index=vecs.index)\n",
        "\n",
        "\n",
        "# Show 20 examples from the test set\n",
        "words_to_sentiment(test_labels).ix[:20]"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:18: DeprecationWarning: \n",
            ".ix is deprecated. Please use\n",
            ".loc for label based indexing or\n",
            ".iloc for positional indexing\n",
            "\n",
            "See the documentation here:\n",
            "http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>fidget</th>\n",
              "      <td>-9.931679</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>interrupt</th>\n",
              "      <td>-9.634706</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>staunchly</th>\n",
              "      <td>1.466919</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>imaginary</th>\n",
              "      <td>-2.989215</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>taxing</th>\n",
              "      <td>0.468522</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>world-famous</th>\n",
              "      <td>6.908561</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>low-cost</th>\n",
              "      <td>9.237223</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>disapointment</th>\n",
              "      <td>-8.737182</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>totalitarian</th>\n",
              "      <td>-10.851580</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>bellicose</th>\n",
              "      <td>-8.328674</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>freezes</th>\n",
              "      <td>-8.456981</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>sin</th>\n",
              "      <td>-7.839670</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>fragile</th>\n",
              "      <td>-4.018289</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>fooled</th>\n",
              "      <td>-4.309344</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>undecided</th>\n",
              "      <td>-2.816172</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>handily</th>\n",
              "      <td>2.339609</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>demonizes</th>\n",
              "      <td>-2.102152</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>easygoing</th>\n",
              "      <td>8.747150</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>unpopular</th>\n",
              "      <td>-7.887475</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>commiserate</th>\n",
              "      <td>1.790899</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "               sentiment\n",
              "fidget         -9.931679\n",
              "interrupt      -9.634706\n",
              "staunchly       1.466919\n",
              "imaginary      -2.989215\n",
              "taxing          0.468522\n",
              "world-famous    6.908561\n",
              "low-cost        9.237223\n",
              "disapointment  -8.737182\n",
              "totalitarian  -10.851580\n",
              "bellicose      -8.328674\n",
              "freezes        -8.456981\n",
              "sin            -7.839670\n",
              "fragile        -4.018289\n",
              "fooled         -4.309344\n",
              "undecided      -2.816172\n",
              "handily         2.339609\n",
              "demonizes      -2.102152\n",
              "easygoing       8.747150\n",
              "unpopular      -7.887475\n",
              "commiserate     1.790899"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "metadata": {
        "id": "D7CPVesRBzed",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Sentiment score for text\n",
        "\n",
        "There are many ways to combine sentiments for word vectors into an overall sentiment score. The simplest way is to average them."
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:07:29.746290Z",
          "start_time": "2019-03-29T15:07:29.740768Z"
        },
        "id": "-XJpCFwjBzee",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import re\n",
        "TOKEN_RE = re.compile(r\"\\w.*?\\b\")\n",
        "# The regex above finds tokens that start with a word-like character (\\w), and continues\n",
        "# matching characters (.+?) until the next word break (\\b). It's a relatively simple\n",
        "# expression that manages to extract something very much like words from text.\n",
        "\n",
        "\n",
        "def text_to_sentiment(text):\n",
        "    tokens = [token.casefold() for token in TOKEN_RE.findall(text)]\n",
        "    sentiments = words_to_sentiment(tokens)\n",
        "    return sentiments['sentiment'].mean()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:07:40.366537Z",
          "start_time": "2019-03-29T15:07:40.351478Z"
        },
        "id": "f_qHy19xBzeg",
        "colab_type": "code",
        "outputId": "54934e84-7980-4e8c-9332-6f8d063e0db7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "cell_type": "code",
      "source": [
        "text_to_sentiment(\"this example is pretty cool\")"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3.889968926086298"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:07:48.548136Z",
          "start_time": "2019-03-29T15:07:48.540902Z"
        },
        "id": "1eJusL1qBzek",
        "colab_type": "code",
        "outputId": "06097bfd-c4d7-4a1c-d2cf-8f0d6af4fb10",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "cell_type": "code",
      "source": [
        "text_to_sentiment(\"meh, this example sucks\")"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "-1.1774475917460698"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "metadata": {
        "id": "vVSJqoSBBzep",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Let’s see what it does with a few variations on a neutral sentence:"
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:08:39.114595Z",
          "start_time": "2019-03-29T15:08:39.099228Z"
        },
        "id": "E-oqx5yFBzeq",
        "colab_type": "code",
        "outputId": "ed354604-7397-42ad-c55d-0edc9db43247",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 69
        }
      },
      "cell_type": "code",
      "source": [
        "print(text_to_sentiment(\"Let's go get Italian food\"))\n",
        "print(text_to_sentiment(\"Let's go get Chinese food\"))\n",
        "print(text_to_sentiment(\"Let's go get Mexican food\"))"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2.0429166109408983\n",
            "1.4094033658140972\n",
            "0.3880198556012173\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:09:13.624994Z",
          "start_time": "2019-03-29T15:09:13.604583Z"
        },
        "id": "ojy7OzeyBzet",
        "colab_type": "code",
        "outputId": "1529467a-56ec-4fe7-c59a-2a89cbba9e10",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 87
        }
      },
      "cell_type": "code",
      "source": [
        "print(text_to_sentiment(\"My name is Emily\"))\n",
        "print(text_to_sentiment(\"My name is Heather\"))\n",
        "print(text_to_sentiment(\"My name is Yvette\"))\n",
        "print(text_to_sentiment(\"My name is Shaniqua\"))\n"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2.228617936474531\n",
            "1.3976291151079159\n",
            "0.9846380213298556\n",
            "-0.47048131775890656\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "lMh897d-Bzew",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The system has associated wildly different sentiments with people’s names. You can look at these examples and many others and see that the sentiment is generally more positive for stereotypically-white names, and more negative for stereotypically-black names."
      ]
    },
    {
      "metadata": {
        "id": "SJLdr7ROBzew",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Ethical problem\n",
        "\n",
        "We want to learn how to not make something like this again. So let’s put more data through it, and statistically measure how bad its bias is.\n",
        "\n",
        "Here we have four lists of names that tend to reflect different ethnic backgrounds, mostly from a United States perspective. The first two are lists of predominantly “white” and “black” names adapted from Caliskan et al.’s article. I also added typically Hispanic names, as well as Muslim names that come from Arabic or Urdu; these are two more distinct groupings of given names that tend to represent your background."
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:10:18.816997Z",
          "start_time": "2019-03-29T15:10:18.787085Z"
        },
        "id": "eSAQ22POBzex",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "NAMES_BY_ETHNICITY = {\n",
        "    # The first two lists are from the Caliskan et al. appendix describing the\n",
        "    # Word Embedding Association Test.\n",
        "    'White': [\n",
        "        'Adam', 'Chip', 'Harry', 'Josh', 'Roger', 'Alan', 'Frank', 'Ian', 'Justin',\n",
        "        'Ryan', 'Andrew', 'Fred', 'Jack', 'Matthew', 'Stephen', 'Brad', 'Greg', 'Jed',\n",
        "        'Paul', 'Todd', 'Brandon', 'Hank', 'Jonathan', 'Peter', 'Wilbur', 'Amanda',\n",
        "        'Courtney', 'Heather', 'Melanie', 'Sara', 'Amber', 'Crystal', 'Katie',\n",
        "        'Meredith', 'Shannon', 'Betsy', 'Donna', 'Kristin', 'Nancy', 'Stephanie',\n",
        "        'Bobbie-Sue', 'Ellen', 'Lauren', 'Peggy', 'Sue-Ellen', 'Colleen', 'Emily',\n",
        "        'Megan', 'Rachel', 'Wendy'\n",
        "    ],\n",
        "\n",
        "    'Black': [\n",
        "        'Alonzo', 'Jamel', 'Lerone', 'Percell', 'Theo', 'Alphonse', 'Jerome',\n",
        "        'Leroy', 'Rasaan', 'Torrance', 'Darnell', 'Lamar', 'Lionel', 'Rashaun',\n",
        "        'Tyree', 'Deion', 'Lamont', 'Malik', 'Terrence', 'Tyrone', 'Everol',\n",
        "        'Lavon', 'Marcellus', 'Terryl', 'Wardell', 'Aiesha', 'Lashelle', 'Nichelle',\n",
        "        'Shereen', 'Temeka', 'Ebony', 'Latisha', 'Shaniqua', 'Tameisha', 'Teretha',\n",
        "        'Jasmine', 'Latonya', 'Shanise', 'Tanisha', 'Tia', 'Lakisha', 'Latoya',\n",
        "        'Sharise', 'Tashika', 'Yolanda', 'Lashandra', 'Malika', 'Shavonn',\n",
        "        'Tawanda', 'Yvette'\n",
        "    ],\n",
        "    \n",
        "    # This list comes from statistics about common Hispanic-origin names in the US.\n",
        "    'Hispanic': [\n",
        "        'Juan', 'José', 'Miguel', 'Luís', 'Jorge', 'Santiago', 'Matías', 'Sebastián',\n",
        "        'Mateo', 'Nicolás', 'Alejandro', 'Samuel', 'Diego', 'Daniel', 'Tomás',\n",
        "        'Juana', 'Ana', 'Luisa', 'María', 'Elena', 'Sofía', 'Isabella', 'Valentina',\n",
        "        'Camila', 'Valeria', 'Ximena', 'Luciana', 'Mariana', 'Victoria', 'Martina'\n",
        "    ],\n",
        "    \n",
        "    # The following list conflates religion and ethnicity, I'm aware. So do given names.\n",
        "    #\n",
        "    # This list was cobbled together from searching baby-name sites for common Muslim names,\n",
        "    # as spelled in English. I did not ultimately distinguish whether the origin of the name\n",
        "    # is Arabic or Urdu or another language.\n",
        "    #\n",
        "    # I'd be happy to replace it with something more authoritative, given a source.\n",
        "    'Arab/Muslim': [\n",
        "        'Mohammed', 'Omar', 'Ahmed', 'Ali', 'Youssef', 'Abdullah', 'Yasin', 'Hamza',\n",
        "        'Ayaan', 'Syed', 'Rishaan', 'Samar', 'Ahmad', 'Zikri', 'Rayyan', 'Mariam',\n",
        "        'Jana', 'Malak', 'Salma', 'Nour', 'Lian', 'Fatima', 'Ayesha', 'Zahra', 'Sana',\n",
        "        'Zara', 'Alya', 'Shaista', 'Zoya', 'Yasmin'\n",
        "    ]\n",
        "}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "8B5G2exKBze1",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now we’ll use Pandas to make a table of these names, their predominant ethnic background, and the sentiment score we get for them:"
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:10:46.744680Z",
          "start_time": "2019-03-29T15:10:46.676412Z"
        },
        "id": "mgePQtf6Bze1",
        "colab_type": "code",
        "outputId": "d07c9804-25ac-46e7-c96b-685847e07524",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 766
        }
      },
      "cell_type": "code",
      "source": [
        "def name_sentiment_table():\n",
        "    frames = []\n",
        "    for group, name_list in sorted(NAMES_BY_ETHNICITY.items()):\n",
        "        lower_names = [name.lower() for name in name_list]\n",
        "        sentiments = words_to_sentiment(lower_names)\n",
        "        sentiments['group'] = group\n",
        "        frames.append(sentiments)\n",
        "\n",
        "    # Put together the data we got from each ethnic group into one big table\n",
        "    return pd.concat(frames)\n",
        "\n",
        "name_sentiments = name_sentiment_table()\n",
        "\n",
        "name_sentiments.ix[::25]"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:12: FutureWarning: \n",
            "Passing list-likes to .loc or [] with any missing label will raise\n",
            "KeyError in the future, you can use .reindex() as an alternative.\n",
            "\n",
            "See the documentation here:\n",
            "http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike\n",
            "  if sys.path[0] == '':\n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:12: FutureWarning: \n",
            "Passing list-likes to .loc or [] with any missing label will raise\n",
            "KeyError in the future, you can use .reindex() as an alternative.\n",
            "\n",
            "See the documentation here:\n",
            "http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike\n",
            "  if sys.path[0] == '':\n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:12: FutureWarning: \n",
            "Passing list-likes to .loc or [] with any missing label will raise\n",
            "KeyError in the future, you can use .reindex() as an alternative.\n",
            "\n",
            "See the documentation here:\n",
            "http://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike\n",
            "  if sys.path[0] == '':\n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:14: DeprecationWarning: \n",
            ".ix is deprecated. Please use\n",
            ".loc for label based indexing or\n",
            ".iloc for positional indexing\n",
            "\n",
            "See the documentation here:\n",
            "http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated\n",
            "  \n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>sentiment</th>\n",
              "      <th>group</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>mohammed</th>\n",
              "      <td>0.834974</td>\n",
              "      <td>Arab/Muslim</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>alya</th>\n",
              "      <td>3.916803</td>\n",
              "      <td>Arab/Muslim</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>terryl</th>\n",
              "      <td>-2.858010</td>\n",
              "      <td>Black</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>josé</th>\n",
              "      <td>0.432956</td>\n",
              "      <td>Hispanic</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>luciana</th>\n",
              "      <td>1.086073</td>\n",
              "      <td>Hispanic</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>hank</th>\n",
              "      <td>0.391858</td>\n",
              "      <td>White</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>megan</th>\n",
              "      <td>2.158679</td>\n",
              "      <td>White</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "          sentiment        group\n",
              "mohammed   0.834974  Arab/Muslim\n",
              "alya       3.916803  Arab/Muslim\n",
              "terryl    -2.858010        Black\n",
              "josé       0.432956     Hispanic\n",
              "luciana    1.086073     Hispanic\n",
              "hank       0.391858        White\n",
              "megan      2.158679        White"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:10:59.406581Z",
          "start_time": "2019-03-29T15:10:59.243215Z"
        },
        "id": "UmGNsOjdBze4",
        "colab_type": "code",
        "outputId": "f263b651-8181-4cca-90f9-b938b6015df2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 383
        }
      },
      "cell_type": "code",
      "source": [
        "import seaborn\n",
        "\n",
        "plot = seaborn.swarmplot(x='group', y='sentiment', data=name_sentiments)\n",
        "plot.set_ylim([-10, 10])"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(-10, 10)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfsAAAFcCAYAAADPkheEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xd8XNWZ+P/PdPUy0qjaqpblJsu2\n3Bu2sQEb04JDryGbZAkk2ZBNssn+gvMKCYHlm8IugYSY0INjejFgimPcO+6yitV7GZXR9PL7Q2bk\nsWzLBFt3ZvS888rrxTlz7/AMI+m599xznqPy+Xw+hBBCCBG21EoHIIQQQoiLS5K9EEIIEeYk2Qsh\nhBBhTpK9EEIIEeYk2QshhBBhTpK9EEIIEeYUSfZlZWUsXbqUF198EYCmpiZuv/12brnlFr7//e/j\ndDoHnfOb3/yGG2+8kZtuuomDBw8Od8hCCCFEyBr2ZG+1WvnVr37FnDlz/H2PP/44t9xyCy+//DLZ\n2dm8+uqrAefs2rWLmpoa1q5dy69//Wt+/etfD3fYQgghRMga9mSv1+t5+umnSUlJ8fft3LmTSy+9\nFIDFixezffv2gHO2b9/O0qVLAcjPz6e7uxuLxTJ8QQshhBAhbNiTvVarJSIiIqDPZrOh1+sBSEpK\noq2tLeD19vZ2EhMT/W2j0TjoGCGEEEKcWdBN0Duf6r3nc4zb7bkQ4QghhBAhT6t0AABRUVHY7XYi\nIiJoaWkJGOIHSElJob293d9ubW3FZDKd8z3NZutFiVUIIYQIRiZT7FlfC4o7+7lz5/Lhhx8CsGHD\nBhYsWBDw+rx58/yvHzlyhJSUFGJiYoY9TiGEECIUqYZ717vDhw/zyCOP0NDQgFarJTU1lccee4yf\n/vSnOBwOMjIyePjhh9HpdPzHf/wHDz/8MBERETz22GPs2bMHlUrFgw8+yLhx487572lr6x2mTySE\nEEIo71x39sOe7IeLJHshhBAjSdAP4wshhBDi4pFkL4QQQoQ5SfZCCCFEmJNkL4QQQoQ5SfZCCCFE\nmJNkL4QQQoQ5SfZCCCFEmJNkL4QQQoQ5SfZCCCFEmJNkL4QQQoQ5SfZCCCFEmJNkL4QQQoQ5SfZC\nCCFEmNMqHYAQQghxMTXUmPlsQzk9Zht5hclcckUhesPISn9yZy+EECJsud0eNrx5hK4OK16vj4pj\nbezeUq10WMNOkr0QQoiw1d1pw25zB/Q1N3QrFI1yJNkLIYQIW/HGSCIidQF96ZnxCkWjHEn2QgwT\nm9vGpvptbKjZiNnepXQ4QowIWq2Gy6+bSGJyFBqNioIJKcxYkKN0WMNO5fP5fEoHcTG0tfUqHYIQ\nfi6Pi4d3/5EWaysAkdpIfjL9e5iikhSOTAgRLkym2LO+Jnf2QgyDwx2l/kQP/Xf525p2KRiREGIk\nkWQvxDBQqVSD+tQM7hNCiItBkr0Qw2BS0jgyY9L97RhdNHMzZikYkRBiJJFn9kIME6fHyb7Wgzg8\nTqalTCZWH6N0SEKIMHKuZ/aS7IUQQogwIBP0hBBCiBFMkr0QQggR5iTZCyGEEGFOkr0QQggR5iTZ\nCyGEEGEuKDb0XbduHW+//ba/ffjwYfbv3+9vT5w4kWnTpvnbzz77LBqNZlhjFEIIIUJV0C2927Vr\nF++//z4PPvigv2/WrFns3LnzS72PLL0TQggxkoTU0rsnnniCe++9V+kwhBBCiLARFMP4Xzh48CDp\n6emYTKaAfqfTyQMPPEBDQwOXX345d99995DvlZgYhVYrQ/0ieFgcffyzegd2t4OF2TNJiUlWOiQh\nxAgRVMn+1Vdf5brrrhvU/+Mf/5irr74alUrFbbfdxvTp0ykqKjrne5nN1osVphBfmtPj4uFdv6fV\n1g7A28c+4icz7iclyjTEmUIIcX5CZhh/586dTJ06dVD/zTffTHR0NFFRUcyePZuysjIFohPiX3e4\n45g/0QPYPXa2Ne5WMCIhxEgSNMm+paWF6Oho9Hp9QP+JEyd44IEH8Pl8uN1u9u3bR0FBgUJRCvGv\n0agGP1LSquUxkxBieATNMH5bWxtGo9Hf/stf/sKMGTOYOnUqaWlprFq1CrVazZIlS5g8ebKCkQrx\n5U1KGkdWbCa1vQ0AxOpjmCdb3AohhknQLb27UGTpnQg2Lo+LA22HcXicFKdMIkYXrXRIQogwIlvc\nCiGEEGEuZCboCSGEEOLCk2QvhBBChDlJ9kIIIUSYk2QvhBBChDlJ9kIIIUSYk2QvhBBChLmgKaoj\nhBDBrGfbVnr370VvSiFx+Qq0sXFKhyTEeZNkL4QQQ+jatJHWF54DoA+wHi8l+/9brWhM4vx53F6O\nH2mmx2wnpyCJtMx4pUMadpLshRBiCL07tge0HTXVOBobMWRkKBSR+DI+fOMINZUdAOzfUcsV108i\nt2BkbTEtz+yFEGIImviE0zo0aGJjlAlGfCndZps/0X/h8N4GhaJRjiR7IYQYQtLV16CJPzn0q1KR\ndPW18sw+RGg0qvPqC3dSG18IIc6D1+XEXlGBzmRCl2xSOhzxJWxcX0rpwWagP9GvvLGYjKyEIc4K\nPbIRTpho77Lxt/dLqWzoJj8znrtXjCM5PlLpsIQQIqj5fD7qq810m21k5ycRGx+hdEgXhST7MPHo\ny/sore3yt8dlJfDjW6YpGJEQQohgIbvehYmyuu6Adnl991mOFEIIIQZIsg8heZmBE4JyM2SCkBBC\niKFJsg8h96wYT/7JhJ+fGcc9K8YrHJEQQoQmq8XB9o2VfPz20UFL88KRPLMPQT6fD5Vq5C0dEUKI\noVSVt7N7cxVOh4cJU9KZNicbAJvViaXHQVJKf32EtX/dRVenzX/eFV+bRO7Y0C60c65n9lJBLwRJ\nohdCiMEsPXY2vHEEr7f/HnbnpiriEiKx9TnZ9mklXq+PuIQIZi7MC0j0AMcPN4d8sj8XSfZCCCHC\nQmNdtz/Rf6GmsoPKY63+/p4uO+VHmwedGxmlG5YYlSLP7IUQ4jxY9u+lec3TdLzzFh6rVelwxBkk\npw4uYRwbF4HHE3gB4LC5mThtYF+D6Fg9U2dnXfT4lCR39kIIMYSebVtpfuZpf7vvyGGyfvpzBSMS\nZ2JMjmbupfns2VKN2+WlYGIq0+ZmcaKsDXP7wAVa/rgUJs8YxcQpGfRZHGRkJaDVahSM/OKTCXpC\nCDGEukcfxlZ2PKAv56GH0aelKxSROBePx4vX60On60/glh47e7bW0G22kVeYzKRpmWE590km6IWR\nioZuKuq7GTMqnjEjcE9mIZSgiTlteFitRh0ZpUwwYkgajRrNKTfqMXERLFpeqFxAQUCSfQjZsLuO\nVz4p97dvurSAy2aMVjAiIUYG48qrsZaW4rX29bevWIE2Xi62g5Wl14HX4yUuYWDvEJ/Ph9fjQ6Md\nmVPVZBg/hHz/8c30Wl3+dmyUjj9+b4GCEQkxcnisVmzHj6FLScWQOUrpcMQZ+Hw+Nn1QxrEDTQDk\nFCRx2bUTqa82s/nDMnp7HGTnG7n0qvEYIsJv9r3Uxg8Tpy8pOb0thLh4NFFRxEwtkUQfxOqqzP5E\nD1Bd3sGxA018/PYxenscANRUdrLrs2qFIlROUCT7nTt3Mnv2bG6//XZuv/12fvWrXwW8vm3bNlat\nWsWNN97IE088oVCUyrtsZtY520IIMZL1mG2D+tqae3E63AF9rc09wxVS0AiaZ/YzZ87k8ccfP+Nr\nDz30EGvWrCE1NZXbbruNyy+/nDFjxgxzhMq7am4OOWmxlNd3UzAqnqK8JKVDEl9CdU8tG6o34vA4\nWZA5mykpRUqHJERYyco3ov5UhfeUdfXjitKoqezA1jfwCDQzK1GJ8BQVNMn+bOrq6oiPjyc9vX+J\nyyWXXML27dtHZLIHKMpLkiQfgnqcvfxx/19wepwAHDdX8L2p32JsYr7CkQkRPuISIll5w2T276zD\n4/YyeXom6aMTWH59EVs+Kqe7y0Z+oYnp87KVDnXYBU2yr6io4Dvf+Q7d3d3cd999zJs3D4C2tjaM\nRqP/OKPRSF1dnVJhCvEvOdpx3J/oAXz4+LztsCR7IS6wzOxEMrMD79ztNhe2PicOm5u+Xicejxet\nLryL6JwuKJJ9Tk4O9913H8uXL6euro477riDDRs2oNfr/+X3TEyMCvuKSCJ0jGEUHAvsy0lOP+fs\nWSHEV+dyuvnbu6XYbf3D+DWVHRza3cDyr42sx2hBkexTU1NZsWIFAFlZWSQnJ9PS0sLo0aNJSUmh\nvb3df2xLSwspKSlDvqfZLLWrRfBIJo35mbPZ2rATHz7GJuRTHD8lLJeIhgNbeRl9hw+hz8wkdvpM\nVOqgmMss/gUdrRZ/ov9CTVVHWP7uBX0Fvbfffpu2tjbuuece2tra6OjoIDU1FYBRo0ZhsVior68n\nLS2NjRs38thjjykcsRBf3s2FX+OK7CU4PE7Sooe+YBXKOL0OvvXYUdLu/AbO5mb6Dn6OzpRCdPEU\nuQAIYg67m4N76uk2W8nOTyIyWnfaBL0EBaNTRlAU1bFYLPzoRz+ip6cHl8vFfffdR0dHB7GxsSxb\ntozdu3f7E/xll13GPffcM+R7huNVmxDi4qv55S9w1NUOdKjVZPz7fTQ+9QR4PADEzp5D+je/rVCE\nYihvvrifpvpuf3vq7NE01HSdrI1vYv7SMWH5zP5cd/ZBkewvBkn2Qoh/Rc1Dv8RRXTXQodEQObYQ\n27GjAcflPvL/0CXJyphg02228vKfdwX0pWXGcd3t0xSKaPgE/TC+OD+dPXae//A4lQ3d5GfGc8fl\nhRjjIpQOS4iwYlyxkqYn/w9O3gclLlmKo6F+8IE+7zBHJs6H3qBFrVYFVBhVqVW8/sI+esw2cscm\nM2/pmBE3gVuzevXq1UoHcTFYrc6hDwoxT7xxiMNVnbjcXlrMNurbLMydJFtshgqb286Opt2Ud50g\n0ZBApFYu1IKRIT2DmJIZ6FNTSbx8OQmLl6COjqF39y7/BUDMtBISLlmscKTiTHQ6DT6vj8a6/mF8\nvUGLw+qi22zD7fLS1mzB54NROeFXWCc62nDW12QYP4T826Mb8ZxytapRq3j6x/IHJxS4PC5+u+dx\nmvtaAIjSRvKTGd8jOVKGgUOFvbaGvs/3o0tJ6Z+hr5WB0WBmbu+jy2wjMlLHGy/uD3gtNSOOr90R\nfsP6MowfJnLSYqls7Aloi9BwuKPUn+gBrG4bWxt3cU3+cgWjEl9GRFY2EVkjr/JaqEpMjiYxORqX\n04PeoMHp8PhfM6XFKBiZMmTtSAi5e8V4slP7E3x2aizfuHK8whGJ86U6Q5/6jL1CiAtJp9ew9KoJ\nxMT1D3Fn5RuZsSBX4aiGnwzjhyCX24NuhE0uCXUur5tHdz9OY18zANG6KH4y/XskRRqHOFMIcSH4\nfD487vAukytL74QIAg6Pk70tB3B4HExLKSbeII9hhLiYus1Wjn7ev7/9hCkZxCdGAmDpdWDrc5Kc\nGoNKFT4jbJLshRBCjCiWHjtr1+zx72VviNBy4z0zOPJ5I/u21eDzQWJyFFfdWEx07NlnsYcSmaAX\nJlxuD+t31FLR0M2YzHhWzM6S4fwQ8nnrId6r+si/n/2y7EVKhyRE2Ko41uZP9NBfQvfI/gb2bhuo\njmhut7J/Ry3zlxUoEeKwkmQfQp7/8DhbD/U/8z1S1UlHt10m6YWIdlsHa468hPdkIZY3K9djikpm\nimmSwpEJEX58Ph86/eAbIY9n8EB2b7d9OEJSnMzGDyG7jrUGtHceaznLkSLYlJtP+BP9F453lisU\njTgf7t4evK7wK84V7j7fWcczf9jC1o/LiYjU+fsNEVpaGnswRATe4+aPHxmbUsmdfQhJjDXQarb5\n28Ywec40EoyOzTyvPqE8T18fTU89gfXYUdSRkZhuvJn4+QuVDkuch9amHrZvrPS3PTYXxTNHU1na\niqXHQdPJqnop6bHoDVoKJqYydmKqUuEOK7mzDyG3LB2L4eTQlEGv4ealYxWOSJyvUbEZXDfmSiI0\nBjQqDfMyZjI7fbrSYYkz6Fz/DtaTm954bTZaX3wed08PPTt30Pjk/9H26j/wWCwKRynOpLVx8MRs\nu82FpccR0KfWqLjqpmLGFaUNV2iKkzv7EDI5P4nH7p1LRX03BaMSiIqQry+ULM26hMWj5uPxedFr\ndEOfIBThqA/c9MbndmP+8H3MH77v77OWHiP7vx8c7tDEENJHxw/qG52TSPnRFrynPK+PHYEbiMmd\nfQipbenl18/v5Y+vHuTXL+yhtkWWF4YajVojiT7IRU8qCmhrYmKxVQTOr3BUV+FobBzOsMR5SEqJ\nYfGKQmLjI4iO0TN7UR4FE1OZfUkeanX/evrYOAPT5+coG6gC5NYwhDz7finNnVYAmjqsPPt+Kb+4\na4bCUQkRXhIuXYbHaqV31050RiPJ13+dzg/WBx6k0aCJGXn11UPBuMnpjJscuBto8czRjMpNpK3Z\nwpjxKWi1I+8+V5J9CKlrtZyzLYT46lRqNcnXXEfyNdf5+5Kuugbb8VI8vb2gUpG08mq0cXEKRim+\njEN76tm2sRKvx8ferdWsvLHYX01vpJBkH0Im5Bg5dKLD3x6XnUhDm4X0pGj/EJUQ4sIzZI4i97eP\nYasoR5dsQp86MmZwhwKXy8PxQ81YehzkjzNhOm03ULvNxfaTiR6gp8vO7i1VLL1qghLhKkbK5YaQ\n7j4nL39URkVDN8ZYA3VtFpwuL8nxEXz/68VkJkcrHaIQQgyrN1/a719Sp1LByhuLGZWT6H+9s62P\ntWt2B5yTlhnHdbefeT97u81FX68Doyk65Ormn6tc7sh7cBHC4qP1/Pu1k/jtt+fQ3GnF6eov0tLe\nbWfdxgqFoxNCiOHV3tLrT/QAPh8c3tcQcExichSJyVEBffnjzlxI5/C+Bp5/Yjv/eGYPrzy9i54u\n2xmPC0UyjB8iPjvQyIbddahUsHhKBn12d8DrLebw+aEUQojzodEMvl91u728u/YAPV128gpNzFyY\nw8obJrNnaw3dZht5hclMmpZJfbWZLR+V09ttJ2+ciVmX5LLt00o87v6bqK5OG7u3VHPpyvAoSS7J\nPgSU1XXx7Pul/vZLH5WTaYqmoa3P3zdtbLISoQkhhGISk6PJH2eisrQNAK1OTVtzD3Zr/83Q/h21\n6PQaikoyiY2PwOf1ERsXgdvl5cM3jvg3yik73IJGo/In+i/0hNFNlCT7EHC0ujOg7QOmFZjITY+j\noa2PojwjK+fmKBKbEEIoadk1Eygs6sTS4yA2zsB76w4FvF57opPaE5001/cP95ceambG/OyAHfEA\nzB1WEoyRdHUOJPi8QtPF/wDDRJJ9CBidMnjSxdjRCUzMNSoQjTgf7bZOdjfvw6A1MDuthChd1NAn\nCSG+NJVKRXZ+EtC/ja1Wp8btGrhDj4k1UHHaJmJ1VWb0Bg1Oh8fflz4qgQlT0tm9pZoes428QhOT\nZ4wang8xDCTZh4BpY5O5tGQU/9zfgNfnw6DTsPlgI6NTYoiL1isdnjhNi7WNR3c/jt3TX497c8N2\nfjbjP9jVso93T2zA4XGwIHMO1+avCLnZvkIEM0OElkXLC9m8oRyH3U366HhK5mZRWdrKqevOIqP1\nXDY/h80b+p/Z548zUTI3G51eEzbP6E8nS+9CyO/Wfs7hqoEh/cn5Sfzg68UKRiTO5M2K9XxU+8+A\nvlUFV/Fa+bv4GPh1u3PCTcxMO/PyHyHEv87j8eJ0uImM6r8Z2r6xks931gGgN2i4+uYpg9bjhwNZ\nehcmjtWYA9pHqjrPcqRQkkY1+Neq3doZkOgBqrprhiskcYG4OjrwOmWP+2Cn0aj9iR4I+GedXoPu\n5O6hI4kk+xAyKiXmnG0RHOZlziJWN/DdZMVmMjdjJioCh+zz43OGOTLxr3J3man51WqqfvIAJx74\nPj3btykdkjhPdpuLnZ+d8Lf7ep3s3lKtXEAKCZpn9o8++ih79+7F7Xbz7W9/m8suu8z/2pIlS0hL\nS0Oj6b8ae+yxx0gdgeUq714+jiffPEyL2UaqMYq7l49TOiRxBsaIRP571gPsbztEpMZAsWkSOo2O\nW8d/nXdPfIjD42Rh5hxKUqcoHao4T+1vvo6jphro3+O+5cXniJ4yFU3kyKqvHoqsfc6A7W0BLD12\nhaJRTlAk+x07dlBeXs7atWsxm81cd911Acke4OmnnyY6emSXg81KjeXX/zaLFrONNGOUTO4KYjH6\naBZkzg7om5M+nTnp0xWKSHwVztO2s/U5HLg72tGMGq1QROILPp+P9hYLeoOG+MTBq14Sk6JIMkXT\ncUpdkoLxZ75ZbG+xUHGshcgoPeMmp2OICIoUeUEExSeZMWMGkydPBiAuLg6bzYbH4/HfyYt+tS29\n/PntIzR1WElPiuLbV08kKzX8JpkIEWyiJxdjP1Hpb+uSTegzMhWMSAA47C7eeeUAbc39O4COL05n\n0fLCgGNUKhVX3jiZfdu+qKBnIndsMu+/eoi6qk6SUmK45IqxuN1e3nrpc7ze/lGAsiMtrLqrJGxu\nqjSrV69erXQQarUanU4HwLp169DpdFx++eX+15977jmOHz/O3/72N6qrq5kzZ86QX4DVGn6TaP74\n6kHqT16dWmwuqpt7uWSK/MER4kLz2GzYKytQGyJQGwxE5o8BlQqPxUJE/hjS7r5HtrgNAgd21VN+\nZGANfXuLhdG5icTERQQcp9dryc5PYuykNExpsWz64Dgnjrfj80GfxUlDTRfWPiftLQPbhlv7nGRm\nJxIbH/hewSw62nDW14Lizv4LH3/8Ma+++irPPPNMQP/3vvc9FixYQHx8PN/97nf58MMPueKKK875\nXomJUWi14TUycKb97BMSo9hb2opOq2bK2BQ0ahV1Lb2s+6SM7j4nS2dksUAuCIQ4b10HD1H5m0fw\n2Gyo9XoKvn8fpvnzSPnmHfDNO5QOT5zC4/IO6uvptFN5rA2P28v0eTlk5yXhcrrZt6MWc4eVwqI0\nWhsDl2Z3m21k5yUNeq9kU+w5l7OFkqBJ9ps3b+app57ir3/9K7Gxgf9xr732Wv8/L1y4kLKysiGT\nvdlsvShxDiefz4fT5cVwcpnI6fvZF2YlcO8jn9Lc2f9Z8zPi+MHXi/nZ0zvotboA2Ffait3qZEqB\n1M4X4nzU/OUZPLb+kqlep5OKvzyDd2xR2AznhpOM7AT2bh9Ywqo3aNi04bi/gt6xg02suquEbZ9W\nUl/dv3R515Yq0kbF0X1K3fu4hAjGT02n9HCzv4xudr4RfaQmpGq2nOvCJCiG8Xt7e/nhD3/ImjVr\nMBqNg1679957Wb58ORqNhmeeeYaZM2dSUFBwzvcM9WH847Vm/ueV/fxjYyXHqjuZmJvEtEITHd02\nLHY3RXlGctJj2Xu8zX+Oube/YtuR6sD1+Co1lBSeeUtHIUSgjrdex3fKWnqfy0ni0v4Jw6qT84hs\nlRW0vvgc3Zv+iUqrwzAqfMqqhgJrn5Pjh5vx+XwUTEjF7faSmh7HqNxEGmq6/Mf5fIAKKo+1BZwf\nHWsgKSWani4byakxzL+sAJ/Px9TZozEmxzBhSgaTSjKpr+oEVERG6Yb3A/6Lgn4Yf/369ZjNZn7w\ngx/4+2bNmkVhYSHLli1j4cKF3HjjjRgMBiZMmDDkXX2o83p9/OWdo/7kXVbfzdpPy1k2fTSVjT30\n9DmpbOwhwjD469PrBj+6SA6hZ07hbEvDjsByuWNWoD5DAR6hrNhZs+n6+CN/W5eczIkffg80GoxX\nrCB+4SLqf/c/+Bz9v5+2suNoYmKInjhJqZBHlG6zldee24fj5DbfqZlxXHvrVNRqFVVl7YOOj441\noFIRUC43IlLHilVFQP9EvPXrDuFxe4mI0nHl14vwen289NROXM7+2vmzLsll2pzsi//hLqKgSPY3\n3ngjN95441lfv/POO7nzzjuHMSJldfc5/Yn+C9XNvTz3wXF/v7nXQUV9NwadBoer/wcyNkrHpSWj\nsDncfLS7Dh8wOiWGZdNleZDSmvpaeOX4G/4qep/UfUZmTDqz0ksUjkyczrTqRrQJiVhLS1HpdfTt\n29v/gttNx1tv4PN4/In+C5b9+yTZD5Mj+xr9iR6gpaGHhhozHa0WqsraiYkzYOnp/36SUqLp7bJh\nNMXQcXLOk1anZtqcLKC/rO6Wj8r9W9varS62bzyBWq3yJ3qAPVtrmDQtE/0ZbrBCRehGHsYSYvSk\nGaP8z+IBxmcnsuVgU8BxbV02Vn9jJpsPNKLVqFk0NYOYSB03XVrA0umj6LO5yUqNkWeNQaCqu3ZQ\nudzqnlpJ9kHGeuwoLS89j6utjdhpJWjiEwYd43U6BvXpU0ZekS+lfLE07lTlR1o4frjF305IjKRk\nQTb/fO84Ha39K5j0ERpmLcxDb9CwfWMlbpeXcUVpARcOAL3ddgynJXWP24vL6ZFkLy4slUrFd79W\nxEsbjtPQ3kdRXhI3LB5Dr9XFvrKBZ0/FY5LJTI7mpksHz19Ijo8kOX44oxbnkh+fjQpVQMLPT8hV\nMCJxOq/TSeOTT+C19ieH3t27iJl2WhEklYrYWXPA66Pr04/B6yVy7Fiip8qGRsNl/JR0jh1s8k/C\nM5qiaT9tpVKX2UZjTTeeUyrnOe0e3G4PWz+u8F8wbGmpIDk1JmDJ3ZjxKUTH6NnycYW/b3RuItGx\nZ38eHgqCYoLexRDqE/TiovTMK0pn+axspo01odOqmZhrpM/mxuXxMrXAxK2XjUUfZssLw1WMPpqk\nCCO1vQ1oVGouzVrIolHzZNQliDga6un6eENAnzoqisTlK3A2NeJzucDjwbJnN/ELF2G64SacLc3Y\njh2l69OPcff2EF00Wb7TiywqWs+Y8SYionTkFSYzf2kBTfU9mDsGRkK1OjVZuUYa67oDzo1PiKS1\nKXB2fe7YZNJHx6PVqZk4NYOSuVmkZcaTYIxEo1UzZnwK8y7NR6MJ/vk1QT9BT5yf6Agdd0k9/JA1\nK71Ehu2DmD4tHXV0NN6+gbKqhlGjSVyyFEddLT2bPwPAa+2j5flnMX39BqyHDvYf6PPRvfFTYqZM\nk2f3wyA+MYrp83L87VmX5NLW3Etvtx2NVs344nTiEiOJT4z0L7FLyYglNn5wMjSlxTK+OH1Qf8GE\nVAomhM/jGdnPPsRYbC5qWnrRH0mtAAAgAElEQVTJTo0lJjI0loMIESqspcdofflFnC3N/R0eDxF5\nefg8Hhw1gVsSx81fSM+WzwL6TDffSuKly4YrXHEKr9dHW3MvmzeU09bc//c/3hjJjPk51FZ2Unak\n/5m+RqvG5/Xh9foomJDC4ivHhcRd+/k41zp7ubMPIfvL2/jzW0dwur3otWq+fc1EphaYlA5LiLAR\nNW48o3/6M0786D/8a+3tJ05gyMkJOE6blETcvPn0bN08sKZLoyF60uRhjlh8Qa1W0dtt9yd6gO5O\nG91mGxXHBkrqetxecsYksWhFYcA+9+FOkn2Qc3u8+Hyg06p55ZNynCeXiDjdXtZ+UiHJPkiZ7V1s\nbdyFx+dhTvp0UqLkewoVrvb2gKI6AGi0ROSPwV5djSYqEtPNtxFVMJb0b/875o82oNLpMC6/Ev0I\n3Ho7mHxR/e5U1j7noBn8Drt7RCV6kGQf1F7/7AQbdtfi88GlJaPosgT+AeqyDF4CJJRncfXxyJ7H\n6XX2z/D9rH47/zXzB8TrY/m87TAOj4MpKUXE6Eb2ls3BypA5Cq3RiLuz09+n0euxHjsKgKe3l7ZX\nXiJmcjGx02cSO32mUqGK0+SOTWbnpirstv5y4VqdmuIZo2hr6g2YmFdYlKZUiIqR2fhB6kh1J8+9\nX4rn5LOlioZuxmcn0tY1UM95XlEaU+TOPujsbt7P3tYD/rbb5yZKG8GbJ97ns4ZtHO44xs7mvZSk\nFBOpleqGwUalVhM1YRLuzg5UGg3xlyzG0dSIp2ugDLXXaiW2ZDraOFnfGkx0Og3540xoNGpS0uNY\neMVYEpOiySs0oVariIrRM31eDoWT0nDYXdRVmVGp+ivqhQOZjR+CapoHTzDMTI6mp89JS6eVVGMU\nV8+XddrByKAZPDzY6eiirrfB3+51WtjauJOVeZcPOlYoz5CZSeb3/sPfdnd14ag64W+rDBFok2Rz\nqWAUlxDJ2ImpmDus/pr2EZE6Zl2S5z+muaGbd9ce9FfJm70oj6mzsxSJd7iExxTEMDQ+O3FQX2VD\nN3WtFpxuL3WtFl744LgCkYmhFJsmMTp2YFthU2QSY+IHX5i5vZ5BfUJZPq+X9rfeoPoXP6fh8d/j\naOi/QEu+9mtE5PYnC3VUFKl33IUmMlLJUMVZ7N5cxT+e2cNHbx3lxSd30FTXdYZjqgPL4W6pPuPz\n/nAy5J39T3/6U377298G9N1zzz2sWbPmogUlIDc9jrtXjGP9jlp8Ph9XzMzipY/KAo45XNV5lrOF\nknQaHT8q+S5HOkpxez0UJU8AYEPNRlpt/Rt1RGgimJsxQ8kwxRmYP/yAznfeAsDZ2ICjro7c3/4P\n2oQEsn7+C1xmM5qYaNS6kTW5K1Q47C727aj1t90uL3u21jB9vootG8rp7rKROzYZa1/gY16324vL\nFdrlcIdy1k/29ttv88orr1BeXs6tt97q73e5XLS3D95ZSFx4CyZnsGByhr+96fNGaloGhvdHmWKU\nCEucB61aS7EpsLjKf06/jx3Ne3G4ncxMm0pSpPEsZwul9B06ENB2mzuxfL6fnq2bcTY1El1UTPKq\nGxSKTgzF7fLi9Zw2897m4oPXD2O39k/aKzvcQmZ24J4Ho3MTiY4J7XK4QzlnUZ2WlhZ+9KMfcf/9\n9/v71Go1Y8aMISFh8AYRwSQci+rUNPfy5JuHae2ykZIQyb9fO4nstLMXURBCfDktLzxH96aN/rZK\nr0ebkIirdWCTlcRll5N8w01Yjx7B1dFOdFExusTBj92EMt5bd5DayoFRz5K52ezdFlgQKSUjlqKS\nUdRUdmBMimLyjFHo9KF/V3+uojrnVUGvt7eXrq7A5x6jRwf3tqnhmOwBfD4fHT12PF4fKQmRUodb\niAvI3d1N4xN/xH7iBOrISIwrr6Z93dqAY/QZGURk59KzfSvQf0Ew6kc/ITIvX4mQxWlcLg9H9jVi\n7ugjZ0wSo3ONPP/E9oDd7YpKMpm/bPAGYqHuKyX7hx56iNdeew2j0cgXh6pUKj755JMLG+UFFq7J\n/mBlO3999xgWm4vUxEjuv34yGcmyXluIC6ln905s5eXoU1PpeOctvJaBXdGiJxfTdzBwuD9mWgkZ\n995/+tuIIFFT2cFnH5bR1+sgd2wyi5aPwxAR+nfyp/tK5XJ37tzJjh07MBjC+3lGKPB4vfzt/VIs\nJwtGtJhtvPJpOT+8YYrCkQkRPnp37aT5L0/624bcXNxqNZ6eHnSmFKKLByd7r9M13GGK89TS2MPG\n9aXY+lzo9BrGF6eHZaIfypBL77KzsyXRK6izx87aT8v567tHOVDeTvdpVfSa2q1nOVMoyevzUm6u\npMHSNOi1MN17Kmx0bQwctXRUVZFy212oIiJwtbXS+sLzaI1JAweoVCQsWTLMUYrzteWjcmx9/Rdj\nLqeHzz4oG5G/g0Ne3qSlpXHrrbdSUlKCRjOwd/r3v//9ixqYAJfby8Mv7qOjxw7A9sPNZCRF0XjK\nvs3FY5LOdrpQSK/Twh/2PUWztX/zjRmpU7lr4s1sbtjBuyc+xOlxMj9zNteNuRK1SkpdBBuV/rRl\ndSoV3Z9txGe3+7vc5k6SvrYKb5+FmGnTicwfM8xRivP1xRa3X+jtceD1+tBoRtZ8pyGTfUJCAnPm\nzBmOWMRpSmvN/kQP4APSk6PJSoulrtXCpFwj1y3IO/sbCEVsqt/mT/QAu1v2MyGpkLXH38BH/x3F\np3WbGRWTIfvbByHjipXYyo7jc/XfDSYsvhRHfV3gQT4fsdOmo08beTXWQ01eoYljBwZG2LLzk8Jm\nS9svY8hkf99992E2m6mvr6eoqAiv14taPfL+QykhNmpwvWaDTkNeRhxXzMwiK1WW3QWjHufgyaEn\numv8if4L1T21kuyDUFThOHJ+/QjWw4fQpaURNbaQnu3bsJUNVKyMLBgriT5EzF86BofdTXVFO16P\nD4/Hg8PuwhARHvXwz9eQyf69997jj3/8I3q9nnfffZdf/epXTJw4kVWrVg1HfCNaTlocsyemsuNI\n/xrf6Agt2w43s+1wMwC3X17I4qmZ53oLoYAZqVPZ1rjLn9xj9THMS5/BloYdAQk/P0H2NghWOqOR\n+IWX+Ntxc+aijo7Csm8v+pRU4hdfqmB04svwAfXVZn+xnfrqLnZ9VsWCy8ae8zyH3Y21z0GCMSos\nljgPufTu+uuv58UXX+Rb3/oWL7zwAna7ndtvv51169YNV4z/knBaelfd3ENPn4tn3jtKj3Vg1m98\ntJ7f3z9fwcjE2ZR2lrOtcRdRuiguHb0QU1QSO5v28m7VBhweBwsz58gmOCHMY7HQ/sar2KuqiBxb\nSPK1X0MdITsYBqOOVgv/eGZPQF9KRizX33H2UbUj+xvZ9kkFbrcXoymaK79eRExc8H+/X2npXWxs\nLJGnbPgQERGBTjeyhj+UlpMWB4DrtDKQTrdXiXDEeRhnLGCcMbBox6z0Ehm2D0Fel5OuTz/BUVtD\n1PiJxM2bT/MzT/uX3zlqa/Ba+0j7xr8pHOnIdvxQM7UnOjEmB1bES0iKIipaH1APPzNrcMVDj8eL\nWq3CYXez9eNyPCf/3na29bF7SzWLV4wbng9ykQyZ7BMTE3njjTdwOBwcOXKE9evXYzRKTW8lLC0Z\nxTvbqgPaQoiLq+Vva+jdtROA3p07cHW003foYMAxlv37lAhNnPT5zlq2bxzYgrilsYcVX58MgEaj\n5orrJ7H14wq6zTbyCpPJH2/C5fSg02twuTz8c30plaVtREbrKZ4xyp/ov9DVGTijPxQNOYzf09PD\nH/7wB3bu3Iler6ekpIT77rtPauMrZNvhZg5WtjM5P5m5k2SCUKjw+rz8s34rR9pLyYhJ4/KcJcTo\npPJhsPPa7VTc/+9wyp9JbXIyKpUaV9vAiouI3Dyyfv4LJUIUwCt/3YX5tJoj190+ld2bq+loszA6\nx8j8ZQXYrE7WrztEt9mG3qDhkisKMbf3sWfrQO18tRqiYyPo7R5YCTV3ST7FM4O7RDx8xWH8uLg4\nfvEL+SEOBgcrO3hhw3EcTg8HKjuIidQxOV/W2YeCD6s/5d2qDQCUmsup623gB9O+o3BUYigqrRZ1\nZCRe60Ai0cTEYrr+6zT9+Uk8ll40CQmYbr5NwShFZJQeMwPfkVan5p/vH/dfAJQdaUGtUWG3ufzr\n7p0OD599WEZKemCC9HphxoIcais76emykVdoYvKM0B9FHTLZv/nmmzz33HP09vYGVB0K9tr44ejv\nH5fhcHoAcDg9/P2Tckn2Qcjj9bCv9SCddjPFpkmkRaewpzWwvGp51wm6HT3EG+IUilKcD5VWS/J1\nq2h9+YX+u3u1mojsHCLy8sn9n9/hbm9Dl5KK6pSCY2L4zVyQw3vrDuE6+fdxyqws9mypDjimodqM\nRhf4PTnsbkxpsdRVmf19Or2G3IJkCsNs5HTIZP+nP/2Jhx56iDRZU6q4zl5HYPuUgjsiePzl0PMc\n7jgGwHtVH3H/lG+SaIinuW9gm9QITQSR2sizvYUIIgmLl+DzeWh7+SXweunetBFXWyujfvif6NMz\nlA5PAOmjE7j93tk01XVjNEUTGx9B6cEmLD0DfzOjYw1o9Rq6TqlAmpwaw/T5ObhcHsqPthITa2DO\n4nz0hvCrnT/kJ8rLy2PmzJkXPZDf/OY3HDhwAJVKxc9+9jMmT57sf23btm387ne/Q6PRsHDhQr77\n3e9e9HiC0czxKWw91BzQ9np91LT0khBjIDFW9jBQWnNfiz/RA3h8HjbWbeGa/OXU9zbS67KgRoXX\n5+G/t/6aZdmLWJa9SLmAxXmx7AucgGc9egRnSwv61FSFIhKnM0ToyClI9reXXjWeT98rpafLTmSU\njuaGHgBUKohLjCQ1I45ZC3PRaNTMX1rA/KXht+XtqYZM9jfddBPf+MY3KC4uDqiNf999912wIHbt\n2kVNTQ1r166lsrKSn/3sZ6xdO7CH9EMPPcSaNWtITU3ltttu4/LLL2fMmPCvRb33eBubPm8gwqDl\nytnZ3HF5IUlxEVQ2dJOfGc/sCWn8/OkdtJhtqFUqrl2Qy8q5OUqHLU7jAxIjEvjV3P9ic8MOXqt4\nB6fXhdPr4s3K9WTHjWZsouyFHszUp28GplKhNujPfLAICumjE7jl27Po6rDyyl93+/t9PkhIjOTS\nleMVjG74DZnsH3nkEaZMmYLP58Ptdl+UILZv387SpUsByM/Pp7u7G4vFQkxMDHV1dcTHx5Oeng7A\nJZdcwvbt28M+2R+r7uSJNw7520eqOnjkO3O59pRa+M99UErLyckmXp+PNzdXMa8oXe7wFZQWnUpR\n8gQOtR8FQK1SU24+wU82/5K8+Bzy4rMHnVPVXSPJPsgZV6zEWnoMn6N/WDh+0RK0CYPXaovgolKp\nUJ+hDr57BNYoGTLZm0wmHn744YsaRHt7OxMnTvS3jUYjbW1txMTE0NbWFrCu32g0UldXd6a3CZCY\nGIVWG7qTZtZ9diKgbXN4qGjqpaHNQmmNmXHZiXT1Be6h7fX58GnU51x+IS6+/1p8L7vq99PU28Zb\npR9ic/dfkJ3oriYucvByu2nZE+Q7C3amqaT9+Qm69n9ORHo6ceNDu8DKSGIyxVIwPoXyYyeXSqpg\n/pKCEfc7N2SyX7BgAa+//jpTp05Fqx04fPToi7fm8ELsNWw2h/Y+7zH6wRcqn+yu4cjJWaNHTnSQ\nnxE4kzslIZKECG3Y1hgIJQWRhcR6E7C73w7o7+zr4roxV/JRzT9RoWJZ9iJMqjT5zkKAu8tCX7cN\nm8GCQ76vkLL4ynFk5CTQY7aRO9aEMTU6LH/nvtI6+7///e+D+lQq1QVdepeSkkJ7e7u/3draislk\nOuNrLS0tpKSkXLB/d7BaNDWT/RXtVNR3owIumZLBllMm5wHUtFi4a/k4dh9rwRgXwVVzc1CrQ3/D\nhnCREmUiKSKRDvvAsp5IbSROj5P/nH4fyZGybDJU2CorqP9/j+Jz9pdcjV+8hNRb71A4KnG+NFo1\nE4pH9sqJIZP9p59+etGDmDdvHv/7v//LTTfdxJEjR0hJSSEmJgaAUaNGYbFYqK+vJy0tjY0bN/LY\nY49d9JiUFmnQ8rPbSqhvsxCh15AcH0lpbRfNnQMjFqaECBYWZ7BwhP8QByu1Ss13Jt/N6xXv0mJt\nw+aycdxcwXFzBZ/UfsaPZ3yP1CiT0mGK89C5/l1/ogfo/udGjCuuQpcoz+2DkcvpobK0FafDQ/54\nE9ExMo/prMn+z3/+M9/+9rf5z//8zzNu7/foo49esCCmTZvGxIkTuemmm1CpVDz44IO8/vrrxMbG\nsmzZMlavXs0DDzwAwIoVK8jNHTlbg44yxfj/+dbLxvLkG4exOtxEGrRcPW/k/HcIVRkxadw35Zvs\naz3ImsMv+vvtHgfbG3dz7ZgVCkYnzpfXEVjjAp8Pr92G+ePd2MrLiMjLJ/HSZai04bc+O9R4PF7e\neHEfHa19AOzZWs2qu0qISxjZdS3OWht/8+bNLFiwgDfeeGPwSSoV11577UUP7qsIx+cxAA6Xh7e3\nVPHJ3nqcbi/jshL47teKiI6QnQiD2YG2I/zl0HMBfctzLpVtboOQx2Kh9eUXsB47iiErm5Rb78BR\nW0PTU0/4j4maOAl9ahpdn37s74tfeAmpd9ytRMjiFFXl7Xzw2uGAvqmzRzN7UfiveDnXM/vBaxJO\nWrBgAQCVlZVcd911Af/fvXv32U4TF5i518FLG8r4w7oDbDnYhNXuZsPuOv/2tqW1XazfUTPEuwil\nTUoaR1Zspr8dq49hXsYsBSMSZ9P69xfp3bUTT28v1iOHafrzn4iaOInIsYUAqGNiSbh0GT3btwac\n17Nt65neTgy3M9y+WnodvPrsXp75wxY2ri/F5fIMf1wKO+uY00cffcSGDRvYvn07ra0Duzu53W5J\n9sPE5/Pxu7Wf09DePxx1sLKDy2eOxuMN/GluaOtTIjzxJWjUGn447V4OtB/B4XEwxVREtC5K6bDE\nGVhLjwW0HTXVtL+2FlvZcQC8ll6a1/wFTUwsXtvA1qeaWNnnIBhk5RsxmqLpPPl3UW/QUHuiE4et\nv05M6cFmIiJ1zFkc/nf6pzprsl+wYAFGo5HDhw8zZ84cf79Kpbqg1fPE2dW1WvyJ/gtVjT3EROqw\n2AbW2BePST79VBGEdBod+fE5OD1OSfRBLCI7h76DAxsX6VJTsdcEjp55+/pIWnkN7a/9A5/bDRoN\nphtuGu5QxRloNGquu20q5UdbcTrdJJtiePcfBwOOaazrUig65Zw12UdERFBSUsKbb76JwWDA5/Nd\nkPXv4vzFxxjQqFUBd/IRBi0LizM4XmfG7vQwe0Iqi6bIbPxQ8Pfjr7O1YSc+fIxNyOc7xXdj0EjJ\n1WCTcsttNFl6sZ84gS41jbR7voVl7y4cVVX+YzSxscQvWkzsrNnYq08QkZWDNiFBwajFqfQGLROn\n9v9ddLk86A1anI6BCrCp6SNvFOasE/S+sGbNGp588kn6+vrvMH0+HyqVimPHjp3rNMWFywS9d7ZV\n8+bmE/h8EBWhxWof+IG94/JCFk3NPMfZIlhUdFXx+31PBvRdX3AVS0YvUCgiMRSvw+Gvie91OGh9\n8Xl69+1Fn5JCyi23EVkwVuEIxfmqPdHJZx+WYemxkz0miSVXjsMQhpOazzVBb8hkv3z5ctasWUNG\nRmjdPYZLsgfo6LbT0WPnT28cosc6MHwfH63n9/fPVzAycS42tw2tWodOrWVH0x5eOPaPgNcvGTWP\nG8Zeo1B0Qow8Ho8XzRlq5YeLr1RBLzs7O+QSfbhJio8gKT4Clyfwusw5AjdzCAVOj4vnjr7CgbbD\nGDQGrs6/gqkpReg1epye/sIsKlRMNU1SOFIhRpZwTvRD0axevXr1uQ4oLy/n1Vdfpauri9LSUo4e\nPcrRo0eZMGHCMIX4r7FanUMfFGIcLg9lp0wsuWJWFuOzpYJXsPmk7jM21fcvw3L73BztOM6CzDlM\nSynC4rTgcDtwel0c6ywn3hBHRkyawhELIcJBdPTZKwUOeWff2tqKXq/n888/D+hftWrVV49MfClf\nW5hHVkoMFQ3djMmMZ/q48N8jIBQ1WJoC2j58NFiaKDZNpCAxn4Mnt7/tcnTz3NFXGJOQS4IhXolQ\nhRAjxJDJ/uGHH8br9dLR0eHfnEYox5QQSXefE9MIL/0YzMYlFrCnZeDiWKvWUtFVhcfn4UT3aUu4\nfF5qe+pJMEmyF0JcPEMm++3bt/Pzn/8cvV7PBx98wG9+8xvmzp3LokWLhiE8caqN++p5YUOZv337\nZWNZPG2UghGJM5mdPp1uZw/bG3fjAzrsnXxa9xnUwZiEwP0MtCoNOfFZygQqhBgxhpyt8Pvf/55/\n/OMf/rv673znO/zpT3+66IGJwd7aWh3Qfvu0tggOKpWKK3Iu5Zdzf4pOE7i850RXDQsy56DX6InX\nx3HnhJuJ0599Bq0QQlwIQyb7qKgokpMHKrQZjUZ0uvBbnxgKXKfNvpfZ+MFPowr8FdOo1TRZmnF6\nnHQ7e3ivagNWl/UsZwshxIUxZLKPiIhg165dAHR1dfHyyy9jMMjewEpYWhI4ZL9sugzhB7vLshej\nYmCL6IlJ46noHqjE1mxtZVuT7DUhhLi4hiyq09TUxOrVq9m5cycGg4Hi4mIefPBBMjODu3JbOBXV\nOdXe421UNnSTnxlPSaFMmAxWVpeVPS2f4wMyY9Ko6alnVEwGnXYzL5auCzj2suzFXJO/XJlAhRBh\n41/a4vYL27dvZ+HChezZs4e8vDzKysrYtGnTBQ1QnL+SQhM3LBkjiT6IWV1WHt79R9aWvck/yt7k\n2SOvMCu9hNz4LFxeV0A9fK1ay4zUqQpGK4QYCYacjb927VpeeOEFPv74Y8aOHctLL73EnXfeyS23\n3DIc8YnTtHfbqGrqJTc9luR4WX4XLI51lNHQ18R441gqu6rptJv9r5kdXexq2svO5n3UWxoB0Km1\nTDUVsSRroRTVEUJcdEMme4PBgF6vZ9OmTVx99dWo1SO33KDSdhxp5q/vHsPr86FWqfjmyvHMniiJ\nQmmvlb/Dp3WbAXiT9cxJnzHomKa+Fn+iB3B53RgjjYyODe7HYUKI8HBemfuXv/wl+/btY+bMmezf\nvx+nM/xK0YaCVzdV4j05xcLr8/HqpkqFIxJ2t51N9dv8bR8+anrrSDQMbHeaYIgnNz570Lle3+DV\nFM19LfzpwDM8uP0RXi9/F5fXPegYIYT4soa8s3/sscdYv349t99+OxqNhoaGBn75y18OR2ziNH22\nwD/8fXZJBErznfzf6e6ccBPVPbX+Z/IGjZ4NNRtps3UAEKmNIEEfxy+2/ZYeZy+z0ktYNeYqnjzw\nN9rtnUB/jX2tWsvV+VcM62cSQoSfIe/sU1JSuOuuu8jLywNg5cqVjBs37qIHJgZbUJwe0F44WXYj\nVFqkNpK5pw3bdzt6+cP+p3iv6iMMGgMx+mjeOvE+7bb+JJ4Zk869k+/h9Yp36bB34vK62NKwg/eq\nPvIn+i8c7Tw+bJ9FCBG+hryzF8HjpiUFZCRHU1HfzZhR8ZLsg8SNhddRaCyg0dJMRXcV5eb+xysu\nr4t15W8Rq4tmY90W//ENlib2tx3E7fMEvE+LtY1IbQQ2t93flxEtczKEEF+dzLYLIWq1ikVTMvnm\nygksmpKJWq0a+iRx0alVaqalTGZl3mXYT0nUAE6Pk5reukHneLwetCpNQN+YhFxuG38DMbpoAHLi\nsmQIXwhxQcidvRAX0BRTEXW9Df52enQaM9Km8WHNxoAJedNSixlrHMMb5e/S5ewhK3YUE4yFpMek\nUpQ0nj63VWrmC3GBOB1uDu1toMdsI7cwmZwxyUOfFGY0q1evXq10EBeD1SorBsTwy4vPRqPSUG9p\nxOV143DbiTfEsWjUPKp7arG7HahVKnQaPYtHz6fZ2kpNTx1mRxdbG3eRE59FSpQJg0ZKUgtxobzz\nygGOH2qmvdVCxdFW4hMiSEqJUTqsCy46+ux/N2QYX4gLSK1So1VrsLptALh8bt6sXI/X56Pd1okX\nLx6fl031W9lUv5Vtjbv853p8Hj6q+adCkQsRnswdVprquwP6jh1oUiga5UiyF+ICa+prGdR3tGPw\nrPq63oZBy/Y8p03aE0J8NQaDBtVp05sMkSNv51ZJ9kJcYBOSCgPaerWOWenTAna/g/4d8Kaaivxt\nFSoWj5o/LDEKMVJExRgonjna39YbNJTMHVzkKtwpPkHP7Xbz85//nNraWjweDz/+8Y+ZPn16wDET\nJ05k2rRp/vazzz6LRqM5/a2ECArTU6fQ4+xlW+MuonVRXJl7Gbnx2dw14SberdqAw+NkYeYcSlKL\nmWKaRFHL57RY2yhKnkBufJbS4Ysz8DoctL74PJb9e9GZUki55XYiCwqUDkucp4IJKTTUmuntspNX\nmILRFK10SMNuyC1uL7bXXnuNQ4cOsXr1asrLy/mv//ovXn311YBjZs2axc6dO7/U+4brFrciOFV1\n11LTU8eYhFxGxUr9g3DTtu4VzB9+4G9rYmPJffR3qHUjbzg41Hg8Xl56agd9vQOTtkvmZjNzYa6C\nUV0c59riVvE7+6uvvpqVK1cCYDQa6erqUjgiIb6cj2s38UbFe0D/UPyt41YxJ2MGRzuOU9pZTmZM\nOjPSpqJWnfmpWYet019pTwQnW3lZQNvT24uruRnD6NFnOUMEi64Oa0CiB6ivNodlsj8XxZO97pQr\n4+eee86f+E/ldDp54IEHaGho4PLLL+fuu+8ezhCFOCuvz8sH1Z/42z58vF/9CS6vm7Vlb/j7K7ur\nuGXcqoBz7W4HTx96nlJzOWqVmqVZl3BN/vJhi12cv4i8fOwnTvjb6uhodKmpCkYkzldcQgQ6vQaX\nc2Dya1LKyLuwHtZkv27dOtatWxfQd//997NgwQJeeukljhw5wlNPPTXovB//+MdcffXVqFQqbrvt\nNqZPn05RUdGg406VmGc7piIAABlFSURBVBiFVivP9cXF5fV6B+1e51V52NK0PaBvR9Me7pl1I0db\ny3C4nZRkFrGtYjul5vL+c3xeNtRsZGnhHHIS5W4x2CR+43Yq+nrp2LmLiNQU8r/zLRIyk5QOS5yn\na2+eyvrXDtJncZIzJonl1xURHTOyalko/swe+i8CPvjgA/70pz9hMJz7C3j00UfJz8/n+uuvP+dx\n8sxeDJc3Kt7j49pN/va1+SvY2/I5dafsX69T68iITvOXzk0wxJMfn8Pe1gMB7/WNibdSklo8PIGL\nL83n9aJSyyKmUOT1enE5PRgiwneexbme2Sv+U1tXV8crr7zC//3f/50x0Z84cYIHHngAn8+H2+1m\n3759FMgsWBFErs1fwTcn3c6yrEX8++S7WZa9iCtyLg14Rj85eUJAjfwuR/eg9zFo9BQmjqGyq5qP\nazdR0zO4pr5QliT60KVWq8M60Q9F8Wf269ato6uri29961v+vjVr1vDss88yY8YMpk6dSlpaGqtW\nrUKtVrNkyRImT56sYMRCBFKpVExNKWJqysCjpSkpRfwo4j52Nu2h0FiA2+sadBcfp4+lJKWYMnMl\niREJ3DD2GrY37ebNyvX+Y24q/BoLMmcP22cRQoSnoBjGvxhkGF8oqaKriicP/A27x45apWZVwVV8\nXPsZnXYz0F9oZ3RsJpXd1f5zbhv3dV6reCdgi9tEQwIPzfvZcIcvhAhBQb30Tohw9Fbl+9g9/Unb\n6/PyzokN/GzmD9jauBO728Hk5Ik8/vlfAs7Z0rBz8GQ/KZ8rhLgAJNkLcRH0OHoC2na3nX/WbWFT\n/TY8Pi8dtk7UKnVAco/URbB49IKApXxLshYOW8xCiPAlyT6EdPbYeeHD41Q0dDMmM547rhhHYuzI\nWj4SKmamTWN99cf+dl58Np/Ubfa3D3UcY1xigX/pnV6tY3nOUvITcsiJG011dy1jEvL+//buPSiq\n++7j+Hu5BwEFBNSoUdSgouINidZbndr4RIJWXMEL0jRtvBCNbSZNmrFGrE7M02TaUaNtrCYpVWxJ\nNAZs1OrESlJFEEWjEptEmxgjQgS5KrDs84ePWzfgLZFdOPt5zTjjufI9HtnPnt855/ejT/CDDq9d\nRIxH9+xbkVe2HOHE2VLbdGS3QJ5OHOTEiuRmGqwNfPBlDoWXTnO/X0f8vf3568fb7NYZff8Ihncc\nSlF1MRFBPQnwuvn9NhGR29E9e4Mo/LzsltPScriZ3BjdeTijOw8HoPRKGW+7ZVLfUG9bp1/73nQN\n6EzXgM7OKlNEXIReGm1FunXwv+W0tFyBPu2YO+DH9GjbnS5+nZjRO57I4N7OLktEXISa8VuR8yVV\nvPbuCT6/WEnXUD+eiIukU3vX6+NZREQau1UzvsK+Faqts+DlqX7/RUTkv3TP3mAU9CKOV55zkMr8\nPDxDQgma8Ajufn7OLknkjinsRURu4/L+f1L059dt09WFp3hg8QtOrEjk7ugBPRGR2yg/8KHd9NWz\nZ7h6/vxN1hZpeRT2IiK34d627TdmuKsZX1oVhb2IyG0EPzoJd///f/jJZCI4Ng6PgADnFiVyF/Q0\nvojIHWi4epWaT/6NZ/sQvMLCnF2OSCN69U5ERMTgbhX2asYXERExOL16J+IgJTVfs+/ch1ytr2VE\np2F0b9vV2SXJXbJUVuLm44PJQx+d0rrof6yIA9TU1/By3qtU1FUCkHPhMM8MfZIu/vc7uTK5E/Xl\n5Xz1h1epOf0x7v7+hM6ajf+QaGeXJXLH1Iwv4gAflRTagh7AYrVw6EK+EyuSu/H1O1upOf0xAJaK\nCope30DDlStOrkrkzinsRRygjadvE/M0iFFrcfXcF3bTDVeuUFdc7KRqRO6ewl7EAXoH9aJvcIRt\nOsw3hJGdYpxYkdwN38h+dtMeQUF4derkpGpE7p5evRNxoE/LznLVcpWIwJ64u2lAo9bCWl9Pyda3\nqMw/jGdoKCHmRLy7dHF2WSJ29J69iIiIwek9exERERemsBcRETE4hb2IiIjBKexFmlF5bQWFl/5N\nTb3eyRYR51EPeiLN5NCFfDadyqDeasHH3Zu5A35Mr8Aezi5LRFyQ06/st27dypgxY0hKSiIpKYl1\n69Y1Wufdd98lPj4es9lMRkaGE6oUuTsN1gbe/ncm9VYLAFcsV9n26d+dXJWIuKoWcWX/yCOP8Oyz\nzza5rLq6mldffZW33noLT09Ppk6dyvjx42nXrp2DqxS5c3UN9VTVVdvNu3y1nHMV56ltqKVbQFfc\nTE7/ri0iLqLFf9oUFBTQv39//P398fHxYfDgweTnq09xadm83b2ICom0m+fl7sWLub/nlcNr+d/c\nVVTX1TipOhFxNS0i7A8dOsTjjz9OcnIyJ0+etFtWUlJCUFCQbTooKIhi9UktrcDsvon8T7cf0C+4\nD6M6PcTF6v/+v/2i8jwfns9xYnUi4koc2oyfkZHR6J77xIkTWbBgAWPHjuXIkSM8++yzZGZm3nQf\nd9rhX2CgLx4e6o5UnOuxDvEA7D+bQ/b5g3bLat1qbtnjlYjIveLQsDebzZjN5psuHzRoEJcuXcJi\nseDufi2oQ0NDKSkpsa1z8eJFBg4ceNufVVpafdt1RBylq1c3fD3uo7r+WtO9m8mNvgGR6tZZRO6Z\nFt1d7vr168nKygLg9OnTBAUF2YIeICoqiuPHj1NeXk5VVRX5+fkMHTrUWeWKfCttPH15esh8RnSM\nZkhoFAsG/pTubbs6uywRcRFOHwjnwoULPPPMM1itVurr63n++ecZMGAAr732GtHR0QwaNIidO3ey\nYcMGTCYTs2bNIi4u7rb71RWTiIi4Eo16JyIiYnAtuhlfREREmleL6FRHxBUUV3/N++eyuWqp5Xud\nYghv+4CzSxIRF6GwF3GA6roaXj68hsq6KuBav/nPDHmSrgGdnVyZiLgCNeOLOMBHX5+yBT1c6zs/\nt+iIEysSEVeisBdxAH9Pv0bz/DzbOKESEXFFCnsRB4gI6km/4D626Q5twhh5/0NOrEhEXIlevRNx\noDOX/8NVSy292oXj7qbunEXk3rnVq3d6QE/EgbrrCXwRcQI144uIiBicwl5ERMTgFPYiIiIGp7AX\nERExOIW9iIiIwelpfBEHKam5xL5zH1BrqWVEp2F0C9B49iLiGAp7EQeoqb/WN35FbSUAB786zDND\nn6SL//1OrkxEXIGa8UUc4KOSQlvQA1isFg5dyHdiRSLiShT2Ig7QxtO3iXnqG19EHENhL+IAvYN6\n0TcowjYd5hvCyE4xTqxIRFyJ+sYXcaBPys5w1VJL78Ce6htfRO4p9Y0v0kL0bNfd2SWIiAtSM76I\niIjBKexFREQMTmEvIiJicAp7ERERg1PYi4iIGJzCXkRExOAU9iIiIgansBcRETE4hb2IiIjBOb0H\nvXXr1vGvf/0LgIaGBkpKSti1a5dt+blz53j00Ufp168fAIGBgaxatcoptYqIiLRGTg/7efPmMW/e\nPAC2bdvG119/3Wid7t27k5aW5ujSREREDKHFNOPX19eTnp7OrFmznF2KiIiIoTj9yv663bt3M3Lk\nSHx8fBotKykpYeHChVy8eJEZM2YQFxd32/0FBvri4aFRxURERBwa9hkZGWRkZNjNW7BgAaNGjeLt\nt98mNTW10Tbt2rXjqaeeIi4ujoqKCsxmMw899BChoaG3/FmlpdX3tHYREZGWrMUMcWs2mzGbzY3m\nV1dXc+HCBTp37txomZ+fH/Hx8QAEBQXRr18/Pvvss9uGvYiIiFzTIu7ZFxYWEh4e3uSygwcP8uKL\nLwLXvhQUFhbSvbvGBBcREblTLSLsi4uLCQoKspu3YsUKvvjiC4YOHcrly5dJSEhg9uzZPPHEE4SF\nhTmpUhERkdbHZLVarc4uojkUF1c4uwQRERGHudU9+xZxZS8iIiLNR2EvIiJicAp7ERERg1PYi4iI\nGJzCXkRExOAU9iIiIgansBcRETE4hb2IiIjBKexFREQMTmEvIiJicAp7ERERg1PYi4iIGJzCXkRE\nxOAU9iIiIgansBcRETE4hb2IiIjBKexFREQMTmEvIiJicAp7ERERg1PYi4iIGJzCXkRExOAU9iIi\nIgansBcRETE4hb2IiIjBKexFREQMTmEvIiJicAp7ERERg1PYi4iIGJzDw/7QoUMMHz6c999/3zav\nsLCQxMREEhMTeeGFFxptU1dXx9NPP8306dOZNWsWX3zxhSNLFhERadUcGvaff/45r7/+OoMHD7ab\nv2LFCp5//nm2bNlCZWUl//znP+2WZ2VlERAQQHp6OnPnzuWVV15xZNkiIiKtmkPDPiQkhDVr1uDv\n72+bV1tby5dffsmAAQMA+P73v8+BAwfstjtw4ADjx48HYMSIEeTn5zuuaBERkVbOoWF/33334e7u\nbjevtLSUgIAA23RwcDDFxcV265SUlBAUFASAm5sbJpOJ2tra5i9YRETEADyaa8cZGRlkZGTYzVuw\nYAGjRo265XZWq/W2+76TdUJC/G+7joiIiCtotrA3m82YzebbrhcUFERZWZltuqioiNDQULt1QkND\nKS4upnfv3tTV1WG1WvHy8rrnNYuIiBiR01+98/T0JDw8nLy8PAB2797d6Or/e9/7Hjt37gTg/fff\nJyYmxuF1ioiItFYm6520id8j+/btY8OGDXz22WcEBQUREhLCxo0b+eSTT1iyZAkNDQ1ERUXxq1/9\nCoB58+axbt06LBYLixcv5uzZs3h5ebFy5Uo6duzoqLJFRERaNYeGvYiIiDie05vxRUREpHkp7EVE\nRAxOYX+PZWVlERkZyaVLl+54m5s9cJiXl8fLL7/M6tWrmTBhgt2y06dPExERQU5Ozl3Vl5SUxOnT\np9m6dSv/+Mc/7mpbublz584xaNAgkpKSSEpKIiEhgby8PJ577jm7rqHvxLhx46iqqmqmSl3TuXPn\nmDJlit281atXs3btWpYsWdJsP3f//v1s3ry52fbvyh599FE+//xz2/Qjjzxi1/tqSkoKCQkJnD59\n2m67U6dOsWrVKgD27t3rMn22NNurd64qKyuLLl26sGvXLqZPn/6d9pWTk8OwYcMoKCigvr6ekydP\n0rdvXwB27NhBly5dvvW+v/nBJ99d9+7dSUtLAyA3N5d169YREhLi5KrkVgICApg/f36z7X/06NHN\ntm9XFxMTQ25uLl27duXSpUvU1NSQm5vLmDFjACgoKCAsLKzRdn369KFPnz4AvPHGGzz00EMu8Sq3\nruzvobKyMo4dO8Zzzz3Hjh07gGtX0suWLWPZsmVcuHDBduU3ffp0u2+ly5cvJzExkZSUFOrq6gDI\nz8+3jSMwZswYMjMzbetnZ2cTFRUFXPtSsHDhQtuy6y0F77zzDlOnTmX69Omkpqba1bp69Wr+8pe/\nkJOTQ0pKCgsXLuQHP/gBb731Fk899RQ//OEP2bNnTzP8K7mGkpISu/4iKisrmTNnDklJSZjNZo4d\nOwbAhx9+SHx8PNOmTeONN96w28dXX33FlClTuHjxoiNLdznXv/i+9tprmM1mEhIS+MMf/gBca2VZ\ntWoVM2bMIDk5mfLy8puey/Hjx/OnP/2JmTNnYjabqaysZOvWrbz00ksArF+/nqlTpzJt2jQOHjzo\nnIM1kJiYGNsr2/n5+cTFxXH06FEAPv30Uzp37oyvry/vvfceycnJTJo0ifPnz9s+L9955x2OHj3K\nz372M2pra9m0aROJiYnMmDGDjRs3OvPQmoXC/h7auXMnY8eOZdSoUZw9e5aioiIAevXqxZIlS7h4\n8SIpKSmkpaURHx9va94rKysjNjaWLVu24O7uTnZ2NrW1tdTW1uLn5wdcu0LYt28fVquV48ePEx4e\njqen5y3r2bBhA6tXryY9PZ1+/fpx5cqVJtc7deoUL7/8Mqmpqbzyyiu8+OKLpKamsnXr1nv4r2N8\nZ86cISkpiWnTprFy5Uoef/xx27Li4mLMZjNpaWn84he/YP369VitVlJTU1m/fj3p6ekcOHDAdo6u\nXr3KL3/5S5YvX96okyn5dq6fn+t/tm3bZrd848aNpKens2XLFrsuvHv06MHmzZvp06cP27Zta/Jc\nAlgsFsLDw9m0aROdO3e2C/SzZ8+ya9cu/va3v/Hb3/7W7ou7fDvR0dEcPnwYuHbLc8SIEVgsFq5c\nuUJubq7toic4OJg333yT0aNHs3v3btv2kydPJiQkhPXr11NUVMTOnTtJT09n06ZN7N69m/Pnzzvl\nuJqLmvHvoaysLObPn4+7uzsTJkzg73//O4BtkJ+QkBCWL1/O6tWrKS8vJzIyEgBvb28GDhwIQP/+\n/Tlz5gz+/v7079/ftm8fHx8efPBBDh8+zN69e5kwYcJtr7xjY2NJSUkhLi6O2NhYfHx8mlyvd+/e\neHl5ERISQrdu3fD19SU4OJiKiorv/G/iSm5sxv/0009ZtGgRERERALRv3561a9eyYcMGamtr8fX1\n5dKlS3h7e9vGffjjH/9o29fSpUsZN26c7baNfHc3nh+41rp1o4cffpjHHnuM2NhY4uLibPOHDx8O\nwMCBAzl48CBTpkxpdC6vGzp0KAAdOnSw+/05efIkUVFRuLm58cADD7BixYpmOUZX0q5dO3x9fSkq\nKqKgoIBFixYxYMAAjh49Sl5eHvHx8eTn5zNkyBAAwsLC7HprvdHx48f5z3/+w+zZswGoqqriyy+/\npFOnTg47nuamK/t75MKFCxQUFLBy5UomTZpEdna2rSn/+hX4qlWrGDlyJJs2bSIlJcW2rclkstuX\nyWQiJyen0YN7EyZM4L333iM7O9uul8Fvbl9fXw/AnDlzWLNmDVarleTkZEpLS5us3cPDo8m/y7fX\no0cPvL29bQM/vfnmm4SFhZGens7SpUuBa4M6NTQ0NLl9WFgY27dvd5mHh1qC1NRUli5dSnFxMUlJ\nSbbfo+tdkVitVkwmU5Pn8robB/q6sQsTd3f3m55r+fZiYmLIzs7GZDLh4+PDkCFDOHLkCMePH2fQ\noEHAzc/JjTw9PRk7dixpaWmkpaWRmZlJdHS0Q47BURT290hWVhYzZ87k3XffZfv27ezcuZPLly/b\n3ZcvLS2la9euWK1W9u7da7s3f+XKFT766CPg2kMlPXr0IC8vz/aN9LqxY8eyZ88eevbsibe3t22+\nn5+f7b5uYWEhVVVVNDQ08Lvf/Y6QkBAee+wxBg4caLhmqZasrKyM4uJiW2BcP/cAe/bsoa6ujsDA\nQCwWC0VFRVitVubMmUN5eTkAixYtYty4cbz66qtOOwZXUlFRwZo1a+jRowdPPvkkbdu2pbKyEsB2\nX/jo0aP07NmzyXN5O5GRkeTn51NfX09JSYndl3359mJiYvjrX/9qaxkdMmQI+/btIyQk5KYtmTcy\nmUxYLBYiIyPJycmhpqYGq9XK8uXLb3rbs7VS2N8jO3bssHvC3WQyMXnyZEpKSmzzEhIS+M1vfsNP\nf/pTJk6cyKFDh/jggw8IDQ0lMzOTmTNn4ubmRnR0tN39+uvuu+8+oqKiePjhh+3m9+7dG19fXxIT\nE9m+fTv3338/bm5utGnThoSEBJKTkzGZTLYnUKV53HhP+IknnuDXv/61rVVn0qRJvP766/zkJz9h\nwIABFBcX8/bbb/PCCy+wcOFCEhMTGT58uN294rlz57J//37bF0FpPv7+/pSWljJ16lRmz55NVFQU\n7dq1A+DEiRMkJyfz8ccfM2nSpJuey1vp3LkzkyZNYtasWaSkpJCUlOSIwzK86OhoTpw4YbswCg4O\npqys7I7HTxk2bBgzZszAx8eH2bNnM3PmTKZNm3bHXxZaE3WXKyJyE+PGjSMzM5M2bdo4uxSR70RX\n9iIiIganK3sRERGD05W9iIiIwSnsRUREDE5hLyIiYnAKexEREYNT2IuIiBic+kYVkSZZrVaWLVtG\nQUEB7du3p0OHDgQGBpKWlsbUqVNpaGhg8eLFrF27ln379uHh4UGvXr1YvHgxRUVFzJgxg/379wPX\n+qGvr6/n5z//OX379mX+/Pnk5ORQVVXFypUrefDBB518tCLGpit7EWnSgQMHOHbsGBkZGfz+97+3\njeJWXV3NmDFjWLx4MUeOHGH37t1s2rSJzZs3U1paSlZW1i33a7FY6NWrF2lpaUyfPp1Vq1Y54nBE\nXJrCXkSadOrUKYYOHYq7uzu+vr62wZesViuDBw8Gro3lEB0dbesWeNiwYRw/fvy2+x45ciQAgwcP\n5pNPPmmmIxCR6xT2ItKkhoYG3Nz++xFx49+vh/s3R1y8PjLcN+d/c7CYG/vy+ua6InLvKexFpEnh\n4eEcPXoUq9VKTU0NH3zwQaN1Bg4cSE5Oji3MDxw4QFRUFH5+fly+fJmamhosFgu5ubl2212/JXD4\n8GEiIiKa/2BEXJwe0BORJo0ZM4YdO3YQHx9Px44dGTRoEB4e9h8ZUVFRTJw40TZiY2RkJLGxsbi5\nufGjH/2I+Ph4unbtSt++fe22O3nyJOnp6Vy+fJmXXnrJkYcl4pLUN76INKmiooI9e/YwefJkTCYT\nc+fOJTY2ltjY2O+034iICE6cONHoi4OINB/9tolIk9q0aUN+fj5//vOf8fb2pnv37kyYMMHZZYnI\nt6ArexEREYPTA3oiIiIGp7AXERExOIW9iIiIwSnsRUREDE5hLyIiYnAKexEREYP7P1pxxWuKh0Rw\nAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 576x396 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "ExecuteTime": {
          "end_time": "2019-03-29T15:11:09.382514Z",
          "start_time": "2019-03-29T15:11:09.222382Z"
        },
        "id": "C_C11E9xBze6",
        "colab_type": "code",
        "outputId": "7552e102-646d-4f26-cc89-349965661d2f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 396
        }
      },
      "cell_type": "code",
      "source": [
        "plot = seaborn.barplot(x='group', y='sentiment', data=name_sentiments, capsize=.1)"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/seaborn/categorical.py:1428: FutureWarning: remove_na is deprecated and is a private function. Do not use.\n",
            "  stat_data = remove_na(group_data)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe0AAAFYCAYAAAB+s6Q9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XlY1WX+//HXOQcUERBS1KwsRTPT\nRC1wbDIdB8lRcskFl8jJXLpkcrTfTDmNmZYzaftXzJo0tcitzEpxQ/1m0mSuSeaSjVfauCEmKCjK\ndn5/+PXIUcGPyzmHG56P6+K6/Ozvw/HwOvf9WW6b0+l0CgAAlHt2XxcAAACsIbQBADAEoQ0AgCEI\nbQAADEFoAwBgCEIbAABD+Pm6gCvJzMzxdQkAAHhNeHhwqctoaQMAYAhCGwAAQxDaAAAYgtAGAMAQ\nhDYAAIYgtAEAMAShDQCAIQhtAAAMQWgDAGAIQhsAAEOU+8eYAgAqrzNn8lRQUGBpXX9/fwUEVPNw\nRb5FaAMAyqWZM9/TqlXL5XQ6La1vs9nVqVNnDR48zMOV+Q7d4wCAcik1dZnlwJYkp7NYqanLPFiR\n7xHaAIByKTa2i2w26zFls9kVG9vFgxX5ns15NV9jfIChOQGg8ip5Tjs3N0ejRye6lr355tsKCrow\njGVFOadd1tCcnNMGAJRbAQHVSg3ioKBgBQeHeLki36J7HAAAQxDaAAAYwqvd43l5eRozZox+/fVX\nnT17ViNGjNDvfvc7b5YAAICxvBraX375pZo3b66hQ4fq4MGDGjx4MKENAIBFXg3tLl0uXIp/+PBh\n1alTx5uHBwDAaD65erxfv346cuSI3n333SuuGxYWKD8/hxeqAgCUZ1WqFLtN16wZpBo1Sr89qiLy\nSWjPnz9fu3bt0l//+lctXrxYNput1HWzsk57sTIAQHmVk5PrNv3rr7nKz69411OXdZ+2V1/tDz/8\noMOHD0uSmjZtqqKiIh0/ftybJQAAYCyvhvbmzZs1c+ZMSdKxY8d0+vRphYWFebMEAACM5dXQ7tev\nn44fP64BAwZo2LBhGjdunOz2ite1AQCAJ3j1nHZAQIBef/11bx4SAIAKg2YuAACGILQBADAEoQ0A\ngCEIbQAADEFoAwBgCEIbAABDENoAABiC0AYAwBCENgAAhiC0AQAwBKENAIAhCG0AAAxBaAMAYAhC\nGwAAQxDaAAAYgtAGAMAQhDYAAIYgtAEAMAShDQCAIQhtAAAMQWgDAGAIP18XAADwrelvrfB1CZYU\nFJxxm05+93/l7x/go2quztBRnW/IfmhpAwBgCEIbAABDENoAABiC0AYAwBCENgAAhiC0AQAwBKEN\nAIAhCG0AAAxBaAMAYAhCGwAAQxDaAAAYgmePA6jQzpzJU0FBgaV1/f39FRBQzcMVAdeO0AZQYc2c\n+Z5WrVoup9NpaX2bza5OnTpr8OBhHq4MuDZ0jwOosFJTl1kObElyOouVmrrMgxUB14eWtpfQRQd4\nX2xsF61atUJOZ7Gl9c+3tIHyitD2ArroAN8YPHiYBgxIcPvCnJubo9GjE13Tb775toKCgiXxhRnl\nH93jXkAXHeA7AQHVFBwc4vo5H9DnBQUFu5YR2CjvCG0viI3tIpvN+q/aZrMrNraLBysCAJiI7nEv\noIsOAHAjENpeEhBQrcwgPt9FBwBAaegeBwDAEIQ2AACGILQBADAEoQ0AgCEIbQAADEFoAwBgCJ/c\n8vXKK69oy5YtKiws1PDhwxUbG+uLMgAAMIrXQ/vbb7/VTz/9pAULFigrK0s9e/YktAEAsMDroR0V\nFaUWLVpIkkJCQpSXl6eioiI5HA5vlwIAgFG8HtoOh0OBgYGSpIULF+rBBx8sM7DDwgLl51fxAr1K\nFfehAmvWDFKNGsGlrA3gRuGzB18ID78x/8d89hjT1atXa+HChZo5c2aZ62VlnfZSRd6Vk5PrNv3r\nr7nKz+e6QMDT+OzBFzIzcyyvW1bA+yS009LS9O6772rGjBkKDuYbLgDg8gqLCuQsLpIkFRSecVt2\n8bTN7pCfw99rtfmC10M7JydHr7zyimbPnq3Q0FBvHx4AYIgff07TgYwdpS5fnz7/ojk23VrnbjVp\n0M6zhfmQ10N72bJlysrK0qhRo1zzJk+erHr16nm7FABAOVZWYF+eUwcydhDaN1J8fLzi4+O9fVgA\ngGFurdNMBzJ2SnJa3OJcS7siYzxtAEC51KRBO0XU/43rnPaVcE4bAAAf8nP4SxU8iK8G9zkAAGAI\nQhsAAEMQ2gAAGILQBgDAEIQ2AACGILQBADAEoQ0AgCEIbQAADEFoAwBgCEIbAABDENoAABiC0AYA\nwBCENgAAhiC0AQAwBKENAIAhCG0AAAxBaAMAYAhCGwAAQxDaAAAYgtAGAMAQhDYAAIbw83UBN9Kf\nX13s6xIsKy484zb93NQVsvsF+Kga6/7nr918XQIAVFq0tAEAMITllvaxY8d06NAhSVK9evVUq1Yt\njxUFAAAudcXQXrZsmd577z1lZmaqbt26kqTDhw+rTp06GjZsmP7whz94vEgAAHCF0B4zZowKCws1\nadIk3XXXXW7Ldu/erRkzZuirr77SpEmTPFokAAC4QmjHxMQoJiZGJ0+evGRZ9erV9dprr2n16tUe\nKw4AAFxwxdAuLi5WYmKiPvzwQzmdTklSYWGhRowYoSVLligmJsYrhQIovzb9v5G+LsGyvKIit+nv\nnv+bqjkcPqrGuqjXp/i6BJQDZYZ2SkqKkpKStH//fjVt2tQ1326364EHHvB4cQAA4IIyQzsuLk5x\ncXFKSkrSU0895a2aAADAZVi65WvYsGFavXq1Tpw44eoil6TevXt7rDAAAODOUmgPGTJENptNt9xy\ni9t8QhsAAO+xFNoFBQWaP3++p2sBAABlsPQY00aNGikrK8vTtQAAgDJYamkfOXJEsbGxioiIkKPE\nrRFz5szxWGEAAMCd5QvRAACAb1nqHo+Ojtbp06e1Z88eRUdHq27duoqKivJ0bQAAoARLof3qq69q\n4cKFWrRokSRpyZIlmjhxokcLAwAA7iyF9qZNmzR16lRVr15dkpSYmKgdO3Z4tDAAAODOUmhXrVpV\nkmSz2SRJRUVFKrro+b0AAMCzLF2I1rp1a/3tb3/T0aNHNWvWLKWmpio6OtrTtQEAgBIshfbo0aO1\nYsUKBQQE6MiRI3r88ccVGxvr6doAAEAJlkJbkh544AFFRka6nj1+6NAh1atXz2OFAQAAd5ZCe/z4\n8frss88UFhYmSXI6nbLZbFq7dq0nawMAACVYCu0tW7Zo48aNrgvSAACA91m6erxJkyYqKCjwdC0A\nAKAMllraHTt2VExMzCXPHv/www+v6aB79uzRiBEj9Mc//lGPPvroNe0DAIDKxlJov/7663r22WdV\nt27d6z7g6dOn9dJLL6lt27bXvS8AACoTS6HdqFEj9ezZ84YcsEqVKpo+fbqmT59+Q/YHAEBlYSm0\nGzZsqGeffVatW7d26x7v3bv31R/Qz09+fpbvNFNYWKD8/BxXXhFeER4e7OsSgEqJz57ZbtT7Zyk9\ns7OzZbfbtW3bNrf51xLaVysr67THjwHrMjNzfF0CUCnx2TPb1bx/ZQW8pdB++eWXLR8MAAB4Rpmh\nPWrUKL311ltq3769a7CQkni4CgAA3lNmaI8dO1aSNHfu3EuW5eXlXdMBf/jhB02ePFkHDx6Un5+f\nVq5cqaSkJIWGhl7T/gAAqCzKDO1atWpJksaNG6f333/fbVmvXr306aefXvUBmzdvruTk5KveDgCA\nyq7M0F68eLHefvttHTp0SB06dHDNLygocAU6AADwjjJDu1u3buratav+/ve/66mnnnLNt9vtql27\ntseLAwAAF1zx6nGHw6FJkyZp9+7dys7Odg3NuW/fPp5qBgCAF1m65WvkyJHatWuX22NMbTYboQ0A\ngBdZCu0DBw5o1apVnq4FAACUwdLQnA0aNFB+fr6nawEAAGWw1NK22+3q2rWrWrRo4fbs8VdeecVj\nhQEAAHeWQvv+++/X/fff7+laAABAGSyFds+ePbVnzx798ssviomJ0cmTJxUSEuLp2iqU4qICyVl0\nYbrwrPvyktM2h+wOf2+VBgAwhKXQnj17tlJSUpSfn6+YmBhNmzZNISEhGjFihKfrqxBO/rJeeZm7\nylzn1x0lny5nU7XwuxRSn6vzAQAXWLoQLSUlRR9//LFq1KghSXrmmWcYLOQqXCmwL+W8hm0AABWd\npdCuXr267PYLq9rtdrdplK1aeFNJl46SVjrb/20DAMAFlrrH69evr6lTp+rkyZNKTU3VsmXLFBER\n4enaKoyQ+m0VdMt9bue0y8Q5bQDAZVhqLo8bN07VqlVTnTp1tHjxYkVGRuqFF17wdG0Vit3hL7tf\ngLUfAhsAcBmWWtr+/v6Kj4/XE088oczMTO3fv19+fpY2BQAAN4illvZLL72k5cuXKzs7WwMGDNBH\nH32k8ePHe7g0AABQkqXQ3rlzp/r06aPly5erR48eeuutt7R//35P1wYAAEqwFNrnh+Ncu3atOnbs\nKEk8ixwAAC+zdGK6QYMG6tq1q8LCwtS0aVN9/vnnrnu2AaA8yy8uVtH/NTwk6Uyx+10cJacdNpuq\ncDsryrEyQ7ugoED+/v6aOHGi9uzZ47rNq1GjRq7BQs6vAwDlzZrs49p2KrfMdWYePeL6t01SZPUg\n/T70Jg9XBlybMr9SDhkyRD///LMcDoeaNm2qKlWqSJKaN2+ukJAQ7d27V0OGDPFKoQBwta4U2Bdz\nXsM2gDeV2dIeO3asnn76adWtW1ft2rXTzTffLEk6fPiw0tLSlJGRocmTJ3ulUAC4Wi2rByn9VK6c\nV15V0oWWNlBelRnajRs31qJFi7RmzRqtW7fO9bzxunXrqlevXvr9738vm+1qHs8JAN7z+9Cb1C4k\n1O2cdlk4p43y7ooXotlsNsXExCgmJsYb9QDADUUIoyKxdPV4SkqKZsyYoRMnTrhu/5LESF8AAHiR\npdBOSkrSxIkTVa9ePU/XAwAASmEptG+//XZFRUV5uhYAAFAGS6HdqlUrvfHGG4qOjpbD4XDNb9u2\nrccKAwAA7iyF9jfffCNJ+u6771zzbDYboQ0AgBdZCu3k5ORL5q1cufKGFwMAAEpnKbQPHTqkjz76\nSFlZWZLODRayYcMGPfTQQx4tDgAAXGDpBsZnnnlGoaGh2rZtm5o3b66srCzXs8cBAIB3WApth8Oh\nYcOGqVatWho4cKDeeecdzZkzx9O1AQCAEiyF9tmzZ3XkyBHZbDb997//lZ+fnw4ePOjp2gAAQAmW\nzmkPGTJE33zzjZ544gl1795dDodDcXFxnq4NAACUYCm0Sz53fOPGjTp16pRq1KjhsaIAAMClLIX2\nwYMHNXnyZGVlZSk5OVmpqamKiorSHXfc4eHygPLhzJk8FRQUWFrX399fAQHVPFwRgMrIUmg///zz\nGjhwoGbNmiVJuuOOO/T8889f9v5toKKZOfM9rVq13G2wnLLYbHZ16tRZgwcP83BlACobSxeiFRQU\nuI2dzXPIUZmkpi6zHNiS5HQWKzV1mQcrAlBZWR5o9uTJk67Q/umnn3T27FmPFQWUJ7GxXWSzWR+T\n2WazKza2iwcrAlBZWeoeT0xMVN++fZWZmamHH35YWVlZevXVVz1dG1AuDB48TAMGJLjOaefm5mj0\n6ETX8jfffFtBQcGuac5pA/AUS6HdoEED9ezZUwUFBdq9e7fat2+vLVu2MGAIKo2AgGqlBnFQULCC\ng0O8XBGAyshSn9/QoUO1b98+FRYWqlGjRvLz81NhYaGnawMAACVYammHhobq5Zdf9nQtAACgDJZC\nu1OnTlq8eLFatWolh8Phml+vXj2PFQYAANxZCu0ff/xRS5YsUWhoqGuezWbT2rVrPVUXAAC4iKXQ\nTk9P16ZNm1SlShVP1wMAAEph6UK05s2bc182AAA+ZqmlnZGRoY4dOyoiIsLtnPa1jKn9z3/+U+np\n6bLZbHruuefUokWLq94HAACVkaXQfvLJJ2/IwTZu3Kj9+/drwYIF2rt3r5577jktWLDghuwbAICK\nzlJoR0dH35CDrV+/3jXMZ0REhE6cOKHc3FwFBQXdkP0DAFCRWX+g8g1w7NgxhYWFuaZvuukmZWZm\nerMEAACMZaml7SlWRk4KCwuUn5/jiutJ0txXBl5vSfCRP876s69LsKzorPvTAF9I/accVX36UbJs\n9uP/45H9dvlwlkf2C+947h99fF0CLPLqX5ratWvr2LFjrumjR48qPDy8zG2ysk57uiyg0sjMzPF1\nCQCuIDw8uNRlXu0e/+1vf6uVK1dKknbs2KHatWtzPhsAAIu82tJu3bq1mjVrpn79+slms+mFF17w\n5uEBADCa10/E/eUvf/H2IQEAqBC82j0OAACuHaENAIAhCG0AAAxBaAMAYAhCGwAAQxDaAAAYgtAG\nAMAQhDYAAIYgtAEAMAShDQCAIQhtAAAMQWgDAGAIQhsAAEMQ2gAAGILQBgDAEIQ2AACGILQBADAE\noQ0AgCEIbQAADEFoAwBgCEIbAABDENoAABiC0AYAwBCENgAAhiC0AQAwBKENAIAhCG0AAAxBaAMA\nYAhCGwAAQxDaAAAYgtAGAMAQhDYAAIYgtAEAMITN6XQ6fV1EWTIzc3xdAuAmJ+ekhg59zDU9ffqH\nCg4O8WFFACqS8PDgUpfR0gYAwBCENgAAhiC0AQAwBKENAIAhCG0AAAxBaAMAYAhCGwAAQxDaAAAY\ngtAGAMAQhDYAAIYgtAEAMAShDQCAIQhtAAAMQWgDAGAIr4f2xo0b1bZtW3355ZfePjQAAEbzamj/\n8ssvmjVrllq3bu3NwwIAUCF4NbTDw8M1depUBQeXPsA3AAC4PD9vHqxatWrePBwAABWKx0L7k08+\n0SeffOI276mnnlK7du2uaj9hYYHy83PcyNKA61KlSrHbdM2aQapRg94jAJ7nsdDu06eP+vTpc937\nyco6fQOqAW6cnJxct+lff81Vfj43YgC4McLDS28E8JcGAABDeDW0165dq4SEBKWlpemNN97Q4MGD\nvXl4AACM5tUL0Tp06KAOHTp485AAAFQYdI8DAGAIQhsAAEMQ2gAAGILQBgDAEIQ2AACGILQBADAE\noQ0AgCEIbQAADEFoAwBgCEIbAABDENoAABiC0AYAwBCENgAAhiC0AQAwBKENAIAhCG0AAAxBaAMA\nYAhCGwAAQxDaAAAYgtAGAMAQhDYAAIYgtAEAMAShDQCAIQhtAAAMQWgDAGAIQhsAAEMQ2gAAGMLP\n1wUAJjhzJk8FBQWSpNzcHLdlF0/7+/srIKCa12oDUHnYnE6n09dFlCUzM+fKKwEeNHPme1q1arms\nflRsNrs6deqswYOHebgyABVReHhwqcvoHgeuIDV1meXAliSns1ipqcs8WBGAyorQBq4gNraLbDbr\nHxWbza7Y2C4erAhAZUX3OGBByXPaV8I5bQDXo6zucS5EAywICKhGEAPwObrHAQAwBKENAIAhCG0A\nAAxBaAMAYAhCGwAAQxDaAAAYgtAGAMAQhDYAAIYgtAEAMAShDQCAIcr9s8cBAMA5tLQBADAEoQ0A\ngCEIbQAADEFoAwBgCEIbAABDENoAABiC0C5FSkqKmjVrpuPHj1vepk2bNpedv3nzZr322mtKSkpS\n586d3Zbt2bNHTZo00YYNG66qvoSEBO3Zs0eLFi3SqlWrrmpblO7AgQNq1aqVEhISlJCQoPj4eG3e\nvFljxozRl19+eVX76tixo06dOuWhSiunAwcO6JFHHnGbl5SUpGnTpmncuHEeO+66des0d+5cj+2/\nMnv44Yf1yy+/uKa7dOmir776yjWdmJio+Ph47dmzx227Xbt2acqUKZKkNWvWKD8/3zsF+5ifrwso\nr1JSUnTbbbdp5cqV6t+//3Xta8OGDYqOjlZ6eroKCwu1c+dO3X333ZKkpUuX6rbbbrvmfV/8BwzX\nr0GDBkpOTpYkbdq0Se+8847Cw8N9XBXKEhISohEjRnhs/w8++KDH9l3ZtWnTRps2bVL9+vV1/Phx\n5eXladOmTWrfvr0kKT09XXXq1Llku6ZNm6pp06aSpNmzZ+s3v/mNqlSp4tXafYGW9mVkZ2fr+++/\n15gxY7R06VJJ51q2L774ol588UUdOXLE1RLr37+/27fEiRMnql+/fkpMTFRBQYEkaevWrWrdurUk\nqX379lqyZIlr/bS0NEVGRko6F+4jR450LTvfcv/888/Vu3dv9e/fXxMmTHCrNSkpSR999JE2bNig\nxMREjRw5UjExMVq4cKH+/Oc/KzY2VqtXr/bAb6lyOHbsmGrXru2azs3N1fDhw5WQkKA+ffro+++/\nlyT9+9//Vq9evdS3b1/Nnj3bbR+HDx/WI488oqNHj3qz9Ern/BfY9957T3369FF8fLzeffddSed6\nPaZMmaIBAwZo0KBBOnnyZKnvZadOnTRjxgwNHDhQffr0UW5urhYtWqTJkydLkqZPn67evXurb9++\n+vbbb33zYiuQNm3aaPPmzZLO/a3s1q2btm3bJknau3evbr31VgUGBmr58uUaNGiQunfvrkOHDrn+\nXn7++efatm2bhg4dqvz8fM2ZM0f9+vXTgAEDNHPmTF++NI8gtC9jxYoV6tChg9q1a6d9+/YpIyND\nktS4cWONGzdOR48eVWJiopKTk9WrVy9Xt1l2drbi4uI0f/58ORwOpaWlKT8/X/n5+QoKCpJ07hv7\n2rVr5XQ6tX37djVs2FD+/v5l1vP+++8rKSlJ8+bNU/PmzXXmzJnLrrdr1y699tprmjBhgl5//XW9\n/PLLmjBhghYtWnQDfzsV388//6yEhAT17dtXkyZN0hNPPOFalpmZqT59+ig5OVlPP/20pk+fLqfT\nqQkTJmj69OmaN2+e1q9f73qPzp49q2eeeUYTJ050C39cu/Pvz/mfzz77zG35zJkzNW/ePM2fP18h\nISGu+REREZo7d66aNm2qzz777LLvpSQVFRWpYcOGmjNnjm699Va3YN63b59Wrlypjz/+WK+++qrb\nF3Bcm6ioKG3ZskXSuVOJ999/v4qKinTmzBlt2rTJ1XipWbOmPvjgAz344INKTU11bd+jRw+Fh4dr\n+vTpysjI0IoVKzRv3jzNmTNHqampOnTokE9el6fQPX4ZKSkpGjFihBwOhzp37qxly5ZJklq0aCFJ\nCg8P18SJE5WUlKSTJ0+qWbNmkqSqVauqZcuWkqR77rlHP//8s4KDg3XPPfe49h0QEKA777xTW7Zs\n0Zo1a9S5c+crtoTj4uKUmJiobt26KS4uTgEBAZdd76677lKVKlUUHh6uO+64Q4GBgapZs6ZycnKu\n+3dSmZTsHt+7d69GjRqlJk2aSJJq1aqladOm6f3331d+fr4CAwN1/PhxVa1aVTfddJMk6V//+pdr\nX+PHj1fHjh1dp0Nw/Uq+P9K53qaSHnroIT3++OOKi4tTt27dXPPbtm0rSWrZsqW+/fZbPfLII5e8\nl+fdd999kqS6deu6fX527typyMhI2e123X777frHP/7hkddYmYSGhiowMFAZGRlKT0/XqFGj1KJF\nC23btk2bN29Wr169tHXrVt17772SpDp16ig7O/uy+9q+fbv279+vxx57TJJ06tQpHTx4UPXq1fPa\n6/E0WtoXOXLkiNLT0zVp0iR1795daWlpri7y8y3iKVOm6IEHHtCcOXOUmJjo2tZms7nty2azacOG\nDZdcoNa5c2ctX75caWlpateuXanbFxYWSpKGDx+uqVOnyul0atCgQcrKyrps7X5+fpf9N65dRESE\nqlatKofDIUn64IMPVKdOHc2bN0/jx4+XJNntdhUXF192+zp16uiLL76oNBfJlAcTJkzQ+PHjlZmZ\nqYSEBNfn6PwwC06nUzab7bLv5Xnn3++S252fX9p7jWvXpk0bpaWlyWazKSAgQPfee6++++47bd++\nXa1atZJU+ntSkr+/vzp06KDk5GQlJydryZIlioqK8spr8BZC+yIpKSkaOHCgFi9erC+++EIrVqzQ\niRMn3M5bZ2VlqX79+nI6nVqzZo3r3PWZM2f0ww8/SDp38URERIQ2b97s+oZ4XocOHbR69Wo1atRI\nVatWdc0PCgpynffcvXu3Tp06peLiYr355psKDw/X448/rpYtW1a47p7yLDs7W5mZma4//Offe0la\nvXq1CgoKFBYWpqKiImVkZMjpdGr48OE6efKkJGnUqFHq2LGj3n77bZ+9hsokJydHU6dOVUREhP70\npz+pRo0ays3NlSTXedNt27apUaNGl30vr6RZs2baunWrCgsLdezYMbcv7bh2bdq00YIFC1w9lffe\ne6/Wrl2r8PDwUnsWS7LZbCoqKlKzZs20YcMG5eXlyel0auLEiaWeTjQVoX2RpUuXul2RbbPZ1KNH\nDx07dsw1Lz4+Xi+99JKGDBmirl27auPGjfr6669Vu3ZtLVmyRAMHDpTdbldUVJTb+ezzqlWrpsjI\nSD300ENu8++66y4FBgaqX79++uKLL3TLLbfIbrerevXqio+P16BBg2Sz2VxXTMIzSp4zHTZsmJ5/\n/nlXL0v37t01a9YsDR48WC1atFBmZqY+/fRTvfDCCxo5cqT69euntm3bup1LffLJJ7Vu3TrXFzp4\nTnBwsLKystS7d2899thjioyMVGhoqCRpx44dGjRokH788Ud179691PeyLLfeequ6d++uRx99VImJ\niUpISPDGy6rwoqKitGPHDlcDp2bNmsrOzi71NtqLRUdHa8CAAQoICNBjjz2mgQMHqm/fvpZD3yQM\nzQmgwuvYsaOWLFmi6tWr+7oU4LrQ0gYAwBC0tAEAMAQtbQAADEFoAwBgCEIbAABDENoAABiC0AYA\nwBA86xKo4JxOp1588UWlp6erVq1aqlu3rsLCwpScnKzevXuruLhYY8eO1bRp07R27Vr5+fmpcePG\nGjt2rDIyMjRgwACtW7dO0rnnfBcWFmr06NG6++67NWLECG3YsEGnTp3SpEmTdOedd/r41QIVGy1t\noIJbv369vv/+e33yySd66623XKNWnT59Wu3bt9fYsWP13XffKTU1VXPmzNHcuXOVlZWllJSUMvdb\nVFSkxo0bKzk5Wf3799eUKVOtBN7aAAABhklEQVS88XKASo3QBiq4Xbt26b777pPD4VBgYKBrkBqn\n0+ka5z09PV1RUVGux7VGR0dr+/btV9z3Aw88IElq3bq1/vOf/3joFQA4j9AGKrji4mLZ7Rc+6iX/\nfT6kLx5h7vxIWBfPv3hQjZLPZrp4XQA3HqENVHANGzbUtm3b5HQ6lZeXp6+//vqSdVq2bKkNGza4\nQnn9+vWKjIxUUFCQTpw4oby8PBUVFWnTpk1u253vat+yZYtrzHEAnsOFaEAF1759ey1dulS9evXS\nzTffrFatWl0y3npkZKS6du3qGqGuWbNmiouLk91uV8+ePdWrVy/Vr19fd999t9t2O3fu1Lx583Ti\nxAlNnjzZmy8LqJR49jhQweXk5Gj16tXq0aOHbDabnnzyScXFxSkuLu669tukSRPt2LHjki8AADyH\nTxtQwVWvXl1bt27Vhx9+qKpVq6pBgwbq3Lmzr8sCcA1oaQMAYAguRAMAwBCENgAAhiC0AQAwBKEN\nAIAhCG0AAAxBaAMAYIj/D2w4WShSIGK2AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 576x396 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "Kmvb2yKmVXkI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Exercise\n",
        "\n",
        "We have used only word labels to build a sentiment classifier, but if we have sentence labels we can build a model that uses both, word and sentence information.\n",
        "\n",
        "To this end, you can use the `imdb` dataset (available in Keras): https://github.com/keras-team/keras/blob/master/examples/imdb_lstm.py \n",
        "\n",
        "Yours tasks are:\n",
        "+ Use the sentiment model we have already developed (based on words), to evaluate the `imdb` dataset. What is the accuracy of this classifier?\n",
        "+ Train a LSTM model to classify sentences based on the `imdb` dataset but, insteasd of learning specific embeddings for this problem, use `GloVe`.  What is the accuracy of this classifier?\n",
        "+ Build and train a hybrid model for taking into account word and sentence labels. What is the accuracy of this classifier when applied the `imdb` dataset ?\n",
        "+  Is there any racial bias in this new classifier? (Check this by classifiying neutral sentences that contain people names). "
      ]
    },
    {
      "metadata": {
        "id": "akmMQJ0kaePD",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# your code here"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}