LAB01_DataPreparation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "64ff0eca",
   "metadata": {},
   "source": [
    "# 2CSSID-TP01. Préparation de données\n",
    "\n",
    "---\n",
    "\n",
    "Dans ce TP, nous allons apprendre quelques méthodes de préparation de données.\n",
    "Nous allons, premierement, implémenter quelques méthodes en utilisant numpy.\n",
    "Ensuite, nous allons utiliser un API plus avancé pour apprendre dans quelle situation ses méthodes sont utilisées."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d47e3d1",
   "metadata": {},
   "source": [
    "- **Binome 01** : Nassim AMEUR\n",
    "- **Binome 02** : Hatem Mohamed ABDELMOUMEN\n",
    "- **Groupe** : Il n'y a qu'un groupe => info inutile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7e9c5947",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy  as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "38726b10",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Tuple"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3118323",
   "metadata": {},
   "source": [
    "## I. Réalisation des algorithmes\n",
    "\n",
    "Cette partie sert à améliorer la compréhension des algorithmes de préparation de données vus en cours en les implémentant à partir de zéro. \n",
    "Pour ce faire, nous allons utiliser la bibliothèque numpy qui est utile dans les calcules surtout matricielles.\n",
    "\n",
    "### I.1. Normalisation\n",
    "\n",
    "Ici, nous allons réaliser les deux fonctions de nomalisation : standard et min-max.\n",
    "Nous allons prendre une matrice $X[N, M]$ de $N$ échantillons et $M$ colonnes.\n",
    "La normalisation standard d'une colonne $j$ peut être décrite comme : \n",
    "$$standard(X_j) = \\frac{X_j - \\mu(X_j)}{\\sigma(X_j)}$$\n",
    "La nomalisation min-max d'une colonne $j$ peut être décrite comme : \n",
    "$$minmax(X_j) = \\frac{X_j - min(X_j)}{max(X_j) - min(X_j)}$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "4e80e66e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([4. , 3. , 0.5]),\n",
       " array([1.87082869, 2.        , 0.5       ]),\n",
       " array([[ 1.60356745,  1.        , -1.        ],\n",
       "        [-1.06904497, -1.        ,  1.        ],\n",
       "        [-0.53452248,  1.        , -1.        ],\n",
       "        [ 0.        , -1.        ,  1.        ]]))"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO: Standardisation d'une matrice\n",
    "# Entrée : la matrice des données (N échantillons X  M caractéristiques)\n",
    "# Sortie : vecteur de M moyennes, vecteur de M écart-types, une matrice normalisée\n",
    "def norm_std(X: np.ndarray) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:\n",
    "    moy   = X.mean(axis=0)\n",
    "    std   = X.std(axis=0)\n",
    "    X_new = (X - moy)/std\n",
    "    return moy, std, X_new\n",
    "\n",
    "\n",
    "#=====================================================================\n",
    "# TEST UNITAIRE\n",
    "#=====================================================================\n",
    "# (array([4. , 3. , 0.5]),\n",
    "#  array([1.87082869, 2.        , 0.5       ]),\n",
    "#  array([[ 1.60356745,  1.        , -1.        ],\n",
    "#         [-1.06904497, -1.        ,  1.        ],\n",
    "#         [-0.53452248,  1.        , -1.        ],\n",
    "#         [ 0.        , -1.        ,  1.        ]]))\n",
    "#---------------------------------------------------------------------\n",
    "\n",
    "X = np.array([\n",
    "    [7, 5, 0],\n",
    "    [2, 1, 1],\n",
    "    [3, 5, 0],\n",
    "    [4, 1, 1],\n",
    "])\n",
    "\n",
    "norm_std(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "82371ce8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([7, 5, 1]),\n",
       " array([2, 1, 0]),\n",
       " array([[1. , 1. , 0. ],\n",
       "        [0. , 0. , 1. ],\n",
       "        [0.2, 1. , 0. ],\n",
       "        [0.4, 0. , 1. ]]))"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO: Normalisation min-max d'une matrice\n",
    "# Entrée : la matrice des données (N échantillons X  M caractéristiques)\n",
    "# Sortie : vecteur de M max, vecteur de M min, une matrice normalisée\n",
    "def norm_minmax(X: np.ndarray) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:\n",
    "    mx    = X.max(axis=0)\n",
    "    mn    = X.min(axis=0)\n",
    "    X_new = (X - mn) / (mx - mn)  \n",
    "    return mx, mn, X_new\n",
    "\n",
    "#=====================================================================\n",
    "# TEST UNITAIRE\n",
    "#=====================================================================\n",
    "# (array([7, 5, 1]),\n",
    "#  array([2, 1, 0]),\n",
    "#  array([[1. , 1. , 0. ],\n",
    "#         [0. , 0. , 1. ],\n",
    "#         [0.2, 1. , 0. ],\n",
    "#         [0.4, 0. , 1. ]]))\n",
    "#---------------------------------------------------------------------\n",
    "\n",
    "X = np.array([\n",
    "    [7, 5, 0],\n",
    "    [2, 1, 1],\n",
    "    [3, 5, 0],\n",
    "    [4, 1, 1],\n",
    "])\n",
    "\n",
    "norm_minmax(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65c342e5",
   "metadata": {},
   "source": [
    "### I.2. Encodage One-Hot\n",
    "\n",
    "Etant donné un vecteur $A[N]$ représentant une caractéristique nominale donnée, on veut encoder les valeurs en utilisant One-Hot. Pour faciliter la tâche, on vous donne l'algorithme détaillé : \n",
    "1. Trouver les valeurs uniques dans le vecteur $A$ ; on appele ça : un vocabulaire $V$\n",
    "1. Créer une matrice $X[N, |V|]$ en recopiant le vecteur $V$ $N$ fois. Dans python, on peut recopier un vecteur en utilisant l'instruction : [V] * N\n",
    "1. Comparer l'égalité entre chaque ligne de $A$ et chaque ligne (qui est un vecteur) de $X$.\n",
    "1. Transformer les booléens vers des entiers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "fa61401a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array(['COLD', 'HOT', 'MILD'], dtype='<U4'),\n",
       " array([[0, 1, 0],\n",
       "        [0, 0, 1],\n",
       "        [1, 0, 0],\n",
       "        [0, 1, 0],\n",
       "        [0, 0, 1]]))"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO: Encodage One-Hot\n",
    "# Entrée : un vecteur d'une caractéristique (N échantillons)\n",
    "# Sortie : vecteur du vocabulaire V, matrice N X |V|\n",
    "def one_hot(A: np.ndarray) -> Tuple[np.ndarray, np.ndarray]:\n",
    "    V = np.unique(A)\n",
    "    X = np.array([V] * A.size)\n",
    "    \n",
    "    comp = (X == np.repeat(A, V.size, axis = 0).reshape(A.size, V.size))\n",
    "    \n",
    "    #Conversion des booléens en entiers\n",
    "    X = comp.astype(int)\n",
    "    return V, X\n",
    "\n",
    "#=====================================================================\n",
    "# TEST UNITAIRE\n",
    "#=====================================================================\n",
    "# (array(['COLD', 'HOT', 'MILD'], dtype='<U4'),\n",
    "#  array([[0, 1, 0],\n",
    "#         [0, 0, 1],\n",
    "#         [1, 0, 0],\n",
    "#         [0, 1, 0],\n",
    "#         [0, 0, 1]]))\n",
    "#---------------------------------------------------------------------\n",
    "\n",
    "A = np.array(['HOT', 'MILD', 'COLD', 'HOT', 'MILD'])\n",
    "one_hot(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "829514ed",
   "metadata": {},
   "source": [
    "### I.3. Binarisation\n",
    "\n",
    "Etant donné un vecteur $A[N]$ représentant une caractéristique numérique donnée, on veut encoder les valeurs en 0 ou 1 selon un seuil $s$.\n",
    "La binarization d'un élément $A_i$ est donnée par :\n",
    "$$A_i' = \\begin{cases}\n",
    "1 & \\text{si } A_i \\ge s\\\\\n",
    "0 & \\text{sinon}\\\\\n",
    "\\end{cases}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "38db2cf8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0, 0, 0, 1, 1])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO: Binarisation\n",
    "# Entrée : un vecteur d'une caractéristique (N échantillons), un seuil de binarisation\n",
    "# Sortie : un vecteur binarisé (N échantillons)\n",
    "def bin(A: np.ndarray, seuil: float) -> np.ndarray:\n",
    "    return (A >= [seuil]*A.size).astype(int)\n",
    "\n",
    "#=====================================================================\n",
    "# TEST UNITAIRE\n",
    "#=====================================================================\n",
    "# array([1, 0, 0, 0, 1, 1])\n",
    "#---------------------------------------------------------------------\n",
    "\n",
    "A = np.array([5, 2, 1, -1, 6, 4])\n",
    "\n",
    "bin(A, 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8826182",
   "metadata": {},
   "source": [
    "### I.4. Analyse en composantes principales (ACP)\n",
    "\n",
    "Algorithme:\n",
    "1. Centrer et réduire les données (Standardisation) (X) pour avoir (X_std).\n",
    "1. Calculer la matrice de covariance (X_cov) en se basant sur (X_std).\n",
    "1. Calculer les valeurs propres (val_prop) et les vecteurs propres (vec_prop) de (X_cov). \n",
    "1. Ordonner les valeurs propres d'une maniere descendante et selon cette ordre, ordonner les vecteurs propres\n",
    "1. Garder seulement les nbr_comp vecteurs propres premiers pour former le vecteur caracteristique (vec_car) \n",
    "1. Transformer les donnees $X\\_reduit = (vec\\_car^T \\cdot X\\_std^T)^T$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "62dd8b68",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-0.47345682 -0.88081703]\n",
      " [-0.6228317   0.33478453]\n",
      " [ 0.6228317  -0.33478453]]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[-2.00488333, -0.74288047],\n",
       "       [ 1.75181002,  0.27206396],\n",
       "       [-0.99259008,  1.14038556],\n",
       "       [ 1.24566339, -0.66956905]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TODO: ACP\n",
    "# Entrée : la matrice des données (N échantillons X  M caractéristiques), nombre de composantes: nbr_comp\n",
    "# Sortie : une matrice reduite (N échantillons X nbr_comp caractéristiques)\n",
    "def ACP(X: 'np.array[M, N]', nbr_comp: int) -> 'np.array[M, nbr_comp]':\n",
    "    \n",
    "    # 1. Centrer et reduire les donnees\n",
    "    X_std = norm_std(X)[2]\n",
    "\n",
    "    # 2. matrice de covariance (X_cov)\n",
    "    X_cov = np.cov(X_std, rowvar = False)\n",
    " \n",
    "    # 3. valeurs propres et vecteurs propres\n",
    "    val_prop, vec_prop = np.linalg.eigh(X_cov)\n",
    "\n",
    "    # 4. Ordonner vec_prop selon l'ordre descendant de val_prop\n",
    "    idx = np.argsort(val_prop)[::-1]\n",
    "    vec_ord = vec_prop[:,idx]\n",
    "\n",
    "    # 5. creer le vecteur caracteristique\n",
    "    vec_car = vec_ord[:,:nbr_comp]\n",
    "    print(vec_car)\n",
    "\n",
    "    # 6. Transformer les donnees\n",
    "    X_reduit = np.matmul(vec_car.T, X_std.T).T\n",
    "\n",
    "    return X_reduit\n",
    "\n",
    "#=====================================================================\n",
    "# TEST UNITAIRE\n",
    "#=====================================================================\n",
    "# array([[-2.00488333, -0.74288047],\n",
    "#        [ 1.75181002,  0.27206396],\n",
    "#        [-0.99259008,  1.14038556],\n",
    "#        [ 1.24566339, -0.66956905]])\n",
    "#---------------------------------------------------------------------\n",
    "\n",
    "X = np.array([\n",
    "    [7, 5, 0],\n",
    "    [2, 1, 1],\n",
    "    [3, 5, 0],\n",
    "    [4, 1, 1],\n",
    "])\n",
    "\n",
    "ACP(X, 2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "308a1df5",
   "metadata": {},
   "source": [
    "## II. Application et analyse\n",
    "\n",
    "Cette partie sert à appliquer les algorithmes, modifier les paramètres et analyser les résultats.\n",
    "\n",
    "**Aucune programmation n'est demandee. Vous pouvez changer le code pour tester ou analyser les resultats sans executer**\n",
    "\n",
    "### II.1. Lecture des données\n",
    "\n",
    "Nous allons lire 4 fichiers : \n",
    "- un fichier CSV avec des colonnes séparées par des virgules ;\n",
    "- un fichier CSV avec des colonnes séparées par des point-virgules ;\n",
    "- un fichier Sqlite ;\n",
    "- un fichier XML."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e67db19a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>Marital-status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>sex</th>\n",
       "      <th>Hours-per-week</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39.0</td>\n",
       "      <td>State-gov</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Adm-clerical</td>\n",
       "      <td>Male</td>\n",
       "      <td>40</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50.0</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Male</td>\n",
       "      <td>13</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Male</td>\n",
       "      <td>40</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>11th</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Male</td>\n",
       "      <td>40</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Prof-specialty</td>\n",
       "      <td>Female</td>\n",
       "      <td>40</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>37.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>Masters</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Female</td>\n",
       "      <td>40</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>49.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>9th</td>\n",
       "      <td>Married-spouse-absent</td>\n",
       "      <td>Other-service</td>\n",
       "      <td>Female</td>\n",
       "      <td>16</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>52.0</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Male</td>\n",
       "      <td>45</td>\n",
       "      <td>&gt;50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>31.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>Masters</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Prof-specialty</td>\n",
       "      <td>Female</td>\n",
       "      <td>50</td>\n",
       "      <td>&gt;50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>42.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Male</td>\n",
       "      <td>40</td>\n",
       "      <td>&gt;50K</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    age         workclass  education         Marital-status  \\\n",
       "0  39.0         State-gov  Bachelors          Never-married   \n",
       "1  50.0  Self-emp-not-inc  Bachelors     Married-civ-spouse   \n",
       "2  38.0           Private    HS-grad               Divorced   \n",
       "3  53.0           Private       11th                    NaN   \n",
       "4  28.0           Private  Bachelors     Married-civ-spouse   \n",
       "5  37.0           Private    Masters     Married-civ-spouse   \n",
       "6  49.0           Private        9th  Married-spouse-absent   \n",
       "7  52.0  Self-emp-not-inc    HS-grad     Married-civ-spouse   \n",
       "8  31.0           Private    Masters          Never-married   \n",
       "9  42.0           Private  Bachelors     Married-civ-spouse   \n",
       "\n",
       "          occupation     sex  Hours-per-week  class  \n",
       "0       Adm-clerical    Male              40  <=50K  \n",
       "1    Exec-managerial    Male              13  <=50K  \n",
       "2  Handlers-cleaners    Male              40  <=50K  \n",
       "3  Handlers-cleaners    Male              40  <=50K  \n",
       "4     Prof-specialty  Female              40  <=50K  \n",
       "5    Exec-managerial  Female              40  <=50K  \n",
       "6      Other-service  Female              16  <=50K  \n",
       "7    Exec-managerial    Male              45   >50K  \n",
       "8     Prof-specialty  Female              50   >50K  \n",
       "9    Exec-managerial    Male              40   >50K  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult1 = pd.read_csv('data/adult1.csv', skipinitialspace=True)\n",
    "adult1.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "890485a8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>class</th>\n",
       "      <th>age</th>\n",
       "      <th>sex</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>marital-status</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>N</td>\n",
       "      <td>25</td>\n",
       "      <td>F</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>40</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>N</td>\n",
       "      <td>18</td>\n",
       "      <td>F</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>30</td>\n",
       "      <td>Never-married</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Y</td>\n",
       "      <td>47</td>\n",
       "      <td>F</td>\n",
       "      <td>Private, Prof-school</td>\n",
       "      <td>60</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Y</td>\n",
       "      <td>50</td>\n",
       "      <td>M</td>\n",
       "      <td>Federal-gov</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>55</td>\n",
       "      <td>Divorced</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>N</td>\n",
       "      <td>47</td>\n",
       "      <td>M</td>\n",
       "      <td>Self-emp-inc</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>60</td>\n",
       "      <td>Divorced</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Y</td>\n",
       "      <td>43</td>\n",
       "      <td>M</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>40</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>N</td>\n",
       "      <td>46</td>\n",
       "      <td>M</td>\n",
       "      <td>Private</td>\n",
       "      <td>5th-6th</td>\n",
       "      <td>40</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>N</td>\n",
       "      <td>35</td>\n",
       "      <td>M</td>\n",
       "      <td>Private</td>\n",
       "      <td>Assoc-voc</td>\n",
       "      <td>40</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>N</td>\n",
       "      <td>41</td>\n",
       "      <td>M</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>48</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>N,30</td>\n",
       "      <td>M</td>\n",
       "      <td>Private, HS-grad</td>\n",
       "      <td>40</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  class age               sex             workclass           education  \\\n",
       "0     N  25                 F               Private        Some-college   \n",
       "1     N  18                 F               Private             HS-grad   \n",
       "2     Y  47                 F  Private, Prof-school                  60   \n",
       "3     Y  50                 M           Federal-gov           Bachelors   \n",
       "4     N  47                 M          Self-emp-inc             HS-grad   \n",
       "5     Y  43                 M               Private        Some-college   \n",
       "6     N  46                 M               Private             5th-6th   \n",
       "7     N  35                 M               Private           Assoc-voc   \n",
       "8     N  41                 M               Private             HS-grad   \n",
       "9  N,30   M  Private, HS-grad                    40  Married-civ-spouse   \n",
       "\n",
       "       hours-per-week      marital-status  \n",
       "0                  40  Married-civ-spouse  \n",
       "1                  30       Never-married  \n",
       "2  Married-civ-spouse                 NaN  \n",
       "3                  55            Divorced  \n",
       "4                  60            Divorced  \n",
       "5                  40  Married-civ-spouse  \n",
       "6                  40  Married-civ-spouse  \n",
       "7                  40  Married-civ-spouse  \n",
       "8                  48  Married-civ-spouse  \n",
       "9                 NaN                 NaN  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "noms = ['class', 'age', 'sex', 'workclass', 'education', 'hours-per-week', 'marital-status']\n",
    "adult2 = pd.read_csv('data/adult2.csv', skipinitialspace=True, sep=';', header=None, names=noms)\n",
    "adult2.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "af026330",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>num</th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>sex</th>\n",
       "      <th>hours-per-day</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>76</td>\n",
       "      <td>Private</td>\n",
       "      <td>Masters</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Y</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>44</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>12.0</td>\n",
       "      <td>Y</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>47</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>Masters</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>10.0</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>20</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>8.0</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>29</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>single</td>\n",
       "      <td>M</td>\n",
       "      <td>8.0</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>32</td>\n",
       "      <td>Self-emp-inc</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Y</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7</td>\n",
       "      <td>17</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10th</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>6.4</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>30</td>\n",
       "      <td>Private</td>\n",
       "      <td>11th</td>\n",
       "      <td>single</td>\n",
       "      <td>M</td>\n",
       "      <td>8.0</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>9</td>\n",
       "      <td>31</td>\n",
       "      <td>Local-gov</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>8.0</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>10</td>\n",
       "      <td>42</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>8.0</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   num  age         workclass     education marital-status sex  hours-per-day  \\\n",
       "0    1   76           Private       Masters        married   M            8.0   \n",
       "1    2   44           Private     Bachelors        married   M           12.0   \n",
       "2    3   47  Self-emp-not-inc       Masters         single   F           10.0   \n",
       "3    4   20           Private  Some-college         single   F            8.0   \n",
       "4    5   29           Private       HS-grad         single   M            8.0   \n",
       "5    6   32      Self-emp-inc       HS-grad        married   M            8.0   \n",
       "6    7   17               NaN          10th         single   F            6.4   \n",
       "7    8   30           Private          11th         single   M            8.0   \n",
       "8    9   31         Local-gov       HS-grad         single   F            8.0   \n",
       "9   10   42           Private       HS-grad        married   M            8.0   \n",
       "\n",
       "  class  \n",
       "0     Y  \n",
       "1     Y  \n",
       "2     N  \n",
       "3     N  \n",
       "4     N  \n",
       "5     Y  \n",
       "6     N  \n",
       "7     N  \n",
       "8     N  \n",
       "9     N  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import sqlite3\n",
    "# établir la connexion avec la base de données\n",
    "con = sqlite3.connect('data/adult3.db')\n",
    "# récupérer le résultat d'une réquête SQL sur cette connexion\n",
    "adult3 = pd.read_sql_query('SELECT * FROM income', con)\n",
    "\n",
    "#remplacer les valeurs \"?\" par NaN de numpy\n",
    "adult3 = adult3.replace('?', np.nan)\n",
    "\n",
    "adult3.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "7095cb63",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: lxml in c:\\users\\admin\\anaconda3\\lib\\site-packages (4.9.3)\n"
     ]
    }
   ],
   "source": [
    "!pip install lxml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "45b43b42",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>sex</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>52</td>\n",
       "      <td>47</td>\n",
       "      <td>Local-gov</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>divorced</td>\n",
       "      <td>F</td>\n",
       "      <td>38</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>53</td>\n",
       "      <td>34</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>40</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>54</td>\n",
       "      <td>33</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>40</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>55</td>\n",
       "      <td>21</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>single</td>\n",
       "      <td>M</td>\n",
       "      <td>35</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>56</td>\n",
       "      <td>52</td>\n",
       "      <td>NaN</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>divorced</td>\n",
       "      <td>M</td>\n",
       "      <td>45</td>\n",
       "      <td>Y</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>57</td>\n",
       "      <td>48</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>46</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>58</td>\n",
       "      <td>23</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>single</td>\n",
       "      <td>M</td>\n",
       "      <td>40</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>59</td>\n",
       "      <td>71</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>divorced</td>\n",
       "      <td>M</td>\n",
       "      <td>2</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>60</td>\n",
       "      <td>29</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>divorced</td>\n",
       "      <td>M</td>\n",
       "      <td>60</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>61</td>\n",
       "      <td>42</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>divorced</td>\n",
       "      <td>M</td>\n",
       "      <td>50</td>\n",
       "      <td>N</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id age         workclass     education marital-status sex hours-per-week  \\\n",
       "0  52  47         Local-gov  Some-college       divorced   F             38   \n",
       "1  53  34           Private       HS-grad         single   F             40   \n",
       "2  54  33           Private     Bachelors         single   F             40   \n",
       "3  55  21           Private       HS-grad         single   M             35   \n",
       "4  56  52               NaN       HS-grad       divorced   M             45   \n",
       "5  57  48           Private       HS-grad        married   M             46   \n",
       "6  58  23           Private     Bachelors         single   M             40   \n",
       "7  59  71  Self-emp-not-inc  Some-college       divorced   M              2   \n",
       "8  60  29           Private       HS-grad       divorced   M             60   \n",
       "9  61  42           Private     Bachelors       divorced   M             50   \n",
       "\n",
       "  class  \n",
       "0     N  \n",
       "1     N  \n",
       "2     N  \n",
       "3     N  \n",
       "4     Y  \n",
       "5     N  \n",
       "6     N  \n",
       "7     N  \n",
       "8     N  \n",
       "9     N  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from lxml import etree\n",
    "\n",
    "#créer le parser et spécifier qu'il doit valider le DTD\n",
    "parser = etree.XMLParser(dtd_validation=True)\n",
    "\n",
    "#analyser le fichier XML en utilisant ce parser\n",
    "arbre = etree.parse('data/adult4.xml', parser)\n",
    "\n",
    "def valeur_noeud(noeud):\n",
    "    \"\"\"Une fonction qui retourne np.nan si le texte du noeud est absent, sinon le texte lui-meme\n",
    "    \"\"\"\n",
    "    return noeud.text if noeud is not None else np.nan\n",
    "\n",
    "\n",
    "# Creer une table pandas vide afin de la remplir a partir des donnees XML\n",
    "noms2 = ['id', 'age', 'workclass', 'education', 'marital-status', 'sex', 'hours-per-week', 'class']\n",
    "adult4 = pd.DataFrame(columns=noms2)\n",
    "\n",
    "# parcourir la structure arbre afin de la transformer a une table pandas\n",
    "for candidat in arbre.getroot():\n",
    "    idi       = candidat.get('id')\n",
    "    age       = valeur_noeud(candidat.find('age')           )\n",
    "    workclass = valeur_noeud(candidat.find('workclass')     )\n",
    "    education = valeur_noeud(candidat.find('education')     )\n",
    "    marital   = valeur_noeud(candidat.find('marital-status'))\n",
    "    sex       = valeur_noeud(candidat.find('sex')           )\n",
    "    hours     = valeur_noeud(candidat.find('hours-per-week'))\n",
    "    klass     = valeur_noeud(candidat.find('class')         )\n",
    "\n",
    "    adult4 = pd.concat([\n",
    "        adult4, \n",
    "        pd.Series([idi, age, workclass, education, marital, sex, hours, klass],index=noms2).to_frame().T\n",
    "        ], axis=0, ignore_index=True)\n",
    "    \n",
    "adult4.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b0af2bd",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Que remarquez-vous concernant l'ordre, le nombre et les noms des caractéristiques dans les 4 datasets ?\n",
    "- Que remarquez-vous à propos des valeurs des 4 tables ?\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- Les quatre datasets ont des caractéristiques presque similaires, mais leurs ordre et nombre varient.\n",
    "    - Nombre de caractéristiques : 8-7-8-8\n",
    "    - Ordre : différent de chaque dataset\n",
    "    - Tous les datasets possède un identificateur, sauf le premier.\n",
    "    - Le premier possède une caractéristique supplémentaire : occupation.\n",
    "    - Les noms des caractéristiques sont aussi différents :\n",
    "        - Mêmes noms commençant par des majuscules et miniscules (par exemple Marital-status pour le premier et marital-status\tpour les autres)\n",
    "        - Le troisième possède la caractéristique \"hours-per-day\" au lieu de \"hours-per-week\" ce qui peut porter confusion avec les autres dataset (significations totalemenet différentes).\n",
    "     \n",
    "    <br/>\n",
    "\n",
    "- En ce qui concerne les valeurs :\n",
    "    - Tout d'abord, nous remarquons bien que les valeurs sont différentes.\n",
    "    - Dans les trois premiers datasets, certaines colonnes contiennent des valeurs manquantes (NaN).\n",
    "    - Il y a des différences dans la façon dont certaines caractéristiques sont représentées :\n",
    "        - Sex = {Male, Female} pour le premier dataset et {M, F} pour les autres\n",
    "        - Classe = {>50, <=50} pour le premier dataset et {N, Y} pour les autres\n",
    "        - les catégories de la colonne Marital-status est différente pour les dataset 1,2 et 3,4.\n",
    "    - Différence dans les types des caractéristiques :\n",
    "        - Age est de type réel pour le premier et de type entier pour les autres\n",
    "    - On peut aussi remarqué que quelques valeurs sont erronées, par exemple les valeurs de {education, hours-per-week, marital-status} sont permutées entre elles"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "564353be",
   "metadata": {},
   "source": [
    "### II.2. Intégration des données\n",
    "\n",
    "Dans cette section, nous allons appliquer des opérations sur les différentes tables. \n",
    "Vous devez à chaque fois figurer l'operation et son utilite."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "37a11f6d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['num',\n",
       " 'age',\n",
       " 'workclass',\n",
       " 'education',\n",
       " 'marital-status',\n",
       " 'sex',\n",
       " 'hours-per-day',\n",
       " 'class']"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Afficher les noms des colonnes de adult3\n",
    "list(adult3.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "e2195b77",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['id',\n",
       " 'age',\n",
       " 'workclass',\n",
       " 'education',\n",
       " 'marital-status',\n",
       " 'sex',\n",
       " 'hours-per-week',\n",
       " 'class']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult3.rename(columns={'num': 'id', 'hours-per-day': 'hours-per-week'}, inplace=True)\n",
    "adult1.rename(columns={'Hours-per-week': 'hours-per-week', 'Marital-status': 'marital-status'}, inplace=True)\n",
    "\n",
    "# Afficher les noms des colonnes de adult3\n",
    "list(adult3.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "300358d6",
   "metadata": {},
   "source": [
    "**TODO: Analyse**\n",
    "\n",
    "- Quelle opération avons-nous appliqué ?\n",
    "- Pourquoi ? (Quel est l'intérêt ?)\n",
    "- Est-ce qu'en appliquant cette opération, nous aurons certains problèmes ?\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'opération effectuée est le renomage des colonnes (caractéristiques) :\n",
    "    - Le but de premier renommage consiste à changer carrément le nom.\n",
    "    - Quant au deuxième, le but est de remplacer les lettres majuscules par des miniscules.\n",
    "\n",
    "- L'intérêt de cette opération est de normaliser la structure des dataset en modifiant les noms de colonnes pour les rendre cohérents (entre tous les datasets).\n",
    "\n",
    "- Normalement nous ne devrions pas rencontrer des problèmes majeurs, mais ça se pourrait qu'il y aura des opérations manipulant les colonnes par leurs noms, donc le fait de les renommer rend ces colonnes (anciens noms) inexistantes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "b82ab545",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['age',\n",
       " 'workclass',\n",
       " 'education',\n",
       " 'marital-status',\n",
       " 'sex',\n",
       " 'hours-per-week',\n",
       " 'class',\n",
       " 'id']"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ordre = ['age', 'workclass', 'education', 'marital-status', 'sex', 'hours-per-week', 'class']\n",
    "adult1 = adult1.reindex(ordre + [\"occupation\"], axis=1)\n",
    "#print adult1.head()\n",
    "adult2 = adult2.reindex(ordre, axis=1)\n",
    "adult3 = adult3.reindex(ordre + ['id'], axis=1)\n",
    "adult4 = adult4.reindex(ordre + ['id'], axis=1)\n",
    "\n",
    "# Afficher les noms des colonnes de adult3\n",
    "list(adult3.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88e2ceb1",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "- Quelle opération avons-nous appliqué ?\n",
    "- Pourquoi ? (Quel est l'intérêt ?)\n",
    "\n",
    "**Réponse**\n",
    "- L'opération réalisée est la réindexation des colonnes des datasets, permettant l'organisation de ces colonnes selon l'ordre indiqué.\n",
    "\n",
    "- Cela permettera de rendre les datasets plus cohérents en terme d'ordre d'organisation en alignant les colonnes dans le même ordre pour chaque dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "4f3e6068",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>sex</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>class</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76</td>\n",
       "      <td>Private</td>\n",
       "      <td>Masters</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>8.0</td>\n",
       "      <td>Y</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>44</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>12.0</td>\n",
       "      <td>Y</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age workclass  education marital-status sex  hours-per-week class  id\n",
       "0   76   Private    Masters        married   M             8.0     Y   1\n",
       "1   44   Private  Bachelors        married   M            12.0     Y   2"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Afficher les deux premières lignes de la table adult3\n",
    "adult3.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "c4688f99",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>sex</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>class</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76</td>\n",
       "      <td>Private</td>\n",
       "      <td>Masters</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>Y</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>44</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>60.0</td>\n",
       "      <td>Y</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age workclass  education marital-status sex  hours-per-week class  id\n",
       "0   76   Private    Masters        married   M            40.0     Y   1\n",
       "1   44   Private  Bachelors        married   M            60.0     Y   2"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult3['hours-per-week'] *= 5\n",
    "\n",
    "# Afficher les deux premières lignes de la table adult3\n",
    "adult3.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c02cd42",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Quelle opération avons-nous appliqué ?\n",
    "\n",
    "- Pourquoi ? (Quel est l'intérêt ?)\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'opération est la normalisation des valeurs selon l'unité de mesure.\n",
    "\n",
    "- Comme le dataset 1, 2 et 4 possèdent \"hours-per-week\" comme caractéristique, alors que le dataset 3 possèd à la place la colonne \"hours-per-day\". Comme les deux unités de meusure sont différentes, il est préférable de les normaliser, de les rendre cohérents. C'est pourauoi nous multiplions le nombre d'heures par jour par le nombre de jours de travail (5 dans notre cas) afin de transformer cette colonne en nombre d'heures par semaine, tout comme les autres datasets => le but est donc de donner une signification au nom de la colonne en normalisant l'unité de mesure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "28a335ae",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>sex</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>class</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>76</td>\n",
       "      <td>Private</td>\n",
       "      <td>Masters</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>Y</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>44</td>\n",
       "      <td>Private</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>60.0</td>\n",
       "      <td>Y</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>47</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>Masters</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>50.0</td>\n",
       "      <td>N</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>20</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>29</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>single</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>32</td>\n",
       "      <td>Self-emp-inc</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>Y</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>17</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10th</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>32.0</td>\n",
       "      <td>N</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>30</td>\n",
       "      <td>Private</td>\n",
       "      <td>11th</td>\n",
       "      <td>single</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>31</td>\n",
       "      <td>Local-gov</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>42</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  age         workclass     education marital-status sex hours-per-week class  \\\n",
       "0  76           Private       Masters        married   M           40.0     Y   \n",
       "1  44           Private     Bachelors        married   M           60.0     Y   \n",
       "2  47  Self-emp-not-inc       Masters         single   F           50.0     N   \n",
       "3  20           Private  Some-college         single   F           40.0     N   \n",
       "4  29           Private       HS-grad         single   M           40.0     N   \n",
       "5  32      Self-emp-inc       HS-grad        married   M           40.0     Y   \n",
       "6  17               NaN          10th         single   F           32.0     N   \n",
       "7  30           Private          11th         single   M           40.0     N   \n",
       "8  31         Local-gov       HS-grad         single   F           40.0     N   \n",
       "9  42           Private       HS-grad        married   M           40.0     N   \n",
       "\n",
       "   id  \n",
       "0   1  \n",
       "1   2  \n",
       "2   3  \n",
       "3   4  \n",
       "4   5  \n",
       "5   6  \n",
       "6   7  \n",
       "7   8  \n",
       "8   9  \n",
       "9  10  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult34 = pd.concat([adult3, adult4], ignore_index=True)\n",
    "adult34.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19e1551c",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Quelle opération avons-nous appliqué ?\n",
    "\n",
    "- Pourquoi ? (Quel est l'intérêt ?)\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'opération réalisée est la concaténation de deux datasets.\n",
    "\n",
    "- L'intérêt de cette opération est de fusionner les données deux datasets en paramètres en un seul dataset, ce qui nous permettra de travailler sur un seul dataset contenant toutes les données au lieu de plusieurs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "582182ff",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>sex</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>class</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>70</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>single</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>70</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>single</td>\n",
       "      <td>M</td>\n",
       "      <td>8</td>\n",
       "      <td>N</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>31</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>30.0</td>\n",
       "      <td>N</td>\n",
       "      <td>46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>31</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>single</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6</td>\n",
       "      <td>N</td>\n",
       "      <td>46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>22</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>24.0</td>\n",
       "      <td>N</td>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>22</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>4.8</td>\n",
       "      <td>N</td>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>36</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>widowed</td>\n",
       "      <td>F</td>\n",
       "      <td>24.0</td>\n",
       "      <td>N</td>\n",
       "      <td>48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>widowed</td>\n",
       "      <td>F</td>\n",
       "      <td>4.8</td>\n",
       "      <td>N</td>\n",
       "      <td>48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>64</td>\n",
       "      <td>Private</td>\n",
       "      <td>11th</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>64</td>\n",
       "      <td>Private</td>\n",
       "      <td>11th</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>8</td>\n",
       "      <td>N</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>43</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>divorced</td>\n",
       "      <td>F</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>43</td>\n",
       "      <td>Federal-gov</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>divorced</td>\n",
       "      <td>F</td>\n",
       "      <td>8</td>\n",
       "      <td>N</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    age    workclass     education marital-status  sex hours-per-week class  \\\n",
       "44   70      Private  Some-college         single    M           40.0     N   \n",
       "94   70      Private  Some-college         single    M              8     N   \n",
       "45   31      Private       HS-grad         single    F           30.0     N   \n",
       "95   31      Private       HS-grad         single  NaN              6     N   \n",
       "46   22      Private  Some-college        married    M           24.0     N   \n",
       "96   22      Private  Some-college        married    M            4.8     N   \n",
       "47   36      Private       HS-grad        widowed    F           24.0     N   \n",
       "97  NaN      Private       HS-grad        widowed    F            4.8     N   \n",
       "48   64      Private          11th        married    M           40.0     N   \n",
       "98   64      Private          11th        married    M              8     N   \n",
       "49   43          NaN  Some-college       divorced    F           40.0     N   \n",
       "99   43  Federal-gov  Some-college       divorced    F              8     N   \n",
       "\n",
       "    id  \n",
       "44  45  \n",
       "94  45  \n",
       "45  46  \n",
       "95  46  \n",
       "46  47  \n",
       "96  47  \n",
       "47  48  \n",
       "97  48  \n",
       "48  49  \n",
       "98  49  \n",
       "49  50  \n",
       "99  50  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Transformer le champs \"id\" à un entier\n",
    "adult34['id'] = pd.to_numeric(adult34['id'], downcast='integer')\n",
    "\n",
    "# Ordonner la table en se basant sur les valeurs de \"id\"\n",
    "adult34 = adult34.sort_values(by='id')\n",
    "\n",
    "# L'opération que vous devez deviner (une opération de vérification)\n",
    "red = adult34[adult34.duplicated('id', keep=False)]\n",
    "red"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31a01085",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Quelle opération avons-nous appliqué ?\n",
    "\n",
    "- Pourquoi ? (Quel est l'intérêt ?)\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'opération en question est la vérification des doublons dans la colonne \"id\".\n",
    "\n",
    "- Son intérêt est de détecter les lignes dont la valeur de la colonne 'id' est dupliquée dans le dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "0da75dc6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>sex</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>class</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>70</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>single</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>70</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>single</td>\n",
       "      <td>M</td>\n",
       "      <td>8</td>\n",
       "      <td>N</td>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>31</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>30.0</td>\n",
       "      <td>N</td>\n",
       "      <td>46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>31</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>single</td>\n",
       "      <td>F</td>\n",
       "      <td>6</td>\n",
       "      <td>N</td>\n",
       "      <td>46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>22</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>24.0</td>\n",
       "      <td>N</td>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>22</td>\n",
       "      <td>Private</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>4.8</td>\n",
       "      <td>N</td>\n",
       "      <td>47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>36</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>widowed</td>\n",
       "      <td>F</td>\n",
       "      <td>24.0</td>\n",
       "      <td>N</td>\n",
       "      <td>48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>36</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>widowed</td>\n",
       "      <td>F</td>\n",
       "      <td>4.8</td>\n",
       "      <td>N</td>\n",
       "      <td>48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>64</td>\n",
       "      <td>Private</td>\n",
       "      <td>11th</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>64</td>\n",
       "      <td>Private</td>\n",
       "      <td>11th</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>8</td>\n",
       "      <td>N</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>43</td>\n",
       "      <td>Federal-gov</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>divorced</td>\n",
       "      <td>F</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>43</td>\n",
       "      <td>Federal-gov</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>divorced</td>\n",
       "      <td>F</td>\n",
       "      <td>8</td>\n",
       "      <td>N</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age    workclass     education marital-status sex hours-per-week class  id\n",
       "44  70      Private  Some-college         single   M           40.0     N  45\n",
       "94  70      Private  Some-college         single   M              8     N  45\n",
       "45  31      Private       HS-grad         single   F           30.0     N  46\n",
       "95  31      Private       HS-grad         single   F              6     N  46\n",
       "46  22      Private  Some-college        married   M           24.0     N  47\n",
       "96  22      Private  Some-college        married   M            4.8     N  47\n",
       "47  36      Private       HS-grad        widowed   F           24.0     N  48\n",
       "97  36      Private       HS-grad        widowed   F            4.8     N  48\n",
       "48  64      Private          11th        married   M           40.0     N  49\n",
       "98  64      Private          11th        married   M              8     N  49\n",
       "49  43  Federal-gov  Some-college       divorced   F           40.0     N  50\n",
       "99  43  Federal-gov  Some-college       divorced   F              8     N  50"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Il y a un problème avec cette forme\n",
    "# en attendant qu'il soit réglé\n",
    "#adult34 = adult34.groupby(\"id\").ffill()\n",
    "\n",
    "adult34.update(adult34.groupby(['id']).ffill())\n",
    "adult34.update(adult34.groupby(['id']).bfill())\n",
    "\n",
    "# L'opération de vérification précédente\n",
    "red = adult34[adult34.duplicated('id', keep=False)]\n",
    "red"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e0889d1",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Quelle opération avons-nous appliqué ?\n",
    "\n",
    "- Pourquoi ? (Quel est l'intérêt ?)\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'opération réalisée est la mise à jour du dataset en remplissant les valeurs manquantes avec les valeurs correspondantes des lignes précédentes ou suivantes ayant le même id.\n",
    "\n",
    "- L'intérêt de cette opération est de traiter les valeurs manquantes afin de compléter les données dans le dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "fcfd9a0a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>sex</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>class</th>\n",
       "      <th>id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: [age, workclass, education, marital-status, sex, hours-per-week, class, id]\n",
       "Index: []"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult34.drop_duplicates('id', keep='last', inplace=True)\n",
    "\n",
    "# On refait la même opération précédente\n",
    "red = adult34[adult34.duplicated('id', keep=False)]\n",
    "red"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac8c4341",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Quelle opération avons-nous appliqué ?\n",
    "\n",
    "- Pourquoi ? (Quel est l'intérêt ?)\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'opération en question est la suppression des doublons en se basant sur la colonne id, en concervant la derniere ligne pour chaque doublon.\n",
    "\n",
    "- L'intérêt de cette opération est de garantir que le dataset ne contient plus de doublons basés sur la colonne id. Comme le id doit être unique, cette opération élimine donc la redondance, assurant l'intégrité des données."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "c9942837",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['age',\n",
       " 'workclass',\n",
       " 'education',\n",
       " 'marital-status',\n",
       " 'sex',\n",
       " 'hours-per-week',\n",
       " 'class',\n",
       " 'occupation']"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(adult1.columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "ff9d6799",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['age',\n",
       " 'workclass',\n",
       " 'education',\n",
       " 'marital-status',\n",
       " 'sex',\n",
       " 'hours-per-week',\n",
       " 'class']"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult1 .drop(['occupation'], axis=1, inplace=True)\n",
    "adult34.drop(['id'        ], axis=1, inplace=True)\n",
    "\n",
    "list(adult1.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bd863b5",
   "metadata": {},
   "source": [
    "**TODO: Analyse**\n",
    "\n",
    "- Quelle opération avons-nous appliqué ?\n",
    "\n",
    "- Pourquoi ? (Quel est l'intérêt ?)\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'opération réalisée est la suppression des colonnes.\n",
    "    \n",
    "    - La colonne 'occupation' pour le premier dataset.\n",
    "    - La colonne 'id' pour le dataset obtenu après la concaténation.\n",
    "\n",
    "- L'intérêt est :\n",
    "\n",
    "    - La première suppression sert à éliminer la colonne en plus afin de garantir que tous les datasets aient la même structure.\n",
    "    - La deuxième suppression a pour but d'éliminer une colonne inutile afin de simplifier la structure du dataset (elle a déjà servi à identifier et éliminer les doublons, on en a plus besoin)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "b0562378",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Never-married', 'Married-civ-spouse', 'Divorced', nan,\n",
       "       'Married-spouse-absent', 'Separated', 'Married-AF-spouse'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# les différentes valeurs du colonne adult1.marital-status\n",
    "adult1['marital-status'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "3e15df2d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['single', 'married', 'divorced', nan], dtype=object)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dic = {\n",
    "    'Never-married'        : 'single'  ,\n",
    "    'Married-civ-spouse'   : 'married' ,\n",
    "    'Married-spouse-absent': 'married' ,\n",
    "    'Married-AF-spouse'    : 'married' ,\n",
    "    'Divorced'             : 'divorced',\n",
    "    'Separated'            : 'divorced',\n",
    "    'Widowed'              : 'widowed'\n",
    "}\n",
    "\n",
    "adult1['marital-status'] = adult1['marital-status'].map(dic)\n",
    "adult2['marital-status'] = adult2['marital-status'].map(dic)\n",
    "\n",
    "# les différentes valeurs du colonne adult1.marital-status après mappage\n",
    "adult1['marital-status'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccd81277",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Quelle opération avons-nous appliqué ?\n",
    "\n",
    "- Pourquoi ? (Quel est l'intérêt ?)\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'opération effectuée est le mappage des valeurs de la colonne 'marital-status' à d'autres valeurs à travers leur regroupement en catégories plus larges et plus significatives.\n",
    "\n",
    "- L'intérêt est la simplification du dataset en réduisant le nombre de catégories par colonnes et les rendre plus significatives. De plus, cela permettra de standariser ces dernières avec les autres datasets pour qu'ils aient tous les mêmes valeurs (les valeurs possibles de chaque colonne sont sont identiques pour tous les datasets)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "cb1d00fb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(194, 7)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# On va appliquer la même opération sur d'autres caractéristiques\n",
    "adult1['sex'  ] = adult1['sex'  ].map({\"Female\": \"F\", \"Male\": \"M\"})\n",
    "adult1['class'] = adult1['class'].map({\"<=50K\": \"N\", \">50K\": \"Y\"})\n",
    "\n",
    "# Ensuite, on fusionne les tables dans une seule\n",
    "adult = pd.concat([adult1, adult2, adult34], ignore_index=True)\n",
    "\n",
    "# dimension de la table adult\n",
    "adult.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55e7d5de",
   "metadata": {},
   "source": [
    "Nos datasets sont maintenant tous homogènes, ayant les mêmes colonnes et les mêmes valeurs. Nous pouvons donc les concaténer afin d'obtenir un seul dataset complet à manipuler."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cde7f137",
   "metadata": {},
   "source": [
    "### II.3. Néttoyage des données\n",
    "\n",
    "Ici, on va appliquer des opérations de néttoyage. C'est à vous de déviner quelle opération a-t-on utilisé et pourquoi.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "8595785f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age                5\n",
       "workclass         10\n",
       "education          1\n",
       "marital-status     4\n",
       "sex                2\n",
       "hours-per-week     2\n",
       "class              0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Afficher le nombre des valeurs nulles dans chaque colonne\n",
    "adult.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "39c8f04b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age               3\n",
       "workclass         0\n",
       "education         0\n",
       "marital-status    0\n",
       "sex               0\n",
       "hours-per-week    0\n",
       "class             0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult.dropna(subset=[\"workclass\", \"education\", \"marital-status\", \"sex\", \"hours-per-week\", \"class\"], inplace=True)\n",
    "adult.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e3d7475",
   "metadata": {},
   "source": [
    "**TODO: Analyse**\n",
    "\n",
    "- Quelle opération a-t-on appliqué ?\n",
    "\n",
    "- Pourquoi ? (Quel est l'intérêt ?)\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'opération réalisée consiste à supprimer les lignes du dataset qui contienneent des valeurs manquantes dans au moins l'une des colonnes spécifiées.\n",
    "\n",
    "- L'intérêt de cette opération est de nettoyer le dataset en supprimant les lignes contenant des valeurs manquantes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "e58fa7de",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age               0\n",
       "workclass         0\n",
       "education         0\n",
       "marital-status    0\n",
       "sex               0\n",
       "hours-per-week    0\n",
       "class             0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult[\"age\"] = pd.to_numeric(adult[\"age\"])\n",
    "adult[\"age\"] = adult.groupby([\"class\", \"education\"])[\"age\"].transform(lambda x: x.fillna(int(round(x.mean()))))\n",
    "adult.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f237fece",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Quelle opération a-t-on appliqué ?\n",
    "\n",
    "- Pourquoi ? (Quel est l'intérêt ?)\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- On a deux opérations, :\n",
    "    - La première convertit la colonne age en un type de données numérique pour garantir que ses valeurs sont toutes de type numérique.\n",
    "    - La deuxième permet de remplir les valeurs manquantes dans la colonne 'age' en se basant sur la moyenne des âges dans chaque groupe défini par les combinaisons de \"class\" et \"education\".\n",
    "\n",
    "- L'intérêt de cette opération est de nettoyer le dataset en remplissant les valeurs manquantes (qui restent de l'étape précédente) afin de garantir que les données sont toutes complètes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aea667a9",
   "metadata": {},
   "source": [
    "**En résumé,** les deux dernières opérations traitent les données manquantes, soit en les supprimant ou en les remplissant par des constantes (une moyenne de groupe pour notre cas), afin de nettoyer nos données et rendre notre dataset complet."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdb47fb3",
   "metadata": {},
   "source": [
    "### II.4. Transformation des données"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "38278ca9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    Bachelors\n",
       "1    Bachelors\n",
       "2      HS-grad\n",
       "4    Bachelors\n",
       "5      Masters\n",
       "6          9th\n",
       "Name: education, dtype: object"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult[\"education\"].head(6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "4cf4dba5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[6.],\n",
       "       [6.],\n",
       "       [8.],\n",
       "       [6.],\n",
       "       [9.],\n",
       "       [3.]])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import OrdinalEncoder\n",
    "ord_enc = OrdinalEncoder()\n",
    "# le résultat c'est un numpy.ndarray\n",
    "education_enc = ord_enc.fit_transform(adult[[\"education\"]])\n",
    "education_enc[:6,]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb5a7ebb",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Quel est le type d'encodage utilisé ?\n",
    "\n",
    "- A votre avis, dans quel cas peut-on utiliser ce type d'encodage ?\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'encodage utilisé est un encodage ordinal (Ordinal Encoder). Il consiste à attribuer des nombres entiers aux catégories (modalités) d'une variable.\n",
    "\n",
    "- Ce type d'encodage semble approprié lorsque les catégories (modalités) d'une variable peuvent être classées dans un certain ordre séquentiel (relation d'ordre)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "1a7a2980",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    M\n",
       "1    M\n",
       "2    M\n",
       "4    F\n",
       "5    F\n",
       "6    F\n",
       "Name: sex, dtype: object"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult[\"sex\"].head(6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "9bb8987b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 1.],\n",
       "       [0., 1.],\n",
       "       [0., 1.],\n",
       "       [1., 0.],\n",
       "       [1., 0.],\n",
       "       [1., 0.]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import OneHotEncoder\n",
    "onehot_enc = OneHotEncoder()\n",
    "# le résultat c'est un numpy.ndarray\n",
    "sex_enc = onehot_enc.fit_transform(adult[[\"sex\"]])\n",
    "sex_enc.toarray()[:6,]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50946472",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Quel est le type d'encodage utilisé ?\n",
    "\n",
    "- A votre avis, dans quel cas peut-on utiliser ce type d'encodage ?\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'encodage utilisé ici est le One-Hot Encoding. Son principe est le suivant : Au lieu de représenter les catégories par leurs valeurs, on crée une nouvelle colonne binaire pour chaque catégorie unique. Chaque colonne binaire indique la présence (1) ou l'absence (0) de la catégorie dans l'observation.\n",
    "\n",
    "- Ce type d'encodage semble approprié lorsque l'ordre entre les catégories d'une variable n'est pas important, ou lorsqu'il n'y a pas une relation directe ou hiérarchique entre les différentes valeurs (chaque valeur est indépendante l'une de l'autre)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "c86b4e21",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    40.0\n",
       "1    13.0\n",
       "2    40.0\n",
       "Name: hours-per-week, dtype: float64"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult[\"hours-per-week\"] = pd.to_numeric(adult[\"hours-per-week\"])\n",
    "adult[\"hours-per-week\"].head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "0fcba4ea",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.49367089],\n",
       "       [0.15189873],\n",
       "       [0.49367089]])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "min_max_scaler = MinMaxScaler()\n",
    "# le résultat c'est un numpy.ndarray\n",
    "hours_per_week_prop = min_max_scaler.fit_transform(adult[[\"hours-per-week\"]])\n",
    "hours_per_week_prop[:3,]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "5be3df27",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>sex</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>class</th>\n",
       "      <th>hours-per-week-prop</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39.0</td>\n",
       "      <td>State-gov</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>single</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0.493671</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50.0</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>married</td>\n",
       "      <td>M</td>\n",
       "      <td>13.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0.151899</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38.0</td>\n",
       "      <td>Private</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>divorced</td>\n",
       "      <td>M</td>\n",
       "      <td>40.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0.493671</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    age         workclass  education marital-status sex  hours-per-week class  \\\n",
       "0  39.0         State-gov  Bachelors         single   M            40.0     N   \n",
       "1  50.0  Self-emp-not-inc  Bachelors        married   M            13.0     N   \n",
       "2  38.0           Private    HS-grad       divorced   M            40.0     N   \n",
       "\n",
       "   hours-per-week-prop  \n",
       "0             0.493671  \n",
       "1             0.151899  \n",
       "2             0.493671  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# pour ajouter la nouvelle caractéristique au dataframe\n",
    "adult[\"hours-per-week-prop\"] = hours_per_week_prop\n",
    "adult.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfa5f9d8",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Comment la normalisation MinMax est calculée ?\n",
    "\n",
    "- Décrire les valeurs résultats (plage de valeurs, etc.) ?\n",
    "\n",
    "- Est-ce que les valeurs du dataset de test sont garanties d'être dans la plage ?\n",
    "\n",
    "- Si oui, expliquer pourquoi. Si non, comment garantir la plage des valeurs ?\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- La nomalisation min-max d'une colonne $j$ peut être calculée selon la formule ci-dessous. Pour chaque valeur dans la colonne, on soustrait la valeur minimale de la colonne à cette valeur. Ensuite, on divise le résultat par la différence entre la valeur maximale et la valeur minimale de la colonne.\n",
    " : \n",
    "\n",
    "$$minmax(X_j) = \\frac{X_j - min(X_j)}{max(X_j) - min(X_j)}$$\n",
    "\n",
    "- Les valeurs résultantes sont dans la plage [0, 1]. En effet, la valeur minimale est normalisée à 0, la valeur maximale est normalisée à 1, donc toutes les autres valeurs sont normalisées entre 0 et 1.\n",
    "\n",
    "    <br/>\n",
    "\n",
    "    - **Pour la valeur minimale :** $minmax(min(X_j)) = \\frac{min(X_j) - min(X_j)}{max(X_j) - min(X_j)} = 0$\n",
    "    \n",
    "    <br/>\n",
    "    \n",
    "    - **Pour la valeur maximale :** $minmax(max(X_j)) = \\frac{max(X_j) - min(X_j)}{max(X_j) - min(X_j)} = 1$\n",
    "\n",
    "    <br/>\n",
    "  \n",
    "- Non, la plage des valeurs dans un dataset de test n'est pas garantie d'être comprise entre 0 et 1 car on pourrait avoir parmi eux des valeurs inférieures au précédent min ou des valeurs supérieures au précédent max (ceux obtenus du dataset d'entraînement). Ces dernières, en leur appliquant la formule, donneront donc des valeurs en dehors de la plage [0, 1]. En effet ceux inférieures au min donneront des valeurs inférieures à 0 et les valeurs supérieures au max donneront des valeurs supérieures à 1. \n",
    "\n",
    "- Pour garantir que les valeurs du dataset de test restent dans la plage [0, 1], nous proposons deux solutions : \n",
    "    - Si on possède de toutes les données y compris celles du test avant l'entraînement, il suffit d'appliquer la formule min-max sur toutes les valeurs du dataset (entraînement et test). Donc on considère le min étant la valeur minimale de la colonne sur toutes les lignes du dataset (entraînement et test) et le max étant la valeur maximale de la colonne sur toutes les lignes du dataset (entraînement et test).\n",
    "    - Sinon, il suffit de retenir les valeurs minimales et maximales de l'ensemble de données d'entraînement. Ensuite, lors de la normalisation des données de test, si une valeur dépasse le maximum, on la fixe au maximum, et si elle est inférieure au minimum, on la remplace par le minimum afin de ne pas dépasser les bornes fixées au préalable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "04c42c4e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    39.0\n",
       "1    50.0\n",
       "2    38.0\n",
       "Name: age, dtype: float64"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult[\"age\"].head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "a1235410",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.3442623 ],\n",
       "       [0.52459016],\n",
       "       [0.32786885]])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "std_scaler = StandardScaler()\n",
    "# le résultat c'est un numpy.ndarray\n",
    "age_normal = min_max_scaler.fit_transform(adult[[\"age\"]])\n",
    "age_normal[:3,]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38be4b39",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Comment la normalisation standard est calculée ?\n",
    "\n",
    "- Décrire les valeurs résultats (plage de valeurs, etc.) ?\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- La normalisation standard d'une colonne $j$ est calculée selon la formule ci-dessous. Pour chaque valeur dans la colonne, on soustrait la moyenne de la colonne à cette valeur. Ensuite, on divise le résultat par l'écart type de la colonne. \n",
    "\n",
    "$$standard(X_j) = \\frac{X_j - \\mu(X_j)}{\\sigma(X_j)}$$\n",
    "\n",
    "- La plage des valeurs normalisées n'est pas bornée comme dans le cas de la normalisation Min-Max. Cependant, la distribution des données sera centrée autour de zéro et mise à l'échelle en fonction de l'écart-type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "712f1f36",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     39.0\n",
       "1     50.0\n",
       "2     38.0\n",
       "4     28.0\n",
       "5     37.0\n",
       "6     49.0\n",
       "7     52.0\n",
       "8     31.0\n",
       "9     42.0\n",
       "10    43.0\n",
       "Name: age, dtype: float64"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult[\"age\"].head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "ac2de190",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.],\n",
       "       [1.],\n",
       "       [0.],\n",
       "       [0.],\n",
       "       [0.],\n",
       "       [1.],\n",
       "       [1.],\n",
       "       [0.],\n",
       "       [1.],\n",
       "       [1.]])"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.preprocessing import Binarizer\n",
    "\n",
    "binarizer = Binarizer(threshold=40)\n",
    "# le résultat c'est un numpy.ndarray\n",
    "age_bin = binarizer.fit_transform(adult[[\"age\"]])\n",
    "age_bin[:10,]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16d52fc1",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Quelle est l'opération appliquée ici ?\n",
    "\n",
    "- Quel est son rôle ?\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'opération appliquée ici est la binarisation de la colonne \"age\" selon un seuil. Cela consiste à remplacer les valeur de la colonne par 1 si la valeur est supérieure ou égale au seuil, 0 sinon.\n",
    "\n",
    "- Le rôle de cette opération est la division des valeurs d'une colonne en deux catégories en se basant sur un seuil. Plus généralement, elle permet de convertir des données continues en données binaires."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e1ade4f",
   "metadata": {},
   "source": [
    "### II.5. ACP\n",
    "\n",
    "Ici, nous allons appliquer l'ACP sur nos donnees. Mais avant ça, nous avons appliqué quelques operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "3888627b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>class</th>\n",
       "      <th>age</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>workclass_Federal-gov</th>\n",
       "      <th>workclass_Local-gov</th>\n",
       "      <th>workclass_Private</th>\n",
       "      <th>workclass_Self-emp-inc</th>\n",
       "      <th>workclass_Self-emp-not-inc</th>\n",
       "      <th>workclass_State-gov</th>\n",
       "      <th>education_11th</th>\n",
       "      <th>...</th>\n",
       "      <th>education_HS-grad</th>\n",
       "      <th>education_Masters</th>\n",
       "      <th>education_Prof-school</th>\n",
       "      <th>education_Some-college</th>\n",
       "      <th>marital-status_divorced</th>\n",
       "      <th>marital-status_married</th>\n",
       "      <th>marital-status_single</th>\n",
       "      <th>marital-status_widowed</th>\n",
       "      <th>sex_F</th>\n",
       "      <th>sex_M</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>N</td>\n",
       "      <td>39.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>N</td>\n",
       "      <td>50.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>N</td>\n",
       "      <td>38.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>N</td>\n",
       "      <td>28.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>N</td>\n",
       "      <td>37.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 27 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  class   age  hours-per-week  workclass_Federal-gov  workclass_Local-gov  \\\n",
       "0     N  39.0            40.0                      0                    0   \n",
       "1     N  50.0            13.0                      0                    0   \n",
       "2     N  38.0            40.0                      0                    0   \n",
       "4     N  28.0            40.0                      0                    0   \n",
       "5     N  37.0            40.0                      0                    0   \n",
       "\n",
       "   workclass_Private  workclass_Self-emp-inc  workclass_Self-emp-not-inc  \\\n",
       "0                  0                       0                           0   \n",
       "1                  0                       0                           1   \n",
       "2                  1                       0                           0   \n",
       "4                  1                       0                           0   \n",
       "5                  1                       0                           0   \n",
       "\n",
       "   workclass_State-gov  education_11th  ...  education_HS-grad  \\\n",
       "0                    1               0  ...                  0   \n",
       "1                    0               0  ...                  0   \n",
       "2                    0               0  ...                  1   \n",
       "4                    0               0  ...                  0   \n",
       "5                    0               0  ...                  0   \n",
       "\n",
       "   education_Masters  education_Prof-school  education_Some-college  \\\n",
       "0                  0                      0                       0   \n",
       "1                  0                      0                       0   \n",
       "2                  0                      0                       0   \n",
       "4                  0                      0                       0   \n",
       "5                  1                      0                       0   \n",
       "\n",
       "   marital-status_divorced  marital-status_married  marital-status_single  \\\n",
       "0                        0                       0                      1   \n",
       "1                        0                       1                      0   \n",
       "2                        1                       0                      0   \n",
       "4                        0                       1                      0   \n",
       "5                        0                       1                      0   \n",
       "\n",
       "   marital-status_widowed  sex_F  sex_M  \n",
       "0                       0      0      1  \n",
       "1                       0      0      1  \n",
       "2                       0      0      1  \n",
       "4                       0      1      0  \n",
       "5                       0      1      0  \n",
       "\n",
       "[5 rows x 27 columns]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Transformer les attributs \n",
    "adult_num = pd.get_dummies(adult, columns=['workclass', 'education', 'marital-status', 'sex'])\n",
    "# Repositionner la classe pour etre en premier\n",
    "adult_num.insert(0, 'class', adult_num.pop('class'))\n",
    "# Nous n'avons pas besoin de ca ici\n",
    "adult_num.pop('hours-per-week-prop') \n",
    "adult_num.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "649a04fc",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Pourquoi nous avons transformé les caractéristiques : 'workclass', 'education', 'marital-status', 'sex' ?\n",
    "\n",
    "- Pourquoi nous avons supprimé la carctéristique 'hours-per-week-prop' ?\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- Les caractéristiques en question sont qualitatives, elles représentent généralement des catégories plutôt que des valeurs numériques. Comme l'ACP s'effectue sur des données quantitatives et non pas qualitatives, il est nécessaire de les convertir en une forme numérique, ce qui est réalisé ici à travers le one-hot encoding.\n",
    "\n",
    "- La caractéristique 'hours-per-week-prop' a été supprimée car elle n'est pas pertinente pour l'analyse de données puisque l'information portée par cette dernière est déjà représenté par la caractéristique 'hours-per-week' (redondance)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "bc0fded5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.90589486, -1.05874639],\n",
       "       [26.18987797, 13.46056046],\n",
       "       [ 0.12921225, -0.450828  ],\n",
       "       [-7.75501444,  5.68754065],\n",
       "       [-0.65016966,  0.16407329]])"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.decomposition import PCA\n",
    "\n",
    "pca_adult = PCA(n_components=2)\n",
    "\n",
    "#exclure la première colonne 'class'\n",
    "adult_pca = pca_adult.fit_transform(adult_num.iloc[:, 1:].values)\n",
    "\n",
    "adult_pca[:5, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "9e74a210",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAGfCAYAAABx3/noAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABXSUlEQVR4nO3df3xT5d0//lcaaPmZIoJtISlBlF8KOhm66urdSkfr2FYWKgpOQbn9WVwrCIK7FfXehFEHrQKCux3wvSdlWCO4qajDFns7BEQZoMCQb0tLaQtOacqvFk/P54/ThKZNmpMmJ+dHXk8fecScnCTXIU3OO9f1vt6XSRRFEUREREQaFKN2A4iIiIj8YaBCREREmsVAhYiIiDSLgQoRERFpFgMVIiIi0iwGKkRERKRZDFSIiIhIsxioEBERkWYxUCEiIiLNYqBCREREmtUtUi+0ZMkSLFy4EHl5eSgsLAQAXLhwAXPnzsXGjRvR1NSEzMxMrFq1CgkJCbKft6WlBSdOnEDfvn1hMpkUaj0RERGFkyiKaGxsxKBBgxAT47/fJCKByu7du7FmzRqMHTvWa/vjjz+Od955B2+88Qbi4+Mxe/ZsOBwOfPLJJ7Kf+8SJE7DZbOFuMhEREUVAdXU1rFar3/sVD1TOnDmDu+++G3/84x/x29/+1rO9oaEBr732GjZs2IDbbrsNALB27VqMGjUKn376KX70ox/Jev6+ffsCkA7UYrGE/wCIiIgo7FwuF2w2m+c87o/igUpubi4mTZqEjIwMr0Blz549uHjxIjIyMjzbRo4cieTkZOzYscNvoNLU1ISmpibP7cbGRgCAxWJhoEJERKQzgdI2FA1UNm7ciM8//xy7d+/ucF9dXR1iY2PRr18/r+0JCQmoq6vz+5yLFy/Gc889F+6mEhERkQYpNuunuroaeXl5eP3119GjR4+wPe/ChQvR0NDguVRXV4ftuYmIiEhbFAtU9uzZg5MnT+KGG25At27d0K1bN2zfvh0vvfQSunXrhoSEBDQ3N+P06dNej6uvr0diYqLf542Li/MM83C4h4iIyNgUG/qZMGEC9u/f77Xtvvvuw8iRI/Hkk0/CZrOhe/fu2LZtG6ZMmQIAOHz4MKqqqpCSkqJUs4iIiMJOFEV8//33EARB7aZohtlsRrdu3UIuHaJYoNK3b19ce+21Xtt69+6Nyy+/3LN91qxZmDNnDvr37w+LxYLHHnsMKSkpsmf8EBERqa25uRm1tbU4d+6c2k3RnF69eiEpKQmxsbFdfo6IFXzzZfny5YiJicGUKVO8Cr4RERHpQUtLCyoqKmA2mzFo0CDExsay+CikHqbm5macOnUKFRUVuPrqqzst6tYZkyiKYpjbF1Eulwvx8fFoaGhgvgoREUXUhQsXUFFRgSFDhqBXr15qN0dzzp07h2PHjmHo0KEdJtbIPX9zrR8iIqIQdbW3wOjC8e+i6tAPERmQIADl5UBtLZCUBKSmAmaz2q0iIp1ioEJE4eN0Anl5wPHjl7ZZrUBREeBwqNcuItIt9lURUXg4nUBOjneQAgA1NdJ2p1OddhGRrjFQIaLQCYLUk+IrN9+9LT9f2o+INGHmzJkwmUxYsmSJ1/bNmzdrauYSAxUiCl15eceelLZEEaiulvYjoo4EASgrA4qLpesIBfU9evTA73//e3z33XcReb2uYKBCRKGrrQ3vfkTRxOkE7HYgPR2YPl26ttsjMlyakZGBxMRELF682O8+b775Jq655hrExcXBbrfjD3/4g+LtaouBChGFLikpvPsRRQuVc7vMZjNeeOEFvPzyyzjuo1d0z549mDp1Ku666y7s378fzz77LJ5++mmsW7dO0Xa1xUCFiEKXmirN7vE3rm0yATabtB8RSTSS2/XLX/4S119/PRYtWtThvmXLlmHChAl4+umnMXz4cMycOROzZ89GQUGBom1qi4EKEYXObJamIAMdgxX37cJC1lMhaktDuV2///3vsX79ehw8eNBr+8GDB3HLLbd4bbvllltw5MiRiC3AyECFiMLD4QBKSoDBg723W63SdtZRIfKmodyuW2+9FZmZmVi4cKHirxUsFnwjovBxOIDsbFamJZJDY7ldS5YswfXXX48RI0Z4to0aNQqffPKJ136ffPIJhg8fDnOEPtcMVIgovMxmIC1N7VYQaZ87t6umxneeiskk3R+h3K4xY8bg7rvvxksvveTZNnfuXIwfPx7//d//jTvvvBM7duzAihUrsGrVqoi0CeDQDxERkTo0mNv1/PPPo6WlxXP7hhtuwKZNm7Bx40Zce+21eOaZZ/D8889j5syZEWuTSRR9hXH6IXeZaCIionC7cOECKioqMHToUPTo0aNrT+JrjSybTQpSdJ7b1dm/j9zzN4d+iIiI1MTcrk4xUCHSMkHglxdRNGBul18MVIi0yld3sNUqjWnrvDuYiEguJtMSaZHKZbWJiLSCgQqR1mikrDYRkRYwUCHSGg2V1SYiUhsDFSKt0VBZbSIitTFQIdIajZXVJiJSEwMVIq1xl9VuX6nSzWSSikFFqKw2EZGaGKgQaY0Gy2oTkXGIooiMjAxkZmZ2uG/VqlXo168fjneWJxdhDFSItMjhAEpKgMGDvbdbrdJ21lEhMhShRUBZZRmK9xejrLIMQotys/pMJhPWrl2LnTt3Ys2aNZ7tFRUVmD9/Pl5++WVYrVbFXj9YDFSItMrhACorgdJSYMMG6bqigkEKkcE4DzphL7IjfX06pjunI319OuxFdjgPKlcvyWazoaioCE888QQqKiogiiJmzZqFiRMn4gc/+AFuv/129OnTBwkJCbjnnnvwzTffeB5bUlKCMWPGoGfPnrj88suRkZGBs2fPKtZWBipEWuYuqz1tmnTN4R4iQ3EedCJnUw6Ou7yHWmpcNcjZlKNosDJjxgxMmDAB999/P1asWIEDBw5gzZo1uO222/CDH/wAn332GbZu3Yr6+npMnToVAFBbW4tp06bh/vvvx8GDB1FWVgaHwwEl1zfm6slERERdFMrqyUKLAHuRvUOQ4maCCVaLFRV5FTDHKPMj5eTJk7jmmmvw7bff4s0338SBAwdQXl6O999/37PP8ePHYbPZcPjwYZw5cwbjxo1DZWUlhgwZEvD5w7F6MntUiIiIVFBeVe43SAEAESKqXdUor1KuuOMVV1yBhx56CKNGjcLkyZPxz3/+E6WlpejTp4/nMnLkSADA0aNHcd1112HChAkYM2YM7rjjDvzxj3/Ed999p1j7AAYqREREqqhtlFe0Ue5+XdWtWzd06yatUXzmzBn8/Oc/x969e70uR44cwa233gqz2YwPP/wQ7733HkaPHo2XX34ZI0aMQEVFhXLtU+yZiUgiCFK5+9paqUhbaipzTdTC94I0JKmvvKKNcvcLhxtuuAFvvvkm7Ha7J3hpz2Qy4ZZbbsEtt9yCZ555BkOGDMFbb72FOXPmKNIm9qgQKcnpBOx2ID0dmD5durbbufqxGvhekMakJqfCarHCBN/FHU0wwWaxITU5csUdc3Nz8e2332LatGnYvXs3jh49ivfffx/33XcfBEHAzp078cILL+Czzz5DVVUVnE4nTp06hVGjRinWJgYqREpxOoGcnI4LDNbUSNt5gowcvhekQeYYM4qypOKO7YMV9+3CrELFEml9GTRoED755BMIgoCJEydizJgxyM/PR79+/RATEwOLxYKPP/4YP/3pTzF8+HD813/9F/7whz/g9ttvV6xNnPVDpARBkH6t+6vuaDJJxdsqKjj0oDS+F6SgUGb9uDkPOpG3Nc8rsdZmsaEwqxCOUfqumxSOWT/MUSFSQnm5/xMjAIgiUF0t7ZeWFrFmRSW+F6RxjlEOZI/IRnlVOWoba5HUNwmpyakR7UnRMkWHfl555RWMHTsWFosFFosFKSkpeO+99zz3X7hwAbm5ubj88svRp08fTJkyBfX19Uo2iSgyamVm6cvdj7qO7wXpgDnGjDR7GqaNmYY0exqDlDYUDVSsViuWLFmCPXv24LPPPsNtt92G7OxsfPnllwCAxx9/HH/961/xxhtvYPv27Thx4gQcLA9ORpAkM0tf7n7UdXwviHQt4jkq/fv3R0FBAXJycjBw4EBs2LABOTk5AIBDhw5h1KhR2LFjB370ox/Jej7mqJAmufMiamqkoYX2mBcROXwvSEHhyFExMl1VphUEARs3bsTZs2eRkpKCPXv24OLFi8jIyPDsM3LkSCQnJ2PHjh1+n6epqQkul8vrQqQ5ZjNQJGXzw9Ru6qH7dmEhT4yRwPeCSNcUD1T279+PPn36IC4uDg8//DDeeustjB49GnV1dYiNjUW/fv289k9ISEBdXZ3f51u8eDHi4+M9F5vNpvAREHWRwwGUlACDB3tvt1ql7RzmjBy+F6QwnU+gVUw4/l0Un/UzYsQI7N27Fw0NDSgpKcGMGTOwffv2Lj/fwoULvarfuVwuBiukXQ4HkJ3NaqhaoMZ7wUq4hte9e3cAwLlz59CzZ0+VW6M9586dA3Dp36krFA9UYmNjcdVVVwEAxo0bh927d6OoqAh33nknmpubcfr0aa9elfr6eiQmJvp9vri4OMTFxSndbKLwMZs57VUrIvleOJ1AXp731GirVRqGYg+OYZjNZvTr1w8nT54EAPTq1Qum9kOMUUgURZw7dw4nT55Ev379YA4hQI94HZWWlhY0NTVh3Lhx6N69O7Zt24YpU6YAAA4fPoyqqiqkpKREullEROHjroTbvtvbXQmXw02G4v5x7Q5W6JJ+/fp12vkgh6KBysKFC3H77bcjOTkZjY2N2LBhA8rKyvD+++8jPj4es2bNwpw5c9C/f39YLBY89thjSElJkT3jh4hIcwRB6knxNTYvilICb36+NAzFYSBDMJlMSEpKwhVXXIGLFy+q3RzN6N69e0g9KW6KBionT57Evffei9raWsTHx2Ps2LF4//338ZOf/AQAsHz5csTExGDKlCloampCZmYmVq1apWSTiIiUxUq4UctsNoflxEzeFA1UXnvttU7v79GjB1auXImVK1cq2QwioshhJVyisOLqyURE4cRKuERhxUCFiCicUlOl2T3+Zn6YTIDNJu1HRAExUCEiCidWwiUKKwYqREThxkq4RGET8ToqRIbFKqTUFqsSE4UFAxWicGAVUvKFVYmJQsahHyI3QQDKyoDiYulaEOQ9zl2FtH3tDHcVUqcz3C0lIooaDFSIACmYsNuB9HRg+nTp2m4PHGQEqkIKSFVI5QY9RETkhYEKUSg9IsFUISUioqAxUKHoFmqPCKuQEhEpioEKRbdQe0RYhZSISFEMVCi6hdojwiqkRESKYqBC0S3UHhFWISUiUhQDFYpu4egRYRVSIiLFsOAbRTd3j0hOjhSUtE2qDaZHhFVIiYgUwUCFyN0j4quybGGh/B4RViElIgo7BipEAHtEiIg0ioEKkRt7RIiINIfJtERERKRZDFSIiIhIszj0Q9FFEJiHQkSkIwxUKHo4nb5n9hQVsdYJEZFGceiHokMoKyQTEZFqGKiQ8YW6QjIREamGgQoZX6grJBMRkWoYqJDxhbpCMhERqYbJtGR8oa6QTKQlnLlGUYY9KmR84VghmUgLnE7AbgfS04Hp06Vru53J4GRoDFTI+NwrJAMdg5VgVkgmUhNnrlGUYqBC0cG9QvLgwd7brVZpO+uokJZx5hpFMeaoUPTgCsmkV8HMXOPCmmQwDFQounCFZNIjzlyjKMahHyIirePMNYpiDFSIiLSOM9c04/Rp4LPPgJMnpdt1dcC+fcD586o2y9AYqJAxCQJQVgYUF0vXTDIkPePMNVUVFAD/8R9AYiJw2WXA+PHAe+9J9735JnDddUDv3tJM8dtvB/70J1WbaziKBiqLFy/G+PHj0bdvX1xxxRWYPHkyDh8+7LXPhQsXkJubi8svvxx9+vTBlClTUF9fr2SzSOuam6Uv3ccek66bm4N7PGtNkBFx5lrEfPMN8NRTwLffSrfr6oCEBODhh4ENG4Ddu6W8fACYNg345BPgtdeAO++U4savvrr0uBUrgKYmdY7DKEyi6Gu+W3hkZWXhrrvuwvjx4/H999/jqaeewoEDB/DVV1+hd+/eAIBHHnkE77zzDtatW4f4+HjMnj0bMTEx+OSTT2S9hsvlQnx8PBoaGmCxWJQ6FIqU+fOBZcu8e0DMZmDOHGDp0sCPd9eaaP9n7f7VyS900jtWplXM6dPS18/y5dLtDz8EfvSjrj/f1q3ApElSbPn008DMmUD37uFoqTHIPn+LEXTy5EkRgLh9+3ZRFEXx9OnTYvfu3cU33njDs8/BgwdFAOKOHTtkPWdDQ4MIQGxoaFCkzRRB8+aJohRi+L7Mm9f547//XhStVv+PN5lE0WaT9iMiaqO0VBQTE0WxZ0/pq+bUqfA876FDonjXXdJX0A03iOLhw+F5XiOQe/6OaI5KQ0MDAKB///4AgD179uDixYvIyMjw7DNy5EgkJydjx44dPp+jqakJLpfL60IG0Nws/ZTpzB/+APzv//rPOeEqyUQUguuuA44ckTpvBwwIz3OOGCGlyu3aJSXcdvYVRb5FLFBpaWlBfn4+brnlFlx77bUAgLq6OsTGxqJfv35e+yYkJKCurs7n8yxevBjx8fGei81mU7rpFAmrVgVOeG1pAe6913/OCWtNEFGQ3n0X+P57qbzSe+91TAEKl/HjgQMHgNtuk77K3HksFFjEApXc3FwcOHAAGzduDOl5Fi5ciIaGBs+luro6TC0kVR09Gtz+vtY3Ya0JIgrC+vVSDsmbb0q3/c3+DpeY1jNuYSFw443AF18o+3pGEZFAZfbs2fjb3/6G0tJSWK1Wz/bExEQ0Nzfj9OnTXvvX19cjMTHR53PFxcXBYrF4XcgAhg0Lbn9f65uw1gQRyfT558ADDwCzZgFTp0b2tR96CBg1CvjZzwBmLwSmaKAiiiJmz56Nt956Cx999BGGDh3qdf+4cePQvXt3bNu2zbPt8OHDqKqqQkpKipJNI6159NHgZy60zzlhrQkikmn+fOCqq4BXXlG+J6W93r2lzuBvvwVefDGyr61HigYqubm5+POf/4wNGzagb9++qKurQ11dHc63lvCLj4/HrFmzMGfOHJSWlmLPnj247777kJKSgh+FMieM9Cc2VpqC3BVtc070WmuCBeqIImbHDmDbNmDx4q5NFxZaBJRVlqF4fzHKKssgtAT/ebXZgF//Wpoj4Cclk1opWkfF5CdMXbt2LWbOnAlAKvg2d+5cFBcXo6mpCZmZmVi1apXfoZ/2WEfFYHzVUQmktLTjQoN6qjXhdAJ5ed7TAaxWqXdIq4EVkY6JIrB9u1RtNtjeFOdBJ/K25uG469Ln1WqxoiirCI5RwX1ev/sO2LIFuOce7X49KUnu+VvRQCUSGKgYUHOzNAvoyBHg9delQVxff6Ymk3RCr6jQ76ecBeqIdMN50ImcTTkQ4f15NUH6vJZMLQk6WIlmcs/fXOuHtKHt0Mc//iGVz1+58tKiGUbMOREEqSfFVxDm3jZrFpCb27WlBIiogy++AG69Nfh6JkKLgLyteR2CFACebflb84MeBtq/H7jvPn68O8NAhdTX2do8es05kUNOgbrTp6XepccfB3r1kobGiKjLvv5a+uj16RPc48qryr2Ge9oTIaLaVY3yquAKSlZXA+vWAadOBdeeaNJN7QZQlPM39OGuk+IORrKz9ZNzIlewhecEQVrGFZC37hERdeBeIDAuLrjH1TbK+7zK3c/N3Q4uXOgfe1RIPXKGPtx1UsxmKWF22jTpWu9BCtD1wnPLlrGfmKiLWtfDRWNjcI9L6ivv8yp3P7czZ6Trnj2Da080YaBC6lFrbR6tTAUOVKDOH0GQhoOIKGhjxkjXu3YF97jU5FRYLVZP4mx7Jphgs9iQmhxcQcldu4CBA4GEhODaE00YqJB61Fibp7N8mEjrrEBdIMEuOUBEAKQi2OvWAT/8YXCPM8eYUZQlfV7bByvu24VZhTDHBNfbe/PNwDPPXCqvTx3xn4bUE+m1edz5MO17cXytGxQp/pKFAwl2yQEiAiD9JpgxA5BZqsuLY5QDJVNLMNji/Xm1Wqxdnpo8aRIwe3bwbYkmrKNC6hEEqTejpkb5Oinu1/I31KR2TRZ3gbrqamDmTGl5VX/MZuDcOamaLxEFTRSliXTDh0urdwRLaBFQXlWO2sZaJPVNQmpyatA9KQDw3HPAhAnAj38cfBuMgHVUSPsiuTaPWvkwcrmThe+5B5g7t/N958xhkEIUApNJSmJdtAg4eTL4x5tjzEizp2HamGlIs6d1KUj5xz+AZ58F/vWv4F8/2jBQIXVFqk6KGvkwXbV0KTBvXscAzWyWtnNqMlHInn9e+khlZwOty89FTEUF8MtfSvkp994b2dfWIw79kDYovTZPWZmUOBuIr3WD1OJeSuDoUSkn5dFH2ZNCFEa7d0vr/UyaBGzcGJlR33//W/p6a2oCPv1UmvETreSev1nwjZQTTPDhHvpQinsqcKB8mNTgphYqKjZWqiNDRIoYPx7YsAF4773Izbr57jvpq/Hdd6M7SAkGh35IGVqaBgxENh+GiHRj8mRgzRrpa6CkBPjrX33/lglFczPw6qtAQwNw1VXAV18BI0aE9zWMjIEKhZ8WpwEDxl43iIhCVlwM/OIXwC23AB99FPrzff89sH49MHIk8PDDwJYt0nb+HgoOc1QovLQ+DRhQPh+GiHRJFIEPPgCeflrKXxk7Fti2DRgwIPjn2rQJeOIJaTLhlCnSDJ9rrw17k3WNOSqkjmCmAauVtKp0PgwR6ZLJBGRmAhMnSjkkH3wAXH65dN/ttwMXL0pDNsOHAxYL0L8/8M03UoLs4cPAoUNSMbkHH5R+A2VnA/ffD/zgB+oel94xUKHw6so0YPZwEJGGmEzSTKBJk6TbFy8CN9wgBSPbtwOvvSbN2hk7Fti3D+jbVwpgRo6UOowB6WtMS7n5esZAhcIr2LL4Tqe0gnLbXhirVUp8Zc4IEWlA9+7A737nvU0QpEu3blynR2n856XwCrQisMkE2GzSflpNuiUiCsBslioIMEhRHv+JKbzkTgMGpJ4UX7nc7m35+dJPFiIKD0GQih8WF0vX/HyRDjBQofCTMw1Y62vvEBmN1mobEcnEHBVShsMhpbz7S5LV09o7RHrnHmZt34PpHmZlHSHSMAYqpJzOpgEHm3RLRF0jCJ0Ps5pM0jBrdjZn25EmceiH1BFM0i0RdR2HWUnnGKiQOrj2DlFkcJiVdI6BCqmHa+8QKY/DrKRzXOuH1MfKtETKca+/VVPjO09FC+tvUVTiWj+kH1x7h0iergT17mHWnBwpKGkbrHCYlXSAQz9kLCxoRUYVSh0UDrOSjnHohzrS61AM1w0io/JXB8XdIyI32NDrZ5sMSe75m4EKedPryT5cX+REWuPOMfE3xZg5JqRTcs/fHPqhS/S6SGCgglYA1w0i/WIdFIpyDFRIEuhkL4rAww8Dr7+uvdwPfpGTkbEOCkU5BiokCXSyB4BTp4Bf/Up7i5nxi5yMjHVQKMoxUCFJsCdxLQ0HBftFzplBpCdcboKiHAMVkgT7a8w9HDR9OvDII8D588q0S45gvsi51D3pDZeboCinaKDy8ccf4+c//zkGDRoEk8mEzZs3e90viiKeeeYZJCUloWfPnsjIyMCRI0eUbBL5E+hk709TE7B6NdCrFzB5siJNC0juF/mWLfpMFiZiHRSKYooGKmfPnsV1112HlStX+rx/6dKleOmll7B69Wrs3LkTvXv3RmZmJi5cuKBks8iXzk72cm3Zol6wEuiLPDubM4NI3xwOoLISKC0FNmyQrisqGKSQ4UWsjorJZMJbb72Fya0nMlEUMWjQIMydOxdPPPEEAKChoQEJCQlYt24d7rrrLlnPyzoqYearjkqwzp0DevYMX5uC4a+gVVmZNMwTSGkpy/kTEQGKFwjUfB2ViooK1NXVISMjw7MtPj4eN910E3bs2OH3cU1NTXC5XF4XCqO2v9r+/Gdg4MDge1jmzVOkabK41w2aNk26dn+oODOIiEg+DeXzqbYoYV1dHQAgISHBa3tCQoLnPl8WL16M5557TtG2Rb22iwT27Ol7MbPOtM0zam4GVq0Cjh4Fhg0DHn0UiI0Ne5MD4hRPIlKCEZcl8Ffp253PF+G8KN3N+lm4cCEaGho8l+rqarWbZGz+cj86c/XV0vX8+VKS7eOPAytWSNe9eknbI41TPIko3DTU6xA2Gqz0rVqgkpiYCACor6/32l5fX++5z5e4uDhYLBavCynMPRz07rvy9i8okIKRgoKOf8yCcOn+SArXFE/WYCEiQL9LjgSiwUrfqgUqQ4cORWJiIrZt2+bZ5nK5sHPnTqSkpKjVLPLHbAZuv12aPRPI228Dy5Z1vk9BAZCRAcyeHbkaLKFO8TTiryciCp4Gex3CRoP5fIoGKmfOnMHevXuxd+9eAFIC7d69e1FVVQWTyYT8/Hz89re/xdtvv439+/fj3nvvxaBBgzwzg0iD3nwT6NHD//0mE/DQQ/I+oNu2AStXRrYGS1eneBr11xMRBU+DvQ5ho8F8PkWTaT/77DOkt5kSOmfOHADAjBkzsG7dOsyfPx9nz57Fgw8+iNOnT+PHP/4xtm7dih6dnQhJXeXlQGd1bkQRaGgI/nndNVjaFQVURNtkYTkC/XoymaRfT9nZ+k+iI6LANNjrEDbufL6aGt/feSaTdH8E8/kU7VFJS0uDKIodLuvWrQMg1VZ5/vnnUVdXhwsXLuDvf/87hg8frmSTKFRKfvC2bAF+/3tpppCWGPnXExEFT4O9DmGjwSUbdDfrh0IQjkRQuR+8mC7+aS1YoN7MIH+M/OuJiIJn9FmEGluygYFKtAhXIqjcD+jjj3e9rWrNDPLHyL+eiCh4Gux1CDsNLdkQsRL6SmEJfRn8Fe9xf6CCjZDdzwd4P2f755s/X5r909XMd7NZKsevRoG4tgRBCuoCjdlWVOj7i4mIguNryRGbTQpSuAZTQHLP3wxUjM59kvWXY9HVk6zcD6i7Mu2hQ8CaNcG3f/lyKVFVbXKDMyKKLkasTBshcs/fqpXQpwgJJhE0mJkwDoc0yyXQBzQ29lKgUVcnJcwG4+jR4PZXinvMtn1wZrXy1xNRNAt2FiEFjYGK0SmZCBrsB3TzZmkKcjDByrBhl/7//HlpwcMjR6Qy/QUFkV2lWW5wRkREYcOhH6MrK5MSZwMpLY3cr4Lz56Vk20BDQW1zVPwFONnZkam9QkREYSX3/M1ZP0anxWl0PXsCq1dLvSOdmTOn8yAFuFQojoiIDImBitFpeRrd0qVSsNL+tc1mafvSpVLvSydDRYIJKPvnFhRnWVF2z48huLpQFZeIiDSLQz/RQsvT6Nwzg44elXJSHn300pTk2bOl9YB8cI4C8rKA4/GXtlkbgKKvh8Hx9tddbo7QIqC8qhy1jbVI6puE1ORUmGMiEMhx9gCFA/+OSCc4PZk60uMXWGYm8MEHHTY7RwE5UwERANp0FJla/5pLPu9asOI86ETe1jwcd10K6KwWK4qyiuAYpWBA5yuQtFql3jC1A0nSD/4dGZ8ev8f9YKBCxuCjR0UwAfZ84LgFXkGKm0kErC6g4rYtMN8+SfaH2HnQiZxNORDh/ZEwtb5IydQSZYKVcBfko+jEvyPjM1ggykCFjOH8eWntnzbK7ED6zMAPLV0HpH0v70MstAiwF9m9elLaMsEEq8WKiryK8A4DKVWQj6IL/46Mz4CBKGf9kDH07ClNQW6jto+8h9b2gVT2fsoUICtLGkaaPVsKftopryr3G6QAgAgR1a5qlFeFeYVkrsxM4cC/I2MTBKknxVe/gntbfn7XlyvROAYqpH2bN3sFK0ln5D0s6QwAUZRmBh1+H8UnPkDZOysh9O7VYUpzbaO8gndy95ONKzNTOPDvyNiiPBBlZVrSh82bPZVpUw9+BWtDKWosgNhJjkrqsU5mBm3dAsfkyZ5icUl95a18LHc/2bgyM4UD/46MLcoDUfaokH707AmsWAHzto9Q9LVUWt/Ufri29XbhVmDLSGlm0PF2Q581FmDKVOD577ag+K5rUPb4L3Hz5T+A1WL1JM62Z4IJNosNqclhLoynxYJ8pD/8OzK2KA9EGaiQLjne/holnw/DYJf3dqsLKNkEZB+SelLaT18GLvXCLLoNmD7qK6T324xhi/ph2r/iIO3u/QD37cKswvDXU9FyQT7SD/4dGVuUB6IMVEi3HG9/jcpnT6P0Xzdjw1sxKF0HVBQCjoNA+ZDW4R4/n+v222sswIsJR/FE3ZUYbBnsdZ/VYlVuajJwaWXmwd6vC6tVfia/IEjrOhUXS9cGTaqjToTj74i0KcoDUU5PJmNwT90DAFFE8bXA9JzgnsIkAgPOAn+4mIZ/J/XDwJ9MxuD+Q7RfmdZgtRUoRAYqCEbtaLnCeBewjgpFnzYfYrm1VjpjbQCK4rLhWLw5DI1TiAFrKxBRJwwUiDJQoejU+iEWqo7Bvnem35lBcnjK8TdpNFhhkS8i0jEWfCPjCCb/wmwG0tJgvncGiuKk2ivtZwbJ5Q5w8pu2QPhgq/byPqK8tgIRRQcGKqRtTqfUa5CeDkyfLl3b7dL2AByLN6OkKbvDzCAEEbiIJqA6Hih/6HbZrxsxUV5bgYiiAwMV0i53/kX7XoOaGmm7zGCl8oVzKO2diw2uiXjuy4EwIfheFk85fpmvGxFRXluBiKIDc1RImxTMv3DOuBF5l+/2qlYbSOk6IK1SWrm5fEQP1P7gKiQlXIXU3/4vzL1lLj4Ubu5/o5oa32uAMEeFiDSMOSqkbwrmXzjW70Llfzei9PRk/PnQaAw4A7/DQSYRsDVcKsdvzwfS77qA6SMOIL3fZtif7gvnjBuDbkNYRHltBSKKDgxUSJu6kn8RRNKtuXcfpC1/C3cXf4k13bJ9DgfJLcefM3S3esEKi3wRkcExUCFtCjb/QoGkW7nl+EUAD1+xG69Pk9YNEs7KXN45XBwOoLISKC0FNmyQrisqGKQQkSEwR4W0KZj8iy1bwlL0TLhwHuUvzUXtG39C0jdNSD0GmEUEXTzO2gAU/Xs8HOt3yX8QEVGUYY4K6Zvc/AtAqkbrK5hxb8vPl1UDxdyjJ9Lmr8K0hRuQdswEc2v3SW2QubKe4SDHKK69Q0QUIgYqpF1y8i+USLpt97pJQY7keArF2Q9BuLt1GCo5WTvTmomIdKSb2g0g8ksQgP79gSVLgFOngIEDpeCh7doWShU9cziA7GygvByplRWw7rs/qHL8nkJxQ6RpzThxApgyBXjzTeaOEBEFgYGK0el1AavOVgRu234li565y/EjDUUzXkGOZTdMYnBrB3UYNpo6FVi6FJg9G4iNDb5NRERRhkM/RhbCTBhVBVORNjVVCmDa57G4mUzSMuipqSE1ybF+F0oqxncsxx9Ah2EjQQDmzgV69QLmzw+pTURE0UATgcrKlStht9vRo0cP3HTTTdi1i7MlQhaG8vOqEITgkmMjWPSsfaG4gWf9l+JvWyjOJ0EACgoYrBARBaB6oPKXv/wFc+bMwaJFi/D555/juuuuQ2ZmJk6ePKl20/Qr2JO9lnQlOTaCRc/aFopbXT8eQOeF4sydTP5vjgEKywvw2PKfoPDPs9HcdD5s7SQiMgrV66jcdNNNGD9+PFasWAEAaGlpgc1mw2OPPYYFCxYEfDzrqPhQViYN8wRSWgqkpSndmuAUF0vDVIFs2ABMm+a9TYV8HF/rBtkapCDFcdD/4+ZnAMtuBoQ2PxXMLcCcK7KxNHezYu0lItIKuedvVZNpm5ubsWfPHixcuNCzLSYmBhkZGdixY4fPxzQ1NaGpqclz2+UKMmlAz+SeiJWaCRMJoSTHtia/RpJj/S5knz2D8v+6B7V1R5D06ZeeQnH+zM8ACm7puF0wAQWntgArJzNYISJqperQzzfffANBEJCQkOC1PSEhAXV1dT4fs3jxYsTHx3suNpstEk1VXzCJsUrOhFFahJJjw8k9HDSt+ADSrssOONyz7ObWG+0PsfX2spNb0Jz7kJRb09ysQIuJiPRD9RyVYC1cuBANDQ2eS3V1tdpNUl6wibE6PNl76H1F4M2bpforfqy6sXW4x98UZ5N0/6rPXwUef5yzg4go6qkaqAwYMABmsxn19fVe2+vr65GYmOjzMXFxcbBYLF4XQ+tKYqzeT/Y6XRFYaBFQVlmG4v++E2WHtkLIfQQYMsRrn6OXyXsuz36cHUREUU7VQCU2Nhbjxo3Dtm3bPNtaWlqwbds2pKSkqNgyDelqifhQT/aCICXlqrVWjc5WBHYedMJeZEf6+nRMd05H+sYs2If9Fc73lgHz5nmCwmHfyXu+tvtxdhARRTPVZ/385S9/wYwZM7BmzRrceOONKCwsxKZNm3Do0KEOuSu+GH7WTyizYICuzYTprCqsRgOFgBScEeQ86ETOphyI8P4omVrHd0qmlsAx7GfAqlVoPnIQvQa8CsEE38M/opSIe+63QGwLZwcR+aTXitvkRRezfgDgzjvvxKlTp/DMM8+grq4O119/PbZu3SorSIkKoSbGBjsTxp0P0z5+defDaHjoxS8FAy+hRUDe1rwOQQoAiBBhggn5W/ORnZcNc34+YgHMWVkvze4R4R2stD7FnH9cClJ0NzuIJxBSmhF/SFGnVO9RCZXhe1QEQZrdU1PjO0/FZJI+pBUVoZ8Q3K/lb6gpnK8VKf4CL3euToiBV1llGdLXB65ZUzqjFGn2NM/t+SsnY9nJLR17Sv4BLP27NNzT678QuOelzxLE5j2ujXWDeAIhpSn8eabIknv+1t2sn6gTycTYrubDaFUEKvTWNsqrRdN+v6W5m3HuqXNYPiwXsy0ZWL5VGu5Z+nfpftmzg5wLwjYzyJMMvL8YZZVlEFqC+HfR65INpB96rrhNIVF96IdkcCfG+vq1WlgYvl8QQRaKEy42o/ydVaitP4qkhGFInfQozN018MveLZjAq4uF4pL6yhua8+zXZmgkNikJ+dNaV4OunQ98WuDZP6jZQe6ZQYC0MnMXOA86kbc1D8ddl/69rBYrirKK4BglI/G6sxOIySSdQLKz9dMTR9oTgc8zaRMDFb1wOKQveiXH/4PIh3GunY+8r5bheJ/WXy91gPWTJ1A0eg4c93XtZBl2EajQm5qcCqvFihpXjc88FRNMsFqsSE1O7XxoxB1gLFsGCELQs4MEE1Be8iJqh55BUtLwoIJGf8nANa4a5GzKkZKBOwtWeAKhSNBzxW0KCYd+9MSdGDttmnQd7l+nMgvFOY/+DTnHCnC8t3cXa01vATnHCuBcq5GaHxGo0GuOMaMoSxqaM7Ubp3HfLswqhHnzlsBDI0uXAufOAcuX49EbHoS5BfAR+0hEKafl0V2AcxRgzwfSZ4iYfvIVpP/zcdif6iXrfQiUDAwA+VvzOx8G4gmEIkHPFbcpJAxU6BIZ+TDCHwqQd6hQOoW120VsvZ3/1TIIFzVQ+j1CFXodoxwomVqCwRbvmjVWi1XqjRieLX9sPTYWyM9H7Mo1mHNFa4Xb9g9rMzvobyOAnKnA8XZ5aHKDxvKqcq/hng7Ng4hqVzXKqzrJS+IJhCJBzxW3KSQMVMhbgEJx5d1rpeEeP98Vogmo7iOg/J1Vyrc1kAgmIjtGOVCZV4nSGaXY4NiA0hmlqMirkIZMupikvDR3M+YN7Lh2kFkE5n0CLN4G5GXBb9AoAnj4YAFen5OBshdnQ7jQsVBcV5OBvfAEQpGg94rb1GUMVKijTqrC1tYflfUUcvdTXATL8ZtjzEizp2HamGlIs6fBHNP6hRnC0IhndtCQhzB7J7xmB5UPAY7Ho9OZQad6A7+K34b0syul4aCFk712CToZ2BeeQChSdLq8BoWGybTkm59CcUkJwwDfC1t33K+V6rODIpGI3JkQh0Zi43oif+Zq4CvLpdk9AGr7BNeMGgswBVvw3MyhuPry4UhKuho3P7pEfjJwZyI1M41I7c8zRRwLvlFQhIvNsD/VCzW9BU9OSlsmEbCeNaPihXMwd4/tODsIgPWMWVuzg5QWzqJ98+d7ZgaV2YH0mUG2pV01XGsDMK3neLwY+1nr3Zfa57UEQKApym6sTEtEMrHgGynC3D0WRaPnAJCCkrbctwtHz/EEKbqYHaS0cA6NtJkZlPrTh2FtNHV4HzrV7uVrLMCL3XfjieYf+k8GllNHxb2AZXm5FJwoNTONiKIOe1SoS3z1lNjOmFHY2lPi7nk53tt34q1JBKxnYlBx/TqYrbbo+OXtq46KzRbS0Ig7GATgs4dLFhHo3QwsumoWrv/xHfim6Vsk9U1CanLqpTwbvw1g2Xwi6hq5528GKtRlneWelG0uRPo/Hw/4HKXrgLRKRM/JTYGhEV9BY1cFtTIz110hohAwUCFVFa95DNPrVgTcb0MJMO0AeHILkTtorDlxCI9XrsE3vbrYw9L6bTBvYIBgxYgLWBJRRDFHhVTVdtZPp/udaf0fLioWEnP3WKRNzsfdj67GarNUKC6o3BW31uBm2cktaJ6YAcyeDZzvWH/FcAtYEpFmMVAhRaROehTWM2a/J0uTCNgagNRjbTby5BYWjsWbUdKUjcGudnfIDVzcKzM3bgNWrpRWZ5482Xsfls0noghhoEKKkDU7aCs6VF0FwJNbGDgWb0blC+dQ2jsXG1wT8VyFHSYE18vitYLzli3ewYqeyua3nZVUVsYeOyKdYcE3UozjvqUoWYuOdVRcUpDiOOjngVo4uRmAuUdPpD1xKU/o2oWTkde0RapmK0PbFZwFE1D+zy2o3bMeSZcPQeotN8NstQauDaN22XzOSiLSPSbTkuI8s4PqjiBp9Z+Rus/luyclQgmYQouA8qpy1DbWyp+GaxDChfPYtvzXyLrwP1Kyra+EW1Hq6Tr3WyC2RVqdOS8LXgGO1WJFUb9pcMx6sfUxbd5QrSRGc1YSkaZx1g9pk/vkAahycnMedCJva57XisFWixVFWUXyq68awPyVk1Fwaot0o22w4p7184m0npBzlLQ6c/uFDz1Va5OfgOOZ4rDWhgkLzkoi0jzO+iFtUnFRMedBJ3I25XgFKQBQ46pBzqYcOA86FXttrQm0MvPSv0vDPX5XZ26NaPK/XgGhXzxwyy3An/7ktYClqjgricgw2KNC6ojwmjBCiwB7kb1DkOLmXnyvIq8iosNAag9DNTedx6o35uHoyUMY9sY2PLpLGu4BIHstIU/RPgAYPx7YtUuRtgaluBiYPj3wfhs2SOX+iSji5J6/mUxL6vCzOrNSyqvK/QYpgNRDUO2qRnlVOdLskWmXFoahYuN6Iv9XrQm3H08GWrZ47pO7OrPXfrt3AzfeqH6woqdZSUTUKQ79UFSobZQ35VnufqHS5DDU5s1AdrbnpqcYXwDu/c6bgdk/BTJH7MbslZNw/rzMJ1BCaqo0nNh+EUg3k0nKpVF7VhIRBcRAhaJCUl95v5zl7hcKoUVA3tY8T55HW2Lrfw/+9UFs+/+3QWiJcM2PzZul1Zlzc5E6/CewCr09ibPttS3aN/lOoNd/AStvBD64Clj5zbvo9fu+mLzsxsi23y2cK1YTkaoYqJDx+CjwlZqcCqvF6v+kCxNsFhtSk5X/hR1oGAoA/n3+38j43wzYi+yR713p2RNYsQLm9z9A0bT/DwA6/Lu1Ldo3ZSqwZaTvp9ri2q1esKJi4jYRhQ8DFTIWp1OalpqeLiVTpqcDdjvMm7egKEv6hd3hpNt6uzCrMCKJrMEML6k9I8kxyoGSqSUYbPE+2VtdQMkm4PZ/tQlS2seArbe3uHarNwzkcACVldJspA0btDMriYhk46wfMg4ZBb6co9AhgdVmsaEwqzBiCaxllWVIX58ue3+1ZiS15ZmddKoCSVPvR+oxaSrz7J9Kwz2B5A6ajBUPvBXGBkV21hgRhR9n/VB0EQSpVLqvuFsUpWDl4YfhWL4c2devQ3kyUHvupCpTgt3DUDWuGp95Ku2pMSOpPXOMWXptexow8BWgcjcA4Eh/eY8/8t1Rz/+HPCWbZfGJogoDFTIGOQW+Tp0CfvUrmAGkeU5saaG9bhd+2ZtjzCjKKkLOphyYYJIVrACRm5EU0K5d0hTk3btx9bfABzIecvVlwwCEYUq2v16zmhppO3NPiAyHOSpkDMGuuOw+sTlDyP3wkw8j5zn95X50JhIzkmTbtQtobESB+adS6Vp/sVbrfQW/+t/Qp2QH6jUDgPx8ro5MZDAMVMgYgi3cFeqJzf3Lvn0vThABkGOUA5V5lfj7PX9H/57+x1AiOSMpKH36oOfb7yDbMl663T5+aL2dbRmP2LienU7JBoD8LQ9D2PC6Z6ZWByyLTxSVGKgYhY8puVElUIEvX7p6YgvjL3tzjBkTrpyAP/78jzC1/tdWpGckdcXmObsuBSvtZFvGY/OcXfIqAzedQvlvfuW/Z0pur1mwvWtEpGkMVIwghCEIw+iswFcgwZ7YFPhl73casMWKkqklml/ZefOcXTj3ZCNyB03GxJ5jkDtoMs492YjNc6RS+rIrA7vL8fvqmWJZfKKoxGRavWNy4SXuAl/tZ4QEEuyJTaFf9o5RDmSPyFZ1kcJQ9OzZx+8UZNmVgVvLrQgQUT4EqC2YgSSxAqm/eAxmd69ZTY3v3iyTSbqfZfGJDIV1VPRMEKSeE38nZfcXd0VFdNWYcM/EqamRhmC++cb3fl399ykrk3qtAiktjejCi1rmXr3a35RskygVkasolArI5WUBx+Mv3W89Y0bR6DlwxP9ICsAB72ClTa2cqAnMKfoYrH6Q3PM3h360Sk7OCZMLfXOvzHz33cCaNdJJLJzrvXDBu6C5p2QDnZfj3zISyJkKHG/3nVXTW0DOsQI4Gz5lWXyKTlE8xK9YoPK73/0ON998M3r16oV+/fr53KeqqgqTJk1Cr169cMUVV2DevHn4/vvvlWqSfsj9g2RyYWBKrPfCBe+6JFA5/uxDUk+KCHQoxy+23s7/ahmEn/+MZfEpuoRhlqGeKTb0s2jRIvTr1w/Hjx/Ha6+9htOnT3vdLwgCrr/+eiQmJqKgoAC1tbW499578cADD+CFF16Q/TqGG/qRUQbe84XMIQj5lOgy9VUh1WaTghSeNP3yVKZtqEHSA/lI3fMNzC1AmR1Inxn48aXXLUfa5HyFW0mkEQYe4pd7/lY8R2XdunXIz8/vEKi89957+NnPfoYTJ04gISEBALB69Wo8+eSTOHXqFGJjY2U9v6EClUB/kADQvz+wadOlwMNuD5xcqMM/YE1rG/RccYW07eRJQ4wZR5w7MAdQfI2I6TmBH7IhcTamPfSywg0j0ggD/yDVfI7Kjh07MGbMGE+QAgCZmZlwuVz48ssv/T6uqakJLpfL62IYgXJOAODbb4GMDClA2bKFQxCR1n5YLiMDmDkTiIuTviT4bx2cNkNz7hk/gSQlDFO2TURawiF+9QKVuro6ryAFgOd2XV2d38ctXrwY8fHxnovNZlO0nREVzB+ae2wSYHJhpET5OLFiHA6gshKpf3wf1kaTJ7m2PZMI2M6YkTrp0ci2j0hNrB8UXKCyYMECmEymTi+HDh1Sqq0AgIULF6KhocFzqa6uVvT1IiqYP7S2FVCzs5lcqLQA1WgFk4iyJQ+j+J+vo6yyDEKLRioD66VisdkMc8ZEFF3zBAB0CFY8M4NGz4G5u7xhYSJD4CzD4Aq+zZ07FzNnzux0nyuvvFLWcyUmJmLXrl1e2+rr6z33+RMXF4e4uDhZr6E7gQpatdd2+nFamu7GJ3Wlk2E55yh33Y9TwOZfAQhyRWCl+Er29awarc0g1nHfUpSsBfK+WobjfS4FVdazZhSOngPHfUsDPocnWVeHRfOIOnDPMszJkYISX/WDDD7EH1SgMnDgQAwcODAsL5ySkoLf/e53OHnyJK5oTUj88MMPYbFYMHr06LC8hu509gfZGQOPTWqGn39j5yip7kf7d8q9IrBq5e91XLHYcd9SZF/8LcrfWYXa+qNIShiG1EmPyupJcR50Im9rnte6QpoIGolC4a/qttUaFbMMFZv1U1VVhW+//RZvv/02CgoKUN5adOyqq65Cnz59PNOTBw0ahKVLl6Kurg733HMP/vM//zO6pycDvn8Jd0aH2d664yPzXjAB9vzW4mQ+emVNMMFqsaIiryKyv+gNPJ2xM86DTuRsyulQ+dZdYE4PayYRdSpKK9MqFqjMnDkT69ev77C9tLQUaa0n1WPHjuGRRx5BWVkZevfujRkzZmDJkiXo1k1+R48hAxXgUm7B1KnSTB9fDHrC0ST3yb/NsJzsuh8zSpFmT1Owce0YeDqjP+4S/Z2t0Nw/pg82Xfss0n72GPNciDRA9enJ69atgyiKHS5pbb4YhwwZgnfffRfnzp3DqVOn8OKLLwYVpBia2QxMmAD88Y/hLwFPwfNRjdaz0m8AclcODpsonM5YXlXeaZACAN+2nEHGvidgf6oXnGvnR6hlRBQqrvWjdUqUgKeuafdeyK77IXPl4LCJwumMwQSDnnWDGKwQ6QJXT9YLg41N6lrreyGcqIH92OOoaf7G94rAaueoRFHF4rLKMqSvlzHc1cokSjOJKl44x2EgIpWoPvRDYeZeEXjaNFZAVVvre2GefjeKJq8G4GNF4NbbhVmFkZ8aG6FFE4UWAWWVZSjeX6x67ZjU5FRYLdYO74M/ogmo7iOg/J1VCreMiELFQIUoBH5XBLZY1Z1lovCQofOgE/YiO9LXp2O6czrS16fDXmSH86A61XnNMWYUZUnBmdxgBQBq649euqGX4nhEUYZDP6R7WijwpYU2+G5Y+IcMtTwN2Fcdlc54VmLWYXE8Ir1TfXpypDBQiW4s8BVZgaYBq5aX04bQIqDs6DZM/VMWvo0Tfde4aZuj8te/+S6O5x4mY9I6kSKYo0KG5/5l3/6k6a4Kq9YwRMg0PAQRaBqwCBHVrmqUV5VHsFXezDFmTLh6Iv44/AmYEGDdoBhzp2s4AZDW09LQe0AUbRiokC4JLQLytub5nG3j3pa/NV87iwPK5XRKM3bS04Hp06Vru10zKzPLnQYc8doxPjjuW4qSIfMw+Kx3z471rBklQ+ZJ6wZ1soYTAO/1tIhIFayuRroUzC/7iFaFDYUO1ueRWxMm4rVj/Ai4blAUFscj0hsGKqRLevplL4sgdD4EYTJJQxDZ2apOTXdPA65x1XRaOyY1WTtLzpu7x0oJs74EWxyP9YyIIo5DP6RLevtlH5BOhiA6mwasau2YrkpNlWb3tK8342YyATabtJ/Gh+WIjIqBCulSoAJfJphgs9g09cu+UzoagtBs7ZiukFscb8sWafitfTDpHpZjsEKkGA79kC65f9nnbMqBCSavYQhd/rIPcghC7botjlEOZI/I1mbtmGC5i+P5qqNSWCgNt9ntmh+WIzIq1lEhXfNVR8VmsaEwq1Bfv+yDWJ/H+a8trB2jBH/5J2Vl0jBPIKWl0vIWRCQLC75R1FC7dyFs3LN+AO9gpU3hMecoaLYqrGEVF0s5KYFs2CCtxUVEsrDgG0UNc4wZafY0TBszDWn2NH0GKUDA9XmEydnGrB2jdcHODCKisGKOCpGWOBxSroOPIYjyyjLj1Y7RA/fMoEDDcqk6Sdwm0hkGKkRaYzb7zHUwXO0YvXDPDMrJkYISX8NyhYVMpCVSCId+iHTCcLVj9CTAsJzaFYOJjIw9KkQ6oceqsIbSybAcESmHgQqRThiudowe+RmWIyLlcOhHTwRBqulQXCxdc+n5qGOoqrBERDKwjopeOJ2+K2cWFXF8PAoZpnYMEUUtFnwzEnchsPZvVZtCYHoMVoSLzSh/ZxVq648iKWEYUic9CnP3WLWbFX24IjARqYCBilG4S6v7W1m3TWl1PZ1cnGvnI++rZTje59LwlfWMGUWj58Bx31IVWxZl2FNHRCphZVqjKC/3H6QAUi9LdbW0n044185HzrECHO/tnWNT01tAzrECONfOV6llUcbdU8cVgYlIwxioaF2tzOJdcvdTmXCxGXlfLZPmq5i87xNbb+d/tQzCxeZINy26CILUk+JvRWBAWhGYCdtEpDIGKlpnsHVGyt9ZJQ33mHzfL5qA6j4Cyt9ZFdmGRRsD9tQRkTGxjoofmplVYbB1Rmrrj4Z1P+oig/XUEZFxMVDxwXnQibyteV4LwFktVhRlFUW+ToXB1hlJShgG1Mncj5RjsJ46IjIuDv204zzoRM6mnA6r1Na4apCzKQfOgyokGBponZHUSY/CesYMk5+5ZiYRsJ0xI3XSo5FtWLRx99SZ/IzBmUyAzaabnjoiMi4GKm0ILQLytub5XEfFvS1/az6EFhUSDB0OoLISKC0FNmyQrisqdBWkAIC5eyyKRs8BgA7Bivt24eg5nddTYYXe0Ll76oCOwYoOe+qIyLgYqLRRXlXeoSelLREiql3VKK9SKcHQvc7ItGnStU5PIo77lqJkyDwMPuvdfutZM0qGzOu8jorTKdWVSU8Hpk+Xru12TqXtijD01AkXm1G2uRDFax5D2eZCztYiorBjwbc2ivcXY7pzesD9Njg2YNqYaSG9FnWhMq1RK/Sqnbjdxcq0ihXtY6Vcoqgg9/zNZNo2kvrKSxyUux91ztw9FmmT8+XtHKDuhxADlC95GLXDziMpfrBu1r7RROJ2F1YEdhftE3t7b3cX7StZi64FK6yUS0TtKDb0U1lZiVmzZmHo0KHo2bMnhg0bhkWLFqG52btreN++fUhNTUWPHj1gs9mwdKl65dNTk1NhtVhh8lPkwwQTbBYbUpOZYBhxndT9cI4C7HlA+qRTmL75V0hfnw57kV2dxOcgaDJxWwbFivaxUi4R+aBYoHLo0CG0tLRgzZo1+PLLL7F8+XKsXr0aTz31lGcfl8uFiRMnYsiQIdizZw8KCgrw7LPP4tVXX1WqWZ0yx5hRlCUlGLYPVty3C7MKdfFL3XD81PNwjgJypgLH2/Uaav5kr+XE7QAUKdrHSrlE5IdigUpWVhbWrl2LiRMn4sorr8QvfvELPPHEE3C2+VX0+uuvo7m5GX/6059wzTXX4K677sKvf/1rLFu2TKlmBeQY5UDJ1BIMtngnGFotVpRMLYl8HRWS+KjnIZiAvCz4/mWv9ZO91hO3O6FI0T5WyiUiPyKao9LQ0ID+/ft7bu/YsQO33norYmMvJVBmZmbi97//Pb777jtcdtllHZ6jqakJTU1Nntsulyvs7XSMciB7RLY2KtOSxEeF3vIhwPF4/w9pe7JPs6dFpp0y1TbKq/gqd79IUqRoHyvlEpEfEZue/PXXX+Pll1/GQw895NlWV1eHhIQEr/3ct+vqfH8TLl68GPHx8Z6LzWZTpL3mGDPS7GmYNmYa0uxpDFLU5qPuR20feQ/V5Mlex4nbihTtY6VcIvIj6EBlwYIFMJlMnV4OHTrk9ZiamhpkZWXhjjvuwAMPPBBSgxcuXIiGhgbPpbq6OqTnIx1pV/cj6Yy8h2nyZK/jxO2wFO1rj5VyiciPoId+5s6di5kzZ3a6z5VXXun5/xMnTiA9PR0333xzhyTZxMRE1NfXe21z305MTPT53HFxcYiLiwu22WQUDgeQnQ2UlyP1RA2sxx5HTfM3PpNSTTDBarFq82TfmridsykHJpi82q+HxG3HfUtRshYd66icNaOwK3VUDLamFRGFj6IF32pqapCeno5x48bhz3/+M8ztvmReeeUV/OY3v0F9fT26d+8OAHjqqafgdDo79Mr4E86Cb6Q/7im+AHye7LWeAO2rjorNYkNhVqGm2+0WdNG+QHzVUbHZpCCFdVSIDEXu+VuxQKWmpgZpaWkYMmQI1q9f7xWkuHtLGhoaMGLECEycOBFPPvkkDhw4gPvvvx/Lly/Hgw8+KOt1GKhQSCd7DVRBVb0yrdZo4D0hIuWpHqisW7cO9913n8/72r7kvn37kJubi927d2PAgAF47LHH8OSTT8p+HQYqBHTxZM8qqEREqlE9UIkUBirUJZ2sGySYRJSvfQ61P7iaPRxERArhWj9E/nRSBdU5UkReFnC8YhFQIW2L+No7RETkEbE6KkSa4acKql7L8RMRGRkDFYo+Pqqb6rkcPxGRkTFQoejjo7qppxy/v4X2NLz2DhGRkTFQoejjowqqnsvxExEZGQMVij4+1g3Sczn+DgQBKCsDioula4HDVUSkXwxUKDq1Wzco9Rhgbei4do2bltfe8eJ0AnY7kJ4OTJ8uXdvt0nYihQktAsoqy1C8vxhllWXM6aKwYB0Vim5tqqA6ux1BzlfPAtBnOf7OasMAkAIzFrIjhfiqEM2p/dQZFnwj6gLdrr0jCFLPiY9p1wCkYMVqBSoqWI6ews695lb7xUF1E+STKhioEHWRLtfeKSuThnkCKS0F0tKUbg1FEaFFgL3I7hXct+Vexbwir0L7nyOKKFamJeoic4wZafY0tZsRHB+1YULaj0im8qpyv0EK4D21X3efK9IEJtMSGYGP2jAh7Uckk9wp+5zaT13FQIXICHzUhvFiMgE2m7QfURjJnbKvi6n9pEkMVIiMwEdtGA/37cJCJtJS2KUmp8JqsXoSZ9vTzdR+0iwGKkRG0a42jIfVyqnJpBhzjBlFWVKQ3D5Ycd8uzCpkIi11GWf9EBlNm9owSEqShnvYk0IK0+3UflINpycTEVFE6XJqP6mG05OJiCiidDm1nzSPOSpERESkWQxUiIiISLMYqBAREZFmMVAhIiIizWKgQkRERJrFQIWIiIg0i4EKERERaRYDFSIiItIsBipERESkWQxUiIiISLMYqBAREZFmMVAhIiIizWKgQkRERJrFQIWIiIg0i4EKERERaRYDFSIiItIsBipERESkWQxUiIiISLMUDVR+8YtfIDk5GT169EBSUhLuuecenDhxwmufffv2ITU1FT169IDNZsPSpUuVbBIRERHpiKKBSnp6OjZt2oTDhw/jzTffxNGjR5GTk+O53+VyYeLEiRgyZAj27NmDgoICPPvss3j11VeVbBYRERHphEkURTFSL/b2229j8uTJaGpqQvfu3fHKK6/gN7/5Derq6hAbGwsAWLBgATZv3oxDhw7Jek6Xy4X4+Hg0NDTAYrEo2XwiIiIKE7nn74jlqHz77bd4/fXXcfPNN6N79+4AgB07duDWW2/1BCkAkJmZicOHD+O7777z+TxNTU1wuVxeFyIiIjImxQOVJ598Er1798bll1+OqqoqbNmyxXNfXV0dEhISvPZ3366rq/P5fIsXL0Z8fLznYrPZlGs8ERERqSroQGXBggUwmUydXtoO28ybNw9ffPEFPvjgA5jNZtx7770IZbRp4cKFaGho8Fyqq6u7/FxERESkbd2CfcDcuXMxc+bMTve58sorPf8/YMAADBgwAMOHD8eoUaNgs9nw6aefIiUlBYmJiaivr/d6rPt2YmKiz+eOi4tDXFxcsM0mIiIiHQo6UBk4cCAGDhzYpRdraWkBIOWZAEBKSgp+85vf4OLFi568lQ8//BAjRozAZZdd1qXXICIiIuNQLEdl586dWLFiBfbu3Ytjx47ho48+wrRp0zBs2DCkpKQAAKZPn47Y2FjMmjULX375Jf7yl7+gqKgIc+bMUapZREREpCOKBSq9evWC0+nEhAkTMGLECMyaNQtjx47F9u3bPUM38fHx+OCDD1BRUYFx48Zh7ty5eOaZZ/Dggw8q1SwiIiLSkYjWUVEC66gQERHpj9zzd9A5KtSOIADl5UBtLZCUBKSmAmaz2q0iIiIyBAYqoXA6gbw84PjxS9usVqCoCHA41GsXERGRQXD15K5yOoGcHO8gBQBqaqTtTqc67SIiIjIQBipdIQhST4qv9B73tvx8aT8iIiLqMgYqXVFe3rEnpS1RBKqrpf2IiIioyxiodEVtbXj3IyIiIp8YqHRFUlJ49yMiIiKfGKh0RWqqNLvHZPJ9v8kE2GzSfkRERNRlDFS6wmyWpiADHYMV9+3CQtZTISIiChEDla5yOICSEmDwYO/tVqu0nXVUiIiIQsaCb6FwOIDsbFamJSIiUggDlVCZzUBaWlieSmgRUF5VjtrGWiT1TUJqcirMMQx6iIgoejFQ0QjnQSfytubhuOtSfRarxYqirCI4RnEYiYiIohNzVDTAedCJnE05XkEKANS4apCzKQfOgyzHT0RE0YmBisqEFgF5W/MgomM5fve2/K35EFpYjp+IiKIPAxWVlVeVd+hJaUuEiGpXNcqrWI6fiIiiDwMVldU2yiuzL3c/IiIiI2GgorKkvvLK7Mvdj4iIyEgYqKgsNTkVVosVJvgux2+CCTaLDanJLMdPRETRh4GKyswxZhRlSeX42wcr7tuFWYWsp0JERFGJgYoGOEY5UDK1BIMt3uX4rRYrSqaWsI4KERFFLZMoih3nxeqIy+VCfHw8GhoaYLFY1G5OSFiZloiIooXc8zcr02qIOcaMNHua2s0gIiLSDA79EBERkWYxUCEiIiLNYqBCREREmsVAhYiIiDSLgQoRERFpFgMVIiIi0iwGKkRERKRZDFSIiIhIsxioEBERkWbpvjKtewUAl8ulckuIiIhILvd5O9BKProPVBobGwEANptN5ZYQERFRsBobGxEfH+/3ft0vStjS0oITJ06gb9++MJlMajcnrFwuF2w2G6qrq3W/4KIcPF7ji7Zj5vEaG483NKIoorGxEYMGDUJMjP9MFN33qMTExMBqtardDEVZLJao+BC48XiNL9qOmcdrbDzeruusJ8WNybRERESkWQxUiIiISLMYqGhYXFwcFi1ahLi4OLWbEhE8XuOLtmPm8RobjzcydJ9MS0RERMbFHhUiIiLSLAYqREREpFkMVIiIiEizGKgQERGRZjFQ0aDKykrMmjULQ4cORc+ePTFs2DAsWrQIzc3NXvvt27cPqamp6NGjB2w2G5YuXapSi0P3u9/9DjfffDN69eqFfv36+dynqqoKkyZNQq9evXDFFVdg3rx5+P777yPb0DBauXIl7HY7evTogZtuugm7du1Su0lh8fHHH+PnP/85Bg0aBJPJhM2bN3vdL4oinnnmGSQlJaFnz57IyMjAkSNH1GlsGCxevBjjx49H3759ccUVV2Dy5Mk4fPiw1z4XLlxAbm4uLr/8cvTp0wdTpkxBfX29Si0OzSuvvIKxY8d6in6lpKTgvffe89xvpGP1ZcmSJTCZTMjPz/dsM9oxP/vsszCZTF6XkSNHeu6P9PEyUNGgQ4cOoaWlBWvWrMGXX36J5cuXY/Xq1Xjqqac8+7hcLkycOBFDhgzBnj17UFBQgGeffRavvvqqii3vuubmZtxxxx145JFHfN4vCAImTZqE5uZm/OMf/8D69euxbt06PPPMMxFuaXj85S9/wZw5c7Bo0SJ8/vnnuO6665CZmYmTJ0+q3bSQnT17Ftdddx1Wrlzp8/6lS5fipZdewurVq7Fz50707t0bmZmZuHDhQoRbGh7bt29Hbm4uPv30U3z44Ye4ePEiJk6ciLNnz3r2efzxx/HXv/4Vb7zxBrZv344TJ07A4XCo2Oqus1qtWLJkCfbs2YPPPvsMt912G7Kzs/Hll18CMNaxtrd7926sWbMGY8eO9dpuxGO+5pprUFtb67n83//9n+e+iB+vSLqwdOlScejQoZ7bq1atEi+77DKxqanJs+3JJ58UR4wYoUbzwmbt2rVifHx8h+3vvvuuGBMTI9bV1Xm2vfLKK6LFYvH6N9CLG2+8UczNzfXcFgRBHDRokLh48WIVWxV+AMS33nrLc7ulpUVMTEwUCwoKPNtOnz4txsXFicXFxSq0MPxOnjwpAhC3b98uiqJ0fN27dxffeOMNzz4HDx4UAYg7duxQq5lhddlll4n/8z//Y+hjbWxsFK+++mrxww8/FP/jP/5DzMvLE0XRmO/vokWLxOuuu87nfWocL3tUdKKhoQH9+/f33N6xYwduvfVWxMbGerZlZmbi8OHD+O6779RooqJ27NiBMWPGICEhwbMtMzMTLpfL80tOL5qbm7Fnzx5kZGR4tsXExCAjIwM7duxQsWXKq6ioQF1dndexx8fH46abbjLMsTc0NACA5/O6Z88eXLx40euYR44cieTkZN0fsyAI2LhxI86ePYuUlBRDH2tubi4mTZrkdWyAcd/fI0eOYNCgQbjyyitx9913o6qqCoA6x6v7RQmjwddff42XX34ZL774omdbXV0dhg4d6rWf+yReV1eHyy67LKJtVFpdXZ1XkAJ4H6+efPPNNxAEwefxHDp0SKVWRYb7vfJ17Hp7H31paWlBfn4+brnlFlx77bUApGOOjY3tkHul52Pev38/UlJScOHCBfTp0wdvvfUWRo8ejb179xruWAFg48aN+Pzzz7F79+4O9xnx/b3pppuwbt06jBgxArW1tXjuueeQmpqKAwcOqHK87FGJoAULFnRIUGp/aX+iqqmpQVZWFu644w488MADKrW8a7pyvER6lpubiwMHDmDjxo1qN0VRI0aMwN69e7Fz50488sgjmDFjBr766iu1m6WI6upq5OXl4fXXX0ePHj3Ubk5E3H777bjjjjswduxYZGZm4t1338Xp06exadMmVdrDHpUImjt3LmbOnNnpPldeeaXn/0+cOIH09HTcfPPNHZJkExMTO2RZu28nJiaGp8EhCvZ4O5OYmNhhVozWjleuAQMGwGw2+3z/9HYswXIfX319PZKSkjzb6+vrcf3116vUqvCYPXs2/va3v+Hjjz+G1Wr1bE9MTERzczNOnz7t9StUz+93bGwsrrrqKgDAuHHjsHv3bhQVFeHOO+803LHu2bMHJ0+exA033ODZJggCPv74Y6xYsQLvv/++4Y65vX79+mH48OH4+uuv8ZOf/CTix8selQgaOHAgRo4c2enFnXNSU1ODtLQ0jBs3DmvXrkVMjPdblZKSgo8//hgXL170bPvwww8xYsQIzQz7BHO8gaSkpGD//v1es2I+/PBDWCwWjB49WqlDUERsbCzGjRuHbdu2eba1tLRg27ZtSElJUbFlyhs6dCgSExO9jt3lcmHnzp26PXZRFDF79my89dZb+OijjzoMyY4bNw7du3f3OubDhw+jqqpKt8fcXktLC5qamgx5rBMmTMD+/fuxd+9ez+WHP/wh7r77bs//G+2Y2ztz5gyOHj2KpKQkdd5jRVJ0KSTHjx8Xr7rqKnHChAni8ePHxdraWs/F7fTp02JCQoJ4zz33iAcOHBA3btwo9urVS1yzZo2KLe+6Y8eOiV988YX43HPPiX369BG/+OIL8YsvvhAbGxtFURTF77//Xrz22mvFiRMninv37hW3bt0qDhw4UFy4cKHKLe+ajRs3inFxceK6devEr776SnzwwQfFfv36ec1q0qvGxkbP+wdAXLZsmfjFF1+Ix44dE0VRFJcsWSL269dP3LJli7hv3z4xOztbHDp0qHj+/HmVW941jzzyiBgfHy+WlZV5fVbPnTvn2efhhx8Wk5OTxY8++kj87LPPxJSUFDElJUXFVnfdggULxO3bt4sVFRXivn37xAULFogmk0n84IMPRFE01rH603bWjyga75jnzp0rlpWViRUVFeInn3wiZmRkiAMGDBBPnjwpimLkj5eBigatXbtWBODz0tY///lP8cc//rEYFxcnDh48WFyyZIlKLQ7djBkzfB5vaWmpZ5/Kykrx9ttvF3v27CkOGDBAnDt3rnjx4kX1Gh2il19+WUxOThZjY2PFG2+8Ufz000/VblJYlJaW+nwvZ8yYIYqiNEX56aefFhMSEsS4uDhxwoQJ4uHDh9VtdAj8fVbXrl3r2ef8+fPio48+Kl522WVir169xF/+8pdePzz05P777xeHDBkixsbGigMHDhQnTJjgCVJE0VjH6k/7QMVox3znnXeKSUlJYmxsrDh48GDxzjvvFL/++mvP/ZE+XpMoiqIyfTVEREREoWGOChEREWkWAxUiIiLSLAYqREREpFkMVIiIiEizGKgQERGRZjFQISIiIs1ioEJERESaxUCFiIiINIuBChEREWkWAxUiIiLSLAYqREREpFkMVIiIiEiz/h/HouniU8+NwAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Extraire la collone 'class' qui est en premier\n",
    "Y = adult_num.iloc[:, 0].values\n",
    "\n",
    "# Trouver les echantillons ayant la class 'N', \n",
    "# la negation est les echantillons ayant la class 'Y' (classement binaire)\n",
    "No_class = Y == 'N'\n",
    "\n",
    "# Ploter les echantillons selon les deux composantes\n",
    "plt.scatter(adult_pca[ No_class,0], adult_pca[ No_class, 1], color='red'  , label='No' )\n",
    "plt.scatter(adult_pca[~No_class,0], adult_pca[~No_class, 1], color='green', label='Yes')\n",
    "\n",
    "circle1 = plt.Circle((39.87348226,  18.02549656), 3, color='b', fill=False, ls='--')\n",
    "ax = plt.gca()\n",
    "ax.add_patch(circle1)\n",
    "\n",
    "plt.legend(loc='upper right')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6bc9368",
   "metadata": {},
   "source": [
    "**TODO: Analyse** \n",
    "\n",
    "- Pour quel but nous avons utilise l'ACP ?\n",
    "\n",
    "- Que pouvons-nous dire a propos de l'echantillon en vert dans les environs de [40, 18] ?\n",
    "\n",
    "- Quelle est la classe la plus homogene ? \n",
    "\n",
    "- Pourquoi ?\n",
    "\n",
    "\n",
    "**Réponse**\n",
    "\n",
    "- L'ACP est utilisée pour la réduction de la dimensionnalité des données afin de visualiser la structure d'un dataset dans un espace réduit. Cela permet ainsi de réduire la complexité des données tout en préservant autant d'informations que possible, facilitant ainsi la visualisation et l'analyse. Dans cet exemple, l'ACP est appliquée pour réduire les caractéristiques à deux composantes principales, ce qui permet de visualiser la distribution des échantillons dans un espace bidimensionnel.\n",
    "\n",
    "- L'échantillon en question est situé loin des autres échantillons de la même classe, c'est-à-dire en dehors de la zone où la majorité des ces derniers se regroupe. Cela signifie que cet échantillon est un point aberrant par rapport à la distribution générale des échantillons, indiquant un comportement unique ou différent par rapport aux autres échantillons de la même classe.\n",
    "\n",
    "- D'après la visualisation, la classe la plus homogène est la classe 'Y', qui est représentée en vert. Les échantillons de cette classe semblent être regroupés plus étroitement, formant un cluster plus compact. \n",
    "\n",
    "- Cette homogénéité peut être due à une similarité plus marquée entre les échantillons de la classe 'Y' en termes de caractéristiques, contrairement à l'autre classe où la variabilité semble plus importante, avec des échantillons plus dispersés."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "7d80279e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Luck is overrated\n",
      "BUT\n",
      "Good luck anayways\n"
     ]
    }
   ],
   "source": [
    "print('Luck is overrated')\n",
    "print('BUT')\n",
    "print('Good luck anayways')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}