LeastSquares.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "In this project, we address a multiclass classification problem using the **Least Squares** algorithm, focusing on the `load_digits()` dataset provided by sklearn.datasets. <br>\n",
    "Our objective is to accurately classify **images of handwritten digits** based on their pixel values.<br>\n",
    "The task involves predicting the correct digit (0 through 9) represented by each 8x8 pixel image in the dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.datasets import load_digits\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score,classification_report\n",
    "from sklearn.preprocessing import LabelBinarizer\n",
    "from sklearn.svm import LinearSVC\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset Overview\n",
    "**Number of Instances:** The `load_digits()` dataset contains 1,797 images.<br>\n",
    "**Features:** Each image is described by 64 features, representing the grayscale pixel values of the 8x8 image.<br>\n",
    "**Labels:** There are 10 distinct labels (0 through 9), corresponding to each digit.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graphical Representation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAGACAYAAAAwHvTQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA+/klEQVR4nO3de3zP9f//8ft7RzPagTnNh80IhTZEomwhh5Ql9IlPWSgVFekTn1TmPKXo5ENoUyRNGklhOXRaUZk+fOrzISZ9RBqT0zA9f3/02/vrbcN4vtjB7Xq5vP/Y6/16PV7P1+v9eB/ue73er7fLGGMEAAAAABa8insAAAAAAEo/ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUuSykpKXK5XGe8rVmzpriH6IiIiAglJCQU9zAKdfz4cT3wwAOqXr26vL29FR0dfcZ5ExISVKFChTPeX6FChWLbzsTERLlcrou6jvx+zcrKck976623NHXq1ALzHjlyRImJicXaw6c/v8qVK6dq1aopLi5OEydO1K+//lpgGZv9uGbNmgLP22XLlikxMfECt6Bw+/bt01//+ldVqVJFLpdL8fHxjtZ32oQJE5SWllZgev7j8/XXX1/U9ec/Lvk3Pz8/hYWFqXXr1ho5cqR27NhxwbV37dqlxMREZWZmOjdgC//+97+VmJjo8RwFLkc+xT0AoDglJyerQYMGBaZfddVVxTCay8s///lPzZgxQy+//LKaNWt21uBwubvllluUkZGh6tWru6e99dZb2rRpk4YMGeIx75EjRzR69GhJUmxs7CUcZUH5z68TJ07o119/1WeffaZJkyZp8uTJWrBggdq3b++ed8CAAerUqdMFradp06bKyMjweN4uW7ZMr776qqPhYuzYsXrvvff0+uuvKyoqSqGhoY7VvhgmTJigHj16FHsAmjBhguLi4nTy5EllZ2frq6++0uuvv64pU6Zo5syZ6tOnz3nX3LVrl0aPHq2IiIiz/lPiUvn3v/+t0aNHKzY2VhEREcU9HKDYECxwWWvUqJGaN29e3MO4LG3atEkBAQEaPHhwcQ+lxDp69KjKlSunsLAwhYWFFfdwztvpz6877rhDQ4cOVZs2bdS9e3dt2bJFVatWlSTVrFlTNWvWvKD1XHHFFbruuuscGfPZbNq0SVFRUef8IHzy5Enl5eXJ39//oo+pNKhXr57H43Pbbbdp2LBhat++vRISEtSkSRM1bty4GEcIwCmcCgWcxdtvvy2Xy6VXXnnFY/qoUaPk7e2tlStXuqeNHj1aLVu2VGhoqK644go1bdpUs2fPljHGY9mIiAh17dpVS5cuVUxMjAICAtSwYUMtXbpU0p+nKTRs2FCBgYFq0aJFgdMV8k8L2rx5s9q1a6fAwECFhYVp8ODBOnLkyDm36ffff9fjjz+uyMhI+fn5KTw8XEOGDNHhw4c95ktNTVXLli0VFBSk8uXLq06dOurXr9856+fm5uof//iHR/1BgwYpJyfHPY/L5dKsWbN09OhR92kSKSkp56xdVLm5uRo2bJiio6MVFBSk0NBQtWrVSosXLy4wr8vl0uDBg/Xmm2+qYcOGKl++vK655hr343GqDz74QNHR0fL391dkZKQmT55cYJ6ePXvq6quv9ph26623yuVyKTU11T3t22+/lcvl0vvvvy/p/05PWbFihfr166ewsDCVL19ex44dK3AqVGxsrD744APt2LHD41STrKwsdwAZPXq0e/qpp4lt2bJFvXv3VpUqVeTv76+GDRvq1Vdf9Rhv/iks8+fP18iRI1WjRg1dccUVat++vf7zn/8U7UE4g1q1aun555/XwYMHNWPGDPf0wk6FOnbsmIYNG6Zq1aqpfPnyuvHGG/XNN98UOMXv9FOhEhIS3Nt0+v6Rzr+3s7Ky5HK5lJ6eru+//97jlMn8+5599lmNGzdOkZGR8vf31+rVqyVJS5YsUatWrVS+fHlVrFhRHTp0UEZGhkf9/G3/7rvv1LNnT3fPPvbYY8rLy9N//vMfderUSRUrVlRERISeffbZc+5nl8ulw4cPa86cOe7xnn4E6+DBg3rwwQdVuXJlVapUSd27d9euXbsK1FqwYIFatWqlwMBAVahQQR07dtSGDRvOOYazCQ0N1YwZM5SXl6cpU6a4p2/dulX33nuv6tWrp/Llyys8PFy33nqr/vWvf7nnWbNmja699lpJ0r333uvevvyjU19//bX++te/KiIiQgEBAYqIiNBdd91V4NSrI0eOuF8Ly5Urp9DQUDVv3lzz58/3mO/rr7/WbbfdptDQUJUrV04xMTF655133PenpKSoZ8+ekqS4uLiL8poGlBoGuAwlJycbSebLL780J06c8Ljl5eV5zPvAAw8YPz8/s379emOMMR9//LHx8vIyTz31lMd8CQkJZvbs2WblypVm5cqVZuzYsSYgIMCMHj3aY77atWubmjVrmkaNGpn58+ebZcuWmZYtWxpfX1/zzDPPmNatW5tFixaZ9957z1x55ZWmatWq5siRI+7l+/bta/z8/EytWrXM+PHjzYoVK0xiYqLx8fExXbt2LbCuvn37uv8+fPiwiY6ONpUrVzYvvPCCSU9PNy+++KIJCgoyN910k/njjz+MMcZ88cUXxuVymb/+9a9m2bJlZtWqVSY5OdncfffdZ92vf/zxh+nYsaPx8fExTz/9tFmxYoWZPHmyCQwMNDExMSY3N9cYY0xGRobp0qWLCQgIMBkZGSYjI8P8+uuvZ6zbt29fExgYWOCxyr8FBgZ6bGdOTo5JSEgwb775plm1apX56KOPzOOPP268vLzMnDlzPGpLMhEREaZFixbmnXfeMcuWLTOxsbHGx8fH/Pjjj+750tPTjbe3t2nTpo1ZtGiRSU1NNddee62pVauWOfWldPr06UaS2bVrlzHGmBMnTpiKFSuagIAAc99997nnmzRpkvHx8TG///67Meb/ejI8PNzcf//95sMPPzQLFy40eXl57vu2b99ujDFm8+bNpnXr1qZatWru/ZeRkWFyc3PNRx99ZCSZ/v37u6dv3brVvVxQUJBp3LixeeONN8yKFSvMsGHDjJeXl0lMTHSPbfXq1e790qdPH/PBBx+Y+fPnm1q1apl69eoVeI6cLn+8+c+Z0x06dMh4e3ubdu3auaeNGjXKnP6WdNdddxkvLy8zYsQIs2LFCjN16lTzl7/8xQQFBXk83vnjXb16tTHGmK1bt5oePXoYSQX2z4X0dm5ursnIyDAxMTGmTp067noHDhww27dvdz9ucXFxZuHChWbFihVm+/btZt68eUaSufnmm01aWppZsGCBadasmfHz8zOffvppgW2vX7++GTt2rFm5cqV54oknjCQzePBg06BBA/PSSy+ZlStXmnvvvddIMu++++5ZH4OMjAwTEBBgunTp4h7v5s2bPR6fOnXqmIcfftgsX77czJo1y4SEhJi4uDiPOuPHjzcul8v069fPLF261CxatMi0atXKBAYGuuudSf7jkpqaesZ5qlevbqKiotx/r1271gwbNswsXLjQrF271rz33nsmPj7eBAQEmB9++MEYY8yBAwfc2/DUU0+5t2/nzp3GGGNSU1PNM888Y9577z2zdu1a8/bbb5u2bduasLAws3fvXve6Bg4caMqXL29eeOEFs3r1arN06VKTlJRkXn75Zfc8q1atMn5+fuaGG24wCxYsMB999JFJSEgwkkxycrIxxphff/3VTJgwwUgyr776apFe04CyimCBy1L+m1JhN29vb495c3NzTUxMjImMjDT//ve/TdWqVU3btm3P+uHq5MmT5sSJE2bMmDGmUqVK7g/sxvz5YT8gIMD8/PPP7mmZmZlGkqlevbo5fPiwe3paWpqRZJYsWeKe1rdvXyPJvPjiix7rHD9+vJFkPvvsM491nfoBbOLEicbLy6vAB76FCxcaSWbZsmXGGGMmT55sJJmcnJyz7cYC8j/UPvvssx7TFyxYYCSZ1157zWM7AgMDi1Q3f5vPdjt1O0+Xl5dnTpw4Yfr3729iYmI87pNkqlat6v6Ab4wxu3fvNl5eXmbixInuaS1btjQ1atQwR48edU/7/fffTWhoqMcH4q1btxpJ5o033jDGGPPZZ58ZSeaJJ54wkZGR7vk6dOhgrr/+evff+T15zz33FBj/6cHCGGNuueUWU7t27QLz7t2710gyo0aNKnBfx44dTc2aNc2BAwc8pg8ePNiUK1fO7Nu3zxjzfx8Iu3Tp4jHfO++84/6wfjbnChbGGFO1alXTsGFD99+nB4vNmzcbSWb48OEey82fP7/A4316sDDGmEGDBhUIKsZceG8bY0zbtm3N1Vdf7TEtP1hERUWZ48ePu6efPHnS1KhRwzRu3NicPHnSPf3gwYOmSpUqHo99/rY///zzHrWjo6ONJLNo0SL3tBMnTpiwsDDTvXv3c4739MCdL//xeeihhzymP/vss0aS+eWXX4wxxvz000/Gx8fHPPzwwx7zHTx40FSrVs306tXrrOsvSrBo2bKlCQgIOOP9eXl55vjx46ZevXpm6NCh7unr16/3+HB/Nnl5eebQoUMmMDDQ43WzUaNGJj4+/qzLNmjQwMTExJgTJ054TO/ataupXr26+7FNTU0t0IPA5YhToXBZe+ONN7R+/XqP21dffeUxj7+/v9555x1lZ2eradOmMsZo/vz58vb29phv1apVat++vYKCguTt7S1fX18988wzys7OLnAVnOjoaIWHh7v/btiwoaQ/T3EpX758gemFXT3l9PO8e/fuLUnuUzAKs3TpUjVq1EjR0dHKy8tz3zp27OhxKkn+aQa9evXSO++8o//9739nrHn6PpBU4ApNPXv2VGBgoD7++OMi1SlMQEBAgccq/xYQEFBg/tTUVLVu3VoVKlSQj4+PfH19NXv2bH3//fcF5o2Li1PFihXdf1etWlVVqlRx7/fDhw9r/fr16t69u8qVK+eer2LFirr11ls9akVFRSkiIkLp6emSpJUrV6px48b629/+pu3bt+vHH3/UsWPH9Nlnn3l8eTnfHXfccWE76Bxyc3P18ccf6/bbb1f58uU9Hv8uXbooNzdXX375pccyt912m8ffTZo0kVR4P54vc9opgqdbu3atpD978FQ9evSQj8+Ffz3wQnv7XG677Tb5+vq6//7Pf/6jXbt26e6775aX1/+91VaoUEF33HGHvvzyywKnLnbt2tXj74YNG8rlcqlz587uaT4+Pqpbt64jj8G5Ht/ly5crLy9P99xzj0e/lCtXTm3btnXkymOn90FeXp4mTJigq666Sn5+fvLx8ZGfn5+2bNlS6HO3MIcOHdLw4cNVt25d+fj4yMfHRxUqVNDhw4c9arRo0UIffvihRowYoTVr1ujo0aMedbZu3aoffvjB/Vp7+nPml19+sT41EChrCBa4rDVs2FDNmzf3uDVr1qzAfHXr1tUNN9yg3Nxc9enTx+PqPJK0bt063XzzzZKkmTNn6vPPP9f69es1cuRISSrwhnX61WT8/PzOOj03N9djuo+PjypVquQxrVq1apKk7OzsM27vnj179N1338nX19fjVrFiRRlj9Ntvv0mSbrzxRqWlpbk/VNSsWVONGjUqcO7x6bKzs+Xj41Pgi8Yul0vVqlU769jOxcvLq8BjlX879YObJC1atEi9evVSeHi45s6dq4yMDK1fv179+vUrsC8lFdiX0p+BMv9x279/v/744w/3Pj5VYdPatWvnDlHp6enq0KGDGjdurKpVqyo9PV2ff/65jh49WmiwOL23nJKdna28vDy9/PLLBR7/Ll26SJL78c93+n7J/zLy6f18vg4fPqzs7GzVqFHjrOOV5P5yd77Cev98XGhvn8vpj1v++At7PGvUqKE//vhD+/fv95he2PO/fPnyHmE2f3phfXy+zvX47tmzR9KfYez0nlmwYEGBfrkQP/30k0cfPPbYY3r66acVHx+v999/X1999ZXWr1+va665psh917t3b73yyisaMGCAli9frnXr1mn9+vUKCwvzqPHSSy9p+PDhSktLU1xcnEJDQxUfH68tW7Z4bP/jjz9eYPsfeughSQWfM8DljqtCAUUwa9YsffDBB2rRooVeeeUV3XnnnWrZsqX7/rffflu+vr5aunSpx4eAwq4h74S8vDxlZ2d7fDDYvXu3pMI/JOerXLmyAgIC9Prrr5/x/nzdunVTt27ddOzYMX355ZeaOHGievfurYiICLVq1arQ5StVqqS8vDzt3bvXI1wYY7R79273f4svtrlz5yoyMlILFizw+ELwsWPHLqheSEiIXC6Xex+fqrBp7dq10+zZs7Vu3Tp99dVXeuqppyRJN910k1auXKkdO3aoQoUKhV7J6GL9JkZISIi8vb119913a9CgQYXOExkZeVHWfboPPvhAJ0+ePOvlcPP7eM+ePR5H9/J738aF9Pa5nP645Y//l19+KTDvrl275OXlpZCQkAta16WS/3qwcOFC1a5d2/H669at0+7du9W/f3/3tLlz5+qee+7RhAkTPOb97bffFBwcfM6aBw4c0NKlSzVq1CiNGDHCPf3YsWPat2+fx7yBgYEaPXq0Ro8erT179riPXtx666364Ycf3Nv/j3/8Q927dy90ffXr1y/q5gKXBYIFcA7/+te/9Mgjj+iee+7RzJkzdf311+vOO+/Uhg0b3B8MXC6XfHx8PE6POnr0qN58882LNq558+bpkUcecf/91ltvSTr7bxd07dpVEyZMUKVKlYr8IdLf319t27ZVcHCwli9frg0bNpzxw1e7du307LPPau7cuRo6dKh7+rvvvqvDhw+rXbt2RVqnrfwf4zr1w97u3bsLvSpUUeRfoWvRokV67rnn3OHx4MGD7qs6napdu3ZyuVx6+umn5eXlpRtvvFGS1L59e/3973/Xjh07dOONN3qcOnO+Tj2icvp0qeBRhfLlyysuLk4bNmxQkyZN3EfDLrWffvpJjz/+uIKCgjRw4MAzzpe/zxYsWKCmTZu6py9cuFB5eXnnXM+p+6GwU+Xy5ylqb5+v+vXrKzw8XG+99ZYef/xxdy8ePnxY7777rvtKURfTmXqkqDp27CgfHx/9+OOPjp+it2/fPj3wwAPy9fX1eK1wuVwFLtP7wQcf6H//+5/q1q3rnnamPne5XDLGFKgxa9YsnTx58ozjqVq1qhISErRx40ZNnTpVR44cUf369VWvXj1t3LixQNA5nVNH84DSjmCBy9qmTZsK/ZASFRWlsLAwHT58WL169VJkZKSmTZsmPz8/vfPOO2ratKnuvfde9xGJW265RS+88IJ69+6t+++/X9nZ2Zo8efJFu469n5+fnn/+eR06dEjXXnutvvjiC40bN06dO3dWmzZtzrjckCFD9O677+rGG2/U0KFD1aRJE/3xxx/66aeftGLFCg0bNkwtW7bUM888o59//lnt2rVTzZo1lZOToxdffFG+vr5q27btGet36NBBHTt21PDhw/X777+rdevW+u677zRq1CjFxMTo7rvvvhi7o4CuXbtq0aJFeuihh9SjRw/t3LlTY8eOVfXq1d2nOZyvsWPHqlOnTurQoYOGDRumkydPatKkSQoMDCzwn9AqVaqoUaNGWrFiheLi4twfINu3b699+/Zp3759euGFF6y2sXHjxlq0aJH++c9/qlmzZu5TxSpWrKjatWtr8eLFateunUJDQ1W5cmVFREToxRdfVJs2bXTDDTfowQcfVEREhA4ePKitW7fq/fffd39Hxin5z6+8vDz9+uuv+vTTT5WcnCxvb2+99957Z/1tjquvvlp33XWXnn/+eXl7e+umm27S5s2b9fzzzysoKKjA6W+F7R9JmjRpkjp37ixvb281adJE48aNu6DePl9eXl569tln1adPH3Xt2lUDBw7UsWPH9NxzzyknJ0dJSUmOretMGjdurDVr1uj9999X9erVVbFixfP6D3tERITGjBmjkSNHatu2berUqZNCQkK0Z88erVu3zv0f/3PZsmWLvvzyS/3xxx/uH8ibPXu2fv/9d73xxhsel2fu2rWrUlJS1KBBAzVp0kTffPONnnvuuQK/cRIVFaWAgADNmzdPDRs2VIUKFVSjRg3VqFFDN954o5577jl3369du1azZ88ucMSjZcuW6tq1q5o0aaKQkBB9//33evPNNz1C34wZM9S5c2d17NhRCQkJCg8P1759+/T999/r22+/dV9CulGjRpKk1157TRUrVlS5cuUUGRlpddoeUCoV5zfHgeJytqtCSTIzZ840xhjzt7/9zZQvX77AZRXzrwAyZcoU97TXX3/d1K9f3/j7+5s6deqYiRMnmtmzZxe4mk/t2rXNLbfcUmBMksygQYM8puVfcea5555zT8u/mtJ3331nYmNjTUBAgAkNDTUPPvigOXTokMfyp18Vypg/L/X51FNPmfr16xs/Pz/35UeHDh1qdu/ebYwxZunSpaZz584mPDzc+Pn5mSpVqpguXbp4XCLzTI4ePWqGDx9uateubXx9fU316tXNgw8+aPbv3+8x3/leFeps8xZ29ZukpCQTERFh/P39TcOGDc3MmTMLvaRpYfvdmML33ZIlS0yTJk3cl/tNSkoqtKYxxgwdOtRIMuPHj/eYXq9ePSPJfPfddx7Tz3YlpcKuCrVv3z7To0cPExwcbFwul8cY0tPTTUxMjPH39y9wBaXt27ebfv36mfDwcOPr62vCwsLM9ddfb8aNG+ee50xX88nvx3Ndief051d+D7Vt29ZMmDCh0MtwFrYfc3NzzWOPPWaqVKliypUrZ6677jqTkZFhgoKCPK4QVNhVoY4dO2YGDBhgwsLC3Ptn+/btVr19tqtCnfocPVVaWppp2bKlKVeunAkMDDTt2rUzn3/+eaHbfuqlUI05c98XNo7CZGZmmtatW5vy5csbSaZt27bGmDP3WmH7MX8b4uLizBVXXGH8/f1N7dq1TY8ePUx6evpZ159fL//m4+NjKlWqZFq1amWefPJJk5WVVWCZ/fv3m/79+5sqVaqY8uXLmzZt2phPP/3UtG3b1j3+fPPnzzcNGjQwvr6+HldC+/nnn80dd9xhQkJCTMWKFU2nTp3Mpk2bCjynR4wYYZo3b25CQkLcr9tDhw41v/32m8d6Nm7caHr16mWqVKlifH19TbVq1cxNN91kpk+f7jHf1KlTTWRkpPH29i7yFauAssZlzDkuzQGgRElISNDChQt16NCh4h4KcMl98cUXat26tebNm+e+EhoAoGTgVCgAQIm0cuVKZWRkqFmzZgoICNDGjRuVlJSkevXqnfHLtACA4kOwAACUSFdccYVWrFihqVOn6uDBg6pcubI6d+6siRMnFrgEKwCg+HEqFAAAAABr/EAeAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGtlIlikpKTI5XLp66+/dqSey+XS4MGDHal1as3ExMQLXv7EiRMaPXq0IiIi5O/vrwYNGujll192boA4L5dDzz311FPq2rWrwsPD5XK5lJCQ4NjYcP7Kes998803GjRokBo3bqyKFSuqatWqat++vVatWuXoGFF0Zb3ndu7cqdtvv1116tRRYGCggoKCFBMTo1deeUV5eXmOjhNFU9Z77nTp6elyuVxyuVz67bffHKlZ3MpEsLgcPPTQQ5o4caIGDRqk5cuX6/bbb9ejjz6qCRMmFPfQUEZNmTJF2dnZuu222+Tn51fcw0EZN3/+fK1bt079+vXT4sWLNWvWLPn7+6tdu3Z64403int4KIMOHz6sK664Qk8//bSWLFmit99+W23atNHDDz+sBx54oLiHhzLu0KFDuu+++1SjRo3iHoqjfIp7ADi3zZs3a/bs2Ro/frz+/ve/S5JiY2OVnZ2tcePG6YEHHlBoaGgxjxJlzcGDB+Xl9ef/Ht58881iHg3KuieeeEKTJ0/2mNalSxc1bdpUY8aM0T333FNMI0NZ1aBBA82ZM8djWufOnfXrr79qzpw5evXVV+Xv719Mo0NZN2LECIWEhOiWW27RuHHjins4jrlsjljk5uZq2LBhio6OVlBQkEJDQ9WqVSstXrz4jMvMmDFDV155pfz9/XXVVVfp7bffLjDP7t27NXDgQNWsWVN+fn6KjIzU6NGjHT2MmpaWJmOM7r33Xo/p9957r44ePaqPPvrIsXXBOaW55yS5QwVKj9Lcc1WqVCkwzdvbW82aNdPOnTsdWw+cVZp77kzCwsLk5eUlb2/vi74unL+y0HOffvqpXnvtNc2aNavM9dllc8Ti2LFj2rdvnx5//HGFh4fr+PHjSk9PV/fu3ZWcnFzgv2FLlizR6tWrNWbMGAUGBmratGm666675OPjox49ekj6swlbtGghLy8vPfPMM4qKilJGRobGjRunrKwsJScnn3VMERERkqSsrKyzzrdp0yaFhYWpWrVqHtObNGnivh8lT2nuOZROZa3n8vLy9Omnn+rqq68+72VxaZSFnjPG6OTJkzp48KBWrFihlJQUDRs2TD4+l81HpFKltPfc0aNH1b9/fw0ZMkRNmzbVkiVLLmg/lFimDEhOTjaSzPr164u8TF5enjlx4oTp37+/iYmJ8bhPkgkICDC7d+/2mL9Bgwambt267mkDBw40FSpUMDt27PBYfvLkyUaS2bx5s0fNUaNGecwXFRVloqKizjnWDh06mPr16xd6n5+fn7n//vvPWQPOKus9d7rAwEDTt2/f814Ozrnces4YY0aOHGkkmbS0tAtaHnYul56bOHGikWQkGZfLZUaOHFnkZeGsy6Hnhg0bZurUqWOOHDlijDFm1KhRRpLZu3dvkZYv6S6rcx1SU1PVunVrVahQQT4+PvL19dXs2bP1/fffF5i3Xbt2qlq1qvtvb29v3Xnnndq6dat+/vlnSdLSpUsVFxenGjVqKC8vz33r3LmzJGnt2rVnHc/WrVu1devWIo3d5XJd0H0oXqW551A6lZWemzVrlsaPH69hw4apW7du5708Lp3S3nMJCQlav369li9frieeeELPPfecHn744SIvj0uvtPbcunXrNHXqVM2YMUMBAQHns8mlxmUTLBYtWqRevXopPDxcc+fOVUZGhtavX69+/fopNze3wPynn3Z06rTs7GxJ0p49e/T+++/L19fX45Z/2N6pS4dVqlTJvc5THT58WMePH+eL2yVUae45lE5lpeeSk5M1cOBA3X///Xruueccrw/nlIWeq1atmpo3b66bb75ZSUlJGjNmjF555RVt2LDB0fXAGaW55/r166fu3burefPmysnJUU5OjnvMv//+uw4ePOjIeorTZXMC4dy5cxUZGakFCxZ4/If/2LFjhc6/e/fuM06rVKmSJKly5cpq0qSJxo8fX2gNpy4h1rhxY7399tvavXu3xxPkX//6lySpUaNGjqwHzirNPYfSqSz0XHJysgYMGKC+fftq+vTpHJEt4cpCz52uRYsWkqT//ve/iomJuajrwvkrzT23efNmbd68WampqQXui4qK0jXXXKPMzExH1lVcLptg4XK55Ofn59GEu3fvPuNVBD7++GPt2bPHffjs5MmTWrBggaKiolSzZk1JUteuXbVs2TJFRUUpJCTkoo29W7dueuqppzRnzhwNHz7cPT0lJUUBAQHq1KnTRVs3Llxp7jmUTqW951JSUjRgwAD97W9/06xZswgVpUBp77nCrF69WpJUt27dS75unFtp7rn83jpVSkqK5syZo7S0NIWHh1+0dV8qZSpYrFq1qtBv5Hfp0kVdu3bVokWL9NBDD6lHjx7auXOnxo4dq+rVq2vLli0FlqlcubJuuukmPf300+6rCPzwww8elygbM2aMVq5cqeuvv16PPPKI6tevr9zcXGVlZWnZsmWaPn26u2kLk/+ida7z8q6++mr1799fo0aNkre3t6699lqtWLFCr732msaNG8epUMWorPac9Oc5pXv37pX05wvxjh07tHDhQklS27ZtFRYWds4acF5Z7bnU1FT1799f0dHRGjhwoNatW+dxf0xMDL8pUEzKas+NGjVKe/bs0Y033qjw8HDl5OToo48+0syZM9WzZ081a9asiHsITiurPRcbG1tg2po1ayRJrVu3VuXKlc+6fKlQ3N8ed0L+VQTOdNu+fbsxxpikpCQTERFh/P39TcOGDc3MmTPd38Y/lSQzaNAgM23aNBMVFWV8fX1NgwYNzLx58wqse+/eveaRRx4xkZGRxtfX14SGhppmzZqZkSNHmkOHDnnUPP0qArVr1za1a9cu0jYeP37cjBo1ytSqVcv4+fmZK6+80rz00kvntZ/gnMuh59q2bXvG7Vu9evX57C44oKz3XN++fYu0fbh0ynrPLVmyxLRv395UrVrV+Pj4mAoVKpgWLVqYl156yZw4ceK89xfslfWeK0xZuyqUyxhj7OMJAAAAgMvZZXNVKAAAAAAXD8ECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsFbkX94+9afTi1vPnj0dqZOUlORInfT0dEfqjBgxwrrG/v37HRiJc2x+JqUk9ZxT8n9h01ZwcLAjdUaNGmVdY/HixQ6MxDn0nKfCfun1QqSlpTlSJzMz07qGU9vklLLSc8OHD3ekjlPvrdu2bXOkTvPmza1r8N5asjn1npiSkuJInfj4eEfqlCRF7TmOWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACw5lPcA7gQSUlJjtSpU6eOI3VCQkIcqbNv3z7rGr169XJgJFJqaqojdeApJyfHkTpt27Z1pE5cXJx1jcWLFzswEpwuOjrakTqrV692pM6BAwccqRMREeFIHXhy4n2xZ8+eDoxEGjhwoCN1ZsyY4UidZs2aWddIT093YCS4WBISEhypk5mZ6UidyxlHLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKz5XOoVNmvWzLpGnTp1HBiJFBUV5Uidbdu2OVJn5cqV1jWc2L+SlJqa6kidsiI6OtqROrGxsY7UcUpmZmZxDwFnEB8f70idjRs3OlInLS3NkTqjRo1ypA48vfbaa9Y1Jk2a5MBIpK+//tqROk69t6anpztSB84LDg52pE5CQoIjdaZOnepInYiICEfqOCErK+uSro8jFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACs+VzqFYaEhFjX+OabbxwYibRt2zZH6jjFqe2CpyFDhljXSExMtK4hSUFBQY7UccqaNWuKewg4g6lTpzpSJysry5E6To1n8eLFjtSBJyfez+rUqePASJyrk56e7kgdJz537N+/34GR4HQJCQmO1ImIiHCkTkpKiiN1nHi9zMnJsa4hOff5pag4YgEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAms+lXmFISIh1jfT0dAdGUvI4sW/279/vwEjKlqlTp1rXSElJsa4hlbzHJzg4uLiHUCY5sV+HDBliXUOS4uPjHanjlISEhOIeAs5g27ZtjtQJDQ11pM7KlStLTJ0OHTo4MJKS9x5go1u3btY1pkyZ4sBIpDlz5jhSxymPPvqodY17773XgZFcehyxAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGDN51KvcP/+/dY1mjVr5sBInBMSEuJIHSe2KzU11YGR4HIRHR1tXSMzM9O6RlmTmJhoXePRRx+1H4iD4uPjHamTk5PjSB2UXE68z0tShw4dHKkzY8YM6xrDhw93YCTSiBEjHKlTEhw4cKBE1JCkvn37OlLHifdEp6SlpRX3EC4IRywAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACs+VzqFW7bts26RrNmzRwYidSzZ88SVccJkyZNKu4hAJe9lJQU6xqxsbHWNSTpmmuucaROWlqaI3UWL15sXSM5OdmBkTgzlrIkKSnJkTrp6emO1AkJCXGkTvv27a1rpKamOjCSsmXNmjXWNYKDg61rSFJ0dLQjdZzYJkmaM2eOdY2cnBz7gRQDjlgAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsOZzqVe4bds26xojRoxwYCRSUlKSI3W++eYbR+o0b97ckTpwXk5OjiN1Fi9e7Eidbt26OVInNjbWukZKSop1jbImMzPTukZ0dLR1DSfrJCYmOlLHid7NysqyH4icez6WFfv373ekzowZMxyp45TU1FTrGgMHDnRgJLhYnHqPDgoKcqTO5fy+yBELAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANZcxhhT3IMAAAAAULpxxAIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwViaCRUpKilwul77++mtH6rlcLg0ePNiRWqfWTExMvKBls7Ky5HK5Cr29/fbbjo4TRVPWey7fpk2b1LNnT4WFhcnf318RERF66KGHnBkgzktZ77nExMQzvs7xWlc8ynrPSdLWrVt19913q1atWgoICFBUVJQee+wxZWdnOzdIFNnl0HP//e9/dccddygkJETly5dXy5YttWTJEucGWMx8insAKLqHH35YvXv39phWr169YhoNyrrVq1frlltu0Q033KDp06ercuXK+umnn7Rhw4biHhrKoAEDBqhTp04Fpt9333368ccfC70PsLF3715dd911uuKKKzR27FjVqlVLGzZs0KhRo7R69Wp988038vIqE/9/RQmRlZWlVq1aqXr16po+fboqVKigf/7zn4qPj1dqaqruuOOO4h6iNYJFKVKrVi1dd911xT0MXAaOHDmiPn366KabbtL7778vl8vlvu/uu+8uxpGhrKpZs6Zq1qzpMS0rK0ubN29Wnz59FBwcXDwDQ5m1ePFiZWdna8GCBWrXrp0kKS4uTseOHdOTTz6pjRs3KiYmpphHibIkKSlJR44c0fLlyxUeHi5J6tSpkxo3bqyhQ4fq9ttvL/VhtnSP/jzk5uZq2LBhio6OVlBQkEJDQ9WqVSstXrz4jMvMmDFDV155pfz9/XXVVVcVeih+9+7dGjhwoGrWrCk/Pz9FRkZq9OjRysvLu5ibg1KgNPdcamqqfvnlF/3973/3CBUo2UpzzxXm9ddflzFGAwYMuKjrwYUrzT3n6+srSQoKCvKYnh9iy5Ur59i64JzS3HOff/65rrnmGneokCRvb2917txZO3fu1Lp16xxbV3G5bI5YHDt2TPv27dPjjz+u8PBwHT9+XOnp6erevbuSk5N1zz33eMy/ZMkSrV69WmPGjFFgYKCmTZumu+66Sz4+PurRo4ekP5uwRYsW8vLy0jPPPKOoqChlZGRo3LhxysrKUnJy8lnHFBERIenP/8oVRVJSkp588kn5+PioadOmeuKJJ3Tbbbed977ApVGae+6TTz6RJJ08eVJt2rTRunXrFBgYqE6dOun5559XjRo1Lmyn4KIqzT13uj/++EMpKSmqW7eu2rZte17L4tIpzT0XHx+vWrVqadiwYZo2bZpq166tb7/9VklJSbr11lvVsGHDC94vuHhKc88dP35coaGhBab7+/tLkr777rvSf2aKKQOSk5ONJLN+/foiL5OXl2dOnDhh+vfvb2JiYjzuk2QCAgLM7t27PeZv0KCBqVu3rnvawIEDTYUKFcyOHTs8lp88ebKRZDZv3uxRc9SoUR7zRUVFmaioqHOOddeuXea+++4z77zzjvn000/NvHnzzHXXXWckmZkzZxZ5m+Gcst5zHTt2NJJMcHCweeKJJ8yqVavM9OnTTaVKlUzdunXN4cOHi7zdcEZZ77nTffjhh0aSmThx4nkvC2dcDj23a9cu06pVKyPJfevZs6fJzc0t6ibDQWW95+Lj401wcLA5ePCgx/QbbrjBSDITJkw4Z42S7rI5FUr68/SO1q1bq0KFCvLx8ZGvr69mz56t77//vsC87dq1U9WqVd1/e3t7684779TWrVv1888/S5KWLl2quLg41ahRQ3l5ee5b586dJUlr164963i2bt2qrVu3nnPc1atX12uvvaaePXuqTZs26t27tz755BPFxMRoxIgRnHZVgpXWnvvjjz8kSXfeeacmTZqkuLg4DRw4ULNnz9bWrVv11ltvFXkf4NIqrT13utmzZ8vHx0cJCQnnvSwurdLac/v371e3bt30+++/a968efrkk080bdo0ffbZZ7rtttt4by3BSmvPDR48WAcOHNA999yjbdu2ac+ePXr66af1xRdfSFKp/36FdBl9x2LRokXq1auXwsPDNXfuXGVkZGj9+vXq16+fcnNzC8xfrVq1M07Lvwzdnj179P7778vX19fjdvXVV0uSfvvtt4u2Pb6+vrrzzjuVnZ2tLVu2XLT14MKV5p6rVKmSJKljx44e0zt27CiXy6Vvv/3WkfXAWaW5507122+/acmSJbrlllsKHSNKjtLcc5MmTVJmZqZWrlyp3r1764YbbtCDDz6oefPmacWKFZo3b54j64GzSnPPtWvXTsnJyfrkk08UFRWlatWqadGiRRo7dqwkeXz3orS6bL5jMXfuXEVGRmrBggUeX0Y9duxYofPv3r37jNPyP3RVrlxZTZo00fjx4wutcbHPQzfGSCobCbcsKs0916RJk7P+bgA9VzKV5p471Ztvvqnjx4/zpe1SoDT3XGZmpsLDw1W9enWP6ddee62kP3/HByVPae45Serbt6/69OmjLVu2yNfXV3Xr1tXEiRPlcrl0ww03OLae4nLZBAuXyyU/Pz+PJty9e/cZryLw8ccfa8+ePe7DZydPntSCBQsUFRXlviRi165dtWzZMkVFRSkkJOTib8QpTpw4oQULFqhy5cqqW7fuJV03iqY099ztt9+ukSNH6sMPP9Ttt9/unv7hhx/KGFP6v1xWRpXmnjvV7NmzVaNGDfdpCCi5SnPP1ahRQx9//LH+97//efynOCMjQ5IKXP4YJUNp7rl8Pj4+7osDHDhwQK+99pq6deum2rVrX/R1X2xlKlisWrWq0G/kd+nSRV27dtWiRYv00EMPqUePHtq5c6fGjh2r6tWrF3oqUeXKlXXTTTfp6aefdl9F4IcffvD4L+6YMWO0cuVKXX/99XrkkUdUv3595ebmKisrS8uWLdP06dPP+sKUHwjOdV7eY489phMnTqh169aqVq2adu7cqZdfflmZmZlKTk6Wt7d3EfcQnFZWe65BgwYaNGiQpk2bpooVK6pz587673//q6eeekoxMTHq1atXEfcQnFZWey7fV199pc2bN+vJJ5/kta2EKKs9N2jQIM2bN08dOnTQiBEj9Je//EWbNm3SuHHjVLVqVfXp06eIewhOK6s99+uvv+r5559X69atVbFiRf3www969tln5eXlpVdffbWIe6eEK+5vjzsh/yoCZ7pt377dGGNMUlKSiYiIMP7+/qZhw4Zm5syZZtSoUeb03SDJDBo0yEybNs1ERUUZX19f06BBAzNv3rwC6967d6955JFHTGRkpPH19TWhoaGmWbNmZuTIkebQoUMeNU+/ikDt2rVN7dq1z7l9s2fPNi1atDChoaHGx8fHhISEmI4dO5rly5ef976CM8p6zxnz55UzkpKSTN26dY2vr6+pXr26efDBB83+/fvPZ1fBIZdDzxljzH333WdcLpf58ccfi7wMLo7Loee+/fZbc/vtt5uaNWsaf39/U6dOHTNgwADz008/nde+gjPKes9lZ2ebm2++2YSFhRlfX19Tq1Yt8/DDD5u9e/ee974qqVzG/P8T9QEAAADgAvENTAAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACAtSL/8vapP51e3NasWeNIncJ+1fFCJCQkOFKnLLL5mZSS1HNOcap3g4ODHakTHR3tSJ2SpKz03JAhQxyp41SvxMfHO1Lnmmuusa5x4MABB0YiRUREOFJn//79F7xsSeq5qVOnOlLHqV5JSUlxpI4T25WTk2Ndw0ll5XUuLS3NkTpOvc7FxsY6UqcsKmrPccQCAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWXMcYUaUaX62KPpciysrIcqVO7dm1H6jhlx44d1jUiIiLsB+KgIrZXoUpSz3Xr1s2ROmlpaY7UGT16tCN1EhMTHalTkpSVnhsyZEhxD8FDZmamI3Wc2K7g4GDrGpIUGxvrSJ2y0nNr1qxxpE5Jex9y4jODU73ilJLQc048ztu3b7cfSAm0ceNG6xrR0dH2A3FQUXuOIxYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWfIp7ABciJyfHkTq1a9d2pM6BAwccqbNmzRrrGsHBwdY1JOf2cVkxevTo4h6Ch7S0tOIeAi6yqVOnFvcQPCQmJjpSJyIiwrpGbGysdQ0UlJmZ6UidrKwsR+okJCQ4UseJ9zOnes6J9/mSwqnPG05Yu3atI3Wc6t3L+TWKIxYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArPkU9wAuRFZWliN1rrnmGkfqBAUFOVInMzPTukZOTo51DRQUHBzsSJ2NGzc6UseJXsHFExsbWyJqOGnIkCHFPQS3+Ph4R+qkpKQ4UqescGp/bNiwwZE6ERERjtRx4n3Rqc8dZUlJ2idOvSakpaU5UsepzwylEUcsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFgjWAAAAACwRrAAAAAAYI1gAQAAAMAawQIAAACANYIFAAAAAGsECwAAAADWCBYAAAAArBEsAAAAAFjzKe4BXIj4+HhH6sTGxjpSJzo62pE6U6ZMcaSOE6ZOnVrcQyhRgoODHamTlZXlSJ0hQ4Y4UictLc26hlPbVJY4sU+cel1x6nXOKU68fq9Zs8a6Bgpy6nXOKW3btnWkTmRkpHUNXucKysnJsa6xceNG+4FI2r9/vyN1XnzxRUfqOPH6HRERYV1DuvS9yxELAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANZ8insAxWnNmjXFPQTHRUREFPcQyqSsrCxH6rRt29aROsHBwY7UmTJlinWNmJgYB0YiZWZmOlKnJHCiX+Lj461rSJIxxpE6To2nLL7ulgTR0dHWNVavXm0/EEmjR492pI5T72dpaWnWNZzqf6feS8oKJ/rWyTol6X1o6tSpjtRxqneLiiMWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1nyKewAXolu3bo7UOXDggCN1EhMTHanjhLS0tOIeQpmUkpLiSJ0pU6Y4UicrK8uROhEREdY14uPjrWtIUmZmpiN1yoqpU6c6Usep17m1a9c6UgcXhxOvCU71ilO968TrkyRt2LDBukZCQoL9QFSyPi+UJU69fzjVu070i1PvrZcaRywAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWPMp7gFciLi4OEfqPProo47UccqcOXOsa6xZs8Z+ICggJSXFkToRERGO1ElISHCkjhP9kpaWZl0DBcXGxjpSp2/fvo7UycnJcaQOLg4nHh+n3j/279/vSJ0DBw44Umfx4sXWNaZOnWo/EBTg1H6Njo52pE5wcLAjdZx4/c7MzLSuURw4YgEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAGsECAAAAgDWCBQAAAABrBAsAAAAA1ggWAAAAAKwRLAAAAABYI1gAAAAAsEawAAAAAGCNYAEAAADAmssYY4p7EAAAAABKN45YAAAAALBGsAAAAABgjWABAAAAwBrBAgAAAIA1ggUAAAAAawQLAAAAANYIFgAAAACsESwAAAAAWCNYAAAAALD2/wDcx16W2USD6QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 800x400 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Load the dataset\n",
    "digits = load_digits()\n",
    "\n",
    "# Set up the figure size and grid\n",
    "fig, axes = plt.subplots(nrows=2, ncols=5, figsize=(8, 4))\n",
    "fig.suptitle('Examples of Handwritten Digits from the Dataset')\n",
    "\n",
    "# Plot examples of the digits\n",
    "for ax, img, label in zip(axes.flat, digits.images, digits.target):\n",
    "    ax.imshow(img, cmap='gray')\n",
    "    ax.set_title(f'Label: {label}')\n",
    "    ax.axis('off')\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Custom Least Squares Classifier\n",
    "We do the following:\n",
    "\n",
    "**Fitting the Model (fit Method):**\n",
    "\n",
    "- Weights Calculation: Computes the weights (coefficients) for the linear model using the pseudo-inverse of the design matrix (X_bias). This is done by multiplying the pseudo-inverse of X_bias with the one-hot encoded target matrix Y. The pseudo-inverse is used to ensure a solution even when the matrix is not invertible, which is common in cases where features are correlated or the system is over-determined.\n",
    "- Target Variable Encoding: Applies one-hot encoding to the target variable y using LabelBinarizer. This transforms the target into a binary matrix necessary for multi-class classification in a least squares context.\n",
    "- Bias Term Addition: Augments the feature matrix X with a bias term (a column of ones). This is a common practice in linear models to account for the intercept in the linear equation.\n",
    "\n",
    "**Making Predictions (predict Method):**\n",
    "- Bias Term Addition: Similar to the fit method, augments the feature matrix X with a bias term for prediction.\n",
    "- Probability Prediction: Uses the learned weights to calculate the probabilities for each class by applying the linear model to the augmented feature matrix.\n",
    "- Class Prediction: Converts the probabilities to class labels by selecting the class with the highest probability for each instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "class LeastSquaresClassifier:\n",
    "    def __init__(self):\n",
    "        self.weights = None\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        # Add a bias term to X\n",
    "        X_bias = np.c_[np.ones((X.shape[0], 1)), X]\n",
    "\n",
    "        # One-hot encode the target variable\n",
    "        lb = LabelBinarizer()\n",
    "        Y = lb.fit_transform(y)\n",
    "\n",
    "        # Calculate weights using the pseudo-inverse\n",
    "        self.weights = np.linalg.pinv(X_bias.T @ X_bias) @ X_bias.T @ Y\n",
    "\n",
    "    def predict(self, X):\n",
    "        # Add a bias term to X\n",
    "        X_bias = np.c_[np.ones((X.shape[0], 1)), X]\n",
    "\n",
    "        # Predict probabilities using the learned weights\n",
    "        probabilities = X_bias @ self.weights\n",
    "\n",
    "        # Convert probabilities to class labels\n",
    "        predictions = np.argmax(probabilities, axis=1)\n",
    "\n",
    "        return predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the Dataset "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the Digits dataset\n",
    "digits = load_digits()\n",
    "X = digits.data\n",
    "y = digits.target\n",
    "\n",
    "\n",
    "# Split the data into training and testing sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Traing and Testing using Custom Least Squares Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set accuracy for Least Squares Classifier: 0.9498956158663883\n",
      "Accuracy for Least Squares Classifier: 0.9444444444444444\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.97      1.00      0.99        33\n",
      "           1       0.90      0.93      0.91        28\n",
      "           2       0.94      0.94      0.94        33\n",
      "           3       0.94      1.00      0.97        34\n",
      "           4       1.00      0.93      0.97        46\n",
      "           5       0.92      0.94      0.93        47\n",
      "           6       0.97      0.97      0.97        35\n",
      "           7       0.97      0.97      0.97        34\n",
      "           8       0.93      0.83      0.88        30\n",
      "           9       0.90      0.93      0.91        40\n",
      "\n",
      "    accuracy                           0.94       360\n",
      "   macro avg       0.94      0.94      0.94       360\n",
      "weighted avg       0.95      0.94      0.94       360\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Instantiate and fit the Least Squares classifier\n",
    "ls_classifier = LeastSquaresClassifier()\n",
    "ls_classifier.fit(X_train, y_train)\n",
    "\n",
    "# Make predictions on the training set\n",
    "y_pred_train = ls_classifier.predict(X_train)\n",
    "\n",
    "# Calculate and print accuracy for the Least Squares Classifier on the training set\n",
    "accuracy_train_ls = accuracy_score(y_train, y_pred_train)\n",
    "print(\"Training set accuracy for Least Squares Classifier:\", accuracy_train_ls)\n",
    "\n",
    "# Make predictions on the test set\n",
    "y_pred = ls_classifier.predict(X_test)\n",
    "\n",
    "# Calculate and print accuracy\n",
    "accuracy_ls = accuracy_score(y_test, y_pred)\n",
    "print(\"Accuracy for Least Squares Classifier:\", accuracy_ls)\n",
    "\n",
    "# Print classification report\n",
    "print(classification_report(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Traing and Testing using LinearSVC from scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set accuracy for LinearSVC: 0.9972164231036882\n",
      "Accuracy for LinearSVC: 0.95\n",
      "\n",
      "Classification Report for LinearSVC:\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.97      1.00      0.99        33\n",
      "           1       0.93      0.96      0.95        28\n",
      "           2       0.94      1.00      0.97        33\n",
      "           3       0.97      0.94      0.96        34\n",
      "           4       1.00      0.96      0.98        46\n",
      "           5       0.94      0.94      0.94        47\n",
      "           6       0.97      0.97      0.97        35\n",
      "           7       1.00      0.94      0.97        34\n",
      "           8       0.82      0.93      0.87        30\n",
      "           9       0.95      0.88      0.91        40\n",
      "\n",
      "    accuracy                           0.95       360\n",
      "   macro avg       0.95      0.95      0.95       360\n",
      "weighted avg       0.95      0.95      0.95       360\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Instantiate and fit scikit-learn's LinearSVC\n",
    "linear_svc_classifier = LinearSVC(loss='squared_hinge', max_iter=10000, dual=False)\n",
    "linear_svc_classifier.fit(X_train, y_train)\n",
    "\n",
    "# Make predictions on the training set\n",
    "y_pred_train = linear_svc_classifier.predict(X_train)\n",
    "\n",
    "# Calculate and print accuracy for LinearSVC on the training set\n",
    "accuracy_train = accuracy_score(y_train, y_pred_train)\n",
    "print(\"Training set accuracy for LinearSVC:\", accuracy_train)\n",
    "\n",
    "# Make predictions on the test set\n",
    "y_pred_linear_svc = linear_svc_classifier.predict(X_test)\n",
    "\n",
    "# Calculate and print accuracy for LinearSVC\n",
    "accuracy_linear_svc = accuracy_score(y_test, y_pred_linear_svc)\n",
    "print(\"Accuracy for LinearSVC:\", accuracy_linear_svc)\n",
    "\n",
    "# Print classification report for LinearSVC\n",
    "print(\"\\nClassification Report for LinearSVC:\")\n",
    "print(classification_report(y_test, y_pred_linear_svc))\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}