From 87f3a4f725ae25d725663f175f037c2fd115a024 Mon Sep 17 00:00:00 2001 From: HYEOK9 Date: Mon, 2 Aug 2021 01:33:03 +0900 Subject: [PATCH] =?UTF-8?q?04~08-=EC=9D=B4=EC=9E=AC=ED=98=81?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...354\235\264\354\236\254\355\230\201.ipynb" | 929 ++++++++++++++++++ 1 file changed, 929 insertions(+) create mode 100644 "assignments/HYEOK9/04~08-\354\235\264\354\236\254\355\230\201.ipynb" diff --git "a/assignments/HYEOK9/04~08-\354\235\264\354\236\254\355\230\201.ipynb" "b/assignments/HYEOK9/04~08-\354\235\264\354\236\254\355\230\201.ipynb" new file mode 100644 index 0000000..3e3cd83 --- /dev/null +++ "b/assignments/HYEOK9/04~08-\354\235\264\354\236\254\355\230\201.ipynb" @@ -0,0 +1,929 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "fb78f666", + "metadata": {}, + "source": [ + "## Module Import" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "570828be", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import pandas as pd\n", + "import matplotlib.pyplot as plt\n", + "import sklearn" + ] + }, + { + "cell_type": "markdown", + "id": "5353befe", + "metadata": {}, + "source": [ + "## Assignment 1.\n", + "우리는 붓꽃의 **꽃받침의 길이, 꽃받침의 너비, 꽃잎의 길이, 꽃잎의 너비**를 통해서, **꽃의 종류**를 구분 해 볼 것입니다. **Input**으로 주어 지는 데이터는 다음과 같습니다.\n", + "\n", + "- Sepal Length: 꽃받침의 길이 정보이다.\n", + "- Sepal Width: 꽃받침의 너비 정보이다.\n", + "- Petal Length: 꽃잎의 길이 정보이다.\n", + "- Petal Width: 꽃잎의 너비 정보이다.\n", + "- Species: 꽃의 종류 정보이다. **setosa / versicolor / virginica** 의 3종류로 구분된다." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "a3b18df3", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "X_train의 shape: (100, 4)\n", + "X_test의 shape: (50, 4)\n", + "y_train의 shape: (100,)\n", + "y_test의 shape: (50,)\n" + ] + } + ], + "source": [ + "from sklearn.datasets import load_iris\n", + "from sklearn.model_selection import train_test_split\n", + "\n", + "data = load_iris()\n", + "X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, test_size=0.33, random_state=42, shuffle=True)\n", + "\n", + "print(\"X_train의 shape:\", X_train.shape)\n", + "print(\"X_test의 shape:\", X_test.shape)\n", + "print(\"y_train의 shape:\", y_train.shape)\n", + "print(\"y_test의 shape:\", y_test.shape)" + ] + }, + { + "cell_type": "markdown", + "id": "9b53defd", + "metadata": {}, + "source": [ + "### 1-1. Data Normalization\n", + "첫 번째로, 데이터를 정규화 하는 것이 중요 할 것 같습니다. 데이터를 정규화 해 보세요.\n", + "\n", + "(Min - Max 정규화를 이용하면 될 것 같죠? hint: ndarray.min(), ndarray.max())" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "25ebc396", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[[0.73684211 0.36842105 0.53947368 0.15789474]\n", + " [0.98684211 0.38157895 0.85526316 0.26315789]\n", + " [0.72368421 0.38157895 0.57894737 0.18421053]\n", + " [0.65789474 0.44736842 0.17105263 0.01315789]\n", + " [1. 0.35526316 0.86842105 0.25 ]\n", + " [0.75 0.34210526 0.52631579 0.11842105]\n", + " [0.67105263 0.43421053 0.17105263 0.01315789]\n", + " [0.64473684 0.44736842 0.15789474 0.02631579]\n", + " [0.65789474 0.48684211 0.23684211 0.03947368]\n", + " [0.64473684 0.25 0.44736842 0.11842105]]\n" + ] + } + ], + "source": [ + "X_train = (X_train-X_train.min())/(X_train.max()-X_train.min())\n", + "X_test = (X_test-X_test.min())/(X_test.max()-X_test.min())\n", + "print(X_train[:10])" + ] + }, + { + "cell_type": "markdown", + "id": "fd97af39", + "metadata": {}, + "source": [ + "## 1-2. Data Training\n", + "그 다음으로는 이제 데이터를 학습 시킬 시간입니다! SVM 모듈을 import 한 후, 학습을 시켜 보도록 하겠습니다." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "9c989b42", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "정확도 (accuracy): 100.0%\n" + ] + } + ], + "source": [ + "from sklearn.svm import SVC\n", + "\n", + "svm = SVC(C=2)\n", + "# \n", + "svm.fit(X_train,y_train)\n", + "y_test = svm.predict(X_test)\n", + "\n", + "\n", + "accuracy = (sum(svm.predict(X_test) == y_test) / 50) * 100\n", + "print(f'정확도 (accuracy): {accuracy}%')" + ] + }, + { + "cell_type": "markdown", + "id": "162aa7df", + "metadata": {}, + "source": [ + "## Assignment 2.\n", + "다음은 **MNIST 데이터**에 대해 분류를 해보는 연습을 해 보겠습니다. **MNIST**는 손글씨 데이터로, **Input Data**는 [28 x 28]의 데이터로 이루어져 있습니다. 일단, 우리가 이를 학습 시키기 전에 한번 데이터를 확인 해 볼까요?\n", + "\n", + "이 예제는 DNN을 이용하기 때문에, **Pytorch**로 진행 하겠습니다. 실습은 **Hyperparameter**만 고치면 됩니다.\n", + "\n", + "**학습률 95%에 도전 해 보세요!**" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "09585b1f", + "metadata": {}, + "outputs": [], + "source": [ + "import time\n", + "import random\n", + "import torch\n", + "import torch.nn as nn\n", + "from torchvision.datasets import MNIST\n", + "from torchvision import transforms\n", + "from torch.utils.data import DataLoader\n", + "import torch.nn.functional as F" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "52693fc8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 이 부분은 절대 변경하지 마세요.\n", + "\n", + "RANDOM_SEED = 123\n", + "DEVICE = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\n", + "\n", + "random.seed(RANDOM_SEED)\n", + "torch.manual_seed(RANDOM_SEED)" + ] + }, + { + "cell_type": "markdown", + "id": "c795ae90", + "metadata": {}, + "source": [ + "## 2-0. Data Load\n", + "데이터를 불러와 보겠습니다. `transforms.Compose`를 이용하여, 데이터를 pytorch에서 사용하는 **Tensor**형으로 바꾸고, 이를 **Gaussian Distribution**으로 정규화합니다." + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "6b9bbb9f", + "metadata": {}, + "outputs": [], + "source": [ + "# 수정 가능한 셀입니다.\n", + "BATCH_SIZE = 64 # 60000을 사용하면, Full-Batch 학습을 진행 합니다." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "d1eb9987", + "metadata": {}, + "outputs": [], + "source": [ + "# Don't Touch!\n", + "custom_train_transform = transforms.Compose([ \n", + " transforms.ToTensor(),\n", + " transforms.Normalize(mean=(0.5,), std=(0.5,))\n", + "])\n", + "\n", + "custom_test_transform = transforms.Compose([\n", + " transforms.ToTensor(),\n", + " transforms.Normalize(mean=(0.5,), std=(0.5,))\n", + "])" + ] + }, + { + "cell_type": "markdown", + "id": "946bb0ec", + "metadata": {}, + "source": [ + "`MNIST`를 이용하여 **MNIST** 데이터를 불러 오고, 이를 transform 해 줍니다. 또한, `DataLoader`를 이용하여, 셔플을 해준 후, 미니 배치를 생성 합니다." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "c0b80568", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n", + "Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to .\\MNIST\\raw\\train-images-idx3-ubyte.gz\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "80b968791680408c87998b08e1827e22", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + " 0%| | 0/9912422 [00:00" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "break minibatch for-loop\n" + ] + } + ], + "source": [ + "for batch_idx, (x, y) in enumerate(train_loader):\n", + " print(' | Batch index:', batch_idx, end='')\n", + " print(' | Batch size:', y.size()[0])\n", + "\n", + " x = x.to(DEVICE)\n", + " y = y.to(DEVICE)\n", + "\n", + " x_numpy = x.numpy()\n", + " y_numpy = y.numpy()\n", + " print('input batch의 모양:', x_numpy.shape)\n", + " plt.imshow(x_numpy[0].reshape(28, 28))\n", + " plt.title(f'Label {y_numpy[0]}')\n", + " plt.show()\n", + "\n", + " print('break minibatch for-loop')\n", + " break" + ] + }, + { + "cell_type": "markdown", + "id": "78dbdd43", + "metadata": {}, + "source": [ + "## 2-1. Deep Neural Network\n", + "아래 셀은 Deep Neural Network를 정의 합니다." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "aed9144c", + "metadata": {}, + "outputs": [], + "source": [ + "# 변경 가능 합니다.\n", + "class DNN(nn.Module):\n", + " def __init__(self, num_features, num_hidden_1, num_hidden_2, num_hidden_3, num_classes):\n", + " \"\"\"\n", + " num_features: input feature 갯수\n", + " num_hidden_1: 첫 번째 레이어의 노드 갯수\n", + " num_hidden_2: 두 번째 레이어의 노드 갯수\n", + " num_hidden_3: 세 번째 레이어의 노드 갯수\n", + " num_classes: 분류하고자 하는 class 갯수\n", + " \"\"\"\n", + " super(DNN, self).__init__()\n", + " \n", + " self.num_classes = num_classes\n", + " \n", + " # 수정 가능!: 레이어를 쌓는 법을 아신다면, 여기서 레이어를 쌓으셔도, 혹은 지우셔도 무방 합니다. (단, nn.Linear만)\n", + " self.linear_1 = nn.Linear(num_features, num_hidden_1)\n", + " self.linear_2 = nn.Linear(num_hidden_1, num_hidden_2)\n", + " self.linear_3 = nn.Linear(num_hidden_2, num_hidden_3)\n", + "\n", + " self.linear_out = nn.Linear(num_hidden_3, num_classes)\n", + " \n", + " def forward(self, x):\n", + " out = self.linear_1(x)\n", + " out = torch.relu(out)\n", + " out = self.linear_2(out)\n", + " out = torch.relu(out)\n", + " out = self.linear_3(out)\n", + " out = torch.relu(out)\n", + " logits = self.linear_out(out)\n", + " probas = torch.sigmoid(logits)\n", + " return logits, probas\n", + "\n", + "# 수정 가능!\n", + "model = DNN(num_features=28*28,\n", + " num_hidden_1=100,\n", + " num_hidden_2=50,\n", + " num_hidden_3=20,\n", + " num_classes=10)\n", + "\n", + "model = model.to(DEVICE)" + ] + }, + { + "cell_type": "markdown", + "id": "956117f2", + "metadata": {}, + "source": [ + "## 2-2. Training\n", + "여기서는 **Optimizer**와, **Epoch**를 설정 합니다. " + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "5925349f", + "metadata": {}, + "outputs": [], + "source": [ + "# 수정 가능!\n", + "LEARNING_LATE = 0.05 # 흠.. 이 친구는 학습률로는 너무 큰 것 같네요..\n", + "NUM_EPOCHS = 5" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "eaee3612", + "metadata": {}, + "outputs": [], + "source": [ + "optimizer = torch.optim.SGD(model.parameters(), lr=LEARNING_LATE)" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "e7d2292a", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Epoch: 001/005 | Batch 000/937 | Cost: 2.2828\n", + "Epoch: 001/005 | Batch 040/937 | Cost: 2.1484\n", + "Epoch: 001/005 | Batch 080/937 | Cost: 1.7366\n", + "Epoch: 001/005 | Batch 120/937 | Cost: 1.1916\n", + "Epoch: 001/005 | Batch 160/937 | Cost: 1.0406\n", + "Epoch: 001/005 | Batch 200/937 | Cost: 0.9223\n", + "Epoch: 001/005 | Batch 240/937 | Cost: 0.6654\n", + "Epoch: 001/005 | Batch 280/937 | Cost: 0.5018\n", + "Epoch: 001/005 | Batch 320/937 | Cost: 0.5587\n", + "Epoch: 001/005 | Batch 360/937 | Cost: 0.3497\n", + "Epoch: 001/005 | Batch 400/937 | Cost: 0.2533\n", + "Epoch: 001/005 | Batch 440/937 | Cost: 0.5394\n", + "Epoch: 001/005 | Batch 480/937 | Cost: 0.3384\n", + "Epoch: 001/005 | Batch 520/937 | Cost: 0.4324\n", + "Epoch: 001/005 | Batch 560/937 | Cost: 0.4670\n", + "Epoch: 001/005 | Batch 600/937 | Cost: 0.3825\n", + "Epoch: 001/005 | Batch 640/937 | Cost: 0.3746\n", + "Epoch: 001/005 | Batch 680/937 | Cost: 0.3031\n", + "Epoch: 001/005 | Batch 720/937 | Cost: 0.4151\n", + "Epoch: 001/005 | Batch 760/937 | Cost: 0.2952\n", + "Epoch: 001/005 | Batch 800/937 | Cost: 0.0914\n", + "Epoch: 001/005 | Batch 840/937 | Cost: 0.2318\n", + "Epoch: 001/005 | Batch 880/937 | Cost: 0.3609\n", + "Epoch: 001/005 | Batch 920/937 | Cost: 0.3543\n", + "Epoch: 001/005 Train Acc.: 91.93% | Test Acc.: 92.05%\n", + "Time elapsed: 0.61 min\n", + "Epoch: 002/005 | Batch 000/937 | Cost: 0.3091\n", + "Epoch: 002/005 | Batch 040/937 | Cost: 0.2163\n", + "Epoch: 002/005 | Batch 080/937 | Cost: 0.3328\n", + "Epoch: 002/005 | Batch 120/937 | Cost: 0.1989\n", + "Epoch: 002/005 | Batch 160/937 | Cost: 0.3994\n", + "Epoch: 002/005 | Batch 200/937 | Cost: 0.2397\n", + "Epoch: 002/005 | Batch 240/937 | Cost: 0.1656\n", + "Epoch: 002/005 | Batch 280/937 | Cost: 0.2632\n", + "Epoch: 002/005 | Batch 320/937 | Cost: 0.1910\n", + "Epoch: 002/005 | Batch 360/937 | Cost: 0.2166\n", + "Epoch: 002/005 | Batch 400/937 | Cost: 0.2527\n", + "Epoch: 002/005 | Batch 440/937 | Cost: 0.1963\n", + "Epoch: 002/005 | Batch 480/937 | Cost: 0.2518\n", + "Epoch: 002/005 | Batch 520/937 | Cost: 0.1424\n", + "Epoch: 002/005 | Batch 560/937 | Cost: 0.3110\n", + "Epoch: 002/005 | Batch 600/937 | Cost: 0.2781\n", + "Epoch: 002/005 | Batch 640/937 | Cost: 0.2678\n", + "Epoch: 002/005 | Batch 680/937 | Cost: 0.3961\n", + "Epoch: 002/005 | Batch 720/937 | Cost: 0.1312\n", + "Epoch: 002/005 | Batch 760/937 | Cost: 0.1270\n", + "Epoch: 002/005 | Batch 800/937 | Cost: 0.1271\n", + "Epoch: 002/005 | Batch 840/937 | Cost: 0.1438\n", + "Epoch: 002/005 | Batch 880/937 | Cost: 0.2130\n", + "Epoch: 002/005 | Batch 920/937 | Cost: 0.2932\n", + "Epoch: 002/005 Train Acc.: 94.24% | Test Acc.: 94.10%\n", + "Time elapsed: 1.15 min\n", + "Epoch: 003/005 | Batch 000/937 | Cost: 0.2285\n", + "Epoch: 003/005 | Batch 040/937 | Cost: 0.3392\n", + "Epoch: 003/005 | Batch 080/937 | Cost: 0.0894\n", + "Epoch: 003/005 | Batch 120/937 | Cost: 0.1540\n", + "Epoch: 003/005 | Batch 160/937 | Cost: 0.0687\n", + "Epoch: 003/005 | Batch 200/937 | Cost: 0.1881\n", + "Epoch: 003/005 | Batch 240/937 | Cost: 0.1823\n", + "Epoch: 003/005 | Batch 280/937 | Cost: 0.1942\n", + "Epoch: 003/005 | Batch 320/937 | Cost: 0.1589\n", + "Epoch: 003/005 | Batch 360/937 | Cost: 0.0991\n", + "Epoch: 003/005 | Batch 400/937 | Cost: 0.1238\n", + "Epoch: 003/005 | Batch 440/937 | Cost: 0.1856\n", + "Epoch: 003/005 | Batch 480/937 | Cost: 0.1634\n", + "Epoch: 003/005 | Batch 520/937 | Cost: 0.1580\n", + "Epoch: 003/005 | Batch 560/937 | Cost: 0.2050\n", + "Epoch: 003/005 | Batch 600/937 | Cost: 0.1828\n", + "Epoch: 003/005 | Batch 640/937 | Cost: 0.1502\n", + "Epoch: 003/005 | Batch 680/937 | Cost: 0.1647\n", + "Epoch: 003/005 | Batch 720/937 | Cost: 0.0720\n", + "Epoch: 003/005 | Batch 760/937 | Cost: 0.2033\n", + "Epoch: 003/005 | Batch 800/937 | Cost: 0.1668\n", + "Epoch: 003/005 | Batch 840/937 | Cost: 0.2332\n", + "Epoch: 003/005 | Batch 880/937 | Cost: 0.1393\n", + "Epoch: 003/005 | Batch 920/937 | Cost: 0.1288\n", + "Epoch: 003/005 Train Acc.: 92.69% | Test Acc.: 92.31%\n", + "Time elapsed: 1.68 min\n", + "Epoch: 004/005 | Batch 000/937 | Cost: 0.2751\n", + "Epoch: 004/005 | Batch 040/937 | Cost: 0.0914\n", + "Epoch: 004/005 | Batch 080/937 | Cost: 0.0505\n", + "Epoch: 004/005 | Batch 120/937 | Cost: 0.0561\n", + "Epoch: 004/005 | Batch 160/937 | Cost: 0.0955\n", + "Epoch: 004/005 | Batch 200/937 | Cost: 0.2285\n", + "Epoch: 004/005 | Batch 240/937 | Cost: 0.1542\n", + "Epoch: 004/005 | Batch 280/937 | Cost: 0.2561\n", + "Epoch: 004/005 | Batch 320/937 | Cost: 0.0894\n", + "Epoch: 004/005 | Batch 360/937 | Cost: 0.2445\n", + "Epoch: 004/005 | Batch 400/937 | Cost: 0.0331\n", + "Epoch: 004/005 | Batch 440/937 | Cost: 0.0181\n", + "Epoch: 004/005 | Batch 480/937 | Cost: 0.2004\n", + "Epoch: 004/005 | Batch 520/937 | Cost: 0.1909\n", + "Epoch: 004/005 | Batch 560/937 | Cost: 0.0417\n", + "Epoch: 004/005 | Batch 600/937 | Cost: 0.1163\n", + "Epoch: 004/005 | Batch 640/937 | Cost: 0.1222\n", + "Epoch: 004/005 | Batch 680/937 | Cost: 0.1415\n", + "Epoch: 004/005 | Batch 720/937 | Cost: 0.0653\n", + "Epoch: 004/005 | Batch 760/937 | Cost: 0.1513\n", + "Epoch: 004/005 | Batch 800/937 | Cost: 0.0719\n", + "Epoch: 004/005 | Batch 840/937 | Cost: 0.0350\n", + "Epoch: 004/005 | Batch 880/937 | Cost: 0.1256\n", + "Epoch: 004/005 | Batch 920/937 | Cost: 0.0653\n", + "Epoch: 004/005 Train Acc.: 95.43% | Test Acc.: 94.93%\n", + "Time elapsed: 2.19 min\n", + "Epoch: 005/005 | Batch 000/937 | Cost: 0.2026\n", + "Epoch: 005/005 | Batch 040/937 | Cost: 0.1414\n", + "Epoch: 005/005 | Batch 080/937 | Cost: 0.0562\n", + "Epoch: 005/005 | Batch 120/937 | Cost: 0.0519\n", + "Epoch: 005/005 | Batch 160/937 | Cost: 0.1836\n", + "Epoch: 005/005 | Batch 200/937 | Cost: 0.1411\n", + "Epoch: 005/005 | Batch 240/937 | Cost: 0.0556\n", + "Epoch: 005/005 | Batch 280/937 | Cost: 0.1112\n", + "Epoch: 005/005 | Batch 320/937 | Cost: 0.0346\n", + "Epoch: 005/005 | Batch 360/937 | Cost: 0.1714\n", + "Epoch: 005/005 | Batch 400/937 | Cost: 0.0754\n", + "Epoch: 005/005 | Batch 440/937 | Cost: 0.1088\n", + "Epoch: 005/005 | Batch 480/937 | Cost: 0.1427\n", + "Epoch: 005/005 | Batch 520/937 | Cost: 0.1141\n", + "Epoch: 005/005 | Batch 560/937 | Cost: 0.1694\n", + "Epoch: 005/005 | Batch 600/937 | Cost: 0.1194\n", + "Epoch: 005/005 | Batch 640/937 | Cost: 0.0535\n", + "Epoch: 005/005 | Batch 680/937 | Cost: 0.0200\n", + "Epoch: 005/005 | Batch 720/937 | Cost: 0.1488\n", + "Epoch: 005/005 | Batch 760/937 | Cost: 0.1087\n", + "Epoch: 005/005 | Batch 800/937 | Cost: 0.1105\n", + "Epoch: 005/005 | Batch 840/937 | Cost: 0.0965\n", + "Epoch: 005/005 | Batch 880/937 | Cost: 0.1250\n", + "Epoch: 005/005 | Batch 920/937 | Cost: 0.0988\n", + "Epoch: 005/005 Train Acc.: 96.90% | Test Acc.: 96.35%\n", + "Time elapsed: 2.64 min\n", + "Total Training Time: 2.64 min\n" + ] + } + ], + "source": [ + "def compute_accuracy_and_loss(model, data_loader, device):\n", + " correct_pred, num_examples = 0, 0\n", + " cross_entropy = 0.\n", + " for i, (features, targets) in enumerate(data_loader):\n", + " \n", + " features = features.view(-1, 28*28).to(device)\n", + " targets = targets.to(device)\n", + "\n", + " logits, probas = model(features)\n", + " cross_entropy += F.cross_entropy(logits, targets).item()\n", + " _, predicted_labels = torch.max(probas, 1)\n", + " num_examples += targets.size(0)\n", + " correct_pred += (predicted_labels == targets).sum()\n", + " return correct_pred.float()/num_examples * 100, cross_entropy/num_examples\n", + " \n", + "\n", + "start_time = time.time()\n", + "train_acc_lst, test_acc_lst = [], []\n", + "train_loss_lst, test_loss_lst = [], []\n", + "\n", + "for epoch in range(NUM_EPOCHS):\n", + " \n", + " model.train()\n", + " \n", + " for batch_idx, (features, targets) in enumerate(train_loader):\n", + " \n", + " ### PREPARE MINIBATCH\n", + " features = features.view(-1, 28*28).to(DEVICE)\n", + " targets = targets.to(DEVICE)\n", + " \n", + " ### FORWARD AND BACK PROP\n", + " logits, probas = model(features) # 모델 계산\n", + " cost = F.cross_entropy(logits, targets) # 크로스 엔트로피 계산\n", + " optimizer.zero_grad() # 기울기 초기화\n", + " \n", + " cost.backward() # 역전파\n", + " \n", + " ### UPDATE MODEL PARAMETERS\n", + " optimizer.step() # step 적용\n", + " \n", + " ### LOGGING\n", + " if not batch_idx % 40:\n", + " print (f'Epoch: {epoch+1:03d}/{NUM_EPOCHS:03d} | '\n", + " f'Batch {batch_idx:03d}/{len(train_loader):03d} |' \n", + " f' Cost: {cost:.4f}')\n", + "\n", + " # 매 Epoch마다 evaluation을 진행합니다. \n", + " # Epoch마다 Loss를 기록하여 학습과정을 살펴보고 Underfitting, Overfitting 여부를 확인합니다.\n", + " model.eval()\n", + " with torch.set_grad_enabled(False): # Gradient 계산이 안되도록\n", + " train_acc, train_loss = compute_accuracy_and_loss(model, train_loader, device=DEVICE) # train acc, loss 계산\n", + " test_acc, test_loss = compute_accuracy_and_loss(model, test_loader, device=DEVICE) # test acc, loss 계산\n", + " \n", + " # list에 train, test의 acc, loss 추가\n", + " train_acc_lst.append(train_acc)\n", + " test_acc_lst.append(test_acc)\n", + " train_loss_lst.append(train_loss)\n", + " test_loss_lst.append(test_loss)\n", + " \n", + " # 로깅\n", + " print(f'Epoch: {epoch+1:03d}/{NUM_EPOCHS:03d} Train Acc.: {train_acc:.2f}%'\n", + " f' | Test Acc.: {test_acc:.2f}%')\n", + " \n", + " # 1 epoch 학습 소요시간\n", + " elapsed = (time.time() - start_time)/60\n", + " print(f'Time elapsed: {elapsed:.2f} min')\n", + "\n", + "# 총 학습 소요시간\n", + "elapsed = (time.time() - start_time)/60\n", + "print(f'Total Training Time: {elapsed:.2f} min')" + ] + }, + { + "cell_type": "markdown", + "id": "02201bb2", + "metadata": {}, + "source": [ + "## 2-3. Evaluation\n", + "테스트 데이터와 학습 데이터의 Loss 변화를 확인 합니다." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "88d756c7", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAEGCAYAAACtqQjWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABEtUlEQVR4nO3dd3gUVffA8e9Jp4Tee+8lgQhIifSiCCogJYAignQEUVAs2H6vHV6UIliR0JsISBMpopQEQu899N5bkvv7YxbeGFKWkMmmnM/z7MPuzL0zZ4fAyZ1yrhhjUEoppZKCm6sDUEoplXZoUlFKKZVkNKkopZRKMppUlFJKJRlNKkoppZKMh6sDcKVcuXKZYsWKuToMpZRKVUJDQ88ZY3LHti5dJ5VixYoREhLi6jCUUipVEZEjca3T019KKaWSjCYVpZRSSUaTilJKqSSTrq+pKKVSrrt37xIeHs6tW7dcHUq65ePjQ6FChfD09HS6jyYVpVSKFB4ejq+vL8WKFUNEXB1OumOM4fz584SHh1O8eHGn++npL6VUinTr1i1y5sypCcVFRIScOXM+9EhRk4pSKsXShOJaiTn+mlQS4drtCEbM38Hlm3ddHYpSSqUomlQSYc+pq0xed4SB0zYTGaXz0SiVFp0/fx4/Pz/8/PzIly8fBQsWvP/5zp078fYNCQlhwIABCe6jdu3aSRLrypUradmyZZJs61HphfpEqF40OyNaVeTtedv5fMkehrUo5+qQlFJJLGfOnISFhQEwYsQIMmfOzJAhQ+6vj4iIwMMj9v9CAwICCAgISHAff//9d5LEmpLoSCWROtcqSqeaRRi/6gC/hh13dThKqWTw4osvMnjwYBo0aMDQoUPZsGEDtWvXxt/fn9q1a7Nnzx7g3yOHESNG8NJLL1G/fn1KlCjB6NGj728vc+bM99vXr1+ftm3bUq5cOYKCgrg3K++iRYsoV64cdevWZcCAAQmOSC5cuMAzzzxDlSpVqFWrFlu3bgVg1apV90da/v7+XL16lZMnTxIYGIifnx+VKlVizZo1j3yMdKTyCEY8XZH9p6/xxqytlMiVmcqFsro6JKXSpPd/28HOE1eSdJsVCmThvacrPnS/vXv3snz5ctzd3bly5QqrV6/Gw8OD5cuX89ZbbzF79uwH+uzevZs///yTq1evUrZsWXr37v3Asx+bN29mx44dFChQgDp16rB27VoCAgJ45ZVXWL16NcWLF6djx44Jxvfee+/h7+/PvHnzWLFiBV27diUsLIwvvviCMWPGUKdOHa5du4aPjw8TJkygWbNmDB8+nMjISG7cuPHQxyMmHakkljF4ebgxtnM1cmbyoucvIZy9etvVUSmlbNauXTvc3d0BuHz5Mu3ataNSpUoMGjSIHTt2xNrnqaeewtvbm1y5cpEnTx5Onz79QJsaNWpQqFAh3Nzc8PPz4/Dhw+zevZsSJUrcf07EmaTy119/0aVLFwAaNmzI+fPnuXz5MnXq1GHw4MGMHj2aS5cu4eHhwWOPPcaPP/7IiBEj2LZtG76+vok9LPfpSCUxzu2DWd3guYnkylOeCV0DaDv+b3pPDmVKj1p4eWiuViopJWZEYZdMmTLdf//OO+/QoEED5s6dy+HDh6lfv36sfby9ve+/d3d3JyIiwqk2906BPYzY+ogIw4YN46mnnmLRokXUqlWL5cuXExgYyOrVq1m4cCFdunTh9ddfp2vXrg+9z+j0f7/E8PCGa2dhchu4HE6lgln5rG1VQo5c5L352xP1g6CUSn0uX75MwYIFAfjpp5+SfPvlypXj4MGDHD58GIDp06cn2CcwMJDg4GDAulaTK1cusmTJwoEDB6hcuTJDhw4lICCA3bt3c+TIEfLkyUOPHj3o3r07mzZteuSYNakkRrYi0HkW3L5qJZabF2lVtQC965dk6oZjTF5/1NURKqWSwRtvvMGbb75JnTp1iIyMTPLtZ8iQgbFjx9K8eXPq1q1L3rx5yZo1/mu3I0aMICQkhCpVqjBs2DB+/vlnAEaNGkWlSpWoWrUqGTJkoEWLFqxcufL+hfvZs2czcODAR45Z0vNv1QEBAeaRJuk6tNpKKgUDoMtcIt296TEphNV7zzL55ZrUKpEz6YJVKp3ZtWsX5cuXd3UYLnft2jUyZ86MMYa+fftSunRpBg0alGz7j+3vQURCjTGx3jOtI5VHUTwQnh0PR/+GOS/jThSjOvhRJGdG+gRvIvzio99JoZRK3yZOnIifnx8VK1bk8uXLvPLKK64OKV62JhURaS4ie0Rkv4gMi2W9iMhox/qtIlLtIfoOEREjIrmiLXvT0X6PiDSz75tFU6kNNP8Edv0Gv79BFm8PJnYN4G5kFD0mhXLjzoMX5JRSylmDBg0iLCyMnTt3EhwcTMaMGV0dUrxsSyoi4g6MAVoAFYCOIlIhRrMWQGnHqycwzpm+IlIYaAIcjbasAtABqAg0B8Y6tmO/Wr2h9gDY+B2s+ZKSuTMzuqM/u09d4fWZW/XCvVIq3bBzpFID2G+MOWiMuQNMA1rHaNMamGQs64BsIpLfib4jgTcAE2Nb04wxt40xh4D9ju0kj8bvQ5X2sOJD2DyZBmXzMLR5ORZuO8nYlQeSLQyllHIlO5NKQeBYtM/hjmXOtImzr4i0Ao4bY7YkYn+ISE8RCRGRkLNnzzr/bRLi5gatvoGSDWH+ANi7lFcCS9CqagG+WLqH5TsffNhJKaXSGjuTSmyF+GOeB4qrTazLRSQjMBx4N5H7wxgzwRgTYIwJyJ07dyxdHoGHFzw/CfJVgpkvIMc38WmbKlQskIVXp4ex/8zVpN2fUkqlMHYmlXCgcLTPhYATTraJa3lJoDiwRUQOO5ZvEpF8Tu7Pft6+EDQLMueBKe3IcOUQE7oE4OPpRo9JoVy+oXOwKJUaPErpe7AePIyrCvFPP/1Ev379kjrkFMHOpLIRKC0ixUXEC+si+vwYbeYDXR13gdUCLhtjTsbV1xizzRiTxxhTzBhTDCuRVDPGnHJsq4OIeItIcayL/xts/H5xy5wHOs+x3k9+jgLuVxjXuTrhF2/QX+dgUSpVuFf6PiwsjF69et2/CyssLAwvL68E+8eXVNIy25KKMSYC6AcsAXYBM4wxO0Skl4j0cjRbBBzEuqg+EegTX98E9rcDmAHsBBYDfY0xSf+Iq7NyloROM+H6WQhuy2P5PPigdSVW7z3Lp4t3uywspVTihYaG8sQTT1C9enWaNWvGyZMnARg9ejQVKlSgSpUqdOjQgcOHDzN+/HhGjhyJn59fvCXljxw5QqNGjahSpQqNGjXi6FHrptaZM2fefwI+MDAQgB07dlCjRg38/PyoUqUK+/bts/9LPyRbC0oaYxZhJY7oy8ZHe2+Avs72jaVNsRifPwY+TmS4Sa9Qdesay5T2MKMLHTvNZOeJK0xYfZDy+X151r+QqyNUKnX4fRic2pa028xXGVp84nRzYwz9+/fn119/JXfu3EyfPp3hw4fzww8/8Mknn3Do0CG8vb25dOkS2bJlo1evXg9M7BWbfv360bVrV1544QV++OEHBgwYwLx58/jggw9YsmQJBQsW5NKlSwCMHz+egQMHEhQUxJ07d2wpDfOo9Il6u5VuAq2+hoMr4de+vNuyHDWL52Do7G1sDb/k6uiUUk66ffs227dvp0mTJvj5+fHRRx8RHh4OQJUqVQgKCmLy5MlxzgYZl3/++YdOnToB0KVLF/766y8A6tSpw4svvsjEiRPvJ4/HH3+c//u//+PTTz/lyJEjZMiQIQm/YdLQ0vfJwT8Irp2CPz7A0zcvY4PeodU3a+k5KZT5/euQx9fH1REqlbI9xIjCLsYYKlasyD///PPAuoULF7J69Wrmz5/Phx9+GOe8Ks4QsW5kHT9+POvXr2fhwoX4+fkRFhZGp06dqFmzJgsXLqRZs2Z89913NGzYMNH7soOOVJJL3cHwWA/4+2tybvuOCV2rc/nmXXpP3sTtiJQ3hFVK/Zu3tzdnz569n1Tu3r3Ljh07iIqK4tixYzRo0IDPPvuMS5cuce3aNXx9fbl6NeHHCGrXrs20adMACA4Opm7dugAcOHCAmjVr8sEHH5ArVy6OHTvGwYMHKVGiBAMGDKBVq1b3pwpOSTSpJBcRaPEplG8FS96i4vllfNGuKqFHLvLuvB1aykWpFM7NzY1Zs2YxdOhQqlatip+fH3///TeRkZF07tyZypUr4+/vz6BBg8iWLRtPP/00c+fOTfBC/ejRo/nxxx+pUqUKv/zyC//9738BeP3116lcuTKVKlUiMDCQqlWrMn36dCpVqoSfnx+7d+9+5Am17KCl7x+l9H1i3L0Fk5+DYxug82y+2JePb/7cz/utKvJC7WLJG4tSKZiWvk8ZtPR9SufpAx2CIWcpmBbE4Mq3aFw+Dx8s2MnfB865OjqllHokmlRcIUN26DwbfLLgNqUdo5rloHiuTPQN3sSxCzoHi1Iq9dKk4ipZC1qJJeIWmWe25/t2JYiMMvSYFML12zoHi1KAXmt0scQcf00qrpSnPHScDpeOUnRJN8Y8X569p68yZOYW/cek0j0fHx/Onz+v/xZcxBjD+fPn8fF5uEce9DkVVyv6OLT9HmZ0pV7Y67zV7EM+WryPr1fsZ0Cj0q6OTimXKVSoEOHh4STpFBXqofj4+FCo0MNV/tCkkhKUfxqe/BwWvkb3THnY4fcSXy3bS7l8vjStmM/V0SnlEp6enhQvXtzVYaiHpKe/UorHXoZ6Q5DNk/g810KqFMrKoOlh7D2tc7AopVIPTSopScO3wb8zHn99zi9Vd5DBy4Mek0K4dCPhuRuUUiol0KSSkohAy1FQuilZVwxjWuB5Tl66Rf+pm4mIjHJ1dEoplSBNKimNuye0+wkK+FNq9QDGBd5hzb5zfPK7zsGilEr5NKmkRF6ZoNMMyFKQRpsHMMQfvvvrELNDw10dmVJKxUuTSkqVKRd0mQPuXvQ9/jpPFjW8OXcbm49edHVkSikVJ00qKVn2YtB5FnLrCqMjP6ZE5ghe+SWU01duuToypZSKlSaVlC5/VWj/Cx4X9jE7+zfcuX2DV34J5dZdnYNFKZXyaFJJDUo2gGfGkenkOn4vHMyWYxd4e952LV+hlEpxNKmkFlXaQdOPyH98MbOL/8as0GP8uPawq6NSdjmzy5p7R6lURsu0pCa1+8OVk1RbN4YvCmRh6CI3yuT1pW7pXK6OTCWVm5dgyXAImwwlG1p3Abp7ujoqpZymI5XUpulHUKkNbS9MpGfWDfSdsokj56+7OiqVFPYugbG1YMtUKNcSDqyAha+BnuZUqYgmldTGzQ2eGQfFA3nj9tfUNmH0mBTCNZ2DJfW6eRHm9oIpz1sTuL283JodtN5rsOlnWDvK1REq5TRNKqmRhze0D0Zyl+cbj5FkOLuV12aEERWlv9GmOrsXwZhasHUGBL4OPVdCwWrWugZvQ6U2sHwEbJ/jyiiVcpomldTKJwt0noV7plxMy/wVu3ZuZfSKfa6OSjnrxgWY3QOmdbQedO2xwioo6uH9vzZubtB6LBSuZY1kjq53XbxKOUmTSmrmmw+6zMHHHeZk/oJfloewePtJV0elErJrAYypCTvmwBPDoMefUMAv9raePtBhijX99LSOcOFgsoaq1MPSpJLa5SqNdJpBTnOBaZm/4u0Z69l96oqro1KxuX4eZnWH6UHgm9dKJg3eBA+v+PtlyglBs8BEQXA7a5SjVApla1IRkeYiskdE9ovIsFjWi4iMdqzfKiLVEuorIh862oaJyFIRKeBYXkxEbjqWh4nIeDu/W4pS+DGk3U+UijzI1+4j6fXzOi5e1zlYUpSdv8LYmtafDYZbCSV/lVib7j9zlVenbebv/ef+tzBnSegwFS4dhemdIeJ2MgWu1MOxLamIiDswBmgBVAA6ikiFGM1aAKUdr57AOCf6fm6MqWKM8QMWAO9G294BY4yf49XLnm+WQpVtjrQcyeMmjIHXv6ZvcKjOwZISXD8HM1+EGV0hSwHrQvwTb8T67Mn12xH85/ddNB+1hnlhJ+gdvIljF278r0HRx607/46shfn99VZjlSLZOVKpAew3xhw0xtwBpgGtY7RpDUwylnVANhHJH19fY0z0czuZAP2XdU/1F6D+Wzzrtpq6R8fy8aJdro4o/TLGumNrTA3rGkrDd+DlPyBfpViaGhZuPUmjL1fx7aqDtKlWiLl9ahNlDH2CN3E7Ilqdt8ptrQv6W6fDyv8k4xdSyjl2JpWCwLFon8Mdy5xpE29fEflYRI4BQfx7pFJcRDaLyCoRqffoXyEVeuINqN6NPh7ziVr3LTNCjiXcRyWta2eskcmsbpCtCLyyGgKHxDo62X/mGl2+tx5izZnZi9m9a/Np2yr4F8nOl+2qsu34ZT74bee/O9UbAv6dYdWnEDYlmb6UUs6xs0yLxLIs5qgirjbx9jXGDAeGi8ibQD/gPeAkUMQYc15EqgPzRKRijJENItIT61QbRYoUcfa7pB4i8NSXRF09zXt7J/HqvOyUzP0q1Ytmd3VkaZ8xsH02LHod7lyDRu9B7QHg/uA/sxt3Ivh6xX6+W3MQH093PmhdkaCaRXF3+9+PftOK+XjliRJ8u+ogAcWy86x/IWvFvWmnLx2D+QMgayEoHphMX1Kp+Nk5UgkHCkf7XAg44WQbZ/oCTAHaABhjbhtjzjvehwIHgDIxOxhjJhhjAowxAblz536oL5RquLnj1u4HogoE8IXHN0yY9DOnLmtxQltdPW1dQJ/dHXKUgFfWQL3BDyQUYwy/bztJ4y9XMW7lAVr7FeTPIfXp+nixfyWUe15vWpYaxXPw1pzt7D199X8r3D3h+UnWBfxpneHsHru/oVJOsTOpbARKi0hxEfECOgDzY7SZD3R13AVWC7hsjDkZX18RKR2tfytgt2N5bscFfkSkBNbF//R7U79nBjw6z8BkL87nEZ/yfz/O0jlY7GCM9TT8mBqwbxk0+RC6L4U85R5oeujcdV74cSO9gzeRJYMns3o9zhftqpIrs3csG7Z4uLvxTUd/Mnl70Gty6L/L8WTIBkEzrQcmg9tap92UcjHbkooxJgLr1NQSYBcwwxizQ0R6ici9O7MWYf3Hvx+YCPSJr6+jzycisl1EtgJNgYGO5YHAVhHZAswCehlj0vcN/Rlz4P3CXLwyZOati2/z2fTlOgdLUrpyEqZ2hDk9IFcZ6PUX1BkAbu7/anbzTiRfLNlDs5Gr2XzkIu89XYEF/esSUCyHU7vJk8WHbzr5c/jcdYbO3vrvv8NsRaDTNLh2FqZ2gDs34t6QUslA0vN/MgEBASYkJMTVYdjv9A5uT2jK0btZWVc/mC4N/V0dUepmDGyZBouHWs+LNHwHavV+IJkYY1i68zQf/LaT45du8px/QYY9WY48vj6J2u24lQf4dPFu3nu6At3qFP/3yt0LYVoQlG8J7SZZJV6UsomIhBpjAmJbpz956UHeingGTaWY2xnKr+zJ2l16R1iiXTkBU9rDvF6Quzz0Wgu1+z2QUA6fu063nzbyyi+hZPb2YHrPWnzV3i/RCQXglcASNC6fl48X7iL0yMV/ryz3FDT7P9j1Gyx7J9H7UOpRaVJJJ9xKBBL5zLdUc9vH7endOHxGS7k8FGNg82SrovCh1dD8E+i2CHKV+lezW3cj+WrpHpqOXE3I4Yu807ICCwbUpWaJnI8cgpub8GW7quTP5kO/KZs4fy3GU/W1ekONnvDPN7Dxu0fen1KJoUklHfHxa8OlwA9pyEa2TezB1ZtaysUpl8Nhchv4ta/18GLvtbGe7lq+8zSNv1rF6BX7aVE5Hytee4LudYvj6Z50/8yyZvRkXFB1zl+/w6vTw4iMPt2BiJXsyjS3bmveuzTJ9quUszSppDM5GvYnvOIrPH13MX98+7rOwRIfYyD0Z2t0cvQfaPE5vLDAuo03mqPnb9D9p428PCmEDJ7uTO1Ri/928CdPlsSf6opPpYJZeb9VRdbsO8foP2JMd+DmDm2+h7yVrIcvT261JQal4qJJJR0q1PZT9udvyTOXfmJ58GeuDidlunQMfnkWfhtglaXv/TfU7PmvC+C37kYyavleGo9cxbqD5xn+ZHkWDazH4yUf/VRXQjo8Vpg21QoxesU+Vu09+++V3pmtue19slmzSV4+bns8St2jSSU9EqFk9x/Zk7kGDfd/QshSLfVxnzEQ8oM1V/yxDfDUl9B1PuT4991WK3afpunI1Yxavo9mFfPxx2v16RFYIklPdcVHRPjomUqUzevLq9M2c/zSzX83yJIfOk2H29esGwtuX419Q0olMU0q6ZR4eFGszywOe5ag4tqBHApb6eqQXO/iEZjUGhYMsqb07fMPPPbyv0Ynxy7coMekEF76KQRPd2HKyzX5uqM/+bLac6orPhm83BkbVI27kYa+wZu4ExGjKnW+SvD8T3BmJ8zsBpERsW5HqaSkSSUd886Ylazd53JOcpB9XmcuHduZcKe0KCrKultq7ONwPNSqq9V1PmQver/JrbuRfP3HPhp/tYq1+88xrEU5fh8YSO1SuVwXN1Aid2Y+a1uFsGOX+L/YqlKXamyNtvYvg9/f0HL5ynZ2FpRUqUDu/EXY2W4GGWe0JOqn1tztvwrPbAVcHVbyuXDImpvk8Boo0QBajbaeUo9m5Z4zjJi/g8Pnb/BU5fwMf6o8BbJlcFHAD3qycn661y3O938dolrR7LSqGuPvL6AbXDwEa/9rncar3d81gap0QUcqigoVqxIWOJGMEZc59+3TcOuyq0OyX1QUrP8WxtWGk1ug1dfQZe6/Ekr4xRu88ksIL/64ETcRfulegzFB1VJUQrlnWItyVC+anWGzt7L/TCzXTxqNgArPwNJ3rNknlbJJgklFRFqKiCafNK5Ro+b8WuY/5LpxiNMT26Xt6WrPH4CfW1qng4rWtq6dVOtqPecB3I6IZMyf+2n81SpW7z3HG83L8vur9ahXOuVWtfZ0d2NMp2pk8HSn9+RNXL8d4/qJmxs8Ox4KPQZzekJ4OihPpFzCmWTRAdgnIp+JSHm7A1Ku83yHbnyfczB5z6/nQnB367f5tCQqCtaNg3F14NR2aD0WgmZZ85E4rN57lhaj1vD5kj00KJuH5a89QZ/6pfD2cI9nwylDvqw+jO7oz/6z1xg+d9uDxUM9M0DHqeCbz7oj7OJhl8Sp0rYEk4oxpjPgjzU/yY8i8o+I9BQRX9ujU8nKw92Nji8P5VvPruQ49BvXFgxzdUhJ59x++LEFLB5mTWjVdx34B90fnZy4dJM+waF0/WEDUcbwU7fHGNe5OgVT4Kmu+NQplYvBjcswL+wEk9cffbBBplxWIo2KgOB2cPPig22UegROndZyzJ44G2uu+PzAs8AmEdErfmlM1oyeNOz+MZNNczJv+pa7a/7r6pAeTVQk/P0NjK8DZ3fBM+Ot5zeyWBez70REMW7lARp9uYoVu88wpGkZlgwKpH7ZPC4OPPH6NihF/bK5+fC3nWw5dunBBrlKQ4dg6yaF6V0gQsv1qKTjzDWVp0VkLrAC8ARqGGNaAFWBITbHp1ygdL4s5G03koWRNfD8413M1hmuDilxzu2DH5rD0uHWnV191oNfx/ujk7X7z9Hiv6v5dPFu6pXOxbJBT9CvYelUcaorPm5uwsjn/cjt602f4E1cvB5L0ihWF1qPse56+22A3mqskowzI5V2wEhjTBVjzOfGmDMAxpgbwEu2RqdcpkmlAhwJHMm6qPJEze0NB/50dUjOi4q0bp8dVwfO74PnJlrXErLkB+DU5Vv0nbKJoO/WExFl+PHFx5jQNYDCOTK6OPCkkz2TF2ODqnHm6i0GzwiLvcZb1fZQ/y3YMhVWabkelTScmqRLRPIBNQADbDTGnLI7sOSQbibpSiRjDK//soaX9/ehlNcFPLr/Dvmrujqs+J3dA/P6wPEQKNcSnvoKfPMCcDcyih/XHmLU8n1ERhn6NihFz8AS+Him7pFJfH755zDv/LqDIU3L0K9h6QcbGGMdry1T4NkJVqJRKgGPNEmXiHQHNgDPAW2BdSKiI5R0QET4oENtPsz2IWcjfIiY1Cbl3jEUGQFrvoLx9eDCQatSb/vJ9xPK3wfO8eR/1/B/i3ZTu2ROlg16ggGNSqfphALQuVZRWvsV4Ktle1m7/9yDDUTg6f9CsXpWaf/DfyV/kCpNSXCkIiJ7gNrGmPOOzzmBv40xZZMhPlvpSMU5xy7c4NWvp/KjeZdM2fPg/vIy6y6ilOL0Tvi1D5zYDOVbWWVJMlsX2k9fucXHC3cxf8sJCufIwHstK9K4Ql4XB5y8rt+OoPWYtVy8foeFA+rFXqfs5kX4vilcOwMvL7cu5isVh0edTjgciP6I7lVA56NNRwrnyMiQzs/w8t0hRF4KxwQ/D3euuzosiLwLqz+HCU/ApaPQ7ido/wtkzsPdyCi+W3OQhl+sZPGOUwxsVJplg55IdwkFIJO3B+M7V+Pm3Uj6TdnE3chYnj/KkB2CZoK7JwS3heuxjGqUcoIzSeU4sF5ERojIe8A6YL+IDBaRwfaGp1KKx0vmpGXLZ+l3uy/mxGaY+aL1n7qrnNoO3zWCFR9Z87P33QAVnwVg/cHztBz9Fx8t3EWN4jlYNiiQQU3KpPlTXfEplceXT9pUIeTIRT79fXfsjbIXg47T4OopmNoR7t6MvZ1S8XAmqRwA5mFdpAf4FTgJ+DpeKp3oUqsoOao/xzt3X4R9S2HBq8l/K2rkXetOpQn14coJeH6SNULJlIszV28xaHoY7Ses49rtCCZ0qc4PLz5G0ZyZkjfGFKpV1QK88HhRvvvrEL9vOxl7o0IB8NwECN8Ic3ulvaoKynYJVik2xrwP4HiC3hhjrtkelUqRRIQPWlei05l2jDl5ib6bJ4Nvfmj4dvIEcHKrde3k1Dao1BZafAaZchIRGcWkf44wctlebkdE0b9hKfrUL0UGr/Q7MonL8KcqsCX8Mq/P2kq5/FkoniuWhFuhNTT5AJa9A38UgybvJ3ucKvVy5u6vSiKyGdgO7BCRUBGpaH9oKiXy8nBjXOfqTPYJ4jf3xtY1jY3f2bvTiDvw539gYgO4ehraB0Pb7yFTTjYevkDLr//igwU78S+anSWDAnmtaVlNKHHw8nBjTFA1PN2F3pNDuXknMvaGtftDwEuwdhSE/pScIapUzpnTXxOAwcaYosaYosBrwER7w1IpWW5fb77tGsDQ290I9a6JWTgEdv1mz85ObrGSyapPoFIb6Lseyrfk7NXbDJ4RRrvx/3D1VgTjO1fn526Pxf6bt/qXgtkyMLK9H3tOX+XtedsfLDwJ1q3GLT6HUk1gwWDYvzz5A1WpkjNJJZMx5v7j1MaYlYD+y03nqhTKxv+18afz5VcIz1geZnWHI/8k3Q4iblsX4Sc0sO5E6jgNnptAhHc2fv77MA2/XMlvW07Qt0FJlg0OpHmlfIij/IpKWP2yeejfsDSzN4UzfWMcN3O6e0C7HyFPBZjxonVzhFIJcCapHBSRd0SkmOP1NnDI7sBUyveMf0G6Blag1YUBXPHJB1Pbw5lYprR9WMc3WRfiV38OVdpbFYXLtiD0yAVafbOW9+bvwK9wNha/GsjrzcqR0UsnME2MgY1KU690Lt6dv4Ptx+OYmM3b1yrA6Z0ZpjwPV+K4wK+UgzNJ5SUgNzDH8coFdLMzKJV6vNG8HFXKlOTpi69xR7xgchu4fDxxG4u4Dcvfh+8aWw/jdZoBz47jXGRGXp+5hTbj/uHijTuMDarGpJdqUDJ35qT9MumMu5swqr0fOTN50Sd4E5dvxnGLeNaC1t/FrctWYrmt9+qouMX7RL2IuANLjDGNky+k5KNP1CeNyzfv8syYteS7sZdgj/dxy1oYXvrdeqDOWeGh1p1dZ3eDf2do+jGR3lmZsv4Iny/Zw407kbxcrwT9G5Yik7eOTJJS6JGLtP/2H+qXzcPErtXjPo24d6k1Gi3dFDpMATe9GSK9SvQT9caYSOCGiGRN5I6bi8geEdkvIg/M+CSW0Y71W0WkWkJ9ReRDR9swEVkqIgWirXvT0X6PiDRLTMzq4WXN4MnErgFsjyzKOz7DMOf3w7QguHsr4c53b8Gyd+H7xnD7KgTNhtZj2HTW0HrMX7zz6w4qFczK4lfrMaxFOU0oNqheNDtvPVme5btO8+3qg3E3LNMUnvwc9i62JjvTcvkqFs78C70FbBORZcD92hzGmAHxdXKMcsYATbBKvWwUkfnGmJ3RmrUASjteNYFxQM0E+n5ujHnHsY8BwLtALxGpgDX1cUWgALBcRMo4EqOyWak8mRnVwY+XJ0VQqtgwuh35EOb0sB5MjOs32mMbrdHJub1Q7QVo+iEXIjPw2eytTNt4jLxZvPmmkz9PVc6vF+Ft1q1OMUKPXOTzJXvwK5yNWiVyxt7wsZetyb3++QayF4fH+yRvoCrFcyapLHS8onPmV5QawH5jzEEAEZkGtAaiJ5XWwCRjnYNbJyLZRCQ/UCyuvo5ZKO/JFC2W1sA0Y8xt4JCI7HfEkIS3JKn4NCqflyFNy/L+EihTcTB1dn0Fvw+1fruNnhTu3oQ/P4Z/xkCWgtB5DpElGjJt41E+W7ye67cj6BlYggGNSpNZRybJQkT4pE1ldp28Qv+pm1nYvy55ssRSeBKgyYdw6QgseQuyFYHyLZM3WJWiOXOhPpsx5ufoL8CZk+UF+XfhyXDHMmfaxNtXRD4WkWNAENZIxdn9ISI9RSRERELOnj3rxNdQD6NP/ZK0rJKfzjsDOFq2O2ycCH999b8GR9fB+Lrw99dQ/UXo/TdbvKvz7Ni1DJ+7nfL5fVk0sB5vPVleE0oy8/XxZFzn6ly7FUG/qZuJiK3wJICbmzX3SsFqMPtlOB6avIGqFM2ZpPJCLMtedKJfbOcrYo5w4moTb19jzHBjTGEgGOj3EPvDGDPBGBNgjAnInTt3rIGrxBMRPmtbhfL5stBydxOuln4W/vgANn4Pi9+ypveNuANdf+Vig095c9ERnhm7llOXb/HfDn5M7VGLMnm1pJyrlM3ny8fPVmLDoQt8sXRv3A29MlrPDmXODVM6wMUjyRekStHiTCoi0lFEfgOKi8j8aK8/gfNObDscKBztcyHghJNtnOkLMAVo8xD7U8kgo5cHE18IwNPDgzYngrhb9AlYOBjWjYHHuhPVay1Tz5WgwZcrmRFyjO51ivPHa0/Q2q+gXjtJAZ6rVohONYswftUBlu08HXfDzHkgaBZE3rZuNb55KdliVClXfCOVv4Evgd2OP++9XgOaO7HtjUBpESkuIl5YF9Hnx2gzH+jquAusFnDZGHMyvr4iEn32oFaO+O5tq4OIeItIcayL/xuciFPZoGC2DIwNqsbBi3cZEDWIqJq94IXf2Fb1XZ79YRtvztlGmby+LBpQj7dbVsDXx9PVIato3m1ZgcoFszJ4RhhHz9+Iu2HustYMm+cPwMwXXDsdgkoRnJqjPtEbF3kSGAW4Az8YYz4WkV4AxpjxYv1a+g1WkroBdDPGhMTV17F8NlAWiAKOAL2MMccd64ZjPawZAbxqjPk9vvj0ORX7TV53hLfnbefF2sWIiIoieP1RcmbyZvhT5XhGRyYp2rELN2j59V8Uyp6B2b1rxz8fTdgUmNfbesao1Tf/vjFDpTnxPafizHTCzwGfAnmwrlsIVgn8LEkdaHLTpJI83pq7jSnrj+Im8ELtYgxqUoYsOjJJFf7YdZruP4fQsUZh/vNclfgbr/gYVn8GDd+BwCHJE6ByifiSijO313wGPG2MSYKiTio9GvF0RYrmyEi90rmpUCDV/y6SrjQqn5c+9UsyduUBqhfNQdvqheJu3OAtuHgYVnxozSJZuW1yhalSEGeSymlNKOpReHm48coTJV0dhkqkwU3KsPnoJd6et42KBbJQPn8cvxiIQOtv4Mpx61RYloJQ9PHkDVa5nDO3FIeIyHTH3WDP3XvZHplSKkXwcHdjdEd/svh40id4E1duxXMx3sPbunCfrQhM62hdwFfpijNJJQvWRfSmwNOOlz5Cq1Q6ktvXm286VePohRsMnbU19om97smYA4JmgrhBcFu47swTCCqtSDCpGGO6xfJ6KTmCU0qlHDWK52Bo87L8vv0U3/+VwJRKOUpAh6nWNAjTOjlXXFSlCc7MUV9GRP4Qke2Oz1UcE3UppdKZHvVK0KxiXj75fTchhy/E37hITXjuWzi2ziocGhVH2ReVpjhz+msi8CZwF8AYsxXrYUSlVDojInzerioFs2eg75RNnLt2O/4OFZ+FxiNg+2z486NkiVG5ljNJJaMxJuaT6RF2BKOUSvmy+HgyLqg6l27cZeC0zURGJfAAdZ1XrakN1nwJm35JlhiV6ziTVM6JSEkcxRlFpC2gE1UrlY5VKJCFD5+pxNr95xm1PJ7Ck2DdavzUl1CyISx4FQ78mSwxKtdwJqn0Bb4FyonIceBVoJedQSmlUr7nAwrzfEAhvl6xnz93n4m/sbsntPsZcpWFGV3h9M7426tUy5m7vw465qjPDZQzxtQ1xmida6UUH7SuRPn8WXh1ehjhF+MpPAngkwWCZoBnRquq8dVTyROkSlbOjFQAMMZcN8ZctTMYpVTq4uPpzrigakRFGfoEb+J2RAKzd2ctBJ2mw40LMLUD3Lkef3uV6jidVJRSKjbFcmXii+ersjX8Mh8tcKKiUwE/aPsDnNwCs3tAVAKJSKUqmlSUUo+sWcV89AwswS/rjvBr2PGEO5RtDs0/hT0LYak+9paWOPPwYzsR8XW8f1tE5ohINftDU0qlJq83K0uNYjkYNnsb+047caa8Zk+o1QfWjYX1E+wPUCULZ0Yq7xhjropIXaAZ8DMwzt6wlFKpjae7G1938ieTtwe9Jody7bYTj7M1/QjKPgWLh8KexfYHqWznTFK5d8LzKWCcMeZXwMu+kJRSqVXeLD583dGfQ+eu8+acbfEXngRwc4c2EyF/VZjVDU6EJUucyj7OJJXjIvIt8DywSES8neynlEqHHi+ZkyHNyvLblhNM+seJpw+8MkHH6ZAxJ0xpD5fD7Q9S2caZ5PA8sARoboy5BOQAXrczKKVU6tYrsCSNyuXho4U72XT0YsIdfPNa5fLv3oDg5+HWFfuDVLZwJqnkBxYaY/aJSH2gHRCzFphSSt3n5iZ89bwfebP40C94Exeu30m4U57y8PwkOLcHZr4AkfFMBqZSLGeSymwgUkRKAd8DxYEptkallEr1sma0Ck+eu3aHV6eHJVx4EqBkA2g5Cg6sgIWvQULXZFSK40xSiTLGRADPAaOMMYOwRi9KKRWvyoWyMqJVRVbvPcs3K/Y716laF6j3Gmz6GdaOsjU+lfScSSp3RaQj0BVY4FjmaV9ISqm0pGONwjznX5BRf+xl9d6zznVq8DZUagPLR8D2ObbGp5KWM0mlG/A48LEx5pCIFAcm2xuWUiqtEBE+erYSZfL4MnDaZk5cuplwJzc3aD0WCteCub3g6Hr7A1VJwpkqxTuBIcA2EakEhBtjPrE9MqVUmpHRy4OxnatxN9LQd8om7kQ4MbWwpw90mAJZC8K0jnDhoP2BqkfmTJmW+sA+YAwwFtgrIoH2hqWUSmtK5s7Mp22qsPnoJf7zuxOFJwEy5YSgWdYF++B2VnVjlaI5c/rrS6CpMeYJY0wgVqmWkfaGpZRKi56qkp9udYrx49rDLNh6wrlOOUtaI5ZLR2F6Z4i4bW+Q6pE4k1Q8jTF77n0wxuxFL9QrpRLpzRblqVYkG0NnbeXA2WvOdSr6ODwzDo6shfn99VbjFMyZpBIqIt+LSH3HayIQandgSqm0ycvDjTFB1fD2dKf35FBu3HGi8CRA5bbQ8B3YOh1W/sfeIFWiOZNUegE7gAHAQGAnTs5RLyLNRWSPiOwXkWGxrBcRGe1YvzV6Sf24+orI5yKy29F+rohkcywvJiI3RSTM8RrvTIxKqeSXP2sG/tvBj31nrjF87vaEC0/eU+818O8Mqz6FMH0GOyWKN6mIiBsQaoz5yhjznDHmWWPMSGNMgic1RcQd6+J+C6AC0FFEKsRo1gIo7Xj1xFFSP4G+y4BKxpgqwF7gzWjbO2CM8XO8nEp8SinXqFc6N4Mal2Hu5uNM2XDUuU4i1hP3xZ+A+QPg0GpbY1QPL96kYoyJAraISJFEbLsGsN8Yc9AYcweYBrSO0aY1MMlY1gHZRCR/fH2NMUsdT/gDrAMKJSI2pVQK0K9BKZ4ok5v35+9ka/gl5zq5e1o1wnKWhGmd4eyehPuoZONsQckdIvKHiMy/93KiX0HgWLTP4Y5lzrRxpi/AS8Dv0T4XF5HNIrJKROrFFpSI9BSREBEJOXvWyad7lVK2cHMTRrX3I1dmL3pP3sSlG04UngTIkM2qauzhDcFt4doZW+NUznMmqbwPtAQ+wLq9+N4rIRLLspgnTuNqk2BfERkORADBjkUngSLGGH9gMDBFRLI8sBFjJhhjAowxAblz507gKyil7JY9kxdjO1fnzNVbDJ6xhShnCk8CZCsCnabBtbMwtQPcuWFvoMopcSYVESklInWMMauiv7D+c3dmFp1woHC0z4WAmDemx9Um3r4i8gJWogsyjit8xpjbxpjzjvehwAGgjBNxKqVczK9wNt5pWYEVu88wbtUB5zsWrA5tv4fjm+DHFnBqm31BKqfEN1IZBVyNZfkNx7qEbARKi0hxEfECOgAxT5vNB7o67gKrBVw2xpyMr6+INAeGAq2MMfd/NRGR3I4L/IhICayL/1rXQalUokutojxdtQBfLt3D3wfOOd+x3FPQ/he4chwm1Ic/PoS7t2yLU8UvvqRSzBizNeZCY0wIUCyhDTsupvfDmjVyFzDDGLNDRHqJyL07sxZh/ce/H5gI9Imvr6PPN4AvsCzGrcOBwFYR2QLMAnoZY7Smg1KphIjwyXOVKZE7MwOmbub0lYdIDOWfhr4boPLzsOYL+LYeHF1nX7AqThLX/eEist8YU+ph16UmAQEBJiQkxNVhKKWi2Xf6Kq3HrKVigSxM6VELT3dnLv1Gs385/DYILh+DGj2g0bvg7WtPsOmUiIQaYwJiWxff39ZGEekRy8a6o0/UK6VsUjqvL/95rjIbD1/k8yWJuF24VGPo8w/UfAU2TIQxtWDfsqQPVMXKI551rwJzRSSI/yWRAMALeNbmuJRS6Vhrv4KEHrnIhNUHqVYkO80r5Xu4DXhnhhafWhN9/drPuu24Snto9h+r8rGyTZwjFWPMaWNMbaxbig87Xu8bYx43xpxKnvCUUunV8KfKU7VwNl6fuYVD564nbiOFa0CvNRD4BmyfDWNqWH9qQUrbODNJ15/GmK8drxXJEZRSSnl7uDOmkz/u7kLvyaHcuhuZuA15eEPD4dBzlfVsy6yXYGpHuHw8aQNWgHMPPyqllEsUyp6Rke392HP6Ku/M2/5oG8tXCV5eDk0/hoMrYWwtCPkBopyYhVI5TZOKUipFa1A2D/0blGJmaDgzNh5LuEN83Nyhdj/o8zcU8IMFg+Dnp+H8QzxwqeKlSUUpleINbFyGuqVy8c6v29lx4vKjbzBHCeg6H1p9bT2FP642/DUKIp2c20XFSZOKUirFc3cT/tvBj+wZvegTvInLN+8++kZFoFpX6Lveug15+XvwXUM4+cAz3+ohaFJRSqUKOTN7MybIn+MXb/L6zC3OT+yVkCz5of1kaPczXDnpKPXygZZ6SSRNKkqpVKN60Ry8+WR5lu48zcQ1SVjaTwQqPmONWqp2gDVfwvi6cOSfpNtHOqFJRSmVqrxUpxhPVs7Hp4v3sP7g+aTdeMYc8MxY6DwHIm/Dj81h4Wtw60rS7icN06SilEpVRIRP21ShaI6M9Ju6mTNXbThNVaoR9P4HavaGjd/D2Mdh79Kk308apElFKZXq+Pp4MrZzNa7eusuAqZuJiLThWRPvzNDiE+i+1Ho/pR3M7gHXk3h0lMZoUlFKpUrl8mXh42cqs+7gBb5atte+HRWuAa+shieGwY65MOYx2DZLS73EQZOKUirValO9EB1rFGHsygPMCnVmQtpE8vCGBm9aySV7MZjd3ZrCWEu9PECTilIqVXvv6QrUKJaDITO30Cc4lHPXbtu3s7wVoPsyaPZ/cHAVjKlpXXPRUi/3aVJRSqVqPp7uTOlRkzeal2X5zjM0HbmaBVtP2LdDN3d4vK81Z0tBf1g4GH5uCef227fPVESTilIq1fNwd6NP/VIsGFCXwtkz0G/KZvoEh3LezlFLjuKOUi/fwKntjlIvI9N9qRdNKkqpNKNMXl9m9659f9TSZORqFm49ad8ORaBaF+i3Aco0heUjYGIDOLnFvn2mcJpUlFJpSvRRS6HsGeg7ZRN9gzfZO2rxzWeVenl+Elw9BRMaWAnm7k379plCaVJRSqVJZfL6Mqd3bV5vVpZlO0/TdORqFm2zcdQCUKG1NWqp2tE6FTa+Lhz52959pjCaVJRSaZaHuxt9G5Tit/51KZAtA32CN9F3is2jlgzZ4Zkx0GUuRN6BH1ukq1IvmlSUUmle2Xy+zO1jjVqW7jhF05Gr+d3uUUvJhtBnHdTq4yj1Ugv2LrF3nymAJhWlVLpwb9SyoH89CmTLQO/gTfSbsokL1+/Yt1OvTND8P9azLd5ZYMrzMPtluH7Ovn26mCYVpVS6UjafL3P61GZI0zIs2XGKJl+tsn/UUvgx62n8+m/CjnkwpgZsnZkmS71oUlFKpTue7m70a1ia3/rXJX82H3oHb6L/1M32jlo8vKD+MOi1BrIXhzkvw5T2cNnG8jIuoElFKZVulcuXhbl96vBakzIs3n6SpiNXsXj7KXt3mqe8Vfm42X/g8BoYUws2fpdmSr1oUlFKpWue7m70b1Sa+f3qkjeLD70mh9J/6mYu2jlqcXOHx/tYpV4KVbfuDvvpqTRR6kWTilJKAeXzZ2Fe3zoMdoxamiTHqCV7MegyD1qPgTM7rFIva76CyLv27tdGtiYVEWkuIntEZL+IDItlvYjIaMf6rSJSLaG+IvK5iOx2tJ8rItmirXvT0X6PiDSz87sppdIeT3c3BsQYtQycZvOoRQT8O0PfjVCmGfzxvlXq5USYffu0kW1JRUTcgTFAC6AC0FFEKsRo1gIo7Xj1BMY50XcZUMkYUwXYC7zp6FMB6ABUBJoDYx3bUUqph3Jv1DKocRkWbj1Jk5GrWbLD5lGLb15o/ws8/wtcOwMTG6bKUi92jlRqAPuNMQeNMXeAaUDrGG1aA5OMZR2QTUTyx9fXGLPUGHOvDOg6oFC0bU0zxtw2xhwC9ju2o5RSD83T3Y2Bja1RSx5fb175JRlGLQAVWkHf9eDnKPUyrg4cXmvvPpOQnUmlIHAs2udwxzJn2jjTF+Al4PeH2B8i0lNEQkQk5OzZs058DaVUelahQBZ+7ffvUctSu0ctGbJb11m6zIOoCPjpSVgwKFWUerEzqUgsy2I+6RNXmwT7ishwIAIIfoj9YYyZYIwJMMYE5M6dO5YuSin1bzFHLT1/CeXVaZu5dMPmUUvJBtYdYo/3g9CfrFIvexbbu89HZGdSCQcKR/tcCIg5HVtcbeLtKyIvAC2BIGPuP5LqzP6UUirR7o1aXm1cmgWOUcuynaft3alXJmj2MXRfDj5ZYWp7mNU9xZZ6sTOpbARKi0hxEfHCuog+P0ab+UBXx11gtYDLxpiT8fUVkebAUKCVMeZGjG11EBFvESmOdfF/g43fTymVDnm6u/Fq4zL82q8OuTJ702NSCIOmh9k/ailUHXqugvpvwc5f4ZvHYOuMFFfqxbak4riY3g9YAuwCZhhjdohILxHp5Wi2CDiIdVF9ItAnvr6OPt8AvsAyEQkTkfGOPjuAGcBOYDHQ1xgTadf3U0qlbxULZOXXvnUY2Kg0v205QZORq1lu96jFwwvqD7VKveQsCXN6WEUqU1CpFzEpLMslp4CAABMSEuLqMJRSqdyOE5d5bcYWdp+6ynP+BXnv6Ypkzehp706jImHDBPjjAxA3aDwCArqDm/3PtItIqDEmILZ1+kS9Uko9oooFsjK/X13rwcktJ2gycpX9oxY3d6jV21Hq5TFYNMS6S+zcPnv3m1BYLt27UkqlEV4ebgxuUoZ5feuQI5MXL08KYfCMMC7fsLnkSvZi1iyTz4yDM7us51pWf+GyUi+aVJRSKglVKugYtTQsxa9h1qjlj102j1pEwK8T9N0AZZvDig9hgmtKvWhSUUqpJObl4cbgpmX51TFq6f5zMo1afPPC85Og/WS47ij1suzdZC31oklFKaVsEnPU0nTUKlbstnnUAlD+aUepl06w9r+OUi9/2b9fNKkopZSt7o1a5vWpQ7YMXrz0UwivzdjC5Zs2j1oyZIfW30DXX8FEWvO1/PYq3Lps6241qSilVDKoXCgr8/vXoX/DUswLO07Tkav4c/cZ+3dcoj70dpR62fSzNdPknt8T7JZYmlSUUiqZeHu481q0UUu3nzYyZGYyjFq8MlqlXl5ebo1gpnawClTaQJOKUkols3ujln4NSjF383GajVzNn3uSYdRSsDr0XAkNhlvPtthAn6jXJ+qVUi60NfwSQ2ZuYe/pa7SrXoi3W1Ygawabn8Z/RPpEvVJKpVBVCmXjt/516dugJHOSc9RiE00qSinlYt4e7rzerBxz+9QmSwYPuv24kTdmbeHKLdc8Ff8oNKkopVQKcW/U0qd+SWaFhtNs5GpWprJRiyYVpZRKQbw93HmjeTnm9qlDZm8PXkxloxZNKkoplQJVLWyNWnpHG7Ws2nvW1WElSJOKUkqlUD6e7gxtXo45feqQyduDF37YwLDZW1P0qEWTilJKpXB+hbOxoH9dej1Rkhkhx2g2cjWrU+ioRZOKUkqlAj6e7gxrUY7ZvWuTyduDro5Ry9UUNmrRpKKUUqmIf5HsKXrUoklFKaVSmeijlgxe7nT9YQNvzkkZoxZNKkoplUr5F8nOwgH1eOWJEkzfeIzmo9awZp9rRy2aVJRSKhXz8XTnzRblmdW7Nj6ebnT5fgNvztnmslGLJhWllEoDqt0btQSWYPrGozQftYa/9p1L9jg0qSilVBrh4+nOm09aoxZvTzc6f7+et+Zu49rtiGSLQZOKUkqlMdWKZGfRgHr0DCzB1A1HaTZydbKNWjSpKKVUGuTj6c5bT5ZnVq/aeHtYo5bhyTBq0aSilFJpWPWi2Vk0sB496hVnimPUsna/faMWTSpKKZXG+Xi6M/ypCszq9TjeHm4EfbeejxfutGVfmlSUUiqdqF40x/1RS5GcmWzZh61JRUSai8geEdkvIsNiWS8iMtqxfquIVEuor4i0E5EdIhIlIgHRlhcTkZsiEuZ4jbfzuymlVGp0b9TSpVZRW7bvYctWARFxB8YATYBwYKOIzDfGRB9ztQBKO141gXFAzQT6bgeeA76NZbcHjDF+Nn0lpZRSCbBzpFID2G+MOWiMuQNMA1rHaNMamGQs64BsIpI/vr7GmF3GmD02xq2UUiqR7EwqBYFj0T6HO5Y508aZvrEpLiKbRWSViNSLrYGI9BSREBEJOXs25VT2VEqptMDOpCKxLDNOtnGmb0wngSLGGH9gMDBFRLI8sBFjJhhjAowxAblz505gk0oppR6GnUklHCgc7XMh4ISTbZzp+y/GmNvGmPOO96HAAaBMoiJXSimVKHYmlY1AaREpLiJeQAdgfow284GujrvAagGXjTEnnez7LyKS23GBHxEpgXXx/2DSfiWllFLxse3uL2NMhIj0A5YA7sAPxpgdItLLsX48sAh4EtgP3AC6xdcXQESeBb4GcgMLRSTMGNMMCAQ+EJEIIBLoZYy5YNf3U0op9SAxJqFLFWlXQECACQkJcXUYSimVqohIqDEmINZ16TmpiMhZ4MgjbCIXkPwTFiRM43o4GtfD0bgeTlqMq6gxJtY7ndJ1UnlUIhISV7Z2JY3r4WhcD0fjejjpLS6t/aWUUirJaFJRSimVZDSpPJoJrg4gDhrXw9G4Ho7G9XDSVVx6TUUppVSS0ZGKUkqpJKNJRSmlVJLRpJIAEflBRM6IyPY41sc50ZiL46ovIpejTVr2bjLEVFhE/hSRXY6J1AbG0ibZj5eTcbniePmIyAYR2eKI6/1Y2rjq58uZ2JL9mDn26+6oRr4glnUuOV5OxOWSY+XY92ER2ebY7wNPeyf5MTPG6CueF1b5l2rA9jjWPwn8jlVZuRawPoXEVR9YkMzHKj9QzfHeF9gLVHD18XIyLlccLwEyO957AuuBWq4+Xg8RW7IfM8d+BwNTYtu3q46XE3G55Fg59n0YyBXP+iQ9ZjpSSYAxZjUQXw2xuCYac3Vcyc4Yc9IYs8nx/iqwiwfnwUn24+VkXMnOcQyuOT56Ol4x75xx1c+XM7ElOxEpBDwFfBdHE5ccLyfiSsmS9JhpUnl0iZ1QLDk87jh98buIVEzOHYtIMcAf6zfc6Fx6vOKJC1xwvBynTMKAM8AyY0yKOV5OxAbJf8xGAW8AUXGsd9XxGkX8cYHr/j0aYKmIhIpIz1jWJ+kx06Ty6BIzoVhy2IRVn6cqVlXnecm1YxHJDMwGXjXGXIm5OpYuyXK8EojLJcfLGBNpjPHDmjOohohUitHEZcfLidiS9ZiJSEvgjLHmS4qzWSzLbD1eTsblsn+PQB1jTDWgBdBXRAJjrE/SY6ZJ5dE99IRiycEYc+Xe6QtjzCLAU0Ry2b1fEfHE+o872BgzJ5YmLjleCcXlquMVbf+XgJVA8xirXP7zFVdsLjhmdYBWInIYmAY0FJHJMdq44nglGJcrf76MMSccf54B5gI1YjRJ0mOmSeXRxTXRmEuJSD4REcf7Glh/1+dt3qcA3wO7jDFfxdEs2Y+XM3G56HjlFpFsjvcZgMbA7hjNXPLz5UxsyX3MjDFvGmMKGWOKYU3ct8IY0zlGs2Q/Xs7E5YqfL8e+MomI7733QFMg5h2jSXrMbJukK60QkalYd27kEpFw4D2si5aYeCYaSwFxtQV6izVp2U2gg3Hc6mGjOkAXYJvjXDzAW0CRaHG54ng5E5crjld+4GexZix1A2YYYxaIExPZJQNnYnPFMXtACjleCcXlqmOVF5jryGcewBRjzGI7j5mWaVFKKZVk9PSXUkqpJKNJRSmlVJLRpKKUUirJaFJRSimVZDSpKKWUSjKaVJSymYhEyv+q04aJyLAk3HYxiaNStVKuoM+pKGW/m45yJ0qleTpSUcpFxJrn4lOx5i3ZICKlHMuLisgfYs1t8YeIFHEszysicx1FCbeISG3HptxFZKJY854sdTwBr5RLaFJRyn4ZYpz+ah9t3RVjTA3gG6xKtzjeTzLGVAGCgdGO5aOBVY6ihNWAHY7lpYExxpiKwCWgja3fRql46BP1StlMRK4ZYzLHsvww0NAYc9BR8PKUMSaniJwD8htj7jqWnzTG5BKRs0AhY8ztaNsohlWWvrTj81DA0xjzUTJ8NaUeoCMVpVzLxPE+rjaxuR3tfSR6rVS5kCYVpVyrfbQ//3G8/xur2i1AEPCX4/0fQG+4P4FWluQKUiln6W80StkvQ7TqyACLjTH3biv2FpH1WL/gdXQsGwD8ICKvA2f5X9XYgcAEEemONSLpDbh8mgWlotNrKkq5iOOaSoAx5pyrY1EqqejpL6WUUklGRypKKaWSjI5UlFJKJRlNKkoppZKMJhWllFJJRpOKUkqpJKNJRSmlVJL5f2MwaQrB2R3FAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plt.plot(range(1, NUM_EPOCHS+1), train_loss_lst, label='Training loss')\n", + "plt.plot(range(1, NUM_EPOCHS+1), test_loss_lst, label='Test loss')\n", + "plt.legend(loc='upper right')\n", + "plt.ylabel('Cross entropy')\n", + "plt.xlabel('Epoch')\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "3b3a090f", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABGwElEQVR4nO3dd3xP1//A8dfJECNixB4RszETEXu3VVp71Kg9S63y1UmH7mqLGqUURe29tahRWxDEJlZsQoZIJPmc3x839VMyPiE3n4z38/HIQ3I/99zzzpW8cz/nnvs+SmuNEEKIjMPO1gEIIYRIWZL4hRAig5HEL4QQGYwkfiGEyGAk8QshRAbjYOsArJEnTx7t7u5u6zCEECJNOXjw4B2tdd6nt6eJxO/u7o6vr6+twxBCiDRFKXUpru0y1COEEBmMqYlfKTVUKeWvlDqulHo3dtsipZRf7MdFpZSfmTEIIYT4L9OGepRSFYC+QDXgEbBRKbVOa93hiX1+AoLNikEIIcSzzBzjLwvs1VqHAyiltgOtgTGxXyugPfDy8xw8KiqKwMBAIiIikilckVplzpyZIkWK4OjoaOtQhEgXzEz8/sDXSilX4CHwBvDkHdq6wE2t9dm4Giul+gH9ANzc3J55PTAwkOzZs+Pu7o7xN0SkR1pr7t69S2BgIMWLF7d1OEKkC6aN8WutTwLfA5uAjcARIPqJXToBCxJoP01r7aO19smb95nZSERERODq6ipJP51TSuHq6irv7IRIRqbe3NVaz9Bae2ut6wFBwFkApZQD0AZY9CLHl6SfMcj/sxDJy+xZPfli/3XDSPT/XuG/CpzSWgea2b8QQqRVIRFRfL76OCERUcl+bLPn8S9TSp0A1gADtdb3Yrd3JIFhnrTg7t27eHl54eXlRYECBShcuPDjrx89epRgW19fX4YMGZJoH7Vq1UqucIUQacjxa8E0n7iTP/Ze4sCFoGQ/vqlP7mqt68azvYeZ/aYEV1dX/Pz8APj8889xdnZmxIgRj1+Pjo7GwSHu0+vj44OPj0+ifezevTtZYk1JMTEx2Nvb2zoMIdKsxQeu8Mkqf3JmdWRhvxr4uOdO9j7kyd1k1KNHD4YPH07Dhg354IMP2L9/P7Vq1aJy5crUqlWL06dPA7Bt2zaaNWsGGH80evXqRYMGDShRogQTJkx4fDxnZ+fH+zdo0IB27drh4eFB586d+XfltPXr1+Ph4UGdOnUYMmTI4+M+6eLFi9StWxdvb2+8vb3/8wdlzJgxVKxYEU9PTz788EMAzp07x6uvvoqnpyfe3t6cP3/+PzEDDBo0iN9//x0wSmp88cUX1KlThyVLljB9+nSqVq2Kp6cnbdu2JTw8HICbN2/SunVrPD098fT0ZPfu3XzyySf8/PPPj487cuTI/5wDITKKh49ieG/JEd5fdpSq7rlZN6SuKUkf0kitnsSMXnOcE9dCkvWY5Qq58Fnz8klud+bMGTZv3oy9vT0hISHs2LEDBwcHNm/ezMcff8yyZcueaXPq1Cm2bt1KaGgoL730EgMGDHhmzvrhw4c5fvw4hQoVonbt2uzatQsfHx/efvttduzYQfHixenUqVOcMeXLl49NmzaROXNmzp49S6dOnfD19WXDhg2sXLmSffv2kTVrVoKCjLeUnTt35sMPP6R169ZERERgsVi4cuVKgt935syZ2blzJ2AMg/Xt2xeAUaNGMWPGDAYPHsyQIUOoX78+K1asICYmhrCwMAoVKkSbNm0YOnQoFouFhQsXsn///iSfdyHSsgt3HjDgj4OcvhnKkFdKM/SV0tjbmTepIV0k/tTkzTfffDzUERwcTPfu3Tl79ixKKaKi4r5J07RpU5ycnHByciJfvnzcvHmTIkWK/GefatWqPd7m5eXFxYsXcXZ2pkSJEo/nt3fq1Ilp06Y9c/yoqCgGDRqEn58f9vb2nDlzBoDNmzfTs2dPsmbNCkDu3LkJDQ3l6tWrtG7dGjASujU6dHj8QDb+/v6MGjWK+/fvExYWRuPGjQH4+++/mTNnDgD29vbkyJGDHDly4OrqyuHDh7l58yaVK1fG1dXVqj6FSA/WH7vO+0uP4mivmNWjKg1eymd6n+ki8T/PlblZsmXL9vjzTz75hIYNG7JixQouXrxIgwYN4mzj5OT0+HN7e3uio6Ot2uff4Z7EjBs3jvz583PkyBEsFsvjZK61fmaqZHzHdHBwwGKxPP766Xn1T37fPXr0YOXKlXh6evL777+zbdu2BOPr06cPv//+Ozdu3KBXr15WfU9CpHWPoi18t+EUM3ddwKtoTiZ39qZwziwp0reM8ZsoODiYwoULAzweD09OHh4eBAQEcPHiRQAWLYr7sYjg4GAKFiyInZ0dc+fOJSYmBoDXXnuNmTNnPh6DDwoKwsXFhSJFirBy5UoAIiMjCQ8Pp1ixYpw4cYLIyEiCg4PZsmVLvHGFhoZSsGBBoqKimDdv3uPtr7zyClOmTAGMm8AhIcbwXOvWrdm4cSMHDhx4/O5AiPTsevBDOk7bw8xdF+hRy53Fb9dMsaQPkvhN9f777/PRRx9Ru3btx8k2OWXJkoVffvmFJk2aUKdOHfLnz0+OHDme2e+dd95h9uzZ1KhRgzNnzjy+Om/SpAktWrTAx8cHLy8vfvzxRwDmzp3LhAkTqFSpErVq1eLGjRsULVqU9u3bU6lSJTp37kzlypXjjevLL7+kevXqNGrUCA8Pj8fbf/75Z7Zu3UrFihWpUqUKx48fByBTpkw0bNiQ9u3by4wgke7tOHObphN2cvpGKJPeqsznLcqTySFlU7GydrjAlnx8fPTTC7GcPHmSsmXL2iii1CMsLAxnZ2e01gwcOJDSpUszbNgwW4eVJBaLBW9vb5YsWULp0qXj3Ef+v0VaF2PRTPz7LD9vOUuZfNn5pYs3JfM6m9qnUuqg1vqZueNyxZ/GTZ8+HS8vL8qXL09wcDBvv/22rUNKkhMnTlCqVCleeeWVeJO+EGnd3bBIeszaz/jNZ2lduTArB9Y2PeknJF3c3M3Ihg0bluau8J9Urlw5AgICbB2GEKY5eOkeg+Yf4u6DR3zbpiIdqxa1ef0pSfxCCGECrTWzdl3km/UnKZQzC8sH1KJC4WfvwdmCJH4hhEhmoRFRfLDsKOuP3aBRufz8+KYnObKknoWEJPELIUQyOnk9hHfmHeJyUDgfv+FB37olbD608zRJ/EIIkUyW+F5h1Ep/cmRxZEHfGlQrbk6tnRclif853b17l1deeQWAGzduYG9vz78rhe3fv59MmTIl2H7btm1kypRJSi8LkQ5ERMXw2arjLPK9Qs0SrkzoVJm82Z0Sb2gjkvifU2JlmROzbds2nJ2dbZ74pYyyEC/m4p0HDJh3iJPXQxjUsBTDGpUxtcBacpB5/Mno4MGD1K9fnypVqtC4cWOuX78OwIQJEyhXrhyVKlWiY8eOXLx4kalTpzJu3Di8vLz4559//nOc+Mo5x8TEMGLECCpWrEilSpWYOHEiAAcOHKBWrVp4enpSrVo1QkND+f333xk0aNDjYzZr1uxxzRxnZ2c+/fRTqlevzp49e/jiiy+oWrUqFSpUoF+/fo/r9cRVnrlr166sWrXq8XE7d+7M6tWrTTunQqRmG/1v0HziTq7df8jMHj6MaPxSqk/6kF6u+Dd8CDeOJe8xC1SE17+zenetNYMHD2bVqlXkzZuXRYsWMXLkSGbOnMl3333HhQsXcHJy4v79++TMmZP+/fvH+y7Bw8MjznLO06ZN48KFCxw+fBgHBweCgoJ49OgRHTp0YNGiRVStWpWQkBCyZEm45seDBw+oUKECX3zxBWDMpf/0008B6Nq1K2vXrqV58+Zxlmfu06cP48aNo2XLlgQHB7N7925mz56dhBMrRNoXFWNhzMZTTP/nAp5FcjC5szdFcmW1dVhWSx+JPxWIjIzE39+fRo0aAcbVecGCBQEe17dp1aoVrVq1SvRY8ZVz3rx5M/3793+8slfu3Lk5duwYBQsWpGrVqgC4uLgkenx7e3vatm37+OutW7cyZswYwsPDCQoKonz58jRo0CDO8sz169dn4MCB3Lp1i+XLl9O2bdt4VxoTIj26ERzBoPmH8L10j241izGyaVmcHNLWcGn6+I1NwpW5WbTWlC9fnj179jzz2rp169ixYwerV6/myy+/fFycLD7xlXOOr4xyXFPFEiqjnDlz5sfj+hEREbzzzjv4+vpStGhRPv/8cyIiIhIs+dy1a1fmzZvHwoULmTlzZoLfixDpya5zdxiy4DAPo2L4uaMXLb0Km9uh1mDCVFAZ408mTk5O3L59+3Hij4qK4vjx449Xr2rYsCFjxox5vDhJ9uzZCQ0NjfNY8ZVzfu2115g6derjev1BQUF4eHhw7do1Dhw4ABglkaOjo3F3d8fPz+9x//GtavXvH4Q8efIQFhbG0qVLAeItzwxGvf3x48cDUL586lkLQQizWCyaiVvO0mXGPnJny8TqQbXNT/pXD8LUOnDnbLIfWhJ/MrGzs2Pp0qV88MEHeHp64uXlxe7du4mJiaFLly5UrFiRypUrM2zYMHLmzEnz5s1ZsWJFnDd34yvn3KdPH9zc3KhUqRKenp7Mnz+fTJkysWjRIgYPHoynpyeNGjUiIiKC2rVrU7x4cSpWrMiIESPw9vaOM+6cOXPSt29fKlasSKtWrR4PGUHc5ZkB8ufPT9myZenZs6cJZ1KI1OXeg0f0mn2AnzadoaVnIVYNqk2pfNnN61Br2PMLzGgMEcEQGfcF4ouQsswiycLDw6lYsSKHDh2Ks/6/GeT/W9jC4cv3GDjvEHfCHvFp83J0ru5m7lO44UGwaiCcXg8vNYWWkyDr8z8EJmWZRbLYvHkzHh4eDB48OMWSvhApTWvN7N0Xaf/rHuzsFMsG1KJLjWLmJv3Le2FqXTi7CZp8Dx3nvVDST0j6uLkrUsyrr77K5cuXbR2GEKYJi4zmw2VHWXv0Oq945GNsey9yZDWxwJrFArvGw99fQc6i0PsvKBz30GxySdOJP74ZLSJ9SQvDkSJ9OH0jlAHzDnLxzgM+aOLB2/VKYGfmA1lht2FFPzj/N5RvDc1/hszmv5NOs4k/c+bM3L17F1dXV0n+6ZjWmrt37z5+jkAIsyw/FMjHK47h7OTIvD41qFnS1dwOL+yAZX2MG7jNxkOVHqZM3YxLmk38RYoUITAwkNu3b9s6FGGyzJkzU6RIEVuHIdKpiKgYRq85wYL9l6lePDcTO1Umn4uJFxqWGNg+BrZ/D3lKQ5flUKCCef3FIc0mfkdHR4oXL27rMIQQadjlu+EMmHeQ49dCGNCgJP9rVAYHexPnvIRch+V94eI/4PkWvPEDOKX82rtpNvELIcSL2HTiJsMX+6GA37r58Gq5/OZ2eG4zLH8bosKh1RTwesvc/hIgiV8IkaFEx1j44a/T/Lo9gAqFXZjSuQpFc5tYYC0mypixs2s85CsPb86CvC+Z158VJPELITKMWyERDFpwmP0Xguhc3Y1PmpUjs6OJBdbuX4GlvSBwP1TpCU2+BceEq+emBFMTv1JqKNAXUMB0rfX42O2DgUFANLBOa/2+mXEIIcTu83cYssCPB5HRjOvgSevKJk8YOLUOVr5j3MxtNxMqtE28TQoxLfErpSpgJP1qwCNgo1JqHVAEaAlU0lpHKqXymRWDEEJYLJop28/z01+nKZ4nG/P7VqdMfhNr7URHwqbPYN8UKOhlJH3Xkub19xzMvOIvC+zVWocDKKW2A60BH+A7rXUkgNb6lokxCCEysPvhjxi++Ah/n7pFc89CfNemItmcTEx7QQGwpCdc94PqA6DRaHBIfWvvmpn4/YGvlVKuwEPgDcAXKAPUVUp9DUQAI7TWB55urJTqB/QDcHNzMzFMIUR6dDTwPgP+OMSt0Ai+aFmermbX2vFfDquHgJ0ddJgHZZuZ19cLMi3xa61PKqW+BzYBYcARjDF9ByAXUAOoCixWSpXQTz2Xr7WeBkwDozqnWXEKIdIXrTV/7LvMl2tOkDe7E0v618KraE7zOox6CH9+DL4zoUhVY2gnZ+q+WDX15q7WegYwA0Ap9Q0QiDEEtDw20e9XSlmAPIA8giuEeCEPIqP5eMUxVvldo+FLeRnb3otc2TKZ1+HtM7C0J9z0h9pD4eVPwN7Egm7JxOxZPfm01reUUm5AG6AmYAFeBrYppcoAmYA7ZsYhhEj/zt4MZcC8QwTcDuO9xi8xoH5JcwusHVkIa4eDY2bovBRKNzKvr2Rm9jz+ZbFj/FHAQK31PaXUTGCmUsofY7ZP96eHeYQQIilW+V3lw2XHyOZkzx+9q1OrVB7zOnv0ANa/B37zoFhtaPsbuBQyrz8TmD3UUzeObY+ALmb2K4TIGCKjY/hy7Qn+2HuZqu65mPSWN/nNLLB28zgs6WGsg1v/A6j3Ptinvedg017EQggBXAkKZ+D8QxwNDObteiUY0fglHM0qsKY1HJoNGz4w6uV3WwUl6pvTVwqQxC+ESHO2nLzJ8MVHsGjNr12r0Lh8AfM6iwiBte+C/zIo0RDaTAPntP3cqSR+IUSaER1jYeymM/yy7TzlCrowpYs3xVyzmdfhNT9jaOf+ZXjlU6g9zJinn8ZJ4hdCpAm3QiMYsuAwewOC6FStKJ81L29egTWtYf80+GsUZMsLPdZBsZrm9GUDkviFEKnevoC7DFpwmNCIKH5805N2VUwssPbwHqwaBKfWQpkmRu38rLnN688GJPELIVIti0Uz7Z8AfvjzNMVyZ2Vu72p4FHAxr8MrB4wyyqHXofE3UOOdFFsHNyVJ4hdCpErB4VH8b8kRNp+8SdOKBfmubUWyZzbpqViLBfZMhC1fgEth6PUnFKliTl+pgCR+IUSqcywwmHfmH+RGcASfNS9Hj1ru5hVYe3AHVvSHc5ugXEtoPgGy5DSnr1RCEr8QItXQWjN//2VGrz5BHudMLHq7Jt5uuczr8OIuWNYbwoOg6U/g0ztdDu08TRK/ECJVCH8UzcgV/qw4fJV6ZfIyvoMXuc0qsGaJgX/GwrZvIHcJeGsxFKxkTl+pkCR+IYTNnbsVxjvzDnL2VhjDG5VhUMNS5hVYC70Jy/vChe1QsT00GwtOJq7IlQpJ4hdC2NSaI9f4cNlRnBztmdOrGnVL5zWvs/N/w/J+EBkGLSeDV+cMMbTzNEn8QgibiIyO4Zt1J5m95xJViuVi0luVKZgjizmdxUQbwzr/jIW8HtB9DeQra05faYAkfiFEigu8F87A+Yc5cuU+feoU54PXPcwrsBYcCMv6wOU94N0NmnwPmbKa01caIYlfCJGitp6+xbBFfsTEaKZ09ub1igXN6+z0RljZH2KioM1vUOlN8/pKQyTxCyFSRIxFM37zGSb+fQ6PAtmZ0qUKxfOYVGAt+hFsGQ17JkGBivDmbHAtaU5faZAkfiFEipi89RwT/z5He58ifNGygnkF1u5dNMouXD0I1fpBoy+N5RHFY5L4hRCmO3LlPj9vOUsLz0KMaedpXkcnVsGqwcbn7ecYT+KKZ0jiF0KY6uGjGIYt9iNfdie+bFnBnE6iIowSygemQ+Eq0G4m5HI3p690QBK/EMJU3244ScDtB8zvU50cWU0osnbnHCztATeOQc1B8Mpn4GDSE7/phCR+IYRptp2+xZw9l+hdpzi1SuVJ/g6OLjGWRbR3hE6L4KUmyd9HOiSJXwhhinsPHvH+0qOUye/Me41fSt6DPwqHDe/D4bngVhPazoAchZO3j3RMEr8QItlprfl4xTHuhT9iVs+qyTuD59ZJYx3c26eh7gho8BHYSypLCjlbQohkt+LwVTb43+CDJh6UL5QjeQ6qNRz+A9a/B07O0HU5lHw5eY6dwSSa+JVSzYD1WmtLCsQjhEjjAu+F89mq41R1z0W/eiWS56CRobB2OBxbDMXrGU/hZs+fPMfOgKwpjtEROKuUGqOUyrhVjYQQiYqxaP63+AgaGNveC/vkKK18/Sj8Wh/8l0LDUdB1pST9F5ToFb/WuotSygXoBMxSSmlgFrBAax1qdoBCiLRjxs4A9l0I4od2lSia+wULoWkNB36DP0dC1tzQfS24106eQDM4q8rhaa1DgGXAQqAg0Bo4pJQabGJsQog05OT1EH788wyNy+enXZUiL3awh/dhcTdYP8IY2um/U5J+MrJmjL850AsoCcwFqmmtbymlsgIngYnmhiiESO0iomIYtsgPlyyOfNO64ostjB540HggK+SaUWen5iCwM6lkcwZlzayeN4FxWusdT27UWocrpXqZE5YQIi0Zu+kMp26EMqtHVVydnZ7vIFrDnsmw+TPIXgh6boSiVZM3UAFYN8bfTSlVQCnVAtDAAa31jdjXtpgdoBAiddtz/i7T/wmgc3U3Gnrke76DhAfBygFwZiN4NIOWkyBLruQNVDyW6PsnpVRvYD/QBmgH7LX2Sl8pNVQp5a+UOq6Uejd22+dKqatKKb/YjzdeIH4hhA2FREQxYskR3F2zMbLpc076u7QHptYx1sN9/Qfo8IckfZNZM9TzPlBZa30XQCnlCuwGZibUSClVAegLVAMeARuVUutiXx6ntf7xuaMWQqQKn686zo2QCJb2r0nWTEl8HtRigV3j4O+vIacb9P4LClU2J1DxH9b8TwUCT07bDAWuWNGuLLBXax0OoJTajjEbSAiRDqw7ep3lh68y9JXSVHZL4hV62C1Y3g8CtkKFttBsPGR2MSVO8SxrEv9VYJ9SahXGGH9LYL9SajiA1npsPO38ga9j3yE8BN4AfIG7wCClVLfYr/+ntb73dGOlVD+gH4Cbm1uSvikhhLluhkQwcuUxPIvkYNDLpZLWOGCbkfQjgqH5BGMB9BeZBSSSzJo5UueBlRhJH2AVcB3IHvsRJ631SeB7YBOwETgCRANTMKaGesUe56d42k/TWvtorX3y5s1rRZhCiJSgtea9pUeJiIphbAcvHO2tnGppiYGt38CcVpA5J/T9G6p0l6RvA9bM6hkNoJTKbnypw6w9uNZ6BjAjtv03QKDW+ua/ryulpgNrkxq0EMJ25u69xI4zt/myVQVK5nW2vuG2b2HHD+DVBd4YA5lMWmhdJMqaWT0VlFKHMYZujiulDiqlyltzcKVUvth/3TBmBS1QShV8YpfWsccVQqQB526F8c36k9Qvk5cu1ZMwBHvhH9jxI3h1hlaTJenbmDVj/NOA4VrrrQBKqQbAdKCWFW2XxY7xRwEDtdb3lFJzlVJeGENHF4G3kx62ECKlRcVYGL7YjyyO9vzQrpL1T+c+uAvL+4JrKXh9jLlBCqtYk/iz/Zv0AbTW25RSVv251lrXjWNb1yTEJ4RIJSZuOcvRwGCmdvEmn0tm6xppDasGQvhdeGuxUUdf2Jw1iT9AKfUJRp0egC7ABfNCEkKkNocu32PS1nO09S5CkwoFE2/wr32/wpkN0OR7KFjJvABFklhzO74XkBdYHvuRB+hpZlBCiNTjQWQ0wxf5UTBHFj5rUc76htePwKZPoMzrUF1GdFOTBK/4lVL2wBKt9aspFI8QIpX5at1JLgWFs7BvDVwyO1rXKDIMlvaCrK7QcrJM2UxlErzi11rHAOFKqWRaNFMIkZZsOXmTBfsv069eCaqXcLW+4Yb34e55aDMNsiWhnUgR1ozxRwDHlFKbgAf/btRaDzEtKiGEzd0Ni+SDZUfxKJCd4Y3KWN/w6BLwmwf13jMWURGpjjWJf13sx5N0XDsKIdIHrTUfLj9GyMNo/uhTHScHe+saBgXA2mFQtAbU/9DcIMVzsybx59Ra//zkBqXUUJPiEUKkAkt8A9l04iajmpbFo4CVxdOiHxnj+nZ20HY62CexWqdIMdbM6ukex7YeyRyHECKVuHw3nNFrjlOzhCu9ahe3vuHfX8C1w9BiklFmWaRa8f5JVkp1At4CiiulVj/xUnaMCptCiHQmxqIZvtgPOzvFj+09sbOzcjbOuc2weyL49IJyLcwNUrywhN6L7caonpmH/1bQDAWOmhmUEMI2pm4/j++le4zv4EXhnFmsaxR6E1b0h3zloPE35gYokkW8iV9rfQm4BNRMuXCEELbifzWYcZvO0LRSQVp6FbKukcUCK9425u13XwOOVv6xEDZlTXXONkqps0qpYKVUiFIqVCkVkhLBCSFSRkRUDMMW+ZE7Wya+blXB+gJsuycYq2g1+RbyPeeauyLFWXPbfQzQPHZhFSFEOvT9xlOcvRXGnF7VyJk1k3WNAn3h7y+hXEuo0sPU+ETysmZWz01J+kKkXzvP3mHWrov0qOVOvTJWrnYXEWxM3cxeEJr/LCUZ0hhrrvh9lVKLMJZfjPx3o9Z6uVlBCSFSRnB4FCOWHKFk3mx80MTDukZaGw9pBQdCzw2QJYkLrQubsybxuwDhwGtPbNMYlTqFEGnYqFX+3AmLZHq32mTJZOXTuYf/AP9l8PIocKtuboDCFNasuSslmIVIh1b5XWXNkWuMeK0MFYtYWYfx9mmjAJt7Xagz3NwAhWmsmdVTRim1RSnlH/t1JaXUKPNDE0KY5dr9h3yy0h9vt5z0r1/SukZREca4vmMWaDMd7Kx8hyBSHWtu7k4HPsJYNxet9VGgo5lBCSHMY7Fo3lt6hGiLZmx7LxzsrUkDwKZP4aY/tJoCLklYhUukOtb8j2fVWu9/alu0GcEIIcw3a/dFdp27yyfNyuGex6rls+HUetj/K9R4B8o0NjdAYTprEv8dpVRJYksxK6XaYZRyEEKkMWduhvL9xlO8WjYfHasWta5R8FVY9Q4UqASvfm5qfCJlWDOrZyAwDfBQSl3FWGi9s6lRCSGS3aNoC+8u9CO7kwPftqlk3dO5lhhY3s8oudxuFjg4mR+oMJ01s3oCgFeVUtkAO611qPlhCSGS27jNZzhxPYTp3XzIm93KBL7jR7i0E1pNhTylzA1QpBgr7+qA1vqBJP307e9TNxm68DC3QiNsHYpIZgcuBjF1+3k6Vi1Ko3L5rWt0aTds/w4qdQCvTuYGKFKULJEjAKMy4zvzDhERZWFfQBDTulWhUpGctg5LJIPQiCiGLfKjaK6sjGpWzrpG4UGwrC/kLAZNf0p8f5GmWH3FL9KvW6ER9J3jS+6smZjTqxr2dop2U/ew4nCgrUMTyeDLtSe4dv8h4zp44uxkxbWe1rB6MITdhHYzwSm7+UGKFGXNA1xvKqWyx34+Sim1XCnlbX5oIiVERMXw9tyD3A+PYnp3H+qVycvqQbXxdsvJsEVH+HrdCaJjLLYOUzynP4/fYLFvIO80KEWVYrmta3TgNzi1Fl79DArLr3p6ZM0V/yda61ClVB2gMTAbmGJuWCIlaK35ePkxDl++z9j2npQvZDy27+rsxNze1elesxjT/7lAz98PEBweZeNoRVLdCo3go+XHqFDYhSGvlLau0Q1/+HMklHoVagw0N0BhM9Yk/pjYf5sCU7TWqwArC3aL1OzXHQEsP3yV4Y3K8HrFghAdCUEBADja2zG6ZQW+b1uRvQF3aTF5J2duyr39tEJrzYfLjvEgMppx7b3I5GDFr/qjB0ZJhiw5jVk8djISnF5Z8z97VSn1K9AeWK+UcrKynUjFNp+4yfcbT9GsUkEGv1wKoh7CH21hYhWj+mKsDlXdWNivBg8iY2g9eRd/Hb9hw6iFtebvv8zfp27x4eselM5v5Rj9xo/gzhlo/Ss4W1mXX6RJ1iTw9sCfQBOt9X0gN/CeNQdXSg1VSvkrpY4rpd596rURSimtlMqTxJjFCzp9I5ShCw9ToVAOfmjnibJEw5KecHEn5C8PqwbCnsmP969SLDdrBtemZD5n+s09yIQtZ7FYtA2/A5GQC3ce8NXak9QtnYfuNd2ta3R8BRyaDXXehZINzQxPpALWJP6CwDqt9VmlVAPgTeDp2j3PUEpVAPoC1QBPoJlSqnTsa0WBRsDl5wtbPK+gB4/oPfsA2ZwcmN7NhywOykj0ZzZA0x+hzxZjKb0/P4at3xgzPICCObKw+O2atKlcmLGbzjBw/iEeRErJptQmOsbCsEV+ZHKw44d2ntjZWfF07r1LsHooFKkKDUeaH6SwOWsS/zIgRilVCpgBFAfmW9GuLLBXax2utY4GtgOtY18bB7xPbP0fkTIeRVsY8MdBboVGMq2bDwVcnIza6kcXwcufQNU+xiP57WZB5S6w/XvY8AFYjFk9mR3t+am9J6OaluXP4zdoO2U3l++G2/i7Ek+avPU8flfu81WrChTIkTnxBjFRsKw3oKHtb2DvaHqMwvasSfyW2MTdBhivtR6G8S4gMf5APaWUq1IqK/AGUFQp1QK4qrU+klBjpVQ/pZSvUsr39u3bVnQnEqK15rPV/uy7EMQP7SrhVTQnbP0aDkyHWoOh7v/+f2c7e2gxCWoOMioyrnoHYoyre6UUfeqWYHavalwPjqDF5J3sOnfHNt+U+I8jV+4z4e+ztPIqRHPPQtY12voNBB6A5uMhl7uZ4YlUxJrEH6WU6gR0A9bGbkv0siB2gfbvgU3ARuAIRjnnkcCnVrSfprX20Vr75M0rN5pe1OzdF1mw/wrvNChJS6/CsHsS7PgBKneFRl8+u1i2UvDaV9BwFBxZAEu6GwtxxKpbOi+rBtYmr7MT3WbuZ9auC2gtb+Bs5eGjGIYt8iNfdidGt6xgXaOAbbBznPEzUKGtqfGJ1MWaxN8TqAl8rbW+oJQqDvyRSBsAtNYztNbeWut6QBBwEWOo6IhS6iJQBDiklCrwPMEL6+w4c5sv1p6gUbn8jHjtJTg0F/4aaYzlN//52aT/L6Wg/nvw+g/GAz3z20Nk2OOX3fNkY8XA2rzikY/Ra07w/tKjREbHxH0sYapvN5wk4M4DfnrTkxxZrBiuCbttVN3MUxpe/978AEWqkmji11qfAEYAx2Jv2AZqrb+z5uBKqXyx/7phDBXN0Vrn01q7a63dgUDAW2stcwRNcv52GAPnH6JM/uyM6+CF3anVsGYIlHzZ+uXzqvcz5nVf3AlzWxl1XGI5OzkwtUsVhr5SmiUHA+nw615uhkiRt5S07fQt5uy5RJ86xalVyopJchYLrBwAD+8b93MyWbkYi0g3rCnZ0AA4C0wGfgHOKKXqWXn8ZUqpE8AaYKDW+t5zximeQ3B4FH1n++Job8f0bj44B+6AZX2M2Rsd/khabXWvTtB+Dlw/Ar83g9Cbj1+ys1MMa1SGqV28OXMzlOYTd3L4svxXp4R7Dx7x3tKjlMnvzIjGL1nXaO8vcG4TNP4aClg5LCTSFWuGen4CXtNa148dsmmMMSsnUVrrulrrclprT631ljhed9day51BE0THWBi04BBX7oUztUsVij7wh4WdIU8ZeGvR813llW0GnZfAvYsws7ExDfAJTSoUZPk7tXBytKPDr3tZ4nsleb4ZESetNR+vOMb98EeM6+BFZkcr3r1dOwybPwePZsYsLpEhWZP4HbXWp//9Qmt9Bitu7grb+nr9Sf45e4evWlWgWpZrMK8dZC8AXVdAllzPf+ASDaD7anh4z0j+t07952WPAi6sHliHqsVz8d7So4xec1yKvJlk+aGrbPC/wfBGLz2us5SgyFCjJINzPmgxMf57OyLdsybxH1RKzVBKNYj9mA4cNDsw8fwW7L/MrF0X6V2nOB1KRMHc1uCYDbqtMn7pX1QRH+i5HrQFZr0OVw/95+Vc2TIxu2c1etUuzqxdF+k+az/3Hjx68X7FY1eCwvls9XGqueemX70S1jVaN8J4t9b2N8hqZaVOkS5Zk/j7A8eBIcBQ4ETsNpEK7Q24yycr/alXJi8f1XaBOa1Ax0C3lZDTLfk6yl8eem0EJ2eY3cK48fsEB3s7Pm1ejh/aVeLAhXu0mLyTUzdCkq//DCzGovnfEuMxmJ/ae2JvzdO5RxbC0YVQ/wMoVsvkCEVql2DiV0rZAQe11mO11m201q211uO01pEpFJ9IgitB4Qz44yBurlmZ1NINh3ltjCGZLssgr5U3/pIidwno9Se4FDIKvJ3585ld3vQpyqK3axAZZaHNL7vZ6H89+ePIYGbsDGD/hSA+a16OormzJt7g7nlYOxyK1YZ6VpXZEulcgolfa23BmHOfjJeKwgyhEVH0nn0Ai4aZncrisqyT8ba+0wIoVNm8jl0KQc8NkK8sLHwLji19ZpfKbrlYM7gOZfJnp/8fhxi76YwUeXtOJ6+H8OOfZ2hSvgDtqhRJvEF0JCztCQ6ZrJ++K9I9a4u0HVdKbVFKrf73w+zAhPViLJp3F/px/vYDfulQHve/ehvTLtvPhuJ1zQ8gmyt0Ww1FaxjTRQ/MeGaX/C6ZWdivBu2qFGHClrO8/cdBwqTIW5JERBlP57pkceSbNhVR1tyc3Tza+FloORlyFDY/SJEmWLPY+mjToxAv5Ic/T7Pl1C2+bF6G2odHGOPtbabBS6+nXBCZXaDLUljSA9YNh4hgqDv8v7s42vNDu0qUL+TCV+tO0uaXXUzr6oN7HnmAyBpjN53h1I1QZvWoSu5sVqyFdOYv2DsZqvYFj6bmByjSjHiv+JVSpZRStbXW25/8wKioKatwpxLLDwUydft5ulQrQpebY+D0enjjB6jUPuWDccxiPBhW8U3YMho2ffa4rPO/lFL0rF2cub2qcSs0khaTdrLjjBThS8ye83eZ/k8Anau70dDDiplZIddhZX/IX8GouSTEExIa6hkPxLXWXnjsa8LGDl2+x4fLj1GjeC5GO81FHV0ML4+Can1tF5S9I7SeBj69Ydd44+rf8mz9nlql8rB6YB0K5cxCj1n7+e2fACnyFo+QiCj+t9gPd9dsjGxaNvEGlhhY0c9YVa3dTHC0ojyzyFASSvzuWuujT2/UWvsC7qZFJKxy7f5D+s05SAGXzMxy34z9gelGGeW6I2wdmrFWa9OfoM5w8J0Jy/sadd+f4uaalWUDatG4fAG+WneS/y0+QkSUFHl72uerjnMzNJKx7T3JmsmK0dmd4+DCDqP4mhmzuUSal1DiT+gyIUtyByKsF/4omr5zfImIimGp12Gy7PnJKK372lep52lMpeDVz+DV0eC/zCgXEfXwmd2yOTkw+S1vhjcqw/LDV+nw6x5uBEuRt3+tO3qd5YevMqhhKSq7WfHE9eV9Ro398m2Mnwkh4pBQ4j+glHpmzEAp1Rt5ctdmLBbNiCVHOHE9hCXVz5Nv9+jEyyvbUp13odl4OPuXMdc/4tmHuOzsFENeKc20rlU4dyuM5pN2cvBS0DP7ZTQ3giP4eMUxPIvmZNDLpRJv8PC+MasqRxFjYZXU+PMgUoWEEv+7QE+l1Dal1E+xH9uBPhhP8AobmPD3WdYfu8GvVa5S9sDIpJVXthWfnkaZgCv7YHZzeHA3zt1eK1+AFQNrkzWTPR2n7WXRgYy7JLPWmveWHiEyOoZx7T1xtE9k5rXWsGYohF4zxvUzW1G7R2RY8f40aa1vaq1rYUznvBj7MVprXVPq59vGuqPXGb/5LB+WuUGjkyOhsE/SyyvbSsV20HEB3D5l1PcJvhrnbmXyZ2fVwNrUKOHKB8uO8dkqf6IyYJG3uXsv8c/ZO4xsWo4SeZ0Tb3BoNpxYadzcL+JjenwibVNpYSaFj4+P9vX1tXUYNuV/NZh2U3fTKs81vg37BJW7OPRY+2KVNm3h4i6Y38GIu9tKcC0Z527RMRbG/HmaaTsCqFEiN790rmLd3PV04NytMJpO+IeaJV2Z1aNq4g9q3ToF0xqAWw3osty4uS4EoJQ6qLV+5kpAfkLSgFuhEfSd44tP5ut8Ez4alT2/8Que1pI+gHtt6LEGoh7AzCZwwz/O3Rzs7fj4jbKM6+DJocv3aT5xJyeupf8ib1ExFoYt8iNrJnvGtK2UeNKPemiUZMiUDVr/KklfWEV+SlK5iKgY3p57EJfwK8xy+Aa7TNmg60rInt/WoT2/QpWh50awc4Df34Ar++PdtXXlIiztX5MYi6btlN2sO5q+i7xN3HKWY1eD+bZNRfK5WDH//s+RcOuEkfTT8s+ESFGS+FMxrTUfLz/GtcsBrMg+BkdiyyvnKmbr0F5c3jJGWeesrjCnJZzfGu+ulYrkZPXg2pQr5MLA+Yf48c/T6bLI28FL95i09RxtvYvQpELBxBucWA2+M4znN0q/an6AIt2QxJ+K/bojgL8Pn2J97rFkjQ4xr7yyreQqZlz55y4B89vDyTXx7pove2bm961Ox6pFmbT1HP3m+hIa8exDYWnVg8hohi/2o2COLHzeolziDe5fgdWDjHdPr3xmfoAiXZHEn0ptPnGTSRsPszLHOHJHXjXKKxf2tnVYyS97fuMmdUEvWNwN/ObHu6uTgz3ftqnIly3Ls+30bVr/spuA22EpF6uJvlp3kstB4Yxt70n2zImsbBoTbczXt8RA2xlGyWUhkkASfyp0+kYo7y3cx7xsP1Ps0VlUSpVXtpUsuYy1gIvXg5UDYO/UeHdVStG1pjtze1cn6MEjWk7exbbTt1Iw2OS3+cRNFuy/TL96JahewjXxBtu/hyt7odm4eGdFCZEQSfypzN2wSPr9vpfx9hPwjD6KajUlZcsr24qTM7y1GMo2h40fwLbvn6ns+aSaJV1ZPag2RXJlpdfvB/h1+/k0WeTtTlgkHy4/StmCLgxvVCbxBhf+gR0/gOdbtqnAKtIFSfypyKNoC+/84cuw8J+prw/AGz+CZwdbh5VyHJyg3e/g1Rm2fQN/fgyW+B/eKpIrK8sG1OT1igX5dsMp3l3kl6aKvGmt+Wj5MUIeRjO+gxdODok8ff3grlHwzrWkUXpbiOdkzUIsIgVorfls1TGaBP5MK4d/oKGNyyvbir0DtJgETi6w9xejtk/zn43tcciayYFJnSpTrqALP/51mvO3w/i1qw+Fc6b+OoJLfAPZdOImo5qW5aUC2RPeWWtYNRDC78Jbi4x3SEI8J7niTyVm775IgcPj6enwpzE9r14qKK9sK3Z20ORbaPAR+P0BS3sYa8fGQynFwIal+K2bDxfvhNNy0k4OXEzdRd4u3w1n9Jrj1CzhSq/axRNvsH8anNkAjb6Agp7mByjSNUn8qcCOM7e5suEnhjosR3t1SV3llW1FKWjwITT5zpjmOb8DPHqQYJNXyuZn5cDauGR25K3pe5m/L3UWeYuxaIYt9sPOTvFje0/s7BL5v75+FP4aBaUbQ/X+KROkSNck8dvY+dth/DV/LJ84zCX6peaoFhMk6T+pxgBo+Qtc2A5zWsHDewnuXiqfMysG1qZ2qTx8vOIYI1cc41F06iryNnX7eQ5euseXLSskPiT16AEs7QVZckOrX+RnQyQLSfw2FBwexewZExnNVB661cfhzRmpu7yyrVTuDG/Ohut+8HszCEt4+maOLI7M6F6V/vVLMm/fZbr8to87YfEPFaUk/6vBjNt0hqaVCtLSq1DiDda/D3fPQdvpkC2P+QGKDEESv41Ex1iYMus3Rj38kfC8XmTpsiBtlFe2lXItjJuaQQEwszHcT3gYx95O8eHrHvzc0YujV+/TYuJO/K8Gp1CwcYuIiuHdRX64Omfi61YVEi/AdmypcY+j7v+MZxyESCaS+G3k98VLGHzrM8JdipO913KjuqJIWMmXodsqY2bLzCZw+0yiTVp6FWZp/1oAtJu6m9VHrpkdZby+33iKc7fC+PFNT3JmTeRp26AAWPMuFK1u3OQWIhlJ4reB9Zs30+7UMCIz5yFnv3Vps7yyrRStBj3WG4u3z2oC1/wSbVKhcA5WD65DpcI5GbLgMN9vPEVMChd523n2DrN2XaRHLXfqls6b8M7Rj2Bpb2N2U9vf4p3KKsTzMjXxK6WGKqX8lVLHlVLvxm77Uil1VCnlp5T6SyllxUBn+nH4yCF8/umNxSEzLv3WSynd51GgglHZ0zGbsZTjpd2JNsnj7MQffarTubobU7adp/fsAwQ/TJkib/fDHzFiyRFK5s3Gh697JN7g7y/h2iFoMRFyupkfoMhwTEv8SqkKQF+gGuAJNFNKlQZ+0FpX0lp7AWuBT82KIbW5ejmAvCs64GQXg2OP1Ti4uts6pLTLtaSR/LMXgLmt4eymRJtkcrDj69YV+bp1BXaevUPrybs4d8v8Im+frDrOnbBIxneoTGbHRG7en9sCuydAlZ5QrqXpsYmMycwr/rLAXq11uNY6GtgOtNZaP7mMUjYg7RVYeQ6h924S9XsLchFCWLtFZC9awdYhpX05CkPPDUap6gUdwX+ZVc06Vy/G/L41CH4YRevJu/j71E3TQlzld5U1R67x7qulqVgkkQXQw27Biv6Qt6zxAJsQJjEz8fsD9ZRSrkqprMAbQFEApdTXSqkrQGfiueJXSvVTSvkqpXxv375tYpjmi3kYwu0pLSgYc4OAV2dQuHxtW4eUfmTLA93XQJFqxrj4wd+talateG5WD65DsTxZ6T3bl8lbzyV7kbdr9x8yaqU/3m456V8/kSqaFguseBsiQ+DNWeCY+ktOiLTLtMSvtT4JfA9sAjYCR4Do2NdGaq2LAvOAQfG0n6a19tFa++TNm8jNsNQsKoIrU1rjFnmGXd4/UrFOM1tHlP5kzmEsUlO6EawZCrt+tqpZ4ZxZWPJ2LVp4FuKHP08zeMFhwh9FJ0tIFotmxJIjxFg04zp44WCfyK/anolw/m/jSj9f2WSJQYj4mHpzV2s9Q2vtrbWuBwQBZ5/aZT7Q1swYbCommmsz3sI9xJflxUbSsEV3W0eUfmXKCh3mQfk2sOlT2Dw6wbLO/8qSyZ7xHbz46HUP1h27Trspewi8F/7C4czafZHd5+/yabNyFHNNZKpu4EHY8gWUbWGM7QthMrNn9eSL/dcNaAMsiL3B+68WwCkzY7AZi4W7C/pS6MYWZuV4h9bdhyf+wI54MQ6ZjOmPVXrAzrGw7n8JlnX+l1KKt+uXZGaPqly5F06LSbvYG3D3ucM4fSOU7zee4tWy+elQtWjCO0eEwLJekL0gSLkOkULMnse/TCl1AlgDDNRa3wO+i53ieRR4DRhqcgwpT2vCVr+H67nl/ObYiVb9RuOY2Ft9kTzs7KHZeKg91FiIfMXbxpx/KzR8KR+rBtYmV1ZHuvy2j7l7LiZ53D8y2ng6N7uTA9+1rZjwH3utYe0wY/3ctr/J8xwixZj6ZIjW+pn1ArXW6XdoJ1bUlm9w9vuN2bop9XuPIVc2WRM1RSlllC/OnBO2jIbIUHjzd3DMnGjTEnmNIm/DFvrxyarjnLgewugWFcjkYN0f7vGbz3LyegjTu/mQxzmREhx+88B/qbH2glsNq44vRHKQy9BkZtnzC447x7A4pj5FO46ldAEXW4eUcdUdDk1/gjMbYV474w+AFVwyOzK9mw+DGpZiwf4rdJq+l1uhEYm2O3AxiKnbz9OxalEalUvkwbzbZ2D9e+Be14hTiBQkiT85HZ6H3Z8fsSGmKsGv/MDLZQvYOiJRtQ+0mW483Tu7BYRbt0CLnZ1iROOXmPyWNyeuhdBi4i6OBt6Pd//QiCiGLfKjaK6sjGpWLuGDR0UYpZYdMkObaVKRVaQ4SfzJ5eRa9KpB7IipyNby39KnvhULZ4uUUelN6Dgfbp2AWa9DiPWF2ppWKsiyAbWwt1O8OXUPKw4HxrnfF2tOcO3+Q8Z18MTZKZER1E2fws1j0GoKuGSoiiUilZDEnxwCtmFZ0pMjuiRT83/Ol+28ZQZPavNSE+i8FIIDjcqeQQFWNy1XyIXVg2pT2S0nwxYd4Zv1J/9T5G2j/w2WHAzknQalqFIsd8IHO70B9v8K1QcYMQlhA5L4X1SgL5YFnQjQBfnA6RN+7l4XJwd5654qFa9rPOUbGWok/5snrG7q6uzE3N7V6V6zGNN2BNBj1n6Cw6O4FRrBxyuOUaGwC0NeKZ3wQUKuwcp3oEAlaDT6Bb8ZIZ6fJP4XcfME+o+23LTkoFf0R4zt0YC82WUxlVStsLdR30fZGcM+gb5WN3W0t2N0ywp816YiewPu0nLyTgbPP8yDyGjGd/BKeOaPJQaW9TUWjW83SxbdETYlif95BV1Az21NSLQDb4Z/wMcdGlC+UCJFuETqkM/DqOyZJadxwzdge5Kad6zmxsJ+NQiLjGHfhSA+et2DUvmyJ9zon5/g0k544wfIU+r5YxciGUjifx4h12FOSyIjHtLuwfu0f7UOTSoUtHVUIilyuUOvPyFXMZj3Jpxal6TmVYrlZu3gOozv4EW3mu4J73xpD2z7Fiq+CV5vPXfIQiQXSfxJFR4Ec1sTHXaHDuEjeKliVQa/LFdwaVL2AtBjHRSoCIu6wpGFSWpeIEdmWlUujJ1dAjfyw4NgWR/IWQyajpWSDCJVkMSfFJGhMK8dlqAA+j4ajqWgNz+085QZPGlZ1tzGOr7udYzyDvumJd+xtYbVgyHsBrSbAZnlYT6ROkjit1ZUBCx8C33Nj4/sh3PcyYvp3XzIkklm8KR5Ts7w1mJ4qSlseA92/GBVZc9E+c6AU2vhlc+gcJUXP54QyUQSvzViomFZb7iwg0kuw1kR7sm0bj4UyJF47ReRRjhmhvZzwLMT/P0V/DXqxZL/zeOw8WMo+QrUjHPJCSFsxtQibemCxWK8XT+1ltWF3uWngMr83LESXkVz2joykdzsHaDlL+DkAnsmQUQwNP856SUVHoXDkp7GAjGtp4KdXF+J1EUSf0K0hj8/hiPzOVTyHYYcr8bAhiVp6VXY1pEJs9jZwevfG0l7xxjjvk6b6Uatf2v9+RHcOQ1dV4BzPvNiFeI5SeJPyPbvYd8UAl/qQbujtWlULj//a/SSraMSZlMKXh5pJP+/RhrJv8NcyJTISloAx1cY6/7WfhdKvmx2pEI8F3kPGp+9U2Dbt4R4tOeN000ok9+FcR28Ep66J9KXWoOgxSQI2Apz28DD+wnvf+8SrB4KhX3g5VEpEqIQz0MSf1z85sPGD4kq04w2VzriYO/A9G4+iVddFOmPd1ejxMLVgzC7GYTdjnu/mChjvj7amLpp75iiYQqRFJL4n3ZyLawahKV4A/qF9+fS/UimdqlC0dxZbR2ZsJXyreCthXDnHMxqYiyV+LRt30Lgfmg2zngqWIhUTBL/kwK2wdKeUNib71xGsfVcCF+1qkC14omU2hXpX6lXodtK44p/ZhPjj8C/ArbDP2Ohcheo2M5mIQphLUn8/wo8CAveAtdSLPMYx7R9t+hdpzgdqrrZOjKRWrjVgB5rISYSZjaG60fhwR1Y3g/ylIbXx9g6QiGsIokfjLrs89qCcz4O1pvJB+uvUL9MXj563cPWkYnUpmAl6LnRWDbx92awoBM8vAftZlo360eIVEASf9AFmNsa7J241mIhfZZfwc01KxM6VcbBXk6PiEOeUkZZZ+e8xrj+a18Zhd6ESCMy9jSV0BswtxXERPLgrTX0WHoTi4YZ3auSI4vMyhAJyFkUev0Fl3eDRzNbRyNEkmTcxB9bXpkHd4jpuoohWyI4f/sBs3tWo3geecsurJDNFco2t3UUQiRZxhzLiAwzFt+4ex46zucHf2e2nLrFZ83LUad0HltHJ4QQpsp4iT+2vDLXDsObs1h+vyRTt5+nc3U3utYoZuvohBDCdBkr8T8ur7wdWk7mUNZafLjsGDVK5ObzFuVlQRUhRIaQcRK/xQJrhhgLY7w+hmvureg35yAFcmRmSucqOMoMHiFEBpExsp3WRpVFv3nQ4GPCK/em7xxfIqJimNHdh1zZklByVwgh0riMkfi3j4G9v0CNd7DUfY8RS45w4noIEztVpnT+7LaOTgghUlT6T/x7p8K2b8CrM7z2NRO2nmP9sRt89LoHDT1kkQwhRMZjauJXSg1VSvkrpY4rpd6N3faDUuqUUuqoUmqFUiqnaQH4LYCNHxgP2DSfwDr/m4zffJa23kXoW7eEad0KIURqZlriV0pVAPoC1QBPoJlSqjSwCaigta4EnAE+MisGHt6DEg2g7Qz8bzzgf0v88HbLyTdtKsgMHiFEhmXmFX9ZYK/WOlxrHQ1sB1prrf+K/RpgL1DEtAhqvgNdlnPrIfSd40vurJn4tasPTg5JXDxbCCHSETMTvz9QTynlqpTKCrwBFH1qn17AhrgaK6X6KaV8lVK+t2/Hs+qRFSJioN/cg9wPj2J6dx/yZnd67mMJIUR6YFri11qfBL7HGNrZCBwB/r3SRyk1MvbrefG0n6a19tFa++TNm/d5Y+Dj5cfwu3KfcR08KV8ox3MdRwgh0hNTb+5qrWdorb211vWAIOAsgFKqO9AM6Ky11mb1/+uOAJYfvsrwRmVoUqGgWd0IIUSaYmp1TqVUPq31LaWUG9AGqKmUagJ8ANTXWoeb2X/hnFloV6UIg18uZWY3QgiRpphdlnmZUsoViAIGaq3vKaUmAU7AptiZNXu11v3N6Ly5ZyGaexYy49BCCJFmmZr4tdZ149gml99CCGFD6f/JXSGEEP8hiV8IITIYSfxCCJHBSOIXQogMRhK/EEJkMJL4hRAig5HEL4QQGYwysWJCslFK3QYuPWfzPMCdZAwnuUhcSSNxJY3ElTSpNS54sdiKaa2fKXaWJhL/i1BK+WqtfWwdx9MkrqSRuJJG4kqa1BoXmBObDPUIIUQGI4lfCCEymIyQ+KfZOoB4SFxJI3EljcSVNKk1LjAhtnQ/xi+EEOK/MsIVvxBCiCdI4hdCiAwmXSR+pdRMpdQtpZR/PK8rpdQEpdQ5pdRRpZR3KomrgVIqWCnlF/vxaQrFVVQptVUpdVIpdVwpNTSOfVL8nFkZV4qfM6VUZqXUfqXUkdi4Rsexjy3OlzVx2eRnLLZve6XUYaXU2jhes8nvpBVx2ep38qJS6lhsn75xvJ6850trneY/gHqAN+Afz+tvABsABdQA9qWSuBoAa21wvgoC3rGfZwfOAOVsfc6sjCvFz1nsOXCO/dwR2AfUSAXny5q4bPIzFtv3cGB+XP3b6nfSirhs9Tt5EciTwOvJer7SxRW/1noHxmLu8WkJzNGGvUBOpZTpq69bEZdNaK2va60PxX4eCpwECj+1W4qfMyvjSnGx5yAs9kvH2I+nZ0XY4nxZE5dNKKWKAE2B3+LZxSa/k1bElVol6/lKF4nfCoWBK098HUgqSCixasa+Vd+glCqf0p0rpdyByhhXi0+y6TlLIC6wwTmLHR7wA24Bm7TWqeJ8WREX2OZnbDzwPmCJ53Vb/XyNJ+G4wDbnSwN/KaUOKqX6xfF6sp6vjJL4VRzbUsOV0SGMWhqewERgZUp2rpRyBpYB72qtQ55+OY4mKXLOEonLJudMax2jtfYCigDVlFIVntrFJufLirhS/HwppZoBt7TWBxPaLY5tpp4vK+Oy1e9kba21N/A6MFApVe+p15P1fGWUxB8IFH3i6yLANRvF8pjWOuTft+pa6/WAo1IqT0r0rZRyxEiu87TWy+PYxSbnLLG4bHnOYvu8D2wDmjz1kk1/xuKLy0bnqzbQQil1EVgIvKyU+uOpfWxxvhKNy1Y/X1rra7H/3gJWANWe2iVZz1dGSfyrgW6xd8ZrAMFa6+u2DkopVUAppWI/r4bx/3E3BfpVwAzgpNZ6bDy7pfg5syYuW5wzpVRepVTO2M+zAK8Cp57azRbnK9G4bHG+tNYfaa2LaK3dgY7A31rrLk/tluLny5q4bPTzlU0plf3fz4HXgKdnAibr+XJ47mhTEaXUAoy78XmUUoHAZxg3utBaTwXWY9wVPweEAz1TSVztgAFKqWjgIdBRx97CN1ltoCtwLHZ8GOBjwO2J2GxxzqyJyxbnrCAwWyllj5EIFmut1yql+j8Rly3OlzVx2epn7Bmp4HxZE5ctzld+YEXs3xsHYL7WeqOZ50tKNgghRAaTUYZ6hBBCxJLEL4QQGYwkfiGEyGAk8QshRAYjiV8IITIYSfxCAEqpGPX/FRn9lFIfJuOx3VU8FVqFsIV0MY9fiGTwMLb0gRDpnlzxC5EAZdRJ/14Zde/3K6VKxW4vppTaooza6FuUUm6x2/MrpVbEFvk6opSqFXsoe6XUdGXUzf8r9klbIWxCEr8QhixPDfV0eOK1EK11NWASRnVHYj+fo7WuBMwDJsRunwBsjy3y5Q0cj91eGpistS4P3AfamvrdCJEAeXJXCEApFaa1do5j+0XgZa11QGwBuRtaa1el1B2goNY6Knb7da11HqXUbaCI1jryiWO4Y5RMLh379QeAo9b6qxT41oR4hlzxC5E4Hc/n8e0Tl8gnPo9B7q8JG5LEL0TiOjzx757Yz3djVHgE6AzsjP18CzAAHi+S4pJSQQphLbnqEMKQ5YmKoAAbtdb/Tul0Ukrtw7hQ6hS7bQgwUyn1HnCb/6+WOBSYppTqjXFlPwCweQlwIZ4kY/xCJCB2jN9Ha33H1rEIkVxkqEcIITIYueIXQogMRq74hRAig5HEL4QQGYwkfiGEyGAk8QshRAYjiV8IITKY/wP5WObd5ctWBgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "plt.plot(range(1, NUM_EPOCHS+1), train_acc_lst, label='Training accuracy')\n", + "plt.plot(range(1, NUM_EPOCHS+1), test_acc_lst, label='Test accuracy')\n", + "plt.legend(loc='upper left')\n", + "plt.ylabel('Cross entropy')\n", + "plt.xlabel('Epoch')\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "ccfebc8d", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Test accuracy: 96.35%\n" + ] + } + ], + "source": [ + "model.eval()\n", + "with torch.set_grad_enabled(False): # save memory during inference\n", + " test_acc, test_loss = compute_accuracy_and_loss(model, test_loader, DEVICE)\n", + " print(f'Test accuracy: {test_acc:.2f}%')" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "0886fd6a", + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAABpCAYAAAAnQqjlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAyZElEQVR4nO29eXQc2XWn+d1YMnLPRAKJlQAIkOBWG1n7IrtKUkmySi3JkiWN3dZ21Fo8Pm7bPdaM3D7usU63x3bPsTXHHltqy7Ita2tJI0uWrMVaSlUl1b6SxX0DAWLfErmvEfHmjwRJkESxSAIEEmB85+RhZiAz4r0f373x4r737hOlFB4eHh4e6xdtrQvg4eHh4bE8PEfu4eHhsc7xHLmHh4fHOsdz5B4eHh7rHM+Re3h4eKxzPEfu4eHhsc7ZEI5cRB4VkQ+v9m8bGU+Ti/E0WRpPl4tZb5o0lCMXkSEReXCty7EUIvJTEVEiYqzydRtKExH5HyKSX/SqiEhulcvQUJoAiEi/iHxXRHIiMisi//calKGhdBGRD4qIc0F7eWCVy9BomlwT+1lVp7ReEZFfx9MKAKXUbwC/ceaziHwecNesQA2AiPiAHwN/A/wvgANsW9NCNQ5PKaVes9aFaBSulf00VI/8lRCRpoXezoyIzC+833TB17aIyLMikhGRb4tIYtHv7xaRJ0UkLSL7rqRXICIx4I+A/2NFKrNCrKUmi84RAn4F+KdlVWaFWENNPgiMK6U+pZQqKKXKSqmXV6ZWy6cR2kqj0QiarKT9rAtHTr2c/wj0Aj1ACfjrC77zfuBDQCdgA38FICJdwPeAPwYSwMeBfxaR5GVe+0+AzwCTy6vCirOWmpzhV4AZ4GdXV4UVZ600uRsYEpEfLIRVHhWRm1agPivFWraVPQuaHBOR/yKrHJq8BBvLfpRSDfMChoAHL+N7u4H5RZ8fBf5s0eddQBXQgU8AX7zg9z8EPrDotx9+hevcDuylHlbZDCjAuJ41ueA3DwOf9NoJPwJqwJsBH/C/A4OA7zrXpR/oo+40bwIOAf/5etbkgt+smP2six65iARF5G9FZFhEstTvYHER0Rd9bWTR+2HABFqo33HfvfAIlBaRNPAaoONVrqkBnwZ+Ryllr2B1VoS10OSC63cD9wNfWGZVVow11KQEPK6U+oFSqgr8OdAM7Fx+rZbPWumilBpUSp1SSrlKqf3AfwXetULVWhYbzX4a5THn1fg9YDtwl1JqUkR2Ay8Bsug73Yve91DvIc1S/8/4olLqI1d4zSj1HvnXRATqd2KAURF5t1Lq51dci5VlLTRZzPuBJ5VSg8s4x0qzVpq8DNx3VSVeHda6rZxBXXDNtWStNVlR+2nEHrkpIv5FLwOIUO/1pBcGHP5oid+9V0R2iUiQ+p3/G0opB/gS8FYReZOI6AvnfGCJgY0LyVCPje1eeD20cPw24JnlVvIKaRRNFvN+4PPLq9ayaCRNvgTcLSIPLvTofpe6wR9egXpeKQ2ji4i8WUTaFt7vAP4L8O0VqueV0DCaLGJl7Wc141WXGc9SF7z+mLpDfRTIA8eAj7EoXr3wtz8FngWywL8CLYvOexfwGJCiPrjwPaDnCuNZm1m7GHlDaQLcAxSAiNdOzv72ncCJhfM+CtxwvetCPcQ0tdBWBqk7Q/N61uRa2Y8snNjDw8PDY53SiKEVDw8PD48rwHPkHh4eHuucZTlyEfklETkqIidE5PdXqlDrGU+TpfF0uRhPk4vxNLk6rjpGvjA6fwx4AzAKPAf8mlLq0MoVb33habI0ni4X42lyMZ4mV89yeuR3AidUfdJ/Ffgq8PaVKda6xdNkaTxdLsbT5GI8Ta6S5SwI6uL8lU+j1KfkvCI+sZSf0DIu2dj4CeFQQ0RmlFJJPE2Aui5lCuVFhy6pi6fJ0mx0XfyEqFBafOi61+QMOeZnF3zKkizHkS+1QuuiOI2IfBT4KICfIHfJ65dxycZmSo0yxyTjDA0vOnxdawJ1XfbzdP6Cw+fp4mkCXOdtZUqNcoQXLzx8XWtyhp+obwxf6u/LCa2Mcv4S1k3A+IVfUkp9Vil1u1LqdhNrGZdrfCwClM/vUVz3mkBdF+rJpM5wkS6eJl5bsQjgnp+a+7rX5HJZjiN/DhgQkT6pJ9b/VeA7K1Os9UmUJkrkAXyeJueI0gTg99rKOTxNLiZKEy4uniZXzlU7clXPCPhb1NM3Hga+rpQ6uFIFW49oorGd3VDfHcbTZAFNNIDTeG3lLJ4mF6OJhp8geJpcMcvKfqiU+j7w/RUqy4agRTpAcUApdftal6XByKy1JmL60MIh0AQ0HZQLto1yXNx8HlY/XcWaa9JoGJgopbxt8q6Q9ZLG1sPj6hFBDBPnrl0cfa8PM1ahtzVFrmIxfbKZwLjO5i8MYY9dFI718FgXeI7cY8MjhokWDpHt8vPm2/ZyZ2SQd4VPM2jDH4Z+mQORLlTAGzTzWL94jtxj47LQE+fmAUYfiJHbWeW/JZ6j28iiYZJy/ByZaMWY8CGV2lqX1sPjqlnfjlwE0esb9yh3Ib7pOtfkOmsQP/VYJqLriN+i1B4kf3OZG3onuMmXJSwmNRzmnDC1tJ9QRsBuuN38Ni6yaAmKZ1crwrp05GJZaOEQ1Zs3M/I6C70sBKcV/nmX2AuTqHwRZy61LKcupg/xW0hbC7WOOOZ0DvfkUP2GcS1uFh4rjjbQx/yeZmZ3Cx/e8zMiepmvZG9gX66bR/fuxJrW6X3Kxj+bw81k17q4GxtNR0wDPdlCdXMSzXbRs2WkWMY+PebZ1DJZn47c50MiYdL9Fre89hjTxQjDg60ExgzCw1F0EWR+HuW++rle+RomEgxgt0bJ9PuJ6oI5bNRnOSzjvCuKXLC4dqV7Nxee/1pe6xpQawmT2ikEt6X5X5te4mjN4q8mHuS54V66fiwEJ4roLx7FLZcvXj7osTwuaDtiGmiWhZuIkNvsR6+Cf87ETJvImI5aC0e+gexnXTnyM73kwut2MvKQor1nmne1Ps/LxR6mMhHs+TD6XA6Vmkc5V9cwxPQhPpPMv7uJyXuBpiotzbOMPZOk/3Q7KlfAmZlZ2YpdBcV33sX4LwhiC1oNQuNC4lAFayqPc+jY5TcUTcfo6kBFghR7Y9TCC0sLREhv0SgnXVy/Cz4XahpS1Ui8LLQ9MoHK5nBm565dJZeJ69Oww4qIvwLAUK2F50/3oJ0KEDk+jzaXxa56sfFlI4LRswk3HqaSDFKNG1SiGtWYUGlSVJJOPaGHpkBXiFFFm/XRdNBHyNII6Bpqlf8bNpr9rDNHbqCFgsztMvjSG/6adr1Im24Q0io8GhlgwgqhMlmcdObqr+EzkWCQ2d3Cf/ulr9NlzNNtZHlr6TdwmiPoSsHs2sfMp/dofOatnyPlhBmvNfGPx+8mRZyYX8N/VEddZsxXdB2nLU6l2c/cTSblxEK9NLjlnmN8uOPnbDfn2GQEmHBKjNhB3hf/MIkjcQxNoJEduanhBh3CvgqaCJN2DHcsQGRM4NQYdi631kVc/yyMUznJGMVNQTK9BqV2Ra29Snt7mre0H+cjiSfRFzqnx2sxHs7ewCOTA+Rm2jDKOgFt9fe32Wj2sy4cuVgWmmVRuWOAqTss5I4MnXqR47UmPj17Ez8Z2Y72oya6xmxUobisazk3byW9LYi+Nc9N1hjfye7mu6M3wr4o+sQwam0WjlxE4pDiY499gO19E3xk0895aPMhfvaWLYzNR+CBO+DVwj8LhqU0cBI19IBNb+s0yUA9j5OG4i0tL9Nvpoho9S9HRKPbKBKJlchvihGpOQ09EKx0wOfi0x1qymWknCA8rBEZt6Hm9cRfCbEs9PZWAFQ2j7Jt3EIRlIsWDiN+P7XtXVSafeQ2GVSjUOy1CSYLtEbztAVzRIwKUaPEVCXK7w2/g6Lto2ybjEw34T8UwEop2o5V8KVKqGp11eu40exnXThyzbKQaITZmy1ufOsRHmg6Sqdh8eNikm8fuQX//gCdn9+LWyy+qv6XRIT0tiAz99q8ZfMxdpom/+d8L4VHW0kesusLRhrEaTXtS+HLxzn61k66++bYnRjnD5JPUVEuOVfhLJmc8nx0FBpgCvhECIqOKfoF36rndXJxCWomQaAjmmWivQkrbZ2X9anRcA1Btxz8eo2yUkyUY8RO2QTGCiinUQY6Gg/x+ah1JkDA1HWkVEaVKygHtHAIFQ0zsydIYZNi061j/GLrCd4WfYndPgO9nnqAk7U8x2vNfHrstRx8aTN6WTByQuugS9P3DuCWyqhadXn2ugw2mv2sC0cusSh2V4Jiu+INiUPssMbR0Nib7yH0fIDYKRtVW970MT0eQ8JhMgPwCzcfJW4W+UK2i5eHuug+bBMcza9Zo1sKSWUIahrxvQneH/8Q29pmeFfb83SZ8+wyM2cfZS+FttBYdQRNBG2DbOEqloUW8JPv0HnjwBE2WfM8X2nnyFwrrRMF9Jk09lWOoWxEzqYu6Egye0czlYSQHbBBAzPdgVYFM98PLtQi4PgVge3zbI+nuTtxiq3WFM+V+nisYPFitoehbIKpVBQnZREY02kbdNFrLnrFxT9Vwq1UrnoMa8XqvMHsZ104cjcRId8TRHWXeG905Kxg++Y66fr+NKQyOPYyH5VbEtRao5i7svxjz6N8Ot3HP52+h9BBP6GnjqJK5YbpjQPYk1MwOUVHbSu5iWaO7e7n6/cL9yYGuSuxb4mewflc2OjchrpNLQ8t4EcSTRS64U87HuFw1cdXUnczPx6j/fgR7Kw31XAx4jMhmSBzQxP+fz/JL7QM84nWnxMUnaM1jbQbYKiaxEFji2+KuFai37AJaxazTomcEv54/M08P9aDOhAldsJl81AZ36kRVC6Ps0jvRrGgjWY/De3Iz8wgKXeEyfRrJJtyaGg8UvLzP2fvZvJIK/HsEG6xeNVOVo/HkFCImde0k94O93e9DMDRYjsjp1tomVWoUhnVoLMbJJMjdNrCNSOcoI9DzT18o2c3ulbXQ0Th0x0swyYZyGO7GmP5GLazdEPVNZcP9T3JA8HjJDSIaOce/kbtCmNOmPFsFF9eYZQbs1erejpI3RyntqmCiY4myzcyPZlEgn6U30KZ58xGK5RQ+SKqWMQtFJZ9nVVF09F8Jgz0MvJgnEKPw/vaj3FDYPRsmCCpV4hoNfxSo6YMHIQZJ8J3slsZL8c5kGonnQ/inAwTmBQiow7B8TLGTA6Vy+NWKmtdy0uyUeynoR25Fg4h4RCZzQblm4u8LnkagC/N3MNL37qRzuMOzszsZY8wX4QItLZQbY2QfSjPZ277CgNmBgiyd66L2H6T6FCloQ3UnpyCqWmiBy1iDweRSBi7NVbP8Ae4hoYdMqgGdY5064gNiSMVjOLSNybHb/BXv/1aojeUuNU/SmRRx+NgtZWfZG4gNxlhU8pFz1cbpoe1mNxAjOk3VLl3YBD9UnN5LxdNx93USqUtSDmhUwueO2doMkpgrB6uaeR2shSaz0SLx5i9Kcbb3/tzbg+d4sHALAHxcSa226UHAeg1HIpumRerEYaqLXzh6XsJDpm07LfZPJpHJk7izMxxZpGF00BPr5dio9hPQztyd+smsltCpHe53N07TLc/xcGqzeG5dqJDLoHJ8rml+VeDaNjNYYodFs3RGTr1HMdrMZ4sRRkbbqZn0MY3XWj8oINSuNUamlbfAtIQObsYQekahuVDBUz0ih9xwZrMIeXzZwooXcNORrCDOkGrStLI4pe6tjNOhZRr8PmJ+9i7v5/YEZ3gWA49lcduQIOtRjR6OlLsCE8CMFht5bHRrVhTxmXFZsX0we7tVJssygmDWkAodAm1iMKOOGCdaxHpeQNfOkZ4NEr8WAvGdBbnxKlrVrcVYaEnLlt6mbk9Qeomxc3B02w25rDEBMBFkXcrPFNpYtKO8URmgKlyhCPjbdSyPuL7TEJTDoGxAtpcFjdfWL+rMzeA/TSuIxdh6o4Ilddl+cDAi/x+yz6eqZh8O7uH1PEEO352CrdQxF3OMnxNyG/yk96q85rEBFtNi8/P38T3Tt1AyzMGgX97Dne9DIq5Tj3EVCxCav7iv4uGXxOUq3CWWJoqPh92740U2gz6Yim2mxkiWv3x8Wgtxr5SLwd/tpWdnxurxz3nM9gNs8T1fEpJ4eObH2GLOYOGznP5PuxnmkgMupf19KZFw5x6KEJtW4mHtu/j9vApNvtmiWtl4ppNcFEvf8YRUq6fPz39Fo4+vZmWfQGig6cb2qlpfgstGmHm9gSbP3KMX42f4qHg1EJPvE5F1Ri2df5i+I0MTycIPR4iNOkw8Pw47sxcfXKBclGOgw0NNX50Vaxz+2lcRw6ggWE4+LUalpjUlM5MNYJWEVSlenVzgTUd0XW0vm7s5jDzO3TKWyrsDE0A8NR0H86+GOGx6tWHbNaapYxKOa+aWqAWMajGhJhZxi+CvjAqX1YmOcePXhZUJocqlRrTUYmAaLgG+KWGgzDrlDiebyU67BIar8CFN+aFBS1aLIqEQxRuaKeYNHB25rm5c4K7IoPs8E0QFBufuEw6FkXXIqkXiGkOEU0R0Yrsjo9yvC9JYTpMUzSMKldwy+W10eFVkK52MrckSe+Eu+On2OUfwxSdkqoyYrsM2018afp1DGUTTO5vw0oJsaEa1lwFlcnWHd5GZh3aT2M7ckApwVEajnKZtJs5kWvBLAiqenWOVvOZSCTCxBvbyG5xefC+l/i15mfYamapKR9Tz7XT/+f7UNVaQ8Z/rxWi6xRbNQpdir7A7HmDNAXXYt4OYpTASacbtvclhon4LZRed+Q5189+J8jBkQ62/+Aobi53UZsRXUcLh7C3d5PvCdD+myf5UOtL3B0YJqkJQc1EQ+e0XWXOtfj/5u9kf7qTe5pPcVfoJAPmHFvMMO9vepo9e4b5ePHd0NWOls7hNuhGFenbWvF9aJJ3JQf5D/GD+MXAQGfUqfDl9D38cHQnwb+NEx4tsG3k2LnBfuXirNfOzTVmre2n4R35GVwUST3L1sgsR7u6qNyzA73mIlUXvVhFm55/ZYF0HRUNoXwGtUSAatQgs90h0p1lT/g03UaWWcdk0PZh5mTdDVotC03HaEuimqLku0HvKbDJd/6y4bJrkqv5EYeGdeIAenMTbmsT1bhLXC8ybjdxqNSFmzVRpdI5Jy6C+HxowSB0JLGbgkzfFqTYqXhHYpCbrTGCoqgBz1RMUk6Yn6TvZaiQ4MhYO27Kx8nmJN+L3cCDnUd5T/w5HEy6zTnMQA036EMvmg236lXz+5FwiFJC49amCXYExgmKj7xb4WhN48nizXz9yB7U6RDNo1n06XmcdGb9PpmuBg1iP+vGkQO8xl/g1vafcvsDp/ji1rvJVX3kyxbFsSgdP4+h1ZYWybGE+e0a1ZhLfCBFb2yeT3T+nFt8s0Q0HRMfX0jv4bGpAUITjWN4q4Hmt8jc20uuR+c1D77Mb7b9lG6jBovWnM3aUUYLcfRKY2tT3rWJ6dstEjtn2GU6PFNs5otH7yQ0bKAWGZD4fOitSeyuBCOvD1PaZPNHr/0Gd/iH6dQVmggnaj5G7AT/19GHmB2PkXzCIH68yLbpNKSziGGArvOd972GIw+18fqWI3wgepzmWIFyS4JAtfGcn9beSnlLktwWl/c2P0mbXkIjyL5qmE+efBunj7Wx429SMDuBm87UF001YgitgWgU+2loR27mFOmZEMNdCfJuBUsMYpqfXdYYr0meJGMHyNQC7DM6Sc02I69gO64Pqn1lQpEyd7WfZmtgmh3mLG16gKKqknGrvDDfw/Bwku7MddRwNR0J+Ml16+T6HW6KjNJt1AjKmUEahxE7zk9nt3N8pI22dGM7cjuoU0kougJFTNHJOX7KOYtYCVg0u0kLBqluTpLfZFHqq5LsyHCHf5g+Q2fKqZJyfXw9fSfHc0nmTiYITmpET5cxR+ZwU/PnPbFZ871MFKJk4kFM0dFFgcalU5iuEU4yRmqHBR1FknoJF9hfrfFI/iaGB1sJD+kwOdvQ4bOGooHsp3EduVI0701jZaP82L+TX295ii49T48R4EZT0Z949uy0wEKHYurGwCueShOXiNTwiUtEE0yEoGbh4nK46uNkrZXjT2xmy48rWKdmaLy+1DVABC0UhLYWYm+e4E+2/IBbfLPEtPrelTXl8IfDv8z+o90knjcY2JvHGJ9saG3KCR3VU2JbdBqAyWoUc8rEmlcsHqlSXW2ceruF0VPgb2/9KlvMedp0g4xb5fPpu3hhvoeRf+kjftJmx9EZSKVRxRJOtXrR9EVfTjE5GedUomVV63o1TN4d4U0fepI7Q4NsMix+WIzx54NvYuxgGzv/bg6Zz2Jnsp4TvxwazH4a15EDWrZIYNKHMRbky7P30OtPsd0/QUQr0awXSGhVeowgTZoiqdcoKodx27go4U1N6Ry169nc7vVPEV4Q28Vl0olxotyGb17wTWRR2fyq13PVEUGzLOjrotATZWfTfm7xzRLXzjUHB8VYNoo1YRKesDHGUw2/i45jQiBYIWrUZ4vYqr6AQ3MWHNNCbNyJ+aGjwtbWWXb45mnRfAzbNoN2Cw9PbGd8somu0w7B4SxMzpy3xPxCxAVsjZxtMeNUKFR8JEou0kihlYXdeWphuCd8gn5zFg2TGTvK+Gwc/6yGzGdR5TJ6shnRNNB1MHRUKIDSBanY4LpIvoiq1VDFEqpmo+za9ef4G9B+GtqRO6MT6FMzbJ3rYPDfdnCgzeLr7RrFdgV9RW7vOc3nen9IDYdhW3iyuI2/PvIA1er5y2drJZP4CxZK4P0f+zf+Y9Px+nHl8NPMTp6Y6CM67OAOjSw7+VbDs+DMpLuTw78ZpX/rJL/e8hQJzXd2FWRNORRdh/nhJrqftwkdS+FMTq15oqNXww4Km2IZ2sylDUYLh5FN7cxtC/CRm3/CXcGTJDSDUafGfxp8D8fHW+n+isGOU2mYnEWVSjiXucR8JBfny5k9ZIZjdBw8hcoXGsbB6Yk4JOJUEooBc4aI5lBUiiczWwk9GyA05aKa49jxAHM3BrADQjUGtYjLppsnCZpVBmeaqeQtovvbCcy4xA/n0KczuHOpjT8dcTENaj8N7chVrYqqVdFGJzBTaaLzScxcBKNgkJUgL2jdfKulg5rSOVVJ8lK6m9JQBL16fo/cXxJiJ2u4PmG2FgagqKrkXIfDmXZSkzE2Zx1Ug+eFWAnE50Pr7qTS00R77xxvbDtMl55HF+vsd4ZtxYjdjJnWsGaL9V7YOpi5oDQImxUsben1BeL3U2sNU0kI2/0TdBo5ZhzFkWorx8daMU77CZ6cwR08fdk9TbWw803N0ZmthdFLGipfaKi2JKEgtZYwbtAlojn4RXCVwtAcXAsqUaHQH6MS08j1geN3caM2/kiFN7YfJmHkedK/hYlijJO5LmphDb0SJhC1sEwDmc/g5guo2urnFV9tGtV+GtqRn8EtlZBqFSmWCI5YhPwWraEAKmjx+aa3gQviKrSqw/b09MUGKIIK+Ch3RnDQcHF5vNzEgVI3Ez/uZusTJXwnJho6/rtszky529LL4d+O09aT4pMD3+VG39x5j4RFt8aHD7+P6SNJeh630Q8M4pQbxyldCteEFqtAXC+goaGhQBacLeBuamX4l/xoW3Ls8k1RdA3+bPzNvDS6ic2f1/CfmsAdn7yicIFrCHrAxqc7VFwDcamvQWigG195SyvTt1u0bJ6hZWF+cw2HjyYfY/v7pyi7JnnHImaU2O6vL4ybs+sdnpBWwRSHf598Bg2Xco9JWZm8XOxhrBzniUdvJHasg+TTc/Ut0jYqDW4/68KRoxTKtuvGccFj3OJkkgpY6uFFLAvZuQU7oOHXajhKcbTcyQuZHkLjCt+JiYaP/y6LhZie1pak1Bmht3+aN7Yf5kbfHB16EBeFi0vGrTLjaEyONxE/qeGfyOCup+3QBAxx0F9hKZcbMKi12PTGc0REUQQmC1GqaQv/qRnswaEruFZ9Fanjh2CwgmXYdUfuQOPszl2nFtUpJRV94RyWGNg41BR06lXeFD6Ig1BTdUvSURSUSdoJUnAtTpebAeizZohoZXb5JoloLrutcVKOnyc295Evh4kNhjFHIvUNKDZaz3wd2M/6cOTLRIvHGH5znGJ/jT3BYVJulf/32dcROeijc38GZzZV74VtRDQdLRREbevh0IdDNHVl+JP+H7DdnCOpW/UdXZRLUdn8xex9PDOzmdZHTVp+Poo7t0TOiQbHVRoOci4/9CKf7vp0rFiZ9lAWXQRTXDaF04xF4ij9yjYF0GNRJBwmvUPxuZu+xmP5HTw+uwWjKMtL5HYNyHUZbLn1NG9oOQyAgU5QNEbdKj8u7GSwlOTZ6V5m02HMI0HMAgSmFUZF4cvWu0Y/bNGpRDUyd5fZ1DbPx/t/yB3WNJ+45UccGujk25E7SfTeSPNL86iXj6xldVeWdWI/G96Ri+lDQkGK3TbdPbMk9SwVBea0SeyUg57KYm+0HsQZROobVodDFJNBtm8f47XJY9zim6NVD57dlgugplwOZ9sZmW5i81gVe+j0GhZ8ZXAXZi/JmUkrSuE6GlVHp6oUGooWK48/UEUFLTS/H3dhKfqrhVYkHMZJxnGba9znr7G/XGA2H0JvwCiUa0DMKmGKTdYtU0NRVYqTtVb25bo5nk4ydTqBb0an9SUbX7qGbzQFpfLZXd6DmzpxmiNUEjFGyi0MbmpltzXN3YFT7LDG+V73DeRTEWKDgQ2yzxTryn42tCPXIhGqd2xjbrOPN96+l/c0P0u7XmTGCWDkBTNrQ4NuGLFcxKg3QDrbmHxNM9l++GTX09xqjRDRDFzOza12UZSV4thUEnPQj5nNbcg8M75jE3R+qZujtwzwzAc72WLO8M6m52k1c3zpba8nfDpOy5NTMDu/ZF6Ws4gw9eYe5u8v87ad+8i4Zb45vht+lKDlaKXhQiudj84zNjHAp27dxr/ceZqxTIzCeARrWqfpiItZcNk2W0Yr1dCm51G1Gm6xhHKcugYiuNOzaOkMvf/iYDcH+Kvym/jKljv437b+hAeDo9zZPcxTdj/FYxbhta7wCrDe7GdDO3KxfBQ6TApdwuvjh/hFf5VRG9JuEL0KetlBuY1ldCuGriOBANVEkFwf0FNitzVKv2me/Yq70Nzm3TJTjo9K2k80JUjZXn+OXASkvvjrlWLkTmqe0AuKWqiPw+VOIlqJHb55isEh/q6/gmv6iJ+IYtZsZHFulrPn1xDTQHw+cpvhXTe8tBCqg/FUjM4TVfwT+cbbVOHEaeJTYRxfH0fb2jGnfCROQHi8RvCF4XrPO5vF5RU2j1dqIcUrkM5ghEJEdt/CjNnEXF+YmOZnR2iKsdY4uVBXw+WYuSrWmf1sbEceCJDeplHrL9FuZCiqKl9I38lTc33ETziYx8dx05m1LuY1QW9pJnf7Jua3G7zxwRe4PXKKTv3i5nXKLvOmh3+H4HEfvXtrBMbnkbGpNSjx1aNFImihINWYYntwkqSx9MC1qlZx0xlie2f45qdfyxd6X8t73vw4fdYMv3nHo4zd3MS/9t2ENtHFlq9F0I4M4ZbKoFy0G7ZTbQsxfZtFocfh7j1HeEfsBb46fxf//cib8D0XJnhgCNWACddUpYLrOCSeniI81oReKqDPF5FCCSeduTi176ueUGEWFEZGJ+/40RDeHt3LLcFhPtH+IZot69zkhHXKerOfV3XkItINfAFop37D/qxS6i9FJAF8DdgMDAHvUUo11uiYaVBJ2rS3ZIlKhYpy2Zfp4uRkkr6pCs7U9FWdtqyKHOQ5KpQRhC766JEBaqrKfp4GuFFEfswaaqKC9V3kC90Ov936U3oNH7C4N+HiKMWkE6LpWZO2J1PIyBROJntViZLWUhOxfBAM4FiKpFH/v+bM9ENtYfqhaKBs3HIZ7fQYHT+okr+pg+fu7MWfrPFrsefRImDe4PBM62YqDzcT8PmQBWdU6QyT7Tap7Cnw7h37eG3kMFvNMkP5ZvJHm2gbcrCXSFt7KV2AARE5zjW2n7NO9cQp9IXdi5a7NEWzQavVV00D7DAt+owstbCqrwp1XvlJ91KaFMmxGpq8GqttP8vlcnrkNvB7SqkXRSQCvLBgkB8EHlZK/ZmI/D7w+8Anrl1RrxwVsOjqn+WtXfsJajZDto+9e/uJH9YwpiavujELwgA3E5UmbFXjWR4modqYYIgEraSYPgA8zBpoIoaBFgxS7kuQua/Mtq5pInKuJ1F0a+yrhhmqtfDF0bsZnmqm91gVGZ9Z2MT66kJNa6mJm8ujVWtYqU6ezA3gizrc4MuwPTjJj7YUyGfDJEIB3EJ9kZlbrcFsivABYf6zm/hmopt/uPF+lK4IDhv4MtCWykFLE+k7t1Ns1yjfkefmriHubRpkl3+M787v5o/nuph7sp3Nj5Uwx9JLT329hC5ATik10Kj2cxEi6C0tkIgxd4uia9cUtwfrN4bvFcO8UOwjMC31efSX6OVfShMdE1vV1kyTtbKf5fKqjlwpNQFMLLzPichhoAt4O/DAwtf+CXiUBmuIrs9gd/MQ94eOYKJIOWEip3Sa9xdh/upDKpYEsKgn6TLEJKgiVCgxwzi3cT8nOABrpIkYBhIMUGo2uKv/JHfGT+FfNLpeVi5HKh28XOhmaH8nwTENa2QSZy61rLjmWmqiKhWcSgUzD0P5ZiYDMSBDt2+OrW2zHG8KIX4/ciY3iOvgFgq4hQKRodPEm5oIvXYbjk8jfLqAXqgitosbDZC6STC2ZvmvN32PXwnPknHL5FzFf0+3M7m/ja6XbLTHXnrFTsGldAHOJK5uSPu5ENF1iIWxW8L4e3L8ctc++s0ULn5eKm7mkcltWGn1qvPIL6WJeS7963VlP8vlimLkIrIZ2AM8A7QtOHmUUhMi0rryxbs69HiMyq1bmd/u453hETYZJXKuxqQdIzjl4huawS2sTH6IkiqQI02MBFUqWBIAtfqaiOlD/BbOjf2M3B+iMFDlPyVfZIs5gyX1/+aachi2A/zlwddRHQvR8bgiOFGA2UtsynEVrJUmLS+XGTS28P/c1Ub/bf8TU2ze3/kUX73H4ZDbT2BKSO4ro5dttHwVWaizHfOT7dGpRWBmTxDXF0A1V7ECNe7veZmdoQn8Wo0nyiafGn0Lh8baCb4QpGd/Ff+p1GU/2V2oC1CDxrOfCxHTh9bfg90S5tRDAWrdFT627VnuD9Xni5+yy3zlyO2YL4bpPHlldnWhJoGFOS/Xs/1cDZftyEUkDPwz8LtKqaxcZr5lEfko8FEAP8GrKeMVI5EIczdY5PpdbrBGSeoWKdchZYfxz9nYo2Mrch1b2bzMU2xnN4aYXO5Q9bXQREwDLRQk3Rcgcv8UD7YO84bABEHtXFzPQTFmN8HBCC2DiujjgzhT08uOly5mLTXxHRhm03icoVAbz+/s59bgEL8SnqW/57t8KXQvj4wMkKrGMPMWgZR1dopGLaJTalfUmmz27BxiIDLDB5ueZKt5LkvmI6Uwe8s9HHi5l+YXNVpenMfdd/iytWuktnKliM+ksilGrtvHbQ8c4aPtj3GjL0eTFuCkXWPcjsCJEJ0/L2AOX34a6EbSpFHs52q5LEcuIiZ1J/5lpdQ3Fw5PiUjHwp2zA1hy5FAp9VngswBRSazKbUtFgmRurNHZO0dCqz8KfytzBz+b2YpRqDczMX2IriGBAOgabi5/RYmOXOXyMk/RTg+t0gWAD4uKKtXPv0qaiGHUUxD0dJLb1kRql/Dh7r0LG+qeeyQctSt8evYXeWKqn+b9DuGhfD1D3wqy1pqoYglRiuRLzfyT73V87aY0f3vzl6hi8mDsIO2+LP/6thspV00yFRO1kITF57PZnZyi1cpzZ+Qk7UaGZl1RVFW+le/lYLGLb+y7FWvER8cBRWQwhzaVWnqq3hXoYlMzV0OXy2ZhKbpEItgDndSiPrK9BtWoULihQjyR4peTL9JrZMm5MOOU+c/D7+TgWActBxXmWAo3e3lL0l9JkzMrcq9H+1kOlzNrRYC/Bw4rpT616E/fAT4A/NnCv9++JiW8Cpyonz07h3hjyyGSuiLjKh6dHmDodJLthRIKEL+F+EwkGkH5TDTbvuyUpUopDvE8ISL0yrazx5N0MsHwmY+roon4fEg4RLkrytwuA217jg/H9i/0JM6l8x2xo3zn6M3IUICBZ0ewR0ZXdK5rI2hSn+tcJPT4UbYeamL43Z08N9DPLv8oDwbSvCmY4feaD5z9vnbBGkT3PNfsI+dW+ebUrRwa7WDTdwyiT5/CXdhF/nJ7nZfSZZhjzQsfG8J+ZGHuNMkmZm8JUmoVInfOsCcxxcc7fsRWQ8MSA5cAx2pVhuwm9r/UR8uLQtNLs9jDI5d1nUu3lbPnuK7sZ7lcTo/8PuB9wH4R2btw7A+oO/Cvi8h/AE4D774mJVwG2qLRZqUENEVua5hQ4BZmdgWpxAW9AlpN0fakAZc5pzzDHJOcJkyMp9WPAdjKjfSy/exUOyDDKmiidvUzfVuEbB+03DLFva2nzutJ1JRDzrXZX96KeThIZFihrkH+6IbSpFyBTJaWA0n+8vsP4Sar3Ng3RtQs02Zl2eyf4+2Rg+RcnW9l95Bz/LjqXKiw5Ph4arKXTC6ItT9IfEYRHkyjCsUrzld/KV2GORZdmGp3TexHCwYRy7r4D81xau0x0AXX0HB8GnZIoxrSKHQKtZjC2p6hPVzgDW1H2OSbwy8OU47N9wtbGCwl+f7gDZRnA7Q9D7ETBUhd/uSBS2kyyiDXUpMLaRT7WS6XM2vlceCVAuKvX9niXDtsV0N0RXpAJ9cdIvLGSV7fNsgzM5uZTEcojkXxv3x554pLCw/yriX/dhv38xP1jQNKqVXRZn5XBP1ts7y78yh/kHwWU3QW9yRqymXc8bE3103bczUCgynca/BI2EiauOUylMsEHi6y7fkotW1dnLqnn1pYUWl1iHdkufWmUwzVknz+wN04hfPjs1LW6HoEkqfzaIOHcebnLzuMciGX0gXFMaXU7Vd56ldFQiGIRxautWgK3UALczeZOCY4AYUTULjJKs2JPP+x/wn6fdPc688RkPoMEhuHKUcxYgf5+xP3Mj8Wo+f7EDo6h5qYxs3lrihOfClNgipCVqUGrrbOV0qj2M9y2dArO88Q0YTXdxzlZDTJ/mgHlaqBT3c4mOng9PE2AmM6/ql1lK4V6tt3aUItCNvic/RYcxeFCgAmHfjq/F08M97LprkyZHJXvpJvnVJPe1zCnMqSOGRiB4RKTKcST/DeqY8hVY3YEf2i3c21KoROZ9Fns7il0hqVfvnk7+tjeo8Bolj0wEEt4RLqTGNpLobuEvJVaQ3m6Axk2OybRReXx8sx0k6QJ3Nbma2EOTjTTj7vJ7A/QMuMIjiSgUwOVV2nCec2mP1cF468SfPzhy0vQwvUehwybpWPDL6bw6PttD0uNO2dqe8evtYFvQJE1xG/RTUuPNB0lF3+sbNbTS3meK2Fbx3ejXEigHb6xFWvZl2PKNtG5XKQy+E/PghAeOGxWbS6Vq+UclYpF3s95wvRdEbfIHzvLX+OX1xCC/XVAB3BEoMaDhVVX6HoUJ/EU1P1ePCXZu/kWDrJ7HNt+GeF1heKdM3lUaMncAtF3DVYvbiSbDT72ZCOXM+WeenwZkZzcUJbK7QbaY5UOpm3Qzyd6mO6EGb+QAvBGSE8UkTSOdwG2prrctBbW3Bamyg3K/p90zRrJRb/d5aVTcpxOFDqRj/lJzyiYJ3VcUU545RV3QE1WILClUe5xA9ovLP1Y4QDFeKBEhoKEYVPd/DrNYq2j3Q5gKOEqq1jOzrVmkG1bCCTfsy8kDjuYmVszOkcks3jlitrsgR9pdlo9rMhHTlTs/R+O0auu4XPvON+WoM5XjrRizFrsunhGi1DaVrSJ6FUxi2V12U+8lpPkrmbg1j9We6wMmcXLZwh5zrsq7bzyMw2Op6wCYzkcAvrN0zgcYUoRevnXkC+YCI9nVQ6W3F0QQkU/Rp2QMMsuIRmykjFQSuUkUoJN5Wuhw5cF6XU2TCC4zhrvuhlJdlo9rMhHbkqV/BPFUEFGd/bxoTVSmRUw5dR+MfykMqgCgXUmSXb6xAjUyI0aZGaDzDuCKbUMBeN2L1Y6eQfx+/jxGA722dKaJk8zobvhnos5szm5Xoqg6VpKJF6EjGfgWsZaOUaeioPtoMqlVDVGm4+v6Ec9iux0exnQzpyt1BA9h3FLxpbHrcQkfrAl+te9g4wjY5z5CTBkwbhbbfx6L3b8EsNU85NjfvK+F1MfLuXrlEHOTqMfZ0YqMfFONMzMDN39rNocnZYz148cHcdtY+NZj8b0pEDZ3Mhb7iNYM/gOqiKQ3jU5W8O3Y+mKTTtXI8hPxKla8QhMFmuz6tu4EbocY1R6uzYAFwH4wOXwwaznw3ryK8X4t/ZT+KRcH1XlkWo2jTqzHZdG/Vm5uGxTDaK/XiOfJ1zJh2rh4fHlbNR7GfDbHjt4eHhcb3iOXIPDw+PdY6oVQzii8gMUABmV+2i154WLq5Pr1IqeTk/9jS5mAVNhl/hPOuVZWkCG7KteJoszRXrsqqOHEBEnr+WiYJWm5Woj6fJtT1PI+BpcjGeJktzNfXxQiseHh4e6xzPkXt4eHisc9bCkX92Da55LVmJ+niaXNvzNAKeJhfjabI0V1yfVY+Re3h4eHisLF5oxcPDw2Ods2qOXER+SUSOisgJEfn91bruSiEi3SLyiIgcFpGDIvI7C8c/KSJjIrJ34fXQFZ533eriaXIxniZLcy108TRZhFLqmr+ob4J3EugHfMA+YNdqXHsF69AB3LrwPgIcA3YBnwQ+fj3q4mniabJWunianP9arR75ncAJpdSgUqoKfBV4+ypde0VQSk0opV5ceJ8DDgNdyzztutbF0+RiPE2W5hro4mmyiNVy5F3AyKLPoyy/ca8ZIrIZ2AM8s3Dot0TkZRH5BxFpuoJTbRhdPE0uxtNkaVZIF0+TRayWI794V1NYl9NlRCQM/DPwu0qpLPAZYAuwG5gA/uJKTrfEsXWni6fJxXiaLM0K6uJpsojVcuSjQPeiz5uA8VW69oohIiZ1wb+slPomgFJqSinlKKVc4O+oP/JdLuteF0+Ti/E0WZoV1sXTZBGr5cifAwZEpE9EfMCvAt9ZpWuvCCIiwN8Dh5VSn1p0vGPR194BHLiC065rXTxNLsbTZGmugS6eJotYlY0llFK2iPwW8EPqo83/oJQ6uBrXXkHuA94H7BeRvQvH/gD4NRHZTf2xbgj42OWecAPo4mlyMZ4mS7OiunianI+3stPDw8NjneOt7PTw8PBY53iO3MPDw2Od4zlyDw8Pj3WO58g9PDw81jmeI/fw8PBY53iO3MPDw2Od4zlyDw8Pj3WO58g9PDw81jn/P8NclgWaBYmcAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "for batch_idx, (x, y) in enumerate(test_loader):\n", + " x = x.to(DEVICE)\n", + " y = y.to(DEVICE)\n", + "\n", + " x_numpy = x.numpy()\n", + " \n", + " fig, axes = plt.subplots(1, 5)\n", + " \n", + " for ax in axes:\n", + " index = random.randint(0, 64)\n", + " logits, probas = model(x[index].view(-1, 28*28))\n", + " _, predicted_labels = torch.max(probas, 1)\n", + " ax.imshow(x_numpy[index].reshape(28, 28))\n", + " ax.set_title(f'Label {predicted_labels[0]}')\n", + " \n", + " plt.show()\n", + " break" + ] + }, + { + "cell_type": "markdown", + "id": "9606aff2", + "metadata": {}, + "source": [ + "## 2-4. Discussion\n", + "1. Train Data에 대한 정확도와, Test Data에 대한 정확도가 왜 다를까요?\n", + "- 입력 해 주세요!\n", + "\n", + "2. 다른 사람들은 정확도가 99퍼가 넘는 모델도 만들던데, DNN의 한계가 있다면 어떤 점이 있을까요? (Hint: 우리는 28x28의 이미지를 768x1로 쫙 펴서 넣어 줬습니다.)\n", + "- 입력 해 주세요!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e72bd14c", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "31bb6f70", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.10" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}