ds8_clustering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 클러스터 분석\n",
    "\n",
    "* Last updated 20211117WED0630"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* C.2 문제의 이해\n",
    "* C.3 클러스터링 종류\n",
    "    * 분할 클러스터링\n",
    "    * 계층 클러스터링\n",
    "    * 밀집도 기반 클러스터링\n",
    "* C.4 k-Means\n",
    "    * C.4.1 알고리듬\n",
    "    * C.4.2 구현\n",
    "        * 데이터\n",
    "        * 단계 1: 설정\n",
    "        * 단계 2: 할당\n",
    "        * 단계 3: 갱신\n",
    "        * 단계 4: 종료\n",
    "* C.5 함수로 만들기\n",
    "* C.6 skleran\n",
    "* 문제: sklearn 클러스터링\n",
    "* C.7 Spark\n",
    "* 문제: Iris 클러스터링\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## C.2 문제의 이해\n",
    "\n",
    "클러스터링은 **Unsupervised learning** 기법이다. \n",
    "의사결정트리, kNN 등과 같이 답을 미리 정해놓고 사례를 훈련해서 모델링을 할 필요가 없다.\n",
    "클러스터링은 훈련없이, 데이터를 그룹화거나 클러스터로 군집화하게 된다.\n",
    "데이터를 특징에 따라 보다 가까운 클러스터로 할당해 묶게 된다.\n",
    "\n",
    "이런 기법이 유용하게 사용되는 분야는 다양하다.\n",
    "예를 들어, 소비자를 서로 다른 특징을 가진 세그먼트로 분류해 보다 유용한 마케팅 기법을 적용할 수 있게 한다.\n",
    "또는 병원에서 환자를 특징으로 분류해, 암, 파킨슨이나 정신병의 환자를 진단하거나 처치에 활용하기도 한다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## C.3 클러스터링 종류\n",
    "\n",
    "### C.3.1 분할 클러스터링\n",
    "\n",
    "분할 클러스터링 Partitioning clustering에서는 클러스터의 개수 **k를 미리 정하고**, 데이터를 k의 그룹으로 구분한다.\n",
    "k 그룹은 완전하게 분리되어, 어떤 데이터이든 여러 그룹에 속해 있을 수 없다.\n",
    "클러스터는 데이터의 평균으로 계산된 중심점을 가지게 된다.\n",
    "따라서 평균에서 멀리 떨어진 이상 값, outlier에 영향을 받는다는 단점이 있다.\n",
    "k-Means가 대표적인 분할 클러스터링 기법이다.\n",
    "\n",
    "### C.3.2 계층 클러스터링\n",
    "\n",
    "계층 클러스터링 hierarchical clustering\n",
    "미리 클러스터의 개수를 지정할 필요가 없다.\n",
    "**Dendrogram**이라고 하는 나무 가지모양으로 데이터를 구분한다.\n",
    "\n",
    "### C.3.2 밀집도 기반 클러스터링\n",
    "\n",
    "밀집도 기반 클러스터링 Density-based clustering은 특정 지역에서의 집적 정도에 따라 클러스터를 구분하는 기법이다. 클러스터의 개수를 미리 지정할 필요 없다. 다만 지역으로 분포된 가운데 데이터가 많이 집적된 곳을 그렇지 않은 곳을 hot spot으로 구분하여 클러스터로 인식하는 것이다.\n",
    "**DBSCAN**을 예로 들 수 있다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## C.4 k-Means\n",
    "\n",
    "### C.4.1 알고리즘\n",
    "\n",
    "* 군집의 갯수 k를 정하고, 중심점 $C_k$을 정한다. 이 중심점은 반복하면서 갱신하게 된다.\n",
    "* 반복:\n",
    "    * 할당 단계: 각 데이터 항목을 중심점과 얼마나 멀리 있는지 계산하여, 클러스터에 할당\n",
    "        * 모든 데이터  ${x_1,x_2,\\cdots,x_n}\\in D$에 대해 반복\n",
    "            * 각 데이터 $x_n$에서 각 클러스터 $C_k$ 거리를 계산 $dist(x_n,m_k)=\\|x_n-m_k\\|$ ($m_k$는 $C_k$ 평균을 말한다)\n",
    "            * $x_n$을 가장 가까운 클러스터 $C_k$에 할당\n",
    "    * 갱신 단계:\n",
    "        * 현재 클러스터에 할당된 데이터로의 평균을 계산하여 새로운 클러스터 $C_k$의 중심점$m_k$를 설정\n",
    "    * 종료 단계:\n",
    "        * 갱신이 더 이상 없으면 종료"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C.4.2 구현\n",
    "\n",
    "#### 데이터\n",
    "\n",
    "다음과 같은 데이터가 있다고 하자.\n",
    "\n",
    "구분 | 1 | 2 |...|...|...|...|...|...|n \n",
    "--|---|---|---|---|---|---|---|---|---\n",
    "x | 2 | 3 | 2 | 5 | 7 | 8 | 9 | 8 | 7\n",
    "y | 5 | 6 | 4 | 7 | 2 | 3 | 4 | 4 | 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "numpy 배열로 데이터를 구성하자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "x = np.array([[2,5],[3,6],[2,4],[5,7],[7,2],[8,3],[9,4],[8,4],[7,3]], dtype = float)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 단계 1: 설정\n",
    "\n",
    "클러스터는 군집의 가운데 있는 점으로, 몇 개로 할 것인지 결정해야 한다.\n",
    "클러스터의 적당한 개수는 통계적으로 결정할 수도 있다.\n",
    "또한 그 중심점을 어떻게 선정할 것인지 결정해야 하는데, 무작위로 하거나 또는 임의로 데이터를 선정할 수 있다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2개로 설정해 놓자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "k = 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "k=2에 따라, centeroid를 임의로 선택한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "centroids = np.array([[2,5],[7,2]], dtype = float)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 단계 2: 할당\n",
    "\n",
    "각 데이터 항목을 중심점과 얼마나 멀리 있는지 계산하여, 클러스터에 할당한다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 거리 계산\n",
    "\n",
    "```norm()``` 함수의 default는 L2라서 생략할 수 있고, Eculdean distance과 동일하다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def normDist(x1, x2):\n",
    "    #dist=np.sqrt(sum(np.power(x1 - x2, 2))) #36.406001444695193\n",
    "    dist = np.linalg.norm(x1 - x2) #36.406001444695193\n",
    "    return dist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 거리를 계산하여 가까운 중심점에 할당\n",
    "\n",
    "x는 $2 \\times 9$ 행렬이다. 첫째 데이터에 대해서 거리를 계산해서, 가까운 중심점에 할당해보자."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "x[0]으로부터 중심점까지의 거리를 구해보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distance 0.000 between [2. 5.] and [2. 5.]\n",
      "Distance 5.831 between [7. 2.] and [2. 5.]\n"
     ]
    }
   ],
   "source": [
    "for j in range(k):\n",
    "    # distance from centroids[j] to x[0]\n",
    "    distJ0 = normDist(centroids[j], x[0])\n",
    "    print (\"Distance {:.3f} between {} and {}\".format(distJ0, centroids[j], x[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그러면 어느 쪽으로 할당되어야 할까?\n",
    "당연히 거리가 짧은 중심점에 배정이 되어야 한다.\n",
    "swap 알고리즘으로 최소 길이를 고르자.\n",
    "swap에 필요한 np.inf는 양수의 무한 값으로 임의의 최대 값을 가진다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distance 0.000 between [2. 5.] and [2. 5.]\n",
      "Distance 5.831 between [7. 2.] and [2. 5.]\n",
      ".....Assigning [2. 5.] to 0th centroid by 0.000\n"
     ]
    }
   ],
   "source": [
    "minDist = np.inf\n",
    "for j in range(k):\n",
    "    # distance from centroids[j] to x[0]\n",
    "    distJ0 = normDist(centroids[j], x[0])\n",
    "    print (\"Distance {:.3f} between {} and {}\".format(distJ0, centroids[j], x[0]))\n",
    "    # pick the closest centroid\n",
    "    if distJ0 < minDist:\n",
    "        minDist = distJ0\n",
    "        minIndex = j\n",
    "print (\".....Assigning {} to {}th centroid by {:.3f}\".format(x[0], minIndex, minDist))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 모든 데이터에 대해 거리를 계산하고 가까운 중심점에 할당."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "clustering 처리결과를 저장하기 위한 저장소를 설정한다.\n",
    "clusterAssigned에는 index, 거리를 저장한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0. 0.]\n",
      " [0. 0.]\n",
      " [0. 0.]\n",
      " [0. 0.]\n",
      " [0. 0.]\n",
      " [0. 0.]\n",
      " [0. 0.]\n",
      " [0. 0.]\n",
      " [0. 0.]]\n"
     ]
    }
   ],
   "source": [
    "clusterAssigned = np.zeros(shape = (x.shape[0],2), dtype = float)\n",
    "print (clusterAssigned)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "각 데이터의 거리를 계산하면, 아래 표와 같이 4개는 C1, 아래 5개는 C2에 할당된다.\n",
    "\n",
    "x | y | C1 [2,5] | C2 [7,2]\n",
    "-----|-----|-----|-----\n",
    "2 | 5 | 0.000 | \n",
    "3 | 6 | 1.414 | \n",
    "2 | 4 | 1.000 | \n",
    "5 | 7 | 3.606 | \n",
    "7 | 2 |  | 0.000\n",
    "8 | 3 |  | 1.414\n",
    "9 | 4 |  | 2.828\n",
    "8 | 4 |  | 2.236\n",
    "7 | 3 |  | 1.000\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(9, 2)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".....Assigning [2. 5.] to 0th centroid by 0.000\n",
      ".....Assigning [3. 6.] to 0th centroid by 1.414\n",
      ".....Assigning [2. 4.] to 0th centroid by 1.000\n",
      ".....Assigning [5. 7.] to 0th centroid by 3.606\n",
      ".....Assigning [7. 2.] to 1th centroid by 0.000\n",
      ".....Assigning [8. 3.] to 1th centroid by 1.414\n",
      ".....Assigning [9. 4.] to 1th centroid by 2.828\n",
      ".....Assigning [8. 4.] to 1th centroid by 2.236\n",
      ".....Assigning [7. 3.] to 1th centroid by 1.000\n"
     ]
    }
   ],
   "source": [
    "row, c = x.shape\n",
    "for i in range(row):\n",
    "    minDist = np.inf   # arbitray max\n",
    "    minIndex = -1      # arbitray index\n",
    "    for j in range(k):\n",
    "        # distance from centroids[j] to x[i]\n",
    "        distJI = normDist(centroids[j], x[i])\n",
    "        # print (\"Distance {:.3f} between {} and {}\".format(distJI, centroids[j], x[i]))\n",
    "        # pick the closest centroid\n",
    "        if distJI < minDist:\n",
    "            minDist = distJI\n",
    "            minIndex = j\n",
    "    print (\".....Assigning {} to {}th centroid by {:.3f}\".format(x[i], minIndex, minDist))\n",
    "    clusterAssigned[i] = [minIndex, minDist]   #-----changed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "각 클러스터에 할당된 인덱스와 거리를 출력해보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".....Assigned clusters: \n",
      "[[0.         0.        ]\n",
      " [0.         1.41421356]\n",
      " [0.         1.        ]\n",
      " [0.         3.60555128]\n",
      " [1.         0.        ]\n",
      " [1.         1.41421356]\n",
      " [1.         2.82842712]\n",
      " [1.         2.23606798]\n",
      " [1.         1.        ]]\n"
     ]
    }
   ],
   "source": [
    "print (\".....Assigned clusters: \\n{}\".format(clusterAssigned))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 단계 3: 갱신\n",
    "\n",
    "현재 클러스터에 할당된 데이터로의 평균 $m_k$을 계산하여 새로운 클러스터 $C_k$의 중심점을 설정한다.\n",
    "새로운 중심점을 구하기 위해서는, 각 중심점에 할당된 데이터들을 모아서 평균을 계산해야 구할 수 있다.\n",
    "앞서 clusterAssigned에는 할당된 인덱스와 거리를 저장해 놓았다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 0., 0., 1., 1., 1., 1., 1.])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clusterAssigned[:,0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "첫 번째 중심점에 할당된 데이터를 출력해 보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[2., 5.],\n",
       "       [3., 6.],\n",
       "       [2., 4.],\n",
       "       [5., 7.]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[clusterAssigned[:,0]==0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "두 번째 중심점에 할당된 데이터를 출력해 보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[7., 2.],\n",
       "       [8., 3.],\n",
       "       [9., 4.],\n",
       "       [8., 4.],\n",
       "       [7., 3.]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[clusterAssigned[:,0]==1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "각 중심점에 할당된 평균으로 새로운 중심점을 계산한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "_centroids=np.zeros(shape=(k,x.shape[1]), dtype=float)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for cent in range(k):\n",
    "    _centroids[cent] = np.mean(x[clusterAssigned[:,0]==cent],axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Centroids updated: \n",
      " [[3.  5.5]\n",
      " [7.8 3.2]]\n"
     ]
    }
   ],
   "source": [
    "print (\"Centroids updated: \\n\", _centroids)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 단계 4: 종료\n",
    "\n",
    "반복을 종료할 것인지 확인한다.\n",
    "소수점을 비교할 경우, 자리수의 정밀도에 따라 동일한 값인지 확인하기 어려울 수 있다.\n",
    "np.equal() 함수를 사용해서 중심점이 갱신되었는지 확인하게 된다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "comparison = np.equal(centroids, _centroids)\n",
    "print(comparison.all())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "또는 np.isclose()는 tolerence를 어느 정도 허용하면서 동일한지 비교하므로 사용할 수 있다. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "comparison = np.isclose(centroids, _centroids)\n",
    "print(comparison.all())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그래프를 그리면 클러스터링을 직관적으로 보다 쉽게 이해할 수 있다.\n",
    "\n",
    "파란점으로 데이터 좌표를, 중심점은 빨간 색으로 그려보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f38b2e7ebe0>"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAOZ0lEQVR4nO3dX4xc5X3G8edZGwTjJqGCbUTj7E56g1pFLdARbQq1Wlyi0CJaVb0gmvQiN9tKKIKmVZTUV6m0F5WqKr2KNAKlSBlAxIlvohSBFFKFixDNGqf8Ma0a6t1Ak3iQSihsVQL59eKM6z+av/Y5c37H+/1Iq9nzzvHsY0v7+Ox73tnXESEAQF4rdQcAAExHUQNAchQ1ACRHUQNAchQ1ACS3v4oXve6666Ldblfx0gBwWdra2notIlbHPVdJUbfbbQ0GgypeGgAuS7a3Jz3H1AcAJEdRA0ByFDUAJEdRA0ByFDUAJDezqG3fYPvEOR9v2L5/GeGw9/T7UrstrawUj/1+3YmA+s1cnhcR/yrpRkmyvU/Sq5KOVZwLe1C/L21sSLu7xfH2dnEsSd1ufbmAui069XFY0vcjYuJ6P+BiHTlytqTP2N0txoG9bNGivkfSI+OesL1he2B7MBwOLz0Z9pydncXGgb1i7qK2faWkuyV9ZdzzEdGLiE5EdFZXx74LEphqbW2xcWCvWOSK+k5JxyPix1WFwd62uSm1WuePtVrFOLCXLVLUH9eEaQ+gDN2u1OtJ6+uSXTz2etxIBDzPnom2D0jakfRLEfGTWed3Op3glzIBwPxsb0VEZ9xzc/32vIh4S9K1paYCAMyFdyYCQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHJzFbXta2wftf2S7ZO2P1J1sMz6fandllZWisd+v+5EAC5n++c87x8kPR4Rf2L7SkmtCjOl1u9LGxvS7m5xvL1dHEtSt1tfLgCXr5lX1LbfJ+mQpAclKSLejojXqw6W1ZEjZ0v6jN3dYhwAqjDP1MeHJA0lfcn2s7YfsH3gwpNsb9ge2B4Mh8PSg2axs7PYOABcqnmKer+kmyV9MSJukvSWpM9eeFJE9CKiExGd1dXVkmPmsba22DgAXKp5ivoVSa9ExDOj46MqintP2tyUWhfM0LdaxTgAVGFmUUfEjyT9wPYNo6HDkl6sNFVi3a7U60nr65JdPPZ63EgEUJ15V318SlJ/tOLjZUmfrC5Sft0uxQxgeeYq6og4IalTcRYAwBi8MxEAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkqOoLwabJgJYonl/ex7OYNNEAEvGFfWi2DQRwJJR1Iti00QAS0ZRL4pNEwEsGUW9KDZNBLBkFPWi2DQRwJKx6uNisGkigCXiihoAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkpurqG2fsv2c7RO2B1UEYXcrABhvkV/K9LsR8VoVIdjdCgAmSzH1we5WADDZvEUdkp6wvWV7Y9wJtjdsD2wPhsPhQiHY3QoAJpu3qG+LiJsl3SnpXtuHLjwhInoR0YmIzurq6kIh2N0KACabq6gj4tXR42lJxyTdUmYIdrcCgMlmFrXtA7bfc+ZzSR+V9HyZIdjdCgAmm2fVx/slHbN95vyHI+LxsoOwuxUAjDezqCPiZUm/toQsAIAxUizPAwBMRlEDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkN3dR295n+1nbX68iSL8vtdvSykrx2O9X8VUANEWTOqHqrPsXOPc+SSclvbfcCMVfamND2t0tjre3i2NJ6nbL/moAsmtSJywjqyNi9kn2QUkPSdqU9OmIuGva+Z1OJwaDwdwh2u3iL3eh9XXp1Km5XwbAZaJJnVBWVttbEdEZ99y8Ux9fkPQZST+b8kU2bA9sD4bD4fzpJO3sLDYO4PLWpE5YRtaZRW37LkmnI2Jr2nkR0YuITkR0VldXFwqxtrbYOIDLW5M6YRlZ57mivlXS3bZPSXpU0u22v1xeBGlzU2q1zh9rtYpxAHtPkzphGVlnFnVEfC4iDkZEW9I9kr4ZEZ8oL0Ix4d7rFXM6dvHY6+W7aQBgOZrUCcvIOtfNxP8/2f4dSX9V9s1EANjrpt1MXGR5niLiW5K+VUImAMCceGciACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1gOXr96V2W1pZKR77/boTpba/7gAA9ph+X9rYkHZ3i+Pt7eJYkrrd+nIlxhU1gOU6cuRsSZ+xu1uMYyyKGsBy7ewsNg6KGsCSra0tNg6KGsCSbW5Krdb5Y61WMY6xKGoAy9XtSr2etL4u2cVjr8eNxClY9QFg+bpdinkBM6+obV9l+7u2v2f7BdufX0YwAOVq2tLlpuWt0jxX1P8r6faIeNP2FZKetv1PEfGdirMBKEnTli43LW/VZl5RR+HN0eEVo4+oNBWAUjVt6XLT8lZtrpuJtvfZPiHptKQnI+KZMeds2B7YHgyHw7JzArgETVu63LS8VZurqCPi3Yi4UdJBSbfY/vCYc3oR0YmIzurqatk5AVyCpi1dblreqi20PC8iXpf0lKSPVRMHQBWatnS5aXmrNs+qj1Xb14w+v1rSHZJeqjoYgPI0bely0/JWzRHT7wva/lVJD0nap6LYH4uIv5n2ZzqdTgwGg9JCAsDlzvZWRHTGPTdzeV5E/Iukm0pPBQCYC28hB4DkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkZha17Q/afsr2i7ZfsH3fMoIBKFe/L7Xb0spK8djv150I89o/xznvSPrLiDhu+z2Stmw/GREvVpwNQEn6fWljQ9rdLY63t4tjSep268uF+cy8oo6IH0bE8dHn/y3ppKQPVB0MQHmOHDlb0mfs7hbjyG+hOWrbbUk3SXpmzHMbtge2B8PhsJx0AEqxs7PYOHKZu6ht/5ykr0q6PyLeuPD5iOhFRCciOqurq2VmBHCJ1tYWG0cucxW17StUlHQ/Ir5WbSQAZdvclFqt88darWIc+c2z6sOSHpR0MiL+vvpIAMrW7Uq9nrS+LtnFY6/HjcSmcERMP8G+TdK3JT0n6Wej4b+OiG9M+jOdTicGg0FpIQHgcmd7KyI6456buTwvIp6W5NJTAQDmwjsTASA5ihoAkqOoASA5ihoAkpu56uOiXtQeStq+yD9+naTXSoxTpSZllZqVt0lZpWblbVJWqVl5LyXrekSMfbdgJUV9KWwPJi1RyaZJWaVm5W1SVqlZeZuUVWpW3qqyMvUBAMlR1ACQXMai7tUdYAFNyio1K2+TskrNytukrFKz8laSNd0cNQDgfBmvqAEA56CoASC5FEXdtA10bV9l+7u2vzfK+/m6M81ie5/tZ21/ve4ss9g+Zfs52ydsp/41jLavsX3U9ku2T9r+SN2ZJrF9w+jf9MzHG7bvrzvXJLb/YvT99bztR2xfVXemaWzfN8r6Qtn/rinmqG1fL+n6czfQlfRHWTfQHf2O7gMR8eZoU4WnJd0XEd+pOdpEtj8tqSPpvRFxV915prF9SlInItK/ycH2Q5K+HREP2L5SUisiXq871yy290l6VdJvRMTFvjmtMrY/oOL76lci4n9sPybpGxHxj/UmG8/2hyU9KukWSW9LelzSn0fEv5fx+imuqJu2gW4U3hwdXjH6qP9/vAlsH5T0B5IeqDvL5cT2+yQdUrGxhiLi7SaU9MhhSd/PWNLn2C/patv7JbUk/WfNeab5ZUnPRMRuRLwj6Z8l/XFZL56iqM81bQPdTEZTCScknZb0ZERkzvsFSZ/R2Y0fsgtJT9jesr1Rd5gpPiRpKOlLo2mlB2wfqDvUnO6R9EjdISaJiFcl/Z2kHUk/lPSTiHii3lRTPS/pt21fa7sl6fclfbCsF09V1LM20M0kIt6NiBslHZR0y+hHn3Rs3yXpdERs1Z1lAbdFxM2S7pR0r+1DdQeaYL+kmyV9MSJukvSWpM/WG2m20RTN3ZK+UneWSWz/vKQ/VPGf4S9KOmD7E/WmmiwiTkr6W0lPqJj2OCHp3bJeP01RN3UD3dGPuk9J+ljdWSa4VdLdo3nfRyXdbvvL9UaabnQ1pYg4LemYinm/jF6R9Mo5P00dVVHc2d0p6XhE/LjuIFP8nqT/iIhhRPxU0tck/VbNmaaKiAcj4tcj4pCk/5L0b2W9doqibtoGurZXbV8z+vxqSXdIeqneVONFxOci4mBEtFX8uPvNiEh7ZWL7wOiGskbTCB9V8WNlOhHxI0k/sH3DaOiwpJQ3wC/wcSWe9hjZkfSbtlujfjis4t5VWrZ/YfS4pmJ++uGyXnvmnolLcqukP5X03GjeV5qxgW7Nrpf00OjO+YqkxyIi/bK3hni/pGPF96b2S3o4Ih6vN9JUn5LUH00nvCzpkzXnmWr0n98dkv6s7izTRMQzto9KOi7pHUnPKv9byb9q+1pJP5V0b5k3llMszwMATJZi6gMAMBlFDQDJUdQAkBxFDQDJUdQAkBxFDQDJUdQAkNz/AZxbQl9weRVDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.scatter(x[:,0],x[:,1],color='blue')\n",
    "plt.scatter(centroids[:,0],centroids[:,1],color='red')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## C.5 함수로 만들기\n",
    "\n",
    "### 거리 계산 함수\n",
    "\n",
    "두 점 사이의 거리를 계산하는 함수이다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def normDist(x1, x2):\n",
    "    #dist=np.sqrt(sum(np.power(x1 - x2, 2))) #36.406001444695193\n",
    "    dist = np.linalg.norm(x1 - x2) #36.406001444695193\n",
    "    return dist"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 클라스터링 함수\n",
    "\n",
    "알고리즘을 정리하면 다음과 같다.\n",
    "* ??1회 반복 후에 여기에 centroids를 _centroids로 갱신해주는 코드가 있어야\n",
    "\n",
    "```python\n",
    "while 무한반복 시작\n",
    "    for 행 갯수만큼 반복\n",
    "        최소길이 초기화\n",
    "        최소인덱스 초기화\n",
    "        for 중심점 개수만큼 반복\n",
    "            중심값과 각 데이터 거리 계산\n",
    "            if 최소 거리값이면, 그 클러스터에 할당\n",
    "                최소값 갱신\n",
    "                최소인덱스 갱신\n",
    "        if 클러스터가 변경되지 않았으면:\n",
    "            무한반복 종료\n",
    "        else:\n",
    "            old 중심점을 new 중심점으로 갱신\n",
    "            무한반복 계속\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def cluster(x, k, centroids):\n",
    "    row, col = x.shape\n",
    "    clusterAssigned = np.zeros(shape=(row, k), dtype=float)\n",
    "    # print (clusterAssigned)\n",
    "    clusterUpdated=True\n",
    "    iter = 1\n",
    "    while(clusterUpdated):\n",
    "        print (\"\\n\\n====== Iteration {} ======\".format(iter))\n",
    "        # STEP 1: assign to closer cluster\n",
    "        for i in range(row):\n",
    "            minDist = np.inf   # set arbitray max to minDist\n",
    "            minIndex = -1      # set arbitray index to minIndex\n",
    "            # compute distance and assign to closer cluster\n",
    "            for j in range(k):\n",
    "                # distance from centroids[j] to x[0]\n",
    "                distJI = normDist(centroids[j], x[i])\n",
    "                # print (\"Distance {:.3f} between {} and {}\".format(distJI, centroids[j], x[i]))\n",
    "                # pick the closest centroid\n",
    "                if distJI < minDist:\n",
    "                    minDist = distJI\n",
    "                    minIndex = j\n",
    "            print (\".....Assigning {} to {}th centroid by {:.3f}\".format(x[i], minIndex, minDist))\n",
    "            clusterAssigned[i] = [minIndex, minDist]   #-----changed\n",
    "        print (\"\\n.....Assigned clusters: \\n {}\".format(clusterAssigned))\n",
    "        # STEP 2: update centroids\n",
    "        _centroids=np.zeros(shape = (k, x.shape[1]), dtype=float)\n",
    "        for cent in range(k):\n",
    "            # read data by index and average them\n",
    "            _centroids[cent] = np.mean(x[clusterAssigned[:,0]==cent],axis=0)\n",
    "        # print (\"Centroids updated: \\n\", _centroids)\n",
    "        # STEP 3: see if centroids is updated. STOP if NOT updated\n",
    "        comparison = np.equal(centroids, _centroids) # np.isclose() in case some tolerence is required\n",
    "        if(comparison.all() == True):\n",
    "            print (\"\\n.....NOT Updating Centroids\\n {} => \\n{}\".format(centroids, _centroids))\n",
    "            clusterUpdated = False\n",
    "            return centroids, clusterAssigned\n",
    "        else:\n",
    "            print (\"\\n.....Updating Centroids\\n {} => \\n{}\".format(centroids, _centroids))\n",
    "            centroids = _centroids\n",
    "            iter += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 그래프\n",
    "\n",
    "데이터의 파란 색으로 산포하고 빨간색으로 중심점을 찍는 그래프를 작성하는 함수이다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def draw(x, centroids):\n",
    "    import matplotlib.pyplot as plt\n",
    "    #plt.subplot(5,2,iter)\n",
    "    plt.scatter(x[:,0], x[:,1], color='blue')\n",
    "    plt.scatter(centroids[:,0], centroids[:,1], color='red')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "아래 데이터로 실행해보자.\n",
    "데이터의 초기 중심점으로 결과가 다르게 나올 수 있다는 점에 유의하자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "====== Iteration 1 ======\n",
      ".....Assigning [2. 5.] to 0th centroid by 0.000\n",
      ".....Assigning [3. 6.] to 0th centroid by 1.414\n",
      ".....Assigning [2. 4.] to 0th centroid by 1.000\n",
      ".....Assigning [5. 7.] to 0th centroid by 3.606\n",
      ".....Assigning [7. 2.] to 1th centroid by 0.000\n",
      ".....Assigning [8. 3.] to 1th centroid by 1.414\n",
      ".....Assigning [9. 4.] to 1th centroid by 2.828\n",
      ".....Assigning [8. 4.] to 1th centroid by 2.236\n",
      ".....Assigning [7. 3.] to 1th centroid by 1.000\n",
      "\n",
      ".....Assigned clusters: \n",
      " [[0.         0.        ]\n",
      " [0.         1.41421356]\n",
      " [0.         1.        ]\n",
      " [0.         3.60555128]\n",
      " [1.         0.        ]\n",
      " [1.         1.41421356]\n",
      " [1.         2.82842712]\n",
      " [1.         2.23606798]\n",
      " [1.         1.        ]]\n",
      "\n",
      ".....Updating Centroids\n",
      " [[2. 5.]\n",
      " [7. 2.]] => \n",
      "[[3.  5.5]\n",
      " [7.8 3.2]]\n",
      "\n",
      "\n",
      "====== Iteration 2 ======\n",
      ".....Assigning [2. 5.] to 0th centroid by 1.118\n",
      ".....Assigning [3. 6.] to 0th centroid by 0.500\n",
      ".....Assigning [2. 4.] to 0th centroid by 1.803\n",
      ".....Assigning [5. 7.] to 0th centroid by 2.500\n",
      ".....Assigning [7. 2.] to 1th centroid by 1.442\n",
      ".....Assigning [8. 3.] to 1th centroid by 0.283\n",
      ".....Assigning [9. 4.] to 1th centroid by 1.442\n",
      ".....Assigning [8. 4.] to 1th centroid by 0.825\n",
      ".....Assigning [7. 3.] to 1th centroid by 0.825\n",
      "\n",
      ".....Assigned clusters: \n",
      " [[0.         1.11803399]\n",
      " [0.         0.5       ]\n",
      " [0.         1.80277564]\n",
      " [0.         2.5       ]\n",
      " [1.         1.44222051]\n",
      " [1.         0.28284271]\n",
      " [1.         1.44222051]\n",
      " [1.         0.82462113]\n",
      " [1.         0.82462113]]\n",
      "\n",
      ".....NOT Updating Centroids\n",
      " [[3.  5.5]\n",
      " [7.8 3.2]] => \n",
      "[[3.  5.5]\n",
      " [7.8 3.2]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAOZ0lEQVR4nO3dX4xc5X3G8edZGwTjJqGCbUTj7E56g1pFLdARbQq1Wlyi0CJaVb0gmvQiN9tKKIKmVZTUV6m0F5WqKr2KNAKlSBlAxIlvohSBFFKFixDNGqf8Ma0a6t1Ak3iQSihsVQL59eKM6z+av/Y5c37H+/1Iq9nzzvHsY0v7+Ox73tnXESEAQF4rdQcAAExHUQNAchQ1ACRHUQNAchQ1ACS3v4oXve6666Ldblfx0gBwWdra2notIlbHPVdJUbfbbQ0GgypeGgAuS7a3Jz3H1AcAJEdRA0ByFDUAJEdRA0ByFDUAJDezqG3fYPvEOR9v2L5/GeGw9/T7UrstrawUj/1+3YmA+s1cnhcR/yrpRkmyvU/Sq5KOVZwLe1C/L21sSLu7xfH2dnEsSd1ufbmAui069XFY0vcjYuJ6P+BiHTlytqTP2N0txoG9bNGivkfSI+OesL1he2B7MBwOLz0Z9pydncXGgb1i7qK2faWkuyV9ZdzzEdGLiE5EdFZXx74LEphqbW2xcWCvWOSK+k5JxyPix1WFwd62uSm1WuePtVrFOLCXLVLUH9eEaQ+gDN2u1OtJ6+uSXTz2etxIBDzPnom2D0jakfRLEfGTWed3Op3glzIBwPxsb0VEZ9xzc/32vIh4S9K1paYCAMyFdyYCQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHJzFbXta2wftf2S7ZO2P1J1sMz6fandllZWisd+v+5EAC5n++c87x8kPR4Rf2L7SkmtCjOl1u9LGxvS7m5xvL1dHEtSt1tfLgCXr5lX1LbfJ+mQpAclKSLejojXqw6W1ZEjZ0v6jN3dYhwAqjDP1MeHJA0lfcn2s7YfsH3gwpNsb9ge2B4Mh8PSg2axs7PYOABcqnmKer+kmyV9MSJukvSWpM9eeFJE9CKiExGd1dXVkmPmsba22DgAXKp5ivoVSa9ExDOj46MqintP2tyUWhfM0LdaxTgAVGFmUUfEjyT9wPYNo6HDkl6sNFVi3a7U60nr65JdPPZ63EgEUJ15V318SlJ/tOLjZUmfrC5Sft0uxQxgeeYq6og4IalTcRYAwBi8MxEAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkqOoLwabJgJYonl/ex7OYNNEAEvGFfWi2DQRwJJR1Iti00QAS0ZRL4pNEwEsGUW9KDZNBLBkFPWi2DQRwJKx6uNisGkigCXiihoAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkqOoASA5ihoAkpurqG2fsv2c7RO2B1UEYXcrABhvkV/K9LsR8VoVIdjdCgAmSzH1we5WADDZvEUdkp6wvWV7Y9wJtjdsD2wPhsPhQiHY3QoAJpu3qG+LiJsl3SnpXtuHLjwhInoR0YmIzurq6kIh2N0KACabq6gj4tXR42lJxyTdUmYIdrcCgMlmFrXtA7bfc+ZzSR+V9HyZIdjdCgAmm2fVx/slHbN95vyHI+LxsoOwuxUAjDezqCPiZUm/toQsAIAxUizPAwBMRlEDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkR1EDQHIUNQAkN3dR295n+1nbX68iSL8vtdvSykrx2O9X8VUANEWTOqHqrPsXOPc+SSclvbfcCMVfamND2t0tjre3i2NJ6nbL/moAsmtSJywjqyNi9kn2QUkPSdqU9OmIuGva+Z1OJwaDwdwh2u3iL3eh9XXp1Km5XwbAZaJJnVBWVttbEdEZ99y8Ux9fkPQZST+b8kU2bA9sD4bD4fzpJO3sLDYO4PLWpE5YRtaZRW37LkmnI2Jr2nkR0YuITkR0VldXFwqxtrbYOIDLW5M6YRlZ57mivlXS3bZPSXpU0u22v1xeBGlzU2q1zh9rtYpxAHtPkzphGVlnFnVEfC4iDkZEW9I9kr4ZEZ8oL0Ix4d7rFXM6dvHY6+W7aQBgOZrUCcvIOtfNxP8/2f4dSX9V9s1EANjrpt1MXGR5niLiW5K+VUImAMCceGciACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1ACRHUQNAchQ1gOXr96V2W1pZKR77/boTpba/7gAA9ph+X9rYkHZ3i+Pt7eJYkrrd+nIlxhU1gOU6cuRsSZ+xu1uMYyyKGsBy7ewsNg6KGsCSra0tNg6KGsCSbW5Krdb5Y61WMY6xKGoAy9XtSr2etL4u2cVjr8eNxClY9QFg+bpdinkBM6+obV9l+7u2v2f7BdufX0YwAOVq2tLlpuWt0jxX1P8r6faIeNP2FZKetv1PEfGdirMBKEnTli43LW/VZl5RR+HN0eEVo4+oNBWAUjVt6XLT8lZtrpuJtvfZPiHptKQnI+KZMeds2B7YHgyHw7JzArgETVu63LS8VZurqCPi3Yi4UdJBSbfY/vCYc3oR0YmIzurqatk5AVyCpi1dblreqi20PC8iXpf0lKSPVRMHQBWatnS5aXmrNs+qj1Xb14w+v1rSHZJeqjoYgPI0bely0/JWzRHT7wva/lVJD0nap6LYH4uIv5n2ZzqdTgwGg9JCAsDlzvZWRHTGPTdzeV5E/Iukm0pPBQCYC28hB4DkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkKGoASI6iBoDkZha17Q/afsr2i7ZfsH3fMoIBKFe/L7Xb0spK8djv150I89o/xznvSPrLiDhu+z2Stmw/GREvVpwNQEn6fWljQ9rdLY63t4tjSep268uF+cy8oo6IH0bE8dHn/y3ppKQPVB0MQHmOHDlb0mfs7hbjyG+hOWrbbUk3SXpmzHMbtge2B8PhsJx0AEqxs7PYOHKZu6ht/5ykr0q6PyLeuPD5iOhFRCciOqurq2VmBHCJ1tYWG0cucxW17StUlHQ/Ir5WbSQAZdvclFqt88darWIc+c2z6sOSHpR0MiL+vvpIAMrW7Uq9nrS+LtnFY6/HjcSmcERMP8G+TdK3JT0n6Wej4b+OiG9M+jOdTicGg0FpIQHgcmd7KyI6456buTwvIp6W5NJTAQDmwjsTASA5ihoAkqOoASA5ihoAkpu56uOiXtQeStq+yD9+naTXSoxTpSZllZqVt0lZpWblbVJWqVl5LyXrekSMfbdgJUV9KWwPJi1RyaZJWaVm5W1SVqlZeZuUVWpW3qqyMvUBAMlR1ACQXMai7tUdYAFNyio1K2+TskrNytukrFKz8laSNd0cNQDgfBmvqAEA56CoASC5FEXdtA10bV9l+7u2vzfK+/m6M81ie5/tZ21/ve4ss9g+Zfs52ydsp/41jLavsX3U9ku2T9r+SN2ZJrF9w+jf9MzHG7bvrzvXJLb/YvT99bztR2xfVXemaWzfN8r6Qtn/rinmqG1fL+n6czfQlfRHWTfQHf2O7gMR8eZoU4WnJd0XEd+pOdpEtj8tqSPpvRFxV915prF9SlInItK/ycH2Q5K+HREP2L5SUisiXq871yy290l6VdJvRMTFvjmtMrY/oOL76lci4n9sPybpGxHxj/UmG8/2hyU9KukWSW9LelzSn0fEv5fx+imuqJu2gW4U3hwdXjH6qP9/vAlsH5T0B5IeqDvL5cT2+yQdUrGxhiLi7SaU9MhhSd/PWNLn2C/patv7JbUk/WfNeab5ZUnPRMRuRLwj6Z8l/XFZL56iqM81bQPdTEZTCScknZb0ZERkzvsFSZ/R2Y0fsgtJT9jesr1Rd5gpPiRpKOlLo2mlB2wfqDvUnO6R9EjdISaJiFcl/Z2kHUk/lPSTiHii3lRTPS/pt21fa7sl6fclfbCsF09V1LM20M0kIt6NiBslHZR0y+hHn3Rs3yXpdERs1Z1lAbdFxM2S7pR0r+1DdQeaYL+kmyV9MSJukvSWpM/WG2m20RTN3ZK+UneWSWz/vKQ/VPGf4S9KOmD7E/WmmiwiTkr6W0lPqJj2OCHp3bJeP01RN3UD3dGPuk9J+ljdWSa4VdLdo3nfRyXdbvvL9UaabnQ1pYg4LemYinm/jF6R9Mo5P00dVVHc2d0p6XhE/LjuIFP8nqT/iIhhRPxU0tck/VbNmaaKiAcj4tcj4pCk/5L0b2W9doqibtoGurZXbV8z+vxqSXdIeqneVONFxOci4mBEtFX8uPvNiEh7ZWL7wOiGskbTCB9V8WNlOhHxI0k/sH3DaOiwpJQ3wC/wcSWe9hjZkfSbtlujfjis4t5VWrZ/YfS4pmJ++uGyXnvmnolLcqukP5X03GjeV5qxgW7Nrpf00OjO+YqkxyIi/bK3hni/pGPF96b2S3o4Ih6vN9JUn5LUH00nvCzpkzXnmWr0n98dkv6s7izTRMQzto9KOi7pHUnPKv9byb9q+1pJP5V0b5k3llMszwMATJZi6gMAMBlFDQDJUdQAkBxFDQDJUdQAkBxFDQDJUdQAkNz/AZxbQl9weRVDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "x = np.array([[2,5],[3,6],[2,4],[5,7],[7,2],[8,3],[9,4],[8,4],[7,3]], dtype = float)\n",
    "#x = np.array([[1, 2],[1, 4],[1, 0],[4, 2],[4, 4],[4, 0]])\n",
    "k = 2\n",
    "centroids = np.array([[2,5],[7,2]], dtype = float)\n",
    "#centroids = np.array([[1,1],[4,1]], dtype = float)\n",
    "centroids, clusterAssigned = cluster(x, k, centroids)\n",
    "draw(x, centroids)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 문제: k-Means 클러스터링\n",
    "\n",
    "```sklearn.datasets.make_blobs```를 사용하여 데이터를 생성한다.\n",
    "```\n",
    "from sklearn.datasets import make_blobs\n",
    "\n",
    "X, y = make_blobs(n_samples=100, centers=4, cluster_std=0.60, random_state=0)\n",
    "```\n",
    "\n",
    "이 데이터를 4개의 클러스터로 군집화하세요.\n",
    "Python으로 구현해서 클러스터링하는데, 앞서 만든 함수 코드를 한 군데 수정해야 한다.\n",
    "\n",
    "(1) 클러스터의 중심점을 몇 회만에 찾는지, 매 반복 중심값만을 출력한다.\n",
    "(2) 배정된 클러스터가 맞는지 비교하여 동일한지 확인하세요."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Parameters\n",
    "* n_samples: 샘플 갯수 (기본 값은 100)\n",
    "* n_features: features 갯수 (기본 값은 2)\n",
    "* centers: 중심점 갯수\n",
    "* cluster_std: 클러스터 표준편차 (기본 값은 1.0)\n",
    "* random_state: 기본 값은 None\n",
    "Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.\n",
    "\n",
    "반환:\n",
    "* X 배열 [n_samples, n_features]\n",
    "* y: 배정된 클러스터 The generated samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_blobs\n",
    "\n",
    "X, y = make_blobs(n_samples=100, centers=4, cluster_std=0.60, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "함수를 약간 수정해야 한다.\n",
    "cluster함수의 3번째 줄을 아래와 같이 수정해야 한다.\n",
    "\n",
    "```python\n",
    "clusterAssigned = np.zeros(shape=(row, 2), dtype=float)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 3, 0, 0, 0, 0, 2, 3, 0, 3, 3, 3, 3, 3, 3, 1, 1, 2, 2, 1, 0, 3,\n",
       "       2, 1, 0, 2, 2, 0, 1, 1, 1, 3, 1, 1, 2, 0, 3, 1, 3, 2, 0, 2, 3, 2,\n",
       "       2, 3, 1, 2, 0, 0, 0, 1, 2, 2, 2, 3, 3, 1, 1, 3, 3, 1, 1, 0, 1, 3,\n",
       "       2, 2, 1, 0, 3, 1, 0, 3, 0, 0, 2, 2, 1, 1, 1, 3, 2, 0, 1, 2, 1, 1,\n",
       "       0, 0, 0, 2, 0, 2, 2, 3, 3, 2, 3, 0])"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "====== Iteration 1 ======\n",
      ".....Assigning [0.669787   3.59540802] to 2th centroid by 1.390\n",
      ".....Assigning [-2.02396992  7.99569054] to 1th centroid by 0.976\n",
      ".....Assigning [1.43289271 4.37679234] to 2th centroid by 0.681\n",
      ".....Assigning [1.06923853 4.53068484] to 2th centroid by 1.071\n",
      ".....Assigning [1.71444449 5.02521524] to 2th centroid by 1.064\n",
      ".....Assigning [0.74387399 4.12240568] to 2th centroid by 1.262\n",
      ".....Assigning [-2.04363942  4.06392123] to 0th centroid by 1.431\n",
      ".....Assigning [-0.67860329  7.88799076] to 1th centroid by 2.324\n",
      ".....Assigning [-0.04749204  5.47425256] to 2th centroid by 2.523\n",
      ".....Assigning [-1.46976488  7.69183251] to 1th centroid by 1.561\n",
      ".....Assigning [-0.86417686  6.86528639] to 1th centroid by 2.419\n",
      ".....Assigning [-1.84838498  6.90859736] to 1th centroid by 1.587\n",
      ".....Assigning [-1.27182547  7.13460392] to 1th centroid by 1.933\n",
      ".....Assigning [-0.93428978  7.73253222] to 1th centroid by 2.083\n",
      ".....Assigning [-0.73415741  7.44484466] to 1th centroid by 2.333\n",
      ".....Assigning [3.18515794 0.08900822] to 3th centroid by 1.495\n",
      ".....Assigning [1.35139348 2.06383637] to 3th centroid by 1.246\n",
      ".....Assigning [-0.09701715  3.48456995] to 2th centroid by 2.159\n",
      ".....Assigning [-1.93150361  2.9369806 ] to 0th centroid by 1.070\n",
      ".....Assigning [2.73890793 0.15676817] to 3th centroid by 1.121\n",
      ".....Assigning [2.33812285 3.43116792] to 2th centroid by 0.662\n",
      ".....Assigning [-1.93673697  7.57276799] to 1th centroid by 1.146\n",
      ".....Assigning [-1.98885366  3.24143178] to 0th centroid by 1.040\n",
      ".....Assigning [ 2.33466499 -0.02408255] to 3th centroid by 1.077\n",
      ".....Assigning [0.43899014 4.53592883] to 2th centroid by 1.650\n",
      ".....Assigning [-1.58597553  2.51979529] to 0th centroid by 1.493\n",
      ".....Assigning [-0.3731388   3.80619114] to 2th centroid by 2.381\n",
      ".....Assigning [0.76752279 4.39759671] to 2th centroid by 1.295\n",
      ".....Assigning [2.94821884 2.03519717] to 3th centroid by 1.404\n",
      ".....Assigning [2.76253526 0.78970876] to 3th centroid by 0.791\n",
      ".....Assigning [1.65179125 0.68193176] to 3th centroid by 0.472\n",
      ".....Assigning [-0.8731169   6.87422542] to 1th centroid by 2.406\n",
      ".....Assigning [2.16172321 0.6565951 ] to 3th centroid by 0.380\n",
      ".....Assigning [2.06156753 1.96918596] to 3th centroid by 0.971\n",
      ".....Assigning [-1.15845639  3.47120626] to 0th centroid by 1.901\n",
      ".....Assigning [1.1641107  3.79132988] to 2th centroid by 0.862\n",
      ".....Assigning [-1.54707525  8.99317925] to 1th centroid by 1.760\n",
      ".....Assigning [1.81336135 1.6311307 ] to 3th centroid by 0.658\n",
      ".....Assigning [0.04968579 8.63737679] to 1th centroid by 3.116\n",
      ".....Assigning [-1.97375691  2.42201914] to 0th centroid by 1.178\n",
      ".....Assigning [1.06269622 5.17635143] to 2th centroid by 1.504\n",
      ".....Assigning [-1.42689992  3.29890112] to 0th centroid by 1.601\n",
      ".....Assigning [-1.63442682  6.50141812] to 1th centroid by 2.027\n",
      ".....Assigning [-2.07459735  3.58809203] to 0th centroid by 1.096\n",
      ".....Assigning [-1.94364473  2.82810154] to 0th centroid by 1.070\n",
      ".....Assigning [-1.98351709  8.3420778 ] to 1th centroid by 1.072\n",
      ".....Assigning [1.53278923 0.55035386] to 3th centroid by 0.648\n",
      ".....Assigning [-1.90841166  3.32374224] to 0th centroid by 1.139\n",
      ".....Assigning [0.67047877 4.04094275] to 2th centroid by 1.330\n",
      ".....Assigning [0.44359863 3.11530945] to 2th centroid by 1.790\n",
      ".....Assigning [0.22459286 4.77028154] to 2th centroid by 1.935\n",
      ".....Assigning [1.868336   0.93136287] to 3th centroid by 0.148\n",
      ".....Assigning [-1.18094952  2.79290301] to 0th centroid by 1.831\n",
      ".....Assigning [-1.78799614  4.0274405 ] to 0th centroid by 1.589\n",
      ".....Assigning [-0.85092246  2.26992336] to 0th centroid by 2.270\n",
      ".....Assigning [-1.26285145  7.39264147] to 1th centroid by 1.840\n",
      ".....Assigning [-0.78518144  8.32956251] to 1th centroid by 2.239\n",
      ".....Assigning [2.49272186 0.97505341] to 3th centroid by 0.493\n",
      ".....Assigning [2.26908736 1.32160756] to 3th centroid by 0.419\n",
      ".....Assigning [-0.5354379   8.02562558] to 1th centroid by 2.465\n",
      ".....Assigning [-1.46065212  7.01048924] to 1th centroid by 1.830\n",
      ".....Assigning [1.41281595 1.5303347 ] to 3th centroid by 0.791\n",
      ".....Assigning [2.13141478 1.13885728] to 3th centroid by 0.191\n",
      ".....Assigning [1.89593761 5.18540259] to 2th centroid by 1.190\n",
      ".....Assigning [1.29297652 1.47930168] to 3th centroid by 0.854\n",
      ".....Assigning [-0.58846002  8.22861825] to 1th centroid by 2.422\n",
      ".....Assigning [-2.31644846  2.6409315 ] to 0th centroid by 0.772\n",
      ".....Assigning [-1.28929999  2.26204536] to 0th centroid by 1.863\n",
      ".....Assigning [2.18023251 1.48364708] to 3th centroid by 0.516\n",
      ".....Assigning [1.24258802 4.50399192] to 2th centroid by 0.910\n",
      ".....Assigning [-1.86880148  8.24441673] to 1th centroid by 1.157\n",
      ".....Assigning [1.35617762 1.43815955] to 3th centroid by 0.779\n",
      ".....Assigning [0.91433877 4.55014643] to 2th centroid by 1.217\n",
      ".....Assigning [-1.52157528  7.84594751] to 1th centroid by 1.486\n",
      ".....Assigning [1.00372519 4.19147702] to 2th centroid by 1.015\n",
      ".....Assigning [1.4949318  3.85848832] to 2th centroid by 0.525\n",
      ".....Assigning [-0.40636864  3.46150906] to 2th centroid by 2.466\n",
      ".....Assigning [-1.68770604  3.3993561 ] to 0th centroid by 1.372\n",
      ".....Assigning [2.29667251 0.48677761] to 3th centroid by 0.593\n",
      ".....Assigning [1.07714851 1.17533301] to 3th centroid by 0.939\n",
      ".....Assigning [ 1.56737975 -0.1381059 ] to 3th centroid by 1.218\n",
      ".....Assigning [-1.69199357  8.76126877] to 1th centroid by 1.513\n",
      ".....Assigning [-1.34796111  3.7137138 ] to 0th centroid by 1.800\n",
      ".....Assigning [0.00793137 4.17614316] to 2th centroid by 2.000\n",
      ".....Assigning [1.67467427 0.68001896] to 3th centroid by 0.456\n",
      ".....Assigning [-0.95855283  2.82487621] to 0th centroid by 2.049\n",
      ".....Assigning [1.5108885 0.9288309] to 3th centroid by 0.494\n",
      ".....Assigning [1.80709613 0.44919077] to 3th centroid by 0.584\n",
      ".....Assigning [1.87271752 4.18069237] to 2th centroid by 0.221\n",
      ".....Assigning [1.01618041 4.48527047] to 2th centroid by 1.097\n",
      ".....Assigning [0.95936074 4.56078645] to 2th centroid by 1.182\n",
      ".....Assigning [-1.30104869  2.25824179] to 0th centroid by 1.854\n",
      ".....Assigning [0.3471383  3.45177657] to 2th centroid by 1.741\n",
      ".....Assigning [-1.12352716  3.16235936] to 0th centroid by 1.883\n",
      ".....Assigning [-2.42165857  3.18151728] to 0th centroid by 0.606\n",
      ".....Assigning [-1.91088578  7.86675906] to 1th centroid by 1.097\n",
      ".....Assigning [-1.73030157  7.42173015] to 1th centroid by 1.395\n",
      ".....Assigning [-1.56784898  3.94588789] to 0th centroid by 1.716\n",
      ".....Assigning [-0.69574048  8.02669661] to 1th centroid by 2.304\n",
      ".....Assigning [-0.55552381  4.69595848] to 2th centroid by 2.649\n",
      "\n",
      ".....Assigned clusters: \n",
      " [[2.         1.39038171]\n",
      " [1.         0.97603959]\n",
      " [2.         0.6808694 ]\n",
      " [2.         1.07142116]\n",
      " [2.         1.06424068]\n",
      " [2.         1.26207595]\n",
      " [0.         1.43057818]\n",
      " [1.         2.32409741]\n",
      " [2.         2.52302284]\n",
      " [1.         1.56095699]\n",
      " [1.         2.41853581]\n",
      " [1.         1.58662436]\n",
      " [1.         1.93274354]\n",
      " [1.         2.08295409]\n",
      " [1.         2.33286093]\n",
      " [3.         1.4948262 ]\n",
      " [3.         1.2459688 ]\n",
      " [2.         2.15943258]\n",
      " [0.         1.0703532 ]\n",
      " [3.         1.1211712 ]\n",
      " [2.         0.66173786]\n",
      " [1.         1.14588633]\n",
      " [0.         1.03957021]\n",
      " [3.         1.0773791 ]\n",
      " [2.         1.65044584]\n",
      " [0.         1.49333913]\n",
      " [2.         2.38103961]\n",
      " [2.         1.29502248]\n",
      " [3.         1.4038348 ]\n",
      " [3.         0.79100091]\n",
      " [3.         0.47161079]\n",
      " [1.         2.40644969]\n",
      " [3.         0.37958046]\n",
      " [3.         0.97113953]\n",
      " [0.         1.90087301]\n",
      " [2.         0.86154172]\n",
      " [1.         1.75994186]\n",
      " [3.         0.65814888]\n",
      " [1.         3.11557902]\n",
      " [0.         1.17781016]\n",
      " [2.         1.50410806]\n",
      " [0.         1.60124506]\n",
      " [1.         2.02744612]\n",
      " [0.         1.09645899]\n",
      " [0.         1.07025022]\n",
      " [1.         1.07249929]\n",
      " [3.         0.64843469]\n",
      " [0.         1.13858427]\n",
      " [2.         1.3301515 ]\n",
      " [2.         1.79026886]\n",
      " [2.         1.93530467]\n",
      " [3.         0.14848052]\n",
      " [0.         1.83080141]\n",
      " [0.         1.588895  ]\n",
      " [0.         2.26970178]\n",
      " [1.         1.84026342]\n",
      " [1.         2.23920358]\n",
      " [3.         0.49335298]\n",
      " [3.         0.41933212]\n",
      " [1.         2.46469532]\n",
      " [1.         1.82995176]\n",
      " [3.         0.7912269 ]\n",
      " [3.         0.19118365]\n",
      " [2.         1.18996146]\n",
      " [3.         0.85417346]\n",
      " [1.         2.42235245]\n",
      " [0.         0.77212233]\n",
      " [0.         1.86308121]\n",
      " [3.         0.51613783]\n",
      " [2.         0.90976962]\n",
      " [1.         1.15730274]\n",
      " [3.         0.77877536]\n",
      " [2.         1.21709548]\n",
      " [1.         1.48642922]\n",
      " [2.         1.01450823]\n",
      " [2.         0.5245183 ]\n",
      " [2.         2.46588372]\n",
      " [0.         1.37171453]\n",
      " [3.         0.59279997]\n",
      " [3.         0.93935964]\n",
      " [3.         1.21755711]\n",
      " [1.         1.51341037]\n",
      " [0.         1.79961659]\n",
      " [2.         1.99984095]\n",
      " [3.         0.45631645]\n",
      " [0.         2.04894482]\n",
      " [3.         0.49426218]\n",
      " [3.         0.58361178]\n",
      " [2.         0.22102163]\n",
      " [2.         1.09699061]\n",
      " [2.         1.18212161]\n",
      " [0.         1.85381789]\n",
      " [2.         1.74140769]\n",
      " [0.         1.88348371]\n",
      " [0.         0.60615785]\n",
      " [1.         1.09723422]\n",
      " [1.         1.39518103]\n",
      " [0.         1.71632178]\n",
      " [1.         2.30441416]\n",
      " [2.         2.64859588]]\n",
      "\n",
      ".....Updating Centroids\n",
      " [[-3.  3.]\n",
      " [-3.  8.]\n",
      " [ 2.  4.]\n",
      " [ 2.  1.]] => \n",
      "[[-1.63043905  3.09879034]\n",
      " [-1.27208964  7.74944718]\n",
      " [ 0.78627354  4.2491454 ]\n",
      " [ 1.97918933  0.97920012]]\n",
      "\n",
      "\n",
      "====== Iteration 2 ======\n",
      ".....Assigning [0.669787   3.59540802] to 2th centroid by 0.664\n",
      ".....Assigning [-2.02396992  7.99569054] to 1th centroid by 0.791\n",
      ".....Assigning [1.43289271 4.37679234] to 2th centroid by 0.659\n",
      ".....Assigning [1.06923853 4.53068484] to 2th centroid by 0.399\n",
      ".....Assigning [1.71444449 5.02521524] to 2th centroid by 1.210\n",
      ".....Assigning [0.74387399 4.12240568] to 2th centroid by 0.134\n",
      ".....Assigning [-2.04363942  4.06392123] to 0th centroid by 1.050\n",
      ".....Assigning [-0.67860329  7.88799076] to 1th centroid by 0.609\n",
      ".....Assigning [-0.04749204  5.47425256] to 2th centroid by 1.482\n",
      ".....Assigning [-1.46976488  7.69183251] to 1th centroid by 0.206\n",
      ".....Assigning [-0.86417686  6.86528639] to 1th centroid by 0.974\n",
      ".....Assigning [-1.84838498  6.90859736] to 1th centroid by 1.019\n",
      ".....Assigning [-1.27182547  7.13460392] to 1th centroid by 0.615\n",
      ".....Assigning [-0.93428978  7.73253222] to 1th centroid by 0.338\n",
      ".....Assigning [-0.73415741  7.44484466] to 1th centroid by 0.618\n",
      ".....Assigning [3.18515794 0.08900822] to 3th centroid by 1.499\n",
      ".....Assigning [1.35139348 2.06383637] to 3th centroid by 1.253\n",
      ".....Assigning [-0.09701715  3.48456995] to 2th centroid by 1.168\n",
      ".....Assigning [-1.93150361  2.9369806 ] to 0th centroid by 0.342\n",
      ".....Assigning [2.73890793 0.15676817] to 3th centroid by 1.120\n",
      ".....Assigning [2.33812285 3.43116792] to 2th centroid by 1.754\n",
      ".....Assigning [-1.93673697  7.57276799] to 1th centroid by 0.688\n",
      ".....Assigning [-1.98885366  3.24143178] to 0th centroid by 0.386\n",
      ".....Assigning [ 2.33466499 -0.02408255] to 3th centroid by 1.064\n",
      ".....Assigning [0.43899014 4.53592883] to 2th centroid by 0.450\n",
      ".....Assigning [-1.58597553  2.51979529] to 0th centroid by 0.581\n",
      ".....Assigning [-0.3731388   3.80619114] to 2th centroid by 1.241\n",
      ".....Assigning [0.76752279 4.39759671] to 2th centroid by 0.150\n",
      ".....Assigning [2.94821884 2.03519717] to 3th centroid by 1.433\n",
      ".....Assigning [2.76253526 0.78970876] to 3th centroid by 0.806\n",
      ".....Assigning [1.65179125 0.68193176] to 3th centroid by 0.442\n",
      ".....Assigning [-0.8731169   6.87422542] to 1th centroid by 0.962\n",
      ".....Assigning [2.16172321 0.6565951 ] to 3th centroid by 0.371\n",
      ".....Assigning [2.06156753 1.96918596] to 3th centroid by 0.993\n",
      ".....Assigning [-1.15845639  3.47120626] to 0th centroid by 0.601\n",
      ".....Assigning [1.1641107  3.79132988] to 2th centroid by 0.594\n",
      ".....Assigning [-1.54707525  8.99317925] to 1th centroid by 1.274\n",
      ".....Assigning [1.81336135 1.6311307 ] to 3th centroid by 0.673\n",
      ".....Assigning [0.04968579 8.63737679] to 1th centroid by 1.592\n",
      ".....Assigning [-1.97375691  2.42201914] to 0th centroid by 0.759\n",
      ".....Assigning [1.06269622 5.17635143] to 2th centroid by 0.968\n",
      ".....Assigning [-1.42689992  3.29890112] to 0th centroid by 0.285\n",
      ".....Assigning [-1.63442682  6.50141812] to 1th centroid by 1.300\n",
      ".....Assigning [-2.07459735  3.58809203] to 0th centroid by 0.661\n",
      ".....Assigning [-1.94364473  2.82810154] to 0th centroid by 0.414\n",
      ".....Assigning [-1.98351709  8.3420778 ] to 1th centroid by 0.926\n",
      ".....Assigning [1.53278923 0.55035386] to 3th centroid by 0.619\n",
      ".....Assigning [-1.90841166  3.32374224] to 0th centroid by 0.358\n",
      ".....Assigning [0.67047877 4.04094275] to 2th centroid by 0.238\n",
      ".....Assigning [0.44359863 3.11530945] to 2th centroid by 1.184\n",
      ".....Assigning [0.22459286 4.77028154] to 2th centroid by 0.766\n",
      ".....Assigning [1.868336   0.93136287] to 3th centroid by 0.121\n",
      ".....Assigning [-1.18094952  2.79290301] to 0th centroid by 0.544\n",
      ".....Assigning [-1.78799614  4.0274405 ] to 0th centroid by 0.942\n",
      ".....Assigning [-0.85092246  2.26992336] to 0th centroid by 1.138\n",
      ".....Assigning [-1.26285145  7.39264147] to 1th centroid by 0.357\n",
      ".....Assigning [-0.78518144  8.32956251] to 1th centroid by 0.757\n",
      ".....Assigning [2.49272186 0.97505341] to 3th centroid by 0.514\n",
      ".....Assigning [2.26908736 1.32160756] to 3th centroid by 0.449\n",
      ".....Assigning [-0.5354379   8.02562558] to 1th centroid by 0.787\n",
      ".....Assigning [-1.46065212  7.01048924] to 1th centroid by 0.763\n",
      ".....Assigning [1.41281595 1.5303347 ] to 3th centroid by 0.790\n",
      ".....Assigning [2.13141478 1.13885728] to 3th centroid by 0.221\n",
      ".....Assigning [1.89593761 5.18540259] to 2th centroid by 1.452\n",
      ".....Assigning [1.29297652 1.47930168] to 3th centroid by 0.849\n",
      ".....Assigning [-0.58846002  8.22861825] to 1th centroid by 0.835\n",
      ".....Assigning [-2.31644846  2.6409315 ] to 0th centroid by 0.825\n",
      ".....Assigning [-1.28929999  2.26204536] to 0th centroid by 0.904\n",
      ".....Assigning [2.18023251 1.48364708] to 3th centroid by 0.543\n",
      ".....Assigning [1.24258802 4.50399192] to 2th centroid by 0.523\n",
      ".....Assigning [-1.86880148  8.24441673] to 1th centroid by 0.775\n",
      ".....Assigning [1.35617762 1.43815955] to 3th centroid by 0.774\n",
      ".....Assigning [0.91433877 4.55014643] to 2th centroid by 0.327\n",
      ".....Assigning [-1.52157528  7.84594751] to 1th centroid by 0.267\n",
      ".....Assigning [1.00372519 4.19147702] to 2th centroid by 0.225\n",
      ".....Assigning [1.4949318  3.85848832] to 2th centroid by 0.809\n",
      ".....Assigning [-0.40636864  3.46150906] to 0th centroid by 1.277\n",
      ".....Assigning [-1.68770604  3.3993561 ] to 0th centroid by 0.306\n",
      ".....Assigning [2.29667251 0.48677761] to 3th centroid by 0.586\n",
      ".....Assigning [1.07714851 1.17533301] to 3th centroid by 0.923\n",
      ".....Assigning [ 1.56737975 -0.1381059 ] to 3th centroid by 1.191\n",
      ".....Assigning [-1.69199357  8.76126877] to 1th centroid by 1.095\n",
      ".....Assigning [-1.34796111  3.7137138 ] to 0th centroid by 0.677\n",
      ".....Assigning [0.00793137 4.17614316] to 2th centroid by 0.782\n",
      ".....Assigning [1.67467427 0.68001896] to 3th centroid by 0.427\n",
      ".....Assigning [-0.95855283  2.82487621] to 0th centroid by 0.726\n",
      ".....Assigning [1.5108885 0.9288309] to 3th centroid by 0.471\n",
      ".....Assigning [1.80709613 0.44919077] to 3th centroid by 0.557\n",
      ".....Assigning [1.87271752 4.18069237] to 2th centroid by 1.089\n",
      ".....Assigning [1.01618041 4.48527047] to 2th centroid by 0.330\n",
      ".....Assigning [0.95936074 4.56078645] to 2th centroid by 0.356\n",
      ".....Assigning [-1.30104869  2.25824179] to 0th centroid by 0.903\n",
      ".....Assigning [0.3471383  3.45177657] to 2th centroid by 0.910\n",
      ".....Assigning [-1.12352716  3.16235936] to 0th centroid by 0.511\n",
      ".....Assigning [-2.42165857  3.18151728] to 0th centroid by 0.796\n",
      ".....Assigning [-1.91088578  7.86675906] to 1th centroid by 0.649\n",
      ".....Assigning [-1.73030157  7.42173015] to 1th centroid by 0.563\n",
      ".....Assigning [-1.56784898  3.94588789] to 0th centroid by 0.849\n",
      ".....Assigning [-0.69574048  8.02669661] to 1th centroid by 0.640\n",
      ".....Assigning [-0.55552381  4.69595848] to 2th centroid by 1.414\n",
      "\n",
      ".....Assigned clusters: \n",
      " [[2.         0.66403439]\n",
      " [1.         0.79117618]\n",
      " [2.         0.65909795]\n",
      " [2.         0.39916619]\n",
      " [2.         1.20987012]\n",
      " [2.         0.13364385]\n",
      " [0.         1.04986294]\n",
      " [1.         0.60944267]\n",
      " [2.         1.48190843]\n",
      " [1.         0.20590034]\n",
      " [1.         0.97372129]\n",
      " [1.         1.0193845 ]\n",
      " [1.         0.61484332]\n",
      " [1.         0.33822309]\n",
      " [1.         0.61818588]\n",
      " [3.         1.49893359]\n",
      " [3.         1.25322122]\n",
      " [2.         1.16823715]\n",
      " [0.         0.34179272]\n",
      " [3.         1.11962791]\n",
      " [2.         1.75423016]\n",
      " [1.         0.68772932]\n",
      " [0.         0.3857559 ]\n",
      " [3.         1.0643961 ]\n",
      " [2.         0.45038927]\n",
      " [0.         0.58069981]\n",
      " [2.         1.24114682]\n",
      " [2.         0.14963082]\n",
      " [3.         1.4332299 ]\n",
      " [3.         0.8059391 ]\n",
      " [3.         0.44221938]\n",
      " [1.         0.96186921]\n",
      " [3.         0.3706651 ]\n",
      " [3.         0.99340733]\n",
      " [0.         0.60121648]\n",
      " [2.         0.59359579]\n",
      " [1.         1.27376863]\n",
      " [3.         0.67269042]\n",
      " [1.         1.59232825]\n",
      " [0.         0.7588718 ]\n",
      " [2.         0.96753322]\n",
      " [0.         0.28543389]\n",
      " [1.         1.2995633 ]\n",
      " [0.         0.66082732]\n",
      " [0.         0.41396887]\n",
      " [1.         0.9259266 ]\n",
      " [3.         0.61901709]\n",
      " [0.         0.35759213]\n",
      " [2.         0.2382368 ]\n",
      " [2.         1.18448725]\n",
      " [2.         0.76620367]\n",
      " [3.         0.12073468]\n",
      " [0.         0.54369835]\n",
      " [0.         0.9419211 ]\n",
      " [0.         1.13783416]\n",
      " [1.         0.35692529]\n",
      " [1.         0.75737268]\n",
      " [3.         0.51354927]\n",
      " [3.         0.44864655]\n",
      " [1.         0.78672123]\n",
      " [1.         0.76263665]\n",
      " [3.         0.79027092]\n",
      " [3.         0.22059691]\n",
      " [2.         1.45187186]\n",
      " [3.         0.84911106]\n",
      " [1.         0.83483793]\n",
      " [0.         0.82476883]\n",
      " [0.         0.90361387]\n",
      " [3.         0.54303324]\n",
      " [2.         0.52265635]\n",
      " [1.         0.77528051]\n",
      " [3.         0.77381352]\n",
      " [2.         0.3271121 ]\n",
      " [1.         0.26749841]\n",
      " [2.         0.22496858]\n",
      " [2.         0.809203  ]\n",
      " [0.         1.27668056]\n",
      " [0.         0.30597269]\n",
      " [3.         0.58589717]\n",
      " [3.         0.92311741]\n",
      " [3.         1.1907812 ]\n",
      " [1.         1.09549178]\n",
      " [0.         0.6767013 ]\n",
      " [2.         0.78175818]\n",
      " [3.         0.42689436]\n",
      " [0.         0.72557566]\n",
      " [3.         0.47100183]\n",
      " [3.         0.55724858]\n",
      " [2.         1.08859834]\n",
      " [2.         0.32956368]\n",
      " [2.         0.35648188]\n",
      " [0.         0.90278451]\n",
      " [2.         0.9102949 ]\n",
      " [0.         0.51088226]\n",
      " [0.         0.79553257]\n",
      " [1.         0.64947871]\n",
      " [1.         0.56334415]\n",
      " [0.         0.84940673]\n",
      " [1.         0.63956672]\n",
      " [2.         1.4142355 ]]\n",
      "\n",
      ".....Updating Centroids\n",
      " [[-1.63043905  3.09879034]\n",
      " [-1.27208964  7.74944718]\n",
      " [ 0.78627354  4.2491454 ]\n",
      " [ 1.97918933  0.97920012]] => \n",
      "[[-1.5772186   3.11456071]\n",
      " [-1.27208964  7.74944718]\n",
      " [ 0.83044547  4.27831711]\n",
      " [ 1.97918933  0.97920012]]\n",
      "\n",
      "\n",
      "====== Iteration 3 ======\n",
      ".....Assigning [0.669787   3.59540802] to 2th centroid by 0.702\n",
      ".....Assigning [-2.02396992  7.99569054] to 1th centroid by 0.791\n",
      ".....Assigning [1.43289271 4.37679234] to 2th centroid by 0.610\n",
      ".....Assigning [1.06923853 4.53068484] to 2th centroid by 0.347\n",
      ".....Assigning [1.71444449 5.02521524] to 2th centroid by 1.157\n",
      ".....Assigning [0.74387399 4.12240568] to 2th centroid by 0.178\n",
      ".....Assigning [-2.04363942  4.06392123] to 0th centroid by 1.058\n",
      ".....Assigning [-0.67860329  7.88799076] to 1th centroid by 0.609\n",
      ".....Assigning [-0.04749204  5.47425256] to 2th centroid by 1.484\n",
      ".....Assigning [-1.46976488  7.69183251] to 1th centroid by 0.206\n",
      ".....Assigning [-0.86417686  6.86528639] to 1th centroid by 0.974\n",
      ".....Assigning [-1.84838498  6.90859736] to 1th centroid by 1.019\n",
      ".....Assigning [-1.27182547  7.13460392] to 1th centroid by 0.615\n",
      ".....Assigning [-0.93428978  7.73253222] to 1th centroid by 0.338\n",
      ".....Assigning [-0.73415741  7.44484466] to 1th centroid by 0.618\n",
      ".....Assigning [3.18515794 0.08900822] to 3th centroid by 1.499\n",
      ".....Assigning [1.35139348 2.06383637] to 3th centroid by 1.253\n",
      ".....Assigning [-0.09701715  3.48456995] to 2th centroid by 1.221\n",
      ".....Assigning [-1.93150361  2.9369806 ] to 0th centroid by 0.396\n",
      ".....Assigning [2.73890793 0.15676817] to 3th centroid by 1.120\n",
      ".....Assigning [2.33812285 3.43116792] to 2th centroid by 1.729\n",
      ".....Assigning [-1.93673697  7.57276799] to 1th centroid by 0.688\n",
      ".....Assigning [-1.98885366  3.24143178] to 0th centroid by 0.431\n",
      ".....Assigning [ 2.33466499 -0.02408255] to 3th centroid by 1.064\n",
      ".....Assigning [0.43899014 4.53592883] to 2th centroid by 0.469\n",
      ".....Assigning [-1.58597553  2.51979529] to 0th centroid by 0.595\n",
      ".....Assigning [-0.3731388   3.80619114] to 2th centroid by 1.293\n",
      ".....Assigning [0.76752279 4.39759671] to 2th centroid by 0.135\n",
      ".....Assigning [2.94821884 2.03519717] to 3th centroid by 1.433\n",
      ".....Assigning [2.76253526 0.78970876] to 3th centroid by 0.806\n",
      ".....Assigning [1.65179125 0.68193176] to 3th centroid by 0.442\n",
      ".....Assigning [-0.8731169   6.87422542] to 1th centroid by 0.962\n",
      ".....Assigning [2.16172321 0.6565951 ] to 3th centroid by 0.371\n",
      ".....Assigning [2.06156753 1.96918596] to 3th centroid by 0.993\n",
      ".....Assigning [-1.15845639  3.47120626] to 0th centroid by 0.550\n",
      ".....Assigning [1.1641107  3.79132988] to 2th centroid by 0.590\n",
      ".....Assigning [-1.54707525  8.99317925] to 1th centroid by 1.274\n",
      ".....Assigning [1.81336135 1.6311307 ] to 3th centroid by 0.673\n",
      ".....Assigning [0.04968579 8.63737679] to 1th centroid by 1.592\n",
      ".....Assigning [-1.97375691  2.42201914] to 0th centroid by 0.798\n",
      ".....Assigning [1.06269622 5.17635143] to 2th centroid by 0.928\n",
      ".....Assigning [-1.42689992  3.29890112] to 0th centroid by 0.238\n",
      ".....Assigning [-1.63442682  6.50141812] to 1th centroid by 1.300\n",
      ".....Assigning [-2.07459735  3.58809203] to 0th centroid by 0.687\n",
      ".....Assigning [-1.94364473  2.82810154] to 0th centroid by 0.465\n",
      ".....Assigning [-1.98351709  8.3420778 ] to 1th centroid by 0.926\n",
      ".....Assigning [1.53278923 0.55035386] to 3th centroid by 0.619\n",
      ".....Assigning [-1.90841166  3.32374224] to 0th centroid by 0.392\n",
      ".....Assigning [0.67047877 4.04094275] to 2th centroid by 0.286\n",
      ".....Assigning [0.44359863 3.11530945] to 2th centroid by 1.226\n",
      ".....Assigning [0.22459286 4.77028154] to 2th centroid by 0.780\n",
      ".....Assigning [1.868336   0.93136287] to 3th centroid by 0.121\n",
      ".....Assigning [-1.18094952  2.79290301] to 0th centroid by 0.510\n",
      ".....Assigning [-1.78799614  4.0274405 ] to 0th centroid by 0.937\n",
      ".....Assigning [-0.85092246  2.26992336] to 0th centroid by 1.114\n",
      ".....Assigning [-1.26285145  7.39264147] to 1th centroid by 0.357\n",
      ".....Assigning [-0.78518144  8.32956251] to 1th centroid by 0.757\n",
      ".....Assigning [2.49272186 0.97505341] to 3th centroid by 0.514\n",
      ".....Assigning [2.26908736 1.32160756] to 3th centroid by 0.449\n",
      ".....Assigning [-0.5354379   8.02562558] to 1th centroid by 0.787\n",
      ".....Assigning [-1.46065212  7.01048924] to 1th centroid by 0.763\n",
      ".....Assigning [1.41281595 1.5303347 ] to 3th centroid by 0.790\n",
      ".....Assigning [2.13141478 1.13885728] to 3th centroid by 0.221\n",
      ".....Assigning [1.89593761 5.18540259] to 2th centroid by 1.399\n",
      ".....Assigning [1.29297652 1.47930168] to 3th centroid by 0.849\n",
      ".....Assigning [-0.58846002  8.22861825] to 1th centroid by 0.835\n",
      ".....Assigning [-2.31644846  2.6409315 ] to 0th centroid by 0.878\n",
      ".....Assigning [-1.28929999  2.26204536] to 0th centroid by 0.900\n",
      ".....Assigning [2.18023251 1.48364708] to 3th centroid by 0.543\n",
      ".....Assigning [1.24258802 4.50399192] to 2th centroid by 0.470\n",
      ".....Assigning [-1.86880148  8.24441673] to 1th centroid by 0.775\n",
      ".....Assigning [1.35617762 1.43815955] to 3th centroid by 0.774\n",
      ".....Assigning [0.91433877 4.55014643] to 2th centroid by 0.284\n",
      ".....Assigning [-1.52157528  7.84594751] to 1th centroid by 0.267\n",
      ".....Assigning [1.00372519 4.19147702] to 2th centroid by 0.194\n",
      ".....Assigning [1.4949318  3.85848832] to 2th centroid by 0.786\n",
      ".....Assigning [-0.40636864  3.46150906] to 0th centroid by 1.221\n",
      ".....Assigning [-1.68770604  3.3993561 ] to 0th centroid by 0.305\n",
      ".....Assigning [2.29667251 0.48677761] to 3th centroid by 0.586\n",
      ".....Assigning [1.07714851 1.17533301] to 3th centroid by 0.923\n",
      ".....Assigning [ 1.56737975 -0.1381059 ] to 3th centroid by 1.191\n",
      ".....Assigning [-1.69199357  8.76126877] to 1th centroid by 1.095\n",
      ".....Assigning [-1.34796111  3.7137138 ] to 0th centroid by 0.642\n",
      ".....Assigning [0.00793137 4.17614316] to 2th centroid by 0.829\n",
      ".....Assigning [1.67467427 0.68001896] to 3th centroid by 0.427\n",
      ".....Assigning [-0.95855283  2.82487621] to 0th centroid by 0.683\n",
      ".....Assigning [1.5108885 0.9288309] to 3th centroid by 0.471\n",
      ".....Assigning [1.80709613 0.44919077] to 3th centroid by 0.557\n",
      ".....Assigning [1.87271752 4.18069237] to 2th centroid by 1.047\n",
      ".....Assigning [1.01618041 4.48527047] to 2th centroid by 0.278\n",
      ".....Assigning [0.95936074 4.56078645] to 2th centroid by 0.310\n",
      ".....Assigning [-1.30104869  2.25824179] to 0th centroid by 0.900\n",
      ".....Assigning [0.3471383  3.45177657] to 2th centroid by 0.957\n",
      ".....Assigning [-1.12352716  3.16235936] to 0th centroid by 0.456\n",
      ".....Assigning [-2.42165857  3.18151728] to 0th centroid by 0.847\n",
      ".....Assigning [-1.91088578  7.86675906] to 1th centroid by 0.649\n",
      ".....Assigning [-1.73030157  7.42173015] to 1th centroid by 0.563\n",
      ".....Assigning [-1.56784898  3.94588789] to 0th centroid by 0.831\n",
      ".....Assigning [-0.69574048  8.02669661] to 1th centroid by 0.640\n",
      ".....Assigning [-0.55552381  4.69595848] to 2th centroid by 1.448\n",
      "\n",
      ".....Assigned clusters: \n",
      " [[2.         0.70155255]\n",
      " [1.         0.79117618]\n",
      " [2.         0.61044251]\n",
      " [2.         0.34743575]\n",
      " [2.         1.15728608]\n",
      " [2.         0.17833395]\n",
      " [0.         1.05774939]\n",
      " [1.         0.60944267]\n",
      " [2.         1.48358885]\n",
      " [1.         0.20590034]\n",
      " [1.         0.97372129]\n",
      " [1.         1.0193845 ]\n",
      " [1.         0.61484332]\n",
      " [1.         0.33822309]\n",
      " [1.         0.61818588]\n",
      " [3.         1.49893359]\n",
      " [3.         1.25322122]\n",
      " [2.         1.22074627]\n",
      " [0.         0.39629858]\n",
      " [3.         1.11962791]\n",
      " [2.         1.72937932]\n",
      " [1.         0.68772932]\n",
      " [0.         0.43074318]\n",
      " [3.         1.0643961 ]\n",
      " [2.         0.46861612]\n",
      " [0.         0.59482989]\n",
      " [2.         1.292872  ]\n",
      " [2.         0.13485876]\n",
      " [3.         1.4332299 ]\n",
      " [3.         0.8059391 ]\n",
      " [3.         0.44221938]\n",
      " [1.         0.96186921]\n",
      " [3.         0.3706651 ]\n",
      " [3.         0.99340733]\n",
      " [0.         0.55005257]\n",
      " [2.         0.59032961]\n",
      " [1.         1.27376863]\n",
      " [3.         0.67269042]\n",
      " [1.         1.59232825]\n",
      " [0.         0.79803287]\n",
      " [2.         0.92758075]\n",
      " [0.         0.2378594 ]\n",
      " [1.         1.2995633 ]\n",
      " [0.         0.68674415]\n",
      " [0.         0.46510963]\n",
      " [1.         0.9259266 ]\n",
      " [3.         0.61901709]\n",
      " [0.         0.39172153]\n",
      " [2.         0.28624454]\n",
      " [2.         1.22565791]\n",
      " [2.         0.78043986]\n",
      " [3.         0.12073468]\n",
      " [0.         0.51038501]\n",
      " [0.         0.93689737]\n",
      " [0.         1.11396515]\n",
      " [1.         0.35692529]\n",
      " [1.         0.75737268]\n",
      " [3.         0.51354927]\n",
      " [3.         0.44864655]\n",
      " [1.         0.78672123]\n",
      " [1.         0.76263665]\n",
      " [3.         0.79027092]\n",
      " [3.         0.22059691]\n",
      " [2.         1.39931325]\n",
      " [3.         0.84911106]\n",
      " [1.         0.83483793]\n",
      " [0.         0.87794386]\n",
      " [0.         0.89982196]\n",
      " [3.         0.54303324]\n",
      " [2.         0.46988361]\n",
      " [1.         0.77528051]\n",
      " [3.         0.77381352]\n",
      " [2.         0.28448069]\n",
      " [1.         0.26749841]\n",
      " [2.         0.19382225]\n",
      " [2.         0.78600146]\n",
      " [0.         1.22117271]\n",
      " [0.         0.30547649]\n",
      " [3.         0.58589717]\n",
      " [3.         0.92311741]\n",
      " [3.         1.1907812 ]\n",
      " [1.         1.09549178]\n",
      " [0.         0.6415165 ]\n",
      " [2.         0.8288359 ]\n",
      " [3.         0.42689436]\n",
      " [0.         0.68312842]\n",
      " [3.         0.47100183]\n",
      " [3.         0.55724858]\n",
      " [2.         1.0468341 ]\n",
      " [2.         0.27807762]\n",
      " [2.         0.3104965 ]\n",
      " [0.         0.89975103]\n",
      " [2.         0.95747329]\n",
      " [0.         0.4562024 ]\n",
      " [0.         0.84709034]\n",
      " [1.         0.64947871]\n",
      " [1.         0.56334415]\n",
      " [0.         0.83137998]\n",
      " [1.         0.63956672]\n",
      " [2.         1.44752726]]\n",
      "\n",
      ".....NOT Updating Centroids\n",
      " [[-1.5772186   3.11456071]\n",
      " [-1.27208964  7.74944718]\n",
      " [ 0.83044547  4.27831711]\n",
      " [ 1.97918933  0.97920012]] => \n",
      "[[-1.5772186   3.11456071]\n",
      " [-1.27208964  7.74944718]\n",
      " [ 0.83044547  4.27831711]\n",
      " [ 1.97918933  0.97920012]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAXaUlEQVR4nO3db4xkWVnH8d8z1TOyLSjSM4lk2akmBkkmRIHpIKgxBjBZViLRaMKmlwAaJ+yIATUxyL7iRccXGpJNYNdMYGFDtxDCHyG4CmtcgyaK9MqKu+xCNrAzDELoHaL8GeLu9jy+uF10TU3dqnurzrn3nHu/n6TSU9XV1adqun731DnPOdfcXQCAdB1puwEAgNkIagBIHEENAIkjqAEgcQQ1ACRuJcaDHj9+3NfX12M8NAB00v333/+4u5+Y9r0oQb2+vq7d3d0YDw0AnWRm58u+x9AHACSOoAaAxBHUAJA4ghoAEkdQA0DiCOoAdnak9XXpyJHi685O2y0C0CVRyvP6ZGdHOnNGuny5uH7+fHFdkjY322sXgO6gR72k2247DOmRy5eL2wEgBIJ6SRcu1LsdAOoiqJd08mS92wGgLoK6glmThVtb0urq1fdfXS1uB4AQCOo5RpOF589L7oeThaOw3tyUzp2ThkPJrPh67hwTiQDCsRjnTNzY2PCubMq0vl6E86ThUHrssaZbA6CrzOx+d9+Y9j161HMwWdguatQBgnouJgvbM2/YCegLgnoOJgvbQ406UCCo52CysD0MOwEFlpBXsLlJMLfh5MnpE7kMO6Fv6FEjWQw7AYXeBDXVA/lh2Ako9CKoqR4o5Hiw2tws6tWvXCm+EtLoo14E9azqgRzDaxEcrIB89SKoy6oERmHVh/CqU+rWl4MXkIteBHVZlcBg0J863aqlbvS8gfT0IqjLqgf296ffv4t1ulVXWLLIBEhPL4K6rHpgOJx+/y7W6VYtdWORCZCeXgS1NL16oE91utMOVq9/fdFTHh+LLjtIuTNeDbSlUlCb2R+Z2UNm9qCZfdDMnha7YSHMmxTrW53u+MFqa0u6++5rx6Jvuunag9cI49VAO+buR21m10v6F0mn3P2HZvZhSfe4+/vLfiaF/agnzw4uFQHU5SCuY9Y+21tbRU972vdH92EvbiCsEPtRr0i6zsxWJK1K+u9QjYuFSbHZZpUsjnreZvV+FkAcc4Pa3b8h6S8lXZD0TUn/6+6fmbyfmZ0xs10z293b2wvf0prmTYrFqBXOqf64bCza7LDd7MUNJMLdZ14k/ZSkf5R0QtJRSX8j6ZZZP3P69Glv23DoXoy+Xn0ZDt23t91XV6++fXW1uH1RMR4zpu1td7Py12h0n5yeE5AzSbtekqlVhj5eKelr7r7n7k9K+pikX4xy1AhoVkVHjGGRhR+zpW745mYRvdOMPnX0bbIVSFWVoL4g6aVmtmpmJukVkh6O26zlzQqZGLXCs8Z8S7M38DLAuplfpY6cTZGABJR1tccvkt4h6RFJD0r6gKQfm3X/FIY+Zpk1LBL6MWcOFwRsyCLDFAxtAOnQjKGPSkFd95J6UDc1Rj03e8sGic1q//5FM397u7iP2eH4PYDmzQrq3p2Ka2fncDx5MCj2+xjVDi/zsX70s7fcMv37U4dGAp5ratHhHE4zBqSvN0vIpauHhKUipEcTjCHCanOz5v4hAdewU0oHdFevgrqJRTC1sjdgWUWf9i0B+qZXQV13eGCRyrna2RuorKLO781pYQ4A9Wsysc6EW5MVEU1O6FHpAaRJSy546Yw6wwNN7RXS9BlV2AMFyE+vgrrO8EBTG+g3HZycGADIT+/K86qWowWsnJup6eBs6nkBCKdXPeo6mqqiaLqsjuoQID8EdYmmNiS66aZr932OGZxstATkZ+4ZXhaRwhlecjDtLDRm0pveJN1xR3vtAtC8EGd4QQTTJhLdpXvuaac9ANJEUEdSZVEJFRgAqiCoI6haG83+HACqIKgVfkl11dpoKjAAVNH7oI6xMrDqkEbMCoyzZ6WVleJxV1aK6wDy1Puqj/X16QtAhsNij6RUHrOOs2elO++89vZbb6WaBEgVVR8zxJjQa3tI49y5ercDSFvvgzrGhF7bi0r29+vdDiBtvQ/qWL3fNs/ePRjUux1A2nof1G33fmM4c6be7QDS1rvd86bp2gleRxOG584Vwx2DQRHSTCQCeSKoO+qOOwhmoCt6P/SxCM45CKBJ9KhrmtzxbrRARurW8AmAdNCjrolzDgJoGkFdEzveAWgaQV0TO94BaBpBXVPby8MB9A9BXVMXF8gASBtVHwvo2gIZAGmjRw0AiSOoASBxBDUAJK5SUJvZM83sI2b2iJk9bGYvi90wAECh6mTi7ZL+3t1/28yOSVqd9wMAgDDmBrWZ/aSkX5H0Bkly9yckPRG3WQCAkSpDH8+VtCfpfWb2BTN7j5n9+OSdzOyMme2a2e7e3l7whgJAX1UJ6hVJL5Z0p7u/SNIPJL1t8k7ufs7dN9x948SJE4GbCQD9VSWoL0q66O6fO7j+ERXBDQBowNygdvdvSfq6mT3/4KZXSPpS1FYBAH6katXHH0raOaj4+KqkN8ZrEgBgXKWgdvcHJG3EbQoAYBpWJgJA4ghqAEgcQY1kcHZ3YDr2o0YSOLs7UI4eNZLA2d2BcgQ1ksDZ3YFyBDWSwNndgXIENZLA2d2BcgQ1ksDZ3YFyBDWSsbkpPfaYdOVK8bXrIZ1jOWKObe4CyvOAFuRYjphjm7vC3D34g25sbPju7m7wxwW6Yn29CLpJw2HxaSJFObY5J2Z2v7tP3VOJoQ+gBamVI1YZ0kitzX1CUAMtSKkccTSkcf685H44pDEZ1im1uW8IaqAFKZUjVl0VmlKb+4agBlqQUjli1SGNlNrcN0wmAj3HJGEamEwEUIohjfQR1MgeizCWw5BG+ghqlMohAKtWLGC2vq0KzQ1BjalyCUD2sUYfENSYKpcAZBEG+oCgxlS5BGDKizBGQ0dm0spK8TXVISSkjaDGVCkH4LhFKhaaGHsfHzqSpP394uv4EFLVduQwV4DI3D345fTp0468bW+7r666FyPUxWV1tbg9Ndvb7sOhu1nxdVYbm3pew+HVv2PaxWx+O3L6f8ByJO16Saay4AWldnaKMekLF4qe9NZW/tUATS3uOHKkiNW6JtvBYpT+YMELFtLFkq2QY++zhiQWHSKabEed9jJE0l0ENXol1Nj7vPLFaWPni7SvantzKafEYghq9Eqo5dKT5Ys3a0cPXV7XzbcU3dlN7fxotZ8kDQbzH3NaO6q2N5dySiyobPB6mQuTiUhZncnHMuMTgTdr27+v+TN+0yYGR48zqx1V2js5MTn++MiDZkwmEtTAAsarOr6msSvjl+Hwmp+79Vb3waD49mBQXA/dnjlNWFiIAxzKzQpqhj6QpNQnxsaHJE6q2ozfzo50992HNdX7+8X1EM8t9g54jIG3i6BGcnIJheuuK75eULUZv5jjyLF3wGMMvF2dDerUe2Qol2IojP89HT8uvfGN0qVLxfferi39QPO7s7GX5ccsp8xlS4GuqhzUZjYwsy+Y2adiNiiEZXpkBHz7UguFyb+nS5ekJ588/P4Htanf1zldHMzuzuayLH+anNveCWWD15MXSX8s6a8lfWrefdueTFx0YoXlumloYmIsRHvqVljk/PeVc9tzoWUnE83sOZJ+XdJ7oh0xAlq0R7bIR2564OGldmqoqj35eb3LnM+kknPbO6Eswccvkj4i6bSkX1VJj1rSGUm7knZPnjzZ7KFowqI9srq1qPQy4kmpFKxKjzq1//eUXj9Uo2XqqCW9WtIdB/8uDerxS9tDH4sGaN2AT+0jOuKY9vd07Jj72lqaQUgHIk/LBvWfS7oo6TFJ35J0WdL2rJ9pO6jdF+tR1P0DZzVYoQ+9t6aeY4jfQwciT0sF9VV3zqRHvYw6bxTeEPTeQgr1WtKByNOsoO5sHfWi6tSipnp2kSalWPOcq1CvJaV03VMrqN39n9z91bEak5u6M+G5rLirI7Wa55yFei1Tq5rB8jrZo26y11qnB97F3mdfem9N/E2Fei0ppeugsjGRZS5tjlGnPGbaxbHDlF/vUJp6jn14LVFOfdrmNOUJvpTbtoxlKhVyqBhp8v8th9cDcfQqqFPutdbpMeXwhl22jU30IEOfJCC1vyl0R6+COlbvJ1RwVnmcHD4Ch2hj7J5qqNexq5+EUpRDByWWXgV1jJBrOjhzCIYQbYzdUw31OuZw4OyCvr/OvQpq9+q91lQXtuTwUTtEG2O/riFfxz739JqSQwclpt4F9TxNLhVf5A2ewx9siDbG7kHl8DriUA4dlJh6H9STYbm2Vu8N3PT+1jl8BAzVxpg91RxeRxzq+4E1i6CO9Yad9mYtu4TeznSZP7zR6yEdnrU6tY/cOQwH5NBGFPp+YE0+qGP+B1U9O8e8AF3kDb/sR7m+/+HGQninq8//N8kHdcyPPGVhOXlZOABn/GUt+7z6/lEwBg5+SNWsoE5ir4+YG/uU7ZOwthZgL4Q5uywtuzkOGx6F18X9VtB9SQR1zI19ysLy9turb6ZUas67ftnNcfqy4VGTOPghR0kEdcxtGSfDcm1Nuu466XWvC7ALWoV3fZ3d9SaxXWV4HPyQoySCOva2jKOw/MAHpB/+ULp0aepIRX2R3/VsVxkeBz/kyIox7LA2NjZ8d3c3+OMua329COdJw2ER5LWNxqjHhz9WV0nTxO3sFKNTFy4Ux9StLf670D4zu9/dN6Z9L4kedVOCj09G6vLW3aS+a6f3im2Z4SigDSttN6BJz3pWMewx7faFbW4GfadPdtJHwzOjX7Xs/QHkp1c96hzULR+j3Azovl4F9Xe+U+/2NtQdnqHcDOi+5IM65PhrDqVZdduYw3MCsJykg3rOwr/acijNqtvGHJ4TgCWVrS1f5hJqm9MYe12E3PQl5o5/dR63zxvZAF2hGXt9JF1HfeRIEc2TzIrSqjZRQg0gpGzrqEOOv4auNabaAkBTkg7qUOOvoce6JaotADQn6aAOtfAvRu+XagsATUk6qKUwy31j9H6ptgDQlOSDOoQYvd8Qvf2zZ6WVleLnV1aK6yGxBwjQDb0I6li932V6+2fPSnfeKe3vF9f394vrocI6xrg8gHYkXZ4XUmpbW66sHIb0uMFAeuqp5R8/+JauAKKaVZ7Xm6BOjVn590L8l6Rcgw7gWtnWUXfZYFDv9rqoSuk+5iD6Y25Qm9kNZnafmX3JzB4ys7c00bCuG+0ZXfX2uqhK6TbmIOJJ8gBYtrZ8dJH0bEkvPvj3MyR9RdKpWT8Taq+Prrv1VvfBoNi/ZDAorofEHiDdFWMfnFBy/rvb3nZfXb36NV1dbeY5KOReH2b2CUnvcvd7y+7DGDUQV6pzELnvgdPmJHywMWozW5f0IkmfC9AuAAtKdQ4i9z1wUt0aonJQm9nTJX1U0lvd/btTvn/GzHbNbHdvby9kGwFMSHUOItWgqyrVA2CloDazoypCesfdPzbtPu5+zt033H3jxIkTIdsIYEKofXBCSzXoqkr1AFil6sMkvVfSw+7+zvhNAlBFiH1wQks16KpK9QA4dzLRzH5Z0j9L+i9Jo2mKt7v7PWU/w2Qi0F+prQLOBSsTASBxrEwEsJAkF3/00ErbDQCQpsma6NHqR4mhjKbRowY6IEbPN/ea6C4hqIFMlIVxrH0/YtREM5SyGCYTgQxMW5p99Kj0Ez8hXbo0/WeWXfYcejl17svLY2MyEcjctGGIJ58sD2lp+dWAoWuiGUpZHEENZGCR0F12NWDoxR9lz+H8eYZC5qHqA8jAyZPThyHKhFoNuLkZblhi1nMYH1sf/V4cokcNZGDaMESZVJY9T6ryHBgKmY6gBlpUtQpichhibU06duzq+6yuStvb6ez7MWnyOZTJZae9JhHUQEvqltWNb8L0+OPSXXelt3nQNOMHo9tuK3rWV64UbZ4ml532mkR5HtCSNs8m0pRZJXkS5XrjZpXnMZkItCT3TfarmFWSNzoYsdPefAQ10JKyKoguffSfdzAKWVXSZYxRAy3JfZP9H5kxIxrjjC99XIZOUAMtiX02kdCBNvXx5syIhj4YxdrXJHnuHvxy+vRpB9Ce7W331VX3Is6Ky+pqcXvIx/ve2vDqG0eX4fCqnx0O3c2Kr4u2wb34+Tm/LluSdr0kU6n6ADoodEVJ2ePt64iOaEqGmBU1eIEdOVJEc0O/rlFsygT0TOiKktLHU7OnHV9mzDvnsW2CGuig0JN4ZT/3zrVmZ0QXHfPOfWyboAY6qEqg7exIx48XwwZmxb/Lgqvs8X7h9sgzohMWnYDNfovVssHrZS5MJgLtmzWJt73tfvTotZNyx46VT/YtOykYclKxLrPpk5BmzbVhHjGZCGBc2eSgFGcJe9tnd8lhuT6TiUBPlU2gzZpUjLGEve2hh9wXFxHUQEfNmkCbNakYo2BjXhVK7IqM2IuLYiOogY6a1Yvd2ipOjjvp2LHDXmbI8JxVhdJURcb4NrGp7tldhqAGOmpWL3ZzU3rf+4oTEIysrRV7XG9uhg/PWUMPbQ+L5IDJRKCjlplAizH5trMzfUvTLq82rIPJRKCHlplAi7FXdtnQQ4wd9rqGoAY6apkJtCbDM/eKjCYQ1ECHLTqB1mR45l6R0QTO8ALgGqOQbOo0WZzpZTZ61ACmyrmcbVLOO+dJBDWAjmuiTjv2gYCgBtBpseu0mzgQVApqM7vRzL5sZo+a2dvC/XoAOBSjZxqj1HBcEwt25ga1mQ0kvVvSqySdknSzmZ0K1wQAiNczjV1qGPtAIFXrUb9E0qPu/lV3f0LShyS9JlwTACBezzR2qWETNedVgvp6SV8fu37x4LarmNkZM9s1s929vb1Q7QMQQA5VD7F6prHrtJuoOQ82meju59x9w903Tpw4EephASwpl/MFxuyZxiw1bGLBTpWg/oakG8auP+fgNgAZyGV3upyXkseuOa8S1J+X9Dwze66ZHZP0WkmfDNsMALE0MdkVAkvJy81dQu7uT5nZmyV9WtJA0l3u/lD0lgEI4uTJ6VuWprg7HUvJp6s0Ru3u97j7z7r7z7h7Bh9EAIzkPKSAAisTgY5jSCF/7J4H9ABDCnmjRw0AiSOoASBxBDUAJI6gBoDEEdQAkDhz9/AParYnaUqJfeuOS3q87UZEwPPKT1efG89rcUN3n7pRUpSgTpWZ7br7RtvtCI3nlZ+uPjeeVxwMfQBA4ghqAEhc34L6XNsNiITnlZ+uPjeeVwS9GqMGgBz1rUcNANkhqAEgcb0KajP7CzN7xMy+aGYfN7Nntt2mUMzsd8zsITO7YmbZl0eZ2Y1m9mUze9TM3tZ2e0Iws7vM7Ntm9mDbbQnJzG4ws/vM7EsHf4NvabtNoZjZ08zs383sPw+e2zvaaEevglrSvZJe4O4/J+krkv6s5faE9KCk35L02bYbsiwzG0h6t6RXSTol6WYzO9Vuq4J4v6Qb225EBE9J+hN3PyXppZL+oCP/X5L0f5Je7u4/L+mFkm40s5c23YheBbW7f8bdnzq4+m8qTtTbCe7+sLt/ue12BPISSY+6+1fd/QlJH5L0mpbbtDR3/6yk77TdjtDc/Zvu/h8H//6epIclXd9uq8LwwvcPrh49uDRegdGroJ7wu5L+ru1GYKrrJX197PpFdeSN33Vmti7pRZI+13JTgjGzgZk9IOnbku5198afW+fO8GJm/yDpp6d86zZ3/8TBfW5T8XFtp8m2LavKcwPaYmZPl/RRSW919++23Z5Q3H1f0gsP5rQ+bmYvcPdG5xk6F9Tu/spZ3zezN0h6taRXeGZF5POeW4d8Q9INY9efc3AbEmVmR1WE9I67f6zt9sTg7v9jZvepmGdoNKh7NfRhZjdK+lNJv+Hul9tuD0p9XtLzzOy5ZnZM0mslfbLlNqGEmZmk90p62N3f2XZ7QjKzE6PqMDO7TtKvSXqk6Xb0KqglvUvSMyTda2YPmNlftd2gUMzsN83soqSXSfpbM/t0221a1MGE75slfVrFxNSH3f2hdlu1PDP7oKR/lfR8M7toZr/XdpsC+SVJr5P08oP31QNmdlPbjQrk2ZLuM7MvquhA3Ovun2q6ESwhB4DE9a1HDQDZIagBIHEENQAkjqAGgMQR1ACQOIIaABJHUANA4v4fhnS5gccFhakAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "k = 4\n",
    "centroids = np.array([[-3,3], [-3,8], [2,4], [2,1]], dtype = float)\n",
    "centroids, clusterAssigned = cluster(X, k, centroids)\n",
    "draw(X, centroids)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## C.6 sklearn\n",
    "\n",
    "### 데이터"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "X = np.array([[2,5],[3,6],[2,4],[5,7],[7,2],[8,3],[9,4],[8,4],[7,3]], dtype = float)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### k-means 클러스터링\n",
    "\n",
    "* n_clusters=2, 클러스터를 2개로 설정,\n",
    "* ```init='random'```은 초기화를 무작위로 설정,\n",
    "* ```random_state=2```에 정수를 입력하면, 그 정수는 중심값을 무작위로 생성하는 seed로 쓰인다. seed를 정하게 되면, 클러스터링을 몇 번 해도 동일한 결과가 만들어질 수 있다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cluster import KMeans\n",
    "\n",
    "kmeans = KMeans(n_clusters=2, init='random', random_state=2).fit(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 클러스터 할당\n",
    "\n",
    "각 데이터 항목에게 배정된 클러스터를 출력하면, 앞서의 결과와 동일하다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cluster label:  [0 0 0 0 1 1 1 1 1]\n"
     ]
    }
   ],
   "source": [
    "print (\"Cluster label: \", kmeans.labels_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "클러스터의 중심점을 출력해보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Centroid:  [[3.  5.5]\n",
      " [7.8 3.2]]\n"
     ]
    }
   ],
   "source": [
    "print (\"Centroid: \", kmeans.cluster_centers_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 예측\n",
    "\n",
    "두 점을 입력하고, 어느 클러스터에 할당되는지 예측해볼 수 있다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predictions:  [0 1]\n"
     ]
    }
   ],
   "source": [
    "print (\"Predictions: \", kmeans.predict([[0, 0], [7, 4]]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 문제: sklearn 클러스터링"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cluster label:  [1 0 1 0 0 1]\n",
      "Centroid:  [[3.         3.33333333]\n",
      " [2.         0.66666667]]\n",
      "Predictions:  [1 0]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.cluster import KMeans\n",
    "import numpy as np\n",
    "X = np.array([[1, 2],[1, 4],[1, 0],[4, 2],[4, 4],[4, 0]])\n",
    "kmeans = KMeans(n_clusters=2, random_state=0).fit(X)\n",
    "print (\"Cluster label: \", kmeans.labels_)\n",
    "print (\"Centroid: \", kmeans.cluster_centers_)\n",
    "print (\"Predictions: \",kmeans.predict([[0, 0], [4, 4]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## C.7 Spark\n",
    "\n",
    "Python 실행경로를 지정해주고, SparkSession을 먼저 생성하자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ[\"PYSPARK_PYTHON\"]=\"/usr/bin/python3\"\n",
    "os.environ[\"PYSPARK_DRIVER_PYTHON\"]=\"/usr/bin/python3\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pyspark\n",
    "\n",
    "myConf=pyspark.SparkConf()\n",
    "spark = pyspark.sql.SparkSession.builder\\\n",
    "    .master(\"local\")\\\n",
    "    .appName(\"myApp\")\\\n",
    "    .config(conf=myConf)\\\n",
    "    .getOrCreate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 데이터 생성\n",
    "\n",
    "numpy 배열 데이터를 생성해보자. 2개의 속성으로 구성된 배열을 2차원으로 가지고 있다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "x = np.array([[2,5],[3,6],[2,4],[5,7],[7,2],[8,3],[9,4],[8,4],[7,3]], dtype = float)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DataFrame\n",
    "\n",
    "Numpy array와 Spark Dataframe은 서로 다른 구조를 가지고 있으며, 데이터타입도 서로 다르다.\n",
    "numpy array는 Pandas를 통하거나, RDD로 만든 후, DataFrame으로 변환해준다.\n",
    "여기서는 RDD를 통해서 DataFrame으로 만들기로 하자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "_xRdd = spark.sparkContext.parallelize(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rdd는 컬럼명이나 데이터타입을 설정하지 않고 만들었다.\n",
    "데이터를 정수형으로 변환해주자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "xRdd = _xRdd.map(lambda x: [int(i) for i in x])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "데이터가 올바르게 읽혀졌는지 collect() 해보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[2, 5], [3, 6], [2, 4], [5, 7], [7, 2], [8, 3], [9, 4], [8, 4], [7, 3]]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xRdd.collect()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이제 DataFrame으로 변환하자. 컬럼명은 같이 리스트 ```[\"x\", \"y\"]```로 입력해준다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+---+---+\n",
      "|  x|  y|\n",
      "+---+---+\n",
      "|  2|  5|\n",
      "|  3|  6|\n",
      "|  2|  4|\n",
      "|  5|  7|\n",
      "|  7|  2|\n",
      "|  8|  3|\n",
      "|  9|  4|\n",
      "|  8|  4|\n",
      "|  7|  3|\n",
      "+---+---+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "xDf = xRdd.toDF([\"x\", \"y\"])\n",
    "xDf.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Features 설정\n",
    "\n",
    "클러스터링에 쓰이게 될 속성을 ```VectorAssembler```로 묶어서 ```features``` 컬럼을 생성해주자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#from pyspark.ml.linalg import Vectors\n",
    "from pyspark.ml.feature import VectorAssembler\n",
    "\n",
    "va = VectorAssembler(inputCols=[\"x\",\"y\"],outputCol=\"features\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```VectorAssembler```를 적용한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "xxDf = va.transform(xDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      " |-- x: long (nullable = true)\n",
      " |-- y: long (nullable = true)\n",
      " |-- features: vector (nullable = true)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "xxDf.printSchema()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+---+---+---------+\n",
      "|  x|  y| features|\n",
      "+---+---+---------+\n",
      "|  2|  5|[2.0,5.0]|\n",
      "|  3|  6|[3.0,6.0]|\n",
      "|  2|  4|[2.0,4.0]|\n",
      "|  5|  7|[5.0,7.0]|\n",
      "|  7|  2|[7.0,2.0]|\n",
      "+---+---+---------+\n",
      "only showing top 5 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "xxDf.show(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### k-means 클러스터링\n",
    "\n",
    "mllib, ml 모두 KMeans 모델을 지원한다.\n",
    "* k를 몇 개, 어떻게 정할지 설정한다.\n",
    "\n",
    "클러스터의 수 k=1로 하고 k-means를 실행해보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from pyspark.ml.clustering import KMeans\n",
    "\n",
    "kmeans = KMeans(k=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = kmeans.fit(xxDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 클러스터 중심점"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([3. , 5.5]), array([7.8, 3.2])]\n"
     ]
    }
   ],
   "source": [
    "print (model.clusterCenters())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 문제: iris 데이터 클러스터링\n",
    "\n",
    "iris는 꽃잎 및 꽃받침의 너비와 높이 데이터이다.\n",
    "이 데이터를 군집화 해보자."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 데이터\n",
    "\n",
    "iris 데이터는 seaborn, sklearn 등 많은 패키지에서 내장해서 제공되고 있다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "\n",
    "dataset = datasets.load_iris()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "sklearn에서 제공하는 데이터는 타입이 다르기 때문에, 바로 사용할 수 없고 변환을 해야 한다.\n",
    "```sklearn.utils.Bunch```라고 하는 타입으로 저장되어 있는데, 이는 **Dictionary 타입**이라서 아래와 같은 키를 입력하여 읽을 수 있다.\n",
    "* 'data': numpy.array로 저장된 속성 데이터\n",
    "* 'target': numpy.array로 저장된 분류 데이터\n",
    "* 'target_names': numpy.array로 저장된 분류 값 ['setosa', 'versicolor', 'virginica']\n",
    "* 'DESCR': 데이터 관련한 설명\n",
    "* 'feature_names': ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "```dataset['data']```는 numpy array 타입이다. 데이터를 읽어보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data = dataset['data']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "인덱스 ```1:5```, 즉 1부터 4까지, 4행을 읽어보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[4.9, 3. , 1.4, 0.2],\n",
       "       [4.7, 3.2, 1.3, 0.2],\n",
       "       [4.6, 3.1, 1.5, 0.2],\n",
       "       [5. , 3.6, 1.4, 0.2]])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[1:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### 그래프\n",
    "\n",
    "변수가 4개이므로, 4차원 그래프를 그려야 맞다.\n",
    "그렇게 되면 명령어가 복잡해지므로, 3차원으로 그래프를 그려보자.\n",
    "gca(projection='3d')라고 해주면 된다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQoAAAD3CAYAAADlsBq6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAACKsklEQVR4nO2dd3gcZ7n2f+/2XfVuW7Yky7339B4S0p0CBAIk5CSHEkI5EErCIeQAJ3zkJIEc0g4QSKgJ2CkQBwiYhBRSbMe2JFu2LKv3tlptb/N+f6xmvFqtpF1pVUL2vi5dsrU778zsztzzlPt5HiGlJI000khjPOhm+wDSSCONuY80UaSRRhoTIk0UaaSRxoRIE0UaaaQxIdJEkUYaaUyINFGkkUYaE8Iwwevp3GkaaUw/xGwfwERIWxRppJHGhEgTRRpppDEh0kSRRhppTIg0UaSRRhoTIk0UaaSRxoRIE0UaaaQxIdJEkUYaaUyINFGkkUYaEyJNFGmkkcaESBNFGmmkMSHSRJFGGmlMiDRRpJFGGhMiTRRppJHGhEgTRRpppDEh0kSRRhppTIiJ+lGkMQ1QFAWfz4dOp8NgMKDX6xFizrckSOM9jDRRzCCklIRCIUKhEMFgEEVRNILQ6/UYjcY0caQxJ5F2PWYIUkoCgQBvvvkmQogR1oROp0NRFLxeLy6XC4fDgdPpxOfzEQqFSA9pSmO2kbYoZgDhcJhgMKhZFLEQQmjkARFSUYlDtSwMBgNGoxG9Xp+2ONKYcaSJYhoR7WrEEsHQ0BA2mw2DYfRXEI84wuHwCJJRicNgMKDT6dLEkca0Ik0U0wRFUUbEIdQbWUpJVVUVwWCQYDCIEILc3Fzy8vLIyclBr9ePWit6e3WNaOIQQmAwGLSfNHGkkWqkiSLFUG/iYDAIjLzJBwcHcbvdLF68mIKCAoQQBINBBgcH6evr4/jx4+j1+hHEoVoV0YhHHGqAVH09TRxppBJigkBZOoqWBGJdjWgroqmpie7ubkKhEKeddtqImzoagUCAwcFB7HY7DocDo9FIXl4eeXl5ZGVlxSWOeMeh/gD09PRQWlqaJo65izn/ZaQtihRBURQCgQBSyhEk4ff7qa6uJjMzk5NOOom33noLiBBEPJI2mUwUFxdTXFysbW+32+no6MDpdGI2mzWLIysrK+4NH2txtLa2UlxcPMriUIOjaeJIYyKkiWKKiHY1ogOQAP39/Rw5coTly5dTVFQEjE0QY8FsNjNv3jzmzZsHgM/nw26309bWhsvlwmKxaBZHRkbGmMQRHfuQUmoxEvV1NTBqMBhGEU0aaaSJYgpQtRGxAUtFUaivr8fhcLBlyxYsFou2TbJEEQuLxcL8+fOZP38+Ukq8Xi92u53m5mZcLhc2m00jDpvNljBxBAIB/H4/ADqdDqPRqFkcaeJII00Uk4SiKDQ2NpKfn4/VatVuJK/XS1VVFYWFhWzdunXUDTZVoohdy2azYbPZKC0tRUqJx+PBbrfT0NCAx+MhMzOTQCCAx+MZcZyx66jEoR5bIBAgEAgAaOKwaFcljfcW0kSRJKIDlna7nezsbO3m6+7upr6+nlWrVpGfnx93+1QSRby1MzIyyMjIYOHChUgpcblcDA4OUl9fj9frJSsrS7M4oi2d6DWANHGkMQJpokgCsa6GKr0Oh8McPXoUn8/Htm3bMJlMY66hEsVMmPJCCLKysjCZTKxfvx4pJU6nE7vdzpEjRwgEAiOIw2w2x10D0sTxXkeaKBJErE5B/fF4PNTW1rJgwQJWrVo1IQHMJFHE23d2djbZ2dmUl5ejKIpGHIcPHyYYDJKTk0NeXh65ublxCS8ecagEqhKHen4ZGRlp4vgXQZooJsBY2ggAt9tNX18fGzduJDs7O6H1ptP1SBY6nY6cnBxycnKoqKhAURQcDoeWVQmHwyOIw2g0jlojnvhrYGCA/v5+lixZou0nNquSxrsLaaIYB2PJsEOhEIcPH8bv97NixYqESQLmFlHEQqfTaW4IRIrZVOJoaWlBSjlCNTpRnYper9csDr/fr2VV0iX17z6kiSIOYmXY0abz0NAQNTU1lJeXY7FYJm1Wz1WyiIZeryc/P18LzIZCIU012tjYOGadSrRrFc/iUBv3RO8nTRxzG2miiIEqRgqHw6Nk2C0tLXR0dLBhwwYyMjI4duxY0je8EIL+/n4aGxtH1HXk5ubGfULPJRgMBgoLCyksLASIW6eSl5c37o2eJo53J+b2lTnDGEuGHQgEqKmpwWKxcNJJJ2lPTjXrkcz6TqcTv9/Pli1bEEJopn1TUxNCCM30H6sgbC7BaDRSVFSkqU7VOpWOjg5cLhdDQ0MT1qmMRRzRvTjSxDH7SBMFY/eNABgYGKC2tpalS5dSUlIyYrtk4g2qEEun07Fq1SpMJhOKolBQUEBBQQEQeULb7XZ6enqor6/HYDCQl5dHfn7+mHUdcwlqnQqAx+Nh/vz5o+pUVOtpojqV8Zr4pIlj5vGeJwo1tffGG29w8sknj3A1Ghoa6OvrY/PmzVit1lHbJkoUvb291NXVsWrVKtrb28d8n9FoHFUQNjAwQFtbG06nE6vVqhHHWPLsuQDVIoutU/F6vQwODiZdp5JI9y/1J00c04P3NFGoVkSsrsHn81FdXU1ubi7btm0b0wWYyPVQFIVjx47hdDrZunUrZrOZjo6OhK0Qs9kct64jWp6tEkc8leVsYSydiNVqxWq1jjqfpqYm3G53wnUqY3X/qq+vZ8mSJVqdSrqkPnV4TxJFPFdDCIGiKPT391NXV8fKlSs1l2AsjGdR+Hw+qqqqKCgo0OIRE20z0b5i6zpcLhd2u52jR4/i9/vJzs7WbrTx1KHTjUQEZYnWqUTLzScqqR8aGkKv14/q/hXtqqSJY3J4zxHFWNoIIQR1dXV4vV7t6T8Rxrrpo12N2JqPVOkoVHl2VlYWZWVlKIrC0NAQdrud9vZ2wuGwllGZ6VTsZJSn49WpHDt2DJ/PN4o4xtpvbHA03TZw6njPEMV42giPx8PQ0BC5ubls3rw54QtHp9Np68GJ8vKhoaGEySZV0Ol05Obmkpuby+LFiwmHw5rmwePxsG/fvhnLqKRCoh5NhIsWLYpbp5Kdna2R4VifdTziSLcNTB7vCaIYSxsB0NnZSUNDA9nZ2SxcuDCpiyTaOhjL1Rhvm+mEXq/XMip2u53169drGZVjx45hNBrJz88ft1PWZCGlTDkRTVSnEgqF8Pv99PT0jFmnoq4TSxyxTXzS3b9G41+eKMbSRoTDYWprawmFQpx00kkcOnQoKU0EnLjp+/r6OHr06JTjGtOJ2IxKdKcsp9OZUCAxUcxE0VtsnUo4HObtt9/G5XJpdSqqhTVWnQok1/3rvUwc/7JEMZ42wul0Ul1dzaJFizQrQqfTTUpl2dvbS39//5TjGjONeJ2yBgYGRgQSVYsj2YzKbFTH6vV6DAYDlZWVwOTqVCDx7l/vtbaB/5JEMVaLOiklbW1ttLa2sm7dOrKysrRtklVZ+nw+jh8/jsFgiNvJaizMZpn5WIjOQEQHEmPjAfn5+eOa9Spm4/xiyXesOpWBgYFx61RiMR5xRKdqzWbzvzRx/MsRRfRgnegvLRgMcujQIQwGAyeffPKoCyMZolBdjXnz5mlxj0QxF4kiFmNlVFTxV3RGJV6Nymycn6Io48ZFEq1TycvLIzs7e8y1oolDJaeamhrKysrIyMjQmvi0t7ezcOHCuEK9dyP+ZYhCdTXUp0VZWZn22uDgIIcOHaKyspL58+fH3T4RopBSUl9fz+DgIFu3bsXlctHb25vUcQoh8Pl87NmzBymlJpga7+KcbURnVCDydB6vRmUuEkUsxqpT6e7u5tixY5p8fqI6FXXfJpNJe08gEOBrX/sad999NytXrkzB2c0+/iWIIlobodfrtZx59OCdTZs2YbPZxlxDFVyNhWi1pupquN3upOMNbreb9vZ21q1bh8lkYnBwkK6uLurq6jCbzVpcYCxJ81yAwWAYs0bl2LFjhEIhcnNzMZvNM1ajkixRxCKReSoqcWRmZo44p3A4PEI6rtfrcbvdZGZmTu2k5hDe1UQRb3yfXq/XfMjq6mqysrI46aSTJryIxgtmqvM5YrMayQQmVTn30NAQlZWVZGdnEw6HR1ycsZJmNaCYn58/aU3GTAROYzMqR48eRafTTUtGZSxMlShiMVadSmtr66g6lVAoNGrfHo9nRAzs3Y53LVGM1aJOp9PhcrnYu3fviME7EyGe6yGl5Pjx49jt9lHzOcbaJh6iNRalpaVxG7zAiVqIBQsWaAHFgYEBTSegtqXLy8ub070rogOJ8aTZE3UCnwzUwPV0YaI6lcOHD48gQ7V2ZSoQQqwAnor6UyVwp5Tyh1NaeBKYu1fbOBhLG6EoCp2dnTgcDk4++eSkLsLYm97v91NVVTXC1YhFIhaFWqauWiONjY0JHU90QLG8vHxEuq+5uVmLC8zF+EZsh6tYaXaswjKaAMfSOySyz5n6DGLrVN566y0qKys1Mvzyl7+Mw+Hg8ccf5/zzz2fx4sWTIjEp5VFg4/A+9UA78ExKTyZBvKuIYrzxfWq/h4yMDIqKipJ+UkUThepqrFixQouSx8N4RCGlpLGxkd7e3hHWyGR1FLHpPjUuMBfjG+MFM+MpLKMb+iqKkpDeIRapdj2SQSwZvvDCC5x55pk4nU7uuOMOfvOb36Ti+zgfOC6lbE7BISeNdw1RjKWNgBODd1avXo0QYtyeD2NBCEE4HKa+vp6BgYG4rkYsxnI9gsEg1dXV2Gy2UWXqqRJcxcYFxopvqI1yZxLJZD1iG/qqGRVV76BmXCaynGabKKKhVqt+6UtfSuVuPgz8NpULJoN3BVGMJ8OOHbwzNDSUtBRb3UdrayslJSVs3bo1oYsu3k2vNt+trKzUAmETbZMKjBXfqK2txe12U1dXN6Pxjck+QWMzKtFpy7q6Okwmk+ZyRWcfZosoxrMoUwUhhAm4Arg9ZYsmiTlNFOPJsF0uF9XV1aMG7ySrsISIq9HU1ERhYSHLly9PeLvom15Vfba1tWnNdyfaZroQK5jas2cPhYWFMxbfSGVgMTZtqdaotLa2jsiowOTJaSqIR1CBQCDV/UAuBt6RUnanctFkMGeJYjxXo729nebmZtauXTtqpoZer0+YKNR2d/39/VRWVo4oGU8E0SMFDx8+DDCi+W48qHqNmbyohRAzGt+YTsFVbI2KmlHp6urC5/Ph9/s1EpyJMv9wODyKKNxu95gPikniI8yi2wFzkCjUgGVzczNms5nCwkLtolMH70DkhoxnQut0OsLh8IT7UXUW2dnZbN26lb6+Pq3wJ1EIIQiFQrz99ttagVki28x2UVii8Y3J3mwzpcyMDiIaDAaNJOx2O7W1tSnLqIwHVWwVDZfLlTKiEEJkABcAn0rJgpPEnCKKaFcjVh8RPXintLR0zDUScT1U3z1aZzGZG7i3txen08nJJ5/8rhwpqCLV+o3ZOD9VlTtWRqW1tXVEBWlubu64ll+y+41GKlWZUko3MH7vgiQghMgH9EBw+McvpQxNtN2cIYrYFnVq70MpJc3NzXR2do7r+6tQt4uHaFcjNquRTGxDURTq6uo0EzPZkYLqsczEDTWZ0vlU6DfmQq1HvIyKWkHa0NCQcCHYeIhnUUyD6zFlDFsmFwKnADpAAcxANfCYEELIcS6WWSeKsVrU6fV6PB4P+/fvx2q1cvLJJ086EwGRAFNVVRVZWVlxsxrJqCwPHjxIUVERy5Yt4+23307kNEccn9/vZ9++fSPKt/Pz86fFNFb3OVlMRr8xW0VhE1kIsRWkakZFPReTyaSdS2w9x1gYiyjmSp2HEEIvpQwD5wDfBR4FBgATkAl0JrLOrBLFeC3qPB4PLS0trFmzZtTgnfEQ78uN52rEIhGiUIVYatPcyVgFTqeTjo4O1q9fr0l91fLt6GrSuTopLJH4htfrJRgMzmiJ9WTSo/EyKgMDA7S0tOByubSMSn5+PlarNe61Nd0xiqlimCQAQsAfpJQ/GuN9417Is0YUY2kj1PqKrq4u5s2blxRJxEJ1NcYb4qNiPKIYy2VJ5qkZXcm6YMEC8vLyCAaDWju3xYsXEwqFRlRhqk/ruTzwJ15849ChQ9TX1xMOh2esPiUVOgqLxcKCBQu0c1EzKvX19SO6gEcHed8FFsU6YDOwCFgmhLgGOAL4hn/6pZS+cZYAZoEoxtNGRJdyr1q1ip6enknvJxAIUF1dTWZm5rhDfFSM57Ko6yQqxIpFKBSipqYGk8nE0qVLcTqdcd9nMBhG9EiIbk/n9XrJysrSTOPZnNsxFtT4htlsZs2aNej1+hmrT0l1rcd4NSqHDx/WSB4YpeBVWwmm4BhygZ8CawEJ/JuU8o0kl1kIfBxoArKAu4F+InGK+cD9wANRLkpczChRjKeNUGdhqMVTDocjoTRnPIRCIfbs2cOyZcs0s3IixLMoHA4HNTU1Sa0TC5fLRVVVFRUVFSxYsIDe3t6E3RWr1Uppaak2IEftMtXe3o6iKHPaTVED0jOl35huZeZYNSrNzc0MDAzQ29tLbm4uiqJgt9sTSpUngAeAP0spPzCszky6HFVK+SfgT0KICsAnpewSQqimdS7gGn7fuDfbjBFF7Pi+aOmtmkGIblCbjHBKhVqI5ff7OeOMM5LykaOJQkpJa2sr7e3tEza8GQ/d3d0cP358hDBssulRIcQoN2VwcJDe3t4RbspcLgqbTv2GShRSSg53uXB4Q1QW2piXPT2iKzWj4nA4mD9/PgUFBQwODvL000/z+OOPY7VaaWtr44YbbqC8vDzp9YUQOcBZwCcApJQBIDCFQ74OaACelFJ6h/fxOaAN+M2sZz3GczU8Hg9VVVWUlJSwYsWKERdXosIpFdEugs1mm3T1qCrq0ul0E6osx4KUkrq6OlwuF9u2bRuRzUhVViA2gh9706luitq+biaR6PmlUr+hWqk/fKmRV+sH0AmBEHDH+5eyaVFOqk4t7n7VDuCFhYV88pOfpK2tTbt2AoFJ39uLgV7g50KIDcA+4AvDuoqEIYRYAqwDLgPeEULUE0mLqiXsqixcEHFv4mJaiULNwasR4+iLRx28s2bNmrgX83h6iFiofqPqItjt9qRvRnXq1549exJWWcZDIBDg4MGD5OXlxZ06NlNFYU6nU7vpXC4X9fX1mpuSCqHReJgMEU5Vv6EoCrU9Xl6pHyDXakAnBN5gmPv/3sgvrt8wbRZWvGCmx+OhtLSUs846aypLG4gEIT8npXxLCPEA8HXgm0muk0lEP7GASIziTMBKxO3YDfxj+H0zn/VQtRGBQIADBw5w+umna6/FDt4ZSzuQCFGomYSenp4RWQ3VGknGZ+3u7sbtdnPKKadMuoWZGtMYLw07U0Vhqj9dVlbGvn37yM3N1TpOq1PC8vPzp8VNSYXFlGx8Q1EUhvwKOiHQDe/bYtDR7wkRlmCYJk9sGgVXbUCblPKt4f/vIEIUSUFKeRC4RQhxDvDmWBmOGU+PRmsjYqcqxRu8MxYm0jWorkZGRsaorEayKsujR4/i9Xqx2WyTJonW1lba2tqmFNOYLgghRrgpql4g1k3Jz89PWTYl1eQzUXwjGAySk5+BlBJ/SMGkFwx6Q6wozsCgm754zXSlR4eDjq1CiBXDna7OBw4nu44QQielVIgIrG4TQjiAQSLKzDagTko5oegqpUQxnjZCLcGOHbwzFsZrdqu231+6dGlcnUWigVBVZVlcXMzKlSt5441kM0+Rc6upqUFRlIRiGnOh1iNWL6C6KTU1NYTD4RHZlOl2UyaLWFdr7969FFjgqgqF3x31EkLHkkIbXzo3+UBiMhhLcJWixrqfA349nPFoAG6cwlr/RiQ4epCIfuI0IoSRIYT4PvDIeDUfKSMKNfUJIyeFSyk5ePAgBoNh0sHB6LVUV2O8J3cigVB1iM/q1asn3QXK6/Xi8XhYtGgRixYtSugpOheIIhrRbkpFRYWWTZkpNyUVUIPkZWVlVFZW8pHzQvQODOJxOmirP0y7EGRm53LYocerGFg1PytlAc6xYhSpIAop5QFg61SXGf6tB7ZJKbX2b0KIJ4GvAr8BdhEho7hIGVHEBish8uR3u93jDt5JFIFAgJqaGqxW64QCqolUlmpn7UTnhcaDSjRmszlhkoATRBEKhfB4PNN+8yVLSrHZlFg3JRUjBKYD0ToKg8HA/OJCKI6cg8fn545naznS40YJK+h0gmvX5/PBbWVT/vzjxcI8Hs9cnBC2BFgihOglIrYKEBFy9RIhE+94G6fU9VBvUFXP0NPTQ2ZmZsIt88fCRK7GWMcRC7UwTO1BMZkLRD23vr4+tm7dyjvvvJNU8E4IQSAQYM+ePZhMJvx+P1lZWRQUFExbz4Sp3AixborL5aK/v19LYar9LFNVtj0VjPXwqO3x0TwUYn5exvDnH+KZ2iG2FDbijRrIPBnyi6cIlVLO+mehIipI+VPgs8AGIi7HucABQDW9x027pjyYGTt455133iEcDk9K56+6Gl1dXUkFCeMRhUo2U1FZhkIhqqursVqtmpxb3VeiGZb+/n7sdjsnnXSSFjh0Op309/fT2toKoF202dnZ02JtKFKyt9nBgCdAeb6NVfMSC7xFpzArKioIh8MjyrYNBgP5+flae4CJjl1Kyd+O9vNcVRcC2L5hHucvL0j5OftDyrCuIrKu0ahHBCWrVq/BoBNT7r8RfbyzUTmbCKSUDwshaoikSiuB3VLKX4vIwV6oirDGQkqJoq+vb1SVpsFgmJQUOxgMajGARCZ9RSM6tSqlpKWlhc7OzillJFQp9uLFi0e4UYnGHKSUHDt2jMHBQa0xbCAQQKfTaYpLiJz3wMAAHR0dHDlyhIyMDI04UjEsR0rJAy818kaDHQkIATeesoiL1yRPnnq9fkQjXL/fz8DAAIFAgLfffnvCJ/VrDXYefbWZTJMeCTzySjM2o47TKvOneJYjsbIkA7NBh8MbwmrU4fSH2FaWi1EfuaZSPT9lLpLFsGy7H3iciJuhCCGKpZQ9TOB2QIqJIhQKjWoIk4xwSoX69LdYLCxbtixpDX+0ylKdYL5t27ZJm4NdXV00NDTEzdgkQhTBYFBzedasWUNdXZ22bSyMRiMlJSWUlJQgpdTK0I8cOUIwGJyyqX+s18ObjYMUZJgQQhAMK/zirTbet7JQu3EmC7PZzPz582lra9OGOEc/qWOP/bX6AcwGHVZT5DyCYYVXjw9MiijG+w7yM0x89/IVPPpqM33uAOeXF3Lz6WVx35usfiOZ45gtCCEMwJeI9KTIJSK4MgMeIq7IhEgpUcyfP38UKUQPDZ4Iajcr1dWora0lHA4n7bfrdDo8Hg/Hjx+fsHVevGOIrUPxeDyjpNjR+xovFatqR5YsWUJJSQlerzfhi0kIQWZmJpmZmZSVlY0y9dWMREFBQcJl6J5AGJ04QVIGHQQVid0TpDgrdcHJeErLWDcl6BX4gyGkxYAAQookwzQ9YuHKQhv3XLUq6e0m0m/4/X46Ozs1q8nv96dsTKIQoglwEokjhKSUk82AFAK3ACuklK6o9RN+Mkx7rUeiFkUwGKSmpgaz2ay5GpNpvQ8RN8Fut7N58+ak0lTRdRjqSMH8/Hw2bdo05k04nkWhWiLr16/XBDhTSY/GmvpqRiK6DH2ioOjiAitmox6HN4jFpOdYT2Qi+3/sOMTFa4r56LbSaTGb47kpiq2Lg53tNHf70Ot1ZFqMXLJycqnqmTL1o/UboVBI61R2+PBhbYwgRFS6qjs5RZwrpexLwTovAyEhhJFIExuGhVgJIaVEEe/LSiRGoUqflyxZMmJoTrJui6IoHDlyBJfLRVlZWdK5bJWYhoaGOHTo0IQjBSH+ja/GI5xO55iWSCoQnZFQFCVuUDQ3N3fE8eVYjfznRUv54d8bOdLtQiBZV5qFQafjhUO9VBbaUh4jiAez2cyWFeU8OK+ENxrt+Hx+VmSHcXQ0sKcluWzKZDqNpQKKomAymSgvL6e8vByfz0draysPPvggF198Maeccgr333//jB9XHAgiLscviOglPERIoyvR/hYzYlGM5XpMFGhMhii8Xi8HDx5k3rx5ZGdnJ+zuREMIQUtLy6jakYm2ibZ6ouMRExWFpTLoNVZQtLOzE5fLRU1NTcQ8zszh13vacflDBMIKmSY9Rr0OnRAYdIJjvZ4ZIQoV83MsXL1xpMYmnpuSl5dHQUFB3F6WqW5akyhixVYWi4XNmzezceNGfvOb30zqGoyBBF4UQkjg/6SUP57kOnqgkUivzDOJFIflEJGEvxEl8x4Ts+Z6xHM1Et02FmrTG1Vl2dXVlXR5bzgcxu12YzKZkgp8RkvNY+MR8RBNFNNpLqtB0cLCQjweDxUVFQwMDPDzv9ZwsCtAUaaJLJOgzx2k1xWgONNESJHMGyNOoSgKASVSaDXdGCubok4Hi80EzfRAJRXxGvqqIjogFa3/zpBStgshioG/CiGOSClfSWaB4T4TbcCtQogyoCNWqp2ICzLtroderx81WGcsVyPetuPFKKSU1NfX43A4tNmjkHwvC4/Hw8GDBzGZTKxYsSKpbIJ648eLR4z3/pm8sKODooEqLyX5AYwiTIniZ8AdpGPARTBgYs2CbM5ZNtqa+N2+Du7523ECIYnFqOOuS5Zxydr4RDgdLoCaTVGng8VmgjIzMwmHw4RCoRmZq6pirDqPFM70aB/+3SOEeAY4CUiKKKSUUghRBHwAuAn4J/B5IcQHASGl/N1ETWtgBiwKg8GA2+1WD5qWlhY6OjrYuHHjhKW4493wqsoyJyeHLVu2jGp6k2gQVLVG1qxZQ1NTU9LBUyEEzc3NBAKBhOIRs13rUZ5vo7HfQ3GWmXyTicXSxHnLc1mdBxlhJwf37xsRFD3W5+N7L9YTViKOrjeocMcfj7KiJJMlRfHTg4mQYH2vm+N9HuZnm1lfmtxclNhMUF9fH3a7nQMHDqDT6TRrIysra1oJeTo7cIvIHA6dlNIpTszk+HaSa6guxYVEmtT8fPg3RFyPM4DfEZF0z24rPNV9CAaDHDp0CKPRmHBx2Fiuh9qoZqy+D4kQRXTNh2qNJJtlCQaD9PX1kZ+fHzceEQ8zTRSx+/rQlvm02L00DUTStKdW5nLDqeWahiI2KPrnxoBGEkIHeglhBV6s7eUz4xDFvhYHv97TTigsuXRtMReuOjEa8pkDnfzwpSaEACnhqo3z+OK5iyd1fup0sKysLNatW0cgENDGH8RzU1KJae7AXQI8M/yZGYDfSCn/PMm1iojEI44QUWVCpP/mQKILzIjr4fV62bNnzyhV40SIJQpVZ9Hd3T1usHEil0UNOGZmZrJlyxYtPpIMUahKzaysrAl7a0RDJYrBwUHa29tnpJV9NOp73SwpsrG8OIMzluSzKM8yyhqLDop2Gjt56uixSAmijNzYAIFgkAF3gPyMkf0rpJQ0OhR+/tZRTa/xwEuNCAEXrirCEwhrJKHXCRQpeeZAF1esK6GycHKq2WgJvclkYt68ecybN2+UmxIIBEZkU6b6mY81oDgVRCGlbCBBMVQCqAfWExl2HB6OVawD9qu7m2iBab06pZT09vYyMDDAKaeckrRJFt1zMLrlfSLVo2O5LGrAsbKyclR8ZLweGNFQm+auX79e64idKFSNxpEjRygrK2NoaIjm5mZNEVhQUJDyilJ1rWcPdvHgP5oIKRKzQce+Fgf/78qVGPVj7+uStSU89kYb9X1ejSRMevhjVRcvVHdxWlkGt5xVRn5eHnq9Hikl7/SEUKSOLEvEDfMEwuyq6eHCVUU4vEGNJIBIDYYO+t2BlBBF7HnHuikOh0Orhp2qmzKWRZGMwG86oQYppZTPCyFswPuJaCieI2KhPBz9vvEwbUSh3tiqVn4yfpt6w6s3t9ryPpHt4t28HR0dNDU1jRlwjE11xkINng4NDWnxiGRcCbWbVigU4vTTT0dRFM11UiP7TU1NWj+DVFaU+kNh7vnrcQKhSIbA4QnyD3eQW56s4cvnV7JyuDBMSkmr3YfTF6Is30qWxcDOf9/Cw680s7fFwZAviDeoUJRpQkp4q8PPkuoO9KF62tw6FuRngFSQnLiBpJSaazPgiYyOdPpCZJh0hCUYdbpJkwQk3qo/Vp4dz02JngyWyH7Hy3rMNkSkk7cNCA4HLZ8nMsujVUoZSCSIqWJaiEKdPF5RUUFhYSEHDx6c1Dp6vZ7BwUG6u7snzCZEI5Yo1BvU7/dz0kknjWlyjud6jKWPSJQogsGg1nTXarViMBhGpHBjI/tDQ0NanEAIoVkb483EDCuSpn4PIUVSWWgbUbvxZuMgvqCCySAIhiVhCUpYodvp5zt/Osb3r1pJaY6FR19r4c+He9ELgdmo4zuXLWdpUQa3nhOJIXzyN1UM+dRJ82Aw6HmrV0dNhyQYDqI7PsgCWxgZCNMb0KHT6zEZ9Fy3bQHPV3dz91/qCctIRWcwLCnKNHL39pUUZEy+Bd9kZ3rEuikej4eBgQHq6urw+/3k5ORodR3xrplQKDSKUFLY3SoVuAQ4FbAPazF8RArAQsNds55jnGY10Ug5UbS0tNDe3q5NHlcUZVLVo+FwmJaWFtxuN6eddlpS/mR0jMLn81FVVUVRURErV66cVJ9ONR4xlrsykeuhbr906VKKi4vp7u4e9/1CiBFxAvXJp87EVK2NnNw8BnwKOgG5ViN3vXCM2k4nCEFpjoXvbV9BhjFyvp5AGKtRh2/4BgVAQnGmGYcvSE2Hkz5XgD8f7iXbokcnBE5fiHv/1sCjH1mnHVtZnpW3mwdx+0P0uYMEwwpdQ36yLQZMBgOKotDuVrjtgqX8dk8bHQ4/xeYw9tY6vvOaD0WCXqfDYtCh1wnuunQ5W8tyx/08JkIqhv8IcWIy2KJFi7QBPwMDAzQ3N2tzPKLL/8eyKObKOEGgA6gb/rd1+CebiACrBPg7aFqLmUuPKopCMBgckdWYyJyPB3XeR15eHmazOemgk+qyqNkRdfpYItvFHmt0PGIsd2W8z7inp4f6+vqEe4XGQ1jo2XHUx9tNQTLNNj6yMRcx5OL/vXicVqeCwWjAZjbR7QqRZ4u4KS12L//7ciNLCqy0twc4bb2BDLMedyCsRa4sBh1GfSSg2OsKYB92C9Qu1hlmPZ1DvhEpz0+fWc6+VgdN/ZHKZJNeJR8FvU4fsTSQ/OLtDpoHfPhDEmdA8P19YUJhiU4wPJQapNTR4/AAU1OCTseUMJUY1DaJgUAAu92ulf/bbDZCodAoNXGKOnADkUnkwF6gXUp5WbLbSyn/wYl2/OO9b2aDmXq9nqVLl464cZINEKkDetXZlY2NjUkfhxACr9dLXV3dqLL38RAdzIwXjxhrX/E+Zykjg40HBgbYunXrlLpbP/bPVv5R30+BzYQ3qPDIG92cXpnLgGKjosSAPxDgQLsbiUQvA4SkHgXB34/2016cgdMZomlPOyAwGXSEFUlYkYSkpKHPgzcY5vnqHrzBMIPeEJkmPQa9wOkLU1kwMrBamGmiNMfCkC+E1ajHoIPmAR8OX5jgMBHohKCx36NZLiFFMuQLsyDXQpczEEnaS4kOsHp6+M2LbQxKK0vm5XL+mtKkHwwzocw0mUwjyv89Hg+1tbW0t7fT0tJCTk4O/f39WlfzFOELQC0RKyBpDFeH6hiZ1VD/LRONT8AsTjOPhVpIpd6YJpMJj8eTtNui9qAIh8MJDSeOhmr9TFSvEW+baITDYaqrqyOFT1Hp18liT/MgBVYjBp3AYIpYBYc6XdjMenR6PVarlfysMK0DXhx+hUgndsgygBU/OrNAUSS+UJiVJRmEFeh2+hnyhQgpCgtyLRRkmFCkxBNw0zQQucltJj3Xbomks1vtXh77Zyu9rgDdQ370QmDUCwIhBUWRIMAdCKPXCW5YZ+WJ6tG9UG49u4Jf7emgtstFjsXIty9bztEeN7843EY47IJqJy9WtfHxdSe0D4lkgOK5ANMJ1U2xWq1UVFRgtVpxOBw8/fTT7Nu3j4997GNcfPHF3HbbbZO2LoQQC4FLgf8m0ksiaQxnM5Ivv46DlBPFZARFakl3Xl7eCJVlstWjbrebqqoqFi1ahNvtnlTDG4/Hw549e+LGI8baJvp81eK0hQsXjjltLN7nc7zXTfugj+IsM8uKMzjS7SIYliwvziDLbMAdCGPQDwuagIW5Fo72uMk0R24QoxCEhi8JdTacKwRS6FHCYVyuIWw66HZ4GfIrhKREIBjyhciyRC4DnRCEwhKDTgwTB/zP3xq4+4oV3PXCMTz+ECaDDocviNsfJqzI4cBmpLjLFwzjDYQ51BPklIpc9jQ78IUUjHpBYaaJc5YX8v7VxZorM+gJcscfjmLUCUwmA4qU7OuX/HtxOQa8IzJAKnHEs+xmq9ZD1VGobsrtt9/OSy+9xJNPPsnbb789VYHXD4l0yJ60eSKEUHX2IYZ7Wgz/VgBFShlMdK0ZsyjGkvaqcYR4Jd3JEIXqsqxdu5acnBxaWlqSPkaXy0VPTw9btmxJ2HyMJkb1XMYaAfD68X5++loT7b0+zh48yq1nl2E1GXi+upufv9mGTkTMdINO4PJHGszkWI3cfNpCfvrPNtyBMAJYVpzBZ84q55FXmjnY7sQdCJFpMqDXCU2fIKUkEJb0+0Gv6MnIzOKCNZk8+Fo7/lDkeOdlCKTOQOeQnwW5VsJhBac/RGmuhSxz5NJweIP86VAPHn+IHGvkJjUbdPSJAJeuK+blugEG3AG6nX7CikRK2Nul8L8fWsDKeZnsbx1icYGVL5y7GPNwQZl6HTj9IfQ6gS5KU2HQgV/qWVB6oqGvOsW9ra0NQKskzcrK0oh6LlSPQuRBsXDhwkkNJlYhhLgM6JFS7hORCV+TxS1EApheIiQRJNJ92xfZjfiplNI/zvYaZoQo1Bs+2vdUG+eOV9KdqBQ7XmFYMlDXUMfVJ9vsJhwOa9PPx4qJHO12cc9f6uhw+AmHJDsOdHC818nHTlrII69GekWa9Hq8gSBNDh+L8qyY9Dr63QH+XtfP/9u+kqM9keZEbzcN8pkna/CHwngDChajHlcgRFg5ESNQAINOsLIkA+egn4+eVc59u5soyTbT5wqg18FQEMqyJC6vwvHOQUJSYDPqR7XEMxl0keaaw1AUidWk59azKlg9L4tv/vGoFqhEgF7Ao6+28KtPbBr3s5uXbSbXaqDPFcBs0OEPKWSYDSwuOBEgjM4ALV68WGtN19HRgdPp1K6b2RjGPJbLkwLSOh24QghxCWABsoUQv5JSfizJdY4QiW9YicQqTEQslKLh33MrRqE2r1GJQi0xt1gs48YRJlJKjlcYlijUeERWVhaVlZV4vRP2GR0BIQRdXV3avJGxfOWjXUO0DfowGfToJEgheL3RwVvNQ4SGb3CrUYciAYSmgjTrdbTZfZTlWynLt/I/fztOVbuTggwD+9si9RqrcixYDDqGvCGGfCEUEXE/bj27nBtOWkBVlZulxZnYvUEKMkwMeUMEFYkE/Bg5Z0UeB9uHkOEwSihEc3+IPKsendBRmmfjQ5vns6/FQV2PG0VGsiWfOqMMIQTnLS/gmUU5vHo8MkHcYtShkwruwMSWoFGv44EPruE//3iUhj4PC3IsfPuy5WRbxr4so1vTqUHF+vp6Ojs76erqmtHxAbES7lRVBUspbwduBxi2KG6bBEkgpfxt9P+Hsyh6KWXSI9anJUYRi2gXYjwJdTJQRV2qNmEyiNVHdHd3J5XKDQQCNDY2YrFYWL9+/bgXic1kIKxI9EKiAC7/iS7hAIqMNJdFRv7tCYYJhMIowOr5J9KHB9uGKMgwEj1O0+0PoRdG5ueYufWcCrLMBtbOz2RRvo1gMOKGWgw6CjOMDPlClBfY6B7y4w2GuWbjPJ6t6ibXZsSoN0dK5of8bJhvZaFNYUOOj7ojtbg9fnRCaulPdf9CCG4+fRFVHc7hQhBJKAzvWzF+ZzAVi/KsPHH9xkQ/8hFQg4pZWVmUlpaSl5cXt6fodE45i9dEZzbiJfEwnPUQwCbgYiJDgJxCiEwirs3XEl1rxlyPUCikpZKSUVnGg7qOKuoaC+N9aao+IlrfkExRmNPppKqqiuLi4lHDmOPhzKUFFGeZ6XP5kWEIRxlKOiLRpVA4YlWgSPrdAQSCHIuBT5xyIiiaazXiCYTIMBvItRroGPRztNuttd5XlIgA65dvtzM/x8JV6yIScSEE37hoGf/1Qh1OXwh/SKEgw8T+tiFc/jCeQBh3IIxJr8Nm0nP5xkWctTRCUH+u7mAo0EyxBRQlDDo9v93bxlUbStDr9WxcmMNdlyzjoVea8fiDnLzAwqfOnN6Zn9FQdRRj9RRVG+FmZ2ePGxSdKqajKlhK+TKRfpeT2VYZLlH/PpE+FucDdwHXE4lTJIwZIQqdTsexY8cwGo1s27Zt0lV7iqJQW1tLKBSacB31po81P8fTRyRKFGqTmg0bNuDxeHA4HBNuYzbq+dn1m7n9mRoauyPxAEWCQR/xzQVQaAF3SCHbrKck24wQEXfiraZBLl8XCWB/+sxyvvunY7QPunAFwyNzXxK+92IDmWYDEolOCF6s7eWL6yMvVxba+Ml167l3dwMH2xwUZ5lxB8IMeoMEwxKDDtz+ME5/iLK8E3EWo9FEGB0dHklY0WEzQoYxxDvvvIPRaKSgoICTF+Zz7s2bsdvtDAwMTOsE8ViMJbiKnXI2UVA0WcQ+HLxe75ybZE8kRmGTUv6XEGK7lPIxIcRvgRchMVUmzIDr4fF46OnpYd68eRNKqMeDOnm8pKSE8vLyCddRZdzRRBEdj9i8eTMA3UM+PIEwpbnWCWMiag8LNXBqNBrHbb/v9ofodQUoyDCRZTGQYwjx6RUB3BVGyldv5oYn3sHtj6gUTXrBgoIMOocCGIVkyO3DpJMEwzo6+gYJhwvR6/XMyzZj0AsQAu9wHEAwbE0Ml4GHwgqZFgNSSjqH/NT2Gzh5+JhMBh1Hu13My7ZoWZKwIskw6yOpUT2EwwpffaaWy9eV8OGtCyjMNDHgDqIQsX4GvQqVhTls27ae/c39/OlYNyLYw8rsENk2Czqdbka7TSWizEwkKKrW0ySS1oz3nadSlZlCGIG24QKxLiHEmUAekcAmnMimj4tp/SbVlGVxcTF5eXmTIgkhhDYQeNWqVVrl30RQZdyqxRAbj5BS8ugrTfztSC96HRRkmPjyWQvGtCjUcYI2m22ECGssK2R/6yB3/fEIQUVBIPj3bUUUh3vYsGFDZAxAholtZbkc7nKhSIWSTCM2ixHpCNDrjaQN9UJHjkVPeUaYv7y2l7+2KjQOgcMv0et1hIbjhZLIt33iMzvx2SElIWXkdZBpNuAPKdhMes09K8owYTbqaOzzoCAIS3imqhtJRI1ZlGnEH5aEwhFScfqDvFTXz4P/aEIZvmlqCzL59EYbzsEBDhw4MK2l89GYjIQ7XlA0ur1eTk4OBQUFYwZF42k3UtXdKsUYIKLJ8AE/ITK5vAP47vDrs1c9qigKx44dw+VysW3bNjo6OiZVGCalJBAIUF9fn5QUG0bewPHiEW822nmxtoeiDBM6naDPFeBnb3fxkWWjL2a1p2Z5efmoMvd4AjNfMMxdzx9BErkpnR4f//tKC7+66SQtNvM/L9bT4fBRkGGkodeD0xcmyxLE5QuSZzXiDoRRJJy+JJ8NK8r4+BP76XMHUSLxTtQolbpnlQvMBoFepyMUVggrkGHSsyxv5IV+4ykLeeDlJpy+EBJYUWxjyBdmyBkgqEgyTXrybEZCiuSV+gGu27oAk0HPvJwI6XoDYWwmPY+/2UqmSY/ZGLHEmgd81DtsrMrPZ/HixWOWzufn56fU2piq4Cq2IEztW9Hf3691AVePWyW8eG7tHLUoTETKzP3AM0S6Zqnt+xOq84BpGlK8b98+CgoKtCfvZMYKqv0sADZs2JC0yk21KKJl4dHxiC6HDySa2CfLbKB10IeijOxC3d/fz5EjRzQhVyziEUW/O0AwFDH/vR4vBkBYrPR7FRYQEVUd63FRYBveJxJfQMEXDA8fu+CMpfnYPUHyM0z8+LVmepzBaCkDCiOtiEwDbC7N4HPnLeb5w3bebh6kJMvMF84uw9lxfMTxbS3P5TuXLae+14PNpGdLWTav1A/w3MFuDnU6qSzKQK8T+IJhciwmTq3M47mqbq19nkEn+MK5i7n/741kmU8U/wnAHz5h30SXzke32GtpadGaxkxUOp8IUl0UFtu3wu/309/fPyIoGq/RTSoqR4UQFiKBRzOR+3OHlPJbk1hH7Zd5FvB54DwhhE1K6QGuAG4FLhBC6KWUE96cKSeKUCjEkiVLRlRrGgwGfL7Eg6yqm1BRUTHp4S5CCA4fPkxeXl7ceo0FuRaEiPRw0OsiUuY182woSiSdqDYC7urqGteaied65NtM6HRgd7jIsBgRBiPhoEJJdiT9qBdgMYjhmwq8AQWdLqJ49AQUBj1BAsEw/qDCugXZ/ODvDQgiXaGklJpUG8CoAxB8/cLFbC6UDPS2cnZOgCvPzhseNWjmcMfo464osFERJWy6YGURp1TkcevvajjS5UJKSbbFwJfOW4zVqOf/XbmS144P4PaHWTM/i2XFGZxROcjLx/rJskRcGZNBx/ICM0KMVgZHt9irrKwcs3R+MhmJ6VZmms3mUUHRrq4unE4n+/btIy8vj8zMTBwORyosCj9wnpTSJSJTvV4TQvxJSvlmkuvkCiEuAD4M5AkhbgAsQoheIoKu+mQWSzlRZGZmjppcncz80Vg3oa+vL2lrxOVy0dfXR3l5OUuWLIn7npMq8rh03Tz+VNOFTgjm5Zj55BlldDfXoygKhw8fRko5YWFZtEXRZvfyw78fp6nXiTnkxa83EcSADEo+f94SijJNKIrC4sWLuczVzK8OuQiEdRE143Bg0WrSISXYPUEuX1vEKeVZFGeaON7nIRwTazAaIk9xo05H/YCfi9ct1kxnu91OT08Pg4ODBAIBOjo6KCgo4LUmJ0/u7SCkSE6uyGVFSQaFGSbWLsjC7gniCyrkWPRA5Hha7F42LMzBatRzwcqRjYw/dWY5FqOOPc0OSrLM3HTaInKki0ACYqvYpjGqtaFmJFRrI5EWddNRZj4W1KCo+r0vWbIEu93OK6+8wn/9139hs9lYuXIll19+eVL9YVUMuwLqfFDj8M9k8q4KEcm2F2gh0lR3PhGXo4pI3IJErAmYBcHVWJBSUldXp8U11KdKsm6LSjSFhYXjynqFENx0ejnbN8zDN/y0V0JB2kMh9u7dS3FxcULZFfWC8QXDfPMPtfQ7vehCPqTRSlmWhf9431LMBsE7zQ5+/noTJy/OZUVJMR9+Xwm58zp5/mAH3nYnmQaJxQhmk4n8TCsPXrsWKSXtg14GPEF0IpKdEIBORFwlq0mHyxfCH1bYfaSf1gEfd16yjPwME/kFBdQN6WjV2wh4OigLBtnxykGeOOQnw2TApwh+/JoDmynSpGZ9aTanVeYSViTlBTZCYUn3kJ9fvh0pTz+9Mo+imOFAZoOOT55RzifPOPG3zk5n0m6EEILs7Gyys7O1jER0i7rMzEzN2ogn0Z9JolCh1nmoQdEPfOAD+P1+jh07hsfj4dChQ5MiCtAUlPuApcBDUsq3kl1DSjkIPA08LYRYDriklJptKZL8kmZUwj0WAoGA1iYu1k1IVNsQq49obGxMaLvCzBMX/4DDjcPhYNOmTePOHO10+Hjo5Qaa+j3MzzJywYIQlgEPfQ4XZp1Cdl4uQqfD7g4Akv/+0zEcnkiM4YVDPdz+/mV0Dfn52ZvtGHSCTIsZpz9EXrYFIyEuKQ2wb98+snPz+ebLA3QOBbAYBEFFkGHS8V+XLOHR19rocQUJhCV5ViNl+VY6h/z88q02Pn/uYu7563FerO0jrCgYCBPIEgzpC8mw9mM1QGuPl7ACQ74wRj283jCgNamBSG9LBbCZ9JgNOl45PsCla4qxGCeWRU81u6FOOVN7P7hcLvr7+6mpqUFRFC1+kJ2drV0fs0EUsfv0er0sWbKEW265ZUprDz/lNwohcokEH9dKKWuSWUPVRwghFhNpiXeLEOJhKeUPhRCfBI4Du2dNRxEP41kF6tSwsWZ0JGJRxOoj1ABqMnLszs5OGhsbsdls45JEIKTwvT/XMTgcaOxy+nmiysPHOEYwrFCYnYXQRZrDKBL2NNmxewKUZFkQRJrKPrmvne4hP7lWI2aDjoIMEx0OHx/aupCLVhdjM0VctYPHO2jo96Ba8gLwhwQlORn8+Lp13L+7gXfahijJMuHwBvAEwrzTNsSe5kF27u/SUp9SkezY38XFa4oiregMRoTOhxyuMTHoBMGQpMfhoSjDSOegF3dQYtQJzqjMx2rU4/SH8ATCExJFqtWJQgiysrLIysqioqKCUCjEwMAAXV1d1NXVYbPZCAQCBAKBUS7vdGKsDtyJdFJLFFLKQSHES8BFQFJEwYmhPjcRcWWeJRIgBVgBFAC7o943LmbM9YgXo2htbaWtrS3ugOLobccjirH6WSY6VlBtmKO6PHv27Bn3/T1OP3ZPQLNEcix6mgaDZGdlcuWWPP5U063pGq7eNC+SWUFoGYpINkEZLgRT6HYGh28uSWGGCZtJr533EYfAHzohqALwBsPsqTpMxsp5nFyeRXWnmwFPiFBYIRBSsBl1/PyfLShSamXdARkhqHOWFbCn2cGAO0Bkj2DUCyQCvR4yrSY+eUoR1a12qrs8LC+0UGwO4gsEkJKErInprnUwGAwj9A9ut5uDBw9y+PBhQqGQprbMycmZVitjLKKYajBTRMb/BYdJwgpcQESCPVkUEiGEDZyIfZg5Mfxn9nQUsYi92cPhMLW1tSiKMuHUsPGIIp4+QkUiLksoFOLgwYNkZ2ezadOmhC7wDLMeZVj9KJUw/XY7QuhYvric0zJNbC3PpdPhY0G2mdIcE/f/vZG2QS9DviDzsy24A2E+sGUBNW0OfrO3QxMrWY16rd8lQK8rQEu/F7NBRyCsaN9mhknPljUrMBt8LBS9LLX5eKtbYjbqWZhr47K1JTxzsBO9iHTb1kdKRzAZBKvmZfLvpy3ihUM9LAlY2dM8hC+kYNBF+l4syrNy/voKLtpUSafDy+7DXXQ5XLjae1mVr6e73U9hYeG46cyZ7A0hhud2GI1GNm3aRCgUYnBwUBP6WSwWrfYj1VPCxtJRpKAN3nzgieE4hQ74nZTy+ckc4vDvt4C1wOXA34UQm4mUmTcns9i0EEWstiA6RqF2gFqwYAGLFi2a8OaMZxkk0s9yIqJQn0TJVrHm2Uxcs2kBv3mzCb/Pi9WWwRklIS3Qt6UsNzIZ3R/ihl8coMfpRycEva4geqHj1nMX876VhfzktSZ0ItKRWsWfDnWzbngOZyCkkGUzsCDHTLczUhXsH5Zivnh0gJtPL2fjggUsXRHksVfrIeDFPuRi39EGcgxmirKMDPnCeAMhTHrBv5+2kF6Hl4Y+N+cuy0enE5xckcfbzQ6CiqQi38onTy/TelHMz7FyzdYy3P5I926TTo6YKJ6VlUVhYeGouSOzWT1pMBgoLCyksLAwrtoyNzdXU1tOlcyiVb8qUmFRSCmriFR7TnUdOfz750KIW4g0rNkGbAfuk8PjCWUCw39gBovCFEXRpNhjdYCKB71er5VKQyQeUV1dTWZm5rj9LKOnjMVCPY5169aRnZ1c31IpJRtzfLBCT878NZF2961HOd7rxqQXzMuKXDz724YYcAc1VyLDLHEGQrxvZSF9rkgrOZNBaDLssCI1QgDIyzBi0us4qSKXvS0OWu0+jHodFQU23mi002r38j/XrCHTYuS0ZSX8dm8HbplD2B8my6hw3gJ4uz2IPtPI5evnce2WUloH/ZE6ESSKIsmz6jmlIodrtyyIe+NYjXqsUe5GdIDR6XTS19enzR1Rn9yz1ZYuFvHUloODg/T19VFfX4/ZbNaOOZFhP7GY5rmjKYWU8mEhxBtE2p2/LaV0JrvGjBCFlBK/309DQwNbt25NKuik1+s1sdZ48zViMZYl0tzcTE9PT9LHAZGL49ChQxgMBq48J6Kv6Bny8t0/eAlW1RBWFE5dnM+X3reEsW6VNruPh//RRL87gD8U6UStFnQVZZ5I/VmNenIsBt5u9GIz6jHpBWX5VnKH3ZNOh5+WAS9LizLIzzBRlGlmqc2A1agnHPDT3d3FTz6+SZv2vXfvXgI6M0NDRvIsuVjMZgacPuZlmwmHw4TD4YgYTB9puT+RdkRNZ0Ika6UqFwcHB8nIyECv18/oTNWJEFuC7vV66e/v14b9RFsbiTS8GYso5tDwHwCEEGuJlJWXEQl3fUQI8aiUcm8y60y766EWU0kp2bp1a9ImnxqjGC8eEQ+xrod6k+v1+kkdh9/v58CBA8yfP5+ysjLt7z95rZl+r8Kioki15msNA6yrzaQiP4MMsx6HJ4hOB1IKzl1WwLdfqMPtj7S694dCWuAzz2bA5T8R8O13BTjc6cJmNmAy6Gjsj4iw8mwmQKIgtVLusALZVgMLciw4nS56eroomrdAs9qiTXGOtfPG8VbCYYXSgizWVC7UTOhwOKx9ZpHZG0JrHjseTCaTJtVWxytEz1RVb1CbzZZya2Oyro7VatUaICuKwuDgoFbboZbO5+fnj3nM8YhiLg3/iZJw/w8RgdV/A0PAZcBDQogPSSkTjlNMK92rFsDixYvxer2T8gt1Oh39/f04nc5x52vE2y56WtjBgwdH3eSJYmhoiOrq6lGDhKSMjPCzGXU4HA4MBiM9QyG++cc6bCY9y4oiqkebWc9J5bmsXZDN//vLMXJtRtyBEL5gGEkkyCiRkd6UgCIlrYNeGvs9lBfYEEjmZ1toc3gZcAcIS8mWslzK8iMmc0m2GaNeR1NnH363E0v+PDaWjayyVU3xszcu59R1S/H5g3iG7PT3dNFUX0dmZiaFhYUUFBRoMSV1yptqmakNeib6HjMzM0fMVFVvQK/Xq1Vl5g0PNZ4qUhETiR5WDJHrJfaYY0cLxtNRuFyuOUMURM3vAL4b5W48JIS4lohqM2FMG1GozV1UC6CxsTHpLzUYDFJfH5FUTzRfIxaqjmJwcJBDhw4lVaIe7zw2btw4IlAlpURRFCoLI23lsm0GmvrduPxhrHrQKZKj3U62lefy0ZMXsaUsl9ou53DPCElBhpl+V5CgIvGHFAw6wQc2LSAQUvhLbQ9VbUP0uSKdrRfmWViUZyHLouekijwqi2xcsKpYm+iVbTGwLsvL37vdZOeVsGFRDluGx/S12b38/Wgf7kCIFfMyOXNJASa9DpPNTLZtpIS6r69PmxOrBgXVtHW0tREKhcZ0UWK/4+g6CXVMX39/P42NjdqTW7U2JoPpEFtZLBZKS0spLS0dNVpQtZACgcCo/fr9/ilpOYQQi4BfEBn3J4EfSykfmMq5EOm+fb8Q4lmgD1gF9AJmIYRFSplQEda0EMXx48ex2+1xpdiJ+qyqNTJ//nyczuRlwTqdjqGhIQYHB8fVacSD6jY1NDSMOA9PIMyOfe002z2sLM5gy6Ic3reyiLZBH+2DPhw+hQyznhyrER0Spy9IW28/79S4KZTzWVhQyMZF2expGiSkSHJtRlbNy2D9whzOWlrAsuJM/tkwQHO/l8oiGx0OL91DAbyBMAWZRlbOz+SMJfnkWE0Yh90ORVGoOXSIwz1h8gpL0Ot0HOtxs7wkE5Nex453OsgwG8g0G3inOdKJ67zlI4Vt0TEHtWBLvZndbjc5OTkUFhaSn5+vxX4URRkxV1a1DsZ7GMSO6VOf3PX19fh8vqTjBOr5T2c6NvaYVQtJvT5VC0l9iEzxWELAl6WU7wghsoB9Qoi/SikPJ7tQlNqylcig4nVE7vcyIqrMnUCmEGJ9ImQxLURRWlpKRUXFiAsmthP3eIiOR6gTzZOBGrT0er2cccYZSQXUhBCEQiEOHz6MyWRi8+bN6HQ6gmGF2585xJFuJwad4PmDXQgdzMu2kGMx8M1LlvPY6y3U97rxBxWEkCjoyMnJ4pS1i9DrvByrO8o2m496o6TbAzazkQyjnhVFmYTCMjLw1+Ej06LHbNCzpSyPt5vsZFoMWIx6hnxhXjk2gDcUJsdiZGGOCV9fKwuKCgiY9SzMjbgiTl+I1+oHWD0/C0WiDfiZl22mttM1iihib+7omIP6RO3r66OxsXFECtJqtY4gDCkloVBIs7Ymumlin9zRcQKTyZRQVmKm5duqhdTZ2cmGDRtwu9309/fz0Y9+lO7ubu6++24uvfRSNm7cmPTaUspOoHP4304hRC1QCiRNFFFrflb993B9hyAyAkAPmGfVorBaraMyDqo6czzTLJ4+wu/3JyXFDgaDHDx4kIyMDHJyckaRhJSS144P8HJdLxaDnqs2zmdp8Ui/ct++fZSWlrJo0SLtb8e6XRzrdZFnNeDyK3iCIUIK+IMejAYdP361mS+et5g7njvCgDtAIKxgMegpyDBhMpn44etdNA8oBMN6AsEQg94gnUMBGnrdNPcOsXlxIZ6gwl9re/EEIy7N5etLWDUvk1MW5/Fmo5152RYk8HaTnQMtgywy+cjMyUHm2tCJE6lVi1E3nH7VEY7Ss/iGu1qpCCkKbzbYOdzlwqATnFaZx8p5o4VrsVZAX18fx44dw+fzkZeXR2FhITk5OQwMDOBwOCgrK9NiG4kGRGPjBKoGoq6ujkAgMKYGYjbqPNT96vV6rXT+xRdf5NRTT6W8vJxnnnlmUkQRDSFEBRE9RdIFYWNBqhJg8Az/KeE06YzlriaSYo+lj0imelQ1B5csWUJOTg6HD48m4n8c6+fhfzRgM+oJKZKD7Q7u3r6asnwbDocDl8vFhg0bRowAkFISVpRhKbUOTyCo1V8Ewwr+kMLbzXa6naV8aMt8/nSoB50QlOZasBj13Pn8UXUl2uw+vMHImD2G06K1fX4WWLvY3Roi06TDYtDR0Ofh6Xc6+ew5i1lWnMnbTYPodYJ+dwCHx4/ic1O+eD65WZk09LnJsRpwB0JYDHp6nAHWlWZRnm+losBGU78HnRDoBFyz6USHrgOtDqrah1iQayEUlvz9aB9ZFiOluWOrGC0Wi5YtUMvZe3t7NQl1ZWUlBoMBk8k0pYCozWbDZrNp+4nWQET3t5wtooCR5QqKomAymfjYx5IevxFv3UwirsEXpZRDU14wBZi29Ggsxrvhx9NHJFqz0dPTQ319vRY8DQQCcbf7U003mcM+O0Sa6/7z+ACmwBANDQ1kZ2eP6GSlmtGLC6yRzMOgT1NI6ojUbqiFE/tbHTh9IXQ6QYHNhMMbom3Qy6A3IrxyeCPpTwnaoB9BpBHNEaeRkBLGGwIdYcxCYnd52VgoyDLrKc4y0zXkx+120zfoZPG8PPKzMyIDeYx6Lllbwuv1duyeIGsXZHFSRR5GvY4rN8yjqd9LIBQppS+M0mo09nvJzzChEwKTITLpvGvINy5RREOv11NYWIjH48Fms7Fs2TLsdjuHDh0iFApRUFBAYWGhls6ebPo1OsWqpnn7+/upra3F7/ej0+kYHBzUqklnA16vNyVt8Iab1ewEfi2lfHrKC6YIM2ZRGAyGuIVhsTd4LBLpjN3Y2Eh/fz9bt27V+hWMJeHW6yI9HwfcgeHtob+vl3adwrZt26iurkZRFAbcAfpcfvKsenKtRixGPd+/ejWPv9HCS3V9DHqChCSElEj1ZWGWmZCiEAwrGIcvVpNBh8+pIAT0Ov14ggqh4fZU0Q1vdUjaBn2EFYlBp0Ov0+MNhjEZ9Ty9r5Uycx0VWTaGfCF6hrysXFhIYZaFIV8Ypy/EaZV5LMqz8eFtowO2Rr2OZcXxL+Aci4F2h09zRwKhSL/MRCGlpKGhAbfbzaZNm9DpdGRnZ1NeXq5Veba3tzM0NBQ3/SqlJBwOa9dFomIvVXFZVlamNbuJriZVNRAzWU3qcrmm3Kp/OIbwGFArpbw/JQeWIsya65FIvcZECIfD1NTUYDKZ2LJly4gLbCyiOK0ynz8f6om02AOElBSttmjujk6n47mDXfx6Xxc6EWmO+9mzF3Py4jz0QnBKRR79zoDW2CUUVjAbdKydn0l5vg27O0B+hommfg8CWDU/i5IsMw/9owkY7i2p7puIdWE26Mg06RkihC8kMeiUyCTzkkz6pA17IJNzdH7WZHvYWGDBG3TTrYAw6TllcSFrFyQnQ1dx0uI8/nCwi06HD0WBRXkWKosSeypKKTly5AgA69atG2VFxlZ5xqZfCwoKKCoqGjP9qloaibThz8jIYOnSpVo1aX9/P4cOHdJ6VxQUFJCdnZ0ysddYrfpToKE4Hfg4UC2EODD8tzuklC9MdeGpYlZcj0TrNcaDWlym+suxGMsSaR7wMj/HEinLDviwmU20BE+0kj/SH+Shd1qwmgzohMAfDHD/7uN89KRSjnS5aej34PQGed+qItoGvDQNeCmwGbls3TxOrczjSJeLtxrtbFyYg90T5ECbg3daHeiHe2KCwOULodOBQS/QD5+6ySDIxhDptCUhrCiUF9jIsxk50txFmzWDq884FSEEwWCQvr4++vr6cLV0ccSRQ1FREfn5+UmJmHKtRj6weYE2tLgk24whAdNdURRqamrIyMigsrIyoS5g8dKvTU1NuFyuMdOvicQ2omtL1GrSzMzMEVZNR0cHR44cmbBTVqKIlwJOhdhKSvkajKn+n1XMqOsRDoeTqtcYC3a7ncOHDydVXKbCH1SwGkCPn8yibDxh8AUjTzIpJXs7g+iI9HUIKdDjDGD3Bvm/15rZWJrDvCwTXQ4ffz3cS2GmieIsE+9fXcw5yyPNbtaXZpNrNdLt9PGD3Y202L0oiiQYloSUMPOyzfiCIcxGA0WZRloGvOgE+EMKbr/CqgwjDm+IUBiq2hx4PB5ybWbKy05U2hqNxlHpy97eXo4fP47ZbKawsJCioqKESqttJr2m8EwEoVCIqqoqCgsLJ6VyhdHp16GhIXp7e+OmX1X3JDq2EU0a45W1x1o1aqes6upqILm+nNEYS749B1v1pwwz6nrY7Xba2toSrteIh7a2Ntra2ti8efOY+fWWAQ/NA146nKNdj7UF8JcqD/k5mfilwB8Mc/6KQi1oWVKQi66rhyG3B1dAEpICq8lAttlA84CHVfOy6HMFcPtDDPmDmPU6zlxyQtYthKC8wIbJoKNzyIeiKBgNenJ0RhzeIGFFkmk2YtQLDEJgMerJMOvJs5nYVm6lMMuEQQj+cawfn8eNyWTGGdah1wma+j387Ugv3kCYNQuyOWtZPoaY9KXH46Gvr29EQLGoqCglprfasnDhwoWT7gcZC51OR25urtbfNDr96vV6ycvLo6ioSDu/WLFXIBDQ5mxMFNuI7pQV25czmS7g8Yhijg7/SRlmxPWQUtLd3c3Q0BCnnnpq0vEIVQRVV1dHMBhk27ZtY5rYf6vt4Qe7jyMEuNx+XBlN3HRGhRYTKZBDfPPydTxXHYlTXHhyCRUFVoY8fmwmPWcsL6ZjKEhNp5OgP4BBByvzJN0uF3qDgabeSIl2YaaJkmwzdk+A/3utibeb7Vy4sohL1kWsJItRRzCsTv+ONMS1mfSctbSAD28rpbbLxeEOJ11OH5vLctELQSCk4AmECYeClFl86G3Z2CwWzAZBx6CPF2v7yDLrsZn0vNEwgF4HZy8b2bbPZrNRVlZGWVkZoVBIC/YNDQ2RnZ09IqCYDNR6mSVLlozbKnCqiE6/KoqC3W7XiMNisWjWhslkIhAI0NXVpZWRx1ob4yG2L2d0F3AhxLgzR95NJeapwrRbFGo8Qq/XU1RUNOkp0u+88w6FhYWsWrVqzCejPxjmgb8fx2rUYzLoUPywc38H5ywvwNlxHIslErQMhBTKCjPpdPh46Wgvbzb0oxOwfcN8tpXnYtLrONQ1xFuNg+RYDVQWZtDQ66KqzUH/kBclHKLAJPAHAnQ4/OiFoH3QFxmvh+SydfPJs5k4fUkeLx3tJ6SE0YlIDGBrRR4rSrJYUZLF2csK+Nk/WxnyhrAYdPS4AmwoMnCkqZf580oozctAyoj2IhBSkFJqKsuSbDM1Hc5RRBENg8Ew4mZQTfympiYMBgNFRUUj6jnGgtvtpqqqilWrVo3b2TzV0Ol0I0rD3W63Zi0Fg0ECgQALFy6kqKhIsyrUmEYy6dfYLuBqDKW5uVkb+BM94SxNFClGdDzCarVqMxuSgdPpxOVysXr1akpLS8d/rz803PYtcmHoRETU9Ma+Ks5ZV87ChQtxeIM88o8GWga81Pe6WZRr5uSKfIKKwh+quijLX8zGRTlsXJTDpWvnsfOdTjoGfWRZTHzrspW0D/r49dttKIpCx6CXYEiiN0i6Br0YDHqer+7hsnXzcftDfObMcnQIWu1esiwG1i/M5sylJwrT8mwmPrKtlJeO9uEJhFmfD8VKH6deuJnfH+gZdl0kG8uyqSzIoLbbpW3rD4axmZOTpqsqwqVLl+Lz+ejt7eXo0aP4/X7y8/MpKioa1WvS4XBw+PBh1q1bN+s3gpoWnT9/Pvv372f+/Pn4fD727Nkzbvo1WbFXbAxFzdi0tLSg1+s16Xp0UDNF/TJ/RqQMvEdKuXZKi6UY0+Z6xOojXC5XwkOAVKg1H7m5uQkFLfNsJvIzTAy4A+RYjfjDCjLs5bT161hYGlFaPrmnjTcb7Rj1ApcvSH1vZJJ5WYENxRNkyBfSxFi5ViP/dtoivEGF7iEv3/nTMbocftyBEMWZJmwmI65AgEyLEZ1UcPoCHO8a4KV9h3m1Q2IwGsmyGPjEqYtYWpTJ/BzzqAa1C3IsXLetlIaGBlwuL2vXbkGv13PT6WX0uQIYdYLCTBPBsORAm5U2uzfSZVwIrtgwumt5orBYLCxatEgz2wcGBujs7NSyA4WFheh0OhobG9m4ceOkukBNB9S+IEuXLtUsDTVQ2dvbOyL9WlhYqN28kxV7RU84U/ff3NzM4OAge/bs0f7ucDgoKSmZ6uk9DjxIpIJ0TmFaiEJtNDPZQT5SSo4fP87g4CDbtm2jtrY2oW31OsF/b1/Ft54/Qmu/C52UfHv7GipLT8ix9zYPYjXqyDAZsJuCuAMhupx+irLM6HWCHMvIj0QIgc2k554Xj9PrDDA/x0K/20/TwLCC0RUclmTrMOhhYUEmL7cEIeRDcQXQmcy8etTHSeUr4naxVqeSGY1G1q9frz2hTHodC3JOZC1MBsG1W0tp6PUQCCssyLGMUFlOBapbWFRUpPnrTU1N9PX1kZmZSXd3t3bTzWabO5/Px4EDB1ixYsWIB0d0oDI6/drc3IzL5SI7O1tLH6vp12hrI9HOXhApCsvNzcVkMlFWVobD4eDpp5/ml7/8Jbm5uVqB2GTiOFLKV4ZrPOYcpoUo9Ho969evH7PB7nhQO2JZrVa2bNmifXmJFoYtyrNy+ymZ9A1K9NLMpoqRjWYyzToG3AqZZpiXY+ZYTwh/UMHlD3HNxvlkxDHnFUWh2e6lJMtEWJE4vGH0ukjrOl9IweULMj/HzMJcCzlWEyaDYH5OHsiI3qOp18Hrb+9jQa5NiwuYzWYtflNQUEB5efmE52bS61g5b3rNfyEEDoeDYDDIWWedpbXSO378OB6Ph/z8fK2p7kzKpVXdzMqVKyeMk0S7DlLKMatf1fRxsmIvNUahFszddNNNHD9+nFNPPRWXy4XX603puc8FzGh6dCLXw+PxcPDgQcrKykbEIxK1RlSSsdlsnLptM++88452AahPkMvXlfDYP1uwe4IoUrJxYTZfOHcJFYVWzIb4mRSdTkeu1YjTH8Zq0KMM+6aFmSYMOkFTv2RhrpUMs54Pbp7Pi7V9OH0hsiwGpMFEUX4+Z59WAUE/vb29VFdXEw6H8fv9lJeXT1qPkGqocnin08nGjRvR6/UYDIYRpeBqEZgql1aJbyoCpongdruprq5m9erVSTdDFkJo6Vc1NhNd/Zqbm6sRnxAiIbHXWOnRxYsXc+qpp6bmpOcYZowoJqrZGBgYoLa2ljVr1ox6YiRCFF6vlwMHDowgGdUSUb98IQRnLC3EoNexr8WBxajnwlVFI6Z6j4X/OK+Su/98jH5/AF8wTFm+lfJ8GwUZITJMej64pZTFhTaKs8zk2kz8/p0OOh1+THodH9g8nwyTAUwGMjIyKCgooLq6mgULFuBwOHjzzTdH6AVmo7BJSsnRo0dRFGWECxSN6CyEKpeOJ8seb+5HsnC5XFRXV7N27dqUNK4dK/2qduaOJj5VrxEr9gqFQqNiNumsxyQRO9tjPLS0tNDZ2cmWLVviqgknIgq13V2sUlOn0xEMBjEajRE9g4hIpk9fUsDpUSKpRLCtIo9HrltPfa8bbyDMnmYHHUM+zHodnzm7giVFJy6S0lwLnz27AncgjM2kx6Q/ceP39/dz7NgxNmzYoAXaYp/UGRkZ2gU72XRyMlAUhUOHDmGxWFixYkVCN3m0XLqiomJUVyz1SZ2srDwaTqeTmpoa1q9fPy1ipnjp1/7+fg4fPkwwGNTcrKysLC22EQ6HcTqd2Gw2gsGg5oLMxQ7cqcSs9lJXFEULVG7dunXMC2q8OaIdHR00NzePUmpKKTEYDLS1tVFaWpoStp+XbWFedoTIzl5WiMsfyZCo6dhoGPU6cq0j/97e3k5HRwebN28eYarHPqnVCP7+/fu1Uu7oAqpUIhwOU1VVRX5+fkJxkrEQm1JU+0eosnKV+BKd2OVwOKitrWXDhg3Tct7xEF2VqtaJqJmgjIwMCgsLGRoawmQyUVhYOMJFaWlpmbL7JYT4LXAOUCiEaAO+JaV8LAWnNmXMGlH4/X4OHjxIcXEx5eXl4z7FxprRET0zNFppqMYjKisr6evr07op5+XlUVxcTG5u7pRNY5NBR74hsQtDLcd2uVxs3rx53CdsbARf9alVvYNq3ufk5Ez5HILBIAcOHKC0tJQFCxZMvEGCiO1WFS2USkRWbrfbOXr0KBs2bJi1tGy86te6ujrcbjdWq5XW1lYtbfrzn/8ck8mUdPwkFlLKj6To8FMOMYF7MOnR1KFQaNTN/cYbb3DKKafgdDqprq5mxYoVCaWR2tvbCQaDVFRUaGtXV1eTkZHBsmXLRlxsKkmoYhj1NVUr0Nvbi8Ph0FJmBQUFKWkbPxZUq0mv1yds1o+FcDhMf38/vb29miR7suegSrIrKyvjTpGfLqg1Fr29vTidzhGpS4PBwMDAgOaapXpe6FRw/PhxfD4fq1evJhgM0t/fz5///GfuueceFEXhnnvu4bLLLpus+zEnK0ajMaMWhU6n01yF2Pb34yF6WpgatCwvLx/1FIyW8MYGBGO1AmrFZUNDAxaLRXstldF7tdIy0fTnRNDr9SOectHnkIx5r2YRYvUIM4HYGovo1KWUkmAwyPr16+cUSagW6Zo1axBCaG6W2WymsrKSO++8k927d7NgwQLOPvvs2T7cacGMWRRSSl599VWsViubNm1Kqiipt7eXwcFBCgsLOXz48KjMiFr5qWY2kn1qu91uent76e3tjaQ9CwspLi6ekm+sPrErKipSodibEGrVaG9vL+FwmIKCAoqLi0dlIIaGhjh06FDKsgipQk9PDw0NDZSUlGC32wkEApq6MlZWPpNobGzE5XKxdu3aEZ/jjh07eOyxx9i1a1cq4l9z3qKYNqKIbnGmPlnVlmnJfrBqND0YDI6SE0+VJGLh9/s10vD7/VogMZkybTVaP9NFVCpGNLZxubTUq1pBu379+hkLECaCrq4uWltb2bhx44jxhv39/fT19eFwOLTpYwUFBTOSCQJoampiaGiItWvXjiCq5557jocffpjnn39+RH/VKSBNFKqIqqKigr6+PsrLy5MK+kgpqa6uxm63c/rpp48KWqaSJGKhlmmr/nRubu4IKXA8qOnPdevWzYn+BGoGorm5mYGBAfLz8ykpKZl2kVSi6Ojo0GZkjGVlqsHE3t5e+vv70el0IzJB0yErb2lpwW63s27duhHf9QsvvMB9993HCy+8kEq37b1NFF1dXRw9epS1a9eSk5NDbW0t8+bNS/gDVi0RtfX7hg0bThzYGEHL6YJ6w/X29jIwMBBX69DR0UF7ezsbNmyYEzehivb2djo7O1m/fj2BQIDe3l76+vo0N6uoqGhWSK2trY2enh42bNiQVDDW7/drbpbX69UqX2NnfkwWra2t9Pf3s379+hHr/fWvf+Xuu+9m165dqe7J8d4liq6uLi16rXZDrqur08zgiRAdtMzNzeXo0aNs2rQpclBRE6mO9njoHvJRkGlm3YLkWppNFqrWoaenh/7+fq2gSEqpSZ/nAqSUNDU14XA4tKlr0Yi+4Xw+35il5tOBlpYWBgYG4h5XMlBni/T19WG32zW9w2Qtpra2Nnp7e9mwYcOIz+Cll17irrvuYteuXSNmvqQI712iCAaDWoGNiuPHj5ORkTFhr0y1J6YatPT5fBw6dIjNmzePcDX+WN3NrppudMM36TnLC7l2y4IZrXBUFIXq6mptaG04HNae0qmUMicLVWcSDAZZtWrVhDd+bPo4KytLiwkk2w1rIqj1JLG+/1ShyspViwlODFtO5Ltob2/XLJzo43r11Ve54447eP7551PWAjAGc54opi09Gk9NmUjNhtoTM1rOrW4XTRJOf4g/H+5hXrYFg06gKJJXjvVz3opCirNmZp5DdPqzrKxsRIfsxsZGPB6PZkGlyixOBGrpuslkYvXq1QmRVWz6OLobltFo1NysqQigVOGZ1+tNOUnASFm52qlK/S5UWblaTxNrxXR0dNDd3T2KJN58802+/vWv88c//nG6SOJdgRnVUYxXaq4WJfl8vlE9MXU6HT6fD4/Hg9VqRQgRGQRMZPhO5D0CnU7gDyU+p3Qq8Pl8VFVVUV5ePiL9Gdshe2BgQIvVZGVlUVxcPK0iL1WSnZeXpwnUkkVsNyyv10tvby+1tbUEg0HtKZ1MJkjNuASDQU2PMN0wmUwsWLCABQsWjIgx1dfXj+i/abfb6ezsHOU27t27ly996Us899xzcUdCvJcwba6HlJJAIDDib52dnXi9XiorK0f8PRQKcfDgQbKzs1m6dGlcpWVHRwddXV2aaZ9fWMRDr3fSOeQjz2rC4QuSazXyjYuXjVkuniqo6c+VK1cmHJhVn9JqXMNsNlNcXJxSkZc6oHn+/PkTtg2cLGIzQTk5E88UUR8CwJTVqamCKitvb2/H5/OxaNEiiouLNfI7cOAAn/nMZ3jmmWdGXa/TgNn/QCbAjBJFT08PDoeDZcuWaX+LTp/GmnZq0BJOKC1V076np4deh4c3+kwMBPVUFGVz3UmlFGVOr9uRqvRntMgL0Mz+ya6p1s5UVFRMR7AtLqJnigwMDMRVuEopqa2txWAwjJLbzza6u7tpbW1lzZo1mkJ09+7dvPLKK9TV1bFz584pTyVPEHPnQxkDM0oU6pNo5cqVwIkeFGr6NHrbRPQRqiinp6cHp9M57T0dpiv9qWYfenp6tMKv6KfbRPB4PFRVVbF8+XKtEGs2EEt+BQUFWmB0yZIlc4okenp6aG5uHqUSrq6u5otf/CJr1qzh4MGD3HnnnVx++eXTfThz54MZA9NGFBC5AaLhcDhobW1l7dq1WtBy48aNI3T9kxVRRfd0sNvtKY0HqJ2fhoaGppzOmwix5DeRyEt1g9asWTPl6sVUQpWwq81eZiOoOxZUkohWgkIkfX/99dfzq1/9ivXr1wNMOFhoPKjtE0pLS3n++edHvPb444/zla98hdLSUg5GOv88KKX86aRPapoxo0Thcrm0QJLP5xt106VKaRkbD7BarRQXF0+qEUwqqz+TxUQiL7Uce66oQFUoiqIFVMvLyzWtg1qzMxtybBXq6MJNmzaN2HdjYyPXXXcdP//5z9m8eXNK9nX//fezd+9ehoaG4hLF3r17efDBB+FdYFFMa9YjtsuVlJKBgQHKysrYsGFDQuXhk91vdNTe7XbT09PD/v37tcE3xcXFmhBsLKjpT7Wpy0ybztF9HWIb2iiKolVaziWSCIfDHDx4kKKiIhYtWgSgNd8pLCwcIcdW52SocY3p7j2hpkpjSaKlpYXrrruOn/70pykjiba2Nnbt2sU3vvEN7r///pSsOZuYsfSo6kebTCaWLl064rV4QctUITq3XllZqaX6qqurkVJqpBFbJKWmP8vKyiY9TDmViG5oY7FYaG1tZdGiRdTX188ZkZeavZo/f/6YjXCip3ItWbJEG0RUW1tLIBCYVBFeIujv76ehoWGUu9He3s6HP/xhHn74YbZt25ay/X3xi1/knnvuwel0jvmenTt38sorr1BdXb0D+A8pZWvKDiDFmBGiiG6cW1tbO+K16Ma3M3GBW61WbTanWvdw9OhRraxZ1UQcOnRoVvo1TISmpibsdrvWOlAduBstLEp17UMiUFOzCxcuTIpYowcRqe3nomelpqK50MDAAPX19WzatGlEELqrq4trr72WBx54gNNPP33S68fi+eefp7i4mC1btvDyyy/Hfc/ll1/ORz7yEcxmM0KIvwJPAOel7CBSjGmNUQSDQZqbm2lvb9eClv/85z857bTTpr3yM1mo+oDW1lZt6lNpaWlK2ualAqpgye/3s3r16jEJQBV5qfGAmRB5qS31ysvLU5aajW7Mo+pOVBdlIpcxGna7nbq6OjZu3Dhiu56eHq655hq+//3v8773vS8lx6zi9ttv55e//CUGgwGfz8fQ0BBXX301v/rVr+K+XwhhAAaklCmpWZ8OTCtR1NTU4PF4WLt2rXaR/vOf/+TUU0/VulGpMxPmAtT059q1a7W4hsPh0ERFBQUFsxKxVwOqBoOB5cuXJ6WGVKXYfX190yLyCgQCHDhwgMWLF09rSz1VINXb24uiKAm5WoODgxw5coRNmzaNIIm+vj6uueYavvOd73DRRRdN2zEDvPzyy9x7772jgpmdnZ2abkgIcTXwNSnlKdN6MFPAtLoepaWlmuQ6GmrQcq6QRHT6U21+a7VateDb4OCgNks1IyNDy6CkulgqHsLhMDU1NWRlZbF48eKkPq94Qd3e3l6qqqq0+MxURF5qCjR6Duh0Qe2QXV5ePsrViqefUUki1pKw2+188IMf5M4775x2kojFnXfeydatW7niiiv43//9X/7whz+o19DngU/M6MEkiWm1KOK1w3vrrbfIzMxk3rx5c8KsVxSFI0eOIIRg5cqV4x5PdHl5X18fJpMp5U/oaKjBwZKSkpTXGqjxGbXEPFmRlzrib7bjOLH6GZVQenp62LRp0wiNjsPh4JprruG2227j6quvnrVjjoPZf1pOgBkjCjVoKaXEbrePMOtLSkpmZULWVNOfHo8nIiUf7rWpkkYq0nyqSR9bdDYdiBV55eTkUFxcPKbIS81grVq1KlWt4FICKSXd3d0cPXoUs9mspcItFgs2m40PfOAD3HrrrVx77bWzfaixSBNFKBQaM2ipCop6enqw2+1kZ2drgbfpJo1Upz/9fr9GGqFQSGvQO5kJ4OrTetmyZdNu0sdCrd/o6emJK/Jyu91UVVXNOSUonFCpqkOD1P6n3/zmN3nllVfYsmULt99+O6eeeuqsq0NjkCaKQCCQUGZDjXJ3d3czMDBAZmamFgtIdbRenWeZTPVnMoguXPN6vUmZ9eqxrV69etaf1tEiL7URjNpLYqYJbCLEkoQKr9fLtddey9VXX82iRYt44YUX+OEPf5hU5mQG8N4mii9+8YvMnz+f7du3s2jRoqSi9U6nU4sFqBLsoqKiKQcQBwYGqKurY+3atTMyVDZe7UZxcXFcV2twcJDa2to5p7aESJv/6upqSkpKGBoa0iZ+xRsJMNNQyTX2c/P5fFx33XVceeWVfOpTn0rZMY5Xw+H3+7n++uvZt28fBQUFPPXUU4n0BXlvE0VbWxtPP/00zzzzDD6fj8suu4zt27cnFb1X25t1d3dPOYDY2dlJa2vriD6eM4lYVyta42C32zl+/Picm5AFJ+aARrf5V62m3t7eWRN5AZorFEsSgUCAj3/847zvfe/j85//fEqJbLwajocffpiqqioeffRRnnzySZ555hmeeuqpiZZ8bxOFtoiU9PT08PTTT/P000/jcDi45JJL2L59e1K6AEDTN/T29o6YnDXeja+mPx0OB+vXr58TzW+jC9fUhjxLlixh3rx5M14oNR7UwrPYKt9oxBN5qXGN6fysVZJYt27dCOswGAxy4403cuqpp3LbbbellCTa2tq44YYbtBqOWKJ4//vfz1133cWpp55KKBRi3rx5WrB7HKSJIh76+vp47rnn2LlzJ93d3Vx88cVceeWVrFq1Kqkv1ev1aqQBxM06RKc/V6xYMdeCWLS0tNDX18eSJUu0m00lQDViP1vo7++nvr5+lBZhPESLvPr7+zGZTJNSVE4EteFR7MSzUCjEzTffzIYNG7jjjjtS7hJ94AMf4Pbbb8fpdMYVUq1du5Y///nPWjp7yZIlvPXWWxO195/zRDEr08wLCwu56aabuOmmmxgcHOQPf/gD3/nOd2hpaeGCCy7gqquuGjV4JR6sVivl5eWUl5drWYfDhw8TDoc1JeXx48e1cufZ1mxEQ0rJ8ePH8Xg8bNy4EZ1OR05ODosXL9YK1w4dOoSiKFMWRk0G0eXYybh4sSIvNYUcXYQ31XPxer1UVVWNIolwOMwtt9zCqlWrpoUkEqnh+FfFrFgUY2FoaIhdu3axc+dOjh07xvnnn8/27dvZsmVLUpZAIBCgs7OThoYGrdltSUnJpFKV0wEpJUeOHAGYUOSlCqN6enq0wrXi4mKysqZvhslYjV2mingiL3WOSKLnoqaOV69ePSI9Gw6H+fznP09JSQnf+973puWzSaSGI+16zDA8Hg8vvPACO3fupKamhnPOOYft27dz8sknT+j3ulwuampqWLFiBZmZmSNSlaq+YTpvtPGgKAo1NTVkZGRQWVmZ1DGohWs9PT24XC7y8/MpLi5OqcK1s7NTa/c3nbESNRvU29vL0NBQQk16fT4fBw4cGCX0UhSFL33pS2RmZnLvvffOiHs5Vg3HQw89RHV1tRbMfPrpp/nd73430XJpokgFfD4fL774Ijt27OCdd97h9NNP56qrruK0004blS4dL/0ZDofp6+uju7sbt9utPZ2TeaJNBaoSNLqpy2ShBhBVhWsqxGqJzAGdDkSLvOx2e9yOZCpJrFy5csTgZ0VR+PrXvw7A//7v/85YDCqaKKJrOHw+Hx//+MfZv38/+fn5PPnkk4l08U4TRaoRCATYvXs3O3fu5I033uCUU07hyiuv5Mwzz2T37t3k5+cnFHxTJ2P19PQwNDREXl6epm+YDtJQJdnT0QgnunBNVVMmW7jW2tqqjdKbzayQmg5XNTR6vZ68vDy6urpYvXr1KJL41re+hdPp5NFHH51zgeokkCaK6UQwGOSVV17hd7/7Hbt27SIvL49vfOMbvP/9708qwq4WFnV3d2v1J+PVOiQL1a9eunRpqofbjkK8wjW1i9dYQcnm5mbsdvuoobxzAUNDQxw8eFA79sLCQrKysigoKODuu++ms7OTxx57bE6kvKeANFHMBO688066u7v50Ic+xLPPPstLL73EunXruPLKK3nf+96XVJGW+nTu7u5OSTdvVTW4atWqEU/DmUJs4ZpKGupnMl1zQFOBQCDA/v37WbZsGfn5+QSDQfr7+/nFL37Bz3/+cywWCw8++CDnnHPOnNKeTAJpopgJ9Pb2UlhYqLkMiqLw5ptvsmPHDv72t7+xfPlyrrzySi688MKkZNuqJqC7u5v+/v6kTXqHw8Hhw4dHCYJmC2qRVE9PD8FgEJ1Oh8FgmDMitGiortqSJUtG1JVIKXnggQfYt28fN910E7t27eLaa6/lrLPOmsWjTR5qE+lhpIlitqEoCu+88w6///3v+ctf/kJFRQVXXHEFF198cVJFV6pJr0rJLRaLJoqK9zRTJ4pt2LBh2rtLJwt1xJ/L5cJoNOL1erUMykwFdsdDMBhk//79VFZWjnDVpJQ88sgjvPbaa/zud79LSQ8Qn8/HWWedhd/vJxQK8YEPfID/+q//GvGe6BkcALfeeis333zzpPYXDofR6/V0d3dTUlKizg1JE8VcgqIoVFdXs2PHDl544QVKSkrYvn07l156adITtqLjAAaDQZOSm0wmurq6aGlpYePGjdPS0GYqiDcHNDZVOV7h2nRDJYnY1npSSh577DFefPFFdu7cmTKVpxo8zczMJBgMcsYZZ/DAAw9wyiknutLFzOCYFDo7Ozl8+DDnn38+L7zwAl/72tc4/fTTOf3007n++uszpZTuVJzPdGFWlJmzBZ1Ox4YNG9iwYQPf/va3qa2tZceOHVx99dXk5uayfft2LrvssoR6P0aPAFDjAAcPHtTGKM5Vkjh8+LA2MkG1HKJrZqIL1+rq6masbgNONOmtqKgY9R088cQT7Nq1i+eeey6lUnB1nIO6/2AwOC0W1d69e/nUpz7FN7/5Td544w3uu+8+6uvrOXDgAMDNQoifSSnH7u0/y3hPWRRjQe1wvWPHDv7whz9gtVq54ooruOKKKygpKUnowpFS0tDQgMPhoKCggN7eXqSU2g042+6HoigcOnQIm82WsNArduLaRO7WVBAKhdi/fz9lZWWjOnr9+te/5sknn+SPf/zjqPkrqUA4HGbLli3U19fz2c9+lu9///sjXn/88ce5/fbbKSoqYvny5fzgBz+YlA7mhRde4Bvf+AZr167ll7/8JcFgkGeeeYZrr732PqAPeFRKOZiSk0ox0kQRAyklTU1N7Ny5k2effRadTsfll1/OlVdeyYIFC+LeYKo5ryjKiMK2QCBAT08PPT09hEIhLeMw070mVJcrOzubxYsXT3qdeJW7qShcC4VCHDhwIO5MkN///vf87Gc/Y9euXdMeEB4cHOSqq67iRz/6EWvXrtX+3t/fT2ZmJmazmf/7v//jqaee4u9///uE6wWDQV599VXOO+88amtraWtrw+fz8bGPfYxHHnmE6667DgAhxHXAycD9UsqWaTq9KSFNFONASkl7ezs7d+7kmWeeIRAIaD01KioqEEJoT2qr1TruxO5gMEhvby/d3d3aRKyZaPoSDoeprq4mPz+fsrKylK3r8/k00lCL8CZDguFwmAMHDrBgwQKtfb2KZ599lkceeYTnn39+xrp9ffvb38Zms3HbbbfFfT0cDpOfn4/D4ZhwLUVR+MQnPkFDQwPBYJB7772XM888k127dnHHHXdw22238fGPfxxACCFKpJTdqT2b1CFNFAlCbdyq9tQYGhriwgsv5JVXXuHuu+9OamZlKBTSpOTJtspLBuoc0OLi4pR38Y5GIBDQ6ml8Pl/C9TQqScQbQbhr1y5+8IMfaEK66UJvby9Go5Hc3Fy8Xi8XXnghX/va17jsssu090TP4HjmmWf4/ve/z5tvvjnmmj6fjyeeeIJPfepTtLS0cNJJJ7Fo0SL27NmjvefFF1/k1ltv5T/+4z/4zGc+k856/Kvi2LFjXHrppRQXF+Pz+bj44ovZvn170j011IxDd3d3Sgu9EpkDOh2IV7imztyIPh+VxNSJbNF48cUX+d73vscLL7ww7b05q6qquOGGG7Qu8R/60Ie48847R9Rv3H777doMjvz8fB555BFWrlw55potLS1ce+21XHrppVx22WUEAgG++tWvYrPZeOGFF4ATPT69Xi/nnXdemij+VfHUU0+RmZnJpZdeit1u5w9/+AM7d+6ktbWVCy+8kCuvvDKhnhrRUBRFu8mmkqZUsweLFi2a1QHLYxWu5ebmUlNTQ1FR0ShL56WXXuKuu+5i165dKRtPOJMIBoMYjUbefvttPve5z3HGGWdw3333AXDuuedisVh44IEHuOWWW/jpT3+q9tNME8V7DWofxZ07d1JfX8/73vc+tm/fzubNm5MmDVVKPjg4mHB1qKporKiomFM3WnSX9fb2dmw2GxUVFSNUrq+++ip33HEHu3btmhMT5JOFKqY6fvw4t9xyCxs2bOAXv/gFH/rQh7j33nsxmUx88IMfZGBggI9+9KP827/9m7ppmijey3C73bzwwgvs2LGDw4cPc+6557J9+3ZOOumkpDQJ8UYZlJSUjKo/UWsjlixZMu3FZ5OBoihUVVWRl5dHfn6+Jlj7y1/+gtfrZffu3fzlL38Z5Yq8GxAKhTAYDCiKwqc//WnmzZvHt7/9bTo6OvjkJz9JQUEBP/nJTzCZTPT29mo6kWEpd5oo0ojA5/Pxl7/8hR07drB//37OOOMMrrzyyrg9NcZDrLbBarVSUlJCZmYmNTU1LF++PGmV6UxATdHm5uZSXl4+4rVnnnmG//mf/8FisWCxWHjsscdYsmTJLB1p8nj44YfJy8vjIx/5CAD33nsvLpdLi0u0tLSwevVqPvGJT3DPPfdoWpCoeo80UaQxGn6/n927d7Njxw7eeustTj31VK2nRjJCJrX+pL29nfb2drKysli4cOG0CKKmArWrV3Z29qgZFwcOHOAzn/kMzz77LIsXL6arq4vc3NxJazMSqd2Y5OyNMaEOuLr55pv5wQ9+wIEDB3jwwQe54447tKrca665huuuu44PfvCD8ZZIE0Ua4yMYDPKPf/yDHTt28Oqrr7J161a2b9/Oueeem5BUOXoOqMFg0LQNav1JqrtfJwspJTU1NWRmZo4Se9XU1PDv//7v/P73v2f58uUp299EtRuTnL0Rd19CCILBIE6nk6985St0dnbyzDPP8Pjjj7N7925MJhPV1dXcfPPNfO5znxtrqTRRpAoVFRVkZWWh1+sxGAzs3bt3tg8p5QiFQrz22mvs2LGDl19+mfXr13PllVdy/vnnx5WAq3MtYrtRw4lRBj09PdoA5eLi4hlt/y+lHCEbj0ZtbS033ngjTz31FKtWrZqW/Xs8Hs444wweeeQRTj75ZO3vk2yAGxeKonD//fezYcMGLrjgAm699VaOHTvG008/zcDAAEePHsXr9XL55ZcDo8rLVaSJIlWoqKhg7969czJINx0Ih8NaT43du3ezYsUKradGRkYGXV1dNDY2JtTrwufzaX0o1Pb/xcXF01I3oUItQDObzSxdunTEa3V1dVx//fX8+te/Zt26dSnf90S1G5OcvTFifb1er9306uSw3/zmNwDcdtttvPXWW/z2t78dkf4dLimPt+ScJ4q51dIoDQ16vZ7TTz9d83m/+tWvcvDgQS644AK2b9/OhRdeSGlpaUL1DxaLhUWLFrFlyxatu/aRI0d46623aGhowOVypfTYpZTU1tZiMplGBSUbGxu54YYbeOKJJ6aFJCDy2R04cIC2tjbefvttampqUr5+OBzmc5/7HI8++ihf+tKXMBgMmmtx7733snHjxlEduudaB7Fk8K6xKBYvXqyp+z71qU/xyU9+crYPaVbw+uuvc9NNN3HBBRfw+uuva0OgL7300qSlzur80O7ubk16rWZQJpuxU2eW6PV6li1bNmIdVbH405/+lG3btk1q/WQRr3Zjsq5HtEVgt9u54IILaG9v55prruEjH/kIDz74IJ/73Oc47bTTkj3MOW9RvGv6Ubz22muUlpbS09PDBRdcwMqVK9917c9SAbfbzV//+lcWLVqkmfc7duzgyiuvJD8/X+upkYgZrQ5Hmj9/via9bmxs1EYZlJSUJFV/olbR6nS6USTR3t7Ohz/8YR599NFpJYnY2o2//vWvfO1rXxvxniuuuIInnniCU089lR07dnDeeecldI4qSezevZuzzz6bp59+mt///vc0NjbyzDPP8M9//hNgBFGMEZN41+FdY1FE46677iIzM3PMCr/3IqSUHDt2TOupYbPZku6poUKtP+np6cHpdCY0ykBKSV1dHVJKrXOWiq6uLj7wgQ/wwx/+cNrJPZHajUnO3gCgurqa22+/nY0bN2Kz2SgrK2PFihWsWbOGp556igMHDvDAAw8ke9hznkneFUThdrtRFIWsrCzcbjcXXHABd955JxdddNFsH9qchDq9Xe2podfrJ+ypMRZi6zVycnIoKSkZUX+iNv4JhUKjRiT29PRwzTXXcM8993D++een/FynG+o5R7sd7e3tHDlyhMcee4x//OMfLFy4kCeeeGJEoViSlkSaKFKBhoYGrrrqKiCSQrzuuuv4xje+MctH9e6AlJK2tjatp0YoFNJ6aiQ7uFlKid1u1yZ6qaMMBgcHCYVCoypn+/r6uPrqq/nud7/7riT1l156ibvvvpv/+7//o7KyknA4jE6nG3GO3/rWt/j1r3/NZz7zGb785S9Pdldpokhj7kBKSVdXl9ZTw+Vycemll7J9+/YRPTQTXcvhcFBXVzcipqH21rTb7Vx99dV885vfHNHb4d2E5uZmHn/8cQ4dOsR///d/s2zZMs2yUFOkEHF31q9fP5VdpYkijbmL3t5enn32WXbu3El/fz8XX3wxV1xxRcI9NRoaGvB4PKxevVrrSl5VVcWPf/xjhoaGuO222/jYxz42A2eSWkS7De3t7fz85z9n37593H333axatUp7PVYXMY5OYiKkieLdjMHBQW6++WZqamoQQvCzn/2MU089dbYPa1owMDCg9dRoa2vj/e9/P1deeeWYE8QaGxtxuVysXbt2BKkMDQ1x3XXXUVhYSHNzM+ecc84owdNcRryb3ePxaKKq73znO6xbty7V2Yw0UbybccMNN3DmmWdy8803EwgE8Hg8szIWcKbhcDi0nhrHjx/XRF6bNm1Cp9Nx+PBhQqHQKBJxu9188IMf5Oabb9YsCbfbPaVmwq2trVx//fV0d3cjhOCTn/wkX/jCF0a85+WXX2b79u1aLcnVV1/NnXfemfS+okniC1/4Arm5uQwODvL9738fvV7Pfffdx1tvvcXtt9/OSSedNOlzioM5TxRIKcf7ec9icHBQVlRUSEVRZvtQZhVOp1M+9dRT8oMf/KBct26dvOiii+Tll18uBwcHpdvt1n76+vrkeeedJx977LGU7r+jo0Pu27dPSinl0NCQXLZsmTx06NCI97z00kvy0ksvTdk+P/3pT8vrr79eVlVVycrKSnnhhRdKh8Mhw+Gw/OY3vym/+c1vpmxfw5joPpz1n3evpnSa0djYSFFRETfeeCObNm3i5ptvxu2e08OcpgWZmZl86EMf4ne/+x033ngj/f39ZGVlcfrpp/PlL3+ZV199FbfbzUc/+lE+9KEPceONN6Z0//Pnz9caF2dlZbFq1Sra29tTuo9oNDQ0kJOTwxNPPMF9993HNddcQ0VFBVdddRUOh4Nvf/vbfPvb3562/c9ZTMAk71ns2bNH6vV6+eabb0oppfz85z8v//M//3OWj2r2oCiKfOihh6TP55NSSunz+eTzzz8vP/GJT8j8/Hx51113Tbv11djYKBctWiQdDseIv7/00ksyPz9frl+/Xl500UWypqYm4TXD4fCovzkcDrlz5055ySWXSCmlfOWVV2RZWZn893//d+09KT7XWbcYJvpJE8UY6OzslOXl5dr/X3nlFe3CSWMknE7ntJOE0+mUmzdvljt37hz1msPhkE6nU0op5a5du+TSpUsTWjP6mB9++GH58MMPy1/+8pdSSilffPFFeeutt0qPxyP/8z//Uz700EMyFAql4EziYtaJYKKfd4XroSgK4XB4Rvc5b948Fi1apA303b17N6tXr57RY3i3YLqHGAWDQa655ho++tGPcvXVV496PTs7W6uiveSSS7Rit/Eg5YmsxUMPPcSjjz6KEII777yT7373u6xbt47GxkY+8YlP8Nxzz3HjjTei1+tRFCX1J/guwJwuCgsGg+j1+lkrz/3Rj37ERz/6UQKBAJWVlfz85z+fleN4L0NKyU033cSqVav40pe+FPc9XV1dWj3L22+/jaIoE84DUUniqaeeYu/evTz55JOsWrWKK6+8knPPPZd58+bx9NNP09PTQ2FhIRaLZYTI6j2HCUyOWYPL5ZL33Xef3Lx5s7z88svlX/7yl1HvCYfDcX3MNP518Oqrr0pArlu3Tm7YsEFu2LBB7tq1Sz7yyCPykUcekVJK+aMf/UiuXr1arl+/Xp588sny9ddfH3M91d1Qf3/lK1+RS5culU888YR0uVxSSilff/11+W//9m8jtpvm62zWXYuJfuasRdHc3Myrr77Kb37zGw4fPsyBAwfYunUr+fn5VFVVsWTJklH5+Sko49KYozjjjDMiwbRxcOutt3LrrbdOuJaMcjeOHDlCeXk599xzD2VlZTz99NOUl5dz8skn8+qrr9Ld3T3CgnivX1dzlihycnJoaWnhxRdfHNGU9I477sDn8/GPf/yDFStW8MADD2gzEtQvU/Uj341f7tGjR7n22mu1/zc0NPDtb3+bL37xi7N3UP8iUEni0Ucf5be//S3btm2jq6uLX/3qV/T29vLlL3+ZFStWEA6H+fGPfzyi3d17HhOYHLOCQCAgpYxEsz/1qU/Ju+66S/r9fvnkk0/KrKwsWVtbK4PBoLz33nvl3//+dymllF//+tflG2+8Ie12+6j1FEV5VwqnQqGQLCkpkU1NTbN9KP8y+Nvf/ibPOOMM2d/fL2+55RZ51VVXaa89+OCD8uyzz5Yvv/yylDLlKdDxMOuuxUQ/c9KiqKur44033uCjH/0on/70p/nKV77CwYMHef311zn33HP5yle+Qnd3N8FgkMHBQVatWsVTTz1FZ2cne/fuZdWqVfzqV7/S2tSrTwQ5bMK+W54Qu3fvZsmSJaMG5qQxeVgsFj796U/z7LPPcvToUXbt2gVExhl+9rOfxe12c/vtt/Poo49OtSL0XwsTMMmsoLGxUX7kIx+RmzZtkhdddJH82te+JkOhkFy3bp08fvy4lFLK5uZm+dhjj8n29nb54x//WG7fvl0ePHhQSinlJz7xCfnnP/9ZShnJq993332ytbV1xD72798vf/SjH0m/3z+zJ5cEbrzxRvmjH/1otg/jXwKHDh2SPT098tChQ7KwsFCuXbtWe+2xxx6TH/zgB7Vr4Ze//OV0aibiYdYthol+5iRRROOdd97RIs4//OEP5Ve/+lXZ3t4+4j2XXXaZfO6556Tb7ZZSSnnuuefKn/zkJ1JKKauqquR//ud/yk2bNsnvfe97Ukop/X6/fPTRR+X27dtn7kSShN/vlwUFBbKrq2u2DyWlaGlpkeecc45ctWqVXL16tfzhD3846j2KosjPfe5zcsmSJXLdunVarcdk8etf/1qedtpp8vDhw1JKKX/xi1/I0047Tf7kJz+RP/rRj+TmzZtlVVXVlPYxRcw6EUz0MyeJIhwOx2X0trY2+alPfUquWbNGbt++Xb7zzjuyqalJbt68Wf7mN7+RUkb8+srKStna2irdbrd8/PHH5VtvvSWPHj0qP//5z8uenh7Z19cnb7nlFvnTn/5U20bKyM35s5/9TD766KMzd7Jj4Nlnn5UXXHDBbB9GypFIkdeuXbvkRRddJBVFkW+88YY86aSTJr2/119/XW7cuFG2tLRIKaVsb2+X77zzjty7d6+84YYb5Fe/+lV54MABKeW0p0DHw6wTwUQ/czJGMVa2orS0lEcffRQ40VXo0UcfJSsri5qaGpqbm/nFL36hjdf75Cc/SWZmJk8//TQNDQ0IIbj33nupra2lubmZr3zlK8CJmMXg4CCvv/66VoQUDoe112Y6g/Lb3/5WG3r7rwS16zeMLPKKVr0+99xzXH/99QghOOWUUxgcHKSzs1PbLhmEw2E2bdrEq6++SmNjI7t370an03H33Xfz+OOPa+9Lp9bHx7vqk4mWcquBpl27dnHDDTdgtVo5//zz8fl8PPDAAxw4cACDwcCjjz7Kc889x3XXXcfKlSsxGo3s2bOHvLw8bTCteoG0trbS3d3NpZdeCqCpQmf6AlJb8seTK/8roampif37948Y9weRrlKLFi3S/r9w4cJJV4yuXr2ayspK7r//flatWsVDDz3ERRddREdHx4j3pUliAsy2STOVH6AU+BuwOOpv+uHfC4B/AvcCnwIagQ8Mv/YocEvM+43AzcBjw/+vBB4E/h9wUpx96wDdbH8G79YfIBPYB1wd57XngTOi/r8b2DrF/VmHf58OHIr3naZ/xv55V9OolLIduAhoE0Lohv8WHv7dAXwWcAAZQB7wJyHEWiIE86K6zPDvXGDb8Hu2AncAzwBVwNeFEMuFEDohRPHw+oqU8r1ZITRFCCGMwE7g11LKp+O8pR1YFPX/hcN/mwqCQoiTgR8B/yGlfFu8W/LkcwBzMkaRDKSUoXFe2w/sBxBC/E5K6RZCrAAUKWX98HvUm30hkYvzL8D/AmcBJUQsEj8RQvoF8N9CCCvQBTwtpfxn9D6FEPrIsu9OEhFC/AcRy0oC1cCNUkpfCtcXwGNArZTy/jHe9gfgViHEk8DJgENK2TmV/UopQ0KIWiJWZYMQQshhEyONifGutigmwrAFIACklG3Dv3cC16uvD/82AluBbsANLAbeT+Tp83EilsZRYBlwCvA2UAP8QAixZXgNdT/haJIYPoZ3BSELIUqBzxMx89cCeuDDKd7N6UQ+0/OEEAeGfy4RQnxaCPHp4fe8ADQA9cBPgFtSsWMp5ZCUsmH432mSSALvigt4shjrqS6ldMS8rgcKgCoppSKE2A+cLaV8jGEXRQiRBXwEeFlK+b/Df7uNiFsD8G9CiA8QcXXuAQ4Mr78GWC6EeEVK2SuE0Kvu0RyFAbAKIYKADeiY4P1JQUr5GhM0kx2+iT+byv2mMTX8SxNFohg2rf9f1JP/JeAbw7GKKuD3gBmoAI4ACCHWAXVArxDifcDtwBVELI7vAM1CiB8A/wE0DVsyEIl3hIAfSCkDM3F+iUJK2S6EuBdoAbzAi1LKFyfYLI33AP6lXY9kocY7pJR/ALYDh4GziTwBlxIhir3Dbz+PiHlsAU4FfiSlPEwkAJoHtAJB4ALgBiHEU0IIM3AS0KCSxFwKqAkh8oic92IiWaMMIcS7b4JPGilHmijiYDjQ1SWl/JGU8sNSyl4i8Yu3gHeG33Y+kWBfIxErQiWQMuAA8DoRt+QPwI1E4h0riLg4RUKIa4UQtjnmK78PaJRS9kopg8DTwGmzfExpzAGkiSIOpJRSRKCP+tsRKeUDUsrgcLyimggh+IhYEGr67hoiGoH9wMVAH1Az7JufAZiGf24Efj2cQZkraAFOEULYhi2d84HaWT6mNOYA0kQxBmQEWtBRzZAMv+aUUn5DSnlwOL7xEPAPIcRvgXOJZEjcwHrg4PC/IeKGPCal/KGU8iIiKchNM3NGE0NK+Rawg4jVVE3k+vjxrB5UGnMC6WBmgohNeap/E0KcC+yTUpYLIb4OuIA9RD7bISKaDb8QYjGQDfx9+GmdB6wlQipzBlLKbwHfmu3jSGNuIU0Uk0BM2jUL+OZwILCdiCjrNSllQAjRBDwuhLiGiE6jEegadm0uATqklP0zfPhppJE0JhpSnEaCEEIsAgqklAdi/i6IkMmNQC/wu2GV4B+Av6majDTSmMtIE8U0YixxlRDCQiSDcslwvUoaacxppIlihiCE0L1b6z/SSCOd9ZghxAuGppHGuwVpiyKNNNKYEOknWxpppDEh0kSRRhppTIg0UaSRRhoTIk0UaaSRxoRIE0UaaaQxIdJEkUYaaUyI/w9au/ghfNWOXwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "ax = plt.figure().gca(projection='3d')\n",
    "ax.scatter(data[:,0], data[:,1], data[:,2])\n",
    "ax.set_xlabel('sepal length')\n",
    "ax.set_ylabel('sepal width')\n",
    "ax.set_zlabel('petal length')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### RDD 생성\n",
    "\n",
    "numpy array는 바로 RDD를 생성할 수 있다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "_xRdd = spark.sparkContext.parallelize(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([5.1, 3.5, 1.4, 0.2])]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "_xRdd.take(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### 형변환\n",
    "\n",
    "앞서 보듯이 numpy.array에 생성되었으므로, array로 구성되어 있다. 이를 소수배열로 변환하자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "xRdd = _xRdd.map(lambda x: [float(i) for i in x])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[5.1, 3.5, 1.4, 0.2]]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xRdd.take(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### k-Means 모델링\n",
    "\n",
    "mllib 라이브러리의 KMeans 모델을 \n",
    "\n",
    "```python\n",
    "KMeanstrain(rdd, k, maxIterations=100, initializationMode='k-means', seed=None, initializationSteps=2, epsilon=0.0001, initialModel=None)\n",
    "```\n",
    "\n",
    "* rdd: RDD 훈련데이터\n",
    "* k: 클러스터 갯수\n",
    "* maxIterations: 최대 반복횟수 (기본 값: 100)\n",
    "* initializationMode: 초기 알고리즘 \"random\" 또는 \"k-means\" (기본 값)\n",
    "* seed: 클러스터 초기화에 필요한 무작위 seed 값 (default는 None (시스템시간에서 seed를 만드는 방식))\n",
    "* initializationSteps: k-means 단계 수 (기본 값: 2)\n",
    "* epsilon: 클러스터 중심값이 이 값이 이하로 움직이면 거리 임계값 (기본 값: 1e-4)\n",
    "* initialModel: 초기 클러스터 중심값 (기본 값: None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from pyspark.mllib.clustering import KMeans\n",
    "\n",
    "model = KMeans.train(xRdd, k = 2, maxIterations = 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "클러스터의 중심값을 출력해보자. k=2로 설정해 놓아서, 2개의 중심값이 출력된다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([5.00566038, 3.36981132, 1.56037736, 0.29056604]),\n",
       " array([6.30103093, 2.88659794, 4.95876289, 1.69587629])]"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.clusterCenters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "이 경우의 오류를 계산해보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "152.34795176035803"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.computeCost(xRdd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "from math import sqrt\n",
    "def error(point):\n",
    "    center = model.centers[model.predict(point)]\n",
    "    return sqrt(sum([x**2 for x in (point - center)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### DataFrame 생성\n",
    "\n",
    "이번에는 DataFrame으로 k-Means 클러스터링을 해보자.\n",
    "앞서 상관관계를 계산하면서 사용했듯이, url으로부터 iris를 읽어보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "file_name = \"https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Spark DataFrame은 url에서 읽어서 생성할 수 없으므로, Pandas로 먼저 읽자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "iris = pd.read_csv(file_name, names=[\"sepal_length\",\"sepal_width\",\"petal_length\", \"petal_width\",\"species\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "irisDf = spark.createDataFrame(iris)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### 통계량\n",
    "\n",
    "데이터는 모두 150개로 결측 값이 없다.\n",
    "petal_length가 다른 측정값에 비해 표준편차가 다소 크다.\n",
    "최대, 최소 값으로 추정하면, 그 측정 값이 다소 퍼져 있다는 것을 알 수 있다.\n",
    "그러나 표준편차 2배 이상인 경우는 많지 않아 보인다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-------+------------------+-------------------+------------------+------------------+--------------+\n",
      "|summary|      sepal_length|        sepal_width|      petal_length|       petal_width|       species|\n",
      "+-------+------------------+-------------------+------------------+------------------+--------------+\n",
      "|  count|               150|                150|               150|               150|           150|\n",
      "|   mean| 5.843333333333335| 3.0540000000000007|3.7586666666666693|1.1986666666666672|          null|\n",
      "| stddev|0.8280661279778637|0.43359431136217375| 1.764420419952262|0.7631607417008414|          null|\n",
      "|    min|               4.3|                2.0|               1.0|               0.1|   Iris-setosa|\n",
      "|    max|               7.9|                4.4|               6.9|               2.5|Iris-virginica|\n",
      "+-------+------------------+-------------------+------------------+------------------+--------------+\n",
      "\n"
     ]
    }
   ],
   "source": [
    "irisDf.describe().show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### features 생성\n",
    "\n",
    "Vector assembler는 변수를 묶어서 하나의 Vector로 만들어 준다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#from pyspark.ml.linalg import Vectors\n",
    "from pyspark.ml.feature import VectorAssembler\n",
    "\n",
    "va = VectorAssembler(inputCols=[\"sepal_length\", \"sepal_width\", \"petal_length\", \"petal_width\"], outputCol=\"features\")\n",
    "xxDf = va.transform(irisDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "features 컬럼이 vector로 만들어진 것을 확인해보자."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      " |-- sepal_length: double (nullable = true)\n",
      " |-- sepal_width: double (nullable = true)\n",
      " |-- petal_length: double (nullable = true)\n",
      " |-- petal_width: double (nullable = true)\n",
      " |-- species: string (nullable = true)\n",
      " |-- features: vector (nullable = true)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "xxDf.printSchema()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+------------+-----------+------------+-----------+-----------+-----------------+\n",
      "|sepal_length|sepal_width|petal_length|petal_width|    species|         features|\n",
      "+------------+-----------+------------+-----------+-----------+-----------------+\n",
      "|         5.1|        3.5|         1.4|        0.2|Iris-setosa|[5.1,3.5,1.4,0.2]|\n",
      "|         4.9|        3.0|         1.4|        0.2|Iris-setosa|[4.9,3.0,1.4,0.2]|\n",
      "|         4.7|        3.2|         1.3|        0.2|Iris-setosa|[4.7,3.2,1.3,0.2]|\n",
      "+------------+-----------+------------+-----------+-----------+-----------------+\n",
      "only showing top 3 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "xxDf.show(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "입력데이터의 타입이 명목일수도 아닐 수도 있는 모르는 경우, Vector Indexer는 이 가운데 범주형 변수를 가려내어 Vector로 만든다. 참고로 String Indexer는 명목변수 컬럼을 인덱스 값으로 변환한다. 요일을 1~7 값으로 변환하는 경우를 예로 들 수 있다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from pyspark.ml.feature import VectorIndexer\n",
    "\n",
    "featureIndexer = VectorIndexer(inputCol=\"features\", \\\n",
    "                               outputCol=\"featuresIndexed\",\\\n",
    "                               maxCategories=4).fit(xxDf)\n",
    "xxxDf = featureIndexer.transform(xxDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      " |-- sepal_length: double (nullable = true)\n",
      " |-- sepal_width: double (nullable = true)\n",
      " |-- petal_length: double (nullable = true)\n",
      " |-- petal_width: double (nullable = true)\n",
      " |-- species: string (nullable = true)\n",
      " |-- features: vector (nullable = true)\n",
      " |-- featuresIndexed: vector (nullable = true)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "xxxDf.printSchema()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+------------+-----------+------------+-----------+-----------+-----------------+-----------------+\n",
      "|sepal_length|sepal_width|petal_length|petal_width|    species|         features|  featuresIndexed|\n",
      "+------------+-----------+------------+-----------+-----------+-----------------+-----------------+\n",
      "|         5.1|        3.5|         1.4|        0.2|Iris-setosa|[5.1,3.5,1.4,0.2]|[5.1,3.5,1.4,0.2]|\n",
      "|         4.9|        3.0|         1.4|        0.2|Iris-setosa|[4.9,3.0,1.4,0.2]|[4.9,3.0,1.4,0.2]|\n",
      "|         4.7|        3.2|         1.3|        0.2|Iris-setosa|[4.7,3.2,1.3,0.2]|[4.7,3.2,1.3,0.2]|\n",
      "|         4.6|        3.1|         1.5|        0.2|Iris-setosa|[4.6,3.1,1.5,0.2]|[4.6,3.1,1.5,0.2]|\n",
      "|         5.0|        3.6|         1.4|        0.2|Iris-setosa|[5.0,3.6,1.4,0.2]|[5.0,3.6,1.4,0.2]|\n",
      "+------------+-----------+------------+-----------+-----------+-----------------+-----------------+\n",
      "only showing top 5 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "xxxDf.show(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### k-Means\n",
    "\n",
    "features, featuresIndexed 어느 것으로 해도 결과는 동일하다.\n",
    "\n",
    "kMeans 모델은 앞서 RDD에서와 같이 초기 값을 설정할 수 있다.\n",
    "```python\n",
    "(featuresCol='features', predictionCol='prediction', k=2, initMode='k-means', initSteps=2, tol=0.0001, maxIter=20, seed=None, distanceMeasure='euclidean', weightCol=None)\n",
    "```\n",
    "* maxIter: 클러스터를 분할하기 위한 최대 반복 횟수 (default: 20)\n",
    "\n",
    "\n",
    "set함수를 이용해서 다음과 같이 할 수 있다.\n",
    "```python\n",
    "kmeans = KMeans()\\\n",
    "        .setK(k)\\\n",
    "        .setSeed(1) \\\n",
    "        .setFeaturesCol(\"indexedFeatures\")\\\n",
    "        .setPredictionCol(\"cluster\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from pyspark.ml.clustering import KMeans\n",
    "\n",
    "kmeans = KMeans(k=2) \\\n",
    "            .setFeaturesCol(\"features\") \\\n",
    "            .setPredictionCol(\"cluster\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = kmeans.fit(xxDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[array([6.30103093, 2.88659794, 4.95876289, 1.69587629]), array([5.00566038, 3.36037736, 1.56226415, 0.28867925])]\n"
     ]
    }
   ],
   "source": [
    "print (model.clusterCenters())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+------------+-----------+------------+-----------+-----------+-----------------+-------+\n",
      "|sepal_length|sepal_width|petal_length|petal_width|    species|         features|cluster|\n",
      "+------------+-----------+------------+-----------+-----------+-----------------+-------+\n",
      "|         5.1|        3.5|         1.4|        0.2|Iris-setosa|[5.1,3.5,1.4,0.2]|      1|\n",
      "|         4.9|        3.0|         1.4|        0.2|Iris-setosa|[4.9,3.0,1.4,0.2]|      1|\n",
      "|         4.7|        3.2|         1.3|        0.2|Iris-setosa|[4.7,3.2,1.3,0.2]|      1|\n",
      "|         4.6|        3.1|         1.5|        0.2|Iris-setosa|[4.6,3.1,1.5,0.2]|      1|\n",
      "|         5.0|        3.6|         1.4|        0.2|Iris-setosa|[5.0,3.6,1.4,0.2]|      1|\n",
      "|         5.4|        3.9|         1.7|        0.4|Iris-setosa|[5.4,3.9,1.7,0.4]|      1|\n",
      "|         4.6|        3.4|         1.4|        0.3|Iris-setosa|[4.6,3.4,1.4,0.3]|      1|\n",
      "|         5.0|        3.4|         1.5|        0.2|Iris-setosa|[5.0,3.4,1.5,0.2]|      1|\n",
      "|         4.4|        2.9|         1.4|        0.2|Iris-setosa|[4.4,2.9,1.4,0.2]|      1|\n",
      "|         4.9|        3.1|         1.5|        0.1|Iris-setosa|[4.9,3.1,1.5,0.1]|      1|\n",
      "|         5.4|        3.7|         1.5|        0.2|Iris-setosa|[5.4,3.7,1.5,0.2]|      1|\n",
      "|         4.8|        3.4|         1.6|        0.2|Iris-setosa|[4.8,3.4,1.6,0.2]|      1|\n",
      "|         4.8|        3.0|         1.4|        0.1|Iris-setosa|[4.8,3.0,1.4,0.1]|      1|\n",
      "|         4.3|        3.0|         1.1|        0.1|Iris-setosa|[4.3,3.0,1.1,0.1]|      1|\n",
      "|         5.8|        4.0|         1.2|        0.2|Iris-setosa|[5.8,4.0,1.2,0.2]|      1|\n",
      "|         5.7|        4.4|         1.5|        0.4|Iris-setosa|[5.7,4.4,1.5,0.4]|      1|\n",
      "|         5.4|        3.9|         1.3|        0.4|Iris-setosa|[5.4,3.9,1.3,0.4]|      1|\n",
      "|         5.1|        3.5|         1.4|        0.3|Iris-setosa|[5.1,3.5,1.4,0.3]|      1|\n",
      "|         5.7|        3.8|         1.7|        0.3|Iris-setosa|[5.7,3.8,1.7,0.3]|      1|\n",
      "|         5.1|        3.8|         1.5|        0.3|Iris-setosa|[5.1,3.8,1.5,0.3]|      1|\n",
      "+------------+-----------+------------+-----------+-----------+-----------------+-------+\n",
      "only showing top 20 rows\n",
      "\n"
     ]
    }
   ],
   "source": [
    "model.transform(xxDf).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 최적의 클러스터 갯수\n",
    "\n",
    "클러스터 분석에 있어, 최적의 갯수를 결정할 필요가 있는데, elbow, silhouette 그리고 gap statistic 방법이 있다.\n",
    "elbow 방법은 클러스터의 개수를 결정하는 직관적 방법이다.\n",
    "오류를 계산하고, 그래프를 그려서 클러스터 개수를 늘려도 그 오류의 기울기가 줄어드는 지점,\n",
    "즉 추세가 꺽이는, 팔꿈치가 접히는 지점이 적당한 개수로 인식하게 된다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Silhouette은 계수로 클러스터링 적합도를 측정한다. 계수는 -1 ~ 1 값을 가진다.\n",
    "* 1: 클러스터가 서로 잘 분리되어 있다는 의미\n",
    "* 0: 클러스터가 서로 차이가 없다는 의미. 클러스터 간의 거리가 유의미하지 않다\n",
    "* -1: 클러스터에 할당된 데이터가 올바르지 못하다는 의미."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "...2 Within Set Sum of Squared Errors 0.8501515983265806\n",
      "...3 Within Set Sum of Squared Errors 0.7354567373091194\n",
      "...4 Within Set Sum of Squared Errors 0.6720731409257744\n",
      "...5 Within Set Sum of Squared Errors 0.6709352176230783\n",
      "...6 Within Set Sum of Squared Errors 0.6451259820011412\n",
      "...7 Within Set Sum of Squared Errors 0.5962165727283095\n",
      "...8 Within Set Sum of Squared Errors 0.6221985662569978\n",
      "...9 Within Set Sum of Squared Errors 0.5003287681787887\n",
      "...10 Within Set Sum of Squared Errors 0.49199492123901184\n",
      "...11 Within Set Sum of Squared Errors 0.47438014361110636\n",
      "...12 Within Set Sum of Squared Errors 0.4321528971979168\n",
      "...13 Within Set Sum of Squared Errors 0.4117501336179155\n",
      "...14 Within Set Sum of Squared Errors 0.3965248682036035\n",
      "...15 Within Set Sum of Squared Errors 0.4472034252455173\n",
      "...16 Within Set Sum of Squared Errors 0.4129720796151931\n"
     ]
    }
   ],
   "source": [
    "from pyspark.ml.evaluation import ClusteringEvaluator\n",
    "#silhouette = np.zeros(15)\n",
    "silhouetteList = list()\n",
    "for i in range(2, 17):\n",
    "    # Trains a k-means model.\n",
    "    kmeans = KMeans().setK(i).setFeaturesCol(\"features\")\n",
    "    model = kmeans.fit(xxDf)\n",
    "    # Evaluate clustering by computing Within Set Sum of Squared Errors\n",
    "    predictions = model.transform(xxxDf)\n",
    "    # Evaluate clustering by computing Silhouette score\n",
    "    evaluator = ClusteringEvaluator().setMetricName(\"silhouette\")\n",
    "    silhouette = evaluator.evaluate(predictions)\n",
    "    #silh_val.append(silhouette)\n",
    "    print(\"...{} Within Set Sum of Squared Errors {}\".format(i, silhouette))\n",
    "    silhouetteList.append(silhouette)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 클러스터 갯수에 따른 오류의 그래프"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그래프를 읽을 때, 클러스터의 갯수가 2개부터 시작한다는 점에 유의하자.\n",
    "클러스터의 갯수가 2일 경우, 계수가 제일 높다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f70cf483b38>]"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAeSElEQVR4nO3deXSW1dX+8e9OAlJBRCSCAgooDohTjUCdFaSAFETQgjjghDgBcUB8a9Vi7a/VSsWpFK04QEFeXlREBSdqRVEJThUEpFgEpBIVCxZlkPP7Y4cSYoYnIcl5huuzVlae4fa5Ny64cufc5+xjIQRERCT1ZcUuQEREqocCXUQkTSjQRUTShAJdRCRNKNBFRNJETqwTN2nSJLRq1SrW6UVEUtL8+fO/CCHklvZetEBv1aoVBQUFsU4vIpKSzGx5We9pyEVEJE0o0EVE0oQCXUQkTSjQRUTShAJdRCRNKNBFRNKEAl1EJE2kXKDPnw833gjq+isisqOUC/Q334Tf/hbmzIldiYhIckm5QL/wQsjN9VAXEZHtUi7Qd90Vhg2D556DDz6IXY2ISPJIuUAHuOIKaNAAfve72JWIiCSPlAz0PfaAIUNg8mRYtix2NSIiySElAx0gPx9ycuCuu2JXIiKSHFI20PfZB84/Hx5+GNasiV2NiEh8KRvoANdfDxs3wj33xK5ERCS+lA70Aw+Evn3h/vth3brY1YiIxJXSgQ5www3w9dcwblzsSkRE4kr5QM/Lgy5dYPRoH34REclUKR/oACNHwurV8PjjsSsREYknLQL91FPh6KPhjjvg++9jVyMiEkdaBLqZX6V//DE8+WTsakRE4kiLQAfo0wfatvWmXWqtKyKZKG0CPTsbRozwfukvvxy7GhGR2pc2gQ5w3nmw995qrSsimSmtAn2XXeCaa/wKfd682NWIiNSutAp0gMGDoVEjtdYVkcyTUKCbWTczW2xmS81sZCnv72tms83sXTP7wMx6VH+piWnYEK68EqZNg8WLY1UhIlL7Kgx0M8sG7ge6A+2AAWbWrsRhNwFTQghHAf2BB6q70MoYOtSHX+68M2YVIiK1K5Er9A7A0hDCshDCJmAy0LvEMQFoWPR4d+Cz6iux8vbaCy6+GB57DFatilmJiEjtSSTQmwMrij1fWfRacbcC55rZSuA54OrSPsjMBptZgZkVFBYWVqHcxF17LWzdCnffXaOnERFJGtV1U3QA8EgIoQXQA3jczH7w2SGEcSGEvBBCXm5ubjWdunStW0P//jB2LKxdW6OnEhFJCokE+iqgZbHnLYpeK+5iYApACGEuUA9oUh0F7owRI+Cbb+CBqCP6IiK1I5FAnwe0NbPWZlYXv+k5vcQxnwKdAczsEDzQa3ZMJQGHHw49esCYMbBhQ+xqRERqVoWBHkLYAlwFzAI+wmezLDCzUWbWq+iwa4FLzex9YBIwKITk6KgyciQUFsL48bErERGpWRYrd/Py8kJBQUGNnycEOP54n+3y8cdQp06Nn1JEpMaY2fwQQl5p76XdStGStrXWXb4cpkyJXY2ISM1J+0AHOP10OPRQtdYVkfSWEYGeleWbSX/4ITz3XOxqRERqRkYEOvic9H33VWtdEUlfGRPoderAddfBnDn+JSKSbjIm0MH7uzRpota6IpKeMirQd93VOzHOmAF//3vsakREqldGBTp4r/T69eGOO2JXIiJSvTIu0Bs39l2NJk3yuekiIuki4wIdfN/RrCy4667YlYiIVJ+MDPQWLeDcc+Ghh7zPi4hIOsjIQAe4/nr47ju4997YlYiIVI+MDfRDDoEzzoD77oP162NXIyKy8zI20MHbAaxdCw8+GLsSEZGdl9GB3rEjnHKK3xzduDF2NSIiOyejAx28te5nn8HEibErERHZORkf6KedBkcd5QuNvv8+djUiIlWX8YG+bQOMxYvh6adjVyMiUnUZH+gAffvC/vtrAwwRSW0KdCA7G0aMgHnzYPbs2NWIiFSNAr3I+edD06baAENEUpcCvUi9epCfDy++6FfqIiKpRoFezJAhkJsLAwfCV1/FrkZEpHIU6MXsvjs8+aS31T3zTNi0KXZFIiKJU6CXcNxx8PDD8OqrfsWuWS8ikipyYheQjAYOhCVLYNQoOOgg7/kiIpLsFOhluPVWD/WRI6FtWx+CERFJZhpyKYMZjB8PnTr5ZhgFBbErEhEpnwK9HPXqwVNPwV57Qa9esHJl7IpERMqmQK9A06YwYwZ88w387Gf+XUQkGSnQE9C+PUyZAh98AOeco66MIpKcFOgJ6tYN7rkHnnnG+76IiCQbzXKphCuv9Da7o0f7dMbBg2NXJCKynQK9kkaPhqVL4YoroE0b6NIldkUiIk5DLpWUkwOTJ8Mhh0C/fvDRR7ErEhFxCQW6mXUzs8VmttTMRpby/h/M7L2iryVm9nW1V5pEGjb0mS/16kHPnvDFF7ErEhFJINDNLBu4H+gOtAMGmFm74seEEPJDCEeGEI4E7gWm1UCtSWW//XzLus8+gz59YOPG2BWJSKZL5Aq9A7A0hLAshLAJmAz0Luf4AcCk6igu2XXsCI8+CnPmwCWXqJGXiMSVSKA3B1YUe76y6LUfMLP9gNbAK2W8P9jMCsysoLCwsLK1JqWzz4bbboMJE+D222NXIyKZrLpvivYHpoYQSl16E0IYF0LICyHk5ebmVvOp4/nFL+C88+CXv/QFSCIiMSQS6KuAlsWetyh6rTT9yZDhluLM4MEH4fjj4YIL4K23YlckIpkokUCfB7Q1s9ZmVhcP7eklDzKzg4E9gLnVW2Jq2GUX3+1on328kdfy5bErEpFMU2GghxC2AFcBs4CPgCkhhAVmNsrMehU7tD8wOYTMvTXYpAk8+6zPeOnZE9ati12RiGQSi5W/eXl5oSBNm4y/9JL3funaFaZP98VIIiLVwczmhxDySntPK0VrQJcu8MAD8PzzcM01sasRkUyha8caMnjwjo28rrwydkUiku4U6DXojju8kdfQobD//j4MIyJSUzTkUoOys2HiRDj8cF+A9OGHsSsSkXSmQK9hDRr4phgNGvjMl88/j12RiKQrDbnUghYtPNRPOAEOPBAaNYI6daBuXf9e/HHJ74m+Vreu/9Do3dunT4pI5lGg15Kjj/ZZLxMnwubNsGnTjt+3Pd6woez3Sn4vbW/TYcPgssvg2mt9kZOIZA7NQ09hW7fCli3bQ375crjrLpg0ycfvL7oIbrgBWrWKXamIVBfNQ09TWVnbh1r22AOOPBIefxyWLIFBg+Dhh+GAA7y/zKJFsasVkZqmQE9DbdrAn/4Ey5bB1VfD//4vtGsHZ50F770XuzoRqSkK9DTWvDn84Q8+FHPjjfDCC3DUUXD66fDGG7GrE5HqpkDPALm5vvnG8uXw6197e9/jjoNTTvG+M5nbTk0kvSjQM0ijRr4Zx/Ll3pJgyRI47TTo1MmbiCnYRVKbAj0D1a8P+fk+xj52LBQW+vz1I46AyZNLnw4pIslPgZ7BdtnF56wvWQKPPeZTIAcMgEMO8RkymzbFrlBEKkOBLuTk+J6oH34IU6f6NMiLL/Ypj/fdB99+G7tCEUmEAl3+KysL+vaF+fPhuedg33192mPr1t45cv362BWKSHkU6PIDZtC9O7z2Gvz1r94t8oYbIC9Pe6WKJDMFupTJDE46yeevv/yyd4o89li1ARZJVgp0Scipp/oVewjeNfL112NXJCIlKdAlYYcd5itM99rL902dMSN2RSJSnAJdKqVVK5gzB9q3hzPOgEceiVyQiPyXAl0qLTcXXnnFWwdceKHPgNEqU5H4FOhSJbvtBs8+C/37+wyY667z/uwiEo92LJIqq1vXd2DKzfXeMGvW+ArTOnViVyaSmRToslOysmDMGGjaFG66Cb780vuv168fuzKRzKMhF9lpZt7Fcdw4mDXLZ8B8+WXsqrb78kvvUyOS7hToUm0uvdR7wbz7rs9VX7Eibj0LFsDAgT7N8qab4tYiUhsU6FKt+vTxq/RVq3xV6cKFtV/D/Plw5pk+tXL6dDjwQHjgAVi3rvZrEalNCnSpdiedBH/7mw9znHACzJ1bO+d94w3o0cN7zsyeDTffDP/8p7cGXr8exo+vnTpEYlGgS4044ghvD9C4MXTuDM8/XzPnCcH7zJx6qm+rN28e/OY3HuS/+hXsuSccc4y/N2aMNu+Q9KZAlxrTpo2vKj34YOjVCyZMqL7PDsHnwR97rN+EXbTIN8T+5z99Q+zdd9/x+Px8+OQTH4IRSVcKdKlRTZt6C94TT/RNNEaP3rnP27rVb7z++MfQsyesXg1//KNvpzd8eNnTJXv3hv3289AXSVcKdKlxDRv6hhlnnQXXXgsjRlS+VcCWLX6F3769f86GDd5H5uOPYcgQqFev/P8+JweGDvWOkfPnV/mPIpLUEgp0M+tmZovNbKmZjSzjmLPNbKGZLTCzv1RvmZLqdtkFJk2CK66AO++Eiy5KbG74pk3w0ENw0EF+hZ+d7RtZL1wIF1xQuVWpF1/s2+vpKl3SVYWBbmbZwP1Ad6AdMMDM2pU4pi1wI3BcCOFQYHj1lyqpLjvb9yi99Va/uu7Tx6+0S/Ptt3DvvbD//j6/vXFjeOopeP99+PnP/bMqa/fdPdSfeMKnVYqkm0Su0DsAS0MIy0IIm4DJQO8Sx1wK3B9CWAsQQlhTvWVKujCDW27xce9nn4WuXWHt2u3vr1/vV/CtW/sQSatWPq/97bd9HDxrJwcJhw71mS4PPLBznyOSjBL559EcKL7mb2XRa8UdCBxoZq+b2Ztm1q20DzKzwWZWYGYFhYWFVatY0sKQId7zZd48n6u+YAHcdpsH+IgRvo/pq6/6mHfXrv6DoDq0aeN93MeOLfu3A5FUVV03RXOAtsDJwADgQTNrVPKgEMK4EEJeCCEvNze3mk4tqapvX5g5Ez791G923nwzHH88vPWW72N64ok1c978fPjqK3j88Zr5fJFYEgn0VUDLYs9bFL1W3EpgeghhcwjhE2AJHvAi5TrlFF9VOnQovPcePP00dOhQs+c8/ng4+mi4+271cJf0kkigzwPamllrM6sL9AdKLs94Cr86x8ya4EMwy6qvTElnRx7pqziPOKJ2zmfmc9YXLfLxeZF0UWGghxC2AFcBs4CPgCkhhAVmNsrMehUdNgv40swWArOB60MISdRAVWRHZ58Ne++tKYySXixE2gwyLy8vFBQURDm3CHjPl1/8Av7+dx/DF0kFZjY/hJBX2ntaKSoZ67LL4Ec/8rF0kXSgQJeMteeecP753lJAs2glHSjQJaMNHw4bN/q8dJFUp0CXjHbwwdC9O9x/vwe7SCpToEvGy8+Hzz/3pl8iqUyBLhmvSxc49FCfwhhp0pdItVCgS8bbttDo/fd9Mw6RVKVAFwEGDoQmTbTQSFKbAl0En49++eUwY4bvgiSSihToIkWuuMJ3QLrnntiViFSNAl2kSLNmMGAAjB8PX38duxqRylOgixSTnw//+Q88+GDsSkQqT4EuUswRR3iP9nvvTWwTa5FkokAXKWH4cFixAv7v/2JXIlI5CnSREnr2hAMO0BRGST0KdJESsrJg2DDf23Tu3NjViCROgS5SikGDoFEjXaVLalGgi5SiQQO49FIfR1++PHY1IolRoIuU4eqrvc/LfffFrkQkMQp0kTK0bAn9+vmc9G++iV2NSMUU6CLlyM+Hf//bV4+KJDsFukg5OnaETp1gzBj4/vvY1YiUT4EuUoH8fPjHP7wTo0gyU6CLVODMM2HffTWFUZKfAl2kAjk5PuPl1Vfh3XdjVyNSNgW6SAIuuQTq14e7745diUjZFOgiCWjUCC66CCZNgtWrY1cjUjoFukiChg3zlroPPBC7EpHSKdBFErT//tCrF4wdC99+G7sakR9SoItUwvDh8MUXMGFC7EpEfkiBLlIJJ50ERx7pN0dDiF2NyI4U6CKVYOYLjRYuhBdeiF2NyI4U6CKV1L8/NGumhUaSfBToIpVUty5ceSXMmuVX6iLJQoEuUgVDhkC9et60SyRZJBToZtbNzBab2VIzG1nK+4PMrNDM3iv6uqT6SxVJHk2awHnnwWOP+awXkWRQYaCbWTZwP9AdaAcMMLN2pRz6RAjhyKKvh6q5TpGkM3w4fPcd/OlPsSsRcTkJHNMBWBpCWAZgZpOB3oBGDyWjtWsHXbvCTTf5rkYHH+xfBx20/XGzZj4zRqQ2JBLozYEVxZ6vBDqWclxfMzsRWALkhxBWlDzAzAYDgwH23XffylcrkmQefRTGjYNFi2DxYnjtNdiwYfv7DRvuGPDbHh9wAOyyS7y6JT1ZqGB1hJn1A7qFEC4pen4e0DGEcFWxY/YEvgkhbDSzy4CfhxBOLe9z8/LyQkFBwU7/AUSSydatsGqVh/uiRdu/Fi+GlSu3H5eVBW3a7Bj22wK/SRNd1UvZzGx+CCGvtPcSuUJfBbQs9rxF0Wv/FUL4stjTh4A7KlukSDrIyvLNpVu2hC5ddnxv/XpYsmTHsF+8GF56CTZu3H5c48bbw/2QQ+CnP4XDDlPIS8USuULPwYdROuNBPg84J4SwoNgxe4cQVhc97gPcEELoVN7n6gpdxH3/PXz66faALx72//qXH9O2LfTr519HHaVwz2TlXaFXGOhFH9ADuBvIBh4OIdxuZqOAghDCdDP7f0AvYAvwFXB5CGFReZ+pQBep2Oefw9NPw9Sp8MorHv6tW28P92OOUbhnmp0O9JqgQBepnC+/3B7uL70Emzf70E6/ftC3L/zkJz7kI+lNgS6SZtauhWee8XCfNQs2bYJ99vENrfv1g+OPh+zs2FVKTSgv0PXzXCQF7bEHnH8+TJ8OhYUwcSJ07AgPPQQnn+zhfvnl8PLLvsuSZAYFukiKa9gQzjkHpk3zcJ8yxUP9scd8pk2zZnDppX4lv3lz7GqlJinQRdJIgwZw1lnwxBMe7tOm+bTHJ56Abt2gaVO48EJ49tkdp0pKelCgi6SpXXeFPn18OGbNGh+e6dULnnoKevaEvfaC66/38XdJDwp0kQxQrx787GfwyCM+FfL55/35738Pp5wCn30Wu0KpDgp0kQxTt64Pv0yY4EMx778PRx/tfWgktSnQRTLY2WfDW2/5jdVTT/UNO7T5depSoItkuEMPhbffhtNP9x7v554L//lP7KqkKhToIsLuu/uMmNtvh0mTfNXp0qWxq5LKUqCLCOBtA/7nf2DmTG8BnJcHM2bErkoqQ4EuIjvo2hXmz4f99/eZMLfc4n3eJfkp0EXkB1q1gjlzYNAgGDXKg33t2thVSUUU6CJSqh/9CB5+GP74R3jxRR+Cef/92FVJeRToIlImMxgyBP72N28V8JOf+Px1SU4KdBGpUKdOPq7eoQOcdx5cfbVaBlTVunU199kKdBFJSNOmvrHGNdfAffepZUBVfPCB7xX7+OM18/kKdBFJWE4O3HUXTJ6slgGV9eabcNJJ/v/wmGNq5hwKdBGptJ//3FsG7Labtwy45x61DCjPK694b/o99/QfgAcfXDPnUaCLSJUceijMm+ctA4YNU8uAsjzzDPTo4VNBX3vNv9cUBbqIVJlaBpRv0iTvSX/44fDqq7D33jV7PgW6iOyUbS0Dnn9eLQOKGzcOBg70DbtfesmHW2qaAl1EqsVPfwoFBdCmja8s/eUvYf362FXF8fvfw2WXQffu/oOuYcPaOa8CXUSqTevW8Prr3jLg17/2IYZBg3xhUibcNA0Bbr7Zt/Y7+2x48klfcVtbFOgiUq22tQx4/XUYMMDH2E86Cdq29ZBfsSJ2hTVj61bvJ3/bbXDxxfCXv/juULVJgS4i1c4Mjj0WHnwQVq+GRx+Fli19GGa//Xx4ZvJk+O672JVWj++/h0su8emb+fn+587Orv06FOgiUqPq14fzz4fZs+Ef//BQX7TIr9733huuuMKnP6bqkMymTdC/P4wfD7fe6guvzOLUokAXkVrTpg386lfwySc+86NHDw/CDh18at/o0bBmTewqE7dhA/TuDVOnepDfcku8MAcFuohEkJUFnTvDxIk+JDN2rF/JX3stNG8OZ5wB06fD5s2xKy3bunXQrRvMmuVDLNdcE7siBbqIRNaokU/xe/NNWLDAbyy++aZf+bZoAddd568nky++8B9Ic+f64qFLLoldkVOgi0jSaNcO7rzTZ8JMnw7HHQdjxkD79tCxo1/Jf/113Bo/+8xn7Xz4ITz1lPe1SRYKdBFJOnXq+OKkadN89eno0fDtt3D55X4j9Zxz4Omna3/h0iefwAknwKef+oKh00+v3fNXxEKkW8t5eXmhoKAgyrlFJPWEAO+84zdR//IX3+O0Th1fWt+tm6/KbN++5m5KfvSRd0z89luYOdNv5MZgZvNDCHmlvqdAF5FUs2kTvPGGXyXPnOkbR4DfUN0W7p07+/h8dXjnHZ87n53t+6sedlj1fG5VlBfoCQ25mFk3M1tsZkvNbGQ5x/U1s2BmpZ5MRKQ61K0LJ58Mv/udb7SxciX8+c/e7XHqVOjXD5o0gRNPhN/8Bt5911dyVsWcOb47U/36/jhmmFekwit0M8sGlgCnASuBecCAEMLCEsftBjwL1AWuCiGUe/mtK3QRqQlbtvgsmW1X7++84683bepX7926Qdeu0LhxxZ/1wgs+hbJlS58337JljZaekJ29Qu8ALA0hLAshbAImA71LOe424HdAmizmFZFUlJPj4+q33+4bW29rPXDKKb7ZxIABkJvrV/OjRsHbb5d+9T5tmt+YPfBA35giGcK8IokEenOgeDudlUWv/ZeZ/RhoGUJ4trwPMrPBZlZgZgWFhYWVLlZEpLKaNfPWA5Mm+SrUuXO9/cDWrb5Uv2NHv3ofOBAmTPBjHnsMzjrL90ydPRv22iv2nyIxOTv7AWaWBYwGBlV0bAhhHDAOfMhlZ88tIlIZ2dnQqZN/3XqrLxB64QUfmpk502fPmPmMms6dfZ55gwaxq05cIoG+Cij+y0aLote22Q1oD/zVfL5QM2C6mfWqaBxdRCSmJk18Tvs55/gV+7vv+tj75s1w441Qr17sCisnkUCfB7Q1s9Z4kPcHztn2Zgjh30CTbc/N7K/AdQpzEUklWVk+xHL00bErqboKx9BDCFuAq4BZwEfAlBDCAjMbZWa9arpAERFJTEJj6CGE54DnSrx2cxnHnrzzZYmISGWpl4uISJpQoIuIpAkFuohImlCgi4ikCQW6iEiaUKCLiKSJaP3QzawQWF7F/7wJ8EU1llObVHscqr32pWrdkNy17xdCyC3tjWiBvjPMrKCs9pHJTrXHodprX6rWDalbu4ZcRETShAJdRCRNpGqgj4tdwE5Q7XGo9tqXqnVDitaekmPoIiLyQ6l6hS4iIiUo0EVE0kTKBbqZdTOzxWa21MxGxq4nUWbW0sxmm9lCM1tgZsNi11QZZpZtZu+a2YzYtVSGmTUys6lmtsjMPjKzn8SuKVFmll/0d+VDM5tkZkm7f46ZPWxma8zsw2KvNTazF83s46Lve8SssSxl1H5n0d+ZD8zsSTNrFLHEhKVUoJtZNnA/0B1oBwwws3Zxq0rYFuDaEEI7oBNwZQrVDjAM3+Ak1YwBZoYQDgaOIEX+DGbWHBgK5IUQ2gPZ+G5hyeoRoFuJ10YCL4cQ2gIvFz1PRo/ww9pfBNqHEA4HlgA31nZRVZFSgQ50AJaGEJaFEDYBk4HekWtKSAhhdQjhnaLH6/FgaR63qsSYWQvgdOCh2LVUhpntDpwI/BkghLAphPB11KIqJwf4kZnlALsCn0Wup0whhL8BX5V4uTfwaNHjR4EzarOmRJVWewjhhaLd2gDexPdSTnqpFujNgRXFnq8kRUKxODNrBRwFvBW5lETdDYwAtkauo7JaA4XA+KLhoofMrH7sohIRQlgF/B74FFgN/DuE8ELcqiqtaQhhddHjfwFNYxazEy4Cno9dRCJSLdBTnpk1AP4PGB5CWBe7noqYWU9gTQhhfuxaqiAH+DHwxxDCUcB/SN5f+3dQNN7cG/+htA9Q38zOjVtV1QWfH51yc6TN7Bf4cOnE2LUkItUCfRXQstjzFkWvpQQzq4OH+cQQwrTY9SToOKCXmf0TH+I61cwmxC0pYSuBlSGEbb8JTcUDPhV0AT4JIRSGEDYD04BjI9dUWZ+b2d4ARd/XRK6nUsxsENATGBhSZMFOqgX6PKCtmbU2s7r4TaLpkWtKiJkZPpb7UQhhdOx6EhVCuDGE0CKE0Ar///1KCCElrhRDCP8CVpjZQUUvdQYWRiypMj4FOpnZrkV/dzqTIjd0i5kOXFD0+ALg6Yi1VIqZdcOHGXuFEDbEridRKRXoRTcprgJm4X+5p4QQFsStKmHHAefhV7jvFX31iF1UBrgamGhmHwBHAr+JW05iin6rmAq8A/wd/7eatMvRzWwSMBc4yMxWmtnFwG+B08zsY/w3jt/GrLEsZdR+H7Ab8GLRv9WxUYtMkJb+i4ikiZS6QhcRkbIp0EVE0oQCXUQkTSjQRUTShAJdRCRNKNBFRNKEAl1EJE38f/Hb7SFAbKQRAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.plot(np.arange(14), silhouetteList[0:14], color='blue')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}