fml_classisication_sloth.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6uOOP18G_Q2V"
   },
   "source": [
    "# Fundamentals of Machine Learning - Sloths classification\n",
    "* Sloths are really special animals\n",
    "* Lets play with a [sloth-dataset](https://www.kaggle.com/datasets/bertiemackie/sloth-species) with records about two species and six sub species\n",
    "* Try to classify the animals corectly!\n",
    "\n",
    "![img01](https://github.com/rasvob/VSB-FEI-Fundamentals-of-Machine-Learning-Exercises/blob/master/images/sloth01.jpg?raw=true)\n",
    "\n",
    "![img02](https://github.com/rasvob/VSB-FEI-Fundamentals-of-Machine-Learning-Exercises/blob/master/images/sloth02.jpg?raw=true)\n",
    "\n",
    "<!-- \n",
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/1/18/Bradypus.jpg\" height=\"500\">\n",
    "\n",
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/9/9b/Choloepus_didactylus_2_-_Buffalo_Zoo.jpg/2560px-Choloepus_didactylus_2_-_Buffalo_Zoo.jpg\" height=\"500\"> -->"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 418,
     "status": "ok",
     "timestamp": 1700766921508,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "rQjMpOi_BHju"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn import datasets\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay\n",
    "import plotly.express as px\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "omWlglftCrCU"
   },
   "source": [
    "# Read dataset from the file\n",
    "* Drop the index file included in file and show the example of it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 22,
     "status": "ok",
     "timestamp": 1700766921783,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "_U6Xg440BYIp"
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"https://raw.githubusercontent.com/rasvob/VSB-FEI-Fundamentals-of-Machine-Learning-Exercises/master/datasets/sloth_data.csv\").drop(columns=[\"Unnamed: 0\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 423
    },
    "executionInfo": {
     "elapsed": 21,
     "status": "ok",
     "timestamp": 1700766921784,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "fMajZuFzG_Et",
    "outputId": "832a99ce-bf7e-4b3c-f6f6-4dd382e109f7"
   },
   "outputs": [],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aDBO9kn7DO5m"
   },
   "source": [
    "# Lets check how the classes are distributed the in the dataset\n",
    "* For both specie and sub_specie of sloths"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 542
    },
    "executionInfo": {
     "elapsed": 1451,
     "status": "ok",
     "timestamp": 1700766923223,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "S63aLNCFCcmw",
    "outputId": "f8fa4297-00f2-4157-f177-0fa521ca2d8a"
   },
   "outputs": [],
   "source": [
    "px.histogram(df, x=\"specie\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 542
    },
    "executionInfo": {
     "elapsed": 10,
     "status": "ok",
     "timestamp": 1700766923224,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "v5mdeYphCl7I",
    "outputId": "a0d82f05-ec8c-4151-8618-55c586a260db"
   },
   "outputs": [],
   "source": [
    "px.histogram(df, x=\"sub_specie\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q0vifeVmDr5x"
   },
   "source": [
    "# Let's check the distribution of the numerical attributes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 542
    },
    "executionInfo": {
     "elapsed": 838,
     "status": "ok",
     "timestamp": 1700766924055,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "av6tuZqsH8x9",
    "outputId": "727e1786-ebb9-4f3b-cd83-26c479f694db"
   },
   "outputs": [],
   "source": [
    "px.box(df, y=[\"claw_length_cm\", \"size_cm\", \"tail_length_cm\", \"weight_kg\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 542
    },
    "executionInfo": {
     "elapsed": 17,
     "status": "ok",
     "timestamp": 1700766924056,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "Q9CcrUDtKOZA",
    "outputId": "a78a8d1a-db18-4e98-9bd3-061e1dc02145"
   },
   "outputs": [],
   "source": [
    "px.box(df, y=[\"claw_length_cm\", \"size_cm\", \"tail_length_cm\", \"weight_kg\"], color=\"specie\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 542
    },
    "executionInfo": {
     "elapsed": 1005,
     "status": "ok",
     "timestamp": 1700766925050,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "BQsA9ggCKZsQ",
    "outputId": "ceb15339-60cf-4639-b325-e62a5dc812bb"
   },
   "outputs": [],
   "source": [
    "px.box(df, y=[\"claw_length_cm\", \"size_cm\", \"tail_length_cm\", \"weight_kg\"], color=\"sub_specie\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 542
    },
    "executionInfo": {
     "elapsed": 36,
     "status": "ok",
     "timestamp": 1700766925050,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "apzUpCUWMQFq",
    "outputId": "4b456b6d-b698-4768-d3dc-66cedb7d3b2f"
   },
   "outputs": [],
   "source": [
    "px.scatter_matrix(df, dimensions=[\"claw_length_cm\", \"size_cm\", \"tail_length_cm\", \"weight_kg\"], color=\"specie\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 542
    },
    "executionInfo": {
     "elapsed": 34,
     "status": "ok",
     "timestamp": 1700766925051,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "qGU9YUehMfaT",
    "outputId": "f9ec3674-f057-4c7e-c145-ae83ccf326dd"
   },
   "outputs": [],
   "source": [
    "px.scatter_matrix(df, dimensions=[\"claw_length_cm\", \"size_cm\", \"tail_length_cm\", \"weight_kg\"], color=\"sub_specie\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rK0vKzJMDygt"
   },
   "source": [
    "# Let's convert the categorical data into numerical represenations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 32,
     "status": "ok",
     "timestamp": 1700766925051,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "h-VJ5wH1B0G3"
   },
   "outputs": [],
   "source": [
    "df[\"endangered\"], unique_endagered = pd.factorize(df[\"endangered\"], sort=True)\n",
    "df[\"specie\"], unique_specie =  pd.factorize(df[\"specie\"], sort=True)\n",
    "df[\"sub_specie\"], unique_sub_specie =  pd.factorize(df[\"sub_specie\"], sort=True)\n",
    "df = df.infer_objects()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 423
    },
    "executionInfo": {
     "elapsed": 30,
     "status": "ok",
     "timestamp": 1700766925051,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "emW0uP0rESTi",
    "outputId": "54a2d696-263a-455d-bab3-7db68e86a768"
   },
   "outputs": [],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SvGC_VCZER0U"
   },
   "source": [
    "# 📌 Parallel coordinates plot is useful when you like to follow the trends in data based on the class they represent\n",
    "* 💡 When many objects are drawn the main clusters may be visible in the plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 542
    },
    "executionInfo": {
     "elapsed": 30,
     "status": "ok",
     "timestamp": 1700766925052,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "GF07zvpDONTj",
    "outputId": "21925b00-dd04-4054-bb2d-9ec54ccd4851"
   },
   "outputs": [],
   "source": [
    "px.parallel_coordinates(df, color=\"specie\", dimensions=[\"claw_length_cm\", \"size_cm\", \"tail_length_cm\", \"weight_kg\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LmmzhHD4EqA2"
   },
   "source": [
    "## It's more complicated when more classes are present in the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 542
    },
    "executionInfo": {
     "elapsed": 27,
     "status": "ok",
     "timestamp": 1700766925052,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "rdUfvykfOuns",
    "outputId": "21556bd5-add7-4bbb-9dce-a42248a219ab"
   },
   "outputs": [],
   "source": [
    "px.parallel_coordinates(df, color=\"sub_specie\", dimensions=[\"claw_length_cm\", \"size_cm\", \"tail_length_cm\", \"weight_kg\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JTnaTuApSs2W"
   },
   "source": [
    "# 🚀 Classification of the data\n",
    "* Split the datasets into `X` input matrix and `y` output vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 25,
     "status": "ok",
     "timestamp": 1700766925052,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "8GjN0d1IS9sW"
   },
   "outputs": [],
   "source": [
    "X = df.drop(columns=[\"specie\", \"sub_specie\"]).values\n",
    "ys = df[\"specie\"].values\n",
    "yss = df[\"sub_specie\"].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 24,
     "status": "ok",
     "timestamp": 1700766925052,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "BVTXIl8HUXFN",
    "outputId": "3ebba041-e97c-49f7-9fc7-a40f9c6dd5a1"
   },
   "outputs": [],
   "source": [
    "X.shape, ys.shape, yss.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ue_RKz5_VrUG"
   },
   "source": [
    "# Split the dataset into training and testing subsets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 22,
     "status": "ok",
     "timestamp": 1700766925053,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "6F-fCsAPVy5v"
   },
   "outputs": [],
   "source": [
    "X_train, X_test, ys_train, ys_test, yss_train, yss_test = train_test_split(X, ys, yss, test_size=0.25, random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 21,
     "status": "ok",
     "timestamp": 1700766925053,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "lQO4LlBdWXCi",
    "outputId": "1bc7603c-91ba-438d-bb61-0add51aa2a36"
   },
   "outputs": [],
   "source": [
    "X_train.shape, ys_train.shape, yss_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_gK6cuqUXfpT"
   },
   "source": [
    "# Let's build the models\n",
    "* 💡 Try KNeighborsClassifier, GaussianNB, DecisionTreeClassifier, RandomForrest and MultiLayerPerceptron aka Neural network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 20,
     "status": "ok",
     "timestamp": 1700766925053,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "FXxdQ3jxXFYd"
   },
   "outputs": [],
   "source": [
    "models = [\n",
    "    [\"KNN(3)\", KNeighborsClassifier(3)],\n",
    "    [\"KNN(5)\", KNeighborsClassifier(5)],\n",
    "    [\"KNN(7)\", KNeighborsClassifier(7)],\n",
    "    [\"GaussianNB\", GaussianNB()],\n",
    "    [\"DecisionTree\", DecisionTreeClassifier()],\n",
    "    [\"RandomForrest(10)\", RandomForestClassifier(10)],\n",
    "    [\"RandomForrest(20)\", RandomForestClassifier(20)],\n",
    "    [\"RandomForrest(50)\", RandomForestClassifier(50)],\n",
    "    [\"RandomForrest(100)\", RandomForestClassifier(100)],\n",
    "    [\"MLP(10)\", MLPClassifier(10)],\n",
    "    [\"MLP(20)\", MLPClassifier(20)],\n",
    "    [\"MLP(50)\", MLPClassifier(50)],\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fit the models and evaluate them "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 6278,
     "status": "ok",
     "timestamp": 1700766931312,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "n-gaTxZgYa8A",
    "outputId": "74c65aef-d153-40c4-de57-2e8b16671667"
   },
   "outputs": [],
   "source": [
    "for name, model in models:\n",
    "  model.fit(X_train, ys_train)\n",
    "  trainp = model.score(X_train, ys_train)\n",
    "  testp = model.score(X_test, ys_test)\n",
    "  print(f\"{name:20s} Train: {trainp:.3f}%    Test: {testp:.3f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xyxkpe0Mbf1v"
   },
   "source": [
    "## Let's check the scaled version of the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 12,
     "status": "ok",
     "timestamp": 1700766931313,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "x_WNNgjCbeer"
   },
   "outputs": [],
   "source": [
    "scaler = MinMaxScaler()\n",
    "scaler.fit(X_train)\n",
    "\n",
    "sX_train = scaler.transform(X_train)\n",
    "sX_test = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fit the models and evaluate them again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 8888,
     "status": "ok",
     "timestamp": 1700766940192,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "Rv869mvmcIz8",
    "outputId": "15561885-21cd-452c-9d7f-200a131ea779"
   },
   "outputs": [],
   "source": [
    "for name, model in models:\n",
    "  model.fit(sX_train, ys_train)\n",
    "  trainp = model.score(sX_train, ys_train)\n",
    "  testp = model.score(sX_test, ys_test)\n",
    "  print(f\"{name:20s} Train: {trainp:.3f}%    Test: {testp:.3f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3x60kcDpc-4V"
   },
   "source": [
    "# ⚡ Let's try more specific classification into 5 classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 20636,
     "status": "ok",
     "timestamp": 1700766960815,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "f3hxUF8tdEPP",
    "outputId": "459022cd-9d6b-4ff6-e932-038e853763cc"
   },
   "outputs": [],
   "source": [
    "print(f\"                     Non-Scaled                      Scaled\")\n",
    "for name, model in models:\n",
    "  model.fit(X_train, yss_train)\n",
    "  trainp = model.score(X_train, yss_train)\n",
    "  testp = model.score(X_test, yss_test)\n",
    "  model.fit(sX_train, yss_train)\n",
    "  strainp = model.score(sX_train, yss_train)\n",
    "  stestp = model.score(sX_test, yss_test)\n",
    "  print(f\"{name:20s} Train: {trainp:.3f}%    Test: {testp:.3f}%   Train: {strainp:.3f}%    Test: {stestp:.3f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wpTpQySDoDw4"
   },
   "source": [
    "## 🌳 Check the confusion matrix of one of the best model - Random Forest with 20 inner trees"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 260,
     "status": "ok",
     "timestamp": 1700767186945,
     "user": {
      "displayName": "Jan Platoš",
      "userId": "17018338888827516123"
     },
     "user_tz": -60
    },
    "id": "goV_akw1nKs_",
    "outputId": "a05e3989-ed1d-42dd-e720-c5b997a69013"
   },
   "outputs": [],
   "source": [
    "model = RandomForestClassifier(20)\n",
    "model.fit(sX_train, yss_train)\n",
    "predicted = model.predict(sX_test)\n",
    "\n",
    "cm = confusion_matrix(yss_test, predicted, labels=model.classes_)\n",
    "cm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ConfusionMatrixDisplay(cm, display_labels=model.classes_).plot()"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "authorship_tag": "ABX9TyMUE1oZPH9SoPvTOrWOxQz+",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}