Ex11.InterpretableML.with.LIME.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://www.th-koeln.de/img/logo.svg\" style=\"float:right;\" width=\"200\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 11th exercise: <font color=\"#C70039\">Interpretable Machine Learning with LIME for Image Classification</font>\n",
    "* Course: AML\n",
    "* Lecturer: <a href=\"https://www.gernotheisenberg.de/\">Gernot Heisenberg</a>\n",
    "* Author of notebook: <a href=\"https://www.gernotheisenberg.de/\">Gernot Heisenberg</a>\n",
    "* Date:   04.09.2024\n",
    "\n",
    "<img src=\"https://miro.medium.com/max/1400/1*K4HCitQxRrjcFekIsSZDSg.png\" style=\"float: center;\" width=\"600\">\n",
    "\n",
    "---------------------------------\n",
    "**GENERAL NOTE 1**: \n",
    "Please make sure you are reading the entire notebook, since it contains a lot of information on your tasks (e.g. regarding the set of certain paramaters or a specific computational trick), and the written mark downs as well as comments contain a lot of information on how things work together as a whole. \n",
    "\n",
    "**GENERAL NOTE 2**: \n",
    "* Please, when commenting source code, just use English language only. \n",
    "* When describing an observation please use English language, too.\n",
    "* This applies to all exercises throughout this course.\n",
    "\n",
    "---------------------------------\n",
    "\n",
    "### <font color=\"ce33ff\">DESCRIPTION</font>:\n",
    "LIME stands for Local Interpretable Model-agnostic Explanations and was developed by Ribeiro et.al. in 2016 (https://arxiv.org/abs/1602.04938).\n",
    "\n",
    "In this notebook LIME is going to be used to generate explanations for an image classification task. \n",
    "The basic idea applying LIME is to understand why a deep neural network predicts that an instance (image) belongs to a certain class (labrador in this case). \n",
    "This notebook is based on previous work by Cristian Arteaga, [arteagac.github.io](https://arteagac.github.io)\n",
    "\n",
    "---------------------------------\n",
    "\n",
    "### <font color=\"FFC300\">TASKS</font>:\n",
    "The tasks that you need to work on within this notebook are always indicated below as bullet points. \n",
    "If a task is more challenging and consists of several steps, this is indicated as well. \n",
    "Make sure you have worked down the task list and commented your doings. \n",
    "This should be done by using markdown.<br> \n",
    "<font color=red>Make sure you don't forget to specify your name and your matriculation number in the notebook.</font>\n",
    "\n",
    "**YOUR TASKS in this exercise are as follows**:\n",
    "1. import the notebook to Google Colab or use your local machine.\n",
    "2. make sure you specified you name and your matriculation number in the header below my name and date. \n",
    "    * set the date too and remove mine.\n",
    "3. read the entire notebook carefully \n",
    "    * add comments whereever you feel it necessary for better understanding\n",
    "    * run the notebook for the first time. \n",
    "4. test your own images to obtain explanations for your classification tasks.\n",
    "5. change some hyperparameters\n",
    "6. describe your observations and the LIME performance\n",
    "---------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "L1MnrZSd06ED"
   },
   "source": [
    "### Imports\n",
    "Import all necessary python utilities for manipulation of images, plotting and numerical analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "id": "uq9q-_zXgqtV",
    "outputId": "3a4be885-98a1-4d53-843e-8b9ca05c0ab5",
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import tensorflow.keras\n",
    "from   tensorflow.keras.applications.imagenet_utils import decode_predictions\n",
    "import skimage.io \n",
    "import skimage.segmentation\n",
    "import copy\n",
    "import sklearn\n",
    "import sklearn.metrics\n",
    "from   sklearn.linear_model import LinearRegression\n",
    "import warnings\n",
    "\n",
    "np.random.seed(666)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JvUcphNbZEWI"
   },
   "source": [
    "### InceptionV3 initialization\n",
    "We are going to use the pre-trained InceptionV3 model available in Keras."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 495
    },
    "id": "kdLdM8v0hbIi",
    "outputId": "ddb7757e-5ba0-469f-88f2-56a66fcabe55",
    "tags": []
   },
   "outputs": [],
   "source": [
    "warnings.filterwarnings('ignore') \n",
    "inceptionV3_model = tensorflow.keras.applications.inception_v3.InceptionV3() #Load pretrained model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "C5eYkf2qgvF-"
   },
   "source": [
    "### Read and pre-process image\n",
    "The instance to be explained (image) is resized and pre-processed to be suitable for Inception V3. This image is saved in the variable `Xi`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 315
    },
    "id": "jgIPG0y3djCF",
    "outputId": "b266b045-4e3c-4654-c3bd-d5084a22905c",
    "tags": []
   },
   "outputs": [],
   "source": [
    "Xi = skimage.io.imread(\"./data/cat-and-dog.jpg\")\n",
    "Xi = skimage.transform.resize(Xi, (299,299)) \n",
    "Xi = (Xi - 0.5)*2 #Inception pre-processing\n",
    "skimage.io.imshow(Xi/2+0.5) # Show image before inception preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2IB6OOmNmfCf"
   },
   "source": [
    "### Predict the class of the input image\n",
    "The Inception V3 model is used to predict the class of the image. The output of the classification is a vector of 1000 proabilities of beloging to each class available in Inception V3. The description of these classes is shown and it can be seen that the \"Labrador Retriever\" is the top class for the given image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 109
    },
    "id": "8LuLr4osmg7K",
    "outputId": "b5c25f56-19f9-4d40-a685-a1e21fd220b0",
    "tags": []
   },
   "outputs": [],
   "source": [
    "np.random.seed(666)\n",
    "preds = inceptionV3_model.predict(Xi[np.newaxis,:,:,:])\n",
    "decode_predictions(preds)[0] #Top 5 classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rjigiO3QrOqy"
   },
   "source": [
    "The indexes (positions) of the top 5 classes are saved in the variable `top_pred_classes`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "Kxfe8MV0rHej",
    "outputId": "53f6f520-ca8b-493e-d323-78beaec7d225",
    "tags": []
   },
   "outputs": [],
   "source": [
    "top_pred_classes = preds[0].argsort()[-5:][::-1]\n",
    "top_pred_classes                #Index of top 5 classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "owMeom1L-Afp"
   },
   "source": [
    "## LIME explanations\n",
    "The following figure illustrates the basic idea behind LIME. The figure shows light and dark gray areas which are the decision boundaries for the classes for each (x1,x2) pairs in the dataset. LIME is able to provide explanations for the predictions of an individual record (blue dot). The  explanations are created by generating a new dataset of perturbations around the instance to be explained (colored markers around the blue dot). The output or class of each generated perturbation is predicted with the machine-learning model (colored markers inside and outside the decision boundaries). The importance of each perturbation is determined by measuring its distance from the original instance to be explained. These distances are converted to weights by mapping the distances to a zero-one scale using a kernel function (see color scale for the weights). All this information: the new generated dataset, its class predictions and its weights are used to fit a simpler model, such as a linear model (blue line), that can be interpreted. The attributes of the simpler model, coefficients for the case of a linear model, are then used to generate explanations.  \n",
    "<img src=\"./images/lime_illustration.png\" alt=\"LIME\" width=\"600\"/>\n",
    "\n",
    "A detailed explanation of each step is shown below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "X_84p1vkjSJa"
   },
   "source": [
    "### Step 1: Create perturbations of image\n",
    "For the case of image explanations, perturbations will be generated by turning on and off some of the superpixels in the image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u_YCGTxUhc1a"
   },
   "source": [
    "#### Extract super-pixels from image\n",
    "Superpixels are generated using the quickshift segmentation algorithm. It can be noted that for the given image, 69 superpixels were generated. The generated superpixels are shown in the image below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "KiuAcTnuhl8r",
    "outputId": "bc88a511-ac94-4b84-c6c5-0a8536bdd072",
    "tags": []
   },
   "outputs": [],
   "source": [
    "superpixels = skimage.segmentation.quickshift(Xi, kernel_size=4,max_dist=200, ratio=0.2)\n",
    "num_superpixels = np.unique(superpixels).shape[0]\n",
    "num_superpixels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 315
    },
    "id": "8fPvSbn0woWP",
    "outputId": "ab584d0e-b045-45f4-f970-4c1e0fe199a8",
    "tags": []
   },
   "outputs": [],
   "source": [
    "skimage.io.imshow(skimage.segmentation.mark_boundaries(Xi/2+0.5, superpixels))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1-jXIePU4fjo"
   },
   "source": [
    "#### Create random perturbations\n",
    "In this example, 150 perturbations were used. However, for real life applications, a larger number of perturbations will produce more reliable explanations. Random zeros and ones are generated and shaped as a matrix with perturbations as rows and superpixels as columns. An example of a perturbation (the first one) is show below. Here, `1` represents that a superpixel is on and `0` represents it is off. Notice that the length of the shown vector corresponds to the number of superpixels in the image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 90
    },
    "id": "YzPpKl83vgWb",
    "outputId": "5ce22ef7-37c0-4a24-f37b-525056b795fe",
    "tags": []
   },
   "outputs": [],
   "source": [
    "num_perturb = 150\n",
    "perturbations = np.random.binomial(1, 0.5, size=(num_perturb, num_superpixels))\n",
    "perturbations[0] #Show example of perturbation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u3uBZ2jK4qlI"
   },
   "source": [
    "The following function `perturb_image` perturbs the given image (`img`) based on a perturbation vector (`perturbation`) and predefined superpixels (`segments`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "hlUXAdhNx01P",
    "tags": []
   },
   "outputs": [],
   "source": [
    "def perturb_image(img,perturbation,segments):\n",
    "  active_pixels = np.where(perturbation == 1)[0]\n",
    "  mask = np.zeros(segments.shape)\n",
    "  for active in active_pixels:\n",
    "      mask[segments == active] = 1 \n",
    "  perturbed_image = copy.deepcopy(img)\n",
    "  perturbed_image = perturbed_image*mask[:,:,np.newaxis]\n",
    "  return perturbed_image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UNQlIbrI6T7r"
   },
   "source": [
    "Let's use the previous function to see what a perturbed image would look like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 315
    },
    "id": "baNh08g76URc",
    "outputId": "2136b39c-4f3c-496f-850a-5ff10d3bc59d",
    "tags": []
   },
   "outputs": [],
   "source": [
    "skimage.io.imshow(perturb_image(Xi/2+0.5,perturbations[0],superpixels))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PlfMjZkH5GUT"
   },
   "source": [
    "### Step 2: Use ML classifier to predict classes of new generated images\n",
    "This is computationally the most expensive step in LIME because a prediction for each perturbed image is computed. From the shape of the predictions, we can see for each of the perturbations the output probability for each of the 1000 classes in Inception V3. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "fel-ZhZQ5Rbo",
    "outputId": "7088d850-6cab-4ac2-f7bf-bf07d7d4780a",
    "tags": []
   },
   "outputs": [],
   "source": [
    "predictions = []\n",
    "for pert in perturbations:\n",
    "  perturbed_img = perturb_image(Xi,pert,superpixels)\n",
    "  pred = inceptionV3_model.predict(perturbed_img[np.newaxis,:,:,:])\n",
    "  predictions.append(pred)\n",
    "\n",
    "predictions = np.array(predictions)\n",
    "predictions.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e9XR8Tkg-FVC"
   },
   "source": [
    "### Step 3: Compute distances between the original image and each of the perturbed images and compute weights (importance) of each perturbed image\n",
    "The distance between each randomly generated pertubation and the image being explained is computed using the cosine distance. For the shape of the `distances` array it can be noted that, as expected, there is a distance for every generated perturbation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "X1bsUoC0-oT7",
    "outputId": "6995cf07-8699-44ec-ac7a-42ca3b4708ac",
    "tags": []
   },
   "outputs": [],
   "source": [
    "original_image = np.ones(num_superpixels)[np.newaxis,:] #Perturbation with all superpixels enabled \n",
    "distances = sklearn.metrics.pairwise_distances(perturbations,original_image, metric='cosine').ravel()\n",
    "distances.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m0oFKD7VUZRT"
   },
   "source": [
    "#### Use kernel function to compute weights\n",
    "The distances are then mapped to a value between zero and one (weight) using a kernel function. An example of a kernel function with different kernel widths is shown in the plot below. Here the x axis represents distances and the y axis the weights. Depeding on how we set the kernel width, it defines how wide we want the \"locality\" around our instance to be. This kernel width can be set based on expected distance values. For the case of cosine distances, we expect them to be somehow stable (between 0 and 1); therefore, no fine tunning of the kernel width might be required."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "goOVV8yLP8WQ"
   },
   "source": [
    "<img src=\"./images/kernel.png\" alt=\"Drawing\" width=\"600\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "hrgL4KW2UdZI",
    "outputId": "57ed7206-8002-48d1-e9f0-1219367756fc",
    "tags": []
   },
   "outputs": [],
   "source": [
    "kernel_width = 0.25\n",
    "weights = np.sqrt(np.exp(-(distances**2)/kernel_width**2)) #Kernel function\n",
    "weights.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c-0pYQ38VsOW"
   },
   "source": [
    "### Step 4: Use `perturbations`, `predictions` and `weights` to fit an explainable (linear) model\n",
    "A weighed linear regression model is fitted using data from the previous steps (perturbations, predictions and weights). Given that the class that we want to explain is labrador, when fitting the linear model we take from the predictions vector only the column corresponding to the top predicted class. Each coefficients in the linear model corresponds to one superpixel in the segmented image. These coefficients represent how important is each superpixel for the prediction of labrador."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 274
    },
    "id": "vKOkz_Y-WL3u",
    "outputId": "6d9932e3-0a69-42f8-f06d-2b73c028e1f9",
    "tags": []
   },
   "outputs": [],
   "source": [
    "class_to_explain = top_pred_classes[0]\n",
    "simpler_model = LinearRegression()\n",
    "simpler_model.fit(X=perturbations, y=predictions[:,:,class_to_explain], sample_weight=weights)\n",
    "coeff = simpler_model.coef_[0]\n",
    "coeff"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "k9EKcFDai5_l"
   },
   "source": [
    "#### Compute top features (superpixels)\n",
    "Now, sort the coefficients to figure out which are the supperpixels, that have larger coefficients (magnitude) for the prediction of labradors. The identifiers of these top features or superpixels are shown below. Even though here the magnitude of the coefficients were used to determine the most important features, other alternatives such as forward or backward elimination can be used for feature importance selection. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "S6Jo9fp2t-jS",
    "outputId": "dd302872-0571-43e7-c30c-01f039449b15",
    "tags": []
   },
   "outputs": [],
   "source": [
    "num_top_features = 4\n",
    "top_features = np.argsort(coeff)[-num_top_features:] \n",
    "top_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Jg4HjJKDwTon"
   },
   "source": [
    "#### Show LIME explanation (image with top features)\n",
    "Let's show the most important superpixels defined in the previous step in an image after covering up less relevant superpixels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 315
    },
    "id": "eF3nyAc2i-Nt",
    "outputId": "02b11565-361d-439d-e2a6-c30a9397b568",
    "tags": []
   },
   "outputs": [],
   "source": [
    "mask = np.zeros(num_superpixels) \n",
    "mask[top_features]= True #Activate top superpixels\n",
    "skimage.io.imshow(perturb_image(Xi/2+0.5,mask,superpixels) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uwCSCztznCZU"
   },
   "source": [
    "This is the final step where we obtain the area of the image that produced the prediction of labrador."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "lime_image.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.17"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}