src/tasks/task-3-model-training/Best-Model/ResNet50/mango-leaf-disease-detection-using-cnn.ipynb

{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<h1 style=\"font-family:Verdana;\"> <center>🍃 Mango Leaf Disease Detection Using Convolutional Neural Networks</center> </h1>\n<p><center style=\"color:#159364; font-family:Helvetica; font-size:20px;\">A Comprehensive Notebook Outlining a Transfer Learning approach for Mango Leaf Disease Classification</center></p>\n\n---\n\n<center><img src='https://img.freepik.com/free-vector/flat-illustration-people-taking-care-plants_52683-64057.jpg?t=st=1694349725~exp=1694350325~hmac=1afea65d489d4174af5e5abb9073ec54f0c4487ecc3444767265fbcadb529ee9' height=150px width=400px></center>\n\n# 👋Thanks for Visiting my Notebook\n<div class=\"alert alert-block alert-info\" style=\"font-size:20px; font-family:verdana; border-radius: 15px; background-color: #cce5ff; border-color: #b8daff;\">\n    📌 Feel free to fork or edit the notebook for your own convenience. If you liked the notebook, consider upvoting. It helps other people discover the notebook as well. Your support inspires me to produce more of these kernels.😊\n</div>\n\n# 🔬Overview \n<div style=\"background-color:#fff1cc; padding: 20px;\">\n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">Mango, known as the 'king of fruits', is one of the most popular, nutritionally rich fruits with unique flavor, fragrance, taste, and health-promoting qualities. While its luscious taste is loved globally, mango trees are often subjected to various diseases that can adversely affect fruit yield. Diseases like 'Anthracnose', 'Bacterial Canker', 'Die Back', and 'Powdery Mildew' are some of the common diseases that affect mango leaves.</p>\n\n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">Early detection of these diseases can aid in proper management and ensure high fruit yield. With advancements in technology, using Convolutional Neural Networks (CNN) to detect and classify these diseases in their early stages can be immensely helpful. This notebook attempts to harness the power of CNNs for mango leaf disease detection and classification.</p>\n\n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">In this analysis, we'll be focusing on detecting the following diseases based on mango leaf images: 'Anthracnose', 'Bacterial Canker', 'Die Back', 'Gall Midge', 'Healthy', 'Powdary Mildew', and 'Scooty Mould'.</p>\n</div>\n\n# ❗Author's Note:\n<div style=\"background-color:#d4f1f4; padding: 20px;\">\n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">Make sure to run the cells from top to bottom with a GPU accelerator. There are some linux commands present in some cells so this is important to take into account. Also, any suggestions, comments, and recommendations to improve the notebook will be highly appreciated. Cheers!</p>\n</div>\n","metadata":{"id":"n6QHvU2dfpFz","papermill":{"duration":0.018811,"end_time":"2023-07-18T09:34:26.687016","exception":false,"start_time":"2023-07-18T09:34:26.668205","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"<div style=\"background-color:#fff1cc; padding: 20px; border-radius: 15px; box-shadow: 0 2px 4px 0 rgba(0, 0, 0, 0.1);\">\n    <h3 style=\"font-size:24px; font-family:verdana; color:#6F4E37;\"><b>🏗️Import Necessary Libraries</b></h3>\n    <p style=\"font-size:20px; font-family:verdana; line-height: 1.7em; text-indent: 20px;\">The code imports various data science libraries, including TensorFlow and scikit-learn, and defines a neural network model using the various CNN architecture. It also includes system libraries for handling file paths and metrics for evaluating model performance. Additionally, it sets up various callbacks and optimizers to optimize the training process.</p>\n</div>\n","metadata":{"id":"kQELj5Dwf3on","papermill":{"duration":0.015782,"end_time":"2023-07-18T09:34:26.732954","exception":false,"start_time":"2023-07-18T09:34:26.717172","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Import Data Science Libraries\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport tensorflow as tf\nfrom sklearn.model_selection import train_test_split\n\n# Tensorflow Libraries\nfrom tensorflow import keras\nfrom tensorflow.keras import layers,models\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.layers import Dense, Dropout\nfrom tensorflow.keras.callbacks import Callback, EarlyStopping,ModelCheckpoint, ReduceLROnPlateau\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.applications import MobileNetV2\nfrom tensorflow.keras import Model\nfrom tensorflow.keras.layers.experimental import preprocessing\n\n# System libraries\nfrom pathlib import Path\nimport os.path\nimport random\nfrom collections import Counter\n\n# Visualization Libraries\nimport matplotlib.cm as cm\nimport cv2\nimport seaborn as sns\nfrom cycler import cycler\nimport textwrap\n\nsns.set_style('darkgrid')\n\n# Metrics\nfrom sklearn.metrics import classification_report, confusion_matrix\nimport itertools","metadata":{"id":"lEzJXgjDf5y8","papermill":{"duration":8.605157,"end_time":"2023-07-18T09:34:35.348966","exception":false,"start_time":"2023-07-18T09:34:26.743809","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:26:39.172449Z","iopub.execute_input":"2023-09-28T02:26:39.173184Z","iopub.status.idle":"2023-09-28T02:26:48.436363Z","shell.execute_reply.started":"2023-09-28T02:26:39.173148Z","shell.execute_reply":"2023-09-28T02:26:48.435439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<div style=\"background-color:#fff1cc; padding: 20px; border-radius: 15px; box-shadow: 0 2px 4px 0 rgba(0, 0, 0, 0.1);\">\n    <h3 style=\"font-size:24px; font-family:verdana; color:#6F4E37;\"><b>🍃 Load and Process Mango Leaf Images</b></h3>\n    <p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">In this notebook, we'll be harnessing the Mango Leaf Disease Dataset. This dataset consists of images of mango leaves affected by various diseases such as: 'Anthracnose', 'Bacterial Canker', 'Die Back', 'Gall Midge', 'Healthy', 'Powdary Mildew', and 'Scooty Mould'. Our objective is to preprocess these images and utilize them to train a Convolutional Neural Network for disease classification.</p>\n</div>\n","metadata":{"id":"Hb_XPhOwiCNY","papermill":{"duration":0.009537,"end_time":"2023-07-18T09:34:35.368640","exception":false,"start_time":"2023-07-18T09:34:35.359103","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Create helper functions\n!wget https://raw.githubusercontent.com/mrdbourke/tensorflow-deep-learning/main/extras/helper_functions.py\n\n# Import series of helper functions for our notebook\nfrom helper_functions import create_tensorboard_callback, plot_loss_curves, unzip_data, compare_historys, walk_through_dir, pred_and_plot\n\n\nBATCH_SIZE = 32\nTARGET_SIZE = (224, 224)\n# Walk through each directory\ndataset = \"/kaggle/input/omdena-mango-leaf/MangoLeafBD_Without_Testset_Augmentation\"\nwalk_through_dir(dataset);","metadata":{"id":"8nD56d7Xxmc3","papermill":{"duration":1.438118,"end_time":"2023-07-18T09:34:36.816440","exception":false,"start_time":"2023-07-18T09:34:35.378322","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:26:48.438168Z","iopub.execute_input":"2023-09-28T02:26:48.438767Z","iopub.status.idle":"2023-09-28T02:26:50.070348Z","shell.execute_reply.started":"2023-09-28T02:26:48.438730Z","shell.execute_reply":"2023-09-28T02:26:50.069211Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Seed Everything to reproduce results for future use cases\ndef seed_everything(seed=42):\n    # Seed value for TensorFlow\n    tf.random.set_seed(seed)\n    \n    # Seed value for NumPy\n    np.random.seed(seed)\n    \n    # Seed value for Python's random library\n    random.seed(seed)\n    \n    # Force TensorFlow to use single thread\n    # Multiple threads are a potential source of non-reproducible results.\n    session_conf = tf.compat.v1.ConfigProto(\n        intra_op_parallelism_threads=1,\n        inter_op_parallelism_threads=1\n    )\n\n    # Make sure that TensorFlow uses a deterministic operation wherever possible\n    tf.compat.v1.set_random_seed(seed)\n\n    sess = tf.compat.v1.Session(graph=tf.compat.v1.get_default_graph(), config=session_conf)\n    tf.compat.v1.keras.backend.set_session(sess)\n\nseed_everything()","metadata":{"execution":{"iopub.status.busy":"2023-09-28T02:26:50.072098Z","iopub.execute_input":"2023-09-28T02:26:50.072862Z","iopub.status.idle":"2023-09-28T02:26:53.517637Z","shell.execute_reply.started":"2023-09-28T02:26:50.072824Z","shell.execute_reply":"2023-09-28T02:26:53.516639Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<div style=\"background-color:#fff1cc; padding: 20px; border-radius: 15px; box-shadow: 0 2px 4px 0 rgba(0, 0, 0, 0.1);\">\n    <h3 style=\"font-size:24px; font-family:verdana; color:#6F4E37;\"><b>📅Placing Data into a Dataframe</b></h3>\n    <p style=\"font-size:20px; font-family:verdana; line-height: 1.7em; text-indent: 20px;\">The first column <b>filepaths</b> contains the file path location of each individual images. The second column <b>labels</b>, on the other hand, contains the class label of the corresponding image from the file path</p>\n</div>\n","metadata":{"id":"MLAnhGlf1hmo","papermill":{"duration":0.019196,"end_time":"2023-07-18T09:34:36.852453","exception":false,"start_time":"2023-07-18T09:34:36.833257","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def create_image_dataframe(dataset_path: str) -> pd.DataFrame:\n    image_dir = Path(dataset_path)\n    \n    # Get filepaths for various extensions\n    filepaths = list(image_dir.glob(r'**/*.JPG')) + \\\n                list(image_dir.glob(r'**/*.jpg')) + \\\n                list(image_dir.glob(r'**/*.png')) + \\\n                list(image_dir.glob(r'**/*.PNG'))\n\n    # Extract labels from filepaths\n    labels = list(map(lambda x: os.path.split(os.path.split(x)[0])[1], filepaths))\n    \n    # Convert lists to Series\n    filepaths_series = pd.Series(filepaths, name='Filepath').astype(str)\n    labels_series = pd.Series(labels, name='Label')\n    \n    # Concatenate filepaths and labels into a DataFrame\n    image_df = pd.concat([filepaths_series, labels_series], axis=1)\n    \n    return image_df\n\n# Usage\ntest_dataset = \"/kaggle/input/omdena-mango-leaf/MangoLeafBD_Without_Testset_Augmentation/Test\"\ntrain_dataset = \"/kaggle/input/omdena-mango-leaf/MangoLeafBD_Without_Testset_Augmentation/Train\"\n\ntest_df = create_image_dataframe(test_dataset)\ntrain_df = create_image_dataframe(train_dataset)","metadata":{"_kg_hide-input":true,"id":"s14XOEp01m_s","papermill":{"duration":0.1107,"end_time":"2023-07-18T09:34:36.982822","exception":false,"start_time":"2023-07-18T09:34:36.872122","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:26:53.521270Z","iopub.execute_input":"2023-09-28T02:26:53.521753Z","iopub.status.idle":"2023-09-28T02:26:53.706885Z","shell.execute_reply.started":"2023-09-28T02:26:53.521715Z","shell.execute_reply":"2023-09-28T02:26:53.705794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df","metadata":{"execution":{"iopub.status.busy":"2023-09-28T02:26:53.708342Z","iopub.execute_input":"2023-09-28T02:26:53.708683Z","iopub.status.idle":"2023-09-28T02:26:53.736417Z","shell.execute_reply.started":"2023-09-28T02:26:53.708648Z","shell.execute_reply":"2023-09-28T02:26:53.735506Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.style.use('fivethirtyeight')\n\n# Get the labels\nlabel_counts = train_df['Label'].value_counts()[:]\n\n# Create a cycler object using the desired colors\ncolor_cycler = (cycler(color=[\"#3B240B\"]))  # Darker coffee brown color\n\n# Set the property cycle of the axes to the created cycler object\nplt.rc('axes', prop_cycle=color_cycler)\n\n# Other styling options\nplt.rc(\"font\", family=\"serif\", size=18.)\nplt.rc(\"savefig\", dpi=200)\nplt.rc(\"legend\", loc=\"best\", fontsize=\"medium\", fancybox=True, framealpha=0.5)\nplt.rc(\"lines\", linewidth=2.5, markersize=10, markeredgewidth=2.5)\n\nplt.figure(figsize=(30, 10), dpi=300)  # Increase figure size\nbars = plt.bar(label_counts.index, label_counts.values, alpha=0.8)\nplt.title('Distribution of Classes in the Dataset', fontsize=20, fontweight='bold')\nplt.xlabel('Label', fontsize=16, fontweight='bold')\nplt.ylabel('Count', fontsize=16, fontweight='bold')\n\n# Wrap x-labels\nwrapped_labels = ['\\n'.join(textwrap.wrap(label, 10)) for label in label_counts.index]\nplt.xticks(range(len(wrapped_labels)), wrapped_labels, fontsize=12, ha='center')  # Change alignment to 'center'\n\nplt.yticks(fontsize=14)\n\n# Adjust y limit to avoid overlapping\nplt.ylim(0, max(label_counts) * 1.1)  # Set y limit 10% higher than the max value\n\n# Add value annotations on top of each bar\nfor bar in bars:\n    yval = bar.get_height()\n    plt.text(bar.get_x() + bar.get_width() / 2, yval + 0.05, int(yval), ha='center', va='bottom')\n\nplt.show()\n","metadata":{"_kg_hide-input":true,"id":"d3-uoP4n1oqK","outputId":"4af0c4a5-c87d-42eb-aa1b-fe89f1fe8876","papermill":{"duration":2.151966,"end_time":"2023-07-18T09:34:39.145009","exception":false,"start_time":"2023-07-18T09:34:36.993043","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:26:53.738003Z","iopub.execute_input":"2023-09-28T02:26:53.738357Z","iopub.status.idle":"2023-09-28T02:26:55.733420Z","shell.execute_reply.started":"2023-09-28T02:26:53.738325Z","shell.execute_reply":"2023-09-28T02:26:55.732545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🔭Visualizing images from the dataset","metadata":{"id":"agb1QMDO1pps","papermill":{"duration":0.014482,"end_time":"2023-07-18T09:34:39.174310","exception":false,"start_time":"2023-07-18T09:34:39.159828","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Set title size and figure background color\nplt.rcParams['axes.titlesize'] = 'small'  # Adjust as needed\nplt.rcParams['figure.facecolor'] = 'white'  # Set figure background color to white\n\n# Display 16 pictures of the dataset with their labels\nrandom_index = np.random.randint(0, len(train_df), 16)\nfig, axes = plt.subplots(nrows=4, ncols=4, figsize=(10, 10),\n                        subplot_kw={'xticks': [], 'yticks': []})\n\nfor i, ax in enumerate(axes.flat):\n    ax.imshow(plt.imread(train_df.Filepath[random_index[i]]))\n    title = train_df.Label[random_index[i]]\n    ax.set_title(\"\\n\".join(textwrap.wrap(title, 20)), fontsize=8)  # Wrap long titles\nplt.tight_layout()\nplt.subplots_adjust(hspace=0.5)  # Increase vertical space between subplots\nplt.show()\n","metadata":{"id":"m4WJVJ7j1rU9","outputId":"777f0f9a-e46c-4475-ca6a-97a4e6fb89a3","papermill":{"duration":4.662798,"end_time":"2023-07-18T09:34:43.851756","exception":false,"start_time":"2023-07-18T09:34:39.188958","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:26:55.734415Z","iopub.execute_input":"2023-09-28T02:26:55.734796Z","iopub.status.idle":"2023-09-28T02:26:57.450084Z","shell.execute_reply.started":"2023-09-28T02:26:55.734762Z","shell.execute_reply":"2023-09-28T02:26:57.449136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<div style=\"background-color:#fff1cc; padding: 20px; border-radius: 15px; box-shadow: 0 2px 4px 0 rgba(0, 0, 0, 0.1);\">\n    <h3 style=\"font-size:24px; font-family:verdana; color:#6F4E37;\"><b>🧮Computing Error Rate Analysis</b></h3>\n    <p style=\"font-size:20px; font-family:verdana; line-height: 1.7em; text-indent: 20px;\">The code below is used for error level analysis (ELA) of images in the context of a animals image classification task.</p>\n    <p style=\"font-size:20px; font-family:verdana; line-height: 1.7em; text-indent: 20px;\">The compute_ela_cv() function takes an image path and a quality parameter, compresses the image using JPEG compression at the given quality, and computes the absolute difference between the compressed and original images. The difference is multiplied by a scale factor and returned as an ELA image.</p>\n    <p style=\"font-size:20px; font-family:verdana; line-height: 1.7em; text-indent: 20px;\">The convert_to_ela_image() function takes an image path and a quality parameter, compresses the image using JPEG compression at the given quality, computes the absolute difference between the compressed and original images, and returns an ELA image. The ELA image is computed using the difference in pixel values between the original and compressed images, and is normalized to enhance the differences.</p>\n    <p style=\"font-size:20px; font-family:verdana; line-height: 1.7em; text-indent: 20px;\">The random_sample() function takes a directory path and an optional file extension, and returns a random file path from that directory with the specified extension (if provided).</p>\n    <p style=\"font-size:20px; font-family:verdana; line-height: 1.7em; text-indent: 20px;\">Finally, the code generates a set of ELA images using the compute_ela_cv() function for a randomly selected image from the test dataset of animals images. The ELA images are generated with decreasing quality levels, resulting in increasing levels of compression and thus increasing levels of error. The resulting images are plotted using matplotlib.</p>\n    <p style=\"font-size:20px; font-family:verdana; line-height: 1.7em; text-indent: 20px;\">Overall, this code provides a way to visually analyze the effect of different levels of JPEG compression on animals images, and may be used to identify optimal quality levels for image compression in the context of a animals image classification task.</p>\n</div>\n","metadata":{"papermill":{"duration":0.030159,"end_time":"2023-07-18T09:34:43.912159","exception":false,"start_time":"2023-07-18T09:34:43.882000","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def compute_ela_cv(path, quality):\n    temp_filename = 'temp_file_name.jpeg'\n    SCALE = 15\n    orig_img = cv2.imread(path)\n    orig_img = cv2.cvtColor(orig_img, cv2.COLOR_BGR2RGB)\n    \n    cv2.imwrite(temp_filename, orig_img, [cv2.IMWRITE_JPEG_QUALITY, quality])\n\n    # read compressed image\n    compressed_img = cv2.imread(temp_filename)\n\n    # get absolute difference between img1 and img2 and multiply by scale\n    diff = SCALE * cv2.absdiff(orig_img, compressed_img)\n    return diff\n\n\ndef convert_to_ela_image(path, quality):\n    temp_filename = 'temp_file_name.jpeg'\n    ela_filename = 'temp_ela.png'\n    image = Image.open(path).convert('RGB')\n    image.save(temp_filename, 'JPEG', quality = quality)\n    temp_image = Image.open(temp_filename)\n\n    ela_image = ImageChops.difference(image, temp_image)\n\n    extrema = ela_image.getextrema()\n    max_diff = max([ex[1] for ex in extrema])\n    if max_diff == 0:\n        max_diff = 1\n\n    scale = 255.0 / max_diff\n    ela_image = ImageEnhance.Brightness(ela_image).enhance(scale)\n    \n    return ela_image\n\n\ndef random_sample(path, extension=None):\n    if extension:\n        items = Path(path).glob(f'*.{extension}')\n    else:\n        items = Path(path).glob(f'*')\n        \n    items = list(items)\n        \n    p = random.choice(items)\n    return p.as_posix()","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.044365,"end_time":"2023-07-18T09:34:43.986621","exception":false,"start_time":"2023-07-18T09:34:43.942256","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:26:57.451447Z","iopub.execute_input":"2023-09-28T02:26:57.452182Z","iopub.status.idle":"2023-09-28T02:26:57.465392Z","shell.execute_reply.started":"2023-09-28T02:26:57.452149Z","shell.execute_reply":"2023-09-28T02:26:57.464053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# View random sample from the dataset\np = random_sample('/kaggle/input/omdena-mango-leaf/MangoLeafBD_Without_Testset_Augmentation/Train/Anthracnose')\norig = cv2.imread(p)\norig = cv2.cvtColor(orig, cv2.COLOR_BGR2RGB) / 255.0\ninit_val = 100\ncolumns = 3\nrows = 3\n\nfig=plt.figure(figsize=(15, 10))\nfig.suptitle('Error Level Analysis', fontsize=20)  # Add super title\nfor i in range(1, columns*rows +1):\n    quality=init_val - (i-1) * 8\n    img = compute_ela_cv(path=p, quality=quality)\n    if i == 1:\n        img = orig.copy()\n    ax = fig.add_subplot(rows, columns, i)\n    ax.title.set_text(f'q: {quality}')\n    plt.imshow(img)\n    ax.set_xticks([])  # Remove x-ticks\n    ax.set_yticks([])  # Remove y-ticks\nplt.show()\n","metadata":{"_kg_hide-input":true,"papermill":{"duration":3.607131,"end_time":"2023-07-18T09:34:47.624008","exception":false,"start_time":"2023-07-18T09:34:44.016877","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:26:57.466882Z","iopub.execute_input":"2023-09-28T02:26:57.467279Z","iopub.status.idle":"2023-09-28T02:26:58.456289Z","shell.execute_reply.started":"2023-09-28T02:26:57.467249Z","shell.execute_reply":"2023-09-28T02:26:58.455465Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<div style=\"background-color:#fff1cc; padding: 20px; border-radius: 15px; box-shadow: 0 2px 4px 0 rgba(0, 0, 0, 0.1);\">\n    <h3 style=\"font-size:24px; font-family:verdana; color:#6F4E37;\"><b>📝Data Preprocessing</b></h3>\n    <p style=\"font-size:20px; font-family:verdana; line-height: 1.7em; text-indent: 20px;\">The data will be split into three different categories: Training, Validation and Testing. The training data will be used to train the deep learning CNN model and its parameters will be fine tuned with the validation data. Finally, the performance of the data will be evaluated using the test data(data the model has not previously seen).</p>\n</div>\n","metadata":{"id":"YwC4EWei1s-V","papermill":{"duration":0.059855,"end_time":"2023-07-18T09:34:47.745599","exception":false,"start_time":"2023-07-18T09:34:47.685744","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Separate in train and test data\ntrain_generator = ImageDataGenerator(\n    preprocessing_function=tf.keras.applications.resnet50.preprocess_input,\n)\n\ntest_generator = ImageDataGenerator(\n    preprocessing_function=tf.keras.applications.resnet50.preprocess_input,\n)","metadata":{"id":"3puUVDwl2Mcz","papermill":{"duration":0.076103,"end_time":"2023-07-18T09:34:47.882175","exception":false,"start_time":"2023-07-18T09:34:47.806072","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:26:58.460775Z","iopub.execute_input":"2023-09-28T02:26:58.461739Z","iopub.status.idle":"2023-09-28T02:26:58.466884Z","shell.execute_reply.started":"2023-09-28T02:26:58.461702Z","shell.execute_reply":"2023-09-28T02:26:58.466042Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\n# Shuffle your dataframe for randomness\ntrain_df = train_df.sample(frac=1).reset_index(drop=True)\n\n# Stratified Split\ntrain_df, val_df = train_test_split(train_df, test_size=0.3, stratify=train_df['Label'])\n\n# Now use train_df with train_images generator and val_df with val_images generator without the subset parameter\ntrain_images = train_generator.flow_from_dataframe(\n    dataframe=train_df,\n    x_col='Filepath',\n    y_col='Label',\n    target_size=TARGET_SIZE,\n    color_mode='rgb',\n    class_mode='categorical',\n    batch_size=BATCH_SIZE,\n    shuffle=True,\n    seed=42\n)\n\nval_images = train_generator.flow_from_dataframe(\n    dataframe=val_df,\n    x_col='Filepath',\n    y_col='Label',\n    target_size=TARGET_SIZE,\n    color_mode='rgb',\n    class_mode='categorical',\n    batch_size=BATCH_SIZE,\n    shuffle=True,\n    seed=42\n)\n\ntest_images = test_generator.flow_from_dataframe(\n    dataframe=test_df,\n    x_col='Filepath',\n    y_col='Label',\n    target_size=TARGET_SIZE,\n    color_mode='rgb',\n    class_mode='categorical',\n    batch_size=BATCH_SIZE,\n    shuffle=False\n)","metadata":{"id":"CsftNShQ2PaK","outputId":"ee3c99b5-3932-4187-f600-1e86cf334026","papermill":{"duration":0.480229,"end_time":"2023-07-18T09:34:48.423519","exception":false,"start_time":"2023-07-18T09:34:47.943290","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:26:58.468243Z","iopub.execute_input":"2023-09-28T02:26:58.469337Z","iopub.status.idle":"2023-09-28T02:27:00.336499Z","shell.execute_reply.started":"2023-09-28T02:26:58.469299Z","shell.execute_reply":"2023-09-28T02:27:00.335581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Function to get distribution from the generator\ndef get_distribution(generator):\n    # Counting occurrences for each class\n    counter = Counter(generator.classes)\n    max_val = float(max(counter.values()))\n    return {k: v/max_val for k, v in counter.items()}\n\n# Print distributions\nprint(get_distribution(train_images))\nprint(get_distribution(val_images))\n","metadata":{"execution":{"iopub.status.busy":"2023-09-28T02:27:00.340026Z","iopub.execute_input":"2023-09-28T02:27:00.340322Z","iopub.status.idle":"2023-09-28T02:27:00.348004Z","shell.execute_reply.started":"2023-09-28T02:27:00.340296Z","shell.execute_reply":"2023-09-28T02:27:00.347030Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Data Augmentation Step\naugment = tf.keras.Sequential([\n  layers.experimental.preprocessing.Resizing(224,224),\n  layers.experimental.preprocessing.Rescaling(1./255),\n  layers.experimental.preprocessing.RandomFlip(\"horizontal\"),\n  layers.experimental.preprocessing.RandomRotation(0.1),\n  layers.experimental.preprocessing.RandomContrast(0.1),\n])","metadata":{"_kg_hide-output":true,"id":"sLbR4WtD2RPg","papermill":{"duration":2.642219,"end_time":"2023-07-18T09:34:51.127418","exception":false,"start_time":"2023-07-18T09:34:48.485199","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:27:00.349697Z","iopub.execute_input":"2023-09-28T02:27:00.350575Z","iopub.status.idle":"2023-09-28T02:27:00.790675Z","shell.execute_reply.started":"2023-09-28T02:27:00.350479Z","shell.execute_reply":"2023-09-28T02:27:00.789647Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<div style=\"background-color:#fff1cc; padding: 20px; border-radius: 15px; box-shadow: 0 2px 4px 0 rgba(0, 0, 0, 0.1);\">\n    <h3 style=\"font-size:24px; font-family:verdana; color:#6F4E37;\"><b>🤹Training the model</b></h3>\n  <p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">The model images will be subjected to a pre-trained CNN model called ResNet50. Three callbacks will be utilized to monitor the training. These are: Model Checkpoint, Early Stopping, Tensorboard callback. The summary of the model hyperparameter is shown as follows:</p>\n\n  <p style=\"font-size:20px\">\n    <strong>Batch size</strong>: 32<br>\n    <strong>Epochs</strong>: 100<br>\n    <strong>Input Shape</strong>: (224, 224, 3)<br>\n    <strong>Output layer</strong>: 8\n  </p>\n</div>\n\n\n","metadata":{"id":"xSEhK2w02Uk-","papermill":{"duration":0.061271,"end_time":"2023-07-18T09:34:51.251098","exception":false,"start_time":"2023-07-18T09:34:51.189827","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Load the pretained model\npretrained_model = tf.keras.applications.resnet50.ResNet50(\n    input_shape=(224, 224, 3),\n    include_top=False,\n    weights='imagenet',\n    pooling='max'\n)\n\npretrained_model.trainable = False","metadata":{"id":"z4VI_UxV2Wp2","papermill":{"duration":4.814894,"end_time":"2023-07-18T09:34:56.127746","exception":false,"start_time":"2023-07-18T09:34:51.312852","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:27:00.792092Z","iopub.execute_input":"2023-09-28T02:27:00.792434Z","iopub.status.idle":"2023-09-28T02:27:03.185644Z","shell.execute_reply.started":"2023-09-28T02:27:00.792399Z","shell.execute_reply":"2023-09-28T02:27:03.184646Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create checkpoint callback\ncheckpoint_path = \"mld_classification_model_checkpoint\"\ncheckpoint_callback = ModelCheckpoint(checkpoint_path,\n                                      save_weights_only=True,\n                                      monitor=\"val_accuracy\",\n                                      save_best_only=True)\n\n# Setup EarlyStopping callback to stop training if model's val_loss doesn't improve for 3 epochs\nearly_stopping = EarlyStopping(monitor = \"val_loss\", # watch the val loss metric\n                               patience = 5,\n                               restore_best_weights = True) # if val loss decreases for 3 epochs in a row, stop training\n\nreduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=3, min_lr=1e-6)","metadata":{"papermill":{"duration":0.073028,"end_time":"2023-07-18T09:34:56.266911","exception":false,"start_time":"2023-07-18T09:34:56.193883","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:27:03.186938Z","iopub.execute_input":"2023-09-28T02:27:03.187307Z","iopub.status.idle":"2023-09-28T02:27:03.196285Z","shell.execute_reply.started":"2023-09-28T02:27:03.187275Z","shell.execute_reply":"2023-09-28T02:27:03.195263Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras import backend as K\n\ndef f1_score(y_true, y_pred):\n    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))\n    possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))\n    predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))\n    precision = true_positives / (predicted_positives + K.epsilon())\n    recall = true_positives / (possible_positives + K.epsilon())\n    f1_val = 2*(precision*recall)/(precision+recall+K.epsilon())\n    return f1_val\n","metadata":{"execution":{"iopub.status.busy":"2023-09-28T02:27:03.197871Z","iopub.execute_input":"2023-09-28T02:27:03.198349Z","iopub.status.idle":"2023-09-28T02:27:03.207917Z","shell.execute_reply.started":"2023-09-28T02:27:03.198316Z","shell.execute_reply":"2023-09-28T02:27:03.206845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Existing model structure\ninputs = pretrained_model.input\nx = augment(inputs)\nx = Dense(128, activation='relu')(pretrained_model.output)\nx = Dropout(0.45)(x)\nx = Dense(256, activation='relu')(x)\nx = Dropout(0.45)(x)\noutputs = Dense(8, activation='softmax')(x)\n\nmodel = Model(inputs=inputs, outputs=outputs)\n\n# Compile with custom F1-Score\nmodel.compile(\n    optimizer=Adam(0.001),\n    loss='categorical_crossentropy',\n    metrics=['accuracy', f1_score]\n)\n\nhistory = model.fit(\n    train_images,\n    steps_per_epoch=len(train_images),\n    validation_data=val_images,\n    validation_steps=len(val_images),\n    epochs=100,\n    callbacks=[\n        early_stopping,\n        create_tensorboard_callback(\"training_logs\", \"mld_classification\"),\n        checkpoint_callback,\n        reduce_lr\n    ]\n)","metadata":{"_kg_hide-output":true,"id":"AkcAsl5H2tYl","outputId":"173d8e87-7f30-401e-fe3b-077c21025b59","papermill":{"duration":577.066476,"end_time":"2023-07-18T09:44:33.395517","exception":false,"start_time":"2023-07-18T09:34:56.329041","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:27:03.209052Z","iopub.execute_input":"2023-09-28T02:27:03.209902Z","iopub.status.idle":"2023-09-28T02:32:25.931887Z","shell.execute_reply.started":"2023-09-28T02:27:03.209868Z","shell.execute_reply":"2023-09-28T02:32:25.930912Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.save_weights('mango_leaf_disease_ResNet50.h5')","metadata":{"execution":{"iopub.status.busy":"2023-09-28T02:32:25.933308Z","iopub.execute_input":"2023-09-28T02:32:25.934141Z","iopub.status.idle":"2023-09-28T02:32:26.321599Z","shell.execute_reply.started":"2023-09-28T02:32:25.934104Z","shell.execute_reply":"2023-09-28T02:32:26.320611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<div style=\"background-color:#fff1cc; padding: 20px; border-radius: 15px; box-shadow: 0 2px 4px 0 rgba(0, 0, 0, 0.1);\">\n    <h3 style=\"font-size:24px; font-family:verdana; color:#6F4E37;\"><b>✔️Model Evaluation</b></h3>\n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">The test dataset will be used to evaluate the performance of the model.One of the metrics that will be tested would be accuracy which measures the fraction of predictions the model got right. Other metrics are as follows:   </p>\n\n<h3>Precision(P):</h3> \n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">The fraction of true positives (TP, correct predictions) from the total amount of relevant results, i.e., the sum of TP and false positives (FP). For multi-class classification problems, P is averaged among the classes. The following is the formula for precision.</p>\n\n<h4>\n  <center>\n    <span style=\"font-size: 1.5em\">\n      $P = \\frac{TP}{TP+FP}$\n    </span>\n  </center>\n</h4>\n\n\n<h3>Recall(R): </h3> \n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">The fraction of TP from the total amount of TP and false negatives (FN). For multi-class classification problems, R gets averaged among all the classes. The following is the formula for recall.</p>\n\n<h4>\n  <center>\n    <span style=\"font-size: 1.5em\">\n      $R = \\frac{TP}{TP+FN}$\n    </span>\n  </center>\n</h4>\n\n\n<h3>F1 score(F1): </h3>\n\n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">The harmonic mean of precision and recall. For multi-class classification problems, F1 gets averaged among all the classes. The following is the formula for F1 score.</p>\n\n<h4>\n  <center>\n    <span style=\"font-size: 1.5em\">\n      $F1 = 2 \\times \\frac{TP \\times FP}{TP + FP}$\n    </span>\n  </center>\n</h4>\n</div>","metadata":{"id":"_BWrofxS2vO0","papermill":{"duration":0.164457,"end_time":"2023-07-18T09:44:33.721214","exception":false,"start_time":"2023-07-18T09:44:33.556757","status":"completed"},"tags":[]}},{"cell_type":"code","source":"results = model.evaluate(test_images, verbose=0)\nprint(\"    Test Loss: {:.5f}\".format(results[0]))\nprint(\"Test Accuracy: {:.2f}%\".format(results[1] * 100))\nprint(\"  Test F1-score: {:.2f}%\".format(results[2] * 100))","metadata":{"id":"CS-g90hJ340B","outputId":"8b35403d-9dc0-4a65-b08e-ce223aef2bdb","papermill":{"duration":3.128423,"end_time":"2023-07-18T09:44:37.009672","exception":false,"start_time":"2023-07-18T09:44:33.881249","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:32:26.323330Z","iopub.execute_input":"2023-09-28T02:32:26.323715Z","iopub.status.idle":"2023-09-28T02:32:29.006318Z","shell.execute_reply.started":"2023-09-28T02:32:26.323680Z","shell.execute_reply":"2023-09-28T02:32:29.005279Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\n\nfile_size = os.path.getsize('mango_leaf_disease_ResNet50.h5') / (1024 * 1024)  # Convert bytes to MB\nprint(f\"Model file size: {file_size:.2f} MB\")","metadata":{"execution":{"iopub.status.busy":"2023-09-28T02:32:29.007916Z","iopub.execute_input":"2023-09-28T02:32:29.008261Z","iopub.status.idle":"2023-09-28T02:32:29.014767Z","shell.execute_reply.started":"2023-09-28T02:32:29.008228Z","shell.execute_reply":"2023-09-28T02:32:29.013302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"total_params = model.count_params()\nprint(f\"Total parameters: {total_params}\")","metadata":{"execution":{"iopub.status.busy":"2023-09-28T02:32:29.016394Z","iopub.execute_input":"2023-09-28T02:32:29.017055Z","iopub.status.idle":"2023-09-28T02:32:29.033855Z","shell.execute_reply.started":"2023-09-28T02:32:29.017018Z","shell.execute_reply":"2023-09-28T02:32:29.032656Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 📉Visualizing loss curves","metadata":{"id":"t5oGHvsG368q","papermill":{"duration":0.176326,"end_time":"2023-07-18T09:44:37.349357","exception":false,"start_time":"2023-07-18T09:44:37.173031","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Extracting F1-score from the history object\naccuracy = history.history['accuracy']\nval_accuracy = history.history['val_accuracy']\n\nloss = history.history['loss']\nval_loss = history.history['val_loss']\n\nepochs = range(len(accuracy))\nf1 = history.history['f1_score']\nval_f1 = history.history['val_f1_score']\n\nfig, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(18, 6))  # Create a 1x3 grid of plots\n\n# Plot accuracy\nax1.plot(epochs, accuracy, 'b', label='Training accuracy')\nax1.plot(epochs, val_accuracy, 'r', label='Validation accuracy')\nax1.set_title('Training and validation accuracy')\nax1.legend()\n\n# Plot loss\nax2.plot(epochs, loss, 'b', label='Training loss')\nax2.plot(epochs, val_loss, 'r', label='Validation loss')\nax2.set_title('Training and validation loss')\nax2.legend()\n\n# Plot F1-score\nax3.plot(epochs, f1, 'b', label='Training F1-score')\nax3.plot(epochs, val_f1, 'r', label='Validation F1-score')\nax3.set_title('Training and validation F1-score')\nax3.legend()\n\nplt.tight_layout()  # Adjust the padding between and around the subplots\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-09-28T02:32:29.035358Z","iopub.execute_input":"2023-09-28T02:32:29.035756Z","iopub.status.idle":"2023-09-28T02:32:30.069386Z","shell.execute_reply.started":"2023-09-28T02:32:29.035719Z","shell.execute_reply":"2023-09-28T02:32:30.068480Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 🔮Making predictions on the Test Data","metadata":{"id":"0BL7xgPz4Fv-","papermill":{"duration":0.163307,"end_time":"2023-07-18T09:44:39.186918","exception":false,"start_time":"2023-07-18T09:44:39.023611","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Predict the label of the test_images\npred = model.predict(test_images)\npred = np.argmax(pred,axis=1)\n\n# Map the labelz\nlabels = (train_images.class_indices)\nlabels = dict((v,k) for k,v in labels.items())\npred = [labels[k] for k in pred]\n\n# Display the result\nprint(f'The first 5 predictions: {pred[:5]}')","metadata":{"id":"KxAegJBB4HlW","outputId":"c836256c-be8e-4346-b6c2-d4c39d030db4","papermill":{"duration":3.463816,"end_time":"2023-07-18T09:44:42.817162","exception":false,"start_time":"2023-07-18T09:44:39.353346","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:32:30.070850Z","iopub.execute_input":"2023-09-28T02:32:30.072067Z","iopub.status.idle":"2023-09-28T02:32:32.258812Z","shell.execute_reply.started":"2023-09-28T02:32:30.072022Z","shell.execute_reply":"2023-09-28T02:32:32.257861Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"  # Display 25 random pictures from the dataset with their labels\nrandom_index = np.random.randint(0, len(test_df) - 1, 15)\nfig, axes = plt.subplots(nrows=3, ncols=5, figsize=(25, 15),\n                        subplot_kw={'xticks': [], 'yticks': []})\n\nfor i, ax in enumerate(axes.flat):\n    ax.imshow(plt.imread(test_df.Filepath.iloc[random_index[i]]))\n    if test_df.Label.iloc[random_index[i]] == pred[random_index[i]]:\n        color = \"green\"\n    else:\n        color = \"red\"\n    ax.set_title(f\"True: {test_df.Label.iloc[random_index[i]]}\\nPredicted: {pred[random_index[i]]}\", color=color)\nplt.show()\nplt.tight_layout()","metadata":{"id":"pWO4e4wb4Iln","outputId":"a58f3382-f991-43ac-e54f-6bd9a36dd25f","papermill":{"duration":4.892842,"end_time":"2023-07-18T09:44:47.875602","exception":false,"start_time":"2023-07-18T09:44:42.982760","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:32:32.260258Z","iopub.execute_input":"2023-09-28T02:32:32.260563Z","iopub.status.idle":"2023-09-28T02:32:34.054627Z","shell.execute_reply.started":"2023-09-28T02:32:32.260538Z","shell.execute_reply":"2023-09-28T02:32:34.053726Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import time\n\nstart_time = time.time()\nmodel.predict(test_images)  # or any other input, just for demonstration\nend_time = time.time()\n\nprint(f\"Inference time: {end_time - start_time:.2f} seconds\")","metadata":{"execution":{"iopub.status.busy":"2023-09-28T02:32:34.056220Z","iopub.execute_input":"2023-09-28T02:32:34.056893Z","iopub.status.idle":"2023-09-28T02:32:35.346557Z","shell.execute_reply.started":"2023-09-28T02:32:34.056856Z","shell.execute_reply":"2023-09-28T02:32:35.345659Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<div style=\"background-color:#fff1cc; padding: 20px; border-radius: 15px; box-shadow: 0 2px 4px 0 rgba(0, 0, 0, 0.1);\">\n    <h3 style=\"font-size:24px; font-family:verdana; color:#6F4E37;\"><b>📊Plotting the Classification Reports and Confusion Matrix</b></h3>\n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\"><b>Confusion matrix</b> and <b>classification report</b> are two important tools used for evaluating the performance of an image classification model.</p>\n\n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">A <b>confusion matrix</b> is a table that summarizes the number of correct and incorrect predictions made by a classification model on a set of test data. It is usually represented as a square matrix with rows and columns representing the predicted and true class labels, respectively. The entries of the matrix indicate the number of test samples that belong to a certain class, and how many of those were classified correctly or incorrectly by the model. A confusion matrix can provide a detailed breakdown of the performance of the model, including measures such as accuracy, precision, recall, and F1-score for each class. It can be used to identify specific areas where the model is making errors, and to diagnose problems with the model's predictions.</p>\n\n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">A <b>classification report</b> is a summary of the key performance metrics for a classification model, including precision, recall, and F1-score, as well as the overall accuracy of the model. It provides a concise overview of the model's performance, typically broken down by class, and can be used to quickly assess the strengths and weaknesses of the model. The report is often presented as a table, with each row representing a class and columns showing various performance metrics. The report may also include other metrics such as support (the number of test samples belonging to a particular class), and the macro- and micro-averages of the performance metrics across all classes.</p>\n\n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\">In image classification, both confusion matrix and classification report are important tools for evaluating the performance of the model, identifying areas for improvement, and making decisions about how to adjust the model's architecture or training parameters.</p>\n</div>","metadata":{"id":"jLYd5vYJ4KTu","papermill":{"duration":0.203504,"end_time":"2023-07-18T09:44:48.282299","exception":false,"start_time":"2023-07-18T09:44:48.078795","status":"completed"},"tags":[]}},{"cell_type":"code","source":"from sklearn.metrics import precision_recall_curve, average_precision_score\nfrom sklearn.preprocessing import LabelBinarizer\nimport matplotlib.pyplot as plt\n\n# Convert labels to one-hot encoding\nlb = LabelBinarizer()\ny_true_onehot = lb.fit_transform(test_df.Label)\n\n# Convert predictions to one-hot encoding\ny_pred_onehot = lb.transform(pred)\n\n# For storing average precision for each class\naverage_precision = {}\n\n# Get class names\nclass_names = lb.classes_\n\n# Plot PR curve for each class\nn_classes = y_true_onehot.shape[1]\nplt.figure(figsize=(15, 12))\n\nfor i in range(n_classes):\n    precision, recall, _ = precision_recall_curve(y_true_onehot[:, i], y_pred_onehot[:, i])\n    average_precision[i] = average_precision_score(y_true_onehot[:, i], y_pred_onehot[:, i])\n    \n    # Printing the precision and recall values for each class using the actual class names\n    print(f\"Class {class_names[i]}:\")\n    print(\"Precision:\", precision)\n    print(\"Recall:\", recall)\n    print(f\"Average Precision: {average_precision[i]:.2f}\\n\")\n    \n    plt.plot(recall, precision, lw=2, label=f'{class_names[i]} (average precision = {average_precision[i]:0.2f})')\n\nplt.xlabel(\"recall\")\nplt.ylabel(\"precision\")\nplt.legend(loc=\"best\")\nplt.title(\"Precision vs. Recall curve\")\nplt.grid(True)\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-09-28T02:32:35.348060Z","iopub.execute_input":"2023-09-28T02:32:35.348438Z","iopub.status.idle":"2023-09-28T02:32:35.905401Z","shell.execute_reply.started":"2023-09-28T02:32:35.348405Z","shell.execute_reply":"2023-09-28T02:32:35.904509Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_test = list(test_df.Label)\nreport = classification_report(y_test, pred, output_dict=True)\ndf = pd.DataFrame(report).transpose()\ndf","metadata":{"id":"bYWvkbXI4Ns2","outputId":"7e04d8a2-3263-4a31-d469-70d8140cb167","papermill":{"duration":0.237432,"end_time":"2023-07-18T09:44:48.721609","exception":false,"start_time":"2023-07-18T09:44:48.484177","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:32:35.906989Z","iopub.execute_input":"2023-09-28T02:32:35.907661Z","iopub.status.idle":"2023-09-28T02:32:35.936176Z","shell.execute_reply.started":"2023-09-28T02:32:35.907623Z","shell.execute_reply":"2023-09-28T02:32:35.935137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Remove the 'accuracy' row\ndf = df.drop(['accuracy'])\n\n# Remove the 'support' column\ndf = df.drop(['support'], axis=1)\n\nplt.figure(figsize=(10, 10))\nsns.heatmap(df, annot=True, cmap='YlOrBr')\nplt.title('Classification Report Heatmap', fontsize=20)\nplt.show()\n","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.764665,"end_time":"2023-07-18T09:44:49.690493","exception":false,"start_time":"2023-07-18T09:44:48.925828","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:32:35.941923Z","iopub.execute_input":"2023-09-28T02:32:35.943026Z","iopub.status.idle":"2023-09-28T02:32:36.427441Z","shell.execute_reply.started":"2023-09-28T02:32:35.942992Z","shell.execute_reply":"2023-09-28T02:32:36.426488Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def make_confusion_matrix(y_true, y_pred, classes=None, figsize=(20, 14), text_size=10, norm=False, savefig=False): \n    \"\"\"Makes a labelled confusion matrix comparing predictions and ground truth labels.\n\n    If classes is passed, confusion matrix will be labelled, if not, integer class values\n  will be used.\n\n  Args:\n    y_true: Array of truth labels (must be same shape as y_pred).\n    y_pred: Array of predicted labels (must be same shape as y_true).\n    classes: Array of class labels (e.g. string form). If `None`, integer labels are used.\n    figsize: Size of output figure (default=(10, 10)).\n    text_size: Size of output figure text (default=15).\n    norm: normalize values or not (default=False).\n    savefig: save confusion matrix to file (default=False).\n  \n  Returns:\n    A labelled confusion matrix plot comparing y_true and y_pred.\n\n  Example usage:\n    make_confusion_matrix(y_true=test_labels, # ground truth test labels\n                          y_pred=y_preds, # predicted labels\n                          classes=class_names, # array of class label names\n                          figsize=(15, 15),\n                          text_size=10)\n    \"\"\"  \n  # Create the confustion matrix\n\n    # Use the fivethirtyeight style\n    plt.style.use('fivethirtyeight')\n    cm = confusion_matrix(y_true, y_pred)\n    cm_norm = cm.astype(\"float\") / cm.sum(axis=1)[:, np.newaxis] # normalize it\n    n_classes = cm.shape[0] # find the number of classes we're dealing with\n\n    # Plot the figure and make it pretty\n    fig, ax = plt.subplots(figsize=figsize)\n    cax = ax.matshow(cm, cmap=plt.cm.YlOrBr) # colors will represent how 'correct' a class is, darker == better\n    fig.colorbar(cax)\n\n    # Are there a list of classes?\n    if classes:\n        labels = classes\n    else:\n        labels = np.arange(cm.shape[0])\n  \n    # Label the axes\n    ax.set(title=\"Confusion Matrix\",\n         xlabel=\"Predicted label\",\n         ylabel=\"True label\",\n         xticks=np.arange(n_classes), # create enough axis slots for each class\n         yticks=np.arange(n_classes), \n         xticklabels=labels, # axes will labeled with class names (if they exist) or ints\n         yticklabels=labels)\n  \n    # Make x-axis labels appear on bottom\n    ax.xaxis.set_label_position(\"bottom\")\n    ax.xaxis.tick_bottom()\n    ### Added: Rotate xticks for readability & increase font size (required due to such a large confusion matrix)\n    plt.xticks(rotation=90, fontsize=text_size)\n    plt.yticks(fontsize=text_size)\n\n    # Set the threshold for different colors\n    threshold = (cm.max() + cm.min()) / 2.\n\n    # Plot the text on each cell\n    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n        if norm:\n            plt.text(j, i, f\"{cm[i, j]} ({cm_norm[i, j]*100:.1f}%)\",\n                horizontalalignment=\"center\",\n                color=\"white\" if cm[i, j] > threshold else \"black\",\n                size=text_size)\n        else:\n            plt.text(j, i, f\"{cm[i, j]}\",\n              horizontalalignment=\"center\",\n              color=\"white\" if cm[i, j] > threshold else \"black\",\n              size=text_size)\n\n  # Save the figure to the current working directory\n    if savefig:\n        fig.savefig(\"confusion_matrix.png\")\n","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.222989,"end_time":"2023-07-18T09:44:50.122370","exception":false,"start_time":"2023-07-18T09:44:49.899381","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:32:36.429004Z","iopub.execute_input":"2023-09-28T02:32:36.429674Z","iopub.status.idle":"2023-09-28T02:32:36.443363Z","shell.execute_reply.started":"2023-09-28T02:32:36.429637Z","shell.execute_reply":"2023-09-28T02:32:36.442438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"make_confusion_matrix(y_test, pred, list(labels.values()))","metadata":{"papermill":{"duration":1.306964,"end_time":"2023-07-18T09:44:51.636470","exception":false,"start_time":"2023-07-18T09:44:50.329506","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:32:36.444775Z","iopub.execute_input":"2023-09-28T02:32:36.445339Z","iopub.status.idle":"2023-09-28T02:32:37.214026Z","shell.execute_reply.started":"2023-09-28T02:32:36.445305Z","shell.execute_reply":"2023-09-28T02:32:37.212916Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<div style=\"background-color:#fff1cc; padding: 20px; border-radius: 15px; box-shadow: 0 2px 4px 0 rgba(0, 0, 0, 0.1);\">\n    <h3 style=\"font-size:24px; font-family:verdana; color:#6F4E37;\"><b>☀️Grad-Cam Visualization</b></h3>\n<p style=\"font-size:20px; font-family:verdana; line-height: 1.7em\"><b>Grad-CAM (Gradient-weighted Class Activation Mapping)</b> is a technique used to visualize the regions of an input image that were most relevant for a neural network's prediction. It allows you to see which regions of the image the model focused on while making its prediction. Grad-CAM is a modification of the CAM technique that extends the latter to any model that uses a convolutional neural network (CNN) as its underlying architecture.</p>\n</div>","metadata":{"papermill":{"duration":0.231634,"end_time":"2023-07-18T09:44:52.109782","exception":false,"start_time":"2023-07-18T09:44:51.878148","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def get_img_array(img_path, size):\n    img = tf.keras.preprocessing.image.load_img(img_path, target_size=size)\n    array = tf.keras.preprocessing.image.img_to_array(img)\n    # We add a dimension to transform our array into a \"batch\"\n    # of size \"size\"\n    array = np.expand_dims(array, axis=0)\n    return array\n\ndef make_gradcam_heatmap(img_array, model, last_conv_layer_name, pred_index=None):\n    # First, we create a model that maps the input image to the activations\n    # of the last conv layer as well as the output predictions\n    grad_model = tf.keras.models.Model(\n        [model.inputs], [model.get_layer(last_conv_layer_name).output, model.output]\n    )\n\n    # Then, we compute the gradient of the top predicted class for our input image\n    # with respect to the activations of the last conv layer\n    with tf.GradientTape() as tape:\n        last_conv_layer_output, preds = grad_model(img_array)\n        if pred_index is None:\n            pred_index = tf.argmax(preds[0])\n        class_channel = preds[:, pred_index]\n    # This is the gradient of the output neuron (top predicted or chosen)\n    # with regard to the output feature map of the last conv layer\n    grads = tape.gradient(class_channel, last_conv_layer_output)\n\n    # This is a vector where each entry is the mean intensity of the gradient\n    # over a specific feature map channel\n    pooled_grads = tf.reduce_mean(grads, axis=(0, 1, 2))\n\n    # We multiply each channel in the feature map array\n    # by \"how important this channel is\" with regard to the top predicted class\n    # then sum all the channels to obtain the heatmap class activation\n    last_conv_layer_output = last_conv_layer_output[0]\n    heatmap = last_conv_layer_output @ pooled_grads[..., tf.newaxis]\n    heatmap = tf.squeeze(heatmap)\n\n    # For visualization purpose, we will also normalize the heatmap between 0 & 1\n    heatmap = tf.maximum(heatmap, 0) / tf.math.reduce_max(heatmap)\n    return heatmap.numpy()\ndef save_and_display_gradcam(img_path, heatmap, cam_path=\"cam.jpg\", alpha=0.4):\n    # Load the original image\n    img = tf.keras.preprocessing.image.load_img(img_path)\n    img = tf.keras.preprocessing.image.img_to_array(img)\n\n    # Rescale heatmap to a range 0-255\n    heatmap = np.uint8(255 * heatmap)\n\n    # Use jet colormap to colorize heatmap\n    jet = cm.get_cmap(\"jet\")\n\n    # Use RGB values of the colormap\n    jet_colors = jet(np.arange(256))[:, :3]\n    jet_heatmap = jet_colors[heatmap]\n\n    # Create an image with RGB colorized heatmap\n    jet_heatmap = tf.keras.preprocessing.image.array_to_img(jet_heatmap)\n    jet_heatmap = jet_heatmap.resize((img.shape[1], img.shape[0]))\n    jet_heatmap = tf.keras.preprocessing.image.img_to_array(jet_heatmap)\n\n    # Superimpose the heatmap on original image\n    superimposed_img = jet_heatmap * alpha + img\n    superimposed_img = tf.keras.preprocessing.image.array_to_img(superimposed_img)\n    # Save the superimposed image\n    superimposed_img.save(cam_path)\n\n    # Display Grad CAM\n#     display(Image(cam_path))\n    \n    return cam_path\n    \npreprocess_input = tf.keras.applications.efficientnet.preprocess_input\ndecode_predictions = tf.keras.applications.efficientnet.decode_predictions\n    \nlast_conv_layer_name = \"conv5_block3_out\"\n\nimg_size = (224,224, 3)\n\n# Remove last layer's softmax\nmodel.layers[-1].activation = None","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.256638,"end_time":"2023-07-18T09:44:52.587988","exception":false,"start_time":"2023-07-18T09:44:52.331350","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:32:37.215902Z","iopub.execute_input":"2023-09-28T02:32:37.216309Z","iopub.status.idle":"2023-09-28T02:32:37.240107Z","shell.execute_reply.started":"2023-09-28T02:32:37.216276Z","shell.execute_reply":"2023-09-28T02:32:37.238383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import textwrap\n\n# Display the part of the pictures used by the neural network to classify the pictures\nfig, axes = plt.subplots(nrows=3, ncols=5, figsize=(15, 10),\n                        subplot_kw={'xticks': [], 'yticks': []})\n\nfor i, ax in enumerate(axes.flat):\n    img_path = test_df.Filepath.iloc[random_index[i]]\n    img_array = preprocess_input(get_img_array(img_path, size=img_size))\n    heatmap = make_gradcam_heatmap(img_array, model, last_conv_layer_name)\n    cam_path = save_and_display_gradcam(img_path, heatmap)\n    ax.imshow(plt.imread(cam_path))\n    \n    # Wrap the title text\n    title_text = f\"True: {test_df.Label.iloc[random_index[i]]}\\nPredicted: {pred[random_index[i]]}\"\n    wrapped_title = '\\n'.join(textwrap.wrap(title_text, 20))  # Adjust the number as needed\n    ax.set_title(wrapped_title)\n    \nplt.tight_layout()\nplt.show()\n","metadata":{"_kg_hide-input":true,"papermill":{"duration":12.086446,"end_time":"2023-07-18T09:45:04.906765","exception":false,"start_time":"2023-07-18T09:44:52.820319","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-09-28T02:32:37.242216Z","iopub.execute_input":"2023-09-28T02:32:37.243423Z","iopub.status.idle":"2023-09-28T02:32:43.419095Z","shell.execute_reply.started":"2023-09-28T02:32:37.243357Z","shell.execute_reply":"2023-09-28T02:32:43.417249Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***\n\n<div style=\"color:white;\n           display:fill;\n           border-radius:5px;\n           background-color:#5642C5;\n           font-size:110%;\n           font-family:Verdana;\n           letter-spacing:0.5px\">\n        <p style=\"padding: 10px;\n              color:white;\">\n            Thanks for viewing my work. If you like it, consider sharing it to others or give feedback to improve the notebook. Have a beautiful day my friend.\n        </p>\n    </div>\n\n<center><img src='https://media4.giphy.com/media/M9gbBd9nbDrOTu1Mqx/giphy.gif?cid=790b7611704aa2ca4e403287801480a6c753abf45f3e6242&rid=giphy.gif&ct=s' \n     height=30px width=160px /></center>","metadata":{"papermill":{"duration":0.235766,"end_time":"2023-07-18T09:45:05.376766","exception":false,"start_time":"2023-07-18T09:45:05.141000","status":"completed"},"tags":[]}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}