Agents/robot_vacuum.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intelligent Agents: Reflex-Based Agents for the Vacuum-cleaner World\n",
    "\n",
    "Student Name: [Add your name]\n",
    "\n",
    "I have used the following AI tools: [list tools]\n",
    "\n",
    "I understand that my submission needs to be my own work: [your initials]\n",
    "\n",
    "## Instructions\n",
    "\n",
    "Total Points: Undergrads 100 + 5 bonus / Graduate students 110\n",
    "\n",
    "Complete this notebook. Use the provided notebook cells and insert additional code and markdown cells as needed. Submit the completely rendered notebook as a HTML file. \n",
    "\n",
    "### Using Visual Studio Code\n",
    "\n",
    "If you use VS code then you can use `Export` (click on `...` in the menu bar) to save your notebook as a HTML file. Note that you have to run all blocks before so the HTML file contains your output.\n",
    "\n",
    "### Using Google Colab\n",
    "\n",
    "In Colab you need to save the notebook on GoogleDrive to work with it. For this you need to mount your google dive and change to the correct directory by uncommenting the following lines and running the code block."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from google.colab import drive\n",
    "# import os\n",
    "#\n",
    "# drive.mount('/content/drive')\n",
    "# os.chdir('/content/drive/My Drive/Colab Notebooks/')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you are done with the assignment and have run all code blocks using `Runtime/Run all`, you can convert the file on your GoogleDrive into HTML be uncommenting the following line and running the block."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !jupyter nbconvert --to html Copy\\ of\\ robot_vacuum.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may have to fix the file location or the file name to match how it looks on your GoogleDrive. You can navigate in Colab to your GoogleDrive using the little folder symbol in the navigation bar to the left."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "In this assignment you will implement a simulator environment for an automatic vacuum cleaner robot, a set of different reflex-based agent programs, and perform a comparison study for cleaning a single room. Focus on the __cleaning phase__ which starts when the robot is activated and ends when the last dirty square in the room has been cleaned. Someone else will take care of the agent program needed to navigate back to the charging station after the room is clean.\n",
    "\n",
    "## PEAS description of the cleaning phase\n",
    "\n",
    "__Performance Measure:__ Each action costs 1 energy unit. The performance is measured as the sum of the energy units used to clean the whole room.\n",
    "\n",
    "__Environment:__ A room with $n \\times n$ squares where $n = 5$. Dirt is randomly placed on each square with probability $p = 0.2$. For simplicity, you can assume that the agent knows the size and the layout of the room (i.e., it knows $n$). To start, the agent is placed on a random square.\n",
    "\n",
    "__Actuators:__ The agent can clean the current square (action `suck`) or move to an adjacent square by going `north`, `east`, `south`, or `west`.\n",
    "\n",
    "__Sensors:__ Four bumper sensors, one for north, east, south, and west; a dirt sensor reporting dirt in the current square.  \n",
    "\n",
    "\n",
    "## The agent program for a simple randomized agent\n",
    "\n",
    "The agent program is a function that gets sensor information (the current percepts) as the arguments. The arguments are:\n",
    "\n",
    "* A dictionary with boolean entries for the for bumper sensors `north`, `east`, `west`, `south`. E.g., if the agent is on the north-west corner, `bumpers` will be `{\"north\" : True, \"east\" : False, \"south\" : False, \"west\" : True}`.\n",
    "* The dirt sensor produces a boolean.\n",
    "\n",
    "The agent returns the chosen action as a string.\n",
    "\n",
    "Here is an example implementation for the agent program of a simple randomized agent:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make sure numpy is installed\n",
    "!pip install -q numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "actions = [\"north\", \"east\", \"west\", \"south\", \"suck\"]\n",
    "\n",
    "def simple_randomized_agent(bumpers, dirty):\n",
    "    return np.random.choice(actions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'north'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# define percepts (current location is NW corner and it is dirty)\n",
    "bumpers = {\"north\" : True, \"east\" : False, \"south\" : False, \"west\" : True}\n",
    "dirty = True\n",
    "\n",
    "# call agent program function with percepts and it returns an action\n",
    "simple_randomized_agent(bumpers, dirty)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Note:__ This is not a rational intelligent agent. It ignores its sensors and may bump into a wall repeatedly or not clean a dirty square. You will be asked to implement rational agents below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple environment example\n",
    "\n",
    "We implement a simple simulation environment that supplies the agent with its percepts.\n",
    "The simple environment is infinite in size (bumpers are always `False`) and every square is always dirty, even if the agent cleans it. The environment function returns a different performance measure than the one specified in the PEAS description! Since the room is infinite and all squares are constantly dirty, the agent can never clean the whole room. Your implementation needs to implement the **correct performance measure.** The energy budget of the agent is specified as `max_steps`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def simple_environment(agent_function, max_steps, verbose = True):\n",
    "    num_cleaned = 0\n",
    "    \n",
    "    for i in range(max_steps):\n",
    "        dirty = True\n",
    "        bumpers = {\"north\" : False, \"south\" : False, \"west\" : False, \"east\" : False}\n",
    "\n",
    "        action = agent_function(bumpers, dirty)\n",
    "        if (verbose): print(\"step\", i , \"- action:\", action) \n",
    "        \n",
    "        if (action == \"suck\"): \n",
    "            num_cleaned = num_cleaned + 1\n",
    "        \n",
    "    return num_cleaned\n",
    "        \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Do one simulation run with a simple randomized agent that has enough energy for 20 steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "step 0 - action: suck\n",
      "step 1 - action: east\n",
      "step 2 - action: west\n",
      "step 3 - action: west\n",
      "step 4 - action: suck\n",
      "step 5 - action: north\n",
      "step 6 - action: east\n",
      "step 7 - action: south\n",
      "step 8 - action: north\n",
      "step 9 - action: north\n",
      "step 10 - action: west\n",
      "step 11 - action: north\n",
      "step 12 - action: north\n",
      "step 13 - action: north\n",
      "step 14 - action: west\n",
      "step 15 - action: suck\n",
      "step 16 - action: east\n",
      "step 17 - action: east\n",
      "step 18 - action: south\n",
      "step 19 - action: west\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "simple_environment(simple_randomized_agent, max_steps = 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tasks\n",
    "\n",
    "## General [10 Points]\n",
    "\n",
    "1. Make sure that you use the latest version of this notebook. \n",
    "2. Your implementation can use libraries like math, numpy, scipy, but not libraries that implement intelligent agents or complete search algorithms. Try to keep the code simple! In this course, we want to learn about the algorithms and we often do not need to use object-oriented design.\n",
    "3. You notebook needs to be formatted professionally. \n",
    "    - Add additional markdown blocks for your description, comments in the code, add tables and use mathplotlib to produce charts where appropriate\n",
    "    - Do not show debugging output or include an excessive amount of output.\n",
    "    - Check that your submitted file is readable and contains all figures.\n",
    "4. Document your code. Use comments in the code and add a discussion of how your implementation works and your design choices.\n",
    "\n",
    "\n",
    "## Task 1: Implement a simulation environment [20 Points]\n",
    "\n",
    "The simple environment above is not very realistic. Your environment simulator needs to follow the PEAS description from above. It needs to:\n",
    "\n",
    "* Initialize the environment by storing the state of each square (clean/dirty) and making some dirty. ([Help with random numbers and arrays in Python](https://github.com/mhahsler/CS7320-AI/blob/master/HOWTOs/random_numbers_and_arrays.ipynb))\n",
    "* Keep track of the agent's position.\n",
    "* Call the agent function repeatedly and provide the agent function with the sensor inputs.  \n",
    "* React to the agent's actions. E.g, by removing dirt from a square or moving the agent around unless there is a wall in the way.\n",
    "* Keep track of the performance measure. That is, track the agent's actions until all dirty squares are clean and count the number of actions it takes the agent to complete the task.\n",
    "\n",
    "The easiest implementation for the environment is to hold an 2-dimensional array to represent if squares are clean or dirty and to call the agent function in a loop until all squares are clean or a predefined number of steps have been reached (i.e., the robot runs out of energy).\n",
    "\n",
    "The simulation environment should be a function like the `simple_environment()` and needs to work with the simple randomized agent program from above. **Use the same environment for all your agent implementations in the tasks below.**\n",
    "\n",
    "*Note on debugging:* Debugging is difficult. Make sure your environment prints enough information when you use `verbose = True`. Also, implementing a function that the environment can use to displays the room with dirt and the current position of the robot at every step is very useful.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code and description goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show that your environment works with the simple randomized agent from above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code and description goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 2:  Implement a simple reflex agent [10 Points] \n",
    "\n",
    "The simple reflex agent randomly walks around but reacts to the bumper sensor by not bumping into the wall and to dirt with sucking. Implement the agent program as a function.\n",
    "\n",
    "_Note:_ Agents cannot directly use variable in the environment. They only gets the percepts as the arguments to the agent function. Use the function signature for the `simple_randomized_agent` function above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code and description goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show how the agent works with your environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code and description goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 3: Implement a model-based reflex agent [20 Points]\n",
    "\n",
    "Model-based agents use a state to keep track of what they have done and perceived so far. Your agent needs to find out where it is located and then keep track of its current location. You also need a set of rules based on the state and the percepts to make sure that the agent will clean the whole room. For example, the agent can move to a corner to determine its location and then it can navigate through the whole room and clean dirty squares.\n",
    "\n",
    "Describe how you define the __agent state__ and how your agent works before implementing it. ([Help with implementing state information on Python](https://github.com/mhahsler/CS7320-AI/blob/master/HOWTOs/store_agent_state_information.ipynb))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your short description of the state and your implementation goes here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show how the agent works with your environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 4: Simulation study [30 Points]\n",
    "\n",
    "Compare the performance (the performance measure is defined in the PEAS description above) of the agents using  environments of different size. Do at least $5 \\times 5$, $10 \\times 10$ and\n",
    "$100 \\times 100$. Use 100 random runs for each. Present the results using tables and graphs. Discuss the differences between the agents. \n",
    "([Help with charts and tables in Python](https://github.com/mhahsler/CS7320-AI/blob/master/HOWTOs/charts_and_tables.ipynb))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fill out the following table with the average performance measure for 100 random runs (you may also create this table with code):\n",
    "\n",
    "| Size     | Randomized Agent | Simple Reflex Agent | Model-based Reflex Agent |\n",
    "|----------|------------------|---------------------|--------------------------|\n",
    "| 5x5     | | | |\n",
    "| 10x10   | | | |\n",
    "| 100x100 | | | |\n",
    "\n",
    "Add charts to compare the performance of the different agents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your graphs and discussion of the results goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task 5: Robustness of the agent implementations [10 Points] \n",
    "\n",
    "Describe how **your agent implementations** will perform \n",
    "\n",
    "* if it is put into a rectangular room with unknown size, \n",
    "* if the cleaning area can have an irregular shape (e.g., a hallway connecting two rooms), or \n",
    "* if the room contains obstacles (i.e., squares that it cannot pass through and trigger the bumper sensors).\n",
    "* if the dirt sensor is not perfect and gives 10% of the time a wrong reading (clean when it is dirty or dirty when it is clean).\n",
    "* if the bumper sensor is not perfect and 10% of the time does not report a wall when there is one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Answers goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Advanced task: Imperfect Dirt Sensor\n",
    "\n",
    "* __Graduate students__ need to complete this task [10 points]\n",
    "* __Undergraduate students__ can attempt this as a bonus task [max +5 bonus points].\n",
    "\n",
    "1. Change your simulation environment to run experiments for the following problem: The dirt sensor has a 10% chance of giving the wrong reading. Perform experiments to observe how this changes the performance of the three implementations. Your model-based reflex agent is likely not able to clean the whole room, so you need to measure performance differently as a tradeoff between energy cost and number of uncleaned squares. \n",
    "\n",
    "2. Design an implement a solution for your model-based agent that will clean better. Show the improvement with experiments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code and discussion goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More advanced implementation tasks\n",
    "\n",
    "* __Obstacles:__ Change your simulation environment to run experiments for the following problem: Add random obstacle squares that also trigger the bumper sensor. The agent does not know where the obstacles are. Perform experiments to observe how this changes the performance of the three implementations. Describe what would need to be done to perform better with obstacles. Add code if you can. \n",
    "\n",
    "* __Agent for and environment with obstacles:__ Implement an agent for an environment where the agent does not know how large the environment is (we assume it is rectangular), where it starts or where the obstacles are. An option would be to always move to the closest unchecked/uncleaned square (note that this is actually depth-first search).\n",
    "\n",
    "* __Utility-based agent:__ Change the environment for a $5 \\times 5$ room, so each square has a fixed probability of getting dirty again. For the implementation, we give the environment a 2-dimensional array of probabilities. The utility of a state is defined as the number of currently clean squares in the room. Implement a utility-based agent that maximizes the expected utility over one full charge which lasts for 100000 time steps. To do this, the agent needs to learn the probabilities with which different squares get dirty again. This is very tricky!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your ideas/code"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "AI",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "toc-autonumbering": false
 },
 "nbformat": 4,
 "nbformat_minor": 4
}