StartHere_ResBaz24Python.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n5iMvwh_hzyO"
      },
      "source": [
        "# Welcome to our session \"Introduction to Python\"! 🐍\n",
        "\n",
        "## Today's example\n",
        "- **import** some data from **spreadsheets**\n",
        "- **plot** data (bar charts, scatter plots)\n",
        "- perform some basic **statistical** analysis\n",
        "- try to make **predictions** using a simple Machine Learning model 🧐\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rLYlAPZnnLNd"
      },
      "source": [
        "<a target=\"_blank\" href=\"https://colab.research.google.com/github/UoA-eResearch/ResBaz24Python/blob/main/StartHere_ResBaz24Python.ipynb\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
        "</a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rqGnLKHVhzyS"
      },
      "source": [
        "# Housekeeping\n",
        "\n",
        "- This is an interactive session; make the most of it; just listening is not enough! There won't be a recording. We are here to help you learn 🧠\n",
        "- Cameras on! 📸\n",
        "- If you have any questions, please ask them in the chat 💬\n",
        "- Please mute your microphone 🎤  \n",
        "- Be kind 😊\n",
        "  - all details of the Code of Conduct 🗞️  can be found [here](https://resbaz.auckland.ac.nz/about/)\n",
        "- This is a 4h session, we have 5 min breaks every hour and a 10 min break after 2h 🕒\n",
        "- You will be able to vote/have an influence on the content of this session 🗳️\n",
        "- This notebook and other resources will be shared with you later, always prioritise following-along over note-taking, etc. 📝\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "khkjWP0lhzyT"
      },
      "source": [
        "# What you can expect\n",
        "<div>\n",
        "<img src=\"https://raw.githubusercontent.com/UoA-eResearch/ResBaz24Python/main/Assets/SessionExpectations.svg\" width=\"800\"/>\n",
        "</div>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p3hOcvRLhzyU"
      },
      "source": [
        "# Defining goals for today's session (will be revisited at the end of the session)\n",
        "\n",
        "- we want you to...\n",
        "  - ... have fun! 🎉\n",
        "  - ... follow along with us and learn some Python 🐍 while working through a few examples\n",
        "    - learning by doing/ *\"live coding\"*\n",
        "    - we mostly follow [The Carpentries style of teaching](https://swcarpentry.github.io/swc-releases/2017.02/instructor-training/15-practices/)\n",
        "\n",
        "- we won't find the time to\n",
        "  - ... start with the basics of coding\n",
        "    - at the University of Auckland, we have these regular sessions:\n",
        "      - [Introduction to Python](https://research-hub.auckland.ac.nz/digital-research-skills/introduction-to-python) (8h)\n",
        "      - [Machine Learning (ML)](https://research-hub.auckland.ac.nz/digital-research-skills/machine-learning-workshop)\n",
        "    - hand-selected links to resoures will follow in the very end\n",
        "  - ... discuss different environments for Python\n",
        "    - we will use Google Colab ([as mentioned](https://resbaz.auckland.ac.nz/setup/) you need a Google account)\n",
        "    - there are use-cases where you won't use a Jupyter Notebook (like this one right here), you can write scripts and run them on the command-line\n",
        "    - you can use an Integrated Development Environment (IDE) like [Visual Studio Code](https://code.visualstudio.com/) or [PyCharm](https://www.jetbrains.com/pycharm/)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Us89EIi_hzyU"
      },
      "source": [
        "# In 3 Statements: How is Python?\n",
        "1. **Awesome**, because it is **abstract** and **versatile**\n",
        "1. The **community** is **huge** (and **AI** is well trained on it): MAKE SURE IT IS OK TO USE THAT ON YOUR POTENTIALLY SENSITIVE DATA\n",
        "2. It is **still a programming** language; you need to learn the **syntax** and the **logic** behind it\n",
        "\n",
        "<div>\n",
        "<img src=\"https://github.com/UoA-eResearch/ResBaz24Python/blob/main/Assets/MemeCoding.JPG?raw=1\" width=\"500\"/>\n",
        "</div>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8T5g2R98hzyV"
      },
      "source": [
        "## Intermezzo: What is a Jupyter Notebook? And or a Google Colab Notebook?\n",
        "\n",
        "- we oversimplify here\n",
        "- Jupyter Notebook\n",
        "  - you can **write** and **run** your **code**, see the **output**, and add **textual descriptions**\n",
        "  - you can **share** your work with others\n",
        "  - you can run it on your **local** machine or on a **server or Virtual Machine**\n",
        "- a web-based environment such as Google Colab or [JupyterHub](https://research-hub.auckland.ac.nz/research-software-and-computing/advanced-compute/nectar-jupyterhub)/[BinderHub](https://research-hub.auckland.ac.nz/research-software-and-computing/advanced-compute/nectar-binderhub), etc.\n",
        "- From a different perspective, cells that can contain\n",
        "  - **markdown** (like this one) which is a way of formatting text with some special characters\n",
        "  - **code**\n",
        "  - you have a **play** button ▶️ to run the code in a cell\n",
        "    - if you look closely, once that is done, you see a number in the brackets `[ ]` on the left side of the cell\n",
        "    - the **sequence matters**\n",
        "      - this is powerful but can also be a source of errors\n",
        "      - you can go back and change only some bits of your code and run it again\n",
        "      - you have to familiarise yourself with this kind of non-linear execution (especially if you are used to writing scripts)\n",
        "\n",
        " Now, gogogo! 🏃‍♂️"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kA_1W2NIhzyV"
      },
      "source": [
        "# Poll #1 Why do you attend this? 🤔"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "tDcPRlid9L2Y"
      },
      "outputs": [],
      "source": [
        "#@markdown ## Enter your answer here and run the cell to send it to us\n",
        "answer = \"\" #@param {type: \"string\"}\n",
        "import requests\n",
        "requests.post(f\"http://resbaz.auckland-cer.cloud.edu.au/\", params={\n",
        "  \"question\": 1499,\n",
        "  \"answer\": answer,\n",
        "  \"participant_name\": \"Anonymous\",\n",
        "  \"is_correct\": True\n",
        "}).json()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KqFB8nKchzyW",
        "vscode": {
          "languageId": "plaintext"
        }
      },
      "source": [
        "# Bare minimum to get started\n",
        "\n",
        "## Variables and data types"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bAsiJoeehzyX"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uNUDlTwahzya"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ukbqf4cChzyb"
      },
      "source": [
        "### more detailed explanation:\n",
        "We just imported a package/library/an external component/something that someone else has built. One repository of such packages is the Python Package Index (PyPI) [pypi.org](https://pypi.org/)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ahNPivfkhzyc"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hH_swRAxhzyc"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JTeEF8Eehzyc"
      },
      "source": [
        "# \"Over to you\"\n",
        "\n",
        "- for some examples, we have 'gamified' our learning experience and have created some neat way of you testing your knowledge\n",
        "- you can input your result\n",
        "- these will be uploaded to a server\n",
        "- we see the result\n",
        "- we share the results (later)\n",
        "- and: Yes! ~~All~~ Most of that was also done in Python, yes, Application Programming Interfaces (APIs) calls and all! 🤯"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GL2kRuXfiBOh"
      },
      "source": [
        "# Exercices Part 1"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "9MvJ7ybnhzyZ"
      },
      "outputs": [],
      "source": [
        "#@markdown ## Enter your name here\n",
        "\n",
        "participant_name = \"\" #@param {type: \"string\"}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "dd_q58PfHpj0"
      },
      "outputs": [],
      "source": [
        "#@markdown ## On a scale from 0 (really bad) to 10 (awesome), how do you like ResBaz so far?\n",
        "\n",
        "rate = 0  #@param {type: \"slider\", min: 0, max: 10}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "M5y3YZ-uIVD7"
      },
      "outputs": [],
      "source": [
        "# @title Submit your opnion on ResBaz\n",
        "import requests\n",
        "if rate == 10:\n",
        "  is_correct = True\n",
        "else:\n",
        "  is_correct = False\n",
        "requests.post(f\"http://resbaz.auckland-cer.cloud.edu.au/\", params={\n",
        "  \"question\": 0,\n",
        "  \"answer\": rate,\n",
        "  \"participant_name\": participant_name,\n",
        "  \"is_correct\": is_correct\n",
        "}).json()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "34kWvqQIiAby"
      },
      "outputs": [],
      "source": [
        "# Exercise 1: sum 1486 and 1985\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "wipoDt5AiJdh"
      },
      "outputs": [],
      "source": [
        "# @title Check Answer for Exercise 1\n",
        "import requests\n",
        "answer = list(Out.values())[-1]\n",
        "if answer == 3471:\n",
        "  is_correct = True\n",
        "else:\n",
        "  is_correct = False\n",
        "requests.post(f\"http://resbaz.auckland-cer.cloud.edu.au/\", params={\n",
        "  \"question\": 1,\n",
        "  \"answer\": answer,\n",
        "  \"participant_name\": participant_name,\n",
        "  \"is_correct\": is_correct\n",
        "}).json()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6Z5AU4xiijvr"
      },
      "outputs": [],
      "source": [
        "# Exercise 2: import math and use the math.sqrt()function to get the square root of 81\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "EI-a2LLoiwQ1"
      },
      "outputs": [],
      "source": [
        "# @title Check Answer Exercise 2\n",
        "answer = list(Out.values())[-1]\n",
        "if answer == 9.0:\n",
        "  is_correct = True\n",
        "else:\n",
        "  is_correct = False\n",
        "requests.post(f\"http://resbaz.auckland-cer.cloud.edu.au/\", params={\n",
        "  \"question\": 2,\n",
        "  \"answer\": answer,\n",
        "  \"participant_name\": participant_name,\n",
        "  \"is_correct\": is_correct\n",
        "}).json()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kbqrwoGFinBx"
      },
      "outputs": [],
      "source": [
        "# Exercise 3: use the print function to print \"Hello world!\"\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "dIf41F2pizdP"
      },
      "outputs": [],
      "source": [
        "# @title Check Answer for Exercise 3\n",
        "answer = list(Out.values())[-1]\n",
        "if answer == \"Hello world!\":\n",
        "  is_correct = True\n",
        "else:\n",
        "  is_correct = False\n",
        "requests.post(f\"http://resbaz.auckland-cer.cloud.edu.au/\", params={\n",
        "  \"question\": 3,\n",
        "  \"answer\": answer,\n",
        "  \"participant_name\": participant_name,\n",
        "  \"is_correct\": is_correct\n",
        "}).json()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "--bm6q36hzyd"
      },
      "source": [
        "## Recap\n",
        "- we have used Python as a calculator\n",
        "- we have imported a library (i.e. math)\n",
        "- we have used a variable to store a value\n",
        "- we have printed the value of a variable\n",
        "\n",
        "## Next\n",
        "- we will store a value in a variable\n",
        "- we will use a different data type"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iRKwbZZ9hzyd"
      },
      "source": [
        "# Variables and data types\n",
        "\n",
        "- Python is dynamically typed (aka duck-typed 🦆)\n",
        "- We don't have to specify what kind of data we want to store in a variable\n",
        "- Python will figure it out for us"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DqW7giBIhzyd"
      },
      "outputs": [],
      "source": [
        "# assign variable\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LlsgN2s0hzye"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MWECwmLehzye"
      },
      "outputs": [],
      "source": [
        "# print variable with f strings"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kGMWM2pdhzye"
      },
      "outputs": [],
      "source": [
        "# and we can make even shorten this further; remember if we take this to an extreme, it is considered Code Golf (https://en.wikipedia.org/wiki/Code_golf) 🏌️\n",
        "# let's create a new variable that holds the non-rounded result\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s3_HWzC_hzyf"
      },
      "source": [
        "This does not only work for numbers or maths, etc. but also for text:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Bh-PPJByhzyf"
      },
      "outputs": [],
      "source": [
        "# assign a string\n",
        "\n",
        "# assign a integer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TM2ikZ1lhzyg"
      },
      "outputs": [],
      "source": [
        "# Let's see what happens when we try to add a string and a number\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ht8NbM5ghzyh"
      },
      "source": [
        "On a side-note: Python errors tend to be 'verbose', they might seem intimidating. Read them bottom to top. Use online resources to get help. There is a big community!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "n9xvlyDOhzyh"
      },
      "outputs": [],
      "source": [
        "# Make Python show us the type of the variable\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2jCr6GVPhzyh"
      },
      "source": [
        "and another example of data types and consequences"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "G58XVdrnhzyh"
      },
      "outputs": [],
      "source": [
        "# Let's try to find out how many decimal points some number has\n",
        "# let's use the number Pi (because it is infinite)\n",
        "\n",
        "\n",
        "# we could think that we can just put a len() around it, but that doesn't work\n",
        "\n",
        "# Convert the float to a string\n",
        "\n",
        "# Get the length of the string\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GG1SZpwDhzyh"
      },
      "outputs": [],
      "source": [
        "# Check variable type"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EYi-AUnWhzyh"
      },
      "source": [
        "on a side-note: floats in Python have about 16 decimal places; the rest is cut off; a discussion (exceeding today's scope) can be found [here](https://en.wikipedia.org/wiki/Double-precision_floating-point_format)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pMHfU-Xqhzyh"
      },
      "source": [
        "# More complex data types\n",
        "\n",
        "- to iterate: This is a 4h intro\n",
        "- we will only show you four other data types\n",
        "  - `list` (list) which is a collection of items\n",
        "  - `dict`  (dictionary) which is a key-value pair\n",
        "  - `np.array` ([numpy](https://numpy.org/) array) which is a collection of items with the same data type\n",
        "    - alternative for use [MatLab](https://www.mathworks.com/products/matlab.html)\n",
        "  - `pd.DataFrame` ([pandas](https://pandas.pydata.org/) DataFrame) which is a 2D data structure with columns that can have different data types\n",
        "    - alternative to [R](https://www.r-project.org/)\n",
        "\t\t- (we will import our spreadsheet into this data type)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "shzMQ51ehzyi"
      },
      "outputs": [],
      "source": [
        "# Lets have a list of names\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_c8qejXhhzyp"
      },
      "outputs": [],
      "source": [
        "# Let's only show the first element of the list\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xg3n4vnthzyp"
      },
      "source": [
        "Wait! Jane? 🤔\n",
        "Yes, Python is zero-indexed! 🤯\n",
        "If we want to see the first element of a list, we need to use `list[0]` 🤯"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ca1V93U6hzyp"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LOo4FGm9hzyp"
      },
      "source": [
        "there you go!\n",
        "\n",
        "\n",
        "(You can also use negative indices to access elements from the end of the list. `list[-1]` will give you the last element of the list)\n",
        "You can also put a list in a list\n",
        "this is then a `list of lists` or a `2D list`\n",
        "\n",
        "<img src=\"https://github.com/UoA-eResearch/ResBaz24Python/blob/main/Assets/veg.png?raw=1\" alt=\"ListOfLists\" style=\"width: 400px\"/>\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TAffWZ0bhzyq"
      },
      "outputs": [],
      "source": [
        "# define a nested list\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "paO3vGsUhzyq"
      },
      "outputs": [],
      "source": [
        "# or more neatly formatted with pprint\n",
        "\n",
        "# Pretty print the list of lists\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8Jp3yl1Vhzyq"
      },
      "source": [
        "let's try to find the cilantro 🌿\n",
        "in other words: lets speficy the *correct cell* in this list of lists"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Gahgf-gYhzyq"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HUtS203xhzyq"
      },
      "outputs": [],
      "source": [
        "# Demonstration: How to get a subset of a list\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uvss36Vthzyr"
      },
      "outputs": [],
      "source": [
        "# Demonstration: How to append a list\n",
        "\n",
        "\n",
        "# what happens if we push the 'play' button on this cell a couple of times?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tyxm6cfDNlJw"
      },
      "source": [
        "# Exercices Part 2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OLqyHsL0Ntbz"
      },
      "outputs": [],
      "source": [
        "# Exercise 4: Use math.sqrt to get the square root of 36 and save it in a variable called square_root\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "OHxzyKowNtcB"
      },
      "outputs": [],
      "source": [
        "# @title Check Answer for Exercise 4\n",
        "import requests\n",
        "if square_root == 6:\n",
        "  is_correct = True\n",
        "else:\n",
        "  is_correct = False\n",
        "requests.post(f\"http://resbaz.auckland-cer.cloud.edu.au/\", params={\n",
        "  \"question\": 4,\n",
        "  \"answer\": square_root,\n",
        "  \"participant_name\": participant_name,\n",
        "  \"is_correct\": is_correct\n",
        "}).json()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "ZSAA6JkrPVCT"
      },
      "outputs": [],
      "source": [
        "#@markdown ## Exercice 5\n",
        "#@markdown ### What is the data type of the variable below?\n",
        "#@markdown my_fruit = \"orange\"\n",
        "\n",
        "data_type = \"int\"  #@param ['int', 'list', 'string', 'dict']"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "5SSkVNAeOkcA"
      },
      "outputs": [],
      "source": [
        "# @title Check Answer for Exercise 5\n",
        "import requests\n",
        "if data_type == \"string\":\n",
        "  is_correct = True\n",
        "else:\n",
        "  is_correct = False\n",
        "requests.post(f\"http://resbaz.auckland-cer.cloud.edu.au/\", params={\n",
        "  \"question\": 5,\n",
        "  \"answer\": data_type,\n",
        "  \"participant_name\": participant_name,\n",
        "  \"is_correct\": is_correct\n",
        "}).json()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "YPm9jXpYQeAE"
      },
      "outputs": [],
      "source": [
        "#@markdown ## Exercice 6\n",
        "#@markdown ### What is the data type of the variable below?\n",
        "#@markdown number_of_oranges = 6\n",
        "\n",
        "data_type = \"list\"  #@param ['int', 'list', 'string', 'dict']"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "iWClycHDQeAS"
      },
      "outputs": [],
      "source": [
        "# @title Check Answer  for Exercise 6\n",
        "import requests\n",
        "if data_type == \"int\":\n",
        "  is_correct = True\n",
        "else:\n",
        "  is_correct = False\n",
        "requests.post(f\"http://resbaz.auckland-cer.cloud.edu.au/\", params={\n",
        "  \"question\": 6,\n",
        "  \"answer\": data_type,\n",
        "  \"participant_name\": participant_name,\n",
        "  \"is_correct\": is_correct\n",
        "}).json()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "isjGA732RqKf"
      },
      "outputs": [],
      "source": [
        "# Exercice 7\n",
        "# Create a list called my_fruits and store banana, apple, orange\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "0DPJL7_CQMDu"
      },
      "outputs": [],
      "source": [
        "# @title Check Answer for Exercise 7\n",
        "import requests\n",
        "import json\n",
        "if \"banana\" in my_fruits and \"apple\" in my_fruits and \"orange\" in my_fruits:\n",
        "  is_correct = True\n",
        "else:\n",
        "  is_correct = False\n",
        "requests.post(f\"http://resbaz.auckland-cer.cloud.edu.au/\", params={\n",
        "  \"question\": 7,\n",
        "  \"answer\": json.dumps(my_fruits),\n",
        "  \"participant_name\": participant_name,\n",
        "  \"is_correct\": is_correct\n",
        "}).json()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MSltTkoXhzyr"
      },
      "source": [
        "## Recap\n",
        "- we have stored values in a variable\n",
        "- we have printed things along the way\n",
        "- we have seen different data types\n",
        "  - `int` (integer), `float` (floating-point number), `str` (string),\n",
        "  -  `list` (list), `dict` (dictionary),\n",
        "  -  `np.array` (numpy array), `pd.DataFrame` (pandas DataFrame)\n",
        "\n",
        "## Next\n",
        "- we will cover big concepts in a very short time frame"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cvcLpgPPhzyr"
      },
      "source": [
        "# \"Quick shots\"\n",
        "\n",
        "- we don't expect you to understand everything\n",
        "- this is fine\n",
        "- these concepts are very powerful (and fundamental)\n",
        "- so let's spend 5min to rapidly go through them\n",
        "\n",
        "## Topics\n",
        "- conditional (=*if statements*)\n",
        "- repetitions (loops)\n",
        "- functions\n",
        "\n",
        "We use this structure to investigate these concepts:\n",
        "\n",
        "- Q1 What does it **do**\n",
        "- Q2 How does it **work**\n",
        "- Q3 How does that **look in code**\n",
        "\n",
        "## Conditional (=*if statements*)\n",
        "\n",
        "### Q1 What does it do\n",
        "- it gives us **options**\n",
        "- you can think of it as a decision tree\n",
        "\n",
        "### Q2 How does it work\n",
        "- you define a **condition**\n",
        "\t- if it is met, something is done\n",
        "\t- if not, something else is done\n",
        "\t- (it can get more complex, but that should suffice for now)\n",
        "\t- there is also Boolean Logic (0 = false, 1 = true) and maths invovled, bit again: out of scope\n",
        "\n",
        "### Q3 How does that look in code"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rfg1o_y4hzys"
      },
      "outputs": [],
      "source": [
        "# Example 1 if-statement: To check if a number is positive\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BWm8k6ahhzys"
      },
      "outputs": [],
      "source": [
        "# Example 1 if-statement: To check if a user is logged in\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "mkYTNI55VU6C"
      },
      "outputs": [],
      "source": [
        "#@markdown ## Check if a number is even\n",
        "\n",
        "number = 0  #@param {type: \"number\"}\n",
        "if number % 2 == 0:\n",
        "  print(\"The number is even!\")\n",
        "else:\n",
        "  print(\"The number is odd!\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WbPlFDFFhzys"
      },
      "source": [
        "## Repetitions (=*loops*)\n",
        "\n",
        "### Q1 What does it do\n",
        "- this prevents you from having to write or repeatedly execute code\n",
        "- you define a condition and as long as it is met, a certain piece of code (a code block, a cell,...) is executed until the condition isn't met anymore (or another condition is met)\n",
        "\n",
        "### Q2 How does it work\n",
        "- we focus on the for loop\n",
        "- iteration/counters are more relevant in other languges; but the same idea applies to Python and might help you in generating an understanding\n",
        "- we write something like: for the time that something is met, go through the indented (pushed a bit to the right) parts of the code\n",
        "\n",
        "### Q3 How does that look in code\n",
        "- side-note: Pythonian is to write something like `for name in names` (singular ... plural) wheras other languages introduce a counter (often the iteration variable `i`) and you write something like `for i < 100`. How you call your counter is up to you! Be consistent! Make sure others (or future-you) can read and understand the code (that also applies to Code-Golf!)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wQPF8cbGhzyt"
      },
      "outputs": [],
      "source": [
        "# Example 2: loops: Iterate over a list of numbers\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uTrDGh_whzyt"
      },
      "outputs": [],
      "source": [
        "# Example 2: loops: Iterate over a list of names\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OquKyBnmhzyt"
      },
      "source": [
        "## Functions\n",
        "\n",
        "### Q1 What does it do\n",
        "- think of it as a variable on steroids\n",
        "- instead of putting a virtual PostIt on a value (for example the integer i = 42 or the string \"hello\"), you encapsulate code-fragments in a function\n",
        "\n",
        "### Q2 How does it work\n",
        "- so you define your function once and can call it several times. From within the code you are writing right now and from other code (another notebook, another py script,...).\n",
        "- We do that in essence, once we call a package's function (math.avg(var)) means: Go to the Math Package and get the instructions/code/cook-book-style-thingy needed to compute the average\n",
        "\n",
        "\n",
        "### Q3 How does that look in code\n",
        "- side-note OOP, [Magic Methods (=Dunder Methods)](https://youtu.be/1I3fuDR2S9A?si=WGxv06r_FM5R8Yut)\n",
        "$\\rightarrow$ We could expand on that probably for days; this leads to the Object Oriented Programming paradigm/a way of doing things. It is good if you can remember this term, but its not vital :)\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4NcbO1NZhzyt"
      },
      "outputs": [],
      "source": [
        "# Example 1: Functions: Add two numbers\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MeoZbxAeWkEn"
      },
      "outputs": [],
      "source": [
        "# Exercice 8\n",
        "# Write a function called check_number\n",
        "# It prints \"Positive\" if the number is positive\n",
        "# \"Negative\" if the number is negative\n",
        "# and \"Zero\" if the number is zero\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "uf664XLRWkE2"
      },
      "outputs": [],
      "source": [
        "# @title Check Answer for Exercise 8\n",
        "import requests\n",
        "def test_check_number():\n",
        "    try:\n",
        "        assert check_number(10) == \"Positive\"\n",
        "        assert check_number(-5) == \"Negative\"\n",
        "        assert check_number(0) == \"Zero\"\n",
        "        return True\n",
        "    except AssertionError:\n",
        "        return False\n",
        "\n",
        "is_correct = test_check_number()\n",
        "requests.post(f\"http://resbaz.auckland-cer.cloud.edu.au/\", params={\n",
        "  \"question\": 8,\n",
        "  \"answer\": In[-1],\n",
        "  \"participant_name\": participant_name,\n",
        "  \"is_correct\": is_correct\n",
        "}).json()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ASgW8YIFhzyu"
      },
      "source": [
        "# It is your choice:\n",
        "\n",
        "- We have **two options** for the next part of the session.\n",
        "- We **might** have time for both, but we will start with the one that gets the most votes.\n",
        "- If we run out of time, we will have all needed information **linked in the notebook** and you can try it out in your own time.\n",
        "- **Again**, it is of utmost importance that you have **fun** and **learn** something new today! 🎉\n",
        "- Python is a vast field and we can **only scratch the surface today**.\n",
        "- No *fomo*! 🙅‍♂️\n",
        "\n",
        "## Option A: The Math problem 🧮\n",
        "- we use a mathematical function, e.g. `y = x^5` or `y = 0.4*x + 3`\n",
        "- we add some random **noise** to the data (fancy: Monte Carlo simulation🎲 $^1$)\n",
        "- we plot the data\n",
        "  \n",
        "$\\rightarrow$ pick this one if you **like math and plotting mathematical functions**\n",
        "\n",
        "## Option B: The waiter and tips 🍽️ 💸\n",
        "- we **import** a given dataset ([seaborn tips](https://github.com/mwaskom/seaborn-data/tree/master?tab=readme-ov-file))\n",
        "- we **plot** the data\n",
        "- we perform some `Exploratory Data Analysis` (EDA)\n",
        "- we will **add other datasets**\n",
        "  \n",
        "$\\rightarrow$ pick this one if you **expect that your research will involve tabular data and you want to see how efficiently this can be done in Python**\n",
        "\n",
        "## If we still have time, for both options we can:\n",
        "- we perform a linear regression (we fit a curve to the data; well, a line to be honest with *linear* regression)\n",
        "- we inspect some statistical values\n",
        "- if we find the time, we also perform a polynomial regression (we fit a curve to the data; this time really *a curve*)\n",
        "\n",
        "$^1$ Monte Carlo simulation is just a fancy way of using randomness and lots of tries to find out the chances of something happening, like picking a red candy from a huge jar. Scientists and mathematicians use this method to solve problems that are too tricky to figure out with just a few calculations."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "-ijBKZqdnLN1"
      },
      "outputs": [],
      "source": [
        "# @title Start voting now, please :)\n",
        "dropdown = 'Option A the Math Problem' # @param [\"Option A the Math Problem\", \"Option B the Waiter and Tips\"]\n",
        "import requests\n",
        "requests.post(f\"http://resbaz.auckland-cer.cloud.edu.au/\", params={\n",
        "  \"question\": 1500,\n",
        "  \"answer\": dropdown,\n",
        "  \"participant_name\": participant_name,\n",
        "  \"is_correct\": True\n",
        "}).json()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4K1PB1IIhzyu"
      },
      "source": [
        "# Option A: The Math problem 🧮"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2k_XB7g_hzyu"
      },
      "source": [
        "### A1: Using a function to create a distribution and fit a linear regression with matplotlib and scipy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jrTwzLVJut7i"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from scipy import stats\n",
        "\n",
        "# Generate 100 x-values\n",
        "\n",
        "\n",
        "# Generate add some noise\n",
        "\n",
        "\n",
        "# Create y values using the linear function y = 0.4*x + 3 + delta\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LOskSv2tut7j"
      },
      "outputs": [],
      "source": [
        "# Plot the scatterplot\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "chX3Y-fCut7j"
      },
      "outputs": [],
      "source": [
        "# Calculate the linear regression\n",
        "\n",
        "# Create the regression line\n",
        "\n",
        "\n",
        "# Plot the scatterplot\n",
        "\n",
        "# Add labels and title\n",
        "\n",
        "\n",
        "# Show the plot\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3zK2p8_dut7j"
      },
      "source": [
        "### A2: Fitting and plotting a non-linear distribution with sklearn and seaborn and pandas"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DFjb6LEthzyu"
      },
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import seaborn as sns\n",
        "\n",
        "# Define a power function\n",
        "\n",
        "\n",
        "# Generate values for x\n",
        "\n",
        "\n",
        "# Compute corresponding y values\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6P12fYmihzyv"
      },
      "source": [
        "#### Plotting alternatives:\n",
        "To get an initial understanding (and not a publishing-ready figure to be fully interpretable by others), we can use:\n",
        "  - we can use `matplotlib.pyplot` and simply put `plt.plot(x,y)`\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "w9Fagppohzyv"
      },
      "outputs": [],
      "source": [
        "# Plot the function using Seaborn\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "07AG-W4whzyv"
      },
      "source": [
        "#### Seaborn: A better workflow\n",
        "- we create a Pandas DataFrame\n",
        "  - think of this as a (fancy/powerful) spreadsheet in Python\n",
        "- we give the columns proper names\n",
        "- then we don't have to use workaround for axis title\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9P_bBmSjhzyw"
      },
      "outputs": [],
      "source": [
        "# import pandas\n",
        "\n",
        "\n",
        "# Create a DataFrame\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hzSj-1athzyw"
      },
      "outputs": [],
      "source": [
        "# Plot the function using Seaborn\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v2g60Dxjhzyw"
      },
      "source": [
        "#### We generate some spreadsheet data with our noise definition\n",
        "\n",
        "- we will use a [Gaussian/Normal distribution](https://numpy.org/doc/stable/reference/random/generated/numpy.random.normal.html) to add some noise to our data\n",
        "- while we are at it, we also plot the new data to compare it"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dt4nm03ehzyw"
      },
      "source": [
        "A2.2: Let's create some noisy data and store it as a spreadsheet and a Pandas Dataframe"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "w9GhQa1thzyw"
      },
      "outputs": [],
      "source": [
        "# Add Gaussian noise\n",
        "\n",
        "\n",
        "# Create the noisy y values based on the existing yClean column\n",
        "\n",
        "# Display the top 10 lines of the DataFrame\n",
        "\n",
        "\n",
        "# Savining a pandas dataframe to csv\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fDPPC63xhzyx"
      },
      "source": [
        "#### A2.3: Let's plot the noisy data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ROAqCmIlhzyx"
      },
      "outputs": [],
      "source": [
        "# Plot our noisy data\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y0yLcLyThzyx"
      },
      "source": [
        "### A2.3 Let's start with the regression (linear and polynomial)\n",
        "\n",
        "- there are various ways of doing this\n",
        "- there are diffferent packages to be used\n",
        "- some are better for visualisation\n",
        "- some offer detailed statistical analysis\n",
        "  - `numpy` is often used, it provides some regression functions, often it is called 'under the hood'\n",
        "  - `scipy` is a more advanced package, it offers more statistical analysis\n",
        "  - `sklearn` is a machine learning package, it offers a lot of different models, including regression models\n",
        "  - `statsmodels` is a package that is often used in academia, it offers a lot of statistical analysis\n",
        "\n",
        "For this session, we will use `sklearn` as it gives us a good balance between visualisation and statistical analysis"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WolgoSFqhzyy"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "from sklearn.preprocessing import PolynomialFeatures\n",
        "from sklearn.linear_model import LinearRegression\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "\n",
        "# Extracting features (X) and target variable (y)\n",
        "\n",
        "# Transforming features to polynomial features of degree 3\n",
        "\n",
        "\n",
        "# Fitting the model\n",
        "\n",
        "\n",
        "# plot distribution\n",
        "\n",
        "\n",
        "# Plotting the polynomial regression curve\n",
        "\n",
        "# Print the coefficients\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2JeA3q7ihzyy"
      },
      "source": [
        "\n",
        "[some theory](https://seaborn.pydata.org/tutorial/relational.html):\n",
        "Scatter plots are highly effective, but there is no universally optimal type of visualisation. Instead, the visual representation should be adapted for the specifics of the dataset and to the question you are trying to answer with the plot.\n",
        "With some datasets, you may want to understand changes in one variable as a function of time, or a similarly continuous variable. In this situation, a good choice is to draw a line plot. In seaborn, this can be accomplished by the lineplot() function, either directly or with relplot() by setting kind=\"line\":"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Mvds1EmZhzyz"
      },
      "source": [
        "# Option B: The Waiter and the tips"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X2CoAFbbhzyz"
      },
      "source": [
        "## The dataset we use:\n",
        "One waiter 🛎️ recorded information about each tip 💸 received\n",
        "- tip 💵\n",
        "- bill in dollars 💵\n",
        "- sex of the bill payer 👫\n",
        "- whether there were smokers in the party 🚬\n",
        "- day of the week 📅\n",
        "- time of day 🕙\n",
        "- size of the party 👥\n",
        "\n",
        "All in all, 244 tips were recorded and published in Bryant, P. G. and Smith, M (1995) Practical Data Analysis: Case Studies in Business Statistics. Homewood, IL: Richard D. Irwin Publishing\n",
        "\n",
        "## Import data\n",
        "\n",
        "In reality, we have several ways how we can get ahold of data.\n",
        "For this course, we will not worry about collecting data (i.e. *going in the field and asking people, sketching rocks, etc.*)\n",
        "We will avoid many challenges by expecting that software used to generate or create the data will have some export functionality to some common file formats.\n",
        "We can import data from...\n",
        "- ... a website (API, a repository (such as GitHub or others), ...)\n",
        "- ... a spreadsheet (local file, uploaded file, etc.)\n",
        "  - we will download an `xlsx` file and then make it more usable in Python\n",
        "  - we will finally save our processed results back as a new Sheet in the `xslx` file\n",
        "\n",
        "### B1: Import data from a respository\n",
        "- we will import an Excel file use the `seaborn` package to import the `tips` dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "E00NqCfohzyz"
      },
      "outputs": [],
      "source": [
        "# Define the file name and URL\n",
        "file_name = \"tips.xlsx\"\n",
        "url = \"https://github.com/UoA-eResearch/ResBaz24Python/raw/main/Datasets/tips.xlsx\"\n",
        "\n",
        "\n",
        "! wget {url}\n",
        "\n",
        "# Read the Excel file and load it into a pandas dataframe\n",
        "tips = pd.read_excel(\"tips.xlsx\", sheet_name=\"tips\")\n",
        "tips.head()\n",
        "\n",
        "\n",
        "# use this https://seaborn.pydata.org/tutorial/regression.html"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CThTC2IUhzy0"
      },
      "source": [
        "### B2: Exploratory Data Analysis (EDA)\n",
        "\n",
        "- now that we have explored how to import data from an Excel file, make it useable as a Pandas Dataframe and save it back to excel, we will perform some basic EDA\n",
        "- we can either\n",
        "  - directly look at the data 🔢 🧐\n",
        "  - plot it 📈 📊\n",
        "- lets try to use only a few lines of code to get an **understanding** of what this 'tips' dataset is\n",
        "- for more details see [this EDA example](https://www.kaggle.com/code/mirjanadmitrovic/data-visualization-using-seaborn-tips-dataset)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "95SCnW3Fhzy0"
      },
      "source": [
        "### B2.1 Looking at the data\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JOJknreqhzy0"
      },
      "outputs": [],
      "source": [
        "# return first 10 rows of the dataframe\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jeCSkgszhzy0"
      },
      "outputs": [],
      "source": [
        "# generate general stats for dataframe"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "529TIKaNhzy0"
      },
      "outputs": [],
      "source": [
        "# demonstrate how to return categorical variables for a column in dataframe\n",
        "\n",
        "\n",
        "# detect labels in categorical variables\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MTSP6Kqshzy1"
      },
      "source": [
        "### B2.2 Plotting the data\n",
        "- firstly with just one line of code\n",
        "- then with some refined package (Seaborn `sns pairplot`)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vvpFPoVwhzy1"
      },
      "outputs": [],
      "source": [
        "# We can plot data using the plotting function available with Pandas\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Q2fi0H0Ihzy1"
      },
      "outputs": [],
      "source": [
        "# We can create more complex plots passing the dataframe to seaborn plotting functions\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4loa4IWghzy1"
      },
      "source": [
        "Poll:\n",
        "What can you observe?\n",
        "How many plots were to be expected? Are there duplicates? Are there too few?\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BW0LoK1Ahzy1"
      },
      "source": [
        "Answer: Our dataset shows 7 categories; here we only see 3 vs. 3 = 9 plots;\n",
        "- why aren't the other plots shown?\n",
        "- Because of their data type!\n",
        "- They are either strings or booleans, and the pairplot function does not know how to plot them.\n",
        "- there are ways to show more data; for example we can include two more dimensions in a 2D plot: Size of the party and the sex of the bill payer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "v2XXcxm7hzy1"
      },
      "outputs": [],
      "source": [
        "# We can use other functions in seaborn to visualise categorical and numerical data on the same plot. e.g. relplot\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sUPc2USqhzy1"
      },
      "source": [
        "Let's have a look at the non-number data types\n",
        "- Don't worry, this looks like 'a lot' but, in essence, it is a loop over all the categorials (so not numeric data types) and then we plot them"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tQB7j9wVhzy2"
      },
      "outputs": [],
      "source": [
        "# Create a figure with subplots\n",
        "\n",
        "\n",
        "# Loop over each categorical column and plot\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tT_WX_4fhzy2"
      },
      "source": [
        "### B2.3 So? What do we learn about our data?\n",
        "\n",
        "- over to you"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h9qWLDyfhzy2"
      },
      "source": [
        "## B3: Adding data\n",
        "  - B4.1 The waiter kept collecting data; we want to append this data to our existing dataset\n",
        "  - B4.2 Another waiter in the same restaurant collected data. But used a different sequence of the columns.\n",
        "  - B4.3 In two other restaurants similar data was collected. We want to merge all this data into one dataset.\n",
        "\n",
        "### B3.1 append with more data\n",
        "- we will append some more data to the `tips` dataframe\n",
        "- to clearly identify where the data stems from, we will add a new column to the existing dataframe named 'source'\n",
        "- when we (accidentally) re-run this cell, we will keep adding data; this is not what we want. We will fix with *a conditional*."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_JeoT4Tnhzy2"
      },
      "outputs": [],
      "source": [
        "# download the file\n",
        "url = \"https://github.com/UoA-eResearch/ResBaz24Python/raw/main/Datasets/tips_dayB.csv\"\n",
        "! wget {url}\n",
        "\n",
        "\n",
        "# add a new column 'source' to the tips dataframe and set it to 'Day1'\n",
        "\n",
        "# Display the top 10 lines of the dataframe\n",
        "\n",
        "\n",
        "# Load the CSV file into a pandas dataframe and add source column\n",
        "\n",
        "\n",
        "# Append the tips2 dataframe to the tips dataframe\n",
        "\n",
        "\n",
        "# Display the top 10 lines of the dataframe\n",
        "\n",
        "# Whats the shape of the new dataframe?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uaaCfnYuhzy2"
      },
      "source": [
        "Let's see if there are duplicates in the data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "D9kyJTFphzy2"
      },
      "outputs": [],
      "source": [
        "# Find duplicated rows\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a4I52eX4hzy3"
      },
      "source": [
        "there is just one duplicated line, that should be fine"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5LyEy-brhzy3"
      },
      "source": [
        "### B3.2 Another waiter in the same restaurant collected data but with different column sequence\n",
        "- we import a dataset from another waiter\n",
        "  - but this person had a different column sequence\n",
        "- we will try to merge these two datasets\n",
        "- then we will try to get an impression of how they differ"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OcXeVFg-hzy3"
      },
      "outputs": [],
      "source": [
        "url = \"https://github.com/UoA-eResearch/ResBaz24Python/raw/main/Datasets//tips_waiterB.csv\"\n",
        "! wget {url}\n",
        "\n",
        "# import the data\n",
        "\n",
        "\n",
        "# add waiter columns to dataframes\n",
        "\n",
        "\n",
        "# concatenate dataframes\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q-DF4r4bhzy3"
      },
      "source": [
        "Poll\n",
        "- what should we do now?\n",
        "- look at the values?\n",
        "- look at some plots?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jQ-Dg9fYhzy4"
      },
      "outputs": [],
      "source": [
        "# How could we look at the variety of distributions for both waiters?\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cB4JP14_nLN7"
      },
      "source": [
        "### B3.3 We have the same dataset collected from another restaurant"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8qD9N0cThzy4"
      },
      "outputs": [],
      "source": [
        "# download the data\n",
        "url = \"https://github.com/UoA-eResearch/ResBaz24Python/raw/main/Datasets/tips_restaurantB.csv\"\n",
        "! wget {url}\n",
        "\n",
        "# import data from another restaurant\n",
        "\n",
        "\n",
        "# add a restaurant column and name each restaurant\n",
        "\n",
        "# concatenate dataframes"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DQ-1e8pLhzy4"
      },
      "outputs": [],
      "source": [
        "# how could visulise the data from each restaurant in the same plot?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3GdStOtanLN7"
      },
      "outputs": [],
      "source": [
        "# How could we return our write our new dataframe to a file containing tips from both restaurants?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T3TIpDmnhzy4"
      },
      "source": [
        "# Revisiting the Goals"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3U2zmiTVhzy4"
      },
      "source": [
        "Did you?\n",
        "  - ... have fun! 🎉\n",
        "  - ... follow along while live coding in Python 🐍\n",
        "  - ... pick up some core coding concepts while doing that?\n",
        "  - ... make a decision if Python is something you want to learn/use? If so:\n",
        "      - Carp\n",
        "      - ML\n",
        "      - hand-selected links below\n",
        "  \n",
        "  Now that we have this experience in Google Colab, which is one flavour of a Jupyter Notebook\n",
        "  - University of Auckland researchers have access to the [Nectar Research Cloud](https://research-hub.auckland.ac.nz/research-software-and-computing/advanced-compute/nectar-research-cloud) and sevices like [JupyterHub](https://research-hub.auckland.ac.nz/research-software-and-computing/advanced-compute/nectar-jupyterhub), [BinderHub](https://research-hub.auckland.ac.nz/research-software-and-computing/advanced-compute/nectar-binderhub), and [Nectar Virtual Desktop](https://research-hub.auckland.ac.nz/research-software-and-computing/advanced-compute/nectar-virtual-desktop)\n",
        "  - if you automate tasks, make machines talk to other machines, etc. you can write scripts and run them on the command-line\n",
        "  - you can use an Integrated Development Environment (IDE) like [Visual Studio Code](https://code.visualstudio.com/) or [PyCharm](https://www.jetbrains.com/pycharm/)\n",
        "\n",
        "Were your goals met? 🎯\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GXE4NClFhzy4"
      },
      "source": [
        "# Resources"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sOXeRHnJhzy4"
      },
      "source": [
        "## Websites for Learning Python:\n",
        "\n",
        "-   [HarvardX: Using Python for Research](https://www.edx.org/learn/python/harvard-university-using-python-for-research)\n",
        "-   [LearnPython.org](https://www.learnpython.org/) This is a free interactive Python tutorial for people who want to learn Python\n",
        "-   [Google\\'s Python Class](https://developers.google.com/edu/python)\n",
        "    Google also has an excellent set of Python tutorials for beginners\n",
        "-   [Codecademy](https://www.codecademy.com/) offers an interactive\n",
        "    Python course\n",
        "-   [30 Days of Python](https://github.com/Asabeneh/30-Days-Of-Python)\n",
        "-   [How to Think Like a Computer Scientist](https://runestone.academy/runestone/books/published/thinkcspy/index.html)\n",
        "\n",
        "## Video for Learning Python\n",
        "\n",
        "-   [freeCodeCamp.org](https://youtu.be/rfscVS0vtbw?si=EiY8F-GupxBrtovb)\n",
        "\n",
        "## Python Blogs:\n",
        "\n",
        "-   [Real Python](http://ttps://realpython.com/) is a comprehensive\n",
        "    Python programming blog with eye-catching infographics, videos, and\n",
        "    an overall fun vibe\n",
        "-   [Planet Python](https://planetpython.org/) is a comprehensive blog\n",
        "    that brings together recent Python-related posts from various other\n",
        "    blogs\n",
        "-   [Finxter](https://blog.finxter.com/) is another Python blog to\n",
        "    follow\n",
        "-   [Full Stack Python](https://www.fullstackpython.com/) is a blog that\n",
        "    provides detailed tutorials on various platforms utilizing Python\n",
        "\n",
        "## Python Games\n",
        "- [CodeCombat](https://codecombat.com/): Learn programming through an interactive game-based platform where players write code to control characters and solve challenges.\n",
        "- [Code on the Cob](https://www.codeonthecob.com/): A webiste to practice coding\n",
        "- [CheckiO](https://checkio.org/): Learn Python coding through solving coding challenges in a gamified environment, with a focus on improving coding skills.\n",
        "- [Advent of Code](https://adventofcode.com/): An annual coding challenge held in December, featuring daily programming puzzles to solve.\n",
        "- [CodinGame](https://www.codingame.com/start/): Learn coding through gamified programming puzzles and challenges, covering various languages and difficulty levels.\n",
        "- [LeetCode](https://leetcode.com/): Practice coding skills through a collection of coding problems, ranging from easy to hard, often used for technical interviews.\n",
        "- [Exercism](https://exercism.org/tracks/python): Learn programming languages through practice problems with mentors, focusing on feedback-driven learning and community support.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5e9QaGI4hzy4"
      },
      "source": []
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}