chapter_00__Pytorch__fundamental.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyNISDW26bC6Evpt6gU+7/w/",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ayush2281/Pytorch-for-ML-DL-/blob/main/chapter_00__Pytorch__fundamental.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 00. Pytorch Fundamental\n"
      ],
      "metadata": {
        "id": "Kxgov1YDL1Yw"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lQcTVh0tLEnt",
        "outputId": "9b1af3d6-4e54-4428-bc92-4a7bc0e33bc5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Hey Im excited to start pytorch\n"
          ]
        }
      ],
      "source": [
        "print(\"Hey Im excited to start pytorch\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The torch library is the core component of PyTorch, and it provides the fundamental building blocks for creating and working with neural networks"
      ],
      "metadata": {
        "id": "L-BVB9D8QBli"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "print(torch.__version__)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OkUXmnnrLj8u",
        "outputId": "6d6e0116-b1b7-4502-db42-1fc63aa9d5db"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2.4.1+cu121\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        " Introduction of tensor\n",
        "\n",
        "Creating Tensor"
      ],
      "metadata": {
        "id": "1TnFE9uwNENm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# scalar\n",
        "scalar = torch.tensor(7)\n",
        "scalar"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TMFL94Z8LxEY",
        "outputId": "45594cd9-130b-4c94-db9e-6bbb2f9826c0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(7)"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "scalar.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ybklLO8CNARW",
        "outputId": "066a3961-908d-452e-fb65-e7a5968cc431"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "0--> dimensions"
      ],
      "metadata": {
        "id": "cM0_4J8vQ2Ej"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# get a tensor back as python int\n",
        "scalar.item()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OQTNgyioQOSQ",
        "outputId": "ef9a48d4-3911-443e-f97b-a5943dfd8be8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "7"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "vector = torch.tensor([7,7])\n",
        "vector"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PVffPVuQQhL4",
        "outputId": "cc38afdc-5638-460a-d3ab-543a40d7e166"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([7, 7])"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "vector.ndim\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6SzwcCc-QuSg",
        "outputId": "53fb0131-744c-456e-9371-9856179b3766"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "vector.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YDkmLvJPQ_R0",
        "outputId": "e6b32141-2ffe-4351-c7ea-e7af94d65afb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([2])"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Matrix\n",
        "MATRIX = torch.tensor([[7,8],\n",
        "                       [9,10]])\n",
        "MATRIX"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FsEdOaXCRS0V",
        "outputId": "7e33bbbd-73ed-4b84-f970-7b82ee8e5e76"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[ 7,  8],\n",
              "        [ 9, 10]])"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "MATRIX.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Vxs3cjrHRd9t",
        "outputId": "352ce9fc-70e8-4a69-a145-4a66aa3d98bf"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "MATRIX.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AEJedzPpRhx8",
        "outputId": "4cad978b-ec03-4f75-e757-32b6c6fbfc5d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([2, 2])"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Tensor\n",
        "Tensor = torch.tensor([[[1,2,3],\n",
        "                      [9,5,2],\n",
        "                      [5,7,6]]])\n",
        "Tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xR8k31qVRoAf",
        "outputId": "e532d82b-6f6f-46b4-c1b0-5896000305a0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[[1, 2, 3],\n",
              "         [9, 5, 2],\n",
              "         [5, 7, 6]]])"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# dimensions\n",
        "Tensor.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aTg3JkO8SbEV",
        "outputId": "c013b8d9-650b-40dd-d83a-9302a30fca77"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "Tensor.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Jb_3F5RhSjEy",
        "outputId": "c473170d-ea61-48ac-9aa2-c180b4b563f6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([1, 3, 3])"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Random Tensors"
      ],
      "metadata": {
        "id": "6GVO0ePFUALl"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Why random tensor?\n",
        "\n",
        "random tensor are very important because the way many network learn is that they start with tensor full of random numbers and then adjust those random numbver sto better represent the data."
      ],
      "metadata": {
        "id": "SVpY-zQkUPLh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Random Tensors\n",
        "random_tensor = torch.rand(3,4)\n",
        "random_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RjVXqEUZSm3g",
        "outputId": "dd8fa12a-4fad-4b1d-eccd-d7c2cafc05fc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0.1268, 0.8988, 0.4644, 0.5064],\n",
              "        [0.4357, 0.1493, 0.6990, 0.1938],\n",
              "        [0.0610, 0.9428, 0.3676, 0.1392]])"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "random_tensor.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WaCxL0AuSqgP",
        "outputId": "6fb0c00e-d7c1-468d-ba10-c3cde28a78b7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# create a random tensor with similar shape to an image tensor\n",
        "random_image_size_tensor  = torch.rand(size=(3,224,224))  # height , width , cooour channel (RGB)\n",
        "random_image_size_tensor.shape, random_image_size_tensor.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HrNs1cZoU4un",
        "outputId": "b007a575-6a17-4a2a-b54c-863f79a59a01"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([3, 224, 224]), 3)"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "## zeros and ones\n",
        "# create a tensor of all zeors\n",
        "zeros = torch.zeros(3,4)\n",
        "zeros"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OYUiLYJWV5aJ",
        "outputId": "04a4b568-f246-4e32-d06d-f38304b8c02f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0.],\n",
              "        [0., 0., 0., 0.]])"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# create an tensor of all ones\n",
        "ones = torch.ones(3,4)\n",
        "ones"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ad-do5fkWmRE",
        "outputId": "46536eed-df5d-4a11-d703-920e29809657"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[1., 1., 1., 1.],\n",
              "        [1., 1., 1., 1.],\n",
              "        [1., 1., 1., 1.]])"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "ones.dtype"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LQjOVAkCXCJD",
        "outputId": "9b20533b-8ab9-4bfa-cdf0-d343d03cf876"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.float32"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "ones.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GDPRbOUIXUAV",
        "outputId": "522043db-76b9-4963-86de-d4b3c3fad8b8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([3, 4])"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "ones.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Av6YWcOVXV0s",
        "outputId": "32af0035-8116-41b7-f517-8c0c588d9990"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Creating a range of tensor and tensor like"
      ],
      "metadata": {
        "id": "1S80InmOXjYi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# use torch.range()\n",
        "# torch.range(0,10)\n",
        "one_to_ten = torch.arange(start=0,end=100, step=10)\n",
        "one_to_ten"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5Y4JrO0qXYW6",
        "outputId": "9a7ddd6c-c414-4d9d-e588-fc2679725c7f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 0, 10, 20, 30, 40, 50, 60, 70, 80, 90])"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# creating the tensor like"
      ],
      "metadata": {
        "id": "ECdQsr1mXuIo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#creating the tensor like\n",
        "ten_zeros = torch.zeros_like(one_to_ten)\n",
        "ten_zeros"
      ],
      "metadata": {
        "id": "aaE6VFchXz8Z",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c371e5d0-99cf-42e9-ae81-b6720047ecc6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Tensor datatypes**"
      ],
      "metadata": {
        "id": "jVjd7JxafWXz"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**NOTE:** : tensor datatype is one of the\n",
        "3 big error you will run with pytorch & deeep learning.\n",
        "1. Tensor not right datatype\n",
        "2. Tensor not right shape\n",
        "3. tensor not on the right device"
      ],
      "metadata": {
        "id": "mXft8wOegp9K"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# float 32  tensor\n",
        "float_32_tensor = torch.tensor([3.0,6.0,9.0],\n",
        "                               dtype=torch.float16)\n",
        "float_32_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CsndXvuke8nM",
        "outputId": "a42d8c1f-0f52-4d10-f4c2-39e0b72a0dc9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([3., 6., 9.], dtype=torch.float16)"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "float_32_tensor.dtype"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5RbOh7jaftj_",
        "outputId": "592fba65-cc7a-41c2-c800-4f8afcc636c0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.float16"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "float_16_tensor = float_32_tensor.type(torch.float16)\n",
        "float_16_tensor\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "THNsYLzqfw9X",
        "outputId": "c2f6bc87-9706-4d54-ef94-0a1d4501596f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([3., 6., 9.], dtype=torch.float16)"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "float_16_tensor * float_32_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PoF1WTh2hwei",
        "outputId": "8c807188-04e0-4886-ebf7-806907e60e65"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 9., 36., 81.], dtype=torch.float16)"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "int_32_tensor = torch.tensor([3,6,9], dtype=torch.long)\n",
        "int_32_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6jXQqxfBiAHa",
        "outputId": "1a6d5308-545f-42d1-9350-376426bb1942"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([3, 6, 9])"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "float_32_tensor * int_32_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "q1rIn1_viRsU",
        "outputId": "93c0c623-80ff-4249-bd94-1cbe8d60caa9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 9., 36., 81.], dtype=torch.float16)"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# create a tensor\n",
        "some_tensor = torch.rand(3,4)\n",
        "some_tensor"
      ],
      "metadata": {
        "id": "961cfn0OiU7r",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fec98163-abca-445f-d3ec-81244ba00dc1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[0.2720, 0.0094, 0.3605, 0.9613],\n",
              "        [0.3142, 0.5819, 0.8168, 0.3588],\n",
              "        [0.7504, 0.0693, 0.3429, 0.3653]])"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# find out details about some tensor\n",
        "print(some_tensor)\n",
        "print(f\"Datatype of tensor: {some_tensor.dtype}\")\n",
        "print(f\"Shape of tensor: {some_tensor.shape}\")\n",
        "print(f\"Device tensor is on: {some_tensor.device}\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FyH_Nav9O2xn",
        "outputId": "583aa763-9b05-4826-d131-53ad6d88f535"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[0.2720, 0.0094, 0.3605, 0.9613],\n",
            "        [0.3142, 0.5819, 0.8168, 0.3588],\n",
            "        [0.7504, 0.0693, 0.3429, 0.3653]])\n",
            "Datatype of tensor: torch.float32\n",
            "Shape of tensor: torch.Size([3, 4])\n",
            "Device tensor is on: cpu\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# both give the same output but one is function and another is attributes\n",
        "some_tensor.size() , some_tensor.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RO3fKc4iPDXQ",
        "outputId": "3204c472-7a0c-4177-835a-5baafeb4c1cb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(torch.Size([3, 4]), torch.Size([3, 4]))"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Manipulating Tensors (Tensors OPeration)\n",
        "  Tensors Operations\n",
        "1. Addition\n",
        "2. Substraction\n",
        "3. Mutiplicatio\n",
        "4. Devision\n",
        "5. Matrix Multipication\n"
      ],
      "metadata": {
        "id": "PiY4-zjmQX1A"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# cfreate an tensor\n",
        "tensor = torch.tensor([1, 2, 3])\n",
        "tensor + 10"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GGP7G8O4PPZh",
        "outputId": "1bb13f19-865a-4dc3-e6e0-a88aac0cbfd1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([11, 12, 13])"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# multiply by 10\n",
        "tensor = torch.tensor([5,5,5])\n",
        "tensor * 10"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z9ftl1IwQ38Q",
        "outputId": "2be68c1c-85f6-4e48-f15e-c84259064e38"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([50, 50, 50])"
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b-Wlt1mKRWKO",
        "outputId": "23d3b975-41d1-45a9-f012-4d5a29d83b10"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([5, 5, 5])"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#  subtract\n",
        "tensor = torch.tensor([9,3,4])\n",
        "tensor -5"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BbdrVR5vRaqp",
        "outputId": "e3923dbe-413c-4b59-804e-038749a60ea5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 4, -2, -1])"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# try out in pytorch build function\n",
        "torch.mul(tensor,10)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ojh2AusDRkka",
        "outputId": "c24171d1-2c08-416f-b40a-6ea238823201"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([90, 30, 40])"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# original value of tensor\n",
        "tensor\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MGMnEh5ZSEt1",
        "outputId": "6bb45679-a206-4f16-b9c2-e74532dd8ffc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([9, 3, 4])"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# divison\n",
        "tensor = torch.tensor([1,2,3])\n",
        "tensor / 2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KPz-wxniSG80",
        "outputId": "f7cde499-ba77-4bad-d958-aa7fbac668e8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([0.5000, 1.0000, 1.5000])"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# there is two ways to matrix multiplication in deep learn and nerual netwrk\n",
        "1. element-wise\n",
        "2. Matrix multiplication (dot product)"
      ],
      "metadata": {
        "id": "1UrBg7KxTwNE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "there are two main rules that performing matrix multiplication needs to satisfy:\n",
        "\n",
        "\n",
        "1. The **inner dimension** must match:\n",
        "\n",
        "\n",
        "*   (3, 2) @ (3,2) wont work\n",
        "*   (3, 2) @ (2,3) will work\n",
        "*   (2,3)  @ (3,2) will work\n",
        "\n",
        "2. the **outer dimension**\n",
        "the resulting matrix has the shape of the **outer dimension**\n",
        "\n",
        "* (2,3) (3,2) -> (2,2)\n",
        "* (3,2) (2,3) -> (3,2)\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "VRrq9fdHXtfG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# element wise\n",
        "# matrix multiplication\n",
        "tensor_1 = torch.tensor([1,5,6]) * torch.tensor([2,3,4])\n",
        "tensor_1"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FjkrEAkNS4au",
        "outputId": "b99732df-4ac9-421c-b6d7-6c289a8cfdd0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([ 2, 15, 24])"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# matrix multilication\n",
        "%%time\n",
        "torch.matmul(tensor_1, tensor_1)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m7eCAPP2TKMH",
        "outputId": "dd0a7749-be9c-4740-e52c-f68319f98fb1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 0 ns, sys: 637 µs, total: 637 µs\n",
            "Wall time: 7.59 ms\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(805)"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# see it is faild because inner dimension is not matched\n",
        "tensr = torch.matmul(torch.rand(3,2), torch.rand(3,2))\n",
        "tensr"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 176
        },
        "id": "WdaFBNjpYlhS",
        "outputId": "d29002e2-04ac-4cb1-f2b1-f71ed1da299e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "RuntimeError",
          "evalue": "mat1 and mat2 shapes cannot be multiplied (3x2 and 3x2)",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-44-ee2af292d2d9>\u001b[0m in \u001b[0;36m<cell line: 2>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# see it is faild because inner dimension is not matched\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mtensr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmatmul\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrand\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrand\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0mtensr\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mRuntimeError\u001b[0m: mat1 and mat2 shapes cannot be multiplied (3x2 and 3x2)"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# see it is faild because inner dimension is not matched\n",
        "tensr = torch.matmul(torch.rand(3,2), torch.rand(2,3))\n",
        "tensr"
      ],
      "metadata": {
        "id": "4p2AMALaY41R"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# matrix multiplication by hand\n",
        "2*2 + 15*15 + 24*24"
      ],
      "metadata": {
        "id": "f10bSJxoUyh_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "value = 0\n",
        "for i in range(len(tensor_1)):\n",
        "  value += tensor_1[i] * tensor_1[i]\n",
        "print(value)\n"
      ],
      "metadata": {
        "id": "5UN3vadiWGCh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**One of the most commin erros in deep : shape error**"
      ],
      "metadata": {
        "id": "iNXcqzV2aFzT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Shape for the matrix multiplication\n",
        "tensor_A = torch.tensor([[1,2],\n",
        "                         [3,4],\n",
        "                         [5,6]])\n",
        "tensor_B = torch.tensor([[7,10],\n",
        "                         [8,11],\n",
        "                         [9,12]])\n",
        "torch.matmul(tensor_A, tensor_B)\n"
      ],
      "metadata": {
        "id": "lGaP8-5KWrh2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "tensor_A.shape, tensor_B.shape"
      ],
      "metadata": {
        "id": "onkB_dsHaxF8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "chanege the shape of any mat"
      ],
      "metadata": {
        "id": "tGqMph8MbF2L"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# we can use the transpose\n",
        "tensor_B.T, tensor_B.T.shape, \"Original shape\",tensor_B.shape\n"
      ],
      "metadata": {
        "id": "kjgAmiZrbDqk"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# th matrix multiplication opearation works with tensor_B is transposed\n",
        "torch.matmul(tensor_A, tensor_B.T)\n"
      ],
      "metadata": {
        "id": "sy6S8FOfbYXT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## finding the min , amx , sum , etc (tensor aggregation)"
      ],
      "metadata": {
        "id": "E1uYIuaVYqc_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# create a tensor\n",
        "x = torch.arange(1, 100, 10)\n",
        "x"
      ],
      "metadata": {
        "id": "7kVQJ4PyZihe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# find the min\n",
        "torch.min(x), x.min()"
      ],
      "metadata": {
        "id": "mHlg-LwebhVD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# find the max\n",
        "torch.max(x), x.max()"
      ],
      "metadata": {
        "id": "GEMIuOi6ZgUj"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# find the mean\n",
        "torch.mean(x.type(torch.float32)), x.type(torch.float32).mean()"
      ],
      "metadata": {
        "id": "JjhvM9G0Z9X5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# find the sum\n",
        "torch.sum(x), x.sum()"
      ],
      "metadata": {
        "id": "8ZQlJ1kYaJWe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Find the positional min and max"
      ],
      "metadata": {
        "id": "2fZ4A8n6bRPA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x\n"
      ],
      "metadata": {
        "id": "ZPTX7ETqa8hX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# find the position in tensor that has the minimun value with argmin()-> returns in\n",
        "x.argmin()"
      ],
      "metadata": {
        "id": "dj3OD8iKbWpC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x[0]"
      ],
      "metadata": {
        "id": "MdBa3mDPbZBd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x.argmax()"
      ],
      "metadata": {
        "id": "BzGkAiXfbg0B"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "find the positional min and  max"
      ],
      "metadata": {
        "id": "UVrHolENc3qz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x[9]"
      ],
      "metadata": {
        "id": "Nm0g2OkIbm72"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x"
      ],
      "metadata": {
        "id": "VWsXdds4bq64"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## reshaping the stacking , squeeezing and unsqueezing tensors\n",
        "\n",
        "* Reshaping - reshapes an input te tensor to defined shape\n",
        "\n",
        "* View - Return a view of an input tensor of certain chape but keep the same memory as the original  tensor\n",
        "\n",
        "* Stacking : combine multiple tensor on top of each other (vstack) or side by side\n",
        "\n",
        "* Unsequeeze : addd a 1 dimension to a target tensor\n",
        "\n",
        "* Permute : Return a view of the input with dimension permute (swapped) in a certain way."
      ],
      "metadata": {
        "id": "2vl2q9BPdyAO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Lets create an tensor\n",
        "import torch\n",
        "x = torch.arange(1., 10.)\n",
        "x, x.shape"
      ],
      "metadata": {
        "id": "lTDgRhxZdLLG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# add an extra dimension\n",
        "x_reshaped = x.reshape(1,9)\n",
        "x_reshaped, x_reshaped.shape"
      ],
      "metadata": {
        "id": "zEjd5UYdfCkX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "   # change the view\n",
        "z = x.view(1, 9)\n",
        "z, z.shape\n"
      ],
      "metadata": {
        "id": "M3zGDsP-fLdU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# changing z changes x because a view a tensor share the same meomory a sthe original\n",
        "z[:, 0] = 5\n",
        "z, x"
      ],
      "metadata": {
        "id": "P1eDefjRgKK_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# stack tye tensor in the top of each other\n",
        "x_stacked = torch.stack([x,x,x,x], dim=0)\n",
        "x_stacked"
      ],
      "metadata": {
        "id": "Mh8Sl7kUgg6k"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#  torch.sequeeze() - remove all single dimension from a target tensor\n",
        "print(f\"Previous tensor: {x_reshaped}\")\n",
        "print(f\"Previous shape: {x_reshaped.shape}\")\n",
        "\n",
        "# remove extra dimension from x_reshaped\n",
        "x_squeezed = x_reshaped.squeeze()\n",
        "print(f\"\\nNew tensor: {x_squeezed}\")\n",
        "print(f\"New shape: {x_squeezed.shape}\")\n",
        "\n"
      ],
      "metadata": {
        "id": "DAgoYtgxg-vp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# torch unsqueeze() - adds dimension to a target tensor at a specific dim (dimension)\n",
        "print(f\"Previous tensor: {x_squeezed}\")\n",
        "print(f\"Previous shape: {x_squeezed.shape}\")\n",
        "\n",
        "# add an extra dimension with unsqueeze\n",
        "x_unsqueezed = x_squeezed.unsqueeze(dim=1)\n",
        "print(f\"\\nNew tensor: {x_unsqueezed}\")\n",
        "print(f\"New shape: {x_unsqueezed.shape}\")"
      ],
      "metadata": {
        "id": "LvgvPrmJhtu1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x_reshaped.squeeze()"
      ],
      "metadata": {
        "id": "FvwuDQ-WhxUI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x_reshaped.squeeze().shape"
      ],
      "metadata": {
        "id": "Yqag8ipmh09W"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# torch.permute - rearrange the dimension of  a target in a specified order\n",
        "x = torch.randn(2,3,5)\n",
        "x.size()\n",
        "torch.permute(x, (2,0,1)).size() # 2,0,1 is showing the index number of the 2,3,5 it will arrange in this manner"
      ],
      "metadata": {
        "id": "Fi35ouU7h848"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## indexing (Selecting data from tensors)\n",
        "\n",
        "indeximg with Pytorch is similar to insexing with Numpy."
      ],
      "metadata": {
        "id": "2Uo3toFyS_4V"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "## creating a tensor\n",
        "import torch\n",
        "x = torch.arange(1,10).reshape(1,3,3)\n",
        "x , x.shape"
      ],
      "metadata": {
        "id": "lqyfLLYNjkag"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# let's index on new tensor\n",
        "x[0]"
      ],
      "metadata": {
        "id": "g2Ca_TfaTbsU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "Pf8_VLPeT5Ns"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#lets index on the middle bracket dim=1\n",
        "x[0] [0]"
      ],
      "metadata": {
        "id": "da3iv1gmTuCG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# lets index the most inner bracket\n",
        "x[0] [0] [0]"
      ],
      "metadata": {
        "id": "QY3mZpVAT6ui"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# lets the inner on the most brackets (last dimenesion)\n",
        "x[0][1][1]"
      ],
      "metadata": {
        "id": "8AUqgmFVUNVQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x[0][2][2]"
      ],
      "metadata": {
        "id": "omXMDld-V41H"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# get all the value of the 0 dimension but only the 1 index value of 1st and 2nd diemsion\n",
        "\n",
        "x[: , 1 , 1]"
      ],
      "metadata": {
        "id": "qmSpL9ZCV_XZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# get the index 0 of 0th and 1sy dimesion and all values of 2nd dimesnion\n",
        "x[0,0,:]"
      ],
      "metadata": {
        "id": "_rEQDv3cWtr9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "VmlEZbJ8Xlpa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# ondex on  the x to return 9\n",
        "print(x[0] [2] [2])\n",
        "\n",
        "# indexing on x to returns 3,6,9\n",
        "print(x[0,:,2])\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 211
        },
        "id": "CqzAzwj4W6LC",
        "outputId": "d8a1dc47-c8a7-4c03-85d7-be457723648d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "name 'x' is not defined",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-47-7974964a5d0f>\u001b[0m in \u001b[0;36m<cell line: 2>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# ondex on  the x to return 9\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;31m# indexing on x to returns 3,6,9\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mNameError\u001b[0m: name 'x' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Pytorch tensors & Numpy\n",
        "\n",
        "Numpy is a popular scientific python numerical computing library.\n",
        "\n",
        "And because of this , Pytorch has functionallity to intteract with it\n",
        "* data in Numpy, want in pytorch -> torch.from_numpy(ndarray)\n",
        "* Pytorcch tensor -> Numpy-> torch.Tensor.numpy()"
      ],
      "metadata": {
        "id": "JZ4-CLpdY3tX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Numpy array to tensor\n",
        "import numpy as np\n",
        "import torch\n",
        "\n",
        "array = np.arange(1.0, 8.0)\n",
        "tensor = torch.from_numpy(array)   # warning when converting from numpy--> pytorch reflects numpy default datatype of the float64 unless specified otheriwse\n",
        "array, tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W3Jxznt3Xyio",
        "outputId": "ddb23469-e1a5-48a4-d2f2-d7bcf5010ae0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([1., 2., 3., 4., 5., 6., 7.]),\n",
              " tensor([1., 2., 3., 4., 5., 6., 7.], dtype=torch.float64))"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# change th evalue of error , what will this do to 'tensor'?\n",
        "array = array+1\n",
        "array, tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hR83sRpjZg_a",
        "outputId": "3ce5037d-785e-4531-f82e-e9568e41cb7a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([2., 3., 4., 5., 6., 7., 8.]),\n",
              " tensor([1., 2., 3., 4., 5., 6., 7.], dtype=torch.float64))"
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Tensor to Numpy array\n",
        "tensor = torch.ones(7)\n",
        "numpy_tensor = tensor.numpy()\n",
        "tensor, numpy_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZpmR12I-Zwid",
        "outputId": "bbf1b002-ceeb-4036-9ace-dfd43ae84654"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tensor([1., 1., 1., 1., 1., 1., 1.]),\n",
              " array([1., 1., 1., 1., 1., 1., 1.], dtype=float32))"
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "numpy_tensor.dtype"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T8sDQ-DhaBXF",
        "outputId": "2330d93b-0e83-430e-9381-018c31c3dff1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "dtype('float32')"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tensor.dtype"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LreHbNwgbC3X",
        "outputId": "1fe619c6-1c8d-4f8f-c87e-ed3f29c60524"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.float32"
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# change the tensor -> what happens to the numpy\n",
        "tensor = tensor + 1\n",
        "tensor, numpy_tensor"
      ],
      "metadata": {
        "id": "DLkBSzmnbHWE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "numpy_tensor.dtype"
      ],
      "metadata": {
        "id": "gegHjNylbSF7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Reproducibility (ttruing to take random out of random)\n",
        "\n",
        "in short how a neural network learns:\n",
        "\n",
        "start with random numbers -> tensor operation -> update random numbers to try and make them better representaion of the data -> again-> again-> again.\n",
        "\n",
        "To reduce the randomness in neural network and pytorch comes with concept of a **random seed**.\n",
        "\n",
        "Essentially what the random seed does is 'floavour' the randomness."
      ],
      "metadata": {
        "id": "4RPYY9EJbmQO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# import torch\n",
        "# create twon random tesors\n",
        "random_tensor_A = torch.rand(3,4)\n",
        "random_tensor_B = torch.rand(3,4)\n",
        "\n",
        "print(random_tensor_A)\n",
        "print(random_tensor_B)\n",
        "print(random_tensor_A == random_tensor_B)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "K5Tck_9rbaLE",
        "outputId": "7e1bae76-4a9e-428c-ca95-0c359c724876"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[0.5615, 0.1303, 0.1790, 0.1256],\n",
            "        [0.5668, 0.6054, 0.9698, 0.9131],\n",
            "        [0.9518, 0.5540, 0.8135, 0.2471]])\n",
            "tensor([[0.1790, 0.9394, 0.9536, 0.3423],\n",
            "        [0.3636, 0.1774, 0.1052, 0.1002],\n",
            "        [0.9070, 0.8514, 0.1779, 0.4880]])\n",
            "tensor([[False, False, False, False],\n",
            "        [False, False, False, False],\n",
            "        [False, False, False, False]])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# lets make some random reproducicle tessors\n",
        "import torch\n",
        "\n",
        "# set the random seed\n",
        "Random_seed = 42\n",
        "torch.manual_seed(Random_seed)\n",
        "random_tensor_C = torch.rand(3,4)\n",
        "\n",
        "torch.manual_seed(Random_seed)\n",
        "random_tensor_D = torch.rand(3,4)\n",
        "\n",
        "print(random_tensor_C)\n",
        "print(random_tensor_D)\n",
        "print(random_tensor_C == random_tensor_D)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PVDOXfXMc_tD",
        "outputId": "c19d33c0-9161-44cb-f967-00eb4542d235"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([[0.8823, 0.9150, 0.3829, 0.9593],\n",
            "        [0.3904, 0.6009, 0.2566, 0.7936],\n",
            "        [0.9408, 0.1332, 0.9346, 0.5936]])\n",
            "tensor([[0.8823, 0.9150, 0.3829, 0.9593],\n",
            "        [0.3904, 0.6009, 0.2566, 0.7936],\n",
            "        [0.9408, 0.1332, 0.9346, 0.5936]])\n",
            "tensor([[True, True, True, True],\n",
            "        [True, True, True, True],\n",
            "        [True, True, True, True]])\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Extra resources for reproducibility:\n",
        "https://pytorch.org/docs/stable/notes/randomness.html"
      ],
      "metadata": {
        "id": "TIzOXDjKgEYg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Running tensors and pytorch objects on the GPUs (and making faster computations)\n",
        "\n",
        "GPUs = faster computation on numbers, thanks to CUDA + NVIDIA Hardware +Pytorch working behind the scenes to make everything hunky dory (good)."
      ],
      "metadata": {
        "id": "muESlGy5hmxR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# ## Getting a GPU\n",
        "# Eassist way to use --> google colab-> what we are usig .\n",
        "# use yourt own gpu but requires the investment because its expensive\n",
        "\n",
        "# you use cloud gpu -GCP, AWS AZURE"
      ],
      "metadata": {
        "id": "GmkWsY1Ue2JB"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "2. Check the GPU access with Pytorch"
      ],
      "metadata": {
        "id": "2SPdCeqvBib7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# cchecking for the GPU acees with pytorch\n",
        "import torch\n",
        "torch.cuda.is_available()"
      ],
      "metadata": {
        "id": "QMKLOhX6iFqP",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "dd6ce77e-a9d6-4262-8210-314ebfbc11c0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {},
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "because I'm using free version of the google colab thats its false."
      ],
      "metadata": {
        "id": "zumYuDXeBv92"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#setup device agnostic code\n",
        "devise = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
        "devise"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "zza-pNh5Btp7",
        "outputId": "fe469814-d746-4fa5-8e6d-0875b33c5ebe"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'cpu'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "CPU because we use free version of colab."
      ],
      "metadata": {
        "id": "k_s1Ek-jCEVb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# count numbe rof devices\n",
        "torch.cuda.device_count()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yr_6zMhhCC_8",
        "outputId": "81e96c98-1e36-453e-c44d-8a51f83740a9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {},
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "For Pytorch since its capable of running compute on the GPU or CPu its best practice to setup device agnostic code:\n",
        "https://pytorch.org/docs/stable/notes/cuda\n",
        "E.g. run on the gpu if available else on CPU by default"
      ],
      "metadata": {
        "id": "f0V9SJM9CzEN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "3.  putting the tensor and models on the GPU\n",
        "the reseas we want to our tensor/model on tye gPU is because using a GPU result in faster computrational."
      ],
      "metadata": {
        "id": "r5iTWcPMCSCa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# create a tesor (default on the Cpu)\n",
        "tensor = torch.tensor([1,2,3])\n",
        "\n",
        "# Tensor not on GPU\n",
        "print(tensor, tensor.device)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iPUnbU4LEed1",
        "outputId": "95b00658-dbeb-4cdb-c397-d0c8d2d439fc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tensor([1, 2, 3]) cpu\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#  Move Tensor to GPU (if available)\n",
        "tensor_on_gpu = tensor.to(devise)\n",
        "tensor_on_gpu"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BcG1U0kvFKXD",
        "outputId": "49bd60d3-c892-4153-d430-1fdc1b9b600d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1, 2, 3])"
            ]
          },
          "metadata": {},
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "it did'nt move on the gpu because we dont have paid plan we are uisng the free plan which is on cpu"
      ],
      "metadata": {
        "id": "fvrCAZ5-F1ki"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 4. moving tensor back to the CPU"
      ],
      "metadata": {
        "id": "8DpKiHNZFxWz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# if tensor is on gpu , can't tranforn it to numpy\n",
        "tensor_on_gpu.numpy()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kVcubnvPGcKr",
        "outputId": "c1cc89b0-66d0-42da-b4be-7a998ec8ded4"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1, 2, 3])"
            ]
          },
          "metadata": {},
          "execution_count": 62
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "it will be show error if we have paid plan or if we are on the gpu"
      ],
      "metadata": {
        "id": "Umzr0KyyGtJ6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# TO FIX THE GPU TENSOR WITH NUMPY ISSUE , WE CAN FIRST SET IT TO THE CPU\n",
        "\n",
        "tensor_back_on_cpu = tensor_on_gpu.cpu().numpy()\n",
        "tensor_back_on_cpu"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rkPy4mJ8GsGX",
        "outputId": "cc2ad170-b19c-4aac-a3b8-4c5cb20603e6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1, 2, 3])"
            ]
          },
          "metadata": {},
          "execution_count": 63
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Caution:** - it is for the GPU . we are using the free colab."
      ],
      "metadata": {
        "id": "XVH7YsFdHb1b"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Excercises & extra cariculum\n",
        "See: https://www.learnpytorch.io/00_pytorch_fundamentals/#extra-curriculum"
      ],
      "metadata": {
        "id": "nNmjw8cwHaw3"
      }
    }
  ]
}