04_python_numpy_basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`numpy` is one of the most important Python packages in science. It implements a new data type, called `ndarray` (*n-dimensional array*), which is optimized for numerical operations. Here are some things you need to know about arrays:\n",
    "\n",
    "1. In arrays all elements must have the same type and hence occupy the same memory space. You cannot define an array in which one element is an `int` and another is a `float` or `bool`.\n",
    "2. Arrays occupy continous segment of memory, as opposed to lists, which are just pointers to different objects in various part of memory\n",
    "3. Arrays have *constant access time*. If you have a very large `list`, getting elements from it will be progressively more slow. This is not the case with arrays: getting elements is always fast. This is a consequence of the first 2 properties of arrays.\n",
    "4. However, insertion of elements in the middle of an array or appending an array is inefficient. Hence, you should always *preallocate* an array, if you can. For example, if you want to make a loop and add a value to an array on each iteration, it is much more efficient to first define an empty array which has the length of the final result, and then change elements on each iteration of the loop. We will see an example in practice later."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First let's import `numpy`. It is customary to import it as `np`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Simplest thing you can do is to make an array from a list, like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = np.array([1,2,3])\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, there are many other ways of creating arrays, `numpy` package includes a lot of functions which create arrays based of various rules, and we will use some of them throughout this notebook. One of them is `np.zeros(n)` which creates an array of length `n` filled with `0`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = np.zeros(4)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you create an array, it has some very helpful attributes, such as \n",
    "- `ndim` (number of dimensions)\n",
    "- `shape` (length of each dimension)\n",
    "- `size` (total number of elements in the array)\n",
    "- `dtype` (element type)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = np.zeros(10)\n",
    "print(x)\n",
    "print(\"ndim: \", x.ndim)\n",
    "print(\"shape:\", x.shape)\n",
    "print(\"size: \", x.size)\n",
    "print(\"dtype:\", x.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see the same for 2-dimensional array (we will take a look at how to create and work with them in more detail later):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = np.zeros((5,5))\n",
    "print(x)\n",
    "print(\"ndim: \", x.ndim)\n",
    "print(\"shape:\", x.shape)\n",
    "print(\"size: \", x.size)\n",
    "print(\"dtype:\", x.dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another useful function to create arrays is `np.arange(n)` which will create an array of integers from `0` to `n` (not including `n`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = np.arange(20)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also specify beginning and end `np.arange(n,m)` to created an array from `n` to `m` (excluding `m`):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = np.arange(10,30)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, you can specify a step:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = np.arange(10,20,0.5)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Array indexing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Array indexing works exactly the same way as `list` indexing. As always in Python, we start counting from zero. `x[0]` is the first element of the array, `x[3]` gives us *4-th* element of the array, etc:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x[3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x[5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# indexing from the end: -1 is the last element\n",
    "x[-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x[-5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a two-dimensional array using another function from `numpy` -- `np.random.randint`, which will give as an array of random integers up to a certain number. We can also specify the size of the array with *keyword argument* `size`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# create two-dimensional array of size (3,5) filled \n",
    "# with random integers from 10 to 50\n",
    "x2 = np.random.randint(10, 50, size=(3,5))\n",
    "x2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When indexing multidimensional array, specify indexes in each dimension, like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# get element which is in position 0 in first dimension, \n",
    "# and position 4 in second dimension\n",
    "x2[0,4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# get the same element using indexing from the end in \n",
    "# the second dimension\n",
    "x2[0,-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also create 3 dimensional arrays (or any number of dimensions, really) by extending same syntax:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x3 = np.random.randint(10,size=(3,5,7))\n",
    "x3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# get a single element from 3 dimensional array\n",
    "x3[1,3,4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x3[:,0,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Change value of elements in the array\n",
    "\n",
    "You can assign individual in the array by using same syntax as getting the element followed by assignment operation (`=`). Let's remember our 2 dimensional array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# modifying items\n",
    "x2[0,0] = -666\n",
    "x2[0,-1] = 999\n",
    "x2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Keep in mind that `numpy` arrays have fixed type, and they will not \"upcast\" automatically! E.g. if you try to store `float` in the array of type `int`, it will try to convert `float` to `int`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x2[0,0] = 3.1415\n",
    "x2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Array slicing\n",
    "Using *:* within brackes we can access slices of the array with the following pattern (same as `list`):\n",
    "        \n",
    "    x[start:stop:step]\n",
    "    \n",
    "If any of these are unspecified, they are assumed as following: `start=0, stop=`*size of dimension*`, step=1`\n",
    "\n",
    "> `np.random.rand` creates uniform random numbers from 0 to 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = np.random.rand(50)\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**: don't confuse: this is not a 2 dimensional array, it is 1 dimensional array with 50 number in it, and it is just displayed as a 2d table for convenience. The following is how a 2 dimensional array of the same size would look like, note that each line has its own brakets (it is basically array of arrays):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "np.random.rand(10,5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let get back to our 1d array with 50 numbers and look at slicing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = np.random.rand(50)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# first 10 elements\n",
    "x[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# from 5th to 10th element\n",
    "x[4:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# from 11th element until the end\n",
    "x[10:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">**Note**: If you were doubting convenience of zero-indexing (like I was in the beginnig), perhaps this is a time to re-evaluate. Note that with zero indexing and not including end-point in the slice, it is very easy to tell how many elements you will get in the output when you slice: `x[:5]` will give you exactly `5` elements. `x[4:10]` will give exactly `10-4 = 6` elements. There is no uncertainty there, it is all very clear and consistent. For comparison, in MATLAB `x(1:5)` gives you `5-1+1 = 5` elements and `x(2:5)` gives you `5-2+1 = 4` elements.\n",
    "\n",
    ">Another strength of this approach is that if you want to extract consequtive intervals, you end one and start next one with the same index, i.e. `x[:5]` and `x[5:]` are *non-overlaping* consequtive intervals. For comparison, in MATLAB this would become `x(1:5)` and `x(6:end)`.\n",
    "\n",
    ">And this is only beginning, in fact any calculations on intervals are immensely simplified in zero-indexing and not including the end-point in the slice.\n",
    "\n",
    ">Perhaps it is better explained by the famous Edsger Dijkstra argument \"Why numbering should start at zero?\": https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some more slicing examples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# every second element\n",
    "x[::2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# every third element\n",
    "x[::3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# reverse array\n",
    "x = np.arange(10)\n",
    "print(x)\n",
    "print(x[::-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can combine slicing in one dimension with precise indexing in another:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# access third column\n",
    "x2[:,2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Pro-tip**: When slicing an array, it is useful to keep in mind that by default the resulting sub-array is not a separate entity in memory, but is actually accessing the memory location of the original array. In programming terms we can say that it is *view* on the same object, not a copy. Here is a simple example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x2 = np.random.randint(0, 10, size=(3,5))\n",
    "x2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# get first two elements from both dimensions\n",
    "x2_sub = x2[:2,:2]\n",
    "x2_sub"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# modify an element in the new array\n",
    "x2_sub[1,1] = 999\n",
    "x2_sub"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# see that the original array also got modified\n",
    "x2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">This behavior is very useful for working with data, because it saves you a lot of memory and speed for useless copying. If you need to make a copy of the array, you must do it explicitly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# make a copy\n",
    "x2_sub = x2[:2,:2].copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# modify a copy and verify that the original array is intact\n",
    "x2_sub[1,1] = -666\n",
    "print(x2_sub)\n",
    "print()\n",
    "print(x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Operations on arrays\n",
    "\n",
    "We use arrays to speed up computations. The key thing to understand here is that when you make an operation on each element of an array or a list separately, each object has to be *dynamically typed*: during the execusion for each element Python core has to look up at the type of the element (whether it is `int`, `float`, `str`, etc) to see what \"flavour\" of the function to apply to the element. For example, in terms of precise operations on memory, convering `float` to `int` -- `int(5.6)` --  is not the same as converting `str` to `int` -- `int('5')` -- therefore before applying a particular routine to the object, Python must check the type. This is slow. Consider the following piece of code, as an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def compute_reciprocals(values):\n",
    "    \n",
    "    # preallocate array for reciprocals, same length \n",
    "    # as `values` input\n",
    "    output = np.empty(len(values))\n",
    "\n",
    "    # compute reciprocal of each element\n",
    "    for i in range(len(values)):\n",
    "        # each time this division runs, Python must \n",
    "        # check types behind the scenes\n",
    "        output[i] = 1.0 / values[i]\n",
    "\n",
    "    return output\n",
    "\n",
    "# example use\n",
    "values = np.arange(1, 10)\n",
    "print(values)\n",
    "compute_reciprocals(values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This works fine. Now let's see how much time it takes to run this function on an array of 1 million integers. We use `%timeit`, which estimates the time it takes to run the function you wrote afterwards:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "big_array = np.random.randint(1, 100, size=1000000)\n",
    "%timeit compute_reciprocals(big_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we do the same thing, but instead we just divide `1` over our `big_array`. `numpy` automatically assumes that we want to divide `1` by each element of the array. Moreover, it has more efficient ways of doing it for us:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%timeit (1/big_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we did the same thing two times in two different ways and checked the time it takes. Results will wary based on the current state of your computer and its CPU speed, but here is what I got: for `compute_reciprocals` function I got `2.87 s`, and for `(1/big_array)` I got `6.17 ms`. This is almost 500 fold difference in run time! Imagine that you had a script using `numpy` which ran for 10 seconds. If you had written it in a wrong way using loops, it would take almost 1.5 hours to run!\n",
    "\n",
    ">**Pro-tip**: although we used `/` to divide `1/big_array`, this operator `/` is actually a shortcut for a function `np.divide':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "np.divide(1,big_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">All `numpy` operators have functions associated with them, here they are:\n",
    "\n",
    "| Operator\t    | Equivalent ufunc    | Description                           |\n",
    "|---------------|---------------------|---------------------------------------|\n",
    "|``+``          |``np.add``           |Addition (e.g., ``1 + 1 = 2``)         |\n",
    "|``-``          |``np.subtract``      |Subtraction (e.g., ``3 - 2 = 1``)      |\n",
    "|``-``          |``np.negative``      |Unary negation (e.g., ``-2``)          |\n",
    "|``*``          |``np.multiply``      |Multiplication (e.g., ``2 * 3 = 6``)   |\n",
    "|``/``          |``np.divide``        |Division (e.g., ``3 / 2 = 1.5``)       |\n",
    "|``//``         |``np.floor_divide``  |Floor division (e.g., ``3 // 2 = 1``)  |\n",
    "|``**``         |``np.power``         |Exponentiation (e.g., ``2 ** 3 = 8``)  |\n",
    "|``%``          |``np.mod``           |Modulus/remainder (e.g., ``9 % 4 = 1``)|\n",
    "\n",
    "> Why would you want to use the full function notation instead of using an operator? There are some curcumstances when using full function notation will give you more flexibility. If you do a lot of crunching on very large numerical datasets and experience problems with speed and/or memory, certainly take a look at the [NumPy](http://www.numpy.org) documentation (especially at the [ufunc](https://docs.scipy.org/doc/numpy-1.10.0/reference/ufuncs.html) section) and [SciPy](http://www.scipy.org) documentation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Aggregator functions\n",
    "\n",
    "Functions which reduce an array (or a dimension of an array) to a single value are called aggregator functions. Some of the most useful include `sum`, `min`, `max`, `mean`, `median`, `std`, etc. Python has in-built versions of some of these functions, but `numpy` versions are much faster and you should be always using them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "big_array = np.random.rand(1000000)\n",
    "%timeit sum(big_array)\n",
    "%timeit np.sum(big_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most of the aggregator function include a sister-function with `nan` prefix, which does the same, but ignores `NaN` (stands for *Not a Number*) elements. `NaN` is usually used as a placeholder for missing data, so these functions are very useful for working with data. We will revisit this in the future lesson.\n",
    "\n",
    "The following table provides a list of useful aggregation functions available in NumPy:\n",
    "\n",
    "|Function name      |   NaN-ignoring version  | Description                                   |\n",
    "|-------------------|---------------------|-----------------------------------------------|\n",
    "| ``np.sum``        | ``np.nansum``       | Compute sum of elements                       |\n",
    "| ``np.prod``       | ``np.nanprod``      | Compute product of elements                   |\n",
    "| ``np.mean``       | ``np.nanmean``      | Compute median of elements                    |\n",
    "| ``np.std``        | ``np.nanstd``       | Compute standard deviation                    |\n",
    "| ``np.var``        | ``np.nanvar``       | Compute variance                              |\n",
    "| ``np.min``        | ``np.nanmin``       | Find minimum value                            |\n",
    "| ``np.max``        | ``np.nanmax``       | Find maximum value                            |\n",
    "| ``np.argmin``     | ``np.nanargmin``    | Find index of minimum value                   |\n",
    "| ``np.argmax``     | ``np.nanargmax``    | Find index of maximum value                   |\n",
    "| ``np.median``     | ``np.nanmedian``    | Compute median of elements                    |\n",
    "| ``np.percentile`` | ``np.nanpercentile``| Compute rank-based statistics of elements     |\n",
    "| ``np.any``        | N/A                 | Evaluate whether any elements are True        |\n",
    "| ``np.all``        | N/A                 | Evaluate whether all elements are True        |\n",
    "\n",
    "We won't discuss each in detail, but feel free to try them for youself."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just to mention 2 things about aggregation functions. \n",
    "\n",
    "**First**, some of them (`sum`, `min`, `max` and some others) can be accessed via method notation, like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# print min, max and sum of the array\n",
    "print('Min:', big_array.min())\n",
    "print('Max:', big_array.max())\n",
    "print('Sum:', big_array.sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will notice this pattern in Python further, it is quite frequent that you can do something using a function, or a method. Usually there is no difference, but sometimes one approach offers some advantage in terms of code length and/or clarity.\n",
    "\n",
    "And **second**, for multidimensional arrays, you can specify `axis` parameter to make aggregation only over a specific axis. By default, they will aggregate over all the array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "multi_dim_array = np.random.randint(100, size=(5,10))\n",
    "multi_dim_array"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# default behavior gives maximum of all elements of the array \n",
    "np.max(multi_dim_array)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# specifying axis gives you control over which dimension is aggregated;\n",
    "# in this particular case, the function will give max of every column \n",
    "np.max(multi_dim_array, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Troubles with floats\n",
    "Example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x = 0.1 + 0.2\n",
    "x == 0.3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why this happens? If you can spare 9 minutes of your time, I suggest you to watch a great explanation on <a href=\"https://www.youtube.com/watch?v=PZRI1IfStY0\">Computerphile</a> youtube channel. You will understand how floating numbers are stored and why the floating arithmetic is not precise.\n",
    "\n",
    "I will attempt to explain it here in a nutshell. The problem is more evident if we just look at what `0.1 + 0.2` gives us:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "0.1+0.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "As you can see, there is a marginal error in the end. Why is it there? The answer has to do with how numbers (in particular, `float` type) are stored in memory: they can only store a certain number of *significant digits*. In situations when the precision requires an infinite number of repeating digits, it will fail. It is easier to understand with an analogy between fractions and decimals.\n",
    "\n",
    "Think about a fraction `1/3`. If you try to write it as a decimal, you'd have to write `0.33333333...` and so on. But what if you were able to only store 5 significant digits after `0.`? You'd have to write `0.33333`, that's the best approximation you can do. Now if you try to do arithmetic with `1/3`, let's say `1/3 + 1/3 + 1/3 = 3`. However, if you try to do it in decimal where you can only store 5 significant digits, you'd get `0.99999`.\n",
    "\n",
    "A similar thing happens in the computers when you write any decimals (like `0.1` and `0.2`), because computers cannot store decimal notation in memory, they have to translate them to *binary*. And sometimes with this translation you get a repeating pattern, just like with `0.33333...`. But computers cannot store infinite numbers, they have to cut it after some digits. This introduces errors when working with `float`, and it will happen in any language, because it is a fundamental property of computers.\n",
    "\n",
    "Based on these difficulties, there are 2 fundamental principles one has to keep in mind when working with `float` type.\n",
    "\n",
    "**First**: Never compare equality of floats, this is exactly same mistake which we did in the script from the survey:\n",
    "    \n",
    "    x = 0.1 + 0.2\n",
    "    y = x == 0.3\n",
    "    \n",
    "The result will sometimes be `True`, sometimes `False`, based on which numbers exactly you try, but the point is that it is *inconsistent*, you cannot trust it. It is usually fine to compare which float is larger though. (**Pro-tip**: if you ever do need to compare equality of floats and cannot get around it, there is function called <a href=\"https://docs.scipy.org/doc/numpy-dev/reference/generated/numpy.isclose.html\">`isclose`</a> in the `numpy` package, which will compare two things up to a specified tolerance).\n",
    "\n",
    "**Second**: Be very careful when doing summation or difference between floats of vastly different order: the smaller ones can get lost in the noise. Consider the following example, where we add `0.001` one thousand times to a large number $10^{13}$, and expect to get $10^{13}+1$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "a = 10.0**13\n",
    "for i in range(1000):\n",
    "    a = a + 0.001\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "As you can see, the error is almost the same size as the number we added.\n",
    "\n",
    "**Pro-tip**: if you even need to do very precise operations with `float` type, check out <a href=\"https://docs.python.org/2/library/decimal.html\">`decimal`</a> module in Python)\n",
    "\n",
    "If you want to dig a little deeper on that, check out this really great page: http://floating-point-gui.de/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.5",
   "language": "python",
   "name": "py35"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}