6. Network Statistical Inference (Student).ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "# Load the data\n",
    "import pandas as pd\n",
    "import networkx as nx\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Introduction\n",
    "\n",
    "In this notebook, we will walk through a hacker's approach to statistical thinking, as applied to network analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Statistics in a Nutshell\n",
    "\n",
    "All of statistics can be broken down into two activities:\n",
    "\n",
    "- Descriptively summarizing data. (a.k.a. **\"descriptive statistics\"**)\n",
    "- Figuring out whether something happened by random chance. (a.k.a. **\"inferential statistics\"**)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Descriptive Statistics\n",
    "\n",
    "- Centrality measures: mean, median, mode\n",
    "- Variance measures: inter-quartile range (IQR), variance and standard deviation\n",
    "\n",
    "### Inferential Statistics\n",
    "\n",
    "- Models of Randomness (see below)\n",
    "- Hypothesis Testing\n",
    "- Fitting Statistical Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "**Models of Randomness**\n",
    "\n",
    "*Discrete Distributions:*\n",
    "\n",
    "- Bernoulli: probability p of success in 1 try (e.g. one flip of coin).\n",
    "- Binomial: probability p of success given n number of tries. `n * bernoulli trials` follows binomial distribution.\n",
    "- Poisson: processes with a per-unit rate.\n",
    "\n",
    "*Continuous Distributions:*\n",
    "\n",
    "- Uniform: equal probability over the range of probable values. Can also be made discrete.\n",
    "- Normal: everyone's favourite."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Tests on Synthetic Social Network\n",
    "\n",
    "In this notebook, we will test whether the synthetic social network we used earlier on follows an Erdos-Renyi model of network connectivity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "G = nx.read_gpickle('Synthetic Social Network.pkl')\n",
    "len(G.nodes())\n",
    "len(G.edges())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "The Erdos-Renyi graph is what is called a \"random graph\" model. In this model, each potential edge in the graph has a probability `p` of being represented in the graph.\n",
    "\n",
    "### Question\n",
    "\n",
    "If a graph has `n` nodes, and `m` edges, assuming that the graph came from an E-R graph, what would be a reasonable estimate for the value `p`?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Exercise\n",
    "\n",
    "Using your answer of `p` above, compute a value of estimated `p` for the synthetic social network data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Write your answer here.\n",
    "    \n",
    "    \n",
    "    \n",
    "\n",
    "p_hat = "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Exercise\n",
    "\n",
    "Using the `networkx` function `erdos_renyi_graph()`, create a single E-R graph, using the value of `p` computed above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Exercise\n",
    "\n",
    "Plot the degree connectivity of the E-R network, sorted from highest to lowest, and plot this on a matplotlib plot. Likewise, do it for the synthetic social network.\n",
    "\n",
    "Hints:\n",
    "\n",
    "- `nx.degree(G)` will give you a dictionary of `{node:degree}`.\n",
    "- The `dict` object has a `dict.values()` function that will return a `dict_values` object, that is iterable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Answer\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Sampling Distributions\n",
    "\n",
    "The E-R graph that we had just created above is one instance of many possible E-R graphs that could be generated, given the parameters **`n`** and **`p`**. \n",
    "\n",
    "The space of all E-R graphs that could be generated is called the **sampling distribution**.\n",
    "\n",
    "Just now, we had drawn one instance from the sampling distribution.\n",
    "\n",
    "To shamelessly borrow a quote from Jake Vanderplas' slides (linked below), in general, computing the sampling distribution is hard, but simulating it is easy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Hypothesis Testing\n",
    "\n",
    "Hypothesis testing is about asking whether the data that we have came from some null model of randomness. First, we set up two competing hypotheses at the beginning, a **null** hypothesis that the data came from some random model, and an **alternative** hypothesis that the data did not come from that random model. Secondly, we compute whether, at some comfortable probabilty of being incorrect, to reject the \"null\" hypothesis, based on the distribution of some test statistic."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Defining a Test Statistic\n",
    "\n",
    "The mean is one of many summarizing statistics that we can use to gauge whether the data on hand came from some random distribution. We can then conduct a hypothesis test to assess how probable it was that the synthetic social network came from an E-R `(n=30, p=0.05)` graph, by comparing the mean of the degree distribution from the synthetic network from the E-R network."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Exercise\n",
    "\n",
    "Compute the mean of the degree distribution of the E-R `(n=30, p=0.05)` graph created above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Answer for E-R graph.\n",
    "er_mean = ____________\n",
    "er_mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "# Answer for synthetic social network.\n",
    "syn_mean = ____________\n",
    "syn_mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Figuring Out the Sampling Distribution\n",
    "\n",
    "A logical procedure for conducting this hypothesis test is to simulate a sampling distribution of ~1000 draws from the E-R `(n=30, p=0.05)` space. We can then compute the probability of seeing the a mean degree distribution greater than or equal to the observed mean."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Exercise\n",
    "\n",
    "Complete the function below to draw 1000 instances of the E-R `(n=30, p=0.05)` graph, and store the distribution of mean node degrees. Ensure that the following parameters are passable into the function:\n",
    "\n",
    "- `n_instances`\n",
    "- `n_nodes`\n",
    "- `p_connection`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Answer\n",
    "\n",
    "def simulate_mean_degree(n_instances, n_nodes, p_connection):\n",
    "    \n",
    "    means = np.zeros(n_instances)\n",
    "    \n",
    "    for i in range(n_instances):\n",
    "        # Fill in the code below.\n",
    "\n",
    "        \n",
    "        \n",
    "        \n",
    "    return means\n",
    "\n",
    "n_instances=1000\n",
    "means = simulate_mean_degree(n_instances=n_instances,\n",
    "                             n_nodes=len(G.nodes()),\n",
    "                             p_connection=p_hat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Plot a histogram of the means, and annotate on the x-axis where the mean of the synthetic network lies.\n",
    "plt.hist(means)\n",
    "plt.axvline(x=syn_mean, linewidth=2, color='red')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Compute the probability that the synthetic network's mean degree would have been seen by chance or not. \n",
    "# This is where the actual hypothesis test comes in.\n",
    "np.sum(means >= syn_mean) / n_instances"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Interim Conclusion\n",
    "\n",
    "We would conclude that the synthetic social network generated above would not have come from an E-R `(n=30, p=0.05)` graph. But what if we had estimated the value of `p_hat` incorrectly?\n",
    "\n",
    "### Exercise\n",
    "\n",
    "Re-calculate `p_hat`, and calculate the probability of seeing the mean degree value by random chance under an E-R model with a new `p_hat` value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "p_hat_new = \n",
    "means_new = simulate_mean_degree(n_instances, 30, p_hat_new)\n",
    "np.sum(means_new >= syn_mean) / n_instances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# Plot this for yourself to convince yourself.\n",
    "plt.hist(means_new)\n",
    "plt.axvline(x=syn_mean, linewidth=2, color='red')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Conclusion\n",
    "\n",
    "We would reject the hypothesis that the synthetic social network came from an E-R graph with `n=30`, `p=0.05`.\n",
    "\n",
    "We would not reject the hypothesis that the synthetic social network came from an E-R graph with `n=30`, `p=0.10`. *That said, we knew in advance that `p=0.10` to begin with.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Further Reading\n",
    "\n",
    "Jake Vanderplas' \"Statistics for Hackers\" slides: https://speakerdeck.com/jakevdp/statistics-for-hackers\n",
    "\n",
    "Allen Downey's \"There is Only One Test\": http://allendowney.blogspot.com/2011/05/there-is-only-one-test.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}