IPythonD3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "____\n",
    "# \\#D3Boulder \n",
    "###@BrianLehman\n",
    "###https://github.com/blehman\n",
    "\n",
    "____\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# IPython & D3\n",
    "Let's start with a few techniques for working with data in ipython and then build a d3 network graph. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The rpy2.ipython extension is already loaded. To reload it, use:\n",
      "  %reload_ext rpy2.ipython\n"
     ]
    }
   ],
   "source": [
    "# import requirments \n",
    "from IPython.display import Image\n",
    "from IPython.display import display\n",
    "from IPython.display import HTML\n",
    "from datetime import *\n",
    "import json\n",
    "from copy import *\n",
    "from pprint import *\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import json\n",
    "from ggplot import *\n",
    "import networkx as nx\n",
    "from networkx.readwrite import json_graph\n",
    "#from __future__ import http_server\n",
    "from BaseHTTPServer import BaseHTTPRequestHandler\n",
    "from IPython.display import IFrame\n",
    "import rpy2\n",
    "%load_ext rpy2.ipython\n",
    "%R require(\"ggplot2\")\n",
    "% matplotlib inline\n",
    "randn = np.random.randn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### JS with IPython?\n",
    "The nice thing about IPython is that we can write in almost any lanaguage. For example, we can use javascript below and pull in the D3 library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "require.config({\n",
       "    paths: {\n",
       "        d3: \"http://d3js.org/d3.v3.min\" //<-- url \n",
       "        //d3: 'd3/d3.min.js' //<-- local path \n",
       "    }\n",
       "});\n",
       "\n",
       "var x = [1,2,3]"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%javascript\n",
    "require.config({\n",
    "    paths: {\n",
    "        d3: \"http://d3js.org/d3.v3.min\" //<-- url \n",
    "        //d3: 'd3/d3.min.js' //<-- local path \n",
    "    }\n",
    "});\n",
    "\n",
    "var x = [1,2,3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python data | D3 Viz\n",
    "A basic method is to serialze your results and then render html that pulls in the data. In this example, we save a json file and then load the html doc in an IFrame. We're now using D3 in ipython!  \n",
    "\n",
    "The example below is adapted from: \n",
    "* Hagberg, A & Schult, D. & Swart, P. Networkx (2011). Github repository, https://github.com/networkx/networkx/tree/master/examples/javascript/force"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"700\"\n",
       "            height=\"350\"\n",
       "            src=\"force/force.html\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.IFrame at 0x11fc2f050>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import json\n",
    "import networkx as nx\n",
    "from networkx.readwrite import json_graph\n",
    "from IPython.display import IFrame\n",
    "\n",
    "\n",
    "G = nx.barbell_graph(6,3)\n",
    "# this d3 example uses the name attribute for the mouse-hover value,\n",
    "# so add a name to each node\n",
    "for n in G:\n",
    "    G.node[n]['name'] = n\n",
    "\n",
    "# write json formatted data\n",
    "d = json_graph.node_link_data(G) # node-link format to serialize\n",
    "\n",
    "# write json\n",
    "json.dump(d, open('force/force.json','w'))\n",
    "\n",
    "# render html inline\n",
    "IFrame('force/force.html', width=700, height=350)\n",
    "#print('Or copy all files in force/ to webserver and load force/force.html')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Passing data from IPython to JS\n",
    "Let's create some random numbers and render them in js (see the [stackoverflow explanation](http://stackoverflow.com/questions/26207668/how-to-use-python-defined-variables-in-javascript-code-within-ipython-notebook) and discussion)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "element.append(\"[ 0.47212583  0.06187296 -0.1809927  -0.03154241]\");"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import Javascript\n",
    "\n",
    "import numpy as np\n",
    "mu, sig = 0.05, 0.2\n",
    "rnd = np.random.normal(loc=mu, scale=sig, size=4)\n",
    "\n",
    "## Use the variable rnd above in Javascript:\n",
    "\n",
    "javascript = 'element.append(\"{}\");'.format(str(rnd))\n",
    "\n",
    "Javascript(javascript)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Passing data from JS to IPython\n",
    "We can also interact with js to define python variables (see [this example](https://jakevdp.github.io/blog/2013/06/01/ipython-notebook-javascript-python-communication/)).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<div style=\"background-color:gainsboro; border:solid black; width:300px; padding:20px;\">\n",
       "Name: <input type=\"text\" id=\"var_name\" value=\"foo\"><br>\n",
       "Value: <input type=\"text\" id=\"var_value\" value=\"bar\"><br>\n",
       "<button onclick=\"set_value()\">Set Value</button>\n",
       "</div>\n",
       "\n",
       "<script type=\"text/Javascript\">\n",
       "    function set_value(){\n",
       "        var var_name = document.getElementById('var_name').value;\n",
       "        var var_value = document.getElementById('var_value').value;\n",
       "        var command = var_name + \" = '\" + var_value + \"'\";\n",
       "        console.log(\"Executing Command: \" + command);\n",
       "        \n",
       "        var kernel = IPython.notebook.kernel;\n",
       "        kernel.execute(command);\n",
       "    }\n",
       "</script>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import HTML\n",
    "\n",
    "input_form = \"\"\"\n",
    "<div style=\"background-color:gainsboro; border:solid black; width:300px; padding:20px;\">\n",
    "Name: <input type=\"text\" id=\"var_name\" value=\"foo\"><br>\n",
    "Value: <input type=\"text\" id=\"var_value\" value=\"bar\"><br>\n",
    "<button onclick=\"set_value()\">Set Value</button>\n",
    "</div>\n",
    "\"\"\"\n",
    "\n",
    "javascript = \"\"\"\n",
    "<script type=\"text/Javascript\">\n",
    "    function set_value(){\n",
    "        var var_name = document.getElementById('var_name').value;\n",
    "        var var_value = document.getElementById('var_value').value;\n",
    "        var command = var_name + \" = '\" + var_value + \"'\";\n",
    "        console.log(\"Executing Command: \" + command);\n",
    "        \n",
    "        var kernel = IPython.notebook.kernel;\n",
    "        kernel.execute(command);\n",
    "    }\n",
    "</script>\n",
    "\"\"\"\n",
    "\n",
    "HTML(input_form + javascript)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Click \"Set Value\" then run the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "print x_var"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Custom D3 module.\n",
    "Now we're having fun. The simplicity of this process wins. We can pass data to javascript via a module called visualize that contains an attribute `plot_circle`, which uses jinja to render our js template. The advantage of using jinja to read our html is apparent: we can pass variables directly from python!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "\n",
       ".overlay {\n",
       "  fill: none;\n",
       "  pointer-events: all;\n",
       "}\n",
       "\n",
       "</style>\n",
       "<div id=\"graph_2\"></div>\n",
       "\n",
       "<script type=\"text/javascript\">\n",
       "\n",
       "graph_2 = function(){\n",
       "\n",
       "var margin = {top: 50, right: 20, bottom: 50, left:20},\n",
       "    width = 700 - margin.left - margin.right,\n",
       "    height = 500 - margin.top - margin.bottom;\n",
       "\n",
       "var svg_2 = d3.select(\"#graph_2\").append(\"svg\")\n",
       "    .attr(\"width\", width + margin.left + margin.right)\n",
       "    .attr(\"height\", height + margin.top + margin.bottom)\n",
       "    .append(\"g\")\n",
       "    .call(d3.behavior.zoom().scaleExtent([0.75, 8]).on(\"zoom\", zoom_2))\n",
       "    .append(\"g\");\n",
       "\n",
       "svg_2.append(\"rect\")\n",
       "    .attr(\"class\", \"overlay\")\n",
       "    .attr(\"width\", width + margin.left + margin.right)\n",
       "    .attr(\"height\", height + margin.top + margin.bottom);\n",
       "\n",
       "    draw = function(nodes, svg){\n",
       "\n",
       "     var x_scale = d3.scale.linear()\n",
       "                    .domain([d3.min(nodes, function(d){ return d['x'];}), \n",
       "                             d3.max(nodes, function(d){ return d['x'];})])\n",
       "                    .range([0, width]);\n",
       "\n",
       "    var y_scale = d3.scale.linear()\n",
       "                    .domain([d3.min(nodes, function(d){ return d['y'];}), \n",
       "                             d3.max(nodes, function(d){ return d['y'];})])\n",
       "                    .range([height, 0]);\n",
       "\n",
       "    svg.selectAll(\"circle\")\n",
       "       .data(nodes)\n",
       "       .enter()\n",
       "       .append(\"circle\")\n",
       "       .attr(\"cx\", function(d) { return x_scale(d['x']); })\n",
       "       .attr(\"cy\", function(d) { return y_scale(d['y']); })\n",
       "       .attr(\"r\",  function(d) { return d['r']; })\n",
       "       .attr(\"fill\", \"red\")\n",
       "       .attr(\"stroke-width\",  0.25)\n",
       "       .attr(\"stroke\",  \"#d3d3d3\");\n",
       "\n",
       "    svg.selectAll(\"text\")\n",
       "       .data(nodes)\n",
       "       .enter()\n",
       "       .append(\"text\")\n",
       "       .attr(\"x\", function(d) { return x_scale(d['x']); })\n",
       "       .attr(\"y\", function(d) { return y_scale(d['y']); })\n",
       "       .attr(\"dy\", \".55em\")\n",
       "       .text(function(d){ return d['name']; });\n",
       "\n",
       "    }\n",
       "\n",
       "    var nodes_2 = [{\"y\": 20, \"x\": 10, \"r\": 15, \"name\": \"circle one\"}, {\"y\": 40, \"x\": 40, \"r\": 5, \"name\": \"circle two\"}, {\"y\": 30, \"x\": 20, \"r\": 8, \"name\": \"circle three\"}, {\"y\": 10, \"x\": 25, \"r\": 10, \"name\": \"circle four\"}];\n",
       "\n",
       "    draw(nodes_2, svg_2);\n",
       "\n",
       "    function zoom_2() {\n",
       "      svg_2.attr(\"transform\", \"translate(\" + d3.event.translate + \")scale(\" + d3.event.scale + \")\");\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "function load_lib(url, callback){\n",
       "  var s = document.createElement('script');\n",
       "  s.src = url;\n",
       "  s.async = true;\n",
       "  s.onreadystatechange = s.onload = callback;\n",
       "  s.onerror = function(){console.warn(\"failed to load library \" + url);};\n",
       "  document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "}\n",
       "\n",
       "if ( typeof(d3) !== \"undefined\" ){\n",
       "    console.log('d3 is defined');\n",
       "    graph_2();\n",
       "}else if(typeof define === \"function\" && define.amd){\n",
       "  console.log('trying d3 with require');\n",
       "  require.config({paths: {d3: \"/nbextensions/d3.v3.min\"}});\n",
       "      console.log('trying /nbextensions/d3.v3.min');\n",
       "        \n",
       "      require([\"d3\"], function(d3){\n",
       "        window.d3 = d3;\n",
       "        console.log('loaded d3 with require');\n",
       "        graph_2();\n",
       "  });\n",
       "}else{\n",
       "    consle.log('trying to load from load_lib');\n",
       "    load_lib(\"http://d3js.org/d3.v3.min.js\", graph_2());\n",
       "}\n",
       "\n",
       "</script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from pythonD3 import visualize\n",
    "data = [{'x': 10, 'y': 20, 'r': 15, 'name': 'circle one'}, \n",
    "        {'x': 40, 'y': 40, 'r': 5, 'name': 'circle two'},\n",
    "        {'x': 20, 'y': 30, 'r': 8, 'name': 'circle three'},\n",
    "        {'x': 25, 'y': 10, 'r': 10, 'name': 'circle four'}]\n",
    "\n",
    "visualize.plot_circle(data, id=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html>\n",
       "<meta charset=\"utf-8\">\n",
       "<style>\n",
       "\n",
       "body {\n",
       "  font: 10px sans-serif;\n",
       "}\n",
       "\n",
       ".chord path {\n",
       "  fill-opacity: .67;\n",
       "  stroke: #000;\n",
       "  stroke-width: .5px;\n",
       "}\n",
       "\n",
       "</style>\n",
       "<body>\n",
       "<div id=\"graph_20\"></div>\n",
       "\n",
       "<script type=\"text/javascript\">\n",
       "graph_20 = function(){\n",
       "\n",
       "    // From http://mkweb.bcgsc.ca/circos/guide/tables/\n",
       "    var matrix = [\n",
       "      [11975,  5871, 8916, 2868],\n",
       "      [ 1951, 10048, 2060, 6171],\n",
       "      [ 8010, 16145, 8090, 8045],\n",
       "      [ 1013,   990,  940, 6907]\n",
       "    ];\n",
       "\n",
       "    var chord = d3.layout.chord()\n",
       "        .padding(.10)\n",
       "        .sortSubgroups(d3.descending)\n",
       "        .matrix(matrix);\n",
       "\n",
       "    var width = window.innerWidth,\n",
       "        height = 500,\n",
       "        innerRadius = Math.min(width, height) * .25,\n",
       "        outerRadius = innerRadius * 1.1;\n",
       "\n",
       "    var fill = d3.scale.ordinal()\n",
       "        .domain(d3.range(4))\n",
       "        .range([\"#000000\", \"#FFDD89\", \"#957244\", \"#F26223\"]);\n",
       "\n",
       "    var svg_20 = d3.select(\"#graph_20\").append(\"svg\")\n",
       "        .attr(\"width\", width)\n",
       "        .attr(\"height\", height)\n",
       "      .append(\"g\")\n",
       "        .attr(\"transform\", \"translate(\" + width / 3.5 + \",\" + height / 3 + \")\");\n",
       "\n",
       "    svg_20.append(\"g\").selectAll(\"path\")\n",
       "        .data(chord.groups)\n",
       "      .enter().append(\"path\")\n",
       "        .style(\"fill\", function(d) { return fill(d.index); })\n",
       "        .style(\"stroke\", function(d) { return fill(d.index); })\n",
       "        .attr(\"d\", d3.svg.arc().innerRadius(innerRadius).outerRadius(outerRadius))\n",
       "        .on(\"mouseover\", fade(.1))\n",
       "        .on(\"mouseout\", fade(1));\n",
       "\n",
       "    var ticks = svg_20.append(\"g\").selectAll(\"g\")\n",
       "        .data(chord.groups)\n",
       "      .enter().append(\"g\").selectAll(\"g\")\n",
       "        .data(groupTicks)\n",
       "      .enter().append(\"g\")\n",
       "        .attr(\"transform\", function(d) {\n",
       "          return \"rotate(\" + (d.angle * 180 / Math.PI - 90) + \")\"\n",
       "              + \"translate(\" + outerRadius + \",0)\";\n",
       "        });\n",
       "\n",
       "    ticks.append(\"line\")\n",
       "        .attr(\"x1\", 1)\n",
       "        .attr(\"y1\", 0)\n",
       "        .attr(\"x2\", 5)\n",
       "        .attr(\"y2\", 0)\n",
       "        .style(\"stroke\", \"#000\");\n",
       "\n",
       "    ticks.append(\"text\")\n",
       "        .attr(\"x\", 8)\n",
       "        .attr(\"dy\", \".35em\")\n",
       "        .attr(\"transform\", function(d) { return d.angle > Math.PI ? \"rotate(180)translate(-16)\" : null; })\n",
       "        .style(\"text-anchor\", function(d) { return d.angle > Math.PI ? \"end\" : null; })\n",
       "        .text(function(d) { return d.label; });\n",
       "\n",
       "    svg_20.append(\"g\")\n",
       "        .attr(\"class\", \"chord\")\n",
       "      .selectAll(\"path\")\n",
       "        .data(chord.chords)\n",
       "      .enter().append(\"path\")\n",
       "        .attr(\"d\", d3.svg.chord().radius(innerRadius))\n",
       "        .style(\"fill\", function(d) { return fill(d.target.index); })\n",
       "        .style(\"opacity\", 1);\n",
       "\n",
       "    // Returns an array of tick angles and labels, given a group.\n",
       "    function groupTicks(d) {\n",
       "      var k = (d.endAngle - d.startAngle) / d.value;\n",
       "      return d3.range(0, d.value, 1000).map(function(v, i) {\n",
       "        return {\n",
       "          angle: v * k + d.startAngle,\n",
       "          label: i % 5 ? null : v / 1000 + \"k\"\n",
       "        };\n",
       "      });\n",
       "    }\n",
       "\n",
       "    // Returns an event handler for fading a given chord group.\n",
       "    function fade(opacity) {\n",
       "      return function(g, i) {\n",
       "        svg_20.selectAll(\".chord path\")\n",
       "            .filter(function(d) { return d.source.index != i && d.target.index != i; })\n",
       "          .transition()\n",
       "            .style(\"opacity\", opacity);\n",
       "      };\n",
       "    }\n",
       "}\n",
       "\n",
       "function load_lib(url, callback){\n",
       "  var s = document.createElement('script');\n",
       "  s.src = url;\n",
       "  s.async = true;\n",
       "  s.onreadystatechange = s.onload = callback;\n",
       "  s.onerror = function(){console.warn(\"failed to load library \" + url);};\n",
       "  document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "}\n",
       "\n",
       "if ( typeof(d3) !== \"undefined\" ){\n",
       "    console.log('d3 is defined');\n",
       "    graph_20();\n",
       "}else if(typeof define === \"function\" && define.amd){\n",
       "  console.log('trying d3 with require');\n",
       "  require.config({paths: {d3: \"/nbextensions/d3.v3.min\"}});\n",
       "      console.log('trying /nbextensions/d3.v3.min');\n",
       "\n",
       "      require([\"d3\"], function(d3){\n",
       "        window.d3 = d3;\n",
       "        console.log('loaded d3 with require');\n",
       "        graph_20();\n",
       "  });\n",
       "}else{\n",
       "    consle.log('trying to load from load_lib');\n",
       "    load_lib(\"http://d3js.org/d3.v3.min.js\", graph_20());\n",
       "}\n",
       "\n",
       "</script>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "visualize.plot_chords(id=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "###What's next?\n",
    "**Now we just need to learn how to write useful javacript!**\n",
    "<br>\n",
    "(enter ...the Jason)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}