02_TensorFlow_Basis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tensor Flow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TensorFlow™ is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team within Google’s AI organization, it comes with strong support for machine learning and deep learning and the flexible numerical computation core is used across many other scientific domains."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "# Create a 1x2 matrix\n",
    "matrix1 = tf.constant([[3., 3.]])\n",
    "\n",
    "# Create a 2x1 matrix\n",
    "matrix2 = tf.constant([[2.],[2.]])\n",
    "\n",
    "# multiplication\n",
    "product = tf.matmul(matrix1, matrix2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here tensorflow does not `run` the calculations but only create a set of operations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the operations in a session"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[12.]]\n"
     ]
    }
   ],
   "source": [
    "# Start a session\n",
    "sess = tf.Session()\n",
    "\n",
    "# 'run(product)' to do all the operations contained in the product\n",
    "# 'result' numpy `ndarray` object\n",
    "result = sess.run(product)\n",
    "print(result)\n",
    "# ==> [[ 12.]]\n",
    "\n",
    "# Close the session\n",
    "sess.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[12.]]\n"
     ]
    }
   ],
   "source": [
    "# Session must be closed to release resources\n",
    "# You can use `with` to do this automatically\n",
    "with tf.Session() as sess:\n",
    "  result = sess.run(product)\n",
    "  print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TensorFlow automatically selects which devices to use for the caculations. Normally if the machine has GPUs, it will try to use the first one it finds.\n",
    "It also allows you select which device to use by yourself.\n",
    "\n",
    "```\n",
    "with tf.Session() as sess:\n",
    "  with tf.device(\"/gpu:1\"):\n",
    "    matrix1 = tf.constant([[3., 3.]])\n",
    "    matrix2 = tf.constant([[2.],[2.]])\n",
    "    product = tf.matmul(matrix1, matrix2)\n",
    "    ...\n",
    "```\n",
    "\n",
    "Supported devices:\n",
    " - \"/cpu:0\": CPU.\n",
    " - \"/gpu:0\": The first GPU, if exists\n",
    " - \"/gpu:1\": The second GPU, etc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('/device:CPU:0', 'CPU')]\n"
     ]
    }
   ],
   "source": [
    "# list available devices\n",
    "from tensorflow.python.client import device_lib\n",
    "\n",
    "def get_available_devices():\n",
    "    local_device_protos = device_lib.list_local_devices()\n",
    "    return [(x.name, x.device_type) for x in local_device_protos]\n",
    "\n",
    "print(get_available_devices())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensor\n",
    "\n",
    "TensorFlow sees tensor as a `n` dimentional array or list.\n",
    "\n",
    "A tensor contains a `rank` and a `shape`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variable\n",
    "\n",
    "Variables store the states of a running graph.\n",
    "\n",
    "The follwing example shows how to implement a simple counter with variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n"
     ]
    }
   ],
   "source": [
    "# create a variable op\n",
    "state = tf.Variable(0, name=\"counter\")\n",
    "\n",
    "# create more ops, which adds 1 to the existing value\n",
    "one = tf.constant(1)\n",
    "new_value = tf.add(state, one)\n",
    "update = tf.assign(state, new_value)\n",
    "\n",
    "# init the variable \n",
    "init_op = tf.initialize_all_variables()\n",
    "\n",
    "# start the session\n",
    "with tf.Session() as sess:\n",
    "  # run with the init_op\n",
    "  sess.run(init_op)\n",
    "  # print the init state\n",
    "  print(sess.run(state))\n",
    "  # run update op and print the state\n",
    "  for _ in range(3):\n",
    "    sess.run(update)\n",
    "    print(sess.run(state))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Normally the parameters of a statistics model can be stored into variables. For example, the weights of a neural network can be put into a tensor variable, which can be updated during training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feed\n",
    "\n",
    "Feed is like `local variables`. It releases after the operation.\n",
    "\n",
    "Warning: \"Feeding\" is the least efficient way to feed data into a TensorFlow program and should only be used for small experiments and debugging."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[271.7932  262.925   260.3382  ... 267.51035 271.60425 271.18796]\n",
      " [258.6997  254.5677  258.29233 ... 260.52664 262.65033 261.86035]\n",
      " [252.65413 248.1597  246.60962 ... 253.30356 258.7171  256.05878]\n",
      " ...\n",
      " [241.45462 235.8337  244.81593 ... 247.6417  246.75197 243.59348]\n",
      " [258.55606 244.85881 249.97055 ... 254.2782  253.41292 253.65778]\n",
      " [254.52028 251.21689 262.70874 ... 257.1659  254.47882 262.87305]]\n"
     ]
    }
   ],
   "source": [
    "x = tf.placeholder(tf.float32, shape=(1024, 1024))\n",
    "y = tf.matmul(x, x)\n",
    "\n",
    "with tf.Session() as sess:\n",
    "#   print(sess.run(y))  # ERROR: will fail because x was not fed.\n",
    "\n",
    "  rand_array = np.random.rand(1024, 1024)\n",
    "  print(sess.run(y, feed_dict={x: rand_array}))  # Will succeed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}