08_Source_Control.ipynb

{
 "metadata": {
  "celltoolbar": "Slideshow",
  "name": "",
  "signature": "sha256:98fa52952c6d543875a93bc69178f28aefa1a1548e27ac558625f83f458fd371"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Source Version Control (with Mercurial)\n",
      "======================================="
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "![Traditional Version Control](files/Images/VCS_001.png)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Traditional source code contol mechanisms were used for managing, amongst other things, source code. Revision Control System (RCS) maintained a history of a file in a sbudirectory where the file is.\n",
      "\n",
      "Concurrent Versioning System (CVS) and Subversion (SVN) expanded that idea by allowing multiple users to work collaboratively on a remote repository of code which tracked the history of the files."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "![Distributed Version Control](files/Images/VCS_002.png)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Distributed version control systems decentralise the entire system by keeping complete repositories on each client machine. They can also be incorporated with centralised solutions."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Mercurial\n",
      "---------\n",
      "\n",
      "Mercurial is a distributed version control system (DVCS) or source code management tool (SCM). It is not the only one and it is not the most popular, but for basic usage, many will have very similar workflows. DVCSs provide a number of benefits to the computational scientist:\n",
      "\n",
      "* They define a set of files that form the 'code'/'project'\n",
      "* They keep track of changes and maintain the history of the code\n",
      "* They synchronise multiple copies of the code"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Older VCS (e.g. CSV, Subversion) relied on a central server to maintain the **repository**, but the recent wave of *distributed* code management tools enable each copy of the code to exist as a standalone repository. I'll be giving a quick introduction to **Mercurial** over the rather more popular **git** for a few reasons:\n",
      "\n",
      "* I use it\n",
      "* 10 times less complex\n",
      "* It is installed on wooki\n",
      "* git was a one man project to manage a single project that was not intended to be a DCVS\n",
      "    * Mercurial has been picked up by facebook and works with huge numbers of massive projects\n",
      "* James Bond vs MacGuyver [http://importantshock.wordpress.com/2008/08/07/git-vs-mercurial/](http://importantshock.wordpress.com/2008/08/07/git-vs-mercurial/)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Hosting\n",
      "-------\n",
      "\n",
      "Although there is no need to maintain a central server with DVCS, it is often a good idea to use a cloud provider to:\n",
      "\n",
      "+ maintain a backup of your code\n",
      "+ help you bridge the firewalls around the university\n",
      "+ collaborate with external users\n",
      "+ provide a platform for publishing code"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "By far the biggest hoster is **github** with many massive open source codes hosted there, but my recommended provider is [http://bitbucket.org](http://bitbucket.org) for a few reasons:\n",
      "\n",
      "* It supports Mercurial which is more ingrained in the Python ecosystem\n",
      "* I personallly find the interface better\n",
      "* Unlimited free private repositories (none for github)\n",
      "* Unlimited collaboration using an educational account (from university email) [here if not automatic](https://www.atlassian.com/software/views/bitbucket-academic-license.jsp)\n",
      "* Corporate enterprise backing\n",
      "* Github has become the 4chan of SCM in my opinion [4chan](https://github.com/visionmedia/n/issues/86) [4chan](https://github.com/MrMEEE/bumblebee-Old-and-abbandoned/commit/a047be85247755cdbe0acce6f1dafc8beb84f2ac)\n",
      "* Github are sexist [scandal](http://techcrunch.com/2014/03/15/julie-ann-horvath-describes-sexism-and-intimidation-behind-her-github-exit/)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Workflow\n",
      "--------\n",
      "\n",
      "In a typical workflow you have a local repository that you work on. Each _incremental_ change that you make needs to be **committed** with a **commit message** that logs the change. Once the local copy is in a stable state all the changes can be **pushed** to the hosting, or another repository. A repository can be **cloned** multiple times by multiple users. Updates from other copies can be **pulled**, conflicting changes may need to be **merged**. If you are working on a new feature and do not want to disrupt the **default** branch you will probably need to work with your own **branches**.\n",
      "\n",
      "For more information see:\n",
      "\n",
      "* [http://mercurial.selenic.com/wiki/Tutorial](http://mercurial.selenic.com/wiki/Tutorial)\n",
      "* [https://web.archive.org/web/20110704200525/http://www.eecs.harvard.edu/~margo/cs161/web/resources/mercurial.html](https://web.archive.org/web/20110704200525/http://www.eecs.harvard.edu/~margo/cs161/web/resources/mercurial.html)\n",
      "\n",
      "When you are getting started it is probably very useful to find a cheat sheet that works for you and keep it handy\n",
      "\n",
      "* [http://mercurial.selenic.com/wiki/QuickReferenceCardsAndCheatSheets](http://mercurial.selenic.com/wiki/QuickReferenceCardsAndCheatSheets)\n",
      "\n",
      "Previously I used the example [here](http://titan.chem.uottawa.ca/CHM8309I/08_Mercurial_Example.html) or [here](08_Mercurial_Example.html) to teach this, so you can read through that if you need a refresher on workflow."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Practical example\n",
      "-----------------\n",
      "\n",
      "Set up your Mercurial options in the `~/.hgrc` file, otherwise you can't do much. I have the following, where vimdiff helps to interactively resolve conflicts with other people editing the same files.\n",
      "\n",
      "    [ui]\n",
      "    username = Firstname Lastname <firstname.lastname@example.net>\n",
      "    merge = vimdiff\n",
      "\n",
      "    [merge-tools]\n",
      "    vimdiff.executable = vim\n",
      "    vimdiff.args = -d $base $local $output $other +close +close\n",
      "\n",
      "    [extensions]\n",
      "    color ="
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "hg clone\n",
      "--------\n",
      "\n",
      "Copies of a repository are called **clones**, you can clone from a local repository or from a remote repository through http or ssh. Clones are useful for working on distinct features concurrently without having to worry about which bit of code belongs to which.\n",
      "\n",
      "`hg clone https://tdaff@bitbucket.org/tdaff/chm8309i_2014`\n",
      "\n",
      "Downloads a copy of the remote repository, and makes your working copy the most recent update."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Inspecting\n",
      "----------\n",
      "\n",
      "* **`hg log`** shows the entire history of the repository\n",
      "    + add `-l 10` to just show latest 10 changes\n",
      "* **`hg heads`** shows latest revision in each **branch**\n",
      "* **`hg id`** shows the **hash** of the **changeset** that is unique to that particular state of the repository (and a branch if there is one)\n",
      "    + A **+** shows that something has changed.\n",
      "    + Adding **-n** gives you the local revision number these only apply to that copy of the repository.\n",
      "* **`hg status`** shows if anything has changed in the repository\n",
      "* **`hg diff`** shows what has changed \n",
      "    * `hg diff file` will just show you one file\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "hg pull\n",
      "-------\n",
      "\n",
      "Get any changes from the default remote repository.\n",
      "\n",
      "You can also get changes from someone else's specific repository, for example, try **`hg pull /home/tdaff/Teaching/chm8309i_2014`** and it will pull in changes that I've made, but not yet published to the bitbucket repository.\n",
      "\n",
      "Inspect the code again, and see what changed.\n",
      "\n",
      "hg update\n",
      "---------\n",
      "\n",
      "Bring the working repository to the latest revision in the current branch.\n",
      "\n",
      "+ **`hg up`** latest revision in current branch\n",
      "    + add `tip` for very latest commit\n",
      "    + add branch name, or `default` for latest commit in that branch\n",
      "+ **`hg up -r <number or hash>`** get specific revision (add **`--clean`** to throw away local changes)"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "hg commit\n",
      "---------\n",
      "\n",
      "Make a change to a file and inspect again. If you are happy with the change then you will want to commit that.\n",
      "\n",
      "+ **`hg commit`** or **`hg ci`** brings up the editor so you can enter a message\n",
      "+ **`hg ci -m \"message in here\"`** automatically fills in the message\n",
      "\n",
      "*Always write clear and descriptive commit messages!* They are a permanent summary of the changes to the code!"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Working with files\n",
      "------------------\n",
      "\n",
      "+ **`hg add`** add all untracked files to the repository **`hg add file`** just adds file\n",
      "    + Do not add files that change constantly, such as output or temporary files\n",
      "    + A file called `.hgignore` can specify files to ignore\n",
      "+ **`hg forget`**, **`hg remove`**, **`hg copy`**, **`hg move`** all work with files to manipulate the repository."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Getting messy (pushes and merges)\n",
      "---------------------------------\n",
      "\n",
      "Once each bit of code it finished it can be **pushed** to the remote repository. If other changes have been pushed in the meantime you will need to **merge** those with yours first.\n",
      "\n",
      "+ **`hg pull`** gets all the pending changes. \n",
      "+ **`hg merge -r tip`** will attempt to combine all the changes together by itself\n",
      "+ resolve conflicts\n",
      "    + if using vimdiff a window will pop up with your file on the left and the other one on the right\n",
      "        + \"crtl-w, =\" readjustst the split\n",
      "        + \"crtl-w, crtl-w\" moves between panes\n",
      "        + \"do\" copies a section from the other pane\n",
      "        + \"dp\" puts a section in the other pane\n",
      "    + otherwise there will be errors like `warning: conflicts during merge.`\n",
      "        + chagned sections will both be put in the file between `<<<<<<< local`, `=======` and `>>>>>>> other`.\n",
      "        + make sure you remove the markers and put code in the middle that makes sense\n",
      "        + mark things done with **`hg resolve -m`**\n",
      "+ **make sure the merge makes sense**! `hg diff` to see the changes.\n",
      "+ **`hg ci`** the merge"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Branching\n",
      "---------\n",
      "\n",
      "Branches let you separate development into separate threads and merge them back in when things have stabilised.\n",
      "\n",
      "+ **`hg branch`** shows the current branch name\n",
      "+ **`hg up branchname`** updates to that branch\n",
      "+ To finish a branch **`hg ci --close-branch`** on the branch you want to close, then **`hg merge`** into the main branch.\n",
      "\n",
      "Branches and their names remain in the history forever. It is also worth looking at bookmarks, which are more lightweight and less permanent."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Your own repositories\n",
      "---------------------\n",
      "\n",
      "You can start repositories from scratch with **`hg init`**, but sometimes it is easier to do that through bitbucket. Log in to [http://bitbucket.org](http://bitbucket.org) and try to create a repository, clone it on your working machine, make some changes and push them to the remote repository."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {
      "slideshow": {
       "slide_type": "slide"
      }
     },
     "source": [
      "Assignment 3\n",
      "------------\n",
      "\n",
      "In assignment 3 you will be working in teams. The repository you will need is at [https://bitbucket.org/tdaff/chm8309i_2014_a3](https://bitbucket.org/tdaff/chm8309i_2014_a3) You will all have write permissions to this repository. Someone in your group should fork the repositoy and submit a pull request when it is ready."
     ]
    }
   ],
   "metadata": {}
  }
 ]
}