Text Processing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Processing Text files\n",
    "\n",
    "- [cat](#1.-cat)\n",
    "- [tac](#2.-tac)\n",
    "- [sort](#3.-sort)\n",
    "- [uniq](#4.-uniq)\n",
    "- [cut](#5.-cut)\n",
    "- [paste](#6.-paste)\n",
    "- [join](#7.-join)\n",
    "- [comm](#8.-comm)\n",
    "- [diff](#9.-diff)\n",
    "- [patch](#10.-patch)\n",
    "- [tr](#11.-tr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. cat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hi^M$\n",
      "^M$\n",
      "this file was ^M$\n",
      "made in ms windows 7.^M$\n",
      "^M$\n",
      "^Ilets see^M$\n",
      "what is going on.^M$\n",
      "^M$\n",
      "  here."
     ]
    }
   ],
   "source": [
    "cat -A a.txt # -vET (All: non-printable, end of lines, tabs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hi$\n",
      "$\n",
      "this file was $\n",
      "made in ms windows 7.$\n",
      "$\n",
      "lets see$\n",
      "what is going on.$\n",
      "$\n",
      "  here."
     ]
    }
   ],
   "source": [
    "tr -dc '[:print:]\\n' < a.txt | cat -A # delete, complement set1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     1\thi\n",
      "     2\t\n",
      "     3\tit's\n",
      "     4\t\n",
      "     5\t\n",
      "     6\t\n",
      "     7\ta new file\n"
     ]
    }
   ],
   "source": [
    "cat -n txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     1\thi\n",
      "     2\t\n",
      "     3\tit's\n",
      "     4\t\n",
      "     5\ta new fiel\n"
     ]
    }
   ],
   "source": [
    "cat -ns txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     1\thi\n",
      "\n",
      "     2\tit's\n",
      "\n",
      "\n",
      "\n",
      "     3\ta new file\n"
     ]
    }
   ],
   "source": [
    "cat -b txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     1\thi\n",
      "     2\tit's\n",
      "     3\ta new file\n"
     ]
    }
   ],
   "source": [
    "cat -b < txt | sed /^$/d"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. tac"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "grep -v '^#' /etc/default/hddtemp | sed '/^$/d' > hddtemp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RUN_DAEMON=\"false\"\n",
      "DISKS_NOPROBE=\"\"\n",
      "INTERFACE=\"127.0.0.1\"\n",
      "RUN_SYSLOG=\"0\"\n",
      "OPTIONS=\"\"\n"
     ]
    }
   ],
   "source": [
    "cat hddtemp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OPTIONS=\"\"\n",
      "RUN_SYSLOG=\"0\"\n",
      "INTERFACE=\"127.0.0.1\"\n",
      "DISKS_NOPROBE=\"\"\n",
      "RUN_DAEMON=\"false\"\n"
     ]
    }
   ],
   "source": [
    "tac hddtemp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dmesg | tac"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. sort"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 76\n",
      "-rwxr-xr-x 1 root root  9791 Jul 23  2016 00_header\n",
      "-rwxr-xr-x 1 root root  6258 Mar 15  2016 05_debian_theme\n",
      "-rwxr-xr-x 1 root root 12512 Mar  2 00:31 10_linux\n",
      "-rwxr-xr-x 1 root root 11082 Jul 23  2016 20_linux_xen\n"
     ]
    }
   ],
   "source": [
    "ls -l /etc/grub.d | head -5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 76\n",
      "-rwxr-xr-x 1 root root 11082 Jul 23  2016 20_linux_xen\n",
      "-rwxr-xr-x 1 root root 12512 Mar  2 00:31 10_linux\n",
      "-rwxr-xr-x 1 root root  6258 Mar 15  2016 05_debian_theme\n",
      "-rwxr-xr-x 1 root root  9791 Jul 23  2016 00_header\n"
     ]
    }
   ],
   "source": [
    "ls -l /etc/grub.d | head -5 | sort -k5 -t ' ' # key [field,to field] | separator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-rwxr-xr-x 1 root root 12512 Mar  2 00:31 10_linux\n",
      "-rwxr-xr-x 1 root root 11082 Jul 23  2016 20_linux_xen\n",
      "-rwxr-xr-x 1 root root  9791 Jul 23  2016 00_header\n",
      "-rwxr-xr-x 1 root root  6258 Mar 15  2016 05_debian_theme\n",
      "total 76\n"
     ]
    }
   ],
   "source": [
    "ls -l /etc/grub.d | head -5 | sort -n -r -k5,5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coca\t1.1\t12:20\n",
      "pepsi\t2.0\t00:11\n",
      "coca\t11.3\t00:33\n",
      "fanta\t3.01\t01:21\n",
      "coca\t4.6\t19:01\n",
      "fanta\t4.0\t15:12\n",
      "pepsi\t2.01\t00:01\n"
     ]
    }
   ],
   "source": [
    "cat txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coca\t1.1\t12:20\n",
      "coca\t11.3\t00:33\n",
      "coca\t4.6\t19:01\n",
      "fanta\t3.01\t01:21\n",
      "fanta\t4.0\t15:12\n",
      "pepsi\t2.0\t00:11\n",
      "pepsi\t2.01\t00:01\n"
     ]
    }
   ],
   "source": [
    "sort txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "coca\t1.1\t12:20\n",
      "coca\t4.6\t19:01\n",
      "coca\t11.3\t00:33\n",
      "fanta\t3.01\t01:21\n",
      "fanta\t4.0\t15:12\n",
      "pepsi\t2.0\t00:11\n",
      "pepsi\t2.01\t00:01\n"
     ]
    }
   ],
   "source": [
    "sort -k1,1 -k2,2n txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total 24\n",
      "-rw-r--r-- 1 root root  309 Aug 12  2016 interfaces\n",
      "drwxr-xr-x 2 root root 4096 Jan 24  2016 interfaces.d/\n",
      "drwxr-xr-x 2 root root 4096 Mar  3 15:18 if-down.d/\n",
      "drwxr-xr-x 2 root root 4096 Mar 21 01:44 if-up.d/\n",
      "drwxr-xr-x 2 root root 4096 Apr 13 11:54 if-post-down.d/\n",
      "drwxr-xr-x 2 root root 4096 Apr 13 11:54 if-pre-up.d/\n"
     ]
    }
   ],
   "source": [
    "ls -lF /etc/network  | sort -k8.4 #char4 in key8"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. uniq"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "grub.d\n",
      "passwd\n",
      "grub.d\n",
      "shadow\n",
      "passwd\n",
      "default\n"
     ]
    }
   ],
   "source": [
    "cat txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "default\n",
      "grub.d\n",
      "passwd\n",
      "shadow\n"
     ]
    }
   ],
   "source": [
    "sort txt | uniq -i # makes output unique, ignore case"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      1 default\n",
      "      2 grub.d\n",
      "      2 passwd\n",
      "      1 shadow\n"
     ]
    }
   ],
   "source": [
    "sort txt | uniq -c # count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "grub.d\n",
      "passwd\n"
     ]
    }
   ],
   "source": [
    "sort txt | uniq -d # only duplicates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "default\n",
      "shadow\n"
     ]
    }
   ],
   "source": [
    "sort txt | uniq -u # only uniqe rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "ln txt h-txt; ln txt h2-txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./txt 394077\n",
      "./h2-txt 394077\n",
      "./h-txt 394077\n"
     ]
    }
   ],
   "source": [
    "find -maxdepth 1 -type f -links +1 -printf \"%p %i\\n\" | tee out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sort out | uniq -d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      3 ./h2-txt 394077\n"
     ]
    }
   ],
   "source": [
    "sort out | uniq -f1 -dc # -f skip n first field"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. cut"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[01;31m\u001b[Kadm\u001b[m\u001b[K:x:4:syslog,milad\n"
     ]
    }
   ],
   "source": [
    "head /etc/group | grep adm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "adm cdrom sudo dip plugdev milad lpadmin sambashare wireshark docker\n"
     ]
    }
   ],
   "source": [
    "cat /etc/group | grep milad | cut -f1 -d: | tr '\\n' ' ' | xargs echo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "milad adm cdrom sudo dip plugdev lpadmin sambashare wireshark docker\n"
     ]
    }
   ],
   "source": [
    "id milad -Gn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root:root:/root\n",
      "daemon:daemon:/usr/sbin\n",
      "bin:bin:/bin\n",
      "sys:sys:/dev\n"
     ]
    }
   ],
   "source": [
    "head -4 /etc/passwd | cut --complement -f2-4,7 -d:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Thu May  4 23:53:18 IRDT 2017$\n"
     ]
    }
   ],
   "source": [
    "date | tee txt | cat -A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 158,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n"
     ]
    }
   ],
   "source": [
    "cut -f4 -d' ' < txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 159,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "53:18\n"
     ]
    }
   ],
   "source": [
    "tr -s ' ' < txt | cut -f4 -d' ' | cut -c 4- #  tr: -s, --squeeze-repeats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. paste"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root:x:0:0:root:/root:/bin/bash\n",
      "daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n",
      "bin:x:2:2:bin:/bin:/usr/sbin/nologin\n",
      "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n",
      "sync:x:4:65534:sync:/bin:/bin/sync\n"
     ]
    }
   ],
   "source": [
    "head -5 /etc/passwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root:/bin/bash\n",
      "daemon:/usr/sbin/nologin\n",
      "bin:/usr/sbin/nologin\n",
      "sys:/usr/sbin/nologin\n",
      "sync:/bin/sync\n"
     ]
    }
   ],
   "source": [
    "cut -f1,7 /etc/passwd -d: | head -5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root:/bin/bash\n",
      "daemon:/usr/sbin/nologin\n",
      "bin:/usr/sbin/nologin\n",
      "sys:/usr/sbin/nologin\n",
      "sync:/bin/sync\n"
     ]
    }
   ],
   "source": [
    "cut -f2-6 --complement /etc/passwd -d: | head -5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      "daemon\n",
      "bin\n",
      "sys\n",
      "sync\n"
     ]
    }
   ],
   "source": [
    "head -5 /etc/passwd | cut -f1 -d: | tee users"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/bin/bash\n",
      "/usr/sbin/nologin\n",
      "/usr/sbin/nologin\n",
      "/usr/sbin/nologin\n",
      "/bin/sync\n"
     ]
    }
   ],
   "source": [
    "head -5 /etc/passwd | cut -f7 -d: | tee shells"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/bin/bash:root\n",
      "/usr/sbin/nologin:daemon\n",
      "/usr/sbin/nologin:bin\n",
      "/usr/sbin/nologin:sys\n",
      "/bin/sync:sync\n"
     ]
    }
   ],
   "source": [
    "paste shells users -d':'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. join"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "foo\t21\n",
      "bar\t22\n",
      "baz\t21\n",
      "---\n",
      "foo\tred\n",
      "bar\tblue\n",
      "baz\tblue\n"
     ]
    }
   ],
   "source": [
    "cat namev; echo ---; cat namet;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "foo 21 red\n",
      "bar 22 blue\n",
      "baz 21 blue\n"
     ]
    }
   ],
   "source": [
    "join namev namet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "21 foo\n",
      "22 bar\n",
      "21 baz\n",
      "---\n",
      "foo red\n",
      "bar pink\n",
      "baz yellow\n"
     ]
    }
   ],
   "source": [
    "cat namev; echo ---; cat namet;"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "foo 21 red\n",
      "bar 22 pink\n",
      "baz 21 yellow\n"
     ]
    }
   ],
   "source": [
    "join -1 2 -2 1 namev namet # -j when both are same"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. comm\n",
    "    Compare sorted files FILE1 and FILE2 line by line.\n",
    "    When FILE1 or FILE2 (not both) is -, read standard input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a\n",
      "b\n",
      "c\n",
      "d\n",
      "---\n",
      "a\n",
      "c\n",
      "x\n",
      "z\n"
     ]
    }
   ],
   "source": [
    "cat f1; echo ---; cat f2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\t\ta\n",
      "b\n",
      "\t\tc\n",
      "d\n",
      "\tx\n",
      "\tz\n"
     ]
    }
   ],
   "source": [
    "comm f1 f2"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "uniq1  u2     same in both"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b\n",
      "d\n",
      "\tx\n",
      "\tz\n"
     ]
    }
   ],
   "source": [
    "comm -3 f1 f2 # do not show 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 9. diff\n",
    "\n",
    "it's telling you how to change the first file to make it match the second file.\n",
    "\n",
    "### 9.1. Default view\n",
    "- First file line numbers\n",
    "- Latter\n",
    "    - a: add\n",
    "    - c: change\n",
    "    - d: delete\n",
    "    - ---: separate file 1, 2\n",
    "- Second file line numbers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "blue\n",
      "red\n",
      "orange\n",
      "-\n",
      "blue\n",
      "pink\n",
      "\n",
      "yellow\n"
     ]
    }
   ],
   "source": [
    "echo - | cat f1 - f2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2,3c2,4\n",
      "< red\n",
      "< orange\n",
      "---\n",
      "> pink\n",
      "> \n",
      "> yellow\n"
     ]
    },
    {
     "ename": "",
     "evalue": "1",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "diff f1 f2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Line 2 to 3 from file one has been changed in file two.\n",
    "We should replace them with 2 to 4 from file two to make them same.`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "blue\n",
      "red\n",
      "orange\n",
      "-\n",
      "blue\n",
      "red\n",
      "yellow\n",
      "orange\n"
     ]
    }
   ],
   "source": [
    "echo - | cat f1 - f2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2a3\n",
      "> yellow\n"
     ]
    },
    {
     "ename": "",
     "evalue": "1",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "diff f1 f2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`After line 2 in file one, add line 3 from file two (which is yellow)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "blue\n",
      "red\n",
      "orange\n",
      "-\n",
      "blue\n",
      "orange\n"
     ]
    }
   ],
   "source": [
    "echo - | cat f1 - f2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2d1\n",
      "< red\n"
     ]
    },
    {
     "ename": "",
     "evalue": "1",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "diff f1 f2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9.2. Context\n",
    "- \\s\\s Same\n",
    "- ! Change\n",
    "- \\- Remove from first file\n",
    "- \\+ Add from second to first one"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ardabil\n",
      "Yazd\n",
      "Esfehan\n",
      "Ahvaz\n",
      "__-_-_-__\n",
      "Tehran\n",
      "Yazd\n",
      "Ahvaz\n",
      "Shiraz\n"
     ]
    }
   ],
   "source": [
    "echo \"__-_-_-__\" | cat f1 - f2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "*** f1\t2017-05-05 19:35:18.449963816 +0430\n",
      "--- f2\t2017-05-05 17:26:31.142020824 +0430\n",
      "***************\n",
      "*** 1,4 ****\n",
      "! Ardabil\n",
      "  Yazd\n",
      "- Esfehan\n",
      "  Ahvaz\n",
      "--- 1,4 ----\n",
      "! Tehran\n",
      "  Yazd\n",
      "  Ahvaz\n",
      "+ Shiraz\n"
     ]
    },
    {
     "ename": "",
     "evalue": "1",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "diff -c f1 f2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### Apply the changes to f1:\n",
    "`Ardabil -> Tehran,\n",
    "Remove Esfehan,\n",
    "Add shiraz`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "diff f1 f2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9.3. Unified"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- f1\t2017-05-05 19:35:18.449963816 +0430\n",
      "+++ f2\t2017-05-05 17:26:31.142020824 +0430\n",
      "@@ -1,4 +1,4 @@\n",
      "-Ardabil\n",
      "+Tehran\n",
      " Yazd\n",
      "-Esfehan\n",
      " Ahvaz\n",
      "+Shiraz\n"
     ]
    },
    {
     "ename": "",
     "evalue": "1",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "diff -u f1 f2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9.4. Column "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ardabil\t\t\t\t\t\t\t      |\tTehran\n",
      "Yazd\t\t\t\t\t\t\t\tYazd\n",
      "Esfehan\t\t\t\t\t\t\t      <\n",
      "Ahvaz\t\t\t\t\t\t\t\tAhvaz\n",
      "\t\t\t\t\t\t\t      >\tShiraz\n"
     ]
    },
    {
     "ename": "",
     "evalue": "1",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "diff -y f1 f2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1c1\n",
      "< Ardabil\n",
      "---\n",
      "> Tehran\n",
      "3d2\n",
      "< Esfehan\n",
      "4a4\n",
      "> Shiraz\n"
     ]
    },
    {
     "ename": "",
     "evalue": "1",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "diff f1 f2 # check with default output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9.5. Create an ed script"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "ename": "",
     "evalue": "1",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "diff f1 f2 -e > changes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4a\n",
      "Shiraz\n",
      ".\n",
      "3d\n",
      "1c\n",
      "Tehran\n",
      ".\n"
     ]
    }
   ],
   "source": [
    "cat changes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "echo 'w' >> changes # hey ed write to file ;)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ardabil\n",
      "Yazd\n",
      "Esfehan\n",
      "Ahvaz\n"
     ]
    }
   ],
   "source": [
    "cat f1 # before running ed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "27\n",
      "25\n"
     ]
    }
   ],
   "source": [
    "ed < changes f1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tehran\n",
      "Yazd\n",
      "Ahvaz\n",
      "Shiraz\n"
     ]
    }
   ],
   "source": [
    "cat f2 # after ed did the job"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "diff f1 f2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9b2d361c3076ad25849ff2fcb87c2c8a  f1\n",
      "9b2d361c3076ad25849ff2fcb87c2c8a  f2\n"
     ]
    }
   ],
   "source": [
    "md5sum f1 f2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 10. patch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1c1\n",
      "< Ardabil\n",
      "---\n",
      "> Tehran\n",
      "3d2\n",
      "< Esfehan\n",
      "4a4\n",
      "> Shiraz\n"
     ]
    },
    {
     "ename": "",
     "evalue": "1",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "diff f1 f2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "ename": "",
     "evalue": "1",
     "output_type": "error",
     "traceback": []
    }
   ],
   "source": [
    "diff -u f1 f2 > diff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- f1\t2017-05-05 18:43:09.809986897 +0430\n",
      "+++ f2\t2017-05-05 17:26:31.142020824 +0430\n",
      "@@ -1,4 +1,4 @@\n",
      "-Ardabil\n",
      "+Tehran\n",
      " Yazd\n",
      "-Esfehan\n",
      " Ahvaz\n",
      "+Shiraz\n"
     ]
    }
   ],
   "source": [
    "cat diff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "patching file f1\n"
     ]
    }
   ],
   "source": [
    "patch < diff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tehran\n",
      "Yazd\n",
      "Ahvaz\n",
      "Shiraz\n"
     ]
    }
   ],
   "source": [
    "cat f1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 11. tr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hereissometext.\n"
     ]
    }
   ],
   "source": [
    "echo \"Here is some text.\" | tr -d ' ' "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A\n",
      "Y\n",
      "E\n",
      "A\n"
     ]
    }
   ],
   "source": [
    "tr -c -d '[:upper:]\\n' < f1 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hi this is a test\n"
     ]
    }
   ],
   "source": [
    "echo \"Hi_this_is_a-TEST\" | tr '[:upper:]_-' '[:lower:] '"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- - -\n",
    "## Lecture notes\n",
    "\n",
    "- All lecture notes are available at github in `ipynb` format: [https://github.com/ravexina/linux-notes](https://github.com/ravexina/linux-notes). \n",
    "\n",
    "\n",
    "- There are `html` exports of project available at: [https://ravexina.github.io/linux-notes](https://ravexina.github.io/linux-notes).\n",
    "- - -\n",
    "\n",
    "## License\n",
    "<a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\">\n",
    "<img alt=\"Creative Commons License\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png\" />\n",
    "</a>\n",
    "\n",
    "<span property=\"dct:title\">Linux Notes</span> by \n",
    "<a href=\"http://github.com/ravexina/Linux-Notes\" property=\"cc:attributionName\" rel=\"cc:attributionURL\">Milad As (Ravexina)</a> is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>.\n",
    "\n",
    "- - -\n",
    "\n",
    "<a rel=\"license\" href=\"https://gitlab.com/ravexina/\"><img alt=\"ravexina's gitlab\" style=\"display:inline;border-width:0;height:30px;float:left;\" src=\"img/logo/logo-square.png\" /></a> \n",
    "\n",
    "<a rel=\"license\" href=\"https://github.com/ravexina/\"><img alt=\"ravexina's github\" style=\"float:right;display:inline;border-width:0;height:30px;\" src=\"img/logo/GitHub_Logo.png\" /></a>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}