2.md

• Adv Data Structures

  • Hash table

• Trees

  • Trees - Notes & Background
  • Binary search trees: BSTs
  • Heap / Priority Queue / Binary Heap
  • balanced search trees (general concept, not details)
  • traversals: preorder, inorder, postorder, BFS, DFS

• Sorting

  • selection
  • insertion
  • heapsort
  • quicksort
  • merge sort

• Graphs

  • directed
  • undirected
  • adjacency matrix
  • adjacency list
  • traversals: BFS, DFS

• Recursion

• Endianness

• Object-Oriented Programming

• Hash table

  • Videos:

    • Hashing with Chaining (video)
    • Table Doubling, Karp-Rabin (video)
    • Open Addressing, Cryptographic Hashing (video)
    • PyCon 2010: The Mighty Dictionary (video)
    • PyCon 2017: The Dictionary Even Mightier (video)
    • (Advanced) Randomization: Universal & Perfect Hashing (video)
    • (Advanced) Perfect hashing (video)

  • Online Courses:

    • Core Hash Tables (video)
    • Data Structures (video)
    • Phone Book Problem (video)
    • distributed hash tables:
      • Instant Uploads And Storage Optimization In Dropbox (video)
      • Distributed Hash Tables (video)

  • Implement with array using linear probing

    • hash(k, m) - m is size of hash table
    • add(key, value) - if key already exists, update value
    • exists(key)
    • get(key)
    • remove(key)

Trees

• Trees - Notes & Background

  • Series: Trees (video)
  • basic tree construction
  • traversal
  • manipulation algorithms
  • BFS(breadth-first search) and DFS(depth-first search) (video)
    • BFS notes:
      • level order (BFS, using queue)
      • time complexity: O(n)
      • space complexity: best: O(1), worst: O(n/2)=O(n)
    • DFS notes:
      • time complexity: O(n)
      • space complexity: best: O(log n) - avg. height of tree worst: O(n)
      • inorder (DFS: left, self, right)
      • postorder (DFS: left, right, self)
      • preorder (DFS: self, left, right)

• Binary search trees: BSTs

  • Binary Search Tree Review (video)
  • Introduction (video)
  • MIT (video)
  • C/C++:
    • Binary search tree - Implementation in C/C++ (video)
    • BST implementation - memory allocation in stack and heap (video)
    • Find min and max element in a binary search tree (video)
    • Find height of a binary tree (video)
    • Binary tree traversal - breadth-first and depth-first strategies (video)
    • Binary tree: Level Order Traversal (video)
    • Binary tree traversal: Preorder, Inorder, Postorder (video)
    • Check if a binary tree is binary search tree or not (video)
    • Delete a node from Binary Search Tree (video)
    • Inorder Successor in a binary search tree (video)
  • Implement:
    • insert // insert value into tree
    • get_node_count // get count of values stored
    • print_values // prints the values in the tree, from min to max
    • delete_tree
    • is_in_tree // returns true if given value exists in the tree
    • get_height // returns the height in nodes (single node's height is 1)
    • get_min // returns the minimum value stored in the tree
    • get_max // returns the maximum value stored in the tree
    • is_binary_search_tree
    • delete_value
    • get_successor // returns next-highest value in tree after given value, -1 if none

• Heap / Priority Queue / Binary Heap

  • visualized as a tree, but is usually linear in storage (array, linked list)
  • Heap
  • Introduction (video)
  • Naive Implementations (video)
  • Binary Trees (video)
  • Tree Height Remark (video)
  • Basic Operations (video)
  • Complete Binary Trees (video)
  • Pseudocode (video)
  • Heap Sort - jumps to start (video)
  • Heap Sort (video)
  • Building a heap (video)
  • MIT: Heaps and Heap Sort (video)
  • CS 61B Lecture 24: Priority Queues (video)
  • Linear Time BuildHeap (max-heap)
  • Implement a max-heap:
    • insert
    • sift_up - needed for insert
    • get_max - returns the max item, without removing it
    • get_size() - return number of elements stored
    • is_empty() - returns true if heap contains no elements
    • extract_max - returns the max item, removing it
    • sift_down - needed for extract_max
    • remove(i) - removes item at index x
    • heapify - create a heap from an array of elements, needed for heap_sort
    • heap_sort() - take an unsorted array and turn it into a sorted array in-place using a max heap or min heap

Sorting

• Notes:

  • Implement sorts & know best case/worst case, average complexity of each:
    • no bubble sort - it's terrible - O(n^2), except when n <= 16
  • Stability in sorting algorithms ("Is Quicksort stable?")
    • Sorting Algorithm Stability
    • Stability In Sorting Algorithms
    • Stability In Sorting Algorithms
    • Sorting Algorithms - Stability
  • Which algorithms can be used on linked lists? Which on arrays? Which on both?
    • I wouldn't recommend sorting a linked list, but merge sort is doable.
    • Merge Sort For Linked List

• For heapsort, see Heap data structure above. Heap sort is great, but not stable

• Sedgewick - Mergesort (5 videos)

  • 1. Mergesort
  • 2. Bottom up Mergesort
  • 3. Sorting Complexity
  • 4. Comparators
  • 5. Stability

• Sedgewick - Quicksort (4 videos)

  • 1. Quicksort
  • 2. Selection
  • 3. Duplicate Keys
  • 4. System Sorts

• UC Berkeley:

  • CS 61B Lecture 29: Sorting I (video)
  • CS 61B Lecture 30: Sorting II (video)
  • CS 61B Lecture 32: Sorting III (video)
  • CS 61B Lecture 33: Sorting V (video)

• Bubble Sort (video)

• Analyzing Bubble Sort (video)

• Insertion Sort, Merge Sort (video)

• Insertion Sort (video)

• Merge Sort (video)

• Quicksort (video)

• Selection Sort (video)

• Merge sort code:

  • Using output array (C)
  • Using output array (Python)
  • In-place (C++)

• Quick sort code:

  • Implementation (C)
  • Implementation (C)
  • Implementation (Python)

• Implement:

  • Mergesort: O(n log n) average and worst case
  • Quicksort O(n log n) average case
  • Selection sort and insertion sort are both O(n^2) average and worst case
  • For heapsort, see Heap data structure above

• Not required, but I recommended them:

  • Sedgewick - Radix Sorts (6 videos)
    • 1. Strings in Java
    • 2. Key Indexed Counting
    • 3. Least Significant Digit First String Radix Sort
    • 4. Most Significant Digit First String Radix Sort
    • 5. 3 Way Radix Quicksort
    • 6. Suffix Arrays
  • Radix Sort
  • Radix Sort (video)
  • Radix Sort, Counting Sort (linear time given constraints) (video)
  • Randomization: Matrix Multiply, Quicksort, Freivalds' algorithm (video)
  • Sorting in Linear Time (video)

As a summary, here is a visual representation of 15 sorting algorithms. If you need more detail on this subject, see "Sorting" section in Additional Detail on Some Subjects

Graphs

Graphs can be used to represent many problems in computer science, so this section is long, like trees and sorting were.

• Notes:

  • There are 4 basic ways to represent a graph in memory:
    • objects and pointers
    • adjacency matrix
    • adjacency list
    • adjacency map
  • Familiarize yourself with each representation and its pros & cons
  • BFS and DFS - know their computational complexity, their trade offs, and how to implement them in real code
  • When asked a question, look for a graph-based solution first, then move on if none

• MIT(videos):

  • Breadth-First Search
  • Depth-First Search

• Skiena Lectures - great intro:

  • CSE373 2012 - Lecture 11 - Graph Data Structures (video)
  • CSE373 2012 - Lecture 12 - Breadth-First Search (video)
  • CSE373 2012 - Lecture 13 - Graph Algorithms (video)
  • CSE373 2012 - Lecture 14 - Graph Algorithms (con't) (video)
  • CSE373 2012 - Lecture 15 - Graph Algorithms (con't 2) (video)
  • CSE373 2012 - Lecture 16 - Graph Algorithms (con't 3) (video)

• Graphs (review and more):

  • 6.006 Single-Source Shortest Paths Problem (video)
  • 6.006 Dijkstra (video)
  • 6.006 Bellman-Ford (video)
  • 6.006 Speeding Up Dijkstra (video)
  • Aduni: Graph Algorithms I - Topological Sorting, Minimum Spanning Trees, Prim's Algorithm - Lecture 6 (video)
  • Aduni: Graph Algorithms II - DFS, BFS, Kruskal's Algorithm, Union Find Data Structure - Lecture 7 (video)
  • Aduni: Graph Algorithms III: Shortest Path - Lecture 8 (video)
  • Aduni: Graph Alg. IV: Intro to geometric algorithms - Lecture 9 (video)
  • CS 61B 2014 (starting at 58:09) (video)
  • CS 61B 2014: Weighted graphs (video)
  • Greedy Algorithms: Minimum Spanning Tree (video)
  • Strongly Connected Components Kosaraju's Algorithm Graph Algorithm (video)

• Full Coursera Course:

  • Algorithms on Graphs (video)

• I'll implement:

  • DFS with adjacency list (recursive)
  • DFS with adjacency list (iterative with stack)
  • DFS with adjacency matrix (recursive)
  • DFS with adjacency matrix (iterative with stack)
  • BFS with adjacency list
  • BFS with adjacency matrix
  • single-source shortest path (Dijkstra)
  • minimum spanning tree
  • DFS-based algorithms (see Aduni videos above):
    • check for cycle (needed for topological sort, since we'll check for cycle before starting)
    • topological sort
    • count connected components in a graph
    • list strongly connected components
    • check for bipartite graph

Even More Knowledge

• Recursion

  • Stanford lectures on recursion & backtracking:
    • Lecture 8 | Programming Abstractions (video)
    • Lecture 9 | Programming Abstractions (video)
    • Lecture 10 | Programming Abstractions (video)
    • Lecture 11 | Programming Abstractions (video)
  • When it is appropriate to use it?
  • How is tail recursion better than not?
    • What Is Tail Recursion Why Is It So Bad?
    • Tail Recursion (video)

• Object-Oriented Programming

  • Optional: UML 2.0 Series (video)
  • SOLID OOP Principles: SOLID Principles (video)

• Endianness

  • Big And Little Endian
  • Big Endian Vs Little Endian (video)
  • Big And Little Endian Inside/Out (video)
    • Very technical talk for kernel devs. Don't worry if most is over your head.
    • The first half is enough.