The Little Book of Algorithms

Version 0.3.9

Author

Duc-Tam Nguyen

Published

October 23, 2025

Content

A Friendly Guide from Numbers to Neural Networks

Licensed under CC BY-NC-SA 4.0.

Chapter 1. Foundations of Algorithms

    1. What Is an Algorithm?
    1. Measuring Time and Space
    1. Big-O, Big-Theta, Big-Omega
    1. Algorithmic Paradigms (Greedy, Divide and Conquer, DP)
    1. Recurrence Relations
    1. Searching Basics
    1. Sorting Basics
    1. Data Structures Overview
    1. Graphs and Trees Overview
    1. Algorithm Design Patterns

Chapter 2. Sorting and Searching

    1. Elementary Sorting (Bubble, Insertion, Selection)
    1. Divide-and-Conquer Sorting (Merge, Quick, Heap)
    1. Counting and Distribution Sorts (Counting, Radix, Bucket)
    1. Hybrid Sorts (IntroSort, Timsort)
    1. Special Sorts (Cycle, Gnome, Comb, Pancake)
    1. Linear and Binary Search
    1. Interpolation and Exponential Search
    1. Selection Algorithms (Quickselect, Median of Medians)
    1. Range Searching and Nearest Neighbor
    1. Search Optimizations and Variants

Chapter 3. Data Structures in Action

    1. Arrays, Linked Lists, Stacks, Queues
    1. Hash Tables and Variants (Cuckoo, Robin Hood, Consistent)
    1. Heaps (Binary, Fibonacci, Pairing)
    1. Balanced Trees (AVL, Red-Black, Splay, Treap)
    1. Segment Trees and Fenwick Trees
    1. Disjoint Set Union (Union-Find)
    1. Probabilistic Data Structures (Bloom, Count-Min, HyperLogLog)
    1. Skip Lists and B-Trees
    1. Persistent and Functional Data Structures
    1. Advanced Trees and Range Queries

Chapter 4. Graph Algorithms

    1. Traversals (DFS, BFS, Iterative Deepening)
    1. Strongly Connected Components (Tarjan, Kosaraju)
    1. Shortest Paths (Dijkstra, Bellman-Ford, A*, Johnson)
    1. Shortest Path Variants (0–1 BFS, Bidirectional, Heuristic A*)
    1. Minimum Spanning Trees (Kruskal, Prim, Borůvka)
    1. Flows (Ford–Fulkerson, Edmonds–Karp, Dinic)
    1. Cuts (Stoer–Wagner, Karger, Gomory–Hu)
    1. Matchings (Hopcroft–Karp, Hungarian, Blossom)
    1. Tree Algorithms (LCA, HLD, Centroid Decomposition)
    1. Advanced Graph Algorithms and Tricks

Chapter 5. Dynamic Programming

    1. DP Basics and State Transitions
    1. Classic Problems (Knapsack, Subset Sum, Coin Change)
    1. Sequence Problems (LIS, LCS, Edit Distance)
    1. Matrix and Chain Problems
    1. Bitmask DP and Traveling Salesman
    1. Digit DP and SOS DP
    1. DP Optimizations (Divide & Conquer, Convex Hull Trick, Knuth)
    1. Tree DP and Rerooting
    1. DP Reconstruction and Traceback
    1. Meta-DP and Optimization Templates

Chapter 6. Mathematics for Algorithms

    1. Number Theory (GCD, Modular Arithmetic, CRT)
    1. Primality and Factorization (Miller–Rabin, Pollard Rho)
    1. Combinatorics (Permutations, Combinations, Subsets)
    1. Probability and Randomized Algorithms
    1. Sieve Methods and Modular Math
    1. Linear Algebra (Gaussian Elimination, LU, SVD)
    1. FFT and NTT (Fast Transforms)
    1. Numerical Methods (Newton, Simpson, Runge–Kutta)
    1. Mathematical Optimization (Simplex, Gradient, Convex)
    1. Algebraic Tricks and Transform Techniques

Chapter 7. Strings and Text Algorithms

    1. String Matching (KMP, Z, Rabin–Karp, Boyer–Moore)
    1. Multi-Pattern Search (Aho–Corasick)
    1. Suffix Structures (Suffix Array, Suffix Tree, LCP)
    1. Palindromes and Periodicity (Manacher)
    1. Edit Distance and Alignment
    1. Compression (Huffman, Arithmetic, LZ77, BWT)
    1. Cryptographic Hashes and Checksums
    1. Approximate and Streaming Matching
    1. Bioinformatics Alignment (Needleman–Wunsch, Smith–Waterman)
    1. Text Indexing and Search Structures

Chapter 8. Geometry, Graphics, and Spatial Algorithms

    1. Convex Hull (Graham, Andrew, Chan)
    1. Closest Pair and Segment Intersection
    1. Line Sweep and Plane Sweep Algorithms
    1. Delaunay and Voronoi Diagrams
    1. Point in Polygon and Polygon Triangulation
    1. Spatial Data Structures (KD, R-tree)
    1. Rasterization and Scanline Techniques
    1. Computer Vision (Canny, Hough, SIFT)
    1. Pathfinding in Space (A*, RRT, PRM)
    1. Computational Geometry Variants and Applications

Chapter 9. Systems, Databases, and Distributed Algorithms

    1. Concurrency Control (2PL, MVCC, OCC)
    1. Logging, Recovery, and Commit Protocols
    1. Scheduling (Round Robin, EDF, Rate-Monotonic)
    1. Caching and Replacement (LRU, LFU, CLOCK)
    1. Networking (Routing, Congestion Control)
    1. Distributed Consensus (Paxos, Raft, PBFT)
    1. Load Balancing and Rate Limiting
    1. Search and Indexing (Inverted, BM25, WAND)
    1. Compression and Encoding in Systems
    1. Fault Tolerance and Replication

Chapter 10. AI, ML, and Optimization

    1. Classical ML (k-means, Naive Bayes, SVM, Decision Trees)
    1. Ensemble Methods (Bagging, Boosting, Random Forests)
    1. Gradient Methods (SGD, Adam, RMSProp)
    1. Deep Learning (Backpropagation, Dropout, Normalization)
    1. Sequence Models (Viterbi, Beam Search, CTC)
    1. Metaheuristics (GA, SA, PSO, ACO)
    1. Reinforcement Learning (Q-learning, Policy Gradients)
    1. Approximation and Online Algorithms
    1. Fairness, Causal Inference, and Robust Optimization
    1. AI Planning, Search, and Learning Systems