Fork it, fill the workbooks, check off the days.
One famous ML algorithm a day, each in a single file, numpy only, done in under 60 seconds. Every algorithm comes with two tests: one that proves it works, and one that breaks it, so you learn exactly where each method fails.
This repo is work to do, not just code to read. Every folder has a workbook.py with TODO bodies: fill it in, run it to check yourself, then peek at impl.py if you get stuck.
flowchart LR
A[Fork] --> B[Fill workbook] --> C[Run self-check] --> D[Check off the day] --> E[Join the Showcase]
git clone https://github.com/RishabhPande/tiny-ml-lab.git
cd tiny-ml-lab
pip install -r requirements.txt
pytest
python algorithms/kmeans/impl.pypython scripts/check_workbooks.py # all days
python scripts/check_workbooks.py perceptron kmeans # selected folders
python scripts/check_workbooks.py --json > workbook-status.jsonThe runner finds workbook folders automatically and runs each in a fresh Python
process from its own directory. It reports todo, error, timeout,
unrecognized, or self-check-ok. Unfilled workbooks print TODO feedback but
normally exit zero; this runner checks their feedback instead of treating that
zero as completion. Exit codes: 0 only when every selected self-check is OK,
1 when work remains or a check fails, 2 for invalid arguments.
Self-check OK is not correctness grading. It only means the workbook's
existing checks ran without their TODO/error feedback. The algorithm tests use
impl.py, not your workbook, so passing those tests does not grade your solution
either. Review your implementation and compare behavior before checking off
PROGRESS.md. This command never fills TODOs or edits your checklist.
Use only trusted local workbooks: running them executes Python with your own
permissions, not in a sandbox. The default 60-second timeout per workbook stops
the direct process, not arbitrary child processes or network access. Change it
with --timeout SECONDS. JSON includes only folder names and statuses, not
workbook output or exception details. No extra dependencies are needed.
| Day | Algorithm | One-liner |
|---|---|---|
| 1 | Linear Regression | Closed-form least squares, and how one outlier bends it |
| 2 | K-Means | Lloyd's algorithm, and why it fails on rings |
| 3 | K-Nearest Neighbors | Majority vote by distance, and the curse of noisy dimensions |
| 4 | Gaussian Naive Bayes | Independent-feature classification, and what duplicated features do to it |
| 5 | Decision Tree | CART-style splits, and how an unlimited tree memorizes noise |
| 6 | PCA | Eigendecomposition, and why it cannot see a circle |
| 7 | Tiny Neural Net | A numpy MLP that solves XOR, and why scaling matters |
| 8 | Logistic Regression | Gradient descent on log loss, and why a straight line cannot solve XOR |
| 9 | Random Forest | Bagged trees with random feature subsets, and how majority vote erases a rare class |
| 10 | DBSCAN | Density clustering, and why one radius cannot handle varying densities |
| 11 | Gradient Boosting | Stumps fit to residuals, and why trees cannot extrapolate |
| 12 | PageRank | Random-surfer ranking, and why a closed loop traps all the rank |
| 13 | Support Vector Machine | Maximum-margin linear classifier, and why a straight boundary cannot solve XOR |
| 14 | Gaussian Mixture Model | Soft clustering by EM, and why overlapping blobs cannot be told apart |
| 15 | Hidden Markov Model | Baum-Welch and Viterbi decoding, and why identical emissions hide the state |
| 16 | Attention | Scaled dot-product self-attention, and why identical keys collapse it to a mean |
| 17 | LoRA | Low-rank adapters on a frozen layer, and why a rank-1 update cannot fix a full-rank task |
| 18 | RAG Retriever | TF-IDF cosine retrieval, and why synonyms it never saw return nothing |
| 19 | Autoencoder | A tiny MLP that compresses and rebuilds, and why a 1D bottleneck cannot hold a circle |
| 20 | Q-Learning | Tabular Q-learning on a gridworld, and why a myopic agent never finds a delayed reward |
| 21 | Isolation Forest | Random splits that isolate outliers fast, and why a tight cluster of anomalies hides itself |
| 22 | Perceptron | The original linear learner, and why it never settles on XOR |
| 23 | AdaBoost | Boosted stumps that correct each other, and why label noise turns the reweighting against it |
| 24 | Ridge Regression | L2-penalized least squares that tames collinear features, and why too much penalty erases the signal |
| 25 | Lasso Regression | L1-penalized least squares that zeroes useless features, and why it hands a shared effect to one twin feature |
Four of the ten test_breaks tests, drawn from this repo's own impl.py files on the same synthetic data and seeds. Regenerate with python docs/make_failure_gallery.py (needs matplotlib; the algorithms stay numpy only).
- One file per algorithm (
impl.py), numpy only, no downloads. - Every demo runs in under 60 seconds on synthetic data with a fixed seed.
- Every algorithm has a
test_workstest (a real numeric assertion) and atest_breakstest (a real failure mode). - Every algorithm has a
workbook.py: the same API with TODO bodies and a standalone self-check. Fill it in yourself before readingimpl.py.
- Start here: copy PROGRESS.md into your fork and check days off there as you finish them.
- Want the next algorithm? Vote in REQUESTS.md.
- Want to add an algorithm? Read CONTRIBUTING.md.
- Finished days in your fork? Add yourself to the Fork Hall of Fame.
MIT. See LICENSE.
