Skip to content

About

One famous ML algorithm a day, each in a single file, numpy only, runnable in 60 seconds, with a test that breaks it.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Tiny ML Lab

One algorithm a day numpy only License: MIT Tests

Fork it, fill the workbooks, check off the days.

One famous ML algorithm a day, each in a single file, numpy only, done in under 60 seconds. Every algorithm comes with two tests: one that proves it works, and one that breaks it, so you learn exactly where each method fails.

This repo is work to do, not just code to read. Every folder has a workbook.py with TODO bodies: fill it in, run it to check yourself, then peek at impl.py if you get stuck.

🔁 How it works

flowchart LR
    A[Fork] --> B[Fill workbook] --> C[Run self-check] --> D[Check off the day] --> E[Join the Showcase]
Loading

⚡ 60-second quickstart

git clone https://github.com/RishabhPande/tiny-ml-lab.git
cd tiny-ml-lab
pip install -r requirements.txt
pytest
python algorithms/kmeans/impl.py

Check your workbooks together

python scripts/check_workbooks.py                         # all days
python scripts/check_workbooks.py perceptron kmeans       # selected folders
python scripts/check_workbooks.py --json > workbook-status.json

The runner finds workbook folders automatically and runs each in a fresh Python process from its own directory. It reports todo, error, timeout, unrecognized, or self-check-ok. Unfilled workbooks print TODO feedback but normally exit zero; this runner checks their feedback instead of treating that zero as completion. Exit codes: 0 only when every selected self-check is OK, 1 when work remains or a check fails, 2 for invalid arguments.

Self-check OK is not correctness grading. It only means the workbook's existing checks ran without their TODO/error feedback. The algorithm tests use impl.py, not your workbook, so passing those tests does not grade your solution either. Review your implementation and compare behavior before checking off PROGRESS.md. This command never fills TODOs or edits your checklist.

Use only trusted local workbooks: running them executes Python with your own permissions, not in a sandbox. The default 60-second timeout per workbook stops the direct process, not arbitrary child processes or network access. Change it with --timeout SECONDS. JSON includes only folder names and statuses, not workbook output or exception details. No extra dependencies are needed.

📅 The days

Day Algorithm One-liner
1 Linear Regression Closed-form least squares, and how one outlier bends it
2 K-Means Lloyd's algorithm, and why it fails on rings
3 K-Nearest Neighbors Majority vote by distance, and the curse of noisy dimensions
4 Gaussian Naive Bayes Independent-feature classification, and what duplicated features do to it
5 Decision Tree CART-style splits, and how an unlimited tree memorizes noise
6 PCA Eigendecomposition, and why it cannot see a circle
7 Tiny Neural Net A numpy MLP that solves XOR, and why scaling matters
8 Logistic Regression Gradient descent on log loss, and why a straight line cannot solve XOR
9 Random Forest Bagged trees with random feature subsets, and how majority vote erases a rare class
10 DBSCAN Density clustering, and why one radius cannot handle varying densities
11 Gradient Boosting Stumps fit to residuals, and why trees cannot extrapolate
12 PageRank Random-surfer ranking, and why a closed loop traps all the rank
13 Support Vector Machine Maximum-margin linear classifier, and why a straight boundary cannot solve XOR
14 Gaussian Mixture Model Soft clustering by EM, and why overlapping blobs cannot be told apart
15 Hidden Markov Model Baum-Welch and Viterbi decoding, and why identical emissions hide the state
16 Attention Scaled dot-product self-attention, and why identical keys collapse it to a mean
17 LoRA Low-rank adapters on a frozen layer, and why a rank-1 update cannot fix a full-rank task
18 RAG Retriever TF-IDF cosine retrieval, and why synonyms it never saw return nothing
19 Autoencoder A tiny MLP that compresses and rebuilds, and why a 1D bottleneck cannot hold a circle
20 Q-Learning Tabular Q-learning on a gridworld, and why a myopic agent never finds a delayed reward
21 Isolation Forest Random splits that isolate outliers fast, and why a tight cluster of anomalies hides itself
22 Perceptron The original linear learner, and why it never settles on XOR
23 AdaBoost Boosted stumps that correct each other, and why label noise turns the reweighting against it
24 Ridge Regression L2-penalized least squares that tames collinear features, and why too much penalty erases the signal
25 Lasso Regression L1-penalized least squares that zeroes useless features, and why it hands a shared effect to one twin feature

💥 Where they break

Four failure modes: K-Means on rings, PCA on a circle, logistic regression on XOR, least squares with an outlier

Four of the ten test_breaks tests, drawn from this repo's own impl.py files on the same synthetic data and seeds. Regenerate with python docs/make_failure_gallery.py (needs matplotlib; the algorithms stay numpy only).

📏 The rules

  • One file per algorithm (impl.py), numpy only, no downloads.
  • Every demo runs in under 60 seconds on synthetic data with a fixed seed.
  • Every algorithm has a test_works test (a real numeric assertion) and a test_breaks test (a real failure mode).
  • Every algorithm has a workbook.py: the same API with TODO bodies and a standalone self-check. Fill it in yourself before reading impl.py.

🙌 How to take part

License

MIT. See LICENSE.

About

One famous ML algorithm a day, each in a single file, numpy only, runnable in 60 seconds, with a test that breaks it.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages