Replace engineering guesswork with mathematical facts.
RepoPulse ingests live pull request data from any GitHub repository, engineers a statistical feature matrix, applies K-Means clustering to auto-discover bottleneck personas, and presents everything inside a production-grade, dark-mode interactive dashboard.
It answers the question: "Why is our CI/CD pipeline slow?" — with data.
RepoPulse/
├── app.R ← Shiny UI + Server (single entry point)
├── setup.R ← One-time dependency installer
├── R/
│ ├── 01_ingest.R ← GitHub REST API fetcher (paginated, cached)
│ ├── 02_features.R ← Feature engineering pipeline
│ └── 03_ml.R ← K-Means clustering + PCA + Silhouette
├── data/
│ └── raw_prs.rds ← Auto-generated cache (24hr TTL)
└── www/
└── logo.svg ← Dashboard branding asset
Rscript setup.RWithout a token you get 60 API requests/hour. With one: 5,000/hour.
Generate a token at: https://github.com/settings/tokens (no scopes needed for public repos)
# Add to ~/.Renviron for persistence:
GITHUB_PAT=ghp_your_token_here
# Or set per-session:
Sys.setenv(GITHUB_PAT = "ghp_your_token_here")shiny::runApp(".")The dashboard opens at http://127.0.0.1:PORT in your browser.
| Component | Description |
|---|---|
| KPI Cards | Total PRs, Avg TTM, Review Comments, Black Hole count |
| Cluster Scatter | Interactive 2D PCA projection of all PRs, coloured by persona |
| Elbow Curve | WCSS vs k — visual validation of chosen cluster count |
| TTM Histogram | Overlaid distributions per persona |
| Churn Boxplot | Code change magnitude by cluster |
Designed to impress technically. Contains:
- Full K-Means objective function in LaTeX
- Lloyd's Algorithm step-by-step with equations
- Silhouette score formula with interpretation
- Feature engineering decisions with justification
- Log-transform and Z-score normalisation rationale
- PCA projection mathematics
- Live cluster centroid statistics table
- Full searchable/filterable PR dataset
- Filter by persona, PR size, time-to-merge range
- Live stats sidebar (auto-updates on filter change)
- Direct links to each PR on GitHub
Raw API Data
│
▼
Feature Engineering
• time_to_merge_hrs = merged_at − created_at
• code_churn = additions + deletions
• review_friction = comments + review_comments
• commits_per_file = commits / (changed_files + 1)
│
▼
Preprocessing
• IQR outlier removal (3× fence)
• log1p transform (right-skewed features)
• Z-score normalisation: z = (x − μ) / σ
│
▼
K-Means Clustering (k=3 default)
Objective: argmin Σ ||x − μᵢ||²
• nstart = 50 (reduces local minima)
• Optimal k via Elbow + Silhouette
│
▼
PCA Projection (2D, visualisation only)
│
▼
Persona Assignment
⚡ Fast Track — low TTM, low friction
⚙️ Average Churn — median behaviour
🕳️ Review Black Hole — high TTM, high friction
In app.R, find the pipeline reactive and change:
raw <- fetch_pull_requests(owner = "OWASP", repo = "BLT")To any public GitHub repo, e.g.:
raw <- fetch_pull_requests(owner = "tidyverse", repo = "ggplot2")
raw <- fetch_pull_requests(owner = "facebook", repo = "react")
raw <- fetch_pull_requests(owner = "microsoft", repo = "vscode")Delete data/raw_prs.rds to force a fresh API fetch.
Stack: R · Shiny · bslib (Bootstrap 5) · Plotly · httr2 · K-Means · PCA · GitHub REST API
Skills demonstrated:
- REST API integration with pagination + rate-limit handling
- Statistical feature engineering on real-world event data
- Unsupervised ML (K-Means) with rigorous validation (Elbow + Silhouette)
- Dimensionality reduction (PCA) for visualisation
- Production-grade reactive web application
- Dark-mode UI/UX with custom CSS design system
Built with R · Powered by the GitHub REST API · No guesswork.