Promote test → main: measurements + measurement-jobs (chat#1796)

[sweetmantech]

Promote the two merged chat#1796 PRs from test to main (prod).

Includes:

• feat(research): measurement-jobs write resource + retryable backfill drain (chat#1796) #668 — measurement-jobs write resource (POST /research/measurement-jobs, source: current|historical) + card-on-file gate on historical + retryable backfill drain / reclaim.
• feat(research): measurements read collection (chat#1796) #669 — measurements read collection (GET /research/tracks|albums/{id}/measurements, series + aggregate=run_rate + latest).

Both verified on preview (results in the respective PRs). Additive — legacy endpoints keep serving. After this lands, the catalog-value-estimator seed (skills#43) works against prod.

---

Summary by cubic

Promotes consolidated research measurement APIs to prod: adds POST /api/research/measurement-jobs for ingest and GET /api/research/tracks|albums/{id}/measurements for reads. Also adds a Songstats card-on-file gate and makes the historical backfill drain retryable with automatic reclaim in maintenance. Legacy endpoints remain available.

• New Features

  • POST /api/research/measurement-jobs — { scope, source }:
    • source: "current" uses the snapshot pipeline (maps snapshot_id → id).
    • source: "historical" enqueues Songstats backfill ranked by latest counts; skips songs already backfilled; bounded concurrency; returns 202.
    • Validation: auth + research credits; historical requires a payment method; current does not.
  • GET /api/research/tracks/{id}/measurements — daily series; ?aggregate=run_rate over a trailing window. {id} can be ISRC or a Spotify track id (resolved to ISRC).
  • GET /api/research/albums/{id}/measurements — latest count per track (thin remap of playcounts). CORS preflight added for new routes.

• Reliability

  • Backfill drain: 408/429/5xx are marked failed (reclaimable); 404 and other permanent 4xx are marked done (terminal).
  • Maintenance cron reclaims failed and stale in_progress rows before draining and reports the reclaimed count.

^{Written for commit 318ae4a. Summary will update on new commits.}

Summary by CodeRabbit

• New Features

  • Added research measurement endpoints for retrieving album and track playcount data
  • Introduced measurement job creation supporting historical backfills and current snapshots
  • Added measurement aggregation options, including run-rate projections
  • Implemented payment method validation for measurement features

• Improvements

  • Enhanced error handling with better categorization for failed requests
  • Improved queue management with staleness detection and recovery mechanisms

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
api	Ready	Preview	Jun 16, 2026 3:35pm

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

Three new REST endpoints are added: GET /api/research/albums/{id}/measurements, GET /api/research/tracks/{id}/measurements, and POST /api/research/measurement-jobs. The PR also introduces historical Songstats backfill enqueueing, Stripe payment-method gating, stale queue row reclamation, and reclassifies non-200 Songstats responses in the backfill worker as retryable vs. terminal.

Changes

Research Measurements & Measurement Jobs API

Layer / File(s)	Summary
Stale backfill queue reclamation and error classification lib/supabase/songstats_backfill_queue/updateSongstatsBackfillQueue.ts, app/workflows/backfillTrackStep.ts, lib/research/playcounts/playcountMaintenanceHandler.ts	Adds reclaimStaleSongstatsBackfillRows (resets stale failed/in_progress rows to pending after 1 hour), reclassifies non-200 Songstats responses into no data 404, retryable, and terminal outcomes with matching done/failed queue statuses, and wires reclamation into playcountMaintenanceHandler before the drain workflow.
Measurement job validation, schema, and payment gating lib/research/measurement_jobs/validateCreateMeasurementJobRequest.ts, lib/research/measurement_jobs/ensureSongstatsPaymentMethod.ts	Defines Zod schema (scope union, source, platforms), implements validateCreateMeasurementJobRequest with auth/JSON/schema validation, and adds ensureSongstatsPaymentMethod which returns a 402 Stripe Checkout redirect when no default payment method is set for historical requests.
Measurement job creation: scope resolution and backfill dispatch lib/research/measurement_jobs/resolveScopeSongs.ts, lib/research/measurement_jobs/enqueueHistoricalBackfill.ts, lib/research/measurement_jobs/createMeasurementJob.ts	resolveScopeSongs maps ISRC/album/catalog scope to a deduplicated ISRC list; enqueueHistoricalBackfill queries existing measurements, skips already-backfilled ISRCs, and upserts queue entries in batches of 25; createMeasurementJob routes historical requests to the backfill enqueue and current requests to createSnapshot with normalized result mapping.
POST measurement-jobs handler and route lib/research/measurement_jobs/createMeasurementJobHandler.ts, app/api/research/measurement-jobs/route.ts	Adds createMeasurementJobHandler with validation-first flow, 202 CORS response on success, structured error mapping, and 500 fallback; wires it to the Next.js route with OPTIONS/POST and maxDuration = 60.
Album measurements: validation, fetch, handler, and route lib/research/measurements/validateGetAlbumMeasurementsRequest.ts, lib/research/measurements/getAlbumMeasurements.ts, lib/research/measurements/getAlbumMeasurementsHandler.ts, app/api/research/albums/[id]/measurements/route.ts	Validates auth and research credits; calls getAlbumPlaycounts and remaps rows to a measurements array keyed by platform_displayed_play_count; handler applies validation-first pattern with success/error response helpers; route wires OPTIONS/GET with maxDuration = 60.
Track measurements: ISRC resolution, validation, data fetch, aggregation, and route lib/research/measurements/resolveTrackIsrc.ts, lib/research/measurements/validateGetTrackMeasurementsRequest.ts, lib/research/measurements/getTrackMeasurements.ts, lib/research/measurements/getTrackMeasurementsHandler.ts, app/api/research/tracks/[id]/measurements/route.ts	resolveTrackIsrc accepts ISRC patterns or Spotify track IDs; validator parses aggregate/window/platform/metric query params with defaults; getTrackMeasurements deducts credits only after data exists and returns either a time series or a run_rate aggregation; handler and route follow the same validation-first pattern.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant Route as Next.js Route
    participant Validator
    participant Logic as Business Logic
    participant DB as Supabase / Stripe

    rect rgba(100, 149, 237, 0.5)
        Note over Client,DB: POST /api/research/measurement-jobs (historical)
        Client->>Route: POST with scope + source=historical
        Route->>Validator: validateCreateMeasurementJobRequest
        Validator->>DB: validateAuthContext
        Validator->>DB: ensureSongstatsPaymentMethod → Stripe customer lookup
        alt No payment method
            DB-->>Validator: 402 + Stripe Checkout URL
            Validator-->>Client: 402 checkoutUrl
        end
        Validator-->>Logic: ValidatedCreateMeasurementJobRequest
        Logic->>DB: resolveScopeSongs → ISRC list
        Logic->>DB: enqueueHistoricalBackfill → upsert songstats_backfill_queue
        Logic-->>Client: 202 enqueued + skipped counts
    end

    rect rgba(144, 238, 144, 0.5)
        Note over Client,DB: GET /api/research/tracks/{id}/measurements
        Client->>Route: GET with id + ?aggregate=run_rate&window=90d
        Route->>Validator: validateGetTrackMeasurementsRequest
        Validator->>DB: auth + ensureResearchCredits
        Validator-->>Logic: ValidatedGetTrackMeasurementsRequest
        Logic->>DB: resolveTrackIsrc → ISRC
        Logic->>DB: fetch song_measurements rows
        Logic->>Logic: computePlaycountDeltas → annualized run_rate
        Logic-->>Client: 200 data.aggregate
    end

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related issues

• Chartmetric API suspended (ToS) — migrate /research to Songstats chat#1777: The new route handlers, measurement fetchers, backfill enqueueing, and Songstats payment gating directly implement the research measurement endpoints described in this issue.
• Consolidate /research play-count endpoints into 2 REST resources (measurements + measurement-jobs) chat#1796: The two new REST resources (/measurements and /measurement-jobs), job-creation dispatch, stale-row reclamation, and payment-method gating correspond to the specific shipped work items documented in this consolidation issue.

Possibly related PRs

• recoupable/api#663: Introduced playcountMaintenanceHandler, which this PR extends by adding the reclaimStaleSongstatsBackfillRows pre-step and updating the cron response shape.
• recoupable/api#662: Both PRs modify backfillTrackStep.ts around non-200 Songstats response handling and backfill queue/quota outcome updates.
• recoupable/api#668: Overlaps with this PR's measurement-jobs write endpoint, stale row reclamation, and backfill-drain retry classification changes.

Poem

🎵 Three routes awoke at measurement time,
Albums and tracks now served in their prime.
Historical depths get queued with care,
Stale rows reclaimed from the job-queue snare.
Stripe stands guard at the historical gate—
Pay up or wait, we'll backfill your fate! 🎸

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Solid & Clean Code	⚠️ Warning	Five explicit code review comments remain unaddressed: double credit deduction in album path, missing Zod ID validation, empty platforms array acceptance, premature credit deduction in track path,...	Apply all five review comment fixes: add .min(1) to platforms schema, validate album id with Zod before credit check, remove pre-deduction in album validator, defer credit deduction to success returns in track measurements, reject invali...

✅ Passed checks (2 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch test

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}