feat: detect services that are running but no longer earning (CashPilot-kbs) - #149
Merged
Conversation
…ot-kbs) Catalog rot is the most common way a user silently stops earning, and the liveness check cannot see the worst case: a service that is UP but no longer paying. The container runs, the collector authenticates, the balance never moves - and every other view of the system looks healthy, so nothing surfaces it. The user finds out by eventually noticing. get_flatlined_services reports a balance that has not moved across 7 recorded days, raised as a distinct "flatline" alert kind and exposed at GET /api/earnings/flatlines. The restraint matters as much as the detection, since the bead is explicit that a report which cries wolf is a report nobody reads: - too little history is not a flatline; a new deployment has not had time to earn anything yet - an always-zero balance is not reported: that is a service that never paid, usually a setup problem the user already knows about, not one that stopped - the window counts DISTINCT RECORDED DAYS, so a collection outage (which records nothing) cannot masquerade as a flat balance record_alert's existing per-kind cooldown keeps this to one notification per service rather than one per collection cycle, and the check is wrapped so a diagnostic can never break the collection run itself. Scope: this is the flatline half. Gone-vs-down streak tracking lives in the weekly liveness script and needs state that survives between CI runs, so it is not in this change.
Owner
Author
|
@coderabbitai review |
📝 WalkthroughWalkthroughChangesFlatline earnings detection
Project release documentation
Estimated code review effort: 3 (Moderate) | ~25 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
✅ Action performedReview finished.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #149 +/- ##
==========================================
+ Coverage 93.91% 94.23% +0.32%
==========================================
Files 35 36 +1
Lines 4123 4475 +352
==========================================
+ Hits 3872 4217 +345
- Misses 251 258 +7
🚀 New features to boost your workflow:
|
codecov flagged the patch: the detection query was tested but the part that actually reaches the user - the alert and the endpoint - was not. Extracted the inline block from the collection loop into _flatline_check so it can be exercised directly, and covered the behaviour that matters: the endpoint returns the flatlined services; a newly detected flatline notifies and the message says how many days; a service already in cooldown does NOT notify again (one per service, not one per collection cycle); and a failing check never raises, because a diagnostic must not be able to take down the collection run it is diagnosing.
# Conflicts: # CHANGELOG.md # app/main.py
# Conflicts: # CHANGELOG.md # app/main.py
# Conflicts: # CHANGELOG.md
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/database.py`:
- Around line 1543-1552: Update the flatline detection flow around the earnings
aggregation query and its direct GET /api/earnings/flatlines caller to filter
results by platforms confirmed active and successfully collected in the current
run, excluding failed or removed services. Propagate that current collection
state into the detection/API path (or reuse persisted equivalent state) rather
than changing only _flatline_check, and add coverage for a service with seven
unchanged historical readings whose current collection fails.
In `@app/main.py`:
- Around line 264-269: The flatline collection must clear the stored cooldown
when a service’s balance has increased. In app/main.py lines 264-269, update the
flow around database.get_flatlined_services and record_alert to detect recovery
from a changed balance and call clear_alerts("flatline", flat["platform"]) only
for that recovery; in tests/test_earnings_flatline.py lines 173-194, add a
regression test covering flatline, balance increase, then flatline again before
cooldown expiry.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 3df2267f-7425-4837-b778-cab039594a44
📒 Files selected for processing (5)
CHANGELOG.mdapp/database.pyapp/main.pydocs/GOAL.mdtests/test_earnings_flatline.py
Owner
Author
|
Closing the review threads on this PR: every finding is fixed on They stayed open only because the fixes landed in a later PR — CodeRabbit auto-resolves a thread when the fix is pushed to the same branch, and these were not. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
Catalog rot is the most common way a user silently stops earning, and the weekly liveness check cannot see the worst case: a service that is up but no longer paying.
The container runs. The collector authenticates. The balance never moves. Every other view of the system looks healthy, so nothing surfaces it — the user finds out by eventually noticing they stopped being paid.
The detection
get_flatlined_services()reports a balance that has not moved across 7 recorded days, raised as a distinctflatlinealert kind and exposed atGET /api/earnings/flatlines.The restraint matters as much
The bead is explicit that a report which cries wolf is a report nobody reads, so most of the logic is about what must not be reported:
record_alert's existing per-kind cooldown keeps this to one notification per service rather than one per collection cycle, and the whole check is wrapped so a diagnostic can never break the collection run itself.Scope
This is the flatline half of the bead. The gone-vs-down streak tracking lives in the weekly liveness script and needs state that survives between CI runs, so it is deliberately not in this change.
Verification
ruff check . && ruff format --check .— cleanpytest --cov=app --cov-fail-under=90— 1342 passed, coverage 93.83%Summary by CodeRabbit
New Features
Documentation