Skip to content

Delete the never-started CaseAutomationService, the case_metrics recompute route and its script - #3104

Merged
G-r-ay merged 6 commits into
mainfrom
cleanup/2905-delete-the-never-started-22359ad9
Oct 24, 2026
Merged

G-r-ay merged 6 commits into
mainfrom
cleanup/2905-delete-the-never-started-22359ad9

Conversation

@G-r-ay

@G-r-ay G-r-ay commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #2905

Nothing ever calls automation_service.start(), so the SLA monitor, the hourly metrics loop and the stale-case detector in CaseAutomationService never ran. CaseMetricsService.calculate_case_metrics had three callers: that loop, the beta POST /api/v1/cases/metrics/calculate/{case_id} (also mounted at /api/cases/metrics), and scripts/calculate_case_metrics.py. No client calls the route. This PR deletes all four.

Where this PR departs from the issue, because the code changed after it was filed:

  • case_metrics_service.py held only calculate_case_metrics, so the whole module is gone. The issue said to keep the class. Its only other users were the router singleton metrics_service and the script, both removed.
  • The daily digest the issue names no longer exists in the class.
  • The logger f-string baseline went from 288 to 281, not 311 to 302.
  • Two router comments named CaseMetricsService as the source of the MTTR definitions. I cut those clauses because the class is gone.

Kept: cluster_findings_by_attack_path, _max_severity and _SEVERITY_ORDER in case_automation_service.py, which services/api/routers/vstrike.py:228 and tests/unit/cases/test_open_case_sla.py:7 import. The case_metrics table, its model and CaseMetricsSchema also stay.

Callers checked

  • grep -rn -E "automation_service|CaseAutomationService|case_automation|calculate_case_metrics|metrics/calculate|CaseMetricsService|case_metrics_service" over the repo, excluding node_modules and .git. Before the change, hits were in the files this PR removes or edits, plus the vstrike importer, the vstrike test above, a docstring in tests/integration/test_vstrike_ingest.py:129, the ambient-state allow-list, and schema.d.ts. No .start() call on the singleton exists anywhere.
  • After the change, only the three vstrike references remain.
  • The same names in *.md, *.yml, *.yaml, *.toml, *.sh, *.ts, *.tsx, *.json, Makefiles and Dockerfiles turned up nothing outside the generated schema.
  • metrics_calculate and metrics/calculate in clients/ turned up nothing outside the generated schema.

Blast radius

  • Safe because: Nothing outside the removed code calls the recompute route, the singleton or calculate_case_metrics. The SLA and metrics loops never started, so no runtime behavior changes.
  • Tried to break it: If something else reached this code, the kept vstrike import would fail, a contract, auth or reachability test would pin the route, or the app would fail to build its OpenAPI spec. I imported the kept functions and the vstrike router and built app.openapi() from the real app. The import printed import ok. The spec lists 7 paths under each of /api/v1/cases/metrics and /api/cases/metrics, and none is calculate. The contract, reachability, security, cases and vstrike tests pass (see Verification).
  • Proof: Level 5. Ran the app (uvicorn services.api.main:app) on main and on this branch against the same throwaway Postgres + Redis and drove the case-metrics routes over HTTP. See "Live app run" under Verification.
  • Severity: Low.
  • Risks:
  • Cleared:
    • core/api/v1/case_metrics_router.py derives MTTR and MTTD in SQL and never read case_metrics through this service.
    • The regenerated schema.d.ts diff is 118 deleted lines in 4 hunks. They are the two calculate paths and their two operations, and no TS file references those operation ids.
    • CaseMetricsSchema keeps its user in tests/unit/storage/test_orm_schema_parity.py.
    • pyflakes reports nothing on the two edited Python files.

Also seen

  • CaseNotificationService.notify_sla_warning (core/cases/case_notification_service.py:148) now has no production caller. Only tests/unit/test_notification_metadata.py calls it. The "sla_warning" type comment at line 30 points at it. I left both alone because they are outside this job.
  • In the full combined run, tests/unit/api/test_api_v1_reachable.py (10 errors) and test_api_v1_dual_mount.py::test_both_paths_return_the_same_response (1 failure) failed with SchemaRevisionError: the database has no alembic_version. The cause is the shared local Postgres. Which tests fail depends on the order of earlier tests in the run. Both files pass alone, on this branch and on main (13 passed each).

Verification

Ran

  • pytest tests/unit/_ratchets: before the baseline change, the logger ratchet failed with "Only 281 f-string log calls remain; lower BASELINE ... from 288 to 281". After the change, 260 passed and 6 skipped (helm not installed). The base had no new commits to merge. Files: pytest-ratchets.txt, pytest-before-baseline.txt.
  • pytest tests/unit/_ratchets tests/unit/api tests/unit/cases tests/unit/storage/test_orm_schema_parity.py tests/security tests/integration/test_vstrike_ingest.py tests/**/test_vstrike*.py with the old baseline: 1488 passed, plus the ratchet step and the DB ordering failures above. File: pytest-before-baseline.txt.
  • pytest tests/unit/api/test_api_v1_dual_mount.py tests/unit/api/test_api_v1_reachable.py: 13 passed alone on the branch and on main. Files: pytest-branch-dualmount-reachable.txt, pytest-base-env-failures.txt.
  • pytest tests/unit/cases tests/integration/test_vstrike_ingest.py tests/unit/storage/test_orm_schema_parity.py tests/unit/api/test_api_v1_contract.py: 302 passed. File: pytest-cases-vstrike-contract.txt.
  • I ran a script that imports the kept vstrike functions and builds the real app's OpenAPI spec. File: route-check.txt.
  • python scripts/generate_frontend_types.py (openapi-typescript 7.13.0) produced 118 deletions in schema.d.ts.

Live app run (before/after)

Throwaway pgvector/pgvector:pg16 + redis:7-alpine containers, schema applied with python -m core.upgrade, app started with DEV_MODE=true uvicorn services.api.main:app. One case created through POST /api/v1/cases. Same DB for both runs.

Before (origin/main 69f6f73):

$ curl -X POST /api/v1/cases/metrics/calculate/case-2026-10-24-bc0719e7
{"case_id":"case-2026-10-24-bc0719e7","time_to_detect":null,...,"sla_met":true,...} [HTTP 200]
$ curl -X POST /api/cases/metrics/calculate/case-2026-10-24-bc0719e7
{"case_id":"case-2026-10-24-bc0719e7",...} [HTTP 200]
GET /api/v1/cases/metrics/{mttr,mttd,summary,by-status,by-priority,breached,analyst-performance}: all [HTTP 200]
$ psql: select case_id, time_to_resolve from case_metrics
 case-2026-10-24-bc0719e7 |        (1 row, written by the route)
openapi calculate paths: ['/api/v1/cases/metrics/calculate/{case_id}', '/api/cases/metrics/calculate/{case_id}']

After (this branch, 1bd3ab0):

$ curl /api/health
{"status":"healthy","version":"0.8.0",...}
$ curl -X POST /api/v1/cases/metrics/calculate/case-2026-10-24-bc0719e7
{"detail":"Not Found"} [HTTP 404]
$ curl -X POST /api/cases/metrics/calculate/case-2026-10-24-bc0719e7
{"detail":"Not Found"} [HTTP 404]
GET /api/cases/metrics/{mttr,mttd,summary,by-status,by-priority,breached,analyst-performance}: all [HTTP 200]
$ diff -r get-base get-branch   # 14 GET bodies, /api/v1 + /api mounts, saved from each run
identical
openapi calculate paths: []   /api/v1/cases/metrics paths: 7

Both startup logs have the same single ERROR line (Bifrost unreachable, no LLM gateway in the throwaway setup) and no tracebacks. The 7 read routes return byte-identical bodies with a case_metrics row present, so they never read that table.

Did not run

  • npx tsc --noEmit. This clone has no node_modules. The schema diff only deletes two paths and two operations that no TS file references, and CI runs tsc.

🤖 Generated with Claude Code

…mpute route and its script

Nothing called automation_service.start(), so its SLA, metrics and
stale-case loops never ran. CaseMetricsService.calculate_case_metrics was
reached only from that loop, the beta POST /cases/metrics/calculate/{case_id}
that no client calls, and scripts/calculate_case_metrics.py. All of them go,
with the now-empty case_metrics_service module. The vstrike clustering code in
case_automation_service.py stays. Lowers the f-string logger ratchet to 281
and regenerates schema.d.ts.

Closes #2905

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
G-r-ay pushed a commit that referenced this pull request Oct 24, 2026
@G-r-ay G-r-ay added the cleanup Opened by the cleanup factory label Oct 24, 2026
cleanup-factory and others added 2 commits October 24, 2026 16:24
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@G-r-ay G-r-ay added the cleanup:blast-high Blast radius: how bad it is if this PR is wrong label Oct 24, 2026
…arted-22359ad9' into cleanup/2905-delete-the-never-started-22359ad9
@G-r-ay

G-r-ay commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

🧹 Asked for proof in the running app, level 5, which a high or critical blast radius needs (1 of 3): a high blast radius proven only to level 4.

deeptempo-factory and others added 2 commits October 24, 2026 22:33
… to 278

Main lowered BASELINE to 285. With this PR's deletions the tree has 278
f-string logger calls, so the ratchet is lowered to match.

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
…te the removed beta route

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
@G-r-ay

G-r-ay commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

🏭 Advisory review (claude-haiku-5-5 + opus advisor) — merge 13db502e12d5. Advice only: nothing was merged or closed on it.

Nothing left on #3104: head 13db502e12d5 has 23 of 23 checks passing and the branch is current with main. Loop stopped.

@G-r-ay

G-r-ay commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

🧹 Proof at 13db502e12d5. Counted by the cleanup factory at this commit; a push or a settled CI run replaces it. Each count says how to reproduce it.

Job: #2905, a remove job in feature case-metrics, from survey 2026-10-22-6035439f-y8bt.

Check Result
Blast radius high: if wrong, several features, or clients outside this repo, break
Size pass (+11 −547)
Callers of what it deletes pass
Style lint pass
CI pass
Ready for review yes: taken out of draft

Blast radius: high. Touches 9 file(s) in the contract, api, ui, tests, docs layer(s), across feature(s) cases, case-metrics, api-platform, vstrike. Why:

  • high: touches a published API contract or generated client: clients/web/src/services/generated/schema.d.ts
  • high: spans 4 features: cases, case-metrics, api-platform, vstrike
  • medium: touches API handlers: core/api/v1/README.md, core/api/v1/case_metrics_router.py
  • medium: touches what users see: clients/web/src/services/generated/schema.d.ts

The author says it is safe because: Nothing outside the removed code calls the recompute route, the singleton or calculate_case_metrics. The SLA and metrics loops never started, so no runtime behavior changes. (proof level 5 of 5, a claim to check, not a count).

How each count was made, and how to rerun it

Size. +11 −547 over the whole diff, as GitHub counts it. A remove job must delete more than it adds. Reproduce with gh pr view 3104 -R Vigil-SOC/vigil --json additions,deletions.

Callers. What the PR deletes and does not define again, searched for across the repo at 13db502e12d5. A symbol is matched as a whole word, and a route by the fixed part of its path.

  • calculate_case_metrics (symbol, from core/cases/case_metrics_service.py): 0 references
  • CaseAutomationService (symbol, from core/cases/case_automation_service.py): 0 references
  • sla_monitor_task (symbol, from core/cases/case_automation_service.py): 0 references
  • _check_sla_deadlines (symbol, from core/cases/case_automation_service.py): 0 references
  • metrics_update_task (symbol, from core/cases/case_automation_service.py): 0 references
  • _update_case_metrics (symbol, from core/cases/case_automation_service.py): 0 references
  • stale_case_detector_task (symbol, from core/cases/case_automation_service.py): 0 references
  • _detect_stale_cases (symbol, from core/cases/case_automation_service.py): 0 references
  • CaseMetricsService (symbol, from core/cases/case_metrics_service.py): 0 references
git fetch origin pull/3104/head
git grep -n -I -F -w -e 'calculate_case_metrics' 13db502e12d5
git grep -n -I -F -w -e 'CaseAutomationService' 13db502e12d5
git grep -n -I -F -w -e 'sla_monitor_task' 13db502e12d5
git grep -n -I -F -w -e '_check_sla_deadlines' 13db502e12d5
git grep -n -I -F -w -e 'metrics_update_task' 13db502e12d5
git grep -n -I -F -w -e '_update_case_metrics' 13db502e12d5
git grep -n -I -F -w -e 'stale_case_detector_task' 13db502e12d5
git grep -n -I -F -w -e '_detect_stale_cases' 13db502e12d5
git grep -n -I -F -w -e 'CaseMetricsService' 13db502e12d5

Style lint. No findings in the prose the PR adds.

CI at 13db502e12d5. 23 of 23 pass. Reproduce with gh pr checks 3104 -R Vigil-SOC/vigil.

Evidence. import-check.txt · pytest-base-env-failures.txt · pytest-before-baseline.txt · pytest-branch-dualmount-reachable.txt · pytest-cases-vstrike-contract.txt · pytest-ratchets.txt · route-check.txt

Machine-readable
{
  "sha": "13db502e12d566566acb151c27f35f590bd497a6",
  "pr": 3104,
  "job": {
    "issue": 2905,
    "kind": "remove",
    "feature": "case-metrics",
    "survey": "2026-10-22-6035439f-y8bt"
  },
  "blast": {
    "severity": "high",
    "reasons": [
      {
        "severity": "high",
        "what": "touches a published API contract or generated client: clients/web/src/services/generated/schema.d.ts"
      },
      {
        "severity": "high",
        "what": "spans 4 features: cases, case-metrics, api-platform, vstrike"
      },
      {
        "severity": "medium",
        "what": "touches API handlers: core/api/v1/README.md, core/api/v1/case_metrics_router.py"
      },
      {
        "severity": "medium",
        "what": "touches what users see: clients/web/src/services/generated/schema.d.ts"
      }
    ],
    "features": [
      "cases",
      "case-metrics",
      "api-platform",
      "vstrike"
    ],
    "layers": [
      "contract",
      "api",
      "ui",
      "tests",
      "docs"
    ],
    "routes": [],
    "safeBecause": "Nothing outside the removed code calls the recompute route, the singleton or `calculate_case_metrics`. The SLA and metrics loops never started, so no runtime behavior changes.",
    "proofLevel": 5
  },
  "size": {
    "additions": 11,
    "deletions": 547
  },
  "verdicts": {
    "size": "pass",
    "callers": "pass",
    "lint": "pass",
    "ci": "pass"
  },
  "callers": [
    {
      "name": "calculate_case_metrics",
      "kind": "symbol",
      "from": "core/cases/case_metrics_service.py",
      "refs": []
    },
    {
      "name": "CaseAutomationService",
      "kind": "symbol",
      "from": "core/cases/case_automation_service.py",
      "refs": []
    },
    {
      "name": "sla_monitor_task",
      "kind": "symbol",
      "from": "core/cases/case_automation_service.py",
      "refs": []
    },
    {
      "name": "_check_sla_deadlines",
      "kind": "symbol",
      "from": "core/cases/case_automation_service.py",
      "refs": []
    },
    {
      "name": "metrics_update_task",
      "kind": "symbol",
      "from": "core/cases/case_automation_service.py",
      "refs": []
    },
    {
      "name": "_update_case_metrics",
      "kind": "symbol",
      "from": "core/cases/case_automation_service.py",
      "refs": []
    },
    {
      "name": "stale_case_detector_task",
      "kind": "symbol",
      "from": "core/cases/case_automation_service.py",
      "refs": []
    },
    {
      "name": "_detect_stale_cases",
      "kind": "symbol",
      "from": "core/cases/case_automation_service.py",
      "refs": []
    },
    {
      "name": "CaseMetricsService",
      "kind": "symbol",
      "from": "core/cases/case_metrics_service.py",
      "refs": []
    }
  ],
  "lint": 0,
  "checks": [
    {
      "name": "CodeQL",
      "bucket": "pass"
    },
    {
      "name": "Build Frontend",
      "bucket": "pass"
    },
    {
      "name": "Backend Image",
      "bucket": "pass"
    },
    {
      "name": "Unit Tests - Backend",
      "bucket": "pass"
    },
    {
      "name": "Helm Install Schema",
      "bucket": "pass"
    },
    {
      "name": "Setup Path (macos-latest)",
      "bucket": "pass"
    },
    {
      "name": "Setup Path (ubuntu-latest)",
      "bucket": "pass"
    },
    {
      "name": "Unit Tests - Agent Layer",
      "bucket": "pass"
    },
    {
      "name": "Lint Backend",
      "bucket": "pass"
    },
    {
      "name": "Unit Tests - Frontend",
      "bucket": "pass"
    },
    {
      "name": "Daemon Image",
      "bucket": "pass"
    },
    {
      "name": "Generated API Types",
      "bucket": "pass"
    },
    {
      "name": "MCP Runner Image",
      "bucket": "pass"
    },
    {
      "name": "Agent Image",
      "bucket": "pass"
    },
    {
      "name": "Lint Dockerfiles",
      "bucket": "pass"
    },
    {
      "name": "Lint Frontend",
      "bucket": "pass"
    },
    {
      "name": "Integration Tests",
      "bucket": "pass"
    },
    {
      "name": "Unit Tests - Backend (DB-backed)",
      "bucket": "pass"
    },
    {
      "name": "Security Scan - Python",
      "bucket": "pass"
    },
    {
      "name": "Security Scan - NPM",
      "bucket": "pass"
    },
    {
      "name": "Analyze (javascript-typescript)",
      "bucket": "pass"
    },
    {
      "name": "Analyze (python)",
      "bucket": "pass"
    },
    {
      "name": "Analyze (actions)",
      "bucket": "pass"
    }
  ],
  "evidence": [
    {
      "name": "import-check.txt",
      "url": "https://github.com/Vigil-SOC/vigil/blob/deslop-evidence/pr-3104/import-check.txt"
    },
    {
      "name": "pytest-base-env-failures.txt",
      "url": "https://github.com/Vigil-SOC/vigil/blob/deslop-evidence/pr-3104/pytest-base-env-failures.txt"
    },
    {
      "name": "pytest-before-baseline.txt",
      "url": "https://github.com/Vigil-SOC/vigil/blob/deslop-evidence/pr-3104/pytest-before-baseline.txt"
    },
    {
      "name": "pytest-branch-dualmount-reachable.txt",
      "url": "https://github.com/Vigil-SOC/vigil/blob/deslop-evidence/pr-3104/pytest-branch-dualmount-reachable.txt"
    },
    {
      "name": "pytest-cases-vstrike-contract.txt",
      "url": "https://github.com/Vigil-SOC/vigil/blob/deslop-evidence/pr-3104/pytest-cases-vstrike-contract.txt"
    },
    {
      "name": "pytest-ratchets.txt",
      "url": "https://github.com/Vigil-SOC/vigil/blob/deslop-evidence/pr-3104/pytest-ratchets.txt"
    },
    {
      "name": "route-check.txt",
      "url": "https://github.com/Vigil-SOC/vigil/blob/deslop-evidence/pr-3104/route-check.txt"
    }
  ]
}

@G-r-ay
G-r-ay merged commit 89d9492 into main Oct 24, 2026
23 checks passed
@G-r-ay
G-r-ay deleted the cleanup/2905-delete-the-never-started-22359ad9 branch October 24, 2026 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cleanup:blast-high Blast radius: how bad it is if this PR is wrong cleanup Opened by the cleanup factory

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Delete the never-started CaseAutomationService loop, the case_metrics recompute route and its backfill script

1 participant