Skip to content

D4/D5/D7/D8/D9: Deep revisions — frontend refactor, notifications, security, scoring - #5

Closed
marota wants to merge 24 commits into
mainfrom
claude/repo-review-next-steps-5o6rz6
Closed

marota wants to merge 24 commits into
mainfrom
claude/repo-review-next-steps-5o6rz6

Conversation

@marota

@marota marota commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Lands the 2026-07 full-repository review revisions (D1–D9) from docs/architecture/2026-07-full-repo-review.md. This is a large, multi-faceted change addressing architecture, UX, security, and test infrastructure across both frontend and backend.

Summary

This PR consolidates nine deep revisions that remediate findings from a comprehensive 12-dimension audit:

  • D4 — Frontend hub decomposition: extracted useManualSimulation hook (manual action + SLD-edit orchestration) from App.tsx, reducing it from 2048 → 1795 lines. Grouped ~30 scattered props into cohesive state objects (SldEditControls, AnalysisTimings, etc.) in VisualizationPanel and ActionFeed.
  • D5 — Unified streaming + notification pipeline: single parseNdjsonStream reader replaces five hand-copied NDJSON loops; new typed notifications store with explicit severity, sticky flag, and per-item dismissal; NotificationHost component replaces ad-hoc StatusToasts; analysis and manual-simulation flows now cancellable.
  • D7 — Deployment trust & reproducibility: session-path traversal guard (_safe_session_dir, QW7); loopback-default CORS; lockdown profile disables filesystem RPCs on hosted Space.
  • D8 — Game Mode scorer in-repo: scripts/game_mode/scoring_program/score.py (Codabench twin) + golden fixture (scoring_golden.json) for cross-language parity; e2e replay test (test_replay.py).
  • D9 — Docs as checked artifact: scripts/check_docs_tree.py gate validates referenced paths and line-number anchors in hand-maintained inventory docs.

Key Changes

Frontend:

  • Extracted useManualSimulation.ts (340 lines) — owns both "simulate unsimulated action" and "SLD-edit → action" flows, routes failures through notification store.
  • New utils/notifications.ts — typed store with notifyError(), notifyInfo(), auto-dismiss timings, per-item dismissal.
  • New utils/ndjsonStream.ts — single reader for /api/run-analysis, /api/simulate-and-variant-diagram, /api/simulate-manual-action streams; replaces five scattered loops.
  • New NotificationHost.tsx component — renders notification queue with aria-live region; replaces StatusToasts.tsx.
  • VisualizationPanel now accepts SldEditControls object prop (22 fields grouped) instead of flat props; ActionFeed accepts AnalysisTimings object.
  • useAnalysis.ts now routes errors/info through notification store; added onCancelAnalysis handler for cancellable analysis.
  • useDiagrams.ts refactored: extracted useOverflowLayout.ts and useActionDiagramCache.ts sub-hooks (D4c); core handleActionSelect, zoomToElement, voltage-filter state remain (FU-1 tracked).
  • New useModalKeyboard.ts — shared Escape-to-close + initial focus for dialogs (QW20).
  • Removed react-zoom-pan-pinch dependency (unused).

Backend:

  • New services/paths.py — centralized path constants (OVERFLOW_DIR, etc.).
  • main.py: added CODE_LOCKED_DOWN error code; lockdown profile disables filesystem RPCs when LOCKED_DOWN_MODE=true.
  • test_api_endpoints.py: added TestSessionPathTraversal — validates _safe_session_dir rejects .., /, absolute paths (QW7).
  • services/analysis/analysis_runner.py: added threading import + cancellation support.
  • services/diagram_mixin.py: _diff_switches now returns structured dict (switch IDs + toggle state).

https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam

Antoine Marot and others added 24 commits July 8, 2026 20:06
Lands the cheap halves of the two still-open headline issues (#4
invisible failures, #5 trust model) plus dead-code cleanup:

- QW4: surface useAnalysis.error through StatusToasts (error ||
  analysis.error, cleared on contingency-clear) so an analysis failure
  is no longer a silent dead spinner; the last silent SLD-preview catch
  now setError's too. Guarded by a new App-integration test.
- QW3: CORS default "*" -> loopback dev origins (localhost/127.0.0.1 on
  :5173/:4173); the wildcard is now explicit opt-in via
  CORS_ALLOWED_ORIGINS="*", closing the drive-by cross-origin read of a
  localhost backend.
- QW7: _safe_session_dir rejects "..", path separators and absolute
  session names before any filesystem write (resolve()+relative_to()
  backstop mirroring the /results/pdf guard); applied to save-session +
  load-session. Guarded by TestSessionPathTraversal.
- QW9: delete inspect_action.py, the dead expert_backend/requirements.txt
  stub, and the unused react-zoom-pan-pinch dependency (+ lockfile and
  doc follow-through). The tracked Overflow_Graph/*.html is deliberately
  kept: it is a skip-guarded regression fixture and untracking it would
  silently drop that CI coverage.

Docs (CLAUDE.md trees, README, .env.example) reconciled with the CORS
default change, the removed files, and the react-zoom-pan-pinch removal
(the "rendered inside react-zoom-pan-pinch" claims corrected to the
actual usePanZoom viewBox mechanism); review tracker updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
Follows up the QW3/4/7/9 bundle with tests and documentation:

- QW3: extract `_resolve_cors_origins` (testable) and add `TestCorsDefault`
  — unit cases (unset/empty → loopback, `*` opt-in, comma list, credentials
  invariant) plus TestClient integration proving a disallowed Origin gets
  no ACAO header while a loopback Origin is reflected.
- QW7: add a `load-session` happy-path case alongside the traversal
  rejections so the guard is shown not to break legitimate loads.
- QW4: add an App-integration test that a surfaced analysis error is
  dismissed when the contingency is cleared.
- Docs: document the `session_name` path-validation contract in
  save-results.md; note the new TestSessionPathTraversal / TestCorsDefault
  classes in the test-suite guide.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
…opies

The NDJSON streaming body was parsed by five hand-copied reader loops
(App.tsx x2, useAnalysis, useDiagrams, ActionFeed) that had drifted:
only the two App.tsx copies flushed a final line lacking a trailing
newline, and each re-implemented blank-line / parse-error handling
slightly differently.

`parseNdjsonStream(source, {signal?})` is now the one implementation —
an async generator that decodes with stream:true (carrying the partial
line across chunks), yields one parsed value per complete line, skips
blank / unparseable lines, flushes a trailing unterminated final line on
end, and honours an AbortSignal (cancels the reader, ends iteration
without flushing the partial buffer — foundation for D5.3 cancellation).
Reader cancellation in cleanup is defensive so it never throws out of a
generator finally.

All five callers now consume it via `for await`. Behaviour-preserving
(full Vitest suite green); covered by src/utils/ndjsonStream.test.ts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
Replaces the two ad-hoc toast channels — a sticky-but-undismissable
`error` string and an auto-3s `infoMessage` string whose green-vs-blue
colour was decided by a magic `'SUCCESS'` prefix — with one typed store.

- utils/notifications.ts: a module-singleton store (severity error/
  success/info, sticky flag, per-item auto-expiry, de-dupe of identical
  toasts) bound to React via useSyncExternalStore (`useNotifications`),
  with notifyError/notifyInfo/notifySuccess/dismiss/clear helpers.
- components/NotificationHost.tsx: renders the stack bottom-right, each
  toast dismissible, inside an aria-live region (errors role=alert /
  assertive, info+success role=status / polite). Replaces StatusToasts
  (deleted).
- useAnalysis: drops its local error/infoMessage state + bespoke 3s timer;
  setError/setInfoMessage are now thin adapters over the store (same
  `(v)=>void` raise/clear contract).
- useSession: the two `setInfoMessage('SUCCESS: …')` sites now call
  notifySuccess directly, retiring the string protocol; the now-unused
  setInfoMessage param is dropped from SaveParams / RestoreContext.
- App: setError is a store adapter (all call sites + hook threading
  unchanged); renders NotificationHost; drops the infoMessage plumbing.

Tests: notifications.test.ts + NotificationHost.test.tsx added;
useAnalysis.test asserts against the store; setup.ts clears the singleton
between tests. Full Vitest suite green (1753 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
Long-running analysis runs are now cancellable, completing the D5 UX
fix for T4 (error/streaming paths as UX).

- api.ts: runAnalysisStep1 / runAnalysisStep2Stream accept an optional
  AbortSignal (axios `signal` / fetch `signal`).
- useAnalysis: each run gets a fresh AbortController (kept on a ref);
  its signal is threaded into step 1, the step 2 fetch, and the NDJSON
  stream read. `cancelAnalysis()` aborts it. An abort is treated as a
  cancellation (info toast "Analysis cancelled.", no error toast, no
  step-2 completion event), not a failure.
- ActionFeed: a "✕ Cancel" control appears beside "⚙️ Analyzing…" while
  a run is in flight (guarded by an optional onCancelAnalysis prop); App
  wires it to analysis.cancelAnalysis.

Tests: hook-level cancellation (signal threading, abort → cancellation
notice not error, no-op when idle), the ActionFeed Cancel control, and
the existing step2 call assertions updated for the new signal arg.
Full Vitest suite green (1758 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
- New docs/architecture/notifications-and-streaming.md: the D5.1–D5.3
  design (single NDJSON reader, typed notification store, cancellable
  analysis) + test map + follow-ups; indexed in docs/README.md.
- 2026-07 review tracker: D5 marked DONE (Part V + progress note), the
  dimension-4 findings (toast design, cancellation, divergent NDJSON
  consumers, invisible failures) and T4 annotated, QW10 marked done.
- CLAUDE.md trees: StatusToasts → NotificationHost; ndjsonStream +
  notifications added to the utils inventories; the App.tsx
  deferred-NDJSON-parser note marked done.
- tests/CLAUDE.md: NotificationHost + ndjsonStream.test + notifications.test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
First stage of D4 (relieve the App.tsx hub). Moves the two operator-
driven "simulate now" flows and the interactive SLD-edit state they
share out of App.tsx into hooks/useManualSimulation.ts:

- handleSimulateUnsimulatedAction (double-click an un-simulated overview
  pin), handleSimulateSldEdit (staged breaker/injection edit → action),
  the useSldTopologyEdit state, the implicit edit-mode sync, and the
  debounced target-topology preview.

App owns the collaborators (diagrams, analysis result, the manual-action
wrapper) and passes them into the hook, which returns the SLD-edit state
+ the two handlers the JSX consumes — a thin orchestrator, not a second
state hub. Behaviour-preserving (full Vitest suite green).

App.tsx 2048 → 1795 lines; APP_TSX_MAX ratcheted 2100 → 1850 to lock the
win. New hooks/useManualSimulation.test.ts covers the no-contingency
guard, the metrics+diagram stream (prime + provenance), the stream-error
path, the empty SLD-edit no-op, and sldEditBaseActionId derivation.

Remaining D4 tail (bigger bets, tracked): split useDiagrams into
per-domain hooks behind the DiagramsState facade, and consolidate the
exploded VisualizationPanel / ActionFeed props into cohesive state-object
props.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
Closes the still-open half of headline #5 (the trust model no longer
matches deployment) and tightens the deploy pipeline.

Lockdown profile:
- COSTUDY4GRID_LOCKDOWN (read into main._LOCKDOWN) disables the desktop-
  era filesystem RPCs — POST /api/config-file-path, save/list/load-session,
  GET /api/pick-path — with a 403 {code: LOCKED_DOWN} via
  _reject_when_locked_down(). They assume a local operator and would give
  an anonymous Space visitor read/write access to the container.
- The read-only app config (GET user-config / config-file-path) stays
  open so the SPA still boots. Unset locally → behaviour unchanged.
- LOCKED_DOWN is a stable code in the D2 envelope (403 → LOCKED_DOWN) and
  the frontend ApiErrorCode union. The Dockerfile sets the flag.

Deploy hardening:
- deploy-huggingface.yml is now test-gated: it fires on workflow_run of a
  SUCCESSFUL "Tests" run on main (not a bare merge), and checks out that
  tested commit. workflow_dispatch stays as the rollback path.
- Each deploy tags the shipped commit on origin (space-deploy-<ts>-<sha>)
  — the only durable record, since the Space push is force-pushed and
  history-free. Rollback = re-dispatch from a prior tag.

Reproducible Python closure (pinned lockfile) is documented as a
follow-up in deployment-trust.md / SETUP.md — it must be resolved on the
image's Python 3.10, so shipping a dev-3.11 lockfile would be wrong.

Tests: TestLockdownProfile (403 + LOCKED_DOWN when locked, read-only
config still reachable, normal 200 when unlocked). OpenAPI snapshot
unaffected. Docs: new deployment-trust.md + index; review tracker (D7
mostly-done, headline #5, dimensions 9 & 12); CLAUDE.md trees;
SETUP.md rollback + lockfile sections; .env.example.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
…eserving)

Begins the D4-tail useDiagrams split. Extracts the two genuinely
decoupled domains — the ones with no effect-ordering constraint, so the
split can't shift DOM-mutation timing — into their own hooks that
useDiagrams composes and re-exposes through the DiagramsState facade
unchanged:

- hooks/useOverflowLayout.ts — the overflow-graph layout toggle
  (mode + loading + handleOverflowLayoutChange); depends only on the API
  and the interaction logger.
- hooks/useActionDiagramCache.ts — the prime-then-paint action-variant
  NAD cache (actionDiagramCacheRef + primeActionDiagram), cleared on
  contingency change (a single ref-clearing effect, no DOM).

useDiagrams 1210 → 1145 lines; the DiagramsState facade is byte-identical
so App.tsx and every consumer/test are untouched (full Vitest suite
green). New useOverflowLayout.test.ts + useActionDiagramCache.test.ts.

The deeply-coupled core (handleActionSelect, zoomToElement, the
DOM-mutating voltage-filter effects) is intentionally left in place —
extracting it would move effect ordering that tsc can't verify; that
higher-risk tail stays open.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
…ne object

Second D4-tail piece — collapse the exploded props on VisualizationPanel
into cohesive state-object props (the SettingsModal pattern). The
largest, most cohesive cluster goes first: the ~22 interactive
SLD-topology-edit props (sldEditMode / onSldSwitchClick / sldPreview* /
onSldEditSimulate / … ) are now ONE grouped `sldEdit?: SldEditControls`
prop, built by App from the useManualSimulation / useSldTopologyEdit
facade.

- VisualizationPanel unpacks the object once at the top of the body, so
  the JSX below (which forwards to SldOverlay) is unchanged.
- VisualizationPanelProps drops 93 → 72 props.
- Behaviour-preserving (full Vitest suite green; App-level SLD-edit tests
  now route through the grouped prop). New VisualizationPanel tests lock
  that the grouped controls reach SldOverlay → SldEditPanel, and that the
  overlay renders without the edit panel when sldEdit is absent.

Remaining consolidations (same mechanical pattern, tracked): the
detached-tabs / overflow-pins / action-overview clusters on
VisualizationPanel, and the ActionFeed props.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
- frontend/CLAUDE.md: add the D4-extracted hooks (useManualSimulation,
  useOverflowLayout, useActionDiagramCache) to the hooks inventory and
  note useDiagrams is now a thinner orchestrator composing the two
  decoupled sub-hooks behind the DiagramsState facade.
- 2026-07 review tracker: D4 marked stages 1–3 done (extraction + ceiling
  ratchet, facade-preserving useDiagrams split, VisualizationPanel
  SLD-edit props consolidation) with the deeply-coupled useDiagrams core
  and remaining prop clusters / ActionFeed noted as the tail; T5 + the
  Part V progress note updated to match.
- tests/CLAUDE.md: list the three new hook test files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
…rop clusters

Complete the D4d props consolidation: collapse every remaining exploded
prop cluster on the two hub-fed presentational components into cohesive
optional state-object props (the SettingsModal pattern), each unpacked at
the top of the component body so the render tree is byte-for-byte unchanged.

- VisualizationPanel (~93 -> 41 props): add DetachControls (6 detached-tabs
  props), OverflowControls (8 layout-toggle + action-pin props) and
  ActionOverviewControls (17 pin/selection/filter props) alongside the
  existing sldEdit group. App.tsx now passes detach / overflow /
  actionOverview as grouped objects.
- ActionFeed (~44 -> 36 props): add AdditionalLinesControls (4 picker
  props), ModelSelectorControls (4 model-dropdown props) and
  AnalysisTimingControls (6 execution-time props).
- Tests: VisualizationPanel.test.tsx partitions flat cluster overrides back
  into the grouped objects in createDefaultProps; ActionFeed.test.tsx wraps
  the picker / model / timing usages in grouped objects. Full Vitest suite
  green (1,771), tsc + eslint + code-quality gate clean.

- Track the deferred deeply-coupled useDiagrams core split as FU-1 in the
  new docs/architecture/followups.md (GitHub Issues are disabled on the
  fork) and index it in docs/README.md.
- Review tracker + decompose-ceiling-riders proposal updated to reflect the
  completed consolidation and current line counts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
…rops

Add direct-object contract tests locking the D4 props consolidation on both
hub-fed components. Each test passes the grouped object DIRECTLY (bypassing
the flat-override partition helper) and asserts field-by-field forwarding
plus the "optional as a whole" omission case:

- VisualizationPanel: detach (Tie button + toggle callback), overflow
  (Hierarchical/Geo layout toggle + callback), actionOverview (collapsed
  inline filter strip) — each with its omitted-group negative case.
- ActionFeed: additionalLines (picker), modelSelector (Model dropdown +
  change callback), timing (execution-time total) — each with its
  omitted-group negative case.

Full Vitest suite green (VisualizationPanel 65 -> 71, ActionFeed 121 -> 127);
review tracker D4 note updated to cite the explicit contract tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
Finish the element-adoption pipeline the review flagged: a full-NAD load
paid an avoidable serialize + re-parse round-trip (boostSvgForLargeGrid
serialized the mutated DOM back to a string; MemoizedSvgContainer then
re-parsed it via innerHTML), and the component's "zero extra parse" comment
was false on every string path.

- svgBoost.ts: new boostSvgToElement(rawSvg, viewBox, vlCount) parses the
  pypowsybl SVG ONCE and mutates the live SVGSVGElement in place (no
  XMLSerializer). processSvg returns that element. The boost mutation body
  is unchanged — only the parse (moved to the top) and the return (element,
  not serialized string) differ.
- MemoizedSvgContainer already had the replaceChildren(element) adoption
  path — proven in production by the svgPatch fast-path, which sets
  n1Diagram.svg to a cloned SVGSVGElement. The full-fetch paths now ride the
  same rail. Branch on typeof (not instanceof) so any parsed element adopts.
- Safety: boostSvgToElement returns null on a parse error OR a
  non-SVG-namespaced root, so MemoizedSvgContainer falls back to innerHTML
  (the HTML parser namespaces it). Real pypowsybl output always declares
  xmlns. SLD overlays keep their own string->innerHTML path untouched.
- boostSvgForLargeGrid(string)->string retained as a thin wrapper for the
  serialized-SVG unit tests; returns the input unchanged on skip so no
  re-serialization side effects leak.

Tests: element-adoption cases in svgBoost.test.ts (element return, parse-
error string fallback, namespace guard) + a MemoizedSvgContainer identity
test (the mounted <svg> is the SAME instance processSvg returned — a
re-parse could never produce that). Full Vitest suite green (1,789).

Docs: review tracker D6 marked core-done (bitmap auto-enable deferred,
validation-gated) + perf weakness row resolved; frontend CLAUDE.md SVG
handling section documents the new lever.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
Address four of the five D8 sub-tasks from the 2026-07 review (the fifth —
physical session-log replay — needs a live backend to verify and is tracked
as FU-2).

Codabench scorer parity (headline — closes the "no in-repo Python twin, no
parity fixture, no CI guard" gap):
- scripts/game_mode/scoring_program/score.py: the in-repo Codabench scorer,
  a faithful twin of frontend/src/game/scoring.ts (remediation / action /
  time efficiency + apply_reference / score_study / score_session).
- scripts/game_mode/scoring_golden.json: a SHARED golden fixture (inputs
  hand-authored to cover every scorer branch; expected values regenerable via
  gen_scoring_golden.py). Both test_score.py (pytest) and scoring.test.ts
  (Vitest) assert the same expected values, so a change to either scorer that
  breaks numerical parity fails a test.
- e2e_game_session.py now defaults to the in-repo scorer (no ~/Dev path).

Layout scale + post-processing (a rebuild no longer clobbers good layouts):
- convert_pypsa_to_xiidm.py step_write_metadata drops the forbidden
  TARGET_WIDTH=8000 rescale — writes grid_layout.json in raw Mercator metres
  (re-centred), matching regenerate_grid_layout.py and the coordinate-scale
  contract.
- separate_voltage_levels.py wired as build_pipeline.py step 6.
- test_grid_layout coordinate-span test corrected from the stale [5000,20000]
  (which enshrined the forbidden scale) to the raw-metres regime.

Provenance:
- build_pipeline.write_provenance_manifest writes provenance.json (git commit,
  params, per-step list, sha256 of each bundle artifact) after the steps run.

Hermetic pipeline slice in CI:
- conftest osm_dir + regenerate_grid_layout guards skip the tests that need the
  uncommitted raw OSM CSVs, so scripts/pypsa_eur + scripts/game_mode pass from
  a fresh clone (228 pass, 9 skip). New test-data-pipeline job in test.yml runs
  them.

Tests: +6 frontend golden-fixture parity cases, +4 Python scorer cases,
+2 provenance-manifest cases, updated build_pipeline step-orchestration tests.
Docs: review tracker D8 marked mostly-done; game-mode-codabench, grid-layout
scale, CLAUDE.md, README, followups (FU-2) updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
Summarise the deep-revision roadmap's current state now that D6 and D8 have
landed alongside the earlier D1/D3/D4/D5/D7 work:

- Part V: a "Status at a glance" table (D1–D9 with one-line status + what
  remains) right under the progress note.
- Part V.5 "What's left": a consolidated open-items summary grouping the
  deep-revision tails (D9 not started; D2 / D7 tails), the tracked follow-ups
  (FU-1 useDiagrams core, FU-2 session-log replay), the still-open quick wins
  by area, and the quick wins already subsumed by shipped deep revisions.
  Ends with a suggested next step (D9).
- Part VI: mark QW1 (root tests, subsumed by D1) and QW18 (single-flight 409,
  subsumed by D3) as done.

Docs-only; no code or test changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
…he (QW15), gate console.log (QW21)

- QW11: DiagramMixin._diff_switches now aligns the two switch `open` columns
  on the shared index and compares in NumPy instead of two O(log n) `.loc`
  lookups per switch inside a Python loop (the action grid carries ~85k
  switches). Semantics unchanged; new unit test test_diff_switches.py.
- QW15: useActionDiagramCache LRU-caps at 3 entries (each pins a processed
  multi-MB SVG element since D6); re-priming refreshes recency. +2 tests.
- QW21: code-quality gate now counts frontend `console.log` and ratchets it
  at the current 25 (console.warn/error still allowed); report + test updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
… legacy-analysis watchdog (QW22)

QW17:
- services/paths.py: one shared OVERFLOW_DIR anchor. main.py's /results/pdf
  serve, the analysis writer (config.SAVE_FOLDER_VISUALIZATION) and the
  load-session copy target now all use it, so read and write agree regardless
  of the process CWD (previously the writer was CWD-relative, the serve
  repo-root-relative — they only coincided when uvicorn ran from the root).
- Fix two reset() leaks: `_n_state_currents` and `_last_action_path` were set
  lazily but never cleared, leaking a prior grid's state into the next study.
- test_reset_completeness.py: an automated invariant that sweeps every
  per-study field a fresh service sets and asserts reset() restores each — so
  a new cache added to __init__ but not reset() now fails a test instead of
  leaking silently.

QW22:
- analysis_runner.run_with_pdf_polling gains a hard deadline
  (_ANALYSIS_DEADLINE_S, env COSTUDY4GRID_ANALYSIS_TIMEOUT_S, 600s default):
  a hung worker no longer loops the generator forever and wedges the D3
  study-mutation gate — it raises TimeoutError which the endpoint turns into a
  {type:'error'} event and releases the gate. Worker is now a daemon thread.
  +2 watchdog tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
…W19), highlight-pipeline dep fix (QW14)

- QW20: new hooks/useModalKeyboard (Escape-to-close, initial focus, focus
  trap, focus restore, role/aria-modal) wired into ConfirmationDialog,
  SettingsModal and ReloadSessionModal. Latest-ref design so the effect
  depends only on isOpen/closeOnEscape (no focus thrash from inline onClose).
  +8 hook tests.
- QW19: detached popup windows now mirror the opener's light/dark theme —
  data-theme + colorScheme stamped at detach time and kept live via a
  MutationObserver on the opener's <html>. Purged the two 'white' literals
  (→ colors.surface). +2 tests.
- QW14: useDiagramHighlights' applyHighlightsForTab now depends on the
  specific `diagrams` members it reads (the two container refs + two
  metaIndexes) instead of the whole `diagrams` object, whose identity churns
  on every pan/zoom settle — so a settle no longer re-runs the whole highlight
  pipeline. +1 regression test.

Full Vitest suite green (1808); tsc + eslint + gate clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
… single CI (QW23), Game-Mode resilience (QW24), Dockerfile hygiene (QW25)

QW6: genericise the seven path-leaking FS/config str(e) error sites in main.py
(save/set config, /api/config, pick-path, save/list/load-session) — each now
logs server-side via logger.exception and returns a generic detail, so absolute
paths / raw exception text no longer reach the client. D2 only covered UNCAUGHT
exceptions; these raise explicit HTTPException(400)s. +2 leak-guard tests.

QW8: pin expert_op4grid_recommender via recommender-pin.txt (one exact version)
for PR + Docker installs, so an upstream release can't silently break a PR or a
rebuild; new weekly canary.yml floats to latest and flags regressions. Consistency
guard test scripts/test_recommender_pin_consistency.py. requirements_py310.txt
un-stalened (0.1.1 -> 0.3.0.post1).

QW23: delete .circleci/config.yml (GitHub Actions is a strict superset); scrub
the CircleCI references from CLAUDE.md / CONTRIBUTING.md / code-quality-analysis.

QW24: Game Mode now recovers from a mid-session backend failure — retryStudy
re-attempts the failed study and finishEarly ends with the already-committed
results instead of destroying them; GameShell surfaces Retry / Finish / Quit.
+ a preset↔overload-data consistency test (every preset contingency exists in
its grid's n1_overload_contingencies.json). +3 tests.

QW25: Dockerfile hygiene — honor $PORT (was decorative; shell-form CMD),
HEALTHCHECK on the always-available /api/user-config, and a committed+tested
scripts/extract_network_zip.py that FAILS LOUDLY on an un-smudged Git-LFS
pointer / corrupt archive instead of baking a broken grid; .dockerignore drops
the test suites from the image. +4 extract-validator tests. (The jupyter-util
runtime-dep slim is deferred — it touches nad/sld render imports.)

Gate + tsc + eslint clean; backend + frontend + scripts suites green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
Add a `--replay <session.json>` mode to scripts/game_mode/e2e_game_session.py
that re-derives TRUSTED per-study finalMaxRho/solved from a session log's
recorded actionsChosen by re-driving the real backend (config → step1 → step2),
falling back to /api/simulate-manual-action for a manual pick that is no longer
in the prioritized set. It writes a reference.json in the apply_reference()
shape and flags studies whose replayed numbers diverge from the self-reported
ones beyond --tolerance (tamper/drift detection; non-zero exit on divergence).

- Extract configure_backend() + analyze_contingency() shared by the greedy
  play path and the replay path so trusted numbers come from one pipeline.
- Hermetic coverage in scripts/game_mode/test_replay.py drives the replay
  machinery against a fake backend client (no FastAPI/pypowsybl/grid) — runs
  in the existing test-data-pipeline CI job. Real-grid reference generation
  stays an on-demand e2e-lane activity.
- Docs: game-mode-codabench.md gains a --replay section; followups.md FU-2
  marked landed (tooling + hermetic coverage).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
The recurring "docs drift" review finding — CLAUDE.md trees referencing
renamed-away files, and file.py:NNN line anchors that had rotted by hundreds
of lines — is now a gate instead of a periodic manual audit.

scripts/check_docs_tree.py scans the four inventory docs and fails on:
  A. a directory-qualified `path.ext` reference that resolves to no real file
     (exempting generated artifacts and paths described AS removed, via a
     ±1-line window so wrapped prose is handled);
  B. any `file.py:NNN` line-number anchor — the convention is a symbol anchor
     (name the function/class), which survives edits.

- Converted all seven pre-existing line anchors to symbol anchors and fixed the
  two drifted path references (StatusToasts row annotated as superseded by
  NotificationHost; _service_integration already described as removed).
- Wired as its own gate step in .github/workflows/code-quality.yml.
- scripts/test_check_docs_tree.py: fixture-driven classification + a real-repo
  self-guard so drift fails the pytest suite too, not only the CI step.
- Docs: code-quality-analysis.md §23; review D8/D9 status marked done.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
GitHub Issues are disabled on the fork, so followups.md is the canonical
issue log. Add self-contained entries for the work that remains after the
QW/FU-2/D9 pass, each pointing at its detailed tracking doc:

- FU-5 — D2 tail: response models on the gzipped endpoints, types.ts
  generation from the OpenAPI snapshot, blanket-handler removal.
- FU-6 — D7 tail: pinned Python-closure lockfile shared by CI + Dockerfile,
  config-merge on upgrade.
- FU-7 — D6 tail: VL-count auto-enable of the bitmap pan/zoom engine
  (validation-gated on operator hardware).
- FU-8 — decompose the size/complexity ceiling-riders (review target #4;
  unblocks FU-3/FU-4, which deferred on the exact-1150 recommender_service).

Also marks FU-2 resolved (replay tooling landed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
Version
- Bump pyproject.toml 0.8.0 -> 0.9.0; README release badge + "current
  release" line; root CLAUDE.md changelog pointer (0.8.0 -> 0.9.0).

CHANGELOG
- Promote [Unreleased] -> [0.9.0] (2026-07-09) with a release summary tying
  together the deep revisions D1-D8 that had accumulated under Unreleased.
- Add this pass's work: backend perf QW11/17/6/22, frontend QW14/15/19/20/21,
  delivery/CI QW8/23/25, Game-Mode resilience QW24 + trusted replay (FU-2),
  and the docs-as-a-checked-artifact gate (D9).
- Update the compare-link footer (0.9.0 tag + Unreleased base).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Am26qvtnHoY1qnmssaSQam
Signed-off-by: Antoine Marot <amarot91@gmail.com>
@marota marota closed this Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant