Repository navigation
Add pluggable recommendation models with registry and UI integration - #145
Merged
Merged
Conversation
Adds expert_backend/recommenders/ as the canonical place where models are registered for Co-Study4Grid. The registry is intentionally small: register(), build_recommender(), list_models() — third-party packages can ship additional models by importing register() and decorating their RecommenderModel subclass at import time. Default model is ExpertRecommender (re-exported from the library); two canonical examples ship out of the box: - RandomRecommender (requires_overflow_graph=False): samples uniformly from the action dictionary, augmented at runtime with synthetic reconnection / load-shedding / curtailment actions derived from the observation. Skips the expensive step-2 overflow-graph build. - RandomOverflowRecommender (requires_overflow_graph=True): samples uniformly from the actions retained by the expert rule filter (context["filtered_candidate_actions"]), so the random pick benefits from the overflow-graph topological analysis without any further scoring. Both models declare a single parameter (n_prioritized_actions) via params_spec(), so the UI can grey out every expert-specific knob when they are selected.
ConfigRequest gains two fields: - model (default "expert"): name of the recommender registered in expert_backend.recommenders. Existing clients keep getting the expert pipeline. - compute_overflow_graph (default True): toggle for the expensive step-2 graph build; only effective when the chosen model declares requires_overflow_graph=True. New endpoint GET /api/models lists every registered model with its label, requires_overflow_graph flag, default flag, and params_spec descriptors. The frontend uses this to populate the model dropdown and render only the parameters the selected model actually consumes. /api/config now echoes the resolved active_model + compute_overflow_graph so the UI can confirm what's in effect (helpful when an unknown model name silently falls back to the default).
Captures `model` + `compute_overflow_graph` from ConfigRequest and exposes them via get_active_model_name() / get_compute_overflow_graph(). The mixin is composed into RecommenderService alongside the existing Diagram/Analysis/Simulation mixins. State init is a no-op `__init__` so we don't depend on cooperative super() in the existing service constructor — `_apply_model_settings()` is called explicitly from `update_config` and `reset()` to keep things deterministic.
…atches Adds expert_backend/recommenders/_service_integration.py which: 1. Mixes ModelSelectionMixin into RecommenderService (state + getters for active model name and compute_overflow_graph toggle). 2. Wraps RecommenderService.update_config so the two new fields from ConfigRequest (model, compute_overflow_graph) are captured every time the operator applies new settings. 3. Wraps RecommenderService.reset so the model state is cleared along with every other cache when a new study is loaded. 4. Replaces RecommenderService.run_analysis_step2 with a model-aware generator: it builds the recommender from the registry, conditionally skips the overflow-graph step when the chosen model does not need it (or the operator disabled it), and threads the recommender all the way through run_analysis_step2_discovery. The integration module is loaded as a side-effect of importing expert_backend.recommenders, so /api/main.py only needs the existing single import to enable everything. This approach avoids rewriting recommender_service.py (~30 KB) and analysis_mixin.py (~14 KB) for a handful of additive changes, keeping the diff focused and reviewable.
…nder - api.ts: ModelDescriptor / ModelParamSpec types, api.getModels(), UserConfig.model + UserConfig.compute_overflow_graph. - useSettings.ts: recommenderModel + computeOverflowGraph state, availableModels fetched on mount, buildConfigRequest passes the new fields through, applyLoadedConfig / persistence cover them. - SettingsModal.tsx: model <select> + Compute Overflow Graph checkbox at the top of the Recommender tab. Each expert-specific parameter input is hidden when the active model does not declare it in params_spec, so operators don't waste time tuning knobs the model will ignore. Default model is "expert", default compute_overflow_graph is true — no behaviour change for existing operators.
Five focused test modules covering the new pluggable pieces: - test_recommenders_registry.py — register/unregister/build idempotence, rejection of empty names, fallback to DEFAULT_MODEL on empty/None, list_models() shape and canonical registrations (expert / random / random_overflow), per-model capability flags + params_spec. - test_random_recommenders.py — RandomRecommender metadata, samples N from dict_action, caps at pool size, augments with reconnections, skips actions env.action_space refuses, ignores entries without content. RandomOverflowRecommender stays inside the filtered set, falls back to dict_action when the filtered set is missing/empty, silently skips unknown ids. Uses random.seed(42) for determinism and mocks env / obs so the suite runs without pypowsybl/grid2op. - test_model_selection_mixin.py — state init, default values, attr resolution (string/whitespace/non-string), bool coercion for the toggle, getter safety before reset. - test_service_integration.py — verifies the side-effects of importing expert_backend.recommenders: mixin attached to RecommenderService, update_config / reset wrapped, run_analysis_step2 replaced, singleton initialised with defaults. Also exercises the model-aware step2 generator: missing context raises ValueError, unknown model name yields a single error event. - test_models_api.py — ConfigRequest defaults to expert with compute_overflow_graph=True, accepts custom model, round-trips through JSON. GET /api/models returns 200, lists the canonical three models, exposes the right capability flags + params_spec shape (every param has name/label/kind/default). Uses fastapi.testclient.TestClient via pytest.importorskip.
…require it When a model declares `requires_overflow_graph: true`, the operator should not be able to skip the step. Three changes enforce this end to end: 1. SettingsModal.tsx: the checkbox is now ALWAYS rendered when a model is selected (instead of being hidden for non-requiring models). For models that require the graph it's forced checked + disabled, label suffixed with "required by this model"; for models that don't, it's shown as an opt-in with the suffix "optional for this model". 2. useSettings.ts: a useEffect watches recommenderModel / availableModels and forces computeOverflowGraph = true whenever the active model needs the graph. Keeps persisted config in sync with what the backend will actually run. 3. _service_integration.py: backend logic switches from AND to OR. `needs_graph = recommender.requires_overflow_graph OR self.get_compute_overflow_graph()`. A model that declares the requirement can never be skipped, even if the client somehow sent `compute_overflow_graph: false`. The /api/run-analysis-step2 result event also echoes the resolved `compute_overflow_graph` (= needs_graph) so the UI can confirm what actually ran.
Action pins were stacking on the overloaded line whenever the
RandomRecommender served as the source: resolveActionAnchor walked
through getActionTargetLines / getActionTargetVoltageLevels, both
returned empty, and the helper fell back to `max_rho_line` — which
is identical across actions that don't reduce the overload.
Two-pronged fix:
1. expert_backend/services/analysis/action_enrichment.py
- `extract_action_topology` now backfills lines_or_bus / lines_ex_bus
/ gens_bus / loads_bus from `dict_action[id]["content"]["set_bus"]`
when the materialised action object exposes them as empty
attributes or position-indexed numpy arrays. Per-field check so
real action attributes are never overwritten.
- `_is_meaningful_dict` truthy-check replaces the unguarded
`if val` that would raise ValueError on numpy arrays with >1
element (the silent reason the topology blob ended up empty).
- `voltage_level_id` is now surfaced on the topology blob when the
dict_action entry carries `VoltageLevelId` (standard pypowsybl
switch-based action shape). Frontends can use it as a hint
without parsing the action ID or description.
2. frontend/src/utils/svg/highlights.ts
- `getActionTargetVoltageLevels` now reads `action_topology
.voltage_level_id` as the highest-priority signal, before falling
back to the description / action ID heuristics. This makes pin
anchoring deterministic for pypowsybl actions whose VL is known
server-side.
Coupling / switch-based actions (e.g. pypowsybl UUID-prefixed
`..._VL_..._coupling`) were rendering with zero badges in the
action card because:
- `getActionTargetLines` skips coupling actions by design,
- `getActionTargetVoltageLevels` only finds the VL when the SVG
metadata index resolves the name from the action ID / description,
which routinely fails on UUID-style IDs.
The recommender pipeline already surfaces the operator-facing target
VL on `action_topology.voltage_level_id` (backed by the dict_action
entry's `VoltageLevelId` — standard pypowsybl switch-based action
shape; commit just before this one). `renderBadges` now reads that
field as the highest-priority signal, before falling back to
load-shedding / curtailment / heuristic resolution. The chip is
clickable (zooms to the VL) and double-clickable (opens the SLD),
matching the existing load-shedding/curtailment VL chips.
…ed network
`AUBE P4_coupling` was being suggested by RandomOverflow even though
`AUBE P4` is not in the loaded small_grid network. Root cause: the
expert `ActionRuleValidator` filters by overflow-graph paths but does
NOT verify that an action's `VoltageLevelId` or `set_bus.lines_*_id`
target an element that actually exists on the loaded network. Dict
entries shipped for a larger grid leak through silently.
Two-pronged fix:
1. New `expert_backend/recommenders/network_existence.py` —
`filter_to_existing_network_elements()` reads the loaded pypowsybl
`Network` (via `inputs.network` exposed by the DTO), builds
`voltage_levels` / `lines ∪ 2_windings_transformers` id sets, and
drops any candidate whose `VoltageLevelId` or `set_bus.lines_*_id`
isn't in them. Conservative: returns the input list unchanged when
the network introspection itself fails — never silently empties
the pool just because the check errored.
2. `RandomOverflowRecommender`: applies the existence filter to
`inputs.filtered_candidate_actions` BEFORE sampling. Also hardens
the empty-set behaviour: when the expert rule filter genuinely
returned `[]` (filter ran, nothing passed the overflow paths),
we return `{}` instead of falling back to the full dict — that
fallback was masking the very bug the model is supposed to avoid.
Fallback to dict_action stays only when `filtered_candidate_actions`
is `None` (= filter didn't run at all), with a loud warning.
3. `RandomRecommender`: applies the same existence filter on the
dict before sampling. Synthetic reconnection / shedding /
curtailment actions remain untouched — they're built from
`obs`/`env`, so by construction they reference elements that
exist on the loaded network.
On larger grids the bug was glaring: RandomOverflow suggestions were
spread all over France while the overflow halo was concentrated around
one substation. Root cause: `ActionRuleValidator.categorize_actions`
(invoked by `_run_expert_action_filter`) removes broadly invalid
actions but does NOT narrow the set to overflow-relevant ones — that
targeting happens inside `ActionDiscoverer`'s per-type
`find_relevant_*` mixins (which consume the dispatch / constrained /
loop / hub path lists built from `g_distribution_graph`).
`RandomOverflowRecommender` skips those mixins (it samples instead of
scores), so its candidate set was the whole rule-validator-approved
subset of the dictionary — effectively the entire dict on large grids
because most actions are individually well-formed.
New module `overflow_path_filter.py`:
- `restrict_to_overflow_paths(candidate_ids, dict_action, distribution_graph, obs, hubs)`
- Extracts the same path targets the expert orchestrator does:
dispatch path lines + constrained path lines → `relevant_lines`;
dispatch loop nodes + blue path nodes + hub substations → `relevant_subs`.
- Keeps an action when ANY of these references matches:
1. `VoltageLevelId` in `relevant_subs` (switch-based / coupling),
2. `content.set_bus.lines_or_id` / `lines_ex_id` / `pst_tap` ∩ `relevant_lines`,
3. action-id suffix for `disco_`/`reco_` synthetic entries.
- Conservative on extraction failure (returns input unchanged so a
buggy graph never silently empties the pool).
- Empty path target sets (no overflow paths) returns [] — that IS the
correct behaviour ("no overflow-relevant actions for this case").
RandomOverflow now stacks three filters before sampling:
1. expert rule filter (`filtered_candidate_actions`),
2. overflow-path filter (new),
3. network-existence filter (existing).
Net effect: pins should now cluster on/near the overflow halo,
matching what the expert recommender would naturally explore.
…tion graph
Logged error:
overflow-path-filter: could not extract path targets
('<' not supported between instances of 'numpy.str_' and 'int');
skipping narrow filter.
`Structured_Overload_Distribution_Graph` can return node identifiers
as integer indices into `obs.name_sub` (legacy) OR as substation-name
strings (current build). My previous code only handled the integer
case (`idx < n_subs`), so the comparison raised on `numpy.str_` and
the whole filter fell through the `except` block — silently disabled,
RandomOverflow reverted to "wide pool" behaviour and pins clustered
all over the map.
Fix: new `_resolve_node_to_name(node, name_sub_arr, n_subs)` helper
that accepts both shapes:
- int / numpy.integer → `name_sub_arr[idx]` lookup
- str / numpy.str_ → return as-is
- bytes → utf-8 decode
- fallback → str(node)
All node coercions in `_extract_path_targets` route through it.
Also:
- str()-coerce every line id and VL hint on the comparison side so
asymmetric str-vs-numpy.str_ set membership doesn't silently miss.
- New segment-scan in `_action_touches_path` for UUID-prefixed
coupling action IDs of the form `<uuid>_<VL>_..._coupling` —
each `_`-split segment is checked against `relevant_subs`,
catching the canonical pypowsybl switch-based shape even when
the dict entry's `VoltageLevelId` is missing.
…t, random-overflow fallback
Four focused test modules covering the recent bug fixes:
- test_network_existence.py — `filter_to_existing_network_elements`
and its helpers. Covers the AUBE P4 regression (action targeting a
VL absent from the loaded grid), conservative fallback when the
network introspection fails, transformer ids accepted as branches,
input-order preservation, and dual `VoltageLevelId` /
`voltage_level_id` key handling.
- test_overflow_path_filter.py — `_resolve_node_to_name` covering both
shapes the distribution graph returns (int / numpy.int_ →
name_sub lookup; str / numpy.str_ / bytes → as-is), plus the
`_extract_path_targets` regression where `numpy.str_` nodes used to
crash `idx < n_subs` and silently disable the filter.
`_action_touches_path` covers VL hint, set_bus, pst_tap, disco/reco
prefix and the UUID segment scan. End-to-end test asserts the filter
still narrows correctly when nodes come back as numpy strings.
- test_action_enrichment.py — `extract_action_topology` covers the
numpy-array attribute case (no ValueError on `if arr`), set_bus
backfill for all four target kinds (lines_or/ex/gens/loads),
voltage_level_id surfacing (upper- and lower-case), switches
fallback, and the combined switch-based pypowsybl shape that
produced the AUBE P4 / MOTTAP3 cases. `_is_meaningful_dict` covers
numpy arrays, empty dicts, lists, None, strings.
- test_random_recommenders.py — extended:
* RandomRecommender: drops dict entries for unknown VLs (regression).
* RandomOverflowRecommender: separates None vs [] semantics
(filter not run → fallback warning; filter ran with 0 result →
return {}). Asserts the three-layer filter chain
(rule → path → existence) drops the right actions at each layer.
Comprehensive walkthrough of the Co-Study4Grid integration: 1. Registry overview (register / build / list_models / DEFAULT_MODEL). 2. Built-in models: Expert (default), Random, RandomOverflow. 3. The three-layer filter chain RandomOverflow applies before sampling (expert rule filter → overflow path filter → network existence) with the rationale for each layer and the conservative-on-failure behaviour. 4. Backend wiring: ConfigRequest fields, /api/models endpoint, _service_integration patches (ModelSelectionMixin, update_config wrap, run_analysis_step2 replacement). 5. Frontend wiring: useSettings state + availableModels fetch, SettingsModal recommender tab (model dropdown, Compute Overflow Graph toggle states, dynamic params_spec rendering), ActionCard VL chip from action_topology.voltage_level_id. 6. Step-by-step guide to add a new model from a third-party package. 7. Test coverage map. 8. Troubleshooting section covering the bugs we hit and fixed (filtered_candidate_actions None, stacked pins, wide pool, numpy.str_ filter crash, Compute-Overflow-Graph toggle locking).
… sessions
The backend already echoes `active_model` and `compute_overflow_graph` in
every `result` event from /api/run-analysis-step2 (see
`_service_integration.py:_run_analysis_step2_with_model`). The frontend
just dropped them on the floor — the saved session JSON had no way to
tell which recommender produced the suggestions, which made reloads
ambiguous (especially when an unknown model name had silently fallen
back to the default).
Two-file change:
1. `frontend/src/types.ts`
- Extends `AnalysisResult` with optional `active_model?: string`
and `compute_overflow_graph?: boolean`.
- Extends `SessionResult.analysis` with `active_model?: string | null`
and `compute_overflow_graph?: boolean | null`.
- Extends `SessionResult.configuration` with `model?` and
`compute_overflow_graph?` so the saved snapshot also captures
what the operator HAD SELECTED (vs. what the backend actually ran,
which lives in `analysis.active_model`).
- Extends `ConfigRequest` with `model?` and `compute_overflow_graph?`
to match the backend Pydantic shape.
- Adds `voltage_level_id?` to `ActionTopology` (companion to the
backend extraction surfaced for pin / chip placement).
2. `frontend/src/utils/sessionUtils.ts`
- `SessionInput` gains `recommenderModel?` and `computeOverflowGraph?`
(snapshot of the settings at save time).
- `buildSessionResult` writes `result.active_model` /
`result.compute_overflow_graph` into `session.analysis`, and
conditionally writes the captured settings into
`session.configuration` (omitted when caller didn't pass them so
legacy callers stay byte-compatible).
The `useSession` hook callsite already spreads every relevant settings
field into `SessionInput`; the next pass through that file should add
`recommenderModel` + `computeOverflowGraph` from `useSettings` so the
two new optional fields actually carry data. The result-event-side
fields (`analysis.active_model`, `analysis.compute_overflow_graph`)
already flow end-to-end with this commit alone.
The recommender is pluggable end-to-end (registry → backend dispatch →
frontend UI). This section walks a developer from zero to a working
plug-in in three steps:
- Blurb at the top + feature bullet under "Contingency analysis &
remediation" so the pluggable interface is discoverable from the
landing.
- New top-level "## Plug Your Own Recommendation Model" section
inserted between Performance Highlights and Architecture, with:
- Table of the three built-in models (Expert default, Random,
RandomOverflow) and what each is good for.
- The three-layer filter chain (rule → path → existence) with
the conservative-on-failure guarantee, pointing at
overflow_path_filter.py and network_existence.py.
- Three-step plug-in guide: implement the contract, register the
class, no further wiring needed (frontend picks it up via
GET /api/models). Includes a copy-pasteable class skeleton
showing every available field on RecommenderInputs.
- References to the library-side contract docs in
Expert_op4grid_recommender and the app-side detailed doc in
docs/recommender_models.md.
- Architecture tree updated to surface `expert_backend/recommenders/`.
- Getting Started step 2 mentions Settings → Recommender; step 8
mentions `analysis.active_model` persistence.
- API Reference: GET /api/models row added; /api/run-analysis-step2
and /api/config descriptions updated to mention `model` and
`active_model` echoing.
- Data Formats: session.json `analysis.active_model` + `compute_overflow_graph`
documented as persisted fields.
- Sessions & replay bullet mentions "active recommender model" in
the Save Results description.
…ackend overview
The pluggable-recommender doc was sitting at docs/recommender_models.md
(root of the docs tree). Moves it under docs/backend/ where it
belongs alongside a new backend-overview doc.
Two files in this commit:
- docs/backend/README.md (NEW) — backend-overview doc. Covers:
* Module layout (services + recommenders) with line-level
annotations so contributors find the right file fast.
* The two-singleton architecture (network_service +
recommender_service with its mixin composition).
* Pluggable recommenders — short blurb + cross-ref to
recommender_models.md.
* Data-flow diagram for /api/config → step1 → step2_graph
→ step2_discovery, explicitly showing the recommender
dispatch point and the active_model echo on the result event.
* Conventions: per-endpoint gzip (with the rationale for NOT
using global middleware on the streaming endpoint), NumPy →
JSON coercion, mixin-based service composition, pre-extraction
+ idempotent helpers, defensive filters.
* Endpoint group summary (linking to top-level README).
* Session persistence cross-ref to save-results.md.
* Test layout summary.
- docs/backend/recommender_models.md — same content as the previous
docs/recommender_models.md, with §4 ("Backend wiring") expanded
to mention the active_model + compute_overflow_graph echo on
the result event (used by the saved-session persistence) and
cross-refs to the new backend-overview README.
A follow-up commit deletes the old docs/recommender_models.md and
updates the top-level README link to point at docs/backend/recommender_models.md.
….json
session.json now captures the pluggable-recommender selection via two
parallel fields:
- configuration.model / configuration.compute_overflow_graph
— what the OPERATOR picked at save time (Settings → Recommender)
- analysis.active_model / analysis.compute_overflow_graph
— what the BACKEND actually executed (echoed in the result event
from /api/run-analysis-step2). Differs from the configured value
only when an unknown name silently fell back to the default.
Updates in this commit:
- "What is saved" — mentions the active recommender model.
- "How to Save" — adds a Settings → Recommender step (step 2) so
operators know the selection is captured.
- "What happens on reload" — restores the model selection on the
Settings dropdown AND forwards it to /api/config so subsequent
runs use the same model as the saved session.
- JSON Structure example — adds `model` + `compute_overflow_graph`
to the `configuration` block, `active_model` + `compute_overflow_graph`
to the `analysis` block, and `voltage_level_id` to the
`action_topology` sub-object (matches the new ActionTopology shape).
- Field Reference: `configuration` table — two new rows for `model`
and `compute_overflow_graph` with legacy-default guidance.
- Field Reference: `analysis` table — two new rows for `active_model`
and `compute_overflow_graph`.
- New top-level "Recommender model persistence" section explaining
the split between `configuration.model` (operator intent) and
`analysis.active_model` (backend ground truth).
- Implementation Details — Save flow step 4 documents the new
`result.active_model` / `result.compute_overflow_graph` propagation
in buildSessionResult; Reload flow step 2 mentions the new setters.
- Testing — sessionUtils test list mentions the new recommender-model
persistence assertions.
Cross-refs to docs/backend/recommender_models.md (the moved
pluggable-models doc).
…ents - Add `model` and `compute_overflow_graph` to `config_loaded` and `settings_applied` event detail payloads (replay-required fields). - Add `active_model` and `compute_overflow_graph` to `analysis_step2_completed` (sourced from the final step-2 stream event). - Update example interaction_log.json with the new fields. - Add a "Pluggable recommender model" section cross-referencing docs/backend/recommender_models.md. - Document the configuration persistence pair (`model` / `compute_overflow_graph`) under "Session reload fidelity".
- Update "Plug Your Own Recommendation Model" → References to point at docs/backend/recommender_models.md (the doc was moved in 68c34a2). - Add a reference to the new docs/backend/README.md backend overview. - Update the architecture tree blurb to mention the new docs/backend/ subfolder.
Folds the previous [Unreleased] entries (config-modal stale-write fix, PyPSA-EUR Mercator layout, Combined-only pin filter) and the pluggable-recommendation-model feature (PR #145, paired with expert_op4grid_recommender 0.2.2) into a tagged 0.7.5 section.
Collaborator
Author
Draft release notes — 0.7.5Same caveat as the Expert_op4grid_recommender one: the GitHub MCP server I have exposes only read-only release endpoints, so I couldn't create the GitHub draft release object directly. Once this PR is merged, paste the body below into https://github.com/marota/Co-Study4Grid/releases/new — same format as the Tag: Feature + polish release headlined by the pluggable recommendation models integration (paired with HighlightsPluggable recommendation models (PR #145, paired with
|
| Name | Label | Requires overflow graph | Best for |
|---|---|---|---|
expert |
Expert system | Yes | Default — rule-based discovery + scoring on every action type. |
random |
Random | No | Sanity-check baseline. Samples uniformly from the action dictionary, augmented with synthetic reconnection / load-shedding / curtailment actions. |
random_overflow |
Random (post overflow analysis) | Yes | "Is the overflow analysis useful?" baseline. Samples uniformly inside the expert-reduced action space (rule filter + overflow paths + network existence). |
- Selecting a model is a one-dropdown gesture in Settings → Recommender. The parameter inputs render dynamically from each model's
params_spec()so the UI hides knobs the active model doesn't consume. - The Compute Overflow Graph (step 1) toggle is locked-on for models that require it (
requires_overflow_graph=True) and editable for the others — letting the operator opt in to the overflow analysis even with the Random model. - The active model is persisted in
session.jsonunderanalysis.active_model(backend ground truth) andconfiguration.model(operator intent at save time); same split forcompute_overflow_graph. - Third-party models plug in with three lines of code — implement the ABC, decorate with
@register, ensure import. The frontend picks them up automatically viaGET /api/models. See the new Plug Your Own Recommendation Model section in the README.
"Combined only" pin filter
New pin-scoped filter on both the Action Overview tab and the Overflow Analysis iframe sidebar. When enabled, the pin layers render combined pairs plus their two constituents (dimmed for context) and drop every other unitary / un-simulated pin — the Action Feed cards remain unfiltered. Round-tripped through the existing cs4g:filters postMessage envelope so both surfaces stay in lock-step.
Config-modal stale-write fix
Switching the config-file path and clicking Apply now actually sends the freshly loaded config to /api/config. The previous behaviour silently sent the previous render's closure values, which the auto-save effect then persisted back into the freshly loaded file — undoing the operator's selection.
PyPSA-EUR grid layouts use raw Mercator metres
The previous 8 000-unit rescaling default forced pypowsybl VL circles to overlap in dense regions like Paris. Default behaviour is now raw projected metres (~1.4 M span for the French grid); pass --target-width N to reproduce the legacy rescaled output.
Added
- Recommender model registry (
expert_backend/recommenders/):registry.py,random_basic.py,random_overflow.py,synthetic_actions.py,overflow_path_filter.py,network_existence.py,_service_integration.py. GET /api/modelsendpoint — returns the full list of registered recommenders with theirparams_spec(), label and capability flags.ConfigRequest.model/ConfigRequest.compute_overflow_graphfields, echoed asactive_model/compute_overflow_graphin the finalresultevent of the step-2 NDJSON stream.- Saved session model echo —
session.analysis.active_model(backend ground truth) +session.configuration.model(operator intent), same split forcompute_overflow_graph. - Frontend types and hooks —
ModelDescriptor/ModelParamSpec,recommenderModel/computeOverflowGraph/availableModelsstate inuseSettings, dynamic dropdown + locked-vs-optional toggle states inSettingsModal, action-card VL chip readsaction_topology.voltage_level_id(so OPEN / CLOSE coupling cards double-click-zoom to the correct VL). - "Combined only" pin filter on the Action Overview tab and the Overflow Analysis iframe.
docs/backend/subfolder —README.md(broader backend overview) andrecommender_models.md(relocated fromdocs/recommender_models.mdwith app-side integration + filter chain + step-by-step guide).- "Plug Your Own Recommendation Model" section in the root
README.md. - Backend tests —
tests/test_recommenders_registry.py,test_random_recommenders.py,test_model_selection_mixin.py,test_service_integration.py,test_models_api.py,test_network_existence.py,test_overflow_path_filter.py,test_action_enrichment.py.
Changed
extract_action_topologyrobustness (expert_backend/services/analysis/action_enrichment.py): backfills emptylines_or_bus/lines_ex_bus/gens_bus/loads_busfromdict_action[id].content.set_bus, surfacesvoltage_level_id, and tolerates numpy arrays via a new_is_meaningful_dicttruthy-check. Fixes pins stacking onmax_rho_linewhen running the Random model.build_recommender_inputspropagation: the expert-rule filter result (context["filtered_candidate_actions"]) is now forwarded to the DTO so sampling models actually see the filtered pool — caught a silent bypass whereRandomOverflowRecommenderran against the full action dictionary while the filter was running upstream.overview_filter_changedinteraction-log event now carries acombined_onlydiscriminator.
Fixed
- Settings modal stale-write on config-file switch:
changeConfigFilePathnow returns the resolvedUserConfig, andconfigRequestFromUserConfigderives the request from it. Regression test infrontend/src/App.configUpload.test.tsx. - PyPSA-EUR grid-layout rescaling — default is now raw Mercator metres.
data/pypsa_eur_fr225_400/grid_layout.jsonanddata/pypsa_eur_fr400/grid_layout.jsonregenerated. - Action overview pin localisation for non-disconnection actions (Random / Random Overflow runs): pins are now anchored on the action's voltage level rather than the contingency
max_rho_line. numpy.str_comparison crash in_resolve_node_to_nameon legacy distribution graphs.
Documentation
docs/features/save-results.md—model/active_model/compute_overflow_graphfield tables + new "Recommender model persistence" section.docs/features/interaction-logging.md—model/compute_overflow_graphadded toconfig_loaded/settings_applied;active_model/compute_overflow_graphadded toanalysis_step2_completed; new "Pluggable recommender model" section.docs/backend/README.md— NEW: backend overview (architecture, mixins, data flow, conventions, endpoints, tests).docs/backend/recommender_models.md— NEW (relocated): app-side integration + filter chain + step-by-step guide.README.md— NEW "Plug Your Own Recommendation Model" section.
Compatibility
modelandcompute_overflow_graphdefault to"expert"andtrueat every entry point that lacks them (older session dumps, missing form values, third-party callers that didn't update their request shape) — byte-for-byte the same behaviour as 0.7.0.- Frontend dynamic UI from
params_spec()— adding a model requires zero UI code; the dropdown and parameter inputs refresh automatically. - Step-2 NDJSON contract unchanged —
active_modelandcompute_overflow_graphare additive fields on the existingresultevent. - Requires
expert_op4grid_recommender>=0.2.2(for theRecommenderModelABC, theRecommenderInputs/RecommenderOutputDTOs, the reusable reassessment phase and the idempotent_run_expert_action_filterhelper). Older versions raise anImportErrorfromexpert_op4grid_recommender.models.baseon backend startup.
Install
pip install "co_study4grid==0.7.5" "expert_op4grid_recommender>=0.2.2"See CHANGELOG.md § 0.7.5 for the full entry.
Full changelog: marota/Co-Study4Grid@0.7.0...0.7.5
Generated by Claude Code
Pre-existing test suites mock `../api` with vi.mock and only provide the methods they exercise. The effect that fetches the recommender model registry was assuming `api.getModels` always exists, which made the whole App tree fail to mount in every test that imports App (95 failures). Skip the fetch when the stub doesn't define `getModels` — the empty `availableModels` state already triggers the fallback "show all parameters" path in SettingsModal.
Pre-existing test suites hand-build a partial `SettingsState` mock without the new `availableModels` field. Default to an empty list locally so the `.find(...)` lookup doesn't crash, and reuse the local `models` variable in the dropdown render so the empty-state fallback (single "Expert system" option) still kicks in.
… apply The model-aware ``_run_analysis_step2_with_model`` was importing ``run_analysis_step2_graph`` / ``…_discovery`` directly from ``expert_op4grid_recommender.main``. Existing tests patch them on ``expert_backend.services.analysis_mixin`` (the long-standing seam used by the legacy step-2 generator); that patch never intercepted the new code path, so the real library functions ran and crashed on ``KeyError: 'backend'`` against the test's mock context. Switching to ``analysis_mixin.run_analysis_step2_graph(...)`` attribute-access at call time honors the module-level patch (the upstream function is re-resolved on every call, so replacing the binding on the ``analysis_mixin`` module is observed). Four backend test regressions resolved.
marota
pushed a commit
that referenced
this pull request
May 14, 2026
Tests for the model-selection feature work shipped across this branch.
Frontend (Vitest, +10 specs):
- ConfirmationDialog: the new `clearSuggested` dialog type renders its
bespoke "kept vs. removed" body and carries a stable testid.
- ActionFeed: the model dropdown renders above Analyze & Suggest,
changing it fires setRecommenderModel + logs
recommender_model_changed{source:'action_feed'}, the dropdown is
omitted when the prop is unwired, the "Suggestions produced by
<model>" reminder shows/hides correctly, the Clear button calls
onClearSuggested, and the Analyze & Suggest slot reappears when only
rejected actions remain (the prioritizedEntries gate).
- api: setRecommenderModel POSTs to /api/recommender-model.
Backend (pytest):
- test_overload_filtering: +5 specs for the step-2 graph cache —
identical signature skips the rebuild + yields cached:true, a
changed additional_lines_to_cut signature rebuilds, the result
event carries active_model (and is None when the getter is absent),
and reset() clears _last_step2_signature / _last_step2_context.
- test_api_endpoints: +4 specs for POST /api/recommender-model
(applies + echoes, optional compute_overflow_graph, 422 on missing
model, 400 on service error). NOTE: this file is not collectable in
the sandbox mock layer — pre-existing PR #145 conftest gap, the
tests follow the file's established pattern and run in CI.
Docs:
- interaction-logging.md: recommender_model_changed +
suggested_actions_cleared event types, their replay details, the
mid-session-model-swap ordering contract, and the wait-point note.
- recommender_models.md: POST /api/recommender-model, the step-2
overflow-graph signature cache, and the ActionFeed model selector /
active-model reminder / Clear button wiring.
- save-results.md: note that the model can also be swapped from the
Analyze & Suggest dropdown (same persisted state).
- state-reset doc + expert_backend/CLAUDE.md: _last_step2_signature
added to the reset() contract.
- README + root CLAUDE.md: /api/recommender-model + /api/models rows
in the API tables; model-selection prose updated for the dropdown
above Analyze & Suggest and the confirmation-gated Clear button.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a pluggable recommendation model system to Co-Study4Grid, allowing operators to select between the expert rule-based system, random baselines, or third-party models. The implementation includes a registry pattern, three-layer filter chain for overflow-aware sampling, comprehensive test coverage, and full frontend integration.
Key Changes
Backend Architecture
expert_backend/recommenders/registry.py): Minimal in-memory registry forRecommenderModelclasses withregister(),build_recommender(),list_models(), andunregister()functionsRandomRecommender(no overflow graph required) — samples uniformly from action dict + synthetic reconnection/shed/curtail actionsRandomOverflowRecommender(requires overflow graph) — samples from actions touching overflow-graph pathsRandomOverflowRecommender:ActionRuleValidator)overflow_path_filter.py) — restricts to dispatch/constrained/loop pathsnetwork_existence.py) — validates target elements exist in loaded network_service_integration.py): PatchesRecommenderServicewithModelSelectionMixinto track active model and overflow-graph toggle; wrapsupdate_config()andrun_analysis_step2()for model dispatchvoltage_level_idhint fromdict_actionfor improved pin placement and VL chip renderingFrontend Integration
/api/modelsendpoint; dynamically shows/hides parameter inputs based on selected model'sparams_specmodelandcompute_overflow_graphfields added toConfigRequestandUserConfig; captured in session.json for reloadModelDescriptorandModelParamSpecinterfaces mirror backend registry outputAPI Changes
POST /api/config: Accepts newmodel(defaults to "expert") andcompute_overflow_graph(defaults to true) fieldsGET /api/models: New endpoint returns JSON array of available models with metadata (name, label, requires_overflow_graph, params_spec)POST /api/run-analysis-step2: Now dispatches to selected model via registry; conditionally skips overflow-graph build if model doesn't require itTesting
test_recommenders_registry.py): Registration, lookup, build, list operationstest_random_recommenders.py): Both models with mocked grid2op/pypowsybl stacktest_overflow_path_filter.py— node resolution (int/numpy.int_/string coercion), path extraction, action-path intersectiontest_network_existence.py— element existence validationtest_action_enrichment.py): Topology extraction,voltage_level_idpropagation, numpy array handlingtest_service_integration.py): Mixin attachment, config wrapping, model dispatchtest_model_selection_mixin.py): State managementtest_models_api.py): ConfigRequest schema,/api/modelsendpointDocumentation
docs/backend/recommender_models.md: Comprehensive guide covering registry, built-in models, filter chain, backend/frontend wiring, and step-by-step integration guide for third-party modelsdocs/backend/README.md: Backend overview with architecture, singletons, mixin pattern, and data flowNotable Implementation Details
https://claude.ai/code/session_01D3rq784pJnzSUxkhS1Wf6P