[gorkem2020]

Admission's utility scoring pays one LLM round trip per candidate. A reset with 10 mapped reflection rows fired 10 calls over roughly 24 seconds. This adds an opt-in utilityMode: "batch" that scores a whole batch's utility with one call per chunk of up to 10 candidates, sharing the instructions and the conversation/source excerpt across the batch.

Default stays standalone; nothing changes for installs that don't opt in.

Changes

• src/admission-control.ts:
  • AdmissionControlConfig.utilityMode gains "batch" alongside "standalone" | "off".
  • AdmissionController.evaluate()'s post-utility logic (confidence, novelty, recency, typePrior, the weighted decision, and audit construction) is extracted into a shared private evaluateWithUtility(), used by both the existing single-candidate path and the new batch path. Only the utility feature's transport changes; everything else is unchanged.
  • AdmissionController.evaluateBatch(items) scores a whole batch: one LLM call per chunk of up to 10 candidates, with the batch prompt listing every candidate indexed and demanding independent, absolute scoring per item (not a ranking within the batch), including a mixed-quality few-shot example. On a malformed or incomplete response (wrong count, missing/invalid index), it falls back to one standalone call per candidate in that chunk. This never fails the batch open or closed, it just costs more calls for that one chunk.
  • The batch prompt is built as { system, user } even though this branch's LlmClient.completeJson() still takes a single string (the two are concatenated here). This keeps the eventual merge with the system/user prompt-architecture split mechanical.
• src/smart-extractor.ts: when admissionControl.utilityMode === "batch", extractAndPersist() evaluates admission for every non-profile candidate in one upfront evaluateBatch() call, instead of one evaluate() call per candidate inside the sequential dedup/persist loop. processCandidate() accepts the precomputed evaluation and skips its own admission call when one is supplied. Profile candidates keep their existing separate admission check inside handleProfileMerge, unaffected by batching. Standalone/off modes are untouched, since no precomputed evaluation is ever produced for them.
• openclaw.plugin.json: configSchema.properties.admissionControl.properties.utilityMode.enum gains "batch" (default stays "standalone"). admissionControl has additionalProperties: false, so without this the new mode would fail config validation before ever reaching runtime code, unconfigurable as shipped.

Tests (all new, TDD red-first)

• test/admission-control-batch-utility.test.mjs: a batch of 3 candidates issues exactly one LLM call and produces three distinct audits; a malformed response falls back to one standalone call per candidate; batches larger than 10 chunk into multiple calls.
• test/smart-extractor-batch-admission.test.mjs: an extraction of 3 candidates in utilityMode: "batch" fires exactly one admission-utility-batch call end to end.
• test/plugin-manifest-regression.mjs: asserts the schema declares "batch" in the utilityMode enum and that the default remains "standalone".

Full local test chain green (npm test, minus the pre-existing host-Ollama port conflict on 11434, unrelated to this change) and npm run build green.

Batched dedup decider and merge writer (2026-07-17)

Extends the batch-call topology to the remaining two capture-path stages, so each pipeline event makes at most one LLM call per stage (extraction, admission utility, dedup decision, merge write):

• One dedup-decision-batch call decides every admitted candidate that needs an LLM verdict (chunked past DEDUP_BATCH_MAX_SIZE of 10). Each numbered candidate block carries its own retrieved-neighbor list; verdict vocabulary, thresholds, match_index semantics, and store operations are unchanged, only the call topology changes.
• One merge-memory-batch call writes every merge verdict, dedup-path and profile-path alike (chunked past MERGE_BATCH_MAX_SIZE of 10). Candidates merging into the same target row are grouped into one job, which also removes the stale-content clobber that precomputed verdicts would otherwise allow when two candidates target one row.
• Zero admitted candidates produce zero dedup calls, zero merge verdicts produce zero writer calls, and a single item still uses the batch shape.
• Per-item fail-closed parity with the single-call paths: a missing or malformed dedup entry degrades only that candidate to CREATE (the single-call unparseable default); a missing or malformed merge entry skips only that job with nothing persisted (the single-call llm-failed path); a failed chunk call degrades every item in that chunk the same way, never crashing and never fanning back out into per-item calls.
• Non-batch admission modes now evaluate per candidate ahead of the processing loop (same call count, earlier timing) so the admitted set is known before the dedup batch runs in every configuration.
• formatCandidateBlock moved to src/extraction-prompts.ts so the admission prompts and the new batched dedup/merge prompts all render numbered candidate blocks through one shared formatter.
• The single-call dedup-decision and merge-memory paths remain as the degraded fallback for candidates without a precomputed vector (batch pre-embed failure) and for direct processCandidate use without a merge queue.

Two deliberate behavior notes: dedup vector searches now run before any merge writes land (previously a later candidate's dedup search could observe an earlier candidate's merge within the same extraction; batch-internal near-duplicate pruning already covers the common case), and a grouped same-target merge emits one persistence notification for its single store write instead of one per candidate.

Update: single-source prompt architecture (2026-07-17)

Extracted a shared src/prompt-blocks.ts module and converged the admission, dedup, and merge-writer prompts onto it, closing the drift between the standalone and batch variants:

• Identity openers: every prompt now opens with its role as line 1. The standalone admission judge and the extraction agent were previously missing an identity line; the dedup decider and merge writer (both topologies) get new identity lines (You are a memory dedup judge... / You are a memory merge writer...).
• Shared category taxonomy and score-tier rubric: composed into admission, dedup, and merge prompts. The rubric now has three tiers (higher/moderate/lower) instead of two, picking up the moderate-events guidance.
• Markdown payload standard: candidate/job payloads move from indentation-only numbered lists to ## Candidates / ### N. category / plain Label: value field lines / #### Existing similar memories nested subsections, matching the extraction prompt's own markdown conventions. Every JSON output contract and few-shot example response is now fenced as a ```json code block.
• buildDedupPrompt and buildMergePrompt (the single-call variants) now take the full candidate object instead of separate positional strings, so they render through the same formatter as the batch variants.

Known gap, intentionally out of scope for this PR: the grounding-awareness rule paragraph (currently only on the standalone admission prompt via a separate, unmerged branch) still cannot be composed into this branch's batch admission lane, because this branch's CandidateMemory type carries no grounding field and its admission prompt shows no conversation excerpt at all. That convergence needs the grounding-awareness branch merged first; flagging it here rather than fabricating a rule that references content this branch never shows the model.

[gorkem2020]

Rebased onto current master as a clean three-commit stack (post #927/#943/#952/#974 merge wave) and marked ready for review.

Notable reconciliation choices: the extraction prompt adopts the landed hypothetical-grounding doctrine verbatim (register stretches, the about/within rule, unsure defaults to constructed); the deterministic constructed short-circuit now runs before any utility call in both the single-candidate path and the batch chunker, so an all-constructed chunk spends zero LLM calls. Prompt-contract tests keep every doctrine assertion and adapt only to the split system/user shape; the batch-embed test pins the batched-utility contract (profile candidates ride the Step-2 pre-compute batch). Full suite and all CI groups green.

[rwmjhb]

Reviewed head 7096c62. The batching tests and full suite pass, but four new production behaviors need correction before merge:

1. openclaw.plugin.json advertises batchChunkSize as the chunk bound for every batch stage, but PluginConfig and parsePluginConfig omit it and the SmartExtractor construction never passes it. Admission utility also hardcodes 10, and the advertised consolidate stage has no implementation. Non-default values are silently ignored. Please wire the setting end to end for the supported stages, normalize its range, and narrow the schema text to behavior that actually exists.

2. scoreUtilityBatch falls back only when completeJson throws. The production LlmClient contract returns null for request, empty-response, and JSON failures, so the normal failure path is logged as a successful batch and every candidate silently receives utility 0.5. Treat null as a call-level failure and perform the documented standalone fallback, with a null-returning production-shaped test.

3. When embedBatch omits a vector, batch admission receives [], skips similarity search, and records novelty=1. processCandidate can later recover the individual embedding but reuses that stale admission result. Skip precomputed admission for missing vectors or recompute it after recovery, and add a partial-batch-vector regression.

4. This PR changes explicitly enabled standalone admission even when utilityMode remains standalone: every preset now defaults utilityVetoThreshold to 0.25, and utility scoring no longer receives the source conversation excerpt. A candidate that previously passed can now be rejected without opting into batching. Preserve the previous standalone default (for example veto=0 and bounded source context) or separate and document this policy migration with an explicit opt-in and compatibility tests.

[gorkem2020]

All four items addressed in a1e1cb5.

1. batchChunkSize is wired end to end: the plugin config type, parsePluginConfig (normalized into 1-50, junk yields the default), and the SmartExtractor construction all carry it, and the extractor injects it into the AdmissionController so the admission-utility stage chunks by the same knob (BATCH_UTILITY_MAX_SIZE stays as the default when unset). The manifest description now names only the stages that actually read it (admission utility, dedup decider, merge writer) and documents the clamp; the consolidate wording is gone. Regressions: parser carry/normalization, and a 7-items-at-chunk-3 test asserting ceil(7/3)=3 calls.

2. scoreUtilityBatch now treats a null completeJson return as the same call-level failure as a throw and performs the documented standalone fallback; the failure log line fires on both shapes. A null-returning production-shaped test pins it: three candidates fan out to three standalone calls carrying their real scores, never the neutral 0.5 batch degradation.

3. A candidate whose batch embedding comes back missing or empty no longer enters the batch: its precomputed entry stays unset, so processCandidate re-evaluates it inline with the individually recovered vector instead of reusing a novelty=1 verdict scored against no matches. The partial-batch-vector regression asserts the vectorless candidate is absent from the batch prompt and gets exactly one inline utility call.

4. Standalone compatibility is restored where it was actually broken: presets now default utilityVetoThreshold to 0, so explicitly enabled standalone admission keeps its previous outcomes and the veto floor is an operator opt-in (the veto suite now pins the default-off contract plus the opted-in behaviors). On the conversation excerpt: that removal is a deliberate design decision of this PR rather than a side effect, and it is pinned by the prompt-shape tests. The judge scores the distilled candidate identically in standalone and batch modes (the batch prompt carries no transcript either, so choosing a mode cannot change scoring inputs), and raw transcript text is where fictional and roleplay framing lives, which the grounding pipeline already classifies before admission; re-feeding transcript to the utility judge would reopen that surface. With the veto defaulted off, composite scoring and thresholds are otherwise unchanged from master. If an excerpt-fed judge is wanted for some installs, we would prefer to add it as an explicit opt-in in a follow-up rather than re-couple utility scoring to transcript text here.

Typecheck, build (dist committed), manifest verifier, and the full suite are green; all six new regressions fail on the previous head.

[rwmjhb]

Thanks for addressing the four review items in a1e1cb5. The current head now has merge conflicts with the latest master after #934 merged, and no CI results are attached to this revision, so the fixes cannot yet be reliably re-reviewed against the code that would merge. Please rebase onto the latest master, resolve the conflicts while preserving the batchChunkSize wiring, null fallback, missing-vector recomputation, and standalone compatibility fixes, rerun the targeted tests and full CI suite, and push the resolved head for re-review.

[gorkem2020]

Rebased onto current master (post #934); the round history is squashed into one commit carrying the four review fixes. Conflict notes: index.ts resolved by keeping this branch's reviewed region and grafting master's new autoCaptureRetainedTextCap helper; the fallback-gating test assertions follow master's now-merged gated behavior; the verdict mock server now serves both dedup response shapes (top-level fields for the single-call path, indexed results for this branch's batched decider) so every gating scenario stays valid on either pipeline; registration files resolved by union. Typecheck, build (dist recommitted), manifest verifier (109 entries), and the full suite (55 files, 0 failures) are green. Mergeable again.

[rwmjhb]

Re-reviewed head a3acf35. The four requested production corrections are present on the rebased head: batchChunkSize is parsed and reaches the normal SmartExtractor admission path, null batch responses use standalone fallback, missing batch vectors are recomputed inline instead of reusing novelty=1, and the runtime utility-veto default is restored to 0. The targeted/full suites and GitHub CI are green.

A few follow-ups remain: the standalone fallback AdmissionController still uses chunk size 10, one mapped-reflection row failure can fail open the whole burst, the manifest still advertises veto default 0.25, and dedup prompts no longer include existing L1 overview details. These should be tightened, but none blocks the core opt-in batching path on this head.

Approving.