integration: emit DecisionOpportunity + episode/credit records across OMP, Hermes, and DeepSeek #62

Activity

1. kvnloo commented on Sep 30, 2026

   

   kvnloo

   on Sep 30, 2026 – with ChatGPT Codex Connector

   OwnerAuthor

   More actions

   Cross-harness children/status:

   • OMP: integration: emit z0 DecisionOpportunity + outcome/credit receipts from the OMP hot path oh-my-pi#109 created.
   • Hermes: reuse [lab:S1] Wire Hermes to z0int shadow decisions on the existing adapter hermes-agent#319; do not create a duplicate.
   • DeepSeek: repo currently returns HTTP 410 for issue creation because Issues are disabled, so keep DeepSeek work tracked here until that repo surface is enabled.

   All three harness paths should converge on the same semantic record, not three harness-specific schemas.
2. kvnloo commented on Oct 4, 2026

   

   kvnloo

   on Oct 4, 2026 – with ChatGPT Codex Connector

   OwnerAuthor

   More actions

   RFC addendum: native telemetry → verified learning, with a minimal Hermes/OMP integration

   Status: proposed; source-audited on October 4, 2026. No live deployment, benchmark, or completed end-to-end integration is claimed.

   Architecture parent: z0#21 and its existing observation RFC. Learning contracts: #54, #58, and this issue. Harness work stays in OMP#109, OMP#111, and Hermes#319, rather than creating another parallel integration program.

   Decision

   Build one passive evidence join, not another tracing system or optimizer.

   Hermes already owns NeMo Relay execution scopes. OMP already owns OpenTelemetry instrumentation. Keep both. Add generic correlation to existing z0 receipts, normalize observations through AgentTrace, and feed the existing episode/credit/evaluation/promotion path.

   No mandatory Relay dependency in z0intelligence or OMP; no second Hermes Relay host; no new profiler UI; no model inference or training triggered synchronously by telemetry.

   What the source audit established

   Surface	Evidence	Consequence
   Canonical map	z0's components, suite, harnesses, mechanisms, and interfaces	Reuse existing owners. Registry membership is not evidence that an integration is running.
   Hermes	Upstream and downstream agent/relay_runtime.py returned the same source blob, 4c8871b98623e09bc3ea2a9cbb7c1a28bb49dcaf, in this audit. Current upstream metrics/configuration documentation describes the built-in runtime.	Configure and correlate the existing runtime; do not resurrect the removed observability/nemo_relay plugin.
   OMP	packages/agent/src/telemetry.ts exposes native spans and callbacks. The map's pinned commit d707478d4c5c72784c058a0163b616988f559230 also contains native telemetry.	Reuse the host configuration seam. Main and the mapped pin differ, including extension attribute naming; validate the installed revision rather than assuming exact parity.
   AgentTrace	README and parser dispatch already cover Hermes/OMP historical logs, reports, and slow-run diagnosis.	Extend this normalizer/report surface. No dedicated ATOF/OTLP path was found in the inspected dispatch; generic JSON support is not proof of complete native-event ingestion.
   z0intelligence	receipt.py, thin Hermes adapter, and zer0.repo.yaml	Receipts already have extra, measurement completeness, outcome joins, and experiment identifiers. Extend those rather than replace the schema.

   The map was reviewed across its component, suite, harness, mechanism, and interface catalogs; implementation inspection concentrated on the integration boundaries above. This is not an audit of every implementation in every suite repository, nor an inspection of the running machine.

   How Hermes actually uses Relay

   There are two different data paths:

   1. Core execution observation: session/turn/model/tool scopes and configured Relay subscribers/exporters.
   2. Shared metrics: a consented, bounded projection into local SQLite counters and optional packages. It omits session/task/request identifiers and rich payloads; it is not sufficient for exact per-decision attribution.

   Use the first path for local correlation. Enabling shared metrics is not a prerequisite for this learning loop. Current upstream documentation specifies nemo-relay >=0.9,<0.10; freeze the actual installed wheel/version and schema rather than copying older version assertions from the architecture RFC.

   Inspect hermes doctor and the effective Relay plugin configuration first. HERMES_NEMO_RELAY_PLUGINS_TOML replaces the user configuration file, but the system layer still applies. Relay plugin/exporter policy is process-wide: profile-local consent does not isolate rich exporters. Separate processes are required when profiles need different trust/export policies. Scope data can include cwd, so path privacy matters too.

   Start with one local, metadata-allowlisted exporter and active adaptive behavior disabled. Consume completed turns rather than waiting for a gateway session that can remain open for days. Preserve session-segment identity when segmentation is enabled. Unsupported native platforms and missing exporters must report reduced coverage, not fabricated empty traces.

   Smallest implementation surface

   1. Add correlation at the existing harness adapter boundary

   Prototype an optional generic extra.execution_ref on the existing DecisionReceipt; do not change z0int.decision_receipt.v1 merely to add vendor fields.

   Use only identities the producer actually knows: harness/session/turn/attempt or native execution identity, plus optional distributed trace_id/span_id. Bind these to the existing immutable receipt record reference. The adapter owns the minimal field contract and validation.

   Do not replace z0's existing trace_id with an OpenTelemetry trace ID or Relay scope UUID. Those identifiers have different semantics. Relay scope IDs stay in the source adapter's mapping, not as a required z0 core type. Preserve one-to-many relationships when several decisions and calls belong to the same execution.

   For Hermes, use its core-owned live context and the thin z0 adapter. Never create a second session scope or manipulate a global scope stack to correlate worker events.

   For OMP, wire existing AgentTelemetryConfig.resolveAttributes, onSpanStart/onSpanEnd, onChatUsage, and onRunEnd at the host boundary. onChatUsage already supplies the span and usage; onRunEnd supplies summary and coverage. Verify how the installed host exposes this configuration before claiming an extension can set it directly. No agent-loop fork is required by the proposed contract.

   Callbacks only attach bounded metadata or enqueue bounded records. No network wait, training, subprocess-per-event, or model call. Existing telemetry callback failure handling is not proof of negligible overhead; measure it.

   2. Normalize and join locally

   Add the narrow native-source support needed in AgentTrace, preserving source identities, timing, parentage, schema version, and fidelity. Begin with one Hermes ATOF fixture and one OMP native telemetry fixture; retain historical-log fallback as lower-fidelity evidence.

   Join normalized execution references to existing decision/outcome and Tokenomics records. Store raw observations with their existing local owner; z0intelligence consumes bounded episode features and evidence references, not a second span database.

   Exact joins require explicit identity. Timestamp proximity may be a diagnostic heuristic but must not become an exact training join. Report matched, partial, ambiguous, and unmatched coverage. Unknown ATOF major versions fail explicitly; compatible minor additions retain unknown fields according to the source format contract.

   Canonical direct usage events take precedence over observer-derived copies. Never count one provider call twice because both a harness receipt and an exported span describe it. Missing usage stays unknown. Preserve logical-call versus physical-attempt distinctions, auxiliary calls, cached-token categories, and late outcomes.

   3. Produce one joined report before building a learner

   Use AgentTrace's existing report surface plus Tokenomics economics and z0 receipt references. The first useful output is: which stage cost time, which z0 decision affected it, what evidence supports the outcome, and what is still unmeasured?

   Do not require a new omp profile command, dashboard, streaming database, or every export format for this slice. Add context-resolution/router timing only where existing evidence cannot answer the first experiment. Do not reinstrument existing model/tool boundaries.

   Mapping the rest of the z0 stack

   These are existing ownership boundaries and proposed connections, not claims that all connections are implemented:

   Owner / related surfaces	Role in this loop
   z0; z0archy with ActiveGraph/graphcon-deck	Canonical contracts and derived architecture/evidence views. No runtime service or policy promotion here.
   aodl; z0 StatePacket/DecisionOpportunity	Preserve authored intent, legal choices, budgets, evidence revision, and independent outcome requirements. Admission is not task success.
   Context resolver; EventLog/OptMem; source memory stores; Hermes LCM; OMP RLM; SoL-Pi ports	Candidate context/retrieval/compression mechanisms. Compare total retrieval, compaction, recall, and downstream model cost without replacing existing memory authorities.
   z0 capability router, legal-action compiler, Jev/Laya/specialists/routines	Select only eligible mechanisms; emit the applied decision and policy revision. Small or confident does not mean eligible or faster.
   kerdoios; provider-router/inference surfaces	Residual provider/resource placement using measured load/quota/latency. Do not introduce a competing allocator inside Relay.
   Hermes, OMP; later DSH, Agent Orchestrator, AgentWeb	Execute through owned seams. DSH uses its session-telemetry/plugin boundary; other harnesses adopt the semantic join rather than Hermes-specific code.
   agenttrace	Cross-source normalization, lineage/fidelity, diagnosis, and comparison presentation.
   tokenomics	Usage, estimates, signed treatment deltas, completeness, and outcome economics. Observer data does not become provider billing truth.
   z0evals; evolution-lab; Hermes self-evolution research	Freeze reproducible studies, compare candidates, produce promotion/rejection evidence. z0 runtime consumes eligible artifacts without depending on the lab during a turn.
   AODL/Bend and Keel/Mesh/privilege-broker companions	Existing authority, proof, transport, and rollback boundaries remain intact. A proof receipt has only its stated scope; telemetry grants no permission.
   Flow, Ripple, Stage Manager, company-os, voice/spatial/history UIs	Optional producers or read-only consumers later; not prerequisites or new controllers for the first slice.
   frontier-kb, verified-OSS-loop and other research/protocol surfaces	External research or publication/workflow evidence only where relevant. Private raw traces stay out of public research repositories.

   Long-term loop

   native execution observations + z0 decision receipts + independent outcomes
       -> fidelity-aware episode/credit join
       -> frozen, grouped evaluation evidence
       -> controlled candidate experiments in Evolution Lab
       -> promoted rule/cache/retrieval recipe/specialist/policy artifact
       -> existing z0 eligibility/authority gate
       -> existing harness adapter applies the bounded choice
       -> observe drift, reject, or deoptimize
   

   Use the existing experiment fields (experiment_id, pair_id, task_snapshot_id, arm_id, treatment_hash, selection_policy, assignment_probability, replay_grade, verifier_class) rather than inventing another experiment registry.

   The initial learner can be a deterministic threshold, cache-validity rule, or even a ranked recommendation. No neural learner is required to establish value. The newer z0 README explicitly treats mechanism choice as evidence-driven, and does not establish a general Bend hot-path speed advantage.

   Reuse Relay's own candidate mechanisms where appropriate. NVIDIA already documents adaptive hints, cache planning, tool-parallelism behavior, and opt-in response caching in its Adaptive plugin. Evaluate these individually under the same frozen outcome/budget contract. One owner per tuning knob; do not let Relay and z0 independently rewrite routing, context, or scheduling policy. Version, provider support, validation, and activation must be proven on the installed build.

   What counts as a speed improvement

   Optimize end-to-end completion time subject to independently checked quality, authority, privacy, and resource budgets. Report success/failure/timeout rates alongside latency so faster failures cannot look like an improvement.

   Include router, retrieval, compaction, model loading, model/tool calls, retries, escalation, auxiliary work, and observation overhead when measured. Report time-to-first-output and task p50/p95 separately. GPU contention and provider-internal latency are not automatically visible from Relay; add host/provider measurements only when an experiment needs them.

   Do not sum nested or parallel spans as wall time. Parentage alone is not a complete dependency graph: critical-path claims need observed dependency/await evidence; otherwise report an estimate or unknown. Preserve cross-host clock uncertainty and uninstrumented residual time.

   Observational traces suggest experiments; they do not prove causal savings. Use grouped, paired trials with frozen work-item/environment/policy snapshots, controlled cache conditions, independent outcomes, and randomized/interleaved arms where appropriate. Record unsuccessful and inconclusive episodes. Report signed deltas, including regressions; a positive-only savings helper is not a treatment-effect estimator.

   NVIDIA's ATOF specification explicitly separates event observation from metric aggregation and replay execution. Replay requires an isolated environment or recorded tool fixtures; never re-run world-mutating tools merely to manufacture a counterfactual. A held-out verifier must not be rewritten by the optimizing system.

   Rollout and exit gates

   P0 — establish the running baseline. Record exact harness refs, Relay wheel/schema, enabled plugins, privacy policy, and z0 policy revision. Capture one real sanitized closed turn per harness. Source presence alone does not pass this gate.

   P1 — passive join. One fixture per harness links state/decision → actual execution → usage → independently supplied outcome. No learning or behavior-changing middleware is enabled.

   P2 — one controlled optimization. Choose the largest measured, addressable bottleneck. A suitable first candidate is bypassing context filtering below a bounded input threshold, but only when the baseline shows filtering overhead matters and existing authority/privacy filters remain unchanged. Compare against the existing policy with independent quality checks before expanding to provider choice, compaction, or fan-out.

   P3 — guarded reuse. Promote the smallest winning artifact with an explicit validity region, version pin, expiry/invalidation, canary exposure, rollback, and drift checks. Preserve abstention and negative evidence. No self-promotion from trace completion.

   P4 — broader optimization. Add interaction/factorial studies only when single-variable experiments leave actionable uncertainty. Add DSH/other harnesses through existing seams; add resource sampling and native Relay adaptive candidates only for demonstrated gaps.

   Acceptance and stop conditions

   • Every supported synthetic fixture has exactly the expected identity joins, including parallel siblings and delayed outcomes; ambiguous joins are never silently accepted.
   • Duplicate exports and re-imports do not inflate usage or episode counts. Crash, cancellation, replay, schema mismatch, and missing-parent cases retain explicit coverage/fidelity states.
   • Passive observation on/off preserves fixture requests, tool inputs/results, persisted state, authority decisions, and verifier outcomes.
   • Payload/privacy fixtures show no secrets, prompts, tool bodies, or identifying paths in default exported records. Sanitization of observation must not mutate the real execution request.
   • Collection uses bounded resources; exporter failures are visible but do not grant authority, trigger another execution, or silently alter the outcome.
   • Report hook overhead, end-to-end overhead, event loss, and memory growth on pinned hardware. Before canary promotion, preregister a workload-specific overhead budget and demonstrate it; no unmeasured “zero overhead” claim.
   • Execution completion, model confidence, reference agreement, and AODL admission never become independent verified success.
   • The first comparison reports all attempted work, signed cost/latency effects, quality results, sample size, and uncertainty, with a frozen manifest in z0evals.
   • No new Relay dependency is required in z0int core or OMP; no runtime enters z0; no optimizer enters AgentTrace; no production turn waits for Evolution Lab.

   Stop or reduce scope if an adapter duplicates native lifecycle instrumentation, forces rich payload retention, cannot preserve identity through concurrency, or costs more than the measured optimization can plausibly recover. The first milestone is a truthful joined episode and report—not an automatically self-improving whole-stack deployment.
3. kvnloo commented on Oct 5, 2026

   

   kvnloo

   on Oct 5, 2026 – with ChatGPT Codex Connector

   OwnerAuthor

   More actions

   Shared-memory ownership is now consolidated in #22, with #66 for runtime identity/scope/snapshot enforcement. Reuse this issue's cross-harness decision/episode identities and outcome joins for the handoff proof; do not introduce a second memory receipt system. Keep capture, retrieval, model-visible injection and independently verified use distinct. Wiring on an integration branch or a captured event does not establish a working adapter on every host.
4. kvnloo commented on Oct 9, 2026

   

   kvnloo

   on Oct 9, 2026

   OwnerAuthor

   More actions

   Hermes cross-harness identity checkpoint — z0-multiharness-20261009

   Downstream capture API and tests published at c71b4e27883ae957fa64ed75364a7e7872a05672 (base 4c41fd5b148fb04e63b1f721ccf3ef564aa78330); 200 focused tests passed in the parent rerun. Credit: Kevin Rajan's requirements/invariants, existing EventLog/scoped-claim and Hermes persistence authors, downstream adapter/tests by Hermes Agent.

   Actual persisted native user inputs produce canonical source references and claim events 536/538; D2 explicitly supersedes D1 within the synthetic controller's workstream scope. Both original events and attribution remain. Two fresh-process replays returned those same event IDs, with exactly two independent native source events. The host adapter admits only the bound, active native user row after identity/hash/schema checks; no model-facing memory-write tool was added.

   Fresh Hermes consumer proof separates the serialized provider request from the real completion. It shows corrected-context injection and response-level use, not verified implementation success. Existing normalize_envelope, close_observation and join_outcome remain unchanged: execution completion and a reported success do not override a failing explicit verifier. Approval text in passive observations adds no authority.

   Private evidence key: multiharness-20261009/hermes/; canonical readiness event 539. Claude/OMP/Codex/final-Hermes outcome joins remain unqualified; retain failure/unknown rather than reporting the complete loop as successful.
5. kvnloo commented on Oct 9, 2026

   

   kvnloo

   on Oct 9, 2026

   Last edited by kvnloo

   OwnerAuthor

   More actions

   Hermes native handoff checkpoint — z0-multiharness-20261009

   A fresh, real Claude Code session consumed the Hermes-captured correction using the Claude-owned adapter at 0b58f42105bb970815af392abf6d453b22bf323c. Its only native task input was the workstream ID plus the existing receipt location. Native CLI UserPromptSubmit hook evidence contains the exact current claim/fresh fixture identity and marks the earlier value replaced. The real Claude response selected the corrected production-reuse strategy and recognized the superseded approach. Native input persistence and behavioral use are checked separately.

   Immutable references: claims 536/538, initial ready 539, native Claude consumer ready event 547; consumer receipt SHA-256 3632fe9034329be302d1fee471d10c3cd42dfca6ae9db5700880b5dde297f905; native stream SHA-256 ecdd5c090351062556496d92ed68dc1c39b7a98ff9053179218e07461b2f7889. Private evidence key: multiharness-20261009/hermes/claude-correction-use.json. These two tracker copies describe ONE measured native session, not independent repetitions.

   No OMP artifact was written by this Claude session. Full qualification remains false: next is Claude-supervised unsteered OMP through the owner's ordinary per-run approval path, fresh native Codex independent exact-artifact verification, then Hermes verified-outcome resumption. Codex's archival artifact replay and deliberately incorrect artifact failure control are explicitly not a fresh corrected-state execution. Four Hermes same-fixture memory controls passed; live approval-dispatch and same-new-artifact verifier-failure gates remain unqualified.

   The earlier local Claude authentication blocker was resolved using the user's existing claude-home configuration. Only its valid access token was borrowed read-only into an isolated native proof home; no refresh grant or existing configuration/history was copied or changed. The existing public capture remains c71b4e27883ae957fa64ed75364a7e7872a05672 (200 focused tests previously executed); this checkpoint introduces no further source change and takes over no peer-owned adapter/launcher work.

   Fresh-launch blocker audit (canonical owner delivery event 565): parent independently executed only the frozen runtime_env functions and existing production request normalizer, without launcher preparation/live execution or inference. The launcher strips an exported scoped-memory environment and assigns a random omp-live-* task; adding scope alone is rejected by the correct production stable-workstream guard. Matching-task control is accepted. This is a missing launcher-host binding, not an established shared-core defect.

   Registered wrapper digest is 801975c522296a1eb8bc2dbd7c8bb28384b0aec3c8603d99e1ed5d5abfd51b88. It still selects historical source 1f8ec56dc6189d8735f1939792049f10fe00067d and a registry different from the frozen integration manifest at 4ac01fc18f80be6a741be6fd583906e68160be28. Owner checkout has since advanced; its pin checks must not be weakened. Recent peer preparations 548/562 use unrelated local/random-run scopes and contain neither current D2 claim identity nor its fresh nonce, so they are not evidence of this correction handoff.

   Private reproduction key: multiharness-20261009/hermes/orchestration-readonly/parent-verification.json; script SHA-256 d027717deb2e3386084bf1044dfd1bb3f5a829a42cc07fc5ca9e3cc0d5f03851. Codex owns the fresh integration/workstream launcher binding; Claude owns OMP supervision. After owner publication, retain the ordinary native per-run approval with full-content/digest review. No historical approval transfers. No peer files, provider defaults, live sessions or gate policies were changed. The native Codex account/model restriction remains a separate owner-recorded gate.
6. kvnloo

   added a commit that references this issue on Oct 25, 2026

   Integration: one stamped live service and shadow source (#219)

   ...

   9db4aab