Skip to content

Agent Observer core: ledger, Codex and Claude adapters, detectors, privacy module - #10

Merged
lukemaj merged 16 commits into
integration/mac-observerfrom
task/1-codex-timeline
Sep 23, 2026
Merged

lukemaj merged 16 commits into
integration/mac-observerfrom
task/1-codex-timeline

Conversation

@lukemaj

@lukemaj lukemaj commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Component PR into integration/mac-observer: Agent Observer's core. The final PR from integration/mac-observer to main follows once the OpenCode, Grok and Model Router adapters land here.

What it adds

  • Schema 2 ledger keyed by session with harness-prefixed natural keys; incremental JSONL sources (offset and tail hash); instruction identity from the AgentsMD direction block, versioned plugin paths and loaded-instruction hashes.
  • Codex and Claude Code adapters with counter semantics per harness: streamed Claude blocks finalize once per message, the Codex token_count fallback reconciles with authoritative usage, integer-only usage buckets, conflicts quarantined.
  • Detectors (repeated reads, skills, commands and failures, test edits after a failure, permission seeking, human corrections, large tool output) with counterexamples; compare --by agentsmd|model|harness|project with per-response model attribution and explicit mixed and unknown groups; session visibility; one marker-owned GitHub summary comment published only on explicit command.
  • agent_observer/privacy.py: one implementation of the ledger privacy rules for every writer. Excerpts only of genuine main-session human input, cut at the first tag-like marker; a privacy version per source that re-imports and corrects older rows; closed error categories; structure-only error excerpts; typed per-family event allowlists (native tool and skill names as identifiers, closed lifecycle names, HTTP codes). docs/contracts.md rules 6 and 7 state them. The ledger directory is 0700 and its files 0600.
  • Marketplace packaging (Codex, Claude Code and Grok manifests, bin/agent-observer, the agent-observer Skill), release and proof workflows, README.

Refs #1, #3, #4, #5, #8, #9.

Proof

Candidate 7694dbd: python3 -m unittest discover -s tests 152 tests OK.

Review

Independent Luna reviews over six rounds (afe9d64, 8f1c385, 76737c7, 96c5215, then 96c5215..7694dbd) drove corrections through Model Router jobs ao-core-a-1, ao-core-b-1, ao-core-a-2, ao-core-b-2, ao-core-4, ao-core-5 and ao-core-6. The last review of 96c5215..7694dbd: APPROVE, no findings.

🤖 Generated with Claude Code

lukemaj and others added 16 commits September 18, 2026 17:12
…y, Codex refit (#1)

Schema 2 keys every row by session with harness-prefixed natural keys, so
re-reads, grown logs and copied files never add usage twice. Append-only
logs resume at their last complete line. Sessions record project, branch,
client and AgentsMD identity from the direction block, versioned plugin
paths, or embedded instruction text matched against the release map;
only hashes are kept. The Codex adapter handles every item type seen in
1,841 local rollouts with zero quarantined records.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ruction identity (#2)

Counts each response once per message id, separates typed and mid-turn
prompts from commands, scaffolding, interrupts and task notifications,
records reads, skill loads, Skill calls, denials, compactions and final
assistant text, and links subagents to their parent session. Identity
comes from the clean direction block (the escaped copy in hook stdout is
ignored) or from loaded instructions matched to a release hash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lback (#3, #8)

Diagnose finds repeated reads and skill loads (broken by edits, compactions
and different ranges), repeated commands and failures, test files edited
between a failing and a passing test run, permission-seeking questions,
corrections, interrupts and denials, and large tool outputs, each with
event references. Compare groups sessions by AgentsMD version, model,
harness or project with sample sizes, excluding subagents. Rollouts
without token_usage_record now count usage from rising token_count totals.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
health compares Claude Code's running-session registry with transcripts
and checks live OpenCode and Codex CLI processes against their stores,
naming each invisible session with its fix. publish renders one aggregate
summary offline and creates or updates a single marker-owned comment on
a PR or commit through the existing GitHub CLI login.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

MIT license, Codex, Claude Code and Grok Build manifests mirrored by the
version policy, a project record for Toolybara, a launcher that runs the
bundled CLI from any directory, a Skill telling agents when and how to use
Observer, and proof and release workflows adapted from Model Router.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r-response model comparison and safe publishing (#1, #4, #5)

Addresses core review findings 4, 5, 8, 9, 10 (report and CLI), 11 and 12
through Model Router job ao-core-b-1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ons; excerpts only for genuine input (#1)

Addresses core review findings 1, 2, 3, 6, 7 and 10 (adapters) through
Model Router job ao-core-a-1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l group, paginated publish lookup (#1, #4, #5)

Addresses core delta review findings 1, 5, 7, 8 and 9 through Model Router
job ao-core-b-2, including three test_accounting assertions that encoded the
old id-in-error behavior.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…integer-only usage buckets, Codex fallback conflicts (#1)

Addresses core delta review findings 2, 3, 4 and 6 through Model Router job
ao-core-a-2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	tests/test_accounting.py
agent_observer/privacy.py implements the ledger privacy rules: excerpts cut
at the first tag-like marker, a privacy version per source that re-imports
and corrects older rows, closed error categories, structure-only error
excerpts and typed per-family event allowlists. Codex and Claude writers and
ingest route through it; native free-text titles are no longer stored.

Implemented through Model Router job ao-core-4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, NULL-safe error dedup (#1)

Addresses the review of 96c5215 through Model Router job ao-core-5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es allowed, free text still fails closed (#1)

Implemented through Model Router job ao-core-6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lukemaj
lukemaj merged commit f10085b into integration/mac-observer Sep 23, 2026
1 check passed
@lukemaj
lukemaj deleted the task/1-codex-timeline branch September 23, 2026 23:43
lukemaj added a commit that referenced this pull request Sep 24, 2026
…ode, OpenCode, Grok Build and Model Router

One private SQLite ledger of the records these tools already keep on this
Mac: usage per response with each harness's own counter meaning, timelines,
behavior detectors, comparison by AgentsMD version, model, harness and
project, session visibility, and one marker-owned GitHub summary on explicit
command. agent_observer/privacy.py enforces the ledger privacy rules for
every writer. Delivered through component PRs #10 to #19, each reviewed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lukemaj lukemaj mentioned this pull request Sep 24, 2026
lukemaj added a commit that referenced this pull request Sep 24, 2026
One private SQLite ledger of the records these tools already keep on this
Mac: usage per response with each harness's own counter meaning, timelines,
behavior detectors, comparison by AgentsMD version, model, harness and
project, session visibility, and one marker-owned GitHub summary on explicit
command. agent_observer/privacy.py enforces the ledger privacy rules for
every writer. Delivered through component PRs #10 to #19, each reviewed.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant