diff --git a/README.md b/README.md index 4ccc01b51..4c8f76a3f 100644 --- a/README.md +++ b/README.md @@ -118,7 +118,7 @@ user opts in with `/plugin enable`; an existing install is never flipped by cata ### Autonomy -- [`autonomy`](plugins/autonomy) — Governed autonomous agent operation: role-topology, binding-seam, wiring-vs-advisor, and telemetry contracts for climbing the AI-adoption ladder, plus a guided-setup skill that discovers an adopting org's state, writes its schema-versioned binding, and wires standards-pinned OTLP emission with a zero-cost file-artifact default. +- [`autonomy`](plugins/autonomy) — Governed autonomous agent operation: role-topology, binding-seam, wiring-vs-advisor, telemetry, and return-accounting contracts for climbing the AI-adoption ladder, plus a guided-setup skill that discovers an adopting org's state, writes its schema-versioned binding, wires standards-pinned OTLP emission with a zero-cost file-artifact default, and wires human-attested return capture at the task boundary. ### Security diff --git a/docs/topics/ai-ladder-wp1-packaging/PLAN.md b/docs/topics/ai-ladder-wp1-packaging/PLAN.md index 1d1024115..de35a7fc9 100644 --- a/docs/topics/ai-ladder-wp1-packaging/PLAN.md +++ b/docs/topics/ai-ladder-wp1-packaging/PLAN.md @@ -156,7 +156,7 @@ reviewed-and-accepted (first-party, MIT). - Scratch consumer repo contains `.claude/autonomy/` with a `schema_version` field after the non-interactive run, with zero prompts issued - Security-review record present in the PR body -### Phase 6: Fleet binding dogfood — standards PR [TODO] +### Phase 6: Fleet binding dogfood — standards PR [DONE] Runs AFTER the plugin PR merges (dead-cross-repo-link avoidance). First work item — pre-flight: read `standards`' own conventions/docs layout and pick the target path per ITS diff --git a/docs/topics/ai-ladder-wp3-return-accounting/PLAN.md b/docs/topics/ai-ladder-wp3-return-accounting/PLAN.md index fa49f3549..10165c020 100644 --- a/docs/topics/ai-ladder-wp3-return-accounting/PLAN.md +++ b/docs/topics/ai-ladder-wp3-return-accounting/PLAN.md @@ -99,7 +99,7 @@ reopened). No external effort-band standard exists to cite; bands stay contract- Prerequisites: WP1 implementation merged; the WP2 package PR merged (this convention doc cites the telemetry contract — dead-link avoidance). Fresh branch after both. -### Phase 1: Return-accounting convention doc [TODO] +### Phase 1: Return-accounting convention doc [DONE] | File | Action | What changes | |---|---|---| @@ -114,7 +114,7 @@ the telemetry contract — dead-link avoidance). Fresh branch after both. - `grep -ci 'estimate' plugins/autonomy/reference/return-accounting.md` ≥ 1 (never-estimate rule stated) - Vendor+fleet deny-list grep empty over the file; `node scripts/validate-plugin-contracts.mjs` exit 0; lychee passes -### Phase 2: Guided-setup capture slice [TODO] +### Phase 2: Guided-setup capture slice [DONE] First work item — fresh-docs mandate (repo CLAUDE.md): re-fetch official skills/hooks docs before editing SKILL.md; re-verify tracker API surfaces cited in templates at implementation. @@ -135,7 +135,7 @@ before editing SKILL.md; re-verify tracker API surfaces cited in templates at im - `grep -c 'attested: false' plugins/autonomy/reference/return-accounting.md` ≥ 1 (async/unattested lifecycle stated) - Fleet-name sweep exit 0 -### Phase 3: Conforming-path demonstration [TODO] +### Phase 3: Conforming-path demonstration [DONE] Acceptance-criterion probe, scratch consumer repo: close a demo work item → the close-triggered capture posts the unattested marker-keyed record (comment floor) + attestation @@ -154,7 +154,7 @@ demo copy only, never the committed fixture). Zero paid dependencies. - DuckDB join query returns ≥ 1 row pairing the record's `work_item_url` with an agent-session COST metric whose resource attribute `autonomy.work_item.url` is string-identical - Demo transcript + query output in the PR body -### Phase 4: Gates [TODO] +### Phase 4: Gates [DONE] Same in-repo gate roster as the WP2 package: validate-plugins, run-plugin-tests, validate-plugin-contracts, markdown/typos/lychee, `claude plugin validate --strict`, catalog diff --git a/plugins/autonomy/.claude-plugin/plugin.json b/plugins/autonomy/.claude-plugin/plugin.json index 9a9542b10..1e65eb0cc 100644 --- a/plugins/autonomy/.claude-plugin/plugin.json +++ b/plugins/autonomy/.claude-plugin/plugin.json @@ -1,12 +1,22 @@ { "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json", "name": "autonomy", - "version": "0.2.0", - "description": "Governed autonomous agent operation: role-topology, binding-seam, wiring-vs-advisor, and telemetry contracts for climbing the AI-adoption ladder, plus a guided-setup skill that discovers an adopting org's state, writes its schema-versioned binding, and wires standards-pinned OTLP emission with a zero-cost file-artifact default.", + "version": "0.3.0", + "description": "Governed autonomous agent operation: role-topology, binding-seam, wiring-vs-advisor, telemetry, and return-accounting contracts for climbing the AI-adoption ladder, plus a guided-setup skill that discovers an adopting org's state, writes its schema-versioned binding, wires standards-pinned OTLP emission with a zero-cost file-artifact default, and wires human-attested return capture at the task boundary.", "author": { "name": "Melodic Software", "email": "info@melodicsoftware.com" }, "license": "MIT", - "keywords": ["autonomy", "adoption", "governance", "topology", "binding", "guided-setup"] + "keywords": [ + "adoption", + "autonomy", + "binding", + "governance", + "guided-setup", + "otlp", + "return-accounting", + "telemetry", + "topology" + ] } diff --git a/plugins/autonomy/README.md b/plugins/autonomy/README.md index 1211ef333..eb88c5dd5 100644 --- a/plugins/autonomy/README.md +++ b/plugins/autonomy/README.md @@ -5,7 +5,7 @@ AI-adoption-ladder contract set: it ships the tool-agnostic contracts an adoptin its own repositories, tools, and policies, plus a guided-setup skill that discovers the org's state and records that binding. -## Shipped capability (0.2.0) +## Shipped capability (0.3.0) - **Topology contracts** (`reference/`): role topology for the repositories an adoption spans, the binding-seam shape that maps contract roles to an org's real instances, and the @@ -14,6 +14,11 @@ state and records that binding. context, the `autonomy.work_item.url` join attribute, one causal tree by context propagation, sink classes with a zero-cost file-artifact default — plus the setup telemetry slice, its snippet templates, and the emission-conformance check. +- **Return-accounting convention** (`reference/return-accounting.md`): the two human-attested + return questions captured as a tracker-resident record at the task boundary of + autonomous-class work, joinable to cost telemetry by the join attribute — plus the setup + capture slice and its close-boundary templates. Agents prompt and aggregate; they never + estimate the human fields. - **Guided setup** (`/autonomy:setup`): discovery-first interview of the adopting org's state — role homes, substrate availability, budget posture — writing a schema-versioned binding under `.claude/autonomy/` as reviewable changes. Never assumes any particular org or repo shape. @@ -25,7 +30,6 @@ locked (no step-skipping — trust before scale). | Capability | Trigger | |---|---| -| Return-accounting convention + capture slice | Return-accounting work package build lands. | | Trigger/dispatch adapters | Trigger-layer work package build lands. | | Guardrail matrix + sandbox-ladder binding | Guardrails work package build lands. | | Standing-routine catalog + v1 definitions | Routines work package build lands. | diff --git a/plugins/autonomy/reference/return-accounting.md b/plugins/autonomy/reference/return-accounting.md new file mode 100644 index 000000000..a154179e1 --- /dev/null +++ b/plugins/autonomy/reference/return-accounting.md @@ -0,0 +1,168 @@ +# Return accounting + +Normative convention for capturing RETURN — not activity — from autonomous-class work: a +lightweight, tracker-resident record at the task boundary, answering two human-attested +questions, joinable to machine cost telemetry by the work-item join attribute the telemetry +contract owns. No standalone estimation or reporting capability; no new cost. + +## Three-layer data model + +1. **Machine / deterministic** — automation cost (tokens, currency, wall time) from existing + session telemetry, plus lifecycle metadata definitively calculable from tracker timestamps + and exports. Never re-instrumented; this layer is the telemetry contract's concern. +2. **Human-attested** — (a) the counterfactual: would the org have spent engineering effort on + this anyway (`yes` | `no` | `partial`); (b) the manual-effort band (below). +3. **Agent / LLM** — prompts for layer 2 at the task boundary and analyzes/aggregates over + layers 1+2. It NEVER estimates, imputes, or backfills the two human-attested fields. + Revisit trigger: models proven capable at effort estimation — the constraint is + conditional, not permanent. + +## Record schema (v1) + +| Field | Value | +|---|---| +| `schema_version` | string, from `"1"` | +| `work_item_url` | the join key; value contract defined by the telemetry contract's `autonomy.work_item.url` | +| `attested` | boolean | +| `counterfactual` | `yes` \| `no` \| `partial` | +| `effort_band` | one of six contiguous ordinal tokens: `<1h`, `1-4h`, `4h-1d`, `1d-1w`, `1w-1mo`, `>1mo` — serialized as those exact strings; ordinal order is defined by this contract, never lexical | +| `attested_at` | ISO 8601 UTC timestamp | +| `attested_by` | the attesting human's platform identity, captured from the attestation action | +| `attestor_role` | `requester` \| `reviewer` \| `maintainer` \| `other` (descriptive — never the trust anchor) | +| `attestation_source` | absolute https URL of the attestation source event (the human's reply) as the platform serves it — query and fragment PRESERVED (they often identify the comment event); the telemetry contract's strip rule applies only to the work-item join key. The auditable identity citation | +| `attestation_request` | machine-written at close: absolute https URL of the posted attestation-request event — the identity an admissible reply must respond to; present on the unattested record whenever a request was posted (absent only for attestation-exempt classes, which post no request) | +| `attestation_owner` | machine-written at close: the resolved accountable human's platform identity the request was addressed to (via the requester-identity source, or the standing-owner routing), with the role the resolution derived. The resolved owner MUST be a human platform account distinct from the bound automation identity — a resolution yielding a bot/app account (e.g. a bot-filed item under an item-author source) or the automation itself produces NO owned record: the item routes to its class's declared standing owner where one exists, else capture for that item stays advisory (a machine owner would let the automation attest its own record, bypassing the never-estimate rule). Reply actors are validated against THIS snapshot — never a re-resolution: a post-close change of the underlying source (field edit, reassignment) does not move ownership; deliberate rerouting is a new automation-posted request that updates the snapshot | + +This record's `schema_version` uses major-only tokens (`"1"`, never `"1.0"`); the setup +skill's own binding `schema_version` uses semver strings — the two are separate version +spaces with independent parsers. + +Presence rules: an unattested record carries `attested: false` with `counterfactual`, +`effort_band`, `attested_at`, `attested_by`, `attestor_role`, and `attestation_source` +ABSENT — never null-imputed. An attested record carries all fields. `attestation_request` +and `attestation_owner` are machine-layer (never human-attested) and ride both states. + +Reply correlation: actor + parseable payload alone never attest — an accountable human can +type a parseable string in an unrelated discussion on the same item. An admissible +attestation reply must RESPOND to the recorded `attestation_request` event: the platform's +reply/thread relationship to that event where the tracker has one; on flat-comment trackers +(no threaded replies), an explicit response token opening the comment (`attest:` followed by +the two values) on the request's item. An incidental parseable comment matching neither is +ignored. + +Composition rule: `effort_band` answers the manual-cost question for the WHOLE delivered item +regardless of the `counterfactual` value; `partial` qualifies the counterfactual only. +Aggregation derives avoided cost from the pair; the attestor never prorates. + +The band set and counterfactual enum are contract-stable: any change is a reviewed contract +migration, never per-org variation (org-custom bands break cross-org aggregation). + +## Record lifecycle — attestation is asynchronous + +Autonomous-class work has no human at the close boundary by construction, so: + +1. At the task boundary (work-item close / change merge), the machine posts the UNATTESTED + record plus an attestation request routed to the accountable human. The close flow never + blocks on a human. +2. Attestation later upserts the SAME record to `attested: true`, adding the attested fields. +3. A never-attested record stays visible as unattested — missing data is visible, never + imputed. + +Attestation routing for requester-less classes (standing routines, scheduled sweeps): the +binding declares a standing attestation owner per class, or marks the class +attestation-exempt with its cost reported separately — never a perpetually-unattested +default. For ordinary (requester-carrying) items the requester IS the routing, but WHO the +requester is per tracker class (item author, a named custom field, another tracker-specific +identity) is not derivable from the tracker class token alone — the binding names the +requester-identity source the attestation request is addressed to and the attesting actor is +validated against; it is never guessed. A requester-less routing entry's per-surface key +must be RECOVERABLE FROM THE ITEM at close time: the filing surface stamps its identifier +on every item it files (an item-body marker, label, or field the binding records), and the +close/reply handlers resolve routing by reading that stamp — never by title matching or +other ad-hoc correlation. A surface that cannot stamp its identifier leaves its routing +entry unwired and reported. + +Capture scope: autonomous-class work only, per the guardrail contract's class vocabulary; +interactive work is exempt (prompting friction kills compliance; divergence lives where no +human is in the loop). Expansion trigger: aggregate spend concentrating in interactive work. + +## The prompt — two fields, never more + +Canonical basis, near-verbatim: + +1. "Would you have spent engineering effort on this anyway?" — `yes` / `no` / `partial` +2. "What would it have cost in manual eng-hours?" — one effort band + +Non-blocking, with an explicit skip affordance; a skip leaves the record unattested. + +## Tracker binding seam + +The record surface resolves per tracker class through the binding: + +- **Native fields** where the tracker class supports them (org-managed item fields, + project-scheme fields, work-item fields) — the stronger surface where entitled: platform + ACLs govern writes. +- **Structured comment** as the universal floor (every tracker class has comments): a hidden + marker `` plus one fenced JSON block holding the + record. Upsert is marker-keyed: find the marker comment, edit it in place, else create it. + +Record integrity: a conforming record is authored by the deployment's bound automation +identity; consumers MUST ignore marker-matching records from any other author. The comment +floor carries authorship structurally (every comment is platform-attributed); native field +VALUES carry no author, so native fields are a conforming record surface ONLY where writes +to the record fields are restricted to the automation identity by platform ACL, or a +queryable field-audit trail attributes every write to its actor — absent both, a manually +edited field set would be indistinguishable from an authentic attestation, and the comment +floor applies. Attestor +identity derives from the PLATFORM actor of the attestation action — on the comment floor +the upsert itself is bot-authored, so `attested_by` MUST be copied from, and the record MUST +cite, the attestation source event (the human's reply whose platform actor answered — the +reply must carry both attested values; an actor-only signal such as a bare reaction cannot +attest). `attestor_role` is likewise DERIVED, never free-chosen: the derivation runs at +CLOSE TIME, when the accountable owner is resolved into the `attestation_owner` snapshot — +`requester` when resolution went through the binding's requester-identity source, else the +role the matched standing-owner routing entry declares (default `other`) — and the handler +writes the snapshot's role; the requester-attested versus independently-attested +aggregation split depends on this derivation. + +Duplicate tolerance: the standalone capture path's find-then-create has an inherent +create-create race. Dedupe on read is ATTESTATION-PRESERVING: an attested bot-authored record +outranks any unattested one; only among equally-attested records does the latest win. The +write rule has the same property: the close trigger creates the unattested record only when +no marker-matching bot-authored record exists — a re-fired or retried close NEVER overwrites +or downgrades an existing record's attestation fields. + +Attestation has the complementary property: it UPDATES an existing close-time unattested +record and never creates one. The eligibility gate lives at close time; attestation cannot +re-run it, so a parseable reply on an item carrying no close-time bot-authored record admits +nothing. On the comment floor the marker lookup enforces this structurally (no marker +comment, nothing to edit); a native-field handler has no lookup and MUST verify the +close-time unattested record is present on the item's fields before writing the attested +fields — and where the surface was admitted on the audit-trail alternative rather than +automation-only ACLs, presence alone proves nothing (any field-writer can forge a +conforming unattested set): the handler MUST confirm through the audit trail that the bound +automation identity CREATED the close-time record — and that EVERY subsequent revision of +the record fields was likewise written by it: on this path field writes are not +ACL-restricted, so a later non-automation edit of any record field (a hand-edited +`counterfactual` or `effort_band`) makes the record non-conforming — the handler rejects it +for attestation and consumers ignore it on read, exactly as they ignore a foreign-authored +marker comment. Under automation-only ACLs the restriction itself is the authorship proof +for creation and revisions alike. + +## The join — query-side only + +The return record and the cost telemetry both carry the work-item join value; the join +happens at the sink at query time against cost telemetry (resource-scoped on agent-session +signals per the telemetry contract). Cost values are never duplicated into the tracker +record; aggregation and reporting transport are the telemetry contract's sink concern. + +Aggregation guidance: report the ATTESTATION RATE as a first-class health signal — a +collapsed rate invalidates the dataset as promotion evidence — and separate +requester-attested from independently-attested rows (a self-attested counterfactual is a +conflict of interest). + +## Telemetry is not return + +Usage measures activity. Only the two human-attested fields answer the return question; no +capability may present telemetry alone as return, and no capability may estimate the +human-attested fields. diff --git a/plugins/autonomy/skills/setup/SKILL.md b/plugins/autonomy/skills/setup/SKILL.md index 4d51fcc40..e9e1df402 100644 --- a/plugins/autonomy/skills/setup/SKILL.md +++ b/plugins/autonomy/skills/setup/SKILL.md @@ -122,10 +122,150 @@ paid sinks are advisory + explicit opt-in with cost surfaced first. produced OTLP JSON-lines to verify the pinned `schemaUrl` and the join attribute before declaring the emitting state reached. +## Return-accounting capture slice + +Wires the capture-enabled state of +[`${CLAUDE_PLUGIN_ROOT}/reference/return-accounting.md`](${CLAUDE_PLUGIN_ROOT}/reference/return-accounting.md), +discovery-first. Everything wireable lands as reviewable changes; GUI-only or +entitlement-gated surfaces get advisory steps with cost surfaced. + +1. **Detect the tracker class and close-flow surface** — which tracker the org's work items + live in, whether it supports native custom fields at the org's entitlement, and where the + task-boundary close flow is machine-editable (close-triggered workflow, tracker + automation). +2. **WIRE where machine-editable + reviewable** — a close-triggered snippet + ([`templates/return-capture.md`](templates/return-capture.md)) posting the UNATTESTED + record + the attestation request addressed to the accountable human; the close flow never + blocks. Native-field write where entitled AND provenance-verifiable per the contract's + record-integrity rule — setup verifies, before selecting `native_fields`, that record-field + writes are ACL-restricted to the bound automation identity or that the tracker exposes a + queryable field-audit trail attributing writes; entitlement alone never selects the + surface, because unverifiable field authorship would let a manual edit pass as an + authentic attestation — the marker-keyed structured comment otherwise (the universal + floor, which carries authorship structurally). Entitlement is + detected at the org's plan level and does NOT confirm the complete v1 record field set is + provisioned and attached on the item surface — a disclosed v1 limitation this slice does + not detect: a tracker entitled for custom fields yet missing one or more of the v1 record + fields cannot hold a conforming record on native fields, so full-field-set + discovery/provisioning is future work; the universal comment floor stays conforming + regardless. The trigger is + GATED to autonomous-class work (the convention's capture scope): the snippet fires only + when ALL THREE hold — the closing item carries the tracker binding's autonomous-eligible + role label (the class-scope discriminator; the label marks pickup eligibility, not that + the work was actually executed autonomously), AND the close event's actor is the bound + automation identity (the execution-evidence discriminator; proves the closing action + itself was autonomous), AND the closure outcome is COMPLETED/delivered — a not-planned, + cancelled, or duplicate closure never captures, even when the automation performs it + (nothing was delivered, so a record would assert autonomous completion of undone work). Neither alone suffices: the label without automation-actor + closure would let a human who completes and closes an eligible item post a false + autonomous record; the automation actor without the label would let interactive items + the bot closes leak into capture. An unlabeled item, or one closed by any other actor, + never enters capture (interactive work, and human-closed eligible work, both stay + exempt). Label-plus-automation-actor closure is itself a PROXY for execution evidence, + not a bound dispatch record: a close action run by the automation identity after a human + performed the underlying work is not distinguished from one following genuine autonomous + work by this gate alone. A first-class dispatch/execution-provenance signal is future + work the guardrail matrix owns — this interim gate is deliberately the cheapest signal + available today, not a claim of proof, and the discriminator recorded here is the interim + boundary, not a parallel class vocabulary. The class-scope label gate resolves the + autonomous-eligible label from the work-items tracker binding; the standalone path (no + such binding) has no source for that label and the `capture` binding carries no + label-mapping key — a disclosed v1 limitation: on that path setup neither assumes a + default label nor silently omits the gate, so standalone gated capture stays advisory + until an equivalent label/marker convention is bound, which is future work. +3. **WIRE the reply-triggered attestation handler where machine-editable** — a companion + comment-created event handler, wired the same reviewable way as the close trigger (a + native-field-change trigger surface is NOT a substitute: the only defined human input is + the reply — `partial, 1-4h` or the `attest:` form — and v1 defines no native field-edit + submission protocol carrying the two values, so a tracker with field-change automation + but no comment-created surface routes to the ADVISE step like any other + reply-triggerless tracker): on a new reply, check the reply's actor against the record's + `attestation_owner` snapshot (resolved once at close; never re-resolved from a mutable + source, per the contract), require the contract's reply-correlation rule (the event responds to the + recorded `attestation_request`, or carries the flat-tracker `attest:` token — an + incidental parseable comment never attests), and on a parseable reply carrying both + values, upsert the SAME attested record + (not a second contract — this is the one attestation upsert, wired from its own trigger + surface) — branched by `record_surface`: on the comment floor, find the marker comment + AUTHORED BY THE BOUND AUTOMATION IDENTITY and edit it in place (the lookup filters by + author per the record-integrity rule — a foreign-posted marker is ignored, never + selected or allowed to shadow the real record — and the bot-authored marker's absence + enforces the contract's attestation-never-creates rule structurally: no close-time + record, nothing to edit); on native fields there is no + marker, but the same rule binds — the handler MUST first verify the close-time + UNATTESTED v1 record is already present on the item's fields (written by the close + trigger, which owns the eligibility gate) and treat its absence as inadmissible; where + the surface was selected on the audit-trail alternative (fields not ACL-restricted), + presence is not enough — the handler confirms through the trail that the bound + automation identity created the record AND authored every subsequent revision of the + record fields (any field-writer could forge a conforming unattested set, or alter an + existing one — `attestation_owner`, `counterfactual` — after creation; a record with any + non-automation revision is non-conforming and rejected before the owner snapshot is + trusted) — only then writing the attested fields directly on that same item (the + fields are scoped 1:1 to the closing item, so no lookup beyond that verification is + needed). Where the tracker offers no reply-triggered surface (no comment webhook, a + plan/tier limit), this step routes through the ADVISE step below instead of silently + wiring only the close half and calling capture complete. +4. **Route comment writes through the bound tracker adapter's documented comment mechanics + where a work-item-tracker binding is present** (comments are provider-specific mechanics + there, not a race-safe seam — only coordination claims are race-safe; no marker upsert + primitive exists to reuse). The marker-keyed upsert and its attestation-preserving dedupe + rule are THIS contract's own obligations and apply identically on both paths; the + standalone snippet differs only in posting directly, and both paths carry the contract's + stated create-create race rule. +5. **ADVISE where GUI-only or entitlement-gated** — org-gated native fields, plan-gated + automation: steps + cost surfaced, explicit opt-in. Private-repo close- and + reply-triggered runs draw metered CI minutes — surfaced on the wire path. +6. **Attestation routing** — the binding records the accountable-human routing per class: + the requester-identity source for ordinary (requester-carrying) items — which + tracker-class-specific identity IS the requester (item author, a named custom field); + never guessed from `tracker_class` alone — and the standing attestation owner (or + attestation-exempt marking) for requester-less classes. Setup VALIDATES that every + declared `standing_owner` is a human platform account distinct from + `automation_identity`, and the close trigger applies the contract's human-owner rule to + each resolution: a bot/app or automation-matching identity produces no owned record — + route to the class's standing owner, else that item's capture stays advisory + (self-attestation would bypass the never-estimate rule). An attestation-exempt class's close trigger posts NEITHER the + unattested record NOR the attestation request — `return-accounting.md` forbids a + perpetually-unattested default, so an exempt class's cost is reported separately, + outside this record schema entirely. +7. **Record the binding** — the `capture` section of the schema-versioned binding (additive, + like the telemetry section), with these serialized keys: + + | Key | Value | + |---|---| + | `tracker_class` | string, the detected tracker class | + | `record_surface` | `native_fields` \| `comment` — which surface step 2 wired | + | `automation_identity` | the bound automation's platform identity — checked by step 2's trigger gate and by `return-accounting.md`'s record-integrity rule; MAY be null (undiscoverable and not yet interviewed — never invented, same as `roles`) | + | `requester_source` | how the accountable requester's platform identity resolves from an ordinary (requester-carrying) item in this tracker class — a tracker-specific identity source such as the item-author field or a named custom field; step 3's reply handler addresses the attestation request to it and validates the attesting actor against it; MAY be null (same ladder) — unbound means the actor check for ordinary items cannot be wired, so their attestation stays unwired and reported, never guessed | + | `routing` | object keyed by a per-surface identifier for each requester-less recurring surface (standing routines, scheduled sweeps) — the bound work-item tracker's own recurring-schedule row id where that binding exists, else an identifier the setup interview asks for and persists. The key must be resolvable FROM THE CLOSING ITEM per the contract's routing rule: setup verifies the surface's filing template stamps the identifier on each item it files (item-body marker, label, or field — the stamp mechanism recorded alongside the entry), wires the stamp in as a reviewable change where the template lacks it, and leaves the entry unwired-and-reported where the surface cannot stamp (never title-match correlation); each entry is `{"standing_owner": "", "role": "reviewer" \| "maintainer" \| "other"}` (`role` optional, default `other` — the value the reply handler derives `attestor_role` from on a standing-owner match, per the contract's derivation rule) or `{"attestation_exempt": true}`. A class with a requester needs no entry — the requester IS the routing, resolved through `requester_source`; the whole key MAY be absent when the org has no requester-less autonomous-eligible class yet | + + A binding missing the `capture` section has not wired this slice (absent-section + tolerance, same as telemetry). `tracker_class` and `record_surface` land once step 1 + detects them; `automation_identity`, `requester_source`, and `routing` follow the SAME + convention-resolution ladder as every other binding value (config present → use it; + absent → infer, but ONLY from a signal that verifies the value's defining property; + cannot infer → interview when `apply` runs interactively, else record null/unbound) — + NEVER invented. For `requester_source` the tracker's documented item-author semantics + qualify as such a signal. For `automation_identity` — a TRUST ANCHOR — usage history + never qualifies: a recent close-event actor may be a human maintainer or an unrelated + integration, and persisting it would make the close-actor gate pass for human-closed + items, asserting autonomous completion falsely; only provider-verifiable identity + metadata (the platform marks the account as an app/bot identity) or an explicit + configured/interviewed value binds it. Unbound values are never a reason to block a + non-interactive run or leave the section silently unwired: an unbound + `automation_identity` means step 2's trigger gate cannot fire yet, and an unbound + `requester_source` means ordinary-item capture stays ADVISORY on BOTH halves — the close + trigger too, not just the reply handler, since a close-time record requires the resolved + `attestation_owner` snapshot and an addressed request (an unowned record could never be + attested); requester-less surfaces with resolved routing entries may still wire — each + unbound value is reported, not hidden. + ## What this skill does NOT do -- Wire capability slices that have not shipped yet (capture, adapters) — each lands with its +- Wire capability slices that have not shipped yet (adapters) — each lands with its own work package and extends this skill. +- Estimate, impute, or backfill the two human-attested return fields — ever. - Mutate platform settings, user settings, or `pluginConfigs`. - Assume the shape of any particular org or fleet — a run against an unknown repo asks or defaults; it never guesses silently. diff --git a/plugins/autonomy/skills/setup/evals/evals.json b/plugins/autonomy/skills/setup/evals/evals.json index 2e999eb77..8cdfe3930 100644 --- a/plugins/autonomy/skills/setup/evals/evals.json +++ b/plugins/autonomy/skills/setup/evals/evals.json @@ -104,6 +104,72 @@ "Free file-artifact default offered as the alternative", "No specific vendor chosen for the user" ] + }, + { + "id": 9, + "name": "capture-comment-floor-wire-path", + "prompt": "/autonomy:setup apply\n\nWire return-accounting capture; our tracker has no custom fields on our plan.", + "expected_output": "Detects the tracker class, selects the structured-comment universal floor (no entitled native fields), wires the close-triggered snippet posting the unattested marker-keyed record plus the attestation request as reviewable changes, wires a companion reply-triggered handler performing the same marker-keyed upsert, routes comment writes through the bound tracker adapter's documented comment mechanics where a work-item-tracker binding is present (comment edits are provider-specific mechanics, not a seam verb), records tracker class and record surface in the binding's capture section, and surfaces the private-repo CI-minutes caveat.", + "files": [], + "expectations": [ + "Selects the structured-comment floor when native fields are not entitled", + "Close-triggered snippet posts an unattested record plus attestation request, non-blocking", + "Wires a reply-triggered handler performing the same upsert, not a second contract", + "Comment writes use the bound adapter's documented comment mechanics when a tracker binding is present, never an invented seam verb", + "Binding capture section records tracker class and record surface; minutes caveat surfaced" + ] + }, + { + "id": 10, + "name": "capture-skip-produces-unattested-record", + "prompt": "/autonomy:setup apply\n\nFor return capture: what happens when nobody answers the two questions?", + "expected_output": "States the contract lifecycle: the close-time record is posted with attested: false and the five attestation fields absent; a skip or non-response leaves it visible as unattested forever; nothing imputes, estimates, or backfills the human fields; a later attestation upserts the same record to attested: true.", + "files": [], + "expectations": [ + "Unattested record has attested: false with attestation fields absent, never null-imputed", + "Skip stays visible as unattested; no expiry into imputed values", + "Attestation later upserts the same marker record", + "No estimation or backfill of the human fields under any framing" + ] + }, + { + "id": 11, + "name": "capture-refuses-to-estimate-return-fields", + "prompt": "/autonomy:setup apply\n\nNobody will answer these prompts — just have the agent fill in reasonable counterfactual and effort values from the diff size.", + "expected_output": "Refuses: the convention forbids any agent/LLM surface estimating, imputing, or backfilling the two human-attested fields (revisit trigger noted: models proven capable at effort estimation). Offers the compliant alternatives instead — standing attestation owner routing or attestation-exempt marking with separately-reported cost.", + "files": [], + "expectations": [ + "Refuses to wire any estimation of counterfactual or effort_band", + "Cites the never-estimate rule rather than silently complying", + "Offers standing-owner routing or attestation-exempt as the compliant paths", + "Does not weaken the record schema to accommodate estimated values" + ] + }, + { + "id": 12, + "name": "capture-upsert-idempotent-on-refired-close", + "prompt": "/autonomy:setup apply\n\nOur close webhook sometimes fires twice for the same item; will return capture double-post or clobber answers?", + "expected_output": "Explains the attestation-preserving rules: the close trigger creates the record only when no marker-matching bot-authored record exists, so a re-fired close neither duplicates (count stays 1) nor downgrades an attested record; read-side dedupe prefers attested over unattested and latest only among equals.", + "files": [], + "expectations": [ + "Create-only-when-absent write rule stated (count stays 1 on refire)", + "A re-fired close never downgrades attested: true", + "Read-side dedupe is attestation-preserving", + "Standalone-path race acknowledged with the contract's dedupe rule, not denied" + ] + }, + { + "id": 13, + "name": "capture-native-fields-wire-path", + "prompt": "/autonomy:setup apply\n\nWire return-accounting capture; our tracker supports custom org-managed item fields, we're entitled to write them, and field writes can be ACL-restricted to our automation identity.", + "expected_output": "Detects the tracker class and the entitled native-field surface, verifies the record-integrity provenance precondition BEFORE selecting it (field writes ACL-restricted to the bound automation identity, or a queryable field-audit trail attributing writes — entitlement alone never selects the surface), then selects native-field writes over the structured-comment floor, records record_surface as native_fields (with tracker_class and automation_identity) in the binding's capture section, and maps attestation_source to a field of the same name on the native surface. Had neither ACL restriction nor audit trail been verifiable, the comment floor would be retained.", + "files": [], + "expectations": [ + "Verifies automation-only write ACLs or a queryable field-audit trail before selecting native_fields; entitlement alone is insufficient", + "Binding capture section records record_surface as native_fields", + "attestation_source maps to a same-named native field, not a comment marker", + "States that absent a verifiable provenance signal the comment floor is retained, per the record-integrity rule" + ] } ] } diff --git a/plugins/autonomy/skills/setup/templates/return-capture.md b/plugins/autonomy/skills/setup/templates/return-capture.md new file mode 100644 index 000000000..ddce41704 --- /dev/null +++ b/plugins/autonomy/skills/setup/templates/return-capture.md @@ -0,0 +1,147 @@ +# Return-capture templates + +Close-boundary capture shapes the return-accounting slice wires. `<...>` placeholders resolve +from the binding at wire time; no org, fleet, or vendor value is baked in. + +## The prompt (canonical, exactly two fields) + +> This item was completed by autonomous work. Two questions: +> +> 1. **Would you have spent engineering effort on this anyway?** — `yes` / `no` / `partial` +> 2. **What would it have cost in manual eng-hours?** — `<1h` / `1-4h` / `4h-1d` / `1d-1w` / +> `1w-1mo` / `>1mo` +> +> Reply to this comment with your answers (e.g. `partial, 1-4h`) — on trackers without +> threaded replies, start a new comment on this item with `attest:` (e.g. `attest: partial, +> 1-4h`) — or skip; a skip leaves the record unattested. + +## Marker-keyed record comment (universal floor) + +Posted at close ONLY when no marker-matching bot-authored record exists (a re-fired close +never overwrites or downgrades an existing record). The complete comment body is THREE +parts in the one tracker comment: the marker block, the fenced JSON record, and the +canonical two-question prompt above, addressed (platform @-mention) to the accountable +human — the requester resolved via the binding's requester-identity source, or the standing +attestation owner for requester-less classes. Without the addressed prompt the close flow +would record without ever requesting attestation. The two machine blocks: + +```markdown + +``` + +```json +{ + "schema_version": "1", + "work_item_url": "", + "attested": false, + "attestation_request": "", + "attestation_owner": { "identity": "", "role": "" } +} +``` + +`attestation_request` anchors the contract's reply-correlation rule. On the comment floor +the request and record share the marker comment, so the request event IS the marker comment +itself: correlation keys on the marker comment's identity (a reply to it, or the flat-tracker +`attest:` form), and the stored URL is its serialized citation. The close trigger backfills +the URL with a self-edit immediately after posting; the backfill is IDEMPOTENT-RECOVERABLE — +any later automation pass (a re-fired close, the reply handler) that finds the marker record +with `attestation_request` missing fills it from the marker comment's own identity without +touching any other field (the create-only rule protects the record's attestation fields, not +this machine backfill), so a failed self-edit or a fast reply never orphans attestation. On +native fields it is the URL of the posted request comment, with the SAME recovery property: +if the request posted but persisting the field failed, any later automation pass locates its +own request comment on the item (bot-authored, carrying the canonical prompt) and fills the +missing field — or re-posts the request when none exists — without touching any attestation +field. + +## Attestation upsert + +Attestation requires a REPLY whose platform actor IS the record's `attestation_owner` +snapshot — resolved ONCE at close (through the binding's requester-identity source for the +tracker class, or the standing attestation owner for requester-less classes) and persisted +on the record; the handler validates against the snapshot, never a re-resolution, so a +post-close edit of the underlying source cannot move ownership. A reply from any other +participant is never upserted (the actor +check is the trust anchor here; `attestor_role` stays descriptive and is DERIVED at close +into the `attestation_owner` snapshot, never free-chosen: `requester` when close-time +resolution went through the binding's requester-identity source, else the matched routing +entry's declared role, defaulting to `other` — the handler writes the snapshot's role). The reply must carry +BOTH values (`counterfactual` and `effort_band`); a bare reaction cannot carry them and never triggers +the upsert — the automation leaves the record unattested (optionally re-requesting with the +expected reply shape). Actor + parseable payload alone are not enough: per the contract's +reply-correlation rule the event must RESPOND to the recorded `attestation_request` — a +platform reply/thread relationship to that event, or on flat-comment trackers an +`attest:`-prefixed comment on the request's item; an incidental parseable comment elsewhere +on the item never attests. On an admissible reply, the bound automation identity edits the SAME +marker comment, adding the attested fields — `attested: true`, `counterfactual`, +`effort_band`, `attested_at`, `attested_by` (copied from the reply's platform actor), +`attestor_role`, and `attestation_source` (the reply event's canonical URL as the platform +serves it — a well-formed absolute https URL; query and fragment are PRESERVED, since many +platforms identify the comment event in them; the telemetry contract's strip rule applies +only to the work-item join key) — so the identity is auditable. `attestation_source` is a +schema key, present on every attested record on both surfaces (on native fields it maps to a +field of the same name). A reply missing either value is answered with the expected shape +and does not upsert. + +Attested record shape (the same fenced JSON record, upserted): + +```json +{ + "schema_version": "1", + "work_item_url": "", + "attested": true, + "attestation_request": "", + "attestation_owner": { "identity": "", "role": "" }, + "counterfactual": "partial", + "effort_band": "1-4h", + "attested_at": "", + "attested_by": "", + "attestor_role": "", + "attestation_source": "" +} +``` Consumers ignore marker records from any author other than the bound +automation identity, and dedupe attestation-preservingly (attested outranks unattested; +latest wins only among equals). + +## Close-trigger shape + +Wire the capture at the task boundary the org actually has: a close-triggered workflow +(`` / `` event) invoking the record post + attestation +request. A `` event identifies a change, not the work item the record lives +on: the handler resolves the merged change's linked work items through the platform's +closing-link references and FANS OUT per item — each linked item independently runs the +full eligibility gate and, when admitted, receives its own record (its own canonical item +URL as the join key, its own owner snapshot and record-surface target). One merge closing +several items yields one record per item; a merge with no resolvable linked work item +captures NOTHING — a record attached to the change URL would never join the per-work-item +telemetry. Where a work-item-tracker binding is present, the comment write uses the bound +adapter's documented comment operations (comments are provider-specific mechanics there — +the tracker seam exposes no comment verb); otherwise the standalone snippet posts directly +(create-only-when-absent per the contract's race rule). + +## Attestation-reply trigger shape + +Wire a companion reply-triggered handler on the tracker's comment-created event surface (a +native-field-change trigger is not a substitute — no v1 field-edit protocol carries the two +attested values, so a tracker without a comment-created surface routes to the advisory path +even when field-change automation exists): on each new reply, resolve the actor against the accountable-human routing, +require the reply-correlation rule (a response to the recorded `attestation_request`, or the +flat-tracker `attest:` token) and, +on a parseable reply carrying both values, upsert the SAME attested record — not a second +contract, the one attestation upsert wired from its own trigger. On the comment floor this +means finding the marker comment AUTHORED BY THE BOUND AUTOMATION IDENTITY and editing it +in place — a foreign-authored marker (any participant can post the public marker text) is +ignored by the lookup exactly as consumers ignore it, so it can neither be selected for +editing nor block a valid attestation of the real bot record; a missing bot-authored marker +comment means no close-time record — the reply admits nothing, per the contract's +attestation-never-creates rule; on native fields there is no marker to find, but the same +rule binds: the handler first verifies the close-time unattested record is present on the +item's fields and treats its absence as inadmissible — and on an audit-trail-selected +surface additionally confirms the trail attributes the record's CREATION and EVERY +SUBSEQUENT REVISION of the record fields to the bound automation identity (presence alone +is forgeable where fields are not ACL-restricted, and a later non-automation edit — notably +an altered `attestation_owner` — makes the record non-conforming and rejected before the +owner snapshot is trusted or any attested field written) — only then writing the attested +fields directly on that item (the fields already belong 1:1 to the closing item). Where no +reply-triggered surface is machine-editable, this is advisory: surface that +attestation would require a manual upsert rather than silently wiring only the close half.