Return accounting: in scope, and in what shape?

[kyle-sexton]

Boris (thread post 3): usage measures activity, not return; return = would you have spent the eng-effort anyway, and what would manual eng-hours have cost. Nothing in the fleet tracks this. Decide: in scope? Shape: convention inside verification:measure vs new capability vs skip. Evidence: gap-map.md #11.

---

Reference: Boris Cherny, "Steps of AI Adoption" (Google Doc)

[kyle-sexton]

🔒 claim: session-61a34b03

[kyle-sexton]

Resolved: return accounting is IN SCOPE for 2→3, shaped as a lightweight convention — no new capability, no new cost.

Q1 — scope: in scope. Boris names the metric verbatim (thread post 3): usage measures activity, not return. 2→3 is exactly where activity and return diverge (more autonomous tasks = more usage regardless of value); scaling without a return signal is the step-3 trap in economic form. Deferral rejected: the dataset cannot be captured retroactively — capture must start before the data is needed.

Q2 — shape: convention at the task boundary (work-item close / PR merge), tracker-resident. Three-layer data model:

1. Machine/deterministic: automation cost (tokens/$, wall time — existing session telemetry) + lifecycle metadata (tracker timestamps, open→close durations, entity lifecycles — definitively calculable from tracker metadata/exports).
2. Human-attested: (a) counterfactual — would you have spent the engineering effort anyway: yes/no/partial; (b) manual-effort band (<1h / 1–4h / 1–2d / 1w+) — bands, not point estimates, because point estimates fail for humans and models alike.
3. Agent/LLM role: prompts for layer 2 at the task boundary and analyzes/aggregates over layers 1+2. NEVER estimates the return fields itself — models are demonstrably poor at effort estimation. This constraint carries a revisit trigger (models proven capable at effort estimation), not a permanent ban.

Aggregation/reporting transport deferred to #247 — whatever sink it picks becomes the join point between the cost half and the value half.

Rejected shapes: new family inside the measurable-delta verification capability (contract misfit — counterfactual has no baseline-before-change); standalone capability now (heavier than the data justifies; graduates later if the convention proves capture happens); fold into #247 (conflates human capture with machine transport).

Q3 — capture scope: autonomous-class work only (classes per the #243 guardrail matrix); interactive sessions exempt — per-task prompting friction kills capture compliance, and the activity-vs-return divergence lives where no human is in the loop. Expansion trigger: post-#247 aggregate reporting shows spend concentrating in interactive work. Optional precision graduation: a per-class matrix column, only if needed.

Boris-alignment verified before locking (session captures, gap-map §Position + item 11): the two human fields are his two questions verbatim; the usage/return distinction is preserved (telemetry alone is never presented as return); convention-before-capability honors trust-before-scale; the step-1 OTel-into-existing-stack guardrail is untouched. Vocabulary is contract-only (work-item tracker, task boundary, autonomous-class) — no vendor or instance names.

[kyle-sexton]

Source anchors (added for durability — the design threads re-anchor to this artifact):

• Boris Cherny, "Step & your role" (Jul 16, 2026) — Google Doc (reliable copy): https://docs.google.com/document/d/1R91ayvj7uvlxgNi--__2-Bf3w8x5r1nF-xIBN7ds8Ns/edit
• Original claude.ai artifact (intermittently erroring as of 2026-07-17): https://claude.ai/code/artifact/bfdfaef9-bc62-4dfe-ba9e-c58a26c9accf