Skip to content

docs(#503): add AC19-AC21 live-Outlook verification runbook and orchestrator memory - #517

Merged
drmoisan merged 2 commits into
mainfrom
bug/ribbon-engine-readiness-guard-503
Aug 8, 2026
Merged

docs(#503): add AC19-AC21 live-Outlook verification runbook and orchestrator memory#517
drmoisan merged 2 commits into
mainfrom
bug/ribbon-engine-readiness-guard-503

Conversation

@drmoisan

@drmoisan drmoisan commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Summary

Documentation-only follow-up to PR #515 (issue #503). Two commits landed on the feature branch after #515 was merged and are not in main. No source, test, or project file is touched.

  • Adds the live-Outlook verification runbook for the three MANUAL-ONLY acceptance criteria AC19, AC20, and AC21, which is the artifact the autonomous-execution mandate requires for a permitted human-interaction exception.
  • Records three orchestrator agent-memory lessons from the Bug: ribbon-engine-readiness-guard #503 delivery.

Why

PR #515 delivered the engine-readiness guard and merged on green CI. Three of its thirty acceptance criteria cannot be satisfied by automated tests: they require a live Outlook desktop process with a real mail profile, which the repository unit-test policy forbids automated tests from depending on. Those three remain unchecked in spec.md by design.

.claude/skills/orchestrate/SKILL.md requires that an unautomatable requirement resolved as a permitted exception be backed by a runbook on disk. That runbook was authored after #515 merged, so it needs this follow-up to reach main.

What Changed

docs/features/active/2026-08-08-ribbon-engine-readiness-guard-503/runbooks/ac19-ac21-live-outlook-verification.runbook.md

Carries the five required sections (Cue, Prerequisites, Step-by-step Instructions, Verification, Source and Citation) with dated Microsoft Learn citations.

Two findings from reading the delivered source materially change how the verification must be run, and neither is stated in spec.md or the existing checklist:

  1. The disabled window is longer than expected. ThisAddIn.cs calls RefreshEngineCommands() after the whole LoadAsync sequence returns, not when the Engines phase completes — and Engines is fourth of five phases. The buttons therefore stay greyed for the entire startup load. The runbook gives the exact log markers for both boundaries: [phase-net] phase=Engines for engine readiness (the AC19 click deadline) and Finished loading globals for the visual state.

  2. The "still loading" message cannot appear in the passing case. When getEnabled is correctly bound and the gate is closed, Office does not dispatch the action at all, so the click never reaches the guard. The existing checklist's yes/no column for that indication is unsatisfiable by design. The runbook replaces it with a three-outcome classification — disabled by getEnabled, enabled but suppressed by the click guard, or throwing — so a verifier can tell a correct result from the original defect.

It also records that the Office "Show VSTO Add-in user interface errors" option does not report a callback signature mismatch: a mismatched callback compiles and simply never runs, so the absence of an error dialog is not evidence of binding.

.claude/agent-memory/orchestrator/

Three notes plus index entries: the two legitimate C# coverage denominators and the JaCoCo-versus-Cobertura gate format; the value of executor preflight in catching gates that pass while verifying nothing; and the Git Bash MSBuild switch-mangling problem with the verified tool paths.

Verification

Completed

Recommended

  • The runbook itself is the pending action: execute it against a live Outlook profile and record the outcome, then check off AC19-AC21 in spec.md only for steps recording PASS.

Backward Compatibility / Migration Notes

None. No runtime code is affected.

Risks and Mitigations

None to the product. The only risk is documentation drift if the startup phase order in ThisAddIn.cs changes; the runbook cites the specific log markers rather than line numbers to limit that exposure.

Review Guide

Read the runbook's Step-by-step Instructions and Verification sections. The three-outcome classification is the part worth scrutiny, since it determines whether a verifier records a pass or a failure.

Follow-ups

GitHub Auto-close

None.

drmoisan and others added 2 commits August 8, 2026 16:53
The three MANUAL-ONLY acceptance criteria require a live Outlook process and
cannot be satisfied by automated tests, so they are resolved as a permitted
exception under the autonomous-execution mandate. This runbook is the artifact
that resolution requires.

Records two findings from reading the delivered source that refine how the
verification must be run:

- The eight buttons re-enable only after the whole LoadAsync sequence returns,
  not when the Engines phase completes, so the disabled window spans the entire
  startup load. Exact log markers for both boundaries are given.
- When getEnabled is correctly bound, Office does not dispatch the action, so
  the "still loading" message cannot appear in the passing case. A three-outcome
  classification replaces the checklist's unsatisfiable yes/no column.

Refs: #503

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F54ScKL18nJNT96WPFFGi4
- C# coverage has two legitimate denominators ~15 points apart; the filtered
  first-party figure is the policy one and the gate needs JaCoCo, not Cobertura
- Executor preflight catches vacuous gates the MCP plan validator cannot see
- The Bash tool mangles MSBuild switch arguments; use pwsh with absolute paths

Refs: #503

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F54ScKL18nJNT96WPFFGi4
@drmoisan
drmoisan force-pushed the bug/ribbon-engine-readiness-guard-503 branch from eeef2b6 to dd3530c Compare August 8, 2026 20:54
@drmoisan
drmoisan merged commit 4444073 into main Aug 8, 2026
2 checks passed
@drmoisan
drmoisan deleted the bug/ribbon-engine-readiness-guard-503 branch August 10, 2026 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant