Continue the Files sidebar AI documentation experiment #409

Purpose

Continue the AI documentation experiment preserved by issue #407 and pull request #408, now merged into dev_jerome.

Keep all experiment source, prompts, data, generated pages, screenshots and results inside tmp/web_doc_by_ai. The normal website and application packages must remain separate. This issue requests continuation of the experiment, not production integration.

Starting point

Read the experiment README, process, and handoff.

The parked workspace passed dependency installation, writer and judge dry runs, seven pure judge tests, 111 website tests, the production website build, and browser loading of all four Files sidebar pages and screenshots. Those checks did not establish successful model generation.

The later GPT-6.1 Sol / high correction attempt was stopped by the user during the writing pass. Existing pages stayed unchanged, and the calibrated judge was not started. Preserve the interrupted evidence in results/known_files_fixes_20261014; do not report the attempt as completed.

Two confirmed documentation defects

1. Relative paths use the Git repository root. With the workspace set to the repository's docs subfolder, copying the path of README.md produced docs/README.md in both the notification and an actual clipboard paste. Correct the workspace-relative description in Copy a file path and Files sidebar section controls, including a clear repository-subfolder example. The captured rule wording conflicts with the implementation and live evidence; record this conflict and cite the implementation rather than inventing behaviour.
2. A newly added workspace needs explicit selection. New workspace… opens the native folder chooser and adds the chosen folder to the workspace list without selecting the workspace. Browse and open project files must tell the reader to click the new workspace entry after adding the folder.

Next experiment cycle

• Resume only after the developer authorises a fresh model run; creating this issue does not start one.
• Use the real generator with gpt-6.1-sol and high effort, limited to docs/agents_rules/user_interface/sidebars/files.md. Do not hand-edit generated pages.
• Preserve the outline, exact prompts and answers, settings, input fingerprints, before/after pages, progress, failures and costs when available.
• Judge the four Files pages with calibrated contract version 3, claude-sonnet-5-5, high effort and the existing fixed rubric. Inspect screenshot pixels. Do not change the rubric to obtain higher scores; recalibrate with independently labelled good and defective examples if the rubric changes.
• Show findings and verification limits, not only average scores. The earlier 5/5 source-only judgement was not acceptance and was superseded by the live findings above.
• Verify the corrected tasks in the real native application built from the captured source and the rendered experimental website. Start the application with --no-splash-screen. Verify relative paths with a repository-subfolder workspace and an actual clipboard paste, and verify adding then selecting a new workspace through the real folder chooser.
• Keep generator and judge operation free of staging, branches, commits, pushes, pull requests, merges and deployment. Any repository publication is a separately authorised developer workflow.

Remaining verification limits

• Notification timing was inconclusive: the accessibility observation did not agree with the source timeout. Do not label this as a confirmed application bug.
• A separate repository-root clipboard control was disturbed by another clipboard writer and is excluded from the evidence.
• Other operating systems, denied clipboard permissions, error paths, duplicate-name creation and restart persistence have not been verified. Either exercise relevant cases or explicitly retain the limits.
• Two pre-existing documentation-updater test failures concern metadata on the unselected Introduction page. Keep those separate from the Files-only result.
• The calibration examples are a small sample and do not establish universal judge reliability.

Acceptance criteria

• Both confirmed defects are corrected by the generator, with source references and matching readable screenshots.
• The calibrated judge reports evidence-backed findings under unchanged settings; blockers cannot be overridden by a high average.
• The corrected native tasks and rendered pages are verified, with raw evidence and explicit remaining limits.
• All experiment edits and artifacts remain under tmp/web_doc_by_ai; the normal website remains unchanged.
• The process and handoff accurately describe the latest result and a reproducible next step.

After reliability is demonstrated, compare cheaper models or lower effort against the same fixed evaluation and live acceptance checks. Production integration remains a later, separately authorised decision.