Skip to content

[spec-review] Update Safe Outputs conformance checker for recent spec changes#32956

Merged
pelikhan merged 1 commit into
mainfrom
fix/safe-outputs-conformance-v1-20-0-64dbfad503626ece
May 18, 2026
Merged

[spec-review] Update Safe Outputs conformance checker for recent spec changes#32956
pelikhan merged 1 commit into
mainfrom
fix/safe-outputs-conformance-v1-20-0-64dbfad503626ece

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

Summary

Updates the Safe Outputs conformance checker script to align with the spec v1.20.0 changes introduced in the recent commit.

Specification Changes Reviewed

  • Commit c2a2a94: Added docs/src/content/docs/reference/safe-outputs-specification.md at v1.20.0
  • Spec v1.20.0 adds Section 10.5 (Warn-Mode Threat Detection Failure Policy) with normative requirements WTD1–WTD3, which shifts the former Section 10.5 (Zero Max / Edge Cases) to Section 10.6
📋 Script Updates & Testing Details

Script Updates

Version Updated

  • Script version bumped from 1.19.0 to 1.20.0 to track the spec version

Section Reference Updated

  • EXEC-002: Section reference corrected from 10.510.6 (zero-max edge case moved when Section 10.5 became the warn-mode policy section)

New Checks Added

  • WTD-001: Validates Reviewable annotation requirements (WTD1): caution [!CAUTION] block, visible agentic threat detected label string, and <!-- gh-aw-threat-detected --> XML comment marker are all present. Checks generate_footer.cjs directly and via the centralised threat_detection_warning.cjs helper.
  • WTD-002: Validates the Convertible fallback for push_to_pull_request_branch (WTD2): the handler must detect GH_AW_DETECTION_CONCLUSION=warning and fall back to a review pull request with the WTD1 annotations. Also checks the handler manager registers the type as Convertible.
  • WTD-003: Validates Abort-class output handling (WTD3): THREAT_WARNING_ABORT_TYPES set exists, abort policy branch stops execution, and a machine-readable threat_detected_abort_policy error outcome is returned. Traceability check for WTD3 requirement ID reference.

Testing

Ran the updated script — all checks passed:

[PASS] WTD-001: Reviewable annotation includes caution block, threat label, and XML marker (WTD1)
[PASS] WTD-002: push_to_pull_request_branch has convertible fallback to create_pull_request (WTD2)
[PASS] WTD-003: Abort-class outputs have threat-detected abort handling and machine-readable error outcomes (WTD3)
PASS: All checks passed

Related Files

  • Specification: docs/src/content/docs/reference/safe-outputs-specification.md
  • Conformance Script: scripts/check-safe-outputs-conformance.sh

Generated by 🔒 Weekly Safe Outputs Specification Review · ● 20M ·

  • expires on May 25, 2026, 5:51 AM UTC

- Update script version from 1.19.0 to 1.20.0
- Update EXEC-002 section reference from Section 10.5 → Section 10.6
  (zero-max edge case moved to 10.6 when 10.5 became warn-mode policy)
- Add WTD-001: check reviewable annotation (caution block, 'agentic threat
- Add WTD-002: check push_to_pull_request_branch convertible fallback to
  create_pull_request under warn-mode threat detection per WTD2
- Add WTD-003: check abort-class outputs produce threat-detected error
  outcomes and are not applied per WTD3

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added automation documentation Improvements or additions to documentation safe-outputs labels May 18, 2026
@github-actions github-actions Bot mentioned this pull request May 18, 2026
@github-actions
Copy link
Copy Markdown
Contributor Author

Please re-run the safe-outputs conformance checker and post the result.

Generated by 👨‍🍳 PR Sous Chef ·

@pelikhan pelikhan merged commit 9d3d52b into main May 18, 2026
@pelikhan pelikhan deleted the fix/safe-outputs-conformance-v1-20-0-64dbfad503626ece branch May 18, 2026 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automation documentation Improvements or additions to documentation safe-outputs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant