Skip to content

Add the interface engine - verify a workflow's contract by name (#374, PR-B) - #375

Merged
ptr727 merged 6 commits into
developfrom
interface-engine-374
Jul 21, 2026
Merged

Add the interface engine - verify a workflow's contract by name (#374, PR-B)#375
ptr727 merged 6 commits into
developfrom
interface-engine-374

Conversation

@ptr727

@ptr727 ptr727 commented Jul 21, 2026

Copy link
Copy Markdown
Owner

Second of the fidelity series (#374). Consumes fidelity: interface: the audit now verifies a carried workflow honors its fixed contract by name and wiring, never the owned body — the virtual-function signature check.

Change

  • spec/audit.py
    • split_jobs(text) — slices a workflow into {job_key: block} by the jobs-level indent, no YAML parser (stdlib), mirroring heading_texts.
    • check_interface(path, contract, text) — required job keys, the ruleset-bound check name:, the release-asset-<branch>-<target> handoff, and per-job required (pattern:/merge-multiple:) and forbidden (artifact-ids:) tokens. All findings DRIFT.
    • An interface unit's absence is DRIFT too (not a hard LETTER) — a workflow's naming varies more than a carried config, so everything here is advisory.
    • --selftest (no network) — 7 fixture cases.
  • spec/files.json — two workflow interface entries: test-pull-request.yml (appliesTo: "*", the universal check-workflow-status job + its ruleset-bound name) and build-release-task.yml (build types, the get-version/github-release jobs + artifact handoff + no-artifact-ids: rule).
  • AUDIT.md — notes the new check, and fixes pre-existing clause-joining semicolons in this file's comments/docstrings so they don't fester.

Verified

  • python3 spec/audit.py --selftest7/7 (conformant, missing job+name, renamed check, artifact-ids fork, missing merge-multiple, owned extra-leaf-job no-finding, split_jobs).
  • Positive control: the hub's own canonical test-pull-request.yml and build-release-task.yml are conformant (0 findings).
  • validate.py green (contracts pass PR-A's value-type guards), lints clean.

Scoping of build-release-task.yml is best-effort (excludes source-only/docs to avoid false "absent" DRIFTs); any mis-scope only yields advisory DRIFTs to triage. It refines as downstreams are audited. Held on develop. PR-C adds the verbatim engine.

… PR-B)

Consume fidelity: interface. The audit now checks that a carried workflow honors
its fixed contract without touching the owned body.

- audit.py: split_jobs slices a workflow into job blocks by indent (no YAML
  parser); check_interface verifies required job keys, the ruleset-bound check
  name, the release-asset-<branch>-<target> handoff, and per-job required
  (pattern:/merge-multiple:) and forbidden (artifact-ids:) tokens. All findings
  are DRIFT - the body is owned and a rename is a hint to verify. An interface
  unit's absence is DRIFT too, not a hard letter, since a workflow's naming
  varies more than a carried config.
- files.json: two workflow interface entries - test-pull-request.yml (universal:
  the check-workflow-status job and its ruleset-bound name) and
  build-release-task.yml (build types: the get-version/github-release jobs, the
  artifact handoff, and the no-artifact-ids fork rule).
- audit.py --selftest exercises the engine on synthetic fixtures (7 cases:
  conformant, missing job+name, renamed check, artifact-ids fork, missing
  merge-multiple, an owned extra leaf job, and split_jobs).
- AUDIT.md notes the new check. Also fix pre-existing clause-joining semicolons
  in this file's comments and docstrings so they do not fester.

Verified: --selftest 7/7 pass, and the hub's own canonical test-pull-request.yml
and build-release-task.yml are conformant to their contracts (0 findings).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 21, 2026 04:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an interface-fidelity audit path so workflow baseline entries can be verified by their contracted wiring (job keys, required job name:, artifact handoff conventions, and forbidden download patterns) without requiring verbatim workflow bodies. This fits the repo’s broader “carry model” governance by making workflow conformance checkable in a structured, low-false-positive way.

Changes:

  • Extend spec/audit.py with a lightweight workflow “jobs” slicer (split_jobs) and an interface-contract checker (check_interface), plus a --selftest fixture suite.
  • Add fidelity: interface entries to spec/files.json for test-pull-request.yml and build-release-task.yml with targeted contract fields.
  • Update AUDIT.md to document the new interface-conformance check and its DRIFT-only semantics.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
spec/files.json Adds two fidelity: interface baseline entries with workflow-specific contract fields.
spec/audit.py Implements the interface engine (split_jobs, check_interface) and wires it into the audit with --selftest.
AUDIT.md Documents the new workflow interface conformance check as part of the mechanized audit.

Comment thread spec/audit.py Outdated
Copilot round-1: "Three recursive trees calls" reads wrong; correct to "tree
calls", consistent with "any tree is truncated" in the same sentence.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 21, 2026 04:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comment thread spec/audit.py
Comment thread spec/audit.py Outdated
Copilot round-2 findings:
- split_jobs: a dedented comment after the jobs children no longer scans into a
  job block or ends the mapping early - it is skipped, so a trailing or
  between-jobs top-level comment is not mis-attributed. Added a split_jobs test
  with a trailing top-level comment.
- check_interface: skip the require/forbid token checks for a job that is
  absent - the requiredJobKeys check already reports the absence, so no
  redundant "job missing required token" for every token it cannot contain.
  Added a self-test: an absent required job reports exactly once.

Verified: --selftest 8/8, hub canonical workflows still conformant.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 21, 2026 04:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

Comment thread spec/audit.py
Comment thread spec/audit.py
Comment thread spec/audit.py
Copilot round-3 findings: check_interface used a raw substring search over the
whole workflow text and job blocks, so a token mentioned only in a comment
counted as signal. A carried task file documents its own contract in comments
(build-release-task.yml names release-asset- and artifact-ids: in prose), so
this could false-pass a missing handoff and false-flag a forbidden token that
appears only in a comment. Scan a code view (comment-only lines dropped) for
requiredCheckName, artifactNameToken, and the per-job require/forbid tokens.
Added self-tests: a forbidden token only in a comment is ignored, and a
required token only in a comment does not count.

Verified: --selftest 10/10, hub canonical workflows still conformant.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 21, 2026 04:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

spec/audit.py:205

  • check_interface() treats requiredCheckName as a raw substring match across the whole workflow (minus comment-only lines). That can false-pass if the string appears in some other field (including the workflow-level name:) and does not actually appear as a job name: in the required job. Since the contract is about the ruleset-bound job check name, match it as a name: line (and prefer searching within the required job block when there is exactly one required job).
    name = contract.get("requiredCheckName")
    if name and name not in code:
        findings.append(("DRIFT", f"interface: {path} missing the ruleset-bound check name '{name}'"))
    tok = contract.get("artifactNameToken")
    if tok and tok not in code:
        findings.append(("DRIFT", f"interface: {path} missing the '{tok}<branch>-<target>' artifact handoff"))

Comment thread spec/audit.py Outdated
Comment thread spec/audit.py
Copilot round-4:
- split_jobs: _JOB_KEY now matches a key with an inline value, and the key line
  is included in its block, so an inline-mapping job (rare, but valid) is
  captured with its content. Added inline-mapping and trailing-comment
  split_jobs self-tests.
- check_interface: match requiredCheckName as a `name:` field value (not a raw
  substring), so the ruleset-bound check name appearing in some other field (a
  run step, the workflow-level name) does not false-pass. Added a self-test for
  the check name present only in a run step.

Verified: --selftest 13/13, hub canonical workflows still conformant.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 21, 2026 04:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

spec/audit.py:205

  • check_interface()'s requiredCheckName scan matches any name: line anywhere in the workflow (including step-level name:). That can false-pass a workflow where the ruleset-bound job name is missing but a step happens to use the same name. Since the contract is explicitly about the ruleset-bound job name, restrict the match to job-level name: keys (the first-level keys inside each job block) and optionally add a selftest case where a step name: equals requiredCheckName but the job-level name is absent.
    name = contract.get("requiredCheckName")
    if name and not re.search(r"^\s*name:\s*['\"]?" + re.escape(name) + r"['\"]?\s*$", code, re.M):
        findings.append(("DRIFT", f"interface: {path} missing the ruleset-bound check name '{name}' as a job name"))
    tok = contract.get("artifactNameToken")
    if tok and tok not in code:
        findings.append(("DRIFT", f"interface: {path} missing the '{tok}<branch>-<target>' artifact handoff"))

Copilot round-5 low-confidence finding (correct): the requiredCheckName scan
matched any name: line, so a step named the same as the ruleset-bound check
could false-pass while the job name was absent. Add job_level_names (a job's
own direct-child name key, at the shallowest block indent, excluding step names
and - name: list items) and match requiredCheckName against it. Added
self-tests for the check name appearing only as a step name or only in a run
step.

Verified: --selftest 14 cases pass, and the hub's test-pull-request.yml job
names ('Check pull request workflow status job', 'Validate sources job') are
extracted at job level so it stays conformant.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 21, 2026 04:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@ptr727
ptr727 merged commit 7e5f187 into develop Jul 21, 2026
7 checks passed
@ptr727
ptr727 deleted the interface-engine-374 branch July 21, 2026 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants