Skip to content

Audit gap: no check that Actions-workflow repos track the github-actions Dependabot ecosystem (ESPHome-Config missed it) #346

Description

@ptr727

Surfaced during the ESPHome-Config conformance standup (ESPHome-Config#46). Two parts: a concrete per-repo drift, and the audit-coverage gap that let it through.

The drift

ptr727/ESPHome-Config's .github/dependabot.yml tracks only the devcontainers ecosystem (on main and develop). It does not track github-actions, even though the repo runs several workflows (test-pull-request.yml, publish-release.yml, merge-bot-pull-request.yml) that pin actions by commit SHA, e.g.:

uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0

Consequence: those pinned SHAs get no bump PRs, so they silently go stale. And the merge-bot just stood up in that repo exists precisely to auto-merge Dependabot PRs - with no github-actions ecosystem, it has no action-update PRs to act on.

This is a divergence from the fleet norm, not a per-repo choice: every other repo I checked tracks it.

repo github-actions ecosystem
Vantage-Config (blessed reference) yes
HomeAutomation-Config yes
ProjectTemplate (hub) yes
ESPHome-Config no (devcontainers only)

The audit-coverage gap (the part the template should own)

spec/audit.py checks Dependabot secret stores (repos/{slug}/dependabot/secrets) but has no check for Dependabot ecosystem coverage. There is nothing asserting that a repo with .github/workflows/*.yml also carries a github-actions entry in .github/dependabot.yml. So ESPHome-Config's missing ecosystem produced no DEFECT/LETTER finding and slipped through the standup audit entirely.

Proposal: add a fleet audit check (spec/audit.py, and a letter in AUDIT.md) that, for any repo with GitHub Actions workflows, verifies .github/dependabot.yml declares a github-actions ecosystem covering the repo's ground-truth branch(es). Optionally generalize it: cross-check the ecosystems a repo's tree implies (e.g. devcontainers for .devcontainer/, github-actions for .github/workflows/, language ecosystems for lockfiles) against what dependabot.yml declares.

Note: the ESPHome-Config driftNotes in registry/repos.json already carry a "Dependabot currently covers only the devcontainers ecosystem" observation - so this was known and noted, but never promoted to an enforced audit check. This issue is to make the audit catch it (and to fix the ESPHome-Config gap itself). Filing separately per the standup owner's request; the standup evidence is in ESPHome-Config#46.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions