Skip to content

feat(agent): add recommendation confidence and provenance #95

Description

@JSONbored

Parent roadmap: #82

Background

Users need to know whether a recommendation is backed by fresh official data, stale cache, or partial GitHub context.

Goal

Attach confidence and provenance to every ranked recommendation.

Current Behavior

Some outputs include data-quality context, but confidence/provenance is not consistently attached to each recommendation.

Desired Behavior

Each action explains which sources support it, how fresh they are, and what would change the recommendation.

Implementation Requirements

  • Add confidence fields to action outputs.
  • Include source list and freshness status.
  • Mark assumptions and user-supplied scenarios.
  • Add low-confidence degradation paths.
  • Keep schema stable for MCP clients.

Public/Private Output Boundaries

Private MCP/API may include scoreability, risk, and confidence. Public GitHub output must use sanitized summaries only and must not expose private scoring, reward/risk, raw trust, wallet/hotkey, or reviewability internals.

Acceptance Criteria

  • Every action has confidence and source summary.
  • Stale recommendations are marked.
  • Missing official data lowers confidence.
  • Public-safe summaries omit private provenance details where needed.

Testing Requirements

  • npm run test:ci must pass.
  • Global coverage must remain at or above 97% for lines, statements, functions, and branches.
  • Aim for 98%+ branch coverage locally to avoid CI variance.
  • Add tests for every new branch, fallback path, sanitizer rule, and regression.
  • Add invariant/property-style tests when behavior depends on sorting, gating, scoring, queue pressure, source-upload safety, public/private boundaries, or upstream drift.
  • Public GitHub output must be tested against forbidden language: wallet, hotkey, raw trust score, payout, reward estimate, farming, private reviewability, and public score estimate.
  • MCP/local tooling must prove source contents are not uploaded.

Additional Test Scenarios

  • Fresh official source fixture.
  • Stale cache fixture.
  • Partial repo sync fixture.
  • User-supplied scenario fixture.
  • Sanitized public summary regression.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions