Skip to content

feat(agent): add closed-loop feedback from accepted recommendations #96

Description

@JSONbored

Parent roadmap: #82
Cross-link: outcome learning analytics loop #278

Background

A base agent should learn whether its guidance helped. It should not learn through opaque AI; it should track deterministic outcomes after recommendations are followed.

Goal

Track recommendation outcomes and feed them into future planning quality signals.

Current behavior

Agent runs can produce actions, but Gittensory does not fully close the loop on whether actions led to merged, closed, stale, ignored, or improved PRs.

Desired behavior

Recommendation snapshots can be evaluated later against GitHub/Gittensor outcomes and used to improve future confidence/ranking. Broader role-aware analytics belong in #278.

Implementation requirements

  • Persist recommendation/action identifiers.
  • Link later PR/issue outcomes where discoverable.
  • Track accepted, ignored, stale, merged, closed, and improved outcomes.
  • Avoid using outcome feedback to publicly shame contributors.
  • Feed aggregate signals into confidence and pattern snapshots.
  • Keep this issue focused on the base feedback mechanism; feat(analytics): add outcome learning loop for recommendation quality #278 owns the larger analytics loop.

Public/private output boundaries

Private MCP/API may include scoreability, risk, and confidence. Public GitHub output must use sanitized summaries only and must not expose private scoring, reward/risk, raw trust, wallet/hotkey, or reviewability internals.

Acceptance criteria

Testing requirements

  • npm run test:ci must pass.
  • Global coverage must remain at or above 97% for lines, statements, functions, and branches.
  • Aim for 98%+ branch coverage locally to avoid CI variance.
  • Add tests for every new branch, fallback path, sanitizer rule, and regression.
  • Add invariant/property-style tests when behavior depends on sorting, gating, scoring, queue pressure, source-upload safety, public/private boundaries, or upstream drift.
  • Public GitHub output must be tested against forbidden language: wallet, hotkey, raw trust score, payout, reward estimate, farming, private reviewability, and public score estimate.
  • MCP/local tooling must prove source contents are not uploaded.

Additional test scenarios

  • Merged-after-recommendation fixture.
  • Closed-after-recommendation fixture.
  • Ignored/stale fixture.
  • Maintainer-lane fixture.
  • Privacy regression.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions