Skip to content

feat(github-agent): add command usefulness feedback - #194

Merged
JSONbored merged 2 commits into
JSONbored:mainfrom
oktofeesh1:codex/github-agent-feedback
Jun 1, 2026
Merged

feat(github-agent): add command usefulness feedback#194
JSONbored merged 2 commits into
JSONbored:mainfrom
oktofeesh1:codex/github-agent-feedback

Conversation

@oktofeesh1

Copy link
Copy Markdown
Contributor

Summary

  • Adds command-answer and command-feedback storage for @gittensory GitHub agent replies.
  • Records deduped usefulness votes from GitHub thumbs-up/thumbs-down reactions and protected app feedback.
  • Exposes aggregate command usefulness metrics through the app API and operator dashboard.

What changed

  • Added separate D1 tables for command answers and feedback with unique actor-answer dedupe.
  • Added public-safe answer markers and aggregate-only feedback prompts to command comments.
  • Added GitHub reaction webhook handling with known-answer, non-bot, maintainer/operator, and confirmed-miner author gates.
  • Added protected app feedback and usefulness endpoints plus regenerated the UI OpenAPI snapshot.
  • Added regression coverage for dedupe, authorization, skip paths, no raw actor storage, public sanitizer boundaries, API errors, and coverage thresholds.

Why

Validation

  • npm run typecheck
  • npm run test:unit -- test/unit/command-feedback.test.ts test/unit/github-commands.test.ts test/unit/queue.test.ts
  • npm run test:integration -- test/integration/api.test.ts
  • npm run test:coverage
  • npm run test:ci
  • Codex Security diff scan: no findings; 9/9 diff worklist rows reviewed and report validated/rendered.

Notes

  • Feedback rows store hashed actor identifiers for dedupe; aggregate APIs expose command counts/rates only.
  • Feedback remains separate from scoring, decision packs, and deterministic recommendation state.

@oktofeesh1
oktofeesh1 requested a review from JSONbored as a code owner June 1, 2026 15:16
@dosubot dosubot Bot added the size:XL label Jun 1, 2026
@oktofeesh1
oktofeesh1 force-pushed the codex/github-agent-feedback branch from 8acfba9 to d7fed30 Compare June 1, 2026 19:15
@superagent-security superagent-security Bot added the pr:flagged PR flagged for review by security analysis. label Jun 1, 2026

@superagent-security superagent-security Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superagent found 1 security concern(s).

Comment thread src/api/routes.ts

@JSONbored JSONbored left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oktofeesh1 This needs another pass. Current main conflicts in src/api/routes.ts, src/db/repositories.ts, src/openapi/spec.ts, test/integration/api.test.ts, and test/unit/queue.test.ts, and Superagent is still action-required.

The usefulness feedback feature is still a reasonable follow-up, but API, repository, OpenAPI, integration-test, and queue code all moved underneath it. Please rebase, address the security scan finding, and rerun the full gate before another review.

@oktofeesh1
oktofeesh1 force-pushed the codex/github-agent-feedback branch from d7fed30 to 12c613b Compare June 1, 2026 20:53
@dosubot dosubot Bot added size:XXL and removed size:XL labels Jun 1, 2026
@superagent-security superagent-security Bot removed the pr:flagged PR flagged for review by security analysis. label Jun 1, 2026
@oktofeesh1
oktofeesh1 force-pushed the codex/github-agent-feedback branch from 12c613b to 27f1231 Compare June 1, 2026 20:57
@dosubot dosubot Bot removed the size:XXL label Jun 1, 2026
@dosubot dosubot Bot added the size:XL label Jun 1, 2026
@dosubot dosubot Bot added size:XL and removed size:XL labels Jun 1, 2026
@oktofeesh1
oktofeesh1 requested a review from JSONbored June 1, 2026 21:10

@JSONbored JSONbored left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oktofeesh1 This is ready from my side. The branch is now green and merge-clean, and the prior conflict/security-scan blockers are resolved.

A few notes:

  • The feedback migration, repository storage, API route, OpenAPI shape, queue handling, and command handling are wired together.
  • The added tests cover feedback persistence, command behavior, OpenAPI coverage, integration routes, and queue processing.
  • No requested changes from me.

@dosubot dosubot Bot added the lgtm label Jun 1, 2026
@JSONbored
JSONbored merged commit 53c8f83 into JSONbored:main Jun 1, 2026
3 checks passed
@dosubot

dosubot Bot commented Jun 1, 2026

Copy link
Copy Markdown

Documentation Updates

1 document(s) were updated by changes in this PR:

Architecture
View Changes
@@ -246,7 +246,7 @@
 
 ### Schema Organization
 
-The schema defines 43 tables organized by functional domain [[64]](https://github.com/JSONbored/gittensory/blob/f63f125e2126dc58ac5a63e88f4f33d1e13b2979/src/db/schema.ts):
+The schema defines 45 tables organized by functional domain [[64]](https://github.com/JSONbored/gittensory/blob/f63f125e2126dc58ac5a63e88f4f33d1e13b2979/src/db/schema.ts):
 
 #### Installation & Repository Management
 
@@ -301,6 +301,8 @@
 - **agentRuns** — Agent orchestration runs with objective and status ([src/db/schema.ts](https://github.com/JSONbored/gittensory/blob/f63f125e2126dc58ac5a63e88f4f33d1e13b2979/src/db/schema.ts#L408-L428))
 - **agentActions** — Agent action recommendations with risk impact ([src/db/schema.ts](https://github.com/JSONbored/gittensory/blob/f63f125e2126dc58ac5a63e88f4f33d1e13b2979/src/db/schema.ts#L430-L457))
 - **agentContextSnapshots** — Agent context with decision pack and signal snapshot IDs ([src/db/schema.ts](https://github.com/JSONbored/gittensory/blob/f63f125e2126dc58ac5a63e88f4f33d1e13b2979/src/db/schema.ts#L459-L474))
+- **githubAgentCommandAnswers** — Stores command invocations and responses from the GitHub agent (@gittensory), including repo, issue number, command name, request/response comment IDs, actor kind, and metadata. Indexed by (repo_full_name, issue_number) and (command, updated_at).
+- **githubAgentCommandFeedback** — Records usefulness votes (thumbs-up/thumbs-down reactions) on command answers, with unique constraint on (answer_id, actor_hash) to deduplicate feedback per actor. Tracks vote, source (reaction or app feedback), actor kind, and links to github_agent_command_answers via foreign key. Indexed by (answer_id, actor_hash), (command, updated_at), and (repo_full_name, issue_number).
 - **syncRuns** — Sync job execution tracking ([src/db/schema.ts](https://github.com/JSONbored/gittensory/blob/f63f125e2126dc58ac5a63e88f4f33d1e13b2979/src/db/schema.ts#L523-L533))
 - **webhookEvents** — GitHub webhook delivery tracking with payload hash ([src/db/schema.ts](https://github.com/JSONbored/gittensory/blob/f63f125e2126dc58ac5a63e88f4f33d1e13b2979/src/db/schema.ts#L510-L521))
 

How did I do? Any feedback?  Join Discord

@JSONbored JSONbored added gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. and removed feature gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(github-agent): add command usefulness feedback loop

2 participants