Skip to content

appmap-review: write novel findings as an action item (fix ticket) a developer can pick up #18

Open

Description

@devin-ai-integration

4. Novel findings should end as an action item a developer (or an AI developer) can pick up

Follow-up from the same readers. The 🔴/🟡/🟢 grading is right and should stay. What is missing is that a novel finding stops at description: it says what the recordings show, but a developer handed the report still has to work out what to change, where, and how they would know it is fixed. Each 🔴/🟡 novel finding should be written like a bug report back to the author, with a fixed shape:

Field What goes there
Title one line, imperative, the behavior that must hold ("OpenID login must not clear an existing user's administrator flag when openid.adminGroup is unset")
Where file(s) and function(s), with line numbers at head
Expected the behavior in plain English, from the user's side
Actual what head does, naming the code path
Evidence the recording (test name) that shows it, and what base vs head each ran
Done when the acceptance condition: which test/recording passes and what it must no longer show (e.g. "no UPDATE tc_users … administrator = false in the recording")

It should also say explicitly whether the PR introduced the defect or it is latent on base — the fix is the same, but the author's obligation differs, and today that only appears in the track label.

Worked example for traccar/traccar#5801 (the one 🔴 in the published set), rendered as a "fix ticket" above the review at https://appmap-playground.fly.dev/case/traccar-openid-login-demotes-administrators-without-admin-group?to=/review :

🔴 High · OpenID login must not clear an existing user's administrator flag when openid.adminGroup is unset

If the server has not been told which single-sign-on group means "administrator", every existing administrator who signs in through single-sign-on is silently downgraded to a normal user on that login.

Where src/main/java/org/traccar/database/OpenIdProvider.java (handleCallback, ~l.177) and src/main/java/org/traccar/api/security/LoginService.java (login, ~l.135)
Expected With no openid.adminGroup configured the provider has no opinion about administrator status; the stored flag is left untouched.
Actual handleCallback computes administrator = adminGroup != null && groups.contains(adminGroup) as a primitive boolean (false when unset); LoginService.login compares it with the stored true and UPDATEs tc_users to administrator = false.
Evidence Recording OpenIdLoginProbeTest.withoutAnAdminGroupAnExistingAdministratorStaysAnAdministrator: base reads the row and writes nothing; head runs DatabaseStorage#updateObject with administrator = false.
Done when administrator is nullable (null when adminGroup is unset) and only synced when non-null; the probe test passes on the fixed revision, its recording shows no UPDATE of tc_users.administrator, and the configured-admin-group recording still shows the sync.

The playground currently hand-writes these tickets per case (infra/demo/plain.json in evlawler/appmap-playground) so viewers see the intended shape; the durable fix is for the skill's Findings template to emit them.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions