Skip to content

feat: match repeated findings across scan history - #575

Merged
mldangelo-oai merged 48 commits into
mainfrom
mdangelo/codex/match-repeated-findings
Sep 4, 2026
Merged

mldangelo-oai merged 48 commits into
mainfrom
mdangelo/codex/match-repeated-findings

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Aug 20, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Match repeated findings across scan history using compact catalogues, paged evidence, and deterministic confirmed identities. This PR now targets main after #574 merged and reuses existing Codex authentication.

Changes

  • Group stable identities and confirmed links into catalogue records, then expand decisions back to every original occurrence.
  • Page catalogues and requested evidence within the upstream message limit. Require complete omitted evidence before accepting a decision, and keep uncertain and independently related findings separate.
  • Preserve cancellation, saved comparisons, sealed artifacts, and the one-extra-call allowance for cost-limited automatic matching.
  • Export matchScanFindings with self-contained public input, result, progress, cancellation, and reasoning-effort types. No additional public CLI syntax or defaults change.
  • Merge main at a82fc14, preserving the canonical plugin source, migration history, plugin metadata, native command authentication, interactive budget increases, shared test fixtures, and existing public API exports.
  • Keep the catalogue and deterministic-identity tests in place of the superseded raw-array batching tests. Give the new authentication regressions nonempty finding pairs so they exercise provider selection.
  • Mark the internal environment-helper re-export as internal so the published matcher declarations compile for installed TypeScript consumers.
  • Forward environment API keys through the Codex SDK authentication option. Preserve the existing OPENAI_API_KEY precedence, native stored credentials, and explicit command-provider selection.

Testing

Merge conflict resolution at 0eb54b1bb4b93d837a332ca699bcb29492fa2470 against main at 6750642c94edce8d60112f816631de05dc10efd6:

  • Preserve catalogue-based matching, its single-turn automatic budget guard, and catalogue tests; combine matcher and severity-classification package checks and accept main's ultra reasoning effort in shared options.
  • Focused catalogue, comparison, and comparison-property tests: 106 passed and 3 Windows-only skips.
  • SDK TypeScript check, plugin bundle generation, package smoke-script syntax, changed-file formatting, and git diff --check passed.
  • The full test suite was left to CI.

Earlier validation recorded for prior heads:

  • Full SDK suite on f17ed63000f7c4cadaa17eecbd4fd1be6c1933d4, seed 12345: 2,273 passed, 43 skipped, 0 failures across 114 files.
  • Focused catalogue, matcher, workbench CLI, sealed-history end-to-end, and component-scan suites: 166 passed, 3 skipped.
  • Credential regressions reproduce the missing SDK API key before the fix and pass afterward; explicit command authentication remains separate from environment keys.
  • Installed CLI end-to-end QA at f17ed63000f7c4cadaa17eecbd4fd1be6c1933d4: four sealed scans, six scan pairs, catalogue and evidence turns, cached reruns without model calls, forced recomputation, related-finding separation, SIGINT, invalid model output, and all 16 sealed artifact hashes preserved.
  • Live installed-package matching with Codex and synthetic findings passed: repeated findings matched while an independent control stayed separate.
  • Native Codex 0.149.1 provider-routing probes passed: a trusted repository attempting to override model_provider and model_providers could not redirect the synthetic API key or finding prompt. Verified both an explicit repository working directory and a process launched from that repository.
  • Python scan-history, database, and migration suites: 197 passed. MCP app suite: all 23 passed.
  • TypeScript and MCP typechecking, generated-model checks, formatting, plugin generation, plugin-source checks, and portable Ruff and source-compatibility checks passed.
  • Package validation at f17ed63000f7c4cadaa17eecbd4fd1be6c1933d4 passed under Node 24.15.0, including archive inspection, public imports, NodeNext declarations, CLI and SDK lifecycle checks, credential locking, MCP initialization, dashboard assets, and nested worker startup without global Codex.

Risk and rollout

#574 is merged. Saved comparisons are not rewritten automatically; the existing forced matching command recomputes them when requested. Preserve globally unique occurrence IDs, confirmed identity validation, read-only matching with repository instructions, tools, MCP, and network disabled, and separate related findings. The single-turn automatic matching guard is not a dollar-accurate spending guarantee.

The bundled Codex 0.149.1 already filters provider, endpoint, and profile settings from project configuration, before merging those project layers. The native-request probes above confirm this existing protection against the automated provider-redirection review concern.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

Existing commit metadata and automated comments contain author contact metadata and account-specific links, so the second attestation remains unchecked. The new commits use GitHub noreply author and committer addresses; fixtures and examples are synthetic.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, 5875cb27239e4a310be1bc3cd4c58b67275b5319, against #574. This is the matcher and existing CLI/SDK integration split from #567. The storage contract is reviewed separately in #574.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 5875cb2723

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

The description now includes the final QA results and merge order. Please review the current head, 5875cb27239e4a310be1bc3cd4c58b67275b5319. The code has not changed since the earlier review.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 5875cb2723

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-03T23:48:55.011791Z 0eb54b1 Manual request
🔒 Security Review ✅ Completed 2026-09-03T23:53:22.682501Z 0eb54b1 New commits

Security findings

Blocking findings (1)

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c3d4a5661b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/scan-comparison.ts

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, c3d4a5661b31870dd2960432f3390f491af5d9ee. The PR description includes the latest cleanup and QA results.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: c3d4a5661b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, c8b75fc305061c6fa0a39e842f111a0d9c9707e4. This also fixes the omitted-evidence case raised in the previous review and adds tests for all four final outcomes.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: c8b75fc305

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, 1b1e546213d1a041e3daf03985f24d24b8ade8a8, against #574. This additive update merges current main, preserves failed-history output semantics, recomputes forced single-pair matches without old semantic groups, and compares complete selected scans before caching automatic results. The PR description includes the regressions, package checks, synthetic live QA, and the remaining full-suite checks.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 1b1e546213

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

The description now includes the completed full-suite runs, installed-package and synthetic live QA, and the green exact-head CI result. Please review the current head, 1b1e546213d1a041e3daf03985f24d24b8ade8a8, against #574. The code has not changed since the previous review.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 1b1e546213

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/scan-comparison.ts Outdated
Keep legacy repositories separate when preparing matching inputs, simplify finding alias bookkeeping, and retain the shared workbench stdin API. Use platform-aware Python probes and exercise comparison payloads larger than command-line limits.
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: 02abe26a52

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kmbroai

kmbroai commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Before I approve this PR, please refresh it onto the current #574 head (6606adae86a86a1b5077ff357feaaffb44b38b97). This branch still contains the older #574 head (08926e62b17c3edfc6b2319483a27f90bb8746d4) and GitHub currently reports the PR as conflicting.

Please resolve the stack against the reconciled canonical plugin source and migration history, preserving #574's migration 40 and synchronized plugin metadata. Then rerun full exact-head CI and request fresh code/security review. The existing successful checks and approval validate the old parent, not the updated stack.

I do not have a separate feature-design blocker; the required change is to reconcile and revalidate the child on its current parent.

@kmbroai kmbroai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved for the feature implementation at 222e398877b37f186171847e920b752c13b37a17.

This approval does not clear the stack-integration blocker: the PR is currently conflicting with its updated #574 base. Resolve that conflict, preserve the reconciled migration and plugin-source changes, and obtain fresh exact-head CI/review before merge.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review current head adeb023ecbd806240b114d7ac6b7fde1ae56894c after the additive merge from its updated base, including the conflict resolution and preserved behavior. Focused validation is recorded in the PR description.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: adeb023ecb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Base automatically changed from mdangelo/codex/scan-comparison-storage to main September 1, 2026 20:44
@github-actions github-actions Bot added the enhancement New feature or request label Sep 1, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Security Review

Here are some automated security review suggestions for this pull request.

Reviewed commit: f17ed63000

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Comment thread sdk/typescript/src/scan-comparison.ts
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review head 0eb54b1bb4b93d837a332ca699bcb29492fa2470 against main at 6750642c94edce8d60112f816631de05dc10efd6. Preserve catalogue-based matching, its single-turn automatic budget guard, and catalogue tests; combine matcher and severity-classification package checks and accept main's ultra reasoning effort in shared options.

Focused catalogue, comparison, and comparison-property tests: 106 passed and 3 Windows-only skips. SDK TypeScript check, plugin bundle generation, package smoke-script syntax, changed-file formatting, and git diff --check passed. The full suite is left to CI.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: 0eb54b1bb4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai
mldangelo-oai merged commit 0d781d7 into main Sep 4, 2026
69 of 85 checks passed
@mldangelo-oai
mldangelo-oai deleted the mdangelo/codex/match-repeated-findings branch September 4, 2026 06:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants