Skip to content

feat(signals): add deterministic copycat containment engine (#1969) - #5973

Closed
andriypolanski wants to merge 1 commit into
JSONbored:mainfrom
andriypolanski:feat-copycat-containment-engine-1969
Closed

feat(signals): add deterministic copycat containment engine (#1969)#5973
andriypolanski wants to merge 1 commit into
JSONbored:mainfrom
andriypolanski:feat-copycat-containment-engine-1969

Conversation

@andriypolanski

Copy link
Copy Markdown
Contributor

Closes #1969

Summary

Phase 1 of #1969 (design from folded #1409): a deterministic, pure copycat/plagiarism containment engine. The gate.copycat.mode / minScore scaffold (#4140) stays call-site-inert — this PR supplies the detection primitive it was threaded for.

Resubmits the approach of closed #5129 with the maintainer-requested multiset fix: candidate shingles are scored as a list (occurrences), prior art as a Set (lookup).

Behavior

  • Containment (0–100) — asymmetric fraction of the candidate's multi-line (3-line) shingles found in prior art. Whitespace/comment/case normalized so reformatting cannot evade it.
  • Direction by timestamp — earlier submission = victim; only candidate_copied can act. Missing / unparseable / tied timestamps → ambiguous → never act.
  • Precision guards — finding only when mode ≠ off, score ≥ threshold (default 85), distinct authors, enough added lines (default 10), and direction is unambiguously candidate_copied.
  • Tier → severitywarn→info, label→warning, block→critical. Public-safe text: score + threshold + optional prior PR number only.

Scope / non-goals

  • In: src/signals/copycat.ts + test/unit/copycat.test.ts; doc-comment updates on the already-scaffolded config fields.
  • Out (deferred Phase 2/3): feeding real prior-art PRs from the queue, actuating copycat label / close / strikes, cross-repo candidate search. Design-spec issue #5746 still covers longer-term algorithm/FPR sign-off for those later slices.

Tests

npx vitest run test/unit/copycat.test.ts --coverage --coverage.include=src/signals/copycat.ts23 pass, 100% statements/branches/functions/lines.

Links

@andriypolanski
andriypolanski marked this pull request as draft July 15, 2026 05:16
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.17%. Comparing base (49883fa) to head (03ed27d).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5973   +/-   ##
=======================================
  Coverage   95.16%   95.17%           
=======================================
  Files         589      590    +1     
  Lines       46783    46846   +63     
  Branches    14938    14966   +28     
=======================================
+ Hits        44521    44584   +63     
  Misses       1511     1511           
  Partials      751      751           
Flag Coverage Δ
shard-1 43.85% <0.00%> (-0.06%) ⬇️
shard-2 36.17% <0.00%> (-0.04%) ⬇️
shard-3 32.07% <0.00%> (+0.09%) ⬆️
shard-4 31.20% <0.00%> (-0.76%) ⬇️
shard-5 32.57% <0.00%> (+0.07%) ⬆️
shard-6 44.61% <100.00%> (-0.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
packages/loopover-engine/src/focus-manifest.ts 99.11% <ø> (ø)
src/signals/copycat.ts 100.00% <100.00%> (ø)
src/types.ts 100.00% <ø> (ø)

@andriypolanski
andriypolanski marked this pull request as ready for review July 15, 2026 05:20
@loopover-orb loopover-orb Bot added gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier. labels Jul 15, 2026
@loopover-orb

loopover-orb Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-15 05:23:52 UTC

5 files · 1 AI reviewer · no blockers · readiness 80/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This adds a pure, dependency-free copycat/containment detection engine (src/signals/copycat.ts) plus a 341-line test suite achieving 100% branch coverage, alongside doc-comment updates clarifying that the already-scaffolded gate.copycat.mode/minScore config is still call-site inert. The core logic — asymmetric multiset shingle containment, timestamp-based direction with ambiguous-on-tie/missing, and the five precision guards gating wouldAct — is internally consistent and correctly traced through assessCopycat, containmentScore, and resolveCopycatDirection. This is explicitly Phase 1 of issue #1969 with no wiring into advisory/processors yet, matching the PR's stated scope.

Nits — 5 non-blocking
  • src/signals/copycat.ts normalizeLine strips everything after the first `//` or `#` unconditionally, which will corrupt lines containing URLs (`https:​//...`) or TS private fields (`#field`) rather than just comments — the header acknowledges this as a deliberate format-agnostic tradeoff, but it's worth flagging since it can both mask real copies and produce false positives on unrelated code that happens to share a truncated prefix.
  • src/signals/copycat.ts:171-174 assessCopycat calls normalizeAddedLines(input.candidateLines) directly for candidateAddedLines and then calls containmentScore(input.candidateLines, ...) which re-runs normalizeAddedLines on the same array internally — minor redundant work, not a hot-path concern given PR-sized inputs but easy to dedupe by threading the already-normalized list through.
  • The codeShingleList short-candidate fallback (src/signals/copycat.ts:~95) collapses to a single whole-block token, which is a real detector limitation, but it's well below the default 10-line minAddedLines floor in practice so its exposure via the exported function alone is fine — just confirm no future caller invokes containmentScore directly bypassing the assessCopycat guards.
  • Consider exposing the already-normalized candidate line list from containmentScore (or accept it as a param) so assessCopycat doesn't normalize the candidate twice.
  • If prior art lines can legitimately contain `#` as non-comment syntax (Ruby symbols, TS private fields) in target ecosystems, consider only stripping `#` when it's clearly a line-leading comment marker rather than anywhere in the line.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #1969
Related work ⚠️ 2 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 198 registered-repo PR(s), 117 merged, 25 issue(s).
Contributor context ✅ Confirmed Gittensor contributor andriypolanski; Gittensor profile; 198 PR(s), 25 issue(s).
Gate result ✅ Passing No configured blocker found.
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Partially addressed
The PR delivers a deterministic containment scoring engine, timestamp-based direction logic, tiered severity mapping, and thorough unit tests, directly matching the 'similarity engine + direction logic' portions of the issue's deliverables. However, it explicitly leaves out actual gate wiring (feeding real prior-art PRs, actuating labels/blocks/strikes) which the issue requires as part of 'tiered

Review context
Contributor next steps
  • Start here: Review top overlaps.
  • Then work through the remaining 2 steps in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb

loopover-orb Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

LoopOver is closing this pull request on the maintainer's behalf (Linked issue #1969 is assigned to the maintainer (@JSONbored) — that work is reserved for the maintainer, so this PR cannot be auto-accepted.). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.

@loopover-orb loopover-orb Bot closed this Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. gittensor:priority Maintainer-selected Gittensor priority — scores a 1.5x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Plagiarism / copycat detection

2 participants