Skip to content

Fix tag merge timeout on unrelated segment payload IDs - #788

Open
jiwenjimiran wants to merge 2 commits into
yourcove:mainfrom
jiwenjimiran:fix/tag-merge-json-scan
Open

jiwenjimiran wants to merge 2 commits into
yourcove:mainfrom
jiwenjimiran:fix/tag-merge-json-scan

Conversation

@jiwenjimiran

@jiwenjimiran jiwenjimiran commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Tag merges currently prefilter stored segment JSON by looking for the source tag ID's digits anywhere in the payload. Imported AI segments contain unrelated numeric scene/span IDs. Merging a tag with ID 51 and only one segment caused the merge request to time out: PostgreSQL estimated roughly 1.55 million matching payload rows before Cove could check whether any were actual tag references.

This change also requires a recognized tag-bearing JSON key before loading a segment payload. It remains a conservative prefilter; TagReferenceJsonRewriter still checks and rewrites the actual reference. No database index or migration is added.

On the affected library, the revised predicate scanned the segment table in 4.3 seconds and returned zero candidate payloads for tag ID 51. This addresses the observed timeout source; very large tag merges can still have other costs from transferring their direct relationships.

Linked issue

None; the failure was found during a local tag-merge pilot.

Type of change

  • Bug fix
  • New feature
  • Refactor / tech debt
  • Docs
  • Other:

AI usage

  • No AI was used for this PR.
  • AI was used for this PR. (The author will mark this during review.)
    • Model(s): OpenAI Codex
    • Where / how: Investigation, code, and focused test drafting.
  • A human (me) has reviewed, understands, and takes full responsibility for every change here including that the design and architecture are sound. (required)

Testing done & evidence

  • dotnet test src/Cove.Tests/Cove.Tests.csproj --filter FullyQualifiedName~TagMergeServiceTests: 12 passed.
  • Added a test with unrelated imported IDs, a nested tag-ID array, and a tag custom-field value.
  • Read-only PostgreSQL EXPLAIN (ANALYZE, BUFFERS) against the affected library: revised predicate returned zero candidates in 4.3 seconds; the old digit-only plan estimated 1.55 million candidates.
  • The live pilot merge was not retried.

Checklist

  • I have read and followed the Contribution Guide.
  • Builds and existing tests pass. (Focused tests passed; CI will run the complete suite.)
  • I added or updated tests where it makes sense.
  • I updated docs where needed. (No user-facing behavior or documentation changed.)
  • This PR is focused and does not bundle unrelated changes.

@jiwenjimiran
jiwenjimiran marked this pull request as ready for review October 10, 2026 02:48
@jiwenjimiran

Copy link
Copy Markdown
Contributor Author

This one is pretty niche and I can work around it, but might bite other people if they are migrating tags in. Fine to close this one if it's unwanted.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant