Skip to content

globalsort: fix stuck when ingesting small kv - #62407

Merged
ti-chi-bot[bot] merged 4 commits into
pingcap:masterfrom
D3Hunter:fix-gsort-stuck-for-smallkv
Jul 15, 2025
Merged

globalsort: fix stuck when ingesting small kv#62407
ti-chi-bot[bot] merged 4 commits into
pingcap:masterfrom
D3Hunter:fix-gsort-stuck-for-smallkv

Conversation

@D3Hunter

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #62388

Problem Summary:

What changed and how does it work?

consider the memory taken by slice header when splitting range

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

same case in the issue, before this pr, 2 out of 3 runs stuck. after this pr, we run 10 times, all success.

  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot Bot added release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 15, 2025
@tiprow

tiprow Bot commented Jul 15, 2025

Copy link
Copy Markdown

Hi @D3Hunter. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@Benjamin2037 Benjamin2037 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot Bot added approved needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 15, 2025
@codecov

codecov Bot commented Jul 15, 2025

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.8131%. Comparing base (0c6cbc1) to head (8cdc3b7).
Report is 11 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #62407        +/-   ##
================================================
+ Coverage   72.9071%   73.8131%   +0.9060%     
================================================
  Files          1754       1785        +31     
  Lines        484998     497265     +12267     
================================================
+ Hits         353598     367047     +13449     
+ Misses       109767     107843      -1924     
- Partials      21633      22375       +742     
Flag Coverage Δ
integration 45.3764% <0.0000%> (?)
unit 72.3016% <100.0000%> (+0.1306%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.6673% <ø> (-0.1132%) ⬇️
parser ∅ <ø> (∅)
br 46.3605% <ø> (+0.0639%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ti-chi-bot ti-chi-bot Bot added the needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. label Jul 15, 2025
@ti-chi-bot

ti-chi-bot Bot commented Jul 15, 2025

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Benjamin2037, joechenrh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jul 15, 2025
@ti-chi-bot

ti-chi-bot Bot commented Jul 15, 2025

Copy link
Copy Markdown

[LGTM Timeline notifier]

Timeline:

  • 2025-07-15 02:23:52.205530249 +0000 UTC m=+2571284.928709231: ☑️ agreed by Benjamin2037.
  • 2025-07-15 02:52:29.975305442 +0000 UTC m=+2573002.698484423: ☑️ agreed by joechenrh.

@Benjamin2037 Benjamin2037 added the needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. label Jul 15, 2025
@Benjamin2037

Copy link
Copy Markdown
Collaborator

/retest

1 similar comment
@Benjamin2037

Copy link
Copy Markdown
Collaborator

/retest

@tiprow

tiprow Bot commented Jul 15, 2025

Copy link
Copy Markdown

@D3Hunter: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
fast_test_tiprow 8cdc3b7 link true /test fast_test_tiprow

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@ti-chi-bot
ti-chi-bot Bot merged commit 65c3cf2 into pingcap:master Jul 15, 2025
27 of 28 checks passed
@ti-chi-bot

Copy link
Copy Markdown
Member

In response to a cherrypick label: new pull request created to branch release-8.5: #62409.

@D3Hunter
D3Hunter deleted the fix-gsort-stuck-for-smallkv branch July 15, 2025 04:38
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this pull request Jul 15, 2025
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot

Copy link
Copy Markdown
Member

In response to a cherrypick label: new pull request created to branch release-8.1: #62410.
But this PR has conflicts, please resolve them!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm needs-cherry-pick-release-8.1 Should cherry pick this PR to release-8.1 branch. needs-cherry-pick-release-8.5 Should cherry pick this PR to release-8.5 branch. release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

global sort importinto stuck at ingest step due to memory limiter

4 participants