gh-108303: Move `Lib/test/sortperf.py` to `Tools/scripts` by sobolevn · Pull Request #114687 · python/cpython

sobolevn · 2024-01-28T20:18:58Z

@corona10 what do you think about moving these case to pyperformance? Would it be benefitial?

Issue: Move test files into test subdirectories #108303

vstinner · 2024-01-29T15:25:49Z

Which change (who) created this script? Is it @corona10?

sobolevn · 2024-01-29T15:35:27Z

It was Guido 27 years ago: https://github.com/python/cpython/commits/main/Lib/test/sortperf.py

I think he agreed on moving it away in Discord's #python-dev channel.

corona10

IMO, the First thing we have to do is add the benchmark script, not remove the file.

bedevere-app · 2024-01-29T17:20:09Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

sobolevn · 2024-01-29T19:04:32Z

@corona10 ok, I will send a PR to pyperformance soon! 👍

This reverts commit 6903baa.

corona10

Even if we just move the script into Tools/script
IMO pyperf based script is more modernized way and easy to compare with improvement patch. WDYT?

sobolevn · 2024-02-09T14:37:32Z

I agree with Guido: it is not really that useful as a benchmark (since it is a micro-benchmark). It is only interesting when working on list.sort directly. And I don't think that this is going to happen anytime soon, since the sorting algorithm is quite old, fast, and stable.

So, I think that we can move this file for now. If someone has a real use-case for list.sort benchmark in pyperf, they can recreate my existing closed PR.

And for now this file would serve more like a history reference.

corona10 · 2024-02-10T06:04:03Z

So, I think that we can move this file for now. If someone has a real use-case for list.sort benchmark in pyperf, they can recreate my existing closed PR.

Ah, I meant rewriting the benchmark script based on pyperf(you already did) and storing it Tools/script/, I did not mean to move new script to pyperformance, I also agree with Guido, it will not be a proper place.

sobolevn · 2024-02-10T09:28:59Z

@corona10 sure!

corona10 · 2024-02-11T12:28:05Z

I will take a look at the PR by tomorrow :)

corona10

By default, all benchmark should be executed.
Benchmarks needs some tune up

.....................
list_sort: Mean +- std dev: 30.8 ms +- 0.4 ms
.....................
list_sort_ascending: Mean +- std dev: 1.76 ms +- 0.10 ms

Execution time is too long :(
(Maybe we need to control inner_loops value)

Tools/scripts/sortperf.py

sobolevn · 2024-02-15T09:05:41Z

@corona10 I've merged your suggestions. Thank you!

I've also optimized the benchmark:

» time python Tools/scripts/sortperf.py  
.....................
list_sort: Mean +- std dev: 29.0 us +- 0.2 us
.....................
list_sort_ascending: Mean +- std dev: 28.9 us +- 0.4 us
.....................
list_sort_ascending_exchanged: Mean +- std dev: 29.2 us +- 0.4 us
.....................
list_sort_ascending_one_percent: Mean +- std dev: 29.5 us +- 0.2 us
.....................
list_sort_ascending_random: Mean +- std dev: 29.6 us +- 0.3 us
.....................
list_sort_descending: Mean +- std dev: 29.5 us +- 0.2 us
.....................
list_sort_duplicates: Mean +- std dev: 30.4 us +- 1.1 us
.....................
list_sort_equal: Mean +- std dev: 29.2 us +- 0.2 us
.....................
list_sort_worst_case: Mean +- std dev: 29.2 us +- 0.2 us
python Tools/scripts/sortperf.py  98.38s user 3.21s system 99% cpu 1:41.75 total

Is it good enough?

corona10

LGTM

@sobolevn Feel free to merge the PR anytime!

sobolevn · 2024-02-18T07:25:11Z

@corona10 Thank you! 🎉

I think that we also need to backport this to 3.11 and 3.12
Do you agree?

corona10 · 2024-02-18T07:25:49Z

Yeah go a head

miss-islington-app · 2024-02-18T07:27:18Z

Thanks @sobolevn for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

…onGH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-app · 2024-02-18T07:27:27Z

GH-115625 is a backport of this pull request to the 3.12 branch.

…onGH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-app · 2024-02-18T07:27:32Z

GH-115626 is a backport of this pull request to the 3.11 branch.

…114687) (#115626) gh-108303: Move `Lib/test/sortperf.py` to `Tools/scripts` (GH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

…114687) (#115625) gh-108303: Move `Lib/test/sortperf.py` to `Tools/scripts` (GH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

…on#114687)

pythongh-108303: Remove Lib/test/sortperf.py

6903baa

sobolevn requested a review from vstinner January 28, 2024 20:18

bedevere-app bot added the awaiting review label Jan 28, 2024

bedevere-app bot mentioned this pull request Jan 28, 2024

Move test files into test subdirectories #108303

Closed

sobolevn added the skip news label Jan 28, 2024

corona10 requested changes Jan 29, 2024

View reviewed changes

bedevere-app bot added awaiting changes and removed awaiting review labels Jan 29, 2024

sobolevn mentioned this pull request Feb 7, 2024

Add list.sort benchmarks python/pyperformance#328

Closed

sobolevn added 2 commits February 9, 2024 14:56

Revert "pythongh-108303: Remove Lib/test/sortperf.py"

145f089

This reverts commit 6903baa.

Move Lib/test/sortperf.py to Tools/scripts

3b38525

sobolevn changed the title ~~gh-108303: Remove Lib/test/sortperf.py~~ gh-108303: Move Lib/test/sortperf.py to Tools/scripts Feb 9, 2024

sobolevn requested a review from corona10 February 9, 2024 11:57

corona10 reviewed Feb 9, 2024

View reviewed changes

sobolevn added 3 commits February 10, 2024 15:17

Use pyperf script

979bca0

Fix CI

052ec30

Merge branch 'main' into issue-108303-sortperf

273eed8

corona10 requested changes Feb 12, 2024

View reviewed changes

Tools/scripts/sortperf.py Outdated Show resolved Hide resolved

Tools/scripts/sortperf.py Outdated Show resolved Hide resolved

Tools/scripts/sortperf.py Outdated Show resolved Hide resolved

Merge branch 'main' into issue-108303-sortperf

6c549d6

Address review

7c2dd81

corona10 approved these changes Feb 17, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting changes labels Feb 17, 2024

sobolevn added needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Feb 18, 2024

sobolevn merged commit f9154f8 into python:main Feb 18, 2024

bedevere-app bot removed the awaiting merge label Feb 18, 2024

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Feb 18, 2024

pythongh-108303: Move Lib/test/sortperf.py to Tools/scripts (pyth…

f52145e

…onGH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-app bot removed the needs backport to 3.12 only security fixes label Feb 18, 2024

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Feb 18, 2024

pythongh-108303: Move Lib/test/sortperf.py to Tools/scripts (pyth…

c846be2

…onGH-114687) (cherry picked from commit f9154f8) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-app bot removed the needs backport to 3.11 only security fixes label Feb 18, 2024

woodruffw pushed a commit to woodruffw-forks/cpython that referenced this pull request Mar 4, 2024

pythongh-108303: Move Lib/test/sortperf.py to Tools/scripts (pyth…

67b6939

…on#114687)

pochmann3 mentioned this pull request Mar 10, 2024

sortperf.py is broken #116576

Closed

sobolevn mentioned this pull request Mar 10, 2024

gh-116576: Fix Tools/scripts/sortperf.py sorting the same list #116577

Merged

diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 17, 2024

pythongh-108303: Move Lib/test/sortperf.py to Tools/scripts (pyth…

3f47339

…on#114687)

LukasWoodtli pushed a commit to LukasWoodtli/cpython that referenced this pull request Jan 22, 2025

pythongh-108303: Move Lib/test/sortperf.py to Tools/scripts (pyth…

4fb68a3

…on#114687)

Uh oh!

Conversation

sobolevn commented Jan 28, 2024 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Jan 29, 2024

Uh oh!

sobolevn commented Jan 29, 2024

Uh oh!

corona10 left a comment

Choose a reason for hiding this comment

Uh oh!

bedevere-app bot commented Jan 29, 2024

Uh oh!

sobolevn commented Jan 29, 2024

Uh oh!

corona10 left a comment

Choose a reason for hiding this comment

Uh oh!

sobolevn commented Feb 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

corona10 commented Feb 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sobolevn commented Feb 10, 2024

Uh oh!

corona10 commented Feb 11, 2024

Uh oh!

corona10 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sobolevn commented Feb 15, 2024

Uh oh!

corona10 left a comment

Choose a reason for hiding this comment

Uh oh!

sobolevn commented Feb 18, 2024

Uh oh!

corona10 commented Feb 18, 2024

Uh oh!

miss-islington-app bot commented Feb 18, 2024

Uh oh!

bedevere-app bot commented Feb 18, 2024

Uh oh!

bedevere-app bot commented Feb 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sobolevn commented Jan 28, 2024 •

edited by bedevere-app bot

Loading

sobolevn commented Feb 9, 2024 •

edited

Loading

corona10 commented Feb 10, 2024 •

edited

Loading

corona10 left a comment •

edited

Loading