[Free Threading] test_opcache: test_binary_subscr_list_int(): list_get_item_ref: Assertion `cap != -1 && cap >= size' failed

[vstinner]

test_binary_subscr_list_int() does crash randomly with an assertion error if Python is built with --disable-gil:

$ ./python -m test test_opcache -v -m test_binary_subscr_list_int -j10 -F
(...)
test_binary_subscr_list_int (test.test_opcache.TestRacesDoNotCrash.test_binary_subscr_list_int) ...

python: Objects/listobject.c:342: list_get_item_ref: Assertion `cap != -1 && cap >= size' failed.
Fatal Python error: Aborted

Thread 0x00007f244350f6c0 (most recent call first):
  File "/home/vstinner/python/main/Lib/test/test_opcache.py", line 638 in write
  File "/home/vstinner/python/main/Lib/threading.py", line 992 in run
  File "/home/vstinner/python/main/Lib/threading.py", line 1041 in _bootstrap_inner
  File "/home/vstinner/python/main/Lib/threading.py", line 1012 in _bootstrap

Thread 0x00007f2443d106c0 (most recent call first):
  File "/home/vstinner/python/main/Lib/test/test_opcache.py", line 639 in write
  File "/home/vstinner/python/main/Lib/threading.py", line 992 in run
  File "/home/vstinner/python/main/Lib/threading.py", line 1041 in _bootstrap_inner
  File "/home/vstinner/python/main/Lib/threading.py", line 1012 in _bootstrap

Current thread 0x00007f2451be5740 (most recent call first):
  File "/home/vstinner/python/main/Lib/test/test_opcache.py", line 632 in read
  File "/home/vstinner/python/main/Lib/test/test_opcache.py", line 586 in assert_races_do_not_crash
  File "/home/vstinner/python/main/Lib/test/test_opcache.py", line 22 in wrapper
  File "/home/vstinner/python/main/Lib/test/test_opcache.py", line 642 in test_binary_subscr_list_int
  (...)

Extension modules: _testinternalcapi (total: 1)

Bug seen on macOS CI, and reproduced on Linux.

Linked PRs

• gh-127521: Mark list as "shared" before resizing if necessary #127524
• [3.13] gh-127521: Mark list as "shared" before resizing if necessary (GH-127524) #127533

[vstinner]

cc @corona10 @ZeroIntensity

According to git bisect, the regression was introduced by commit e271340.

commit e2713409cff5b71b1176b0e3fa63dae447548672
Author: Donghee Na <donghee.na@python.org>
Date:   Mon Dec 2 10:38:17 2024 +0900

    gh-115999: Add partial free-thread specialization for BINARY_SUBSCR (gh-127227)

[colesbury]

I think I understand the issue. It's a preexisting bug in the free threading listobject.c implementation. We need to be setting the _PyObject_GC_IS_SHARED flag when resizing if the current thread doesn't own the list.