Skip to content

CLAUDE.md

Latest commit

 

History

History
99 lines (70 loc) · 3.54 KB

CLAUDE.md

File metadata and controls

99 lines (70 loc) · 3.54 KB

Claude Code Instructions

See README.md for project overview and usage.

Test Results

No issues were found. The lock-free __contains__ implementation passed all stress tests.

Lock-Free Implementation Details

The patch lock_free_change.patch modifies CPython's Objects/setobject.c to make __contains__ lock-free. Key code paths to understand:

Critical Functions

  • set_lookkey_threadsafe() - Lock-free lookup path for free-threaded builds
  • set_compare_threadsafe() - Entry comparison with _Py_TryIncrefCompare
  • ensure_shared_on_read() - Marks set as shared on first cross-thread access
  • set_table_resize() - Resize with QSBR cleanup of old table
  • set_swap_bodies() - Swaps two sets' internals (used by __iand__, etc.)

Key Constants

  • PySet_MINSIZE = 8 - Threshold between smalltable and malloc'd table
  • SET_LOOKKEY_CHANGED = -2 - Signals that set was mutated during lookup

Race Condition Patterns

The implementation handles these concurrent scenarios:

  1. Table pointer changes during lookup - Resize sets table=NULL temporarily
  2. Entry key changes during comparison - Detected via startkey != ep->key check
  3. QSBR delayed freeing - Old tables freed after grace period when set is shared
  4. Mask/table ordering - mask updated before table during resize

Test Files

File Purpose
main.py Main stress test with 6 modes
test_mutating_keys.py Tests keys that mutate set in __hash__/__eq__
test_resize_race.py Targeted resize race condition tests
run_tests.sh Runs all tests

Test Modes (main.py --mode)

Mode Target Verification
standard Basic concurrent read/write Contiguous value ranges
resize set_table_resize() + QSBR Crash/exception only
swap set_swap_bodies() via __iand__ Crash/exception only
discard Dummy entry handling Crash/exception only
hash_collision Linear probing paths Crash/exception only
threshold Small/large table transitions Crash/exception only

Test Design Notes

  • Modes with clear() cannot verify contiguous ranges - Multiple writers calling clear() wipe each other's work. These modes only check for crashes/exceptions.
  • Only standard mode verifies value integrity - Writers have non-overlapping ranges, no clear() calls.
  • Use threading.Barrier - Ensures all threads start simultaneously for maximum contention.

Running Tests

# Quick test
./run_tests.sh --quick

# Full test suite
./run_tests.sh

# Thorough (longer duration, more threads)
./run_tests.sh --thorough

# Single mode
$PYTHON main.py --mode resize --duration 10 --readers 8

# Mutating keys (exercises SET_LOOKKEY_CHANGED path)
$PYTHON test_mutating_keys.py --duration 5

Debugging Tips

  • If a test fails with "Missing N values", check if that mode uses clear() - may be a test bug, not implementation bug.
  • test_resize_race.py --test N runs individual targeted tests to isolate issues.
  • Increase --duration and --readers to increase likelihood of hitting race windows.
  • The hash_collision mode forces linear probing by using keys with identical hashes.

Potential Additional Tests

These are ideas for further stress testing if needed:

  • frozenset operations (immutable, different code paths)
  • Set operations that iterate (union, intersection, difference)
  • Weak references to set elements during concurrent modification
  • TSAN/ASAN builds for memory safety verification