_winapi.LCMapStringEx fails when encountering a string containing null characters

[finnagin]

Bug report

This bug was encountered when working through gh-106816, specifically this comment chain.

_winapi.LCMapStringEx() raises the error ValueError: embedded null character when passed a string containing a null character.

This can be reproduced by running the following:

import _winapi
_winapi.LCMapStringEx('abc\x00def')

From the discussion in the linked PR it looks like the LCMapStringEx implementaion in 3.11 has already been improved to handle null characters and so maybe the same change can be brought forward into the 3.12 and 3.13 implementations?

Your environment

• CPython versions tested on: 3.13.0a0
• Operating system and architecture: Windows 11 on amd64

Linked PRs

• gh-106844: Fix null-bytes handling in LCMapStringEx in _winapi #106857
• gh-106844: Fix null-bytes handling in LCMapStringEx in _winapi #107832
• [3.12] gh-106844: Fix issues in _winapi.LCMapStringEx (GH-107832) #107874
• [3.11] gh-106844: Fix issues in _winapi.LCMapStringEx (GH-107832) #107875

[serhiy-storchaka]

There is a problem not only with the null character handling.

I tested on 3.12rc1. Note that -1 passed as the input size, so LCMapStringEx() determines the size of the string by the terminating null character, so it is not a limitation of the API.

>>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**32+5))
>>> len(x)
5
>>> del x
>>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**31+5))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**30+5))
>>> len(x)
1073741829
>>> len(x) - 2**30
5
>>> del x
>>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**31-1))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**31-2))
>>> len(x) - 2**31
-2
>>> del x
>>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**32-2))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError
>>> x = _winapi.LCMapStringEx(_winapi.LOCALE_NAME_INVARIANT, _winapi.LCMAP_LOWERCASE, 'x'*(2**32-1))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [WinError 6] The handle is invalid

• Size up to 2**31-2 including works correctly.
• Size between 2**31-1 and 2**32-2 raises MemoryError. I think that LCMapStringEx() return a correct size, but converted to int it becomes negative, than it is promoted to Py_ssize_t, and interpreted as a very large size_t value, which causes a MemoryError.
• Size 2**32-1 raises OSError. LCMapStringEx() returns the output size including the terminating null character. (int)(2**32-1 + 1) == 0, and it is interpreted as a signal of error.
• Size larger than 2**32-1 loses highest bits.

Rejecting size larger than INT_MAX is safe, in any case LCMapStringEx() does not support it now (and in worst case it silently returns a truncated result). Perhaps size exactly equal to INT_MAX produces an OSError on 32-bit Windows. And with different content we can get an OSError for strings smaller than INT_MAX characters.

Perhaps we can make it supporting strings between 2**31-1 and 2**32-2 characters long if cast sizes to unsigned int. But I am not sure that it is guaranteed and that we need this for normcase().

[serhiy-storchaka]

Perhaps size exactly equal to INT_MAX produces an OSError on 32-bit Windows.

No, it always less than INT_MAX on 32-bit Windows.