Intl: a -u- extension carrying more than one key is read whole (backport of #3594)

[lahma]

Backport of #3594 (main squash 53ef37e79), which closed #3573.

The defect, live on 4.x

new Intl.DateTimeFormat('en-US-u-ca-buddhist-nu-arab') resolves to a bare en-US with gregory and latn, losing both keys. Either key on its own is read; the two together lose both, and the resolved locale loses the extension with them. Intl.NumberFormat has it too, while Intl.Locale and Intl.getCanonicalLocales read the same tag correctly.

Each constructor carried its own scanner, and each ended a key's value by consuming the next key into it — ca read back as "buddhist-nu", which resolves to no calendar, so the option was dropped and the extension never made it into the resolved locale. Eight such scanners existed under Jint/Native/Intl.

Measured on this branch, before and after. DTF is new Intl.DateTimeFormat(tag, { hour: 'numeric' }).resolvedOptions(), NF is new Intl.NumberFormat(tag).resolvedOptions():

requested tag	before	after
en-US-u-ca-buddhist-nu-arab	DTF en-US / gregory / latn	DTF en-US-u-ca-buddhist-nu-arab / buddhist / arab
en-US-u-nu-arab-ca-buddhist	DTF en-US / gregory / latn	DTF en-US-u-ca-buddhist-nu-arab / buddhist / arab
en-US-u-ca-buddhist-hc-h23	DTF en-US / gregory / latn / h12	DTF en-US-u-ca-buddhist-hc-h23 / buddhist / latn / h23
en-US-u-hc-h23-nu-arab-ca-buddhist	DTF en-US-u-nu-arab / gregory / arab / h12	DTF en-US-u-ca-buddhist-hc-h23-nu-arab / buddhist / arab / h23
en-US-u-ca-islamic-civil-nu-arab	DTF en-US / gregory / latn	DTF en-US-u-ca-islamic-civil-nu-arab / islamic-civil / arab
en-US-u-foo-ca-buddhist-nu-arab	DTF en-US / gregory / latn	DTF en-US-u-ca-buddhist-nu-arab / buddhist / arab
en-US-u-nu-arab-x-private	DTF en-US / latn	DTF en-US-u-nu-arab / arab
en-US-u-ca-buddhist-nu-arab	NF en-US / latn	NF en-US-u-nu-arab / arab
en-US-u-nu-arab-ca-buddhist	NF en-US / latn	NF en-US-u-nu-arab / arab

Every "after" value is what ICU — and therefore V8 (Node 24, full ICU) — answers for the same tag.

This changes what scripts see

Stated plainly, as #3556 did for read-only host collections. A script reading resolvedOptions() for a tag that carries more than one -u- key gets different values than it did: the keys it asked for, and a locale that carries them. A tag with one key, or none, resolves exactly as before — those lanes are pinned in the new tests precisely so the change stays confined to the case that was broken. Anything downstream that formats through such a formatter (dates in a Buddhist calendar, numbers in Arabic-Indic digits) formats differently as a result, which is the point of the fix.

ResolveLocale step 13's precedence is untouched, and now pinned: an option still supersedes the tag's keyword and still removes that keyword from the resolved locale — for one key, and for one key beside another.

The fix

The eight scanners are replaced by one. UnicodeExtension owns the rule https://unicode.org/reports/tr35/#unicode_locale_extensions states — a key is two characters, a value is the 3-to-8-character subtags after it, and both end at the next key or the next singleton — and answers for reading a keyword, enumerating them, removing the sequence, and rewriting one key. Intl.Locale's own whole-tag parser and CanonicalizeUnicodeLocaleId take their key test from it as well, so nothing in the engine can drift about where one keyword ends and the next begins.

Divergence from the main PR

This is not a blind pick; four files needed a merge, and all four for the same reason: main's Intl has moved on around this code (the provider chain that is out of scope for 4.x), so git apply -3 pulled in signature changes that belong to other PRs.

file	what main's patch wanted	what landed here
DurationFormatConstructor	private string ResolveNumberingSystem(…, string locale) reading IntlUtilities.IsSupportedNumberingSystem / GetLocaleDefaultNumberingSystem	4.x's private static string ResolveNumberingSystem(string?, string?) kept as-is; only ParseNumberingSystemExtension is deleted and its call site replaced with UnicodeExtension.GetKeywordValue. Main's spec-citation doc comment is kept — it describes step 13 either way.
NumberFormatConstructor, RelativeTimeFormatConstructor	private bool IsSupportedNumberingSystem(…)	kept private static bool; only ExtractNumberingSystemFromLocale is deleted
DateTimeFormatConstructor	delete a region main no longer has CanonicalizeCalendar in	CanonicalizeCalendar kept, ExtractUnicodeExtensionFromLocale deleted
Polyfills.cs	insert the string.Concat(ROS<char>, ROS<char>) block where 4.x has its #if NETFRAMEWORK span-Contains block	inserted ahead of it, both intact

docs/v5-migration.md dropped — it does not exist here. Jint/Native/Intl/UnicodeExtension.cs, CollatorConstructor, DefaultCldrProvider, IntlUtilities and LocaleConstructor applied cleanly and are byte-identical to main's.

TFM check (asked for explicitly, since 4.x is net462;netstandard2.0;netstandard2.1;net8.0;net10.0):

• The string.Concat span polyfill's guard is #if !NET8_0_OR_GREATER, which covers net462, netstandard2.0 and netstandard2.1 here exactly as it covers net472, netstandard2.0 and netstandard2.1 on main — same three downlevel assets, same reason. 4.x's Polyfills already uses the C# 14 extension(string) member syntax the block needs (string.Join(char, …) right above it). No existing Concat polyfill to collide with.
• UnicodeExtension.IsKey is shared by IntlUtilities.CanonicalizeUnicodeLocaleId and LocaleConstructor and uses nothing newer than ReadOnlySpan<char>.
• All five assets build with 0 errors and 0 warnings.

Evidence

Jint.Tests/Runtime/IntlUnicodeExtensionTests.cs (42 cases) run against unfixed 4.x first. To make the file compile before the fix, UnicodeExtension.cs and the polyfill were added on their own — nothing calls the class at that point, so engine behaviour is unchanged and the reader's own unit tests act as a control:

leg	before	after
net472	Failed: 11, Passed: 31, Total: 42	Failed: 0, Passed: 42
net10.0	Failed: 11, Passed: 31, Total: 42	Failed: 0, Passed: 42

The 31 that passed before are the 23 GetKeywordValue / RemoveSequence / WithKeyword cases (the new reader is right before any consumer uses it), plus Intl.Collator's three keys and RelativeTimeFormat / DurationFormat's nu — those two scanners split the sequence on - and looked for a key by name, so they already survived a second key. The 11 failures are Intl.DateTimeFormat (7) and Intl.NumberFormat (2) losing keys, plus the multi-subtag/attribute case and the option-precedence case.

All 280 Jint.Tests Intl cases pass on both legs. Full dotnet build -c Release: 0 errors, 0 warnings. Full dotnet test -c Release:

• Jint.Tests 7041/7041 (net10.0), 6956/6956 (net472)
• Jint.Tests.PublicInterface 1820/1820 (net10.0), 1812/1812 (net472)
• Jint.Tests.CommonScripts 28/28 both legs, Jint.Tests.SourceGenerators 52/52
• Jint.Tests.Test262 102,499 passed / 0 failed / 185 skipped of 102,684 — exactly the 4.x control, with not even the usual supportedLocalesOf load flake this run. The pass count is unchanged, so no exclusion becomes removable: every test the corpus runs through these constructors was already passing with a single-key tag, and the multi-key tags the defect lost are ones test262 does not exercise here.