Yarr: JIT lookbehinds, first-code-point alternation dispatch, Boyer-Moore and /u·/v spec alignment, RegExp correctness fixes - #299
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThe change enables supported lookbehind patterns in the Yarr RegExp JIT through mirrored backward execution, adds direction-aware matching and dispatch infrastructure, updates alternation optimization, and introduces stress coverage comparing JIT and interpreter results. ChangesLookbehind JIT support
Suggested reviewers: 🚥 Pre-merge checks | ✅ 3 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (3 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@JSTests/stress/regexp-lookbehind-jit.js`:
- Around line 7-12: Update shouldBe to compare expected and actual values
recursively using Object.is, including each array element, so undefined captures
remain distinct from null. Preserve the existing mismatch error reporting and
message behavior while replacing the JSON.stringify-based equality check.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: d5f1f314-6f67-4953-aba9-cd18c8087599
📒 Files selected for processing (3)
JSTests/stress/regexp-lookbehind-jit.jsSource/JavaScriptCore/runtime/RegExp.cppSource/JavaScriptCore/yarr/YarrJIT.cpp
💤 Files with no reviewable changes (1)
- Source/JavaScriptCore/runtime/RegExp.cpp
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I didn't find any bugs, but this is a substantial JIT codegen change — new backward-matching primitives, mirrored-disjunction construction, and direction-aware addressing across many term types — so it warrants a human reviewer familiar with Yarr.
What was reviewed:
- Backward input-claim/read/end-of-input primitives and the
negativePositionOffset→index + (k-1)addressing transform. - Capture start/end slot swapping in backward frames vs. the interpreter's end-first recording.
mirrorDisjunctionForLookbehindfallback paths (backrefs, nested lookaheads, quantified groups >1, surrogate decoding) — each still sets aJITFailureReasonso the interpreter path remains reachable.- The Boyer-Moore change treating parenthetical assertions as zero-width — checked that ignoring the assertion keeps the leading-char sets an over-approximation.
Extended reasoning...
Overview
This PR adds lookbehind-assertion support to the Yarr regex JIT. Previously, any pattern containing a lookbehind was rejected wholesale (both by a pre-check in RegExp::compile/compileMatchOnly and by JITFailureReason::Lookbehind) and fell back to the bytecode interpreter. The change compiles each lookbehind body as a term-reversed mirrored copy driven through the existing forward machinery, with a per-op MatchDirection that flips the input primitives (claim, read, end-of-input, consume/rewind, capture slot order, and BOL/EOL/word-boundary edge tests). It touches ~500 lines in YarrJIT.cpp, removes two guard lines in RegExp.cpp, and adds a new stress test.
Security risks
Regex JIT is security-sensitive: it emits machine code that indexes into string buffers. The new backward addressing (input[index + (k-1)], branchSub32 on the sign bit for input claims) must never read outside [input, input+length). The PR's argument that index and claim counts are both < 2^31 (so a borrowing sub sets the sign bit) looks sound given containsUnsignedLengthPattern() gates the JIT, and the k==0 peek case (input[index-1]) is only reached by anchors that first check index != 0. I did not find an OOB read path, but this is exactly the kind of invariant a Yarr expert should sanity-check across all the term-type call sites.
Level of scrutiny
High. This is new codegen logic in a JIT compiler with a novel coordinate transform (mirrored/virtual input positions) threaded through many existing generate/backtrack functions. The correctness surface is large — every direction-branched primitive (fixed/greedy/non-greedy pattern-char and char-class, anchors, word boundaries, nested-alternative check-adjust, parenthetical-assertion begin/end) has to agree on what index and inputPosition mean in a backward frame. The differential fuzzing (6000 patterns × 25 inputs × all flags, byte-identical vs. interpreter) and test262 lookBehind suite passing in both modes are strong evidence, but not a substitute for a design-level read by someone who owns this code.
Other factors
- The bug-hunting system found no issues.
- The mirrored-copy approach deliberately leaves the original
YarrPatternuntouched, so unsupported bodies (backrefs, nested lookaheads,maxCount > 1groups, surrogate decoding) cleanly setm_failureReasonand fall through to the pre-existing interpreter path — I verified each of those bailout branches is reachable and sets a reason before returning. - The Boyer-Moore change (
return cursorforParentheticalAssertion) is independently reasonable: assertions are zero-width, so skipping them keeps the leading-character sets a valid over-approximation. - The PR description explicitly flags follow-up refactors (moving mirroring into
YarrPatternConstructor, reversed packed constants) as out of scope, which is a design decision worth a maintainer's nod.
Given the scope and the codegen/security surface, deferring to a human reviewer.
Preview Builds
|
dylan-conway
changed the title
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Source/JavaScriptCore/yarr/YarrJIT.cpp`:
- Around line 5198-5229: Make the dispatch backtrack handling in the
op.m_dispatch branch explicitly enforce the Forward-direction invariant before
adjusting m_regs.index, or replace the unconditional sub32 with the
direction-aware adjustment used by the non-dispatch alternative path. Anchor the
change at the op.m_checkAdjust block and preserve the existing frame-resume jump
behavior.
In `@Source/JavaScriptCore/yarr/YarrPattern.cpp`:
- Around line 2608-2647: Update the repeated-alternative eligibility check
before constructing groupDisjunction so repeatedCount == 0 returns immediately,
while preserving the existing threshold behavior for non-empty ranges. Ensure
the loop that populates groupDisjunction and its final last() access only
execute when at least one repeated alternative exists.
- Around line 2637-2664: Mark the synthesized alternative returned by
body->addNewAlternative(...) as the last alternative by setting
wrapped->m_isLastAlternative to true before appending the group term, while
preserving the existing terminal marker handling for
groupDisjunction->m_alternatives.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: de1af30c-ea73-43b9-8a5c-31469f281caa
📒 Files selected for processing (5)
Source/JavaScriptCore/runtime/OptionsList.hSource/JavaScriptCore/yarr/Yarr.hSource/JavaScriptCore/yarr/YarrJIT.cppSource/JavaScriptCore/yarr/YarrPattern.cppSource/JavaScriptCore/yarr/YarrPattern.h
There was a problem hiding this comment.
♻️ Duplicate comments (1)
Source/JavaScriptCore/yarr/YarrPattern.cpp (1)
2613-2680: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
wrappedalternative never getsm_isLastAlternative = true.The prior crash-on-empty-group finding (
repeatedCount < 2 ||guard) is fixed here. However, the new body-level alternative created at Line 2669 (wrapped) is the only remaining alternative afteralternatives.shrink(firstRepeated)appends it, yet nothing setswrapped->m_isLastAlternative = true. Every absorbed alternative had this flag explicitly cleared (Line 2652), and the codebase's consistent convention (checkForTerminalParentheses,atomParenthesesEnd,factorAlternatives,mergeSharedPrefix, and this same function's owngroupDisjunction->m_alternatives.last()->m_isLastAlternative = trueat Line 2660) confirmsaddNewAlternativedoes not default this to true. Net effect: after this transform, no alternative inbody->m_alternativesis marked as last.🐛 Proposed fix
PatternAlternative* wrapped = body->addNewAlternative(hasCaptures ? firstCaptureId - 1 : m_pattern.m_numSubpatterns); wrapped->m_lastSubpatternId = groupLastSubpatternId; wrapped->m_containsBOL = containsBOL; wrapped->m_startsWithBOL = startsWithBOLCount == groupDisjunction->m_alternatives.size(); + wrapped->m_isLastAlternative = true; groupDisjunction->m_parent = wrapped;#!/bin/bash # Confirm the default value of m_isLastAlternative and how it's consumed for body-level alternatives. rg -n 'm_isLastAlternative' Source/JavaScriptCore/yarr/YarrPattern.h Source/JavaScriptCore/yarr/YarrJIT.cpp Source/JavaScriptCore/yarr/YarrInterpreter.cpp🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Source/JavaScriptCore/yarr/YarrPattern.cpp` around lines 2613 - 2680, Mark the newly created body-level alternative as terminal in wrapAlternativesForDispatch by setting wrapped->m_isLastAlternative to true after addNewAlternative. Preserve the existing terminal marking for the nested group’s final alternative.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@Source/JavaScriptCore/yarr/YarrPattern.cpp`:
- Around line 2613-2680: Mark the newly created body-level alternative as
terminal in wrapAlternativesForDispatch by setting wrapped->m_isLastAlternative
to true after addNewAlternative. Preserve the existing terminal marking for the
nested group’s final alternative.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 159f500d-9431-4417-ae9e-a88a6e336ff1
📒 Files selected for processing (4)
JSTests/stress/regexp-lookbehind-jit.jsSource/JavaScriptCore/runtime/OptionsList.hSource/JavaScriptCore/yarr/YarrJIT.cppSource/JavaScriptCore/yarr/YarrPattern.cpp
There was a problem hiding this comment.
Thanks — the fixup commit addresses the two functional items from the last pass (the repeatedCount < 2 guard in wrapAlternativesForDispatch and WTF_MAKE_TZONE_ALLOCATED_IMPL(DispatchInfo)). No new issues found on this revision, but this is a large change to Yarr JIT codegen and pattern rewriting that should get a human review.
Checked this round:
- Mirrored-frame primitives (claim/release,
negativePositionUntilInputBeginaddressing, capture start/end swap, BOL/EOL/\b) for forward/backward symmetry. wrapAlternativesForDispatch: the concern that the synthesized body alternative lacksm_isLastAlternativewas examined and ruled out — the flag is not load-bearing on the body disjunction path.tryPrepareDispatchgating (Forward-only, FixedCount 1:1, non-terminal/non-string-list,m_minimumSize > 0) vs. the assumptions the dispatch generate/backtrack paths make.- The remaining open inline comment (duplicate comment sentence) is cosmetic only.
Extended reasoning...
Overview
This PR adds three substantial features to the Yarr regex engine across ~1,500 lines of new logic in YarrJIT.cpp and YarrPattern.cpp: (1) JIT compilation of lookbehind assertions via term-reversed "mirrored" bodies with a per-op MatchDirection that flips the input-claim / character-addressing / anchor / capture primitives; (2) an engine-neutral pattern rewrite pass (factorAndWrapAlternatives) that stably sorts literal-leading alternatives, hoists shared prefixes into synthesized (?:…) groups, and folds large top-level alternations into a single group; and (3) a first-character dispatch scheme in the JIT that reads one character, routes through a binary decision tree to per-character chains, and threads chain continuation through a new BackTrackInfoParenthesesOnce::chainResume frame slot. It also removes the blanket m_containsLookbehinds JIT bailout in RegExp.cpp, adds four new Options, bumps YarrStackSpaceForBackTrackInfoParenthesesOnce 2→3, and fixes a pre-existing m_startsWithBOL bubbling bug in atomParenthesesEnd/quantifyAtom.
Prior review and current state
My previous run flagged three items. The fixup commit (b9615a1) addressed the two functional ones: wrapAlternativesForDispatch now guards repeatedCount < 2 so --regExpAlternationGroupThreshold=0 on an all-once-through body no longer reaches .last() on an empty vector, and WTF_MAKE_TZONE_ALLOCATED_IMPL(DispatchInfo) is now present. The third (a duplicated comment sentence) remains and is purely cosmetic. This run's finder/verifier pass raised and refuted the "body left with no m_isLastAlternative" concern for wrapAlternativesForDispatch — the body-disjunction path in the JIT does not rely on that flag the way nested alternatives do. No other findings.
Security risks
Regex JIT codegen is security-sensitive: it emits machine code that reads attacker-controlled input strings against attacker-controlled patterns. The new backward addressing (input[index + (k-1)] with k==0 mapping to index-1), the signed borrow-based bounds check, the frame-slot indirect jump through chainResume, and the pattern-tree restructuring (reparenting, capture-span recomputation, terminal-mark clearing) all have OOB-read / control-flow-hijack surface if any invariant is off by one. Nothing concrete was found, and the PR's differential harnesses (JIT vs. interpreter over hundreds of thousands of cases) plus the full stress/test262 pass are strong evidence — but this is exactly the kind of change where a second pair of expert eyes on the index arithmetic and frame layout is warranted.
Level of scrutiny
High. This is production-critical engine code (every regex in every Bun program), the diff is large, it introduces new JIT control-flow patterns (patched frame stores driving indirect jumps), and it changes observable regex semantics for a family of BOL-anchored patterns (a bug fix, but still a behavior change). It is well outside the "simple/mechanical" bar for auto-approval.
Other factors
The verification story is unusually thorough (randomized differential fuzzing against both the interpreter and node, enumerated small-alternation coverage, full stress + test262), and everything is behind runtime options with sensible defaults. The design is internally well-documented. Outstanding non-blocking bot comments remain (CodeRabbit's defensive-assert suggestion on the dispatch backtrack path, the test helper's undefined-vs-null comparison, and my comment-cosmetics nit). None of those block merge, but the scope alone means this needs a human reviewer familiar with Yarr.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
Source/JavaScriptCore/yarr/YarrPattern.cpp (1)
2665-2673:⚠️ Potential issue | 🟠 MajorRestore the terminal marker on the synthesized body alternative.
wrappedis created withoutm_isLastAlternative, while the previous terminal marker was cleared from the moved alternatives. The body can therefore have no terminal alternative. Setwrapped->m_isLastAlternative = truebefore appending the group.Proposed fix
PatternAlternative* wrapped = body->addNewAlternative(hasCaptures ? firstCaptureId - 1 : m_pattern.m_numSubpatterns); wrapped->m_lastSubpatternId = groupLastSubpatternId; +wrapped->m_isLastAlternative = true; wrapped->m_containsBOL = containsBOL;🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Source/JavaScriptCore/yarr/YarrPattern.cpp` around lines 2665 - 2673, Set wrapped->m_isLastAlternative to true immediately after creating the synthesized alternative in the surrounding group-wrapping logic, before appending the ParenthesesSubpattern group, so the synthesized body retains the terminal-alternative marker.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@JSTests/stress/regexp-lookbehind-jit.js`:
- Around line 148-160: Update the pre-existing JIT defect comment above the
useRegExpJIT assertions to include a link or identifier for a tracking issue
covering the three divergent regular-expression cases. Keep the existing
per-tier expectations and test logic unchanged.
---
Duplicate comments:
In `@Source/JavaScriptCore/yarr/YarrPattern.cpp`:
- Around line 2665-2673: Set wrapped->m_isLastAlternative to true immediately
after creating the synthesized alternative in the surrounding group-wrapping
logic, before appending the ParenthesesSubpattern group, so the synthesized body
retains the terminal-alternative marker.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: beb7c340-33af-444d-b0b3-9869b71be2aa
📒 Files selected for processing (4)
JSTests/stress/regexp-lookbehind-jit.jsSource/JavaScriptCore/yarr/YarrJIT.cppSource/JavaScriptCore/yarr/YarrPattern.cppSource/JavaScriptCore/yarr/YarrPattern.h
There was a problem hiding this comment.
Thanks for the fixes in b9615a1 — the repeatedCount < 2 clamp and WTF_MAKE_TZONE_ALLOCATED_IMPL(DispatchInfo) are both in place now. I didn't find any new issues in this revision, but given the scope (backward-matching JIT primitives, engine-neutral pattern rewrites, first-character dispatch codegen and backtracking) this really wants eyes from someone who knows Yarr well.
Checked this run: the new m_containsBOLGroupBubble fallback keeps BOL-group lookbehind patterns on the interpreter without narrowing which non-lookbehind patterns reach the JIT; the synthesized wrapper alternative's unset m_isLastAlternative was examined and found harmless for body-level consumers; the test's JSON.stringify comparator masking undefined vs null was noted as a test-quality nit only. The remaining open inline notes (duplicate comment line, dispatch-backtrack direction assert) are cosmetic.
Extended reasoning...
Overview
This PR adds ~1,500 lines to YarrJIT.cpp and ~340 to YarrPattern.cpp, spanning three interacting features: (1) JIT compilation of lookbehind bodies via term-reversed mirroring with a per-op MatchDirection that flips input-claim/read/anchor/capture primitives; (2) an engine-neutral pattern rewrite that stably sorts and factors literal-leading alternatives and folds large top-level alternations into a synthetic (?:...) group; and (3) a first-character dispatch JIT path that reads one character, routes through a binary decision tree to per-character chains, and threads failure through a new chainResume frame slot. It also adds four runtime options, grows BackTrackInfoParenthesesOnce by one slot, adds a m_containsBOLGroupBubble pattern flag, and includes a stress test.
Security risks
This is JIT code generation for attacker-controlled input (regex patterns and subject strings) in a JavaScript engine. The backward-matching path introduces new signed-arithmetic index handling and a new BaseIndex addressing mode; the dispatch path stores and indirect-jumps through code addresses in the match frame; the pattern rewrite restructures the AST that downstream passes (setupOffsets, terminal/string-list optimizations, JIT op-compile) consume. Any of these could plausibly harbor OOB reads, index-register corruption, or backtracking-state confusion that only manifests on specific pattern shapes. The differential harnesses described in the PR are strong evidence of correctness, but this class of change is exactly where subtle exploitable bugs hide.
Level of scrutiny
High. This is neither mechanical nor small; it touches production-critical, security-sensitive codegen with many interacting invariants (frame offsets, checked-input accounting, direction state, capture-span bookkeeping, terminal/string-list marks). It is well outside the "approve without human review" bar.
Other factors
Since my previous inline comments the author pushed a fix commit addressing the threshold=0 crash and the missing TZone _IMPL, plus a follow-up (1e122871) that replaces the earlier BOL-bubbling narrowing with a per-pattern m_containsBOLGroupBubble flag so lookbehind patterns of that shape stay on the interpreter — a more conservative fix. The bug-hunting system found nothing this run; verifiers specifically examined and refuted the "wrapper alternative not marked m_isLastAlternative" concern and the test comparator's undefined/null conflation. The remaining open inline items (a duplicated comment sentence in wrapAlternativesForDispatch, and CodeRabbit's suggestion to assert Forward direction in the dispatch backtrack) are non-blocking cosmetics. Deferring to a human Yarr reviewer.
There was a problem hiding this comment.
Beyond the two inline nits, I also checked the open coderabbit concern about wrapped->m_isLastAlternative in wrapAlternativesForDispatch — m_isLastAlternative is only read for {Simple,}Nested/StringList alternative ops (YarrJIT.cpp:4792/4826), never for body alternatives, so leaving it unset on the synthesized body alternative is harmless. The earlier regExpAlternationGroupThreshold=0 and missing WTF_MAKE_TZONE_ALLOCATED_IMPL(DispatchInfo) findings are addressed in the current diff.
Extended reasoning...
The two new inline findings are cosmetic (a decimal-vs-hex mismatch in a verboseRegExpCompilation dump, and two empty #if/#endif pairs left after removing their bodies). The separately-raised concern that wrapAlternativesForDispatch leaves the body with no m_isLastAlternative marker was verified not to matter: that flag is consulted only in the nested/string-list alternative codegen paths, and the synthesized alternative is a body alternative compiled via BodyAlternativeBegin/Next/End, which never reads it.
dylan-conway
changed the title
dylan-conway
changed the title
There was a problem hiding this comment.
No new issues found in this pass, but this is a large change to the Yarr JIT — new backward-matching codegen for lookbehinds (including unicode), pattern-level alternation rewrites, first-character dispatch, frame-layout growth, and a Boyer-Moore search rewrite — so it needs a human reviewer.
What was reviewed:
- Direction-aware input primitives and
negativeOffsetIndexedAddressbackward mapping — offsets stay within claimed input and the negative-frontier borrow is undone on failure. factorAlternatives/wrapAlternativesForDispatch— stable sort preserves leftmost-first ordering within same-first-character groups; terminal/string-list shapes are left untouched.- Frame-size bumps in
Yarr.hline up with the newBackTrackInfoParenthesesOnce::chainResumeandBackTrackInfoBackReference::backwardSpanEdgeslots. findBestCharacterSequencerewrite — empty-union ranges are skipped so an all-above-Latin1 class can't score as the best window.
Extended reasoning...
Overview
The PR spans seven files but the substance is in YarrJIT.cpp (~1.8k new lines) and YarrPattern.cpp (~400 new lines). It: (1) removes the compile-time lookbehind gate in RegExp.cpp and JIT-compiles lookbehind bodies via a term-reversed mirror driven through the existing forward machinery, with a per-op MatchDirection flipping the input-claim / addressing / end-of-input / capture-slot primitives; (2) adds a leftward unicode code-point reader and backward variants of every character-class and pattern-character quantifier form; (3) adds engine-neutral alternation prefix factoring and a body-folding wrapper in the pattern constructor; (4) adds first-character dispatch for once-groups in the JIT (binary decision tree over Latin-1 → per-character chain of applicable alternatives, with a new chainResume frame slot); (5) rewrites the Boyer-Moore sub-range scoring and adds a subject-frequency gate on the SIMD search loop; and (6) fixes several pre-existing correctness bugs (BOL/word-boundary position tests, firstCharacterAdditionalReadSize scoping, quantified-split capture clearing, dotAll dangling-surrogate rejection).
Security risks
Regex JIT is memory-safety–critical. The new backward addressing path computes input[index + (k-1)] from a leftward-moving frontier and relies on signed-branch underflow detection for input claims; an off-by-one in the offset mapping or a missed borrow-undo would be an OOB read against attacker-controlled subject strings. The dispatch machinery stores and later jumps through a code address in the runtime frame (chainResume), which is the same shape as the existing returnAddress slot but is a new indirect-jump surface. The alternation-factoring rewrite mutates the pattern tree (reparenting, capture-range recomputation) before offset assignment; a mis-bracketed capture span would corrupt the output vector. I did not find a concrete flaw in any of these paths, and the PR's differential and fuzz coverage is extensive, but the surface area is exactly the kind that has historically produced exploitable JIT bugs.
Level of scrutiny
High. This is production-critical engine code, not sandbox or tooling, and it removes a safety fallback (lookbehinds previously always ran in the interpreter). The mirroring design, the frame-layout changes shared with the interpreter, and the pattern-rewrite semantics are all architectural decisions a maintainer should sign off on.
Other factors
The verification story is strong — full test262 RegExp on both tiers, three JIT-vs-interpreter differential corpora, a large enumerated small-alternation sweep, and randomized fuzzing across quantifiers/lookarounds/unicode — and the added regexp-lookbehind-jit.js stress file pins the reproducers. All prior review threads are resolved; the two inline nits still open (unguarded readUnicodeCharBackward call sites on non-x64/arm64/riscv64 builds, and a cosmetic double blank line) don't affect Bun's shipped configs. Net: nothing blocking from automated review, but scope and risk class put this well outside auto-approval.
There was a problem hiding this comment.
No new issues found this pass, but this is a ~2,000-line change to the Yarr JIT's core codegen and pattern layer (mirrored lookbehind compilation, direction-aware primitives, alternation prefix factoring/reordering, first-character dispatch chains, Boyer-Moore range scoring), so it needs a human reviewer.
What was reviewed:
- Direction-aware input-claim/addressing/backtrack primitives and the mirrored-body offset assignment — checked that forward paths are byte-identical when
m_direction == Forward. - Alternation factoring's stable sort + barrier rules and
wrapAlternativesForDispatch— the compile-timealternationFactoringMinRun = 8guard makes the empty-.last()path unreachable. - Boyer-Moore
findBestCharacterSequencerewrite — theisAllSet()break and empty-mapcontinuecover the any-character / above-Latin1-only cases the tests pin. - Frame-slot growth (
YarrStackSpaceForBackTrackInfo*) matches the newBackTrackInfo*struct fields.
The two open inline nits (missing #if ENABLE(YARR_JIT_UNICODE_EXPRESSIONS) around five readUnicodeCharBackward call sites; a stray double blank line) are cosmetic / non-Bun-target only.
Extended reasoning...
Overview
This PR adds lookbehind JIT compilation to Yarr (previously an interpreter-only fallback), engine-neutral alternation prefix factoring and top-level folding in YarrPattern.cpp, first-character dispatch for wide alternations in YarrJIT.cpp, and reworks Boyer-Moore search-range scoring and SIMD-vs-scalar loop selection. It touches YarrJIT.cpp (~1,700 net lines), YarrPattern.cpp (~450 net lines), YarrPattern.h, Yarr.h, YarrJIT.h, RegExp.cpp, and adds two stress tests.
Security risks
Regex JIT is security-sensitive: it emits machine code from attacker-controllable patterns and subjects. The new backward-frame addressing (negativeOffsetIndexedAddress Backward branch, claimBackwardPairLead, checkNotEnoughInputBackward), the dispatch chain's patched frame stores (chainResume), and the pattern-level alternative reordering all create new surfaces for out-of-bounds reads or backtracking-state corruption if an invariant is violated. The PR's own verification (test262, differential JIT-vs-interpreter fuzzing, 52k BM cases) is substantial, but the design decisions — mirroring via deep-copied disjunctions, per-op direction stamping, forward-only dispatch — deserve a human reviewer familiar with Yarr's frame layout and backtracking model.
Level of scrutiny
High. This is core-engine JIT codegen with correctness and memory-safety implications, plus several large-scale design choices (mirrored-body compilation strategy, always-on pattern rewrites that reorder alternatives, frame-slot growth affecting the interpreter). It is well outside the "simple, mechanical, or obvious" bar for auto-approval.
Other factors
All prior review threads on this PR are resolved except two of my own cosmetic nits (unguarded readUnicodeCharBackward call sites for non-x64/arm64 targets, and a leftover double blank line); neither affects Bun's shipped builds. The bug-hunting pass on the current head found nothing new. Preview builds are green across all commits. No human has reviewed yet.
|
(Claude wrote this comment.) Pre-1.4 review, part 1/7 — cross-cutting engine integration & resource safety. Six more focused passes (lookbehind codegen, YarrPattern rewrites, first-char dispatch, Boyer-Moore, differential fuzzer, historical-bug regression scan) are running against Debug+ASAN and Release builds of ab39369 and will be posted as they finish. Verified sound
Findings
Reproducers/benchmarks for all of the above are in |
|
(Claude wrote this comment.) Part 2/7 — lookbehind / mirrored-body codegen ( Method: full read of the Backward paths, then a 3-way differential fuzzer (branch JIT vs Findings
Believed sound (checked explicitly)Every Backward read is Harness (reusable post-rebase): |
|
(Claude wrote this comment.) Part 3/7 — Boyer-Moore first-character search ( Method: end-to-end read, then ~360k randomized 4-way differential cases (branch JIT / branch interp / pre-PR JIT / node; 8-bit, 8-bit + one 16-bit char, full 16-bit; lengths 0–300; lastIndex ≠ 0; g/y/u/v/i/s/m; ⅓ of cases with 8–15 alternatives to trigger factoring), an exhaustive short-subject sweep (49 BM-eligible patterns × 11 flag sets × ~9.5k subjects of length 0–5, under 5 different sampler warm-ups so both vector and scalar loops were exercised), and 5k cases on Debug+ASAN. Verdict on the BM changes themselves: sound. Zero JIT-vs-oracle divergences attributable to BM, zero ASAN reports, zero asserts. Specifically checked: Finding (same root cause as part 2 #1, independently found, with the realistic shape)Wrong result / false negative in the "x😀 y😀 z-".match(/\p{Extended_Pictographic}|[a-z]-/gu) // branch & baseline JIT: ["z-"] interp/node: ["😀","😀","z-"]
/😀|k_/iu.exec("a😀") // branch & baseline JIT: null interp/node: ["😀"]Pre-PR JIT is equally wrong (for the older any-read reason), so not a regression, but this one root cause accounts for essentially all ~150 residual JIT≠node cases across these sweeps, and the emoji-alternation shape is realistic. Fix: key Pre-existing, out of scope, noted for the record: interpreter matches Harness: |
|
(Claude wrote this comment.) Part 4/7 — engine-neutral YarrPattern rewrites (prefix factoring 🔴 Finding 1 — cubic time / cubic peak memory in prefix factoring (PR-introduced; DoS; both tiers). I consider this a merge blocker.
Re-verified on this machine, branch Release vs pre-PR Release,
A < 1 MB regex source OOM-kills a Linux process, from 🔴 Finding 2 — wrong match (false positive) for a negative assertion whose body is BOL-anchored via a nested positive lookahead (PR-introduced in 5cd184b; both tiers; low real-world likelihood).
/(?!(?=^a))a/.exec("a") // branch JIT+interp: ["a"] baseline JIT+interp: null node: null
/(?!(?=^))a|q/.exec("a") // same(Re-verified locally.) The partial-filter sibling Pre-existing, adjacent, not from this PRInterpreter tries once-through alternatives in the wrong order after the first position ( Checked and sound
Harness: |
|
(Claude wrote this comment.) Part 5/7 — historical-bug regression scan. Harvested 517
No in-tree regression test passes on baseline and fails on the branch. No ASAN reports, crashes, or hangs across any of it. 🔴 Finding — resurfaced bug class G (b319370): unicode lookbehind ending in a variable-width greedy class is O(n³) in the new backward JIT path. New regression vs pre-PR.
Latin1 subjects and non- Medium — pre-existing interpreter bugs now visible as tier divergence (JIT correct = V8)Same family as parts 2/3: non-BMP inside lookbehind in the bytecode interpreter, e.g. Info — pre-existing, both tiers, deliberately preservedZero-width patterns match mid-surrogate-pair under Not run: WebCore-only tests (b308707 etc.), |
|
(Claude wrote this comment.) Part 6/7 — first-character dispatch, inline literal alternatives, Verdict: no memory-safety or wrong-result bug found; this sub-feature looks sound. ~22.5k random cases on Debug+ASAN JIT (0 crashes/asserts), ~200k branch-vs-merge-base-build (0 regressions; every diff is a branch improvement), ~240k branch-vs-node, plus hand suites for icase / backtracking+quantifiers+captures+backrefs / big alternations / assertions / edges — identical across branch JIT, branch interp, main, node except pre-existing items below. Checked explicitly:
Findings
Not this PR, surfaced by the differential
Harness: |
|
(Claude wrote this comment.) Part 7/7 — black-box differential fuzzer + suites. Volume: 564k branch-JIT-vs-branch-interp cases, 564k branch-vs-baseline-JIT, 500k three-way with node 26.5, 123k on Debug+ASAN (JIT vs interp), across five generators (lookbehind-heavy incl. nested lookaround/backrefs/anchors/all flags/lastIndex sweeps/astral + lone surrogates; trie-generated shared-prefix alternations up to 150 alts; Boyer-Moore bait on Latin1 vs Latin1+one-16-bit-char; the real isbot pattern × 100k UA strings × i/iu/iv/g/y/d; mutations of every regex literal in JSTests/stress + LayoutTests/fast/regex). Plus JSTests/stress Crashes / asserts / ASAN: none. 0 non-zero exits, 0 ASSERT/ASAN across all of it. No engine-specific hangs. isbot: zero divergences across all five engines on 100k UAs — the motivating case is solid. Suites: no stress test passes on baseline and fails on branch; test262 only delta is explained below. Triage of the "new vs baseline" divergences it reportedMy baseline binary is current
Harness (parallel, seeded, re-runnable post-rebase): |
|
(Claude wrote this comment.) Review summary & recommendation for Bun 1.4Seven independent passes (parts 1–7 above), ~1.9M differential regex executions across branch-JIT / branch-interp / merge-base / current-main / node, ~190k of them under Debug+ASAN, all 290 historical Yarr regression tests, test262 RegExp on both tiers, the real isbot pattern on 100k UAs, and targeted adversarial reads of every new code path. Every finding below was re-verified by hand on this machine. Memory safetyNothing found. Zero ASAN reports, zero asserts, zero crashes. Backward reads are bounded by 0 (checked with planted sentinels at Must fix before merge (all PR-introduced, all small)
Should fix (cheap, same PR)
Accept / follow-up issues
RecommendationDon't merge ab39369 as-is. Do merge for 1.4 if items 1–5 land in the next couple of days, followed by a clean re-run of the seven harnesses in |
Patterns containing a lookbehind previously fell back entirely to the bytecode interpreter (JITFailureReason::Lookbehind at compile time, and a pre-check in RegExp::compile that never even attempted the JIT). A single lookbehind therefore sent an otherwise cheap pattern to the interpreter; the isbot user-agent regex (~110 alternatives, four of them lookbehinds) ran ~40x slower than with the assertions JIT'd. A lookbehind body matches right-to-left ending at the assertion point. Matching backward over the input is equivalent to matching a term-reversed copy of the body forward over the reversed prefix, so each body is compiled as a mirrored copy (terms reversed, input positions reassigned with the same width rules as setupAlternativeOffsets) driven through the existing forward machinery. The original YarrPattern is untouched, keeping the bytecode fallback valid. Each op carries a MatchDirection; in a backward body the index register is the leftward-moving left frontier, and only the primitives flip: input claims subtract (branchSub32 on the borrow's sign bit), characters are read at input[index + (k - 1)], end-of-input is index == 0, greedy consumption and give-back reverse sign, captures write the real position with start/end slots swapped (backward captures record end-first, matching the interpreter), and BOL/EOL/word-boundary anchors get mirrored edge tests. Nested lookbehinds are mirrored on demand. Still handled by the interpreter, exactly as before: surrogate-pair decoding (unicode 16-bit), backreferences inside a body, lookaheads nested inside a body, and quantified groups with maxCount > 1 inside a body. Character fusion (multi-char wide loads) is disabled inside backward bodies since ascending memory holds descending pattern positions. collectBoyerMooreInfoFromTerm now treats a parenthetical assertion as zero-width (like BOL/EOL/word-boundary) instead of aborting collection: an assertion consumes no input, so the leading-character sets remain a valid over-approximation.
…group A large top-level alternation is matched by attempting each alternative in turn at every candidate position, so cost grows linearly with the alternative count regardless of how quickly each alternative fails. Two pattern-level rewrites cut the number of alternatives entered. Alternation prefix factoring: alternatives are tried leftmost-first, but their order is only observable between alternatives that can match at the same starting position. Alternatives that must begin with different literal characters have disjoint starting points, so a maximal run of consecutive alternatives that each begin with a fixed, case-sensitive literal character is stably sorted by that character (stability keeps same-first-character alternatives in source order) and each group sharing its leading literal is merged into one alternative whose common prefix is hoisted over a non-capturing group of the suffixes: /aq|bx|ar|by/ becomes /a(?:q|r)|b(?:x|y)/. Any alternative that does not begin with such a character (class, group, anchor, optional atom, the empty alternative) is a barrier no reordering crosses, and the rewrite recurses into the factored suffixes. Group folding: the body's repeated (non-once-through) alternatives, when numerous, are folded into a single alternative holding one non-capturing group -- /X|Y|Z/ is exactly /(?:X|Y|Z)/ -- whose capture range brackets the captures it absorbs. Nested alternatives claim and release their own input, giving each alternative an index-neutral entry that a following change dispatches on the first character. Both rewrites are engine-neutral pattern equivalences (the interpreter and the JIT see the same factored form) and run before setupOffsets(), which lays the synthesized groups out like hand-written ones. Controlled by useRegExpAlternationFactoring / regExpAlternationGroupThreshold.
Trying each alternative of a nested disjunction in sequence costs an entry attempt (claim, character load, compare, release, jump) per alternative at every candidate position, so a large alternation scales linearly with its alternative count even when each alternative fails on its first character. For a once-quantified fixed group whose alternatives all consume at least one character, compute per alternative a first-character set (an over-approximation of the characters a match can start with; Latin-1 tracked precisely, wider characters and the empty prefix widen to "any"). Every alternative's first character sits at the group's start frame position, which is inside input the enclosing alternative already claimed, so it is read once with no bounds check. A binary decision tree over that character's value then jumps to the ordered chain of exactly the alternatives whose set contains it; an "any" alternative appears in every chain, so leftmost-alternative-wins ordering is preserved. When no alternative can start with the character the group fails immediately. Each chain is a sequence of stubs that store the address of the next stub into a new frame slot (BackTrackInfoParenthesesOnce::chainResume, patched at link time) before jumping into the alternative; a failing alternative releases its claim and continues its own chain through that slot rather than falling to a fixed successor, and an exhausted chain joins the group's failure flow. Entry into an alternative is now a labelled chain target rather than a fallthrough, and backtracking back into a matched alternative reuses the still-valid resume address. Combined with the alternation prefix factoring, the isbot user-agent regex over the 361,829-string corpus goes from ~620 ms (factoring alone) to ~120 ms; the sequential JIT was ~750 ms and the interpreter ~32,000 ms. Controlled by useRegExpAlternationDispatch / regExpAlternationDispatchThreshold; interpreter behaviour is unchanged and JIT results match it byte-for-byte across differential corpora.
… and dead checks, test probe, harness imports - accumulateCaptureRange reads each parenthesis term's [subpatternId, lastSubpatternId] bracket instead of recursing through the nesting (no isSafeToRecurse needed before setupOffsets()). - emitInlineLiteralAlternative's 64-bit fused loads sit behind the same 64-bit-CPU guard as the term path's; generateBitInTableSIMDSearch drops the checks canUseVectorScan() already made; the wide-routing fallback restores a stubCount snapshot rather than re-summing. - regexp-unicode-property-escape-ignore-case.js's sixth probe is U+01C4 (uppercase DZ) rather than a repeat of U+01C5, with the expectation table regenerated. - drive.mjs writes filename-safe output names for file: profiles; run262.mjs imports tidied; stray blank lines / a column-0 line.
…apture-range uses containsAnyCaptures(); Tools/yarr-fuzz gains the corpus mutator and the reference-matcher oracle No behaviour change in the engine (the same instructions are emitted). Tools/yarr-fuzz/regex-fuzz.js is synced with the version the driver expects (file: replay profiles, strings/wide profiles), and mutate.mjs / refmatch.mjs / refgen.mjs / refrun.js / refcheck.mjs are added with a README section.
Jarred-Sumner
changed the title
… entry's leading slot m_namedGroupToParenIndices maps a name to [id, id] or, once duplicated, to [duplicateNamedGroupId, id1, id2, ...]; comparing the closing group's subpatternId against slot [0] could take a duplicate-group id for a subpattern id and resolve \k<name> to an unnamed enclosing group. No observable difference (such a group encloses the reference, so both tiers read it as unset), but the lookup no longer depends on that.
…P latch's feature guard to its member's, whitespace; drive.mjs unused imports No-Verification-Needed: dead-code/whitespace/import removal only; identical generated code.
| const HERE = (typeof arguments !== "undefined" && arguments[0]) || (typeof process !== "undefined" && process.argv[2]) || "."; | ||
| const isbot = JSON.parse(RF(HERE + "/isbot-pattern.json")); |
There was a problem hiding this comment.
🟡 Under Node the HERE detection at L5 picks up the CommonJS module wrapper's arguments object — arguments[0] is the (truthy) exports object, not a CLI arg — so HERE becomes {} and L6 calls readFileSync("[object Object]/isbot-pattern.json") → ENOENT. The file's own L1 comment documents node bench.js as a supported invocation. Fix: check process.argv[2] first, or guard with typeof arguments[0] === "string". Dev-tool only; zero product impact.
Extended reasoning...
What
Tools/yarr-fuzz/bench.js L5:
const HERE = (typeof arguments !== "undefined" && arguments[0]) || (typeof process !== "undefined" && process.argv[2]) || ".";The intent is: under jsc use the shell's global CLI-argument array (arguments[0]), under Node use process.argv[2], else fall back to ".". But under Node.js a .js file loaded as CommonJS is wrapped in (function(exports, require, module, __filename, __dirname){ … }), so at module scope typeof arguments === "object" and arguments[0] is the module's exports object — an empty {}, which is truthy. The || chain therefore short-circuits to {}, the process.argv[2] branch is never reached, and HERE is an object rather than a path string.
L6 then evaluates RF(HERE + "/isbot-pattern.json"). String concatenation on an object gives "[object Object]/isbot-pattern.json", and readFileSync throws ENOENT immediately.
Step-by-step proof
- Write the exact expression to a
.jsfile (notnode -e, which has no module wrapper):—$ printf 'const HERE=(typeof arguments!=="undefined"&&arguments[0])||(typeof process!=="undefined"&&process.argv[2])||".";console.log(typeof arguments, JSON.stringify(HERE), HERE+"/x")' > t.js $ node t.js somedir object {} [object Object]/xargumentsexists at module scope,arguments[0]is{}, and the intendedsomedirargument inprocess.argv[2]is never consulted. - Running
node bench.jsfromTools/yarr-fuzz/therefore reaches L6 withHERE = {}and throws infs.readFileSyncon"[object Object]/isbot-pattern.json"before any benchmark runs. jsc bench.js -- <dir>works, because in the jsc shell the globalargumentsreally is the CLI-arg array (a string at index 0, orundefinedwhen none is passed).
Why nothing catches it
The file's own header comment (L1) says jsc bench.js (or node bench.js), and README.md lists bench.js as one of the reusable tools, so node bench.js is a documented invocation — but these are standalone scripts under Tools/, not compiled or exercised by any CI job in this repository. On the author's side the benchmarks were presumably run under jsc (the shell whose numbers the PR quotes), where the expression works.
Impact
Zero product impact. This is a dev-only benchmark script, not shipped in any Bun/JSC build and not run by CI. The Yarr changes themselves are unaffected. It is the same class as the earlier yarr-fuzz tool nits on this PR (run262.mjs/triage-base.mjs hardcoded paths, drive.mjs unused imports): a PR-introduced tool that does not work as its own comment says under one of its two documented hosts.
Fix
Test the Node path first (Node has both process and arguments; jsc has only arguments), or require the jsc-path value to be a string:
const HERE = (typeof process !== "undefined" && process.argv[2])
|| (typeof arguments !== "undefined" && typeof arguments[0] === "string" && arguments[0])
|| ".";Either ordering fix or the typeof … === "string" guard alone is sufficient; combining them is belt-and-braces.
| // astral / special characters spliced into literals | ||
| [2, () => { const n = pick(terms.filter(t => t.type === "Character")); if (!n) return null; return [splice(src, n, text(src, n) + pick(["\u{1F600}", "\\u{1F600}", "\uD83D", "é", "ſ", "K", "\\u212A", "ß", "\u{10400}"])), flags.includes("u") || flags.includes("v") || chance(0.5) ? flags : flags + "u"]; }], | ||
| // anchors / boundaries / modifiers | ||
| [2, () => { const n = pick(terms); if (!n) return null; return [splice(src, n, pick(["^", "$", "\\b", "\\B", "(?i:", "(?-i:", "(?s:"]).replace(/\($/, "(") + text(src, n) + (["(?i:", "(?-i:", "(?s:"].some(x => false) ? "" : "")), flags]; }], |
There was a problem hiding this comment.
🟡 Two subexpressions on this line are dead editing debris: .replace(/\($/, "(") is a no-op (nothing in the pick list ends with a bare ( — the modifier openers end with :), and (["(?i:", "(?-i:", "(?s:"].some(x => false) ? "" : "") is always "" (.some(x => false) is always false, and both ternary branches are ""). So when pick() returns one of the three modifier openers the emitted pattern has no closing ), always fails compiles() (L115), and the mutation step is discarded — 3 of 7 options are dead weight, and redundant with L81 which already handles modifier groups correctly. Fix: drop "(?i:", "(?-i:", "(?s:" from the pick list and delete the two dead subexpressions. Zero product impact — dev-only fuzz tool.
Extended reasoning...
What
Tools/yarr-fuzz/mutate.mjs L80 is the "anchors / boundaries / modifiers" mutation op:
[2, () => { const n = pick(terms); if (!n) return null; return [splice(src, n,
pick(["^", "$", "\\b", "\\B", "(?i:", "(?-i:", "(?s:"]).replace(/\($/, "(")
+ text(src, n)
+ (["(?i:", "(?-i:", "(?s:"].some(x => false) ? "" : "")
), flags]; }],Two subexpressions in it are dead editing debris:
.replace(/\($/, "(")— the regex matches a literal(at end-of-string and replaces it with(, which is a no-op by construction. It also never matches: none of the seven pick-list strings end with(— the four anchor/boundary strings contain no parenthesis at all, and the three modifier openers"(?i:","(?-i:","(?s:"all end with:.(["(?i:", "(?-i:", "(?s:"].some(x => false) ? "" : "")— the.some()callback ignoresxand returnsfalseunconditionally, so.some(...)is alwaysfalse; and both branches of the ternary are"". The whole subexpression is therefore the constant"".
Step-by-step proof
Take a mutation step where pick() returns "(?i:":
pick([...])→"(?i:"..replace(/\($/, "(")on"(?i:"—/\($/requires the last character to be(, but the last character is:, so no match; result is still"(?i:".["(?i:", "(?-i:", "(?s:"].some(x => false)— the callback returnsfalsefor every element, so.some()returnsfalse.false ? "" : ""→"".- The spliced fragment is
"(?i:" + text(src, n) + ""— an unclosed modifier group. splice(src, n, ...)inserts an unmatched(into an otherwise-balanced source, sonew RegExp(s2, f2)throwsSyntaxError: Invalid regular expression: missing ).compiles(s2, f2)(L89) returnsfalse, and the loop at L115 (if (s2.length > 600 || !compiles(s2, f2)) break;) discards the whole mutation step.
The same holds for "(?-i:" and "(?s:". So 3 of the 7 options in this weight-2 op are guaranteed to produce a syntax error and be filtered out — dead weight in the mutation-op distribution.
The very next op at L81 already handles modifier groups correctly:
[2, () => { ... const m = pick(["(?i:", "(?s:", "(?m:", "(?-i:", "(?i-s:"]); return [splice(src, n, m + text(src, n) + ")"), ...]; }],— it appends the closing ) and even adjusts flags for -i. So L80's three modifier entries are also redundant with L81, not merely broken.
Why it looks like this
This is unmistakable unfinished-refactor debris. The .replace(/\($/, ...) and the ternary keyed on the modifier-opener list both suggest an earlier form where the pick list contained something ending in ( and the ternary appended ")" when a modifier opener was chosen — presumably before that responsibility was moved to L81. What remains is the scaffold with both effects neutered.
Why nothing catches it / impact
mutate.mjs is a standalone Node script under Tools/yarr-fuzz/, not compiled or exercised by any CI job. compiles() (L89) is expressly there to filter out invalid mutations, so the broken cases are silently absorbed — no wrong output, no crash, no test failure. The tool still works; L81 covers modifier groups. Zero product impact.
The only effect is on fuzzing coverage: this weight-2 op wastes ~3/7 of its picks (and one whole mutation step each time, since L115 breaks the step loop), slightly under-representing the "anchors / boundaries" mutations relative to their intended weight. Same class of PR-introduced editing debris as the other Tools/yarr-fuzz nits already flagged on this PR (unused imports, hardcoded scratchpad paths).
Fix
Drop the three modifier entries and the two dead subexpressions, leaving L80 as the anchors/boundaries op only (L81 already covers modifier groups):
// anchors / boundaries
[2, () => { const n = pick(terms); if (!n) return null; return [splice(src, n, pick(["^", "$", "\\b", "\\B"]) + text(src, n)), flags]; }],…ect load the RegExp JIT emits Sanitizers do not instrument JIT-generated code, so an out-of-bounds read of the subject string by a compiled RegExp (past the end after a surrogate-pair peek, or before the start from a lookbehind) is invisible to them unless it happens to fault. With --verifyRegExpJITReads=1 each of the JIT's subject loads (single units, surrogate-pair peeks in the shared thunks and inline, the backward reader, fused 16/32/64-bit literal compares, the Boyer-Moore scalar and vector scans) is preceded by a probe that recomputes the effective address range from the live registers and compares it with [input, input + length * unitSize), reporting the emitting source line and aborting on a stray read. The input and length registers are never written by generated code, so the probe reads them back directly. Off by default and free when off (the check is made at code-generation time). Validated by planting four bugs of that shape (thunk pair read without its end check, backward reader without its start check, lookbehind offset mapping off by one, vector-scan bound loose by 8 bytes): each is reported within seconds by Tools/yarr-fuzz's new `verifyreads` configuration, which the un-modified tree passes. drive.mjs also gains a `gcstress` configuration (--collectContinuously).
| const ve = encMatch(vre.exec(subject), !!pat.flags.d); | ||
| vre.lastIndex = 0; | ||
| const vt = vre.test(subject); | ||
| const enc = JSON.stringify(ve && ve.map((v) => typeof v === "string" ? v : v)) + "|" + vt; |
There was a problem hiding this comment.
🟡 Nit: .map((v) => typeof v === "string" ? v : v) has identical ternary branches, so it is an identity map — dead editing debris (compare L512, where the string branch is safeStr(v)). No functional impact: baseE at L423 and the width variant at L451 both stringify the raw encMatch result without a map, so the comparison at L442 is self-consistent either way. Drop the .map(...) — JSON.stringify(ve) + "|" + vt. Zero product impact (dev-only fuzz tool).
Extended reasoning...
What
Tools/yarr-fuzz/regex-fuzz.js L441, inside the metamorphic-variant loop:
const enc = JSON.stringify(ve && ve.map((v) => typeof v === "string" ? v : v)) + "|" + vt;Both branches of the ternary return v, so .map((v) => typeof v === "string" ? v : v) is an identity map — the whole subexpression is equivalent to JSON.stringify(ve). This is unmistakable editing debris: the shape matches the file's own L512 (r.e.map((v) => typeof v === "string" ? safeStr(v) : ...)), so safeStr(v) (or some other transform) was presumably intended for the string branch and then dropped without deleting the scaffolding.
Step-by-step proof
encMatch(L370-377) returns eithernullor a plainArraybuilt asconst out = [m.index]; ...; return out;— no extra own enumerable properties.- At L439,
ve = encMatch(vre.exec(subject), ...), soveisnullor such a plain array. ve && ve.map((v) => typeof v === "string" ? v : v): whenveisnullthe&&short-circuits tonull; when it is an array,.mapreturns a fresh array withf(v) = vat every index.JSON.stringifyon a plainArraywalks index0..length-1only, so a shallow copy produced by.map(x => x)stringifies identically to the original. Hence L441 ≡JSON.stringify(ve) + "|" + vt.baseEat L423 isJSON.stringify(r.e) + "|" + r.t— the rawencMatchresult of the original pattern, with no map — and the width-variant at L451 isJSON.stringify(oe) + "|" + ot, likewise unmapped. So the comparison at L442 (if (enc !== baseE)) is self-consistent with or without the.map: both sides encode rawencMatchoutput.
Therefore the .map(...) is dead code with no observable effect on the metamorphic oracle.
Why nothing catches it
Nothing in the toolchain flags a ternary with identical branches or a no-op .map; regex-fuzz.js is a standalone fuzz-driver script under Tools/, not compiled or exercised by any CI job in this repository, and the metamorphic check produces the same enc string with or without the map, so runtime behaviour is identical.
Impact
Zero product impact. Tools/yarr-fuzz/regex-fuzz.js is a dev-only differential-fuzzing driver, not shipped in any Bun/JSC build and not run by CI. The Yarr changes themselves are unaffected. This is the same class of PR-introduced dev-tool editing debris as the open mutate.mjs L80 comment (.some(x => false) ? "" : ""), at a distinct file/location no existing PR comment covers.
Fix
Drop the identity .map:
const enc = JSON.stringify(ve) + "|" + vt;(If a safeStr transform was intended here, it would have to be applied to baseE at L423 and the width-variant enc at L451 as well to keep the comparison self-consistent — but those already compare raw encMatch output, and the metamorphic oracle only ever compares within one engine, so lone-surrogate encoding is not a concern; simply removing the map is correct.)
|
exciting, will this be upstreamed? |
`[\P{Number}&&\P{Alphabetic}]` and friends unioned the inverted operand
instead of intersecting or subtracting it. The engine fix landed in
oven-sh/WebKit#299 and reached Bun with the WebKit bump in #37352; this
adds the coverage for it.
The four tests on the previously broken path fail on the build before
that bump (447082ab) and pass on the build main pins now.
`[\P{Number}&&\P{Alphabetic}]` and friends unioned the inverted operand
instead of intersecting or subtracting it. The engine fix landed in
oven-sh/WebKit#299 and reached Bun with the WebKit bump in #37352; this
adds the coverage for it.
The four tests on the previously broken path fail on the build before
that bump (447082ab) and pass on the build main pins now.
Upgrades the WebKit fork to upstream WebKit/WebKit@47f7250137c6
(2026-08-16) via oven-sh/WebKit#455: 846 upstream commits since the
previous merge base `3722912ff800` (2026-08-02), 235 of them in
JavaScriptCore, WTF or bmalloc.
`WEBKIT_VERSION` is pinned to oven-sh/WebKit@eeab04040fa6, the fork
`main` after oven-sh/WebKit#455 merged, plus oven-sh/WebKit#463
(URLParser host scanning, WTF only); its `autobuild-eeab04040fa6...`
release has all 42 variants. (The PR initially pinned the #455 preview
build while that PR was open.)
### Bun-side changes
- `root.h`: `<JavaScriptCore/HandleSet.h>` no longer exists (`Strong<>`
slots moved to `StrongSet`, upstream `ff64aee116d4`).
- `ScriptFetchParameters::Type` gained `Text` (import-text, upstream
`49246d2612`) ahead of the fork's `HostDefined`, so the ordinal Bun's
transpiler emits for host-defined import types
(`to_script_fetch_parameters_type`) is 5 instead of 4; the
static_asserts in `BunAnalyzeTranspiledModule.cpp` pin both values. With
the fork, `with { type: "text" }` still parses as a host-defined type,
so Bun's own text loader keeps handling it on every file type.
- `NodeVMSyntheticModule.cpp`: `SyntheticModuleRecord::create()` takes
the record's `SourceProviderSourceType` (it only feeds the module kind
attached to errors).
- `NodeVM.cpp`: the import attributes switch covers `Type::Text`.
- `wtf-bindings.cpp`: `StackBounds::currentThreadStackBounds()` is
private to `Thread` upstream (`f6bc402b83`);
`Bun__StackCheck__initialize` uses the once-per-thread accessor the fork
adds.
### Visible to JavaScript after this upgrade
- `Iterator.prototype.chunks` / `windows` / `join` and `Iterator.zip` /
`zipKeyed` are enabled by default (upstream flipped the flags;
`chunks`/`windows` also follow the latest spec text and throw on
non-integral sizes).
- intl-era-monthcode (Stage 4) is unconditional:
`Intl.supportedValuesOf("calendar")` returns the proposal's 16 calendars
(`islamic` and `islamic-rgsa` are gone, Temporal rejects them as
calendar ids), era / eraYear / monthCode handling reworked across the
non-ISO calendars.
- `Array.prototype.sort()` without a comparator is stable for small
buckets of equal keys (was not) and faster on string arrays.
- Temporal: a batch of spec fixes (constructor `newTarget` order,
Duration rounding in exact arithmetic, `.with()` field resolution, time
zone string parsing follows the spec's parse records, DST gap range
checks).
- `/^[\q{ab|c|1}&&\P{L}]$/v` no longer matches `"ab"` (ported upstream
fix, the one Yarr change of this range that #299 did not already
contain).
- Deliberately unchanged in the fork, each flagged in oven-sh/WebKit#455
so it can be revisited separately: `Buffer` `kMaxLength` /
`MAX_ARRAY_BUFFER_SIZE` stays 4 GB (upstream went to 16 GB),
`NUMBER_OF_PROCESSORS` does not influence
`navigator.hardwareConcurrency` / `os.availableParallelism()`
(upstream's WTF now reads it), import-text is not exposed (`type:
"text"` stays Bun's).
### WebKit-side notes (details in oven-sh/WebKit#455)
- Yarr is kept at the fork's version (#299); upstream's Yarr commits of
this range were checked one by one and upstream's new regexp JSTests run
against the fork's engine. Not yet ported: one JIT optimization and the
default-off `\A` `\z` buffer boundaries.
- Windows ARM64 now uses `__builtin_frame_address(1)` in JIT operations
like every other platform: upstream deleted the `topCallFrame` fallback
the fork had selected there since the January bring-up ("crashes in DFG
operations" back then). Checked on a Windows 11 ARM64 machine with a
debug build of this branch against the preview WebKit: a workload that
tiers up to DFG and FTL (6 compiles each, `reportCompileTimes`) and
calls operations for 300k iterations runs clean; debug builds assert
`topCallFrame == callFrame` in every operation, so a wrong frame address
would have fired immediately. The windows-aarch64 lanes of this PR cover
the rest.
- Other resolutions: `GCCompletionCallback`, `StrongSet` and
`reconcileWeakReferencesAtGCEnd` renames applied to fork code,
upstream's own CMake 4.4 fix replaces the fork's,
`SyntheticModuleRecord` lazy exports kept on top of upstream's source
type plumbing.
- `JSType.h` did not change, so `src/jsc/JSType.rs` stays valid. ICU is
unchanged (the fork's 78.3 bump is already in the current pin). Bytecode
caches are keyed on the WebKit version and invalidate on their own.
### Binary size
The stripped binaries grow 448 KB to 800 KB per target against main
(0.6% to 0.9%; the size check's 0.5 MB threshold trips on darwin,
android and freebsd), acknowledged with `[skip size check]` in 95d581dd.
Comparing the non-LTO linux-x64 WebKit prebuilts of the old and new pin:
`libJavaScriptCore.a` object code grows a net 90 KB spread over 117
object files (StrongSet replacing HandleSet, the typed array sort
rewrite, intl-era-monthcode, memory64/table64, the new Air analyses,
builtins metadata), `libWTF.a` 2 KB, `libbmalloc.a` unchanged; the
remainder of the per-binary delta is LTO inlining of the changed engine
headers into Bun's own objects. The zipped artifacts are slightly
smaller than main's, so the added bytes are highly compressible.
### How did you verify your code works?
- `bun run jsc:build:debug` and `bun run build:local -p '42'` on Linux
x64 against the merged tree.
- oven-sh/WebKit#455 built on every platform variant as a preview before
merging; the merged release pinned here built the same way.
- The JS-visible changes listed above and the fork-side decisions
(`type: "text"` staying host-defined on both module paths,
`NUMBER_OF_PROCESSORS` being ignored, the 4 GB limit) were checked
against this build and against the previous pin with a throwaway test;
`test/js/bun/jsc/webkit-upgrade-3722912f.test.ts` still passes. No test
file is added in this PR.
- With the locally linked build: `test/js/bun/jsc`, `jsc-stress`,
`node/vm`, `node/module`, `bun/resolve`, `node/buffer`,
`node/worker_threads`, `bun/wasm`, `node/util` (78 files, 2034 tests);
the only failures are 5 s timeout / RSS threshold tests that a debug
build of current main fails identically on the same machine, and the
scenarios behind them behave the same with both builds when run
directly.
- JSTests: upstream's 21 new regexp tests plus the 318 `regexp*` /
`string-*` / `yarr*` stress tests against the fork's Yarr, JIT and
interpreter modes (see oven-sh/WebKit#455 for the four explained
failures).
<details>
<summary>JavaScriptCore / WTF / bmalloc changes in
WebKit/WebKit@3722912ff800...47f7250137c6 (235 commits; the ones that
matter to an embedder)</summary>
### Highlights
- `2c2c1af35743` ArrayBuffer / Wasm memory sizing overhaul: upstream
raises 64-bit `MAX_ARRAY_BUFFER_SIZE` from 4 GiB to 16 GiB (the Bun fork
pins it back to 4 GiB under `BUN_JSC_ADDITIONS` because
`buffer.constants.MAX_LENGTH` derives from it), fixes
`ArrayBuffer.prototype.slice` truncating byte lengths to 32 bits, fixes
growing shared memory64 past 4 GiB, and makes typed-array string keys
past `MAX_ARRAY_INDEX` reach the element.
- `ff64aee116d4` `Strong<>` root slots move from
`HandleSet`/`HandleBlock` to new `StrongSet`/`StrongBlock` (faster and
smaller for embedders that create/destroy many `JSC::Strong` handles, as
Bun does); `HandleSet.h` is gone and `Heap::handleSet()` is now
`Heap::strongSet()` (Bun's `root.h` already switched).
- `f6bc402b8344` `StackBounds::currentThreadStackBounds()` is now
private (on Linux it can re-parse `/proc/self/maps` per call); Bun's
`Bun__StackCheck__initialize` called it directly and now goes through a
`USE(BUN_JSC_ADDITIONS)`-only `currentThreadStackBoundsForEmbedder()`
shim.
- `5fc5182bcf83` `WTF::numberOfProcessorCores()` upstream now honors
`NUMBER_OF_PROCESSORS`; kept out of Bun builds in the fork (it feeds
`navigator.hardwareConcurrency` / `os.availableParallelism()` and would
override the fork's cgroup aware count), so nothing changes for Bun.
- `49246d261276` Implements the import-text proposal behind new
`useImportText` (default true); `ScriptFetchParameters::Type` and
`SourceProviderSourceType` gain `Text` (Bun's `HostDefined` tag moves
from 4 to 5), `SyntheticModuleRecord::create` / `AbstractModuleRecord`
take a `SourceProviderSourceType`; the fork keeps `"text"` as
`HostDefined` so Bun's own text loader still wins.
- `547e1555ce4d` `Iterator.prototype.chunks/windows` (and via yaml-only
flips `Iterator.prototype.join`, `Iterator.zip`/`zipKeyed`) become
enabled by default in this range and Bun does not override the flags, so
they become visible to Bun users with this upgrade.
- `99473681ff5e` intl-era-monthcode (Stage 4) is now unconditional:
`Intl.supportedValuesOf("calendar")` returns the fixed 16-calendar list,
`islamic`/`islamic-rgsa` are dropped as Temporal calendar ids, and
era/eraYear/monthCode handling is reworked across all non-ISO calendars.
- `a011564b98ab` `Array.prototype.sort()` with no comparator was not
stable for buckets of <32 equal-key entries (spec violation); now stable
(and `6380373fc6a1` makes it 1.2x-3.9x faster on string arrays).
- `f641af0b8e47` DFG-inlined single-element `Array.prototype.unshift`
was missing a write barrier, so the shifted element could be hidden from
the concurrent collector; fixes a potential GC use-after-free/crash in
optimized code.
- `7ff1104e4d0b` DFG no longer re-speculates GlobalProperty scope
accesses (e.g. `console`, `process`) after a BadCache exit, fixing
repeated OSR exits when such globals are redefined.
- `f2b02eb84f25` `MicrotaskQueue::performMicrotaskCheckpoint` skips
`drain()` on an empty queue; an empty `VM::drainMicrotasks()` halves in
cost (Bun calls this after every task).
### Runtime / builtins
- `2c2c1af35743` Overhauled ArrayBuffer / Wasm memory sizing: upstream
raises the 64-bit `MAX_ARRAY_BUFFER_SIZE` from 4 GiB to 16 GiB, caps
memory64 at 262144 pages (over-declared modules now fail
`WebAssembly.Module`), caps a single resizable/growable buffer's
`maxByteLength` reservation at 1/4 of the primitive address-space
budget, stops GCing while holding the buffer-memory lock (fixes growing
a shared memory64 buffer past 4 GiB), fixes
`ArrayBuffer.prototype.slice` truncating byte lengths to 32 bits, and
makes typed-array string keys past `MAX_ARRAY_INDEX` (e.g.
`"4294967295"`) reach the element for get/set/define/delete. (The Bun
fork pins `MAX_ARRAY_BUFFER_SIZE` back to 4 GiB under
`BUN_JSC_ADDITIONS` in `Source/JavaScriptCore/runtime/PageCount.h`
because `buffer.constants.MAX_LENGTH` in `src/jsc/bindings/JSBuffer.h`
is derived from it.)
- `40d37f36527f` Follow-up: module parsing accepts arbitrarily large
memory64 limits (rejected at instantiate/grow instead);
`PageCount::maxPageCount` becomes a `uint64_t` and `PageCount::bytes()`
saturates instead of wrapping.
- `a011564b98ab` `Array.prototype.sort()` with no comparator was not
stable for buckets of <32 equal-key entries (spec violation); now uses a
stable sort.
- `6380373fc6a1` `Array.prototype.sort()` with no comparator rewritten
as an in-place counting sort over UTF-16 (still stable); 1.2x-3.9x
faster on string arrays such as `Object.keys(o).sort()`.
- `547e1555ce4d` `Iterator.prototype.chunks/windows` aligned to the
latest spec: non-number or non-integral size now throws TypeError (was
ToNumber coercion), invalid arguments close the underlying iterator,
`undersized` only defaults when `undefined`. These methods become
enabled by default in this range via `793e36fb835e` (yaml-only, outside
these paths); Bun does not override the flag, so they appear on
`Iterator.prototype` after this upgrade.
- `7417386b7da1` `Iterator.prototype.join` aligned to spec: a separator
is still emitted for `undefined`/`null` elements; builds the result with
a RopeBuilder; OOM closes the iterator. Enabled by default in this range
via `e9a62e6b4da5` (yaml-only), so `Iterator.prototype.join` now exists
in Bun. (`Iterator.zip`/`zipKeyed` are likewise enabled by
`934bb002485a`, yaml-only.)
- `c0625bcafb6c` `ErrorInstance` is now subclassable by embedders
(exported constructor/method-table entries plus a `finishCreation(VM&,
StackTraceCapturePolicy)` that captures no stack and adds no own props),
used by WebCore to make `Error.isError(new DOMException())` true;
`CloneSerializerBase` now consults the embedder's `dumpDerivedTerminal`
before its generic ErrorInstance path. Bun's `JSDOMException`
(`src/jsc/bindings/webcore/JSDOMException.h`) is still a plain wrapper,
so no behavior change in Bun unless adopted.
- `fedbb7bdc250`
`Int8Array/Uint8Array/Uint8ClampedArray.prototype.sort()` uses a SIMD
presorted check plus counting sort (2.5x-12x faster, ~38x on presorted
input).
- `4f3ecec97431` `JSON.stringify` fast path now accepts final objects
with non-`Object.prototype` prototypes (class instances) when the chain
has no `toJSON` (~3.5x on such payloads); also fixes
`noSideEffectMayHaveNonIndexProperty()` checking static properties on
the wrong chain entry.
- `8b5e6ebb64e6` FastStringifier caches buffer pointer/length across
property-name emission (reland of `ea3fbb33caa5`, which was reverted in
`ba1d526398de` for a perf regression; value half dropped).
- `da12fb32aeb9` FastStringifier adds a 4-7 byte two-window copy tier
and removes the 8-byte loop; faster `JSON.stringify` of short Latin-1
keys.
- `01ea2a8eb955` `String.prototype.split` no longer atomizes results
when the subject is not an atom string (~3.8x faster on runtime-built
strings; results are plain substrings now).
- `81a11702ef82` The `str.replace(/^\s+/, "")` / `/\s+$/` trim fast path
was unreachable once the caller tiered up to DFG/FTL; now applies in all
tiers (4.5x-5x).
- `a4df93500a72` `Array.prototype.join` / `toString` on Int32 arrays
writes numbers directly for any separator (~2x);
`JSOnlyStringsAndInt32sJoiner::tryJoin` is now templated on indexing
shape.
- `2af38faaec70` DFG `Function.prototype.bind` strength reduction now
also fires for method structures (`this.onClick.bind(this)` on class
methods, ~4.6x).
- `deb0d2fa4be6` BigInt add/sub/mul get fixed-size fast paths, squaring
optimization and carry handling that avoids flag spills on arm64.
- `0270fd0a8d77` BigInt Crandall modular reduction made branch-free for
the first corrective subtract (faster big modular arithmetic).
- `5d6747ef60d4` (parser) see below; memory-visible: closures no longer
retain all call arguments when an inner arrow uses object shorthand.
- `a73e86f9a37f` `Set.prototype`, `WeakRef` and `FinalizationRegistry`
are no longer materialized in `JSGlobalObject::init()`;
WeakRef/FinalizationRegistry become lazy static-table globals (~6.6 KB
saved per global object; `propertyNames->WeakRef` /
`->FinalizationRegistry` removed).
- `8d33a8ff591d` `UnlinkedFunctionExecutable` stores
`parentScopeTDZVariables` inline (RareData allocations drop ~100x in
let/const-heavy code at the same 96-byte cell size); bytecode cache
encoding in `CachedTypes.cpp` changed (Bun keys its cache on the WebKit
version, so old caches are simply invalidated).
- `edd953757f9f` `StructureRareData` shrunk back from 104 to 96 bytes
(cell 112 -> 96) with a static_assert so it does not regress.
- `f2b02eb84f25` `MicrotaskQueue::performMicrotaskCheckpoint` skips
`drain()` on an empty queue; an empty `VM::drainMicrotasks()` halves in
cost (Bun calls this after every task).
- `13dc8fa6e3d5` VM startup: AtomStringTable and BuiltinNames'
private-name set reserve capacity up front (fewer rehashes during VM
construction).
- `81d660ceeb2e` Builtin executable metadata (line counts, parameter
counts, etc.) is precomputed by the builtins generator instead of at VM
launch; `BuiltinCodeIndex::NumberOfBuiltinCodes` replaced by
`numberOfBuiltinCodes`. The free `JSC::createBuiltinExecutable()` used
by Bun's generated builtins is unchanged.
- `c1b19d012809` JIT thunks split into eagerly- and lazily-created sets
(less work at VM startup; `JITThunks::ctiStub` now takes `VM&`).
- `bff3814d76f7` Linux: checkpoint OSR side-state handling used uncached
stack bounds, which glibc implements by re-reading `/proc/self/maps` on
every call; now uses the thread's cached bounds (also on the release
path).
- `c00fd8a9713c` Baseline JIT gets an inline atom-identity fast path for
`switch` on strings; new option `maximumInlineStringSwitchCaseCount`
(default 64).
- `0c51f43daa3b` Wasm OMG recognizes naive byte-copy/fill loops and
prepends `memory.copy`/`memory.fill` fast paths; new option
`useWasmByteLoopReplacement` (default true).
- `c7ed9fcf7957` 32-bit only: typed-array put with an out-of-range
canonical numeric index keeps the index as `uint64_t` until
bounds-checked.
- `a53d011599e7` Tree-wide rename, no behavior change:
`finalizeUnconditionally` -> `reconcileWeakReferencesAtGCEnd` on
ErrorInstance, Structure, StructureRareData, SymbolTable, InferredValue,
JSWeakObjectRef, JSFinalizationRegistry, FunctionExecutable, etc.;
`Heap::finalizeUnconditionalFinalizers` ->
`reconcileWeakReferencesAtGCEnd`; `finalizerSet(For)` ->
`weakReconciliationSet(For)`; `ScriptExecutable::finalizeCodeBlockEdge`
-> `jettisonCodeBlockEdgeIfDead` (Bun only references the old names in
comments).
### Parser / bytecompiler
- `5d6747ef60d4` Object-literal shorthand inside an arrow function no
longer marks the enclosing function as using `eval`, so it stops
materializing `arguments` into its scope; closures returned from such
functions no longer keep all call arguments alive (memory + faster
function entry).
- `69b336c0ac05` `SourceProviderCacheItem` (one per function >16 chars
parsed, retained until full GC) is now a proper trailing array of
`PackedRefPtr`; ~12% less malloc memory for the source-provider cache on
large bundles.
- `9f770b1bd595` `Parser::useVariable` remembers the last variable added
and skips the set insertion on repeats (parse speed).
- `8aa3307b46af` Single-line-comment scanning and the arrow-function /
destructuring paths are moved out of the lexer and
`parseAssignmentExpression` hot loops (lower register pressure; parse
speed, no logic change).
- `71c68f4b3b35` `Lexer::lexExpectIdentifier()` removed; the vectorized
`parseIdentifier()` is now faster, so this shrinks hot code (header API
removal, internal to the parser).
### Intl / Temporal
- `99473681ff5e` intl-era-monthcode (Stage 4) is implemented
unconditionally and the previously default-off `useIntlEraMonthcode`
option is removed: `Intl.supportedValuesOf("calendar")` now returns the
proposal's fixed 16-calendar list, `islamic`/`islamic-rgsa` are dropped
as Temporal calendar ids (`islamic` maps to `islamic-tbla` in
DateTimeFormat, unknown calendars fall back to the locale default),
era/eraYear/monthCode handling reworked across all non-ISO calendars
with chinese/dangi falling back to ISO fields beyond +/-10000 instead of
throwing, and DateTimeFormat's era-text override only applies when an
era field was requested.
- `9ef04dabf52d` `Intl.Locale.prototype.getCollations()` etc. now return
sorted arrays per spec.
- `171864159318` DateTimeFormat with islamic-civil/tbla/umalqura
calendars rendered pre-Hijra years as e.g. `-332 Before Hijra`; now `333
Before Hijra` (computed from the calendar, works with `year:
"2-digit"`).
- `b2ec9a4586ee` `formatToParts()` now emits the separating space that
`format()` inserts before a synthesized coptic/islamic era, so joined
parts equal `format()` again.
- `33a5272cf9ac` `String.prototype.localeCompare(x, "locale")` (string
locale, no options) caches the collator per global object; the common
sort-comparator pattern is ~50x faster.
- `7d0200e4e6ed` That cache is invalidated when the user preferred
languages change (it returned stale orderings for unavailable locales
like `"xx"`).
- `b48f01b7f1b1` All eight Temporal constructors now validate fields
before reading `newTarget.prototype` (spec order; `Reflect.construct`
with a throwing prototype getter gets the RangeError);
`ZonedDateTime.prototype.with` now range-checks epoch nanoseconds;
`tryCreateIfValid`-style helpers renamed to
`createTemporalDate`/`createTemporalZonedDateTime`/... taking a
`TemporalNewTarget`.
- `11615f86705a` Duration rounding decisions now use exact Int128
instead of doubles, fixing wrong results such as `until(...,
{smallestUnit:"month", roundingMode:"ceil"})` returning `P1M` instead of
`P29DT1H`, and the half-even branch of ApplyUnsignedRoundingMode.
- `399973c04a04` `.with()` on all Temporal types now goes through spec
`ISODateToFields`/`CalendarMergeFields` (year-only changes on lunisolar
calendars pick the right month); fixes `PlainYearMonth.add/subtract`
shifting months by -2 for buddhist/roc/japanese in ISO years ~1-1582;
`ZonedDateTime.prototype.with` restored to spec step order.
- `22a13eb9ee2f` Time-zone string parsing follows the spec's parse
records: bracket annotations are now accepted on all six string
productions (`"2024-12[Europe/Berlin]"`, `"12:00[Europe/Berlin]"`, ...),
`"T12+01"` is rejected as an unavailable named zone instead of resolving
to `+01:00`, and IANA-name syntax drops the 14-char limit (accepting
e.g. `[..]`).
- `284afdacfb77` Non-ISO field resolution at range edges:
`PlainYearMonth.toPlainDate({day: 256})` no longer wraps the day to 0
(produced a live `...-01-00` date); chinese/dangi arithmetic at extreme
years no longer throws; `dateUntil` used the wrong year kind on ICU 76.
- `6fd438a4aef2` DST-gap disambiguation re-enters the epoch range check,
so `ZonedDateTime.from("+275760-10-05T02:30[Australia/Sydney]")` throws
instead of creating an out-of-range value; also fixes which candidate is
picked in gaps.
- `4f049dc9046e` `monthCode` given a non-string now throws TypeError
again in `PlainDateTime.from`/`PlainDate.with` (regression from
consolidation); ISO `.with()` no longer regulates day/month twice;
getter order test added.
- `89c1884e15a9` `PlainDate` construction clamps out-of-range years
itself (was a debug assertion crash); fixes `PlainYearMonth.toPlainDate`
clamp direction under `overflow: "constrain"` and a UB cast in the
PlainMonthDay constructor.
- `642d9211add2` ICU failures inside the calendar/time-zone bridges now
propagate as errors instead of being folded into plausible values (e.g.
hebrew `M05L` silently becoming `M06`, a sticky UErrorCode making
`getTimeZoneTransition` return bogus transitions).
- `b2233ac17643` Fixes uninitialized members in duration nudging, an
overflowable day bound, and a debug-only assertion crash when a
zero-length nudge window lands on a day a zone skips; removes dead
duration helpers.
- `8776c95a1b0a` Temporal time-zone cache widened from 8 to 16 entries
(parity with V8 on the duration-total benchmark).
- (`e07ecf4c4a07`, `ee16ce938a8f`, `1375d28c26b3`, `e165fd1fce9a`,
`f6c491404f2d`: internal Temporal refactors declared no-behavior-change;
omitted.)
### Modules
- `49246d261276` Implements the import-text proposal: `import x from
"./a.txt" with { type: "text" }` / dynamic import are handled by JSC as
synthetic default-export modules, gated by new option `useImportText`
(default true, generated from the preferences yaml). Adds
`ScriptFetchParameters::Type::Text` and
`SourceProviderSourceType::Text`, and
`AbstractModuleRecord`/`CyclicModuleRecord`/`SyntheticModuleRecord`
constructors and `SyntheticModuleRecord::create` now take a
`SourceProviderSourceType`. The Bun fork keeps `"text"` as a
`HostDefined` type in `ScriptFetchParameters::parseType` so Bun's own
text loader still wins; Bun's `HostDefined` tag moved from 4 to 5
(static_asserts in `src/jsc/bindings/BunAnalyzeTranspiledModule.cpp` and
`to_script_fetch_parameters_type` in `src/js_printer/lib.rs` are already
updated).
- `cc673d7b23bf` import-defer updated to proposal PRs #85/#87:
`ReadyForSyncExecution` and `GatherAsynchronousTransitiveDependencies`
now use `IsModuleSCCEvaluated` (new
`CyclicModuleRecord::isSCCEvaluated()`), so touching a deferred
namespace whose dependency sits in a still-awaiting TLA cycle correctly
throws "Unable to synchronously evaluate deferred module" instead of
evaluating early (and a debug assertion no longer fires). Bun already
forces `useImportDefer` on; upstream also flipped its default on in
`85e82ceefe1b` (yaml-only).
### API
- `62692012c98a` `HeapFinalizerCallback` renamed to
`GCCompletionCallback` (header `heap/HeapFinalizerCallback.h` ->
`heap/GCCompletionCallback.h`; `Heap::add/removeHeapFinalizerCallback`
-> `add/removeGCCompletionCallback`); the C entry points
`JSContextGroupAddHeapFinalizer` / `JSContextGroupRemoveHeapFinalizer`
keep their names and behavior.
### Embedder-relevant API changes
- Removed: `heap/HeapFinalizerCallback.h` / class
`HeapFinalizerCallback`, `Heap::addHeapFinalizerCallback`,
`Heap::removeHeapFinalizerCallback` -> `GCCompletionCallback.h`,
`Heap::addGCCompletionCallback`, `Heap::removeGCCompletionCallback`
(`62692012c98a`).
- Renamed: `T::finalizeUnconditionally(VM&, CollectionScope)` ->
`T::reconcileWeakReferencesAtGCEnd` on ErrorInstance, Structure,
StructureRareData, StructureTransitionTable, SymbolTable, InferredValue,
JSWeakObjectRef, JSFinalizationRegistry, FunctionExecutable,
GlobalExecutable, UnlinkedFunctionExecutable, CodeBlock;
`Heap::finalizeUnconditionalFinalizers` ->
`reconcileWeakReferencesAtGCEnd`;
`Heap::finalizeMarkedUnconditionalFinalizers` ->
`reconcileWeakReferencesInMarkedCells`; IsoCellSet
`finalizerSet`/`finalizerSetFor` ->
`weakReconciliationSet`/`weakReconciliationSetFor`;
`ScriptExecutable::finalizeCodeBlockEdge` ->
`jettisonCodeBlockEdgeIfDead`; `JITPlan::finalizeInGC` ->
`reconcileWeakReferencesAtGCEnd` (`a53d011599e7`). Any embedder class
registered for unconditional finalization must rename its method.
- Enums: `ScriptFetchParameters::Type` gains `Text` after `JSON` (shifts
any embedder-appended values); `SourceProviderSourceType` gains `Text`
between `JSON` and `ImportMap` (shifts `ImportMap` and any
embedder-appended values; exhaustive switches need a case);
`SourceProvider::isModuleType()` now also true for `Text`
(`49246d261276`).
- Signatures: `AbstractModuleRecord(VM&, Structure*, Identifier,
SourceProviderSourceType)`, `CyclicModuleRecord(...,
SourceProviderSourceType)`,
`SyntheticModuleRecord::create(JSGlobalObject*, VM&, Structure*, const
Identifier&, SourceProviderSourceType)`; new
`SyntheticModuleRecord::createTextModule` (`49246d261276`). Bun's
`NodeVMSyntheticModule.cpp` already passes the new argument.
- `ArrayBuffer::grow(const AbstractLocker&, VM&, size_t, bool)` removed;
replaced by `tryGrow(const AbstractLocker&, size_t, bool,
BufferMemoryResult::Kind&)` (the `grow(VM&, ...)` overload remains); new
`maxGrowableBufferReservationBytes` in `BufferMemoryHandle.h`; new
`Gigacage::primitiveAddressSpaceBudget`; `isCanonicalNumericIndexString`
gains an optional `std::optional<uint64_t>*` out-parameter (source
compatible); 64-bit `MAX_ARRAY_BUFFER_SIZE` is 16 GiB upstream (fork
keeps 4 GiB) (`2c2c1af35743`).
- `PageCount::maxPageCount` is now `uint64_t` with a much larger value;
`PageCount::bytes()` saturates (`40d37f36527f`).
- `ErrorInstance`: constructor and
`getOwnPropertySlot`/`put`/`defineOwnProperty`/`deleteProperty`/`getOwnSpecialPropertyNames`
are now `JS_EXPORT_PRIVATE`; new protected `finishCreation(VM&,
StackTraceCapturePolicy)`; `CloneSerializerBase::dumpIfTerminal` calls
`dumpDerivedTerminal` before the ErrorInstance path (`c0625bcafb6c`).
- `CommonIdentifiers`: `propertyNames->WeakRef` and
`propertyNames->FinalizationRegistry` removed;
WeakRef/FinalizationRegistry structures/prototypes become lazy accessors
(`a73e86f9a37f`).
- `BuiltinCodeIndex::NumberOfBuiltinCodes` removed ->
`JSC::numberOfBuiltinCodes`; new `BuiltinSourceMetadata` /
`s_JSCBuiltinSourceMetadata`; member
`BuiltinExecutables::createBuiltinExecutable` gains a metadata parameter
(free `JSC::createBuiltinExecutable()` and public static
`BuiltinExecutables::createExecutable()` unchanged) (`81d660ceeb2e`).
- `JSOnlyStringsAndInt32sJoiner::tryJoin` is now
`template<IndexingType>` (`a4df93500a72`);
`JITThunks::ctiStub(CommonJITThunkID)` now takes `VM&` first
(`c1b19d012809`); `Lexer::lexExpectIdentifier()` removed
(`71c68f4b3b35`); Temporal `try*` creation helpers replaced by
`createTemporal*(…, TemporalNewTarget)` free functions and
`TemporalPlainDate::mergeDateFields` removed (`b48f01b7f1b1`,
`4f049dc9046e`); `IntlObject.h` calendar-ID table drops `islamic` and
`islamic-rgsa`, `Options::useIntlEraMonthcode` removed (`99473681ff5e`).
- New options: `useImportText` (true),
`maximumInlineStringSwitchCaseCount` (64), `useWasmByteLoopReplacement`
(true). Defaults flipped to true in this range but via yaml-only commits
outside these paths: `useIteratorChunking` (`793e36fb835e`),
`useIteratorJoin` (`e9a62e6b4da5`), `useJointIteration`
(`934bb002485a`), `useImportDefer` (`85e82ceefe1b`); Bun overrides none
of the first three, so `Iterator.prototype.chunks/windows/join` and
`Iterator.zip/zipKeyed` become visible to Bun users with this upgrade.
### GC / heap
- `ff64aee116d45c` `Strong<>` root slots now live in new
`StrongBlock`/`StrongSet` (libpas-style bump+freelist pages, empty
blocks returned to the OS, no write barrier on set) replacing
`HandleSet`/`HandleBlock`; faster and smaller for embedders that
create/destroy many `JSC::Strong` handles (Bun does);
`Heap::handleSet()` is now `Heap::strongSet()` and `HandleSet.h` is gone
(Bun's `root.h` already switched to `StrongSet.h` in this PR). Follow-up
`55659d048725` drops a dead `USE(JSVALUE64_32)` branch from
`StrongBlock.h`.
- `f641af0b8e47` DFG-inlined single-element `Array.prototype.unshift` on
contiguous arrays was missing a write barrier, so the shifted element
could be hidden from the concurrent collector; fixes a potential GC
use-after-free/crash in optimized code.
- `6bdb4f69e23b` VM/Heap teardown (`lastChanceToFinalize`) uses a new
`StopAllocatingMode::ForGood` that skips recomputing allocation bitmaps;
faster VM destruction (e.g. Worker exit);
`MarkedSpace::stopAllocatingForGood()` removed.
- `a53d011599e7` Rename-only: `finalizeUnconditionally()` on all cell
types/VM becomes `reconcileWeakReferencesAtGCEnd()`,
`Heap::finalizeUnconditionalFinalizers` ->
`reconcileWeakReferencesAtGCEnd`, IsoCellSet `finalizerSet` ->
`weakReconciliationSet`, `ScriptExecutable::finalizeCodeBlockEdge` ->
`jettisonCodeBlockEdgeIfDead`; no behavior change (Bun only mentions the
old name in comments in `src/jsc/bindings/ErrorStackTrace.cpp`,
`JSCTaskScheduler.cpp`, `FormatStackTraceForJS.cpp`).
- `5602ec36107b` Rename-only follow-up: `visitWeak()` on
CallLinkInfo/PropertyInlineCache/InlineCacheHandler/JITStubRoutine/PolymorphicCallStubRoutine/MicrotaskCall
-> `reconcileWeakReferencesAtGCEnd()`;
`AccessCase`/`PolymorphicAccess::visitWeak` -> `isStillLive`.
- `3d37c6da40ba` Rename-only:
`GetByStatus`/`PutByStatus`/`InByStatus`/`DeleteByStatus`/`CallLinkStatus`/private-brand
statuses and their variants `finalize()` -> `isStillLive()`.
- `62692012c98a` Rename-only: `HeapFinalizerCallback` ->
`GCCompletionCallback` (header renamed too),
`Heap::add/removeHeapFinalizerCallback` ->
`add/removeGCCompletionCallback`; C API `JSContextGroupAddHeapFinalizer`
unchanged.
- `f4da7823ee1d` Rename-only: `Heap::finalize` ->
`runCollectionEpilogue` (and `needFinalize` bits); the only observable
change is the `--logGC=1` phase label "finalize" is now "epilogue".
### LLInt / Baseline / DFG / FTL / B3
- `a02f99629f76` FTL OSR-exit compiler hit `RELEASE_ASSERT_NOT_REACHED`
(crash) when exiting with a `PhantomNewArrayWithButterfly` whose
butterfly was still live (`DataFormatStorage`); now passed through like
`DataFormatJS`.
- `91d96b29d6b2` DFG
`AbstractInterpreter::forAllValues`/`dump`/`SafeToExecute` now handle
tuple nodes; the DFG-inlined `StringIterator.prototype.next` followed by
a structure transition in the same block asserted in debug builds and
silently skipped the tuple's values in release.
- `7ff1104e4d0b` DFG no longer re-speculates
`op_get_from_scope`/`op_put_to_scope` GlobalProperty accesses (e.g.
`console`, `process`, any global-object property) after a BadCache exit;
emits a generic IC instead, fixing repeated OSR exits when such globals
are redefined.
- `7600ab4bec97` DFG stops inlining varargs calls (`f(...args)`,
`f.apply`) once a `VarargsOverflow` exit has been seen at that site,
fixing perpetual OSR exit/recompile loops.
- `fbb79b137a90` Baseline JIT read the 1-byte
`maxArgumentCountIncludingThisForVarargs` profile with a 32-bit compare
(picking up adjacent bytes), so varargs argument-count feedback fed to
the DFG was wrong; now `load8` + compare.
- `465d5ab28c60` `String.prototype.substring` is now inlined in DFG/FTL
(shares `slice` lowering: empty/one-char/whole-string/rope fast paths);
1.6-2.1x faster in microbenchmarks.
- `fb299342a580` RegExp `test`/`exec` first-character filter now also
applies when the subject is an Untyped edge (runtime string check),
widening the fast path for real-world code.
- `c00fd8a9713c` Baseline JIT gets an inline pointer-identity dispatch
for `switch` on strings when the scrutinee is an atom (previously always
called the hashing slow path); new option
`maximumInlineStringSwitchCaseCount` (default 64).
- `a4df93500a72` `Array.prototype.join`/`toString` on Int32 arrays now
uses `JSOnlyStringsAndInt32sJoiner` for any separator (was only for
`""`), ~2x faster (one-line DFGOperations change; mostly runtime/).
- `0d25934d08a8` VM-independent JIT thunks (polymorphic call thunks,
most IC handler thunks) are generated once per process and shared across
VMs; less per-VM startup work and JIT memory when creating many VMs
(Workers); `JITThunks::ctiStub` now takes `VM&`, handler generators no
longer take `VM&`.
- `c1b19d012809` Remaining VM-dependent thunks split into eager
(exception/native-call/virtual-call) and lazily generated (IC
transition/custom-accessor handlers), so short-lived VMs do not generate
thunks they never use.
- `f40dcdd0730d` LLInt function prologue zeroes the new frame 16 bytes
per iteration with a hoisted zero register (4 instructions/16 bytes on
ARM64, 5 on x64, was 12); `76f57a9311b1` extends it to ARM64E (not built
by Bun).
- `bbab514b1010` DFG/FTL `LazyJSValue::emit` leaked a `StringImpl` ref
per emitted string constant when compilation was abandoned (JIT memory
exhausted or code block invalidated before finalize); now held in a
`RefPtr`.
- `74091f918bfc` New Air `Padding` pseudo-op that emits no bytes
replaces most `Nop` padding, and `reportUsedRegisters` is skipped for
Wasm OMG; faster OMG compiles with no extra `nop`s in generated code.
- `5821b05faa72` Air `TmpWidth` and `UseCounts` are now built in a
single graph walk via new `InstAnalyzer`; faster FTL/OMG register
allocation.
- `4a10860dc35c` Faster Air liveness (`WTF::Liveness` no longer re-walks
blocks or zeroes gen/kill sets; new
`forEachLiveAtHeadNotLiveAtTail`/`...TailNotLiveAtHead`), ~17% off
greedy allocator `buildLiveRanges`.
- `ee4f0240590d` Air DCE worklist seeded in reverse program order, ~20%
faster phase; `ae85b80e5fbe` same phase avoids Vector element removal.
- `4af3bbad9cda` Air, BBQ and Baseline JIT code-generation loops skip
disassembler-only label creation and hoist loop invariants; lower
compile latency in all JIT tiers.
- `6589b2e5c18c` WasmGC `struct.new`/`array.new` codegen tightened (new
`JITAllocator::variableNonNullWithConstantCellSize`, narrower B3
effects, constant-size array allocation folding); faster WasmGC
allocation and more B3 load motion around it.
- `2ec06de15a0d` B3 CSE stops walking every predecessor block for WasmGC
`struct.get`/`struct.set` when no other access to that field exists;
faster OMG compile of WasmGC modules.
- `53517eb3a2b8` / `31f35870966c` Wasm `memory.copy` and `memory.fill`
runtime operations inline small-size copies/fills before falling back to
`memcpy`/`memset`; faster small bulk-memory ops.
- `d02c68d04f96` IPInt mis-decoded `memory.size`/`memory.grow` when the
memory-index immediate took more than one LEB byte (multi-memory, on by
default), desynchronizing the following instructions; also removes the
`parseMemoryIndexForBulkOp` spec-test workaround.
- `9226ba78d93d` `DFG::enableInt52()` removed; Int52 speculation is
unconditional now that the only 64-bit JIT backends remain (no behavior
change on x64/arm64).
- `bf1dab73b14d` / `6010a9ea6ce6` / `84f83abd45c9` 32-bit/ARMv7 JIT
leftovers removed: `ARMv7Assembler.h` deleted, 32-bit DataFormats/GPR
pairs/OSR-entry paths dropped, `branchIfNumber`/`branchIfNotNumber` lose
their scratch-register parameter, `CCallHelpers` `extraGPRArgs` removed;
no codegen change on 64-bit.
- `2a8926009f45` `USE(BUILTIN_FRAME_ADDRESS)` removed (always on for JIT
platforms); `JSWebAssemblyInstance::temporaryCallFrame()` and its field
removed. The fork had it off on Windows ARM64; that configuration no
longer exists (see above).
- `ac2afd10b8ac` Yarr JIT sub-feature flags
(`YARR_JIT_ALL_PARENS_EXPRESSIONS`, `YARR_JIT_BACKREFERENCES`,
`YARR_JIT_REGEXP_TEST_INLINE`, `YARR_JIT_UNICODE_EXPRESSIONS`) removed
as always-on for x64/arm64, with matching DFG/FTL ifdef cleanup; no
behavior change.
- `ef6d9ba26b17` / `56baf6e01b3d` Linux RT-thread removal briefly set
JIT worklist threads to `ThreadQOS::Utility`, then was reverted for
JetStream/Speedometer regressions; net zero change to JSC.
### Bytecode / CodeBlock
- `8d33a8ff591d` `m_parentScopeTDZVariables` moves back into
`UnlinkedFunctionExecutable` (name stored as `m_ecmaName` + `m_hasName`
bit), so the 80-byte RareData is no longer malloc'ed for ~30-40% of
executables in let/const/class-heavy code; also changes the
`CachedTypes` bytecode-cache layout (Bun keys its cache version on
`BUN_WEBKIT_VERSION`, so old `--bytecode` artifacts are invalidated as
with any bump).
- `b00e0c35f823` Slow-path location and per-site register fields move
from `PropertyInlineCache` into `RepatchingPropertyInlineCache`; handler
ICs shrink 128->112 bytes, baseline unlinked ICs 40->32, DFG unlinked
ICs 64->40 (~465 KB saved on Octane typescript).
### Embedder-relevant API changes
- Removed headers: `heap/HandleSet.h`, `heap/HandleBlock.h`,
`heap/HandleBlockInlines.h` (use `heap/StrongSet.h` /
`heap/StrongBlock.h`); `assembler/ARMv7Assembler.h`. Renamed header:
`heap/HeapFinalizerCallback.h` -> `heap/GCCompletionCallback.h`.
- `Heap::handleSet()` -> `Heap::strongSet()`; `HandleSet::heapFor(slot)`
-> `StrongSet::setFor(slot)`; `HandleSet` -> `StrongSet`.
- `HeapFinalizerCallback` -> `GCCompletionCallback`;
`Heap::addHeapFinalizerCallback/removeHeapFinalizerCallback` ->
`addGCCompletionCallback/removeGCCompletionCallback` (C API
`JSContextGroupAdd/RemoveHeapFinalizer` unchanged).
- `finalizeUnconditionally()` -> `reconcileWeakReferencesAtGCEnd()` on
`VM`, `ErrorInstance`, `JSFinalizationRegistry`, `JSWeakObjectRef`,
`Structure`, `StructureRareData`, `SymbolTable`, `WeakMapImpl`,
`InferredValue`, `UnlinkedFunctionExecutable`, `FunctionExecutable`,
`GlobalExecutable`, `CodeBlock`, `JSWebAssemblyInstance`, `JITPlan` (was
`finalizeInGC`);
`Heap::ScriptExecutableSpaceAndSets::finalizerSet/finalizerSetFor` ->
`weakReconciliationSet/weakReconciliationSetFor`;
`ScriptExecutable::finalizeCodeBlockEdge` ->
`jettisonCodeBlockEdgeIfDead`;
`CodeBlock::finalizeLLIntInlineCaches/finalizeJITInlineCaches` ->
`reconcileLLIntInlineCachesAtGCEnd/reconcileJITInlineCachesAtGCEnd`;
`RecordedStatuses::finalize` -> `reconcileWeakReferences`.
- `visitWeak()` -> `reconcileWeakReferencesAtGCEnd()` on `CallLinkInfo`,
`DirectCallLinkInfo`, `PropertyInlineCache`, `InlineCacheHandler`,
`JITStubRoutine` (incl. the virtual `...Impl`),
`PolymorphicCallStubRoutine`, `MicrotaskCall`;
`AccessCase::visitWeak`/`PolymorphicAccess::visitWeak` -> `isStillLive`;
`*Status::finalize()`/`*Variant::finalize()` -> `isStillLive()`.
- `Heap::finalize` -> `Heap::runCollectionEpilogue`;
`MarkedSpace::stopAllocatingForGood()` removed;
`MarkedBlock::Handle::stopAllocating` and
`LocalAllocator::stopAllocating` gain a `StopAllocatingMode` parameter.
- `JITThunks::ctiStub(CommonJITThunkID)` -> `ctiStub(VM&,
CommonJITThunkID)`; `polymorphicThunk()`,
`polymorphicThunkForClosure()`, `polymorphicTopTierThunk[ForClosure]()`,
`returnFromBaselineGenerator()` and the VM-independent IC handler
generators in `InlineCacheCompiler.h` no longer take `VM&`;
`JSC_FOR_EACH_COMMON_THUNK` is now the union of
`JSC_FOR_EACH_VM_INDEPENDENT_COMMON_THUNK` and
`JSC_FOR_EACH_VM_DEPENDENT_{EAGER,LAZY}_COMMON_THUNK`.
- `AssemblyHelpers::branchIfNumber/branchIfNotNumber(JSValueRegs, GPRReg
scratch, ...)` overloads removed (now `(JSValueRegs,
TagRegistersMode)`); `storeValue(JSValue, Address, JSValueRegs)` ->
`storeValue(JSValue, Address)`; `DataFormat.h`
`isJSFormat/isJSInt32/isJSDouble/isJSCell/isJSBoolean` removed;
`DFG::enableInt52()` removed.
- `USE(BUILTIN_FRAME_ADDRESS)` macro removed (`DECLARE_CALL_FRAME` is
unconditionally builtin-frame-address based);
`JSWebAssemblyInstance::temporaryCallFrame()/setTemporaryCallFrame()/offsetOfTemporaryCallFrame()`
removed; `ENABLE(YARR_JIT_*)` sub-flags listed above removed;
`Yarr::JITFailureReason::{DecodeSurrogatePair,BackReference,ParenthesizedSubpattern}`
removed; `WTF_CPU_ARM_VFP_V3_D32/V2` removed.
- New JSC option: `maximumInlineStringSwitchCaseCount` (default 64).
`--logGC` phase label "finalize" -> "epilogue".
- Bun impact: only the `HandleSet.h` removal required a source change
(`src/jsc/bindings/root.h`, already in this PR's diff); the other
renamed symbols are not referenced by Bun's C++ apart from stale
comments naming `finalizeUnconditionally` in
`src/jsc/bindings/ErrorStackTrace.cpp`,
`src/jsc/bindings/JSCTaskScheduler.cpp`, and
`src/jsc/bindings/FormatStackTraceForJS.cpp`.
### WebAssembly
- `2c2c1af35743` Overhauls ArrayBuffer/Wasm::Memory sizing for memory64:
`MAX_ARRAY_BUFFER_SIZE` goes from 4 GiB to 16 GiB on 64-bit (the fork
keeps 4 GiB under `BUN_JSC_ADDITIONS`, so not in Bun; Bun's
`Buffer.kMaxLength`/`MAX_LENGTH` derive from this macro in
`src/jsc/bindings/JSBuffer.h`, and `src/jsc/array_buffer.rs` `MAX_SIZE`
is a hard-coded `u32::MAX`), memory32 capped at 4 GiB and memory64 at 16
GiB, growing a shared memory64 past 4 GiB no longer crashes, and a
memory's buffer now advertises the maximum it can actually grow to; no
GC is triggered while holding the buffer-memory lock.
- `40d37f36527f` Follow-up: memory64 modules may declare arbitrarily
large page limits (parsing accepts them, as for table64); the 16 GiB cap
is enforced when the Memory is created or grown at runtime instead of
failing `WebAssembly.Module()`.
- `d6d09268899b` BBQ and OMG now always emit explicit bounds checks for
memory64 (and non-zero multi-memory) accesses via
`ModuleInformation::memoryModeForAccess()`; signaling-mode fast paths
are reserved for 32-bit memory 0 (previously a release-assert
crash/unsafe path once memory64 code tiered up).
- `72928a517633` Instances whose module declares no memory now still
reserve and zero the memory-0 cached base/size slot that every wasm
entry reads (previously it overlapped the import call-link area).
- `bfe5073f4c99` Fixes a crash when an imported memory is grown while a
multi-memory instance is only partially linked (e.g. after a LinkError
on a later import).
- `f771c5060cd7` `ref.func`, `table.get` and `array.init_elem` slow
paths now set up a FrameTracer since they can allocate wrapper functions
and GC (fixes crashes/ShadowChicken corruption).
- `0a704bb74f1e` IPInt->BBQ loop OSR entry now rejects a stack pointer
exactly at the soft stack limit (and underflow) instead of crashing
inside BBQ.
- `0c51f43daa3b` OMG recognizes naive byte-at-a-time copy/fill loops and
prepends a guarded `memory.copy`/`memory.fill` fast path; new option
`useWasmByteLoopReplacement` (default on).
- `ca730ef8b0fe` Wasm-to-JS import stubs convert an already-BigInt i64
return value inline instead of calling out to `operationConvertToI64`
(faster imports returning i64).
- `6589b2e5c18c` Tighter WasmGC struct/array allocation codegen
(constant cell size with variable allocator, DFG-like effect model so
allocations no longer clobber loads, constant-size array.new folded).
- `3eee8becf0b5` WasmGC struct layouts fill alignment gaps with smaller
fields (V8 heuristic), shrinking structs that interleave narrow and wide
fields; adds `$vm.wasmStructFieldOffsets`/`wasmStructPayloadSize`.
- `4687d7ecfefa` BBQ skips null checks for `ref.as_non_null`, `call_ref`
and `throw_ref` on non-nullable reference types, matching OMG.
- `74091f918bfc` New Air `Padding` pseudo-op that emits no code; OMG
stops running `reportUsedRegisters`, cutting OMG compile time without
the nop-related regression.
- `4af3bbad9cda` Faster JIT code emission loops in Air, BBQ and baseline
(skip disassembler-only labels, hoist loop invariants).
- `099f93fe4993` memory64/table64 JS API fixes: i64 address values are
round-tripped as BigInt in descriptors, imports and type reflection, and
a memory64's maximum bytes is clamped to what ArrayBuffer supports; adds
`addressValueFromUint64` helper.
- `b8af849be6f0` table64: `WebAssembly.Table.prototype.length` returns a
BigInt for i64 tables and `grow()` throws RangeError on an out-of-range
delta, per JS API spec.
- `b91045c99b1b` table64 maximum sizes are no longer silently truncated
to 32 bits (`Table::maximum()` is now 64-bit).
- `e942b93cdaa0` Active element segment offsets into a table64 are read
as i64 and no longer truncated to uint32.
- `02cdfb795a84` BBQ/OMG zero-extend i32 table indices when calling into
the uint64 table operations (table64 correctness).
- `aa8167a2feb9` Oversized table declarations are accepted at parse time
and rejected when the table is created/grown, so type reflection reports
the declared sizes and the failure happens at instantiation.
- `47f20d8cfd63` `call_indirect` in unreachable code now validates the
table element type and that the type index is a function type;
previously-accepted invalid modules now fail with CompileError.
- `2e8a96a8c585` memarg offsets are decoded as u64 for both memory32 and
memory64 (range-checked for memory32), and call/table immediates in
unreachable code are scanned correctly.
- `64153f963497` memory64 memarg immediates in unreachable code were
decoded differently from reachable code, producing spurious parse errors
on valid modules.
- `d319ee7c278e` A module declaring a memory64 together with any other
memory is now rejected regardless of declaration order (JSC supports
memory64 only as a single memory).
- `d02c68d04f96` `memory.size`/`memory.grow` in IPInt now record the
memidx immediate length, fixing non-minimal LEB encodings of the memory
index under multi-memory; drops the `parseMemoryIndexForBulkOp` hack.
- `102fd6db184d` OMG now passes the memory index when building
loads/stores, so accesses to non-zero memories are marked trapping
correctly under multi-memory.
- `de45b9be42db` `memory.init` overflow check uses 64-bit arithmetic
(memory64); dead `Wasm::Memory::fill/copy` removed.
- `b1b0566f244e` `table.copy` detects source/destination aliasing by
table identity rather than index, so the same table imported under two
indices copies with overlap semantics.
- `2194da86b382` Spec-aligned limits: tag/exception section limit raised
100,000 -> 1,000,000, tables may have exactly 10,000,000 entries (was
exclusive), `maxTableInitializationEntries` removed, exception-section
error message fixed.
- `9b3637884b68` `WebAssembly.Global.prototype.value` setter called with
no argument now treats it as `undefined` instead of throwing a
not-enough-arguments TypeError (WPT behavior).
- `707048fdabb7` `WebAssembly.Memory.prototype.type()` (type reflection,
behind `useWasmJSTypes`) reports the current size as `minimum`, not the
initially declared size.
- `0cc69e2993f4` / `57a1c44be6eb` BBQ pointer materialization takes a
uint64 offset (no truncation for >4 GiB memory64 addresses) and queries
address-form validity with the actual access width (folds more offsets
into addressing).
- `01a43483d35f` `WasmCalleeGroup` stops using
`ThreadSafeWeakOrStrongPtr`, which is removed from WTF
(`wtf/ThreadSafeWeakPtr.h`) as prep for making `ThreadSafeWeakPtr`
thread-safe.
- `a53d011599e7` / `5602ec36107b` Heap-wide renames reaching wasm:
`finalizeUnconditionally` -> `reconcileWeakReferencesAtGCEnd` (and
`Heap`/`IsoCellSet` accessors), `visitWeak` family ->
`reconcileWeakReferencesAtGCEnd`/`isStillLive`; no behavior change.
- `bf1dab73b14d` / `84f83abd45c9` / `6010a9ea6ce6` / `2a8926009f45`
Post-32-bit-JIT-removal cleanups touching BBQ/JSToWasm/WasmToJS: 32-bit
register pairs and scratch registers dropped, `ARMv7Assembler.h`
deleted, `USE(BUILTIN_FRAME_ADDRESS)` made unconditional; no behavior
change on x64/arm64.
### RegExp (Yarr)
- `yarr/` and `RegExp.cpp` stay at the fork's version
(oven-sh/WebKit#299 already contains the lookbehind JIT and most of the
fixes upstream landed in this range); see oven-sh/WebKit#455 for the
commit-by-commit status. Net new for Bun from this range: the `&&` /
`--` with `\P{..}` fix (`7b5e7da783f5`, ported), nothing else. Not yet
in the fork: the BMP code-unit read optimization (`bbc000ae4f3d`), the
default-off `\A` `\z` buffer boundaries (`2f66f5ed23f9`, `37f4628ab5fe`)
and the `ENABLE(YARR_JIT_*)` ifdef removal (`ac2afd10b8ac`; the fork
keeps those macros defined, so the mentions of their removal below do
not apply to this build).
### Inspector / debugger
- `1a7f711d887e` `Debugger::sourceParsed` for WebAssembly modules now
reports the module's `sourceMappingURL` custom section, so
`Debugger.scriptParsed` for wasm scripts carries a source map URL that
inspector frontends can use to map byte offsets to source.
- `49246d261276` Implements the import-text proposal (`import x from
"./f.txt" with { type: "text" }` and the dynamic-import form) behind new
`useImportText` (default on); in this area it only teaches
`InspectorDebuggerAgent` about the new `SourceProviderSourceType::Text`,
but the module-loader API changes (listed below) affect embedders with
custom loaders.
- `4ccb3f3a1c85` / `af624adbb3bc` / `de82d6282625` / `86575c4e1516` /
`87399235b55a` / `fcd024f84dbb` / `1240a421fe56` Protocol schema churn
in the WebCore-only Canvas and Recording domains plus a new generic
`Size` type in `GenericTypes.json`; these flow into
`CombinedDomains.json` (and therefore into regenerated
bun-inspector-protocol types) but change no JSC agent behavior.
### Build / scripts
- `81d660ceeb2e` wkbuiltins generator now precomputes builtin executable
metadata (`BuiltinSourceMetadata`) at build time instead of scanning
sources at VM startup; `BuiltinExecutables::createBuiltinExecutable`
gains a metadata parameter (the free `JSC::createBuiltinExecutable(VM&,
...)` that Bun uses is unchanged).
- `ff64aee116d4` `HandleSet`/`HandleBlock` replaced by
`StrongSet`/`StrongBlock` (Sources.txt/CMakeLists updated): `Strong<>`
slots are allocated from a libpas-style segregated freelist, cheaper and
smaller; `<JavaScriptCore/HandleSet.h>` no longer exists and
`Heap::handleSet()` is now `Heap::strongSet()`.
- `62692012c98a` `heap/HeapFinalizerCallback.{h,cpp}` renamed to
`GCCompletionCallback.{h,cpp}` with
`Heap::add/removeHeapFinalizerCallback` ->
`add/removeGCCompletionCallback`; the C API
`JSContextGroupAdd/RemoveHeapFinalizer` keeps its names.
- `3c64729cefbc` Fixes a clang 18 `-Wthread-safety-precise`/constexpr
build break in `WasmCalleeGroup.cpp`.
- `b9d3ef9f6a0f` Removes the dead `JettisonDueToProfiledWatchpoint`
value from the profiler's `JettisonReason` enum.
### Embedder-relevant API changes
- `MAX_ARRAY_BUFFER_SIZE` (runtime/PageCount.h) is now `1 << 34` on
64-bit (was `1 << 32`); `PageCount::maxPageCount` is public and
redefined; `Wasm::maxMemoryPages` renamed `maxMemory32Pages`,
`maxMemory64Pages` redefined, `maxTableInitializationEntries` removed;
new
`Wasm::maxDeclarablePages/maxBufferByteLength/maxAllocatableBytes(AddressType)`;
`Gigacage::primitiveAddressSpaceBudget` added in bmalloc.
- `ArrayBuffer::grow(const AbstractLocker&, VM&, ...)` replaced by
`ArrayBuffer::tryGrow(const AbstractLocker&, size_t, bool,
BufferMemoryResult::Kind&)`; `Wasm::Memory::fill()`/`copy()` removed
(use `Wasm::memoryFill/memoryCopy`); `Wasm::Table::maximum()` is now
64-bit.
- WTF: `ThreadSafeWeakOrStrongPtr` removed from
`wtf/ThreadSafeWeakPtr.h`; `USE(BUILTIN_FRAME_ADDRESS)` removed
(`DECLARE_CALL_FRAME`/`DECLARE_WASM_CALL_FRAME` always use the
frame-address form);
`ENABLE(YARR_JIT_ALL_PARENS_EXPRESSIONS|YARR_JIT_BACKREFERENCES|YARR_JIT_BACKREFERENCES_FOR_16BIT_EXPRS|YARR_JIT_UNICODE_EXPRESSIONS|YARR_JIT_REGEXP_TEST_INLINE)`
removed from PlatformEnable.h.
- Yarr:
`JITFailureReason::{DecodeSurrogatePair,BackReference,ParenthesizedSubpattern}`
removed; `Yarr::parse()` gained a defaulted trailing
`allowRegExpBufferBoundaries` parameter; new
`Options::useRegExpBufferBoundaries` (off by default).
- Module loading (from import-text): `SourceProviderSourceType::Text`
inserted before `ImportMap` (renumbers `ImportMap`; Bun's fork also
appends `BunTranspiledModule`), `ScriptFetchParameters::Type::Text`
inserted before `HostDefined` (HostDefined becomes 5, matching the
updated static_asserts in
`src/jsc/bindings/BunAnalyzeTranspiledModule.cpp`),
`SyntheticModuleRecord::create()` and the `AbstractModuleRecord`
constructor now take a `SourceProviderSourceType` (already adapted in
`src/jsc/bindings/NodeVMSyntheticModule.cpp`), new
`SyntheticModuleRecord::createTextModule()`, new
`Options::useImportText` (on by default).
- Heap renames: `T::finalizeUnconditionally()` ->
`reconcileWeakReferencesAtGCEnd()`,
`Heap::finalizeUnconditionalFinalizers()` ->
`reconcileWeakReferencesAtGCEnd()`, `Heap::...::finalizerSetFor()` ->
`weakReconciliationSetFor()`, `CallLinkInfo::visitWeak` and friends ->
`reconcileWeakReferencesAtGCEnd`; `HeapFinalizerCallback` ->
`GCCompletionCallback` (header renamed);
`HandleSet.h`/`HandleBlock.h`/`HandleBlockInlines.h` removed in favor of
`StrongSet.h`/`StrongBlock.h`, `Heap::handleSet()` -> `strongSet()`
(Bun's `src/jsc/bindings/root.h` include already switched).
- `BuiltinExecutables::createBuiltinExecutable()`/`createExecutable()`
gained `const BuiltinSourceMetadata&` overloads (member function
signature changed; free function unchanged);
`JettisonReason::JettisonDueToProfiledWatchpoint` removed;
`assembler/ARMv7Assembler.h` deleted;
`Wasm::ModuleInformation::memoryModeForAccess()` added.
### WTF
- `f6bc402b8344` `StackBounds::currentThreadStackBounds()` is now
private (only `Thread`/`StackStats` may call it; other code is meant to
read the cached `Thread::currentSingleton().stack()`) because on Linux
it can re-parse `/proc/self/maps` on every call. Bun's
`Bun__StackCheck__initialize` called it once per thread, including on
non-WTF threads, and now uses the
`currentThreadStackBoundsForEmbedder()` accessor the fork adds under
`USE(BUN_JSC_ADDITIONS)`. `36403ca62849` re-adds `WTF_EXPORT_PRIVATE` on
`currentThreadStackBoundsInternal()`.
- `5fc5182bcf83` `WTF::numberOfProcessorCores()` now honors a
`NUMBER_OF_PROCESSORS` env var (after the existing
`WTF_numberOfProcessorCores`) before asking the OS. Bun reports this
value as `navigator.hardwareConcurrency` / `os.availableParallelism()`
and it would take precedence over the fork's affinity/cgroup aware
count, so the fork keeps this lookup out of Bun builds
(oven-sh/WebKit#455, 8fc20b18b9); no change for Bun.
- `957b52180bee` `MemoryPressureHandler` no longer inherits
`CanMakeWeakPtr` (timers bind to the singleton via lambdas); fixes a
debug-build WeakPtr thread assertion when the singleton is first touched
off the main thread (JSC's `FullGCActivityCallback` does this, e.g. from
a Worker), and `s_hasCreatedMemoryPressureHandler` is now only set once
the singleton really exists.
- `4a10860dc35c` `WTF::Liveness` iterates less (no separate boundary
pass, no zeroing of the gen store) and gains
`forEachLiveAtHeadNotLiveAtTail` / `forEachLiveAtTailNotLiveAtHead`;
used by the Air greedy register allocator (~17% faster
`buildLiveRanges`), i.e. lower DFG/FTL/OMG compile latency.
- `01a43483d35f` `ThreadSafeWeakOrStrongPtr` removed from
`wtf/ThreadSafeWeakPtr.h` (its only user, `Wasm::CalleeGroup`, was
rewritten); groundwork for shrinking `ThreadSafeWeakPtr` to one pointer
and making it atomic.
- `ac2afd10b8ac` Removes the `ENABLE_YARR_JIT_*` sub-feature macros
upstream (unconditional on x64/arm64). The fork keeps them defined
because its YarrJIT still tests them; no behavior change either way.
- `6010a9ea6ce6` ARMv7 JIT removal follow-ups: drops
`CPU(ARM_VFP_V2)`/`CPU(ARM_VFP_V3_D32)`, simplifies
`ASSERT_VALID_CODE_POINTER`, `ENABLE(JUMP_ISLANDS)` is now arm64-only
and `LLINT_EMBEDDED_OPCODE_ID` drops Thumb2; no effect on x64/arm64
builds.
- `2a8926009f45` `USE(BUILTIN_FRAME_ADDRESS)` macro removed; JSC now
unconditionally uses `__builtin_frame_address` on JIT platforms. The
fork had it disabled on Windows ARM64 only; that fallback is gone with
this merge (see the Windows ARM64 note above).
- `ef6d9ba26b17` removed Linux real-time threads in favor of nice/RTKit
priorities, `44bab332e0f1` fixed its JSCOnly build, and `56baf6e01b3d`
reverted the whole thing for ~2% JetStream3/Speedometer3 regressions:
net zero change to `Threading.h`/`AutomaticThread`/`RealTimeThreads.cpp`
in this range.
- `1240a421fe56` Additive
`JSON::Array::set{Boolean,Integer,Double,String,Value,Object,Array}(index,
…)` and `JSON::ArrayOf<T>::setItem(index, …)` (in-place replacement;
`RELEASE_ASSERT`s index in range) in `wtf/JSONValues.h`, which Bun's
inspector/profiler bindings include.
- `5720766c8056` Reverts the IPC URL-size limit, removing the
`WTF::maxURLLength` constant from `wtf/URL.h`; no URL parsing behavior
change.
- `3089b5074c3d` Deletes the empty `wtf/text/WYHash.h`; any `#include`
of it now fails (Bun has none).
- `e9a62e6b4da5` `85e82ceefe1b` `793e36fb835e` `934bb002485a`
`2f66f5ed23f9` `49246d261276` `99473681ff5e` only touch
`Scripts/Preferences/UnifiedWebPreferences.yaml` on the WTF side,
mirroring JSC option changes (iterator join / import defer / iterator
chunking / joint iteration flipped to default-on, new RegExp
buffer-boundaries and import-text prefs, `IntlEraMonthcodeEnabled` pref
removed since the feature is now unconditional); the actual behavior
lives in the JavaScriptCore commits. All other yaml-only commits in this
range are WebCore/WebKit feature flags and irrelevant to Bun.
**Embedder-relevant API changes**
- `StackBounds::currentThreadStackBounds()` is private (`friend class
Thread`); replacement is `Thread::currentSingleton().stack()`
(`f6bc402b8344`).
- `WTF::ThreadSafeWeakOrStrongPtr` removed (`01a43483d35f`).
- Header `wtf/text/WYHash.h` removed (`3089b5074c3d`); header
`wtf/Nonallocatable.h` added and `RefCountedWithInlineWeakPtrBase`
removed / `RefCountedWithInlineWeakPtr<T>` made non-`new`-able
(`f880bc57ad50`).
- `using WTF::Task` removed from `wtf/CoroutineUtilities.h`
(`9f82586af24c`); `WTF::maxURLLength` removed from `wtf/URL.h`
(`5720766c8056`).
- `MemoryPressureHandler` no longer derives from `CanMakeWeakPtr` and
lost its no-op `ref()`/`deref()` (`957b52180bee`).
- Config macros removed: `USE(BUILTIN_FRAME_ADDRESS)`,
`ENABLE(YARR_JIT_ALL_PARENS_EXPRESSIONS)`,
`ENABLE(YARR_JIT_REGEXP_TEST_INLINE)`,
`ENABLE(YARR_JIT_BACKREFERENCES)`,
`ENABLE(YARR_JIT_BACKREFERENCES_FOR_16BIT_EXPRS)`,
`ENABLE(YARR_JIT_UNICODE_EXPRESSIONS)`, `CPU(ARM_VFP_V2)`,
`CPU(ARM_VFP_V3_D32)`; `ENABLE(JUMP_ISLANDS)` now arm64-only; new
`ENABLE(JIT_CAGE_RELAXATION)`.
- Additive only: `JSON::Array::set*`/`ArrayOf<T>::setItem`,
`Liveness::forEachLiveAt{Head,Tail}NotLiveAt{Tail,Head}`,
`WTF::isInBaseSystem()` (Cocoa port only, not compiled in JSCOnly/Bun),
`numberOfProcessorCores()` reading `NUMBER_OF_PROCESSORS`.
### bmalloc
- `2c2c1af35743` Adds `Gigacage::primitiveAddressSpaceBudget` (a
`constexpr uint64_t`, 64 GB on 64-bit desktop/server targets, 16 GB on
iOS/32-bit) to `Gigacage.h`, defined *outside* `#if GIGACAGE_ENABLED` so
it exists even when the Gigacage is compiled out;
`primitiveGigacageSize` is now derived from it (same value as before, so
no cage-size change). This is the bmalloc half of the ArrayBuffer/Wasm
memory64 sizing overhaul: JSC's `BufferMemoryHandle.h` uses it to cap
the virtual reservation of any one resizable `ArrayBuffer` / growable
`SharedArrayBuffer` / `WebAssembly.Memory` at budget/4 (16 GB on Bun's
platforms), which is what lets those buffers reach the new 16 GB
`MAX_ARRAY_BUFFER_SIZE` (previously 4 GB) and lets memory64 grow past 4
GB without crashing. (The fork pins `MAX_ARRAY_BUFFER_SIZE` at 4 GB, so
in Bun only the crash fix applies.)
Embedder-relevant API changes (bmalloc):
- Added: `Gigacage::primitiveAddressSpaceBudget` (`constexpr uint64_t`)
in `Source/bmalloc/bmalloc/Gigacage.h`; `primitiveGigacageSize`
unchanged in type and value. No removals or renames. Bun's own C++
(`src/jsc/bindings`) references no Gigacage symbols, so nothing on the
Bun side needs updating for this area.
</details>
<!-- robobun:evidence:begin -->
---
**[decide:webkit]** gate passed · iteration 0 · 8 files touched
<!-- robobun:evidence:end -->
---------
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>
3 participants
Yarr (JavaScriptCore's RegExp engine): JIT-compile lookbehinds, factor and dispatch wide alternations on their first code point, extend the Boyer-Moore search, align
/u·/vsemantics with the spec, and fix a set of correctness bugs in both tiers.Motivating case: oven-sh/bun#5197 — the
isbotuser-agent regex (~110 alternatives, four lookbehinds) ran ~200× slower in Bun than in Node because any lookbehind sent the whole pattern to the bytecode interpreter and wide alternations were tried one alternative at a time.What changes
Lookbehinds are JIT-compiled. Each lookbehind body is compiled as a term-reversed mirrored copy driven through the existing forward machinery, with a per-op
MatchDirectionflipping only the primitives (input claim, character addressing, end-of-input, greedy signs, capture start/end slots, mirrored^ $ \b). Unicode patterns are covered (leftward code-point reader, backward astral literals/classes in every quantifier form, backreferences with a frame slot for the span edge, quantified/nested groups, forward assertions nested in a mirrored body); backward greedy variable-width classes under/ubacktrack in O(1) per step. If a mirrored body's layout disagrees with the pattern's offsets the compile falls back to the interpreter instead of asserting.Wide alternations are cheaper. Engine-neutral
YarrPatternrewrites factor shared literal prefixes (/aq|bx|ar|by/→/a(?:q|r)|b(?:x|y)/, recursively and inside existing groups, but never beneath a group that can repeat — see below) and fold a wide top-level alternation into one non-capturing group; the rewrite moves terms rather than copying and runs against a per-pattern work budget, so adversarial inputs stay linear in memory. In the JIT, a once-through group's alternatives are dispatched on their first code point: a decision tree over Latin-1, and a second one over code-point ranges above it (whole code points when the compile decodes surrogate pairs), each leaf jumping to the ordered chain of alternatives that can start there. Short literal alternatives are compared inline in the chain with 16/32/64-bit loads and emit no separate body; other alternatives are entered through a stub (newchainResumeslot in the once-group frame). Sizing keeps small patterns in their existing code shape (bodies of < 16 alternatives are folded only if some alternative already needs a Yarr frame, groups dispatch from 4 alternatives / 12 literal characters, at most 2,048 chains / 4,096 stubs, and preparation abandons or falls back the moment a bound is passed). A top-level alternation whose alternatives' first-character sets are pairwise disjoint (keyword lists) is dispatched directly, with no group and no frame, so.test()loops keepRegExpTestInline.Classes that contain strings (
\q{…},\p{RGI_Emoji}and the other properties of strings, results of/vset operations) expand to one alternative per string, longest first, then the single characters, then the empty member; a list of four or more strings gets its own once-through group so the dispatcher serves it even under a quantifier./^\p{RGI_Emoji}$/v.test()6–9 µs → 0.03–0.05 µs (V8 0.05–0.09);\p{RGI_Emoji}+21 µs → 0.02 µs per emoji;string-width@8in Bun goes from ~15× slower than Node to slightly faster.The Boyer-Moore search reaches more patterns and no longer depends on the first subject a RegExp saw. Every sub-range of candidate positions is scored by the stride it buys against how often its union hits; positions matching every character are excluded; sampled character frequencies are used only when the sample is long enough to resolve them; a single-position range is scored for the 16-byte vector scan when the compile can emit it. On x86_64 the vector scan is always used for Latin-1 subjects (as before); arm64 keeps a measured candidate-frequency gate.
Spec alignment for
/u·/vmatch starts (both tiers). AlastIndexthat splits a surrogate pair maps to the pair's lead, and a non-sticky match whose start would split a pair is never reported (RegExpBuiltinExec'sinputIndex+ AdvanceStringIndex). This is the one intentional remaining difference from V8, which attempts mid-pair positions.Resource behaviour.
maxRegExpStackSize128 → 192 MB (interpreter frames grew with the new once-group slots; nested-quantifier tokenizers over ~600 KB inputs otherwise gave up ~20% sooner than before). The interpreter'sBumpPointerAllocatorkeeps up to 64 KB of already-mapped overflow pools between matches instead of mapping/unmapping them around every match (js-tokens-style tokenizers: 29k mmap/munmap pairs → 49), andVM::deleteAllCodereleases them.Behaviour changes vs. current
main(each was JSC ≠ V8/spec unless noted)RegExp semantics — both tiers unless noted:
\p{…}/\P{…}under/iuand/ivtake part in case folding (standalone, in classes, negated classes,/vset operations), per CharacterSetMatcher; test262's RegExp subset goes 2533 → 2535/2535./.*X.*/fast path): honouredlastIndexunder/s; a leading^is re-checked whenlastIndex > 0(also/m); anXthat can consume a line terminator no longer widens to the wrong line; captures inside lookarounds inXare seen. Linear in all^cases; U+2028/2029 handled.String.prototype.replacematches at and updateslastIndex(C++ fast paths, DFG empty-replacement operation, DFG constant folding).\k<name>to (duplicate-)named groups captured before the lookbehind.^groups (/(?:^)?a/,/(?:^b)?a|de/);^-anchored alternatives are never filtered beneath an inverted assertion./v: set operations keep a class's astral width; an inverted property escape honours a pending&&/--; nested negated class with astral input; chained--; unions of string-bearing operands are merged; duplicate\q{}members are one member; the empty\q{}member matches last;[\q{zz}b]no longer matches U+0000;[\q{ab}\-]/[[a-z]\-]accepted and[\d&&-]rejected as the grammar says./ulookbehinds; once-through alternative order after the first position.<class><literal>after an astral char not at position 0; string-list group with an astral literal on 16-bit input; a unicode backreference walk past a lone-surrogate capture's end (unbounded read); the dotAll class rejects a dangling surrogate under/u; a{0,n}split copy that ran zero iterations no longer clears captures it shares with its mandatory sibling; a lazily quantified backreference whose extra iteration fails (zero-width, not enough input, or different characters) fails the term as the interpreter does —/z(bb)(?:.\1*?)+z/on"zbbcz"+"c".repeat(30)returned null after 700 ms onmain's JIT."\u{1F600} x".replace(/\s*/gu, "")(and DFG's folding of it) advances past a surrogate pair after an empty/umatch rather than looping.RegExp::matchInlinekeeps the non-unicode/8-bit paths to a flag test plus a tail call.Performance (x86_64, retired instructions, jsc shell, this branch ÷
main; wall-clock tables for arm64 in the PR thread).test()/(?<=X[^"]*)!/uover 5 KB ×2000\b(?:…)\b/gover 300 KB/NEEDLE\d+/,/zqx\d/,/zalpha|qbravo|xcharlie/) over 130 KB/^\p{RGI_Emoji}$/v.test,\p{RGI_Emoji}+per emoji/\d+px/g,/#[0-9a-fA-F]{6}\b/g, trailing-space/[ \t]+$/gm\s+split; email validate; semver ×400k.test()loops (bool / http-method / ext / alt8)\p{RGI_Emoji}0.5×Known trade-offs kept: picomatch-globstar-class patterns run ~1.6×
main's JIT instructions (optional^groups no longer take the once-through path that produced wrong results);(…)+groups still fall back to the interpreter once the JIT's stack-carved paren-context pool is exhausted (~60k iterations of a dispatched string list;mainbehaves the same, follow-up planned to move that pool to the heap).Verification
Against current
mainand Node 24–26, on Release, RelWithDebInfo and Debug+ASAN builds of this head:regexp*/*lookbehind*/*yarr*/string-*{match,replace,split}*(311 files) on JIT,--useRegExpJIT=0, all new gates off, Debug+ASAN JIT and interpreter: identical tomainin every configuration. test262 RegExp/String subset (2,535 files) × both tiers: 2535/2535.Tools/yarr-fuzz, in tree): grammar-generated (10 profiles: lookbehind nesting, wide/shared-prefix and wide-headed alternations,/u·/vwith astral text and lone surrogates,\q{}/string-property classes, Boyer-Moore bait, deep nesting, DFG constant folding, small hot patterns), structure-mutated real-world regexes (60k patterns from a 685k-pattern npm/PyPI corpus, AST-level mutations), and a spec-literal reference matcher as an engine-independent oracle; every case as exec/test/match/matchAll/replace/split/search with sticky andlastIndexsweeps, 8-bit and forced 16-bit subjects, plus metamorphic wrappers. ≈ 20 M cases across JIT / interpreter / each gate off / eager DFG-FTL /main/ V8, ≈ 1.5 M under Debug+ASAN. Result on this head: 0 crashes or sanitizer reports; JIT ≡ interpreter except step-budget exhaustion on catastrophic patterns (pre-existing, both directions); gates-off ≡ default; every difference from V8 is the mid-surrogate rule above, JSC's existing size limits, or a V8 bug; every difference frommainis one of the fixes above./vwhere they compile: same conclusion; 20 nested-quantifier tokenizers × 0.4–1 MB inputs identical across all engines.--verifyRegExpJITReadsoption bounds-checks every load the RegExp JIT emits against the subject; four planted bugs of that shape (pair peek without its end check, backward reader without its start check, lookbehind offset off by one, vector-scan bound loose by 8 bytes) are each caught within seconds by the fuzzer, and the unmodified tree is clean over 830k verified cases (all profiles plus 58k mutated real-world patterns). A continuous-GC configuration (30k cases + the key stress tests) is also clean.New tests:
regexp-lookbehind-jit.js,regexp-review-regressions.js,regexp-interpreter-unicode-lookbehind.js,regexp-interpreter-once-through-order.js,regexp-unicode-no-mid-surrogate-pair-start.js,regexp-bm-search-any-character-position.js,regexp-unicode-fixed-class-then-single-class-backtrack.js,regexp-unicode-empty-match-replace-advances-past-surrogate-pair.js,regexp-dotstar-enclosure-soundness.js,regexp-unicode-property-escape-ignore-case.js,regexp-lookbehind-forward-reference-resolution.js,regexp-sticky-non-global-replace-uses-lastindex.js,regexp-class-strings-longest-first.js,regexp-shared-code-across-realms.js.Pre-existing issues seen and deliberately left for follow-ups (present on
main):/vicase folding of multi-character\q{}members inside set operations; a family of/vclass-set syntax leniencies ([a\d&&c],[a[b]--c]accepted; a raw U+0000 inside a/vclass rejected);{n}above 2³²; the interpreter reporting no match rather than an error when its step budget is exhausted.Options
useRegExpLookbehindJIT,useRegExpAlternationFactoring,useRegExpAlternationDispatch(default true; each restores the previous behaviour of that feature —BUN_JSC_useRegExpAlternationDispatch=0etc. in Bun, and…Dispatch=0alone restoresmain's exact code shape for\p{RGI_Emoji});regExpDispatchMaxInlineLiteralLength(default 32; 0 never compares alternatives inline) trades dispatch code size for speed;maxRegExpStackSizedefault 192 MB;verifyRegExpJITReads(default false) is the fuzzing aid described above.