Prevent WebGL atlas pages from exceeding texture capacity by anthonykim1 · Pull Request #6043 · xtermjs/xterm.js

anthonykim1 · 2026-07-10T00:44:45Z

Part of: #6038
Part of: microsoft/vscode#322756
Follow-up to: #6042

Keep the existing exact-four, same-sized atlas page merge behavior.
Evict and rebuild the glyph atlas when it reaches maxAtlasPages and that merge is unavailable, instead of adding a page the renderer cannot use.
Apply the same capacity check before creating an oversized-glyph page.
Increment pageLayoutVersion after eviction so every renderer sharing the atlas rebuilds its cached glyph mappings.
Clamp the texture upload loop as a defensive backstop so an unexpected overflow cannot throw while rendering.
Add deterministic Chromium coverage for normal page overflow, oversized glyphs, shared terminals, rendered pixels after eviction, and mid-frame retry stability.

TDD evidence:

Red: 8237004d5 reproduces both overflow paths on the PR 1 baseline with TypeError: Cannot read properties of undefined (reading 'version').
Green: 2b982bc39 keeps the atlas within renderer capacity and passes both tests.

Steps to verify:

npm run build
npm run esbuild
npm run esbuild-demo-client
npm run esbuild-demo-server
npm run test-integration-chromium -- --suite=addon-webgl
npm run lint-changes

Inspirations from:

Preserving exact-four merging comes from the guard added to fix the earlier stack overflow in TextureAtlas._createNewPage.
The capacity invariant comes from GlyphRenderer allocating its shader and texture arrays from TextureAtlas.maxAtlasPages, including one texture per supported page in the _atlasTextures initialization.
The eviction cache reset follows the existing cache-clearing behavior in TextureAtlas.clearTexture, while additionally removing the discarded pages.
Handling oversized glyphs at the same capacity boundary comes from the existing direct _overflowSizePage insertion path in TextureAtlas.
Mid-frame eviction recovery relies on the existing full-model retry behavior in WebglRenderer.renderRows.

anthonykim1 · 2026-07-13T18:19:09Z

    }
  }

+  private _evictAllPages(): void {


@Tyriar Would this be all we need in order to evict pages properly?

… budget The fragment shader has sampler slots for maxAtlasPages (16 on most Macs) and leaves outColor uninitialized for any higher page index, so glyphs rasterized onto pages past the budget render as garbled pixels. Long sessions grow past the budget via the merge fallback, and the previous wipe fix re-activated those unbindable pages, so every atlas wipe re-allocated glyphs onto them (post-wipe allocation prefers the last, highest-index active page) and garbled whole panes mid-stream. Fix, matching the direction xterm.js maintainers are pursuing upstream (xtermjs/xterm.js#6043): a shared _evictAllPages resets the atlas to one fresh page, called from clearTexture and from the two allocation paths that could otherwise push a page past the budget (merge fallback and oversized-glyph page creation), so the page count can never exceed the renderer's texture capacity. Defensive backstops: a one-time warn plus bind-loop clamp, and an else branch in the generated shader so an unexpected overflow renders blank instead of undefined pixels.

… budget (#8672) * fix(terminal): keep WebGL glyph atlas pages within the shader sampler budget The fragment shader has sampler slots for maxAtlasPages (16 on most Macs) and leaves outColor uninitialized for any higher page index, so glyphs rasterized onto pages past the budget render as garbled pixels. Long sessions grow past the budget via the merge fallback, and the previous wipe fix re-activated those unbindable pages, so every atlas wipe re-allocated glyphs onto them (post-wipe allocation prefers the last, highest-index active page) and garbled whole panes mid-stream. Fix, matching the direction xterm.js maintainers are pursuing upstream (xtermjs/xterm.js#6043): a shared _evictAllPages resets the atlas to one fresh page, called from clearTexture and from the two allocation paths that could otherwise push a page past the budget (merge fallback and oversized-glyph page creation), so the page count can never exceed the renderer's texture capacity. Defensive backstops: a one-time warn plus bind-loop clamp, and an else branch in the generated shader so an unexpected overflow renders blank instead of undefined pixels. * test(terminal): cover WebGL atlas sampler budget * fix(terminal): align WebGL atlas invalidation source

Upstream 8aab310..699f553 is entirely addon-webgl atlas work: - xtermjs#6042 replaces the consume-once ITextureAtlas.beginFrame() flag with a monotonic pageLayoutVersion, so every renderer sharing an atlas learns about a page-layout change instead of only the first one to ask. - xtermjs#6043 adds _evictAllPages() so the atlas can't grow past the renderer's texture capacity, plus a defensive clamp in GlyphRenderer.render. - xtermjs#6056 adds atlas overflow/stress/shared-garble test coverage. The merge is textually clean; the semantic follow-up to _evictAllPages is in the next commit.

Upstream xtermjs#6043 added _evictAllPages(), which destroys every atlas page and drops the two upstream caches of IRasterizedGlyph records. The SDF path keeps a third such cache: _sdfGlyphCache, keyed by shape rather than shape+color, because a distance field depends only on the glyph outline and color is applied per cell in the shader. Nothing conflicted, because _evictAllPages is new code that has never seen the fork — but the records it left behind carry texturePage indices into the array it just emptied. Since eviction also clears _cacheMap, the very next glyph is a cache miss that routes into the SDF path, hits the surviving canonical record, and indexes past the end of _pages while registering its color alias. That throws in GlyphRenderer.updateCell rather than degrading. WebglSdfAtlasEviction covers it, alongside the upstream tests for the raster equivalent. It asserts the general invariant rather than the cache being empty, since the glyph flood that forces the eviction immediately refills it: no cached SDF record may point at a page that no longer exists. Without the fix it reports 191 stale records and a renderer throw. Also rewrites the addon import in the three merged-in upstream test files to the published fork name, without which they do not typecheck, and records both this and the merge-review process in FORK.md.

…r, Safari autofill (#2130) Bump the terminal engine from **`@xterm/xterm@6.1.0-beta.289` / `@xterm/addon-webgl@0.20.0-beta.288`** to the latest published betas — **`6.1.0-beta.300` / `0.20.0-beta.297`**. kolu rides the 6.1 beta line via `pnpm.overrides` (stable latest is still 6.0.0). Other addons and `@xterm/headless` stay on their package ranges, same as the previous bump (#1830). ### What's new in xterm.js (since beta.289) | Upstream | What it does for tiles | | --- | --- | | [xtermjs#6043](xtermjs/xterm.js#6043) | When the **shared WebGL glyph atlas** would exceed texture capacity, it **evicts and rebuilds** instead of adding a page the renderer can't use — and bumps `pageLayoutVersion` so every live tile sharing the atlas refreshes. Follow-on to the page-layout invalidation we already took in #1830. | | [xtermjs#6055](xtermjs/xterm.js#6055) | `clearTexture()` now bumps the same layout version. Clearing the atlas on **one** tile no longer leaves **siblings painting stale glyphs** from wiped atlas pages (VS Code hit this on OS resume; multi-tile WebGL is exactly our setup). | | [xtermjs#6057](xtermjs/xterm.js#6057) | Helper textarea gets `autocomplete="off"`, so **iPadOS Safari** stops offering password / AutoFill suggestions when a terminal focuses. Desktop unchanged. | Also upstream in the range: extra WebGL atlas stress coverage for TUI-style alternate-buffer redraws (xtermjs#6056), plus dependency and docs-only noise. ### This PR - `pnpm.overrides`: xterm **289→300**, addon-webgl **288→297** (matched pair) - Refresh `fetchPnpmDeps` hash in `nix/workspace.nix` - Changelog under *Inside a Tile* naming the three user-visible upstream fixes - `@kolu/xterm-kit` contract suites green on the new pin (94/94) ### Try it locally ```sh nix run github:juspay/kolu/chore/xterm-beta-300 ``` _Generated by Grok Build (model `grok-4.5`)._

The star tier: each distinct Home — a chat's Row.Project, the same identity the picker groups by — is a sun. One project holds the galactic centre; several share the ring, spread wide-axis first so two stars land left and right, each with its own tighter system of planets. Residence wins over lineage: a chat spawned into another star's system is a planet of the star it lives under, and the spawn edge alone carries the bloodline between systems. A ghost with no row seats under the shared "unknown" star — its home is genuinely unanswerable, never guessed. A sun's radiance is its population: heat lifts the colour from warm gold toward white as chats gather, the corona widens and multiplies, three planets grow four twinkling diffraction spikes, five erupt periodic solar flares — all deterministic in (clock, home, population), so pinned test clocks render the same sky. Sun glyphs replay last: a star can never be erased by its planets. Safe-mode comments corrected per upstream research: no app-side output shaping is proven to prevent the shared-atlas corruption (xtermjs/xterm.js#6043) — safe mode is pressure relief until the xterm.js 6.1.0 atlas-eviction fix ships in stable VS Code. Fenced gate: dev.sh iso test pfm green (54 packages ok); all three cosmos goldens watched failing before regeneration.

… budget (stablyai#8672) * fix(terminal): keep WebGL glyph atlas pages within the shader sampler budget The fragment shader has sampler slots for maxAtlasPages (16 on most Macs) and leaves outColor uninitialized for any higher page index, so glyphs rasterized onto pages past the budget render as garbled pixels. Long sessions grow past the budget via the merge fallback, and the previous wipe fix re-activated those unbindable pages, so every atlas wipe re-allocated glyphs onto them (post-wipe allocation prefers the last, highest-index active page) and garbled whole panes mid-stream. Fix, matching the direction xterm.js maintainers are pursuing upstream (xtermjs/xterm.js#6043): a shared _evictAllPages resets the atlas to one fresh page, called from clearTexture and from the two allocation paths that could otherwise push a page past the budget (merge fallback and oversized-glyph page creation), so the page count can never exceed the renderer's texture capacity. Defensive backstops: a one-time warn plus bind-loop clamp, and an else branch in the generated shader so an unexpected overflow renders blank instead of undefined pixels. * test(terminal): cover WebGL atlas sampler budget * fix(terminal): align WebGL atlas invalidation source

anthonykim1 and others added 4 commits July 9, 2026 15:51

Add WebGL atlas page overflow repros

8237004

Prevent WebGL atlas page overflow

2b982bc

Merge branch 'master' into anthonykim1/webgl-atlas-page-cap-eviction

5501e93

Log to measure eviction time

6791a0a

anthonykim1 self-assigned this Jul 13, 2026

anthonykim1 added this to the 7.0.0 milestone Jul 13, 2026

anthonykim1 requested review from Tyriar and jerch July 13, 2026 18:08

Don't mess with formatting

06ce754

anthonykim1 commented Jul 13, 2026

View reviewed changes

anthonykim1 added 2 commits July 13, 2026 11:23

Useful comments.

2ae6e3a

Clean up WebGL atlas overflow coverage

2ea67f9

anthonykim1 marked this pull request as ready for review July 13, 2026 18:51

brennanb2025 mentioned this pull request Jul 14, 2026

fix(terminal): keep WebGL glyph atlas pages within the shader sampler budget stablyai/orca#8672

Merged

anthonykim1 merged commit ce21694 into xtermjs:master Jul 15, 2026
12 checks passed

This was referenced Jul 17, 2026

Add WebGL atlas alternate-buffer stress test and refine the manual rendering demo #6056

Merged

Bump xterm to 6.1.0-beta.291 microsoft/vscode#326772

Merged

binaricat mentioned this pull request Jul 24, 2026

fix(terminal): prevent Herdr rendering freezes binaricat/Netcatty#2472

Merged

srid mentioned this pull request Aug 9, 2026

chore(deps): bump xterm.js to beta.300 — atlas eviction, sibling-clear, Safari autofill juspay/kolu#2130

Merged

Kitenite mentioned this pull request Aug 9, 2026

fix(desktop): bump @xterm/addon-webgl to 0.20.0-beta.297 for atlas page eviction ParadevAI/Paradev#6287

Merged

tjx666 mentioned this pull request Aug 11, 2026

为 Terminal 增加可回退的 WebGL 渲染模式 tjx666/caxri#96

Closed

6 tasks

This was referenced Oct 27, 2026

Terminal glyphs become fragments or wrong characters while buffer text stays correct getpaseo/paseo#6506

Closed

fix(app): update xterm to prevent shared-atlas glyph corruption getpaseo/paseo#8013

Merged

Navigation Menu

Repository navigation

Prevent WebGL atlas pages from exceeding texture capacity - #6043

Prevent WebGL atlas pages from exceeding texture capacity#6043
anthonykim1 merged 7 commits into
xtermjs:masterfrom
anthonykim1:anthonykim1/webgl-atlas-page-cap-eviction

anthonykim1 commented Jul 10, 2026 •

edited

Loading

Uh oh!

anthonykim1 Jul 13, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

anthonykim1 commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anthonykim1 Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

anthonykim1 commented Jul 10, 2026 •

edited

Loading

anthonykim1 Jul 13, 2026 •

edited

Loading