Repository navigation
[lexical-code-prism][lexical-code-shiki][lexical-mark][lexical-website][lexical-y] Bug Fix: Keep comment ids and NodeState when a block change moves or re-creates text - #9503
Merged
Conversation
zurfyx
requested review from
acywatson,
etrepum,
ivailop7 and
potatowagon
as code owners
October 28, 2026 23:34
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
meta-cla
Bot
added
the
CLA Signed
lexical
@lexical/a11y
@lexical/clipboard
@lexical/code
@lexical/code-core
@lexical/code-prism
@lexical/code-shiki
@lexical/compiler
@lexical/devtools-core
@lexical/dragon
@lexical/eslint-plugin
@lexical/extension
@lexical/fast-check
@lexical/file
@lexical/hashtag
@lexical/headless
@lexical/history
@lexical/html
@lexical/internal
@lexical/link
@lexical/list
@lexical/mark
@lexical/markdown
@lexical/mdast
@lexical/offset
@lexical/overflow
@lexical/plain-text
@lexical/react
@lexical/rich-text
@lexical/selection
@lexical/table
@lexical/tailwind
@lexical/text
@lexical/utils
@lexical/y
@lexical/yjs
commit: |
etrepum
reviewed
Oct 29, 2026
| * that `to` has for the keys `from` does not set (such as state that a node | ||
| * replacement initializes). | ||
| */ | ||
| function $copyNodeState(from: LexicalNode, to: LexicalNode): void { |
Collaborator
There was a problem hiding this comment.
I would probably make this an @internal function in lexical since it's going to be used in every highlighter (with possible other use cases), and it could go through NodeState.getInternalState to read unknown/known directly instead of passing through JSON serialization.
vjeux
approved these changes
Oct 29, 2026
…e][lexical-y] Bug Fix: Keep comment ids and NodeState when a block change moves or re-creates text ## Description `MarkStateExtension` gives inserted text its comment ids with a root transform that compares the blocks around the caret with the previous editor state. That comparison only covered the sibling blocks of the caret's block and treated a nested block (the items of a list) as one opaque unit. Text that a block change moved in from outside it had no counterpart, so it was taken for text typed into an existing node: it lost its own ids and got the ids shared by its neighbours. The core keeps the NodeState in these conversions; with the transform disabled, the ids survive. - Turning a lone list item into a paragraph, quote or heading (`$setBlocksType`) removed every comment id from its text, with or without `@lexical/y`. - Turning a paragraph into a list item that joins the list above it (`$insertList`) removed the comment from a link when the item above ended with the same text, or gave the moved text the comment of the item above (#9490). - Converting a list item to a highlighted code block, which re-creates the text in new nodes, lost its ids too. - Turning a highlighted code block into a paragraph lost them whenever the inserted-text rule could not run (with the code selected, or for pasted code): Prism and Shiki replace each code highlight node that leaves a code block with `$createTextNode(text)`, which drops its NodeState. The comparison now: - keeps the ids of a leaf that existed before the update and kept its text, wherever the update moved it, and of the text a moved TextNode kept at each end when the same update also typed into it (each run of inserted units between kept ones is an insertion, with the ids shared by its neighbours); - compares a nested block that the update changed by its text, like the blocks around it (an unchanged one stays a single unit, so an edit next to a large nested list does not read it); - also compares the following block when it lost its first children, as a list does when its first item is converted. `@lexical/code-prism` and `@lexical/code-shiki` now copy the NodeState of a code highlight node (flat states included) into the TextNode that replaces it, keeping any state that a TextNode replacement sets, so any NodeState survives leaving a code block, not only comment ids. A node that code moves (rather than creates) into the middle of a marked run now keeps exactly its own ids instead of taking the run's; `comments.md` says so, and that block changes keep every id. Independent of #9489 and #9496. It merges with #9489 without conflicts of its own (that PR's `comments.md` conflict with main is unrelated). With #9496 it has two one-line conflicts in `MarkStateExtension.ts`, at the two `$collectSegments` calls: #9496 adds a `flavor` first argument and this adds a `$isUnchangedBlock` last one, so both become `$collectSegments(flavor, windows, points, $isUnchangedBlock)`. Closes #9490 ## Test plan `MarkStateBlockConversion.test.ts` states the invariant (converting blocks keeps the ids of every character) over every source and target (paragraph, heading, quote, code with and without highlighting, bullet, number and check items, nested items as sources, no list as a target), with links, comments on part, inside a link, overlapping or on all of the text, neighbouring lists that the conversion merges with (including text that lines up and shared ids), the caret or a range at several places, text typed in the same update, and pasted blocks: 27,072 cases. Each also checks that the conversion happened. It includes both reports and a comment on code text turned into a paragraph; the lexical-y test round-trips the list item case through a Yjs document, and the code-prism and code-shiki tests check that NodeState (nested and flat) survives leaving a code block. ### Before New tests against `main`'s sources: ``` $ pnpm exec vitest run --project unit packages/lexical-mark/src/__tests__/unit/MarkStateBlockConversion.test.ts packages/lexical-y/src/__tests__/unit/MarkStateBlockType.test.ts packages/lexical-code-prism/src/__tests__/unit/CodePrismLeaveCodeNodeState.test.ts packages/lexical-code-shiki/src/__tests__/unit/CodeShikiLeaveCodeNodeState.test.ts FAIL ... > reported cases > a paragraph turned into a list item keeps the comment on a link - "a:c0", - "b:c0", - "c:c0", - "d:c0", + "a:", + "b:", + "c:", + "d:", FAIL ... > MarkStateExtension with @lexical/y > a list item with a comment turned into a paragraph keeps the comment Expected: "paragraph("alpha "[] "beta"["c1"])" Received: "paragraph("alpha beta"[])" FAIL ... CodePrismLeaveCodeNodeState.test.ts > code highlight nodes turned back into text when they leave a code block keep their NodeState AssertionError: expected [ [ true, 'note', '' ] ] to deeply equal [ [ true, 'note', 'kept' ] ] ... Test Files 4 failed (4) Tests 89 failed | 107 passed (196) ``` 3,804 of the 27,072 matrix cases fail, for example: ``` "bullet -> paragraph, plain, marks part, alone, selection 17: before a: l: p: h: a: : b:c1 e:c1 t:c1 a:c1 : g: a: m: m: a: : after a: l: p: h: a: : b: e: t: a: : g: a: m: m: a: :" ``` ### After ``` $ pnpm exec vitest run --project unit packages/lexical-mark/src/__tests__/unit/MarkStateBlockConversion.test.ts packages/lexical-y/src/__tests__/unit/MarkStateBlockType.test.ts packages/lexical-code-prism/src/__tests__/unit/CodePrismLeaveCodeNodeState.test.ts packages/lexical-code-shiki/src/__tests__/unit/CodeShikiLeaveCodeNodeState.test.ts Test Files 4 passed (4) Tests 196 passed (196) $ pnpm exec vitest run --project unit packages/lexical-y packages/lexical-mark/ packages/lexical-list packages/lexical-selection packages/lexical-code packages/lexical-table packages/lexical-rich-text packages/lexical-markdown Test Files 250 passed | 2 skipped (252) Tests 7251 passed | 16 expected fail | 3 skipped (7270) $ MARK_STATE_PROPERTY_SEEDS=400 MARK_STATE_PROPERTY_STEPS=120 pnpm exec vitest run --project unit packages/lexical-mark/src/__tests__/unit/MarkStateProperty.test.ts Tests 1 passed (1) $ pnpm run ci-check Found 0 errors ``` Unit tests only (Node/jsdom); no browser or E2E run. Not covered (the matrix skips them, with the reason): comment ids on part of a code highlighting token, and marked text that a range conversion merges into a highlighted code block, which re-tokenizing drops (#9491); and typing inside a link in the same update that turns it into highlighted code, where the highlighter puts the caret back at a wrong offset, so the typed text is not before the caret as the inserted-text rule needs.
zurfyx
force-pushed
the
zurfyx/keep-node-state-on-block-convert
branch
from
October 29, 2026 17:18
80cf3c9 to
576b48d
Compare
zurfyx
added a commit
that referenced
this pull request
Oct 29, 2026
…maVersion ## Description Rebasing this PR onto `main` brings in two tests that landed after it was written and bind a plain `Y.Doc` without a `schemaVersion`, which this PR's `YCollabExtension` and `@lexical/y/headless` config no longer accept: - `TableOnOpen.test.ts` (#9501): the document is made with `createYDoc` from the seeded state, and the seed and the extension get `schemaVersion: 1`. - `MarkStateBlockType.test.ts` (#9503): both documents are made with `createYDoc`, and the extension gets `schemaVersion: 1`. ## Test plan ### Before ``` $ npx tsc --noEmit -p tsconfig.json packages/lexical-y/src/__tests__/unit/MarkStateBlockType.test.ts(54,44): error TS2741: Property '[createdByCreateYDoc]' is missing in type 'Doc' but required in type 'YDoc'. packages/lexical-y/src/__tests__/unit/TableOnOpen.test.ts(58,9): error TS2345: Argument of type '{ extensions: ... }' is not assignable to parameter of type 'HeadlessYConfig'. packages/lexical-y/src/__tests__/unit/TableOnOpen.test.ts(62,55): error TS2741: Property '[createdByCreateYDoc]' is missing in type 'Doc' but required in type 'YDoc'. ``` ### After ``` $ pnpm run ci-check Found 0 errors $ npx vitest run --project unit Test Files 563 passed | 3 skipped (566) Tests 14641 passed | 46 expected fail | 6 skipped (14693) ```
etrepum
pushed a commit
to etrepum/lexical
that referenced
this pull request
Oct 29, 2026
…ed code block's selection as a caret again, so comment ids are kept ## Description The previous fix kept both ends of a range selection across a highlighter retokenize. Upstream's meta#9503, merged in through the PR head, has `MarkStateExtension` carry comment ids onto the new tokens only while the selection is collapsed, so with the range kept, converting selected commented text to a code block dropped its comments. The highlighters restore the selection as a caret at the anchor again, as before, still by text offset, and the range tests are dropped. ## Test plan ### Before ``` $ ./node_modules/.bin/vitest run --project unit packages/lexical-mark/src/__tests__/unit/MarkStateBlockConversion.test.ts Tests 19 failed | 170 passed (189) ``` ### After ``` $ ./node_modules/.bin/vitest run --project unit packages/lexical-mark packages/lexical-code-shiki packages/lexical-code-prism packages/lexical-code packages/lexical-code-core packages/lexical-playground/__tests__/unit Tests 1821 passed | 1 skipped (1822) ``` Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zurfyx
added a commit
that referenced
this pull request
Nov 2, 2026
…become text ## Description `registerLexicalTextEntity` turns an entity back into plain text when it stops matching (typing right before a hashtag, deleting its `#`, joining blocks), with `$replaceWithSimpleText`. That built the new `TextNode` from the entity's text, format, style and detail, but not its NodeState, so anything an app keeps on the characters as NodeState (comment ids, suggestion marks, block ids) was dropped (#9703). The same happened in the other direction: plain text with NodeState that becomes an entity gave the entity none. What carries, and why: - **Demotion (entity to text): all of the entity's NodeState**, nested and flat, known and not yet parsed (read from JSON), the same value objects (ids compared by identity stay identical). NodeState is the ad-hoc data an app keeps on text; what is inherent to the entity type is its class (its fields) and its mode, which stay with the type, as before. This is what `splitText` already does for the pieces it cuts, and what the code highlighters do since #9503 when a code token becomes text. A key that the plain text type declares with a different `StateConfig` is parsed with the text type's config. Format, style and detail carry as before. - **Promotion (text to entity): the text's NodeState too**, for every key `createNode` did not set (its state wins), with one exception: text that the entity's own transform splits off its end in the same update (text typed or pasted right after a hashtag). `splitText` gives that piece a copy of the entity's NodeState, but those characters were just inserted, so it is not theirs; an entity made from it keeps the old behaviour (MarkStateExtension relies on telling inserted text from marked text this way). An entity turned into text and back in one update gets its own NodeState back. - **Token and segmented entities** that text follows are turned into text so that they merge with it and are parsed again. With their NodeState kept they do not merge with text that has other NodeState, and the transforms would make the entity again and turn it back forever (on `main` this already looped whenever that text had NodeState, or another format). So, as in #9686, when the same entity (same parent, node before it and text) comes back to be turned into text a third time in one update (or updates batched into one commit), it is kept, with its NodeState. It takes the text its match extends into, as an entity in normal mode does, only when that text would merge with it (the same format, style, detail and NodeState; the selection stays where it was in that text); text with other NodeState or format stays apart, so it neither takes the entity's NodeState nor makes the entity drop its own. Until the third time the transforms run as before. The count belongs to the pending editor state, so an update that fails does not count toward the next. - The text split off the end of an entity is tracked per editor, for every registration, so an entity of another registration made from it (a keyword typed after a hashtag) does not take the copy either. Two text nodes merge only when their NodeState is equal, so text that keeps an entity's NodeState stays its own node next to text with other NodeState, as text in another format does. That is the point (the characters keep their comment), and it is the only structural difference: text with no NodeState merges as before. Callers in this repo, all through `registerLexicalTextEntity`: `@lexical/hashtag` (`registerLexicalHashtag`, `HashtagExtension`, `LexicalHashtagPlugin`), `@lexical/react` `useLexicalTextEntity`, and the playground's `KeywordsExtension`. Mentions (typeahead) and `@lexical/link` autolink have their own transforms and do not share this path. Fixes #9703 ## Test plan `.changeset/lexical-text-entity-node-state.md` (`@lexical/text`, patch, `[fix]`). `registerLexicalTextEntityNodeState.test.ts` (new, 22 tests): #9703's repro; deleting a hashtag's `#`; merging into neighbours with the same NodeState and, without NodeState, merging as before; promotion with NodeState, `createNode`'s state winning, and text inserted after a hashtag not taking the copy; a reference-typed value carried as the same object; NodeState read from JSON and not parsed yet; NodeState declared in the entity's `$config` (nested and flat) and exported nested on the text; a key the text type declares with another config; a hashtag inside a MarkNode; undo and redo; and every path that turns an entity into text: the text transform with no match, a token entity that text follows (same NodeState: it takes the text; other NodeState: kept, the text stays apart; the selection after pasted text; text with its own NodeState, which loops on `main`), the next entity that stops matching, adjacent entities, an entity followed by another, and a join across text in another format. `registerLexicalTextEntityNodeStateProperty.test.ts` (new): random layouts of entities (hashtags, and token mentions) and text in several formats, with NodeState on both and MarkNodes around some, and random edits (typing, backspace and delete, joining blocks, removing ranges across nodes). Each entity has a letter no other text has; after every edit that turns an entity into text, every character of it that is still there has the entity's NodeState and MarkNode (or sits inside another entity that grew over it, which takes characters with its own NodeState, as before). 200 seeds per matcher (`TEXT_ENTITY_NODE_STATE_SEEDS`). Before (main): ``` $ pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeState.test.ts packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts × #9703: a hashtag that stops matching keeps its NodeState when it becomes text (typing before it) × deleting the # of a hashtag keeps its NodeState (reverse transform, no match) × a hashtag that becomes text merges into neighbors with the same NodeState × plain text with NodeState that becomes a hashtag keeps it × state that createNode gives the entity wins over the text it replaces × a reference-typed state value is carried as the same object × NodeState read from JSON (not parsed yet) is carried and exported × NodeState declared in the entity $config (nested and flat) is carried to the text × a key the plain text type declares with another config is parsed with that config × a hashtag inside a MarkNode stays inside it when it becomes text × undo and redo of a demotion restore the entity and the text with their NodeState × the text transform: the entity before it stops matching with its text (prevMatch null) × with the same NodeState as the text: it keeps it and takes the text × with other NodeState than the text: kept with its NodeState, the text stays apart × typed text after it: the selection stays after the typed text × with other NodeState than text that its match extends into, which keeps it apart: kept, each keeps its own × the text transform: the entity after it stops matching (nextMatch null) × the reverse transform: adjacent entities both become text × the reverse transform: an entity followed by an entity × hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters × mention: an edit that turns an entity into text keeps its NodeState and marks on its characters Tests 21 failed | 3 passed (24) ``` After: ``` $ pnpm exec vitest run --project unit packages/lexical-text packages/lexical-hashtag packages/lexical-mark packages/lexical-history Tests 1375 passed (1375) $ TEXT_ENTITY_NODE_STATE_SEEDS=20000 pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts ✓ hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters ✓ mention: an edit that turns an entity into text keeps its NodeState and marks on its characters ``` The existing `@lexical/text` tests pass unchanged, including `registerLexicalTextEntityEquivalence.test.ts` (JSON, selection, mutations and keys after random sessions, against the helper before #9483, which has no NodeState in its layouts), and `MarkStateExtension`'s hashtag tests (text inserted after a marked hashtag is not marked). A differential against `main` over random editing sessions (typing, backspace and delete, joins across paragraphs, headings, quotes, list items and table cells, ranges, formatting, paste with NodeState, undo and redo, IME composition) with hashtags, a keyword matcher, a token and a segmented matcher, autolink and MarkNodes: the NodeState of an entity's characters is kept in tens of thousands of steps where `main` drops it; no text is lost; token and segmented entities that loop on `main` settle. Where NodeState now keeps text apart, the entities can differ from `main`, as they do next to text in another format; a keyword-style matcher can then reach #9637's loop in a layout `main` merged (a few per 10,000 sessions; hashtags never do), which #9637's fix settles.
zurfyx
added a commit
that referenced
this pull request
Nov 2, 2026
…hen they become text ## Description `registerLexicalTextEntity` turns an entity back into plain text when it stops matching (typing right before a hashtag, deleting its `#`, joining blocks), with `$replaceWithSimpleText`. That built the new `TextNode` from the entity's text, format, style and detail, but not its NodeState, so anything an app keeps on the characters as NodeState (comment ids, suggestion marks, block ids) was dropped (#9703). The same happened in the other direction: plain text with NodeState that becomes an entity gave the entity none. What carries, and why: - **Demotion (entity to text): all of the entity's NodeState**, nested and flat, known and not yet parsed (read from JSON), the same value objects (ids compared by identity stay identical). NodeState is the ad-hoc data an app keeps on text; what is inherent to the entity type is its class (its fields) and its mode, which stay with the type, as before. This is what `splitText` already does for the pieces it cuts, and what the code highlighters do since #9503 when a code token becomes text. A key that the plain text type declares with a different `StateConfig` is parsed with the text type's config. Format, style and detail carry as before. - **Promotion (text to entity): the text's NodeState too**, for every key `createNode` did not set (its state wins), with one exception: text that the entity's own transform splits off its end in the same update (text typed or pasted right after a hashtag). `splitText` gives that piece a copy of the entity's NodeState, but those characters were just inserted, so it is not theirs; an entity made from it keeps the old behaviour (MarkStateExtension relies on telling inserted text from marked text this way). An entity turned into text and back in one update gets its own NodeState back. - **Token and segmented entities** that text follows are turned into text so that they merge with it and are parsed again. With their NodeState kept they do not merge with text that has other NodeState, and the transforms would make the entity again and turn it back forever (on `main` this already looped whenever that text had NodeState, or another format). So, as in #9686, when the same entity (same parent, text, and place: the first node before it that is no text entity, and the size of the entities between) comes back to be turned into text a third time in one update (or updates batched into one commit), it is kept, with its NodeState. It takes the text its match extends into, as an entity in normal mode does, only when that text would merge with it (the same format, style and detail, and NodeState equal by the rule text nodes merge by: `StateConfig.isEqual`, so two distinct value objects are two values; the selection stays where it was in that text); text with other NodeState or format stays apart, so it neither takes the entity's NodeState nor makes the entity drop its own. Until the third time the transforms run as before. The count belongs to the pending editor state, so an update that fails does not count toward the next. - The text split off the end of an entity is tracked per editor, for every registration, so an entity of another registration made from it (a keyword typed after a hashtag) does not take the copy either. - **Core: typing inside a segmented node** (outside composition) replaces it with a plain `TextNode` built from its text, format and style, the same way `$replaceWithSimpleText` did, and dropped its NodeState the same way. That node now keeps it (`$carryNodeState`, internal to `lexical`, with the same rules as above). Segmented entities that this change keeps or makes reach it more often, so it is fixed here. Two text nodes merge only when their NodeState is equal, so text that keeps an entity's NodeState stays its own node next to text with other NodeState, as text in another format does. That is the point (the characters keep their comment), and it is the only structural difference: text with no NodeState merges as before. Callers in this repo, all through `registerLexicalTextEntity`: `@lexical/hashtag` (`registerLexicalHashtag`, `HashtagExtension`, `LexicalHashtagPlugin`), `@lexical/react` `useLexicalTextEntity`, and the playground's `KeywordsExtension`. Mentions (typeahead) and `@lexical/link` autolink have their own transforms and do not share this path. Fixes #9703 ## Test plan `.changeset/lexical-text-entity-node-state.md` (`lexical` and `@lexical/text`, patch, `[fix]`). `LexicalSelection.test.ts`: typing inside a segmented node keeps its NodeState (fails on `main`). `registerLexicalTextEntityNodeState.test.ts` (new, 23 tests): #9703's repro; deleting a hashtag's `#`; merging into neighbours with the same NodeState and, without NodeState, merging as before; promotion with NodeState, `createNode`'s state winning, and text inserted after a hashtag not taking the copy; a reference-typed value carried as the same object; NodeState read from JSON and not parsed yet; NodeState declared in the entity's `$config` (nested and flat) and exported nested on the text; a key the text type declares with another config; a hashtag inside a MarkNode; undo and redo; and every path that turns an entity into text: the text transform with no match, a token entity that text follows (same NodeState: it takes the text; other NodeState: kept, the text stays apart; the selection after pasted text; text with its own NodeState, which loops on `main`; a value object distinct from the text's, equal as JSON), the next entity that stops matching, adjacent entities, an entity followed by another, and a join across text in another format. `registerLexicalTextEntityNodeStateProperty.test.ts` (new): random layouts of entities (hashtags, and token mentions) and text in several formats, with NodeState on both and MarkNodes around some, and random edits (typing, backspace and delete, joining blocks, removing ranges across nodes). Each entity has a letter no other text has; after every edit that turns an entity into text, every character of it that is still there has the entity's NodeState and MarkNode (or sits inside another entity that grew over it, which takes characters with its own NodeState, as before). 200 seeds per matcher (`TEXT_ENTITY_NODE_STATE_SEEDS`). Before (main): ``` $ pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeState.test.ts packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts × #9703: a hashtag that stops matching keeps its NodeState when it becomes text (typing before it) × deleting the # of a hashtag keeps its NodeState (reverse transform, no match) × a hashtag that becomes text merges into neighbors with the same NodeState × plain text with NodeState that becomes a hashtag keeps it × state that createNode gives the entity wins over the text it replaces × a reference-typed state value is carried as the same object × NodeState read from JSON (not parsed yet) is carried and exported × NodeState declared in the entity $config (nested and flat) is carried to the text × a key the plain text type declares with another config is parsed with that config × a hashtag inside a MarkNode stays inside it when it becomes text × undo and redo of a demotion restore the entity and the text with their NodeState × the text transform: the entity before it stops matching with its text (prevMatch null) × with the same NodeState as the text: it keeps it and takes the text × with other NodeState than the text: kept with its NodeState, the text stays apart × typed text after it: the selection stays after the typed text × with other NodeState than text that its match extends into, which keeps it apart: kept, each keeps its own × with a value object distinct from the text after it, equal as JSON: each keeps its own × the text transform: the entity after it stops matching (nextMatch null) × the reverse transform: adjacent entities both become text × the reverse transform: an entity followed by an entity × hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters × mention: an edit that turns an entity into text keeps its NodeState and marks on its characters Tests 22 failed | 3 passed (25) ``` After: ``` $ pnpm exec vitest run --project unit packages/lexical-text packages/lexical-hashtag packages/lexical-mark packages/lexical-history Tests 1376 passed (1376) $ pnpm exec vitest run --project unit packages/lexical/src/__tests__/unit/LexicalSelection.test.ts Tests 101 passed (101) $ TEXT_ENTITY_NODE_STATE_SEEDS=20000 pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts ✓ hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters ✓ mention: an edit that turns an entity into text keeps its NodeState and marks on its characters ``` The existing `@lexical/text` tests pass unchanged, including `registerLexicalTextEntityEquivalence.test.ts` (JSON, selection, mutations and keys after random sessions, against the helper before #9483, which has no NodeState in its layouts), and `MarkStateExtension`'s hashtag tests (text inserted after a marked hashtag is not marked). A differential against `main` over random editing sessions (typing, backspace and delete, joins across paragraphs, headings, quotes, list items and table cells, ranges, formatting, paste with NodeState, undo and redo, IME composition) with hashtags, a keyword matcher, a token and a segmented matcher, autolink and MarkNodes: the NodeState of an entity's characters is kept in tens of thousands of steps where `main` drops it; no text is lost; token and segmented entities that loop on `main` settle. Where NodeState now keeps text apart, the entities can differ from `main`, as they do next to text in another format; a keyword-style matcher can then reach #9637's loop in a layout `main` merged (a few per 10,000 sessions; hashtags never do), which #9637's fix settles.
zurfyx
added a commit
that referenced
this pull request
Nov 2, 2026
…hen they become text ## Description `registerLexicalTextEntity` turns an entity back into plain text when it stops matching (typing right before a hashtag, deleting its `#`, joining blocks), with `$replaceWithSimpleText`. That built the new `TextNode` from the entity's text, format, style and detail, but not its NodeState, so anything an app keeps on the characters as NodeState (comment ids, suggestion marks, block ids) was dropped (#9703). The same happened in the other direction: plain text with NodeState that becomes an entity gave the entity none. What carries, and why: - **Demotion (entity to text): all of the entity's NodeState**, nested and flat, known and not yet parsed (read from JSON), the same value objects (ids compared by identity stay identical). NodeState is the ad-hoc data an app keeps on text; what is inherent to the entity type is its class (its fields) and its mode, which stay with the type, as before. This is what `splitText` already does for the pieces it cuts, and what the code highlighters do since #9503 when a code token becomes text. A key that the plain text type declares with a different `StateConfig` is parsed with the text type's config. Format, style and detail carry as before. - **Promotion (text to entity): the text's NodeState too**, for every key `createNode` did not set (its state wins), with one exception: text that the entity's own transform splits off its end in the same update (text typed or pasted right after a hashtag). `splitText` gives that piece a copy of the entity's NodeState, but those characters were just inserted, so it is not theirs; an entity made from it keeps the old behaviour (MarkStateExtension relies on telling inserted text from marked text this way). An entity turned into text and back in one update gets its own NodeState back. - **Token and segmented entities** that text follows are turned into text so that they merge with it and are parsed again. With their NodeState kept they do not merge with text that has other NodeState, and the transforms would make the entity again and turn it back forever (on `main` this already looped whenever that text had NodeState, or another format). So, as in #9686, when the same entity (same parent, text, and place: the first node before it that is no text entity, and the size of the entities between) comes back to be turned into text a third time in one update (or updates batched into one commit), it is kept, with its NodeState. It takes the text its match extends into, as an entity in normal mode does, only when that text would merge with it (the same format, style and detail, and NodeState equal by the rule text nodes merge by: `StateConfig.isEqual`, so two distinct value objects are two values; the selection stays where it was in that text); text with other NodeState or format stays apart, so it neither takes the entity's NodeState nor makes the entity drop its own. Until the third time the transforms run as before. The count belongs to the pending editor state, so an update that fails does not count toward the next. - The text split off the end of an entity is tracked per editor, for every registration, so an entity of another registration made from it (a keyword typed after a hashtag) does not take the copy either. - **Core: editing inside a segmented node** replaces it with a plain `TextNode` built from its text, format and style, the same way `$replaceWithSimpleText` did, and dropped its NodeState the same way: typing inside it (outside composition, `LexicalSelection`) and removing a range that starts or ends inside it (`LexicalCaretUtils`). Both now keep it (`$carryNodeState`, internal to `lexical`, with the same rules as above; keys the new node already holds, such as state a registered `TextNode` replacement sets, keep their value). Segmented entities that this change keeps or makes reach these paths more often, so they are fixed here. Two text nodes merge only when their NodeState is equal, so text that keeps an entity's NodeState stays its own node next to text with other NodeState, as text in another format does. That is the point (the characters keep their comment), and it is the only structural difference: text with no NodeState merges as before. Callers in this repo, all through `registerLexicalTextEntity`: `@lexical/hashtag` (`registerLexicalHashtag`, `HashtagExtension`, `LexicalHashtagPlugin`), `@lexical/react` `useLexicalTextEntity`, and the playground's `KeywordsExtension`. Mentions (typeahead) and `@lexical/link` autolink have their own transforms and do not share this path. Fixes #9703 ## Test plan `.changeset/lexical-text-entity-node-state.md` (`lexical` and `@lexical/text`, patch, `[fix]`). `LexicalSelection.test.ts` (3 tests, each fails on `main`): typing inside a segmented node keeps its NodeState; a registered `TextNode` replacement keeps the NodeState it sets and takes the rest; removing a range from inside a segmented node to the next block keeps the NodeState of what is left. `registerLexicalTextEntityNodeState.test.ts` (new, 23 tests): #9703's repro; deleting a hashtag's `#`; merging into neighbours with the same NodeState and, without NodeState, merging as before; promotion with NodeState, `createNode`'s state winning, and text inserted after a hashtag not taking the copy; a reference-typed value carried as the same object; NodeState read from JSON and not parsed yet; NodeState declared in the entity's `$config` (nested and flat) and exported nested on the text; a key the text type declares with another config; a hashtag inside a MarkNode; undo and redo; and every path that turns an entity into text: the text transform with no match, a token entity that text follows (same NodeState: it takes the text; other NodeState: kept, the text stays apart; the selection after pasted text; text with its own NodeState, which loops on `main`; a value object distinct from the text's, equal as JSON), the next entity that stops matching, adjacent entities, an entity followed by another, and a join across text in another format. `registerLexicalTextEntityNodeStateProperty.test.ts` (new): random layouts of entities (hashtags, and token mentions) and text in several formats, with NodeState on both and MarkNodes around some, and random edits (typing, backspace and delete, joining blocks, removing ranges across nodes). Each entity has a letter no other text has; after every edit that turns an entity into text, every character of it that is still there has the entity's NodeState and MarkNode (or sits inside another entity that grew over it, which takes characters with its own NodeState, as before). 200 seeds per matcher (`TEXT_ENTITY_NODE_STATE_SEEDS`). Before (main): ``` $ pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeState.test.ts packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts × #9703: a hashtag that stops matching keeps its NodeState when it becomes text (typing before it) × deleting the # of a hashtag keeps its NodeState (reverse transform, no match) × a hashtag that becomes text merges into neighbors with the same NodeState × plain text with NodeState that becomes a hashtag keeps it × state that createNode gives the entity wins over the text it replaces × a reference-typed state value is carried as the same object × NodeState read from JSON (not parsed yet) is carried and exported × NodeState declared in the entity $config (nested and flat) is carried to the text × a key the plain text type declares with another config is parsed with that config × a hashtag inside a MarkNode stays inside it when it becomes text × undo and redo of a demotion restore the entity and the text with their NodeState × the text transform: the entity before it stops matching with its text (prevMatch null) × with the same NodeState as the text: it keeps it and takes the text × with other NodeState than the text: kept with its NodeState, the text stays apart × typed text after it: the selection stays after the typed text × with other NodeState than text that its match extends into, which keeps it apart: kept, each keeps its own × with a value object distinct from the text after it, equal as JSON: each keeps its own × the text transform: the entity after it stops matching (nextMatch null) × the reverse transform: adjacent entities both become text × the reverse transform: an entity followed by an entity × hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters × mention: an edit that turns an entity into text keeps its NodeState and marks on its characters Tests 22 failed | 3 passed (25) ``` After: ``` $ pnpm exec vitest run --project unit packages/lexical-text packages/lexical-hashtag packages/lexical-mark packages/lexical-history Tests 1376 passed (1376) $ pnpm exec vitest run --project unit packages/lexical/src/__tests__/unit/LexicalSelection.test.ts Tests 103 passed (103) $ TEXT_ENTITY_NODE_STATE_SEEDS=20000 pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts ✓ hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters ✓ mention: an edit that turns an entity into text keeps its NodeState and marks on its characters ``` The existing `@lexical/text` tests pass unchanged, including `registerLexicalTextEntityEquivalence.test.ts` (JSON, selection, mutations and keys after random sessions, against the helper before #9483, which has no NodeState in its layouts), and `MarkStateExtension`'s hashtag tests (text inserted after a marked hashtag is not marked). A differential against `main` over random editing sessions (typing, backspace and delete, joins across paragraphs, headings, quotes, list items and table cells, ranges, formatting, paste with NodeState, undo and redo, IME composition) with hashtags, a keyword matcher, a token and a segmented matcher, autolink and MarkNodes: the NodeState of an entity's characters is kept in tens of thousands of steps where `main` drops it; no text is lost; token and segmented entities that loop on `main` settle. Where NodeState now keeps text apart, the entities can differ from `main`, as they do next to text in another format; a keyword-style matcher can then reach #9637's loop in a layout `main` merged (a few per 10,000 sessions; hashtags never do), which #9637's fix settles.
zurfyx
added a commit
that referenced
this pull request
Nov 2, 2026
…hen they become text ## Description `registerLexicalTextEntity` turns an entity back into plain text when it stops matching (typing right before a hashtag, deleting its `#`, joining blocks), with `$replaceWithSimpleText`. That built the new `TextNode` from the entity's text, format, style and detail, but not its NodeState, so anything an app keeps on the characters as NodeState (comment ids, suggestion marks, block ids) was dropped (#9703). The same happened in the other direction: plain text with NodeState that becomes an entity gave the entity none. What carries, and why: - **Demotion (entity to text): all of the entity's NodeState**, nested and flat, known and not yet parsed (read from JSON), the same value objects (ids compared by identity stay identical). NodeState is the ad-hoc data an app keeps on text; what is inherent to the entity type is its class (its fields) and its mode, which stay with the type, as before. This is what `splitText` already does for the pieces it cuts, and what the code highlighters do since #9503 when a code token becomes text. A key that the plain text type declares with a different `StateConfig` is parsed with the text type's config. Format, style and detail carry as before. - **Promotion (text to entity): the text's NodeState too**, for every key `createNode` did not set (its state wins), with one exception: text that the entity's own transform splits off its end in the same update (text typed or pasted right after a hashtag). `splitText` gives that piece a copy of the entity's NodeState, but those characters were just inserted, so it is not theirs; an entity made from it keeps the old behaviour (MarkStateExtension relies on telling inserted text from marked text this way). An entity turned into text and back in one update gets its own NodeState back. - **Token and segmented entities** that text follows are turned into text so that they merge with it and are parsed again. With their NodeState kept they do not merge with text that has other NodeState, and the transforms would make the entity again and turn it back forever (on `main` this already looped whenever that text had NodeState, or another format). So, as in #9686, when the same entity (same parent, text, and place: the first node before it that is no text entity, and the size of the entities between) comes back to be turned into text a third time in one update (or updates batched into one commit), it is kept, with its NodeState. It takes the text its match extends into, as an entity in normal mode does, only when that text would merge with it (the same format, style and detail, and NodeState equal by the rule text nodes merge by: `StateConfig.isEqual`, so two distinct value objects are two values; the selection stays where it was in that text); text with other NodeState or format stays apart, so it neither takes the entity's NodeState nor makes the entity drop its own. Until the third time the transforms run as before. The count belongs to the pending editor state, so an update that fails does not count toward the next. - The text split off the end of an entity is tracked per editor, for every registration, so an entity of another registration made from it (a keyword typed after a hashtag) does not take the copy either. - **Core: editing inside a segmented node** replaces it with a plain `TextNode` built from its text, format and style, the same way `$replaceWithSimpleText` did, and dropped its NodeState the same way (and its detail, such as `unmergeable`, which `$replaceWithSimpleText` kept): typing inside it (outside composition, `LexicalSelection`) and removing a range that starts or ends inside it (`LexicalCaretUtils`). Both now keep them (detail, and NodeState with `$carryNodeState`, internal to `lexical`, with the same rules as above; keys the new node already holds, such as state a registered `TextNode` replacement sets, keep their value). Segmented entities that this change keeps or makes reach these paths more often, so they are fixed here. Two text nodes merge only when their NodeState is equal, so text that keeps an entity's NodeState stays its own node next to text with other NodeState, as text in another format does. That is the point (the characters keep their comment), and it is the only structural difference: text with no NodeState merges as before. Callers in this repo, all through `registerLexicalTextEntity`: `@lexical/hashtag` (`registerLexicalHashtag`, `HashtagExtension`, `LexicalHashtagPlugin`), `@lexical/react` `useLexicalTextEntity`, and the playground's `KeywordsExtension`. Mentions (typeahead) and `@lexical/link` autolink have their own transforms and do not share this path. Fixes #9703 ## Test plan `.changeset/lexical-text-entity-node-state.md` (`lexical` and `@lexical/text`, patch, `[fix]`). `LexicalSelection.test.ts` (3 tests, each fails on `main`): typing inside a segmented node keeps its NodeState and detail; a registered `TextNode` replacement keeps the NodeState it sets and takes the rest; removing a range from inside a segmented node to the next block keeps the NodeState and detail of what is left. `registerLexicalTextEntityNodeState.test.ts` (new, 23 tests): #9703's repro; deleting a hashtag's `#`; merging into neighbours with the same NodeState and, without NodeState, merging as before; promotion with NodeState, `createNode`'s state winning, and text inserted after a hashtag not taking the copy; a reference-typed value carried as the same object; NodeState read from JSON and not parsed yet; NodeState declared in the entity's `$config` (nested and flat) and exported nested on the text; a key the text type declares with another config; a hashtag inside a MarkNode; undo and redo; and every path that turns an entity into text: the text transform with no match, a token entity that text follows (same NodeState: it takes the text; other NodeState: kept, the text stays apart; the selection after pasted text; text with its own NodeState, which loops on `main`; a value object distinct from the text's, equal as JSON), the next entity that stops matching, adjacent entities, an entity followed by another, and a join across text in another format. `registerLexicalTextEntityNodeStateProperty.test.ts` (new): random layouts of entities (hashtags, and token mentions) and text in several formats, with NodeState on both and MarkNodes around some, and random edits (typing, backspace and delete, joining blocks, removing ranges across nodes). Each entity has a letter no other text has; after every edit that turns an entity into text, every character of it that is still there has the entity's NodeState and MarkNode (or sits inside another entity that grew over it, which takes characters with its own NodeState, as before). 200 seeds per matcher (`TEXT_ENTITY_NODE_STATE_SEEDS`). Before (main): ``` $ pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeState.test.ts packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts × #9703: a hashtag that stops matching keeps its NodeState when it becomes text (typing before it) × deleting the # of a hashtag keeps its NodeState (reverse transform, no match) × a hashtag that becomes text merges into neighbors with the same NodeState × plain text with NodeState that becomes a hashtag keeps it × state that createNode gives the entity wins over the text it replaces × a reference-typed state value is carried as the same object × NodeState read from JSON (not parsed yet) is carried and exported × NodeState declared in the entity $config (nested and flat) is carried to the text × a key the plain text type declares with another config is parsed with that config × a hashtag inside a MarkNode stays inside it when it becomes text × undo and redo of a demotion restore the entity and the text with their NodeState × the text transform: the entity before it stops matching with its text (prevMatch null) × with the same NodeState as the text: it keeps it and takes the text × with other NodeState than the text: kept with its NodeState, the text stays apart × typed text after it: the selection stays after the typed text × with other NodeState than text that its match extends into, which keeps it apart: kept, each keeps its own × with a value object distinct from the text after it, equal as JSON: each keeps its own × the text transform: the entity after it stops matching (nextMatch null) × the reverse transform: adjacent entities both become text × the reverse transform: an entity followed by an entity × hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters × mention: an edit that turns an entity into text keeps its NodeState and marks on its characters Tests 22 failed | 3 passed (25) ``` After: ``` $ pnpm exec vitest run --project unit packages/lexical-text packages/lexical-hashtag packages/lexical-mark packages/lexical-history Tests 1376 passed (1376) $ pnpm exec vitest run --project unit packages/lexical/src/__tests__/unit/LexicalSelection.test.ts Tests 103 passed (103) $ TEXT_ENTITY_NODE_STATE_SEEDS=20000 pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts ✓ hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters ✓ mention: an edit that turns an entity into text keeps its NodeState and marks on its characters ``` The existing `@lexical/text` tests pass unchanged, including `registerLexicalTextEntityEquivalence.test.ts` (JSON, selection, mutations and keys after random sessions, against the helper before #9483, which has no NodeState in its layouts), and `MarkStateExtension`'s hashtag tests (text inserted after a marked hashtag is not marked). A differential against `main` over random editing sessions (typing, backspace and delete, joins across paragraphs, headings, quotes, list items and table cells, ranges, formatting, paste with NodeState, undo and redo, IME composition) with hashtags, a keyword matcher, a token and a segmented matcher, autolink and MarkNodes: the NodeState of an entity's characters is kept in tens of thousands of steps where `main` drops it; no text is lost; token and segmented entities that loop on `main` settle. Where NodeState now keeps text apart, the entities can differ from `main`, as they do next to text in another format; a keyword-style matcher can then reach #9637's loop in a layout `main` merged (a few per 10,000 sessions; hashtags never do), which #9637's fix settles.
zurfyx
added a commit
that referenced
this pull request
Nov 2, 2026
…ir NodeState when they become text ## Description `registerLexicalTextEntity` turns an entity back into plain text when it stops matching (typing right before a hashtag, deleting its `#`, joining blocks), with `$replaceWithSimpleText`. That built the new `TextNode` from the entity's text, format, style and detail, but not its NodeState, so anything an app keeps on the characters as NodeState (comment ids, suggestion marks, block ids) was dropped (#9703). The same happened in the other direction: plain text with NodeState that becomes an entity gave the entity none. What carries, and why: - **Demotion (entity to text): all of the entity's NodeState**, nested and flat, known and not yet parsed (read from JSON), the same value objects (ids compared by identity stay identical). NodeState is the ad-hoc data an app keeps on text; what is inherent to the entity type is its class (its fields) and its mode, which stay with the type, as before. This is what `splitText` already does for the pieces it cuts, and what the code highlighters do since #9503 when a code token becomes text. A key that the plain text type declares with a different `StateConfig` is parsed with the text type's config. Format, style and detail carry as before. - **Promotion (text to entity): the text's NodeState too**, for every key `createNode` did not set (its state wins), with one exception: text that the entity's own transform splits off its end in the same update (text typed or pasted right after a hashtag). `splitText` gives that piece a copy of the entity's NodeState, but those characters were just inserted, so it is not theirs; an entity made from it keeps the old behaviour (MarkStateExtension relies on telling inserted text from marked text this way). An entity turned into text and back in one update gets its own NodeState back. - **Token and segmented entities** that text follows are turned into text so that they merge with it and are parsed again. With their NodeState kept they do not merge with text that has other NodeState, and the transforms would make the entity again and turn it back forever (on `main` this already looped whenever that text had NodeState, or another format). So, as in #9686, when the same entity (same parent, text, and place: the first node before it that is no text entity, and the size of the entities between) comes back to be turned into text a third time in one update (or updates batched into one commit), it is kept, with its NodeState. It takes the text its match extends into, as an entity in normal mode does, only when that text would merge with it (the same format, style and detail, and NodeState equal by the rule text nodes merge by: `StateConfig.isEqual`, so two distinct value objects are two values; the selection stays where it was in that text); text with other NodeState or format stays apart, so it neither takes the entity's NodeState nor makes the entity drop its own. Until the third time the transforms run as before. The count belongs to the pending editor state, so an update that fails does not count toward the next. - The text split off the end of an entity is tracked per editor, for every registration, so an entity of another registration made from it (a keyword typed after a hashtag) does not take the copy either. - **`MarkStateExtension`**: text inserted at the edge of a comment lands in the commented node, and an entity transform that makes a hashtag of it (pasting ` #tag` right after a comment) splits it off with a copy of that node's comment ids, which the hashtag now keeps. `MarkStateExtension` already treats such a copy as not the inserted text's own when the piece is plain text (`$isSplitOffEntity`); it now does the same for a text entity made in the update whose ids are the same value as the ids of a node that existed (a copy, never ids of its own: pasted or inserted content brings its own value), so the hashtag is not added to the comment, as on `main` and with `MarkNode`. - **Core: editing inside a segmented node** replaces it with a plain `TextNode` built from its text, format and style, the same way `$replaceWithSimpleText` did, and dropped its NodeState the same way (and its detail, such as `unmergeable`, which `$replaceWithSimpleText` kept): typing inside it (outside composition, `LexicalSelection`) and removing a range that starts or ends inside it (`LexicalCaretUtils`). Both now keep them (detail, and NodeState with `$carryNodeState`, internal to `lexical`, with the same rules as above; keys the new node already holds, such as state a registered `TextNode` replacement sets, keep their value). Segmented entities that this change keeps or makes reach these paths more often, so they are fixed here. Two text nodes merge only when their NodeState is equal, so text that keeps an entity's NodeState stays its own node next to text with other NodeState, as text in another format does. That is the point (the characters keep their comment), and it is the only structural difference: text with no NodeState merges as before. Callers in this repo, all through `registerLexicalTextEntity`: `@lexical/hashtag` (`registerLexicalHashtag`, `HashtagExtension`, `LexicalHashtagPlugin`), `@lexical/react` `useLexicalTextEntity`, and the playground's `KeywordsExtension`. Mentions (typeahead) and `@lexical/link` autolink have their own transforms and do not share this path. Fixes #9703 ## Test plan `.changeset/lexical-text-entity-node-state.md` (`lexical` and `@lexical/text`, patch, `[fix]`). `MarkStateExtension.test.ts`: a hashtag pasted or inserted at the edge of a comment (after it, and before it) is not added to it, with `MarkStateExtension` and with `MarkNode`. `LexicalSelection.test.ts` (3 tests, each fails on `main`): typing inside a segmented node keeps its NodeState and detail; a registered `TextNode` replacement keeps the NodeState it sets and takes the rest; removing a range from inside a segmented node to the next block keeps the NodeState and detail of what is left. `registerLexicalTextEntityNodeState.test.ts` (new, 23 tests): #9703's repro; deleting a hashtag's `#`; merging into neighbours with the same NodeState and, without NodeState, merging as before; promotion with NodeState, `createNode`'s state winning, and text inserted after a hashtag not taking the copy; a reference-typed value carried as the same object; NodeState read from JSON and not parsed yet; NodeState declared in the entity's `$config` (nested and flat) and exported nested on the text; a key the text type declares with another config; a hashtag inside a MarkNode; undo and redo; and every path that turns an entity into text: the text transform with no match, a token entity that text follows (same NodeState: it takes the text; other NodeState: kept, the text stays apart; the selection after pasted text; text with its own NodeState, which loops on `main`; a value object distinct from the text's, equal as JSON), the next entity that stops matching, adjacent entities, an entity followed by another, and a join across text in another format. `registerLexicalTextEntityNodeStateProperty.test.ts` (new): random layouts of entities (hashtags, and token mentions) and text in several formats, with NodeState on both and MarkNodes around some, and random edits (typing, backspace and delete, joining blocks, removing ranges across nodes). Each entity has a letter no other text has; after every edit that turns an entity into text, every character of it that is still there has the entity's NodeState and MarkNode (or sits inside another entity that grew over it, which takes characters with its own NodeState, as before). 200 seeds per matcher (`TEXT_ENTITY_NODE_STATE_SEEDS`). Before (main): ``` $ pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeState.test.ts packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts × #9703: a hashtag that stops matching keeps its NodeState when it becomes text (typing before it) × deleting the # of a hashtag keeps its NodeState (reverse transform, no match) × a hashtag that becomes text merges into neighbors with the same NodeState × plain text with NodeState that becomes a hashtag keeps it × state that createNode gives the entity wins over the text it replaces × a reference-typed state value is carried as the same object × NodeState read from JSON (not parsed yet) is carried and exported × NodeState declared in the entity $config (nested and flat) is carried to the text × a key the plain text type declares with another config is parsed with that config × a hashtag inside a MarkNode stays inside it when it becomes text × undo and redo of a demotion restore the entity and the text with their NodeState × the text transform: the entity before it stops matching with its text (prevMatch null) × with the same NodeState as the text: it keeps it and takes the text × with other NodeState than the text: kept with its NodeState, the text stays apart × typed text after it: the selection stays after the typed text × with other NodeState than text that its match extends into, which keeps it apart: kept, each keeps its own × with a value object distinct from the text after it, equal as JSON: each keeps its own × the text transform: the entity after it stops matching (nextMatch null) × the reverse transform: adjacent entities both become text × the reverse transform: an entity followed by an entity × hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters × mention: an edit that turns an entity into text keeps its NodeState and marks on its characters Tests 22 failed | 3 passed (25) ``` After: ``` $ pnpm exec vitest run --project unit packages/lexical-text packages/lexical-hashtag packages/lexical-mark packages/lexical-history Tests 1376 passed (1376) $ pnpm exec vitest run --project unit packages/lexical/src/__tests__/unit/LexicalSelection.test.ts Tests 103 passed (103) $ TEXT_ENTITY_NODE_STATE_SEEDS=20000 pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts ✓ hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters ✓ mention: an edit that turns an entity into text keeps its NodeState and marks on its characters ``` The existing `@lexical/text` tests pass unchanged, including `registerLexicalTextEntityEquivalence.test.ts` (JSON, selection, mutations and keys after random sessions, against the helper before #9483, which has no NodeState in its layouts), and `MarkStateExtension`'s hashtag tests (text inserted after a marked hashtag is not marked). A differential against `main` over random editing sessions (typing, backspace and delete, joins across paragraphs, headings, quotes, list items and table cells, ranges, formatting, paste with NodeState, undo and redo, IME composition) with hashtags, a keyword matcher, a token and a segmented matcher, autolink and MarkNodes: the NodeState of an entity's characters is kept in tens of thousands of steps where `main` drops it; no text is lost; token and segmented entities that loop on `main` settle. Where NodeState now keeps text apart, the entities can differ from `main`, as they do next to text in another format; a keyword-style matcher can then reach #9637's loop in a layout `main` merged (a few per 10,000 sessions; hashtags never do), which #9637's fix settles.
zurfyx
added a commit
that referenced
this pull request
Nov 2, 2026
…hen they become text ## Description `registerLexicalTextEntity` turns an entity back into plain text when it stops matching (typing right before a hashtag, deleting its `#`, joining blocks), with `$replaceWithSimpleText`. That built the new `TextNode` from the entity's text, format, style and detail, but not its NodeState, so anything an app keeps on the characters as NodeState (comment ids, suggestion marks, block ids) was dropped (#9703). The same happened in the other direction: plain text with NodeState that becomes an entity gave the entity none. What carries, and why: - **Demotion (entity to text): all of the entity's NodeState**, nested and flat, known and not yet parsed (read from JSON), the same value objects (ids compared by identity stay identical). NodeState is the ad-hoc data an app keeps on text; what is inherent to the entity type is its class (its fields) and its mode, which stay with the type, as before. This is what `splitText` already does for the pieces it cuts, and what the code highlighters do since #9503 when a code token becomes text. A key that the plain text type declares with a different `StateConfig` is parsed with the text type's config. Format, style and detail carry as before. - **Promotion (text to entity): the text's NodeState too**, for every key `createNode` did not set (its state wins), with two exceptions, both characters that were just inserted into a node and so only have a copy of its NodeState: text that the entity's own transform splits off its end in the same update (text typed or pasted right after a hashtag), and an entity made only of text inserted at the start or the end of a node that existed in this update, when no entity was turned into text in that block in the update (pasting ` #tag` right after commented text: the hashtag does not join the comment, as on `main` and as `MarkStateExtension`'s rule for text inserted at a comment's edge says). `splitText` gives that piece a copy of the entity's NodeState, but those characters were just inserted, so it is not theirs; an entity made from it keeps the old behaviour (MarkStateExtension relies on telling inserted text from marked text this way). An entity turned into text and back in one update gets its own NodeState back. - **Token and segmented entities** that text follows are turned into text so that they merge with it and are parsed again. With their NodeState kept they do not merge with text that has other NodeState, and the transforms would make the entity again and turn it back forever (on `main` this already looped whenever that text had NodeState, or another format). So, as in #9686, when the same entity (same parent, text, and place: the first node before it that is no text entity, and the size of the entities between) comes back to be turned into text a third time in one update (or updates batched into one commit), it is kept, with its NodeState. It takes the text its match extends into, as an entity in normal mode does, only when that text would merge with it (the same format, style and detail, and NodeState equal by the rule text nodes merge by: `StateConfig.isEqual`, so two distinct value objects are two values; the selection stays where it was in that text); text with other NodeState or format stays apart, so it neither takes the entity's NodeState nor makes the entity drop its own. Until the third time the transforms run as before. The count belongs to the pending editor state, so an update that fails does not count toward the next. - The text split off the end of an entity is tracked per editor, for every registration, so an entity of another registration made from it (a keyword typed after a hashtag) does not take the copy either. - **Core: editing inside a segmented node** replaces it with a plain `TextNode` built from its text, format and style, the same way `$replaceWithSimpleText` did, and dropped its NodeState the same way (and its detail, such as `unmergeable`, which `$replaceWithSimpleText` kept): typing inside it (outside composition, `LexicalSelection`) and removing a range that starts or ends inside it (`LexicalCaretUtils`). Both now keep them (detail, and NodeState with `$carryNodeState`, internal to `lexical`, with the same rules as above; keys the new node already holds, such as state a registered `TextNode` replacement sets, keep their value). Segmented entities that this change keeps or makes reach these paths more often, so they are fixed here. Two text nodes merge only when their NodeState is equal, so text that keeps an entity's NodeState stays its own node next to text with other NodeState, as text in another format does. That is the point (the characters keep their comment), and it is the only structural difference: text with no NodeState merges as before. Callers in this repo, all through `registerLexicalTextEntity`: `@lexical/hashtag` (`registerLexicalHashtag`, `HashtagExtension`, `LexicalHashtagPlugin`), `@lexical/react` `useLexicalTextEntity`, and the playground's `KeywordsExtension`. Mentions (typeahead) and `@lexical/link` autolink have their own transforms and do not share this path. Fixes #9703 ## Test plan `.changeset/lexical-text-entity-node-state.md` (`lexical` and `@lexical/text`, patch, `[fix]`). `MarkStateExtension.test.ts`: a hashtag pasted or inserted at the edge of a comment (after it, and before it) is not added to it, with `MarkStateExtension` and with `MarkNode` (as on `main`). `LexicalSelection.test.ts` (3 tests, each fails on `main`): typing inside a segmented node keeps its NodeState and detail; a registered `TextNode` replacement keeps the NodeState it sets and takes the rest; removing a range from inside a segmented node to the next block keeps the NodeState and detail of what is left. `registerLexicalTextEntityNodeState.test.ts` (new, 24 tests): #9703's repro; deleting a hashtag's `#`; merging into neighbours with the same NodeState and, without NodeState, merging as before; promotion with NodeState, `createNode`'s state winning, text inserted after a hashtag not taking the copy, and a hashtag made only of inserted text not taking it; a reference-typed value carried as the same object; NodeState read from JSON and not parsed yet; NodeState declared in the entity's `$config` (nested and flat) and exported nested on the text; a key the text type declares with another config; a hashtag inside a MarkNode; undo and redo; and every path that turns an entity into text: the text transform with no match, a token entity that text follows (same NodeState: it takes the text; other NodeState: kept, the text stays apart; the selection after pasted text; text with its own NodeState, which loops on `main`; a value object distinct from the text's, equal as JSON), the next entity that stops matching, adjacent entities, an entity followed by another, and a join across text in another format. `registerLexicalTextEntityNodeStateProperty.test.ts` (new): random layouts of entities (hashtags, and token mentions) and text in several formats, with NodeState on both and MarkNodes around some, and random edits (typing, backspace and delete, joining blocks, removing ranges across nodes). Each entity has a letter no other text has; after every edit that turns an entity into text, every character of it that is still there has the entity's NodeState and MarkNode (or sits inside another entity that grew over it, which takes characters with its own NodeState, as before). 200 seeds per matcher (`TEXT_ENTITY_NODE_STATE_SEEDS`). Before (main): ``` $ pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeState.test.ts packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts × #9703: a hashtag that stops matching keeps its NodeState when it becomes text (typing before it) × deleting the # of a hashtag keeps its NodeState (reverse transform, no match) × a hashtag that becomes text merges into neighbors with the same NodeState × plain text with NodeState that becomes a hashtag keeps it × state that createNode gives the entity wins over the text it replaces × a reference-typed state value is carried as the same object × NodeState read from JSON (not parsed yet) is carried and exported × NodeState declared in the entity $config (nested and flat) is carried to the text × a key the plain text type declares with another config is parsed with that config × a hashtag inside a MarkNode stays inside it when it becomes text × undo and redo of a demotion restore the entity and the text with their NodeState × the text transform: the entity before it stops matching with its text (prevMatch null) × with the same NodeState as the text: it keeps it and takes the text × with other NodeState than the text: kept with its NodeState, the text stays apart × typed text after it: the selection stays after the typed text × with other NodeState than text that its match extends into, which keeps it apart: kept, each keeps its own × with a value object distinct from the text after it, equal as JSON: each keeps its own × the text transform: the entity after it stops matching (nextMatch null) × the reverse transform: adjacent entities both become text × the reverse transform: an entity followed by an entity × hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters × mention: an edit that turns an entity into text keeps its NodeState and marks on its characters Tests 22 failed | 3 passed (25) ``` After: ``` $ pnpm exec vitest run --project unit packages/lexical-text packages/lexical-hashtag packages/lexical-mark packages/lexical-history Tests 1378 passed (1378) $ pnpm exec vitest run --project unit packages/lexical/src/__tests__/unit/LexicalSelection.test.ts Tests 103 passed (103) $ TEXT_ENTITY_NODE_STATE_SEEDS=20000 pnpm exec vitest run --project unit packages/lexical-text/src/__tests__/unit/registerLexicalTextEntityNodeStateProperty.test.ts ✓ hashtag: an edit that turns an entity into text keeps its NodeState and marks on its characters ✓ mention: an edit that turns an entity into text keeps its NodeState and marks on its characters ``` The existing `@lexical/text` tests pass unchanged, including `registerLexicalTextEntityEquivalence.test.ts` (JSON, selection, mutations and keys after random sessions, against the helper before #9483, which has no NodeState in its layouts), and `MarkStateExtension`'s hashtag tests (text inserted after a marked hashtag is not marked). A differential against `main` over random editing sessions (typing, backspace and delete, joins across paragraphs, headings, quotes, list items and table cells, ranges, formatting, paste with NodeState, undo and redo, IME composition) with hashtags, a keyword matcher, a token and a segmented matcher, autolink and MarkNodes: the NodeState of an entity's characters is kept in tens of thousands of steps where `main` drops it; no text is lost; token and segmented entities that loop on `main` settle. Where NodeState now keeps text apart, the entities can differ from `main`, as they do next to text in another format; a keyword-style matcher can then reach #9637's loop in a layout `main` merged (a few per 10,000 sessions; hashtags never do), which #9637's fix settles.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
MarkStateExtensiongives inserted text its comment ids with a root transformthat compares the blocks around the caret with the previous editor state. That
comparison only covered the sibling blocks of the caret's block and treated a
nested block (the items of a list) as one opaque unit. Text that a block
change moved in from outside it had no counterpart, so it was taken for text
typed into an existing node: it lost its own ids and got the ids shared by
its neighbours. The core keeps the NodeState in these conversions; with the
transform disabled, the ids survive.
(
$setBlocksType) removed every comment id from its text, with or without@lexical/y.(
$insertList) removed the comment from a link when the item above endedwith the same text, or gave the moved text the comment of the item above
(Bug: [lexical-mark] Converting a paragraph to a list changes MarkStateExtension comment ids on text that did not change #9490).
text in new nodes, lost its ids too.
inserted-text rule could not run (with the code selected, or for pasted
code): Prism and Shiki replace each code highlight node that leaves a code
block with
$createTextNode(text), which drops its NodeState.The comparison now:
wherever the update moved it, and of the text a moved TextNode kept at each
end when the same update also typed into it (each run of inserted units
between kept ones is an insertion, with the ids shared by its neighbours);
blocks around it (an unchanged one stays a single unit, so an edit next to
a large nested list does not read it);
list does when its first item is converted.
@lexical/code-prismand@lexical/code-shikinow copy the NodeState of acode highlight node (flat states included) into the TextNode that replaces
it, keeping any state that a TextNode replacement sets, so any NodeState
survives leaving a code block, not only comment ids.
A node that code moves (rather than creates) into the middle of a marked run
now keeps exactly its own ids instead of taking the run's;
comments.mdsaysso, and that block changes keep every id.
Independent of #9489 and #9496. It merges with #9489 without conflicts of its
own (that PR's
comments.mdconflict with main is unrelated). With #9496 ithas two one-line conflicts in
MarkStateExtension.ts, at the two$collectSegmentscalls: #9496 adds aflavorfirst argument and this addsa
$isUnchangedBlocklast one, so both become$collectSegments(flavor, windows, points, $isUnchangedBlock).Closes #9490
Test plan
MarkStateBlockConversion.test.tsstates the invariant (converting blockskeeps the ids of every character) over every source and target (paragraph,
heading, quote, code with and without highlighting, bullet, number and check
items, nested items as sources, no list as a target), with links, comments
on part, inside a link, overlapping or on all of the text, neighbouring
lists that the conversion merges with (including text that lines up and
shared ids), the caret or a range at several places, text typed in the same
update, and pasted blocks: 27,072 cases. Each also checks that the
conversion happened. It includes both reports and a comment on code text
turned into a paragraph; the lexical-y test round-trips the list item case
through a Yjs document, and the code-prism and code-shiki tests check that
NodeState (nested and flat) survives leaving a code block.
Before
New tests against
main's sources:3,804 of the 27,072 matrix cases fail, for example:
After
Unit tests only (Node/jsdom); no browser or E2E run. Not covered (the
matrix skips them, with the reason): comment ids on part of a code
highlighting token, and marked text that a range conversion merges into a
highlighted code block, which re-tokenizing drops (#9491); and typing inside
a link in the same update that turns it into highlighted code, where the
highlighter puts the caret back at a wrong offset, so the typed text is not
before the caret as the inserted-text rule needs.