Skip to content

KSES: Keep opening brackets encoded inside allowed code elements - #15971

Open
robelsust wants to merge 3 commits into
WordPress:trunkfrom
robelsust:fix/66673-kses-code-block-shortcodes
Open

robelsust wants to merge 3 commits into
WordPress:trunkfrom
robelsust:fix/66673-kses-code-block-shortcodes

Conversation

@robelsust

Copy link
Copy Markdown

Description

The Block Editor's Code block escapes [ as &#91; when saving post content so shortcode examples inside <pre class="wp-block-code"><code>...</code></pre> stay as literal text rather than executing via do_shortcode(). Since [64233], wp_sanitize_html_kses() decodes character references in #text nodes via get_modifiable_text() and only re-encodes \r, <, &, and >, turning &#91; back into [ when content is filtered for users without unfiltered_html (such as Authors).

This PR updates wp_sanitize_html_kses() in src/wp-includes/kses.php to:

  • Track nesting depth inside allowed HTML <code> elements ($code_depth).
  • Re-encode opening square brackets ([) as &#91; in #text nodes while $code_depth > 0, leaving brackets outside allowed <code> elements unchanged.
  • Add idempotency, entity-normalization, nesting/unmatched-tag, disallowed-<code>, and wp_kses_post() + do_shortcode() regression tests in tests/phpunit/tests/kses.php.

Trac ticket: https://core.trac.wordpress.org/ticket/66673

Testing Instructions

  1. Log in as a user without the unfiltered_html capability (for example, an Author), or run:
    wp_kses_post( '<pre class="wp-block-code"><code>&#91;audio src="https://example.com/a.mp3"]</code></pre>' );
  2. Before this patch: wp_kses_post() returns <pre class="wp-block-code"><code>[audio src="https://example.com/a.mp3"]</code></pre>, and do_shortcode() renders the shortcode on the front end.
  3. After this patch: wp_kses_post() preserves &#91; inside <code>, and do_shortcode() leaves the Code block example unexecuted while still executing shortcodes outside <code>.

Automated Checks (PHP 8.4)

  • ./vendor/bin/phpunit tests/phpunit/tests/kses.php: OK (413 tests, 1447 assertions)
  • ./vendor/bin/phpunit --group kses,shortcode: Tests: 1032, Assertions: 3010, Skipped: 4. (0 failures)
  • ./vendor/bin/phpcs src/wp-includes/kses.php tests/phpunit/tests/kses.php: 2 / 2 (100%) passed with 0 errors and 0 warnings
  • Note: Local ./vendor/bin/phpstan (2.2.13) did not match the repository's composer.json / phpstan.neon.dist (DisallowedBacktickRule), so PHPStan relies on CI.

AI Assistance

AI assistance was used (Antigravity / Gemini) to investigate the regression in wp_sanitize_html_kses(), implement the $code_depth fix, write unit tests, run local PHPUnit/PHPCS verification, and draft this PR description. All code changes, diffs, and test results were reviewed by the contributor (therssoftware / robelsust).


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

Preserve character-encoded opening square brackets inside allowed HTML `<code>` elements in `wp_sanitize_html_kses()` by encoding `[` as `&WordPress#91;` in text nodes while `$code_depth > 0`. This prevents shortcode examples saved by the Code block from decoding back into executable shortcodes when posts are filtered for users without `unfiltered_html`.

Props therssoftware.
See #66673.
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs.

Core Committers: Use this line as a base for the props when committing in SVN:

Props therssoftware, dmsnell.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.


Props Bot commands

You can trigger Props Bot actions by commenting on this PR:

  • @propsbot add <username> to give props to a GitHub or WordPress.org user. Requires write access.
  • @propsbot remove <username> to leave a user out of the props. Requires write access.
  • @propsbot refresh to refresh this list. Adding the props-bot label works too.

@github-actions

Copy link
Copy Markdown

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

  • All changes will be lost when closing a tab with a Playground instance.
  • All changes will be lost when refreshing the page.
  • A fresh instance is created each time the link below is clicked.
  • Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
    it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

@mmtr
mmtr requested review from dmsnell and sirreal November 13, 2026 09:49

@dmsnell dmsnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See note in https://core.trac.wordpress.org/ticket/66208#comment:52.

There is a more-significant approach that needs to be taken instead of focusing on this one particular case.

We can leave escaped characters escaped.

See also #10600 for some prior work.

…special-casing code elements.

Replace the code-element-specific bracket encoding in `wp_sanitize_html_kses()` with general character-reference preservation and normalization for `#text` tokens using `WP_HTML_Decoder::read_character_reference()`, aligned with the approach in WordPress#10600 and #66208:
- Leave non-syntax character references escaped (normalizing missing semicolons, leading zeros, and hex casing).
- Unhide ASCII letters, digits, and colons.
- Normalize numeric syntax character references (`&`, `"`, `'`, `<`, `>`) to named forms and escape stray `&`, `<`, and `>`.

Update `tests/phpunit/tests/kses.php` to cover general character-reference preservation and normalization.

Props therssoftware.
See #66673.
@robelsust

Copy link
Copy Markdown
Author

Thanks for the review and pointers to #66208 and #10600, @dmsnell!

I've reworked this PR to remove the <code>-specific [ encoding and instead preserve and normalize escaped character references across #text nodes in wp_sanitize_html_kses() using WP_HTML_Decoder::read_character_reference() (matching the approach in #10600).

With this change:

  • Already-escaped character references like &#91;, &#x5b; (normalized to &#x5B;), &lbrack;, &nbsp;, etc. remain escaped in text nodes rather than being decoded and lost.
  • Literal unescaped [ characters are left as-is.
  • ASCII letters, digits, and colons are unhidden, numeric syntax entities are normalized to named entities, and stray &, <, > are escaped.

All kses, blocks, and shortcode tests pass locally. Ready for another look!

@dmsnell dmsnell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@robelsust I don’t think we just want to copy my work from #10600 and paste it here under your name 🙃

Did you respond to @sirreal’s feedback from there? Also, there’s no need to unhide characters at this point; we’re just preserving behaviors and not enhancing it

Comment thread src/wp-includes/kses.php Outdated
if ( $is_in_text_integration_point && '#text' === $token_type ) {
if ( '#text' === $token_type ) {
$text = '';
} elseif ( $is_in_text_integration_point ) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what was your reasoning behind making this change?

Comment thread src/wp-includes/kses.php
$span = $this->bookmarks['text'];
$raw_text = substr( $this->html, $span->start, $span->length );
$this->release_bookmark( 'text' );
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there’s a method that already exists on the class doing this

Comment thread src/wp-includes/kses.php
$output .= "\n{$text}";
} else {
$output .= $text;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what’s the reasoning for this change?

…hiding or rewriting.

Address review feedback on PR WordPress#15971:
- Remove the character-unhiding and entity-rewriting logic copied from WordPress#10600.
- Restore the MathML text integration point handling and `$needs_special_newline` logic in `wp_sanitize_html_kses()`.
- Add `escape_text_preserving_character_references()` to leave valid HTML character references as written while escaping stray `&`, `<`, `>`, and `\r`.

Props therssoftware.
See #66673.
@robelsust

Copy link
Copy Markdown
Author

Thank you for calling that out, @dmsnell — you're completely right, and I apologize for copying the normalization/unhiding logic from #10600 and dropping the integration-point and $needs_special_newline handling.

I've pushed a clean update (a8286dd75ac) that:

  1. Removes all of the Formatting: Introduce normalizing function for escaped HTML. #10600 unhiding and entity-rewriting logic.
  2. Restores the $is_in_text_integration_point and $needs_special_newline handling as they were.
  3. Replaces normalize_escaped_text() with a minimal escape_text_preserving_character_references() helper that only escapes stray &, <, >, and \r while leaving existing valid HTML character references as written (no unhiding or rewriting).

Regarding set_bookmark( 'text' ): get_modifiable_text() decodes all character references (turning &#91; into [), so once it returns we can't tell whether an input bracket was originally escaped (&#91;) or literal ([). Using the token span here is only to inspect the raw text for existing character references while still preserving the newline/\x00 preprocessing from get_modifiable_text(). If there's an existing method on WP_HTML_Tag_Processor that exposes the raw span without decoding that I missed, I'd be glad to switch to it.

@dmsnell dmsnell left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You’re absolutely right, @robelsust, there’s no “existing method on WP_HTML_Tag_Processor that exposes the raw span without decoding” but I wasn’t talking about the Tag Processor, but rather this class, which is a sub-class of it.

This is probably a fairly good place to scope-limit the work for WordPress 7.2. It doesn’t normalize character references or add missing semicolons, but we can continue the work post-release.

I would like to see us ensure that all risky inputs are properly normalized, so if you are willing I would be happy if you could explore that (all character references are terminated with a semicolon, all numeric character references contain no leading zeros, all hexadecimal character references use upper-case letters for A–F)

Comment thread src/wp-includes/kses.php
$decoded = WP_HTML_Decoder::read_character_reference( 'data', $text, $next_amp, $token_length );
if ( isset( $decoded ) ) {
$result .= substr( $text, $next_amp, $token_length );
$at = $next_amp + $token_length;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here we would rather skip over this than make two additional string copies. consider, for example, a text that’s fully normalized with no unescaped & characters and plenty of normative character references.

with this code we end up performing two string copies per character reference but we could instead perform none, returning the original string

Comment thread src/wp-includes/kses.php
$result .= strtr( substr( $text, $at, $next_amp - $at ), $replacements );
}

$decoded = WP_HTML_Decoder::read_character_reference( 'data', $text, $next_amp, $token_length );

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by hard-coding data here we have made this otherwise-useful function unusable for HTML attributes. that is fine for this limited private function, but we should name it explicitly by calling it something like escape_text_node_preserving_character_references()

Comment thread src/wp-includes/kses.php
}

return $result;
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you move this above sanitize(), and probably above the foreign-element handling helpers? above or below set_attribute() would be good

'leading zeros and missing semicolon' => array(
'<code>&#0091;gallery]</code> and <p>&#91gallery]</p>',
'<code>&#0091;gallery]</code> and <p>&#91gallery]</p>',
),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because I think the goal is going to be to normalize these, let’s avoid hard-coding unwanted outputs in the test suite

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants