Skip to content

Filter invisible Unicode after HTML entity normalization #3101

Description

@SamMorrowDrums

Describe the bug

Invisible-character filtering currently occurs before HTML sanitization can decode character entities. Encoded invisible or bidirectional characters can therefore survive filtering. The removal set also omits relevant format characters and variation-selector ranges. Normalize first, then apply a carefully scoped Unicode filter with regression tests.

Affected version

Current main as of 2026-08-19.

Steps to reproduce the behavior

  1. Sanitize text containing numeric HTML entities for invisible or bidirectional Unicode characters.
  2. Observe that HTML processing decodes the entity after the initial invisible-character pass.
  3. See the resulting invisible character remain in output.

Expected vs actual behavior

Expected: invisible-character policy applies to the final normalized text and covers the intended Unicode ranges.

Actual: entity decoding can occur after filtering, and some relevant characters are not covered.

Logs

N/A

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions