Skip to content

feat(rules): GEN-002 should check locale content for diacritic corruption, not just parity #516

Description

@rhuanbarreto

Found by an audit that asked, for every agent-memory claim, "would a rule have caught this?"

The gap

GEN-002's companion rules already enforce translation structure:

  • i18n-page-parity — missing and orphan translations, both locales, error severity
  • i18n-translation-drift — an English page changed without its locale sibling in the same changeset

Neither looks at translation content. A Norwegian page whose diacritics have been stripped (monster for mønster), ASCII-approximated (oe/aa/ae), or HTML-entity-escaped (ø) passes both rules, because the file exists and was touched.

That is a real failure mode for machine-assisted translation, and it is invisible in review to anyone who does not read Norwegian.

Proposed rule

A third GEN-002 companion rule scanning locale content for corruption signatures:

  • HTML entities for Nordic and Portuguese letters (ø å æ ç ã …) anywhere in nb/ or pt-br/
  • mojibake sequences from double-encoded UTF-8 (ø, Ã¥, æ, ç, ã)
  • optionally, a nb page containing zero characters from [øæå], which is implausible for a page of Norwegian prose and catches wholesale diacritic stripping

ARCH-021-ascii-only-powershell-scripts.rules.ts is the working idiom: it reads file contents through ctx.readFile over ctx.scopedFiles and reports per-line violations.

Deliberately not proposed

Do not implement this as a byte-length-versus-character-length comparison. Every legitimately accented character is multi-byte in UTF-8, so that check flags correct prose — a wrong suggestion that reached agent memory in this PR before review caught it.

Similarly, the case-insensitive default of PowerShell Select-String makes an ad-hoc Ã/Â grep match legitimate ã/â, reporting every accented line as corrupt. A rule using explicit code points avoids both traps, which is the argument for making this a rule instead of a documented command.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions