Skip to content

False Turkish encoding corruption warnings during code review on Windows PowerShell #13755

Open

Description

@3rt4nm4n

What version of the IDE extension are you using?

26.304.20706

What subscription do you have?

Plus

Which IDE are you using?

VS Code

What platform is your computer?

Microsoft Windows NT 10.0.26200.0 x64

What issue are you seeing?

On Windows PowerShell, Codex sometimes tells me Turkish characters are broken (for example İ, ş, ç) even though the file is actually UTF-8 and looks correct in my IDE. It seems like Codex is reading terminal rendered text and treating that as real file corruption. I need Codex to validate encoding more reliably before warning about text issues. Right now, I get incorrect encoding warnings while reviewing or editing .tsx files that contain Turkish text, and this leads to confusion and extra work.

What steps can reproduce the bug?

You can reproduce this by opening a .tsx file that contains Turkish UTF-8 text, then reading that file through Windows PowerShell output. In some cases, PowerShell shows mojibake characters (for example İ, ş, ç) and Codex treats that terminal rendering as if the source file itself is corrupted even though the file is valid UTF-8 and displays correctly in the IDE.

Example source:
example.tsx:
const label = "Hızlı Geçiş Sistemi";
Then run this in powershell:
Get-Content .\src\app\routes\pages\example.tsx
Codex may read the console output and incorrectly report Turkish character corruption as a real file issue.

What is the expected behavior?

Codex should distinguish between terminal display encoding issues and actual file encoding issues. If the source file is valid UTF-8 and renders correctly in the IDE Codex should not report character corruption. Before flagging text corruption Codex should verify file content using a reliable validation instead of relying only on PowerShell rendered output.

Additional information

Shortly: IDE text rendering is correct but the issue appears in terminal rendered output interpretation.

Activity

  1. added
    bugSomething isn't working
    windows-osIssues related to Codex on Windows systems
    code-reviewIssues relating to code reviews performed by codex
    on Mar 6, 2026
  2. github-actions commented on Mar 6, 2026

    @github-actions
    Contributor

    Potential duplicates detected. Please review them and close your issue if it is a duplicate.

    Powered by Codex Action

  3. aymenmerabta5 commented on Mar 6, 2026

    @aymenmerabta5

    Arabic encording corruption as well

  4. ugurgok commented on Mar 14, 2026

    @ugurgok

    Use PowerShell 7.5.4.

  5. aymenmerabta5 commented on Mar 14, 2026

    @aymenmerabta5

    @ok-oai any one from openai its urgent

  6. 3rt4nm4n commented on Mar 18, 2026

    @3rt4nm4n
    Author

    Use PowerShell 7.5.4.

    Thanks, I tested this and can confirm that on PowerShell 7.5.5, the same file is displayed correctly without any issues.

    So switching to PowerShell 7 does resolve the problem.

    However, I believe this still represents a usability issue. Most Windows users are still on PowerShell 5.1 by default and may not be aware of encoding differences or the need to upgrade.

  7. evrenonur commented on Jun 19, 2026

    @evrenonur

    I investigated this while reproducing #15422 / #13755 on Windows PowerShell 5.1.

    Root-cause hypothesis: the shell-command wrapper currently pins [Console]::OutputEncoding, but Windows PowerShell can still decode BOM-less UTF-8 Markdown through the active ANSI code page unless the surrounding PowerShell process is consistently forced to UTF-8. In practice, commands like Get-Content / Select-String can return mojibake for Turkish characters, which then makes generated patches mismatch the file bytes.

    I prepared a focused branch with a potential fix here:

    https://github.com/evrenonur/codex/tree/codex/powershell-utf8-text-io

    Summary of the approach:

    • run chcp.com 65001 in the Windows PowerShell startup prefix
    • set [Console]::InputEncoding, [Console]::OutputEncoding, and $OutputEncoding to UTF-8 without BOM
    • default Get-Content and Select-String to UTF-8 for the tool-call shell process
    • add a Windows-only regression test for BOM-less UTF-8 Markdown containing Turkish characters

    Validation I ran locally:

    • cargo fmt --package codex-shell-command
    • cargo test -p codex-shell-command powershell --lib

    I noticed the current contributing guide says external PRs are invitation-only, so I am leaving this as analysis and a concrete patch outline in the issue thread rather than relying on an unsolicited PR.

  8. 3rt4nm4n commented on Jun 28, 2026

    @3rt4nm4n
    Author

    Use PowerShell 7.5.4.

    I'm using PowerShell 7.5.4, and while it fixed the mojibake issue for me for some time, I've restarted encountering the same problem recently. Codex sometimes doesn't preserve Turkish characters correctly when generating or editing text, even though the same file already contains correctly written Turkish text.

    When I read the file with PowerShell 7.5.4, the existing Turkish text is displayed correctly, so this no longer seems to be a PowerShell rendering issue. However, Codex still generates new text with incorrect Turkish characters. With PowerShell 5.x, at least the existing text was displayed incorrectly.

    It seems to ignore the existing context unless the prompt explicitly tells it to preserve the Turkish alphabet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingcode-reviewIssues relating to code reviews performed by codexextensionIssues related to the VS Code extensionwindows-osIssues related to Codex on Windows systems

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions