Repository navigation
Conversation
The bls, ecb, eia, imf and oecd build hooks read the cache generator's output with the locale encoding. uv runs PEP 517 builds with PYTHONIOENCODING=utf-8, so the generator writes UTF-8, and on Windows with a CJK code page (cp932, cp936, cp949, cp950) the hook fails with UnicodeDecodeError as soon as a line holds a non-ASCII character, such as the em dash in the ECB and IMF summary lines. Building these providers from source (dev_install.py) then aborts. Run the generator with PYTHONIOENCODING=utf-8 and decode its output as UTF-8, replacing undecodable bytes, so the build no longer depends on the console code page.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request the OpenBB Platform
Description
On Windows with a CJK code page (here cp932, Japanese),
python openbb_platform/dev_install.pystops while buildingopenbb-ecb:The bls, ecb, eia, imf and oecd build hooks run the cache generator with
text=True, so they decode its output with the locale encoding. uv runs PEP 517 builds withPYTHONIOENCODING=utf-8:backslashreplace, so the generator writes UTF-8, and the first non-ASCII character breaks the read. For ECB and IMF that is the em dash in the final summary line, after the whole download (about six minutes for ECB). cp936, cp949 and cp950 fail the same way. Without uv'sPYTHONIOENCODING, the generator writes to the pipe with cp932 instead, andprintof that em dash raisesUnicodeEncodeErrorin the generator.This sets
PYTHONIOENCODING=utf-8for the generator and reads its output withencoding="utf-8", errors="replace"in all five hooks.How has this been tested?
Built the ECB wheel with
uv build --wheelfrom a copy ofopenbb_platform/providers/ecbwithout the cache file, on Windows 11 with the cp932 code page:develop: fails with theUnicodeDecodeErrorabove (exit 2).this branch:
Successfully built openbb_ecb-2.0.0-py3-none-any.whl, with the summary line printed intact.All five hooks: ran
initialize()with a stub generator that prints the same kind of em dash line underPYTHONIOENCODING=utf-8:backslashreplace, as uv sets it. Ondevelopeach raisesUnicodeDecodeError: 'cp932' codec can't decode byte 0x94; on this branch each reads the output and marks the cache file.ruff checkandruff format --checkpass on the five files.Checklist
feature/feature-nameorhotfix/hotfix-name.