Repository navigation
perf: Defer heavy imports on Python 3.15 with lazy imports - #2564
Open
karangattu wants to merge 3 commits into
Open
karangattu wants to merge 3 commits into
karangattu wants to merge 3 commits into
Conversation
`import shiny` currently loads narwhals, shinychat, and the CLI/server stack (uvicorn, click, questionary, prompt_toolkit, watchfiles) even when none of it is used. Declare the heavy modules in `__lazy_modules__` so Python 3.15+ defers them until first use (PEP 810); the declaration is ignored on Python 3.11-3.14, where behavior is unchanged. Warm-cache `import shiny` on CPython 3.15.0 drops from ~310ms to ~140ms (median, scripts/benchmark_startup.py). Reification-on-first-use is covered by tests/pytest/test_lazy_imports.py (skipped before 3.15).
Pin the encoding for theme CSS/SCSS, `shiny create` templates, bundled Agent Skills, and pytest generator data. These previously used the locale encoding, so non-ASCII content could mis-handle on Python 3.14 and older (e.g. cp1252 locales on Windows); Python 3.15 already defaults to UTF-8.
karangattu
added this pull request to stack #2565
October 15, 2026 04:00
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #2519 — this PR targets
add-python-3-15-supportand will retarget tomainautomatically when #2519 merges.What
Python 3.15 adoption follow-up from #2519:
Lazy imports (PEP 810) on Python 3.15+.
__lazy_modules__declarations defer the modules that dominateimport shinybut are only needed for specific features:narwhalsand the render modules:shiny.render._data_frame,shiny.render._data_frame_utils._selection,shiny.render._data_frame_utils._types,shiny.render._rendershinychat:shiny.ui._chat,shiny.ui._markdown_stream(mirrored inshiny.express.ui)shiny._main): uvicorn, click, questionary, prompt_toolkit, watchfilesEvery deferred module reifies transparently on first use.
__lazy_modules__is ignored on Python 3.11–3.14, so behavior there is unchanged.Explicit UTF-8 for text file I/O (theme CSS/SCSS,
shiny createtemplates, bundled Agent Skills, pytest generator data). These previously used the locale encoding, so non-ASCII content could mis-handle on Python 3.14 and older (e.g. cp1252 locales on Windows); 3.15 already defaults to UTF-8.Measured effect
scripts/benchmark_startup.py(included in this PR): median wall-clock of fresh subprocesses, so the numbers include interpreter startup, on a warm filesystem cache — Apple M2 Pro (macOS), shown with min in parentheses.import shinyfrom shiny import App, uiApp(ui.page_fluid(ui.h1("hi")), None))The Python 3.12 column is unaffected by this PR (the declarations are inert before 3.15); it's included for the cross-version comparison. Reproduce with:
Notes and limitations
sentinel,frozendict, PEP 798 unpacking in comprehensions,TypeForm) were deliberately not adopted: they would need per-version gating and don't have a current use in this codebase.App start time (
shiny run)Time from launching
python -m shiny run app.py --port Nto the firstHTTP 200on/, for a minimal app (ui.input_text+@render.text). Median of 12 runs after 2 warmups, fresh subprocess per run, warm caches, Apple M2 Pro (macOS), same wheel for the two 3.15 environments (3.15 base =add-python-3-15-supporttip).On the full
shiny runpath,shiny._main(uvicorn/click/questionary) is needed by the CLI itself, so the saving comes fromshinychatand thenarwhalsrender stack no longer being loaded before the server is ready: 3.15 alone is roughly even with 3.12 here, while 3.15 + this PR is ~20% faster to first response (min: −28% vs 3.15 base). Afterimport app, none ofshinychat,narwhals,questionary,prompt_toolkit,uvicorn,watchfiles,clickare insys.moduleson 3.15 + this PR; all of them are on 3.15 base and on 3.12.