[artcashin]

Problem

StaticCommandRunner._command awaits every command on the server's event loop. Provider code does CPU-bound work in sync and async functions alike, so one heavy command blocks every route until it finishes, /widgets.json included. Clients time out, health checks fail, and the API looks wedged. Examples we hit: CBOE options strategy pricing (pandas), CFTC curve bootstraps over whole DTCC day files, FFIEC peer-group averages.

Change

Each provider gets one persistent event loop in its own daemon thread, created on first use (_provider_loop). _command runs the command there and awaits the result from the server loop.

• A slow provider blocks only itself. The server loop and other providers keep answering.
• Loop-bound provider state keeps working. A provider's commands always run on the same loop, so asyncio locks and cached aiohttp/SQLite backends stay valid. A fresh loop per command (asyncio.run in a thread) breaks such providers: in openbb-cboe, openbb-deribit, openbb-nasdaq and openbb-tmx it raised "bound to a different event loop". It also leaked native memory per call in our tests.
• One loop per provider, across routers. The provider comes from provider_choices, else the command's top-level package with the openbb_ prefix dropped, so a provider's standard-model and custom router commands share a loop.

Tests

Four new tests in tests/app/test_command_runner.py:

• one loop per provider, with the prefix ignored;
• a provider's commands share a loop that is never the caller's;
• provider_choices picks the loop;
• a CPU-bound command in one provider doesn't delay another provider or the caller's loop.

All four fail without the change; the file passes 69/69 with it. tests/app and tests/api pass: 1,401 tests and 1,213 subtests. The one failure, test_version::test_is_git_repo_true_for_repo_root, comes from running in a container against a git worktree and is unrelated.

Results from a deployment

We've run this as a patched command_runner.py in OpenBQ's Docker image, calling all 995 widgets one at a time. The number of widgets during which /widgets.json took over 1 s fell from 34 (worst 88.7 s) to 7 (worst 2.3 s), with the same response statuses.

Not included

Commands of the same provider still interleave on that provider's loop as before, so one provider's heavy command can still delay its own other commands. We serialize CBOE and CFTC commands in our deployment, but that's tuning, so it isn't in this PR.

🤖 Generated with Claude Code

[CLAassistant]

All committers have signed the CLA.

[artcashin]

Could a maintainer approve the workflow runs for this PR? They're waiting for approval because it comes from a fork (action_required on openbb-platform-api, openbb-mcp-server, linting, CodeQL and the PR checks). Locally, tests/app/test_command_runner.py passes 69/69, including the four new tests, and tests/app + tests/api pass 1,401 tests and 1,213 subtests. Thanks!

[artcashin]

@dlee-bq, a gentle reminder on this one when you have a moment. It's ready for review and still merges cleanly into develop; the one commit since its base doesn't touch these files.

The workflow runs are still waiting for maintainer approval, because the PR comes from a fork. Locally, tests/app/test_command_runner.py passes 69/69, including the four new tests, and tests/app plus tests/api pass 1,401 tests and 1,213 subtests. Thanks!

[dlee-bq]

Hi @artcashin. Thanks for the PR, however I do not believe this is a viable solution because it defeats the Python interface's behavior that runs functions in a new event loop - effectively making concurrency not possible.

In a process pool, the same load runs 3.5× faster, and light same-provider requests are served by free workers. This works with the REST API and the Python interface.