Skip to content

think: messenger concurrency is hardcoded and cannot be configured #2313

Description

@harryy2510

Describe the bug

This is a feature request rather than a defect.

The Chat instance ThinkMessengerRuntime.createChat() builds hardcodes its concurrency:

concurrency: { debounceMs: 600, strategy: "burst" },

There is no way for a Think subclass to change it. Every deployment pays a 600ms pause before anyone gets a reply, and nobody can opt into queue, concurrent, debounce, or drop, or tune the burst window, even though the Chat SDK supports all of them through that same option.

The 600ms burst is a sensible default and should stay. The issue is only that it is unreachable.

To Reproduce

  1. Subclass Think and configure a messenger.
  2. Try to make the messenger runtime use concurrent or queue, or shorten the 600ms debounce.
  3. There is no supported way to do it. messageConcurrency governs submits on the agent's own chat surface, not inbound messenger traffic.

Expected behavior

A Think subclass should be able to choose its messenger concurrency, the way it already can for messageConcurrency, with the current behaviour as the default.

Version: @cloudflare/think@0.19.0 (reproduces on main)

Additional context

I have a branch ready that:

  • adds Think.messengerConcurrency, typed MessengerConcurrency (ConcurrencyStrategy | ConcurrencyConfig, exactly what the Chat SDK's concurrency option accepts)
  • defaults it to a new exported DEFAULT_MESSENGER_CONCURRENCY, the same debounced burst as today, so existing agents are unaffected
  • carries the property on MessengerThinkHost as optional, with the runtime falling back to the same constant, so existing implementations of that exported interface stay valid and nothing breaks

main...harryy2510:agents:feat/think-configurable-messenger-concurrency

Naming follows the existing messageConcurrency, and both docstrings cross-reference each other so the distinction is discoverable. If you would rather see this on MessengerDefinition (per messenger) than on the agent (per host), I am glad to move it.

One note on testing: Chat keeps _concurrencyStrategy and _concurrencyConfig private and exposes no accessor, so I could not assert on the constructed instance without reaching into privates. The branch instead proves createChat sources the value from the host via a recording getter on the fake host, plus a test that the shipped default has not silently changed. If you have a seam you would prefer, I will use it.

Includes a minor changeset. Validated with Node 24 / pnpm 11.9.0: full packages/think suite (45 files, 938 tests) passing, tsgo clean, oxlint 0/0, oxfmt --check clean, pnpm run build and pnpm run check:exports green.

I opened this as an issue because pull requests from forks appear to be disabled on this repo. Happy to submit it as a PR instead if that changes.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions