docs: position the development runner as not for production use#995
docs: position the development runner as not for production use#995mattshep wants to merge 4 commits into
Conversation
The development runner (pipecat.runner.run) is a local-dev tool: its session-start and transport-webhook endpoints are unauthenticated, unsigned, and unthrottled, and a single request can create paid resources. Rather than harden it toward production parity, make the docs state the boundary clearly. - self-hosting: pull the runner out of the numbered options; lead with a boundary warning; the real options are now "build your own dispatcher" and "use a managed runtime". Add a prototyping-only section covering the minimum reverse-proxy smarts to put in front of the runner and where that stops being enough. - overview: drop "the runner directly" from the list of production session-start servers. - running-bots-locally: replace the "genuinely a fine production target" closing note with a clear dev-tool-not-a-production-server boundary.
|
🔍 Mintlify preview for this branch: https://daily-ms-improve-self-hosting-docs.mintlify.site |
| For production-grade images, Pipecat Cloud's [base image](https://github.com/daily-co/pipecat-cloud-images) (`dailyco/pipecat-base`) is what most of the [examples](https://github.com/pipecat-ai/pipecat-examples) build from. It's also a reasonable starting point for self-hosted deployments — it provides a sensible runtime environment for a bot file, and you can use it without using PCC. | ||
|
|
||
| ## When the development runner isn't enough | ||
| ## The runner is a development tool, not a production server |
There was a problem hiding this comment.
I wonder if we should add a callout for this right at the beginning of this page and link to this section.
What do you think ?
markbackman
left a comment
markbackman
left a comment
There was a problem hiding this comment.
Nice! Thanks for the update.
| For production-grade images, Pipecat Cloud's [base image](https://github.com/daily-co/pipecat-cloud-images) (`dailyco/pipecat-base`) is what most of the [examples](https://github.com/pipecat-ai/pipecat-examples) build from. It's also a reasonable starting point for self-hosted deployments — it provides a sensible runtime environment for a bot file, and you can use it without using PCC. | ||
|
|
||
| ## When the development runner isn't enough | ||
| ## The runner is a development tool, not a production server |
There was a problem hiding this comment.
The section above this one—"Containerizing for deployment"—feels like it already leads the reader down the path of deploying the runner to production. When the reader then gets to this section, it's a bit of whiplash: "hey, I thought you just suggested I could deploy this runner..."
So it seems to me that a bit more restructuring is in order for this page. Actually, I wonder if "Containerizing for deployment" doesn't even really belong on a page called "Running bots locally" in the first place...the information in it might more naturally live in the "self-hosting" page.
| ## What plays the runner's role | ||
|
|
||
| A practical upper bound per host is "however many concurrent bot subprocesses one host can sustain", which depends heavily on what your pipeline does. CPU-light pipelines that mostly proxy between transport and third-party services can pack many sessions per host; pipelines with local STT/TTS, custom models, or significant VAD work will saturate sooner. | ||
| The two options below span the range from "you run everything yourself" to "a managed runtime runs the bot for you". The first is self-hosting in the literal sense; the second is an alternative to it. They're walked through together because the decision space is the same one ("what serves session-start and runs the bot?") and most teams evaluating self-hosting are also weighing whether they want to be in that business at all. |
There was a problem hiding this comment.
Nit: "span the range of" would work better if there were more than two options. Maybe something more like:
The two options can below can roughly be summarized as "run everything yourself" and "let a managed runtime do it for you".
There was a problem hiding this comment.
Also: "They're walked through together because..." would read more smoothly as "Both options are presented together here because..."
|
|
||
| ## Exposing the runner for prototyping | ||
|
|
||
| Sometimes you need something reachable before you've built either option above — a demo, an alpha test, a prototype a handful of people will hit. If you're going to put the development runner on a public address at all, treat it as **prototyping-only** and put a reverse proxy in front of it that adds, at minimum: |
There was a problem hiding this comment.
I wonder if we can be a smidge softer here, with something like "you can do nothing, but be warned". Like:
If you're going to put the development runner on a public address at all, treat it as **prototyping-only**. We recommend putting a reverse proxy in front of it that adds, at a minimum:
| - **Rate limiting** — the runner has no backpressure. A per-IP and global request cap at the proxy bounds both load and cost if the address is discovered. | ||
| - **TLS termination** — the runner speaks plain HTTP. | ||
|
|
||
| Two things this does _not_ solve, which is why it stays prototyping-only: |
There was a problem hiding this comment.
I would tweak this wording to avoid conveying "if you solve for this very specific list of things, then it will be production-ready". Maybe something like:
Notably, the above does _not_ address the following:
kompfner
left a comment
kompfner
left a comment
There was a problem hiding this comment.
A few suggestions, but overall this is a nice clarification of our recommendations!
| @@ -1,25 +1,33 @@ | |||
| --- | |||
| title: "Self-hosting in production" | |||
There was a problem hiding this comment.
Question: should we broaden the framing of this page to "Running bots in production" instead of "Self-hosting in production"? Only one of the two options presented on this page can be described as "self-hosting".
There was a problem hiding this comment.
Then the next section—"Self-hosting patterns"—should be renamed to "Production hosting patterns" or something, accordingly, too.
|
Going through all the review notes, I'll commit some updates in a bit. |
- Rename "Self-hosting in production" -> "Running bots in production" (only
one of the two options is literally self-hosting); rename the nav group to
"Production hosting patterns" and update card/link labels. Slug unchanged.
- Move the containerizing content off "Running bots locally" onto the
production page, reframed as building a per-session bot image (dispatcher
injects room/token; process runs one conversation and exits) rather than
running the dev runner as a server. Drops the misleading
`CMD python bot.py --host 0.0.0.0` production entrypoint.
- Scope the security claims accurately: POST /start and the Daily dial-in
webhook are unauthenticated, but the runner does sign WS tokens and verify
WhatsApp webhooks, so "no request signing" was overbroad.
- Soften the prototyping guidance ("we recommend"), reword the "does not
solve" list to avoid implying a checklist to production-readiness, and
explain the localhost/--host 0.0.0.0 nuance in context.
- Add a callout at the top of "Running bots locally" linking to the boundary
section.
|
All review feedback should now be addressed. @markbackman you might want to take a second look as well as I renamed a couple of pages. |
There was a problem hiding this comment.
After the title change, should the name of the file change accordingly?
There was a problem hiding this comment.
(If so: maybe we'd need to do something to ensure that old links continue to work)
| ## What plays the runner's role | ||
|
|
||
| A practical upper bound per host is "however many concurrent bot subprocesses one host can sustain", which depends heavily on what your pipeline does. CPU-light pipelines that mostly proxy between transport and third-party services can pack many sessions per host; pipelines with local STT/TTS, custom models, or significant VAD work will saturate sooner. | ||
| The two options below can roughly be summarized as "run everything yourself" and "let a managed runtime do it for you". The first is self-hosting in the literal sense; the second is an alternative to it. Both options are presented together here because the decision is the same one ("what serves session-start and runs the bot?"), and most teams evaluating self-hosting are also weighing whether they want to be in that business at all. |
There was a problem hiding this comment.
Nit: now that this page isn't titled "self-hosting", we don't need to sort of apologize for one of the options not technically being self-hosting.
| The two options below can roughly be summarized as "run everything yourself" and "let a managed runtime do it for you". The first is self-hosting in the literal sense; the second is an alternative to it. Both options are presented together here because the decision is the same one ("what serves session-start and runs the bot?"), and most teams evaluating self-hosting are also weighing whether they want to be in that business at all. | |
| The two options below can roughly be summarized as "run everything yourself" and "let a managed runtime do it for you". |
| Running bots in production means putting one of the options below in front of | ||
| your bots — not exposing the runner to real traffic. If you're only | ||
| prototyping, [Exposing the runner for | ||
| prototyping](#exposing-the-runner-for-prototyping) covers the minimum you'd | ||
| want in front of it, and where that stops being enough. |
There was a problem hiding this comment.
| Running bots in production means putting one of the options below in front of | |
| your bots — not exposing the runner to real traffic. If you're only | |
| prototyping, [Exposing the runner for | |
| prototyping](#exposing-the-runner-for-prototyping) covers the minimum you'd | |
| want in front of it, and where that stops being enough. | |
| Running bots in production means choosing one of the below options — not exposing the runner to real traffic. If you're only | |
| prototyping, [Exposing the runner for | |
| prototyping](#exposing-the-runner-for-prototyping) covers the minimum you'd | |
| want in front of it, and where that stops being enough. |
| [Pipecat Cloud](/pipecat-cloud/introduction) is one example: Daily's managed runtime, purpose-built for Pipecat. The development runner's session-start API is deliberately shaped the same way as PCC's (`POST /start`, `/sessions/{id}/...`), which means a bot file and client built against the runner work against PCC unchanged. Other managed runtimes — like AWS Bedrock AgentCore — are the same shape, with their own platform-specific tradeoffs. | ||
|
|
||
| If you're considering self-hosting partly because you already have an ops team and existing infrastructure to leverage, options 1–2 are probably the right starting point. If you'd rather not build any of that surface, option 3 is the path designed to skip it. | ||
| If you're considering self-hosting partly because you already have an ops team and existing infrastructure to leverage, Option 1 is probably the right starting point. If you'd rather not build any of that surface, Option 2 is the path designed to skip it. |
There was a problem hiding this comment.
I'd delete this holdover paragraph—there's only 1 self-hosting option presented now.
|
|
||
| ## What you'll need to address either way | ||
|
|
||
| Whichever option you pick, a handful of concerns show up in production. None of them have a single right answer; the goal here is just to name them so they don't surprise you. |
There was a problem hiding this comment.
Shoot, sorry, just noticed this. This section ("what you'll need to address either way") doesn't make all that much sense anymore, now that we're framing the choice as between self-hosting and using a managed runner (as opposed to various self-hosting options).
The minimal change might be to reframe this section as something like "Self-hosting considerations" (with wording adjusted accordingly throughout)—these considerations only apply if you go with option 1.
|
So sorry, @mattshep, just noticed that we still have a bunch of language in the "running in production" guide that really only made sense when the guide was focused on self-hosting and only presented the managed option as a bonus. Now that the guide is broadly focused on how to run bots in production, all the stale language should be expunged or updated accordingly. Should've noticed this on my first pass through! |
- "choosing one of the options below" instead of "putting one in front of your bots" (the managed-runtime option isn't something you put in front). - Drop the "self-hosting in the literal sense" aside now that the page isn't titled self-hosting. - Remove the Option 1-vs-2 chooser paragraph (holdover from the three-option structure). - Reframe "What you'll need to address either way" -> "Self-hosting considerations": these are Option 1's to own; a managed runtime handles most.
Match the file/slug to the retitled page. Renames self-hosting.mdx -> running-bots-in-production.mdx, updates the nav path and all internal links, and adds redirects so existing URLs keep working: - /pipecat/deployment/self-hosting -> /pipecat/deployment/running-bots-in-production (current live slug) - /deployment/self-hosting -> /pipecat/deployment/running-bots-in-production (legacy top-level, repointed to avoid a redirect chain)
|
Thanks @kompfner — the latest two pushes close out this round:
One correction worth surfacing from the earlier security wording: I scoped the "unauthenticated" claim to |
4 participants
What & why
We've been receiving security advisories for the development runner (
pipecat.runner.run) — most recently an unauthenticated dial-in-webhook endpoint that lets a remote caller spawn a bot and create a paid Daily room at the operator's expense. Internally we've decided not to harden the dev runner toward production parity; it exists to streamline local development. Our answer for production is a proper dispatcher, a managed runtime, or Pipecat Cloud.This updates the deployment docs to state that boundary clearly, rather than presenting the runner as a viable production target (as PR #824 did). It aligns with the maintainer position on the advisory: "this is a dev runner; NOT meant for production use."
Changes
self-hosting.mdx— Pull the runner out of the numbered options. Lead with a<Warning>stating it's not built or supported for production (unauthenticated/unsigned/unthrottled endpoints; a single request can create paid resources; SPOF). The two real options become Build your own dispatcher and Use a managed runtime. Add a new "Exposing the runner for prototyping" section — the minimum reverse-proxy smarts (auth on/start, rate limiting, TLS), explicitly capped at prototyping/alpha, plus the two things it doesn't solve (transport webhooks have a different caller than/start; lifecycle/capacity/isolation untouched).overview.mdx— Drop "the runner directly" from the list of things that can serve production session-start traffic.running-bots-locally.mdx— Replace the "genuinely a fine production target" closing note with a clear dev-tool-not-a-production-server boundary. Local-dev happy path is otherwise untouched.Deliberately no links to specific security advisories or CVE IDs — citing them invites a "just patch these and it's fine" read, which reintroduces the "sort-of production-ready" framing we're trying to kill.
Follow-ups (not in this PR)
--host 0.0.0.0containerizing example inrunning-bots-locally.mdxstill binds the runner to all interfaces; left as-is to keep scope tight, but it's the one remaining spot that soft-contradicts the new posture.