Repository navigation
Feature request: prompt fetches rate limited like feature flag local evaluation #106802
Description
Is your feature request related to a problem?
Customer hitting an API rate limit for a prompt. Currently polling prompts on a 5min fixed interval. Have multiple backends polling.
Describe the solution you'd like
Prompt fetches could behave more like feature flag evaluation:
- One endpoint that returns all the prompts a client needs in one request (see Feature request: prompt management - fetch all prompts at a label in one request #95460 for the label-aware collection fetch).
- A rate limit for prompt reads that is separate from the general API limit and sized for many polling clients, like the flag definitions endpoint.
- Cacheable responses. Prompts do not have to be real time. Data that is a few minutes old is acceptable, so a CDN or HTTP cache layer (for example
Cache-Controlwith a short max-age, or ETags so that an unchanged poll is cheap) is a good fit.
Describe alternatives you've considered
- A customer-hosted shared cache or proxy. It works, but it adds infrastructure and a failure point.
- Longer SDK cache TTLs. They lower the rate per process, but the total still scales with the process count.
Additional context
Top priority from customer feedback. Related: #95460 (fetch all prompts at a label in one request), PostHog/posthog-js#4353 (SDK batch fetch).
Activity
jurajmajerik commentedon Oct 15, 2026
Closing. Here is where each ask landed:
get_all()in posthog-python (7.55.0+) andgetAll()in@posthog/ai(8.13.0+) fetch every prompt at a label in one call.600/minutebudget per personal API key, the same as feature flag local evaluation, separate from the general API limits. Docs: docs(api): list the prompt fetch rate limit posthog.com#21247.304 Not Modifiedwould still count against it. A CDN cannot cache responses that are authenticated with a personal API key. The SDKs cache prompts for 5 minutes and, since fix(ai): coalesce concurrent prompt fetches and time out stalled requests posthog-js#5266 and fix(prompts): share one request between concurrent fetches of the same prompt posthog-python#1059, share one request between concurrent callers in a process, so the request rate scales with process count and poll interval only.With a 5 minute poll, the limit covers about 3,000 processes polling one prompt each, or 3,000
get_all()calls per label. If a fleet needs more than that, reopen this with the numbers.