Skip to content

docs(proxy): add Grafana Cloud Pyroscope user and API token configura… #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mubashir1osmani merged 1 commit into from
May 21, 2026
Merged

docs(proxy): add Grafana Cloud Pyroscope user and API token configura… #52

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/proxy/config_settings.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -863,6 +863,8 @@ router_settings:
| PYROSCOPE_APP_NAME | Application name reported to Pyroscope. Required when LITELLM_ENABLE_PYROSCOPE is true. No default.
| PYROSCOPE_SERVER_ADDRESS | Pyroscope server URL to send profiles to. Required when LITELLM_ENABLE_PYROSCOPE is true. No default.
| PYROSCOPE_SAMPLE_RATE | Optional. Sample rate for Pyroscope profiling (integer). No default; when unset, the pyroscope-io library default is used.
| PYROSCOPE_GRAFANA_USER | Optional. Grafana Cloud Pyroscope user/tenant ID for basic auth. Required when PYROSCOPE_GRAFANA_API_TOKEN is set.
| PYROSCOPE_GRAFANA_API_TOKEN | Optional. Grafana Cloud API/access policy token for Pyroscope basic auth. Required when PYROSCOPE_GRAFANA_USER is set.
| LITELLM_MASTER_KEY | Master key for proxy authentication
| LITELLM_MAX_BUDGET_PER_SESSION_TTL | TTL in seconds for session budget counters used by the max-budget-per-session limiter. Default is 3600 (1 hour)
| LITELLM_MAX_ITERATIONS_TTL | TTL in seconds for session iteration counters used by the max-iterations limiter. Default is 3600 (1 hour)
Expand Down
14 changes: 14 additions & 0 deletions docs/proxy/pyroscope_profiling.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ LiteLLM proxy can send continuous CPU profiles to [Grafana Pyroscope](https://gr
| `PYROSCOPE_APP_NAME` | Yes (when enabled) | Application name shown in the Pyroscope UI. |
| `PYROSCOPE_SERVER_ADDRESS` | Yes (when enabled) | Pyroscope server URL (e.g. `http://localhost:4040`). |
| `PYROSCOPE_SAMPLE_RATE` | No | Sample rate (integer). If unset, the pyroscope-io library default is used. |
| `PYROSCOPE_GRAFANA_USER` | No | Grafana Cloud Pyroscope user/tenant ID. Required when `PYROSCOPE_GRAFANA_API_TOKEN` is set. |
| `PYROSCOPE_GRAFANA_API_TOKEN` | No | Grafana Cloud API/access policy token. Used as the Pyroscope basic auth password. |

3. **Start the proxy**; profiling will begin automatically when the proxy starts.

Expand All @@ -34,6 +36,18 @@ LiteLLM proxy can send continuous CPU profiles to [Grafana Pyroscope](https://gr
litellm --config config.yaml
```

For Grafana Cloud Pyroscope, use the Profiles endpoint as `PYROSCOPE_SERVER_ADDRESS`
and set the Grafana Cloud credentials:

```bash
export LITELLM_ENABLE_PYROSCOPE=true
export PYROSCOPE_APP_NAME=litellm-proxy
export PYROSCOPE_SERVER_ADDRESS=https://profiles-prod-<region>.grafana.net
export PYROSCOPE_GRAFANA_USER=<grafana-cloud-pyroscope-user>
export PYROSCOPE_GRAFANA_API_TOKEN=<grafana-cloud-api-or-access-policy-token>
litellm --config config.yaml
```

4. **View profiles** in the Pyroscope (or Grafana) UI and select your `PYROSCOPE_APP_NAME`.

## Notes
Expand Down