docs: add llms.md — AWF /reflect endpoint guide for LLM tool configuration#32744
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Optimized in commit |
There was a problem hiding this comment.
Pull request overview
Adds internal AWF documentation explaining how tools running inside the agent container can discover LLM gateway endpoints/models at runtime via the api-proxy /reflect route.
Changes:
- Added a new guide (
.github/aw/llms.md) describing/reflect, default provider ports, and example configuration snippets for common SDKs/clients. - Linked the new guide from the Reference Documentation table in
.github/aw/github-agentic-workflows.md.
Show a summary per file
| File | Description |
|---|---|
| .github/aw/llms.md | New documentation for discovering and using api-proxy LLM endpoints/models via /reflect. |
| .github/aw/github-agentic-workflows.md | Adds a reference-table link to the new LLM endpoint discovery doc. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (2)
.github/aw/llms.md:53
- This configuration example contradicts the earlier guidance to not pass raw API keys/tokens to the proxy. Many OpenAI clients only require a non-empty key locally; consider using a placeholder value (or clarifying when it is/isn't safe/necessary to reuse
$COPILOT_GITHUB_TOKEN) so readers don't propagate real credentials unnecessarily.
```bash
# Use Copilot as the OpenAI backend
export OPENAI_BASE_URL="http://api-proxy:10002/v1"
export OPENAI_API_KEY="$COPILOT_GITHUB_TOKEN"
**.github/aw/llms.md:81**
* The Gemini model-list command uses `http://api-proxy:10003/v1/models`, but existing AWF reflect examples/tests use a `v1beta/models` route for Gemini (and the doc above recommends using `models_url`). To avoid a likely 404, resolve and call Gemini’s `models_url` from `/reflect` (or update the hard-coded path to match the gateway).
Gemini format → { models: [{name: "models/gemini-..."}] }
curl -sf http://api-proxy:10003/v1/models | jq '[.models[].name | ltrimstr("models/")]'
- Files reviewed: 2/2 changed files
- Comments generated: 4
| ## /reflect Response | ||
|
|
||
| ```json | ||
| { | ||
| "endpoints": [ | ||
| { "provider": "openai", "port": 10000, "configured": true, "models": ["gpt-4o", "o1-mini"], "models_url": "http://api-proxy:10000/v1/models" }, | ||
| { "provider": "anthropic", "port": 10001, "configured": true, "models": ["claude-sonnet-4-5"], "models_url": "http://api-proxy:10001/v1/models" }, | ||
| { "provider": "copilot", "port": 10002, "configured": true, "models": null, "models_url": "http://api-proxy:10002/models" }, | ||
| { "provider": "gemini", "port": 10003, "configured": false, "models": null, "models_url": null } | ||
| ], | ||
| "models_fetch_complete": true | ||
| } | ||
| ``` |
| All ports use the OpenAI-compatible API format. The api-proxy injects auth headers automatically — **do not pass raw API keys** to these URLs. | ||
|
|
|
|
||
| ```bash | ||
| export ANTHROPIC_BASE_URL="http://api-proxy:10001" | ||
| export ANTHROPIC_API_KEY="$ANTHROPIC_API_KEY" |
| | Provider | Port | Base URL | Credentials env var | | ||
| |---|---|---|---| | ||
| | `openai` / `codex` | 10000 | `http://api-proxy:10000/v1` | `OPENAI_API_KEY` | | ||
| | `anthropic` | 10001 | `http://api-proxy:10001/v1` (or no `/v1` for native SDK) | `ANTHROPIC_API_KEY` | | ||
| | `copilot` | 10002 | `http://api-proxy:10002/v1` | `COPILOT_GITHUB_TOKEN` | | ||
| | `gemini` | 10003 | `http://api-proxy:10003/v1` | `GEMINI_API_KEY` | |
Tools that need OpenAI/Anthropic LLM access inside the AWF agent container had no documented way to discover the api-proxy endpoints, ports, or available model names at runtime.
Changes
.github/aw/llms.md(new) — documents the AWF api-proxy/reflectendpoint:provider,port,configured,models,models_url,models_fetch_complete)openai→10000,anthropic→10001,copilot→10002,gemini→10003/reflectand enumerate models viamodels_url/reflectat runtimenetwork.allowedentries needed;api-proxyis in the AWF sandbox network.github/aw/github-agentic-workflows.md— addedLLM API endpoint discoveryrow linking tollms.mdin the Reference Documentation tableExample: discover and configure