| private |
true |
| name |
Dev Auto Model Smoke Test |
| description |
Minimal inference and safe-output test for Pi, Codex, and Claude using auto on GitHub. |
| intent |
Detect whether an engine can complete GitHub inference and emit a safe output using auto. |
| true |
|
| permissions |
| contents |
copilot-requests |
read |
write |
|
| engine |
| id |
model-provider |
pi |
github |
|
| model |
auto |
| checkout |
false |
| tools |
|
| safe-outputs |
|
| timeout-minutes |
5 |
| evals |
| id |
question |
auto_inference_completed |
Did the agent call noop exactly once with message AUTO_MODEL_OK without performing any other task? |
|
|
Call the noop safe-output tool exactly once with message AUTO_MODEL_OK.
Do not inspect the repository, run commands, or perform any other task.