Pydantic AI as an agentic engine

[dsfaccini]

hey guys! @strawgate brought agentic workflows to our attention a couple months ago and we've been
using them since then (https://github.com/pydantic/pydantic-ai/tree/main/.github/workflows, pydantic/pydantic-ai#7211, pydantic/pydantic-ai#7253 (review)).

in parallel we've been running our own handrolled agent that does triage and review in our repo pydantic/pydantic-ai#7235 (comment).

we like agentic workflows and want to keep using them, while also dogfooding our own stack, which
is why we want to build a Pydantic AI based engine pydantic/pydantic-ai-harness#569 following the third-party model in your https://github.github.io/gh-aw/reference/engines/

claude's helping me build it so I'll leave its description below, + some questions we have for you guys so we can make the right choices

thank you for your time and attention!

---

the shape. An owner-maintained engine definition (engine: {id: pydantic-ai, behaviors: ...})
published in pydantic-ai-harness and imported
pinned to a tag, backed by a headless coding-agent CLI installed via runtimes: {uv: {}} +
pre-agent-steps (the aider.md pattern), with secret-strategy: universal-llm-consumer for
auth. Nothing needs to live in your repo or binary -- users import the definition from ours.

built already (pydantic-ai-harness#569):
the engine definition and a smoke workflow, compile-tested:

• a new pydantic-ai engine id imported from a shared definition compiles clean with --strict on
  v0.85.4. we bisected the releases: v0.84.3 still rejects new ids, the Refactoring opencode integration for shared workflows #50145 fix lands in v0.84.4.
  we'll document v0.85.4 as our minimum supported version.
• cross-repo pinned imports (owner/repo/path.md@tag) resolve, SHA-pin, and vendor correctly --
  tag, branch, and bare-SHA refs all work.
• one docs/UX note from testing: the bare-string form (engine: pydantic-ai) errors even when the
  definition is imported -- only the object form (engine: {id: pydantic-ai}) works, and the error
  message doesn't hint at that. cost us a minute; will cost every new user a minute.

being built now: the CLI itself. pydantic-ai-harness ships all the pieces a coding agent needs
(sandboxed filesystem + shell tools, repo context loading, planning, context compaction) but no
console entry point yet -- that's the gap we're closing, designed against your engine contract from
day one: prompt in, one-shot run, exit code out, MCP config consumed from the gateway so safe
outputs flow through the standard safeoutputs server.

three questions where your answers change what we build:

1. install. behaviors.installation.package-manager values other than npm compile clean but
   silently emit no install step. we're fine on the pre-agent-steps path -- but a compile warning
   for non-npm values would save the next Python/Rust engine author a confusing afternoon. want an
   issue/PR for that?
2. logs and metrics. behavior-defined engines get no log parser: no conversation rendering in
   the step summary, and gh aw logs / gh aw audit report zero tokens and turns (AI-credit
   accounting through the proxy still works). we get why -- GitHub Agentic Workflow Engine Enhancement Proposal #20416 scoped declarative log parsing
   out as a non-goal -- but that was before imported engines became the recommended route for third
   parties. is there appetite for a hook here, e.g. a documented streaming-JSONL schema plus
   parse_custom_log.cjs promoted to a real contract? we'd emit whatever schema you standardize,
   and we're happy to contribute the parser.
3. discoverability. once our definition is live and smoke-tested, would you take a link to it
   from the third-party agent guide?
   not asking for a row in the engines table or any support commitment from you -- just a pointer
   for people searching "how do I run X on gh-aw".

whatever shape this takes, we intend to maintain it long-term -- so if any assumption above looks
wrong, tell us now and we'll adjust before we ship :)

[pelikhan]

@copilot allow allow behavior driver parsers to define log parsers.

[pelikhan]

Added log-parser in the latest build.

[pelikhan]

@copilot add a definition based pydantic.md shared agentic workflow and a smoke-pydantic.md smoke Aw to test the integration. Review the issue to find more information about the pydantic-ai-harness.

[DouweM]

Related work on the pydantic-ai side: pydantic/pydantic-ai#7276

[dsfaccini]

hey @pelikhan , thank you for giving this a shot

I tried out the engine but as expected it doesn't work (as mentioned above, there's some more work to be done on our side)

the biggest problem is that the agent that the engine creates has no shell or file system.

we'd like to keep working on this, thus would it be possible to reopen the issue and revert the current integration? or at least remove it from docs/announcements so people aren't compelled to using it? otherwise people will think that our stuff doesn't work.

if you'll allow us we'll create a PR ourselves after we've done the work on our side.