diff --git a/docs/decisions/0035-dotnet-agent-hooks-enforcement.md b/docs/decisions/0035-dotnet-agent-hooks-enforcement.md new file mode 100644 index 00000000000..69a84736299 --- /dev/null +++ b/docs/decisions/0035-dotnet-agent-hooks-enforcement.md @@ -0,0 +1,42 @@ +--- +status: proposed +contact: MohammadHaroonAbuomar +date: 2026-08-07 +deciders: agent-framework .NET maintainers +--- + +# .NET agent-hooks enforcement: composed factory over three seams + +## Context and Problem Statement + +The [AGENT-HOOKS-0.1](https://github.com/responsibleai/agent-hooks) interception contract shipped for Python as a first-class experimental core feature (#7515): a middleware bundle emitting eight interception points with three-verdict, fail-closed enforcement, transform write-back, buffered streaming, and verdict-before-durability persistence gating. The .NET side needs the same semantics, but the .NET framework has no category-based middleware lists — interception is decorator composition (`DelegatingAIAgent`, Microsoft.Extensions.AI `DelegatingChatClient`, the function-invocation middleware seam). How should the contract's indivisibility and enforcement properties be realized in that model? + +## Decision Drivers + +- Identical enforcement semantics to the merged Python feature (same spec, same fail-closed rules), diverging only where the .NET seam model requires it — never by weakening an enforcement property. +- Partial installation of the enforcement must be impossible or loudly rejected, not silently degraded. +- Denied content must never become durable; transformed content must persist post-transform. +- No changes to existing framework source; the optional native-runtime dependency (`ResponsibleAI.AgentHooks`) must not be referenced by core packages. + +## Decision Outcome + +**A single factory (`AsAIAgentWithAgentHooks`, per-run and host-owned-session overloads) in a new package `Microsoft.Agents.AI.AgentHooks` composes the full enforcement itself** instead of exposing middleware values: + +- **Seam order (fixed by construction):** `AgentHooksAgent` (agent seam: `agent_startup`/`input`/`output`/`agent_shutdown`, per-run `AsyncLocal` state, buffered streaming, persistence gate) → framework function-invocation middleware (`pre_tool_call`/`post_tool_call`) → `ChatClientAgent` with its default pipeline → `AgentHooksChatClient` **below** `FunctionInvokingChatClient` (so `pre_model_call`/`post_model_call` bracket every model service call of the tool loop individually). +- **Indivisibility:** the seam decorators are `internal`; only the factory composes them. Two pipeline-replacement affordances of `ChatClientAgent` are rejected loudly (fail closed): a caller-supplied per-run `ChatClientFactory` (the framework's own function-middleware factory is recognized and allowed — it wraps, not replaces), and a supplied chat client that already contains a `FunctionInvokingChatClient` (it would execute tools below the verdicts). +- **Verdict-before-durability:** end-of-run history and context-provider writes defer behind the `output` verdict via gating provider wrappers installed by the factory (dropped on deny, flushed post-transform with verdicted-message substitution for streamed runs). The implicit default `InMemoryChatHistoryProvider` is materialized and gated, with the history-conflict flags set to mimic implicit-default semantics. Per-service-call persistence sits above the chat seam, so it is covered by its own `post_model_call` verdict. Per-run provider overrides are wrapped in both `AdditionalProperties` dictionaries, copy-on-write. Nested agents persist inline at their own boundaries (they have their own providers) — no run-identity bookkeeping is needed, unlike Python. +- **Fail-closed error behavior:** interceptor crashes/timeouts surface as `host_error:*` denies; enforcement-layer failures at the tool seam halt the run through `FunctionInvocationContext.Terminate` (the loop's only loud escape — thrown exceptions are converted to tool errors by the loop, which would fail open); wire projections run inside the guarded blocks; failure notifications to providers are redacted (empty request messages) once a deny/halt stands. +- **Streaming:** fully buffered per the spec's `buffered_output` semantics — zero egress ahead of a verdict; transformed responses re-derive the released updates (preserving continuation tokens) so egress never diverges from verdicted content. + +### Considered Alternatives + +- **Port Python's middleware-value model (a `MiddlewareBundle` type):** rejected — .NET has no middleware list to put a bundle into; indivisibility via runtime validation is weaker than construction ownership. +- **Core-framework persistence gate (as Python added in `_sessions.py`):** rejected — unnecessary in .NET; construction ownership of the provider instances gives the same property with zero core changes. +- **Per-run `ChatClientFactory` as the chat-seam install point:** rejected — it wraps the whole pipeline above the function-invocation loop, so per-model-call points would be impossible. + +## Consequences + +- Good: zero existing-source changes; the optional native dependency is isolated in one leaf package; enforcement properties are structural rather than convention-based. +- Accepted: the package ships in the release solution filter as an **alpha** package (maintainer decision on the PR) — the version suffix follows the maturity of the `ResponsibleAI.AgentHooks` dependency it is built on, and the whole surface stays `[Experimental]`; a sample follows once the API shape settles. +- Known limitations (documented on the factory): hosted (service-executed) tools never reach the function seam and are intercepted via the `post_model_call` content projection; service-managed (conversation-id) history is durable at the service and ungateable; the deferred-OTel decorator sits above the chat seam, so sensitive-data request spans observe pre-transform content; a chat-seam projection failure fails the run closed but without a synthesized `host_error` record (SDK affordance gap, responsibleai/agent-hooks#70). +- The trust model is the spec's: cooperative contract, not a security boundary — the misuse rejections catch accidental foot-guns loudly, not in-process adversaries. diff --git a/dotnet/Directory.Packages.props b/dotnet/Directory.Packages.props index 3ba3f3b13b5..4a472fc8050 100644 --- a/dotnet/Directory.Packages.props +++ b/dotnet/Directory.Packages.props @@ -108,6 +108,7 @@ + diff --git a/dotnet/agent-framework-dotnet.slnx b/dotnet/agent-framework-dotnet.slnx index 9442f98cb18..380ebba2067 100644 --- a/dotnet/agent-framework-dotnet.slnx +++ b/dotnet/agent-framework-dotnet.slnx @@ -604,6 +604,7 @@ + @@ -641,6 +642,7 @@ + diff --git a/dotnet/agent-framework-release.slnf b/dotnet/agent-framework-release.slnf index 01f099e7819..8d31405597f 100644 --- a/dotnet/agent-framework-release.slnf +++ b/dotnet/agent-framework-release.slnf @@ -4,6 +4,7 @@ "projects": [ "src\\Microsoft.Agents.AI.A2A\\Microsoft.Agents.AI.A2A.csproj", "src\\Microsoft.Agents.AI.Abstractions\\Microsoft.Agents.AI.Abstractions.csproj", + "src\\Microsoft.Agents.AI.AgentHooks\\Microsoft.Agents.AI.AgentHooks.csproj", "src\\Microsoft.Agents.AI.Anthropic\\Microsoft.Agents.AI.Anthropic.csproj", "src\\Microsoft.Agents.AI.GitHub.Copilot\\Microsoft.Agents.AI.GitHub.Copilot.csproj", "src\\Microsoft.Agents.AI.Harness\\Microsoft.Agents.AI.Harness.csproj", diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/AgentHooksChatClientExtensions.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/AgentHooksChatClientExtensions.cs new file mode 100644 index 00000000000..3337f35926e --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/AgentHooksChatClientExtensions.cs @@ -0,0 +1,266 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Linq; +using AgentHooks; +using Microsoft.Extensions.AI; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// Factory for AGENT-HOOKS-0.1 enforced agents: composes a +/// whose runs emit every applicable interception point of the agent-hooks control +/// contract and enforce the combined verdicts fail-closed. +/// +/// +/// +/// The enforcement is one coherent feature riding three seams that this factory installs +/// as one indivisible unit (the seam decorators are internal, so a partial install is +/// impossible by construction): +/// +/// agent_startup / input / output / agent_shutdown at the agent seam, +/// pre_model_call / post_model_call at the chat seam (below the function-invocation loop, so every model service call is bracketed individually), +/// pre_tool_call / post_tool_call at the function-invocation seam. +/// +/// +/// +/// Enforcement semantics (): every interception +/// point is emitted before the guarded action runs (pre points) or before its result is +/// incorporated (post points); emission failures inside the SDK synthesize +/// host_error:* denies and are treated as blocks — the feature never fails open. +/// Transform verdicts are written back into the native values (messages, arguments, +/// results) so the framework executes exactly the value the interceptors approved, and +/// rich (non-text) content is preserved as content objects, never flattened to text. A +/// deny at input, pre_model_call, post_model_call or output +/// terminates the run: propagates to the +/// caller of +/// (for streaming runs, when the stream is consumed, with zero updates released). A deny +/// at the tool seam blocks the tool call and surfaces a tool-error payload to the model +/// so the agent loop can continue; a host_error:* deny there additionally halts +/// the run. Streaming is fail-closed by buffering: no partial content ever egresses +/// ahead of a verdict. +/// +/// +/// Durable history persistence is gated behind the verdicts: end-of-run history and +/// context-provider writes are deferred until the output verdict permits the +/// content (denied content never becomes durable; transformed content persists +/// post-transform), and per-service-call history persistence sits above the chat seam, +/// so it only ever observes responses its own post_model_call verdict permitted — +/// a permitted per-service-call write remains durable even if the run's output is +/// later denied. Nested and sibling agents (including sub-agents invoked as tools) have +/// their own providers and persist inline at their own run boundaries. Residual +/// limitation: history managed server-side by the model service (a conversation id) is +/// durable at the service the moment the model call executes and cannot be gated by any +/// framework layer. +/// +/// +/// Known limitation — service-side (hosted) tool execution: tools executed by the model +/// provider itself never pass through the function-invocation seam, so +/// pre_tool_call / post_tool_call cannot intercept them. Their calls and +/// outputs are surfaced faithfully in the post_model_call content projection, +/// where interceptors can observe and deny/transform the response that carries them. +/// +/// +/// Composition order: decorators applied to the returned agent run outside the +/// enforcement boundary (outer position is outer trust — the final output point +/// still guards whatever egresses); decorators applied to the supplied chat +/// client run inside it, below the verdicts. Install exactly one enforcement per agent: +/// nesting one guarded agent's seams inside another fails closed, a supplied client that +/// already contains a function-invocation loop is rejected (it would execute tools below +/// the verdicts), and per-run +/// callbacks are rejected on guarded agents (they would replace the guarded pipeline). +/// +/// +/// Observability note: the agent's built-in deferred-OpenTelemetry decorator sits above +/// the enforcement's chat seam, so when sensitive-data telemetry is enabled its +/// request-side spans capture the request content before any +/// pre_model_call transform is applied (an observer channel inside the +/// enforcement boundary, analogous to outer-position middleware in the Python feature). +/// Response-side telemetry observes only verdicted content; a denied call surfaces as an +/// error span with no response content. +/// +/// +/// Session scoping: by default each agent run is one agent-hooks session (fresh emitter +/// and sequence, agent_startup/agent_shutdown bracket the run). A host +/// that owns a longer-lived session constructs its own +/// and and uses the +/// host-owned overload; the enforcement then emits only the per-run points and the host +/// owns the session boundaries. +/// +/// +public static class AgentHooksChatClientExtensions +{ + /// + /// Creates an over with + /// AGENT-HOOKS-0.1 enforcement installed on every seam, one agent-hooks session per + /// run. + /// + /// The chat client the agent talks to. It is decorated by the enforcement's chat + /// seam before the agent's default pipeline is applied, so every model service call is bracketed. + /// The agent-hooks enforcement options; at least one interceptor is required. + /// Optional agent options; a copy is used, with any configured history and + /// context providers wrapped so durable writes obey verdict-before-durability. + /// Optional service provider passed through to the agent. + /// The enforced agent. + /// or is . + /// has no interceptors. + public static AIAgent AsAIAgentWithAgentHooks( + this IChatClient chatClient, + AgentHooksOptions hooksOptions, + ChatClientAgentOptions? agentOptions = null, + IServiceProvider? services = null) + { + _ = Throw.IfNull(chatClient); + _ = Throw.IfNull(hooksOptions); + if (hooksOptions.Interceptors.Count == 0) + { + throw new ArgumentException( + "agent-hooks enforcement requires at least one interceptor (an emitter with zero interceptors " + + "fails closed on every emission).", + nameof(hooksOptions)); + } + + var configuration = new AgentHooksConfiguration + { + Interceptors = [.. hooksOptions.Interceptors], + Resolver = hooksOptions.Resolver, + Mode = hooksOptions.Mode, + Composition = hooksOptions.Composition, + IdentityProvider = hooksOptions.IdentityProvider, + Timeout = hooksOptions.Timeout, + RecordSink = hooksOptions.RecordSink, + }; + + return Compose(chatClient, configuration, agentOptions, services); + } + + /// + /// Creates an over with + /// AGENT-HOOKS-0.1 enforcement bound to a host-owned session: the fully configured + /// and matching are used for + /// every run, only the per-run points (input through output) are + /// emitted, and the host owns the agent_startup / agent_shutdown + /// session boundaries. + /// + /// The chat client the agent talks to. + /// The host-owned, fully configured . + /// The host-owned matching . + /// Optional agent options; a copy is used, with providers wrapped as in the per-run overload. + /// Optional service provider passed through to the agent. + /// The enforced agent. + /// , or is . + public static AIAgent AsAIAgentWithAgentHooks( + this IChatClient chatClient, + InterceptionEmitter emitter, + AgentContextBuilder builder, + ChatClientAgentOptions? agentOptions = null, + IServiceProvider? services = null) + { + _ = Throw.IfNull(chatClient); + _ = Throw.IfNull(emitter); + _ = Throw.IfNull(builder); + + var configuration = new AgentHooksConfiguration + { + Interceptors = [], + Emitter = emitter, + Builder = builder, + }; + + return Compose(chatClient, configuration, agentOptions, services); + } + + private static AgentHooksAgent Compose( + IChatClient chatClient, AgentHooksConfiguration configuration, ChatClientAgentOptions? agentOptions, IServiceProvider? services) + { + if (chatClient.GetService() is not null) + { + // A supplied client that already contains a function-invocation loop would + // sit BELOW the enforcement's chat seam, inverting the seam order: tools + // would execute before any post_model_call verdict could deny the + // tool-calling response, and the function seam would never see them. + throw new ArgumentException( + "The chat client supplied to the agent-hooks factory must not already contain a " + + $"{nameof(FunctionInvokingChatClient)}: it would execute tools below the enforcement's chat seam, " + + "before any post_model_call verdict and outside the tool seam. Supply the raw chat client instead — " + + "the agent installs its own function-invocation loop above the enforcement.", + nameof(chatClient)); + } + + // Diagnostics logger, resolved the same way the agent resolves its own. + configuration.Logger = + ((services?.GetService(typeof(Extensions.Logging.ILoggerFactory)) as Extensions.Logging.ILoggerFactory) + ?? chatClient.GetService() + ?? Extensions.Logging.Abstractions.NullLoggerFactory.Instance) + .CreateLogger("Microsoft.Agents.AI.AgentHooks"); + + var options = agentOptions?.Clone() ?? new ChatClientAgentOptions(); + if (options.UseProvidedChatClientAsIs) + { + // UseProvidedChatClientAsIs signals a fully custom, do-not-touch client + // stack — which is incompatible with this factory by definition: it always + // decorates the supplied client with the enforcement's chat seam and relies + // on the agent's default pipeline placing the function-invocation loop above + // that seam. Honoring the flag would silently change where (and whether) the + // seams sit, so it is rejected loudly instead. + throw new ArgumentException( + $"{nameof(ChatClientAgentOptions.UseProvidedChatClientAsIs)} is not supported with the agent-hooks " + + "factory: the factory always decorates the supplied chat client with the enforcement's chat seam " + + "and relies on the agent's default pipeline above it. Supply the raw chat client and let the " + + "factory compose the stack.", + nameof(agentOptions)); + } + + bool perServiceCallPersistence = options.RequirePerServiceCallChatHistoryPersistence; + + // Durability gating: wrap the durable-write providers so end-of-run writes defer + // behind the output verdict. The wrappers belong to this composition only, so + // other agents sharing the same underlying providers are unaffected. + if (options.ChatHistoryProvider is null) + { + // With no provider configured, the agent creates a default + // InMemoryChatHistoryProvider internally — which this factory would never + // see, so denied output would become durable session history on the + // zero-config path. Materialize the default here and gate it. An explicitly + // configured provider changes the agent's conflict handling for + // service-managed history (it warns/throws by default), so the conflict + // flags are set to mimic the implicit default: silently disengage. + options.ChatHistoryProvider = new InMemoryChatHistoryProvider(); + options.WarnOnChatHistoryProviderConflict = false; + options.ThrowOnChatHistoryProviderConflict = false; + options.ClearOnChatHistoryProviderConflict = true; + } + + options.ChatHistoryProvider = new AgentHooksGatingChatHistoryProvider( + options.ChatHistoryProvider, configuration, perServiceCallPersistence); + + if (options.AIContextProviders is not null) + { + options.AIContextProviders = options.AIContextProviders + .Select(AIContextProvider (provider) => new AgentHooksGatingAIContextProvider(provider, configuration, perServiceCallPersistence)) + .ToList(); + } + + // Chat seam: decorate the supplied client so the agent's default pipeline + // (including the function-invocation loop) is built on top of it — every model + // service call is bracketed individually. + var guardedClient = new AgentHooksChatClient(chatClient, configuration); + var chatAgent = new ChatClientAgent(guardedClient, options, loggerFactory: null, services); + + // Function seam: bracket every host-executed tool invocation. Skipped when the + // agent has no function-invocation loop (nothing executes tools framework-side; + // hosted tools surface at post_model_call). + AIAgent innerAgent = chatAgent; + if (chatAgent.GetService() is not null) + { + innerAgent = new AIAgentBuilder(chatAgent) + .Use(AgentHooksFunctionMiddleware.CreateCallback(configuration)) + .Build(); + } + + // Agent seam, outermost: owns the per-run state, the run bracket and the + // persistence gate. + return new AgentHooksAgent(innerAgent, configuration); + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/AgentHooksOptions.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/AgentHooksOptions.cs new file mode 100644 index 00000000000..390527921a0 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/AgentHooksOptions.cs @@ -0,0 +1,70 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using AgentHooks; +using Microsoft.Extensions.AI; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// Options controlling the AGENT-HOOKS-0.1 enforcement installed by +/// . +/// +public sealed class AgentHooksOptions +{ + private readonly List> _interceptors = []; + + /// + /// Initializes a new instance of the class. + /// + /// The agent-hooks interceptors to register. At least one interceptor is required + /// (an emitter with zero interceptors fails closed on every emission). + /// or one of its elements is . + public AgentHooksOptions(params IEnumerable interceptors) + { + _ = Throw.IfNull(interceptors); + + // Materialized exactly once: the sequence may be single-enumeration. + foreach (var interceptor in interceptors) + { + _ = this.AddInterceptor(interceptor); + } + } + + /// Register an interceptor, optionally with a payload-free name recorded on the records' verdict summaries. + /// The interceptor to register. + /// An optional registration name. + /// This options instance. + public AgentHooksOptions AddInterceptor(IInterceptor interceptor, string? name = null) + { + _ = Throw.IfNull(interceptor); + this._interceptors.Add(new KeyValuePair(name, interceptor)); + return this; + } + + /// Gets the registered interceptors, in registration order. + public IReadOnlyList> Interceptors => this._interceptors; + + /// Gets or sets the optional approval resolver consulted for liftable denies. + public IApprovalResolver? Resolver { get; set; } + + /// Gets or sets whether verdicts are enforced (default) or recorded without acting. + public EnforcementMode Mode { get; set; } = EnforcementMode.Enforce; + + /// Gets or sets the composition profile and knobs; uses the SDK default + /// (sequential/first_deny, on_approval: stop). + public CompositionConfig? Composition { get; set; } + + /// Gets or sets the identity provider; uses the SDK default + /// (jcs-sha256). Use for identity-unbound records. + public IdentityProvider? IdentityProvider { get; set; } + + /// Gets or sets the per-interceptor/resolver timeout; uses the + /// spec-recommended 5 seconds. + public TimeSpan? Timeout { get; set; } + + /// Gets or sets an optional callback receiving every interception record. + public Action? RecordSink { get; set; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/AgentHooksWriteBackException.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/AgentHooksWriteBackException.cs new file mode 100644 index 00000000000..f9ebef2c183 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/AgentHooksWriteBackException.cs @@ -0,0 +1,29 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// A transform verdict could not be converted back into the native framework value. +/// +/// +/// Thrown (and deliberately never caught by this package) so an unappliable transform +/// fails the run closed instead of silently proceeding with the untransformed value. +/// +internal sealed class AgentHooksWriteBackException : InvalidOperationException +{ + public AgentHooksWriteBackException() + { + } + + public AgentHooksWriteBackException(string message) + : base(message) + { + } + + public AgentHooksWriteBackException(string message, Exception innerException) + : base(message, innerException) + { + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/InputCodec.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/InputCodec.cs new file mode 100644 index 00000000000..5ee5a3df068 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/InputCodec.cs @@ -0,0 +1,100 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Collections.Generic; +using System.Linq; +using System.Text.Json.Nodes; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// input: the run's input messages <-> the spec's input payload. +internal static class InputCodec +{ + /// Project one input message with the spec's input role mapping. + public static JsonObject MessageToWire(ChatMessage message) => new() + { + ["role"] = Wire.InputRole(message.Role), + ["content"] = Wire.ContentsToWire(message.Contents), + }; + + /// + /// Project run input per the spec's input payload schema: a single plain-text + /// message projects as its content string (so string-matching perimeter guards fire); + /// multi-message or rich input projects as a list of per-message objects. + /// + /// + /// The content and role are returned alongside the payload object so callers never + /// have to re-read them out of the payload by property name — the projection is the + /// single producer, and this shape makes the "both fields always exist" invariant + /// hold by construction rather than by lookup. + /// + public static (JsonObject Payload, JsonNode? Content, string Role) ToWire(IReadOnlyList messages) + { + JsonNode? content; + string role; + if (messages.Count == 1) + { + content = Wire.ContentsToWire(messages[0].Contents); + role = Wire.InputRole(messages[0].Role); + } + else + { + var parts = new JsonArray(); + foreach (var message in messages) + { + parts.Add(MessageToWire(message)); + } + + content = parts; + role = "user"; + } + + return (new JsonObject { ["content"] = content, ["role"] = role }, content, role); + } + + /// Write a transformed input target back into the run's message list. + public static void WriteBack(List messages, JsonObject before, JsonNode? after) + { + if (after is null || Wire.WireEquals(after, before)) + { + return; + } + + if (after is not JsonObject afterObject) + { + throw new AgentHooksWriteBackException("agent-hooks input transform must produce an input object target."); + } + + var afterRole = afterObject["role"]; + if (!Wire.WireEquals(afterRole, before["role"])) + { + // The role field is per-message only for single-message input; for + // multi-message input the top-level role is synthetic and a transform + // against it is ambiguous. + if (messages.Count != 1 || (afterRole as JsonValue)?.TryGetValue(out string? newRole) is not true) + { + throw new AgentHooksWriteBackException( + "agent-hooks input transform changed the input role in a way that cannot be written back."); + } + + messages[0].Role = new ChatRole(newRole!); + } + + var afterContent = afterObject["content"]; + if (Wire.WireEquals(afterContent, before["content"])) + { + return; + } + + if (messages.Count == 1 && !Wire.LooksLikeMessageObjects(afterContent)) + { + messages[0].Contents = Wire.WireToContents(afterContent, "input"); + return; + } + + List beforeList = [.. messages.Select(MessageToWire)]; + var rebuilt = Wire.WriteBackMessageList([.. messages], beforeList, afterContent, "input"); + messages.Clear(); + messages.AddRange(rebuilt); + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ModelRequestCodec.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ModelRequestCodec.cs new file mode 100644 index 00000000000..586694c71ad --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ModelRequestCodec.cs @@ -0,0 +1,26 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Collections.Generic; +using System.Linq; +using System.Text.Json.Nodes; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// pre_model_call: the outgoing request messages <-> the spec's messages list. +internal static class ModelRequestCodec +{ + public static JsonArray ToWire(IReadOnlyList messages) => Wire.MessagesToWire(messages); + + /// Return the transformed message list, or when the target is untouched. + public static List? WriteBack(IReadOnlyList messages, JsonArray before, JsonNode? after) + { + if (Wire.WireEquals(after, before)) + { + return null; + } + + List beforeList = [.. before.Cast()]; + return Wire.WriteBackMessageList(messages, beforeList, after, "pre_model_call"); + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ModelResponseCodec.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ModelResponseCodec.cs new file mode 100644 index 00000000000..fdfec95aca2 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ModelResponseCodec.cs @@ -0,0 +1,307 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text.Json.Nodes; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// post_model_call: the assembled chat response <-> the spec's response payload. +/// +/// +/// Host-executed tool calls ride tool_calls (they drive the function seam); +/// service-executed (informational-only) tool calls are part of the model response itself +/// and are surfaced in content so hosted tool activity is interceptable here even +/// though the function seam never sees it. +/// +internal static class ModelResponseCodec +{ + private static bool IsHostExecutedCall(AIContent content) => + content is FunctionCallContent { InformationalOnly: false }; + + /// Project the response content (everything except host-executed tool calls). + public static JsonNode? ContentToWire(IList messages) + { + var parts = new JsonArray(); + foreach (var message in messages) + { + List visible = [.. message.Contents.Where(content => !IsHostExecutedCall(content))]; + if (visible.Count == 0) + { + continue; + } + + parts.Add(new JsonObject + { + ["role"] = Wire.RoleString(message.Role), + ["content"] = Wire.ContentsToWire(visible), + }); + } + + if (parts.Count == 0) + { + return null; + } + + if (parts.Count == 1 && parts[0]!["content"] is JsonValue value && value.TryGetValue(out string? text)) + { + return JsonValue.Create(text); + } + + return parts; + } + + /// Project the host-executed tool calls (the ones the function seam will bracket). + public static JsonArray ToolCallsToWire(IList messages) + { + var calls = new JsonArray(); + foreach (var message in messages) + { + foreach (var content in message.Contents) + { + if (content is FunctionCallContent { InformationalOnly: false } call) + { + calls.Add(new JsonObject + { + ["id"] = call.CallId ?? string.Empty, + ["name"] = call.Name ?? string.Empty, + ["args"] = Wire.ArgumentsToWire(call.Arguments), + }); + } + } + } + + return calls; + } + + public static JsonObject ToWire(ChatResponse response) => new() + { + ["content"] = ContentToWire(response.Messages), + ["tool_calls"] = ToolCallsToWire(response.Messages), + ["finish_reason"] = Wire.FinishReasonString(response.FinishReason), + }; + + /// Write a transformed post_model_call target back into the chat response. Returns whether it changed. + public static bool WriteBack(ChatResponse response, JsonObject before, JsonNode? after) + { + if (after is null || Wire.WireEquals(after, before)) + { + return false; + } + + if (after is not JsonObject afterObject) + { + throw new AgentHooksWriteBackException("agent-hooks post_model_call transform must produce a response object."); + } + + bool changed = false; + var afterFinish = afterObject["finish_reason"]; + if (!Wire.WireEquals(afterFinish, before["finish_reason"])) + { + if ((afterFinish as JsonValue)?.TryGetValue(out string? finish) is not true) + { + throw new AgentHooksWriteBackException("agent-hooks post_model_call transform must keep finish_reason a string."); + } + + response.FinishReason = new ChatFinishReason(finish!); + changed = true; + } + + var afterCalls = afterObject["tool_calls"]; + if (!Wire.WireEquals(afterCalls, before["tool_calls"])) + { + changed |= WriteBackToolCalls(response, afterCalls); + } + + var afterContent = afterObject["content"]; + if (!Wire.WireEquals(afterContent, before["content"])) + { + WriteBackContent(response, afterContent); + changed = true; + } + + return changed; + } + + /// Reconcile transformed tool_calls with the response's function-call contents. + private static bool WriteBackToolCalls(ChatResponse response, JsonNode? afterCalls) + { + if (afterCalls is not JsonArray callsArray) + { + throw new AgentHooksWriteBackException("agent-hooks post_model_call transform must keep tool_calls a list."); + } + + // Validate the complete shape up front, before any reconciliation: every call — + // kept or added — must carry a non-empty string id, a non-empty string name and + // object-valued args, and ids must be unique (duplicates would silently collapse + // during reconciliation). An invalid shape fails closed rather than becoming a + // malformed native call. + List<(string Id, string Name, JsonObject Args)> wireCalls = []; + Dictionary callsById = []; + foreach (var item in callsArray) + { + if (item is not JsonObject callObject) + { + throw new AgentHooksWriteBackException("agent-hooks post_model_call transform produced a tool call that is not an object."); + } + + if ((callObject["id"] as JsonValue)?.TryGetValue(out string? id) is not true || string.IsNullOrEmpty(id)) + { + throw new AgentHooksWriteBackException( + "agent-hooks post_model_call transform must give each tool call a non-empty string id."); + } + + if ((callObject["name"] as JsonValue)?.TryGetValue(out string? name) is not true || string.IsNullOrEmpty(name)) + { + throw new AgentHooksWriteBackException( + "agent-hooks post_model_call transform must keep each tool call's name a non-empty string."); + } + + if (callObject["args"] is not JsonObject args) + { + throw new AgentHooksWriteBackException( + "agent-hooks post_model_call transform must keep each tool call's args an object."); + } + + if (!callsById.TryAdd(id!, (name!, args))) + { + throw new AgentHooksWriteBackException( + "agent-hooks post_model_call transform produced two tool calls with the same id."); + } + + wireCalls.Add((id!, name!, args)); + } + + HashSet consumed = []; + bool changed = false; + foreach (var message in response.Messages) + { + List kept = []; + foreach (var content in message.Contents) + { + if (content is not FunctionCallContent { InformationalOnly: false } call) + { + kept.Add(content); + continue; + } + + if (!callsById.TryGetValue(call.CallId ?? string.Empty, out var wire)) + { + changed = true; // the transform dropped this tool call + continue; + } + + consumed.Add(call.CallId ?? string.Empty); + if (wire.Name != call.Name || !Wire.WireEquals(Wire.ArgumentsToWire(call.Arguments), wire.Args)) + { + kept.Add(new FunctionCallContent(call.CallId ?? string.Empty, wire.Name, WireArgsToNative(wire.Args))); + changed = true; + } + else + { + kept.Add(content); + } + } + + if (kept.Count != message.Contents.Count || !kept.SequenceEqual(message.Contents)) + { + message.Contents = kept; + } + } + + List added = []; + foreach (var (id, name, args) in wireCalls) + { + if (!consumed.Contains(id)) + { + added.Add(new FunctionCallContent(id, name, WireArgsToNative(args))); + changed = true; + } + } + + if (added.Count > 0) + { + var target = response.Messages.LastOrDefault(m => Wire.RoleString(m.Role) == "assistant"); + if (target is not null) + { + target.Contents = [.. target.Contents, .. added]; + } + else + { + response.Messages.Add(new ChatMessage(ChatRole.Assistant, added)); + } + } + + return changed; + } + + private static Dictionary WireArgsToNative(JsonObject wireArgs) + { + Dictionary native = []; + foreach (var (key, value) in wireArgs) + { + native[key] = value?.DeepClone(); + } + + return native; + } + + /// Rebuild the response's visible content from a transformed response.content value, preserving host-executed tool calls. + private static void WriteBackContent(ChatResponse response, JsonNode? afterContent) + { + List calls = [.. response.Messages + .SelectMany(message => message.Contents) + .Where(IsHostExecutedCall)]; + + List baseMessages; + if (afterContent is null) + { + baseMessages = []; + } + else if (afterContent is JsonValue value && value.TryGetValue(out string? text)) + { + baseMessages = [new ChatMessage(ChatRole.Assistant, text)]; + } + else if (afterContent is JsonArray array) + { + baseMessages = []; + foreach (var item in array) + { + if (item is not JsonObject wireMessage || !wireMessage.ContainsKey("content")) + { + throw new AgentHooksWriteBackException("agent-hooks post_model_call transform produced content without role/content."); + } + + string role = (wireMessage["role"] as JsonValue)?.GetValue() ?? "assistant"; + baseMessages.Add(new ChatMessage(new ChatRole(role), Wire.WireToContents(wireMessage["content"], "post_model_call"))); + } + } + else + { + throw new AgentHooksWriteBackException("agent-hooks post_model_call transform produced unsupported content."); + } + + if (calls.Count > 0) + { + if (baseMessages.Count > 0 && Wire.RoleString(baseMessages[^1].Role) == "assistant") + { + baseMessages[^1].Contents = [.. baseMessages[^1].Contents, .. calls]; + } + else + { + baseMessages.Add(new ChatMessage(ChatRole.Assistant, calls)); + } + } + + // Mutate the response's message list in place: deferred persistence callbacks and + // outer layers hold references to this list, and must observe the transformed content. + response.Messages.Clear(); + foreach (var message in baseMessages) + { + response.Messages.Add(message); + } + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/OutputCodec.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/OutputCodec.cs new file mode 100644 index 00000000000..6c26ba2531e --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/OutputCodec.cs @@ -0,0 +1,83 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Collections.Generic; +using System.Linq; +using System.Text.Json.Nodes; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// output: the final agent response <-> the spec's output payload. +internal static class OutputCodec +{ + /// Project the run output: a single plain-text message as a string, else per-message objects. + public static JsonNode? ToWire(AgentResponse response) + { + var parts = Wire.MessagesToWire(response.Messages); + if (parts.Count == 1 && parts[0]!["content"] is JsonValue value && value.TryGetValue(out string? text)) + { + return JsonValue.Create(text); + } + + return parts; + } + + /// Write a transformed output target back into the agent response. Returns whether it changed. + /// + /// Mutations happen in place (message contents and the response's message list) so + /// that persistence deferred behind the run gate — which holds references to the same + /// message objects — observes the transformed content, never the pre-transform value. + /// + public static bool WriteBack(AgentResponse response, JsonNode? beforeContent, JsonNode? after) + { + if (after is null) + { + return false; + } + + if (after is not JsonObject afterObject) + { + throw new AgentHooksWriteBackException("agent-hooks output transform must produce an output object target."); + } + + var afterContent = afterObject["content"]; + if (Wire.WireEquals(afterContent, beforeContent)) + { + return false; + } + + List originals = [.. response.Messages]; + if (afterContent is JsonValue value && value.TryGetValue(out string? text)) + { + if (originals.Count == 1) + { + originals[0].Contents = Wire.WireToContents(afterContent, "output"); + } + else + { + ReplaceMessages(response, [new ChatMessage(ChatRole.Assistant, text)]); + } + + return true; + } + + if (afterContent is null) + { + ReplaceMessages(response, []); + return true; + } + + List beforeList = [.. originals.Select(Wire.MessageToWire)]; + ReplaceMessages(response, Wire.WriteBackMessageList(originals, beforeList, afterContent, "output")); + return true; + } + + private static void ReplaceMessages(AgentResponse response, List messages) + { + response.Messages.Clear(); + foreach (var message in messages) + { + response.Messages.Add(message); + } + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ToolArgumentsCodec.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ToolArgumentsCodec.cs new file mode 100644 index 00000000000..066b6769305 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ToolArgumentsCodec.cs @@ -0,0 +1,56 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Collections.Generic; +using System.Text.Json.Nodes; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// pre_tool_call: the native tool arguments <-> the spec's args object. +internal static class ToolArgumentsCodec +{ + public static JsonObject ToWire(IDictionary? arguments) => Wire.ArgumentsToWire(arguments); + + /// + /// Merge a transformed args target back onto the native arguments. + /// + /// + /// Returns the effective wire args, and sets to the merged + /// native arguments (or when untouched). Only the keys the + /// transform actually changed (or added/removed) are taken from the wire value; + /// untouched keys keep their original native values, so non-JSON-native argument + /// values survive a transform that did not touch them. + /// + public static JsonObject WriteBack( + IDictionary arguments, JsonObject before, JsonNode? after, out Dictionary? merged) + { + if (after is not JsonObject effective) + { + throw new AgentHooksWriteBackException("agent-hooks pre_tool_call transform must produce an arguments object."); + } + + if (Wire.WireEquals(effective, before)) + { + merged = null; + return effective; + } + + merged = []; + foreach (var (key, value) in arguments) + { + if (effective.ContainsKey(key)) + { + merged[key] = value; + } + } + + foreach (var (key, value) in effective) + { + if (!before.ContainsKey(key) || !Wire.WireEquals(before[key], value)) + { + merged[key] = value?.DeepClone(); + } + } + + return effective; + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ToolResultCodec.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ToolResultCodec.cs new file mode 100644 index 00000000000..424a3d3195c --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/ToolResultCodec.cs @@ -0,0 +1,81 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Collections.Generic; +using System.Text.Json; +using System.Text.Json.Nodes; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// post_tool_call: the native tool result <-> the spec's result value. +internal static class ToolResultCodec +{ + /// + /// Project a tool result faithfully, unwrapping framework content containers: text + /// content projects as its text, function-result content projects as its canonical + /// result value, and any other content projects as its full content object. + /// + public static JsonNode? ToWire(object? value) + { + switch (value) + { + case null: + return null; + case string s: + return JsonValue.Create(s); + case TextContent text: + return JsonValue.Create(text.Text ?? string.Empty); + case FunctionResultContent { Result: not null } result: + return ToWire(result.Result); + case AIContent content: + return JsonSerializer.SerializeToNode(content, typeof(AIContent), Wire.JsonOptions); + case IList { Count: 1 } single: + // The canonical single-content result projects as the content's value + // itself, matching what the model sees. + return ToWire(single[0]); + case IEnumerable contents: + var array = new JsonArray(); + foreach (var item in contents) + { + array.Add(ToWire(item)); + } + + return array; + default: + return Wire.ValueToWire(value); + } + } + + /// + /// Convert a transformed post_tool_call value back into the native result + /// shape. A wire value the interceptors left untouched maps back to the untouched + /// native result; text-content wrappers are preserved when shape-compatible; + /// otherwise the transformed wire value becomes the result as-is (the function + /// invocation layer serializes JSON values faithfully). + /// + public static object? WriteBack(object? original, JsonNode? before, JsonNode? after) + { + if (Wire.WireEquals(after, before)) + { + return original; + } + + if (original is string && after is JsonValue afterString && afterString.TryGetValue(out string? text)) + { + return text; + } + + if (original is TextContent && after is JsonValue afterText && afterText.TryGetValue(out string? content)) + { + return new TextContent(content); + } + + if (original is IList { Count: 1 } single && single[0] is TextContent && + after is JsonValue afterValue && afterValue.TryGetValue(out string? singleText)) + { + return new List { new TextContent(singleText) }; + } + + return after?.DeepClone(); + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/Wire.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/Wire.cs new file mode 100644 index 00000000000..0d0d1693274 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Codecs/Wire.cs @@ -0,0 +1,290 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text.Json; +using System.Text.Json.Nodes; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// Wire building blocks shared by the per-point codecs (framework values <-> AGENT-HOOKS wire JSON). +/// +/// +/// One codec per interception point owns both directions of the wire conversion: +/// ToWire projects the native framework value into the spec's payload, and +/// WriteBack converts the (possibly transformed) wire target back into the native +/// value. Every WriteBack implements the same rule exactly once per point: a wire +/// value the interceptors left untouched maps back to the untouched native value — only +/// genuine transforms modify native state, and an untranslatable transform throws +/// (fail closed) rather than being dropped. +/// +internal static class Wire +{ + /// The serializer options used for all content projections. + public static JsonSerializerOptions JsonOptions { get; } = AIJsonUtilities.DefaultOptions; + + public static bool WireEquals(JsonNode? left, JsonNode? right) => JsonNode.DeepEquals(left, right); + + public static string RoleString(ChatRole? role) + { + var value = role?.Value; + return string.IsNullOrEmpty(value) ? "user" : value!; + } + + /// Map a framework role onto the spec's input role enum (user | system | external). + public static string InputRole(ChatRole? role) + { + var value = RoleString(role); + return value is "user" or "system" ? value : "external"; + } + + public static string FinishReasonString(ChatFinishReason? finishReason) + { + var value = finishReason?.Value; + return string.IsNullOrEmpty(value) ? "stop" : value!; + } + + /// Project message contents faithfully: plain text as a string, rich content as content objects. + public static JsonNode? ContentsToWire(IList contents) + { + if (contents.Count == 1 && contents[0] is TextContent text) + { + return JsonValue.Create(text.Text ?? string.Empty); + } + + var array = new JsonArray(); + foreach (var content in contents) + { + array.Add(JsonSerializer.SerializeToNode(content, typeof(AIContent), JsonOptions)); + } + + return array; + } + + public static JsonObject MessageToWire(ChatMessage message) => new() + { + ["role"] = RoleString(message.Role), + ["content"] = ContentsToWire(message.Contents), + }; + + public static JsonArray MessagesToWire(IEnumerable messages) + { + var array = new JsonArray(); + foreach (var message in messages) + { + array.Add(MessageToWire(message)); + } + + return array; + } + + /// Decode a transformed wire content value back into framework objects. + public static List WireToContents(JsonNode? value, string point) + { + if (value is null) + { + return []; + } + + if (value is JsonValue jsonValue && jsonValue.TryGetValue(out string? s)) + { + return [new TextContent(s)]; + } + + List items = value switch + { + JsonObject o => [o], + JsonArray a => [.. a], + _ => throw new AgentHooksWriteBackException($"agent-hooks {point} transform produced an unsupported content value type."), + }; + + List contents = []; + foreach (var item in items) + { + if (item is JsonValue itemValue && itemValue.TryGetValue(out string? itemText)) + { + contents.Add(new TextContent(itemText)); + continue; + } + + if (item is JsonObject itemObject && itemObject.ContainsKey("$type")) + { + try + { + var content = JsonSerializer.Deserialize(itemObject, JsonOptions); + if (content is not null) + { + contents.Add(content); + continue; + } + } + catch (Exception exception) when (exception is JsonException or NotSupportedException) + { + throw new AgentHooksWriteBackException($"agent-hooks {point} transform produced an undecodable content item."); + } + } + + throw new AgentHooksWriteBackException($"agent-hooks {point} transform produced an unsupported content item."); + } + + return contents; + } + + public static bool LooksLikeMessageObjects(JsonNode? value) => + value is JsonArray array && array.Count > 0 && + array.All(item => item is JsonObject o && o.ContainsKey("content")); + + /// + /// Convert a transformed wire message list back into framework messages. + /// + /// + /// The transformed list is authoritative. Entries are matched to original messages by + /// projection identity rather than list position, so a removal or insertion in the + /// middle does not shift content onto the wrong original: + /// + /// An entry equal to an (unconsumed) original's projection reuses that + /// original untouched; originals skipped over were removed by the transform. + /// A changed entry mutates the next unconsumed original in place only when + /// that original's projection is not preserved later in the transformed list (i.e. it was + /// modified, not shifted) and its role is unchanged. + /// Anything else (insertions, role changes) becomes a new . + /// + /// + public static List WriteBackMessageList( + IReadOnlyList originals, IReadOnlyList before, JsonNode? after, string point) + { + if (after is not JsonArray afterArray) + { + throw new AgentHooksWriteBackException($"agent-hooks {point} transform must produce a list of messages."); + } + + List afterItems = []; + foreach (var item in afterArray) + { + if (item is not JsonObject itemObject || !itemObject.ContainsKey("content")) + { + throw new AgentHooksWriteBackException($"agent-hooks {point} transform produced a message without role/content."); + } + + afterItems.Add(itemObject); + } + + List result = []; + int cursor = 0; + for (int index = 0; index < afterItems.Count; index++) + { + var item = afterItems[index]; + int? matchIndex = null; + for (int position = cursor; position < originals.Count; position++) + { + if (WireEquals(before[position], item)) + { + matchIndex = position; + break; + } + } + + if (matchIndex is int match) + { + result.Add(originals[match]); + cursor = match + 1; + continue; + } + + if (cursor < originals.Count) + { + var candidateProjection = before[cursor]; + bool preservedLater = afterItems.Skip(index + 1).Any(later => WireEquals(later, candidateProjection)); + string role = (item["role"] as JsonValue)?.GetValue() ?? "user"; + if (!preservedLater && role == (candidateProjection["role"] as JsonValue)?.GetValue()) + { + var message = originals[cursor]; + cursor++; + message.Contents = WireToContents(item["content"], point); + result.Add(message); + continue; + } + } + + string newRole = (item["role"] as JsonValue)?.GetValue() ?? "user"; + result.Add(new ChatMessage(new ChatRole(newRole), WireToContents(item["content"], point))); + } + + return result; + } + + /// Project tool-call arguments as the spec's args object. + public static JsonObject ArgumentsToWire(IDictionary? arguments) + { + var result = new JsonObject(); + if (arguments is not null) + { + foreach (var (key, value) in arguments) + { + result[key] = ValueToWire(value); + } + } + + return result; + } + + /// Project one runtime value into wire JSON, never throwing (repr fallback, matching the Python feature's make_json_safe). + public static JsonNode? ValueToWire(object? value) + { + if (value is null) + { + return null; + } + + if (value is JsonNode node) + { + return node.DeepClone(); + } + + try + { + return JsonSerializer.SerializeToNode(value, value.GetType(), JsonOptions); + } + catch (Exception exception) when (exception is JsonException or NotSupportedException or InvalidOperationException) + { + return JsonValue.Create(value.ToString()); + } + } + + public static JsonObject? UsageToWire(UsageDetails? usage) + { + if (usage is null) + { + return null; + } + + var result = new JsonObject(); + if (usage.InputTokenCount is long input) + { + result["input_token_count"] = input; + } + + if (usage.OutputTokenCount is long output) + { + result["output_token_count"] = output; + } + + if (usage.TotalTokenCount is long total) + { + result["total_token_count"] = total; + } + + if (usage.AdditionalCounts is not null) + { + foreach (var (key, count) in usage.AdditionalCounts) + { + result[key] = count; + } + } + + return result.Count > 0 ? result : null; + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksAgent.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksAgent.cs new file mode 100644 index 00000000000..a6bc2d5d8c3 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksAgent.cs @@ -0,0 +1,432 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Globalization; +using System.Linq; +using System.Threading; +using System.Threading.Tasks; +using AgentHooks; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// Run bracket: emits agent_startup, input, output and +/// agent_shutdown, owns the per-run enforcement state shared with the chat and +/// function seams, and releases the run's deferred durable persistence only after the +/// output verdict permits the content. +/// +/// +/// Streaming runs are fail-closed by buffering: the inner stream is fully consumed, the +/// output verdict is applied to the assembled response, deferred persistence is +/// flushed (or dropped on deny), and only then are the (possibly re-derived) updates +/// released. A deny releases zero updates and surfaces +/// when the stream is consumed. +/// +internal sealed class AgentHooksAgent : DelegatingAIAgent +{ + private const string FrameworkName = "agent-framework"; + + private readonly AgentHooksConfiguration _configuration; + + internal AgentHooksAgent(AIAgent innerAgent, AgentHooksConfiguration configuration) + : base(innerAgent) + { + this._configuration = configuration; + } + + /// + protected override async Task RunCoreAsync( + IEnumerable messages, AgentSession? session = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) + { + var state = this.CreateRunState(); + var previous = AgentHooksRunState.Current; + AgentHooksRunState.Current = state; + string shutdownReason = "completed"; + try + { + List messageList = [.. messages]; + await this.EmitRunStartAsync(state, messageList, options, cancellationToken).ConfigureAwait(false); + + var response = await this.InnerAgent.RunAsync(messageList, session, this.WrapRunOptions(options), cancellationToken).ConfigureAwait(false); + + if (state.Halted is Exception halted) + { + // The enforcement layer itself failed mid-run: strand the deferred + // persistence (fail closed) and surface the halt to the caller. + state.Denied = true; + state.Gate.Drop(); + throw halted; + } + + _ = await EmitOutputAsync(state, response, cancellationToken).ConfigureAwait(false); + + // The verdict permitted the content: release the persistence the run + // deferred behind the gate. A deny drops it instead, so denied content + // never becomes durable, and transformed content persists post-transform + // (the deferred persists substitute the verdicted messages). + state.VerdictedResponseMessages = response.Messages; + await state.Gate.FlushAsync(cancellationToken).ConfigureAwait(false); + return response; + } + catch (InterceptionBlockedException) + { + state.Denied = true; + state.Gate.Drop(); + shutdownReason = "error"; + throw; + } + catch (OperationCanceledException) + { + shutdownReason = "cancelled"; + throw; + } + catch (Exception) + { + shutdownReason = "error"; + throw; + } + finally + { + await this.EmitShutdownAsync(state, shutdownReason).ConfigureAwait(false); + AgentHooksRunState.Current = previous; + } + } + + /// + protected override async IAsyncEnumerable RunCoreStreamingAsync( + IEnumerable messages, AgentSession? session = null, AgentRunOptions? options = null, [System.Runtime.CompilerServices.EnumeratorCancellation] CancellationToken cancellationToken = default) + { + // All guarded work (including full consumption of the inner stream) happens in + // the helper, so a deny surfaces when the returned stream is consumed and zero + // updates egress ahead of the verdict. + var released = await this.RunStreamingGuardedAsync(messages, session, options, cancellationToken).ConfigureAwait(false); + foreach (var update in released) + { + yield return update; + } + } + + private async Task> RunStreamingGuardedAsync( + IEnumerable messages, AgentSession? session, AgentRunOptions? options, CancellationToken cancellationToken) + { + var state = this.CreateRunState(); + var previous = AgentHooksRunState.Current; + AgentHooksRunState.Current = state; + string shutdownReason = "completed"; + try + { + List messageList = [.. messages]; + await this.EmitRunStartAsync(state, messageList, options, cancellationToken).ConfigureAwait(false); + + List buffered = []; + await foreach (var update in this.InnerAgent.RunStreamingAsync(messageList, session, this.WrapRunOptions(options), cancellationToken).ConfigureAwait(false)) + { + buffered.Add(update); + } + + if (state.Halted is Exception halted) + { + state.Denied = true; + state.Gate.Drop(); + throw halted; + } + + var response = buffered.ToAgentResponse(); + bool transformed = await EmitOutputAsync(state, response, cancellationToken).ConfigureAwait(false); + state.VerdictedResponseMessages = response.Messages; + await state.Gate.FlushAsync(cancellationToken).ConfigureAwait(false); + + // No-divergence rule: a transformed output re-derives the released updates + // from the verdicted response, so streamed egress can never diverge from the + // verdicted content. + return transformed ? RederiveUpdates(response) : buffered; + } + catch (InterceptionBlockedException) + { + state.Denied = true; + state.Gate.Drop(); + shutdownReason = "error"; + throw; + } + catch (OperationCanceledException) + { + shutdownReason = "cancelled"; + throw; + } + catch (Exception) + { + shutdownReason = "error"; + throw; + } + finally + { + await this.EmitShutdownAsync(state, shutdownReason).ConfigureAwait(false); + AgentHooksRunState.Current = previous; + } + } + + /// + /// Re-derive stream updates from the (transformed) verdicted response, preserving the + /// response-level metadata that + /// does not project — currently the , + /// without which a transformed streaming background response could not be resumed. + /// + internal static IReadOnlyList RederiveUpdates(AgentResponse response) + { + var updates = response.ToAgentResponseUpdates(); + if (response.ContinuationToken is { } continuationToken) + { + if (updates.Length == 0) + { + updates = [new AgentResponseUpdate { AgentId = response.AgentId, ResponseId = response.ResponseId }]; + } + + updates[^1].ContinuationToken = continuationToken; + } + + return updates; + } + + private AgentHooksRunState CreateRunState() + { + var configuration = this._configuration; + if (configuration is { Emitter: not null, Builder: not null }) + { + // Session-scoped: the host constructed the emitter/builder pair and owns + // the agent-hooks session boundaries — one session (one sequence, one + // record trail, one approval ledger) spans multiple agent runs, so this + // agent emits only the per-run points and never agent_startup / + // agent_shutdown (the host brackets the session itself). + return new AgentHooksRunState(configuration.Emitter, configuration.Builder, sessionScoped: true, configuration); + } + + string agentId = this.Id ?? this.Name ?? "agent"; + var builder = new AgentContextBuilder( + agentId, + framework: FrameworkName, + sessionId: Guid.NewGuid().ToString("N", CultureInfo.InvariantCulture), + agentName: this.Name); + var emitter = new InterceptionEmitter(configuration.Mode, configuration.Resolver, configuration.Timeout); + if (configuration.Composition is not null) + { + _ = emitter.SetComposition(configuration.Composition); + } + + if (configuration.IdentityProvider is not null) + { + _ = emitter.SetIdentityProvider(configuration.IdentityProvider); + } + + if (configuration.RecordSink is not null) + { + _ = emitter.SetRecordSink(configuration.RecordSink); + } + + foreach (var (name, interceptor) in configuration.Interceptors) + { + _ = emitter.Register(interceptor, name); + } + + // Not session-scoped: the default scoping is one agent-hooks session per run. + // The emitter and builder created above are fresh for this run — fresh session + // id, sequence numbering starting at zero, and an isolated record trail — so + // concurrent runs on this agent cannot interleave their emissions, and + // agent_startup / agent_shutdown bracket the run (emitted by this agent, see + // EmitRunStartAsync / EmitShutdownAsync, which skip them when session-scoped). + return new AgentHooksRunState(emitter, builder, sessionScoped: false, configuration); + } + + /// Emit agent_startup (per-run sessions) and input; apply input transforms. + private async Task EmitRunStartAsync( + AgentHooksRunState state, List messages, AgentRunOptions? options, CancellationToken cancellationToken) + { + if (!state.SessionScoped) + { + _ = await state.Emitter.EmitAsync(state.Builder.AgentStartup(this.ResolveToolNames(options)), cancellationToken).ConfigureAwait(false); + } + + // The projection returns content and role directly (single producer, invariant + // by construction) so nothing here re-reads payload properties by name. + var before = InputCodec.ToWire(messages); + var outcome = await state.Emitter.EmitAsync( + state.Builder.Input(before.Content?.DeepClone(), before.Role), cancellationToken).ConfigureAwait(false); + InputCodec.WriteBack(messages, before.Payload, outcome.Target); + } + + /// Emit output over the assembled response; apply output transforms. Returns whether the response changed. + private static async Task EmitOutputAsync(AgentHooksRunState state, AgentResponse response, CancellationToken cancellationToken) + { + var before = OutputCodec.ToWire(response); + var outcome = await state.Emitter.EmitAsync(state.Builder.Output(before), cancellationToken).ConfigureAwait(false); + return OutputCodec.WriteBack(response, before, outcome.Target); + } + + /// Best-effort agent_shutdown (per-run sessions only; blocks there are record-only). + private async Task EmitShutdownAsync(AgentHooksRunState state, string reason) + { + if (state.SessionScoped) + { + return; + } + + try + { + _ = await state.Emitter.EmitUncheckedAsync(state.Builder.AgentShutdown(reason)).ConfigureAwait(false); + } + catch (Exception exception) when (exception is not OutOfMemoryException) + { + // agent_shutdown is a best-effort trail closure: a failure to emit it + // (including I/O-style faults from a record sink) must not mask the run's + // own outcome, which is already propagating — but it is logged so the + // incomplete session trail is trackable. Truly fatal exceptions + // (OutOfMemory; StackOverflow is uncatchable) are not swallowed. + state.Configuration.Logger.LogAgentShutdownEmissionFailed(reason, exception); + } + } + + /// Project the registered tool names for agent_startup (spec tools_registered). + /// + /// This is deliberately the run-start snapshot: the tools declared on the agent and + /// on the run options when the run begins. Tools registered dynamically during the + /// run (for example by context providers during run preparation) cannot be known at + /// agent_startup time; they surface in each pre_model_call emission's + /// tools projection (the completed per-call set) and are bracketed by + /// pre_tool_call/post_tool_call like any other tool when invoked. + /// + private List ResolveToolNames(AgentRunOptions? options) + { + List names = []; + if (this.GetService()?.Tools is { } agentTools) + { + names.AddRange(agentTools.Select(tool => tool.Name)); + } + + if (options is ChatClientAgentRunOptions { ChatOptions.Tools: { } runTools }) + { + names.AddRange(runTools.Select(tool => tool.Name)); + } + + return names; + } + + /// + /// Guard per-run options against enforcement bypasses. + /// + /// + /// + /// A per-run would replace the + /// guarded chat pipeline — and the tool-wrapping stage riding it — silently removing the chat and tool seams, + /// so it is rejected loudly (fail closed). + /// A override can ride either the base + /// (merged into the chat options with precedence by the + /// agent) or ; both would bypass the gating wrapper installed at + /// construction, so both are wrapped — on a clone, never mutating the caller's options. + /// + /// + private AgentRunOptions? WrapRunOptions(AgentRunOptions? options) + { + if (options is null) + { + return null; + } + + if (options is ChatClientAgentRunOptions { ChatClientFactory: { } factory } && !IsFrameworkFunctionMiddlewareFactory(factory)) + { + throw new InvalidOperationException( + $"A per-run {nameof(ChatClientAgentRunOptions.ChatClientFactory)} is not supported on an " + + "agent-hooks-guarded agent: it would replace the guarded chat pipeline (and the tool-wrapping " + + "stage riding it), silently removing the pre/post_model_call and pre/post_tool_call seams. " + + "Decorate the chat client supplied to the agent-hooks factory instead."); + } + + bool wrapBase = this.HasUnwrappedProviderOverride(options.AdditionalProperties); + bool wrapChat = options is ChatClientAgentRunOptions { ChatOptions.AdditionalProperties: { } chatProperties } && + this.HasUnwrappedProviderOverride(chatProperties); + if (!wrapBase && !wrapChat && options is not ChatClientAgentRunOptions) + { + return options; + } + + // Copy-on-write: Clone() deep-copies both dictionaries, so the caller's options + // are never mutated. Chat-typed options are ALWAYS cloned, even when nothing + // needs wrapping: the framework's function-invocation middleware (including this + // composition's own tool seam) chains its factory onto the options instance it + // receives in place, so forwarding the caller's instance would leak that factory + // into it — reusing the same options for a second run would then trip the + // rejection above, and concurrent reuse would race. + var cloned = options.Clone(); + this.WrapProviderOverride(cloned.AdditionalProperties); + if (cloned is ChatClientAgentRunOptions clonedChatOptions) + { + this.WrapProviderOverride(clonedChatOptions.ChatOptions?.AdditionalProperties); + } + + return cloned; + } + + /// + /// Whether a per-run chat-client factory was installed by the framework's own + /// function-invocation middleware (an agent decorator composed outside this agent). + /// + /// + /// That factory wraps the pipeline it is given (it only rewrites the run's tools to + /// add the middleware bracket), so the enforcement seams below stay intact — outer + /// position is outer trust, exactly like any other decorator on the returned agent. + /// A caller-supplied factory chained through it is still rejected by walking the + /// chain. The type check is intentionally narrow: anything unrecognized stays + /// rejected (fail closed). + /// + private static bool IsFrameworkFunctionMiddlewareFactory(Func? factory) + { + while (factory is not null) + { + if (factory.Method.DeclaringType?.FullName?.StartsWith( + "Microsoft.Agents.AI.FunctionInvocationDelegatingAgent", StringComparison.Ordinal) is not true) + { + return false; + } + + // The framework middleware chains any pre-existing factory into its closure; + // walk it so a caller-supplied factory cannot ride in unnoticed. + factory = factory.Target?.GetType() + .GetFields(System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags.NonPublic) + .Where(field => field.FieldType == typeof(Func)) + .Select(field => (Func?)field.GetValue(factory.Target)) + .FirstOrDefault(value => value is not null); + } + + return true; + } + + private bool HasUnwrappedProviderOverride(AdditionalPropertiesDictionary? properties) => + properties is not null && + properties.TryGetValue(out ChatHistoryProvider? overrideProvider) && + overrideProvider is not null && + !this.IsOwnGatingWrapper(overrideProvider); + + private void WrapProviderOverride(AdditionalPropertiesDictionary? properties) + { + if (properties is not null && + properties.TryGetValue(out ChatHistoryProvider? overrideProvider) && + overrideProvider is not null && + !this.IsOwnGatingWrapper(overrideProvider)) + { + bool perServiceCall = this.GetService()?.RequirePerServiceCallChatHistoryPersistence is true; + properties[typeof(ChatHistoryProvider).FullName!] = + new AgentHooksGatingChatHistoryProvider(overrideProvider, this._configuration, perServiceCall); + } + } + + /// + /// Whether is a gating wrapper owned by this + /// installation. Ownership matters: a gating wrapper belonging to a different + /// agent-hooks installation runs inline under this run's state (its own gate is not + /// covering here), so it must be re-wrapped like any unguarded provider — skipping + /// it would let a denied run's history persist through the foreign wrapper. + /// + private bool IsOwnGatingWrapper(ChatHistoryProvider provider) => + provider is AgentHooksGatingChatHistoryProvider wrapper && + ReferenceEquals(wrapper.Configuration, this._configuration); +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksChatClient.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksChatClient.cs new file mode 100644 index 00000000000..e7d8f48bb57 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksChatClient.cs @@ -0,0 +1,198 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// Model bracket: emits pre_model_call and post_model_call around each +/// individual model service call. +/// +/// +/// +/// Installed by the agent-hooks factory directly on the supplied chat client, so it sits +/// below in the agent's default pipeline and +/// brackets every service call of the tool loop individually. +/// +/// +/// Streaming is fail-closed by buffering (spec §12.1 buffered_output): the model +/// stream is fully consumed internally, the post_model_call verdict is applied to +/// the assembled response, and only then are the (possibly re-derived) updates released. +/// No partial content ever egresses ahead of the verdict. +/// +/// +/// Durability note: PerServiceCallChatHistoryPersistingChatClient sits above this +/// decorator, so a permitted (and possibly transformed) response is what gets persisted, +/// and a denied response throws before the persister ever sees it — verdict precedes +/// durability by pipeline order at this seam. +/// +/// +internal sealed class AgentHooksChatClient : DelegatingChatClient +{ + private readonly AgentHooksConfiguration _configuration; + + internal AgentHooksChatClient(IChatClient innerClient, AgentHooksConfiguration configuration) + : base(innerClient) + { + this._configuration = configuration; + } + + public override async Task GetResponseAsync( + IEnumerable messages, ChatOptions? options = null, CancellationToken cancellationToken = default) + { + var state = this.RequireRunState(); + string modelId = this.ResolveModelId(options); + var effectiveMessages = await this.EmitPreModelCallAsync(state, modelId, messages, options, cancellationToken).ConfigureAwait(false); + + var response = await base.GetResponseAsync(effectiveMessages, options, cancellationToken).ConfigureAwait(false); + + await EmitPostModelCallAsync(state, modelId, response, cancellationToken).ConfigureAwait(false); + return response; + } + + public override async IAsyncEnumerable GetStreamingResponseAsync( + IEnumerable messages, ChatOptions? options = null, [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + var state = this.RequireRunState(); + string modelId = this.ResolveModelId(options); + var effectiveMessages = await this.EmitPreModelCallAsync(state, modelId, messages, options, cancellationToken).ConfigureAwait(false); + + // Spec §12.1: the complete response is assembled before post_model_call is + // emitted, and nothing (updates or tool calls) is released beforehand. A deny + // throws before any update egresses. + List buffered = []; + await foreach (var update in base.GetStreamingResponseAsync(effectiveMessages, options, cancellationToken).ConfigureAwait(false)) + { + buffered.Add(update); + } + + var response = buffered.ToChatResponse(); + bool changed = await EmitPostModelCallAsync(state, modelId, response, cancellationToken).ConfigureAwait(false); + + // No-divergence rule: a transformed response re-derives the released updates + // from the verdicted content; otherwise the buffered updates replay as-is. + foreach (var update in changed ? response.ToChatResponseUpdates() : (IEnumerable)buffered) + { + yield return update; + } + } + + private AgentHooksRunState RequireRunState() + { + var state = AgentHooksRunState.Current + ?? throw new InvalidOperationException( + "The agent-hooks chat seam was invoked without an active agent-hooks run. The agent-hooks " + + "decorators must be installed as one unit by the agent-hooks factory; do not extract or reuse " + + "the inner chat client outside the agent it guards."); + + if (!ReferenceEquals(state.Configuration, this._configuration)) + { + throw new InvalidOperationException( + "The agent-hooks chat seam found an active agent-hooks run owned by a different agent-hooks " + + "installation. Nesting one agent-hooks-guarded agent's chat client inside another guarded agent " + + "is not supported: emissions would silently bind to the wrong emitter."); + } + + return state; + } + + /// + /// Project the per-call effective tool set into the spec's optional + /// pre_model_call tools field ({name, description?}). + /// + /// + /// This is the completed set for the call — including tools registered dynamically + /// by context providers during run preparation — whereas agent_startup's + /// tools_registered is the run-start snapshot. + /// + private static System.Text.Json.Nodes.JsonArray? ProjectTools(ChatOptions? options) + { + if (options?.Tools is not { Count: > 0 } tools) + { + return null; + } + + var projected = new System.Text.Json.Nodes.JsonArray(); + foreach (var tool in tools) + { + var entry = new System.Text.Json.Nodes.JsonObject { ["name"] = tool.Name }; + if (!string.IsNullOrEmpty(tool.Description)) + { + entry["description"] = tool.Description; + } + + projected.Add((System.Text.Json.Nodes.JsonNode)entry); + } + + return projected; + } + + private string ResolveModelId(ChatOptions? options) => + options?.ModelId + ?? this.GetService()?.DefaultModelId + ?? this.InnerClient.GetType().Name; + + private async Task> EmitPreModelCallAsync( + AgentHooksRunState state, string modelId, IEnumerable messages, ChatOptions? options, CancellationToken cancellationToken) + { + List messageList = [.. messages]; + try + { + // Projection and write-back run inside the guarded block: a failure there is + // an enforcement-layer failure, so this run's gated persistence is refused + // (fail closed) before the exception fails the run. + var before = ModelRequestCodec.ToWire(messageList); + var outcome = await state.Emitter.EmitAsync( + state.Builder.PreModelCall(modelId, before, ProjectTools(options)), cancellationToken).ConfigureAwait(false); + return ModelRequestCodec.WriteBack(messageList, before, outcome.Target) ?? messageList; + } + catch (OperationCanceledException) + { + throw; + } + catch (Exception) + { + state.Denied = true; + throw; + } + } + + /// Emit post_model_call over the assembled response; apply transforms. Returns whether the response changed. + private static async Task EmitPostModelCallAsync( + AgentHooksRunState state, string modelId, ChatResponse response, CancellationToken cancellationToken) + { + try + { + // Projection and write-back run inside the guarded block (see + // EmitPreModelCallAsync). §6.1 on deny: the denied response must not be + // incorporated; downstream persistence (the per-service-call persister sits + // above this seam) never runs, and any later gated persist for this run is + // refused via the denied flag. + var before = ModelResponseCodec.ToWire(response); + var outcome = await state.Emitter.EmitAsync( + state.Builder.PostModelCall( + response.ModelId ?? modelId, + before["content"]?.DeepClone(), + (System.Text.Json.Nodes.JsonArray)before["tool_calls"]!.DeepClone(), + Wire.FinishReasonString(response.FinishReason), + Wire.UsageToWire(response.Usage), + response.ResponseId), + cancellationToken).ConfigureAwait(false); + return ModelResponseCodec.WriteBack(response, before, outcome.Target); + } + catch (OperationCanceledException) + { + throw; + } + catch (Exception) + { + state.Denied = true; + throw; + } + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksConfiguration.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksConfiguration.cs new file mode 100644 index 00000000000..5380db00dfc --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksConfiguration.cs @@ -0,0 +1,43 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using AgentHooks; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// The configuration shared by all seams composed by one factory call. +/// +/// +/// Reference identity of this object is the ownership token: the chat and tool seams +/// only bind to an ambient run state created by their own factory call, so nesting two +/// agent-hooks-enabled agents can never silently misroute emissions. +/// +internal sealed class AgentHooksConfiguration +{ + public required IReadOnlyList> Interceptors { get; init; } + + public IApprovalResolver? Resolver { get; init; } + + public EnforcementMode Mode { get; init; } = EnforcementMode.Enforce; + + public CompositionConfig? Composition { get; init; } + + public IdentityProvider? IdentityProvider { get; init; } + + public TimeSpan? Timeout { get; init; } + + public Action? RecordSink { get; init; } + + /// Host-owned session: when set, the middleware emits only the per-run points on this emitter. + public InterceptionEmitter? Emitter { get; init; } + + /// Host-owned session: the builder matching . + public AgentContextBuilder? Builder { get; init; } + + /// The logger for enforcement diagnostics (swallowed best-effort failures are tracked here). + public ILogger Logger { get; set; } = NullLogger.Instance; +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksFunctionMiddleware.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksFunctionMiddleware.cs new file mode 100644 index 00000000000..1a9d2f19dae --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksFunctionMiddleware.cs @@ -0,0 +1,241 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Globalization; +using System.Linq; +using System.Text.Json.Nodes; +using System.Threading; +using System.Threading.Tasks; +using AgentHooks; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// Tool bracket: emits pre_tool_call and post_tool_call around each +/// host-executed function invocation. +/// +/// +/// +/// Installed by the agent-hooks factory through the framework's function-invocation +/// middleware seam (), +/// which wraps every so this callback brackets the invocation. +/// +/// +/// A policy deny blocks the tool call — the tool is not executed (or its result is +/// discarded) and a tool-error payload is surfaced to the model so the agent loop can +/// continue, per the spec's block-propagation rules. A host_error:* deny (the +/// enforcement layer itself failed) additionally halts the run: the loop is terminated +/// via — the only loud escape from the +/// function-invocation loop, which converts thrown exceptions into tool errors and keeps +/// running (fail open) — and the agent seam rethrows the failure at the run boundary. +/// +/// +/// Approval flows pass through structurally: the framework requests tool approvals +/// before ever invoking the wrapped function, so an unapproved tool never reaches this +/// seam, and the approved replay enters through pre_tool_call when it actually +/// executes. +/// +/// +/// Known limitation (service-side tool execution): tools executed by the model provider +/// itself never pass through the function-invocation seam, so pre_tool_call / +/// post_tool_call cannot intercept them. Their calls and outputs are surfaced in +/// the post_model_call content projection, where interceptors can observe and +/// deny/transform the response that carries them. +/// +/// +internal static class AgentHooksFunctionMiddleware +{ + internal static Func>, CancellationToken, ValueTask> CreateCallback( + AgentHooksConfiguration configuration) => + (agent, context, next, cancellationToken) => InvokeAsync(configuration, context, next, cancellationToken); + + private static async ValueTask InvokeAsync( + AgentHooksConfiguration configuration, + FunctionInvocationContext context, + Func> next, + CancellationToken cancellationToken) + { + var state = AgentHooksRunState.Current; + if (state is null) + { + // No run state means the agent seam never ran (the guarded agent's inner + // pieces were extracted and reused). The tool is never dispatched and the + // loop is stopped — throwing here would be converted into a tool error by + // the function-invocation loop and the run would continue unguarded. + context.Terminate = true; + return new JsonObject + { + ["error"] = "The agent-hooks function seam was invoked without an active agent-hooks run. " + + "The agent-hooks decorators must be installed as one unit by the agent-hooks factory.", + }; + } + + if (!ReferenceEquals(state.Configuration, configuration)) + { + // A different installation owns the innermost run state (nested guarded + // agents sharing seams): binding to it would silently misroute emissions. + return HaltEnforcementFailure( + state, + context, + new InvalidOperationException( + "The agent-hooks function seam found an active agent-hooks run owned by a different " + + "agent-hooks installation. Nesting one guarded agent's tools inside another guarded agent " + + "is not supported."), + "pre_tool_call"); + } + + string callId = context.CallContent?.CallId is { Length: > 0 } id + ? id + : Guid.NewGuid().ToString("N", CultureInfo.InvariantCulture); + string name = context.Function?.Name ?? context.CallContent?.Name ?? "unknown"; + + JsonObject args; + try + { + // The projection runs inside the guarded block: a projection failure (e.g. a + // poisoned argument value whose serialization throws) is an enforcement-layer + // failure and must halt the run, not crash it with a trail gap. + args = ToolArgumentsCodec.ToWire(context.Arguments); + var outcome = await state.Emitter.EmitAsync(state.Builder.PreToolCall(callId, name, args), cancellationToken).ConfigureAwait(false); + args = ToolArgumentsCodec.WriteBack(context.Arguments, args, outcome.Target, out var merged); + if (merged is not null) + { + // The transform rewrote (some of) the arguments: execute the approved + // values, keeping untouched keys' original native values. The argument + // dictionary is mutated in place to preserve any framework-managed + // context riding on it. + foreach (var key in context.Arguments.Keys.ToArray()) + { + if (!merged.ContainsKey(key)) + { + _ = context.Arguments.Remove(key); + } + } + + foreach (var (key, value) in merged) + { + context.Arguments[key] = value; + } + } + } + catch (InterceptionBlockedException exception) + { + // §6.2: the tool is not dispatched and no post_tool_call is emitted. + return Block(state, context, exception, "pre_tool_call"); + } + catch (OperationCanceledException) + { + throw; + } + catch (Exception exception) + { + return HaltEnforcementFailure(state, context, exception, "pre_tool_call"); + } + + object? result; + try + { + result = await next(context, cancellationToken).ConfigureAwait(false); + } + catch (OperationCanceledException) + { + throw; + } + catch (Exception invokeException) + { + // The invocation errored: the contract still brackets it (is_error=true). + // Only the exception type name crosses the boundary (spec §6.3/§14). + try + { + _ = await state.Emitter.EmitAsync( + state.Builder.PostToolCall(callId, name, args, JsonValue.Create(invokeException.GetType().Name), isError: true), + cancellationToken).ConfigureAwait(false); + } + catch (InterceptionBlockedException blocked) + { + // A policy deny over an already-errored call changes nothing (the + // result is discarded either way); a host error still halts the run. + MaybeHalt(state, context, blocked, "post_tool_call"); + } + catch (OperationCanceledException) + { + throw; + } + catch (Exception emitException) + { + _ = HaltEnforcementFailure(state, context, emitException, "post_tool_call"); + } + + throw; + } + + try + { + var value = ToolResultCodec.ToWire(result); + var outcome = await state.Emitter.EmitAsync(state.Builder.PostToolCall(callId, name, args, value), cancellationToken).ConfigureAwait(false); + return ToolResultCodec.WriteBack(result, value, outcome.Target); + } + catch (InterceptionBlockedException exception) + { + // §6.1: the result must be discarded as if the call had errored. + return Block(state, context, exception, "post_tool_call"); + } + catch (OperationCanceledException) + { + throw; + } + catch (Exception exception) + { + return HaltEnforcementFailure(state, context, exception, "post_tool_call"); + } + } + + /// Enforce a tool-seam deny: surface a tool error and, on host errors, halt the run. + private static JsonObject Block( + AgentHooksRunState state, FunctionInvocationContext context, InterceptionBlockedException exception, string point) + { + var payload = new JsonObject + { + ["error"] = $"Tool call blocked by agent-hooks at {point}.", + ["reason"] = exception.Result.Verdict.Reason ?? "deny", + }; + if (exception.Result.Verdict.Message is string message) + { + payload["message"] = message; + } + + MaybeHalt(state, context, exception, point); + return payload; + } + + private static void MaybeHalt( + AgentHooksRunState state, FunctionInvocationContext context, InterceptionBlockedException exception, string point) + { + if (exception.Result.Verdict.Reason?.StartsWith("host_error:", StringComparison.Ordinal) is true) + { + // The enforcement layer itself failed (interceptor crash/timeout, invalid + // context): continuing the loop would run unguarded. Halt the run; the + // agent seam rethrows the block to the caller at the run boundary. + state.Halted = exception; + context.Terminate = true; + } + } + + /// + /// Route an unexpected failure inside the enforcement layer through the fail-closed + /// halt path: the function-invocation loop converts thrown exceptions into + /// tool-error results and keeps running, so for a failure of the enforcement layer + /// itself that would fail open. Instead the loop is stopped via + /// and the agent seam rethrows the + /// failure at the run boundary. + /// + private static JsonObject HaltEnforcementFailure( + AgentHooksRunState state, FunctionInvocationContext context, Exception exception, string point) + { + string message = $"agent-hooks {point} enforcement failed: {exception.GetType().Name}"; + state.Halted = exception as InvalidOperationException ?? new InvalidOperationException(message, exception); + context.Terminate = true; + return new JsonObject { ["error"] = message }; + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksGatingAIContextProvider.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksGatingAIContextProvider.cs new file mode 100644 index 00000000000..1ebcf592465 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksGatingAIContextProvider.cs @@ -0,0 +1,70 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Threading; +using System.Threading.Tasks; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// Wraps an of the guarded agent so its run-end durable +/// writes defer behind the run's output verdict, mirroring the history gating. +/// +internal sealed class AgentHooksGatingAIContextProvider : AIContextProvider +{ + private readonly AIContextProvider _inner; + + /// The installation this wrapper belongs to (ownership check for re-wrapping foreign wrappers). + internal AgentHooksConfiguration Configuration { get; } + private readonly bool _endOfRunDeferral; + + internal AgentHooksGatingAIContextProvider( + AIContextProvider inner, AgentHooksConfiguration configuration, bool perServiceCallPersistence) + { + this._inner = inner; + this.Configuration = configuration; + this._endOfRunDeferral = !perServiceCallPersistence; + } + + /// + public override IReadOnlyList StateKeys => this._inner.StateKeys; + + /// + protected override ValueTask InvokingCoreAsync(InvokingContext context, CancellationToken cancellationToken = default) => + this._inner.InvokingAsync(context, cancellationToken); + + /// + protected override ValueTask InvokedCoreAsync(InvokedContext context, CancellationToken cancellationToken = default) + { + if (context.InvokeException is not null) + { + // See the history wrapper: failure notifications pass through for cleanup, + // redacted (empty request messages) once a run-level deny or halt stands. + var state = AgentHooksRunState.Current; + if (state is not null && ReferenceEquals(state.Configuration, this.Configuration) && + (state.Denied || state.Halted is not null)) + { + context = new InvokedContext(context.Agent, context.Session, [], context.InvokeException); + } + + return this._inner.InvokedAsync(context, cancellationToken); + } + + return PersistenceGating.GateAsync( + this.Configuration, + this._endOfRunDeferral, + (state, ct) => + { + var effective = state?.VerdictedResponseMessages is { } verdicted && context.ResponseMessages is not null + ? new InvokedContext(context.Agent, context.Session, context.RequestMessages, verdicted) + : context; + return this._inner.InvokedAsync(effective, ct); + }, + cancellationToken); + } + + /// + public override object? GetService(Type serviceType, object? serviceKey = null) => + base.GetService(serviceType, serviceKey) ?? this._inner.GetService(serviceType, serviceKey); +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksGatingChatHistoryProvider.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksGatingChatHistoryProvider.cs new file mode 100644 index 00000000000..66f631c4d02 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksGatingChatHistoryProvider.cs @@ -0,0 +1,82 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// Wraps the guarded agent's so that durable history +/// writes obey verdict-before-durability: end-of-run writes defer behind the run's +/// output verdict (flushed post-transform, dropped on deny), while writes already +/// covered by their own post_model_call verdict (per-service-call persistence) +/// run inline. +/// +internal sealed class AgentHooksGatingChatHistoryProvider : ChatHistoryProvider +{ + private readonly ChatHistoryProvider _inner; + + /// The installation this wrapper belongs to (ownership check for re-wrapping foreign wrappers). + internal AgentHooksConfiguration Configuration { get; } + private readonly bool _endOfRunDeferral; + + internal AgentHooksGatingChatHistoryProvider( + ChatHistoryProvider inner, AgentHooksConfiguration configuration, bool perServiceCallPersistence) + { + this._inner = inner; + this.Configuration = configuration; + this._endOfRunDeferral = !perServiceCallPersistence; + } + + /// + public override IReadOnlyList StateKeys => this._inner.StateKeys; + + /// + protected override ValueTask> InvokingCoreAsync(InvokingContext context, CancellationToken cancellationToken = default) => + this._inner.InvokingAsync(context, cancellationToken); + + /// + protected override ValueTask InvokedCoreAsync(InvokedContext context, CancellationToken cancellationToken = default) + { + if (context.InvokeException is not null) + { + // Failure notifications are cleanup control flow (the default provider + // stores nothing for them) and pass through — but once a run-level deny or + // halt is standing they are REDACTED: the notification would carry the + // denied turn's request messages, which must not reach provider code, while + // providers that release per-run resources on the failure signal must still + // be notified. + var state = AgentHooksRunState.Current; + if (state is not null && ReferenceEquals(state.Configuration, this.Configuration) && + (state.Denied || state.Halted is not null)) + { + context = new InvokedContext(context.Agent, context.Session, [], context.InvokeException); + } + + return this._inner.InvokedAsync(context, cancellationToken); + } + + return PersistenceGating.GateAsync( + this.Configuration, + this._endOfRunDeferral, + (state, ct) => + { + // A deferred (end-of-run) persist substitutes the verdicted response + // messages: for streamed runs the captured context holds the inner + // agent's own pre-verdict message list, and the output transform must + // be what becomes durable. + var effective = state?.VerdictedResponseMessages is { } verdicted && context.ResponseMessages is not null + ? new InvokedContext(context.Agent, context.Session, context.RequestMessages, verdicted) + : context; + return this._inner.InvokedAsync(effective, ct); + }, + cancellationToken); + } + + /// + public override object? GetService(Type serviceType, object? serviceKey = null) => + base.GetService(serviceType, serviceKey) ?? this._inner.GetService(serviceType, serviceKey); +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksLogMessages.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksLogMessages.cs new file mode 100644 index 00000000000..5965b35dce7 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksLogMessages.cs @@ -0,0 +1,16 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using Microsoft.Extensions.Logging; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// Source-generated log messages for the agent-hooks enforcement. +internal static partial class AgentHooksLogMessages +{ + [LoggerMessage( + EventId = 1, + Level = LogLevel.Warning, + Message = "agent-hooks failed to emit agent_shutdown (reason: {Reason}); the session trail is incomplete.")] + public static partial void LogAgentShutdownEmissionFailed(this ILogger logger, string reason, Exception exception); +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksRunState.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksRunState.cs new file mode 100644 index 00000000000..779cef62339 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/AgentHooksRunState.cs @@ -0,0 +1,67 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Threading; +using AgentHooks; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// Per-run enforcement state shared by the seams via an . +/// +internal sealed class AgentHooksRunState +{ + private static readonly AsyncLocal s_current = new(); + + public AgentHooksRunState(InterceptionEmitter emitter, AgentContextBuilder builder, bool sessionScoped, AgentHooksConfiguration configuration) + { + this.Emitter = emitter; + this.Builder = builder; + this.SessionScoped = sessionScoped; + this.Configuration = configuration; + } + + /// The run state covering the current async flow, if any. + public static AgentHooksRunState? Current + { + get => s_current.Value; + set => s_current.Value = value; + } + + public InterceptionEmitter Emitter { get; } + + public AgentContextBuilder Builder { get; } + + /// Whether the session (and its startup/shutdown boundaries) is host-owned. + public bool SessionScoped { get; } + + public AgentHooksConfiguration Configuration { get; } + + /// + /// Set when the enforcement layer itself failed (interceptor host error at the tool + /// seam, projection bug): the run must not egress; the agent seam rethrows this at + /// the run boundary. + /// + public Exception? Halted { get; set; } + + /// + /// Set when a run-level verdict denied content. Once set, the run's durable + /// persistence is refused fail-closed (denied content never becomes durable, and the + /// denied turn's request messages are not persisted either). + /// + public bool Denied { get; set; } + + /// The gate deferring this run's end-of-run durable writes behind the output verdict. + public RunPersistenceGate Gate { get; } = new(); + + /// + /// The run's final (output-verdicted, post-transform) response messages, set by the + /// agent seam before the gate is flushed. Deferred end-of-run persists substitute + /// these for the response messages they captured, so streamed runs — whose inner + /// agent assembled its own message list before the output verdict existed — persist + /// the verdicted content, never the pre-transform value. + /// + public IList? VerdictedResponseMessages { get; set; } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/PersistenceGating.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/PersistenceGating.cs new file mode 100644 index 00000000000..6d50eb57486 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/PersistenceGating.cs @@ -0,0 +1,58 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Threading; +using System.Threading.Tasks; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// Shared deferral rule for the gating provider wrappers: durable writes issued inside a +/// guarded run wait behind the run's persistence gate until the covering verdict permits +/// the content; once a run-level verdict has denied, writes are refused outright. +/// +internal static class PersistenceGating +{ + /// + /// Defer behind the active run's gate when the ambient run + /// state belongs to ; run it inline otherwise. + /// + /// + /// The wrappers are installed only on the agent the agent-hooks factory itself + /// composed, so nested or sibling agents (which have their own providers) always + /// persist inline at their own run boundaries — no run-identity bookkeeping is + /// needed. Per-service-call persistence never reaches this gate un-verdicted either: + /// the per-service-call persister sits above the agent-hooks chat seam, so its writes + /// are already covered by their own post_model_call verdict and are executed + /// inline here when the end-of-run notification is skipped; a permitted + /// per-service-call write therefore remains durable even if the run's output + /// is later denied — unless a deny is already standing, in which case everything + /// (including the denied turn's request messages) is refused. + /// + public static ValueTask GateAsync( + AgentHooksConfiguration configuration, bool endOfRunDeferral, Func persist, CancellationToken cancellationToken) + { + var state = AgentHooksRunState.Current; + if (state is null || !ReferenceEquals(state.Configuration, configuration)) + { + return persist(null, cancellationToken); + } + + if (state.Denied || state.Halted is not null) + { + // Fail closed: denied content (and the denied turn's request messages) + // never becomes durable. + return default; + } + + if (endOfRunDeferral) + { + // The deferred callback receives the run state at flush time so it can + // substitute the verdicted (post-transform) response messages. + state.Gate.Collect(ct => persist(state, ct)); + return default; + } + + return persist(null, cancellationToken); + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/RunPersistenceGate.cs b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/RunPersistenceGate.cs new file mode 100644 index 00000000000..4e444594e84 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Core/RunPersistenceGate.cs @@ -0,0 +1,62 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Threading; +using System.Threading.Tasks; + +namespace Microsoft.Agents.AI.AgentHooks; + +/// +/// Collects a guarded run's durable persistence side effects so they only execute after +/// the covering verdict permits the content. +/// +/// +/// The .NET equivalent of the Python feature's run persistence gate, radically simplified +/// by construction ownership: the gate is consulted only by the gating provider wrappers +/// that the agent-hooks factory itself installed on its own agent, so nested or sibling +/// agents (which have their own providers) always persist inline at their own run +/// boundaries, with no run-identity bookkeeping. +/// +internal sealed class RunPersistenceGate +{ + private readonly object _lock = new(); + private List>? _pending; + + /// Queue one deferred persistence callback. + public void Collect(Func persist) + { + lock (this._lock) + { + (this._pending ??= []).Add(persist); + } + } + + /// Execute the deferred persistence in order (the covering verdict permitted the content). + public async ValueTask FlushAsync(CancellationToken cancellationToken) + { + List>? pending; + lock (this._lock) + { + pending = this._pending; + this._pending = null; + } + + if (pending is not null) + { + foreach (var persist in pending) + { + await persist(cancellationToken).ConfigureAwait(false); + } + } + } + + /// Discard the deferred persistence (the covering verdict denied the content). + public void Drop() + { + lock (this._lock) + { + this._pending = null; + } + } +} diff --git a/dotnet/src/Microsoft.Agents.AI.AgentHooks/Microsoft.Agents.AI.AgentHooks.csproj b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Microsoft.Agents.AI.AgentHooks.csproj new file mode 100644 index 00000000000..c086b2200f0 --- /dev/null +++ b/dotnet/src/Microsoft.Agents.AI.AgentHooks/Microsoft.Agents.AI.AgentHooks.csproj @@ -0,0 +1,46 @@ + + + + + $(TargetFrameworksCore) + + false + + $(NoWarn);MEAI001;MAAI001 + + + + true + + + + + alpha + + + + + + + + + + + + + + + + Microsoft Agent Framework - Responsible AI Agent-Hooks Protocol Support + Microsoft Agent Framework support for the Agent-Hooks interception protocol (AGENT-HOOKS-0.1, maintained by the Responsible AI project at github.com/responsibleai/agent-hooks): fail-closed policy interception across the agent, model-call and tool-call seams, with transform write-back, buffered streaming and verdict-gated history persistence. + AI, Artificial Intelligence, Agent, SDK, Framework, Agent-Hooks, Responsible AI, Guardrails, Policy, Interception + + + + + + + diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksBoundaryRegressionTests.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksBoundaryRegressionTests.cs new file mode 100644 index 00000000000..c0223baa44a --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksBoundaryRegressionTests.cs @@ -0,0 +1,477 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Threading.Tasks; +using AgentHooks; +using Microsoft.Extensions.AI; +using static Microsoft.Agents.AI.AgentHooks.UnitTests.TestHelpers; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +/// +/// Regressions for the structural boundary with , mined +/// from the review probes: the default history provider must be gated, per-run options +/// must not open bypass routes, and seam-order inversions are rejected loudly. +/// +public class AgentHooksBoundaryRegressionTests +{ + [Theory] + [InlineData(false)] + [InlineData(true)] + public async Task DefaultProviderDeniedOutputNeverBecomesDurableAsync(bool streaming) + { + // Arrange: NO ChatHistoryProvider configured — the zero-config path where the + // agent's implicit default InMemoryChatHistoryProvider must still be gated. + const string Marker = "SECRET-TOKEN-42"; + var client = new MockChatClient().EnqueueText(Marker).EnqueueText("second turn ok"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new ContentDenyGuard(InterceptionPoint.Output, Marker))); + var session = await agent.CreateSessionAsync(); + + // Act: first run is denied at output. + if (streaming) + { + _ = await Assert.ThrowsAsync(async () => + { + await foreach (var _ in agent.RunStreamingAsync(UserMessage("hi"), session)) + { + } + }); + } + else + { + _ = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"), session)); + } + + // Assert: the denied content is not in the serialized session state and does not + // replay to the model on the next run. + var state = await agent.SerializeSessionAsync(session); + Assert.DoesNotContain(Marker, state.GetRawText(), StringComparison.Ordinal); + + _ = await agent.RunAsync(UserMessage("next question"), session); + Assert.True(client.Requests.Count > 1); + Assert.DoesNotContain(client.Requests[1], message => message.Text.Contains(Marker, StringComparison.Ordinal)); + } + + [Fact] + public async Task DefaultProviderPermittedOutputStillPersistsAsync() + { + // Arrange: gating the implicit default must not break normal session history. + var client = new MockChatClient().EnqueueText("first answer").EnqueueText("second answer"); + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(new AllowGuard())); + var session = await agent.CreateSessionAsync(); + + // Act + _ = await agent.RunAsync(UserMessage("first question"), session); + _ = await agent.RunAsync(UserMessage("second question"), session); + + // Assert: the second request carries the first turn from session history. + Assert.Contains(client.Requests[1], message => message.Text == "first answer"); + } + + [Fact] + public async Task BaseAdditionalPropertiesProviderOverrideIsGatedAsync() + { + // Arrange: the override rides the BASE AgentRunOptions.AdditionalProperties, + // which the agent merges into the chat options with precedence. + var overrideProvider = new RecordingHistoryProvider(); + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Deny("egress_blocked")))); + var session = await agent.CreateSessionAsync(); + var runOptions = new ChatClientAgentRunOptions { AdditionalProperties = [] }; + runOptions.AdditionalProperties!.Add(overrideProvider); + + // Act + _ = await Assert.ThrowsAsync( + () => agent.RunAsync(UserMessage("hi"), session, runOptions)); + + // Assert + Assert.Empty(overrideProvider.Stored); + } + + [Fact] + public async Task BaseOverrideDisplacingWrappedChatOptionsOverrideIsGatedAsync() + { + // Arrange: the same provider on BOTH dictionaries — the base-level entry + // displaces the ChatOptions-level one during the agent's options merge, so both + // must be wrapped. + var overrideProvider = new RecordingHistoryProvider(); + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Deny("egress_blocked")))); + var session = await agent.CreateSessionAsync(); + var runOptions = new ChatClientAgentRunOptions + { + ChatOptions = new ChatOptions { AdditionalProperties = [] }, + AdditionalProperties = [], + }; + runOptions.ChatOptions.AdditionalProperties!.Add(overrideProvider); + runOptions.AdditionalProperties!.Add(overrideProvider); + + // Act + _ = await Assert.ThrowsAsync( + () => agent.RunAsync(UserMessage("hi"), session, runOptions)); + + // Assert + Assert.Empty(overrideProvider.Stored); + } + + [Fact] + public async Task PlainAgentRunOptionsProviderOverrideIsGatedAsync() + { + // Arrange: a plain AgentRunOptions (converted to ChatClientAgentRunOptions by + // the tool-seam decorator, preserving AdditionalProperties) must be guarded too. + var overrideProvider = new RecordingHistoryProvider(); + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Deny("egress_blocked")))); + var session = await agent.CreateSessionAsync(); + var runOptions = new AgentRunOptions { AdditionalProperties = [] }; + runOptions.AdditionalProperties!.Add(overrideProvider); + + // Act + _ = await Assert.ThrowsAsync( + () => agent.RunAsync(UserMessage("hi"), session, runOptions)); + + // Assert + Assert.Empty(overrideProvider.Stored); + } + + [Fact] + public async Task ForeignGatingWrapperOverrideIsRewrappedAsync() + { + // Arrange: a gating wrapper belonging to a DIFFERENT agent-hooks installation + // (e.g. extracted from another guarded agent) is passed as a per-run provider + // override. It runs inline under this run's state — its own gate is not covering + // here — so skipping the re-wrap because "it is already a gating wrapper" would + // let this run's denied history persist straight through it. + var recording = new RecordingHistoryProvider(); + var foreignConfiguration = new AgentHooksConfiguration + { + Interceptors = [new KeyValuePair(null, new AllowGuard())], + }; + var foreignWrapper = new AgentHooksGatingChatHistoryProvider(recording, foreignConfiguration, perServiceCallPersistence: false); + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Deny("egress_blocked")))); + var session = await agent.CreateSessionAsync(); + var runOptions = new ChatClientAgentRunOptions { AdditionalProperties = [] }; + runOptions.AdditionalProperties!.Add(foreignWrapper); + + // Act + _ = await Assert.ThrowsAsync( + () => agent.RunAsync(UserMessage("hi"), session, runOptions)); + + // Assert: the denied run's history never reached the underlying provider. + Assert.Empty(recording.Stored); + } + + [Fact] + public async Task CallerRunOptionsAreNeverMutatedAsync() + { + // Arrange + var overrideProvider = new RecordingHistoryProvider(); + var client = new MockChatClient().EnqueueText("fine"); + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(new AllowGuard())); + var session = await agent.CreateSessionAsync(); + var runOptions = new ChatClientAgentRunOptions { AdditionalProperties = [] }; + runOptions.AdditionalProperties!.Add(overrideProvider); + + // Act + _ = await agent.RunAsync(UserMessage("hi"), session, runOptions); + + // Assert: copy-on-write — the caller's dictionary still holds the original, + // unwrapped provider instance. + _ = runOptions.AdditionalProperties.TryGetValue(out ChatHistoryProvider? stillThere); + Assert.Same(overrideProvider, stillThere); + } + + [Fact] + public async Task ReusedRunOptionsAcrossSequentialRunsWorkAsync() + { + // Arrange: the framework's function-invocation middleware chains its factory + // onto the options instance it receives; forwarding the caller's instance would + // leak that factory into it and trip the rejection on the second run. + var client = new MockChatClient().EnqueueText("one").EnqueueText("two"); + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(new AllowGuard())); + var runOptions = new ChatClientAgentRunOptions(); + + // Act + var first = await agent.RunAsync(UserMessage("a"), null, runOptions); + var second = await agent.RunAsync(UserMessage("b"), null, runOptions); + + // Assert: both runs succeed and the caller's options were never mutated. + Assert.Equal("one", first.Text); + Assert.Equal("two", second.Text); + Assert.Null(runOptions.ChatClientFactory); + } + + [Fact] + public async Task OuterFunctionMiddlewareCompositionIsSupportedAsync() + { + // Arrange: the framework's function-invocation middleware composed OUTSIDE the + // guarded agent (outer position, outer trust). Its per-run factory wraps the + // guarded pipeline instead of replacing it, so it must be allowed — and the + // enforcement's own tool seam must still bracket the invocation. + bool outerMiddlewareSaw = false; + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("done"); + var guard = new AllowGuard(); + var guarded = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(guard), AgentOptionsWithTools(WeatherTool())); + var composed = new AIAgentBuilder(guarded) + .Use(async (agent, context, next, cancellationToken) => + { + outerMiddlewareSaw = true; + return await next(context, cancellationToken); + }) + .Build(); + + // Act + var response = await composed.RunAsync(UserMessage("weather?")); + + // Assert + Assert.Equal("done", response.Text); + Assert.True(outerMiddlewareSaw); + Assert.Contains("pre_tool_call", guard.Points); + Assert.Contains("post_tool_call", guard.Points); + } + + [Fact] + public async Task CallerFactorySmuggledThroughOuterFunctionMiddlewareIsRejectedAsync() + { + // Arrange: a caller-supplied ChatClientFactory hidden behind the framework's + // outer function-invocation middleware (which chains pre-existing factories into + // its own) must still be rejected — the chain is walked. + var client = new MockChatClient().EnqueueText("never"); + var bypassClient = new MockChatClient().EnqueueText("bypassed"); + var guarded = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(new AllowGuard())); + var composed = new AIAgentBuilder(guarded) + .Use((agent, context, next, cancellationToken) => next(context, cancellationToken)) + .Build(); + var runOptions = new ChatClientAgentRunOptions { ChatClientFactory = _ => bypassClient }; + + // Act / Assert + var exception = await Assert.ThrowsAsync( + () => composed.RunAsync(UserMessage("hi"), null, runOptions)); + Assert.Contains("ChatClientFactory", exception.Message, StringComparison.Ordinal); + Assert.Equal(0, bypassClient.CallCount); + } + + [Fact] + public void RederivedUpdatesPreserveTheContinuationToken() + { + // Arrange: a (transformed) background streaming response carries a continuation + // token that ToAgentResponseUpdates() does not project. + var token = ResponseContinuationToken.FromBytes(new byte[] { 1, 2, 3 }); + var response = new AgentResponse(new ChatMessage(ChatRole.Assistant, "[final]")) { ContinuationToken = token }; + + // Act + var updates = AgentHooksAgent.RederiveUpdates(response); + + // Assert: the token rides the last released update, so the response remains resumable. + Assert.Same(token, updates[^1].ContinuationToken); + + // And a message-less response still releases a metadata-only update carrying it. + var empty = new AgentResponse { ContinuationToken = token }; + var emptyUpdates = AgentHooksAgent.RederiveUpdates(empty); + Assert.Same(token, Assert.Single(emptyUpdates).ContinuationToken); + } + + [Fact] + public async Task PerRunChatClientFactoryIsRejectedAsync() + { + // Arrange: a per-run ChatClientFactory would swap out the guarded pipeline (and + // the tool wrapping riding it) — loud rejection, nothing egresses. + var client = new MockChatClient().EnqueueText("never"); + var bypassClient = new MockChatClient().EnqueueText("bypassed-content"); + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(new AllowGuard())); + var runOptions = new ChatClientAgentRunOptions { ChatClientFactory = _ => bypassClient }; + + // Act / Assert + var exception = await Assert.ThrowsAsync( + () => agent.RunAsync(UserMessage("hi"), null, runOptions)); + Assert.Contains("ChatClientFactory", exception.Message, StringComparison.Ordinal); + Assert.Equal(0, client.CallCount); + Assert.Equal(0, bypassClient.CallCount); + } + + [Fact] + public void UseProvidedChatClientAsIsIsRejected() + { + // Arrange: UseProvidedChatClientAsIs signals a fully custom, do-not-touch client + // stack — incompatible with a factory whose job is to decorate that client and + // rely on the agent's default pipeline above it. Honoring it would silently + // change where (and whether) the seams sit. + var client = new MockChatClient(); + + // Act / Assert + var exception = Assert.Throws(() => client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new AllowGuard()), + new ChatClientAgentOptions { UseProvidedChatClientAsIs = true })); + Assert.Contains("UseProvidedChatClientAsIs", exception.Message, StringComparison.Ordinal); + } + + [Fact] + public void SuppliedClientContainingFunctionInvocationIsRejected() + { + // Arrange: a supplied client that already contains a FunctionInvokingChatClient + // would execute tools below the chat seam, before any post_model_call verdict. + var mock = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("done"); + var suppliedWithFicc = new FunctionInvokingChatClient(mock); + + // Act / Assert + var exception = Assert.Throws( + () => suppliedWithFicc.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new AllowGuard()), AgentOptionsWithTools(WeatherTool()))); + Assert.Contains("FunctionInvokingChatClient", exception.Message, StringComparison.Ordinal); + } + + [Fact] + public async Task DeniedRunFailureNotificationsAreRedactedForBothProviderKindsAsync() + { + // Arrange: a post_model_call deny makes the inner agent's run fail, which sends + // failure notifications to BOTH provider kinds. Those notifications must still + // arrive (failure-cleanup contract) but with the denied turn's request messages + // redacted. + var historyProvider = new RecordingHistoryProvider(); + var contextProvider = new RecordingContextProvider(); + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PostModelCall, Verdict.Deny("bad_response"))), + new ChatClientAgentOptions + { + ChatHistoryProvider = historyProvider, + AIContextProviders = [contextProvider], + }); + var session = await agent.CreateSessionAsync(); + + // Act + _ = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"), session)); + + // Assert: both providers were notified of the failure, with zero request messages. + var historyNotification = Assert.Single(historyProvider.FailureNotifications); + Assert.Equal(0, historyNotification.RequestMessageCount); + var contextNotification = Assert.Single(contextProvider.FailureNotifications); + Assert.Equal(0, contextNotification.RequestMessageCount); + Assert.Empty(historyProvider.Stored); + Assert.Empty(contextProvider.StoredResponses); + } + + [Fact] + public async Task OrdinaryFailureNotificationsPassThroughUnredactedAsync() + { + // Arrange: a plain model failure (no verdict involved) — providers must receive + // the full failure notification, request messages included. + var historyProvider = new RecordingHistoryProvider(); + var contextProvider = new RecordingContextProvider(); + var client = new MockChatClient().EnqueueThrow(new TimeoutException("model down")); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new AllowGuard()), + new ChatClientAgentOptions + { + ChatHistoryProvider = historyProvider, + AIContextProviders = [contextProvider], + }); + var session = await agent.CreateSessionAsync(); + + // Act + _ = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"), session)); + + // Assert + var historyNotification = Assert.Single(historyProvider.FailureNotifications); + Assert.IsType(historyNotification.Exception); + Assert.True(historyNotification.RequestMessageCount > 0); + var contextNotification = Assert.Single(contextProvider.FailureNotifications); + Assert.True(contextNotification.RequestMessageCount > 0); + } + + [Fact] + public async Task ContextProviderAddedToolsAreBracketedByTheToolSeamAsync() + { + // Arrange: NO agent-level tools — the only tool is registered dynamically by a + // context provider during run preparation, i.e. after agent_startup was emitted. + bool invoked = false; + var provider = new ToolAddingContextProvider(WeatherTool(_ => invoked = true)); + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("done"); + var guard = new AllowGuard(); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(guard), + new ChatClientAgentOptions { AIContextProviders = [provider] }); + var session = await agent.CreateSessionAsync(); + + // Act + var response = await agent.RunAsync(UserMessage("weather?"), session); + + // Assert (enforcement): the provider-added tool still flows through the guarded + // pipeline and is bracketed by the tool seam like any other tool. + Assert.True(invoked); + Assert.Equal("done", response.Text); + Assert.Contains("pre_tool_call", guard.Points); + Assert.Contains("post_tool_call", guard.Points); + + // Assert (audit): agent_startup's tools_registered is the run-start snapshot + // (empty here — the provider had not run yet), while the pre_model_call tools + // projection carries the completed per-call set including the provider's tool. + var startupTools = Assert.IsType( + guard.Context("agent_startup")["agent_init"]?["tools_registered"]); + Assert.Empty(startupTools); + var callTools = Assert.IsType(guard.Contexts("pre_model_call")[0]["tools"]); + Assert.Contains(callTools, tool => tool?["name"]?.GetValue() == "get_weather"); + } + + [Fact] + public async Task ContextProviderAddedToolDenyBlocksInvocationAsync() + { + // Arrange: a pre_tool_call deny must block a provider-added tool exactly like a + // constructor-registered one. + bool invoked = false; + var provider = new ToolAddingContextProvider(WeatherTool(_ => invoked = true)); + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("recovered"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PreToolCall, Verdict.Deny("tool_blocked"))), + new ChatClientAgentOptions { AIContextProviders = [provider] }); + var session = await agent.CreateSessionAsync(); + + // Act + var response = await agent.RunAsync(UserMessage("weather?"), session); + + // Assert: the tool never ran and the loop continued with the tool-error payload. + Assert.False(invoked); + Assert.Equal("recovered", response.Text); + } + + [Fact] + public async Task PoisonedToolArgumentProjectionFailsClosedAsync() + { + // Arrange: an argument value whose serialization throws. The projection failure + // surfaces at the chat seam (post_model_call projects the tool-call args before + // the function loop ever invokes): the run must fail closed — no tool execution, + // no silent continuation, gated persistence refused, trail closed as error. + bool invoked = false; + var tool = AIFunctionFactory.Create((object p) => { invoked = true; return "ran"; }, "poison_tool"); + var provider = new RecordingHistoryProvider(); + var options = AgentOptionsWithTools(tool); + options.ChatHistoryProvider = provider; + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "poison_tool", new() { ["p"] = new PoisonedValue() }) + .EnqueueText("recovered"); + var guard = new AllowGuard(); + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(guard), options); + var session = await agent.CreateSessionAsync(); + + // Act / Assert + _ = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("go"), session)); + Assert.False(invoked); + Assert.Empty(provider.Stored); + Assert.Equal("error", guard.Context("agent_shutdown")["summary"]?["reason"]?.GetValue()); + } +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksCodecTests.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksCodecTests.cs new file mode 100644 index 00000000000..e22b47da3db --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksCodecTests.cs @@ -0,0 +1,262 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text.Json.Nodes; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +public class AgentHooksCodecTests +{ + [Fact] + public void WireEqualityDistinguishesBoolFromNumber() + { + // Assert + Assert.False(Wire.WireEquals(JsonValue.Create(true), JsonValue.Create(1))); + Assert.False(Wire.WireEquals(JsonValue.Create(false), JsonValue.Create(0))); + Assert.True(Wire.WireEquals(JsonValue.Create(1), JsonValue.Create(1))); + } + + [Fact] + public void InputCodecMapsRolesOntoTheSpecEnum() + { + // Arrange / Act / Assert: user and system pass through; anything else is external. + Assert.Equal("user", Wire.InputRole(ChatRole.User)); + Assert.Equal("system", Wire.InputRole(ChatRole.System)); + Assert.Equal("external", Wire.InputRole(ChatRole.Assistant)); + Assert.Equal("external", Wire.InputRole(ChatRole.Tool)); + } + + [Fact] + public void ToolArgumentsCodecMergesOnlyChangedKeys() + { + // Arrange: a non-JSON-native original value that a faithful projection cannot + // round-trip; the transform touches a different key. + var opaque = new byte[] { 1, 2, 3 }; + var arguments = new AIFunctionArguments { ["location"] = "Paris", ["blob"] = opaque }; + var before = ToolArgumentsCodec.ToWire(arguments); + var after = (JsonObject)before.DeepClone(); + after["location"] = "Berlin"; + + // Act + _ = ToolArgumentsCodec.WriteBack(arguments, before, after, out var merged); + + // Assert: only the changed key was taken from the wire; the untouched key keeps + // its original native value by identity. + Assert.NotNull(merged); + Assert.Equal("Berlin", (merged!["location"] as JsonNode)?.GetValue()); + Assert.Same(opaque, merged["blob"]); + } + + [Fact] + public void ToolArgumentsCodecUntouchedTargetIsANoOp() + { + // Arrange + var arguments = new AIFunctionArguments { ["location"] = "Paris" }; + var before = ToolArgumentsCodec.ToWire(arguments); + + // Act + _ = ToolArgumentsCodec.WriteBack(arguments, before, before.DeepClone(), out var merged); + + // Assert + Assert.Null(merged); + } + + [Fact] + public void ToolArgumentsCodecNonObjectTransformFailsClosed() + { + // Arrange + var arguments = new AIFunctionArguments { ["location"] = "Paris" }; + var before = ToolArgumentsCodec.ToWire(arguments); + + // Act / Assert + _ = Assert.Throws( + () => ToolArgumentsCodec.WriteBack(arguments, before, JsonValue.Create("nope"), out _)); + } + + [Fact] + public void MessageListWriteBackMatchesByIdentityNotPosition() + { + // Arrange: three originals; the transform removes the middle one. + List originals = + [ + new ChatMessage(ChatRole.User, "first"), + new ChatMessage(ChatRole.User, "second"), + new ChatMessage(ChatRole.User, "third"), + ]; + List before = [.. originals.Select(Wire.MessageToWire)]; + var after = new JsonArray(before[0].DeepClone(), before[2].DeepClone()); + + // Act + var result = Wire.WriteBackMessageList(originals, before, after, "test"); + + // Assert: removal did not shift content onto the wrong original. + Assert.Equal(2, result.Count); + Assert.Same(originals[0], result[0]); + Assert.Same(originals[2], result[1]); + Assert.Equal("third", result[1].Text); + } + + [Fact] + public void ModelResponseCodecSurfacesHostedToolCallsInContent() + { + // Arrange: one host-executed call and one service-executed (informational) call. + var hostCall = new FunctionCallContent("call-1", "local_tool", new Dictionary()); + var hostedCall = new FunctionCallContent("call-2", "hosted_tool", new Dictionary()) + { + InformationalOnly = true, + }; + var response = new ChatResponse(new ChatMessage(ChatRole.Assistant, [hostCall, hostedCall])); + + // Act + var wire = ModelResponseCodec.ToWire(response); + + // Assert: the host-executed call rides tool_calls; the hosted call is part of + // the response content, where it stays interceptable. + var calls = Assert.IsType(wire["tool_calls"]); + Assert.Single(calls); + Assert.Equal("local_tool", calls[0]?["name"]?.GetValue()); + Assert.Contains("hosted_tool", wire["content"]?.ToJsonString(), StringComparison.Ordinal); + } + + [Theory] + [InlineData("""[{"id":"","name":"tool","args":{}}]""")] // empty id + [InlineData("""[{"id":1,"name":"tool","args":{}}]""")] // non-string id + [InlineData("""[{"name":"tool","args":{}}]""")] // missing id + [InlineData("""[{"id":"c1","name":"","args":{}}]""")] // empty name + [InlineData("""[{"id":"c1","args":{}}]""")] // missing name + [InlineData("""[{"id":"c1","name":"tool"}]""")] // missing args + [InlineData("""[{"id":"c1","name":"tool","args":"nope"}]""")] // non-object args + [InlineData("""[{"id":"c1","name":"a","args":{}},{"id":"c1","name":"b","args":{}}]""")] // duplicate ids + public void ToolCallTransformValidationFailsClosedOnMalformedShapes(string toolCallsJson) + { + // Arrange: a transform that adds/reshapes tool calls with an invalid shape must + // fail closed instead of producing malformed native calls. + var response = new ChatResponse(new ChatMessage(ChatRole.Assistant, "text")); + var before = ModelResponseCodec.ToWire(response); + var after = (JsonObject)before.DeepClone(); + after["tool_calls"] = JsonNode.Parse(toolCallsJson); + + // Act / Assert + _ = Assert.Throws(() => ModelResponseCodec.WriteBack(response, before, after)); + } + + [Fact] + public void MessageListWriteBackDefaultsMissingRoleToUserMatchingPython() + { + // Arrange: the merged Python codec defaults a missing per-message role to "user" + // in message-list write-backs (str(item.get("role") or "user")); this port + // mirrors that documented behavior exactly. + List originals = [new ChatMessage(ChatRole.User, "original")]; + List before = [.. originals.Select(Wire.MessageToWire)]; + var after = new JsonArray(new JsonObject { ["content"] = "rewritten" }); + + // Act + var result = Wire.WriteBackMessageList(originals, before, after, "test"); + + // Assert: the role-less entry adopted the user default, so the original user + // message was mutated in place rather than replaced. + Assert.Same(originals[0], Assert.Single(result)); + Assert.Equal("rewritten", result[0].Text); + Assert.Equal(ChatRole.User, result[0].Role); + } + + [Fact] + public void ResponseContentWriteBackDefaultsMissingRoleToAssistantMatchingPython() + { + // Arrange: the merged Python response codec defaults a missing role to + // "assistant" (str(wire_message.get("role") or "assistant")); mirrored here. + var response = new ChatResponse(new ChatMessage(ChatRole.Assistant, "original")); + var before = ModelResponseCodec.ToWire(response); + var after = (JsonObject)before.DeepClone(); + after["content"] = new JsonArray(new JsonObject { ["content"] = "rewritten" }); + + // Act + _ = ModelResponseCodec.WriteBack(response, before, after); + + // Assert + var message = Assert.Single(response.Messages); + Assert.Equal(ChatRole.Assistant, message.Role); + Assert.Equal("rewritten", message.Text); + } + + [Fact] + public void ToolResultCodecUntouchedTargetKeepsOriginalIdentity() + { + // Arrange + var original = new Dictionary { ["value"] = 42 }; + var wire = ToolResultCodec.ToWire(original); + + // Act + var result = ToolResultCodec.WriteBack(original, wire, wire?.DeepClone()); + + // Assert + Assert.Same(original, result); + } + + [Fact] + public void ToolResultCodecPreservesTextContentWrappers() + { + // Arrange: the canonical single-text-content result shape. + List original = [new TextContent("raw")]; + var wire = ToolResultCodec.ToWire(original); + + // Act + var result = ToolResultCodec.WriteBack(original, wire, JsonValue.Create("clean")); + + // Assert + var contents = Assert.IsType>(result); + Assert.Equal("clean", Assert.IsType(contents[0]).Text); + } + + [Fact] + public void OutputCodecUntouchedTargetIsANoOp() + { + // Arrange + var message = new ChatMessage(ChatRole.Assistant, "answer"); + var response = new AgentResponse(message); + var before = OutputCodec.ToWire(response); + + // Act + bool changed = OutputCodec.WriteBack(response, before, new JsonObject { ["content"] = before?.DeepClone() }); + + // Assert + Assert.False(changed); + Assert.Same(message, response.Messages.Single()); + } + + [Fact] + public void OutputCodecUnsupportedTransformFailsClosed() + { + // Arrange + var response = new AgentResponse(new ChatMessage(ChatRole.Assistant, "answer")); + var before = OutputCodec.ToWire(response); + + // Act / Assert: a non-object output target cannot be written back. + _ = Assert.Throws( + () => OutputCodec.WriteBack(response, before, JsonValue.Create(42))); + } + + [Fact] + public void WireToContentsDecodesRichContentAndFailsClosedOnGarbage() + { + // Arrange + var text = Wire.WireToContents(JsonValue.Create("plain"), "test"); + + // Assert: strings decode as text content. + Assert.Equal("plain", Assert.IsType(Assert.Single(text)).Text); + + // Assert: a round-tripped rich content object decodes back to its type. + var image = new DataContent(new byte[] { 1, 2, 3 }, "image/png"); + var projected = Wire.ContentsToWire([new TextContent("t"), image]); + var decoded = Wire.WireToContents(projected, "test"); + Assert.Equal(2, decoded.Count); + _ = Assert.IsType(decoded[1]); + + // Assert: an unsupported item fails closed instead of being dropped. + _ = Assert.Throws( + () => Wire.WireToContents(new JsonArray(new JsonObject { ["no_type"] = "x" }), "test")); + } +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksEnforcementTests.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksEnforcementTests.cs new file mode 100644 index 00000000000..0ad653a56f7 --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksEnforcementTests.cs @@ -0,0 +1,694 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Concurrent; +using System.Collections.Generic; +using System.Linq; +using System.Text.Json.Nodes; +using System.Threading; +using System.Threading.Tasks; +using AgentHooks; +using Microsoft.Extensions.AI; +using static Microsoft.Agents.AI.AgentHooks.UnitTests.TestHelpers; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +public class AgentHooksEnforcementTests +{ + // ------------------------------------------------------------------------- + // Session shape and projections + // ------------------------------------------------------------------------- + + [Fact] + public async Task FullToolRunEmitsCompleteOrderedSessionAsync() + { + // Arrange + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("It is sunny."); + var guard = new AllowGuard(); + var records = new ConcurrentQueue(); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(guard) { RecordSink = records.Enqueue }, + AgentOptionsWithTools(WeatherTool())); + + // Act + var response = await agent.RunAsync(UserMessage("weather in paris?")); + + // Assert + Assert.Equal("It is sunny.", response.Text); + Assert.Equal( + [ + "agent_startup", "input", + "pre_model_call", "post_model_call", + "pre_tool_call", "post_tool_call", + "pre_model_call", "post_model_call", + "output", "agent_shutdown", + ], + guard.Points); + Assert.All(records, record => Assert.Equal(records.First().SessionId, record.SessionId)); + } + + [Fact] + public async Task InputProjectionIsFaithfulAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("hi"); + var guard = new AllowGuard(); + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(guard)); + + // Act + _ = await agent.RunAsync(UserMessage("hello agent")); + + // Assert: a single plain-text message projects as its content string. + var input = guard.Context("input"); + Assert.Equal("hello agent", input["input"]?["content"]?.GetValue()); + Assert.Equal("user", input["input"]?["role"]?.GetValue()); + var startup = guard.Context("agent_startup"); + Assert.NotNull(startup["agent_init"]?["tools_registered"]); + } + + [Fact] + public async Task RichContentIsPreservedInProjectionsAsync() + { + // Arrange + var image = new DataContent(new byte[] { 1, 2, 3 }, "image/png"); + var client = new MockChatClient() + .EnqueueResponse(new ChatResponse(new ChatMessage(ChatRole.Assistant, [new TextContent("look"), image]))); + var guard = new AllowGuard(); + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(guard)); + + // Act + var response = await agent.RunAsync(UserMessage("show me")); + + // Assert: rich content is projected as content objects, not flattened to text, + // and the untouched response keeps the original content instances. + var output = guard.Context("output"); + var parts = Assert.IsType(output["output"]?["content"]); + var contentList = Assert.IsType(parts[0]?["content"]); + Assert.Equal(2, contentList.Count); + Assert.Same(image, response.Messages.SelectMany(m => m.Contents).OfType().Single()); + } + + [Fact] + public async Task ToolCallsRideToolCallsProjectionAsync() + { + // Arrange + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("done"); + var guard = new AllowGuard(); + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(guard), AgentOptionsWithTools(WeatherTool())); + + // Act + _ = await agent.RunAsync(UserMessage("weather?")); + + // Assert: the host-executed call is in tool_calls with faithful args. + var postModel = guard.Contexts("post_model_call")[0]; + var calls = Assert.IsType(postModel["response"]?["tool_calls"]); + Assert.Equal("get_weather", calls[0]?["name"]?.GetValue()); + Assert.Equal("Paris", calls[0]?["args"]?["location"]?.GetValue()); + } + + // ------------------------------------------------------------------------- + // Deny before execution, per seam + // ------------------------------------------------------------------------- + + [Fact] + public async Task InputDenyBlocksRunBeforeModelCallAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("never"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Input, Verdict.Deny("blocked_input")))); + + // Act / Assert + var exception = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"))); + Assert.Equal("blocked_input", exception.Result.Verdict.Reason); + Assert.Equal(0, client.CallCount); + } + + [Fact] + public async Task PreModelCallDenyBlocksModelDispatchAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("never"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PreModelCall, Verdict.Deny("no_model")))); + + // Act / Assert + _ = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"))); + Assert.Equal(0, client.CallCount); + } + + [Fact] + public async Task PostModelCallDenyDiscardsResponseAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PostModelCall, Verdict.Deny("bad_response")))); + + // Act / Assert + _ = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"))); + Assert.Equal(1, client.CallCount); + } + + [Fact] + public async Task PreToolCallDenyBlocksToolAndContinuesLoopAsync() + { + // Arrange + bool invoked = false; + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("recovered"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PreToolCall, Verdict.Deny("tool_blocked"))), + AgentOptionsWithTools(WeatherTool(_ => invoked = true))); + + // Act + var response = await agent.RunAsync(UserMessage("weather?")); + + // Assert: the tool never ran, the model saw a tool-error payload, the loop continued. + Assert.False(invoked); + Assert.Equal("recovered", response.Text); + var secondRequest = client.Requests[1]; + var result = secondRequest.SelectMany(m => m.Contents).OfType().Single(); + string serialized = System.Text.Json.JsonSerializer.Serialize(result.Result); + Assert.Contains("blocked by agent-hooks at pre_tool_call", serialized, StringComparison.Ordinal); + Assert.Contains("tool_blocked", serialized, StringComparison.Ordinal); + } + + [Fact] + public async Task PostToolCallDenyDiscardsResultAsync() + { + // Arrange + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("recovered"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PostToolCall, Verdict.Deny("result_blocked"))), + AgentOptionsWithTools(WeatherTool())); + + // Act + var response = await agent.RunAsync(UserMessage("weather?")); + + // Assert: the result was discarded and replaced with a tool-error payload. + Assert.Equal("recovered", response.Text); + var result = client.Requests[1].SelectMany(m => m.Contents).OfType().Single(); + string serialized = System.Text.Json.JsonSerializer.Serialize(result.Result); + Assert.DoesNotContain("weather:Paris", serialized, StringComparison.Ordinal); + Assert.Contains("result_blocked", serialized, StringComparison.Ordinal); + } + + [Fact] + public async Task OutputDenyBlocksResponseAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Deny("egress_blocked")))); + + // Act / Assert + var exception = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"))); + Assert.Equal("egress_blocked", exception.Result.Verdict.Reason); + } + + // ------------------------------------------------------------------------- + // Transform write-back, per seam + // ------------------------------------------------------------------------- + + [Fact] + public async Task InputTransformWritesBackIntoRunMessagesAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("ok"); + var transform = TransformTarget(new JsonObject { ["content"] = "[clean]", ["role"] = "user" }); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Input, transform))); + + // Act + _ = await agent.RunAsync(UserMessage("dirty input")); + + // Assert: the model received exactly the transformed input. + Assert.Equal("[clean]", client.Requests[0].Last().Text); + } + + [Fact] + public async Task PreModelCallTransformWritesBackIntoRequestAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("ok"); + var transform = TransformTarget(new JsonArray(new JsonObject { ["role"] = "user", ["content"] = "[redacted]" })); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PreModelCall, transform))); + + // Act + _ = await agent.RunAsync(UserMessage("sensitive")); + + // Assert + Assert.Equal("[redacted]", client.Requests[0].Single().Text); + } + + [Fact] + public async Task PreToolCallTransformWritesBackIntoArgumentsAsync() + { + // Arrange + string? seenLocation = null; + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("done"); + var transform = TransformTarget(new JsonObject { ["location"] = "Berlin" }); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PreToolCall, transform)), + AgentOptionsWithTools(WeatherTool(location => seenLocation = location))); + + // Act + _ = await agent.RunAsync(UserMessage("weather?")); + + // Assert: the tool executed the approved (transformed) arguments. + Assert.Equal("Berlin", seenLocation); + } + + [Fact] + public async Task PostToolCallTransformWritesBackIntoResultAsync() + { + // Arrange + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("done"); + var transform = TransformTarget((JsonNode)"weather:[redacted]"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PostToolCall, transform)), + AgentOptionsWithTools(WeatherTool())); + + // Act + _ = await agent.RunAsync(UserMessage("weather?")); + + // Assert: the model saw the transformed result, not the raw one. + var result = client.Requests[1].SelectMany(m => m.Contents).OfType().Single(); + string serialized = System.Text.Json.JsonSerializer.Serialize(result.Result); + Assert.Contains("weather:[redacted]", serialized, StringComparison.Ordinal); + Assert.DoesNotContain("weather:Paris", serialized, StringComparison.Ordinal); + } + + [Fact] + public async Task PostModelCallTransformWritesBackIntoResponseAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("raw"); + var transform = TransformTarget(new JsonObject + { + ["content"] = "[filtered]", + ["tool_calls"] = new JsonArray(), + ["finish_reason"] = "stop", + }); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PostModelCall, transform))); + + // Act + var response = await agent.RunAsync(UserMessage("hi")); + + // Assert + Assert.Equal("[filtered]", response.Text); + } + + [Fact] + public async Task OutputTransformWritesBackIntoResponseAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("raw output"); + var transform = TransformTarget(new JsonObject { ["content"] = "[final]" }); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, transform))); + + // Act + var response = await agent.RunAsync(UserMessage("hi")); + + // Assert + Assert.Equal("[final]", response.Text); + } + + // ------------------------------------------------------------------------- + // Streaming: buffered, zero egress on deny, no divergence + // ------------------------------------------------------------------------- + + [Fact] + public async Task StreamingBuffersUntilAllVerdictsPermitAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("streamed text"); + var guard = new AllowGuard(); + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(guard)); + + // Act + var updates = await CollectAsync(agent.RunStreamingAsync(UserMessage("hi"))); + + // Assert: content egressed and every point was emitted before release. + Assert.Equal("streamed text", string.Concat(updates.Select(u => u.Text))); + Assert.Contains("output", guard.Points); + } + + [Fact] + public async Task StreamingOutputDenyReleasesNothingAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Deny("egress_blocked")))); + + // Act + int released = 0; + var exception = await Assert.ThrowsAsync(async () => + { + await foreach (var _ in agent.RunStreamingAsync(UserMessage("hi"))) + { + released++; + } + }); + + // Assert: the deny surfaced at consumption with zero updates egressed. + Assert.Equal(0, released); + Assert.Equal("egress_blocked", exception.Result.Verdict.Reason); + } + + [Fact] + public async Task StreamingPostModelCallDenyReleasesNothingAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PostModelCall, Verdict.Deny("bad_response")))); + + // Act + int released = 0; + _ = await Assert.ThrowsAsync(async () => + { + await foreach (var _ in agent.RunStreamingAsync(UserMessage("hi"))) + { + released++; + } + }); + + // Assert + Assert.Equal(0, released); + } + + [Fact] + public async Task StreamingOutputTransformRewritesUpdatesAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("raw"); + var transform = TransformTarget(new JsonObject { ["content"] = "[final]" }); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, transform))); + + // Act + var updates = await CollectAsync(agent.RunStreamingAsync(UserMessage("hi"))); + + // Assert: released updates are re-derived from the verdicted response — streamed + // egress never diverges from the transformed content. + Assert.Equal("[final]", string.Concat(updates.Select(u => u.Text))); + } + + // ------------------------------------------------------------------------- + // Error paths: fail closed, complete trail + // ------------------------------------------------------------------------- + + [Fact] + public async Task ToolExceptionIsBracketedWithErrorPostToolCallAsync() + { + // Arrange + var tool = AIFunctionFactory.Create( + new Func(location => throw new InvalidOperationException("tool exploded")), "get_weather"); + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("recovered"); + var guard = new AllowGuard(); + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(guard), AgentOptionsWithTools(tool)); + + // Act + _ = await agent.RunAsync(UserMessage("weather?")); + + // Assert: the errored call is bracketed with is_error=true and only the + // exception type name crosses the boundary. + var postTool = guard.Context("post_tool_call"); + Assert.True(postTool["tool_result"]?["is_error"]?.GetValue()); + Assert.Equal("InvalidOperationException", postTool["tool_result"]?["value"]?.GetValue()); + } + + [Fact] + public async Task InterceptorCrashAtToolSeamFailsClosedAndHaltsRunAsync() + { + // Arrange + bool invoked = false; + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("never"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new CrashingGuard(InterceptionPoint.PreToolCall)), + AgentOptionsWithTools(WeatherTool(_ => invoked = true))); + + // Act / Assert: the crash synthesizes a host_error deny, the tool never runs, + // and the whole run halts instead of continuing unguarded. + var exception = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("weather?"))); + Assert.False(invoked); + Assert.StartsWith("host_error:", exception.Result.Verdict.Reason, StringComparison.Ordinal); + } + + [Fact] + public async Task InterceptorCrashAtInputFailsClosedAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("never"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new CrashingGuard(InterceptionPoint.Input))); + + // Act / Assert + var exception = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"))); + Assert.StartsWith("host_error:", exception.Result.Verdict.Reason, StringComparison.Ordinal); + Assert.Equal(0, client.CallCount); + } + + [Fact] + public async Task DeniedRunStillClosesSessionTrailAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("secret"); + var guard = new AllowGuard(); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(guard).AddInterceptor(new PointGuard(InterceptionPoint.Output, Verdict.Deny("no")))); + + // Act + _ = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"))); + + // Assert: the trail is closed with an error shutdown. + var shutdown = guard.Context("agent_shutdown"); + Assert.Equal("error", shutdown["summary"]?["reason"]?.GetValue()); + } + + // ------------------------------------------------------------------------- + // Concurrency and session scoping + // ------------------------------------------------------------------------- + + [Fact] + public async Task ConcurrentRunsAreIsolatedAsync() + { + // Arrange + var client = new MockChatClient(); + for (int i = 0; i < 8; i++) + { + _ = client.EnqueueText("ok"); + } + + var records = new ConcurrentQueue(); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new AllowGuard()) { RecordSink = records.Enqueue }); + + // Act + await Task.WhenAll(Enumerable.Range(0, 8).Select(i => agent.RunAsync(UserMessage($"run {i}")))); + + // Assert: eight distinct per-run sessions, each with a complete bracket. + var sessions = records.GroupBy(record => record.SessionId).ToList(); + Assert.Equal(8, sessions.Count); + Assert.All(sessions, session => + { + Assert.Contains(session, record => record.InterceptionPoint == InterceptionPoint.AgentStartup); + Assert.Contains(session, record => record.InterceptionPoint == InterceptionPoint.Output); + Assert.Contains(session, record => record.InterceptionPoint == InterceptionPoint.AgentShutdown); + }); + } + + [Fact] + public async Task SequentialRunsGetFreshSessionsAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("one").EnqueueText("two"); + var records = new ConcurrentQueue(); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new AllowGuard()) { RecordSink = records.Enqueue }); + + // Act + _ = await agent.RunAsync(UserMessage("first")); + _ = await agent.RunAsync(UserMessage("second")); + + // Assert + Assert.Equal(2, records.Select(record => record.SessionId).Distinct().Count()); + } + + [Fact] + public async Task HostOwnedSessionSpansRunsAsync() + { + // Arrange + var guard = new AllowGuard(); + var emitter = new InterceptionEmitter().Register(guard); + var builder = new AgentContextBuilder("host-agent", "host", "session-42"); + var client = new MockChatClient().EnqueueText("one").EnqueueText("two"); + var agent = client.AsAIAgentWithAgentHooks(emitter, builder); + + // Act + _ = await agent.RunAsync(UserMessage("first")); + _ = await agent.RunAsync(UserMessage("second")); + + // Assert: only per-run points, one shared session, continuous sequence. + Assert.DoesNotContain("agent_startup", guard.Points); + Assert.DoesNotContain("agent_shutdown", guard.Points); + Assert.All(emitter.Records, record => Assert.Equal("session-42", record.SessionId)); + Assert.Equal(emitter.Records.Count, emitter.Records.Select(record => record.Sequence).Distinct().Count()); + } + + // ------------------------------------------------------------------------- + // Modes and the approval seam + // ------------------------------------------------------------------------- + + [Fact] + public async Task EvaluateOnlyRecordsButNeverBlocksAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("flows"); + var records = new ConcurrentQueue(); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Input, Verdict.Deny("would_block"))) + { + Mode = EnforcementMode.EvaluateOnly, + RecordSink = records.Enqueue, + }); + + // Act + var response = await agent.RunAsync(UserMessage("hi")); + + // Assert: the deny is recorded but the run proceeds untouched. + Assert.Equal("flows", response.Text); + var inputRecord = records.Single(record => record.InterceptionPoint == InterceptionPoint.Input); + Assert.Equal(Decision.Deny, inputRecord.Verdict.Decision); + Assert.True(inputRecord.Proceeds); + } + + [Fact] + public async Task LiftableDenyIsResolvedThroughTheApprovalSeamAsync() + { + // Arrange: a liftable deny plus a resolver that approves it. + var client = new MockChatClient().EnqueueText("approved output"); + var resolver = new ApprovingResolver(); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Escalate("needs_review"))) + { + Resolver = resolver, + }); + + // Act + var response = await agent.RunAsync(UserMessage("hi")); + + // Assert: the approval lifted the deny and the run egressed. + Assert.Equal("approved output", response.Text); + Assert.True(resolver.Consulted); + } + + private sealed class ApprovingResolver : IApprovalResolver + { + public bool Consulted { get; private set; } + + public ValueTask ResolveAsync(ApprovalRequest request, CancellationToken ct = default) + { + this.Consulted = true; + return new(new ApprovalResolution(ApprovalOutcome.Approve, request.ContextIdentity, Verdict.Allow)); + } + } + + // ------------------------------------------------------------------------- + // Misuse fails closed (partial-install impossibility) + // ------------------------------------------------------------------------- + + [Fact] + public async Task ExtractedChatClientFailsClosedOutsideItsRunAsync() + { + // Arrange + var client = new MockChatClient().EnqueueText("never"); + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(new AllowGuard())); + var extracted = agent.GetService(); + Assert.NotNull(extracted); + + // Act / Assert: the chat seam refuses to run without its agent seam's state. + _ = await Assert.ThrowsAsync( + () => extracted!.GetResponseAsync(UserMessage("hi"))); + Assert.Equal(0, client.CallCount); + } + + [Fact] + public async Task ForeignSeamNestingFailsLoudlyAsync() + { + // Arrange: agent B is (mis)built over agent A's guarded chat client, so A's chat + // seam runs inside B's run state. + var client = new MockChatClient().EnqueueText("never"); + var configurationA = new AgentHooksConfiguration + { + Interceptors = [new KeyValuePair(null, new AllowGuard())], + }; + var clientOfA = new AgentHooksChatClient(client, configurationA); + var agentB = clientOfA.AsAIAgentWithAgentHooks(new AgentHooksOptions(new AllowGuard())); + + // Act / Assert + var exception = await Assert.ThrowsAsync(() => agentB.RunAsync(UserMessage("hi"))); + Assert.Contains("different agent-hooks installation", exception.Message, StringComparison.Ordinal); + Assert.Equal(0, client.CallCount); + } + + [Fact] + public void FactoryRequiresInterceptors() + { + // Arrange + var client = new MockChatClient(); + + // Act / Assert + _ = Assert.Throws(() => client.AsAIAgentWithAgentHooks(new AgentHooksOptions())); + } + + [Fact] + public async Task NestedGuardedAgentsStayIsolatedAsync() + { + // Arrange: a guarded sub-agent invoked as a tool of a guarded outer agent. + var subClient = new MockChatClient().EnqueueText("sub says hi"); + var subGuard = new AllowGuard(); + var subAgent = subClient.AsAIAgentWithAgentHooks(new AgentHooksOptions(subGuard)); + var subTool = AIFunctionFactory.Create( + async () => (await subAgent.RunAsync(UserMessage("inner"))).Text, "ask_sub_agent"); + + var outerClient = new MockChatClient() + .EnqueueFunctionCall("call-1", "ask_sub_agent", []) + .EnqueueText("outer done"); + var outerGuard = new AllowGuard(); + var outerAgent = outerClient.AsAIAgentWithAgentHooks( + new AgentHooksOptions(outerGuard), AgentOptionsWithTools(subTool)); + + // Act + var response = await outerAgent.RunAsync(UserMessage("go")); + + // Assert: both runs completed with their own complete, separate sessions. + Assert.Equal("outer done", response.Text); + Assert.Contains("pre_tool_call", outerGuard.Points); + Assert.Equal( + ["agent_startup", "input", "pre_model_call", "post_model_call", "output", "agent_shutdown"], + subGuard.Points); + Assert.DoesNotContain("pre_tool_call", subGuard.Points); + } +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksPersistenceTests.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksPersistenceTests.cs new file mode 100644 index 00000000000..773827f0bda --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/AgentHooksPersistenceTests.cs @@ -0,0 +1,287 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Linq; +using System.Text.Json.Nodes; +using System.Threading.Tasks; +using AgentHooks; +using Microsoft.Extensions.AI; +using static Microsoft.Agents.AI.AgentHooks.UnitTests.TestHelpers; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +/// +/// Verdict-before-durability: denied content never becomes durable, transformed content +/// persists post-transform, per-service-call persistence is covered by its own verdict, +/// and nested runs persist inline at their own boundaries. +/// +public class AgentHooksPersistenceTests +{ + [Theory] + [InlineData(false)] + [InlineData(true)] + public async Task DeniedOutputNeverBecomesDurableHistoryAsync(bool streaming) + { + // Arrange + var provider = new RecordingHistoryProvider(); + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Deny("egress_blocked"))), + new ChatClientAgentOptions { ChatHistoryProvider = provider }); + var session = await agent.CreateSessionAsync(); + + // Act + if (streaming) + { + _ = await Assert.ThrowsAsync(async () => + { + await foreach (var _ in agent.RunStreamingAsync(UserMessage("hi"), session)) + { + } + }); + } + else + { + _ = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"), session)); + } + + // Assert: neither the denied response nor the denied turn's input persisted. + Assert.Equal(0, provider.StoreCalls); + Assert.Empty(provider.Stored); + } + + [Theory] + [InlineData(false)] + [InlineData(true)] + public async Task TransformedOutputIsPersistedPostTransformAsync(bool streaming) + { + // Arrange + var provider = new RecordingHistoryProvider(); + var client = new MockChatClient().EnqueueText("raw output"); + var transform = TransformTarget(new JsonObject { ["content"] = "[final]" }); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, transform)), + new ChatClientAgentOptions { ChatHistoryProvider = provider }); + var session = await agent.CreateSessionAsync(); + + // Act + if (streaming) + { + _ = await CollectAsync(agent.RunStreamingAsync(UserMessage("hi"), session)); + } + else + { + _ = await agent.RunAsync(UserMessage("hi"), session); + } + + // Assert: what became durable is the verdicted (transformed) content. + Assert.Equal(1, provider.StoreCalls); + string storedText = string.Concat(provider.Stored.Where(m => m.Role == ChatRole.Assistant).Select(m => m.Text)); + Assert.Equal("[final]", storedText); + } + + [Fact] + public async Task PermittedRunPersistsExactlyOnceAsync() + { + // Arrange + var provider = new RecordingHistoryProvider(); + var client = new MockChatClient().EnqueueText("fine"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new AllowGuard()), + new ChatClientAgentOptions { ChatHistoryProvider = provider }); + var session = await agent.CreateSessionAsync(); + + // Act + _ = await agent.RunAsync(UserMessage("hi"), session); + + // Assert + Assert.Equal(1, provider.StoreCalls); + Assert.Contains(provider.Stored, message => message.Text == "fine"); + Assert.Contains(provider.Stored, message => message.Text == "hi"); + } + + [Fact] + public async Task DeniedOutputNeverReachesContextProvidersAsync() + { + // Arrange + var contextProvider = new RecordingContextProvider(); + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Deny("no"))), + new ChatClientAgentOptions { AIContextProviders = [contextProvider] }); + var session = await agent.CreateSessionAsync(); + + // Act + _ = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"), session)); + + // Assert + Assert.Equal(0, contextProvider.StoreCalls); + } + + [Fact] + public async Task PermittedRunReachesContextProvidersPostVerdictAsync() + { + // Arrange + var contextProvider = new RecordingContextProvider(); + var client = new MockChatClient().EnqueueText("raw"); + var transform = TransformTarget(new JsonObject { ["content"] = "[final]" }); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, transform)), + new ChatClientAgentOptions { AIContextProviders = [contextProvider] }); + var session = await agent.CreateSessionAsync(); + + // Act + _ = await agent.RunAsync(UserMessage("hi"), session); + + // Assert + Assert.Equal(1, contextProvider.StoreCalls); + Assert.Contains(contextProvider.StoredResponses, message => message.Text == "[final]"); + } + + [Fact] + public async Task PerServiceCallPersistenceIsCoveredByItsOwnVerdictAsync() + { + // Arrange: per-service-call persistence with a run whose output is denied. + var provider = new RecordingHistoryProvider(); + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("done"); + var options = AgentOptionsWithTools(WeatherTool()); + options.ChatHistoryProvider = provider; + options.RequirePerServiceCallChatHistoryPersistence = true; + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Deny("egress_blocked"))), + options); + var session = await agent.CreateSessionAsync(); + + // Act + _ = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("weather?"), session)); + + // Assert: history persisted under a permitted post_model_call verdict remains + // durable even though the run's output was later denied. + Assert.True(provider.StoreCalls >= 1); + Assert.Contains(provider.Stored, message => message.Contents.OfType().Any()); + } + + [Fact] + public async Task DeniedModelResponseNeverPersistsPerServiceCallAsync() + { + // Arrange: per-service-call persistence with the first model response denied. + var provider = new RecordingHistoryProvider(); + var client = new MockChatClient().EnqueueText("secret"); + var options = new ChatClientAgentOptions + { + ChatHistoryProvider = provider, + RequirePerServiceCallChatHistoryPersistence = true, + }; + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.PostModelCall, Verdict.Deny("bad_response"))), + options); + var session = await agent.CreateSessionAsync(); + + // Act + _ = await Assert.ThrowsAsync(() => agent.RunAsync(UserMessage("hi"), session)); + + // Assert: neither the denied response nor the denied turn's request messages persisted. + Assert.DoesNotContain(provider.Stored, message => message.Text == "secret"); + Assert.DoesNotContain(provider.Stored, message => message.Text == "hi"); + } + + [Fact] + public async Task PerServiceCallPersistenceStillPersistsOnAllowAsync() + { + // Arrange + var provider = new RecordingHistoryProvider(); + var client = new MockChatClient() + .EnqueueFunctionCall("call-1", "get_weather", new() { ["location"] = "Paris" }) + .EnqueueText("done"); + var options = AgentOptionsWithTools(WeatherTool()); + options.ChatHistoryProvider = provider; + options.RequirePerServiceCallChatHistoryPersistence = true; + var agent = client.AsAIAgentWithAgentHooks(new AgentHooksOptions(new AllowGuard()), options); + var session = await agent.CreateSessionAsync(); + + // Act + var response = await agent.RunAsync(UserMessage("weather?"), session); + + // Assert: both service calls persisted. + Assert.Equal("done", response.Text); + Assert.True(provider.StoreCalls >= 2); + Assert.Contains(provider.Stored, message => message.Text == "done"); + } + + [Theory] + [InlineData(false)] + [InlineData(true)] + public async Task OuterDenyNeverDropsPermittedNestedRunHistoryAsync(bool streaming) + { + // Arrange: a guarded sub-agent (own provider, own session) invoked as a tool of + // a guarded outer agent whose output is denied. + var subProvider = new RecordingHistoryProvider(); + var subClient = new MockChatClient().EnqueueText("sub answer"); + var subAgent = subClient.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new AllowGuard()), + new ChatClientAgentOptions { ChatHistoryProvider = subProvider }); + var subTool = AIFunctionFactory.Create( + async () => + { + var subSession = await subAgent.CreateSessionAsync(); + return (await subAgent.RunAsync(UserMessage("inner"), subSession)).Text; + }, + "ask_sub_agent"); + + var outerProvider = new RecordingHistoryProvider(); + var outerClient = new MockChatClient() + .EnqueueFunctionCall("call-1", "ask_sub_agent", []) + .EnqueueText("outer secret"); + var outerOptions = AgentOptionsWithTools(subTool); + outerOptions.ChatHistoryProvider = outerProvider; + var outerAgent = outerClient.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Deny("egress_blocked"))), + outerOptions); + var outerSession = await outerAgent.CreateSessionAsync(); + + // Act + if (streaming) + { + _ = await Assert.ThrowsAsync(async () => + { + await foreach (var _ in outerAgent.RunStreamingAsync(UserMessage("go"), outerSession)) + { + } + }); + } + else + { + _ = await Assert.ThrowsAsync(() => outerAgent.RunAsync(UserMessage("go"), outerSession)); + } + + // Assert: the outer deny dropped only the outer run's persistence; the nested + // run's fully-permitted history persisted inline at its own run boundary. + Assert.Empty(outerProvider.Stored); + Assert.Contains(subProvider.Stored, message => message.Text == "sub answer"); + } + + [Fact] + public async Task PerRunHistoryProviderOverrideIsGatedTooAsync() + { + // Arrange: a per-run ChatHistoryProvider override smuggled through the run + // options would bypass the construction-time wrapper; the agent seam wraps it. + var overrideProvider = new RecordingHistoryProvider(); + var client = new MockChatClient().EnqueueText("secret"); + var agent = client.AsAIAgentWithAgentHooks( + new AgentHooksOptions(new PointGuard(InterceptionPoint.Output, Verdict.Deny("egress_blocked")))); + var session = await agent.CreateSessionAsync(); + var runOptions = new ChatClientAgentRunOptions + { + ChatOptions = new ChatOptions { AdditionalProperties = [] }, + }; + runOptions.ChatOptions.AdditionalProperties!.Add(overrideProvider); + + // Act + _ = await Assert.ThrowsAsync( + () => agent.RunAsync(UserMessage("hi"), session, runOptions)); + + // Assert + Assert.Empty(overrideProvider.Stored); + } +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Microsoft.Agents.AI.AgentHooks.UnitTests.csproj b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Microsoft.Agents.AI.AgentHooks.UnitTests.csproj new file mode 100644 index 00000000000..935a4b676df --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Microsoft.Agents.AI.AgentHooks.UnitTests.csproj @@ -0,0 +1,13 @@ + + + + + net10.0 + + + + + + + + diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/AllowGuard.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/AllowGuard.cs new file mode 100644 index 00000000000..6968934c892 --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/AllowGuard.cs @@ -0,0 +1,49 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Collections.Generic; +using System.Linq; +using System.Text.Json.Nodes; +using System.Threading; +using System.Threading.Tasks; +using AgentHooks; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +/// Allows everything and records every context it sees (point + deep clone). +internal sealed class AllowGuard : IInterceptor +{ + private readonly object _lock = new(); + + public List<(string Point, JsonObject Context)> Seen { get; } = []; + + public List Points + { + get { lock (this._lock) { return [.. this.Seen.Select(entry => entry.Point)]; } } + } + + public JsonObject Context(string point) + { + lock (this._lock) + { + return this.Seen.First(entry => entry.Point == point).Context; + } + } + + public List Contexts(string point) + { + lock (this._lock) + { + return [.. this.Seen.Where(entry => entry.Point == point).Select(entry => entry.Context)]; + } + } + + public ValueTask InterceptAsync(AgentContext context, CancellationToken ct = default) + { + lock (this._lock) + { + this.Seen.Add((context.InterceptionPoint.ToWireName(), (JsonObject)context.Json.DeepClone())); + } + + return new(Verdict.Allow); + } +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/ContentDenyGuard.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/ContentDenyGuard.cs new file mode 100644 index 00000000000..32da94bb134 --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/ContentDenyGuard.cs @@ -0,0 +1,17 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Threading; +using System.Threading.Tasks; +using AgentHooks; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +/// Denies at one point only when the projected context contains a marker string. +internal sealed class ContentDenyGuard(InterceptionPoint point, string marker) : IInterceptor +{ + public ValueTask InterceptAsync(AgentContext context, CancellationToken ct = default) => + context.InterceptionPoint == point && context.Json.ToJsonString().Contains(marker, StringComparison.Ordinal) + ? new(Verdict.Deny("marker_blocked")) + : new(Verdict.Allow); +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/CrashingGuard.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/CrashingGuard.cs new file mode 100644 index 00000000000..4df2c662a2b --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/CrashingGuard.cs @@ -0,0 +1,17 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Threading; +using System.Threading.Tasks; +using AgentHooks; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +/// Throws at one interception point, allow elsewhere (drives host_error:interceptor_failed). +internal sealed class CrashingGuard(InterceptionPoint point) : IInterceptor +{ + public ValueTask InterceptAsync(AgentContext context, CancellationToken ct = default) => + context.InterceptionPoint == point + ? throw new InvalidOperationException("guard crashed") + : new(Verdict.Allow); +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/MockChatClient.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/MockChatClient.cs new file mode 100644 index 00000000000..6a6406e0d2a --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/MockChatClient.cs @@ -0,0 +1,98 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Runtime.CompilerServices; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +/// A scriptable chat client: queued responses, recorded requests. +internal sealed class MockChatClient : IChatClient +{ + private readonly object _lock = new(); + + public Queue, ChatResponse>> Responses { get; } = new(); + + public List> Requests { get; } = []; + + public int CallCount + { + get { lock (this._lock) { return this.Requests.Count; } } + } + + public MockChatClient EnqueueText(string text) + { + lock (this._lock) + { + this.Responses.Enqueue(_ => new ChatResponse(new ChatMessage(ChatRole.Assistant, text))); + } + + return this; + } + + public MockChatClient EnqueueResponse(ChatResponse response) + { + lock (this._lock) + { + this.Responses.Enqueue(_ => response); + } + + return this; + } + + public MockChatClient EnqueueThrow(Exception exception) + { + lock (this._lock) + { + this.Responses.Enqueue(_ => throw exception); + } + + return this; + } + + public MockChatClient EnqueueFunctionCall(string callId, string name, Dictionary arguments) + { + lock (this._lock) + { + this.Responses.Enqueue(_ => new ChatResponse( + new ChatMessage(ChatRole.Assistant, [new FunctionCallContent(callId, name, arguments)]))); + } + + return this; + } + + private ChatResponse NextResponse(IEnumerable messages) + { + lock (this._lock) + { + List request = [.. messages]; + this.Requests.Add(request); + return this.Responses.Dequeue()(request); + } + } + + public Task GetResponseAsync( + IEnumerable messages, ChatOptions? options = null, CancellationToken cancellationToken = default) => + Task.FromResult(this.NextResponse(messages)); + + public async IAsyncEnumerable GetStreamingResponseAsync( + IEnumerable messages, ChatOptions? options = null, [EnumeratorCancellation] CancellationToken cancellationToken = default) + { + var response = this.NextResponse(messages); + await Task.Yield(); + foreach (var update in response.ToChatResponseUpdates()) + { + yield return update; + } + } + + public object? GetService(Type serviceType, object? serviceKey = null) => + serviceKey is null && serviceType.IsInstanceOfType(this) ? this : null; + + public void Dispose() + { + } +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/PointGuard.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/PointGuard.cs new file mode 100644 index 00000000000..85ff3049e8f --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/PointGuard.cs @@ -0,0 +1,24 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Threading; +using System.Threading.Tasks; +using AgentHooks; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +/// Returns a configured verdict at one interception point, allow elsewhere. +internal sealed class PointGuard(InterceptionPoint point, Verdict verdict) : IInterceptor +{ + public int Hits; + + public ValueTask InterceptAsync(AgentContext context, CancellationToken ct = default) + { + if (context.InterceptionPoint == point) + { + Interlocked.Increment(ref this.Hits); + return new(verdict); + } + + return new(Verdict.Allow); + } +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/PoisonedValue.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/PoisonedValue.cs new file mode 100644 index 00000000000..2d5b3ca0ded --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/PoisonedValue.cs @@ -0,0 +1,11 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +/// An argument value whose serialization throws (drives projection-failure halts). +internal sealed class PoisonedValue +{ + public string Boom => throw new ArgumentException("poisoned getter"); +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/RecordingContextProvider.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/RecordingContextProvider.cs new file mode 100644 index 00000000000..2699b3cd6ba --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/RecordingContextProvider.cs @@ -0,0 +1,40 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +/// A context provider that records its run-end (durable) notifications and failure notifications. +internal sealed class RecordingContextProvider : AIContextProvider +{ + public List StoredResponses { get; } = []; + + public List<(Exception Exception, int RequestMessageCount)> FailureNotifications { get; } = []; + + public int StoreCalls; + + protected override ValueTask ProvideAIContextAsync(InvokingContext context, CancellationToken cancellationToken = default) => + new(new AIContext()); + + protected override ValueTask InvokedCoreAsync(InvokedContext context, CancellationToken cancellationToken = default) + { + if (context.InvokeException is not null) + { + this.FailureNotifications.Add((context.InvokeException, context.RequestMessages.Count())); + } + + return base.InvokedCoreAsync(context, cancellationToken); + } + + protected override ValueTask StoreAIContextAsync(InvokedContext context, CancellationToken cancellationToken = default) + { + _ = Interlocked.Increment(ref this.StoreCalls); + this.StoredResponses.AddRange(context.ResponseMessages ?? []); + return default; + } +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/RecordingHistoryProvider.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/RecordingHistoryProvider.cs new file mode 100644 index 00000000000..4c8dab1b231 --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/RecordingHistoryProvider.cs @@ -0,0 +1,41 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +/// A chat history provider that records exactly what becomes durable (and what failure notifications carry). +internal sealed class RecordingHistoryProvider : ChatHistoryProvider +{ + public List Stored { get; } = []; + + public List<(Exception Exception, int RequestMessageCount)> FailureNotifications { get; } = []; + + public int StoreCalls; + + protected override ValueTask> ProvideChatHistoryAsync(InvokingContext context, CancellationToken cancellationToken = default) => + new([.. this.Stored]); + + protected override ValueTask InvokedCoreAsync(InvokedContext context, CancellationToken cancellationToken = default) + { + if (context.InvokeException is not null) + { + this.FailureNotifications.Add((context.InvokeException, context.RequestMessages.Count())); + } + + return base.InvokedCoreAsync(context, cancellationToken); + } + + protected override ValueTask StoreChatHistoryAsync(InvokedContext context, CancellationToken cancellationToken = default) + { + _ = Interlocked.Increment(ref this.StoreCalls); + this.Stored.AddRange(context.RequestMessages); + this.Stored.AddRange(context.ResponseMessages ?? []); + return default; + } +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/TestHelpers.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/TestHelpers.cs new file mode 100644 index 00000000000..4068a86c52a --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/TestHelpers.cs @@ -0,0 +1,44 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Text.Json.Nodes; +using System.Threading.Tasks; +using AgentHooks; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +internal static class TestHelpers +{ + public static List UserMessage(string text) => [new ChatMessage(ChatRole.User, text)]; + + public static AIFunction WeatherTool(Action? onInvoke = null) => + AIFunctionFactory.Create( + (string location) => + { + onInvoke?.Invoke(location); + return $"weather:{location}"; + }, + "get_weather"); + + public static ChatClientAgentOptions AgentOptionsWithTools(params AITool[] tools) => new() + { + Name = "assistant", + ChatOptions = new ChatOptions { Tools = [.. tools] }, + }; + + public static Verdict TransformTarget(JsonNode? value) => + new(Decision.Transform, Transform: new Transform("$target", value)); + + public static async Task> CollectAsync(IAsyncEnumerable stream) + { + List updates = []; + await foreach (var update in stream) + { + updates.Add(update); + } + + return updates; + } +} diff --git a/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/ToolAddingContextProvider.cs b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/ToolAddingContextProvider.cs new file mode 100644 index 00000000000..7cfe398fb24 --- /dev/null +++ b/dotnet/tests/Microsoft.Agents.AI.AgentHooks.UnitTests/Support/ToolAddingContextProvider.cs @@ -0,0 +1,14 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Extensions.AI; + +namespace Microsoft.Agents.AI.AgentHooks.UnitTests; + +/// A context provider that registers an additional tool during run preparation. +internal sealed class ToolAddingContextProvider(AITool tool) : AIContextProvider +{ + protected override ValueTask ProvideAIContextAsync(InvokingContext context, CancellationToken cancellationToken = default) => + new(new AIContext { Tools = [tool] }); +}