.NET: Give a hosted agent a single source of conversation history - #7525
Conversation
The handler used to fetch the platform conversation history and prepend it to the input of every turn. For a ChatClientAgent that runs in parallel with its own chat history provider, so the conversation had two sources at once. It also had a hidden cost: platform items carry no chat-history source marker, so the agent's provider stored them again as if this turn had written them, leaving a second copy of the conversation inside the persisted session that then diverges from the platform. Make the chat history provider the single source for a ChatClientAgent: - Add FoundryChatHistoryProvider, which reads the conversation through ResponseContext.GetHistoryAsync (it already resolves previous_response_id and the conversation the request belongs to) and stores nothing, because the platform persists the response items itself. An instance is created per request because it holds that request's context, and it is passed as a run-scoped override so the host does not have to mutate the agent. - Register it only when the agent was created without a chat history provider. When one was supplied at construction, that provider owns the conversation and the platform history is not used at all. - Stop adding the platform history to the input for a ChatClientAgent, since the provider now delivers it. A workflow hosted as an agent is not a ChatClientAgent and has no provider pipeline, so it keeps receiving the platform history from the handler exactly as before.
Cover the three symptoms the previous handler produced, each verified to fail when the handler is reverted to fetching the platform history into the turn input: - the conversation the service already keeps was copied into the persisted agent session by the default in-memory history provider; - a custom history provider was asked to write that same conversation into its own database, because platform items carry no chat-history source marker and so look like content this turn produced; - an agent with its own provider received both that provider's history and the platform's in a single request. Also state precisely, in the provider's remarks, why nothing is written back: for a stored request the response orchestrator hands the finished response to its responses provider, which persists the input and output items that a later turn then reads back through GetHistoryAsync; for a non-stored request nothing is persisted and nothing is readable, so the request is self-contained either way.
A conversation can mix turns the service stores with turns it does not. History is resolved from previous_response_id or the conversation regardless of the current request's store flag, so an unstored turn still reads the stored ones back, but the service records nothing for it and a later turn would never see it again. Reading the platform history through FoundryChatHistoryProvider alone lost those turns: from the second turn onwards the handler treats the session as a resume and stops feeding history in, and the provider kept nothing of its own, so an unstored turn simply vanished from the conversation. A regression test drives three turns of one conversation, the first stored and the rest not, and without this change the model receives only [second question, ok, third question]: the stored opening turn is gone. Give the provider both halves instead of choosing one: - reading returns what the service serves, followed by the turns kept in the session, which are by definition later than anything the service recorded; - writing keeps a turn only when the service was not asked to store it, so a stored turn is never duplicated and an unstored one is never lost. The turns are held in the agent session under the provider's own state key, so they travel with the session the host already persists.
A conversation can move between stored and unstored turns, and the unstored ones live only in the agent session. Going back to a stored turn after that would have the service record it on top of turns the service never saw, so anyone reading the conversation back from the service would find an answer with no question. Refuse it before the model is called instead of writing that gap. Cover the whole shape with a walkthrough of nine turns over one conversation and three provider instances, each with its own session: - an instance that never took an unstored turn starts from the turn the service last saved, and does not see another instance's unstored turns; - an instance that did keeps reading the saved turns and adds its own on top; - asking such an instance for a stored turn is refused, twice, while unstored turns keep working; - a turn stored from one instance does not appear for another, because it sits on a different branch of the conversation and so is not among the turns leading to what that other instance last saved.
The turns the service was not asked to store are written into the agent session's state bag under this provider's own state key, and a new provider is built for every request, so nothing is held on the provider object itself. The walkthrough named its three threads after provider instances, which read as if the object carried the memory. Name them after the sessions they are, and add a test that pins the behaviour down: a turn kept through one provider object is read back by a different one given the same session, and is absent for one given another session.
The session decides what is kept, but the provider still decides two things: which service-side conversation is read, because it holds the request's response context, and whether the turn is kept at all, because it holds the request's store flag. Add two tests that separate those from the session: - two providers reading one session, each built for a request of a different conversation, return the same kept turn behind different served turns; - two providers writing to one session, one for a stored request and one for an unstored one, leave only the unstored turn behind.
The comment stated that a workflow hosted as an agent has no provider pipeline without saying what that means. It derives from AIAgent directly, so it never calls a ChatHistoryProvider and does not read the run options' additional properties: the provider could not reach it even if it were registered.
The handler decided that a turn was resuming an existing conversation by looking for state on the session. That reading broke once the handler itself started writing to the session before the check: it records the caller's identity there, so a session created moments earlier already carried state and the very first turn of a conversation looked like a resume. Its history was then never fetched, and the agent answered knowing nothing of a conversation the service was already holding. It only showed up when hosted, because running locally there is no identity to record. Let the store answer the question instead. GetSessionAsync now returns null when nothing is stored rather than quietly handing back a new session, so a non-null result means a prior turn established this session and nothing else has to be inferred. Callers that just want a usable session can use the new GetOrCreateSessionAsync, which is written in terms of GetSessionAsync so a store overriding one gets the other for free. Both store implementations and their tests follow the plain-lookup contract: a miss creates nothing, deserializes nothing, and touches no directory.
FoundryChatHistoryProvider is internal, so the attribute reached no caller: the marker exists to warn people consuming the public surface. It also does not follow from the base type, which does not carry one, and most internal types in this package have none either. Removing it leaves two usings behind, so they go as well.
An agent refuses a second history manager once the model reports a conversation id of its own, which happens as soon as the container lets the model keep the conversation. The guard is meant for an application that configured a provider by hand and would otherwise end up with two of them. Here the host is the one supplying the provider, deliberately and for every turn, so the guard was rejecting the arrangement it is hosting: the first turn failed while streaming, and every later one failed before reaching the model at all. Turn the three conflict settings off on the agent the host is serving, and let the provider decide what reaches the model. A test drives two turns of one conversation against a model that reports a conversation id and asserts both complete.
A request asking the hosting service not to store the response was honoured there and nowhere else, so the service behind the agent's own chat client kept recording the conversation and reporting an id for it. A caller opting out of storage still ended up with a stored conversation, and the container went on continuing it. Only that direction travels. Carrying store=true across would either force storage on a container whose author turned it off on purpose or change nothing, since storing is already the default.
The host no longer supplies a chat history provider of its own. It writes the turns the service holds into the provider the agent already created for itself, and only when that is the stock in-memory one, so an agent given a provider keeps sole control of its storage and the model receives the conversation once. A conversation the caller stops asking the service to store moves into the session state and stays there. The session's conversation id no longer names anything the service records and cannot be cleared, so the session is cloned without it on that single turn. Asking for a stored turn afterwards is refused: the service would record a turn whose predecessors it does not hold. An agent that does not read history through a provider, a hosted workflow for example, is still given its prior turns as input, now marked as chat history so no provider along the way stores them as new.
There was a problem hiding this comment.
🟡 Changes recommended
The store=false propagation via ChatOptions.RawRepresentationFactory can suppress agent/container RawRepresentationFactory behavior due to ChatClientAgent’s factory chaining semantics, risking incorrect request shaping.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
This PR adjusts the Foundry hosted-agent request pipeline to ensure each turn’s conversation history comes from exactly one place (service history, provider/session state, or the model’s own conversation id), preventing duplicate replay/storage and fixing incorrect “resume” detection.
Changes:
- Update hosted session store semantics so
GetSessionAsyncis a pure lookup returningnullwhen nothing is persisted, and addGetOrCreateSessionAsyncfor callers that want a usable session. - Rework
AgentFrameworkResponseHandlerhistory routing: preload service history into the defaultInMemoryChatHistoryProviderforChatClientAgents, stamp replayed service history asChatHistorywhen passed as input, and use store presence (not session state) as the resume signal. - Propagate
store=falseinto chat-client options viaCreateResponseOptions.StoredOutputEnabled = false.
File summaries
| File | Description |
|---|---|
| dotnet/tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/FileSystemAgentSessionStoreTests.cs | Updates tests to reflect GetSessionAsync now returning null on cache miss and adds coverage for GetOrCreateSessionAsync. |
| dotnet/tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/AgentFrameworkResponseHandlerTests.cs | Adds regression coverage for resume detection, single-source history routing, and stored/unstored conversation continuity. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/InputConverter.cs | Attempts to forward store=false to the underlying chat client via RawRepresentationFactory. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/InMemoryAgentSessionStore.cs | Makes GetSessionAsync a pure lookup returning null when not found. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/HostedSessionJsonUtilities.cs | Adds a serialization shape to clone a ChatClientAgentSession without a conversationId. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/FileSystemAgentSessionStore.cs | Makes GetSessionAsync return null on missing/empty session file. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/AgentSessionStore.cs | Updates GetSessionAsync contract to nullable and introduces GetOrCreateSessionAsync. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/AgentFrameworkResponseHandler.cs | Centralizes conversation ownership per turn, fixes resume detection, and prevents history duplication/storage. |
Review details
- Files reviewed: 8/8 changed files
- Comments generated: 1
- Review effort level: Lite
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
ChatClientAgent chains a request's raw representation factory with the agent's by taking the agent's only when the request's returns null. The factory added for an unstored turn always answers, so anything the container configured on the agent's ChatOptions was silently dropped for that turn. The agent's factory is now invoked first and its result is what carries the setting. A result that is not a CreateResponseOptions belongs to some other chat client, which has no notion of storing a response, so it is handed back untouched.
|
Thanks for the review, I will be creating a follow up PR with the suggestions, merging this as a good progress, to unblock subsequent other works. Thanks! |
Five points raised on microsoft#7525 were marked resolved without a code change, and the code they pointed at was still there. A hosted workflow session is now recognised by its full type name, so a session of the same short name from another namespace is not mistaken for one. The test double moves into the namespace it stands in for, otherwise it would no longer exercise the check. The per-run chat history provider is handed over on AgentRunOptions.AdditionalProperties, which ChatClientAgent copies onto the chat options with precedence, rather than being written onto the chat options here. The test that pins down who supplies the history said the agent's own provider is used, while it asserts the opposite, so it is renamed after what it checks. Reading a response back in the hosted integration tests no longer swallows every failure: only "not stored" and "not readable through this endpoint" are, so an expired token or a server fault cannot be mistaken for an absent response and pass the test. Also fills in the readiness message for the case where storing is explicitly allowed.
…t#7572) * Let the container choose who stores a hosted turn, and say so when it is stored twice Turning storage off downstream was unconditional and silent. It is now a container choice, and a deployment that ends up storing anyway is reported instead of quietly recording the conversation in two places nothing reconciles. FoundryResponsesOptions, passed through AddFoundryResponses, carries two settings. AllowStoredOutputEnabled defaults to false, which is when hosting turns storage off for every run and checks the result. Setting it to true leaves the agent's own configuration exactly as the container built it, and nothing is checked, overridden, or refused. IncludeReasoningEncryptedContent applies while storage is off, asking for the encrypted form of the reasoning tokens so reasoning survives between turns, mirroring AsIChatClientWithStoredOutputDisabled. Two checks replace the 400 that used to refuse a session carrying a conversation id. The readiness probe runs each registered agent with its chat client swapped for one that calls nothing, so the request the agent builds on its own is visible without leaving the container, and an agent asking for its responses to be stored keeps the container out of rotation. Per request, a conversation id on the session after the run means the agent's own service kept the turn, which fails with 501 and leaves the session unsaved so later turns do not resume onto it. A misconfigured container is a server problem, not a bad request, hence 5xx. Only a confirmed "this asks to be stored" fails either check. An agent that is not a ChatClientAgent, a request shape carrying no such setting, and a run that could not be completed all pass: this package cannot tell what those would do. * Rename the stored-session flag to say what it means * Say plainly what server-side storage does to a hosted turn * Read the store gate as an allow, and align the messages The flag that decides whether the session may be saved reads as an allow at every use, while the test it comes from keeps saying what is not allowed, so neither side has to be read inside out. The wording now matches what the readiness probe says: server side storage must be off, because with it on the agent's own service records a conversation and response nothing tracks while the hosted agent records its own for the same request. The message the readiness probe raises no longer travels through a shared constant, since each check says its own thing. * Address the review comments left open on the merged PR Five points raised on microsoft#7525 were marked resolved without a code change, and the code they pointed at was still there. A hosted workflow session is now recognised by its full type name, so a session of the same short name from another namespace is not mistaken for one. The test double moves into the namespace it stands in for, otherwise it would no longer exercise the check. The per-run chat history provider is handed over on AgentRunOptions.AdditionalProperties, which ChatClientAgent copies onto the chat options with precedence, rather than being written onto the chat options here. The test that pins down who supplies the history said the agent's own provider is used, while it asserts the opposite, so it is renamed after what it checks. Reading a response back in the hosted integration tests no longer swallows every failure: only "not stored" and "not readable through this endpoint" are, so an expired token or a server fault cannot be mistaken for an absent response and pass the test. Also fills in the readiness message for the case where storing is explicitly allowed. * Address the review on microsoft#7572 Four findings, all real, all in code this branch introduced. A container that allows its own service to keep the conversation was still being handed the platform history on every turn. That service replays the earlier turns itself, so the model was getting each of them twice, which is the very thing this work exists to prevent. The history now goes in only while nothing else holds it: the first turn of such a conversation still gets it, and the service takes over from there. A turn that fails for storing downstream was announcing itself as completed first and only then failing, leaving the caller with two different answers for the same turn. The completed event is now held back until the run is wound up and the session can be read, because the id of any conversation the agent's service kept only lands there at the very end. The readiness probe replaced the chat client but left the agent's chat history provider running, so a provider backed by a database was reading and writing on every probe, and adding the probe's empty turn to a real conversation. It is stood down for that run now. The probe also treated any cancellation as the health check's own, so a timeout inside an agent could fail readiness. Only a cancellation of the health check's token is left to propagate. Fixing the completed event turned up a latent problem: the terminal event types are named the same in two namespaces this file pulls in, and the short name binds to the ones the response stream never produces, so �vt is ResponseCompletedEvent was quietly always false. The three terminal types are now named explicitly. * Let the chat history provider carry the conversation The handler used to read the hosting service's record of the conversation and prepend it to the input of every run, then work out who should not get it: a resumed workflow by the name of its session type, and a container whose own service already holds the conversation. Two exceptions, a type name matched as a string, and a shape where the same turns could arrive from two directions. An agent that reads its history through a provider is now given one, seeded with that record, for the length of the run. The turns arrive the way the agent expects them rather than as fresh input, so nothing is stored back as if it had just been said, and the provider is dropped when the run ends. Only the new input is passed to the run now. Everything else supplies its own history and is left alone: an agent built with a provider keeps using it, an agent whose service keeps the conversation reads it from there, and an agent that is not a ChatClientAgent, a hosted workflow for instance, carries the conversation in its own session state and wants only the new input. The workflow session type name check is gone with it. The session is saved on every turn again. It was being withheld when the agent's own service had kept the turn, which is a decision about that service, not about the session; nothing this handler adds for a turn reaches the session anyway. * Fail a turn to skip its session, and name the store check after what it detects The session was being withheld from the store on a condition about the agent's own service rather than about the turn, and guarded by an emptiness check on a key that is never empty. A turn that is being failed now says so, and only that skips the save. A turn that ends incomplete, waiting on OAuth consent or interrupted by a shutdown, is not a failure: the caller comes back for it and needs the state built up so far, the tool approval ids among it. The session key is resolved once as a value that always exists, so both the load and the save use it without asking again whether it is there. CheckNotAllowedStoreUsage and notAllowedStoreUsageDetected now read as what they are: a check for an agent storing when it should not, and the flag saying it was seen. * Read a hosted response through the agent client, and only forgive a 404 Reading a response back tried the project-level client first and then the per-agent one, swallowing 403 as well as 404 to get past the first. The project-level client cannot see a hosted agent's responses at all, so that attempt only ever produced the 403 the catch then had to forgive, and any other 403, an authorization failure for instance, was read as "nothing is stored" and passed the test. Only the per-agent client is used now, and only a 404 counts as not stored. Verified against the service: a well-formed id it has no response for answers 404 invalid_request_error "Response '...' not found", the same id through the project-level client answers 403 session_not_accessible, and a malformed id answers 400. Everything but the 404 now surfaces. * Move the store setting next to the code that reads and writes it The two halves of the stored output concern lived in a shared helper: one that installs the factory turning storage off, and one that reads back what a request would have asked for. Each had exactly one caller, so the helper only added a hop. They now sit in the converter that builds the request and in the probe client that inspects it, and the helper keeps just the error the handler throws. The test double standing in for a hosted workflow session is also gone. It was declared inside the Workflows namespace because the handler used to recognise a resumed workflow by the full name of its session type; that comparison no longer exists, so the double only needs to not be a ChatClientAgent. * Say that the stored output setting could not be determined, which is the case being logged
Motivation & Context
A hosted agent quietly kept a second copy of its conversation.
Two different things record a hosted turn. The AgentServer SDK's storage provider records it
around the container's handler, and that record is the conversation the caller reads back.
Separately, the service behind the agent's own chat client records the turn again whenever the
agent runs with storing left on, onto a trail of its own. Nothing reads that second trail, and
nothing reconciles it with the first.
The same conversation could also reach the model twice, once as input from the platform's record
and once replayed by the agent itself, consuming extra tokens for no gain.
Description & Review Guide
What are the major changes?
One source of history, and one recording of each turn.
ResponseContext.GetHistoryAsync) and is passed as input, marked as chat historyVolatileChatHistoryProvider, which holds messages in a field and is dropped when the run endsCreateResponseOptionsandChatCompletionOptions)service_managed_chat_history_not_supportedAgentSessionStore.GetSessionAsyncnow returnsnullwhen nothing is stored, andGetOrCreateSessionAsyncis added alongside itWhat a single turn looks like now:
graph LR C["Caller"] -->|"turn, store as the caller asked"| P["AgentServer storage provider<br/>records the turn"] P --> H["Container handler<br/>reads that record back as the input history"] H -->|"store off"| S["The agent's own service<br/>answers and records nothing"] S --> PWhat is the impact of these changes?
ChatClientAgent, as in the first hosted agent sampleA new integration test covers this against a live Foundry project. The container agent is an
ordinary
ChatClientAgent, wrapped so that after each run it reports back the conversation itsown run left behind, and the test goes looking for it on the service. Finding it means a second
copy exists.
AgentSessionStorehere is the one inMicrosoft.Agents.AI.Foundry.Hosting, which partitionsper user; the separate type of the same name in the framework's own hosting package is
untouched.
GetSessionAsyncchanges its return type and its meaning. The type is public, butthe handler is its only caller and the package is still in preview, so both in-box stores are
updated here and nothing else has to follow.
What do you want reviewers to focus on?
Whether refusing a session that carries a conversation id is the right call, or whether such a
container should instead be allowed to run with its service holding the conversation and the
handler standing down entirely.
Related Issue
N/A
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.