Complete OPC UA Part 11 Historical Access

[marcschier]

Description

Completes the OPC UA Part 11 Historical Access work requested by #4387 and
the historian framework integration requested by #4400.

The conformance/evidence catalog covers all 37 released UACore 1.05
Historical Access profiles: 15 Server facets and 22 Client facets, together
with the standard aggregate functions and their conformance units. Server
facets are advertised only when the resolved providers expose their required
interfaces and capabilities.

Server and provider support

• Completes raw, modified, at-time, processed, annotation, structured, and event history.
• Dispatches best-effort, bulk, and transactional updates with aligned statuses, old values, diagnostics, and rollback state.
• Supports mixed history-update detail types and batched structured/annotation operations.
• Uses composite identities and exclusive cursors for same-timestamp structured, modified, and event values.
• Captures live data and events, installs historical configuration objects, and emits typed audit payloads.
• Derives aggregate monitoring filters from per-node historian capabilities and rejects unsupported processed aggregates before provider dispatch.
• Supports tri-state Historizing: set, clear, or preserve provider-owned state.

Client support

• Exposes modified values together with ModificationInfo.
• Adds event-history reads and event insert/replace/update/delete operations.
• Adds structured and annotation batching with strict response validation.
• Shares continuation handling and release-on-abandon across read families.
• Accepts inherited standard event fields rooted at event subtypes.

Review hardening

• Registers DI-selected historians before node-manager reconciliation and capability publication, while preserving provider ownership.
• Uses committed Raft read barriers for linearizable TryGetAsync and ScanAsync.
• Keeps claimed continuation state immutable until a fresh successor is saved, restoring the original after transient failures.
• Uses exact-incarnation compare-and-delete so delayed cleanup cannot remove a restored same-ID continuation.
• Cleans up indeterminate initial and replacement saves, and propagates shutdown cancellation through cleanup resolution.
• Preserves legacy continuation envelopes/cursors and distinct annotation parent/property identities.
• Deletes every structured entry at a requested timestamp and uses the complete modified-history ordering tuple.
• Shares native/fallback at-time and AnnotationCount calculations, including reverse endpoints and cross-framework fractional intervals.
• Primes modified aggregate items before buffered live delivery and deduplicates history/live overlap using provider-specific identities.
• Keeps committed Modify results successful while reporting priming failures
  through the monitored item's notification queue.
• Keeps protected-notification state live-only for queues larger than one.
  Queue size one retains the newest notification without pinning errors or
  transferring priority during resizing. Durable restore retains ordinary
  definitions, last values/errors, and raw queues without recreating priority.
• Isolates aggregate calculation, modification preparation, and historical/live
  handoff in AggregationFilterHandler, retaining existing provider/factory
  wiring and supported past-start behavior. The larger-queue protection policy
  is unchanged pending the open review discussion.
• Uses one IMonitoringFilter owner for protocol filters or the aggregation
  handler, eliminating parallel effective-filter storage and retaining the
  current definition during preparation. The original client filter stays distinct.
• Integrates current master, retaining custom monitored-item creation decisions and per-node history callbacks.

Redundant-server alignment and samples

• Adds a protected shared historian with immutable segments, manifest CAS publication, writer fencing, and restart recovery.
• Persists portable continuation state before exposing it to clients and transfers ownership during mirrored-session takeover.
• Rejects eventual-consistency and active/active multi-writer historian configurations rather than silently degrading.
• ReferenceServer and ConsoleReferenceClient demonstrate discovery, paging, raw/modified/at-time/processed reads, annotation/structured CRUD, event capture, and event CRUD.
• RedundantServer and RedundantClient demonstrate active/passive history, continuation recovery after replica termination, and post-promotion writes.
• Updates Historical Access, High Availability, profile, migration, and sample documentation.

Compatibility

• New 2.0 collection boundaries use ArrayOf<T> and public byte payloads use ByteString.
• Removes the legacy public virtual SamplingGroupManager.CreateMonitoredItem /
  ModifyMonitoredItem entry points and protected creation factory, as agreed
  in review. Migration guidance points to the monitored-item manager and
  construction-decision APIs.
• Retains established persisted formats and current write layouts. Retired interim
  notification-state formats (sample v2 / shared v4) are rejected, as agreed in review.
• Keeps the historian integration compatible with .NET Framework and NativeAOT.

Validation

Latest CI source repair

73c619435 forwards the existing cancellation token at both predefined-node
creation calls in the PR-added History event fixture. The merged lifecycle
overload exposed these two CA2016 errors, which blocked History jobs, the
solution build matrix, and CodeQL on GitHub and Azure. The exact compiler
failure was reproduced locally; the repaired fixture passes 4/4 integration
cases on net10.0 and 4/4 on net48 with clean scoped diagnostics. No assertions,
tests, coverage thresholds, or CI configuration were weakened.

The repaired History jobs on Linux/macOS and all four Azure solution builds
now pass. The Windows all-TFM GitHub job instead lost its hosted runner's
connection; no build log was recoverable, and GitHub recorded a runner-loss
annotation rather than a compiler/test diagnostic. One unchanged retry was
requested. Remaining checks are monitored with
gh pr checks --watch --fail-fast --interval 30; CI is not yet reported as green.

Latest base integration

bd52e5fea merges upstream master at da7412c42. The lifecycle conflict
retains upstream's shared cancellation-aware registration helper and the
historian behavior. The hosted-historian fixture forwards its existing token
to the new lifecycle overload. GDS project renames and all three AOT test
hosts are retained, with their project references resolving after the merge.

Integrated scope	net10.0	net48
Server lifecycle, fluent authoring/import, historian, monitored-item and hosting regressions	732 passed	732 passed
NodeState lifecycle regressions	216 passed	216 passed
Automatically merged Raft client fixtures	28 passed	28 passed

No selected tests failed or were skipped. The manually resolved files pass
scoped formatting/analyzer checks. One CA1861 warning remains in the incoming
NodeSetImportIntegrationTests.cs; that file is byte-identical to upstream
master and was not rewritten or suppressed during integration.

Latest feedback follow-up

102954eaa replaces split filter references with the shared IMonitoringFilter
contract implemented by protocol filters and AggregationFilterHandler.
Targeted server regressions pass 103/103 on net10.0 and 103/103 on net48;
scoped formatting/analyzers are clean, and the updated XML-documentation
inventory has zero gaps across 188 non-generated PR C# files.

93c5b5db8 renames AggregationFilterHandler, makes it the owner of the
server-revised aggregate filter while preserving the original client filter,
and removes the retired interim notification-state readers. Server regressions
pass 101/101 and shared-store regressions 64/64, on both net10.0 and
net48, with no failures or skips. Runtime-file formatting/analyzers are clean.

5ccf45630 completes the public/internal XML-documentation sweep. The Roslyn
inventory finds zero undocumented eligible declarations across all 186
non-generated PR C# files; the 110-file documentation slice changes only XML
comments and spacing. f17765475 applies the three requested Markdown edits.

Earlier feedback follow-ups

a53950737 removes single-slot notification pinning and extracts aggregate
implementation from MonitoredItem. Targeted sampling, aggregate modification,
history/live handoff, queue/lifecycle, restore, and serialization cases pass
101/101 on net10.0 and 101/101 on net48, without failures or skips.
Scoped formatting/analyzers are clean; the focused review found no significant
issues. The final formatted monitored-item fixtures also pass 45/45 on net48.

The preceding b21555568 removes the legacy sampling API bridge and durable
notification-priority state. Its targeted Server cases passed 97/97 and shared
subscription-store cases passed 64/64 on each framework. The broader evidence
below remains pinned to its original tested commit, not attributed to these
follow-ups.

Earlier integrated-implementation validation

Validated implementation commit: 3efae597f. The following commit,
f6dbefb1c, only adds two explanatory comment lines for the final standards
review; it changes no executable code.

Scope	net10.0	net48
Server	5,187 passed; 56 skipped	Affected surface: 2,160 passed; 1 reproduced baseline failure; 56 skipped
Client	2,202 passed; 2 skipped	2,204 passed; 5 skipped
History integration	519 passed; 21 skipped	519 passed; 21 skipped
Redundancy server	641 passed	641 passed
Redundancy client	124 passed	124 passed
Core shared-store regression suite	23 passed	23 passed

All six affected samples passed non-incremental Release/net10.0 builds with
zero warnings and errors: ConsoleReferenceClient, ConsoleReferenceServer,
RedundantServer, RedundantClient, MinimalBoilerServer, and
PumpDeviceIntegrationServer.

The published self-contained win-x64 NativeAOT executable is 116,924,928
bytes and has no CLR header or managed runtime dependency. Direct execution
passed HistoryAotTests 3/3 and RaftAotTests 2/2.

Final standards and specification reviews found no confirmed outstanding
high/medium functional gaps or hard standards violations. The remaining
constructor-purpose clarification is addressed in f6dbefb1c.

Reproduced baseline exceptions

The net48 hosted-startup timeout also reproduces in a pristine worktree at
the previous PR head, 24cf1416d, in
RegisterPostStartRegistriesWiresHistorianAndAliasStoresOnPlainServerAsync,
with the same subsequent certificate-leak assertion. The corresponding
renamed case fails before startup in the reviewed branch as well. This is
reported separately from the net48 affected-surface results, not counted as
a passing full Server suite. The net48 matrix selected the historian,
aggregate, monitored-item, fluent, and continuation surfaces and excluded
the Hosting namespace.

DurableDataValueQueueVerifyReferenceBatchingAsync failed at item 1000 in
the net48 affected-surface run and passed its single isolated rerun. The same
failure at the same assertion reproduces on pristine 849e16acc; the test,
sample queue, queue factory, and batch persistor are unchanged. The original
failure remains in the table rather than being erased by the rerun.

CI source repairs

• 4be7eb8f7 keeps diagnostics/configuration managers out of distributed
  address-space replication. Those managers inherit the source interface,
  but their live server state and redundancy metadata must remain replica-local.
  The ownership regression passes on net10.0 and net48, and all nine
  redundancy sample CI tests pass locally.
• d17f467b3 completes the sample startup repair: core managers are also
  replica-local; the startup banner no longer advertises readiness before
  initialization; the producer starts after distributed services; and native
  Raft startup no longer forces disruptive extra elections. The full sample
  suite passed 9/9, with initialized RedundancySupport=Hot, and focused
  Raft/address-space regressions passed 8/8 and 3/3.
• 81b0b72be moves the unchanged distributed-historian NativeAOT scenario
  into a smaller, always-run companion executable to bound Apple's linker
  input. All 137 AOT tests remain present; the moved test and helpers are
  unchanged. GitHub and Azure publish/run both native executables, with
  separate result directories where they share a job. The companion passed
  actual Windows NativeAOT execution. Linux and Intel macOS CI passed both
  executables, but this split alone did not fix ARM64 macOS linking.
  The remaining historian graph is rooted by ReferenceServer startup, not
  just the relocated test. No test exclusion or speculative linker setting
  was added to conceal it.
• 8a5842913 isolates the independent MCP test graph into an always-run
  native host. The existing MCP source and assembly leak hooks are linked
  unchanged, and evaluated compiler inputs prove exactly one owner for
  every test source: 133 main + 3 MCP + 1 historian = 137 declarations.
  All three hosts run on every AOT platform. The main Windows native object
  is 9.35% smaller; both changed publishes had zero diagnostics. MCP passed
  3/3; the identical main executable passed 133/133 on one unfiltered
  followup after two initial endpoint-discovery connection failures, which
  remain recorded in the local evidence.
• The previous Intel macOS job successfully published and executed its native
  tests, then failed on artifact-upload DNS (ENOTFOUND). No upload gate,
  shared environment setting, or cache configuration was disabled.

The first repair round also encountered the unrelated
MetricsAreEmittedForChannelLifetimeAsync timing failure on macOS Core.
Neither that test nor channel-manager code is changed by this PR, and its
assertions remain intact. It passed unchanged on the final repair head.

On d17f467b3, the Linux and macOS redundancy sample jobs pass; the remaining
failure was ARM64 Apple's too many large addends assertion. The verified
upstream relocation-anchor fix (dotnet/runtime#124721, backport
dotnet/runtime#132171) is not present in the shipping ILCompiler 10.0.11.
An experimental multi-module probe failed and was rejected, not added to CI.
On 8a5842913,
aot-macos-latest
successfully publishes and executes all three native hosts on ARM64:
136 main runtime cases, 1 historian case, and 3 MCP cases, with no skips.
The previous Apple linker blocker is resolved. Linux NativeAOT also passes.
No assertion, test, shared environment setting, cache, or required gate was
weakened.

The same CI run reports a separate macOS Sessions teardown failure: all
662 executed tests pass, but the global leak assertion detects one
undisposed certificate. The MCP partition changes no runtime, Sessions, or
test-fixture code. A single unchanged diagnostic rerun is queued once GitHub
permits rerunning that job; the failure is not suppressed or counted as green.
Other CI checks are still pending.

Related issues

• Fixes Part 11: IHistorianTransactionalProvider is never called, and HistoryClient drops ModificationInfo and has no event API #4387
• Fixes Historian: aggregate-filter revision, processed-read aggregate gate, initial-value priming and Historize() forcing Historizing=true #4400

Checklist

• I have signed the CLA and read the contributing guide.
• I have added regression tests and directly related documentation.
• I have retained working reference and redundant-server sample wiring.
• Final validation and review evidence is complete, including the baseline exceptions above.
• All required CI checks pass; the fail-fast rerun for 73c619435 is pending.

[Copilot]

🔵 Needs a closer look

It introduces broad new client/server historian behaviors and a public API shape change that warrant final human review despite strong test coverage.

Pull request overview

This PR closes the Part 11 historian gaps from #4387 by aligning server dispatch behavior with the transactional-provider contract, extending the HistoryClient to preserve modified-history audit metadata, and adding full event-history read/update APIs. It also consolidates client-side continuation-point paging so all history read modes share consistent pagination and cleanup behavior.

Changes:

• HistorianDispatcher now prefers IHistorianTransactionalProvider atomic Insert/Replace/Update paths when available.
• HistoryClient.ReadModifiedAsync now yields ModifiedDataValue (value + ModificationInfo), and HistoryClient gains event history read/insert/replace/update/delete APIs.
• Annotation writes/removals are batched via UpdateStructureDataDetails, and history-read continuation/paging logic is centralized.

File summaries

File	Description
tests/Opc.Ua.Server.Tests/Historian/HistorianTransactionalDispatcherTests.cs	Adds dispatcher-level coverage ensuring transactional providers use atomic operations and roll back on collision.
tests/Opc.Ua.History.Tests/HistoryClientIntegrationTests.cs	Extends integration coverage for transactional rollback behavior and modified-history audit metadata.
tests/Opc.Ua.History.Tests/HistoryClientEventIntegrationTests.cs	Adds end-to-end event history and batched annotation integration tests.
tests/Opc.Ua.Client.Tests/Historian/HistoryClientUnitTests.cs	Updates unit tests for the ReadModifiedAsync return type and metadata assertions.
tests/Opc.Ua.Client.Tests/Historian/HistoryClientPart11Tests.cs	Adds unit tests for Part 11 gaps: modified audit pairing, event paging/cleanup, event update detail construction, structured remove batching.
src/Opc.Ua.Server/Historian/HistorianDispatcher.cs	Routes raw Insert/Replace/Update through transactional atomic methods when supported.
src/Opc.Ua.Client/Historian/ModifiedDataValue.cs	Introduces the client-side ModifiedDataValue record struct for modified-history reads.
src/Opc.Ua.Client/Historian/HistoryClient.Updates.cs	Adds shared HistoryUpdate helpers and UpdateStructureDataAsync.
src/Opc.Ua.Client/Historian/HistoryClient.Paging.cs	Centralizes paginated HistoryRead loop + continuation-point release and decoding helpers.
src/Opc.Ua.Client/Historian/HistoryClient.Extras.cs	Switches annotation reads to the shared paging helper and adds batched WriteAnnotationsAsync.
src/Opc.Ua.Client/Historian/HistoryClient.Events.cs	Adds event-history read and update APIs on the client.
src/Opc.Ua.Client/Historian/HistoryClient.cs	Uses shared paging + update helpers; changes ReadModifiedAsync to return ModifiedDataValue.
docs/WhatsNewIn2.0.md	Documents modified-history audit metadata and event-history support in 2.0.
docs/README.md	Updates the Historical Access doc blurb to include event reads and audit metadata.
docs/Profiles.md	Updates feature coverage to include modified audit metadata and event history.
docs/HistoricalAccess.md	Updates Part 11 documentation to match new transactional routing, modified return type, and event/annotation batching APIs.

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The new shared history paging cleanup path can throw ObjectDisposedException during best-effort continuation-point release, which risks surfacing exceptions during enumerator disposal and masking the original failure.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 1
• Review effort level: Lite

[romanett]

Quite a big one, but i think it is okay

[romanett]

The MonitoredItem has to many changes here, it should not carry more logic than needed. Basically its task is to store values in a queue and publish them on demand.

updated