Skip to content

Add compile-time RPC codec scope routing - #320

Closed
SunSi12138 wants to merge 290 commits into
mainfrom
fix/311-codec-scope-routing
Closed

SunSi12138 wants to merge 290 commits into
mainfrom
fix/311-codec-scope-routing

Conversation

@SunSi12138

@SunSi12138 SunSi12138 commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Status: superseded by #386 — do not merge.

This PR is retained as the research/review history for the codec-routing work, including the API-5 per-Contract experiments and review iterations. The final implementation moved to #386 as a clean, single-parent port using the assembly-owned codec-routing model. The rejected transitional/API-5 machinery on this branch is intentionally not carried forward.

#386 exact head 668ad51ee263768fd62850b10ce218b113f8e59b is ready for review with PR Quick, CodeQL, Codec Mobile Evidence, and Codec Padding Security Evidence green.

Closes #311.

This PR adds compile-time RpcCodecRoute / RpcCodecScope routing with assembly-owned codec routing.

Ownership model

The RPC Contract remains the RPC identity boundary, but the Contract assembly is the codec-route configuration, ownership, publication, and freeze boundary.

For one generated Contract assembly:

Contract assembly × closed CLR payload type T
    -> exactly one final RPC Codec binding

All [RpcContract] interfaces emitted from the same Contract assembly therefore share the same immutable final Codec graph. Two Contracts in the same assembly do not independently select different codecs for the same closed T.

Different Contract assemblies may select different codec routes/policies.

Public routing surface

RpcCodecRoute is assembly-scoped. Contract/interface-local route configuration is intentionally not part of the final API.

Selection precedence remains deterministic:

explicit custom IRpcCodec<T>
> explicit RpcCodecAdapter / serializer selector
> matching assembly RpcCodecRoute
> SharpLink generated/default Codec
> builtin/native Codec
> eligible unmanaged fallback
> unsupported diagnostic

Conflicting explicit selections or overlapping assembly routes with incompatible adapters produce deterministic compile-time diagnostics.

Generator / manifest / runtime model

The generator computes and publishes one assembly-owned final RPC Codec graph for the Contract assembly. Generated proxies/stubs bind to that graph at construction/registration time; endpoint runtime codec overrides cannot mutate RPC wire semantics after the assembly graph is frozen.

Contract Type remains the identity used for RPC method/service lookup, but it is not an independent codec-policy ownership key. Runtime resolution may start from a Contract Type and resolve to its owning assembly graph; this must not imply separate same-assembly codec policy.

Legacy per-Contract/API-5 artifacts such as ContractCodecSets and same-assembly distinct codec-provider selection are transitional implementation history and are not part of the final target model.

Structural and ownership validation

Generated manifest validation is assembly-owned. Manifest codec metadata must belong to the manifest owner assembly and must be structurally valid before preparation/adoption. Dynamic registration and direct/catalog RuntimeContext paths use the same assembly-level validation boundary.

Compatibility and lifecycle

Compatibility identity for RPC codec bindings is assembly-scoped:

(Contract assembly, T)
    -> Kind / SchemaId / WireFormatId

Generated dependency closure is frozen with the assembly-owned graph so dynamic registration/replacement/unload keeps adapter scopes and generated modules alive until calls drain.

Acceptance boundary

The final implementation must satisfy these invariants:

  • RpcCodecRoute configuration is assembly-level only.
  • One Contract assembly publishes/owns one final RPC Codec graph.
  • Same-assembly Contracts share that graph; there is no same-assembly per-Contract codec-policy split.
  • Different Contract assemblies may have different codec routes.
  • Contract Type remains the RPC identity boundary without becoming an independent codec-policy key.
  • Generated proxy/stub and static server automatic/replacement paths use the owning assembly's frozen codec graph.
  • Dynamic and direct/catalog RuntimeContext paths validate assembly-owned manifest metadata before adoption.
  • Endpoint/runtime codec overrides do not mutate frozen RPC wire semantics.
  • Dynamic generation, multi-cluster, streaming, package, NativeAOT, and unload/replacement lifecycle coverage remains intact.

Validation results will be recorded from the exact final head only after PR Quick, Release Gate, and CodeQL are green.

@SunSi12138 SunSi12138 closed this Aug 23, 2026
@SunSi12138 SunSi12138 reopened this Aug 23, 2026
@SunSi12138 SunSi12138 closed this Aug 23, 2026
@SunSi12138 SunSi12138 reopened this Aug 23, 2026
@SunSi12138 SunSi12138 closed this Aug 23, 2026
@SunSi12138 SunSi12138 reopened this Aug 23, 2026

@SunSi12138 SunSi12138 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found two correctness gaps worth addressing before merge: the Native builtin override is currently context-global rather than Contract-owner-scoped, and route selection still cannot fail at compile time when the selected adapter cannot create a codec for the closed payload type.

Comment thread src/SharpLink.Runtime/Codec/RpcCodecProvider.cs
Comment thread src/SharpLink.Generator/RpcGenerator.CodecRoutes.cs

@SunSi12138 SunSi12138 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up after rechecking the generated paths and #313: only the context-wide Native builtin leakage on response/stream paths remains as a correctness concern here. The closed-T adapter capability comment is withdrawn and resolved; SharpPack-specific compile-time capability/formatter validation belongs to #313, with #311's acceptance wording needing alignment to that split.

@SunSi12138 SunSi12138 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found one additional owner-scoping correctness gap on the final head: owner scoping needs to propagate through native generated codec dependency construction, not only the outer artifact binding.

Comment thread src/SharpLink.Runtime/Codec/RpcManifestCodecProvider.cs Outdated

Copy link
Copy Markdown
Owner Author

Handoff / 当前工作状态(2026-08-26)

这是给下一段对话/接手者的 canonical continuation checkpoint。请以这里记录的目标和当前 exact head 为准,不要从历史 detour 推断最终设计。

1. Repo / PR / exact head

  • Repo: SunSi12138/SharpLink
  • PR: Add compile-time RPC codec scope routing #320Add compile-time RPC codec scope routing
  • Branch: fix/311-codec-scope-routing
  • Current exact head: e30bf93b9a2bb8dc5391c6f965f4fe2c6464cbcb
  • PR remains open. Do not merge without explicit authorization.
  • Historical commit 709eccf7e971ec980b6e1527cb5493372574ff8e had green CI but was the per-Contract detour; it is not final validation and must not be used as the target architecture.

2. Canonical architecture target

The target is assembly-owned RPC codec routing:

(Contract assembly, closed CLR payload type T)
    -> one final RPC codec binding

Invariants:

  • RpcCodecRoute configuration is assembly-level only.
  • A Contract assembly owns/publishes/freezes one codec graph.
  • Contracts in the same assembly share the graph and cannot independently choose different codecs for the same closed T.
  • Different Contract assemblies may use different routes.
  • Contract Type remains RPC identity / lookup input, but it is not the codec-policy ownership key.
  • Generated proxy/stub/static server and dynamic/direct/catalog paths must resolve the owning assembly graph consistently.
  • Dynamic/direct manifest adoption must validate metadata against the manifest owner assembly.
  • Legacy/API-5 ContractCodecSets and same-assembly per-Contract provider selection are transitional/history, not the final ownership model.

3. Open API decision: UseCodec<T>

A design inconsistency was identified: if the generated assembly graph is already frozen, a runtime-facing UseCodec<T> API that appears to replace RPC codecs becomes misleading if it cannot affect generated RPC wire behavior.

Current preferred direction is remove generated-RPC runtime UseCodec<T> override capability rather than make arbitrary endpoint/client per-T overrides part of the model. If hard removal is too breaking for the current version, obsolete/migrate it explicitly; do not leave a public API that silently does nothing for generated RPC.

If runtime deployment customization is later required, the narrower acceptable model is:

assembly defaults / profile
    -> server/client selects/replaces whole assembly graph before Build()
    -> Build() freezes runtime context

Same-assembly Contracts still cannot diverge inside one runtime context. Avoid post-Build hot reload and arbitrary per-Contract/per-T endpoint override. In that model compatibility identity becomes closer to (assembly, codec-profile, T).

Before implementing API removal, inspect the exact current public definitions/usages in SharpLinkServerBuilder and serializer-builder paths so manual/non-generated serialization behavior is not accidentally removed.

4. Current CI status on exact head

At e30bf93b9a2bb8dc5391c6f965f4fe2c6464cbcb:

  • CodeQL: ✅ success — run 32954485173
  • PR Quick: ❌ failure — run 32954485100
    • Build: ✅
    • Unit Tests: ✅
    • Generator Tests: ❌ current blocker
    • failing Generator Tests job: 98133020055
  • Release Gate: ❌ failure — run 32954485151

Progress before this checkpoint: failures were reduced from a large regression set to 5 unit failures, then 1, then a compile issue; the current head has both Build and Unit Tests green. The next conversation should inspect Generator Tests job 98133020055 first and fix only the exact failure found there. Do not guess/invert tests merely to make CI green.

5. Two unresolved P1 review threads

There are currently two unresolved P1 threads. Both originated from the older API-5/per-Contract direction and should not be implemented literally against the restored assembly-owned architecture.

  1. Thread PRRT_kwDOROK3ys6cTTRB, inline comment 2925944492src/SharpLink.Runtime/SharpLinkDynamicModule.cs

    • Original request concerned ContractCodecSets ownership/structural validation and foreign Contract Type hijacking.
    • Final assembly-owned interpretation: dynamic/direct adoption validates manifest data against the manifest owner assembly; per-Contract Type is not an independent codec-policy ownership slot.
  2. Thread PRRT_kwDOROK3ys6cTVS1, inline comment 2925946075src/SharpLink.Server/SharpLinkServerBuilder.cs

    • Original request expected Contract-Type provider selection so same-assembly Contract A/B can use different codecs.
    • That behavior is intentionally superseded: same-assembly Contracts share one assembly codec graph.

Do not resolve these threads until final semantics are implemented and exact-final-head CI evidence is available. Then reply with the superseding architecture and regression/CI evidence before resolving.

6. Files most likely relevant next

Production/API:

  • src/SharpLink.Server/SharpLinkServerBuilder.cs
  • src/SharpLink.Runtime/Codec/RpcManifestCodecProvider.cs
  • src/SharpLink.Runtime/SharpLinkRuntimeContext.cs
  • src/SharpLink.Runtime/RpcGeneratedCodecResolver.cs
  • src/SharpLink.Runtime/RpcCodecScopeRuntime.cs
  • src/SharpLink.Runtime/RpcGeneratedManifestCatalog.cs
  • src/SharpLink.Runtime/SharpLinkDynamicModule.cs
  • src/SharpLink.Runtime/SharpLinkGeneratedManifestStructureValidator.cs
  • src/SharpLink.Abstractions/ISharpLinkGeneratedAssemblyManifest.cs
  • src/SharpLink.Abstractions/AssemblyCodecScopeDescriptor.cs
  • src/SharpLink.Abstractions/IRpcCodecScope.cs
  • generator RpcGenerator.*Codec*, ManifestEmitter, ProxyEmitter, ServerEmitter, MethodsEmitter, Symbols, Diagnostics files.

Relevant tests include:

  • test/SharpLink.UnitTests/Runtime/RpcCodecPolicyRegressionTests.cs
  • test/SharpLink.UnitTests/Runtime/RpcCodecRouteRuntimeTests.cs
  • test/SharpLink.UnitTests/Runtime/RpcEnumCodecOverrideRegressionTests.cs
  • test/SharpLink.UnitTests/Runtime/RpcManifestCodecOwnershipRegressionTests.cs
  • test/SharpLink.UnitTests/Runtime/SharpLinkRuntimeContextTests.cs
  • test/SharpLink.UnitTests/Builder/SerializerBuilderTests.cs

7. PR-body inconsistency to clean up

The current PR body still has a precedence line roughly saying:

explicit custom IRpcCodec<T>
> explicit RpcCodecAdapter / serializer selector
> matching assembly RpcCodecRoute
> ...

while the same body says endpoint/runtime overrides cannot mutate frozen generated RPC wire semantics. If those “explicit” entries refer to runtime builder configuration, the two statements conflict. Once UseCodec<T> lifecycle/scope is finalized, update the PR body so the precedence and freeze model describe one coherent API.

8. Recommended continuation order

  1. Inspect/fix the exact Generator Tests failure from job 98133020055 on e30bf93....
  2. Run PR Quick / Release Gate / CodeQL on the resulting exact head and record the results.
  3. Inspect UseCodec<T> and serializer-selector public API/usages; implement the chosen removal/obsoletion cleanly for generated RPC while preserving any deliberately supported manual/non-generated behavior.
  4. Rewrite/delete runtime-override tests that are invalid under that API decision; do not merely flip expectations without explaining the lifecycle boundary.
  5. Update PR body precedence/freeze wording.
  6. Re-run all required CI on the exact final head.
  7. Reply to and resolve the two P1 threads only with final semantics + exact-head evidence.
  8. Do not merge unless explicitly authorized.

9. Commit/worktree note

Work in this session has been applied through the GitHub connector, which commits changes directly to the branch. There is no local SharpLink working tree in this execution environment containing additional unstaged/staged edits to preserve. Therefore no empty/checkpoint commit was created solely for handoff; the remote branch itself is the committed checkpoint at e30bf93b9a2bb8dc5391c6f965f4fe2c6464cbcb.

SunSi12138 commented Aug 27, 2026

Copy link
Copy Markdown
Owner Author

Status update: this PR is now superseded by #386.

#320 is being retained as the research/review history for the codec-routing work, including the API-5 per-Contract experiments and the review iterations that led to the final ownership model. It should not be merged.

The final implementation moved to the clean, single-parent port in #386, which uses the assembly-owned codec-routing model and intentionally does not carry forward the rejected transitional/API-5 machinery from this branch.

As of #386 head 668ad51ee263768fd62850b10ce218b113f8e59b, PR Quick, CodeQL, Codec Mobile Evidence, and Codec Padding Security Evidence are all green, and the PR is open/mergeable/ready for review.

Closing #320 as superseded; keeping it available as implementation and review history.

@SunSi12138 SunSi12138 closed this Aug 27, 2026
SunSi12138 added a commit that referenced this pull request Aug 30, 2026
* feat: port assembly-owned codec routing onto the current dev line

Selectively port the final assembly-owned Codec routing design from PR
#320 (fix/311-codec-scope-routing) onto dev. Per-Contract/API-5
transitional machinery (ContractCodecSets, per-Contract publication,
same-assembly per-Contract policy splits) is deliberately not carried
over; the PR body's final ownership contract is the spec.

- Public API: assembly-only RpcCodecRoute/RpcCodecScope; singular
  ContractCodecs/ContractDependencies on the generated assembly
  manifest; IRpcContractCodecProviderResolver;
  RpcGeneratedCodecFactoryKind; codec-aware IRpcClientStreamSink overload.
- Generator: assembly-owned final Codec graph (explicit > route >
  default), frozen graph published in the assembly manifest, proxy/stub
  construction-time binding, and implicit final-selection identities
  (Native/UnsafeBlit) materialized into format-2 compatibility manifests
  so UnsafeBlit <-> Adapter transitions are wire breaks.
- Runtime: RpcManifestCodecProvider as the per-assembly owner provider,
  RpcGeneratedCodecResolver, assembly-level manifest structure
  validation, and the request-level stream drain barrier.
- Client/Server: proxy/stub registration binds the owning assembly
  provider; multi-cluster dependency closure includes
  ContractDependencies; builder UseCodec<T> removed; dynamic replacement
  validates assembly-owned dependency closure.
- Tests: assembly-owned routing/compatibility/ownership suites ported;
  per-instance codec-independence and same-assembly split tests removed;
  fixture projects are real generated contract assemblies.

Local validation (net10.0, Release): Unit 1223/1223, Generator 171/171,
LoadTest 60/60, Integration 344/344.

* test: preload generated fixture assemblies to stabilize catalog snapshots

The generated contract fixtures register their manifests into the global
catalogs from a module initializer. Lazy mid-suite loads raced the
weak-catalog tests that snapshot and restore catalog counts, so force
the fixture assemblies to load before any test executes.

* test: harden generated bridge async waits on loaded CI runners

* ci: pin browser desktop verification to producer SDK

* fix: preserve assembly ownership for custom RPC codecs

Address the review findings on #386:
- classify adapter-free non-native generated factories as direct/custom construction
- keep Contract-only [RpcCodec] bindings out of the global graph while freezing them into ContractCodecs
- publish ContractDependencies as the RPC-only dependency delta
- add generator regressions covering custom ownership, standalone behavior, and dependency separation

* test: place generated factory kind regression in runtime suite

Keep Generator.Tests independent from the real Abstractions project. The generator-only ownership/dependency regressions remain there, while the real IRpcGeneratedCodecFactory default-kind behavior is covered by UnitTests.

* fix(client): validate RPC-only manifest dependencies

* test(client): cover RPC-only dynamic dependencies

* style: restore final newline in client registration source

* test(client): import client builder namespace

* test(generator): distinguish CLR refs from module dependencies

* fix: bump generated descriptor ABI identity

* fix: emit current generated ABI identity

* fix: protect contract dependencies during client unregister

* fix: keep custom codec helpers out of module dependencies

* test: keep custom codec helpers outside module closure

* test: cover contract dependency drain and stale ABI rejection

* chore: keep ABI fix diff focused

* test: add LINQ import for reflection helper

* test: isolate stale ABI compatibility branch

* test(generator): align ABI fixtures with current descriptor

* fix(generator): align route classification with custom codecs

* fix(generator): tighten codec policy ownership and dependencies

* fix(generator): derive module dependencies from generated graph

* fix(generator): let explicit builtin policy outrank native routes

* test(generator): cover second codec policy review regressions

* test(generator): retire ambient codec policy assumptions

* fix(generator): align final codec ownership boundaries

* test(generator): cover final codec ownership review cases

* fix(generator): keep owner policy out of contract defaults

* fix(generator): reuse existing contract ownership probe

* fix(generator): keep contract custom fallbacks out of global codecs

* test(generator): stop owning manifestless foreign contracts

* fix: derive RPC behavior from final graphs

* test(generator): lock final codec for top-level requests

* fix(generator): canonicalize tuple codec identities

* fix(generator): close final codec identity and compatibility gaps

* test: isolate global codec catalog regression

* style: restore final newline

* fix(runtime): fail closed for implicit-only generated owners

* fix(generator): close late review codec identity gaps

* test(generator): cover fourth review codec identity regressions

* test(runtime): cover implicit-only late owner fail-closed

* test(runtime): avoid unassigned-field warning

* test(generator): match canonical review types robustly

* test(generator): resolve nullable codec by manifest type identity

* fix(generator): close final codec review gaps

* test(generator): cover final codec review gaps

* refactor(codec): remove Native route scope

* refactor(codec): define framework wire primitive boundary

* refactor(codec): route only configurable payloads

* refactor(codec): freeze framework primitive policy

* test(codec): make enum wire policy fixed

* test(codec): route configurable composites not primitives

* test(codec): align route suite with configurable scopes

* test(codec): fix compatibility boundary for framework primitives

* test(codec): keep framework enums outside routes

* test(codec): use configurable payloads for owner binding

* test(codec): reject framework primitive rebinding

* test(codec): remove primitive custom policy surface

* test(codec): keep primitive request framing fixed

* test(codec): reject primitive element rebinding

* test(codec): keep enum compatibility framework-owned

* test(codec): lock framework wire primitive boundary

* docs(codec): document fixed framework wire primitives

* refactor(codec): freeze nullable framework primitives

* test(codec): freeze nullable primitive wrappers

* test(generator): use direct-codec fixture for tuple alias

* refactor(codec): make RpcCodecAdapter adapter-only

* test(generator): remove direct codec alias coverage

* refactor(codec): replace Direct factory kind with Custom

* refactor(generator): remove Direct codec kind

* refactor(generator): remove native route shim and direct adapter model

* refactor(generator): simplify codec ownership and UnsafeBlit identity

* refactor(generator): make adapter analysis adapter-only

* refactor(runtime): validate custom codec factory kind

* refactor(runtime): replace direct factory semantics with custom

* docs(codec): document single custom codec binding API

* refactor(generator): remove direct codec emitter

* refactor(generator): remove direct manifest kind

* test(generator): keep UnsafeBlit identity wire-only

* refactor(generator): keep codec routing separate from compatibility identity

* test(codec): drop compatibility-only routing regressions

* docs(codec): defer stable identity model to #396

* fix(generator): preserve legacy codec manifest ownership

* test(generator): keep route coverage independent of compatibility identity

* test(generator): keep enum policy coverage routing-only

* test(generator): restore legacy manifest compatibility coverage

* style(generator): restore final newline

* fix(generator): keep legacy type-level adapter defaults

* fix(generator): keep explicit type adapters contract-owned

* fix(generator): preserve adapter selection diagnostic

* refactor(codec): remove public generated factory kind

* refactor(runtime): validate adapter-backed factories structurally

* refactor(runtime): infer adapter-backed factories from metadata

* refactor(generator): stop emitting runtime factory kind

* test(runtime): remove public factory kind coverage

* test(runtime): keep custom factory adapter-free

* fix(generator): keep factory emission adapter-metadata-only

* test(server): keep custom factory adapter-free

* fix(generator): keep adapter selection diagnostics structured

* fix(generator): stop parsing diagnostic presentation text

* fix(generator): preserve explicit adapter ownership

* fix(generator): retain identical default codec publication

---------

Co-authored-by: sunsi <suns12138@163.com>
@SunSi12138
SunSi12138 deleted the fix/311-codec-scope-routing branch September 9, 2026 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[codec][generator] Simplify assembly-owned codec routing

1 participant