Skip to content

[Enhancement]: PII content redaction plugin improvements #525

Description

@bbednarski9

Affected area

  • Plugins
  • Middleware or guardrails
  • Observability or exporters
  • Documentation or examples

Problem or opportunity

Composable PII redaction profiles added in #512 always own the complete sanitization boundary. PiiRedactionProfile exposes only enabled, mode, priority, builtin, and local; profile_as_legacy_config currently forces input, output, mark, tool_input, and tool_output to true for every enabled profile.

Profile-array validation also rejects those legacy surface fields. This creates three limitations:

  1. A composed policy cannot apply different profiles to different observability surfaces.
  2. A component-wide LLM codec is required even when the intended profiles only need tool, mark, or generic-context sanitization.
  3. Generic non-LLM/non-tool scope sanitization is coupled to the LLM/tool input and output callbacks. Merely exposing the existing flags per profile would therefore still not allow an LLM-and-tool-only policy that preserves generic scope context.

The third limitation is visible with preset = "trajectory_context": enabling the LLM and tool surfaces also sanitizes unrelated scope metadata, including classification fields. See #528.

The current mark surface also contains semantically related events such as llm.chunk, nemo_relay.llm.optimization, and skill.load. A surface model must not expose streaming or tool content merely because a user selected a high-level llm or tool group without a literal mark flag.

Legacy single-policy configuration supports some surface selection, so migrating to profile composition loses an existing capability. It does not, however, independently control generic context.

Proposed enhancement

Add explicit per-profile surface selection and make generic trajectory context independently selectable from managed LLM and tool content.

One possible high-level shape is:

[[components.config.profiles]]
enabled = true
mode = "builtin"
priority = 80
surfaces = ["llm", "tool"]

[components.config.profiles.builtin]
preset = "trajectory_context"

An equivalent typed object with directional controls would also work:

[components.config.profiles.surfaces]
llm_input = true
llm_output = true
tool_input = true
tool_output = true
generic_scope_start = false
generic_scope_end = false
mark = false

The exact serialized shape can follow the repository's schema and editor conventions, but it should provide these semantics:

  • LLM request and response sanitization can be selected independently.
  • Tool input and output sanitization can be selected independently.
  • Generic non-LLM/non-tool scope data and metadata can be selected independently from the LLM and tool surfaces.
  • Remaining/custom mark sanitization can be selected independently.
  • If high-level semantic groups such as llm and tool are exposed, they include their associated content-bearing marks, such as llm.chunk, so buffered and streaming privacy behavior remains consistent.
  • Omitted surface configuration defaults to the current all-surfaces behavior, preserving feat(pii)!: support composable redaction profiles #512 compatibility and the existing privacy boundary.
  • Changing the omitted default to only LLM and tool surfaces requires an explicit schema or preset version change rather than silently weakening existing configurations.
  • Every enabled profile must select at least one effective surface.
  • The component-wide codec is required only when at least one enabled profile selects a codec-dependent LLM surface.
  • Priority ordering, array-order tie-breaking, disabled-profile validation, and atomic rollback remain unchanged.
  • If a preset requires specific surfaces for its privacy guarantee, validation and documentation state that constraint explicitly.

This enhancement controls where a profile runs. The separate correctness issue in #528 controls which trusted structural fields trajectory_context must preserve when generic context is selected.

Runtime contract and binding impact

The serialized profile contract should remain snake_case and consistent across Rust, Python, Node.js, and experimental bindings that expose PII redaction configuration helpers.

This changes the Rust PiiRedactionProfile public configuration type and its JSON Schema/editor representation. Language helpers, generated schemas, and documentation should expose the same defaults and validation semantics.

Existing profile-array configurations that omit surface selection must retain their current behavior. Buffered and streaming events must resolve to the same semantic surface policy.

Alternatives considered

  • Keep one all-surfaces component: works for simple policies but cannot express surface-specific compositions.
  • Expose only the existing five legacy flags per profile: improves composition but leaves generic scope sanitization coupled to LLM/tool selection.
  • Treat all marks as a completely separate opt-in surface: risks leaking llm.chunk or other semantically associated content when users select LLM/tool groups.
  • Default new surface lists to LLM and tool only in schema version 1: silently weakens existing privacy behavior.
  • Use multiple top-level pii_redaction components: unsupported because the component is a singleton.
  • Use raw middleware for surface-specific behavior: possible, but loses the typed, validated, composable first-party PII policy contract.
  • Keep using legacy mode: preserves the original flags but does not support multiple ordered policies and still couples generic scope events to LLM/tool callbacks.

Acceptance criteria

  • An enabled PII redaction profile can select LLM input, LLM output, tool input, and tool output independently.
  • Generic non-LLM/non-tool scope start and end sanitization can be selected independently from LLM and tool surfaces.
  • Remaining/custom mark sanitization can be selected independently.
  • High-level LLM/tool groups, if provided, include their associated content-bearing marks so streaming and buffered privacy behavior is equivalent.
  • Omitting per-profile surface selection preserves the current all-surfaces behavior.
  • Validation rejects an enabled profile with no effective selected surfaces.
  • codec is required exactly when an enabled profile selects a codec-dependent LLM surface.
  • Registration installs callbacks only for the effective surfaces selected by each profile.
  • Profile priority ordering, array-order tie-breaking, disabled profiles, and rollback behavior remain correct.
  • Tests cover mixed-surface profiles, generic scope independence, content-bearing marks, conditional codec requirements, and compatibility with existing profile arrays.
  • CLI editor/schema support and relevant binding helpers expose the new configuration.
  • PII redaction documentation includes a composed LLM-and-tool-only example and explains the backward-compatible default.

Relates to #512, #513, and #528.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions