[BUG]: Define a consistent missing/malformed-attribute policy for APL #145

Context

This came from the review of PR #114 (restrict effect). The review noted that missing or malformed attributes tend to result in "unconstrained" rather than "deny" in several places, and that — individually minor, together — this weakens APL's fail-closed story. We need to define a consistent policy, potentially including load-time shape validation and guidance that when-gated restrictions need a default-deny companion.

This issue records the problem, a cited survey of how mature engines handle it, and a proposed direction — for comment before we build the enforcement pieces.

The problem

APL policies read attributes (subject.role, session.labels, data.agents[...].allowed_models, args.amount, …). At request time any can be missing (host didn't populate it, token lacked the claim) or malformed (present but the wrong shape). Today the deny behavior depends on which code path hits the missing value:

• require(X) / deny-lists → missing ⇒ deny (fail closed). require compiles to IsFalse(X), absent is falsy.
• Authorization comparisons → missing ⇒ comparison false ⇒ usually fail-closed for allow-lists (absent role won't satisfy role == 'admin').
• when-gated effects → missing ⇒ gate false ⇒ effect skipped ⇒ fail-open when the effect was protective. This is the problematic case.

Three concrete instances the review flagged:

1. deny_models fail-open — an unresolvable data.* deny-list reference resolved to an empty vec, which (if it was the only field) discarded the whole constraint → unconstrained routing. Fixed in feat: add restrict effect #114 (unresolvable deny reference now denies; see RestrictResolveError).
2. when-gated restrict skipped on a missing attribute — e.g. when: "subject.eu_resident" do: [restrict: {allow_regions: [eu]}]. If subject.eu_resident is absent, the gate is false, the restrict never fires, and the request routes unconstrained. The engine can't distinguish "absent" from "legitimately false," so a missing attribute silently removes a protection. Open.
3. Shape mismatches vanishing during bag flattening — a wrong-shaped data.*/args value is silently skipped by walk, so the key never lands in the bag and a comparison rule that depended on it never matches. Partially addressed in feat: add restrict effect #114 for restrict refs (a wrong-shape deny ref now denies); the general case is open.

How existing engines handle missing attributes

A cited survey of OPA/Rego, AWS Cedar, Google CEL, OASIS XACML, and AWS IAM. There is a convergent pattern — but it converges on goals, not a single mechanism, and the modern engines deliberately do not blanket-deny on a missing attribute.

The convergent goals:

• (a) Treat "missing" as a distinct value, not silently false — holds for Rego (undefined), Cedar/CEL (error), XACML (empty bag / Indeterminate); weak for IAM (mostly match/no-match; negated operators match a missing key — a polarity asymmetry that's the tell it's match-logic, not a real three-valued algebra).
• (b) Explicit presence-guard operators — universal (has, exists, MustBePresent, Null/...IfExists).
• (c) Fail-closed at the top-level default-deny, not per-attribute — universal.
• (d) Author/load-time schema validation to catch missing/mis-typed references — first-class in Cedar, type-checked in CEL; absent as an enforced gate in OPA and IAM.

Key nuance (important for our design): the two most modern engines deliberately drop, not deny, an errored attribute access. Cedar explicitly rejected deny-on-error on safety grounds ("one bad 101st policy could deny everything") — an erroring policy is skipped, so an erroring permit loses its grant and an erroring forbid loses its deny. CEL's commutative boolean algebra makes false && <error> evaluate to false. Safety lives at the top-level default-deny frame plus load-time validation, not in a blanket per-attribute deny. (NIST SP 800-162 mandates deny-by-default generally but does not prescribe deny-on-missing-attribute; SP 1800-3 treats missing attributes as an expected runtime condition.)

Sources: OPA · Cedar auth · Cedar validation · CEL spec · XACML 3.0 core · IAM condition operators · NIST SP 800-162

Where APL sits

• ✅ Presence guard — APL has exists() (the has analogue).
• ✅ Fail-closed require() — require(X) denies on absent (Rego-style at the rule level).
• ✅ Top-level posture — a phase with no allow and a trailing deny fails closed.
• ⚠️ No distinct "missing" value — eval_condition collapses absent into false (for IsTrue/comparison) or true (for IsFalse). This is why a when-gated protective effect silently skips: there's no Indeterminate/undefined to propagate.
• ⚠️ No load-time schema validation — the Cedar-style "catch it before runtime" piece is absent.

Proposed direction (for comment)

1. Document the posture (this issue → design doc): missing is a distinct outcome; authorization predicates treat absent as falsy (fail-closed for allow-lists); when-gated effects skip on an absent gate, which is fail-open for protective effects — so those need explicit handling.
2. Adopt the "default-deny companion" pattern as authoring guidance — a protective effect gated on a possibly-absent attribute should be paired so absence fails safe: a trailing default deny, an else branch that restricts, or require(exists(subject.eu_resident)) to force presence. This is literally Rego's default allow := false.
3. Explicitly do NOT adopt blanket deny-on-missing — the modern-engine consensus (Cedar's explicit rejection, CEL's error-drop) is against it; it trades an availability hazard for a fail-open one. Safety belongs at the top-level default-deny + presence guards + validation, not per-attribute.
4. Follow-up (separate issues), in rough priority:
   • Load-time / config-load shape validation for referenced data.* attributes (Cedar-style; note it's opt-in even in Cedar). Catches typos and wrong shapes before runtime.
   • Lint a when-gated protective effect (restrict/deny) that lacks a default-deny companion.
   • (Larger, optional) three-valued evaluation — give eval_condition a real Unknown/Indeterminate for a missing attribute instead of coercing to bool, propagate it through when gates, and have the top-level combiner treat it as deny unless the author guarded with exists(). This is the XACML/Cedar-validated model done properly — but Cedar/CEL deliberately chose error-and-drop over it, so it's a legitimate but non-consensus path.

Already handled in PR #114

• deny_models unresolvable reference now denies (fail-closed); literal deny-lists unaffected.
• A restrict string-set reference pointing at a wrong-shaped value now resolves to "unresolved" → denies for deny-lists.
• Unrankable max_cost_tier (backend and ceiling) fails closed in the matcher.

Questions for discussion

• Is authoring guidance + opt-in load-time validation the right stopping point for v1, or do we want the three-valued Indeterminate model as the principled fix?
• For load-time validation, what's the source of truth for expected attribute shapes — the attribute_files tree, an explicit schema block, or inferred from usage?
• Should a when-gated protective effect without a default-deny companion be a lint warning or a hard load error?