diff --git a/.agents/skills/generate-sandbox-policy/SKILL.md b/.agents/skills/generate-sandbox-policy/SKILL.md index cfdc14e1b5..741bee2b0e 100644 --- a/.agents/skills/generate-sandbox-policy/SKILL.md +++ b/.agents/skills/generate-sandbox-policy/SKILL.md @@ -384,6 +384,7 @@ Before presenting the policy to the user, verify correctness **and** flag breadt - [ ] Middleware `order` values are unique and no selected chain exceeds 10 stages - [ ] No fail-closed middleware selector can cover a `tls: skip` endpoint - [ ] Any required WebSocket control advertises `WEBSOCKET_MESSAGE/PRE_CREDENTIALS`, and the user understands that V1 does not inspect binary messages +- [ ] Endpoints contributed by a credentialed provider are not L4-only or `tls: skip` unless `allow_uninspected_credentials: true` explicitly records the exception ### Schema Warnings (log-only, but should be fixed) @@ -418,6 +419,7 @@ Evaluate the generated policy for overly broad access and **include warnings in | **Broad CIDR** in `allowed_ips` (e.g., `10.0.0.0/8`) | "This `allowed_ips` entry covers a very broad range. Consider narrowing to a specific subnet (e.g., `10.0.5.0/24`) to minimize exposure." | | **`on_error: fail_open`** | "This middleware can be bypassed when it is unavailable, rejects configuration, returns an invalid result, or exceeds its body limit. Use `fail_closed` unless availability is more important than this control." | | **Broad middleware host selector** | "This middleware attaches independently of the admitting network rule to every matching destination, then runs only for operation bindings its implementation advertises. Narrow `endpoints.include` or add exclusions if the attachment is not required for every matching host." | +| **`allow_uninspected_credentials: true`** | "This endpoint may carry provider credentials on traffic OpenShell cannot inspect or rewrite. Prefer an inspected protocol and credential rewrite; keep this exception only when raw traffic is required." | Format breadth warnings clearly in the output, e.g.: diff --git a/.agents/skills/openshell-cli/cli-reference.md b/.agents/skills/openshell-cli/cli-reference.md index ec529508de..2cd5881ab9 100644 --- a/.agents/skills/openshell-cli/cli-reference.md +++ b/.agents/skills/openshell-cli/cli-reference.md @@ -384,6 +384,7 @@ Incrementally merge live network policy changes into the current sandbox policy. Notes: - The sandbox name defaults to the last-used sandbox. +- `--add-endpoint` options are comma-separated: `allowed-ip=`, `websocket-credential-rewrite`, `request-body-credential-rewrite`, and `allow-uninspected-credentials`. The last option is a security-sensitive exception for provider-credentialed L4-only, `tls: skip`, or otherwise uninspectable traffic. - `--add-allow` and `--add-deny` operate on REST and WebSocket endpoints. Use full YAML for JSON-RPC, MCP, SQL, or other policy structure. - `--wait` cannot be combined with `--dry-run`. - Use `policy set` when replacing the full policy or changing static sections. diff --git a/architecture/gateway.md b/architecture/gateway.md index db8f2508e8..29155466dd 100644 --- a/architecture/gateway.md +++ b/architecture/gateway.md @@ -174,7 +174,7 @@ Supported auth modes: | Plaintext | Local development or a trusted reverse proxy boundary. | | Unauthenticated local users | Trusted Kubernetes dev or fully trusted proxy deployments only. | | Cloudflare JWT | Edge-authenticated deployments where Cloudflare Access supplies identity. | -| OIDC | Bearer-token auth for users, with browser PKCE or client credentials login. | +| OIDC | Bearer-token auth for users, with browser or device-code PKCE and client credentials login. | The CLI persists the scopes requested during OIDC login in gateway metadata and reuses them when refreshing an access token. This preserves the intended API diff --git a/architecture/security-policy.md b/architecture/security-policy.md index ff285a6a11..0cc3aa618d 100644 --- a/architecture/security-policy.md +++ b/architecture/security-policy.md @@ -97,6 +97,70 @@ raw relay by default. A `protocol: rest` endpoint can opt in to after an allowed `101` upgrade; server-to-client traffic and all other upgraded protocols remain raw passthrough. +## Credentialed Endpoints + +OpenShell keeps provider credentials on paths it can inspect or rewrite by +default. The gateway derives credential provenance from the attached providers +and stamps it onto the effective policy at composition time. This provenance is +internal, contains no credential identifiers or values, and is never trusted +from user-authored policy. + +Every evaluation clears provenance across the whole policy and re-derives it +from two sources: + +- the endpoints of attached provider profiles that carry credentials, and +- the valid `credential_binding` entries of the sandbox policy that name an + attached provider whose profile is endpointless. + +A binding reduces to a host and port scope only. Dropping the path is +deliberate: a path is not observable on an L4 or `tls: skip` endpoint, so a +path-scoped derivation would omit the marker on exactly the surfaces the +uninspected-credential gate exists to catch. A malformed binding — empty +provider, missing host, or a port outside `1..=65535` — fails the evaluation +instead of contributing a scope. Bindings naming an endpointful profile or an +unattached provider contribute nothing; the gateway rejects those uses +separately. + +Both sources merge into one deduplicated scope set, and each endpoint is +stamped once per evaluation from that set, so binding-derived scopes reach the +same gates as profile-derived ones. The stamp is an assignment, not an +accumulation, so an endpoint that stops matching a credentialed scope — or +whose binding was removed — loses its marker in the same pass. This must remain +a full recomputation: a delta-based derivation would let a series of +individually valid edits reach a state no single edit would have admitted. + +Credentialed L4-only and `tls: skip` endpoints fail policy validation unless the +public `allow_uninspected_credentials` escape hatch is explicitly enabled. The +flag defaults to `false` and is security-flagged in policy approval flows. +Incremental merges only ever add the flag to a matching endpoint; clearing it +requires removing the endpoint or replacing the policy. + +The network supervisor independently enforces the same boundary. Credentialed +WebSocket upgrades use the parsed relay, binary frames fail closed, and text +placeholders require rewrite. REST bodies can continue streaming when body +rewrite is disabled, but the relay withholds enough trailing bytes to detect a +placeholder split across reads before forwarding its marker. Explicitly opted-in +endpoints retain raw passthrough behavior. + +Denials emit both the relevant network activity and a detection finding. Events +identify only the destination, policy, and traffic surface; they never include +credential names, placeholders, body content, or secret values. + +Credential provenance is gateway-derived and deliberately absent from the policy +YAML schema, so it does not survive a policy that never transits the gateway. +Gateway-delivered policy is the authoritative source for this control, and a +policy without provenance applies neither the raw-tunnel refusal nor the +WebSocket binary-frame refusal. The request-body backstop still applies, because +it keys off the presence of a secret resolver rather than endpoint provenance. + +Two paths load a policy without provenance. A supervisor booting from a +container-image policy is a bounded window: that policy is resynchronized to the +gateway, which then serves a stamped effective policy. An explicit local Rego and +data override is permanent, because gateway revisions are observed for settings +and providers but never replace the local policy. When that override is combined +with injected provider credentials, the supervisor emits a high-severity +detection finding at startup naming the inactive controls. + ## Live Updates The gateway stores sandbox-authored policy revisions separately from derived diff --git a/crates/openshell-bootstrap/src/oidc_token.rs b/crates/openshell-bootstrap/src/oidc_token.rs index 3fa0730956..f21a115c80 100644 --- a/crates/openshell-bootstrap/src/oidc_token.rs +++ b/crates/openshell-bootstrap/src/oidc_token.rs @@ -39,6 +39,11 @@ pub fn oidc_token_path(gateway_name: &str) -> Result { Ok(user_gateway_dir(gateway_name)?.join("oidc_token.json")) } +/// Path to the one-shot marker that asks the next browser login to prompt. +pub fn oidc_login_prompt_required_path(gateway_name: &str) -> Result { + Ok(user_gateway_dir(gateway_name)?.join("oidc_login_prompt_required")) +} + /// Store an OIDC token bundle for a gateway. pub fn store_oidc_token(gateway_name: &str, bundle: &OidcTokenBundle) -> Result<()> { let path = oidc_token_path(gateway_name)?; @@ -50,6 +55,7 @@ pub fn store_oidc_token(gateway_name: &str, bundle: &OidcTokenBundle) -> Result< .into_diagnostic() .wrap_err_with(|| format!("failed to write OIDC token to {}", path.display()))?; set_file_owner_only(&path)?; + clear_oidc_login_prompt(gateway_name)?; Ok(()) } @@ -76,6 +82,33 @@ pub fn remove_oidc_token(gateway_name: &str) -> Result<()> { Ok(()) } +/// Mark the next interactive OIDC login as requiring a fresh `IdP` prompt. +pub fn request_oidc_login_prompt(gateway_name: &str) -> Result<()> { + let path = oidc_login_prompt_required_path(gateway_name)?; + ensure_parent_dir_restricted(&path)?; + std::fs::write(&path, b"1\n") + .into_diagnostic() + .wrap_err_with(|| format!("failed to write {}", path.display()))?; + set_file_owner_only(&path)?; + Ok(()) +} + +/// Return whether the next interactive OIDC login should request a fresh prompt. +pub fn oidc_login_prompt_required(gateway_name: &str) -> bool { + oidc_login_prompt_required_path(gateway_name).is_ok_and(|path| path.exists()) +} + +/// Clear the one-shot fresh-login marker for a gateway. +pub fn clear_oidc_login_prompt(gateway_name: &str) -> Result<()> { + let path = oidc_login_prompt_required_path(gateway_name)?; + if path.exists() { + std::fs::remove_file(&path) + .into_diagnostic() + .wrap_err_with(|| format!("failed to remove {}", path.display()))?; + } + Ok(()) +} + /// Check if the stored access token is expired or near expiry. /// /// Returns `true` if the token expires within the next 30 seconds. @@ -129,4 +162,36 @@ mod tests { assert!(remove_oidc_token("../escape").is_err()); }); } + + #[test] + fn oidc_login_prompt_marker_is_per_gateway() { + let tmp = tempfile::tempdir().unwrap(); + with_tmp_xdg(tmp.path(), || { + assert!(!oidc_login_prompt_required("alpha")); + assert!(!oidc_login_prompt_required("beta")); + + request_oidc_login_prompt("alpha").unwrap(); + + assert!(oidc_login_prompt_required("alpha")); + assert!(!oidc_login_prompt_required("beta")); + + let bundle = OidcTokenBundle { + access_token: "token".to_string(), + refresh_token: None, + expires_at: None, + issuer: "https://issuer.example.com".to_string(), + client_id: "openshell-cli".to_string(), + }; + store_oidc_token("alpha", &bundle).unwrap(); + + assert!(!oidc_login_prompt_required("alpha")); + + request_oidc_login_prompt("alpha").unwrap(); + assert!(oidc_login_prompt_required("alpha")); + + clear_oidc_login_prompt("alpha").unwrap(); + + assert!(!oidc_login_prompt_required("alpha")); + }); + } } diff --git a/crates/openshell-cli/src/commands/gateway.rs b/crates/openshell-cli/src/commands/gateway.rs index acf87a1466..7a3ebf01f0 100644 --- a/crates/openshell-cli/src/commands/gateway.rs +++ b/crates/openshell-cli/src/commands/gateway.rs @@ -905,6 +905,26 @@ pub async fn gateway_add( false } } + } else if is_browser_suppressed() { + match crate::oidc_auth::oidc_device_code_flow( + issuer, + oidc_client_id, + oidc_audience, + oidc_scopes, + gateway_insecure, + ) + .await + { + Ok(bundle) => { + openshell_bootstrap::oidc_token::store_oidc_token(name, &bundle)?; + eprintln!("{} Authenticated via device code", "✓".green().bold()); + true + } + Err(e) => { + eprintln!("{} Authentication failed: {e}", "!".yellow()); + false + } + } } else { match crate::oidc_auth::oidc_browser_auth_flow( issuer, @@ -912,6 +932,7 @@ pub async fn gateway_add( oidc_audience, oidc_scopes, gateway_insecure, + false, ) .await { @@ -1116,6 +1137,8 @@ pub async fn gateway_login(name: &str, gateway_insecure: bool) -> Result<()> { .unwrap_or("openshell-cli"); let audience = metadata.oidc_audience.as_deref(); let scopes = metadata.oidc_scopes.as_deref(); + let force_fresh_login = + openshell_bootstrap::oidc_token::oidc_login_prompt_required(name); let bundle = if std::env::var("OPENSHELL_OIDC_CLIENT_SECRET").is_ok() { crate::oidc_auth::oidc_client_credentials_flow( @@ -1126,6 +1149,15 @@ pub async fn gateway_login(name: &str, gateway_insecure: bool) -> Result<()> { gateway_insecure, ) .await? + } else if is_browser_suppressed() { + crate::oidc_auth::oidc_device_code_flow( + issuer, + client_id, + audience, + scopes, + gateway_insecure, + ) + .await? } else { crate::oidc_auth::oidc_browser_auth_flow( issuer, @@ -1133,12 +1165,14 @@ pub async fn gateway_login(name: &str, gateway_insecure: bool) -> Result<()> { audience, scopes, gateway_insecure, + force_fresh_login, ) .await? }; let username = jwt_preferred_username(&bundle.access_token); openshell_bootstrap::oidc_token::store_oidc_token(name, &bundle)?; + openshell_bootstrap::oidc_token::clear_oidc_login_prompt(name)?; if let Some(user) = username { eprintln!( @@ -1184,6 +1218,7 @@ pub fn gateway_logout(name: &str) -> Result<()> { match metadata.auth_mode.as_deref() { Some("oidc") => { openshell_bootstrap::oidc_token::remove_oidc_token(name)?; + openshell_bootstrap::oidc_token::request_oidc_login_prompt(name)?; } Some("cloudflare_jwt") => { openshell_bootstrap::edge_token::remove_edge_token(name)?; @@ -1430,6 +1465,9 @@ fn remove_gateway_registration(name: &str) { if let Err(err) = openshell_bootstrap::oidc_token::remove_oidc_token(name) { tracing::debug!("failed to remove oidc token: {err}"); } + if let Err(err) = openshell_bootstrap::oidc_token::clear_oidc_login_prompt(name) { + tracing::debug!("failed to clear oidc login prompt marker: {err}"); + } if let Err(err) = remove_gateway_metadata(name) { tracing::debug!("failed to remove gateway metadata: {err}"); } diff --git a/crates/openshell-cli/src/main.rs b/crates/openshell-cli/src/main.rs index 7cefd3669a..fc7728c15d 100644 --- a/crates/openshell-cli/src/main.rs +++ b/crates/openshell-cli/src/main.rs @@ -1172,7 +1172,8 @@ enum GatewayCommands { /// Authenticate with an edge-authenticated or OIDC gateway. /// /// Opens a browser for the edge proxy's login flow and stores the - /// token locally. Use this to re-authenticate when a token expires. + /// token locally. After `gateway logout`, OIDC browser login requests a + /// fresh identity-provider prompt so you can switch users. #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] Login { /// Gateway name (defaults to the active gateway). @@ -1183,7 +1184,9 @@ enum GatewayCommands { /// Clear stored authentication credentials for a gateway. /// /// Removes the locally stored OIDC token or edge token so subsequent - /// commands require re-authentication via `gateway login`. + /// commands require re-authentication via `gateway login`. For OIDC + /// gateways, the next browser login asks the identity provider for a fresh + /// login instead of silently reusing an existing browser session. #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] Logout { /// Gateway name (defaults to the active gateway). @@ -1791,6 +1794,8 @@ enum PolicyCommands { name: Option, /// Add or merge an endpoint: host:port[:access[:protocol[:enforcement[:options]]]]. + /// Options include allowed-ip=..., credential rewrite flags, and + /// allow-uninspected-credentials. #[arg(long = "add-endpoint")] add_endpoints: Vec, diff --git a/crates/openshell-cli/src/oidc_auth.rs b/crates/openshell-cli/src/oidc_auth.rs index 2aacdb0c9c..2a5c41a5fa 100644 --- a/crates/openshell-cli/src/oidc_auth.rs +++ b/crates/openshell-cli/src/oidc_auth.rs @@ -3,10 +3,10 @@ //! OIDC authentication flows for CLI gateway login. //! -//! Implements Authorization Code + PKCE (interactive browser flow) and -//! Client Credentials (CI/automation) `OAuth2` grant types against a -//! Keycloak-compatible OIDC provider. - +//! Implements Authorization Code + PKCE (interactive browser flow), +//! Device Authorization Grant (headless flow), and Client Credentials +//! (CI/automation) `OAuth2` grant types against a Keycloak-compatible +//! OIDC provider. use bytes::Bytes; use http_body_util::Full; use hyper::service::service_fn; @@ -14,7 +14,7 @@ use hyper::{Method, Response, StatusCode}; use hyper_util::rt::{TokioExecutor, TokioIo}; use hyper_util::server::conn::auto::Builder; use miette::{IntoDiagnostic, Result}; -use oauth2::basic::BasicClient; +use oauth2::basic::{BasicClient, BasicTokenResponse}; use oauth2::{ AuthType, AuthUrl, AuthorizationCode, ClientId, ClientSecret, CsrfToken, PkceCodeChallenge, RedirectUrl, Scope, TokenResponse, TokenUrl, @@ -37,6 +37,31 @@ struct OidcDiscovery { issuer: String, authorization_endpoint: String, token_endpoint: String, + device_authorization_endpoint: Option, +} + +/// Device authorization response from the provider. +#[derive(Debug, Deserialize)] +struct DeviceAuthorizationResponse { + device_code: String, + user_code: String, + verification_uri: String, + verification_uri_complete: Option, + expires_in: u64, + #[serde(default = "default_interval")] + interval: u64, +} + +fn default_interval() -> u64 { + 5 +} + +/// Device token polling error responses (RFC 8628). +#[derive(Debug, Deserialize)] +struct DeviceTokenErrorResponse { + error: String, + #[serde(default)] + error_description: String, } /// Discover OIDC endpoints from the issuer's well-known configuration. @@ -96,6 +121,55 @@ fn build_ci_scopes(scopes: Option<&str>) -> Vec { .collect() } +fn interactive_authorization_params( + audience: Option<&str>, + force_fresh_login: bool, +) -> Vec<(&'static str, String)> { + let mut params = Vec::new(); + if force_fresh_login { + params.push(("prompt", "login".to_string())); + } + if let Some(aud) = audience { + params.push(("audience", aud.to_string())); + } + params +} + +fn device_authorization_form( + client_id: &str, + scopes: &str, + audience: Option<&str>, + code_challenge: &str, + code_challenge_method: &str, +) -> Vec<(&'static str, String)> { + let mut params = vec![ + ("client_id", client_id.to_string()), + ("scope", scopes.to_string()), + ("code_challenge", code_challenge.to_string()), + ("code_challenge_method", code_challenge_method.to_string()), + ]; + if let Some(audience) = audience { + params.push(("audience", audience.to_string())); + } + params +} + +fn device_token_form( + client_id: &str, + device_code: &str, + code_verifier: &str, +) -> Vec<(&'static str, String)> { + vec![ + ( + "grant_type", + "urn:ietf:params:oauth:grant-type:device_code".to_string(), + ), + ("device_code", device_code.to_string()), + ("client_id", client_id.to_string()), + ("code_verifier", code_verifier.to_string()), + ] +} + /// Run the OIDC Authorization Code + PKCE browser flow. /// /// Opens the user's browser to the Keycloak login page and waits for @@ -106,6 +180,7 @@ pub async fn oidc_browser_auth_flow( audience: Option<&str>, scopes: Option<&str>, insecure: bool, + force_fresh_login: bool, ) -> Result { let discovery = discover(issuer, insecure).await?; @@ -130,10 +205,14 @@ pub async fn oidc_browser_auth_flow( let (mut auth_url, csrf_token) = auth_request.url(); - // Append audience parameter for providers like Entra ID where the API - // audience differs from the client ID. - if let Some(aud) = audience { - auth_url.query_pairs_mut().append_pair("audience", aud); + // After `gateway logout`, ask the IdP for a fresh login prompt so the user + // can switch browser identity. Ordinary repeated logins may reuse SSO. + let params = interactive_authorization_params(audience, force_fresh_login); + { + let mut query = auth_url.query_pairs_mut(); + for (key, value) in ¶ms { + query.append_pair(key, value); + } } let (tx, rx) = oneshot::channel::(); @@ -226,6 +305,162 @@ pub async fn oidc_client_credentials_flow( )) } +/// Run the OIDC Device Authorization Grant flow (RFC 8628). +/// +/// Prompts the user to visit a verification URL and enter a code on any device +/// with a browser. Polls the token endpoint until the user completes authorization +/// or the device code expires. +pub async fn oidc_device_code_flow( + issuer: &str, + client_id: &str, + audience: Option<&str>, + scopes: Option<&str>, + insecure: bool, +) -> Result { + let discovery = discover(issuer, insecure).await?; + + let device_auth_endpoint = discovery.device_authorization_endpoint.as_deref().ok_or_else(|| { + miette::miette!( + "The OIDC provider does not advertise a device_authorization_endpoint.\n\ + Enable the device authorization grant on this client, or use client credentials for headless automation." + ) + })?; + + // Step 1: Request device and user codes + let http = http_client(insecure); + let scopes_param = build_scopes(scopes) + .iter() + .map(|s| s.to_string()) + .collect::>() + .join(" "); + + // Use PKCE for the device flow as well as the browser flow. Keycloak + // requires these parameters when the public client enforces S256 PKCE. + let (pkce_challenge, pkce_verifier) = PkceCodeChallenge::new_random_sha256(); + let form_params = device_authorization_form( + client_id, + &scopes_param, + audience, + pkce_challenge.as_str(), + pkce_challenge.method().as_str(), + ); + + let device_auth_resp = http + .post(device_auth_endpoint) + .form(&form_params) + .send() + .await + .into_diagnostic()?; + + if !device_auth_resp.status().is_success() { + let status = device_auth_resp.status(); + let body = device_auth_resp.text().await.unwrap_or_default(); + return Err(miette::miette!( + "Device authorization request failed (status {status}): {body}" + )); + } + + let device_auth: DeviceAuthorizationResponse = + device_auth_resp.json().await.into_diagnostic()?; + + // Step 2: Display instructions to the user + eprintln!(); + eprintln!(" To authenticate, visit:"); + if let Some(uri_complete) = &device_auth.verification_uri_complete { + eprintln!(" {uri_complete}"); + } else { + eprintln!(" {}", device_auth.verification_uri); + eprintln!(); + eprintln!(" And enter this code:"); + eprintln!(" {}", device_auth.user_code); + } + eprintln!(); + eprintln!(" Waiting for authorization..."); + + // Step 3: Poll the token endpoint + let start_time = std::time::Instant::now(); + let expires_duration = Duration::from_secs(device_auth.expires_in); + let mut poll_interval = Duration::from_secs(device_auth.interval); + + loop { + if start_time.elapsed() >= expires_duration { + return Err(miette::miette!( + "Device code expired after {} seconds. Please try again.", + device_auth.expires_in + )); + } + + tokio::time::sleep(poll_interval).await; + + let token_params = + device_token_form(client_id, &device_auth.device_code, pkce_verifier.secret()); + + let poll_resp = http + .post(&discovery.token_endpoint) + .form(&token_params) + .send() + .await + .into_diagnostic()?; + + let status = poll_resp.status(); + + if status.is_success() { + // A successful HTTP status is not sufficient: require a valid OAuth + // token response before persisting credentials. + let token_response: BasicTokenResponse = poll_resp + .json() + .await + .map_err(|error| miette::miette!("invalid device token response: {error}"))?; + + return bundle_from_device_token_response(&token_response, issuer, client_id); + } + + // Parse error response + let error_resp: DeviceTokenErrorResponse = match poll_resp.json().await { + Ok(e) => e, + Err(_) => { + return Err(miette::miette!( + "Token polling failed with status {status} and unparseable response" + )); + } + }; + + match error_resp.error.as_str() { + "authorization_pending" => { + // Keep polling + debug!("Device authorization pending, continuing to poll"); + } + "slow_down" => { + // Increase polling interval per RFC 8628 + poll_interval += Duration::from_secs(5); + debug!( + "Received slow_down, increasing interval to {:?}", + poll_interval + ); + } + "access_denied" => { + return Err(miette::miette!( + "Authorization was denied by the user or administrator" + )); + } + "expired_token" => { + return Err(miette::miette!("Device code expired. Please try again.")); + } + _ => { + let desc = if error_resp.error_description.is_empty() { + String::new() + } else { + format!(": {}", error_resp.error_description) + }; + return Err(miette::miette!( + "Device authorization failed: {}{desc}", + error_resp.error + )); + } + } + } +} + /// Refresh an OIDC token using the `refresh_token` grant. /// /// Reuses the configured login scopes when supplied so providers can select @@ -308,7 +543,7 @@ pub async fn ensure_valid_oidc_token(gateway_name: &str, insecure: bool) -> Resu // ── Helpers ────────────────────────────────────────────────────────── fn bundle_from_oauth2_response( - resp: &oauth2::basic::BasicTokenResponse, + resp: &BasicTokenResponse, issuer: &str, client_id: &str, ) -> OidcTokenBundle { @@ -326,6 +561,20 @@ fn bundle_from_oauth2_response( } } +fn bundle_from_device_token_response( + resp: &BasicTokenResponse, + issuer: &str, + client_id: &str, +) -> Result { + if resp.access_token().secret().trim().is_empty() { + return Err(miette::miette!( + "invalid device token response: access_token is empty" + )); + } + + Ok(bundle_from_oauth2_response(resp, issuer, client_id)) +} + /// Percent-decode a URL query parameter value. fn percent_decode(s: &str) -> String { let mut out = Vec::with_capacity(s.len()); @@ -537,6 +786,82 @@ mod tests { assert!(scopes.is_empty()); } + #[test] + fn interactive_authorization_params_force_fresh_login() { + assert_eq!( + interactive_authorization_params(Some("api://openshell"), true), + vec![ + ("prompt", "login".to_string()), + ("audience", "api://openshell".to_string()), + ] + ); + assert_eq!( + interactive_authorization_params(None, true), + vec![("prompt", "login".to_string())] + ); + assert_eq!( + interactive_authorization_params(Some("api://openshell"), false), + vec![("audience", "api://openshell".to_string())] + ); + assert!(interactive_authorization_params(None, false).is_empty()); + } + + #[test] + fn device_authorization_form_includes_pkce_and_audience() { + let params = device_authorization_form( + "openshell-cli", + "openid profile", + Some("openshell-api"), + "test-challenge", + "S256", + ); + let params: std::collections::HashMap<_, _> = params.into_iter().collect(); + + assert_eq!( + params.get("client_id").map(String::as_str), + Some("openshell-cli") + ); + assert_eq!( + params.get("scope").map(String::as_str), + Some("openid profile") + ); + assert_eq!( + params.get("audience").map(String::as_str), + Some("openshell-api") + ); + assert_eq!( + params.get("code_challenge").map(String::as_str), + Some("test-challenge") + ); + assert_eq!( + params.get("code_challenge_method").map(String::as_str), + Some("S256") + ); + } + + #[test] + fn device_token_form_includes_pkce_verifier() { + let params = device_token_form("openshell-cli", "device-code", "test-verifier"); + let params: std::collections::HashMap<_, _> = params.into_iter().collect(); + + assert_eq!( + params.get("grant_type").map(String::as_str), + Some("urn:ietf:params:oauth:grant-type:device_code") + ); + assert_eq!( + params.get("device_code").map(String::as_str), + Some("device-code") + ); + assert_eq!( + params.get("client_id").map(String::as_str), + Some("openshell-cli") + ); + assert_eq!( + params.get("code_verifier").map(String::as_str), + Some("test-verifier") + ); + } + #[test] fn bundle_from_response_sets_fields() { use oauth2::basic::BasicTokenResponse; @@ -572,4 +897,130 @@ mod tests { assert_eq!(refreshed.client_id, previous.client_id); assert_eq!(refreshed.expires_at, Some(300)); } + + #[test] + fn discovery_missing_device_endpoint_is_optional() { + let discovery_json = "{\"issuer\":\"https://issuer.example\",\"authorization_endpoint\":\"https://issuer.example/auth\",\"token_endpoint\":\"https://issuer.example/token\"}"; + let discovery: OidcDiscovery = serde_json::from_str(discovery_json).unwrap(); + assert!(discovery.device_authorization_endpoint.is_none()); + assert_eq!(discovery.issuer, "https://issuer.example"); + } + + #[test] + fn discovery_with_device_endpoint_is_captured() { + let discovery_json = "{\"issuer\":\"https://issuer.example\",\"authorization_endpoint\":\"https://issuer.example/auth\",\"token_endpoint\":\"https://issuer.example/token\",\"device_authorization_endpoint\":\"https://issuer.example/device\"}"; + let discovery: OidcDiscovery = serde_json::from_str(discovery_json).unwrap(); + assert_eq!( + discovery.device_authorization_endpoint.as_deref(), + Some("https://issuer.example/device") + ); + } + + #[test] + fn device_auth_response_parses_minimal() { + let json = "{\"device_code\":\"GmRhmhcxhwAzkoEqiMEg_DnyEysNkuNhszIySk9eS\",\"user_code\":\"WDJB-MJHT\",\"verification_uri\":\"https://example.com/device\",\"expires_in\":1800}"; + let resp: DeviceAuthorizationResponse = serde_json::from_str(json).unwrap(); + assert_eq!( + resp.device_code, + "GmRhmhcxhwAzkoEqiMEg_DnyEysNkuNhszIySk9eS" + ); + assert_eq!(resp.user_code, "WDJB-MJHT"); + assert_eq!(resp.verification_uri, "https://example.com/device"); + assert_eq!(resp.expires_in, 1800); + assert_eq!(resp.interval, 5); + assert!(resp.verification_uri_complete.is_none()); + } + + #[test] + fn device_auth_response_parses_complete() { + let json = "{\"device_code\":\"test-device-code\",\"user_code\":\"TEST-CODE\",\"verification_uri\":\"https://example.com/device\",\"verification_uri_complete\":\"https://example.com/device?user_code=TEST-CODE\",\"expires_in\":900,\"interval\":10}"; + let resp: DeviceAuthorizationResponse = serde_json::from_str(json).unwrap(); + assert_eq!(resp.interval, 10); + assert_eq!( + resp.verification_uri_complete.as_deref(), + Some("https://example.com/device?user_code=TEST-CODE") + ); + } + + #[test] + fn device_token_error_response_parses() { + let json = "{\"error\":\"authorization_pending\",\"error_description\":\"User has not authorized yet\"}"; + let resp: DeviceTokenErrorResponse = serde_json::from_str(json).unwrap(); + assert_eq!(resp.error, "authorization_pending"); + assert_eq!(resp.error_description, "User has not authorized yet"); + } + + #[test] + fn device_token_error_response_defaults_empty_description() { + let json = "{\"error\":\"slow_down\"}"; + let resp: DeviceTokenErrorResponse = serde_json::from_str(json).unwrap(); + assert_eq!(resp.error, "slow_down"); + assert_eq!(resp.error_description, ""); + } + + #[test] + fn device_token_response_complete_is_typed() { + let response: BasicTokenResponse = serde_json::from_value(serde_json::json!({ + "access_token": "device-access-token", + "token_type": "Bearer", + "expires_in": 3600, + "refresh_token": "device-refresh-token" + })) + .unwrap(); + let bundle = + bundle_from_device_token_response(&response, "https://issuer.example", "test-client") + .unwrap(); + assert_eq!(bundle.access_token, "device-access-token"); + assert_eq!( + bundle.refresh_token.as_deref(), + Some("device-refresh-token") + ); + assert_eq!(bundle.issuer, "https://issuer.example"); + assert_eq!(bundle.client_id, "test-client"); + assert!(bundle.expires_at.is_some()); + } + + #[test] + fn device_token_response_minimal_is_typed() { + let response: BasicTokenResponse = serde_json::from_value(serde_json::json!({ + "access_token": "device-access-only", + "token_type": "Bearer" + })) + .unwrap(); + let bundle = + bundle_from_device_token_response(&response, "https://issuer.example", "test-client") + .unwrap(); + assert_eq!(bundle.access_token, "device-access-only"); + assert!(bundle.refresh_token.is_none()); + assert!(bundle.expires_at.is_none()); + } + + #[test] + fn device_token_response_requires_access_token() { + let result = serde_json::from_value::(serde_json::json!({ + "token_type": "Bearer" + })); + assert!(result.is_err()); + } + + #[test] + fn device_token_response_requires_token_type() { + let result = serde_json::from_value::(serde_json::json!({ + "access_token": "device-access-token" + })); + assert!(result.is_err()); + } + + #[test] + fn device_token_response_rejects_empty_access_token() { + let response: BasicTokenResponse = serde_json::from_value(serde_json::json!({ + "access_token": "", + "token_type": "Bearer" + })) + .unwrap(); + + let result = + bundle_from_device_token_response(&response, "https://issuer.example", "test-client"); + assert!(result.is_err()); + } } diff --git a/crates/openshell-cli/src/policy_update.rs b/crates/openshell-cli/src/policy_update.rs index 1f1f647506..e21054f46d 100644 --- a/crates/openshell-cli/src/policy_update.rs +++ b/crates/openshell-cli/src/policy_update.rs @@ -351,6 +351,8 @@ fn parse_add_endpoint_spec(spec: &str) -> Result { Ok(endpoint) } +const ALLOWED_IP_OPTION_PREFIX: &str = "allowed-ip="; + fn apply_add_endpoint_options( spec: &str, endpoint: &mut NetworkEndpoint, @@ -368,6 +370,9 @@ fn apply_add_endpoint_options( )); } match option { + "allow-uninspected-credentials" => { + endpoint.allow_uninspected_credentials = true; + } "websocket-credential-rewrite" => { ensure_websocket_credential_rewrite_protocol(spec, endpoint)?; endpoint.websocket_credential_rewrite = true; @@ -376,37 +381,40 @@ fn apply_add_endpoint_options( ensure_request_body_credential_rewrite_protocol(spec, endpoint)?; endpoint.request_body_credential_rewrite = true; } - _ => { - let Some(allowed_ip) = option.strip_prefix("allowed-ip=") else { - return Err(miette!( - "--add-endpoint options segment supports only 'websocket-credential-rewrite', 'request-body-credential-rewrite', and 'allowed-ip='; got '{option}' in '{spec}'" - )); - }; - let allowed_ip = allowed_ip.trim(); - if allowed_ip.is_empty() { - return Err(miette!( - "--add-endpoint allowed-ip option must include a CIDR or IP value in '{spec}'" - )); - } - if allowed_ip.contains(char::is_whitespace) { - return Err(miette!( - "--add-endpoint allowed-ip option must not contain whitespace in '{spec}'" - )); - } - if !endpoint - .allowed_ips - .iter() - .any(|existing| existing == allowed_ip) - { - endpoint.allowed_ips.push(allowed_ip.to_string()); + _ if option.starts_with(ALLOWED_IP_OPTION_PREFIX) => { + let allowed_ip = + parse_allowed_ip_value(spec, &option[ALLOWED_IP_OPTION_PREFIX.len()..])?; + if !endpoint.allowed_ips.contains(&allowed_ip) { + endpoint.allowed_ips.push(allowed_ip); } } + _ => { + return Err(miette!( + "--add-endpoint options segment supports only 'allow-uninspected-credentials', 'websocket-credential-rewrite', 'request-body-credential-rewrite', and 'allowed-ip='; got '{option}' in '{spec}'" + )); + } } } Ok(()) } +/// Validate the value part of an `allowed-ip=` endpoint option. +fn parse_allowed_ip_value(spec: &str, value: &str) -> Result { + let allowed_ip = value.trim(); + if allowed_ip.is_empty() { + return Err(miette!( + "--add-endpoint allowed-ip option must include a CIDR or IP value in '{spec}'" + )); + } + if allowed_ip.contains(char::is_whitespace) { + return Err(miette!( + "--add-endpoint allowed-ip option must not contain whitespace in '{spec}'" + )); + } + Ok(allowed_ip.to_string()) +} + fn parse_host(flag: &str, spec: &str, host: &str) -> Result { let host = host.trim(); if host.is_empty() { @@ -454,6 +462,7 @@ fn dedup_strings(values: &[String]) -> Vec { mod tests { use super::{ PolicyUpdatePlan, build_policy_update_plan as build_policy_update_plan_with_options, + parse_allowed_ip_value, }; use openshell_policy::PolicyMergeOp; @@ -604,6 +613,25 @@ mod tests { assert!(endpoint.request_body_credential_rewrite); } + #[test] + fn parse_add_endpoint_enables_allow_uninspected_credentials() { + let plan = build_policy_update_plan( + &["api.vendor.example:443::::allow-uninspected-credentials".to_string()], + &[], + &[], + &[], + &[], + &[], + None, + ) + .expect("plan should build"); + + let PolicyMergeOp::AddRule { rule, .. } = &plan.preview_operations[0] else { + panic!("expected add-rule preview"); + }; + assert!(rule.endpoints[0].allow_uninspected_credentials); + } + #[test] fn parse_add_endpoint_merges_allowed_ips_with_websocket_options() { let plan = build_policy_update_plan( @@ -665,6 +693,28 @@ mod tests { assert!(error.to_string().contains("allowed-ip option")); } + #[test] + fn parse_allowed_ip_value_accepts_trimmed_cidr_and_ip() { + assert_eq!( + parse_allowed_ip_value("spec", "10.0.0.0/8").expect("CIDR should parse"), + "10.0.0.0/8" + ); + assert_eq!( + parse_allowed_ip_value("spec", " 192.168.1.10 ").expect("IP should parse"), + "192.168.1.10" + ); + } + + #[test] + fn parse_allowed_ip_value_rejects_empty_and_interior_whitespace() { + let empty = parse_allowed_ip_value("spec", " ").expect_err("empty value must fail"); + assert!(empty.to_string().contains("must include a CIDR or IP")); + + let spaced = + parse_allowed_ip_value("spec", "10.0.0.0/8 172.16.0.0/12").expect_err("must fail"); + assert!(spaced.to_string().contains("must not contain whitespace")); + } + #[test] fn websocket_credential_rewrite_rejects_l4_endpoint() { let error = build_policy_update_plan( diff --git a/crates/openshell-core/src/policy.rs b/crates/openshell-core/src/policy.rs index 1645b9da44..20212fd2e3 100644 --- a/crates/openshell-core/src/policy.rs +++ b/crates/openshell-core/src/policy.rs @@ -92,6 +92,19 @@ pub enum LandlockCompatibility { HardRequirement, } +/// Accepted `landlock.compatibility` values in their proto string form. +/// +/// Single source of truth shared by YAML parsing, proto→runtime conversion, +/// and gateway policy validation so the accepted set cannot drift. +pub const LANDLOCK_COMPATIBILITY_VALUES: [&str; 2] = ["best_effort", "hard_requirement"]; + +/// Returns `true` if `value` is an accepted `landlock.compatibility` string. +/// +/// The empty string is accepted and defaults to `best_effort`. +pub fn is_valid_landlock_compatibility(value: &str) -> bool { + value.is_empty() || LANDLOCK_COMPATIBILITY_VALUES.contains(&value) +} + // ============================================================================ // Proto to Rust type conversions // ============================================================================ @@ -114,7 +127,11 @@ impl TryFrom for SandboxPolicy { .map(FilesystemPolicy::from) .unwrap_or_default(), network, - landlock: proto.landlock.map(LandlockPolicy::from).unwrap_or_default(), + landlock: proto + .landlock + .map(LandlockPolicy::try_from) + .transpose()? + .unwrap_or_default(), process: proto.process.map(ProcessPolicy::from).unwrap_or_default(), }) } @@ -138,14 +155,20 @@ impl From for FilesystemPolicy { } } -impl From for LandlockPolicy { - fn from(proto: ProtoLandlockPolicy) -> Self { - let compatibility = if proto.compatibility == "hard_requirement" { - LandlockCompatibility::HardRequirement - } else { - LandlockCompatibility::BestEffort +impl TryFrom for LandlockPolicy { + type Error = miette::Error; + + fn try_from(proto: ProtoLandlockPolicy) -> Result { + let compatibility = match proto.compatibility.as_str() { + "best_effort" | "" => LandlockCompatibility::BestEffort, + "hard_requirement" => LandlockCompatibility::HardRequirement, + otherwise => miette::bail!( + "invalid landlock.compatibility {:?}; accepted: {}", + otherwise, + LANDLOCK_COMPATIBILITY_VALUES.join(", ") + ), }; - Self { compatibility } + Ok(Self { compatibility }) } } @@ -165,3 +188,49 @@ impl From for ProcessPolicy { } } } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn try_from_maps_known_compatibility_values() { + for (input, expected) in [ + ("", LandlockCompatibility::BestEffort), + ("best_effort", LandlockCompatibility::BestEffort), + ("hard_requirement", LandlockCompatibility::HardRequirement), + ] { + let proto = ProtoLandlockPolicy { + compatibility: input.into(), + }; + let policy = LandlockPolicy::try_from(proto).expect("should convert"); + assert_eq!( + std::mem::discriminant(&policy.compatibility), + std::mem::discriminant(&expected), + "input {input:?} mapped to unexpected variant", + ); + } + } + + #[test] + fn try_from_rejects_invalid_compatibility() { + let proto = ProtoLandlockPolicy { + compatibility: "hard-requirement".into(), + }; + let err = LandlockPolicy::try_from(proto).expect_err("should reject"); + let msg = format!("{err:?}"); + assert!( + msg.contains("best_effort") && msg.contains("hard_requirement"), + "error should list accepted values, got: {msg}", + ); + } + + #[test] + fn is_valid_landlock_compatibility_accepts_empty_and_known() { + assert!(is_valid_landlock_compatibility("")); + assert!(is_valid_landlock_compatibility("best_effort")); + assert!(is_valid_landlock_compatibility("hard_requirement")); + assert!(!is_valid_landlock_compatibility("nope")); + assert!(!is_valid_landlock_compatibility("BestEffort")); + } +} diff --git a/crates/openshell-core/src/secrets.rs b/crates/openshell-core/src/secrets.rs index 6450da4f61..bef39c8fe2 100644 --- a/crates/openshell-core/src/secrets.rs +++ b/crates/openshell-core/src/secrets.rs @@ -13,6 +13,24 @@ const PROVIDER_ALIAS_MARKER: &str = "OPENSHELL-RESOLVE-ENV-"; /// Public access to the placeholder prefix for fail-closed scanning in other modules. pub const PLACEHOLDER_PREFIX_PUBLIC: &str = PLACEHOLDER_PREFIX; pub const PROVIDER_ALIAS_MARKER_PUBLIC: &str = PROVIDER_ALIAS_MARKER; +/// Longest wire form of a reserved marker: percent-encoding expands every +/// marker byte to three bytes (`%XX`), and detection decodes in a single pass. +const LONGEST_RESERVED_MARKER_WIRE_BYTES: usize = + 3 * if PLACEHOLDER_PREFIX.len() > PROVIDER_ALIAS_MARKER.len() { + PLACEHOLDER_PREFIX.len() + } else { + PROVIDER_ALIAS_MARKER.len() + }; + +/// Retain this many trailing bytes when scanning a streamed request body so a +/// reserved marker split across reads cannot be forwarded before detection. +/// +/// A marker is only detected while all of its wire bytes sit in the scan buffer +/// at once, so the retained window must hold every byte of the longest form but +/// the last. A window shorter than that lets a caller split a fully +/// percent-encoded marker so its leading bytes are forwarded before the rest +/// arrives, and the reassembled remainder no longer decodes to the marker. +pub const CREDENTIAL_MARKER_SCAN_TAIL_BYTES: usize = LONGEST_RESERVED_MARKER_WIRE_BYTES; /// Characters that are valid in an env var key name (used to extract /// placeholder boundaries within concatenated strings like path segments). @@ -36,6 +54,15 @@ pub fn contains_reserved_credential_marker(value: &str) -> bool { contains_raw_reserved_marker(&decoded) } +pub fn contains_reserved_credential_marker_bytes(value: &[u8]) -> bool { + if value.is_empty() { + return false; + } + String::from_utf8_lossy(value) + .split('\0') + .any(contains_reserved_credential_marker) +} + // --------------------------------------------------------------------------- // Error and result types // --------------------------------------------------------------------------- @@ -1373,6 +1400,45 @@ mod tests { // === Existing tests (preserved) === + #[test] + fn byte_marker_detection_handles_raw_encoded_and_binary_input() { + assert!(contains_reserved_credential_marker_bytes( + b"openshell:resolve:env:API_TOKEN" + )); + assert!(contains_reserved_credential_marker_bytes( + b"openshell%3Aresolve%3Aenv%3AAPI_TOKEN" + )); + assert!(!contains_reserved_credential_marker_bytes(&[ + 0xff, 0x00, 0x01, 0x02 + ])); + } + + fn fully_percent_encoded(marker: &str) -> String { + const HEX: &[u8; 16] = b"0123456789ABCDEF"; + let mut encoded = String::with_capacity(marker.len() * 3); + for byte in marker.bytes() { + encoded.push('%'); + encoded.push(char::from(HEX[usize::from(byte >> 4)])); + encoded.push(char::from(HEX[usize::from(byte & 0x0f)])); + } + encoded + } + + #[test] + fn scan_tail_window_covers_longest_encoded_marker_form() { + for marker in [PLACEHOLDER_PREFIX, PROVIDER_ALIAS_MARKER] { + let encoded = fully_percent_encoded(marker); + assert!( + contains_reserved_credential_marker(&encoded), + "fully encoded {marker} must be detected" + ); + assert!( + CREDENTIAL_MARKER_SCAN_TAIL_BYTES >= encoded.len() - 1, + "scan window must retain every byte of {encoded} but the last" + ); + } + } + #[test] fn provider_env_is_replaced_with_placeholders() { let (child_env, resolver) = SecretResolver::from_provider_env( diff --git a/crates/openshell-policy/src/lib.rs b/crates/openshell-policy/src/lib.rs index 55d6cf1e40..42c28ae3be 100644 --- a/crates/openshell-policy/src/lib.rs +++ b/crates/openshell-policy/src/lib.rs @@ -74,11 +74,19 @@ struct FilesystemDef { read_write: Vec, } +#[derive(Debug, Default, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +enum LandlockCompatibilityDef { + #[default] + BestEffort, + HardRequirement, +} + #[derive(Debug, Serialize, Deserialize)] #[serde(deny_unknown_fields)] struct LandlockDef { - #[serde(default, skip_serializing_if = "String::is_empty")] - compatibility: String, + #[serde(default)] + compatibility: LandlockCompatibilityDef, } #[derive(Debug, Serialize, Deserialize)] @@ -103,6 +111,10 @@ struct NetworkPolicyRuleDef { #[derive(Debug, Serialize, Deserialize)] #[serde(deny_unknown_fields)] +#[allow( + clippy::struct_excessive_bools, + reason = "Endpoint DTO mirrors independent policy schema toggles." +)] struct NetworkEndpointDef { #[serde(default, skip_serializing_if = "String::is_empty")] host: String, @@ -144,6 +156,10 @@ struct NetworkEndpointDef { /// placeholders before forwarding upstream. Defaults to false. #[serde(default, skip_serializing_if = "std::ops::Not::not")] request_body_credential_rewrite: bool, + /// Explicitly permits credentials on traffic paths that `OpenShell` cannot + /// inspect or rewrite. Defaults to false. + #[serde(default, skip_serializing_if = "std::ops::Not::not")] + allow_uninspected_credentials: bool, #[serde(default, skip_serializing_if = "String::is_empty")] persisted_queries: String, #[serde(default, skip_serializing_if = "BTreeMap::is_empty")] @@ -745,6 +761,10 @@ fn to_proto(raw: PolicyFile) -> Result { allow_encoded_slash: e.allow_encoded_slash, websocket_credential_rewrite: e.websocket_credential_rewrite, request_body_credential_rewrite: e.request_body_credential_rewrite, + allow_uninspected_credentials: e.allow_uninspected_credentials, + // Provider credential provenance is derived by the + // gateway and cannot be authored in policy YAML. + provider_credentialed: false, // Advisor provenance is internal runtime state, not // a user-authored policy schema field. advisor_proposed: false, @@ -798,7 +818,10 @@ fn to_proto(raw: PolicyFile) -> Result { read_write: fs.read_write, }), landlock: raw.landlock.map(|ll| LandlockPolicy { - compatibility: ll.compatibility, + compatibility: match ll.compatibility { + LandlockCompatibilityDef::BestEffort => "best_effort".to_string(), + LandlockCompatibilityDef::HardRequirement => "hard_requirement".to_string(), + }, }), process: raw.process.map(|p| ProcessPolicy { run_as_user: p.run_as_user, @@ -813,16 +836,28 @@ fn to_proto(raw: PolicyFile) -> Result { // Proto → YAML conversion // --------------------------------------------------------------------------- -fn from_proto(policy: &SandboxPolicy) -> PolicyFile { +fn from_proto(policy: &SandboxPolicy) -> Result { let filesystem_policy = policy.filesystem.as_ref().map(|fs| FilesystemDef { include_workdir: fs.include_workdir, read_only: fs.read_only.clone(), read_write: fs.read_write.clone(), }); - let landlock = policy.landlock.as_ref().map(|ll| LandlockDef { - compatibility: ll.compatibility.clone(), - }); + let landlock = match policy.landlock.as_ref() { + Some(ll) => { + let compatibility = match ll.compatibility.as_str() { + "hard_requirement" => LandlockCompatibilityDef::HardRequirement, + "best_effort" | "" => LandlockCompatibilityDef::BestEffort, + otherwise => miette::bail!( + "invalid landlock.compatibility {:?}; accepted: {}", + otherwise, + openshell_core::policy::LANDLOCK_COMPATIBILITY_VALUES.join(", ") + ), + }; + Some(LandlockDef { compatibility }) + } + _ => None, + }; let process = policy.process.as_ref().and_then(|p| { if p.run_as_user.is_empty() && p.run_as_group.is_empty() { @@ -901,6 +936,7 @@ fn from_proto(policy: &SandboxPolicy) -> PolicyFile { allow_encoded_slash: e.allow_encoded_slash, websocket_credential_rewrite: e.websocket_credential_rewrite, request_body_credential_rewrite: e.request_body_credential_rewrite, + allow_uninspected_credentials: e.allow_uninspected_credentials, persisted_queries: e.persisted_queries.clone(), graphql_persisted_queries: e .graphql_persisted_queries @@ -945,14 +981,14 @@ fn from_proto(policy: &SandboxPolicy) -> PolicyFile { let network_middlewares = middleware::from_proto(&policy.network_middlewares); - PolicyFile { + Ok(PolicyFile { version: policy.version, filesystem_policy, landlock, process, network_policies, network_middlewares, - } + }) } // --------------------------------------------------------------------------- @@ -1007,7 +1043,7 @@ pub fn parse_sandbox_policy(yaml: &str) -> Result { /// canonical YAML field names (e.g. `filesystem_policy`, not `filesystem`) /// and is round-trippable through `parse_sandbox_policy`. pub fn serialize_sandbox_policy(policy: &SandboxPolicy) -> Result { - let yaml_repr = from_proto(policy); + let yaml_repr = from_proto(policy)?; serde_yml::to_string(&yaml_repr) .into_diagnostic() .wrap_err("failed to serialize policy to YAML") @@ -1018,7 +1054,7 @@ pub fn serialize_sandbox_policy(policy: &SandboxPolicy) -> Result { /// The shape mirrors the YAML schema used by [`serialize_sandbox_policy`], so /// automation can use the same documented field names in either format. pub fn sandbox_policy_to_json_value(policy: &SandboxPolicy) -> Result { - let json_repr = from_proto(policy); + let json_repr = from_proto(policy)?; serde_json::to_value(&json_repr) .into_diagnostic() .wrap_err("failed to serialize policy to JSON") @@ -1175,6 +1211,8 @@ pub enum PolicyViolation { policy_name: String, host: String, }, + /// `landlock.compatibility` has an unrecognized value. + InvalidLandlockCompatibility { value: String }, } impl fmt::Display for PolicyViolation { @@ -1285,6 +1323,13 @@ impl fmt::Display for PolicyViolation { '{policy_name}' tls: skip endpoint '{host}'" ) } + Self::InvalidLandlockCompatibility { value } => { + write!( + f, + "invalid landlock.compatibility '{value}'; accepted: {}", + openshell_core::policy::LANDLOCK_COMPATIBILITY_VALUES.join(", ") + ) + } } } } @@ -1329,6 +1374,17 @@ pub fn validate_sandbox_policy( } } + // Check landlock compatibility mode is a recognized value. Direct gRPC/SDK + // clients bypass YAML serde validation, so reject invalid values here at the + // gateway create path rather than deferring rejection to sandbox startup. + if let Some(ref landlock) = policy.landlock + && !openshell_core::policy::is_valid_landlock_compatibility(&landlock.compatibility) + { + violations.push(PolicyViolation::InvalidLandlockCompatibility { + value: landlock.compatibility.clone(), + }); + } + // Check filesystem paths if let Some(ref fs) = policy.filesystem { let total_paths = fs.read_only.len() + fs.read_write.len(); @@ -2010,6 +2066,80 @@ network_policies: assert_eq!(violations.len(), 2); } + #[test] + fn parse_rejects_invalid_landlock_compatibility() { + let err = parse_sandbox_policy("version: 1\nlandlock:\n compatibility: bogus\n") + .expect_err("should reject invalid YAML enum value"); + let msg = format!("{err:?}"); + assert!( + msg.contains("best_effort") && msg.contains("hard_requirement"), + "error should list accepted values, got: {msg}", + ); + } + + #[test] + fn parse_accepts_known_landlock_compatibility() { + for value in ["best_effort", "hard_requirement"] { + let yaml = format!("version: 1\nlandlock:\n compatibility: {value}\n"); + let policy = parse_sandbox_policy(&yaml).expect("should parse"); + assert_eq!( + policy.landlock.as_ref().expect("landlock").compatibility, + value, + ); + } + } + + #[test] + fn validate_rejects_invalid_landlock_compatibility_proto() { + let mut policy = restrictive_default_policy(); + policy.landlock = Some(LandlockPolicy { + compatibility: "nope".into(), + }); + let violations = validate_sandbox_policy(&policy).unwrap_err(); + assert!( + violations + .iter() + .any(|v| matches!(v, PolicyViolation::InvalidLandlockCompatibility { .. })), + "expected InvalidLandlockCompatibility, got: {violations:?}", + ); + } + + #[test] + fn validate_accepts_empty_landlock_compatibility() { + // Empty string is the proto default and maps to best_effort. + let mut policy = restrictive_default_policy(); + policy.landlock = Some(LandlockPolicy { + compatibility: String::new(), + }); + assert!(validate_sandbox_policy(&policy).is_ok()); + } + + #[test] + fn serialize_rejects_invalid_landlock_compatibility() { + // Old policies persisted before gateway validation can hold invalid + // values; serialization must error rather than normalize to best_effort. + let mut policy = restrictive_default_policy(); + policy.landlock = Some(LandlockPolicy { + compatibility: "hard-requirement".to_string(), + }); + let err = serialize_sandbox_policy(&policy).expect_err("should reject"); + let msg = format!("{err:?}"); + assert!( + msg.contains("best_effort") && msg.contains("hard_requirement"), + "error should list accepted values, got: {msg}", + ); + } + + #[test] + fn serialize_accepts_empty_landlock_compatibility() { + // Empty string is the proto default; serialize must not error on it. + let mut policy = restrictive_default_policy(); + policy.landlock = Some(LandlockPolicy { + compatibility: String::new(), + }); + assert!(serialize_sandbox_policy(&policy).is_ok()); + } + #[test] fn validate_rejects_invalid_middleware_control_fields() { let cases = [ @@ -3486,6 +3616,32 @@ network_policies: assert!(yaml_out.contains("request_body_credential_rewrite: true")); } + #[test] + fn round_trip_preserves_allow_uninspected_credentials() { + let yaml = r" +version: 1 +network_policies: + vendor_api: + endpoints: + - host: api.vendor.example + port: 443 + tls: skip + allow_uninspected_credentials: true +"; + let proto1 = parse_sandbox_policy(yaml).expect("parse failed"); + let yaml_out = serialize_sandbox_policy(&proto1).expect("serialize failed"); + let proto2 = parse_sandbox_policy(&yaml_out).expect("re-parse failed"); + + let ep = &proto2.network_policies["vendor_api"].endpoints[0]; + assert!(ep.allow_uninspected_credentials); + assert!( + !ep.provider_credentialed, + "provider provenance must not be authorable from policy YAML" + ); + assert!(yaml_out.contains("allow_uninspected_credentials: true")); + assert!(!yaml_out.contains("provider_credentialed")); + } + #[test] fn websocket_credential_rewrite_defaults_false() { let yaml = r" @@ -3504,6 +3660,8 @@ network_policies: let ep = &proto.network_policies["gateway"].endpoints[0]; assert!(!ep.websocket_credential_rewrite); assert!(!ep.request_body_credential_rewrite); + assert!(!ep.allow_uninspected_credentials); + assert!(!ep.provider_credentialed); } #[test] diff --git a/crates/openshell-policy/src/merge.rs b/crates/openshell-policy/src/merge.rs index ef77c2aaad..75bc700975 100644 --- a/crates/openshell-policy/src/merge.rs +++ b/crates/openshell-policy/src/merge.rs @@ -1282,6 +1282,7 @@ fn merge_endpoint( existing.allow_encoded_slash |= incoming.allow_encoded_slash; existing.websocket_credential_rewrite |= incoming.websocket_credential_rewrite; existing.request_body_credential_rewrite |= incoming.request_body_credential_rewrite; + existing.allow_uninspected_credentials |= incoming.allow_uninspected_credentials; existing.advisor_proposed |= incoming.advisor_proposed; normalize_endpoint(existing); Ok(()) @@ -3100,6 +3101,48 @@ mod tests { assert!(endpoint.request_body_credential_rewrite); } + #[test] + fn add_rule_merges_allow_uninspected_credentials_flag() { + let mut policy = restrictive_default_policy(); + policy.network_policies.insert( + "existing".to_string(), + NetworkPolicyRule { + name: "existing".to_string(), + endpoints: vec![NetworkEndpoint { + host: "api.vendor.example".to_string(), + port: 443, + ports: vec![443], + ..Default::default() + }], + ..Default::default() + }, + ); + + let incoming = NetworkPolicyRule { + name: "incoming".to_string(), + endpoints: vec![NetworkEndpoint { + host: "api.vendor.example".to_string(), + port: 443, + ports: vec![443], + allow_uninspected_credentials: true, + ..Default::default() + }], + ..Default::default() + }; + + let result = merge_policy( + policy, + &[PolicyMergeOp::AddRule { + rule_name: "allow_api_vendor_example_443".to_string(), + rule: incoming, + }], + ) + .expect("merge should succeed"); + + let endpoint = &result.policy.network_policies["existing"].endpoints[0]; + assert!(endpoint.allow_uninspected_credentials); + } + #[test] fn add_allow_expands_access_preset() { let mut policy = restrictive_default_policy(); diff --git a/crates/openshell-providers/src/profiles.rs b/crates/openshell-providers/src/profiles.rs index 42c82c0c2a..c96e39674a 100644 --- a/crates/openshell-providers/src/profiles.rs +++ b/crates/openshell-providers/src/profiles.rs @@ -195,6 +195,10 @@ pub struct DiscoveryProfile { // GraphqlOperation, or NetworkBinary, add it here and in both conversion // directions unless the import/lint path explicitly rejects it. #[derive(Debug, Clone, Deserialize, Serialize, PartialEq, Eq)] +#[allow( + clippy::struct_excessive_bools, + reason = "Endpoint profile mirrors independent policy schema toggles." +)] pub struct EndpointProfile { pub host: String, #[serde(default, skip_serializing_if = "is_zero")] @@ -221,6 +225,8 @@ pub struct EndpointProfile { pub websocket_credential_rewrite: bool, #[serde(default, skip_serializing_if = "is_false")] pub request_body_credential_rewrite: bool, + #[serde(default, skip_serializing_if = "is_false")] + pub allow_uninspected_credentials: bool, #[serde(default, skip_serializing_if = "String::is_empty")] pub persisted_queries: String, #[serde(default, skip_serializing_if = "HashMap::is_empty")] @@ -651,6 +657,21 @@ impl ProviderTypeProfile { } diagnostics } + + /// Whether attaching this profile makes its network endpoints credentialed. + /// + /// Profiles do not currently map individual credentials to individual + /// endpoints, so the safe interpretation is that any declared credential + /// can be used with every endpoint in the same profile. Endpoint signing is + /// also credential-bearing even when placement metadata is implicit. + #[must_use] + pub fn has_credentialed_endpoints(&self) -> bool { + !self.credentials.is_empty() + || self + .endpoints + .iter() + .any(|endpoint| !endpoint.credential_signing.trim().is_empty()) + } } #[allow(clippy::trivially_copy_pass_by_ref)] @@ -1074,6 +1095,8 @@ fn endpoint_to_proto(endpoint: &EndpointProfile) -> NetworkEndpoint { allow_encoded_slash: endpoint.allow_encoded_slash, websocket_credential_rewrite: endpoint.websocket_credential_rewrite, request_body_credential_rewrite: endpoint.request_body_credential_rewrite, + allow_uninspected_credentials: endpoint.allow_uninspected_credentials, + provider_credentialed: false, advisor_proposed: false, persisted_queries: endpoint.persisted_queries.clone(), graphql_persisted_queries: endpoint @@ -1123,6 +1146,7 @@ fn endpoint_from_proto(endpoint: &NetworkEndpoint) -> EndpointProfile { allow_encoded_slash: endpoint.allow_encoded_slash, websocket_credential_rewrite: endpoint.websocket_credential_rewrite, request_body_credential_rewrite: endpoint.request_body_credential_rewrite, + allow_uninspected_credentials: endpoint.allow_uninspected_credentials, persisted_queries: endpoint.persisted_queries.clone(), graphql_persisted_queries: endpoint .graphql_persisted_queries @@ -2189,6 +2213,27 @@ pub fn validate_profile_set( } } } + + if profile.has_credentialed_endpoints() + && !endpoint.allow_uninspected_credentials + && (endpoint.protocol.trim().is_empty() + || endpoint.tls.trim().eq_ignore_ascii_case("skip")) + { + let mode = if endpoint.protocol.trim().is_empty() { + "L4-only" + } else { + "tls: skip" + }; + diagnostics.push(ProfileValidationDiagnostic::error( + source, + profile_id, + format!("endpoints[{index}].allow_uninspected_credentials"), + format!( + "credentialed endpoint '{}:{}' uses {mode}; configure L7 inspection or explicitly set allow_uninspected_credentials: true", + endpoint.host, endpoint.port + ), + )); + } } for (index, binary) in profile.binaries.iter().enumerate() { @@ -3338,6 +3383,8 @@ endpoints: - host: alpha.default.svc.cluster.local port: 80 path: /v1/** + protocol: rest + access: full ", ) .expect("profile should parse"); @@ -3378,6 +3425,8 @@ endpoints: - host: alpha.default.svc.cluster.local port: 80 path: /v1/** + protocol: rest + access: full ", ) .expect("profile should parse"); @@ -3470,6 +3519,7 @@ endpoints: - method: POST path: /admin/** allow_encoded_slash: true + allow_uninspected_credentials: true binaries: - path: /usr/bin/custom harness: true @@ -3503,6 +3553,8 @@ binaries: assert_eq!(rest_ep.tls, "terminate"); assert_eq!(rest_ep.allowed_ips, vec!["10.0.0.0/24"]); assert!(rest_ep.allow_encoded_slash); + assert!(rest_ep.allow_uninspected_credentials); + assert!(!rest_ep.provider_credentialed); assert_eq!( rest_ep .rules @@ -3521,9 +3573,93 @@ binaries: assert_eq!(reprotoo.endpoints[1].rules.len(), 1); assert_eq!(reprotoo.endpoints[1].deny_rules.len(), 1); assert_eq!(reprotoo.endpoints[1].ports, vec![443, 8443]); + assert!(reprotoo.endpoints[1].allow_uninspected_credentials); + assert!(!reprotoo.endpoints[1].provider_credentialed); assert!(reprotoo.binaries[0].harness); } + #[test] + fn profile_classifies_declared_credentials_and_signing_as_credentialed() { + let with_declared_credential = parse_profile_yaml( + r" +id: credentialed +display_name: Credentialed +credentials: + - name: token + env_vars: [TOKEN] +endpoints: + - host: api.example.com + port: 443 +", + ) + .expect("profile should parse"); + assert!(with_declared_credential.has_credentialed_endpoints()); + + let with_signing = parse_profile_yaml( + r" +id: signed +display_name: Signed +credentials: [] +endpoints: + - host: s3.example.com + port: 443 + credential_signing: sigv4 +", + ) + .expect("profile should parse"); + assert!(with_signing.has_credentialed_endpoints()); + + let plain = parse_profile_yaml( + r" +id: plain +display_name: Plain +credentials: [] +endpoints: + - host: pypi.org + port: 443 +", + ) + .expect("profile should parse"); + assert!(!plain.has_credentialed_endpoints()); + } + + #[test] + fn credentialed_profile_requires_opt_in_for_l4_endpoint() { + let profile = parse_profile_yaml( + r" +id: raw +display_name: Raw +credentials: + - name: token + env_vars: [TOKEN] +endpoints: + - host: raw.example.com + port: 443 +", + ) + .expect("profile should parse"); + let diagnostics = validate_profile_set(&[("raw.yaml".to_string(), profile)]); + assert!(diagnostics.iter().any(|diagnostic| { + diagnostic.field == "endpoints[0].allow_uninspected_credentials" + })); + + let opted_in = parse_profile_yaml( + r" +id: raw +display_name: Raw +credentials: + - name: token + env_vars: [TOKEN] +endpoints: + - host: raw.example.com + port: 443 + allow_uninspected_credentials: true +", + ) + .expect("profile should parse"); + assert!(validate_profile_set(&[("raw.yaml".to_string(), opted_in)]).is_empty()); + } + #[test] fn validate_profile_set_returns_all_discoverable_diagnostics() { let profile = parse_profile_yaml( diff --git a/crates/openshell-sandbox/src/lib.rs b/crates/openshell-sandbox/src/lib.rs index 9394037c84..c1dbada149 100644 --- a/crates/openshell-sandbox/src/lib.rs +++ b/crates/openshell-sandbox/src/lib.rs @@ -26,9 +26,9 @@ use tracing::{debug, info, warn}; use openshell_core::PolicyValidationFailureMode; use openshell_ocsf::{ - ActionId, ActivityId, AppLifecycleBuilder, ConfigStateChangeBuilder, DetectionFindingBuilder, - DispositionId, FindingInfo, OcsfEvent, SandboxContext, SeverityId, StateId, StatusId, - ocsf_emit, + ActionId, ActivityId, AppLifecycleBuilder, ConfidenceId, ConfigStateChangeBuilder, + DetectionFindingBuilder, DispositionId, FindingInfo, OcsfEvent, SandboxContext, SeverityId, + StateId, StatusId, ocsf_emit, }; // --------------------------------------------------------------------------- @@ -332,6 +332,14 @@ pub async fn run_sandbox( (provider_credentials, provider_env) }; + if credential_gating_unavailable( + &loaded_policy_origin, + provider_credentials.resolver().is_some(), + network_enabled, + ) { + report_credential_gating_unavailable(); + } + // Shared agent-proposals feature flag. Seed from the same initial settings // snapshot that produced the policy so networking and process setup agree // before the poll loop starts reconciling later changes. @@ -2606,6 +2614,63 @@ fn unchanged_policy_revision_ready_to_ack( candidate.filter(|_| !policy_runtime_changed || policy_runtime_reconciled) } +/// Whether the credential-provenance gates cannot apply to the loaded policy. +/// +/// The gateway derives `provider_credentialed` and deliberately keeps it out of +/// the policy YAML schema, so a local-file policy never carries it and never +/// will: gateway revisions are observed for settings and providers but must not +/// replace the local OPA policy. Provider credentials still arrive from the +/// gateway on that path, so the raw-tunnel and WebSocket binary-frame refusals +/// have nothing to match on. The request-body backstop is unaffected because it +/// keys off the secret resolver rather than endpoint provenance. +fn credential_gating_unavailable( + origin: &LoadedPolicyOrigin, + has_resolver: bool, + network_enabled: bool, +) -> bool { + network_enabled && has_resolver && matches!(origin, LoadedPolicyOrigin::LocalOverride) +} + +/// Report that credential provenance is unavailable for the loaded policy. +/// +/// Carries no credential name, host, or value: the finding states which +/// controls are inactive, nothing about what they would have protected. +fn report_credential_gating_unavailable() { + ocsf_emit!( + DetectionFindingBuilder::new(ocsf_ctx()) + .activity(ActivityId::Open) + .severity(SeverityId::High) + .confidence(ConfidenceId::High) + .is_alert(true) + .finding_info( + FindingInfo::new( + "credential-gating-unavailable", + "Credential Provenance Unavailable", + ) + .with_desc( + "Provider credentials are injected, but the loaded policy comes from local \ + files and carries no gateway-derived credential provenance. Uninspected \ + credentialed tunnels and WebSocket binary frames are not refused. Load \ + policy from the gateway to enable these controls." + ), + ) + .evidence_pairs(&[ + ("policy_source", "local-override"), + ("uninspected_connect_gate", "inactive"), + ("websocket_binary_gate", "inactive"), + ("request_body_backstop", "active"), + ]) + .remediation( + "Remove the local policy override so the gateway-delivered effective policy \ + applies, or detach provider credentials from this sandbox." + ) + .message( + "Credential provenance unavailable for local-file policy; uninspected credential gates inactive" + ) + .build() + ); +} + /// Deliver policy status updates independently from policy reconciliation. /// /// The channel is FIFO, so a delayed older status can never arrive after a @@ -5235,6 +5300,40 @@ filesystem_policy: ); } + #[test] + fn credential_gating_unavailable_for_local_override_with_credentials() { + assert!(credential_gating_unavailable( + &LoadedPolicyOrigin::LocalOverride, + true, + true + )); + } + + #[test] + fn credential_gating_available_without_local_override_or_credentials() { + // A gateway policy is stamped with provenance, so the gates apply. + assert!(!credential_gating_unavailable( + &LoadedPolicyOrigin::Gateway { + revision: None, + has_last_valid_policy: true, + }, + true, + true + )); + // No provider credentials means there is nothing to leak. + assert!(!credential_gating_unavailable( + &LoadedPolicyOrigin::LocalOverride, + false, + true + )); + // Without networking the proxy never evaluates endpoint provenance. + assert!(!credential_gating_unavailable( + &LoadedPolicyOrigin::LocalOverride, + true, + false + )); + } + #[test] fn policy_status_outbox_preserves_all_revision_order() { let (sender, mut receiver) = tokio::sync::mpsc::unbounded_channel(); diff --git a/crates/openshell-server/src/grpc/policy.rs b/crates/openshell-server/src/grpc/policy.rs index a63121ae05..a2e8a7f0a1 100644 --- a/crates/openshell-server/src/grpc/policy.rs +++ b/crates/openshell-server/src/grpc/policy.rs @@ -52,7 +52,7 @@ use openshell_core::telemetry::{ use openshell_core::{ VERSION, endpoint_path::EndpointPathPattern, - host_pattern::host_matches, + host_pattern::{host_matches, host_patterns_overlap}, settings::{self, SettingValueKind}, }; use openshell_ocsf::{ @@ -345,6 +345,9 @@ fn summarize_endpoint(endpoint: &NetworkEndpoint) -> String { if endpoint.request_body_credential_rewrite { parts.push("request_body_credential_rewrite=true".to_string()); } + if endpoint.allow_uninspected_credentials { + parts.push("allow_uninspected_credentials=true".to_string()); + } if !endpoint.allowed_ips.is_empty() { parts.push(format!("allowed_ips={}", endpoint.allowed_ips.len())); } @@ -995,19 +998,23 @@ async fn auto_approve_chunk( .as_ref() .map(|spec| spec.providers.as_slice()) .unwrap_or_default(); - let provider_layers = provider_policy_layers_for_sandbox( + let merge_validation = sandbox_policy_merge_validation_data( state, context.workspace, context.sandbox, provider_names, ) .await?; - let (version, hash) = merge_chunk_into_policy( + let credential_binding_context = merge_validation.credential_binding_context(); + let (version, hash) = merge_chunk_into_policy_with_validation( state.store.as_ref(), sandbox_id, context.workspace, &chunk, - &provider_layers, + PolicyMergeValidationContext { + provider_layers: &merge_validation.provider_layers, + credential_binding: Some(&credential_binding_context), + }, ) .await?; let chunk_summary = summarize_draft_chunk_rule(&chunk)?; @@ -1063,7 +1070,12 @@ async fn current_effective_policy_for_sandbox( sandbox: &Sandbox, sandbox_id: &str, ) -> Result { - let mut policy = if let Some(record) = state + let provider_names = sandbox + .spec + .as_ref() + .map(|spec| spec.providers.clone()) + .unwrap_or_default(); + let policy = if let Some(record) = state .store .get_latest_policy(sandbox_id) .await @@ -1079,6 +1091,16 @@ async fn current_effective_policy_for_sandbox( .unwrap_or_default() }; + effective_policy_for_source(state, catalog, workspace, &provider_names, policy).await +} + +async fn effective_policy_for_source( + state: &ServerState, + catalog: &EffectiveProviderProfileCatalog, + workspace: &str, + provider_names: &[String], + mut policy: ProtoSandboxPolicy, +) -> Result { let global_settings = load_global_settings(state.store.as_ref()).await?; let policy_source = decode_policy_from_global_settings(&global_settings)?.map_or( PolicySource::Sandbox, @@ -1090,23 +1112,27 @@ async fn current_effective_policy_for_sandbox( let providers_v2_enabled = bool_setting_enabled(&global_settings, settings::PROVIDERS_V2_ENABLED_KEY)?; - if providers_v2_enabled && !matches!(policy_source, PolicySource::Global) { - let provider_names = sandbox - .spec - .as_ref() - .map(|spec| spec.providers.clone()) - .unwrap_or_default(); - let provider_layers = profile_provider_policy_layers_with_catalog( - state.store.as_ref(), - catalog, - workspace, - &provider_names, - ) - .await?; - if !provider_layers.is_empty() { - policy = compose_effective_policy(&policy, &provider_layers); - } + clear_provider_credentialed_markers(&mut policy); + let mut provider_context = provider_policy_context_with_catalog( + state.store.as_ref(), + catalog, + workspace, + provider_names, + ) + .await?; + if providers_v2_enabled + && !matches!(policy_source, PolicySource::Global) + && !provider_context.layers.is_empty() + { + policy = compose_effective_policy(&policy, &provider_context.layers); } + let policy_credential_bindings = policy_static_credential_endpoint_bindings(Some(&policy))?; + extend_credentialed_scopes_from_policy_bindings( + &mut provider_context.credentialed_scopes, + &policy_credential_bindings, + &provider_context.endpointless_provider_names, + ); + stamp_provider_credentialed_endpoints(&mut policy, &provider_context.credentialed_scopes); Ok(policy) } @@ -1427,13 +1453,14 @@ async fn provider_policy_layers_for_sandbox( .provider_profile_sources .snapshot_catalog(state.store.as_ref(), workspace) .await?; - let layers = profile_provider_policy_layers_with_catalog( + let layers = provider_policy_context_with_catalog( state.store.as_ref(), &catalog, workspace, provider_names, ) - .await?; + .await? + .layers; debug!( sandbox_id = %sandbox.object_id(), provider_layer_count = layers.len(), @@ -1538,13 +1565,14 @@ async fn validate_provider_composition_for_existing_sandboxes( .expect("catalog was inserted for sandbox workspace"); let base_policy = current_base_policy_for_sandbox(state.store.as_ref(), &sandbox).await?; - let provider_layers = profile_provider_policy_layers_with_catalog( + let provider_layers = provider_policy_context_with_catalog( state.store.as_ref(), catalog, &workspace, provider_names, ) - .await?; + .await? + .layers; validate_candidate_effective_policy(&base_policy, &provider_layers).map_err(|error| { Status::failed_precondition(format!( "cannot activate provider policy composition: sandbox '{}/{}' has an invalid effective policy: {}", @@ -1564,6 +1592,27 @@ async fn validate_provider_composition_for_existing_sandboxes( Ok(()) } +pub(super) async fn validate_candidate_sandbox_credential_policy( + state: &ServerState, + workspace: &str, + provider_names: &[String], + policy: Option<&ProtoSandboxPolicy>, +) -> Result<(), Status> { + let catalog = state + .provider_profile_sources + .snapshot_catalog(state.store.as_ref(), workspace) + .await?; + let effective = effective_policy_for_source( + state, + &catalog, + workspace, + provider_names, + policy.cloned().unwrap_or_default(), + ) + .await?; + validate_uninspected_credentialed_endpoints(&effective) +} + fn truncate_for_log(input: &str, max_chars: usize) -> String { let mut chars = input.chars(); let truncated: String = chars.by_ref().take(max_chars).collect(); @@ -1839,6 +1888,13 @@ pub(super) async fn handle_get_sandbox_config( load_sandbox_settings(state.store.as_ref(), &workspace, sandbox.object_name()).await?; let providers_v2_enabled = bool_setting_enabled(&global_settings, settings::PROVIDERS_V2_ENABLED_KEY)?; + let mut provider_policy_context = provider_policy_context_with_catalog( + state.store.as_ref(), + &provider_profile_catalog, + &workspace, + &sandbox_provider_names, + ) + .await?; let mut global_policy_version: u32 = 0; @@ -1858,28 +1914,42 @@ pub(super) async fn handle_get_sandbox_config( } } + if let Some(source_policy) = policy.as_mut() { + // Never trust provenance supplied by a persisted/user-authored policy. + // The gateway derives it from the attached provider catalog below. + clear_provider_credentialed_markers(source_policy); + } + if providers_v2_enabled && !matches!(policy_source, PolicySource::Global) && let Some(source_policy) = policy.as_ref() + && !provider_policy_context.layers.is_empty() { - let provider_layers = profile_provider_policy_layers_with_catalog( - state.store.as_ref(), - &provider_profile_catalog, - &workspace, - &sandbox_provider_names, - ) - .await?; - if !provider_layers.is_empty() { - let effective_policy = compose_effective_policy(source_policy, &provider_layers); - validate_policy_safety(&effective_policy).map_err(|error| { - Status::failed_precondition(format!( - "provider composition produced an invalid effective policy: {}", - error.message() - )) - })?; - policy_hash = deterministic_policy_hash(&effective_policy); - policy = Some(effective_policy); - } + let effective_policy = + compose_effective_policy(source_policy, &provider_policy_context.layers); + validate_policy_safety(&effective_policy).map_err(|error| { + Status::failed_precondition(format!( + "provider composition produced an invalid effective policy: {}", + error.message() + )) + })?; + policy_hash = deterministic_policy_hash(&effective_policy); + policy = Some(effective_policy); + } + + let policy_credential_bindings = policy_static_credential_endpoint_bindings(policy.as_ref())?; + extend_credentialed_scopes_from_policy_bindings( + &mut provider_policy_context.credentialed_scopes, + &policy_credential_bindings, + &provider_policy_context.endpointless_provider_names, + ); + if let Some(effective_policy) = policy.as_mut() { + stamp_provider_credentialed_endpoints( + effective_policy, + &provider_policy_context.credentialed_scopes, + ); + report_uninspected_credentialed_endpoints(effective_policy, &sandbox_id); + policy_hash = deterministic_policy_hash(effective_policy); } if let Some(policy) = policy.as_ref() { @@ -1904,7 +1974,6 @@ pub(super) async fn handle_get_sandbox_config( state.config.policy_validation_failure_mode, state.sandbox_jwt_issuer.is_some(), ); - let policy_credential_bindings = policy_static_credential_endpoint_bindings(policy.as_ref())?; if let Some(policy) = policy.as_ref() { validate_policy_credential_bindings_for_sandbox( state.as_ref(), @@ -2168,13 +2237,42 @@ async fn profile_provider_policy_layers( profile_provider_policy_layers_with_catalog(store, &catalog, workspace, provider_names).await } +#[cfg(test)] async fn profile_provider_policy_layers_with_catalog( store: &Store, catalog: &EffectiveProviderProfileCatalog, workspace: &str, provider_names: &[String], ) -> Result, Status> { + Ok( + provider_policy_context_with_catalog(store, catalog, workspace, provider_names) + .await? + .layers, + ) +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub(super) struct CredentialedEndpointScope { + host: String, + ports: Vec, +} + +#[derive(Debug, Default)] +struct ProviderPolicyContext { + layers: Vec, + credentialed_scopes: Vec, + endpointless_provider_names: HashSet, +} + +async fn provider_policy_context_with_catalog( + store: &Store, + catalog: &EffectiveProviderProfileCatalog, + workspace: &str, + provider_names: &[String], +) -> Result { let mut layers = Vec::new(); + let mut credentialed_scopes = Vec::new(); + let mut endpointless_provider_names = HashSet::new(); for name in provider_names { let provider = store @@ -2199,13 +2297,190 @@ async fn profile_provider_policy_layers_with_catalog( }; let rule_name = openshell_policy::provider_rule_name(provider.object_name()); + let mut rule = profile.network_policy_rule(&rule_name); + if rule.endpoints.is_empty() { + endpointless_provider_names.insert(name.clone()); + } + if profile.has_credentialed_endpoints() { + for endpoint in &mut rule.endpoints { + endpoint.provider_credentialed = true; + let scope = CredentialedEndpointScope { + host: endpoint.host.to_ascii_lowercase(), + ports: endpoint_ports(endpoint), + }; + if !credentialed_scopes.contains(&scope) { + credentialed_scopes.push(scope); + } + } + } layers.push(ProviderPolicyLayer { rule_name: rule_name.clone(), - rule: profile.network_policy_rule(&rule_name), + rule, }); } - Ok(layers) + Ok(ProviderPolicyContext { + layers, + credentialed_scopes, + endpointless_provider_names, + }) +} + +fn endpoint_ports(endpoint: &NetworkEndpoint) -> Vec { + if endpoint.ports.is_empty() { + (endpoint.port > 0) + .then_some(endpoint.port) + .into_iter() + .collect() + } else { + endpoint.ports.clone() + } +} + +fn extend_credentialed_scopes_from_policy_bindings( + scopes: &mut Vec, + bindings: &HashMap>, + endpointless_provider_names: &HashSet, +) { + for (provider_name, provider_bindings) in bindings { + if !endpointless_provider_names.contains(provider_name) { + continue; + } + for binding in provider_bindings { + let scope = CredentialedEndpointScope { + host: binding.host.to_ascii_lowercase(), + ports: vec![binding.port], + }; + if !scopes.contains(&scope) { + scopes.push(scope); + } + } + } +} + +fn endpoint_matches_credentialed_scope( + endpoint: &NetworkEndpoint, + scope: &CredentialedEndpointScope, +) -> bool { + if !host_patterns_overlap(&endpoint.host, &scope.host).unwrap_or(false) { + return false; + } + let endpoint_ports = endpoint_ports(endpoint); + endpoint_ports.is_empty() + || scope.ports.is_empty() + || endpoint_ports.iter().any(|port| scope.ports.contains(port)) +} + +pub(super) fn clear_provider_credentialed_markers(policy: &mut ProtoSandboxPolicy) { + for rule in policy.network_policies.values_mut() { + for endpoint in &mut rule.endpoints { + endpoint.provider_credentialed = false; + } + } +} + +fn stamp_provider_credentialed_endpoints( + policy: &mut ProtoSandboxPolicy, + scopes: &[CredentialedEndpointScope], +) { + for rule in policy.network_policies.values_mut() { + for endpoint in &mut rule.endpoints { + endpoint.provider_credentialed = scopes + .iter() + .any(|scope| endpoint_matches_credentialed_scope(endpoint, scope)); + } + } +} + +/// A credentialed endpoint whose configured mode disables the L7 inspection a +/// reviewer would expect, without an explicit `allow_uninspected_credentials`. +struct UninspectedCredentialedEndpoint { + rule_name: String, + host: String, + port: u32, + mode: &'static str, +} + +/// Scan the effective policy for credentialed endpoints on uninspected modes. +/// +/// Explicit opt-ins are logged and skipped. Never logs credential names, +/// placeholders, or secret values. +fn find_uninspected_credentialed_endpoint( + policy: &ProtoSandboxPolicy, +) -> Option { + for (rule_name, rule) in &policy.network_policies { + for endpoint in &rule.endpoints { + if !endpoint.provider_credentialed { + continue; + } + + let mode = if endpoint.protocol.trim().is_empty() { + "L4-only" + } else if endpoint.tls.trim().eq_ignore_ascii_case("skip") { + "tls: skip" + } else { + continue; + }; + + if endpoint.allow_uninspected_credentials { + warn!( + rule_name, + host = %endpoint.host, + ports = ?endpoint_ports(endpoint), + mode, + "credentialed endpoint explicitly allows uninspected traffic" + ); + continue; + } + + return Some(UninspectedCredentialedEndpoint { + rule_name: rule_name.clone(), + host: endpoint.host.clone(), + port: endpoint_ports(endpoint) + .first() + .copied() + .unwrap_or(endpoint.port), + mode, + }); + } + } + None +} + +/// Admission gate for policy-authoring paths (create, attach, operator config +/// update). Rejects credentialed endpoints that would lose L7 inspection. +fn validate_uninspected_credentialed_endpoints(policy: &ProtoSandboxPolicy) -> Result<(), Status> { + let Some(violation) = find_uninspected_credentialed_endpoint(policy) else { + return Ok(()); + }; + + warn!( + rule_name = %violation.rule_name, + host = %violation.host, + port = violation.port, + mode = violation.mode, + "rejecting uninspected credentialed endpoint" + ); + Err(Status::failed_precondition(format!( + "credentialed endpoint '{}:{}' in rule '{}' uses {}; configure L7 inspection or explicitly set allow_uninspected_credentials: true", + violation.host, violation.port, violation.rule_name, violation.mode + ))) +} + +/// Delivery-path reporting for an already-persisted policy. Sandbox config +/// delivery must not fail closed here: refusing the config would crash-loop a +/// running supervisor. The runtime backstop denies the traffic instead. +fn report_uninspected_credentialed_endpoints(policy: &ProtoSandboxPolicy, sandbox_id: &str) { + if let Some(violation) = find_uninspected_credentialed_endpoint(policy) { + warn!( + sandbox_id, + rule_name = %violation.rule_name, + host = %violation.host, + port = violation.port, + mode = violation.mode, + "delivering credentialed endpoint without L7 inspection; the sandbox proxy will deny this traffic unless allow_uninspected_credentials is set" + ); + } } pub(super) fn bool_setting_enabled(settings: &StoredSettings, key: &str) -> Result { @@ -2429,6 +2704,7 @@ async fn handle_update_config_inner( let mut new_policy = req.policy.ok_or_else(|| { Status::invalid_argument("policy is required for global policy update") })?; + clear_provider_credentialed_markers(&mut new_policy); normalize_process_identity_for_driver(&mut new_policy, state.compute.driver_kind()); validate_no_reserved_provider_policy_keys(&new_policy)?; validate_policy_safety(&new_policy)?; @@ -2714,23 +2990,10 @@ async fn handle_update_config_inner( .ok_or_else(|| Status::internal("sandbox has no spec"))?; let merge_ops = parse_merge_operations(&req.merge_operations)?; validate_merge_operations_for_server(&merge_ops)?; - let provider_layers = - provider_policy_layers_for_sandbox(state, &workspace, &sandbox, &spec.providers) + let merge_validation = + sandbox_policy_merge_validation_data(state, &workspace, &sandbox, &spec.providers) .await?; - let provider_profile_catalog = state - .provider_profile_sources - .snapshot_catalog(state.store.as_ref(), &workspace) - .await?; - let provider_records = super::provider::load_provider_environment_records( - state.store.as_ref(), - &workspace, - &spec.providers, - ) - .await?; - let credential_binding_context = PolicyCredentialBindingValidationContext { - catalog: &provider_profile_catalog, - records: &provider_records, - }; + let credential_binding_context = merge_validation.credential_binding_context(); let atomic_context = AtomicPolicyWriteContext { expected_resource_version: req.expected_resource_version, provenance: &req.annotations, @@ -2747,7 +3010,7 @@ async fn handle_update_config_inner( baseline_policy.as_ref(), &merge_ops, PolicyMergeValidationContext { - provider_layers: &provider_layers, + provider_layers: &merge_validation.provider_layers, credential_binding: Some(&credential_binding_context), }, Some(&atomic_context), @@ -2814,6 +3077,7 @@ async fn handle_update_config_inner( let mut new_policy = req .policy .ok_or_else(|| Status::invalid_argument("policy is required"))?; + clear_provider_credentialed_markers(&mut new_policy); normalize_process_identity_for_driver(&mut new_policy, state.compute.driver_kind()); let global_settings = load_global_settings(state.store.as_ref()).await?; @@ -2873,6 +3137,18 @@ async fn handle_update_config_inner( &effective_policy, ) .await?; + // Sandbox-authored syncs replay a policy the supervisor already discovered + // on disk. Rejecting it here would crash-loop the sandbox instead of + // surfacing an operator decision, so only operator-authored updates gate. + if !sandbox_caller { + validate_candidate_sandbox_credential_policy( + state, + &workspace, + &spec.providers, + Some(&new_policy), + ) + .await?; + } let _sandbox_sync_guard = if backfill_policy.is_some() { Some(state.compute.sandbox_sync_guard().await) @@ -3797,14 +4073,18 @@ async fn handle_approve_draft_chunk_inner( .as_ref() .map(|spec| spec.providers.as_slice()) .unwrap_or_default(); - let provider_layers = - provider_policy_layers_for_sandbox(state, &workspace, &sandbox, provider_names).await?; - let (version, hash) = merge_chunk_into_policy( + let merge_validation = + sandbox_policy_merge_validation_data(state, &workspace, &sandbox, provider_names).await?; + let credential_binding_context = merge_validation.credential_binding_context(); + let (version, hash) = merge_chunk_into_policy_with_validation( state.store.as_ref(), &sandbox_id, &workspace, &chunk, - &provider_layers, + PolicyMergeValidationContext { + provider_layers: &merge_validation.provider_layers, + credential_binding: Some(&credential_binding_context), + }, ) .await?; let chunk_summary = summarize_draft_chunk_rule(&chunk)?; @@ -4024,8 +4304,13 @@ async fn handle_approve_all_draft_chunks_inner( .as_ref() .map(|spec| spec.providers.as_slice()) .unwrap_or_default(); - let provider_layers = - provider_policy_layers_for_sandbox(state, &workspace, &sandbox, provider_names).await?; + let merge_validation = + sandbox_policy_merge_validation_data(state, &workspace, &sandbox, provider_names).await?; + let credential_binding_context = merge_validation.credential_binding_context(); + let merge_validation_context = PolicyMergeValidationContext { + provider_layers: &merge_validation.provider_layers, + credential_binding: Some(&credential_binding_context), + }; let mut bulk_candidate = current_base_policy_for_sandbox(state.store.as_ref(), &sandbox).await?; for chunk in &pending_chunks { @@ -4043,9 +4328,21 @@ async fn handle_approve_all_draft_chunks_inner( bulk_candidate = merge_policy(bulk_candidate, &operations) .map_err(map_policy_merge_error)? .policy; + validate_policy_safety(&bulk_candidate)?; + validate_candidate_effective_policy(&bulk_candidate, &merge_validation.provider_layers)?; + let mut prefix_effective_policy = if merge_validation.provider_layers.is_empty() { + bulk_candidate.clone() + } else { + compose_effective_policy(&bulk_candidate, &merge_validation.provider_layers) + }; + let prefix_bindings = + policy_static_credential_endpoint_bindings(Some(&prefix_effective_policy))?; + validate_operator_merged_credential_policy( + &mut prefix_effective_policy, + &prefix_bindings, + &credential_binding_context, + )?; } - validate_policy_safety(&bulk_candidate)?; - validate_candidate_effective_policy(&bulk_candidate, &provider_layers)?; for chunk in &pending_chunks { let security_notes = current_draft_chunk_security_notes(chunk)?; @@ -4070,12 +4367,12 @@ async fn handle_approve_all_draft_chunks_inner( "ApproveAllDraftChunks: merging chunk" ); - let (version, hash) = merge_chunk_into_policy( + let (version, hash) = merge_chunk_into_policy_with_validation( state.store.as_ref(), &sandbox_id, &workspace, chunk, - &provider_layers, + merge_validation_context, ) .await?; last_version = version; @@ -4634,6 +4931,12 @@ fn generate_security_notes(rule: &NetworkPolicyRule) -> String { for endpoint in &rule.endpoints { let host = endpoint.host.to_lowercase(); + if endpoint.allow_uninspected_credentials { + notes.push(format!( + "Endpoint '{host}' explicitly allows credentials on traffic OpenShell cannot inspect or rewrite." + )); + } + // Flag destinations that are an internal/private address. Parse the host as // an IP literal and defer to the canonical RFC-accurate classifier // (openshell-core net::is_internal_ip) rather than naive string prefixes: @@ -4959,13 +5262,105 @@ struct AtomicPolicyWriteContext<'a> { struct PolicyCredentialBindingValidationContext<'a> { catalog: &'a EffectiveProviderProfileCatalog, records: &'a [super::provider::ProviderEnvironmentRecord], + credentialed_scopes: &'a [CredentialedEndpointScope], + endpointless_provider_names: &'a HashSet, +} + +struct SandboxPolicyMergeValidationData { + provider_layers: Vec, + catalog: EffectiveProviderProfileCatalog, + records: Vec, + credentialed_scopes: Vec, + endpointless_provider_names: HashSet, +} + +impl SandboxPolicyMergeValidationData { + fn credential_binding_context(&self) -> PolicyCredentialBindingValidationContext<'_> { + PolicyCredentialBindingValidationContext { + catalog: &self.catalog, + records: &self.records, + credentialed_scopes: &self.credentialed_scopes, + endpointless_provider_names: &self.endpointless_provider_names, + } + } +} + +async fn sandbox_policy_merge_validation_data( + state: &ServerState, + workspace: &str, + sandbox: &Sandbox, + provider_names: &[String], +) -> Result { + let global_settings = load_global_settings(state.store.as_ref()).await?; + let composition_enabled = provider_policy_composition_enabled_in(&global_settings)?; + let catalog = state + .provider_profile_sources + .snapshot_catalog(state.store.as_ref(), workspace) + .await?; + let ProviderPolicyContext { + layers, + credentialed_scopes, + endpointless_provider_names, + } = provider_policy_context_with_catalog( + state.store.as_ref(), + &catalog, + workspace, + provider_names, + ) + .await?; + let provider_layers = if composition_enabled { + layers + } else { + Vec::new() + }; + debug!( + sandbox_id = %sandbox.object_id(), + provider_layer_count = provider_layers.len(), + "Composed provider policy and credential context for merge validation" + ); + let records = super::provider::load_provider_environment_records( + state.store.as_ref(), + workspace, + provider_names, + ) + .await?; + Ok(SandboxPolicyMergeValidationData { + provider_layers, + catalog, + records, + credentialed_scopes, + endpointless_provider_names, + }) } +#[derive(Clone, Copy)] struct PolicyMergeValidationContext<'a> { provider_layers: &'a [ProviderPolicyLayer], credential_binding: Option<&'a PolicyCredentialBindingValidationContext<'a>>, } +fn validate_operator_merged_credential_policy( + effective_policy: &mut ProtoSandboxPolicy, + bindings: &HashMap>, + context: &PolicyCredentialBindingValidationContext<'_>, +) -> Result<(), Status> { + validate_policy_credential_binding_context( + context.catalog, + context.records, + effective_policy, + bindings, + )?; + let mut credentialed_scopes = context.credentialed_scopes.to_vec(); + extend_credentialed_scopes_from_policy_bindings( + &mut credentialed_scopes, + bindings, + context.endpointless_provider_names, + ); + clear_provider_credentialed_markers(effective_policy); + stamp_provider_credentialed_endpoints(effective_policy, &credentialed_scopes); + validate_uninspected_credentialed_endpoints(effective_policy) +} + async fn apply_merge_operations_with_retry( store: &Store, sandbox_id: &str, @@ -4998,19 +5393,14 @@ async fn apply_merge_operations_with_retry( } validate_policy_safety(&new_policy)?; validate_candidate_effective_policy(&new_policy, provider_layers)?; - let effective_policy = if provider_layers.is_empty() { + let mut effective_policy = if provider_layers.is_empty() { new_policy.clone() } else { compose_effective_policy(&new_policy, provider_layers) }; let bindings = policy_static_credential_endpoint_bindings(Some(&effective_policy))?; if let Some(context) = validation_context.credential_binding { - validate_policy_credential_binding_context( - context.catalog, - context.records, - &effective_policy, - &bindings, - )?; + validate_operator_merged_credential_policy(&mut effective_policy, &bindings, context)?; } if let Some(ref current) = latest @@ -5102,12 +5492,12 @@ async fn apply_merge_operations_with_retry( ))) } -pub(super) async fn merge_chunk_into_policy( +async fn merge_chunk_into_policy_with_validation( store: &Store, sandbox_id: &str, workspace: &str, chunk: &DraftChunkRecord, - provider_layers: &[ProviderPolicyLayer], + validation_context: PolicyMergeValidationContext<'_>, ) -> Result<(i64, String), Status> { let rule = NetworkPolicyRule::decode(chunk.proposed_rule.as_slice()) .map_err(|e| Status::internal(format!("decode proposed_rule failed: {e}")))?; @@ -5122,14 +5512,32 @@ pub(super) async fn merge_chunk_into_policy( workspace, None, &operations, + validation_context, + None, + ) + .await + .map(|(version, hash, _)| (version, hash)) +} + +#[cfg(test)] +async fn merge_chunk_into_policy( + store: &Store, + sandbox_id: &str, + workspace: &str, + chunk: &DraftChunkRecord, + provider_layers: &[ProviderPolicyLayer], +) -> Result<(i64, String), Status> { + merge_chunk_into_policy_with_validation( + store, + sandbox_id, + workspace, + chunk, PolicyMergeValidationContext { provider_layers, credential_binding: None, }, - None, ) .await - .map(|(version, hash, _)| (version, hash)) } async fn remove_chunk_from_policy( @@ -5527,6 +5935,164 @@ mod tests { }) } + #[test] + fn provider_credentialed_stamping_matches_host_patterns_and_ports() { + let mut policy = ProtoSandboxPolicy { + network_policies: HashMap::from([( + "test".to_string(), + NetworkPolicyRule { + endpoints: vec![ + NetworkEndpoint { + host: "api.example.com".to_string(), + port: 443, + provider_credentialed: true, + ..Default::default() + }, + NetworkEndpoint { + host: "api.example.com".to_string(), + port: 8443, + provider_credentialed: true, + ..Default::default() + }, + NetworkEndpoint { + host: "*.api.example.com".to_string(), + port: 443, + provider_credentialed: true, + ..Default::default() + }, + ], + ..Default::default() + }, + )]), + ..Default::default() + }; + let scopes = vec![CredentialedEndpointScope { + host: "*.example.com".to_string(), + ports: vec![443], + }]; + + clear_provider_credentialed_markers(&mut policy); + stamp_provider_credentialed_endpoints(&mut policy, &scopes); + + let endpoints = &policy.network_policies["test"].endpoints; + assert!(endpoints[0].provider_credentialed); + assert!(!endpoints[1].provider_credentialed); + assert!(!endpoints[2].provider_credentialed); + } + + #[test] + fn policy_bindings_add_scopes_only_for_attached_endpointless_providers() { + let mut scopes = vec![CredentialedEndpointScope { + host: "profile.example.com".to_string(), + ports: vec![443], + }]; + let bindings = HashMap::from([ + ( + "bound".to_string(), + vec![ + StaticCredentialEndpointBinding { + host: "API.Bound.Example".to_string(), + port: 8443, + path: "/v1".to_string(), + }, + StaticCredentialEndpointBinding { + host: "api.bound.example".to_string(), + port: 8443, + path: "/v2".to_string(), + }, + ], + ), + ( + "endpointful".to_string(), + vec![StaticCredentialEndpointBinding { + host: "profile-bound.example".to_string(), + port: 443, + path: String::new(), + }], + ), + ( + "unattached".to_string(), + vec![StaticCredentialEndpointBinding { + host: "unattached.example".to_string(), + port: 443, + path: String::new(), + }], + ), + ]); + + extend_credentialed_scopes_from_policy_bindings( + &mut scopes, + &bindings, + &HashSet::from(["bound".to_string()]), + ); + + assert_eq!( + scopes, + vec![ + CredentialedEndpointScope { + host: "profile.example.com".to_string(), + ports: vec![443], + }, + CredentialedEndpointScope { + host: "api.bound.example".to_string(), + ports: vec![8443], + }, + ] + ); + } + + #[test] + fn credentialed_l4_and_tls_skip_require_explicit_opt_in() { + let endpoint = |protocol: &str, tls: &str, allow: bool| NetworkEndpoint { + host: "api.vendor.example".to_string(), + port: 443, + protocol: protocol.to_string(), + tls: tls.to_string(), + provider_credentialed: true, + allow_uninspected_credentials: allow, + ..Default::default() + }; + let policy = |endpoint| ProtoSandboxPolicy { + network_policies: HashMap::from([( + "vendor".to_string(), + NetworkPolicyRule { + endpoints: vec![endpoint], + ..Default::default() + }, + )]), + ..Default::default() + }; + + assert!( + validate_uninspected_credentialed_endpoints(&policy(endpoint("", "", false))).is_err() + ); + assert!( + validate_uninspected_credentialed_endpoints(&policy(endpoint("rest", "skip", false))) + .is_err() + ); + assert!( + validate_uninspected_credentialed_endpoints(&policy(endpoint("", "", true))).is_ok() + ); + + let mut plain = endpoint("", "", false); + plain.provider_credentialed = false; + assert!(validate_uninspected_credentialed_endpoints(&policy(plain)).is_ok()); + } + + #[test] + fn security_notes_flag_allow_uninspected_credentials() { + let notes = generate_security_notes(&NetworkPolicyRule { + endpoints: vec![NetworkEndpoint { + host: "api.vendor.example".to_string(), + port: 443, + allow_uninspected_credentials: true, + ..Default::default() + }], + ..Default::default() + }); + assert!(notes.contains("cannot inspect or rewrite")); + } + #[test] fn security_notes_use_canonical_internal_ip_classifier() { // RFC 1918 is 172.16.0.0/12 only: the old starts_with("172.") prefix @@ -6501,6 +7067,8 @@ mod tests { endpoints: vec![NetworkEndpoint { host: host.to_string(), port: 443, + protocol: "rest".to_string(), + access: "full".to_string(), ..Default::default() }], ..Default::default() @@ -6873,6 +7441,13 @@ mod tests { assert_eq!(layers.len(), 1); assert_eq!(layers[0].rule_name, "_provider_work_github"); assert_eq!(layers[0].rule.endpoints.len(), 3); + assert!( + layers[0] + .rule + .endpoints + .iter() + .all(|endpoint| endpoint.provider_credentialed) + ); assert!( layers[0] .rule @@ -7247,6 +7822,159 @@ mod tests { assert!(error.message().contains("already defines endpoints")); } + #[tokio::test] + async fn update_config_gates_uninspected_endpointless_credential_binding() { + use openshell_core::proto::{ + ProviderProfile, ProviderProfileCategory, StoredProviderProfile, + }; + + let state = test_server_state().await; + state + .store + .put_message(&StoredProviderProfile { + metadata: Some(openshell_core::proto::datamodel::v1::ObjectMeta { + id: "profile-endpointless-gating".to_string(), + name: "endpointless-gating".to_string(), + workspace: "default".to_string(), + ..Default::default() + }), + profile: Some(ProviderProfile { + id: "endpointless-gating".to_string(), + display_name: "Endpointless Gating".to_string(), + category: ProviderProfileCategory::Other as i32, + endpoints: Vec::new(), + ..Default::default() + }), + }) + .await + .unwrap(); + state + .store + .put_message(&test_provider("work-endpointless", "endpointless-gating")) + .await + .unwrap(); + let mut sandbox = test_sandbox( + "sb-endpointless-gating", + "endpointless-gating", + ProtoSandboxPolicy::default(), + vec!["work-endpointless".to_string()], + ); + sandbox.spec.as_mut().unwrap().policy = None; + state.store.put_message(&sandbox).await.unwrap(); + + let l4 = + test_policy_with_credential_binding("bound", "api.bound.example", "work-endpointless"); + let add_bound_rule = |policy: &ProtoSandboxPolicy| PolicyMergeOperation { + operation: Some(policy_merge_operation::Operation::AddRule( + openshell_core::proto::AddNetworkRule { + rule_name: "bound".to_string(), + rule: Some(policy.network_policies["bound"].clone()), + }, + )), + }; + let l4_error = handle_update_config( + &state, + with_user(Request::new(UpdateConfigRequest { + name: "endpointless-gating".to_string(), + workspace: "default".to_string(), + policy: Some(l4.clone()), + ..Default::default() + })), + ) + .await + .expect_err("L4-only credential binding must be rejected"); + assert_eq!(l4_error.code(), Code::FailedPrecondition); + assert!(l4_error.message().contains("L4-only")); + + let mut tls_skip = l4.clone(); + let tls_endpoint = &mut tls_skip + .network_policies + .get_mut("bound") + .unwrap() + .endpoints[0]; + tls_endpoint.protocol = "rest".to_string(); + tls_endpoint.access = "full".to_string(); + tls_endpoint.tls = "skip".to_string(); + let tls_error = handle_update_config( + &state, + with_user(Request::new(UpdateConfigRequest { + name: "endpointless-gating".to_string(), + workspace: "default".to_string(), + policy: Some(tls_skip.clone()), + ..Default::default() + })), + ) + .await + .expect_err("tls: skip credential binding must be rejected"); + assert_eq!(tls_error.code(), Code::FailedPrecondition); + assert!(tls_error.message().contains("tls: skip")); + + let merge_l4_error = handle_update_config( + &state, + with_user(Request::new(UpdateConfigRequest { + name: "endpointless-gating".to_string(), + workspace: "default".to_string(), + merge_operations: vec![add_bound_rule(&l4)], + ..Default::default() + })), + ) + .await + .expect_err("L4-only credential binding merge must be rejected"); + assert_eq!(merge_l4_error.code(), Code::FailedPrecondition); + assert!(merge_l4_error.message().contains("L4-only")); + + let merge_tls_error = handle_update_config( + &state, + with_user(Request::new(UpdateConfigRequest { + name: "endpointless-gating".to_string(), + workspace: "default".to_string(), + merge_operations: vec![add_bound_rule(&tls_skip)], + ..Default::default() + })), + ) + .await + .expect_err("tls: skip credential binding merge must be rejected"); + assert_eq!(merge_tls_error.code(), Code::FailedPrecondition); + assert!(merge_tls_error.message().contains("tls: skip")); + + assert!( + state + .store + .get_latest_policy("sb-endpointless-gating") + .await + .unwrap() + .is_none(), + "rejected policies must not leave a revision in history" + ); + + let mut opted_in = l4; + opted_in + .network_policies + .get_mut("bound") + .unwrap() + .endpoints[0] + .allow_uninspected_credentials = true; + handle_update_config( + &state, + with_user(Request::new(UpdateConfigRequest { + name: "endpointless-gating".to_string(), + workspace: "default".to_string(), + merge_operations: vec![add_bound_rule(&opted_in)], + ..Default::default() + })), + ) + .await + .expect("explicit opt-in must admit the endpointless credential binding merge"); + assert!( + state + .store + .get_latest_policy("sb-endpointless-gating") + .await + .unwrap() + .is_some() + ); + } + #[tokio::test] async fn update_config_rejects_sigv4_without_credential_source_before_persisting_revision() { let state = test_server_state().await; @@ -7857,12 +8585,20 @@ mod tests { .put_message(&test_provider("work-github", "github")) .await .unwrap(); + let mut policy = test_policy_with_rule("custom_github", "api.github.com"); + let endpoint = &mut policy + .network_policies + .get_mut("custom_github") + .expect("custom rule") + .endpoints[0]; + endpoint.protocol = "rest".to_string(); + endpoint.access = "read-only".to_string(); state .store .put_message(&test_sandbox( "sb-overlap", "overlap", - test_policy_with_rule("custom_github", "api.github.com"), + policy, vec!["work-github".to_string()], )) .await @@ -8019,6 +8755,14 @@ mod tests { .credential_binding = Some(NetworkCredentialBinding { provider: "work-cloud".to_string(), }); + let bound_endpoint = &mut policy + .network_policies + .get_mut("cloud_api") + .unwrap() + .endpoints[0]; + bound_endpoint.protocol = "rest".to_string(); + bound_endpoint.access = "full".to_string(); + bound_endpoint.tls = "terminate".to_string(); openshell_policy::ensure_sandbox_process_identity(&mut policy); state .store @@ -8051,6 +8795,11 @@ mod tests { .unwrap() .into_inner(); + assert!( + config.policy.as_ref().unwrap().network_policies["cloud_api"].endpoints[0] + .provider_credentialed, + "config delivery must derive provenance from the endpointless binding" + ); assert_eq!( environment.environment.get("CLOUD_TOKEN"), Some(&"cloud-secret".to_string()) @@ -8106,6 +8855,10 @@ mod tests { .unwrap() .into_inner(); + assert!( + next_config.policy.as_ref().unwrap().network_policies["cloud_api"].endpoints[0] + .provider_credentialed + ); assert_ne!( config.provider_env_revision, next_config.provider_env_revision, "changing the policy binding must rotate the provider environment revision" @@ -8137,6 +8890,15 @@ mod tests { ) .await .expect("removing a policy binding must succeed"); + let unbound_config = handle_get_sandbox_config( + &state, + with_user(Request::new(GetSandboxConfigRequest { + sandbox_id: "sb-policy-binding".to_string(), + })), + ) + .await + .unwrap() + .into_inner(); let unbound_environment = handle_get_sandbox_provider_environment( &state, with_user(Request::new(GetSandboxProviderEnvironmentRequest { @@ -8148,6 +8910,11 @@ mod tests { .unwrap() .into_inner(); + assert!( + !unbound_config.policy.as_ref().unwrap().network_policies["cloud_api"].endpoints[0] + .provider_credentialed, + "removing the binding must clear the derived provenance" + ); assert_ne!( next_environment.provider_env_revision, unbound_environment.provider_env_revision, "removing the binding must rotate the provider environment revision" @@ -8206,6 +8973,8 @@ mod tests { host: "api.dynamic.example.test".to_string(), port: 443, path: "/**".to_string(), + protocol: "rest".to_string(), + access: "full".to_string(), ..Default::default() }], ..Default::default() @@ -8380,6 +9149,8 @@ mod tests { host: endpoint_host.to_string(), port: 443, path: "/**".to_string(), + protocol: "rest".to_string(), + access: "full".to_string(), ..Default::default() }], ..Default::default() @@ -8565,6 +9336,8 @@ mod tests { endpoints: vec![NetworkEndpoint { host: "api.custom.example".to_string(), port: 443, + protocol: "rest".to_string(), + access: "full".to_string(), ..Default::default() }], binaries: Vec::new(), @@ -9766,6 +10539,7 @@ mod tests { endpoints: vec![NetworkEndpoint { host: "api.github.com".to_string(), port: 443, + allow_uninspected_credentials: true, ..Default::default() }], binaries: vec![NetworkBinary { @@ -9813,8 +10587,8 @@ mod tests { .into_inner(); assert_eq!(draft_policy.draft_version, 1); assert_eq!(draft_policy.chunks.len(), 1); - // The proposal is L4 to a host with a credential in scope, so the - // prover emits a HIGH finding and the chunk stays pending for the + // The proposal explicitly opts in to L4 credentials. The prover emits + // a HIGH finding and the security note keeps the chunk pending for the // manual approve path this test exercises. assert_eq!(draft_policy.chunks[0].status, "pending"); let chunk_id = draft_policy.chunks[0].id.clone(); @@ -12511,9 +13285,9 @@ mod tests { use openshell_core::proto::{NetworkBinary, NetworkEndpoint, SandboxPhase, SandboxSpec}; let state = test_server_state().await; - // Attach a github provider so the L4 proposal below has a credential - // in scope and the prover emits a HIGH finding — keeps the chunk - // pending so this cross-sandbox approve check is reachable. + // Attach a github provider so the explicitly opted-in L4 proposal + // below has a credential in scope and stays pending, keeping this + // cross-sandbox approve check reachable. state .store .put_message(&test_provider("github-pat", "github")) @@ -12564,6 +13338,7 @@ mod tests { endpoints: vec![NetworkEndpoint { host: "api.github.com".to_string(), port: 443, + allow_uninspected_credentials: true, ..Default::default() }], binaries: vec![NetworkBinary { diff --git a/crates/openshell-server/src/grpc/sandbox.rs b/crates/openshell-server/src/grpc/sandbox.rs index 9338956950..d5dd4e04e4 100644 --- a/crates/openshell-server/src/grpc/sandbox.rs +++ b/crates/openshell-server/src/grpc/sandbox.rs @@ -271,11 +271,19 @@ async fn handle_create_sandbox_inner( // Docker and Podman preserve omitted identity fields for OCI USER // fallback. Other drivers retain the legacy persisted sandbox defaults. if let Some(ref mut policy) = spec.policy { + super::policy::clear_provider_credentialed_markers(policy); normalize_process_identity_for_driver(policy, state.compute.driver_kind()); validate_no_reserved_provider_policy_keys(policy)?; validate_policy_safety(policy)?; crate::middleware::validate_policy(state.middleware_registry.as_ref(), policy).await?; } + super::policy::validate_candidate_sandbox_credential_policy( + state, + &workspace, + &spec.providers, + spec.policy.as_ref(), + ) + .await?; let id = uuid::Uuid::new_v4().to_string(); let name = if request.name.is_empty() { @@ -573,6 +581,13 @@ pub(super) async fn handle_attach_sandbox_provider( &candidate_spec.providers, ) .await?; + super::policy::validate_candidate_sandbox_credential_policy( + state, + &workspace, + &candidate_spec.providers, + candidate_spec.policy.as_ref(), + ) + .await?; let provider_name = request.provider_name.clone(); let attached = Arc::new(AtomicBool::new(false)); diff --git a/crates/openshell-supervisor-network/data/sandbox-policy.rego b/crates/openshell-supervisor-network/data/sandbox-policy.rego index 780080c54a..4b2977b936 100644 --- a/crates/openshell-supervisor-network/data/sandbox-policy.rego +++ b/crates/openshell-supervisor-network/data/sandbox-policy.rego @@ -868,6 +868,25 @@ matched_endpoint_config := _matching_endpoint_configs[0] if { count(_matching_endpoint_configs) > 0 } +# --- Credential provenance view (credential gating only) --- +# Deliberately separate from `_matching_endpoint_configs`. The credential guard +# must also see L4-only endpoints, which carry no extended config. Widening +# `endpoint_has_extended_config` instead would let such an endpoint become +# element [0] of the shared list and shadow the TLS mode, SSRF allowlist, and +# L7 protocol of an inspected endpoint on the same host:port. +_policy_credential_guards(policy) := [ep | + some ep + ep := policy.endpoints[_] + endpoint_matches_request(ep, input.network) +] + +endpoint_credential_guards := [cfg | + some pname + _matching_policy_names[pname] + cfgs := _policy_credential_guards(data.network_policies[pname]) + cfg := cfgs[_] +] + # Expose middleware policy data to Rust. Selection and validation stay in Rust; # Rego does not evaluate middleware selectors. network_middlewares := object.get(data, "network_middlewares", {}) diff --git a/crates/openshell-supervisor-network/src/l7/mod.rs b/crates/openshell-supervisor-network/src/l7/mod.rs index 3b2849f5a3..9279d3f089 100644 --- a/crates/openshell-supervisor-network/src/l7/mod.rs +++ b/crates/openshell-supervisor-network/src/l7/mod.rs @@ -127,6 +127,10 @@ pub struct L7EndpointConfig { /// Opt-in rewrite of credential placeholders in supported textual REST /// request bodies before forwarding upstream. pub request_body_credential_rewrite: bool, + /// Explicit opt-in to credential-bearing traffic that cannot be inspected. + pub allow_uninspected_credentials: bool, + /// Internal gateway-derived credential provenance for this endpoint. + pub provider_credentialed: bool, /// When true, client-to-server GraphQL-over-WebSocket operation messages /// are classified with the same operation policy used by GraphQL-over-HTTP. pub websocket_graphql_policy: bool, @@ -210,6 +214,9 @@ pub fn parse_l7_config(val: ®orus::Value) -> Option { get_object_bool(val, "websocket_credential_rewrite").unwrap_or(false); let request_body_credential_rewrite = get_object_bool(val, "request_body_credential_rewrite").unwrap_or(false); + let allow_uninspected_credentials = + get_object_bool(val, "allow_uninspected_credentials").unwrap_or(false); + let provider_credentialed = get_object_bool(val, "provider_credentialed").unwrap_or(false); let websocket_graphql_policy = protocol == L7Protocol::Websocket && endpoint_has_graphql_policy(val); let graphql_max_body_bytes = get_object_u64(val, "graphql_max_body_bytes") @@ -265,6 +272,8 @@ pub fn parse_l7_config(val: ®orus::Value) -> Option { allow_encoded_slash, websocket_credential_rewrite, request_body_credential_rewrite, + allow_uninspected_credentials, + provider_credentialed, websocket_graphql_policy, credential_signing, signing_service, @@ -272,6 +281,24 @@ pub fn parse_l7_config(val: ®orus::Value) -> Option { }) } +pub(crate) fn emit_uninspected_credential_finding(host: &str, policy_name: &str, surface: &str) { + let event = openshell_ocsf::DetectionFindingBuilder::new(openshell_ocsf::ctx::ctx()) + .severity(openshell_ocsf::SeverityId::High) + .finding_info(openshell_ocsf::FindingInfo::new( + "openshell.credentials.traffic_uninspectable", + "Credential-bearing traffic cannot be inspected", + )) + .evidence_pairs(&[ + ("policy", policy_name), + ("host", host), + ("surface", surface), + ("disposition", "denied"), + ]) + .message("Uninspected credential-bearing traffic denied") + .build(); + openshell_ocsf::ocsf_emit!(event); +} + impl L7EndpointConfig { pub fn matches_path(&self, path: &str) -> bool { endpoint_path_matches(&self.path, path) @@ -284,6 +311,10 @@ impl L7EndpointConfig { self.path.chars().filter(|c| *c != '*').count() } } + + pub fn deny_uninspected_body_credentials(&self, has_resolver: bool) -> bool { + !self.allow_uninspected_credentials && (self.provider_credentialed || has_resolver) + } } pub fn endpoint_path_matches(pattern: &str, path: &str) -> bool { @@ -302,6 +333,34 @@ pub fn parse_tls_mode(val: ®orus::Value) -> TlsMode { } } +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct EndpointCredentialGuard { + pub provider_credentialed: bool, + pub allow_uninspected_credentials: bool, + pub has_l7_protocol: bool, + pub tls: TlsMode, +} + +impl EndpointCredentialGuard { + pub fn blocks_uninspected(self) -> bool { + self.provider_credentialed && !self.allow_uninspected_credentials + } + + pub fn blocks_connect(self) -> bool { + self.blocks_uninspected() && (!self.has_l7_protocol || self.tls == TlsMode::Skip) + } +} + +pub fn parse_endpoint_credential_guard(val: ®orus::Value) -> EndpointCredentialGuard { + EndpointCredentialGuard { + provider_credentialed: get_object_bool(val, "provider_credentialed").unwrap_or(false), + allow_uninspected_credentials: get_object_bool(val, "allow_uninspected_credentials") + .unwrap_or(false), + has_l7_protocol: get_object_str(val, "protocol").is_some(), + tls: parse_tls_mode(val), + } +} + /// Extract a bool value from a regorus object. Returns `None` when the key /// is absent or not a boolean. fn get_object_bool(val: ®orus::Value, key: &str) -> Option { @@ -1696,6 +1755,43 @@ mod tests { assert!(parse_l7_config(&val).is_none()); } + #[test] + fn parse_endpoint_credential_guard_handles_l4_and_opt_in() { + let guarded = regorus::Value::from_json_str( + r#"{"host":"api.example.com","ports":[443],"provider_credentialed":true}"#, + ) + .unwrap(); + let guard = parse_endpoint_credential_guard(&guarded); + assert!(guard.blocks_connect()); + + let opted_in = regorus::Value::from_json_str( + r#"{"host":"api.example.com","ports":[443],"provider_credentialed":true,"allow_uninspected_credentials":true}"#, + ) + .unwrap(); + assert!(!parse_endpoint_credential_guard(&opted_in).blocks_connect()); + } + + #[test] + fn generic_resolver_enables_rest_body_backstop_without_provider_marker() { + let val = regorus::Value::from_json_str( + r#"{"protocol":"rest","host":"api.example.com","ports":[443]}"#, + ) + .unwrap(); + let config = parse_l7_config(&val).unwrap(); + assert!(!config.deny_uninspected_body_credentials(false)); + assert!(config.deny_uninspected_body_credentials(true)); + + let opted_in = regorus::Value::from_json_str( + r#"{"protocol":"rest","host":"api.example.com","ports":[443],"allow_uninspected_credentials":true}"#, + ) + .unwrap(); + assert!( + !parse_l7_config(&opted_in) + .unwrap() + .deny_uninspected_body_credentials(true) + ); + } + #[test] fn parse_l7_config_allow_encoded_slash_defaults_false() { let val = regorus::Value::from_json_str( diff --git a/crates/openshell-supervisor-network/src/l7/relay.rs b/crates/openshell-supervisor-network/src/l7/relay.rs index 1e1af2105e..dc6ecc866f 100644 --- a/crates/openshell-supervisor-network/src/l7/relay.rs +++ b/crates/openshell-supervisor-network/src/l7/relay.rs @@ -328,6 +328,7 @@ pub(crate) struct UpgradeRelayOptions<'a> { #[derive(Default)] pub(crate) struct WebSocketUpgradeBehavior { pub(crate) credential_rewrite: bool, + pub(crate) deny_uninspected_credentials: bool, pub(crate) message_policy: WebSocketMessagePolicy, pub(crate) permessage_deflate: bool, } @@ -834,6 +835,8 @@ where ), request_body_credential_rewrite: config.protocol == L7Protocol::Rest && config.request_body_credential_rewrite, + deny_uninspected_credentials: config + .deny_uninspected_body_credentials(ctx.secret_resolver.is_some()), credential_signing: config.credential_signing, signing_service: &config.signing_service, signing_region: &config.signing_region, @@ -1067,7 +1070,8 @@ where && (options.websocket.message_policy.inspects_messages() || options.websocket.permessage_deflate || options.websocket.credential_rewrite - || options.middleware_session.is_some()); + || options.middleware_session.is_some() + || options.websocket.deny_uninspected_credentials); let relay_mode = if use_websocket_relay { "websocket parsed relay" } else { @@ -1135,6 +1139,7 @@ where compression, middleware_session: options.middleware_session.take(), middleware_context: options.ctx, + deny_uninspected_credentials: options.websocket.deny_uninspected_credentials, }, ) .await; @@ -1199,6 +1204,8 @@ pub(crate) fn upgrade_options<'a>( let websocket_credential_rewrite = matches!(config.protocol, L7Protocol::Rest | L7Protocol::Websocket) && config.websocket_credential_rewrite; + let deny_uninspected_credentials = + config.provider_credentialed && !config.allow_uninspected_credentials; let websocket_message_policy = if config.protocol == L7Protocol::Websocket { if config.websocket_graphql_policy { WebSocketMessagePolicy::Graphql @@ -1212,6 +1219,7 @@ pub(crate) fn upgrade_options<'a>( websocket_request, websocket: WebSocketUpgradeBehavior { credential_rewrite: websocket_credential_rewrite, + deny_uninspected_credentials, message_policy: websocket_message_policy, permessage_deflate: false, }, @@ -1240,6 +1248,7 @@ pub(crate) fn websocket_extension_mode( if inspecting_middleware_session || config.protocol == L7Protocol::Websocket || (config.protocol == L7Protocol::Rest && config.websocket_credential_rewrite) + || (config.provider_credentialed && !config.allow_uninspected_credentials) { WebSocketExtensionMode::PermessageDeflate } else { @@ -1524,6 +1533,8 @@ where ), request_body_credential_rewrite: config.protocol == L7Protocol::Rest && config.request_body_credential_rewrite, + deny_uninspected_credentials: config + .deny_uninspected_body_credentials(ctx.secret_resolver.is_some()), credential_signing: config.credential_signing, signing_service: &config.signing_service, signing_region: &config.signing_region, @@ -6962,6 +6973,8 @@ network_policies: allow_encoded_slash, websocket_credential_rewrite: false, request_body_credential_rewrite: false, + allow_uninspected_credentials: false, + provider_credentialed: false, websocket_graphql_policy: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: String::new(), @@ -7315,6 +7328,8 @@ network_policies: allow_encoded_slash: false, websocket_credential_rewrite: true, request_body_credential_rewrite: false, + allow_uninspected_credentials: false, + provider_credentialed: false, websocket_graphql_policy: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: String::new(), @@ -7515,6 +7530,8 @@ network_policies: allow_encoded_slash: false, websocket_credential_rewrite: true, request_body_credential_rewrite: false, + allow_uninspected_credentials: false, + provider_credentialed: false, websocket_graphql_policy: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: String::new(), @@ -7639,6 +7656,8 @@ network_policies: allow_encoded_slash: false, websocket_credential_rewrite: true, request_body_credential_rewrite: false, + allow_uninspected_credentials: false, + provider_credentialed: false, websocket_graphql_policy: true, credential_signing: crate::l7::CredentialSigning::None, signing_service: String::new(), diff --git a/crates/openshell-supervisor-network/src/l7/rest.rs b/crates/openshell-supervisor-network/src/l7/rest.rs index f9aaaf18a9..93315a671a 100644 --- a/crates/openshell-supervisor-network/src/l7/rest.rs +++ b/crates/openshell-supervisor-network/src/l7/rest.rs @@ -14,7 +14,8 @@ use base64::Engine as _; use miette::{IntoDiagnostic, Result, miette}; use openshell_core::proto::{ExistingHeaderAction, HeaderMutation, header_mutation}; use openshell_core::secrets::{ - SecretResolver, contains_reserved_credential_marker, rewrite_http_header_block, + CREDENTIAL_MARKER_SCAN_TAIL_BYTES, SecretResolver, contains_reserved_credential_marker, + contains_reserved_credential_marker_bytes, rewrite_http_header_block, }; use openshell_ocsf::ctx::ctx as ocsf_ctx; use sha1::{Digest, Sha1}; @@ -726,6 +727,7 @@ where generation_guard, websocket_extensions: WebSocketExtensionMode::Preserve, request_body_credential_rewrite: false, + deny_uninspected_credentials: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: "", signing_region: "", @@ -750,6 +752,7 @@ pub(crate) struct RelayRequestOptions<'a> { pub(crate) generation_guard: Option<&'a PolicyGenerationGuard>, pub(crate) websocket_extensions: WebSocketExtensionMode, pub(crate) request_body_credential_rewrite: bool, + pub(crate) deny_uninspected_credentials: bool, pub(crate) credential_signing: crate::l7::CredentialSigning, pub(crate) signing_service: &'a str, pub(crate) signing_region: &'a str, @@ -1087,6 +1090,22 @@ where if !body.body.is_empty() { upstream.write_all(&body.body).await.into_diagnostic()?; } + } else if options.deny_uninspected_credentials { + if let Err(error) = relay_request_body_with_marker_guard( + req, + client, + upstream, + &rewrite_result.rewritten, + &req.raw_header[header_end..], + options.generation_guard, + ) + .await + { + if error.to_string().contains("credential placeholder") { + emit_uninspected_body_credential_denial(req, &options); + } + return Err(error); + } } else { ensure_credential_generation_current(options)?; upstream @@ -1139,6 +1158,218 @@ where Ok(outcome) } +#[derive(Default)] +struct ReservedMarkerStreamGuard { + pending: Vec, +} + +impl ReservedMarkerStreamGuard { + fn push(&mut self, bytes: &[u8]) -> Result> { + self.pending.extend_from_slice(bytes); + if contains_reserved_credential_marker_bytes(&self.pending) { + return Err(miette!( + "request body credential placeholder denied because rewrite is disabled" + )); + } + let safe_len = self + .pending + .len() + .saturating_sub(CREDENTIAL_MARKER_SCAN_TAIL_BYTES); + Ok(self.pending.drain(..safe_len).collect()) + } + + fn finish(mut self) -> Result> { + if contains_reserved_credential_marker_bytes(&self.pending) { + return Err(miette!( + "request body credential placeholder denied because rewrite is disabled" + )); + } + Ok(std::mem::take(&mut self.pending)) + } +} + +async fn relay_request_body_with_marker_guard( + req: &L7Request, + client: &mut C, + upstream: &mut U, + headers: &[u8], + already_read: &[u8], + generation_guard: Option<&PolicyGenerationGuard>, +) -> Result<()> +where + C: AsyncRead + Unpin, + U: AsyncWrite + Unpin, +{ + upstream.write_all(headers).await.into_diagnostic()?; + match req.body_length { + BodyLength::None => { + let mut scanner = ReservedMarkerStreamGuard::default(); + let safe = scanner.push(already_read)?; + upstream.write_all(&safe).await.into_diagnostic()?; + upstream + .write_all(&scanner.finish()?) + .await + .into_diagnostic()?; + } + BodyLength::ContentLength(len) => { + let initial_len = usize::try_from(len) + .unwrap_or(usize::MAX) + .min(already_read.len()); + let mut scanner = ReservedMarkerStreamGuard::default(); + let safe = scanner.push(&already_read[..initial_len])?; + upstream.write_all(&safe).await.into_diagnostic()?; + let mut remaining = len.saturating_sub(initial_len as u64); + let mut buf = vec![0u8; RELAY_BUF_SIZE]; + while remaining > 0 { + let to_read = usize::try_from(remaining) + .unwrap_or(buf.len()) + .min(buf.len()); + let n = client.read(&mut buf[..to_read]).await.into_diagnostic()?; + if n == 0 { + return Err(miette!( + "Connection closed with {remaining} body bytes remaining" + )); + } + if let Some(guard) = generation_guard { + guard.ensure_current()?; + } + let safe = scanner.push(&buf[..n])?; + upstream.write_all(&safe).await.into_diagnostic()?; + remaining -= n as u64; + } + upstream + .write_all(&scanner.finish()?) + .await + .into_diagnostic()?; + } + BodyLength::Chunked => { + relay_chunked_with_marker_guard(client, upstream, already_read, generation_guard) + .await?; + } + } + Ok(()) +} + +async fn write_chunk(writer: &mut W, payload: &[u8]) -> Result<()> { + if payload.is_empty() { + return Ok(()); + } + writer + .write_all(format!("{:X}\r\n", payload.len()).as_bytes()) + .await + .into_diagnostic()?; + writer.write_all(payload).await.into_diagnostic()?; + writer.write_all(b"\r\n").await.into_diagnostic()?; + Ok(()) +} + +async fn relay_chunked_with_marker_guard( + client: &mut C, + upstream: &mut U, + already_read: &[u8], + generation_guard: Option<&PolicyGenerationGuard>, +) -> Result<()> +where + C: AsyncRead + Unpin, + U: AsyncWrite + Unpin, +{ + let mut read_state = ChunkedReadState { + buffered_pos: 0, + wire_bytes: 0, + max_wire_bytes: None, + }; + let mut scanner = ReservedMarkerStreamGuard::default(); + + loop { + let size_line = read_chunked_line(client, already_read, &mut read_state, generation_guard) + .await + .map_err(CollectChunkedError::into_report)?; + let size_line = std::str::from_utf8(&size_line) + .map_err(|_| miette!("Invalid UTF-8 in chunk-size line"))?; + let size_token = size_line + .split(';') + .next() + .map(str::trim) + .unwrap_or_default(); + let chunk_size = usize::from_str_radix(size_token, 16) + .map_err(|_| miette!("Invalid chunk size token: {size_token:?}"))?; + + if chunk_size == 0 { + write_chunk(upstream, &scanner.finish()?).await?; + upstream.write_all(b"0\r\n").await.into_diagnostic()?; + loop { + let trailer = + read_chunked_line(client, already_read, &mut read_state, generation_guard) + .await + .map_err(CollectChunkedError::into_report)?; + if contains_reserved_credential_marker_bytes(&trailer) { + return Err(miette!( + "request body credential placeholder denied because rewrite is disabled" + )); + } + upstream.write_all(&trailer).await.into_diagnostic()?; + upstream.write_all(b"\r\n").await.into_diagnostic()?; + if trailer.is_empty() { + return Ok(()); + } + } + } + + let mut remaining = chunk_size; + while remaining > 0 { + let block_len = remaining.min(RELAY_BUF_SIZE); + let mut block = Vec::with_capacity(block_len); + read_buffered_exact( + client, + already_read, + &mut read_state, + block_len, + &mut block, + generation_guard, + ) + .await + .map_err(CollectChunkedError::into_report)?; + write_chunk(upstream, &scanner.push(&block)?).await?; + remaining -= block_len; + } + + let mut terminator = Vec::with_capacity(2); + read_buffered_exact( + client, + already_read, + &mut read_state, + 2, + &mut terminator, + generation_guard, + ) + .await + .map_err(CollectChunkedError::into_report)?; + if terminator.as_slice() != b"\r\n" { + return Err(miette!("Chunk missing terminating CRLF")); + } + } +} + +fn emit_uninspected_body_credential_denial(req: &L7Request, options: &RelayRequestOptions<'_>) { + let event = openshell_ocsf::NetworkActivityBuilder::new(openshell_ocsf::ctx::ctx()) + .activity(openshell_ocsf::ActivityId::Traffic) + .action(openshell_ocsf::ActionId::Denied) + .disposition(openshell_ocsf::DispositionId::Blocked) + .severity(openshell_ocsf::SeverityId::High) + .status(openshell_ocsf::StatusId::Failure) + .dst_endpoint(openshell_ocsf::Endpoint::from_domain( + options.host, + options.port, + )) + .message(format!( + "{} request body credential traffic denied for {}:{}", + req.action, options.host, options.port + )) + .build(); + openshell_ocsf::ocsf_emit!(event); + crate::l7::emit_uninspected_credential_finding(options.host, "", "http-request-body"); +} + struct PreparedRequestBody { headers: Vec, body: Vec, @@ -1840,12 +2071,7 @@ fn is_rewritable_content_type(content_type: Option<&str>) -> bool { } fn body_bytes_contain_reserved_marker(body: &[u8]) -> bool { - if body.is_empty() { - return false; - } - String::from_utf8_lossy(body) - .split('\0') - .any(contains_reserved_credential_marker) + contains_reserved_credential_marker_bytes(body) } fn set_content_length(headers: &[u8], len: usize) -> Result> { @@ -6985,6 +7211,109 @@ mod tests { assert!(!lower.contains("upgrade: h2c")); } + #[test] + fn streamed_body_guard_detects_marker_split_across_reads() { + let mut guard = ReservedMarkerStreamGuard::default(); + assert!(guard.push(b"prefix-openshell:res").is_ok()); + let error = guard + .push(b"olve:env:API_TOKEN-suffix") + .expect_err("split marker must be detected"); + assert!(error.to_string().contains("rewrite is disabled")); + } + + /// Worst case for the retained window: a fully percent-encoded marker, the + /// longest detectable wire form, delivered one byte short and then + /// completed. A window narrower than that form drains its leading bytes + /// before the match completes, and the remainder decodes to a non-marker. + #[test] + fn streamed_body_guard_detects_fully_encoded_marker_completed_by_one_byte() { + const ENCODED: &str = "%6F%70%65%6E%73%68%65%6C%6C%3A%72%65%73%6F%6C%76%65%3A%65%6E%76%3A"; + assert!( + contains_reserved_credential_marker(ENCODED), + "fixture must be a recognized marker form" + ); + assert_eq!( + ENCODED.len(), + 3 * openshell_core::secrets::PLACEHOLDER_PREFIX_PUBLIC.len(), + "fixture must stay the fully encoded form of the current marker" + ); + let (head, tail) = ENCODED.as_bytes().split_at(ENCODED.len() - 1); + + let mut guard = ReservedMarkerStreamGuard::default(); + let forwarded = guard.push(head).expect("partial marker is not a match"); + let error = guard + .push(tail) + .expect_err("encoded marker completed across reads must be detected"); + + assert!(error.to_string().contains("rewrite is disabled")); + assert!( + forwarded.is_empty(), + "no marker byte may be forwarded early" + ); + } + + #[tokio::test] + async fn guarded_content_length_does_not_forward_placeholder() { + let body = b"prefix-openshell:resolve:env:API_TOKEN-suffix"; + let split = 20; + let mut raw_header = format!( + "POST /api HTTP/1.1\r\nHost: api.example.com\r\nContent-Length: {}\r\n\r\n", + body.len() + ) + .into_bytes(); + raw_header.extend_from_slice(&body[..split]); + let req = L7Request { + action: "POST".to_string(), + target: "/api".to_string(), + query_params: HashMap::new(), + raw_header, + body_length: BodyLength::ContentLength(body.len() as u64), + }; + let (mut client_side, mut proxy_client) = tokio::io::duplex(1024); + client_side.write_all(&body[split..]).await.unwrap(); + drop(client_side); + let (mut proxy_upstream, mut upstream_side) = tokio::io::duplex(4096); + + let error = relay_http_request_with_options_guarded( + &req, + &mut proxy_client, + &mut proxy_upstream, + RelayRequestOptions { + deny_uninspected_credentials: true, + host: "api.example.com", + port: 443, + ..Default::default() + }, + ) + .await + .expect_err("placeholder must fail closed"); + assert!(error.to_string().contains("rewrite is disabled")); + + drop(proxy_upstream); + let mut forwarded = Vec::new(); + upstream_side.read_to_end(&mut forwarded).await.unwrap(); + assert!(!contains_reserved_credential_marker_bytes(&forwarded)); + } + + #[tokio::test] + async fn guarded_chunked_body_detects_encoded_placeholder() { + let wire = b"8\r\nprefix-o\r\n1D\r\npenshell%3Aresolve%3Aenv%3AKEY\r\n0\r\n\r\n"; + let (mut upstream_writer, mut upstream_reader) = tokio::io::duplex(4096); + let error = relay_chunked_with_marker_guard( + &mut tokio::io::empty(), + &mut upstream_writer, + wire, + None, + ) + .await + .expect_err("encoded placeholder must fail closed"); + assert!(error.to_string().contains("rewrite is disabled")); + drop(upstream_writer); + let mut forwarded = Vec::new(); + upstream_reader.read_to_end(&mut forwarded).await.unwrap(); + assert!(!String::from_utf8_lossy(&forwarded).contains("env%3AKEY")); + } + #[tokio::test] async fn relay_request_body_rewrites_provider_alias_header_and_urlencoded_token() { let (_, resolver) = SecretResolver::from_provider_env( diff --git a/crates/openshell-supervisor-network/src/l7/websocket.rs b/crates/openshell-supervisor-network/src/l7/websocket.rs index db73104dd3..3ce39744a2 100644 --- a/crates/openshell-supervisor-network/src/l7/websocket.rs +++ b/crates/openshell-supervisor-network/src/l7/websocket.rs @@ -491,6 +491,7 @@ pub(super) struct RelayOptions<'a> { pub(super) compression: WebSocketCompression, pub(super) middleware_session: Option, pub(super) middleware_context: Option<&'a L7EvalContext>, + pub(super) deny_uninspected_credentials: bool, } /// Relay an upgraded WebSocket connection with optional client text inspection, @@ -811,6 +812,18 @@ where } } OPCODE_BINARY => { + if options.deny_uninspected_credentials { + emit_uninspected_credential_denial( + host, + port, + options.policy_name, + "websocket-binary", + ); + return Err(terminate( + WebSocketTerminationCause::PolicyDenial, + miette!("websocket binary frame denied for credentialed endpoint"), + )); + } let initial_size = usize::try_from(frame.payload_len).unwrap_or(usize::MAX); let coverage = options .middleware_session @@ -1172,6 +1185,27 @@ where "websocket text message is not valid UTF-8" ))) })?; + let live_resolver = options.provider_credentials.map(|credentials| { + let (resolver, revision) = + credentials.resolver_for_endpoint_with_revision(host, port, options.target); + ( + resolver, + crate::l7::rest::CredentialGenerationGuard::new(credentials, revision), + ) + }); + let resolver = live_resolver + .as_ref() + .map_or(options.resolver, |(resolver, _)| resolver.as_deref()); + if options.deny_uninspected_credentials + && resolver.is_none() + && contains_reserved_credential_marker(&text) + { + emit_uninspected_credential_denial(host, port, options.policy_name, "websocket-text"); + return Err(terminate( + WebSocketTerminationCause::PolicyDenial, + miette!("websocket credential placeholder denied because rewrite is disabled"), + )); + } // Built-in transport/GraphQL inspection sees the original unresolved // message. External transformations run next, then policy is re-evaluated @@ -1223,17 +1257,6 @@ where } ensure_generation_current(host, port, options)?; - let live_resolver = options.provider_credentials.map(|credentials| { - let (resolver, revision) = - credentials.resolver_for_endpoint_with_revision(host, port, options.target); - ( - resolver, - crate::l7::rest::CredentialGenerationGuard::new(credentials, revision), - ) - }); - let resolver = live_resolver - .as_ref() - .map_or(options.resolver, |(resolver, _)| resolver.as_deref()); let replacements = if let Some(resolver) = resolver { resolver .rewrite_websocket_text_placeholders(&mut text) @@ -1323,6 +1346,23 @@ where .await } +fn emit_uninspected_credential_denial(host: &str, port: u16, policy_name: &str, surface: &str) { + let event = NetworkActivityBuilder::new(openshell_ocsf::ctx::ctx()) + .activity(ActivityId::Traffic) + .action(ActionId::Denied) + .disposition(DispositionId::Blocked) + .severity(SeverityId::High) + .status(StatusId::Failure) + .dst_endpoint(Endpoint::from_domain(host, port)) + .firewall_rule(policy_name, "l7-websocket") + .message(format!( + "WebSocket credential traffic denied for {host}:{port}" + )) + .build(); + ocsf_emit!(event); + crate::l7::emit_uninspected_credential_finding(host, policy_name, surface); +} + fn inspect_websocket_text_message( host: &str, port: u16, @@ -2362,6 +2402,7 @@ network_policies: compression: WebSocketCompression::None, middleware_session: None, middleware_context: None, + deny_uninspected_credentials: false, } } @@ -2391,6 +2432,7 @@ network_policies: compression: WebSocketCompression::None, middleware_session: None, middleware_context: None, + deny_uninspected_credentials: false, }; let result = relay_client_to_server( &mut relay_read, @@ -2669,6 +2711,7 @@ network_policies: compression: WebSocketCompression::None, middleware_session: None, middleware_context: None, + deny_uninspected_credentials: false, }; let result = relay_client_to_server( &mut relay_read, @@ -2729,6 +2772,48 @@ network_policies: ); } + #[tokio::test] + async fn guarded_websocket_uses_path_scoped_live_resolver() { + let state = bound_websocket_provider_state(); + let placeholder = b"openshell:resolve:env:v1_DISCORD_BOT_TOKEN"; + let input = masked_frame(true, OPCODE_TEXT, placeholder); + let (mut client_write, mut relay_read) = tokio::io::duplex(4096); + let (mut relay_write, mut upstream_read) = tokio::io::duplex(4096); + client_write.write_all(&input).await.unwrap(); + drop(client_write); + + let mut options = RelayOptions { + policy_name: "test-policy", + assembly_budget: WebSocketAssemblyBudget::default(), + resolver: None, + generation_guard: None, + provider_credentials: Some(&state), + target: "/socket", + inspector: None, + compression: WebSocketCompression::None, + middleware_session: None, + middleware_context: None, + deny_uninspected_credentials: true, + }; + let result = relay_client_to_server( + &mut relay_read, + &mut relay_write, + "gateway.example.test", + 443, + &mut options, + ) + .await; + assert!( + result.is_ok(), + "path-scoped live resolver must satisfy the credential guard: {result:?}" + ); + + drop(relay_write); + let mut output = Vec::new(); + upstream_read.read_to_end(&mut output).await.unwrap(); + assert_eq!(decode_masked_text_frame(&output), "real-token"); + } + #[tokio::test] async fn websocket_rewrite_rejects_revocation_before_frame_write() { let state = bound_websocket_provider_state(); @@ -2755,6 +2840,7 @@ network_policies: compression: WebSocketCompression::PermessageDeflate, middleware_session: None, middleware_context: None, + deny_uninspected_credentials: false, }; let reached_write = tokio::sync::Barrier::new(2); let release_write = tokio::sync::Barrier::new(2); @@ -2798,6 +2884,44 @@ network_policies: ); } + async fn run_client_to_server_guarded(input: Vec) -> (Result<()>, Vec) { + let (mut client_write, mut relay_read) = tokio::io::duplex(MAX_TEXT_MESSAGE_BYTES + 1024); + let (mut relay_write, mut upstream_read) = tokio::io::duplex(MAX_TEXT_MESSAGE_BYTES + 1024); + + client_write.write_all(&input).await.unwrap(); + drop(client_write); + + let mut options = RelayOptions { + policy_name: "test-policy", + assembly_budget: WebSocketAssemblyBudget::default(), + resolver: None, + generation_guard: None, + provider_credentials: None, + target: "/", + inspector: None, + compression: WebSocketCompression::None, + middleware_session: None, + middleware_context: None, + deny_uninspected_credentials: true, + }; + let result = relay_client_to_server( + &mut relay_read, + &mut relay_write, + "gateway.example.test", + 443, + &mut options, + ) + .await; + drop(relay_write); + + let mut output = Vec::new(); + upstream_read.read_to_end(&mut output).await.unwrap(); + ( + result.map(|_| ()).map_err(|termination| termination.error), + output, + ) + } + async fn run_client_to_server_with_graphql_policy( input: Vec, resolver: Option<&SecretResolver>, @@ -2853,6 +2977,7 @@ network_policies: compression: WebSocketCompression::None, middleware_session: None, middleware_context: None, + deny_uninspected_credentials: false, }; let result = relay_client_to_server( &mut relay_read, @@ -2890,6 +3015,7 @@ network_policies: compression: WebSocketCompression::PermessageDeflate, middleware_session: None, middleware_context: None, + deny_uninspected_credentials: false, }; let result = relay_client_to_server( &mut relay_read, @@ -3331,6 +3457,7 @@ network_policies: compression: WebSocketCompression::None, middleware_session: None, middleware_context: None, + deny_uninspected_credentials: false, }, ) .await @@ -3646,6 +3773,7 @@ network_policies: compression: WebSocketCompression::None, middleware_session: Some(session), middleware_context: None, + deny_uninspected_credentials: false, }, ) .await @@ -3773,6 +3901,7 @@ network_policies: compression: WebSocketCompression::None, middleware_session: Some(session), middleware_context: None, + deny_uninspected_credentials: false, }, ) .await @@ -4381,6 +4510,7 @@ network_policies: compression: WebSocketCompression::None, middleware_session: Some(session), middleware_context: Some(&ctx), + deny_uninspected_credentials: false, }, ) .await @@ -4517,6 +4647,7 @@ network_policies: compression: WebSocketCompression::None, middleware_session: Some(session), middleware_context: None, + deny_uninspected_credentials: false, }, ) .await @@ -4688,6 +4819,7 @@ network_policies: compression: WebSocketCompression::None, middleware_session: Some(session), middleware_context: None, + deny_uninspected_credentials: false, }, ) .await @@ -4782,6 +4914,7 @@ network_policies: compression: WebSocketCompression::None, middleware_session: Some(session), middleware_context: None, + deny_uninspected_credentials: false, }, ) .await @@ -5049,6 +5182,40 @@ network_policies: assert_eq!(output, frame); } + #[tokio::test] + async fn credentialed_endpoint_denies_binary_frame_without_opt_in() { + let frame = masked_frame(true, OPCODE_BINARY, &[0, 1, 2, 3, 255]); + + let (result, output) = run_client_to_server_guarded(frame).await; + + assert!( + result + .unwrap_err() + .to_string() + .contains("binary frame denied") + ); + assert!(output.is_empty()); + } + + #[tokio::test] + async fn credentialed_endpoint_denies_text_placeholder_without_rewrite() { + let frame = masked_frame( + true, + OPCODE_TEXT, + br#"{"token":"openshell:resolve:env:API_TOKEN"}"#, + ); + + let (result, output) = run_client_to_server_guarded(frame).await; + + assert!( + result + .unwrap_err() + .to_string() + .contains("rewrite is disabled") + ); + assert!(output.is_empty()); + } + #[tokio::test] async fn rejects_reserved_opcode() { let err = run_client_to_server(masked_frame(true, 0x3, b"reserved")) diff --git a/crates/openshell-supervisor-network/src/opa.rs b/crates/openshell-supervisor-network/src/opa.rs index 6dd92b40db..33c14e9bd8 100644 --- a/crates/openshell-supervisor-network/src/opa.rs +++ b/crates/openshell-supervisor-network/src/opa.rs @@ -846,6 +846,38 @@ impl OpaEngine { } } + /// Query every matching endpoint for credential-provenance gating. + /// + /// Unlike [`Self::query_endpoint_configs_with_generation`], this includes + /// L4-only endpoints, which carry no extended L7 config. It is answered by + /// a dedicated Rego rule so credential gating cannot alter which endpoint + /// the TLS mode and SSRF allowlist are read from. + pub fn query_endpoint_credential_guards( + &self, + input: &NetworkInput, + ) -> Result> { + let input_json = network_input_json(input); + + let mut engine = self + .engine + .lock() + .map_err(|_| miette::miette!("OPA engine lock poisoned"))?; + + engine + .set_input_json(&input_json.to_string()) + .map_err(|e| miette::miette!("{e}"))?; + + let val = engine + .eval_rule("data.openshell.sandbox.endpoint_credential_guards".into()) + .map_err(|e| miette::miette!("{e}"))?; + + match val { + regorus::Value::Undefined => Ok(Vec::new()), + regorus::Value::Array(values) => Ok(values.to_vec()), + other => Ok(vec![other]), + } + } + /// Query the ordered middleware chain for an admitted destination. pub fn query_middleware_chain_with_generation( &self, @@ -1780,6 +1812,12 @@ fn proto_to_opa_data_json(proto: &ProtoSandboxPolicy, entrypoint_pid: u32) -> St if e.request_body_credential_rewrite { ep["request_body_credential_rewrite"] = true.into(); } + if e.allow_uninspected_credentials { + ep["allow_uninspected_credentials"] = true.into(); + } + if e.provider_credentialed { + ep["provider_credentialed"] = true.into(); + } if !e.credential_signing.is_empty() { ep["credential_signing"] = e.credential_signing.clone().into(); } @@ -5663,6 +5701,87 @@ process: .expect("Failed to load allowed_ips test data") } + /// An L4-only credentialed endpoint must not join the shared endpoint-config + /// list: `query_endpoint_config` returns only its first element, so joining + /// it would let the L4 endpoint shadow the inspected endpoint's TLS mode and + /// SSRF allowlist on the same host:port. + #[test] + fn credential_guard_does_not_shadow_inspected_endpoint_config() { + const OVERLAPPING_DATA: &str = r#" +network_policies: + telemetry_l4: + name: telemetry_l4 + endpoints: + - host: api.example.com + port: 443 + provider_credentialed: true + allow_uninspected_credentials: true + binaries: + - { path: /usr/bin/curl } + inspected_api: + name: inspected_api + endpoints: + - host: api.example.com + port: 443 + protocol: rest + access: full + tls: skip + allowed_ips: ["10.0.5.0/24"] + binaries: + - { path: /usr/bin/curl } +filesystem_policy: + include_workdir: true + read_only: [] + read_write: [] +landlock: + compatibility: best_effort +process: + run_as_user: sandbox + run_as_group: sandbox +"#; + let engine = OpaEngine::from_strings(TEST_POLICY, OVERLAPPING_DATA) + .expect("overlapping endpoint policy should load"); + let input = NetworkInput { + host: "api.example.com".into(), + port: 443, + binary_path: PathBuf::from("/usr/bin/curl"), + binary_sha256: "unused".into(), + ancestors: vec![], + cmdline_paths: vec![], + }; + + let configs = engine + .query_endpoint_configs_with_generation(&input) + .unwrap() + .0; + assert_eq!( + configs.len(), + 1, + "only the inspected endpoint carries extended config" + ); + let config = crate::l7::parse_l7_config(&configs[0]).expect("inspected endpoint config"); + assert_eq!(config.protocol, crate::l7::L7Protocol::Rest); + assert_eq!(config.tls, crate::l7::TlsMode::Skip); + assert_eq!( + engine.query_allowed_ips(&input).unwrap(), + vec!["10.0.5.0/24"], + "SSRF allowlist must still come from the inspected endpoint" + ); + + let guards = engine.query_endpoint_credential_guards(&input).unwrap(); + assert_eq!( + guards.len(), + 2, + "credential gating must still see the L4-only endpoint" + ); + assert!( + guards + .iter() + .map(crate::l7::parse_endpoint_credential_guard) + .any(|guard| guard.provider_credentialed) + ); + } + #[test] fn allowed_ips_mode2_host_plus_ips_allows() { let engine = allowed_ips_engine(); diff --git a/crates/openshell-supervisor-network/src/policy_local.rs b/crates/openshell-supervisor-network/src/policy_local.rs index 13744a63f9..8951d6936e 100644 --- a/crates/openshell-supervisor-network/src/policy_local.rs +++ b/crates/openshell-supervisor-network/src/policy_local.rs @@ -1167,6 +1167,8 @@ fn network_endpoint_from_json( allow_encoded_slash: endpoint.allow_encoded_slash, websocket_credential_rewrite: false, request_body_credential_rewrite: false, + allow_uninspected_credentials: false, + provider_credentialed: false, advisor_proposed: false, // GraphQL persisted-query knobs and path scoping default empty — // agent proposals don't author them today. diff --git a/crates/openshell-supervisor-network/src/proxy.rs b/crates/openshell-supervisor-network/src/proxy.rs index 1671d8f532..daaccd0d6e 100644 --- a/crates/openshell-supervisor-network/src/proxy.rs +++ b/crates/openshell-supervisor-network/src/proxy.rs @@ -1408,6 +1408,7 @@ async fn handle_tcp_connection( // connect and before `200 Connection Established`. hydrate_tls_mode(&opa_engine, &mut decision); let effective_tls_skip = decision.endpoint.tls_mode == crate::l7::TlsMode::Skip; + let credential_guard = query_endpoint_credential_guard(&opa_engine, &decision, &host_lc, port)?; let sandbox_entrypoint_pid = entrypoint_pid.load(Ordering::Acquire); @@ -1511,6 +1512,51 @@ async fn handle_tcp_connection( return Ok(()); } + if credential_guard.blocks_connect() { + const DETAIL: &str = + "credentialed endpoint requires L7 inspection; raw tunnel is not explicitly allowed"; + let event = NetworkActivityBuilder::new(openshell_ocsf::ctx::ctx()) + .activity(ActivityId::Open) + .action(ActionId::Denied) + .disposition(DispositionId::Blocked) + .severity(SeverityId::High) + .status(StatusId::Failure) + .dst_endpoint(Endpoint::from_domain(&host_lc, port)) + .src_endpoint_addr(workload_addr.ip(), workload_addr.port()) + .actor_process( + Process::from_bypass(&binary_str, &pid_str, &ancestors_str) + .with_cmd_line(&cmdline_str), + ) + .firewall_rule(policy_str, "credentials") + .message(format!( + "CONNECT refused for {host_lc}:{port}: uninspected credential traffic" + )) + .status_detail(DETAIL) + .build(); + ocsf_emit!(event); + crate::l7::emit_uninspected_credential_finding( + &host_lc, + policy_str, + if effective_tls_skip { "tls-skip" } else { "l4" }, + ); + emit_activity_simple(activity_tx.as_ref(), true, "uninspected_credentials"); + emit_denial( + &denial_tx, + &host_lc, + port, + &binary_str, + &decision, + DETAIL, + "connect-uninspected-credentials", + ); + respond( + &mut client, + &build_json_error_response(403, "Forbidden", "uninspected_credentials", DETAIL), + ) + .await?; + return Ok(()); + } + // CONNECT must use one policy generation from authorization through route // hydration and relay startup. A later L7 lookup must never make a stale // L4 allow appear current. @@ -3008,6 +3054,55 @@ fn query_tls_mode( } } +fn query_endpoint_credential_guard( + engine: &OpaEngine, + decision: &EgressDecision, + host: &str, + port: u16, +) -> Result { + let has_policy = match &decision.action { + NetworkAction::Allow { matched_policy } => matched_policy.is_some(), + NetworkAction::Deny { .. } => false, + }; + if !has_policy { + return Ok(crate::l7::EndpointCredentialGuard::default()); + } + + let input = crate::opa::NetworkInput { + host: host.to_string(), + port, + binary_path: decision.binary.clone().unwrap_or_default(), + binary_sha256: String::new(), + ancestors: decision.ancestors.clone(), + cmdline_paths: decision.cmdline_paths.clone(), + }; + let values = engine.query_endpoint_credential_guards(&input)?; + let credentialed: Vec<_> = values + .iter() + .map(crate::l7::parse_endpoint_credential_guard) + .filter(|guard| guard.provider_credentialed) + .collect(); + if credentialed.is_empty() { + return Ok(crate::l7::EndpointCredentialGuard::default()); + } + + Ok(crate::l7::EndpointCredentialGuard { + provider_credentialed: true, + allow_uninspected_credentials: credentialed + .iter() + .all(|guard| guard.allow_uninspected_credentials), + has_l7_protocol: credentialed.iter().all(|guard| guard.has_l7_protocol), + tls: if credentialed + .iter() + .any(|guard| guard.tls == crate::l7::TlsMode::Skip) + { + crate::l7::TlsMode::Skip + } else { + crate::l7::TlsMode::Auto + }, + }) +} + /// When the policy endpoint host is a literal IP address, the user has /// explicitly declared intent to allow that destination. Synthesize an /// `allowed_ips` entry so the existing allowlist-validation path is used @@ -4145,6 +4240,7 @@ struct ForwardRelayOptions<'a> { websocket_extensions: crate::l7::rest::WebSocketExtensionMode, secret_resolver: Option<&'a SecretResolver>, request_body_credential_rewrite: bool, + deny_uninspected_credentials: bool, credential_signing: crate::l7::CredentialSigning, signing_service: &'a str, signing_region: &'a str, @@ -4189,6 +4285,7 @@ where generation_guard: Some(options.generation_guard), websocket_extensions: options.websocket_extensions, request_body_credential_rewrite: options.request_body_credential_rewrite, + deny_uninspected_credentials: options.deny_uninspected_credentials, credential_signing: options.credential_signing, signing_service: options.signing_service, signing_region: options.signing_region, @@ -4484,6 +4581,7 @@ async fn handle_forward_proxy( let mut forward_websocket_request = crate::l7::rest::request_is_websocket_upgrade(&forward_request_bytes); let mut request_body_credential_rewrite = false; + let mut deny_uninspected_credentials = false; let mut l7_activity_pending = false; // 4b. If the endpoint has L7 config, evaluate the request against @@ -4540,7 +4638,7 @@ async fn handle_forward_proxy( let mut l7_ctx = relay::http_context( &decision, provider_credentials, - secret_resolver, + secret_resolver.clone(), activity_tx.cloned(), dynamic_credentials.clone(), agent_proposals, @@ -4733,6 +4831,9 @@ async fn handle_forward_proxy( websocket_extensions = crate::l7::relay::websocket_extension_mode(&l7_config.config, false); request_body_credential_rewrite = l7_config.config.protocol == crate::l7::L7Protocol::Rest && l7_config.config.request_body_credential_rewrite; + deny_uninspected_credentials = l7_config + .config + .deny_uninspected_body_credentials(secret_resolver.is_some()); forward_upgrade_config = Some(l7_config.config.clone()); forward_upgrade_target = path.clone(); forward_upgrade_query_params = query_params.clone(); @@ -5435,6 +5536,7 @@ async fn handle_forward_proxy( websocket_extensions, secret_resolver: secret_resolver.as_deref(), request_body_credential_rewrite, + deny_uninspected_credentials, credential_signing, signing_service, signing_region, @@ -6539,6 +6641,8 @@ network_policies: allow_encoded_slash: false, websocket_credential_rewrite, request_body_credential_rewrite: false, + allow_uninspected_credentials: false, + provider_credentialed: false, websocket_graphql_policy: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: String::new(), @@ -7178,6 +7282,7 @@ network_policies: websocket_extensions: crate::l7::rest::WebSocketExtensionMode::Preserve, secret_resolver: resolver, request_body_credential_rewrite, + deny_uninspected_credentials: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: "", signing_region: "", @@ -7400,6 +7505,7 @@ network_policies: websocket_extensions, secret_resolver: None, request_body_credential_rewrite: false, + deny_uninspected_credentials: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: "", signing_region: "", @@ -7709,6 +7815,8 @@ network_policies: allow_encoded_slash: false, websocket_credential_rewrite: false, request_body_credential_rewrite: false, + allow_uninspected_credentials: false, + provider_credentialed: false, websocket_graphql_policy: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: String::new(), @@ -7727,6 +7835,8 @@ network_policies: allow_encoded_slash: false, websocket_credential_rewrite: false, request_body_credential_rewrite: false, + allow_uninspected_credentials: false, + provider_credentialed: false, websocket_graphql_policy: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: String::new(), @@ -10489,6 +10599,7 @@ network_policies: websocket_extensions: crate::l7::rest::WebSocketExtensionMode::Preserve, secret_resolver: Some(&resolver), request_body_credential_rewrite: true, + deny_uninspected_credentials: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: "", signing_region: "", @@ -10569,6 +10680,7 @@ network_policies: websocket_extensions: crate::l7::rest::WebSocketExtensionMode::Preserve, secret_resolver: Some(&resolver), request_body_credential_rewrite: false, + deny_uninspected_credentials: false, credential_signing: crate::l7::CredentialSigning::SigV4NoBody, signing_service: "execute-api", signing_region: "us-west-2", @@ -10657,6 +10769,7 @@ network_policies: websocket_extensions: crate::l7::rest::WebSocketExtensionMode::Preserve, secret_resolver: None, request_body_credential_rewrite: false, + deny_uninspected_credentials: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: "", signing_region: "", @@ -10706,6 +10819,7 @@ network_policies: websocket_extensions: crate::l7::rest::WebSocketExtensionMode::Preserve, secret_resolver: None, request_body_credential_rewrite: false, + deny_uninspected_credentials: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: "", signing_region: "", diff --git a/crates/openshell-supervisor-network/src/proxy/relay.rs b/crates/openshell-supervisor-network/src/proxy/relay.rs index 314ab53312..1ec122d11f 100644 --- a/crates/openshell-supervisor-network/src/proxy/relay.rs +++ b/crates/openshell-supervisor-network/src/proxy/relay.rs @@ -410,6 +410,8 @@ mod tests { allow_encoded_slash: false, websocket_credential_rewrite: false, request_body_credential_rewrite: false, + allow_uninspected_credentials: false, + provider_credentialed: false, websocket_graphql_policy: false, credential_signing: crate::l7::CredentialSigning::None, signing_service: String::new(), diff --git a/crates/openshell-supervisor-process/src/sandbox/linux/landlock.rs b/crates/openshell-supervisor-process/src/sandbox/linux/landlock.rs index bf42faede8..0a5bbfdc79 100644 --- a/crates/openshell-supervisor-process/src/sandbox/linux/landlock.rs +++ b/crates/openshell-supervisor-process/src/sandbox/linux/landlock.rs @@ -144,6 +144,14 @@ fn prepare_with_path_open_mode( } if read_only.is_empty() && read_write.is_empty() { + if matches!( + policy.landlock.compatibility, + LandlockCompatibility::HardRequirement + ) { + miette::bail!( + "landlock.compatibility is hard_requirement but no filesystem paths are configured" + ); + } return Ok(None); } @@ -522,6 +530,68 @@ mod tests { panic!("hard_requirement should accept mixed directory and device paths: {err}"); } } + #[test] + fn prepare_hard_requirement_no_paths_aborts() { + // Zero configured paths under hard_requirement must fail startup rather + // than silently running without filesystem restrictions. + let policy = hard_requirement_policy(vec![], vec![]); + let Err(err) = prepare(&policy, None) else { + panic!("should abort with no paths"); + }; + let msg = err.to_string(); + assert!( + msg.contains("hard_requirement") && msg.contains("no filesystem paths"), + "error should explain the empty hard_requirement policy: {msg}" + ); + } + + #[test] + fn prepare_best_effort_no_paths_is_noop() { + let policy = SandboxPolicy { + version: 1, + filesystem: FilesystemPolicy { + read_only: vec![], + read_write: vec![], + include_workdir: false, + }, + network: NetworkPolicy::default(), + landlock: LandlockPolicy { + compatibility: LandlockCompatibility::BestEffort, + }, + process: ProcessPolicy::default(), + }; + let prepared = prepare(&policy, None).expect("best_effort no-op should succeed"); + assert!(prepared.is_none(), "no paths should produce no ruleset"); + } + + #[test] + fn prepare_include_workdir_counts_as_configured_path() { + // With no explicit paths but include_workdir set, the workdir must be + // treated as a configured path — so the zero-path abort must NOT fire. + let policy = SandboxPolicy { + version: 1, + filesystem: FilesystemPolicy { + read_only: vec![], + read_write: vec![], + include_workdir: true, + }, + network: NetworkPolicy::default(), + landlock: LandlockPolicy { + compatibility: LandlockCompatibility::HardRequirement, + }, + process: ProcessPolicy::default(), + }; + // Any error (e.g. Landlock unavailable on this host) is acceptable, but + // it must not be the "no filesystem paths" abort. + if let Err(err) = prepare(&policy, Some("/tmp")) { + let msg = err.to_string(); + assert!( + !msg.contains("no filesystem paths"), + "workdir should count as a configured path: {msg}" + ); + } + } + fn tailored_access(path: &Path, requested_access: BitFlags) -> BitFlags { let path_fd = PathFd::new(path).unwrap(); access_for_path_fd(&path_fd, requested_access, ABI::V2).unwrap() diff --git a/crates/openshell-supervisor-process/src/sandbox/linux/mod.rs b/crates/openshell-supervisor-process/src/sandbox/linux/mod.rs index 107a50e370..8afb3caef2 100644 --- a/crates/openshell-supervisor-process/src/sandbox/linux/mod.rs +++ b/crates/openshell-supervisor-process/src/sandbox/linux/mod.rs @@ -99,14 +99,60 @@ pub fn log_sandbox_readiness(policy: &SandboxPolicy, workdir: Option<&str>) { let total_paths = read_only.len() + read_write.len(); if total_paths == 0 { - openshell_ocsf::ocsf_emit!( - openshell_ocsf::ConfigStateChangeBuilder::new(openshell_ocsf::ctx::ctx()) - .severity(openshell_ocsf::SeverityId::Informational) - .status(openshell_ocsf::StatusId::Success) - .state(openshell_ocsf::StateId::Other, "skipped") - .message("Landlock filesystem sandbox skipped: no paths configured".to_string()) - .build() - ); + if matches!( + policy.landlock.compatibility, + openshell_core::policy::LandlockCompatibility::HardRequirement + ) { + // hard_requirement with no paths is fatal (see `landlock::prepare`). + // Emit a failure state so operators don't see a misleading "skipped" + // success event immediately before startup aborts. + openshell_ocsf::ocsf_emit!( + openshell_ocsf::ConfigStateChangeBuilder::new(openshell_ocsf::ctx::ctx()) + .severity(openshell_ocsf::SeverityId::High) + .status(openshell_ocsf::StatusId::Failure) + .state(openshell_ocsf::StateId::Other, "invalid") + .message( + "Landlock hard_requirement but no filesystem paths configured; \ + sandbox startup will abort" + .to_string(), + ) + .build() + ); + // Dual-emit a security finding for the unsafe policy (per OCSF + // guidance: pair the domain event with a DetectionFinding). + openshell_ocsf::ocsf_emit!( + openshell_ocsf::DetectionFindingBuilder::new(openshell_ocsf::ctx::ctx()) + .activity(openshell_ocsf::ActivityId::Open) + .severity(openshell_ocsf::SeverityId::High) + .confidence(openshell_ocsf::ConfidenceId::High) + .is_alert(true) + .finding_info( + openshell_ocsf::FindingInfo::new( + "landlock-hard-requirement-no-paths", + "Landlock Hard Requirement Without Paths", + ) + .with_desc( + "landlock.compatibility is hard_requirement but no filesystem \ + paths are configured; sandbox startup will abort.", + ), + ) + .message( + "Landlock hard_requirement with no filesystem paths configured".to_string(), + ) + .build() + ); + } else { + openshell_ocsf::ocsf_emit!( + openshell_ocsf::ConfigStateChangeBuilder::new(openshell_ocsf::ctx::ctx()) + .severity(openshell_ocsf::SeverityId::Informational) + .status(openshell_ocsf::StatusId::Success) + .state(openshell_ocsf::StateId::Other, "skipped") + .message( + "Landlock filesystem sandbox skipped: no paths configured".to_string(), + ) + .build() + ); + } return; } diff --git a/docs/kubernetes/ingress.mdx b/docs/kubernetes/ingress.mdx index 66178bc3a0..51284465d9 100644 --- a/docs/kubernetes/ingress.mdx +++ b/docs/kubernetes/ingress.mdx @@ -91,7 +91,7 @@ Envoy Gateway only terminates TLS here — it does not perform OIDC. Do not enab Because Envoy terminates TLS, the OpenShell gateway never sees a client certificate, so client mTLS cannot provide identity on this path. Use OIDC bearer tokens for client identity instead. -For headless agents and CI, the CLI obtains the token via the OAuth2 client-credentials grant (no browser): set `OPENSHELL_OIDC_CLIENT_SECRET` to the OAuth client secret before running `openshell gateway add`. The client id comes from `--oidc-client-id` (default `openshell-cli`); pass it explicitly if your IdP client uses a different id. Interactive human users get the browser-based Authorization Code + PKCE flow by default. +For an interactive login on a headless machine, set `OPENSHELL_NO_BROWSER=1` before running `openshell gateway add`. The CLI uses the Device Authorization Grant with S256 PKCE and prompts the user to approve the login from another browser. For unattended agents and CI, set `OPENSHELL_OIDC_CLIENT_SECRET` to use the OAuth2 client-credentials grant instead. The client id comes from `--oidc-client-id` (default `openshell-cli`); pass it explicitly if your identity provider uses a different id. Interactive users with a local browser get the Authorization Code flow with PKCE by default. ### Provide a TLS certificate diff --git a/docs/reference/gateway-auth.mdx b/docs/reference/gateway-auth.mdx index 1d318f109f..e5dcb65adf 100644 --- a/docs/reference/gateway-auth.mdx +++ b/docs/reference/gateway-auth.mdx @@ -120,10 +120,12 @@ openshell gateway add https://gateway.example.com \ --oidc-audience openshell-cli ``` -When you register or log in to an OIDC gateway, the CLI uses the Authorization Code flow with PKCE. It opens a browser, receives the authorization code on a localhost callback, exchanges the code for tokens, and stores the token bundle under the gateway credential directory. If `OPENSHELL_OIDC_CLIENT_SECRET` is set, the CLI uses the client credentials flow instead. Use that mode for CI and other non-interactive automation. +When you register or log in to an OIDC gateway, the CLI uses the Authorization Code flow with PKCE. It opens a browser, receives the authorization code on a localhost callback, exchanges the code for tokens, and stores the token bundle under the gateway credential directory. After `openshell gateway logout`, the next browser login asks the identity provider for a fresh login prompt so you can choose a different browser user instead of silently reusing the previous session. If `OPENSHELL_OIDC_CLIENT_SECRET` is set, the CLI uses the client credentials flow instead. Use that mode for CI and other non-interactive automation. The connection flow: +For a headless environment, set `OPENSHELL_NO_BROWSER=1` before registering or logging in to the gateway. When this variable is set and `OPENSHELL_OIDC_CLIENT_SECRET` is not configured, the CLI uses the Device Authorization Grant (RFC 8628) with S256 PKCE. This flow prompts the user to visit a verification URL on any device with a browser and enter a displayed code. The CLI polls the token endpoint until the user completes authorization. This requires the OIDC client to have the device authorization grant enabled on the identity provider. + 1. The CLI loads the stored OIDC token bundle. 2. If the access token is expired and a refresh token is available, the CLI refreshes it with the OIDC scopes saved in the gateway metadata. 3. The CLI connects to the gateway and attaches `authorization: Bearer ` metadata to each gRPC request. diff --git a/docs/reference/policy-schema.mdx b/docs/reference/policy-schema.mdx index e6dcf12a01..f1aad9d517 100644 --- a/docs/reference/policy-schema.mdx +++ b/docs/reference/policy-schema.mdx @@ -92,14 +92,16 @@ Configures [Landlock LSM](https://docs.kernel.org/security/landlock.html) enforc **Compatibility modes:** -| Value | Kernel ABI unavailable | Individual path inaccessible | All paths inaccessible | -|---|---|---|---| -| `best_effort` | Warns and continues without Landlock. | Skips the path, applies remaining rules. | Warns and continues without Landlock (refuses to apply an empty ruleset). | -| `hard_requirement` | Aborts sandbox startup. | Aborts sandbox startup. | Aborts sandbox startup. | +| Value | No paths configured | Kernel ABI unavailable | Individual path inaccessible | All paths inaccessible | +|---|---|---|---|---| +| `best_effort` | Landlock skipped (no-op). | Warns and continues without Landlock. | Skips the path, applies remaining rules. | Warns and continues without Landlock (refuses to apply an empty ruleset). | +| `hard_requirement` | Aborts sandbox startup. | Aborts sandbox startup. | Aborts sandbox startup, except in Kubernetes sidecar (current-user) mode where the inaccessible path is skipped. | Aborts sandbox startup. | `best_effort` (the default) is appropriate for most deployments. It handles missing paths gracefully. For example, `/app` might not exist in every container image but is included in the baseline path set for containers that do have it. Individual missing paths are skipped while the remaining filesystem rules are still enforced. -`hard_requirement` is for environments where any gap in filesystem isolation is unacceptable. If a listed path cannot be opened for any reason (missing, permission denied, symlink loop), sandbox startup fails immediately rather than running with reduced protection. +`hard_requirement` is for environments where any gap in filesystem isolation is unacceptable. If a listed path cannot be opened for any reason (missing, permission denied, symlink loop), sandbox startup fails immediately rather than running with reduced protection. Configuring `hard_requirement` with no filesystem paths is also a startup error. + +In Kubernetes sidecar (current-user) mode the sandbox cannot distinguish an intentionally denied path from a misconfigured one, so an individual inaccessible path is skipped and the remaining rules are applied instead of aborting. The other `hard_requirement` failures (kernel ABI unavailable, no paths configured, all paths inaccessible) still abort startup. When a path is skipped under `best_effort`, the sandbox logs a warning that includes the path, the specific error, and a human-readable reason (for example, "path does not exist" or "permission denied"). @@ -163,16 +165,17 @@ Each endpoint defines a reachable destination and optional inspection rules. | `host` | string | Yes | Hostname or IP address. Supports a `*` wildcard inside the first DNS label only: `*.example.com`, `**.example.com`, and intra-label patterns like `*-aiplatform.googleapis.com` are accepted; bare `*`/`**`, TLD wildcards (`*.com`), and wildcards outside the first label are rejected at load time. | | `port` | integer | Yes | TCP port number. | | `path` | string | No | Optional HTTP path glob used to select between L7 endpoints that share the same host and port. Empty means all paths. Use this when REST and GraphQL live under the same host, such as `/repos/**` and `/graphql`. | -| `protocol` | string | No | Set to `rest` for HTTP method/path inspection, `websocket` for RFC 6455 upgrade and client text-message inspection, `graphql` for GraphQL-over-HTTP operation inspection, `mcp` for MCP Streamable HTTP request inspection, or `json-rpc` for generic JSON-RPC-over-HTTP method inspection. WebSocket endpoints can also use GraphQL operation rules for GraphQL-over-WebSocket traffic. Omit for TCP passthrough. | -| `tls` | string | No | TLS handling mode. The proxy auto-detects TLS by peeking the first bytes of each connection and terminates it for inspected HTTPS traffic, so this field is optional in most cases. Set to `skip` to disable auto-detection for edge cases such as client-certificate mTLS or non-standard protocols. The values `terminate` and `passthrough` are deprecated and log a warning; they are still accepted for backward compatibility but have no effect on behavior. | +| `protocol` | string | No | Set to `rest` for HTTP method/path inspection, `websocket` for RFC 6455 upgrade and client text-message inspection, `graphql` for GraphQL-over-HTTP operation inspection, `mcp` for MCP Streamable HTTP request inspection, or `json-rpc` for generic JSON-RPC-over-HTTP method inspection. WebSocket endpoints can also use GraphQL operation rules for GraphQL-over-WebSocket traffic. Omit for TCP passthrough. Provider-credentialed endpoints require an inspected protocol unless `allow_uninspected_credentials` is explicitly set. | +| `tls` | string | No | TLS handling mode. The proxy auto-detects TLS by peeking the first bytes of each connection and terminates it for inspected HTTPS traffic, so this field is optional in most cases. Set to `skip` to disable auto-detection for edge cases such as client-certificate mTLS or non-standard protocols. Provider-credentialed endpoints reject `tls: skip` unless `allow_uninspected_credentials` is explicitly set. The values `terminate` and `passthrough` are deprecated and log a warning; they are still accepted for backward compatibility but have no effect on behavior. | | `enforcement` | string | No | `enforce` actively blocks disallowed requests. `audit` logs violations but allows traffic through. | | `access` | string | No | Access preset. One of `read-only`, `read-write`, or `full`. Mutually exclusive with `rules`. Not valid on `protocol: mcp` or `protocol: json-rpc`; MCP uses explicit rules unless `mcp.allow_all_known_mcp_methods: true` enables the endpoint method profile, and JSON-RPC always uses explicit rules. | | `rules` | list of allow rule objects | No | Fine-grained protocol-specific allow rules. Mutually exclusive with `access`. | | `deny_rules` | list of deny rule objects | No | L7 deny rules that block specific requests even when allowed by `access` or `rules`. Deny rules take precedence over allow rules. | | `allowed_ips` | list of string | No | CIDR or IP allowlist for SSRF override. Exact user-declared hostname endpoints may resolve to RFC 1918 private addresses without this field, but wildcard, hostless, and policy-advisor-proposed endpoints still require `allowed_ips` for private resolved IPs. Entries overlapping loopback (`127.0.0.0/8`), link-local (`169.254.0.0/16`), or unspecified (`0.0.0.0`) are rejected at load time. | | `allow_encoded_slash` | bool | No | When `true`, L7 request parsing preserves `%2F` inside path segments instead of rejecting it. Use this for registries and APIs such as npm scoped packages (`/@scope%2Fname`). Defaults to `false`. | -| `websocket_credential_rewrite` | bool | No | When `true` on a `protocol: rest` or `protocol: websocket` endpoint, OpenShell rewrites credential placeholders in client-to-server WebSocket text messages after an allowed HTTP `101` upgrade. Binary frames are relayed but not rewritten. Defaults to `false`. | -| `request_body_credential_rewrite` | bool | No | When `true` on a `protocol: rest` endpoint, OpenShell rewrites credential placeholders in UTF-8 `application/json`, `application/x-www-form-urlencoded`, and `text/*` request bodies before forwarding upstream. The proxy buffers at most 256 KiB and updates `Content-Length` after rewriting. For chunked requests, the limit counts framing, extensions, and trailers. Defaults to `false`. Mutually exclusive with `credential_signing`. | +| `websocket_credential_rewrite` | bool | No | When `true` on a `protocol: rest` or `protocol: websocket` endpoint, OpenShell rewrites credential placeholders in client-to-server WebSocket text messages after an allowed HTTP `101` upgrade. On provider-credentialed endpoints without `allow_uninspected_credentials`, OpenShell uses the parsed relay and rejects binary frames; text frames containing placeholders fail closed when rewrite is disabled. Defaults to `false`. | +| `request_body_credential_rewrite` | bool | No | When `true` on a `protocol: rest` endpoint, OpenShell rewrites credential placeholders in UTF-8 `application/json`, `application/x-www-form-urlencoded`, and `text/*` request bodies before forwarding upstream. The proxy buffers at most 256 KiB and updates `Content-Length` after rewriting. For chunked requests, the limit counts framing, extensions, and trailers. When rewrite is disabled and the sandbox has provider credentials, ordinary bodies continue to stream, but a reserved credential placeholder is rejected before its marker reaches upstream, including for providers without endpoint profiles. Defaults to `false`. Mutually exclusive with `credential_signing`. | +| `allow_uninspected_credentials` | bool | No | Explicit security-sensitive opt-in that permits a provider-credentialed endpoint to use traffic paths OpenShell cannot inspect or rewrite, including L4-only and `tls: skip` tunnels. Defaults to `false`. Policy proposals that set it require explicit security-flagged approval. | | `credential_signing` | string | No | Proxy-side credential signing mode. When set, the proxy strips the sandbox client's `Authorization` header and re-signs with real provider credentials. Values: `sigv4` (auto-detect payload mode from client headers), `sigv4:body` (buffer and hash body, max 10 MiB), `sigv4:no_body` (unsigned payload, stream body). Mutually exclusive with `request_body_credential_rewrite`. See [AWS SigV4](/providers/aws-sigv4). | | `signing_service` | string | No | AWS service name for SigV4 signing (e.g. `bedrock`, `s3`, `sts`). Required when `credential_signing` is set. | | `signing_region` | string | No | AWS region override for SigV4 signing (e.g. `us-east-1`). When omitted, the region is extracted from the endpoint hostname. Required for non-standard AWS endpoints where the region cannot be inferred. | diff --git a/docs/sandboxes/policies.mdx b/docs/sandboxes/policies.mdx index f57a21fc3f..8cb7398efc 100644 --- a/docs/sandboxes/policies.mdx +++ b/docs/sandboxes/policies.mdx @@ -62,7 +62,7 @@ network_middlewares: Static sections are locked at sandbox creation. Changing them requires destroying and recreating the sandbox. Dynamic sections can be updated on a running sandbox with `openshell policy update` for incremental merges or `openshell policy set` for full replacement, and take effect without restarting. -When a hot reload changes rules, the supervisor publishes a new policy generation and closes connections pinned to the previous generation. This includes HTTP keep-alive tunnels, `tls: skip`, non-HTTP payloads, HTTP upgrades such as WebSocket, and long-lived response streams such as SSE. Most clients reconnect automatically, and the next connection or request is evaluated against the current policy. A parsed WebSocket relay closes with code `1012` when its attached policy generation becomes stale. Use `protocol: websocket` when policy should stay attached to the RFC 6455 upgrade and client text messages after the allowed upgrade. Add `websocket_credential_rewrite: true` only when the relay should rewrite credential placeholders in client-to-server WebSocket text messages. Add `request_body_credential_rewrite: true` only on inspected REST endpoints that need OpenShell to rewrite placeholders in supported text request bodies. +When a hot reload changes rules, the supervisor publishes a new policy generation and closes connections pinned to the previous generation. This includes HTTP keep-alive tunnels, `tls: skip`, non-HTTP payloads, HTTP upgrades such as WebSocket, and long-lived response streams such as SSE. Most clients reconnect automatically, and the next connection or request is evaluated against the current policy. A parsed WebSocket relay closes with code `1012` when its attached policy generation becomes stale. Use `protocol: websocket` when policy should stay attached to the RFC 6455 upgrade and client text messages after the allowed upgrade. Provider-credentialed endpoints reject L4-only and `tls: skip` modes by default. On a credentialed WebSocket upgrade, OpenShell keeps the connection on the parsed relay and rejects binary frames unless the endpoint explicitly sets `allow_uninspected_credentials: true`. Add `websocket_credential_rewrite: true` when the relay should rewrite credential placeholders in client-to-server WebSocket text messages. Add `request_body_credential_rewrite: true` only on inspected REST endpoints that need OpenShell to rewrite placeholders in supported text request bodies. | Section | Type | Description | |---|---|---| @@ -316,6 +316,7 @@ Examples: | Example | Meaning | |---|---| | `pypi.org:443` | Add a plain L4 endpoint. The proxy allows the TCP stream and does not inspect HTTP requests. | +| `telemetry.example.com:443::::allow-uninspected-credentials` | Explicitly allow a provider-credentialed L4 endpoint after accepting that OpenShell cannot inspect or rewrite its traffic. | | `api.github.com:443:read-only:rest:enforce` | Add a REST endpoint with the `read-only` preset expanded by the policy engine into GET, HEAD, and OPTIONS access. | | `api.example.com:443:read-write:rest:enforce:request-body-credential-rewrite` | Add a REST endpoint that rewrites credential placeholders in supported text request bodies. | | `realtime.example.com:443:read-write:websocket:enforce` | Add a WebSocket endpoint with the `read-write` preset expanded by the policy engine into the upgrade `GET` and client `WEBSOCKET_TEXT` access. | @@ -327,6 +328,8 @@ Use the `websocket-credential-rewrite` endpoint option with `protocol: websocket Use the `request-body-credential-rewrite` endpoint option with `protocol: rest` when an API expects OpenShell-managed credentials in UTF-8 JSON, form, or text request bodies. OpenShell buffers up to 256 KiB, rewrites recognized credential placeholders, updates `Content-Length`, and rejects unresolved placeholders instead of forwarding them. For chunked requests, the 256 KiB limit counts the complete wire representation, including framing, extensions, and trailers. The option is rejected for WebSocket, GraphQL, SQL, and plain L4 endpoints. +Use `allow-uninspected-credentials` only when a provider-credentialed endpoint must remain L4-only, use `tls: skip`, or carry uninspectable WebSocket traffic. Without this explicit opt-in, the gateway rejects credentialed L4-only and `tls: skip` endpoints. REST bodies without placeholders continue to work when body rewrite is disabled; a body containing an OpenShell credential placeholder fails closed. + Credential rewrite recognizes the canonical `openshell:resolve:env:KEY` placeholder form and whole-token provider-shaped aliases such as `provider-OPENSHELL-RESOLVE-ENV-API_TOKEN` when the referenced environment key exists in the configured provider credentials. Static provider placeholders resolve only when the request host, port, and path @@ -603,7 +606,7 @@ Allow `pip install` and `uv pip install` to reach PyPI: - { path: /usr/local/bin/uv } ``` -Endpoints without `protocol` use TCP passthrough, where the proxy allows the stream without inspecting payloads. If the stream is HTTP and TLS is auto-terminated, the proxy can still rewrite configured credential placeholders and closes keep-alive passthrough tunnels on policy reload before forwarding another request. WebSocket text-frame policy requires an explicit `protocol: websocket` endpoint. WebSocket payload credential rewrite can also be enabled on a `protocol: rest` compatibility endpoint with `websocket_credential_rewrite: true`. REST request body credential rewrite requires an inspected `protocol: rest` endpoint with `request_body_credential_rewrite: true`. +Endpoints without `protocol` use TCP passthrough, where the proxy allows the stream without inspecting payloads. If the stream is HTTP and TLS is auto-terminated, the proxy can still rewrite configured credential placeholders and closes keep-alive passthrough tunnels on policy reload before forwarding another request. Provider-credentialed endpoints cannot use this shape unless `allow_uninspected_credentials: true` records the exception. WebSocket text-frame policy requires an explicit `protocol: websocket` endpoint. WebSocket payload credential rewrite can also be enabled on a `protocol: rest` compatibility endpoint with `websocket_credential_rewrite: true`. REST request body credential rewrite requires an inspected `protocol: rest` endpoint with `request_body_credential_rewrite: true`. diff --git a/docs/sandboxes/providers-v2.mdx b/docs/sandboxes/providers-v2.mdx index 9395de3156..c6028b2a7c 100644 --- a/docs/sandboxes/providers-v2.mdx +++ b/docs/sandboxes/providers-v2.mdx @@ -388,6 +388,7 @@ endpoints: allow_encoded_slash: false websocket_credential_rewrite: false request_body_credential_rewrite: false + allow_uninspected_credentials: false persisted_queries: deny graphql_max_body_bytes: 65536 rules: @@ -435,7 +436,7 @@ credential declared under `credentials`. OpenShell scans the referenced credential's `env_vars` in order and stores the first non-empty local environment value under the actual environment variable key. -`endpoints` contains the same endpoint object shape as sandbox network policy. A profile can use access presets, protocol-specific allow rules, deny rules, WebSocket credential rewriting, request body credential rewriting, GraphQL fields, and SSRF IP allowlists. +`endpoints` contains the same endpoint object shape as sandbox network policy. A profile can use access presets, protocol-specific allow rules, deny rules, WebSocket credential rewriting, request body credential rewriting, GraphQL fields, and SSRF IP allowlists. Because profile credentials are not mapped to individual endpoints, OpenShell conservatively treats every endpoint in a profile that declares credentials as credentialed. Such endpoints require L7 inspection and cannot use `tls: skip` unless the profile explicitly sets `allow_uninspected_credentials: true`. `binaries` contains the executable paths allowed to reach the profile endpoints when the profile contributes policy to a sandbox. diff --git a/docs/security/best-practices.mdx b/docs/security/best-practices.mdx index 8bbcc604d4..29f095d6f8 100644 --- a/docs/security/best-practices.mdx +++ b/docs/security/best-practices.mdx @@ -96,8 +96,8 @@ The `protocol` field on an endpoint controls whether the proxy inspects individu | Aspect | Detail | |---|---| -| Default | Endpoints without a `protocol` field use L4-only enforcement: the proxy checks host, port, and binary, then relays the TCP stream without inspecting payloads. | -| What you can change | Add `protocol: rest` to enable per-request HTTP method/path inspection, `protocol: websocket` to inspect RFC 6455 upgrade handshakes and client text messages, or `protocol: graphql` to inspect GraphQL-over-HTTP operation type, operation name, and root fields. WebSocket endpoints can also use GraphQL operation rules for GraphQL-over-WebSocket messages. Pair inspected protocols with `rules` or access presets (`full`, `read-only`, `read-write`). REST endpoints that need credential placeholders in supported text request bodies can set `request_body_credential_rewrite: true`. | +| Default | Endpoints without a `protocol` field use L4-only enforcement: the proxy checks host, port, and binary, then relays the TCP stream without inspecting payloads. Provider-credentialed endpoints reject this mode unless an operator explicitly opts in. | +| What you can change | Add `protocol: rest` to enable per-request HTTP method/path inspection, `protocol: websocket` to inspect RFC 6455 upgrade handshakes and client text messages, or `protocol: graphql` to inspect GraphQL-over-HTTP operation type, operation name, and root fields. WebSocket endpoints can also use GraphQL operation rules for GraphQL-over-WebSocket messages. Pair inspected protocols with `rules` or access presets (`full`, `read-only`, `read-write`). REST endpoints that need credential placeholders in supported text request bodies can set `request_body_credential_rewrite: true`. Set `allow_uninspected_credentials: true` only as an explicit exception for credentialed traffic that cannot use an inspected path. | | Risk if relaxed | L4-only endpoints allow the agent to send any data through the tunnel after the initial connection is permitted. The proxy cannot see HTTP methods, paths, or GraphQL operations. Adding `access: full` with L7 inspection enables observability but permits all inspected actions. | | Recommendation | Use `protocol: rest` with specific `rules` for APIs where intent is encoded in method and path. Add `request_body_credential_rewrite: true` only for REST APIs that require OpenShell-managed credentials in UTF-8 JSON, form, or text request bodies. Use `protocol: graphql` for GraphQL-over-HTTP APIs where destructive operations are body-encoded. Use `protocol: websocket` for RFC 6455 endpoints, with explicit `GET` and `WEBSOCKET_TEXT` rules for raw text protocols or explicit GraphQL operation rules for GraphQL-over-WebSocket. Prefer `access: read-only` or explicit allowlists, and deny hash-only persisted queries unless you maintain a trusted registry. Omit `protocol` for non-HTTP protocols. For WebSocket endpoints that must carry placeholder credentials in client text frames, add `websocket_credential_rewrite: true`. | @@ -123,7 +123,7 @@ This enables credential injection and L7 inspection without explicit configurati | Default | Auto-detect and terminate. OpenShell generates the sandbox CA at startup and injects it into the process trust stores (`NODE_EXTRA_CA_CERTS`, `DENO_CERT`, `SSL_CERT_FILE`, `REQUESTS_CA_BUNDLE`, `CURL_CA_BUNDLE`, `GIT_SSL_CAINFO`). | | What you can change | Set `tls: skip` on an endpoint to disable TLS detection and termination for that endpoint. Use this for client-certificate mTLS to upstream or non-standard binary protocols. | | Risk if relaxed | `tls: skip` disables placeholder credential rewriting, dynamic token grant injection, and L7 inspection for that endpoint. The proxy relays encrypted traffic without seeing the contents. | -| Recommendation | Use auto-detect (the default) for most endpoints. Use `tls: skip` only when the upstream requires the client's own TLS certificate (mTLS) or uses a non-HTTP protocol. | +| Recommendation | Use auto-detect (the default) for most endpoints. Use `tls: skip` only when the upstream requires the client's own TLS certificate (mTLS) or uses a non-HTTP protocol. A provider-credentialed endpoint also requires the explicit `allow_uninspected_credentials: true` exception. | ### SSRF Protection diff --git a/e2e/rust/Cargo.toml b/e2e/rust/Cargo.toml index 7f6c6467d9..0b8b56f3e5 100644 --- a/e2e/rust/Cargo.toml +++ b/e2e/rust/Cargo.toml @@ -116,6 +116,11 @@ name = "websocket_conformance" path = "tests/websocket_conformance.rs" required-features = ["e2e-host-gateway"] +[[test]] +name = "credential_gating" +path = "tests/credential_gating.rs" +required-features = ["e2e-host-gateway"] + [[test]] name = "user_namespaces" path = "tests/user_namespaces.rs" diff --git a/e2e/rust/tests/credential_gating.rs b/e2e/rust/tests/credential_gating.rs new file mode 100644 index 0000000000..c65694d625 --- /dev/null +++ b/e2e/rust/tests/credential_gating.rs @@ -0,0 +1,843 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#![cfg(feature = "e2e")] + +//! E2E coverage for credentialed endpoint admission and REST body backstops. + +use std::io::Write; +use std::process::Stdio; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +use base64::Engine as _; +use openshell_e2e::harness::binary::openshell_cmd; +use openshell_e2e::harness::sandbox::SandboxGuard; +use sha1::{Digest, Sha1}; +use tempfile::NamedTempFile; +use tokio::io::{AsyncReadExt, AsyncWriteExt}; +use tokio::net::{TcpListener, TcpStream}; +use tokio::task::JoinHandle; + +const PROFILE_ID: &str = "e2e-credential-gating"; +const PROVIDER_NAME: &str = "e2e-credential-gating"; +const TEST_HOST: &str = "host.openshell.internal"; +const TOKEN_ENV: &str = "E2E_GATING_TOKEN"; +const TEST_SECRET: &str = "e2e-gating-secret-value"; +const PLACEHOLDER_PREFIX: &str = "openshell:resolve:env:"; +const WEBSOCKET_GUID: &str = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"; + +async fn run_cli(args: &[&str]) -> (bool, String) { + let mut command = openshell_cmd(); + command + .args(args) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); + let output = command.output().await.expect("spawn openshell CLI"); + ( + output.status.success(), + format!( + "{}{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ), + ) +} + +/// Retry a delete until it takes effect. +/// +/// The gateway refuses to delete a provider or a profile that a sandbox still +/// references, and sandbox teardown drains asynchronously, so a single attempt +/// can be rejected right after a sandbox was removed. +async fn delete_until_gone(args: &[&str]) -> Result<(), String> { + const ATTEMPTS: u32 = 40; + let mut last_output = String::new(); + for _ in 0..ATTEMPTS { + let (deleted, output) = run_cli(args).await; + if deleted || output.to_lowercase().contains("not found") { + return Ok(()); + } + last_output = output; + tokio::time::sleep(Duration::from_millis(500)).await; + } + Err(format!( + "'{}' still failing after {ATTEMPTS} attempts:\n{last_output}", + args.join(" ") + )) +} + +/// Strict teardown for the install paths: every test resource must be gone +/// before it is recreated, otherwise creation fails with "already exists". +async fn ensure_provider_resources_absent() -> Result<(), String> { + delete_until_gone(&["provider", "delete", PROVIDER_NAME]).await?; + delete_until_gone(&["provider", "profile", "delete", PROFILE_ID]).await +} + +/// Best-effort teardown. Never fails the test: it also runs on the failure +/// path, where the original assertion is the interesting one. +async fn cleanup_provider_resources() { + if let Err(error) = ensure_provider_resources_absent().await { + eprintln!("provider cleanup did not settle: {error}"); + } +} + +fn write_provider_profile(rest_port: u16, websocket_port: u16) -> Result { + let mut file = tempfile::Builder::new() + .suffix(".yaml") + .tempfile() + .map_err(|error| format!("create profile: {error}"))?; + let profile = format!( + r"id: {PROFILE_ID} +display_name: E2E Credential Gating +category: other +credentials: + - name: token + env_vars: [{TOKEN_ENV}] + required: true + auth_style: bearer + header_name: authorization +endpoints: + - host: {TEST_HOST} + port: {rest_port} + protocol: rest + access: full + - host: {TEST_HOST} + port: {websocket_port} + protocol: websocket + access: read-write +binaries: + - path: /usr/bin/python* + - path: /usr/local/bin/python* + - path: /sandbox/.uv/python/*/bin/python* +", + ); + file.write_all(profile.as_bytes()) + .map_err(|error| format!("write profile: {error}"))?; + file.flush() + .map_err(|error| format!("flush profile: {error}"))?; + Ok(file) +} + +async fn install_provider(rest_port: u16, websocket_port: u16) -> Result<(), String> { + ensure_provider_resources_absent().await?; + let profile = write_provider_profile(rest_port, websocket_port)?; + let profile_path = profile + .path() + .to_str() + .ok_or_else(|| "profile path is not UTF-8".to_string())?; + let (imported, output) = + run_cli(&["provider", "profile", "import", "--file", profile_path]).await; + if !imported { + return Err(format!("profile import failed:\n{output}")); + } + let credential = format!("{TOKEN_ENV}={TEST_SECRET}"); + let (created, output) = run_cli(&[ + "provider", + "create", + "--name", + PROVIDER_NAME, + "--type", + PROFILE_ID, + "--credential", + &credential, + ]) + .await; + if !created { + return Err(format!("provider create failed:\n{output}")); + } + Ok(()) +} + +fn write_endpointless_provider_profile() -> Result { + let mut file = tempfile::Builder::new() + .suffix(".yaml") + .tempfile() + .map_err(|error| format!("create endpointless profile: {error}"))?; + let profile = format!( + r"id: {PROFILE_ID} +display_name: E2E Endpointless Credential Gating +category: other +credentials: + - name: token + env_vars: [{TOKEN_ENV}] + required: true + auth_style: bearer + header_name: authorization +binaries: + - path: /usr/bin/python* + - path: /usr/local/bin/python* + - path: /sandbox/.uv/python/*/bin/python* +", + ); + file.write_all(profile.as_bytes()) + .map_err(|error| format!("write endpointless profile: {error}"))?; + file.flush() + .map_err(|error| format!("flush endpointless profile: {error}"))?; + Ok(file) +} + +async fn install_endpointless_provider() -> Result<(), String> { + ensure_provider_resources_absent().await?; + let profile = write_endpointless_provider_profile()?; + let profile_path = profile + .path() + .to_str() + .ok_or_else(|| "endpointless profile path is not UTF-8".to_string())?; + let (imported, output) = + run_cli(&["provider", "profile", "import", "--file", profile_path]).await; + if !imported { + return Err(format!("endpointless profile import failed:\n{output}")); + } + let credential = format!("{TOKEN_ENV}={TEST_SECRET}"); + let (created, output) = run_cli(&[ + "provider", + "create", + "--name", + PROVIDER_NAME, + "--type", + PROFILE_ID, + "--credential", + &credential, + ]) + .await; + if !created { + return Err(format!("endpointless provider create failed:\n{output}")); + } + Ok(()) +} + +#[derive(Clone, Copy)] +enum EndpointMode { + L4, + TlsSkip, + L4OptIn, + RestBody { rewrite: bool }, + WebSocket, +} + +#[derive(Clone, Copy)] +enum CredentialSource { + ProviderProfile, + PolicyBinding, +} + +fn write_policy( + port: u16, + mode: EndpointMode, + credential_source: CredentialSource, +) -> Result { + let mut file = NamedTempFile::new().map_err(|error| format!("create policy: {error}"))?; + let endpoint_options = match mode { + EndpointMode::L4 => String::new(), + EndpointMode::TlsSkip => { + " protocol: rest\n access: full\n tls: skip\n".to_string() + } + EndpointMode::L4OptIn => " allow_uninspected_credentials: true\n".to_string(), + EndpointMode::RestBody { rewrite } => format!( + " protocol: rest\n access: full\n request_body_credential_rewrite: {rewrite}\n" + ), + EndpointMode::WebSocket => { + " protocol: websocket\n access: read-write\n".to_string() + } + }; + let credential_binding = match credential_source { + CredentialSource::ProviderProfile => String::new(), + CredentialSource::PolicyBinding => { + format!(" credential_binding:\n provider: {PROVIDER_NAME}\n") + } + }; + let policy = format!( + r#"version: 1 +filesystem_policy: + include_workdir: true + read_only: [/usr, /lib, /proc, /dev/urandom, /app, /etc, /var/log] + read_write: [/sandbox, /tmp, /dev/null] +landlock: + compatibility: best_effort +process: + run_as_user: sandbox + run_as_group: sandbox +network_policies: + credential_gating: + name: credential_gating + endpoints: + - host: {TEST_HOST} + port: {port} +{endpoint_options}{credential_binding} allowed_ips: + - "10.0.0.0/8" + - "172.0.0.0/8" + - "192.168.0.0/16" + - "fc00::/7" + binaries: + - path: /usr/bin/python* + - path: /usr/local/bin/python* + - path: /sandbox/.uv/python/*/bin/python* +"#, + ); + file.write_all(policy.as_bytes()) + .map_err(|error| format!("write policy: {error}"))?; + file.flush() + .map_err(|error| format!("flush policy: {error}"))?; + Ok(file) +} + +#[derive(Debug, Default, Clone, Copy)] +struct BodyObservation { + saw_placeholder: bool, + saw_secret: bool, +} + +struct HttpProbeServer { + port: u16, + observations: Arc>>, + task: JoinHandle<()>, +} + +impl HttpProbeServer { + async fn start() -> Result { + let listener = TcpListener::bind(("0.0.0.0", 0)) + .await + .map_err(|error| format!("bind HTTP probe: {error}"))?; + let port = listener + .local_addr() + .map_err(|error| format!("read HTTP probe address: {error}"))? + .port(); + let observations = Arc::new(Mutex::new(Vec::new())); + let task_observations = Arc::clone(&observations); + let task = tokio::spawn(async move { + loop { + let Ok((stream, _)) = listener.accept().await else { + break; + }; + let observations = Arc::clone(&task_observations); + tokio::spawn(async move { + let _ = handle_http_probe(stream, observations).await; + }); + } + }); + Ok(Self { + port, + observations, + task, + }) + } + + async fn wait_for_observations(&self, count: usize) -> Vec { + for _ in 0..100 { + let observations = self.observations.lock().unwrap().clone(); + if observations.len() >= count { + return observations; + } + tokio::time::sleep(Duration::from_millis(100)).await; + } + self.observations.lock().unwrap().clone() + } +} + +impl Drop for HttpProbeServer { + fn drop(&mut self) { + self.task.abort(); + } +} + +struct BinaryWebSocketProbeServer { + port: u16, + handshake_seen: Arc, + binary_seen: Arc, + task: JoinHandle<()>, +} + +impl BinaryWebSocketProbeServer { + async fn start() -> Result { + let listener = TcpListener::bind(("0.0.0.0", 0)) + .await + .map_err(|error| format!("bind WebSocket probe: {error}"))?; + let port = listener + .local_addr() + .map_err(|error| format!("read WebSocket probe address: {error}"))? + .port(); + let handshake_seen = Arc::new(AtomicBool::new(false)); + let binary_seen = Arc::new(AtomicBool::new(false)); + let task_handshake_seen = Arc::clone(&handshake_seen); + let task_binary_seen = Arc::clone(&binary_seen); + let task = tokio::spawn(async move { + loop { + let Ok((stream, _)) = listener.accept().await else { + break; + }; + let handshake_seen = Arc::clone(&task_handshake_seen); + let binary_seen = Arc::clone(&task_binary_seen); + tokio::spawn(async move { + let _ = + handle_binary_websocket_probe(stream, handshake_seen, binary_seen).await; + }); + } + }); + Ok(Self { + port, + handshake_seen, + binary_seen, + task, + }) + } + + async fn wait_for_handshake(&self) -> bool { + for _ in 0..100 { + if self.handshake_seen.load(Ordering::Acquire) { + return true; + } + tokio::time::sleep(Duration::from_millis(50)).await; + } + false + } +} + +impl Drop for BinaryWebSocketProbeServer { + fn drop(&mut self) { + self.task.abort(); + } +} + +async fn recv_until(stream: &mut TcpStream, marker: &[u8]) -> std::io::Result> { + let mut received = Vec::new(); + let mut buffer = [0_u8; 1024]; + loop { + let read = stream.read(&mut buffer).await?; + if read == 0 { + return Ok(received); + } + received.extend_from_slice(&buffer[..read]); + if received + .windows(marker.len()) + .any(|window| window == marker) + { + return Ok(received); + } + } +} + +fn websocket_header_value(request: &str, name: &str) -> Option { + request.lines().find_map(|line| { + let (header, value) = line.split_once(':')?; + header + .trim() + .eq_ignore_ascii_case(name) + .then(|| value.trim().to_string()) + }) +} + +fn websocket_accept(key: &str) -> String { + let mut hasher = Sha1::new(); + hasher.update(key.as_bytes()); + hasher.update(WEBSOCKET_GUID.as_bytes()); + base64::engine::general_purpose::STANDARD.encode(hasher.finalize()) +} + +async fn handle_binary_websocket_probe( + mut stream: TcpStream, + handshake_seen: Arc, + binary_seen: Arc, +) -> std::io::Result<()> { + let request_bytes = recv_until(&mut stream, b"\r\n\r\n").await?; + let request = String::from_utf8_lossy(&request_bytes); + let key = websocket_header_value(&request, "Sec-WebSocket-Key").ok_or_else(|| { + std::io::Error::new(std::io::ErrorKind::InvalidData, "missing WebSocket key") + })?; + let accept = websocket_accept(&key); + let response = format!( + "HTTP/1.1 101 Switching Protocols\r\n\ + Upgrade: websocket\r\n\ + Connection: Upgrade\r\n\ + Sec-WebSocket-Accept: {accept}\r\n\ + \r\n" + ); + stream.write_all(response.as_bytes()).await?; + handshake_seen.store(true, Ordering::Release); + + let mut header = [0_u8; 2]; + if tokio::time::timeout(Duration::from_secs(5), stream.read_exact(&mut header)) + .await + .is_ok_and(|result| result.is_ok()) + && header[0] & 0x0f == 0x02 + { + binary_seen.store(true, Ordering::Release); + } + Ok(()) +} + +async fn handle_http_probe( + mut stream: TcpStream, + observations: Arc>>, +) -> std::io::Result<()> { + let mut received = Vec::new(); + let mut buffer = [0_u8; 4096]; + let mut expected_total = None; + loop { + let read = tokio::time::timeout(Duration::from_secs(10), stream.read(&mut buffer)).await; + let Ok(Ok(read)) = read else { + break; + }; + if read == 0 { + break; + } + received.extend_from_slice(&buffer[..read]); + if expected_total.is_none() + && let Some(header_end) = received.windows(4).position(|window| window == b"\r\n\r\n") + { + let header_end = header_end + 4; + let headers = String::from_utf8_lossy(&received[..header_end]); + let content_length = headers + .lines() + .find_map(|line| { + let (name, value) = line.split_once(':')?; + name.eq_ignore_ascii_case("content-length") + .then(|| value.trim().parse::().ok()) + .flatten() + }) + .unwrap_or(0); + expected_total = Some(header_end + content_length); + } + if expected_total.is_some_and(|expected| received.len() >= expected) { + break; + } + } + + let observation = BodyObservation { + saw_placeholder: received + .windows(PLACEHOLDER_PREFIX.len()) + .any(|window| window == PLACEHOLDER_PREFIX.as_bytes()), + saw_secret: received + .windows(TEST_SECRET.len()) + .any(|window| window == TEST_SECRET.as_bytes()), + }; + observations.lock().unwrap().push(observation); + if expected_total.is_some_and(|expected| received.len() >= expected) { + let result = if observation.saw_secret && !observation.saw_placeholder { + "BODY_REWRITTEN" + } else { + "BODY_BAD" + }; + let response = format!( + "HTTP/1.1 200 OK\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{result}", + result.len() + ); + stream.write_all(response.as_bytes()).await?; + } + Ok(()) +} + +fn body_client_script(port: u16) -> String { + format!( + r#" +import os +import socket +import urllib.parse + +host = {TEST_HOST:?} +port = {port} +token = os.environ[{TOKEN_ENV:?}] +proxy_url = next(os.environ[name] for name in + ("HTTP_PROXY", "http_proxy", "HTTPS_PROXY", "https_proxy", "ALL_PROXY", "all_proxy") + if os.environ.get(name)) +proxy = urllib.parse.urlparse(proxy_url) + +with socket.create_connection((proxy.hostname, proxy.port or 80), timeout=10) as sock: + target = f"{{host}}:{{port}}" + sock.sendall(f"CONNECT {{target}} HTTP/1.1\r\nHost: {{target}}\r\n\r\n".encode("ascii")) + response = b"" + while b"\r\n\r\n" not in response: + chunk = sock.recv(4096) + if not chunk: + break + response += chunk + if not response.startswith(b"HTTP/1.1 200"): + raise RuntimeError("CONNECT failed") + body = ("prefix-" + token + "-suffix").encode("utf-8") + request = ( + f"POST /token HTTP/1.1\r\nHost: {{target}}\r\n" + f"Content-Type: text/plain\r\nContent-Length: {{len(body)}}\r\nConnection: close\r\n\r\n" + ).encode("ascii") + body + sock.sendall(request) + sock.settimeout(3) + response = b"" + while True: + try: + chunk = sock.recv(4096) + except socket.timeout: + break + if not chunk: + break + response += chunk + print("BODY_REWRITTEN" if b"BODY_REWRITTEN" in response else "BODY_DENIED") +"# + ) +} + +fn binary_websocket_client_script(port: u16) -> String { + format!( + r#" +import base64 +import os +import socket +import struct +import urllib.parse + +host = {TEST_HOST:?} +port = {port} +proxy_url = next(os.environ[name] for name in + ("HTTP_PROXY", "http_proxy", "HTTPS_PROXY", "https_proxy", "ALL_PROXY", "all_proxy") + if os.environ.get(name)) +proxy = urllib.parse.urlparse(proxy_url) + +def recv_until(sock, marker): + data = b"" + while marker not in data: + chunk = sock.recv(4096) + if not chunk: + break + data += chunk + return data + +def recv_exact(sock, size): + data = b"" + while len(data) < size: + chunk = sock.recv(size - len(data)) + if not chunk: + break + data += chunk + return data + +with socket.create_connection((proxy.hostname, proxy.port or 80), timeout=10) as sock: + target = f"{{host}}:{{port}}" + sock.sendall(f"CONNECT {{target}} HTTP/1.1\r\nHost: {{target}}\r\n\r\n".encode("ascii")) + if not recv_until(sock, b"\r\n\r\n").startswith(b"HTTP/1.1 200"): + raise RuntimeError("CONNECT failed") + key = base64.b64encode(os.urandom(16)).decode("ascii") + request = ( + f"GET /ws HTTP/1.1\r\nHost: {{target}}\r\n" + "Upgrade: websocket\r\nConnection: Upgrade\r\n" + f"Sec-WebSocket-Key: {{key}}\r\nSec-WebSocket-Version: 13\r\n\r\n" + ) + sock.sendall(request.encode("ascii")) + if not recv_until(sock, b"\r\n\r\n").startswith(b"HTTP/1.1 101"): + raise RuntimeError("upgrade failed") + payload = b"binary-credential-channel" + mask = os.urandom(4) + masked = bytes(byte ^ mask[index % 4] for index, byte in enumerate(payload)) + frame = bytes([0x82, 0x80 | len(payload)]) + mask + masked + sock.sendall(frame) + sock.settimeout(3) + denied = False + try: + header = recv_exact(sock, 2) + if not header: + denied = True + elif len(header) == 2: + opcode = header[0] & 0x0f + masked = bool(header[1] & 0x80) + payload_length = header[1] & 0x7f + if opcode == 0x08 and not masked and payload_length < 126: + close_payload = recv_exact(sock, payload_length) + denied = ( + len(close_payload) >= 2 + and struct.unpack("!H", close_payload[:2])[0] == 1008 + ) + except socket.timeout: + denied = True + print("BINARY_DENIED" if denied else "BINARY_FORWARDED") +"# + ) +} + +async fn sandbox_create_failure(policy: &NamedTempFile) -> String { + let policy_path = policy.path().to_str().expect("policy path is UTF-8"); + let (success, output) = run_cli(&[ + "sandbox", + "create", + "--policy", + policy_path, + "--provider", + PROVIDER_NAME, + "--", + "echo", + "must-not-run", + ]) + .await; + assert!(!success, "sandbox create unexpectedly succeeded:\n{output}"); + output +} + +async fn assert_gateway_admission( + port: u16, + credential_source: CredentialSource, +) -> Result<(), String> { + let l4 = write_policy(port, EndpointMode::L4, credential_source)?; + let l4_error = sandbox_create_failure(&l4).await; + assert!(l4_error.contains("credentialed endpoint"), "{l4_error}"); + assert!(l4_error.contains("L4-only"), "{l4_error}"); + + let tls_skip = write_policy(port, EndpointMode::TlsSkip, credential_source)?; + let tls_error = sandbox_create_failure(&tls_skip).await; + assert!(tls_error.contains("credentialed endpoint"), "{tls_error}"); + assert!( + tls_error.contains("tls:") && tls_error.contains("skip"), + "{tls_error}" + ); + + let opt_in = write_policy(port, EndpointMode::L4OptIn, credential_source)?; + let opt_in_path = opt_in + .path() + .to_str() + .ok_or_else(|| "opt-in policy path is not UTF-8".to_string())?; + let accepted_marker = match credential_source { + CredentialSource::ProviderProfile => "OPT_IN_ACCEPTED", + CredentialSource::PolicyBinding => "ENDPOINTLESS_OPT_IN_ACCEPTED", + }; + let mut sandbox = SandboxGuard::create(&[ + "--policy", + opt_in_path, + "--provider", + PROVIDER_NAME, + "--", + "echo", + accepted_marker, + ]) + .await?; + assert!(sandbox.create_output.contains(accepted_marker)); + sandbox.cleanup().await; + Ok(()) +} + +async fn run_body_sandbox( + port: u16, + mode: EndpointMode, + credential_source: CredentialSource, +) -> Result { + let policy = write_policy(port, mode, credential_source)?; + let policy_path = policy + .path() + .to_str() + .ok_or_else(|| "body policy path is not UTF-8".to_string())?; + let script = body_client_script(port); + let mut sandbox = SandboxGuard::create(&[ + "--policy", + policy_path, + "--provider", + PROVIDER_NAME, + "--", + "python3", + "-c", + &script, + ]) + .await?; + let output = sandbox.create_output.clone(); + sandbox.cleanup().await; + Ok(output) +} + +async fn assert_rest_body_backstop(server: &HttpProbeServer) -> Result<(), String> { + let denied = run_body_sandbox( + server.port, + EndpointMode::RestBody { rewrite: false }, + CredentialSource::ProviderProfile, + ) + .await?; + assert!(denied.contains("BODY_DENIED")); + + let rewritten = run_body_sandbox( + server.port, + EndpointMode::RestBody { rewrite: true }, + CredentialSource::ProviderProfile, + ) + .await?; + assert!(rewritten.contains("BODY_REWRITTEN")); + assert!(!rewritten.contains(TEST_SECRET)); + assert!(!rewritten.contains(PLACEHOLDER_PREFIX)); + + let observations = server.wait_for_observations(2).await; + assert_eq!(observations.len(), 2, "observations: {observations:?}"); + assert!(!observations[0].saw_placeholder); + assert!(!observations[0].saw_secret); + assert!(!observations[1].saw_placeholder); + assert!(observations[1].saw_secret); + Ok(()) +} + +async fn assert_websocket_binary_denied(server: &BinaryWebSocketProbeServer) -> Result<(), String> { + let policy = write_policy( + server.port, + EndpointMode::WebSocket, + CredentialSource::ProviderProfile, + )?; + let policy_path = policy + .path() + .to_str() + .ok_or_else(|| "WebSocket policy path is not UTF-8".to_string())?; + let script = binary_websocket_client_script(server.port); + let mut sandbox = SandboxGuard::create(&[ + "--policy", + policy_path, + "--provider", + PROVIDER_NAME, + "--", + "python3", + "-c", + &script, + ]) + .await?; + assert!(sandbox.create_output.contains("BINARY_DENIED")); + sandbox.cleanup().await; + assert!( + server.wait_for_handshake().await, + "upstream should receive the WebSocket handshake" + ); + assert!( + !server.binary_seen.load(Ordering::Acquire), + "credentialed WebSocket binary frame reached upstream" + ); + Ok(()) +} + +#[tokio::test] +async fn credentialed_endpoint_gates_work_end_to_end() { + let server = HttpProbeServer::start().await.expect("start HTTP probe"); + let websocket_server = BinaryWebSocketProbeServer::start() + .await + .expect("start WebSocket probe"); + install_provider(server.port, websocket_server.port) + .await + .expect("install credentialed provider"); + + let result = async { + assert_gateway_admission(server.port, CredentialSource::ProviderProfile).await?; + assert_rest_body_backstop(&server).await?; + assert_websocket_binary_denied(&websocket_server).await + } + .await; + + cleanup_provider_resources().await; + result.expect("credential gating E2E"); + + install_endpointless_provider() + .await + .expect("install endpointless provider"); + let endpointless_result = async { + assert_gateway_admission(server.port, CredentialSource::PolicyBinding).await?; + let denied = run_body_sandbox( + server.port, + EndpointMode::RestBody { rewrite: false }, + CredentialSource::PolicyBinding, + ) + .await?; + assert!(denied.contains("BODY_DENIED")); + let observations = server.wait_for_observations(3).await; + assert_eq!(observations.len(), 3, "observations: {observations:?}"); + assert!(!observations[2].saw_placeholder); + assert!(!observations[2].saw_secret); + Ok::<(), String>(()) + } + .await; + cleanup_provider_resources().await; + endpointless_result.expect("endpointless provider credential gating E2E"); +} diff --git a/proto/sandbox.proto b/proto/sandbox.proto index 23cbbbae55..95df265ff2 100644 --- a/proto/sandbox.proto +++ b/proto/sandbox.proto @@ -186,6 +186,13 @@ message NetworkEndpoint { // endpointless provider profile. Profiles that already define endpoints // continue to use those profile endpoints as their credential boundary. NetworkCredentialBinding credential_binding = 24; + // Explicitly permits credential-bearing traffic to use paths that OpenShell + // cannot inspect or rewrite. Defaults to false. This is a security-sensitive + // escape hatch and must be explicitly approved. + bool allow_uninspected_credentials = 25; + // Internal gateway-derived marker indicating that this endpoint belongs to + // an attached credentialed provider. User-authored values are ignored. + bool provider_credentialed = 26; } // MCP options are grouped so MCP-specific policy can grow without adding more diff --git a/providers/copilot.yaml b/providers/copilot.yaml index 1b219fd221..cc7e5145c3 100644 --- a/providers/copilot.yaml +++ b/providers/copilot.yaml @@ -42,8 +42,10 @@ endpoints: enforcement: enforce - host: telemetry.enterprise.githubcopilot.com port: 443 + allow_uninspected_credentials: true - host: default.exp-tas.com port: 443 + allow_uninspected_credentials: true binaries: - /usr/bin/copilot - /usr/lib/node_modules/@github/copilot/node_modules/@github/**/copilot diff --git a/sdk/go/openshell/v1/internal/converter/coverage_test.go b/sdk/go/openshell/v1/internal/converter/coverage_test.go index 34cdc0e05e..33edebf64f 100644 --- a/sdk/go/openshell/v1/internal/converter/coverage_test.go +++ b/sdk/go/openshell/v1/internal/converter/coverage_test.go @@ -130,6 +130,8 @@ func TestConverterCoversAllProtoFields_NetworkEndpoint(t *testing.T) { "path": true, "websocket_credential_rewrite": true, "request_body_credential_rewrite": true, + "allow_uninspected_credentials": true, + "provider_credentialed": true, "advisor_proposed": true, "credential_signing": true, "signing_service": true, diff --git a/sdk/go/openshell/v1/internal/converter/network_policy.go b/sdk/go/openshell/v1/internal/converter/network_policy.go index e90cf209d8..81ff463d3d 100644 --- a/sdk/go/openshell/v1/internal/converter/network_policy.go +++ b/sdk/go/openshell/v1/internal/converter/network_policy.go @@ -76,6 +76,8 @@ func policyNetworkEndpointFromProto(ep *sbv1.NetworkEndpoint) types.PolicyNetwor Path: ep.GetPath(), WebsocketCredentialRewrite: ep.GetWebsocketCredentialRewrite(), RequestBodyCredentialRewrite: ep.GetRequestBodyCredentialRewrite(), + AllowUninspectedCredentials: ep.GetAllowUninspectedCredentials(), + ProviderCredentialed: ep.GetProviderCredentialed(), AdvisorProposed: ep.GetAdvisorProposed(), CredentialSigning: ep.GetCredentialSigning(), SigningService: ep.GetSigningService(), @@ -136,6 +138,8 @@ func policyNetworkEndpointToProto(ep *types.PolicyNetworkEndpoint) *sbv1.Network Path: ep.Path, WebsocketCredentialRewrite: ep.WebsocketCredentialRewrite, RequestBodyCredentialRewrite: ep.RequestBodyCredentialRewrite, + AllowUninspectedCredentials: ep.AllowUninspectedCredentials, + ProviderCredentialed: ep.ProviderCredentialed, AdvisorProposed: ep.AdvisorProposed, CredentialSigning: ep.CredentialSigning, SigningService: ep.SigningService, diff --git a/sdk/go/openshell/v1/internal/converter/network_policy_test.go b/sdk/go/openshell/v1/internal/converter/network_policy_test.go index d623cbc903..5f83a82645 100644 --- a/sdk/go/openshell/v1/internal/converter/network_policy_test.go +++ b/sdk/go/openshell/v1/internal/converter/network_policy_test.go @@ -33,6 +33,8 @@ func TestNetworkPolicyRuleFromProto(t *testing.T) { Path: "/api/v1", WebsocketCredentialRewrite: true, RequestBodyCredentialRewrite: false, + AllowUninspectedCredentials: true, + ProviderCredentialed: true, AdvisorProposed: true, CredentialSigning: "sigv4", SigningService: "bedrock", @@ -110,6 +112,8 @@ func TestNetworkPolicyRuleFromProto(t *testing.T) { assert.Equal(t, "/api/v1", ep.Path) assert.True(t, ep.WebsocketCredentialRewrite) assert.False(t, ep.RequestBodyCredentialRewrite) + assert.True(t, ep.AllowUninspectedCredentials) + assert.True(t, ep.ProviderCredentialed) assert.True(t, ep.AdvisorProposed) assert.Equal(t, "sigv4", ep.CredentialSigning) assert.Equal(t, "bedrock", ep.SigningService) @@ -188,6 +192,8 @@ func TestNetworkPolicyRuleRoundTrip(t *testing.T) { Path: "/graphql", WebsocketCredentialRewrite: false, RequestBodyCredentialRewrite: true, + AllowUninspectedCredentials: true, + ProviderCredentialed: true, AdvisorProposed: false, CredentialSigning: "sigv4", SigningService: "bedrock", @@ -257,6 +263,8 @@ func TestNetworkPolicyRuleRoundTrip(t *testing.T) { assert.Equal(t, original.Endpoints[0].AllowedIPs, roundTrip.Endpoints[0].AllowedIPs) assert.Equal(t, original.Endpoints[0].AllowEncodedSlash, roundTrip.Endpoints[0].AllowEncodedSlash) assert.Equal(t, original.Endpoints[0].GraphqlMaxBodyBytes, roundTrip.Endpoints[0].GraphqlMaxBodyBytes) + assert.Equal(t, original.Endpoints[0].AllowUninspectedCredentials, roundTrip.Endpoints[0].AllowUninspectedCredentials) + assert.Equal(t, original.Endpoints[0].ProviderCredentialed, roundTrip.Endpoints[0].ProviderCredentialed) assert.Equal(t, original.Endpoints[0].AdvisorProposed, roundTrip.Endpoints[0].AdvisorProposed) assert.Equal(t, original.Endpoints[0].CredentialSigning, roundTrip.Endpoints[0].CredentialSigning) assert.Equal(t, original.Endpoints[0].SigningService, roundTrip.Endpoints[0].SigningService) diff --git a/sdk/go/openshell/v1/types/network_policy.go b/sdk/go/openshell/v1/types/network_policy.go index ed6938b9b8..d357cca873 100644 --- a/sdk/go/openshell/v1/types/network_policy.go +++ b/sdk/go/openshell/v1/types/network_policy.go @@ -34,13 +34,19 @@ type PolicyNetworkEndpoint struct { Path string WebsocketCredentialRewrite bool RequestBodyCredentialRewrite bool - AdvisorProposed bool - CredentialSigning string - SigningService string - SigningRegion string - JSONRPCMaxBodyBytes uint32 - Mcp *McpOptions - CredentialBinding *NetworkCredentialBinding + // AllowUninspectedCredentials explicitly permits credential-bearing traffic + // on paths OpenShell cannot inspect or rewrite. + AllowUninspectedCredentials bool + // ProviderCredentialed is gateway-derived provenance indicating that the + // endpoint belongs to an attached credentialed provider. + ProviderCredentialed bool + AdvisorProposed bool + CredentialSigning string + SigningService string + SigningRegion string + JSONRPCMaxBodyBytes uint32 + Mcp *McpOptions + CredentialBinding *NetworkCredentialBinding } // NetworkCredentialBinding binds an endpoint to static credentials from an attached provider. diff --git a/sdk/go/proto/sandboxv1/sandbox.pb.go b/sdk/go/proto/sandboxv1/sandbox.pb.go index 48250ccc72..25ad8295ce 100644 --- a/sdk/go/proto/sandboxv1/sandbox.pb.go +++ b/sdk/go/proto/sandboxv1/sandbox.pb.go @@ -730,8 +730,15 @@ type NetworkEndpoint struct { // endpointless provider profile. Profiles that already define endpoints // continue to use those profile endpoints as their credential boundary. CredentialBinding *NetworkCredentialBinding `protobuf:"bytes,24,opt,name=credential_binding,json=credentialBinding,proto3" json:"credential_binding,omitempty"` - unknownFields protoimpl.UnknownFields - sizeCache protoimpl.SizeCache + // Explicitly permits credential-bearing traffic to use paths that OpenShell + // cannot inspect or rewrite. Defaults to false. This is a security-sensitive + // escape hatch and must be explicitly approved. + AllowUninspectedCredentials bool `protobuf:"varint,25,opt,name=allow_uninspected_credentials,json=allowUninspectedCredentials,proto3" json:"allow_uninspected_credentials,omitempty"` + // Internal gateway-derived marker indicating that this endpoint belongs to + // an attached credentialed provider. User-authored values are ignored. + ProviderCredentialed bool `protobuf:"varint,26,opt,name=provider_credentialed,json=providerCredentialed,proto3" json:"provider_credentialed,omitempty"` + unknownFields protoimpl.UnknownFields + sizeCache protoimpl.SizeCache } func (x *NetworkEndpoint) Reset() { @@ -932,6 +939,20 @@ func (x *NetworkEndpoint) GetCredentialBinding() *NetworkCredentialBinding { return nil } +func (x *NetworkEndpoint) GetAllowUninspectedCredentials() bool { + if x != nil { + return x.AllowUninspectedCredentials + } + return false +} + +func (x *NetworkEndpoint) GetProviderCredentialed() bool { + if x != nil { + return x.ProviderCredentialed + } + return false +} + // MCP options are grouped so MCP-specific policy can grow without adding more // top-level NetworkEndpoint fields. Current enforcement targets the active // 2025-11-25 Streamable HTTP/tools behavior, while preserving space for @@ -2080,7 +2101,8 @@ const file_sandbox_proto_rawDesc = "" + "\ainclude\x18\x01 \x03(\tR\ainclude\x12\x18\n" + "\aexclude\x18\x02 \x03(\tR\aexclude\"6\n" + "\x18NetworkCredentialBinding\x12\x1a\n" + - "\bprovider\x18\x01 \x01(\tR\bprovider\"\xe3\t\n" + + "\bprovider\x18\x01 \x01(\tR\bprovider\"\xdc\n" + + "\n" + "\x0fNetworkEndpoint\x12\x12\n" + "\x04host\x18\x01 \x01(\tR\x04host\x12\x12\n" + "\x04port\x18\x02 \x01(\rR\x04port\x12\x1a\n" + @@ -2108,7 +2130,9 @@ const file_sandbox_proto_rawDesc = "" + "\x0esigning_region\x18\x15 \x01(\tR\rsigningRegion\x124\n" + "\x17json_rpc_max_body_bytes\x18\x16 \x01(\rR\x13jsonRpcMaxBodyBytes\x122\n" + "\x03mcp\x18\x17 \x01(\v2 .openshell.sandbox.v1.McpOptionsR\x03mcp\x12]\n" + - "\x12credential_binding\x18\x18 \x01(\v2..openshell.sandbox.v1.NetworkCredentialBindingR\x11credentialBinding\x1ar\n" + + "\x12credential_binding\x18\x18 \x01(\v2..openshell.sandbox.v1.NetworkCredentialBindingR\x11credentialBinding\x12B\n" + + "\x1dallow_uninspected_credentials\x18\x19 \x01(\bR\x1ballowUninspectedCredentials\x123\n" + + "\x15provider_credentialed\x18\x1a \x01(\bR\x14providerCredentialed\x1ar\n" + "\x1cGraphqlPersistedQueriesEntry\x12\x10\n" + "\x03key\x18\x01 \x01(\tR\x03key\x12<\n" + "\x05value\x18\x02 \x01(\v2&.openshell.sandbox.v1.GraphqlOperationR\x05value:\x028\x01\"\xb6\x01\n" +