feat: add reusable network access group APIs

api/.env.example

@@ -17,6 +17,13 @@ PYTHONIOENCODING=utf-8
 
 # Console API base URL
 CONSOLE_API_URL=http://localhost:5001
+
+# Optional internal SaaS policy API; defaults to BILLING_API_URL when empty.
+NETWORK_ACCESS_API_URL=
+# Current-IP preview only. Comma-separated CIDRs, e.g. 172.18.0.0/16.
+# Required for Cloud IP previews. Match Gateway's proxy boundary; never trust caller
+# headers unless the edge sanitizes XFF or appends its verified socket peer.
+NETWORK_ACCESS_TRUSTED_PROXY_CIDRS=
 CONSOLE_WEB_URL=http://localhost:3000
 
 # Service API base URL

api/.importlinter

@@ -171,6 +171,21 @@ forbidden_modules =
     sqlalchemy
     werkzeug
 
+[importlinter:contract:network-access-group-service-boundary]
+name = Network access group application service is framework and persistence neutral
+type = forbidden
+source_modules =
+    services.network_access_group_service
+forbidden_modules =
+    configs
+    controllers
+    extensions
+    flask
+    models
+    repositories
+    sqlalchemy
+    werkzeug
+
 [importlinter:contract:dataset-application-boundary]
 name = Dataset application services and contracts are framework and persistence neutral
 type = forbidden

api/configs/feature/__init__.py

@@ -426,6 +426,19 @@ class EndpointConfig(BaseSettings):
 
     TRIGGER_URL: str = Field(description="Template url for triggers", default="http://localhost:5001")
 
+    NETWORK_ACCESS_API_URL: str = Field(
+        description="Optional internal SaaS API base URL for network access policy requests; "
+        "falls back to BILLING_API_URL",
+        default="",
+    )
+
+    NETWORK_ACCESS_TRUSTED_PROXY_CIDRS: str = Field(
+        description="Comma-separated trusted proxy CIDRs for network-access current-IP previews. "
+        "Only verified socket peers may supply X-Forwarded-For; empty disables Cloud IP previews. "
+        "Match the Gateway trust boundary and sanitize or append the client IP at the edge.",
+        default="",
+    )
+
 
 class FileAccessConfig(BaseSettings):
     """

api/controllers/API_SCHEMA_GUIDE.md

@@ -224,6 +224,95 @@ return dump_response(WorkflowRunDetailResponse, workflow_run)
 Use manual `model_validate(...).model_dump(...)` only when the endpoint needs behavior that `dump_response(...)` does
 not provide, such as returning a non-dict payload, intentionally excluding fields, or composing a `(body, status)` tuple.
 
+## Public access-error contracts
+
+IP-policy denials must be indistinguishable from each surface's native
+authentication/not-found errors. The Gateway terminates denied requests before
+execution; Core owns the corresponding native error baselines. Keep their JSON
+bytes (sorted keys, compact separators, no trailing newline), status and
+application-owned headers aligned. Do not add policy names, internal reasons,
+`ip_access_denied`, or a policy-specific response header.
+
+| Surface | Error contract |
+| --- | --- |
+| Service API | 401 `unauthorized`, `Access token is invalid`; missing/malformed Bearer uses `Authorization header must be provided and start with 'Bearer'` |
+| OAuth OpenAPI | Its own 401 `unauthorized` / `invalid_token`, not the Service API's token message |
+| MCP | 404 JSON-RPC `-32600` / `Server Not Found`, echoing a string/number request ID or null |
+| Workflow Webhook | 404 standard Werkzeug `NotFound()` HTML without capability ID/private detail; not JSON |
+| Plugin callback | 404 JSON `{"error":"Endpoint not found"}`; not the Workflow Webhook HTML |
+| WebApp | Canonical 404 `app_not_found` / `App not found.`, with optional trusted `client_ip` |
+| HITL form / upload | Native form 404 / invalid-upload-token 403, not a WebApp error page |
+
+These errors use `Cache-Control: no-store`. JSON uses `Content-Type:
+application/json`; Workflow Webhook uses `text/html; charset=utf-8`. Service API
+401 also includes `WWW-Authenticate: Bearer realm="api"`; OpenAPI preserves its
+native challenge/extra fields. For callers: if an API key is confirmed correct,
+contact the App administrator to check whether the source IP is allowed.
+
+MCP resolves unavailable identity before validating an invocation. Its error-only
+ID parser accepts at most 64 KiB of strict UTF-8 JSON, nesting depth 64; absent,
+invalid or oversized IDs become null. Numeric lexemes retain precision. Allowed
+requests retain their existing body and protocol-validation path. Do not create
+EndUsers or execute tools just to construct a not-found response.
+
+MCP identity preflight uses a short read session that ends before body reads or
+parsing. Normal execution rechecks the same server/App/tenant identity in its
+own transaction and uses fresh ORM objects; deletion, disabling or rebinding
+during parsing must not execute against the stale identity. Construct opaque
+404 responses outside both sessions. The error ID reader reuses existing raw
+Werkzeug body-cache bytes without changing that cache, and otherwise reads at
+most the limit plus one sentinel byte; it must not use unbounded `get_data()`
+for unknown-length bodies. This error-only limit does not cap valid MCP bodies.
+
+Real dependency failures stay fail-closed errors, not a missing-policy allow.
+The Gateway can present an unresolved canonical WebApp reference as opaque404
+while still terminating the request; it must never forward it because the
+read-side miss could be replication lag. Internal audit/metrics retain the true
+policy-denied or resolution-failure reason even though the public response does
+not disclose it.
+
+### WebApp and Console 404 metadata
+
+The Web and Console blueprints add optional request-local `client_ip` metadata
+to their JSON app-not-found responses. This is transport error metadata,
+not an App property, a new endpoint, or a field in successful response DTOs.
+
+- Public WebApp errors use only `code: app_not_found`, `message: App not found.`,
+  `status: 404` and optional `client_ip`. Strip exception-specific details that
+  would reveal whether an App exists. Authenticated Console management errors
+  preserve their original code/message/extra fields.
+- A typed `app_not_found` 404 is eligible. Generic 404s are eligible only for
+  matched GET app-identity routes: Web bootstrap (`site`, `parameters`, `meta`,
+  `passport`, `login/status`, `webapp/access-mode`) and Console App/Agent,
+  installed-App detail/parameters/meta, and trial-App detail/parameters.
+- The field is a single normalized IP resolved using the configured trusted
+  proxy boundary. Do not accept a caller's query/body IP or use unverified
+  forwarding headers. If trust configuration or resolution is unavailable,
+  omit `client_ip` instead of guessing a proxy address or changing the 404 to 503.
+- These eligible errors use `Cache-Control: no-store`, because their IP metadata
+  must not be shared between callers through URL-based error caching.
+- Known unavailable/unpublished WebApp identity errors use this same404; arbitrary
+  400s, auth errors and dependency503s do not. Public login/status and access-mode
+  must validate a supplied App reference even when user authentication is disabled.
+- HITL keeps its own canonical form/token error; no App IP is attached. Ordinary
+  resource404s, unmatched routes, streams and Inner API contracts are unchanged.
+  In particular, an authenticated Console trial403 is not converted to404.
+
+For example, either an unavailable Web passport or a Gateway WebApp denial is:
+
+```json
+{
+  "code": "app_not_found",
+  "message": "App not found.",
+  "status": 404,
+  "client_ip": "203.0.113.42"
+}
+```
+
+The existing frontend error transport already accepts optional `client_ip`;
+success OpenAPI/TypeScript/Zod/oRPC contracts do not change for this enrichment.
+Add coverage at the final serialized-response boundary, including exceptions
+with their own `data`, so Flask-RESTX cannot silently discard the metadata.
 ## Error Responses
 
 Every 4xx/5xx response is documented with the surface's error body: `ErrorResponse` (`code`, `message`, `status`,

api/controllers/common/access_response.py

@@ -0,0 +1,152 @@
+"""Native, non-cacheable access errors shared with the public gateway.
+
+Keep these byte-level contracts local to their owning surfaces: Service API and
+OpenAPI authentication, unavailable MCP identities, and missing trigger endpoints.
+Successful responses and other APIs retain their existing serialization.
+"""
+
+import json
+from dataclasses import dataclass
+from typing import Any
+
+from flask import Blueprint, Response, request
+from werkzeug.exceptions import HTTPException, NotFound
+from werkzeug.wrappers import Response as WerkzeugResponse
+
+MCP_ERROR_BODY_LIMIT = 64 * 1024
+MCP_ERROR_MAX_DEPTH = 64
+MISSING_BEARER_MESSAGE = "Authorization header must be provided and start with 'Bearer'"
+
+
+def _json(value: Any) -> str:
+    # Match Go's non-HTML-escaping JSON encoder, including its JS-safe separators.
+    return (
+        json.dumps(value, ensure_ascii=False, separators=(",", ":"), sort_keys=True, allow_nan=False)
+        .replace("\u2028", "\\u2028")
+        .replace("\u2029", "\\u2029")
+    )
+
+
+def register_auth_error_response(blueprint: Blueprint, *, www_authenticate: str | None = None) -> None:
+    """Canonicalize local final 401 bodies, preserving native message/extra fields.
+
+    The Service API supplies its Bearer challenge explicitly. OpenAPI retains its
+    own challenge and OAuth error fields; other surfaces do not register this hook.
+    """
+
+    @blueprint.after_request
+    def auth_error_response(response: Response) -> Response:
+        if response.status_code != 401 or not response.is_json or response.is_streamed:
+            return response
+        payload = response.get_json(silent=True)
+        if not isinstance(payload, dict):
+            return response
+        response.set_data(_json(payload))
+        response.headers["Content-Type"] = "application/json"
+        response.headers["Cache-Control"] = "no-store"
+        if www_authenticate is not None:
+            response.headers["WWW-Authenticate"] = www_authenticate
+        return response
+
+
+@dataclass(frozen=True)
+class _JSONNumber:
+    """An already-validated JSON number lexeme, without float/int conversion."""
+
+    value: str
+
+
+def _within_json_depth(body: bytes) -> bool:
+    """Match the gateway's bounded parser; brackets inside strings do not count."""
+    depth = 0
+    in_string = False
+    escaped = False
+    for char in body:
+        if in_string:
+            if escaped:
+                escaped = False
+            elif char == 92:  # backslash
+                escaped = True
+            elif char == 34:  # quote
+                in_string = False
+        elif char == 34:
+            in_string = True
+        elif char in (91, 123):  # array or object
+            depth += 1
+            if depth > MCP_ERROR_MAX_DEPTH:
+                return False
+        elif char in (93, 125):
+            depth -= 1
+    return True
+
+
+def _invalid_constant(_value: str) -> None:
+    raise ValueError("Non-JSON numeric constant")
+
+
+def _parse_mcp_error_request_id(body: bytes) -> str:
+    """Parse only an error response's ID, retaining its original numeric lexeme."""
+    try:
+        if len(body) > MCP_ERROR_BODY_LIMIT or not _within_json_depth(body):
+            return "null"
+        payload = json.loads(
+            body.decode("utf-8"),
+            parse_int=_JSONNumber,
+            parse_float=_JSONNumber,
+            parse_constant=_invalid_constant,
+        )
+        if not isinstance(payload, dict):
+            return "null"
+        request_id = payload.get("id")
+        if isinstance(request_id, _JSONNumber):
+            return request_id.value
+        if isinstance(request_id, str):
+            # Reject unpaired surrogates instead of creating invalid UTF-8.
+            request_id.encode("utf-8")
+            return _json(request_id)
+    except (ValueError, UnicodeError, RecursionError):
+        pass
+    return "null"
+
+
+def _mcp_request_id() -> str:
+    """Read bounded raw bytes, including bodies already cached by request logging.
+
+    Werkzeug's public get_data() reuses _cached_data but otherwise reads the
+    entire stream. Its stream property does not replay that cache. Keep this
+    read-only private-attribute compatibility here: it avoids unbounded reads
+    for unknown Content-Length and never installs a partial request cache.
+    """
+    try:
+        if request.content_length is not None and request.content_length > MCP_ERROR_BODY_LIMIT:
+            return "null"
+        cached = vars(request).get("_cached_data")
+        if cached is not None:
+            return _parse_mcp_error_request_id(cached) if isinstance(cached, bytes) else "null"
+        body = request.stream.read(MCP_ERROR_BODY_LIMIT + 1)
+    except HTTPException:
+        return "null"
+    return _parse_mcp_error_request_id(body)
+
+
+def mcp_server_not_found_response() -> Response:
+    """Opaque MCP lookup failure; no app/server identity or policy is disclosed."""
+    body = '{"error":{"code":-32600,"message":"Server Not Found"},"id":' + _mcp_request_id() + ',"jsonrpc":"2.0"}'
+    return Response(body, status=404, content_type="application/json", headers={"Cache-Control": "no-store"})
+
+
+def webhook_not_found_response() -> WerkzeugResponse:
+    """Use Werkzeug's normal HTML 404 without caller IDs or internal detail."""
+    response = NotFound().get_response()
+    response.headers["Cache-Control"] = "no-store"
+    return response
+
+
+def plugin_endpoint_not_found_response() -> Response:
+    """Plugin callbacks own a JSON not-found shape, unlike native Webhooks."""
+    return Response(
+        _json({"error": "Endpoint not found"}),
+        status=404,
+        content_type="application/json",
+        headers={"Cache-Control": "no-store"},
+    )