Skip to content

Add a generic, extensible rest-endpoint provider SPI - #5656

Open
noCharger wants to merge 24 commits into
opensearch-project:mainfrom
noCharger:feature/ppl-rest-framework
Open

Add a generic, extensible rest-endpoint provider SPI#5656
noCharger wants to merge 24 commits into
opensearch-project:mainfrom
noCharger:feature/ppl-rest-framework

Conversation

@noCharger

@noCharger noCharger commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Description

Rebuild of the reverted PPL rest command (#5599, reverted in #5635) as a separated, generic, extensible framework. rest <name> exposes read-only, fixed-schema management endpoints as a relational row source behind one operator allow-list, so results feed straight into where / stats / head.

Key design points

  • Generic endpoint SPI. A thin new ppl-rest-spi module exposes RestEndpointProvider (an ExtensiblePlugin loadExtensions SPI) plus RestEndpointDefinition / RestEndpointHandler / RestEndpointContext and Column / ColumnType / ArgSpec. New endpoints are registrations, not new grammar keywords — the language stays a single generic rest <name> command.
  • Built-in is a uniform SPI client. /_cluster/health ships via CoreEndpointsProvider, which fetches through the SPI context's transport node client (ctx.client()) exactly like an external plugin — there is no privileged in-repo path. ColumnType keeps the SPI module free of any dependency on the sql engine's ExprType.
  • Allow-list choke point. plugins.ppl.rest.allowed_endpoints is the single resolve() gate; it defaults to ["/_cluster/health"] (health on, every other endpoint off until an operator opts in). Health stays gated by the cluster's own cluster:monitor/health permission.
  • Lazy relational scan. The handler runs at scan execution, never at planning, so EXPLAIN is side-effect free and where / head / stats compose and push down. RestRequest mirrors the system-index OpenSearchSystemIndexScan behind one shared catalog enumerator.
  • Central redaction seam. Rows pass through a single Redactor ((endpoint, row) -> row) at the choke point before coercion; the default is Redactor.NONE (no-op). Redaction lives at the platform choke point rather than in the endpoint SPI.
  • Scope. This PR adds the framework plus the /_cluster/health reference endpoint; further endpoints are left to follow-up PRs.

Related Issues

Rebuild of #5599 (reverted in #5635) as a separated, extensible framework.

How it works: handler + ctx flow (bootstrap to fetch)

Each endpoint's handler is captured into its Endpoint at bootstrap and travels bundled inside it; the per-query ctx (args + transport client) is assembled at search(); the two meet at Endpoint.toRows -> handler.fetch(ctx). The registry and redactor cross the bootstrap-to-query Guice gap through static holders.

flowchart TB
  subgraph BOOT["Phase 1 · Plugin bootstrap (once per node; loadExtensions runs before the Guice injector)"]
    direction TB
    prov["RestEndpointProvider SPI<br/>CoreEndpointsProvider.getEndpoints()<br/>→ RestEndpointDefinition(name, schema, argSpec, handler)"]
    load["SQLPlugin.loadExtensions<br/>loader.loadExtensions(RestEndpointProvider.class)"]
    build["publishRestCommandRegistries()<br/>new RestEndpointRegistry(providers)<br/>each Definition → new Endpoint(def)<br/>★ Endpoint.handler = def.handler()"]
    hold["RestEndpointRegistryHolder.set(registry)<br/>RedactorHolder.set(Redactor.NONE)"]
    prov -->|discovered| load
    load -->|providers| build
    build -->|publish| hold
  end
  subgraph QUERY["Phase 2 · Query execution (per query: rest '/_cluster/health' ...)"]
    direction TB
    gt["OpenSearchStorageEngine.restTable()<br/>registry = RestEndpointRegistryHolder.get()<br/>redactor = RedactorHolder.get()"]
    src["new RestCatalogSource(registry, spec, client, redactor)<br/>★ endpoint = registry.resolve(spec.endpoint) «allow-list gate»"]
    scan["CalciteEnumerableCatalogScan<br/>source.createRequest()"]
    req["RestCatalogSource.createRequest()<br/>→ new RestRequest(client, endpoint, spec, redactor)"]
    en["OpenSearchCatalogEnumerator<br/>request.search()"]
    se["RestRequest.search()<br/>★ ctx = RestEndpointContext.of(spec.args, nodeClient)<br/>endpoint.toRows(ctx, redactor)"]
    tr["Endpoint.toRows(ctx, redactor)<br/>★★ handler.fetch(ctx) «handler + ctx meet»<br/>→ redactor.redact(endpoint, raw) → coerce → rows"]
    gt --> src
    src --> scan
    scan --> req
    req --> en
    en --> se
    se --> tr
  end
  hold -.->|"registry / redactor (static bridge: bootstrap → query)"| gt
  build ==>|"handler travels bundled inside Endpoint"| tr
Loading

Testing

  • Unit tests green on JDK 21: rest endpoint registry, catalog source, cross-plugin extensibility, the centralized Redactor seam, storage engine, and settings.
  • CalcitePPLRestIT and RestCommandSecurityIT cover /_cluster/health plus the allow-list and per-arg gates.

Check List

  • New functionality includes testing.
  • New functionality has been documented.
    • New functionality has javadoc added.
    • New functionality has a user manual doc added (docs/user/ppl/cmd/rest.md).
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

noCharger added 14 commits June 30, 2026 00:54
Add a leading `rest <endpoint>` command that exposes a curated, read-only,
fixed-schema set of in-cluster management endpoints as a PPL table, modeled as
a system row source bridged through visitRelation (the same seam as describe
and the system-index family), so it runs on the Calcite engine without the
unsupported table-function path.

- Grammar/AST: REST/TIMEOUT tokens, restCommand rule, RestRelation, AstBuilder
  visit, query anonymizer.
- Execution: RestSourceTable -> CalciteLogicalRestScan / CalciteEnumerableRestScan;
  RestEndpointRegistry (read-only allow-list + fixed schema + accepted args);
  RestEnumerator/RestRequest dispatch via OpenSearchClient (NodeClient in-cluster,
  RestClient standalone).
- 9 endpoints: cluster health/state/settings, cat indices/nodes/cluster_manager/
  plugins/shards, resolve/index.
- Output shaping: numeric type normalization, id-to-name resolution, role-name
  expansion, structural flattening, graceful null degrade.
- Args: count caps emitted rows; timeout reserved but rejected with 400; get-args
  applied server-side with per-arg value validation (local on health, health on
  cat/indices, expand_wildcards on resolve/index). Undeclared arg or out-of-domain
  value is rejected with a 400. level and include_defaults are deferred to a later
  release; flat_settings is dropped as redundant.
- Error handling: blank endpoint, negative count, disallowed arg, and uncoercible
  value all surface clean 400s rather than 500s.

Tests: CalcitePPLRestIT 25/25, RestEndpointRegistryTest 16, RestSourceTableTest 10.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
…xes; comment cleanup

- /_cluster/settings: run the persistent and transient tiers through the node SettingsFilter
  (published via RestSettingsFilterHolder from SQLPlugin#getRestHandlers) so Property.Filtered
  and plugin-registered pattern settings are redacted exactly as the native
  GET /_cluster/settings endpoint. Remove the dead secretFields column-filter, which was the
  wrong shape for the (setting, value, tier) rows.
- Parser: add TIMEOUT to searchableKeyWord so a bare 'timeout' term still matches searchLiteral.
- coerce(): narrow the catch to IllegalArgumentException | ClassCastException; add empty-string
  guards in toNumber/toBoolean.
- spotlessApply formatting; drop outdated and redundant comments.

Tests: RestEndpointRegistryTest, RestSourceTableTest, OpenSearchNodeClientClusterSettingsFilterTest green.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
- clusterSettings: fail closed (throw IllegalStateException) when the node
  SettingsFilter is unavailable, instead of returning unredacted settings
- collectSettings: handle list-type settings via getAsList fallback
- decodeRestSpec: reject a blank/missing endpoint with a clear error
- docs: correct rest.md allow-list table (9 endpoints + accepted args),
  quote endpoint literals, fix timeout/get-arg descriptions, add security note
- register docs/user/ppl/cmd/rest.md in docs/category.json (deterministic
  single-node examples: number_of_nodes=1, cluster_manager count=1)

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
decodeRestSpec is only ever called behind an isRestSource gate today, but as
a public decoder it must not assume its precondition. Without the guard a
malformed token would surface an opaque StringIndexOutOfBoundsException from
substring; now it throws a clear IllegalArgumentException instead. Addresses
the PR opensearch-project#5599 Code Suggestions finding (importance 8).

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
…fetch

- CalciteExplainIT.explainRestCommand: single-quote the endpoint literal; the
  explain harness inlines the query into a JSON body without escaping, so a
  double-quoted literal produced an invalid payload and a 400 (integration CI
  failure).
- OpenSearchNodeClient.clusterSettings: resolve the SettingsFilter and fail
  closed BEFORE fetching cluster state, so settings are never read into memory
  when the redaction filter is unavailable.
- OpenSearchRestClient.clusterState: narrow filter_path to nodes.*.name so node
  IPs/attributes are not over-fetched; manager-name resolution is preserved.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
… decode

- AnalyticsEngineCompatIT: assert | rest '/_cluster/health' behaves identically
  with the analytics engine enabled (rest is never routed to DataFusion).
- SystemIndexUtils.fromHex: reject an odd-length hex body so a crafted source name
  that passes the isRestSource suffix check fails clearly rather than silently
  dropping the trailing half-byte.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
On a cluster started with cluster.pluggable.dataformat=composite,
RestUnifiedQueryAction.isAnalyticsIndex() routed every non-system-catalog
PPL query to the analytics engine. The rest command's reserved in-cluster
source (REST...__REST_SOURCE) has no backing index and only resolves on the
Calcite path, so it was routed to DataFusion and failed with
"Table 'REST...__REST_SOURCE' not found".

Fix: exclude isRestSource(name) alongside isSystemCatalog(name) so a rest
source falls back to the default (Calcite) pipeline and is never routed to
the analytics engine.

- RestUnifiedQueryActionTest: unit repro under cluster-composite.
- integ-test analyticsEngineCompat testcluster set composite-default so the
  existing rest coexistence IT exercises this routing exclusion.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
Collapse the two near-duplicate Calcite scan hierarchies -- SHOW/DESCRIBE system
tables and the rest command -- into one generic OpenSearchCatalogTable whose
per-endpoint behavior is supplied by a pluggable CatalogSource.

- OpenSearchSystemIndex + RestSourceTable -> one OpenSearchCatalogTable backed by
  SystemIndexCatalogSource / RestCatalogSource.
- Two Abstract/Logical/Enumerable scans + two enumerators + two converter rules ->
  AbstractCalciteCatalogScan / CalciteLogicalCatalogScan / CalciteEnumerableCatalogScan
  (+ rest-only CalciteScannableCatalogScan) / OpenSearchCatalogEnumerator /
  EnumerableCatalogScanRule.
- Concerns stay per-source: system tables keep the real V2 implement() path; rest is
  Calcite-only (implement() throws) and opts into Scannable for the collect short-circuit.

No behavior change: dispatch, schemas, V2 support, and the Scannable marker are
preserved; this is pure de-duplication of the Calcite scan plumbing.

Verified: opensearch compileJava/compileTestJava green; ppl and integ-test test-compile
green; affected unit tests pass.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
Two dynamic cluster settings gate the rest command per deployment:
- plugins.ppl.rest.redaction.enabled (default false): mask network
  identifiers in _cat/* cells and availability-zone names in
  _cluster/settings values.
- plugins.ppl.rest.allowed_endpoints (default all): restrict which
  endpoints are served; an empty list disables the rest command.

Both default to open-source parity: all endpoints served, no masking.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
The shared language-grammar carried REST and TIMEOUT lexer tokens and the
restCommand/restArgument parser rules with no consumer: async-query-core
has no rest visitor, and the rest command grammar lives in the ppl module.
Remove the dead rules.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
Verify the rest command is subject to the security plugin fine grained access control: a caller without cluster:monitor privilege is denied the cat and cluster endpoints, a caller holding the privilege can run them, and the resolve index endpoint is filtered to the caller authorized indices. Test indices are created idempotently, and denials are asserted by the security denial reason in the response body because a denied action on the Calcite only rest path surfaces as a wrapped error carrying that reason. Calcite fallback is disabled so the denial reason is not replaced by an unsupported command error.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
plugins.ppl.rest.redaction.enabled and plugins.ppl.rest.allowed_endpoints were dynamic cluster settings, so they could be changed at runtime through _cluster/settings or the _plugins/_query/settings endpoint, which could override an operator-configured value. Drop Setting.Property.Dynamic so both are node-level settings set in the node config; the engine rejects runtime updates on both paths. Register them without an update consumer and read the node-configured value.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
Flip plugins.ppl.rest.allowed_endpoints default from ["*"] to empty so
open source ships with the rest command closed. Deployments opt specific
endpoints in via the setting (AOS enables the ones it supports; AOSS
leaves it empty and stays disabled). Enforcement already treats an empty
or missing list as disabled. Opt the integration-test clusters into all
endpoints so the rest ITs still exercise the enabled path.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
@noCharger
noCharger force-pushed the feature/ppl-rest-framework branch from 8bb45cd to 2c6901d Compare July 28, 2026 03:57
…ework

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>

# Conflicts:
#	ppl/src/main/antlr/OpenSearchPPLLexer.g4
#	ppl/src/main/antlr/OpenSearchPPLParser.g4
#	ppl/src/main/java/org/opensearch/sql/ppl/parser/AstBuilder.java
- ArgSpec: drop the unused csv-arg path (csvArg builder, csvArgs field, csv branch); no endpoint declares a csv arg (leftover from the deferred _cat endpoints).

- RedactionRegistry: drop unused isEmpty() (consumers call mask()).

- RestRequest: drop the unused 3-arg convenience ctor; fix a stale comment left by the ctx.client() refactor.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 5ed738d

@noCharger noCharger self-assigned this Jul 28, 2026
@noCharger noCharger added feature calcite calcite migration releated v3.9.0 skip-diff-analyzer Maintainer to skip code-diff-analyzer check, after reviewing issues in AI analysis. labels Jul 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 84f5773

…e centralized Redactor

The framework now exposes one Redactor seam ((endpoint, row) -> row, with a NONE no-op) published through RedactorHolder and applied once per raw row at the rest choke point before coercion; the default is NONE. Removes RedactionClass, RedactionRegistry, RedactionRegistryHolder and the per-column redaction tag on Column, so the framework stays field- and facet-agnostic, one coordinated masking pass avoids independent maskers interfering on the same value, and redaction has a single central owner.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
@noCharger
noCharger force-pushed the feature/ppl-rest-framework branch from 84f5773 to 7637f4d Compare July 28, 2026 08:24
@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 7637f4d

…mmand index

Shorten the ppl-rest-spi javadoc and the visitRestCommand comment to a concise, consistent style, correct the rest.md endpoint note, and list the rest command as 3.9 (experimental) in the PPL command index.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 7b63421

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 65d3f86

…tartup

If two providers register the same endpoint name, keep the first registration and log a warning for the duplicate instead of throwing during createComponents. This bounds a naming collision to the offending endpoint rather than failing plugin/node bootstrap, and the first registration still wins so a later provider cannot replace it.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 8d371a8

The doctest, integTest, and RestCommandSecurityIT clusters allow-listed endpoints that are not registered in this version; only /_cluster/health exists (and is the default). Trim each to /_cluster/health and correct the comments, which wrongly said the command is disabled by default.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 6a4e66b

…ssage

Reword the resolve() error messages to "Only the following endpoints are supported: <list>", enumerating the registered allow-list (currently /_cluster/health) instead of promising a vaguer "read-only in-cluster endpoints" category. The "is not allow-listed" clause is kept so callers (and the IT assertion) still see that phrase.

Signed-off-by: Louis Chu <lingzhichu.clz@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 0ff4abc

@noCharger noCharger moved this from Todo to In progress in PPL 2026 Roadmap Jul 28, 2026
Comment thread core/src/main/java/org/opensearch/sql/utils/SystemIndexUtils.java
}
registry.resolve(spec.getEndpoint());
List<String> allowed = settings.getSettingValue(Settings.Key.PPL_REST_ALLOWED_ENDPOINTS);
if (allowed == null || !(allowed.contains("*") || allowed.contains(spec.getEndpoint()))) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should "*" apply to extension-provided endpoints? With the extensible registry, a cluster configured with ["*"] will automatically enable endpoints added by a newly installed or upgraded RestEndpointProvider, without an explicit allow-list change. This seems to weaken the endpoint-by-endpoint opt-in and security-review boundary. Could "*" be limited to built-in endpoints, with external endpoints requiring explicit names?

Comment on lines +62 to +66
LOG.warn(
"rest endpoint [{}] is already registered; ignoring the duplicate from provider [{}]",
definition.name(),
provider.getClass().getName());
continue;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Endpoint names are global, while the operator allow-list identifies only the endpoint name and not the contributing provider. If two external providers register the same name, the current first-wins behavior binds that allow-list entry to whichever provider is loaded first. Was this ambiguity considered? Would it be safer to disable only the conflicting endpoint, while allowing the node and the remaining endpoints to start normally?

Comment thread ppl-rest-spi/README.md
Comment on lines +3 to +6
The extension point for the PPL `rest` command. Any OpenSearch plugin can contribute read-only,
fixed-schema management endpoints that become queryable as `rest '<name>'`, without touching the
PPL grammar and without depending on the sql plugin's internals. The built-in `/_cluster/health`
endpoint is just one client of this same contract.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could u make the statement more generic. It is not just extension point for the PPL. This solution provide a generic a way to let other plugin to integrate PPL without introduce new command.

The built-in /_cluster/health endpoint is just one client of this same contract. this is just example use case. mention it in example section.

Comment thread ppl-rest-spi/README.md
`loadExtensions` discovers your provider:

```gradle
dependencies { compileOnly project(':ppl-rest-spi') } // or the published artifact

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be clear step by step guide.
Other plugins need published artifact, right?

Comment thread ppl-rest-spi/README.md

## Rules and guarantees

- Endpoints are read-only. The handler produces rows; the sql side coerces each value to the

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sql -> ppl

Comment thread ppl-rest-spi/README.md

- Endpoints are read-only. The handler produces rows; the sql side coerces each value to the
declared `ColumnType` and rejects an uncoercible value with a clear error.
- Endpoint names are global. If two providers register the same name, the first registration wins

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case all the endpoint is regestered at plugins.ppl.rest.allowed_endpoints. It should no conflict, right?

Comment thread ppl-rest-spi/README.md
and the duplicate is ignored with a logged warning.
- An endpoint must be both registered by a provider and present in `allowed_endpoints`; anything
else is rejected before any transport call.
- Redaction of sensitive values is applied centrally at the row-shaping choke point; a provider

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this mean?

Comment thread core/src/main/java/org/opensearch/sql/utils/SystemIndexUtils.java
Comment thread integ-test/build.gradle
Comment on lines +426 to +427
// Composite-default cluster: PPL queries route to the analytics engine unless excluded.
setting 'cluster.pluggable.dataformat', 'composite'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR should not releated to analytics engine?

Comment on lines +60 to +61
throw new IllegalStateException(
"the rest command endpoint registry is not initialized on this node");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not good user facing error.


private Table restTable(String name) {
RestSpec spec = decodeRestSpec(name);
RestEndpointRegistry registry = RestEndpointRegistryHolder.get();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Why use static class? It is hard to mock and test. Does singleton better?

Comment on lines +44 to +54
List.of(
Column.of("cluster_name", STRING),
Column.of("status", STRING),
Column.of("number_of_nodes", INTEGER),
Column.of("number_of_data_nodes", INTEGER),
Column.of("active_primary_shards", INTEGER),
Column.of("active_shards", INTEGER),
Column.of("relocating_shards", INTEGER),
Column.of("initializing_shards", INTEGER),
Column.of("unassigned_shards", INTEGER),
Column.of("timed_out", BOOLEAN)))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may not required to expose the schema. single string column should be fine. Then use spath command to extract.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

calcite calcite migration releated feature skip-diff-analyzer Maintainer to skip code-diff-analyzer check, after reviewing issues in AI analysis. v3.9.0

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

4 participants