Skip to content

Add RBAC coverage for OpenSearch proxy endpoint - #1096

Merged
mmguero merged 2 commits into
cisagov:contributor-prsfrom
pavanchow:rbac-opensearch-proxy
Oct 9, 2026
Merged

mmguero merged 2 commits into
cisagov:contributor-prsfrom
pavanchow:rbac-opensearch-proxy

Conversation

@pavanchow

@pavanchow pavanchow commented Aug 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds role-based access control to the OpenSearch reverse proxy (/mapi/opensearch/...), splitting endpoints into two tiers:

  • Reader tier - _search, _msearch, _count, _field_caps, _validate/query - accessible to all authenticated users
  • Admin tier - everything else (_cluster, _snapshot, _cat, _bulk, writes, scroll/PIT management, etc.) - requires admin role

This addresses the gap identified in GHSA-jr6p-63pg-hr6g where, after the URI normalization fix, all OpenSearch endpoints still shared a single authorization level.

Changes

  • nginx/lua/nginx_auth_helpers.lua - Two new RBAC entries with PCRE2 patterns: a reader-tier allowlist anchored with $, and a catch-all admin-tier rule for everything under /mapi/opensearch
  • nginx/lua/test_opensearch_rbac.lua - 48-case regression test suite covering reader endpoints, admin endpoints, normalization bypass attempts (%3f-encoded ?, case folding, dot-dot traversal, encoded underscores), and neighbouring route isolation
  • .github/workflows/nginx-build-and-push-ghcr.yml - Wires the new test suite into CI alongside the existing test_normalize_uri.lua

Design decisions

  • The reader pattern is a strict allowlist anchored with $. normalize_uri_for_rbac() strips the query string before percent-decoding, so real query strings don't reach the pattern, and a %3f-encoded ? decodes into the path and fails the anchor.
  • The index slot [^/_][^/]* rejects a leading _, which keeps API paths such as /_cluster/_search out of the reader tier. It does not reject a leading ., so dot-prefixed system indices such as .kibana can be searched by readers (see Known limitations).
  • _all/ is explicitly allowed as an index position for multi-index search.
  • Scroll, PIT, and search template endpoints are admin-only even though they live under _search/, because they create or delete server-side state. A reader can still pass ?scroll= on a plain _search, which opens a scroll context that expires after its keepalive. The reader can't continue or clear it.
  • The test harness uses lrexlib's PCRE binding (rex_pcre2 or rex_pcre) so the patterns behave as they do under OpenResty's ngx.re.

Known limitations

  • _search and _msearch can still reach system indices like .kibana if the index name slot matches. Consider an additional deny rule for .* and _* prefixed indices if that's a concern.
  • Readers cannot continue a scroll (admin-only by design).
  • An unset ROLE_ADMIN fails open. This is existing behaviour, not introduced here.

CI note

The nginx-build-and-push-ghcr.yml workflow only triggers on pushes to main/staging and manual dispatch. It has no pull_request trigger, so CI will not run automatically on this PR. Adding pull_request to that workflow would also trigger image builds/pushes, so that's left to the maintainer.

Local test output (reproducible):

# Run both test suites
lua nginx/lua/test_normalize_uri.lua    # 90 passed, 0 failed
lua nginx/lua/test_opensearch_rbac.lua  # 48 passed, 0 failed

# Requires: luarocks install lua-rex-pcre2

Test plan

  • All 48 RBAC test cases pass locally
  • All 90 existing normalize_uri test cases still pass
  • CI passes with lua-rex-pcre2 dependency (requires manual dispatch or merge to main)
  • Manual verification: reader user can _search but gets 403 on _cluster/settings

@mmguero

mmguero commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Thanks for your contribution. Just FYI, it might be a week or two before I am able to review it. I appreciate your patience.

@pavanchow

Copy link
Copy Markdown
Contributor Author

No rush at all, take your time. Happy to address any feedback when you get to it.

@sylvesterkaczmarek

Copy link
Copy Markdown
Contributor

I traced this against the full check_rbac() path rather than only the table entry. /mapi/opensearch/ is the directly exposed nginx OpenSearch proxy, and the new prefix is matched against the normalized pre-rewrite request URI after percent-decoding, repeated-slash/traversal collapse and case folding.

Once the rule matches, ROLE_ADMIN is resolved through the same environment-role mechanism as the other protected routes, so the previous No role restrictions, access granted fallback is no longer reachable for this endpoint.

I also checked the surrounding route order: the new prefix does not capture the existing Dashboards or Arkime routes, and the normalization closes the obvious encoding, case and path-normalization bypasses. I don't see a blocking RBAC issue in the current change.

Given that this closes direct access to cluster-level APIs, an automated viewer-vs-admin regression would be useful if there is an existing nginx authentication test harness, but the runtime boundary itself looks correct.

@mmguero mmguero added nginx Relating to Malcolm's use of nginx security Related to issues with bearing on the security of Malcolm itself labels Sep 15, 2026
@mmguero mmguero added this to Malcolm Sep 15, 2026
@mmguero mmguero added this to the v26.10.0 milestone Sep 15, 2026
@mmguero mmguero moved this to Review in Malcolm Sep 15, 2026
@mmguero mmguero self-assigned this Oct 6, 2026
@mmguero

mmguero commented Oct 7, 2026

Copy link
Copy Markdown
Collaborator

Thanks for digging into this, yeah you're right about that gap. With ROLE_BASED_ACCESS=true, /mapi/opensearch doesn't match anything in path_role_envs, so it falls through to the default allow, and any authenticated user can reach cluster settings, snapshots, index deletion and mapping changes.

My hesitation is with making the whole endpoint admin-only. None of Malcolm's own components go through /mapi/opensearch/, so we're good there. Dashboards, Arkime and the API all talk to OpenSearch directly on the internal network, and sensors use the separate 9200 listener. The people who do use this path would be like analysts running searches against arkime_sessions3-* and malcolm_beats_* from curl, scripts or notebooks. A user with ROLE_READ_ACCESS or one of the Dashboards read roles can already see that data in Dashboards and Arkime. If we lock the passthrough to ROLE_ADMIN, those users lose direct query access to data they're already allowed to read.

I'd like to split this into two tiers instead. Read-only query APIs go to anyone who can already read data, and everything else stays admin-only. The issue is that path_role_envs matches on path, not HTTP method. So the reader tier can only include endpoints that are read-only no matter which method is used. That rules out _mapping, for example, since PUT /<index>/_mapping modifies the index.

What about something like this?

    -- OpenSearch proxy: read-only query APIs for data readers (method-independent endpoints only)
    { pattern = "^/mapi/opensearch/(?:[^/]+/)?_(?:search|msearch|count|field_caps|validate/query)(?:$|[/?])", roles = {
        "ROLE_ADMIN",
        "ROLE_DASHBOARDS_READ_ACCESS",
        "ROLE_DASHBOARDS_READ_ALL_APPS_ACCESS",
        "ROLE_DASHBOARDS_READ_WRITE_ACCESS",
        "ROLE_DASHBOARDS_READ_WRITE_ALL_APPS_ACCESS",
        "ROLE_READ_ACCESS",
        "ROLE_READ_WRITE_ACCESS"
    }},

    -- OpenSearch proxy: everything else (cluster, snapshot, index admin, writes)
    { pattern = "^/mapi/opensearch", roles = {
        "ROLE_ADMIN"
    }},

A few thoughts on placement and matching:

  • Both entries need to go above the Dashboards entry, with the more specific one first, because get_allowed_roles_for_path returns the first match.
  • The patterns are lowercase because check_rbac() matches against the normalized URI, which is percent-decoded and case-folded.
  • The (?:$|[/?]) tail allows query strings and _search/scroll while stopping a prefix like _searchfoo from matching.
  • The reader role list is the same one the Dashboards entry uses, so this doesn't give anyone query access they don't already have through Dashboards.

The only limitation we can't fix at this layer. _msearch puts its target indices in the request body, so a reader could point it at a system index such as .kibana that has no Dashboards index pattern. Closing that would need a method- and body-aware check in Lua. I'd rather handle that separately (if we decide we really need to) than hold up this fix for it.

Would you be up for reworking the PR along these lines?

@sylvesterkaczmarek

Copy link
Copy Markdown
Contributor

Thanks @mmguero, I agree with a reader tier for genuinely query-only endpoints and an admin-only fallback. One caution with the proposed PCRE: the trailing (?:$|[/?]) also authorizes every _search/... subpath for reader roles. That includes DELETE /_search/scroll/_all (clears open scroll contexts) and DELETE /_search/point_in_time/_all (deletes PIT contexts), while POST /<index>/_search/point_in_time creates PITs. These are not method-independent read-only routes. The OpenSearch docs spell out those methods: https://docs.opensearch.org/latest/api-reference/search-apis/scroll/ and https://docs.opensearch.org/latest/api-reference/search-apis/point-in-time-api/. I would anchor the allowlist at each complete query endpoint (and optional query string), without a trailing arbitrary slash/subpath; let the subsequent admin rule handle nested _search/* routes. Please add positive reader-query and negative reader-context-management cases before merging. I am not the PR author, so @pavanchow would need to apply the route split. The separate cross-index read scope concern you noted for _msearch is also worth tracking for unscoped _search.

@pavanchow

pavanchow commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor Author

Reworked the RBAC config with your two-tier design. Key change from your proposed regex:

Anchor at (?:$|[?]) instead of (?:$|[/?])

As @sylvesterkaczmarek pointed out, allowing / after the query endpoints would also match destructive subpaths:

  • DELETE _search/scroll/_all — clears all scroll contexts
  • DELETE _search/point_in_time/_all — clears all PIT contexts

These are cluster-wide destructive operations that should stay admin-only. The (?:$|[?]) anchor ensures reader-tier access only covers the exact query endpoint (with optional query string), and everything else falls through to the admin-only rule.

Reader-tier endpoints: _search, _msearch, _count, _field_caps, _validate/query

Reader role list matches the existing Dashboards entry — same roles that can already reach Dashboards can now query OpenSearch directly, which is what they need for saved searches and visualizations.


Update: Rebased onto current main (7cfac5b) and added test_opensearch_rbac.lua with 46 test cases covering reader-tier query endpoints, admin-tier scroll/PIT/cluster/write operations, URI normalization (uppercase, percent-encoding, doubled slashes, dot-dot traversal), and neighbouring route isolation. All 46 pass. Also tightened the index segment to [^/_][^/]*/ so paths like /_cluster/_search can't match.

@pavanchow
pavanchow force-pushed the rbac-opensearch-proxy branch from dce2716 to 861ac4b Compare October 9, 2026 13:53
Split /mapi/opensearch access into two tiers:
- Reader tier: _search, _msearch, _count, _field_caps, _validate
  operations allowed for all dashboard/read roles
- Admin tier: everything else (cluster health, snapshots, index
  management, bulk writes) restricted to ROLE_ADMIN only

Add 46-case Lua test suite covering reader access, admin-only
operations, URI normalization, and neighbouring-route isolation.
@pavanchow
pavanchow force-pushed the rbac-opensearch-proxy branch from 861ac4b to 6be4891 Compare October 9, 2026 13:59
- Remove dead [?] alternative from reader-tier pattern; anchor with $ only
- Add two %3f-encoded ? test cases to confirm admin-only classification
- Wire test_opensearch_rbac.lua into the nginx CI workflow
- Install lua-rex-pcre2 dependency for RBAC tests
@pavanchow

Copy link
Copy Markdown
Contributor Author

Follow-up to the rework: pushed a second commit (2bdceee) that tightens the reader regex further.

What changed:

  • Removed the [?] alternative entirely. The reader pattern now anchors with $ only, so query strings can't be used to extend the match past the endpoint name.
  • Added 2 test cases for %3f-encoded ? bypass attempts (both correctly blocked as admin-only).
  • Wired the RBAC test suite into the CI workflow alongside the existing test_normalize_uri.lua.

On @sylvesterkaczmarek's scroll/PIT concern: scroll endpoints (_search/scroll, _search/scroll/_all) and point-in-time endpoints (_search/point_in_time) are now explicitly tested as admin-only (test cases 25-27 in test_opensearch_rbac.lua). They fail to match the reader pattern because it anchors at $ with no / alternative.

CI note: the nginx-build-and-push-ghcr.yml workflow only triggers on pushes to main/staging and manual dispatch, so CI won't run on this PR automatically. Local results: 48/48 RBAC tests pass, 90/90 normalize tests pass. Repro commands are in the PR description.

Ready for review whenever you have time.

@mmguero

mmguero commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the rework, looks good. The $-only anchor is right since normalize_uri_for_rbac() strips the query string before decoding, and the scroll, PIT and template cases cover Sylvester's points.

One correction to the original "design decisions" bit of the description before merge, and I can make that myself if you don't mind, or you're welcome to. The index slot [^/_][^/]* blocks a leading _ but not a leading ., so /mapi/opensearch/.kibana/_search does match the reader tier. That's consistent with the _msearch and _all limitation already listed, so no code change is needed. Something like:

Design decisions

  • The reader pattern is a strict allowlist anchored with $. normalize_uri_for_rbac() strips the query string before percent-decoding, so real query strings don't reach the pattern, and a %3f-encoded ? decodes into the path and fails the anchor.
  • The index slot [^/_][^/]* rejects a leading _, which keeps API paths such as /_cluster/_search out of the reader tier. It does not reject a leading ., so dot-prefixed system indices such as .kibana can be searched by readers (see Known limitations).
  • _all/ is explicitly allowed as an index position for multi-index search.
  • Scroll, PIT, and search template endpoints are admin-only even though they live under _search/, because they create or delete server-side state. A reader can still pass ?scroll= on a plain _search, which opens a scroll context that expires after its keepalive. The reader can't continue or clear it.
  • The test harness uses lrexlib's PCRE binding (rex_pcre2 or rex_pcre) so the patterns behave as they do under OpenResty's ngx.re.

@pavanchow

Copy link
Copy Markdown
Contributor Author

Updated the Design decisions section with your wording — the index slot correction and the four points (query-string handling, index slot, scroll behaviour, lrexlib harness) are in. No code changes. Ready whenever you are.

@mmguero
mmguero changed the base branch from main to contributor-prs October 9, 2026 16:22
@mmguero
mmguero merged commit f6609a9 into cisagov:contributor-prs Oct 9, 2026
@mmguero mmguero mentioned this pull request Oct 29, 2026
@mmguero mmguero moved this from Done to Released in Malcolm Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

nginx Relating to Malcolm's use of nginx security Related to issues with bearing on the security of Malcolm itself

Projects

Status: Released

Development

Successfully merging this pull request may close these issues.

3 participants