[aws.cloudtrail] Fix ARN partition grok pattern for aws-cn/aws-iso* (7.3.x backport) by efd6 · Pull Request #21180 · elastic/integrations

efd6 · 2026-09-10T20:35:23Z

Backport of the fix from #20403 to the 7.3.x line.

The grok processor that extracts the session name from aws.cloudtrail.user_identity.arn used the pattern arn:(aws|aws-us-gov), which silently dropped aws-cn and aws-iso* partition ARNs, leaving user.changes.name unpopulated. This widens the pattern to arn:[a-z0-9-]+.

…d aws-iso* The grok pattern used to extract the session name from aws.cloudtrail.user_identity.arn matched only arn:aws:... and arn:aws-us-gov:... partitions. Identities from aws-cn and aws-iso* partitions were silently ignored, leaving user.changes.name unpopulated. Widening the character class from (aws|aws-us-gov) to [a-z0-9-] covers all current and future AWS partition strings without requiring updates for new partition names. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

github-actions · 2026-09-10T20:36:33Z

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

elastic-vault-github-plugin-prod · 2026-09-10T20:43:07Z

✅ All changelog entries have the correct PR link.

elastic-vault-github-plugin-prod · 2026-09-10T20:43:23Z

✅ Package owners are in sync with main.

infra-vault-gh-plugin-prod · 2026-09-10T20:43:47Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

vera-review-bot · 2026-09-10T20:51:55Z

      field: aws.cloudtrail.user_identity.arn
      patterns:
-        - "arn:(aws|aws-us-gov):sts:.*/%{GREEDYDATA:_tmp.session_name}$"
+        - "arn:[a-z0-9-]+:sts:.*/%{GREEDYDATA:_tmp.session_name}$"


Severity: 🟡 Medium confidence: high path: packages/aws/data_stream/cloudtrail/elasticsearch/ingest_pipeline/default.yml:1096

Backported grok pattern drops the leading ^ anchor that the main-branch fix (PR 20403) carries; restore it so the 7.3.x pipeline matches main.

Details

This PR describes itself as a backport of the ARN partition change from PR 20403. The diff of PR 20403 replaces the old arn:(aws|aws-us-gov):sts:... pattern with '^arn:[a-z0-9-]+:sts:.*/%{GREEDYDATA:_tmp.session_name}$', i.e. start-anchored. This backport lands "arn:[a-z0-9-]+:sts:.*/%{GREEDYDATA:_tmp.session_name}$" with no ^. Without the start anchor grok scans for arn: anywhere in the value, so the pattern is unanchored (pipeline checklist: missing grok anchoring) and differs from the fix it claims to backport. A faithful copy also keeps later cherry-picks onto the 7.3.x line conflict-free.

Recommendation:

Use the exact pattern merged on main, including the start anchor.

Suggested change

- "arn:[a-z0-9-]+:sts:.*/%{GREEDYDATA:_tmp.session_name}$"

- '^arn:[a-z0-9-]+:sts:.*/%{GREEDYDATA:_tmp.session_name}$'

🤖 AI-Generated Review | Vera Review Bot - v0.4.1 | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

vera-review-bot · 2026-09-10T20:51:57Z

Review summary

Issues found across the latest commits ae4c979 — 2 medium, 1 low

🟡 Backported grok pattern drops the leading ^ anchor that the main-branch fix (PR 20403) carries (link) (Unresolved)

Package-level:

🟡 No pipeline test fixture exercises an aws-cn or aws-iso* ARN, so the widened pattern is untested

Copy test-us-gov-arn.log to a new test-cn-arn.log with the partition and region changed, using placeholder identifiers only:

{"eventVersion":"1.11","userIdentity":{"type":"AssumedRole","principalId":"AROAEXAMPLE:i-example1234567890","arn":"arn:aws-cn:sts::111111111111:assumed-role/ExampleRole/i-example1234567890","accountId":"111111111111","accessKeyId":"ASIAEXAMPLE","sessionContext":{"sessionIssuer":{"type":"Role","principalId":"AROAEXAMPLE","arn":"arn:aws-cn:iam::111111111111:role/ExampleRole","accountId":"111111111111","userName":"ExampleRole"},"attributes":{"creationDate":"2026-01-05T17:53:33Z","mfaAuthenticated":"false"}}},"eventTime":"2026-01-05T18:46:26Z","eventSource":"ssm.amazonaws.com","eventName":"UpdateInstanceInformation","awsRegion":"cn-north-1","sourceIPAddress":"81.2.69.144","userAgent":"aws-sdk-go/1.55.5","requestID":"be50efb8-2ecb-4a40-a9ac-533ffa351EXAMPLE","eventID":"46a04795-4e02-4710-a395-5a0f578cfEXAMPLE","readOnly":false,"eventType":"AwsApiCall","managementEvent":true,"recipientAccountId":"111111111111","eventCategory":"Management"}

Then regenerate the expected output with elastic-package test pipeline -g.

🔵 PR description says the bug left user.changes.name unpopulated, but on the 7.3 line the session name feeds related.user

aws.cloudtrail: widen ARN partition grok pattern to include aws-cn and aws-iso* (7.3.x)

Backport of the ARN partition change from PR 20403 to the 7.3.x line.

The grok processor that extracts the assumed-role session name from
aws.cloudtrail.user_identity.arn matched only the aws and aws-us-gov
partitions, so identities from aws-cn and aws-iso* partitions were
silently skipped and the session name was never added to related.user.
The partition alternation is replaced with the character class
[a-z0-9-]+, which covers all current and future AWS partition strings.

A new commit triggers another review — at most once every 15 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot - v0.4.1 | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

infra-vault-gh-plugin-prod · 2026-09-10T20:55:43Z

💔 Build Failed

Buildkite Build
Commit: ae4c979

Failed CI Steps

Check integrations aws

History

cc @efd6

github-actions · 2026-09-10T21:08:02Z

TL;DR

The failure is not from the CloudTrail grok change itself; it fails during package installation because Kibana rejects uploading aws by name on this stack. Immediate action is to adjust the installation step for bundled packages (or run this check against a stack where aws is not bundled), then re-run CI.

Remediation

In .buildkite/scripts/common.sh, update the install path used by run_tests_package() for bundled packages so aws does not use plain elastic-package install (install_package() at .buildkite/scripts/common.sh#L984-L989); either skip install for bundled packages and continue tests, or switch to a supported overwrite/force install path if available in your CI environment.
Re-run Check integrations aws after that change to confirm the package test phase proceeds past installation.

Investigation details

Root Cause

run_tests_package() always executes package installation in local-stack mode before tests (.buildkite/scripts/common.sh#L1032-L1060), and install_package() uses a plain elastic-package install call (.buildkite/scripts/common.sh#L984-L989).

On this Buildkite run, that install call failed with Kibana API 400 because aws is already present as a bundled/registry package in the 9.6 snapshot stack, so zip-install is rejected before test suites begin.

Evidence

Build: https://buildkite.com/elastic/integrations/builds/49469
Job/step: Check integrations aws (test-integrations-aws)
Key log excerpt:

2026-09-10T20:55:12.555Z--- [aws] test installation
2026-09-10T20:55:12.555ZInstall package: aws
2026-09-10T20:55:19.023ZError: can't install the package: could not zip-install package; API status code = 400; response body = {"statusCode":400,"error":"Bad Request","message":"Cannot upload a package whose name already exists in the package registry or as a bundled package: aws"}
2026-09-10T20:55:19.027Z[aws] run_tests_package failed

Verification

Not run locally (this environment does not support the Buildkite Docker-in-Docker stack used by the job).

Follow-up

If you prefer to keep installation coverage, consider adding package-name-aware handling in CI for bundled packages so aws backports can still run the rest of the suite deterministically.

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

efd6 requested review from a team as code owners September 10, 2026 20:35

aws.cloudtrail: fix changelog PR link

ae4c979

efd6 self-assigned this Sep 10, 2026

efd6 added Integration:aws AWS bugfix Pull request that fixes a bug issue Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Sep 10, 2026

vera-review-bot Bot reviewed Sep 10, 2026

View reviewed changes

github-actions Bot mentioned this pull request Sep 10, 2026

[aw] No-Op Runs #19221

Open

Navigation Menu

Uh oh!

[aws.cloudtrail] Fix ARN partition grok pattern for aws-cn/aws-iso* (7.3.x backport) - #21180

[aws.cloudtrail] Fix ARN partition grok pattern for aws-cn/aws-iso* (7.3.x backport)#21180
efd6 wants to merge 2 commits into
backport-aws-7.3from
cloudtrail/backport-7.3-arn-cn

efd6 commented Sep 10, 2026

Uh oh!

github-actions Bot commented Sep 10, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Sep 10, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Sep 10, 2026

Uh oh!

infra-vault-gh-plugin-prod Bot commented Sep 10, 2026

Uh oh!

vera-review-bot Bot Sep 10, 2026

Uh oh!

vera-review-bot Bot commented Sep 10, 2026

Uh oh!

infra-vault-gh-plugin-prod Bot commented Sep 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 10, 2026

Root Cause

Evidence

Verification

Follow-up

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	- "arn:[a-z0-9-]+:sts:.*/%{GREEDYDATA:_tmp.session_name}$"
	- '^arn:[a-z0-9-]+:sts:.*/%{GREEDYDATA:_tmp.session_name}$'

Uh oh!

Conversation

efd6 commented Sep 10, 2026

Uh oh!

github-actions Bot commented Sep 10, 2026

✅ Elastic Docs Style Checker (Vale)

Uh oh!

elastic-vault-github-plugin-prod Bot commented Sep 10, 2026

Uh oh!

elastic-vault-github-plugin-prod Bot commented Sep 10, 2026

Uh oh!

infra-vault-gh-plugin-prod Bot commented Sep 10, 2026

Uh oh!

vera-review-bot Bot Sep 10, 2026

Choose a reason for hiding this comment

Uh oh!

vera-review-bot Bot commented Sep 10, 2026

Review summary

Uh oh!

infra-vault-gh-plugin-prod Bot commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Failed CI Steps

History

Uh oh!

github-actions Bot commented Sep 10, 2026

TL;DR

Remediation

Root Cause

Evidence

Verification

Follow-up

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

infra-vault-gh-plugin-prod Bot commented Sep 10, 2026 •

edited

Loading