[nicholasberlin]

Proposed commit message

WHAT: Extended the IAM event type copy directives (ADD_USER, DEL_USER, ADD_GROUP, DEL_GROUP, USER_CHAUTHTOK, USER_LOGOUT, USER_START, ACCT_LOCK) to source ECS name fields from the enriched uppercase counterparts (AUID, UID, ID) in addition to the existing lowercase numeric fields. When auditd appends enriched context using the Unit Separator (U+001D, \x1d) as delimiter, the KV processor correctly extracts auditd.log.AUID, .UID, and .ID, but the copy directives never read them — so user.name, user.effective.name, user.target.name, and group.name were silently absent while the numeric user.id / user.effective.id fields were present.

Also added two gsub processors in raw.yml to normalize the malformed format where res=success is directly concatenated with a following uppercase KV key (res=successUID="root"), which previously caused UID not to be extracted and event.outcome to be set to unknown. Added placeholder filtering ("unset", "?") in the copy script. Added auditd.log.ID to the field definitions.

WHY: Customer reported that a number of critical Elastic Security detection rules were silently failing because user.name and related fields were not populated for these IAM event types. The root cause is that modern auditd (and auditd on SELinux systems) commonly uses the \x1d delimiter format, and all affected customers running 9.x agent versions would hit this issue.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• Pipeline tests regenerated with elastic-package test pipeline --generate against stack 9.4.3, all 12 pass
• Verified ECS output for all 8 affected event types via test inspection

How to test this PR locally

cd packages/auditd
elastic-package stack up -d --services=elasticsearch
elastic-package test pipeline
elastic-package stack down

The 9 new entries at the end of test-auditd-useradd.log cover all 8 affected event types. Inspect test-auditd-useradd.log-expected.json events 8–16 and confirm user.name, user.effective.name, user.target.name, and group.name are populated.

Related issues

• Closes [Auditd Logs Integration]: user.name, user.target.name and user.effective.name not getting extracted for Account deletion #20135
• Relates [Auditd Logs Integration]: user.name, user.target.name and user.effective.name not getting extracted for Account creation #20137
• Relates [Auditd Logs Integration]: user.name, user.target.name and user.effective.name not getting extracted for Password Change #20138
• Relates [Auditd Logs Integration]: user.name, user.target.name and user.effective.name not getting extracted for Account Lockout #20139
• Relates [Auditd Logs Integration]: group.name, user.name and user.effective.name not getting extracted for Group Deletion #20140

[github-actions]

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

---

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

[github-actions]

TL;DR

The auditd package check failed because the PR adds the exported field auditd.log.ID to packages/auditd/data_stream/log/fields/fields.yml, but the generated packages/auditd/docs/README.md was not regenerated. No pipeline or source-code failure is indicated.

Remediation

• Run elastic-package build for packages/auditd (or regenerate the package README) and commit the resulting packages/auditd/docs/README.md update, including auditd.log.ID.
• Re-run .buildkite/scripts/test_one_package.sh packages/auditd origin/main 67b107508f569ef0a0f913190e684a68a2635f7f or the equivalent package check.

Investigation details

Root Cause

The PR commit introduces the ID field in the enriched auditd field definitions, while the generated README still lists auditd.log.UID immediately after auditd.log.SYSCALL and omits auditd.log.ID. The package linter compares generated output with the committed README and fails on this mismatch.

Evidence

• Build: https://buildkite.com/elastic/integrations/builds/46293
• Job/step: Check integrations auditd
• Key log excerpt: README.md is outdated. Rebuild the package with elastic-packagebuild followed by +| auditd.log.ID | | keyword | in the generated output.

Verification

• The failed job reached package format/lint and exited before pipeline tests; no additional tests were run during this investigation.

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[infra-vault-gh-plugin-prod]

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

[stanek-michal]

looks like UNLOCK also needs a copy directive like LOCK:

Summary

The PR correctly fixes the root cause: for modern auditd with U+001D enrichment, the copy directives in default.yml only sourced lowercase numeric fields (auid, uid, id) and never read the enriched uppercase counterparts (AUID, UID, ID) that carry the human-readable names. The fix is straightforward and well-tested across all 8 stated event types. There is one significant omission and a couple of minor observations.

Findings

Significant

ACCT_UNLOCK still has no copy directives — same root cause as ACCT_LOCK before this fix

default.yml lines 1059–1067 (both main and the PR branch):

ACCT_UNLOCK:
  - event:
      action:
        - unlocked-account
      category:
        - iam
      type:
        - user
        - info

ACCT_LOCK (which this PR fixes) was in exactly the same situation and is now corrected. ACCT_UNLOCK is a symmetric IAM operation and would produce the same missing-field symptoms for customers running the enriched format. It should receive the same copy directives added to ACCT_LOCK.

Minor

GRP_MGMT and USER_MGMT still have lowercase-only copy directives — pre-existing, out of stated scope

These two types have copy directives but they reference only auid/uid/id, not AUID/UID/ID. They'd exhibit the same missing-name behavior under the enriched format. They're not mentioned in the PR's issue list, so this may be intentional scoping. Worth a brief note in the description or a follow-up issue if so.

GRP_MGMT maps uid → group — pre-existing semantic oddity, not introduced here

GRP_MGMT:
  - copy:
      - from: [auid]
        to: user
      - from: [uid]
        to: group      # uid (process owner, typically 0/root) mapped to group field
      - from: [uid]
        to: user.effective

uid is the process UID (usually root=0) and not a group identifier. This pre-dates this PR and is outside its scope, but it would produce group.id = "0" for all GRP_MGMT events, which seems wrong. Flagging for awareness.

Verdict

REQUEST_CHANGES — the ACCT_UNLOCK omission is a directly parallel case to the ACCT_LOCK fix: same event category, same enrichment format, same missing fields for end users. It should be included in this fix. Everything else in the PR is correct and well-tested.

[nicholasberlin]

looks like UNLOCK also needs a copy directive like LOCK:

5ec48a0

[elastic-vault-github-plugin-prod]

✅ All changelog entries have the correct PR link.

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[infra-vault-gh-plugin-prod]

💚 Build Succeeded

• Buildkite Build
• Commit: 5ec48a0

History

• 💚 Build #46485 succeeded d7ecbde
• 💔 Build #46477 failed 9eedc03
• 💚 Build #46369 succeeded 08c1f8b
• 💔 Build #46293 failed 67b1075