[system][auth] Remove deprecated user.name/user.id duplication for IAM events

[nicholasberlin]

Proposed commit message (updated)

[system][auth] Remove deprecated user.name/user.id duplication for IAM events

Mechanically this deletes the five *_compat processors that 2.22.0
introduced for backward compatibility; for existing fixture lines the
regenerated pipeline test expectations differ only by the removal of the
duplicated fields. It also replaces set_user_target_name_chauthtok (which
copied user.name into user.target.name) with renames that route the account
parsed from non-success passwd:chauthtok messages (the user= of an
authentication failure, or the quoted user of the "does not exist" message)
straight to user.target.name, the same way the successful "password changed
for" case already works. A user.name populated upstream is left untouched,
and chauthtok accounts are excluded from the generic renames into user.name.
New fixture lines cover both messages, and a new fixture with a
pre-populated user.name covers all three chauthtok paths.

Ships as 3.0.0 with a breaking-change changelog entry, which Kibana shows
for acknowledgment on UI-initiated upgrades. The bundled New users and
groups dashboard already reads user.target.* as of 2.24.0 (#20352).

Original proposed commit message

Contract phase of the expand-and-contract migration started in 2.22.0
(#20339): the auth data stream no longer writes the
affected account to user.name/user.id for useradd account creation,
usermod account changes (for example group membership or expiration changes),
userdel deletion, and PAM password change events. The affected account remains
in user.target.name (and user.target.id for useradd); user.name on these
events is reserved for the acting user, matching ECS semantics. On journald
input, useradd events now keep the actor UID from journald metadata in
user.id instead of having it overwritten by the created account's UID.

Mechanically this deletes the five *_compat processors that 2.22.0
introduced for backward compatibility and regenerates the pipeline test
expectations, whose diff is exactly the removal of the duplicated fields. It
also replaces set_user_target_name_chauthtok (which copied user.name into
user.target.name) with renames that route the account parsed from
non-success passwd:chauthtok messages (the user= of an
authentication failure, or the quoted user of the debug-level
does not exist message) straight to user.target.name, the same way the
successful password changed for case already works. A user.name populated
upstream is left untouched, and chauthtok accounts are excluded from the
generic renames into user.name. Fixture lines cover both messages.

Ships as 3.0.0 with a breaking-change changelog entry so Kibana requires
users to acknowledge the change before upgrading. The bundled New users and
groups dashboard already reads user.target.* as of 2.24.0 (#20352).

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs. N/A, pipeline-only change
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines. N/A
• I have verified that any added dashboard complies with Kibana's Dashboard good practices N/A

Author's Checklist

• [system] Use user.target.* in New users and groups dashboard #20352 merged and shipped as 2.24.0; this branch has been brought up
  to date with main so the diff now contains only the 3.0.0 change.
• Breaking change: detections, dashboards, and saved searches keyed on
  user.name/user.id for these IAM events stop matching new data. The
  changelog entry (surfaced by Kibana's upgrade acknowledgment prompt) tells
  users what changed, why, and to migrate content to user.target.*.
• Timing: 2.22.0 announced the deprecation. Reviewers should confirm
  enough release time has passed between the expand and contract phases for
  users to migrate, and hold this PR if not.
• Follow-up (separate PR in elastic/detection-rules, which is the source
  for security_detection_engine): the Linux User Account Creation rule
  (edfd5ca9-9d6c-44d9-b615-1e56b920219c) still matches as intended, but the
  osquery investigation queries in its note field use {{user.name}} and
  should switch to {{user.target.name}}.

How to test this PR locally

cd packages/system
elastic-package stack up -d
elastic-package test pipeline --data-streams auth

Or simulate directly:

POST _ingest/pipeline/logs-system.auth-<version>/_simulate

with useradd/usermod/userdel/passwd-chauthtok sample events (see
#20105) and verify the affected account appears only in
user.target.* (plus related.user), with user.name/user.id no longer
duplicated.

Related issues

• Closes [System] : ECS user field mapping issues in password change, account creation, and group membership events #20105
• Requires [system] Use user.target.* in New users and groups dashboard #20352 (merged, 2.24.0)
• Relates [system][auth] Populate user.target.* for IAM events (expand phase) #20339

[github-actions]

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

---

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

[infra-vault-gh-plugin-prod]

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

[nicholasberlin]

The deprecation release was here: #20105 (comment)

July 24, 2026

Not sure how long of a grace period.

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[nicholasberlin]

The deprecation release was here: #20105 (comment)

July 24, 2026

Not sure how long of a grace period.

[Copilot]

Pull request overview

This PR implements the contract phase of the System auth ingest pipeline’s expand/contract migration by removing the deprecated duplication of affected-account fields into user.name/user.id for IAM events, aligning emitted documents with ECS semantics (actor in user.*, affected account in user.target.*).

Changes:

• Removes the *_compat processors that duplicated user.target.* into user.name/user.id for useradd/usermod/userdel/PAM chauthtok IAM events.
• Bumps the System package version to 3.0.0 and adds a breaking-change changelog entry describing the field behavior change.
• Regenerates pipeline test expected outputs to reflect the removal of duplicated user.name/user.id (including journald cases where user.id is no longer overwritten).

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
packages/system/manifest.yml	Bumps package version to 3.0.0 for the breaking change.
packages/system/data_stream/auth/elasticsearch/ingest_pipeline/message.yml	Removes backward-compat field duplication processors for IAM events.
packages/system/data_stream/auth/_dev/test/pipeline/test-usermod-group.log-expected.json	Updates expectations to no longer include duplicated user.name for usermod group changes.
packages/system/data_stream/auth/_dev/test/pipeline/test-userdel.log-expected.json	Updates expectations to no longer include duplicated user.name for userdel events.
packages/system/data_stream/auth/_dev/test/pipeline/test-pam-extraction.log-expected.json	Updates expectations to no longer duplicate PAM target user into user.name.
packages/system/data_stream/auth/_dev/test/pipeline/test-pam-chauthtok-iso8601.log-expected.json	Updates ISO8601 PAM chauthtok expectations to remove user.name duplication.
packages/system/data_stream/auth/_dev/test/pipeline/test-auth.log-expected.json	Updates baseline auth fixture expectations to remove useradd user.name/user.id duplication.
packages/system/data_stream/auth/_dev/test/pipeline/test-auth-ubuntu1204.log-expected.json	Updates Ubuntu fixture expectations to remove duplicated user.* fields on IAM events.
packages/system/data_stream/auth/_dev/test/pipeline/test-auth-rhel79.log-expected.json	Updates RHEL fixture expectations to remove duplicated user.name on IAM events.
packages/system/data_stream/auth/_dev/test/pipeline/test-auth-debian11.json-expected.json	Updates journald fixture expectations to stop overwriting user.id with the created account’s UID and remove user.name duplication.
packages/system/data_stream/auth/_dev/test/pipeline/test-auth-debian11-preserve-original.json-expected.json	Same as above for the preserve-original variant.
packages/system/changelog.yml	Adds 3.0.0 breaking-change entry documenting the removal of duplicated fields.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[botelastic]

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[infra-vault-gh-plugin-prod]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[Copilot]

Copilot review overview

🟡 Changes recommended

The PAM rename may overwrite actor identity by moving pre-existing user.name into user.target.name.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

• Avoid moving pre-existing user.name into user.target.name · New

[nicholasberlin]

@chrisberkhout small follow-up to the chauthtok change: Copilot pointed out that renaming user.name → user.target.name would also move a user.name populated upstream (e.g. by an agent processor) and then block rename_pam_target. 69fe93b routes the parsed account (_temp.pam_user / _temp.user) directly to user.target.name for chauthtok instead, the same way _temp.pam_target already works. Same fixture result; happy to go back to the plain rename if you prefer it.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Chauthtok fallback paths can still write affected accounts to user.name when targets already exist, and coverage is incomplete.

Review effort: Lite
Findings: None

Resolved since last review (1)

• Avoid moving pre-existing user.name into user.target.name

[nicholasberlin]

Follow-ups from the latest bot passes, in af50455:

• Copilot's overview noted the chauthtok renames could still fall through to user.name when user.target.name is already populated (the guard skips the chauthtok rename, then rename_pam_user/rename_user pick the _temp.* field up). Both generic renames now skip _temp.category == 'chauthtok', so a chauthtok account is either placed in user.target.name or dropped, never misattributed to user.name.
• Added the debug-level pam_unix user "nosuchuser" does not exist in /etc/passwd line to test-pam-chauthtok-iso8601.log so the _temp.user path is covered too.
• Refreshed the proposed commit message in the description (Vera's note) to describe the direct routing rather than "a rename".

As before, the new expected entry is hand-derived; CI will confirm.

[Copilot]

Copilot review overview

🔵 Needs a closer look

Add regression fixtures verifying that an existing actor in user.name is preserved while chauthtok accounts are routed to user.target.name.

Review effort: Lite
Findings: None

[vera-review-bot]

🟢 No issues across the latest commits af50455.

Review summary

Issues found across earlier commits 69fe93b — 1 low

Package-level:

• 🔵 Proposed commit message

  system: stop duplicating the affected account into user.name/user.id for auth IAM events
  
  This is the contract phase of the expand-and-contract migration started in
  2.22.0 (elastic/integrations#​20339). The auth data stream no longer writes
  the affected account to user.name/user.id for useradd account creation,
  usermod account changes, userdel deletion, and PAM password change events.
  The affected account remains in user.target.name (and user.target.id for
  useradd), and user.name on these events is reserved for the acting user,
  matching ECS semantics. On journald input, useradd events now keep the actor
  UID from journald metadata in user.id instead of having it overwritten by
  the created account's UID.
  
  The five *_compat set processors introduced in 2.22.0 are removed, and the
  chauthtok handling becomes a rename so that failed passwd:chauthtok messages
  (authentication failure; ... user=bob) also carry the account only in
  user.target.name, consistent with the successful password-changed case. A
  fixture line covers this case.
  
  Ships as 3.0.0 with a breaking-change changelog entry so Kibana requires
  users to acknowledge the change before upgrading. The bundled New users and
  groups dashboard already reads user.target.* as of 2.24.0 (#​20352).
  
  Closes #​20105
  

Issues found across earlier commits 987c7d1…108f265 (229 commits) — 1 low

• 🔵 The new changelog entries link to the tracking issue (link) (Outdated)

A new commit triggers another review — at most once every 15 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot - v0.4.2 | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

[mergify]

This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again.

[Copilot]

Copilot review overview

🟡 Changes recommended

Add chauthtok fixtures verifying an existing actor remains in user.name while the affected account moves to user.target.name.

Review effort: Lite
Findings: 1

Open (1)

• Add chauthtok fixtures preserving existing actor in user.name · New

[Copilot]

Copilot review overview

🟢 Approval recommended

No unresolved review issues were identified, and all reviewed assessments indicate readiness.

Review effort: Lite
Findings: None

Resolved since last review (1)

• Add chauthtok fixtures preserving existing actor in user.name

[elastic-vault-github-plugin-prod]

✅ All changelog entries have the correct PR link.

[infra-vault-gh-plugin-prod]

💚 Build Succeeded

• Buildkite Build
• Commit: 5026a21

History

• 💚 Build #50376 succeeded af50455
• 💚 Build #50373 succeeded 8bddcbe
• 💚 Build #49619 succeeded 5d8fc7b
• 💔 Build #49616 failed 108f265
• 💚 Build #47801 succeeded 479ef39
• 💚 Build #47059 succeeded 40db538

cc @chrisberkhout @nicholasberlin

[elastic-vault-github-plugin-prod]

Package system - 3.0.0 containing this change is available at https://epr.elastic.co/package/system/3.0.0/