Skip to content

[system][auth] Populate user.target.* for IAM events (expand phase) - #20339

Merged
nicholasberlin merged 1 commit into
mainfrom
fix/system-auth-iam-user-target
Jul 24, 2026
Merged

[system][auth] Populate user.target.* for IAM events (expand phase)#20339
nicholasberlin merged 1 commit into
mainfrom
fix/system-auth-iam-user-target

Conversation

@nicholasberlin

@nicholasberlin nicholasberlin commented Jul 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Proposed commit message

Adds ECS target-user mappings for Linux account-management (IAM) events in the
system.auth pipeline, where the affected account was previously only available
in user.name (semantically the actor) or not extracted at all
(#20105):

  • useradd: new user: name=..., UID=... maps the created account to
    user.target.name/user.target.id.
  • usermod: group membership additions and other account changes
    (change user 'X' expiration ...) map the modified account to
    user.target.name.
  • PAM chauthtok (passwd): password changed for X maps the affected account
    to user.target.name.
  • userdel: new grok patterns extract user.target.name and group.name from
    delete 'X' from (shadow) group 'Y' (with event.action: group-member-removed, mirroring usermod's group-member-added),
    delete user 'X', and removed (shadow) group 'X'. Previously nothing was
    extracted from these messages.

To avoid a breaking change this is the expand half of an expand-and-contract
migration: the affected account is still duplicated into user.name (and
user.id for useradd) everywhere it was written before, so existing outputs
are unchanged apart from the new fields. The duplication is deprecated; a
future major version (contract phase) will stop writing user.name/user.id
for these events once users have had a chance to move detections and
dashboards to user.target.* (user.name = actor, user.target.name =
account the action was performed on). The compatibility processors are
tagged *_compat so the contract-phase removal is mechanical. The changelog
entry announces the deprecation.

related.user includes the affected accounts as before.

The regenerated expected files also pick up 2025 -> 2026 year changes for
test logs with year-less syslog timestamps; the date processor fills in the
current year at execution time, so these are required for the tests to pass.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs. N/A, pipeline-only change
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines. N/A
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices N/A

Author's Checklist

  • Non-breaking by design: pipeline outputs are identical to the current
    release except for added user.target.* (and group.name/event.action
    on the new userdel extractions) — verified by diffing regenerated expected
    files against main.
  • Follow-up (contract phase, future major version): remove the four
    *_compat processors, ship a type: breaking-change changelog entry, and
    migrate content still reading the affected account from user.name
    (e.g. the "[Logs System] New users and groups" dashboard).

How to test this PR locally

cd packages/system
elastic-package stack up -d
elastic-package test pipeline --data-streams auth

Or simulate directly, e.g.:

POST _ingest/pipeline/logs-system.auth-<version>/_simulate

with the sample events from #20105 (useradd, usermod,
userdel, and passwd chauthtok messages) and verify user.target.name,
group.name, and related.user are populated as described, while
user.name/user.id keep their previous values.

Related issues

@nicholasberlin
nicholasberlin requested review from a team as code owners July 23, 2026 17:49
@nicholasberlin nicholasberlin added the bugfix Pull request that fixes a bug issue label Jul 23, 2026
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

@andrewkroh andrewkroh added Integration:system System Team:Security-Linux Platform Linux Platform Security team [elastic/sec-linux-platform] labels Jul 23, 2026
@infra-vault-gh-plugin-prod

infra-vault-gh-plugin-prod Bot commented Jul 23, 2026

Copy link
Copy Markdown

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

@mergify

mergify Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@nicholasberlin
[system][auth] Populate user.target.* for IAM events (expand phase)
845e82c 
Adds ECS target-user mappings for Linux account-management events in the
system.auth pipeline (#20105):

- useradd: "new user: name=..., UID=..." maps the created account to
  user.target.name/user.target.id.
- usermod: group membership additions and other account changes
  ("change user 'X' expiration ...") map the modified account to
  user.target.name.
- PAM chauthtok (passwd): "password changed for X" maps the affected
  account to user.target.name.
- userdel: new grok patterns extract user.target.name and group.name from
  "delete 'X' from (shadow) group 'Y'" (with event.action
  group-member-removed), "delete user 'X'", and "removed (shadow) group".

To avoid a breaking change this is the expand half of an
expand-and-contract migration: the affected account is still duplicated
into user.name (and user.id for useradd) everywhere it was written
before, so existing outputs are unchanged apart from the new fields.
The duplication is deprecated; a future major version (contract phase)
will stop writing user.name/user.id for these events once users have
had a chance to move detections and dashboards to user.target.*. The
compatibility processors are tagged *_compat so the contract-phase
removal is mechanical.

related.user includes the target names as before.
@nicholasberlin
nicholasberlin force-pushed the fix/system-auth-iam-user-target branch from c1cb6cd to 845e82c Compare July 24, 2026 15:14
@nicholasberlin nicholasberlin changed the title [system][auth] Map affected accounts to user.target.name for IAM events [system][auth] Populate user.target.* for IAM events (expand phase) Jul 24, 2026
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

✅ All changelog entries have the correct PR link.

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@infra-vault-gh-plugin-prod

infra-vault-gh-plugin-prod Bot commented Jul 24, 2026

Copy link
Copy Markdown

💚 Build Succeeded

History

@nicholasberlin
nicholasberlin merged commit 08c3a36 into main Jul 24, 2026
9 checks passed
@nicholasberlin
nicholasberlin deleted the fix/system-auth-iam-user-target branch July 24, 2026 16:55
This was referenced Jul 24, 2026
Merged
Merged
nicholasberlin added a commit that referenced this pull request Jul 24, 2026
@nicholasberlin
[system][auth] Remove deprecated user.name/user.id duplication for IA…
6b7ceb9 
…M events

Contract phase of the expand-and-contract migration started in 2.22.0
(#20339): the auth data stream no longer writes the
affected account to user.name/user.id for useradd account creation,
usermod group membership, userdel deletion, and PAM password change
events. The affected account remains in user.target.name (and
user.target.id for useradd); user.name on these events is reserved for
the acting user, matching ECS semantics. On journald input, useradd
events now keep the actor UID from journald metadata in user.id instead
of having it overwritten by the created account's UID.

Mechanically this deletes the five *_compat processors that 2.22.0
introduced for backward compatibility and regenerates the pipeline test
expectations, whose diff is exactly the removal of the duplicated
fields.

Ships as 3.0.0 with a breaking-change changelog entry so Kibana
requires users to acknowledge the change before upgrading. The bundled
New users and groups dashboard already reads user.target.* as of
2.23.0.
nicholasberlin added a commit that referenced this pull request Aug 3, 2026
@nicholasberlin
[system] Use user.target.* in New users and groups dashboard
c9013d4 
Updates the "[Logs System] New users and groups" dashboard to read the
created account from user.target.name/user.target.id instead of
user.name/user.id in the four New users panels (table, over time, by
shell, by home directory). The New groups panels are unchanged, as
groupadd events still map group.name/group.id.

Since system 2.22.0 (#20339) useradd events populate
user.target.* with the created account; user.name/user.id only carry a
deprecated duplication that a future major version will remove
(contract phase of the expand-and-contract migration,
#20105). Moving the dashboard to the new fields now
means it keeps working after that removal.

Note the New users panels only show events ingested with system 2.22.0
or later, where user.target.* was introduced for useradd events.
nicholasberlin added a commit that referenced this pull request Aug 3, 2026
@nicholasberlin
[system][auth] Remove deprecated user.name/user.id duplication for IA…
23bcd0d 
…M events

Contract phase of the expand-and-contract migration started in 2.22.0
(#20339): the auth data stream no longer writes the
affected account to user.name/user.id for useradd account creation,
usermod group membership, userdel deletion, and PAM password change
events. The affected account remains in user.target.name (and
user.target.id for useradd); user.name on these events is reserved for
the acting user, matching ECS semantics. On journald input, useradd
events now keep the actor UID from journald metadata in user.id instead
of having it overwritten by the created account's UID.

Mechanically this deletes the five *_compat processors that 2.22.0
introduced for backward compatibility and regenerates the pipeline test
expectations, whose diff is exactly the removal of the duplicated
fields.

Ships as 3.0.0 with a breaking-change changelog entry so Kibana
requires users to acknowledge the change before upgrading. The bundled
New users and groups dashboard already reads user.target.* as of
2.23.0.
nicholasberlin added a commit that referenced this pull request Aug 3, 2026
@nicholasberlin
[system][auth] Remove deprecated user.name/user.id duplication for IA…
40db538 
…M events

Contract phase of the expand-and-contract migration started in 2.22.0
(#20339): the auth data stream no longer writes the
affected account to user.name/user.id for useradd account creation,
usermod group membership, userdel deletion, and PAM password change
events. The affected account remains in user.target.name (and
user.target.id for useradd); user.name on these events is reserved for
the acting user, matching ECS semantics. On journald input, useradd
events now keep the actor UID from journald metadata in user.id instead
of having it overwritten by the created account's UID.

Mechanically this deletes the five *_compat processors that 2.22.0
introduced for backward compatibility and regenerates the pipeline test
expectations, whose diff is exactly the removal of the duplicated
fields.

Ships as 3.0.0 with a breaking-change changelog entry so Kibana
requires users to acknowledge the change before upgrading. The bundled
New users and groups dashboard already reads user.target.* as of
2.23.0.
nicholasberlin added a commit that referenced this pull request Aug 3, 2026
@nicholasberlin
[system] Use user.target.* in New users and groups dashboard
22ed856 
Updates the "[Logs System] New users and groups" dashboard to read the
created account from user.target.name/user.target.id instead of
user.name/user.id in the four New users panels (table, over time, by
shell, by home directory). The New groups panels are unchanged, as
groupadd events still map group.name/group.id.

Since system 2.22.0 (#20339) useradd events populate
user.target.* with the created account; user.name/user.id only carry a
deprecated duplication that a future major version will remove
(contract phase of the expand-and-contract migration,
#20105). Moving the dashboard to the new fields now
means it keeps working after that removal.

Note the New users panels only show events ingested with system 2.22.0
or later, where user.target.* was introduced for useradd events.
nicholasberlin added a commit that referenced this pull request Sep 14, 2026
@nicholasberlin
[system] Use user.target.* in New users and groups dashboard
d3aec31 
Updates the "[Logs System] New users and groups" dashboard to read the
created account from user.target.name/user.target.id instead of
user.name/user.id in the four New users panels (table, over time, by
shell, by home directory). The New groups panels are unchanged, as
groupadd events still map group.name/group.id.

Since system 2.22.0 (#20339) useradd events populate
user.target.* with the created account; user.name/user.id only carry a
deprecated duplication that a future major version will remove
(contract phase of the expand-and-contract migration,
#20105). Moving the dashboard to the new fields now
means it keeps working after that removal.

Note the New users panels only show events ingested with system 2.22.0
or later, where user.target.* was introduced for useradd events.
nicholasberlin added a commit that referenced this pull request Sep 14, 2026
@nicholasberlin
[system][auth] Remove deprecated user.name/user.id duplication for IA…
108f265 
…M events

Contract phase of the expand-and-contract migration started in 2.22.0
(#20339): the auth data stream no longer writes the
affected account to user.name/user.id for useradd account creation,
usermod group membership, userdel deletion, and PAM password change
events. The affected account remains in user.target.name (and
user.target.id for useradd); user.name on these events is reserved for
the acting user, matching ECS semantics. On journald input, useradd
events now keep the actor UID from journald metadata in user.id instead
of having it overwritten by the created account's UID.

Mechanically this deletes the five *_compat processors that 2.22.0
introduced for backward compatibility and regenerates the pipeline test
expectations, whose diff is exactly the removal of the duplicated
fields.

Ships as 3.0.0 with a breaking-change changelog entry so Kibana
requires users to acknowledge the change before upgrading. The bundled
New users and groups dashboard already reads user.target.* as of
2.24.0.
nicholasberlin added a commit that referenced this pull request Sep 14, 2026
@nicholasberlin
[system][auth] Remove deprecated user.name/user.id duplication for IA…
5d8fc7b 
…M events

Contract phase of the expand-and-contract migration started in 2.22.0
(#20339): the auth data stream no longer writes the
affected account to user.name/user.id for useradd account creation,
usermod account changes (for example group membership or expiration
changes), userdel deletion, and PAM password change events. The affected
account remains in user.target.name (and user.target.id for useradd);
user.name on these events is reserved for the acting user, matching ECS
semantics. On journald input, useradd events now keep the actor UID from
journald metadata in user.id instead of having it overwritten by the
created account's UID.

Mechanically this deletes the five *_compat processors that 2.22.0
introduced for backward compatibility and regenerates the pipeline test
expectations, whose diff is exactly the removal of the duplicated
fields.

Ships as 3.0.0 with a breaking-change changelog entry so Kibana
requires users to acknowledge the change before upgrading. The bundled
New users and groups dashboard already reads user.target.* as of
2.24.0.
nicholasberlin added a commit that referenced this pull request Sep 15, 2026
@nicholasberlin
[system] Use user.target.* in New users and groups dashboard (#20352)
8b82b93 
Updates the "[Logs System] New users and groups" dashboard to read the
created account from user.target.name/user.target.id instead of
user.name/user.id in the four New users panels (table, over time, by
shell, by home directory). The New groups panels are unchanged, as
groupadd events still map group.name/group.id.

Since system 2.22.0 (#20339) useradd events populate
user.target.* with the created account; user.name/user.id only carry a
deprecated duplication that a future major version will remove
(contract phase of the expand-and-contract migration,
#20105). Moving the dashboard to the new fields now
means it keeps working after that removal.

Note the New users panels only show events ingested with system 2.22.0
or later, where user.target.* was introduced for useradd events.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@fearful-symmetry fearful-symmetry fearful-symmetry approved these changes
@stanek-michal stanek-michal stanek-michal approved these changes

Assignees

No one assigned

Labels

bugfix Pull request that fixes a bug issue Integration:system System Team:Security-Linux Platform Linux Platform Security team [elastic/sec-linux-platform]

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[System] : ECS user field mapping issues in password change, account creation, and group membership events

4 participants

@nicholasberlin @fearful-symmetry @stanek-michal @andrewkroh