Skip to content

[Feature]: Data-driven generic template for Filebeat modules (replace beats_filebeat_modules) #510

Open

Description

@afeefghannam89

Today beats_filebeat_modules is a flat list of module names; the role only runs filebeat modules enable, so a module's config (paths, vars, filesets) can't be customized — only the system module is templated.

Replace the flat beats_filebeat_modules name-list with a data-driven variable (a list of modules with their config), rendered inline into filebeat.yml (filebeat.modules:) by a shared template/macro. This one list is the declarative desired state: modules in the list are enabled and configured; modules removed on a later run disappear — no separate disable list, no enabled: false needed.

Render inline, not into modules.d/*.yml: inline is re-rendered wholesale each run → removal is automatic; per-file modules.d/ rendering would orphan files (module stays active) and need cleanup, and modules.d settings take precedence over the main config. Drop the imperative filebeat modules enable; leave modules.d/ untouched.

Re-adapt the setup task here too (supersedes #508's current-architecture fix): run filebeat setup when this template changes (register/is changed, run_once) so pipelines for newly added filesets load; inline means setup --pipelines sees the modules directly, so the --modules workaround from #508 is no longer needed.

Docs: the module structure is documented once in a shared roles/beats/docs/beats-modules.md (common model + a Filebeat section with examples), linked from the beats_filebeat_modules description; the README table shows only a short default + the pointer.

Breaking: beats_filebeat_modules changes from a name list to a data structure. Shares the macro + docs page with #505 and the Auditbeat modules issue. Related: #393 (filebeat inputs).


Depends on: #393 (shared plugin-as-data macro).

Activity

  1. added
    featureNew capability or new variable for users
    breaking-changeChanges behaviour or variables so users must adapt
    and removed on Jul 23, 2026
  2. added this to the 2.0.0 milestone on Jul 23, 2026
  3. changed the title [-][Feature]: Data-driven generic template for Filebeat modules (replace beats_filebeat_modules)[/-] [+][Feature]: Data-driven generic template for Beats modules (Filebeat + Metricbeat)[/+] on Jul 24, 2026
  4. changed the title [-][Feature]: Data-driven generic template for Beats modules (Filebeat + Metricbeat)[/-] [+][Feature]: Data-driven generic template for Filebeat modules (replace beats_filebeat_modules)[/+] on Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    breaking-changeChanges behaviour or variables so users must adaptcomponent:filebeatroles/beats - Filebeat onlyfeatureNew capability or new variable for users

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions