Skip to content

[sql_server_input_otel][mysql_input_otel] Add system and policy tests for logs and metrics signals - #18658

Merged
mrodm merged 7 commits into
elastic:mainfrom
mrodm:add-tests-sql_mysql
Apr 29, 2026
Merged

mrodm merged 7 commits into
elastic:mainfrom
mrodm:add-tests-sql_mysql

Conversation

@mrodm

@mrodm mrodm commented Apr 27, 2026 •

Copy link
Copy Markdown
Collaborator

Proposed commit message

Add signal-specific system tests and policy tests to the sql_server_input_otel and mysql_input_otel integrations.

WHAT:

sql_server_input_otel

  • Enable test-logs-config with signal_types: logs, covering both db.server.top_query and db.server.query_sample events, including fields_present assertions aligned with the upstream receiver's field names (user.name instead of sqlserver.username; sqlserver.query_plan removed due to the value exceeding the ES ignore_above keyword limit). Uses top_query_lookback_time: 20s to match the workload's 20s lock window.
  • Add test-metrics-config (currently skipped pending multisignal input support in elastic-package).
  • Enable log event collection (enable_query_sample_events, enable_top_query_events, top_query_lookback_time: 20s) in the default and datasource configs so the default system test also exercises log collection alongside metrics.
  • Remove blanket fields_present from the default and datasource configs — assertions are now handled per-signal config file.
  • Fix the Docker workload to hold an exclusive table lock (TABLOCKX) so foreground SELECTs appear as suspended sessions in sys.dm_exec_requests, making them reliably visible to the query_sample collector.
  • Remove the skip directive from all 5 existing policy tests (the multisignal input limitation does not affect policy tests).
  • Add test-resource-attributes policy test: exercises the resource_attributes YAML var with a mix of enabled/disabled attributes.
  • Add test-logs-custom-collection policy test: verifies non-default values for query-sample and top-query collection settings are correctly rendered.

mysql_input_otel

  • Add test-logs-config and test-metrics-config signal-specific system tests with fields_present assertions using correct attribute names from the upstream mysqlreceiver documentation.
  • Add policy tests for logs and metrics enabling.
  • Restrict the default system test to a multisignal run (no signal_types filter), dropping fields_present in favour of the signal-specific configs.

WHY:
These tests ensure that both OTel SQL integrations collect the expected fields for each signal type, catch regressions in field naming conventions, and validate that policy rendering produces correct agent configurations for all supported configuration options.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.~
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • All policy tests for both sql_server_input_otel and mysql_input_otel pass (elastic-package test policy -v)
  • All system tests for both sql_server_input_otel and mysql_input_otel pass (elastic-package test system -v)

How to test this PR locally

elastic-package stack update -v -version 9.4.0-SNAPSHOT
elastic-package stack up -v -d --version 9.4.0-SNAPSHOT

# SQL Server OTel — policy tests
elastic-package -C packages/sql_server_input_otel test policy -v

# SQL Server OTel — system tests (requires a running stack + Docker)
elastic-package -C packages/sql_server_input_otel  test system -v

# MySQL OTel — system tests
elastic-package -C packages/mysql_input_otel  test system -v

elastic-package stack down -v

Related issues


This PR was generated with the assistance of Claude (claude-sonnet-4-6).

mrodm and others added 3 commits April 27, 2026 16:34
- Enable test-logs-config with signal_types: logs, covering both
  db.server.top_query and db.server.query_sample events
- Add test-metrics-config for metrics signal validation
- Remove fields_present from test-default and test-datasource configs;
  signal-type-specific assertions are now handled per config file
- Fix workload to hold an exclusive lock (TABLOCKX) so foreground
  SELECTs appear as suspended sessions in sys.dm_exec_requests,
  making them reliably visible to the query_sample collector
- Remove sqlserver.query_plan from fields_present: the XML plan content
  exceeds the Elasticsearch keyword ignore_above limit and lands in
  _ignored, making it not indexable/searchable
- Replace sqlserver.username with user.name: the receiver follows ECS
  convention and emits user.name instead of a custom sqlserver.* field

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove skip directive from all 5 existing policy tests; the
  multisignal input limitation does not affect policy tests since they
  only validate rendered config output without running the agent
- Add test-resource-attributes: exercises the resource_attributes YAML
  var with a mix of enabled/disabled attributes (sqlserver.computer.name,
  sqlserver.instance.name, server.address, host.name)
- Add test-logs-custom-collection: verifies that non-default values for
  query_sample and top_query collection settings are correctly rendered
  (max_rows_per_query: 200, lookback_time: 120s, max_query_sample_count:
  500, top_query_count: 100, collection_interval: 30s)
- All 7 policy tests pass (elastic-package test policy -v confirmed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add signal-specific system tests for logs (test-logs-config.yml) and
metrics (test-metrics-config.yml) with fields_present assertions using
the correct attribute names from the upstream mysqlreceiver documentation.
Update the default system test to restrict it to the metrics signal so
fields_present assertions are not drowned out by log documents.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mrodm mrodm self-assigned this Apr 27, 2026
@mrodm mrodm added Integration:mysql_input_otel MySQL (OpenTelemetry) Integration:sql_server_input_otel SQL Server OpenTelemetry Input Package labels Apr 27, 2026
mrodm and others added 4 commits April 27, 2026 18:50
… tests

- Fix missing unit suffix: top_query_lookback_time: 60 → 60s (mysql)
- Add service_notify_signal: SIGHUP to mysql test-metrics-config.yml for consistency with default and logs configs
- Remove dead commented-out skip block from sql_server test-logs-config.yml
- Remove sqlserver.username tombstone comment from sql_server test-logs-config.yml
- Remove commented-out image: lines from sql_server docker-compose.yml (build: stanza is sufficient)
- Remove trailing blank lines from sql_server test-metrics-config.yml

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove the skip block from all three existing policy tests and add two
new ones: test-logs-enabled exercises the top_query_collection template
branch with custom collection settings, and test-metrics-enabled
overrides the metrics configuration with non-default optional metrics.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The MySQL receiver expects an integer (seconds), not a duration string.
Reverts the 60 → 60s change from the previous cleanup commit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Reduce top_query_lookback_time from 60s to 20s in test-logs-config.yml
- Enable query_sample and top_query events in test-default-config.yml and
  test-datasource-config.yml, with top_query_lookback_time: 20s

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@elasticmachine

Copy link
Copy Markdown

💚 Build Succeeded

History

cc @mrodm

@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

Pinging @elastic/ecosystem (Team:Ecosystem)

@mrodm mrodm changed the title [sql_server_input_otel][mysql_input_otel] Add system and policy tests for logs and metrics signals - WIP [sql_server_input_otel][mysql_input_otel] Add system and policy tests for logs and metrics signals Apr 29, 2026
mrodm added a commit to elastic/elastic-package that referenced this pull request Apr 29, 2026
…and policy tests for logs and metrics (#3508)

Update the sql_server_input_otel and mysql_input_otel test packages
to add signal-specific system and policy tests, aligned with the changes
in elastic/integrations#18658.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@mrodm
mrodm merged commit e6319e3 into elastic:main Apr 29, 2026
9 checks passed
herrBez pushed a commit to herrBez/integrations that referenced this pull request Jun 1, 2026
… for logs and metrics signals (elastic#18658)

Add signal-specific system tests and policy tests for logs and metrics
to the sql_server_input_otel and mysql_input_otel packages.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Integration:mysql_input_otel MySQL (OpenTelemetry) Integration:sql_server_input_otel SQL Server OpenTelemetry Input Package Team:Ecosystem Packages Ecosystem team [elastic/ecosystem]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[sql_server_input_otel, mysql_input_otel]: Add signal_types to system test configs when elastic-package enables dynamic signal testing

4 participants