Skip to content

[meta] Upgrade integrations to ECS 8.17 #11952

Open

Description

@taylor-swanson

Guide

It is recommended to split the changes into smaller PRs to limit the number of changed files. Generally speaking, 10 integrations per PR is a good number to target.

Automated method

Use the ecs-update tool which can be found here.

cd packages/
ecs-update -pr 999999 -ecs-git-ref v8.17.0 -ecs-version 8.17.0 -owner elastic/owner-name-here package_1 [package_2 ...]

Once PR is filed, the changelogs will need to be updated with the correct PR number.

Manual method

Update ECS references in integrations to version 8.17.

  • Update reference in _dev/build/build.yml to v8.17.0
  • Update package changelog/manifest and regenerate README.
dependencies:
  ecs:
-    reference: "git@v8.11.0"
+    reference: "git@v8.17.0"

ECS 8.17 Changes

https://github.com/elastic/ecs/releases/tag/v8.17.0

Schema Changes

Bugfixes

Improvements

Integrations

@elastic/cloud-security-posture:

  • cloud_security_posture

@elastic/ecosystem:

  • elastic_package_registry

@elastic/elastic-agent:

  • elastic_agent

@elastic/elastic-agent-data-plane:

  • filestream
  • journald
  • linux
  • log
  • windows

@elastic/fleet:

  • fleet_server

@elastic/obs-cloudnative-monitoring:

  • containerd
  • docker
  • istio
  • kubernetes
  • kubernetes_otel
  • nginx_ingress_controller

@elastic/obs-ds-hosted-services:

  • aws
  • aws_logs
  • awsfirehose
  • azure
  • azure_metrics
  • gcp
  • gcp_metrics

@elastic/obs-ds-intake-services:

  • apm
  • profiler_collector
  • profiler_symbolizer

@elastic/obs-infraobs-integrations:

  • activemq
  • airflow [airflow] Make Airflow package GA #15287
  • apache
  • apache_spark
  • apache_tomcat
  • awsfargate
  • azure_app_service
  • azure_application_insights
  • azure_billing
  • azure_functions
  • azure_logs
  • azure_openai
  • cassandra
  • ceph
  • cisco_meraki_metrics
  • citrix_adc
  • cockroachdb
  • coredns
  • couchbase
  • couchdb
  • etcd
  • gcp_vertexai
  • golang
  • hadoop
  • haproxy
  • ibmmq
  • iis
  • influxdb
  • jolokia
  • kafka
  • kafka_log
  • memcached
  • microsoft_sqlserver
  • mongodb
  • mongodb_atlas
  • mysql
  • nagios_xi
  • nats
  • nginx
  • nginx_ingress_controller_otel
  • oracle
  • oracle_weblogic
  • panw_metrics
  • php_fpm
  • postgresql
  • prometheus
  • prometheus_input
  • rabbitmq
  • redis
  • redisenterprise
  • salesforce
  • spring_boot
  • sql
  • stan
  • statsd_input [statsd_input] Make StatsD input package GA #15125
  • system
  • tomcat
  • traefik
  • vsphere
  • websphere_application_server
  • zookeeper

@elastic/sec-deployment-and-devices:

@elastic/sec-linux-platform:

  • auditd
  • auditd_manager
  • cloud_defend
  • fim
  • network_traffic
  • sysmon_linux
  • system_audit

@elastic/sec-windows-platform: #12636

  • hid_bravura_monitor
  • microsoft_dhcp
  • microsoft_dnsserver
  • microsoft_exchange_server
  • mysql_enterprise
  • windows_etw
  • winlog

@elastic/security-asset-management:

  • osquery_manager

@elastic/security-service-integrations:

  • Update Security Service integrations to ECS 8.17.0 (part 1) #14158

    • claroty_ctd
    • entityanalytics_ad
    • sysdig
    • tychon
  • 1password

  • abnormal_security

  • akamai

  • amazon_security_lake

  • atlassian_bitbucket

  • atlassian_confluence

  • atlassian_jira

  • auth0

  • authentik

  • aws_bedrock

  • azure_blob_storage

  • azure_frontdoor

  • azure_network_watcher_nsg

  • azure_network_watcher_vnet

  • barracuda

  • barracuda_cloudgen_firewall

  • bbot

  • bitdefender

  • bitwarden

  • blacklens

  • box_events

  • canva

  • carbon_black_cloud

  • carbonblack_edr

  • cel

  • checkpoint_harmony_endpoint

  • cisa_kevs

  • cisco_duo

  • cisco_meraki

  • cisco_secure_endpoint

  • cisco_umbrella

  • cloudflare

  • cloudflare_logpush

  • corelight

  • cribl

  • crowdstrike

  • cyberark_pta

  • cyberarkpas

  • cybereason

  • cylance

  • darktrace

  • digital_guardian

  • entityanalytics_entra_id

  • entityanalytics_okta

  • eset_protect

  • f5

  • f5_bigip

  • falco

  • fireeye

  • first_epss

  • forcepoint_web

  • forgerock

  • gcp_pubsub

  • gigamon

  • github

  • gitlab

  • google_cloud_storage

  • google_scc

  • google_workspace

  • http_endpoint

  • httpjson

  • imperva_cloud_waf

  • infoblox_bloxone_ddi

  • infoblox_nios

  • jamf_compliance_reporter

  • jamf_pro

  • jamf_protect

  • jumpcloud

  • keycloak

  • lastpass

  • lumos

  • lyve_cloud

  • m365_defender

  • mattermost

  • menlo

  • microsoft_defender_cloud

  • microsoft_defender_endpoint

  • microsoft_exchange_online_message_trace

  • microsoft_sentinel

  • mimecast

  • netskope

  • o365

  • okta

  • opencanary

  • panw_cortex_xdr

  • ping_one

  • pps

  • prisma_access

  • prisma_cloud

  • proofpoint_on_demand

  • proofpoint_tap

  • pulse_connect_secure

  • qualys_vmdr

  • rapid7_insightvm

  • santa

  • sentinel_one

  • sentinel_one_cloud_funnel

  • servicenow

  • slack

  • snyk

  • sophos_central

  • spycloud

  • sublime_security

  • symantec_edr_cloud

  • symantec_endpoint

  • symantec_endpoint_security

  • tanium

  • teleport

  • tenable_io

  • tenable_sc

  • threat_map

  • thycotic_ss

  • ti_abusech

  • ti_anomali

  • ti_cif3

  • ti_crowdstrike

  • ti_custom

  • ti_cybersixgill

  • ti_eclecticiq

  • ti_eset

  • ti_maltiverse

  • ti_mandiant_advantage

  • ti_misp

  • ti_opencti

  • ti_otx

  • ti_rapid7_threat_command

  • ti_recordedfuture

  • ti_threatconnect

  • ti_threatq

  • ti_util

  • tines

  • trellix_edr_cloud

  • trellix_epo_cloud

  • trend_micro_vision_one

  • trendmicro

  • vectra_detect

  • websocket

  • wiz

  • zerofox

  • zeronetworks

  • zoom

  • zscaler_zia

  • zscaler_zpa

@elastic/stack-monitoring:

  • beat
  • elasticsearch
  • enterprisesearch
  • kibana
  • logstash
  • platform_observability

Activity

  1. added
    Team:FleetFleet team [elastic/fleet]
    Team:Elastic-AgentPlatform - Ingest - Agent [elastic/elastic-agent]
    Team:Asset MgtSecurity Assets Management team [elastic/security-asset-management]
    Team:EcosystemPackages Ecosystem team [elastic/ecosystem]
    Team:Cloudnative-MonitoringCloud Native Monitoring team [elastic/obs-cloudnative-monitoring]
    Team:Cloud SecurityCloud Security team [elastic/cloud-security-posture]
    Team:Security-Service IntegrationsSecurity Service Integrations team [elastic/security-service-integrations]
    Team:Security-Deployment and DevicesDEPRECATED Deployment and Devices Security team [elastic/sec-deployment-and-devices]
    Team:Security-Linux PlatformLinux Platform Security team [elastic/sec-linux-platform]
    Team:Security-Windows PlatformSecurity Windows Platform team [elastic/sec-windows-platform]
    Team:Stack MonitoringStack Monitoring team [elastic/stack-monitoring]
    Team:obs-ds-hosted-servicesObservability Hosted Services team [elastic/obs-ds-hosted-services]
    on Dec 2, 2024
  2. consulthys commented on Dec 19, 2024

    @consulthys
    Contributor

    @elastic/stack-monitoring:

    • beat[ ] elasticsearch[ ] enterprisesearch[ ] kibana[ ] logstash[ ] platform_observability

    Hey @taylor-swanson thanks for initiating this. What is the deadline for this change?
    Also, can you share whether this should be done before or after https://github.com/elastic/ingest-dev/issues/4484 ?

  3. taylor-swanson commented on Dec 19, 2024

    @taylor-swanson
    ContributorAuthor

    Hey @consulthys!

    While it is nice to have ECS updated, it isn't critical it needs to be done immediately, unless an integration needs to take advantage of a change in the new version. We haven't issued a new ECS release in a while (8.11.0 was the previous version), but at least back then, we'd work on getting the integrations updated within a short period of time (perhaps a week or so?) after the release.

    The ecs-update tool (see the issue description above) definitely helps make the process of updating integrations less painful.

    Regarding how it relates to the issue you linked, that's actually a good question. As far as I know, I don't think 9.0 will have any issues using a 8.16 (or any 8.x version) ECS. There isn't a strong version enforcement like there is for kibana.version in a package manifest. I'll raise that question with the ECS team, though. If we do need a 9.0 version, we'll for sure get another issue out like this one.

  4. changed the title [-][meta] Upgrade integrations to ECS 8.16[/-] [+][meta] Upgrade integrations to ECS 8.17[/+] on Jan 8, 2025
  5. taylor-swanson commented on Jan 8, 2025

    @taylor-swanson
    ContributorAuthor

    After discussing with the ECS working group, we've decided to hold off on upgrading to 8.16.0 due to link rendering issues. We will be fixing these issues for 8.17.0 and then work can resume on upgrading integrations. If the integrations you are working on don't have any issues with links and you would still like to upgrade to 8.16.0, that is okay.

    I don't have a date for the ECS release, but I'd imagine it's relatively soon given that the 8.17 stack is already out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Team:Asset MgtSecurity Assets Management team [elastic/security-asset-management]Team:Cloud SecurityCloud Security team [elastic/cloud-security-posture]Team:Cloudnative-MonitoringCloud Native Monitoring team [elastic/obs-cloudnative-monitoring]Team:EcosystemPackages Ecosystem team [elastic/ecosystem]Team:Elastic-AgentPlatform - Ingest - Agent [elastic/elastic-agent]Team:Elastic-Agent-Data-PlaneAgent Data Plane team [elastic/elastic-agent-data-plane]Team:FleetFleet team [elastic/fleet]Team:Security-Service IntegrationsSecurity Service Integrations team [elastic/security-service-integrations]Team:Stack MonitoringStack Monitoring team [elastic/stack-monitoring]Team:obs-ds-hosted-servicesObservability Hosted Services team [elastic/obs-ds-hosted-services]meta

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions