Skip to content

Arkime_Capture_Statistics - #640

Merged
mmguero merged 5 commits into
idaholab:stagingfrom
scott-jeffery:main
Sep 4, 2025
Merged

mmguero merged 5 commits into
idaholab:stagingfrom
scott-jeffery:main

Conversation

@scott-jeffery

@scott-jeffery scott-jeffery commented Sep 3, 2025 •

Copy link
Copy Markdown
Collaborator

This PR is the Arkime part of Issue #703

Arkime updates the arkime_stats_v30 index with internally calculated capture statistics. The method used to determine capture statistics depends on the Arkime reader used. For example: the tpacketv3 reader determines capture statistics via a getsockopt call. These statistics are updated in a single cumulative record for each Arkime instance. If Malcolm is run along with a Hedgehog sensor there will be two records in the arkime_stats_v30 index, one for Arkime running under Malcolm and one for Arkime running under Hedgehog. (In reality there will be three records as arkime keeps an initial "dummy" record which we filter out when displayed)

The update interval for each arkime_stats_v30 record varies depending how Arkime is run. Arkime starts a separate thread with several timers that may or may not cause a record update on each pass. Typically Arkime running under the Malcolm profile will update the capture statistics about every 10 minutes while Arkime run under Hedgehog can be as often as every two seconds.

A new visualization pane has been added to the "Packet Capture Statistics" Malcolm dashboard. This required a new Index Pattern (arkime_stats_template.json) to make the index fields available as visualization elements. Since Arkime updates a single cumulative record, rather than adding time-series records like Zeek and Suricata do, the shared-object-creation.sh shell script was updated to handle time-series index patterns as well as non-time-series index patterns with a "null" timeFieldName.

Further work could be done to better understand the Arkime update timing and reader dependent capture data sources. This PR marks the first step in displaying the capture data that Arkime makes available in an index.

@mmguero
mmguero changed the base branch from main to staging September 4, 2025 19:23
@mmguero
mmguero merged commit d018c0c into idaholab:staging Sep 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants