Skip to content

Wait for positive document counts before reporting OpenSearch logs ready - #1232

Merged
mmguero merged 1 commit into
cisagov:contributor-prsfrom
sylvesterkaczmarek:fix/opensearch-wait-for-positive-doc-counts-20261013
Oct 13, 2026
Merged

mmguero merged 1 commit into
cisagov:contributor-prsfrom
sylvesterkaczmarek:fix/opensearch-wait-for-positive-doc-counts-20261013

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown
Contributor

Description
opensearch_status.sh -w can report that logs exist when an index has an absent, null, invalid, or numeric-zero document count. The JSON branch compares the value only with the string "0"; the text fallback assumes a populated seventh column and also checks string inequality.

Require a positive numeric document count in both branches. The text fallback explicitly requests h=docs.count to avoid reading a shifted column. Keep the primary-index and distinct Arkime fallback behavior unchanged.

Related issue(s)
No existing issue identified. Reproduced against upstream main at aa29d73; no overlapping open PR was found.

Type of change

  • Bug fix
  • New feature or enhancement
  • Documentation
  • Other

How has this been tested?
python3 -m unittest discover -s scripts/installer/tests/unit -p test_opensearch_log_readiness.py -v

All six tests pass, including subcases for absent/null/zero/invalid counts, valid numeric/string counts, the no-jq fallback, and distinct Arkime index patterns. The suite runs the actual shell script and real curl/jq/awk against a loopback-only HTTP fixture; only retry sleeps are replaced. Regression cases fail on unchanged upstream because the script returns before the fixture supplies a positive count.

Bash syntax, Python compilation, Ruff correctness rules (E4/E7/E9/F), and git diff --check pass. Testing used macOS ARM64; no live OpenSearch cluster, credentials, or production data were involved.

Documentation

  • No user-documentation update is needed; this restores the documented wait-for-logs behavior.

Was AI used in writing the code, tests, or description for this pull request?

  • No
  • Yes

ChatGPT (OpenAI) generated the patch, regression tests, and this description. Desktop Commander and GitHub CLI were used to inspect the upstream code, check for duplicates, compare diffs, reproduce failures before the fix, and run the tests above. Maintainer review is pending.

Additional context
The explicit CAT column selection follows the OpenSearch CAT Indices API. No new dependencies or public endpoints are introduced.

@mmguero mmguero self-assigned this Oct 13, 2026
@mmguero mmguero added the opensearch Relating to Malcolm's use of OpenSearch label Oct 13, 2026
@mmguero mmguero added this to the v26.10.0 milestone Oct 13, 2026
@mmguero mmguero added this to Malcolm Oct 13, 2026
@mmguero mmguero moved this to Review in Malcolm Oct 13, 2026
@mmguero
mmguero changed the base branch from main to contributor-prs October 13, 2026 16:57
@mmguero
mmguero merged commit 2ff9e8a into cisagov:contributor-prs Oct 13, 2026
@github-project-automation github-project-automation Bot moved this from Review to Done in Malcolm Oct 13, 2026
@mmguero mmguero mentioned this pull request Oct 29, 2026
@mmguero mmguero moved this from Done to Released in Malcolm Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

opensearch Relating to Malcolm's use of OpenSearch

Projects

Status: Released

Development

Successfully merging this pull request may close these issues.

2 participants