Repository navigation
Simplify container log parsing and handle timestamped JSON - #1243
Merged
mmguero merged 1 commit intoOct 15, 2026
Conversation
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Refactor the container-log formatter to isolate JSON timestamp extraction, Supercronic event rendering, Filebeat event rendering, and per-service routing. This reduces
ProcessLogLinecyclomatic complexity from F (41) to B (9) as measured by Radon, while retaining its existing formatting and sanitization behaviour.The refactor also fixes a reproducible exception: when a container log includes an ISO-8601 timestamp before a message, the original code calls
messageStrSplit[1:].join(' '). Lists do not have ajoinmethod, so timestamped JSON and timestamped plain-text lines crash. Usepartition(' ')to extract the timestamp safely, preserving the remaining text. A JSON timestamp field continues to take precedence over the outer timestamp.Related issue(s)
Refs #765 (cyclomatic-complexity reduction in Malcolm's Python scripts). The timestamp parsing bug is fixed in the same localized function.
Type of change
How has this been tested?
13 focused tests passed, including four cases that fail on unchanged upstream
maindue to the timestamped-line exception. Other tests cover ordinary JSON and plain-text logs, ISO timestamp handling and precedence, Filebeat summaries and noisy-key removal, Dashboards rendering, Supercronic success/start suppression, debug output, job errors and credential redaction.Radon reports
ProcessLogLine41/F → 9/B, with each extracted helper rated A or B. Both files pass diff checks; the new tests pass Ruff correctness lint. The changed source passes targeted Ruff checks excluding unrelated, pre-existing import/unused-variable and bare-exception findings elsewhere inmalcolm_common.py.The wider installer unit suite was run locally: 72 tests, 4 failures and 6 errors. Those same 4 failures and 6 errors occur on unmodified upstream
main(59 tests) in unrelated configuration/validation suites; no additional failures were introduced. Real container deployments, PCAP ingest, and end-to-end log pipelines were not exercised.Documentation
Was AI used in writing the code, tests, or description for this pull request?
ChatGPT assisted with identifying the bug, refactoring the formatter, writing the regression cases and drafting this description. I inspected the result and ran the tests and checks described above.
Additional context
No new dependencies or configuration options are introduced. The existing log redaction and ignore filtering are preserved.