You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Opt-in, local-only Observe diagnostics for investigating high-retention results without collecting command arguments, paths, search patterns, URLs, or output contents.
Implemented
Disabled by default; enable with CONTEXTCRAWLER_OBSERVE_DIAGNOSTICS=1.
Writes bounded JSONL under the platform-local ContextCrawler data directory. Fixed-schema fields are allowlisted tool/filter labels, static outcome reasons, input/output byte and line counts, available grep match/group counts, and repeated non-empty line counts.
Duplicate-line analysis samples at most 10,000 non-empty lines per input/output and marks sampled counts.
Unix files/directories use private permissions; the log rotates at 5 MiB and retains one previous file.
Diagnostics write failures warn on stderr but do not alter the Observe result.
Documented the enable/disable, location, privacy boundary, sampling, and cleanup.
Validation and deployment
Unit and real-binary tests verify the fixed schema, no content/path/command serialization, output identity with diagnostics enabled/disabled, write-failure behavior, permissions, rotation, and bounded sampling.
Deployed from source to Brain and Pinky; verified the installed binaries, persistent opt-in, and live synthetic records on both hosts.
cargo test --all-targets, cargo test (including doctests), cargo clippy --all-targets, rustfmt, and git diff --check passed.
Added contextcrawler observe-report for reviewing the opt-in trace. By default it summarizes the active and rotated JSONL logs; --input <file> supports a copied log. It reports timestamp range, byte/line totals, repeated-line and match counts, sampled records, and breakdowns by allowlisted tool/filter/reason labels. Unknown labels map to other; raw JSON values and paths are not printed. It rejects non-regular or >5 MiB input files and reports malformed records explicitly. Integration tests cover active+rotated aggregation and malicious labels not leaking into report output.
Deployment follow-up: the first live report exposed older diagnostics records missing duplicate-count and/or sampling fields. The parser now accepts the earlier record shapes and reports repetition/sampling coverage separately, rather than rejecting the entire log or presenting unavailable metrics as zero. Verified contextcrawler observe-report on Brain and Pinky against their existing logs. Those 3 Brain / 5 Pinky records are still synthetic deployment probes; no representative real Observe traffic has accumulated yet.
Activity
thehoff commentedon Oct 17, 2026
Added
contextcrawler observe-reportfor reviewing the opt-in trace. By default it summarizes the active and rotated JSONL logs;--input <file>supports a copied log. It reports timestamp range, byte/line totals, repeated-line and match counts, sampled records, and breakdowns by allowlisted tool/filter/reason labels. Unknown labels map toother; raw JSON values and paths are not printed. It rejects non-regular or >5 MiB input files and reports malformed records explicitly. Integration tests cover active+rotated aggregation and malicious labels not leaking into report output.thehoff commentedon Oct 17, 2026
Deployment follow-up: the first live report exposed older diagnostics records missing duplicate-count and/or sampling fields. The parser now accepts the earlier record shapes and reports repetition/sampling coverage separately, rather than rejecting the entire log or presenting unavailable metrics as zero. Verified
contextcrawler observe-reporton Brain and Pinky against their existing logs. Those 3 Brain / 5 Pinky records are still synthetic deployment probes; no representative real Observe traffic has accumulated yet.