Skip to content

Logs viewer: more detail and history for troubleshooting searches, downloads, and post-processing #1007

Open
3 of 4 issues completed

Description

@codynerren-dotcom

Describe the request
The Settings → Logs page is a big help (the Warning/Info/Debug dial, the view filter, copy). When I'm trying to figure out why an issue wasn't found or didn't import, though, I still end up in the container's log files. Things that would let me stay in the app:

  • A level beyond Debug, or per-area debug toggles (search/indexers, DDL, download clients, post-processing, metadata), so I can turn up detail for the part I'm chasing without flooding the whole log
  • Text search and filtering by series/issue, provider/indexer, or component, on top of the severity filter
  • Older history: read the rotated files too, or download the full log/archive from the page. Right now it only reads the current comicarr.log (max 5,000 lines), so after a rotation the lines I need are gone
  • More context in the log lines themselves: for each search, which providers/indexers were tried with what query, how many results came back, and why each candidate was rejected; for downloads, each DDL/client step and its result; for post-processing, the source path, destination path, and the step that failed, with the full error
  • Optional: from an issue's page, a link to its recent activity/log lines

How did you come up with the idea?
I troubleshoot grabs that fail or never import (DDL lookups with no results, post-processing permission errors) and always need more context than the Info level shows, plus history from before the last rotation.

Environment

  • Comicarr version: 0.41.4
  • Install method: Docker (compose)
  • OS: ZimaOS
  • Branch / commit: n/a

Additional information
Builds on #611 / #617 (log level dial and Settings log viewer) and relates to #957 (search loop log volume). Per-area verbosity would also help with the volume problem in #957.

Activity

  1. frankieramirez commented on Oct 8, 2026

    @frankieramirez
    Owner

    This was generated by AI during triage.

    Agent Brief

    Category: enhancement
    Summary: Extend Settings → Logs with retained history, focused finding/export and consistent troubleshooting context.

    Current behavior:
    get_recent_logs(ctx, lines) deliberately exposes only the current comicarr.log, capped at 5,000 lines. The viewer already offers severity filtering, Copy, Refresh, rotation and saved/effective/restart level context. Both old/current log formats and multiline traceback severity are supported. #957's bounded INFO search summary is already implemented; the remaining request is not. Rotated-file access was deferred from #617's first cut, not rejected. No matching .out-of-scope/ record exists.

    Desired behavior and delegated decisions:

    • Keep Warning/Info/Debug as the single capture control, following CONTEXT.md and docs/architecture/logging-levels.md. No fourth level or per-subsystem capture toggles.
    • Let the operator select current or retained Comicarr log files. Add literal, case-insensitive text search and component display filters combined with severity, so a Series/issue/provider/query can be found by its logged text.
    • Search the selected file with bounded memory and return the newest matching records within a documented cap, in chronological display order. Preserve traceback blocks when their header or continuation matches; show selected file and loaded/matched/truncated scope honestly.
    • Copy exports the filtered view. Add a redacted download of the selected full retained file, read in bounded chunks; do not present the capped view as a full archive.
    • Enumerate only known Comicarr current/rotation filenames beneath the configured log directory. Accept opaque file selectors rather than client-provided paths, refuse symlinks/nonregular files, and handle rotation/disappearance/unreadable files with a useful error and refresh.
    • Keep all read/search/export operations session protected. Use existing server-side sensitive-text/provider-secret redaction for every output. Describe exported logs as operator diagnostics that can contain library names/paths; they are separate from the public-attachment Support bundle, whose allowlist remains unchanged.
    • At Debug, standardize searchable component/operation, known Series/issue identity, safe provider name, actual query, returned versus consumed counts and evaluated candidate verdict/reason codes. Keep INFO summaries bounded as in Search loop logs ~5 MB/min at INFO; rotation keeps only minutes of history #957.
    • At download/post-processing failure boundaries, include failing step, operation identity, available source/destination paths and full exception traceback. Do not dump authenticated URLs, provider tuples or raw response bodies.
    • Observe EvaluationSession results already consumed by the workflow. Diagnostics must not consume extra candidates, re-query providers, change selection/pack claims or guess totals.

    Key interfaces:

    • get_recent_logs(ctx, lines) and authenticated system log routes: extend with selected retained source and filter scope while preserving current callers.
    • LogsTab / useLogs / logLines: file selection, literal/component/severity filters, multiline rendering, Copy and export.
    • redact_sensitive_text(): common server redaction for historical as well as current output.
    • EvaluationSession.evaluate() / public candidate verdict: observational diagnostics only.
    • postprocessing.run() and download execution failure boundaries: contextual exception logging.

    Acceptance criteria:

    • Current and rotated-file fixtures can be selected, searched and exported. Combined text/component/severity filtering and chronological ordering work across both log formats, unknown lines and multiline tracebacks.
    • A match older than the newest 5,000 unfiltered lines can be found. The UI distinguishes bounded matches from full-file export and reports truncation.
    • Credentials/provider keys are absent from view, Copy and export; full exported files are processed without loading the entire file into memory.
    • Unauthenticated requests, path traversal/arbitrary selectors, symlinks and nonregular files are refused. Missing, rotating and unreadable files produce actionable states rather than misleading empty results.
    • Stubbed multi-provider search records real queries, honest result/consumption counts and final evaluated rejection reasons. Provider calls, candidate consumption, accepted selection and pack claims are unchanged; Search loop logs ~5 MB/min at INFO; rotation keeps only minutes of history #957 INFO bounds remain intact.
    • DDL/client failures and a post-processing permission failure expose operation/failing-step context, known paths and traceback, with sensitive values redacted.
    • Existing live-level, startup-source and rotation behavior remains covered. Relevant backend/frontend tests, frontend typecheck and npm run lint pass; add a changeset and update current-file-only log API documentation.

    Verification: Source inspection confirms current-tail-only behavior and missing viewer find/history/export. Existing Debug logging covers some requested details but failure context is inconsistent; postprocessing.run() logs an exception string without a traceback. This enhancement has not been tested against a live container.

    Out of scope:

    • New capture levels/subsystem capture switches, retention-default changes, a log database, live streaming, or Support bundle expansion.
    • Optional issue-page activity shortcut, deferred from this bounded first delivery.
    • Any acquisition/evaluation behavior change.
  2. added
    ready-for-agentFully specified and ready for an autonomous implementation agent
    and removed
    needs-triageNeeds maintainer evaluation and triage
    ready-for-agentFully specified and ready for an autonomous implementation agent
    on Oct 8, 2026
  3. frankieramirez commented on Oct 8, 2026

    @frankieramirez
    Owner

    Split into four sub-issues so each lands as its own reviewable PR:

    This issue stays open as the parent and no longer carries ready-for-agent.

  4. added
    ready-for-agentFully specified and ready for an autonomous implementation agent
    and removed
    ready-for-agentFully specified and ready for an autonomous implementation agent
    on Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions