perf(cli): verbose prompt rendering hangs / OOMs on large markdown prompts due to Rich markup parsing

[hertznsk]

Summary

When conductor run logs a rendered prompt in verbose mode, it passes the raw prompt string to rich.Panel. If the prompt contains markdown with many [...] brackets (e.g. Renovate / Node.js changelogs with links like [text](url) or [`hash`](url)), Rich interprets the brackets as markup tags. This creates thousands of unmatched style spans and triggers catastrophic CPU/memory scaling in rich.markup.render / rich.text.Text.render, eventually hanging the process or getting OOM-killed.

The same prompt with --quiet runs in seconds without any memory spike.

Reproduction

A minimal reproduction renders a realistic markdown block through rich.Panel:

import time
from rich.console import Console
from rich.panel import Panel

# Realistic Node.js-style changelog line with many brackets
line = (
    "- **crypto**: harden KeyObject internal slots (Filip Skokan) "
    "[[`ce84aef37d`](https://github.com/nodejs/node/commit/ce84aef37d)] "
    "[#63111](https://github.com/nodejs/node/pull/63111)\n"
)

# 100 KB of such markdown
content = line * 1000

console = Console(force_terminal=False, width=80)
start = time.perf_counter()
console.print(Panel(content, title="Prompt", border_style="dim"))
print(f"rendered in {(time.perf_counter() - start) * 1000:.1f} ms")

In Conductor the problematic path is:

• src/conductor/executor/agent.py:308 calls _verbose_log_section(f"Prompt for '{agent.name}'", rendered_prompt)
• src/conductor/cli/run.py:379 passes the raw string to _verbose_console.print(Panel(content, ...))

Observed behavior

Measurements on a 80-column console, output redirected to /dev/null:

Synthetic markdown scaling

size	plain text	markdown with brackets	slowdown
10 KB	3 ms	26 ms	9x
25 KB	4 ms	147 ms	33x
50 KB	8 ms	642 ms	78x
100 KB	15 ms	3261 ms	215x

Peak RSS for 100 KB markdown reached 311 MB, while plain text of the same size stayed at 3.6 MB.

Real changelog subsample (/tmp/changelog.md, ~1 MB total)

lines	chars	render time	peak RSS
100	18 427	66 ms	10 MB
500	93 494	2.1 s	208 MB
1 000	189 617	9.7 s	866 MB
2 000	354 497	38 s	3.1 GB
5 707	998 791	>60 s / OOM	>60 s / OOM

A 200 KB real-world prompt (~1 000 lines) already takes 10 seconds and 866 MB just to render the verbose panel. The full 1 MB file exceeds the 60-second timeout and is killed.

Root cause

Console.print(Panel(str, ...)) invokes rich.markup.render on the string because the default Console has markup parsing enabled. The regex RE_TAGS (\[(\\\*)?([#/@a-z][^[]*?)\]) matches every [...] in markdown links and code spans. Unmatched tags are converted into style spans, and Text.render then performs expensive sorting/stack operations over those spans.

cProfile shows the hot path:

ncalls  tottime  cumtime  filename:lineno(function)
...     ...      11.490   rich/segment.py:309(split_and_crop_lines)
...     ...      10.497   {method 'extend' of 'list' objects}
...     ...      10.288   rich/console.py:1300(render)
...     ...       9.588   rich/panel.py:141(__rich_console__)

With bracket-heavy markdown the span count explodes, turning the per-line wrap/render loop into effectively quadratic or worse behavior.

Expected behavior

Verbose logging should not make a workflow unusable just because the prompt contains markdown links. The console preview should render quickly and with bounded memory, regardless of prompt size.

Possible fixes

1. Disable markup parsing for prompt content — the smallest fix. Wrap the content in rich.text.Text(content) before passing it to Panel. This bypasses rich.markup.render entirely and avoids the span explosion.

   from rich.text import Text
   _verbose_console.print(Panel(Text(content), title=f"[cyan]{title}[/cyan]", border_style="dim"))

2. Truncate console preview — large prompts should not be dumped in full into the terminal scrollback. A preview of, say, 10 000 characters is enough for debugging and keeps rendering fast.

3. Gate full prompt logging behind a separate debug flag — keep the default verbose output short, and only write the complete prompt to the file log (--log-file) or when CONDUCTOR_LOG_PROMPTS=1 is set.

4. Use markup=False / highlight=False on the content print call as a defensive default.

External alternatives such as manual box drawing or Textual's virtualized viewport were also considered, but option 1 is the minimal change.

Workaround for users

Run with --quiet:

conductor --quiet run workflow.yaml

This skips the verbose panel rendering entirely and avoids the hang / memory spike.

[hertznsk]

Just in case: I understand that it's not normal to cram 6,000 lines into a prompt.

[jrob5756]

Fixed on main by #387.

#387 targeted the MarkupError crash in #382, but it removed this issue's root cause at the same time, since both come from Rich parsing [...] in agent-supplied text as console markup. Both sinks are covered: the console path now renders Panel(Text(content), ...), and init_file_logging builds its console with markup=False.

Measured on main at abe55b5, using the reproduction from this issue:

content	before	now
175 KB	63 s, 993 MB	0.31 s, 2.8 MB
1 MB	OOM / >60 s timeout	2.2 s, 15 MB

The file log keeps bracketed tokens and link URLs verbatim, so the full prompt is still recoverable with --log-file.

The 1 MB case is worth calling out: it could not complete at all before, and it is the size that motivated the report.

Regression coverage is in tests/test_cli/test_markup_safety.py. It guards the quiet failure too, not just the crash: opening tags like [bold] are consumed by Rich without raising, so a crash-only suite could not distinguish a real fix from one that silently drops agent text.

Thanks hertznsk for the detailed profiling and scaling table here. They made verifying #387 straightforward.