Skip to content

[NVBug 6534797] Finish polish items on model-call-capture.mdx docs #4494

Closed

Description

@qiaochuz-nv

Queue rank 7/48 · Difficulty 2/5 · Status on main: partially_fixed

NVBug: https://nvbugspro.nvidia.com/bug/6534797 · NVBug priority: P1-Should have · ARB: Ananth Subramaniam
GitHub: Fix PR #2415 · Epic (tangential, closed) #1434

Summary

Docs-only bug: a list of requested improvements to fern/.../model-server/model-call-capture.mdx. PR #2415 (commit 3260c981e, 2026-08-10) rewrote the page and landed after the last bug comment (2026-08-06), so most requests are already done on main. A few low-priority polish items remain, and the NVBug is still open.

Reproduction

Read fern/versions/latest/pages/model-server/model-call-capture.mdx and .../model-server/index.mdx.

Root cause

Original 0.5.0 page lacked a copy-paste run workflow, led with author-only helper APIs, lacked JSON samples, had an oversized "Agent observations" section, used jargon ("kill-shaped"), and was not linked from the model-server index.

Status on main (partially_fixed)

Fixed by PR #2415 and follow-ups, verified in current main (HEAD 75bb53e):

  • Copy-paste workflow: "## Run an evaluation with capture" (lines 16-61).
  • Consumer-first; helpers moved to "## If you build a custom agent" (lines 235-253).
  • Samples: ng_model_call_capture with gaps + full metrics (lines 86-126); raw CaptureStore JSONL (lines 148-176).
  • "Agent observations" trimmed to "## Add agent-side evidence" linking rollout-evidence (lines 255-265).
  • Index card added: index.mdx "## Observe model calls" (lines 61-71).
  • Regression test tests/unit_tests/test_fern_docs_links.py guards the above.
    Still open (minor):
  • "kill-shaped" is removed from latest/v0.5.0 but still present in the v0.5.1 snapshot (model-call-capture.mdx:154).
  • model_call_capture_dir Default cell still reads "required when enabled" (line 213); bug asked for a dash + "Required when observability_enabled is true".
  • No cross-link from "top-level global-config fields" (line 208) to reference/configuration.mdx.
  • Title still "Model-call capture" (line 2) vs requested "Model-Call Capture"; maintainer deliberately encoded lowercase in the index-card test, likely WONTFIX.
  • anthropic-messages.mdx mixes "model-call capture" / "Model-call capture" (lines 81, 141).

Proposed fix

In model-call-capture.mdx (and synced snapshots v0.5.0/v0.5.1 the test checks): set Default to "—" and move "Required when observability_enabled is true" into Description; link "global config" to /main/reference/configuration; remove the lingering "kill-shaped" in v0.5.1. Optionally align Anthropic-page link casing. Confirm title-casing intent with maintainer before touching it (also requires updating the index card and the test assertion at line 143). Extend test_fern_docs_links.py with the new assertions.

Validation

QA regression: tests/unit_tests/test_fern_docs_links.py (pytest tests/unit_tests/test_fern_docs_links.py -x) is the only confirmed guard. Run pre-commit run --files fern/**. Docs-only; no model compute.

Triage

Difficulty 2/5 — pure docs cleanup, root cause pinpointed, but spans several synced .mdx files plus a test assertion. Needs GPU/model: no. External dependency: none.

Reviewer notes

Verifier confirmed partially_fixed and difficulty 2. New finding folded in: "kill-shaped" is NOT fully removed — it survives in the v0.5.1 snapshot (model-call-capture.mdx:154), so the draft overstated removal. Softened the Validation claim: the only confirmed regression guard is tests/unit_tests/test_fern_docs_links.py (the "testcases/ reference this bug" claim was unverified). PR #2415 provenance and remaining open items confirmed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions