Docs audit — request logs — 2026-08-17 #69

Audited: the request logs surface of mnfst/manifest at main 633455b98 (v6.18.0), against the docs at 7a5cdbb.

Scoped run, topic request logs. Pages in scope: request-logs.mdx, observability.mdx, reference/telemetry.mdx. Platform window: the whole logs and data/telemetry surface at main, plus the PRs that built it (#2317 removed body storage in v6.12.0; #2565 per-attempt logs and storage backends, #2627 logs on by default for new harnesses, and #2678 API surface on the request all shipped in v6.18.0).

Findings

1 · [WRONG] The data page says inline image data is stripped out. A stored log keeps the images, and everything else the request carried, exactly as sent

Page: https://manifest.build/docs/reference/telemetry (What Manifest stores, the first paragraph)

What actually happens:

1. You turn logs on for a harness.
2. Your agent sends a request carrying an inline image, a token, a customer record, anything.
3. Manifest stores the exact body it forwarded to the provider: images included, nothing removed, nothing shortened, no size limit.
4. Anyone with access to that workspace's Requests page reads it back word for word on the attempt's Raw tab.
5. The stripping the page describes happens on a throwaway copy Manifest uses to pick a model. That copy is never stored and never sent anywhere.

Options:

1. Drop the stripping claim from the metadata paragraph, and state in the bodies paragraph that a stored body is kept exactly as sent. Recommended (this is the privacy page: the current order makes the reader apply the protection to stored bodies).
2. Keep the sentence but scope it to the model-choosing step, and add one line to the bodies paragraph saying stored bodies are unfiltered and uncapped.
3. Drop the stripping claim only, and leave the bodies paragraph as it is.

Suggested wording (option 1), replacing the paragraph at reference/telemetry.mdx:14:

Manifest always keeps metadata about each request: model, provider, tier, token counts, cost, latency, and an error message. Error text is scrubbed for secrets before it lands, and the request headers Manifest stores drop credentials and IP addresses. The usage and cost views in the dashboard are built from that metadata alone. See Observability for what the record looks like in practice.

and replacing the paragraph at reference/telemetry.mdx:18:

A stored body is exactly what went over the wire: every message, system instruction, tool definition and inline image, at full length. Manifest doesn't redact or truncate it, and there is no size limit. Stored bodies are held outside the database, in the object storage your instance is configured for, and deleted on a retention schedule. With logs off for a harness, that harness's prompts and completions stay between you and the provider.

2 · [MISSING-CASE] The retention table promises Pro a year. When the subscription stops being active, the workspace drops to the 7-day window and the next nightly cleanup deletes the rest

Page: https://manifest.build/docs/request-logs (Retention, the deployment table)

What actually happens:

1. You are on Pro, and your logs are kept for a year.
2. Your card fails, or you cancel. The subscription is no longer active and no longer in a trial.
3. The nightly cleanup stops treating the workspace as Pro.
4. On its next run, in one pass, it deletes every stored body older than 7 days.
5. The requests stay listed with their cost and model. The conversations are gone, and nothing in the dashboard says they expired: the tabs are simply not there any more.

Options:

1. Add one line under the table: the Pro window applies while the subscription is active or trialing, and a lapsed one falls back to the free window on the next nightly cleanup. Recommended (smallest edit, covers cancellation and payment failure at once).
2. Replace the two Cloud rows with rows keyed on the subscription's state: active or in trial, canceled, payment failed, no subscription.
3. Say it in the sentence about what the cleanup removes rather than in the table.

Suggested wording (option 1), inserted at request-logs.mdx:66, between the table and "Self-hosted installs can override this...":

Cloud Pro keeps 365 days only while the subscription is active or in a trial. A canceled workspace, or one whose payment failed, drops to 7 days, and the nightly cleanup applies that on its next run, deleting everything older than 7 days.

3 · [MISSING-CASE] A self-hosted install can show logs switched on everywhere and store nothing, with no sign of it in the dashboard

Page: https://manifest.build/docs/request-logs (Self-hosted storage, around the warning about partial storage settings)

What actually happens:

1. You configure log storage and get one value wrong: a bucket with no region, one half of a key pair, a mistyped storage mode. Or you set the mode to off on purpose.
2. Manifest starts normally and serves every request. Nothing fails.
3. Each harness's Settings page still shows the switch on, and flipping it still confirms logs are enabled.
4. Requests keep appearing on the Requests page, and their attempts have no Messages, Tools or Raw tab, exactly as if logs had never been turned on.
5. The only statement of the problem is one warning line in the server's own startup output.

Options:

1. Add a short "Check it's working" step to the self-hosted storage section: the warning line to grep for, and the symptom in the dashboard. Recommended (turns a silent multi-week data loss into a 30-second check).
2. Extend the existing warning with one sentence saying the dashboard cannot show this, and naming the symptom.
3. Leave it: treat the existing warning about partial settings as sufficient.

Suggested wording (option 1), inserted at request-logs.mdx:119, after the "Full variable list" line and before "Platforms without a persistent disk". The command is Compose-specific; on another host the reader greps their own process output:

Check it's working

A storage problem never stops traffic, and the dashboard never mentions it. Flipping Enable logs on still confirms the change, and attempts come back without Messages, Tools, or Raw tabs, exactly as if logs were off. The server output is the only place the problem shows.

docker compose logs manifest | grep "Request recording storage"

A line reading Request recording storage unavailable: <reason> means Manifest is storing nothing, and the reason names what to fix. No such line means the backend resolved.

4 · [WRONG] Both pages say a request Manifest blocked itself has zero attempts. The Attempts list actually shows one entry for it

Page: https://manifest.build/docs/observability (Requests and provider attempts, the paragraph after the table) and https://manifest.build/docs/request-logs (What gets recorded, the last paragraph)

What actually happens:

1. Your agent sends a request and Manifest rejects it itself: a usage limit, a rate limit, a malformed body, an internal failure.
2. The request appears on the Requests page like any other.
3. Opening it shows an Attempts list with one entry, labelled with the model you asked for, or "No provider" when the request carried none, and the failure status.
4. That entry has a Details tab, plus a Request headers tab when headers were sent. No Messages, Tools or Raw tab, because no provider was called and no conversation was stored.
5. The "Manifest rejected this request before contacting a provider" screen that the zero-attempt wording describes appears only when the request truly has no attempt at all, which is not the case for any of the blocks the two pages give as examples.

Options:

1. Correct both sentences: a blocked request still shows one entry in the Attempts list, and that entry has no conversation to read. Recommended (keeps the point each page was making, fixes the count).
2. Fix only the request-logs sentence, where the missing tabs are explained, and drop the example from the observability counting rule.
3. Replace the example on both pages with the fallback and Autofix case, which is the one the counting rule really needs.

Suggested wording (option 1), replacing the paragraph at observability.mdx:25:

A request that succeeds on the first try is one Request with one attempt. A request that fails twice before succeeding is still one Request, with three attempts. A request Manifest blocked itself, like a hard limit or a malformed body, is one Request with one attempt that never reached a provider. That attempt is listed under the model you asked for, or No provider when the request named none, and carries the error that stopped it.

and replacing the paragraph at request-logs.mdx:28:

A request Manifest blocked itself, on a hard limit or a malformed body, still lists one attempt, labelled with the model you asked for or No provider. It carries the failure status and no Messages, Tools, or Raw tab: Manifest sent nothing to a provider, so there is nothing to read back.

5 · [MISSING-CASE] The telemetry field table does not match the report that is actually sent: agent groupings can arrive as combined keys, three fields carry an "unknown" bucket the table never names, and the message count counts provider calls

Page: https://manifest.build/docs/reference/telemetry (What gets sent, the field table)

What actually happens:

1. You point the report at your own collector, exactly as the page describes.
2. Agent groupings arrive with combined keys for anything classed as "other", the category and the word "other" joined, not the bare names the example shows.
3. The provider, tier and auth-type groupings can each carry an "unknown" bucket, for activity Manifest blocked before a provider or tier was picked. The table names only the "custom" bucket.
4. The message count is one per call to a provider, so a request that fell back twice counts three, and blocked requests count as well.
5. A dashboard built from the documented shape mislabels the combined keys, drops the unknown buckets, and reads the count as request volume.

Options:

1. Correct the five rows in place, keeping the examples but showing the real key shapes. Recommended (the page promises a private collector "receives the same payload documented above").
2. Add a short "shape notes" paragraph under the table instead of touching the rows.
3. Fix only the agent grouping row, where a wrong key shape actually breaks a collector, and leave the counting nuance.

Suggested wording (option 1), replacing the rows at reference/telemetry.mdx:42-45:

| `messages_total` | `1284` | Daily activity per install. One count per provider attempt, so a request that fell back twice counts three. Requests Manifest blocked count too |
| `messages_by_provider` | `{"anthropic": 700, "openai": 500}` | Provider mix. Anything we don't recognize collapses to `"custom"`, so self-hosted provider names and URLs stay local. Activity with no provider, like a blocked request, lands in `"unknown"` |
| `messages_by_tier` | `{"default": 900, "simple": 300, ...}` | Routing tier usage. A tier outside the known set collapses to `"other"`, and activity with no tier lands in `"unknown"` |
| `messages_by_auth_type` | `{"api_key": 1200, "subscription": 84}` | API key, paid subscription or local usage. Anything else collapses to `"other"`, and activity with no auth type lands in `"unknown"` |

and the row at reference/telemetry.mdx:51:

| `agents_by_platform` | `{"openclaw": 3, "hermes": 1}` | Which agent clients people use. A harness whose platform is `other` is keyed by its category instead: `personal:other`, `app:other` or `coding:other` |

6 · [OBSOLETE] The data page tells users to turn off "message recording" on the Settings page. The product calls it Logs, and the same page says so two paragraphs earlier

Page: https://manifest.build/docs/reference/telemetry (What Manifest stores, the note at the end of the section)

What actually happens:

1. You decide one harness should not have its prompts stored.
2. The note tells you to turn message recording off on that harness's Settings page.
3. On that page nothing is called message recording: the section is Logs, the switch is Enable logs.
4. Two paragraphs above, the same page already sends you to Settings → Logs.

Options:

1. Align the note with the shipped names and with the paragraph above. Recommended.
2. Same, and drop the old name from the page's search keywords.
3. Leave it: the old name still helps people who search for it.

Suggested wording (option 1), replacing the note at reference/telemetry.mdx:20-24:

<Note>
  If a harness handles data you'd rather Manifest never hold onto, switch
  **Enable logs** off in its **Settings → Logs** section. Metadata collection
  isn't affected either way.
</Note>

7 · [IMAGE] The Messages tab, the screen the whole feature exists for, has no screenshot

Page: https://manifest.build/docs/request-logs (Reading it back, directly under the paragraph describing the conversation view and its side rail)

The shot must show one attempt's Messages tab: the turn rail on the left with its filter box and role pills, one turn selected, and the conversation card open in the main pane.

Reply with the finding number and the chosen option
(e.g. "3 → option 2", or "3: other idea, let's discuss").
PRs are then made manually with the agent, never by the harness.
Rejected findings: reply "N → reject" and they will never be re-raised.