Skip to content

Docs audit — introduction (getting started) — 2026-08-07 #49

Closed

Description

@SebConejo

Scoped audit: getting started (introduction page). Docs at 09540c8, platform at e82126878. Pages checked: introduction.mdx and the pages it references.

Correct and untouched: the OpenAI/Anthropic-compatible endpoint, the four provider families, the Autofix and Observability cards, Cloud vs self-hosted restrictions, no demo credentials, no internal names.

Scope note: findings 2 to 5 target pages the introduction links to, not introduction.mdx itself. They stay here because they are already raised; future scoped audits keep findings inside their own page list (out-of-scope observations now go to a backlog).

Findings

1 · [OBSOLETE] The introduction card still says "Message recording"

Page: https://manifest.build/docs/introduction (Key features, the feature cards)

Options:

  1. Rename it "Request logs" (the title of the page it opens). Recommended.
  2. Rename it "Logs" (the dashboard's word).
Evidence (for the agent)
  • introduction.mdx:29 — <Card title="Message recording" icon="messages-square" href="/request-logs">
  • Target page title: "Request logs" (request-logs.mdx:2).
  • The old name also survives in: observability.mdx:123, autofix.mdx:186, deploy/fly.mdx:75, deploy/railway.mdx:59, deploy/render.mdx:67. Same PR.

2 · [WRONG] The self-hosting page says "the first account you create becomes the admin", implying an open signup. In reality there is no signup: first access takes you to a setup screen where you create the one admin account

Page: https://manifest.build/docs/self-hosted (intro paragraph and both install tabs)

What actually happens:

  1. You install Manifest and open the dashboard for the first time.
  2. Manifest detects the empty instance and takes you to a setup screen titled "Create the admin account for your instance".
  3. You fill in email and password. That account is the admin. There is no other signup path.

Options:

  1. Write "on first access, Manifest takes you to a setup screen where you create the admin account" in the three spots. Recommended.
  2. Keep the current wording, if you consider the setup screen close enough to a signup form.
Evidence (for the agent)
  • self-hosted.mdx:14 and :72 — "The first account you create becomes the admin."; :42 — "sign up for an account". Correct wording at :323 and reference/environment-variables.mdx:48.
  • Code: empty database → redirect to /setup, createFirstAdmin (setup.service.ts:84-157, Setup.tsx:68,127).

3 · [WRONG] The doc says an unknown model name "falls back to your routing config". False: the request fails with M302

Page: https://manifest.build/docs/self-hosted (First request, the "Send a request" step)

What actually happens:

  1. Your app sends a request with a model name.
  2. If the name is auto, Manifest picks the model for you (routing).
  3. If it is a real model name, Manifest sends the request to that exact model, no substitution.
  4. If that model is unreachable, the request fails with error M302. Nothing falls back.

Options:

  1. Replace with the sentence llm-gateway.mdx:89 already uses: unlisted model ID → M302, send auto for routing. Recommended.
  2. Delete the false half of the sentence, keep the rest.
Evidence (for the agent)
  • self-hosted.mdx:150 — "Any other name is treated as an explicit choice, and falls back to your routing config if it matches nothing you have connected."
  • Code: proxy.service.ts:899-916 (resolution), :275-287 (M302 raise). Dev comment: "Every public proxy surface treats a concrete model as an explicit route." No fallback path for explicit models.
  • Reproduced live on Cloud (2026-08-07): POST /v1/chat/completions with "model": "gpt-modele-qui-nexiste-pas" → [🦚 Manifest M302] Model "gpt-modele-qui-nexiste-pas" is not available for this agent. Use GET /v1/models.... Same request with "model": "auto" → routed, served by o3.
  • A user believes a typo'd model is caught by routing. It fails.

4 · [WRONG] The fresh-install error table describes M100 with M101's definition, and M101 is missing

Page: https://manifest.build/docs/self-hosted (First request, the error-codes note at the end)

What actually happens:

  1. You install Manifest, create your harness, and send a request before connecting any provider.
  2. The request fails with M101: "You're connected, but no providers are set up yet."
  3. M100 is a different situation: routing picked a provider whose API key is missing.

Options:

  1. Add an M101 row and give M100 its real definition ("routing picked a provider with no key on file"). Recommended.
  2. Replace M100 with M101, keep three rows.
Evidence (for the agent)
  • self-hosted.mdx:163 — "M100 | The key is fine; no provider is connected yet for the model being routed to".
  • Code: error-codes.ts:7-84 (M100 = provider API key missing; M101 = no providers configured); resolve.service.ts:426 (null route → M101).
  • errors/M100.mdx and errors/M101.mdx are already correct; the table contradicts them.
  • Reproduced live on a throwaway local instance (2026-08-07): (a) providers connected but none usable, "model": "auto" → [🦚 Manifest M101] You're connected, but no providers are set up yet.; (b) zero providers → M101 again; (c) provider enabled with a model in catalog but no API key on file, "model": "openai/gpt-4o" → [🦚 Manifest M100] No openai API key yet. The fresh-install error is M101; M100 means a picked provider misses its key. The doc table swaps them.

5 · [WRONG] MANIFEST_MODE is required by four deploy guides and by a product error message, but the environment-variables reference does not document it

Page: https://manifest.build/docs/reference/environment-variables (the variables table), and the guides https://manifest.build/docs/deploy/aws, /deploy/render, /deploy/easypanel, /deploy/koyeb

What actually happens:

  1. Unset, Manifest auto-detects its mode: a Docker, Podman or Kubernetes runtime means self-hosted; anything else means cloud.
  2. On some hosting platforms the container leaves none of those three traces, so a self-hosted install is mistaken for cloud (stricter URL rules, cloud-only behaviors).
  3. That is why the four deploy guides tell you to set MANIFEST_MODE=selfhosted, and why Manifest's own error message says: "set MANIFEST_MODE=selfhosted if you are running Manifest yourself".
  4. A user who meets the variable in a guide or in that error and looks it up in the environment-variables reference finds nothing.

Fix: add a MANIFEST_MODE row to the reference table: values selfhosted, local (legacy alias), cloud; unset = auto-detection (Docker, Podman, Kubernetes); set selfhosted on platforms where detection cannot see the container.

Evidence (for the agent)
  • reference/environment-variables.mdx — no MANIFEST_MODE row (grep negative).
  • deploy/aws.mdx:67, deploy/render.mdx:50, deploy/easypanel.mdx:62, deploy/koyeb.mdx:19,54,59 — guides instruct MANIFEST_MODE=selfhosted.
  • Code, detection: detect-self-hosted.ts — priority: explicit MANIFEST_MODE wins; else /.dockerenv, /run/.containerenv, or KUBERNETES_SERVICE_HOST → self-hosted; else cloud.
  • Code, user-facing error: url-validation.ts:156-162 — "Only https URLs are allowed in cloud mode. Self-hosted installs allow http:// for private destinations — set MANIFEST_MODE=selfhosted if you are running Manifest yourself (e.g. inside Docker, Podman, or Kubernetes)."
  • The variable was first classified internal-only by the engineer report; wrongly: it is printed in public docs and in product output, so it is public API surface.

6 · [MISSING-CASE] The free plan quota (10,000 requests/month) is written nowhere except inside an error page

Page: https://manifest.build/docs/introduction (Cloud or self-hosted?, the plan-limits sentence)

What actually happens:

  1. On Cloud Free, the sidebar shows a meter: "N / 10,000 requests" for the month.
  2. At 10,000 the requests are blocked with error M204, which invites you to upgrade to Pro.
  3. The counter resets at the start of the next month.

Options:

  1. Add one sentence to the introduction: "The free plan includes 10,000 routed requests per month." Recommended.
  2. Leave it: the number stays in one place (errors/M204) and never goes stale elsewhere.
Evidence (for the agent)
  • introduction.mdx:51 — "Cloud is also subject to plan limits". No number.
  • Only occurrence: the error message quoted in errors/M204.mdx:13.
  • Code: plan-limits.ts:9-16 (10,000/month), PlanPicker.tsx:6-21.

Reply with the finding number and the chosen option
(e.g. "3 → option 2", or "3: other idea, let's discuss").
PRs are then made manually with the agent, never by the harness.
Rejected findings: reply "N → reject" and they will never be re-raised.

Activity

  1. added
    docs-syncFiled by the docs-sync harness
    obsoleteDocs describe something removed
    internal-leakInternal info exposed to users
    wrongDocs say it incorrectly
    missing-caseA user situation is not covered
    on Aug 7, 2026
  2. added a commit that references this issue on Aug 7, 2026
  3. SebConejo commented on Aug 7, 2026

    @SebConejo
    MemberAuthor

    All findings accepted by Seb. Resolution PR: #50 (all recommended options applied; findings 3 and 4 backed by live reproductions). The issue closes when the PR merges.

  4. added a commit that references this issue on Aug 7, 2026
  5. SebConejo commented on Aug 7, 2026

    @SebConejo
    MemberAuthor

    Correction on finding 5: rejected by Seb after discussion. The in-place instructions (deploy guides and the product's own error message) are sufficient; no reference row is added. PR #50 updated accordingly. Findings 1-4 and 6 remain applied.

  6. added a commit that references this issue on Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    docs-syncFiled by the docs-sync harnessinternal-leakInternal info exposed to usersmissing-caseA user situation is not coveredobsoleteDocs describe something removedwrongDocs say it incorrectly

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions