fix(observability): repair Grafana Tempo trace pipeline

[JSONbored]

Parent: #1819

Context

The self-host stack includes OTEL collector, Tempo, and Grafana provisioning so operators can inspect review traces. Current operator experience reports Grafana errors when attempting to view traces, which blocks the intended latency/debugging workflow.

This is separate from Sentry error reporting. Sentry remains optional; Grafana/Tempo traces are the local self-host observability path.

Requirements

• Reproduce the Grafana trace-viewing error in the self-host stack.
• Verify the complete local trace path: app or CLI emits OTLP traces, OTEL collector receives them, collector exports to Tempo, Tempo stores them, Grafana datasource can query them.
• Check Grafana Tempo datasource provisioning, Tempo config, OTEL collector config, service names, ports, profiles, and trace IDs in logs/metrics.
• Confirm whether traces are expected only when OTEL_TRACES_EXPORTER=otlp is set, and make the disabled state clear in docs/UI.
• Add actionable logs or health panels for trace pipeline failures where practical.
• Keep all trace attributes scrubbed: no prompts, raw diffs, review bodies, tokens, auth headers, private repo config, or local auth paths.

Deliverables

• Root-cause analysis for Grafana trace errors.
• Fixes to OTEL collector, Tempo, Grafana datasource, app trace emission, or docs as needed.
• Smoke test or reproducible manual check that creates a trace and opens it in Grafana.
• Documentation updates for enabling and troubleshooting traces.
• Optional dashboard/resource-hub update that shows whether Tempo has recent spans.

Acceptance criteria

• With observability enabled and traces configured, at least one self-host review/job produces a trace visible in Grafana Tempo Explore.
• When traces are disabled, Grafana/docs make that state understandable instead of looking broken.
• Trace pipeline failures are diagnosable from logs, metrics, or a troubleshooting section.
• No sensitive review payloads or secrets appear in trace attributes.

Validation

• Boot the observability profile and verify OTEL collector, Tempo, and Grafana are healthy.
• Emit a controlled test trace or run a review/job with tracing enabled.
• Open the trace in Grafana and confirm spans render without datasource errors.
• Run focused self-host OTEL/Grafana tests and docs/UI validation if files change.

Ownership

Maintainer-only. No gittensor:* labels.