compare replays migrate-time verdicts for non-PromQL panels instead of re-executing, so its ERROR count can never be cleared

[shmsr]

Summary

obs-migrate compare does not re-execute the target query for non-PromQL
panels. For every Datadog panel it replays the comparison.status that
migrate recorded into verification_packets.json, and maps target_broken
onto its own ERROR verdict — labelled "mode": "live_source", with no
timestamp and no hint the evidence is from an earlier run.

The result is a verdict that cannot be refreshed. Fix the target, re-run
compare, and it reports the same failures with the same message and still
exits 1.

This matters because of ordering. On a fresh cluster the documented workflow is
migrate → seed → verify, so the --validate inside migrate necessarily runs
before the target has any data. Every panel is recorded target_broken, and
compare then reports the empty cluster forever.

Reproduction

Against a local 9.6.0-SNAPSHOT stack, 6 live Datadog dashboards (62 panels):

obs-migrate migrate --source datadog --input-mode api --output-dir OUT \
  --es-url http://localhost:9200 --kibana-url http://localhost:5601 \
  --field-profile otel --preflight --validate --upload --ensure-data-views

obs-migrate seed-sample-data --artifact-dir OUT/dashboards \
  --es-url http://localhost:9200 --api-key dummy      # 1573 docs, errors=0

obs-migrate compare --artifact-dir OUT/dashboards \
  --es-url http://localhost:9200 --api-key dummy

{ "panels": 62, "ERROR": 35, "STRUCTURAL": 27 }

One of the 35:

panel:  Currently reporting APM hosts
mode:   live_source
reason: Target runtime validation did not produce a trustworthy runnable result:
        Found 1 problem
        line 3:29: Unknown column [datadog_apm_host_instance]

The column is not unknown

datadog_apm_host_instance exists in the index the panel reads, with data, and
the exact query compare reports runs cleanly — including with the same
?_tstart/?_tend binding, and in a window with no data:

POST /_query   ->  documents_found: 1573, rows_emitted: 6445

Independently, verifier.live_validate over the same artifacts after seeding:

live ES|QL validation: 27 queries | ok=27 data_gap=0 REAL_BUGS=0 other=0

Evidence that nothing is re-executed

1. Monkeypatching parity_oracle._run_query to log any request containing
   datadog_apm_host_instance — never fires during a compare run.
2. Clearing only packets[i].comparison.status (target_broken -> not_run)
   in a copy of verification_packets.json takes the identical run from
   ERROR: 35, STRUCTURAL: 27 to STRUCTURAL: 62, ERROR: 0.

Scrubbing the surrounding recorded fields instead (validation,
target_execution, comparison.counterexamples, and the same blocks in
migration_report.json — 280 blocks in total) changes nothing, which
isolates comparison.status as the single input behind the verdict.

Where

observability_migration/app/cli.py

• _live_source_row (line 1824) reads pkt["comparison"]["status"]
• _LIVE_COMPARISON_VERDICTS (line 1816) maps target_broken -> ERROR
• called at line 1779, on the non-PromQL branch of _run_compare

The PromQL branch above it does execute (_run_query), so the two halves of
compare answer with different freshness under one verdict vocabulary.

Expected

Some way for the verdict to reflect the cluster as it is at compare time.
Options, roughly in order of how much they change the contract:

1. Re-execute the target query on the non-PromQL branch, as the command's
   own help implies ("run the emitted ES|QL … on the target cluster").
2. Keep replaying, but say so — carry the migrate-time timestamp into the
   row and give the replayed verdicts a distinct mode (e.g. recorded, not
   live_source), so a stale ERROR is legible as stale.
3. Do not fail the exit code on replayed evidence — a recorded
   target_broken is not something compare observed.

(1) is what an operator expects; (2) is the minimum that stops the report
misrepresenting itself.

Notes

Not a duplicate of #246, which is about resolving specific compare/data-readiness
errors in a release batch; this is about compare reporting verdicts it did not
observe.

Found while testing the Datadog path for #454.