[Adityakk9031]

Summary

The distance scorers (min_distance_to_goal and reached_goal_state) consume distance signals recorded in step.result.info.

1. Non-finite distance signals (NaN / inf): When sensors warm up, track is temporarily lost, or cameras experience occlusion during initial rollout steps, embodiments may record NaN in info["distance"]. Because Python's min([nan, 0.02]) returns nan, reached_goal_state evaluates nan <= threshold as False, causing runs that reached the goal to be marked as failed. In min_distance_to_goal, Score(value=nan) propagates into downstream evaluation aggregations (such as mean across scenes), turning aggregate benchmark scores into NaN. Furthermore, if info["distance"] is unconvertible or None, it raised unhandled TypeError or ValueError.
2. Missing signal fallback: When all recorded distance signals in an episode are NaN, min_distance_to_goal previously returned nan instead of inf ("no distance signal recorded").
3. Threshold validation: reached_goal_state(threshold) silently accepted invalid thresholds, including negative values (which can never be achieved), NaN, and booleans.

This change filters _distances to finite non-negative values, ignoring missing, non-numeric, non-finite, and negative entries. It also validates that reached_goal_state threshold is a finite, non-negative real number.

Changes

• src/inspect_robots/scorer.py:
  • Update _distances to filter out non-finite (NaN, ±inf), negative, boolean, and non-numeric values.
  • Validate threshold in _ReachedGoalState at initialization to ensure it is a finite non-negative real number.
• tests/test_scorers.py: Add unit tests for NaN handling alongside valid distances, filtering of invalid entries, fallback when no distance signals exist, and threshold validation.
• changelog.d/+distance-scorers-nan-and-threshold-validation.fixed.md: Add changelog fragment.

[robocurve-pr-reviewer]

REQUIRE_REVIEWER. No verdict issued. This contributor has used their $20 monthly review allowance across their PRs.

Next step: @jeqcho, the service did not start a sandbox or make a model call. Review manually; the allowance resets at the start of the next UTC month.

This is a limitation or failure of the reviewer, not a finding against the contribution. No approval, merge or closure recommendation was issued.

Commit: 0be5b0243332ada61793970784b332da62a7ca83
Run reference: b43b9b7f8a849c0b05ddcafa74e2c279780851377a5b2568. Reason: contributor_budget_exhausted.

Budget: this run booked $0.000 for 0 model calls, plus $0.000 sandbox allowance. This revision has used $0.000 of $5.000 across all runs; $0.000 remains under all spending caps. These are conservative ledger amounts, not an invoice.

[SIDDARTHAREDDY8]

Two edge cases in the new validation in src/inspect_robots/scorer.py that the added tests do not cover:

1. Overflow escapes both new guards. In _distances, the float(raw) conversion is wrapped in except (TypeError, ValueError), but float() on a large int (for example a distance recorded as 10**400) raises OverflowError, so one bad signal would still crash the scorer instead of being treated as unobserved. The same applies in _ReachedGoalState.__post_init__: float(self.threshold) on a large int raises OverflowError rather than the TypeError/ValueError the validation otherwise raises. Adding OverflowError to the catch in _distances (and converting it in __post_init__) would close that hole.

2. The threshold type check is stricter than the distance path, and inconsistent for NumPy scalars. _distances accepts anything float() can coerce, but __post_init__ requires isinstance(threshold, int | float). np.float64 passes because it subclasses float, while np.float32(0.05) (and Decimal) raises TypeError, even though it would have worked before this change via the min(dists) <= self.threshold comparison. Consider a numbers.Real check or the same try-float() coercion used for distances, plus a test with a np.float32 threshold.

The core NaN/inf filtering in _distances itself looks right, including the explicit bool rejection.

[Adityakk9031]

@jeqcho please have a look