Skip to content

feat: add Video Depth Anything prediction - #2576

Merged
inureyes merged 7 commits into
mainfrom
feature/issue-2352-video-depth-anything
Oct 23, 2026
Merged

inureyes merged 7 commits into
mainfrom
feature/issue-2352-video-depth-anything

Conversation

@inureyes

@inureyes inureyes commented Oct 23, 2026 •

Copy link
Copy Markdown
Member

Video Depth Anything checkpoints currently fail dense-model dispatch, so image and video depth inference cannot use the six published Small, Base, and Large relative/metric models. This change adds the shared-DINOv2 temporal DPT implementation, exact OpenCV-compatible preprocessing, bounded 32-frame sliding-window inference, consecutive video sampling, source-resolution depth outputs, registry metadata, documentation, and authentic opt-in parity/load tests.

Validation

  • cargo check -p mlxcel --profile test-fast --features rocm --lib --tests
  • focused VDA unit suite: 14 passed, 0 failed
  • all six pinned checkpoints load sequentially with exact layout and F32 dtype checks
  • ROCm full 40-frame Small relative parity: relative RMS 4.912878453e-6
  • ROCm full 40-frame Small metric parity: relative RMS 7.973837467e-7
  • real 64-frame ROCm CLI inference for both Small checkpoints: F32 [64, 96, 128] depth tensors and 64 valid 16-bit previews per model
  • strict CPU CLI 40-frame parity against authenticated CPU references: Small relative RMS 5.142513694620207e-6, Small metric RMS 7.445840126949833e-7 (threshold 1e-4)
  • CPU capture recovery independently reviewed: each underlying CLI exited 0, each outer wrapper exited 1 after inference on a display-case assertion, and the saved-output validator exited 0 without a model rerun; the exact pre-edit capture script bytes were not preserved, so the authenticated post-edit archive is explicitly excluded as execution proof
  • consecutive-prefix video sampling regression
  • independent implementation and security/performance reviews
  • environment-variable documentation and formatting checks
  • complete authentic CPU reference fixtures, provenance, archived run/recovery evidence, and exact binary-asset allowances
  • recovered relative CPU oracle independently audited: exact artifact/provenance identities and CPU-to-pinned-GPU relative RMS 3.4021821538643326e-7; wrapper exit 127, Python exit unknown, and lost completion log are preserved explicitly
  • clean metric CPU oracle: two complete temporal windows and terminal fixture PASS with zero status
  • rebased cleanly onto f69dd301; all six issue commits are patch-equivalent, and the eight lightweight repository contracts, format, diff, license-header, fixture-checksum, and binary-asset checks pass
  • cargo clippy -p mlxcel --profile test-fast --features rocm --lib --tests -- -D warnings
  • add English and Korean technical reports

Closes #2352

@inureyes inureyes added status:in-progress Currently being worked on type:enhancement New features, capabilities, or significant additions priority:low Low priority area:models Model architectures, weights, loading, metadata modelsize:small 4-bit checkpoint <= 10GB; fast iteration, safe for smoke tests modeltype:vlm Vision-language model status:review Under review and removed status:in-progress Currently being worked on labels Oct 23, 2026
@inureyes
inureyes force-pushed the feature/issue-2352-video-depth-anything branch from 8630c9a to 45e895a Compare October 23, 2026 11:09
@inureyes
inureyes marked this pull request as ready for review October 23, 2026 11:29
@inureyes inureyes added status:done Completed and removed status:review Under review labels Oct 23, 2026
@inureyes
inureyes merged commit edb9f03 into main Oct 23, 2026
29 checks passed
@inureyes
inureyes deleted the feature/issue-2352-video-depth-anything branch October 23, 2026 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:models Model architectures, weights, loading, metadata modelsize:small 4-bit checkpoint <= 10GB; fast iteration, safe for smoke tests modeltype:vlm Vision-language model priority:low Low priority status:done Completed type:enhancement New features, capabilities, or significant additions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Port Video Depth Anything (video_depth_anything) video depth

1 participant