Skip to content

Feature: frame generation for games with motion vectors (MTL4FXFrameInterpolator) #1509

Closed

Description

@dracinn

Request

An opt-in frame-generation pass in the present path, built on macOS 26 / iOS 26's MTL4FXFrameInterpolator, scoped to games that hand the host usable motion vectors — in practice, the NVN2 DLSS path.

Why this is feasible here

The host already encodes Metal 4 MetalFX work in the right places: MTL4FXTemporalScaler for NVN2's DLSS and MTL4FXSpatialScaler for DM_UPSCALER=metalfx / DM_PRESENT_UPSCALE. The interpolator's inputs line up with what the DLSS eval already collects:

  • colorTexture / prevColorTexture: the present thread sees both frames
  • depthTexture / motionTexture: the DLSS path already takes the game's depth and motion textures plus jitter (the NGX feature flags give the MV layout)
  • deltaTime: the present/vsync cadence
  • uiTexture: optional; games that composite UI into the presented frame can leave it out and accept HUD artifacts, or the UI composite pass could feed it where one is identifiable
  • shouldResetHistory: scene-cut / first-frame heuristics the present already tracks

For games without a motion source (NDSR-upscaler games like Bananza, Ocarina of Time), MV-less interpolation is the mushy case, so I'd scope this to interpolate only when the frame carried usable motion data — an env switch / launcher toggle like DM_PRESENT_UPSCALE, default off.

Use case

30 fps Switch 2 titles presenting at 60, or 60 → 120 on a 120 Hz screen. Caveats are the usual: one frame of added latency, interpolation artifacts, no benefit where MVs are absent.

Environment to test on

Apple silicon Mac, Xcode 27.2 / Metal 32023.922, building Switch 2 games from NUPs through the mac-app path.

Activity

  1. Repository owner locked and limited conversation to collaborators on Nov 10, 2026
  2. converted this issue into a discussion #2130 on Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions