feat: export deviceToExecutionProviders and getSupportedDevices#1646
feat: export deviceToExecutionProviders and getSupportedDevices#1646TimPietrusky wants to merge 1 commit into
Conversation
Export two device utility functions from the public API: - `deviceToExecutionProviders(device)` — maps a device string to the ONNX execution providers list (already existed internally) - `getSupportedDevices()` — returns the platform-specific supported devices array in priority order (new convenience wrapper) This lets downstream libraries implement their own device fallback logic without hardcoding platform-specific device lists. Closes huggingface#1645
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
nico-martin
left a comment
There was a problem hiding this comment.
Thanks for looking itno this. It's agreat freature, but I have two small issues that should be adressed before merging.
| export { DynamicCache } from './cache_utils.js'; | ||
|
|
||
| // Device utilities | ||
| export { deviceToExecutionProviders, getSupportedDevices } from './backends/onnx.js'; |
There was a problem hiding this comment.
This adds runtime exports from the package entry point, but the checked-in generated declaration files are not updated. packages/transformers/types/transformers.d.ts still does not export deviceToExecutionProviders or getSupportedDevices, and packages/transformers/types/backends/onnx.d.ts still does not contain getSupportedDevices.
TypeScript consumers importing the new public API from @huggingface/transformers will therefore get a type error even though the runtime export exists. Please run the type generation step or add the generated declaration updates to this PR.
| * @returns {import("../utils/devices.js").DeviceType[]} The supported devices. | ||
| */ | ||
| export function getSupportedDevices() { | ||
| return /** @type {import("../utils/devices.js").DeviceType[]} */ ([...supportedDevices]); |
There was a problem hiding this comment.
Now that deviceToExecutionProviders is exported publicly, returning supportedDevices directly for device === "auto" exposes mutable internal state:
case 'auto':
return supportedDevices;A caller can accidentally or intentionally mutate the returned array, for example deviceToExecutionProviders('auto').length = 0, which changes the library's internal device order for later calls. getSupportedDevices() already avoids this by returning [...supportedDevices]; this path should do the same or otherwise return an immutable copy.
Summary
Export two device utility functions from the public API so downstream libraries can implement their own device fallback logic:
deviceToExecutionProviders(device)— maps a device string (e.g."auto","cuda","cpu") to the corresponding ONNX execution providers list. Already existed internally, just not re-exported.getSupportedDevices()— returns a copy of the platform-specificsupportedDevicesarray in priority order (e.g.["cuda", "webgpu", "cpu"]on Linux x64,["coreml", "webgpu", "cpu"]on macOS).Why
When
device="auto"and a provider crashes hard (e.g. CUDA whenlibcudnn.sois missing — see #1642), downstream code needs to retry providers individually. Without access to the device list, libraries have to hardcode their own copy of the platform detection logic, which drifts when transformers.js updates.Example usage
Changes
src/backends/onnx.js: addedgetSupportedDevices()functionsrc/transformers.js: re-exporteddeviceToExecutionProvidersandgetSupportedDevicesCloses #1645