Skip to content

feat: export deviceToExecutionProviders and getSupportedDevices#1646

Open
TimPietrusky wants to merge 1 commit into
huggingface:mainfrom
TimPietrusky:export-device-utils
Open

feat: export deviceToExecutionProviders and getSupportedDevices#1646
TimPietrusky wants to merge 1 commit into
huggingface:mainfrom
TimPietrusky:export-device-utils

Conversation

@TimPietrusky

Copy link
Copy Markdown

Summary

Export two device utility functions from the public API so downstream libraries can implement their own device fallback logic:

  • deviceToExecutionProviders(device) — maps a device string (e.g. "auto", "cuda", "cpu") to the corresponding ONNX execution providers list. Already existed internally, just not re-exported.
  • getSupportedDevices() — returns a copy of the platform-specific supportedDevices array in priority order (e.g. ["cuda", "webgpu", "cpu"] on Linux x64, ["coreml", "webgpu", "cpu"] on macOS).

Why

When device="auto" and a provider crashes hard (e.g. CUDA when libcudnn.so is missing — see #1642), downstream code needs to retry providers individually. Without access to the device list, libraries have to hardcode their own copy of the platform detection logic, which drifts when transformers.js updates.

Example usage

import { getSupportedDevices } from '@huggingface/transformers';

// Walk devices one at a time, catching provider crashes
for (const device of getSupportedDevices()) {
  try {
    model = await AutoModel.from_pretrained(modelId, { device });
    break;
  } catch (err) {
    console.warn(`device=${device} failed: ${err.message}`);
  }
}

Changes

  • src/backends/onnx.js: added getSupportedDevices() function
  • src/transformers.js: re-exported deviceToExecutionProviders and getSupportedDevices

Closes #1645

Export two device utility functions from the public API:

- `deviceToExecutionProviders(device)` — maps a device string to
  the ONNX execution providers list (already existed internally)
- `getSupportedDevices()` — returns the platform-specific supported
  devices array in priority order (new convenience wrapper)

This lets downstream libraries implement their own device fallback
logic without hardcoding platform-specific device lists.

Closes huggingface#1645
@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@nico-martin nico-martin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @TimPietrusky

Thanks for looking itno this. It's agreat freature, but I have two small issues that should be adressed before merging.

export { DynamicCache } from './cache_utils.js';

// Device utilities
export { deviceToExecutionProviders, getSupportedDevices } from './backends/onnx.js';

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds runtime exports from the package entry point, but the checked-in generated declaration files are not updated. packages/transformers/types/transformers.d.ts still does not export deviceToExecutionProviders or getSupportedDevices, and packages/transformers/types/backends/onnx.d.ts still does not contain getSupportedDevices.

TypeScript consumers importing the new public API from @huggingface/transformers will therefore get a type error even though the runtime export exists. Please run the type generation step or add the generated declaration updates to this PR.

* @returns {import("../utils/devices.js").DeviceType[]} The supported devices.
*/
export function getSupportedDevices() {
return /** @type {import("../utils/devices.js").DeviceType[]} */ ([...supportedDevices]);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that deviceToExecutionProviders is exported publicly, returning supportedDevices directly for device === "auto" exposes mutable internal state:

case 'auto':
    return supportedDevices;

A caller can accidentally or intentionally mutate the returned array, for example deviceToExecutionProviders('auto').length = 0, which changes the library's internal device order for later calls. getSupportedDevices() already avoids this by returning [...supportedDevices]; this path should do the same or otherwise return an immutable copy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Export supportedDevices / deviceToExecutionProviders from public API

4 participants