Skip to content

initial usbgpu support - #37906

Merged
Armandpl merged 111 commits into
masterfrom
usbgpu
May 20, 2026
Merged

Armandpl merged 111 commits into
masterfrom
usbgpu

Conversation

@Armandpl

@Armandpl Armandpl commented Apr 24, 2026 •

Copy link
Copy Markdown
Contributor

This PR:

  • compiles big model on boot if gpu present
  • chooses big model on onroad transition if gpu present + compiled, and displays state in ui

  • allow different devices for modeld/dmonitoringmodeld
    • compile path still need DEV set for OnnxRunner
    • runtime path doesn't and explicitely sets input devices
  • display/track state: compile state, in use
  • build both big and smol models
  • measure compile time again
    • can we build for target camera only?
  • add firmware to agnos
  • double check we need GMMU=0
  • make hotplug work
  • make_input_queues shouldn't rely on Device.default?
  • benchmark how long before we can detect gpu on boot? -> do we need sconscript to poll?
    -> instant don't need to poll
  • double check ldw or re-export model with ll/road edges

later:

  • add icon to big ui
  • add to model replay
  • re export big model in bf16/fp16

@github-actions

github-actions Bot commented Apr 24, 2026 •

Copy link
Copy Markdown
Contributor

@Armandpl
Armandpl marked this pull request as ready for review May 5, 2026 02:54
@Armandpl
Armandpl force-pushed the usbgpu branch 3 times, most recently from 37caaab to 16bb145 Compare May 10, 2026 22:46
@Armandpl Armandpl changed the title usbgpu usbgpu support May 10, 2026
Comment thread release/build_stripped.sh Outdated
@Armandpl
Armandpl merged commit 52e1826 into master May 20, 2026
13 checks passed
@Armandpl
Armandpl deleted the usbgpu branch May 20, 2026 05:42
erikfleischer pushed a commit to erikfleischer/openpilot that referenced this pull request May 22, 2026
* zero ll patched big model

* probe in a subprocess so usbgpu lock gets released

* compiles

* runs

* num_jobs gets overwritten, use side effect

* poll tg devices

* make sure build crashes on missing gpu

* fine not to rely on Device.default

* seperate tg env for each model runner

* comment

* Revert "seperate tg env for each model runner"

This reverts commit f6470cc.

* env is shared, gate on flag

* no fallback warp dev must be set

* build for current device only, unless pc/release

* comment

* list

* listen for plug in

* add icon to status bar, read params on every frame (?)

* log available devices

* try copy out when loading?

* Revert "log available devices"

This reverts commit e8c52a5.

* Revert "try copy out when loading?"

This reverts commit 518f403.

* don't trigger device probe/caching on modeld prepare

* re-export with ll and road edges

* dont cache devices in manager process

* get USBGPU from params

* no usbgpu env

* missed one

* sconscript don't poll

* unconditional env

* always explicitely set devices on input tensors

* set DEV so amd uses right compiler and iface??

* fix flag

* bump tg

* rm xdg_cache_home

* tg don't bump all the way

* missing gmmu=0 at compile time

* dm set dev

* tg backend

* update gitignore

* missing import

* unused imports

* rely on Device.DEFAULT at compile time (already the case bc onnxrunner)

* comments

* dm warp needs DEV set too

* build both smol and big

* misc typos

* set dev at compile time

* don't need

* DEV=CPU when getting metadata, ensure we don't grab gpu lock

* this would also grab lock

* put bool

* warp compile always prepare only

* missed one

* poll ui

* missing here

* don't force usbgpu at build time

* tmp patch fetch_fw

* catch all, follow hardwared patterns

* simpler

* compile make input queues

* revert this

* group this more readable

* rm empty line

* make dummy frame using numpy

* revert compile make input queues

* no compiler at runtime

* cleanup

* fine to rebuild all on change to device node for now

* fix usbgpu_present

* fix sconscript

* no size in header stream decompress

* DEBUG=2

* minimal viable feedback

* egpu gray

* oops

* gotta do this actually

* modeld build only depends on modeld devices

* don't ship onnx to release? or chunk

* don't need

* can only set compiler on dev=

* none device works, will use default

* make linter happy

* chunk agnostic onnx input to compile_modeld

* chunk big onnx

* +x chunker

* fix #!

* and don't ship chunked onnx to release

* firmware now in correct location

* better err on missing onnx/chunk

* SConscript also need to accept chunked onnx

* metadata also need to load maybe chunked

* dedupe cmd

* this needs to be on cpu

* devices are set in the tgflags, we already depend on them

* rebuilding on changed order is fine

* read file chunked can already load either chunked or not

* chunk all big onnx

* less confusing

* unused import

* python device to load onnx bytes

* default device for runners, python for metadata

* why not

* chunked to shm
cursor Bot pushed a commit to jmbrunick/openpilot that referenced this pull request Sep 16, 2026
* zero ll patched big model

* probe in a subprocess so usbgpu lock gets released

* compiles

* runs

* num_jobs gets overwritten, use side effect

* poll tg devices

* make sure build crashes on missing gpu

* fine not to rely on Device.default

* seperate tg env for each model runner

* comment

* Revert "seperate tg env for each model runner"

This reverts commit f6470cc.

* env is shared, gate on flag

* no fallback warp dev must be set

* build for current device only, unless pc/release

* comment

* list

* listen for plug in

* add icon to status bar, read params on every frame (?)

* log available devices

* try copy out when loading?

* Revert "log available devices"

This reverts commit e8c52a5.

* Revert "try copy out when loading?"

This reverts commit 518f403.

* don't trigger device probe/caching on modeld prepare

* re-export with ll and road edges

* dont cache devices in manager process

* get USBGPU from params

* no usbgpu env

* missed one

* sconscript don't poll

* unconditional env

* always explicitely set devices on input tensors

* set DEV so amd uses right compiler and iface??

* fix flag

* bump tg

* rm xdg_cache_home

* tg don't bump all the way

* missing gmmu=0 at compile time

* dm set dev

* tg backend

* update gitignore

* missing import

* unused imports

* rely on Device.DEFAULT at compile time (already the case bc onnxrunner)

* comments

* dm warp needs DEV set too

* build both smol and big

* misc typos

* set dev at compile time

* don't need

* DEV=CPU when getting metadata, ensure we don't grab gpu lock

* this would also grab lock

* put bool

* warp compile always prepare only

* missed one

* poll ui

* missing here

* don't force usbgpu at build time

* tmp patch fetch_fw

* catch all, follow hardwared patterns

* simpler

* compile make input queues

* revert this

* group this more readable

* rm empty line

* make dummy frame using numpy

* revert compile make input queues

* no compiler at runtime

* cleanup

* fine to rebuild all on change to device node for now

* fix usbgpu_present

* fix sconscript

* no size in header stream decompress

* DEBUG=2

* minimal viable feedback

* egpu gray

* oops

* gotta do this actually

* modeld build only depends on modeld devices

* don't ship onnx to release? or chunk

* don't need

* can only set compiler on dev=

* none device works, will use default

* make linter happy

* chunk agnostic onnx input to compile_modeld

* chunk big onnx

* +x chunker

* fix #!

* and don't ship chunked onnx to release

* firmware now in correct location

* better err on missing onnx/chunk

* SConscript also need to accept chunked onnx

* metadata also need to load maybe chunked

* dedupe cmd

* this needs to be on cpu

* devices are set in the tgflags, we already depend on them

* rebuilding on changed order is fine

* read file chunked can already load either chunked or not

* chunk all big onnx

* less confusing

* unused import

* python device to load onnx bytes

* default device for runners, python for metadata

* why not

* chunked to shm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants