Repository navigation

hexagon: support for multi-device model split (aka row-split) - #28589

Merged

max-krasnyansky merged 68 commits into

ggml-org:masterfrom

qualcomm:hexagon-mdev

Sep 12, 2026

Merged

hexagon: support for multi-device model split (aka row-split)#28589
max-krasnyansky merged 68 commits into
ggml-org:masterfrom
qualcomm:hexagon-mdev

Commits on Sep 11, 2026
- hex-row-split: add support for multi-device row spliting
  
  alexlu-qc
  and
  max-krasnyansky
  committed
  71fcfed
- hex-mdev: add work splitting to fused kernels
  max-krasnyansky
  committed
  0abdcee
- hex-mdev: use mdev_ prefix for all multi-device state
  max-krasnyansky
  committed
  e13454c
- hex-mdev: make device configuration more expressive to support device groups
  max-krasnyansky
  committed
  dd24175
- hex-mdev: fix mdev session init
  max-krasnyansky
  committed
  f7a6d7f
- hex-mdev: fused nx (2x,3x) matmuls must update row counts for each w/o
  max-krasnyansky
  committed
  89aa735
- hex-mdev: fix MUL_MAT work partitioning bugs introduced by mdev
  max-krasnyansky
  committed
  fbd8641
- hex-cont: fix crashes with new tests due to wrong striding
  max-krasnyansky
  committed
  cd3f00a
- hex-mdev: move fences after l2flushes
  max-krasnyansky
  committed
  20c1f98
- hex-cont: fix work splitting for mnpu -- align chunks to cachelines
  max-krasnyansky
  committed
  35c9f7b
- hex-mdev: fix CPY tests with multi-dev
  max-krasnyansky
  committed
  9bdfc49
- hex-mmid: fix work partitioning with mnpu
  max-krasnyansky
  committed
  60244bb
- hex-mm: fix test failures with mdev
  max-krasnyansky
  committed
  428e33f
- hex-binary: fix work partitioning for mdev
  max-krasnyansky
  committed
  165a86c
- hex-argsort: fix mdev partitioning
  max-krasnyansky
  committed
  5ddb1ad
- hex-mdev: fix work partitioning and general updates for all simple ops
  max-krasnyansky
  committed
  01acec1
- hex-fa: fix mdev work splitting issues
  max-krasnyansky
  committed
  1ad87b4
- hex-mdev: fixing more failing ops test
  max-krasnyansky
  committed
  d2995e3
- hex-mdev: update the rest of the ops
  max-krasnyansky
  committed
  a48a66f
- hex-mdev: refactor all mdev splitting logic to be contained within if (mdev_count > 1) {...}
  max-krasnyansky
  committed
  165fbc8
- hex-mdev: fix macros
  max-krasnyansky
  committed
  26b767a
- hex-mdev: simplify session flush logic
  max-krasnyansky
  committed
  446c7d9
- hex-sync: fix recursion in session flush
  max-krasnyansky
  committed
  ca23460
- hex-mdev: factor out fence buffer and allocator
  max-krasnyansky
  committed
  b0c236a
- hex-fence: make fence allocation more robust with reserved slots for mdev
  max-krasnyansky
  committed
  7c742bb
- hex-mdev: keep all mdev state in htp_mdev_group
  max-krasnyansky
  committed
  c5ea84e
- hex-mdev: further cleanup mdev group handling at the host
  max-krasnyansky
  committed
  78cd361
- hex-mdev: update group idx in the opbatch before serializing
  max-krasnyansky
  committed
  e4df2c6
- hex-batch: remove separate op_pending and use batch_req/rsp_seq
  max-krasnyansky
  committed
  007b1df
- hex-async: workaround another missing tensor_init in ggml-meta
  max-krasnyansky
  committed
  7c22ca9
- hex-fence: cleanup and robustify fences and error handling in multi-device scenarios
  max-krasnyansky
  committed
  207076e
- hex-ar: improve ALLREDUCE error handling
  max-krasnyansky
  committed
  3c10d6f
- hex-async: robust error handling for op_cpy_fence
  max-krasnyansky
  committed
  930917a
- hex-async: use seq0 from allreduce context to allocate fence_seq
  max-krasnyansky
  committed
  2dbb100
- hex-mdev: fix remaining issues with fence and barrier clearing in CPY_FENCE
  max-krasnyansky
  committed
  2dea460
- hex-misc: realign macros and fix misplaces trace events
  max-krasnyansky
  committed
  03a5f17
- hex-misc: align macros
  max-krasnyansky
  committed
  bd6cc5c
- hex-mdev: fix unclone buffer re-entrancy
  max-krasnyansky
  committed
  500641d
- hex-glu: fix mdev partitioning logic
  max-krasnyansky
  committed
  b08a823
- hex-mdev: make buffer uncloning/cleanup work with tensor-split scenarios
  max-krasnyansky
  committed
  295f70c
- hex-mdev: tighten up the can_split check in act-ops
  max-krasnyansky
  committed
  132aab6
- hex-mdev: factor out common bits of the partitioning logic
  max-krasnyansky
  committed
  647612f
- hex-mm: minor realignment of the macros
  max-krasnyansky
  committed
  94eea9f
- hex-bufs: fix incorrectly placed assert for MAX_BUFS
  max-krasnyansky
  committed
  ca94ef2
- hex-pad: tighten up gating checks for PAD
  max-krasnyansky
  committed
  fa1f7df
- hex-kparams: make sure all kernels properly use kparams->n_threads
  max-krasnyansky
  committed
  0e45847
- hex-docs: update user and developer docs with new features and detailed guide for ops development
  max-krasnyansky
  committed
  01ee391
- hex-scripts: update run script to properly parse dev groups
  max-krasnyansky
  committed
  8c983ac
- hex-misc: formatting
  max-krasnyansky
  committed
  08054ec
- hex-sess: minor cleanup for session init
  max-krasnyansky
  committed
  e40f24b
- hex-ar: fix vtcm size calc in allreduce kparams
  max-krasnyansky
  committed
  9005032
- hex-scripts: fix flake8 warnings
  max-krasnyansky
  committed
  d57cb39
- hex-rope: update ROPE to support mdev work split
  max-krasnyansky
  committed
  d896ff8
- hex-ops: remove redunant checks and minor reformat
  max-krasnyansky
  committed
  97d0ae1
- hex-dev-guide: update dev-guide to avoid redundant null checks
  max-krasnyansky
  committed
  ecc59fc
- hex-async: improve event_wait, event_sync and fence implementations
  max-krasnyansky
  committed
  1163872
- hex-async: remove synchronous flush from event_sync
  max-krasnyansky
  committed
  3c3ca80
- hex-async: symplify fence recovery protocol and make sync more robust
  max-krasnyansky
  committed
  eb2023c
- hex-async: futher simplify error recovery for fences
  max-krasnyansky
  committed
  b752c7c
- hex-err: return status instead of just -1
  max-krasnyansky
  committed
  b70b163
- hex-async: print all seq nums in hex
  max-krasnyansky
  committed
  1966ad1
- hex-async: make sure fences flush dirty ranges
  max-krasnyansky
  committed
  a76241b
- hex-async: add dirty ranges merging to reduce fence flushes
  max-krasnyansky
  committed
  d1194be
- hex-async: properly sync before freeing the event
  max-krasnyansky
  committed
  2833f16
- hex-async: make sure fence owner session is not overriden
  max-krasnyansky
  committed
  f733de4
- hex-async: more fence write order more robust
  max-krasnyansky
  committed
  250e3c9
- hex-async: make sure not to fuse ALLREDUCE+ADD if their dsts overlap
  max-krasnyansky
  committed
  2a8cb53
- hex-fusion: cleanup redundant checks
  max-krasnyansky
  committed
  9b9bdf1

Commits

Commits on Sep 11, 2026