Repository navigation
openvino: merge refactor PR - isolate op support, manage buffers, document API - #284
Draft
zhaixuejun1993 with Copilot wants to merge 135 commits into
Draft
zhaixuejun1993 with Copilot wants to merge 135 commits into
zhaixuejun1993 with Copilot wants to merge 135 commits into
Conversation
…g_src to recorde the src ggml tensor for OpenVINO dynamic shape infer
enable qwen35 Fix after rebase remove logging
…t reason: the backend test initializes unary op inputs over a wide range, [-150, 150]. For FP32, exp(x) overflows around x ~= 88.7, so this test can randomly generate values right in or beyond the overflow region
In stateful mode the NEOX RoPE branch fed rank-3 data ([S, n_heads, head_size]) into the Multiply against the rank-4 cos/sin tables ([1, S, 1, n_dims/2]). That mixed-rank broadcast is miscomputed by the OpenVINO GPU plugin, corrupting the rotated Q/K and producing garbage output (e.g. Phi-3-mini). Lift the data to rank-4 before the split/ Multiply so the operands are equal-rank, matching what the TYPE_NORMAL branch already does. CPU and stateless paths are unaffected. Phi-3-mini-Q4_K_M, wiki.test perplexity, GPU stateful: before: PPL = 27120.43 after: PPL = 6.2263 (CPU reference: 6.2251)
…ov name in ov bk; 3) fix issue in arch test & op test with latest code update
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Add runtime configuration entries for the newly recognized OpenVINO environment variables. Document GGML_OPENVINO_COMPILED_MODEL_CACHE_DIR as the frontend compiled-model cache used to export and import compiled blobs for matching single-graph models. Document GGML_OPENVINO_MEMORY_OPTIMIZE as the umbrella switch, including how GGML_OPENVINO_REDUCE_COMPILE_MEM and the GPU-only GGML_OPENVINO_RELEASE_WEIGHTS override or inherit from it.
1. GGML_OP_PAD was missing from compute_node_dynamic_dims(), causing a crash
on decode for models that pad the token embedding (n_embd -> n_embd_inp).
PAD never reorders/merges dims, so it keeps the same dynamic dim index as
its source.
2. process_view_input_new() chained VIEW inputs through src[0] (the
immediate op-graph parent) using offsets treated as relative to that
parent. But ggml_tensor::view_offs is always absolute from the true root
allocation (ggml collapses VIEW-of-VIEW chains internally). For the
per-layer deepstack view ("embd (view)", whose src[0] is "embd" - itself
an already-narrowed, zero-offset VIEW of the padded root, with the SAME
ggml shape as the deepstack view but a different absolute offset), this
caused an out-of-bounds re-slice that silently fell back to returning the
wrong (already-resolved sibling) tensor. In practice every deepstack ADD
ended up adding the real base token embedding into the residual stream
instead of zero, corrupting generation ("Hello my name is 1000000..."
instead of coherent text). Fixed by detecting this pattern (same shape as
the immediate src, different absolute offset) and re-slicing directly
from the untouched root tensor using the innermost view's absolute
offset.
Also adds a GGML_OPENVINO_DEBUG_NODE=<name1>,<name2>,... env var that attaches
extra debug Result nodes for arbitrary intermediate tensors, without binding
them to any ggml buffer (avoiding the risk of reading a ggml buffer that has
since been overwritten by a later in-place op). This was instrumental in
diagnosing bug #2 above and is left in as a general-purpose debugging aid.
IMROPE's inp_pos tensor packs 4 stacked t/h/w/e position planes into ne[0] = 4*n_tokens instead of one value per token. On NPU's static-shape path, inp_pos was padded/shaped as if it held a single plane, which interleaved padding across the 4 planes and desynced later reshapes from the rest of the (chunk_size-wide) graph. - add GgmlOvDecoder::get_inp_pos_n_planes() to detect IMROPE's 4-plane layout - get_graph_input_shape(): size inp_pos as n_planes * chunk_size (prefill) or n_planes (decode) instead of assuming 1 value per token - get_ov_input_tensor_static_prefill(): pad each plane to chunk_size independently instead of one flat block - get_ov_input_tensor_static_decode(): copy n_planes contiguous values instead of asserting/copying a single scalar
reject CPY with quantized destination as unsupported
…ent API Assisted-by: Claude Sonnet Co-authored-by: zhaixuejun1993 <52686861+zhaixuejun1993@users.noreply.github.com>
Copilot created this pull request from a session on behalf of
zhaixuejun1993
August 14, 2026 01:44
View session
wine99
force-pushed
the
dev_backend_openvino
branch
2 times, most recently
from
August 18, 2026 01:01
678ba21 to
16f69ac
Compare
ravi9
force-pushed
the
dev_backend_openvino
branch
from
August 18, 2026 14:43
eb54a50 to
04b5691
Compare
ravi9
force-pushed
the
dev_backend_openvino
branch
from
August 25, 2026 16:53
d691121 to
407e209
Compare
wine99
force-pushed
the
dev_backend_openvino
branch
from
August 28, 2026 01:36
01d5a57 to
40d3fdd
Compare
ravi9
force-pushed
the
dev_backend_openvino
branch
2 times, most recently
from
September 3, 2026 21:10
fd9bc04 to
33237ab
Compare
ravi9
force-pushed
the
dev_backend_openvino
branch
from
September 14, 2026 17:49
37b4e1d to
926ef68
Compare
wine99
force-pushed
the
dev_backend_openvino
branch
from
September 16, 2026 08:07
0365b00 to
37b53fd
Compare
ravi9
force-pushed
the
dev_backend_openvino
branch
from
September 17, 2026 17:37
40a2a96 to
972d231
Compare
ravi9
force-pushed
the
dev_backend_openvino
branch
6 times, most recently
from
October 3, 2026 19:48
308ccd8 to
836d571
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports and rebases PR #278 (
zhaixuejun1993/llama.cpp:xuejun/refactor-ggml-openvino) onto the latestdev_backend_openvino, resolving all merge conflicts.Overview
Structural refactor of the OpenVINO backend, splitting a monolithic
ggml-openvino.cppinto focused modules, plus conflict resolution to incorporate fixes that landed ondev_backend_openvinoafter the PR diverged.Changes
New modules extracted from
ggml-openvino.cpp:ggml-openvino-op-support.cpp/h- op support policy (ggml_openvino_device_supports_op_impl), previously inline; includes full per-op policy table in header commentsggml-openvino-buffer.cpp/h- buffer allocation and tensor extra managementggml-openvino-weight-buffer-release.cpp/h-GGML_OPENVINO_RELEASE_WEIGHTSRSS reclaim logicggml-openvino.cpp: now a thin orchestration layer;device_supports_opdelegates toggml_openvino_device_supports_op_impl;buffer_type_contextgainsbool is_hostfieldConflict resolutions:
ggml-openvino-op-support.cpp(GGML_OP_CPY): added quantized-destination guard fromae02f5eafter the existing BF16 src/dst check:utils.cpp(is_model_splitted): retainedGGML_OPENVINO_ENABLE_FALLBACKenv-var gate fromc66a9c9openvino/op/sqr.cpp: trailing newline preserved (matches base)tests/test-llama-archs.cpp: auto-merged correctly;has_openvinoskip removed, Meta tensor-split config always appended; new arch entries (QWEN3TTS,MUSE_GLIMMER,GRANITE_SWITCH) and updated WebGPU/HIP skip logic retained from baseAdditional information
Merge commit on branch
copilot/xuejunrefactor-ggml-openvino(rebased ontoravi9/llama.cpp:dev_backend_openvinoat37b164f).Requirements