Repository navigation

fallback ov to vulkan - #47

Open

zhaixuejun1993 wants to merge 253 commits into

ravi9:dev_backend_openvinofrom

zhaixuejun1993:xuejun/fallback-ov-to-gpu-v1

Open

fallback ov to vulkan#47
zhaixuejun1993 wants to merge 253 commits into
ravi9:dev_backend_openvinofrom
zhaixuejun1993:xuejun/fallback-ov-to-gpu-v1

This pull request is big! We're only showing the most recent 250 commits

Commits on Jan 15, 2026
- Solve the issue of abnormal model output caused by using OpenVINO ADD operator
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  faa4a7d
- Add OpenVINO MUL operator to GGML of Llama.cpp.
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  adc2c70
- Add compile options
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  0a81aa1
- add OpenVINO frontend convert process steps
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  77d6814
- add get openvino available ops function
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  ee31dc1
- Add PoC of integration of openvino frontend. Main changes: ggml-ov-frontend-utils, GraphIterator, Decoder
  
  yumengbo
  authored and
  cavusmustafa
  committed
  171c468
- Implement GgmlOvDecoder. Add dump functions.
  
  yumengbo
  authored and
  cavusmustafa
  committed
  34e826a
- Convert subgraph with add, sub, mul, div op to ov model and do infer on openvino device
  
  yumengbo
  authored and
  cavusmustafa
  committed
  9b7b63d
- Add GGML_OV_FRONTEND option. Add readme.
  
  yumengbo
  authored and
  cavusmustafa
  committed
  31bd816
- Change output for infer request to set output tensor. Support scale, view op.
  
  yumengbo
  authored and
  cavusmustafa
  committed
  5b46dc2
- add GET_ROWS operator of OpenVINO to GGML of llama.cpp
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  49804f4
- Update build.md and add operation mapping(GGML to OpenVINO)
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  80c330a
- add the rms_norm operator implemented using OpenVINO to the GGML backend of llama.cpp
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  8c5a609
- Fix issue for output memory copy of infer request
  
  yumengbo
  authored and
  cavusmustafa
  committed
  e95f29c
- Change to implementation following pytorch frontend
  
  yumengbo
  authored and
  cavusmustafa
  committed
  b100f89
- Add support for UNARY SILU op . Fix pytorch impl bugs.
  
  yumengbo
  authored and
  cavusmustafa
  committed
  590f587
- Support Softmax op
  
  yumengbo
  authored and
  cavusmustafa
  committed
  d218c61
- Support Softmax op
  
  yumengbo
  authored and
  cavusmustafa
  committed
  8aba03b
- Support ROPE op.
  
  yumengbo
  authored and
  cavusmustafa
  committed
  2353c73
- Add support for RMS_NORM OP
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  0f7d07d
- Add MUL_MAT,CPY,CONT as operators implemented in OpenVINO for GGML backend
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  2b04bd4
- Move CPY from GGML OV Backend to OV Frontend
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  cb2729b
- add implementation of MUL_MAT, CPY, CONT of GGML ops using OV ops
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  8484769
- add implementation of CPY when the output tensor is non-contiguous
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  57582fd
- add tmp source code files
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  afb8594
- Execute singel CONT operator is OK
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  081b526
- Execute CONT & VIEW operators in OV Frontend is OK
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  901f734
- OV Frontend supports GET_ROWS/RMS_NORM/MUL/MUL_MAT graph conversion of consecutive OPs
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  95ae982
- OV Frontend supports GET_ROWS/RMS_NORM/MUL/MUL_MAT/ROPE/SCALE/SOFTMAX/ADD adjacent op graph conversion
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  9a7b7d8
- Change the input parameter shape of CONT operator
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  f98d215
- Change the input and ouput node shape of MUL_MAT operator
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  f37fa21
- Change the input and ouput node shape of MUL_MAT operator
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  246a2d1
- change CONT and MULMAT input node shape
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  d05c458
- All adjacent ops can conversion but calculation result is wrong and need debugging
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  e08a7fd
- 1. All operators implemented using OpenVINO can be successfully executed individually.
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  cff473a
- 1. Update the implementation of CPY node when it's non-contiguous
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  467a5dd
- Minor Update
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  b14b49d
- Try to add VIEW node to OV Frontend and have some issues that need to be dealt with
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  19ec9b6
- 1. In the Prompt process and predict first token stage, the PERMUTE node needs to be integrated into the OV Frontend
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  b02265a
- add debug info
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  8020138
- Process Prompt and predict first token is OK
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  8ae700a
- 1. Solve the AC issue of Permute+VIEW and MULMAL issue in the phase of “1. Process Prompt and predict the first token”.
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  eac9a99
- 1. Delete some comments
  
  zhanmyz
  authored and
  cavusmustafa
  committed
  84be5c6
- * Use find_package in CMake to configure OpenVINO
  
  wine99
  authored and
  cavusmustafa
  committed
  651b2c0
- change op mappings to list in openvino_supports_op
  
  wine99
  authored and
  cavusmustafa
  committed
  91d2a19
- 2nd+ token correct by fix CPY in OV, remove single op backend compute code
  
  wine99
  authored and
  cavusmustafa
  committed
  8d263bd
- Arbitrary token len (>32) work; Fix bug in mulmat
  
  wine99
  authored and
  cavusmustafa
  committed
  8b40886
- FEAT: do PERMUTE eagerly
  
  wine99
  authored and
  cavusmustafa
  committed
  6ed44a3
- FEAT: Add interleaved mode for ROPE
  
  wine99
  authored and
  cavusmustafa
  committed
  0c7b026
- REFACTOR: support weigts as constant
  
  wine99
  authored and
  cavusmustafa
  committed
  c04966c
- STYLE: minor refactor
  
  wine99
  authored and
  cavusmustafa
  committed
  96ba47d
- PERF: share const nodes for weights for diff infer
  
  wine99
  authored and
  cavusmustafa
  committed
  d3bdca2
- BUILD: update build doc, add cmake preset, add CACHE_DIR env var
  
  wine99
  authored and
  cavusmustafa
  committed
  0a8cc9a
- FEAT: improve debug capability
  
  wine99
  authored and
  cavusmustafa
  committed
  7d5e234
- PERF: compile once (dynamic graph + cache)
  
  wine99
  authored and
  cavusmustafa
  committed
  a8e5efa
- Rebase - Bring up to date and fix build process
  
  virajwad
  authored and
  cavusmustafa
  committed
  ffabe95
- fix build error
  
  wine99
  authored and
  cavusmustafa
  committed
  4c905b2
- FIX: backend buffer type issue
  
  wine99
  authored and
  cavusmustafa
  committed
  a0b3052
- STYLE: clang-format
  
  wine99
  authored and
  cavusmustafa
  committed
  f15a2cc
- FEAT: Add all conversion code from ov side
  
  wine99
  authored and
  cavusmustafa
  committed
  0d009fe
- PERF: favor low precision matmul
  
  wine99
  authored and
  cavusmustafa
  committed
  cdf5370
- STYLE and minor REFACTOR
  
  wine99
  authored and
  cavusmustafa
  committed
  0d505b4
- FIX: Re-add tensor names in cgraph, Add another case for RESHAPE
  
  wine99
  authored and
  cavusmustafa
  committed
  041d220
- FIX: input shape of KQ_mask
  
  wine99
  authored and
  cavusmustafa
  committed
  c57f614
- PERF: add weight constant in parallel
  
  wine99
  authored and
  cavusmustafa
  committed
  a30dc6e
- FIX: set_max_token_len
  
  wine99
  authored and
  cavusmustafa
  committed
  8ac5c22
- PERF: use Slice+Concat in writing cache_v
  
  wine99
  authored and
  cavusmustafa
  committed
  d7cc802
- Update build doc
  
  wine99
  authored and
  cavusmustafa
  committed
  fd32436
- Add cgraph tensor output name to OV op name
  
  wine99
  authored and
  cavusmustafa
  committed
  8ce5cc5
- Update openvino build instructions
  
  ravi9
  authored and
  cavusmustafa
  committed
  3051d5a
- Add initial NPU support
  
  wine99
  authored and
  cavusmustafa
  committed
  7fec223
- draft NPU support version 2: prefill + kvcache
  
  wine99
  authored and
  cavusmustafa
  committed
  34531ab
- NPU support version 2: prefill + kvcache
  
  wine99
  authored and
  cavusmustafa
  committed
  d9ca8f5
- Change due to ggml cgraph changes, not correct yet
  
  wine99
  authored and
  cavusmustafa
  committed
  f7ad779
- Change due to ggml cgraph changes, llama-3.2 CPU work
  
  wine99
  authored and
  cavusmustafa
  committed
  592d7f8
- Add AMD64 to CMakeLists
  
  wine99
  authored and
  cavusmustafa
  committed
  e27738a
- Change due to ggml cgraph changes, all device work
  
  wine99
  authored and
  cavusmustafa
  committed
  42d4240
- Refactor: clean, fix warning
  
  wine99
  authored and
  cavusmustafa
  committed
  593484c
- Update clang-format
  
  wine99
  authored and
  cavusmustafa
  committed
  8afee79
- Statful transformation for CPU GPU
  
  wine99
  authored and
  cavusmustafa
  committed
  4c582ac
- Add SwiGLU
  
  wine99
  authored and
  cavusmustafa
  committed
  73ee84f
- Fuse to SDPA
  
  wine99
  authored and
  cavusmustafa
  committed
  ebc4fc9
- Replace Concat with Broadcast in MulMat for GQA
  
  wine99
  authored and
  cavusmustafa
  committed
  bf5414c
- Pull out indices creation for kv cache update
  
  wine99
  authored and
  cavusmustafa
  committed
  acf358d
- Refactor: remove past_token_len from extra_inputs
  
  wine99
  authored and
  cavusmustafa
  committed
  0fa7a5e
- Fix Phi3 SwiGLU and SoftMax
  
  wine99
  authored and
  cavusmustafa
  committed
  3533c14
- Pull out sin cos from rope
  
  wine99
  authored and
  cavusmustafa
  committed
  a80da69
- Reduce memory: free ov weights node after graph conversion
  
  wine99
  authored and
  cavusmustafa
  committed
  f3c0519
- Fix CPY due to cgraph change
  
  wine99
  authored and
  cavusmustafa
  committed
  d61f83c
- Added OpenVINO CI/CD. Updated docs
  
  ravi9
  authored and
  cavusmustafa
  committed
  ea75772
- Fix llama-cli
  
  wine99
  authored and
  cavusmustafa
  committed
  1ed49bb
- Fix Phi3 ROPE; Add test-backend-ops
  
  wine99
  authored and
  cavusmustafa
  committed
  44f4cf3
- Fix NPU
  
  wine99
  authored and
  cavusmustafa
  committed
  6dc4b90
- Fix llama-bench; Clang-format
  
  wine99
  authored and
  cavusmustafa
  committed
  75eec62
- Fix llama-perplexity
  
  wine99
  authored and
  cavusmustafa
  committed
  4e7f04a
- temp. changes for mark decomp
  cavusmustafa
  committed
  9cf56d6
- matmul in fp32
  
  wine99
  authored and
  cavusmustafa
  committed
  01cdf4a
- mulmat input conversion fix
  cavusmustafa
  committed
  e2fdc1b
- mulmat type conversion update
  cavusmustafa
  committed
  93b2d09
- add mark decomp pass
  cavusmustafa
  committed
  1a19566
- Revert changes in fuse_to_sdpa
  
  wine99
  authored and
  cavusmustafa
  committed
  43489bb
- Update build.md
  
  ravi9
  authored and
  cavusmustafa
  committed
  2f99135
- Fix test-backend-ops
  
  wine99
  authored and
  cavusmustafa
  committed
  fc86534
- Skip test-thread-safety; Run ctest only in ci/run.sh
  
  wine99
  authored and
  cavusmustafa
  committed
  1141350
- Use CiD for NPU
  
  wine99
  authored and
  cavusmustafa
  committed
  37ff226
- Optimize tensor conversion, improve TTFT
  
  wine99
  authored and
  cavusmustafa
  committed
  9a91ca6
- Support op SET_ROWS
  
  wine99
  authored and
  cavusmustafa
  committed
  63d000b
- Fix NPU
  
  wine99
  authored and
  cavusmustafa
  committed
  7bda502
- Remove CPY
  
  wine99
  authored and
  cavusmustafa
  committed
  839f8c6
- Fix test-backend-ops
  
  wine99
  authored and
  cavusmustafa
  committed
  f4123be
- Minor updates for raising PR
  
  wine99
  authored and
  cavusmustafa
  committed
  a7b611b
- Perf: RMS fused to OV internal RMS op
  
  wine99
  authored and
  cavusmustafa
  committed
  14c8a85
- Fix after rebasing
  
  wine99
  authored and
  cavusmustafa
  committed
  65e1b1a
- Change openvino device_type to GPU; Enable flash_attn
  
  wine99
  authored and
  cavusmustafa
  committed
  56d5967
- Update supports_buft and supports_op for quantized models
  
  wine99
  authored and
  cavusmustafa
  committed
  3e897df
- Add quant weight conversion functions from genai gguf reader
  
  wine99
  authored and
  cavusmustafa
  committed
  d4ca760
- Quant models run with accuracy issue
  
  wine99
  authored and
  cavusmustafa
  committed
  663a0b8
- Fix accuracy: disable cpu_repack
  
  wine99
  authored and
  cavusmustafa
  committed
  6ab76ed
- Fix CI; Disable test-backend-ops
  
  wine99
  authored and
  cavusmustafa
  committed
  dd80b04
- Fix Q4_1
  
  wine99
  authored and
  cavusmustafa
  committed
  a1ce428
- Fix test-backend-ops: Treat quantized tensors as weights
  
  wine99
  authored and
  cavusmustafa
  committed
  9900245
- Add NPU Q4_0 support
  
  wine99
  authored and
  cavusmustafa
  committed
  9ca53c7
- NPU perf: eliminate zp
  
  wine99
  authored and
  cavusmustafa
  committed
  82c9833
- Dequantize q4_1 q4_k q6_k for NPU
  
  wine99
  authored and
  cavusmustafa
  committed
  b593428
- Add custom quant type: q8_1_c, q4_0_128
  
  wine99
  authored and
  cavusmustafa
  committed
  6926655
- Set m_is_static=false as default in decoder
  
  wine99
  authored and
  cavusmustafa
  committed
  c5231a2
- Simpilfy translation of get_rows
  
  wine99
  authored and
  cavusmustafa
  committed
  810eb48
- Fix after rebasing
  
  wine99
  authored and
  cavusmustafa
  committed
  0f7b253
- Improve debug util; Eliminate nop ReshapeReshape
  
  wine99
  authored and
  cavusmustafa
  committed
  2ad1147
- STYLE: make get_types_to_requant a function
  
  wine99
  authored and
  cavusmustafa
  committed
  dc77cbb
- Support BF16 model
  
  wine99
  authored and
  cavusmustafa
  committed
  bcc343a
- Fix NPU compile
  
  wine99
  authored and
  cavusmustafa
  committed
  434059a
- WA for npu 1st token acc issue
  
  wine99
  authored and
  cavusmustafa
  committed
  da2cc99
- Apply EliminateZP only for npu
  
  wine99
  authored and
  cavusmustafa
  committed
  be07073
- Add GeGLU
  
  wine99
  authored and
  cavusmustafa
  committed
  5975612
- Fix Hunyuan
  
  wine99
  authored and
  cavusmustafa
  committed
  7d81861
- Support iSWA
  
  wine99
  authored and
  cavusmustafa
  committed
  9de874c
- Fix NPU accuracy
  
  wine99
  authored and
  cavusmustafa
  committed
  602f9ca
- Fix ROPE accuracy when freq_scale != 1
  
  wine99
  authored and
  cavusmustafa
  committed
  1a38339
- Minor: not add attention_size_swa for non-swa model
  
  wine99
  authored and
  cavusmustafa
  committed
  67e178a
- Minor refactor
  
  wine99
  authored and
  cavusmustafa
  committed
  2f1d50f
- Add Q5_K to support phi-3-q4_k_m
  
  wine99
  authored and
  cavusmustafa
  committed
  e4bfe5a
- Requantize Q6_K (gs16) to gs32 on GPU
  
  wine99
  authored and
  cavusmustafa
  committed
  f3afa7b
- Fix after rebasing
  
  wine99
  authored and
  cavusmustafa
  committed
  fdadca1
- Always apply Eliminate_ZP to fix GPU compile issue on some platforms
  
  wine99
  authored and
  cavusmustafa
  committed
  973a80f
- kvcachefusion support
  cavusmustafa
  committed
  c112bc4
- env variable GGML_OPENVINO_DISABLE_SDPA_OPTIMIZATION added
  cavusmustafa
  committed
  e725292
- Fix for Phi3
  cavusmustafa
  committed
  05d7aba
- Fix llama-cli (need to run with --no-warmup)
  
  wine99
  authored and
  cavusmustafa
  committed
  a9371ea
- Fix add_sliced_mask; Revert mulmat, softmax; Remove input attention_size, iSWA model not working
  
  wine99
  authored and
  cavusmustafa
  committed
  8b82d11
- fix after rebasing
  
  wine99
  authored and
  cavusmustafa
  committed
  299f492
- Fix llama-3-8b and phi3-mini q4_0 NPU
  
  wine99
  authored and
  cavusmustafa
  committed
  2d2f00a
- Update to OV-2025.3 and CMakeLists.txt
  
  ravi9
  authored and
  cavusmustafa
  committed
  841d673
- Add OV CI cache
  
  wine99
  authored and
  cavusmustafa
  committed
  4c8406e
- Apply CISC review and update CI to OV2025.3
  
  ravi9
  authored and
  cavusmustafa
  committed
  38e8a19
- Update CI to run OV dep install before build
  
  ravi9
  authored and
  cavusmustafa
  committed
  45af912
- Update OV dockerfile to use OV2025.3 and update build docs
  
  ravi9
  authored and
  cavusmustafa
  committed
  3a1129e
- Style: use switch in supports_ops
  
  wine99
  authored and
  cavusmustafa
  committed
  bd3093f
- Style: middle ptr and ref align, omit optional struct keyword
  
  wine99
  authored and
  cavusmustafa
  committed
  eba8113
- NPU Unify PD (#14)
  
  wine99
  authored and
  cavusmustafa
  committed
  b8690bc
- Clean placeholders in ggml-openvino.cpp
  
  wine99
  authored and
  cavusmustafa
  committed
  303923a
- NPU unify PD (handled internally)
  
  wine99
  authored and
  cavusmustafa
  committed
  ea2c99b
- change graph to 4d, support multi sequences
  
  wine99
  authored and
  cavusmustafa
  committed
  072dde0
- Fix llama-bench
  
  wine99
  authored and
  cavusmustafa
  committed
  ae404f7
- Fix NPU
  
  wine99
  authored and
  cavusmustafa
  committed
  531941b
- Update ggml-decoder.cpp
  
  I-N-T-E-L
  authored and
  cavusmustafa
  committed
  047bfb5
- Update ggml-decoder.cpp
  
  I-N-T-E-L
  authored and
  cavusmustafa
  committed
  11b4cc5
- Update ggml-decoder.cpp
  
  I-N-T-E-L
  authored and
  cavusmustafa
  committed
  bed4952
- Update ggml-decoder.cpp
  
  I-N-T-E-L
  authored and
  cavusmustafa
  committed
  4a57b37
- Update ggml-decoder.cpp
  
  I-N-T-E-L
  authored and
  cavusmustafa
  committed
  98396b2
- Update ggml-decoder.cpp
  
  I-N-T-E-L
  authored and
  cavusmustafa
  committed
  4400b5c
- Remove the second decoder for node. Moving the function into the model decoder
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  ae93651
- Fix error for naive
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  992dea7
- NPU prefill chunking
  
  wine99
  authored and
  cavusmustafa
  committed
  38254cf
- NPU fix llama-bench
  
  wine99
  authored and
  cavusmustafa
  committed
  59e7e7c
- fallback naive run with accuracy issue
  
  wine99
  authored and
  cavusmustafa
  committed
  65348b5
- NPU support llma-perplexity -b 512 --no-warmup
  
  wine99
  authored and
  cavusmustafa
  committed
  808619e
- Refactor: split ov_graph_compute for dynamic and static
  
  wine99
  authored and
  cavusmustafa
  committed
  2a9d4ca
- remove unused API GgmlOvDecoder::get_output_stride(const std::string & name)
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  0ea8238
- minor update due to ov 2025.4
  
  wine99
  authored and
  cavusmustafa
  committed
  8f4ee4e
- remove unused API GgmlOvDecoder::get_output_names()
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  497964a
- remove unused API get_output_shape(const std::string & name)
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  f516db1
- Modified API GgmlOvDecoder::get_output_type(const std::string & name)
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  6d7a0d6
- Removed API GgmlOvDecoder::get_output_op_params(const std::string & name)
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  ba852f2
- Removed API get_output_ggml_tensor(const std::string & name)
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  111c96c
- Removed API m_outputs
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  8ff73e5
- Removed m_output_names
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  197ed99
- Removed API GgmlOvDecoder::get_input_names()
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  95c3071
- Removed API GgmlOvDecoder::get_input_stride(const std::string& name)
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  cd61178
- Removed API get_input_type
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  891a3be
- Removed API get_input_type
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  42ca27f
- Removed API GgmlOvDecoder::get_input_shape(const std::string & name)
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  acb8a01
- Removed API GgmlOvDecoder::get_input_op_params(const std::string & name)
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  47c91db
- Fix error for decoder cache
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  91a1b20
- Reuse cached decoder
  
  wine99
  authored and
  cavusmustafa
  committed
  28da9a9
- GPU remove Q6_K requantization
  
  wine99
  authored and
  cavusmustafa
  committed
  469325c
- NPU fix wrong model output shape
  
  wine99
  authored and
  cavusmustafa
  committed
  ae01322
- NPU fix q4 perf regression
  
  wine99
  authored and
  cavusmustafa
  committed
  c9234b4
- Remove unused variable nodes
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  9e3163e
- Fix decoder can_reuse for llama-bench
  
  wine99
  authored and
  cavusmustafa
  committed
  0ef2e5e
- Update build.md for Windows
  
  I-N-T-E-L
  authored and
  cavusmustafa
  committed
  ae53363
- backend buffer: allocate on host
  
  wine99
  authored and
  cavusmustafa
  committed
  22d9c17
- Use shared_buffer for GPU NPU; Refactor
  
  wine99
  authored and
  cavusmustafa
  committed
  72bba82
- Add ov_backend_host_buffer; Use cached remote context
  
  wine99
  authored and
  cavusmustafa
  committed
  3fdcb6a
- Put kvcache on GPU
  
  wine99
  authored and
  cavusmustafa
  committed
  d757849
- Use ggml_aligned_malloc
  
  wine99
  authored and
  cavusmustafa
  committed
  8273a7c
- only use remote tensor for kvcache
  
  wine99
  authored and
  cavusmustafa
  committed
  88d1d17
- only use remote tensor for kvcache for GPU
  
  wine99
  authored and
  cavusmustafa
  committed
  a356b44
- FIX: use remote tensor from singleton
  
  wine99
  authored and
  cavusmustafa
  committed
  cfc4713
- Update build.md to include OpenCL
  
  wine99
  authored and
  cavusmustafa
  committed
  52a4401
- NPU always requant to q4_0_128
  
  wine99
  authored and
  cavusmustafa
  committed
  c1142dd
- Optimize symmetric quant weight extraction: use single zp
  
  wine99
  authored and
  cavusmustafa
  committed
  67c9720
- Use Q8_0_C in token embd, lm_head, and for 5 and 6 bits quant
  
  wine99
  authored and
  cavusmustafa
  committed
  4e45177
- Update build.md
  
  wine99
  authored and
  cavusmustafa
  committed
  f5c71e3
- Support -ctk f32
  
  wine99
  authored and
  cavusmustafa
  committed
  0d6f253
- Initial stateful graph support
  cavusmustafa
  committed
  5f30eac
- Update ggml/src/ggml-openvino/ggml-decoder.cpp
  
  cavusmustafa
  and
  ynimmaga
  committed
  d2fc152
- code cleanup
  cavusmustafa
  committed
  981ec65
- npu perf fix
  cavusmustafa
  committed
  a40a5df
- requant to f16 for Q6 embed on NPU
  cavusmustafa
  committed
  a81b202
- Update ggml/src/ggml-openvino/ggml-decoder.cpp
  cavusmustafa
  committed
  a92ecee
- Update ggml/src/ggml-openvino/ggml-openvino-extra.cpp
  cavusmustafa
  committed
  599335c
- Create OPENVINO.md in llama.cpp backend docs
  
  ynimmaga
  authored and
  cavusmustafa
  committed
  416556a
- Update OPENVINO.md
  
  ynimmaga
  authored and
  cavusmustafa
  committed
  25e6525
- Update OPENVINO.md
  
  ynimmaga
  authored and
  cavusmustafa
  committed
  9ba3247
- Update OPENVINO.md
  
  ynimmaga
  authored and
  cavusmustafa
  committed
  61552e4
- Update build.md
  
  ynimmaga
  authored and
  cavusmustafa
  committed
  63eed0d
- Update OPENVINO.md
  
  ynimmaga
  authored and
  cavusmustafa
  committed
  f44c60e
- Update OPENVINO.md
  
  ynimmaga
  authored and
  cavusmustafa
  committed
  e9ed5c4
- Update OPENVINO.md
  
  ynimmaga
  authored and
  cavusmustafa
  committed
  d3649c1
- kq_mask naming fix
  cavusmustafa
  committed
  d7dccf8
Commits on Jan 16, 2026
- Syntax correction for workflows build file
  cavusmustafa
  committed
  aa4bc90
Commits on Jan 21, 2026
- Change ov backend buffer is_host to false
  wine99
  committed
  9a15c8b
Commits on Feb 11, 2026
- Fix llama-bench -p -n where p<=256
  wine99
  committed
  8fb20b2
- Fix --direct-io 0
  wine99
  committed
  1c0a47a
- Don't put kvcache on GPU in stateful mode
  wine99
  committed
  c840210
- Remove hardcode names
  wine99
  committed
  d398214
- Fix stateful shapes
  wine99
  committed
  26328fe
- Simplification for stateful and update output shape processing
  
  cavusmustafa
  authored and
  wine99
  committed
  3259921
- Remove hardcode names
  wine99
  committed
  18ab0f5
- Avoid re-compilation in llama-bench
  wine99
  committed
  b6c0697
- Extract zp directly instead of bias
  wine99
  committed
  0ee7e05
- Refactor weight tensor processing
  wine99
  committed
  900dd76
- Merge branch 'master' into dev_backend_openvino
  wine99
  committed
  7b3b65b
- create_weight_node accept non-ov backend buffer
  wine99
  committed
  1d4ec1b
- remove changes in llama-graph.cpp
  wine99
  committed
  e059015
Commits on Feb 12, 2026
- stateful masking fix (#38)
  cavusmustafa
  authored
  0d74aba
- Fix test-backend-ops crash glu, get_rows, scale, rms_norm, add
  wine99
  committed
  d5d673c
- Add ov falback to CPU machinisim & verified with OV without ROPE support
  
  zhaixuejun1993
  authored and
  cavusmustafa
  committed
  046ba55
Commits on Feb 15, 2026
- fallback to vulkan, verified with ROPE
  cavusmustafa
  committed
  f7cf4e9

Commits

Commits on Jan 15, 2026