Repository navigation

Tool calls support improvements - #11

Closed

mario7421 wants to merge 144 commits into

ngxson:xsn/tool_callfrom

mario7421:tool-call-improvements

Closed

Tool calls support improvements#11
mario7421 wants to merge 144 commits into
ngxson:xsn/tool_callfrom
mario7421:tool-call-improvements

Commits on Aug 30, 2024
- Correct typo run_llama2.sh > run-llama2.sh (#9149)
  MakeDecisionWorth
  authored
  cddae48
- llama : fix llama_split_mode enum values in main_gpu document (#9057)
  kou
  authored
  0ab30f8
Commits on Aug 31, 2024
- llama : fix typo in xcda_array_view comment [no ci] (#9132)
  danbev
  authored
  49271ef
- sgemm : improved Q4_0 and Q8_0 performance via 4xN and Mx4 gemm (#8908)
  Srihari-mcw
  authored
  ea5d747
- nix: fix CUDA build - replace deprecated autoAddOpenGLRunpathHook
  
  enolan
  authored and
  SomeoneSerge
  committed
  a47667c
Commits on Sep 1, 2024
- llama : support RWKV v6 models (#8980)
  
  authored
  8f1d81a
Commits on Sep 2, 2024
- llama : minor style
  ggerganov
  committed
  c6d4cb4
- build(nix): Package gguf-py (#5664)
  ditsuke
  authored
  9c1ba55
- llama-cli : remove duplicated log message (#9275)
  nbcsm
  authored
  b60074f
- server : refactor multitask handling (#9274)
  
  ngxson
  and
  ggerganov
  authored
  6e7d133
- ggml : add pthread includes on FreeBSD (#9258)
  yurivict
  authored
  f771d06
- docker : fix missing binaries in full-cuda image (#9278)
  slaren
  authored
  048de84
- src: make tail invalid when kv cell is intersection for mamba (#9249)
  kylo5aby
  authored
  f148516
- server : test script : add timeout for all requests (#9282)
  ngxson
  authored
  48baa61
Commits on Sep 3, 2024
- readme : refactor API section + remove old hot topics
  ggerganov
  authored
  b69a480
- llama-bench : add JSONL (NDJSON) output mode (#9288)
  akx
  authored
  8962422
- flake.lock: Update (#9261)
  
  ggerganov
  and
  github-actions[bot]
  authored
  7605ae7
Commits on Sep 4, 2024
- readme : rename result_format to response_format (#9300)
  iscy
  authored
  9379d3c
- rpc : make RPC servers come first in the device list (#9296)
  rgerganov
  authored
  82e3b03
- Fix broken links in docker.md (#9306)
  carlory
  authored
  c8671ae
- [SYCL] Fix DMMV dequantization (#9279)
  OuadiElfarouki
  authored
  5910ea9
- ggml : AVX2 support for Q4_0_8_8 (#8713)
  Srihari-mcw
  authored
  581c305
Commits on Sep 5, 2024
- llama-bench : fix NUL terminators in CPU name (#9313)
  slaren
  authored
  bdf314f
- cuda : fix defrag with quantized KV (#9319)
  slaren
  authored
  4db0478
- CMake fix: host for msvc compiler can only be x86 or x64 (#8624)
  Xarbirus
  authored
  1031771
- Update build.yml (#9184)
  awatuna
  authored
  32b2ec8
Commits on Sep 6, 2024
- ggml-quants : ternary packing for TriLMs and BitNet b1.58 (#8151)
  compilade
  authored
  9bc6db2
- Improve Vulkan shader build system (#9239)
  mtavenrath
  authored
  8ebe8dd
- server : fix missing lock (#9334)
  ngxson
  authored
  4a1411b
- ggml : fix build break for the vulkan-debug (#9265)
  cyzero-kim
  authored
  409dc4f
- batched-bench : add `--output-format jsonl` option (#9293)
  akx
  authored
  815b1fb
- llama-bench : log benchmark progress (#9287)
  akx
  authored
  134bc38
- server : simplify state machine for slot (#9283)
  
  ngxson
  and
  ggerganov
  authored
  9b2c24c
Commits on Sep 7, 2024
- ci : disable rocm image creation (#9340)
  slaren
  authored
  6c89eb0
- ggml : fix missing `cpu_set_t` on emscripten (#9336)
  ngxson
  authored
  947538a
- llama : refactor sampling v2 (#9294)
  ggerganov
  authored
  df270ef
- ggml : always check bounds on get_rows operations (#9354)
  slaren
  authored
  e32d081
- common : refactor arg parser (#9308)
  ngxson
  authored
  1b9ae51
- llamafile : disable sgemm for batch-size 1 (#9330)
  netrunnereve
  authored
  e536426
- llama : sanitize invalid tokens (#9357)
  ggerganov
  authored
  faf69d4
- llama : fix empty ring buffer push (#9358)
  ggerganov
  authored
  f12295b
- llama.android : fix build (#9350)
  ggerganov
  authored
  a5b5d9a
Commits on Sep 8, 2024
- llama : set attrs of mislabelled EOT/EOM tokens (#9348)
  bakkot
  authored
  fbb7fcf
- ggml : fix cont with transposed tensors when one dimension is 1 (ggml/934)
  
  smeso
  and
  ggerganov
  committed
  efe6a83
- cuda : mark BF16 CONT as unsupported
  ggerganov
  committed
  51d964a
- cann : add Ascend NPU support (whisper/2336)
  
  MengqingCao
  authored and
  ggerganov
  committed
  d2d3200
- cann : fix doxy (ggml/0)
  ggerganov
  committed
  ba1cf84
- ggml: fix ggml_graph_cpy undefined behavior (ggml/943)
  
  JohannesGaessler
  authored and
  ggerganov
  committed
  dbbebca
- tests: add gradient tests for all backends (ggml/932)
  
  JohannesGaessler
  authored and
  ggerganov
  committed
  202084d
- vulkan: correctly report support for OP_CONT (ggml/946)
  
  smeso
  authored and
  ggerganov
  committed
  9cb9260
- vulkan: add dryrun support to sin and cos ops (ggml/947)
  
  smeso
  authored and
  ggerganov
  committed
  406c1a3
- scripts : option to increase git patch context
  ggerganov
  committed
  60a3107
- sync : ggml
  ggerganov
  committed
  385decb
- metal : update support condition for im2col + fix warning (#0)
  ggerganov
  committed
  a876861
- imatrix : fix arg parser for imatrix (#9366)
  ngxson
  authored
  00b02bb
- llama : sanitize tokens in the upper bound (#9359)
  slaren
  authored
  eae5971
- [SYCL] add check malloc result on device (#9346)
  
  NeoZhangJianyu
  and
  arthw
  authored
  2a358fb
- llama : refactor samplers internal implementation (#9370)
  slaren
  authored
  19f4a7b
- common : restore --n-gpu-layers (#9371)
  slaren
  authored
  a249843
- common : bring back missing args, add env var duplication check (#9375)
  ngxson
  authored
  3f7ccfd
- cuda : fix FA Q src index (1 -> 0) (#9374)
  ggerganov
  authored
  e079bff
- Overlap cmdbuffer creation and cmdbuffer execution in Vulkan backend by submitting smaller cmdbuffers early. (#9118)
  mtavenrath
  authored
  daa9623
Commits on Sep 9, 2024
- Arm AArch64: Documentation updates (#9321)
  eddnjjn
  authored
  b2e89a3
- rpc : update README [no ci] (#9320)
  rgerganov
  authored
  54f376d
- readme : add LLMUnity to UI projects (#9381)
  amakropoulos
  authored
  5ed0875
- CUDA: fix variable name conflict for Windows build (#9382)
  JohannesGaessler
  authored
  8e6e2fb
- readme : update hot topics
  ggerganov
  authored
  38ca6f6
- llama : minor sampling refactor (2) (#9386)
  slaren
  authored
  5fb5e24
- ggml : vector length agnostic SVE support (#9290)
  
  Vithulep
  and
  ggerganov
  authored
  5fac4d5
- rpc : fix segfault with nkvo (#9389)
  
  rgerganov
  and
  slaren
  authored
  293bebe
- common : move arg parser code to `arg.cpp` (#9388)
  
  ngxson
  and
  ggerganov
  authored
  bfe76d4
Commits on Sep 10, 2024
- make : do not run llama-gen-docs when building (#9399)
  slaren
  authored
  fb3f249
- RWKV v6: Add time_mix_decay_w1/w2 in quant exclusion list (#9387)
  MollySophia
  authored
  0b4ac75
- llama : update llm_build_copy_mask_state comment [no ci] (#9385)
  danbev
  authored
  83008b7
- metal : fix compile warning with GGML_METAL_NDEBUG (#0)
  ggerganov
  committed
  00ba2ff
- llama : move random seed generation to the samplers (#9398)
  slaren
  authored
  49006c6
- enable --special arg for llama-server (#9419)
  matteoserva
  authored
  8d300bd
- arg : bring back missing ifdef (#9411)
  ngxson
  authored
  6cd4e03
- flake.lock: Update (#9360)
  
  ggerganov
  and
  github-actions[bot]
  authored
  cb9c933
Commits on Sep 11, 2024
- sycl : update support conditions (#9394)
  Alberto Cabrera Pérez
  authored
  51b6038
- musa: remove Clang builtins mapping (#9421)
  yeahdongcn
  authored
  b34e023
- batched-bench : remove unused code (#9305)
  ggerganov
  authored
  d2b496b
- CUDA: fix --split-mode row race condition (#9413)
  JohannesGaessler
  authored
  5af118e
- feat: Implements retrying logic for downloading models using --model-url flag (#9255)
  
  authored
  67155ab
- files : remove accidentally added `lora_test` submodule (#9430)
  ngxson
  authored
  5bb2c5d
- llava : correct args for minicpmv-cli (#9429)
  ngxson
  authored
  0996c55
- py : support converting local models (#7547)
  EvilFreelancer
  authored
  8db003a
- llama : skip token bounds check when evaluating embeddings (#9437)
  slaren
  authored
  1b28061
Commits on Sep 12, 2024
- Add Jais to list of supported models (#9439)
  
  fmz
  and
  fmz
  authored
  449ccfb
- cann: Fix error when running a non-exist op (#9424)
  bachelor-dou
  authored
  df4b794
- enhance run script to be easy to change the parameters (#9448)
  
  NeoZhangJianyu
  and
  arthw
  authored
  c9c8575
- ggml : hide ggml_object, ggml_cgraph, ggml_hash_set (#9408)
  ggerganov
  authored
  d6a04f8
- riscv : modify Makefile and add a RISCV_VECT to print log info (#9442)
  Tameem-10xE
  authored
  2b00fa7
- py : add special tokens in hf_converter for RWKV v6 (#9428)
  MollySophia
  authored
  39f852f
- cmake : fixed the order of linking libraries for llama-quantize (#9450)
  Xarbirus
  authored
  ff76e18
- ci : bump actions/checkout to v4 (#9377)
  trivikr
  authored
  3c26a16
- py : add Phi-1.5/Phi-2 tokenizer (#9361)
  daminho
  authored
  c837981
- ci : update HIP SDK to 24.Q3 (ROCm 6.1) (#9329)
  no1wudi
  authored
  4dc4f5f
- cmake : fix for builds without `GGML_CDEF_PUBLIC` (#9338)
  Xarbirus
  authored
  2a82511
- lora : raise error if lm_head is ignored (#9103)
  ngxson
  authored
  d4c3c10
- llava : fix the script error in MobileVLM README (#9054)
  fengerhu1
  authored
  e665744
- cann: Add host buffer type for Ascend NPU (#9406)
  bachelor-dou
  authored
  e6b7801
- server : Add option to return token pieces in /tokenize endpoint (#9108)
  
  mathijshenquet
  and
  ngxson
  authored
  7820364
Commits on Sep 13, 2024
- feat: remove a sampler from a chain (#9445)
  giladgd
  authored
  bd35cb0
- llama : llama_perf + option to disable timings during decode (#9355)
  
  ggerganov
  and
  ngxson
  authored
  0abc6a2
- server : add loading html page while model is loading (#9468)
  
  ngxson
  and
  VJHack
  authored
  feff4aa
Commits on Sep 14, 2024
Commits on Sep 15, 2024
- cmake : try to fix sycl+intel build (#9487)
  Xarbirus
  authored
  7596487
- readme : update tools list (#9475)
  OLSecret
  authored
  d6b37c8
- py : add "LLaMAForCausalLM" conversion support (#9485)
  
  csabakecskemeti
  and
  Csaba Kecskemeti
  authored
  3c7989f
- cmake : correct order of sycl flags (#9497)
  Xarbirus
  authored
  6988da9
- gguf-split : add basic checks (#9499)
  slaren
  authored
  e6deac3
- common : reimplement logging (#9418)
  ggerganov
  authored
  6262d13
Commits on Sep 16, 2024
- flake.lock: Update (#9488)
  ggerganov
  authored
  90a2fff
- metal : handle zero-sized allocs (#9466)
  ggerganov
  authored
  c4965a6
- main : option to disable context shift (#9484)
  
  VJHack
  and
  ggerganov
  authored
  441b72b
- llama : support MiniCPM3 (#9322)
  
  CarryFun
  and
  Fanruikai
  authored
  95ca851
- llama : support OLMoE (#9462)
  2015aroras
  authored
  0aadac1
- ggml : IQ4_NL sgemm + Q4_0 AVX optimization (#9422)
  netrunnereve
  authored
  5c3d0f1
- cmake : do not hide GGML options + rename option (#9465)
  ggerganov
  authored
  19514d6
- convert : identify missing model files (#9397)
  compilade
  authored
  d54c21d
- ggml : link MATH_LIBRARY not by its full path (#9339)
  Xarbirus
  authored
  a6a3a5c
- llama : rename n_embed to n_embd in rwkv6_time_mix (#9504)
  danbev
  authored
  acb2c32
- ggml : move common CPU backend impl to new header (#9509)
  slaren
  authored
  23e0d70
Commits on Sep 17, 2024
- llama : add llama_n_head() (#9512)
  Xarbirus
  authored
  37f3a38
- llama : support IBM Granite architecture (#9412)
  gabe-l-hart
  authored
  0d2ec43
- unicode : add <algorithm> (#9508)
  ykhrustalev
  authored
  503147a
- threadpool : skip polling for unused threads (#9461)
  max-krasnyansky
  authored
  0226613
- llama : fix n_vocab init for 'no_vocab' case (#9511)
  
  Xarbirus
  and
  ggerganov
  authored
  8344ef5
- arg : add env variable for parallel (#9513)
  bertwagner
  authored
  8b836ae
- llama-bench: correct argument parsing error message (#9524)
  Xarbirus
  authored
  7be099f
Commits on Sep 18, 2024
Commits on Sep 19, 2024
- imatrix : disable prompt escape by default (#9543)
  CISC
  authored
  eca0fab
- server : clean-up completed tasks from waiting list (#9531)
  ggerganov
  authored
  6026da5
Commits on Sep 20, 2024
- Tool calls support improvements (support null content in messages, handle correctly previous messages that contains tool calls, improvements in prompt format)
  mario7421
  committed
  d3830ad
Commits on Sep 22, 2024
- Merge branch 'master' into tool-call-improvements
  mario7421
  committed
  b67b817

Commits

Commits on Aug 30, 2024

Commits on Aug 31, 2024

Commits on Sep 1, 2024

Commits on Sep 2, 2024

Commits on Sep 3, 2024

Commits on Sep 4, 2024

Commits on Sep 5, 2024

Commits on Sep 6, 2024

Commits on Sep 7, 2024

Commits on Sep 8, 2024

Commits on Sep 9, 2024

Commits on Sep 10, 2024

Commits on Sep 11, 2024

Commits on Sep 12, 2024

Commits on Sep 13, 2024

Commits on Sep 14, 2024

Commits on Sep 15, 2024

Commits on Sep 16, 2024