Note: This issue was copied from ggml-org#4491
Original Author: @cebtenzzre
Original Issue Number: ggml-org#4491
Created: 2023-12-16T03:15:15Z
We should deprecate these functions so people do not use them in new code. But we would first have to stop using them in our own code, which is easier said than done. For example, I tried to understand what beam search was using for a batch size, and gave up:
https://github.com/ggerganov/llama.cpp/blob/88ae8952b65cbf32eb1f5703681ea592e510e570/llama.cpp#L8000-L8003
see also ggml-org#4274
Note: This issue was copied from ggml-org#4491
Original Author: @cebtenzzre
Original Issue Number: ggml-org#4491
Created: 2023-12-16T03:15:15Z
We should deprecate these functions so people do not use them in new code. But we would first have to stop using them in our own code, which is easier said than done. For example, I tried to understand what beam search was using for a batch size, and gave up:
https://github.com/ggerganov/llama.cpp/blob/88ae8952b65cbf32eb1f5703681ea592e510e570/llama.cpp#L8000-L8003
see also ggml-org#4274