Negative Prompt support - not found

[zsogitbe]

llama.cpp supports negative prompt strings (see main.cpp in Examples), but I cannot find any support here. Haven't I looked well or it does not exist yet?

[martindevans]

LLamaSharp doesn't support negative prompts in an easy to use way.

All the necessary low level components are there, but they haven't been put together into an easy to use high level system that allows you to use it by just supplying a string as a negative prompt.

[zsogitbe]

I see. This is an important feature since we can control model ethics with it easily. It is probably not a lot of work for someone who knows the code already very well.

[martindevans]

Applying the guidance logits is fairly simple (I've actually just put together the code to do that).

The tricky part is that you need a second entire sequence running in parallel to the first to generate those guidance logits. This really needs batched execution to be fast, which is supported in LLamaSharp, but only very recently and it doesn't work with any kind of high level executor yet.

[martindevans]

@zsogitbe I've implemented a demo for how to use classifier free guidance using the new batched executor. As I mentioned this is not as high level as the existing executors, but it should at least demo how things work at the lower level for now. If you could pull #536, test it out and leave any feedback on that PR, it would be very appreciated!

[zsogitbe]

Added a possible bug issue: ggml-org/llama.cpp#5709