Implement the im_start / im_stop funkiness #94

Activity

1. lukehinds

   assigned

   jhrozek

   on Nov 26, 2024
2. lukehinds

   added this to the 0.1.0 (flagship) milestone on Nov 26, 2024
3. jhrozek

   added

   flagship[python]

   on Nov 27, 2024
4. jhrozek

   closed this as completedin #121on Nov 29, 2024
5. jhrozek

   added a commit that references this issue on Nov 29, 2024

   Normalize and denormalize llamacpp streaming reply

   ...

   This issue will close once commit dc988a3 is merged into the 'main' branch.

   dc988a3
6. jhrozek commented on Nov 29, 2024

   

   jhrozek

   on Nov 29, 2024

   Contributor

   More actions

   I actually realized we need to do this for vllm as well, while it does use the openai message format it also uses the im_start/im_end tags. We side-stepped the problem in the llama.cpp provider to not use them, but not the vllm provider.
7. jhrozek

   reopened this on Nov 29, 2024
8. lukehinds commented on Dec 2, 2024

   

   lukehinds

   on Dec 2, 2024

   Author

   More actions

   How are we on this @jhrozek ?
9. lukehinds commented on Dec 2, 2024

   

   lukehinds

   on Dec 2, 2024

   Author

   More actions

   SHould be ready to merge @jhrozek will comment out function for now, in case we need it, but until then the provider routing should take care fo this.
10. jhrozek commented on Dec 2, 2024

    

    jhrozek

    on Dec 2, 2024

    Contributor

    More actions

    This was finally merged
11. jhrozek

    closed this as completedon Dec 2, 2024