This repository was archived by the owner on Jun 5, 2025. It is now read-only.
Repository navigation
This repository was archived by the owner on Jun 5, 2025. It is now read-only.
Remove LiteLLM and build native implementation #348
Description
Activity
Hi @lukehinds what issues do you hit with LiteLLM ? - litellm maintainer
hey @ishaan-jaff , sorry for late reply and thanks for all your contributions.
Some of the main things:
- logging is really noisy, can't figure out how to setlogger or monkey patch it out: [Feature]: Turn off or tune down noisy library logging BerriAI/litellm#4528
- what we gain from litellm is an easy way to talk to multiple providers, but since we receive the provider-native data and litellm expects to receive openAI formatted data and replies with openAI formatted data, we still need to implement our own normalizers
- we get behaviour like throwing away some data fields. This has not been a problem so far, but might be in the future.
- Use ollama python client for completion #241
Not knocking your efforts, LiteLLM is an awesome project and you have a lot of support coverage to maintain! We just need something it bit more specific to our needs.
While LiteLLM is very useful (huge coverage of providers), we always invariably hit issues around certain idiomatic approaches taken by LiteLLM, and feel we need to take a different approach. It therefore makes sense to build our own native implementation
If possible, we should initially seek to support both LiteLLM and the CodeGate implementation (if viable and not to heavy a lift).