[Enhancement] LiteLLM Integration

README.md

@@ -93,7 +93,7 @@ poetry run python main.py --task examples.glaive_code_assistant --num_records=1
 See full steps in <a href="https://servicenow.github.io/SyGra/sygra_library/">Sygra Library</a>.
 
 ```bash
-pip install sygra   
+pip install sygra  
 ```
 
 ```python
@@ -137,6 +137,8 @@ For example, openai client is being supported by Huggingface TGI, vLLM server an
 
 <kbd> ![ModelClient](https://raw.githubusercontent.com/ServiceNow/SyGra/refs/heads/main/docs/resources/images/component_model_client.png) </kbd>
 
+####  Integration with [LiteLLM](https://github.com/BerriAI/litellm)
+SyGra now integrates with LiteLLM—unlocking more flexibility, expanded model compatibility, and seamless end-to-end integrations.
 ## Task Components
 
 SyGra supports extendability and ease of implementation—most tasks are defined as graph configuration YAML files. Each task consists of two major components: a graph configuration and Python code to define conditions and processors.
@@ -145,7 +147,7 @@ YAML contains various parts:
 - **Data configuration** : Configure file or huggingface as source and sink for the task.
 - **Data transformation** : Configuration to transform the data into the format it can be used in the graph.
 - **Node configuration** : Configure nodes and corresponding properties, preprocessor and post processor.
-- **Edge configuration** : Connect the nodes configured above with or without conditions. 
+- **Edge configuration** : Connect the nodes configured above with or without conditions.
 - **Output configuration** : Configuration for data tranformation before writing the data into sink.
 
 A node is defined by the node module, supporting types like LLM call, multiple LLM call, lambda node, and sampler node.  
@@ -165,7 +167,7 @@ To contact us, please send us an [email](mailto:sygra_team@servicenow.com)!
 
 The package is licensed by ServiceNow, Inc. under the Apache 2.0 license. See [LICENSE](LICENSE) for more details.
 
----
+## Questions?
 
-**Questions?**  
+Ask SyGra's [DeepWiki](https://deepwiki.com/ServiceNow/SyGra) </br>
 Open an [issue](https://github.com/ServiceNow/SyGra/issues) or start a [discussion](https://github.com/ServiceNow/SyGra/discussions)! Contributions are welcome.

docs/getting_started/model_configuration.md

@@ -31,7 +31,7 @@ For `mixtral_8x7b` and `gpt4`, set:
 You should use a `.env` file at the project root or set these in your shell environment.
 
 **Note:**
-If you want to define a list of URLs for any model, you can use pipe (`|`) as a separator. For example, if you have a model called `mixtral_8x7b` with URLs `https://myserver/models/mixtral-8x7b` and `https://myserver/models/mixtral-8x7b-2`, you can set the following environment variables as shown in examples below. 
+If you want to define a list of URLs for any model, you can use pipe (`|`) as a separator. For example, if you have a model called `mixtral_8x7b` with URLs `https://myserver/models/mixtral-8x7b` and `https://myserver/models/mixtral-8x7b-2`, you can set the following environment variables as shown in examples below.
 
 ### Example `.env`:
 
@@ -45,23 +45,24 @@ SYGRA_MIXTRAL_8X7B_CHAT_TEMPLATE={% for m in messages %} ... {% endfor %}
 ### Configuration Properties
 
 
-| Key                         | Description                                                                                                                |
-|-----------------------------|----------------------------------------------------------------------------------------------------------------------------|
-| `model_type`                | Type of backend server (`tgi`, `vllm`, `openai`, `azure_openai`, `azure`, `mistralai`, `ollama`, `triton`)                 |
-| `model_name`                | Model name for your deployments (for Azure/Azure OpenAI)                                                                   |
-| `api_version`               | API version for Azure or Azure OpenAI                                                                                      |
-| `hf_chat_template_model_id` | Hugging Face model ID                                                                                                      |
-| `multi_modal`               | *(Optional)* Boolean: Set this to false if the model is not multi-modal (default: true)                                    |
-| `completions_api`           | *(Optional)* Boolean: use completions API instead of chat completions API (default: false)                                 |
-| `modify_tokenizer`          | *(Optional)* Boolean: apply custom chat template and modify the base model tokenizer (default: false)                      |
-| `special_tokens`            | *(Optional)* List of special stop tokens used in generation                                                                |
-| `post_process`              | *(Optional)* Post processor after model inference (e.g. `models.model_postprocessor.RemoveThinkData`)                      |
-| `parameters`                | *(Optional)* Generation parameters (see below)                                                                             |
-| `chat_template_params`      | *(Optional)* Chat template parameters (e.g. `reasoning_effort` for `gpt-oss-120b`) <br/> when `completions_api` is enabled |
-| `ssl_verify`                | *(Optional)* Verify SSL certificate (default: true)                                                                        |
-| `ssl_cert`                  | *(Optional)* Path to SSL certificate file                                                                                  |
-| `json_payload`              | *(Optional)* Boolean: use JSON payload instead of JSON string for `http client` based models (default: false)              |
-| `headers`                   | *(Optional)* Dictionary of headers to be sent with the request for `http client` based models                              |
+| Key                         | Description                                                                                                                                             |
+|-----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `model_type`                | Type of backend server (`tgi`, `vllm`, `openai`, `azure_openai`, `azure`, `mistralai`, `ollama`, `triton`)                                              |
+| `model_name`                | Model name for your deployments (for Azure/Azure OpenAI)                                                                                                |
+| `api_version`               | API version for Azure or Azure OpenAI                                                                                                                   |
+| `multi_modal`               | *(Optional)* Boolean: Set this to false if the model is not multi-modal (default: true)                                                                 |
+| `backend`                   | *(Optional)* Backend for the model (default: `litellm` for litellm supported models, `custom` for other models) Supported values: `litellm`, `custom`   |
+| `completions_api`           | *(Optional)* Boolean: use completions API instead of chat completions API (default: false) <br/> Supported models: `tgi`, `vllm`, `ollama`              |
+| `hf_chat_template_model_id` | *(Optional)* Hugging Face model ID. Make sure to set this when completions_api is set to `true`                                                         |
+| `modify_tokenizer`          | *(Optional)* Boolean: apply custom chat template and modify the base model tokenizer (default: false)                                                   |
+| `special_tokens`            | *(Optional)* List of special stop tokens used in generation                                                                                             |
+| `post_process`              | *(Optional)* Post processor after model inference (e.g. `models.model_postprocessor.RemoveThinkData`)                                                   |
+| `parameters`                | *(Optional)* Generation parameters (see below)                                                                                                          |
+| `chat_template_params`      | *(Optional)* Chat template parameters (e.g. `reasoning_effort` for `gpt-oss-120b`) <br/> when `completions_api` is enabled                              |
+| `ssl_verify`                | *(Optional)* Verify SSL certificate (default: true)                                                                                                     |
+| `ssl_cert`                  | *(Optional)* Path to SSL certificate file                                                                                                               |
+| `json_payload`              | *(Optional)* Boolean: use JSON payload instead of JSON string for `http client` based models (default: false)                                           |
+| `headers`                   | *(Optional)* Dictionary of headers to be sent with the request for `http client` based models                                                           |
 ![Note](https://img.shields.io/badge/Note-important-yellow)  
 > - Do **not** include `url`, `auth_token`, or `api_key` in your YAML config. These are sourced from environment variables as described above.<br>
 > - If you want to set **ssl_verify** to **false** globally, you can set `ssl_verify:false` under `model_config` section in config/configuration.yaml
@@ -148,7 +149,7 @@ qwen3-32b-triton:
   hf_chat_template_model_id: Qwen/Qwen3-32B
   post_process: sygra.core.models.model_postprocessor.RemoveThinkData
   model_type: triton
-  payload_key: default 
+  payload_key: default
   # Uses default payload format defined in config/payload_cfg.json.
   # Add/Update the payload_cfg.json if you need to use a different payload format with new key.
   parameters:
@@ -159,4 +160,4 @@ qwen3-32b-triton:
 > **Important:**
 If you set modify_tokenizer: true for a model, you must provide the corresponding chat template in your environment as SYGRA_<MODEL_NAME>_CHAT_TEMPLATE.
 Otherwise, exception will be raised during the model initialization.
----
+---

main.py

@@ -31,10 +31,9 @@ def check_model_availability(task_name):
             logger.error(f"Model {mn} has no model configuration. Exiting the process.")
             sys.exit(1)
         mc["name"] = mn
-        backend = mc.get("backend", "default")
 
         # create model object for inference
-        mod = ModelFactory.create_model(mc, backend)
+        mod = ModelFactory.create_model(mc)
         model_param = ModelParams(url=mc.get("url"), auth_token=mc.get("auth_token"))
         # inference
         status = mod.ping()

pyproject.toml

@@ -57,6 +57,8 @@ dependencies = [
   "soundfile>=0.13,<1.0",
   "types-pyyaml>=6.0,<7.0",
   "fasttext-wheel (>=0.9.2,<0.10.0)",
+  "litellm (>=1.79.3,<2.0.0)",
+  "boto3 (>=1.40.71,<2.0.0)",
 ]
 
 [project.optional-dependencies]
@@ -138,6 +140,8 @@ pandarallel = "^1.6"
 sentence-transformers = "^5.1"
 soundfile = "^0.13"
 types-pyyaml = "^6.0"
+litellm = "^1.79.3"
+boto3 = "^1.40.71"
 
 [tool.poetry.group.ui.dependencies]
 streamlit = { version = "^1.49", markers = "python_version != '3.9.7'" }

sygra/core/graph/nodes/agent_node.py

@@ -34,7 +34,9 @@ def _initialize_model(self):
         configuration from the node configuration.
         """
 
-        self.model = ModelFactory.create_model(self.node_config["model"], constants.BACKEND)
+        self.model = ModelFactory.create_model(
+            self.node_config["model"], constants.MODEL_BACKEND_LANGGRAPH
+        )
 
     async def _exec_wrapper(self, state: dict[str, Any]) -> dict[str, Any]:
 

sygra/core/models/client/client_factory.py

@@ -291,7 +291,6 @@ def _create_http_client(
             An HttpClient instance
         """
         model_config = utils.get_updated_model_config(model_config)
-        utils.validate_required_keys(["url", "auth_token"], model_config, "model")
         ssl_verify = bool(model_config.get("ssl_verify", True))
         ssl_cert = model_config.get("ssl_cert")
         json_payload = model_config.get("json_payload", False)