[FEATURE] Add api_base support for LiteLLM semantic embedding providers

[lastguru-net]

Problem

Basic Memory's experimental LiteLLM embedding provider currently supports provider/model selection, dimensions, role-specific input_type, batching, and request concurrency, but it does not expose an api_base / custom endpoint setting for embedding calls.

That makes it awkward or impossible to use Basic Memory with OpenAI-compatible embedding servers that are not hosted at the default provider endpoint.

A concrete example is running llama-server --embedding directly and exposing its OpenAI-compatible /v1/embeddings API. LiteLLM supports this shape by calling embeddings with an openai/... model string and an api_base, but Basic Memory does not currently provide a config field that reaches litellm.aembedding(...).

Why this matters

This is useful for local and self-hosted embedding deployments.

For example, Ollama can accept batched embedding input, but in some local setups it still runs embedding work through a single llama.cpp slot, which makes large Basic Memory reindexes extremely slow. Running llama-server --embedding directly with multiple slots can be materially faster while still exposing an OpenAI-compatible embeddings endpoint.

Current code shape

From a quick read of current main:

• src/basic_memory/repository/litellm_provider.py

  • LiteLLMEmbeddingProvider accepts api_key, timeout, dimensions, batch size, concurrency, and role settings.
  • _embed() builds params for litellm.aembedding(...).
  • The params include model, input, drop_params, timeout, optionally dimensions, api_key, and input_type.
  • There is no api_base.

• src/basic_memory/config.py

  • Semantic embedding config includes provider, model, dimensions, dimension forwarding, batch size, request concurrency, document/query input types, sync batch size, and FastEmbed runtime knobs.
  • There is no endpoint/base URL field for semantic embedding providers.

• src/basic_memory/repository/embedding_provider_factory.py

  • The factory passes LiteLLM model, batch size, request concurrency, input types, and dimension forwarding into LiteLLMEmbeddingProvider.
  • There is no endpoint/base URL field in the provider cache key or provider constructor call.

Suggested solution

Add api_base option and pass it to the LiteLLM embedding providers.

[phernandez]

@claude can you look at this?

[claude]

Claude finished @phernandez's task in 3m 2s —— View job • Create PR ➔

---

Implementing api_base support for LiteLLM embedding providers

• Gather context and read relevant files
• Understand the request — thread api_base through config → factory → provider → litellm.aembedding()
• Add semantic_embedding_api_base field to BasicMemoryConfig in config.py
• Add api_base param to LiteLLMEmbeddingProvider (init, runtime_log_attrs, identity_key, _embed)
• Update embedding_provider_factory.py cache key and factory to pass api_base
• Add tests for api_base forwarding (provider + factory)
• Push changes — blocked by GitHub App write permission (see below)

Changes made

src/basic_memory/config.py — new semantic_embedding_api_base: str | None field (default None), positioned near the other LiteLLM-specific config fields. Documented as applicable only when semantic_embedding_provider='litellm'.

src/basic_memory/repository/litellm_provider.py:

• __init__ — new api_base: str | None = None keyword param, stored as self._api_base
• runtime_log_attrs — includes api_base in startup log output when set
• identity_key — encodes api_base so changing the endpoint invalidates cached vectors
• _embed — passes api_base to litellm.aembedding(**params) when set

src/basic_memory/repository/embedding_provider_factory.py:

• ProviderCacheKey type — added str | None slot for api_base
• _provider_cache_key — includes app_config.semantic_embedding_api_base
• create_embedding_provider — passes api_base=app_config.semantic_embedding_api_base to LiteLLMEmbeddingProvider

tests/repository/test_litellm_provider.py — five new tests:

• test_litellm_provider_api_base_forwarded — verifies api_base reaches litellm.aembedding
• test_litellm_provider_api_base_omitted_when_none — verifies api_base is absent by default
• test_runtime_log_attrs_includes_api_base_when_set
• test_runtime_log_attrs_omits_api_base_when_not_set
• test_identity_key_includes_api_base
• test_factory_forwards_litellm_api_base — verifies the factory wires the config field through

Usage example (after merging)

{
  "semantic_embedding_provider": "litellm",
  "semantic_embedding_model": "openai/my-model",
  "semantic_embedding_api_base": "http://localhost:8080",
  "semantic_embedding_dimensions": 1536
}

Or via environment variable: BASIC_MEMORY_SEMANTIC_EMBEDDING_API_BASE=http://localhost:8080

Push status

The commit is ready locally but the push was denied (Permission to basicmachines-co/basic-memory.git denied to github-actions[bot]). A maintainer will need to push or merge the branch claude/issue-1005-20260617-2159.

[kingpro1985]

I saw this while looking at custom model/provider setup issues: "[FEATURE] Add api_base support for LiteLLM semantic embedding providers".

If it helps your testing, CloudGPU exposes a hosted OpenAI-compatible endpoint that can route to GPT, Claude, Gemini, DeepSeek, Kimi and GLM behind one API key.

Base URL: https://gpuapis.com/api/v1
Setup guide: https://gpuapis.com/use-cases/deepseek-gemini-claude-api/

I can share a small test key/credit and a cURL request if you want to verify it in your current client before doing any integration work.