Repository navigation
feat(providers): watsonx.ai embeddings - #1356
Merged
vgmartinez merged 3 commits intoNov 15, 2026
Merged
Conversation
The proxy prices a call from canonical.Usage, but the OpenAI embeddings adapter only kept the usage inside the embedding extension. Every OpenAI-format embeddings call (OpenAI, Azure, Mistral, openai_compatible) was therefore recorded without tokens or cost, and dollar budgets never counted it. The Vertex predict and Bedrock Titan adapters already set Usage; this one now does too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KF89yDnBWqZ6yB3bLqAtb9
A watsonx registry now serves /v1/embeddings. The client translates the
OpenAI embeddings body to /ml/v1/text/embeddings with the registry's IAM
token and project, and the answer back to OpenAI's shape.
- input: a string or a list of strings. Token arrays are a 400, since
watsonx.ai embeds text. More than 1,000 inputs (IBM's limit per call)
are split into calls whose vectors are joined in order and whose
token counts are summed.
- encoding_format base64, which the OpenAI SDKs request by default,
returns little-endian float32 vectors as OpenAI does.
- dimensions is a 400: watsonx.ai models return their own fixed size,
and a silently different length would break a vector store.
- usage.prompt_tokens comes from input_token_count, so calls are priced.
- The region listing now includes embedding models. A live model can
carry capability flags ("chat", "embed"), which the catalog uses when
it has no row of its own for the model, so a watsonx embedding model
is not shown as a chat model.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KF89yDnBWqZ6yB3bLqAtb9
Contributor
Trivy code scan — HIGH/CRITICALNo HIGH or CRITICAL vulnerabilities found. (run) |
develop's A2A skill scan (#1353) added skill_scan fields to the agent card and registry responses without regenerating docs/, so OpenAPI Drift fails on develop and on every PR based on it. This is the output of make docs with the pinned swag; nothing else changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KF89yDnBWqZ6yB3bLqAtb9
vgmartinez
marked this pull request as ready for review
November 15, 2026 16:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #1349 (native IBM watsonx.ai provider). A watsonx registry now serves
/v1/embeddings, and embeddings calls are now priced: a fix in the shared OpenAI embeddings adapter also makes OpenAI, Azure, Mistral and openai_compatible embeddings record their tokens.Commits
1.
fix(adapter): report the tokens of an OpenAI embeddings responsedecodeResponseMetainpkg/app/proxy/provider.goprices a call fromcanonical.Usage.OpenAIEmbeddingsAdapter.DecodeResponseonly kept the usage insideProviderExtensions["embedding"]. As a result, every OpenAI-format embeddings call ondevelopis recorded without tokens or cost, and dollar budgets never count it. That covers OpenAI, Azure, Mistral, openai_compatible, and watsonx through this PR.The Vertex predict and Bedrock Titan adapters already set
Usage; this one now does too.TestProviderInvoke_VertexPredictCrossFormatToOpenAInow assertsresp.Usage. It failed before the one-line fix and passes after.Cohere embeddings have the same symptom for a different reason:
cohereEmbedResponsedoes not readmeta.billed_units. That is left for a separate change.2.
feat(providers): watsonx.ai embeddingsThe client translates the OpenAI embeddings body to
POST /ml/v1/text/embeddings(with the registry's IAM token and project ID) and the answer back to OpenAI's shape:input: string or list of stringsinputs. More than 1,000 (IBM's per-call limit) are split into several calls; vectors are joined in order and token counts summed.input: token arrays400: watsonx.ai embeds text.encoding_format: base64(the OpenAI SDKs' default)dimensions400: watsonx.ai models return their own fixed size, and a silently different length would break a vector store.modelmodel_id, with the same precedence as chat (configured, then body, then default).{object: list, data[].embedding, model, usage: {prompt_tokens, total_tokens}}.usage.prompt_tokenscomes frominput_token_count, so the call is priced.watsonx errors (for example IBM's 512-token input limit) come back as OpenAI error envelopes with IBM's trace id. A 401 retries once with a fresh IAM token, as chat does.
Model list:
function: embedding) as well as chat ones. Frankfurt and Dallas both serveibm/granite-embedding-278m-multilingual,intfloat/multilingual-e5-large, twoibm/slate-*models andsentence-transformers/all-minilm-l6-v2.providers.LiveModel/appcatalog.LiveModelgain optionalCapabilities("chat","embed").deploymentModelsandLiveCatalogLister.Listuse them when the catalog has no row of its own for a model, so a watsonx embedding model is not shown as a chat model. A catalog row keeps its own flags.SupportsCapability(watsonx, embeddings)is now true, so routing puts watsonx registries in the embeddings pool. The console's provider card picks up the capability from the catalog.3.
docs(openapi): regenerate for the A2A skill scan (not this PR's change)The first CI run failed OpenAPI Drift. Running
make docswith the pinnedswagondevelopd7f267e alone produces the same diff:skill_scanfields from #1353, which merged without regeneratingdocs/. Nothing in it concerns watsonx. This commit is thatmake docsoutput.go test ./docs/...passes, and OpenAPI Drift is green on 00d04d3. It can also go todevelopon its own.Testing
pkg/infra/providers/watsonx(90% coverage) has new tests covering:developd7f267e):go test -race ./...: 215 packages ok, 0 failures.go test -tags functional -race -p 1 ./tests/functional/... ./pkg/infra/database/migrations) against Postgres 16 and Redis: 717 passed, 0 failed, 4 skipped. The skips are environmental: noOPENAI_API_KEY, plan rate limiter disabled, needs a real TrustGuard, and RUN-1621.golangci-lintv2.14.0 on the touched packages: 0 issues.addlicense -checkandscripts/check-comments.shpass.Not yet verified against live watsonx: no IBM Cloud key was available here. Request and response shapes follow IBM's
ibm-watsonx-aiSDK (model_id,inputs,project_id,parameters.return_options;results[].embedding,input_token_count; 1,000 inputs per call). The regional model specs were read live. Please run one embeddings call with a real key before merging.Not included
truncate_input_tokens: a text over 512 tokens is IBM's400, like an over-long OpenAI input. A registry option to truncate could follow if needed.ibm/*) cover them.Docs: NeuralTrust/docs#561.
🤖 Generated with Claude Code
https://claude.ai/code/session_01KF89yDnBWqZ6yB3bLqAtb9