Skip to content

Gateway /v1/images (generations, then edits) #507

Closed

Description

@vgmartinez

Goal

Proxy OpenAI-compatible Images API through TrustGate (POST /{consumer}/v1/images/generations, then edits) under the same consumer auth, policies, and multi-registry routing as chat.

Why

OpenRouter / LiteLLM / Portkey expose image generation as a first-class route. TrustGate path resolver today is chat / embeddings / rerank only.

Architecture (locked)

  • Capability is a property of the provider (e.g. ImagesClient on the provider client). Registry inherits it (API key + config only).
  • Consumer may attach many registries. For an images request:
    1. Build CandidateSet as today (intent, allowlists, roles).
    2. Pre-filter to registries whose provider implements images capability (do not rely only on late invoke type-assert + retry like embeddings today).
    3. LB / fallback only among those.
    4. Pinned model/registry that lacks images → terminal 400 (no silent hop).
    5. Auto/pool with zero capable registries → no backend available.
  • Never route outside the consumer’s configured registries.
  • Model layer: only image-capable model ids (catalog / allowlist).

Routes

TrustGate (consumer-facing) Upstream
POST /{consumer}/v1/images/generations OpenAI/Azure /v1/images/generations
POST /{consumer}/v1/images/edits (phase 2) multipart edits

OpenRouter uses POST /api/v1/images (not .../generations): expose OpenAI-shaped path to clients; map OR registries in the provider client.

Add CapabilityImages in proxy_path_resolver.go.

Provider rollout

  1. OpenAI + Azure OpenAI
  2. openaicompat
  3. OpenRouter (path map)
  4. Gemini native — out of v1 unless OpenAI-compat wire proven

Governance

Consumer auth; rate/size limits; model allowlist; metrics/cost (unpriced OK until #505 covers non-token).

Out of scope

Video (#509); chat vision parts (#510); marketplace provider.order.

Acceptance criteria

  • POST /{slug}/v1/images/generations for OpenAI registry
  • Candidate pre-filter by provider images capability; pin incapable → 400
  • Consumer with OpenAI+Groq routes images only to OpenAI when unpinned
  • Path resolver + functional tests + docs
  • PR only (never merge from automation)

Related

#506 models · #508 audio · #509 video · #510 chat multimodal · #503/#504 patterns

Activity

  1. vgmartinez commented on Aug 22, 2026

    @vgmartinez
    ContributorAuthor

    Design note: OpenRouter path mismatch

    OpenAI/Azure: POST /v1/images/generations.
    OpenRouter: POST /api/v1/images (not .../generations), plus GET /api/v1/images/models.

    TrustGate should expose the OpenAI-shaped consumer route (/{slug}/v1/images/generations) and map OpenRouter registries to OR’s /images in the provider client. Do not make clients learn two paths.

    Gemini native image (generateContent / Imagen predict): out of v1 unless an OpenAI-compat wire is proven.

  2. vgmartinez commented on Aug 22, 2026

    @vgmartinez
    ContributorAuthor

    Architecture lock (Victor): Registry vs Consumer

    • Registry = provider connection + API key → owns capabilities (chat / embeddings / images / audio_speech / audio_transcription / video / chat multimodal parts).
    • Consumer = may attach many registries + LB + fallback chain (existing forwarder / CandidateSet / ModelPolicies).

    How multimodal selects a backend

    1. Build the candidate set as today (intent, allowlists, roles).
    2. Filter out registries whose provider does not implement the required capability client (same idea as embeddings needing EmbeddingsClient).
    3. Run LB / fallback only among remaining consumer registries — never hop to a registry outside the consumer.
    4. Pinned model/registry (@provider/model / short model that resolves to one registry): if that registry lacks capability → 400 with clear reason (do not silently strip parts or re-route).
    5. Auto/pool with zero capable registries → existing style no backend available / no backends in pool.
    6. GET /v1/models (Proxy GET /v1/models for consumers (SDK model discovery) #506): union of models from the consumer’s registries, annotated/filtered by modality so UIs don’t offer Whisper on a chat-only registry.

    This is not OpenRouter marketplace shopping; it is capability-aware selection inside the consumer’s configured pool.

  3. vgmartinez commented on Aug 22, 2026

    @vgmartinez
    ContributorAuthor

    Implementation nuance vs today’s embeddings

    Embeddings/rerank currently type-assert at invoke; missing capability is retryable, so unpinned LB may try another registry after a failed attempt. That works but wastes a call and is awkward for multimodal.

    Prefer for #507–#510: capability filter on CandidateSet before LB/pin selection; keep invoke type-assert as belt-and-suspenders. Pinned incapable → terminal 400 (existing pin = no failover).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions