diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..a1dfa41 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,105 @@ +# @corbits/reranking — Architecture + +## Overview + +The package is a one-shot JSON client, not a streaming inference turn. A caller +hands `rerankDocuments` a query, a candidate list, a style-addressed config, and +transport dependencies. An adapter builds the provider request and parses the +reply into positions; the client maps those positions back onto caller ids and +sorts by score descending. + +## Components + +| Component | Role | +| --------- | ---- | +| `rerankDocuments` | Public orchestration: validate config, short-circuit empty input, resolve an adapter, send, map, sort. | +| `RerankAdapter` | One wire format: `buildRequest` / `parseResponse`, optional `extractRetryAfterMs`. Mirrors the inference harness `ProviderAdapter` so the two read as the same kind of object. | +| Adapter registry | Maps `apiStyle` strings to adapters. Built-ins cover TEI, Cohere (also Jina's `/rerank`), and Voyage. | +| JSON request runner | Single POST of a built request: classify failures, retry or abort, return a JSON body. | + +Callers inject `deps` (`fetch` and a scheduler). A full harness dependency bag +satisfies this structurally; assembling an inference-provider registry is not +required to rerank. + +## Control flow + +``` +query + docs + config + deps + │ + ▼ + validate config + │ + ├── docs.length === 0 → [] + │ + ▼ + registry.resolve(apiStyle) + │ + ▼ + adapter.buildRequest(query, docs, { baseURL, model, apiKey }) + │ + ▼ + JSON POST (classify + retry) + │ + ▼ + adapter.parseResponse(body) → [{ index, score }] + │ + ▼ + map through docs[index] → { id, score } + │ + ▼ + sort score descending +``` + +## Adapter registry + +Reranking is a real adapter boundary, not a knob: the built-in styles disagree +on path, request field names, and where scores live in the reply. + +`createRerankAdapterRegistry` closes over a private `Map` copy of the style → +adapter table. Lookups never consult `Object.prototype`, so an `apiStyle` from +config cannot resolve to an inherited function and fail later as an opaque +`TypeError`. Unknown styles fail immediately, by name. + +Pass `options.registry` to add a house format without forking the package. +`apiStyle` is an open string for that reason; the registry, not a closed enum, +is the source of truth for what is legal. + +Parsers are pure (no per-request state), so a shared adapter instance is safe. +`parseResponse` returns positions into the `docs` array it was given; id mapping +and bounds checks live in one place (`rerankDocuments`), not in each adapter. + +## Index mapping + +Every built-in protocol addresses documents by their position in the *request* +array. TEI's reply is explicitly unordered. Results are therefore mapped through +`docs[index]`. An out-of-range index raises rather than attaching a score to the +wrong document. + +`RerankDoc.id` exists because the wire formats only speak in array offsets. +Translating back to a stable identifier is this package's job, not the +caller's. + +## Failure modes + +| Condition | Behavior | +| --------- | -------- | +| Empty `docs` | Return `[]`; no request. | +| Invalid config | Throw a validation error; no request. | +| Unknown `apiStyle` | Throw naming the known styles. | +| Adapter-required `model` missing (Cohere, Voyage) | Throw before the request. TEI serves whatever model it was started with. | +| Reply index out of range | Throw; do not invent a document. | +| Transport, HTTP error, or 200 with a non-JSON body | Raise a request error carrying the classified reason and URL. Never swallowed. | + +Retry and classification are the same policy a chat call uses: retryables back +off; credential failures abort. A reranker outage is visible so the call site +can fall back to its fused ordering. + +## Boundaries + +- **This package owns:** style adapters, index→id mapping, config validation, + one-shot JSON transport wrapping the harness classifiers. +- **The harness owns:** `fetch`, scheduling, retry policy, error taxonomy. +- **The caller owns:** candidate selection, document ids, fallback when ranking + fails, which `apiStyle` and endpoint to use. +- **Not this package:** embedding generation, retrieval fusion, OpenAI-shaped + rerank (there isn't one). diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md new file mode 100644 index 0000000..9b85456 --- /dev/null +++ b/IMPLEMENTATION.md @@ -0,0 +1,148 @@ +# @corbits/reranking — Implementation + +Package: `@corbits/reranking` `0.1.0`. License: LGPL-2.1-only. + +## Runtime + +- **Bun >= 1.2.0** — development runtime (`bun test`, `bun run build`). +- **Node >= 24** — consumes built `dist/`. Native Node does not load this + package's TypeScript source. +- **TypeScript 5.9.3**, ES modules. Default export is `dist/index.js` with + types from `dist/index.d.ts`. The `intx-src` export condition points at + `src/index.ts` for source-linked Interchange workspaces. + +## Dependencies + +| Package | Version | Role | +| ------- | ------- | ---- | +| `@intx/inference` | ^0.3.0 | `createDefaultScheduler`, `BuiltRequest`, classifiers, `createDefaultRetryPolicy` | +| `@intx/types` | ^0.3.0 | `RetryPolicy`, `InferenceError` | +| `@intx/log` | ^0.3.0 | Harness logging peer | +| `arktype` | ^2.1.29 | `RerankConfigSchema` and per-style response shapes | + +Dev: `@intx/inference-testing` 0.3.0, `@types/bun` 1.3.9, `prettier` 3.6.2. + +## Public surface (`src/index.ts`) + +`rerankDocuments`, `RerankConfigSchema`, `createRerankAdapterRegistry`, +`rerankAdapterRegistry`, `rerankAdapters`, `runJSONRequest`, +`extractRetryAfterMs`, `ModelRequestError`, and the associated types +(`RerankDoc`, `RerankResult`, `RerankAdapter`, `RerankAPIStyle`, …). + +## Quickstart + +Matches the README. Registry install, then `rerankDocuments`: + +```bash +npm add @corbits/reranking +pnpm add @corbits/reranking +yarn add @corbits/reranking +bun add @corbits/reranking +``` + +```ts +import { createDefaultScheduler } from "@intx/inference"; +import { rerankDocuments } from "@corbits/reranking"; + +const deps = { fetch, scheduler: createDefaultScheduler() }; +const ranked = await rerankDocuments( + "how do I deploy to staging?", + [ + { id: "doc-1", text: "Staging deploys run from main…" }, + { id: "doc-2", text: "On-call rotation is weekly…" }, + ], + { baseURL: "http://localhost:8085", apiStyle: "tei" }, + { deps }, +); +``` + +```ts +import { createDefaultScheduler } from "@intx/inference"; +import { rerankDocuments } from "@corbits/reranking"; + +const deps = { fetch, scheduler: createDefaultScheduler() }; + +const ranked = await rerankDocuments( + "how do I deploy to staging?", + [ + { id: "doc-1", text: "Staging deploys run from main." }, + { id: "doc-2", text: "On-call rotation is weekly." }, + { id: "doc-3", text: "The billing export lives in finance." }, + ], + { + baseURL: "https://api.cohere.com", + apiStyle: "cohere", // also Jina's /rerank; TEI: "tei"; Voyage: "voyage" + model: "rerank-v3.5", + apiKey: process.env.COHERE_API_KEY, + }, + { deps }, +); + +for (const { id, score } of ranked) { + console.log(id, score); +} +``` + +`deps` is `{ fetch, scheduler }`. A full harness `Dependencies` object also +satisfies this. + +## Config (`RerankConfigSchema`) + +| Field | Rule | +| ----- | ---- | +| `baseURL` | string (provider root, no path suffix beyond what the adapter appends) | +| `apiStyle` | string key into the registry (`tei`, `cohere`, `voyage`, or a custom name) | +| `model` | optional; required at request-build time for Cohere and Voyage | +| `apiKey` | optional; sent as `Authorization: Bearer …` when set | +| `timeoutMs` | optional, `number > 0`; default per-attempt ceiling is 30s | + +`RerankOptions`: required `deps`; optional `retryPolicy`, `registry`, `signal`. + +## Built-in wire formats + +| `apiStyle` | endpoint | request | response | +| ---------- | -------- | ------- | -------- | +| `tei` | `/rerank` | `{query, texts}` | `[{index, score}]` | +| `cohere` | `/v2/rerank` | `{query, documents}` plus `model` | `{results: [{index, relevance_score}]}` | +| `voyage` | `/v1/rerank` | `{query, documents}` plus `model` | `{data: [{index, relevance_score}]}` | + +Jina's `/rerank` uses Cohere's shape and is served by the `cohere` adapter. + +## Errors + +Every transport, HTTP-status, or 200-non-JSON failure raises +`ModelRequestError` (`error.name === "ModelRequestError"`), with classified +`reason` (`InferenceError`) and `url`. + +`@corbits/embedding` and `@corbits/reranking` each carry their own copy of this +class until the shared transport is upstreamed in `@intx/inference`. `instanceof` +does not hold across the two — catch on `error.name`. + +`runJSONRequest` in `src/request.ts` is duplicated (modulo comments) with the +embedding package. Fix both copies or neither. + +## Layout + +``` +src/ + index.ts barrel + rerank.ts rerankDocuments + RerankConfigSchema + adapters.ts TEI / Cohere / Voyage + createRerankAdapterRegistry + request.ts runJSONRequest, ModelRequestError, extractRetryAfterMs + rerank.test.ts bun:test +``` + +## Development + +```bash +git clone https://github.com/corbitsdev/corbits-reranking.git +cd corbits-reranking +bun install +bun run build # tsc -p tsconfig.build.json +bun run test # bun test ./src +bun run typecheck # tsc --noEmit +``` + +Publish is `bun run build && npm publish` (`publishConfig.access: public`). +The npm tarball includes `dist`, `README.md`, and `LICENSE` — not these P/A/I +docs. diff --git a/PRODUCT.md b/PRODUCT.md new file mode 100644 index 0000000..34d4bf1 --- /dev/null +++ b/PRODUCT.md @@ -0,0 +1,46 @@ +# @corbits/reranking — Product + +## What it is + +A cross-encoder rerank client published as `@corbits/reranking`. Given a query +and a candidate set, callers get `{id, score}` rows sorted descending — better +answers from the same retrieval, without rewriting the pipeline per provider. + +The public entry is `rerankDocuments`. + +## Why it exists + +Retrieval already produced a candidate list. Reranking is the step that scores +those candidates against the query so the best evidence surfaces first. + +Unlike embeddings, reranking has no OpenAI-compatible standard — TEI does not +serve an OpenAI-shaped rerank route at all — so teams otherwise fork a client +per vendor. This package is that adapter boundary: one call, caller-owned +document ids, provider chosen at config time. + +## Who it's for + +Authors of retrieval, RAG, and search pipelines who already have documents with +stable ids and need a reranker to order them. Not an embedding client, not a +ranker that invents ids, not a policy layer that decides what to do when the +reranker is down. + +## What users can do + +- Install from the registry (`npm` / `pnpm` / `yarn` / `bun` add + `@corbits/reranking`) and call `rerankDocuments` with a query, `{id, text}` + documents, a config (`baseURL`, `apiStyle`, optional `model` / `apiKey`), and + harness `deps`. +- Rank against a local TEI server or a hosted Cohere, Jina (`cohere` style), or + Voyage reranker without changing the call shape. +- Keep their own document ids: the result is `{id, score}`, not array offsets. +- Pass an empty candidate list and get an empty result with no network request. +- Decide at the call site how to degrade when the reranker fails. This package + does not swallow errors; a reranker outage is a policy decision for the + pipeline (typically: keep the fused retrieval order and continue). + +## Out of scope + +- Embedding generation (`@corbits/embedding`). +- Choosing or fusing retrieval results. +- Hiding transport or protocol failures behind a silent fallback.