Skip to content

Support batch vector query API and shared flat KNN scan #6821

Closed

Description

@BubbleCal

Parent: #6820
Linear: https://linear.app/lancedb/issue/OSS-923/support-batch-vector-query-api-and-shared-flat-knn-scan
Linear parent: https://linear.app/lancedb/issue/OSS-922/vector-search-performance-optimization-roadmap
Related historical issue: #876

Goal

Allow one vector search request to contain multiple query vectors and make the first implementation immediately useful for flat KNN by sharing scan/load work across the query batch.

Scope

  • Add or finalize the public API shape for batch query vectors in Rust and Python.
  • Return results with an explicit per-query discriminator, for example query_index, so callers can split top-k results by query vector.
  • Apply limit/top-k per query, not globally across all query vectors.
  • For flat KNN, load each data batch once and evaluate all query vectors against that loaded batch.
  • It is acceptable for the first version to keep ANN/indexed search out of scope, as long as the behavior is explicit and covered by tests.

Acceptance criteria

  • Rust tests compare batch query results against repeated single-query results.
  • Python tests cover 2-D query input and validate output grouping by query id.
  • Docs/docstrings explain the batch input shape and result contract.
  • A small flat KNN benchmark demonstrates lower tail latency versus issuing repeated independent queries, especially when I/O or decode cost dominates.

Activity

  1. LeoReeYang commented on May 18, 2026

    @LeoReeYang
    Contributor

    hey, I would like to take this one

  2. added a commit that references this issue on May 26, 2026
    98390e1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

A-indexVector index, linalg, tokenizerA-pythonPython bindingsenhancementNew feature or requestperformancerustRust related tasks

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions