Repository navigation
Tracking: data sources beyond the database category (observability, control plane, messaging, vector, lakehouse) #1089
Description
Tracking epic for data systems that sit outside the classic database-IDE category: observability stores, cloud-native control planes, messaging systems, vector databases and lakehouse catalogs.
It is a sibling of #424, not a replacement: #424 stays the coverage map for database providers, and this issue covers the systems a database IDE usually does not reach.
Why this list exists
#424 records the positioning this builds on: we cannot win on driver count against the JDBC-era tools, and our asymmetry is the HTTP-native, cloud-native and AI side.
The first two systems in this category are already under way:
- Prometheus (PromQL over the HTTP API) - design approved, Prometheus provider: PromQL over the HTTP API, with VictoriaMetrics as a relative #1085.
- Apache Kafka (read-only browsing of topics, partitions, consumer groups and messages) - design in progress; the Node client is decided by a measurement, not by documentation. design approved, Kafka provider: read-only browsing over the Kafka protocol, with Redpanda as a relative #1088
Each later candidate reuses what those two establish: the flat object tree, JSON object sources, the status field for an object worth acting on, the third queryLanguage mechanism, and the rule that a new provider touches no other provider (#1085, sections 3.2 and 3.5).
Rules
- Definition of Done. A candidate is added when it clears the seven gates of Tracking: database provider coverage - shipped, queued, and declined #424's "Definition of Done - per provider", unchanged.
This issue does not restate them. - Claim discipline. "No database editor supports X" is a hypothesis until it is measured.
Before any candidate's absence in competitors is used in published copy, it is checked against DBeaver (Community and PRO), DataGrip, CloudBeaver, Beekeeper Studio and TablePlus, with the source recorded here. - One tracking surface per name. A system already tracked in Tracking: database provider coverage - shipped, queued, and declined #424 stays there and is linked from here, never tracked twice.
A system that speaks the wire protocol of a driver we ship is a relative of that provider (Tracking: database provider coverage - shipped, queued, and declined #424 Phase 0), not a new type-id. - Architecture. Each new type-id follows the isolation and SOLID rules of Prometheus provider: PromQL over the HTTP API, with VictoriaMetrics as a relative #1085 section 3.5, and its security requirements are written before its code, as in Prometheus provider: PromQL over the HTTP API, with VictoriaMetrics as a relative #1085 section 3.6.
- Scoring. Before a candidate is scheduled it is scored on Tracking: database provider coverage - shipped, queued, and declined #424's rubric (Demand, Transport, Packaging, Differentiation), from measurements: competitor support, demand signals, client health (latest release, native addon or not, Bun compatibility), and whether a single container can serve the gate-4 probe.
Candidates
Observability, alongside Prometheus
- Grafana Loki (LogQL). Logs over an HTTP API with no runtime dependency. Its label and stream model is close to Prometheus's, so "metrics and logs in one tool" is one story and one technical pattern.
- VictoriaLogs (LogsQL) and Quickwit. Log and search engines over HTTP.
- InfluxDB 3 (SQL and InfluxQL over HTTP) is tracked in Tracking: database provider coverage - shipped, queued, and declined #424, "Open questions".
Cloud-native control plane
- etcd. The database behind Kubernetes: browsing by key prefix through the JSON gateway or a pure-JavaScript client. It also holds Kubernetes secrets, so its security design comes first: read-only from the start and bound to the product's RBAC.
- Consul KV. Service catalog and key-value store over HTTP.
- Steampipe. Presents cloud and SaaS APIs (AWS, GitHub, Kubernetes and others) as tables over the PostgreSQL wire protocol, so it is a candidate relative of the
postgresprovider: no driver work, only a probe. Once probed it is recorded in Tracking: database provider coverage - shipped, queued, and declined #424 Phase 0.
Messaging, alongside Kafka
- NATS JetStream. A CNCF project with streams, a key-value store and an object store, and a pure-JavaScript client; the key-value side browses like a database.
- RabbitMQ and Apache Pulsar. Queues and topics can be inspected through their management REST APIs, but both fit the object model less well.
- Redpanda speaks the Kafka wire protocol and is a relative of the Kafka provider, recorded with it.
AI
- Vector databases (Qdrant, Milvus, Weaviate, Chroma) are tracked in Tracking: database provider coverage - shipped, queued, and declined #424, "Open questions". Qdrant is the first to measure there: a single container and the cleanest HTTP API of the four.
Lakehouse
- Apache Iceberg REST Catalog. Browsing tables, snapshots and schema history without an engine. Querying is already served by DuckDB and Trino, so this provider's role is a catalog browser.
Multi-model and edge cases
- SurrealDB. SurrealQL over HTTP and WebSocket, with a fast-growing community.
- Google Cloud Firestore. A managed service with high demand; like Tracking: database provider coverage - shipped, queued, and declined #424 Phase 3, gate 4 needs a real account.
- TigerBeetle and FoundationDB. Genuine outliers with no query language; weak candidates for now.
Proposed order after Kafka and Prometheus
Not a schedule: each step is confirmed by its score first.
- Loki: the Prometheus story and the Prometheus pattern.
- Steampipe: a label, with no driver work.
- Qdrant (in Tracking: database provider coverage - shipped, queued, and declined #424): the AI positioning.
- NATS JetStream: the Kafka messaging story.
- etcd: once its security design is done.
First task
- A measured comparison of every unticked candidate above, scored on Tracking: database provider coverage - shipped, queued, and declined #424's rubric, recorded as a comment here with a source per fact.
Activity
DevvoLazza commentedon Sep 24, 2026
Hi! I’d like to take this
DevvoLazza commentedon Sep 24, 2026
Research snapshot (2026-09-24 UTC) for the first task in #1089. This compares all 15 names in unticked candidate checkboxes, including the two starting points. Scores use #424's 1–5 rubric in order Demand / Transport / Packaging / Differentiation; higher is better. They are prioritisation estimates, not a claim that a provider has passed #424's seven gates.
How I scored. Demand is a deliberately narrow, reproducible LibreDB request signal: 5 = a dedicated provider issue/spec (#1085 or #1088), 3 = explicitly in #1089's proposed next order, 2 = named in this epic only, 1 = explicitly described here as a weak outlier. A title search of this repository's issues found dedicated titles for Prometheus and Kafka only. This does not measure wider customer demand; collect user requests before scheduling any of the remaining names. Transport is implementation cost (5 = documented HTTP endpoint with no extra runtime client). Packaging is runtime native-dependency risk (5 = direct HTTP, no client package). Differentiation is an estimate from the advertised native support audit below, not a published “nobody supports X” claim. Rubric and gates; candidate list and proposed order.
Competitor audit (advertised native support, checked 2026-09-24). Sources checked: DBeaver PRO vs Community table and PRO database list, DataGrip supported data sources plus its Kafka plugin page, CloudBeaver database catalog, Beekeeper Studio edition matrix, and TablePlus supported list. Positive matches above are explicit. Other names were not listed as native connectors in those pages; that is not evidence that custom JDBC/ODBC drivers, plugins, or other editions cannot connect. DBeaver/CloudBeaver's “Kafka (ksqlDB)” is not proof of raw Kafka protocol browsing.
Boundary for this pass. These are documentation and issue measurements, not live gate-4 probes; the exact server version, captured payloads, permission failure, cancellation, and Bun smoke tests remain open for each unshipped name. Keep Steampipe and the vector candidates in #424, and Redpanda with Kafka, rather than creating duplicate type-id tracking here. A single container is documented for the rows that say so, not yet proven against all seven gates.
cevheri commentedon Sep 24, 2026
Hi Lazzo, thank for offering to take this, but I have opened it as a taks and assigned it to myself. I will close it in the coming days. could you take some other unassigned issues?
cevheri commentedon Sep 24, 2026
this is a greate list
my short plan is:
cevheri commentedon Sep 24, 2026
also db2 is waiting, I need to generate seperated docker image :( for our arm support
cevheri commentedon Sep 24, 2026
firestore also planned
cevheri commentedon Oct 27, 2026
Closing this epic: Prometheus, Kafka, etcd, InfluxDB and Firestore from the plan above have shipped, along with RabbitMQ and the vector stores from #424. Loki and Pulsar are the two planned names left; each gets its own design issue, in the #1085 shape, when it is scheduled. The scored comparison above stays as the reference for the other candidates.