[Dariusch]

Description

Adds Google BigQuery as an engine (bigquery), with no Google SDK: every statement is a REST jobs.query call over the shared node transport, and jose (already a dependency) signs service-account tokens.
It covers the editor (results, schema tree, autocomplete, a live cost estimate), monitoring and seeding. Agent and MCP run_read_query execution, and clause-aware completion, follow in separate PRs.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Code refactoring
• Performance improvement
• Test addition or update

Related Issue

Part of #424.

Changes Made

• Provider (src/lib/db/providers/sql/bigquery/): connect, query with paging and polling, typed cell decoding (INT64 as string, TIMESTAMP, STRUCT, ARRAY), error mapping, best-effort jobs.cancel on abort, and a datasets → tables / views / materialized views object surface.
• Credentials: a pasted service-account key or access token; with BIGQUERY_SERVER_CREDENTIALS=true and only for operator-seeded connections, the server's ADC file or the metadata server (GKE Workload Identity). Tokens are cached and single-flighted.
• Job location: the connection's location field, else the default dataset's location, else the most common dataset location, so table-less statements do not fall back to US.
• Cost estimate: EXPLAIN is a free dry run reporting bytes processed, and the editor shows a live "This query will process X when run" badge.
• Schema and autocomplete: two region-level INFORMATION_SCHEMA reads per location, cached, with a REST fallback; on a 3,000-table project this took the schema load from about 78 s to under 10 s.
• Monitoring: overview, performance, slow queries, active jobs, table stats and storage from INFORMATION_SCHEMA, cached for 60 s.
• SQL grammar for BigQuery, every fact measured with dry runs: # comments, backslash escapes (including in backtick names), triple-quoted strings, flat block comments, […] subscripts.
• Shared code: a tablesNeedContainer capability and a labelQualified table-item flag (completion for other engines is unchanged), a showSslPanel UI flag, triple-quoted strings and backslash-escaped backticks in the span reader (off for every other grammar), and the server forcing managed: false on inline connections.
• Seeding and form: seed type: bigquery with database (project), schema (default dataset), password (key JSON or token) and location; the form hides host, port, SSL and SSH.
• Docs: new docs/providers/bigquery.md; BigQuery added to docs/SEED_CONNECTIONS.md, docs/API_DOCS.md, the READMEs and listings (thirty external engines); BIGQUERY_SERVER_CREDENTIALS, GOOGLE_APPLICATION_CREDENTIALS and GCE_METADATA_HOST in .env.example.
• Chart: bumped to 0.1.86 because an upstream test pins the chart description to the engine count; metadata only, no template or value changes.

Testing

• I have tested this locally
• I have added/updated tests
• All existing tests pass

Live, against a real BigQuery project (data in europe-west3): connect, queries, schema tree and autocomplete, the dry-run estimate, monitoring, and every grammar fact via dry runs.

Automated: new unit tests for the provider, transport, auth and tokens, decoding, the grammar and qualified completion labels, plus tests/integration/db/bigquery-provider.test.ts, which runs the real provider, transport and auth against a fake REST server, including the shared object-surface conformance check.

bun run test passes apart from tests/unit/db/etcd/tls-handshake.test.ts, which also fails on main.
Format, lint, typecheck, knip, chart:check, channels:showcase:check, readme:check, security:check, build, build:lib, attw, lib:closure and the Windows launcher's gofmt/go vet/go test pass.

Test Environment

• LibreDB Studio Version: 0.18.3 (+ this branch)
• Browser: Google Chrome
• OS: Linux (Debian 13)
• Node.js/Bun Version: Bun 1.4.2
• Database Type: Google BigQuery

Screenshots (if applicable)

Checklist

• My code follows the project's code style guidelines
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have updated the documentation accordingly
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• The required CI test job passes the 100% line-coverage gate (bun run test:coverage and bun run coverage:check)
• If I changed src/lib/db/providers/, I updated the matching docs/providers/ documentation and tests/integration/db/ tests in the same PR (provider triad)
• Any dependent changes have been merged and published

Additional Notes

• No live BigQuery in CI: BigQuery has no local emulator that implements jobs.query and INFORMATION_SCHEMA faithfully, so the integration test uses a fake REST server and live behaviour was verified by hand.
• Known limits, documented: an abort while the first jobs.query POST is still in flight cannot cancel the job; the shared inventory limit (1,000 container/kind pairs) truncates autocomplete above about 333 datasets.
• Reviewing: the last commit applies the first review: the MCP fence, the completion rewrite and agent/MCP execution are reverted to main, so review the full diff rather than commit by commit.
• Chart bump: the chart version bump to 0.1.86 may be something maintainers prefer to own at release; happy to drop or adjust it.

[cevheri]

Thanks, this is a solid provider: the triad is complete, there is no new dependency, and nothing reverts main. I'm requesting changes because three pieces change behaviour beyond BigQuery, plus a few small fixes. A new provider should not change how any other engine behaves.

1. Keep the MCP statement fence on main's reading for every engine.
   Passing connection.type into checkReadOnlyStatement changes the verdict on PostgreSQL, DuckDB and SQL Server too: SELECT /* /* */ 1; DROP TABLE t; */ was MULTIPLE_STATEMENTS on main and is now admitted there, and SELECT [a--b] FROM t flips the other way. No test pins those engines' verdicts.
   Revert src/lib/mcp/guards/execution-fence.ts and the call in run-read-query.ts to main's form. BigQuery's own queryReadOnly already applies its grammar.
   Done when: checkReadOnlyStatement matches main.

2. Move the clause-aware completion rewrite into its own PR.
   The sql-completions.ts change alters completion for every engine, for example tables are no longer offered in a SELECT list. That deserves its own review.
   Keep only what BigQuery needs here (tablesNeedContainer and the qualified label), the way Databend and S3 added their completion facts without touching the shared flow.
   Done when: completion for a non-BigQuery connection behaves exactly as on main.

3. Split agent and MCP execution on BigQuery into a follow-up PR.
   On the engines that execute today, the boundary includes the identity: PostgreSQL refuses broad roles and SQL Server checks at open that the principal cannot write. Here a key that can write is accepted, and the grammar comment notes remote functions are not covered.
   Land the editor provider first. In the follow-up we can talk about an identity check (for example a testIamPermissions probe that refuses write permissions).
   Done when: AGENT_EXECUTION_ENGINES, the MCP engine list and their tests are as on main.

4. Document the server-credential variables in .env.example.
   BIGQUERY_SERVER_CREDENTIALS, GOOGLE_APPLICATION_CREDENTIALS and GCE_METADATA_HOST are read in auth.ts but are not in .env.example, where every env var lives.
   Done when: all three are there with an example value and a one-line explanation.

5. Fix two small text slips.
   In src/lib/sql/grammar.ts, BIGQUERY_GRAMMAR sits between Databend's docblock and DATABEND_GRAMMAR, so that docblock now belongs to the wrong constant. DOCKERHUB.md says "The thirty-one row", which should be "thirty-first".
   Done when: each grammar constant directly follows its own docblock and the ordinal is fixed.

Please also replace Closes #<issue number> in the description with a reference to #424. I'll approve the CI run once these are in.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[cevheri]

One more ask, on top of the review above. A new provider is a big change, so next time please open an issue first so we can agree on the scope before the code. For this one, the integration test mocks the HTTP layer, so nothing here has run against an engine. docs/providers/bigquery.md says no emulator covers jobs.query and INFORMATION_SCHEMA, but goccy/bigquery-emulator implements jobs.query, getQueryResults with paging, dataset and table listing, and the SCHEMATA, TABLES and COLUMNS views. That covers connect, the schema tree, a query and pagination. Dry run and bytes processed are not covered, so those stay mocked.

6. Add a live check against the emulator.
   Read a server-side BIGQUERY_EMULATOR_HOST (env only, never a connection field) in transport.ts, then add tests/live/bigquery-live-check.ts in the shape of tests/live/databend-live-check.ts, run against ghcr.io/goccy/bigquery-emulator.
   Done when: the script connects, lists the datasets and tables, reads columns, runs a query across two pages and exits non-zero on any failure, and the doc line about the emulator says what it covers and what it does not.

[Dariusch]

Yeah, I will create an issue for sure next time. For us, this initially started as a PoC, but I made quick progress and did not want to keep it from the project.

We did not have a good experience with https://github.com/goccy/bigquery-emulator for our CI pipelines. This is likely due to our setup; I will look into adding it to this project/PR.

[cevheri]

how was your contribution experience with libredb-studio?

[cevheri]

thanks Dariusch this is a big PR, and last commit fixed the live check, merging shortly and will see next release

[Dariusch]

Thank you.
Tomorrow, I will reply to the contribution ask and create the follow-up Pull Requests. It is getting late here in Germany

[cevheri]

Thanks @Dariusch, BigQuery is merged. Two follow-ups, in one PR when you have time:

1. Send no Google credential to the emulator. apiOrigin (src/lib/db/providers/sql/bigquery/transport.ts:142) reads BIGQUERY_EMULATOR_HOST in the server. When it is set, every API request carries the connection's real access token to that host, over plain HTTP for a bare host:port, so a variable left set on a server sends live tokens in clear text. With the variable set, skip the token source and send no authorization header; the emulator checks none.
   Done when: a case in tests/unit/db/bigquery/transport.test.ts with the variable and a key JSON shows no request to oauth2.googleapis.com and no authorization header on the API request, and tests/live/bigquery-live-check.ts still passes.

2. Replay recorded answers in the integration test. Its header says every answer is built from the documented REST shapes, while gate 4 in Tracking: database provider coverage - shipped, queued, and declined #424 asks for answers captured from a live instance, with the date in the test header. You have a real project, so record them there with scrubbed captures (the pattern is tests/live/databend-evidence.ts and tests/fixtures/databend/README.md) and replay them.
   Done when: every fake answer in tests/integration/db/bigquery-provider.test.ts comes from a committed capture, and its header names the capture date and region.