Skip to content

feat(neo4j): read-only Neo4j 5.26 provider on a shared graph layer - #1239

Merged
cevheri merged 74 commits into
mainfrom
feat/neo4j-provider
Oct 3, 2026
Merged

cevheri merged 74 commits into
mainfrom
feat/neo4j-provider

Conversation

@cevheri

@cevheri cevheri commented Oct 3, 2026 •

Copy link
Copy Markdown
Member

Adds Neo4j 5.26 LTS as a read-only engine, on a new shared graph layer that later graph engines join through an engine profile.

What it does:

  • Connects over the bolt scheme family only (bolt, bolt+s, bolt+ssc), with the SSH tunnel; no routing neo4j:// scheme, no Aura.
  • Browses one database per connection: node labels and relationship types with their properties as columns, indexes and constraints.
  • Runs read-only Cypher from the editor; nodes, relationships and paths arrive as tagged JSON cells, with the graph type in the column header.
  • Refuses every write before it is sent, in three layers: a token-level read policy (procedures, functions and SHOW forms are allowlists; LOAD CSV, APOC, GDS and parameters are refused), a server EXPLAIN classification gate, and a READ session.
  • Ships its own Cypher editor language and schema completion, plan mode, and the MCP metadata tools. Agent execution and MCP run_read_query do not serve Neo4j.

Layout: src/lib/db/graph/ (pure core: lexer, read policy, quoting, generators, object model), src/lib/db/graph/bolt/ (the only place that imports neo4j-driver-lite 6.2.0, pinned exactly; no telemetry, no logging, no retries, no managed transactions), GraphBaseProvider, and src/lib/db/providers/graph/neo4j/. No other provider directory is touched.

Release note: DatabaseType gains neo4j, QueryTab["type"] gains cypher, and ProviderCapabilities.queryLanguage gains cypher; consumers with exhaustive switches over these need an arm.

Known limits, each in docs/providers/neo4j.md or the backlog: no graph view; one database per connection; verified TLS through an SSH tunnel is refused; monitoring lists are bounded at the shared query limit; 2025.x and 2026.x servers connect untested.

Verification: every local gate, including build, build:lib, attw and 100% line coverage (83598/83598). Two external design reviews ran before code (Codex, Cursor, OpenCode, Antigravity) and an adversarial review of the whole diff after it. A live pass in a real browser against Neo4j 5.26.31 with the seed graph covered the dialog, the tree, Cypher tabs and completion, the value forms and export, every refusal class (node and relationship counts unchanged), cancel, monitoring and plan mode; a regression pass on SQLite, PostgreSQL and SQL Server (queries, Format, the safety dialog, agent plan and auto mode) found no change. Local reviews by OpenCode and Antigravity followed; Cursor and Kiro were over their usage limits. Confirmed findings are fixed with tests, among them two read-policy bypasses measured on the live server: a carriage return ending a line comment, and the \u0060 escape the server decodes before lexing, which let a non-allowlisted procedure run.

cevheri added 30 commits October 3, 2026 07:26
… inputs as escapes

A backticked parameter that spans lines carries over as the backtick state, and digit-group underscores in numbers are not read; both are now documented where they live. The test's astral-plane inputs are written as \u{1F600} escapes.
… or SHOW as a key by the read policy

A scoped CALL's scope is now walked by the same rules instead of skipped, so a denied word,
a parameter or a namespace there is refused. One backticked name holding dots and called is
read as a qualified function name, so the namespace and function allowlists apply to it.
CALL and SHOW after a dot or before a colon are property or map keys, not clauses.
…__proto__ column as a key

A session the driver refuses to open is now a classified GraphClientError.
A close failure after a complete result is classified; after a run error or a
cancellation the first error stands. Rows and converted maps write keys as own
properties, so a column or key named __proto__ keeps its cell. UUID, Vector and
unbound relationships are recognised by the driver's predicate or class, and the
depth bound is pinned at its boundary for points, byte arrays, nodes,
relationships and paths.
…s client on reconnect

Two runs sent under one queryId are both reached by cancelQuery, and a connect on a
connected provider closes the old client and aborts its statements first, so no driver
leaks and a failed reconnect leaves no stale session. The getHealth docblock records why
there is no unhealthy form, and the default-factory test is named for what it checks.
…plicate captures

The evidence harness no longer imports neo4j-driver-lite: the cancel is
measured as the query route sends it, an abort of createBoltClient's run,
with the time until the transaction leaves SHOW TRANSACTIONS.
recordedGraphClient refuses two captures of one database and statement,
and the duplicate explain-show capture is gone. The TLS row of the error
test pins that the only TLS capture comes back as category connection.
Fixtures regenerated from Neo4j 5.26.31.
A missing or non-string owningConstraint is now a wrong-shape QueryError
in both readers; indexRows no longer reads a missing field as unique.
…time, pin the backtick advice

A refused label, relationship type or index listing no longer throws the whole overview: the label count becomes a floor of 0 and an unreadable type or index count is said in the size text. The table stats run their label counts one after another, so a refresh holds one session. The integration test pins the backtick advice for a denied word as a property or map key, and the backticked form running. Docblocks say why no database count is read and why lastRead stays unread.
neo4j joins DatabaseType, and every exhaustive record, census and registry answers for it.

Record rows:
- SHIPPED.neo4j: true; EXTERNAL.neo4j: true.
- READ_ONLY_ENFORCED.neo4j: true (v1 refuses every write whatever readOnly says).
- MCP_EXPOSABLE.neo4j: true (list_connections and inspect_schema; no queryReadOnly, so no run_read_query).
- WIRE_COMPATIBLE_ENGINES: no row.
- factory: case "neo4j" imports ./providers/graph/neo4j/index; the Supported types string names neo4j.
- DB_UI_CONFIG.neo4j: label "Neo4j", icon Neo4jIcon, color text-hue-fuchsia-alt, defaultPort "7687", showConnectionStringToggle false, connectionFields host, port, user, password, database, SSL panel and SSH tunnel offered, fieldHints on user (the read-only sentence) and database ("Leave empty to use the server's home database.").
- Neo4jIcon: a generic graph glyph, three circles joined by three lines, 24x24, strokeWidth 1.5.
- Colour: fuchsia joins IDENTITY_ALTS, with --studio-hue-fuchsia-alt (fuchsia-900 light, fuchsia-300 dark).
- SHOWCASE_RANK.neo4j: 11, after cassandra; couchbase to libredb move down one (12 to 20).
- ENGINE_FENCE_TAGS.neo4j: true; QUERY_FENCE_ALIASES gains cypher, ALIAS_ENGINES does not (the cql rule).
- LITERAL_ESCAPE.neo4j: "backslash" (Cypher escapes with a backslash and has no doubled quote).
- positionalPlaceholder("neo4j"): null.
- STANDS_ALONE.neo4j: NOTHING_STANDS_ALONE; BINARY_LITERAL.neo4j: "standard-hex".
- NON_SQL_DIALECTS gains neo4j.
- NON_SQL_DESTRUCTIVE_VOCABULARY.neo4j: empty operation set, a reader answering [], decidesAlone true.
- NO_COLUMN_MODIFICATION.neo4j: "Neo4j", "A node label groups nodes whose properties are not declared columns, so there is no column definition to change."
- NO_TRANSACTION_WRAPPER gains neo4j: its text is a Cypher statement, not SQL.
- NO_TABLE_DDL gains neo4j: a label is not a table with declared columns.
- Seed type enum gains "neo4j"; selectableTypes gains "neo4j"; CENSUS_CONNECTION.neo4j: unconnected("neo4j").

The provider no longer declares containerPathShapes: absent reads as exact, and the
container-path census requires a declaring provider to call assertContainerPathShape, which
GraphBaseProvider does not (it refuses any other container itself, as etcd and Kafka do).

Census moves, each from EXTERNAL_DATABASE_TYPES.length (20) plus the embedded store:
census fleets 20 to 21, edit abstainers 16 to 17, source census kinds 95 to 99 and 103 to 107
(neo4j adds four kinds, none source-bearing), source abstainers gain neo4j.
… Cypher cases through the corpus

A SHOW completion replaced every word after SHOW, so its range held a space and could span lines.
It now replaces only the word being typed and inserts and filters on the words still to come.
The multi-line comment, string and backticked parameter cases move into CYPHER_CORPUS, and the Cypher profile refusal test moves off the #1085 docblock it did not match.
neo4j-driver-lite joins the library build's external list with the other
drivers, so dist/neo4j-*.mjs imports it and dist/neo4j-*.js requires it
rather than carrying a copy.

The Next.js build bundles it, and that works: it stays out of
serverExternalPackages. Measured 2026-10-03 on Next.js 16.3.6 (Turbopack):
bun run build put neo4j-driver-core and neo4j-driver-bolt-connection into
two server chunks; next start, as playwright.config.ts runs it, against
the compose neo4j service (5.26.31, seeded) answered
POST /api/db/query with MATCH (n:Person) RETURN n LIMIT 1 as HTTP 200,
one Node row ("Ayse Yilmaz") with columnTypes {"n":"Node"}.
…knip passes

Both were exported but only used inside their own file, which failed knip in the required check.
The tsup comment now says why the Bolt driver is external rather than how it is pinned.
docs/providers/neo4j.md is the provider's prime reference, and tests/unit/db/neo4j/provider-doc.test.ts reads every list, sentence, bound and measurement it quotes back from the code, the captures and the compose file. The repository docs gain the provider and the graph layer (an "Adding a graph engine" section), numerals are re-derived, SECURITY.md records the read policy as control 3.10, and the backlog files D140 to D143, U72 and B93.
cevheri added 14 commits October 3, 2026 13:20
…s active

f6da1f2 maps Running to active and updated monitoring.test.ts, but this read's test still expected Running and failed.
…called or not

The doc said apoc. and gds. are refused "called as a procedure or a function", while the policy checks every qualified name, so `apoc`.name, a property of a variable named apoc, is refused with a message about the network (review N3).
The strictness stays, fail closed; the doc now states that scope, and the refusal adds that a name starting the namespace is refused in any position, called or not, so a variable of that name must be renamed.
A label cannot carry a dot, so the advice names a variable only.
…ck advice

MATCH (n:CALL) RETURN n was refused as "CALL  is not allowed" with an empty procedure, and MATCH (n:SHOW) RETURN n as "SHOW ) is not allowed", the closing parenthesis read into the form (review N4).
A CALL followed by no name, or by a keyword as in WITH n AS call RETURN call, is now refused with the subject CALL; the SHOW subject collects only the words, names and strings typed after it.
Both refusals add the backtick advice the denied-word refusal gives when the word may be a name: CALL with no procedure name after it, SHOW with no form word after it.
The advice sentence is one helper the three refusals share, and section 3.3 quotes both new sentences.
listObjects kept only the entries of the catalog's answer and dropped its truncated flag, so a listing cut at CATALOG_ROW_BOUND reached the tree as if whole, against the catalog's "never cut silently" (review N5).
DatabaseObject[] has no field for a cut, so the cut is now logged through logError, as the table stats log the labels they leave out, and the tree shows the rows read.
The catalog docblock and the bounds row now say what holds: a count reports a floor, a detail is refused, and a tree listing is logged on the server with no marker in the tree.
Two overlapping connect() calls on one provider both saw no session, each built and verified a client, and the second session written orphaned the first client's driver unclosed (review N6).
connect() now memoises the attempt in flight, so an overlapping call awaits it, shares its failure, and the next call after it settles reconnects as before.

The factory's own race, two first requests for one connection each connecting a provider while the later cache write orphans the earlier one, predates this provider but now leaks a Bolt driver with up to four pooled sockets; it is filed as D144.
The antigravity review said a non-version prefix such as CYPHER runtime=slotted breaks the gate because EXPLAIN lands before CYPHER.
Measured on the compose server (neo4j:5.26.31-community) on 2026-10-03 with cypher-shell, that claim is false, and a neighbouring case is real:

- EXPLAIN CYPHER runtime=slotted MATCH (n) RETURN n: parses, Statement READ_ONLY.
- CYPHER runtime=slotted EXPLAIN MATCH (n) RETURN n: parses, Statement READ_ONLY.
- EXPLAIN CYPHER 5 runtime=slotted MATCH (n) RETURN n and CYPHER 5 runtime=slotted EXPLAIN MATCH (n) RETURN n: parse, READ_ONLY.
- CYPHER 5 EXPLAIN runtime=slotted MATCH (n) RETURN n: syntax error, "Invalid input 'runtime'".

The policy allows both CYPHER runtime=slotted and CYPHER 5 runtime=slotted, and the gate sent the second as CYPHER 5 EXPLAIN runtime=slotted ..., so the server refused the check and a valid read was refused.
The gate now keeps the whole prefix, the version and every option=value pair, in front of EXPLAIN, sliced from the statement's text; a prefix without options is sent as before.
…in one step

Windows reports every directory as mode 0o40666 and has no process uid, so the POSIX check refused every CA directory there; the rule is now caDirectoryIsPrivate, which checks only the directory shape on win32, where the user's temp directory ACL is the protection. The tests point TEMP and TMP as well as TMPDIR at their root, because os.tmpdir() reads those on Windows (Cross-platform Tests, windows-latest). The file is written with flag wx and read back only when it exists, which removes the check-then-write race CodeQL js/file-system-race reported.
…son reads them

A bare sort() orders by UTF-16 code unit, so every capitalised label came before every lower-case one, and the first 50 labels of the table stats were the capitalised ones (SonarCloud S2871, five sites). Each now sorts with localeCompare in the en locale, as the object listing does.
…ket no other second-server spec spends

On CI the user's 120 query slots were spent by the earlier second-server specs and their retries, so this spec's Test Connection answered Too many requests in all three attempts. The bucket is keyed on the account; the dialog is the same for either role.
The Cypher tab type renders in graph-cypher with no formatter, read from the
editor registry #1241 introduced instead of the arm editorLanguageForTabType
carried before the merge. Cypher declares a language and no dialect, as PromQL
does, so resolveTabType keeps its language rung and QUERY_DIALECTS has no row.
Generated by tests/fixtures/dialect-registry/golden-dialect-outputs.ts. The only
change is the added neo4j entry; every other type-id answers as before.
# Conflicts:
#	docs/ADDING_A_PROVIDER.md
#	docs/SEED_CONNECTIONS.md
…ader allowlist

#1243 holds negated language reads to the closed list; this PR's diagram gate reads queryLanguage !== "cypher" on purpose, so every other language keeps the diagram it had before the gate.
@cevheri
cevheri merged commit 23f5c85 into main Oct 3, 2026
20 checks passed
@cevheri
cevheri deleted the feat/neo4j-provider branch October 3, 2026 12:09
@sonarqubecloud

sonarqubecloud Bot commented Oct 3, 2026

Copy link
Copy Markdown

cevheri added a commit that referenced this pull request Oct 3, 2026
The census holds one row per shipped type-id, so rebasing onto main with
the Neo4j provider (#1239) made the record miss a member: a type error and
two failing cases. Neo4j offers no per-row maintenance control and declares
no extension of spec 3.11, so its row is the empty one.
cevheri added a commit that referenced this pull request Oct 3, 2026
Brings in the Neo4j provider (#1239), the dialect and dialect editor
registries (#1241, #1243), the node HTTP transport (#1242), host-box
addresses and credential warnings (#1244) and the Oracle UROWID fix.

Adapted to main:
- engine counts re-derived for 21 external engines (22 type-ids) in
  every listing, README, chart, operator CSV and package description,
  naming both Db2 LUW and Neo4j; showcase ranks renumbered
- Db2 needs no registry entry: it declares no queryDialect and opens
  sql tabs; the dialect golden file gains its db2 record
- census counts, fingerprint line references and field-copy tests
  updated for both providers
- Db2 backlog entries D140-D145 renumbered to D145-D150 behind
  Neo4j's D140-D144
- chart stays 0.1.74 (unreleased) with both changes in its metadata;
  operator bundle regenerated
cevheri added a commit that referenced this pull request Oct 3, 2026
The census holds one row per shipped type-id, so rebasing onto main with
the Neo4j provider (#1239) made the record miss a member: a type error and
two failing cases. Neo4j offers no per-row maintenance control and declares
no extension of spec 3.11, so its row is the empty one.
cevheri added a commit that referenced this pull request Oct 3, 2026
…he engine principal (#1246)

* test(maintenance): pin every shipped provider's per-row maintenance controls

* feat(maintenance): declare typed-target, preview and the engine principal on the provider contract

* test(maintenance): hold a typed-target confirmation to a per-row control

* feat(audit): carry the engine principal as engine_user on the audit line

* feat(maintenance): write the engine principal on every maintenance audit row

* feat(maintenance): serve a per-object preview at POST /api/db/maintenance/preview

* feat(monitoring): read a maintenance preview from the monitoring hook

* feat(maintenance): the typed-target and preview dialog for a per-row operation

* feat(operations): offer declared per-row operations with their typed target and preview

* feat(monitoring): offer declared per-row operations on the Tables tab

* feat(object-tree): offer declared per-row operations in the row menu

* feat(schema-explorer): offer declared per-row operations in the mobile row menu

* docs: document typed-target, the maintenance preview and engine_user

* test(maintenance): drop unused mock parameters from the per-row surface tests

* docs(security): re-derive the role-denial sites from the source

The table and its count missed the two account-administration routes,
and the preview route was added on top of that stale base. Seven
handler sites call auditRoleDenial, plus the proxy; a unit test now
derives the set from src/app and holds the table and the numerals to it.

Fixes review findings SC-1 and R1.

* fix(maintenance): answer no preview for an operation no row offers

The preview route served a preview for a spec declaring preview: true
with both placements false, which no surface draws. It now answers the
no-preview 400 unless the per-row placement is offered, and the
capability test holds every shipped spec that declares a preview to
perEntity: true and a provider that implements previewMaintenance.

Fixes review finding B1.

* fix(maintenance): key preview facts so a repeated label draws cleanly

Facts were keyed by label alone, and a preview that names each vector
field under one label logged a duplicate-key warning. Each fact is now
keyed by its label and how many facts before it share that label.

Fixes review finding B2.

* fix(maintenance): answer 400 to a body that is not a JSON object

Both maintenance routes destructured the parsed body directly, so null
answered 500 with the raw TypeError message and text that is not JSON
answered 500 as well. A body that is not a JSON object now answers 400
"Invalid request body" before any provider is opened, on the existing
route and on the preview route that copies its checks.

Fixes review finding B3.

* docs(rate-limit): re-count the query bucket's call sites

The comment said seventeen direct call sites while its own grep already
answered twenty-one on main, and the preview route made it twenty-two.
It now names all twenty-two files, the two that hold two call sites
each, and the thirty-two route files in total.

Fixes review findings B4 and R2.

* test(exports): say that the packed-tarball check is not tracked

The docblock described a tarball type check as if the repository ran
it. It was a one-off pre-merge measurement in a gitignored folder, so
the docblock now says what this file covers and what it does not.

Addresses review findings SC-2 and R3; a tracked tarball check in CI
is deferred.

* docs(maintenance): one sentence per line, and name the failed row

Split two sentences that shared a line, and call the engine_user rows
completed, failed (success: false) and thrown, since "refused" read as
the route's own 400 refusals, which write no maintenance row.

Fixes review findings R4 and R5.

* test(maintenance): spread the real auth module in the preview route test

The new route test copied the five-key auth stub that BACKLOG D85 records as debt, which also missed two of
the module's seven exports. It now spreads the real module and replaces getSession alone, the form D85
prefers, and D85 is re-measured: 40 files mock the module, eight of them by spreading it.

* test(maintenance): pin neo4j's per-row maintenance surface in the census

The census holds one row per shipped type-id, so rebasing onto main with
the Neo4j provider (#1239) made the record miss a member: a type error and
two failing cases. Neo4j offers no per-row maintenance control and declares
no extension of spec 3.11, so its row is the empty one.

* test(maintenance): pin db2 in the census of per-row maintenance controls

The census holds one row per shipped type-id, so rebasing onto main with
the Db2 provider (#1238) made the record miss a member. Db2 offers analyze
and optimize per row, on both tabs and both row menus, and declares no
operation outside MaintenanceType: 11 of 22 type-ids now offer per-row
controls on the tabs and 10 on the row menus.

* fix(maintenance): hold a declared per-row operation to the kinds its spec names

Main gained MaintenanceOperationSpec.kinds with the Db2 provider (#786):
a per-row control is offered only on the object kinds its operation names.
The two row menus pass the row kind for analyze and vacuum, and now pass
it for the declared operations outside MaintenanceType as well, so a
declared operation that names its kinds is not offered on a view. The
Operations and Tables tabs list tables and name no kind, as before.
cevheri added a commit that referenced this pull request Oct 5, 2026
…d three security fixes (#1531)

Carries eight connection types for seven engines: etcd (#1228), IBM Db2 LUW
(#1238), Neo4j with the Graph tab (#1239, #1289), Qdrant (#1270), Milvus
(#1283), InfluxDB and InfluxDB 3 (#1309) and Oxia (#1310). Also CapRover and
DigitalOcean discovery (#1502, #1378, #1528), launch sign-in for hosting
platforms (#1516), and any number of local accounts with passkeys (#1122,
#1195).

Three security fixes: the OIDC state cookie is no longer accepted as a
session (#1380), a non-admin role's DuckDB connection opens with file and
network access off (#1519), and SQLite, DuckDB and LibreDB connections refuse
Studio's own storage and credential files, with SQLite open to admins only in
the editor, agent runs and MCP alike (#1523, #1529). Chart 0.1.78 is flagged
containsSecurityUpdates, and its description calls Trino by its own name, the
rename #1158 left for this bump.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants