Skip to content

fix(ai,nodes): SQL error text and ClickHouse session docs (#2713 to stage) - #2733

Merged
nihalnihalani merged 13 commits into
rocketride-org:stagefrom
nihalnihalani:fix/RR-2717-sql-formatter-stage
Oct 20, 2026
Merged

nihalnihalani merged 13 commits into
rocketride-org:stagefrom
nihalnihalani:fix/RR-2717-sql-formatter-stage

Conversation

@nihalnihalani

@nihalnihalani nihalnihalani commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Release backport of #2713 to stage. #2713 fixes the SQL nodes' error text and the ClickHouse session docs.

dylan-savage's review of #2713, and what changed

This backport carries #2713 as it stands after review 5588942190 (CHANGES_REQUESTED on #2713). It does not carry the earlier, larger version. The review asked for four changes. All four are in b856c10b7/9508c8f67/425934045 on develop, which are 90fc69f2b/93ff489ef/e79f14c57 here.

1. Move the ClickHouse fix into the ClickHouse node

Review: almost all of the change sat in the shared db_global_base.py, which db_mysql, db_postgres, rocketride_sql and db_clickhouse all inherit. ClickHouse mentions there went from 12 lines to 137. The MySQL bug QA found came from exactly that placement: ClickHouse's failed at position marker in the shared fallback cut a MySQL SIGNAL reading Upload failed at position 3 down to Upload.

Now:

  • nodes/src/nodes/db_clickhouse/IGlobal.py overrides _format_db_error, as every node already overrides _connection_params and _build_connection_url.
    • It takes clickhouse-driver's error from .orig or from the dbapi wrapper.
    • It drops the driver's doubled class name. On 23.x the driver writes DB::ParsingException. DB::ParsingException: ….
    • It returns a throwIf message whole.
    • It cuts everything else at the first place ClickHouse repeats the statement or its input.
    • Anything else goes to super().
  • The shared fallback is develop's minus the five ClickHouse markers, so the SIGNAL bug is fixed by deleting code. Lines mentioning ClickHouse in db_global_base.py are down to 4.
  • The pymysql branch is develop's again. The reviewer withdrew the point about db_mysql pointed at ClickHouse's MySQL port. The db_mysql README now says to use db_clickhouse for ClickHouse, and that db_mysql pointed at ClickHouse's MySQL interface returns ClickHouse's text as is.
  • The session hook in db_instance_base.py stays where it was. The review called it fine.

2. Cut the test file down

Review: test_format_db_error_dialects.py was 1,929 lines and 55 tests, larger than the module it tests, and most of it pinned regex alternatives to exact strings.

Now: that file is deleted. nodes/test/db_clickhouse/test_clickhouse_errors.py replaces it:

  • One table of live captures from ClickHouse 23.8.16.16 and 25.8.33.6, each row run through both driver routes and asserting properties, not exact strings:
    • no bound value comes back;
    • the sentence survives;
    • no stack trace remains;
    • the Error <code>: prefix is present.
  • Tests for throwIf, the hand-off to super(), which formatter is wired in, the class-only note, and timing.

The eight develop tests that ran ClickHouse text through the shared base moved into that table. test_db_base.py gained:

  • live MySQL rows (SIGNAL whole, 1064 sentence);
  • generic texts that must come back unchanged.

3. The Row N: pattern took cubic time on whitespace

Review: :?\s*\nRow\ \d+:\nColumn\ \d+, sat behind the pattern's leading \s*. Matching took 0.46, 3.5 and 28.6 s on 1k, 2k and 4k spaces; develop takes 8, 34 and 143 ms. Live, a 3,000-space bind value stalled the engine for 12 s.

Now: no pattern has a leading \s*, and Row N: is bounded: (?:\n:\ ?)?\nRow\ \d+:\nColumn\ \d+,. The table below was measured on #2713's head, whose code is identical to this branch's. The input has a whitespace run before any marker. The "Before" column is mine at 1k and 2k, and formatter_engineer's at 4k. The develop column is formatter_engineer's. This branch's own run gave the node 0.07 / 0.25 / 1.07 ms at 1,000 / 4,000 / 20,000 spaces.

Spaces Before develop Node, now Shared, now
1,000 0.47 s 8.9 ms 0.07 ms 0.01 ms
2,000 3.6 s 35 ms 0.13 ms 0.01 ms
4,000 28 s 141 ms 0.26 ms 0.01 ms
100,000 — — 5.8 ms 0.2 ms

The node and the shared base each have a two-stage timing test: 1,000 characters in under 0.1 s, then 20,000 in under 1 s. A cubic regression fails the first stage; develop's quadratic behaviour fails the second.

4. Trim the comments

Review: 66% of the added source lines were comments or docstrings, the _format_db_error docstring alone was 171 lines, and 38% of the whole PR was comments, docstrings or README text.

Now:

  • Docstrings are a few lines each, and comments explain only a non-obvious why. Of the 693 added Python lines, 97 (14%) are comments or docstrings: 24 comment lines and 73 docstring lines.
  • The evidence, history and measurements are in this description and the commit messages.
  • The two READMEs say what the code does in one paragraph each.
  • The out-of-scope edits are reverted: the db_postgres and rocketride_sql README sentences, and an extra session test.

Behaviour changes the review did not ask for

  • Runtime contexts are now cut. : while executing 'FUNCTION …' repeats the function's arguments, which can be bound values.

  • throwIf messages lose their context. The caller's message comes back without its runtime context. The context repeats that message and the condition's arguments.

  • db_mysql pointed at ClickHouse's MySQL interface returns ClickHouse's text untrimmed. On the live captures, 9 of 12 such cases carry bound values, against 4 on develop. The review withdrew that setup, and the db_mysql README says to use db_clickhouse.

Commit map

The repo squash-merges, so here is the map from #2713's commits to this branch:

develop (#2713) stage (this PR) What
fc6afce8a 7c0978bdb per-driver markers, throwIf protection
b17797128 74598aa21 ClickHouse session wording
4f60846e3 86dc05749 live captures in tests
220950bea b75c04665 review round 1
6c18adbf3 cae0b716b ClickHouse markers only on ClickHouse text
ade0f1310 43c4e37a3 before: echo
4aff10d3a 49fead741 older servers' class name
d2613ba89 f39a5cb58 review round 2: 23.x driver text, more echoes
8ecdad4d1 e8ff98ffb review round 2 docs
93636ff37 61bad5d59 two db_mysql sentences
b856c10b7 90fc69f2b review round 3: ClickHouse formatting moved into the node; linear patterns; table-driven tests
9508c8f67 93ff489ef review round 3: docs shortened; out-of-scope edits reverted
425934045 e79f14c57 test comment: which rows are constructed

Type

Bug fix: release backport of #2713 to stage.

Testing

On this branch's head, e79f14c57 (the last commit changes only a test comment):

  • Unit tests. packages/ai/tests/ai/common/database, packages/ai/tests/database and nodes/test/db_clickhouse: 377 passed, 0 failed. The run imported this worktree's own source.
  • Fails before. On pristine stage with this PR's test files added: 49 of the 377 tests fail, all of them in the files this PR adds or changes.
  • Live against ClickHouse 23.8.16.16 and 25.8.33.6 and MySQL 8.4.11: the 32 ClickHouse cases on the node's own routes (its SQLAlchemy connector and the dbapi layer) give 0 leaks and 0 lost sentences, and every result is identical to fix(ai,nodes): cut remaining ClickHouse echoes, keep other dialects' error text whole, fix ClickHouse session docs #2713's head. throwIf text and MySQL SIGNAL and 1064 cases come back whole. The same run's other leaks are the expected ones: db_mysql pointed at ClickHouse's MySQL interface (9, unsupported), and replayed str() / clickhouse-connect payloads that no node produces (4).
  • Docs gates, run locally because stage PRs do not run the "Docs site build" workflow. All rc 0: validate-node-readme.py (db_clickhouse, db_mysql, --all), validate-client-docs.py, docs:check, docs:test, docs:build, ruff 0.16.5 check and format --check, and gitleaks over the 13 commits.

Checklist

Linked Issue

Resolves #2717 together with #2713. Closing keywords have no effect on stage, so #2717 should be closed by hand once both have landed. Related follow-up: #2732.

AI-assisted: written and verified with Claude Code.

🤖 Generated with Claude Code

nihalnihalani and others added 9 commits October 20, 2026 05:20
… other dialects' text

`_format_db_error` ran one marker list over every driver's message. Two
defects followed, both found by a QA run on live MySQL 8.4 and ClickHouse 25.8:

- ClickHouse still echoed the statement, with the bind values
  clickhouse-driver interpolates client-side, in three places the markers
  missed: a named CTE or lambda scope (`in scope x AS (SELECT ...)`,
  `in scope x -> ...`), code 215's `In query SELECT ...`, and a mutation's
  `while processing: '<expr>'`. These are now cut, along with the same CTE
  form after `In query` and the old interpreter's `: While processing`. The
  lookaheads still need the shape ClickHouse's formatter prints, a keyword
  followed by a space included, so prose that shares the words survives.
- One dialect's markers cut another's own text: a MySQL SIGNAL reading
  "Upload failed at position 3 of batch" came back as "Upload", and
  `throwIf(1, 'In scope SELECT secret')` came back as `DB::Exception:`. A
  branch that recognised its driver now cuts only with that driver's
  markers: none for pymysql, since MySQL appends nothing and SQLAlchemy's
  tail is never in `args`, and ClickHouse's for clickhouse-driver. Every
  marker still applies, as a backstop, on the branches that cannot tell
  which driver wrote the text. A throwIf failure (code 395) keeps the
  caller's message whole. The runtime context after it is kept too, unless
  a marker fires inside it, in which case it is dropped whole.

Of the 106 distinct native messages QA captured, exactly the 24 cases above
change, and no bound value survives that did not before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit fc6afce)
The `execute` tool description and the db_clickhouse README told callers to
recover a session with `rollback` or `rollback to savepoint <name>`, and the
begin/commit/rollback tools described a transaction. On ClickHouse,
clickhouse-sqlalchemy's native connector makes commit and rollback no-ops, so
every statement is executed when it is sent. Savepoints do not exist either:
SAVEPOINT is a syntax error. An agent following the advice would believe its
work was discarded when it was not.

`_sessionsAreTransactions()` is false for the clickhouse dialect. The four
session-related tool descriptions read it and say what a ClickHouse session
actually does; every other dialect keeps its existing wording. The README's
tool table and session paragraph now say the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit b177971)
…ith live captures

The error-formatter tests built some messages by hand because no live
server was available. Those shapes are now captured from ClickHouse
25.8.33.6 and MySQL 8.4.11 with clickhouse-driver 0.2.9 and pymysql, and
each test names the statement it was captured for.

- The multi-argument lambda scope, `In query` after a named CTE, and the
  old interpreter's `: While processing` arrive as the tests assumed. Only
  the database name in the 215 message and the column in the alias test
  change.
- A lazy throwIf inside `if()` reports an `if(...)` runtime context, which
  ClickHouse truncates with `...` in the middle of a literal. The formatted
  result is the same.
- A throwIf inside a scalar subquery gets no `In scope`, before or after
  its runtime context. A new test pins this.
- MySQL returns the hand-built SIGNAL texts verbatim.

The formatter itself does not change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 4f60846)
… narrow

Answers dylan-savage's review on rocketride-org#2713.

- pymysql's `(errno, message)` branch keeps a strip again. The pair names
  the wire protocol, not the server: db_mysql pointed at ClickHouse's MySQL
  interface receives `(47, "Code: 47. DB::Exception: ... in scope SELECT
  ... '<bound value>'")`. Text naming `DB::Exception:` is now cut with
  ClickHouse's markers, throwIf protection included, and the duplicated
  `Code: N.` prefix is dropped. MySQL's own text keeps SQLAlchemy's
  markers only, so SIGNAL text stays whole.
- The backstop used by the SQLSTATE, `args[0]`, `str(orig)` and `str(exc)`
  branches is back to the pre-rocketride-org#2713 marker set. The ClickHouse-only
  markers apply only to ClickHouse's text. `clickhouse_driver.dbapi`'s
  `OperationalError(ServerException)` is unwrapped into the ClickHouse
  branch, so that layer keeps every echo fix.
- New ClickHouse echoes are cut:
  - the input a parser prints after ` here: `, which for
    `INSERT ... VALUES` is every later row with its bound values;
  - a function-call scope, `In scope nofunc('<value>')`.
- The throwIf split anchors on `: while executing 'FUNCTION `, so a
  caller's own "while executing" stays in the message. The code check now
  runs before any partitioning, and `Stack trace:` is defined once.
- `_sessionsAreTransactions()` is True in the shared base and overridden in
  db_clickhouse's IInstance. The wording uses the node's display name. The
  `session_id` schema text and the unknown-session error no longer say
  "transaction" on a node without transactions. The README's session_id,
  allow-execute and throwIf wording now match.

Tests use messages captured live from ClickHouse 25.8.33.6, both native and
over its MySQL interface. 16 of them fail on 4f60846 and pass here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 220950b)
Follow-up from an independent review of the previous commit.

- A `.code` / `.message` pair is not ClickHouse's shape alone, so its text
  is cut with ClickHouse's markers only when it opens with `DB::Exception:`.
  Other text gets the backstop again, as it did on develop. The
  one-element-`args` unwrap applies only when the inner text is
  ClickHouse's. Before this, another driver's wrapped error could lose the
  backstop's DETAIL/LINE cut.
- ClickHouse text is recognised with one anchored pattern,
  `\A(?:Code: N. )?(?=DB::Exception:)`, on both the native and the
  MySQL-wire paths. MySQL text that merely mentions the phrase stays whole.
- The new node test loads `db_clickhouse.IInstance` through a bare package
  module, so collection no longer runs the node's `depends()`.
- Docstring corrections: the `(int, str)` strip, SQLAlchemy's tail markers,
  and the commit/rollback method docstrings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 6c18adb)
ValuesBlockInputFormat expects `,` or `(` after a closed row. When it finds
anything else, the error is code 27: `Cannot parse input: expected '('
before: '<rest of the VALUES input>'`. The rest of the input is quoted
there, with every later bound value in it. Agents produce this easily by
writing another dialect's upsert, such as `ON DUPLICATE KEY UPDATE s = $3`
or `ON CONFLICT ... DO UPDATE`.

`FORMAT CSV` adds per-row `parsed text` lines after the same phrase, and
those repeat the earlier rows' values. ` before: '` is now a ClickHouse-only
marker. It is cut natively and over the MySQL interface.

Every message in the new tests was captured live from ClickHouse 25.8.33.6.
The two extra `here:` cases also come from that capture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit ade0f13)
Older servers name their parse errors' class `DB::ParsingException:`. Those
are the errors that carry the `here:` and `before:` echoes. Detection now
accepts any `DB::<name>Exception:`, so that text still gets ClickHouse's
markers, natively and over the MySQL interface.

Also corrects the module comment: it no longer says the backstop runs every
group, and it no longer says the ClickHouse-only lookaheads belong to the
backstop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 4aff10d)
…echoes

Answers dylan-savage's review 5578809432 on rocketride-org#2713.

- clickhouse-driver writes an older server's exception subclass twice,
  `DB::ParsingException. DB::ParsingException: ...`. The start-anchored
  check missed that, so on 23.x the `before:` echo and its bound values
  reached the caller. Detection now has one anchored pattern per arrival
  shape, each read off live 23.8.16.16 and 25.8.33.6 servers:
  - the driver's `.message`, with the repeated class name dropped;
  - `str()` of the driver and dbapi exceptions (`Code: N.` and a newline,
    optionally behind clickhouse-sqlalchemy's `Orig exception: `), and
    clickhouse-connect's one-line wrapper when both codes are equal. On
    these backstop branches the cut is the earlier of the unchanged
    backstop and ClickHouse's markers, so it can only remove more;
  - the MySQL interface's `Code: <errno>. `, which must now repeat the
    pair's own errno. A MySQL SIGNAL whose text starts with
    `DB::Exception:` stays whole.
  Text that only contains ClickHouse's class on a later line is not
  reclassified.
- More echoes are cut, all captured live: ` at: ` after a quoted type
  (code 53), the expression after `Cannot deduce template of expression:`
  (62) and `(result column not found):` (36), a text input format's
  `Row N:` / `Column N` diagnostic (117, 72, CSV/TSV), and `in scope` in
  any expression shape. The throwIf runtime-context check keeps the old
  `in scope` shapes, so a caller's "not in scope for this trigger" passes.
- When ClickHouse's markers leave only the exception class, the error says
  the message only repeated the statement instead of returning a bare
  header.
- Shared markers are defined once (the backstop pattern is unchanged),
  `_strip_clickhouse_detail` has one return path, and the dbapi layer is
  classified once. The docstring now says MySQL 1064 quotes the rest of
  the statement, up to 80 characters.

47 new tests; 29 of them fail on 4aff10d.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit d2613ba)
Answers items C, D and G of dylan-savage's review 5578809432 on rocketride-org#2713.

- db_mysql README: documents what the pymysql branch does. Only
  SQLAlchemy's tail is removed, a SIGNAL's text comes back whole, and a
  1064 `near '...'` quotes the rest of the statement from where the parser
  stopped, up to 80 characters, so a value bound after that point can
  appear. It also states when text from ClickHouse's MySQL interface is
  treated as ClickHouse's.
- db_clickhouse README: the removed shapes are now a list that matches the
  markers, plus the 23.x class prefix, the `str()` and clickhouse-connect
  shapes, and the bare-message fallback.
- The four SQL node READMEs no longer say the full text "stays in the
  server log". It is written to the engine log; only the error `execute`
  returns is trimmed.
- `begin`, `commit` and `rollback` docstrings are provider-accurate.
- A new test pins that db_mysql, db_postgres and rocketride_sql keep
  transactional sessions, so only db_clickhouse opts out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 8ecdad4)
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review: 2
  • develop
  • release/.*

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: rocketride-org/rocketride-server/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 816b8411-bf6f-4bc6-9918-689bc3a1d367

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor
🤖 Internal: Discord sync marker

Auto-managed by the Discord notification workflow. Stores the linked Discord message ID and forum thread ID. Do not edit or delete.

@github-actions github-actions Bot added docs Documentation module:nodes Python pipeline nodes module:ai AI/ML modules labels Oct 20, 2026
A SIGNAL's text is cut at SQLAlchemy's markers like any MySQL message, so
"returned exactly as written, whatever words it contains" overstated it.
A SIGNAL that copies ClickHouse's whole prefix can get the fixed "only
repeated the statement" note appended, so "nothing is added" becomes "no
part of the statement is added".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 93636ff)
nihalnihalani and others added 3 commits October 20, 2026 23:58
…e node

Answers dylan-savage's review 5588942190 on rocketride-org#2713, items 1-3.

- db_clickhouse's IGlobal overrides `_format_db_error`. Every
  clickhouse-driver error, found in `.orig` or as the only argument of a
  dbapi error, is cut at the first place ClickHouse repeats the statement
  or its input. The driver's doubled class name is dropped. A `throwIf`
  (code 395) message comes back whole with its runtime context dropped.
  Anything else goes to the shared formatter.
- The shared fallback no longer carries ClickHouse's markers, so MySQL's
  own text (a SIGNAL reading "Upload failed at position 3 of batch") is no
  longer cut by them. The pymysql branch is develop's again; db_mysql
  pointed at ClickHouse's MySQL interface is not supported.
- No pattern has a leading `\s*` any more. The `Row N:` marker is bounded.
  A 4,000-space run that took 28 s to format now takes 0.26 ms (develop:
  141 ms); 100,000 spaces take 6 ms.
- Tests: the 1,929-line dialects file is replaced by one table of live
  ClickHouse 23.8.16.16 / 25.8.33.6 captures that asserts properties (no
  bound value, the sentence survives, no stack trace) on both driver
  routes, plus two-stage timing tests. The eight develop tests that ran
  ClickHouse text through the shared base move into that table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit b856c10)
…de does

Answers item 4 of dylan-savage's review 5588942190 on rocketride-org#2713.

- db_clickhouse README: one paragraph that lists what the node removes, and
  states that a `throwIf` message comes back whole while a 395 without its
  runtime context gets the markers.
- db_mysql README: only SQLAlchemy's tail is removed. A SIGNAL's text comes
  back whole. A 1064 `near '...'` quotes up to 80 characters of the rest of
  the statement, so a later bound value can appear (rocketride-org#2732). ClickHouse
  should use db_clickhouse.
- Both rewritten paragraphs say the full text is written to the server log,
  not that it stays there.
- `begin`, `commit` and `rollback` docstrings are one line each. The
  session hook and the provider-aware tool descriptions are unchanged.
- Reverted as out of scope: the db_postgres and rocketride_sql README
  sentences, and the extra session pin test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 9508c8f)
The table comment said the last two rows were constructed. The constructed
rows are the four added after the list, with ids starting `constructed-`;
the last two rows of the list are live captures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 4259340)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation module:ai AI/ML modules module:nodes Python pipeline nodes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants