feat(postgresql): index a pre-built tsvector, making setweight() weighting reachable

[jeremydmiller]

Closes #541. Raised from JasperFx/marten#5298.

The constraint

FullTextIndexDefinition.Columns unconditionally wrapped whatever it was given:

get => new[] { $"to_tsvector('{regConfig}',{DocumentConfig.Trim()})" };

So DocumentConfig was always text to be converted, and there was no way to hand the type an expression that is already a tsvector. PostgreSQL's weighting works by labelling each member's vector and concatenating the vectors, not the text — so the expression is a tsvector at the top level, and passing it as DocumentConfig produced to_tsvector('english', setweight(…) || setweight(…)): a type error, not a weighted index.

The shape

TsVectorExpression, consumed by Columns without the to_tsvector wrapping, plus a ForTsVector factory. A factory rather than a constructor overload — the existing constructor takes a string in that position followed by three optional string?s, so an overload would be ambiguous wherever the optionals are omitted and would silently pick wrong where they are not.

var weighted =
    "setweight(to_tsvector('english', coalesce(data ->> 'Title', '')), 'A') || " +
    "setweight(to_tsvector('english', coalesce(data ->> 'Body', '')), 'B')";

var index = FullTextIndexDefinition.ForTsVector(tableName, weighted);

Nothing changes for anyone who does not opt in. Left unset, every existing definition emits the byte-identical DDL it always did — pinned by a theory over four DocumentConfig/RegConfig combinations, because a changed index expression means Weasel drops and recreates the index, which on a large table is an outage rather than a migration.

One property, so the vectors cannot drift

The issue's actual requirement. IndexedTsVector yields the vector this index is built over whichever way it was configured, and the DDL is generated from it and nothing else:

public string IndexedTsVector =>
    tsVectorExpression == null
        ? $"to_tsvector('{regConfig}',{DocumentConfig.Trim()})"
        : parenthesize(tsVectorExpression);

Marten's FullTextWhereFragment reads the expression back off the index definition for exactly this reason, and ranking makes the coupling stricter: a ts_rank computed over a different vector than the one @@ filtered on is silently wrong rather than merely slow. A test asserts IndexedTsVector is exactly what lands in the DDL for both configurations.

Two things only a real server showed

Both were caught by integration tests, not by reading the code.

1. The DDL was a syntax error. setweight(…) || setweight(…) is an operator expression, and PostgreSQL lets an index expression go bare only when it is a function call or a column reference. IndexDefinition.correctedExpression supplies exactly one pair of parentheses, which the USING gin (…) argument list consumes — so the expression needs its own. It is now parenthesized on the way in, unless the caller already did it. Canonicalization strips parentheses entirely, so delta detection is unaffected either way.

2. A weighted index re-read as changed on every migration. PostgreSQL renders setweight's weight argument with an explicit cast when it gives the index expression back:

expected: … setweightto_tsvector'english',coalescedata->>'title','','a' || …
actual:   … setweightto_tsvector'english',coalescedata->>'title','','a'::"char" || …

That cast is in the actual and never in the expected, so the index would have been dropped and recreated on every single migration — on a table big enough to want ranked search, the expensive kind of wrong. CanonicizeDdl now strips ::"char" alongside the ::text and ::regconfig it already handled; this is the same class of automatic cast the existing fts_index_comparison_must_take_into_account_automatic_cast test was written for.

Tests

Verified against PostgreSQL 17 in a dedicated database.

full_text_index_over_a_prebuilt_tsvector.cs (unit) — DDL shape, no double conversion, name derivation and prefixing, IndexedTsVector for both configurations and its equality with what lands in the DDL, empty expression refused, clearing the expression falling back cleanly, and the byte-identical theory for untouched definitions.

prebuilt_tsvector_index_deltas.cs (integration) — the weighted index is accepted by the server and round-trips with no delta; a single setweight (legal unparenthesized) still round-trips, so the wrapping does not break the case that never needed it; an already-parenthesized expression round-trips; a changed weight is still detected as an update; and ts_rank actually puts a title match above a description match, so the tests prove the expression means what the issue was raised to make it mean rather than merely that Weasel emits and re-reads a string.

Full Weasel.Postgresql.Tests (971) and Weasel.Core.Tests (320) green.

Docs

docs/postgresql/tables.md gains a Full Text Indexes section with a weighting subsection, from compilable mdsnippets in src/DocSamples.

🤖 Generated with Claude Code