feat: Add SQL support for INTERSECT and EXCEPT ops

[alexander-beedie]

Features

• Adds SQL support for EXCEPT¹ and INTERSECT² ops.
• Fixes ARRAY_TO_STRING so that it works with non-string arrays (casts inner type).
• Added recognition of TABLE xyz³ PostgreSQL shortcut syntax.
• Added some SQL docs and extended/clarified array-function examples.
• Miscellaneous SQLSyntaxError message tidy-ups.

Also

• Upgrades sqlparser-rs to the latest released version (0.47.0); this actually turned into majority of the PR, as various aspects of Function representation were consolidated⁴, requiring non-trivial changes on our side. However, it does leave things in a better place, and allows for additional centralisation/validation of function properties & syntax 👌
• Taking advantage of this we now catch a lot of additional unsupported syntax.

Follow-up

• Already added internal support for [EXCEPT | INTERSECT] BY NAME, but it won't get parsed until we pick up a new sqlparser-rs with this PR merged: Support use of BY NAME quantifier across all set ops apache/datafusion-sqlparser-rs#1309 (no rush).

Examples

import polars as pl

lf1 = pl.LazyFrame({
    "id": [1, 2, 3],
    "name": ["Alice", "Bob", "Charlie"],
})

lf2 = pl.LazyFrame({
    "id": [2, 3, 4],
    "age": [30, 25, 45],
    "name": ["Bob", "Charlie", "David"],
})

EXCEPT:

pl.sql("""
  SELECT id, name FROM lf1
  EXCEPT
  SELECT id, name FROM lf2
""").collect()

# shape: (1, 2)
# ┌─────┬───────┐
# │ id  ┆ name  │
# │ --- ┆ ---   │
# │ i64 ┆ str   │
# ╞═════╪═══════╡
# │ 1   ┆ Alice │
# └─────┴───────┘

INTERSECT:

pl.sql("""
  SELECT id, name FROM lf1
  INTERSECT
  SELECT id, name FROM lf2
""").collect()

# shape: (2, 2)
# ┌─────┬─────────┐
# │ id  ┆ name    │
# │ --- ┆ ---     │
# │ i64 ┆ str     │
# ╞═════╪═════════╡
# │ 2   ┆ Bob     │
# │ 3   ┆ Charlie │
# └─────┴─────────┘

Footnotes

1. https://www.postgresql.org/docs/current/sql-select.html#SQL-EXCEPT ↩

2. https://www.postgresql.org/docs/current/sql-select.html#SQL-INTERSECT ↩

3. https://www.postgresql.org/docs/current/sql-select.html#SQL-TABLE ↩

4. https://github.com/sqlparser-rs/sqlparser-rs/pull/1247 ↩

[codecov]

Codecov Report

Attention: Patch coverage is 82.91317% with 61 lines in your changes missing coverage. Please review.

Project coverage is 81.03%. Comparing base (4584b45) to head (e21097f).
Report is 11 commits behind head on main.

Files	Patch %	Lines
crates/polars-sql/src/functions.rs	88.03%	31 Missing ⚠️
crates/polars-sql/src/context.rs	75.64%	19 Missing ⚠️
crates/polars-sql/src/sql_expr.rs	45.00%	11 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #16960      +/-   ##
==========================================
+ Coverage   80.95%   81.03%   +0.07%     
==========================================
  Files        1445     1445              
  Lines      189822   190276     +454     
  Branches     2712     2712              
==========================================
+ Hits       153675   154191     +516     
+ Misses      35647    35586      -61     
+ Partials      500      499       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.