test: move string expression coverage from Scala suites to Comet SQL tests #6626

Part of #6615.

What / Why

This issue moves string expression coverage to Comet SQL tests under sql-tests/expressions/string/. The sources are CometStringExpressionSuite, the Spark 3.x CometStringDecodeSuite, and the string tests in CometExpressionSuite (substring, like, contains, char/varchar, read-side padding). Regex tests are in #6627. Static analysis on main at 3bc2faa found the following:

Most of the work extends existing fixtures, which are thinner than the Scala tests. They have:

• no genuinely decomposed or combining characters (the Scala "combining" rows are themselves precomposed; see below);
• no binary lpad/rpad;
• no Parquet string-predicate pushdown;
• no concat_ws with scalar subqueries or native array arguments;
• no case-sensitivity check for contains;
• no escapes or Contains rewrite in like.

Work

Line numbers refer to CometStringExpressionSuite, except L902, L914, L1269, L1306, L1321, L1349, L1573, L1952 and L2909, which are in CometExpressionSuite.

Existing fixtures to extend

• string_lpad.sql / string_rpad.sql (L66, L70, L74)
  • Add decomposed 'é' and Telugu rows, also with a NULL length and with length 4.
  • Add literal non-positive lengths with a column string (lpad(s, -6, 'x'), lpad(s, 0)) and the multi-character pad 'zzz' with a column length.
• split_rust.sql (L196, L207, L228)
  • Add limits 3 and 0, and the regex separators '\\s+' and '\\d+'. The backslashes must be doubled; see test: move regex coverage to Comet SQL tests and fix fixtures whose patterns lose their backslashes #6627.
  • Add UTF-8 rows: CJK, emoji, a genuinely decomposed accent, RTL scripts, supplementary characters, mixed text with '[a-z]+', and a zero-width space.
• chr.sql (L334)
  • Add negative and large column values (61231, -61231, -1700, 512) and the literals 555, -555 and cast(0.5 as int).
• init_cap_enabled.sql (L354): add 'james ähtäri'.
• string_trim.sql (L378)
  • Add trim('SL', s), ltrim('SL', s) and rtrim('SL', s).
  • Add btrim(s), btrim('SL', s) (where the column is the trim set) and btrim(s, 'SL').
  • Add upper(trim(s)) and its variants.
• string_instr.sql, string_replace.sql, string_translate_enabled.sql (L409)
  • Add instr(s, '手机') over '苹果手机', the two-argument replace(s, 'SQL'), and a native translate(s, 'SL苹', '123').
• length.sql (L423): add an X'00FF' row to the binary table.
• substring.sql
  • Covers nine CometStringExpressionSuite tests (L577-L759) and CometExpressionSuite L902 and L914.
  • Rows: 'a', 'ab', 'x', 'hello', 'world!', mixed-width 'aé苹😀', '苹果手机', '😀🎉🔥', and real decomposed, precomposed and Telugu strings.
  • Add the missing start/length combinations, the -2147483647 cases, the literal (-300, 3), substring(s, -2, -2), substring(s, 0, 0), the two-argument substring(s, 0), and four multi-byte literals.
• like.sql
  • L1321: a '%..%' pattern (the Contains rewrite) and use in a WHERE clause over mixed-case data.
  • L1349: LIKE '%\_%' and LIKE '%$_%' ESCAPE '$'. Spark's unescaping keeps \_ and \% as written.
• contains.sql (L1573): rows that differ only in case, a needle at the start of the string, and use in a WHERE clause.
• misc/codegen_dispatch_disabled_fallback.sql (L1952)
  • Add initcap with expect_fallback(spark.comet.exec.scalaUDF.codegen.enabled=false).
  • The opt-in half is already in init_cap_enabled.sql.
• decode.sql, plus a new decode_disabled.sql (MaxSparkVersion: 3.5), for the CometStringDecodeSuite test
  • Set spark.comet.expression.StringDecode.enabled=false.
  • Assert expect_fallback(Expression support is disabled. Set spark.comet.expression.StringDecode.enabled=true to enable it.).

New fixtures

• binary_pad.sql (L95, L99): binary lpad/rpad with column and literal str/len/pad combinations. These go through the dispatcher as StaticInvoke.
• like_parquet_pushdown.sql (L456, L496)
  • Use a ConfigMatrix over spark.sql.parquet.filterPushdown.stringPredicate (and parquet.enable.dictionary), plus parquet.block.size=512.
  • Write one-file tables with range(0, n, 1, 1) and add LIKE / NOT LIKE filters.
• concat_ws_scalar_subquery.sql (L782): dispatcher off, a 32-row single-file table, and six scalar-subquery queries.
• concat_ws_native.sql (L799)
  • Turn the dispatcher off and set StringSplit.allowIncompatible=true, so a plain query proves native execution.
  • Add concat_ws('', arr, s, array('x', NULL, 'y')).
• char_varchar.sql
  • L1269: char(7), char(8) and varchar(6) columns under substring.
  • L2909: a CHAR(2) column holding 'é', 'é', '' and 'ab'.
• collation_native_opt_in.sql (L43, MinSparkVersion: 4.0)
  • Dispatcher off, with Concat.allowIncompatible=true and Reverse.allowIncompatible=true.
  • expect_native(concat) / expect_native(reverse) over CAST('abc' AS STRING COLLATE UTF8_LCASE). Because CometCast folds a literal cast, this does produce the collated Literal the Scala test builds.

Blocked

• CometExpressionSuite L1306 (char/varchar over-length values). It needs a table over files that already contain over-length values. A fixture cannot point a table at a location, and the write-side length check rejects such values. Keep it in Scala unless the framework gains LOCATION support.

Stays in Scala

• L444 reads test-data/dictionary-string-binary.parquet, an Arrow dictionary-typed file that SQL cannot write.

Notes on the Scala tests

• "split string with UTF-8 characters". Its "combining" row is byte-identical to the precomposed row: both use U+00E9 and U+00EF.
• "substring - unicode multi-byte characters". "café" // combining accent is precomposed. Neither test covers combining characters, so the fixtures add real ones.
• "Upper and Lower" and "trim". They set Upper.allowIncompatible / Lower.allowIncompatible, which do nothing: the native path is gated on spark.comet.caseConversion.enabled. Both tests therefore run the dispatcher.
• "lpad/rpad with NULL length" says the generator never emits NULL integers. It does now.
• All-literal pad combinations. Spark folds them in the Scala suite. They really run in Comet once in a fixture.
• concat_ws.sql:44-45 uses spark_answer_only on all-literal arguments, and its comment says the plan dispatches. Pin it with expect_dispatch(concat_ws).

Done when

• Every test in the table below is accounted for. Either a fixture covers it (the same inputs, configs and assertion strength, or stronger), the fixture that already covers it is cited, or it stays in Scala for the reason given above.
• Each converted Scala test is deleted in the PR that adds its coverage. A suite left empty is deleted and removed from the suite lists in .github/workflows/pr_build_linux.yml and pr_build_macos.yml.
• The new and changed fixtures pass on every Spark profile. Apply run-all-spark-profiles, because pull requests run only the default profile.