String </> (and ORDER BY/MIN/MAX) compare by UTF-16 code units instead of Unicode code points #4300

Version

6.2.0

What happened?

Reporter: ting668

Environment

• Apache Jena Version: 6.2.0
• Java Version: 21.0.12+8-LTS

Description

While testing Apache Jena using an intuitive approach inspired by attribute constraints, I found that ordering comparisons on xsd:string and simple literals compare by UTF-16 code unit order instead of Unicode code point order, which flips the result for any BMP character at/above U+E000 versus an astral character.

SPARQL 1.1 §17.3 maps </> on simple literals and xsd:string to fn:compare, which compares by code point. U+FFFD (65533) and U+FFFE (65534) are both numerically smaller than U+1D11E (MUSICAL SYMBOL G CLEF, 119070), so both comparisons below must be true. Jena 6.2.0 returns false for both, because in UTF-16 the astral character is encoded as a surrogate pair starting at U+D800, which sorts before both U+FFFD and U+FFFE.

How to Reproduce and Expected Behavior

Note: The queries below are a minimized, simplified example reproducing the bug.

Buggy query (no data needed; both strings use SPARQL escape syntax):

SELECT ('\uFFFD' < '\U0001D11E' AS ?a)
       ('\uFFFE' < '\U0001D11E' AS ?b)
WHERE {}

Expected behavior (code point order: 65533 and 65534 are both < 119070):

a=true, b=true

Actual behavior (Jena 6.2.0):

a=false, b=false

(Control: SELECT ('a' < 'b' AS ?a) WHERE {} returns true, so BMP-internal comparisons work; the flip appears exactly when one side is an astral character encoded as a surrogate pair.)

The same wrong order surfaces in data-facing operations: with a property holding '\uFFFD' and '\U0001D11E' values, ORDER BY puts the astral value first, and MIN() returns '\U0001D11E' although U+FFFD is the code-point minimum — so this is not specific to the < operator but to the shared string comparison.

The comparison appears to delegate to Java String.compareTo, which compares UTF-16 code units; comparing by code point (or equivalently comparing String.codePointAt sequences, or using a code-point-aware comparator) would match fn:compare.

Relevant output and stacktrace

Are you interested in making a pull request?

None