Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions docs/reference/database-indexes.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Create indexes on encrypted columns when:
- The table has a significant number of rows (typically > 1000)
- You frequently query by equality on that column
- Query performance is important
- The column contains searchable index terms (hmac_256, blake3, or ore)
- The column contains searchable index terms (hmac_256, blake3, ore, or ope)

---

Expand All @@ -49,7 +49,7 @@ For PostgreSQL to use an index on encrypted columns, **all** of these conditions
The encrypted data must contain the index term types that support the operation:

- **Equality queries** - Require `unique` index config (adds `hm` hmac_256 or `b3` blake3 terms)
- **Range queries** - Require `ore` index config (adds `ob` ore_block_u64_8_256 terms)
- **Range queries** - Require `ore` index config (adds `ob` ore_block_u64_8_256 terms) **or** `ope` index config (adds `opf` ope_cllw_u64_65 / `opv` ope_cllw_var_8 terms)
- **Pattern matching** - Typically scans (bloom filters don't use B-tree indexes)

**Example:**
Expand Down Expand Up @@ -149,14 +149,16 @@ Bitmap Heap Scan on users

### Range Queries

When encrypted column has `ob` (ore_block_u64_8_256) index terms:
When encrypted column has `ob` (ore_block_u64_8_256), `opf` (ope_cllw_u64_65), or `opv` (ope_cllw_var_8) index terms:

```sql
SELECT * FROM events
WHERE encrypted_date < $1::eql_v2_encrypted
ORDER BY encrypted_date DESC;
```

The encrypted operator class transparently dispatches to whichever ordered term is present on the column, so range queries against an `ore`-configured column and an `ope`-configured column have identical SQL.

### GROUP BY

Encrypted columns can be used in GROUP BY with indexes:
Expand Down Expand Up @@ -215,6 +217,8 @@ B-tree indexes **only work** with:
- `hm` (hmac_256) - for equality
- `b3` (blake3) - for equality
- `ob` (ore_block_u64_8_256) - for range queries
- `opf` (ope_cllw_u64_65) - for range queries (fixed-width OPE)
- `opv` (ope_cllw_var_8) - for range queries (variable-width OPE)

They **do not work** with:
- `bf` (bloom_filter) - pattern matching
Expand Down Expand Up @@ -404,7 +408,7 @@ If you see `Seq Scan`, ensure:
| Feature | B-tree Index | GIN Index |
|---------|-------------|-----------|
| **Use case** | Equality, range queries | JSONB containment |
| **Index terms** | `hm`, `b3`, `ob` | `sv` (via jsonb_array) |
| **Index terms** | `hm`, `b3`, `ob`, `opf`, `opv` | `sv` (via jsonb_array) |
| **Operators** | `=`, `<`, `>`, `<=`, `>=` | `@>`, `<@` |
| **Function** | Direct column reference | `eql_v2.jsonb_array()` |

Expand All @@ -417,10 +421,13 @@ If you see `Seq Scan`, ensure:
**Check 1: Verify data has index terms**

```sql
-- Check if data contains hm (hmac_256) or b3 (blake3) for equality
-- Check if data contains hm (hmac_256) or b3 (blake3) for equality,
-- ob (ore) for range, or opf/opv (ope) for range
SELECT encrypted_email::jsonb ? 'hm' AS has_hmac,
encrypted_email::jsonb ? 'b3' AS has_blake3,
encrypted_email::jsonb ? 'ob' AS has_ore
encrypted_email::jsonb ? 'ob' AS has_ore,
encrypted_email::jsonb ? 'opf' AS has_ope_fixed,
encrypted_email::jsonb ? 'opv' AS has_ope_var
FROM users LIMIT 1;
```

Expand Down Expand Up @@ -457,7 +464,7 @@ WHERE tablename = 'users'
1. **Ensure index exists and is being used** - Use `EXPLAIN ANALYZE`
2. **Check table has been ANALYZEd** - Run `ANALYZE table_name`
3. **Consider index selectivity** - Very small tables might not use indexes
4. **Check for appropriate search config** - Equality needs `unique`, ranges need `ore`
4. **Check for appropriate search config** - Equality needs `unique`, ranges need `ore` or `ope`

---

Expand Down
11 changes: 10 additions & 1 deletion docs/reference/index-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Add an index to an encrypted column. Returns the updated configuration as JSONB.
SELECT eql_v2.add_search_config(
'table_name', -- Name of the table
'column_name', -- Name of the column
'index_name', -- Index kind ('unique', 'match', 'ore', 'ste_vec')
'index_name', -- Index kind ('unique', 'match', 'ore', 'ope', 'ste_vec')
'cast_as', -- PostgreSQL type to cast decrypted data ('text', 'int', etc.)
'opts' -- Index options as JSONB (optional)
);
Expand Down Expand Up @@ -109,6 +109,15 @@ If you're using n-gram as a token filter, then a token that is already shorter t
However, if that same short string only appears as a part of a larger token, then it will not match that record.
Try to ensure that the string you search for is at least as long as the `tokenLength` of the index, except in the specific case where you know that there are shorter tokens to match, _and_ you are explicitly OK with not returning records that have that short string as part of a larger token.

#### `ore` vs `ope`

Both `ore` and `ope` enable the same ordered-comparison surface (`<`, `<=`, `=`, `>`, `>=`, `BETWEEN`, `ORDER BY`, `MIN`/`MAX`).

- **`ore`** uses Order-Revealing Encryption (`ore_block_u64_8_256`, payload field `ob`). Ciphertexts compare via a custom per-byte protocol implemented in `eql_v2.compare_ore_block_u64_8_256`. This is the default ordered-search index.
- **`ope`** uses CLWW Order-Preserving Encryption — `ope_cllw_u64_65` (fixed-width, payload field `opf`) for numeric types and `ope_cllw_var_8` (variable-width, payload field `opv`) for text-shaped values. OPE ciphertexts compare with **standard lexicographic byte ordering**, which makes them usable in environments that can only sort `bytea` natively (e.g. some pluggable storage layers without custom comparators).
Comment on lines +116 to +117

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Use CLLW consistently in the OPE docs.

These lines say CLWW, but the new type names in this PR are ope_cllw_*. Fixing the spelling here will make the docs easier to search and less confusing.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/reference/index-config.md` around lines 116 - 117, Documentation
mistakenly spells the OPE scheme as "CLWW" while the new cipher names use
"CLLW"; update the text to consistently use "CLLW" (e.g., change "CLWW
Order-Preserving Encryption" to "CLLW Order-Preserving Encryption") and ensure
references to the cipher identifiers ope_cllw_u64_65 and ope_cllw_var_8 remain
correct so searches and examples match the actual type names used elsewhere.


`eql_v2.compare()` and the `<` / `<=` / `>` / `>=` operators dispatch automatically to whichever ordered terms are present on the encrypted value, so application queries do not change when switching between `ore` and `ope`.

#### Options for ste_vec indexes (`opts`)

An ste_vec index on an encrypted JSONB column enables the use of PostgreSQL's `@>` and `<@` [containment operators](https://www.postgresql.org/docs/16/functions-json.html#FUNCTIONS-JSONB-OP-TABLE).
Expand Down
Loading
Loading