Skip to content

Partitioned type: lookups on a secondary index are pruned to the wrong bucket, returning nothing and letting UNIQUE duplicates through #5589

Description

@lvca

On a type using PartitionedBucketSelectionStrategy, a lookup on any index other than the partition index is pruned to the wrong bucket. Two consequences, both silent:

  1. the lookup returns nothing;
  2. a secondary UNIQUE index stops enforcing its constraint, because the commit-time duplicate check reads through the same pruned path.

No exception is raised in either case.

Root cause

TypeIndex.getIndexesByKeys (engine/src/main/java/com/arcadedb/index/TypeIndex.java:567) prunes with:

final int bucketIndex = type.getBucketIndexByKeys(keys, ...);
if (bucketIndex > -1) {
  final List<String> propNames = getPropertyNames();
  List<IndexInternal> polymorphicIndexesOnKeys = type.getPolymorphicBucketIndexByBucketId(
      type.getBuckets(false).get(bucketIndex).getFileId(), propNames);
  ...
  return polymorphicIndexesOnKeys;
}

keys are the values of this index's properties, and PartitionedBucketSelectionStrategy.getBucketIdByKeys hashes whatever it is handed without checking that those properties are the partition properties. But a record is placed by getBucketIdByRecord, which hashes the partition properties. For any index other than the partition one the two hashes are unrelated, so the pruned search reads a bucket the record is not in.

LocalDocumentType.getPolymorphicBucketIndexByBucketId (:880) does filter by property name, but only after the wrong bucket has already been chosen, so it cannot catch this.

The query planner guards its own bucket-pruning rule by comparing the partition properties against the ones in the predicate: SelectExecutionPlanner:2017 and PartitionPruning:63 both read partitioned.getProperties() and bail when it does not line up. The index path has no equivalent guard.

Round-robin and thread strategies are unaffected: ThreadBucketSelectionStrategy.getBucketIdByKeys returns -1, so pruning never fires.

Reproducer

Added as engine/src/test/java/com/arcadedb/partitioning/PartitionedSecondaryIndexLookupTest.java. Schema: 8 buckets, UNIQUE on tenant_id (the partition key) and UNIQUE on code, strategy partitioned('tenant_id').

CREATE PROPERTY PartitionedSecondary.tenant_id STRING;
CREATE PROPERTY PartitionedSecondary.code STRING;
CREATE INDEX ON PartitionedSecondary(tenant_id) UNIQUE;
CREATE INDEX ON PartitionedSecondary(code) UNIQUE;
ALTER TYPE PartitionedSecondary BucketSelectionStrategy `partitioned('tenant_id')`;

Results on current main:

case expected actual
200 rows inserted, looked up by code via TypeIndex.get 200 found 0 found
same, via SELECT ... WHERE code = ? 200 found 0 found
same schema on the default round-robin strategy (control) 200 found 200 found
38 inserts duplicating an existing code 38 rejected 6 accepted, 32 rejected as DuplicatedKeyException, 0 unrelated errors

The rows are stored correctly in every case (countType returns 200); only the index path is wrong.

Suggested fix

Guard the pruning in getIndexesByKeys the way the planner already does: only prune when the strategy is a PartitionedBucketSelectionStrategy and its getProperties() matches this index's getPropertyNames(). Otherwise fall through to the existing return indexesOnBuckets fan-out, which is correct if slower. That also keeps the existing isNeedsRepartition() bail-out semantics.

Worth deciding separately whether ALTER TYPE ... BucketSelectionStrategy partitioned(...) should warn (or refuse) when the type carries indexes on other properties, since today the strategy attaches silently and the breakage only shows up at read time.

Impact

Anyone using partitioned on a type with more than one index. The duplicate-admission half means a UNIQUE index can already hold duplicate keys on disk, so a fix should probably come with a note that affected indexes need REBUILD INDEX and a duplicate check.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions