On a type using PartitionedBucketSelectionStrategy, a lookup on any index other than the partition index is pruned to the wrong bucket. Two consequences, both silent:
- the lookup returns nothing;
- a secondary UNIQUE index stops enforcing its constraint, because the commit-time duplicate check reads through the same pruned path.
No exception is raised in either case.
Root cause
TypeIndex.getIndexesByKeys (engine/src/main/java/com/arcadedb/index/TypeIndex.java:567) prunes with:
final int bucketIndex = type.getBucketIndexByKeys(keys, ...);
if (bucketIndex > -1) {
final List<String> propNames = getPropertyNames();
List<IndexInternal> polymorphicIndexesOnKeys = type.getPolymorphicBucketIndexByBucketId(
type.getBuckets(false).get(bucketIndex).getFileId(), propNames);
...
return polymorphicIndexesOnKeys;
}
keys are the values of this index's properties, and PartitionedBucketSelectionStrategy.getBucketIdByKeys hashes whatever it is handed without checking that those properties are the partition properties. But a record is placed by getBucketIdByRecord, which hashes the partition properties. For any index other than the partition one the two hashes are unrelated, so the pruned search reads a bucket the record is not in.
LocalDocumentType.getPolymorphicBucketIndexByBucketId (:880) does filter by property name, but only after the wrong bucket has already been chosen, so it cannot catch this.
The query planner guards its own bucket-pruning rule by comparing the partition properties against the ones in the predicate: SelectExecutionPlanner:2017 and PartitionPruning:63 both read partitioned.getProperties() and bail when it does not line up. The index path has no equivalent guard.
Round-robin and thread strategies are unaffected: ThreadBucketSelectionStrategy.getBucketIdByKeys returns -1, so pruning never fires.
Reproducer
Added as engine/src/test/java/com/arcadedb/partitioning/PartitionedSecondaryIndexLookupTest.java. Schema: 8 buckets, UNIQUE on tenant_id (the partition key) and UNIQUE on code, strategy partitioned('tenant_id').
CREATE PROPERTY PartitionedSecondary.tenant_id STRING;
CREATE PROPERTY PartitionedSecondary.code STRING;
CREATE INDEX ON PartitionedSecondary(tenant_id) UNIQUE;
CREATE INDEX ON PartitionedSecondary(code) UNIQUE;
ALTER TYPE PartitionedSecondary BucketSelectionStrategy `partitioned('tenant_id')`;
Results on current main:
| case |
expected |
actual |
200 rows inserted, looked up by code via TypeIndex.get |
200 found |
0 found |
same, via SELECT ... WHERE code = ? |
200 found |
0 found |
| same schema on the default round-robin strategy (control) |
200 found |
200 found |
38 inserts duplicating an existing code |
38 rejected |
6 accepted, 32 rejected as DuplicatedKeyException, 0 unrelated errors |
The rows are stored correctly in every case (countType returns 200); only the index path is wrong.
Suggested fix
Guard the pruning in getIndexesByKeys the way the planner already does: only prune when the strategy is a PartitionedBucketSelectionStrategy and its getProperties() matches this index's getPropertyNames(). Otherwise fall through to the existing return indexesOnBuckets fan-out, which is correct if slower. That also keeps the existing isNeedsRepartition() bail-out semantics.
Worth deciding separately whether ALTER TYPE ... BucketSelectionStrategy partitioned(...) should warn (or refuse) when the type carries indexes on other properties, since today the strategy attaches silently and the breakage only shows up at read time.
Impact
Anyone using partitioned on a type with more than one index. The duplicate-admission half means a UNIQUE index can already hold duplicate keys on disk, so a fix should probably come with a note that affected indexes need REBUILD INDEX and a duplicate check.
On a type using
PartitionedBucketSelectionStrategy, a lookup on any index other than the partition index is pruned to the wrong bucket. Two consequences, both silent:No exception is raised in either case.
Root cause
TypeIndex.getIndexesByKeys(engine/src/main/java/com/arcadedb/index/TypeIndex.java:567) prunes with:keysare the values of this index's properties, andPartitionedBucketSelectionStrategy.getBucketIdByKeyshashes whatever it is handed without checking that those properties are the partition properties. But a record is placed bygetBucketIdByRecord, which hashes the partition properties. For any index other than the partition one the two hashes are unrelated, so the pruned search reads a bucket the record is not in.LocalDocumentType.getPolymorphicBucketIndexByBucketId(:880) does filter by property name, but only after the wrong bucket has already been chosen, so it cannot catch this.The query planner guards its own bucket-pruning rule by comparing the partition properties against the ones in the predicate:
SelectExecutionPlanner:2017andPartitionPruning:63both readpartitioned.getProperties()and bail when it does not line up. The index path has no equivalent guard.Round-robin and thread strategies are unaffected:
ThreadBucketSelectionStrategy.getBucketIdByKeysreturns-1, so pruning never fires.Reproducer
Added as
engine/src/test/java/com/arcadedb/partitioning/PartitionedSecondaryIndexLookupTest.java. Schema: 8 buckets,UNIQUEontenant_id(the partition key) andUNIQUEoncode, strategypartitioned('tenant_id').Results on current
main:codeviaTypeIndex.getSELECT ... WHERE code = ?codeDuplicatedKeyException, 0 unrelated errorsThe rows are stored correctly in every case (
countTypereturns 200); only the index path is wrong.Suggested fix
Guard the pruning in
getIndexesByKeysthe way the planner already does: only prune when the strategy is aPartitionedBucketSelectionStrategyand itsgetProperties()matches this index'sgetPropertyNames(). Otherwise fall through to the existingreturn indexesOnBucketsfan-out, which is correct if slower. That also keeps the existingisNeedsRepartition()bail-out semantics.Worth deciding separately whether
ALTER TYPE ... BucketSelectionStrategy partitioned(...)should warn (or refuse) when the type carries indexes on other properties, since today the strategy attaches silently and the breakage only shows up at read time.Impact
Anyone using
partitionedon a type with more than one index. The duplicate-admission half means a UNIQUE index can already hold duplicate keys on disk, so a fix should probably come with a note that affected indexes needREBUILD INDEXand a duplicate check.