Skip to content

[fix][broker] Prevent BookkeeperSchemaStorage and BookkeeperBucketSnapshotStorage from bypassing namespace bookie affinity - #26703

Closed
void-ptr974 wants to merge 2 commits into
apache:masterfrom
void-ptr974:fix/aux-ledger-placement-schema-bucket
Closed

void-ptr974 wants to merge 2 commits into
apache:masterfrom
void-ptr974:fix/aux-ledger-placement-schema-bucket

Conversation

@void-ptr974

Copy link
Copy Markdown
Contributor

Motivation

When a namespace has a bookie affinity group, or strict bookie affinity selects the isolated placement policy, managed-ledger data uses the matching placement-aware BookKeeper client. BookkeeperSchemaStorage and BookkeeperBucketSnapshotStorage instead created their auxiliary ledgers with their independently configured default clients and did not persist the selected placement policy in ledger metadata.

As a result, newly created schema and delayed-delivery bucket ledgers could be placed outside the namespace's configured bookie groups. Bookie replacement for those ledgers also lacked the placement metadata needed to preserve the same policy.

Modifications

  • Resolve the schema or delayed-bucket owner topic before creating a ledger.
  • Borrow the placement-aware BookKeeper client introduced by [improve][broker] Add topic-aware BookKeeper context for auxiliary ledger placement #26702 and add the matching standard placement-policy metadata to the ledger.
  • Recognize canonical schema storage keys, including encoded local names and scalable topic names.
  • Preserve the existing default-client behavior for arbitrary and legacy schema storage keys that do not identify an unambiguous owner topic.
  • Keep placement failures strict: do not silently fall back to the default client.
  • Preserve delayed-bucket retry behavior by exposing policy lookup and client creation failures as BucketSnapshotPersistenceException.

Impact

  • Only newly created schema and delayed-delivery bucket ledgers are affected; existing ledgers and their read/delete paths are unchanged.
  • Namespaces without a custom or strict placement policy continue to use the shared default BookKeeper client and do not receive placement metadata.
  • Placement clients are cached by policy configuration and borrowed from managed-ledger storage; this does not create one BookKeeper client per topic.
  • Canonical three-part schema keys inherit the owner namespace policy. Arbitrary and legacy keys continue to use the schema storage's existing default client.

Stack

Depends on #26702. This branch contains the dependency commit; the commit specific to this PR is a7b2ccb439f.

Tests

  • ./gradlew :pulsar-broker:test --no-build-cache --tests org.apache.pulsar.broker.service.schema.BookkeeperSchemaStorageTest --tests org.apache.pulsar.broker.service.schema.SchemaServiceTest --tests org.apache.pulsar.broker.delayed.BookkeeperBucketSnapshotStorageTest --tests org.apache.pulsar.broker.delayed.bucket.BookkeeperBucketSnapshotStoragePlacementTest
  • ./gradlew :pulsar-broker:checkstyleMain :pulsar-broker:checkstyleTest

Resolve topic placement configuration in one shared path and pair it with the cached BookKeeper client and matching ledger metadata.

Assisted-by: Codex
…gers

Resolve the owner topic before creating schema and delayed-delivery bucket ledgers, then use the shared placement-aware BookKeeper client and persist the matching policy metadata. Preserve arbitrary schema keys and bucket retry semantics.

Assisted-by: Codex
@void-ptr974

Copy link
Copy Markdown
Contributor Author

Closing in favor of #26705. The schema and delayed-delivery bucket change remains the second of the three commits in that unified PR, whose description now documents the complete problem, implementation, and impact.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant