[improve][metadata] Replace findByIndex with streaming scanByIndex (point + range)

[merlimat]

Motivation

PIP-473 (metadata-driven transactions for scalable topics) needs to scan secondary indexes by range, not just point lookup — the idx:txn-by-deadline sweep finds open txns past their deadline (key <= now), and the GC sweep finds finalized txns past their retention window. The existing MetadataStore.findByIndex is point-only and returns a materialized List<GetResult>, which doesn't fit either use case.

This PR replaces findByIndex with a single streaming method that subsumes both shapes.

Modifications

API change on MetadataStore:

CompletableFuture<Void> scanByIndex(
    String scanPathPrefix, String indexName,
    String fromKeyInclusive, String toKeyInclusive,  // null = unbounded on that side
    Predicate<GetResult> fallbackFilter,
    ScanConsumer consumer,
    Set<Option> opts)

• Both bounds are inclusive. Point lookup is fromKey == toKey == key. Half-open ranges use null. Listener shape matches scanChildren's existing ScanConsumer.
• findByIndex is removed in the same PR — one method, no separate point-lookup variant. All call sites are migrated.

Oxia (native): client.list(start, end, ListOption.UseIndex(indexName)) returns matching primary keys; results are then fetched via storeGet and streamed to the consumer. Range translation: [scanPathPrefix/fromKey, scanPathPrefix/toKey + "~") — the trailing ~ sentinel widens Oxia's half-open range to include toKey for the secondary-key shapes callers use today (numeric timestamps, fixed-tag enums). Documented in code.

ZK / Memory / RocksDB compatibility layer in AbstractMetadataStore: lists children under scanPathPrefix, fetches each, streams those matching fallbackFilter. The fallback ignores fromKey/toKey — callers encode the range in fallbackFilter.

ScanConsumer.collectInto(List) static helper for callers that prefer the materialized-list shape on top of the streaming API.

Migrated callers:

• ScalableTopicResources.findScalableTopicsAsync (broker)
• MetadataStoreSecondaryIndexTest, MetadataCacheSecondaryIndexTest (added a small in-test findByIndex helper that wraps scanByIndex + collectInto)
• DualMetadataStore wrapper

Verifying this change

• All existing findByIndex* tests migrated to scanByIndex + collectInto and continue to pass on every backend (ZK, Memory, RocksDB, Oxia, MockZooKeeper).
• New scanByIndexInclusiveRange and scanByIndexUnboundedFromKey cover the range-query shape end-to-end (zero-padded numeric secondary keys mirroring PIP-473's idx:txn-by-deadline).

Local results: full :pulsar-metadata:test runs cleanly modulo the recurring BookKeeper.AuditorLedgerCheckerTest.setUp BindException flake (port collision; unrelated to this change).

Does this pull request potentially affect one of the following parts:

• The public MetadataStore API — findByIndex is removed and replaced with scanByIndex. All in-tree callers are migrated in this PR. External implementations of MetadataStore would need the same migration; the new method has a default that returns a not-supported failure, matching the previous default behavior.

Matching PR in forked repository

PR in forked repository: https://github.com/merlimat/pulsar/pull/new/mmerli/metadata-scan-by-index

[lhotari]

LGTM