Cosmos: query_items_change_feed reture same resualt with is_start_from_beginning parameters

[nonokangwei]

• Package Name: azure-cosmos
• Package Version: 4.0.0
• Operating System: Linux
• Python Version: 3.5.3

Describe the bug
query_items_change_feed fucntion return the same result with the parameter is_start_from_beginning=True and False

To Reproduce
Steps to reproduce the behavior:

python -m venv venv
source venv/bin/activate
pip install azure-cosmos==4.0.0

Query

client = cosmos_client.CosmosClient(HOST, {'masterKey': MASTER_KEY})
container = client.get_database_client(DATABASE_ID).get_container_client(CONTAINER_ID)
container.query_items_change_feed(is_start_from_beginning=True)
# with is_start_from_beginning=false
container.query_items_change_feed()

Expected behavior
Read the change feed from the last checkpoint, tried with the parameter continunation, did not find the sample to fetch the continunation token.

[annatisch]

Thanks @nonokangwei - I will try to repro today :)

[annatisch]

Hi @nonokangwei - I put together some sample code based on our tests demonstrating the continuation support that is hopefully helpful:

document_definition = {'pk': 'pk', 'id':'doc1'}
collection.create_item(body=document_definition)

query_iterable = collection.query_items_change_feed(
    partition_key_range_id=pkRangeId,
    is_start_from_beginning=True,
)
iter_list = list(query_iterable)
self.assertEqual(len(iter_list), 1)

continuation = collection.client_connection.last_response_headers['etag']

# Create two new documents. Verify that change feed contains the 2 new documents
document_definition = {'pk': 'pk', 'id': 'doc2'}
collection.create_item(body=document_definition)
document_definition = {'pk': 'pk', 'id': 'doc3'}
collection.create_item(body=document_definition)

# verify iterator
 query_iterable = created_collection.query_items_change_feed(
    partition_key_range_id=pkRangeId,
    continuation=continuation,
    max_item_count=pageSize
)

Here is the actual test with more detail:
https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/cosmos/azure-cosmos/test/test_query.py#L50

As for the behaviour of is_start_from_beginning - @southpolesteve could you please weigh in with regards to the expected service behaviour of this options as presenting in the OP snippet?

[southpolesteve]

@nonokangwei Is this a partitioned collection? You'll need to pass the partition key range id as the example above

@annatisch Thanks for investigating. The API is a bit strange here. Partition Key Range ID is an integer representing the physical partition that a logical partition key maps to. But it's not something we normally expose to the user. I don't see any methods in the SDK that let a user convert a partition key to a partition key range id. This might just be a bug. Instead the SDK will need to accept the partition key and set that as the header. This is what JS does and forces the user to supply the partition key: https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/cosmosdb/cosmos/samples/ChangeFeed.ts#L73-L78

[nonokangwei]

@annatisch @southpolesteve i do some test based on the @annatisch sample code, if i do not give the partition key range id paramterer it seems not works. as @southpolesteve said i review the SDK, it seems no function for customer to get the partition key range id. So what is the suggested method to get the partition key range id, i only saw the description in rest-api interface.

[southpolesteve]

Relabeling this as a bug. Partition Key Range Id shouldn't be exposed here. We should instead be accepting a partition key and change feed is scoped to a single partition key value. Cross partition change feed is something we would eventually like to support, but it requires some additional work.