Token returned by QueryStateAsync() does not work for subsequent queries

[koumdev]

Hello!

I'm using Azure Cosmos DB state store and the QueryStateAsync() method. I want to get all of my data and used var query = "{\"filter\": {}}", but I run into an exception ("state store statestorecosmos query failed: context canceled") if my database has more than 999 rows. I then tried to use pagination, but I'm running into exceptions again with the pagination token returned from Azure Cosmos DB.

Expected Behavior

var query = "{\"filter\": {}, \"page\": {\"limit\": 100}}";
var employeesResponse = await _daprClient.QueryStateAsync<Employee>(STORE_NAME, query, cancellationToken: cancellation);
// employeesResponse contains the first 100 items as expected

var paginationToken = employeesResponse.Token;

query = $"{{\"filter\": {{}}, \"page\": {{\"limit\": 100, \"token\": {paginationToken}}}}}";
employeesResponse = await _daprClient.QueryStateAsync<Employee>(STORE_NAME, query, cancellationToken: cancellation);
// Should return the next 100 items

Actual Behavior

var query = "{\"filter\": {}, \"page\": {\"limit\": 100}}";
var employeesResponse = await _daprClient.QueryStateAsync<Employee>(STORE_NAME, query, cancellationToken: cancellation); 
// employeesResponse contains the first 100 items as expected, good so far

var paginationToken = employeesResponse.Token;

// Printing paginationToken outputs {"token":"-RID:~R4EMAJWkwMToAwAAAAAAAA==#RT:1#TRC:1000#ISV:2#IEO:65567#QCF:8","range":{"min":"","max":"FF"}}

query = $"{{\"filter\": {{}}, \"page\": {{\"limit\": 100, \"token\": {paginationToken}}}}}";
employeesResponse = await _daprClient.QueryStateAsync<Employee>(STORE_NAME, query, cancellationToken: cancellation);
// QueryStateAsync() throws an exception

QueryStateAsync() throws an exception with details: state store statestorecosmos query failed: failed to parse JSON query body: json: cannot unmarshal object into Go struct field Pagination.page.token of type string

Modifying the paginationToken to remove the first '{' and last '}' to have "token":"-RID:~R4EMAJWkwMToAwAAAAAAAA==#RT:1#TRC:1000#ISV:2#IEO:65567#QCF:8","range":{"min":"","max":"FF"}
Then modifying the query to query = $"{{\"filter\": {{}}, \"page\": {{\"limit\": 100, {paginationToken}}}}}";
QueryStateAsync() throws an exception with details "Invalid Continuation Token\r\nActivityId: fc7cfc39-cd1d-4e4e-a9d4-7ead0d367bac, Microsoft.Azure.Documents.Common/2.14.0".

I'm using Dapr 1.13.0.

Thanks in advance!

[philliphoff]

This is probably better asked in dapr/components-contrib as it's unlikely to be specific to the .NET SDK. That said, the Dapr state store query API is considered to be in "alpha" (and has been for some time) and not all components support it (or all aspects of it). The query API is unlikely to make it out of alpha due to changes in priorities as well as other competing proposals for competing functionality.

[koumdev]

Thank you @philliphoff. I found actually what was the problem with the token... the quotes must be escaped :(. However it still doesn't work, so I've posted the issue here.