Dynamodb With Boto3 - Limit Acts As Page Size

According to the boto3 docs, the limit argument in query allows you to to limit the number of evaluated objects in your DynamoDB table/GSI. However, LastEvaluatedKey isn't returned

Solution 1:

Your example code is missing the critical part where LastEvaluatedKey is fed back into the query, as an ExclusiveStartKey parameter! So you are retrying the same query in a loop, rather than continuing where the previous query stopped.

For example, here is working code (I generated an array, it's not a cool generator like you did ;-)):

def full_query(table, **kwargs):
    response = table.query(**kwargs)
    items = response['Items']
    while'LastEvaluatedKey'in response:
        response = table.query(ExclusiveStartKey=response['LastEvaluatedKey'], **kwards)
        items.extend(response['Items'])
    return items

You can now run

full_query(Limit=37, KeyConditions={...})

And get all the results, fetched in batches of 37.