What Is The Correct Way To Get The Previous Page Of Results Given An NDB Cursor?

I'm working on providing an API via GAE that will allow users to page forwards and backwards through a set of entities. I've reviewed the section about cursors on the NDB Queries d

Solution 1:

To make the example from the docs a little clearer let's forget about the datastore for a moment and work with a list instead:

# some_list = [4, 6, 1, 12, 15, 0, 3, 7, 10, 11, 8, 2, 9, 14, 5, 13]

# Set up.
q = Bar.query()

q_forward = q.order(Bar.key)
# This puts the elements of our list into the following order:
# ordered_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]

q_reverse = q.order(-Bar.key)
# Now we reversed the order for backwards paging: 
# reversed_list = [15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

# Fetch a page going forward.

bars, cursor, more = q_forward.fetch_page(10)
# This fetches the first 10 elements from ordered_list(!) 
# and yields the following:
# bars = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# cursor = [... 9, CURSOR-> 10 ...]
# more = True
# Please notice the right-facing cursor.

# Fetch the same page going backward.

rev_cursor = cursor.reversed()
# Now the cursor is facing to the left:
# rev_cursor = [... 9, <-CURSOR 10 ...]

bars1, cursor1, more1 = q_reverse.fetch_page(10, start_cursor=rev_cursor)
# This uses reversed_list(!), starts at rev_cursor and fetches 
# the first ten elements to it's left:
# bars1 = [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

So the example from the docs fetches the same page from two different directions in two differents orders. This is not what you want to achieve.

It seems you already found a solution that covers your use case pretty well but let me suggest another:

Simply reuse cursor1 to go back to page2.
If we're talking frontend and the current page is page3, this would mean assigning cursor3 to the 'next'-button and cursor1 to the 'previous'-button.

That way you have to reverse neither the query nor the cursor(s).

Solution 2:

I took the liberty of changing the Bar model to a Character model. The example looks more Pythonic IMO ;-)

I wrote a quick unit test to demonstrate the pagination, ready for copy-pasting:

import unittest

from google.appengine.datastore import datastore_stub_util
from google.appengine.ext import ndb
from google.appengine.ext import testbed


class Character(ndb.Model):
    name = ndb.StringProperty()

class PaginationTest(unittest.TestCase):
    def setUp(self):
        tb = testbed.Testbed()
        tb.activate()
        self.addCleanup(tb.deactivate)
        tb.init_memcache_stub()
        policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(
            probability=1)
        tb.init_datastore_v3_stub(consistency_policy=policy)

        characters = [
            Character(id=1, name='Luigi Vercotti'),
            Character(id=2, name='Arthur Nudge'),
            Character(id=3, name='Harry Bagot'),
            Character(id=4, name='Eric Praline'),
            Character(id=5, name='Ron Obvious'),
            Character(id=6, name='Arthur Wensleydale')]
        ndb.put_multi(characters)
        query = Character.query().order(Character.key)
        # Fetch second page
        self.page = query.fetch_page(2, offset=2)

    def test_current_page(self):
        characters, _cursor, more = self.page
        self.assertSequenceEqual(
            ['Harry Bagot', 'Eric Praline'],
            [character.name for character in characters])
        self.assertTrue(more)

    def test_next_page(self):
        _characters, cursor, _more = self.page
        query = Character.query().order(Character.key)
        characters, cursor, more = query.fetch_page(2, start_cursor=cursor)

        self.assertSequenceEqual(
            ['Ron Obvious', 'Arthur Wensleydale'],
            [character.name for character in characters])
        self.assertFalse(more)

    def test_previous_page(self):
        _characters, cursor, _more = self.page
        # Reverse the cursor (point it backwards).
        cursor = cursor.reversed()
        # Also reverse the query order.
        query = Character.query().order(-Character.key)
        # Fetch with an offset equal to the previous page size.
        characters, cursor, more = query.fetch_page(
            2, start_cursor=cursor, offset=2)
        # Reverse the results (undo the query reverse ordering).
        characters.reverse()

        self.assertSequenceEqual(
            ['Luigi Vercotti', 'Arthur Nudge'],
            [character.name for character in characters])
        self.assertFalse(more)

Some explanation:

The setUp method first initializes the required stubs. Then the 6 example characters are put with an id so the order isn't random. Since there are 6 characters we have 3 pages of 2 characters. The second page is fetched directly using an ordered query and an offset of 2. Note the offset, this is key for the example.

test_current_page verifies that the two middle characters are fetched. Characters are compared by name for readability. ;-)

test_next_page fetches the next (third) page and verifies the names of the expected characters. Everything is quite straight forward so far.

Now test_previous_page is interesting. This does a couple of things, first the cursor is reversed so the cursor now points backwards instead of forward. (This improves readability, it should work without this, but the offset will be different, I'll leave this as an exercise for the reader.) Next a query is created with a reverse ordering, this is necessary because the offset cannot be negative and you want to have previous entities. Then results are fetched with an offset equal to the page length of the current page. Else the query will return the same results, but reversed (like in the question). Now because the query was reverse-ordered the results are all backwards. We simply reverse the results list in-place to fix this. Last but not least, the expected names are asserted.

Side note: Since this involves global queries the probability is set to 100%, in production (because of the eventual consistency) putting and querying right after will most likely fail.