Itertools.islice Compared To List Slice

I've been trying to apply an algorithm to reduce a python list into a smaller one based on a certain criteria. Due to the large volume of the original list, in the order of 100k el

Solution 1:

islice works with arbitrary iterables. To do this, rather than jumping straight to the nth element, it has to iterate over the first n-1, throwing them away, then yield the ones you want.

Check out the pure Python implementation from the itertools documentation:

def islice(iterable, *args):
    # islice('ABCDEFG', 2) --> A B
    # islice('ABCDEFG', 2, 4) --> C D
    # islice('ABCDEFG', 2, None) --> C D E F G
    # islice('ABCDEFG', 0, None, 2) --> A C E G
    s = slice(*args)
    it = iter(xrange(s.start or 0, s.stop or sys.maxint, s.step or 1))
    nexti = next(it)
    for i, element in enumerate(iterable):
        if i == nexti:
            yield element
            nexti = next(it)

---

Speaking of the itertools documentation, if I was trying to do this operation, I'd probably use the grouper recipe. It won't actually save you any memory, but it could if you rewrote it to be lazier, which wouldn't be tough.

from __future__ import division

from itertools import izip_longest
defgrouper(n, iterable, fillvalue=None):
    "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
    args = [iter(iterable)] * n
    return izip_longest(fillvalue=fillvalue, *args)

reducedVec = []
for chunk in grouper(ratio, vec):
    ifsum(1for x in chunk if x == 'F') > ratio / 3:
        reducedVec.append('F')
    else:
        reducedVec.append('T')

I like using grouper to abstract away the consecutive slices and find this code a lot easier to read than the original

Solution 2:

My guess would be that using islice() involves a Python function call for each element of vec, while the extended slice notation is understood by the parser and translates directly to CPython calls.