Applying Several Functions To Each Row Of An Array

I have a numpy array which has only a few non-zero entries which can be either positive or negative. E.g. something like this: myArray = np.array([[ 0. , 0. , 0.

Solution 1:

Here's what I have tried - using reduce, map. I am not sure how fast this is - but is this what you are trying to do?

Edit 4: Simplest and most readable - Make l a numpy array and then greatly simplifies where.

import numpy as np
import time

l = np.array([1.0, 2.0, 3.0])

defposFunc(x,y):
    return x*y

defnegFunc(x,y):
    return x*(y+1)

defmyFunc(x, y):
    if x > 0:
        return posFunc(x, y)
    if x < 0:
        return negFunc(x, y)
    else:
        return1.0

myArray = np.array([
        [ 0.,0.,0.],
        [ 0.32, -6.79,  0.],
        [ 0.,0.,0.],
        [ 0.,1.5,0.],
        [ 0.,0., -1.71]])

t1 = time.time()
a = np.array([reduce(lambda x, (y,z): x*myFunc(z,l[y]), enumerate(x), 1) for x in myArray])
t2 = time.time()
print (t2-t1)*1000000print a

Basically let's just look at last line it says cumulatively multiply things in enumerate(xx), starting with 1 (last parameter to reduce). myFunc simply takes the element in myArray(row) and element @ index row in l and multiplies them as needed.

My output is not same as yours - so I am not sure whether this is exactly what you want, but may be you can follow the logic.

Also I am not so sure how fast this will be for huge arrays.

edit: Following is a 'pure numpy way' to do this.

my = myArray # just for brevity

t1 = time.time() 
# First set the positive and negative values# complicated - [my.itemset((x,y), posFunc(my.item(x,y), l[y])) for (x,y) in zip(*np.where(my > 0))]# changed to my = np.where(my > 0, my*l, my)
# complicated - [my.itemset((x,y), negFunc(my.item(x,y), l[y])) for (x,y) in zip(*np.where(my < 0))]# changed to my = np.where(my < 0, my*(l+1), my)
# print my - commented out to time it.# Now set the zeroes to 1.0smy = np.where(my == 0.0, 1.0, my)
# print my  - commented out to time it

a = np.prod(my, axis=1)
t2 = time.time()
print (t2-t1)*1000000print a

Let me try to explain the zip(*np.where(my != 0)) part as best as I can. np.where simply returns two numpy arrays first array is an index of row, second array is an index of column that matches the condition (my != 0) in this case. We take a tuple of those indices and then use array.itemset and array.item, thankfully, column index is available for free to us, so we can just take the element @ that index in the list l. This should be faster than previous (and by orders of magnitude readable!!). Need to timeit to find out whether it indeed is.

Edit 2: Don't have to call separately for positive and negative can be done with one call np.where(my != 0).

Solution 2:

So, let's see if I understand your question.

1. You want to map elements of your matrix to a new matrix such that:
   • 0 maps to 1
   • x>0 maps to funPos(x)
   • x<0 maps to funNeg(x)
2. You want to calculate the product of all elements in the rows this new matrix.

So, here's how I would go about doing it:

1:

def myFun(a):
    if a==0:
        return1if a>0:
        return funPos(a)
    if a<0:
        return funNeg(a)

newFun = np.vectorize(myFun)
newArray = newFun(myArray)

And for 2:

np.prod(newArray, axis = 1)

---

Edit: To pass the index to funPos, funNeg, you can probably do something like this:

# Python 2.7
r,c = myArray.shape
ctr = -1# I don't understand why this should be -1 instead of 0defmyFun(a):
    global ctr
    global c
    ind = ctr % c
    ctr += 1if a==0:
        return1if a>0:
        return funPos(a,l[ind])
    if a<0:
        return funNeg(a,l[ind])

Solution 3:

I think this numpy function would be helpful to you

numpy.apply_along_axis

Here is one implementation. Also I would warn against checking if the sum of the array is 0. Comparing floats to 0 can give unexpected behavior due to machine accuracy constraints. Also if you have -5 and 5 the sum is zero and I'm not sure thats what you want. I used numpy's any() function to see if anything was nonzero. For simplicity I also pulled your list (my_list) into global scope.

import numpy as np


my_list = 1.1, 2., 3.4deffunc_pos(val1, val2):
    return val1 * val2

deffunc_neg(val1, val2):
    return val1 *(val2 + 1)


defmy_generator(row):
    for i, a inenumerate(row):
        if a > 0:
            yield func_pos(a, my_list[i])
        elif a < 0:
            yield func_neg(a, my_list[i])
        else:
            yield1defreduce_row(row):
    ifnot row.any():
        return1.0else:
        return np.prod(np.fromiter(my_generator(row), dtype=float))


defmain():
    myArray = np.array([
            [ 0.        ,  0.        ,  0.        ],
            [ 0.32, -6.79,  0.        ],
            [ 0.        ,  0.        ,  0.        ],
            [ 0.        ,  1.5        ,  0.        ],
            [ 0.        ,  0.        , -1.71]])
    return np.apply_along_axis(reduce_row, axis=1, arr=myArray)

There are probably faster implmentations, I think apply_along_axis is really just a loop under the covers.

I didn't test, but I bet this is faster than what you started with, and should be more memory efficient.

Solution 4:

I've tried your example with the masking function of numpy arrays. However, I couldn't find a solution to replace the values in your array by funPos or funNeg.

So my suggestion would be to try this using pandas instead as it conserves indices while masking.

See my example:

import numpy as np
import pandas as pd

deffunPos(a, b):
    return a * b
deffunNeg(a, b):
    return a * (b + 1)

myPosFunc = np.vectorize(funPos) #vectorized form of funPos
myNegFunc = np.vectorize(funNeg) #vectorized form of funNeg#Input
I = [1.0, 2.0, 3.0]    
x = pd.DataFrame([
    [ 0.,0.,0.],
    [ 0.32, -6.79,  0.],
    [ 0.,0.,0.],
    [ 0.,1.5,0.],
    [ 0.,0., -1.71]])

b = pd.DataFrame(myPosFunc(x[x>0], I)) #calculate all positive values
c = pd.DataFrame(myNegFunc(x[x<0], I)) #calculate all negative values   
b = b.combineMult(c) #put values of c in b
b = b.fillna(1) #replace all missing values that were '0' in the raw array
y = b.product() #multiply all elements in one row#Outputprint ('final result')
print (y)
print (y.tolist())