Matrices Are Not Aligned Error: Python Scipy Fmin_bfgs

Problem Synopsis: When attempting to use the scipy.optimize.fmin_bfgs minimization (optimization) function, the function throws a derphi0 = np.dot(gfk, pk) ValueError: matr

Solution 1:

In case anyone else encounters this problem ....

1) ERROR 1: As noted in the comments, I incorrectly returned the value from my gradient as a multidimensional array (m,n) or (m,1). fmin_bfgs seems to require a 1d array output from the gradient (that is, you must return a (m,) array and NOT a (m,1) array. Use scipy.shape(myarray) to check the dimensions if you are unsure of the return value.

The fix involved adding:

grad = numpy.ndarray.flatten(grad)

just before returning the gradient from your gradient function. This "flattens" the array from (m,1) to (m,). fmin_bfgs can take this as input.

2) ERROR 2: Remember, the fmin_bfgs seems to work with NONlinear functions. In my case, the sample that I was initially working with was a LINEAR function. This appears to explain some of the anomalous results even after the flatten fix mentioned above. For LINEAR functions, fmin, rather than fmin_bfgs, may work better.

QED

Solution 2:

As of current scipy version you need not pass fprime argument. It will compute the gradient for you without any issues. You can also use 'minimize' fn and pass method as 'bfgs' instead without providing gradient as argument.