Cryptic Scipy "could Not Convert Integer Scalar" Error

I am constructing a sparse vector using a scipy.sparse.csr_matrix like so: csr_matrix((values, (np.zeros(len(indices)), indices)), shape = (1, max_index)) This works fine for most

Solution 1:

Might it be that max_index > 2**31 ? Try this, just to make sure:

csr_matrix((vals, (np.zeros(10), inds/2)), shape = (1, max_index/2))

Solution 2:

The max index you are giving is less than the maximum index of the rows you are supplying.

This sparse.csr_matrix((vals, (np.zeros(10), inds)), shape = (1, np.max(inds)+1)) works fine with me.

Although making a .todense() results in memory error for the large size of the matrix

Solution 3:

Uncommenting the sum_duplicates - function will lead to other errors. But this fix: strange error when creating csr_matrix also solves your problem. You can extend the version_check to newer versions of scipy.

import scipy 
import scipy.sparse  
if scipy.__version__ in ("0.14.0", "0.14.1", "0.15.1"): 
    _get_index_dtype = scipy.sparse.sputils.get_index_dtype 
    def_my_get_index_dtype(*a, **kw): 
        kw.pop('check_contents', None) 
        return _get_index_dtype(*a, **kw) 
    scipy.sparse.compressed.get_index_dtype = _my_get_index_dtype 
    scipy.sparse.csr.get_index_dtype = _my_get_index_dtype 
    scipy.sparse.bsr.get_index_dtype = _my_get_index_dtype