Copy Numpy Recarray To Ndarray

I have a process that requires extracting data from a numpy recarray to a ndarray where I then do some vector math. (The recarray comes from a pytables table.read() function.) I wa

Solution 1:

This is a future warning, not an error. The change has been postponed to 1.16. It has to do with multifield indexing, your [['X','Y','Z']] step.

In [56]: dt = np.dtype([('ID', '<i8'), ('X', '<f8'), ('Y', '<f8'), ('Z', '<f8'), ('FREQ', '<i8')])
In [57]: arr = np.ones(3, dtype=dt)
In [58]: arr       # a structured array, recarray is just variation
Out[58]: 
array([(1, 1., 1., 1., 1), (1, 1., 1., 1., 1), (1, 1., 1., 1., 1)],
      dtype=[('ID', '<i8'), ('X', '<f8'), ('Y', '<f8'), ('Z', '<f8'), ('FREQ', '<i8')])

It's quiet when you just view the fields:

In[59]: arr[['X','Y','Z']]
Out[59]: 
array([(1., 1., 1.), (1., 1., 1.), (1., 1., 1.)],
      dtype=[('X', '<f8'), ('Y', '<f8'), ('Z', '<f8')])

But it warns of a change when you try to do something with them. Note it still does the action.

In [60]: arr[['X','Y','Z']].view('float64')
/usr/local/bin/ipython3:1: FutureWarning: Numpy has detected that you may be viewing or writing to an array returned by selecting multiple fields in a structured array. 

This code may break in numpy 1.16 because this will return a view instead of a copy-- see release notes for details.
  #!/usr/bin/python3
Out[60]: array([1., 1., 1., 1., 1., 1., 1., 1., 1.])

A way to silence the warning is to add copy() after the indexing:

In [62]: arr[['X','Y','Z']].copy().view('float64')
Out[62]: array([1., 1., 1., 1., 1., 1., 1., 1., 1.])

Currently this view change works. But in the planned change, the arr[['X','Y','Z']] data layout will be different, and the view won't work. There's some complex business about offsets.