Is Ndarray Faster Than Recarray Access?

I was able to copy my recarray data to a ndarray, do some calculations and return the ndarray with updated values. Then, I discovered the append_fields() capability in numpy.lib.

Solution 1:

Updated 15-November-2018 I expanded my timing tests to clarify differences in performance for ndarray, structured array, recarray and masked array (type of record array?). There are subtle differences in each. See discussion here: numpy-discussion:structured-arrays-recarrays-and-record-arrays

Here are result of my performance tests. I built a very simple example (using 1 of my HDF5 data sets) to compare performance with the same data stored in 4 types of arrays: ndarray, structured array, recarray and masked array. After the arrays are constructed, they are passed to a function that simply loops thru each row and extracts 12 values from each row. The functions are called from the timeit function with a single pass (number=1). This test only measures the array read function, and avoids all other calculations. Results given below for 9,000 rows:

forndarray: 0.034137165047070615forstructured array: 0.1306827116913577forrecarray: 0.446010040784266formasked array: 31.33269560998199

Based on this test, access performance decreases with each type. Access times for structured array and recarray are 4x-13x slower than ndarray access (but all are only a fraction of second). However, ndarray access is 1000x faster than masked array access. That explains the seconds to minutes difference I see in my complete example. Hopefully this data is useful to others that encounter this issue.