[Numpy-discussion] re cfunctions help with concatenating (vstack, hstack, etc.)

[Robert Kern]

On Mon, Dec 7, 2009 at 18:00, John [H2O] <washakie at gmail.com> wrote:
>
> Hello (Pierre?),
>
> I'm trying to work more with structured arrays, which at times seems great,
> and at others (due to my lack of familiarity) very frustrating.
>
> Anyway, right now I'm writing a bit of code to read a series of files with
> x,y,z data. I'm creating record arrays for each file a read. Once I have
> them all read, I just want to load them into one big array. But it doesn't
> seem as straight forward as concatenation:
>
> In [102]: type(D); D.shape; D.dtype.names
> Out[102]: <class 'numpy.core.records.recarray'>
> Out[102]: (3025,)
> Out[102]: ('datetime', 'lon', 'lat', 'elv', 'co')
>
> In [103]: type(D2); D2.shape; D2.dtype.names
> Out[103]: <class 'numpy.core.records.recarray'>
> Out[103]: (3445,)
> Out[103]: ('datetime', 'lon', 'lat', 'elv', 'co')
>
> In [104]: C = rf.stack_arrays((D,D2))
>
> In [105]: type(C); C.shape; C.dtype.names
> Out[105]: <class 'numpy.ma.core.MaskedArray'>
> Out[105]: (6470,)
> Out[105]: ('datetime', 'lon', 'lat', 'elv', 'co')
>
> In [106]: C.datetime
> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
>
> /xnilu_wrk/flex_wrk/jfb/RESEARCH_ARCTIC/Arctic_CO/<ipython console> in
> <module>()
>
> AttributeError: 'MaskedArray' object has no attribute 'datetime'
>
> In [107]: C[0]
> Out[107]: (datetime.datetime(2008, 6, 29, 14, 50, tzinfo=<UTC>),
> 248.83900164808117, 53.949661856137709, -0.31834712473582627,
> 112.91218844292422)
>
> In [108]:
>
> So it seems I end up with a masked array of the correct length, but it is an
> array of tuples and no longer a record array. Am I missing a step?

It's still a structured array (albeit a masked one); however, it is no
longer a recarray. recarray is just a convenience class that provide
attribute access. Use C['datetime'] to access the datetime column.
Out[107] is not a tuple but a record scalar. Look at the docstring of
numpy.lib.recfunctions.stack_arrays() to see the asrecarray option
which will make the output a recarray (or the MaskedRecords subclass
if you retain the default usemask=True option).

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco