[Numpy-discussion] List with numpy semantics

[josef.pktd at gmail.com]

On Tue, Nov 2, 2010 at 10:21 PM,  <josef.pktd at gmail.com> wrote:
> On Tue, Nov 2, 2010 at 10:02 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> Gerrit Holl <gerrit.holl at gmail.com> writes:
>>> On 31 October 2010 17:10, Nikolaus Rath <Nikolaus at rath.org> wrote:
>>>> Hello,
>>>>
>>>> I have a couple of numpy arrays which belong together. Unfortunately
>>>> they have different dimensions, so I can't bundle them into a higher
>>>> dimensional array.
>>>>
>>>> My solution was to put them into a Python list instead. But
>>>> unfortunately this makes it impossible to use any ufuncs.
>>>>
>>>> Has someone else encountered a similar problem and found a nice
>>>> solution? Something like a numpy list maybe?
>>>
>>> You could try a record array with a clever dtype, maybe?
>>
>> It seems that this requires more cleverness than I have... Could you
>> give me an example? How do I replace l in the following code with a
>> record array?
>>
>> l = list()
>> l.append(np.arange(3))
>> l.append(np.arange(42))
>> l.append(np.arange(9))
>>
>> for i in range(len(l)):
>>   l[i] += 32
>
> Depending on how you want to use it, it might be more convenient to
> use masked arrays or fill with nan (like pandas and larry) to get a
> rectangular array. it might be more convenient for some things, but if
> the sizes differ a lot then it might not be more efficient.

another option I sometimes use (e.g. for unbalanced panel data), is to
just stack them on top of each other into one long 1d array, and keep
track which is which, e.g. keeping the (start-end) indices or using an
indicator array. For example, with an integer label array np.bincount
is very fast to work with it.
This is mainly an advantage if there are many short arrays and many
operations have to applied to all of them.

Josef

> Josef
>
>>
>> Thanks,
>>
>>   -Nikolaus
>>
>> --
>>  »Time flies like an arrow, fruit flies like a Banana.«
>>
>>  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>