[PYTHON MATRIX-SIG] Another way to crash the interpreter using long ints...

[Jim Hugunin]

tim@lassi.ece.uiuc.edu wrote:
> 
> I've been trying to look into it from this end, unfortunatley, my
> understanding of the python internals is somewhat (ok very)
> sketchy. For what its worth, here's what I've noticed:
> 
> 1) Long ints work most of the time on alpha2:
> 
> >>> arange(1,5l,2)
> 1L 3L
> >>> arange(1,5l)[::2]
> 1L 3L

I've figured out the source of the bug here.  I'm going to explain in
some detail for those who are interested.

1) An array of long ints is represented as an array of generic python
objects.  Elements of this array could be anything from a user-defined
class to other strange builtin types.  Operations on these arrays are
obviously going to be less efficient that on corresponding arrays of raw
C data, but they shouldn't be too bad.

2) Many C functions (take, concatenate, repeat, choose, ...) use memcpy
on portions of an array in order to copy the raw contents of one array
to a new one.

3) When you copy a pointer to a PyObject, you need to remember to INCREF
that object.  This is in contrast to every other potential type of
array.

4) I forgot to add in the PyArray_INCREF call (INCREF's all the python
objects in the array) to virtually all of the array C functions when I
rewrote them (to avoid other potential memory leaks) for 1.0a1.

I've now fixed the obvious problems for alpha3, but I'd have to admit
that remembering all of the required INCREF's and DECREF's can sometimes
be a confusing process (and tracking down the error can be even worse).

-Jim

=================
MATRIX-SIG  - SIG on Matrix Math for Python

send messages to: matrix-sig@python.org
administrivia to: matrix-sig-request@python.org
=================