Negative array indicies and slice()

[Chris Kaynor]

On Mon, Oct 29, 2012 at 11:00 AM, Andrew Robinson
<andrew3 at r3dsolutions.com> wrote:
>
> Let's look at the source code rather than the web notes -- the source must
> be the true answer anyhow.
>
> I downloaded the source code for python 3.3.0, as the tbz;
> In the directory "Python-3.3.0/Python", look at Python-ast.c, line 2089 &
> ff.
>
> Clearly a slice is malloced for a slice_ty type.
> It has four elements: kind, lower, upper, and step.
>
> So, tracing it back to the struct definition...
>
> "Include/Python-ast.h"  has "typedef struct _slice *slice_ty;"
>
> And, here's the answer!:
>
> enum _slice_kind {Slice_kind=1, ExtSlice_kind=2, Index_kind=3};
> struct _slice {
>         enum _slice_kind kind;
>         union {
>                 struct {
>                         expr_ty lower;
>                         expr_ty upper;
>                         expr_ty step;
>                 } Slice;
>
>                 struct {
>                         asdl_seq *dims;
>                 } ExtSlice;
>
>                 struct {
>                         expr_ty value;
>                 } Index;
>
>         } v;
> };
>
>
> So, slice() does indeed have arbitrary python types included in it; contrary
> to what I read elsewhere.
> expr_ty is a pointer to an arbitrary expression, so the actual structure is
> 4 pointers, at 32 bits each = 16 bytes.
> The size of the structure itself, given in an earlier post, is 20 bytes --
> which means one more pointer is involved, perhaps the one pointing to the
> slice structure itself.
>
> Hmm...!
>
> An empty tuple gives sys.getsizeof( () ) = 24.
>
> But, I would expect a tuple to be merely a list of object pointers; hence I
> would expect 4 bytes for len(), and then a head pointer 4 bytes, and then a
> pointer for each object.
> 3 objects gives 12 bytes, + 8 = 16 bytes.
>
> Then we need one more pointer so Python knows where the struct is...
> So a Tuple of 3 objects ought to fit nicely into 20 bytes; the same size as
> slice() --
>
> but it's 24, even when empty...
> And 36 when initialized...
> What are the extra 16 bytes for?

Every Python object requires two pieces of data, both of which are
pointer-sized (one is a pointer, one is an int the size of a pointer).
These are: a pointer to the object's type, and the object's reference
count.

A tuple actually does not need a head pointer: the head pointer is
merely an offset from the tuple's pointer. It merely has a ref count,
type, an item count, and pointers to its contents.

A slice has the same type pointer and reference count, then three
pointers to the start, stop, and step objects. This means a slice
object should be the same size as a two-item tuple: the tuple needs a
count, while that is fixed at 3 for a slice (though some items may be
unset).

NOTE: The above is taken from reading the source code for Python 2.6.
For some odd reason, I am getting that an empty tuple consists of 6
pointer-sized objects (48 bytes on x64), rather than the expected 3
pointer-sized (24 bytes on x64). Slices are showing up as the expected
5 pointer-sized (40 bytes on x64), and tuples grow at the expected 1
pointer (8 bytes on x64) per item. I imagine I am missing something,
but cannot figure out what that would be.

>
> All I see is:
> typedef struct { object** whatever } PyTupleObject;
>