Negative array indicies and slice()

[Andrew Robinson]

Hi Ian,

There are several interesting/thoughtful things you have written.
I like the way you consider a problem before knee jerk answering.

The copying you mention (or realloc) doesn't re-copy the objects on the 
list.
It merely re-copies the pointer list to those objects. So lets see what 
it would do...

I have seen doubling as the supposed re-alloc method, but I'll assume 
1.25 --
so, 1.25**x = 20million, is 76 copies (max).

The final memory copy would leave about a 30MB hole.
And my version of Python operates initially with a 7MB virtual footprint.

Sooo.... If the garbage collection didn't operate at all, the copying 
would waste around:

 >>> z,w = 30e6,0
 >>> while (z>1): w,z = w+z, z/1.25
...
 >>> print(w)
149999995.8589521

eg: 150MB cummulative.
The doubles would amount to 320Megs max.

Not enough to fill virtual memory up; nor even cause a swap on a 2GB 
memory machine.
It can hold everything in memory at once.

So, I don't think Python's memory management is the heart of the problem,
although memory wise-- it does require copying around 50% of the data.

As an implementation issue, though, the large linear array may cause 
wasteful caching/swapping loops, esp, on smaller machines.

On 10/29/2012 10:27 AM, Ian Kelly wrote:
> Yes, I misconstrued your question.  I thought you wanted to change the
> behavior of slicing to wrap around the end when start>  stop instead
> of returning an empty sequence. ...  Chris has
> already given ...  You
> could also use map for this:
>
> new_seq = list(map(old_seq.__getitem__, iterable))
MMM... interesting.

I am not against changing the behavior, but I do want solutions like you 
are offering.
As I am going to implement a python interpreter, in C,  being able to do 
things differently could significantly reduce the interpreter's size.

However, I want to break existing scripts very seldom...

> I'm aware of what is possible in C with pointer arithmetic. This is 
> Python, though, and Python by design has neither pointers nor pointer 
> arithmetic. In any case, initializing the pointer to the end of the 
> array would still not do what you want, since the positive indices 
> would then extend past the end of the array. 

Yes, *and* if you have done assembly language programming -- you know 
that testing for sign is a trivial operation.  It doesn't even require a 
subtraction.  Hence, at the most basic machine level -- changing the 
base pointer *once* during a slice operation is going to be far more 
efficient than performing multiple subtractions from the end of an 
array, as the Python API defines.
I'll leave out further gory details... but it is a Python interpreter 
built in "C" issue.