The future of Python immutability

[John Nagle]

Bearophile wrote:
> John Nagle:
>> The concept here is that objects have an "owner", which is either
>> a thread or some synchronized object.   Locking is at the "owner"
>> level.  This is simple until "ownership" needs to be transferred.
>> Can this be made to work in a Pythonic way, without explicit
>> syntax?
>>
>> What we want to happen, somehow, is to
>> transfer the ownership of "words" from the calling thread to the object
>> in "putitem", and transfer it to the calling thread in "getitem".
>> How can this be done?
> 
> There are several people that have found ways to implement such owning
> of variables, for example Bartosz Milewski:
> 
> http://bartoszmilewski.wordpress.com/2009/06/02/race-free-multithreading-ownership/
> 
> It requires some extra complexities, so statically typed languages
> usually don't implement this idea, even if it avoids bugs in user
> code.
> Implementing it in Python in a simple enough way looks like a good
> topic for advanced research :-)

    I hadn't seen that implementation for D.  That's an interesting idea.
I'm certainly not the first to go down this road.  But nobody seems to have
thought hard about this for Python yet.  I looked at this years ago for C++,
but you get bogged down in the same problems that keep C++ "smart pointer"
implementations from being airtight.

    In a static language like D, it takes a lot of declarations to make this
go.  In Python, more of the ownership decisions have to be implicit, to keep
to a declaration-free Pythonic style.

    That article has a policy that "every object has one owner, for life".
That allows compile-time checking, but it's very limiting.  You can't,
for example, pass an array into a member function of a synchronized object
as a parameter and have the object keep a reference to it.  (That's what
"Queue" does. for example.) Milewski would have you make a copy in
that situation.  I felt that was too restrictive, but if you make
that restriction, it cuts down on overhead and simplifies the implementation.
The toughest part of this approach is doing a handoff from one owner to another.

    Python has the advantage that a sizable fraction of its objects, especially
the very common ones like numbers and strings, are immutable.  Immutable objects
can be shared without problems (other than storage management, but there are
ways to handle that.)  So the programmer doesn't have to obsess over "who owns
a string" when a string parameter is passed to a function.  So at least we
don't have to sweat the ownership issue for little stuff.

    It's worth looking at this for Python because CPython's current thread model
behaves terribly on multiprocessors.  We really do need something better.

					John Nagle