Jython, GILs and object locking.

[Jon Franz_antispam]

Daniel wrote:
> A test would still be needed in the ref counting macros to check for 
> global/thread local. I don't know the overhead for this, but as there 
<<SNIP>>
> But removing the object from the list wouldn't make it local again as 
> there could be other global objects pointing to it.
> 
> I think "painful to implement" doesn't even begin to describe it.

All the reasons given above are why I said it would be painful - it
just appears we have different definitions of painful.  I don't use it 
lightly.

Jumping over to another part of this conversation....

Harri Pesonen wrote:
> You are probably asking next "how threads can communicate". They can 
> communicate with messages, and via special shared interpreter state, 
> which needs a new "shared" keyword in python. Access to these shared 
> variables is synchronized, all other variables are local to thread, and 
> they need not be synchronized

Unless these shared variables had different semantics than all 
other python variables, that is, assignment by value, not reference, 
then shared variables can have the same issues as described above 
by Daniel - assignment to/from local variables will complicate 
matters greatly.

..................
But the message passing scheme brings to mind a new idea
for a different take on reference counting, once again, painful,
this time especially so.  Remember that I'm throwing ideas out
as I have them, and I'm not edorsing the ideas, just bringing them
up.

Move reference counting to its own thread.  Py_INCREF and 
Py_DECREF would send messages to this thread, thus avoiding
locking.
The reference counting thread would then be responsible for all
GC, and timing of GC for an object would never be guaranteed -
it could be collected almost immediately, or in 2 seconds when the
ref-count thread has worked its way through its message queue
and 'gets to' the final DECREF message.  Internally, the GC thread
(that fits more than calling it a ref-count thread) would need to
keep track of which thread the references were held for - thus
thread-exit would trigger a DECREF for all outstanding refs held 
by a thread.  Storing the ownership of a reference internally to
the GC would seem preferable to storing it thread-local,
since an abnormal thread-exit could prevent DECREF messages
from being delivered by a thread on exit.

Pros:
-  the GC thread can continue in the 
background while the rest of the process exits.  This doesn't 
apply to most users, but for programs will large amounts of 
objects (>1000000), the time it takes to tear-down a python 
process can be excessive, and leave you waiting for a command 
prompt (if you happen to have launched the process from one.).
(I can provide an example script for anyone running on a large 
memory (>4GB) system for those curious about this 
phenomenon.)

- no locking for ref-counting

Cons:
- Ugly, very painful to implement.

- It is unclear what thread-context del methods on an object
should be considered for and DECREFs they trigger.  
Possibly the thread that created the object?

- True benefit (speed wise) unknown since impact of extra 
thread and message passing is unknown.

~Jon Franz