Design thought for callbacks

[Frank Millman]

"Steven D'Aprano" <steve+comp.lang.python at pearwood.info> wrote in message 
news:54e8af1b$0$12976$c3e8da3$5496439d at news.astraweb.com...
> Frank Millman wrote:
>
>> I tried something similar a while ago, and I did find a gotcha.
>>
>> The problem lies in this phrase - "if they are no longer alive, they are
>> automatically removed from the WeakSet, preventing me from accidentally
>> calling them when they are dead."
>>
>> I found that the reference was not removed immediately, but was waiting 
>> to
>> be garbage collected. During that window, I could call the callback, 
>> which
>> resulted in an error.
>
> I don't understand how this could possibly work. (Or fail to work, as the
> case may be.)
>
> If the callback has been garbage collected, then you cannot call it, 
> because
> you don't have any references to it and so cannot refer to it in any way.
>
> If the callback has *not* been garbage collected, then you can safely call
> it. You have a reference to the callback, therefore it exists. (If Python
> ever garbage collects an object that still has references to it, that 
> would
> be a critical bug, and you would likely get some sort of seg fault).
>
> The only thing I can think of is you have a situation where your callback
> refers to another object, B, via a weak reference. Once all the regular
> strong references to the callback and B are gone, theoretically you could
> have a race condition where the callback is waiting to be garbage 
> collected
> but B has already been garbage collected. If, in that window, you call the
> callback, *and* if the callback fails to correctly check that the weak
> reference to B still exists, then you could get a Python exception.
>
> The solution is simple: anytime you have a weak reference, you must always
> check it before you use it.
>
> Other than that, I cannot see how calling a function which has *not* yet
> been garbage collected can fail, just because the only reference still
> existing is a weak reference.
>

You are right. I tried to reproduce the problem and I can't.

Before describing what I think was happening, I want to clarify something.

Most of this thread uses the word 'callback' in the sense of an 
'asynchronous' scenario - the caller wants something to happen some time in 
the future, and then forget about it, but it is important that it does 
actually happen eventually.

That is not what I was doing, and it is not what I thought the OP was asking 
for.

"In order to inform users that certain bits of state have changed, I require 
them to register a callback with my code."

This sounds to me like a pub/sub scenario. When a 'listener' object comes 
into existence it is passed a reference to a 'controller' object that holds 
state. It wants to be informed when the state changes, so it registers a 
callback function with the controller. When the controller detects a change 
in state, it calls all the callback functions, thereby notifying each 
listener. When the listener goes out of scope, it is important that it 
deregisters with the controller.

Now back to my scenario. You are right that so long as the controller 
maintains a reference to the callback function, the listener cannot be 
garbage collected, and therefore the callback will always succeed.

As far as I can remember, I had a situation where the listener used the 
information to pass the information to a gui on a client. When the listener 
was no longer required, a close() fiunction was called which cleaned up and 
closed connections. When the callback was called after the listener was 
closed, whatever it was trying to do failed (I forget the details).

Hope this makes sense.

Frank