A Twisted Design Decision

[Chris Mellon]

On Wed, Jan 28, 2009 at 10:05 AM, koranthala <koranthala at gmail.com> wrote:
> On Jan 28, 8:36 pm, Jean-Paul Calderone <exar... at divmod.com> wrote:
>> On Wed, 28 Jan 2009 06:30:32 -0800 (PST), koranthala <koranth... at gmail.com> wrote:
>> >On Jan 28, 7:10 pm, Jean-Paul Calderone <exar... at divmod.com> wrote:
>> >> On Wed, 28 Jan 2009 02:02:57 -0800 (PST), koranthala <koranth... at gmail.com> wrote:
>> >> >On Jan 27, 9:27 pm, koranthala <koranth... at gmail.com> wrote:
>> >> >> On Jan 27, 6:57 pm, Jean-Paul Calderone <exar... at divmod.com> wrote:
>> >> > [snip]
>>
>> >> >> Thank you Jean-Paul.
>> >> >> My code is more complex than what I have mentioned. When I mentioned
>> >> >> msg.send, the msg object actually gets the data from DB etc to send.
>> >> >> And there are many other items being done.
>> >> >> I will try to see whether I can change the code to incorporate what
>> >> >> you mentioned.
>>
>> >> >> I rewrote most of my code after learning just raw deferreds - I had
>> >> >> planned to study inlineCallbacks - but then it slipped my mind  - now
>> >> >> it has come to bit me. :-(
>>
>> >> >Hi,
>> >> >  I tried to update the code as per the suggestion, but to no avail.
>> >> >  My system uses Python2.4.3 (cannot move to 2.5) - so I tried to
>> >> >rewrite with deferredGenerators - since I thought inlineCallbacks are
>> >> >similar to deferredGenerators.
>>
>> >> >  But I cannot seem to rewrite it in a format where the functional
>> >> >encapsulation is not broken.
>> >> >  i.e. as I mentioned in the first example - I have to pass SELF to
>> >> >child objects for them to modify it.
>>
>> >> Why?  You don't do this in the original version of your code.  Why do
>> >> it after switching to Twisted (particularly since you seem to want *not*
>> >> to)?
>>
>> >> Jean-Paul
>>
>> >Without that, I am unable to increment success and failure counters
>> >which are part of the message handler object.
>> >In the original version, if send fails the return value of
>> >protocol.send is propogated back to msg.send and to msg handler.send.
>> >But in twisted, it is not so. So, I have to pass in SELF through to
>> >increment success and failure counter.
>> >Is it possible otherwise in twisted?
>>
>> Why isn't the return value of protocol.send propagated back to msg.send?
>> It sounds like it should be.
>>
>> Jean-Paul
>
> Thank you very much again Jean-Paul for helping me out.
> I am unable to understand how I will be able to propogate the return
> value of protocol.send to msg.send.
> Maybe I am being foolish - but my understanding is as follows.
>
> In a non-reactor pattern scenario:
> msg_handler.send_message calls msg.send which inturn calls
> protocol.send.
> So, the reply to protocol.send actually goes up the stack till
> msg_handler.send_message wherein I can increment/decrement success/
> failure counter.
>
> In reactor pattern:
> msg_handler.send_message calls msg.send which call protocol.send which
> causes a deferred to be created.
> Now, when the deferred finishes its work, reactor calls the callback
> associated - but the original context (stack etc) is lost.
> Now, the only mechanism of interaction is via the parameters passed in
> the callback.
> This means that msg_handler has to pass in its object to msg.send
> which inturn has to send either msg_handler or self to protocol.send
> so that it is stored in the parameter to the callback.
> When callback is hit, I use this parameter to call methods in each
> object.
>
> This is what I was trying to say in my first mail that - Twisted,
> being twisted in its behavior is causing quite a lot of
> confusion in design decisions - because now I have to break functional
> encapsulation - by asking lower layer objects to handler upper layer
> objects behaviors.
>
> As I said earlier, maybe I am being completely stupid -  there might
> be a very easy and obvious solution. But I cannot seem to get it at
> all.
>

I'm sure you're not completely stupid, but there is an easy and
obvious solution - you should be returning the *deferred* up the
callstack. That will allow the MessageHandler to attach
callbacks/errbacks to it, without needing to pass self all the way
down the call stack and have a proxy object manage it.