[py-dev] new Channel callback

[Jan Balster]

holger krekel wrote:
> On Thu, Apr 14, 2005 at 17:41 +0200, Jan Balster wrote:
> 
>>holger krekel wrote:
>>
>>>On Thu, Apr 14, 2005 at 16:18 +0200, Jan Balster wrote:
>>>
>>>>if I use the new channel.setcallback I still have to call gateway.exit()
>>>>at the end. So I need a thread to wait for the remote side to finish.
>>>>
>>>>Is there a way to avoid this? Calling gateway.exit() form the callback
>>>>would be fine for me, but I'm afraid that won't work.
>>>
>>>
>>>Maybe that should work, although the gateway/channel shutdown-process 
>>>is rather involved so it might actually not work.  It's worth
>>>just testing it out, i'd say. 
>>
>>Hurray! It works. But it seems to me, using the callback is slower than
>>receiving all items in a loop (13 secs vs. 15 secs).
> 
> 
> That would be very odd.  Do you repeatedly get this? 
> Btw, i'd recommend to add a test for exiting from a callback
> because i am currently revising the threading mechanism :-) 
Ok.

I did some benchmarking. Callback is faster than Loop, except for small
numbers of received items.

Jan

benchmark output:
Receiving 10000 elements
use_callback:   3.702
use_loop:       4.522
******************** The winner is Callback

Receiving 2000 elements
use_callback:   0.919
use_loop:       1.175
******************** The winner is Callback

Receiving 1000 elements
use_callback:   0.573
use_loop:       0.588
******************** The winner is Callback

Receiving 100 elements
use_callback:   0.270
use_loop:       0.250
******************** The winner is Loop

Receiving 50 elements
use_callback:   0.274
use_loop:       0.235
******************** The winner is Loop

Receiving 10 elements
use_callback:   0.230
use_loop:       0.219
******************** The winner is Loop
close failed: [Errno 9] Bad file descriptor


program:
import py

remote_code = '''
import sha
number = channel.receive()
for i in range(number):
    s = sha.new(str(i)).hexdigest() * 10
    channel.send(s)
'''

def use_callback(number):
    gw = py.execnet.PopenGateway(py.std.sys.executable)
    channel = gw.newchannel()
    l = []
    channel.setcallback(l.append)
    channel.remote_exec(remote_code)
    channel.send(number)
    channel.waitclose(100.0)
    channel.close()
    channel.gateway.exit()

def use_loop(number):
    gw = py.execnet.PopenGateway(py.std.sys.executable)
    channel = gw.newchannel()
    l = []
    channel.remote_exec(remote_code)
    channel.send(number)
    try:
        while 1:
            l.append(channel.receive())
    except EOFError:
        pass
    channel.waitclose(100.0)
    channel.close()
    channel.gateway.exit()


def test(number):
    print
    print 'Receiving', number, 'elements'
    t1 = py.std.time.time()
    use_callback(number)
    t1 = py.std.time.time() - t1
    print 'use_callback: \t%0.3f' % t1
    t2 = py.std.time.time()
    use_loop(number)
    t2 = py.std.time.time() - t2
    print 'use_loop:  \t%0.3f' % t2
    print '*'* 20, 'The winner is',
    if t1 < t2:
        print 'Callback'
    else:
        print 'Loop'


if __name__ == '__main__':
    for i in (10000, 2000, 1000, 100, 50, 10):
        test(i)