Different execution time in python code between embedded or standalone

[Pau Freixes]

Hi Gabriel,

The source code for embedded mode it's the same, look this code :

_nrdigest = 0
_const_b = 20
_f = None
_signal = False


def handler_alrm(signum, frame):
    global _signal
    global _nrdigest
    global _f


    _signal = True

def try_me():
    global _nrdigest
    global _f
    global _signal

    _f = open("/dev/urandom","r")
    while _signal is not True:
        buff = _f.read(_const_b)
        md5.md5(buff).hexdigest()
        _nrdigest = _nrdigest + 1

    if _f is not None :
        _f.close()

# Define entry point with one input variable
# req is a instance of Request object, usefull members of this object are:
# req.input is a dictionary with input.xml variables
# req.constants is a dictionary with constants defined into signature.xml
# req.output is void dictionary for full with output variables
# req.config is a dictionary with config values take from namespace
# req.apdn_pid is a pid of aplication


def main( req ):
    global _nrdigest


    signal.signal(signal.SIGALRM, handler_alrm)
    signal.alarm(req.input['time'])


    try_me()

    req.output['count'] = _nrdigest

    return req.OK


if __name__ == "__main__":

    # test code
    class test_req:
        pass

    req = test_req()
    req.input = { 'time' : 10 }
    req.output = { 'ret' : 0, 'count' : 0 }
    req.OK = 1

    main(req)

    print "Reached %d digests" % req.output['count']

When this code it's run into embeded mode the main function it's called
automatically form  one  c code with PyObject_CallObject function, and
Request object it's build into c code. When you run this code into
"standalone" or tradicional method with bash, the main function it's called
from "main(req)" call.

I know the cost of search for global variable it's more bigger than local
variable, but this cost it's the same for emedded mode and standalone mode
:), and the same for read urandom. And read urandom don't have a lock
implicit - the const_b is 20 bytes.

My problem it's that don't understand because the same program run into
embedded mode or in bash mode have a different results !!! what do you think
about this ? may be a GIL ?


PD: no seguro que no llego :), a ver si alguien mas se apunta a la discusion
!

On Wed, Jun 4, 2008 at 4:45 AM, Gabriel Genellina <gagsl-py2 at yahoo.com.ar>
wrote:

> En Tue, 03 Jun 2008 16:58:12 -0300, Pau Freixes <pfreixes at milnou.net>
> escribió:
>
>
>  Hi list,
>>
>> First Hello to all, this is my and hope not end message to the list :P
>>
>> This last months I have been writting a  program in c like to mod_python
>> for
>> embedding python language, it's a middleware for dispatch and execute
>> python
>> batch programs into several nodes. Now I'm writing some python program for
>> test how scale this into several nodes and comparing with "standalone"
>> performance.
>>
>> I found a very strange problem with one application named md5challenge,
>> this
>> aplication try to calculate the max number md5 digest in several seconds,
>> md5challenge use a simple signal alarm for stop program when time has
>> passed. This is the code of python script
>>
>> def handler_alrm(signum, frame):
>>    global _signal
>>    global _nrdigest
>>    global _f
>>
>>
>>    _signal = True
>>
>> def try_me():
>>    global _nrdigest
>>    global _f
>>    global _signal
>>
>>    _f = open("/dev/urandom","r")
>>    while _signal is not True:
>>        buff = _f.read(_const_b)
>>        md5.md5(buff).hexdigest()
>>        _nrdigest = _nrdigest + 1
>>
>>    if _f is not None :
>>        _f.close()
>>
>> def main( req ):
>>    global _nrdigest
>>
>>
>>    signal.signal(signal.SIGALRM, handler_alrm)
>>    signal.alarm(req.input['time'])
>>
>>
>>    try_me()
>>
>>    req.output['count'] = _nrdigest
>>
>>    return req.OK
>>
>>
>> if __name__ == "__main__":
>>
>>    # test code
>>    class test_req:
>>        pass
>>
>>    req = test_req()
>>    req.input = { 'time' : 10 }
>>    req.output = { 'ret' : 0, 'count' : 0 }
>>    req.OK = 1
>>
>>    main(req)
>>
>>    print "Reached %d digests" % req.output['count']
>>
>>
>> When I try to run this program in standalone into my Pentium Dual Core
>> md4challenge reached 1.000.000 milion keys in 10 seconds but when i try to
>> run this in embedded mode md5challenge reached about 200.000 more keys !!!
>> I
>> repeat this test many times and  always  wins  embedded mode  !!!  What's
>> happen ?
>>
>> Also I tested to erase read dependencies from /dev/random, and calculate
>> all
>> keys from same buffer. In this case embedded mode win always also, and the
>> difference are more bigger !!!
>>
>> Thks to all, can anybody help to me ?
>>
>
> So the above code corresponds to the standalone version - what about the
> embedded version? Are you sure it is exactly the *same* code? All those
> global statements are suspicious, and you don't even need most of them. Note
> that looking up a name in the global namespace is much slower than using a
> local name.
> Also, you're including the time it takes the OS to *generate* several
> megabytes of random data from /dev/urandom (how big is _const_b?).
> Usually it's easier (and more accurate) to measure the time it takes to
> compute a long task (let's say, how much time it takes to compute 1000000
> md5 values). You're doing it backwards instead.
> I'd rewrite the test as:
>
> def try_me():
>  from md5 import md5
>  buff = os.urandom(_const_b)
>  for i in xrange(1000000):
>    md5(buff).hexdigest()
>
> def main(req):
>  t0 = time.clock()
>  try_me()
>  t1 = time.clock()
>  # elapsed time = t1-t0
>
>
> PS: Recuerdo que respondí esto en la lista de Python en castellano, pero
> ahora veo que mi mensaje nunca llegó :(
>
> --
> Gabriel Genellina
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>



-- 
Pau Freixes
Linux GNU/User
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20080604/be2979a4/attachment-0001.html>