Magic Optimisation

[Bengt Richter]

On 5 Sep 2005 07:27:41 -0700, "Paul McGuire" <ptmcg at austin.rr.com> wrote:

>I still think there are savings to be had by looping inside the
>try-except block, which avoids many setup/teardown exception handling
>steps.  This is not so pretty in another way (repeated while on
>check()), but I would be interested in your timings w.r.t. your current
>code.
>
>    def loop(self):
>        self_pool_popleft = self.pool.popleft
>        self_pool_append = self.pool.append
>        self_call_exit_funcs = self.call_exit_funcs
>        check = self.pool.__len__
>        while check() > 0:
>            try:
>                while check() > 0:
>                    task = self_pool_popleft()
>                    task.next()
>                    self_pool_append(task)
>            except StopIteration:
>                self_call_exit_funcs(task) 
>

Why not let popleft trigger an exception out of while True instead,
and prevent tasks from raising StopIteration, and let them yield a None
to indicate keep scheduling with no special action, and something else
for optional differentiation of various exit options, e.g., zero for die,
and nonzero for suspension waiting for event(s) E.g., a returned integer
could be an event mask or single index (+ vs -) for thing(s) to wait for.
If you work things right, event check in the loop can be an if like
if waitedfor&events: process_events(),  which most of the time is a fast no-op).

Then (without event stuff, and untested ;-) maybe something like:

     def loop(self):
         self_pool_popleft = self.pool.popleft
         self_pool_append = self.pool.append
         self_call_exit_funcs = self.call_exit_funcs
         try:
             while True:
                 task = self_pool_popleft()
                 if task.next() is None:
                     self_call_exit_funcs(task) 
                 else:    
                     self_pool_append(task)
         except Indexerror:
             pass

You could even consider putting the bound task.next methods in
the deque instead of the task, and using deque rotation instead
of popping and appending. Then, if you put the task.next's in
reverse order in the deque to start with, self_pool[-1] will be
the first, and self_pool_rotate() will bring the next into position.
Which would make it look like (untested!):

     def loop(self):
         self_pool_pop = self.pool.pop
         self_call_exit_funcs = self.call_exit_funcs
         self_pool_rotate = self.pool.rotate
         try:
             while True:
                 if self.pool[-1]() is None:
                     self_call_exit_funcs(self_pool_pop())
                 else:    
                     self_pool_rotate()
         except Indexerror:
             pass

IWT if the pool remains unchanged most of the time, pool_rotate() ought
to be faster than popping and appending. Note that exit_funcs will need
a mapping of task.next -> task most likely, unless communication is
entirely via a mutable task state object, and all that's needed is to
me map to that and mess with exit state there and trigger a final .next()
to wrap up the generator.

Regards,
Bengt Richter