[Python-Dev] Musings on concurrency and scoping ("replacing" Javascript)

[Phillip J. Eby]

At 05:17 PM 7/6/2006 -0500, Ka-Ping Yee wrote:
>On Wed, 21 Jun 2006, Brett Cannon wrote:
> > I have been working on a design doc for restricted execution of Python
> > as part of my dissertation for getting Python into Firefox to replace
> > JavaScript on the web.
>
>I've been doing a bunch of Firefox extension programming in Javascript
>and suddenly a few of the recent topics here came together in my head
>in a silent kapow of thoughts.  This is kind of a side note to the
>security discussion, but they're all interconnected: network
>programming, concurrency, lexical scoping, security.
>
>Client-side web scripting tends to have a callback/continuation-ish
>concurrency style because it has to deal with network transactions
>(which can stall for long periods of time) in a user interface that
>is expected to stay always responsive.  The Firefox API is full of
>listeners/observers, events, and continuation-like things.  So one
>thing to consider is that, when Python is used for these purposes,
>it may be written in a specialized style.
>
>As i write JavaScript in this style i find i use nested functions
>a lot.  When i want to set up a callback that uses variables in the
>current context, the natural thing to do is to define a new function
>in the local namespace.  And if that function has to also provide a
>callback, then it has another function nested within it and so on.
>
>     function spam() {
>         var local_A = do_work();
>         do_network_transaction(
>             new function(result_1) {
>                 var local_B = do_work(result_1);
>                 do_network_transaction(
>                     new function(result_2) {
>                         do_work(local_A, local_B, result_1, result_2);
>                         ...
>                     }
>                 );
>             }
>         );
>     }
>
>So it is a possible consequence of embedding Python in Firefox that
>people will be using nested functions and lexical scoping in Python
>more often, which makes the recent discussion about access to
>enclosing scopes more significant.

As much as I'd love to have the nested scope feature, I think it's only 
right to point out that the above can be rewritten as something like this 
in Python 2.5:

     def spam():
         local_A = do_work()
         result_1 = yield do_network_transaction()
         local_B = do_work(result_1)
         result_2 = yield do_network_transaction()
         do_work(local_A, local_B, result_1, result_2)
         ...

All you need is an appropriate trampoline (possibly just a decorator) that 
takes the objects yielded by the function, and uses them up to set up 
callbacks that resume the generator with the returned result.


>This is even related to security as well.  Namespaces and lexical
>scoping are a natural and visually apparent way to limit access.
>If, for example, result_1 and result_2 in the above example are
>security-sensitive objects like secret keys or networking functions,
>you can see just by inspection that they cannot leak outside of
>spam() except by directly being passed in an outgoing function call.
>
>The standard Pythonic response to nested functions is to translate
>them into classes.  But the nested function style has two advantages:
>
>     1.  Variables are more appropriately scoped; they exist
>         only where they are meaningful.  (In a class, all the
>         variables would be mixed into one namespace, where
>         some of them would be invalid some of the time.)
>
>     2.  Local variables are private.  (Class instances don't
>         get their own private namespaces to play in.)
>
>The first becomes more significant when in a more continuation-y
>style because it helps keep the various continuations from
>interfering with each other.  The second becomes more significant
>if you care about restricting untrusted Python code.

These are all legitimate benefits of a functional style; it's just not 
necessarily the case that they also argue in favor of write access to outer 
function scopes, since this isn't necessary for writing callbacks, as in 
your original example.  Co-routines are clearly superior to nested-function 
callbacks when the intended control flow is relatively linear.  This is, if 
I recall correctly, why we added them.  :)