The Cost of Dynamism (was Re: Pyhon 2.x or 3.x, which is faster?)

[BartC]

On 13/03/2016 09:39, Steven D'Aprano wrote:
> On Sun, 13 Mar 2016 04:54 am, BartC wrote:

>> Common sense tells you it is unlikely.
>
> Perhaps your common sense is different from other people's common sense. To
> me, and many other Python programmers, it's common sense that being able to
> replace functions or methods on the fly is a useful feature worth having.

Worth having but at significant cost. But look at my jpeg test (look at 
any other similar programs); how many function names are used 
dynamically? How many functions *really* need to be dynamic?

>> (Have you tried looking at the CPython sources?

> Right. Now add *on top of that complexity* the extra complexity needed to
> manage not one, but *two* namespaces for every scope: one for "variables"
> and one for "functions and methods".

No, there's one namespace per scope. (Otherwise you could have a 
function 'f' and a variable 'f' in each scope.)

Perhaps what you mean is the number of different kinds of identifiers 
there can be. At the minute, apart from obvious, fixed, reserved words 
(I assume there are some!), there seems to be just one kind. The 
different categories (function, variable, class, built-in, module etc) 
are sorted out at *run-time*.

Some of this will just move to *compile-time*. Same amount of 
complexity, but now you do it just once at compile-time, instead of a 
billion times at run-time.

>> def f(): return "One"
>> def g(): return "Two"
>>
>> h=f

> Let me see if I can draw you a more complete picture. Suppose I have a
> function that relies on (let's say) something random or unpredictable:
>
> def get_data():
>      return time.strftime("%S:%H")

> Now I use `get_data` in another function:
>
> def spam():
>      value = get_stuff().replace(":", "")

(I assume you mean get_data here.)

> How do I test the `spam` function? I cannot easily predict what the
> `get_data` function will return.
>
> In Python, I can easily monkey-patch this for the purposes of testing, or
> debugging, by introducing a test double (think of "stunt double") to
> replace the real `get_data` function:
>
> import mymodule
> mymodule.get_data = lambda: "1:1"
> assert spam() == "Spam to the 11"

(How do you get back the original get_data?) But this looks a very 
dangerous technique. Suppose, during the test, that another function in 
mymodule, or one that imports it, needs access to the original get_data 
function to work properly? Now it will get back nonsense.

> How would you do it, when functions are constant? You would have to re-write
> the module to allow it:

There are a dozen ways of doing it. It may involve temporary renaming or 
hiding. But what you don't want is for production code to be lumbered 
with all these lookups (and having to sort out arguments, keywords and 
defaults) at runtime, just to make it a bit easier for debug code to run.

I think anyway that any Python program using dynamic functions, can be 
trivially transformed to one that uses static functions. It won't be 
pretty, but any function:

  def f(): whatever

could be rewritten as:

  def __f(): whatever
  f = __f()

But now the static name __f is available for direct use, and can be 
depended on not to change.

(Perhaps such a convention can be used anyway. A functions that starts 
with "__" or uses some other device, the compiler and runtime will know 
it will always be that function, and could allow some optimisations.)

(It's not quite so trivial for import module names, as you can't really 
just rename all modules! But in theory it could be done.)

> I once monkey-patched the `len` built-in so I could monitor the progress of
> a long-running piece of code that wasn't written to give any feedback.

These ought to be tricks that you do with source code. It shouldn't be 
necessary for an implementation to allow that.

(But doesn't len() already have a mechanism where you can override it 
anyway?)

-- 
Bartc