[Python-Dev] Impact of Namedtuple on startup time

[Raymond Hettinger]

> On Jul 17, 2017, at 8:49 AM, Guido van Rossum <guido at python.org> wrote:
> 
>  The approach of generating source code and exec()ing it, is a cool demonstration of Python's expressive power, but it's always been my sense that whenever we encounter a popular idiom that uses exec() and eval(), we should augment the language (or the builtins) to avoid these calls -- that's for example how we ended up with getattr().

FYI, the proposal (from Jelle) isn't to remove exec.  It is to only exec a smaller piece of code and make the rest of it static.  

It isn't bad idea, it just complicates the implementation (generating _source lazily) and the subsequence maintenance (which is currently really easy).

> Concluding, I think we should move on from the original implementation and optimize the heck out of namedtuple. The original has served us well. The world is constantly changing. Python should adapt to the (happy) fact that it's being used for systems larger than any of us could imagine 15 years ago.

Okay, then Nick and I are overruled.  I'll move Jelle's patch forward.  We'll also need to lazily generate _source but I don't think that will be hard.

One minor grumble:  I think we need to give careful cost/benefit considerations to optimizations that complicate the implementation.  Over the last several years, the source for Python has grown increasingly complicated.  Fewer people understand it now. It is much harder to newcomers to on-ramp.  The old-timers (myself included) find that their knowledge is out of date.  And complexity leads to bugs (the C optimization of random number seeding caused a major bug in the 3.6.0 release; the C optimization of the lru_cache resulted in multiple releases having a hard to find threading bugs, etc.).  It is becoming increasingly difficult to look at code and tell whether it is correct (I still don't fully understand the implications of the recursive constant folding in the peephole optimizer for example).    In the case of this named tuple proposal, the complexity is manageable, but the overall trend isn't good and I get the feeling the aggressive optimization is causing us to forget key parts of the zen-of-python.

Cheers,


Raymond


P.S.  Ironically, a lot of my consulting work comes from people who have created something complex our of something that could have been simple.  So, I in a strange way, I should be happy about these trends -- just saying ;-)