[Python-Dev] namedtuple implementation grumble

[Glenn Linderman]

On 6/7/2014 7:50 AM, Antoine Pitrou wrote:
> Le 07/06/2014 09:25, R. David Murray a écrit :
>> On Fri, 06 Jun 2014 19:50:57 +0100, Chris Withers 
>> <chris at simplistix.co.uk> wrote:
>>> I guess I could duck-type it based on the _fields attribute but that
>>> feels implicit and fragile.
>>>
>>> What do you guys suggest?
>>
>> I seem to remember a previous discussion that concluded that duck typing
>> based on _fields was the way to go.  (It's a public API, despite the _,
>> due to name-tuple's attribute namespacing issues.)
>
> There could be many third-party classes with a _fields member, so that 
> sounds rather fragile.
> There doesn't seem to be any technical reason barring the addition of 
> a common base class for namedtuples.
>
> Regards
>
> Antoine.

A common base class sounds like a good idea, to me, at a minimum, to 
help identify all the namedtuple derivatives.


On 6/7/2014 7:46 AM, Nick Coghlan wrote:
> On 7 June 2014 04:50, Chris Withers <chris at simplistix.co.uk> wrote:
>> Curious as to what lead to that implementation approach? What does it buy
>> that couldn't have been obtained by a mixin providing the functionality?
> In principle, you could get the equivalent of collections.namedtuple
> through dynamically constructed classes. In practice, that's actually
> easier said than done, so the fact the current implementation works
> fine for almost all purposes acts as a powerful disincentive to
> rewriting it. The current implementation is also *really* easy to
> understand, while writing out the dynamic type creation explicitly
> would likely require much deeper knowledge of the type machinery to
> follow.
I wonder if the dynamically constructed classes approach could lead to 
the same space and time efficiencies... seems like I recall there being 
a discussion of efficiency, I think primarily space efficiency, as a 
justification for the present implementation. namedtuple predates of the 
improvements in metaclasses, also, which may be a justification for the 
present implementation.

I bumped into namedtuple when I first started coding in Python, I was 
looking for _some way_, _any way_ to achieve an unmutable class with 
named members, and came across Raymond's recipe, which others have 
linked to... and learned, at the time, that he was putting it into 
Python stdlib.  I found it far from "*really* easy to understand", 
although at that point in my Python knowledge, I highly doubt a 
metaclass implementation would have been easier to understand... but 
learning metaclasses earlier than I did might have been good for my 
general understanding of Python, and more useful in the toolbox than an 
implementation like namedtuple. I did, however, find and suggest a fix 
for a bug in the namedtuple implementation that Raymond was rather 
surprised that he had missed, although I would have to pick through the 
email archives to remember now what it was, or any other details about 
it... but it was in time to get fixed before the first release of Python 
that included namedtuple, happily.

I wouldn't be opposed to someone rewriting namedtuple using metaclasses, 
to compare the implementations from an understandability and from an 
efficiency standpoint... but I don't think my metaclass skills are 
presently sufficient to make the attempt myself.

I also seem to recall that somewhere in the (lengthy) Enum discussions, 
that Enum uses a technique similar to namedtuple, again for an 
efficiency reason, even though it also uses metaclasses in its 
implementation.

I wonder if, if the reasons were well understood by someone that 
understand Python internals far better than I do, if they point out some 
capability that is missing from metaclasses that lead to these decisions 
to use string parsing and manipulation as a basis for implementing 
classes with metaclass-like behaviors, yet not use the metaclass feature 
set to achieve those behaviors.

Glenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140607/e65af8df/attachment.html>