[Python-ideas] namedtuple is not as good as it should be

[Christian Tismer]

Hi Raymond,

On 10.06.13 01:54, Raymond Hettinger wrote:
>
> On Jun 9, 2013, at 5:55 PM, Christian Tismer <tismer at stackless.com 
> <mailto:tismer at stackless.com>> wrote:
>
>> Without being very explicit, namedtuple makes you use it happily 
>> instead of tuples.
>> But instead of using a native, simple type, you now use a not-so-simple,
>> user-defined type.
>> This type
>>
>> - is not built in
>> - has a class definition
>> - needs a global, constant definition _somewhrere_
>>
>> Typically, you run a script interactively, and typically you need to 
>> pickle
>> some __main__.somename namedtuple class instances.
>
> Does you whole issue boil down to this?  If you interactively define
> any function, class, named tuple, or enumeration, the pickle module
> will only store its name, not its definition.   Then the unpickler
> can't find it later.
>

No, this is just part of it, and just distracting a bit:

If you start to implement things, namedtuple is great.
When you implement your structure definitions with namedtuple,
all good.
But when you continue and create tables, which are typically not having
a static class definition and you come from the dynamic of tuples,
the problem suddenly becomes very visible.

The whole issue is that a dynamically usable structure like tuple suddenly
becomes statically defined, and you need to cope with that.

The problem is not per se:
Sure I know how pickling works and what I have to do to pickle anything.
But names for tuples is such a small thing that is not worth to need to
dig into that whole pickling issue, because all the reasons why pickling
of classes is done by module lookup do not apply here, at least not
for a native named tuple:

I don't need to subclass and invent fancy methods in the first place.
When I derive a class from a tuple, well ok. I asked for it, and I need
to cope with it.
But namedtuple as a tool pretty much pretends to be almost a builtin
and therefore should not introduce the pickling problem, which is not
there when you use tuples instead.

> The essential problem being that data gets stored separately
> from its definition (similar to the relationship
> between a database and its ORM models).
>
> I don't know if it will help in your case, but one thing I've seen done
> it is store the definitions and data together.   The developments
> in Python 3 may help in this regard.  In Py3.3, you can access
> (and possibly store) the definition using the _source attribute.
> And in Py3.4,  your will be able to specify the module
> (see http://bugs.python.org/issue17941 ).
>

I was actually reading about that discussion of the separation issue,
the enum discussion as well, and that brought me back to the tuples.
Although I use Py3.3, the _source attribute was not obvious, yet.
Actually I hoped for the Py3.4 solution of http://bugs.python.org/issue17941

It also related to the sys._getframe() issue that needs to live on for a 
while
because of the implied need to take care of __module__ in enum and 
namedtuple.

I think that we need to go this route at all is a fundamental misconception
and goes into the wrong direction.
I think defining constants like namedtuple or enum is not a matter for 
classes,
but more like a _prototype_, conceptually.
This prototype needs to live somewhere to be used as a blueprint.
Reconstruction should not need to require a static class definition,
but a simple prototype tuple, which can be pickled.

(this is not solving the problem, but just trying to change how we think
  of it.)

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20130610/2c622f70/attachment-0001.html>