[Python-Dev] Meta-reflections
Kevin Jacobs
jacobs@penguin.theopalgroup.com
Wed, 20 Feb 2002 08:30:05 -0500 (EST)
Paul, thanks for the very constructive feedback!
On Wed, 20 Feb 2002, Moore, Paul wrote:
> > I fully intend to provide a reference implementation of
> > some of these ideas. In fact, its likely to be a fairly
> > small patch. However, I still don't know what the ideal
> > semantics are. I would very much value your input on the
> > matter, even if on a purely theoretical level. So, lets
> > start with the premise that __attrs__ is a declaration like
> > __slots__ except that:
>
> It seems relevant to me that your choice of name ("attrs") indicates a
> relationship with attributes - which your ideas seem to deny.
You are right -- lets call them 'slotattrs', since they should ideally have
virtually the same semantics as attributes, except they are allocated like
slots.
> > 1) the namespace of these 'attrs' is flat -- repeated
> > names in descendant classes either results in an error or
> > silently re-using the existing slot. This maintains the
> > traditional flat instance namespace of attributes.
>
> FWIW, I disagree with this completely. I would expect slots with a
> particular name in derived classes to hide the same-named slots in base
> classes. Whether or not the base class slot is available via some sort of
> "super" shenannigans is less relevant. But hiding semantics is critical. How
> do you expect to reliably make it possible to derive from a class with slots
> otherwise? Perl gets into this sort of mess with its implementation of
> objects.
Attributes currently have a flat namespace, and the construct that I feel is
most natural would maintain that characteristic. e.g.:
class Base:
def __init__(self):
self.foo = 1
class Derived(Base):
def __init__(self):
Base.__init__(self)
self.foo = 2 # this is the same foo as in Base
Python already implements a form data hiding semantics in a different way,
so I'm not sure it is a good idea to add another ad-hoc method to do the
same thing. The current way to implement data hiding is by using the
namespace munging trick by prefixing symbols with '__', which is then munged
by prepending an underscore and the name of the class.
class Foo:
__var = 1
dir(Foo)
> ['_Foo__var', '__doc__', '__module__']
> > 2) A complete and immutable list of slots is available
> > as a member of each type to allow for easy and efficient
> > reflection. (though I am also in favor of working on
> > better formal reflection APIs)
>
> Agreed - up to a point. I don't see a need for a way to distinguish between
> slots and "normal" attributes, personally. But I don't do anything fancy
> here, so my experience isn't very indicative.
Without a more formal reflection API, the traditional way to get all normal
dictionary attributes is by using instance.__dict__.keys(). All I'm
proposing is that instance.__slotattrs__ (or possibly
instance.__class__.__slotattrs__) returns a list of objects that reveal the
name of all slots in the instance (including those declared in base
classes). I am not sure what that list should look like, though here are
the current suggestions:
1) __slotattrs__ = ('a','b')
2) # slot descriptor objects -- the repr is shown here
__slotattrs__ = ('<member 'a' of 'Baz' objects>',
<member 'b' of 'Baz' objects>')
The only issue that concerns me is that I am not sure if the slot to slot
name mapping should be fixed. The intrinsic definition of a slot is a type
and the offset of the slot in the type. The name is just a binding to a
slot descriptor, so it "feels" unnecessary to make that immutable. It
either case, it is not a big issue for me.
> > 3) These 'attrs' are to otherwise have the same semantics
> > as normal __dict__ instance attributes. e.g., they should
> > be automatically picklable, they can have properties
> > assigned to them, etc.
>
> I think I agree here. However, if you want slots to behave like normal
> attributes, except for the flat namespace, I see no value. Why have the
> exception at all?
Attributes currently have a flat namespace? I must not have been clear -- I
_do_ want my slotattrs to be allocated like slots, mainly for efficiency
reasons.
> Hmm, this raises the question of why we have slots at all. If they act
> exactly like attributes, why have them? As a user, I perceive them as an
> efficiency thing - they save the memory associated with a dictionary, and
> are probably faster as well. There can be tradeoffs which you pay for that
> efficiency, but that's all. No semantic difference. Actually, that's pretty
> much how the descrintro document describes slots. Strange that...
EXACTLY! I want to use slots (or slotattrs, or whatever you call them) to
be solely an allocation declaration. For small objects that you expect to
create many, many instance of, they make a huge difference. I have some
results that I measured on various implementations of a class to represent
database rows. The purpose of this class is to give simple dictionary-like
attribute access semantics to tuples returned as a result of default DB-API
relational database queries. The trick is to add the ability to access
fields by name (instead of by only by index) without incurring the overhead
of allocating a dictionary for every instance. Below are results of a
benchmark that compare various ways of implementing this class:
time
SIZE (sec) Bytes/row
-------- ------ ---------
overhead: 4,744KB 0.56 -
tuple: 18,948KB 2.49 73
C extension w/ slots: 18,924KB 4.85 73
native dict*: 117MB 13.50 589
Python class w/ slots: 18,960KB 17.23 73
Python class w/o slots: 117MB 24.09 589
* the native dict implementation does not allow indexed access,
and is only included as a reference point.
[For more details and discussion of this specific application, please see
this URL: http://opensource.theopalgroup.com/ ]
Thanks,
-Kevin
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com
Fax: (216) 986-0714 WWW: http://www.theopalgroup.com