Issue520644
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2002-02-20 20:50 by pedronis, last changed 2022-04-10 16:05 by admin. This issue is now closed.
Files | ||||
---|---|---|---|---|
File name | Uploaded | Description | Edit | |
sltattr.py | pedronis, 2002-02-22 01:33 | slots more like attrs illustrative code |
Messages (9) | |||
---|---|---|---|
msg9324 - (view) | Author: Samuele Pedroni (pedronis) * | Date: 2002-02-20 20:50 | |
[Posted on behalf of Kevin Jacobs] I have been hacking on ways to make lighter-weight Python objects using the __slots__ mechanism that came with Python 2.2 new- style class. Everything has gone swimmingly until I noticed that slots do not get pickled/cPickled at all! Here is a simple test case: import pickle,cPickle class Test(object): __slots__ = ['x'] def __init__(self): self.x = 66666 test = Test() pickle_str = pickle.dumps( test ) cpickle_str = cPickle.dumps( test ) untest = pickle.loads( pickle_str ) untestc = cPickle.loads( cpickle_str ) print untest.x # raises AttributeError print untextc.x # raises AttributeError ... see http://aspn.activestate.com/ASPN/Mail/Message/python- dev/1031499 |
|||
msg9325 - (view) | Author: Kevin Jacobs (jacobs99) | Date: 2002-02-21 17:51 | |
Logged In: YES user_id=459565 This bug raises questions about what a slot really is. After a fair amount of discussion on Python-dev, we have come up with basically two answers: 1) a slot is a struct-member that is part of the private implementation of an object. Slots should have their own semantics and not be expected to act like Python instance attributes. 2) slots should be treated just like dict instance attributes except they are allocated statically within the object itself, and require slightly different reflection methods. Under (1), this bug isn't really a bug. The class should implement a __reduce__ function or otherwise hook into the copy_reg system. Under (2), this bug is just the tip of the iceberg. There are about 8 other problems with the current slot implementation that need to be resolved before slots act almost identically to normal instance attributes. Thankfully, I am fairly confident that I can supply patches that can achieve this, though I am waiting for Guido to comment on this issue when he returns from his trip. |
|||
msg9326 - (view) | Author: Samuele Pedroni (pedronis) * | Date: 2002-02-22 01:33 | |
Logged In: YES user_id=61408 some slots more like attrs illustrative python code |
|||
msg9327 - (view) | Author: Kevin Jacobs (jacobs99) | Date: 2002-02-22 14:52 | |
Logged In: YES user_id=459565 Samuele's sltattr.py is an interesting approach, though I am not entirely sure it is necessary or feasible sufficiently address the significant problems with slots via proxying __dict__ (see #5 below). Here is a mostly complete list of smaller changes that are somewhat orthogonal to how we address accesses to __dict__: 1) Flatten slot lists: Change obj.__class__.__slots__ to return an immutable list of all slot descriptors in the object (including all those of base classes). The motivation for this is similar in spirit to storing a flattened __mro__. The advantages of this change are: a) allows for fast and explicit object reflection that correctly finds all dict attributes, all slot attributes. b) allows reflection implementations (like vars (object) and pickle) to treat dict and slot attrs differently if we choose not to proxy __dict__. This has several advantages, as explained in change #2. Also importantly, this way it is not possible to "lose" descriptors permanently by deleting them from obj.__class__.__dict__. 2) Update reflection API even if we do not choose to proxy __dict__: Alter vars(object) to return a dictionary of all attributes, including both the contents of the non-proxied __dict__ and the valid attributes that result from iterating over __slots__ and evaluating the descriptors. The details of how this is best implemented depend on how we wish to define the behavior of modifying the resulting dictionary. It could be either: a) explicitly immutable, which involves creating proxy objects b) mutable, which involves copying c) undefined, which means implicitly immutable Aside from the questions over the nature of the return type, this implementation (coupled with #1) has distinct advantages. Specifically the native object.__dict__ has a very natural internal representation that pairs attribute names directly with values. In contrast, a fair amount of additional work is needed to extract the slots that store values and create a dictionary of their names and values. Other implementations will require a great deal more work since they would have to traverse though base classes to collecting slot descriptors. 3) Flatten slot inheritance: Update the new-style object inheritance mechanism to re-use slots of the same name, rather than creating a new slot and hiding the old. This makes the inheritance semantics of slots equivalent to those of normal instance attributes and avoids introducing an ad-hoc and obscure method of data hiding. 4) Update standard library to use new reflection API (and make them robust to properies at the same time) if we choose not to proxy __dict__. Virtually all of the changes are simple and involve updating these constructs: a) obj.__dict__ b) obj.__dict__[blah] c) obj.__dict__[blah] = x (What these will become depends on other factors, including the context and semantics of vars(obj).) Here is a fairly complete list of Python 2.2 modules that will need to be updated: copy, copy_reg, inspect, pickle, pydoc, cPickle, Bastion, codeop, dis, doctest, gettext, ihooks, imputil, knee, pdb, profile, rexec, rlcompleter, tempfile, unittest, xmllib, xmlrpclib 5) (NB: potentially controversial and not required) We could alter the descriptor protocol to make slots (and properties) more transparent when the values they reference do not exist. Here is an example to illustrate this: class A(object): foo = 1 class B(A): __slots__ = ('foo',) b = B() print b.foo > 1 or AttributeError? Currently an AttributeError is raised. However, it is a fairly easy change to make AttributeErrors signal that attribute resolution is to continue until either a valid descriptor is evaluated, an instance-attribute is found, or until the resolution fails after search the meta-type, the type and the instance dictionary. The problem illustrated by the above code also occurs when trying to create proxies for __dict__, if the proxy worked on the basis of the collected slot descriptors (__allslots__ in Samuele's example). I am prepared to submit patches to address each of these issues. However, I do want feedback beforehand, so that I do not waste time implementing something that will never be accepted. |
|||
msg9328 - (view) | Author: Kevin Jacobs (jacobs99) | Date: 2002-02-22 15:03 | |
Logged In: YES user_id=459565 Oops. Please ignore the last paragraph of point #5. Samuele's __allslots__ is fine with regard to the example I presented. |
|||
msg9329 - (view) | Author: Samuele Pedroni (pedronis) * | Date: 2002-02-28 19:29 | |
Logged In: YES user_id=61408 [Guido on python-dev] In particular, the fact that instances of classes with __slots__ appear picklable but lose all their slot values is a bug -- these should either not be picklable unless you add a __reduce__ method, or they should be pickled properly. ... I haven't made up my mind on how to fix this -- it would be nice if __slots__ would automatically be pickled, but it's tricky (although I think it's doable -- without ever referencing the __slots__ variable :-). [pedronis - my 2cts] unless you plan some low-level (non-python-level) solution, I think a main question is whether member and properties are distinguishable and maybe whether among members basic type members (file.softspace etc) and __slots__ members are distinguishable It would be somehow strange and redundant if properties value would be automatically pickled (I see them as computed value) In java (bean) properties are not pickled and even fields (= slots) can be marked as transient to avoid their serialization. In your picture it seems that all those things are not to be dinstinguished, so probably no automatic serialization, if there are members and given that actually files for example cannot be pickled, would be a reasonable solution. Otherwise (distinguishable case) other automatic approaches can make sense too. Just some - I hope valuable - input and my opinion. |
|||
msg9330 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2002-02-28 19:36 | |
Logged In: YES user_id=6380 Good point. So maybe it should be up to the class to define how to pickle slots. An alternative idea could look at the type of descriptors; slots use a different type than properties. |
|||
msg9331 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2002-03-07 02:43 | |
Logged In: YES user_id=6380 I've decided on a "fix" that will go into 2.2.1. When a new-style class defines slots and does not define __getstate__, a dummy __getstate__ will be inserted into the class that raises an exception. If the class wants slots to be pickled, it should define a __getstate__ method. In 2.3, we may revise this decision. The solution space is large, and I don't want to rush it, hence the decision to make it fail cleanly in 2.2.1 -- that's the best I can do. The 2.2.1 solution will be compatible with whatever we decide to do in 2.3. |
|||
msg9332 - (view) | Author: Guido van Rossum (gvanrossum) * | Date: 2002-03-14 23:07 | |
Logged In: YES user_id=6380 I've checked in the proposed fix as typeobject.c rev. 2.129. Also a test for the new behavior to test_descr.py and a NEWS entry. These should all be added to 2.2.1 as well. Closing now. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-10 16:05:00 | admin | set | github: 36129 |
2002-02-20 20:50:36 | pedronis | create |