[New-bugs-announce] [issue43460] Exception copy error
Douglas Raillard
report at bugs.python.org
Wed Mar 10 06:30:36 EST 2021
New submission from Douglas Raillard <douglas.raillard at arm.com>:
Instances of subclasses of BaseException created with keyword argument fail to copy properly as demonstrated by:
import copy
class E(BaseException):
def __init__(self, x):
self.x=x
# works fine
e = E(None)
copy.copy(e)
# raises
e = E(x=None)
copy.copy(e)
This seems to affect all Python versions I've tested (3.6 <= Python <= 3.9).
I've currently partially worked around the issue with a custom pickler that just restores __dict__, but:
* "args" is not part of __dict__, and setting "args" key in __dict__ does not create a "working object" (i.e. the key is set, but is ignored for all intents and purposes except direct lookup in __dict__)
* pickle is friendly: you can provide a custom pickler that chooses the reduce function for each single class.
copy module is much less friendly: copyreg.pickle() only allow registering custom functions for specific classes. That means there is no way (that I know) to make copy.copy() select a custom reduce for a whole subclass tree.
One the root of the issue:
* exception from the standard library prevent keyword arguments (maybe because of that issue ?), but there is no such restriction on user-defined classes.
* the culprit is BaseException_reduce() (in Objects/exceptions.c) [1]
It seems that the current behavior is a consequence of the __dict__ being created lazily, I assume for speed and memory efficiency
There seems to be a few approaches that would solve the issue:
* keyword arguments passed to the constructor could be fused with the positional arguments in BaseException_new (using the signature, but signature might be not be available for extension types I suppose)
* keyword arguments could just be stored like "args" in a "kwargs" attribute in PyException_HEAD, so they are preserved and passed again to __new__ when the instance is restored upon copying/pickling.
* the fact that keyword arguments were used could be saved as a bool in PyException_HEAD. When set, this flag would make BaseException_reduce() only use __dict__ and not "args". This would technically probably be a breaking change, but the only cases I can think of where this would be observable are a bit far fetched (if __new__ or __init__ have side effects beyond storing attributes in __dict__).
[1] https://github.com/python/cpython/blob/master/Objects/exceptions.c#L134
----------
messages: 388427
nosy: douglas-raillard-arm
priority: normal
severity: normal
status: open
title: Exception copy error
type: behavior
_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue43460>
_______________________________________
More information about the New-bugs-announce
mailing list