A "scopeguard" for Python

[Robert Kern]

On 2010-03-04 12:37 PM, Alf P. Steinbach wrote:
> * Robert Kern:
>> On 2010-03-04 10:56 AM, Alf P. Steinbach wrote:
>>> * Robert Kern:
>>>> On 2010-03-03 18:49 PM, Alf P. Steinbach wrote:
> [snippety]
>>>>
>>>>> If you call the possibly failing operation "A", then that systematic
>>>>> approach goes like this: if A fails, then it has cleaned up its own
>>>>> mess, but if A succeeds, then it's the responsibility of the calling
>>>>> code to clean up if the higher level (multiple statements) operation
>>>>> that A is embedded in, fails.
>>>>>
>>>>> And that's what Marginean's original C++ ScopeGuard was designed for,
>>>>> and what the corresponding Python Cleanup class is designed for.
>>>>
>>>> And try: finally:, for that matter.
>>>
>>> Not to mention "with".
>>>
>>> Some other poster made the same error recently in this thread; it is a
>>> common fallacy in discussions about programming, to assume that since
>>> the same can be expressed using lower level constructs, those are all
>>> that are required.
>>>
>>> If adopted as true it ultimately means the removal of all control
>>> structures above the level of "if" and "goto" (except Python doesn't
>>> have "goto").
>>
>> What I'm trying to explain is that the with: statement has a use even
>> if Cleanup doesn't. Arguing that Cleanup doesn't improve on try:
>> finally: does not mean that the with: statement doesn't improve on
>> try: finally:.
>
> That's a different argument, essentially that you see no advantage for
> your current coding patterns.
>
> It's unconnected to the argument I responded to.
>
> The argument that I responded to, that the possibility of expressing
> things at the level of try:finally: means that a higher level construct
> is superfluous, is still meaningless.

I am attacking your premise that the "with Cleanup():" construct is higher level 
than try: finally:. It isn't. It provides the same level of abstraction as try: 
finally:.

This is distinct from the accepted uses of the with: statement which *are* 
higher level than try: finally: and which do confer practical benefits over 
using try: finally: despite being syntactical sugar for try: finally:.

>>>>>> Both formulations can be correct (and both work perfectly fine with
>>>>>> the chdir() example being used). Sometimes one is better than the
>>>>>> other, and sometimes not. You can achieve both ways with either your
>>>>>> Cleanup class or with try: finally:.
>>>>>>
>>>>>> I am still of the opinion that Cleanup is not an improvement over
>>>>>> try:
>>>>>> finally: and has the significant ugliness of forcing cleanup code
>>>>>> into
>>>>>> callables. This significantly limits what you can do in your cleanup
>>>>>> code.
>>>>>
>>>>> Uhm, not really. :-) As I see it.
>>>>
>>>> Well, not being able to affect the namespace is a significant
>>>> limitation. Sometimes you need to delete objects from the namespace in
>>>> order to ensure that their refcounts go to zero and their cleanup code
>>>> gets executed.
>>>
>>> Just a nit (I agree that a lambda can't do this, but as to what's
>>> required): assigning None is sufficient for that[1].
>>
>> Yes, but no callable is going to allow you to assign None to names in
>> that namespace, either. Not without sys._getframe() hackery, in any case.
>>
>>> However, note that the current language doesn't guarantee such cleanup,
>>> at least as far as I know.
>>>
>>> So while it's good practice to support it, to do everything to let it
>>> happen, it's presumably bad practice to rely on it happening.
>>>
>>>
>>>> Tracebacks will keep the namespace alive and all objects in it.
>>>
>>> Thanks!, I hadn't thought of connecting that to general cleanup actions.
>>>
>>> It limits the use of general "with" in the same way.
>>
>> Not really.
>
> Sorry, it limits general 'with' in /exactly/ the same way.
>
>> It's easy to write context managers that do that [delete objects from
>> the namespace].
>
> Sorry, no can do, as far as I know; your following example quoted below
> is an example of /something else/.

Okay, so what do you mean by 'the use of general "with"'? I'm talking about 
writing a context manager or using the @contextmanager decorator to do some 
initialization and then later cleaning up that initialization. That cleaning up 
may entail deleting an object. You are correct that the context manager can't 
affect the namespace of the with: clause, but that's not the initialization that 
it would need to clean up.

Yes, you can write code with a with: statement where you try to clean up stuff 
that happened inside of the clause (you did), but that's not how the with: 
statement was ever intended to be used nor is it good practice to do so because 
of that limitation. Context managers are designed to initialize specific things, 
then clean them up. I thought you were talking about the uses of the with: 
statement as described in PEP-343, not every possible misuse of the with: statement.

> And adding on top of irrelevancy, for the pure technical aspect it can
> be accomplished in the same way using Cleanup (I provide an example below).
>
> However, doing that would generally be worse than pointless since with
> good coding practices the objects would become unreferenced anyway.
>
>
>> You put the initialization code in the __enter__() method, assign
>> whatever objects you want to keep around through the with: clause as
>> attributes on the manager, then delete those attributes in the
>> __exit__().
>
> Analogously, if one were to do this thing, then it could be accomplished
> using a Cleanup context manager as follows:
>
> foo = lambda: None
> foo.x = create_some_object()
> at_cleanup.call( lambda o = foo: delattr( o, "x" ) )
>
> ... except that
>
> 1) for a once-only case this is less code :-)

Not compared to a try: finally:, it isn't.

> 2) it is a usage that I wouldn't recommend; instead I recommend adopting
> good
> coding practices where object references aren't kept around.

Many of the use cases of the with: statement involve creating an object (like a 
lock or a transaction object), keeping it around for the duration of the "# Do 
stuff" block, and then finalizing it.

>> Or, you use the @contextmanager decorator to turn a generator into a
>> context manager, and you just assign to local variables and del them
>> in the finally: clause.
>
> Uhm, you don't need a 'finally' clause when you define a context manager.

When you use the @contextmanager decorator, you almost always do. See the 
Examples section of PEP 343:

   http://www.python.org/dev/peps/pep-0343/

> Additionally, you don't need to 'del' the local variables in
> @contextmanager decorated generator.
>
> The local variables cease to exist automatically.

True.

>> What you can't do is write a generic context manager where the
>> initialization happens inside the with: clause and the cleanup actions
>> are registered callables. That does not allow you to affect the
>> namespace.
>
> If you mean that you can't introduce direct local variables and have
> them deleted by "registered callables" in a portable way, then right.
>
> But I can't think of any example where that would be relevant; in
> particular what matters for supporting on-destruction cleanup is whether
> you keep any references or not, not whether you have a local variable of
> any given name.

Well, local variables keep references to objects. Variable assignment followed 
by deletion is a very readable way to keep an object around for a while then 
remove it later. If you have to go through less readable contortions to keep the 
object around when it needs to be and clean it up later, then that is a mark 
against your approach.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco