[CentralOH] Fwd: Files and context manager

[Brian Costlow]

forgot to reply all on this..so forwarding to the group


---------- Forwarded message ----------
From: Brian Costlow <brian.costlow at gmail.com>
Date: Thu, Mar 11, 2010 at 3:55 PM
Subject: Re: [CentralOH] Files and context manager
To: Mark Erbaugh <mark at microenh.com>


On Thu, Mar 11, 2010 at 3:02 PM, Mark Erbaugh <mark at microenh.com> wrote:
> Brian,
>
> Thanks for the reply.  The point of my question was
>
> I guess the real point about
>
> x = open(filename,'rb').read()
>
> is that close is never called. Is that safe, just relying on the file object's destructor.

Not really. If you are doing something that opens a lot of files you
can get in a situation where file handles run out.
This can especially bite if you handle errors somewhere upstream, and
don't exit, or if you end up with circular refs somehow.
Hard to debug because everything usually seems fine, but you run out
of file handles.
Or, in Windows, you can leave files possibly unaccessible by other apps.
If it's just a command line util that opens a couple of files, does
something, then exits, you can get away with it. But I don't think
it's good practice.



>
> My second question was more about the details of context managers:
>
>
> with open(filename,'rb') as f:
>        return f.read()
>
> What happens when a return statement terminates a context manager rather than falling off the end of the code.

It doesn't need to 'fall off the end', just exit the block, which the
return does. Here's some sample code that shows this.
Substitute a real file path for /some/file/path,

import os

def foo(file_path):
       with open(file_path,'rb') as f:
               fnum = f.fileno()
               return (f.read(),fnum)
               print 'unreachable\n'

contents,fid = foo('/some/file/path')
os.fstat(fid)

Will raise a OSError: [Errno 9] Bad file descriptor on the descriptor
since it's been closed.



>
> Is it a shortcut for
>
> try:
>  x = open(filename,'rb')
>  return x.read()
> finally:
>  x.close()
>
> Thanks,
> Mark
>
> On Mar 10, 2010, at 5:34 PM, Brian Costlow wrote:
>
>> Hi Mark,
>>
>> Yes. Although you can get that same level of safety without them.
>>
>> The way contexts are used with file objects in python is they are a shortcut for
>>
>> try:
>>    x = open(filename,'rb').read()
>>    # do stuff with x
>> finally:
>>    x.close()
>>
>> If you need to do error handling though, you still need to catch
>> exceptions. But if you are catching them farther upstream you don't
>> need to clutter your code with try blocks just for the file
>> open/read/close idiom.
>>
>> Not sure what you mean by a good idea. If you have code in a bunch of
>> places in an app that opens and reads files by filename, of course
>> putting that in a function is a good idea. But if you are asking, is
>> it okay to just read a (potentially large) file into memory using
>> read(), I do it all the time for files in the 5-35 mb range in an app
>> that checksums images.
>>
>>
>>
>>
>> On Wed, Mar 10, 2010 at 3:39 PM, Mark Erbaugh <mark at microenh.com> wrote:
>>>
>>>
>>> A common idiom with older versions of Python is:
>>>
>>> x = open(filename,'rb').read()
>>>
>>> With context managers in 2.6+ you could say:
>>>
>>> with open(filename,'rb') as f:
>>> x = f.read()
>>>
>>> Is the later safer in that it explicitly will call close() whereas the
>>> former depends on the file destructor to close the file?
>>>
>>>
>>> I forgot the second half of my question.
>>> Is a function like:
>>> def getData(filename):
>>> with open(filename,'rb') as f:
>>>              return f.read()
>>>
>>> a good idea?
>>> _______________________________________________
>>> CentralOH mailing list
>>> CentralOH at python.org
>>> http://mail.python.org/mailman/listinfo/centraloh
>>>
>>>
>
>