Python 3.2 | WIndows 7 -- Multiprocessing and files not closing

[Isaac Gerg]

On Thu, Oct 10, 2013 at 2:49 PM, Isaac Gerg <isaac.gerg at gergltd.com> wrote:

>
>
>
> On Thu, Oct 10, 2013 at 2:41 PM, Ned Batchelder <ned at nedbatchelder.com>wrote:
>
>> On 10/10/13 12:44 PM, Isaac Gerg wrote:
>>
>>> Sorry, I am just providing pseudo code since I the code i have is quite
>>> large.
>>>
>>> As I mentioned, the code works fine when I remove the multirpcessing
>>> stuff so the filename is not the issue (though you are right in your
>>> correction).
>>>
>>> Someone with the same problem posted a smaller, more complete example
>>> here:
>>>
>>> http://stackoverflow.com/**questions/948119/preventing-**
>>> file-handle-inheritance-in-**multiprocessing-lib<http://stackoverflow.com/questions/948119/preventing-file-handle-inheritance-in-multiprocessing-lib>
>>>
>>> None of the solutions posted work.
>>>
>>
>> (BTW: it's better form to reply beneath the original text, not above it.)
>>
>> None of the solutions try the obvious thing of closing the file before
>> spawning more processes.  Would that work for you?  A "with" statement is a
>> convenient way to do this:
>>
>>     with open(filename,'r') as f:
>>         data = f.read()
>>
>> The file is closed automatically when the with statement ends.
>>
>> --Ned.
>>
>>
>>> On Thursday, October 10, 2013 12:38:19 PM UTC-4, Piet van Oostrum wrote:
>>>
>>>> Isaac Gerg <isaac.gerg at gergltd.com> writes:
>>>>
>>>>
>>>>
>>>>  I have a function that looks like the following:
>>>>>
>>>>
>>>>
>>>> That doesn't look like a function
>>>>
>>>>
>>>>
>>>>  #-----------------------------**----
>>>>> filename = 'c:\testfile.h5'
>>>>>
>>>>
>>>>
>>>> Your filename is most probably wrong. It should be something like:
>>>>
>>>>
>>>>
>>>> filename = 'c:/testfile.h5'
>>>>
>>>> filename = 'c:\\testfile.h5'
>>>>
>>>> filename = r'c:\testfile.h5'
>>>>
>>>> --
>>>>
>>>> Piet van Oostrum <piet at vanoostrum.org>
>>>>
>>>> WWW: http://pietvanoostrum.com/
>>>>
>>>> PGP key: [8DAE142BE17999C4]
>>>>
>>>
>>
> I will try what you suggest and see if it works.
>
> Additionally, is there a place on the web to view this conversation and
> reply?  Currently, I am only able to access this list through email.
>


Ned, I am unable to try what you suggest.  The multiprocess.Process call is
within a class but its target is a static method outside of the class thus
no pickling.  I cannot close the file and then reopen after the
multiprocess.Process call because other threads may be reading from the
file during that time.  Is there a way in Python 3.2 to prevent the
multiprocess.Process from inheriting the file descriptors from the parent
process OR, is there a way to ensure that the multiprocess is completely
closed and garbaged collected by the time I want to use os.remove()?

Isaac
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20131010/5277b20a/attachment.html>