cannot open file in write mode, no such file or directory

[David Bolen]

haynesc at gmail.com writes:

> I'm having a problem where when trying to open a file in write mode, I
> get an IOError stating no such file or directory.  I'm calling an
> external program which takes an input file and produces an output file
> repeatedly, simulating the input file separately for each replicate.
> The error occurs when trying to open the input file to write out the
> new data.  The problem is difficult to reproduce since it only shows up
> once every few thousand replicates.  I've tried using both os.system
> and os.popen to invoke the external program.  Originally I was running
> this on cygwin, but also tried under windows.

You might be hitting a race condition where the OS is still
considering the file to be in use when you get around to rewriting it,
even if the using application has just exited.  I've run into similar
problems when trying to rename temporary files under NT based systems.

The problem can be obscured because some of the Win32-specific IO
errors can turn into more generic IOError exceptions at the Python
level due to incomplete mappings available for all Win32 errors.  In
particular, a lot of Win32-layer failures turn into EINVAL errno's at
the C RTL level, which Python in turn translates to ENOENT (which is
the file not found).  So the IOError exception at the Python level can
be misleading.

Since it sounds like you can reproduce the problem relatively easily
(just run your application several thousand times), a quick check for
this condition would be to trap the IOError, delay a few seconds (say
5-10 to be absolutely sure, although in the cases I've run into 2-3 is
generally more than enough), and retry the operation.  If that
succeeds, then this might be the issue you're hitting.

-- David