[Python-ideas] Implementation of shutil.move

[David Townshend]

I''ve done some more reading and this is where I've got to:

shutil.move() uses either os.rename() or shutil.copyfile() (via
shutil.copytree()) depending on the situation. shutil.copyfile() uses
open().  So to implement a safe move function it would seem to be necessary
to do the same to copyfile(), and possibly open().

open(), in my opinon, already behaves as it should.  It would be possible to
add a slightly safer implementation by writing to a temporary file first,
but this would not always be desired or even possible. Perhaps an
alternative function could be added if the idea is popular enough?

I would expect copyfile(), like move(), to fail if the destination exists.
 This is not the current behaviour, so both functions could benefit from
this.

>From what I've read, there are several ways of ensuring that these functions
fail if destination exists depending on the platform and filesystem, but
there is no uniform way to do it. One approach would be to try all possible
methods and hope that at least one works, with a simple "if os.exists(dst):
fail" fallback.  The documentation would state that "An exception occurs if
the destination exists.  This check is done is as safe a way possible to
avoid race conditions where the system supports it."  An additional measure
of safety on copyfile() would be to write to a temporary file first, then
use move. This would allow rollback in case of failure during the copy, but
as with open(), its not always the most appropriate approach.

Adding new copyfile() and move() functions would mean also mean adding new
copy(), copy2() and copytree() functions, perhaps as copy3() and
copytree2(). This seems to be getting rather messy - three slightly
different copy functions, so it might still be better to add an optional
argument to these.  Alternatively, a new module could be added dedicated to
safe file operations.

On Sun, Aug 14, 2011 at 5:23 PM, David Townshend <aquavitae69 at gmail.com>wrote:

> Sorry, yes. That is what I meant.
> On Aug 14, 2011 5:12 PM, "Devin Jeanpierre" <jeanpierreda at gmail.com>
> wrote:
> >> Why do you think that move on remote file systems use copy? From past
> >> experience and recent tests I can confirm that shutil.move() uses rename
> >> on remote CIFS and NFS file systems
> >
> > I believe what he meant to say was "if you move from one filesystem to
> another".
> >
> > shutil.move tries to do a copy and delete if rename fails with an
> OSError.
> >
> > Devin
> >
> > On Sun, Aug 14, 2011 at 10:39 AM, Christian Heimes <lists at cheimes.de>
> wrote:
> >> Am 14.08.2011 16:09, schrieb David Townshend:
> >>> It seems there's a second problem too - move on remote file systems use
> copy
> >>> rather than rename, so changing the implementation means changing it
> for
> >>> copy too, which is more difficult. Maybe the best option is to try to
> apply
> >>> some sort of locking mechanism, but I can't see how right now.
> >>
> >> Why do you think that move on remote file systems use copy? From past
> >> experience and recent tests I can confirm that shutil.move() uses rename
> >> on remote CIFS and NFS file systems.
> >>
> >>
> >> _______________________________________________
> >> Python-ideas mailing list
> >> Python-ideas at python.org
> >> http://mail.python.org/mailman/listinfo/python-ideas
> >>
> > _______________________________________________
> > Python-ideas mailing list
> > Python-ideas at python.org
> > http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20110815/da83ecce/attachment.html>