[Patches] [ python-Patches-1239890 ] Prevent race condition in os.makedirs

[SourceForge.net]

Patches item #1239890, was opened at 2005-07-17 19:42
Message generated for change (Comment added) made by gbrandl
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1239890&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Out of Date
Priority: 5
Private: No
Submitted By: Nir Soffer (nirs)
Assigned to: Nobody/Anonymous (nobody)
Summary: Prevent race condition in os.makedirs

Initial Comment:
makedirs check if a directory exists, and call mkdir to create it. In 
rare cases, the directory might have been created by another 
process or thread in the time passed from the exists() call to the 
mkdir() call.

To prevent this rare race condition, use try: expect: to create the 
directory, then ignore "File exists" errors, which mean the directory 
was there.

----------------------------------------------------------------------

>Comment By: Georg Brandl (gbrandl)
Date: 2006-12-09 09:24

Message:
Logged In: YES 
user_id=849994
Originator: NO

Something similar has been committed as patch #1608267.

----------------------------------------------------------------------

Comment By: Nir Soffer (nirs)
Date: 2005-12-05 12:46

Message:
Logged In: YES 
user_id=832344

Sorry, here is the patch.

----------------------------------------------------------------------

Comment By: Nir Soffer (nirs)
Date: 2005-12-05 01:21

Message:
Logged In: YES 
user_id=832344

Please review this new patch, tested with current 2.5 code.

I factored duplicate code in makedirs and removedirs into private _splits

function, kind of the "super" version of os.path.split for the "super"
directory 
utilities.

I also simplified the test code for makedirs, and added more test cases.

----------------------------------------------------------------------

Comment By: Nir Soffer (nirs)
Date: 2005-12-05 01:08

Message:
Logged In: YES 
user_id=832344

I deleted both patches as they are both wrong:

The patch against 2.4.1 will not raise OSError when trying to create
existing 
directories.

The simpler code for 2.5 will not work because os.path.normpath is not
safe 
to use if the path contain symbolic links.

----------------------------------------------------------------------

Comment By: Nir Soffer (nirs)
Date: 2005-12-02 18:11

Message:
Logged In: YES 
user_id=832344

The 2.5 code is only a proof of concept, don't use it as is :-)

All the directories are handled in the same way - ignore OSError for 
existing directories, which is not compatible with the docs, that say it
should 
raise an OSError for the last directory.  This has to be fixed of course.

I'll get 2.5 development code and submit a real working patch.






----------------------------------------------------------------------

Comment By: Richard Barran (rbarran)
Date: 2005-12-02 16:07

Message:
Logged In: YES 
user_id=1207189

Hi,
Your patch "makedirs.py" for python 2.5 will fail under
Windows if a drive letter is specified:

>>> os.getcwd()
'C:\\Temp\\makedirs'
>>> os.listdir('.')
['makedirs.py', 'makedirs.pyc', 'test.py']
>>> makedirs.makedirs('c:/temp/makedirs/a/b')
>>> os.listdir('.')
['makedirs.py', 'makedirs.pyc', 'temp', 'test.py']

It will create the following path:
C:\Temp\makedirs\temp\makedirs\a\b

Also I ran it through the test suite (lib\test\test_os.py)
and it failed one test:

======================================================================
FAIL: test_makedir (__main__.MakedirTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_os.py", line 327, in test_makedir
    self.failUnlessRaises(OSError, os.makedirs, os.curdir)
AssertionError: OSError not raised

----------------------------------------------------------------------

I haven't looked into *why* this happens - I'll dig a bit
deeper into this subject sometime next week.

HTH,
Richard

----------------------------------------------------------------------

Comment By: Nir Soffer (nirs)
Date: 2005-07-17 21:13

Message:
Logged In: YES 
user_id=832344

Here is another patch, using simpler design, maybe for 2.5?

The attached patch will try once to create each leaf in the path, ignoring

existing leafs. 

This should work for most cases. If one run application that can delete
its 
own directories, he should use proper locking.


----------------------------------------------------------------------

Comment By: Georg Brandl (birkenfeld)
Date: 2005-07-17 20:13

Message:
Logged In: YES 
user_id=1188172

http://sourceforge.net/tracker/index.php?func=detail&aid=1223238&group_id=5470&atid=105470

----------------------------------------------------------------------

Comment By: Nir Soffer (nirs)
Date: 2005-07-17 20:06

Message:
Logged In: YES 
user_id=832344

I can't find such bug or patch.

----------------------------------------------------------------------

Comment By: Georg Brandl (birkenfeld)
Date: 2005-07-17 19:57

Message:
Logged In: YES 
user_id=1188172

See bug #1223238.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1239890&group_id=5470