[New-bugs-announce] [issue24886] open fails randomly on AIX

[Arnon Yaari]

New submission from Arnon Yaari:

We are using Python 2.7.8 on AIX 7.1 TL 3.
On rare occasions, calls to open files with mode "w" fail with: IOError: File exists.

      File "/root/jenkins/workspace/powertools/eggs/infi.credentials_store-0.1-py2.7.egg/infi/credentials_store/base.py", line 175, in set_credentials
        with open(self.get_file_path(), 'w') as f:
    IOError: [Errno 17] File exists: '/root/.infinidat/infinihost'

This happens randomly on multiple systems and different files (including e.g. "/dev/null").
This is very strange because "File exists" should only be raised if we open the file with the 'x' flag (os.O_EXCL) and we don't.
After debugging, we discovered that this happens due to a bug in AIX. On AIX, fopen checks for mode[2] (where 'mode' is the second argument passed to it - in our case, "w") regardless of its length. fopen checks this byte against 'x' to decide if O_EXCL should be used. In the case of passing "w" as 'mode', mode[2] will contain undefined data -- and in rare cases it will contain the byte 'x'.
This was reported to IBM and APARs exist for this issue:
http://www.ibm.com/support/docview.wss?uid=isg1IV64453 (see related APARs for various AIX versions in this page)

Python can and should work around this because the code in fileobject.c is also partly to blame:
Python 2.7.x contains the following lines:

    /* probably need to replace 'U' by 'rb' */
    newmode = PyMem_MALLOC(strlen(mode) + 3);

For the 'mode' parameter, Python allocates 3 additional bytes that are not always used, and these bytes contain uninitialized data. The byte 'x' that causes the problem comes from this allocation. Python should set the bytes to zero to avoid this issue.
I'm attaching a patch. Note that this applies only to the 2.x branch.

----------
files: aix_fopen_fix.diff
keywords: patch
messages: 248769
nosy: wiggin15
priority: normal
severity: normal
status: open
title: open fails randomly on AIX
versions: Python 2.7
Added file: http://bugs.python.org/file40202/aix_fopen_fix.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue24886>
_______________________________________