The opener parameter of Python 3 open() built-in

[Steven D'Aprano]

On Mon, 03 Sep 2012 23:19:51 -0400, Dennis Lee Bieber wrote:

> On 04 Sep 2012 01:13:09 GMT, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> declaimed the following in
> gmane.comp.python.general:
> 
> 
> 
>> Why does the open builtin need this added complexity? Why not just call
>> os.open directly? Or for more complex openers, just call the opener
>> directly?
>>
> 	Because os.open() returns a low-level file descriptor, not a 
> Python file object?

Good point.

But you can wrap the call to os.open, as you mention below. The only 
complication is that you have to give the mode twice, converting between 
low-level O_* integer modes and high-level string modes:

a = os.open('/tmp/foo', os.O_WRONLY | os.O_CREAT)
b = os.fdopen(a, 'w')


But to some degree, you still have to do that with the opener argument, 
at least in your own head.


>> What is the rationale for complicating open instead of telling people
>> to just call their opener directly?
> 
> 	To avoid the new syntax would mean coding the example as
> 
> 	f = os.fdopen(os.open("newfile", flags | os.O_EXCL), "w")
> 
> which does NOT look any cleaner to me... 

Well, I don't know about that. Once you start messing about with low-
level O_* flags, it's never going to exactly be clean no matter what you 
do. But I think a one-liner like the above *is* cleaner than a three-
liner like the original:

def opener(file, flags):
    return os.open(file, flags | os.O_EXCL)

open("newfile", "w", opener=opener)

although I accept that this is a matter of personal taste.

Particularly if the opener is defined far away from where you eventually 
use it. A lambda is arguably better from that perspective:

open("newfile", "w", 
     opener=lambda file, flags: os.open(file, flags | os.O_EXCL)
     )

but none of these solutions are exactly neat or clean. You still have to 
mentally translate between string modes and int modes, and make sure 
you're not passing the wrong mode:

py> open('junk', 'w').write('hello world')
11
py> open('junk', 'r', opener=lambda file, flags: os.open(file, flags | 
os.O_TRUNC)).read()  # oops
''

so it's not exactly a high-level interface.

In my opinion, a cleaner, more Pythonic interface would be either:

* allow built-in open to take numeric modes:

  open(file, os.O_CREAT | os.O_WRONLY | os.O_EXCL)

* or even more Pythonic, expose those numeric modes using strings:

  open(file, 'wx')


That's not as general as an opener, but it covers the common use-case and 
for everything else, write a helper function.


> Especially not if "opener" is to be used in more than one location.

The usual idiom for fixing the "used more than once" is "write a helper", 
not "add a callback function to a builtin" :)


> Furthermore, using "opener" could
> allow for a localized change to affect all open statements in the module
> -- change file path, open for string I/O rather than file I/O, etc.


A common idiom for that is to shadow open in the module, like this:

_open = open
def open(file, *args):
    file = file.lowercase()
    return _open(file, *args)



-- 
Steven