Help replacing os.system call with subprocess call

[Matt Nordhoff]

Matt Nordhoff wrote:
> David Pratt wrote:
>> Hi. I am trying to replace a system call with a subprocess call. I have 
>> tried subprocess.Popen and subprocess.call with but have not been 
>> successful. The command line would be:
>>
>> svnadmin dump /my/repository > svndump.db
>>
>> This is what I am using currently:
>>
>> os.system('svnadmin dump %s > %s' % (svn_dir,
>>      os.path.join(backup_dir, 'svndump.db')))
>>
>> Many thanks.
> 
> Try this:
> 
> import os.path
> import subprocess
> 
> p = subprocess.Popen(
>     ['svnadmin', 'dump', svndir],
>     stdout=subprocess.PIPE,
> )
> 
> fh = open(os.path.join(backup_dir, 'svndump.db'), 'wb')
> 
> while True:
>     chunk = p.stdout.read(2**20) # 1 MB
>     if not chunk:
>         break
>     fh.write(chunk)
> 
> fh.close()
> 
> It reads svnadmin's stdout in 1 MB chunks, in case it's large enough
> that reading the whole thing into RAM at once would be a bad idea.
> 
> No error handling. For one, you might want to add a try...finally to
> ensure that fh will get closed. (Or if you have Python 2.5, use a with
> statement! :-) ) Also, Popen will raise an OSError if svnadmin can't be
> found or something. And this isn't even considering svnadmin erroring out...
> 
> svnadmin's stderr will go to your stderr.
> 
> I didn't test it, but I'm pretty sure it will work. (I spotted a syntax
> error while writing that though.) I don't have much experience with
> Popen's stdio objects, so it's possible you'd need to do something like
> call p.wait() to wait for it to exit before being able to read its stdout.
> 
> It could be slower than the os.system version, since now Python is doing
> all of the I/O, instead of your shell, but I doubt that'll be a big problem.
> 
> (Also, insert suggestion about using a good VCS. ;-) )

This might work too:

import os.path
import subprocess

fh = open(os.path.join(backup_dir, 'svndump.db'), 'wb')

p = subprocess.Popen(
    ['svnadmin', 'dump', svndir],
    stdout=fh,
)

# Wait for svnadmin to exit so that fh can be closed
p.wait()
fh.close()

With this, svnadmin's stdout is directly set to the file. Python isn't
being an intermediary reading from stdout and writing it to the file. At
least, that's how I expect it would work.

I didn't test this either. In particular, if Popen doesn't behave how I
expect it should, it might buffer the entirety of stdout in memory or
something, which would be bad if your svn repo is really large...
--