Moving to subprocess from os.popen--pipe closes prematurely

[Kevin Walzer]

Hello,

I'm trying to move from os.popen to using subprocess, and I'm having 
trouble with the pipe suddenly closing.

My old code looked something like this:

   #build and format a list of all packages
     def getAllPackages(self):

         self.masterlist=[]

         for self.catname in self.catlist:
             time.sleep(.3)
             self.getpackages = os.popen('%s list --section=%s' % 
(self.finkpath, self.catname), 'r', os.O_NONBLOCK)
             for line in self.getpackages:
                 newline = line.split('\t')
                 rawcat = newline[0]
                 if rawcat == '(i)':
                     firstcat=rawcat.replace('(i)', 'outdated')
                 elif rawcat == ' i ':
                     firstcat=rawcat.replace('i', 'current')
                 elif rawcat == ' p ':
                     firstcat=rawcat.replace('p', 'provided')
                 else:
                     firstcat = rawcat
                 self.packagelist = (firstcat, newline[1], newline[2], 
self.catname, newline[3].strip('\n'))
                 self.masterlist.append(self.packagelist)


Using the time.sleep function and adding the os.O_NONBLOCK to the 
os.popen call let the data stream come through uninterrupted.

Here is the comparable code using subprocess:

   def getAllPackages(self):

         self.masterlist=[]
         self.showProgress()
         self.status.set('Getting all packages')
         self.update()
         for self.catname in self.catlist:
             self.update()
             self.status.set('Getting all packages by category: %s' % 
self.catname)
             self.getpackages = Popen('%s list --section=%s' % 
(self.finkpath, self.catname), shell=True, bufsize=0, stdout=PIPE).stdout
             time.sleep(5)
             fcntl.fcntl(self.getpackages, fcntl.F_SETFL, os.O_NONBLOCK)
             print "getting %s" % self.catname
             self.update()
             for line in self.getpackages:
                 print line
                 time.sleep(.1)
                 self.update()

                 newline = line.split('\t')
                 rawcat = newline[0]
                 if rawcat == '(i)':
                     firstcat=rawcat.replace('(i)', 'outdated')
                 elif rawcat == ' i ':
                     firstcat=rawcat.replace('i', 'current')
                 elif rawcat == ' p ':
                     firstcat=rawcat.replace('p', 'provided')
                 else:
                     firstcat = rawcat
                 self.packagelist = (firstcat, newline[1], newline[2], 
self.catname, newline[3].strip('\n'))


I am using the fcntl call to set the pipe to nonblocking mode, and am 
trying to time.sleep to give the buffer time to get some data. However, 
this code crashes with the error "IOError: [Errno 35] Resource 
temporarily unavailable."

Any advice to get the subprocess bits working so that the buffer does 
not close prematurely would be appreciated.

Thank you,
Kevin

-- 
Kevin Walzer
Code by Kevin
http://www.codebykevin.com