[Tutor] os.listdir blocks threads?

[Pierre Barbier de Reuille]

Hello,

first, I believe listdir is implemented so as to block other threads
because underlying C functions are *not* thread safe ! So if you try
reading two directories in two different threads you may end up with
really strange things happening !

For your process problem, if you want to have full control of the
process while abstracting the pipe things, you may want to use the
popen2 module and, more precisely, the Popen3 or Popen4 objects that
will give you access to the PID.

However, be aware that functions involving PID are always OS-dependant.
So you may want to abstract it a bit to call UNIX, Windows or OS-X
functions depending on the current plateform.

Pierre

Wolfgang Braun a écrit :
> Hello List,
> 
> 
> I try to read large directories off network shares (samba3,NT) with
> os.listdir(). The listdir() takes up to 5 minutes and blocks the rest of
> the program (gui refreshes, ...)
> 
> The idea now is to put the listdir call into a separate thread so the
> program can go on doing stuff (see below).
> 
> Unfortunately, as soon as I start a rdir instance the whole python
> process locks up and waits for the os.listdir to return.
> 
> Plan b was to popen('/bin/ls -1 '%dir) in the rdir thread which works
> better but is a bit kludgy (needs to work on NT, too) and the program
> cannot be shudown cleanly unless the rdir thread has finished.
> 
> 
> Obviously I'm doing something wrong? What would be the Right Way to
> handle this situation?
> 
> 
> Thanks,
> Wolfgang
> 
> 
> # --- skeleton listdir in a thread
> 
> class rdir(threading.Thread):
>     def __init__(self,dir,glob=None):
>         super(rdir,self).__init__()
>         self.dir=dir
>         self.content=()
> 
>     def start(self):
>         self.setDaemon(True)
>         super(dircache,self).start()
> 
>     def run(self):
>             self.content=os.listdir(self.dir)
> 	    # alternatively os.popen(' /bin/ls -1U '%dir)
>             # stuff to keep run() from returning
> 
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> http://mail.python.org/mailman/listinfo/tutor
> 

-- 
Pierre Barbier de Reuille

INRA - UMR Cirad/Inra/Cnrs/Univ.MontpellierII AMAP
Botanique et Bio-informatique de l'Architecture des Plantes
TA40/PSII, Boulevard de la Lironde
34398 MONTPELLIER CEDEX 5, France

tel   : (33) 4 67 61 65 77    fax   : (33) 4 67 61 56 68