Problems with file IO in a thread, and shutil

Roopesh roopesh.raj at gmail.com
Wed Jun 25 11:11:13 EDT 2008


Hi,

I have a multithreaded application. There are two threads, T1 and T2.
Suppose that there are two folders A, B.  Thread T1 fetches data from
network and creates files in folder A and after creating each file, it
moves the file to folder B, using shutil.move().

Thread T2, takes files from folder B and processes it.
Note: Only the thread T1 has access to the files in folder A.

psuedo code
=========
class T1(thread):
   def run():
     self.download()

  def download():
     data = from_network
     filename = os.path.join ( "A", "file1")

     f = open ( filename, "w")
     for d in data:
	f.write ( d + "\n" )
     f.flush()
     f.close()

    shutil.move(os.path.join ( "A", "file1"),  os.path.join("B",
"file1"))


class T2(thread):
  run()
      process(listdir(os.path.join("B")))

All the files has similar contents. But in some cases(very rare), when
thread T1 tries to move the newly created file from folder A to folder
B, an exception occurs (on shutil.move()).

The exception is WindowsError(32, 'The process cannot access the file
because it is being used by another process'). I looked inside the
shutil.move. What I found was due to this WindowsError, the usual
os.rename() inside shutil.move() fails and the file is copied using
copy2(src, dst). Finally os.unlink() also failed. (So though copying
occurred, deleting the source file failed)

I am not getting why this error comes. Has anyone faced similar
situation? Please help me.

Thanks and Regards
Roopesh



More information about the Python-list mailing list