filecmp.dircmp performance

dads wayne.dads.bell at gmail.com
Sat Jan 8 09:44:05 EST 2011


I'm creating a one way sync program, it's to automate backing up data
over the wan from our shops to a server at head office. It uses
filecmp.dircmp() but the performance seems poor to me.

        for x in dc.diff_files:
            srcfp = os.path.join(src, x)
            self.fn777(srcfp)
            if os.path.isfile(srcfp):
                try:
                    shutil.copy2(srcfp, dst)
                    self.lg.add_diffiles(src, x)
                except Exception, e:
                    self.lg.add_errors(e)

I tested it at a store which is only around 50 miles away on a 10Mbps
line, the directory has 59 files that are under 100KB. When it gets to
dc.diff_files it takes 15mins to complete. Looking at the filecmp.py
it's only using os.stat, it seems excessively long.

code:
http://pastebin.com/QskXGDQT



More information about the Python-list mailing list