[Tutor] How to make ftplib show progress while uploading a large file

[Terry Carroll]

On Mon, 5 Feb 2007, Terry Carroll wrote:

> On Sun, 4 Feb 2007, [ISO-8859-1] Magnus Wirström wrote:
> 
> > I'm workinga on a program that will upload a large file to a server 
> > using ftp. I'm using ftplib to do this. I'm using a gui with wxpython 
> > and i would like to have a progressbar showing in % how much have been 
> > transfered. I have been googling but i can't make any sense of what i 
> > have found. does anyone have a good example or could explain how to do 
> > this ?
> 
> Magnus --
> 
> When you installed wxPython, did you also download and install the "Docs, 
> Demo, Samples, etc."?  There are great examples of nearly all wxPython 
> dialogs, including ProgressDialog.
> 
> The hard part is going to get ftplib to talk to your dialog.  You're 
> uploading.  ftplib's download methods (retrbinary and retrlines) include a 
> callback option, which would let you update the progress as you went, but 
> the upload methods (storbinary and storlines) does not.[1]
> 
> You could either a) install the patch from [2] on your system (or override 
> storbinary and/or storlines as appropriate; 

I couldn't resist.  This intrigued me, partially because I will have a 
similar problem in a project I have coming up.  Here's an example.  Most 
of this is derived either from the callback patch I mentioned or from 
the wxPython sample code.  Very little is actual original thought.

First, set up ftplib, but with Phil Schwartz's patch to support the 
callback argument.  

############## begin code ###############################

import ftplib

def my_storlines(self, cmd, fp, callback=None):   # patched for callback
    '''Store a file in line mode.'''
    CRLF = ftplib.CRLF                            # patched for callback
    self.voidcmd('TYPE A')
    conn = self.transfercmd(cmd)
    while 1:
        buf = fp.readline()
        if not buf: break
        if buf[-2:] != CRLF:
            if buf[-1] in CRLF: buf = buf[:-1]
            buf = buf + CRLF
        conn.sendall(buf)
        if callback: callback(buf)                 # patched for callback
    conn.close()
    return self.voidresp()

ftplib.FTP.storlines = my_storlines  # use the patched version

############## end code ###############################

In the above, I just copied storlines out of ftplib.py, and edited it
where the "patched for callback" comments are.  Note: this is for a text
file upload; the same would work for a binary file by patching storbinary
instead.


Now: set up a class that will start a wxPython ProgressDialog, but with 
a callback method added:


############## begin code ###############################
class FTPProgressDialog:
    
    def __init__(self, fname):
        import wx, os
        statinfo = os.stat(fname)
        self.filesize = statinfo.st_size
        self.so_far = 0
        self.app = wx.PySimpleApp()
        self.dlg = wx.ProgressDialog("Upload progress",
                       fname+":",
                       maximum = self.filesize,
                       style = wx.PD_APP_MODAL
                        | wx.PD_ELAPSED_TIME
                        | wx.PD_ESTIMATED_TIME
                        | wx.PD_REMAINING_TIME
                        )

    def asciicallback(self, buffer):
        """
        just used for testing, w/o wxPython
        """
        self.so_far = self.so_far+len(buffer)-1
        pct = float(self.so_far)/self.filesize
        print "so far:", self.so_far, pct
        return

    def wxcallback(self, buffer):
        self.so_far = self.so_far+len(buffer)-1
        self.dlg.Update(self.so_far)
        return

    def close(self):
        self.dlg.Destroy()
        self.app.Destroy()

############## end code ###############################

The code above is not pretty; and it would have been cleaner to 
have FTPProgressDialog inherit from wx.ProgressDialog.  Because the 
class actually starts a wx app, I'll bet you'd run into trouble if you 
tried to use this in an existing wx application.  But that didn't 
occur to me until I was nearly done and starting to get bored.  :-)

Now, to use them:


############## begin code ###############################

filename = "samplefile.txt"

ftpconn = ftplib.FTP('127.0.0.1')  # small local FTP server
ftpconn.set_pasv(False)            # my server doesn't support PASV
ftpconn.login()
trans_file = open(filename)
FTPprogress = FTPProgressDialog(filename)
ftpconn.storlines("STOR "+filename, trans_file, callback=FTPprogress.wxcallback)
ftpconn.quit
FTPprogress.close()

############## end code ###############################

And that, as they say, is that.