twisted: problem with sftp-client

[Jean-Paul Calderone]

On Wed, 23 Jan 2008 15:43:47 +0100, Kristian Domke <news at neither-nor.net> wrote:
>Hello Folks,
>
>I don't know, if it is ok to post large portions of code, but I have
>really no idea where the problem lies, so I don't have much of a choice.
>
>I am programming a tool, which in the end shall connect to an
>sftp-server, take the list of files in a specified directory, searches
>for some special names and mailes them to different persons.
>
>I am only at the start at the moment. As orientation I use the cftp.py
>programm from twisted.conch with some alterations because of not having
>userinteraction.
>
>At its current state the programm should do nothing but getting the
>listing of files from a server. It works fine with the cfto.py, so the
>server is ok.
>
>But while the cftp.py script gets 'files' as an exceptions.EOFError
>exeption in the StdioClient._cbReadFile at the end of the listing, I
>just get another list and run into wall (aka Traceback):
>
>> 2008/01/23 16:14 +0200 [SSHChannel session (0) on SSHService
>> ssh-connection on SimpleTransport,client] Unhandled Error
>> 	Traceback (most recent call last):
>> 	  File "/usr/lib/python2.5/site-packages/twisted/python/log.py", >
>line 48, in callWithLogger
>> 	    return callWithContext({"system": lp}, func, *args, **kw)
>> 	  File "/usr/lib/python2.5/site-packages/twisted/python/log.py", >
>line 33, in callWithContext
>> 	    return context.call({ILogContext: newCtx}, func, *args,
>> **kw)
>> 	  File
>> "/usr/lib/python2.5/site-packages/twisted/python/context.py", line 59,
>> in callWithContext
>> 	    return self.currentContext().callWithContext(ctx, func,
>> *args, **kw)
>> 	  File
>> "/usr/lib/python2.5/site-packages/twisted/python/context.py", line 37,
>> in callWithContext
>> 	    return func(*args,**kw)
>> 	--- <exception caught here> ---
>> 	  File
>> "/usr/lib/python2.5/site-packages/twisted/conch/ssh/filetransfer.py",
>> line 52, in dataReceived
>> 	    f(data)
>> 	  File
>> "/usr/lib/python2.5/site-packages/twisted/conch/ssh/filetransfer.py",
>> line 694, in packet_STATUS
>> 	    msg, data = getNS(data)
>> 	  File
>> "/usr/lib/python2.5/site-packages/twisted/conch/ssh/common.py", line
>> 39, in getNS
>> 	    l, = struct.unpack('!L',s[c:c+4])
>> 	  File "struct.py", line 87, in unpack
>> 	    return o.unpack(s)
>> 	struct.error: unpack requires a string argument of length 4
>
>
>I have no Idea, and hope here is someone who can help me.
>
>Kristian
>
> [snip]
>
>    def _cbOpenList(self, directory):
>        files = []
>        log.msg('direc.:%s' % str(directory))
>        log.msg('direc.:%s' % type(directory))
>        log.msg('direc.:%s' % str(directory.parent))
>        log.msg('direc.:%s' % type(directory.parent))
>        d = directory.read()
>        d.addCallback(self._cbReadFile, files, directory)
>        d.addErrback(self._ebRaeadFile, files, directory)

You typo'd the errback method name above.  This doesn't cause the problem
you're having, though.

There appears to be a bug in FX_EOF handling in filetransfer.py which
results in the problem you see.  It's not completely clear to me why
cftp.py doesn't encounter the same issue.  I filed #3009 in the Twisted
issue tracker for this problem.  I twiddled the source a bit and seem to
have fixed it, so I think the issue should be easily resolvable.  If you
want, you can help out with this by writing unit tests for the issue and
submitting a patch.

Contributing to Twisted is described on this page:

  http://twistedmatrix.com/trac/wiki/TwistedDevelopment

Jean-Paul