Unpythonic Python

[David Abrahams]

I started having some weird problems with Python recently; they're so
weird that I can't begin to explain them.  All I can do is describe
the symptoms and hope someone else has a clue.  So here goes:

FreeBSD 4.2, Python 2.2.2. 
 
I have a nightly cron job that downloads the boost cvs tarball from
SourceForge and bunzip2s it.  For about a year everything worked with
no problems. About a month ago the download started getting truncated
with no error reported.  Then bunzip2 reports corruption, of course.

I took the salient part of the download script, and added a reporthook
(undocumented in urllib, BTW) to the urlretrieve call:
--
import urllib
import os

def dump(*args):
    print args

#print 'downloading...'
os.chdir('/tmp')
urllib.urlretrieve('http://cvs.sourceforge.net/cvstarballs/boost-cvsroot.tar.bz2',
                  'boost-cvsroot.tar.bz2', dump)
---

When a recent download was truncated, the last lines of the dump were:

(1014, 8192, 34441987)
(1015, 8192, 34441987)
(1016, 8192, 34441987)
(1017, 8192, 34441987)
(1018, 8192, 34441987)
(1019, 8192, 34441987)
(1020, 8192, 34441987)
(1021, 8192, 34441987)
(1022, 8192, 34441987)
(1023, 8192, 34441987)

is 1023 a coincidence?  Maybe; here's the tail of another failure:

(2439, 8192, 34455413)
(2440, 8192, 34455413)
(2441, 8192, 34455413)
(2442, 8192, 34455413)
(2443, 8192, 34455413)
(2444, 8192, 34455413)
(2445, 8192, 34455413)
(2446, 8192, 34455413)
(2447, 8192, 34455413)
(2448, 8192, 34455413)

So I figured maybe we needed a newer version of Python.  I asked my
sysadmin at stlport.com to upgrade Python to the most recent release,
and all of a sudden my incoming mail started looping (see below).

I am classifying spam with SpamBayes and on my system the only way to
get it sorted into IMAP folders after classification is to send it to
myself.  Only messages lacking an X-Spambayes-Classification get
classified and sent back out, so I guess when Python was upgraded the
classification stopped adding the headers?  My sysadmin rolled Python
back to 2.2.2 and the mail problems stopped.  But I still have the
truncated download problem.

Any clues?
Thanks in advance!

-Dave

--

From: <MAILER-DAEMON at stlport.com>
Subject: Undeliverable mail: RE: What's wrong with this?
To: <agurtovoy at meta-comm.com>
Date: Mon, 23 Aug 2004 17:08:45 -0700

Failed to deliver to 'dave'
mail loop: too many hops (too many 'Received:' header fields)


Reporting-MTA: dns; stlport.com

Original-Recipient: rfc822;<dave>
Final-Recipient: system;<dave>
Action: failed
Status: 5.0.0
[3. text/rfc822-headers]

Received: by stlport.com (CommuniGate Pro PIPE 4.2)
  with PIPE id 817189; Mon, 23 Aug 2004 17:08:45 -0700
Received: by stlport.com (CommuniGate Pro PIPE 4.2)
  with PIPE id 817183; Mon, 23 Aug 2004 17:08:26 -0700
<snip>
Received: from [12.163.41.8] (HELO expressmail.office.meta)
  by stlport.com (CommuniGate Pro SMTP 4.2)
  with SMTP id 817122 for dave at boost-consulting.com; Mon, 23 Aug 2004
17:04:16 -0700
Received-SPF: error
 receiver=stlport.com; client-ip=12.163.41.8;
envelope-from=agurtovoy at meta-comm.com
Received: by expressmail.office.meta with Internet Mail Service
(5.5.2653.19)
	id <RGXKFLXQ>; Mon, 23 Aug 2004 19:03:42 -0500
Message-ID: <838645D2D4A9A64ABD56D8C001F07E023533F9 at expressmail.office.meta>
From: Aleksey Gurtovoy <agurtovoy at meta-comm.com>
To: 'David Abrahams' <dave at boost-consulting.com>
Subject: RE: What's wrong with this?
Date: Mon, 23 Aug 2004 19:03:42 -0500
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"