Python crashed on me

[Bengt Richter]

On Thu, 27 Jun 2002 20:50:42 +0400, Oleg Broytmann <phd at phd.pp.ru> wrote:

>Hello!
>
>   Recently Python (one program that I am dbugging) started to crash.
>FreeBSD kills it with "Bus error", Linux with "Segmentation fault".
>
>   I think the program crashed in the cPickle.dump(file, 1), because after
>the crash the file is corrupted and cannot be load back (upon loading
>cPickle complain EOFile). The error is reproduceable.
>
>   How can I debug the situation? May be my data grew too big? The pickle
>file is about 800K.
>
>   Details. The program is my URL robot
>(http://phd.pp.ru/Software/Python/#bookmarks_db). The robot runs
>urllib.urlretrive() on my bookmarks file and report the results. Between
>runs data may be saved in different formats, including pickle.
>   While trying to understand what is going on and make full run faster I
>replaced the call to urlretrive with fake function that returns nothing.
>The robot crashed with the same segfault.
>
>   Python is
>
>Python 2.2.1 (#2, Apr 10 2002, 15:15:56)
>[GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2
>
To eliminate pickle, you could substitute a dump pickle.
If it's happening in pickle, you could substite a file object of your own,
to get a clue what it died on, e.g., you might import the following dummy
output class POUT, and first get a file object with opt=None to see if pickle
dies doing dummy output. Then use the 'n' option and some start count to get
nearer the end and see what the last number is. It should just keep printing
on the same line, so you can watch them whiz by without scrolling overhead, at least.

Then back off a few dozen, and run it with opt='all' and start=<whatever, near the end>
to get the last few objects and presumably the one it died on.

====< pout.py >========================================
class POUT:
    def __init__(self, opt=None, start=0):
        self.opt = opt
        self.start = start
        self.n = 0
    def write(self, s):
        if not self.opt: return
        self.n +=1
        if self.n < self.start: return
        if self.opt == 'n':
            print ('\r%d' % self.n),
        else:
            print '%8d: %s' % (self.n, `s`)

def test(opt='all',start=0):
    import pickle
    f = POUT(opt,start)
    print "---< pickle dump binary of 'example object' and 0x12345678 >---"
    pickle.dump('example object', f, 1)
    pickle.dump(0x12345678, f, 1)
    print "---------------------------------------------------------------"

if __name__=='__main__':
    test()
====</pout.py >========================================

HTH

Regards,
Bengt Richter