Too big of a list? and other problems

Brian bnblazer at gmail.com
Mon May 22 20:19:12 EDT 2006


First off, I am sorry for cluttering this group with my inept
questions, but I am stuck again despite a few hours of hair pulling.

I have a function (below) that takes a list of html pages that have
images on them (not porn but boats).  This function then (supposedly)
goes through and extracts the links to those images and puts them into
a list, appending with each iteration of the for loop.  The list of
html pages is 82 items long and each page has multiple image links.
When the function gets to item 77 or so, the list gets all funky.
Sometimes it goes empty, and others it is a much more abbreviated list
than I expect - it should have roughly 750 image links.

When I looked at it while running, it appears as if my regex is
actually appending a tuple (I think) of the results it finds to the
list.  My best guess is that the list is getting too big and croaks.
Since one of the objects of the function is also to be able to count
the items in the list, I am getting some strange errors there as well.

Here is the code:

def countPics(linkList):
    foundPics = []
    count = 0
    for link in linkList:
        picPage =
urllib.urlopen("http://continuouswave.com/whaler/cetacea/" +
                                 link)
        count = count +1
        print 'got page', count
        html = picPage.read()
        picPage.close()
        pics = re.compile(r"images/.*\.jpeg")
        foundPics.append(pics.findall(html))
        #print len(foundPics)
    print "found", len(foundPics), "pictures"
    print foundPics

Again, I sincerely appreciate the answers, time and patience this group
is giving me.

Thank you for any help you can provide in showing me where I am going
wrong.
Brian




More information about the Python-list mailing list