fork seems to make urlopen into a black hole?

BobAalsma overhaalsgang_24_bob at me.com
Tue Jan 14 15:04:42 EST 2014


A program took much too long to check some texts collected from web pages.
As this could be made parallel easily, I put in fork.
And the result seems to be that the program simply stops in the line with urlopen. Any suggestions?

Relevant part:

try:
            print 'urlopen by', kind_nummer, '- before'
            extern_adres = urlopen(adres).geturl()
            print 'urlopen by', kind_nummer, '- after'
        except IOError:
            tekst_string = 'URL "' + adres +'" geeft IO Error'
            print tekst_string, 'door', kind_nummer
            bloklijst_verslag.append(tekst_string + '\n')
            tekst_string = '\t**** Deze pagina wordt overgeslagen\n'
            bloklijst_verslag.append(tekst_string + '\n')
            adres = ''
        except:
            fout = sys.exc_info()[:2]
            tekst_string = 'URL "' + adres + '" geeft fout ' + fout
            print tekst_string, 'door', kind_nummer
            bloklijst_verslag.append(tekst_string + '\n')
            tekst_string = '\t**** Deze pagina wordt overgeslagen\n'
            bloklijst_verslag.append(tekst_string + '\n')
            adres = ''
        else:
            print 'urlopen by', kind_nummer, '- else'
            if extern_adres != adres:
                tekst_string = '\t**** redirect naar ' + extern_adres
                bloklijst_verslag.append(tekst_string + '\n')

>From this block, the only response seen is the first print statement ("url open by .... - before") and then nothing else seems to happen in that child.
This happens for all children: the expected number of children is reached and they all reach (only) this point.



More information about the Python-list mailing list