Tips on Speeding up Python Execution

[Abhijeet Mahagaonkar]

Thats awesome. Its time I migrate to 3 :)

On Fri, Apr 8, 2011 at 11:29 PM, Raymond Hettinger <python at rcn.com> wrote:

> On Apr 8, 12:25 am, Chris Angelico <ros... at gmail.com> wrote:
> > On Fri, Apr 8, 2011 at 5:04 PM, Abhijeet Mahagaonkar
> >
> > <abhijeet.mano... at gmail.com> wrote:
> > > I was able to isolate that major chunk of run time is eaten up in
> opening a
> > > webpages, reading from them and extracting text.
> > > I wanted to know if there is a way to concurrently calling the
> functions.
> >
> > So, to clarify: you have code that's loading lots of separate pages,
> > and the time is spent waiting for the internet? If you're saturating
> > your connection, then this won't help, but if they're all small pages
> > and they're coming over the internet, then yes, you certainly CAN
> > fetch them concurrently. As the Perl folks say, There's More Than One
> > Way To Do It; one is to spawn a thread for each request, then collect
> > up all the results at the end. Look up the 'threading' module for
> > details:
> >
> > http://docs.python.org/library/threading.html
>
> The docs for Python3.2 have a nice example for downloading multiple
> webpages in parallel:
>
>
> http://docs.python.org/py3k/library/concurrent.futures.html#threadpoolexecutor-example
>
> Raymond
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20110409/38434562/attachment-0001.html>