Processing a file using multithreads

Tim Roberts timr at probo.com
Sat Sep 10 01:43:19 EDT 2011


Abhishek Pratap <abhishek.vit at gmail.com> wrote:
>
>My application is not I/O bound as far as I can understand it. Each
>line is read and then processed independently of each other. May be
>this might sound I/O intensive as #N files will be read but I think if
>I have 10 processes running under a parent then it might not be a
>bottle neck.

Your conclusion doesn't follow from your premise.  If you are only doing a
little bit of processing on each line, then you almost certainly WILL be
I/O bound.  You will spend most of your time waiting for the disk to
deliver more data.  In that case, multithreading is not a win.  The threads
will all compete with each other for the disk.
-- 
Tim Roberts, timr at probo.com
Providenza & Boekelheide, Inc.



More information about the Python-list mailing list