[Python-Dev] (no subject)

[David Beazley]

On Mon 15/03/10  4:34 AM , "Martin v. Löwis" martin at v.loewis.de sent:
> > So, just to be clear about the my bug report, it
> is directly related> to the problem of overlapping I/O requests with
> CPU-bound processing.> This kind of scenario comes up in the context of
> many> applications--especially those based on
> cooperating processes,> multiprocessing, and message passing.
> 
> How so? if you have cooperating processes, multiprocessing, and message
> passing, you *don't* have CPU bound threads along with IO
> "bound"threads in the same process - you may not have threads at all!!!
> 

You're right in that end user will probably not be using threads in this situation.  However, threads are still often used inside the associated programming 
libraries and frameworks that provide communication.   For example, threads might be used to implement queuing, background monitoring, event notification, 
routing, and other similar low-level features.   Just as an example, the multiprocessing module currently uses background threads as part of its implementation 
of queues.   In a similar vein, threads are sometimes used in asynchronous I/O frameworks (e.g., Twisted) to handle certain kinds of deferred operations.  
Bottom line: just because a user isn't directly programming with threads doesn't mean that threads aren't there.

> > In any case, if the GIL can be improved in a way
> that is simple and> which either improves or doesn't negatively
> impact the performance of> existing applications, why wouldn't you want to
> do it?  Seems like a> no-brainer.
> 
> Unfortunately, a simple improvement doesn't really seem to exist.
> 

Well, I think this problem can be fixed in a manner that is pretty well isolated to just one specific part of Python (the GIL)--especially since several prototypes, 
including my own, have already demonstrated that it's possible.    In any case, as it stands now, the current "new GIL" ignores about 40 years of work concerning 
operating system thread/process scheduling and the efficient handling of I/O requests.   I suspect that I'm not the only one who be disappointed if the Python 
community simply blew it off and said it's not an issue.    Trying to fix it is a worthwhile goal.

Cheers,
Dave