[Patches] [ python-Patches-1434657 ] Patch to support lots of file descriptors

[SourceForge.net]

Patches item #1434657, was opened at 2006-02-19 17:34
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1434657&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Core (C code)
Group: Python 2.4
>Status: Closed
>Resolution: Rejected
Priority: 5
Private: No
Submitted By: Sven Berkvens-Matthijsse (svenberkvens)
Assigned to: Nobody/Anonymous (nobody)
Summary: Patch to support lots of file descriptors

Initial Comment:
Off-the-shelf Python on FreeBSD (and probably other
OS'es as well) has a limit on the number of file
descriptors that select can use. This is limited by the
size of the fdset structures.

On FreeBSD, and probably in many OS'es, a user can set
the size of the fdset structures if he defines
FD_SETSIZE before he includes sys/types.h.

My patch sets the FD_SETSIZE to a substantial number.


----------------------------------------------------------------------

>Comment By: Martin v. Löwis (loewis)
Date: 2007-02-14 11:12

Message:
Logged In: YES 
user_id=21627
Originator: NO

Apparently, there is no activity on this patch, so I'm eventually
rejecting it.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2006-02-28 23:48

Message:
Logged In: YES 
user_id=21627

This test is not good enough. On systems where FD_SETSIZE is not
"settable", 
it still is a macro, and setting it will succeed. The problem is that on
these 
systems, select(2) might refuse to process more than FD_SETSIZE file 
descriptors, and/or fd_set might just buffer-overrun.

In these cases, the cure would be worse than the desease: you would be
able 
to pass that many file descriptors to select, but the system would never
tell 
you when they are ready.

If you want to use many file descriptors, just use poll(2) instead of
select(2).

----------------------------------------------------------------------

Comment By: Sven Berkvens-Matthijsse (svenberkvens)
Date: 2006-02-28 09:51

Message:
Logged In: YES 
user_id=175845

Oops, 10000 should obivously match 10240 here...

Sven

----------------------------------------------------------------------

Comment By: Sven Berkvens-Matthijsse (svenberkvens)
Date: 2006-02-28 09:50

Message:
Logged In: YES 
user_id=175845

Is it perhaps possible to add a test to the configuring
process that tests whether something like:

#define FD_SETSIZE 10240
#include <sys/types.h>
#include <unistd.h>

int main(int argc, char **argv)
{
#if FD_SETSIZE == 10000
    return 0;
#else
    return 1;
#endif
}

can be preprocessed, compiled and run correctly? This will
most probably fail on systems where FD_SETSIZE is not
settable, and succeed on systems where it is.

Thanks for responding,
Sven

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2006-02-27 22:54

Message:
Logged In: YES 
user_id=21627

That patch is wrong: you should only change the value of FD_SETSIZE on
systems 
where this actually works. Unfortunately, I cannot think of a method to
determine 
this reliably.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1434657&group_id=5470