checking a thread has started

[Michael Fuhr]

Deepak Sarda <deepak.sarda at gmail.com> writes:

> The server is on Redhat with a custom 2.4.20 kernel. The script is not
> running through mod_python - just a regular #!/usr/bin/python2.2

The shebang line doesn't control how the web server starts the
script, so that doesn't tell us anything.  Does the web server have
mod_python installed at all?  If so then it's possible that all
files ending in .py or all files in a certain directory are run
under mod_python.

You can examine os.environ['GATEWAY_INTERFACE'] to see how your
script is run: the value should be "CGI/1.1" if you're running
via CGI and "Python-CGI/1.1" if you're under mod_python.  This
environment variable probably won't be set if you run the script
from the command line, so use has_key() or trap KeyError to avoid
exceptions.

It may or may not matter whether you're running under mod_python,
but let's find out for sure anyway.  Then we can better understand
the extent of relationship between your script and Apache.

> I see what you mean and it's true that the script actually runs as a
> local user.

What do you mean "runs as a local user"?  CGI programs usually run
as whatever user the web server runs as, although it's possible to
have them run as the script's owner (e.g., by setting up suEXEC).

> But there is certainly a relation between the server and
> the threads because in httpd.conf - there's a directive which says:
>
> StartServers 8
>
> which means there are 8 idle server instances running at any given
> time.

That's not what StartServers means according to the Apache
documentation:

http://httpd.apache.org/docs/mod/core.html#startservers

"The StartServers directive sets the number of child server processes
created on startup.  As the number of processes is dynamically
controlled depending on the load, there is usually little reason
to adjust this parameter."

> Now I assume  my main script call take up one instance. Then
> when new threads are requested - each thread takes up an instance so
> we have 7 new threads. Which is exactly what happens!

That doesn't make sense given the description of StartServers.
Besides, as I recall, Apache 1.3.27, which you said you were using,
doesn't even use threads -- each instance of the web server is run
in a separate process.  Have you tested your hypothesis by altering
the value of StartServers and checking whether your script behaves
differently?

StartServers is 5 on one of my machines, yet I just ran a few
tests and successfully created 1000 threads under both CGI
and mod_python.

> The program _always_ successfully creates and completes  execution of
> exactly seven new threads leaving the remaining eight requested
> threads in no man's land!

How are you determining which threads are starting and which aren't?

> So there must be some relation between apache and the script. Also, as
> I said earlier, when run through a shell as the same user - the script
> works perfectly fine.

Something's going on but we lack sufficent evidence to say what.
We haven't seen your code yet -- could you post the smallest possible
script that demonstrates the problem?

-- 
Michael Fuhr
http://www.fuhr.org/~mfuhr/