object references/memory access

["Martin v. Löwis"]

>     I have searched a good deal about this topic and have not found
> any good information yet. It seems that the people asking all want
> something a bit different than what I want and also don't divulge much
> about their intentions. I wish to improve the rate of data transfer
> between two python programs on the same machine.

I don't understand why you want exactly this. Wouldn't it be
sufficient/better if the response time for a request as seen
by the client web browser would improve? Why is it necessary
to start optimizing at the data transfer rate?

As a starting point, I would try to eliminate the CGI part. There
are two ways to do that:
a) run the Python code inside the Apache process, and
b) use a single Python server (possibly shared with the database
   process), and connect this to Apache through the
   reverse proxy protocol.

The cost you observe might be in the repeated creation of
new processes, and the repeated establishment of new TCP
connections. Either solution would drop some of that overhead.

> I am fairly ignorant of how Apache works with the
> CGI module but here is what I'd like to do. I want to somehow let the
> server print out to the user's browser instead of the search script in
> order to cut out the time of sending the results over the socket.

That is not possible. The CGI script does not "print out to the
user's browser". Instead, it prints to its stdout, which is a pipe
being read by Apache; Apache then copies all data to the socket
going to the user's browser (possibly after manipulating the
headers also).

> Mainly, I'd like
> to know if there is any kind of descriptor or ID that can be passed
> and used by another process to print output to the user's browser
> instead of the script that Apache invoked.

No. The CGI script has a file handle, and it is not possible to pass
a file handle to a different process.

> If there is not a good Pythonic way to do the above, I am open to
> mixing in some C to do the job if that is what it takes.

No, it's not Python that fails to support that - it's the operating
system. See above for solutions that avoid one such copying in
the first place.

Regards,
Martin