Python - CGI-BIN - Apache Timeout Problem

[Sean Cavanaugh (scavanau)]

Hey Chris,

Thanks for your quick reply!  I switched my code to->

proc = subprocess.Popen(['/usr/local/bin/python', 'tests.py'], stdout=subprocess.PIPE)
out, err = proc.communicate()
print out
proc.stdout.close()

It still dead locked.  Interestingly enough When I did a #python tests.py on the command line even that was taking awhile to print out to the command line so I decided to restart my webserver... wow from mucking before something must have been running in the background still.  I got the script down to 2 seconds or so... 

Now it still works but faster when I do #python main.py it generates all the text to the command line but the website still hangs when I go to http://webserver/main.py... I am not sure what is going wrong... no error in the /var/log except for the eventual timeout after a couple minutes goes by.

-S

-----Original Message-----
From: chris at rebertia.com [mailto:chris at rebertia.com] On Behalf Of Chris Rebert
Sent: Friday, March 02, 2012 3:23 PM
To: Sean Cavanaugh (scavanau)
Cc: python-list at python.org
Subject: Re: Python - CGI-BIN - Apache Timeout Problem

On Fri, Mar 2, 2012 at 12:09 PM, Sean Cavanaugh (scavanau)
<scavanau at cisco.com> wrote:
<snip>
> THE PROBLEM:
>
> When I execute the scripts from the command line (#python main.py) it
> generates it fine (albeit slowly), it prints all the html code out including
> the script.  The ‘core’ part of the script dumbed down to the lowest level
> is->
>
>         proc = subprocess.Popen(['/usr/local/bin/python', 'tests.py'],
> stdout=subprocess.PIPE)
>         output = proc.stdout.read()

Note the red warning box about possible deadlock with .stdout.read()
and friends:
http://docs.python.org/library/subprocess.html#popen-objects

>         print output
>         proc.stdout.close()

As the docs advise, try using .communicate()
[http://docs.python.org/library/subprocess.html#subprocess.Popen.communicate
] instead:
    proc = subprocess.Popen(…)
    out, err = proc.communicate()
    print out

> When I open main.py and execute the script it just hangs… it seems to
> execute the script (I see pcap fires on the interface that I am testing on
> the firewall) but its not executing correctly… or loading the entire
> webpage…the webpage keeps chugging along and eventually gives me an error
> timeout.

The hanging makes me suspect that the aforementioned deadlock is occurring.

Cheers,
Chris
--
http://chrisrebert.com