file.close() freezes my script

[Bob Halley]

schwerdy at web.de (schwerdy) writes:

> I created a script that handles SC and CSV files. To convert CSV (or
> in my case: pipe ('|') seperated values) to SC, I want to hunt my CSV
> lines through /usr/bin/psc. Therefore I did:
> 
> tofilter, fromfilter, error = os.popen3("psc -S -L -d '|'")
> map(tofilter.write,lines)
> tofilter.close()
> 
> For some CSV-files my script freezes at tofilter.close(). Removing
> some lines from the "lines" sequence (that represents the CSV-file)
> always helps. But imho "psc" is not the cause. Piping my CSV-file
> through psc in bash always works. Isn't it crazy? I tried python2.2
> and python2.3 from the debian distribution.
> 
> Anyone who has an explanation, workaround, hint or anything?

This sounds like a classic pipe deadlock.  My guess is that the child
process is blocked trying to write to its stdout, which you haven't
read.  The parent is blocking trying to finish all that writing.

The issue is described in the Python Library Reference, section 6.8.2,
"Flow Control Issues".

If you wrote psc's output to a file, used select(), or used different
threads (e.g. a writer thread and a reader thread), you could avoid
the deadlock.

/Bob