(embedding) running Python as coprocess

[William Park]

Jp Calderone <exarkun at intarweb.us> wrote:
> [-- text/plain, encoding quoted-printable, 42 lines --]
> 
> On Wed, Feb 19, 2003 at 09:40:09PM +0000, William Park wrote:
>> I've managed to fully embed Python into Bash.  But, the result is 1.4Meg
>> in size.  I'm wondering if there is better and more general way of
>> running Python as coprocess in the background.
>> 
>> That is, with Python running in the background, I want to send one batch
>> of commands to Python and read the result back, send another batch and
>> read the result, and so on.  Python should not terminate between
>> batches, because I want to access data from previous calls.
>> 
>> I've tried something like
>>     python <fifo.in >fifo.out &
>>     echo "print 2.0+3.0" > fifo.in
>>     cat fifo.out
>> I get the result, but Python terminates as soon as EOF is reached in the
>> <stdin> stream.
>> 
>> My embedded Python will stay alive between batches.  But, this solves
>> problem only for Python.  Any insight to solving this problem for a any
>> external program (ie. Perl, Ruby, etc.) would be greatly appreciated.
>> 
> 
> Instead of running -just- Python, run a Python program.  It can attach
> itself to a couple fifos (or a unix socket, or a tcp socket, or ...),
> read until EOF, process the string (easy, with exec/eval), write the
> results out, then re-open the input file.

Thanks Jp.  I overlooked 'exec' function in Python.

I was initially thinking about writing a C wrapper program, which will
read the batch commands from 'stdin' (redirected from FIFO), pass it to
PyRun_SimpleString(), and repeat.  The result will in 'stdout'
(redirected to FIFO), and reading it would be up to the calling program.  

But, somehow, I ended up with full-blown embedded Python within Bash
shell. :-)

> 
> There's also a project that keeps a Python interpreter running and
> lets you use a mini-interpreter (for the #!) to connect to it, but I
> forget the name.  It was mentioned on this group a week or two ago, so
> maybe you can find it w/ a little searching.

-- 
William Park, Open Geometry Consulting, <opengeometry at yahoo.ca>
Linux solution for data management and processing.