How to install on Windows?

[Gregory (Grisha) Trubetskoy]

On Fri, 6 Dec 2002, Wolfgang Strobl wrote:

> "Gregory (Grisha) Trubetskoy" <grisha at ispol.com>:
>
> >> The problem _I_ am struggling with right now is how to write an output
> >> filter with mod_python for processing very large files, without reading
> >> the whole file into memory. Any suggestions?
> >
> >An apache *filter* has no notion of file size,
>
> That's fine with me. Replace "file" by "conent to be filtered".
>
> >it just processes data as
> >it is written out and has no control over how much data is passed through
> >it.
>
> I'm asking wether it reads the whole input into memory before giving it
> to my output filter, or perhaps collects all the written output before
> actually delivering it, and how to avoid that behaviour.

This depens on what "it" is, and "it" can be a number of things, it could
be a standard apache handler that reads a static file and sends it out, or
it could be output from a cgi script. But you can't control how much is
read and written from within a filter, you are at "it"'s mercy.

> >You are probably referring to a handler. But in any event I don't see a
> >problem - read a little from the file, write to the network, repeat -
> >there is no reason to read large files into memory.
>
> Didn't work. I read the input in small chunks (filter.read(1024) or
> sumething) and even added a flush after the write. but it behaved no
> differently from doing a .read() followed from a single write(): the
> memory footprint of the process grew beyond belief, before I killed the
> process.

It's possible that there is a bug of some kind - after all filters are new
in mod_python. There is quite a number of variables at play here... If the
output isn't in chunked encoding, then the content length filter will
cache the data until end of stream.

I suggest you take this discussion to the mod_python list, you will get a
much better response there. Make sure to provide as much detail as
possible, as described here:

http://www.modpython.org/live/mod_python-3.0.1/doc-html/inst-trouble.html

Grisha