feature request / max size limit on StringIO

[Bengt Richter]

On Tue, 25 Oct 2005 14:15:02 -0400, "Clark C. Evans" <cce at clarkevans.com> wrote:

>Hello.  I've not been able to use cStringIO since I have the need to
>ensure that the memory buffers created are bounded within a resonable
>limit set by specifications.  No, this code does not properly belong
>in my application as the modules that use "files" should not have
>to care about any resource limitations that may be imposed.
>
>class LimitedBuffer(StringIO):
>    def __init__(self, buffer = None, maxsize = 5 * 1000 * 1000):
>        StringIO.__init__(self,buffer)
>        self.cursize = 0
>        self.maxsize = maxsize
>    def write(self,str):
>        self.cursize += len(str)
>        if self.cursize > self.maxsize:
>            raise IOError("allocated buffer space exceeded")
>        return StringIO.write(self,str)

You might want to use StringIO's own knowledge of its writing position (self.pos or self.tell())
and then you wouldn't need cursize, nor to worry about whether seek has been used to
reposition, maybe writing the same section of file over and over for some reason.

I'm not sure whether seek beyond the end actually causes allocation, but I don't think so.
So you should be ok just checking self.pos+len(strarg)> self.maxsize.
There's also some interesting things going on with self.buf and self.buflist attributes, which
probably is not doing as simple a job of buffering as you might think. Maybe you'll want to wrap
a byte array or an mmap instance to store your info, depending on what you are doing?

Regards,
Bengt Richter