marshal.dumps quadratic growth and marshal.dump not allowing file-like objects

[Christian Heimes]

Raymond Hettinger wrote:
> When more space is needed, the resize operation over-allocates by
> double the previous need plus 1K.  This should give amortized O(1)
> performance just like list.append().
> 
> However, when that strategy requests more than 32Mb, the resizing
> becomes less aggressive and grows only in 1MB blocks and giving your
> observed nasty quadratic behavior.

The marshal code has been revamped in Python 2.6. The old code in Python
2.5 uses a linear growth strategy:

        size = PyString_Size(p->str);
        newsize = size + 1024;
        if (_PyString_Resize(&p->str, newsize) != 0) {
                p->ptr = p->end = NULL;
        }

Anyway marshal should not be used by user code to serialize objects.
It's only meant for Python byte code. Please use the pickle/cPickle
module instead.

Christian