using strings from extension module question + possible documentation error

[Stefan Sonnenberg-Carstens]

Am 24.12.2010 13:31, schrieb Oleg Leschov:
> Hi,
>
> I am writing an extension module in which I want to do some heavy
> number crunching on large amount of data.
>
> The input data is prepared using Python code and passed to this
> extension module via strings containing binary data, and the result is
> also passed back as a list of pretty large strings containing binary
> data.
>
> So first I thought I'll just pass the empty strings made in Python
> along with input data strings, so my extension code would just fill
> those empty strings with the results.
>
> But then I read the PyString docs, it says I must not modify any
> strings even though it seems to be possible...
strings are immutable.
If you pass a string and the underlying C module changes it's contents, 
this idiom is broken.
The source for endless pain ...
> Ok then I decided I'll create a list and fill it with strings from my
> C extension code..
>
> However to avoid data copying I wish to fill the newly created (using
> PyString_FromString(NULL,x) ) strings' buffers (obtained with
> PyString_AsString) with data after creating them in my extension
> module.
You could as an alternative just use byte arrays. These are changeable.
> The question is - is this allowed? Because the doc says this
>> The data must not be modified in any way, unless the string was just created using PyString_FromStringAndSize(NULL, size).
Once the string has been used by python code, you should not change it.
strings are immutable in python, so that every operation on a string 
returns a new one.
See above.
> but what exactly does "just created" mean? will it not be considered
Just created means just that. It is created and not been passed back to 
the interpreter.
So long you may change it.
> "just created" if I call any more Python stuff after
> PyString_FromString, like another PyString_FromString along with
> PyString_AsString? Which I certainly intend to do since I first create
> all strings I want, and then do my calculations which fill those with
> actual data.
>
>
> Another question is this - why does PyString_AsString doc states that
>> Return a NUL-terminated representation of the contents of string.
> when strings may contain binary data and thus NOT NUL-terminated in
> general? is this a documentation error or I can't access binary
> strings using PyString_AsString ?
Read carefully: NUL-terminated representation of the contents of the string.
It may contain other data, but the C-API will take care that this 
"contract" will be valid.
> P.S. the doc quotes are from
> http://docs.python.org/release/2.6.6/c-api/string.html
There do they live, indeed.