[IronPython] Portable use of pickle.dumps()

[Michael Foord]

Michael Foord wrote:
> [snip...]
> Here is an example of getting a byte array from a binary pickle in 
> IronPython:
>
> >>> import pickle
> >>> class A(object):
> ...  b = 'hello'
> ...  c = (None, 'fish', 7.2, 7j)
> ...  a = {1: 2}
> ...
> >>> p = pickle.dumps(A(), protocol=2)
> >>> p
> u'\x80\x02c__main__\nA\nq\x00)\x81q\x01}q\x02b.'
> >>> from System import Array, Byte
> >>> a = Array[Byte](tuple(Byte(ord(c)) for c in p))
> >>> a
> Array[Byte]((<System.Byte object at 0x0000000000000033 [128]>, 
> <System.Byte obje...
>

And the converse:

 >>> p2 = ''.join(chr(c) for c in a)
 >>> a2 = pickle.loads(p2)
 >>> a2
<A object at 0x000000000000004E>
 >>> a2.a
{1: 2}
 >>> a2.b
'hello'
 >>> a2.c
(None, 'fish', 7.2, 7j)


Michael


> I hope this is at least slightly helpful. :-)
>
> Michael
>
>
>
>> My basic issue is that the 'str' unavoidably implies certain 
>> semantics when
>> calling .NET APIs from IronPython. These APIs interpret str as text 
>> rather
>> than just bytes, which therefore gets transformed by various text 
>> encodings,
>> such as UTF-8 to UTF-16. Such encodings are undesirable for my 
>> pickled data
>> since the result is no longer necessarily a valid pickle.   I suppose 
>> the
>> intention in Python 3.0 is that 'bytes' doesn't carry any semantics 
>> with it,
>> its just data, which is why pickle.dumps() in Python 3.0 returns bytes
>> rather than str.
>>
>> I want to push plain old byte arrays into the database from both 
>> CPython and
>> IronPython, so I can avoid any head-scratching confusion with database
>> adapters and/or databases inappropriately encoding or decoding my data.
>>
>>  
>>> For example "data = [ord(c) for c in some_string]" has behaved as 
>>> expected many times for me in IronPython (and could help you turn 
>>> strings into bytes).
>>>     
>>
>> Thanks. I'll try something based on that.
>>
>>  
>>> Is this a theoretical problem at this stage or an actual problem?
>>>     
>>
>> Its an actual problem with SQLiteParameter.Value from the SQLite ADO.NET
>> provider.  I think our original CPython code is a bit sloppy with 
>> respect to
>> the distinction between text strings and byte arrays, so I'll 
>> probably need
>> to tighten things up on both sides.
>>
>> Would you agree tha using unicode() and bytes() everywhere and avoiding
>> str() gives code that has the same meaning in Python 2.6, IronPython 
>> 2.6 and
>> Python 3.0?  Do you think this would be a good guideline to follow 
>> until we
>> can leave Python 2.x behind?
>>
>> Many thanks,
>>
>> Rob
>>
>>
>>
>>   
>
>


-- 
http://www.ironpythoninaction.com/