marshal vs pickle

[Jean-Paul Calderone]

On Thu, 01 Nov 2007 21:15:06 -0000, Aaron Watters <aaron.watters at gmail.com> wrote:
>On Nov 1, 4:59 pm, Jean-Paul Calderone <exar... at divmod.com> wrote:
>> On Thu, 01 Nov 2007 20:35:15 -0000, Aaron Watters <aaron.watt... at gmail.com> wrote:
>> >On Nov 1, 2:15 pm, Raymond Hettinger <pyt... at rcn.com> wrote:
>> >> On Nov 1, 4:45 am, Aaron Watters <aaron.watt... at gmail.com> wrote:
>>
>> >> > Marshal is more secure than pickle
>>
>> >> "More" or "less" make little sense in a security context which
>> >> typically is an all or nothing affair.  Neither module is designed for
>> >> security.  From the docs for marshal:
>>
>> >> '''
>> >> Warning: The marshal module is not intended to be secure against
>> >> erroneous or maliciously constructed data. Never unmarshal data
>> >> received from an untrusted or unauthenticated source.
>> >> '''
>>
>> >> If security is a focus, then use xmlrpc or some other tool that
>> >> doesn't construct arbitrary code objects.
>>
>> >I disagree.  Xmlrpc is insecure if you compile
>> >and execute  one of the strings
>> >you get from it.  Marshal is similarly insecure if you evaluate a code
>> >object it hands you.  If you aren't that dumb, then neither one
>> >is a problem.  As far as I'm concerned marshal.load is not any
>> >more insecure than file.read.
>>
>> You're mistaken.
>>
>>   $ python
>>   Python 2.4.3 (#2, Oct  6 2006, 07:52:30)
>>   [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
>>   Type "help", "copyright", "credits" or "license" for more information.
>>   >>> import marshal
>>   >>> marshal.loads('RKp,U\xf7`\xef\xe77\xc1\xea\xd8\xec\xbe\\')
>>   Segmentation fault
>>
>> Plenty of other nasty stuff can happen when you call marshal.loads, too.
>
>I'll grant you the above as a denial of service attack.  You are right
>that I was mistaken in that sense.  (btw, it doesn't core dump for
>2.5.1)
>
>That is/was a bug in marshal.  Someone should fix it.  Properly
>implemented,
>marshal is not fundamentally insecure.

One can then ask the question of whether or not marshal is properly
implemented in any extant version of CPython. ;)  It isn't much
comfort to know that marshal is ideologically sound after someone
uses it to exploit your service.

>Can you give me an example
>where someone can erase the filesystem using marshal.load?  I saw one
>for pickle.load once.
>

Many bugs which lead to a segfault can also be exploited to execute
arbitrary code.  Not all such bugs can be.  I haven't looked closely
at the marshal source code to determine if it can be or not in this
case.

My observations agree with yours, for what it's worth.  A cursory
investigation doesn't reveal any inputs which cause segfaults in
trunk at HEAD with marshal.loads(), although there are still many which
will cause it to allocate huge amounts of memory, another effective
DoS attack.

Jean-Paul