[Python-Dev] Let's update CObject API so it is safe and regular!

[Guido van Rossum]

2009/4/1 Larry Hastings <larry at hastings.org>:
>
> Jim Fulton wrote:
>
> The only type-safety mechanism for a CObject is it's identity.  If you want
> to make sure you're using the foomodule api, make sure the address of the
> CObject is the same as the address of the api object exported by the module.
>
> That doesn't help.  Here's a program that crashes the interpreter, something
> I shouldn't be able to do from pure Python:
>
> import _socket
> import cStringIO
> cStringIO.cStringIO_CAPI = _socket.CAPI
>
> import cPickle
> s = cPickle.dumps([1, 2, 3])
>
> How can cPickle determine that cStringIO.cStringIO_CAPI is legitimate?

This is a bug in cPickle. It calls the PycString_IMPORT macro at the
very end of its init_stuff() function without checking for success.
This macro calls PyCObject_Import("cStringIO", "cStringIO_CAPI") which
in turn calls PyCObject_AsVoidPtr() on the object that it finds as
cStringIO.cStringIO_CAPI, and this function *does* do a type check and
sets an exception if the object isn't a PyCObject instance. However
cPickle's initialization doesn't check for errors immediately and
apparently some later code overrides the exception.

The fix should be simple: insert

  if (PyErr_Occurred()) return -1;

immediately after the line

  PycString_IMPORT;

in init_stuff() in cPickle.c. This will cause the import of cPickle to
fail with an exception and all should be well.

I have to say, I haven't understood this whole thread, but I'm
skeptical about a redesign. But perhaps you can come up with an
example that doesn't rely on this cPickle bug?

--Guido

> That would break backward compatibility. Are you proposing this for Python
> 3?
>
> I'm proposing this for Python 3.1.  My understanding is that breaking
> backwards compatibility is still on the table, which is why I wrote the
> patch the way I did.  If we have to preserve the existing API, I still think
> we should add new APIs and deprecate the old ones.
>
> It's worth noting that there's been demand for this for a long time.  Check
> out this comment from Include/datetime.h:
>
> #define PyDateTime_IMPORT \
>         PyDateTimeAPI = (PyDateTime_CAPI*) PyCObject_Import("datetime", \
>                                                             "datetime_CAPI")
>
> /* This macro would be used if PyCObject_ImportEx() was created.
> #define PyDateTime_IMPORT \
>         PyDateTimeAPI = (PyDateTime_CAPI*) PyCObject_ImportEx("datetime", \
>                                                             "datetime_CAPI",
> \
>
> DATETIME_API_MAGIC)
> */
>
> That was checked in by Tim Peters on 2004-06-20, r36214.  (At least, in the
> py3k/trunk branch; I'd hope it would be the same revision number in other
> branches.)
>
>
> /larry/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)