Unpacking a python list in C?

[Ken Seehof]

Michael Hudson wrote:
> Ken Seehof <kseehof at neuralintegrator.com> writes:
> 
> > Here's some real code that's pretty close to what you want.  You can
> > use PyList_* instead of PySequence_* if speed is more important than
> > flexibility (PySequence works with any sequence object).
> > This example is a little different than what you requested (it's a
> > working python extension function).  But it does show how to iterate
> > a python list.
> > 
> > /////////////////////////
> > // syntax: obj = manifold([seq])
> > // returns: new manifold object
> > //
> > // Creates a manifold object, and initializes with sequence
> 
> Are you not doing any error checking for clarity?

Nah, just plain laziness :-)  I think the only error checking I
missed is the sequence type check.

> > static PyObject *module_manifold(PyObject *self, PyObject *args)
> > {
> >     PyObject *seq = NULL;
> > 	if (!PyArg_ParseTuple(args, "|O", &seq)) return NULL;
> 
> It would be wise to check that "seq" is indeed a sequence at some
> point...

That's a good idea.  Thanks.
 
> Also, if you call this function with no argument it will return NULL
> but with no exception set... don't do that.

Not true.  Actually, calling with no argument is valid for "|O", and
in this case leaves seq=NULL (which skips the sequence initialization
as intended).  Calling with incorrect arguments (in this case, two or
more args) results in PyArg_ParseTuple returning NULL, with the
appropriate TypeError set automagically.  When my function returns
NULL, I get the correct exception.

>>> manifold([2,3,4])
<2, 3, 4>
>>> manifold()
<>
>>> manifold([2,3,4], 3.4)
Traceback (most recent call last):
  File "<input>", line 1, in ?
TypeError: function takes at most 1 argument (2 given)

> > 	manifold__object *k = manifold__new();
> > 
> >     if (seq)
> >     {
> >         for (int i=0; i<PySequence_Length(seq); i++)
> 
> Also PySequence_Length isn't reliable; consider
> 
> class Devious:
>     def __len__(self):
>         return 9
>     def __getitem__(self, i):
>         raise IndexError
> 
> >         {
> >             PyObject *obj = PySequence_GetItem(seq, i);
> >             manifold__insert_obj(k, k->root, obj);
> 
> ... so obj could be NULL here; manifold__insert_obj will likely try to
> INCREF it... boom!

Yup, boom it is.  I'd better fix that.  Thanks.

> >         }
> >     }
> > 
> >     return (PyObject *)k;
> > }
> 
> The best way to nullify the threat of such devious code is to use the
> "PySequence_Fast" API, like so:
> 
> static PyObject *module_manifold(PyObject *self, PyObject *args)
> {
>     PyObject *seq;
>     int len;
>     if (!PyArg_ParseTuple(args, "O", &seq)) return NULL;
> 
>     manifold__object *k = manifold__new();
> 
>     seq = PySequence_Fast(seq, "manifold: sole argument must be 
> sequence");
>     if (!seq) return NULL;
> 
>     len = PySequence_Fast_GET_SIZE(seq);
> 
>     for (int i=0; i < len; i++) {
>         PyObject *obj = PySequence_Fast_GET_ITEM(seq, i);
>         manifold__insert_obj(k, k->root, obj);
>     }
> 
>     return (PyObject *)k;
> }
> 
> as PySequence_Fast knows how to cope with the most devious of user
> classes.  I'm not sure when it appeared (I don't *think* it's in
> 1.5.2, unfortunately, though there's always the option of wholesale
> lifting of code from Python's codebase to your own).
> 
> Docs here:
> 
>     http://www.python.org/dev/doc/devel/api/sequence.html
> 
> Mild apologies for picking on Ken's code like this; more people should
> know about this API, though.

Thanks, most of your suggestions are very helpful and all your suggestions
are appreciated.  I look forward to putting this stuff on SourceForge
to get more feedback, when I get some free time.

In case you're wondering, "manifold" is a new kind of list.

- safe deletion during iteration
- O(constant) fast __contains__ (in) operator
- global explicit object removal (fast)
- python thread safe

Basically it solves the problem of explicit removal of an object from
all "lists" of which it is a member, especially in situations where
object removal of any kind would normally be hazardous.  I've been using
it for a couple weeks, and it seems stable.  The sequence initialization
that I posted is one of a few recent additions for completeness.

> Cheers,
> M.
> 
> -- 
>   The Programmer's Quick Guide To Python (Time Machine version):
>     You try to shoot yourself in the foot, only to realize that
>     there's no need, since Guido thoughtfully shot you in the foot 
>     years ago.                     -- Nick Mathewson, comp.lang.python
> -- 
> http://mail.python.org/mailman/listinfo/python-list

Thanks,
- Ken Seehof