[C++-sig] pyplusplus howto?

[Matthias Baas]

Allen Bierbaum wrote:
> To help out I setup a temporary location for a pyplusplus faq.  Take a
> look at: https://realityforge.vrsource.org/view/PyOpenSG/PyPlusPlusFaq
> .  I haven't copied over all the questions that I have seen on the
> list yet but it is a start.  I would appreciate it if other people
> that are interested in using pyplusplus could help out by either
> migrating some of the questions/answers from the list or adding new
> questions and entries of their own.  If you want to ask questions or
> contribute to the faq you can register for the wiki at
> https://realityforge.vrsource.org/view/TWiki/TWikiRegistration .

Actually, instead of providing solutions using the current pyplusplus 
API I'd rather like to discuss a new/modified API that is already more 
straightforward to use.

As you've noticed from that other thread I'm currently evaluating 
pyplusplus by trying to wrap the SDK of a commercial software package. 
I'm not finished with that yet as I don't have a working version of the 
bindings yet (but I think I'm close), so I don't have any final 
conclusions yet, but so far it boils down to this: I like the concept 
and the functionality of pyplusplus, but I don't like the API (or 
rather: there is no API I could like or dislike, instead I have to mess 
right with the internals of pyplusplus). But then, it's still at version 
0.6.1, so it's ok if there's no absolutely stable and mature API. At 
least, so far the low level API has been flexible enough to deal with 
the problems I had and Roman has always been helpful with providing a 
solution (or fixing bugs). And I still believe that a higher level API 
could just be built on top of the existing package.

The question now is, what would an "ideal API" look like?

What I don't like about pyplusplus' way of filtering is that it 
encourages "messy" coding style ("spaghetti code"). You basically have 
one big filter function that must contain all rules and exceptions there 
are. That's fine when you have some global rules that apply to all 
classes, but when you need to do individual modifications to a 
particular class you end up with a lot of long winded tests. Then it's 
the same with the function that assigns call policies (i.e. the code 
dealing with a particular class is spread over your script whereas I 
would like to keep it together in one place). For such "specialized" 
changes that only affect one particular class or method, the API that 
Pyste offers is much more suited in my opinion (that's why I added my 
own functions).

One thing I didn't fully understand is the rationale behind the 
make_flatten() function. Why should this be used by a user of 
pyplusplus? The declarations are supposed to be seen as a tree, right? 
Then why don't we leave it what it is and provide an appropriate 
iterator instead? For example, in my script I defined the following 
generator:

# iterdecls
def iterdecls(rootdecl):
     """Iterate over one or more declaration trees.

     rootdecl can be either a single declaration, a list of declarations
     or None. A declaration must be an object derived from the declaration_t
     class.
     """
     if rootdecl==None:
         return

     if type(rootdecl) is not list:
         rootdecl = [rootdecl]

     for root in rootdecl:
         yield root
         childs = getattr(root, "declarations", None)
         for node in iterdecls(childs):
             yield node


Note that list(iterdecls(decl)) is equivalent to calling 
make_flatten(decl). But when you just want to iterate over the 
declarations you don't have to create a flattened list at all but 
instead write:

for decl in iterdecls(rootdecl):
   ...

With a slightly different interface the tree traversal can be guided so 
that parts of the tree are pruned and not traversed at all (something 
you cannot do with a flattened list).

The I defined a find_class() function that looks like this:

# find_class
def find_class(name):
     """Search for a class with a particular name.
     """
     global decls

     if name[:2]!="::":
         name = "::"+name

     res = declarations.find_first_declaration(decls, 
type=declarations.class_t, fullname=name)
     if res==None:
         raise RuntimeError, "Class '%s' not found"%name
     return res

The global decls variable was the root of the entire declaration tree 
(maybe it would be nice if pyplusplus would already store this). 
Basically, the function is equivalent to find_first_declaration(), but 
an important difference is that an exception is generated when the class 
was not found. You could argue that you would sooner or later also get 
an exception as find_first_declaration() returns None when it doesn't 
find the declaration, but this would hide the "true" source of the error 
and you wouldn't know which (missing) class was responsible for the error.
Then by defining an alias Class=find_class you already have something 
that looks familiar to Pyste users:

Foo = Class("Foo")

In this case, Foo is the declaration object corresponding to the class 
"Foo". Here, it would be nice if Foo would provide some convenience 
methods such as finding declarations of methods, etc. I did that with a 
function instead:

# find_method
def find_method(classdecl, name, retval=None, args=None):
     """Search for a method with a particular name.

     classdecl is the declaration object of the class where the method 
should
     be searched. name is the method name, retval the type of the return
     value and args a list of argument types. All types (retval and args)
     are given as strings.
     """
     res = []
     for decl in iterdecls(classdecl):
         if not isinstance(decl, declarations.member_calldef_t):
             continue
         if decl.name!=name:
             continue
         if retval!=None:
             if decl.return_type==None or 
decl.return_type.decl_string!=retval:
                 continue
         if args!=None:
             if len(args)!=len(decl.arguments):
                 continue
             cont_flag = False
             for arg,argument in zip(args, decl.arguments):
                 if arg!=argument.type.decl_string:
                     cont_flag = True
                     break
             if cont_flag:
                 continue

         res.append(decl)
     if res==[]:
         raise RuntimeError, "method '%s::%s' not 
found"%(declarations.full_name(classdecl), name)
     return res

This function returns a list of matching declarations. For example, you 
can invoke it like this:

methods = find_method(Foo, "bar", retval="bool", args=["int", "float"]) )

The next function I defined is the ignore function:

# ignore
def ignore(decls):
     """Mark one or more declarations as being ignored.
     """
     if type(decls) is not list:
         decls = [decls]

     for decl in decls:
         decl._ignore_flag = True

This just adds the attribute _ignore_flag to the declarations which 
indicates that this declaration should not appear in the bindings.
The filter could then just look like this:

def filter(decl):
     """Filter function.
     """
     # Check if the declaration was explicitly ignored
     if getattr(decl, "_ignore_flag", False):
         return False

     # default behavior
     return True

With this scheme (i.e. keeping the declaration tree and only mark the 
nodes that should be included/excluded) you can work with the binding in 
several ways. For example, you could either mark everything as being 
ignored and then selectively add the methods you want or just work the 
other way and include everything and then ignore some particular methods.
A similar scheme could be devised for assigning call policies, etc.

The bottom line: My suggestion would be to make the declaration tree 
more "feature rich" and make it the main data structure that the user 
will interact with. Each node should have a specialized API to 
manipulate the way that a binding for this particular object is created. 
The code creators that are later instantiated could get their parameters 
from the declaration tree.

- Matthias -