[C++-sig] Re: Pyste: problems with inheritance across headers.

[Prabhu Ramachandran]

>>>>> "DA" == David Abrahams <dave at boost-consulting.com> writes:

    DA> Prabhu Ramachandran <prabhu at aero.iitm.ernet.in> writes: 

    >> The only trouble we saw with this approach was for the case
    >> without --multiple.  This will require some modifications to
    >> the SingleCodeUnit class but Nicodemus said this should not be
    >> a big problem.  Essentially before saving the SingleCodeUnit
    >> we'd need to order the methods correctly.

    DA> I admit to not having a deep understanding of the issues here,
    DA> but it seems like you might solve some significant ordering
    DA> problems by parsing the XML and using pickle/shelve to create
    DA> representations of the classes that are quickly readable by
    DA> Python, then making a separate pass over these data
    DA> structures.

Thanks for the hint.  We did think about this.  We first thought that
it was GCCXML that was taking much of the time to generate the XML.
So we thought that if we cached the XML files we could generate the
XML in separate steps and then use the generated XML files when
generating the wrapper code.  However subsequent timing tests by
Nicodemus revealed that parsing the generated XML file takes as much
time if not more than generating the XML file.  So we thought of
saving the parsed XML information into a picked file.  However as a
first attempt we went ahead with caching the XML files.  It should be
easy to move to using a pickled (.pystec) file once the other bugs are
resolved.

We did not think of using shelve.  That is a good idea but we think we
will have problems when trying to build the wrappers incrementally and
will also have trouble with dependencies and build systems.  Neither
me nor Nicodemus are experts with build systems though, so
clarifications from experts would help.  Essentially, with the
'.pystec' approach we will generate a file containing parsed
information from the XML file, one for each header.  With shelve we
will have one shelf with all the pickles.  When generating incremental
compiles we thought that it would be better to have the pystec
information in separate units in order to avoid dependency problems.
To clarify, this is a contrived example specifically for the
--multiple mode:

 file.pyste  depends on  header1.hpp, header2.hpp

 header{1,2}.hpp + file.pyste generates a header{1,2}.pystec
                                                     
 file.pyste + header{1,2}.pystec generates _file.cpp + _main.cpp

 _file.cpp, _main.cpp generates the module.

Generalize this to several pyste files and you'll see why we prefer
.pystec's to one single shelf.  If one of the interface files (.pyste)
changes, the build system will need to update the shelf.  The build
system might even delete the shelf so we'll loose all the other
information.  From the perspective of dependencies it might be better
to save one .pystec for each .pyste file.  We might do that later on.
Hmm, perhaps one shelf per .pyste file containing individual
.pystec's.  Anyway, things are still quite experimental and in flux.
We hope that we end up with a decent solution though.

I hope this is clear.  If there is something incorrect here please let
us know.

Thanks.

cheers,
prabhu