[C++-sig] Improved pyste

[Niall Douglas]

On 7 Jun 2005 at 14:18, Ralf W. Grosse-Kunstleve wrote:

> > > * Added new SplitOutput() directive which can split output into
> > > multiple files, thus avoiding MSVC's "internal structure overflow"
> > > error. Note that the implementation of this is pretty hacky
> > 
> > Can you say a bit more about how this works?

Yeah it's very easy. For example:

class A
{
  virtual void foo();
};

class B
{
  virtual void foo();
}

in file TestFile.h with TestFile.pyste containing SplitOutput("A") 
causes generation of _TestFile.cpp and _TestFile1.cpp.

_TestFile.cpp contain everything not split ie; class B. 
_TestFile1.cpp contains just the class_<"A"> and the A_Wrapper class. 
All headers, PCH etc. are the same for both files.

This good when any one of your classes alone isn't complex enough to 
kill MSVC, but doesn't help you if any single class is. However it's 
not much of an additional step to get pyste to pass the class_<> 
instance to the subsiduary c++ file so that more defs can be applied 
to it.

I could have had pyste do this automatically, but the problem is that 
complexity has little to do with simple line counts - a short class 
definition can trip the compiler as much as a much longer one. Hence 
I went with the new directive so the user can specify manually.

You can specify SplitOutput() many times, with each class going into 
its own .cpp file. I had to do this with FXMDIButton.h as each class 
alone was too big to coexist with any other, and it defined like six 
MDI buttons.

I should warn you that the pyste I supplied outputs TnFOX-only code 
as I had to patch BPL to unlock the interpreter around entering C++, 
though it's trivial to return to the default pyste output.

> FWIW: I've never seen the dreaded "internal structure overflow" error
> with Visual C++ 7.1. Since the compiler is available free of charge I
> wonder why anybody is still "investing" time working around the
> numerous problems in VC6 and VC 7.0:

Oh no, it's MSVC7.1 is the problem. Some *nine* files of 238 total 
output by pyste cause fatal internal structure overflows but then I 
can see why, the output is about 50-80Kb long with multiple 
polymorphic classes with multiple inheritance all kicking along. To 
give you some idea, the whole of the TnFOX python bindings is around 
110,000 lines of code which is nearly that of the TnFOX core library 
itself. The DLL output was around 16Mb release build and takes about 
six hours to link and comdat optimise.

As you might imagine, binding TnFOX to Python has pushed BPL itself, 
pyste and the compilers & linkers to their limit. It was after all 
the cause of me patching GCC with the visibility patch.

I'd imagine MSVC8 is better, but I haven't tried it. GCC is just 
fine.

Cheers,
Niall