[C++-sig] pyste and big files

[Nicodemus]

Nicodemus wrote:

>
>
> Niki Spahiev wrote:
>
>> I got msvc internal overflow with pyste generated files. Can pyste 
>> write separate finctions for each class?
>
>
> I don't understand what you mean by "functions for each class", but 
> you can always generate various little modules, one for each pyste 
> file if you want. If you're using a single pyste file for the entire 
> library, break it into smaller ones, one for each submodule your 
> library has (or any other criteria you find appropriate). Note that 
> you can still access the library in Python as a single module:
>
> pyste --module=sub_module1 sub_module1.pyste
> pyste --module=sub_module2 sub_module2.pyste
> pyste --module=sub_module3 sub_module3.pyste
>
> (Each sub_module is a subset of your library called here "library" ;))
>
> Then you create a "library.py", which imports the sub modules:
>
> # file library.py
> from sub_module1 import *
> from sub_module2 import *
> from sub_module3 import *
>
>
> And then as a user:
>
> >>> import library
> >>> # use library normally
>
> Which is the same as:
>
> pyste --module=library library.pyste
>
> >>> import library
> >>> # use library normally too
>
> Which I believe it's what you're doing right now.
>
> I recommend this approach anyway for large libraries, even if you're 
> not having the problems you're describing. 


Hi All!

To solve this kind of problem, and to help the developer while 
developing libraries, I implemented a new mode of code generation in 
Pyste. Using the flag --multiple, Pyste will generate various cpps (one 
for each class in the pyste files) in a directory, instead of the usual 
single one. The user can then compile them all and link them into a 
single library, acomplishing the same results of the single-cpp-mode. 
The advantages of this method are:

- There's no need to recompile the entire module when a single class 
changes.
- In single mode, a library with about 20-30 classes can easily use 
300mb+ of memory to compile. Breaking it into smaller cpps solves the 
problem.

The drawback of this approach is that a compilation starting from zero 
can take about 4x times more than the single-cpp-method... but in a low 
memory machine it may actually be faster.

More details of usage in the documentation.

Regards,
Nicodemus.