[SciPy-Dev] On pulling fwrap refactor into upstream SciPy

[Dag Sverre Seljebotn]

On 02/09/2011 12:20 AM, Pauli Virtanen wrote:
> On Tue, 08 Feb 2011 21:58:08 +0000, Pauli Virtanen wrote:
>    
>> On Tue, 08 Feb 2011 21:48:28 +0100, Dag Sverre Seljebotn wrote:
>>      
> [clip]
>    
>>> Not code-wise, but history-wise.
>>>
>>> That is, I have several times changed the code generation of Fwrap,
>>> regenerated the boilerplate, and put the manual changes back in (using
>>> git).
>>>        
>> [clip]
>>
>> Clever, that's doable with git. However, it's not very pretty, and I
>> have a nasty feeling about mixing autogenerated code with manual code in
>> the long term.
>>      
> Ok, some backing arguments for the gut feeling: the approach moves a part
> of the logical content of the code out of the files, to the version
> history. The top-level meaning of the code cannot be understood by
> reading it, as boilerplate cannot be distinguished from customizations.
> Instead, one needs to use "git blame" or some such tool to spot points
> that contain more information than the *.pyf files.
>
> Using some sort of "inline patch" approach would put the information
> back. Before a manual edit:
>
> 	some
> 	automatically
> 	generated
> 	code
>
> and after:
>
> 	some
> 	#<  automatically
> 	#<  generated
> 	manually edited
> 	#>
> 	code
>
> or:
>
> 	some
> 	#<
> 	manually edited
> 	#>
> 	code
>
> Suitably smart code generator could in principle extract and re-apply
> such inline patches automatically, but the main point IMHO would be to
> preserve readability.
>    

First: When I had to go another round with regenerating my wrappers, I 
realized that while merging back changes works beautifully, the git 
history becomes incredibly messy and difficult to track from my approach 
(a) every Fwrap-generated file has its own branch which *does* show up 
in the history, b) I become too afraid to do any rebasing less I loose 
the auto-generated version (and managed to break the scheme in a merge I 
did), and thus the tool got in my way, which is never good).

So what I did now is to have a subdirectory in every package (which is 
currently named ".fwrap+pyf", we can get back to that) which stores the 
pristine auto-generated copies. "fwrap update" invokes Unix "merge", and 
"fwrap mergetool" currently invokes kdiff3 with the right arguments.

This works much better with my workflow, since I decouple how I work 
with git from how I work with fwrap. And the merges are as painless as 
they use to be.

And you can always get the manual modifications by

diff wrapper.pyx .fwrap+pyf/wrapper.pyx

(which I suppose I could roll into "fwrap diff").

Commenting on your suggestion: Assuming the (much more failsafe) merging 
strategy I described above is kept, this just boils down to a SciPy 
project convention, right? A problem is that many of the manual patches 
changes only some parts of a line (e.g., change one argument of the call 
to Fortran). How about just a comment at the top of the function body 
listing higher-level descriptions of every manual change done, and then 
one can consult 'fwrap diff' for further details?

Not sure if I'll have (paid) time to insert such comments tough...

Dag Sverre