[Pythonmac-SIG] Adding functionality to existing MacPython installations

[Jack Jansen]

On Friday, September 26, 2003, at 01:17 AM, Bob Ippolito wrote:

>> I think this would lead us down a path where we have "functionality 
>> addition"
>> releases for MacPython, orthogonal to the 2.3.X bugfix releases that
>> are common to Python. In other words, at some point we create a
>> "MacPython 2.3.X FA 1" release, that contains a number of modules that
>> we expect to be included in 2.4. We take care that these FA releases
>> work with any 2.3.X.
>>
>> (The alternative would be to de-couple MacPython distributions from
>> core-Python distributions, similar to the Windows situation of 
>> PythonWin
>> and PythonLabs-Python-for-Windows. But if we want all the good stuff
>> to be picked up by Apple eventually and end up on every Mac in the 
>> universe
>> I think trying to stick with the unified distribution is definitely
>> worth the extra hassle).
>
> I just want to have the most capable Python I can possibly have, and 
> I'd like to see other people have the same.  I think that this problem 
> is mostly solved with Package Manager between releases, but I don't 
> have enough experience releasing software for the general public (or 
> this community) to really make an educated choice between either of 
> the two.
>
> As you can see, I'm not the kind of person who tries to do everything 
> in pure Python with as few dependencies as possible.  I just want to 
> write the least amount of code to do the most correct and complete 
> thing for the task at hand.  Now that we have Package Manager, I think 
> it's rather easy to get away with this :)

One of my primary concerns is maintainability. Not only do I want a 
good Python *now*, I also want
it to be good in ten years time.

When I took over MacPython maintainance from Guido I first started with 
adding a couple of
modules I really needed (macfs) or thought were fun (macspeech). I 
wrote these by hand, and
put in just the stuff I needed. Then I added to it when requests came 
in, or when new
Apple technologies came out.

Over the ten years that I've maintained MacPython this has turned out 
to be a very bad idea.
Macfs needed work for every release of MacOS, and lots of really useful 
things (like FSRefs)
got delayed far too long. And macspeech went basically the other way: 
it was never really
updated, so while it looked fun at the time of Python 1.3 it looked 
really clunky and
outdated when I finally put it out of its misery for 2.3.

If I compare the effort put into macfs with that put into, say, 
Carbon.Win and friends
the latter was a *lot* less work, even though the underlying toolkits 
went through
serious changes (such as the Appearance stuff at the time of MacOS8). I 
basically did
the appearance work in a couple of days, without any real understanding 
of how it
worked:-)

>> 2. For LaunchServices and TECManager, why did you use Pyrex and not 
>> bgen?
>
> I used Pyrex and not bgen because I understand it a lot better, and 
> I'm much quicker at using it (I'm extremely proficient at regexing C 
> to Pyrex after doing a few wrappers with it).  It supports keyword 
> arguments very easily so I can make it act like Python would, it looks 
> like Python and is relatively easy for non-C programmers to 
> understand, it does all of the conversion to and from Python stuff 
> automatically and makes exception handling and callbacks easy, and 
> there's a decent amount of code in there that just can't be generated 
> automagically.  The portion of the APIs I wrapped aren't too large, so 
> it wasn't an enormous task.

But (again with my MacPython-maintainer hat on): if you didn't wrap all 
of it people are going to
come and ask for the missing bits eventually. And if this happens in 3 
years time and you've lost
all interest in Python by that time and moved on to VB programming that 
task of adding it manually
will come to me (or, in case I've left Python for VB, to whoever is 
then in charge of MacPython:-).

With the current setup, whoever maintains MacPython needs to understand
one tool only, bgen. And while bgen is clunky, and I'd be open to 
replacing
it with something better, it is the only tool that will do most of the
work automatically.

>  As for the automatic generation, I am actually working on a C header 
> -> Pyrex thing that I've made good progress with.  Unlike bgen, it 
> actually tokenizes the C (using a modified version of someone else's 
> lexer) and breaks it up into abstract syntax trees.

I've been looking at this too, for bgen. The main reason is that I want 
to be able to wrap
C++ classes eventually, and bgen's simple-minded regex parser isn't 
going to cut
it. And Apple's headers are more and more manually written (in stead of 
generated,
as in the Universal Header days), so the regexps need more and more 
twiddling anyway.

I've looked at the SWIG XML-based format, but it's too tied to SWIG, 
and the output
is gigantic. I've talked to Dave Beazly over the years about somehow 
getting some of
the good bits of bgen into swig and the reverse, but we're both too 
busy to actually make
anything happen.

I've now started looking at gcc-xml, and that looks very promising. 
But: my time
is limited right now, so I didn't look hard enough yet to find the 
actual showstoppers:-)

Did you look at gcc-xml? And if you did but found it lacking: could your
ASTs be exported as XML so I could feed them into bgen?
--
Jack Jansen, <Jack.Jansen at cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman