[Cython] Build module script

[Chuck Blake]

>>> On 21 April 2011 08:57, Vitja Makarov <vitja.makarov at gmail.com> wrote:
>> I mean simple script that run cython compiler and then build shared
>> python module.
>> It's much better then manually run cython then gcc or write makefile for this.
>
>Ah, I see. Yeah that might be convenient. Although there is always pyximport.

I have a script I can contribute called pycc that might be a bit more
than "simple".

  pycc [-c] [pyrexc|cython opts] [-- C-compiler opts] <FILE.pyx> [Clink opts]

as in

  pycc -c foo.pyx

to produce foo.so in the simple case, or maybe, just as an example

  pycc -c -- -fmudflap foo.pyx -lmudflap

if you had gcc as a compiler and wanted to turn on mudflap.

The script itself is a Python script which uses an
    exec(file(os.environ["HOME"] + "/.pyccrc").read())
to configure things in the Python language itself for what C compiler and
what default compile flags, C & Python include paths, libraries, etc.

It only works on Unix right now.  I'm not sure how hard it would be
to adapt it to work on both Unix and Windows.

It actually also works for "programs" rather than "modules" if you
drop the -c as in

  pycc -- -I/usr/include/X11 whatever.pyx -lX11

to produce a --embed generated program called "whatever".  In all,
it makes for a pycc that behaves a bit more like traditional compiler
front ends such as 'gcc' that take source all the way to a variety
of outputs usable by the host OS.  It works well in traditional
Makefiles or just from the command line as one-off invocations as
you're doing initial writing and testing or even "just seeing"
if Cython works/helps in some isolated situation with some module
or script.

--

There's also some complexity that dates to pre-embed days, but
which I personally find useful (or Pyrex users might find useful
since I believe Pyrex does not have --embed).  If you do not have
the "-c" and you do have a ##MAIN## comment in the source file,
it will indent all code after ##MAIN## into a "Main()" function
called at startup.  If there's no special comment it falls back
to --embed.  (we could spell this some other way if anyone wants
more like the compiler option comment syntax "# cython: MAIN").

The upside of this feature is that with almost no mucking about
with your Python script other than adding a comment, it makes
variables local to Main() which allows optimizations in the main
script logic that potentially make things much faster.  I see 2X
speed-ups in a few of my own speed critical programs.  It probably
would speed up CPython interpretation, too, but not nearly as much
as the Cython operation in my various experiments.

The downside of the feature is that does fiddle with global/local
scope or mixed statements + defines + statments + defines.  Those
are all "very uncommon" patterns in my usage.  Also, it does have a
big ##MAIN## warning marker to let a reader know something special
might be going on semantics-wise.  And, hey, 2X speed for a 1-line
diff. :)

So, you can probably see why I keep that feature around, but I could
also see if people didn't want an auto-functionizing-of-Main feature.
Having it around also doesn't seem harmful if the spelling of the
comment marker is nigh impossible to occur by accident.  If you
wanted me to delete that from the script before submission, I could,
but it'd require a bit work and testing.  Let me know if you want it.

--

An entirely alternate proposal that might be better, but would also be
more involved would be to make the "cython" program grow functionality
to make it more like a traditional compiler front end.  As things stand,
"cython" (inheriting use-concept from Pyrex) works more like "cc -E"
where output needs to fed to another stage to be compiled/linked.