[Cython] dynamically compile and import pure python modules

[mark florisson]

On 28 April 2012 18:55, Stefan Behnel <stefan_ml at behnel.de> wrote:
> mark florisson, 27.04.2012 22:38:
>> On 27 April 2012 21:16, Stefan Behnel wrote:
>>> What about this deal: we remove the hard bootstrap dependency on the fused
>>> types code (and maybe other Cython specific features) and require its
>>> compilation at install time in Py2.4 (and maybe even 2.5). That would allow
>>> us to use newer Python syntax (and even Cython supported syntax) there
>>> (except for fused types, obviously). Failure to compile the module in
>>> Python 2.4/5 at install time would then abort the installation. Bad luck
>>> for the user, but easy to fix by installing a newer Python version.
>>>
>>> That would give us the advantage of not needing to pollute user home
>>> directories with shared libraries at runtime (which I would consider a very
>>> annoying property).
>>
>> I think it's fine to require compiling in the installed case (or will
>> that be a problem for some package managers?). In the non-installed
>> case with python versions smaller than needed, would you prefer a
>> pyximport or an error message telling you to install Cython? Because
>> for development you really don't want to install every time.
>
> I think it's fine to require at least Python 2.6 for Cython core
> development. Just the installation (basically, what we test in Jenkins
> anyway) should work in Py2.4 and shouldn't require any rebuilds at runtime.
>
> Stefan
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

Well, sometimes stuff works in say, 2.7 but fails in 2.4. In that case
you really have to test with the failing python versions, which means
you'd have to reinstall every time you want to try the tests again
(this is handled automatically for py3k, which runs the 2to3 tool).
I'm also sure many users just clone from git, add the directory to
PYTHONPATH and work from there.

So I guess what I'm saying is, it's fine to mandate compilation at
compile time (don't allow flags to disable compilation), and (for me),
pyximport is totally fine, but Cython must be workable (all
functionality), without needing to install or build, in all versions.
That means either not using the with statement, or compiling with
pyximport in certain versions in certain situations only (i.e., only
in incompatible python version in case the user neither built nor
installed Cython). I don't think that's a problem, if people don't
like to have those shared library modules (will they even notice?) in
their user directories, they can install or build Cython.

Finally, my attachment to the with statement here is mostly to the
aesthetics of the resulting code, rewriting my pull request is not so
much work, so we can leave that out of consideration here.