[Python-Dev] AST optimizer implemented in Python

[Brett Cannon]

On Sun, Aug 12, 2012 at 6:07 PM, Eric Snow <ericsnowcurrently at gmail.com>wrote:

> On Aug 12, 2012 12:56 PM, "PJ Eby" <pje at telecommunity.com> wrote:
> > I'm not sure if this is directly related or not, but making this
> > mechanism support custom compilation for new filename suffixes would
> > be nice, especially for various e.g. HTML/XML templating systems that
> > compile to Python or bytecode.
> >
> > Specifically, having a way to add a new source suffix (e.g. ".kid",
> > ".zpt", etc.) and a matching compilation function, such that it's
> > automatically picked up for compilation by both the filesystem and zip
> > importers would be awesome.  It'd also allow for DSLs and syntax
> > experiments using alternative filename extensions.
>
> +1
>
> I'm hacking around this right now for a project I'm working on.  I
> definitely do this through the import system.  Inserting a look-alike path
> hook and monkeypatching the cached path entry finders is not difficult, but
> certainly fragile and less than ideal.
>
Why are you doing that? Can't you just use FileFinder with new loaders and
file suffixes? Why do you feel the need to much with any cache?

> Consequently I've been looking into simple and not-so-simple solutions to
> making it easier to add new suffixes to be handled.  The
> source/pyc-to-code-to-module path during imports is so prevalent and
> critical that it may benefit from its own model.
>

3.4 will expose more of this. The source-to-code method would get you the
transformation that PJE wants while 3.3 already has FileFinder handle the
finding of the proper files for you. Toss in a method or two to help in
parsing byte-compiled files and writing them and most of the process is
then exposed.


>   This is similar to how the path-based import subsystem has its own
> special-case model.  Or source-based imports just need better
> special-casing in the path-based import subsystem.  Perhaps just adding and
> using sys.source_suffixes as a mapping of suffixes to loader classes.
>

This is starting to get off-topic, but no more adding stuff to sys in
regards to import. There is already too much global state that is
preventing good encapsulation and we don't need to make it any worse. The
more we add to sys in regards to import the farther we get from any import
engine solution that we might want to evolve towards.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120812/28daba95/attachment-0001.html>