[Python-ideas] .pyu nicode syntax symbols (was Re: Empty set, Empty dict)

[Andrew Barnert]

(Replies to both Guido's top-post and Nick's reply-post below.)

On Monday, June 30, 2014 7:19 PM, Guido van Rossum <guido at python.org> wrote:


>Like bytecode, the compiler's workings are not part of the language spec, and are likely to change incompatibly between versions and not work for anything besides CPython. I don't really want to go there (cool though it sounds for wannabe compiler hackers).


But CPython does expose bytecode via the dis module, parts of inspect, etc. For that matter, it exposes some of the compiler's workings (especially if you consider everything up to AST generation part of the compiler, since every step up to there is exposed, including doing the whole thing in one whack with PyCF_ONLY_AST). So, I don't see how exposing the AST-to-bytecode transformation part (or, while we're at it, the .pyc generation part) is any more unportable than what's already there.

That being said, I can appreciate that it would almost certainly take a lot more work, and a lot riskier work to do that, so the same tradeoff could easily go the other way in this case. (Not to mention that the dis module and so on are already there, while the patches Nick was talking about, much less something brand new, are obviously not.)

>On Mon, Jun 30, 2014 at 7:15 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>>On 30 Jun 2014 16:51, "Andrew Barnert" <abarnert at yahoo.com.dmarc.invalid> wrote:
>>>
>>> First, two quick side notes:
>>>
>>> It might be nice if the compiler were as easy to hook as the importer. Alternatively, it might be nice if there were a way to do "inline bytecode assembly" in CPython, similar to the way you do inline assembly in many C compilers, so the answer to random's question is just "asm [('BUILD_SET', 0)]" or something similar. Either of those would make this problem trivial.
>>Eugene Toder & Dave Malcolm have some interesting patches on the tracker to help enhance the compiler (in particular, Dave's allowed compiler plugins to be written in Python). Neither set of patches made it to being merge ready, though, and they'll be rather stale at this point.


Thanks!

Are you referring to Dave Malcolm's patch to adding a hook for an AST optimizer (in Python) right before compiling the AST to code (http://bugs.python.org/issue10399 and related)? 

If so, I don't think that would actually help here. Unless it's possible to say "BUILD_SET 0" in AST, but in that case, we don't need any new compiler hooks; just use an import hook the same way MacroPy does. (Doing it without import hooks would be a little nicer, but it's not essential.)

The only patch I could find by Eugene Toder is one to reenable constant folding on -0, which I think was already committed in 3.3, and doesn't seem relevant anyway. Is there something else I should be searching for?