[Patches] [ python-Patches-1206077 ] updates for the compiler package

SourceForge.net noreply at sourceforge.net
Sun May 22 09:55:10 CEST 2005


Patches item #1206077, was opened at 2005-05-21 09:23
Message generated for change (Comment added) made by sxanth
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1206077&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Library (Lib)
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Stelios (sxanth)
Assigned to: Nobody/Anonymous (nobody)
Summary: updates for the compiler package

Initial Comment:
The compiler package produces lower quality bytecode
since it does not do
any of the peephole optimizations.  I tried to fix that
to make the package
produce bytecode which is as good as the internal
compiler.  Yeah, probably
wasted time but because python is so great it was much
easier than I thought
and now the package produces rather superior bytecode!

The patch implements all the optimizations mentioned in
compile.c and
additionally:

- constant folding (or however this is called).  For
example, it turns
  x = -(1<<2) to the constant -4.  Also nested tuples
of constants are
  converted to one big tuple constant.

- The optimization which checks JUMPs that lead to
other JUMPs is extended
  to re-run the optimization when something is
modified.  For example,
  the builtin optimizer can't do:
	if (a and b) and c
  where there is a JUMP_IF_FALSE which leads to a
JUMP_IF_FALSE which leads
  to a third JUMP_IF_FALSE.
	

While I was at it, I discovered several "bugs" which I
couldn't resist
attempting to fix.

- Implemented LIST_APPEND for list comprehensions

- The docstrings of functions where added in the
co_consts of the module

- All the names that appeared in a function where added
to co_varnames.
  Not sure about the fix

- Made co_filename and co_name interns

- In generator expressions the outmost-iterable's names
where considered
  freevars and therefore in:
	def bar(y):
	    return foo (x for x in y)
  the generator expression was considered a closure


I've used the new compiler for my own benchmark suite
and all went well.
I've also managed to just *compile* the standard
library and with the
new compiler it turns out 40167 bytes smaller (!).  But
really, somebody
should run the regression tests with it (I can't due to
inefficient
hardware).  IFF it passes the tests if *could* be used
to re-compile
the standard library for more efficient code.

The patch is vs 2.4.1.
However, the patch it's not 100% ready to be applied to
mainline; but it's easier to
review.  The maintainer should tell me how to proceed :-)


Further Questions:

- generator expressions do not emit
SETUP_LOOP/POP_BLOCK.  Should they?

- What is the status of the package?  I discovered all
those bugs in a couple
  of hours.  Does this mean that there are many many
others?  How many new ones
  has my patch added?  Is the package not suitable for
serious development?

----------------------------------------------------------------------

>Comment By: Stelios (sxanth)
Date: 2005-05-22 07:55

Message:
Logged In: YES 
user_id=667962

Thanks for the instructions to regtest the compiler.

I was hoping for a quick review by someone who knows well
the compiler package to confirm that I'm doing the right
thing in the places with XXX. It'll save me a lot of time.

Hmmm. Since the patch is not ready to be applied, you can
close this  one if you want and we'll keep the link as a
reference for future work.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2005-05-21 12:19

Message:
Logged In: YES 
user_id=6656

Why do you say this patch isn't ready to apply?  Oh, I've
read it now :)

> What is the status of the package?

Not as well looked after as it should be :-/

I'm not sure most of the changes you describe are really
bugs, more inefficiencies (e.g. compiler generated list
comps still work even if they don't emit LIST_APPEND).

Running "./python Lib/test/regrtest.py test_compiler -uall"
will compile the entire stdlib with Lib/compiler (this is
probably an overnight job) then running just plain "./python
Lib/test/regrtest.py" will run the test suite using the
compiled copy.

It probably makes sense to submit a number of patches,
particularly for the optimization versus straight fix divide.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1206077&group_id=5470


More information about the Patches mailing list