[pypy-dev] running the pypy toolchain

[Terrence Cole]

On Thu, 2009-08-13 at 12:27 +1000, Ben Mellor wrote:
> On Wed, 12 Aug 2009 17:44:02 -0700
> Terrence Cole <list-sink at trainedmonkeystudios.org> wrote:
> 
> > On Tue, 2009-08-11 at 03:16 -0600, Maciej Fijalkowski wrote: 
> > > Hi.
> > > 
> > > It's really cool that you find pypy as a good platform for writing
> > > interpreters, we're definitely
> > > happy with that :-) In general, we try to remove stuff from lang
> > > rather then put more of it
> > > there. So js/smalltalk/befunge interpreters would hopefully soon
> > > become projects on
> > > its own. 
> > 
> > Thank you for the reply.  I was thinking that the current situation in
> > the lang directory would be unmanagable and I was wondering what I was
> > missing :-).  
> > 
> > I have not run into any documentation on making the pypy toolchain run
> > from an installed location, rather than from the svn checkout dir.
> > Since I don't even understand how autopath works yet, I figured that
> > getting something more sophisticated working than what everyone else is
> > doing in the lang directory would be an adventure better left for later.
> > Is there a plan for making pypy installable?  Is it already possible and
> > I just haven't looked hard enough?
> 
> What I'm currently doing with an interpreter I'm messing around with is to just
> put the path to PyPy in my PYTHONPATH (I'm using the 1.1.0 release right now,
> but I think it would work for an svn checkout as well). That works for running
> on top of CPython, even with my interpreter project directory completely
> outside of the PyPy tree.

That did the trick!  My test suite is a bit of a kludge now though since
I can't reference my interpreter from an absolute pypy path.  I have
added .. to my sys.path manually to get them working again, but it's
very ugly.  Is there a better way to handle this?

> Translating is a little less satisfactory. I have a makefile with the following:
> 
> 
> # Defualt place to look for PyPy being installed.
> PYPY_DIR :=	~/pypy-1.1.0/
> 
> # Set target-specific variable to set the translation backend based on the name
> # of the target, e.g. fundy-jvm uses the jvm backend.
> fundy-%: BACKEND =	--backend=$(@:fundy-%=%)
> 
> # Default target: fundy translated with the C backend.
> .PHONY: default
> default: fundy-c
> 
> fundy-%: *.py fundy.grammar
> 	python $(PYPY_DIR)/pypy/translator/goal/translate.py $(OPT) $(BACKEND)
> 	$(BATCH) $(TRANSLATE_FLAGS) target_fundy
> 
> 
> (plus a few default settings for the variables I haven't defined here, but
> they're pretty trivial), and a target_fundy.py file that provides the
> target(driver, args) function.
> 
> Then I can just type make, or make fundy-jvm, etc, to get the interpreter
> translated for any backend (although the C one is the only one working on my
> system).
> 
> I started with this project in the lang directory of a PyPy svn checkout, but
> when the 1.1.0 release was made and I ended up using my interpreter as the
> basis for a university essay on programming languages, it spurred me to figure
> out how to get it working as a more independent project.
> 
> I didn't find much information in the PyPy docs on how to do this, I just
> hacked it together. If there are better ways of going about it, I'd like to
> know!

I suppose using PYTHONPATH should be obvious, in retrospect.  Usually,
when I have these sorts of problems, I create a setup.py and just get
everything installed into site-packages.  In fact, I do think that
having pypy in an installed location would be more satisfying than
messing with the path, but I suppose this works for now.

> I do have ~/pypy-1.1.0/pypy/bin in my PATH (translatorshell.py is just too
> nifty), and I have thought it would be handy if there were an entry point
> executable for the translation process in there as well. If translate.py or
> something equivalent was there, then installing PyPy would consist of untarring
> it, setting PYTHONPATH and PATH, and then interpreter projects could just use
> it a little more easily.

Agreed.  It would be very nice to have a pypy.translate program
in /usr/bin.  There are some (non-interpreter) projects I would consider
writing in RPython just for the performance benefits of translation.

> I guess there's a difference between people who are installing the PyPy python
> interpreter, and those who are installing the PyPy translation toolchain.

Very true.  If all you want is pypy-*, how much more do you need
installed than the executable and the modules directory?

>  But
> what the docs say now about how to build things with the PyPy toolchain is to
> cd to pypy/bin/translator/goal and run translate.py targetfile, which
> looks a bit like to make things able to be translated you have to put a
> targetfile there (and it has to be able to import the rest of your project,
> which means either your project is in the PyPy tree, or it's on your
> PYTHONPATH).

That was exactly my thinking as well.

> -- Ben

Thank you for the help.  That was exactly what I needed and I'm very
glad I did not have to go blundering about relearning all of that for
myself.

- Terrence

> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev