RFC: Viper: yet another python implementation

[John Max Skaller]

On 15 Aug 1999 23:05:29 GMT, m.faassen at vet.uu.nl (Martijn Faassen) wrote:

>John (Max) Skaller <skaller at maxtal.com.au> wrote:

>I'm sure type inference can be done (at least in a limited fashion), but
>it would be nice to have an intermediate dropoff point; a viperi or
>viperc that would work with type annotations. 

	It is always possible to 'do' type inference; this
isn't an issue. The issue is whether I can construct an algorithm
that gives useful (and correct) information, and whether I can
then use that information to optimise performance.

	Type annotations are planned; the syntax will
probably be:

	def f(x:int): pass

because (a) that is fairly standard in other languages, and
(b) it is probably the syntax Guido will adopt. However,
a more transparent scheme is possible (traits a la Swallow; nice idea)

>This is useable already,
>especially if it could be used to create fast Python extension modules.
>(something you aren't aiming for right now, I gather)

	I will look at doing this, but it is unclear if enough
benefits can be obtained: Python is highly polymorphic.

>If the project has usuable results at many stages, you have a bigger chance
>to get the 'open source' effect going (if you're aiming for that), too.

	That is true, but see above. I need to get
much faster performance for interscript, and it isn't
clear the a module compiler can do that. But I will
definitely look at the idea, although the output
would have to be either C/1.5.2 or Java (Jpython).

>Ah, so the reason you don't use Python itself as the interpreter is
>learning, and also as an intermediate dropoff point.

	Not really: C isn't suitable for writing complex
software. I would be mad to use something as low level
as C for programming a compiler/interpreter where 
speed is far less important than the speed of the 
code which is output. Especially while prototyping.

	Secondly, it is easier to extend my own
code than the existing Python code.

>> It is possible the first back end will actually general Ocaml
>> rather than C (since Viper is currently implemented in ocaml).
>
>Hm, why Ocaml? 

	First, it is a powerful language, and is especially
good for implementing language translators.

	Secondly, it can generate assembler output,
so the results are on par with C, sometimes even faster.

	Third, it is based on mathematical foundations.

	it contains many of the best features of
functional, procedural, and object oriented code,
reasonably well integrated, and again _founded_
on mathematics.

	Finally, it is readily available and has a small
footprint.

	To put it in a nutshell, it is easily
the best statically typed programming language
I have seen: and the margin is massive.

>I don't know this language, though I heard some
>interesting things about it. But it begs the question why you're 
>not using Python itself.

	Are you kidding? You really cannot be serious.
Python doesn't have the kinds of constructions needed
for complex software design. Perhaps, after I'm finished
with Viper, it will provide better facilities: one of the aims
is to provide powerful constructions in a unified manner,
but using the nice Python syntax.

>but as you are aiming for a Python compiler
>this can't last. :) And I'd definitely consider at least a C 
>backend for the compiler, because C is so common. 

	I am considering the following backends:

	1) Python 
	2) C
	3) C++
	4) Assembler (via ocaml)
	5) Ocaml

>But Python's C interface is well known and I believe also quite stable;

	No, there was an upgrade from 1.5.1 to 1.5.2,
1.6 may be different, 2.0 will certainly be different.

>I can see where you're coming from, but I'd still seriously consider
>adding at least a CPython interface early on -- the more stages in your
>project that are usable, the higher the chance is your project will
>bear fruit.

	A CPython interface is indeed possible,
but it will be a very _difficult_ problem. The reason
is the the Viper architecture is quite different to
the CPython 1. For a start, there is a built in 
garbage collector.  All the objects are represented
differently. The binding will be _much_ harder
than a completely fresh implementation.

	Note that JPython doesn't support C modules either.

>I can see where full program analysis may help with type inference, 
>of course. But if you go by the intermediate route of manual adding of
>types, it doesn't sound that hard to support some interface with Python.

	True. But that won't work for existing code: I want to
be able to compile _existing_ Python sources.

>Of course you need to restrict the modules that can be called by Viper
>code to the set of modules supported by Viper;

	Yes. So I have to implement all the common library
modules that do not have a Python .py equivalent.
Note that even 'string' has Python source, and,
surprisingly, almost all of  Tkinter is pure python.

>Hm, reading this it seems you're talking about the other thing; supporting
>C Python modules in Viper? I was talking about allowing people to Viper-compile
>their Python modules (as long as they are Viper compliant), and use them
>from Python. But as you say, you can use Pure Python modules in Viper
>as well; if they restrict themselves to the Viper subset. 

	It would be nice to be able to generate compatible
C modules, since that would allow incremental speed up of
existing python programs. 

	However, several competent people have tried this
and the results ar barely better than the bytecode interpreter.

	OTOH experiments in JPython indicate up to 10000X
faster performance after whole program analysis (so I read,
and I believe it).

John Max Skaller                ph:61-2-96600850              
mailto:skaller at maxtal.com.au       10/1 Toxteth Rd 
http://www.maxtal.com.au/~skaller  Glebe 2037 NSW AUSTRALIA