SoC project: Python-Haskell bridge - request for feedback

[Michał Janeczek]

On 27 Mar 2008 23:49:46 -0700, Paul Rubin
<"http://phr.cx"@nospam.invalid> wrote:
> "Michaâ Janeczek" <janeczek at gmail.com> writes:
>  > However, the type of embedding that I imagined would be mostly
>  > pure functions, since Python can deal with IO rather well. It'd also
>  > be applicable in situations where we want to add some functionality to
>  > to existing, large Python project, where the complete rewrite
>  > would be infeasible.
>
>  Of course I can't say what functions someone else would want to use,
>  but I'm not seeing very convincing applications of this myself.  There
>  aren't that many prewritten Haskell libraries (especially monomorphic
>  ones) that I could see using that way.  And if I'm skilled enough with
>  Haskell to write the functions myself, I'd probably rather embed
>  Python in a Haskell app than the other way around.

I was talking mostly about writing new components there, not just
using libraries. It'd apply to situations where there already exists
a large Python infrastructure.

>  > I didn't mention this in this first draft, but I don't know (yet)
>  > how to support those "fancy" types. The plan for now is to export
>  > monomorphic functions only.
>
>  This probably loses most of the interesting stuff: parser combinators,
>  functional data structures like zippers, etc.  Unless you mean to
>  use templates to make specialized versions?

That is true, but as you said before, using them directly from
Python would be unwieldy. In most cases writing a thin wrapper
in a language of the library that is going to be used is a much
more natural way (same might apply to calling Python from Haskell).

The plan for now is to start with monomorphic functions, then
add support for parametrically polymorphic functions
(that is ones that don't care about the type at all, i.e. List.length),
and then possibly extend this to some type classes that we
can reasonably expect to find on the Python side (Eq, Ord, etc.)

>  > As for GC, I think having the two systems involved is unavoidable if
>  > I want to have first class functions on both sides.
>
>  This just seems worse and worse the more I think about it.  Remember
>  that GHC uses a copying gc so there is no finalization and therefore
>  no way to notify python that a reference has been freed.  And you'd
>  probably have to put Haskell pointers into Python's heap objects so
>  that the Haskell gc wouldn't have to scan the whole Python heap.
>  Also, any low level GHC gc stuff (not sure if there would be any)
>  might have to be redone for GHC 6.10(?) which is getting a new
>  (parallel) gc.  Maybe I'm not thinking of this the right way though, I
>  haven't looked at the low level ghc code.

GHC does support finalization, at least for foreign pointers.
I don't think there is going to be any major change to the
foreign interface, either. Otherwise I wouldn't be bothering myself really ;)
The idea here is to have Haskell track most of Python objects
(I think that'll be much easier), and the other way around when
it's really necessary.

>  Keep in mind also that Python style tends to not use complex data
>  structures and fancy sharing of Haskell structures may not be in the
>  Python style.  Python uses extensible lists and mutable dictionaries
>  for just about everything, relying on the speed of the underlying C
>  functions to do list operations very fast (C-coded O(n) operation
>  faster than interpreted O(log n) operation for realistic n).  So maybe
>  this type of sharing won't be so useful.

Yes, but for now I think it is necessary for function callbacks
(Haskell calling Python giving it a higher order function). And if
we decide to support that, we might as well investigate other
use cases.

>  It may be simplest to just marshal data structures across a message
>  passing interface rather than really try to share values between the
>  two systems.
I agree that it'll be simplest, that's why I chose it as a first step :)

>  For fancy functional structures, from a Python
>  programmer's point of view, it is probably most useful to just pick a
>  few important ones and code them in C from scratch for direct use in
>  Python.  Hedgehog Lisp (google for it) has a nice C implementation of
>  functional maps that could probably port easily to the Python C API,
>  and I've been sort of wanting to do that.  It would be great if
>  you beat me to it.

Nice link about Hedgehog lisp :)
As for porting its data structures to Python, I don't think
it is a large enough project for SoC. And I also hope that
the Haskell<->Python can bring more benefits to the
users of both languages.

>  One thing I highly recommend is that you join the #haskell channel on
>  irc.freenode.net.  There are a lot of real experts there (I'm just
>  newbie) who can advise you better than I can, and you can talk to them
>  in real time.

Yes, I do drop by in there sometimes.. But what I'm after here
is an opinion of a Python programmer :)

Thanks again :)
Michal