Serializing functions

[Aaron Staley]

I am one of the developer's of PiCloud.

To answer your question, we wrote a custom subclass of Pickler to
pickle functions.  As Robert pointed out, the library is LGPL, so you
can see (and use) the source code.  I also presented the details on a
poster at PyCon 2010.  You can see it here:
http://blog.picloud.com/?p=96


On Jun 17, 9:26 am, Robert Kern <robert.k... at gmail.com> wrote:
> On 6/17/10 8:23 AM, Matteo Landi wrote:
>
>
>
>
>
> > Some weeks ago, here on the mailing list I read about picloud[1], a
> > python library used for cloud-computing; I was impressed by its
> > simplicity, here is an example:
>
> >>>> import cloud
> >>>> def square(x):
> > ...  return x * x
> >>>> cloud.call(square, 10)
> >>>> cloud.result()
> > 100
>
> > So, I tried to figure out how to achieve the same result, i.e. define a
> > local generic function and send it somewhere for remote execution, and
> > the get the result back.
> > So I thought about serialization (pickle): I made a few tries and it
> > seemed to work.. but I was wrong, because I never tried to load pickled
> > data from another session different from the one used to pickle data
> > itself. If you try and pickle a function, it is not pickled as a whole,
> > indeed, once you unpickle it, it will raise an exception telling you
> > that the target function was not found in the current module.
>
> > So I'm here, with nothing in my hands; how would you implement this?
>
> PiCloud's client library is LGPLed. Go take a look for yourself. It's pretty nifty.
>
> [Disclosure: My employer, Enthought, has partnered with PiCloud to provide our
> Enthought Python Distribution to PiCloud users. However, I say that it's nifty
> purely because it is.]
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
>   that is made terrible by our own mad attempt to interpret it as though it had
>   an underlying truth."
>    -- Umberto Eco