[C++-sig] Re: cPickle in extension module

[Norbert Riedlin]

> I won't be able to devote any time to it for at least a week. Maybe
someone
> else will step forward in the meantime if you can post a *minimal*
> reproducible test case.
>
> -Dave
>
OK, I'll try to make it short:

<code>
// pickletest.cpp
#include "picklemodule.h"

#include <boost/python/detail/wrap_python.hpp>
#include <boost/python/class_builder.hpp>
#include <boost/python/objects.hpp>

PYTHON_API std::string ToScript(PyObject* msg);
PYTHON_API PyObject* FromScript(const std::string& msg);

BOOST_PYTHON_MODULE_INIT(pickletest)
{
  try {
    python::module_builder module("pickletest");

    // pickle
    module.def(&FromScript,   "get_object");
    module.def(&ToScript,       "to_script");
  }
  catch(...) {
    python::handle_exception(); // Deal with the exception for Python
  }
}

PYTHON_API std::string ToScript(PyObject* msg)
{
  return Netatec::PickleModule::Instance().Dumps(msg);
}

PYTHON_API PyObject* FromScript(const std::string& msg)
{
  return Netatec::PickleModule::Instance().Loads(msg);
}

// picklemodule.h, methods inlined for brevity

namespace Netatec {
  class PickleModule {
    private:
      PickleModuleMgr() {
         PyObject* mod = PyImport_ImportModule("cPickle");
         if(mod == 0) {
           PyErr_SetString(PyExc_ImportError, "no module cPickle found");
           throw boost::python::error_already_set();
         }
         m_dumps = PyObject_GetAttrString(mod, "dumps");
         m_loads = PyObject_GetAttrString(mod, "loads");

        Py_XDECREF(mod);
      }

    public:
      static PickleModule& Instance() {
        static PickleModule theModule;
        return theModule;
      }

      ~PickleModuleMgr() {
          Py_XDECREF(m_dumps);
          Py_XDECREF(m_loads);
      }
      std::string Dumps(PyObject* theObject) const {
        const int binary = 0; // 1: binary, 0: text
        PyObject* str = PyObject_CallFunction(m_dumps, "Ni",
                                              theObject, binary);
        if(str == 0) {
          throw boost::python::error_already_set();
        }
        std::string result = PyString_AsString(str);
        Py_XDECREF(str);
        return result;
      }

      PyObject* Loads(const std::string& theString) const {
        return PyObject_CallFunction(m_loads, "s", theString.c_str());
      }
    private:
      PyObject* m_dumps;
      PyObject* m_loads;
  };
}
</code>

Then I compile and link the extensionmodule (BTW, Im using VC++6 an gcc
2.95.2 and (now) boost.python 1.28 ).
In python I have the folowing code:

<code>
# test.py

# A simple testclass, to show that the __del__ message for a loaded object
won't be called:
class test:
  def __init__(self, p1, p2): # so we have some data to pickle
    self.p1 = p1
    self.p2 = p2

  def __del__(self):
    print "__del__-method called", self.p1, self.p2

import pickletest # the extensionmodule

def save():
  t=test(1, 2)
  s=pickletest.to_script(t)
  open("pickled.txt", "w").write(s)
  # here t.__del__() is invoked

def load():
  f=open("pickled.txt", "r")
  s = f.readlines()
  # append all lines in f. I'm sure there is a more elegant way, but who
cares...
  s1 =""
  for i in s:
    s1 += i
  t=pickletest.get_object(s1)
  # here t.__del__() is NOT invoked!!!

save()
load()

</code>

I hope the code isn't too extensive.
Thanks in advance for your help

Norbert

--
Norbert Riedlin
Tel.: +49 (0)7243-2176-24
Fax.: +49 (0)7243-2176-19
mailto:norbert.riedlin at netatec.de

netatec GmbH
Am Hardtwald 3
76275 Ettlingen