PyCObject & malloc creating memory leak

[Antoine Pitrou]

On Thu, 30 Sep 2010 04:06:03 -0700 (PDT)
Tom Conneely <tom.conneely at gmail.com> wrote:
> Thanks for your reply, you've given me plenty to think about
> 
> On Sep 29, 11:51 pm, Antoine Pitrou <solip... at pitrou.net> wrote:
> >
> > > My original plan was to have the data processing and data acquisition
> > > functions running in separate processes, with a multiprocessing.Queue
> > > for passing the raw data packets. The raw data is read in as a char*,
> > > with a non constant length, hence I have allocated memory using
> > > PyMem_Malloc and I am returning from the acquisition function a
> > > PyCObject containing a pointer to this char* buffer, along with a
> > > destructor.
> >
> > That sounds overkill, and I also wonder how you plan to pass that
> > object in a multiprocessing Queue (which relies on objects being
> > pickleable). Why don't you simply create a PyString object instead?
> 
> Could you elaborate on why you feel this is overkill? Also, your right
> about
> passing the PyCObjects through a Queue, something which I hadn't
> really
> considered, so I've switched to using python strings as you
> suggested,
> an overhead I hoped to avoid but you can't win them all I suppose.

Well, there should be no overhead. Actually, a string should be cheaper
since:
- the string contents are allocated inline with the PyObject header
- while your PyCObject contents were allocated separately (two
  allocations rather than one)

Regards

Antoine.