[pypy-dev] Memap in numpypy

[Amaury Forgeot d'Arc]

Hi, this is definitely doable, but needs some work.
Here are the tasks I identified:

- copy "memmap.py" from the official numpy, and write a unit test (there is
a nice docstring in the previous file)

- add support in numpy for buffers with a fixed address
  (in interp-level terms: a RWBuffer with a get_raw_address() method)

- have buffer(mmap) return such a buffer, very very similar to
array.ArrayType.

The three tasks are quite independent, not too difficult, and could be a
nice start for newcomers...
I'll be happy to help.



2013/6/14 Mike Beller <mike at tradeworx.com>

> Hello
>
> Do you think it is likely that the memap capabilities of numpy will find
> their way in to numpypy any time soon?
>
> It seems to me that some people think memap is a relatively unimportant
> aspect of numpy.  But I do not think so.  Because of the way the linux IO
> subsystem and virtual memory systems interact, memap enables numpy to have
> high performance access to very large data sets -- it helps make numpy
> relevant to "Big Data".
>
> The code to allow numpy to support memap doesn't seem very large.  But,
> while I have tried reading through the code, I really can not tell whether
> the same is true for numpypy, or whether it is a large endeavor (for
> example, due to some kind of pypy memory management architectural issue).
>
> I'm interested in any input on this.
>
> Mike Beller
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>


-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130614/19305b6a/attachment.html>