[Ironpython-users] numpy in IronPython

[Olof Bjarnason]

Why isn't CPython+NumPy+SciPy (or what you need on top of NumPy)
enough? It's been tested and maintained for a long time, and works
quite well?

It does seem like a daunting task to try and build and maintain
something separate from the mainline NumPy/SciPy community...



On 3 June 2014 13:39, Doug Blank <doug.blank at gmail.com> wrote:
> I've tried to capture all of the links and options towards developing a
> numpy for IronPython here:
>
> http://calicoproject.org/Numpy
>
> Please let me know if I have missed something, or if you would like to add
> to that page.
>
> My summary so far:
>
> 1) a new wrapper (based on IronClad or Cython) would create fast code, and
> could utilize the current and future versions of numpy, but would (I
> suspect) be low-level, high-maintenance in keeping it working on
> multi-platforms/architectures. This wrapper work might also allow other
> packages in the SciPy to work. Requires low-level C#, CLR, and
> IronPython-specific skills to develop (a small group of people have these
> skills; only IronPython users would benefit).
>
> 2) A pure-Python version would be a lot of work (perhaps building on PyPy's
> RPython version and converting their C) and be slow, but would be little
> maintenance as most of the details for the current version of numpy would be
> static. Requires generic Python skills to develop (a large group of people
> have these skills; any generic Python implementation could use).
>
> -Doug
>
>
> On Mon, Jun 2, 2014 at 2:09 PM, Doug Blank <doug.blank at gmail.com> wrote:
>>
>> On Mon, Jun 2, 2014 at 1:41 PM, Steve Baer <steve at mcneel.com> wrote:
>>>
>>> > That is the code here that I mentioned in the first post:
>>> > https://www.enthought.com/repo/.iron/
>>>
>>> Unless I'm missing something, that's the binary installation and not the
>>> actual source code. I believe Enthought used managed C++ which does tie it
>>> to Windows, but it would also be good to see since it would give me a good
>>> idea about the effort involved in trying to make a C# DLL with pInvokes to a
>>> C DLL. I've done a lot of this type of work in the past and it usually isn't
>>> very hard once I've gotten things set up correctly (like having an
>>> executable to automatically generate the pinvokes from the exported C
>>> functions.)
>>
>>
>> I think you are correct. A bit more googling found:
>>
>> http://blog.enthought.com/python/scipy-for-net/#.U4y83x92nfE
>>
>> which states:
>>
>> """
>> The first release of SciPy and NumPy for .NET are available now as binary
>> distributions from SciPy.org or directly from Enthought. All of the code for
>> these and the supporting projects are open source and available at the links
>> below.
>>
>> NumPy for .NET:https://github.com/numpy/numpy-refactor
>> SciPy for .NET:https://github.com/jasonmccampbell/scipy-refactor
>> Cython for .NET:
>> https://bitbucket.org/jasonmccampbell/cython-for-ironpython
>> FWrap extensions:https://github.com/jasonmccampbell/fwrap
>> """
>>
>> So it looks like the Enthought project was using the
>> cython-for-ironpython.
>>
>> Please let us know what you think about making this work for C#; thanks!
>>
>> -Doug
>>
>>>
>>>
>>> -Steve
>>>
>>> Steve Baer
>>> Robert McNeel & Associates
>>> www.rhino3d.com
>>>
>>>
>>> On Mon, Jun 2, 2014 at 10:28 AM, Doug Blank <doug.blank at gmail.com> wrote:
>>>>
>>>> On Mon, Jun 2, 2014 at 1:13 PM, Steve Baer <steve at mcneel.com> wrote:
>>>>>
>>>>> We should also check with the Enthought guys to see what they did and
>>>>> if they are willing to share.
>>>>
>>>>
>>>> That is the code here that I mentioned in the first post:
>>>>
>>>> https://www.enthought.com/repo/.iron/
>>>>
>>>> I can't tell what license this is done under. Also, it is Windows only.
>>>>
>>>> It looks like a Cython for IronPython might be closer to working:
>>>>
>>>> https://bitbucket.org/cwitty/cython-for-ironpython/overview
>>>>
>>>> Especially here considering these:
>>>>
>>>>
>>>> https://bitbucket.org/cwitty/cython-for-ironpython/src/9dc7e1a2d56a/Cython/Includes/posix/?at=default
>>>>
>>>> https://bitbucket.org/cwitty/cython-for-ironpython/commits/9dc7e1a2d56a3e3c8a7b3c7a5d23073a54b2b814
>>>>
>>>> But, I haven't worked much with Python and C libraries.
>>>>
>>>> -Doug
>>>>
>>>>>
>>>>>
>>>>> -Steve
>>>>>
>>>>>
>>>>> On Sun, Jun 1, 2014 at 6:29 AM, Doug Blank <doug.blank at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> On Sun, Jun 1, 2014 at 3:05 AM, Jeff Hardy <jdhardy at gmail.com> wrote:
>>>>>>>
>>>>>>> On Sat, May 31, 2014 at 3:23 PM, Doug Blank <doug.blank at gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On Fri, May 30, 2014 at 5:22 PM, Steve Baer <steve at mcneel.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I would definitely be interested in helping, but don't exactly know
>>>>>>>>> where to start. We have a lot of users who would love to get access to numpy
>>>>>>>>> on our OSX and 64bit windows versions of our product.  This is only going to
>>>>>>>>> become a bigger problem in the future since we will probably only have a
>>>>>>>>> 64bit version for Window in the next version of Rhino.
>>>>>>>>
>>>>>>>>
>>>>>>>> I agree. Let's start a serious discussion about how to solve the
>>>>>>>> lack of numpy in IronPython.
>>>>>>>>
>>>>>>>> We could look into using ctypes (IronClad?) and wrap what already
>>>>>>>> exists.
>>>>>>>>
>>>>>>>> We could look into a cross-platform DLL drop-in replacement.
>>>>>>>>
>>>>>>>> Between speed and compatibility, initially I'm most interested in
>>>>>>>> compatibility. But speed should be a long term goal.
>>>>>>>>
>>>>>>>> We could write a pure-python prototype initially, and slowly move
>>>>>>>> that to C#, or another CLR language. That would be useful for all
>>>>>>>> non-C-based Python implementations, and would probably be quickest to write
>>>>>>>> and test.
>>>>>>>>
>>>>>>>> A related note: Python3 just added a new matrix multiplication
>>>>>>>> operator [1]. Hope to see more numpy-related functionality in standard
>>>>>>>> Python in the future.
>>>>>>>>
>>>>>>>> Other ideas? Where to start?
>>>>>>>
>>>>>>>
>>>>>>> I would start by asking the NumPy team what the best option is, and
>>>>>>> seeing what the NumPyPy team are doing - the more work that can be shared,
>>>>>>> the better.
>>>>>>
>>>>>>
>>>>>> That is a good idea, and I started to try to define what we might mean
>>>>>> by "best option." Almost all of the work I have seen coming from most
>>>>>> related projects would define best in terms of speed. However, this comes at
>>>>>> a pretty big cost in terms of maintenance---for example, having wrapped C
>>>>>> libraries compiled for each platform, for each bus size (32, 64).
>>>>>>
>>>>>> In poking around, I think the Jython needs might be closest to ours.
>>>>>> Looks like they have two starts: a pure-Java library based on the older
>>>>>> Numeric [1], and a native interface for talking directly to C [2]. The
>>>>>> native interface is a long term project, and is not yet to the point of
>>>>>> working with numpy. As a quick test, I tried to IKVM-convert the pure Java
>>>>>> jar file into a DLL. I think that could work (eventually) but would require
>>>>>> bringing a lot of Jython, and would probably always be a little wonky. In
>>>>>> addition, that API is the older Numeric.
>>>>>>
>>>>>> A third option, as mentioned by Ivan, is to have a bridge to CPython
>>>>>> interfacing with numpy. But that would make CPython a dependency---not
>>>>>> really something that many IronPython users would appreciate.
>>>>>>
>>>>>> Of course, if one is looking for doing numeric operations, you could
>>>>>> use a different .NET/Mono math library. But what I am interested in is the
>>>>>> numpy API, so that other code will be usable in IronPython.
>>>>>>
>>>>>> To me, it looks like the best bet at this point in time is to write
>>>>>> our own. The next question is: write it in pure Python, or C#/F#/etc.
>>>>>>
>>>>>> If we write it in pure Python, there is the chance that some Jython
>>>>>> developers (and maybe other Python implementation people) might be
>>>>>> interested in helping. It would be slow, but we could rely on Python for
>>>>>> handling type operations (float times int) to do the right thing. It would
>>>>>> also be immediately usable by future CPython users. It could be the case
>>>>>> that a future Python implementation could do some JITting to make it run
>>>>>> fast enough. (A pure Python version might also be useful for educational
>>>>>> uses, as I presume it would be more easily understood by Python students).
>>>>>>
>>>>>> If we write it as a CLR library, it would be as fast as managed code
>>>>>> would allow, and be available for other CLR languages (like F#, Boo, etc).
>>>>>> But we would probably be alone in developing it, as it is mostly of interest
>>>>>> to Python users using numpy and the CLR.
>>>>>>
>>>>>> I guess I am leaning towards a pure Python implementation of the
>>>>>> latest numpy API. Perhaps followed up by a DLL version.
>>>>>>
>>>>>> -Doug
>>>>>>
>>>>>> [1] - https://bitbucket.org/zornslemon/jnumeric-ra/overview
>>>>>> [2] - http://jyni.org/
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> - Jeff
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
> _______________________________________________
> Ironpython-users mailing list
> Ironpython-users at python.org
> https://mail.python.org/mailman/listinfo/ironpython-users
>