[Cython] array expressions

[Robert Bradshaw]

On Sun, Oct 21, 2012 at 5:39 AM, mark florisson
<markflorisson88 at gmail.com> wrote:
> On 16 October 2012 18:48, Robert Bradshaw <robertwb at gmail.com> wrote:
>> On Sun, Oct 14, 2012 at 6:13 AM, mark florisson
>> <markflorisson88 at gmail.com> wrote:
>>> On 14 October 2012 14:05, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>>> mark florisson, 14.10.2012 13:59:
>>>>> The problem with minivect as a package is that it caters to different
>>>>> projects, which have different requirements. Cython and minivect are
>>>>> quite closely coupled, and any future change, or in the future any
>>>>> older version may not have the functionality Cython needs, it's not
>>>>> exactly a stable API at this point.
>>>>
>>>> Ok, understood.
>>>>
>>>>
>>>>> For instance Numba needs python
>>>>> 2.7, whereas Cython needs to be compatible with python 2.4.
>>>>>
>>>>> Before releasing minivect I'll verify every time that it doesn't break
>>>>> Cython, but I currently have no real promises for backwards or forward
>>>>> compatibility. And that is really because not all use cases have yet
>>>>> been anticipated, and some really require a change, as I've already
>>>>> seen with Numba.
>>>>>
>>>>> We could list minivect as a dependency, which works for
>>>>> easy_install/pip users, but I just foresee numerous people running
>>>>> into problems that didn't install with pip, and I don't think an
>>>>> exclusion of a 300kb addition is worth any of that.
>>>>
>>>> Fine. In that case, I'm for not making minivect a separate package at all
>>>> but including it directly and considering it a part of Cython (and Numba
>>>> etc.) until there is enough of an interface to make it a reusable separate
>>>> package, or at least to support a separate installation and independent
>>>> update. Basically, if you can't update it separately, there's no use in
>>>> installing it separately.
>>>>
>>>> As long as we handle this so, we should take care to keep the generic parts
>>>> in their separate package directory and the Cython specific parts in
>>>> Cython, and try to keep the interface between the two as cleanly separate
>>>> as possible, so that we can actually reach a point where both have an
>>>> interface. I would guess that the need to support Numba from the same
>>>> source base will encourage this kind of separation anyway.
>>>
>>> Yes, definitely.
>>>
>>>> Note that this means that minivect will fall under the release schedules of
>>>> Cython and Numba (independently), instead of really having its own releases.
>>>
>>> It can have its own releases as well, but currently there isn't much
>>> point :) Minivect can be developed independent of the releases, since
>>> Cython and Numba need to explicitly pull in the changes. Let's make a
>>> habit of squashing the minivect pulls to avoid its history.
>>>
>>> I'll also wait for Dag and Robert to see if they have a (final)
>>> opinion before merging the subtree.
>>
>> As I mentioned on the pull request, looks good to me. Given the
>> (somewhat) tight coupling with the AST but the desire to use the
>> codebase for multiple projects, a subtree seems to make the most sense
>> (until/if we have some kind of a plugin system).
>>
>> - Robert
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
>
> Great, thanks for the review Robert. There seems to be a refcount
> issue with python 3:
>
> cython at sage:/jenkins/workspaces/cython-mark-build/PYVERSION/py31/build/lib.linux-x86_64-3.1-pydebug$
> /jenkins/workspaces/cython-mark-build/PYVERSION/py31/python/bin/python
> Python 3.1.5+ (default:5a6fa1b8767f, Apr 11 2012, 23:32:58)
> [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from Cython.Compiler import Main
> [98632 refs]
>>>> ^D
> [98632 refs]
> python: Modules/gcmodule.c:327: visit_decref: Assertion
> `gc->gc.gc_refs != 0' failed.
>
> The object being visited is a UtilityCode object. I think the error
> means the object is being visited more often than it's refcount value,
> which means it's not increffed properly somewhere. I'm not sure how/if
> my changes introduced this, has this problem been encountered recently
> in Cython's master?

Not that I'm aware of. The refnanny is supposed to catch this kind of
thing, is it enabled?

- Robert