[Cython] array expressions

[mark florisson]

On 16 October 2012 18:48, Robert Bradshaw <robertwb at gmail.com> wrote:
> On Sun, Oct 14, 2012 at 6:13 AM, mark florisson
> <markflorisson88 at gmail.com> wrote:
>> On 14 October 2012 14:05, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>> mark florisson, 14.10.2012 13:59:
>>>> The problem with minivect as a package is that it caters to different
>>>> projects, which have different requirements. Cython and minivect are
>>>> quite closely coupled, and any future change, or in the future any
>>>> older version may not have the functionality Cython needs, it's not
>>>> exactly a stable API at this point.
>>>
>>> Ok, understood.
>>>
>>>
>>>> For instance Numba needs python
>>>> 2.7, whereas Cython needs to be compatible with python 2.4.
>>>>
>>>> Before releasing minivect I'll verify every time that it doesn't break
>>>> Cython, but I currently have no real promises for backwards or forward
>>>> compatibility. And that is really because not all use cases have yet
>>>> been anticipated, and some really require a change, as I've already
>>>> seen with Numba.
>>>>
>>>> We could list minivect as a dependency, which works for
>>>> easy_install/pip users, but I just foresee numerous people running
>>>> into problems that didn't install with pip, and I don't think an
>>>> exclusion of a 300kb addition is worth any of that.
>>>
>>> Fine. In that case, I'm for not making minivect a separate package at all
>>> but including it directly and considering it a part of Cython (and Numba
>>> etc.) until there is enough of an interface to make it a reusable separate
>>> package, or at least to support a separate installation and independent
>>> update. Basically, if you can't update it separately, there's no use in
>>> installing it separately.
>>>
>>> As long as we handle this so, we should take care to keep the generic parts
>>> in their separate package directory and the Cython specific parts in
>>> Cython, and try to keep the interface between the two as cleanly separate
>>> as possible, so that we can actually reach a point where both have an
>>> interface. I would guess that the need to support Numba from the same
>>> source base will encourage this kind of separation anyway.
>>
>> Yes, definitely.
>>
>>> Note that this means that minivect will fall under the release schedules of
>>> Cython and Numba (independently), instead of really having its own releases.
>>
>> It can have its own releases as well, but currently there isn't much
>> point :) Minivect can be developed independent of the releases, since
>> Cython and Numba need to explicitly pull in the changes. Let's make a
>> habit of squashing the minivect pulls to avoid its history.
>>
>> I'll also wait for Dag and Robert to see if they have a (final)
>> opinion before merging the subtree.
>
> As I mentioned on the pull request, looks good to me. Given the
> (somewhat) tight coupling with the AST but the desire to use the
> codebase for multiple projects, a subtree seems to make the most sense
> (until/if we have some kind of a plugin system).
>
> - Robert
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

Great, thanks for the review Robert. There seems to be a refcount
issue with python 3:

cython at sage:/jenkins/workspaces/cython-mark-build/PYVERSION/py31/build/lib.linux-x86_64-3.1-pydebug$
/jenkins/workspaces/cython-mark-build/PYVERSION/py31/python/bin/python
Python 3.1.5+ (default:5a6fa1b8767f, Apr 11 2012, 23:32:58)
[GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from Cython.Compiler import Main
[98632 refs]
>>> ^D
[98632 refs]
python: Modules/gcmodule.c:327: visit_decref: Assertion
`gc->gc.gc_refs != 0' failed.

The object being visited is a UtilityCode object. I think the error
means the object is being visited more often than it's refcount value,
which means it's not increffed properly somewhere. I'm not sure how/if
my changes introduced this, has this problem been encountered recently
in Cython's master?