[Cython] some pull requests for 0.16?

[Robert Bradshaw]

On Wed, Mar 21, 2012 at 6:47 AM, mark florisson
<markflorisson88 at gmail.com> wrote:
> On 20 March 2012 18:51, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> mark florisson, 20.03.2012 17:40:
>>> On 18 March 2012 11:58, Stefan Behnel wrote:
>>>> I put up two new pull requests on github:
>>>>
>>>> Implementation of PEP 380 (yield from):
>>>> https://github.com/cython/cython/pull/96
>>>>
>>>> Rewrite of dict iteration:
>>>> https://github.com/cython/cython/pull/95
>>>>
>>>> Given that the release of 0.16 has currently slowed down a bit, and given
>>>> that these are really nice features, could someone (and especially Mark, as
>>>> the responsible release manager) take a look at them and give an opinion
>>>> regarding their suitability for 0.16?
>>>
>>> Sorry about the slowdown, my BT internet connection at home got closed
>>> off, so working on a release is rather hard, hopefully that will be
>>> fixed within the next few days.

Thanks for the status update. Is there a separate release branch, or
are you working off the github master? Is there a list of
issues/blockers somewhere?

>>> Anyway, during the sprints Dag worked
>>> on the numpy attributes accesses to generate non-deprecated numpy
>>> code, I'm not sure how far along that is and if we can get that in as
>>> well, or have it wait until 0.16.1.
>>
>> I wouldn't mind if my changes waited for 0.16.1 as well, as long as that's
>> not too long from now. There will certainly be things to fix after the
>> release anyway.
>>
>>
>>> I worked on enhancing fused types runtime dispatching (e.g. select the
>>> right specialization at runtime from the argument types), especially
>>> on matching buffers. It's also a bit of a rewrite as the complexity
>>> grew quite a bit and ndarray and fused types were broken in subtle
>>> ways (I would do the pull request now if I hadn't left the code at
>>> home :).
>>
>> I took a tiny look at that when I wrote a test case and was surprised to
>> see that it imports "sys" inside of the generated function. Looks like
>> there's a bit left to optimise.
>>
>> I faintly remember a blog post (or more than one) by PJ Eby when he wrote a
>> "generic functions" module. It uses type dispatching as well, and he wrote
>> about the ways he found to tune that. Might be an interesting read.
>>
>
> Thanks, I'll look into that. Maybe the modules could be bound as
> locals through default arguments.
>
>>> Finally, there is a serious bug in the reversed(range()) optimization
>>> that Mike Graham discovered when working on the range() optimization
>>> with a runtime step, which is basically wrong for any step that is not
>>> 1 or -1 (it simply swaps the bounds with an offset by 1, but it
>>> actually has to figure out what the actual last value was in the range
>>> and use that as the start instead). I personally consider this a
>>> blocker, so I'll try contacting Mike and see if he is still
>>> (interested in) working on that.
>>
>> Interesting. Yes, I can well imagine that being wrong. These things are
>> exceedingly tricky.
>>
>> However, it's easy to disable the optimisation for a specific case - likely
>> easier than fixing it.
>>
>> Also, I don't consider this a blocker. It's not like it works in the
>> current release, so it's not a regression compared to that.
>>
>
> Yes, it's not a regression, but I find it hard to release something
> that is so broken, and I feel it kind of represents a lack of care
> which may make people believe Cython can't be taken seriously.

I'm usually in the "not a regression" camp as well, but this is pretty
fundamental. Sounds like an easy fix, or at least an easy-to-disable
optimization.

>>> As for the two pull requests, both are quite large, but the dict
>>> iteration rewrite is more like an enhancement whereas the 'yield from'
>>> is really a new (somewhat big) feature. I would personally prefer to
>>> merge only enhancements and bug fixes, but not new features.
>>
>> Funny. I would have proposed the opposite, because I consider the "yield
>> from" implementation (as a new language feature) safer than the dict
>> iteration (which impacts existing code). As I said, I wouldn't mind
>> postponing both to 0.16.1, but I think it would be better to get them out
>> and have people use them, so that we can also fix the bugs earlier.
>>
>
> A good point. My reasoning was that we want less new functionality per
> release, and keeping 'yield from' out of 0.16 makes us push out the
> next release earlier as we will have pending functionality we want to
> get out there. If the dict iteration is tested well enough (and it
> looked to me like that), then hopefully code in sage or lxml will
> break if there are leftover bugs, or our next beta might catch them
> (or some). If you do want it in now, then that's fine with me as well,
> you have a point.

Has this been pushed into the devel branch for sufficient testing
there (e.g. on Sage)? Personally, I'd like to see the release get out
sooner rather than later.

- Robert