[Numpy-discussion] Change in scalar upcasting rules for 1.6.x?

[Charles R Harris]

On Mon, Feb 13, 2012 at 10:01 PM, Travis Oliphant <travis at continuum.io>wrote:

>
> On Feb 13, 2012, at 10:14 PM, Charles R Harris wrote:
>
>
>
> On Mon, Feb 13, 2012 at 9:04 PM, Travis Oliphant <travis at continuum.io>wrote:
>
>> I disagree with your assessment of the subscript operator, but I'm sure
>> we will have plenty of time to discuss that.  I don't think it's correct to
>> compare  the corner cases of the fancy indexing and regular indexing to the
>> corner cases of type coercion system.    If you recall, I was quite nervous
>> about all the changes you made to the coercion rules because I didn't
>> believe you fully understood what had been done before and I knew there was
>> not complete test coverage.
>>
>> It is true that both systems have emerged from a long history and could
>> definitely use fresh perspectives which we all appreciate you and others
>> bringing.   It is also true that few are aware of the details of how things
>> are actually implemented and that there are corner cases that are basically
>> defined by the algorithm used (this is more true of the type-coercion
>> system than fancy-indexing, however).
>>
>> I think it would have been wise to write those extensive tests prior to
>> writing new code.   I'm curious if what you were expecting for the output
>> was derived from what earlier versions of NumPy produced.    NumPy has
>> never been in a state where you could just re-factor at will and assume
>> that tests will catch all intended use cases.   Numeric before it was not
>> in that state either.   This is a good goal, and we always welcome new
>> tests.    It just takes a lot of time and a lot of tedious work that the
>> volunteer labor to this point have not had the time to do.
>>
>> Very few of us have ever been paid to work on NumPy directly and have
>> often been trying to fit in improvements to the code base between other
>> jobs we are supposed to be doing.    Of course, you and I are hoping to
>> change that this year and look forward to the code quality improving
>> commensurately.
>>
>> Thanks for all you are doing.   I also agree that Rolf and Charles
>> have-been and are invaluable in the maintenance and progress of NumPy and
>> SciPy.   They deserve as much praise and kudos as anyone can give them.
>>
>>
> Well, the typecasting wasn't perfect and, as Mark points out, it wasn't
> commutative. The addition of float16 also complicated the picture, and user
> types is going to do more in that direction. And I don't see how a new
> developer should be responsible for tests enforcing old traditions, the
> original developers should be responsible for those. But history is
> history, it didn't happen that way, and here we are.
>
> That said, I think we need to show a little flexibility in the corner
> cases. And going forward I think that typecasting is going to need a
> rethink.
>
>
> No argument on any of this.   It's just that this needs to happen at NumPy
> 2.0, not in the NumPy 1.X series.   I think requiring a re-compile is
> far-less onerous than changing the type-coercion subtly in a 1.5 to 1.6
> release.      That's my major point, and I'm surprised others are more
> cavalier about this.
>
> New developers are awesome, and the life-blood of a project.   But, you
> have to respect the history of a code-base and if you are re-factoring code
> that might create a change in corner-cases, then you are absolutely
> responsible for writing the tests if they aren't there already.    That is
> a pretty simple rule.
>
> If you are changing semantics and are not doing a new major version number
> that you can document the changes in, then any re-factor needs to have
> tests written *before* the re-factor to ensure behavior does not change.
> That might be annoying, for sure, and might make you curse the original
> author for not writing the tests you wish were already written --- but it
> doesn't change the fact that a released code has many, many tests already
> written for it in the way of applications and users.   All of these are
> outside of the actual code-base, and may rely on behavior that you can't
> just change even if you think it needs to change.  Bug-fixes are different,
> of course, but it can sometimes be difficult to discern what is a "bug" and
> what is just behavior that seems inappropriate.
>
> Type-coercion, in particular, can be a difficult nut to crack because
> NumPy doesn't always control what happens and is trying to work-within
> Python's stunted type-system.   I've often thought that it might be easier
> if NumPy were more tightly integrated into Python.   For example, it would
> be great if NumPy's Int-scalar was the same thing as Python's int.  Same
> for float and complex.    It would also be nice if you could specify scalar
> literals with different precisions in Python directly.    I've often wished
> that NumPy developers had more access to all the great language people who
> have spent their time on IronPython, Jython, and PyPy instead.
>
>
Going on about casting, somewhere I might still have a table I generated
back around 1.3. Also, Numpy has a mixed C/Numarray system that leads to
some problems, so that should be rationalized at some point.

<snip>

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120213/29d9f803/attachment.html>