[Numpy-discussion] tests for casting table? (was: Numpy 1.7b1 API change cause big trouble)

[Frédéric Bastien]

Hi,

Finally, the change about the casting rule was done in NumPy 1.6. It
is our test that checked specifically for numpy 1.6 behavior. But
adding the test to make sure it don't change is an excellent idea.

Fred

On Thu, Sep 20, 2012 at 7:30 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
> On Thu, Sep 20, 2012 at 2:20 PM, Travis Oliphant <travis at continuum.io>
> wrote:
>>
>> Here are a couple of scripts that might help (I used them to compare
>> casting tables between various versions of NumPy):
>>
>> Casting Table Creation Script
>> ========================
>> import numpy as np
>>
>> operators = np.set_numeric_ops().values()
>> types = '?bhilqpBHILQPfdgFDGO'
>> to_check = ['add', 'divide', 'minimum', 'maximum', 'remainder',
>> 'true_divide', 'logical_or', 'bitwise_or', 'right_shift', 'less', 'equal']
>> operators = [op for op in operators if op.__name__ in to_check]
>>
>>
>> def type_wrap(op):
>>     def func(obj1, obj2):
>>         try:
>>             result = op(obj1, obj2)
>>             char = result.dtype.char
>>         except:
>>             char = 'X'
>>         return char
>>
>>     return func
>>
>> def coerce():
>>     result = {}
>>     for op in operators:
>>         d = {}
>>         name = op.__name__
>>         print name
>>         op = type_wrap(op)
>>         for type1 in types:
>>             s1 = np.dtype(type1).type(2)
>>             a1 = np.dtype(type1).type([1,2,3])
>>             for type2 in types:
>>                 s2 = np.dtype(type2).type(1)
>>                 a2 = np.dtype(type2).type([2,3,4])
>>                 codes = []
>>                 # scalar <op> scalar
>>                 codes.append(op(s1, s2))
>>                 # scalar <op> array
>>                 codes.append(op(s1, a2))
>>                 # array <op> scalar
>>                 codes.append(op(a1, s2))
>>                 # array <op> array
>>                 codes.append(op(a1, a2))
>>                 d[type1,type2] = codes
>>         result[name] = d
>>
>>         #for check_key in to_check:
>>         # for key in result.keys():
>>         #    if key == check_key:
>>         #        continue
>>         #    if result[key] == result[check_key]:
>>         #        del result[key]
>>         #assert set(result.keys()) == set(to_check)
>>     return result
>>
>> import sys
>> if sys.maxint > 2**33:
>>     bits = 64
>> else:
>>     bits = 32
>>
>> def write():
>>     import cPickle
>>     file = open('coercion-%s-%sbit.pkl'%(np.__version__, bits),'w')
>>     cPickle.dump(coerce(),file,protocol=2)
>>     file.close()
>>
>> if __name__ == '__main__':
>>     write()
>>
>>
>>
>>
>>
>> Comparison Script
>> ================
>>
>> import numpy as np
>>
>>
>> def compare(result1, result2):
>>     for op in result1.keys():
>>         print "**** ", op, " ****"
>>         if op not in result2:
>>             print op, " not in the first"
>>         table1 = result1[op]
>>         table2 = result2[op]
>>         if table1 == table2:
>>             print "Tables are the same"
>>         else:
>>             if set(table1.keys()) != set(table2.keys()):
>>                 print "Keys are not the same"
>>                 continue
>>             for key in table1.keys():
>>                 if table1[key] != table2[key]:
>>                     print "Different at ", key, ": ", table1[key],
>> table2[key]
>>
>> import cPickle
>> import sys
>>
>> if __name__ == '__main__':
>>     name1 = 'coercion-1.5.1-64bit.pkl'
>>     name2 = 'coercion-1.6.1-64bit.pkl'
>>
>>     if len(sys.argv) > 1:
>>         name1 = 'coercion-%s-64bit.pkl' % sys.argv[1]
>>     if len(sys.argv) > 2:
>>         name2 = 'coercion-%s-64bit.pkl' % sys.argv[2]
>>     result1 = cPickle.load(open(name1))
>>     result2 = cPickle.load(open(name2))
>>     compare(result1, result2)
>>
>>
>>
>> On Sep 20, 2012, at 3:09 PM, Nathaniel Smith wrote:
>>
>> > On Mon, Sep 17, 2012 at 10:22 AM, Matthew Brett
>> > <matthew.brett at gmail.com> wrote:
>> >> Hi,
>> >>
>> >> On Sun, Sep 9, 2012 at 6:12 PM, Frédéric Bastien <nouiz at nouiz.org>
>> >> wrote:
>> >>> The third is releated to change to the casting rules in numpy. Before
>> >>> a scalar complex128 * vector float32 gived a vector of dtype
>> >>> complex128. Now it give a vector of complex64. The reason is that now
>> >>> the scalar of different category only change the category, not the
>> >>> precision. I would consider a must that we warn clearly about this
>> >>> interface change. Most people won't see it, but people that optimize
>> >>> there code heavily could depend on such thing.
>> >>
>> >> It seems to me that it would be a very good idea to put the casting
>> >> table results into the tests to make sure we are keeping track of this
>> >> kind of thing.
>> >>
>> >> I'm happy to try to do it if no-one else more qualified has time.
>> >
>> > I haven't seen any PRs show up from anyone else in the last few days,
>> > and this would indeed be an excellent test to have, so that would be
>> > awesome.
>> >
>
>
> IIRC, there are some scripts in the numpy repository. But I forget where I
> saw them.
>
> Chuck
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>