[Numpy-discussion] problems with auto detection and broadcasting for userdefined numpy types

[Tom Denniston]

I have a userdefined numpy type for mx.DateTime and one for
mx.DateTimeDelta.  Info on these packages is available here
(http://www.egenix.com/files/python/eGenix-mx-Extensions.html).

In any case most things work.

What I am left with is three problems:


1. I cannot figure out how to cleanly or even at all, get type detection to work
So this works:
In [35]: numpy.array([D('2002-01-02 00:00:00.00'), D('2002-01-03
00:00:00.00')], dtype=numpy.mxDateTimeType)
Out[35]: array([2002-01-02 00:00:00.00, 2002-01-03 00:00:00.00],
dtype=mxNumpyDateTime)
But this doesn't:
In [36]: numpy.array([D('2002-01-02 00:00:00.00'), D('2002-01-03 00:00:00.00')])
Out[36]: array([2002-01-02 00:00:00.00, 2002-01-03 00:00:00.00], dtype=object)

I.e. if I don't specify the type it gives me an object dtype.  This is
unlike the behavior for strings, ints, etc.  I think I know where the
C code is that controls this but it is not obvious to me how to change
it without making it dependent on mx.DateTime at compile time or
having to put a bunch of #define #ifdef magic around it.  It would be
preferable to figure out a way to specify autodetection, from the
outside, so to speak.

2. I can get printing to work but only using a quasi hack in the numpy
arrayprint.py code.  It works with or without presense of the mx
libraries but doesn't seem to me entirely clean.

3. Broadcasting has bugs.  See example below:
The basic case (2,) - (2,) arrays works.
In [37]: numpy.array([D('2002-01-02 00:00:00.00'), D('2002-01-03
00:00:00.00')], dtype=numpy.mxDateTimeType) -
numpy.array([DateTime.DateTimeDeltaFromDays(1)],
dtype=numpy.mxDateTimeDeltaType)
Out[37]: array([2002-01-01 00:00:00.00, 2002-01-02 00:00:00.00],
dtype=mxNumpyDateTime)
The broadcasting case of (2,) - (1,) works:
In [39]: numpy.array([D('2002-01-02 00:00:00.00'), D('2002-01-03
00:00:00.00')], dtype=numpy.mxDateTimeType) -
numpy.array([DateTime.DateTimeDeltaFromDays(20)],
dtype=numpy.mxDateTimeDeltaType)
Out[39]: array([2001-12-13 00:00:00.00, 2001-12-14 00:00:00.00],
dtype=mxNumpyDateTime)
The extremely similar (2,) - () array:
In [40]: numpy.array([D('2002-01-02 00:00:00.00'), D('2002-01-03
00:00:00.00')], dtype=numpy.mxDateTimeType) -
numpy.array(DateTime.DateTimeDeltaFromDays(20),
dtype=numpy.mxDateTimeDeltaType)
---------------------------------------------------------------------------
<type 'exceptions.TypeError'>             Traceback (most recent call last)

/home/tom/src/<ipython console> in <module>()

<type 'exceptions.TypeError'>: function not supported for these types,
and can't coerce safely to supported types
Finally the (2,) - scalar doesn't work either:

In [41]: numpy.array([D('2002-01-02 00:00:00.00'), D('2002-01-03
00:00:00.00')], dtype=numpy.mxDateTimeType) -
DateTime.DateTimeDeltaFromDays(20)
---------------------------------------------------------------------------
<type 'exceptions.TypeError'>             Traceback (most recent call last)

/home/tom/src/<ipython console> in <module>()

<type 'exceptions.TypeError'>: function not supported for these types,
and can't coerce safely to supported types

The final one might not work even if broadcasting worked properly
simply because of the autodetection problem listed in 2.  The (2,) -
(), however, should work, I would think, because it has the type
explicitly set.



--Tom