Getting Pandas NaT to propagate like NaN

Skip Montanaro skip.montanaro at gmail.com
Tue Oct 30 08:59:05 EDT 2018


I'm trying to take the min and max of a couple Pandas Series objects in the
face of NaT. np.minimum and np.maximum work the way I want if the elements
are floats. For example:

>>> s1
0    0.0
1    1.8
2    3.6
3    5.4
dtype: float64
>>> s2
0    10.0
1    17.0
2     NaN
3    14.0
dtype: float64
>>> np.maximum(s1, s2)
0    10.0
1    17.0
2     NaN
3    14.0
dtype: float64
>>> np.minimum(s1, s2)
0    0.0
1    1.8
2    NaN
3    5.4
dtype: float64

This doesn't work if s1 and s2 are datetime64[ns] objects:

>>> s1
0   2199-12-31
1   2199-12-31
2   2199-12-31
3   2199-12-31
dtype: datetime64[ns]
>>> s2
0          NaT
1   2018-10-30
2          NaT
3          NaT
dtype: datetime64[ns]
>>> np.maximum(s1, s2)
0   2199-12-31
1   2199-12-31
2   2199-12-31
3   2199-12-31
dtype: datetime64[ns]
>>> np.minimum(s1, s2)
0   2199-12-31
1   2018-10-30
2   2199-12-31
3   2199-12-31
dtype: datetime64[ns]

After doing a bit of reading, I came to realize NaT is only approximately
NaN, the latter having a proper floating point representation. Further
reading suggested no simple way to have NaT "pollute" these comparisons.
What's the correct way to get NaT to propagate in min/max comparisons the
way NaN does in a floating point context?

Thanks,

Skip



More information about the Python-list mailing list