[Python-Dev] datetime nanosecond support

[Vincenzo Ampolo]

On 07/24/2012 06:46 PM, Guido van Rossum wrote:
> 
> You're welcome.

Hi Guido,

I'm glad you spent your time reading my mail. I would have never
imagined that my mail could come to your attention.

> Have you read PEP 410 and my rejection of it
> (http://mail.python.org/pipermail/python-dev/2012-February/116837.html)?
> Even though that's about using Decimal for timestamps, it could still
> be considered related.

I've read it and point 5 is very like in this issue. You said:

"[...]
I see only one real use case for nanosecond precision: faithful
copying of the mtime/atime recorded by filesystems, in cases where the
filesystem (like e.g. ext4) records these times with nanosecond
precision. Even if such timestamps can't be trusted to be accurate,
converting them to floats and back loses precision, and verification
using tools not written in Python will flag the difference. But for
this specific use case a much simpler set of API changes will suffice;
only os.stat() and os.utime() need to change slightly (and variants of
os.stat() like os.fstat()).
[...]"

I think that's based on a wrong hypothesis: just one case -> let's
handle in a different way (modifying os.stat() and os.utime()).
I would say: It's not just one case, there are at lest other two
scenarios. One is like mine, parsing network packets, the other one is
in parsing stock trading data.
But in this case there is no os.stat() or os.utime() that can be
modified. I've to write my own class to deal with time and loose all the
power and flexibility that the datetime module adds to the python language.

> Not every use case deserves an API change. :-)
> 
> First you will have to show how you'd have to code this *without*
> nanosecond precision in datetime and how tedious that is. (I expect
> that representing the timestamp as a long integer expressing a posix
> timestamp times a billion would be very reasonable.)

Yeah that's exactly how we built our Time class to handle this, and we
wrote also a Duration class to represent timedelta.
The code we developed is 383 python lines long but is not comparable
with all the functionalities that the datetime module offers and it's
also really slow compared to native datetime module which is written in C.

As you may think using that approach in a web application is very
limiting since there is no strftime() in this custom class.

I cannot share the code right now since It's copyrighted by the company
I work for but I've asked permission to do so. I just need to wait
tomorrow morning (PDT time) so they approve my request. Looking at the
code you can see how tedious is to try to remake all the conversions
that are already implemented on the datetime module.
Just let me know if you actually want to have a look at the code.

> 
> I didn't read the entire bug, but it mentioned something about storing
> datetimes in databases. Do databases support nanosecond precision?
> 

Yeah. According to
http://wiki.ispirer.com/sqlways/postgresql/data-types/timestamp at least
Oracle support timestamps with nanoseconds accuracy, SQL server supports
100 nanosecond accuracy.
Since I use Postgresql personally the best way to accomplish it (also
suggested by the #postgresql on freenode) is to store the timestamp with
nanosecond (like 1343158283.880338907242) as bigint and let the ORM (so
a python ORM) do all the conversion job.
An yet again, having nanosecond support in datetime would make things
much more easy.

While writing this mail Chris Lambacher answered with more data about
nanosecond support on databases

Best Regards,
-- 
Vincenzo Ampolo
http://vincenzo-ampolo.net
http://goshawknest.wordpress.com