[Python-Dev] strftime/strptime locale funnies...

[Brett Cannon]

On 4/5/06, Donovan Baarda <abo at minkirri.apana.org.au> wrote:
> G'day,
>
> Just noticed on Debian (testing), Ubuntu (warty?), and RedHat (old)
> based systems Python's time.strptime() seems to ignore the environment's
> Locale and just uses "C".
>
> Last time I looked at this, time.strptime() leveraged off the platform's
> strptime(), which meant it had all the extra features, bugs and
> missingness of the platform's implementation.
>
> We now seem to be using a Python implementation in _strptime.py. This
> implementation does Locale's by feeding a magic date to time.strftime()
> and figuring out how it formats it.
>

The Python implementation of time.strptime() has been in Python since
summer 2002 so it was first introduced in 2.3 (I can't believe I have
been on python-dev that long!).

> This revealed that time.strftime() is not honouring the Locale settings,
> which is causing the new Python strptime() to also get it wrong.
>

This isn't time.strftime() .  If you look at Modules/timemodule.c:450
you will find ``buflen = strftime(outbuf, i, fmt, &buf);`` for the
actual strftime() call.  Before that the only things that could
possibly change the locale are localtime() or gettmarg().  Everything
else is error-checking of arguments.

> $ set | grep "^LC\|LANG"
> GDM_LANG=en_AU.UTF-8
> LANG=en_AU.UTF-8
> LANGUAGE=en_AU.UTF-8
> LC_COLLATE=C
>
> $ date -d "1999-02-22" +%x
> 22/02/99
>
> $ python
> ...
> >>> import time
> >>> time.strftime("%x", time.strptime("1999-02-22","%Y-%m-%d"))
> '02/22/99'
>
> This is consistent across all three platforms for multiple Python
> versions, including 2.1 and 1.5 (where they were available) which BTW
> don't use the Python implementation of strptime().
>
> This suggests that all three of these platforms have a broken libc
> strftime() implementation... but all three? And why does date work?
>

Beats me.  This could be a locale thing.  If I remember correctly
Python assumes the C locale on some things.  I suspect the reason for
this is in the locale module or libc.  But you can't even find the
word 'locale' or 'Locale' in timemodule.c nor do I know of any calls
that mess with the locale, so I doubt 'time' is at fault for this.

> Can others reproduce this? Have I done something stupid? Is this a bug,
> and in what, libc or Python?
>
> Slightly OT, is it wise to use a Python strptime() on platforms that
> have a perfectly good one in libc? The Python reverse-engineering of
> libc's strftime() output to figure out locale formatting is clever,
> but...
>

The reason it was made the default implementation is for consistency
across platforms.  Since the trouble to implement a pure Python
version was done and it is not a performance critical operation,
consistency across platforms was deemed more important than allowing
various platforms to support whatever directives they chose and having
people writing code the relied upon it.

Plus it has been in there for so long there would be
backwards-compatibility issues if we switched this now.

> I see there have already been bugs submitted about strftime/strptime
> non-symmetry for things like support of extensions. There has also been
> a bug against strptime() Locale switching not working because of caching
> Locale formatting info from the strftime() analysis,

None open, right?  Those should all be closed and fixed.

-Brett