[Python-Dev] [Python-checkins] cpython (merge 3.4 -> default): Issue #22165: Fixed test_undecodable_filename on non-UTF-8 locales.

Sun Aug 17 11:37:44 CEST 2014

This change is okay and not harmful. But I think, It might still not fix
the encoding issue that we encountered on Mac.

[localhost cpython]$ hg log -l 1
changeset:   92128:7cdc941d5180
tag:         tip
parent:      92126:3153a400b739
parent:      92127:a894b629bbea
user:        Serhiy Storchaka <storchaka at gmail.com>
date:        Sun Aug 17 12:21:06 2014 +0300
description:
Issue #22165: Fixed test_undecodable_filename on non-UTF-8 locales.

[localhost cpython]$ ./python.exe -m test.regrtest test_httpservers
[1/1] test_httpservers
test test_httpservers failed -- Traceback (most recent call last):
  File "/Users/skumaran/python/cpython/Lib/test/test_httpservers.py", line
283, in test_undecodable_filename
    .encode(enc, 'surrogateescape'), body)
AssertionError: b'href="%40test_5809_tmp%ED%B3%A7w%ED%B3%B0.txt"' not found
in b'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "
http://www.w3.org/TR/html4/strict.dtd">\n<html>\n<head>\n<meta
http-equiv="Content-Type" content="text/html;
charset=utf-8">\n<title>Directory listing for
tmpj54lc8m1/</title>\n</head>\n<body>\n<h1>Directory listing for
tmpj54lc8m1/</h1>\n<hr>\n<ul>\n<li><a
href="%40test_5809_tmp%25E7w%25F0.txt">@test_5809_tmp%E7w%F0.txt</a></li>\n<li><a
href="test">test</a></li>\n</ul>\n<hr>\n</body>\n</html>\n'

1 test failed:
    test_httpservers

The underlying problem seems to be difference in which os.listdir() which
uses C-API  and os.fsdecode represent the decoded chars. Ref:
http://bugs.python.org/issue22165#msg225428

On Sun, Aug 17, 2014 at 2:52 PM, serhiy.storchaka <
python-checkins at python.org> wrote:

> http://hg.python.org/cpython/rev/7cdc941d5180
> changeset:   92128:7cdc941d5180
> parent:      92126:3153a400b739
> parent:      92127:a894b629bbea
> user:        Serhiy Storchaka <storchaka at gmail.com>
> date:        Sun Aug 17 12:21:06 2014 +0300
> summary:
>   Issue #22165: Fixed test_undecodable_filename on non-UTF-8 locales.
>
> files:
>   Lib/test/test_httpservers.py |  5 +++--
>   1 files changed, 3 insertions(+), 2 deletions(-)
>
>
> diff --git a/Lib/test/test_httpservers.py b/Lib/test/test_httpservers.py
> --- a/Lib/test/test_httpservers.py
> +++ b/Lib/test/test_httpservers.py
> @@ -272,6 +272,7 @@
>      @unittest.skipUnless(support.TESTFN_UNDECODABLE,
>                           'need support.TESTFN_UNDECODABLE')
>      def test_undecodable_filename(self):
> +        enc = sys.getfilesystemencoding()
>          filename = os.fsdecode(support.TESTFN_UNDECODABLE) + '.txt'
>          with open(os.path.join(self.tempdir, filename), 'wb') as f:
>              f.write(support.TESTFN_UNDECODABLE)
> @@ -279,9 +280,9 @@
>          body = self.check_status_and_reason(response, 200)
>          quotedname = urllib.parse.quote(filename, errors='surrogatepass')
>          self.assertIn(('href="%s"' % quotedname)
> -                      .encode('utf-8', 'surrogateescape'), body)
> +                      .encode(enc, 'surrogateescape'), body)
>          self.assertIn(('>%s<' % html.escape(filename))
> -                      .encode('utf-8', 'surrogateescape'), body)
> +                      .encode(enc, 'surrogateescape'), body)
>          response = self.request(self.tempdir_name + '/' + quotedname)
>          self.check_status_and_reason(response, 200,
>                                       data=support.TESTFN_UNDECODABLE)
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> https://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20140817/a45b830c/attachment.html>