[Numpy-svn] [numpy/numpy] a9d58a: FIX: Fixes the PyUnicodeObject problem in py-3.3

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/numpy/numpy
  Commit: a9d58ab42da8d2ed9071044848a54c5e066b557a
      https://github.com/numpy/numpy/commit/a9d58ab42da8d2ed9071044848a54c5e066b557a
  Author: Ondrej Certik <ondrej.certik at gmail.com>
  Date:   2012-08-03 (Fri, 03 Aug 2012)

  Changed paths:
    M numpy/core/src/multiarray/scalarapi.c
    M numpy/core/src/multiarray/scalartypes.c.src

  Log Message:
  -----------
  FIX: Fixes the PyUnicodeObject problem in py-3.3

Previously NumPy did not compile in Python 3.3 due to the changes in PEP 393.
This patch simply calls PyUnicode_FromKindAndData() from the new Python 3.3 API
and possibly swaps the data before calling it if needed. The data in NumPy is always UCS4 and the PyUnicode_FromKindAndData() internally converts it to UCS1, UCS2 or UCS4 depending on the maximum code point.

The following tests now fail, because they produce invalid unicode in the
process (will be fixed in the next patch):

======================================================================
ERROR: Check byteorder of 0-dimensional objects
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ondrej/py33/lib/python3.3/site-packages/numpy/core/tests/test_unicode.py", line 277, in test_values0D
    self.assertTrue(ua[()] != ua2[()])
SystemError: invalid maximum character passed to PyUnicode_New

======================================================================
ERROR: Check byteorder of multi-dimensional objects
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ondrej/py33/lib/python3.3/site-packages/numpy/core/tests/test_unicode.py", line 297, in test_valuesMD
    self.assertTrue(ua[0,0,0] != ua2[0,0,0])
SystemError: invalid maximum character passed to PyUnicode_New

======================================================================
ERROR: Check byteorder of single-dimensional objects
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ondrej/py33/lib/python3.3/site-packages/numpy/core/tests/test_unicode.py", line 286, in test_valuesSD
    self.assertTrue(ua[0] != ua2[0])
SystemError: invalid maximum character passed to PyUnicode_New

======================================================================
ERROR: Check byteorder of 0-dimensional objects
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ondrej/py33/lib/python3.3/site-packages/numpy/core/tests/test_unicode.py", line 277, in test_values0D
    self.assertTrue(ua[()] != ua2[()])
SystemError: invalid maximum character passed to PyUnicode_New

======================================================================
ERROR: Check byteorder of multi-dimensional objects
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ondrej/py33/lib/python3.3/site-packages/numpy/core/tests/test_unicode.py", line 297, in test_valuesMD
    self.assertTrue(ua[0,0,0] != ua2[0,0,0])
SystemError: invalid maximum character passed to PyUnicode_New

======================================================================
ERROR: Check byteorder of single-dimensional objects
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/ondrej/py33/lib/python3.3/site-packages/numpy/core/tests/test_unicode.py", line 286, in test_valuesSD
    self.assertTrue(ua[0] != ua2[0])
SystemError: invalid maximum character passed to PyUnicode_New


  Commit: 4234b6b13e3ee9da6fc1c24e9e8c442d77587837
      https://github.com/numpy/numpy/commit/4234b6b13e3ee9da6fc1c24e9e8c442d77587837
  Author: Ondrej Certik <ondrej.certik at gmail.com>
  Date:   2012-08-03 (Fri, 03 Aug 2012)

  Changed paths:
    M numpy/core/tests/test_unicode.py

  Log Message:
  -----------
  FIX: Make sure the tests produce valid unicode

The tests are testing byte order for unicode, so we can only use such unicode
data, so that both versions (swapped and unswapped) are valid unicode.


  Commit: 09d2c51fa1d09b17060a8545b925f4dded9dedb1
      https://github.com/numpy/numpy/commit/09d2c51fa1d09b17060a8545b925f4dded9dedb1
  Author: Ondrej Certik <ondrej.certik at gmail.com>
  Date:   2012-08-03 (Fri, 03 Aug 2012)

  Changed paths:
    M numpy/core/src/multiarray/scalarapi.c

  Log Message:
  -----------
  Follow the C guidelines


  Commit: f2ac38f09ff258339ef44572a3abba02019e1f55
      https://github.com/numpy/numpy/commit/f2ac38f09ff258339ef44572a3abba02019e1f55
  Author: Ondrej Certik <ondrej.certik at gmail.com>
  Date:   2012-08-03 (Fri, 03 Aug 2012)

  Changed paths:
    M numpy/core/src/multiarray/scalarapi.c

  Log Message:
  -----------
  Use PyUnicode_DecodeUTF32()

This function handles the swapping automatically and it returns a unicode
object in one of: UCS1, UCS2 or UCS4 internal Python format.


  Commit: 4676f33f9c77b04e9c599e642de7ab465f48ea8f
      https://github.com/numpy/numpy/commit/4676f33f9c77b04e9c599e642de7ab465f48ea8f
  Author: Travis E. Oliphant <teoliphant at gmail.com>
  Date:   2012-08-03 (Fri, 03 Aug 2012)

  Changed paths:
    M numpy/core/src/multiarray/scalarapi.c
    M numpy/core/src/multiarray/scalartypes.c.src
    M numpy/core/tests/test_unicode.py

  Log Message:
  -----------
  Merge pull request #372 from certik/py3.3-cleaner

A fix for the €˜PyUnicodeObject Python 3.3


Compare: https://github.com/numpy/numpy/compare/fd15162fbff5...4676f33f9c77