[C++-sig] Boost.Python py3k support mid-term status report

[Haoyu Bai]

Hi,

There is a list of milestones we have reached for the GSoC py3k support project:

1. Made Boost.Python compiled successfully with Python 3
2. Almost all the test cases passed, except some test cases need to be
clean up to compatible with Python 3.
3. Basic unicode support by using Python API.
4. The bjam based BPL test suite runner will automatically call 2to3
to convert the source code of test cases to Python 3 code when needed.

I also have found some Python bugs when working on this project
(mainly 2to3 and document), and submitted reports to Python issue
tracker, and they have been fixed quickly. I also gained basic
knowledge about Bjam, so the next hacking on Boost.Build would be
easier.

I'm currently working on the test suite clean up. doctests are heavily
used in Boost.Python test suite, and 2to3 is not clever enough to deal
with all of that. (For example, the output part of doctest will be
untouched by 2to3, so if the expected output is "42L", then it will
reamain 42L after 2to3.)

The next step will be focus on the building system. The concept of
Bjam is complex and much different from the build systems I'm familiar
with. So I'd expect to put more effort on it.

Now we only have basic unicode support. The PyUnicode and C++ string
conversion is simply done by the PyUnicode_AsUTF8String and
PyUnicode_FromStringAndSize API. Converting Unicode to const char* is
a bit more complex - it is done by an API _PyUnicode_AsString, which
is an API marked only for Python internal use. When asked why this API
is not public, Benjamin Peterson answered "because implicitly encoding
unicode is bad practice in Python 3" [1]. However, use this API is
almost the only way to keep the Boost.Python PyUniocde -> const char*
converting semantic unchanged. I'll look into the Boost Unicode
project to see if we can make use of this library.

Stefan suggested me to look into the Py_Finalize issue. I did a little
experiments but didn't see the problem occurred. Can anyone help to
provide an example that produce the issue or point out details about
this issue? Thanks!

The boost SVN branch of this projec is located at the following address:

https://svn.boost.org/svn/boost/sandbox-branches/bhy/py3k/

If you are interesting, please checkout it, review it and  try it.
I'll happily provide help if you want to migrate your extension module
to py3k.

Thank you!

Reference:
[1] http://bugs.python.org/issue6223

-- Haoyu Bai