From jython-checkins at python.org  Wed May  7 19:44:45 2014
From: jython-checkins at python.org (jim.baker)
Date: Wed,  7 May 2014 19:44:45 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Bundle_Apache_commons_compr?=
 =?utf-8?q?ession_support_jar_with_jar_jar_links?=
Message-ID: <3gP4v137Wfz7Ljn@mail.python.org>

http://hg.python.org/jython/rev/5d9ada3c7a62
changeset:   7221:5d9ada3c7a62
user:        Jim Baker <jim.baker at rackspace.com>
date:        Wed May 07 11:45:01 2014 -0600
summary:
  Bundle Apache commons compression support jar with jar jar links

files:
  build.xml |  3 ++-
  1 files changed, 2 insertions(+), 1 deletions(-)


diff --git a/build.xml b/build.xml
--- a/build.xml
+++ b/build.xml
@@ -148,7 +148,6 @@
             <pathelement path="${extlibs.dir}/antlr-3.1.3.jar" />
             <pathelement path="${extlibs.dir}/stringtemplate-3.2.1.jar" />
             <pathelement path="${extlibs.dir}/livetribe-jsr223-2.0.6.jar" />
-
             <pathelement path="${extlibs.dir}/commons-compress-1.4.1.jar"/>
             <pathelement path="${extlibs.dir}/asm-4.0.jar" />
             <pathelement path="${extlibs.dir}/asm-commons-4.0.jar" />
@@ -574,6 +573,8 @@
             <zipfileset src="extlibs/asm-commons-4.0.jar"/>
             <zipfileset src="extlibs/asm-util-4.0.jar"/>
             <rule pattern="org.objectweb.asm.**" result="org.python.objectweb.asm. at 1"/>
+	    <zipfileset src="extlibs/commons-compress-1.4.1.jar"/>
+	    <rule pattern="org.apache.**" result="org.python.apache. at 1"/>
             <zipfileset src="extlibs/guava-13.0.1.jar"/>
             <rule pattern="com.google.**" result="org.python.google. at 1"/>
             <zipfileset src="extlibs/icu4j-52_1.jar"/>

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Thu May  8 06:56:30 2014
From: jython-checkins at python.org (jim.baker)
Date: Thu,  8 May 2014 06:56:30 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Combine_changesets_from_htt?=
 =?utf-8?q?ps=3A//bitbucket=2Eorg/jimbaker/jython-socket-reboot?=
Message-ID: <3gPMp61nh1z7Ljs@mail.python.org>

http://hg.python.org/jython/rev/107fe4a4c96b
changeset:   7222:107fe4a4c96b
user:        Jim Baker <jim.baker at rackspace.com>
date:        Wed May 07 22:41:56 2014 -0600
summary:
  Combine changesets from https://bitbucket.org/jimbaker/jython-socket-reboot

files:
  CPythonLib.includes                      |     4 +
  Lib/BaseHTTPServer.py                    |   614 ++
  Lib/SocketServer.py                      |  1462 +++---
  Lib/_socket.py                           |  2059 ++++++++++
  Lib/_sslcerts.py                         |   239 +
  Lib/asynchat.py                          |   295 -
  Lib/asyncore.py                          |   705 ---
  Lib/distutils/command/install.py         |    93 +-
  Lib/ftplib.py                            |  1047 -----
  Lib/logging/config.py                    |   909 ----
  Lib/netrc.py                             |   119 -
  Lib/robotparser.py                       |   222 -
  Lib/select.py                            |   274 +-
  Lib/socket.py                            |  1993 +---------
  Lib/ssl.py                               |   267 +-
  Lib/telnetlib.py                         |   663 ---
  Lib/test/test_httplib.py                 |   472 --
  Lib/test/test_httpservers.py             |   543 --
  Lib/test/test_logging.py                 |   172 +-
  Lib/test/test_mhlib.py                   |    57 +-
  Lib/test/test_select.py                  |    67 +-
  Lib/test/test_socket.py                  |   285 +-
  Lib/test/test_socket_ssl.py              |    80 -
  Lib/test/test_socketserver.py            |     3 +-
  Lib/test/test_urllib2_localnet.py        |   557 --
  Lib/uu.py                                |   211 -
  Lib/zlib.py                              |    84 +-
  build.xml                                |    14 +
  extlibs/bcpkix-jdk15on-150.jar           |   Bin 
  extlibs/bcprov-jdk15on-150.jar           |   Bin 
  extlibs/netty-buffer-4.0.18.Final.jar    |   Bin 
  extlibs/netty-codec-4.0.18.Final.jar     |   Bin 
  extlibs/netty-common-4.0.18.Final.jar    |   Bin 
  extlibs/netty-handler-4.0.18.Final.jar   |   Bin 
  extlibs/netty-transport-4.0.18.Final.jar |   Bin 
  35 files changed, 4559 insertions(+), 8951 deletions(-)


diff --git a/CPythonLib.includes b/CPythonLib.includes
--- a/CPythonLib.includes
+++ b/CPythonLib.includes
@@ -27,6 +27,7 @@
 ast.py
 atexit.py
 asynchat.py
+asyncore.py
 BaseHTTPServer.py
 base64.py
 bdb.py
@@ -101,6 +102,7 @@
 mimify.py
 multifile.py
 mutex.py
+netrc.py
 nntplib.py
 numbers.py
 nturl2path.py
@@ -149,6 +151,7 @@
 symbol.py
 sysconfig.py
 tabnanny.py
+telnetlib.py
 this.py
 textwrap.py
 tempfile.py
@@ -164,6 +167,7 @@
 UserDict.py
 UserList.py
 UserString.py
+uu.py
 uuid.py
 warnings.py
 whichdb.py
diff --git a/Lib/BaseHTTPServer.py b/Lib/BaseHTTPServer.py
new file mode 100644
--- /dev/null
+++ b/Lib/BaseHTTPServer.py
@@ -0,0 +1,614 @@
+"""HTTP server base class.
+
+Note: the class in this module doesn't implement any HTTP request; see
+SimpleHTTPServer for simple implementations of GET, HEAD and POST
+(including CGI scripts).  It does, however, optionally implement HTTP/1.1
+persistent connections, as of version 0.3.
+
+Contents:
+
+- BaseHTTPRequestHandler: HTTP request handler base class
+- test: test function
+
+XXX To do:
+
+- log requests even later (to capture byte count)
+- log user-agent header and other interesting goodies
+- send error log to separate file
+"""
+
+
+# See also:
+#
+# HTTP Working Group                                        T. Berners-Lee
+# INTERNET-DRAFT                                            R. T. Fielding
+# <draft-ietf-http-v10-spec-00.txt>                     H. Frystyk Nielsen
+# Expires September 8, 1995                                  March 8, 1995
+#
+# URL: http://www.ics.uci.edu/pub/ietf/http/draft-ietf-http-v10-spec-00.txt
+#
+# and
+#
+# Network Working Group                                      R. Fielding
+# Request for Comments: 2616                                       et al
+# Obsoletes: 2068                                              June 1999
+# Category: Standards Track
+#
+# URL: http://www.faqs.org/rfcs/rfc2616.html
+
+# Log files
+# ---------
+#
+# Here's a quote from the NCSA httpd docs about log file format.
+#
+# | The logfile format is as follows. Each line consists of:
+# |
+# | host rfc931 authuser [DD/Mon/YYYY:hh:mm:ss] "request" ddd bbbb
+# |
+# |        host: Either the DNS name or the IP number of the remote client
+# |        rfc931: Any information returned by identd for this person,
+# |                - otherwise.
+# |        authuser: If user sent a userid for authentication, the user name,
+# |                  - otherwise.
+# |        DD: Day
+# |        Mon: Month (calendar name)
+# |        YYYY: Year
+# |        hh: hour (24-hour format, the machine's timezone)
+# |        mm: minutes
+# |        ss: seconds
+# |        request: The first line of the HTTP request as sent by the client.
+# |        ddd: the status code returned by the server, - if not available.
+# |        bbbb: the total number of bytes sent,
+# |              *not including the HTTP/1.0 header*, - if not available
+# |
+# | You can determine the name of the file accessed through request.
+#
+# (Actually, the latter is only true if you know the server configuration
+# at the time the request was made!)
+
+__version__ = "0.3"
+
+__all__ = ["HTTPServer", "BaseHTTPRequestHandler"]
+
+import sys
+import time
+import socket # For gethostbyaddr()
+from warnings import filterwarnings, catch_warnings
+with catch_warnings():
+    if sys.py3kwarning:
+        filterwarnings("ignore", ".*mimetools has been removed",
+                        DeprecationWarning)
+    import mimetools
+import SocketServer
+
+# Default error message template
+DEFAULT_ERROR_MESSAGE = """\
+<head>
+<title>Error response</title>
+</head>
+<body>
+<h1>Error response</h1>
+<p>Error code %(code)d.
+<p>Message: %(message)s.
+<p>Error code explanation: %(code)s = %(explain)s.
+</body>
+"""
+
+DEFAULT_ERROR_CONTENT_TYPE = "text/html"
+
+def _quote_html(html):
+    return html.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
+
+class HTTPServer(SocketServer.TCPServer):
+
+    allow_reuse_address = 1    # Seems to make sense in testing environment
+
+    def server_bind(self):
+        """Override server_bind to store the server name."""
+        SocketServer.TCPServer.server_bind(self)
+        try:
+            host, port = self.socket.getsockname()[:2]
+            self.server_name = socket.getfqdn(host)
+            self.server_port = port
+        except socket.error:
+            pass
+
+    def server_activate(self):
+        SocketServer.TCPServer.server_activate(self)
+        # Adding a second call to getsockname() because of this issue
+        # http://wiki.python.org/jython/NewSocketModule#Deferredsocketcreationonjython
+        host, port = self.socket.getsockname()[:2]
+        self.server_name = socket.getfqdn(host)
+        self.server_port = port
+
+
+class BaseHTTPRequestHandler(SocketServer.StreamRequestHandler):
+
+    """HTTP request handler base class.
+
+    The following explanation of HTTP serves to guide you through the
+    code as well as to expose any misunderstandings I may have about
+    HTTP (so you don't need to read the code to figure out I'm wrong
+    :-).
+
+    HTTP (HyperText Transfer Protocol) is an extensible protocol on
+    top of a reliable stream transport (e.g. TCP/IP).  The protocol
+    recognizes three parts to a request:
+
+    1. One line identifying the request type and path
+    2. An optional set of RFC-822-style headers
+    3. An optional data part
+
+    The headers and data are separated by a blank line.
+
+    The first line of the request has the form
+
+    <command> <path> <version>
+
+    where <command> is a (case-sensitive) keyword such as GET or POST,
+    <path> is a string containing path information for the request,
+    and <version> should be the string "HTTP/1.0" or "HTTP/1.1".
+    <path> is encoded using the URL encoding scheme (using %xx to signify
+    the ASCII character with hex code xx).
+
+    The specification specifies that lines are separated by CRLF but
+    for compatibility with the widest range of clients recommends
+    servers also handle LF.  Similarly, whitespace in the request line
+    is treated sensibly (allowing multiple spaces between components
+    and allowing trailing whitespace).
+
+    Similarly, for output, lines ought to be separated by CRLF pairs
+    but most clients grok LF characters just fine.
+
+    If the first line of the request has the form
+
+    <command> <path>
+
+    (i.e. <version> is left out) then this is assumed to be an HTTP
+    0.9 request; this form has no optional headers and data part and
+    the reply consists of just the data.
+
+    The reply form of the HTTP 1.x protocol again has three parts:
+
+    1. One line giving the response code
+    2. An optional set of RFC-822-style headers
+    3. The data
+
+    Again, the headers and data are separated by a blank line.
+
+    The response code line has the form
+
+    <version> <responsecode> <responsestring>
+
+    where <version> is the protocol version ("HTTP/1.0" or "HTTP/1.1"),
+    <responsecode> is a 3-digit response code indicating success or
+    failure of the request, and <responsestring> is an optional
+    human-readable string explaining what the response code means.
+
+    This server parses the request and the headers, and then calls a
+    function specific to the request type (<command>).  Specifically,
+    a request SPAM will be handled by a method do_SPAM().  If no
+    such method exists the server sends an error response to the
+    client.  If it exists, it is called with no arguments:
+
+    do_SPAM()
+
+    Note that the request name is case sensitive (i.e. SPAM and spam
+    are different requests).
+
+    The various request details are stored in instance variables:
+
+    - client_address is the client IP address in the form (host,
+    port);
+
+    - command, path and version are the broken-down request line;
+
+    - headers is an instance of mimetools.Message (or a derived
+    class) containing the header information;
+
+    - rfile is a file object open for reading positioned at the
+    start of the optional input data part;
+
+    - wfile is a file object open for writing.
+
+    IT IS IMPORTANT TO ADHERE TO THE PROTOCOL FOR WRITING!
+
+    The first thing to be written must be the response line.  Then
+    follow 0 or more header lines, then a blank line, and then the
+    actual data (if any).  The meaning of the header lines depends on
+    the command executed by the server; in most cases, when data is
+    returned, there should be at least one header line of the form
+
+    Content-type: <type>/<subtype>
+
+    where <type> and <subtype> should be registered MIME types,
+    e.g. "text/html" or "text/plain".
+
+    """
+
+    # The Python system version, truncated to its first component.
+    sys_version = "Python/" + sys.version.split()[0]
+
+    # The server software version.  You may want to override this.
+    # The format is multiple whitespace-separated strings,
+    # where each string is of the form name[/version].
+    server_version = "BaseHTTP/" + __version__
+
+    # The default request version.  This only affects responses up until
+    # the point where the request line is parsed, so it mainly decides what
+    # the client gets back when sending a malformed request line.
+    # Most web servers default to HTTP 0.9, i.e. don't send a status line.
+    default_request_version = "HTTP/0.9"
+
+    def parse_request(self):
+        """Parse a request (internal).
+
+        The request should be stored in self.raw_requestline; the results
+        are in self.command, self.path, self.request_version and
+        self.headers.
+
+        Return True for success, False for failure; on failure, an
+        error is sent back.
+
+        """
+        self.command = None  # set in case of error on the first line
+        self.request_version = version = self.default_request_version
+        self.close_connection = 1
+        requestline = self.raw_requestline
+        requestline = requestline.rstrip('\r\n')
+        self.requestline = requestline
+        words = requestline.split()
+        if len(words) == 3:
+            command, path, version = words
+            if version[:5] != 'HTTP/':
+                self.send_error(400, "Bad request version (%r)" % version)
+                return False
+            try:
+                base_version_number = version.split('/', 1)[1]
+                version_number = base_version_number.split(".")
+                # RFC 2145 section 3.1 says there can be only one "." and
+                #   - major and minor numbers MUST be treated as
+                #      separate integers;
+                #   - HTTP/2.4 is a lower version than HTTP/2.13, which in
+                #      turn is lower than HTTP/12.3;
+                #   - Leading zeros MUST be ignored by recipients.
+                if len(version_number) != 2:
+                    raise ValueError
+                version_number = int(version_number[0]), int(version_number[1])
+            except (ValueError, IndexError):
+                self.send_error(400, "Bad request version (%r)" % version)
+                return False
+            if version_number >= (1, 1) and self.protocol_version >= "HTTP/1.1":
+                self.close_connection = 0
+            if version_number >= (2, 0):
+                self.send_error(505,
+                          "Invalid HTTP Version (%s)" % base_version_number)
+                return False
+        elif len(words) == 2:
+            command, path = words
+            self.close_connection = 1
+            if command != 'GET':
+                self.send_error(400,
+                                "Bad HTTP/0.9 request type (%r)" % command)
+                return False
+        elif not words:
+            return False
+        else:
+            self.send_error(400, "Bad request syntax (%r)" % requestline)
+            return False
+        self.command, self.path, self.request_version = command, path, version
+
+        # Examine the headers and look for a Connection directive
+        self.headers = self.MessageClass(self.rfile, 0)
+
+        conntype = self.headers.get('Connection', "")
+        if conntype.lower() == 'close':
+            self.close_connection = 1
+        elif (conntype.lower() == 'keep-alive' and
+              self.protocol_version >= "HTTP/1.1"):
+            self.close_connection = 0
+        return True
+
+    def handle_one_request(self):
+        """Handle a single HTTP request.
+
+        You normally don't need to override this method; see the class
+        __doc__ string for information on how to handle specific HTTP
+        commands such as GET and POST.
+
+        """
+        try:
+            self.raw_requestline = self.rfile.readline(65537)
+            if len(self.raw_requestline) > 65536:
+                self.requestline = ''
+                self.request_version = ''
+                self.command = ''
+                self.send_error(414)
+                return
+            if not self.raw_requestline:
+                self.close_connection = 1
+                return
+            if not self.parse_request():
+                # An error code has been sent, just exit
+                return
+            mname = 'do_' + self.command
+            if not hasattr(self, mname):
+                self.send_error(501, "Unsupported method (%r)" % self.command)
+                return
+            method = getattr(self, mname)
+            method()
+            self.wfile.flush() #actually send the response if not already done.
+        except socket.timeout, e:
+            #a read or a write timed out.  Discard this connection
+            self.log_error("Request timed out: %r", e)
+            self.close_connection = 1
+            return
+
+    def handle(self):
+        """Handle multiple requests if necessary."""
+        self.close_connection = 1
+
+        self.handle_one_request()
+        while not self.close_connection:
+            self.handle_one_request()
+
+    def send_error(self, code, message=None):
+        """Send and log an error reply.
+
+        Arguments are the error code, and a detailed message.
+        The detailed message defaults to the short entry matching the
+        response code.
+
+        This sends an error response (so it must be called before any
+        output has been generated), logs the error, and finally sends
+        a piece of HTML explaining the error to the user.
+
+        """
+
+        try:
+            short, long = self.responses[code]
+        except KeyError:
+            short, long = '???', '???'
+        if message is None:
+            message = short
+        explain = long
+        self.log_error("code %d, message %s", code, message)
+        # using _quote_html to prevent Cross Site Scripting attacks (see bug #1100201)
+        content = (self.error_message_format %
+                   {'code': code, 'message': _quote_html(message), 'explain': explain})
+        self.send_response(code, message)
+        self.send_header("Content-Type", self.error_content_type)
+        self.send_header('Connection', 'close')
+        self.end_headers()
+        if self.command != 'HEAD' and code >= 200 and code not in (204, 304):
+            self.wfile.write(content)
+
+    error_message_format = DEFAULT_ERROR_MESSAGE
+    error_content_type = DEFAULT_ERROR_CONTENT_TYPE
+
+    def send_response(self, code, message=None):
+        """Send the response header and log the response code.
+
+        Also send two standard headers with the server software
+        version and the current date.
+
+        """
+        self.log_request(code)
+        if message is None:
+            if code in self.responses:
+                message = self.responses[code][0]
+            else:
+                message = ''
+        if self.request_version != 'HTTP/0.9':
+            self.wfile.write("%s %d %s\r\n" %
+                             (self.protocol_version, code, message))
+            # print (self.protocol_version, code, message)
+        self.send_header('Server', self.version_string())
+        self.send_header('Date', self.date_time_string())
+
+    def send_header(self, keyword, value):
+        """Send a MIME header."""
+        if self.request_version != 'HTTP/0.9':
+            self.wfile.write("%s: %s\r\n" % (keyword, value))
+
+        if keyword.lower() == 'connection':
+            if value.lower() == 'close':
+                self.close_connection = 1
+            elif value.lower() == 'keep-alive':
+                self.close_connection = 0
+
+    def end_headers(self):
+        """Send the blank line ending the MIME headers."""
+        if self.request_version != 'HTTP/0.9':
+            self.wfile.write("\r\n")
+
+    def log_request(self, code='-', size='-'):
+        """Log an accepted request.
+
+        This is called by send_response().
+
+        """
+
+        self.log_message('"%s" %s %s',
+                         self.requestline, str(code), str(size))
+
+    def log_error(self, format, *args):
+        """Log an error.
+
+        This is called when a request cannot be fulfilled.  By
+        default it passes the message on to log_message().
+
+        Arguments are the same as for log_message().
+
+        XXX This should go to the separate error log.
+
+        """
+
+        self.log_message(format, *args)
+
+    def log_message(self, format, *args):
+        """Log an arbitrary message.
+
+        This is used by all other logging functions.  Override
+        it if you have specific logging wishes.
+
+        The first argument, FORMAT, is a format string for the
+        message to be logged.  If the format string contains
+        any % escapes requiring parameters, they should be
+        specified as subsequent arguments (it's just like
+        printf!).
+
+        The client ip address and current date/time are prefixed to every
+        message.
+
+        """
+
+        sys.stderr.write("%s - - [%s] %s\n" %
+                         (self.client_address[0],
+                          self.log_date_time_string(),
+                          format%args))
+
+    def version_string(self):
+        """Return the server software version string."""
+        return self.server_version + ' ' + self.sys_version
+
+    def date_time_string(self, timestamp=None):
+        """Return the current date and time formatted for a message header."""
+        if timestamp is None:
+            timestamp = time.time()
+        year, month, day, hh, mm, ss, wd, y, z = time.gmtime(timestamp)
+        s = "%s, %02d %3s %4d %02d:%02d:%02d GMT" % (
+                self.weekdayname[wd],
+                day, self.monthname[month], year,
+                hh, mm, ss)
+        return s
+
+    def log_date_time_string(self):
+        """Return the current time formatted for logging."""
+        now = time.time()
+        year, month, day, hh, mm, ss, x, y, z = time.localtime(now)
+        s = "%02d/%3s/%04d %02d:%02d:%02d" % (
+                day, self.monthname[month], year, hh, mm, ss)
+        return s
+
+    weekdayname = ['Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat', 'Sun']
+
+    monthname = [None,
+                 'Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun',
+                 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec']
+
+    def address_string(self):
+        """Return the client address formatted for logging.
+
+        This version looks up the full hostname using gethostbyaddr(),
+        and tries to find a name that contains at least one dot.
+
+        """
+
+        host, port = self.client_address[:2]
+        return socket.getfqdn(host)
+
+    # Essentially static class variables
+
+    # The version of the HTTP protocol we support.
+    # Set this to HTTP/1.1 to enable automatic keepalive
+    protocol_version = "HTTP/1.0"
+
+    # The Message-like class used to parse headers
+    MessageClass = mimetools.Message
+
+    # Table mapping response codes to messages; entries have the
+    # form {code: (shortmessage, longmessage)}.
+    # See RFC 2616.
+    responses = {
+        100: ('Continue', 'Request received, please continue'),
+        101: ('Switching Protocols',
+              'Switching to new protocol; obey Upgrade header'),
+
+        200: ('OK', 'Request fulfilled, document follows'),
+        201: ('Created', 'Document created, URL follows'),
+        202: ('Accepted',
+              'Request accepted, processing continues off-line'),
+        203: ('Non-Authoritative Information', 'Request fulfilled from cache'),
+        204: ('No Content', 'Request fulfilled, nothing follows'),
+        205: ('Reset Content', 'Clear input form for further input.'),
+        206: ('Partial Content', 'Partial content follows.'),
+
+        300: ('Multiple Choices',
+              'Object has several resources -- see URI list'),
+        301: ('Moved Permanently', 'Object moved permanently -- see URI list'),
+        302: ('Found', 'Object moved temporarily -- see URI list'),
+        303: ('See Other', 'Object moved -- see Method and URL list'),
+        304: ('Not Modified',
+              'Document has not changed since given time'),
+        305: ('Use Proxy',
+              'You must use proxy specified in Location to access this '
+              'resource.'),
+        307: ('Temporary Redirect',
+              'Object moved temporarily -- see URI list'),
+
+        400: ('Bad Request',
+              'Bad request syntax or unsupported method'),
+        401: ('Unauthorized',
+              'No permission -- see authorization schemes'),
+        402: ('Payment Required',
+              'No payment -- see charging schemes'),
+        403: ('Forbidden',
+              'Request forbidden -- authorization will not help'),
+        404: ('Not Found', 'Nothing matches the given URI'),
+        405: ('Method Not Allowed',
+              'Specified method is invalid for this resource.'),
+        406: ('Not Acceptable', 'URI not available in preferred format.'),
+        407: ('Proxy Authentication Required', 'You must authenticate with '
+              'this proxy before proceeding.'),
+        408: ('Request Timeout', 'Request timed out; try again later.'),
+        409: ('Conflict', 'Request conflict.'),
+        410: ('Gone',
+              'URI no longer exists and has been permanently removed.'),
+        411: ('Length Required', 'Client must specify Content-Length.'),
+        412: ('Precondition Failed', 'Precondition in headers is false.'),
+        413: ('Request Entity Too Large', 'Entity is too large.'),
+        414: ('Request-URI Too Long', 'URI is too long.'),
+        415: ('Unsupported Media Type', 'Entity body in unsupported format.'),
+        416: ('Requested Range Not Satisfiable',
+              'Cannot satisfy request range.'),
+        417: ('Expectation Failed',
+              'Expect condition could not be satisfied.'),
+
+        500: ('Internal Server Error', 'Server got itself in trouble'),
+        501: ('Not Implemented',
+              'Server does not support this operation'),
+        502: ('Bad Gateway', 'Invalid responses from another server/proxy.'),
+        503: ('Service Unavailable',
+              'The server cannot process the request due to a high load'),
+        504: ('Gateway Timeout',
+              'The gateway server did not receive a timely response'),
+        505: ('HTTP Version Not Supported', 'Cannot fulfill request.'),
+        }
+
+
+def test(HandlerClass = BaseHTTPRequestHandler,
+         ServerClass = HTTPServer, protocol="HTTP/1.0"):
+    """Test the HTTP request handler class.
+
+    This runs an HTTP server on port 8000 (or the first command line
+    argument).
+
+    """
+
+    if sys.argv[1:]:
+        port = int(sys.argv[1])
+    else:
+        port = 8000
+    server_address = ('', port)
+
+    HandlerClass.protocol_version = protocol
+    httpd = ServerClass(server_address, HandlerClass)
+
+    sa = httpd.socket.getsockname()
+    print "Serving HTTP on", sa[0], "port", sa[1], "..."
+    httpd.serve_forever()
+
+
+if __name__ == '__main__':
+    test()
diff --git a/Lib/SocketServer.py b/Lib/SocketServer.py
--- a/Lib/SocketServer.py
+++ b/Lib/SocketServer.py
@@ -1,723 +1,739 @@
-"""Generic socket server classes.
-
-This module tries to capture the various aspects of defining a server:
-
-For socket-based servers:
-
-- address family:
-        - AF_INET{,6}: IP (Internet Protocol) sockets (default)
-        - AF_UNIX: Unix domain sockets
-        - others, e.g. AF_DECNET are conceivable (see <socket.h>
-- socket type:
-        - SOCK_STREAM (reliable stream, e.g. TCP)
-        - SOCK_DGRAM (datagrams, e.g. UDP)
-
-For request-based servers (including socket-based):
-
-- client address verification before further looking at the request
-        (This is actually a hook for any processing that needs to look
-         at the request before anything else, e.g. logging)
-- how to handle multiple requests:
-        - synchronous (one request is handled at a time)
-        - forking (each request is handled by a new process)
-        - threading (each request is handled by a new thread)
-
-The classes in this module favor the server type that is simplest to
-write: a synchronous TCP/IP server.  This is bad class design, but
-save some typing.  (There's also the issue that a deep class hierarchy
-slows down method lookups.)
-
-There are five classes in an inheritance diagram, four of which represent
-synchronous servers of four types:
-
-        +------------+
-        | BaseServer |
-        +------------+
-              |
-              v
-        +-----------+        +------------------+
-        | TCPServer |------->| UnixStreamServer |
-        +-----------+        +------------------+
-              |
-              v
-        +-----------+        +--------------------+
-        | UDPServer |------->| UnixDatagramServer |
-        +-----------+        +--------------------+
-
-Note that UnixDatagramServer derives from UDPServer, not from
-UnixStreamServer -- the only difference between an IP and a Unix
-stream server is the address family, which is simply repeated in both
-unix server classes.
-
-Forking and threading versions of each type of server can be created
-using the ForkingMixIn and ThreadingMixIn mix-in classes.  For
-instance, a threading UDP server class is created as follows:
-
-        class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
-
-The Mix-in class must come first, since it overrides a method defined
-in UDPServer! Setting the various member variables also changes
-the behavior of the underlying server mechanism.
-
-To implement a service, you must derive a class from
-BaseRequestHandler and redefine its handle() method.  You can then run
-various versions of the service by combining one of the server classes
-with your request handler class.
-
-The request handler class must be different for datagram or stream
-services.  This can be hidden by using the request handler
-subclasses StreamRequestHandler or DatagramRequestHandler.
-
-Of course, you still have to use your head!
-
-For instance, it makes no sense to use a forking server if the service
-contains state in memory that can be modified by requests (since the
-modifications in the child process would never reach the initial state
-kept in the parent process and passed to each child).  In this case,
-you can use a threading server, but you will probably have to use
-locks to avoid two requests that come in nearly simultaneous to apply
-conflicting changes to the server state.
-
-On the other hand, if you are building e.g. an HTTP server, where all
-data is stored externally (e.g. in the file system), a synchronous
-class will essentially render the service "deaf" while one request is
-being handled -- which may be for a very long time if a client is slow
-to reqd all the data it has requested.  Here a threading or forking
-server is appropriate.
-
-In some cases, it may be appropriate to process part of a request
-synchronously, but to finish processing in a forked child depending on
-the request data.  This can be implemented by using a synchronous
-server and doing an explicit fork in the request handler class
-handle() method.
-
-Another approach to handling multiple simultaneous requests in an
-environment that supports neither threads nor fork (or where these are
-too expensive or inappropriate for the service) is to maintain an
-explicit table of partially finished requests and to use select() to
-decide which request to work on next (or whether to handle a new
-incoming request).  This is particularly important for stream services
-where each client can potentially be connected for a long time (if
-threads or subprocesses cannot be used).
-
-Future work:
-- Standard classes for Sun RPC (which uses either UDP or TCP)
-- Standard mix-in classes to implement various authentication
-  and encryption schemes
-- Standard framework for select-based multiplexing
-
-XXX Open problems:
-- What to do with out-of-band data?
-
-BaseServer:
-- split generic "request" functionality out into BaseServer class.
-  Copyright (C) 2000  Luke Kenneth Casson Leighton <lkcl at samba.org>
-
-  example: read entries from a SQL database (requires overriding
-  get_request() to return a table entry from the database).
-  entry is processed by a RequestHandlerClass.
-
-"""
-
-# Author of the BaseServer patch: Luke Kenneth Casson Leighton
-
-# XXX Warning!
-# There is a test suite for this module, but it cannot be run by the
-# standard regression test.
-# To run it manually, run Lib/test/test_socketserver.py.
-
-__version__ = "0.4"
-
-
-import socket
-import select
-import sys
-import os
-try:
-    import threading
-except ImportError:
-    import dummy_threading as threading
-
-select_fn = select.select
-if sys.platform.startswith('java'):
-    select_fn = select.cpython_compatible_select
-
-__all__ = ["TCPServer","UDPServer","ForkingUDPServer","ForkingTCPServer",
-           "ThreadingUDPServer","ThreadingTCPServer","BaseRequestHandler",
-           "StreamRequestHandler","DatagramRequestHandler",
-           "ThreadingMixIn", "ForkingMixIn"]
-if hasattr(socket, "AF_UNIX"):
-    __all__.extend(["UnixStreamServer","UnixDatagramServer",
-                    "ThreadingUnixStreamServer",
-                    "ThreadingUnixDatagramServer"])
-
-class BaseServer:
-
-    """Base class for server classes.
-
-    Methods for the caller:
-
-    - __init__(server_address, RequestHandlerClass)
-    - serve_forever(poll_interval=0.5)
-    - shutdown()
-    - handle_request()  # if you do not use serve_forever()
-    - fileno() -> int   # for select()
-
-    Methods that may be overridden:
-
-    - server_bind()
-    - server_activate()
-    - get_request() -> request, client_address
-    - handle_timeout()
-    - verify_request(request, client_address)
-    - server_close()
-    - process_request(request, client_address)
-    - shutdown_request(request)
-    - close_request(request)
-    - handle_error()
-
-    Methods for derived classes:
-
-    - finish_request(request, client_address)
-
-    Class variables that may be overridden by derived classes or
-    instances:
-
-    - timeout
-    - address_family
-    - socket_type
-    - allow_reuse_address
-
-    Instance variables:
-
-    - RequestHandlerClass
-    - socket
-
-    """
-
-    timeout = None
-
-    def __init__(self, server_address, RequestHandlerClass):
-        """Constructor.  May be extended, do not override."""
-        self.server_address = server_address
-        self.RequestHandlerClass = RequestHandlerClass
-        self.__is_shut_down = threading.Event()
-        self.__shutdown_request = False
-
-    def server_activate(self):
-        """Called by constructor to activate the server.
-
-        May be overridden.
-
-        """
-        pass
-
-    def serve_forever(self, poll_interval=0.5):
-        """Handle one request at a time until shutdown.
-
-        Polls for shutdown every poll_interval seconds. Ignores
-        self.timeout. If you need to do periodic tasks, do them in
-        another thread.
-        """
-        self.__is_shut_down.clear()
-        try:
-            while not self.__shutdown_request:
-                # XXX: Consider using another file descriptor or
-                # connecting to the socket to wake this up instead of
-                # polling. Polling reduces our responsiveness to a
-                # shutdown request and wastes cpu at all other times.
-                r, w, e = select_fn([self], [], [], poll_interval)
-                if self in r:
-                    self._handle_request_noblock()
-        finally:
-            self.__shutdown_request = False
-            self.__is_shut_down.set()
-
-    def shutdown(self):
-        """Stops the serve_forever loop.
-
-        Blocks until the loop has finished. This must be called while
-        serve_forever() is running in another thread, or it will
-        deadlock.
-        """
-        self.__shutdown_request = True
-        self.__is_shut_down.wait()
-
-    # The distinction between handling, getting, processing and
-    # finishing a request is fairly arbitrary.  Remember:
-    #
-    # - handle_request() is the top-level call.  It calls
-    #   select, get_request(), verify_request() and process_request()
-    # - get_request() is different for stream or datagram sockets
-    # - process_request() is the place that may fork a new process
-    #   or create a new thread to finish the request
-    # - finish_request() instantiates the request handler class;
-    #   this constructor will handle the request all by itself
-
-    def handle_request(self):
-        """Handle one request, possibly blocking.
-
-        Respects self.timeout.
-        """
-        # Support people who used socket.settimeout() to escape
-        # handle_request before self.timeout was available.
-        timeout = self.socket.gettimeout()
-        if timeout is None:
-            timeout = self.timeout
-        elif self.timeout is not None:
-            timeout = min(timeout, self.timeout)
-        fd_sets = select_fn([self], [], [], timeout)
-        if not fd_sets[0]:
-            self.handle_timeout()
-            return
-        self._handle_request_noblock()
-
-    def _handle_request_noblock(self):
-        """Handle one request, without blocking.
-
-        I assume that select_fn has returned that the socket is
-        readable before this function was called, so there should be
-        no risk of blocking in get_request().
-        """
-        try:
-            request, client_address = self.get_request()
-        except socket.error:
-            return
-        if self.verify_request(request, client_address):
-            try:
-                self.process_request(request, client_address)
-            except:
-                self.handle_error(request, client_address)
-                self.shutdown_request(request)
-
-    def handle_timeout(self):
-        """Called if no new request arrives within self.timeout.
-
-        Overridden by ForkingMixIn.
-        """
-        pass
-
-    def verify_request(self, request, client_address):
-        """Verify the request.  May be overridden.
-
-        Return True if we should proceed with this request.
-
-        """
-        return True
-
-    def process_request(self, request, client_address):
-        """Call finish_request.
-
-        Overridden by ForkingMixIn and ThreadingMixIn.
-
-        """
-        self.finish_request(request, client_address)
-        self.shutdown_request(request)
-
-    def server_close(self):
-        """Called to clean-up the server.
-
-        May be overridden.
-
-        """
-        pass
-
-    def finish_request(self, request, client_address):
-        """Finish one request by instantiating RequestHandlerClass."""
-        self.RequestHandlerClass(request, client_address, self)
-
-    def shutdown_request(self, request):
-        """Called to shutdown and close an individual request."""
-        self.close_request(request)
-
-    def close_request(self, request):
-        """Called to clean up an individual request."""
-        pass
-
-    def handle_error(self, request, client_address):
-        """Handle an error gracefully.  May be overridden.
-
-        The default is to print a traceback and continue.
-
-        """
-        print '-'*40
-        print 'Exception happened during processing of request from',
-        print client_address
-        import traceback
-        traceback.print_exc() # XXX But this goes to stderr!
-        print '-'*40
-
-
-class TCPServer(BaseServer):
-
-    """Base class for various socket-based server classes.
-
-    Defaults to synchronous IP stream (i.e., TCP).
-
-    Methods for the caller:
-
-    - __init__(server_address, RequestHandlerClass, bind_and_activate=True)
-    - serve_forever(poll_interval=0.5)
-    - shutdown()
-    - handle_request()  # if you don't use serve_forever()
-    - fileno() -> int   # for select()
-
-    Methods that may be overridden:
-
-    - server_bind()
-    - server_activate()
-    - get_request() -> request, client_address
-    - handle_timeout()
-    - verify_request(request, client_address)
-    - process_request(request, client_address)
-    - shutdown_request(request)
-    - close_request(request)
-    - handle_error()
-
-    Methods for derived classes:
-
-    - finish_request(request, client_address)
-
-    Class variables that may be overridden by derived classes or
-    instances:
-
-    - timeout
-    - address_family
-    - socket_type
-    - request_queue_size (only for stream sockets)
-    - allow_reuse_address
-
-    Instance variables:
-
-    - server_address
-    - RequestHandlerClass
-    - socket
-
-    """
-
-    address_family = socket.AF_INET
-
-    socket_type = socket.SOCK_STREAM
-
-    request_queue_size = 5
-
-    allow_reuse_address = False
-
-    def __init__(self, server_address, RequestHandlerClass, bind_and_activate=True):
-        """Constructor.  May be extended, do not override."""
-        BaseServer.__init__(self, server_address, RequestHandlerClass)
-        self.socket = socket.socket(self.address_family,
-                                    self.socket_type)
-        if bind_and_activate:
-            self.server_bind()
-            self.server_activate()
-
-    def server_bind(self):
-        """Called by constructor to bind the socket.
-
-        May be overridden.
-
-        """
-        if self.allow_reuse_address:
-            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-        self.socket.bind(self.server_address)
-        self.server_address = self.socket.getsockname()
-
-    def server_activate(self):
-        """Called by constructor to activate the server.
-
-        May be overridden.
-
-        """
-        self.socket.listen(self.request_queue_size)
-        # Adding a second call to getsockname() because of this issue
-        # http://wiki.python.org/jython/NewSocketModule#Deferredsocketcreationonjython
-        self.server_address = self.socket.getsockname()
-
-    def server_close(self):
-        """Called to clean-up the server.
-
-        May be overridden.
-
-        """
-        self.socket.close()
-
-    def fileno(self):
-        """Return socket file number.
-
-        Interface required by select().
-
-        """
-        return self.socket.fileno()
-
-    def get_request(self):
-        """Get the request and client address from the socket.
-
-        May be overridden.
-
-        """
-        return self.socket.accept()
-
-    def shutdown_request(self, request):
-        """Called to shutdown and close an individual request."""
-        try:
-            #explicitly shutdown.  socket.close() merely releases
-            #the socket and waits for GC to perform the actual close.
-            request.shutdown(socket.SHUT_WR)
-        except socket.error:
-            pass #some platforms may raise ENOTCONN here
-        self.close_request(request)
-
-    def close_request(self, request):
-        """Called to clean up an individual request."""
-        request.close()
-
-
-class UDPServer(TCPServer):
-
-    """UDP server class."""
-
-    allow_reuse_address = False
-
-    socket_type = socket.SOCK_DGRAM
-
-    max_packet_size = 8192
-
-    def get_request(self):
-        data, client_addr = self.socket.recvfrom(self.max_packet_size)
-        return (data, self.socket), client_addr
-
-    def server_activate(self):
-        # No need to call listen() for UDP.
-        pass
-
-    def shutdown_request(self, request):
-        # No need to shutdown anything.
-        self.close_request(request)
-
-    def close_request(self, request):
-        # No need to close anything.
-        pass
-
-class ForkingMixIn:
-
-    """Mix-in class to handle each request in a new process."""
-
-    timeout = 300
-    active_children = None
-    max_children = 40
-
-    def collect_children(self):
-        """Internal routine to wait for children that have exited."""
-        if self.active_children is None: return
-        while len(self.active_children) >= self.max_children:
-            # XXX: This will wait for any child process, not just ones
-            # spawned by this library. This could confuse other
-            # libraries that expect to be able to wait for their own
-            # children.
-            try:
-                pid, status = os.waitpid(0, 0)
-            except os.error:
-                pid = None
-            if pid not in self.active_children: continue
-            self.active_children.remove(pid)
-
-        # XXX: This loop runs more system calls than it ought
-        # to. There should be a way to put the active_children into a
-        # process group and then use os.waitpid(-pgid) to wait for any
-        # of that set, but I couldn't find a way to allocate pgids
-        # that couldn't collide.
-        for child in self.active_children:
-            try:
-                pid, status = os.waitpid(child, os.WNOHANG)
-            except os.error:
-                pid = None
-            if not pid: continue
-            try:
-                self.active_children.remove(pid)
-            except ValueError, e:
-                raise ValueError('%s. x=%d and list=%r' % (e.message, pid,
-                                                           self.active_children))
-
-    def handle_timeout(self):
-        """Wait for zombies after self.timeout seconds of inactivity.
-
-        May be extended, do not override.
-        """
-        self.collect_children()
-
-    def process_request(self, request, client_address):
-        """Fork a new subprocess to process the request."""
-        self.collect_children()
-        pid = os.fork()
-        if pid:
-            # Parent process
-            if self.active_children is None:
-                self.active_children = []
-            self.active_children.append(pid)
-            self.close_request(request) #close handle in parent process
-            return
-        else:
-            # Child process.
-            # This must never return, hence os._exit()!
-            try:
-                self.finish_request(request, client_address)
-                self.shutdown_request(request)
-                os._exit(0)
-            except:
-                try:
-                    self.handle_error(request, client_address)
-                    self.shutdown_request(request)
-                finally:
-                    os._exit(1)
-
-
-class ThreadingMixIn:
-    """Mix-in class to handle each request in a new thread."""
-
-    # Decides how threads will act upon termination of the
-    # main process
-    daemon_threads = False
-
-    def process_request_thread(self, request, client_address):
-        """Same as in BaseServer but as a thread.
-
-        In addition, exception handling is done here.
-
-        """
-        try:
-            self.finish_request(request, client_address)
-            self.shutdown_request(request)
-        except:
-            self.handle_error(request, client_address)
-            self.shutdown_request(request)
-
-    def process_request(self, request, client_address):
-        """Start a new thread to process the request."""
-        t = threading.Thread(target = self.process_request_thread,
-                             args = (request, client_address))
-        if self.daemon_threads:
-            t.setDaemon (1)
-        t.start()
-
-
-class ForkingUDPServer(ForkingMixIn, UDPServer): pass
-class ForkingTCPServer(ForkingMixIn, TCPServer): pass
-
-class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
-class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass
-
-if hasattr(socket, 'AF_UNIX'):
-
-    class UnixStreamServer(TCPServer):
-        address_family = socket.AF_UNIX
-
-    class UnixDatagramServer(UDPServer):
-        address_family = socket.AF_UNIX
-
-    class ThreadingUnixStreamServer(ThreadingMixIn, UnixStreamServer): pass
-
-    class ThreadingUnixDatagramServer(ThreadingMixIn, UnixDatagramServer): pass
-
-class BaseRequestHandler:
-
-    """Base class for request handler classes.
-
-    This class is instantiated for each request to be handled.  The
-    constructor sets the instance variables request, client_address
-    and server, and then calls the handle() method.  To implement a
-    specific service, all you need to do is to derive a class which
-    defines a handle() method.
-
-    The handle() method can find the request as self.request, the
-    client address as self.client_address, and the server (in case it
-    needs access to per-server information) as self.server.  Since a
-    separate instance is created for each request, the handle() method
-    can define arbitrary other instance variariables.
-
-    """
-
-    def __init__(self, request, client_address, server):
-        self.request = request
-        self.client_address = client_address
-        self.server = server
-        self.setup()
-        try:
-            self.handle()
-        finally:
-            self.finish()
-
-    def setup(self):
-        pass
-
-    def handle(self):
-        pass
-
-    def finish(self):
-        pass
-
-
-# The following two classes make it possible to use the same service
-# class for stream or datagram servers.
-# Each class sets up these instance variables:
-# - rfile: a file object from which receives the request is read
-# - wfile: a file object to which the reply is written
-# When the handle() method returns, wfile is flushed properly
-
-
-class StreamRequestHandler(BaseRequestHandler):
-
-    """Define self.rfile and self.wfile for stream sockets."""
-
-    # Default buffer sizes for rfile, wfile.
-    # We default rfile to buffered because otherwise it could be
-    # really slow for large data (a getc() call per byte); we make
-    # wfile unbuffered because (a) often after a write() we want to
-    # read and we need to flush the line; (b) big writes to unbuffered
-    # files are typically optimized by stdio even when big reads
-    # aren't.
-    rbufsize = -1
-    wbufsize = 0
-
-    # A timeout to apply to the request socket, if not None.
-    timeout = None
-
-    # Disable nagle algorithm for this socket, if True.
-    # Use only when wbufsize != 0, to avoid small packets.
-    disable_nagle_algorithm = False
-
-    def setup(self):
-        self.connection = self.request
-        if self.timeout is not None:
-            self.connection.settimeout(self.timeout)
-        if self.disable_nagle_algorithm:
-            self.connection.setsockopt(socket.IPPROTO_TCP,
-                                       socket.TCP_NODELAY, True)
-        self.rfile = self.connection.makefile('rb', self.rbufsize)
-        self.wfile = self.connection.makefile('wb', self.wbufsize)
-
-    def finish(self):
-        if not self.wfile.closed:
-            self.wfile.flush()
-        self.wfile.close()
-        self.rfile.close()
-
-
-class DatagramRequestHandler(BaseRequestHandler):
-
-    # XXX Regrettably, I cannot get this working on Linux;
-    # s.recvfrom() doesn't return a meaningful client address.
-
-    """Define self.rfile and self.wfile for datagram sockets."""
-
-    def setup(self):
-        try:
-            from cStringIO import StringIO
-        except ImportError:
-            from StringIO import StringIO
-        self.packet, self.socket = self.request
-        self.rfile = StringIO(self.packet)
-        self.wfile = StringIO()
-
-    def finish(self):
-        self.socket.sendto(self.wfile.getvalue(), self.client_address)
+"""Generic socket server classes.
+
+This module tries to capture the various aspects of defining a server:
+
+For socket-based servers:
+
+- address family:
+        - AF_INET{,6}: IP (Internet Protocol) sockets (default)
+        - AF_UNIX: Unix domain sockets
+        - others, e.g. AF_DECNET are conceivable (see <socket.h>
+- socket type:
+        - SOCK_STREAM (reliable stream, e.g. TCP)
+        - SOCK_DGRAM (datagrams, e.g. UDP)
+
+For request-based servers (including socket-based):
+
+- client address verification before further looking at the request
+        (This is actually a hook for any processing that needs to look
+         at the request before anything else, e.g. logging)
+- how to handle multiple requests:
+        - synchronous (one request is handled at a time)
+        - forking (each request is handled by a new process)
+        - threading (each request is handled by a new thread)
+
+The classes in this module favor the server type that is simplest to
+write: a synchronous TCP/IP server.  This is bad class design, but
+save some typing.  (There's also the issue that a deep class hierarchy
+slows down method lookups.)
+
+There are five classes in an inheritance diagram, four of which represent
+synchronous servers of four types:
+
+        +------------+
+        | BaseServer |
+        +------------+
+              |
+              v
+        +-----------+        +------------------+
+        | TCPServer |------->| UnixStreamServer |
+        +-----------+        +------------------+
+              |
+              v
+        +-----------+        +--------------------+
+        | UDPServer |------->| UnixDatagramServer |
+        +-----------+        +--------------------+
+
+Note that UnixDatagramServer derives from UDPServer, not from
+UnixStreamServer -- the only difference between an IP and a Unix
+stream server is the address family, which is simply repeated in both
+unix server classes.
+
+Forking and threading versions of each type of server can be created
+using the ForkingMixIn and ThreadingMixIn mix-in classes.  For
+instance, a threading UDP server class is created as follows:
+
+        class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
+
+The Mix-in class must come first, since it overrides a method defined
+in UDPServer! Setting the various member variables also changes
+the behavior of the underlying server mechanism.
+
+To implement a service, you must derive a class from
+BaseRequestHandler and redefine its handle() method.  You can then run
+various versions of the service by combining one of the server classes
+with your request handler class.
+
+The request handler class must be different for datagram or stream
+services.  This can be hidden by using the request handler
+subclasses StreamRequestHandler or DatagramRequestHandler.
+
+Of course, you still have to use your head!
+
+For instance, it makes no sense to use a forking server if the service
+contains state in memory that can be modified by requests (since the
+modifications in the child process would never reach the initial state
+kept in the parent process and passed to each child).  In this case,
+you can use a threading server, but you will probably have to use
+locks to avoid two requests that come in nearly simultaneous to apply
+conflicting changes to the server state.
+
+On the other hand, if you are building e.g. an HTTP server, where all
+data is stored externally (e.g. in the file system), a synchronous
+class will essentially render the service "deaf" while one request is
+being handled -- which may be for a very long time if a client is slow
+to read all the data it has requested.  Here a threading or forking
+server is appropriate.
+
+In some cases, it may be appropriate to process part of a request
+synchronously, but to finish processing in a forked child depending on
+the request data.  This can be implemented by using a synchronous
+server and doing an explicit fork in the request handler class
+handle() method.
+
+Another approach to handling multiple simultaneous requests in an
+environment that supports neither threads nor fork (or where these are
+too expensive or inappropriate for the service) is to maintain an
+explicit table of partially finished requests and to use select() to
+decide which request to work on next (or whether to handle a new
+incoming request).  This is particularly important for stream services
+where each client can potentially be connected for a long time (if
+threads or subprocesses cannot be used).
+
+Future work:
+- Standard classes for Sun RPC (which uses either UDP or TCP)
+- Standard mix-in classes to implement various authentication
+  and encryption schemes
+- Standard framework for select-based multiplexing
+
+XXX Open problems:
+- What to do with out-of-band data?
+
+BaseServer:
+- split generic "request" functionality out into BaseServer class.
+  Copyright (C) 2000  Luke Kenneth Casson Leighton <lkcl at samba.org>
+
+  example: read entries from a SQL database (requires overriding
+  get_request() to return a table entry from the database).
+  entry is processed by a RequestHandlerClass.
+
+"""
+
+# Author of the BaseServer patch: Luke Kenneth Casson Leighton
+
+# XXX Warning!
+# There is a test suite for this module, but it cannot be run by the
+# standard regression test.
+# To run it manually, run Lib/test/test_socketserver.py.
+
+__version__ = "0.4"
+
+
+import socket
+import select
+import sys
+import os
+import errno
+try:
+    import threading
+except ImportError:
+    import dummy_threading as threading
+
+__all__ = ["TCPServer","UDPServer","ForkingUDPServer","ForkingTCPServer",
+           "ThreadingUDPServer","ThreadingTCPServer","BaseRequestHandler",
+           "StreamRequestHandler","DatagramRequestHandler",
+           "ThreadingMixIn", "ForkingMixIn"]
+if hasattr(socket, "AF_UNIX"):
+    __all__.extend(["UnixStreamServer","UnixDatagramServer",
+                    "ThreadingUnixStreamServer",
+                    "ThreadingUnixDatagramServer"])
+
+def _eintr_retry(func, *args):
+    """restart a system call interrupted by EINTR"""
+    while True:
+        try:
+            return func(*args)
+        except (OSError, select.error) as e:
+            if e.args[0] != errno.EINTR:
+                raise
+
+class BaseServer:
+
+    """Base class for server classes.
+
+    Methods for the caller:
+
+    - __init__(server_address, RequestHandlerClass)
+    - serve_forever(poll_interval=0.5)
+    - shutdown()
+    - handle_request()  # if you do not use serve_forever()
+    - fileno() -> int   # for select()
+
+    Methods that may be overridden:
+
+    - server_bind()
+    - server_activate()
+    - get_request() -> request, client_address
+    - handle_timeout()
+    - verify_request(request, client_address)
+    - server_close()
+    - process_request(request, client_address)
+    - shutdown_request(request)
+    - close_request(request)
+    - handle_error()
+
+    Methods for derived classes:
+
+    - finish_request(request, client_address)
+
+    Class variables that may be overridden by derived classes or
+    instances:
+
+    - timeout
+    - address_family
+    - socket_type
+    - allow_reuse_address
+
+    Instance variables:
+
+    - RequestHandlerClass
+    - socket
+
+    """
+
+    timeout = None
+
+    def __init__(self, server_address, RequestHandlerClass):
+        """Constructor.  May be extended, do not override."""
+        self.server_address = server_address
+        self.RequestHandlerClass = RequestHandlerClass
+        self.__is_shut_down = threading.Event()
+        self.__shutdown_request = False
+
+    def server_activate(self):
+        """Called by constructor to activate the server.
+
+        May be overridden.
+
+        """
+        pass
+
+    def serve_forever(self, poll_interval=0.5):
+        """Handle one request at a time until shutdown.
+
+        Polls for shutdown every poll_interval seconds. Ignores
+        self.timeout. If you need to do periodic tasks, do them in
+        another thread.
+        """
+        self.__is_shut_down.clear()
+        try:
+            while not self.__shutdown_request:
+                # XXX: Consider using another file descriptor or
+                # connecting to the socket to wake this up instead of
+                # polling. Polling reduces our responsiveness to a
+                # shutdown request and wastes cpu at all other times.
+                r, w, e = _eintr_retry(select.select, [self], [], [],
+                                       poll_interval)
+                if self in r:
+                    self._handle_request_noblock()
+        finally:
+            self.__shutdown_request = False
+            self.__is_shut_down.set()
+
+    def shutdown(self):
+        """Stops the serve_forever loop.
+
+        Blocks until the loop has finished. This must be called while
+        serve_forever() is running in another thread, or it will
+        deadlock.
+        """
+        self.__shutdown_request = True
+        self.__is_shut_down.wait()
+
+    # The distinction between handling, getting, processing and
+    # finishing a request is fairly arbitrary.  Remember:
+    #
+    # - handle_request() is the top-level call.  It calls
+    #   select, get_request(), verify_request() and process_request()
+    # - get_request() is different for stream or datagram sockets
+    # - process_request() is the place that may fork a new process
+    #   or create a new thread to finish the request
+    # - finish_request() instantiates the request handler class;
+    #   this constructor will handle the request all by itself
+
+    def handle_request(self):
+        """Handle one request, possibly blocking.
+
+        Respects self.timeout.
+        """
+        # Support people who used socket.settimeout() to escape
+        # handle_request before self.timeout was available.
+        timeout = self.socket.gettimeout()
+        if timeout is None:
+            timeout = self.timeout
+        elif self.timeout is not None:
+            timeout = min(timeout, self.timeout)
+        fd_sets = _eintr_retry(select.select, [self], [], [], timeout)
+        if not fd_sets[0]:
+            self.handle_timeout()
+            return
+        self._handle_request_noblock()
+
+    def _handle_request_noblock(self):
+        """Handle one request, without blocking.
+
+        I assume that select.select has returned that the socket is
+        readable before this function was called, so there should be
+        no risk of blocking in get_request().
+        """
+        try:
+            request, client_address = self.get_request()
+        except socket.error:
+            return
+        if self.verify_request(request, client_address):
+            try:
+                self.process_request(request, client_address)
+            except:
+                self.handle_error(request, client_address)
+                self.shutdown_request(request)
+
+    def handle_timeout(self):
+        """Called if no new request arrives within self.timeout.
+
+        Overridden by ForkingMixIn.
+        """
+        pass
+
+    def verify_request(self, request, client_address):
+        """Verify the request.  May be overridden.
+
+        Return True if we should proceed with this request.
+
+        """
+        return True
+
+    def process_request(self, request, client_address):
+        """Call finish_request.
+
+        Overridden by ForkingMixIn and ThreadingMixIn.
+
+        """
+        self.finish_request(request, client_address)
+        self.shutdown_request(request)
+
+    def server_close(self):
+        """Called to clean-up the server.
+
+        May be overridden.
+
+        """
+        pass
+
+    def finish_request(self, request, client_address):
+        """Finish one request by instantiating RequestHandlerClass."""
+        self.RequestHandlerClass(request, client_address, self)
+
+    def shutdown_request(self, request):
+        """Called to shutdown and close an individual request."""
+        self.close_request(request)
+
+    def close_request(self, request):
+        """Called to clean up an individual request."""
+        pass
+
+    def handle_error(self, request, client_address):
+        """Handle an error gracefully.  May be overridden.
+
+        The default is to print a traceback and continue.
+
+        """
+        print '-'*40
+        print 'Exception happened during processing of request from',
+        print client_address
+        import traceback
+        traceback.print_exc() # XXX But this goes to stderr!
+        print '-'*40
+
+
+class TCPServer(BaseServer):
+
+    """Base class for various socket-based server classes.
+
+    Defaults to synchronous IP stream (i.e., TCP).
+
+    Methods for the caller:
+
+    - __init__(server_address, RequestHandlerClass, bind_and_activate=True)
+    - serve_forever(poll_interval=0.5)
+    - shutdown()
+    - handle_request()  # if you don't use serve_forever()
+    - fileno() -> int   # for select()
+
+    Methods that may be overridden:
+
+    - server_bind()
+    - server_activate()
+    - get_request() -> request, client_address
+    - handle_timeout()
+    - verify_request(request, client_address)
+    - process_request(request, client_address)
+    - shutdown_request(request)
+    - close_request(request)
+    - handle_error()
+
+    Methods for derived classes:
+
+    - finish_request(request, client_address)
+
+    Class variables that may be overridden by derived classes or
+    instances:
+
+    - timeout
+    - address_family
+    - socket_type
+    - request_queue_size (only for stream sockets)
+    - allow_reuse_address
+
+    Instance variables:
+
+    - server_address
+    - RequestHandlerClass
+    - socket
+
+    """
+
+    address_family = socket.AF_INET
+
+    socket_type = socket.SOCK_STREAM
+
+    request_queue_size = 5
+
+    allow_reuse_address = False
+
+    def __init__(self, server_address, RequestHandlerClass, bind_and_activate=True):
+        """Constructor.  May be extended, do not override."""
+        BaseServer.__init__(self, server_address, RequestHandlerClass)
+        self.socket = socket.socket(self.address_family,
+                                    self.socket_type)
+        if bind_and_activate:
+            self.server_bind()
+            self.server_activate()
+
+    def server_bind(self):
+        """Called by constructor to bind the socket.
+
+        May be overridden.
+
+        """
+        if self.allow_reuse_address:
+            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        self.socket.bind(self.server_address)
+        try:
+            self.server_address = self.socket.getsockname()
+        except socket.error:
+            # Jython may raise ENOTCONN here;
+            # we will pick up again in server_activate
+            pass
+
+    def server_activate(self):
+        """Called by constructor to activate the server.
+
+        May be overridden.
+
+        """
+        self.socket.listen(self.request_queue_size)
+        # Adding a second call to getsockname() because of this issue
+        # http://wiki.python.org/jython/NewSocketModule#Deferredsocketcreationonjython
+        self.server_address = self.socket.getsockname()
+        
+    def server_close(self):
+        """Called to clean-up the server.
+
+        May be overridden.
+
+        """
+        self.socket.close()
+
+    def fileno(self):
+        """Return socket file number.
+
+        Interface required by select().
+
+        """
+        return self.socket.fileno()
+
+    def get_request(self):
+        """Get the request and client address from the socket.
+
+        May be overridden.
+
+        """
+        return self.socket.accept()
+
+    def shutdown_request(self, request):
+        """Called to shutdown and close an individual request."""
+        try:
+            #explicitly shutdown.  socket.close() merely releases
+            #the socket and waits for GC to perform the actual close.
+            request.shutdown(socket.SHUT_WR)
+        except socket.error:
+            pass #some platforms may raise ENOTCONN here
+        self.close_request(request)
+
+    def close_request(self, request):
+        """Called to clean up an individual request."""
+        request.close()
+
+
+class UDPServer(TCPServer):
+
+    """UDP server class."""
+
+    allow_reuse_address = False
+
+    socket_type = socket.SOCK_DGRAM
+
+    max_packet_size = 8192
+
+    def get_request(self):
+        data, client_addr = self.socket.recvfrom(self.max_packet_size)
+        return (data, self.socket), client_addr
+
+    def server_activate(self):
+        # No need to call listen() for UDP.
+        pass
+
+    def shutdown_request(self, request):
+        # No need to shutdown anything.
+        self.close_request(request)
+
+    def close_request(self, request):
+        # No need to close anything.
+        pass
+
+class ForkingMixIn:
+
+    """Mix-in class to handle each request in a new process."""
+
+    timeout = 300
+    active_children = None
+    max_children = 40
+
+    def collect_children(self):
+        """Internal routine to wait for children that have exited."""
+        if self.active_children is None: return
+        while len(self.active_children) >= self.max_children:
+            # XXX: This will wait for any child process, not just ones
+            # spawned by this library. This could confuse other
+            # libraries that expect to be able to wait for their own
+            # children.
+            try:
+                pid, status = os.waitpid(0, 0)
+            except os.error:
+                pid = None
+            if pid not in self.active_children: continue
+            self.active_children.remove(pid)
+
+        # XXX: This loop runs more system calls than it ought
+        # to. There should be a way to put the active_children into a
+        # process group and then use os.waitpid(-pgid) to wait for any
+        # of that set, but I couldn't find a way to allocate pgids
+        # that couldn't collide.
+        for child in self.active_children:
+            try:
+                pid, status = os.waitpid(child, os.WNOHANG)
+            except os.error:
+                pid = None
+            if not pid: continue
+            try:
+                self.active_children.remove(pid)
+            except ValueError, e:
+                raise ValueError('%s. x=%d and list=%r' % (e.message, pid,
+                                                           self.active_children))
+
+    def handle_timeout(self):
+        """Wait for zombies after self.timeout seconds of inactivity.
+
+        May be extended, do not override.
+        """
+        self.collect_children()
+
+    def process_request(self, request, client_address):
+        """Fork a new subprocess to process the request."""
+        self.collect_children()
+        pid = os.fork()
+        if pid:
+            # Parent process
+            if self.active_children is None:
+                self.active_children = []
+            self.active_children.append(pid)
+            self.close_request(request) #close handle in parent process
+            return
+        else:
+            # Child process.
+            # This must never return, hence os._exit()!
+            try:
+                self.finish_request(request, client_address)
+                self.shutdown_request(request)
+                os._exit(0)
+            except:
+                try:
+                    self.handle_error(request, client_address)
+                    self.shutdown_request(request)
+                finally:
+                    os._exit(1)
+
+
+class ThreadingMixIn:
+    """Mix-in class to handle each request in a new thread."""
+
+    # Decides how threads will act upon termination of the
+    # main process
+    daemon_threads = False
+
+    def process_request_thread(self, request, client_address):
+        """Same as in BaseServer but as a thread.
+
+        In addition, exception handling is done here.
+
+        """
+        try:
+            self.finish_request(request, client_address)
+            self.shutdown_request(request)
+        except:
+            self.handle_error(request, client_address)
+            self.shutdown_request(request)
+
+    def process_request(self, request, client_address):
+        """Start a new thread to process the request."""
+        t = threading.Thread(target = self.process_request_thread,
+                             args = (request, client_address))
+        t.daemon = self.daemon_threads
+        t.start()
+
+
+class ForkingUDPServer(ForkingMixIn, UDPServer): pass
+class ForkingTCPServer(ForkingMixIn, TCPServer): pass
+
+class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
+class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass
+
+if hasattr(socket, 'AF_UNIX'):
+
+    class UnixStreamServer(TCPServer):
+        address_family = socket.AF_UNIX
+
+    class UnixDatagramServer(UDPServer):
+        address_family = socket.AF_UNIX
+
+    class ThreadingUnixStreamServer(ThreadingMixIn, UnixStreamServer): pass
+
+    class ThreadingUnixDatagramServer(ThreadingMixIn, UnixDatagramServer): pass
+
+class BaseRequestHandler:
+
+    """Base class for request handler classes.
+
+    This class is instantiated for each request to be handled.  The
+    constructor sets the instance variables request, client_address
+    and server, and then calls the handle() method.  To implement a
+    specific service, all you need to do is to derive a class which
+    defines a handle() method.
+
+    The handle() method can find the request as self.request, the
+    client address as self.client_address, and the server (in case it
+    needs access to per-server information) as self.server.  Since a
+    separate instance is created for each request, the handle() method
+    can define arbitrary other instance variariables.
+
+    """
+
+    def __init__(self, request, client_address, server):
+        self.request = request
+        self.client_address = client_address
+        self.server = server
+        self.setup()
+        try:
+            self.handle()
+        finally:
+            self.finish()
+
+    def setup(self):
+        pass
+
+    def handle(self):
+        pass
+
+    def finish(self):
+        pass
+
+
+# The following two classes make it possible to use the same service
+# class for stream or datagram servers.
+# Each class sets up these instance variables:
+# - rfile: a file object from which receives the request is read
+# - wfile: a file object to which the reply is written
+# When the handle() method returns, wfile is flushed properly
+
+
+class StreamRequestHandler(BaseRequestHandler):
+
+    """Define self.rfile and self.wfile for stream sockets."""
+
+    # Default buffer sizes for rfile, wfile.
+    # We default rfile to buffered because otherwise it could be
+    # really slow for large data (a getc() call per byte); we make
+    # wfile unbuffered because (a) often after a write() we want to
+    # read and we need to flush the line; (b) big writes to unbuffered
+    # files are typically optimized by stdio even when big reads
+    # aren't.
+    rbufsize = -1
+    wbufsize = 0
+
+    # A timeout to apply to the request socket, if not None.
+    timeout = None
+
+    # Disable nagle algorithm for this socket, if True.
+    # Use only when wbufsize != 0, to avoid small packets.
+    disable_nagle_algorithm = False
+
+    def setup(self):
+        self.connection = self.request
+        if self.timeout is not None:
+            self.connection.settimeout(self.timeout)
+        if self.disable_nagle_algorithm:
+            self.connection.setsockopt(socket.IPPROTO_TCP,
+                                       socket.TCP_NODELAY, True)
+        self.rfile = self.connection.makefile('rb', self.rbufsize)
+        self.wfile = self.connection.makefile('wb', self.wbufsize)
+
+    def finish(self):
+        if not self.wfile.closed:
+            try:
+                self.wfile.flush()
+            except socket.error:
+                # An final socket error may have occurred here, such as
+                # the local error ECONNABORTED.
+                pass
+        self.wfile.close()
+        self.rfile.close()
+
+
+class DatagramRequestHandler(BaseRequestHandler):
+
+    # XXX Regrettably, I cannot get this working on Linux;
+    # s.recvfrom() doesn't return a meaningful client address.
+
+    """Define self.rfile and self.wfile for datagram sockets."""
+
+    def setup(self):
+        try:
+            from cStringIO import StringIO
+        except ImportError:
+            from StringIO import StringIO
+        self.packet, self.socket = self.request
+        self.rfile = StringIO(self.packet)
+        self.wfile = StringIO()
+
+    def finish(self):
+        self.socket.sendto(self.wfile.getvalue(), self.client_address)
diff --git a/Lib/_socket.py b/Lib/_socket.py
new file mode 100644
--- /dev/null
+++ b/Lib/_socket.py
@@ -0,0 +1,2059 @@
+import array
+import encodings.idna
+import errno
+import jarray
+import logging
+import pprint
+import sys
+import time
+import _google_ipaddr_r234
+from collections import namedtuple, Sequence
+from contextlib import contextmanager
+from functools import partial, wraps
+from itertools import chain
+from numbers import Number
+from StringIO import StringIO
+from threading import Condition, Lock
+from types import MethodType, NoneType
+
+import java
+from java.io import IOException, InterruptedIOException
+from java.lang import Thread
+from java.net import InetAddress, InetSocketAddress
+from java.nio.channels import ClosedChannelException
+from java.util import NoSuchElementException
+from java.util.concurrent import (
+    ArrayBlockingQueue, CopyOnWriteArrayList, CountDownLatch, LinkedBlockingQueue,
+    RejectedExecutionException, ThreadFactory, TimeUnit)
+from java.util.concurrent.atomic import AtomicBoolean
+from javax.net.ssl import SSLPeerUnverifiedException
+
+try:
+    # jarjar-ed version
+    from org.python.netty.bootstrap import Bootstrap, ChannelFactory, ServerBootstrap
+    from org.python.netty.buffer import PooledByteBufAllocator, Unpooled
+    from org.python.netty.channel import ChannelInboundHandlerAdapter, ChannelInitializer, ChannelOption
+    from org.python.netty.channel.nio import NioEventLoopGroup
+    from org.python.netty.channel.socket import DatagramPacket
+    from org.python.netty.channel.socket.nio import NioDatagramChannel, NioSocketChannel, NioServerSocketChannel
+except ImportError:
+    # dev version from extlibs
+    from io.netty.bootstrap import Bootstrap, ChannelFactory, ServerBootstrap
+    from io.netty.buffer import PooledByteBufAllocator, Unpooled
+    from io.netty.channel import ChannelInboundHandlerAdapter, ChannelInitializer, ChannelOption
+    from io.netty.channel.nio import NioEventLoopGroup
+    from io.netty.channel.socket import DatagramPacket
+    from io.netty.channel.socket.nio import NioDatagramChannel, NioSocketChannel, NioServerSocketChannel
+
+
+log = logging.getLogger("_socket")
+
+
+def _debug():
+    FORMAT = '%(asctime)-15s %(threadName)s %(levelname)s %(funcName)s %(message)s %(sock)s'
+    logging.basicConfig(format=FORMAT, level=logging.DEBUG)
+
+
+# Constants
+###########
+
+has_ipv6 = True # IPV6 FTW!
+_GLOBAL_DEFAULT_TIMEOUT = object()
+_EPHEMERAL_ADDRESS = InetSocketAddress(0)
+_BOUND_EPHEMERAL_ADDRESS = object()
+
+# FIXME most constants should come from JNR if possible; they may be
+# arbitrary for the implementation of socket/ssl/select purposes, but
+# some misbehaved code may want to use the arbitrary numbers
+
+SHUT_RD   = 0
+SHUT_WR   = 1
+SHUT_RDWR = 2
+
+AF_UNSPEC = 0
+AF_INET   = 2
+AF_INET6  = 23
+
+AI_PASSIVE     = 1
+AI_CANONNAME   = 2
+AI_NUMERICHOST = 4
+AI_V4MAPPED    = 8
+AI_ALL         = 16
+AI_ADDRCONFIG  = 32
+AI_NUMERICSERV = 1024
+
+EAI_NONAME     = -2
+EAI_SERVICE    = -8
+EAI_ADDRFAMILY = -9
+
+NI_NUMERICHOST              = 1
+NI_NUMERICSERV              = 2
+NI_NOFQDN                   = 4
+NI_NAMEREQD                 = 8
+NI_DGRAM                    = 16
+NI_MAXSERV                  = 32
+NI_IDN                      = 64
+NI_IDN_ALLOW_UNASSIGNED     = 128
+NI_IDN_USE_STD3_ASCII_RULES = 256
+NI_MAXHOST                  = 1025
+
+SOCK_DGRAM     = 1
+SOCK_STREAM    = 2
+SOCK_RAW       = 3 # not supported
+SOCK_RDM       = 4 # not supported
+SOCK_SEQPACKET = 5 # not supported
+
+SOL_SOCKET = 0xFFFF
+
+IPPROTO_AH       =  51 # not supported
+IPPROTO_DSTOPTS  =  60 # not supported
+IPPROTO_ESP      =  50 # not supported
+IPPROTO_FRAGMENT =  44 # not supported
+IPPROTO_GGP      =   3 # not supported
+IPPROTO_HOPOPTS  =   0 # not supported
+IPPROTO_ICMP     =   1 # not supported
+IPPROTO_ICMPV6   =  58 # not supported
+IPPROTO_IDP      =  22 # not supported
+IPPROTO_IGMP     =   2 # not supported
+IPPROTO_IP       =   0
+IPPROTO_IPV4     =   4 # not supported
+IPPROTO_IPV6     =  41 # not supported
+IPPROTO_MAX      = 256 # not supported
+IPPROTO_ND       =  77 # not supported
+IPPROTO_NONE     =  59 # not supported
+IPPROTO_PUP      =  12 # not supported
+IPPROTO_RAW      = 255 # not supported
+IPPROTO_ROUTING  =  43 # not supported
+IPPROTO_TCP      =   6
+IPPROTO_UDP      =  17
+
+SO_ACCEPTCONN  = 1
+SO_BROADCAST   = 2
+SO_ERROR       = 4
+SO_KEEPALIVE   = 8
+SO_LINGER      = 16
+SO_OOBINLINE   = 32
+SO_RCVBUF      = 64
+SO_REUSEADDR   = 128
+SO_SNDBUF      = 256
+SO_TIMEOUT     = 512
+SO_TYPE        = 1024
+
+# Options with negative constants are not supported
+# They are being added here so that code that refers to them
+# will not break with an AttributeError
+
+SO_DEBUG            = -1
+SO_DONTROUTE        = -1
+SO_EXCLUSIVEADDRUSE = -8
+SO_RCVLOWAT         = -16
+SO_RCVTIMEO         = -32
+SO_REUSEPORT        = -64
+SO_SNDLOWAT         = -128
+SO_SNDTIMEO         = -256
+SO_USELOOPBACK      = -512
+
+TCP_NODELAY    = 2048
+
+INADDR_ANY = "0.0.0.0"
+INADDR_BROADCAST = "255.255.255.255"
+
+IN6ADDR_ANY_INIT = "::"
+
+POLLIN   = 1
+POLLOUT  = 2
+POLLPRI  = 4   # Ignored - not supportable on Java
+POLLERR  = 8
+POLLHUP  = 16
+POLLNVAL = 32  # Polled when not open - no Netty channel
+
+
+# Specific constants for socket-reboot:
+
+# Keep the highest possible precision for converting from Python's use
+# of floating point for durations to Java's use of both a long
+# duration and a specific unit, in this case TimeUnit.NANOSECONDS
+_TO_NANOSECONDS = 1000000000
+
+_PEER_CLOSED = object()
+
+
+# Event loop management
+#######################
+
+# Use daemon threads for the event loop group. This is just fine
+# because these threads only handle ephemeral data, such as performing
+# SSL wrap/unwrap.
+
+class DaemonThreadFactory(ThreadFactory):
+    def newThread(self, runnable):
+        t = Thread(runnable)
+        t.daemon = True
+        return t
+
+
+# This number should be configurable by the user. 10 is the default
+# number as of 4.0.17 of Netty. FIXME this default may be based on core count.
+
+NIO_GROUP = NioEventLoopGroup(10, DaemonThreadFactory())
+
+def _check_threadpool_for_pending_threads():
+    pending_threads = []
+    for t in NIO_GROUP:
+        pending_count = t.pendingTasks()
+        if pending_count > 0:
+            pending_threads.append((t, pending_count))
+    log.debug("Pending threads in Netty pool: %s", pprint.pformat(pending_threads),  extra={"sock": "*"})
+    return pending_threads
+
+
+def _shutdown_threadpool():
+    log.debug("Shutting down thread pool...", extra={"sock": "*"})
+    # FIXME this timeout probably should be configurable; for client
+    # usage that have completed this probably only produces scary
+    # messages at worst, but TBD; in particular this may because we
+    # are seeing closes both in SSL and at the socket level
+
+    NIO_GROUP.shutdownGracefully(0, 100, TimeUnit.MILLISECONDS)
+    log.debug("Shut down thread pool", extra={"sock": "*"})
+
+# Ensure deallocation of thread pool if PySystemState.cleanup is
+# called; this includes in the event of sigterm
+
+sys.registerCloser(_shutdown_threadpool)
+
+
+# Error management
+##################
+
+class error(IOError): pass
+class herror(error): pass
+class gaierror(error): pass
+class timeout(error): pass
+class SSLError(error): pass
+
+SSL_ERROR_SSL = 1
+SSL_ERROR_WANT_READ = 2
+SSL_ERROR_WANT_WRITE = 3
+SSL_ERROR_WANT_X509_LOOKUP = 4
+SSL_ERROR_SYSCALL = 5
+SSL_ERROR_ZERO_RETURN = 6
+SSL_ERROR_WANT_CONNECT = 7
+SSL_ERROR_EOF = 8
+SSL_ERROR_INVALID_ERROR_CODE = 9
+
+
+def _add_exception_attrs(exc):
+    exc.errno = exc[0]
+    exc.strerror = exc[1]
+    return exc
+
+
+def _unmapped_exception(exc):
+    return _add_exception_attrs(error(-1, 'Unmapped exception: %s' % exc))
+
+
+def java_net_socketexception_handler(exc):
+    if exc.message.startswith("Address family not supported by protocol family"):
+        return _add_exception_attrs(
+            error(errno.EAFNOSUPPORT, 
+                  'Address family not supported by protocol family: See http://wiki.python.org/jython/NewSocketModule#IPV6_address_support'))
+    return _unmapped_exception(exc)
+
+
+def would_block_error(exc=None):
+    return _add_exception_attrs(
+        error(errno.EWOULDBLOCK, 'The socket operation could not complete without blocking'))
+
+
+_exception_map = {
+
+    # javaexception : callable that raises the python equivalent exception, or None to stub out as unmapped
+
+    IOException            : lambda x: error(errno.ECONNRESET, 'Software caused connection abort'),
+    InterruptedIOException : lambda x: timeout(None, 'timed out'),
+    
+    java.net.BindException            : lambda x: error(errno.EADDRINUSE, 'Address already in use'),
+    java.net.ConnectException         : lambda x: error(errno.ECONNREFUSED, 'Connection refused'),
+    java.net.NoRouteToHostException   : lambda x: error(errno.EHOSTUNREACH, 'No route to host'),
+    java.net.PortUnreachableException : None,
+    java.net.ProtocolException        : None,
+    java.net.SocketException          : java_net_socketexception_handler,
+    java.net.SocketTimeoutException   : lambda x: timeout(None, 'timed out'),
+    java.net.UnknownHostException     : lambda x: gaierror(errno.EGETADDRINFOFAILED, 'getaddrinfo failed'),
+
+    java.nio.channels.AlreadyConnectedException       : lambda x: error(errno.EISCONN, 'Socket is already connected'),
+    java.nio.channels.AsynchronousCloseException      : None,
+    java.nio.channels.CancelledKeyException           : None,
+    java.nio.channels.ClosedByInterruptException      : None,
+    java.nio.channels.ClosedChannelException          : lambda x: error(errno.ECONNRESET, 'Socket closed'),
+    java.nio.channels.ClosedSelectorException         : None,
+    java.nio.channels.ConnectionPendingException      : None,
+    java.nio.channels.IllegalBlockingModeException    : None,
+    java.nio.channels.IllegalSelectorException        : None,
+    java.nio.channels.NoConnectionPendingException    : None,
+    java.nio.channels.NonReadableChannelException     : None,
+    java.nio.channels.NonWritableChannelException     : None,
+    java.nio.channels.NotYetBoundException            : None,
+    java.nio.channels.NotYetConnectedException        : None,
+    java.nio.channels.UnresolvedAddressException      : lambda x: gaierror(errno.EGETADDRINFOFAILED, 'getaddrinfo failed'),
+    java.nio.channels.UnsupportedAddressTypeException : None,
+
+    SSLPeerUnverifiedException: lambda x: SSLError(SSL_ERROR_SSL, "FIXME"),
+}
+
+
+def _map_exception(java_exception):
+    mapped_exception = _exception_map.get(java_exception.__class__)
+    if mapped_exception:
+        py_exception = mapped_exception(java_exception)
+    else:
+        py_exception = error(-1, 'Unmapped exception: %s' % java_exception)
+    py_exception.java_exception = java_exception
+    return _add_exception_attrs(py_exception)
+
+
+def raises_java_exception(method_or_function):
+    """Maps java socket exceptions to the equivalent python exception.
+    Also sets _last_error on socket objects so as to support SO_ERROR.
+    """
+    
+    @wraps(method_or_function)
+    def handle_exception(*args, **kwargs):
+        is_socket = len(args) > 0 and isinstance(args[0], _realsocket)
+        try:
+            try:
+                return method_or_function(*args, **kwargs)
+            except java.lang.Exception, jlx:
+                raise _map_exception(jlx)
+        except error, e:
+            if is_socket:
+                args[0]._last_error = e[0]
+            raise
+        else:
+            if is_socket:
+                args[0]._last_error = 0
+    return handle_exception
+
+
+# select support
+################
+
+class _Select(object):
+
+    def __init__(self, rlist, wlist, xlist):
+        self.cv = Condition()
+        self.rlist = frozenset(rlist)
+        self.wlist = frozenset(wlist)
+        self.xlist = frozenset(xlist)
+
+    def _normalize_sockets(self, socks):
+        # Get underlying socket, via fileno lookup
+        _socks = []
+        for sock in socks:
+            try:
+                _sock = sock.fileno()
+                _sock._register_selector  # double check our API requirements
+                _socks.append(_sock)
+            except AttributeError:
+                raise error(errno.EBADF, "Bad file descriptor: %s" % (sock,))
+        return _socks
+
+    def notify(self, sock, **_):
+        with self.cv:
+            self.cv.notify()
+
+    def __str__(self):
+        return "_Select(r={},w={},x={})".format(list(self.rlist), list(self.wlist), list(self.xlist))
+
+    @contextmanager
+    def _register_sockets(self, socks):
+        socks = self._normalize_sockets(socks)
+        for sock in socks:
+            sock._register_selector(self)
+        yield self
+        for sock in socks:
+            sock._unregister_selector(self)
+
+    def __call__(self, timeout):
+        started = time.time()
+        with self.cv, self._register_sockets(chain(self.rlist, self.wlist, self.xlist)):
+            while True:
+                # Checking if sockets are ready (readable OR writable)
+                # converts selection from detecting edges to detecting levels
+                selected_rlist = set(sock for sock in self.rlist if sock.fileno()._readable())
+                selected_wlist = set(sock for sock in self.wlist if sock.fileno()._writable())
+                # FIXME add support for exceptions
+                selected_xlist = []
+
+                # As usual with condition variables, we need to ensure
+                # there's not a spurious wakeup; this test also ensures
+                # shortcircuiting if the socket was in fact ready for
+                # reading/writing/exception before the select call
+                if selected_rlist or selected_wlist:
+                    return sorted(selected_rlist), sorted(selected_wlist), sorted(selected_xlist)
+                elif timeout is not None and time.time() - started >= timeout:
+                    return [], [], []
+                self.cv.wait(timeout)
+
+
+# poll support
+##############
+
+_PollNotification = namedtuple("_PollNotification", ["sock", "fd", "exception", "hangup"])
+
+
+class poll(object):
+    
+    def __init__(self):
+        self.queue = LinkedBlockingQueue()
+        self.registered = dict()  # fd -> eventmask
+
+    def notify(self, sock, exception=None, hangup=False):
+        notification = _PollNotification(
+            sock=sock,
+            fd=sock.fileno(),
+            exception=exception,
+            hangup=hangup)
+        self.queue.put(notification)
+
+    def register(self, fd, eventmask=POLLIN|POLLPRI|POLLOUT):
+        self.registered[fd] = eventmask
+        # NOTE in case fd != sock in a future release, modifiy accordingly
+        sock = fd
+        sock._register_selector(self)
+        self.notify(sock)  # Ensure we get an initial notification
+
+    def modify(self, fd, eventmask):
+        if fd not in self.registered:
+            raise error(errno.ENOENT, "No such file or directory")
+        self.registered[fd] = eventmask
+
+    def unregister(self, fd):
+        del self.registered[fd]
+        sock = fd
+        sock._unregister_selector(self)
+
+    def _event_test(self, notification):
+        # Performs standard edge vs event polling, except that we get
+        # edges around errors and hangup
+        if notification is None:
+            return None, 0
+        mask = self.registered.get(notification.sock, 0)   # handle if concurrently removed, by simply ignoring
+        event = 0
+        if mask & POLLIN and notification.sock._readable():
+            event |= POLLIN
+        if mask & POLLOUT and notification.sock._writable():
+            event |= POLLOUT
+        if mask & POLLERR and notification.exception:
+            event |= POLLERR
+        if mask & POLLHUP and (notification.hangup or not notification.sock.channel):
+            event |= POLLHUP
+        if mask & POLLNVAL and not notification.sock.peer_closed:
+            event |= POLLNVAL
+        return notification.fd, event
+
+    def poll(self, timeout=None):
+        if not timeout or timeout < 0:
+            # Simplify logic around timeout resets
+            timeout = None
+        else:
+            timeout /= 1000.  # convert from milliseconds to seconds
+
+        while True:
+            if timeout is None:
+                notification = self.queue.take()
+            elif timeout > 0:
+                started = time.time()
+                timeout_in_ns = int(timeout * _TO_NANOSECONDS)
+                notification = self.queue.poll(timeout_in_ns, TimeUnit.NANOSECONDS)
+                # Need to reset the timeout, because this notification
+                # may not be of interest when masked out
+                timeout = timeout - (time.time() - started)
+            else:
+                return []
+
+            if notification is None:
+                continue
+            
+            # Pull as many outstanding notifications as possible out
+            # of the queue
+            notifications = [notification]
+            self.queue.drainTo(notifications)
+            log.debug("Got notification(s) %s", notifications, extra={"sock": "MODULE"})
+            result = []
+            socks = set()
+
+            # But given how we notify, it's possible to see possible
+            # multiple notifications. Just return one (fd, event) for a
+            # given socket
+            for notification in notifications:
+                if notification.sock not in socks:
+                    fd, event = self._event_test(notification)
+                    if event:
+                        result.append((fd, event))
+                        socks.add(notification.sock)
+            # Repump sockets to pick up a subsequent level change
+            for sock in socks:
+                self.notify(sock)
+            if result:
+                return result
+
+
+# integration with Netty
+########################
+
+class PythonInboundHandler(ChannelInboundHandlerAdapter):
+
+    def __init__(self, sock):
+        self.sock = sock
+        log.debug("Initializing inbound handler", extra={"sock": self.sock})
+
+    def channelActive(self, ctx):
+        log.debug("Channel is active", extra={"sock": self.sock})
+        self.sock._notify_selectors()
+        ctx.fireChannelActive()
+
+    def channelRead(self, ctx, msg):
+        log.debug("Channel read message %s", msg, extra={"sock": self.sock})
+        msg.retain()  # bump ref count so it can be used in the blocking queue
+        self.sock.incoming.put(msg)
+        self.sock._notify_selectors()
+        ctx.fireChannelRead(msg)
+
+    def channelWritabilityChanged(self, ctx):
+        log.debug("Channel ready for write", extra={"sock": self.sock})
+        self.sock._notify_selectors()
+        ctx.fireChannelWritabilityChanged()
+
+    def exceptionCaught(self, ctx, cause):
+        log.debug("Channel caught exception %s", cause, extra={"sock": self.sock})
+        self.sock._notify_selectors(exception=cause)
+        ctx.fireExceptionCaught(cause) 
+
+
+class ChildSocketHandler(ChannelInitializer):
+
+    def __init__(self, parent_socket):
+        self.parent_socket = parent_socket
+
+    def initChannel(self, child_channel):
+        child = ChildSocket()
+        child.proto = IPPROTO_TCP
+        child._init_client_mode(child_channel)
+
+        # Get most current options from the parent. This enables any subsequent divergence.
+        #
+        # It's OK that this copy could occur without a mutex, given that such iteration
+        # is guaranteed to be weakly consistent
+        child.options = dict(((option, value) for option, value in self.parent_socket.options.iteritems()))
+        if child.options:
+            log.debug("Setting inherited options %s", child.options, extra={"sock": child})
+            config = child_channel.config()
+            for option, value in child.options.iteritems():
+                config.setOption(option, value)
+
+        log.debug("Notifing listeners of parent socket %s", self.parent_socket, extra={"sock": child})
+        self.parent_socket.child_queue.put(child)
+        self.parent_socket._notify_selectors()
+        log.debug("Notified listeners of parent socket %s with queue %s",
+                  self.parent_socket, self.parent_socket.child_queue, extra={"sock": child})
+
+        # Must block until the child socket is actually "used". This is
+        # because there may be some additional setup required, such as
+        # wrapping the socket, before the child is ready to read.
+
+        def unlatch_child(_):
+            # FIXME when bound methods are supported for single method interfaces
+            child._unlatch()
+
+        # Ensure that this handler will not block if the channel is closed,
+        # otherwise this handler will simply sit idly as a pending task in the Netty
+        # thread pool
+        child_channel.closeFuture().addListener(unlatch_child)
+
+        child._wait_on_latch()
+        log.debug("Socket initChannel completed waiting on latch", extra={"sock": child})
+
+
+# FIXME raise exceptions for ops not permitted on client socket, server socket
+UNKNOWN_SOCKET, CLIENT_SOCKET, SERVER_SOCKET, DATAGRAM_SOCKET = range(4)
+
+
+# These are the only socket protocols we currently support, so it's easy to map as follows:
+
+_socket_options = {
+    IPPROTO_TCP: {
+        (SOL_SOCKET,  SO_KEEPALIVE):   (ChannelOption.SO_KEEPALIVE, bool),
+        (SOL_SOCKET,  SO_LINGER):      (ChannelOption.SO_LINGER, int),
+        (SOL_SOCKET,  SO_RCVBUF):      (ChannelOption.SO_RCVBUF, int),
+        (SOL_SOCKET,  SO_REUSEADDR):   (ChannelOption.SO_REUSEADDR, bool),
+        (SOL_SOCKET,  SO_SNDBUF):      (ChannelOption.SO_SNDBUF, int),
+        # FIXME SO_TIMEOUT needs to be handled by an IdleStateHandler -
+        # ChannelOption.SO_TIMEOUT really only applies to OIO (old) socket channels,
+        # we want to use NIO ones
+        (SOL_SOCKET,  SO_TIMEOUT):     (ChannelOption.SO_TIMEOUT, int),
+        (IPPROTO_TCP, TCP_NODELAY):    (ChannelOption.TCP_NODELAY, bool),
+    },
+    IPPROTO_UDP: {
+        (SOL_SOCKET,  SO_BROADCAST):   (ChannelOption.SO_BROADCAST, bool),
+        (SOL_SOCKET,  SO_LINGER):      (ChannelOption.SO_LINGER, int),
+        (SOL_SOCKET,  SO_RCVBUF):      (ChannelOption.SO_RCVBUF, int),
+        (SOL_SOCKET,  SO_REUSEADDR):   (ChannelOption.SO_REUSEADDR, bool),
+        (SOL_SOCKET,  SO_SNDBUF):      (ChannelOption.SO_SNDBUF, int),
+        (SOL_SOCKET,  SO_TIMEOUT):     (ChannelOption.SO_TIMEOUT, int),
+    }
+}
+
+# actual socket support
+#######################
+
+class _realsocket(object):
+
+    def __init__(self, family=None, type=None, proto=None):
+        # FIXME verify args are correct
+        self.family = family
+        self.type = type
+        if proto is None:
+            if type == SOCK_STREAM:
+                proto = IPPROTO_TCP
+            elif type == SOCK_DGRAM:
+                proto = IPPROTO_UDP
+        self.proto = proto
+
+        self._last_error = 0  # supports SO_ERROR
+        self.connected = False
+        self.timeout = _defaulttimeout
+        self.channel = None
+        self.bind_addr = _EPHEMERAL_ADDRESS
+        self.bind_timestamp = None   # Handle Netty race condition on bound addresses
+        self.selectors = CopyOnWriteArrayList()
+        self.options = {}  # deferred options until bootstrap
+        self.peer_closed = False
+
+        # Reference count this underlying socket
+        self.open_lock = Lock()
+        self.open_count = 1
+
+        if self.type == SOCK_DGRAM:
+            self.socket_type = DATAGRAM_SOCKET
+            self.incoming = LinkedBlockingQueue()  # list of read buffers
+            self.incoming_head = None  # allows msg buffers to be broken up
+            self.python_inbound_handler = None
+            self._can_write = True
+        else:
+            self.socket_type = UNKNOWN_SOCKET
+
+    def __repr__(self):
+        return "<_realsocket at {:#x} type={} open_count={} channel={} timeout={}>".format(
+            id(self), self.socket_type, self.open_count, self.channel, self.timeout)
+
+    def _unlatch(self):
+        pass  # no-op once mutated from ChildSocket to normal _socketobject
+
+    def _register_selector(self, selector):
+        self.selectors.addIfAbsent(selector)
+
+    def _unregister_selector(self, selector):
+        return self.selectors.remove(selector)
+
+    def _notify_selectors(self, exception=None, hangup=False):
+        for selector in self.selectors:
+            selector.notify(self, exception=exception, hangup=hangup)
+
+    @raises_java_exception
+    def _handle_channel_future(self, future, reason):
+        # All differences between nonblocking vs blocking with optional timeouts
+        # is managed by this method.
+        #
+        # All sockets can be selected on, regardless of blocking/nonblocking state.
+
+        def workaround_jython_bug_for_bound_methods(_):
+            # FIXME check for errors, notify the corresponding socket accordingly
+            # FIXME wrapper function is needed because of http://bugs.jython.org/issue2115
+            self._notify_selectors()
+
+        future.addListener(workaround_jython_bug_for_bound_methods)
+
+        if self.timeout is None:
+            log.debug("Syncing on future %s for %s", future, reason, extra={"sock": self})
+            return future.sync()
+        elif self.timeout:
+            self._handle_timeout(future.await, reason)
+            if not future.isSuccess():
+                raise future.cause()
+            return future
+        else:
+            return future
+
+    def setblocking(self, flag):
+        if flag:
+            self.settimeout(None)
+        else:
+            self.settimeout(0.0)
+
+    def settimeout(self, timeout):
+        self.timeout = _calctimeoutvalue(timeout)
+
+    def gettimeout(self):
+        return self.timeout
+
+    def _handle_timeout(self, waiter, reason):
+        timeout_in_ns = int(self.timeout * _TO_NANOSECONDS)
+        log.debug("Waiting for up to %.2fs for %s", self.timeout, reason, extra={"sock": self})
+        started = time.time()
+        result = waiter(timeout_in_ns, TimeUnit.NANOSECONDS)
+        log.debug("Completed in %.2fs",  time.time() - started, extra={"sock": self})
+        if not result:
+            # above predicate handles either the case the waiter is
+            # returning a value or in the case ChannelFuture#await,
+            # that the timeout expired, in which case False is
+            # returned
+            if self.timeout == 0:
+                raise error(errno.ETIMEDOUT, "Connection timed out")
+            else:
+                raise timeout("timed out")
+        return result
+
+    def bind(self, address):
+        # Netty 4 supports binding a socket to multiple addresses;
+        # apparently this is the not the case for C API sockets
+        self.bind_addr = _get_jsockaddr(address, self.family, self.type, self.proto, AI_PASSIVE)
+        self._datagram_connect()  # as necessary
+
+    # CLIENT METHODS
+    # Calling connect/connect_ex means this is a client socket; these
+    # in turn use _connect, which uses Bootstrap, not ServerBootstrap
+
+    def _init_client_mode(self, channel=None):
+        # this is client socket specific 
+        self.socket_type = CLIENT_SOCKET
+        self.incoming = LinkedBlockingQueue()  # list of read buffers
+        self.incoming_head = None  # allows msg buffers to be broken up
+        self.python_inbound_handler = None
+        self._can_write = True
+        self.connect_handlers = []
+        self.connected = False
+        if channel:
+            log.debug("Setting up channel %s", channel, extra={"sock": self})
+            self.channel = channel
+            self.python_inbound_handler = PythonInboundHandler(self)
+            self.connect_handlers = [self.python_inbound_handler]
+            self.connected = True
+
+    def _connect(self, addr):
+        log.debug("Begin connection to %s", addr, extra={"sock": self})
+        addr = _get_jsockaddr(addr, self.family, self.type, self.proto, 0)
+        self._init_client_mode()
+        self.connected = True
+        self.python_inbound_handler = PythonInboundHandler(self)
+        bootstrap = Bootstrap().group(NIO_GROUP).channel(NioSocketChannel)
+        for option, value in self.options.iteritems():
+            bootstrap.option(option, value)
+
+        # FIXME really this is just for SSL handling, so make more
+        # specific than a list of connect_handlers
+        if self.connect_handlers:
+            for handler in self.connect_handlers:
+                bootstrap.handler(handler)
+        else:
+            bootstrap.handler(self.python_inbound_handler)
+
+        if self.bind_addr:
+            log.debug("Connect %s to %s", self.bind_addr, addr, extra={"sock": self})
+            bind_future = bootstrap.bind(self.bind_addr)
+            self._handle_channel_future(bind_future, "local bind")
+            self.channel = bind_future.channel()
+            future = self.channel.connect(addr)
+        else:
+            log.debug("Connect to %s", addr, extra={"sock": self})
+            future = bootstrap.connect(addr)
+            self.channel = future.channel()
+            
+        self._handle_channel_future(future, "connect")
+        self.bind_timestamp = time.time()
+
+    def _post_connect(self):
+        # Post-connect step is necessary to handle SSL setup,
+        # otherwise the read adapter can race in seeing encrypted
+        # messages from the peer
+        if self.connect_handlers:
+            self.channel.pipeline().addLast(self.python_inbound_handler)
+        
+        def peer_closed(x):
+            log.debug("Peer closed channel %s", x, extra={"sock": self})
+            self.incoming.put(_PEER_CLOSED)
+            self._notify_selectors(hangup=True)
+
+        self.channel.closeFuture().addListener(peer_closed)
+
+    def connect(self, addr):
+        # Unwrapped sockets can immediately perform the post-connect step
+        if self.socket_type == DATAGRAM_SOCKET:
+            self._datagram_connect(addr)
+            log.debug("Completed datagram connection to %s", addr, extra={"sock": self})
+        else:
+            self._connect(addr)
+            self._post_connect()
+            log.debug("Completed connection to %s", addr, extra={"sock": self})
+
+    def connect_ex(self, addr):
+        self.connect(addr)
+        if self.timeout is None:
+            return errno.EISCONN
+        else:
+            return errno.EINPROGRESS
+
+
+    # SERVER METHODS
+    # Calling listen means this is a server socket
+
+    def listen(self, backlog):
+        self.socket_type = SERVER_SOCKET
+
+        b = ServerBootstrap()
+        self.group = NioEventLoopGroup(10, DaemonThreadFactory())
+        b.group(self.group)
+        b.channel(NioServerSocketChannel)
+        b.option(ChannelOption.SO_BACKLOG, backlog)
+        for option, value in self.options.iteritems():
+            b.option(option, value)
+            # Note that child options are set in the child handler so
+            # that they can take into account any subsequent changes,
+            # plus have shadow support
+
+        self.child_queue = ArrayBlockingQueue(backlog)
+        self.child_handler = ChildSocketHandler(self)
+        b.childHandler(self.child_handler)
+
+        future = b.bind(self.bind_addr)
+        self._handle_channel_future(future, "listen")
+        self.bind_timestamp = time.time()
+        self.channel = future.channel()
+        log.debug("Bound server socket to %s", self.bind_addr, extra={"sock": self})
+
+    def accept(self):
+        if self.timeout is None:
+            log.debug("Blocking indefinitely for child on queue %s", self.child_queue, extra={"sock": self})
+            child = self.child_queue.take()
+        elif self.timeout:
+            log.debug("Timed wait for child on queue %s", self.child_queue, extra={"sock": self})
+            child = self._handle_timeout(self.child_queue.poll, "accept")
+        else:
+            log.debug("Polling for child on queue %s", self.child_queue, extra={"sock": self})
+            child = self.child_queue.poll()
+            if child is None:
+                raise error(errno.EWOULDBLOCK, "Resource temporarily unavailable")
+        peername = child.getpeername() if child else None
+        log.debug("Got child %s connected to %s", child, peername, extra={"sock": self})
+        return child, peername
+
+    # DATAGRAM METHODS
+    
+    def _datagram_connect(self, addr=None):
+        # FIXME raise exception if not of the right family
+        if addr is not None:
+            addr = _get_jsockaddr(addr, self.family, self.type, self.proto, 0)
+
+        if not self.connected and self.socket_type == DATAGRAM_SOCKET:
+            log.debug("Binding datagram socket to %s", self.bind_addr, extra={"sock": self})
+            self.connected = True
+            self.python_inbound_handler = PythonInboundHandler(self)
+            bootstrap = Bootstrap().group(NIO_GROUP).channel(NioDatagramChannel)
+            bootstrap.handler(self.python_inbound_handler)
+            for option, value in self.options.iteritems():
+                bootstrap.option(option, value)
+
+            future = bootstrap.register()
+            self._handle_channel_future(future, "register")
+            self.channel = future.channel()
+            self._handle_channel_future(self.channel.bind(self.bind_addr), "bind")
+
+        if addr is not None:
+            # Handles the relatively rare case that this is a
+            # CONNECTED datagram socket, which Netty does not
+            # support in its bootstrap setup.
+            log.debug("Connecting datagram socket to %s", addr, extra={"sock": self})
+            future = self.channel.connect(addr)
+            self._handle_channel_future(future, "connect")
+
+    def sendto(self, string, arg1, arg2=None):
+        # Unfortunate arg overloading
+        if arg2 is not None:
+            flags = arg1
+            address = arg2
+        else:
+            flags = None
+            address = arg1
+
+        address =  _get_jsockaddr(address, self.family, self.type, self.proto, 0)
+
+        log.debug("Sending datagram to %s <<<{!r:.20}>>>".format(string), address, extra={"sock": self})
+        self._datagram_connect()
+        packet = DatagramPacket(Unpooled.wrappedBuffer(string), address)
+        future = self.channel.writeAndFlush(packet)
+        self._handle_channel_future(future, "sendto")
+        return len(string)
+
+
+    # FIXME implement these methods
+
+    def recvfrom_into(self, buffer, nbytes=0, flags=0):
+        raise NotImplementedError()
+
+    def recv_into(self, buffer, nbytes=0, flags=0):
+        raise NotImplementedError()
+
+    # GENERAL METHODS
+                                             
+    def close(self):
+        with self.open_lock:
+            self.open_count -= 1
+            if self.open_count > 0:
+                log.debug("Open count > 0, so not closing underlying socket", extra={"sock": self})
+                return
+
+            if self.channel is None:
+                return
+
+            try:
+                self.channel.close().sync()
+            except RejectedExecutionException:
+                # Do not care about tasks that attempt to schedule after close
+                pass
+            if self.socket_type == SERVER_SOCKET:
+                self.group.shutdownGracefully(0, 100, TimeUnit.MILLISECONDS)
+                while True:
+                    child = self.child_queue.poll()
+                    if child is None:
+                        break
+                    log.debug("Closed child socket %s not yet accepted", child, extra={"sock": self})
+                    child.close()
+
+            log.debug("Closed socket", extra={"sock": self})
+
+    def shutdown(self, how):
+        self._verify_channel()
+        if how & SHUT_RD:
+            try:
+                self.channel.pipeline().remove(self.python_inbound_handler)
+            except NoSuchElementException:
+                pass  # already removed, can safely ignore (presumably)
+        if how & SHUT_WR:
+            self._can_write = False
+            
+    def _readable(self):
+        if self.socket_type == CLIENT_SOCKET or self.socket_type == DATAGRAM_SOCKET:
+            return ((self.incoming_head is not None and self.incoming_head.readableBytes()) or
+                    self.incoming.peek())
+        elif self.socket_type == SERVER_SOCKET:
+            return bool(self.child_queue.peek())
+        else:
+            return False
+
+    def _writable(self):
+        return self.channel and self.channel.isActive() and self.channel.isWritable()
+
+    can_write = _writable
+
+    def _verify_channel(self):
+        if self.channel is None:
+            log.debug("Channel is not connected or setup", extra={"sock": self})
+            raise error(errno.ENOTCONN, "Socket is not connected")
+
+    @raises_java_exception
+    def send(self, data, flags=0):
+        # FIXME this almost certainly needs to chunk things
+        self._verify_channel()
+        data = str(data)  # FIXME temporary fix if data is of type buffer
+        log.debug("Sending data <<<{!r:.20}>>>".format(data), extra={"sock": self})
+
+        if self.socket_type == DATAGRAM_SOCKET:
+            packet = DatagramPacket(Unpooled.wrappedBuffer(data), self.channel.remoteAddress())
+            future = self.channel.writeAndFlush(packet)
+            self._handle_channel_future(future, "send")
+            return len(data)
+
+        if not self._can_write:
+            raise error(errno.ENOTCONN, 'Socket not connected')
+        future = self.channel.writeAndFlush(Unpooled.wrappedBuffer(data))
+        self._handle_channel_future(future, "send")
+        # FIXME are we sure we are going to be able to send this much data, especially async?
+        return len(data)
+    
+    sendall = send   # FIXME see note above!
+
+    def _get_incoming_msg(self, reason):
+        if self.incoming_head is None:
+            if self.timeout is None:
+                if self.peer_closed:
+                    return None
+                self.incoming_head = self.incoming.take()
+            elif self.timeout:
+                if self.peer_closed:
+                    return None
+                self.incoming_head = self._handle_timeout(self.incoming.poll, reason)
+            else:
+                self.incoming_head = self.incoming.poll()  # Could be None
+                if self.incoming_head is None:
+                    # FIXME FIXME C socket semantics return a '' after the first EAGAIN (not certain if this gets reset or not)
+                    log.debug("No data yet for socket", extra={"sock": self})
+                    raise error(errno.EAGAIN, "Resource temporarily unavailable")
+
+        # Only return _PEER_CLOSED once
+        msg = self.incoming_head
+        if msg is _PEER_CLOSED:
+            self.incoming_head = None
+            self.peer_closed = True
+        return msg
+
+    @raises_java_exception
+    def _get_message(self, bufsize, reason):
+        self._datagram_connect()
+        self._verify_channel()
+        msg = self._get_incoming_msg(reason)
+
+        if self.socket_type == DATAGRAM_SOCKET:
+            if msg is None:
+                return None, None
+            elif msg is _PEER_CLOSED:
+                return "", None
+        else:
+            if msg is None:
+                return None, self.channel.remoteAddress()
+            elif msg is _PEER_CLOSED:
+                return "", self.channel.remoteAddress()
+
+        if self.socket_type == DATAGRAM_SOCKET:
+            content = msg.content()
+            sender = msg.sender()
+        else:
+            content = msg
+            sender = self.channel.remoteAddress()
+        msg_length = content.readableBytes()
+        buf = jarray.zeros(min(msg_length, bufsize), "b")
+        content.readBytes(buf)
+        if content.readableBytes() == 0:
+            msg.release()  # return msg ByteBuf back to Netty's pool
+            self.incoming_head = None
+        return buf.tostring(), sender
+
+    def recv(self, bufsize, flags=0):
+        self._verify_channel()
+        log.debug("Waiting on recv", extra={"sock": self})
+        # For obvious reasons, concurrent reads on the same socket
+        # have to be locked; I don't believe it is the job of recv to
+        # do this; in particular this is the policy of SocketChannel,
+        # which underlies Netty's support for such channels.
+        data, _ = self._get_message(bufsize, "recv")
+        log.debug("Received <<<{!r:.20}>>>".format(data), extra={"sock": self})
+        return data
+        
+    def recvfrom(self, bufsize, flags=0):
+        self._verify_channel()
+        data, sender = self._get_message(bufsize, "recvfrom")
+        remote_addr = sender.getHostString(), sender.getPort()
+        log.debug("Received from sender %s <<<{!r:20}>>>".format(data), remote_addr, extra={"sock": self})
+        return data, remote_addr
+
+    def fileno(self):
+        return self
+
+    def setsockopt(self, level, optname, value):
+        try:
+            option, cast = _socket_options[self.proto][(level, optname)]
+        except KeyError:
+            raise error(errno.ENOPROTOOPT, "Protocol not available")
+
+        # FIXME for NIO sockets, SO_TIMEOUT doesn't work - should use
+        # IdleStateHandler instead
+        cast_value = cast(value)
+        self.options[option] = cast_value
+        log.debug("Setting option %s to %s", optname, value, extra={"sock": self})
+        if self.channel:
+            self.channel.config().setOption(option, cast(value))
+
+    def getsockopt(self, level, optname, buflen=None):
+        # Pseudo options for interrogating the status of this socket
+        if level == SOL_SOCKET:
+            if optname == SO_ACCEPTCONN:
+                if self.socket_type == SERVER_SOCKET:
+                    return 1
+                elif self.type == SOCK_STREAM:
+                    return 0
+                else:
+                    raise error(errno.ENOPROTOOPT, "Protocol not available")
+            if optname == SO_TYPE:
+                return self.type
+            if optname == SO_ERROR:
+                last_error = self._last_error
+                self._last_error = 0
+                return last_error
+
+        # Normal options
+        try:
+            option, _ = _socket_options[self.proto][(level, optname)]
+        except KeyError:
+            raise error(errno.ENOPROTOOPT, "Protocol not available")
+        log.debug("Shadow option settings %s", self.options, extra={"sock": self})
+        return self.options.get(option, 0)
+
+    def getsockname(self):
+        if self.channel is None:
+            if self.bind_addr == _EPHEMERAL_ADDRESS:
+                raise error(errno.ENOTCONN, "Socket is not connected")
+            else:
+                return self.bind_addr.getHostString(), self.bind_addr.getPort()
+        # Netty 4 currently races between bind to ephemeral port and the availability
+        # of the local address for the channel. Workaround for now is to poll.
+        while True:
+            local_addr = self.channel.localAddress()
+            if local_addr:
+                break
+            if time.time() - self.bind_timestamp > 1:
+                # Presumably after a second something is completely wrong,
+                # so punt
+                raise error(errno.ENOTCONN, "Socket is not connected")
+            log.debug("Poll for local address", extra={"sock": self})
+            time.sleep(0.1)  # completely arbitrary
+        return local_addr.getHostString(), local_addr.getPort()
+
+    def getpeername(self):
+        self._verify_channel()
+        remote_addr = self.channel.remoteAddress()
+        if remote_addr is None:
+            raise error(errno.ENOTCONN, "Socket is not connected")
+        return remote_addr.getHostString(), remote_addr.getPort()
+
+
+_socketmethods = (
+    'bind', 'connect', 'connect_ex', 'fileno', 'listen',
+    'getpeername', 'getsockname', 'getsockopt', 'setsockopt',
+    'sendall', 'setblocking',
+    'settimeout', 'gettimeout', 'shutdown')
+
+
+# All the method names that must be delegated to either the real socket
+# object or the _closedsocket object.
+# For socket-reboot, this also means anything used by _Select
+
+_delegate_methods = (
+    "recv", "recvfrom", "recv_into", "recvfrom_into",
+    "send", "sendto", "fileno")
+
+class _closedsocket(object):
+
+    def close(self):
+        pass  # Should be able to close repeatedly
+
+    def _dummy(*args):
+        raise error(errno.EBADF, 'Bad file descriptor')
+
+    # All _delegate_methods must also be initialized here.
+    fileno = send = recv = recv_into = sendto = recvfrom = recvfrom_into = _dummy
+
+    __getattr__ = _dummy
+
+
+# Wrapper around platform socket objects. This implements
+# a platform-independent dup() functionality. The
+# implementation currently relies on reference counting
+# to close the underlying socket object.
+class _socketobject(object):
+
+    __doc__ = _realsocket.__doc__
+
+
+    def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0, _sock=None):
+        if _sock is None:
+            _sock = _realsocket(family, type, proto)
+        self._sock = _sock
+        for method in _delegate_methods:
+            setattr(self, method, getattr(_sock, method))
+
+    def close(self, _closedsocket=_closedsocket,
+              _delegate_methods=_delegate_methods, setattr=setattr):
+        # This function should not reference any globals. See issue #808164.
+        self._sock.close()
+        self._sock = _closedsocket()
+        dummy = self._sock._dummy
+        for method in _delegate_methods:
+            setattr(self, method, dummy)
+    close.__doc__ = _realsocket.close.__doc__
+
+    def fileno(self):
+        return self._sock
+
+    def accept(self):
+        sock, addr = self._sock.accept()
+        return _socketobject(_sock=sock), addr
+    accept.__doc__ = _realsocket.accept.__doc__
+
+    def dup(self):
+        """dup() -> socket object
+
+        Return a new socket object connected to the same system resource."""
+
+        if isinstance(self._sock, _closedsocket):
+            return _socketobject(_sock=_closedsocket())
+        with self._sock.open_lock:
+            self._sock.open_count += 1
+            return _socketobject(_sock=self._sock)
+
+    def makefile(self, mode='r', bufsize=-1):
+        """makefile([mode[, bufsize]]) -> file object
+
+        Return a regular file object corresponding to the socket.  The mode
+        and bufsize arguments are as for the built-in open() function."""
+
+        if isinstance(self._sock, _closedsocket):
+            return _fileobject(_closedsocket(), mode, bufsize, close=True)
+        with self._sock.open_lock:
+            self._sock.open_count += 1
+            return _fileobject(self._sock, mode, bufsize, close=True)
+
+    family = property(lambda self: self._sock.family, doc="the socket family")
+    type = property(lambda self: self._sock.type, doc="the socket type")
+    proto = property(lambda self: self._sock.proto, doc="the socket protocol")
+
+
+def meth(name,self,*args):
+    return getattr(self._sock,name)(*args)
+
+
+for _m in _socketmethods:
+    p = partial(meth,_m)
+    p.__name__ = _m
+    p.__doc__ = getattr(_realsocket,_m).__doc__
+    m = MethodType(p,None,_socketobject)
+    setattr(_socketobject,_m,m)
+
+
+socket = SocketType = _socketobject
+
+
+class ChildSocket(_realsocket):
+    
+    def __init__(self):
+        super(ChildSocket, self).__init__()
+        self.active = AtomicBoolean()
+        self.active_latch = CountDownLatch(1)
+
+    def _ensure_post_connect(self):
+        do_post_connect = not self.active.getAndSet(True)
+        if do_post_connect:
+            self._post_connect()
+            self.active_latch.countDown()
+
+    def _wait_on_latch(self):
+        log.debug("Waiting for activity", extra={"sock": self})
+        self.active_latch.await()
+        log.debug("Latch released, can now proceed", extra={"sock": self})
+
+    # FIXME raise exception for accept, listen, bind, connect, connect_ex
+
+    # All ops that allow us to characterize the mode of operation of
+    # this socket as being either Start TLS or SSL when
+    # connected. These should be ops that send/receive/change
+    # connection, not metadata.
+
+    def send(self, data):
+        self._ensure_post_connect()
+        return super(ChildSocket, self).send(data)
+
+    sendall = send
+
+    def recv(self, bufsize, flags=0):
+        self._ensure_post_connect()
+        return super(ChildSocket, self).recv(bufsize, flags)
+
+    def recvfrom(self, bufsize, flags=0):
+        self._ensure_post_connect()
+        return super(ChildSocket, self).recvfrom(bufsize, flags)
+
+    def setblocking(self, mode):
+        self._ensure_post_connect()
+        return super(ChildSocket, self).setblocking(mode)
+
+    # FIXME FIXME FIXME other ops.
+
+    def close(self):
+        self._ensure_post_connect()
+        super(ChildSocket, self).close()
+
+    def shutdown(self, how):
+        self._ensure_post_connect()
+        super(ChildSocket, self).shutdown(how)
+
+    def __del__(self):
+        # Required in the case this child socket never becomes active.
+        # This cleanup will ensure that the pending thread for the
+        # handler is released when a GC happens, not necessarily
+        # before shutdown of course.  Naturally no extra work will be
+        # done in setting up the channel.
+        self.active_latch.countDown()
+        self.close()
+
+
+# EXPORTED constructors
+
+def socket(family=None, type=None, proto=None):
+    return _socketobject(family, type, proto)
+
+
+def select(rlist, wlist, xlist, timeout=None):
+    for lst in (rlist, wlist, xlist):
+        if not isinstance(lst, Sequence):
+            raise TypeError("arguments 1-3 must be sequences")
+    if not(timeout is None or isinstance(timeout, Number)):
+        raise TypeError("timeout must be a float or None")
+    if timeout is not None and timeout < 0:
+        raise error(errno.EINVAL, "Invalid argument")
+    return _Select(rlist, wlist, xlist)(timeout)
+
+
+def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
+                      source_address=None):
+    """Connect to *address* and return the socket object.
+
+    Convenience function.  Connect to *address* (a 2-tuple ``(host,
+    port)``) and return the socket object.  Passing the optional
+    *timeout* parameter will set the timeout on the socket instance
+    before attempting to connect.  If no *timeout* is supplied, the
+    global default timeout setting returned by :func:`getdefaulttimeout`
+    is used.  If *source_address* is set it must be a tuple of (host, port)
+    for the socket to bind as a source address before making the connection.
+    An host of '' or port 0 tells the OS to use the default.
+    """
+
+    host, port = address
+    err = None
+    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
+        af, socktype, proto, canonname, sa = res
+        sock = None
+        try:
+            sock = socket(af, socktype, proto)
+            if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
+                sock.settimeout(timeout)
+            if source_address:
+                sock.bind(source_address)
+            sock.connect(sa)
+            return sock
+
+        except error as _:
+            err = _
+            if sock is not None:
+                sock.close()
+
+    if err is not None:
+        raise err
+    else:
+        raise error("getaddrinfo returns an empty list")
+
+
+# MISCELLANEOUS module level functions
+
+_defaulttimeout = None
+
+def _calctimeoutvalue(value):
+    if value is None:
+        return None
+    try:
+        floatvalue = float(value)
+    except:
+        raise TypeError('Socket timeout value must be a number or None')
+    if floatvalue < 0.0:
+        raise ValueError("Socket timeout value cannot be negative")
+    return floatvalue
+
+def getdefaulttimeout():
+    return _defaulttimeout
+
+def setdefaulttimeout(timeout):
+    global _defaulttimeout
+    _defaulttimeout = _calctimeoutvalue(timeout)
+
+
+# Define data structures to support IPV4 and IPV6.
+# FIXME are these ip address classes required by CPython API? Must they be old-style classes?
+
+class _ip_address_t: pass
+
+class _ipv4_address_t(_ip_address_t):
+
+    def __init__(self, sockaddr, port, jaddress):
+        self.sockaddr = sockaddr
+        self.port     = port
+        self.jaddress = jaddress
+
+    def __getitem__(self, index):
+        if   0 == index:
+            return self.sockaddr
+        elif 1 == index:
+            return self.port
+        else:
+            raise IndexError()
+
+    def __len__(self):
+        return 2
+
+    def __str__(self):
+        return "('%s', %d)" % (self.sockaddr, self.port)
+
+    __repr__ = __str__
+
+class _ipv6_address_t(_ip_address_t):
+
+    def __init__(self, sockaddr, port, jaddress):
+        self.sockaddr = sockaddr
+        self.port     = port
+        self.jaddress = jaddress
+
+    def __getitem__(self, index):
+        if   0 == index:
+            return self.sockaddr
+        elif 1 == index:
+            return self.port
+        elif 2 == index:
+            return 0
+        elif 3 == index:
+            return self.jaddress.scopeId
+        else:
+            raise IndexError()
+
+    def __len__(self):
+        return 4
+
+    def __str__(self):
+        return "('%s', %d, 0, %d)" % (self.sockaddr, self.port, self.jaddress.scopeId)
+
+    __repr__ = __str__
+
+
+
+def _get_jsockaddr(address_object, family, sock_type, proto, flags):
+    addr = _get_jsockaddr2(address_object, family, sock_type, proto, flags)
+    log.debug("Address %s for %s", addr, address_object, extra={"sock": "*"})
+    return addr
+
+def _get_jsockaddr2(address_object, family, sock_type, proto, flags):
+    # Is this an object that was returned from getaddrinfo? If so, it already contains an InetAddress
+    if isinstance(address_object, _ip_address_t):
+        return java.net.InetSocketAddress(address_object.jaddress, address_object[1]) 
+    # The user passed an address tuple, not an object returned from getaddrinfo
+    # So we must call getaddrinfo, after some translations and checking
+    if address_object is None:
+        address_object = ("", 0)
+    error_message = "Address must be a 2-tuple (ipv4: (host, port)) or a 4-tuple (ipv6: (host, port, flow, scope))"
+    if not isinstance(address_object, tuple) or \
+            ((family == AF_INET and len(address_object) != 2) or (family == AF_INET6 and len(address_object) not in [2,4] )) or \
+            not isinstance(address_object[0], (basestring, NoneType)) or \
+            not isinstance(address_object[1], (int, long)):
+        raise TypeError(error_message)
+    if len(address_object) == 4 and not isinstance(address_object[3], (int, long)):
+        raise TypeError(error_message)
+    hostname = address_object[0]
+    if hostname is not None:
+        hostname = hostname.strip()
+    port = address_object[1]
+    if family == AF_INET and sock_type == SOCK_DGRAM and hostname == "<broadcast>":
+        hostname = INADDR_BROADCAST
+    if hostname in ["", None]:
+        if flags & AI_PASSIVE:
+            hostname = {AF_INET: INADDR_ANY, AF_INET6: IN6ADDR_ANY_INIT}[family]
+        else:
+            hostname = "localhost"
+    if isinstance(hostname, unicode):
+        hostname = encodings.idna.ToASCII(hostname)
+    addresses = getaddrinfo(hostname, port, family, sock_type, proto, flags)
+    if len(addresses) == 0:
+        raise gaierror(errno.EGETADDRINFOFAILED, 'getaddrinfo failed')
+    return java.net.InetSocketAddress(addresses[0][4].jaddress, port)
+
+
+def _is_ip_address(addr, version=None):
+    try:
+        _google_ipaddr_r234.IPAddress(addr, version)
+        return True
+    except ValueError:
+        return False
+
+
+def is_ipv4_address(addr):
+    return _is_ip_address(addr, 4)
+
+
+def is_ipv6_address(addr):
+    return _is_ip_address(addr, 6)
+
+
+def is_ip_address(addr):
+    return _is_ip_address(addr)
+
+
+# Workaround for this (predominantly windows) issue
+# http://wiki.python.org/jython/NewSocketModule#IPV6_address_support
+
+_ipv4_addresses_only = False
+
+def _use_ipv4_addresses_only(value):
+    global _ipv4_addresses_only
+    _ipv4_addresses_only = value
+
+
+def _getaddrinfo_get_host(host, family, flags):
+    if not isinstance(host, basestring) and host is not None:
+        raise TypeError("getaddrinfo() argument 1 must be string or None")
+    if flags & AI_NUMERICHOST:
+        if not is_ip_address(host):
+            raise gaierror(EAI_NONAME, "Name or service not known")
+        if family == AF_INET and not is_ipv4_address(host):
+            raise gaierror(EAI_ADDRFAMILY, "Address family for hostname not supported")
+        if family == AF_INET6 and not is_ipv6_address(host):
+            raise gaierror(EAI_ADDRFAMILY, "Address family for hostname not supported")
+    if isinstance(host, unicode):
+        host = encodings.idna.ToASCII(host)
+    return host
+
+
+def _getaddrinfo_get_port(port, flags):
+    if isinstance(port, basestring):
+        try:
+            int_port = int(port)
+        except ValueError:
+            if flags & AI_NUMERICSERV:
+                raise gaierror(EAI_NONAME, "Name or service not known")
+            # Lookup the service by name
+            try:
+                int_port = getservbyname(port)
+            except error:
+                raise gaierror(EAI_SERVICE, "Servname not supported for ai_socktype")
+    elif port is None:
+        int_port = 0
+    elif not isinstance(port, (int, long)):
+        raise error("Int or String expected")
+    else:
+        int_port = int(port)
+    return int_port % 65536
+
+
+ at raises_java_exception
+def getaddrinfo(host, port, family=AF_UNSPEC, socktype=0, proto=0, flags=0):
+    if _ipv4_addresses_only:
+        family = AF_INET
+    if not family in [AF_INET, AF_INET6, AF_UNSPEC]:
+        raise gaierror(errno.EIO, 'ai_family not supported')
+    host = _getaddrinfo_get_host(host, family, flags)
+    port = _getaddrinfo_get_port(port, flags)
+    if socktype not in [0, SOCK_DGRAM, SOCK_STREAM]:
+        raise error(errno.ESOCKTNOSUPPORT, "Socket type is not supported")
+    filter_fns = []
+    filter_fns.append({
+        AF_INET:   lambda x: isinstance(x, java.net.Inet4Address),
+        AF_INET6:  lambda x: isinstance(x, java.net.Inet6Address),
+        AF_UNSPEC: lambda x: isinstance(x, java.net.InetAddress),
+    }[family])
+    if host in [None, ""]:
+        if flags & AI_PASSIVE:
+             hosts = {AF_INET: [INADDR_ANY], AF_INET6: [IN6ADDR_ANY_INIT], AF_UNSPEC: [INADDR_ANY, IN6ADDR_ANY_INIT]}[family]
+        else:
+             hosts = ["localhost"]
+    else:
+        hosts = [host]
+    results = []
+    for h in hosts:
+        for a in java.net.InetAddress.getAllByName(h):
+            if len([f for f in filter_fns if f(a)]):
+                family = {java.net.Inet4Address: AF_INET, java.net.Inet6Address: AF_INET6}[a.getClass()]
+                if flags & AI_CANONNAME:
+                    canonname = str(a.getCanonicalHostName())
+                else:
+                    canonname = ""
+                sockaddr = str(a.getHostAddress())
+                # TODO: Include flowinfo and scopeid in a 4-tuple for IPv6 addresses
+                sock_tuple = {AF_INET : _ipv4_address_t, AF_INET6 : _ipv6_address_t}[family](sockaddr, port, a)
+                if socktype == 0:
+                    socktypes = [SOCK_DGRAM, SOCK_STREAM]
+                else:
+                    socktypes = [socktype]
+                for result_socktype in socktypes:
+                    result_proto = {SOCK_DGRAM: IPPROTO_UDP, SOCK_STREAM: IPPROTO_TCP}[result_socktype]
+                    if proto in [0, result_proto]:
+                        # The returned socket will only support the result_proto
+                        # If this does not match the requested proto, don't return it
+                        results.append((family, result_socktype, result_proto, canonname, sock_tuple))
+    return results
+
+
+
+def htons(x): return x
+def htonl(x): return x
+def ntohs(x): return x
+def ntohl(x): return x
+
+ at raises_java_exception
+def inet_pton(family, ip_string):
+    if family == AF_INET:
+        if not is_ipv4_address(ip_string):
+            raise error("illegal IP address string passed to inet_pton")
+    elif family == AF_INET6:
+        if not is_ipv6_address(ip_string):
+            raise error("illegal IP address string passed to inet_pton")
+    else:
+        raise error(errno.EAFNOSUPPORT, "Address family not supported by protocol")
+    ia = java.net.InetAddress.getByName(ip_string)
+    bytes = []
+    for byte in ia.getAddress():
+        if byte < 0:
+            bytes.append(byte+256)
+        else:
+            bytes.append(byte)
+    return "".join([chr(byte) for byte in bytes])
+
+ at raises_java_exception
+def inet_ntop(family, packed_ip):
+    jByteArray = array.array("b", packed_ip)
+    if family == AF_INET:
+        if len(jByteArray) != 4:
+            raise ValueError("invalid length of packed IP address string")
+    elif family == AF_INET6:
+        if len(jByteArray) != 16:
+            raise ValueError("invalid length of packed IP address string")
+    else:
+        raise ValueError("unknown address family %s" % family)
+    ia = java.net.InetAddress.getByAddress(jByteArray)
+    return ia.getHostAddress()
+
+def inet_aton(ip_string):
+    return inet_pton(AF_INET, ip_string)
+
+def inet_ntoa(packed_ip):
+    return inet_ntop(AF_INET, packed_ip)
+
+
+
+# Various toplevel functions for the socket module
+##################################################
+
+def _gethostbyaddr(name):
+    # This is as close as I can get; at least the types are correct...
+    addresses = InetAddress.getAllByName(gethostbyname(name))
+    names = []
+    addrs = []
+    for addr in addresses:
+        names.append(str(addr.getHostName()))
+        addrs.append(str(addr.getHostAddress()))
+    return names, addrs
+
+ at raises_java_exception
+def getfqdn(name=None):
+    """
+    Return a fully qualified domain name for name. If name is omitted or empty
+    it is interpreted as the local host.  To find the fully qualified name,
+    the hostname returned by gethostbyaddr() is checked, then aliases for the
+    host, if available. The first name which includes a period is selected.
+    In case no fully qualified domain name is available, the hostname is retur
+    New in version 2.0.
+    """
+    if not name:
+        name = gethostname()
+    names, addrs = _gethostbyaddr(name)
+    for a in names:
+        if a.find(".") >= 0:
+            return a
+    return name
+
+ at raises_java_exception
+def gethostname():
+    return str(InetAddress.getLocalHost().getHostName())
+
+ at raises_java_exception
+def gethostbyname(name):
+    return str(InetAddress.getByName(name).getHostAddress())
+
+#
+# Skeleton implementation of gethostbyname_ex
+# Needed because urllib2 refers to it
+#
+
+ at raises_java_exception
+def gethostbyname_ex(name):
+    return name, [], gethostbyname(name)
+
+ at raises_java_exception
+def gethostbyaddr(name):
+    names, addrs = _gethostbyaddr(name)
+    return names[0], names, addrs
+
+
+try:
+    from jnr.netdb import Service, Protocol
+
+    def getservbyname(service_name, protocol_name=None):
+        service = Service.getServiceByName(service_name, protocol_name)
+        if service is None:
+            raise error('service/proto not found')
+        return service.getPort()
+
+    def getservbyport(port, protocol_name=None):
+        service = Service.getServiceByPort(port, protocol_name)
+        if service is None:
+            raise error('port/proto not found')
+        return service.getName()
+
+    def getprotobyname(protocol_name=None):
+        proto = Protocol.getProtocolByName(protocol_name)
+        if proto is None:
+            raise error('protocol not found')
+        return proto.getProto()
+
+except ImportError:
+    def getservbyname(service_name, protocol_name=None):
+        return None
+
+    def getservbyport(port, protocol_name=None):
+        return None
+
+    def getprotobyname(protocol_name=None):
+        return None
+
+
+def _getnameinfo_get_host(address, flags):
+    if not isinstance(address, basestring):
+        raise TypeError("getnameinfo() address 1 must be string, not None")
+    if isinstance(address, unicode):
+        address = encodings.idna.ToASCII(address)
+    jia = InetAddress.getByName(address)
+    result = jia.getCanonicalHostName()
+    if flags & NI_NAMEREQD:
+        if is_ip_address(result):
+            raise gaierror(EAI_NONAME, "Name or service not known")
+    elif flags & NI_NUMERICHOST:
+        result = jia.getHostAddress()
+    # Ignoring NI_NOFQDN for now
+    if flags & NI_IDN:
+        result = encodings.idna.ToASCII(result)
+    return result
+
+def _getnameinfo_get_port(port, flags):
+    if not isinstance(port, (int, long)):
+        raise TypeError("getnameinfo() port number must be an integer")
+    if flags & NI_NUMERICSERV:
+        return port
+    proto = None
+    if flags & NI_DGRAM:
+        proto = "udp"
+    return getservbyport(port, proto)
+
+ at raises_java_exception
+def getnameinfo(sock_addr, flags):
+    if not isinstance(sock_addr, tuple) or len(sock_addr) < 2:
+        raise TypeError("getnameinfo() argument 1 must be a tuple")
+    host = _getnameinfo_get_host(sock_addr[0], flags)
+    port = _getnameinfo_get_port(sock_addr[1], flags)
+    return (host, port)
+
+
+
+class _fileobject(object):
+    """Faux file object attached to a socket object."""
+
+    default_bufsize = 8192
+    name = "<socket>"
+
+    __slots__ = ["mode", "bufsize", "softspace",
+                 # "closed" is a property, see below
+                 "_sock", "_rbufsize", "_wbufsize", "_rbuf", "_wbuf", "_wbuf_len",
+                 "_close"]
+
+    def __init__(self, sock, mode='rb', bufsize=-1, close=False):
+        self._sock = sock
+        self.mode = mode # Not actually used in this version
+        if bufsize < 0:
+            bufsize = self.default_bufsize
+        self.bufsize = bufsize
+        self.softspace = False
+        # _rbufsize is the suggested recv buffer size.  It is *strictly*
+        # obeyed within readline() for recv calls.  If it is larger than
+        # default_bufsize it will be used for recv calls within read().
+        if bufsize == 0:
+            self._rbufsize = 1
+        elif bufsize == 1:
+            self._rbufsize = self.default_bufsize
+        else:
+            self._rbufsize = bufsize
+        self._wbufsize = bufsize
+        # We use StringIO for the read buffer to avoid holding a list
+        # of variously sized string objects which have been known to
+        # fragment the heap due to how they are malloc()ed and often
+        # realloc()ed down much smaller than their original allocation.
+        self._rbuf = StringIO()
+        self._wbuf = [] # A list of strings
+        self._wbuf_len = 0
+        self._close = close
+
+    def _getclosed(self):
+        return self._sock is None
+    closed = property(_getclosed, doc="True if the file is closed")
+
+    def close(self):
+        try:
+            if self._sock:
+                self.flush()
+        finally:
+            if self._close:
+                self._sock.close()
+            self._sock = None
+
+    def __del__(self):
+        try:
+            self.close()
+        except:
+            # close() may fail if __init__ didn't complete
+            pass
+
+    def flush(self):
+        if self._wbuf:
+            data = "".join(self._wbuf)
+            self._wbuf = []
+            self._wbuf_len = 0
+            buffer_size = max(self._rbufsize, self.default_bufsize)
+            data_size = len(data)
+            write_offset = 0
+            # FIXME apparently this doesn't yet work on jython,
+            # despite our work on memoryview/buffer support
+            view = data # memoryview(data)
+            try:
+                while write_offset < data_size:
+                    chunk = view[write_offset:write_offset+buffer_size]
+                    self._sock.sendall(chunk)
+                    write_offset += buffer_size
+            finally:
+                if write_offset < data_size:
+                    remainder = data[write_offset:]
+                    del view, data  # explicit free
+                    self._wbuf.append(remainder)
+                    self._wbuf_len = len(remainder)
+
+    def fileno(self):
+        return self._sock.fileno()
+
+    def write(self, data):
+        data = str(data) # XXX Should really reject non-string non-buffers
+        if not data:
+            return
+        self._wbuf.append(data)
+        self._wbuf_len += len(data)
+        if (self._wbufsize == 0 or
+            (self._wbufsize == 1 and '\n' in data) or
+            (self._wbufsize > 1 and self._wbuf_len >= self._wbufsize)):
+            self.flush()
+
+    def writelines(self, list):
+        # XXX We could do better here for very long lists
+        # XXX Should really reject non-string non-buffers
+        lines = filter(None, map(str, list))
+        self._wbuf_len += sum(map(len, lines))
+        self._wbuf.extend(lines)
+        if (self._wbufsize <= 1 or
+            self._wbuf_len >= self._wbufsize):
+            self.flush()
+
+    def read(self, size=-1):
+        # Use max, disallow tiny reads in a loop as they are very inefficient.
+        # We never leave read() with any leftover data from a new recv() call
+        # in our internal buffer.
+        rbufsize = max(self._rbufsize, self.default_bufsize)
+        # Our use of StringIO rather than lists of string objects returned by
+        # recv() minimizes memory usage and fragmentation that occurs when
+        # rbufsize is large compared to the typical return value of recv().
+        buf = self._rbuf
+        buf.seek(0, 2)  # seek end
+        if size < 0:
+            # Read until EOF
+            self._rbuf = StringIO()  # reset _rbuf.  we consume it via buf.
+            while True:
+                try:
+                    data = self._sock.recv(rbufsize)
+                except error, e:
+                    if e.args[0] == errno.EINTR:
+                        continue
+                    raise
+                if not data:
+                    break
+                buf.write(data)
+            return buf.getvalue()
+        else:
+            # Read until size bytes or EOF seen, whichever comes first
+            buf_len = buf.tell()
+            if buf_len >= size:
+                # Already have size bytes in our buffer?  Extract and return.
+                buf.seek(0)
+                rv = buf.read(size)
+                self._rbuf = StringIO()
+                self._rbuf.write(buf.read())
+                return rv
+
+            self._rbuf = StringIO()  # reset _rbuf.  we consume it via buf.
+            while True:
+                left = size - buf_len
+                # recv() will malloc the amount of memory given as its
+                # parameter even though it often returns much less data
+                # than that.  The returned data string is short lived
+                # as we copy it into a StringIO and free it.  This avoids
+                # fragmentation issues on many platforms.
+                try:
+                    data = self._sock.recv(left)
+                except error, e:
+                    if e.args[0] == errno.EINTR:
+                        continue
+                    raise
+                if not data:
+                    break
+                n = len(data)
+                if n == size and not buf_len:
+                    # Shortcut.  Avoid buffer data copies when:
+                    # - We have no data in our buffer.
+                    # AND
+                    # - Our call to recv returned exactly the
+                    #   number of bytes we were asked to read.
+                    return data
+                if n == left:
+                    buf.write(data)
+                    del data  # explicit free
+                    break
+                assert n <= left, "recv(%d) returned %d bytes" % (left, n)
+                buf.write(data)
+                buf_len += n
+                del data  # explicit free
+                #assert buf_len == buf.tell()
+            return buf.getvalue()
+
+    def readline(self, size=-1):
+        buf = self._rbuf
+        buf.seek(0, 2)  # seek end
+        if buf.tell() > 0:
+            # check if we already have it in our buffer
+            buf.seek(0)
+            bline = buf.readline(size)
+            if bline.endswith('\n') or len(bline) == size:
+                self._rbuf = StringIO()
+                self._rbuf.write(buf.read())
+                return bline
+            del bline
+        if size < 0:
+            # Read until \n or EOF, whichever comes first
+            if self._rbufsize <= 1:
+                # Speed up unbuffered case
+                buf.seek(0)
+                buffers = [buf.read()]
+                self._rbuf = StringIO()  # reset _rbuf.  we consume it via buf.
+                data = None
+                recv = self._sock.recv
+                while True:
+                    try:
+                        while data != "\n":
+                            data = recv(1)
+                            if not data:
+                                break
+                            buffers.append(data)
+                    except error, e:
+                        # The try..except to catch EINTR was moved outside the
+                        # recv loop to avoid the per byte overhead.
+                        if e.args[0] == errno.EINTR:
+                            continue
+                        raise
+                    break
+                return "".join(buffers)
+
+            buf.seek(0, 2)  # seek end
+            self._rbuf = StringIO()  # reset _rbuf.  we consume it via buf.
+            while True:
+                try:
+                    data = self._sock.recv(self._rbufsize)
+                except error, e:
+                    if e.args[0] == errno.EINTR:
+                        continue
+                    raise
+                if not data:
+                    break
+                nl = data.find('\n')
+                if nl >= 0:
+                    nl += 1
+                    buf.write(data[:nl])
+                    self._rbuf.write(data[nl:])
+                    del data
+                    break
+                buf.write(data)
+            return buf.getvalue()
+        else:
+            # Read until size bytes or \n or EOF seen, whichever comes first
+            buf.seek(0, 2)  # seek end
+            buf_len = buf.tell()
+            if buf_len >= size:
+                buf.seek(0)
+                rv = buf.read(size)
+                self._rbuf = StringIO()
+                self._rbuf.write(buf.read())
+                return rv
+            self._rbuf = StringIO()  # reset _rbuf.  we consume it via buf.
+            while True:
+                try:
+                    data = self._sock.recv(self._rbufsize)
+                except error, e:
+                    if e.args[0] == errno.EINTR:
+                        continue
+                    raise
+                if not data:
+                    break
+                left = size - buf_len
+                # did we just receive a newline?
+                nl = data.find('\n', 0, left)
+                if nl >= 0:
+                    nl += 1
+                    # save the excess data to _rbuf
+                    self._rbuf.write(data[nl:])
+                    if buf_len:
+                        buf.write(data[:nl])
+                        break
+                    else:
+                        # Shortcut.  Avoid data copy through buf when returning
+                        # a substring of our first recv().
+                        return data[:nl]
+                n = len(data)
+                if n == size and not buf_len:
+                    # Shortcut.  Avoid data copy through buf when
+                    # returning exactly all of our first recv().
+                    return data
+                if n >= left:
+                    buf.write(data[:left])
+                    self._rbuf.write(data[left:])
+                    break
+                buf.write(data)
+                buf_len += n
+                #assert buf_len == buf.tell()
+            return buf.getvalue()
+
+    def readlines(self, sizehint=0):
+        total = 0
+        list = []
+        while True:
+            line = self.readline()
+            if not line:
+                break
+            list.append(line)
+            total += len(line)
+            if sizehint and total >= sizehint:
+                break
+        return list
+
+    # Iterator protocols
+
+    def __iter__(self):
+        return self
+
+    def next(self):
+        line = self.readline()
+        if not line:
+            raise StopIteration
+        return line
diff --git a/Lib/_sslcerts.py b/Lib/_sslcerts.py
new file mode 100644
--- /dev/null
+++ b/Lib/_sslcerts.py
@@ -0,0 +1,239 @@
+import logging
+import sys
+import uuid
+from array import array
+from contextlib import closing
+from StringIO import StringIO
+
+from java.io import BufferedInputStream, BufferedReader, FileReader, InputStreamReader, ByteArrayInputStream
+from java.security import KeyStore, Security
+from java.security.cert import CertificateException, CertificateFactory
+from javax.net.ssl import (
+    X509KeyManager, X509TrustManager, KeyManagerFactory, SSLContext, TrustManager, TrustManagerFactory)
+
+try:
+    # jarjar-ed version
+    from org.python.bouncycastle.asn1.pkcs import PrivateKeyInfo
+    from org.python.bouncycastle.cert import X509CertificateHolder
+    from org.python.bouncycastle.cert.jcajce import JcaX509CertificateConverter
+    from org.python.bouncycastle.jce.provider import BouncyCastleProvider
+    from org.python.bouncycastle.openssl import PEMKeyPair, PEMParser
+    from org.python.bouncycastle.openssl.jcajce import JcaPEMKeyConverter
+except ImportError:
+    # dev version from extlibs
+    from org.bouncycastle.asn1.pkcs import PrivateKeyInfo
+    from org.bouncycastle.cert import X509CertificateHolder
+    from org.bouncycastle.cert.jcajce import JcaX509CertificateConverter
+    from org.bouncycastle.jce.provider import BouncyCastleProvider
+    from org.bouncycastle.openssl import PEMKeyPair, PEMParser
+    from org.bouncycastle.openssl.jcajce import JcaPEMKeyConverter
+
+log = logging.getLogger("ssl")
+
+
+# FIXME what happens if reloaded?
+Security.addProvider(BouncyCastleProvider())
+
+
+# build the necessary certificate with a CertificateFactory; this can take the pem format:
+# http://docs.oracle.com/javase/7/docs/api/java/security/cert/CertificateFactory.html#generateCertificate(java.io.InputStream)
+
+# not certain if we can include a private key in the pem file; see 
+# http://stackoverflow.com/questions/7216969/getting-rsa-private-key-from-pem-base64-encoded-private-key-file
+
+
+# helpful advice for being able to manage ca_certs outside of Java's keystore
+# specifically the example ReloadableX509TrustManager
+# http://jcalcote.wordpress.com/2010/06/22/managing-a-dynamic-java-trust-store/
+
+# in the case of http://docs.python.org/2/library/ssl.html#ssl.CERT_REQUIRED
+
+# http://docs.python.org/2/library/ssl.html#ssl.CERT_NONE
+# https://github.com/rackerlabs/romper/blob/master/romper/trust.py#L15
+#
+# it looks like CERT_OPTIONAL simply validates certificates if
+# provided, probably something in checkServerTrusted - maybe a None
+# arg? need to verify as usual with a real system... :)
+
+# http://alesaudate.wordpress.com/2010/08/09/how-to-dynamically-select-a-certificate-alias-when-invoking-web-services/
+# is somewhat relevant for managing the keyfile, certfile
+
+
+def _get_ca_certs_trust_manager(ca_certs):
+    trust_store = KeyStore.getInstance(KeyStore.getDefaultType())
+    trust_store.load(None, None)
+    num_certs_installed = 0
+    with open(ca_certs) as f:
+        cf = CertificateFactory.getInstance("X.509")
+        for cert in cf.generateCertificates(BufferedInputStream(f)):
+            trust_store.setCertificateEntry(str(uuid.uuid4()), cert)
+            num_certs_installed += 1
+    tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm())
+    tmf.init(trust_store)
+    log.debug("Installed %s certificates", num_certs_installed, extra={"sock": "*"})
+    return tmf
+
+
+def _stringio_as_reader(s):
+    return BufferedReader(InputStreamReader(ByteArrayInputStream(bytearray(s.getvalue()))))
+
+
+def _extract_readers(cert_file):
+    private_key = StringIO()
+    certs = StringIO()
+    output = certs
+    with open(cert_file) as f:
+        for line in f:
+            if line.startswith("-----BEGIN PRIVATE KEY-----"):
+                output = private_key
+            output.write(line)
+            if line.startswith("-----END PRIVATE KEY-----"):
+                output = certs
+    return _stringio_as_reader(private_key), _stringio_as_reader(certs)
+
+
+def _get_openssl_key_manager(cert_file, key_file=None):
+    paths = [key_file] if key_file else []
+    paths.append(cert_file)
+
+    # Go from Bouncy Castle API to Java's; a bit heavyweight for the Python dev ;)
+    key_converter = JcaPEMKeyConverter().setProvider("BC")
+    cert_converter = JcaX509CertificateConverter().setProvider("BC")
+
+    private_key = None
+    certs = []
+    for path in paths:
+        for br in _extract_readers(path):
+            while True:
+                obj = PEMParser(br).readObject()
+                if obj is None:
+                    break
+                if isinstance(obj, PEMKeyPair):
+                    private_key = key_converter.getKeyPair(obj).getPrivate()
+                elif isinstance(obj, PrivateKeyInfo):
+                    private_key = key_converter.getPrivateKey(obj)
+                elif isinstance(obj, X509CertificateHolder):
+                    certs.append(cert_converter.getCertificate(obj))
+
+    assert private_key, "No private key loaded"
+    key_store = KeyStore.getInstance(KeyStore.getDefaultType())
+    key_store.load(None, None)
+    key_store.setKeyEntry(str(uuid.uuid4()), private_key, [], certs)
+    kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm())
+    kmf.init(key_store, [])
+    return kmf
+
+
+def _get_ssl_context(keyfile, certfile, ca_certs):
+    if certfile is None and ca_certs is None:
+        log.debug("Using default SSL context", extra={"sock": "*"})
+        return SSLContext.getDefault()
+    else:
+        log.debug("Setting up a specific SSL context for keyfile=%s, certfile=%s, ca_certs=%s",
+                  keyfile, certfile, ca_certs, extra={"sock": "*"})
+        if ca_certs:
+            # should support composite usage below
+            trust_managers = _get_ca_certs_trust_manager(ca_certs).getTrustManagers()
+        else:
+            trust_managers = None
+        if certfile:
+            key_managers = _get_openssl_key_manager(certfile, keyfile).getKeyManagers()
+        else:
+            key_managers = None
+
+        # FIXME FIXME for performance, cache this lookup in the future
+        # to avoid re-reading files on every lookup
+        context = SSLContext.getInstance("SSL")
+        context.init(key_managers, trust_managers, None)
+        return context
+
+
+# CompositeX509KeyManager and CompositeX509TrustManager allow for mixing together Java built-in managers
+# with new managers to support Python ssl.
+#
+# See http://tersesystems.com/2014/01/13/fixing-the-most-dangerous-code-in-the-world/
+# for a good description of this composite approach.
+#
+# Ported to Python from http://codyaray.com/2013/04/java-ssl-with-multiple-keystores
+# which was inspired by http://stackoverflow.com/questions/1793979/registering-multiple-keystores-in-jvm
+
+class CompositeX509KeyManager(X509KeyManager):
+                                                   
+    def __init__(self, key_managers):
+        self.key_managers = key_managers
+
+    def chooseClientAlias(self, key_type, issuers, socket):
+        for key_manager in self.key_managers:
+            alias = key_manager.chooseClientAlias(key_type, issuers, socket)
+            if alias:
+                return alias;
+        return None
+
+    def chooseServerAlias(self, key_type, issuers, socket):
+        for key_manager in self.key_managers:
+            alias = key_manager.chooseServerAlias(key_type, issuers, socket)
+            if alias:
+                return alias;
+        return None
+    
+    def getPrivateKey(self, alias):
+        for key_manager in self.key_managers:
+            private_key = keyManager.getPrivateKey(alias)
+            if private_key:
+                return private_key
+        return None
+
+    def getCertificateChain(self, alias):
+        for key_manager in self.key_managers:
+            chain = key_manager.getCertificateChain(alias)
+            if chain:
+                return chain
+        return None
+
+    def getClientAliases(self, key_type, issuers):
+        aliases = []
+        for key_manager in self.key_managers:
+            aliases.extend(key_manager.getClientAliases(key_type, issuers))
+        if not aliases:
+            return None
+        else:
+            return aliases
+
+    def getServerAliases(self, key_type, issuers):
+        aliases = []
+        for key_manager in self.key_managers:
+            aliases.extend(key_manager.getServerAliases(key_type, issuers))
+        if not aliases:
+            return None
+        else:
+            return aliases
+
+
+class CompositeX509TrustManager(X509TrustManager):
+
+    def __init__(self, trust_managers):
+        self.trust_managers = trust_managers
+
+    def checkClientTrusted(self, chain, auth_type):
+        for trust_manager in self.trust_managers:
+            try:
+                trustManager.checkClientTrusted(chain, auth_type);
+                return
+            except CertificateException:
+                pass
+        raise CertificateException("None of the TrustManagers trust this certificate chain")
+
+    def checkServerTrusted(self, chain, auth_type):
+        for trust_manager in self.trust_managers:
+            try:
+                trustManager.checkServerTrusted(chain, auth_type);
+                return
+            except CertificateException:
+                pass
+        raise CertificateException("None of the TrustManagers trust this certificate chain")
+
+    def getAcceptedIssuers(self):
+        certs = []
+        for trust_manager in self.trust_managers:
+            certs.extend(trustManager.getAcceptedIssuers())
+        return certs
diff --git a/Lib/asynchat.py b/Lib/asynchat.py
deleted file mode 100644
--- a/Lib/asynchat.py
+++ /dev/null
@@ -1,295 +0,0 @@
-# -*- Mode: Python; tab-width: 4 -*-
-#       Id: asynchat.py,v 2.26 2000/09/07 22:29:26 rushing Exp
-#       Author: Sam Rushing <rushing at nightmare.com>
-
-# ======================================================================
-# Copyright 1996 by Sam Rushing
-#
-#                         All Rights Reserved
-#
-# Permission to use, copy, modify, and distribute this software and
-# its documentation for any purpose and without fee is hereby
-# granted, provided that the above copyright notice appear in all
-# copies and that both that copyright notice and this permission
-# notice appear in supporting documentation, and that the name of Sam
-# Rushing not be used in advertising or publicity pertaining to
-# distribution of the software without specific, written prior
-# permission.
-#
-# SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
-# INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN
-# NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR
-# CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
-# OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
-# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
-# CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-# ======================================================================
-
-r"""A class supporting chat-style (command/response) protocols.
-
-This class adds support for 'chat' style protocols - where one side
-sends a 'command', and the other sends a response (examples would be
-the common internet protocols - smtp, nntp, ftp, etc..).
-
-The handle_read() method looks at the input stream for the current
-'terminator' (usually '\r\n' for single-line responses, '\r\n.\r\n'
-for multi-line output), calling self.found_terminator() on its
-receipt.
-
-for example:
-Say you build an async nntp client using this class.  At the start
-of the connection, you'll have self.terminator set to '\r\n', in
-order to process the single-line greeting.  Just before issuing a
-'LIST' command you'll set it to '\r\n.\r\n'.  The output of the LIST
-command will be accumulated (using your own 'collect_incoming_data'
-method) up to the terminator, and then control will be returned to
-you - by calling your self.found_terminator() method.
-"""
-
-import socket
-import asyncore
-from collections import deque
-
-class async_chat (asyncore.dispatcher):
-    """This is an abstract class.  You must derive from this class, and add
-    the two methods collect_incoming_data() and found_terminator()"""
-
-    # these are overridable defaults
-
-    ac_in_buffer_size       = 4096
-    ac_out_buffer_size      = 4096
-
-    def __init__ (self, conn=None):
-        self.ac_in_buffer = ''
-        self.ac_out_buffer = ''
-        self.producer_fifo = fifo()
-        asyncore.dispatcher.__init__ (self, conn)
-
-    def collect_incoming_data(self, data):
-        raise NotImplementedError, "must be implemented in subclass"
-
-    def found_terminator(self):
-        raise NotImplementedError, "must be implemented in subclass"
-
-    def set_terminator (self, term):
-        "Set the input delimiter.  Can be a fixed string of any length, an integer, or None"
-        self.terminator = term
-
-    def get_terminator (self):
-        return self.terminator
-
-    # grab some more data from the socket,
-    # throw it to the collector method,
-    # check for the terminator,
-    # if found, transition to the next state.
-
-    def handle_read (self):
-
-        try:
-            data = self.recv (self.ac_in_buffer_size)
-        except socket.error, why:
-            self.handle_error()
-            return
-
-        self.ac_in_buffer = self.ac_in_buffer + data
-
-        # Continue to search for self.terminator in self.ac_in_buffer,
-        # while calling self.collect_incoming_data.  The while loop
-        # is necessary because we might read several data+terminator
-        # combos with a single recv(1024).
-
-        while self.ac_in_buffer:
-            lb = len(self.ac_in_buffer)
-            terminator = self.get_terminator()
-            if not terminator:
-                # no terminator, collect it all
-                self.collect_incoming_data (self.ac_in_buffer)
-                self.ac_in_buffer = ''
-            elif isinstance(terminator, int) or isinstance(terminator, long):
-                # numeric terminator
-                n = terminator
-                if lb < n:
-                    self.collect_incoming_data (self.ac_in_buffer)
-                    self.ac_in_buffer = ''
-                    self.terminator = self.terminator - lb
-                else:
-                    self.collect_incoming_data (self.ac_in_buffer[:n])
-                    self.ac_in_buffer = self.ac_in_buffer[n:]
-                    self.terminator = 0
-                    self.found_terminator()
-            else:
-                # 3 cases:
-                # 1) end of buffer matches terminator exactly:
-                #    collect data, transition
-                # 2) end of buffer matches some prefix:
-                #    collect data to the prefix
-                # 3) end of buffer does not match any prefix:
-                #    collect data
-                terminator_len = len(terminator)
-                index = self.ac_in_buffer.find(terminator)
-                if index != -1:
-                    # we found the terminator
-                    if index > 0:
-                        # don't bother reporting the empty string (source of subtle bugs)
-                        self.collect_incoming_data (self.ac_in_buffer[:index])
-                    self.ac_in_buffer = self.ac_in_buffer[index+terminator_len:]
-                    # This does the Right Thing if the terminator is changed here.
-                    self.found_terminator()
-                else:
-                    # check for a prefix of the terminator
-                    index = find_prefix_at_end (self.ac_in_buffer, terminator)
-                    if index:
-                        if index != lb:
-                            # we found a prefix, collect up to the prefix
-                            self.collect_incoming_data (self.ac_in_buffer[:-index])
-                            self.ac_in_buffer = self.ac_in_buffer[-index:]
-                        break
-                    else:
-                        # no prefix, collect it all
-                        self.collect_incoming_data (self.ac_in_buffer)
-                        self.ac_in_buffer = ''
-
-    def handle_write (self):
-        self.initiate_send ()
-
-    def handle_close (self):
-        self.close()
-
-    def push (self, data):
-        self.producer_fifo.push (simple_producer (data))
-        self.initiate_send()
-
-    def push_with_producer (self, producer):
-        self.producer_fifo.push (producer)
-        self.initiate_send()
-
-    def readable (self):
-        "predicate for inclusion in the readable for select()"
-        return (len(self.ac_in_buffer) <= self.ac_in_buffer_size)
-
-    def writable (self):
-        "predicate for inclusion in the writable for select()"
-        # return len(self.ac_out_buffer) or len(self.producer_fifo) or (not self.connected)
-        # this is about twice as fast, though not as clear.
-        return not (
-                (self.ac_out_buffer == '') and
-                self.producer_fifo.is_empty() and
-                self.connected
-                )
-
-    def close_when_done (self):
-        "automatically close this channel once the outgoing queue is empty"
-        self.producer_fifo.push (None)
-
-    # refill the outgoing buffer by calling the more() method
-    # of the first producer in the queue
-    def refill_buffer (self):
-        while 1:
-            if len(self.producer_fifo):
-                p = self.producer_fifo.first()
-                # a 'None' in the producer fifo is a sentinel,
-                # telling us to close the channel.
-                if p is None:
-                    if not self.ac_out_buffer:
-                        self.producer_fifo.pop()
-                        self.close()
-                    return
-                elif isinstance(p, str):
-                    self.producer_fifo.pop()
-                    self.ac_out_buffer = self.ac_out_buffer + p
-                    return
-                data = p.more()
-                if data:
-                    self.ac_out_buffer = self.ac_out_buffer + data
-                    return
-                else:
-                    self.producer_fifo.pop()
-            else:
-                return
-
-    def initiate_send (self):
-        obs = self.ac_out_buffer_size
-        # try to refill the buffer
-        if (len (self.ac_out_buffer) < obs):
-            self.refill_buffer()
-
-        if self.ac_out_buffer and self.connected:
-            # try to send the buffer
-            try:
-                num_sent = self.send (self.ac_out_buffer[:obs])
-                if num_sent:
-                    self.ac_out_buffer = self.ac_out_buffer[num_sent:]
-
-            except socket.error, why:
-                self.handle_error()
-                return
-
-    def discard_buffers (self):
-        # Emergencies only!
-        self.ac_in_buffer = ''
-        self.ac_out_buffer = ''
-        while self.producer_fifo:
-            self.producer_fifo.pop()
-
-
-class simple_producer:
-
-    def __init__ (self, data, buffer_size=512):
-        self.data = data
-        self.buffer_size = buffer_size
-
-    def more (self):
-        if len (self.data) > self.buffer_size:
-            result = self.data[:self.buffer_size]
-            self.data = self.data[self.buffer_size:]
-            return result
-        else:
-            result = self.data
-            self.data = ''
-            return result
-
-class fifo:
-    def __init__ (self, list=None):
-        if not list:
-            self.list = deque()
-        else:
-            self.list = deque(list)
-
-    def __len__ (self):
-        return len(self.list)
-
-    def is_empty (self):
-        return not self.list
-
-    def first (self):
-        return self.list[0]
-
-    def push (self, data):
-        self.list.append(data)
-
-    def pop (self):
-        if self.list:
-            return (1, self.list.popleft())
-        else:
-            return (0, None)
-
-# Given 'haystack', see if any prefix of 'needle' is at its end.  This
-# assumes an exact match has already been checked.  Return the number of
-# characters matched.
-# for example:
-# f_p_a_e ("qwerty\r", "\r\n") => 1
-# f_p_a_e ("qwertydkjf", "\r\n") => 0
-# f_p_a_e ("qwerty\r\n", "\r\n") => <undefined>
-
-# this could maybe be made faster with a computed regex?
-# [answer: no; circa Python-2.0, Jan 2001]
-# new python:   28961/s
-# old python:   18307/s
-# re:        12820/s
-# regex:     14035/s
-
-def find_prefix_at_end (haystack, needle):
-    l = len(needle) - 1
-    while l and not haystack.endswith(needle[:l]):
-        l -= 1
-    return l
diff --git a/Lib/asyncore.py b/Lib/asyncore.py
deleted file mode 100644
--- a/Lib/asyncore.py
+++ /dev/null
@@ -1,705 +0,0 @@
-# -*- Mode: Python -*-
-#   Id: asyncore.py,v 2.51 2000/09/07 22:29:26 rushing Exp
-#   Author: Sam Rushing <rushing at nightmare.com>
-
-# ======================================================================
-# Copyright 1996 by Sam Rushing
-#
-#                         All Rights Reserved
-#
-# Permission to use, copy, modify, and distribute this software and
-# its documentation for any purpose and without fee is hereby
-# granted, provided that the above copyright notice appear in all
-# copies and that both that copyright notice and this permission
-# notice appear in supporting documentation, and that the name of Sam
-# Rushing not be used in advertising or publicity pertaining to
-# distribution of the software without specific, written prior
-# permission.
-#
-# SAM RUSHING DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE,
-# INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN
-# NO EVENT SHALL SAM RUSHING BE LIABLE FOR ANY SPECIAL, INDIRECT OR
-# CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
-# OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
-# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
-# CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-# ======================================================================
-
-"""Basic infrastructure for asynchronous socket service clients and servers.
-
-There are only two ways to have a program on a single processor do "more
-than one thing at a time".  Multi-threaded programming is the simplest and
-most popular way to do it, but there is another very different technique,
-that lets you have nearly all the advantages of multi-threading, without
-actually using multiple threads. it's really only practical if your program
-is largely I/O bound. If your program is CPU bound, then pre-emptive
-scheduled threads are probably what you really need. Network servers are
-rarely CPU-bound, however.
-
-If your operating system supports the select() system call in its I/O
-library (and nearly all do), then you can use it to juggle multiple
-communication channels at once; doing other work while your I/O is taking
-place in the "background."  Although this strategy can seem strange and
-complex, especially at first, it is in many ways easier to understand and
-control than multi-threaded programming. The module documented here solves
-many of the difficult problems for you, making the task of building
-sophisticated high-performance network servers and clients a snap.
-"""
-
-import select
-import socket
-import sys
-import time
-import warnings
-
-import os
-from errno import EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET, EINVAL, \
-     ENOTCONN, ESHUTDOWN, EINTR, EISCONN, EBADF, ECONNABORTED, EPIPE, EAGAIN, \
-     errorcode
-
-_DISCONNECTED = frozenset((ECONNRESET, ENOTCONN, ESHUTDOWN, ECONNABORTED, EPIPE,
-                           EBADF))
-
-try:
-    socket_map
-except NameError:
-    socket_map = {}
-
-def _strerror(err):
-    try:
-        return os.strerror(err)
-    except (ValueError, OverflowError, NameError):
-        if err in errorcode:
-            return errorcode[err]
-        return "Unknown error %s" %err
-
-class ExitNow(Exception):
-    pass
-
-_reraised_exceptions = (ExitNow, KeyboardInterrupt, SystemExit)
-
-def read(obj):
-    try:
-        obj.handle_read_event()
-    except _reraised_exceptions:
-        raise
-    except:
-        obj.handle_error()
-
-def write(obj):
-    try:
-        obj.handle_write_event()
-    except _reraised_exceptions:
-        raise
-    except:
-        obj.handle_error()
-
-def _exception(obj):
-    try:
-        obj.handle_expt_event()
-    except _reraised_exceptions:
-        raise
-    except:
-        obj.handle_error()
-
-def readwrite(obj, flags):
-    try:
-        if flags & select.POLLIN:
-            obj.handle_read_event()
-        if flags & select.POLLOUT:
-            obj.handle_write_event()
-        if flags & select.POLLPRI:
-            obj.handle_expt_event()
-        if flags & (select.POLLHUP | select.POLLERR | select.POLLNVAL):
-            obj.handle_close()
-    except socket.error, e:
-        if e.args[0] not in _DISCONNECTED:
-            obj.handle_error()
-        else:
-            obj.handle_close()
-    except _reraised_exceptions:
-        raise
-    except:
-        obj.handle_error()
-
-def poll(timeout=0.0, map=None):
-    if map is None:
-        map = socket_map
-    if map:
-        r = []; w = []; e = []
-        for fd, obj in map.items():
-            is_r = obj.readable()
-            is_w = obj.writable()
-            if is_r:
-                r.append(fd)
-            # accepting sockets should not be writable
-            if is_w and not obj.accepting:
-                w.append(fd)
-            if is_r or is_w:
-                e.append(fd)
-        if [] == r == w == e:
-            time.sleep(timeout)
-            return
-
-        try:
-            r, w, e = select.select(r, w, e, timeout)
-        except select.error, err:
-            if err.args[0] != EINTR:
-                raise
-            else:
-                return
-
-        for fd in r:
-            obj = map.get(fd)
-            if obj is None:
-                continue
-            read(obj)
-
-        for fd in w:
-            obj = map.get(fd)
-            if obj is None:
-                continue
-            write(obj)
-
-        for fd in e:
-            obj = map.get(fd)
-            if obj is None:
-                continue
-            _exception(obj)
-
-def poll2(timeout=0.0, map=None):
-    # Use the poll() support added to the select module in Python 2.0
-    if map is None:
-        map = socket_map
-    if timeout is not None:
-        # timeout is in milliseconds
-        timeout = int(timeout*1000)
-    pollster = select.poll()
-    if map:
-        for fd, obj in map.items():
-            flags = 0
-            if obj.readable():
-                flags |= select.POLLIN | select.POLLPRI
-            # accepting sockets should not be writable
-            if obj.writable() and not obj.accepting:
-                flags |= select.POLLOUT
-            if flags:
-                # Only check for exceptions if object was either readable
-                # or writable.
-                flags |= select.POLLERR | select.POLLHUP | select.POLLNVAL
-                pollster.register(fd, flags)
-        try:
-            r = pollster.poll(timeout)
-        except select.error, err:
-            if err.args[0] != EINTR:
-                raise
-            r = []
-        for fd, flags in r:
-            obj = map.get(fd)
-            if obj is None:
-                continue
-            readwrite(obj, flags)
-
-poll3 = poll2                           # Alias for backward compatibility
-
-def jython_poll_fun(timeout=0.0, map=None):
-    # On jython, select.poll() is the mechanism to use,
-    # select.select is implemented on top of it.
-    # Also, we have to use a cache of such objects, because of problems with frequent 
-    # creation and destruction  of such objects on windows
-    # "select() crashes with IOException": http://bugs.jython.org/issue1291
-    # So this function is basically the same function as poll2 above, except
-    # with the select.poll() functionality wrapped in a try..finally clause.
-    if map is None:
-        map = socket_map
-    if timeout is not None:
-        # timeout is in milliseconds
-        timeout = int(timeout*1000)
-    if map:
-        try:
-            pollster = select._poll_object_cache.get_poll_object()
-            for fd, obj in map.items():
-                flags = 0
-                if obj.readable():
-                    flags |= select.POLLIN | select.POLLPRI
-                # accepting sockets should not be writable
-                if obj.writable() and not obj.accepting:
-                    flags |= select.POLLOUT
-                if flags:
-                    # Only check for exceptions if object was either readable
-                    # or writable.
-                    flags |= select.POLLERR | select.POLLHUP | select.POLLNVAL
-                    pollster.register(obj, flags)
-            try:
-                r = pollster.poll(timeout)
-            except select.error, err:
-                if err.args[0] != EINTR:
-                    raise
-                r = []
-            for obj, flags in r:
-                # obj = map.get(fd)
-                if obj is None:
-                    continue
-                readwrite(obj, flags)
-        finally:
-            select._poll_object_cache.release_poll_object(pollster)
-
-def loop(timeout=30.0, use_poll=False, map=None, count=None):
-    if map is None:
-        map = socket_map
-
-    if use_poll and hasattr(select, 'poll'):
-        poll_fun = poll2
-    else:
-        poll_fun = poll
-    if sys.platform.startswith('java'):
-        poll_fun = jython_poll_fun
-
-    if count is None:
-        while map:
-            poll_fun(timeout, map)
-
-    else:
-        while map and count > 0:
-            poll_fun(timeout, map)
-            count = count - 1
-
-class dispatcher:
-
-    debug = False
-    connected = False
-    accepting = False
-    connecting = False
-    closing = False
-    addr = None
-    ignore_log_types = frozenset(['warning'])
-
-    def __init__(self, sock=None, map=None):
-        if map is None:
-            self._map = socket_map
-        else:
-            self._map = map
-
-        self._fileno = None
-
-        if sock:
-            # Set to nonblocking just to make sure for cases where we
-            # get a socket from a blocking source.
-            sock.setblocking(0)
-            self.set_socket(sock, map)
-            self.connected = True
-            # The constructor no longer requires that the socket
-            # passed be connected.
-            try:
-                self.addr = sock.getpeername()
-            except socket.error, err:
-                if err.args[0] in (ENOTCONN, EINVAL):
-                    # To handle the case where we got an unconnected
-                    # socket.
-                    self.connected = False
-                else:
-                    # The socket is broken in some unknown way, alert
-                    # the user and remove it from the map (to prevent
-                    # polling of broken sockets).
-                    self.del_channel(map)
-                    raise
-        else:
-            self.socket = None
-
-    def __repr__(self):
-        status = [self.__class__.__module__+"."+self.__class__.__name__]
-        if self.accepting and self.addr:
-            status.append('listening')
-        elif self.connected:
-            status.append('connected')
-        if self.addr is not None:
-            try:
-                status.append('%s:%d' % self.addr)
-            except TypeError:
-                status.append(repr(self.addr))
-        return '<%s at %#x>' % (' '.join(status), id(self))
-
-    __str__ = __repr__
-
-    def add_channel(self, map=None):
-        #self.log_info('adding channel %s' % self)
-        if map is None:
-            map = self._map
-        map[self._fileno] = self
-
-    def del_channel(self, map=None):
-        fd = self._fileno
-        if map is None:
-            map = self._map
-        if fd in map:
-            #self.log_info('closing channel %d:%s' % (fd, self))
-            del map[fd]
-        self._fileno = None
-
-    def create_socket(self, family, type):
-        self.family_and_type = family, type
-        sock = socket.socket(family, type)
-        sock.setblocking(0)
-        self.set_socket(sock)
-
-    def set_socket(self, sock, map=None):
-        self.socket = sock
-##        self.__dict__['socket'] = sock
-        # On jython, the socket object itself is what is watchable.
-        # http://mail.python.org/pipermail/python-dev/2007-May/073443.html
-        self._fileno = sock
-        self.add_channel(map)
-
-    def set_reuse_addr(self):
-        # try to re-use a server port if possible
-        try:
-            self.socket.setsockopt(
-                socket.SOL_SOCKET, socket.SO_REUSEADDR,
-                self.socket.getsockopt(socket.SOL_SOCKET,
-                                       socket.SO_REUSEADDR) | 1
-                )
-        except socket.error:
-            pass
-
-    # ==================================================
-    # predicates for select()
-    # these are used as filters for the lists of sockets
-    # to pass to select().
-    # ==================================================
-
-    def readable(self):
-        return True
-
-    def writable(self):
-        return True
-
-    # ==================================================
-    # socket object methods.
-    # ==================================================
-
-    def listen(self, num):
-        self.accepting = True
-        if os.name == 'nt' and num > 5:
-            num = 5
-        return self.socket.listen(num)
-
-    def bind(self, addr):
-        self.addr = addr
-        return self.socket.bind(addr)
-
-    def connect(self, address):
-        self.connected = False
-        self.connecting = True
-        err = self.socket.connect_ex(address)
-        if err in (EINPROGRESS, EALREADY, EWOULDBLOCK) \
-        or err == EINVAL and os.name in ('nt', 'ce'):
-            self.addr = address
-            return
-        if err in (0, EISCONN):
-            self.addr = address
-            self.handle_connect_event()
-        else:
-            raise socket.error(err, errorcode[err])
-
-    def accept(self):
-        # XXX can return either an address pair or None
-        try:
-            conn, addr = self.socket.accept()
-        except TypeError:
-            return None
-        except socket.error as why:
-            if why.args[0] in (EWOULDBLOCK, ECONNABORTED, EAGAIN):
-                return None
-            else:
-                raise
-        else:
-            return conn, addr
-
-    def send(self, data):
-        try:
-            result = self.socket.send(data)
-            return result
-        except socket.error, why:
-            if why.args[0] == EWOULDBLOCK:
-                return 0
-            elif why.args[0] in _DISCONNECTED:
-                self.handle_close()
-                return 0
-            else:
-                raise
-
-    def recv(self, buffer_size):
-        try:
-            data = self.socket.recv(buffer_size)
-            if not data:
-                # a closed connection is indicated by signaling
-                # a read condition, and having recv() return 0.
-                self.handle_close()
-                return ''
-            else:
-                return data
-        except socket.error, why:
-            # winsock sometimes throws ENOTCONN
-            if why.args[0] in _DISCONNECTED:
-                self.handle_close()
-                return ''
-            else:
-                raise
-
-    def close(self):
-        self.connected = False
-        self.accepting = False
-        self.connecting = False
-        self.del_channel()
-        try:
-            self.socket.close()
-        except socket.error, why:
-            if why.args[0] not in (ENOTCONN, EBADF):
-                raise
-
-    # cheap inheritance, used to pass all other attribute
-    # references to the underlying socket object.
-    def __getattr__(self, attr):
-        try:
-            retattr = getattr(self.socket, attr)
-        except AttributeError:
-            raise AttributeError("%s instance has no attribute '%s'"
-                                 %(self.__class__.__name__, attr))
-        else:
-            msg = "%(me)s.%(attr)s is deprecated. Use %(me)s.socket.%(attr)s " \
-                  "instead." % {'me': self.__class__.__name__, 'attr':attr}
-            warnings.warn(msg, DeprecationWarning, stacklevel=2)
-            return retattr
-
-    # log and log_info may be overridden to provide more sophisticated
-    # logging and warning methods. In general, log is for 'hit' logging
-    # and 'log_info' is for informational, warning and error logging.
-
-    def log(self, message):
-        sys.stderr.write('log: %s\n' % str(message))
-
-    def log_info(self, message, type='info'):
-        if type not in self.ignore_log_types:
-            print '%s: %s' % (type, message)
-
-    def handle_read_event(self):
-        if self.accepting:
-            # accepting sockets are never connected, they "spawn" new
-            # sockets that are connected
-            self.handle_accept()
-        elif not self.connected:
-            if self.connecting:
-                self.handle_connect_event()
-            self.handle_read()
-        else:
-            self.handle_read()
-
-    def handle_connect_event(self):
-        err = self.socket.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR)
-        if err != 0:
-            raise socket.error(err, _strerror(err))
-        self.handle_connect()
-        self.connected = True
-        self.connecting = False
-
-    def handle_write_event(self):
-        if self.accepting:
-            # Accepting sockets shouldn't get a write event.
-            # We will pretend it didn't happen.
-            return
-
-        if not self.connected:
-            if self.connecting:
-                self.handle_connect_event()
-        self.handle_write()
-
-    def handle_expt_event(self):
-        # handle_expt_event() is called if there might be an error on the
-        # socket, or if there is OOB data
-        # check for the error condition first
-        err = self.socket.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR)
-        if err != 0:
-            # we can get here when select.select() says that there is an
-            # exceptional condition on the socket
-            # since there is an error, we'll go ahead and close the socket
-            # like we would in a subclassed handle_read() that received no
-            # data
-            self.handle_close()
-        else:
-            self.handle_expt()
-
-    def handle_error(self):
-        nil, t, v, tbinfo = compact_traceback()
-
-        # sometimes a user repr method will crash.
-        try:
-            self_repr = repr(self)
-        except:
-            self_repr = '<__repr__(self) failed for object at %0x>' % id(self)
-
-        self.log_info(
-            'uncaptured python exception, closing channel %s (%s:%s %s)' % (
-                self_repr,
-                t,
-                v,
-                tbinfo
-                ),
-            'error'
-            )
-        self.handle_close()
-
-    def handle_expt(self):
-        self.log_info('unhandled incoming priority event', 'warning')
-
-    def handle_read(self):
-        self.log_info('unhandled read event', 'warning')
-
-    def handle_write(self):
-        self.log_info('unhandled write event', 'warning')
-
-    def handle_connect(self):
-        self.log_info('unhandled connect event', 'warning')
-
-    def handle_accept(self):
-        self.log_info('unhandled accept event', 'warning')
-
-    def handle_close(self):
-        self.log_info('unhandled close event', 'warning')
-        self.close()
-
-# ---------------------------------------------------------------------------
-# adds simple buffered output capability, useful for simple clients.
-# [for more sophisticated usage use asynchat.async_chat]
-# ---------------------------------------------------------------------------
-
-class dispatcher_with_send(dispatcher):
-
-    def __init__(self, sock=None, map=None):
-        dispatcher.__init__(self, sock, map)
-        self.out_buffer = ''
-
-    def initiate_send(self):
-        num_sent = 0
-        num_sent = dispatcher.send(self, self.out_buffer[:512])
-        self.out_buffer = self.out_buffer[num_sent:]
-
-    def handle_write(self):
-        self.initiate_send()
-
-    def writable(self):
-        return (not self.connected) or len(self.out_buffer)
-
-    def send(self, data):
-        if self.debug:
-            self.log_info('sending %s' % repr(data))
-        self.out_buffer = self.out_buffer + data
-        self.initiate_send()
-
-# ---------------------------------------------------------------------------
-# used for debugging.
-# ---------------------------------------------------------------------------
-
-def compact_traceback():
-    t, v, tb = sys.exc_info()
-    tbinfo = []
-    if not tb: # Must have a traceback
-        raise AssertionError("traceback does not exist")
-    while tb:
-        tbinfo.append((
-            tb.tb_frame.f_code.co_filename,
-            tb.tb_frame.f_code.co_name,
-            str(tb.tb_lineno)
-            ))
-        tb = tb.tb_next
-
-    # just to be safe
-    del tb
-
-    file, function, line = tbinfo[-1]
-    info = ' '.join(['[%s|%s|%s]' % x for x in tbinfo])
-    return (file, function, line), t, v, info
-
-def close_all(map=None, ignore_all=False):
-    if map is None:
-        map = socket_map
-    for x in map.values():
-        try:
-            x.close()
-        except OSError, x:
-            if x.args[0] == EBADF:
-                pass
-            elif not ignore_all:
-                raise
-        except _reraised_exceptions:
-            raise
-        except:
-            if not ignore_all:
-                raise
-    map.clear()
-
-# Asynchronous File I/O:
-#
-# After a little research (reading man pages on various unixen, and
-# digging through the linux kernel), I've determined that select()
-# isn't meant for doing asynchronous file i/o.
-# Heartening, though - reading linux/mm/filemap.c shows that linux
-# supports asynchronous read-ahead.  So _MOST_ of the time, the data
-# will be sitting in memory for us already when we go to read it.
-#
-# What other OS's (besides NT) support async file i/o?  [VMS?]
-#
-# Regardless, this is useful for pipes, and stdin/stdout...
-
-if os.name == 'posix':
-    import fcntl
-
-    class file_wrapper:
-        # Here we override just enough to make a file
-        # look like a socket for the purposes of asyncore.
-        # The passed fd is automatically os.dup()'d
-
-        def __init__(self, fd):
-            self.fd = os.dup(fd)
-
-        def recv(self, *args):
-            return os.read(self.fd, *args)
-
-        def send(self, *args):
-            return os.write(self.fd, *args)
-
-        def getsockopt(self, level, optname, buflen=None):
-            if (level == socket.SOL_SOCKET and
-                optname == socket.SO_ERROR and
-                not buflen):
-                return 0
-            raise NotImplementedError("Only asyncore specific behaviour "
-                                      "implemented.")
-
-        read = recv
-        write = send
-
-        def close(self):
-            os.close(self.fd)
-
-        def fileno(self):
-            return self.fd
-
-    class file_dispatcher(dispatcher):
-
-        def __init__(self, fd, map=None):
-            dispatcher.__init__(self, None, map)
-            self.connected = True
-            try:
-                fd = fd.fileno()
-            except AttributeError:
-                pass
-            self.set_file(fd)
-            # set it to non-blocking mode
-            flags = fcntl.fcntl(fd, fcntl.F_GETFL, 0)
-            flags = flags | os.O_NONBLOCK
-            fcntl.fcntl(fd, fcntl.F_SETFL, flags)
-
-        def set_file(self, fd):
-            self.socket = file_wrapper(fd)
-            self._fileno = self.socket.fileno()
-            self.add_channel()
diff --git a/Lib/distutils/command/install.py b/Lib/distutils/command/install.py
--- a/Lib/distutils/command/install.py
+++ b/Lib/distutils/command/install.py
@@ -6,7 +6,7 @@
 
 # This module should be kept compatible with Python 2.1.
 
-__revision__ = "$Id: install.py 43363 2006-03-27 21:55:21Z phillip.eby $"
+__revision__ = "$Id$"
 
 import sys, os, string
 from types import *
@@ -16,8 +16,11 @@
 from distutils.errors import DistutilsPlatformError
 from distutils.file_util import write_file
 from distutils.util import convert_path, subst_vars, change_root
+from distutils.util import get_platform
 from distutils.errors import DistutilsOptionError
-from glob import glob
+from site import USER_BASE
+from site import USER_SITE
+
 
 if sys.version < "2.2":
     WINDOWS_SCHEME = {
@@ -51,13 +54,20 @@
         'scripts': '$base/bin',
         'data'   : '$base',
         },
+    'unix_user': {
+        'purelib': '$usersite',
+        'platlib': '$usersite',
+        'headers': '$userbase/include/python$py_version_short/$dist_name',
+        'scripts': '$userbase/bin',
+        'data'   : '$userbase',
+        },
     'nt': WINDOWS_SCHEME,
-    'mac': {
-        'purelib': '$base/Lib/site-packages',
-        'platlib': '$base/Lib/site-packages',
-        'headers': '$base/Include/$dist_name',
-        'scripts': '$base/Scripts',
-        'data'   : '$base',
+    'nt_user': {
+        'purelib': '$usersite',
+        'platlib': '$usersite',
+        'headers': '$userbase/Python$py_version_nodot/Include/$dist_name',
+        'scripts': '$userbase/Scripts',
+        'data'   : '$userbase',
         },
     'os2': {
         'purelib': '$base/Lib/site-packages',
@@ -66,6 +76,13 @@
         'scripts': '$base/Scripts',
         'data'   : '$base',
         },
+    'os2_home': {
+        'purelib': '$usersite',
+        'platlib': '$usersite',
+        'headers': '$userbase/include/python$py_version_short/$dist_name',
+        'scripts': '$userbase/bin',
+        'data'   : '$userbase',
+        },
     'java': {
         'purelib': '$base/Lib/site-packages',
         'platlib': '$base/Lib/site-packages',
@@ -93,6 +110,8 @@
          "(Unix only) prefix for platform-specific files"),
         ('home=', None,
          "(Unix only) home directory to install under"),
+        ('user', None,
+         "install in user site-package '%s'" % USER_SITE),
 
         # Or, just set the base director(y|ies)
         ('install-base=', None,
@@ -144,7 +163,7 @@
          "filename in which to record list of installed files"),
         ]
 
-    boolean_options = ['compile', 'force', 'skip-build']
+    boolean_options = ['compile', 'force', 'skip-build', 'user']
     negative_opt = {'no-compile' : 'compile'}
 
 
@@ -155,6 +174,7 @@
         self.prefix = None
         self.exec_prefix = None
         self.home = None
+        self.user = 0
 
         # These select only the installation base; it's up to the user to
         # specify the installation scheme (currently, that means supplying
@@ -173,6 +193,8 @@
         self.install_lib = None         # set to either purelib or platlib
         self.install_scripts = None
         self.install_data = None
+        self.install_userbase = USER_BASE
+        self.install_usersite = USER_SITE
 
         self.compile = None
         self.optimize = None
@@ -248,6 +270,11 @@
             raise DistutilsOptionError, \
                   "must supply either home or prefix/exec-prefix -- not both"
 
+        if self.user and (self.prefix or self.exec_prefix or self.home or
+                self.install_base or self.install_platbase):
+            raise DistutilsOptionError("can't combine user with with prefix/"
+                                       "exec_prefix/home or install_(plat)base")
+
         # Next, stuff that's wrong (or dubious) only on certain platforms.
         if os.name != "posix":
             if self.exec_prefix:
@@ -283,10 +310,13 @@
                             'dist_fullname': self.distribution.get_fullname(),
                             'py_version': py_version,
                             'py_version_short': py_version[0:3],
+                            'py_version_nodot': py_version[0] + py_version[2],
                             'sys_prefix': prefix,
                             'prefix': prefix,
                             'sys_exec_prefix': exec_prefix,
                             'exec_prefix': exec_prefix,
+                            'userbase': self.install_userbase,
+                            'usersite': self.install_usersite,
                            }
         self.expand_basedirs()
 
@@ -308,6 +338,10 @@
 
         self.dump_dirs("post-expand_dirs()")
 
+        # Create directories in the home dir:
+        if self.user:
+            self.create_home_path()
+
         # Pick the actual directory to install all modules to: either
         # install_purelib or install_platlib, depending on whether this
         # module distribution is pure or not.  Of course, if the user
@@ -322,7 +356,8 @@
         # Convert directories from Unix /-separated syntax to the local
         # convention.
         self.convert_paths('lib', 'purelib', 'platlib',
-                           'scripts', 'data', 'headers')
+                           'scripts', 'data', 'headers',
+                           'userbase', 'usersite')
 
         # Well, we're not actually fully completely finalized yet: we still
         # have to deal with 'extra_path', which is the hack for allowing
@@ -359,7 +394,7 @@
                 opt_name = opt[0]
                 if opt_name[-1] == "=":
                     opt_name = opt_name[0:-1]
-                if self.negative_opt.has_key(opt_name):
+                if opt_name in self.negative_opt:
                     opt_name = string.translate(self.negative_opt[opt_name],
                                                 longopt_xlate)
                     val = not getattr(self, opt_name)
@@ -383,7 +418,13 @@
                       "installation scheme is incomplete")
             return
 
-        if self.home is not None:
+        if self.user:
+            if self.install_userbase is None:
+                raise DistutilsPlatformError(
+                    "User base directory is not specified")
+            self.install_base = self.install_platbase = self.install_userbase
+            self.select_scheme("unix_user")
+        elif self.home is not None:
             self.install_base = self.install_platbase = self.home
             self.select_scheme("unix_home")
         else:
@@ -408,7 +449,13 @@
 
     def finalize_other (self):          # Windows and Mac OS for now
 
-        if self.home is not None:
+        if self.user:
+            if self.install_userbase is None:
+                raise DistutilsPlatformError(
+                    "User base directory is not specified")
+            self.install_base = self.install_platbase = self.install_userbase
+            self.select_scheme(os.name + "_user")
+        elif self.home is not None:
             self.install_base = self.install_platbase = self.home
             self.select_scheme("unix_home")
         else:
@@ -438,7 +485,7 @@
         for attr in attrs:
             val = getattr(self, attr)
             if val is not None:
-                if os.name == 'posix':
+                if os.name == 'posix' or os.name == 'nt':
                     val = os.path.expanduser(val)
                 val = subst_vars(val, self.config_vars)
                 setattr(self, attr, val)
@@ -503,6 +550,16 @@
             attr = "install_" + name
             setattr(self, attr, change_root(self.root, getattr(self, attr)))
 
+    def create_home_path(self):
+        """Create directories under ~
+        """
+        if not self.user:
+            return
+        home = convert_path(os.path.expanduser("~"))
+        for name, path in self.config_vars.iteritems():
+            if path.startswith(home) and not os.path.isdir(path):
+                self.debug_print("os.makedirs('%s', 0700)" % path)
+                os.makedirs(path, 0700)
 
     # -- Command execution methods -------------------------------------
 
@@ -511,6 +568,14 @@
         # Obviously have to build before we can install
         if not self.skip_build:
             self.run_command('build')
+            # If we built for any other platform, we can't install.
+            build_plat = self.distribution.get_command_obj('build').plat_name
+            # check warn_dir - it is a clue that the 'install' is happening
+            # internally, and not to sys.path, so we don't check the platform
+            # matches what we are running.
+            if self.warn_dir and build_plat != get_platform():
+                raise DistutilsPlatformError("Can't install when "
+                                             "cross-compiling")
 
         # Run all sub-commands (at least those that need to be run)
         for cmd_name in self.get_sub_commands():
diff --git a/Lib/ftplib.py b/Lib/ftplib.py
deleted file mode 100644
--- a/Lib/ftplib.py
+++ /dev/null
@@ -1,1047 +0,0 @@
-"""An FTP client class and some helper functions.
-
-Based on RFC 959: File Transfer Protocol (FTP), by J. Postel and J. Reynolds
-
-Example:
-
->>> from ftplib import FTP
->>> ftp = FTP('ftp.python.org') # connect to host, default port
->>> ftp.login() # default, i.e.: user anonymous, passwd anonymous@
-'230 Guest login ok, access restrictions apply.'
->>> ftp.retrlines('LIST') # list directory contents
-total 9
-drwxr-xr-x   8 root     wheel        1024 Jan  3  1994 .
-drwxr-xr-x   8 root     wheel        1024 Jan  3  1994 ..
-drwxr-xr-x   2 root     wheel        1024 Jan  3  1994 bin
-drwxr-xr-x   2 root     wheel        1024 Jan  3  1994 etc
-d-wxrwxr-x   2 ftp      wheel        1024 Sep  5 13:43 incoming
-drwxr-xr-x   2 root     wheel        1024 Nov 17  1993 lib
-drwxr-xr-x   6 1094     wheel        1024 Sep 13 19:07 pub
-drwxr-xr-x   3 root     wheel        1024 Jan  3  1994 usr
--rw-r--r--   1 root     root          312 Aug  1  1994 welcome.msg
-'226 Transfer complete.'
->>> ftp.quit()
-'221 Goodbye.'
->>>
-
-A nice test that reveals some of the network dialogue would be:
-python ftplib.py -d localhost -l -p -l
-"""
-
-#
-# Changes and improvements suggested by Steve Majewski.
-# Modified by Jack to work on the mac.
-# Modified by Siebren to support docstrings and PASV.
-# Modified by Phil Schwartz to add storbinary and storlines callbacks.
-# Modified by Giampaolo Rodola' to add TLS support.
-#
-
-import os
-import sys
-
-# Import SOCKS module if it exists, else standard socket module socket
-try:
-    import SOCKS; socket = SOCKS; del SOCKS # import SOCKS as socket
-    from socket import getfqdn; socket.getfqdn = getfqdn; del getfqdn
-except ImportError:
-    import socket
-from socket import _GLOBAL_DEFAULT_TIMEOUT
-
-__all__ = ["FTP","Netrc"]
-
-# Magic number from <socket.h>
-MSG_OOB = 0x1                           # Process data out of band
-
-
-# The standard FTP server control port
-FTP_PORT = 21
-
-
-# Exception raised when an error or invalid response is received
-class Error(Exception): pass
-class error_reply(Error): pass          # unexpected [123]xx reply
-class error_temp(Error): pass           # 4xx errors
-class error_perm(Error): pass           # 5xx errors
-class error_proto(Error): pass          # response does not begin with [1-5]
-
-
-# All exceptions (hopefully) that may be raised here and that aren't
-# (always) programming errors on our side
-all_errors = (Error, IOError, EOFError)
-
-
-# Line terminators (we always output CRLF, but accept any of CRLF, CR, LF)
-CRLF = '\r\n'
-
-# The class itself
-class FTP:
-
-    '''An FTP client class.
-
-    To create a connection, call the class using these arguments:
-            host, user, passwd, acct, timeout
-
-    The first four arguments are all strings, and have default value ''.
-    timeout must be numeric and defaults to None if not passed,
-    meaning that no timeout will be set on any ftp socket(s)
-    If a timeout is passed, then this is now the default timeout for all ftp
-    socket operations for this instance.
-
-    Then use self.connect() with optional host and port argument.
-
-    To download a file, use ftp.retrlines('RETR ' + filename),
-    or ftp.retrbinary() with slightly different arguments.
-    To upload a file, use ftp.storlines() or ftp.storbinary(),
-    which have an open file as argument (see their definitions
-    below for details).
-    The download/upload functions first issue appropriate TYPE
-    and PORT or PASV commands.
-'''
-
-    debugging = 0
-    host = ''
-    port = FTP_PORT
-    sock = None
-    file = None
-    welcome = None
-    passiveserver = 1
-
-    # Initialization method (called by class instantiation).
-    # Initialize host to localhost, port to standard ftp port
-    # Optional arguments are host (for connect()),
-    # and user, passwd, acct (for login())
-    def __init__(self, host='', user='', passwd='', acct='',
-                 timeout=_GLOBAL_DEFAULT_TIMEOUT):
-        self.timeout = timeout
-        if host:
-            self.connect(host)
-            if user:
-                self.login(user, passwd, acct)
-
-    def connect(self, host='', port=0, timeout=-999):
-        '''Connect to host.  Arguments are:
-         - host: hostname to connect to (string, default previous host)
-         - port: port to connect to (integer, default previous port)
-        '''
-        if host != '':
-            self.host = host
-        if port > 0:
-            self.port = port
-        if timeout != -999:
-            self.timeout = timeout
-        self.sock = socket.create_connection((self.host, self.port), self.timeout)
-        self.af = self.sock.family
-        self.file = self.sock.makefile('rb')
-        self.welcome = self.getresp()
-        return self.welcome
-
-    def getwelcome(self):
-        '''Get the welcome message from the server.
-        (this is read and squirreled away by connect())'''
-        if self.debugging:
-            print '*welcome*', self.sanitize(self.welcome)
-        return self.welcome
-
-    def set_debuglevel(self, level):
-        '''Set the debugging level.
-        The required argument level means:
-        0: no debugging output (default)
-        1: print commands and responses but not body text etc.
-        2: also print raw lines read and sent before stripping CR/LF'''
-        self.debugging = level
-    debug = set_debuglevel
-
-    def set_pasv(self, val):
-        '''Use passive or active mode for data transfers.
-        With a false argument, use the normal PORT mode,
-        With a true argument, use the PASV command.'''
-        self.passiveserver = val
-
-    # Internal: "sanitize" a string for printing
-    def sanitize(self, s):
-        if s[:5] == 'pass ' or s[:5] == 'PASS ':
-            i = len(s)
-            while i > 5 and s[i-1] in '\r\n':
-                i = i-1
-            s = s[:5] + '*'*(i-5) + s[i:]
-        return repr(s)
-
-    # Internal: send one line to the server, appending CRLF
-    def putline(self, line):
-        line = line + CRLF
-        if self.debugging > 1: print '*put*', self.sanitize(line)
-        self.sock.sendall(line)
-
-    # Internal: send one command to the server (through putline())
-    def putcmd(self, line):
-        if self.debugging: print '*cmd*', self.sanitize(line)
-        self.putline(line)
-
-    # Internal: return one line from the server, stripping CRLF.
-    # Raise EOFError if the connection is closed
-    def getline(self):
-        line = self.file.readline()
-        if self.debugging > 1:
-            print '*get*', self.sanitize(line)
-        if not line: raise EOFError
-        if line[-2:] == CRLF: line = line[:-2]
-        elif line[-1:] in CRLF: line = line[:-1]
-        return line
-
-    # Internal: get a response from the server, which may possibly
-    # consist of multiple lines.  Return a single string with no
-    # trailing CRLF.  If the response consists of multiple lines,
-    # these are separated by '\n' characters in the string
-    def getmultiline(self):
-        line = self.getline()
-        if line[3:4] == '-':
-            code = line[:3]
-            while 1:
-                nextline = self.getline()
-                line = line + ('\n' + nextline)
-                if nextline[:3] == code and \
-                        nextline[3:4] != '-':
-                    break
-        return line
-
-    # Internal: get a response from the server.
-    # Raise various errors if the response indicates an error
-    def getresp(self):
-        resp = self.getmultiline()
-        if self.debugging: print '*resp*', self.sanitize(resp)
-        self.lastresp = resp[:3]
-        c = resp[:1]
-        if c in ('1', '2', '3'):
-            return resp
-        if c == '4':
-            raise error_temp, resp
-        if c == '5':
-            raise error_perm, resp
-        raise error_proto, resp
-
-    def voidresp(self):
-        """Expect a response beginning with '2'."""
-        resp = self.getresp()
-        if resp[:1] != '2':
-            raise error_reply, resp
-        return resp
-
-    def abort(self):
-        '''Abort a file transfer.  Uses out-of-band data.
-        This does not follow the procedure from the RFC to send Telnet
-        IP and Synch; that doesn't seem to work with the servers I've
-        tried.  Instead, just send the ABOR command as OOB data.'''
-        line = 'ABOR' + CRLF
-        if self.debugging > 1: print '*put urgent*', self.sanitize(line)
-        self.sock.sendall(line, MSG_OOB)
-        resp = self.getmultiline()
-        if resp[:3] not in ('426', '225', '226'):
-            raise error_proto, resp
-
-    def sendcmd(self, cmd):
-        '''Send a command and return the response.'''
-        self.putcmd(cmd)
-        return self.getresp()
-
-    def voidcmd(self, cmd):
-        """Send a command and expect a response beginning with '2'."""
-        self.putcmd(cmd)
-        return self.voidresp()
-
-    def sendport(self, host, port):
-        '''Send a PORT command with the current host and the given
-        port number.
-        '''
-        hbytes = host.split('.')
-        pbytes = [repr(port//256), repr(port%256)]
-        bytes = hbytes + pbytes
-        cmd = 'PORT ' + ','.join(bytes)
-        return self.voidcmd(cmd)
-
-    def sendeprt(self, host, port):
-        '''Send a EPRT command with the current host and the given port number.'''
-        af = 0
-        if self.af == socket.AF_INET:
-            af = 1
-        if self.af == socket.AF_INET6:
-            af = 2
-        if af == 0:
-            raise error_proto, 'unsupported address family'
-        fields = ['', repr(af), host, repr(port), '']
-        cmd = 'EPRT ' + '|'.join(fields)
-        return self.voidcmd(cmd)
-
-    def makeport(self):
-        '''Create a new socket and send a PORT command for it.'''
-        err = None
-        sock = None
-        for res in socket.getaddrinfo(None, 0, self.af, socket.SOCK_STREAM, 0, socket.AI_PASSIVE):
-            af, socktype, proto, canonname, sa = res
-            try:
-                sock = socket.socket(af, socktype, proto)
-                sock.bind(sa)
-            except socket.error, err:
-                if sock:
-                    sock.close()
-                sock = None
-                continue
-            break
-        if sock is None:
-            if err is not None:
-                raise err
-            else:
-                raise socket.error("getaddrinfo returns an empty list")
-        sock.listen(1)
-        port = sock.getsockname()[1] # Get proper port
-        host = self.sock.getsockname()[0] # Get proper host
-        if self.af == socket.AF_INET:
-            resp = self.sendport(host, port)
-        else:
-            resp = self.sendeprt(host, port)
-        if self.timeout is not _GLOBAL_DEFAULT_TIMEOUT:
-            sock.settimeout(self.timeout)
-        return sock
-
-    def makepasv(self):
-        if self.af == socket.AF_INET:
-            host, port = parse227(self.sendcmd('PASV'))
-        else:
-            host, port = parse229(self.sendcmd('EPSV'), self.sock.getpeername())
-        return host, port
-
-    def ntransfercmd(self, cmd, rest=None):
-        """Initiate a transfer over the data connection.
-
-        If the transfer is active, send a port command and the
-        transfer command, and accept the connection.  If the server is
-        passive, send a pasv command, connect to it, and start the
-        transfer command.  Either way, return the socket for the
-        connection and the expected size of the transfer.  The
-        expected size may be None if it could not be determined.
-
-        Optional `rest' argument can be a string that is sent as the
-        argument to a REST command.  This is essentially a server
-        marker used to tell the server to skip over any data up to the
-        given marker.
-        """
-        size = None
-        if self.passiveserver:
-            host, port = self.makepasv()
-            conn = socket.create_connection((host, port), self.timeout)
-            try:
-                if rest is not None:
-                    self.sendcmd("REST %s" % rest)
-                resp = self.sendcmd(cmd)
-                # Some servers apparently send a 200 reply to
-                # a LIST or STOR command, before the 150 reply
-                # (and way before the 226 reply). This seems to
-                # be in violation of the protocol (which only allows
-                # 1xx or error messages for LIST), so we just discard
-                # this response.
-                if resp[0] == '2':
-                    resp = self.getresp()
-                if resp[0] != '1':
-                    raise error_reply, resp
-            except:
-                conn.close()
-                raise
-        else:
-            sock = self.makeport()
-            try:
-                if rest is not None:
-                    self.sendcmd("REST %s" % rest)
-                resp = self.sendcmd(cmd)
-                # See above.
-                if resp[0] == '2':
-                    resp = self.getresp()
-                if resp[0] != '1':
-                    raise error_reply, resp
-                conn, sockaddr = sock.accept()
-                if self.timeout is not _GLOBAL_DEFAULT_TIMEOUT:
-                    conn.settimeout(self.timeout)
-            finally:
-                sock.close()
-        if resp[:3] == '150':
-            # this is conditional in case we received a 125
-            size = parse150(resp)
-        return conn, size
-
-    def transfercmd(self, cmd, rest=None):
-        """Like ntransfercmd() but returns only the socket."""
-        return self.ntransfercmd(cmd, rest)[0]
-
-    def login(self, user = '', passwd = '', acct = ''):
-        '''Login, default anonymous.'''
-        if not user: user = 'anonymous'
-        if not passwd: passwd = ''
-        if not acct: acct = ''
-        if user == 'anonymous' and passwd in ('', '-'):
-            # If there is no anonymous ftp password specified
-            # then we'll just use anonymous@
-            # We don't send any other thing because:
-            # - We want to remain anonymous
-            # - We want to stop SPAM
-            # - We don't want to let ftp sites to discriminate by the user,
-            #   host or country.
-            passwd = passwd + 'anonymous@'
-        resp = self.sendcmd('USER ' + user)
-        if resp[0] == '3': resp = self.sendcmd('PASS ' + passwd)
-        if resp[0] == '3': resp = self.sendcmd('ACCT ' + acct)
-        if resp[0] != '2':
-            raise error_reply, resp
-        return resp
-
-    def retrbinary(self, cmd, callback, blocksize=8192, rest=None):
-        """Retrieve data in binary mode.  A new port is created for you.
-
-        Args:
-          cmd: A RETR command.
-          callback: A single parameter callable to be called on each
-                    block of data read.
-          blocksize: The maximum number of bytes to read from the
-                     socket at one time.  [default: 8192]
-          rest: Passed to transfercmd().  [default: None]
-
-        Returns:
-          The response code.
-        """
-        self.voidcmd('TYPE I')
-        conn = self.transfercmd(cmd, rest)
-        while 1:
-            data = conn.recv(blocksize)
-            if not data:
-                break
-            callback(data)
-        conn.close()
-        return self.voidresp()
-
-    def retrlines(self, cmd, callback = None):
-        """Retrieve data in line mode.  A new port is created for you.
-
-        Args:
-          cmd: A RETR, LIST, NLST, or MLSD command.
-          callback: An optional single parameter callable that is called
-                    for each line with the trailing CRLF stripped.
-                    [default: print_line()]
-
-        Returns:
-          The response code.
-        """
-        if callback is None: callback = print_line
-        resp = self.sendcmd('TYPE A')
-        conn = self.transfercmd(cmd)
-        fp = conn.makefile('rb')
-        while 1:
-            line = fp.readline()
-            if self.debugging > 2: print '*retr*', repr(line)
-            if not line:
-                break
-            if line[-2:] == CRLF:
-                line = line[:-2]
-            elif line[-1:] == '\n':
-                line = line[:-1]
-            callback(line)
-        fp.close()
-        conn.close()
-        return self.voidresp()
-
-    def storbinary(self, cmd, fp, blocksize=8192, callback=None, rest=None):
-        """Store a file in binary mode.  A new port is created for you.
-
-        Args:
-          cmd: A STOR command.
-          fp: A file-like object with a read(num_bytes) method.
-          blocksize: The maximum data size to read from fp and send over
-                     the connection at once.  [default: 8192]
-          callback: An optional single parameter callable that is called on
-                    on each block of data after it is sent.  [default: None]
-          rest: Passed to transfercmd().  [default: None]
-
-        Returns:
-          The response code.
-        """
-        self.voidcmd('TYPE I')
-        conn = self.transfercmd(cmd, rest)
-        while 1:
-            buf = fp.read(blocksize)
-            if not buf: break
-            conn.sendall(buf)
-            if callback: callback(buf)
-        conn.close()
-        return self.voidresp()
-
-    def storlines(self, cmd, fp, callback=None):
-        """Store a file in line mode.  A new port is created for you.
-
-        Args:
-          cmd: A STOR command.
-          fp: A file-like object with a readline() method.
-          callback: An optional single parameter callable that is called on
-                    on each line after it is sent.  [default: None]
-
-        Returns:
-          The response code.
-        """
-        self.voidcmd('TYPE A')
-        conn = self.transfercmd(cmd)
-        while 1:
-            buf = fp.readline()
-            if not buf: break
-            if buf[-2:] != CRLF:
-                if buf[-1] in CRLF: buf = buf[:-1]
-                buf = buf + CRLF
-            conn.sendall(buf)
-            if callback: callback(buf)
-        conn.close()
-        return self.voidresp()
-
-    def acct(self, password):
-        '''Send new account name.'''
-        cmd = 'ACCT ' + password
-        return self.voidcmd(cmd)
-
-    def nlst(self, *args):
-        '''Return a list of files in a given directory (default the current).'''
-        cmd = 'NLST'
-        for arg in args:
-            cmd = cmd + (' ' + arg)
-        files = []
-        self.retrlines(cmd, files.append)
-        return files
-
-    def dir(self, *args):
-        '''List a directory in long form.
-        By default list current directory to stdout.
-        Optional last argument is callback function; all
-        non-empty arguments before it are concatenated to the
-        LIST command.  (This *should* only be used for a pathname.)'''
-        cmd = 'LIST'
-        func = None
-        if args[-1:] and type(args[-1]) != type(''):
-            args, func = args[:-1], args[-1]
-        for arg in args:
-            if arg:
-                cmd = cmd + (' ' + arg)
-        self.retrlines(cmd, func)
-
-    def rename(self, fromname, toname):
-        '''Rename a file.'''
-        resp = self.sendcmd('RNFR ' + fromname)
-        if resp[0] != '3':
-            raise error_reply, resp
-        return self.voidcmd('RNTO ' + toname)
-
-    def delete(self, filename):
-        '''Delete a file.'''
-        resp = self.sendcmd('DELE ' + filename)
-        if resp[:3] in ('250', '200'):
-            return resp
-        else:
-            raise error_reply, resp
-
-    def cwd(self, dirname):
-        '''Change to a directory.'''
-        if dirname == '..':
-            try:
-                return self.voidcmd('CDUP')
-            except error_perm, msg:
-                if msg.args[0][:3] != '500':
-                    raise
-        elif dirname == '':
-            dirname = '.'  # does nothing, but could return error
-        cmd = 'CWD ' + dirname
-        return self.voidcmd(cmd)
-
-    def size(self, filename):
-        '''Retrieve the size of a file.'''
-        # The SIZE command is defined in RFC-3659
-        resp = self.sendcmd('SIZE ' + filename)
-        if resp[:3] == '213':
-            s = resp[3:].strip()
-            try:
-                return int(s)
-            except (OverflowError, ValueError):
-                return long(s)
-
-    def mkd(self, dirname):
-        '''Make a directory, return its full pathname.'''
-        resp = self.sendcmd('MKD ' + dirname)
-        return parse257(resp)
-
-    def rmd(self, dirname):
-        '''Remove a directory.'''
-        return self.voidcmd('RMD ' + dirname)
-
-    def pwd(self):
-        '''Return current working directory.'''
-        resp = self.sendcmd('PWD')
-        return parse257(resp)
-
-    def quit(self):
-        '''Quit, and close the connection.'''
-        resp = self.voidcmd('QUIT')
-        self.close()
-        return resp
-
-    def close(self):
-        '''Close the connection without assuming anything about it.'''
-        if self.file is not None:
-            self.file.close()
-        if self.sock is not None:
-            self.sock.close()
-        self.file = self.sock = None
-
-try:
-    import ssl
-    ssl.PROTOCOL_TLSv1
-except (ImportError, AttributeError):
-    pass
-else:
-    class FTP_TLS(FTP):
-        '''A FTP subclass which adds TLS support to FTP as described
-        in RFC-4217.
-
-        Connect as usual to port 21 implicitly securing the FTP control
-        connection before authenticating.
-
-        Securing the data connection requires user to explicitly ask
-        for it by calling prot_p() method.
-
-        Usage example:
-        >>> from ftplib import FTP_TLS
-        >>> ftps = FTP_TLS('ftp.python.org')
-        >>> ftps.login()  # login anonymously previously securing control channel
-        '230 Guest login ok, access restrictions apply.'
-        >>> ftps.prot_p()  # switch to secure data connection
-        '200 Protection level set to P'
-        >>> ftps.retrlines('LIST')  # list directory content securely
-        total 9
-        drwxr-xr-x   8 root     wheel        1024 Jan  3  1994 .
-        drwxr-xr-x   8 root     wheel        1024 Jan  3  1994 ..
-        drwxr-xr-x   2 root     wheel        1024 Jan  3  1994 bin
-        drwxr-xr-x   2 root     wheel        1024 Jan  3  1994 etc
-        d-wxrwxr-x   2 ftp      wheel        1024 Sep  5 13:43 incoming
-        drwxr-xr-x   2 root     wheel        1024 Nov 17  1993 lib
-        drwxr-xr-x   6 1094     wheel        1024 Sep 13 19:07 pub
-        drwxr-xr-x   3 root     wheel        1024 Jan  3  1994 usr
-        -rw-r--r--   1 root     root          312 Aug  1  1994 welcome.msg
-        '226 Transfer complete.'
-        >>> ftps.quit()
-        '221 Goodbye.'
-        >>>
-        '''
-        ssl_version = ssl.PROTOCOL_TLSv1
-
-        def __init__(self, host='', user='', passwd='', acct='', keyfile=None,
-                     certfile=None, timeout=_GLOBAL_DEFAULT_TIMEOUT):
-            self.keyfile = keyfile
-            self.certfile = certfile
-            self._prot_p = False
-            FTP.__init__(self, host, user, passwd, acct, timeout)
-
-        def login(self, user='', passwd='', acct='', secure=True):
-            if secure and not isinstance(self.sock, ssl.SSLSocket):
-                self.auth()
-            return FTP.login(self, user, passwd, acct)
-
-        def auth(self):
-            '''Set up secure control connection by using TLS/SSL.'''
-            if isinstance(self.sock, ssl.SSLSocket):
-                raise ValueError("Already using TLS")
-            if self.ssl_version == ssl.PROTOCOL_TLSv1:
-                resp = self.voidcmd('AUTH TLS')
-            else:
-                resp = self.voidcmd('AUTH SSL')
-            self.sock = ssl.wrap_socket(self.sock, self.keyfile, self.certfile,
-                                        ssl_version=self.ssl_version)
-            self.file = self.sock.makefile(mode='rb')
-            return resp
-
-        def prot_p(self):
-            '''Set up secure data connection.'''
-            # PROT defines whether or not the data channel is to be protected.
-            # Though RFC-2228 defines four possible protection levels,
-            # RFC-4217 only recommends two, Clear and Private.
-            # Clear (PROT C) means that no security is to be used on the
-            # data-channel, Private (PROT P) means that the data-channel
-            # should be protected by TLS.
-            # PBSZ command MUST still be issued, but must have a parameter of
-            # '0' to indicate that no buffering is taking place and the data
-            # connection should not be encapsulated.
-            self.voidcmd('PBSZ 0')
-            resp = self.voidcmd('PROT P')
-            self._prot_p = True
-            return resp
-
-        def prot_c(self):
-            '''Set up clear text data connection.'''
-            resp = self.voidcmd('PROT C')
-            self._prot_p = False
-            return resp
-
-        # --- Overridden FTP methods
-
-        def ntransfercmd(self, cmd, rest=None):
-            conn, size = FTP.ntransfercmd(self, cmd, rest)
-            if self._prot_p:
-                conn = ssl.wrap_socket(conn, self.keyfile, self.certfile,
-                                       ssl_version=self.ssl_version)
-            return conn, size
-
-        def retrbinary(self, cmd, callback, blocksize=8192, rest=None):
-            self.voidcmd('TYPE I')
-            conn = self.transfercmd(cmd, rest)
-            try:
-                while 1:
-                    data = conn.recv(blocksize)
-                    if not data:
-                        break
-                    callback(data)
-                # shutdown ssl layer
-                if isinstance(conn, ssl.SSLSocket):
-                    conn.unwrap()
-            finally:
-                conn.close()
-            return self.voidresp()
-
-        def retrlines(self, cmd, callback = None):
-            if callback is None: callback = print_line
-            resp = self.sendcmd('TYPE A')
-            conn = self.transfercmd(cmd)
-            fp = conn.makefile('rb')
-            try:
-                while 1:
-                    line = fp.readline()
-                    if self.debugging > 2: print '*retr*', repr(line)
-                    if not line:
-                        break
-                    if line[-2:] == CRLF:
-                        line = line[:-2]
-                    elif line[-1:] == '\n':
-                        line = line[:-1]
-                    callback(line)
-                # shutdown ssl layer
-                if isinstance(conn, ssl.SSLSocket):
-                    conn.unwrap()
-            finally:
-                fp.close()
-                conn.close()
-            return self.voidresp()
-
-        def storbinary(self, cmd, fp, blocksize=8192, callback=None, rest=None):
-            self.voidcmd('TYPE I')
-            conn = self.transfercmd(cmd, rest)
-            try:
-                while 1:
-                    buf = fp.read(blocksize)
-                    if not buf: break
-                    conn.sendall(buf)
-                    if callback: callback(buf)
-                # shutdown ssl layer
-                if isinstance(conn, ssl.SSLSocket):
-                    conn.unwrap()
-            finally:
-                conn.close()
-            return self.voidresp()
-
-        def storlines(self, cmd, fp, callback=None):
-            self.voidcmd('TYPE A')
-            conn = self.transfercmd(cmd)
-            try:
-                while 1:
-                    buf = fp.readline()
-                    if not buf: break
-                    if buf[-2:] != CRLF:
-                        if buf[-1] in CRLF: buf = buf[:-1]
-                        buf = buf + CRLF
-                    conn.sendall(buf)
-                    if callback: callback(buf)
-                # shutdown ssl layer
-                if isinstance(conn, ssl.SSLSocket):
-                    conn.unwrap()
-            finally:
-                conn.close()
-            return self.voidresp()
-
-    __all__.append('FTP_TLS')
-    all_errors = (Error, IOError, EOFError, ssl.SSLError)
-
-
-_150_re = None
-
-def parse150(resp):
-    '''Parse the '150' response for a RETR request.
-    Returns the expected transfer size or None; size is not guaranteed to
-    be present in the 150 message.
-    '''
-    if resp[:3] != '150':
-        raise error_reply, resp
-    global _150_re
-    if _150_re is None:
-        import re
-        _150_re = re.compile("150 .* \((\d+) bytes\)", re.IGNORECASE)
-    m = _150_re.match(resp)
-    if not m:
-        return None
-    s = m.group(1)
-    try:
-        return int(s)
-    except (OverflowError, ValueError):
-        return long(s)
-
-
-_227_re = None
-
-def parse227(resp):
-    '''Parse the '227' response for a PASV request.
-    Raises error_proto if it does not contain '(h1,h2,h3,h4,p1,p2)'
-    Return ('host.addr.as.numbers', port#) tuple.'''
-
-    if resp[:3] != '227':
-        raise error_reply, resp
-    global _227_re
-    if _227_re is None:
-        import re
-        _227_re = re.compile(r'(\d+),(\d+),(\d+),(\d+),(\d+),(\d+)')
-    m = _227_re.search(resp)
-    if not m:
-        raise error_proto, resp
-    numbers = m.groups()
-    host = '.'.join(numbers[:4])
-    port = (int(numbers[4]) << 8) + int(numbers[5])
-    return host, port
-
-
-def parse229(resp, peer):
-    '''Parse the '229' response for a EPSV request.
-    Raises error_proto if it does not contain '(|||port|)'
-    Return ('host.addr.as.numbers', port#) tuple.'''
-
-    if resp[:3] != '229':
-        raise error_reply, resp
-    left = resp.find('(')
-    if left < 0: raise error_proto, resp
-    right = resp.find(')', left + 1)
-    if right < 0:
-        raise error_proto, resp # should contain '(|||port|)'
-    if resp[left + 1] != resp[right - 1]:
-        raise error_proto, resp
-    parts = resp[left + 1:right].split(resp[left+1])
-    if len(parts) != 5:
-        raise error_proto, resp
-    host = peer[0]
-    port = int(parts[3])
-    return host, port
-
-
-def parse257(resp):
-    '''Parse the '257' response for a MKD or PWD request.
-    This is a response to a MKD or PWD request: a directory name.
-    Returns the directoryname in the 257 reply.'''
-
-    if resp[:3] != '257':
-        raise error_reply, resp
-    if resp[3:5] != ' "':
-        return '' # Not compliant to RFC 959, but UNIX ftpd does this
-    dirname = ''
-    i = 5
-    n = len(resp)
-    while i < n:
-        c = resp[i]
-        i = i+1
-        if c == '"':
-            if i >= n or resp[i] != '"':
-                break
-            i = i+1
-        dirname = dirname + c
-    return dirname
-
-
-def print_line(line):
-    '''Default retrlines callback to print a line.'''
-    print line
-
-
-def ftpcp(source, sourcename, target, targetname = '', type = 'I'):
-    '''Copy file from one FTP-instance to another.'''
-    if not targetname: targetname = sourcename
-    type = 'TYPE ' + type
-    source.voidcmd(type)
-    target.voidcmd(type)
-    sourcehost, sourceport = parse227(source.sendcmd('PASV'))
-    target.sendport(sourcehost, sourceport)
-    # RFC 959: the user must "listen" [...] BEFORE sending the
-    # transfer request.
-    # So: STOR before RETR, because here the target is a "user".
-    treply = target.sendcmd('STOR ' + targetname)
-    if treply[:3] not in ('125', '150'): raise error_proto  # RFC 959
-    sreply = source.sendcmd('RETR ' + sourcename)
-    if sreply[:3] not in ('125', '150'): raise error_proto  # RFC 959
-    source.voidresp()
-    target.voidresp()
-
-
-class Netrc:
-    """Class to parse & provide access to 'netrc' format files.
-
-    See the netrc(4) man page for information on the file format.
-
-    WARNING: This class is obsolete -- use module netrc instead.
-
-    """
-    __defuser = None
-    __defpasswd = None
-    __defacct = None
-
-    def __init__(self, filename=None):
-        if filename is None:
-            if "HOME" in os.environ:
-                filename = os.path.join(os.environ["HOME"],
-                                        ".netrc")
-            else:
-                raise IOError, \
-                      "specify file to load or set $HOME"
-        self.__hosts = {}
-        self.__macros = {}
-        fp = open(filename, "r")
-        in_macro = 0
-        while 1:
-            line = fp.readline()
-            if not line: break
-            if in_macro and line.strip():
-                macro_lines.append(line)
-                continue
-            elif in_macro:
-                self.__macros[macro_name] = tuple(macro_lines)
-                in_macro = 0
-            words = line.split()
-            host = user = passwd = acct = None
-            default = 0
-            i = 0
-            while i < len(words):
-                w1 = words[i]
-                if i+1 < len(words):
-                    w2 = words[i + 1]
-                else:
-                    w2 = None
-                if w1 == 'default':
-                    default = 1
-                elif w1 == 'machine' and w2:
-                    host = w2.lower()
-                    i = i + 1
-                elif w1 == 'login' and w2:
-                    user = w2
-                    i = i + 1
-                elif w1 == 'password' and w2:
-                    passwd = w2
-                    i = i + 1
-                elif w1 == 'account' and w2:
-                    acct = w2
-                    i = i + 1
-                elif w1 == 'macdef' and w2:
-                    macro_name = w2
-                    macro_lines = []
-                    in_macro = 1
-                    break
-                i = i + 1
-            if default:
-                self.__defuser = user or self.__defuser
-                self.__defpasswd = passwd or self.__defpasswd
-                self.__defacct = acct or self.__defacct
-            if host:
-                if host in self.__hosts:
-                    ouser, opasswd, oacct = \
-                           self.__hosts[host]
-                    user = user or ouser
-                    passwd = passwd or opasswd
-                    acct = acct or oacct
-                self.__hosts[host] = user, passwd, acct
-        fp.close()
-
-    def get_hosts(self):
-        """Return a list of hosts mentioned in the .netrc file."""
-        return self.__hosts.keys()
-
-    def get_account(self, host):
-        """Returns login information for the named host.
-
-        The return value is a triple containing userid,
-        password, and the accounting field.
-
-        """
-        host = host.lower()
-        user = passwd = acct = None
-        if host in self.__hosts:
-            user, passwd, acct = self.__hosts[host]
-        user = user or self.__defuser
-        passwd = passwd or self.__defpasswd
-        acct = acct or self.__defacct
-        return user, passwd, acct
-
-    def get_macros(self):
-        """Return a list of all defined macro names."""
-        return self.__macros.keys()
-
-    def get_macro(self, macro):
-        """Return a sequence of lines which define a named macro."""
-        return self.__macros[macro]
-
-
-
-def test():
-    '''Test program.
-    Usage: ftp [-d] [-r[file]] host [-l[dir]] [-d[dir]] [-p] [file] ...
-
-    -d dir
-    -l list
-    -p password
-    '''
-
-    if len(sys.argv) < 2:
-        print test.__doc__
-        sys.exit(0)
-
-    debugging = 0
-    rcfile = None
-    while sys.argv[1] == '-d':
-        debugging = debugging+1
-        del sys.argv[1]
-    if sys.argv[1][:2] == '-r':
-        # get name of alternate ~/.netrc file:
-        rcfile = sys.argv[1][2:]
-        del sys.argv[1]
-    host = sys.argv[1]
-    ftp = FTP(host)
-    ftp.set_debuglevel(debugging)
-    userid = passwd = acct = ''
-    try:
-        netrc = Netrc(rcfile)
-    except IOError:
-        if rcfile is not None:
-            sys.stderr.write("Could not open account file"
-                             " -- using anonymous login.")
-    else:
-        try:
-            userid, passwd, acct = netrc.get_account(host)
-        except KeyError:
-            # no account for host
-            sys.stderr.write(
-                    "No account -- using anonymous login.")
-    ftp.login(userid, passwd, acct)
-    for file in sys.argv[2:]:
-        if file[:2] == '-l':
-            ftp.dir(file[2:])
-        elif file[:2] == '-d':
-            cmd = 'CWD'
-            if file[2:]: cmd = cmd + ' ' + file[2:]
-            resp = ftp.sendcmd(cmd)
-        elif file == '-p':
-            ftp.set_pasv(not ftp.passiveserver)
-        else:
-            ftp.retrbinary('RETR ' + file, \
-                           sys.stdout.write, 1024)
-    ftp.quit()
-
-
-if __name__ == '__main__':
-    test()
diff --git a/Lib/logging/config.py b/Lib/logging/config.py
deleted file mode 100644
--- a/Lib/logging/config.py
+++ /dev/null
@@ -1,909 +0,0 @@
-# Copyright 2001-2010 by Vinay Sajip. All Rights Reserved.
-#
-# Permission to use, copy, modify, and distribute this software and its
-# documentation for any purpose and without fee is hereby granted,
-# provided that the above copyright notice appear in all copies and that
-# both that copyright notice and this permission notice appear in
-# supporting documentation, and that the name of Vinay Sajip
-# not be used in advertising or publicity pertaining to distribution
-# of the software without specific, written prior permission.
-# VINAY SAJIP DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING
-# ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
-# VINAY SAJIP BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR
-# ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER
-# IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
-# OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-
-"""
-Configuration functions for the logging package for Python. The core package
-is based on PEP 282 and comments thereto in comp.lang.python, and influenced
-by Apache's log4j system.
-
-Copyright (C) 2001-2010 Vinay Sajip. All Rights Reserved.
-
-To use, simply 'import logging' and log away!
-"""
-
-import sys, logging, logging.handlers, socket, struct, os, traceback, re
-import types, cStringIO
-
-try:
-    import thread
-    import threading
-except ImportError:
-    thread = None
-
-from SocketServer import ThreadingTCPServer, StreamRequestHandler
-
-
-DEFAULT_LOGGING_CONFIG_PORT = 9030
-
-if sys.platform == "win32":
-    RESET_ERROR = 10054   #WSAECONNRESET
-else:
-    RESET_ERROR = 104     #ECONNRESET
-
-#
-#   The following code implements a socket listener for on-the-fly
-#   reconfiguration of logging.
-#
-#   _listener holds the server object doing the listening
-_listener = None
-
-def fileConfig(fname, defaults=None, disable_existing_loggers=True):
-    """
-    Read the logging configuration from a ConfigParser-format file.
-
-    This can be called several times from an application, allowing an end user
-    the ability to select from various pre-canned configurations (if the
-    developer provides a mechanism to present the choices and load the chosen
-    configuration).
-    """
-    import ConfigParser
-
-    cp = ConfigParser.ConfigParser(defaults)
-    if hasattr(fname, 'readline'):
-        cp.readfp(fname)
-    else:
-        cp.read(fname)
-
-    formatters = _create_formatters(cp)
-
-    # critical section
-    logging._acquireLock()
-    try:
-        logging._handlers.clear()
-        del logging._handlerList[:]
-        # Handlers add themselves to logging._handlers
-        handlers = _install_handlers(cp, formatters)
-        _install_loggers(cp, handlers, disable_existing_loggers)
-    finally:
-        logging._releaseLock()
-
-
-def _resolve(name):
-    """Resolve a dotted name to a global object."""
-    name = name.split('.')
-    used = name.pop(0)
-    found = __import__(used)
-    for n in name:
-        used = used + '.' + n
-        try:
-            found = getattr(found, n)
-        except AttributeError:
-            __import__(used)
-            found = getattr(found, n)
-    return found
-
-def _strip_spaces(alist):
-    return map(lambda x: x.strip(), alist)
-
-def _encoded(s):
-    return s if isinstance(s, str) else s.encode('utf-8')
-
-def _create_formatters(cp):
-    """Create and return formatters"""
-    flist = cp.get("formatters", "keys")
-    if not len(flist):
-        return {}
-    flist = flist.split(",")
-    flist = _strip_spaces(flist)
-    formatters = {}
-    for form in flist:
-        sectname = "formatter_%s" % form
-        opts = cp.options(sectname)
-        if "format" in opts:
-            fs = cp.get(sectname, "format", 1)
-        else:
-            fs = None
-        if "datefmt" in opts:
-            dfs = cp.get(sectname, "datefmt", 1)
-        else:
-            dfs = None
-        c = logging.Formatter
-        if "class" in opts:
-            class_name = cp.get(sectname, "class")
-            if class_name:
-                c = _resolve(class_name)
-        f = c(fs, dfs)
-        formatters[form] = f
-    return formatters
-
-
-def _install_handlers(cp, formatters):
-    """Install and return handlers"""
-    hlist = cp.get("handlers", "keys")
-    if not len(hlist):
-        return {}
-    hlist = hlist.split(",")
-    hlist = _strip_spaces(hlist)
-    handlers = {}
-    fixups = [] #for inter-handler references
-    for hand in hlist:
-        sectname = "handler_%s" % hand
-        klass = cp.get(sectname, "class")
-        opts = cp.options(sectname)
-        if "formatter" in opts:
-            fmt = cp.get(sectname, "formatter")
-        else:
-            fmt = ""
-        try:
-            klass = eval(klass, vars(logging))
-        except (AttributeError, NameError):
-            klass = _resolve(klass)
-        args = cp.get(sectname, "args")
-        args = eval(args, vars(logging))
-        h = klass(*args)
-        if "level" in opts:
-            level = cp.get(sectname, "level")
-            h.setLevel(logging._levelNames[level])
-        if len(fmt):
-            h.setFormatter(formatters[fmt])
-        if issubclass(klass, logging.handlers.MemoryHandler):
-            if "target" in opts:
-                target = cp.get(sectname,"target")
-            else:
-                target = ""
-            if len(target): #the target handler may not be loaded yet, so keep for later...
-                fixups.append((h, target))
-        handlers[hand] = h
-    #now all handlers are loaded, fixup inter-handler references...
-    for h, t in fixups:
-        h.setTarget(handlers[t])
-    return handlers
-
-
-def _install_loggers(cp, handlers, disable_existing_loggers):
-    """Create and install loggers"""
-
-    # configure the root first
-    llist = cp.get("loggers", "keys")
-    llist = llist.split(",")
-    llist = list(map(lambda x: x.strip(), llist))
-    llist.remove("root")
-    sectname = "logger_root"
-    root = logging.root
-    log = root
-    opts = cp.options(sectname)
-    if "level" in opts:
-        level = cp.get(sectname, "level")
-        log.setLevel(logging._levelNames[level])
-    for h in root.handlers[:]:
-        root.removeHandler(h)
-    hlist = cp.get(sectname, "handlers")
-    if len(hlist):
-        hlist = hlist.split(",")
-        hlist = _strip_spaces(hlist)
-        for hand in hlist:
-            log.addHandler(handlers[hand])
-
-    #and now the others...
-    #we don't want to lose the existing loggers,
-    #since other threads may have pointers to them.
-    #existing is set to contain all existing loggers,
-    #and as we go through the new configuration we
-    #remove any which are configured. At the end,
-    #what's left in existing is the set of loggers
-    #which were in the previous configuration but
-    #which are not in the new configuration.
-    existing = list(root.manager.loggerDict.keys())
-    #The list needs to be sorted so that we can
-    #avoid disabling child loggers of explicitly
-    #named loggers. With a sorted list it is easier
-    #to find the child loggers.
-    existing.sort()
-    #We'll keep the list of existing loggers
-    #which are children of named loggers here...
-    child_loggers = []
-    #now set up the new ones...
-    for log in llist:
-        sectname = "logger_%s" % log
-        qn = cp.get(sectname, "qualname")
-        opts = cp.options(sectname)
-        if "propagate" in opts:
-            propagate = cp.getint(sectname, "propagate")
-        else:
-            propagate = 1
-        logger = logging.getLogger(qn)
-        if qn in existing:
-            i = existing.index(qn) + 1 # start with the entry after qn
-            prefixed = qn + "."
-            pflen = len(prefixed)
-            num_existing = len(existing)
-            while i < num_existing:
-                if existing[i][:pflen] == prefixed:
-                    child_loggers.append(existing[i])
-                i += 1
-            existing.remove(qn)
-        if "level" in opts:
-            level = cp.get(sectname, "level")
-            logger.setLevel(logging._levelNames[level])
-        for h in logger.handlers[:]:
-            logger.removeHandler(h)
-        logger.propagate = propagate
-        logger.disabled = 0
-        hlist = cp.get(sectname, "handlers")
-        if len(hlist):
-            hlist = hlist.split(",")
-            hlist = _strip_spaces(hlist)
-            for hand in hlist:
-                logger.addHandler(handlers[hand])
-
-    #Disable any old loggers. There's no point deleting
-    #them as other threads may continue to hold references
-    #and by disabling them, you stop them doing any logging.
-    #However, don't disable children of named loggers, as that's
-    #probably not what was intended by the user.
-    for log in existing:
-        logger = root.manager.loggerDict[log]
-        if log in child_loggers:
-            logger.level = logging.NOTSET
-            logger.handlers = []
-            logger.propagate = 1
-        elif disable_existing_loggers:
-            logger.disabled = 1
-
-
-
-IDENTIFIER = re.compile('^[a-z_][a-z0-9_]*$', re.I)
-
-
-def valid_ident(s):
-    m = IDENTIFIER.match(s)
-    if not m:
-        raise ValueError('Not a valid Python identifier: %r' % s)
-    return True
-
-
-# The ConvertingXXX classes are wrappers around standard Python containers,
-# and they serve to convert any suitable values in the container. The
-# conversion converts base dicts, lists and tuples to their wrapped
-# equivalents, whereas strings which match a conversion format are converted
-# appropriately.
-#
-# Each wrapper should have a configurator attribute holding the actual
-# configurator to use for conversion.
-
-class ConvertingDict(dict):
-    """A converting dictionary wrapper."""
-
-    def __getitem__(self, key):
-        value = dict.__getitem__(self, key)
-        result = self.configurator.convert(value)
-        #If the converted value is different, save for next time
-        if value is not result:
-            self[key] = result
-            if type(result) in (ConvertingDict, ConvertingList,
-                                ConvertingTuple):
-                result.parent = self
-                result.key = key
-        return result
-
-    def get(self, key, default=None):
-        value = dict.get(self, key, default)
-        result = self.configurator.convert(value)
-        #If the converted value is different, save for next time
-        if value is not result:
-            self[key] = result
-            if type(result) in (ConvertingDict, ConvertingList,
-                                ConvertingTuple):
-                result.parent = self
-                result.key = key
-        return result
-
-    def pop(self, key, default=None):
-        value = dict.pop(self, key, default)
-        result = self.configurator.convert(value)
-        if value is not result:
-            if type(result) in (ConvertingDict, ConvertingList,
-                                ConvertingTuple):
-                result.parent = self
-                result.key = key
-        return result
-
-class ConvertingList(list):
-    """A converting list wrapper."""
-    def __getitem__(self, key):
-        value = list.__getitem__(self, key)
-        result = self.configurator.convert(value)
-        #If the converted value is different, save for next time
-        if value is not result:
-            self[key] = result
-            if type(result) in (ConvertingDict, ConvertingList,
-                                ConvertingTuple):
-                result.parent = self
-                result.key = key
-        return result
-
-    def pop(self, idx=-1):
-        value = list.pop(self, idx)
-        result = self.configurator.convert(value)
-        if value is not result:
-            if type(result) in (ConvertingDict, ConvertingList,
-                                ConvertingTuple):
-                result.parent = self
-        return result
-
-class ConvertingTuple(tuple):
-    """A converting tuple wrapper."""
-    def __getitem__(self, key):
-        value = tuple.__getitem__(self, key)
-        result = self.configurator.convert(value)
-        if value is not result:
-            if type(result) in (ConvertingDict, ConvertingList,
-                                ConvertingTuple):
-                result.parent = self
-                result.key = key
-        return result
-
-class BaseConfigurator(object):
-    """
-    The configurator base class which defines some useful defaults.
-    """
-
-    CONVERT_PATTERN = re.compile(r'^(?P<prefix>[a-z]+)://(?P<suffix>.*)$')
-
-    WORD_PATTERN = re.compile(r'^\s*(\w+)\s*')
-    DOT_PATTERN = re.compile(r'^\.\s*(\w+)\s*')
-    INDEX_PATTERN = re.compile(r'^\[\s*(\w+)\s*\]\s*')
-    DIGIT_PATTERN = re.compile(r'^\d+$')
-
-    value_converters = {
-        'ext' : 'ext_convert',
-        'cfg' : 'cfg_convert',
-    }
-
-    # We might want to use a different one, e.g. importlib
-    importer = __import__
-
-    def __init__(self, config):
-        self.config = ConvertingDict(config)
-        self.config.configurator = self
-
-    def resolve(self, s):
-        """
-        Resolve strings to objects using standard import and attribute
-        syntax.
-        """
-        name = s.split('.')
-        used = name.pop(0)
-        try:
-            found = self.importer(used)
-            for frag in name:
-                used += '.' + frag
-                try:
-                    found = getattr(found, frag)
-                except AttributeError:
-                    self.importer(used)
-                    found = getattr(found, frag)
-            return found
-        except ImportError:
-            e, tb = sys.exc_info()[1:]
-            v = ValueError('Cannot resolve %r: %s' % (s, e))
-            v.__cause__, v.__traceback__ = e, tb
-            raise v
-
-    def ext_convert(self, value):
-        """Default converter for the ext:// protocol."""
-        return self.resolve(value)
-
-    def cfg_convert(self, value):
-        """Default converter for the cfg:// protocol."""
-        rest = value
-        m = self.WORD_PATTERN.match(rest)
-        if m is None:
-            raise ValueError("Unable to convert %r" % value)
-        else:
-            rest = rest[m.end():]
-            d = self.config[m.groups()[0]]
-            #print d, rest
-            while rest:
-                m = self.DOT_PATTERN.match(rest)
-                if m:
-                    d = d[m.groups()[0]]
-                else:
-                    m = self.INDEX_PATTERN.match(rest)
-                    if m:
-                        idx = m.groups()[0]
-                        if not self.DIGIT_PATTERN.match(idx):
-                            d = d[idx]
-                        else:
-                            try:
-                                n = int(idx) # try as number first (most likely)
-                                d = d[n]
-                            except TypeError:
-                                d = d[idx]
-                if m:
-                    rest = rest[m.end():]
-                else:
-                    raise ValueError('Unable to convert '
-                                     '%r at %r' % (value, rest))
-        #rest should be empty
-        return d
-
-    def convert(self, value):
-        """
-        Convert values to an appropriate type. dicts, lists and tuples are
-        replaced by their converting alternatives. Strings are checked to
-        see if they have a conversion format and are converted if they do.
-        """
-        if not isinstance(value, ConvertingDict) and isinstance(value, dict):
-            value = ConvertingDict(value)
-            value.configurator = self
-        elif not isinstance(value, ConvertingList) and isinstance(value, list):
-            value = ConvertingList(value)
-            value.configurator = self
-        elif not isinstance(value, ConvertingTuple) and\
-                 isinstance(value, tuple):
-            value = ConvertingTuple(value)
-            value.configurator = self
-        elif isinstance(value, basestring): # str for py3k
-            m = self.CONVERT_PATTERN.match(value)
-            if m:
-                d = m.groupdict()
-                prefix = d['prefix']
-                converter = self.value_converters.get(prefix, None)
-                if converter:
-                    suffix = d['suffix']
-                    converter = getattr(self, converter)
-                    value = converter(suffix)
-        return value
-
-    def configure_custom(self, config):
-        """Configure an object with a user-supplied factory."""
-        c = config.pop('()')
-        if not hasattr(c, '__call__') and hasattr(types, 'ClassType') and type(c) != types.ClassType:
-            c = self.resolve(c)
-        props = config.pop('.', None)
-        # Check for valid identifiers
-        kwargs = dict([(k, config[k]) for k in config if valid_ident(k)])
-        result = c(**kwargs)
-        if props:
-            for name, value in props.items():
-                setattr(result, name, value)
-        return result
-
-    def as_tuple(self, value):
-        """Utility function which converts lists to tuples."""
-        if isinstance(value, list):
-            value = tuple(value)
-        return value
-
-class DictConfigurator(BaseConfigurator):
-    """
-    Configure logging using a dictionary-like object to describe the
-    configuration.
-    """
-
-    def configure(self):
-        """Do the configuration."""
-
-        config = self.config
-        if 'version' not in config:
-            raise ValueError("dictionary doesn't specify a version")
-        if config['version'] != 1:
-            raise ValueError("Unsupported version: %s" % config['version'])
-        incremental = config.pop('incremental', False)
-        EMPTY_DICT = {}
-        logging._acquireLock()
-        try:
-            if incremental:
-                handlers = config.get('handlers', EMPTY_DICT)
-                for name in handlers:
-                    if name not in logging._handlers:
-                        raise ValueError('No handler found with '
-                                         'name %r'  % name)
-                    else:
-                        try:
-                            handler = logging._handlers[name]
-                            handler_config = handlers[name]
-                            level = handler_config.get('level', None)
-                            if level:
-                                handler.setLevel(logging._checkLevel(level))
-                        except StandardError, e:
-                            raise ValueError('Unable to configure handler '
-                                             '%r: %s' % (name, e))
-                loggers = config.get('loggers', EMPTY_DICT)
-                for name in loggers:
-                    try:
-                        self.configure_logger(name, loggers[name], True)
-                    except StandardError, e:
-                        raise ValueError('Unable to configure logger '
-                                         '%r: %s' % (name, e))
-                root = config.get('root', None)
-                if root:
-                    try:
-                        self.configure_root(root, True)
-                    except StandardError, e:
-                        raise ValueError('Unable to configure root '
-                                         'logger: %s' % e)
-            else:
-                disable_existing = config.pop('disable_existing_loggers', True)
-
-                logging._handlers.clear()
-                del logging._handlerList[:]
-
-                # Do formatters first - they don't refer to anything else
-                formatters = config.get('formatters', EMPTY_DICT)
-                for name in formatters:
-                    try:
-                        formatters[name] = self.configure_formatter(
-                                                            formatters[name])
-                    except StandardError, e:
-                        raise ValueError('Unable to configure '
-                                         'formatter %r: %s' % (name, e))
-                # Next, do filters - they don't refer to anything else, either
-                filters = config.get('filters', EMPTY_DICT)
-                for name in filters:
-                    try:
-                        filters[name] = self.configure_filter(filters[name])
-                    except StandardError, e:
-                        raise ValueError('Unable to configure '
-                                         'filter %r: %s' % (name, e))
-
-                # Next, do handlers - they refer to formatters and filters
-                # As handlers can refer to other handlers, sort the keys
-                # to allow a deterministic order of configuration
-                handlers = config.get('handlers', EMPTY_DICT)
-                for name in sorted(handlers):
-                    try:
-                        handler = self.configure_handler(handlers[name])
-                        handler.name = name
-                        handlers[name] = handler
-                    except StandardError, e:
-                        raise ValueError('Unable to configure handler '
-                                         '%r: %s' % (name, e))
-                # Next, do loggers - they refer to handlers and filters
-
-                #we don't want to lose the existing loggers,
-                #since other threads may have pointers to them.
-                #existing is set to contain all existing loggers,
-                #and as we go through the new configuration we
-                #remove any which are configured. At the end,
-                #what's left in existing is the set of loggers
-                #which were in the previous configuration but
-                #which are not in the new configuration.
-                root = logging.root
-                existing = root.manager.loggerDict.keys()
-                #The list needs to be sorted so that we can
-                #avoid disabling child loggers of explicitly
-                #named loggers. With a sorted list it is easier
-                #to find the child loggers.
-                existing.sort()
-                #We'll keep the list of existing loggers
-                #which are children of named loggers here...
-                child_loggers = []
-                #now set up the new ones...
-                loggers = config.get('loggers', EMPTY_DICT)
-                for name in loggers:
-                    name = _encoded(name)
-                    if name in existing:
-                        i = existing.index(name)
-                        prefixed = name + "."
-                        pflen = len(prefixed)
-                        num_existing = len(existing)
-                        i = i + 1 # look at the entry after name
-                        while (i < num_existing) and\
-                              (existing[i][:pflen] == prefixed):
-                            child_loggers.append(existing[i])
-                            i = i + 1
-                        existing.remove(name)
-                    try:
-                        self.configure_logger(name, loggers[name])
-                    except StandardError, e:
-                        raise ValueError('Unable to configure logger '
-                                         '%r: %s' % (name, e))
-
-                #Disable any old loggers. There's no point deleting
-                #them as other threads may continue to hold references
-                #and by disabling them, you stop them doing any logging.
-                #However, don't disable children of named loggers, as that's
-                #probably not what was intended by the user.
-                for log in existing:
-                    logger = root.manager.loggerDict[log]
-                    if log in child_loggers:
-                        logger.level = logging.NOTSET
-                        logger.handlers = []
-                        logger.propagate = True
-                    elif disable_existing:
-                        logger.disabled = True
-
-                # And finally, do the root logger
-                root = config.get('root', None)
-                if root:
-                    try:
-                        self.configure_root(root)
-                    except StandardError, e:
-                        raise ValueError('Unable to configure root '
-                                         'logger: %s' % e)
-        finally:
-            logging._releaseLock()
-
-    def configure_formatter(self, config):
-        """Configure a formatter from a dictionary."""
-        if '()' in config:
-            factory = config['()'] # for use in exception handler
-            try:
-                result = self.configure_custom(config)
-            except TypeError, te:
-                if "'format'" not in str(te):
-                    raise
-                #Name of parameter changed from fmt to format.
-                #Retry with old name.
-                #This is so that code can be used with older Python versions
-                #(e.g. by Django)
-                config['fmt'] = config.pop('format')
-                config['()'] = factory
-                result = self.configure_custom(config)
-        else:
-            fmt = config.get('format', None)
-            dfmt = config.get('datefmt', None)
-            result = logging.Formatter(fmt, dfmt)
-        return result
-
-    def configure_filter(self, config):
-        """Configure a filter from a dictionary."""
-        if '()' in config:
-            result = self.configure_custom(config)
-        else:
-            name = config.get('name', '')
-            result = logging.Filter(name)
-        return result
-
-    def add_filters(self, filterer, filters):
-        """Add filters to a filterer from a list of names."""
-        for f in filters:
-            try:
-                filterer.addFilter(self.config['filters'][f])
-            except StandardError, e:
-                raise ValueError('Unable to add filter %r: %s' % (f, e))
-
-    def configure_handler(self, config):
-        """Configure a handler from a dictionary."""
-        formatter = config.pop('formatter', None)
-        if formatter:
-            try:
-                formatter = self.config['formatters'][formatter]
-            except StandardError, e:
-                raise ValueError('Unable to set formatter '
-                                 '%r: %s' % (formatter, e))
-        level = config.pop('level', None)
-        filters = config.pop('filters', None)
-        if '()' in config:
-            c = config.pop('()')
-            if not hasattr(c, '__call__') and hasattr(types, 'ClassType') and type(c) != types.ClassType:
-                c = self.resolve(c)
-            factory = c
-        else:
-            klass = self.resolve(config.pop('class'))
-            #Special case for handler which refers to another handler
-            if issubclass(klass, logging.handlers.MemoryHandler) and\
-                'target' in config:
-                try:
-                    config['target'] = self.config['handlers'][config['target']]
-                except StandardError, e:
-                    raise ValueError('Unable to set target handler '
-                                     '%r: %s' % (config['target'], e))
-            elif issubclass(klass, logging.handlers.SMTPHandler) and\
-                'mailhost' in config:
-                config['mailhost'] = self.as_tuple(config['mailhost'])
-            elif issubclass(klass, logging.handlers.SysLogHandler) and\
-                'address' in config:
-                config['address'] = self.as_tuple(config['address'])
-            factory = klass
-        kwargs = dict([(k, config[k]) for k in config if valid_ident(k)])
-        try:
-            result = factory(**kwargs)
-        except TypeError, te:
-            if "'stream'" not in str(te):
-                raise
-            #The argument name changed from strm to stream
-            #Retry with old name.
-            #This is so that code can be used with older Python versions
-            #(e.g. by Django)
-            kwargs['strm'] = kwargs.pop('stream')
-            result = factory(**kwargs)
-        if formatter:
-            result.setFormatter(formatter)
-        if level is not None:
-            result.setLevel(logging._checkLevel(level))
-        if filters:
-            self.add_filters(result, filters)
-        return result
-
-    def add_handlers(self, logger, handlers):
-        """Add handlers to a logger from a list of names."""
-        for h in handlers:
-            try:
-                logger.addHandler(self.config['handlers'][h])
-            except StandardError, e:
-                raise ValueError('Unable to add handler %r: %s' % (h, e))
-
-    def common_logger_config(self, logger, config, incremental=False):
-        """
-        Perform configuration which is common to root and non-root loggers.
-        """
-        level = config.get('level', None)
-        if level is not None:
-            logger.setLevel(logging._checkLevel(level))
-        if not incremental:
-            #Remove any existing handlers
-            for h in logger.handlers[:]:
-                logger.removeHandler(h)
-            handlers = config.get('handlers', None)
-            if handlers:
-                self.add_handlers(logger, handlers)
-            filters = config.get('filters', None)
-            if filters:
-                self.add_filters(logger, filters)
-
-    def configure_logger(self, name, config, incremental=False):
-        """Configure a non-root logger from a dictionary."""
-        logger = logging.getLogger(name)
-        self.common_logger_config(logger, config, incremental)
-        propagate = config.get('propagate', None)
-        if propagate is not None:
-            logger.propagate = propagate
-
-    def configure_root(self, config, incremental=False):
-        """Configure a root logger from a dictionary."""
-        root = logging.getLogger()
-        self.common_logger_config(root, config, incremental)
-
-dictConfigClass = DictConfigurator
-
-def dictConfig(config):
-    """Configure logging using a dictionary."""
-    dictConfigClass(config).configure()
-
-
-def listen(port=DEFAULT_LOGGING_CONFIG_PORT):
-    """
-    Start up a socket server on the specified port, and listen for new
-    configurations.
-
-    These will be sent as a file suitable for processing by fileConfig().
-    Returns a Thread object on which you can call start() to start the server,
-    and which you can join() when appropriate. To stop the server, call
-    stopListening().
-    """
-    if not thread:
-        raise NotImplementedError("listen() needs threading to work")
-
-    class ConfigStreamHandler(StreamRequestHandler):
-        """
-        Handler for a logging configuration request.
-
-        It expects a completely new logging configuration and uses fileConfig
-        to install it.
-        """
-        def handle(self):
-            """
-            Handle a request.
-
-            Each request is expected to be a 4-byte length, packed using
-            struct.pack(">L", n), followed by the config file.
-            Uses fileConfig() to do the grunt work.
-            """
-            import tempfile
-            try:
-                conn = self.connection
-                chunk = conn.recv(4)
-                if len(chunk) == 4:
-                    slen = struct.unpack(">L", chunk)[0]
-                    chunk = self.connection.recv(slen)
-                    while len(chunk) < slen:
-                        chunk = chunk + conn.recv(slen - len(chunk))
-                    try:
-                        import json
-                        d =json.loads(chunk)
-                        assert isinstance(d, dict)
-                        dictConfig(d)
-                    except:
-                        #Apply new configuration.
-
-                        file = cStringIO.StringIO(chunk)
-                        try:
-                            fileConfig(file)
-                        except (KeyboardInterrupt, SystemExit):
-                            raise
-                        except:
-                            traceback.print_exc()
-                    if self.server.ready:
-                        self.server.ready.set()
-            except socket.error, e:
-                if not isinstance(e.args, tuple):
-                    raise
-                else:
-                    errcode = e.args[0]
-                    if errcode != RESET_ERROR:
-                        raise
-
-    class ConfigSocketReceiver(ThreadingTCPServer):
-        """
-        A simple TCP socket-based logging config receiver.
-        """
-
-        allow_reuse_address = 1
-
-        def __init__(self, host='localhost', port=DEFAULT_LOGGING_CONFIG_PORT,
-                     handler=None, ready=None):
-            ThreadingTCPServer.__init__(self, (host, port), handler)
-            logging._acquireLock()
-            self.abort = 0
-            logging._releaseLock()
-            self.timeout = 1
-            self.ready = ready
-
-        def serve_until_stopped(self):
-            if sys.platform.startswith('java'):
-                from select import cpython_compatible_select as select
-            else:
-                from select import select
-            abort = 0
-            while not abort:
-                rd, wr, ex = select([self.socket.fileno()],
-                                     [], [],
-                                     self.timeout)
-                if rd:
-                    self.handle_request()
-                logging._acquireLock()
-                abort = self.abort
-                logging._releaseLock()
-            self.socket.close()
-
-    class Server(threading.Thread):
-
-        def __init__(self, rcvr, hdlr, port):
-            super(Server, self).__init__()
-            self.rcvr = rcvr
-            self.hdlr = hdlr
-            self.port = port
-            self.ready = threading.Event()
-
-        def run(self):
-            server = self.rcvr(port=self.port, handler=self.hdlr,
-                               ready=self.ready)
-            if self.port == 0:
-                self.port = server.server_address[1]
-            self.ready.set()
-            global _listener
-            logging._acquireLock()
-            _listener = server
-            logging._releaseLock()
-            server.serve_until_stopped()
-
-    return Server(ConfigSocketReceiver, ConfigStreamHandler, port)
-
-def stopListening():
-    """
-    Stop the listening server which was created with a call to listen().
-    """
-    global _listener
-    logging._acquireLock()
-    try:
-        if _listener:
-            _listener.abort = 1
-            _listener = None
-    finally:
-        logging._releaseLock()
diff --git a/Lib/netrc.py b/Lib/netrc.py
deleted file mode 100644
--- a/Lib/netrc.py
+++ /dev/null
@@ -1,119 +0,0 @@
-"""An object-oriented interface to .netrc files."""
-
-# Module and documentation by Eric S. Raymond, 21 Dec 1998
-
-from __future__ import with_statement
-import os, shlex
-
-__all__ = ["netrc", "NetrcParseError"]
-
-
-class NetrcParseError(Exception):
-    """Exception raised on syntax errors in the .netrc file."""
-    def __init__(self, msg, filename=None, lineno=None):
-        self.filename = filename
-        self.lineno = lineno
-        self.msg = msg
-        Exception.__init__(self, msg)
-
-    def __str__(self):
-        return "%s (%s, line %s)" % (self.msg, self.filename, self.lineno)
-
-
-class netrc:
-    def __init__(self, file=None):
-        if file is None:
-            try:
-                file = os.path.join(os.environ['HOME'], ".netrc")
-            except KeyError:
-                raise IOError("Could not find .netrc: $HOME is not set")
-        self.hosts = {}
-        self.macros = {}
-        with open(file) as fp:
-            self._parse(file, fp)
-
-    def _parse(self, file, fp):
-        lexer = shlex.shlex(fp)
-        lexer.wordchars += r"""!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~"""
-        lexer.commenters = lexer.commenters.replace('#', '')
-        while 1:
-            # Look for a machine, default, or macdef top-level keyword
-            toplevel = tt = lexer.get_token()
-            if not tt:
-                break
-            elif tt[0] == '#':
-                fp.readline();
-                continue;
-            elif tt == 'machine':
-                entryname = lexer.get_token()
-            elif tt == 'default':
-                entryname = 'default'
-            elif tt == 'macdef':                # Just skip to end of macdefs
-                entryname = lexer.get_token()
-                self.macros[entryname] = []
-                lexer.whitespace = ' \t'
-                while 1:
-                    line = lexer.instream.readline()
-                    if not line or line == '\012':
-                        lexer.whitespace = ' \t\r\n'
-                        break
-                    self.macros[entryname].append(line)
-                continue
-            else:
-                raise NetrcParseError(
-                    "bad toplevel token %r" % tt, file, lexer.lineno)
-
-            # We're looking at start of an entry for a named machine or default.
-            login = ''
-            account = password = None
-            self.hosts[entryname] = {}
-            while 1:
-                tt = lexer.get_token()
-                if (tt=='' or tt == 'machine' or
-                    tt == 'default' or tt =='macdef'):
-                    if password:
-                        self.hosts[entryname] = (login, account, password)
-                        lexer.push_token(tt)
-                        break
-                    else:
-                        raise NetrcParseError(
-                            "malformed %s entry %s terminated by %s"
-                            % (toplevel, entryname, repr(tt)),
-                            file, lexer.lineno)
-                elif tt == 'login' or tt == 'user':
-                    login = lexer.get_token()
-                elif tt == 'account':
-                    account = lexer.get_token()
-                elif tt == 'password':
-                    password = lexer.get_token()
-                else:
-                    raise NetrcParseError("bad follower token %r" % tt,
-                                          file, lexer.lineno)
-
-    def authenticators(self, host):
-        """Return a (user, account, password) tuple for given host."""
-        if host in self.hosts:
-            return self.hosts[host]
-        elif 'default' in self.hosts:
-            return self.hosts['default']
-        else:
-            return None
-
-    def __repr__(self):
-        """Dump the class data in the format of a .netrc file."""
-        rep = ""
-        for host in self.hosts.keys():
-            attrs = self.hosts[host]
-            rep = rep + "machine "+ host + "\n\tlogin " + repr(attrs[0]) + "\n"
-            if attrs[1]:
-                rep = rep + "account " + repr(attrs[1])
-            rep = rep + "\tpassword " + repr(attrs[2]) + "\n"
-        for macro in self.macros.keys():
-            rep = rep + "macdef " + macro + "\n"
-            for line in self.macros[macro]:
-                rep = rep + line
-            rep = rep + "\n"
-        return rep
-
-if __name__ == '__main__':
-    print netrc()
diff --git a/Lib/robotparser.py b/Lib/robotparser.py
deleted file mode 100644
--- a/Lib/robotparser.py
+++ /dev/null
@@ -1,222 +0,0 @@
-""" robotparser.py
-
-    Copyright (C) 2000  Bastian Kleineidam
-
-    You can choose between two licenses when using this package:
-    1) GNU GPLv2
-    2) PSF license for Python 2.2
-
-    The robots.txt Exclusion Protocol is implemented as specified in
-    http://info.webcrawler.com/mak/projects/robots/norobots-rfc.html
-"""
-import urlparse
-import urllib
-
-__all__ = ["RobotFileParser"]
-
-
-class RobotFileParser:
-    """ This class provides a set of methods to read, parse and answer
-    questions about a single robots.txt file.
-
-    """
-
-    def __init__(self, url=''):
-        self.entries = []
-        self.default_entry = None
-        self.disallow_all = False
-        self.allow_all = False
-        self.set_url(url)
-        self.last_checked = 0
-
-    def mtime(self):
-        """Returns the time the robots.txt file was last fetched.
-
-        This is useful for long-running web spiders that need to
-        check for new robots.txt files periodically.
-
-        """
-        return self.last_checked
-
-    def modified(self):
-        """Sets the time the robots.txt file was last fetched to the
-        current time.
-
-        """
-        import time
-        self.last_checked = time.time()
-
-    def set_url(self, url):
-        """Sets the URL referring to a robots.txt file."""
-        self.url = url
-        self.host, self.path = urlparse.urlparse(url)[1:3]
-
-    def read(self):
-        """Reads the robots.txt URL and feeds it to the parser."""
-        opener = URLopener()
-        f = opener.open(self.url)
-        lines = [line.strip() for line in f]
-        f.close()
-        self.errcode = opener.errcode
-        if self.errcode in (401, 403):
-            self.disallow_all = True
-        elif self.errcode >= 400:
-            self.allow_all = True
-        elif self.errcode == 200 and lines:
-            self.parse(lines)
-
-    def _add_entry(self, entry):
-        if "*" in entry.useragents:
-            # the default entry is considered last
-            if self.default_entry is None:
-                # the first default entry wins
-                self.default_entry = entry
-        else:
-            self.entries.append(entry)
-
-    def parse(self, lines):
-        """parse the input lines from a robots.txt file.
-           We allow that a user-agent: line is not preceded by
-           one or more blank lines."""
-        # states:
-        #   0: start state
-        #   1: saw user-agent line
-        #   2: saw an allow or disallow line
-        state = 0
-        linenumber = 0
-        entry = Entry()
-
-        for line in lines:
-            linenumber += 1
-            if not line:
-                if state == 1:
-                    entry = Entry()
-                    state = 0
-                elif state == 2:
-                    self._add_entry(entry)
-                    entry = Entry()
-                    state = 0
-            # remove optional comment and strip line
-            i = line.find('#')
-            if i >= 0:
-                line = line[:i]
-            line = line.strip()
-            if not line:
-                continue
-            line = line.split(':', 1)
-            if len(line) == 2:
-                line[0] = line[0].strip().lower()
-                line[1] = urllib.unquote(line[1].strip())
-                if line[0] == "user-agent":
-                    if state == 2:
-                        self._add_entry(entry)
-                        entry = Entry()
-                    entry.useragents.append(line[1])
-                    state = 1
-                elif line[0] == "disallow":
-                    if state != 0:
-                        entry.rulelines.append(RuleLine(line[1], False))
-                        state = 2
-                elif line[0] == "allow":
-                    if state != 0:
-                        entry.rulelines.append(RuleLine(line[1], True))
-                        state = 2
-        if state == 2:
-            self._add_entry(entry)
-
-
-    def can_fetch(self, useragent, url):
-        """using the parsed robots.txt decide if useragent can fetch url"""
-        if self.disallow_all:
-            return False
-        if self.allow_all:
-            return True
-        # search for given user agent matches
-        # the first match counts
-        parsed_url = urlparse.urlparse(urllib.unquote(url))
-        url = urlparse.urlunparse(('', '', parsed_url.path,
-            parsed_url.params, parsed_url.query, parsed_url.fragment))
-        url = urllib.quote(url)
-        if not url:
-            url = "/"
-        for entry in self.entries:
-            if entry.applies_to(useragent):
-                return entry.allowance(url)
-        # try the default entry last
-        if self.default_entry:
-            return self.default_entry.allowance(url)
-        # agent not found ==> access granted
-        return True
-
-
-    def __str__(self):
-        return ''.join([str(entry) + "\n" for entry in self.entries])
-
-
-class RuleLine:
-    """A rule line is a single "Allow:" (allowance==True) or "Disallow:"
-       (allowance==False) followed by a path."""
-    def __init__(self, path, allowance):
-        if path == '' and not allowance:
-            # an empty value means allow all
-            allowance = True
-        self.path = urllib.quote(path)
-        self.allowance = allowance
-
-    def applies_to(self, filename):
-        return self.path == "*" or filename.startswith(self.path)
-
-    def __str__(self):
-        return (self.allowance and "Allow" or "Disallow") + ": " + self.path
-
-
-class Entry:
-    """An entry has one or more user-agents and zero or more rulelines"""
-    def __init__(self):
-        self.useragents = []
-        self.rulelines = []
-
-    def __str__(self):
-        ret = []
-        for agent in self.useragents:
-            ret.extend(["User-agent: ", agent, "\n"])
-        for line in self.rulelines:
-            ret.extend([str(line), "\n"])
-        return ''.join(ret)
-
-    def applies_to(self, useragent):
-        """check if this entry applies to the specified agent"""
-        # split the name token and make it lower case
-        useragent = useragent.split("/")[0].lower()
-        for agent in self.useragents:
-            if agent == '*':
-                # we have the catch-all agent
-                return True
-            agent = agent.lower()
-            if agent in useragent:
-                return True
-        return False
-
-    def allowance(self, filename):
-        """Preconditions:
-        - our agent applies to this entry
-        - filename is URL decoded"""
-        for line in self.rulelines:
-            if line.applies_to(filename):
-                return line.allowance
-        return True
-
-class URLopener(urllib.FancyURLopener):
-    def __init__(self, *args):
-        urllib.FancyURLopener.__init__(self, *args)
-        self.errcode = 200
-
-    def prompt_user_passwd(self, host, realm):
-        ## If robots.txt file is accessible only with a password,
-        ## we act as if the file wasn't there.
-        return None, None
-
-    def http_error_default(self, url, fp, errcode, errmsg, headers):
-        self.errcode = errcode
-        return urllib.FancyURLopener.http_error_default(self, url, fp, errcode,
-                                                        errmsg, headers)
diff --git a/Lib/select.py b/Lib/select.py
--- a/Lib/select.py
+++ b/Lib/select.py
@@ -1,264 +1,12 @@
-"""
-This is an select module for use on JVMs >= 1.5.
-It is documented, along with known issues and workarounds, on the jython wiki.
-http://wiki.python.org/jython/SelectModule
-"""
+# dispatches to _socket for actual implementation
 
-import java.nio.channels.SelectableChannel
-import java.nio.channels.SelectionKey
-import java.nio.channels.Selector
-from java.nio.channels.SelectionKey import OP_ACCEPT, OP_CONNECT, OP_WRITE, OP_READ
-
-import errno
-import os
-import Queue
-import socket
-
-class error(Exception): pass
-
-ALL = None
-
-_exception_map = {
-
-# (<javaexception>, <circumstance>) : lambda: <code that raises the python equivalent>
-
-(java.nio.channels.ClosedChannelException, ALL) : error(errno.ENOTCONN, 'Socket is not connected'),
-(java.nio.channels.CancelledKeyException, ALL) : error(errno.ENOTCONN, 'Socket is not connected'),
-(java.nio.channels.IllegalBlockingModeException, ALL) : error(errno.ESOCKISBLOCKING, 'socket must be in non-blocking mode'),
-}
-
-def _map_exception(exc, circumstance=ALL):
-    try:
-        mapped_exception = _exception_map[(exc.__class__, circumstance)]
-        mapped_exception.java_exception = exc
-        return mapped_exception
-    except KeyError:
-        return error(-1, 'Unmapped java exception: <%s:%s>' % (exc.toString(), circumstance))
-
-POLLIN   = 1
-POLLOUT  = 2
-
-# The following event types are completely ignored on jython
-# Java does not support them, AFAICT
-# They are declared only to support code compatibility with cpython
-
-POLLPRI  = 4
-POLLERR  = 8
-POLLHUP  = 16
-POLLNVAL = 32
-
-def _getselectable(selectable_object):
-    try:
-        channel = selectable_object.getchannel()
-    except:
-        try:
-            channel = selectable_object.fileno().getChannel()
-        except:
-            raise TypeError("Object '%s' is not watchable" % selectable_object,
-                            errno.ENOTSOCK)
-    
-    if channel and not isinstance(channel, java.nio.channels.SelectableChannel):
-        raise TypeError("Object '%s' is not watchable" % selectable_object,
-                        errno.ENOTSOCK)
-    return channel
-
-class poll:
-
-    def __init__(self):
-        self.selector = java.nio.channels.Selector.open()
-        self.chanmap = {}
-        self.unconnected_sockets = []
-
-    def _register_channel(self, socket_object, channel, mask):
-        jmask = 0
-        if mask & POLLIN:
-            # Note that OP_READ is NOT a valid event on server socket channels.
-            if channel.validOps() & OP_ACCEPT:
-                jmask = OP_ACCEPT
-            else:
-                jmask = OP_READ
-        if mask & POLLOUT:
-            if channel.validOps() & OP_WRITE:
-                jmask |= OP_WRITE
-            if channel.validOps() & OP_CONNECT:
-                jmask |= OP_CONNECT
-        selectionkey = channel.register(self.selector, jmask)
-        self.chanmap[channel] = (socket_object, selectionkey)
-
-    def _check_unconnected_sockets(self):
-        temp_list = []
-        for socket_object, mask in self.unconnected_sockets:
-            channel = _getselectable(socket_object)
-            if channel is not None:
-                self._register_channel(socket_object, channel, mask)
-            else:
-                temp_list.append( (socket_object, mask) )
-        self.unconnected_sockets = temp_list
-
-    def register(self, socket_object, mask = POLLIN|POLLOUT|POLLPRI):
-        try:
-            channel = _getselectable(socket_object)
-            if channel is None:
-                # The socket is not yet connected, and thus has no channel
-                # Add it to a pending list, and return
-                self.unconnected_sockets.append( (socket_object, mask) )
-                return
-            self._register_channel(socket_object, channel, mask)
-        except java.lang.Exception, jlx:
-            raise _map_exception(jlx)
-
-    def unregister(self, socket_object):
-        try:
-            channel = _getselectable(socket_object)
-            self.chanmap[channel][1].cancel()
-            del self.chanmap[channel]
-        except java.lang.Exception, jlx:
-            raise _map_exception(jlx)
-
-    def _dopoll(self, timeout):
-        if timeout is None or timeout < 0:
-            self.selector.select()
-        else:
-            try:
-                timeout = int(timeout)
-                if not timeout:
-                    self.selector.selectNow()
-                else:
-                    # No multiplication required: both cpython and java use millisecond timeouts
-                    self.selector.select(timeout)
-            except ValueError, vx:
-                raise error("poll timeout must be a number of milliseconds or None", errno.EINVAL)
-        # The returned selectedKeys cannot be used from multiple threads!
-        return self.selector.selectedKeys()
-
-    def poll(self, timeout=None):
-        try:
-            self._check_unconnected_sockets()
-            selectedkeys = self._dopoll(timeout)
-            results = []
-            for k in selectedkeys.iterator():
-                jmask = k.readyOps()
-                pymask = 0
-                if jmask & OP_READ: pymask |= POLLIN
-                if jmask & OP_WRITE: pymask |= POLLOUT
-                if jmask & OP_ACCEPT: pymask |= POLLIN
-                if jmask & OP_CONNECT: pymask |= POLLOUT
-                # Now return the original userobject, and the return event mask
-                results.append( (self.chanmap[k.channel()][0], pymask) )
-            return results
-        except java.lang.Exception, jlx:
-            raise _map_exception(jlx)
-
-    def _deregister_all(self):
-        try:
-            for k in self.selector.keys():
-                k.cancel()
-            # Keys are not actually removed from the selector until the next select operation.
-            self.selector.selectNow()
-        except java.lang.Exception, jlx:
-            raise _map_exception(jlx)
-
-    def close(self):
-        try:
-            self._deregister_all()
-            self.selector.close()
-        except java.lang.Exception, jlx:
-            raise _map_exception(jlx)
-
-def _calcselecttimeoutvalue(value):
-    if value is None:
-        return None
-    try:
-        floatvalue = float(value)
-    except Exception, x:
-        raise TypeError("Select timeout value must be a number or None")
-    if value < 0:
-        raise error("Select timeout value cannot be negative", errno.EINVAL)
-    if floatvalue < 0.000001:
-        return 0
-    return int(floatvalue * 1000) # Convert to milliseconds
-
-# This cache for poll objects is required because of a bug in java on MS Windows
-# http://bugs.jython.org/issue1291
-
-class poll_object_cache:
-
-    def __init__(self):
-        self.is_windows = os._name == 'nt'
-        if self.is_windows:
-            self.poll_object_queue = Queue.Queue()
-        import atexit
-        atexit.register(self.finalize)
-
-    def get_poll_object(self):
-        if not self.is_windows:
-            return poll()
-        try:
-            return self.poll_object_queue.get(False)
-        except Queue.Empty:
-            return poll()
-
-    def release_poll_object(self, pobj):
-        if self.is_windows:
-            pobj._deregister_all()
-            self.poll_object_queue.put(pobj)
-        else:
-            pobj.close()
-
-    def finalize(self):
-        if self.is_windows:
-            while True:
-                try:
-                    p = self.poll_object_queue.get(False)
-                    p.close()
-                except Queue.Empty:
-                    return
-
-_poll_object_cache = poll_object_cache()
-
-def native_select(read_fd_list, write_fd_list, outofband_fd_list, timeout=None):
-    timeout = _calcselecttimeoutvalue(timeout)
-    # First create a poll object to do the actual watching.
-    pobj = _poll_object_cache.get_poll_object()
-    try:
-        registered_for_read = {}
-        # Check the read list
-        for fd in read_fd_list:
-            pobj.register(fd, POLLIN)
-            registered_for_read[fd] = 1
-        # And now the write list
-        for fd in write_fd_list:
-            if fd in registered_for_read:
-                # registering a second time overwrites the first
-                pobj.register(fd, POLLIN|POLLOUT)
-            else:
-                pobj.register(fd, POLLOUT)
-        results = pobj.poll(timeout)
-        # Now start preparing the results
-        read_ready_list, write_ready_list, oob_ready_list = [], [], []
-        for fd, mask in results:
-            if mask & POLLIN:
-                read_ready_list.append(fd)
-            if mask & POLLOUT:
-                write_ready_list.append(fd)
-        return read_ready_list, write_ready_list, oob_ready_list
-    finally:
-        _poll_object_cache.release_poll_object(pobj)
-
-select = native_select
-
-def cpython_compatible_select(read_fd_list, write_fd_list, outofband_fd_list, timeout=None):
-    # First turn all sockets to non-blocking
-    # keeping track of which ones have changed
-    modified_channels = []
-    try:
-        for socket_list in [read_fd_list, write_fd_list, outofband_fd_list]:
-            for s in socket_list:
-                channel = _getselectable(s)
-                if channel.isBlocking():
-                    modified_channels.append(channel)
-                    channel.configureBlocking(0)
-        return native_select(read_fd_list, write_fd_list, outofband_fd_list, timeout)
-    finally:
-        for channel in modified_channels:
-            channel.configureBlocking(1)
+from _socket import (
+    POLLIN,
+    POLLOUT,
+    POLLPRI,
+    POLLERR,
+    POLLHUP,
+    POLLNVAL,
+    error,
+    #poll,
+    select)
diff --git a/Lib/socket.py b/Lib/socket.py
--- a/Lib/socket.py
+++ b/Lib/socket.py
@@ -1,1889 +1,142 @@
-"""
-This is an updated socket module for use on JVMs >= 1.5; it is derived from the old jython socket module.
-It is documented, along with known issues and workarounds, on the jython wiki.
-http://wiki.python.org/jython/NewSocketModule
-"""
+# dispatches to _socket for actual implementation
 
-_defaulttimeout = None
+from _socket import (
+    socket, error, herror, gaierror, timeout, has_ipv6,
 
-import errno
-import jarray
-import string
-import struct
-import sys
-import threading
-import time
-import types
+    create_connection,
 
-# Java.io classes
-import java.io.BufferedInputStream
-import java.io.BufferedOutputStream
-# Java.io exceptions
-import java.io.InterruptedIOException
-import java.io.IOException
+    getdefaulttimeout,
+    setdefaulttimeout,
+    
+    getfqdn,
+    gethostbyaddr,
+    gethostbyname,
+    gethostbyname_ex,
+    gethostname,
+    getprotobyname,
+    getservbyname,
+    getservbyport,
 
-# Java.lang classes
-import java.lang.String
-# Java.lang exceptions
-import java.lang.Exception
+    AF_UNSPEC,
+    AF_INET,
+    AF_INET6,
 
-# Java.net classes
-import java.net.DatagramPacket
-import java.net.InetAddress
-import java.net.InetSocketAddress
-import java.net.Socket
-# Java.net exceptions
-import java.net.BindException
-import java.net.ConnectException
-import java.net.NoRouteToHostException
-import java.net.PortUnreachableException
-import java.net.ProtocolException
-import java.net.SocketException
-import java.net.SocketTimeoutException
-import java.net.UnknownHostException
+    AI_PASSIVE,
+    AI_CANONNAME,
+    AI_NUMERICHOST,
+    AI_V4MAPPED,
+    AI_ALL,
+    AI_ADDRCONFIG,
+    AI_NUMERICSERV,
 
-# Java.nio classes
-import java.nio.ByteBuffer
-import java.nio.channels.DatagramChannel
-import java.nio.channels.ServerSocketChannel
-import java.nio.channels.SocketChannel
-# Java.nio exceptions
-import java.nio.channels.AlreadyConnectedException
-import java.nio.channels.AsynchronousCloseException
-import java.nio.channels.CancelledKeyException
-import java.nio.channels.ClosedByInterruptException
-import java.nio.channels.ClosedChannelException
-import java.nio.channels.ClosedSelectorException
-import java.nio.channels.ConnectionPendingException
-import java.nio.channels.IllegalBlockingModeException
-import java.nio.channels.IllegalSelectorException
-import java.nio.channels.NoConnectionPendingException
-import java.nio.channels.NonReadableChannelException
-import java.nio.channels.NonWritableChannelException
-import java.nio.channels.NotYetBoundException
-import java.nio.channels.NotYetConnectedException
-import java.nio.channels.UnresolvedAddressException
-import java.nio.channels.UnsupportedAddressTypeException
+    EAI_NONAME,
+    EAI_SERVICE,
+    EAI_ADDRFAMILY,
+    
+    NI_NUMERICHOST,
+    NI_NUMERICSERV,
+    NI_NOFQDN,
+    NI_NAMEREQD,
+    NI_DGRAM,
+    NI_MAXSERV,
+    NI_IDN,
+    NI_IDN_ALLOW_UNASSIGNED,
+    NI_IDN_USE_STD3_ASCII_RULES,
+    NI_MAXHOST,
 
-# Javax.net.ssl classes
-import javax.net.ssl.SSLSocketFactory
-# Javax.net.ssl exceptions
-javax.net.ssl.SSLException
-javax.net.ssl.SSLHandshakeException
-javax.net.ssl.SSLKeyException
-javax.net.ssl.SSLPeerUnverifiedException
-javax.net.ssl.SSLProtocolException
+    SHUT_RD,
+    SHUT_WR,
+    SHUT_RDWR,
 
-import org.python.core.io.DatagramSocketIO
-import org.python.core.io.ServerSocketIO
-import org.python.core.io.SocketIO
-from org.python.core.Py import newString as asPyString
+    SOCK_DGRAM,
+    SOCK_STREAM,
+    SOCK_RAW,
+    SOCK_RDM,
+    SOCK_SEQPACKET,
+    
+    SOL_SOCKET,
+    # not supported, but here for apparent completeness
+    IPPROTO_AH,
+    IPPROTO_DSTOPTS,
+    IPPROTO_ESP,
+    IPPROTO_FRAGMENT,
+    IPPROTO_GGP,
+    IPPROTO_HOPOPTS,
+    IPPROTO_ICMP,
+    IPPROTO_ICMPV6,
+    IPPROTO_IDP,
+    IPPROTO_IGMP,
+    IPPROTO_IP, # supported
+    # not supported
+    IPPROTO_IPV4,
+    IPPROTO_IPV6,
+    IPPROTO_MAX,
+    IPPROTO_ND,
+    IPPROTO_NONE,
+    IPPROTO_PUP,
+    IPPROTO_RAW,
+    IPPROTO_ROUTING,
+    IPPROTO_TCP, # supported
+    IPPROTO_UDP, # supported
 
-class error(IOError): pass
-class herror(error): pass
-class gaierror(error): pass
-class timeout(error): pass
-class sslerror(error): pass
+    # supported
+    SO_BROADCAST,
+    SO_KEEPALIVE,
+    SO_LINGER,
+    SO_RCVBUF,
+    SO_REUSEADDR,
+    SO_SNDBUF,
+    SO_TIMEOUT,
+    TCP_NODELAY,
 
-def _add_exception_attrs(exc):
-    setattr(exc, 'errno', exc[0])
-    setattr(exc, 'strerror', exc[1])
-    return exc
+    # pseudo options
+    SO_ACCEPTCONN,
+    SO_ERROR,
+    SO_TYPE,
 
-def _unmapped_exception(exc):
-    return _add_exception_attrs(error(-1, 'Unmapped exception: %s' % exc))
+    # unsupported, will return errno.ENOPROTOOPT if actually used
+    SO_OOBINLINE,
+    SO_DEBUG,
+    SO_DONTROUTE,
+    SO_EXCLUSIVEADDRUSE,
+    SO_RCVLOWAT,
+    SO_RCVTIMEO,
+    SO_REUSEPORT,
+    SO_SNDLOWAT,
+    SO_SNDTIMEO,
+    SO_USELOOPBACK,
+    
+    INADDR_ANY,
+    INADDR_BROADCAST,
+    IN6ADDR_ANY_INIT,
 
-def java_net_socketexception_handler(exc):
-    if exc.message.startswith("Address family not supported by protocol family"):
-        return _add_exception_attrs(error(errno.EAFNOSUPPORT, 
-                'Address family not supported by protocol family: See http://wiki.python.org/jython/NewSocketModule#IPV6_address_support'))
-    return _unmapped_exception(exc)
+    _GLOBAL_DEFAULT_TIMEOUT,
 
-def would_block_error(exc=None):
-    return _add_exception_attrs(error(errno.EWOULDBLOCK, 'The socket operation could not complete without blocking'))
+    is_ipv4_address, is_ipv6_address, is_ip_address,
+    getaddrinfo,
+    getnameinfo,
+    htons,
+    htonl,
+    ntohs,
+    ntohl,
+    inet_aton,
+    inet_ntoa,
+    inet_pton,
+    inet_ntop,
 
-ALL = None
+    _fileobject,
+    _get_jsockaddr
+)
 
-_ssl_message = ": Differences between the SSL socket behaviour of cpython vs. jython are explained on the wiki:  http://wiki.python.org/jython/NewSocketModule#SSL_Support"
 
-_exception_map = {
+def supports(feature):
+    # FIXME this seems to be Jython internals specific, and for
+    # testing only; consider removing since it really no longer
+    # matters
+ 
+    if feature == "idna":
+        return True
+    raise KeyError("Unknown feature", feature)
 
-# (<javaexception>, <circumstance>) : callable that raises the python equivalent exception, or None to stub out as unmapped
 
-(java.io.IOException, ALL)            : lambda x: error(errno.ECONNRESET, 'Software caused connection abort'),
-(java.io.InterruptedIOException, ALL) : lambda x: timeout(None, 'timed out'),
 
-(java.net.BindException, ALL)            : lambda x: error(errno.EADDRINUSE, 'Address already in use'),
-(java.net.ConnectException, ALL)         : lambda x: error(errno.ECONNREFUSED, 'Connection refused'),
-(java.net.NoRouteToHostException, ALL)   : lambda x: error(errno.EHOSTUNREACH, 'No route to host'),
-(java.net.PortUnreachableException, ALL) : None,
-(java.net.ProtocolException, ALL)        : None,
-(java.net.SocketException, ALL)          : java_net_socketexception_handler,
-(java.net.SocketTimeoutException, ALL)   : lambda x: timeout(None, 'timed out'),
-(java.net.UnknownHostException, ALL)     : lambda x: gaierror(errno.EGETADDRINFOFAILED, 'getaddrinfo failed'),
-
-(java.nio.channels.AlreadyConnectedException, ALL)       : lambda x: error(errno.EISCONN, 'Socket is already connected'),
-(java.nio.channels.AsynchronousCloseException, ALL)      : None,
-(java.nio.channels.CancelledKeyException, ALL)           : None,
-(java.nio.channels.ClosedByInterruptException, ALL)      : None,
-(java.nio.channels.ClosedChannelException, ALL)          : lambda x: error(errno.EPIPE, 'Socket closed'),
-(java.nio.channels.ClosedSelectorException, ALL)         : None,
-(java.nio.channels.ConnectionPendingException, ALL)      : None,
-(java.nio.channels.IllegalBlockingModeException, ALL)    : None,
-(java.nio.channels.IllegalSelectorException, ALL)        : None,
-(java.nio.channels.NoConnectionPendingException, ALL)    : None,
-(java.nio.channels.NonReadableChannelException, ALL)     : None,
-(java.nio.channels.NonWritableChannelException, ALL)     : None,
-(java.nio.channels.NotYetBoundException, ALL)            : None,
-(java.nio.channels.NotYetConnectedException, ALL)        : None,
-(java.nio.channels.UnresolvedAddressException, ALL)      : lambda x: gaierror(errno.EGETADDRINFOFAILED, 'getaddrinfo failed'),
-(java.nio.channels.UnsupportedAddressTypeException, ALL) : None,
-
-# These error codes are currently wrong: getting them correct is going to require
-# some investigation. Cpython 2.6 introduced extensive SSL support.
-
-(javax.net.ssl.SSLException, ALL)                        : lambda x: sslerror(-1, 'SSL exception'+_ssl_message),
-(javax.net.ssl.SSLHandshakeException, ALL)               : lambda x: sslerror(-1, 'SSL handshake exception'+_ssl_message),
-(javax.net.ssl.SSLKeyException, ALL)                     : lambda x: sslerror(-1, 'SSL key exception'+_ssl_message),
-(javax.net.ssl.SSLPeerUnverifiedException, ALL)          : lambda x: sslerror(-1, 'SSL peer unverified exception'+_ssl_message),
-(javax.net.ssl.SSLProtocolException, ALL)                : lambda x: sslerror(-1, 'SSL protocol exception'+_ssl_message),
-
-}
-
-def _map_exception(java_exception, circumstance=ALL):
-    mapped_exception = _exception_map.get((java_exception.__class__, circumstance))
-    if mapped_exception:
-        py_exception = mapped_exception(java_exception)
-    else:
-        py_exception = error(-1, 'Unmapped exception: %s' % java_exception)
-    setattr(py_exception, 'java_exception', java_exception)
-    return _add_exception_attrs(py_exception)
-
-from functools import wraps
-
-# Used to map java exceptions to the equivalent python exception
-# And to set the _last_error attribute on socket objects, to support SO_ERROR
-def raises_java_exception(method_or_function):
-    @wraps(method_or_function)
-    def handle_exception(*args, **kwargs):
-        is_socket = (len(args) > 0 and isinstance(args[0], _nonblocking_api_mixin))
-        try:
-            try:
-                return method_or_function(*args, **kwargs)
-            except java.lang.Exception, jlx:
-                raise _map_exception(jlx)
-        except error, e:
-            if is_socket:
-                setattr(args[0], '_last_error', e[0])
-            raise
-        else:
-            if is_socket:
-                setattr(args[0], '_last_error', 0)
-    return handle_exception
-
-_feature_support_map = {
-    'ipv6': True,
-    'idna': False,
-    'tipc': False,
-}
-
-def supports(feature, *args):
-    if len(args) == 1:
-        _feature_support_map[feature] = args[0]
-    return _feature_support_map.get(feature, False)
-
-MODE_BLOCKING    = 'block'
-MODE_NONBLOCKING = 'nonblock'
-MODE_TIMEOUT     = 'timeout'
-
-_permitted_modes = (MODE_BLOCKING, MODE_NONBLOCKING, MODE_TIMEOUT)
-
-SHUT_RD   = 0
-SHUT_WR   = 1
-SHUT_RDWR = 2
-
-AF_UNSPEC = 0
-AF_INET   = 2
-AF_INET6  = 23
-
-AI_PASSIVE     = 1
-AI_CANONNAME   = 2
-AI_NUMERICHOST = 4
-AI_V4MAPPED    = 8
-AI_ALL         = 16
-AI_ADDRCONFIG  = 32
-AI_NUMERICSERV = 1024
-
-EAI_NONAME     = -2
-EAI_SERVICE    = -8
-EAI_ADDRFAMILY = -9
-
-NI_NUMERICHOST              = 1
-NI_NUMERICSERV              = 2
-NI_NOFQDN                   = 4
-NI_NAMEREQD                 = 8
-NI_DGRAM                    = 16
-NI_MAXSERV                  = 32
-NI_IDN                      = 64
-NI_IDN_ALLOW_UNASSIGNED     = 128
-NI_IDN_USE_STD3_ASCII_RULES = 256
-NI_MAXHOST                  = 1025
-
-# For some reason, probably historical, SOCK_DGRAM and SOCK_STREAM are opposite values of what they are on cpython.
-# I.E. The following is the way they are on cpython
-# SOCK_STREAM    = 1
-# SOCK_DGRAM     = 2
-# At some point, we should probably switch them around, which *should* not affect anybody
-
-SOCK_DGRAM     = 1
-SOCK_STREAM    = 2
-SOCK_RAW       = 3 # not supported
-SOCK_RDM       = 4 # not supported
-SOCK_SEQPACKET = 5 # not supported
-
-SOL_SOCKET = 0xFFFF
-
-IPPROTO_AH       =  51 # not supported
-IPPROTO_DSTOPTS  =  60 # not supported
-IPPROTO_ESP      =  50 # not supported
-IPPROTO_FRAGMENT =  44 # not supported
-IPPROTO_GGP      =   3 # not supported
-IPPROTO_HOPOPTS  =   0 # not supported
-IPPROTO_ICMP     =   1 # not supported
-IPPROTO_ICMPV6   =  58 # not supported
-IPPROTO_IDP      =  22 # not supported
-IPPROTO_IGMP     =   2 # not supported
-IPPROTO_IP       =   0
-IPPROTO_IPV4     =   4 # not supported
-IPPROTO_IPV6     =  41 # not supported
-IPPROTO_MAX      = 256 # not supported
-IPPROTO_ND       =  77 # not supported
-IPPROTO_NONE     =  59 # not supported
-IPPROTO_PUP      =  12 # not supported
-IPPROTO_RAW      = 255 # not supported
-IPPROTO_ROUTING  =  43 # not supported
-IPPROTO_TCP      =   6
-IPPROTO_UDP      =  17
-
-SO_ACCEPTCONN  = 1
-SO_BROADCAST   = 2
-SO_ERROR       = 4
-SO_KEEPALIVE   = 8
-SO_LINGER      = 16
-SO_OOBINLINE   = 32
-SO_RCVBUF      = 64
-SO_REUSEADDR   = 128
-SO_SNDBUF      = 256
-SO_TIMEOUT     = 512
-SO_TYPE        = 1024
-
-TCP_NODELAY    = 2048
-
-INADDR_ANY = "0.0.0.0"
-INADDR_BROADCAST = "255.255.255.255"
-
-IN6ADDR_ANY_INIT = "::"
-
-# Options with negative constants are not supported
-# They are being added here so that code that refers to them
-# will not break with an AttributeError
-
-SO_DEBUG            = -1
-SO_DONTROUTE        = -1
-SO_EXCLUSIVEADDRUSE = -8
-SO_RCVLOWAT         = -16
-SO_RCVTIMEO         = -32
-SO_REUSEPORT        = -64
-SO_SNDLOWAT         = -128
-SO_SNDTIMEO         = -256
-SO_USELOOPBACK      = -512
-
-__all__ = [
-    # Families
-    'AF_UNSPEC', 'AF_INET', 'AF_INET6', 
-    # getaddrinfo and getnameinfo flags
-    'AI_PASSIVE', 'AI_CANONNAME', 'AI_NUMERICHOST', 'AI_V4MAPPED',
-    'AI_ALL', 'AI_ADDRCONFIG', 'AI_NUMERICSERV', 'EAI_NONAME', 
-    'EAI_SERVICE', 'EAI_ADDRFAMILY',
-    'NI_NUMERICHOST', 'NI_NUMERICSERV', 'NI_NOFQDN', 'NI_NAMEREQD',
-    'NI_DGRAM', 'NI_MAXSERV', 'NI_IDN', 'NI_IDN_ALLOW_UNASSIGNED',
-    'NI_IDN_USE_STD3_ASCII_RULES', 'NI_MAXHOST',
-    # socket types
-    'SOCK_DGRAM', 'SOCK_STREAM', 'SOCK_RAW', 'SOCK_RDM', 'SOCK_SEQPACKET',
-    # levels
-    'SOL_SOCKET',
-    # protocols
-    'IPPROTO_AH', 'IPPROTO_DSTOPTS', 'IPPROTO_ESP', 'IPPROTO_FRAGMENT',
-    'IPPROTO_GGP', 'IPPROTO_HOPOPTS', 'IPPROTO_ICMP', 'IPPROTO_ICMPV6',
-    'IPPROTO_IDP', 'IPPROTO_IGMP', 'IPPROTO_IP', 'IPPROTO_IPV4',
-    'IPPROTO_IPV6', 'IPPROTO_MAX', 'IPPROTO_ND', 'IPPROTO_NONE',
-    'IPPROTO_PUP', 'IPPROTO_RAW', 'IPPROTO_ROUTING', 'IPPROTO_TCP', 
-    'IPPROTO_UDP',
-    # Special hostnames
-    'INADDR_ANY', 'INADDR_BROADCAST', 'IN6ADDR_ANY_INIT',
-    # support socket options
-    'SO_BROADCAST', 'SO_KEEPALIVE', 'SO_LINGER', 'SO_OOBINLINE',
-    'SO_RCVBUF', 'SO_REUSEADDR', 'SO_SNDBUF', 'SO_TIMEOUT', 'TCP_NODELAY',
-    # unsupported socket options
-    'SO_ACCEPTCONN', 'SO_DEBUG', 'SO_DONTROUTE', 'SO_ERROR',
-    'SO_EXCLUSIVEADDRUSE', 'SO_RCVLOWAT', 'SO_RCVTIMEO', 'SO_REUSEPORT',
-    'SO_SNDLOWAT', 'SO_SNDTIMEO', 'SO_TYPE', 'SO_USELOOPBACK',
-    # functions
-    'getfqdn', 'gethostname', 'gethostbyname', 'gethostbyaddr',
-    'getservbyname', 'getservbyport', 'getprotobyname', 'getaddrinfo',
-    'getnameinfo', 'getdefaulttimeout', 'setdefaulttimeout', 'htons',
-    'htonl', 'ntohs', 'ntohl', 'inet_pton', 'inet_ntop', 'inet_aton',
-    'inet_ntoa', 'create_connection', 'socket', 'ssl',
-    # exceptions
-    'error', 'herror', 'gaierror', 'timeout', 'sslerror',
-    # classes
-    'SocketType', 
-    # Misc flags     
-    'has_ipv6', 'SHUT_RD', 'SHUT_WR', 'SHUT_RDWR',
-]
-
-def _constant_to_name(const_value, expected_name_starts):
-    sock_module = sys.modules['socket']
-    try:
-        for name in dir(sock_module):
-            if getattr(sock_module, name) is const_value:
-                for name_start in expected_name_starts:
-                    if name.startswith(name_start):
-                        return name
-        return "Unknown"
-    finally:
-        sock_module = None
-
-import _google_ipaddr_r234
-
-def _is_ip_address(addr, version=None):
-    try:
-        _google_ipaddr_r234.IPAddress(addr, version)
-        return True
-    except ValueError:
-        return False
-
-def is_ipv4_address(addr):
-    return _is_ip_address(addr, 4)
-
-def is_ipv6_address(addr):
-    return _is_ip_address(addr, 6)
-
-def is_ip_address(addr):
-    return _is_ip_address(addr)
-
-class _nio_impl:
-
-    timeout = None
-    mode = MODE_BLOCKING
-
-    def config(self, mode, timeout):
-        self.mode = mode
-        if self.mode == MODE_BLOCKING:
-            self.jchannel.configureBlocking(1)
-        if self.mode == MODE_NONBLOCKING:
-            self.jchannel.configureBlocking(0)
-        if self.mode == MODE_TIMEOUT:
-            self.jchannel.configureBlocking(1)
-            self._timeout_millis = int(timeout*1000)
-            self.jsocket.setSoTimeout(self._timeout_millis)
-
-    def getsockopt(self, level, option):
-        if (level, option) in self.options:
-            result = getattr(self.jsocket, "get%s" % self.options[ (level, option) ])()
-            if option == SO_LINGER:
-                if result == -1:
-                    enabled, linger_time = 0, 0
-                else:
-                    enabled, linger_time = 1, result
-                return struct.pack('ii', enabled, linger_time)
-            return result
-        else:
-            raise error(errno.ENOPROTOOPT, "Socket option '%s' (level '%s') not supported on socket(%s)" % \
-                (_constant_to_name(option, ['SO_', 'TCP_']), _constant_to_name(level, ['SOL_', 'IPPROTO_']), str(self.jsocket)))
-
-    def setsockopt(self, level, option, value):
-        if (level, option) in self.options:
-            if option == SO_LINGER:
-                values = struct.unpack('ii', value)
-                self.jsocket.setSoLinger(*values)
-            else:
-                getattr(self.jsocket, "set%s" % self.options[ (level, option) ])(value)
-        else:
-            raise error(errno.ENOPROTOOPT, "Socket option '%s' (level '%s') not supported on socket(%s)" % \
-                (_constant_to_name(option, ['SO_', 'TCP_']), _constant_to_name(level,  ['SOL_', 'IPPROTO_']), str(self.jsocket)))
-
-    def close(self):
-        self.jsocket.close()
-
-    def getchannel(self):
-        return self.jchannel
-
-    def fileno(self):
-        return self.socketio
-
-class _client_socket_impl(_nio_impl):
-
-    options = {
-        (SOL_SOCKET,  SO_KEEPALIVE):   'KeepAlive',
-        (SOL_SOCKET,  SO_LINGER):      'SoLinger',
-        (SOL_SOCKET,  SO_OOBINLINE):   'OOBInline',
-        (SOL_SOCKET,  SO_RCVBUF):      'ReceiveBufferSize',
-        (SOL_SOCKET,  SO_REUSEADDR):   'ReuseAddress',
-        (SOL_SOCKET,  SO_SNDBUF):      'SendBufferSize',
-        (SOL_SOCKET,  SO_TIMEOUT):     'SoTimeout',
-        (IPPROTO_TCP, TCP_NODELAY):    'TcpNoDelay',
-    }
-
-    def __init__(self, socket=None, pending_options=None):
-        if socket:
-            self.jchannel = socket.getChannel()
-        else:
-            self.jchannel = java.nio.channels.SocketChannel.open()
-        self.jsocket = self.jchannel.socket()
-        self.socketio = org.python.core.io.SocketIO(self.jchannel, 'rw')
-        if pending_options:
-            for level, optname in pending_options.keys():
-                self.setsockopt(level, optname, pending_options[ (level, optname) ])
-
-    def bind(self, jsockaddr, reuse_addr):
-        self.jsocket.setReuseAddress(reuse_addr)
-        self.jsocket.bind(jsockaddr)
-
-    def connect(self, jsockaddr):
-        if self.mode == MODE_TIMEOUT:
-            self.jsocket.connect (jsockaddr, self._timeout_millis)
-        else:
-            self.jchannel.connect(jsockaddr)
-
-    def finish_connect(self):
-        return self.jchannel.finishConnect()
-
-    def _do_read_net(self, buf):
-        # Need two separate implementations because the java.nio APIs do not support timeouts
-        return self.jsocket.getInputStream().read(buf)
-
-    def _do_read_nio(self, buf):
-        bytebuf = java.nio.ByteBuffer.wrap(buf)
-        count = self.jchannel.read(bytebuf)
-        return count
-
-    def _do_write_net(self, buf):
-        self.jsocket.getOutputStream().write(buf)
-        return len(buf)
-
-    def _do_write_nio(self, buf):
-        bytebuf = java.nio.ByteBuffer.wrap(buf)
-        count = self.jchannel.write(bytebuf)
-        return count
-
-    def read(self, buf):
-        if self.mode == MODE_TIMEOUT:
-            return self._do_read_net(buf)
-        else:
-            return self._do_read_nio(buf)
-
-    def write(self, buf):
-        if self.mode == MODE_TIMEOUT:
-            return self._do_write_net(buf)
-        else:
-            return self._do_write_nio(buf)
-
-    def shutdown(self, how):
-        if how in (SHUT_RD, SHUT_RDWR):
-            self.jsocket.shutdownInput()
-        if how in (SHUT_WR, SHUT_RDWR):
-            self.jsocket.shutdownOutput()
-
-    def getsockname(self):
-        return (self.jsocket.getLocalAddress().getHostAddress(), self.jsocket.getLocalPort())
-
-    def getpeername(self):
-        return (self.jsocket.getInetAddress().getHostAddress(), self.jsocket.getPort() )
-
-class _server_socket_impl(_nio_impl):
-
-    options = {
-        (SOL_SOCKET, SO_RCVBUF):      'ReceiveBufferSize',
-        (SOL_SOCKET, SO_REUSEADDR):   'ReuseAddress',
-        (SOL_SOCKET, SO_TIMEOUT):     'SoTimeout',
-    }
-
-    def __init__(self, jsockaddr, backlog, reuse_addr):
-        self.pending_client_options = {}
-        self.jchannel = java.nio.channels.ServerSocketChannel.open()
-        self.jsocket = self.jchannel.socket()
-        self.jsocket.setReuseAddress(reuse_addr)
-        self.jsocket.bind(jsockaddr, backlog)
-        self.socketio = org.python.core.io.ServerSocketIO(self.jchannel, 'rw')
-
-    def accept(self):
-        if self.mode in (MODE_BLOCKING, MODE_NONBLOCKING):
-            new_cli_chan = self.jchannel.accept()
-            if new_cli_chan is not None:
-                return _client_socket_impl(new_cli_chan.socket(), self.pending_client_options)
-            else:
-                return None
-        else:
-            # In timeout mode now
-            new_cli_sock = self.jsocket.accept()
-            return _client_socket_impl(new_cli_sock, self.pending_client_options)
-
-    def shutdown(self, how):
-        # This is no-op on java, for server sockets.
-        # What the user wants to achieve is achieved by calling close() on
-        # java/jython. But we can't call that here because that would then
-        # later cause the user explicit close() call to fail
-        pass
-
-    def getsockopt(self, level, option):
-        if self.options.has_key( (level, option) ):
-            return _nio_impl.getsockopt(self, level, option)
-        elif _client_socket_impl.options.has_key( (level, option) ):
-            return self.pending_client_options.get( (level, option), None)
-        else:
-            raise error(errno.ENOPROTOOPT, "Socket option '%s' (level '%s') not supported on socket(%s)" % \
-                (_constant_to_name(option, ['SO_', 'TCP_']), _constant_to_name(level,  ['SOL_', 'IPPROTO_']), str(self.jsocket)))
-
-    def setsockopt(self, level, option, value):
-        if self.options.has_key( (level, option) ):
-            _nio_impl.setsockopt(self, level, option, value)
-        elif _client_socket_impl.options.has_key( (level, option) ):
-            self.pending_client_options[ (level, option) ] = value
-        else:
-            raise error(errno.ENOPROTOOPT, "Socket option '%s' (level '%s') not supported on socket(%s)" % \
-                (_constant_to_name(option, ['SO_', 'TCP_']), _constant_to_name(level,  ['SOL_', 'IPPROTO_']), str(self.jsocket)))
-
-    def getsockname(self):
-        return (self.jsocket.getInetAddress().getHostAddress(), self.jsocket.getLocalPort())
-
-    def getpeername(self):
-        # Not a meaningful operation for server sockets.
-        raise error(errno.ENOTCONN, "Socket is not connected")
-
-class _datagram_socket_impl(_nio_impl):
-
-    options = {
-        (SOL_SOCKET, SO_BROADCAST):   'Broadcast',
-        (SOL_SOCKET, SO_RCVBUF):      'ReceiveBufferSize',
-        (SOL_SOCKET, SO_REUSEADDR):   'ReuseAddress',
-        (SOL_SOCKET, SO_SNDBUF):      'SendBufferSize',
-        (SOL_SOCKET, SO_TIMEOUT):     'SoTimeout',
-    }
-
-    def __init__(self, jsockaddr=None, reuse_addr=0):
-        self.jchannel = java.nio.channels.DatagramChannel.open()
-        self.jsocket = self.jchannel.socket()
-        if jsockaddr is not None:
-            self.jsocket.setReuseAddress(reuse_addr)
-            self.jsocket.bind(jsockaddr)
-        self.socketio = org.python.core.io.DatagramSocketIO(self.jchannel, 'rw')
-
-    def connect(self, jsockaddr):
-        self.jchannel.connect(jsockaddr)
-
-    def disconnect(self):
-        """
-            Disconnect the datagram socket.
-            cpython appears not to have this operation
-        """
-        self.jchannel.disconnect()
-
-    def shutdown(self, how):
-        # This is no-op on java, for datagram sockets.
-        # What the user wants to achieve is achieved by calling close() on
-        # java/jython. But we can't call that here because that would then
-        # later cause the user explicit close() call to fail
-        pass
-
-    def _do_send_net(self, byte_array, socket_address, flags):
-        # Need two separate implementations because the java.nio APIs do not support timeouts
-        num_bytes = len(byte_array)
-        if self.jsocket.isConnected() and socket_address is None:
-            packet = java.net.DatagramPacket(byte_array, num_bytes)
-        else:
-            packet = java.net.DatagramPacket(byte_array, num_bytes, socket_address)
-        self.jsocket.send(packet)
-        return num_bytes
-
-    def _do_send_nio(self, byte_array, socket_address, flags):
-        byte_buf = java.nio.ByteBuffer.wrap(byte_array)
-        if self.jchannel.isConnected() and socket_address is None:
-            bytes_sent = self.jchannel.write(byte_buf)
-        else:
-            bytes_sent = self.jchannel.send(byte_buf, socket_address)
-        return bytes_sent
-
-    def sendto(self, byte_array, jsockaddr, flags):
-        if self.mode == MODE_TIMEOUT:
-            return self._do_send_net(byte_array, jsockaddr, flags)
-        else:
-            return self._do_send_nio(byte_array, jsockaddr, flags)
-
-    def send(self, byte_array, flags):
-        if self.mode == MODE_TIMEOUT:
-            return self._do_send_net(byte_array, None, flags)
-        else:
-            return self._do_send_nio(byte_array, None, flags)
-
-    def _do_receive_net(self, return_source_address, num_bytes, flags):
-        byte_array = jarray.zeros(num_bytes, 'b')
-        packet = java.net.DatagramPacket(byte_array, num_bytes)
-        self.jsocket.receive(packet)
-        bytes_rcvd = packet.getLength()
-        if bytes_rcvd < num_bytes:
-            byte_array = byte_array[:bytes_rcvd]
-        return_data = byte_array.tostring()
-        if return_source_address:
-            host = None
-            if packet.getAddress():
-                host = packet.getAddress().getHostAddress()
-            port = packet.getPort()
-            return return_data, (host, port)
-        else:
-            return return_data
-
-    def _do_receive_nio(self, return_source_address, num_bytes, flags):
-        byte_array = jarray.zeros(num_bytes, 'b')
-        byte_buf = java.nio.ByteBuffer.wrap(byte_array)
-        source_address = self.jchannel.receive(byte_buf)
-        if source_address is None and not self.jchannel.isBlocking():
-            raise would_block_error()
-        byte_buf.flip() ; bytes_read = byte_buf.remaining()
-        if bytes_read < num_bytes:
-            byte_array = byte_array[:bytes_read]
-        return_data = byte_array.tostring()
-        if return_source_address:
-            return return_data, (source_address.getAddress().getHostAddress(), source_address.getPort())
-        else:
-            return return_data
-
-    def recvfrom(self, num_bytes, flags):
-        if self.mode == MODE_TIMEOUT:
-            return self._do_receive_net(1, num_bytes, flags)
-        else:
-            return self._do_receive_nio(1, num_bytes, flags)
-
-    def recv(self, num_bytes, flags):
-        if self.mode == MODE_TIMEOUT:
-            return self._do_receive_net(0, num_bytes, flags)
-        else:
-            return self._do_receive_nio(0, num_bytes, flags)
-
-    def getsockname(self):
-        return (self.jsocket.getLocalAddress().getHostAddress(), self.jsocket.getLocalPort())
-
-    def getpeername(self):
-        peer_address = self.jsocket.getInetAddress()
-        if peer_address is None:
-            raise error(errno.ENOTCONN, "Socket is not connected")
-        return (peer_address.getHostAddress(), self.jsocket.getPort() )
-
-has_ipv6 = True # IPV6 FTW!
-
-# Name and address functions
-
-def _gethostbyaddr(name):
-    # This is as close as I can get; at least the types are correct...
-    addresses = java.net.InetAddress.getAllByName(gethostbyname(name))
-    names = []
-    addrs = []
-    for addr in addresses:
-        names.append(asPyString(addr.getHostName()))
-        addrs.append(asPyString(addr.getHostAddress()))
-    return (names, addrs)
-
- at raises_java_exception
-def getfqdn(name=None):
-    """
-    Return a fully qualified domain name for name. If name is omitted or empty
-    it is interpreted as the local host.  To find the fully qualified name,
-    the hostname returned by gethostbyaddr() is checked, then aliases for the
-    host, if available. The first name which includes a period is selected.
-    In case no fully qualified domain name is available, the hostname is retur
-    New in version 2.0.
-    """
-    if not name:
-        name = gethostname()
-    names, addrs = _gethostbyaddr(name)
-    for a in names:
-        if a.find(".") >= 0:
-            return a
-    return name
-
- at raises_java_exception
-def gethostname():
-    return asPyString(java.net.InetAddress.getLocalHost().getHostName())
-
- at raises_java_exception
-def gethostbyname(name):
-    return asPyString(java.net.InetAddress.getByName(name).getHostAddress())
-
-#
-# Skeleton implementation of gethostbyname_ex
-# Needed because urllib2 refers to it
-#
-
- at raises_java_exception
-def gethostbyname_ex(name):
-    return (name, [], gethostbyname(name))
-
- at raises_java_exception
-def gethostbyaddr(name):
-    names, addrs = _gethostbyaddr(name)
-    return (names[0], names, addrs)
-
-def getservbyname(service_name, protocol_name=None):
-    try:
-        from jnr.netdb import Service
-    except ImportError:
-        return None
-    service = Service.getServiceByName(service_name, protocol_name)
-    if service is None:
-        raise error('service/proto not found')
-    return service.getPort()
-
-def getservbyport(port, protocol_name=None):
-    try:
-        from jnr.netdb import Service
-    except ImportError:
-        return None
-    service = Service.getServiceByPort(port, protocol_name)
-    if service is None:
-        raise error('port/proto not found')
-    return service.getName()
-
-def getprotobyname(protocol_name=None):
-    try:
-        from jnr.netdb import Protocol
-    except ImportError:
-        return None
-    proto = Protocol.getProtocolByName(protocol_name)
-    if proto is None:
-        raise error('protocol not found')
-    return proto.getProto()
-
-def _realsocket(family = AF_INET, sock_type = SOCK_STREAM, protocol=0):
-    assert family in (AF_INET, AF_INET6), "Only AF_INET and AF_INET6 sockets are currently supported on jython"
-    assert sock_type in (SOCK_DGRAM, SOCK_STREAM), "Only SOCK_STREAM and SOCK_DGRAM sockets are currently supported on jython"
-    if sock_type == SOCK_STREAM:
-        if protocol != 0:
-            assert protocol == IPPROTO_TCP, "Only IPPROTO_TCP supported on SOCK_STREAM sockets"
-        else:
-            protocol = IPPROTO_TCP
-        result = _tcpsocket()
-    else:
-        if protocol != 0:
-            assert protocol == IPPROTO_UDP, "Only IPPROTO_UDP supported on SOCK_DGRAM sockets"
-        else:
-            protocol = IPPROTO_UDP
-        result = _udpsocket()
-    setattr(result, "family", family)
-    setattr(result, "type",   sock_type)
-    setattr(result, "proto",  protocol)
-    return result
-
-#
-# Attempt to provide IDNA (RFC 3490) support.
-#
-# Try java.net.IDN, built into java 6
-#
-
-idna_libraries = [
-    ('java.net.IDN', 'toASCII', 'toUnicode', 
-        'ALLOW_UNASSIGNED', 'USE_STD3_ASCII_RULES', 
-        java.lang.IllegalArgumentException)
-]
-  
-for idna_lib, efn, dfn, au, usar, exc in idna_libraries:
-    try:
-        m = __import__(idna_lib, globals(), locals(), [efn, dfn, au, usar])
-        encode_fn = getattr(m, efn)
-        def _encode_idna(name):
-            try:
-                return encode_fn(name)
-            except exc:
-                raise UnicodeEncodeError(name)
-        decode_fn = getattr(m, dfn)
-        def _decode_idna(name, flags=0):
-            try:
-                jflags = 0
-                if flags & NI_IDN_ALLOW_UNASSIGNED:
-                    jflags |= au
-                if flags & NI_IDN_USE_STD3_ASCII_RULES:
-                    jflags |= usar
-                return decode_fn(name, jflags)
-            except Exception, x:
-                raise UnicodeDecodeError(name)
-        supports('idna', True)
-        break
-    except (AttributeError, ImportError), e:
-        pass
-else:
-    _encode_idna = lambda x: x.encode("ascii")
-    _decode_idna = lambda x, y=0: x.decode("ascii")
-
-#
-# Define data structures to support IPV4 and IPV6.
-#
-
-class _ip_address_t: pass
-
-class _ipv4_address_t(_ip_address_t):
-
-    def __init__(self, sockaddr, port, jaddress):
-        self.sockaddr = sockaddr
-        self.port     = port
-        self.jaddress = jaddress
-
-    def __getitem__(self, index):
-        if   0 == index:
-            return self.sockaddr
-        elif 1 == index:
-            return self.port
-        else:
-            raise IndexError()
-
-    def __len__(self):
-        return 2
-
-    def __str__(self):
-        return "('%s', %d)" % (self.sockaddr, self.port)
-
-    __repr__ = __str__
-
-class _ipv6_address_t(_ip_address_t):
-
-    def __init__(self, sockaddr, port, jaddress):
-        self.sockaddr = sockaddr
-        self.port     = port
-        self.jaddress = jaddress
-
-    def __getitem__(self, index):
-        if   0 == index:
-            return self.sockaddr
-        elif 1 == index:
-            return self.port
-        elif 2 == index:
-            return 0
-        elif 3 == index:
-            return self.jaddress.scopeId
-        else:
-            raise IndexError()
-
-    def __len__(self):
-        return 4
-
-    def __str__(self):
-        return "('%s', %d, 0, %d)" % (self.sockaddr, self.port, self.jaddress.scopeId)
-
-    __repr__ = __str__
-
-def _get_jsockaddr(address_object, family, sock_type, proto, flags):
-    # Is this an object that was returned from getaddrinfo? If so, it already contains an InetAddress
-    if isinstance(address_object, _ip_address_t):
-        return java.net.InetSocketAddress(address_object.jaddress, address_object[1]) 
-    # The user passed an address tuple, not an object returned from getaddrinfo
-    # So we must call getaddrinfo, after some translations and checking
-    if address_object is None:
-        address_object = ("", 0)
-    error_message = "Address must be a 2-tuple (ipv4: (host, port)) or a 4-tuple (ipv6: (host, port, flow, scope))"
-    if not isinstance(address_object, tuple) or \
-            ((family == AF_INET and len(address_object) != 2) or (family == AF_INET6 and len(address_object) not in [2,4] )) or \
-            not isinstance(address_object[0], (basestring, types.NoneType)) or \
-            not isinstance(address_object[1], (int, long)):
-        raise TypeError(error_message)
-    if len(address_object) == 4 and not isinstance(address_object[3], (int, long)):
-        raise TypeError(error_message)
-    hostname = address_object[0]
-    if hostname is not None:
-        hostname = hostname.strip()
-    port = address_object[1]
-    if family == AF_INET and sock_type == SOCK_DGRAM and hostname == "<broadcast>":
-        hostname = INADDR_BROADCAST
-    if hostname in ["", None]:
-        if flags & AI_PASSIVE:
-            hostname = {AF_INET: INADDR_ANY, AF_INET6: IN6ADDR_ANY_INIT}[family]
-        else:
-            hostname = "localhost"
-    if isinstance(hostname, unicode):
-        hostname = _encode_idna(hostname)
-    addresses = getaddrinfo(hostname, port, family, sock_type, proto, flags)
-    if len(addresses) == 0:
-        raise gaierror(errno.EGETADDRINFOFAILED, 'getaddrinfo failed')
-    return java.net.InetSocketAddress(addresses[0][4].jaddress, port)
-
-# Workaround for this (predominantly windows) issue
-# http://wiki.python.org/jython/NewSocketModule#IPV6_address_support
-
-_ipv4_addresses_only = False
-
-def _use_ipv4_addresses_only(value):
-    global _ipv4_addresses_only
-    _ipv4_addresses_only = value
-
-def _getaddrinfo_get_host(host, family, flags):
-    if not isinstance(host, basestring) and host is not None:
-        raise TypeError("getaddrinfo() argument 1 must be string or None")
-    if flags & AI_NUMERICHOST:
-        if not is_ip_address(host):
-            raise gaierror(EAI_NONAME, "Name or service not known")
-        if family == AF_INET and not is_ipv4_address(host):
-            raise gaierror(EAI_ADDRFAMILY, "Address family for hostname not supported")
-        if family == AF_INET6 and not is_ipv6_address(host):
-            raise gaierror(EAI_ADDRFAMILY, "Address family for hostname not supported")
-    if isinstance(host, unicode):
-        host = _encode_idna(host)
-    return host
-
-def _getaddrinfo_get_port(port, flags):
-    if isinstance(port, basestring):
-        try:
-            int_port = int(port)
-        except ValueError:
-            if flags & AI_NUMERICSERV:
-                raise gaierror(EAI_NONAME, "Name or service not known")
-            # Lookup the service by name
-            try:
-                int_port = getservbyname(port)
-            except error:
-                raise gaierror(EAI_SERVICE, "Servname not supported for ai_socktype")
-    elif port is None:
-        int_port = 0
-    elif not isinstance(port, (int, long)):
-        raise error("Int or String expected")
-    else:
-        int_port = int(port)
-    return int_port % 65536
-
- at raises_java_exception
-def getaddrinfo(host, port, family=AF_UNSPEC, socktype=0, proto=0, flags=0):
-    if _ipv4_addresses_only:
-        family = AF_INET
-    if not family in [AF_INET, AF_INET6, AF_UNSPEC]:
-        raise gaierror(errno.EIO, 'ai_family not supported')
-    host = _getaddrinfo_get_host(host, family, flags)
-    port = _getaddrinfo_get_port(port, flags)
-    if socktype not in [0, SOCK_DGRAM, SOCK_STREAM]:
-        raise error(errno.ESOCKTNOSUPPORT, "Socket type %s is not supported" % _constant_to_name(socktype, ['SOCK_']))
-    filter_fns = []
-    filter_fns.append({
-        AF_INET:   lambda x: isinstance(x, java.net.Inet4Address),
-        AF_INET6:  lambda x: isinstance(x, java.net.Inet6Address),
-        AF_UNSPEC: lambda x: isinstance(x, java.net.InetAddress),
-    }[family])
-    if host in [None, ""]:
-        if flags & AI_PASSIVE:
-             hosts = {AF_INET: [INADDR_ANY], AF_INET6: [IN6ADDR_ANY_INIT], AF_UNSPEC: [INADDR_ANY, IN6ADDR_ANY_INIT]}[family]
-        else:
-             hosts = ["localhost"]
-    else:
-        hosts = [host]
-    results = []
-    for h in hosts:
-        for a in java.net.InetAddress.getAllByName(h):
-            if len([f for f in filter_fns if f(a)]):
-                family = {java.net.Inet4Address: AF_INET, java.net.Inet6Address: AF_INET6}[a.getClass()]
-                if flags & AI_CANONNAME:
-                    canonname = asPyString(a.getCanonicalHostName())
-                else:
-                    canonname = ""
-                sockaddr = asPyString(a.getHostAddress())
-                # TODO: Include flowinfo and scopeid in a 4-tuple for IPv6 addresses
-                sock_tuple = {AF_INET : _ipv4_address_t, AF_INET6 : _ipv6_address_t}[family](sockaddr, port, a)
-                if socktype == 0:
-                    socktypes = [SOCK_DGRAM, SOCK_STREAM]
-                else:
-                    socktypes = [socktype]
-                for result_socktype in socktypes:
-                    result_proto = {SOCK_DGRAM: IPPROTO_UDP, SOCK_STREAM: IPPROTO_TCP}[result_socktype]
-                    if proto in [0, result_proto]:
-                        # The returned socket will only support the result_proto
-                        # If this does not match the requested proto, don't return it
-                        results.append((family, result_socktype, result_proto, canonname, sock_tuple))
-    return results
-
-def _getnameinfo_get_host(address, flags):
-    if not isinstance(address, basestring):
-        raise TypeError("getnameinfo() address 1 must be string, not None")
-    if isinstance(address, unicode):
-        address = _encode_idna(address)
-    jia = java.net.InetAddress.getByName(address)
-    result = jia.getCanonicalHostName()
-    if flags & NI_NAMEREQD:
-        if is_ip_address(result):
-            raise gaierror(EAI_NONAME, "Name or service not known")
-    elif flags & NI_NUMERICHOST:
-        result = jia.getHostAddress()
-    # Ignoring NI_NOFQDN for now
-    if flags & NI_IDN:
-        result = _decode_idna(result, flags)
-    return result
-
-def _getnameinfo_get_port(port, flags):
-    if not isinstance(port, (int, long)):
-        raise TypeError("getnameinfo() port number must be an integer")
-    if flags & NI_NUMERICSERV:
-        return port
-    proto = None
-    if flags & NI_DGRAM:
-        proto = "udp"
-    return getservbyport(port, proto)
-
- at raises_java_exception
-def getnameinfo(sock_addr, flags):
-    if not isinstance(sock_addr, tuple) or len(sock_addr) < 2:
-        raise TypeError("getnameinfo() argument 1 must be a tuple")
-    host = _getnameinfo_get_host(sock_addr[0], flags)
-    port = _getnameinfo_get_port(sock_addr[1], flags)
-    return (host, port)
-
-def getdefaulttimeout():
-    return _defaulttimeout
-
-def _calctimeoutvalue(value):
-    if value is None:
-        return None
-    try:
-        floatvalue = float(value)
-    except:
-        raise TypeError('Socket timeout value must be a number or None')
-    if floatvalue < 0.0:
-        raise ValueError("Socket timeout value cannot be negative")
-    if floatvalue < 0.000001:
-        return 0.0
-    return floatvalue
-
-def setdefaulttimeout(timeout):
-    global _defaulttimeout
-    try:
-        _defaulttimeout = _calctimeoutvalue(timeout)
-    finally:
-        _nonblocking_api_mixin.timeout = _defaulttimeout
-
-def htons(x): return x
-def htonl(x): return x
-def ntohs(x): return x
-def ntohl(x): return x
-
- at raises_java_exception
-def inet_pton(family, ip_string):
-    if family == AF_INET:
-        if not is_ipv4_address(ip_string):
-            raise error("illegal IP address string passed to inet_pton")
-    elif family == AF_INET6:
-        if not is_ipv6_address(ip_string):
-            raise error("illegal IP address string passed to inet_pton")
-    else:
-        raise error(errno.EAFNOSUPPORT, "Address family not supported by protocol")
-    ia = java.net.InetAddress.getByName(ip_string)
-    bytes = []
-    for byte in ia.getAddress():
-        if byte < 0:
-            bytes.append(byte+256)
-        else:
-            bytes.append(byte)
-    return "".join([chr(byte) for byte in bytes])
-
- at raises_java_exception
-def inet_ntop(family, packed_ip):
-    jByteArray = jarray.array(packed_ip, 'b')
-    if family == AF_INET:
-        if len(jByteArray) != 4:
-            raise ValueError("invalid length of packed IP address string")
-    elif family == AF_INET6:
-        if len(jByteArray) != 16:
-            raise ValueError("invalid length of packed IP address string")
-    else:
-        raise ValueError("unknown address family %s" % family)
-    ia = java.net.InetAddress.getByAddress(jByteArray)
-    return ia.getHostAddress()
-
-def inet_aton(ip_string):
-    return inet_pton(AF_INET, ip_string)
-
-def inet_ntoa(packed_ip):
-    return inet_ntop(AF_INET, packed_ip)
-
-class _nonblocking_api_mixin:
-
-    mode            = MODE_BLOCKING
-    reference_count = 0
-    close_lock      = threading.Lock()
-
-    def __init__(self):
-        self.timeout = _defaulttimeout
-        if self.timeout is not None:
-            self.mode = MODE_TIMEOUT
-        self.pending_options = {
-            (SOL_SOCKET, SO_REUSEADDR):  0,
-        }
-
-    def gettimeout(self):
-        return self.timeout
-
-    def settimeout(self, timeout):
-        self.timeout = _calctimeoutvalue(timeout)
-        if self.timeout is None:
-            self.mode = MODE_BLOCKING
-        elif self.timeout < 0.000001:
-            self.mode = MODE_NONBLOCKING
-        else:
-            self.mode = MODE_TIMEOUT
-        self._config()
-
-    def setblocking(self, flag):
-        if flag:
-            self.mode = MODE_BLOCKING
-            self.timeout = None
-        else:
-            self.mode = MODE_NONBLOCKING
-            self.timeout = 0.0
-        self._config()
-
-    def getblocking(self):
-        return self.mode == MODE_BLOCKING
-
-    @raises_java_exception
-    def setsockopt(self, level, optname, value):
-        if self.sock_impl:
-            self.sock_impl.setsockopt(level, optname, value)
-        else:
-            self.pending_options[ (level, optname) ] = value
-
-    @raises_java_exception
-    def getsockopt(self, level, optname):
-        # Handle "pseudo" options first
-        if level == SOL_SOCKET and optname == SO_TYPE:
-            return getattr(self, "type")
-        if level == SOL_SOCKET and optname == SO_ERROR:
-            return_value = self._last_error
-            self._last_error = 0
-            return return_value
-        # Now handle "real" options
-        if self.sock_impl:
-            return self.sock_impl.getsockopt(level, optname)
-        else:
-            return self.pending_options.get( (level, optname), None)
-
-    @raises_java_exception
-    def shutdown(self, how):
-        assert how in (SHUT_RD, SHUT_WR, SHUT_RDWR)
-        if not self.sock_impl:
-            raise error(errno.ENOTCONN, "Transport endpoint is not connected")
-        self.sock_impl.shutdown(how)
-
-    @raises_java_exception
-    def close(self):
-        if self.sock_impl:
-            self.sock_impl.close()
-
-    @raises_java_exception
-    def getsockname(self):
-        if self.sock_impl is None:
-            # If the user has already bound an address, return that
-            if self.local_addr:
-                return self.local_addr
-            # The user has not bound, connected or listened
-            # This is what cpython raises in this scenario
-            raise error(errno.EINVAL, "Invalid argument")
-        return self.sock_impl.getsockname()
-
-    @raises_java_exception
-    def getpeername(self):
-        if self.sock_impl is None:
-            raise error(errno.ENOTCONN, "Socket is not connected")
-        return self.sock_impl.getpeername()
-
-    def _config(self):
-        assert self.mode in _permitted_modes
-        if self.sock_impl:
-            self.sock_impl.config(self.mode, self.timeout)
-            for level, optname in self.pending_options.keys():
-                if optname != SO_REUSEADDR:
-                    self.sock_impl.setsockopt(level, optname, self.pending_options[ (level, optname) ])
-
-    def getchannel(self):
-        if not self.sock_impl:
-            return None
-        return self.sock_impl.getchannel()
-
-    def fileno(self):
-        if not self.sock_impl:
-            return None
-        return self.sock_impl.fileno()
-
-    def _get_jsocket(self):
-        return self.sock_impl.jsocket
-
-class _tcpsocket(_nonblocking_api_mixin):
-
-    sock_impl   = None
-    istream     = None
-    ostream     = None
-    local_addr  = None
-    server      = 0
-    _last_error = 0
-
-    def __init__(self):
-        _nonblocking_api_mixin.__init__(self)
-
-    def getsockopt(self, level, optname):
-        if level == SOL_SOCKET and optname == SO_ACCEPTCONN:
-            return self.server
-        return _nonblocking_api_mixin.getsockopt(self, level, optname)
-
-    @raises_java_exception
-    def bind(self, addr):
-        assert not self.sock_impl
-        assert not self.local_addr
-        # Do the address format check
-        _get_jsockaddr(addr, self.family, self.type, self.proto, AI_PASSIVE)
-        self.local_addr = addr
-
-    @raises_java_exception
-    def listen(self, backlog):
-        "This signifies a server socket"
-        assert not self.sock_impl
-        self.server = 1
-        self.sock_impl = _server_socket_impl(_get_jsockaddr(self.local_addr, self.family, self.type, self.proto, AI_PASSIVE), 
-                              backlog, self.pending_options[ (SOL_SOCKET, SO_REUSEADDR) ])
-        self._config()
-
-    @raises_java_exception
-    def accept(self):
-        "This signifies a server socket"
-        if not self.sock_impl:
-            self.listen()
-        assert self.server
-        new_sock = self.sock_impl.accept()
-        if not new_sock:
-            raise would_block_error()
-        cliconn = _tcpsocket()
-        cliconn.pending_options[ (SOL_SOCKET, SO_REUSEADDR) ] = new_sock.jsocket.getReuseAddress()
-        cliconn.sock_impl = new_sock
-        cliconn._setup()
-        return cliconn, new_sock.getpeername()
-
-    def _do_connect(self, addr):
-        assert not self.sock_impl
-        self.sock_impl = _client_socket_impl()
-        if self.local_addr: # Has the socket been bound to a local address?
-            self.sock_impl.bind(_get_jsockaddr(self.local_addr, self.family, self.type, self.proto, 0), 
-                                 self.pending_options[ (SOL_SOCKET, SO_REUSEADDR) ])
-        self._config() # Configure timeouts, etc, now that the socket exists
-        self.sock_impl.connect(_get_jsockaddr(addr, self.family, self.type, self.proto, 0))
-
-    @raises_java_exception
-    def connect(self, addr):
-        "This signifies a client socket"
-        self._do_connect(addr)
-        self._setup()
-
-    @raises_java_exception
-    def connect_ex(self, addr):
-        "This signifies a client socket"
-        if not self.sock_impl:
-            self._do_connect(addr)
-        if self.sock_impl.finish_connect():
-            self._setup()
-            if self.mode == MODE_NONBLOCKING:
-                return errno.EISCONN
-            return 0
-        return errno.EINPROGRESS
-
-    def _setup(self):
-        if self.mode != MODE_NONBLOCKING:
-            self.istream = self.sock_impl.jsocket.getInputStream()
-            self.ostream = self.sock_impl.jsocket.getOutputStream()
-
-    @raises_java_exception
-    def recv(self, n):
-        if not self.sock_impl: raise error(errno.ENOTCONN, 'Socket is not connected')
-        if self.sock_impl.jchannel.isConnectionPending():
-            self.sock_impl.jchannel.finishConnect()
-        data = jarray.zeros(n, 'b')
-        m = self.sock_impl.read(data)
-        if m == -1:#indicates EOF has been reached, so we just return the empty string
-            return ""
-        elif m <= 0:
-            if self.mode == MODE_NONBLOCKING:
-                raise would_block_error()
-            return ""
-        if m < n:
-            data = data[:m]
-        return data.tostring()
-
-    @raises_java_exception
-    def recvfrom(self, n):
-        return self.recv(n), self.getpeername()
-
-    @raises_java_exception
-    def send(self, s):
-        if not self.sock_impl: raise error(errno.ENOTCONN, 'Socket is not connected')
-        if self.sock_impl.jchannel.isConnectionPending():
-            self.sock_impl.jchannel.finishConnect()
-        numwritten = self.sock_impl.write(s)
-        if numwritten == 0 and self.mode == MODE_NONBLOCKING:
-            raise would_block_error()
-        return numwritten
-
-    sendall = send
-
-    @raises_java_exception
-    def close(self):
-        if self.istream:
-            self.istream.close()
-        if self.ostream:
-            self.ostream.close()
-        if self.sock_impl:
-            self.sock_impl.close()
-
-
-class _udpsocket(_nonblocking_api_mixin):
-
-    sock_impl   = None
-    connected   = False
-    local_addr  = None
-    _last_error = 0
-
-    def __init__(self):
-        _nonblocking_api_mixin.__init__(self)
-
-    @raises_java_exception
-    def bind(self, addr):
-        assert not self.sock_impl
-        assert not self.local_addr
-        # Do the address format check
-        _get_jsockaddr(addr, self.family, self.type, self.proto, AI_PASSIVE)
-        self.local_addr = addr
-        self.sock_impl = _datagram_socket_impl(_get_jsockaddr(self.local_addr, self.family, self.type, self.proto, AI_PASSIVE), 
-                                                self.pending_options[ (SOL_SOCKET, SO_REUSEADDR) ])
-        self._config()
-
-    def _do_connect(self, addr):
-        assert not self.connected, "Datagram Socket is already connected"
-        if not self.sock_impl:
-            self.sock_impl = _datagram_socket_impl()
-            self._config()
-        self.sock_impl.connect(_get_jsockaddr(addr, self.family, self.type, self.proto, 0))
-        self.connected = True
-
-    @raises_java_exception
-    def connect(self, addr):
-        self._do_connect(addr)
-
-    @raises_java_exception
-    def connect_ex(self, addr):
-        if not self.sock_impl:
-            self._do_connect(addr)
-        return 0
-
-    @raises_java_exception
-    def sendto(self, data, p1, p2=None):
-        if not p2:
-            flags, addr = 0, p1
-        else:
-            flags, addr = 0, p2
-        if not self.sock_impl:
-            self.sock_impl = _datagram_socket_impl()
-            self._config()
-        byte_array = java.lang.String(data).getBytes('iso-8859-1')
-        result = self.sock_impl.sendto(byte_array, _get_jsockaddr(addr, self.family, self.type, self.proto, 0), flags)
-        return result
-
-    @raises_java_exception
-    def send(self, data, flags=None):
-        if not self.connected: raise error(errno.ENOTCONN, "Socket is not connected")
-        byte_array = java.lang.String(data).getBytes('iso-8859-1')
-        return self.sock_impl.send(byte_array, flags)
-
-    @raises_java_exception
-    def recvfrom(self, num_bytes, flags=None):
-        """
-        There is some disagreement as to what the behaviour should be if
-        a recvfrom operation is requested on an unbound socket.
-        See the following links for more information
-        http://bugs.jython.org/issue1005
-        http://bugs.sun.com/view_bug.do?bug_id=6621689
-        """
-        # This is the old 2.1 behaviour
-        #assert self.sock_impl
-        # This is amak's preferred interpretation
-        #raise error(errno.ENOTCONN, "Recvfrom on unbound udp socket meaningless operation")
-        # And this is the option for cpython compatibility
-        if not self.sock_impl:
-            self.sock_impl = _datagram_socket_impl()
-            self._config()
-        return self.sock_impl.recvfrom(num_bytes, flags)
-
-    @raises_java_exception
-    def recv(self, num_bytes, flags=None):
-        if not self.sock_impl:
-            raise error(errno.ENOTCONN, "Socket is not connected")
-        return self.sock_impl.recv(num_bytes, flags)
-
-    def __del__(self):
-        self.close()
-
-_socketmethods = (
-    'bind', 'connect', 'connect_ex', 'fileno', 'listen',
-    'getpeername', 'getsockname', 'getsockopt', 'setsockopt',
-    'sendall', 'setblocking',
-    'settimeout', 'gettimeout', 'shutdown', 'getchannel')
-
-# All the method names that must be delegated to either the real socket
-# object or the _closedsocket object.
-_delegate_methods = ("recv", "recvfrom", "recv_into", "recvfrom_into",
-                     "send", "sendto")
-
-class _closedsocket(object):
-    __slots__ = []
-    def _dummy(*args):
-        raise error(errno.EBADF, 'Bad file descriptor')
-    # All _delegate_methods must also be initialized here.
-    send = recv = recv_into = sendto = recvfrom = recvfrom_into = _dummy
-    __getattr__ = _dummy
-
-_active_sockets = set()
-
-def _closeActiveSockets():
-    for socket in _active_sockets.copy():
-        try:
-            socket.close()
-        except error:
-            msg = 'Problem closing socket: %s: %r' % (socket, sys.exc_info())
-            print >> sys.stderr, msg
-
-class _socketobject(object):
-
-    __doc__ = _realsocket.__doc__
-
-    __slots__ = ["_sock", "__weakref__"] + list(_delegate_methods)
-
-    def __init__(self, family=AF_INET, type=SOCK_STREAM, proto=0, _sock=None):
-        if _sock is None:
-            _sock = _realsocket(family, type, proto)
-            _sock.reference_count += 1
-        elif isinstance(_sock, _nonblocking_api_mixin):
-            _sock.reference_count += 1
-        self._sock = _sock
-        for method in _delegate_methods:
-            meth = getattr(_sock, method, None)
-            if meth:
-                setattr(self, method, meth)
-        _active_sockets.add(self)
-
-    def close(self):
-        try:
-            _active_sockets.remove(self)
-        except KeyError:
-            pass
-        _sock = self._sock
-        if isinstance(_sock, _nonblocking_api_mixin):
-            _sock.close_lock.acquire()
-            try:
-                _sock.reference_count -=1
-                if not _sock.reference_count:
-                    _sock.close()
-                self._sock = _closedsocket()
-                dummy = self._sock._dummy
-                for method in _delegate_methods:
-                    setattr(self, method, dummy)
-                self.send = self.recv = self.sendto = self.recvfrom = \
-                    self._sock._dummy
-            finally:
-                _sock.close_lock.release()
-    #close.__doc__ = _realsocket.close.__doc__
-
-    def accept(self):
-        sock, addr = self._sock.accept()
-        return _socketobject(_sock=sock), addr
-    #accept.__doc__ = _realsocket.accept.__doc__
-
-    def dup(self):
-        """dup() -> socket object
-
-        Return a new socket object connected to the same system resource."""
-        _sock = self._sock
-        if not isinstance(_sock, _nonblocking_api_mixin):
-            return _socketobject(_sock=_sock)
-
-        _sock.close_lock.acquire()
-        try:
-            duped = _socketobject(_sock=_sock)
-        finally:
-            _sock.close_lock.release()
-        return duped
-
-    def makefile(self, mode='r', bufsize=-1):
-        """makefile([mode[, bufsize]]) -> file object
-
-        Return a regular file object corresponding to the socket.  The mode
-        and bufsize arguments are as for the built-in open() function."""
-        _sock = self._sock
-        if not isinstance(_sock, _nonblocking_api_mixin):
-            return _fileobject(_sock, mode, bufsize)
-
-        _sock.close_lock.acquire()
-        try:
-            fileobject = _fileobject(_sock, mode, bufsize)
-        finally:
-            _sock.close_lock.release()
-        return fileobject
-
-    family = property(lambda self: self._sock.family, doc="the socket family")
-    type = property(lambda self: self._sock.type, doc="the socket type")
-    proto = property(lambda self: self._sock.proto, doc="the socket protocol")
-
-    _s = ("def %s(self, *args): return self._sock.%s(*args)\n\n"
-          #"%s.__doc__ = _realsocket.%s.__doc__\n")
-          )
-    for _m in _socketmethods:
-        #exec _s % (_m, _m, _m, _m)
-        exec _s % (_m, _m)
-    del _m, _s
-
-socket = SocketType = _socketobject
-
-class _fileobject(object):
-    """Faux file object attached to a socket object."""
-
-    default_bufsize = 8192
-    name = "<socket>"
-
-    __slots__ = ["mode", "bufsize", "softspace",
-                 # "closed" is a property, see below
-                 "_sock", "_rbufsize", "_wbufsize", "_rbuf", "_wbuf",
-                 "_close"]
-
-    def __init__(self, sock, mode='rb', bufsize=-1, close=False):
-        self._sock = sock
-        if isinstance(sock, _nonblocking_api_mixin):
-            sock.reference_count += 1
-        self.mode = mode # Not actually used in this version
-        if bufsize < 0:
-            bufsize = self.default_bufsize
-        self.bufsize = bufsize
-        self.softspace = False
-        if bufsize == 0:
-            self._rbufsize = 1
-        elif bufsize == 1:
-            self._rbufsize = self.default_bufsize
-        else:
-            self._rbufsize = bufsize
-        self._wbufsize = bufsize
-        self._rbuf = "" # A string
-        self._wbuf = [] # A list of strings
-        self._close = close
-
-    def _getclosed(self):
-        return self._sock is None
-    closed = property(_getclosed, doc="True if the file is closed")
-
-    def close(self):
-        try:
-            if self._sock:
-                self.flush()
-        finally:
-            if self._sock:
-                if isinstance(self._sock, _nonblocking_api_mixin):
-                    self._sock.reference_count -= 1
-                    if not self._sock.reference_count or self._close:
-                        self._sock.close()
-                elif self._close:
-                    self._sock.close()
-            self._sock = None
-
-    def __del__(self):
-        try:
-            self.close()
-        except:
-            # close() may fail if __init__ didn't complete
-            pass
-
-    def flush(self):
-        if self._wbuf:
-            buffer = "".join(self._wbuf)
-            self._wbuf = []
-            self._sock.sendall(buffer)
-
-    def fileno(self):
-        return self._sock.fileno()
-
-    def write(self, data):
-        data = str(data) # XXX Should really reject non-string non-buffers
-        if not data:
-            return
-        self._wbuf.append(data)
-        if (self._wbufsize == 0 or
-            self._wbufsize == 1 and '\n' in data or
-            self._get_wbuf_len() >= self._wbufsize):
-            self.flush()
-
-    def writelines(self, list):
-        # XXX We could do better here for very long lists
-        # XXX Should really reject non-string non-buffers
-        self._wbuf.extend(filter(None, map(str, list)))
-        if (self._wbufsize <= 1 or
-            self._get_wbuf_len() >= self._wbufsize):
-            self.flush()
-
-    def _get_wbuf_len(self):
-        buf_len = 0
-        for x in self._wbuf:
-            buf_len += len(x)
-        return buf_len
-
-    def read(self, size=-1):
-        data = self._rbuf
-        if size < 0:
-            # Read until EOF
-            buffers = []
-            if data:
-                buffers.append(data)
-            self._rbuf = ""
-            if self._rbufsize <= 1:
-                recv_size = self.default_bufsize
-            else:
-                recv_size = self._rbufsize
-            while True:
-                data = self._sock.recv(recv_size)
-                if not data:
-                    break
-                buffers.append(data)
-            return "".join(buffers)
-        else:
-            # Read until size bytes or EOF seen, whichever comes first
-            buf_len = len(data)
-            if buf_len >= size:
-                self._rbuf = data[size:]
-                return data[:size]
-            buffers = []
-            if data:
-                buffers.append(data)
-            self._rbuf = ""
-            while True:
-                left = size - buf_len
-                recv_size = max(self._rbufsize, left)
-                data = self._sock.recv(recv_size)
-                if not data:
-                    break
-                buffers.append(data)
-                n = len(data)
-                if n >= left:
-                    self._rbuf = data[left:]
-                    buffers[-1] = data[:left]
-                    break
-                buf_len += n
-            return "".join(buffers)
-
-    def readline(self, size=-1):
-        data = self._rbuf
-        if size < 0:
-            # Read until \n or EOF, whichever comes first
-            if self._rbufsize <= 1:
-                # Speed up unbuffered case
-                assert data == ""
-                buffers = []
-                recv = self._sock.recv
-                while data != "\n":
-                    data = recv(1)
-                    if not data:
-                        break
-                    buffers.append(data)
-                return "".join(buffers)
-            nl = data.find('\n')
-            if nl >= 0:
-                nl += 1
-                self._rbuf = data[nl:]
-                return data[:nl]
-            buffers = []
-            if data:
-                buffers.append(data)
-            self._rbuf = ""
-            while True:
-                data = self._sock.recv(self._rbufsize)
-                if not data:
-                    break
-                buffers.append(data)
-                nl = data.find('\n')
-                if nl >= 0:
-                    nl += 1
-                    self._rbuf = data[nl:]
-                    buffers[-1] = data[:nl]
-                    break
-            return "".join(buffers)
-        else:
-            # Read until size bytes or \n or EOF seen, whichever comes first
-            nl = data.find('\n', 0, size)
-            if nl >= 0:
-                nl += 1
-                self._rbuf = data[nl:]
-                return data[:nl]
-            buf_len = len(data)
-            if buf_len >= size:
-                self._rbuf = data[size:]
-                return data[:size]
-            buffers = []
-            if data:
-                buffers.append(data)
-            self._rbuf = ""
-            while True:
-                data = self._sock.recv(self._rbufsize)
-                if not data:
-                    break
-                buffers.append(data)
-                left = size - buf_len
-                nl = data.find('\n', 0, left)
-                if nl >= 0:
-                    nl += 1
-                    self._rbuf = data[nl:]
-                    buffers[-1] = data[:nl]
-                    break
-                n = len(data)
-                if n >= left:
-                    self._rbuf = data[left:]
-                    buffers[-1] = data[:left]
-                    break
-                buf_len += n
-            return "".join(buffers)
-
-    def readlines(self, sizehint=0):
-        total = 0
-        list = []
-        while True:
-            line = self.readline()
-            if not line:
-                break
-            list.append(line)
-            total += len(line)
-            if sizehint and total >= sizehint:
-                break
-        return list
-
-    # Iterator protocols
-
-    def __iter__(self):
-        return self
-
-    def next(self):
-        line = self.readline()
-        if not line:
-            raise StopIteration
-        return line
-
-_GLOBAL_DEFAULT_TIMEOUT = object()
-
-def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
-                      source_address=None):
-    """Connect to *address* and return the socket object.
-
-    Convenience function.  Connect to *address* (a 2-tuple ``(host,
-    port)``) and return the socket object.  Passing the optional
-    *timeout* parameter will set the timeout on the socket instance
-    before attempting to connect.  If no *timeout* is supplied, the
-    global default timeout setting returned by :func:`getdefaulttimeout`
-    is used.  If *source_address* is set it must be a tuple of (host, port)
-    for the socket to bind as a source address before making the connection.
-    An host of '' or port 0 tells the OS to use the default.
-    """
-
-    host, port = address
-    err = None
-    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
-        af, socktype, proto, canonname, sa = res
-        sock = None
-        try:
-            sock = socket(af, socktype, proto)
-            if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
-                sock.settimeout(timeout)
-            if source_address:
-                sock.bind(source_address)
-            sock.connect(sa)
-            return sock
-
-        except error as _:
-            err = _
-            if sock is not None:
-                sock.close()
-
-    if err is not None:
-        raise err
-    else:
-        raise error("getaddrinfo returns an empty list")
-
-# Define the SSL support
-
-class ssl:
-
-    @raises_java_exception
-    def __init__(self, jython_socket_wrapper, keyfile=None, certfile=None):
-        self.jython_socket_wrapper = jython_socket_wrapper
-        jython_socket = self.jython_socket_wrapper._sock
-        self.java_ssl_socket = self._make_ssl_socket(jython_socket)
-        self._in_buf = java.io.BufferedInputStream(self.java_ssl_socket.getInputStream())
-        self._out_buf = java.io.BufferedOutputStream(self.java_ssl_socket.getOutputStream())
-
-    def _make_ssl_socket(self, jython_socket, auto_close=0):
-        java_net_socket = jython_socket._get_jsocket()
-        assert isinstance(java_net_socket, java.net.Socket)
-        host = java_net_socket.getInetAddress().getHostAddress()
-        port = java_net_socket.getPort()
-        factory = javax.net.ssl.SSLSocketFactory.getDefault();
-        java_ssl_socket = factory.createSocket(java_net_socket, host, port, auto_close)
-        java_ssl_socket.setEnabledCipherSuites(java_ssl_socket.getSupportedCipherSuites())
-        java_ssl_socket.startHandshake()
-        return java_ssl_socket
-
-    @raises_java_exception
-    def read(self, n=4096):
-        data = jarray.zeros(n, 'b')
-        m = self._in_buf.read(data, 0, n)
-        if m <= 0:
-            return ""
-        if m < n:
-            data = data[:m]
-        return data.tostring()
-
-    recv = read
-
-    @raises_java_exception
-    def write(self, s):
-        self._out_buf.write(s)
-        self._out_buf.flush()
-        return len(s)
-
-    send = sendall = write
-
-    def makefile(self, mode='r', bufsize=-1):
-        return _fileobject(self, mode, bufsize)
-
-    def _get_server_cert(self):
-        return self.java_ssl_socket.getSession().getPeerCertificates()[0]
-
-    @raises_java_exception
-    def server(self):
-        cert = self._get_server_cert()
-        return cert.getSubjectDN().toString()
-
-    @raises_java_exception
-    def issuer(self):
-        cert = self._get_server_cert()
-        return cert.getIssuerDN().toString()
-
-    def close(self):
-        self.jython_socket_wrapper.close()
-
-def test():
-    s = socket(AF_INET, SOCK_STREAM)
-    s.connect(("", 80))
-    s.send("GET / HTTP/1.0\r\n\r\n")
-    while 1:
-        data = s.recv(2000)
-        print data
-        if not data:
-            break
-
-if __name__ == '__main__':
-    test()
diff --git a/Lib/ssl.py b/Lib/ssl.py
--- a/Lib/ssl.py
+++ b/Lib/ssl.py
@@ -1,10 +1,257 @@
-"""
-This module provides very limited support for the SSL module on jython.
-
-See the jython wiki for more information.
-http://wiki.python.org/jython/SSLModule
-"""
-
-import socket
-
-wrap_socket = socket.ssl
+import logging
+
+try:
+    # jarjar-ed version
+    from org.python.netty.channel import ChannelInitializer
+    from org.python.netty.handler.ssl import SslHandler
+except ImportError:
+    # dev version from extlibs
+    from io.netty.channel import ChannelInitializer
+    from io.netty.handler.ssl import SslHandler
+
+from _socket import (
+    SSLError, raises_java_exception,
+    SSL_ERROR_SSL,
+    SSL_ERROR_WANT_READ,
+    SSL_ERROR_WANT_WRITE,
+    SSL_ERROR_WANT_X509_LOOKUP,
+    SSL_ERROR_SYSCALL,
+    SSL_ERROR_ZERO_RETURN,
+    SSL_ERROR_WANT_CONNECT,
+    SSL_ERROR_EOF,
+    SSL_ERROR_INVALID_ERROR_CODE)
+from _sslcerts import _get_ssl_context
+
+from java.text import SimpleDateFormat
+from java.util import Locale, TimeZone
+from javax.naming.ldap import LdapName
+from javax.security.auth.x500 import X500Principal
+
+
+log = logging.getLogger("socket")
+
+
+CERT_NONE, CERT_OPTIONAL, CERT_REQUIRED = range(3)
+
+# FIXME need to map to java names as well; there's also possibility some difference between 
+# SSLv2 (Java) and PROTOCOL_SSLv23 (Python) but reading the docs suggest not
+# http://docs.oracle.com/javase/7/docs/technotes/guides/security/StandardNames.html#SSLContext
+
+# Currently ignored, since we just use the default in Java. FIXME
+PROTOCOL_SSLv2, PROTOCOL_SSLv3, PROTOCOL_SSLv23, PROTOCOL_TLSv1 = range(4)
+_PROTOCOL_NAMES = {PROTOCOL_SSLv2: 'SSLv2', PROTOCOL_SSLv3: 'SSLv3', PROTOCOL_SSLv23: 'SSLv23', PROTOCOL_TLSv1: 'TLSv1'}
+
+_rfc2822_date_format = SimpleDateFormat("MMM dd HH:mm:ss yyyy z", Locale.US)
+_rfc2822_date_format.setTimeZone(TimeZone.getTimeZone("GMT"))
+
+_ldap_rdn_display_names = {
+    # list from RFC 2253
+    "CN": "commonName",
+    "L":  "localityName",
+    "ST": "stateOrProvinceName",
+    "O":  "organizationName",
+    "OU": "organizationalUnitName",
+    "C":  "countryName",
+    "STREET": "streetAddress",
+    "DC": "domainComponent",
+    "UID": "userid"
+}
+
+_cert_name_types = [
+    # FIXME only entry 2 - DNS - has been confirmed w/ cpython;
+    # everything else is coming from this doc:
+    # http://docs.oracle.com/javase/7/docs/api/java/security/cert/X509Certificate.html#getSubjectAlternativeNames()
+    "other",
+    "rfc822",
+    "DNS",
+    "x400Address",
+    "directory",
+    "ediParty",
+    "uniformResourceIdentifier",
+    "ipAddress",
+    "registeredID"]
+
+
+class SSLInitializer(ChannelInitializer):
+
+    def __init__(self, ssl_handler):
+        self.ssl_handler = ssl_handler
+
+    def initChannel(self, ch):
+        pipeline = ch.pipeline()
+        pipeline.addLast("ssl", self.ssl_handler) 
+
+
+class SSLSocket(object):
+    
+    def __init__(self, sock,
+                 keyfile, certfile, ca_certs,
+                 do_handshake_on_connect, server_side):
+        self.sock = sock
+        self._sock = sock._sock  # the real underlying socket
+        self.context = _get_ssl_context(keyfile, certfile, ca_certs)
+        self.engine = self.context.createSSLEngine()
+        self.engine.setUseClientMode(not server_side)
+        self.ssl_handler = SslHandler(self.engine)
+        self.already_handshaked = False
+        self.do_handshake_on_connect = do_handshake_on_connect
+
+        if self.do_handshake_on_connect and hasattr(self._sock, "connected") and self._sock.connected:
+            self.already_handshaked = True
+            log.debug("Adding SSL handler to pipeline after connection", extra={"sock": self._sock})
+            self._sock.channel.pipeline().addFirst("ssl", self.ssl_handler)
+            self._sock._post_connect()
+            self._sock._notify_selectors()
+            self._sock._unlatch()
+
+        def handshake_step(result):
+            log.debug("SSL handshaking %s", result, extra={"sock": self._sock})
+            if not hasattr(self._sock, "activity_latch"):  # need a better discriminant
+                self._sock._post_connect()
+            self._sock._notify_selectors()
+
+        self.ssl_handler.handshakeFuture().addListener(handshake_step)
+        if self.do_handshake_on_connect and self.already_handshaked:
+            self.ssl_handler.handshakeFuture().sync()
+            log.debug("SSL handshaking completed", extra={"sock": self._sock})
+
+    def connect(self, addr):
+        log.debug("Connect SSL with handshaking %s", self.do_handshake_on_connect, extra={"sock": self._sock})
+        self._sock._connect(addr)
+        if self.do_handshake_on_connect:
+            self.already_handshaked = True
+            if self._sock.connected:
+                log.debug("Already connected, adding SSL handler to pipeline...", extra={"sock": self._sock})
+                self._sock.channel.pipeline().addFirst("ssl", self.ssl_handler)
+            else:
+                log.debug("Not connected, adding SSL initializer...", extra={"sock": self._sock})
+                self._sock.connect_handlers.append(SSLInitializer(self.ssl_handler))
+
+    # Various pass through methods to the wrapper socket
+
+    def send(self, data):
+        return self.sock.send(data)
+
+    def sendall(self, data):
+        return self.sock.sendall(data)
+
+    def recv(self, bufsize, flags=0):
+        return self.sock.recv(bufsize, flags)
+
+    def close(self):
+        self.sock.close()
+
+    def setblocking(self, mode):
+        self.sock.setblocking(mode)
+
+    def settimeout(self, timeout):
+        self.sock.settimeout(timeout)
+
+    def gettimeout(self):
+        return self.sock.gettimeout()
+
+    def makefile(self, mode='r', bufsize=-1):
+        return self.sock.makefile(mode, bufsize)
+
+    def shutdown(self, how):
+        self.sock.shutdown(how)
+
+    # Need to work with the real underlying socket as well
+
+    def _readable(self):
+        return self._sock._readable()
+
+    def _writable(self):
+        return self._sock._writable()
+
+    def _register_selector(self, selector):
+        self._sock._register_selector(selector)
+
+    def _unregister_selector(self, selector):
+        return self._sock._unregister_selector(selector)
+
+    def _notify_selectors(self):
+        self._sock._notify_selectors()
+
+    def do_handshake(self):
+        if not self.already_handshaked:
+            log.debug("Not handshaked, so adding SSL handler", extra={"sock": self._sock})
+            self.already_handshaked = True
+            self._sock.channel.pipeline().addFirst("ssl", self.ssl_handler)
+
+    def getpeername(self):
+        return self.sock.getpeername()
+
+    def fileno(self):
+        return self
+
+    @raises_java_exception
+    def getpeercert(self, binary_form=False):
+        cert = self.engine.getSession().getPeerCertificates()[0]
+        if binary_form:
+            return cert.getEncoded()
+        dn = cert.getSubjectX500Principal().getName()
+        ldapDN = LdapName(dn)
+        # FIXME given this tuple of a single element tuple structure assumed here, is it possible this is
+        # not actually the case, eg because of multi value attributes?
+        rdns = tuple((((_ldap_rdn_display_names.get(rdn.type), rdn.value),) for rdn in ldapDN.getRdns()))
+        # FIXME is it str? or utf8? or some other encoding? maybe a bug in cpython?
+        alt_names = tuple(((_cert_name_types[type], str(name)) for (type, name) in cert.getSubjectAlternativeNames()))
+        pycert = {
+            "notAfter": _rfc2822_date_format.format(cert.getNotAfter()),
+            "subject": rdns,
+            "subjectAltName": alt_names, 
+        }
+        return pycert
+
+    @raises_java_exception
+    def issuer(self):
+        return self.getpeercert().getIssuerDN().toString()
+
+    def cipher(self):
+        session = self._sslsocket.session
+        suite = str(session.cipherSuite)
+        if "256" in suite:  # FIXME!!! this test usually works, but there must be a better approach
+            strength = 256
+        elif "128" in suite:
+            strength = 128
+        else:
+            strength = None
+        return suite, str(session.protocol), strength
+
+
+
+# instantiates a SSLEngine, with the following things to keep in mind:
+
+# FIXME not yet supported
+# suppress_ragged_eofs - presumably this is an exception we can detect in Netty, the underlying SSLEngine certainly does
+# ssl_version - use SSLEngine.setEnabledProtocols(java.lang.String[])
+# ciphers - SSLEngine.setEnabledCipherSuites(String[] suites)
+
+def wrap_socket(sock, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE,
+                ssl_version=None, ca_certs=None, do_handshake_on_connect=True,
+                suppress_ragged_eofs=True, ciphers=None):
+    return SSLSocket(
+        sock, 
+        keyfile=keyfile, certfile=certfile, ca_certs=ca_certs,
+        server_side=server_side,
+        do_handshake_on_connect=do_handshake_on_connect)
+
+
+def unwrap_socket(sock):
+    # FIXME removing SSL handler from pipeline should suffice, but low pri for now
+    raise NotImplemented()
+
+
+# Underlying Java does a good job of managing entropy, so these are just no-ops
+
+def RAND_status():
+    return True
+
+def RAND_egd(path):
+    pass
+
+def RAND_add(bytes, entropy):
+    pass
+
+
diff --git a/Lib/telnetlib.py b/Lib/telnetlib.py
deleted file mode 100644
--- a/Lib/telnetlib.py
+++ /dev/null
@@ -1,664 +0,0 @@
-r"""TELNET client class.
-
-Based on RFC 854: TELNET Protocol Specification, by J. Postel and
-J. Reynolds
-
-Example:
-
->>> from telnetlib import Telnet
->>> tn = Telnet('www.python.org', 79)   # connect to finger port
->>> tn.write('guido\r\n')
->>> print tn.read_all()
-Login       Name               TTY         Idle    When    Where
-guido    Guido van Rossum      pts/2        <Dec  2 11:10> snag.cnri.reston..
-
->>>
-
-Note that read_all() won't read until eof -- it just reads some data
--- but it guarantees to read at least one byte unless EOF is hit.
-
-It is possible to pass a Telnet object to select.select() in order to
-wait until more data is available.  Note that in this case,
-read_eager() may return '' even if there was data on the socket,
-because the protocol negotiation may have eaten the data.  This is why
-EOFError is needed in some cases to distinguish between "no data" and
-"connection closed" (since the socket also appears ready for reading
-when it is closed).
-
-To do:
-- option negotiation
-- timeout should be intrinsic to the connection object instead of an
-  option on one of the read calls only
-
-"""
-
-
-# Imported modules
-import sys
-import socket
-import select
-import os
-if os.name == 'java':
-    from select import cpython_compatible_select as select
-else:
-    from select import select
-del os
-
-
-__all__ = ["Telnet"]
-
-# Tunable parameters
-DEBUGLEVEL = 0
-
-# Telnet protocol defaults
-TELNET_PORT = 23
-
-# Telnet protocol characters (don't change)
-IAC  = chr(255) # "Interpret As Command"
-DONT = chr(254)
-DO   = chr(253)
-WONT = chr(252)
-WILL = chr(251)
-theNULL = chr(0)
-
-SE  = chr(240)  # Subnegotiation End
-NOP = chr(241)  # No Operation
-DM  = chr(242)  # Data Mark
-BRK = chr(243)  # Break
-IP  = chr(244)  # Interrupt process
-AO  = chr(245)  # Abort output
-AYT = chr(246)  # Are You There
-EC  = chr(247)  # Erase Character
-EL  = chr(248)  # Erase Line
-GA  = chr(249)  # Go Ahead
-SB =  chr(250)  # Subnegotiation Begin
-
-
-# Telnet protocol options code (don't change)
-# These ones all come from arpa/telnet.h
-BINARY = chr(0) # 8-bit data path
-ECHO = chr(1) # echo
-RCP = chr(2) # prepare to reconnect
-SGA = chr(3) # suppress go ahead
-NAMS = chr(4) # approximate message size
-STATUS = chr(5) # give status
-TM = chr(6) # timing mark
-RCTE = chr(7) # remote controlled transmission and echo
-NAOL = chr(8) # negotiate about output line width
-NAOP = chr(9) # negotiate about output page size
-NAOCRD = chr(10) # negotiate about CR disposition
-NAOHTS = chr(11) # negotiate about horizontal tabstops
-NAOHTD = chr(12) # negotiate about horizontal tab disposition
-NAOFFD = chr(13) # negotiate about formfeed disposition
-NAOVTS = chr(14) # negotiate about vertical tab stops
-NAOVTD = chr(15) # negotiate about vertical tab disposition
-NAOLFD = chr(16) # negotiate about output LF disposition
-XASCII = chr(17) # extended ascii character set
-LOGOUT = chr(18) # force logout
-BM = chr(19) # byte macro
-DET = chr(20) # data entry terminal
-SUPDUP = chr(21) # supdup protocol
-SUPDUPOUTPUT = chr(22) # supdup output
-SNDLOC = chr(23) # send location
-TTYPE = chr(24) # terminal type
-EOR = chr(25) # end or record
-TUID = chr(26) # TACACS user identification
-OUTMRK = chr(27) # output marking
-TTYLOC = chr(28) # terminal location number
-VT3270REGIME = chr(29) # 3270 regime
-X3PAD = chr(30) # X.3 PAD
-NAWS = chr(31) # window size
-TSPEED = chr(32) # terminal speed
-LFLOW = chr(33) # remote flow control
-LINEMODE = chr(34) # Linemode option
-XDISPLOC = chr(35) # X Display Location
-OLD_ENVIRON = chr(36) # Old - Environment variables
-AUTHENTICATION = chr(37) # Authenticate
-ENCRYPT = chr(38) # Encryption option
-NEW_ENVIRON = chr(39) # New - Environment variables
-# the following ones come from
-# http://www.iana.org/assignments/telnet-options
-# Unfortunately, that document does not assign identifiers
-# to all of them, so we are making them up
-TN3270E = chr(40) # TN3270E
-XAUTH = chr(41) # XAUTH
-CHARSET = chr(42) # CHARSET
-RSP = chr(43) # Telnet Remote Serial Port
-COM_PORT_OPTION = chr(44) # Com Port Control Option
-SUPPRESS_LOCAL_ECHO = chr(45) # Telnet Suppress Local Echo
-TLS = chr(46) # Telnet Start TLS
-KERMIT = chr(47) # KERMIT
-SEND_URL = chr(48) # SEND-URL
-FORWARD_X = chr(49) # FORWARD_X
-PRAGMA_LOGON = chr(138) # TELOPT PRAGMA LOGON
-SSPI_LOGON = chr(139) # TELOPT SSPI LOGON
-PRAGMA_HEARTBEAT = chr(140) # TELOPT PRAGMA HEARTBEAT
-EXOPL = chr(255) # Extended-Options-List
-NOOPT = chr(0)
-
-class Telnet:
-
-    """Telnet interface class.
-
-    An instance of this class represents a connection to a telnet
-    server.  The instance is initially not connected; the open()
-    method must be used to establish a connection.  Alternatively, the
-    host name and optional port number can be passed to the
-    constructor, too.
-
-    Don't try to reopen an already connected instance.
-
-    This class has many read_*() methods.  Note that some of them
-    raise EOFError when the end of the connection is read, because
-    they can return an empty string for other reasons.  See the
-    individual doc strings.
-
-    read_until(expected, [timeout])
-        Read until the expected string has been seen, or a timeout is
-        hit (default is no timeout); may block.
-
-    read_all()
-        Read all data until EOF; may block.
-
-    read_some()
-        Read at least one byte or EOF; may block.
-
-    read_very_eager()
-        Read all data available already queued or on the socket,
-        without blocking.
-
-    read_eager()
-        Read either data already queued or some data available on the
-        socket, without blocking.
-
-    read_lazy()
-        Read all data in the raw queue (processing it first), without
-        doing any socket I/O.
-
-    read_very_lazy()
-        Reads all data in the cooked queue, without doing any socket
-        I/O.
-
-    read_sb_data()
-        Reads available data between SB ... SE sequence. Don't block.
-
-    set_option_negotiation_callback(callback)
-        Each time a telnet option is read on the input flow, this callback
-        (if set) is called with the following parameters :
-        callback(telnet socket, command, option)
-            option will be chr(0) when there is no option.
-        No other action is done afterwards by telnetlib.
-
-    """
-
-    def __init__(self, host=None, port=0,
-                 timeout=socket._GLOBAL_DEFAULT_TIMEOUT):
-        """Constructor.
-
-        When called without arguments, create an unconnected instance.
-        With a hostname argument, it connects the instance; port number
-        and timeout are optional.
-        """
-        self.debuglevel = DEBUGLEVEL
-        self.host = host
-        self.port = port
-        self.timeout = timeout
-        self.sock = None
-        self.rawq = ''
-        self.irawq = 0
-        self.cookedq = ''
-        self.eof = 0
-        self.iacseq = '' # Buffer for IAC sequence.
-        self.sb = 0 # flag for SB and SE sequence.
-        self.sbdataq = ''
-        self.option_callback = None
-        if host is not None:
-            self.open(host, port, timeout)
-
-    def open(self, host, port=0, timeout=socket._GLOBAL_DEFAULT_TIMEOUT):
-        """Connect to a host.
-
-        The optional second argument is the port number, which
-        defaults to the standard telnet port (23).
-
-        Don't try to reopen an already connected instance.
-        """
-        self.eof = 0
-        if not port:
-            port = TELNET_PORT
-        self.host = host
-        self.port = port
-        self.timeout = timeout
-        self.sock = socket.create_connection((host, port), timeout)
-
-    def __del__(self):
-        """Destructor -- close the connection."""
-        self.close()
-
-    def msg(self, msg, *args):
-        """Print a debug message, when the debug level is > 0.
-
-        If extra arguments are present, they are substituted in the
-        message using the standard string formatting operator.
-
-        """
-        if self.debuglevel > 0:
-            print 'Telnet(%s,%s):' % (self.host, self.port),
-            if args:
-                print msg % args
-            else:
-                print msg
-
-    def set_debuglevel(self, debuglevel):
-        """Set the debug level.
-
-        The higher it is, the more debug output you get (on sys.stdout).
-
-        """
-        self.debuglevel = debuglevel
-
-    def close(self):
-        """Close the connection."""
-        if self.sock:
-            self.sock.close()
-        self.sock = 0
-        self.eof = 1
-        self.iacseq = ''
-        self.sb = 0
-
-    def get_socket(self):
-        """Return the socket object used internally."""
-        return self.sock
-
-    def fileno(self):
-        """Return the fileno() of the socket object used internally."""
-        return self.sock.fileno()
-
-    def write(self, buffer):
-        """Write a string to the socket, doubling any IAC characters.
-
-        Can block if the connection is blocked.  May raise
-        socket.error if the connection is closed.
-
-        """
-        if IAC in buffer:
-            buffer = buffer.replace(IAC, IAC+IAC)
-        self.msg("send %r", buffer)
-        self.sock.sendall(buffer)
-
-    def read_until(self, match, timeout=None):
-        """Read until a given string is encountered or until timeout.
-
-        When no match is found, return whatever is available instead,
-        possibly the empty string.  Raise EOFError if the connection
-        is closed and no cooked data is available.
-
-        """
-        n = len(match)
-        self.process_rawq()
-        i = self.cookedq.find(match)
-        if i >= 0:
-            i = i+n
-            buf = self.cookedq[:i]
-            self.cookedq = self.cookedq[i:]
-            return buf
-        s_reply = ([self], [], [])
-        s_args = s_reply
-        if timeout is not None:
-            s_args = s_args + (timeout,)
-            from time import time
-            time_start = time()
-        while not self.eof and select(*s_args) == s_reply:
-            i = max(0, len(self.cookedq)-n)
-            self.fill_rawq()
-            self.process_rawq()
-            i = self.cookedq.find(match, i)
-            if i >= 0:
-                i = i+n
-                buf = self.cookedq[:i]
-                self.cookedq = self.cookedq[i:]
-                return buf
-            if timeout is not None:
-                elapsed = time() - time_start
-                if elapsed >= timeout:
-                    break
-                s_args = s_reply + (timeout-elapsed,)
-        return self.read_very_lazy()
-
-    def read_all(self):
-        """Read all data until EOF; block until connection closed."""
-        self.process_rawq()
-        while not self.eof:
-            self.fill_rawq()
-            self.process_rawq()
-        buf = self.cookedq
-        self.cookedq = ''
-        return buf
-
-    def read_some(self):
-        """Read at least one byte of cooked data unless EOF is hit.
-
-        Return '' if EOF is hit.  Block if no data is immediately
-        available.
-
-        """
-        self.process_rawq()
-        while not self.cookedq and not self.eof:
-            self.fill_rawq()
-            self.process_rawq()
-        buf = self.cookedq
-        self.cookedq = ''
-        return buf
-
-    def read_very_eager(self):
-        """Read everything that's possible without blocking in I/O (eager).
-
-        Raise EOFError if connection closed and no cooked data
-        available.  Return '' if no cooked data available otherwise.
-        Don't block unless in the midst of an IAC sequence.
-
-        """
-        self.process_rawq()
-        while not self.eof and self.sock_avail():
-            self.fill_rawq()
-            self.process_rawq()
-        return self.read_very_lazy()
-
-    def read_eager(self):
-        """Read readily available data.
-
-        Raise EOFError if connection closed and no cooked data
-        available.  Return '' if no cooked data available otherwise.
-        Don't block unless in the midst of an IAC sequence.
-
-        """
-        self.process_rawq()
-        while not self.cookedq and not self.eof and self.sock_avail():
-            self.fill_rawq()
-            self.process_rawq()
-        return self.read_very_lazy()
-
-    def read_lazy(self):
-        """Process and return data that's already in the queues (lazy).
-
-        Raise EOFError if connection closed and no data available.
-        Return '' if no cooked data available otherwise.  Don't block
-        unless in the midst of an IAC sequence.
-
-        """
-        self.process_rawq()
-        return self.read_very_lazy()
-
-    def read_very_lazy(self):
-        """Return any data available in the cooked queue (very lazy).
-
-        Raise EOFError if connection closed and no data available.
-        Return '' if no cooked data available otherwise.  Don't block.
-
-        """
-        buf = self.cookedq
-        self.cookedq = ''
-        if not buf and self.eof and not self.rawq:
-            raise EOFError, 'telnet connection closed'
-        return buf
-
-    def read_sb_data(self):
-        """Return any data available in the SB ... SE queue.
-
-        Return '' if no SB ... SE available. Should only be called
-        after seeing a SB or SE command. When a new SB command is
-        found, old unread SB data will be discarded. Don't block.
-
-        """
-        buf = self.sbdataq
-        self.sbdataq = ''
-        return buf
-
-    def set_option_negotiation_callback(self, callback):
-        """Provide a callback function called after each receipt of a telnet option."""
-        self.option_callback = callback
-
-    def process_rawq(self):
-        """Transfer from raw queue to cooked queue.
-
-        Set self.eof when connection is closed.  Don't block unless in
-        the midst of an IAC sequence.
-
-        """
-        buf = ['', '']
-        try:
-            while self.rawq:
-                c = self.rawq_getchar()
-                if not self.iacseq:
-                    if c == theNULL:
-                        continue
-                    if c == "\021":
-                        continue
-                    if c != IAC:
-                        buf[self.sb] = buf[self.sb] + c
-                        continue
-                    else:
-                        self.iacseq += c
-                elif len(self.iacseq) == 1:
-                    # 'IAC: IAC CMD [OPTION only for WILL/WONT/DO/DONT]'
-                    if c in (DO, DONT, WILL, WONT):
-                        self.iacseq += c
-                        continue
-
-                    self.iacseq = ''
-                    if c == IAC:
-                        buf[self.sb] = buf[self.sb] + c
-                    else:
-                        if c == SB: # SB ... SE start.
-                            self.sb = 1
-                            self.sbdataq = ''
-                        elif c == SE:
-                            self.sb = 0
-                            self.sbdataq = self.sbdataq + buf[1]
-                            buf[1] = ''
-                        if self.option_callback:
-                            # Callback is supposed to look into
-                            # the sbdataq
-                            self.option_callback(self.sock, c, NOOPT)
-                        else:
-                            # We can't offer automatic processing of
-                            # suboptions. Alas, we should not get any
-                            # unless we did a WILL/DO before.
-                            self.msg('IAC %d not recognized' % ord(c))
-                elif len(self.iacseq) == 2:
-                    cmd = self.iacseq[1]
-                    self.iacseq = ''
-                    opt = c
-                    if cmd in (DO, DONT):
-                        self.msg('IAC %s %d',
-                            cmd == DO and 'DO' or 'DONT', ord(opt))
-                        if self.option_callback:
-                            self.option_callback(self.sock, cmd, opt)
-                        else:
-                            self.sock.sendall(IAC + WONT + opt)
-                    elif cmd in (WILL, WONT):
-                        self.msg('IAC %s %d',
-                            cmd == WILL and 'WILL' or 'WONT', ord(opt))
-                        if self.option_callback:
-                            self.option_callback(self.sock, cmd, opt)
-                        else:
-                            self.sock.sendall(IAC + DONT + opt)
-        except EOFError: # raised by self.rawq_getchar()
-            self.iacseq = '' # Reset on EOF
-            self.sb = 0
-            pass
-        self.cookedq = self.cookedq + buf[0]
-        self.sbdataq = self.sbdataq + buf[1]
-
-    def rawq_getchar(self):
-        """Get next char from raw queue.
-
-        Block if no data is immediately available.  Raise EOFError
-        when connection is closed.
-
-        """
-        if not self.rawq:
-            self.fill_rawq()
-            if self.eof:
-                raise EOFError
-        c = self.rawq[self.irawq]
-        self.irawq = self.irawq + 1
-        if self.irawq >= len(self.rawq):
-            self.rawq = ''
-            self.irawq = 0
-        return c
-
-    def fill_rawq(self):
-        """Fill raw queue from exactly one recv() system call.
-
-        Block if no data is immediately available.  Set self.eof when
-        connection is closed.
-
-        """
-        if self.irawq >= len(self.rawq):
-            self.rawq = ''
-            self.irawq = 0
-        # The buffer size should be fairly small so as to avoid quadratic
-        # behavior in process_rawq() above
-        buf = self.sock.recv(50)
-        self.msg("recv %r", buf)
-        self.eof = (not buf)
-        self.rawq = self.rawq + buf
-
-    def sock_avail(self):
-        """Test whether data is available on the socket."""
-        return select([self], [], [], 0) == ([self], [], [])
-
-    def interact(self):
-        """Interaction function, emulates a very dumb telnet client."""
-        if sys.platform == "win32":
-            self.mt_interact()
-            return
-        while 1:
-            rfd, wfd, xfd = select([self, sys.stdin], [], [])
-            if self in rfd:
-                try:
-                    text = self.read_eager()
-                except EOFError:
-                    print '*** Connection closed by remote host ***'
-                    break
-                if text:
-                    sys.stdout.write(text)
-                    sys.stdout.flush()
-            if sys.stdin in rfd:
-                line = sys.stdin.readline()
-                if not line:
-                    break
-                self.write(line)
-
-    def mt_interact(self):
-        """Multithreaded version of interact()."""
-        import thread
-        thread.start_new_thread(self.listener, ())
-        while 1:
-            line = sys.stdin.readline()
-            if not line:
-                break
-            self.write(line)
-
-    def listener(self):
-        """Helper for mt_interact() -- this executes in the other thread."""
-        while 1:
-            try:
-                data = self.read_eager()
-            except EOFError:
-                print '*** Connection closed by remote host ***'
-                return
-            if data:
-                sys.stdout.write(data)
-            else:
-                sys.stdout.flush()
-
-    def expect(self, list, timeout=None):
-        """Read until one from a list of a regular expressions matches.
-
-        The first argument is a list of regular expressions, either
-        compiled (re.RegexObject instances) or uncompiled (strings).
-        The optional second argument is a timeout, in seconds; default
-        is no timeout.
-
-        Return a tuple of three items: the index in the list of the
-        first regular expression that matches; the match object
-        returned; and the text read up till and including the match.
-
-        If EOF is read and no text was read, raise EOFError.
-        Otherwise, when nothing matches, return (-1, None, text) where
-        text is the text received so far (may be the empty string if a
-        timeout happened).
-
-        If a regular expression ends with a greedy match (e.g. '.*')
-        or if more than one expression can match the same input, the
-        results are undeterministic, and may depend on the I/O timing.
-
-        """
-        re = None
-        list = list[:]
-        indices = range(len(list))
-        for i in indices:
-            if not hasattr(list[i], "search"):
-                if not re: import re
-                list[i] = re.compile(list[i])
-        if timeout is not None:
-            from time import time
-            time_start = time()
-        while 1:
-            self.process_rawq()
-            for i in indices:
-                m = list[i].search(self.cookedq)
-                if m:
-                    e = m.end()
-                    text = self.cookedq[:e]
-                    self.cookedq = self.cookedq[e:]
-                    return (i, m, text)
-            if self.eof:
-                break
-            if timeout is not None:
-                elapsed = time() - time_start
-                if elapsed >= timeout:
-                    break
-                s_args = ([self.fileno()], [], [], timeout-elapsed)
-                r, w, x = select(*s_args)
-                if not r:
-                    break
-            self.fill_rawq()
-        text = self.read_very_lazy()
-        if not text and self.eof:
-            raise EOFError
-        return (-1, None, text)
-
-
-def test():
-    """Test program for telnetlib.
-
-    Usage: python telnetlib.py [-d] ... [host [port]]
-
-    Default host is localhost; default port is 23.
-
-    """
-    debuglevel = 0
-    while sys.argv[1:] and sys.argv[1] == '-d':
-        debuglevel = debuglevel+1
-        del sys.argv[1]
-    host = 'localhost'
-    if sys.argv[1:]:
-        host = sys.argv[1]
-    port = 0
-    if sys.argv[2:]:
-        portstr = sys.argv[2]
-        try:
-            port = int(portstr)
-        except ValueError:
-            port = socket.getservbyname(portstr, 'tcp')
-    tn = Telnet()
-    tn.set_debuglevel(debuglevel)
-    tn.open(host, port, timeout=0.5)
-    tn.interact()
-    tn.close()
-
-if __name__ == '__main__':
-    test()
diff --git a/Lib/test/test_httplib.py b/Lib/test/test_httplib.py
deleted file mode 100644
--- a/Lib/test/test_httplib.py
+++ /dev/null
@@ -1,472 +0,0 @@
-import httplib
-import array
-import httplib
-import StringIO
-import socket
-import errno
-
-import unittest
-TestCase = unittest.TestCase
-
-from test import test_support
-
-HOST = test_support.HOST
-
-class FakeSocket:
-    def __init__(self, text, fileclass=StringIO.StringIO):
-        self.text = text
-        self.fileclass = fileclass
-        self.data = ''
-
-    def sendall(self, data):
-        self.data += ''.join(data)
-
-    def makefile(self, mode, bufsize=None):
-        if mode != 'r' and mode != 'rb':
-            raise httplib.UnimplementedFileMode()
-        return self.fileclass(self.text)
-
-class EPipeSocket(FakeSocket):
-
-    def __init__(self, text, pipe_trigger):
-        # When sendall() is called with pipe_trigger, raise EPIPE.
-        FakeSocket.__init__(self, text)
-        self.pipe_trigger = pipe_trigger
-
-    def sendall(self, data):
-        if self.pipe_trigger in data:
-            raise socket.error(errno.EPIPE, "gotcha")
-        self.data += data
-
-    def close(self):
-        pass
-
-class NoEOFStringIO(StringIO.StringIO):
-    """Like StringIO, but raises AssertionError on EOF.
-
-    This is used below to test that httplib doesn't try to read
-    more from the underlying file than it should.
-    """
-    def read(self, n=-1):
-        data = StringIO.StringIO.read(self, n)
-        if data == '':
-            raise AssertionError('caller tried to read past EOF')
-        return data
-
-    def readline(self, length=None):
-        data = StringIO.StringIO.readline(self, length)
-        if data == '':
-            raise AssertionError('caller tried to read past EOF')
-        return data
-
-
-class HeaderTests(TestCase):
-    def test_auto_headers(self):
-        # Some headers are added automatically, but should not be added by
-        # .request() if they are explicitly set.
-
-        class HeaderCountingBuffer(list):
-            def __init__(self):
-                self.count = {}
-            def append(self, item):
-                kv = item.split(':')
-                if len(kv) > 1:
-                    # item is a 'Key: Value' header string
-                    lcKey = kv[0].lower()
-                    self.count.setdefault(lcKey, 0)
-                    self.count[lcKey] += 1
-                list.append(self, item)
-
-        for explicit_header in True, False:
-            for header in 'Content-length', 'Host', 'Accept-encoding':
-                conn = httplib.HTTPConnection('example.com')
-                conn.sock = FakeSocket('blahblahblah')
-                conn._buffer = HeaderCountingBuffer()
-
-                body = 'spamspamspam'
-                headers = {}
-                if explicit_header:
-                    headers[header] = str(len(body))
-                conn.request('POST', '/', body, headers)
-                self.assertEqual(conn._buffer.count[header.lower()], 1)
-
-    def test_putheader(self):
-        conn = httplib.HTTPConnection('example.com')
-        conn.sock = FakeSocket(None)
-        conn.putrequest('GET','/')
-        conn.putheader('Content-length',42)
-        self.assertTrue('Content-length: 42' in conn._buffer)
-
-    def test_ipv6host_header(self):
-        # Default host header on IPv6 transaction should wrapped by [] if
-        # its actual IPv6 address
-        expected = 'GET /foo HTTP/1.1\r\nHost: [2001::]:81\r\n' \
-                   'Accept-Encoding: identity\r\n\r\n'
-        conn = httplib.HTTPConnection('[2001::]:81')
-        sock = FakeSocket('')
-        conn.sock = sock
-        conn.request('GET', '/foo')
-        self.assertTrue(sock.data.startswith(expected))
-
-        expected = 'GET /foo HTTP/1.1\r\nHost: [2001:102A::]\r\n' \
-                   'Accept-Encoding: identity\r\n\r\n'
-        conn = httplib.HTTPConnection('[2001:102A::]')
-        sock = FakeSocket('')
-        conn.sock = sock
-        conn.request('GET', '/foo')
-        self.assertTrue(sock.data.startswith(expected))
-
-
-class BasicTest(TestCase):
-    def test_status_lines(self):
-        # Test HTTP status lines
-
-        body = "HTTP/1.1 200 Ok\r\n\r\nText"
-        sock = FakeSocket(body)
-        resp = httplib.HTTPResponse(sock)
-        resp.begin()
-        self.assertEqual(resp.read(), 'Text')
-        self.assertTrue(resp.isclosed())
-
-        body = "HTTP/1.1 400.100 Not Ok\r\n\r\nText"
-        sock = FakeSocket(body)
-        resp = httplib.HTTPResponse(sock)
-        self.assertRaises(httplib.BadStatusLine, resp.begin)
-
-    def test_bad_status_repr(self):
-        exc = httplib.BadStatusLine('')
-        self.assertEqual(repr(exc), '''BadStatusLine("\'\'",)''')
-
-    def test_partial_reads(self):
-        # if we have a lenght, the system knows when to close itself
-        # same behaviour than when we read the whole thing with read()
-        body = "HTTP/1.1 200 Ok\r\nContent-Length: 4\r\n\r\nText"
-        sock = FakeSocket(body)
-        resp = httplib.HTTPResponse(sock)
-        resp.begin()
-        self.assertEqual(resp.read(2), 'Te')
-        self.assertFalse(resp.isclosed())
-        self.assertEqual(resp.read(2), 'xt')
-        self.assertTrue(resp.isclosed())
-
-    def test_host_port(self):
-        # Check invalid host_port
-
-        # Note that httplib does not accept user:password@ in the host-port.
-        for hp in ("www.python.org:abc", "user:password at www.python.org"):
-            self.assertRaises(httplib.InvalidURL, httplib.HTTP, hp)
-
-        for hp, h, p in (("[fe80::207:e9ff:fe9b]:8000", "fe80::207:e9ff:fe9b",
-                          8000),
-                         ("www.python.org:80", "www.python.org", 80),
-                         ("www.python.org", "www.python.org", 80),
-                         ("www.python.org:", "www.python.org", 80),
-                         ("[fe80::207:e9ff:fe9b]", "fe80::207:e9ff:fe9b", 80)):
-            http = httplib.HTTP(hp)
-            c = http._conn
-            if h != c.host:
-                self.fail("Host incorrectly parsed: %s != %s" % (h, c.host))
-            if p != c.port:
-                self.fail("Port incorrectly parsed: %s != %s" % (p, c.host))
-
-    def test_response_headers(self):
-        # test response with multiple message headers with the same field name.
-        text = ('HTTP/1.1 200 OK\r\n'
-                'Set-Cookie: Customer="WILE_E_COYOTE";'
-                ' Version="1"; Path="/acme"\r\n'
-                'Set-Cookie: Part_Number="Rocket_Launcher_0001"; Version="1";'
-                ' Path="/acme"\r\n'
-                '\r\n'
-                'No body\r\n')
-        hdr = ('Customer="WILE_E_COYOTE"; Version="1"; Path="/acme"'
-               ', '
-               'Part_Number="Rocket_Launcher_0001"; Version="1"; Path="/acme"')
-        s = FakeSocket(text)
-        r = httplib.HTTPResponse(s)
-        r.begin()
-        cookies = r.getheader("Set-Cookie")
-        if cookies != hdr:
-            self.fail("multiple headers not combined properly")
-
-    def test_read_head(self):
-        # Test that the library doesn't attempt to read any data
-        # from a HEAD request.  (Tickles SF bug #622042.)
-        sock = FakeSocket(
-            'HTTP/1.1 200 OK\r\n'
-            'Content-Length: 14432\r\n'
-            '\r\n',
-            NoEOFStringIO)
-        resp = httplib.HTTPResponse(sock, method="HEAD")
-        resp.begin()
-        if resp.read() != "":
-            self.fail("Did not expect response from HEAD request")
-
-    @unittest.skipIf(test_support.is_jython, "FIXME: not working on Jython")
-    def test_send_file(self):
-        expected = 'GET /foo HTTP/1.1\r\nHost: example.com\r\n' \
-                   'Accept-Encoding: identity\r\nContent-Length:'
-
-        body = open(__file__, 'rb')
-        conn = httplib.HTTPConnection('example.com')
-        sock = FakeSocket(body)
-        conn.sock = sock
-        conn.request('GET', '/foo', body)
-        self.assertTrue(sock.data.startswith(expected))
-
-    def test_send(self):
-        expected = 'this is a test this is only a test'
-        conn = httplib.HTTPConnection('example.com')
-        sock = FakeSocket(None)
-        conn.sock = sock
-        conn.send(expected)
-        self.assertEqual(expected, sock.data)
-        sock.data = ''
-        conn.send(array.array('c', expected))
-        self.assertEqual(expected, sock.data)
-        sock.data = ''
-        conn.send(StringIO.StringIO(expected))
-        self.assertEqual(expected, sock.data)
-
-    def test_chunked(self):
-        chunked_start = (
-            'HTTP/1.1 200 OK\r\n'
-            'Transfer-Encoding: chunked\r\n\r\n'
-            'a\r\n'
-            'hello worl\r\n'
-            '1\r\n'
-            'd\r\n'
-        )
-        sock = FakeSocket(chunked_start + '0\r\n')
-        resp = httplib.HTTPResponse(sock, method="GET")
-        resp.begin()
-        self.assertEqual(resp.read(), 'hello world')
-        resp.close()
-
-        for x in ('', 'foo\r\n'):
-            sock = FakeSocket(chunked_start + x)
-            resp = httplib.HTTPResponse(sock, method="GET")
-            resp.begin()
-            try:
-                resp.read()
-            except httplib.IncompleteRead, i:
-                self.assertEqual(i.partial, 'hello world')
-                self.assertEqual(repr(i),'IncompleteRead(11 bytes read)')
-                self.assertEqual(str(i),'IncompleteRead(11 bytes read)')
-            else:
-                self.fail('IncompleteRead expected')
-            finally:
-                resp.close()
-
-    def test_chunked_head(self):
-        chunked_start = (
-            'HTTP/1.1 200 OK\r\n'
-            'Transfer-Encoding: chunked\r\n\r\n'
-            'a\r\n'
-            'hello world\r\n'
-            '1\r\n'
-            'd\r\n'
-        )
-        sock = FakeSocket(chunked_start + '0\r\n')
-        resp = httplib.HTTPResponse(sock, method="HEAD")
-        resp.begin()
-        self.assertEqual(resp.read(), '')
-        self.assertEqual(resp.status, 200)
-        self.assertEqual(resp.reason, 'OK')
-        self.assertTrue(resp.isclosed())
-
-    def test_negative_content_length(self):
-        sock = FakeSocket('HTTP/1.1 200 OK\r\n'
-                          'Content-Length: -1\r\n\r\nHello\r\n')
-        resp = httplib.HTTPResponse(sock, method="GET")
-        resp.begin()
-        self.assertEqual(resp.read(), 'Hello\r\n')
-        resp.close()
-
-    def test_incomplete_read(self):
-        sock = FakeSocket('HTTP/1.1 200 OK\r\nContent-Length: 10\r\n\r\nHello\r\n')
-        resp = httplib.HTTPResponse(sock, method="GET")
-        resp.begin()
-        try:
-            resp.read()
-        except httplib.IncompleteRead as i:
-            self.assertEqual(i.partial, 'Hello\r\n')
-            self.assertEqual(repr(i),
-                             "IncompleteRead(7 bytes read, 3 more expected)")
-            self.assertEqual(str(i),
-                             "IncompleteRead(7 bytes read, 3 more expected)")
-        else:
-            self.fail('IncompleteRead expected')
-        finally:
-            resp.close()
-
-    def test_epipe(self):
-        sock = EPipeSocket(
-            "HTTP/1.0 401 Authorization Required\r\n"
-            "Content-type: text/html\r\n"
-            "WWW-Authenticate: Basic realm=\"example\"\r\n",
-            b"Content-Length")
-        conn = httplib.HTTPConnection("example.com")
-        conn.sock = sock
-        self.assertRaises(socket.error,
-                          lambda: conn.request("PUT", "/url", "body"))
-        resp = conn.getresponse()
-        self.assertEqual(401, resp.status)
-        self.assertEqual("Basic realm=\"example\"",
-                         resp.getheader("www-authenticate"))
-
-    def test_filenoattr(self):
-        # Just test the fileno attribute in the HTTPResponse Object.
-        body = "HTTP/1.1 200 Ok\r\n\r\nText"
-        sock = FakeSocket(body)
-        resp = httplib.HTTPResponse(sock)
-        self.assertTrue(hasattr(resp,'fileno'),
-                'HTTPResponse should expose a fileno attribute')
-
-    # Test lines overflowing the max line size (_MAXLINE in http.client)
-
-    def test_overflowing_status_line(self):
-        self.skipTest("disabled for HTTP 0.9 support")
-        body = "HTTP/1.1 200 Ok" + "k" * 65536 + "\r\n"
-        resp = httplib.HTTPResponse(FakeSocket(body))
-        self.assertRaises((httplib.LineTooLong, httplib.BadStatusLine), resp.begin)
-
-    def test_overflowing_header_line(self):
-        body = (
-            'HTTP/1.1 200 OK\r\n'
-            'X-Foo: bar' + 'r' * 65536 + '\r\n\r\n'
-        )
-        resp = httplib.HTTPResponse(FakeSocket(body))
-        self.assertRaises(httplib.LineTooLong, resp.begin)
-
-    def test_overflowing_chunked_line(self):
-        body = (
-            'HTTP/1.1 200 OK\r\n'
-            'Transfer-Encoding: chunked\r\n\r\n'
-            + '0' * 65536 + 'a\r\n'
-            'hello world\r\n'
-            '0\r\n'
-        )
-        resp = httplib.HTTPResponse(FakeSocket(body))
-        resp.begin()
-        self.assertRaises(httplib.LineTooLong, resp.read)
-
-
-class OfflineTest(TestCase):
-    def test_responses(self):
-        self.assertEqual(httplib.responses[httplib.NOT_FOUND], "Not Found")
-
-
-class SourceAddressTest(TestCase):
-    def setUp(self):
-        self.serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.port = test_support.bind_port(self.serv)
-        self.source_port = test_support.find_unused_port()
-        self.serv.listen(5)
-        self.conn = None
-
-    def tearDown(self):
-        if self.conn:
-            self.conn.close()
-            self.conn = None
-        self.serv.close()
-        self.serv = None
-
-    def testHTTPConnectionSourceAddress(self):
-        self.conn = httplib.HTTPConnection(HOST, self.port,
-                source_address=('', self.source_port))
-        self.conn.connect()
-        self.assertEqual(self.conn.sock.getsockname()[1], self.source_port)
-
-    @unittest.skipIf(not hasattr(httplib, 'HTTPSConnection'),
-                     'httplib.HTTPSConnection not defined')
-    def testHTTPSConnectionSourceAddress(self):
-        self.conn = httplib.HTTPSConnection(HOST, self.port,
-                source_address=('', self.source_port))
-        # We don't test anything here other the constructor not barfing as
-        # this code doesn't deal with setting up an active running SSL server
-        # for an ssl_wrapped connect() to actually return from.
-
-
-class TimeoutTest(TestCase):
-    PORT = None
-
-    def setUp(self):
-        self.serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        TimeoutTest.PORT = test_support.bind_port(self.serv)
-        self.serv.listen(5)
-
-    def tearDown(self):
-        self.serv.close()
-        self.serv = None
-
-    def testTimeoutAttribute(self):
-        '''This will prove that the timeout gets through
-        HTTPConnection and into the socket.
-        '''
-        # default -- use global socket timeout
-        self.assertTrue(socket.getdefaulttimeout() is None)
-        socket.setdefaulttimeout(30)
-        try:
-            httpConn = httplib.HTTPConnection(HOST, TimeoutTest.PORT)
-            httpConn.connect()
-        finally:
-            socket.setdefaulttimeout(None)
-        self.assertEqual(httpConn.sock.gettimeout(), 30)
-        httpConn.close()
-
-        # no timeout -- do not use global socket default
-        self.assertTrue(socket.getdefaulttimeout() is None)
-        socket.setdefaulttimeout(30)
-        try:
-            httpConn = httplib.HTTPConnection(HOST, TimeoutTest.PORT,
-                                              timeout=None)
-            httpConn.connect()
-        finally:
-            socket.setdefaulttimeout(None)
-        self.assertEqual(httpConn.sock.gettimeout(), None)
-        httpConn.close()
-
-        # a value
-        httpConn = httplib.HTTPConnection(HOST, TimeoutTest.PORT, timeout=30)
-        httpConn.connect()
-        self.assertEqual(httpConn.sock.gettimeout(), 30)
-        httpConn.close()
-
-
-class HTTPSTimeoutTest(TestCase):
-# XXX Here should be tests for HTTPS, there isn't any right now!
-
-    def test_attributes(self):
-        # simple test to check it's storing it
-        if hasattr(httplib, 'HTTPSConnection'):
-            h = httplib.HTTPSConnection(HOST, TimeoutTest.PORT, timeout=30)
-            self.assertEqual(h.timeout, 30)
-
-    @unittest.skipIf(not hasattr(httplib, 'HTTPS'), 'httplib.HTTPS not available')
-    def test_host_port(self):
-        # Check invalid host_port
-
-        # Note that httplib does not accept user:password@ in the host-port.
-        for hp in ("www.python.org:abc", "user:password at www.python.org"):
-            self.assertRaises(httplib.InvalidURL, httplib.HTTP, hp)
-
-        for hp, h, p in (("[fe80::207:e9ff:fe9b]:8000", "fe80::207:e9ff:fe9b",
-                          8000),
-                         ("pypi.python.org:443", "pypi.python.org", 443),
-                         ("pypi.python.org", "pypi.python.org", 443),
-                         ("pypi.python.org:", "pypi.python.org", 443),
-                         ("[fe80::207:e9ff:fe9b]", "fe80::207:e9ff:fe9b", 443)):
-            http = httplib.HTTPS(hp)
-            c = http._conn
-            if h != c.host:
-                self.fail("Host incorrectly parsed: %s != %s" % (h, c.host))
-            if p != c.port:
-                self.fail("Port incorrectly parsed: %s != %s" % (p, c.host))
-
-
-def test_main(verbose=None):
-    test_support.run_unittest(HeaderTests, OfflineTest, BasicTest, TimeoutTest,
-                              HTTPSTimeoutTest, SourceAddressTest)
-
-if __name__ == '__main__':
-    test_main()
diff --git a/Lib/test/test_httpservers.py b/Lib/test/test_httpservers.py
deleted file mode 100644
--- a/Lib/test/test_httpservers.py
+++ /dev/null
@@ -1,543 +0,0 @@
-"""Unittests for the various HTTPServer modules.
-
-Written by Cody A.W. Somerville <cody-somerville at ubuntu.com>,
-Josip Dzolonga, and Michael Otteneder for the 2007/08 GHOP contest.
-"""
-
-import os
-import sys
-import re
-import base64
-import shutil
-import urllib
-import httplib
-import tempfile
-import unittest
-import CGIHTTPServer
-
-
-from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
-from SimpleHTTPServer import SimpleHTTPRequestHandler
-from CGIHTTPServer import CGIHTTPRequestHandler
-from StringIO import StringIO
-from test import test_support
-
-
-threading = test_support.import_module('threading')
-
-
-class NoLogRequestHandler:
-    def log_message(self, *args):
-        # don't write log messages to stderr
-        pass
-
-class SocketlessRequestHandler(SimpleHTTPRequestHandler):
-    def __init__(self):
-        self.get_called = False
-        self.protocol_version = "HTTP/1.1"
-
-    def do_GET(self):
-        self.get_called = True
-        self.send_response(200)
-        self.send_header('Content-Type', 'text/html')
-        self.end_headers()
-        self.wfile.write(b'<html><body>Data</body></html>\r\n')
-
-    def log_message(self, fmt, *args):
-        pass
-
-
-class TestServerThread(threading.Thread):
-    def __init__(self, test_object, request_handler):
-        threading.Thread.__init__(self)
-        self.request_handler = request_handler
-        self.test_object = test_object
-
-    def run(self):
-        self.server = HTTPServer(('', 0), self.request_handler)
-        self.test_object.PORT = self.server.socket.getsockname()[1]
-        self.test_object.server_started.set()
-        self.test_object = None
-        try:
-            self.server.serve_forever(0.05)
-        finally:
-            self.server.server_close()
-
-    def stop(self):
-        self.server.shutdown()
-
-
-class BaseTestCase(unittest.TestCase):
-    def setUp(self):
-        self._threads = test_support.threading_setup()
-        os.environ = test_support.EnvironmentVarGuard()
-        self.server_started = threading.Event()
-        self.thread = TestServerThread(self, self.request_handler)
-        self.thread.start()
-        self.server_started.wait()
-
-    def tearDown(self):
-        self.thread.stop()
-        os.environ.__exit__()
-        test_support.threading_cleanup(*self._threads)
-
-    def request(self, uri, method='GET', body=None, headers={}):
-        self.connection = httplib.HTTPConnection('localhost', self.PORT)
-        self.connection.request(method, uri, body, headers)
-        return self.connection.getresponse()
-
-class BaseHTTPRequestHandlerTestCase(unittest.TestCase):
-    """Test the functionality of the BaseHTTPServer focussing on
-    BaseHTTPRequestHandler.
-    """
-
-    HTTPResponseMatch = re.compile('HTTP/1.[0-9]+ 200 OK')
-
-    def setUp (self):
-        self.handler = SocketlessRequestHandler()
-
-    def send_typical_request(self, message):
-        input_msg = StringIO(message)
-        output = StringIO()
-        self.handler.rfile = input_msg
-        self.handler.wfile = output
-        self.handler.handle_one_request()
-        output.seek(0)
-        return output.readlines()
-
-    def verify_get_called(self):
-        self.assertTrue(self.handler.get_called)
-
-    def verify_expected_headers(self, headers):
-        for fieldName in 'Server: ', 'Date: ', 'Content-Type: ':
-            self.assertEqual(sum(h.startswith(fieldName) for h in headers), 1)
-
-    def verify_http_server_response(self, response):
-        match = self.HTTPResponseMatch.search(response)
-        self.assertTrue(match is not None)
-
-    def test_http_1_1(self):
-        result = self.send_typical_request('GET / HTTP/1.1\r\n\r\n')
-        self.verify_http_server_response(result[0])
-        self.verify_expected_headers(result[1:-1])
-        self.verify_get_called()
-        self.assertEqual(result[-1], '<html><body>Data</body></html>\r\n')
-
-    def test_http_1_0(self):
-        result = self.send_typical_request('GET / HTTP/1.0\r\n\r\n')
-        self.verify_http_server_response(result[0])
-        self.verify_expected_headers(result[1:-1])
-        self.verify_get_called()
-        self.assertEqual(result[-1], '<html><body>Data</body></html>\r\n')
-
-    def test_http_0_9(self):
-        result = self.send_typical_request('GET / HTTP/0.9\r\n\r\n')
-        self.assertEqual(len(result), 1)
-        self.assertEqual(result[0], '<html><body>Data</body></html>\r\n')
-        self.verify_get_called()
-
-    def test_with_continue_1_0(self):
-        result = self.send_typical_request('GET / HTTP/1.0\r\nExpect: 100-continue\r\n\r\n')
-        self.verify_http_server_response(result[0])
-        self.verify_expected_headers(result[1:-1])
-        self.verify_get_called()
-        self.assertEqual(result[-1], '<html><body>Data</body></html>\r\n')
-
-    def test_request_length(self):
-        # Issue #10714: huge request lines are discarded, to avoid Denial
-        # of Service attacks.
-        result = self.send_typical_request(b'GET ' + b'x' * 65537)
-        self.assertEqual(result[0], b'HTTP/1.1 414 Request-URI Too Long\r\n')
-        self.assertFalse(self.handler.get_called)
-
-
-class BaseHTTPServerTestCase(BaseTestCase):
-    class request_handler(NoLogRequestHandler, BaseHTTPRequestHandler):
-        protocol_version = 'HTTP/1.1'
-        default_request_version = 'HTTP/1.1'
-
-        def do_TEST(self):
-            self.send_response(204)
-            self.send_header('Content-Type', 'text/html')
-            self.send_header('Connection', 'close')
-            self.end_headers()
-
-        def do_KEEP(self):
-            self.send_response(204)
-            self.send_header('Content-Type', 'text/html')
-            self.send_header('Connection', 'keep-alive')
-            self.end_headers()
-
-        def do_KEYERROR(self):
-            self.send_error(999)
-
-        def do_CUSTOM(self):
-            self.send_response(999)
-            self.send_header('Content-Type', 'text/html')
-            self.send_header('Connection', 'close')
-            self.end_headers()
-
-    def setUp(self):
-        BaseTestCase.setUp(self)
-        self.con = httplib.HTTPConnection('localhost', self.PORT)
-        self.con.connect()
-
-    def test_command(self):
-        self.con.request('GET', '/')
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 501)
-
-    def test_request_line_trimming(self):
-        self.con._http_vsn_str = 'HTTP/1.1\n'
-        self.con.putrequest('XYZBOGUS', '/')
-        self.con.endheaders()
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 501)
-
-    def test_version_bogus(self):
-        self.con._http_vsn_str = 'FUBAR'
-        self.con.putrequest('GET', '/')
-        self.con.endheaders()
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 400)
-
-    def test_version_digits(self):
-        self.con._http_vsn_str = 'HTTP/9.9.9'
-        self.con.putrequest('GET', '/')
-        self.con.endheaders()
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 400)
-
-    def test_version_none_get(self):
-        self.con._http_vsn_str = ''
-        self.con.putrequest('GET', '/')
-        self.con.endheaders()
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 501)
-
-    def test_version_none(self):
-        # Test that a valid method is rejected when not HTTP/1.x
-        self.con._http_vsn_str = ''
-        self.con.putrequest('CUSTOM', '/')
-        self.con.endheaders()
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 400)
-
-    def test_version_invalid(self):
-        self.con._http_vsn = 99
-        self.con._http_vsn_str = 'HTTP/9.9'
-        self.con.putrequest('GET', '/')
-        self.con.endheaders()
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 505)
-
-    def test_send_blank(self):
-        self.con._http_vsn_str = ''
-        self.con.putrequest('', '')
-        self.con.endheaders()
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 400)
-
-    def test_header_close(self):
-        self.con.putrequest('GET', '/')
-        self.con.putheader('Connection', 'close')
-        self.con.endheaders()
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 501)
-
-    def test_head_keep_alive(self):
-        self.con._http_vsn_str = 'HTTP/1.1'
-        self.con.putrequest('GET', '/')
-        self.con.putheader('Connection', 'keep-alive')
-        self.con.endheaders()
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 501)
-
-    def test_handler(self):
-        self.con.request('TEST', '/')
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 204)
-
-    def test_return_header_keep_alive(self):
-        self.con.request('KEEP', '/')
-        res = self.con.getresponse()
-        self.assertEqual(res.getheader('Connection'), 'keep-alive')
-        self.con.request('TEST', '/')
-        self.addCleanup(self.con.close)
-
-    def test_internal_key_error(self):
-        self.con.request('KEYERROR', '/')
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 999)
-
-    def test_return_custom_status(self):
-        self.con.request('CUSTOM', '/')
-        res = self.con.getresponse()
-        self.assertEqual(res.status, 999)
-
-
-class SimpleHTTPServerTestCase(BaseTestCase):
-    class request_handler(NoLogRequestHandler, SimpleHTTPRequestHandler):
-        pass
-
-    def setUp(self):
-        BaseTestCase.setUp(self)
-        self.cwd = os.getcwd()
-        basetempdir = tempfile.gettempdir()
-        os.chdir(basetempdir)
-        self.data = 'We are the knights who say Ni!'
-        self.tempdir = tempfile.mkdtemp(dir=basetempdir)
-        self.tempdir_name = os.path.basename(self.tempdir)
-        temp = open(os.path.join(self.tempdir, 'test'), 'wb')
-        temp.write(self.data)
-        temp.close()
-
-    def tearDown(self):
-        try:
-            os.chdir(self.cwd)
-            try:
-                shutil.rmtree(self.tempdir)
-            except OSError:
-                pass
-        finally:
-            BaseTestCase.tearDown(self)
-
-    def check_status_and_reason(self, response, status, data=None):
-        body = response.read()
-        self.assertTrue(response)
-        self.assertEqual(response.status, status)
-        self.assertIsNotNone(response.reason)
-        if data:
-            self.assertEqual(data, body)
-
-    def test_get(self):
-        #constructs the path relative to the root directory of the HTTPServer
-        response = self.request(self.tempdir_name + '/test')
-        self.check_status_and_reason(response, 200, data=self.data)
-        response = self.request(self.tempdir_name + '/')
-        self.check_status_and_reason(response, 200)
-        response = self.request(self.tempdir_name)
-        self.check_status_and_reason(response, 301)
-        response = self.request('/ThisDoesNotExist')
-        self.check_status_and_reason(response, 404)
-        response = self.request('/' + 'ThisDoesNotExist' + '/')
-        self.check_status_and_reason(response, 404)
-        f = open(os.path.join(self.tempdir_name, 'index.html'), 'w')
-        response = self.request('/' + self.tempdir_name + '/')
-        self.check_status_and_reason(response, 200)
-
-        # chmod() doesn't work as expected on Windows, and filesystem
-        # permissions are ignored by root on Unix.
-        if os.name == 'posix' and os.geteuid() != 0:
-            os.chmod(self.tempdir, 0)
-            response = self.request(self.tempdir_name + '/')
-            self.check_status_and_reason(response, 404)
-            os.chmod(self.tempdir, 0755)
-
-    def test_head(self):
-        response = self.request(
-            self.tempdir_name + '/test', method='HEAD')
-        self.check_status_and_reason(response, 200)
-        self.assertEqual(response.getheader('content-length'),
-                         str(len(self.data)))
-        self.assertEqual(response.getheader('content-type'),
-                         'application/octet-stream')
-
-    def test_invalid_requests(self):
-        response = self.request('/', method='FOO')
-        self.check_status_and_reason(response, 501)
-        # requests must be case sensitive,so this should fail too
-        response = self.request('/', method='get')
-        self.check_status_and_reason(response, 501)
-        response = self.request('/', method='GETs')
-        self.check_status_and_reason(response, 501)
-
-
-cgi_file1 = """\
-#!%s
-
-print "Content-type: text/html"
-print
-print "Hello World"
-"""
-
-cgi_file2 = """\
-#!%s
-import cgi
-
-print "Content-type: text/html"
-print
-
-form = cgi.FieldStorage()
-print "%%s, %%s, %%s" %% (form.getfirst("spam"), form.getfirst("eggs"),
-                          form.getfirst("bacon"))
-"""
-
-
- at unittest.skipIf(hasattr(os, 'geteuid') and os.geteuid() == 0,
-        "This test can't be run reliably as root (issue #13308).")
-class CGIHTTPServerTestCase(BaseTestCase):
-    class request_handler(NoLogRequestHandler, CGIHTTPRequestHandler):
-        pass
-
-    def setUp(self):
-        BaseTestCase.setUp(self)
-        self.parent_dir = tempfile.mkdtemp()
-        self.cgi_dir = os.path.join(self.parent_dir, 'cgi-bin')
-        os.mkdir(self.cgi_dir)
-
-        # The shebang line should be pure ASCII: use symlink if possible.
-        # See issue #7668.
-        if hasattr(os, 'symlink'):
-            self.pythonexe = os.path.join(self.parent_dir, 'python')
-            os.symlink(sys.executable, self.pythonexe)
-        else:
-            self.pythonexe = sys.executable
-
-        self.file1_path = os.path.join(self.cgi_dir, 'file1.py')
-        with open(self.file1_path, 'w') as file1:
-            file1.write(cgi_file1 % self.pythonexe)
-        os.chmod(self.file1_path, 0777)
-
-        self.file2_path = os.path.join(self.cgi_dir, 'file2.py')
-        with open(self.file2_path, 'w') as file2:
-            file2.write(cgi_file2 % self.pythonexe)
-        os.chmod(self.file2_path, 0777)
-
-        self.cwd = os.getcwd()
-        os.chdir(self.parent_dir)
-
-    def tearDown(self):
-        try:
-            os.chdir(self.cwd)
-            if self.pythonexe != sys.executable:
-                os.remove(self.pythonexe)
-            os.remove(self.file1_path)
-            os.remove(self.file2_path)
-            os.rmdir(self.cgi_dir)
-            os.rmdir(self.parent_dir)
-        finally:
-            BaseTestCase.tearDown(self)
-
-    def test_url_collapse_path(self):
-        # verify tail is the last portion and head is the rest on proper urls
-        test_vectors = {
-            '': '//',
-            '..': IndexError,
-            '/.//..': IndexError,
-            '/': '//',
-            '//': '//',
-            '/\\': '//\\',
-            '/.//': '//',
-            'cgi-bin/file1.py': '/cgi-bin/file1.py',
-            '/cgi-bin/file1.py': '/cgi-bin/file1.py',
-            'a': '//a',
-            '/a': '//a',
-            '//a': '//a',
-            './a': '//a',
-            './C:/': '/C:/',
-            '/a/b': '/a/b',
-            '/a/b/': '/a/b/',
-            '/a/b/.': '/a/b/',
-            '/a/b/c/..': '/a/b/',
-            '/a/b/c/../d': '/a/b/d',
-            '/a/b/c/../d/e/../f': '/a/b/d/f',
-            '/a/b/c/../d/e/../../f': '/a/b/f',
-            '/a/b/c/../d/e/.././././..//f': '/a/b/f',
-            '../a/b/c/../d/e/.././././..//f': IndexError,
-            '/a/b/c/../d/e/../../../f': '/a/f',
-            '/a/b/c/../d/e/../../../../f': '//f',
-            '/a/b/c/../d/e/../../../../../f': IndexError,
-            '/a/b/c/../d/e/../../../../f/..': '//',
-            '/a/b/c/../d/e/../../../../f/../.': '//',
-        }
-        for path, expected in test_vectors.iteritems():
-            if isinstance(expected, type) and issubclass(expected, Exception):
-                self.assertRaises(expected,
-                                  CGIHTTPServer._url_collapse_path, path)
-            else:
-                actual = CGIHTTPServer._url_collapse_path(path)
-                self.assertEqual(expected, actual,
-                                 msg='path = %r\nGot:    %r\nWanted: %r' %
-                                 (path, actual, expected))
-
-    @unittest.skipIf(test_support.is_jython, "FIXME: not working on Jython")
-    def test_headers_and_content(self):
-        res = self.request('/cgi-bin/file1.py')
-        self.assertEqual(('Hello World\n', 'text/html', 200),
-            (res.read(), res.getheader('Content-type'), res.status))
-
-    @unittest.skipIf(test_support.is_jython, "FIXME: not working on Jython")
-    def test_post(self):
-        params = urllib.urlencode({'spam' : 1, 'eggs' : 'python', 'bacon' : 123456})
-        headers = {'Content-type' : 'application/x-www-form-urlencoded'}
-        res = self.request('/cgi-bin/file2.py', 'POST', params, headers)
-
-        self.assertEqual(res.read(), '1, python, 123456\n')
-
-    def test_invaliduri(self):
-        res = self.request('/cgi-bin/invalid')
-        res.read()
-        self.assertEqual(res.status, 404)
-
-    @unittest.skipIf(test_support.is_jython, "FIXME: not working on Jython")
-    def test_authorization(self):
-        headers = {'Authorization' : 'Basic %s' %
-                   base64.b64encode('username:pass')}
-        res = self.request('/cgi-bin/file1.py', 'GET', headers=headers)
-        self.assertEqual(('Hello World\n', 'text/html', 200),
-                (res.read(), res.getheader('Content-type'), res.status))
-
-    @unittest.skipIf(test_support.is_jython, "FIXME: not working on Jython")
-    def test_no_leading_slash(self):
-        # http://bugs.python.org/issue2254
-        res = self.request('cgi-bin/file1.py')
-        self.assertEqual(('Hello World\n', 'text/html', 200),
-             (res.read(), res.getheader('Content-type'), res.status))
-
-    @unittest.skipIf(test_support.is_jython, "FIXME: not working on Jython")
-    def test_os_environ_is_not_altered(self):
-        signature = "Test CGI Server"
-        os.environ['SERVER_SOFTWARE'] = signature
-        res = self.request('/cgi-bin/file1.py')
-        self.assertEqual((b'Hello World\n', 'text/html', 200),
-                (res.read(), res.getheader('Content-type'), res.status))
-        self.assertEqual(os.environ['SERVER_SOFTWARE'], signature)
-
-
-class SimpleHTTPRequestHandlerTestCase(unittest.TestCase):
-    """ Test url parsing """
-    def setUp(self):
-        self.translated = os.getcwd()
-        self.translated = os.path.join(self.translated, 'filename')
-        self.handler = SocketlessRequestHandler()
-
-    def test_query_arguments(self):
-        path = self.handler.translate_path('/filename')
-        self.assertEqual(path, self.translated)
-        path = self.handler.translate_path('/filename?foo=bar')
-        self.assertEqual(path, self.translated)
-        path = self.handler.translate_path('/filename?a=b&spam=eggs#zot')
-        self.assertEqual(path, self.translated)
-
-    def test_start_with_double_slash(self):
-        path = self.handler.translate_path('//filename')
-        self.assertEqual(path, self.translated)
-        path = self.handler.translate_path('//filename?foo=bar')
-        self.assertEqual(path, self.translated)
-
-
-def test_main(verbose=None):
-    try:
-        cwd = os.getcwd()
-        test_support.run_unittest(BaseHTTPRequestHandlerTestCase,
-                                  SimpleHTTPRequestHandlerTestCase,
-                                  BaseHTTPServerTestCase,
-                                  SimpleHTTPServerTestCase,
-                                  CGIHTTPServerTestCase
-                                 )
-    finally:
-        os.chdir(cwd)
-
-if __name__ == '__main__':
-    test_main()
diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py
--- a/Lib/test/test_logging.py
+++ b/Lib/test/test_logging.py
@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 #
-# Copyright 2001-2010 by Vinay Sajip. All Rights Reserved.
+# Copyright 2001-2012 by Vinay Sajip. All Rights Reserved.
 #
 # Permission to use, copy, modify, and distribute this software and its
 # documentation for any purpose and without fee is hereby granted,
@@ -18,7 +18,7 @@
 
 """Test harness for the logging module. Run all tests.
 
-Copyright (C) 2001-2010 Vinay Sajip. All Rights Reserved.
+Copyright (C) 2001-2012 Vinay Sajip. All Rights Reserved.
 """
 
 import logging
@@ -31,6 +31,7 @@
 import gc
 import json
 import os
+import random
 import re
 import select
 import socket
@@ -40,6 +41,7 @@
 import tempfile
 from test.test_support import captured_stdout, run_with_locale, run_unittest
 import textwrap
+import time
 import unittest
 import warnings
 import weakref
@@ -272,6 +274,8 @@
             ('INF.BADPARENT', 'INFO', '4'),
         ])
 
+    def test_invalid_name(self):
+        self.assertRaises(TypeError, logging.getLogger, any)
 
 class BasicFilterTest(BaseTest):
 
@@ -562,6 +566,38 @@
     datefmt=
     """
 
+    # config1a moves the handler to the root.
+    config1a = """
+    [loggers]
+    keys=root,parser
+
+    [handlers]
+    keys=hand1
+
+    [formatters]
+    keys=form1
+
+    [logger_root]
+    level=WARNING
+    handlers=hand1
+
+    [logger_parser]
+    level=DEBUG
+    handlers=
+    propagate=1
+    qualname=compiler.parser
+
+    [handler_hand1]
+    class=StreamHandler
+    level=NOTSET
+    formatter=form1
+    args=(sys.stdout,)
+
+    [formatter_form1]
+    format=%(levelname)s ++ %(message)s
+    datefmt=
+    """
+
     # config2 has a subtle configuration error that should be reported
     config2 = config1.replace("sys.stdout", "sys.stbout")
 
@@ -640,6 +676,44 @@
     datefmt=
     """
 
+    # config7 adds a compiler logger.
+    config7 = """
+    [loggers]
+    keys=root,parser,compiler
+
+    [handlers]
+    keys=hand1
+
+    [formatters]
+    keys=form1
+
+    [logger_root]
+    level=WARNING
+    handlers=hand1
+
+    [logger_compiler]
+    level=DEBUG
+    handlers=
+    propagate=1
+    qualname=compiler
+
+    [logger_parser]
+    level=DEBUG
+    handlers=
+    propagate=1
+    qualname=compiler.parser
+
+    [handler_hand1]
+    class=StreamHandler
+    level=NOTSET
+    formatter=form1
+    args=(sys.stdout,)
+
+    [formatter_form1]
+    format=%(levelname)s ++ %(message)s
+    datefmt=
+    """
+
     def apply_config(self, conf):
         file = cStringIO.StringIO(textwrap.dedent(conf))
         logging.config.fileConfig(file)
@@ -703,6 +777,49 @@
     def test_config6_ok(self):
         self.test_config1_ok(config=self.config6)
 
+    def test_config7_ok(self):
+        with captured_stdout() as output:
+            self.apply_config(self.config1a)
+            logger = logging.getLogger("compiler.parser")
+            # See issue #11424. compiler-hyphenated sorts
+            # between compiler and compiler.xyz and this
+            # was preventing compiler.xyz from being included
+            # in the child loggers of compiler because of an
+            # overzealous loop termination condition.
+            hyphenated = logging.getLogger('compiler-hyphenated')
+            # All will output a message
+            logger.info(self.next_message())
+            logger.error(self.next_message())
+            hyphenated.critical(self.next_message())
+            self.assert_log_lines([
+                ('INFO', '1'),
+                ('ERROR', '2'),
+                ('CRITICAL', '3'),
+            ], stream=output)
+            # Original logger output is empty.
+            self.assert_log_lines([])
+        with captured_stdout() as output:
+            self.apply_config(self.config7)
+            logger = logging.getLogger("compiler.parser")
+            self.assertFalse(logger.disabled)
+            # Both will output a message
+            logger.info(self.next_message())
+            logger.error(self.next_message())
+            logger = logging.getLogger("compiler.lexer")
+            # Both will output a message
+            logger.info(self.next_message())
+            logger.error(self.next_message())
+            # Will not appear
+            hyphenated.critical(self.next_message())
+            self.assert_log_lines([
+                ('INFO', '4'),
+                ('ERROR', '5'),
+                ('INFO', '6'),
+                ('ERROR', '7'),
+            ], stream=output)
+            # Original logger output is empty.
+            self.assert_log_lines([])
+
 class LogRecordStreamHandler(StreamRequestHandler):
 
     """Handler for a streaming logging request. It saves the log message in the
@@ -755,14 +872,6 @@
         self.finished = threading.Event()
 
     def serve_until_stopped(self):
-        if sys.platform.startswith('java'):
-            # XXX: There's a problem using cpython_compatibile_select
-            # here: it seems to be due to the fact that
-            # cpython_compatible_select switches blocking mode on while
-            # a separate thread is reading from the same socket, causing
-            # a read of 0 in LogRecordStreamHandler.handle (which
-            # deadlocks this test)
-            self.socket.setblocking(0)
         while not self.abort:
             rd, wr, ex = select.select([self.socket.fileno()], [], [],
                                        self.timeout)
@@ -1766,6 +1875,47 @@
         self.assertTrue(c2 is c3)
 
 
+class HandlerTest(BaseTest):
+
+    @unittest.skipIf(os.name in ('java', 'nt'), 'WatchedFileHandler not appropriate for Jython or Windows.')
+    @unittest.skipUnless(threading, 'Threading required for this test.')
+    def test_race(self):
+        # Issue #14632 refers.
+        def remove_loop(fname, tries):
+            for _ in range(tries):
+                try:
+                    os.unlink(fname)
+                except OSError:
+                    pass
+                time.sleep(0.004 * random.randint(0, 4))
+
+        del_count = 500
+        log_count = 500
+
+        for delay in (False, True):
+            fd, fn = tempfile.mkstemp('.log', 'test_logging-3-')
+            os.close(fd)
+            remover = threading.Thread(target=remove_loop, args=(fn, del_count))
+            remover.daemon = True
+            remover.start()
+            h = logging.handlers.WatchedFileHandler(fn, delay=delay)
+            f = logging.Formatter('%(asctime)s: %(levelname)s: %(message)s')
+            h.setFormatter(f)
+            try:
+                for _ in range(log_count):
+                    time.sleep(0.005)
+                    r = logging.makeLogRecord({'msg': 'testing' })
+                    h.handle(r)
+            finally:
+                remover.join()
+                try:
+                    h.close()
+                except ValueError:
+                    pass
+                if os.path.exists(fn):
+                    os.unlink(fn)
+
+
 # Set the locale to the platform-dependent default.  I have no idea
 # why the test does this, but in any case we save the current locale
 # first and restore it at the end.
@@ -1775,7 +1925,7 @@
                  CustomLevelsAndFiltersTest, MemoryHandlerTest,
                  ConfigFileTest, SocketHandlerTest, MemoryTest,
                  EncodingTest, WarningsTest, ConfigDictTest, ManagerTest,
-                 ChildLoggerTest)
+                 ChildLoggerTest, HandlerTest)
 
 if __name__ == "__main__":
     test_main()
diff --git a/Lib/test/test_mhlib.py b/Lib/test/test_mhlib.py
--- a/Lib/test/test_mhlib.py
+++ b/Lib/test/test_mhlib.py
@@ -7,10 +7,10 @@
 ###      mhlib.  It should.
 
 import unittest
-from test.test_support import is_jython, run_unittest, TESTFN, TestSkipped
+from test.test_support import is_jython, run_unittest, TESTFN, import_module
 import os, StringIO
 import sys
-import mhlib
+mhlib = import_module('mhlib', deprecated=True)
 
 if (sys.platform.startswith("win") or sys.platform=="riscos" or
       sys.platform.startswith("atheos") or (is_jython and os._name != 'posix')):
@@ -21,8 +21,8 @@
     # link counts, and that causes test_listfolders() here to get back
     # an empty list from its call of listallfolders().
     # The other tests here pass on Windows.
-    raise TestSkipped("skipped on %s -- " % sys.platform +
-                      "too many Unix assumptions")
+    raise unittest.SkipTest("skipped on %s -- " % sys.platform +
+                            "too many Unix assumptions")
 
 _mhroot = TESTFN+"_MH"
 _mhpath = os.path.join(_mhroot, "MH")
@@ -148,7 +148,7 @@
         writeCurMessage('inbox', 2)
         mh = getMH()
 
-        eq = self.assertEquals
+        eq = self.assertEqual
         eq(mh.getprofile('Editor'), 'emacs')
         eq(mh.getprofile('not-set'), None)
         eq(mh.getpath(), os.path.abspath(_mhpath))
@@ -171,37 +171,34 @@
 
     def test_listfolders(self):
         mh = getMH()
-        eq = self.assertEquals
+        eq = self.assertEqual
 
         folders = mh.listfolders()
         folders.sort()
         eq(folders, ['deep', 'inbox', 'wide'])
 
-        #link counts from os.stat always return 0 in jython, which causes
-        #lisallfolders and listsubfolders to return empty lists.
-        if not sys.platform.startswith("java"):
-            folders = mh.listallfolders()
-            folders.sort()
-            tfolders = map(normF, ['deep', 'deep/f1', 'deep/f2', 'deep/f2/f3',
+        folders = mh.listallfolders()
+        folders.sort()
+        tfolders = map(normF, ['deep', 'deep/f1', 'deep/f2', 'deep/f2/f3',
                                 'inbox', 'wide'])
-            tfolders.sort()
-            eq(folders, tfolders)
+        tfolders.sort()
+        eq(folders, tfolders)
 
-            folders = mh.listsubfolders('deep')
-            folders.sort()
-            eq(folders, map(normF, ['deep/f1', 'deep/f2']))
+        folders = mh.listsubfolders('deep')
+        folders.sort()
+        eq(folders, map(normF, ['deep/f1', 'deep/f2']))
 
-            folders = mh.listallsubfolders('deep')
-            folders.sort()
-            eq(folders, map(normF, ['deep/f1', 'deep/f2', 'deep/f2/f3']))
-            eq(mh.listsubfolders(normF('deep/f2')), [normF('deep/f2/f3')])
+        folders = mh.listallsubfolders('deep')
+        folders.sort()
+        eq(folders, map(normF, ['deep/f1', 'deep/f2', 'deep/f2/f3']))
+        eq(mh.listsubfolders(normF('deep/f2')), [normF('deep/f2/f3')])
 
-            eq(mh.listsubfolders('inbox'), [])
-            eq(mh.listallsubfolders('inbox'), [])
+        eq(mh.listsubfolders('inbox'), [])
+        eq(mh.listallsubfolders('inbox'), [])
 
     def test_sequence(self):
         mh = getMH()
-        eq = self.assertEquals
+        eq = self.assertEqual
         writeCurMessage('wide', 55)
 
         f = mh.openfolder('wide')
@@ -256,12 +253,12 @@
 
     def test_modify(self):
         mh = getMH()
-        eq = self.assertEquals
+        eq = self.assertEqual
 
         mh.makefolder("dummy1")
-        self.assert_("dummy1" in mh.listfolders())
+        self.assertIn("dummy1", mh.listfolders())
         path = os.path.join(_mhpath, "dummy1")
-        self.assert_(os.path.exists(path))
+        self.assertTrue(os.path.exists(path))
 
         f = mh.openfolder('dummy1')
         def create(n):
@@ -313,12 +310,12 @@
 
         mh.deletefolder('dummy1')
         mh.deletefolder('dummy2')
-        self.assert_('dummy1' not in mh.listfolders())
-        self.assert_(not os.path.exists(path))
+        self.assertNotIn('dummy1', mh.listfolders())
+        self.assertTrue(not os.path.exists(path))
 
     def test_read(self):
         mh = getMH()
-        eq = self.assertEquals
+        eq = self.assertEqual
 
         f = mh.openfolder('inbox')
         msg = f.openmessage(1)
diff --git a/Lib/test/test_select.py b/Lib/test/test_select.py
--- a/Lib/test/test_select.py
+++ b/Lib/test/test_select.py
@@ -134,16 +134,11 @@
         else:
             self.fail("Unregistering socket that is not registered should have raised KeyError")
 
-#
-# using the test_socket thread based server/client management, for convenience.
-#
-class ThreadedPollClientSocket(test_socket.ThreadedTCPSocketTest):
 
-    HOST = HOST
-    PORT = PORT
+class ThreadedPollClientSocket(test_socket.SocketConnectedTest):
 
     def testSocketRegisteredBeforeConnected(self):
-        self.cli_conn = self.serv.accept()
+        pass
 
     def _testSocketRegisteredBeforeConnected(self):
         timeout = 1000 # milliseconds
@@ -152,7 +147,7 @@
         poll_object.register(self.cli, select.POLLOUT)
         result_list = poll_object.poll(timeout)
         result_sockets = [r[0] for r in result_list]
-        self.failIf(self.cli in result_sockets, "Unconnected client socket should not have been selectable")
+        self.failUnless(self.cli in result_sockets, "Unconnected client socket should be selectable")
         # Now connect the socket, but DO NOT register it again
         self.cli.setblocking(0)
         self.cli.connect( (self.HOST, self.PORT) )
@@ -161,25 +156,8 @@
         result_sockets = [r[0] for r in result_list]
         self.failUnless(self.cli in result_sockets, "Connected client socket should have been selectable")
 
-    def testSocketMustBeNonBlocking(self):
-        self.cli_conn = self.serv.accept()
-
-    def _testSocketMustBeNonBlocking(self):
-        self.cli.setblocking(1)
-        self.cli.connect( (self.HOST, self.PORT) )
-        timeout = 1000 # milliseconds
-        poll_object = select.poll()
-        try:
-            poll_object.register(self.cli)
-        except select.error, se:
-            self.failUnlessEqual(se[0], errno.ESOCKISBLOCKING)
-        except Exception, x:
-            self.fail("Registering blocking socket should have raised select.error, not %s" % str(x))
-        else:
-            self.fail("Registering blocking socket should have raised select.error")
-
     def testSelectOnSocketFileno(self):
-        self.cli_conn = self.serv.accept()
+        pass
 
     def _testSelectOnSocketFileno(self):
         self.cli.setblocking(0)
@@ -190,42 +168,13 @@
         except Exception, x:
             self.fail("Selecting on socket.fileno() should not have raised exception: %s" % str(x))
 
-class TestPipes(unittest.TestCase):
-
-    verbose = 1
-
-    def test(self):
-        import sys
-        from test.test_support import verbose
-        if sys.platform[:3] in ('win', 'mac', 'os2', 'riscos'):
-            if verbose:
-                print "Can't test select easily on", sys.platform
-            return
-        cmd = 'for i in 0 1 2 3 4 5 6 7 8 9; do echo testing...; sleep 1; done'
-        p = os.popen(cmd, 'r')
-        for tout in (0, 1, 2, 4, 8, 16) + (None,)*10:
-            if verbose:
-                print 'timeout =', tout
-            rfd, wfd, xfd = select.select([p], [], [], tout)
-            if (rfd, wfd, xfd) == ([], [], []):
-                continue
-            if (rfd, wfd, xfd) == ([p], [], []):
-                line = p.readline()
-                if verbose:
-                    print repr(line)
-                if not line:
-                    if verbose:
-                        print 'EOF'
-                    break
-                continue
-            self.fail('Unexpected return values from select(): %s' % str(rfd, wfd, xfd))
-        p.close()
 
 def test_main():
 
-    if test_support.is_jython:
-        del TestPipes.test
-    test_support.run_unittest(__name__)
+    tests = [TestSelectInvalidParameters, TestSelectClientSocket, TestPollClientSocket, ThreadedPollClientSocket]
+    suites = [unittest.makeSuite(klass, 'test') for klass in tests]
+    test_support._run_suite(unittest.TestSuite(suites))
+
 
 if __name__ == "__main__":
     test_main()
diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -7,6 +7,7 @@
 import jarray
 import Queue
 import platform
+import pprint
 import select
 import socket
 import struct
@@ -15,6 +16,7 @@
 import thread, threading
 from weakref import proxy
 from StringIO import StringIO
+from _socket import _check_threadpool_for_pending_threads
 
 PORT = 50100
 HOST = 'localhost'
@@ -124,15 +126,26 @@
         if not self.server_ready.isSet():
             self.server_ready.set()
         self.client_ready.wait()
-
+        
     def _tearDown(self):
-        self.done.wait()
+        self.done.wait()   # wait for the client to exit
         self.__tearDown()
 
+        msg = None
         if not self.queue.empty():
             msg = self.queue.get()
-            self.fail(msg)
 
+        # Wait up to one second for there not to be pending threads
+        for i in xrange(10):
+            pending_threads = _check_threadpool_for_pending_threads()
+            if len(pending_threads) == 0:
+                break
+            time.sleep(0.1)
+
+        if pending_threads or msg:
+            self.fail("msg={} Pending threads in Netty pool={}".format(msg, pprint.pformat(pending_threads)))
+
+            
     def clientRun(self, test_func):
         self.server_ready.wait()
         self.client_ready.set()
@@ -150,10 +163,6 @@
 
     def clientTearDown(self):
         self.done.set()
-        if sys.platform[:4] != 'java':
-            # This causes the whole process to exit on jython
-            # Probably related to problems with daemon status of threads
-            thread.exit()
 
 class ThreadedTCPSocketTest(SocketTCPTest, ThreadableTest):
 
@@ -275,23 +284,6 @@
         socket.SOL_SOCKET
         socket.SO_REUSEADDR
 
-    def testConstantToNameMapping(self):
-        # Testing for mission critical constants
-        for name, expected_name_starts in [
-            ('IPPROTO_ICMP',  ['IPPROTO_']),
-            ('IPPROTO_TCP',   ['IPPROTO_']),
-            ('IPPROTO_UDP',   ['IPPROTO_']),
-            ('SO_BROADCAST',  ['SO_', 'TCP_']),
-            ('SO_KEEPALIVE',  ['SO_', 'TCP_']),
-            ('SO_ACCEPTCONN', ['SO_', 'TCP_']),
-            ('SO_DEBUG',      ['SO_', 'TCP_']),
-            ('SOCK_DGRAM',    ['SOCK_']),
-            ('SOCK_RAW',      ['SOCK_']),
-            ('SOL_SOCKET',    ['SOL_', 'IPPROTO_']),
-            ('TCP_NODELAY',   ['SO_', 'TCP_']),
-            ]:
-            self.failUnlessEqual(socket._constant_to_name(getattr(socket, name), expected_name_starts), name)
-
     def testHostnameRes(self):
         # Testing hostname resolution mechanisms
         hostname = socket.gethostname()
@@ -715,7 +707,7 @@
         for expected_value in values:
             sock.setsockopt(level, option, expected_value)
             retrieved_value = sock.getsockopt(level, option)
-            msg = "Retrieved option(%s, %s) value %s != %s(value set)" % (level, option, retrieved_value, expected_value)
+            msg = "TCP Retrieved option(%s, %s) value %s != %s(value set)" % (level, option, retrieved_value, expected_value)
             if option == socket.SO_RCVBUF:
                 self.assert_(retrieved_value >= expected_value, msg)
             else:
@@ -729,7 +721,7 @@
             sock.bind( (HOST, PORT) )
             retrieved_option_value = sock.getsockopt(level, option)
             self.failUnlessEqual(retrieved_option_value, values[-1], \
-                 "Option value '(%s, %s)'='%s' did not propagate to implementation socket: got %s" % (level, option, values[-1], retrieved_option_value) )
+                 "UDP Option value '(%s, %s)'='%s' did not propagate to implementation socket: got %s" % (level, option, values[-1], retrieved_option_value) )
             self._testSetAndGetOption(sock, level, option, values)
         finally:
             sock.close()
@@ -747,10 +739,10 @@
             self._testSetAndGetOption(sock, level, option, values)
             # now connect the socket i.e. cause the implementation socket to be created
             # First bind, so that the SO_REUSEADDR setting propagates
-            sock.bind( (HOST, PORT+1) )
+            #sock.bind( (HOST, PORT+1) )
             sock.connect( (HOST, PORT) )
             retrieved_option_value = sock.getsockopt(level, option)
-            msg = "Option value '%s'='%s' did not propagate to implementation socket: got %s" % (option, values[-1], retrieved_option_value)
+            msg = "TCP client option value '%s'='%s' did not propagate to implementation socket: got %s" % (option, values[-1], retrieved_option_value)
             if option in (socket.SO_RCVBUF, socket.SO_SNDBUF):
                 # NOTE: there's no guarantee that bufsize will be the
                 # exact setsockopt value, particularly after
@@ -765,6 +757,7 @@
             server_sock.close()
             if sock:
                 sock.close()
+            pass
 
     def _testTCPClientInheritedOption(self, level, option, values):
         cli_sock = accepted_sock = None
@@ -780,7 +773,7 @@
             cli_sock.connect( (HOST, PORT) )
             accepted_sock = server_sock.accept()[0]
             retrieved_option_value = accepted_sock.getsockopt(level, option)
-            msg = "Option value '(%s,%s)'='%s' did not propagate to accepted socket: got %s" % (level, option, values[-1], retrieved_option_value)
+            msg = "TCP client inherited option value '(%s,%s)'='%s' did not propagate to accepted socket: got %s" % (level, option, values[-1], retrieved_option_value)
             if option == socket.SO_RCVBUF:
                 # NOTE: see similar bsd/solaris workaround above
                 self.assert_(retrieved_option_value >= values[-1], msg)
@@ -789,6 +782,7 @@
             self._testSetAndGetOption(accepted_sock, level, option, values)
         finally:
             server_sock.close()
+            time.sleep(1)
             if cli_sock:
                 cli_sock.close()
             if accepted_sock:
@@ -803,7 +797,7 @@
             sock.bind( (HOST, PORT) )
             sock.listen(50)
             retrieved_option_value = sock.getsockopt(level, option)
-            msg = "Option value '(%s,%s)'='%s' did not propagate to implementation socket. Got %s" % (level, option, values[-1], retrieved_option_value)
+            msg = "TCP server option value '(%s,%s)'='%s' did not propagate to implementation socket. Got %s" % (level, option, values[-1], retrieved_option_value)
             if option == socket.SO_RCVBUF:
                 # NOTE: see similar bsd/solaris workaround above
                 self.assert_(retrieved_option_value >= values[-1], msg)
@@ -849,19 +843,20 @@
         self._testOption(socket.SOL_SOCKET, socket.SO_KEEPALIVE, [0, 1])
         self._testInheritedOption(socket.SOL_SOCKET, socket.SO_KEEPALIVE, [0, 1])
 
-    def testSO_LINGER(self):
-        self.test_tcp_client = 1
-        self.test_tcp_server = 1
-        off = struct.pack('ii', 0, 0)
-        on_2_seconds = struct.pack('ii', 1, 2)
-        self._testOption(socket.SOL_SOCKET, socket.SO_LINGER, [off, on_2_seconds])
-        self._testInheritedOption(socket.SOL_SOCKET, socket.SO_LINGER, [off, on_2_seconds])
+    # def testSO_LINGER(self):
+    #     self.test_tcp_client = 1
+    #     self.test_tcp_server = 1
+    #     off = struct.pack('ii', 0, 0)
+    #     on_2_seconds = struct.pack('ii', 1, 2)
+    #     self._testOption(socket.SOL_SOCKET, socket.SO_LINGER, [off, on_2_seconds])
+    #     self._testInheritedOption(socket.SOL_SOCKET, socket.SO_LINGER, [off, on_2_seconds])
 
-    def testSO_OOBINLINE(self):
-        self.test_tcp_client = 1
-        self.test_tcp_server = 1
-        self._testOption(socket.SOL_SOCKET, socket.SO_OOBINLINE, [0, 1])
-        self._testInheritedOption(socket.SOL_SOCKET, socket.SO_OOBINLINE, [0, 1])
+    # # WILL NOT FIX
+    # def testSO_OOBINLINE(self):
+    #     self.test_tcp_client = 1
+    #     self.test_tcp_server = 1
+    #     self._testOption(socket.SOL_SOCKET, socket.SO_OOBINLINE, [0, 1])
+    #     self._testInheritedOption(socket.SOL_SOCKET, socket.SO_OOBINLINE, [0, 1])
 
     def testSO_RCVBUF(self):
         self.test_udp        = 1
@@ -889,8 +884,7 @@
         self.test_tcp_client = 1
         self.test_tcp_server = 1
         self._testOption(socket.SOL_SOCKET, socket.SO_TIMEOUT, [0, 1, 1000])
-        # We don't test inheritance here because both server and client sockets have SO_TIMEOUT
-        # but it doesn't inherit.
+        self._testInheritedOption(socket.SOL_SOCKET, socket.SO_TIMEOUT, [0, 1, 1000])
 
     def testTCP_NODELAY(self):
         self.test_tcp_client = 1
@@ -921,18 +915,26 @@
                     self.fail("getsocket(SO_ACCEPTCONN) on valid socket type should not have raised exception: %s" % (str(se)))
 
     def testSO_ERROR(self):
-        for socket_type in [socket.SOCK_STREAM, socket.SOCK_DGRAM]:
-            s = socket.socket(socket.AF_INET, socket_type)
-            self.failUnlessEqual(s.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR), 0)
-            try:
-                # Now cause an error
-                s.connect(("localhost", 100000))
-                self.fail("Operation '%s' that should have failed to generate SO_ERROR did not" % operation)
-            except socket.error, se:
-                so_error = s.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR)
-                self.failUnlessEqual(so_error, se[0])
-            # Now retrieve the option again - it should be zero
-            self.failUnlessEqual(s.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR), 0)
+        good = bad = None
+
+        try:
+            good = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            good.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+            good.bind((HOST, PORT))
+            good.listen(1)
+            bad = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            bad.bind((HOST, PORT))
+            bad.listen(1)
+            self.fail("Listen operation against same port did not generate an expected error")
+        except socket.error, se:
+            self.failUnlessEqual(bad.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR), se[0])
+            # try again, should now be reset
+            self.failUnlessEqual(bad.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR), 0)
+        finally:
+            if good is not None:
+                good.close()
+            if bad is not None:
+                bad.close()
 
     def testSO_TYPE(self):
         for socket_type in [socket.SOCK_STREAM, socket.SOCK_DGRAM]:
@@ -1041,7 +1043,7 @@
     def testFromFd(self):
         # Testing fromfd()
         if not hasattr(socket, "fromfd"):
-            return # On Windows, this doesn't exist
+            return # On Windows or Jython, this doesn't exist
         fd = self.cli_conn.fileno()
         sock = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM)
         msg = sock.recv(1024)
@@ -1081,6 +1083,7 @@
         dup_conn = self.cli_conn.dup()
         msg = dup_conn.recv(len('and ' + MSG))
         self.assertEqual(msg, 'and ' +  MSG)
+        dup_conn.close()  # need to ensure all sockets are closed
 
     def _testDup(self):
         self.serv_conn.send(MSG)
@@ -1127,7 +1130,7 @@
     def testSendtoAndRecvTimeoutMode(self):
         # Need to test again in timeout mode, which follows
         # a different code path
-        self.serv.settimeout(10)
+        self.serv.settimeout(1)
         msg = self.serv.recv(len(MSG))
         self.assertEqual(msg, MSG)
 
@@ -1147,14 +1150,15 @@
     def testSendAndRecvTimeoutMode(self):
         # Need to test again in timeout mode, which follows
         # a different code path
-        self.serv.settimeout(10)
+        self.serv.settimeout(5)
         # Testing send() and recv() over connect'ed UDP
         msg = self.serv.recv(len(MSG))
         self.assertEqual(msg, MSG)
 
     def _testSendAndRecvTimeoutMode(self):
         self.cli.connect( (self.HOST, self.PORT) )
-        self.cli.settimeout(10)
+        self.cli.settimeout(5)
+        time.sleep(1)
         self.cli.send(MSG, 0)
 
     def testRecvFrom(self):
@@ -1168,12 +1172,12 @@
     def testRecvFromTimeoutMode(self):
         # Need to test again in timeout mode, which follows
         # a different code path
-        self.serv.settimeout(10)
+        self.serv.settimeout(1)
         msg, addr = self.serv.recvfrom(len(MSG))
         self.assertEqual(msg, MSG)
 
     def _testRecvFromTimeoutMode(self):
-        self.cli.settimeout(10)
+        self.cli.settimeout(1)
         self.cli.sendto(MSG, 0, (self.HOST, self.PORT))
 
     def testSendtoEightBitSafe(self):
@@ -1242,13 +1246,6 @@
         end = time.time()
         self.assert_((end - start) < 1.0, "Error setting non-blocking mode.")
 
-    def testGetBlocking(self):
-        # Testing whether set blocking works
-        self.serv.setblocking(0)
-        self.failUnless(not self.serv.getblocking(), "Getblocking return true instead of false")
-        self.serv.setblocking(1)
-        self.failUnless(self.serv.getblocking(), "Getblocking return false instead of true")
-
     def testAcceptNoConnection(self):
         # Testing non-blocking accept returns immediately when no connection
         self.serv.setblocking(0)
@@ -1267,21 +1264,22 @@
     def testAcceptConnection(self):
         # Testing non-blocking accept works when connection present
         self.serv.setblocking(0)
-        read, write, err = select.select([self.serv], [], [])
+
+        # this can potentially race with the client, so we need to loop
+        while True:
+            read, write, err = select.select([self.serv], [], [], 0.1)
+            if read or write or err:
+                break
         if self.serv in read:
             conn, addr = self.serv.accept()
+            conn.close()
         else:
             self.fail("Error trying to do accept after select: server socket was not in 'read'able list")
 
     def _testAcceptConnection(self):
         # Make a connection to the server
         self.cli.connect((self.HOST, self.PORT))
-
-    #
-    # AMAK: 20070311
-    # Introduced a new test for non-blocking connect
-    # Renamed old testConnect to testBlockingConnect
-    #
+        time.sleep(1)
 
     def testBlockingConnect(self):
         # Testing blocking connect
@@ -1300,21 +1298,20 @@
         # Testing non-blocking connect
         self.cli.setblocking(0)
         result = self.cli.connect_ex((self.HOST, self.PORT))
-        rfds, wfds, xfds = select.select([], [self.cli], [])
+        while True:
+            rfds, wfds, xfds = select.select([self.cli], [self.cli], [], 0.1)
+            if rfds or wfds or xfds:
+                break
         self.failUnless(self.cli in wfds)
         try:
             self.cli.send(MSG)
         except socket.error:
             self.fail("Sending on connected socket should not have raised socket.error")
 
-    #
-    # AMAK: 20070518
-    # Introduced a new test for connect with bind to specific local address
-    #
-
     def testConnectWithLocalBind(self):
         # Test blocking connect
         conn, addr = self.serv.accept()
+        conn.close()  # Closing the server socket does not close this client socket
 
     def _testConnectWithLocalBind(self):
         # Testing blocking connect with local bind
@@ -1342,18 +1339,28 @@
 
     def testRecvData(self):
         # Testing non-blocking recv
-        conn, addr = self.serv.accept()
-        conn.setblocking(0)
-        rfds, wfds, xfds = select.select([conn], [], [])
-        if conn in rfds:
-            msg = conn.recv(len(MSG))
-            self.assertEqual(msg, MSG)
-        else:
-            self.fail("Non-blocking socket with data should been in read list.")
+        conn, addr = self.serv.accept()  # server socket is blocking
+        conn.setblocking(0)              # but now the child socket is not
+
+        try:
+            # this can potentially race with the client, so we need to loop
+            while True:
+                rfds, wfds, xfds = select.select([conn], [], [], 0.1)
+                if rfds or wfds or xfds:
+                    break
+
+            if conn in rfds:
+                msg = conn.recv(len(MSG))
+                self.assertEqual(msg, MSG)
+            else:
+                self.fail("Non-blocking socket with data should been in read list.")
+        finally:
+            conn.close()
 
     def _testRecvData(self):
         self.cli.connect((self.HOST, self.PORT))
         self.cli.send(MSG)
+        #time.sleep(0.5)
 
     def testRecvNoData(self):
         # Testing non-blocking recv
@@ -1365,10 +1372,13 @@
             pass
         else:
             self.fail("Non-blocking recv of no data should have raised socket.error.")
+        finally:
+            conn.close()
 
     def _testRecvNoData(self):
         self.cli.connect((self.HOST, self.PORT))
-        time.sleep(0.1)
+        time.sleep(1)  # Without a sleep, we may not see the connect, because the channel will be closed
+
 
 class NonBlockingUDPTests(ThreadedUDPSocketTest): pass
 
@@ -1453,6 +1463,7 @@
             self.failUnlessEqual(se[0], errno.EBADF)
         else:
             self.fail("Original socket did not close")
+
         try:
             self.dup_conn1.send(MSG)
         except socket.error, se:
@@ -1541,7 +1552,7 @@
 
     def testClosedAttr(self):
         self.assert_(not self.serv_file.closed)
-
+        
     def _testClosedAttr(self):
         self.assert_(not self.cli_file.closed)
 
@@ -1627,7 +1638,7 @@
     def testConnectTimeout(self):
         cli = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         cli.settimeout(0.1)
-        host = '192.168.192.168'
+        host = '192.0.2.42'  # address in TEST-NET-1, guaranteed to not be routeable
         try:
             cli.connect((host, 5000))
         except socket.timeout, st:
@@ -1643,7 +1654,7 @@
         _saved_timeout = socket.getdefaulttimeout()
         socket.setdefaulttimeout(0.1)
         cli = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        host = '192.168.192.168'
+        host = '192.0.2.42'  # address in TEST-NET-1, guaranteed to not be routeable
         try:
             cli.connect((host, 5000))
         except socket.timeout, st:
@@ -1654,24 +1665,22 @@
             self.fail('''Client socket timeout should have raised
 socket.timeout.  This tries to connect to %s in the assumption that it isn't
 used, but if it is on your network this failure is bogus.''' % host)
-        socket.setdefaulttimeout(_saved_timeout)
+        finally:
+            socket.setdefaulttimeout(_saved_timeout)
 
     def testRecvTimeout(self):
         def raise_timeout(*args, **kwargs):
             cli_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-            cli_sock.connect( (HOST, PORT) )
+            cli_sock.connect( (self.HOST, self.PORT) )
             cli_sock.settimeout(1)
             cli_sock.recv(1024)
         self.failUnlessRaises(socket.timeout, raise_timeout,
                               "TCP socket recv failed to generate a timeout exception (TCP)")
 
-    # Disable this test, but leave it present for documentation purposes
-    # socket timeouts only work for read and accept, not for write
-    # http://java.sun.com/j2se/1.4.2/docs/api/java/net/SocketTimeoutException.html
     def estSendTimeout(self):
         def raise_timeout(*args, **kwargs):
             cli_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-            cli_sock.connect( (HOST, PORT) )
+            cli_sock.connect( (self.HOST, self.PORT) )
             # First fill the socket
             cli_sock.settimeout(1)
             sent = 0
@@ -1683,7 +1692,7 @@
 
     def testSwitchModes(self):
         cli_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        cli_sock.connect( (HOST, PORT) )
+        cli_sock.connect( (self.HOST, self.PORT) )
         # set non-blocking mode
         cli_sock.setblocking(0)
         # then set timeout mode
@@ -1695,11 +1704,6 @@
         else:
             pass
 
-#
-# AMAK: 20070307
-# Corrected the superclass of UDPTimeoutTest
-#
-
 class UDPTimeoutTest(SocketUDPTest):
 
     def testUDPTimeout(self):
@@ -1812,15 +1816,6 @@
                 self.failUnlessEqual(expected_canonname, canonname, "For hostname '%s' and flags %d, canonname '%s' != '%s'" % (host_param, flags, expected_canonname, canonname) )
                 self.failUnlessEqual(expected_sockaddr, sockaddr[0], "For hostname '%s' and flags %d, sockaddr '%s' != '%s'" % (host_param, flags, expected_sockaddr, sockaddr[0]) )
 
-    def testIPV4AddressesOnly(self):
-        socket._use_ipv4_addresses_only(True)
-        def doAddressTest(addrinfos):
-            for family, socktype, proto, canonname, sockaddr in addrinfos:
-                self.failIf(":" in sockaddr[0], "Incorrectly received IPv6 address '%s'" % (sockaddr[0]) )
-        doAddressTest(socket.getaddrinfo("localhost", 0, socket.AF_INET6, socket.SOCK_STREAM, 0, 0))
-        doAddressTest(socket.getaddrinfo("localhost", 0, socket.AF_UNSPEC, socket.SOCK_STREAM, 0, 0))
-        socket._use_ipv4_addresses_only(False)
-
     def testAddrTupleTypes(self):
         ipv4_address_tuple = socket.getaddrinfo("localhost", 80, socket.AF_INET, socket.SOCK_STREAM, 0, 0)[0][4]
         self.failUnlessEqual(ipv4_address_tuple[0], "127.0.0.1")
@@ -1966,21 +1961,28 @@
         for address, flags, expected in [
             ( ("127.0.0.1", 25),  0,                     "smtp" ),
             ( ("127.0.0.1", 25),  socket.NI_NUMERICSERV, 25     ),
-            ( ("127.0.0.1", 513), socket.NI_DGRAM,       "who"  ),
-            ( ("127.0.0.1", 513), 0,                     "login"),
+
+            # This portion of the test does not suceed on OS X;
+            # the above entries probably suffice
+            # ( ("127.0.0.1", 513), socket.NI_DGRAM,       "who"  ),
+            # ( ("127.0.0.1", 513), 0,                     "login"),
         ]:
             result = socket.getnameinfo(address, flags)
             self.failUnlessEqual(result[1], expected)
 
-    def testHost(self):
-        for address, flags, expected in [
-            ( ("www.python.org", 80),  0,                     "dinsdale.python.org"),
-            ( ("www.python.org", 80),  socket.NI_NUMERICHOST, "82.94.164.162"      ),
-            ( ("www.python.org", 80),  socket.NI_NAMEREQD,    "dinsdale.python.org"),
-            ( ("82.94.164.162",  80),  socket.NI_NAMEREQD,    "dinsdale.python.org"),
-        ]:
-            result = socket.getnameinfo(address, flags)
-            self.failUnlessEqual(result[0], expected)
+
+    # This test currently fails due to the recent changes (as of March 2014) at python.org:
+    # TBD perhaps there are well-known addresses that guarantee stable resolution
+
+    # def testHost(self):
+    #     for address, flags, expected in [
+    #         ( ("www.python.org", 80),  0,                     "dinsdale.python.org"),
+    #         ( ("www.python.org", 80),  socket.NI_NUMERICHOST, "82.94.164.162"      ),
+    #         ( ("www.python.org", 80),  socket.NI_NAMEREQD,    "dinsdale.python.org"),
+    #         ( ("82.94.164.162",  80),  socket.NI_NAMEREQD,    "dinsdale.python.org"),
+    #     ]:
+    #         result = socket.getnameinfo(address, flags)
+    #         self.failUnlessEqual(result[0], expected)
 
     def testNI_NAMEREQD(self):
         # This test may delay for some seconds
@@ -2069,7 +2071,7 @@
             ( socket.AF_INET,  0,                 socket.AI_PASSIVE, ("", 80),            [socket.INADDR_ANY]),
             ( socket.AF_INET6, 0,                 0,                 ("", 80),            ["localhost"]),
             ( socket.AF_INET6, 0,                 socket.AI_PASSIVE, ("", 80),            [socket.IN6ADDR_ANY_INIT, "0:0:0:0:0:0:0:0"]),
-            ( socket.AF_INET,  socket.SOCK_DGRAM, 0,                 ("<broadcast>", 80), [socket.INADDR_BROADCAST]),
+            ( socket.AF_INET,  socket.SOCK_DGRAM, 0,                 ("<broadcast>", 80), ["broadcasthost"]),
             ]:
             sockaddr = socket._get_jsockaddr(addr_tuple, family, sock_type, 0, flags)
             self.failUnless(sockaddr.hostName in expected, "_get_jsockaddr returned wrong hostname '%s' for special hostname '%s'(family=%d)" % (sockaddr.hostName, addr_tuple[0], family))
@@ -2152,16 +2154,6 @@
         else:
             self.fail("Send on unconnected socket raised exception")
 
-    def testSocketNotBound(self):
-        try:
-            result = self.s.recv(1024)
-        except socket.error, se:
-            self.failUnlessEqual(se[0], errno.ENOTCONN)
-        except Exception, x:
-            self.fail("Receive on unbound socket raised wrong exception: %s" % x)
-        else:
-            self.fail("Receive on unbound socket raised exception")
-
     def testClosedSocket(self):
         self.s.close()
         try:
@@ -2215,6 +2207,17 @@
         else:
             self.fail("Binding to already bound host/port should have raised exception")
 
+    def testSocketNotBound(self):
+        try:
+            result = self.s.recv(1024)
+        except socket.error, se:
+            self.failUnlessEqual(se[0], errno.ENOTCONN)
+        except Exception, x:
+            self.fail("Receive on unbound socket raised wrong exception: %s" % x)
+        else:
+            self.fail("Receive on unbound socket raised exception")
+
+
 class TestJythonUDPExceptions(TestJythonExceptionsShared, unittest.TestCase):
 
     def setUp(self):
@@ -2358,7 +2361,7 @@
         try:
             self.s.getsockname()
         except socket.error, se:
-            if se[0] == errno.EINVAL:
+            if se[0] == errno.ENOTCONN:
                 return
         self.fail("getsockname() on unconnected socket should have raised socket.error")
 
@@ -2451,6 +2454,9 @@
             try:
                 self.s.getpeername()
             except socket.error, se:
+                # FIXME Apparently Netty's doesn't set remoteAddress, even if connected, for datagram channels
+                # so we may have to shadow
+                print "\n\n\ngetpeername()", self.s._sock.channel
                 self.fail("getpeername() on connected UDP socket should not have raised socket.error")
             self.failUnlessEqual(self.s.getpeername(), self._udp_peer.getsockname())
         finally:
@@ -2491,12 +2497,15 @@
         TestGetSockAndPeerNameTCPServer, 
         TestGetSockAndPeerNameUDP,
     ]
+
     if hasattr(socket, "socketpair"):
         tests.append(BasicSocketPairTest)
+
     if sys.platform[:4] == 'java':
         tests.append(TestJythonTCPExceptions)
         tests.append(TestJythonUDPExceptions)
         tests.append(TestJython_get_jsockaddr)
+
     # TODO: Broadcast requires permission, and is blocked by some firewalls
     # Need some way to discover the network setup on the test machine
     if False:
diff --git a/Lib/test/test_socket_ssl.py b/Lib/test/test_socket_ssl.py
deleted file mode 100644
--- a/Lib/test/test_socket_ssl.py
+++ /dev/null
@@ -1,80 +0,0 @@
-# Test just the SSL support in the socket module, in a moderately bogus way.
-
-from test import test_support
-import socket
-import time
-
-# Optionally test SSL support.  This requires the 'network' resource as given
-# on the regrtest command line.
-skip_expected = not (test_support.is_resource_enabled('network') and
-                     hasattr(socket, "ssl"))
-
-def test_basic():
-    test_support.requires('network')
-
-    import urllib
-
-    socket.RAND_status()
-    try:
-        socket.RAND_egd(1)
-    except TypeError:
-        pass
-    else:
-        print "didn't raise TypeError"
-    socket.RAND_add("this is a random string", 75.0)
-
-    f = urllib.urlopen('https://sf.net')
-    buf = f.read()
-    f.close()
-
-def test_rude_shutdown():
-    # This test deadlocks, see http://bugs.jython.org/issue1049
-    if test_support.is_jython:
-        return
-    try:
-        import thread
-    except ImportError:
-        return
-
-    # some random port to connect to
-    PORT = 9934
-    def listener():
-        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        s.bind(('', PORT))
-        s.listen(5)
-        s.accept()
-        del s
-        thread.exit()
-
-    def connector():
-        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        s.connect(('localhost', PORT))
-        try:
-            ssl_sock = socket.ssl(s)
-        except socket.sslerror:
-            pass
-        else:
-            raise test_support.TestFailed, \
-                        'connecting to closed SSL socket failed'
-
-    thread.start_new_thread(listener, ())
-    time.sleep(1)
-    connector()
-
-def test_https_socket():
-    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-    s.connect(('www.verisign.com', 443))
-    ssl_sock = socket.ssl(s)
-    ssl_sock.server()
-    ssl_sock.issuer()
-    s.close()    
-
-def test_main():
-    if not hasattr(socket, "ssl"):
-        raise test_support.TestSkipped("socket module has no ssl support")
-    test_rude_shutdown()
-    test_basic()
-    test_https_socket()
-
-if __name__ == "__main__":
-    test_main()
diff --git a/Lib/test/test_socketserver.py b/Lib/test/test_socketserver.py
--- a/Lib/test/test_socketserver.py
+++ b/Lib/test/test_socketserver.py
@@ -33,8 +33,7 @@
         signal.alarm(n)
 
 select_fn = select.select
-if test.test_support.is_jython:
-    select_fn = select.cpython_compatible_select
+
 
 def receive(sock, n, timeout=20):
     r, w, x = select_fn([sock], [], [], timeout)
diff --git a/Lib/test/test_urllib2_localnet.py b/Lib/test/test_urllib2_localnet.py
deleted file mode 100644
--- a/Lib/test/test_urllib2_localnet.py
+++ /dev/null
@@ -1,557 +0,0 @@
-#!/usr/bin/env python
-
-import urlparse
-import urllib2
-import BaseHTTPServer
-import unittest
-import hashlib
-
-from test import test_support
-
-if test_support.is_jython:
-    import socket
-    # Working around an IPV6 problem on Windows
-    socket._use_ipv4_addresses_only(True)
-
-mimetools = test_support.import_module('mimetools', deprecated=True)
-threading = test_support.import_module('threading')
-
-# Loopback http server infrastructure
-
-class LoopbackHttpServer(BaseHTTPServer.HTTPServer):
-    """HTTP server w/ a few modifications that make it useful for
-    loopback testing purposes.
-    """
-
-    def __init__(self, server_address, RequestHandlerClass):
-        BaseHTTPServer.HTTPServer.__init__(self,
-                                           server_address,
-                                           RequestHandlerClass,
-                                           True)
-
-        host, port = self.socket.getsockname()[:2]
-        self.server_name = socket.getfqdn(host)
-        self.server_port = port
-
-        # Set the timeout of our listening socket really low so
-        # that we can stop the server easily.
-        self.socket.settimeout(1.0)
-
-    def get_request(self):
-        """BaseHTTPServer method, overridden."""
-
-        request, client_address = self.socket.accept()
-
-        # It's a loopback connection, so setting the timeout
-        # really low shouldn't affect anything, but should make
-        # deadlocks less likely to occur.
-        request.settimeout(10.0)
-
-        return (request, client_address)
-
-class LoopbackHttpServerThread(threading.Thread):
-    """Stoppable thread that runs a loopback http server."""
-
-    def __init__(self, request_handler):
-        threading.Thread.__init__(self)
-        self._stop = False
-        self.ready = threading.Event()
-        request_handler.protocol_version = "HTTP/1.0"
-        self.httpd = LoopbackHttpServer(('127.0.0.1', 0),
-                                        request_handler)
-        #print "Serving HTTP on %s port %s" % (self.httpd.server_name,
-        #                                      self.httpd.server_port)
-        self.port = self.httpd.server_port
-
-    def stop(self):
-        """Stops the webserver if it's currently running."""
-
-        # Set the stop flag.
-        self._stop = True
-
-        self.join()
-
-    def run(self):
-        self.ready.set()
-        while not self._stop:
-            self.httpd.handle_request()
-
-# Authentication infrastructure
-
-class DigestAuthHandler:
-    """Handler for performing digest authentication."""
-
-    def __init__(self):
-        self._request_num = 0
-        self._nonces = []
-        self._users = {}
-        self._realm_name = "Test Realm"
-        self._qop = "auth"
-
-    def set_qop(self, qop):
-        self._qop = qop
-
-    def set_users(self, users):
-        assert isinstance(users, dict)
-        self._users = users
-
-    def set_realm(self, realm):
-        self._realm_name = realm
-
-    def _generate_nonce(self):
-        self._request_num += 1
-        nonce = hashlib.md5(str(self._request_num)).hexdigest()
-        self._nonces.append(nonce)
-        return nonce
-
-    def _create_auth_dict(self, auth_str):
-        first_space_index = auth_str.find(" ")
-        auth_str = auth_str[first_space_index+1:]
-
-        parts = auth_str.split(",")
-
-        auth_dict = {}
-        for part in parts:
-            name, value = part.split("=")
-            name = name.strip()
-            if value[0] == '"' and value[-1] == '"':
-                value = value[1:-1]
-            else:
-                value = value.strip()
-            auth_dict[name] = value
-        return auth_dict
-
-    def _validate_auth(self, auth_dict, password, method, uri):
-        final_dict = {}
-        final_dict.update(auth_dict)
-        final_dict["password"] = password
-        final_dict["method"] = method
-        final_dict["uri"] = uri
-        HA1_str = "%(username)s:%(realm)s:%(password)s" % final_dict
-        HA1 = hashlib.md5(HA1_str).hexdigest()
-        HA2_str = "%(method)s:%(uri)s" % final_dict
-        HA2 = hashlib.md5(HA2_str).hexdigest()
-        final_dict["HA1"] = HA1
-        final_dict["HA2"] = HA2
-        response_str = "%(HA1)s:%(nonce)s:%(nc)s:" \
-                       "%(cnonce)s:%(qop)s:%(HA2)s" % final_dict
-        response = hashlib.md5(response_str).hexdigest()
-
-        return response == auth_dict["response"]
-
-    def _return_auth_challenge(self, request_handler):
-        request_handler.send_response(407, "Proxy Authentication Required")
-        request_handler.send_header("Content-Type", "text/html")
-        request_handler.send_header(
-            'Proxy-Authenticate', 'Digest realm="%s", '
-            'qop="%s",'
-            'nonce="%s", ' % \
-            (self._realm_name, self._qop, self._generate_nonce()))
-        # XXX: Not sure if we're supposed to add this next header or
-        # not.
-        #request_handler.send_header('Connection', 'close')
-        request_handler.end_headers()
-        request_handler.wfile.write("Proxy Authentication Required.")
-        return False
-
-    def handle_request(self, request_handler):
-        """Performs digest authentication on the given HTTP request
-        handler.  Returns True if authentication was successful, False
-        otherwise.
-
-        If no users have been set, then digest auth is effectively
-        disabled and this method will always return True.
-        """
-
-        if len(self._users) == 0:
-            return True
-
-        if 'Proxy-Authorization' not in request_handler.headers:
-            return self._return_auth_challenge(request_handler)
-        else:
-            auth_dict = self._create_auth_dict(
-                request_handler.headers['Proxy-Authorization']
-                )
-            if auth_dict["username"] in self._users:
-                password = self._users[ auth_dict["username"] ]
-            else:
-                return self._return_auth_challenge(request_handler)
-            if not auth_dict.get("nonce") in self._nonces:
-                return self._return_auth_challenge(request_handler)
-            else:
-                self._nonces.remove(auth_dict["nonce"])
-
-            auth_validated = False
-
-            # MSIE uses short_path in its validation, but Python's
-            # urllib2 uses the full path, so we're going to see if
-            # either of them works here.
-
-            for path in [request_handler.path, request_handler.short_path]:
-                if self._validate_auth(auth_dict,
-                                       password,
-                                       request_handler.command,
-                                       path):
-                    auth_validated = True
-
-            if not auth_validated:
-                return self._return_auth_challenge(request_handler)
-            return True
-
-# Proxy test infrastructure
-
-class FakeProxyHandler(BaseHTTPServer.BaseHTTPRequestHandler):
-    """This is a 'fake proxy' that makes it look like the entire
-    internet has gone down due to a sudden zombie invasion.  It main
-    utility is in providing us with authentication support for
-    testing.
-    """
-
-    def __init__(self, digest_auth_handler, *args, **kwargs):
-        # This has to be set before calling our parent's __init__(), which will
-        # try to call do_GET().
-        self.digest_auth_handler = digest_auth_handler
-        BaseHTTPServer.BaseHTTPRequestHandler.__init__(self, *args, **kwargs)
-
-    def log_message(self, format, *args):
-        # Uncomment the next line for debugging.
-        #sys.stderr.write(format % args)
-        pass
-
-    def do_GET(self):
-        (scm, netloc, path, params, query, fragment) = urlparse.urlparse(
-            self.path, 'http')
-        self.short_path = path
-        if self.digest_auth_handler.handle_request(self):
-            self.send_response(200, "OK")
-            self.send_header("Content-Type", "text/html")
-            self.end_headers()
-            self.wfile.write("You've reached %s!<BR>" % self.path)
-            self.wfile.write("Our apologies, but our server is down due to "
-                              "a sudden zombie invasion.")
-
-# Test cases
-
-class BaseTestCase(unittest.TestCase):
-    def setUp(self):
-        self._threads = test_support.threading_setup()
-
-    def tearDown(self):
-        test_support.threading_cleanup(*self._threads)
-
-
-class ProxyAuthTests(BaseTestCase):
-    URL = "http://localhost"
-
-    USER = "tester"
-    PASSWD = "test123"
-    REALM = "TestRealm"
-
-    def setUp(self):
-        super(ProxyAuthTests, self).setUp()
-        self.digest_auth_handler = DigestAuthHandler()
-        self.digest_auth_handler.set_users({self.USER: self.PASSWD})
-        self.digest_auth_handler.set_realm(self.REALM)
-        def create_fake_proxy_handler(*args, **kwargs):
-            return FakeProxyHandler(self.digest_auth_handler, *args, **kwargs)
-
-        self.server = LoopbackHttpServerThread(create_fake_proxy_handler)
-        self.server.start()
-        self.server.ready.wait()
-        proxy_url = "http://127.0.0.1:%d" % self.server.port
-        handler = urllib2.ProxyHandler({"http" : proxy_url})
-        self.proxy_digest_handler = urllib2.ProxyDigestAuthHandler()
-        self.opener = urllib2.build_opener(handler, self.proxy_digest_handler)
-
-    def tearDown(self):
-        self.server.stop()
-        super(ProxyAuthTests, self).tearDown()
-
-    def test_proxy_with_bad_password_raises_httperror(self):
-        self.proxy_digest_handler.add_password(self.REALM, self.URL,
-                                               self.USER, self.PASSWD+"bad")
-        self.digest_auth_handler.set_qop("auth")
-        self.assertRaises(urllib2.HTTPError,
-                          self.opener.open,
-                          self.URL)
-
-    def test_proxy_with_no_password_raises_httperror(self):
-        self.digest_auth_handler.set_qop("auth")
-        self.assertRaises(urllib2.HTTPError,
-                          self.opener.open,
-                          self.URL)
-
-    def test_proxy_qop_auth_works(self):
-        self.proxy_digest_handler.add_password(self.REALM, self.URL,
-                                               self.USER, self.PASSWD)
-        self.digest_auth_handler.set_qop("auth")
-        result = self.opener.open(self.URL)
-        while result.read():
-            pass
-        result.close()
-
-    def test_proxy_qop_auth_int_works_or_throws_urlerror(self):
-        self.proxy_digest_handler.add_password(self.REALM, self.URL,
-                                               self.USER, self.PASSWD)
-        self.digest_auth_handler.set_qop("auth-int")
-        try:
-            result = self.opener.open(self.URL)
-        except urllib2.URLError:
-            # It's okay if we don't support auth-int, but we certainly
-            # shouldn't receive any kind of exception here other than
-            # a URLError.
-            result = None
-        if result:
-            while result.read():
-                pass
-            result.close()
-
-
-def GetRequestHandler(responses):
-
-    class FakeHTTPRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler):
-
-        server_version = "TestHTTP/"
-        requests = []
-        headers_received = []
-        port = 80
-
-        def do_GET(self):
-            body = self.send_head()
-            if body:
-                self.wfile.write(body)
-
-        def do_POST(self):
-            content_length = self.headers['Content-Length']
-            post_data = self.rfile.read(int(content_length))
-            self.do_GET()
-            self.requests.append(post_data)
-
-        def send_head(self):
-            FakeHTTPRequestHandler.headers_received = self.headers
-            self.requests.append(self.path)
-            response_code, headers, body = responses.pop(0)
-
-            self.send_response(response_code)
-
-            for (header, value) in headers:
-                self.send_header(header, value % self.port)
-            if body:
-                self.send_header('Content-type', 'text/plain')
-                self.end_headers()
-                return body
-            self.end_headers()
-
-        def log_message(self, *args):
-            pass
-
-
-    return FakeHTTPRequestHandler
-
-
-class TestUrlopen(BaseTestCase):
-    """Tests urllib2.urlopen using the network.
-
-    These tests are not exhaustive.  Assuming that testing using files does a
-    good job overall of some of the basic interface features.  There are no
-    tests exercising the optional 'data' and 'proxies' arguments.  No tests
-    for transparent redirection have been written.
-    """
-
-    def setUp(self):
-        proxy_handler = urllib2.ProxyHandler({})
-        opener = urllib2.build_opener(proxy_handler)
-        urllib2.install_opener(opener)
-        super(TestUrlopen, self).setUp()
-
-    def start_server(self, responses):
-        handler = GetRequestHandler(responses)
-
-        self.server = LoopbackHttpServerThread(handler)
-        self.server.start()
-        self.server.ready.wait()
-        port = self.server.port
-        handler.port = port
-        return handler
-
-
-    def test_redirection(self):
-        expected_response = 'We got here...'
-        responses = [
-            (302, [('Location', 'http://localhost:%s/somewhere_else')], ''),
-            (200, [], expected_response)
-        ]
-
-        handler = self.start_server(responses)
-
-        try:
-            f = urllib2.urlopen('http://localhost:%s/' % handler.port)
-            data = f.read()
-            f.close()
-
-            self.assertEqual(data, expected_response)
-            self.assertEqual(handler.requests, ['/', '/somewhere_else'])
-        finally:
-            self.server.stop()
-
-
-    def test_404(self):
-        expected_response = 'Bad bad bad...'
-        handler = self.start_server([(404, [], expected_response)])
-
-        try:
-            try:
-                urllib2.urlopen('http://localhost:%s/weeble' % handler.port)
-            except urllib2.URLError, f:
-                pass
-            else:
-                self.fail('404 should raise URLError')
-
-            data = f.read()
-            f.close()
-
-            self.assertEqual(data, expected_response)
-            self.assertEqual(handler.requests, ['/weeble'])
-        finally:
-            self.server.stop()
-
-
-    def test_200(self):
-        expected_response = 'pycon 2008...'
-        handler = self.start_server([(200, [], expected_response)])
-
-        try:
-            f = urllib2.urlopen('http://localhost:%s/bizarre' % handler.port)
-            data = f.read()
-            f.close()
-
-            self.assertEqual(data, expected_response)
-            self.assertEqual(handler.requests, ['/bizarre'])
-        finally:
-            self.server.stop()
-
-    def test_200_with_parameters(self):
-        expected_response = 'pycon 2008...'
-        handler = self.start_server([(200, [], expected_response)])
-
-        try:
-            f = urllib2.urlopen('http://localhost:%s/bizarre' % handler.port, 'get=with_feeling')
-            data = f.read()
-            f.close()
-
-            self.assertEqual(data, expected_response)
-            self.assertEqual(handler.requests, ['/bizarre', 'get=with_feeling'])
-        finally:
-            self.server.stop()
-
-
-    def test_sending_headers(self):
-        handler = self.start_server([(200, [], "we don't care")])
-
-        try:
-            req = urllib2.Request("http://localhost:%s/" % handler.port,
-                                  headers={'Range': 'bytes=20-39'})
-            urllib2.urlopen(req)
-            self.assertEqual(handler.headers_received['Range'], 'bytes=20-39')
-        finally:
-            self.server.stop()
-
-    def test_basic(self):
-        handler = self.start_server([(200, [], "we don't care")])
-
-        try:
-            open_url = urllib2.urlopen("http://localhost:%s" % handler.port)
-            for attr in ("read", "close", "info", "geturl"):
-                self.assertTrue(hasattr(open_url, attr), "object returned from "
-                             "urlopen lacks the %s attribute" % attr)
-            try:
-                self.assertTrue(open_url.read(), "calling 'read' failed")
-            finally:
-                open_url.close()
-        finally:
-            self.server.stop()
-
-    def test_info(self):
-        handler = self.start_server([(200, [], "we don't care")])
-
-        try:
-            open_url = urllib2.urlopen("http://localhost:%s" % handler.port)
-            info_obj = open_url.info()
-            self.assertIsInstance(info_obj, mimetools.Message,
-                                  "object returned by 'info' is not an "
-                                  "instance of mimetools.Message")
-            self.assertEqual(info_obj.getsubtype(), "plain")
-        finally:
-            self.server.stop()
-
-    def test_geturl(self):
-        # Make sure same URL as opened is returned by geturl.
-        handler = self.start_server([(200, [], "we don't care")])
-
-        try:
-            open_url = urllib2.urlopen("http://localhost:%s" % handler.port)
-            url = open_url.geturl()
-            self.assertEqual(url, "http://localhost:%s" % handler.port)
-        finally:
-            self.server.stop()
-
-
-    def test_bad_address(self):
-        # Make sure proper exception is raised when connecting to a bogus
-        # address.
-        self.assertRaises(IOError,
-                          # Given that both VeriSign and various ISPs have in
-                          # the past or are presently hijacking various invalid
-                          # domain name requests in an attempt to boost traffic
-                          # to their own sites, finding a domain name to use
-                          # for this test is difficult.  RFC2606 leads one to
-                          # believe that '.invalid' should work, but experience
-                          # seemed to indicate otherwise.  Single character
-                          # TLDs are likely to remain invalid, so this seems to
-                          # be the best choice. The trailing '.' prevents a
-                          # related problem: The normal DNS resolver appends
-                          # the domain names from the search path if there is
-                          # no '.' the end and, and if one of those domains
-                          # implements a '*' rule a result is returned.
-                          # However, none of this will prevent the test from
-                          # failing if the ISP hijacks all invalid domain
-                          # requests.  The real solution would be to be able to
-                          # parameterize the framework with a mock resolver.
-                          urllib2.urlopen, "http://sadflkjsasf.i.nvali.d./")
-
-    def test_iteration(self):
-        expected_response = "pycon 2008..."
-        handler = self.start_server([(200, [], expected_response)])
-        try:
-            data = urllib2.urlopen("http://localhost:%s" % handler.port)
-            for line in data:
-                self.assertEqual(line, expected_response)
-        finally:
-            self.server.stop()
-
-    def ztest_line_iteration(self):
-        lines = ["We\n", "got\n", "here\n", "verylong " * 8192 + "\n"]
-        expected_response = "".join(lines)
-        handler = self.start_server([(200, [], expected_response)])
-        try:
-            data = urllib2.urlopen("http://localhost:%s" % handler.port)
-            for index, line in enumerate(data):
-                self.assertEqual(line, lines[index],
-                                 "Fetched line number %s doesn't match expected:\n"
-                                 "    Expected length was %s, got %s" %
-                                 (index, len(lines[index]), len(line)))
-        finally:
-            self.server.stop()
-        self.assertEqual(index + 1, len(lines))
-
-def test_main():
-    # We will NOT depend on the network resource flag
-    # (Lib/test/regrtest.py -u network) since all tests here are only
-    # localhost.  However, if this is a bad rationale, then uncomment
-    # the next line.
-    #test_support.requires("network")
-
-    test_support.run_unittest(ProxyAuthTests, TestUrlopen)
-
-if __name__ == "__main__":
-    test_main()
diff --git a/Lib/uu.py b/Lib/uu.py
deleted file mode 100755
--- a/Lib/uu.py
+++ /dev/null
@@ -1,211 +0,0 @@
-#! /usr/bin/env python
-
-# Copyright 1994 by Lance Ellinghouse
-# Cathedral City, California Republic, United States of America.
-#                        All Rights Reserved
-# Permission to use, copy, modify, and distribute this software and its
-# documentation for any purpose and without fee is hereby granted,
-# provided that the above copyright notice appear in all copies and that
-# both that copyright notice and this permission notice appear in
-# supporting documentation, and that the name of Lance Ellinghouse
-# not be used in advertising or publicity pertaining to distribution
-# of the software without specific, written prior permission.
-# LANCE ELLINGHOUSE DISCLAIMS ALL WARRANTIES WITH REGARD TO
-# THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
-# FITNESS, IN NO EVENT SHALL LANCE ELLINGHOUSE CENTRUM BE LIABLE
-# FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
-# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
-# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
-# OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-#
-# Modified by Jack Jansen, CWI, July 1995:
-# - Use binascii module to do the actual line-by-line conversion
-#   between ascii and binary. This results in a 1000-fold speedup. The C
-#   version is still 5 times faster, though.
-# - Arguments more compliant with python standard
-
-"""Implementation of the UUencode and UUdecode functions.
-
-encode(in_file, out_file [,name, mode])
-decode(in_file [, out_file, mode])
-"""
-
-import binascii
-import os
-import sys
-
-__all__ = ["Error", "encode", "decode"]
-
-class Error(Exception):
-    pass
-
-def encode(in_file, out_file, name=None, mode=None):
-    """Uuencode file"""
-    #
-    # If in_file is a pathname open it and change defaults
-    #
-
-    close_in_file = False
-    close_out_file = False
-
-    if in_file == '-':
-        in_file = sys.stdin
-    elif isinstance(in_file, basestring):
-        if name is None:
-            name = os.path.basename(in_file)
-        if mode is None:
-            try:
-                mode = os.stat(in_file).st_mode
-            except AttributeError:
-                pass
-        in_file = open(in_file, 'rb')
-        close_in_file = True
-    #
-    # Open out_file if it is a pathname
-    #
-    if out_file == '-':
-        out_file = sys.stdout
-    elif isinstance(out_file, basestring):
-        out_file = open(out_file, 'w')
-        close_out_file = True
-    #
-    # Set defaults for name and mode
-    #
-    if name is None:
-        name = '-'
-    if mode is None:
-        mode = 0666
-    #
-    # Write the data
-    #
-    out_file.write('begin %o %s\n' % ((mode&0777),name))
-    data = in_file.read(45)
-    while len(data) > 0:
-        out_file.write(binascii.b2a_uu(data))
-        data = in_file.read(45)
-    out_file.write(' \nend\n')
-
-    # Jython and other implementations requires files to be explicitly
-    # closed if we don't want to wait for GC
-    if close_in_file:
-        in_file.close()
-    if close_out_file:
-        out_file.close()
-
-def decode(in_file, out_file=None, mode=None, quiet=0):
-    """Decode uuencoded file"""
-
-    close_in_file = False
-    close_out_file = False
-
-    #
-    # Open the input file, if needed.
-    #
-    if in_file == '-':
-        in_file = sys.stdin
-    elif isinstance(in_file, basestring):
-        close_in_file = True
-        in_file = open(in_file)
-    #
-    # Read until a begin is encountered or we've exhausted the file
-    #
-    while True:
-        hdr = in_file.readline()
-        if not hdr:
-            raise Error('No valid begin line found in input file')
-        if not hdr.startswith('begin'):
-            continue
-        hdrfields = hdr.split(' ', 2)
-        if len(hdrfields) == 3 and hdrfields[0] == 'begin':
-            try:
-                int(hdrfields[1], 8)
-                break
-            except ValueError:
-                pass
-    if out_file is None:
-        out_file = hdrfields[2].rstrip()
-        if os.path.exists(out_file):
-            raise Error('Cannot overwrite existing file: %s' % out_file)
-    if mode is None:
-        mode = int(hdrfields[1], 8)
-    #
-    # Open the output file
-    #
-    opened = False
-    if out_file == '-':
-        out_file = sys.stdout
-    elif isinstance(out_file, basestring):
-        close_out_file = True
-        fp = open(out_file, 'wb')
-        try:
-            os.path.chmod(out_file, mode)
-        except AttributeError:
-            pass
-        out_file = fp
-        opened = True
-    #
-    # Main decoding loop
-    #
-    s = in_file.readline()
-    while s and s.strip() != 'end':
-        try:
-            data = binascii.a2b_uu(s)
-        except binascii.Error, v:
-            # Workaround for broken uuencoders by /Fredrik Lundh
-            nbytes = (((ord(s[0])-32) & 63) * 4 + 5) // 3
-            data = binascii.a2b_uu(s[:nbytes])
-            if not quiet:
-                sys.stderr.write("Warning: %s\n" % v)
-        out_file.write(data)
-        s = in_file.readline()
-    if not s:
-        raise Error('Truncated input file')
-    if opened:
-        out_file.close()
-
-    # Jython and other implementations requires files to be explicitly
-    # closed if we don't want to wait for GC
-    if close_in_file:
-        in_file.close()
-    if close_out_file:
-        out_file.close()
-
-def test():
-    """uuencode/uudecode main program"""
-
-    import optparse
-    parser = optparse.OptionParser(usage='usage: %prog [-d] [-t] [input [output]]')
-    parser.add_option('-d', '--decode', dest='decode', help='Decode (instead of encode)?', default=False, action='store_true')
-    parser.add_option('-t', '--text', dest='text', help='data is text, encoded format unix-compatible text?', default=False, action='store_true')
-
-    (options, args) = parser.parse_args()
-    if len(args) > 2:
-        parser.error('incorrect number of arguments')
-        sys.exit(1)
-
-    input = sys.stdin
-    output = sys.stdout
-    if len(args) > 0:
-        input = args[0]
-    if len(args) > 1:
-        output = args[1]
-
-    if options.decode:
-        if options.text:
-            if isinstance(output, basestring):
-                output = open(output, 'w')
-            else:
-                print sys.argv[0], ': cannot do -t to stdout'
-                sys.exit(1)
-        decode(input, output)
-    else:
-        if options.text:
-            if isinstance(input, basestring):
-                input = open(input, 'r')
-            else:
-                print sys.argv[0], ': cannot do -t from stdin'
-                sys.exit(1)
-        encode(input, output)
-
-if __name__ == '__main__':
-    test()
diff --git a/Lib/zlib.py b/Lib/zlib.py
--- a/Lib/zlib.py
+++ b/Lib/zlib.py
@@ -16,11 +16,11 @@
 import array
 import binascii
 import jarray
+from cStringIO import StringIO
 
+from java.lang import Long, String, System
 from java.util.zip import Adler32, Deflater, Inflater, DataFormatException
-from java.lang import Long, String
 
-from cStringIO import StringIO
 
 class error(Exception):
     pass
@@ -56,7 +56,6 @@
 def crc32(string, value=0):
     return binascii.crc32(string, value)
 
-
 def compress(string, level=6):
     if level < Z_BEST_SPEED or level > Z_BEST_COMPRESSION:
         raise error, "Bad compression level"
@@ -77,7 +76,8 @@
     finally:
         inflater.end()
 
-class compressobj:
+
+class compressobj(object):
     # all jython uses wbits for is deciding whether to skip the header if it's negative
     def __init__(self, level=6, method=DEFLATED, wbits=MAX_WBITS,
                        memLevel=0, strategy=0):
@@ -108,14 +108,26 @@
             self._ended = True
         return last
 
-class decompressobj:
-    # all jython uses wbits for is deciding whether to skip the header if it's negative
+
+class decompressobj(object):
+
     def __init__(self, wbits=MAX_WBITS):
-        if abs(wbits) > MAX_WBITS or abs(wbits) < 8:
+
+        # Jython only uses wbits to determine to skip the header if it's negative;
+        # but apparently there are some tests around this that we do some bogus
+        # param checking
+
+        if abs(wbits) < 8:
             raise ValueError, "Invalid initialization option"
+        if abs(wbits) > 16:  # NOTE apparently this also implies being negative in CPython/zlib
+            wbits = -1
+
         self.inflater = Inflater(wbits < 0)
+        self._ended = False
         self.unused_data = ""
-        self._ended = False
+        self.unconsumed_tail = ""
+        self.gzip = wbits < 0
+        self.gzip_header_skipped = False
 
     def decompress(self, string, max_length=0):
         if self._ended:
@@ -132,7 +144,13 @@
         if max_length < 0:
             raise ValueError("max_length must be a positive integer")
 
+        # Suppress gzip header if present and wbits < 0
+        if self.gzip and not self.gzip_header_skipped:
+            string = _skip_gzip_header(string)
+            self.gzip_header_skipped = True
+
         string = _to_input(string)
+
         self.inflater.setInput(string)
         inflated = _get_inflate_data(self.inflater, max_length)
 
@@ -146,6 +164,7 @@
         return inflated
 
     def flush(self, length=None):
+        # FIXME close input streams if gzip
         if self._ended:
             raise error("decompressobj may not be used after flush()")
         if length is None:
@@ -193,3 +212,52 @@
             break
     s.seek(0)
     return s.read()
+
+
+
+FTEXT = 1
+FHCRC = 2
+FEXTRA = 4
+FNAME = 8
+FCOMMENT = 16
+
+def _skip_gzip_header(string):
+    # per format specified in http://tools.ietf.org/html/rfc1952
+    
+    # could we use bytearray instead?
+    s = array.array("B", string)
+
+    id1 = s[0]
+    id2 = s[1]
+
+    # Check gzip magic
+    if id1 != 31 or id2 != 139:
+        return string
+
+    cm = s[2]
+    flg = s[3]
+    mtime = s[4:8]
+    xfl = s[8]
+    os = s[9]
+
+    # skip fixed header, then figure out variable parts
+    s = s[10:]
+
+    if flg & FEXTRA:
+        # skip extra field
+        xlen = s[0] + s[1] * 256  # MSB ordering
+        s = s[2 + xlen:]
+    if flg & FNAME:
+        # skip filename
+        s = s[s.find("\x00")+1:]
+    if flg & FCOMMENT:
+        # skip comment
+        s = s[s.find("\x00")+1:]
+    if flg & FHCRC:
+        # skip CRC16 for the header - might be nice to check of course
+        s = s[2:]
+    
+    return s.tostring()
+
+
+
diff --git a/build.xml b/build.xml
--- a/build.xml
+++ b/build.xml
@@ -573,12 +573,26 @@
             <zipfileset src="extlibs/asm-commons-4.0.jar"/>
             <zipfileset src="extlibs/asm-util-4.0.jar"/>
             <rule pattern="org.objectweb.asm.**" result="org.python.objectweb.asm. at 1"/>
+	    <zipfileset src="extlibs/bcpkix-jdk15on-150.jar" excludes="META-INF/*.SF"/>
+	    <rule pattern="org.bouncycastle.**" result="org.python.bouncycastle. at 1"/>
+	    <zipfileset src="extlibs/bcprov-jdk15on-150.jar" excludes="META-INF/*.SF"/>
+	    <rule pattern="org.bouncycastle.**" result="org.python.bouncycastle. at 1"/>
 	    <zipfileset src="extlibs/commons-compress-1.4.1.jar"/>
 	    <rule pattern="org.apache.**" result="org.python.apache. at 1"/>
             <zipfileset src="extlibs/guava-13.0.1.jar"/>
             <rule pattern="com.google.**" result="org.python.google. at 1"/>
             <zipfileset src="extlibs/icu4j-52_1.jar"/>
             <rule pattern="com.ibm.icu.**" result="org.python.icu. at 1"/>
+            <zipfileset src="extlibs/netty-buffer-4.0.18.Final.jar"/>
+            <rule pattern="io.netty.**" result="org.python.netty. at 1"/>
+            <zipfileset src="extlibs/netty-codec-4.0.18.Final.jar"/>
+            <rule pattern="io.netty.**" result="org.python.netty. at 1"/>
+            <zipfileset src="extlibs/netty-common-4.0.18.Final.jar"/>
+            <rule pattern="io.netty.**" result="org.python.netty. at 1"/>
+            <zipfileset src="extlibs/netty-handler-4.0.18.Final.jar"/>
+            <rule pattern="io.netty.**" result="org.python.netty. at 1"/>
+            <zipfileset src="extlibs/netty-transport-4.0.18.Final.jar"/>
+            <rule pattern="io.netty.**" result="org.python.netty. at 1"/>
             <zipfileset src="extlibs/jffi-arm-Linux.jar"/>
             <zipfileset src="extlibs/jffi-Darwin.jar"/>
             <zipfileset src="extlibs/jffi-i386-FreeBSD.jar"/>
diff --git a/extlibs/bcpkix-jdk15on-150.jar b/extlibs/bcpkix-jdk15on-150.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..5dc125fc4945a73e628fec34001503899e9c0b10
GIT binary patch
[stripped]
diff --git a/extlibs/bcprov-jdk15on-150.jar b/extlibs/bcprov-jdk15on-150.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..d4b510d7894afa3775fdfb7ab2b3a704baf82c99
GIT binary patch
[stripped]
diff --git a/extlibs/netty-buffer-4.0.18.Final.jar b/extlibs/netty-buffer-4.0.18.Final.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..4ef63f516ba9c0dbb951c85a7b1382f9d3c1f23f
GIT binary patch
[stripped]
diff --git a/extlibs/netty-codec-4.0.18.Final.jar b/extlibs/netty-codec-4.0.18.Final.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..248d26950900a16fa5612b06aaaf0a58d6b000fc
GIT binary patch
[stripped]
diff --git a/extlibs/netty-common-4.0.18.Final.jar b/extlibs/netty-common-4.0.18.Final.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..ee7491c13be9228430aecbe358297f2aada76db1
GIT binary patch
[stripped]
diff --git a/extlibs/netty-handler-4.0.18.Final.jar b/extlibs/netty-handler-4.0.18.Final.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..22eb114f569f9c6f5ca05cf4a935c9913f5c29d8
GIT binary patch
[stripped]
diff --git a/extlibs/netty-transport-4.0.18.Final.jar b/extlibs/netty-transport-4.0.18.Final.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..133c0fa65c71555ed90c960bf95233167aacc4c1
GIT binary patch
[stripped]

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Fri May  9 20:44:58 2014
From: jython-checkins at python.org (jim.baker)
Date: Fri,  9 May 2014 20:44:58 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Fix_os=2Estat_behaviour_for?=
 =?utf-8?q?_paths_with_trailing_slash_when_the_path_exists_as_a?=
Message-ID: <3gQL7Z1vf2z7Ljc@mail.python.org>

http://hg.python.org/jython/rev/a7b71df0cfd3
changeset:   7223:a7b71df0cfd3
parent:      7181:6e438088c0e3
user:        Indra Talip <indra.talip at gmail.com>
date:        Fri Apr 25 15:50:35 2014 +1000
summary:
  Fix os.stat behaviour for paths with trailing slash when the path exists as a file

files:
  Lib/test/test_os_jy.py                        |  15 ++++++++++
  src/org/python/modules/posix/PosixModule.java |  10 ++++++-
  2 files changed, 24 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_os_jy.py b/Lib/test/test_os_jy.py
--- a/Lib/test/test_os_jy.py
+++ b/Lib/test/test_os_jy.py
@@ -76,10 +76,25 @@
         os.removedirs(p)
 
 
+class OSStatTestCase(unittest.TestCase):
+
+    def setUp(self):
+        open(test_support.TESTFN, 'w').close()
+
+    def tearDown(self):
+        if os.path.exists(test_support.TESTFN):
+            os.remove(test_support.TESTFN)
+
+    def test_stat_with_trailing_slash(self):
+        self.assertRaises(OSError, os.stat, test_support.TESTFN + os.path.sep)
+        self.assertRaises(OSError, os.lstat, test_support.TESTFN + os.path.sep)
+
+
 def test_main():
     test_support.run_unittest(
         OSFileTestCase, 
         OSDirTestCase,
+        OSStatTestCase,
     )
 
 if __name__ == '__main__':
diff --git a/src/org/python/modules/posix/PosixModule.java b/src/org/python/modules/posix/PosixModule.java
--- a/src/org/python/modules/posix/PosixModule.java
+++ b/src/org/python/modules/posix/PosixModule.java
@@ -950,7 +950,15 @@
 
         @Override
         public PyObject __call__(PyObject path) {
-            return PyStatResult.fromFileStat(posix.stat(absolutePath(path)));
+            String absolutePath = absolutePath(path);
+
+            // round tripping from a string to a file to a string loses
+            // trailing slashes so add them back back in to get correct posix.stat
+            // behaviour if path is not a directory.
+            if (asPath(path).endsWith(File.separator)) {
+                absolutePath = absolutePath + File.separator;
+            }
+            return PyStatResult.fromFileStat(posix.stat(absolutePath));
         }
     }
 }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Fri May  9 20:44:59 2014
From: jython-checkins at python.org (jim.baker)
Date: Fri,  9 May 2014 20:44:59 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_Merged_pull_request?=
Message-ID: <3gQL7b3kYBz7Ljt@mail.python.org>

http://hg.python.org/jython/rev/ac645c8c4208
changeset:   7224:ac645c8c4208
parent:      7222:107fe4a4c96b
parent:      7223:a7b71df0cfd3
user:        Jim Baker <jim.baker at rackspace.com>
date:        Fri May 09 12:25:41 2014 -0600
summary:
  Merged pull request

files:
  Lib/test/test_os_jy.py                        |  15 ++++++++++
  src/org/python/modules/posix/PosixModule.java |  10 ++++++-
  2 files changed, 24 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_os_jy.py b/Lib/test/test_os_jy.py
--- a/Lib/test/test_os_jy.py
+++ b/Lib/test/test_os_jy.py
@@ -77,10 +77,25 @@
         os.removedirs(p)
 
 
+class OSStatTestCase(unittest.TestCase):
+
+    def setUp(self):
+        open(test_support.TESTFN, 'w').close()
+
+    def tearDown(self):
+        if os.path.exists(test_support.TESTFN):
+            os.remove(test_support.TESTFN)
+
+    def test_stat_with_trailing_slash(self):
+        self.assertRaises(OSError, os.stat, test_support.TESTFN + os.path.sep)
+        self.assertRaises(OSError, os.lstat, test_support.TESTFN + os.path.sep)
+
+
 def test_main():
     test_support.run_unittest(
         OSFileTestCase, 
         OSDirTestCase,
+        OSStatTestCase,
     )
 
 if __name__ == '__main__':
diff --git a/src/org/python/modules/posix/PosixModule.java b/src/org/python/modules/posix/PosixModule.java
--- a/src/org/python/modules/posix/PosixModule.java
+++ b/src/org/python/modules/posix/PosixModule.java
@@ -967,7 +967,15 @@
 
         @Override
         public PyObject __call__(PyObject path) {
-            return PyStatResult.fromFileStat(posix.stat(absolutePath(path)));
+            String absolutePath = absolutePath(path);
+
+            // round tripping from a string to a file to a string loses
+            // trailing slashes so add them back back in to get correct posix.stat
+            // behaviour if path is not a directory.
+            if (asPath(path).endsWith(File.separator)) {
+                absolutePath = absolutePath + File.separator;
+            }
+            return PyStatResult.fromFileStat(posix.stat(absolutePath));
         }
     }
 }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 00:09:25 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 00:09:25 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Fix_the_struct_module_so_th?=
 =?utf-8?q?at_struct=2EStruct_class_can_be_derived_from=2E?=
Message-ID: <3gQQgT2KH0z7LjP@mail.python.org>

http://hg.python.org/jython/rev/06a9cc256459
changeset:   7225:06a9cc256459
parent:      7181:6e438088c0e3
user:        Indra Talip <indra.talip at gmail.com>
date:        Sat Mar 22 14:32:48 2014 +1100
summary:
  Fix the struct module so that struct.Struct class can be derived from.

org.python.modules.struct now implements ClassDictInit allowing for PyStruct
to be exposed as a type instead of a reflected function. Modified PyStruct to
allow the generated PyStructDerived class to be instantiated thereby allowing
derived classes.

files:
  src/org/python/modules/PyStruct.java        |    41 +-
  src/org/python/modules/PyStructDerived.java |  1137 ++++++++++
  src/org/python/modules/struct.java          |     8 +-
  src/templates/PyStruct.derived              |     4 +
  4 files changed, 1168 insertions(+), 22 deletions(-)


diff --git a/src/org/python/modules/PyStruct.java b/src/org/python/modules/PyStruct.java
--- a/src/org/python/modules/PyStruct.java
+++ b/src/org/python/modules/PyStruct.java
@@ -1,5 +1,6 @@
 package org.python.modules;
 
+import org.python.core.ArgParser;
 import org.python.core.Py;
 import org.python.core.PyArray;
 import org.python.core.PyNewWrapper;
@@ -12,7 +13,7 @@
 import org.python.expose.ExposedNew;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "Struct", base = PyObject.class)
+ at ExposedType(name = "struct.Struct", base = PyObject.class)
 public class PyStruct extends PyObject {
     public static final PyType TYPE = PyType.fromClass(PyStruct.class);
     
@@ -30,27 +31,29 @@
         return TYPE;
     }
 
-    public PyStruct(PyObject[] args, String[] keywords) {
-        final int nargs = args.length;
-        if (keywords.length > 0 ) {
-            for (String keyword : keywords) {
-                if (!keyword.equals("format")) {
-                    throw Py.TypeError("'" + keyword + "' is an invalid keyword argument for this function");
-                }
-            }
-        }
-        else if (nargs < 1 || nargs > 1) {
-            throw Py.TypeError("Struct() takes exactly 1 argument (" + nargs + " given)");
-        }
-        format = args[0].toString();
-        format_def = struct.whichtable(format);
-        size = struct.calcsize(format, format_def);
+    public PyStruct(PyString format) {
+        this(TYPE, format);
     }
-    
+
+    public PyStruct(PyType type, PyString format) {
+        super(type);
+        this.format = format.toString();
+        this.format_def = struct.whichtable(this.format);
+        this.size = struct.calcsize(this.format, this.format_def);
+    }
+
     @ExposedNew
     final static PyObject Struct___new__ (PyNewWrapper new_, boolean init,
             PyType subtype, PyObject[] args, String[] keywords) {
-         return new PyStruct(args, keywords);
+        ArgParser ap = new ArgParser("Struct", args, keywords, new String[] {"format"}, 1);
+
+        PyObject format = ap.getPyObject(0);
+        if (!(format instanceof PyString)) {
+            throw Py.TypeError("coercing to Unicode: need string, '"
+                    + format.getType().fastGetName() + "' type found");
+        }
+
+        return new PyStruct(TYPE, (PyString) format);
     }
 
     @ExposedMethod
@@ -85,4 +88,4 @@
             throw struct.StructError("unpack_from str size does not match format");
         return struct.unpack(format_def, size, format, new struct.ByteStream(s, offset));
     }
-}
\ No newline at end of file
+}
diff --git a/src/org/python/modules/PyStructDerived.java b/src/org/python/modules/PyStructDerived.java
new file mode 100644
--- /dev/null
+++ b/src/org/python/modules/PyStructDerived.java
@@ -0,0 +1,1137 @@
+/* Generated file, do not modify.  See jython/src/templates/gderived.py. */
+package org.python.modules;
+
+import java.io.Serializable;
+import org.python.core.*;
+
+public class PyStructDerived extends PyStruct implements Slotted {
+
+    public PyObject getSlot(int index) {
+        return slots[index];
+    }
+
+    public void setSlot(int index,PyObject value) {
+        slots[index]=value;
+    }
+
+    private PyObject[]slots;
+
+    private PyObject dict;
+
+    public PyObject fastGetDict() {
+        return dict;
+    }
+
+    public PyObject getDict() {
+        return dict;
+    }
+
+    public void setDict(PyObject newDict) {
+        if (newDict instanceof PyStringMap||newDict instanceof PyDictionary) {
+            dict=newDict;
+        } else {
+            throw Py.TypeError("__dict__ must be set to a Dictionary "+newDict.getClass().getName());
+        }
+    }
+
+    public void delDict() {
+        // deleting an object's instance dict makes it grow a new one
+        dict=new PyStringMap();
+    }
+
+    public PyStructDerived(PyType subtype,PyString format) {
+        super(subtype,format);
+        slots=new PyObject[subtype.getNumSlots()];
+        dict=subtype.instDict();
+    }
+
+    public PyString __str__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__str__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__str__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__str__();
+    }
+
+    public PyString __repr__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__repr__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__repr__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__repr__();
+    }
+
+    public PyString __hex__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__hex__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__hex__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__hex__();
+    }
+
+    public PyString __oct__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__oct__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__oct__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__oct__();
+    }
+
+    public PyFloat __float__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__float__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyFloat)
+                return(PyFloat)res;
+            throw Py.TypeError("__float__"+" returned non-"+"float"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__float__();
+    }
+
+    public PyComplex __complex__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__complex__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyComplex)
+                return(PyComplex)res;
+            throw Py.TypeError("__complex__"+" returned non-"+"complex"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__complex__();
+    }
+
+    public PyObject __pos__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__pos__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__pos__();
+    }
+
+    public PyObject __neg__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__neg__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__neg__();
+    }
+
+    public PyObject __abs__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__abs__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__abs__();
+    }
+
+    public PyObject __invert__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__invert__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__invert__();
+    }
+
+    public PyObject __reduce__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__reduce__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__reduce__();
+    }
+
+    public PyObject __dir__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__dir__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__dir__();
+    }
+
+    public PyObject __add__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__add__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__add__(other);
+    }
+
+    public PyObject __radd__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__radd__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__radd__(other);
+    }
+
+    public PyObject __sub__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__sub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__sub__(other);
+    }
+
+    public PyObject __rsub__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rsub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rsub__(other);
+    }
+
+    public PyObject __mul__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__mul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__mul__(other);
+    }
+
+    public PyObject __rmul__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rmul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rmul__(other);
+    }
+
+    public PyObject __div__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__div__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__div__(other);
+    }
+
+    public PyObject __rdiv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rdiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rdiv__(other);
+    }
+
+    public PyObject __floordiv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__floordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__floordiv__(other);
+    }
+
+    public PyObject __rfloordiv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rfloordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rfloordiv__(other);
+    }
+
+    public PyObject __truediv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__truediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__truediv__(other);
+    }
+
+    public PyObject __rtruediv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rtruediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rtruediv__(other);
+    }
+
+    public PyObject __mod__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__mod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__mod__(other);
+    }
+
+    public PyObject __rmod__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rmod__(other);
+    }
+
+    public PyObject __divmod__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__divmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__divmod__(other);
+    }
+
+    public PyObject __rdivmod__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rdivmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rdivmod__(other);
+    }
+
+    public PyObject __rpow__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rpow__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rpow__(other);
+    }
+
+    public PyObject __lshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__lshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__lshift__(other);
+    }
+
+    public PyObject __rlshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rlshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rlshift__(other);
+    }
+
+    public PyObject __rshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rshift__(other);
+    }
+
+    public PyObject __rrshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rrshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rrshift__(other);
+    }
+
+    public PyObject __and__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__and__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__and__(other);
+    }
+
+    public PyObject __rand__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rand__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rand__(other);
+    }
+
+    public PyObject __or__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__or__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__or__(other);
+    }
+
+    public PyObject __ror__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ror__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ror__(other);
+    }
+
+    public PyObject __xor__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__xor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__xor__(other);
+    }
+
+    public PyObject __rxor__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rxor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rxor__(other);
+    }
+
+    public PyObject __lt__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__lt__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__lt__(other);
+    }
+
+    public PyObject __le__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__le__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__le__(other);
+    }
+
+    public PyObject __gt__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__gt__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__gt__(other);
+    }
+
+    public PyObject __ge__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ge__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ge__(other);
+    }
+
+    public PyObject __eq__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__eq__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__eq__(other);
+    }
+
+    public PyObject __ne__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ne__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ne__(other);
+    }
+
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
+    public PyObject __iadd__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iadd__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__iadd__(other);
+    }
+
+    public PyObject __isub__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__isub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__isub__(other);
+    }
+
+    public PyObject __imul__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__imul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__imul__(other);
+    }
+
+    public PyObject __idiv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__idiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__idiv__(other);
+    }
+
+    public PyObject __ifloordiv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ifloordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ifloordiv__(other);
+    }
+
+    public PyObject __itruediv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__itruediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__itruediv__(other);
+    }
+
+    public PyObject __imod__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__imod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__imod__(other);
+    }
+
+    public PyObject __ipow__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ipow__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ipow__(other);
+    }
+
+    public PyObject __ilshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ilshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ilshift__(other);
+    }
+
+    public PyObject __irshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__irshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__irshift__(other);
+    }
+
+    public PyObject __iand__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iand__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__iand__(other);
+    }
+
+    public PyObject __ior__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ior__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ior__(other);
+    }
+
+    public PyObject __ixor__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ixor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ixor__(other);
+    }
+
+    public PyObject __int__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__int__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger||res instanceof PyLong)
+                return res;
+            throw Py.TypeError("__int__"+" should return an integer");
+        }
+        return super.__int__();
+    }
+
+    public PyObject __long__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__long__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyLong||res instanceof PyInteger)
+                return res;
+            throw Py.TypeError("__long__"+" returned non-"+"long"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__long__();
+    }
+
+    public int hashCode() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__hash__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger) {
+                return((PyInteger)res).getValue();
+            } else
+                if (res instanceof PyLong) {
+                    return((PyLong)res).getValue().intValue();
+                }
+            throw Py.TypeError("__hash__ should return a int");
+        }
+        if (self_type.lookup("__eq__")!=null||self_type.lookup("__cmp__")!=null) {
+            throw Py.TypeError(String.format("unhashable type: '%.200s'",getType().fastGetName()));
+        }
+        return super.hashCode();
+    }
+
+    public PyUnicode __unicode__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__unicode__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyUnicode)
+                return(PyUnicode)res;
+            if (res instanceof PyString)
+                return new PyUnicode((PyString)res);
+            throw Py.TypeError("__unicode__"+" should return a "+"unicode");
+        }
+        return super.__unicode__();
+    }
+
+    public int __cmp__(PyObject other) {
+        PyType self_type=getType();
+        PyObject[]where_type=new PyObject[1];
+        PyObject impl=self_type.lookup_where("__cmp__",where_type);
+        // Full Compatibility with CPython __cmp__:
+        // If the derived type don't override __cmp__, the
+        // *internal* super().__cmp__ should be called, not the
+        // exposed one. The difference is that the exposed __cmp__
+        // throws a TypeError if the argument is an instance of the same type.
+        if (impl==null||where_type[0]==TYPE||Py.isSubClass(TYPE,where_type[0])) {
+            return super.__cmp__(other);
+        }
+        PyObject res=impl.__get__(this,self_type).__call__(other);
+        if (res==Py.NotImplemented) {
+            return-2;
+        }
+        int c=res.asInt();
+        return c<0?-1:c>0?1:0;
+    }
+
+    public boolean __nonzero__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__nonzero__");
+        if (impl==null) {
+            impl=self_type.lookup("__len__");
+            if (impl==null)
+                return super.__nonzero__();
+        }
+        PyObject o=impl.__get__(this,self_type).__call__();
+        Class c=o.getClass();
+        if (c!=PyInteger.class&&c!=PyBoolean.class) {
+            throw Py.TypeError(String.format("__nonzero__ should return bool or int, returned %s",self_type.getName()));
+        }
+        return o.__nonzero__();
+    }
+
+    public boolean __contains__(PyObject o) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__contains__");
+        if (impl==null)
+            return super.__contains__(o);
+        return impl.__get__(this,self_type).__call__(o).__nonzero__();
+    }
+
+    public int __len__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__len__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger)
+                return((PyInteger)res).getValue();
+            throw Py.TypeError("__len__ should return a int");
+        }
+        return super.__len__();
+    }
+
+    public PyObject __iter__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iter__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        impl=self_type.lookup("__getitem__");
+        if (impl==null)
+            return super.__iter__();
+        return new PySequenceIter(this);
+    }
+
+    public PyObject __iternext__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("next");
+        if (impl!=null) {
+            try {
+                return impl.__get__(this,self_type).__call__();
+            } catch (PyException exc) {
+                if (exc.match(Py.StopIteration))
+                    return null;
+                throw exc;
+            }
+        }
+        return super.__iternext__(); // ???
+    }
+
+    public PyObject __finditem__(PyObject key) { // ???
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
+            try {
+                return impl.__get__(this,self_type).__call__(key);
+            } catch (PyException exc) {
+                if (exc.match(Py.LookupError))
+                    return null;
+                throw exc;
+            }
+        return super.__finditem__(key);
+    }
+
+    public PyObject __finditem__(int key) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
+            try {
+                return impl.__get__(this,self_type).__call__(new PyInteger(key));
+            } catch (PyException exc) {
+                if (exc.match(Py.LookupError))
+                    return null;
+                throw exc;
+            }
+        return super.__finditem__(key);
+    }
+
+    public PyObject __getitem__(PyObject key) {
+        // Same as __finditem__, without swallowing LookupErrors. This allows
+        // __getitem__ implementations written in Python to raise custom
+        // exceptions (such as subclasses of KeyError).
+        //
+        // We are forced to duplicate the code, instead of defining __finditem__
+        // in terms of __getitem__. That's because PyObject defines __getitem__
+        // in terms of __finditem__. Therefore, we would end with an infinite
+        // loop when self_type.lookup("__getitem__") returns null:
+        //
+        //  __getitem__ -> super.__getitem__ -> __finditem__ -> __getitem__
+        //
+        // By duplicating the (short) lookup and call code, we are safe, because
+        // the call chains will be:
+        //
+        // __finditem__ -> super.__finditem__
+        //
+        // __getitem__ -> super.__getitem__ -> __finditem__ -> super.__finditem__
+
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__(key);
+        return super.__getitem__(key);
+    }
+
+    public void __setitem__(PyObject key,PyObject value) { // ???
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setitem__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(key,value);
+            return;
+        }
+        super.__setitem__(key,value);
+    }
+
+    public PyObject __getslice__(PyObject start,PyObject stop,PyObject step) { // ???
+        if (step!=null) {
+            return __getitem__(new PySlice(start,stop,step));
+        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            return impl.__get__(this,self_type).__call__(indices[0],indices[1]);
+        }
+        return super.__getslice__(start,stop,step);
+    }
+
+    public void __setslice__(PyObject start,PyObject stop,PyObject step,PyObject value) {
+        if (step!=null) {
+            __setitem__(new PySlice(start,stop,step),value);
+            return;
+        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            impl.__get__(this,self_type).__call__(indices[0],indices[1],value);
+            return;
+        }
+        super.__setslice__(start,stop,step,value);
+    }
+
+    public void __delslice__(PyObject start,PyObject stop,PyObject step) {
+        if (step!=null) {
+            __delitem__(new PySlice(start,stop,step));
+            return;
+        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            impl.__get__(this,self_type).__call__(indices[0],indices[1]);
+            return;
+        }
+        super.__delslice__(start,stop,step);
+    }
+
+    public void __delitem__(PyObject key) { // ???
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delitem__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(key);
+            return;
+        }
+        super.__delitem__(key);
+    }
+
+    public PyObject __call__(PyObject args[],String keywords[]) {
+        ThreadState ts=Py.getThreadState();
+        if (ts.recursion_depth++>ts.systemState.getrecursionlimit())
+            throw Py.RuntimeError("maximum __call__ recursion depth exceeded");
+        try {
+            PyType self_type=getType();
+            PyObject impl=self_type.lookup("__call__");
+            if (impl!=null)
+                return impl.__get__(this,self_type).__call__(args,keywords);
+            return super.__call__(args,keywords);
+        } finally {
+            --ts.recursion_depth;
+        }
+    }
+
+    public PyObject __findattr_ex__(String name) {
+        return Deriveds.__findattr_ex__(this,name);
+    }
+
+    public void __setattr__(String name,PyObject value) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setattr__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(PyString.fromInterned(name),value);
+            return;
+        }
+        super.__setattr__(name,value);
+    }
+
+    public void __delattr__(String name) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delattr__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(PyString.fromInterned(name));
+            return;
+        }
+        super.__delattr__(name);
+    }
+
+    public PyObject __get__(PyObject obj,PyObject type) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__get__");
+        if (impl!=null) {
+            if (obj==null)
+                obj=Py.None;
+            if (type==null)
+                type=Py.None;
+            return impl.__get__(this,self_type).__call__(obj,type);
+        }
+        return super.__get__(obj,type);
+    }
+
+    public void __set__(PyObject obj,PyObject value) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__set__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(obj,value);
+            return;
+        }
+        super.__set__(obj,value);
+    }
+
+    public void __delete__(PyObject obj) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delete__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(obj);
+            return;
+        }
+        super.__delete__(obj);
+    }
+
+    public PyObject __pow__(PyObject other,PyObject modulo) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__pow__");
+        if (impl!=null) {
+            PyObject res;
+            if (modulo==null) {
+                res=impl.__get__(this,self_type).__call__(other);
+            } else {
+                res=impl.__get__(this,self_type).__call__(other,modulo);
+            }
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__pow__(other,modulo);
+    }
+
+    public void dispatch__init__(PyObject[]args,String[]keywords) {
+        Deriveds.dispatch__init__(this,args,keywords);
+    }
+
+    public PyObject __index__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__index__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger||res instanceof PyLong) {
+                return res;
+            }
+            throw Py.TypeError(String.format("__index__ returned non-(int,long) (type %s)",res.getType().fastGetName()));
+        }
+        return super.__index__();
+    }
+
+    public Object __tojava__(Class c) {
+        // If we are not being asked by the "default" conversion to java, then
+        // we can provide this as the result, as long as it is a instance of the
+        // specified class. Without this, derived.__tojava__(PyObject.class)
+        // would broke. (And that's not pure speculation: PyReflectedFunction's
+        // ReflectedArgs asks for things like that).
+        if ((c!=Object.class)&&(c!=Serializable.class)&&(c.isInstance(this))) {
+            return this;
+        }
+        // Otherwise, we call the derived __tojava__, if it exists:
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__tojava__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__(Py.java2py(c)).__tojava__(Object.class);
+        return super.__tojava__(c);
+    }
+
+    public Object __coerce_ex__(PyObject o) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__coerce__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(o);
+            if (res==Py.NotImplemented)
+                return Py.None;
+            if (!(res instanceof PyTuple))
+                throw Py.TypeError("__coerce__ didn't return a 2-tuple");
+            return((PyTuple)res).getArray();
+        }
+        return super.__coerce_ex__(o);
+    }
+
+    public String toString() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__repr__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (!(res instanceof PyString))
+                throw Py.TypeError("__repr__ returned non-string (type "+res.getType().fastGetName()+")");
+            return((PyString)res).toString();
+        }
+        return super.toString();
+    }
+
+}
diff --git a/src/org/python/modules/struct.java b/src/org/python/modules/struct.java
--- a/src/org/python/modules/struct.java
+++ b/src/org/python/modules/struct.java
@@ -19,6 +19,7 @@
 import org.python.core.PyTuple;
 
 import java.math.BigInteger;
+import org.python.core.ClassDictInit;
 import org.python.core.PyArray;
 
 /**
@@ -252,7 +253,7 @@
  * @author Finn Bock, bckfnn at pipmail.dknet.dk
  * @version struct.java,v 1.6 1999/04/17 12:04:34 fb Exp
  */
-public class struct {
+public class struct implements ClassDictInit {
 
     /**
      * Exception raised on various occasions; argument is a
@@ -1148,8 +1149,9 @@
         return dict;
     }
     
-    public static PyStruct Struct(PyObject[] args, String[] keywords) {
-        return new PyStruct(args, keywords);
+    public static void classDictInit(PyObject dict) {
+        dict.__setitem__("Struct", PyStruct.TYPE);
     }
+
 }
 
diff --git a/src/templates/PyStruct.derived b/src/templates/PyStruct.derived
new file mode 100644
--- /dev/null
+++ b/src/templates/PyStruct.derived
@@ -0,0 +1,4 @@
+base_class: PyStruct
+want_dict: true
+ctr: PyString format
+incl: object

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 00:09:26 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 00:09:26 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Add_mapping_for_PyStruct=2E?=
 =?utf-8?q?derived_to_org=2Epython=2Emodules=2EPyStructDerived?=
Message-ID: <3gQQgV3hr4z7Ljl@mail.python.org>

http://hg.python.org/jython/rev/00525e068a0a
changeset:   7226:00525e068a0a
user:        Indra Talip <indra.talip at gmail.com>
date:        Mon Mar 24 20:46:50 2014 +1100
summary:
  Add mapping for PyStruct.derived to org.python.modules.PyStructDerived

files:
  src/templates/mappings |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/src/templates/mappings b/src/templates/mappings
--- a/src/templates/mappings
+++ b/src/templates/mappings
@@ -11,6 +11,7 @@
 PyBZ2File.derived:org.python.modules.bz2.PyBZ2FileDerived
 PyBZ2Compressor.derived:org.python.modules.bz2.PyBZ2CompressorDerived
 PyBZ2Decompressor.derived:org.python.modules.bz2.PyBZ2DecompressorDerived
+PyStruct.derived:org.python.modules.PyStructDerived
 array.derived:org.python.core.PyArrayDerived
 bytearray.derived:org.python.core.PyByteArrayDerived
 classmethod.derived:org.python.core.PyClassMethodDerived

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 00:09:28 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 00:09:28 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_Merged_pull_request?=
Message-ID: <3gQQgX1n5Lz7Ljm@mail.python.org>

http://hg.python.org/jython/rev/0431ab4d1e44
changeset:   7227:0431ab4d1e44
parent:      7224:ac645c8c4208
parent:      7226:00525e068a0a
user:        Jim Baker <jim.baker at rackspace.com>
date:        Fri May 09 15:42:27 2014 -0600
summary:
  Merged pull request https://bitbucket.org/jython/jython/pull-request/28/fix-the-struct-module-so-that-structstruct

files:
  src/org/python/modules/PyStruct.java        |    41 +-
  src/org/python/modules/PyStructDerived.java |  1137 ++++++++++
  src/org/python/modules/struct.java          |     8 +-
  src/templates/PyStruct.derived              |     4 +
  src/templates/mappings                      |     1 +
  5 files changed, 1169 insertions(+), 22 deletions(-)


diff --git a/src/org/python/modules/PyStruct.java b/src/org/python/modules/PyStruct.java
--- a/src/org/python/modules/PyStruct.java
+++ b/src/org/python/modules/PyStruct.java
@@ -1,5 +1,6 @@
 package org.python.modules;
 
+import org.python.core.ArgParser;
 import org.python.core.Py;
 import org.python.core.PyArray;
 import org.python.core.PyNewWrapper;
@@ -12,7 +13,7 @@
 import org.python.expose.ExposedNew;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "Struct", base = PyObject.class)
+ at ExposedType(name = "struct.Struct", base = PyObject.class)
 public class PyStruct extends PyObject {
     public static final PyType TYPE = PyType.fromClass(PyStruct.class);
     
@@ -30,27 +31,29 @@
         return TYPE;
     }
 
-    public PyStruct(PyObject[] args, String[] keywords) {
-        final int nargs = args.length;
-        if (keywords.length > 0 ) {
-            for (String keyword : keywords) {
-                if (!keyword.equals("format")) {
-                    throw Py.TypeError("'" + keyword + "' is an invalid keyword argument for this function");
-                }
-            }
-        }
-        else if (nargs < 1 || nargs > 1) {
-            throw Py.TypeError("Struct() takes exactly 1 argument (" + nargs + " given)");
-        }
-        format = args[0].toString();
-        format_def = struct.whichtable(format);
-        size = struct.calcsize(format, format_def);
+    public PyStruct(PyString format) {
+        this(TYPE, format);
     }
-    
+
+    public PyStruct(PyType type, PyString format) {
+        super(type);
+        this.format = format.toString();
+        this.format_def = struct.whichtable(this.format);
+        this.size = struct.calcsize(this.format, this.format_def);
+    }
+
     @ExposedNew
     final static PyObject Struct___new__ (PyNewWrapper new_, boolean init,
             PyType subtype, PyObject[] args, String[] keywords) {
-         return new PyStruct(args, keywords);
+        ArgParser ap = new ArgParser("Struct", args, keywords, new String[] {"format"}, 1);
+
+        PyObject format = ap.getPyObject(0);
+        if (!(format instanceof PyString)) {
+            throw Py.TypeError("coercing to Unicode: need string, '"
+                    + format.getType().fastGetName() + "' type found");
+        }
+
+        return new PyStruct(TYPE, (PyString) format);
     }
 
     @ExposedMethod
@@ -85,4 +88,4 @@
             throw struct.StructError("unpack_from str size does not match format");
         return struct.unpack(format_def, size, format, new struct.ByteStream(s, offset));
     }
-}
\ No newline at end of file
+}
diff --git a/src/org/python/modules/PyStructDerived.java b/src/org/python/modules/PyStructDerived.java
new file mode 100644
--- /dev/null
+++ b/src/org/python/modules/PyStructDerived.java
@@ -0,0 +1,1137 @@
+/* Generated file, do not modify.  See jython/src/templates/gderived.py. */
+package org.python.modules;
+
+import java.io.Serializable;
+import org.python.core.*;
+
+public class PyStructDerived extends PyStruct implements Slotted {
+
+    public PyObject getSlot(int index) {
+        return slots[index];
+    }
+
+    public void setSlot(int index,PyObject value) {
+        slots[index]=value;
+    }
+
+    private PyObject[]slots;
+
+    private PyObject dict;
+
+    public PyObject fastGetDict() {
+        return dict;
+    }
+
+    public PyObject getDict() {
+        return dict;
+    }
+
+    public void setDict(PyObject newDict) {
+        if (newDict instanceof PyStringMap||newDict instanceof PyDictionary) {
+            dict=newDict;
+        } else {
+            throw Py.TypeError("__dict__ must be set to a Dictionary "+newDict.getClass().getName());
+        }
+    }
+
+    public void delDict() {
+        // deleting an object's instance dict makes it grow a new one
+        dict=new PyStringMap();
+    }
+
+    public PyStructDerived(PyType subtype,PyString format) {
+        super(subtype,format);
+        slots=new PyObject[subtype.getNumSlots()];
+        dict=subtype.instDict();
+    }
+
+    public PyString __str__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__str__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__str__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__str__();
+    }
+
+    public PyString __repr__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__repr__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__repr__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__repr__();
+    }
+
+    public PyString __hex__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__hex__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__hex__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__hex__();
+    }
+
+    public PyString __oct__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__oct__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__oct__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__oct__();
+    }
+
+    public PyFloat __float__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__float__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyFloat)
+                return(PyFloat)res;
+            throw Py.TypeError("__float__"+" returned non-"+"float"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__float__();
+    }
+
+    public PyComplex __complex__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__complex__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyComplex)
+                return(PyComplex)res;
+            throw Py.TypeError("__complex__"+" returned non-"+"complex"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__complex__();
+    }
+
+    public PyObject __pos__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__pos__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__pos__();
+    }
+
+    public PyObject __neg__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__neg__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__neg__();
+    }
+
+    public PyObject __abs__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__abs__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__abs__();
+    }
+
+    public PyObject __invert__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__invert__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__invert__();
+    }
+
+    public PyObject __reduce__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__reduce__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__reduce__();
+    }
+
+    public PyObject __dir__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__dir__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        return super.__dir__();
+    }
+
+    public PyObject __add__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__add__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__add__(other);
+    }
+
+    public PyObject __radd__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__radd__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__radd__(other);
+    }
+
+    public PyObject __sub__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__sub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__sub__(other);
+    }
+
+    public PyObject __rsub__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rsub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rsub__(other);
+    }
+
+    public PyObject __mul__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__mul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__mul__(other);
+    }
+
+    public PyObject __rmul__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rmul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rmul__(other);
+    }
+
+    public PyObject __div__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__div__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__div__(other);
+    }
+
+    public PyObject __rdiv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rdiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rdiv__(other);
+    }
+
+    public PyObject __floordiv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__floordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__floordiv__(other);
+    }
+
+    public PyObject __rfloordiv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rfloordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rfloordiv__(other);
+    }
+
+    public PyObject __truediv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__truediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__truediv__(other);
+    }
+
+    public PyObject __rtruediv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rtruediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rtruediv__(other);
+    }
+
+    public PyObject __mod__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__mod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__mod__(other);
+    }
+
+    public PyObject __rmod__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rmod__(other);
+    }
+
+    public PyObject __divmod__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__divmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__divmod__(other);
+    }
+
+    public PyObject __rdivmod__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rdivmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rdivmod__(other);
+    }
+
+    public PyObject __rpow__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rpow__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rpow__(other);
+    }
+
+    public PyObject __lshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__lshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__lshift__(other);
+    }
+
+    public PyObject __rlshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rlshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rlshift__(other);
+    }
+
+    public PyObject __rshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rshift__(other);
+    }
+
+    public PyObject __rrshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rrshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rrshift__(other);
+    }
+
+    public PyObject __and__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__and__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__and__(other);
+    }
+
+    public PyObject __rand__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rand__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rand__(other);
+    }
+
+    public PyObject __or__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__or__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__or__(other);
+    }
+
+    public PyObject __ror__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ror__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ror__(other);
+    }
+
+    public PyObject __xor__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__xor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__xor__(other);
+    }
+
+    public PyObject __rxor__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rxor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__rxor__(other);
+    }
+
+    public PyObject __lt__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__lt__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__lt__(other);
+    }
+
+    public PyObject __le__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__le__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__le__(other);
+    }
+
+    public PyObject __gt__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__gt__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__gt__(other);
+    }
+
+    public PyObject __ge__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ge__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ge__(other);
+    }
+
+    public PyObject __eq__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__eq__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__eq__(other);
+    }
+
+    public PyObject __ne__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ne__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ne__(other);
+    }
+
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
+    public PyObject __iadd__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iadd__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__iadd__(other);
+    }
+
+    public PyObject __isub__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__isub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__isub__(other);
+    }
+
+    public PyObject __imul__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__imul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__imul__(other);
+    }
+
+    public PyObject __idiv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__idiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__idiv__(other);
+    }
+
+    public PyObject __ifloordiv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ifloordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ifloordiv__(other);
+    }
+
+    public PyObject __itruediv__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__itruediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__itruediv__(other);
+    }
+
+    public PyObject __imod__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__imod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__imod__(other);
+    }
+
+    public PyObject __ipow__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ipow__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ipow__(other);
+    }
+
+    public PyObject __ilshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ilshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ilshift__(other);
+    }
+
+    public PyObject __irshift__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__irshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__irshift__(other);
+    }
+
+    public PyObject __iand__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iand__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__iand__(other);
+    }
+
+    public PyObject __ior__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ior__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ior__(other);
+    }
+
+    public PyObject __ixor__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ixor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__ixor__(other);
+    }
+
+    public PyObject __int__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__int__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger||res instanceof PyLong)
+                return res;
+            throw Py.TypeError("__int__"+" should return an integer");
+        }
+        return super.__int__();
+    }
+
+    public PyObject __long__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__long__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyLong||res instanceof PyInteger)
+                return res;
+            throw Py.TypeError("__long__"+" returned non-"+"long"+" (type "+res.getType().fastGetName()+")");
+        }
+        return super.__long__();
+    }
+
+    public int hashCode() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__hash__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger) {
+                return((PyInteger)res).getValue();
+            } else
+                if (res instanceof PyLong) {
+                    return((PyLong)res).getValue().intValue();
+                }
+            throw Py.TypeError("__hash__ should return a int");
+        }
+        if (self_type.lookup("__eq__")!=null||self_type.lookup("__cmp__")!=null) {
+            throw Py.TypeError(String.format("unhashable type: '%.200s'",getType().fastGetName()));
+        }
+        return super.hashCode();
+    }
+
+    public PyUnicode __unicode__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__unicode__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyUnicode)
+                return(PyUnicode)res;
+            if (res instanceof PyString)
+                return new PyUnicode((PyString)res);
+            throw Py.TypeError("__unicode__"+" should return a "+"unicode");
+        }
+        return super.__unicode__();
+    }
+
+    public int __cmp__(PyObject other) {
+        PyType self_type=getType();
+        PyObject[]where_type=new PyObject[1];
+        PyObject impl=self_type.lookup_where("__cmp__",where_type);
+        // Full Compatibility with CPython __cmp__:
+        // If the derived type don't override __cmp__, the
+        // *internal* super().__cmp__ should be called, not the
+        // exposed one. The difference is that the exposed __cmp__
+        // throws a TypeError if the argument is an instance of the same type.
+        if (impl==null||where_type[0]==TYPE||Py.isSubClass(TYPE,where_type[0])) {
+            return super.__cmp__(other);
+        }
+        PyObject res=impl.__get__(this,self_type).__call__(other);
+        if (res==Py.NotImplemented) {
+            return-2;
+        }
+        int c=res.asInt();
+        return c<0?-1:c>0?1:0;
+    }
+
+    public boolean __nonzero__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__nonzero__");
+        if (impl==null) {
+            impl=self_type.lookup("__len__");
+            if (impl==null)
+                return super.__nonzero__();
+        }
+        PyObject o=impl.__get__(this,self_type).__call__();
+        Class c=o.getClass();
+        if (c!=PyInteger.class&&c!=PyBoolean.class) {
+            throw Py.TypeError(String.format("__nonzero__ should return bool or int, returned %s",self_type.getName()));
+        }
+        return o.__nonzero__();
+    }
+
+    public boolean __contains__(PyObject o) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__contains__");
+        if (impl==null)
+            return super.__contains__(o);
+        return impl.__get__(this,self_type).__call__(o).__nonzero__();
+    }
+
+    public int __len__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__len__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger)
+                return((PyInteger)res).getValue();
+            throw Py.TypeError("__len__ should return a int");
+        }
+        return super.__len__();
+    }
+
+    public PyObject __iter__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iter__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        impl=self_type.lookup("__getitem__");
+        if (impl==null)
+            return super.__iter__();
+        return new PySequenceIter(this);
+    }
+
+    public PyObject __iternext__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("next");
+        if (impl!=null) {
+            try {
+                return impl.__get__(this,self_type).__call__();
+            } catch (PyException exc) {
+                if (exc.match(Py.StopIteration))
+                    return null;
+                throw exc;
+            }
+        }
+        return super.__iternext__(); // ???
+    }
+
+    public PyObject __finditem__(PyObject key) { // ???
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
+            try {
+                return impl.__get__(this,self_type).__call__(key);
+            } catch (PyException exc) {
+                if (exc.match(Py.LookupError))
+                    return null;
+                throw exc;
+            }
+        return super.__finditem__(key);
+    }
+
+    public PyObject __finditem__(int key) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
+            try {
+                return impl.__get__(this,self_type).__call__(new PyInteger(key));
+            } catch (PyException exc) {
+                if (exc.match(Py.LookupError))
+                    return null;
+                throw exc;
+            }
+        return super.__finditem__(key);
+    }
+
+    public PyObject __getitem__(PyObject key) {
+        // Same as __finditem__, without swallowing LookupErrors. This allows
+        // __getitem__ implementations written in Python to raise custom
+        // exceptions (such as subclasses of KeyError).
+        //
+        // We are forced to duplicate the code, instead of defining __finditem__
+        // in terms of __getitem__. That's because PyObject defines __getitem__
+        // in terms of __finditem__. Therefore, we would end with an infinite
+        // loop when self_type.lookup("__getitem__") returns null:
+        //
+        //  __getitem__ -> super.__getitem__ -> __finditem__ -> __getitem__
+        //
+        // By duplicating the (short) lookup and call code, we are safe, because
+        // the call chains will be:
+        //
+        // __finditem__ -> super.__finditem__
+        //
+        // __getitem__ -> super.__getitem__ -> __finditem__ -> super.__finditem__
+
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__(key);
+        return super.__getitem__(key);
+    }
+
+    public void __setitem__(PyObject key,PyObject value) { // ???
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setitem__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(key,value);
+            return;
+        }
+        super.__setitem__(key,value);
+    }
+
+    public PyObject __getslice__(PyObject start,PyObject stop,PyObject step) { // ???
+        if (step!=null) {
+            return __getitem__(new PySlice(start,stop,step));
+        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            return impl.__get__(this,self_type).__call__(indices[0],indices[1]);
+        }
+        return super.__getslice__(start,stop,step);
+    }
+
+    public void __setslice__(PyObject start,PyObject stop,PyObject step,PyObject value) {
+        if (step!=null) {
+            __setitem__(new PySlice(start,stop,step),value);
+            return;
+        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            impl.__get__(this,self_type).__call__(indices[0],indices[1],value);
+            return;
+        }
+        super.__setslice__(start,stop,step,value);
+    }
+
+    public void __delslice__(PyObject start,PyObject stop,PyObject step) {
+        if (step!=null) {
+            __delitem__(new PySlice(start,stop,step));
+            return;
+        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            impl.__get__(this,self_type).__call__(indices[0],indices[1]);
+            return;
+        }
+        super.__delslice__(start,stop,step);
+    }
+
+    public void __delitem__(PyObject key) { // ???
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delitem__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(key);
+            return;
+        }
+        super.__delitem__(key);
+    }
+
+    public PyObject __call__(PyObject args[],String keywords[]) {
+        ThreadState ts=Py.getThreadState();
+        if (ts.recursion_depth++>ts.systemState.getrecursionlimit())
+            throw Py.RuntimeError("maximum __call__ recursion depth exceeded");
+        try {
+            PyType self_type=getType();
+            PyObject impl=self_type.lookup("__call__");
+            if (impl!=null)
+                return impl.__get__(this,self_type).__call__(args,keywords);
+            return super.__call__(args,keywords);
+        } finally {
+            --ts.recursion_depth;
+        }
+    }
+
+    public PyObject __findattr_ex__(String name) {
+        return Deriveds.__findattr_ex__(this,name);
+    }
+
+    public void __setattr__(String name,PyObject value) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setattr__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(PyString.fromInterned(name),value);
+            return;
+        }
+        super.__setattr__(name,value);
+    }
+
+    public void __delattr__(String name) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delattr__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(PyString.fromInterned(name));
+            return;
+        }
+        super.__delattr__(name);
+    }
+
+    public PyObject __get__(PyObject obj,PyObject type) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__get__");
+        if (impl!=null) {
+            if (obj==null)
+                obj=Py.None;
+            if (type==null)
+                type=Py.None;
+            return impl.__get__(this,self_type).__call__(obj,type);
+        }
+        return super.__get__(obj,type);
+    }
+
+    public void __set__(PyObject obj,PyObject value) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__set__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(obj,value);
+            return;
+        }
+        super.__set__(obj,value);
+    }
+
+    public void __delete__(PyObject obj) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delete__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(obj);
+            return;
+        }
+        super.__delete__(obj);
+    }
+
+    public PyObject __pow__(PyObject other,PyObject modulo) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__pow__");
+        if (impl!=null) {
+            PyObject res;
+            if (modulo==null) {
+                res=impl.__get__(this,self_type).__call__(other);
+            } else {
+                res=impl.__get__(this,self_type).__call__(other,modulo);
+            }
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__pow__(other,modulo);
+    }
+
+    public void dispatch__init__(PyObject[]args,String[]keywords) {
+        Deriveds.dispatch__init__(this,args,keywords);
+    }
+
+    public PyObject __index__() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__index__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger||res instanceof PyLong) {
+                return res;
+            }
+            throw Py.TypeError(String.format("__index__ returned non-(int,long) (type %s)",res.getType().fastGetName()));
+        }
+        return super.__index__();
+    }
+
+    public Object __tojava__(Class c) {
+        // If we are not being asked by the "default" conversion to java, then
+        // we can provide this as the result, as long as it is a instance of the
+        // specified class. Without this, derived.__tojava__(PyObject.class)
+        // would broke. (And that's not pure speculation: PyReflectedFunction's
+        // ReflectedArgs asks for things like that).
+        if ((c!=Object.class)&&(c!=Serializable.class)&&(c.isInstance(this))) {
+            return this;
+        }
+        // Otherwise, we call the derived __tojava__, if it exists:
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__tojava__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__(Py.java2py(c)).__tojava__(Object.class);
+        return super.__tojava__(c);
+    }
+
+    public Object __coerce_ex__(PyObject o) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__coerce__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(o);
+            if (res==Py.NotImplemented)
+                return Py.None;
+            if (!(res instanceof PyTuple))
+                throw Py.TypeError("__coerce__ didn't return a 2-tuple");
+            return((PyTuple)res).getArray();
+        }
+        return super.__coerce_ex__(o);
+    }
+
+    public String toString() {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__repr__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (!(res instanceof PyString))
+                throw Py.TypeError("__repr__ returned non-string (type "+res.getType().fastGetName()+")");
+            return((PyString)res).toString();
+        }
+        return super.toString();
+    }
+
+}
diff --git a/src/org/python/modules/struct.java b/src/org/python/modules/struct.java
--- a/src/org/python/modules/struct.java
+++ b/src/org/python/modules/struct.java
@@ -19,6 +19,7 @@
 import org.python.core.PyTuple;
 
 import java.math.BigInteger;
+import org.python.core.ClassDictInit;
 import org.python.core.PyArray;
 
 /**
@@ -252,7 +253,7 @@
  * @author Finn Bock, bckfnn at pipmail.dknet.dk
  * @version struct.java,v 1.6 1999/04/17 12:04:34 fb Exp
  */
-public class struct {
+public class struct implements ClassDictInit {
 
     /**
      * Exception raised on various occasions; argument is a
@@ -1148,8 +1149,9 @@
         return dict;
     }
     
-    public static PyStruct Struct(PyObject[] args, String[] keywords) {
-        return new PyStruct(args, keywords);
+    public static void classDictInit(PyObject dict) {
+        dict.__setitem__("Struct", PyStruct.TYPE);
     }
+
 }
 
diff --git a/src/templates/PyStruct.derived b/src/templates/PyStruct.derived
new file mode 100644
--- /dev/null
+++ b/src/templates/PyStruct.derived
@@ -0,0 +1,4 @@
+base_class: PyStruct
+want_dict: true
+ctr: PyString format
+incl: object
diff --git a/src/templates/mappings b/src/templates/mappings
--- a/src/templates/mappings
+++ b/src/templates/mappings
@@ -11,6 +11,7 @@
 PyBZ2File.derived:org.python.modules.bz2.PyBZ2FileDerived
 PyBZ2Compressor.derived:org.python.modules.bz2.PyBZ2CompressorDerived
 PyBZ2Decompressor.derived:org.python.modules.bz2.PyBZ2DecompressorDerived
+PyStruct.derived:org.python.modules.PyStructDerived
 array.derived:org.python.core.PyArrayDerived
 bytearray.derived:org.python.core.PyByteArrayDerived
 classmethod.derived:org.python.core.PyClassMethodDerived

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 00:30:44 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 00:30:44 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Refactor_decompressing_a_bz?=
 =?utf-8?q?2_file_to_stream_the_decompressed_data=2E?=
Message-ID: <3gQR844Dxhz7LjW@mail.python.org>

http://hg.python.org/jython/rev/1241914a8648
changeset:   7228:1241914a8648
parent:      7174:b2890af7a5e8
user:        Indra Talip <indra.talip at gmail.com>
date:        Fri Jan 03 08:46:47 2014 +1100
summary:
  Refactor decompressing a bz2 file to stream the decompressed data.
implemented decompressing bz2 files in terms of pep 3116? and using streaming
reads instead of fully decompressing in memory.

files:
  src/org/python/modules/bz2/PyBZ2File.java |  302 ++++-----
  1 files changed, 144 insertions(+), 158 deletions(-)


diff --git a/src/org/python/modules/bz2/PyBZ2File.java b/src/org/python/modules/bz2/PyBZ2File.java
--- a/src/org/python/modules/bz2/PyBZ2File.java
+++ b/src/org/python/modules/bz2/PyBZ2File.java
@@ -1,19 +1,21 @@
 package org.python.modules.bz2;
 
 import java.io.BufferedInputStream;
-import java.io.ByteArrayOutputStream;
 import java.io.File;
 import java.io.FileInputStream;
+import java.io.FileNotFoundException;
 import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
 import java.util.ArrayList;
-import java.util.Arrays;
 import java.util.Iterator;
 
 import org.apache.commons.compress.compressors.bzip2.BZip2CompressorInputStream;
 import org.apache.commons.compress.compressors.bzip2.BZip2CompressorOutputStream;
 import org.python.core.ArgParser;
 import org.python.core.Py;
+import org.python.core.PyException;
 import org.python.core.PyIterator;
 import org.python.core.PyList;
 import org.python.core.PyLong;
@@ -21,8 +23,13 @@
 import org.python.core.PyObject;
 import org.python.core.PySequence;
 import org.python.core.PyString;
-import org.python.core.PyTuple;
 import org.python.core.PyType;
+import org.python.core.io.BinaryIOWrapper;
+import org.python.core.io.BufferedReader;
+import org.python.core.io.IOBase;
+import org.python.core.io.StreamIO;
+import org.python.core.io.TextIOBase;
+import org.python.core.io.UniversalIOWrapper;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
 import org.python.expose.ExposedNew;
@@ -32,22 +39,22 @@
 public class PyBZ2File extends PyObject {
 
     public static final PyType TYPE = PyType.fromClass(PyBZ2File.class);
+    private int buffering;
 
-    @ExposedGet
-    public PyObject newlines = null;
+    @ExposedGet(name = "newlines")
+    public PyObject PyBZ2File_newlines() {
+        if (buffer != null) {
+            return buffer.getNewlines();
+        } else {
+            return Py.None;
+        }
+    }
 
-    private byte[] fileData = null;
-    private int offset = 0;
+    private TextIOBase buffer;
     private String fileName = null;
     private String fileMode = "";
     private boolean inIterMode = false;
     private boolean inUniversalNewlineMode = false;
-    private final ArrayList<String> validNewlines = new ArrayList<String>();
-    {
-        validNewlines.add("\n");
-        validNewlines.add("\r");
-        validNewlines.add("\r\n");
-    }
 
     private BZip2CompressorOutputStream writeStream = null;
 
@@ -86,9 +93,9 @@
     private void BZ2File___init__(PyString inFileName, String mode,
             int buffering, int compresslevel) {
         try {
-
             fileName = inFileName.asString();
             fileMode = mode;
+            this.buffering = buffering;
 
             // check universal newline mode
             if (mode.contains("U")) {
@@ -104,30 +111,34 @@
                 writeStream = new BZip2CompressorOutputStream(
                         new FileOutputStream(fileName), compresslevel);
             } else {
-                FileInputStream fin = new FileInputStream(fileName);
-                BufferedInputStream bin = new BufferedInputStream(fin);
-                BZip2CompressorInputStream bZin = new BZip2CompressorInputStream(
-                        bin);
-
-                ByteArrayOutputStream buffer = new ByteArrayOutputStream();
-
-                final byte[] buf = new byte[100];
-                int n = 0;
-                while (-1 != (n = bZin.read(buf))) {
-                    buffer.write(buf, 0, n);
-                }
-                fileData = buffer.toByteArray();
-
-                buffer.close();
-                bZin.close();
-                bin.close();
-                fin.close();
+                makeReadBuffer();
             }
         } catch (IOException e) {
             throw Py.IOError("File " + fileName + " not found,");
         }
     }
 
+    private void makeReadBuffer() {
+        try {
+            FileInputStream fin = new FileInputStream(fileName);
+            BufferedInputStream bin = new BufferedInputStream(fin);
+            BZip2CompressorInputStream bZin = new BZip2CompressorInputStream(
+                    bin, true);
+            BufferedReader bufferedReader = new BufferedReader(
+                    new SkippableStreamIO(bZin, true), buffering);
+
+            if (inUniversalNewlineMode) {
+                buffer = new UniversalIOWrapper(bufferedReader);
+            } else {
+                buffer = new BinaryIOWrapper(bufferedReader);
+            }
+        } catch (FileNotFoundException fileNotFoundException) {
+            throw Py.IOError(fileNotFoundException);
+        } catch (IOException io) {
+            throw Py.IOError(io);
+        }
+    }
+
     @ExposedMethod
     public void __del__() {
         BZ2File_close();
@@ -135,8 +146,6 @@
 
     @ExposedMethod
     public void BZ2File_close() {
-        fileData = null;
-
         if (writeStream != null) {
             BZ2File_flush();
             try {
@@ -146,6 +155,9 @@
                 throw Py.IOError(e.getMessage());
             }
         }
+        if (buffer != null) {
+            buffer.close();
+        }
     }
 
     private void BZ2File_flush() {
@@ -158,16 +170,6 @@
         }
     }
 
-    private byte[] peek() {
-
-        byte[] buf = new byte[1];
-        if (fileData.length > offset) {
-            buf[0] = fileData[offset + 1];
-        }
-
-        return buf;
-    }
-
     @ExposedMethod
     public PyObject BZ2File_read(PyObject[] args, String[] kwds) {
         checkInIterMode();
@@ -176,59 +178,17 @@
                 new String[] { "size" }, 0);
 
         int size = ap.getInt(0, -1);
+        final String data = buffer.read(size);
 
-        byte[] buf = _BZ2File_read(size);
-
-        return new PyString(new String(buf));
-    }
-
-    private byte[] _BZ2File_read(int size) {
-        byte[] buf = null;
-        if (size == 0) {
-            return new byte[0];
-        } else if (size > 0) {
-            buf = new byte[size];
-        } else {
-            buf = new byte[fileData.length - offset];
-        }
-
-        int readbytes = 0;
-        for (int i = offset, j = 0; i < fileData.length && j < buf.length; i++, j++) {
-            buf[j] = fileData[i];
-
-            String possibleNewline = new String(new byte[] { buf[j] });
-            if (possibleNewline.equals("\r")) { // handle CRLF
-                buf[j] = '\n';
-                if (fileData[i + 1] == '\n') { // check if next character part
-                                               // of newline
-                    possibleNewline = possibleNewline
-                            + new String(new byte[] { fileData[i + 1] });
-                    buf = Arrays.copyOf(buf, buf.length - 1); // adjust buffer
-                                                              // size
-                    i++;
-                }
-            }
-            if (validNewlines.contains(possibleNewline)) {
-                addNewlineMarker(possibleNewline);
-            }
-
-            offset++;
-            readbytes++;
-        }
-
-        if (readbytes == 0) {
-            return new byte[0];
-        }
-
-        return buf;
-
+        return new PyString(data);
     }
 
     @ExposedMethod
     public PyObject BZ2File_next(PyObject[] args, String[] kwds) {
-        if (fileData == null) {
+        if (buffer == null || buffer.closed()) {
             throw Py.ValueError("Cannot call next() on closed file");
         }
+
         inIterMode = true;
         return null;
     }
@@ -242,57 +202,7 @@
 
         int size = ap.getInt(0, -1);
 
-        StringBuilder line = new StringBuilder();
-
-        byte[] buf = null;
-        int readSize = 0;
-        while ((buf = _BZ2File_read(1)).length > 0) {
-            line.append(new String(buf));
-            // handle newlines
-            boolean mustBreak = false;
-            if (inUniversalNewlineMode) {
-                if ((char) buf[0] == '\r') {
-                    if (peek()[0] == '\n') {
-                        buf = _BZ2File_read(1);
-                        mustBreak = true;
-                    }
-                    line.replace(line.length() - 1, line.length(), new String(
-                            "\n"));
-                    mustBreak = true;
-                } else if ((char) buf[0] == '\n'
-                        || (size > -1 && (readSize >= size))) {
-                    mustBreak = true;
-                }
-
-            } else {
-                if ((char) buf[0] == '\n' || (size > -1 && (readSize >= size))) {
-                    mustBreak = true;
-                }
-            }
-
-            if (mustBreak) {
-                break;
-            }
-        }
-
-        return new PyString(line.toString());
-    }
-
-    private void addNewlineMarker(String newline) {
-        if (newlines == null) {
-            newlines = new PyString(newline);
-        } else {
-            if (newlines instanceof PyString) {
-                if (!newlines.equals(new PyString(newline))) {
-                    newlines = new PyTuple(newlines, new PyString(newline));
-                }
-            } else {
-                if (!newlines.__contains__(new PyString(newline))) {
-                    newlines = newlines.__add__(new PyTuple(new PyString(
-                            newline)));
-                }
-            }
-        }
+        return new PyString(buffer.readline(size));
     }
 
     @ExposedMethod
@@ -300,10 +210,9 @@
         checkInIterMode();
 
         // make sure file data valid
-        if (fileData == null) {
+        if (buffer == null || buffer.closed()) {
             throw Py.ValueError("Cannot call readlines() on a closed file");
         }
-
         PyList lineList = new PyList();
 
         PyString line = null;
@@ -335,38 +244,65 @@
         int newOffset = ap.getInt(0);
         int whence = ap.getInt(1, 0);
 
+        if (fileMode.contains("w")) {
+            Py.IOError("seek works only while reading");
+        }
+
         // normalise offset
-        int finalOffset = 0;
+        long currentPos = buffer.tell();
+
+        long finalOffset = 0;
         switch (whence) {
         case 0: // offset from start of file
-            if (newOffset > fileData.length) {
-                finalOffset = fileData.length;
-            } else {
-                finalOffset = newOffset;
-            }
+            finalOffset = newOffset;
             break;
         case 1: // move relative to current position
-            finalOffset = offset + newOffset;
+            finalOffset = currentPos + newOffset;
+
             break;
         case 2: // move relative to end of file
-            finalOffset = fileData.length + newOffset;
+            long fileSize = currentPos;
+
+            // in order to seek from the end of the stream we need to fully read
+            // the decompressed stream to get the size
+            for (;;) {
+                final String data = buffer.read(IOBase.DEFAULT_BUFFER_SIZE);
+                if (data.isEmpty()) {
+                    break;
+                }
+                fileSize += data.length();
+            }
+
+            finalOffset = fileSize + newOffset;
+
+            // close and reset the buffer
+            buffer.close();
+            makeReadBuffer();
+
+            break;
         }
 
         if (finalOffset < 0) {
             finalOffset = 0;
-        } else {
-            if (finalOffset > fileData.length) {
-                finalOffset = fileData.length;
-            }
+        }
+
+        // can't seek backwards so close and reopen the stream at the start
+        if (whence != 2 && finalOffset < currentPos) {
+            buffer.close();
+            makeReadBuffer();
         }
 
         // seek operation
-        offset = finalOffset;
+        buffer.seek(finalOffset, 0);
     }
 
     @ExposedMethod
     public PyLong BZ2File_tell() {
-        return new PyLong(offset);
+        if (buffer == null) {
+            return Py.newLong(0);
+        } else {
+            return Py.newLong(buffer.tell());
+        }
     }
 
     @ExposedMethod
@@ -447,7 +383,7 @@
                 throw Py.ValueError("Stream closed");
             }
         } else if (fileMode.contains("r")) {
-            if (fileData == null) {
+            if (buffer == null || buffer.closed()) {
                 throw Py.ValueError("Stream closed");
             }
         }
@@ -461,4 +397,54 @@
         BZ2File_close();
         return false;
     }
+
+    private static class SkippableStreamIO extends StreamIO {
+        private long position = 0;
+
+        public SkippableStreamIO(InputStream inputStream, boolean closefd) {
+            super(inputStream, closefd);
+        }
+
+        @Override
+        public int readinto(ByteBuffer buf) {
+            int bytesRead = 0;
+            try {
+                bytesRead = super.readinto(buf);
+            } catch (PyException pyex) {
+                // translate errors on read of decompressed stream to EOFError
+                throw Py.EOFError(pyex.value.asStringOrNull());
+            }
+
+            position += bytesRead;
+            return bytesRead;
+        }
+
+        @Override
+        public long tell() {
+            return position;
+        }
+
+        @Override
+        public long seek(long offset, int whence) {
+            long skipBytes = offset - position;
+            if (whence != 0 || skipBytes < 0) {
+                throw Py.IOError("can only seek forward");
+            }
+
+            if (skipBytes == 0) {
+                return position;
+            } else {
+                long skipped = 0;
+                try {
+                    skipped = asInputStream().skip(skipBytes);
+                } catch (IOException ex) {
+                    throw Py.IOError(ex);
+                }
+                long newPosition = position + skipped;
+                position = newPosition;
+
+                return newPosition;
+            }
+        }
+    }
 }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 00:30:46 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 00:30:46 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Defer_the_creation_of_BZip2?=
 =?utf-8?q?CompressorInputStream_until_a_read_occurs?=
Message-ID: <3gQR8602Kpz7Ljg@mail.python.org>

http://hg.python.org/jython/rev/b73f07d78c1c
changeset:   7229:b73f07d78c1c
user:        Indra Talip <indra.talip at gmail.com>
date:        Tue Jan 07 21:15:01 2014 +1100
summary:
  Defer the creation of BZip2CompressorInputStream until a read occurs
This is because the BZip2CompressorInputStream does an initial read on the
underlying InputStream that causes an exception to be thrown on construction
of the PyBZ2File instead of the read. Deferring creating the read buffer
corrects a number of the tests in test_tarfile that were failing due to the
exception thrown on construction rather than on read.

files:
  src/org/python/modules/bz2/PyBZ2File.java |  49 ++++++++--
  1 files changed, 36 insertions(+), 13 deletions(-)


diff --git a/src/org/python/modules/bz2/PyBZ2File.java b/src/org/python/modules/bz2/PyBZ2File.java
--- a/src/org/python/modules/bz2/PyBZ2File.java
+++ b/src/org/python/modules/bz2/PyBZ2File.java
@@ -55,6 +55,8 @@
     private String fileMode = "";
     private boolean inIterMode = false;
     private boolean inUniversalNewlineMode = false;
+    private boolean needReadBufferInit = false;
+    private boolean inReadMode = false;
 
     private BZip2CompressorOutputStream writeStream = null;
 
@@ -111,7 +113,13 @@
                 writeStream = new BZip2CompressorOutputStream(
                         new FileOutputStream(fileName), compresslevel);
             } else {
-                makeReadBuffer();
+                File f = new File(fileName);
+                if (!f.exists()) {
+                    throw new FileNotFoundException();
+                }
+
+                inReadMode = true;
+                needReadBufferInit = true;
             }
         } catch (IOException e) {
             throw Py.IOError("File " + fileName + " not found,");
@@ -134,8 +142,8 @@
             }
         } catch (FileNotFoundException fileNotFoundException) {
             throw Py.IOError(fileNotFoundException);
-        } catch (IOException io) {
-            throw Py.IOError(io);
+        } catch (IOException ioe) {
+            throw Py.EOFError(ioe.getMessage());
         }
     }
 
@@ -155,6 +163,8 @@
                 throw Py.IOError(e.getMessage());
             }
         }
+
+        needReadBufferInit = false;
         if (buffer != null) {
             buffer.close();
         }
@@ -173,6 +183,7 @@
     @ExposedMethod
     public PyObject BZ2File_read(PyObject[] args, String[] kwds) {
         checkInIterMode();
+        checkReadBufferInit();
 
         ArgParser ap = new ArgParser("read", args, kwds,
                 new String[] { "size" }, 0);
@@ -185,6 +196,8 @@
 
     @ExposedMethod
     public PyObject BZ2File_next(PyObject[] args, String[] kwds) {
+        checkReadBufferInit();
+
         if (buffer == null || buffer.closed()) {
             throw Py.ValueError("Cannot call next() on closed file");
         }
@@ -196,6 +209,7 @@
     @ExposedMethod
     public PyString BZ2File_readline(PyObject[] args, String[] kwds) {
         checkInIterMode();
+        checkReadBufferInit();
 
         ArgParser ap = new ArgParser("read", args, kwds,
                 new String[] { "size" }, 0);
@@ -208,6 +222,7 @@
     @ExposedMethod
     public PyList BZ2File_readlines(PyObject[] args, String[] kwds) {
         checkInIterMode();
+        checkReadBufferInit();
 
         // make sure file data valid
         if (buffer == null || buffer.closed()) {
@@ -224,10 +239,15 @@
     }
 
     private void checkInIterMode() {
-        if (fileMode.contains("r")) {
-            if (inIterMode) {
-                throw Py.ValueError("Cannot mix iteration and reads");
-            }
+        if (inReadMode && inIterMode) {
+            throw Py.ValueError("Cannot mix iteration and reads");
+        }
+    }
+
+    private void checkReadBufferInit() {
+        if (inReadMode && needReadBufferInit) {
+            makeReadBuffer();
+            needReadBufferInit = false;
         }
     }
 
@@ -238,16 +258,18 @@
 
     @ExposedMethod
     public void BZ2File_seek(PyObject[] args, String[] kwds) {
+        if (!inReadMode) {
+            Py.IOError("seek works only while reading");
+        }
+
         ArgParser ap = new ArgParser("seek", args, kwds, new String[] {
                 "offset", "whence" }, 1);
 
+        checkReadBufferInit();
+
         int newOffset = ap.getInt(0);
         int whence = ap.getInt(1, 0);
 
-        if (fileMode.contains("w")) {
-            Py.IOError("seek works only while reading");
-        }
-
         // normalise offset
         long currentPos = buffer.tell();
 
@@ -298,6 +320,7 @@
 
     @ExposedMethod
     public PyLong BZ2File_tell() {
+        checkReadBufferInit();
         if (buffer == null) {
             return Py.newLong(0);
         } else {
@@ -347,7 +370,7 @@
     }
 
     private void checkFileWritable() {
-        if (fileMode.contains("r")) {
+        if (inReadMode) {
             throw Py.IOError("File in read-only mode");
         }
         if (writeStream == null) {
@@ -382,7 +405,7 @@
             if (writeStream == null) {
                 throw Py.ValueError("Stream closed");
             }
-        } else if (fileMode.contains("r")) {
+        } else if (inReadMode && !needReadBufferInit) {
             if (buffer == null || buffer.closed()) {
                 throw Py.ValueError("Stream closed");
             }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 00:30:47 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 00:30:47 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Refactor_compressing_data_t?=
 =?utf-8?q?o_a_bz2_file_to_use_the_PEP3116_infrastructure=2E?=
Message-ID: <3gQR872WPcz7Ljc@mail.python.org>

http://hg.python.org/jython/rev/b1304c8ba094
changeset:   7230:b1304c8ba094
user:        Indra Talip <indra.talip at gmail.com>
date:        Tue Jan 07 21:15:42 2014 +1100
summary:
  Refactor compressing data to a bz2 file to use the PEP3116 infrastructure.

files:
  src/org/python/modules/bz2/PyBZ2File.java |  54 ++++------
  1 files changed, 21 insertions(+), 33 deletions(-)


diff --git a/src/org/python/modules/bz2/PyBZ2File.java b/src/org/python/modules/bz2/PyBZ2File.java
--- a/src/org/python/modules/bz2/PyBZ2File.java
+++ b/src/org/python/modules/bz2/PyBZ2File.java
@@ -7,8 +7,8 @@
 import java.io.FileOutputStream;
 import java.io.IOException;
 import java.io.InputStream;
+import java.io.OutputStream;
 import java.nio.ByteBuffer;
-import java.util.ArrayList;
 import java.util.Iterator;
 
 import org.apache.commons.compress.compressors.bzip2.BZip2CompressorInputStream;
@@ -26,6 +26,7 @@
 import org.python.core.PyType;
 import org.python.core.io.BinaryIOWrapper;
 import org.python.core.io.BufferedReader;
+import org.python.core.io.BufferedWriter;
 import org.python.core.io.IOBase;
 import org.python.core.io.StreamIO;
 import org.python.core.io.TextIOBase;
@@ -52,13 +53,12 @@
 
     private TextIOBase buffer;
     private String fileName = null;
-    private String fileMode = "";
     private boolean inIterMode = false;
     private boolean inUniversalNewlineMode = false;
     private boolean needReadBufferInit = false;
     private boolean inReadMode = false;
+    private boolean inWriteMode = false;
 
-    private BZip2CompressorOutputStream writeStream = null;
 
     public PyBZ2File() {
         super(TYPE);
@@ -96,7 +96,6 @@
             int buffering, int compresslevel) {
         try {
             fileName = inFileName.asString();
-            fileMode = mode;
             this.buffering = buffering;
 
             // check universal newline mode
@@ -105,13 +104,18 @@
             }
 
             if (mode.contains("w")) {
+                inWriteMode = true;
                 File f = new File(fileName);
                 if (!f.exists()) {
                     f.createNewFile();
                 }
 
-                writeStream = new BZip2CompressorOutputStream(
+                BZip2CompressorOutputStream writeStream = new BZip2CompressorOutputStream(
                         new FileOutputStream(fileName), compresslevel);
+                buffer = new BinaryIOWrapper(
+                            new BufferedWriter(
+                                new SkippableStreamIO(writeStream, true),
+                                buffering));
             } else {
                 File f = new File(fileName);
                 if (!f.exists()) {
@@ -154,29 +158,16 @@
 
     @ExposedMethod
     public void BZ2File_close() {
-        if (writeStream != null) {
-            BZ2File_flush();
-            try {
-                writeStream.close();
-                writeStream = null;
-            } catch (IOException e) {
-                throw Py.IOError(e.getMessage());
-            }
-        }
-
         needReadBufferInit = false;
+        BZ2File_flush();
         if (buffer != null) {
             buffer.close();
         }
     }
 
     private void BZ2File_flush() {
-        if (writeStream != null) {
-            try {
-                writeStream.flush();
-            } catch (IOException e) {
-                throw Py.IOError(e.getMessage());
-            }
+        if (buffer != null) {
+            buffer.flush();
         }
     }
 
@@ -339,16 +330,8 @@
         if (data.getType() == PyNone.TYPE) {
             throw Py.TypeError("Expecting str argument");
         }
-        byte[] buf = ap.getString(0).getBytes();
 
-        try {
-            synchronized (this) {
-                writeStream.write(buf);
-            }
-        } catch (IOException e) {
-
-            throw Py.IOError(e.getMessage());
-        }
+        buffer.write(ap.getString(0));
     }
 
     @ExposedMethod
@@ -373,7 +356,8 @@
         if (inReadMode) {
             throw Py.IOError("File in read-only mode");
         }
-        if (writeStream == null) {
+
+        if (buffer == null || buffer.closed()) {
             throw Py.ValueError("Stream closed");
         }
     }
@@ -401,8 +385,8 @@
 
     @ExposedMethod
     public PyObject BZ2File___enter__() {
-        if (fileMode.contains("w")) {
-            if (writeStream == null) {
+        if (inWriteMode) {
+            if (buffer == null) {
                 throw Py.ValueError("Stream closed");
             }
         } else if (inReadMode && !needReadBufferInit) {
@@ -428,6 +412,10 @@
             super(inputStream, closefd);
         }
 
+        public SkippableStreamIO(OutputStream outputStream, boolean closefd) {
+            super(outputStream, closefd);
+        }
+
         @Override
         public int readinto(ByteBuffer buf) {
             int bytesRead = 0;

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 00:30:48 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 00:30:48 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_Merged_pull_request?=
Message-ID: <3gQR886hrxz7Ljc@mail.python.org>

http://hg.python.org/jython/rev/e467aa777b11
changeset:   7231:e467aa777b11
parent:      7227:0431ab4d1e44
parent:      7230:b1304c8ba094
user:        Jim Baker <jim.baker at rackspace.com>
date:        Fri May 09 16:31:05 2014 -0600
summary:
  Merged pull request https://bitbucket.org/jython/jython/pull-request/22/fixes-for-bz2-module

files:
  src/org/python/modules/bz2/PyBZ2File.java |  385 ++++-----
  1 files changed, 191 insertions(+), 194 deletions(-)


diff --git a/src/org/python/modules/bz2/PyBZ2File.java b/src/org/python/modules/bz2/PyBZ2File.java
--- a/src/org/python/modules/bz2/PyBZ2File.java
+++ b/src/org/python/modules/bz2/PyBZ2File.java
@@ -1,19 +1,21 @@
 package org.python.modules.bz2;
 
 import java.io.BufferedInputStream;
-import java.io.ByteArrayOutputStream;
 import java.io.File;
 import java.io.FileInputStream;
+import java.io.FileNotFoundException;
 import java.io.FileOutputStream;
 import java.io.IOException;
-import java.util.ArrayList;
-import java.util.Arrays;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
 import java.util.Iterator;
 
 import org.apache.commons.compress.compressors.bzip2.BZip2CompressorInputStream;
 import org.apache.commons.compress.compressors.bzip2.BZip2CompressorOutputStream;
 import org.python.core.ArgParser;
 import org.python.core.Py;
+import org.python.core.PyException;
 import org.python.core.PyIterator;
 import org.python.core.PyList;
 import org.python.core.PyLong;
@@ -21,8 +23,14 @@
 import org.python.core.PyObject;
 import org.python.core.PySequence;
 import org.python.core.PyString;
-import org.python.core.PyTuple;
 import org.python.core.PyType;
+import org.python.core.io.BinaryIOWrapper;
+import org.python.core.io.BufferedReader;
+import org.python.core.io.BufferedWriter;
+import org.python.core.io.IOBase;
+import org.python.core.io.StreamIO;
+import org.python.core.io.TextIOBase;
+import org.python.core.io.UniversalIOWrapper;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
 import org.python.expose.ExposedNew;
@@ -32,24 +40,25 @@
 public class PyBZ2File extends PyObject {
 
     public static final PyType TYPE = PyType.fromClass(PyBZ2File.class);
+    private int buffering;
 
-    @ExposedGet
-    public PyObject newlines = null;
+    @ExposedGet(name = "newlines")
+    public PyObject PyBZ2File_newlines() {
+        if (buffer != null) {
+            return buffer.getNewlines();
+        } else {
+            return Py.None;
+        }
+    }
 
-    private byte[] fileData = null;
-    private int offset = 0;
+    private TextIOBase buffer;
     private String fileName = null;
-    private String fileMode = "";
     private boolean inIterMode = false;
     private boolean inUniversalNewlineMode = false;
-    private final ArrayList<String> validNewlines = new ArrayList<String>();
-    {
-        validNewlines.add("\n");
-        validNewlines.add("\r");
-        validNewlines.add("\r\n");
-    }
+    private boolean needReadBufferInit = false;
+    private boolean inReadMode = false;
+    private boolean inWriteMode = false;
 
-    private BZip2CompressorOutputStream writeStream = null;
 
     public PyBZ2File() {
         super(TYPE);
@@ -86,9 +95,8 @@
     private void BZ2File___init__(PyString inFileName, String mode,
             int buffering, int compresslevel) {
         try {
-
             fileName = inFileName.asString();
-            fileMode = mode;
+            this.buffering = buffering;
 
             // check universal newline mode
             if (mode.contains("U")) {
@@ -96,38 +104,53 @@
             }
 
             if (mode.contains("w")) {
+                inWriteMode = true;
                 File f = new File(fileName);
                 if (!f.exists()) {
                     f.createNewFile();
                 }
 
-                writeStream = new BZip2CompressorOutputStream(
+                BZip2CompressorOutputStream writeStream = new BZip2CompressorOutputStream(
                         new FileOutputStream(fileName), compresslevel);
+                buffer = new BinaryIOWrapper(
+                            new BufferedWriter(
+                                new SkippableStreamIO(writeStream, true),
+                                buffering));
             } else {
-                FileInputStream fin = new FileInputStream(fileName);
-                BufferedInputStream bin = new BufferedInputStream(fin);
-                BZip2CompressorInputStream bZin = new BZip2CompressorInputStream(
-                        bin);
+                File f = new File(fileName);
+                if (!f.exists()) {
+                    throw new FileNotFoundException();
+                }
 
-                ByteArrayOutputStream buffer = new ByteArrayOutputStream();
-
-                final byte[] buf = new byte[100];
-                int n = 0;
-                while (-1 != (n = bZin.read(buf))) {
-                    buffer.write(buf, 0, n);
-                }
-                fileData = buffer.toByteArray();
-
-                buffer.close();
-                bZin.close();
-                bin.close();
-                fin.close();
+                inReadMode = true;
+                needReadBufferInit = true;
             }
         } catch (IOException e) {
             throw Py.IOError("File " + fileName + " not found,");
         }
     }
 
+    private void makeReadBuffer() {
+        try {
+            FileInputStream fin = new FileInputStream(fileName);
+            BufferedInputStream bin = new BufferedInputStream(fin);
+            BZip2CompressorInputStream bZin = new BZip2CompressorInputStream(
+                    bin, true);
+            BufferedReader bufferedReader = new BufferedReader(
+                    new SkippableStreamIO(bZin, true), buffering);
+
+            if (inUniversalNewlineMode) {
+                buffer = new UniversalIOWrapper(bufferedReader);
+            } else {
+                buffer = new BinaryIOWrapper(bufferedReader);
+            }
+        } catch (FileNotFoundException fileNotFoundException) {
+            throw Py.IOError(fileNotFoundException);
+        } catch (IOException ioe) {
+            throw Py.EOFError(ioe.getMessage());
+        }
+    }
+
     @ExposedMethod
     public void __del__() {
         BZ2File_close();
@@ -135,100 +158,41 @@
 
     @ExposedMethod
     public void BZ2File_close() {
-        fileData = null;
-
-        if (writeStream != null) {
-            BZ2File_flush();
-            try {
-                writeStream.close();
-                writeStream = null;
-            } catch (IOException e) {
-                throw Py.IOError(e.getMessage());
-            }
+        needReadBufferInit = false;
+        BZ2File_flush();
+        if (buffer != null) {
+            buffer.close();
         }
     }
 
     private void BZ2File_flush() {
-        if (writeStream != null) {
-            try {
-                writeStream.flush();
-            } catch (IOException e) {
-                throw Py.IOError(e.getMessage());
-            }
+        if (buffer != null) {
+            buffer.flush();
         }
     }
 
-    private byte[] peek() {
-
-        byte[] buf = new byte[1];
-        if (fileData.length > offset) {
-            buf[0] = fileData[offset + 1];
-        }
-
-        return buf;
-    }
-
     @ExposedMethod
     public PyObject BZ2File_read(PyObject[] args, String[] kwds) {
         checkInIterMode();
+        checkReadBufferInit();
 
         ArgParser ap = new ArgParser("read", args, kwds,
                 new String[] { "size" }, 0);
 
         int size = ap.getInt(0, -1);
+        final String data = buffer.read(size);
 
-        byte[] buf = _BZ2File_read(size);
-
-        return new PyString(new String(buf));
-    }
-
-    private byte[] _BZ2File_read(int size) {
-        byte[] buf = null;
-        if (size == 0) {
-            return new byte[0];
-        } else if (size > 0) {
-            buf = new byte[size];
-        } else {
-            buf = new byte[fileData.length - offset];
-        }
-
-        int readbytes = 0;
-        for (int i = offset, j = 0; i < fileData.length && j < buf.length; i++, j++) {
-            buf[j] = fileData[i];
-
-            String possibleNewline = new String(new byte[] { buf[j] });
-            if (possibleNewline.equals("\r")) { // handle CRLF
-                buf[j] = '\n';
-                if (fileData[i + 1] == '\n') { // check if next character part
-                                               // of newline
-                    possibleNewline = possibleNewline
-                            + new String(new byte[] { fileData[i + 1] });
-                    buf = Arrays.copyOf(buf, buf.length - 1); // adjust buffer
-                                                              // size
-                    i++;
-                }
-            }
-            if (validNewlines.contains(possibleNewline)) {
-                addNewlineMarker(possibleNewline);
-            }
-
-            offset++;
-            readbytes++;
-        }
-
-        if (readbytes == 0) {
-            return new byte[0];
-        }
-
-        return buf;
-
+        return new PyString(data);
     }
 
     @ExposedMethod
     public PyObject BZ2File_next(PyObject[] args, String[] kwds) {
-        if (fileData == null) {
+        checkReadBufferInit();
+
+        if (buffer == null || buffer.closed()) {
             throw Py.ValueError("Cannot call next() on closed file");
         }
+
         inIterMode = true;
         return null;
     }
@@ -236,74 +200,25 @@
     @ExposedMethod
     public PyString BZ2File_readline(PyObject[] args, String[] kwds) {
         checkInIterMode();
+        checkReadBufferInit();
 
         ArgParser ap = new ArgParser("read", args, kwds,
                 new String[] { "size" }, 0);
 
         int size = ap.getInt(0, -1);
 
-        StringBuilder line = new StringBuilder();
-
-        byte[] buf = null;
-        int readSize = 0;
-        while ((buf = _BZ2File_read(1)).length > 0) {
-            line.append(new String(buf));
-            // handle newlines
-            boolean mustBreak = false;
-            if (inUniversalNewlineMode) {
-                if ((char) buf[0] == '\r') {
-                    if (peek()[0] == '\n') {
-                        buf = _BZ2File_read(1);
-                        mustBreak = true;
-                    }
-                    line.replace(line.length() - 1, line.length(), new String(
-                            "\n"));
-                    mustBreak = true;
-                } else if ((char) buf[0] == '\n'
-                        || (size > -1 && (readSize >= size))) {
-                    mustBreak = true;
-                }
-
-            } else {
-                if ((char) buf[0] == '\n' || (size > -1 && (readSize >= size))) {
-                    mustBreak = true;
-                }
-            }
-
-            if (mustBreak) {
-                break;
-            }
-        }
-
-        return new PyString(line.toString());
-    }
-
-    private void addNewlineMarker(String newline) {
-        if (newlines == null) {
-            newlines = new PyString(newline);
-        } else {
-            if (newlines instanceof PyString) {
-                if (!newlines.equals(new PyString(newline))) {
-                    newlines = new PyTuple(newlines, new PyString(newline));
-                }
-            } else {
-                if (!newlines.__contains__(new PyString(newline))) {
-                    newlines = newlines.__add__(new PyTuple(new PyString(
-                            newline)));
-                }
-            }
-        }
+        return new PyString(buffer.readline(size));
     }
 
     @ExposedMethod
     public PyList BZ2File_readlines(PyObject[] args, String[] kwds) {
         checkInIterMode();
+        checkReadBufferInit();
 
         // make sure file data valid
-        if (fileData == null) {
+        if (buffer == null || buffer.closed()) {
             throw Py.ValueError("Cannot call readlines() on a closed file");
         }
-
         PyList lineList = new PyList();
 
         PyString line = null;
@@ -315,10 +230,15 @@
     }
 
     private void checkInIterMode() {
-        if (fileMode.contains("r")) {
-            if (inIterMode) {
-                throw Py.ValueError("Cannot mix iteration and reads");
-            }
+        if (inReadMode && inIterMode) {
+            throw Py.ValueError("Cannot mix iteration and reads");
+        }
+    }
+
+    private void checkReadBufferInit() {
+        if (inReadMode && needReadBufferInit) {
+            makeReadBuffer();
+            needReadBufferInit = false;
         }
     }
 
@@ -329,44 +249,74 @@
 
     @ExposedMethod
     public void BZ2File_seek(PyObject[] args, String[] kwds) {
+        if (!inReadMode) {
+            Py.IOError("seek works only while reading");
+        }
+
         ArgParser ap = new ArgParser("seek", args, kwds, new String[] {
                 "offset", "whence" }, 1);
 
+        checkReadBufferInit();
+
         int newOffset = ap.getInt(0);
         int whence = ap.getInt(1, 0);
 
         // normalise offset
-        int finalOffset = 0;
+        long currentPos = buffer.tell();
+
+        long finalOffset = 0;
         switch (whence) {
         case 0: // offset from start of file
-            if (newOffset > fileData.length) {
-                finalOffset = fileData.length;
-            } else {
-                finalOffset = newOffset;
-            }
+            finalOffset = newOffset;
             break;
         case 1: // move relative to current position
-            finalOffset = offset + newOffset;
+            finalOffset = currentPos + newOffset;
+
             break;
         case 2: // move relative to end of file
-            finalOffset = fileData.length + newOffset;
+            long fileSize = currentPos;
+
+            // in order to seek from the end of the stream we need to fully read
+            // the decompressed stream to get the size
+            for (;;) {
+                final String data = buffer.read(IOBase.DEFAULT_BUFFER_SIZE);
+                if (data.isEmpty()) {
+                    break;
+                }
+                fileSize += data.length();
+            }
+
+            finalOffset = fileSize + newOffset;
+
+            // close and reset the buffer
+            buffer.close();
+            makeReadBuffer();
+
+            break;
         }
 
         if (finalOffset < 0) {
             finalOffset = 0;
-        } else {
-            if (finalOffset > fileData.length) {
-                finalOffset = fileData.length;
-            }
+        }
+
+        // can't seek backwards so close and reopen the stream at the start
+        if (whence != 2 && finalOffset < currentPos) {
+            buffer.close();
+            makeReadBuffer();
         }
 
         // seek operation
-        offset = finalOffset;
+        buffer.seek(finalOffset, 0);
     }
 
     @ExposedMethod
     public PyLong BZ2File_tell() {
-        return new PyLong(offset);
+        checkReadBufferInit();
+        if (buffer == null) {
+            return Py.newLong(0);
+        } else {
+            return Py.newLong(buffer.tell());
+        }
     }
 
     @ExposedMethod
@@ -380,16 +330,8 @@
         if (data.getType() == PyNone.TYPE) {
             throw Py.TypeError("Expecting str argument");
         }
-        byte[] buf = ap.getString(0).getBytes();
 
-        try {
-            synchronized (this) {
-                writeStream.write(buf);
-            }
-        } catch (IOException e) {
-
-            throw Py.IOError(e.getMessage());
-        }
+        buffer.write(ap.getString(0));
     }
 
     @ExposedMethod
@@ -411,10 +353,11 @@
     }
 
     private void checkFileWritable() {
-        if (fileMode.contains("r")) {
+        if (inReadMode) {
             throw Py.IOError("File in read-only mode");
         }
-        if (writeStream == null) {
+
+        if (buffer == null || buffer.closed()) {
             throw Py.ValueError("Stream closed");
         }
     }
@@ -442,12 +385,12 @@
 
     @ExposedMethod
     public PyObject BZ2File___enter__() {
-        if (fileMode.contains("w")) {
-            if (writeStream == null) {
+        if (inWriteMode) {
+            if (buffer == null) {
                 throw Py.ValueError("Stream closed");
             }
-        } else if (fileMode.contains("r")) {
-            if (fileData == null) {
+        } else if (inReadMode && !needReadBufferInit) {
+            if (buffer == null || buffer.closed()) {
                 throw Py.ValueError("Stream closed");
             }
         }
@@ -461,4 +404,58 @@
         BZ2File_close();
         return false;
     }
+
+    private static class SkippableStreamIO extends StreamIO {
+        private long position = 0;
+
+        public SkippableStreamIO(InputStream inputStream, boolean closefd) {
+            super(inputStream, closefd);
+        }
+
+        public SkippableStreamIO(OutputStream outputStream, boolean closefd) {
+            super(outputStream, closefd);
+        }
+
+        @Override
+        public int readinto(ByteBuffer buf) {
+            int bytesRead = 0;
+            try {
+                bytesRead = super.readinto(buf);
+            } catch (PyException pyex) {
+                // translate errors on read of decompressed stream to EOFError
+                throw Py.EOFError(pyex.value.asStringOrNull());
+            }
+
+            position += bytesRead;
+            return bytesRead;
+        }
+
+        @Override
+        public long tell() {
+            return position;
+        }
+
+        @Override
+        public long seek(long offset, int whence) {
+            long skipBytes = offset - position;
+            if (whence != 0 || skipBytes < 0) {
+                throw Py.IOError("can only seek forward");
+            }
+
+            if (skipBytes == 0) {
+                return position;
+            } else {
+                long skipped = 0;
+                try {
+                    skipped = asInputStream().skip(skipBytes);
+                } catch (IOException ex) {
+                    throw Py.IOError(ex);
+                }
+                long newPosition = position + skipped;
+                position = newPosition;
+
+                return newPosition;
+            }
+        }
+    }
 }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 04:59:19 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 04:59:19 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Upgraded_apache_commons_com?=
	=?utf-8?q?press_to_1=2E8?=
Message-ID: <3gQY5z0z0Rz7LjP@mail.python.org>

http://hg.python.org/jython/rev/8c3154683142
changeset:   7232:8c3154683142
parent:      7227:0431ab4d1e44
user:        Jim Baker <jim.baker at rackspace.com>
date:        Fri May 09 20:56:00 2014 -0600
summary:
  Upgraded apache commons compress to 1.8

files:
  build.xml                          |    4 ++--
  extlibs/commons-compress-1.4.1.jar |  Bin 
  extlibs/commons-compress-1.8.jar   |  Bin 
  3 files changed, 2 insertions(+), 2 deletions(-)


diff --git a/build.xml b/build.xml
--- a/build.xml
+++ b/build.xml
@@ -148,7 +148,7 @@
             <pathelement path="${extlibs.dir}/antlr-3.1.3.jar" />
             <pathelement path="${extlibs.dir}/stringtemplate-3.2.1.jar" />
             <pathelement path="${extlibs.dir}/livetribe-jsr223-2.0.6.jar" />
-            <pathelement path="${extlibs.dir}/commons-compress-1.4.1.jar"/>
+            <pathelement path="${extlibs.dir}/commons-compress-1.8.jar"/>
             <pathelement path="${extlibs.dir}/asm-4.0.jar" />
             <pathelement path="${extlibs.dir}/asm-commons-4.0.jar" />
             <pathelement path="${extlibs.dir}/asm-util-4.0.jar" />
@@ -577,7 +577,7 @@
 	    <rule pattern="org.bouncycastle.**" result="org.python.bouncycastle. at 1"/>
 	    <zipfileset src="extlibs/bcprov-jdk15on-150.jar" excludes="META-INF/*.SF"/>
 	    <rule pattern="org.bouncycastle.**" result="org.python.bouncycastle. at 1"/>
-	    <zipfileset src="extlibs/commons-compress-1.4.1.jar"/>
+	    <zipfileset src="extlibs/commons-compress-1.8.jar"/>
 	    <rule pattern="org.apache.**" result="org.python.apache. at 1"/>
             <zipfileset src="extlibs/guava-13.0.1.jar"/>
             <rule pattern="com.google.**" result="org.python.google. at 1"/>
diff --git a/extlibs/commons-compress-1.4.1.jar b/extlibs/commons-compress-1.4.1.jar
deleted file mode 100644
index b58761e81235e247e363cdf67819e63577ae0e99..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/commons-compress-1.8.jar b/extlibs/commons-compress-1.8.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..940b0687323428c7ef6063a07a43eb77fb9cbea1
GIT binary patch
[stripped]

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 04:59:20 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 04:59:20 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython_=28merge_default_-=3E_default?=
	=?utf-8?q?=29=3A_Merged_head?=
Message-ID: <3gQY604X4Cz7Ljw@mail.python.org>

http://hg.python.org/jython/rev/e7da164c360b
changeset:   7233:e7da164c360b
parent:      7232:8c3154683142
parent:      7231:e467aa777b11
user:        Jim Baker <jim.baker at rackspace.com>
date:        Fri May 09 20:58:42 2014 -0600
summary:
  Merged head

files:
  src/org/python/modules/bz2/PyBZ2File.java |  385 ++++-----
  1 files changed, 191 insertions(+), 194 deletions(-)


diff --git a/src/org/python/modules/bz2/PyBZ2File.java b/src/org/python/modules/bz2/PyBZ2File.java
--- a/src/org/python/modules/bz2/PyBZ2File.java
+++ b/src/org/python/modules/bz2/PyBZ2File.java
@@ -1,19 +1,21 @@
 package org.python.modules.bz2;
 
 import java.io.BufferedInputStream;
-import java.io.ByteArrayOutputStream;
 import java.io.File;
 import java.io.FileInputStream;
+import java.io.FileNotFoundException;
 import java.io.FileOutputStream;
 import java.io.IOException;
-import java.util.ArrayList;
-import java.util.Arrays;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.ByteBuffer;
 import java.util.Iterator;
 
 import org.apache.commons.compress.compressors.bzip2.BZip2CompressorInputStream;
 import org.apache.commons.compress.compressors.bzip2.BZip2CompressorOutputStream;
 import org.python.core.ArgParser;
 import org.python.core.Py;
+import org.python.core.PyException;
 import org.python.core.PyIterator;
 import org.python.core.PyList;
 import org.python.core.PyLong;
@@ -21,8 +23,14 @@
 import org.python.core.PyObject;
 import org.python.core.PySequence;
 import org.python.core.PyString;
-import org.python.core.PyTuple;
 import org.python.core.PyType;
+import org.python.core.io.BinaryIOWrapper;
+import org.python.core.io.BufferedReader;
+import org.python.core.io.BufferedWriter;
+import org.python.core.io.IOBase;
+import org.python.core.io.StreamIO;
+import org.python.core.io.TextIOBase;
+import org.python.core.io.UniversalIOWrapper;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
 import org.python.expose.ExposedNew;
@@ -32,24 +40,25 @@
 public class PyBZ2File extends PyObject {
 
     public static final PyType TYPE = PyType.fromClass(PyBZ2File.class);
+    private int buffering;
 
-    @ExposedGet
-    public PyObject newlines = null;
+    @ExposedGet(name = "newlines")
+    public PyObject PyBZ2File_newlines() {
+        if (buffer != null) {
+            return buffer.getNewlines();
+        } else {
+            return Py.None;
+        }
+    }
 
-    private byte[] fileData = null;
-    private int offset = 0;
+    private TextIOBase buffer;
     private String fileName = null;
-    private String fileMode = "";
     private boolean inIterMode = false;
     private boolean inUniversalNewlineMode = false;
-    private final ArrayList<String> validNewlines = new ArrayList<String>();
-    {
-        validNewlines.add("\n");
-        validNewlines.add("\r");
-        validNewlines.add("\r\n");
-    }
+    private boolean needReadBufferInit = false;
+    private boolean inReadMode = false;
+    private boolean inWriteMode = false;
 
-    private BZip2CompressorOutputStream writeStream = null;
 
     public PyBZ2File() {
         super(TYPE);
@@ -86,9 +95,8 @@
     private void BZ2File___init__(PyString inFileName, String mode,
             int buffering, int compresslevel) {
         try {
-
             fileName = inFileName.asString();
-            fileMode = mode;
+            this.buffering = buffering;
 
             // check universal newline mode
             if (mode.contains("U")) {
@@ -96,38 +104,53 @@
             }
 
             if (mode.contains("w")) {
+                inWriteMode = true;
                 File f = new File(fileName);
                 if (!f.exists()) {
                     f.createNewFile();
                 }
 
-                writeStream = new BZip2CompressorOutputStream(
+                BZip2CompressorOutputStream writeStream = new BZip2CompressorOutputStream(
                         new FileOutputStream(fileName), compresslevel);
+                buffer = new BinaryIOWrapper(
+                            new BufferedWriter(
+                                new SkippableStreamIO(writeStream, true),
+                                buffering));
             } else {
-                FileInputStream fin = new FileInputStream(fileName);
-                BufferedInputStream bin = new BufferedInputStream(fin);
-                BZip2CompressorInputStream bZin = new BZip2CompressorInputStream(
-                        bin);
+                File f = new File(fileName);
+                if (!f.exists()) {
+                    throw new FileNotFoundException();
+                }
 
-                ByteArrayOutputStream buffer = new ByteArrayOutputStream();
-
-                final byte[] buf = new byte[100];
-                int n = 0;
-                while (-1 != (n = bZin.read(buf))) {
-                    buffer.write(buf, 0, n);
-                }
-                fileData = buffer.toByteArray();
-
-                buffer.close();
-                bZin.close();
-                bin.close();
-                fin.close();
+                inReadMode = true;
+                needReadBufferInit = true;
             }
         } catch (IOException e) {
             throw Py.IOError("File " + fileName + " not found,");
         }
     }
 
+    private void makeReadBuffer() {
+        try {
+            FileInputStream fin = new FileInputStream(fileName);
+            BufferedInputStream bin = new BufferedInputStream(fin);
+            BZip2CompressorInputStream bZin = new BZip2CompressorInputStream(
+                    bin, true);
+            BufferedReader bufferedReader = new BufferedReader(
+                    new SkippableStreamIO(bZin, true), buffering);
+
+            if (inUniversalNewlineMode) {
+                buffer = new UniversalIOWrapper(bufferedReader);
+            } else {
+                buffer = new BinaryIOWrapper(bufferedReader);
+            }
+        } catch (FileNotFoundException fileNotFoundException) {
+            throw Py.IOError(fileNotFoundException);
+        } catch (IOException ioe) {
+            throw Py.EOFError(ioe.getMessage());
+        }
+    }
+
     @ExposedMethod
     public void __del__() {
         BZ2File_close();
@@ -135,100 +158,41 @@
 
     @ExposedMethod
     public void BZ2File_close() {
-        fileData = null;
-
-        if (writeStream != null) {
-            BZ2File_flush();
-            try {
-                writeStream.close();
-                writeStream = null;
-            } catch (IOException e) {
-                throw Py.IOError(e.getMessage());
-            }
+        needReadBufferInit = false;
+        BZ2File_flush();
+        if (buffer != null) {
+            buffer.close();
         }
     }
 
     private void BZ2File_flush() {
-        if (writeStream != null) {
-            try {
-                writeStream.flush();
-            } catch (IOException e) {
-                throw Py.IOError(e.getMessage());
-            }
+        if (buffer != null) {
+            buffer.flush();
         }
     }
 
-    private byte[] peek() {
-
-        byte[] buf = new byte[1];
-        if (fileData.length > offset) {
-            buf[0] = fileData[offset + 1];
-        }
-
-        return buf;
-    }
-
     @ExposedMethod
     public PyObject BZ2File_read(PyObject[] args, String[] kwds) {
         checkInIterMode();
+        checkReadBufferInit();
 
         ArgParser ap = new ArgParser("read", args, kwds,
                 new String[] { "size" }, 0);
 
         int size = ap.getInt(0, -1);
+        final String data = buffer.read(size);
 
-        byte[] buf = _BZ2File_read(size);
-
-        return new PyString(new String(buf));
-    }
-
-    private byte[] _BZ2File_read(int size) {
-        byte[] buf = null;
-        if (size == 0) {
-            return new byte[0];
-        } else if (size > 0) {
-            buf = new byte[size];
-        } else {
-            buf = new byte[fileData.length - offset];
-        }
-
-        int readbytes = 0;
-        for (int i = offset, j = 0; i < fileData.length && j < buf.length; i++, j++) {
-            buf[j] = fileData[i];
-
-            String possibleNewline = new String(new byte[] { buf[j] });
-            if (possibleNewline.equals("\r")) { // handle CRLF
-                buf[j] = '\n';
-                if (fileData[i + 1] == '\n') { // check if next character part
-                                               // of newline
-                    possibleNewline = possibleNewline
-                            + new String(new byte[] { fileData[i + 1] });
-                    buf = Arrays.copyOf(buf, buf.length - 1); // adjust buffer
-                                                              // size
-                    i++;
-                }
-            }
-            if (validNewlines.contains(possibleNewline)) {
-                addNewlineMarker(possibleNewline);
-            }
-
-            offset++;
-            readbytes++;
-        }
-
-        if (readbytes == 0) {
-            return new byte[0];
-        }
-
-        return buf;
-
+        return new PyString(data);
     }
 
     @ExposedMethod
     public PyObject BZ2File_next(PyObject[] args, String[] kwds) {
-        if (fileData == null) {
+        checkReadBufferInit();
+
+        if (buffer == null || buffer.closed()) {
             throw Py.ValueError("Cannot call next() on closed file");
         }
+
         inIterMode = true;
         return null;
     }
@@ -236,74 +200,25 @@
     @ExposedMethod
     public PyString BZ2File_readline(PyObject[] args, String[] kwds) {
         checkInIterMode();
+        checkReadBufferInit();
 
         ArgParser ap = new ArgParser("read", args, kwds,
                 new String[] { "size" }, 0);
 
         int size = ap.getInt(0, -1);
 
-        StringBuilder line = new StringBuilder();
-
-        byte[] buf = null;
-        int readSize = 0;
-        while ((buf = _BZ2File_read(1)).length > 0) {
-            line.append(new String(buf));
-            // handle newlines
-            boolean mustBreak = false;
-            if (inUniversalNewlineMode) {
-                if ((char) buf[0] == '\r') {
-                    if (peek()[0] == '\n') {
-                        buf = _BZ2File_read(1);
-                        mustBreak = true;
-                    }
-                    line.replace(line.length() - 1, line.length(), new String(
-                            "\n"));
-                    mustBreak = true;
-                } else if ((char) buf[0] == '\n'
-                        || (size > -1 && (readSize >= size))) {
-                    mustBreak = true;
-                }
-
-            } else {
-                if ((char) buf[0] == '\n' || (size > -1 && (readSize >= size))) {
-                    mustBreak = true;
-                }
-            }
-
-            if (mustBreak) {
-                break;
-            }
-        }
-
-        return new PyString(line.toString());
-    }
-
-    private void addNewlineMarker(String newline) {
-        if (newlines == null) {
-            newlines = new PyString(newline);
-        } else {
-            if (newlines instanceof PyString) {
-                if (!newlines.equals(new PyString(newline))) {
-                    newlines = new PyTuple(newlines, new PyString(newline));
-                }
-            } else {
-                if (!newlines.__contains__(new PyString(newline))) {
-                    newlines = newlines.__add__(new PyTuple(new PyString(
-                            newline)));
-                }
-            }
-        }
+        return new PyString(buffer.readline(size));
     }
 
     @ExposedMethod
     public PyList BZ2File_readlines(PyObject[] args, String[] kwds) {
         checkInIterMode();
+        checkReadBufferInit();
 
         // make sure file data valid
-        if (fileData == null) {
+        if (buffer == null || buffer.closed()) {
             throw Py.ValueError("Cannot call readlines() on a closed file");
         }
-
         PyList lineList = new PyList();
 
         PyString line = null;
@@ -315,10 +230,15 @@
     }
 
     private void checkInIterMode() {
-        if (fileMode.contains("r")) {
-            if (inIterMode) {
-                throw Py.ValueError("Cannot mix iteration and reads");
-            }
+        if (inReadMode && inIterMode) {
+            throw Py.ValueError("Cannot mix iteration and reads");
+        }
+    }
+
+    private void checkReadBufferInit() {
+        if (inReadMode && needReadBufferInit) {
+            makeReadBuffer();
+            needReadBufferInit = false;
         }
     }
 
@@ -329,44 +249,74 @@
 
     @ExposedMethod
     public void BZ2File_seek(PyObject[] args, String[] kwds) {
+        if (!inReadMode) {
+            Py.IOError("seek works only while reading");
+        }
+
         ArgParser ap = new ArgParser("seek", args, kwds, new String[] {
                 "offset", "whence" }, 1);
 
+        checkReadBufferInit();
+
         int newOffset = ap.getInt(0);
         int whence = ap.getInt(1, 0);
 
         // normalise offset
-        int finalOffset = 0;
+        long currentPos = buffer.tell();
+
+        long finalOffset = 0;
         switch (whence) {
         case 0: // offset from start of file
-            if (newOffset > fileData.length) {
-                finalOffset = fileData.length;
-            } else {
-                finalOffset = newOffset;
-            }
+            finalOffset = newOffset;
             break;
         case 1: // move relative to current position
-            finalOffset = offset + newOffset;
+            finalOffset = currentPos + newOffset;
+
             break;
         case 2: // move relative to end of file
-            finalOffset = fileData.length + newOffset;
+            long fileSize = currentPos;
+
+            // in order to seek from the end of the stream we need to fully read
+            // the decompressed stream to get the size
+            for (;;) {
+                final String data = buffer.read(IOBase.DEFAULT_BUFFER_SIZE);
+                if (data.isEmpty()) {
+                    break;
+                }
+                fileSize += data.length();
+            }
+
+            finalOffset = fileSize + newOffset;
+
+            // close and reset the buffer
+            buffer.close();
+            makeReadBuffer();
+
+            break;
         }
 
         if (finalOffset < 0) {
             finalOffset = 0;
-        } else {
-            if (finalOffset > fileData.length) {
-                finalOffset = fileData.length;
-            }
+        }
+
+        // can't seek backwards so close and reopen the stream at the start
+        if (whence != 2 && finalOffset < currentPos) {
+            buffer.close();
+            makeReadBuffer();
         }
 
         // seek operation
-        offset = finalOffset;
+        buffer.seek(finalOffset, 0);
     }
 
     @ExposedMethod
     public PyLong BZ2File_tell() {
-        return new PyLong(offset);
+        checkReadBufferInit();
+        if (buffer == null) {
+            return Py.newLong(0);
+        } else {
+            return Py.newLong(buffer.tell());
+        }
     }
 
     @ExposedMethod
@@ -380,16 +330,8 @@
         if (data.getType() == PyNone.TYPE) {
             throw Py.TypeError("Expecting str argument");
         }
-        byte[] buf = ap.getString(0).getBytes();
 
-        try {
-            synchronized (this) {
-                writeStream.write(buf);
-            }
-        } catch (IOException e) {
-
-            throw Py.IOError(e.getMessage());
-        }
+        buffer.write(ap.getString(0));
     }
 
     @ExposedMethod
@@ -411,10 +353,11 @@
     }
 
     private void checkFileWritable() {
-        if (fileMode.contains("r")) {
+        if (inReadMode) {
             throw Py.IOError("File in read-only mode");
         }
-        if (writeStream == null) {
+
+        if (buffer == null || buffer.closed()) {
             throw Py.ValueError("Stream closed");
         }
     }
@@ -442,12 +385,12 @@
 
     @ExposedMethod
     public PyObject BZ2File___enter__() {
-        if (fileMode.contains("w")) {
-            if (writeStream == null) {
+        if (inWriteMode) {
+            if (buffer == null) {
                 throw Py.ValueError("Stream closed");
             }
-        } else if (fileMode.contains("r")) {
-            if (fileData == null) {
+        } else if (inReadMode && !needReadBufferInit) {
+            if (buffer == null || buffer.closed()) {
                 throw Py.ValueError("Stream closed");
             }
         }
@@ -461,4 +404,58 @@
         BZ2File_close();
         return false;
     }
+
+    private static class SkippableStreamIO extends StreamIO {
+        private long position = 0;
+
+        public SkippableStreamIO(InputStream inputStream, boolean closefd) {
+            super(inputStream, closefd);
+        }
+
+        public SkippableStreamIO(OutputStream outputStream, boolean closefd) {
+            super(outputStream, closefd);
+        }
+
+        @Override
+        public int readinto(ByteBuffer buf) {
+            int bytesRead = 0;
+            try {
+                bytesRead = super.readinto(buf);
+            } catch (PyException pyex) {
+                // translate errors on read of decompressed stream to EOFError
+                throw Py.EOFError(pyex.value.asStringOrNull());
+            }
+
+            position += bytesRead;
+            return bytesRead;
+        }
+
+        @Override
+        public long tell() {
+            return position;
+        }
+
+        @Override
+        public long seek(long offset, int whence) {
+            long skipBytes = offset - position;
+            if (whence != 0 || skipBytes < 0) {
+                throw Py.IOError("can only seek forward");
+            }
+
+            if (skipBytes == 0) {
+                return position;
+            } else {
+                long skipped = 0;
+                try {
+                    skipped = asInputStream().skip(skipBytes);
+                } catch (IOException ex) {
+                    throw Py.IOError(ex);
+                }
+                long newPosition = position + skipped;
+                position = newPosition;
+
+                return newPosition;
+            }
+        }
+    }
 }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 07:13:26 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 07:13:26 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Update_IDEA_for_extlibs?=
Message-ID: <3gQc4k2CVDz7LjR@mail.python.org>

http://hg.python.org/jython/rev/6679b496a4de
changeset:   7234:6679b496a4de
user:        Jim Baker <jim.baker at rackspace.com>
date:        Fri May 09 22:52:23 2014 -0600
summary:
  Update IDEA for extlibs

files:
  .idea/libraries/extlibs.xml |  41 +-----------------------
  Jython27.iml                |   2 +-
  2 files changed, 3 insertions(+), 40 deletions(-)


diff --git a/.idea/libraries/extlibs.xml b/.idea/libraries/extlibs.xml
--- a/.idea/libraries/extlibs.xml
+++ b/.idea/libraries/extlibs.xml
@@ -1,47 +1,10 @@
 <component name="libraryTable">
   <library name="extlibs">
     <CLASSES>
-      <root url="jar://$PROJECT_DIR$/extlibs/mysql-connector-java-5.1.6.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-ppc64-Linux.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/stringtemplate-3.2.1.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jarjar-0.7.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-x86_64-Windows.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/livetribe-jsr223-2.0.6.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-i386-FreeBSD.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/antlr-3.1.3.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-Darwin.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/libreadline-java-0.8.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-i386-OpenBSD.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jnr-netdb-0.4.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-i386-Linux.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/profile.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-s390x-Linux.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-ppc-AIX.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/antlr-runtime-3.1.3.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/xercesImpl-2.11.0.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/xml-apis-2.11.0.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/servlet-api-2.5.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jnr-posix.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-sparcv9-SunOS.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-x86_64-OpenBSD.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/antlr-2.7.7.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-i386-SunOS.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-i386-Windows.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-x86_64-SunOS.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jline-1.0.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-ppc-Linux.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-x86_64-Linux.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-x86_64-FreeBSD.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/postgresql-8.3-603.jdbc4.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/jffi-sparc-SunOS.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/junit-4.10.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/guava-13.0.1.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/asm-4.0.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/asm-commons-4.0.jar!/" />
-      <root url="jar://$PROJECT_DIR$/extlibs/asm-util-4.0.jar!/" />
+      <root url="file://$PROJECT_DIR$/extlibs" />
     </CLASSES>
     <JAVADOC />
     <SOURCES />
+    <jarDirectory url="file://$PROJECT_DIR$/extlibs" recursive="false" />
   </library>
 </component>
\ No newline at end of file
diff --git a/Jython27.iml b/Jython27.iml
--- a/Jython27.iml
+++ b/Jython27.iml
@@ -7,7 +7,6 @@
     </content>
     <orderEntry type="inheritedJdk" />
     <orderEntry type="sourceFolder" forTests="false" />
-    <orderEntry type="library" name="extlibs" level="project" />
     <orderEntry type="module-library">
       <library>
         <CLASSES>
@@ -17,6 +16,7 @@
         <SOURCES />
       </library>
     </orderEntry>
+    <orderEntry type="library" name="extlibs" level="project" />
   </component>
 </module>
 

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 07:13:28 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 07:13:28 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Upgrade_JNR_and_JFFI_jars_t?=
	=?utf-8?q?o_latest_versions?=
Message-ID: <3gQc4m0tkqz7Ljf@mail.python.org>

http://hg.python.org/jython/rev/01460e803ef3
changeset:   7235:01460e803ef3
user:        Jim Baker <jim.baker at rackspace.com>
date:        Fri May 09 22:52:45 2014 -0600
summary:
  Upgrade JNR and JFFI jars to latest versions

files:
  build.xml                                            |   58 ++-------
  extlibs/jffi-1.2.6.jar                               |  Bin 
  extlibs/jffi-1.2.7-native.jar                        |  Bin 
  extlibs/jffi-1.2.7.jar                               |  Bin 
  extlibs/jffi-Darwin.jar                              |  Bin 
  extlibs/jffi-arm-Linux.jar                           |  Bin 
  extlibs/jffi-i386-FreeBSD.jar                        |  Bin 
  extlibs/jffi-i386-Linux.jar                          |  Bin 
  extlibs/jffi-i386-OpenBSD.jar                        |  Bin 
  extlibs/jffi-i386-SunOS.jar                          |  Bin 
  extlibs/jffi-i386-Windows.jar                        |  Bin 
  extlibs/jffi-ppc-AIX.jar                             |  Bin 
  extlibs/jffi-ppc-Linux.jar                           |  Bin 
  extlibs/jffi-ppc64-Linux.jar                         |  Bin 
  extlibs/jffi-s390x-Linux.jar                         |  Bin 
  extlibs/jffi-sparc-SunOS.jar                         |  Bin 
  extlibs/jffi-sparcv9-SunOS.jar                       |  Bin 
  extlibs/jffi-x86_64-FreeBSD.jar                      |  Bin 
  extlibs/jffi-x86_64-Linux.jar                        |  Bin 
  extlibs/jffi-x86_64-OpenBSD.jar                      |  Bin 
  extlibs/jffi-x86_64-SunOS.jar                        |  Bin 
  extlibs/jffi-x86_64-Windows.jar                      |  Bin 
  extlibs/jnr-constants-0.8.4.jar                      |  Bin 
  extlibs/jnr-constants-0.8.5.jar                      |  Bin 
  extlibs/jnr-ffi-0.7.10.jar                           |  Bin 
  extlibs/jnr-ffi-1.0.10.jar                           |  Bin 
  extlibs/jnr-netdb-1.1.1.jar                          |  Bin 
  extlibs/jnr-netdb-1.1.2.jar                          |  Bin 
  extlibs/jnr-posix-2.4.0.jar                          |  Bin 
  extlibs/jnr-posix-3.0.1.jar                          |  Bin 
  src/org/python/modules/posix/PythonPOSIXHandler.java |    8 +-
  31 files changed, 19 insertions(+), 47 deletions(-)


diff --git a/build.xml b/build.xml
--- a/build.xml
+++ b/build.xml
@@ -154,29 +154,12 @@
             <pathelement path="${extlibs.dir}/asm-util-4.0.jar" />
             <pathelement path="${extlibs.dir}/guava-13.0.1.jar" />
             <pathelement path="${extlibs.dir}/icu4j-52_1.jar" />
-            <pathelement path="${extlibs.dir}/jffi-arm-Linux.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-Darwin.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-i386-FreeBSD.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-i386-Linux.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-i386-OpenBSD.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-i386-SunOS.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-i386-Windows.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-ppc-AIX.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-ppc-Linux.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-ppc64-Linux.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-s390x-Linux.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-sparc-SunOS.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-sparcv9-SunOS.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-x86_64-FreeBSD.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-x86_64-Linux.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-x86_64-OpenBSD.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-x86_64-SunOS.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-x86_64-Windows.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-1.2.6.jar"/>
-            <pathelement path="${extlibs.dir}/jnr-ffi-0.7.10.jar"/>
-            <pathelement path="${extlibs.dir}/jnr-netdb-1.1.1.jar"/>
-            <pathelement path="${extlibs.dir}/jnr-posix-2.4.0.jar"/>
-            <pathelement path="${extlibs.dir}/jnr-constants-0.8.4.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-1.2.7.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-1.2.7-native.jar"/>
+            <pathelement path="${extlibs.dir}/jnr-constants-0.8.5.jar"/>
+            <pathelement path="${extlibs.dir}/jnr-ffi-1.0.10.jar"/>
+            <pathelement path="${extlibs.dir}/jnr-netdb-1.1.2.jar"/>
+            <pathelement path="${extlibs.dir}/jnr-posix-3.0.1.jar"/>
             <pathelement path="${extlibs.dir}/jline-1.0.jar" />
         </path>
 
@@ -593,29 +576,12 @@
             <rule pattern="io.netty.**" result="org.python.netty. at 1"/>
             <zipfileset src="extlibs/netty-transport-4.0.18.Final.jar"/>
             <rule pattern="io.netty.**" result="org.python.netty. at 1"/>
-            <zipfileset src="extlibs/jffi-arm-Linux.jar"/>
-            <zipfileset src="extlibs/jffi-Darwin.jar"/>
-            <zipfileset src="extlibs/jffi-i386-FreeBSD.jar"/>
-            <zipfileset src="extlibs/jffi-i386-Linux.jar"/>
-            <zipfileset src="extlibs/jffi-i386-OpenBSD.jar"/>
-            <zipfileset src="extlibs/jffi-i386-SunOS.jar"/>
-            <zipfileset src="extlibs/jffi-i386-Windows.jar"/>
-            <zipfileset src="extlibs/jffi-ppc-AIX.jar"/>
-            <zipfileset src="extlibs/jffi-ppc-Linux.jar"/>
-            <zipfileset src="extlibs/jffi-ppc64-Linux.jar"/>
-            <zipfileset src="extlibs/jffi-s390x-Linux.jar"/>
-            <zipfileset src="extlibs/jffi-sparc-SunOS.jar"/>
-            <zipfileset src="extlibs/jffi-sparcv9-SunOS.jar"/>
-            <zipfileset src="extlibs/jffi-x86_64-FreeBSD.jar"/>
-            <zipfileset src="extlibs/jffi-x86_64-Linux.jar"/>
-            <zipfileset src="extlibs/jffi-x86_64-OpenBSD.jar"/>
-            <zipfileset src="extlibs/jffi-x86_64-SunOS.jar"/>
-            <zipfileset src="extlibs/jffi-x86_64-Windows.jar"/>
-            <zipfileset src="extlibs/jffi-1.2.6.jar"/>
-            <zipfileset src="${extlibs.dir}/jnr-ffi-0.7.10.jar"/>
-	    <zipfileset src="${extlibs.dir}/jnr-netdb-1.1.1.jar"/>
-	    <zipfileset src="${extlibs.dir}/jnr-posix-2.4.0.jar"/>
-            <zipfileset src="${extlibs.dir}/jnr-constants-0.8.4.jar"/>
+            <zipfileset src="${extlibs.dir}/jffi-1.2.7.jar"/>
+            <zipfileset src="${extlibs.dir}/jffi-1.2.7-native.jar"/>
+            <zipfileset src="${extlibs.dir}/jnr-constants-0.8.5.jar"/>
+            <zipfileset src="${extlibs.dir}/jnr-ffi-1.0.10.jar"/>
+	    <zipfileset src="${extlibs.dir}/jnr-netdb-1.1.2.jar"/>
+	    <zipfileset src="${extlibs.dir}/jnr-posix-3.0.1.jar"/>
             <zipfileset src="extlibs/xml-apis-2.11.0.jar" excludes="META-INF/services/*"/>
             <zipfileset src="extlibs/xercesImpl-2.11.0.jar" excludes="META-INF/services/*"/>
             <rule pattern="org.apache.xml.**" result="org.python.apache.xml. at 1"/>
diff --git a/extlibs/jffi-1.2.6.jar b/extlibs/jffi-1.2.6.jar
deleted file mode 100644
index a6b2be6c39798f0fc5958aa309924506b74f0f9f..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-1.2.7-native.jar b/extlibs/jffi-1.2.7-native.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0b7404c256d1d4277b58c9c559bf24938f11404e
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-1.2.7.jar b/extlibs/jffi-1.2.7.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..68c8db05c248ab54aba86d03d3c7904334468770
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-Darwin.jar b/extlibs/jffi-Darwin.jar
deleted file mode 100644
index 009191ac8c0dcaf63267ea0103e73ea2a9895e43..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-arm-Linux.jar b/extlibs/jffi-arm-Linux.jar
deleted file mode 100644
index b3c5d977c1b7ad3fc196bfe2cd24fe56129dc8f9..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-i386-FreeBSD.jar b/extlibs/jffi-i386-FreeBSD.jar
deleted file mode 100644
index 8235a3484884243b1d17d5e19340e4166f3b0b55..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-i386-Linux.jar b/extlibs/jffi-i386-Linux.jar
deleted file mode 100644
index 7e23b400cce5e3b4861141ee4f10730fbcbdb5de..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-i386-OpenBSD.jar b/extlibs/jffi-i386-OpenBSD.jar
deleted file mode 100644
index 8235a3484884243b1d17d5e19340e4166f3b0b55..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-i386-SunOS.jar b/extlibs/jffi-i386-SunOS.jar
deleted file mode 100644
index 68bcacc39ac49a01929be37ec67c6784612bc966..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-i386-Windows.jar b/extlibs/jffi-i386-Windows.jar
deleted file mode 100644
index 90a6baecf76870212dc4f8cb7aa56e0962d1f858..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-ppc-AIX.jar b/extlibs/jffi-ppc-AIX.jar
deleted file mode 100644
index 8235a3484884243b1d17d5e19340e4166f3b0b55..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-ppc-Linux.jar b/extlibs/jffi-ppc-Linux.jar
deleted file mode 100644
index 8235a3484884243b1d17d5e19340e4166f3b0b55..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-ppc64-Linux.jar b/extlibs/jffi-ppc64-Linux.jar
deleted file mode 100644
index 8235a3484884243b1d17d5e19340e4166f3b0b55..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-s390x-Linux.jar b/extlibs/jffi-s390x-Linux.jar
deleted file mode 100644
index 8235a3484884243b1d17d5e19340e4166f3b0b55..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-sparc-SunOS.jar b/extlibs/jffi-sparc-SunOS.jar
deleted file mode 100644
index 8235a3484884243b1d17d5e19340e4166f3b0b55..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-sparcv9-SunOS.jar b/extlibs/jffi-sparcv9-SunOS.jar
deleted file mode 100644
index 306788c0810baeccb93f191316844d5d0d9e6254..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-x86_64-FreeBSD.jar b/extlibs/jffi-x86_64-FreeBSD.jar
deleted file mode 100644
index 32824a7e209b57e06808d4de65de4443225a5349..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-x86_64-Linux.jar b/extlibs/jffi-x86_64-Linux.jar
deleted file mode 100644
index 99deb014d458c5a42b5290832b8141acf42212dd..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-x86_64-OpenBSD.jar b/extlibs/jffi-x86_64-OpenBSD.jar
deleted file mode 100644
index 8235a3484884243b1d17d5e19340e4166f3b0b55..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-x86_64-SunOS.jar b/extlibs/jffi-x86_64-SunOS.jar
deleted file mode 100644
index b8240422d442b4bcf0e95d6b6a0a02714aeac827..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-x86_64-Windows.jar b/extlibs/jffi-x86_64-Windows.jar
deleted file mode 100644
index 667e48db8193809b21118cff14b23184d558821a..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-constants-0.8.4.jar b/extlibs/jnr-constants-0.8.4.jar
deleted file mode 100644
index f0ff6bb1a1796560c09121b4366a5d04032156af..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-constants-0.8.5.jar b/extlibs/jnr-constants-0.8.5.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..fe11c5752485174311ae271f4ca8ce76a496ff63
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-ffi-0.7.10.jar b/extlibs/jnr-ffi-0.7.10.jar
deleted file mode 100644
index 9e2e64f276334bed30c62d12823d5088b5672425..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-ffi-1.0.10.jar b/extlibs/jnr-ffi-1.0.10.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..0e90154fcbf11a4ce289ec5b93cfe975cee6fa50
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-netdb-1.1.1.jar b/extlibs/jnr-netdb-1.1.1.jar
deleted file mode 100644
index 731f6ec0fdc5c420ec55edec6c28092b06c70e32..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-netdb-1.1.2.jar b/extlibs/jnr-netdb-1.1.2.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..d082053606b4718b13648a1ca17c0ffc43eb66a6
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-posix-2.4.0.jar b/extlibs/jnr-posix-2.4.0.jar
deleted file mode 100644
index 36f3fc1d217091941a1079e9ebccc70094a5ede6..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-posix-3.0.1.jar b/extlibs/jnr-posix-3.0.1.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..94f9bf3a7d19baf0071e12eb24f14a0d0cef6c0b
GIT binary patch
[stripped]
diff --git a/src/org/python/modules/posix/PythonPOSIXHandler.java b/src/org/python/modules/posix/PythonPOSIXHandler.java
--- a/src/org/python/modules/posix/PythonPOSIXHandler.java
+++ b/src/org/python/modules/posix/PythonPOSIXHandler.java
@@ -20,7 +20,13 @@
  */
 public class PythonPOSIXHandler implements POSIXHandler {
 
-	public void error(Errno error, String extraData) {
+    public void error(Errno error, String extraData) {
+        // XXX: extraData (filename) could have been unicode!
+        // http://bugs.jython.org/issue1825
+        throw Py.OSError(error, Py.newString(extraData));
+    }
+
+    public void error(Errno error, String methodName, String extraData) {
         // XXX: extraData (filename) could have been unicode!
         // http://bugs.jython.org/issue1825
         throw Py.OSError(error, Py.newString(extraData));

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 17:38:01 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 17:38:01 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Make_binascii_methods_take_?=
 =?utf-8?q?objects_implementing_the_BufferProtocol_interface=2E?=
Message-ID: <3gQsxP5S59z7Ljb@mail.python.org>

http://hg.python.org/jython/rev/355bb70327e0
changeset:   7236:355bb70327e0
user:        Indra Talip <indra.talip at gmail.com>
date:        Sat May 10 15:52:54 2014 +1000
summary:
  Make binascii methods take objects implementing the BufferProtocol interface.
Change return type on various binascii methods to PyString to prevent automatic
conversion to PyUnicode.

Unlike the other methods converted binascii.b2a_qp does not operate on the PyBuffer
object, rather it converts it to a java String to reuse the existing behaviour.
binascii.b2a_qp could probably be made more efficient if it is made to operate on
a PyBuffer directly rather than converting to a java String first.

files:
  src/org/python/modules/binascii.java |  668 ++++++++------
  1 files changed, 379 insertions(+), 289 deletions(-)


diff --git a/src/org/python/modules/binascii.java b/src/org/python/modules/binascii.java
--- a/src/org/python/modules/binascii.java
+++ b/src/org/python/modules/binascii.java
@@ -12,7 +12,10 @@
 import java.util.regex.Pattern;
 
 import org.python.core.ArgParser;
+import org.python.core.BufferProtocol;
 import org.python.core.Py;
+import org.python.core.PyBUF;
+import org.python.core.PyBuffer;
 import org.python.core.PyException;
 import org.python.core.PyObject;
 import org.python.core.PyString;
@@ -272,65 +275,70 @@
      * binary data. Lines normally contain 45 (binary) bytes, except for the
      * last line. Line data may be followed by whitespace.
      */
-    public static PyString a2b_uu(String ascii_data) {
+    public static PyString a2b_uu(BufferProtocol bp) {
         int leftbits = 0;
         int leftchar = 0;
 
-        if (ascii_data.length() == 0)
-            return new PyString("");
+        StringBuilder bin_data = new StringBuilder();
+        PyBuffer ascii_data = bp.getBuffer(PyBUF.SIMPLE);
+        try {
+            if (ascii_data.getLen() == 0)
+                return new PyString("");
+
+
+            char this_ch;
+            int i;
+
+            int ascii_len = ascii_data.getLen()-1;
+
+            int bin_len = (ascii_data.intAt(0) - ' ') & 077;
+
+            for (i = 0; bin_len > 0 && ascii_len > 0; i++, ascii_len--) {
+                this_ch = (char) ascii_data.intAt(i+1);
+                if (this_ch == '\n' || this_ch == '\r' || ascii_len <= 0) {
+                    // Whitespace. Assume some spaces got eaten at
+                    // end-of-line. (We check this later)
+                    this_ch = 0;
+                } else {
+                    // Check the character for legality
+                    // The 64 in stead of the expected 63 is because
+                    // there are a few uuencodes out there that use
+                    // '@' as zero instead of space.
+                    if ( this_ch < ' ' || this_ch > (' ' + 64)) {
+                        throw new PyException(Error, "Illegal char");
+                    }
+                    this_ch = (char)((this_ch - ' ') & 077);
+                }
+                // Shift it in on the low end, and see if there's
+                // a byte ready for output.
+                leftchar = (leftchar << 6) | (this_ch);
+                leftbits += 6;
+                if (leftbits >= 8) {
+                    leftbits -= 8;
+                    bin_data.append((char)((leftchar >> leftbits) & 0xff));
+                    leftchar &= ((1 << leftbits) - 1);
+                    bin_len--;
+                }
+            }
+
+            // Finally, check that if there's anything left on the line
+            // that it's whitespace only.
+            while (ascii_len-- > 0) {
+                this_ch = (char) ascii_data.intAt(++i);
+                // Extra '@' may be written as padding in some cases
+                if (this_ch != ' ' && this_ch != '@' &&
+                         this_ch != '\n' && this_ch != '\r') {
+                    throw new PyException(Error, "Trailing garbage");
+                }
+            }
         
-        StringBuilder bin_data = new StringBuilder();
-
-        char this_ch;
-        int i;
-
-        int ascii_len = ascii_data.length()-1;
-
-        int bin_len = (ascii_data.charAt(0) - ' ') & 077;
-
-        for (i = 0; bin_len > 0 && ascii_len > 0; i++, ascii_len--) {
-            this_ch = ascii_data.charAt(i+1);
-            if (this_ch == '\n' || this_ch == '\r' || ascii_len <= 0) {
-                // Whitespace. Assume some spaces got eaten at
-                // end-of-line. (We check this later)
-                this_ch = 0;
-            } else {
-                // Check the character for legality
-                // The 64 in stead of the expected 63 is because
-                // there are a few uuencodes out there that use
-                // '@' as zero instead of space.
-                if ( this_ch < ' ' || this_ch > (' ' + 64)) {
-                    throw new PyException(Error, "Illegal char");
-                }
-                this_ch = (char)((this_ch - ' ') & 077);
-            }
-            // Shift it in on the low end, and see if there's
-            // a byte ready for output.
-            leftchar = (leftchar << 6) | (this_ch);
-            leftbits += 6;
-            if (leftbits >= 8) {
-                leftbits -= 8;
-                bin_data.append((char)((leftchar >> leftbits) & 0xff));
-                leftchar &= ((1 << leftbits) - 1);
-                bin_len--;
-            }
+            // finally, if we haven't decoded enough stuff, fill it up with zeros
+            for (; i<bin_len; i++)
+                    bin_data.append((char)0);
+        } finally {
+            ascii_data.release();
         }
         
-        // Finally, check that if there's anything left on the line
-        // that it's whitespace only.
-        while (ascii_len-- > 0) {
-            this_ch = ascii_data.charAt(++i);
-            // Extra '@' may be written as padding in some cases
-            if (this_ch != ' ' && this_ch != '@' &&
-                     this_ch != '\n' && this_ch != '\r') {
-                throw new PyException(Error, "Trailing garbage");
-            }
-        }
-        
-        // finally, if we haven't decoded enough stuff, fill it up with zeros
-        for (; i<bin_len; i++)
-        	bin_data.append((char)0);
-        
         return new PyString(bin_data.toString());
     }
 
@@ -346,36 +354,41 @@
      * is the converted line, including a newline char. The length of
      * <i>data</i> should be at most 45.
      */
-    public static PyString b2a_uu(String bin_data) {
+    public static PyString b2a_uu(BufferProtocol bp) {
         int leftbits = 0;
         char this_ch;
         int leftchar = 0;
 
-        int bin_len = bin_data.length();
-        if (bin_len > 45) {
-            // The 45 is a limit that appears in all uuencode's
-            throw new PyException(Error, "At most 45 bytes at once");
-        }
+        PyBuffer bin_data = bp.getBuffer(PyBUF.SIMPLE);
 
         StringBuilder ascii_data = new StringBuilder();
+        try {
+            int bin_len = bin_data.getLen();
+            if (bin_len > 45) {
+                // The 45 is a limit that appears in all uuencode's
+                throw new PyException(Error, "At most 45 bytes at once");
+            }
 
-        // Store the length */
-        ascii_data.append((char)(' ' + (bin_len & 077)));
+            // Store the length */
+            ascii_data.append((char)(' ' + (bin_len & 077)));
 
-        for (int i = 0; bin_len > 0 || leftbits != 0; i++, bin_len--) {
-            // Shift the data (or padding) into our buffer
-            if (bin_len > 0)    // Data
-                leftchar = (leftchar << 8) | bin_data.charAt(i);
-            else  // Padding
-                leftchar <<= 8;
-            leftbits += 8;
+            for (int i = 0; bin_len > 0 || leftbits != 0; i++, bin_len--) {
+                // Shift the data (or padding) into our buffer
+                if (bin_len > 0)    // Data
+                    leftchar = (leftchar << 8) | (char) bin_data.intAt(i);
+                else  // Padding
+                    leftchar <<= 8;
+                leftbits += 8;
 
-            // See if there are 6-bit groups ready
-            while (leftbits >= 6) {
-                this_ch = (char)((leftchar >> (leftbits-6)) & 0x3f);
-                leftbits -= 6;
-                ascii_data.append((char)(this_ch + ' '));
+                // See if there are 6-bit groups ready
+                while (leftbits >= 6) {
+                    this_ch = (char)((leftchar >> (leftbits-6)) & 0x3f);
+                    leftbits -= 6;
+                    ascii_data.append((char)(this_ch + ' '));
+                }
             }
+        } finally {
+            bin_data.release();
         }
         ascii_data.append('\n'); // Append a courtesy newline
 
@@ -383,8 +396,8 @@
     }
 
 
-    private static int binascii_find_valid(String s, int offset, int num) {
-        int slen = s.length() - offset;
+    private static int binascii_find_valid(PyBuffer b, int offset, int num) {
+        int blen = b.getLen() - offset;
 
         /* Finds & returns the (num+1)th
         ** valid character for base64, or -1 if none.
@@ -392,8 +405,8 @@
 
         int ret = -1;
 
-        while ((slen > 0) && (ret == -1)) {
-            int c = s.charAt(offset);
+        while ((blen > 0) && (ret == -1)) {
+            int c = b.intAt(offset);
             short b64val = table_a2b_base64[c & 0x7f];
             if (((c <= 0x7f) && (b64val != -1)) ) {
                 if (num == 0)
@@ -402,7 +415,7 @@
             }
 
             offset++;
-            slen--;
+            blen--;
         }
         return ret;
     }
@@ -417,52 +430,57 @@
      * Convert a block of base64 data back to binary and return the
      * binary data. More than one line may be passed at a time.
      */
-    public static PyString a2b_base64(String ascii_data) {
+    public static PyString a2b_base64(BufferProtocol bp) {
         int leftbits = 0;
         char this_ch;
         int leftchar = 0;
         int quad_pos = 0;
 
-        int ascii_len = ascii_data.length();
+        PyBuffer ascii_data = bp.getBuffer(PyBUF.SIMPLE);
+        int ascii_len = ascii_data.getLen();
 
         int bin_len = 0;
         StringBuilder bin_data = new StringBuilder();
 
-        for(int i = 0; ascii_len > 0 ; ascii_len--, i++) {
-            // Skip some punctuation
-            this_ch = ascii_data.charAt(i);
-            if (this_ch > 0x7F || this_ch == '\r' ||
-                      this_ch == '\n' || this_ch == ' ')
-                continue;
+        try {
+            for(int i = 0; ascii_len > 0 ; ascii_len--, i++) {
+                // Skip some punctuation
+                this_ch = (char) ascii_data.intAt(i);
+                if (this_ch > 0x7F || this_ch == '\r' ||
+                          this_ch == '\n' || this_ch == ' ')
+                    continue;
 
-            if (this_ch == BASE64_PAD) {
-                if (quad_pos < 2 || (quad_pos == 2 &&
-                         binascii_find_valid(ascii_data, i, 1) != BASE64_PAD))
+                if (this_ch == BASE64_PAD) {
+                    if (quad_pos < 2 || (quad_pos == 2 &&
+                             binascii_find_valid(ascii_data, i, 1) != BASE64_PAD))
+                        continue;
+                    else {
+                        // A pad sequence means no more input.
+                        // We've already interpreted the data
+                        // from the quad at this point.
+                        leftbits = 0;
+                        break;
+                    }
+                }
+
+                short this_v = table_a2b_base64[this_ch];
+                if (this_v == -1)
                     continue;
-                else {
-                    // A pad sequence means no more input.
-                    // We've already interpreted the data
-                    // from the quad at this point.
-                    leftbits = 0;
-                    break;
+
+                // Shift it in on the low end, and see if there's
+                // a byte ready for output.
+                quad_pos = (quad_pos + 1) & 0x03;
+                leftchar = (leftchar << 6) | (this_v);
+                leftbits += 6;
+                if (leftbits >= 8) {
+                    leftbits -= 8;
+                    bin_data.append((char)((leftchar >> leftbits) & 0xff));
+                    bin_len++;
+                    leftchar &= ((1 << leftbits) - 1);
                 }
             }
-
-            short this_v = table_a2b_base64[this_ch];
-            if (this_v == -1)
-                continue;
-
-            // Shift it in on the low end, and see if there's
-            // a byte ready for output.
-            quad_pos = (quad_pos + 1) & 0x03;
-            leftchar = (leftchar << 6) | (this_v);
-            leftbits += 6;
-            if (leftbits >= 8) {
-                leftbits -= 8;
-                bin_data.append((char)((leftchar >> leftbits) & 0xff));
-                bin_len++;
-                leftchar &= ((1 << leftbits) - 1);
-            }
+        } finally {
+            ascii_data.release();
         }
         // Check that no bits are left
         if (leftbits != 0) {
@@ -482,29 +500,34 @@
      * Convert binary data to a line of ASCII characters in base64 coding.
      * The return value is the converted line, including a newline char.
      */
-    public static PyString b2a_base64(String bin_data) {
+    public static PyString b2a_base64(BufferProtocol bp) {
         int leftbits = 0;
         char this_ch;
         int leftchar = 0;
 
         StringBuilder ascii_data = new StringBuilder();
 
-        int bin_len = bin_data.length();
-        if (bin_len > BASE64_MAXBIN) {
-            throw new PyException(Error,"Too much data for base64 line");
-        }
+        PyBuffer bin_data = bp.getBuffer(PyBUF.SIMPLE);
+        try {
+            int bin_len = bin_data.getLen();
+            if (bin_len > BASE64_MAXBIN) {
+                throw new PyException(Error,"Too much data for base64 line");
+            }
 
-        for (int i = 0; bin_len > 0 ; bin_len--, i++) {
-            // Shift the data into our buffer
-            leftchar = (leftchar << 8) | bin_data.charAt(i);
-            leftbits += 8;
+            for (int i = 0; bin_len > 0 ; bin_len--, i++) {
+                // Shift the data into our buffer
+                leftchar = (leftchar << 8) | (char) bin_data.intAt(i); //charAt(i);
+                leftbits += 8;
 
-            // See if there are 6-bit groups ready
-            while (leftbits >= 6) {
-                this_ch = (char)((leftchar >> (leftbits-6)) & 0x3f);
-                leftbits -= 6;
-                ascii_data.append((char)table_b2a_base64[this_ch]);
+                // See if there are 6-bit groups ready
+                while (leftbits >= 6) {
+                    this_ch = (char)((leftchar >> (leftbits-6)) & 0x3f);
+                    leftbits -= 6;
+                    ascii_data.append((char)table_b2a_base64[this_ch]);
+                }
             }
+        } finally {
+            bin_data.release();
         }
         if (leftbits == 2) {
             ascii_data.append((char)table_b2a_base64[(leftchar&3) << 4]);
@@ -530,39 +553,43 @@
      * binary bytes, or (in case of the last portion of the binhex4 data)
      * have the remaining bits zero.
      */
-    public static PyTuple a2b_hqx(String ascii_data) {
+    public static PyTuple a2b_hqx(BufferProtocol bp) {
         int leftbits = 0;
         char this_ch;
         int leftchar = 0;
 
         boolean done = false;
-
-        int len = ascii_data.length();
+        PyBuffer ascii_data = bp.getBuffer(PyBUF.SIMPLE);
+        int len = ascii_data.getLen();
 
         StringBuilder bin_data = new StringBuilder();
 
-        for(int i = 0; len > 0 ; len--, i++) {
-            // Get the byte and look it up
-            this_ch = (char) table_a2b_hqx[ascii_data.charAt(i)];
-            if (this_ch == SKIP)
-                continue;
-            if (this_ch == FAIL) {
-                throw new PyException(Error, "Illegal char");
+        try {
+            for(int i = 0; len > 0 ; len--, i++) {
+                // Get the byte and look it up
+                this_ch = (char) table_a2b_hqx[ascii_data.intAt(i)];
+                if (this_ch == SKIP)
+                    continue;
+                if (this_ch == FAIL) {
+                    throw new PyException(Error, "Illegal char");
+                }
+                if (this_ch == DONE) {
+                    // The terminating colon
+                    done = true;
+                    break;
+                }
+
+                // Shift it into the buffer and see if any bytes are ready
+                leftchar = (leftchar << 6) | (this_ch);
+                leftbits += 6;
+                if (leftbits >= 8) {
+                    leftbits -= 8;
+                    bin_data.append((char)((leftchar >> leftbits) & 0xff));
+                    leftchar &= ((1 << leftbits) - 1);
+                }
             }
-            if (this_ch == DONE) {
-                // The terminating colon
-                done = true;
-                break;
-            }
-
-            // Shift it into the buffer and see if any bytes are ready
-            leftchar = (leftchar << 6) | (this_ch);
-            leftbits += 6;
-            if (leftbits >= 8) {
-                leftbits -= 8;
-                bin_data.append((char)((leftchar >> leftbits) & 0xff));
-                leftchar &= ((1 << leftbits) - 1);
-            }
+        } finally {
+            ascii_data.release();
         }
 
         if (leftbits != 0 && !done) {
@@ -570,7 +597,7 @@
                                   "String has incomplete number of bytes");
         }
 
-        return new PyTuple(Py.java2py(bin_data.toString()), Py.newInteger(done ? 1 : 0));
+        return new PyTuple(new PyString(bin_data.toString()), Py.newInteger(done ? 1 : 0));
     }
 
 
@@ -582,37 +609,42 @@
      * Perform binhex4 style RLE-compression on <i>data</i> and return the
      * result.
      */
-    static public String rlecode_hqx(String in_data) {
-        int len = in_data.length();
+    static public PyString rlecode_hqx(BufferProtocol bp) {
+        PyBuffer in_data = bp.getBuffer(PyBUF.SIMPLE);
+        int len = in_data.getLen();
 
         StringBuilder out_data = new StringBuilder();
 
-        for (int in=0; in < len; in++) {
-            char ch = in_data.charAt(in);
-            if (ch == RUNCHAR) {
-                // RUNCHAR. Escape it.
-                out_data.append(RUNCHAR);
-                out_data.append(0);
-            } else {
-                // Check how many following are the same
-                int inend;
-                for (inend=in+1; inend < len &&
-                                 in_data.charAt(inend) == ch &&
-                                 inend < in+255; inend++)
-                    ;
-                if (inend - in > 3) {
-                    // More than 3 in a row. Output RLE.
-                    out_data.append(ch);
+        try {
+            for (int in=0; in < len; in++) {
+                char ch = (char) in_data.intAt(in);
+                if (ch == RUNCHAR) {
+                    // RUNCHAR. Escape it.
                     out_data.append(RUNCHAR);
-                    out_data.append((char) (inend-in));
-                    in = inend-1;
+                    out_data.append(0);
                 } else {
-                    // Less than 3. Output the byte itself
-                    out_data.append(ch);
+                    // Check how many following are the same
+                    int inend;
+                    for (inend=in+1; inend < len &&
+                                     (char) in_data.intAt(inend) == ch &&
+                                     inend < in+255; inend++)
+                        ;
+                    if (inend - in > 3) {
+                        // More than 3 in a row. Output RLE.
+                        out_data.append(ch);
+                        out_data.append(RUNCHAR);
+                        out_data.append((char) (inend-in));
+                        in = inend-1;
+                    } else {
+                        // Less than 3. Output the byte itself
+                        out_data.append(ch);
+                    }
                 }
             }
+        } finally {
+            in_data.release();
         }
-        return out_data.toString();
+        return new PyString(out_data.toString());
     }
 
 
@@ -625,25 +657,31 @@
      * resulting string. The argument should already be RLE-coded, and have a
      * length divisible by 3 (except possibly the last fragment).
      */
-    public static PyString b2a_hqx(String bin_data) {
+    public static PyString b2a_hqx(BufferProtocol bp) {
         int leftbits = 0;
         char this_ch;
         int leftchar = 0;
 
-        int len = bin_data.length();
+        PyBuffer bin_data = bp.getBuffer(PyBUF.SIMPLE);
+        int len = bin_data.getLen();
 
         StringBuilder ascii_data = new StringBuilder();
 
-        for(int i = 0; len > 0; len--, i++) {
-            // Shift into our buffer, and output any 6bits ready
-            leftchar = (leftchar << 8) | bin_data.charAt(i);
-            leftbits += 8;
-            while (leftbits >= 6) {
-                this_ch = (char) ((leftchar >> (leftbits-6)) & 0x3f);
-                leftbits -= 6;
-                ascii_data.append((char) table_b2a_hqx[this_ch]);
+        try {
+            for(int i = 0; len > 0; len--, i++) {
+                // Shift into our buffer, and output any 6bits ready
+                leftchar = (leftchar << 8) | (char) bin_data.intAt(i);
+                leftbits += 8;
+                while (leftbits >= 6) {
+                    this_ch = (char) ((leftchar >> (leftbits-6)) & 0x3f);
+                    leftbits -= 6;
+                    ascii_data.append((char) table_b2a_hqx[this_ch]);
+                }
             }
+        } finally {
+            bin_data.release();
         }
+
         // Output a possible runt byte
         if (leftbits != 0) {
             leftchar <<= (6-leftbits);
@@ -667,60 +705,65 @@
      * unless data input data ends in an orphaned repeat indicator, in which
      * case the <tt>Incomplete</tt> exception is raised.
      */
-    static public String rledecode_hqx(String in_data) {
+    static public PyString rledecode_hqx(BufferProtocol bp) {
         char in_byte, in_repeat;
+        
+        PyBuffer in_data = bp.getBuffer(PyBUF.SIMPLE);
+        int in_len = in_data.getLen();
+        int i = 0;
+        
+        StringBuilder out_data = new StringBuilder();
+        try {
+            // Empty string is a special case
+            if (in_len == 0)
+                return Py.EmptyString;
 
-        int in_len = in_data.length();
-        int i = 0;
 
-        // Empty string is a special case
-        if (in_len == 0)
-            return "";
-
-        StringBuilder out_data = new StringBuilder();
-
-        // Handle first byte separately (since we have to get angry
-        // in case of an orphaned RLE code).
-        if (--in_len < 0) throw new PyException(Incomplete);
-        in_byte = in_data.charAt(i++);
-
-        if (in_byte == RUNCHAR) {
+            // Handle first byte separately (since we have to get angry
+            // in case of an orphaned RLE code).
             if (--in_len < 0) throw new PyException(Incomplete);
-            in_repeat = in_data.charAt(i++);
-
-            if (in_repeat != 0) {
-                // Note Error, not Incomplete (which is at the end
-                // of the string only). This is a programmer error.
-                throw new PyException(Error, "Orphaned RLE code at start");
-            }
-            out_data.append(RUNCHAR);
-        } else {
-            out_data.append(in_byte);
-        }
-
-        while (in_len > 0) {
-            if (--in_len < 0) throw new PyException(Incomplete);
-            in_byte = in_data.charAt(i++);
+            in_byte = (char) in_data.intAt(i++);
 
             if (in_byte == RUNCHAR) {
                 if (--in_len < 0) throw new PyException(Incomplete);
-                in_repeat = in_data.charAt(i++);
+                in_repeat = (char) in_data.intAt(i++);
 
-                if (in_repeat == 0) {
-                    // Just an escaped RUNCHAR value
-                    out_data.append(RUNCHAR);
-                } else {
-                    // Pick up value and output a sequence of it
-                    in_byte = out_data.charAt(out_data.length()-1);
-                    while (--in_repeat > 0)
-                        out_data.append(in_byte);
+                if (in_repeat != 0) {
+                    // Note Error, not Incomplete (which is at the end
+                    // of the string only). This is a programmer error.
+                    throw new PyException(Error, "Orphaned RLE code at start");
                 }
+                out_data.append(RUNCHAR);
             } else {
-                // Normal byte
                 out_data.append(in_byte);
             }
+
+            while (in_len > 0) {
+                if (--in_len < 0) throw new PyException(Incomplete);
+                in_byte = (char) in_data.intAt(i++);
+
+                if (in_byte == RUNCHAR) {
+                    if (--in_len < 0) throw new PyException(Incomplete);
+                    in_repeat = (char) in_data.intAt(i++);
+
+                    if (in_repeat == 0) {
+                        // Just an escaped RUNCHAR value
+                        out_data.append(RUNCHAR);
+                    } else {
+                        // Pick up value and output a sequence of it
+                        in_byte = out_data.charAt(out_data.length()-1);
+                        while (--in_repeat > 0)
+                            out_data.append(in_byte);
+                    }
+                } else {
+                    // Normal byte
+                    out_data.append(in_byte);
+                }
+            }
+        } finally {
+            in_data.release();
         }
-        return out_data.toString();
+        return new PyString(out_data.toString());
     }
 
 
@@ -734,13 +777,18 @@
      * Compute the binhex4 crc value of <i>data</i>, starting with an initial
      * <i>crc</i> and returning the result.
      */
-    public static int crc_hqx(String bin_data, int crc) {
-        int len = bin_data.length();
+    public static int crc_hqx(BufferProtocol bp, int crc) {
+        PyBuffer bin_data = bp.getBuffer(PyBUF.SIMPLE);
+        int len = bin_data.getLen();
         int i = 0;
 
-        while(len-- > 0) {
-            crc=((crc<<8)&0xff00) ^
-                       crctab_hqx[((crc>>8)&0xff)^bin_data.charAt(i++)];
+        try {
+            while(len-- > 0) {
+                crc=((crc<<8)&0xff00) ^
+                           crctab_hqx[((crc>>8)&0xff)^ (char) bin_data.intAt(i++)];
+            }
+        } finally {
+            bin_data.release();
         }
 
         return crc;
@@ -804,20 +852,25 @@
 0x2d02ef8dL
 };
 
-    public static int crc32(String bin_data) {
-        return crc32(bin_data, 0);
+    public static int crc32(BufferProtocol bp) {
+        return crc32(bp, 0);
     }
 
-    public static int crc32(String bin_data, long crc) {
-        int len = bin_data.length();
+    public static int crc32(BufferProtocol bp, long crc) {
+        PyBuffer bin_data = bp.getBuffer(PyBUF.SIMPLE);
+        int len = bin_data.getLen();
 
         crc &= 0xFFFFFFFFL;
         crc = crc ^ 0xFFFFFFFFL;
-        for (int i = 0; i < len; i++) {
-            char ch = bin_data.charAt(i);
-            crc = (int)crc_32_tab[(int) ((crc ^ ch) & 0xffL)] ^ (crc >> 8);
-            /* Note:  (crc >> 8) MUST zero fill on left */
-            crc &= 0xFFFFFFFFL;
+        try {
+            for (int i = 0; i < len; i++) {
+                char ch = (char) bin_data.intAt(i);
+                crc = (int)crc_32_tab[(int) ((crc ^ ch) & 0xffL)] ^ (crc >> 8);
+                /* Note:  (crc >> 8) MUST zero fill on left */
+                crc &= 0xFFFFFFFFL;
+            }
+        } finally {
+            bin_data.release();
         }
         if (crc >= 0x80000000)
             return -(int)(crc+1 & 0xFFFFFFFF);
@@ -834,23 +887,27 @@
         "This function is also available as \"hexlify()\"."
     );
 
-    public static PyString b2a_hex(String argbuf) {
-        int arglen = argbuf.length();
+    public static PyString b2a_hex(BufferProtocol bp) {
+        PyBuffer argbuf = bp.getBuffer(PyBUF.SIMPLE);
+        int arglen = argbuf.getLen();
 
         StringBuilder retbuf = new StringBuilder(arglen*2);
 
-        /* make hex version of string, taken from shamodule.c */
-        for (int i = 0; i < arglen; i++) {
-            char ch = argbuf.charAt(i);
-            retbuf.append(hexdigit[(ch >>> 4) & 0xF]);
-            retbuf.append(hexdigit[ch & 0xF]);
+        try {
+            /* make hex version of string, taken from shamodule.c */
+            for (int i = 0; i < arglen; i++) {
+                char ch = (char) argbuf.intAt(i);
+                retbuf.append(hexdigit[(ch >>> 4) & 0xF]);
+                retbuf.append(hexdigit[ch & 0xF]);
+            }
+        } finally {
+            argbuf.release();
         }
         return new PyString(retbuf.toString());
-
     }
 
 
-    public static PyString hexlify(String argbuf) {
+    public static PyString hexlify(BufferProtocol argbuf) {
         return b2a_hex(argbuf);
     }
 
@@ -864,30 +921,34 @@
     );
 
 
-    public static PyString a2b_hex(String argbuf) {
-        int arglen = argbuf.length();
+    public static PyString a2b_hex(BufferProtocol bp) {
+        PyBuffer argbuf = bp.getBuffer(PyBUF.SIMPLE);
+        int arglen = argbuf.getLen();
 
+        StringBuilder retbuf = new StringBuilder(arglen/2);
         /* XXX What should we do about strings with an odd length?  Should
          * we add an implicit leading zero, or a trailing zero?  For now,
          * raise an exception.
          */
-        if (arglen % 2 != 0)
-            throw Py.TypeError("Odd-length string");
+        try {
+            if (arglen % 2 != 0)
+                throw Py.TypeError("Odd-length string");
 
-        StringBuilder retbuf = new StringBuilder(arglen/2);
-
-        for (int i = 0; i < arglen; i += 2) {
-            int top = Character.digit(argbuf.charAt(i), 16);
-            int bot = Character.digit(argbuf.charAt(i+1), 16);
-            if (top == -1 || bot == -1)
-                throw Py.TypeError("Non-hexadecimal digit found");
-            retbuf.append((char) ((top << 4) + bot));
+            for (int i = 0; i < arglen; i += 2) {
+                int top = Character.digit(argbuf.intAt(i), 16);
+                int bot = Character.digit(argbuf.intAt(i+1), 16);
+                if (top == -1 || bot == -1)
+                    throw Py.TypeError("Non-hexadecimal digit found");
+                retbuf.append((char) ((top << 4) + bot));
+            }
+        } finally {
+            argbuf.release();
         }
         return new PyString(retbuf.toString());
     }
 
 
-    public static PyString unhexlify(String argbuf) {
+    public static PyString unhexlify(BufferProtocol argbuf) {
         return a2b_hex(argbuf);
     }
 
@@ -920,36 +981,49 @@
     public static PyString a2b_qp(PyObject[] arg, String[] kws)
     {
         ArgParser ap = new ArgParser("a2b_qp", arg, kws, new String[] {"s", "header"});
-        String s = ap.getString(0);
+
+        PyObject pyObject = ap.getPyObject(0);
+        BufferProtocol bp;
+        if (pyObject instanceof BufferProtocol) {
+            bp = (BufferProtocol) pyObject;
+        } else {
+            throw Py.TypeError("expected something conforming to the buffer protocol, got "
+                    + pyObject.getType().fastGetName());
+        }
+
         StringBuilder sb = new StringBuilder();
         boolean header = getIntFlagAsBool(ap, 1, 0, "an integer is required");
 
-        if (header)
-        	s = UNDERSCORE.matcher(s).replaceAll(" ");
-        
-        for (int i=0, m=s.length(); i<m;) {
-        	char c = s.charAt(i++);
-        	if (c == '=') {
-        		if (i < m) {
-        			c = s.charAt(i++);
-        			if (c == '=') {
-        				sb.append(c);
-                                } else if (c == ' ') {
-                                    sb.append("= ");     
-        			} else if ((c >= '0' && c <= '9' || c >= 'A' && c <= 'F') && i < m) {
-        				char nc = s.charAt(i++);
-        				if ((nc >= '0' && nc <= '9' || nc >= 'A' && nc <= 'F')) {
-        					sb.append((char)(Character.digit(c, 16) << 4 | Character.digit(nc, 16)));
-        				} else {
-        					sb.append('=').append(c).append(nc);
-        				}
-        			} else if (c != '\n') {
-        				sb.append('=').append(c);
-        			}
-        		}
-        	} else {
-        		sb.append(c);
-        	}
+        PyBuffer ascii_data = bp.getBuffer(PyBUF.SIMPLE);
+        try {
+            for (int i=0, m=ascii_data.getLen(); i<m;) {
+                    char c = (char) ascii_data.intAt(i++);
+                    if (header && c == '_') {
+                        sb.append(' ');
+                    } else if (c == '=') {
+                            if (i < m) {
+                                    c = (char) ascii_data.intAt(i++);
+                                    if (c == '=') {
+                                            sb.append(c);
+                                    } else if (c == ' ') {
+                                        sb.append("= ");
+                                    } else if ((c >= '0' && c <= '9' || c >= 'A' && c <= 'F') && i < m) {
+                                            char nc = (char) ascii_data.intAt(i++);
+                                            if ((nc >= '0' && nc <= '9' || nc >= 'A' && nc <= 'F')) {
+                                                    sb.append((char)(Character.digit(c, 16) << 4 | Character.digit(nc, 16)));
+                                            } else {
+                                                    sb.append('=').append(c).append(nc);
+                                            }
+                                    } else if (c != '\n') {
+                                            sb.append('=').append(c);
+                                    }
+                            }
+                    } else {
+                            sb.append(c);
+                    }
+            }
+        } finally {
+            ascii_data.release();
         }
         return new PyString(sb.toString());
     }
@@ -965,11 +1039,25 @@
 
     public static PyString b2a_qp(PyObject[] arg, String[] kws) {
         ArgParser ap = new ArgParser("b2a_qp", arg, kws, new String[] {"s", "quotetabs", "istext", "header"});
-        String s = ap.getString(0);
         boolean quotetabs = getIntFlagAsBool(ap, 1, 0, "an integer is required");
         boolean istext = getIntFlagAsBool(ap, 2, 1, "an integer is required");
         boolean header = getIntFlagAsBool(ap, 3, 0, "an integer is required");
 
+        PyObject pyObject = ap.getPyObject(0);
+        BufferProtocol bp;
+        if (pyObject instanceof BufferProtocol) {
+            bp = (BufferProtocol) pyObject;
+        } else {
+            throw Py.TypeError("expected something conforming to the buffer protocol, got "
+                    + pyObject.getType().fastGetName());
+        }
+
+        PyBuffer bin_data = bp.getBuffer(PyBUF.SIMPLE);
+
+        // TODO make this operate on the raw buffer rather converting it to java String.
+        String s = bin_data.toString();
+        bin_data.release();
+
         String lineEnd;
         int pos = s.indexOf('\n');
         if (pos > 0 && s.charAt(pos-1) == '\r') {
@@ -979,11 +1067,13 @@
         	lineEnd = "\n";
         	s = RN_TO_N.matcher(s).replaceAll("\n");
         }
+
         StringBuilder sb = new StringBuilder();
         int count = 0;
+
         for (int i=0, m=s.length(); i<m; i++) {
         	char c = s.charAt(i);
-                
+
                 // RFC 1521 requires that the line ending in a space or tab must have
                 // that trailing character encoded.
                 if (lineEnding(s, lineEnd, i)) {

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 17:38:03 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 17:38:03 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Fix_binascii=2Etest=5Fhqx_b?=
 =?utf-8?q?y_appending_a_0_byte_rather_than_the_integer_0_after?=
Message-ID: <3gQsxR0DTZz7Ljb@mail.python.org>

http://hg.python.org/jython/rev/240d60136e84
changeset:   7237:240d60136e84
user:        Indra Talip <indra.talip at gmail.com>
date:        Sat May 10 15:52:56 2014 +1000
summary:
  Fix binascii.test_hqx by appending a 0 byte rather than the integer 0 after encountering a RUN_CHAR.

files:
  src/org/python/modules/binascii.java |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/src/org/python/modules/binascii.java b/src/org/python/modules/binascii.java
--- a/src/org/python/modules/binascii.java
+++ b/src/org/python/modules/binascii.java
@@ -621,7 +621,7 @@
                 if (ch == RUNCHAR) {
                     // RUNCHAR. Escape it.
                     out_data.append(RUNCHAR);
-                    out_data.append(0);
+                    out_data.append((char) 0);
                 } else {
                     // Check how many following are the same
                     int inend;

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 17:38:04 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 17:38:04 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Fix_b2a=5Fqp_so_that_lines_?=
 =?utf-8?q?with_a_solitary_period_=22=2E=22_are_encoded=2E?=
Message-ID: <3gQsxS1yVMz7Ljh@mail.python.org>

http://hg.python.org/jython/rev/a7b01f4cfb8d
changeset:   7238:a7b01f4cfb8d
user:        Indra Talip <indra.talip at gmail.com>
date:        Sat May 10 15:52:59 2014 +1000
summary:
  Fix b2a_qp so that lines with a solitary period "." are encoded.

files:
  src/org/python/modules/binascii.java |  5 +++++
  1 files changed, 5 insertions(+), 0 deletions(-)


diff --git a/src/org/python/modules/binascii.java b/src/org/python/modules/binascii.java
--- a/src/org/python/modules/binascii.java
+++ b/src/org/python/modules/binascii.java
@@ -1081,6 +1081,11 @@
                     sb.append(lineEnd);
                     if (lineEnd.length() > 1) i++;
                 }
+                else if ((c == '.') && (count == 0) && endOfLine(s, lineEnd, i + 1)) {
+                    // RFC 1521 requires that a solitary "." alone on a line is encoded.
+                    count += 3;
+                    qpEscape(sb, c);
+                }
                 else if ((c == '\t' || c == ' ' ) && endOfLine(s, lineEnd, i + 1)) {
                     count += 3;
                     qpEscape(sb, c);

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 10 17:38:05 2014
From: jython-checkins at python.org (jim.baker)
Date: Sat, 10 May 2014 17:38:05 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Make_b2a=5Fqp_operate_on_th?=
 =?utf-8?q?e_PyBuffer_directly_instead_of_converting_to_a_String=2E?=
Message-ID: <3gQsxT4mHKz7Ljt@mail.python.org>

http://hg.python.org/jython/rev/d3bad92dfc5b
changeset:   7239:d3bad92dfc5b
user:        Indra Talip <indra.talip at gmail.com>
date:        Sat May 10 15:53:02 2014 +1000
summary:
  Make b2a_qp operate on the PyBuffer directly instead of converting to a String.
Basically (re-)port the C Python binascii b2a_qp substituing a PyBuffer and StringBuilder.

files:
  src/org/python/modules/binascii.java |  164 +++++++-------
  1 files changed, 84 insertions(+), 80 deletions(-)


diff --git a/src/org/python/modules/binascii.java b/src/org/python/modules/binascii.java
--- a/src/org/python/modules/binascii.java
+++ b/src/org/python/modules/binascii.java
@@ -1053,93 +1053,97 @@
         }
 
         PyBuffer bin_data = bp.getBuffer(PyBUF.SIMPLE);
+        int datalen = bin_data.getLen();
+        StringBuilder sb = new StringBuilder(datalen);
+        try {
 
-        // TODO make this operate on the raw buffer rather converting it to java String.
-        String s = bin_data.toString();
-        bin_data.release();
+            String lineEnd = "\n";
+            // Work out if line endings should be crlf.
+            for (int i = 0, m = bin_data.getLen(); i < m; i++) {
+                if ('\n' == bin_data.intAt(i)) {
+                    if (i > 0 && '\r' == bin_data.intAt(i-1)) {
+                        lineEnd = "\r\n";
+                    }
+                    break;
+                }
+            }
 
-        String lineEnd;
-        int pos = s.indexOf('\n');
-        if (pos > 0 && s.charAt(pos-1) == '\r') {
-        	lineEnd = "\r\n";
-        	s = N_TO_RN.matcher(s).replaceAll("\r\n");
-        } else {
-        	lineEnd = "\n";
-        	s = RN_TO_N.matcher(s).replaceAll("\n");
+            int count = 0;
+            int MAXLINESIZE = 76;
+
+            int in = 0;
+            while (in < datalen) {
+                char ch = (char) bin_data.intAt(in);
+                if ((ch > 126) ||
+                        (ch == '=') ||
+                        (header && ch == '_') ||
+                        ((ch == '.') && (count == 0) &&
+                         ((in+1 == datalen) || (char) bin_data.intAt(in+1) == '\n' || (char) bin_data.intAt(in+1) == '\r')) ||
+                        (!istext && ((ch == '\r') || (ch == '\n'))) ||
+                        ((ch == '\t' || ch == ' ') && (in + 1 == datalen)) ||
+                        ((ch < 33) &&
+                         (ch != '\r') && (ch != '\n') &&
+                         (quotetabs ||
+                          (!quotetabs && ((ch != '\t') && (ch != ' '))))))
+                {
+                    if ((count + 3 )>= MAXLINESIZE) {
+                        sb.append('=');
+                        sb.append(lineEnd);
+                        count = 0;
+                    }
+                    qpEscape(sb, ch);
+                    in++;
+                    count += 3;
+                }
+                else {
+                    if (istext &&
+                            ((ch == '\n') ||
+                             ((in+1 < datalen) && (ch == '\r') &&
+                              (bin_data.intAt(in+1) == '\n'))))
+                    {
+                        count = 0;
+                        /* Protect against whitespace on end of line */
+                        int out = sb.length();
+                            if (out > 0 && ((sb.charAt(out-1) == ' ') || (sb.charAt(out-1) == '\t'))) {
+                                ch = sb.charAt(out-1);
+                                sb.setLength(out-1);
+                                qpEscape(sb, ch);
+                            }
+
+                            sb.append(lineEnd);
+                            if (ch == '\r') {
+                                in+=2;
+                            } else {
+                                in++;
+                            }
+                    }
+                    else {
+                        if ((in + 1 != datalen) &&
+                            ((char) bin_data.intAt(in+1) != '\n') &&
+                            (count + 1) >= MAXLINESIZE) {
+                            sb.append('=');
+                            sb.append(lineEnd);
+                            count = 0;
+                            }
+                        count++;
+                        if (header && ch == ' ') {
+                            sb.append('_');
+                            in++;
+                        }
+                        else {
+                            sb.append(ch);
+                            in++;
+                        }
+                    }
+                }
+            }
+        } finally {
+            bin_data.release();
         }
 
-        StringBuilder sb = new StringBuilder();
-        int count = 0;
-
-        for (int i=0, m=s.length(); i<m; i++) {
-        	char c = s.charAt(i);
-
-                // RFC 1521 requires that the line ending in a space or tab must have
-                // that trailing character encoded.
-                if (lineEnding(s, lineEnd, i)) {
-                    count = 0;
-                    sb.append(lineEnd);
-                    if (lineEnd.length() > 1) i++;
-                }
-                else if ((c == '.') && (count == 0) && endOfLine(s, lineEnd, i + 1)) {
-                    // RFC 1521 requires that a solitary "." alone on a line is encoded.
-                    count += 3;
-                    qpEscape(sb, c);
-                }
-                else if ((c == '\t' || c == ' ' ) && endOfLine(s, lineEnd, i + 1)) {
-                    count += 3;
-                    qpEscape(sb, c);
-                }
-                else if (('!' <= c && c <= '<')
-        		|| ('>' <= c && c <= '^')
-        		|| ('`' <= c && c <= '~')
-        		|| (c == '_' && !header)
-        		|| (c == '\n' || c == '\r' && istext)) {
-//        		if (count == 75 && i < s.length() - 1) {
-                        if (count == 75 && !endOfLine(s, lineEnd, i + 1)) {
-        			sb.append("=").append(lineEnd);
-        			count = 0;
-        		}
-        		sb.append(c);
-        		count++;
-        	}
-        	else if (!quotetabs && (c == '\t' || c == ' ')) {
-        		if (count >= 72) {
-        			sb.append("=").append(lineEnd);
-        			count = 0;
-        		}
-        		
-        		if (count >= 71) {
-        			count += 3;
-        			qpEscape(sb, c);
-        		} else {
-        			if (c == ' ' && header)
-        				sb.append('_');
-        			else
-        				sb.append(c);
-        			count += 1;
-        		}
-        	} else {
-        		if (count >= 72) {
-        			sb.append("=").append(lineEnd);
-        			count = 0;
-        		}
-        		count += 3;
-        		qpEscape(sb, c);
-        	}
-        }
         return new PyString(sb.toString());
     }
     
-    private static boolean endOfLine(String s, String lineEnd, int i) {
-        return (s.length() == i || lineEnding(s, lineEnd, i));
-    }
-    
-    private static boolean lineEnding(String s, String lineEnd, int i) {
-        return 
-            (s.length() > i && s.substring(i).startsWith(lineEnd));
-    }
-    
 /*
     public static void main(String[] args) {
         String l = b2a_uu("Hello");

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sun May 11 06:32:01 2014
From: jython-checkins at python.org (jim.baker)
Date: Sun, 11 May 2014 06:32:01 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Upgraded_Guava_collections_?=
	=?utf-8?q?library_to_17=2E0=2E?=
Message-ID: <3gRC6T2r8Sz7LjP@mail.python.org>

http://hg.python.org/jython/rev/3059c41e3838
changeset:   7240:3059c41e3838
user:        Jim Baker <jim.baker at rackspace.com>
date:        Sat May 10 22:31:48 2014 -0600
summary:
  Upgraded Guava collections library to 17.0.
Updated PyDefaultDict to use LoadingCache idiom instead of deprecated
ComputingMap; fixes bug 2087.
Only use computed loads with __getitem__, not other get usage; fixes
bug 2133 re memory leaks.

files:
  Lib/test/test_defaultdict_jy.py                        |   21 ++++++-
  build.xml                                              |    4 +-
  extlibs/guava-13.0.1.jar                               |  Bin 
  extlibs/guava-17.0.jar                                 |  Bin 
  src/org/python/modules/_collections/PyDefaultDict.java |   31 ++++++---
  5 files changed, 42 insertions(+), 14 deletions(-)


diff --git a/Lib/test/test_defaultdict_jy.py b/Lib/test/test_defaultdict_jy.py
--- a/Lib/test/test_defaultdict_jy.py
+++ b/Lib/test/test_defaultdict_jy.py
@@ -74,9 +74,28 @@
         for i in xrange(size):
             self.assertEqual(counters[i].get(), 0, counters)
 
+class GetVariantsTestCase(unittest.TestCase):
+
+    #http://bugs.jython.org/issue2133
+
+    def test_get_does_not_vivify(self):
+        d = defaultdict(list)
+        self.assertEquals(d.get("foo"), None)
+        self.assertEquals(d.items(), [])
+
+    def test_get_default_does_not_vivify(self):
+        d = defaultdict(list)
+        self.assertEquals(d.get("foo", 42), 42)
+        self.assertEquals(d.items(), [])
+
+    def test_getitem_does_vivify(self):
+        d = defaultdict(list)
+        self.assertEquals(d["vivify"], [])
+        self.assertEquals(d.items(), [("vivify", [])]) 
+
 
 def test_main():
-    test_support.run_unittest(PickleTestCase, ThreadSafetyTestCase)
+    test_support.run_unittest(PickleTestCase, ThreadSafetyTestCase, GetVariantsTestCase)
 
 
 if __name__ == '__main__':
diff --git a/build.xml b/build.xml
--- a/build.xml
+++ b/build.xml
@@ -152,7 +152,7 @@
             <pathelement path="${extlibs.dir}/asm-4.0.jar" />
             <pathelement path="${extlibs.dir}/asm-commons-4.0.jar" />
             <pathelement path="${extlibs.dir}/asm-util-4.0.jar" />
-            <pathelement path="${extlibs.dir}/guava-13.0.1.jar" />
+            <pathelement path="${extlibs.dir}/guava-17.0.jar" />
             <pathelement path="${extlibs.dir}/icu4j-52_1.jar" />
             <pathelement path="${extlibs.dir}/jffi-1.2.7.jar"/>
             <pathelement path="${extlibs.dir}/jffi-1.2.7-native.jar"/>
@@ -562,7 +562,7 @@
 	    <rule pattern="org.bouncycastle.**" result="org.python.bouncycastle. at 1"/>
 	    <zipfileset src="extlibs/commons-compress-1.8.jar"/>
 	    <rule pattern="org.apache.**" result="org.python.apache. at 1"/>
-            <zipfileset src="extlibs/guava-13.0.1.jar"/>
+            <zipfileset src="extlibs/guava-17.0.jar"/>
             <rule pattern="com.google.**" result="org.python.google. at 1"/>
             <zipfileset src="extlibs/icu4j-52_1.jar"/>
             <rule pattern="com.ibm.icu.**" result="org.python.icu. at 1"/>
diff --git a/extlibs/guava-13.0.1.jar b/extlibs/guava-13.0.1.jar
deleted file mode 100644
index 09c5449115df66fdd2611e2c4f8c362fbb6aaff3..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/guava-17.0.jar b/extlibs/guava-17.0.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..661fc7473f8760f5f81874ddc1fcc0b5634fd6cf
GIT binary patch
[stripped]
diff --git a/src/org/python/modules/_collections/PyDefaultDict.java b/src/org/python/modules/_collections/PyDefaultDict.java
--- a/src/org/python/modules/_collections/PyDefaultDict.java
+++ b/src/org/python/modules/_collections/PyDefaultDict.java
@@ -4,6 +4,9 @@
 import java.util.Map;
 import java.util.concurrent.ConcurrentMap;
 
+import com.google.common.cache.CacheBuilder;
+import com.google.common.cache.CacheLoader;
+import com.google.common.cache.LoadingCache;
 import org.python.core.Py;
 import org.python.core.PyDictionary;
 import org.python.core.PyObject;
@@ -40,10 +43,10 @@
      * argument to the constructor, if present, or to None, if absent.
      */
     private PyObject defaultFactory = Py.None;
-    private final ConcurrentMap<PyObject, PyObject> backingMap;
+    private final LoadingCache<PyObject, PyObject> backingMap;
 
     public ConcurrentMap<PyObject, PyObject> getMap() {
-        return backingMap;
+        return backingMap.asMap();
     }
 
     public PyDefaultDict() {
@@ -52,17 +55,16 @@
 
     public PyDefaultDict(PyType subtype) {
         super(subtype, false);
-        backingMap =
-                new MapMaker().makeComputingMap(
-                new Function<PyObject, PyObject>() {
-
-                    public PyObject apply(PyObject key) {
+        backingMap = CacheBuilder.newBuilder().build(
+                new CacheLoader<PyObject, PyObject>() {
+                    public PyObject load(PyObject key) {
                         if (defaultFactory == Py.None) {
                             throw Py.KeyError(key);
                         }
                         return defaultFactory.__call__();
                     }
-                });
+                }
+        );
     }
 
     public PyDefaultDict(PyType subtype, Map<PyObject, PyObject> map) {
@@ -165,11 +167,18 @@
     @ExposedMethod(doc = BuiltinDocs.dict___getitem___doc)
     protected final PyObject defaultdict___getitem__(PyObject key) {
         try {
-            return getMap().get(key);
-//        } catch (ComputationException ex) {
-//            throw Py.RuntimeError(ex.getCause());
+            return backingMap.get(key);
         } catch (Exception ex) {
             throw Py.KeyError(key);
         }
     }
+
+    public PyObject get(PyObject key, PyObject defaultObj) {
+        PyObject value = getMap().get(key);
+        if (value != null) {
+            return value;
+        } else {
+            return defaultObj;
+        }
+    }
 }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sun May 11 07:37:13 2014
From: jython-checkins at python.org (jim.baker)
Date: Sun, 11 May 2014 07:37:13 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Updated_*Derived=2Ejava_cla?=
 =?utf-8?q?sses_as_generated_by_src/templates/gderived=2Epy=2E?=
Message-ID: <3gRDYj0KKzz7Lk0@mail.python.org>

http://hg.python.org/jython/rev/2ce763cf289d
changeset:   7241:2ce763cf289d
user:        Jim Baker <jim.baker at rackspace.com>
date:        Sat May 10 23:37:01 2014 -0600
summary:
  Updated *Derived.java classes as generated by src/templates/gderived.py.
Fixes bug 2119.

files:
  src/org/python/antlr/ast/AssertDerived.java                              |    12 +
  src/org/python/antlr/ast/AssignDerived.java                              |    12 +
  src/org/python/antlr/ast/AttributeDerived.java                           |    12 +
  src/org/python/antlr/ast/AugAssignDerived.java                           |    12 +
  src/org/python/antlr/ast/BinOpDerived.java                               |    12 +
  src/org/python/antlr/ast/BoolOpDerived.java                              |    12 +
  src/org/python/antlr/ast/BreakDerived.java                               |    12 +
  src/org/python/antlr/ast/CallDerived.java                                |    12 +
  src/org/python/antlr/ast/ClassDefDerived.java                            |    12 +
  src/org/python/antlr/ast/CompareDerived.java                             |    12 +
  src/org/python/antlr/ast/ContinueDerived.java                            |    12 +
  src/org/python/antlr/ast/DeleteDerived.java                              |    12 +
  src/org/python/antlr/ast/DictDerived.java                                |    12 +
  src/org/python/antlr/ast/EllipsisDerived.java                            |    12 +
  src/org/python/antlr/ast/ExceptHandlerDerived.java                       |    12 +
  src/org/python/antlr/ast/ExecDerived.java                                |    12 +
  src/org/python/antlr/ast/ExprDerived.java                                |    12 +
  src/org/python/antlr/ast/ExpressionDerived.java                          |    12 +
  src/org/python/antlr/ast/ExtSliceDerived.java                            |    12 +
  src/org/python/antlr/ast/ForDerived.java                                 |    12 +
  src/org/python/antlr/ast/FunctionDefDerived.java                         |    12 +
  src/org/python/antlr/ast/GeneratorExpDerived.java                        |    12 +
  src/org/python/antlr/ast/GlobalDerived.java                              |    12 +
  src/org/python/antlr/ast/IfDerived.java                                  |    12 +
  src/org/python/antlr/ast/IfExpDerived.java                               |    12 +
  src/org/python/antlr/ast/ImportDerived.java                              |    12 +
  src/org/python/antlr/ast/ImportFromDerived.java                          |    12 +
  src/org/python/antlr/ast/IndexDerived.java                               |    12 +
  src/org/python/antlr/ast/InteractiveDerived.java                         |    12 +
  src/org/python/antlr/ast/LambdaDerived.java                              |    12 +
  src/org/python/antlr/ast/ListCompDerived.java                            |    12 +
  src/org/python/antlr/ast/ListDerived.java                                |    12 +
  src/org/python/antlr/ast/ModuleDerived.java                              |    12 +
  src/org/python/antlr/ast/NameDerived.java                                |    12 +
  src/org/python/antlr/ast/NumDerived.java                                 |    12 +
  src/org/python/antlr/ast/PassDerived.java                                |    12 +
  src/org/python/antlr/ast/PrintDerived.java                               |    12 +
  src/org/python/antlr/ast/RaiseDerived.java                               |    12 +
  src/org/python/antlr/ast/ReprDerived.java                                |    12 +
  src/org/python/antlr/ast/ReturnDerived.java                              |    12 +
  src/org/python/antlr/ast/SliceDerived.java                               |    12 +
  src/org/python/antlr/ast/StrDerived.java                                 |    12 +
  src/org/python/antlr/ast/SubscriptDerived.java                           |    12 +
  src/org/python/antlr/ast/SuiteDerived.java                               |    12 +
  src/org/python/antlr/ast/TryExceptDerived.java                           |    12 +
  src/org/python/antlr/ast/TryFinallyDerived.java                          |    12 +
  src/org/python/antlr/ast/TupleDerived.java                               |    12 +
  src/org/python/antlr/ast/UnaryOpDerived.java                             |    12 +
  src/org/python/antlr/ast/WhileDerived.java                               |    12 +
  src/org/python/antlr/ast/WithDerived.java                                |    12 +
  src/org/python/antlr/ast/YieldDerived.java                               |    12 +
  src/org/python/antlr/ast/aliasDerived.java                               |    12 +
  src/org/python/antlr/ast/argumentsDerived.java                           |    12 +
  src/org/python/antlr/ast/comprehensionDerived.java                       |    12 +
  src/org/python/antlr/ast/keywordDerived.java                             |    12 +
  src/org/python/antlr/op/AddDerived.java                                  |    12 +
  src/org/python/antlr/op/AndDerived.java                                  |    12 +
  src/org/python/antlr/op/AugLoadDerived.java                              |    12 +
  src/org/python/antlr/op/AugStoreDerived.java                             |    12 +
  src/org/python/antlr/op/BitAndDerived.java                               |    12 +
  src/org/python/antlr/op/BitOrDerived.java                                |    12 +
  src/org/python/antlr/op/BitXorDerived.java                               |    12 +
  src/org/python/antlr/op/DelDerived.java                                  |    12 +
  src/org/python/antlr/op/DivDerived.java                                  |    12 +
  src/org/python/antlr/op/EqDerived.java                                   |    12 +
  src/org/python/antlr/op/FloorDivDerived.java                             |    12 +
  src/org/python/antlr/op/GtDerived.java                                   |    12 +
  src/org/python/antlr/op/GtEDerived.java                                  |    12 +
  src/org/python/antlr/op/InDerived.java                                   |    12 +
  src/org/python/antlr/op/InvertDerived.java                               |    12 +
  src/org/python/antlr/op/IsDerived.java                                   |    12 +
  src/org/python/antlr/op/IsNotDerived.java                                |    12 +
  src/org/python/antlr/op/LShiftDerived.java                               |    12 +
  src/org/python/antlr/op/LoadDerived.java                                 |    12 +
  src/org/python/antlr/op/LtDerived.java                                   |    12 +
  src/org/python/antlr/op/LtEDerived.java                                  |    12 +
  src/org/python/antlr/op/ModDerived.java                                  |    12 +
  src/org/python/antlr/op/MultDerived.java                                 |    12 +
  src/org/python/antlr/op/NotDerived.java                                  |    12 +
  src/org/python/antlr/op/NotEqDerived.java                                |    12 +
  src/org/python/antlr/op/NotInDerived.java                                |    12 +
  src/org/python/antlr/op/OrDerived.java                                   |    12 +
  src/org/python/antlr/op/ParamDerived.java                                |    12 +
  src/org/python/antlr/op/PowDerived.java                                  |    12 +
  src/org/python/antlr/op/RShiftDerived.java                               |    12 +
  src/org/python/antlr/op/StoreDerived.java                                |    12 +
  src/org/python/antlr/op/SubDerived.java                                  |    12 +
  src/org/python/antlr/op/UAddDerived.java                                 |    12 +
  src/org/python/antlr/op/USubDerived.java                                 |    12 +
  src/org/python/core/ClasspathPyImporterDerived.java                      |    12 +
  src/org/python/core/PyArrayDerived.java                                  |    12 +
  src/org/python/core/PyBaseExceptionDerived.java                          |    12 +
  src/org/python/core/PyByteArrayDerived.java                              |    12 +
  src/org/python/core/PyClassMethodDerived.java                            |    12 +
  src/org/python/core/PyComplexDerived.java                                |    12 +
  src/org/python/core/PyDictionaryDerived.java                             |    12 +
  src/org/python/core/PyEnumerateDerived.java                              |    12 +
  src/org/python/core/PyFloatDerived.java                                  |    12 +
  src/org/python/core/PyFrozenSetDerived.java                              |    12 +
  src/org/python/core/PyIntegerDerived.java                                |    12 +
  src/org/python/core/PyListDerived.java                                   |    12 +
  src/org/python/core/PyLongDerived.java                                   |    12 +
  src/org/python/core/PyModuleDerived.java                                 |    12 +
  src/org/python/core/PyPropertyDerived.java                               |    12 +
  src/org/python/core/PySetDerived.java                                    |    12 +
  src/org/python/core/PyStringDerived.java                                 |    12 +
  src/org/python/core/PySuperDerived.java                                  |    12 +
  src/org/python/core/PyTupleDerived.java                                  |    12 +
  src/org/python/core/PyTypeDerived.java                                   |    12 +
  src/org/python/core/PyUnicodeDerived.java                                |    12 +
  src/org/python/modules/_collections/PyDefaultDictDerived.java            |    12 +
  src/org/python/modules/_collections/PyDequeDerived.java                  |    12 +
  src/org/python/modules/_csv/PyDialectDerived.java                        |    12 +
  src/org/python/modules/_functools/PyPartialDerived.java                  |    12 +
  src/org/python/modules/_io/PyFileIODerived.java                          |   390 +--
  src/org/python/modules/_io/PyIOBaseDerived.java                          |  1400 ++++-----
  src/org/python/modules/_io/PyRawIOBaseDerived.java                       |  1437 ++++-----
  src/org/python/modules/_weakref/ReferenceTypeDerived.java                |    12 +
  src/org/python/modules/bz2/PyBZ2CompressorDerived.java                   |    12 +
  src/org/python/modules/bz2/PyBZ2DecompressorDerived.java                 |    12 +
  src/org/python/modules/bz2/PyBZ2FileDerived.java                         |    12 +
  src/org/python/modules/itertools/PyTeeIteratorDerived.java               |    12 +
  src/org/python/modules/itertools/chainDerived.java                       |    12 +
  src/org/python/modules/itertools/combinationsDerived.java                |    12 +
  src/org/python/modules/itertools/combinationsWithReplacementDerived.java |    12 +
  src/org/python/modules/itertools/compressDerived.java                    |    12 +
  src/org/python/modules/itertools/countDerived.java                       |    12 +
  src/org/python/modules/itertools/cycleDerived.java                       |    12 +
  src/org/python/modules/itertools/dropwhileDerived.java                   |    12 +
  src/org/python/modules/itertools/groupbyDerived.java                     |    12 +
  src/org/python/modules/itertools/ifilterDerived.java                     |    12 +
  src/org/python/modules/itertools/ifilterfalseDerived.java                |    12 +
  src/org/python/modules/itertools/isliceDerived.java                      |    12 +
  src/org/python/modules/itertools/izipDerived.java                        |    12 +
  src/org/python/modules/itertools/izipLongestDerived.java                 |    12 +
  src/org/python/modules/itertools/permutationsDerived.java                |    12 +
  src/org/python/modules/itertools/productDerived.java                     |    12 +
  src/org/python/modules/itertools/repeatDerived.java                      |    12 +
  src/org/python/modules/itertools/starmapDerived.java                     |    12 +
  src/org/python/modules/itertools/takewhileDerived.java                   |    12 +
  src/org/python/modules/random/PyRandomDerived.java                       |    12 +
  src/org/python/modules/thread/PyLocalDerived.java                        |    12 +
  src/org/python/modules/zipimport/zipimporterDerived.java                 |    12 +
  143 files changed, 2960 insertions(+), 1947 deletions(-)


diff --git a/src/org/python/antlr/ast/AssertDerived.java b/src/org/python/antlr/ast/AssertDerived.java
--- a/src/org/python/antlr/ast/AssertDerived.java
+++ b/src/org/python/antlr/ast/AssertDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/AssignDerived.java b/src/org/python/antlr/ast/AssignDerived.java
--- a/src/org/python/antlr/ast/AssignDerived.java
+++ b/src/org/python/antlr/ast/AssignDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/AttributeDerived.java b/src/org/python/antlr/ast/AttributeDerived.java
--- a/src/org/python/antlr/ast/AttributeDerived.java
+++ b/src/org/python/antlr/ast/AttributeDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/AugAssignDerived.java b/src/org/python/antlr/ast/AugAssignDerived.java
--- a/src/org/python/antlr/ast/AugAssignDerived.java
+++ b/src/org/python/antlr/ast/AugAssignDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/BinOpDerived.java b/src/org/python/antlr/ast/BinOpDerived.java
--- a/src/org/python/antlr/ast/BinOpDerived.java
+++ b/src/org/python/antlr/ast/BinOpDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/BoolOpDerived.java b/src/org/python/antlr/ast/BoolOpDerived.java
--- a/src/org/python/antlr/ast/BoolOpDerived.java
+++ b/src/org/python/antlr/ast/BoolOpDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/BreakDerived.java b/src/org/python/antlr/ast/BreakDerived.java
--- a/src/org/python/antlr/ast/BreakDerived.java
+++ b/src/org/python/antlr/ast/BreakDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/CallDerived.java b/src/org/python/antlr/ast/CallDerived.java
--- a/src/org/python/antlr/ast/CallDerived.java
+++ b/src/org/python/antlr/ast/CallDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ClassDefDerived.java b/src/org/python/antlr/ast/ClassDefDerived.java
--- a/src/org/python/antlr/ast/ClassDefDerived.java
+++ b/src/org/python/antlr/ast/ClassDefDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/CompareDerived.java b/src/org/python/antlr/ast/CompareDerived.java
--- a/src/org/python/antlr/ast/CompareDerived.java
+++ b/src/org/python/antlr/ast/CompareDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ContinueDerived.java b/src/org/python/antlr/ast/ContinueDerived.java
--- a/src/org/python/antlr/ast/ContinueDerived.java
+++ b/src/org/python/antlr/ast/ContinueDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/DeleteDerived.java b/src/org/python/antlr/ast/DeleteDerived.java
--- a/src/org/python/antlr/ast/DeleteDerived.java
+++ b/src/org/python/antlr/ast/DeleteDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/DictDerived.java b/src/org/python/antlr/ast/DictDerived.java
--- a/src/org/python/antlr/ast/DictDerived.java
+++ b/src/org/python/antlr/ast/DictDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/EllipsisDerived.java b/src/org/python/antlr/ast/EllipsisDerived.java
--- a/src/org/python/antlr/ast/EllipsisDerived.java
+++ b/src/org/python/antlr/ast/EllipsisDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ExceptHandlerDerived.java b/src/org/python/antlr/ast/ExceptHandlerDerived.java
--- a/src/org/python/antlr/ast/ExceptHandlerDerived.java
+++ b/src/org/python/antlr/ast/ExceptHandlerDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ExecDerived.java b/src/org/python/antlr/ast/ExecDerived.java
--- a/src/org/python/antlr/ast/ExecDerived.java
+++ b/src/org/python/antlr/ast/ExecDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ExprDerived.java b/src/org/python/antlr/ast/ExprDerived.java
--- a/src/org/python/antlr/ast/ExprDerived.java
+++ b/src/org/python/antlr/ast/ExprDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ExpressionDerived.java b/src/org/python/antlr/ast/ExpressionDerived.java
--- a/src/org/python/antlr/ast/ExpressionDerived.java
+++ b/src/org/python/antlr/ast/ExpressionDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ExtSliceDerived.java b/src/org/python/antlr/ast/ExtSliceDerived.java
--- a/src/org/python/antlr/ast/ExtSliceDerived.java
+++ b/src/org/python/antlr/ast/ExtSliceDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ForDerived.java b/src/org/python/antlr/ast/ForDerived.java
--- a/src/org/python/antlr/ast/ForDerived.java
+++ b/src/org/python/antlr/ast/ForDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/FunctionDefDerived.java b/src/org/python/antlr/ast/FunctionDefDerived.java
--- a/src/org/python/antlr/ast/FunctionDefDerived.java
+++ b/src/org/python/antlr/ast/FunctionDefDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/GeneratorExpDerived.java b/src/org/python/antlr/ast/GeneratorExpDerived.java
--- a/src/org/python/antlr/ast/GeneratorExpDerived.java
+++ b/src/org/python/antlr/ast/GeneratorExpDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/GlobalDerived.java b/src/org/python/antlr/ast/GlobalDerived.java
--- a/src/org/python/antlr/ast/GlobalDerived.java
+++ b/src/org/python/antlr/ast/GlobalDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/IfDerived.java b/src/org/python/antlr/ast/IfDerived.java
--- a/src/org/python/antlr/ast/IfDerived.java
+++ b/src/org/python/antlr/ast/IfDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/IfExpDerived.java b/src/org/python/antlr/ast/IfExpDerived.java
--- a/src/org/python/antlr/ast/IfExpDerived.java
+++ b/src/org/python/antlr/ast/IfExpDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ImportDerived.java b/src/org/python/antlr/ast/ImportDerived.java
--- a/src/org/python/antlr/ast/ImportDerived.java
+++ b/src/org/python/antlr/ast/ImportDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ImportFromDerived.java b/src/org/python/antlr/ast/ImportFromDerived.java
--- a/src/org/python/antlr/ast/ImportFromDerived.java
+++ b/src/org/python/antlr/ast/ImportFromDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/IndexDerived.java b/src/org/python/antlr/ast/IndexDerived.java
--- a/src/org/python/antlr/ast/IndexDerived.java
+++ b/src/org/python/antlr/ast/IndexDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/InteractiveDerived.java b/src/org/python/antlr/ast/InteractiveDerived.java
--- a/src/org/python/antlr/ast/InteractiveDerived.java
+++ b/src/org/python/antlr/ast/InteractiveDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/LambdaDerived.java b/src/org/python/antlr/ast/LambdaDerived.java
--- a/src/org/python/antlr/ast/LambdaDerived.java
+++ b/src/org/python/antlr/ast/LambdaDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ListCompDerived.java b/src/org/python/antlr/ast/ListCompDerived.java
--- a/src/org/python/antlr/ast/ListCompDerived.java
+++ b/src/org/python/antlr/ast/ListCompDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ListDerived.java b/src/org/python/antlr/ast/ListDerived.java
--- a/src/org/python/antlr/ast/ListDerived.java
+++ b/src/org/python/antlr/ast/ListDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ModuleDerived.java b/src/org/python/antlr/ast/ModuleDerived.java
--- a/src/org/python/antlr/ast/ModuleDerived.java
+++ b/src/org/python/antlr/ast/ModuleDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/NameDerived.java b/src/org/python/antlr/ast/NameDerived.java
--- a/src/org/python/antlr/ast/NameDerived.java
+++ b/src/org/python/antlr/ast/NameDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/NumDerived.java b/src/org/python/antlr/ast/NumDerived.java
--- a/src/org/python/antlr/ast/NumDerived.java
+++ b/src/org/python/antlr/ast/NumDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/PassDerived.java b/src/org/python/antlr/ast/PassDerived.java
--- a/src/org/python/antlr/ast/PassDerived.java
+++ b/src/org/python/antlr/ast/PassDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/PrintDerived.java b/src/org/python/antlr/ast/PrintDerived.java
--- a/src/org/python/antlr/ast/PrintDerived.java
+++ b/src/org/python/antlr/ast/PrintDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/RaiseDerived.java b/src/org/python/antlr/ast/RaiseDerived.java
--- a/src/org/python/antlr/ast/RaiseDerived.java
+++ b/src/org/python/antlr/ast/RaiseDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ReprDerived.java b/src/org/python/antlr/ast/ReprDerived.java
--- a/src/org/python/antlr/ast/ReprDerived.java
+++ b/src/org/python/antlr/ast/ReprDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/ReturnDerived.java b/src/org/python/antlr/ast/ReturnDerived.java
--- a/src/org/python/antlr/ast/ReturnDerived.java
+++ b/src/org/python/antlr/ast/ReturnDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/SliceDerived.java b/src/org/python/antlr/ast/SliceDerived.java
--- a/src/org/python/antlr/ast/SliceDerived.java
+++ b/src/org/python/antlr/ast/SliceDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/StrDerived.java b/src/org/python/antlr/ast/StrDerived.java
--- a/src/org/python/antlr/ast/StrDerived.java
+++ b/src/org/python/antlr/ast/StrDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/SubscriptDerived.java b/src/org/python/antlr/ast/SubscriptDerived.java
--- a/src/org/python/antlr/ast/SubscriptDerived.java
+++ b/src/org/python/antlr/ast/SubscriptDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/SuiteDerived.java b/src/org/python/antlr/ast/SuiteDerived.java
--- a/src/org/python/antlr/ast/SuiteDerived.java
+++ b/src/org/python/antlr/ast/SuiteDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/TryExceptDerived.java b/src/org/python/antlr/ast/TryExceptDerived.java
--- a/src/org/python/antlr/ast/TryExceptDerived.java
+++ b/src/org/python/antlr/ast/TryExceptDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/TryFinallyDerived.java b/src/org/python/antlr/ast/TryFinallyDerived.java
--- a/src/org/python/antlr/ast/TryFinallyDerived.java
+++ b/src/org/python/antlr/ast/TryFinallyDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/TupleDerived.java b/src/org/python/antlr/ast/TupleDerived.java
--- a/src/org/python/antlr/ast/TupleDerived.java
+++ b/src/org/python/antlr/ast/TupleDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/UnaryOpDerived.java b/src/org/python/antlr/ast/UnaryOpDerived.java
--- a/src/org/python/antlr/ast/UnaryOpDerived.java
+++ b/src/org/python/antlr/ast/UnaryOpDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/WhileDerived.java b/src/org/python/antlr/ast/WhileDerived.java
--- a/src/org/python/antlr/ast/WhileDerived.java
+++ b/src/org/python/antlr/ast/WhileDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/WithDerived.java b/src/org/python/antlr/ast/WithDerived.java
--- a/src/org/python/antlr/ast/WithDerived.java
+++ b/src/org/python/antlr/ast/WithDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/YieldDerived.java b/src/org/python/antlr/ast/YieldDerived.java
--- a/src/org/python/antlr/ast/YieldDerived.java
+++ b/src/org/python/antlr/ast/YieldDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/aliasDerived.java b/src/org/python/antlr/ast/aliasDerived.java
--- a/src/org/python/antlr/ast/aliasDerived.java
+++ b/src/org/python/antlr/ast/aliasDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/argumentsDerived.java b/src/org/python/antlr/ast/argumentsDerived.java
--- a/src/org/python/antlr/ast/argumentsDerived.java
+++ b/src/org/python/antlr/ast/argumentsDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/comprehensionDerived.java b/src/org/python/antlr/ast/comprehensionDerived.java
--- a/src/org/python/antlr/ast/comprehensionDerived.java
+++ b/src/org/python/antlr/ast/comprehensionDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/ast/keywordDerived.java b/src/org/python/antlr/ast/keywordDerived.java
--- a/src/org/python/antlr/ast/keywordDerived.java
+++ b/src/org/python/antlr/ast/keywordDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/AddDerived.java b/src/org/python/antlr/op/AddDerived.java
--- a/src/org/python/antlr/op/AddDerived.java
+++ b/src/org/python/antlr/op/AddDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/AndDerived.java b/src/org/python/antlr/op/AndDerived.java
--- a/src/org/python/antlr/op/AndDerived.java
+++ b/src/org/python/antlr/op/AndDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/AugLoadDerived.java b/src/org/python/antlr/op/AugLoadDerived.java
--- a/src/org/python/antlr/op/AugLoadDerived.java
+++ b/src/org/python/antlr/op/AugLoadDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/AugStoreDerived.java b/src/org/python/antlr/op/AugStoreDerived.java
--- a/src/org/python/antlr/op/AugStoreDerived.java
+++ b/src/org/python/antlr/op/AugStoreDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/BitAndDerived.java b/src/org/python/antlr/op/BitAndDerived.java
--- a/src/org/python/antlr/op/BitAndDerived.java
+++ b/src/org/python/antlr/op/BitAndDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/BitOrDerived.java b/src/org/python/antlr/op/BitOrDerived.java
--- a/src/org/python/antlr/op/BitOrDerived.java
+++ b/src/org/python/antlr/op/BitOrDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/BitXorDerived.java b/src/org/python/antlr/op/BitXorDerived.java
--- a/src/org/python/antlr/op/BitXorDerived.java
+++ b/src/org/python/antlr/op/BitXorDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/DelDerived.java b/src/org/python/antlr/op/DelDerived.java
--- a/src/org/python/antlr/op/DelDerived.java
+++ b/src/org/python/antlr/op/DelDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/DivDerived.java b/src/org/python/antlr/op/DivDerived.java
--- a/src/org/python/antlr/op/DivDerived.java
+++ b/src/org/python/antlr/op/DivDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/EqDerived.java b/src/org/python/antlr/op/EqDerived.java
--- a/src/org/python/antlr/op/EqDerived.java
+++ b/src/org/python/antlr/op/EqDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/FloorDivDerived.java b/src/org/python/antlr/op/FloorDivDerived.java
--- a/src/org/python/antlr/op/FloorDivDerived.java
+++ b/src/org/python/antlr/op/FloorDivDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/GtDerived.java b/src/org/python/antlr/op/GtDerived.java
--- a/src/org/python/antlr/op/GtDerived.java
+++ b/src/org/python/antlr/op/GtDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/GtEDerived.java b/src/org/python/antlr/op/GtEDerived.java
--- a/src/org/python/antlr/op/GtEDerived.java
+++ b/src/org/python/antlr/op/GtEDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/InDerived.java b/src/org/python/antlr/op/InDerived.java
--- a/src/org/python/antlr/op/InDerived.java
+++ b/src/org/python/antlr/op/InDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/InvertDerived.java b/src/org/python/antlr/op/InvertDerived.java
--- a/src/org/python/antlr/op/InvertDerived.java
+++ b/src/org/python/antlr/op/InvertDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/IsDerived.java b/src/org/python/antlr/op/IsDerived.java
--- a/src/org/python/antlr/op/IsDerived.java
+++ b/src/org/python/antlr/op/IsDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/IsNotDerived.java b/src/org/python/antlr/op/IsNotDerived.java
--- a/src/org/python/antlr/op/IsNotDerived.java
+++ b/src/org/python/antlr/op/IsNotDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/LShiftDerived.java b/src/org/python/antlr/op/LShiftDerived.java
--- a/src/org/python/antlr/op/LShiftDerived.java
+++ b/src/org/python/antlr/op/LShiftDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/LoadDerived.java b/src/org/python/antlr/op/LoadDerived.java
--- a/src/org/python/antlr/op/LoadDerived.java
+++ b/src/org/python/antlr/op/LoadDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/LtDerived.java b/src/org/python/antlr/op/LtDerived.java
--- a/src/org/python/antlr/op/LtDerived.java
+++ b/src/org/python/antlr/op/LtDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/LtEDerived.java b/src/org/python/antlr/op/LtEDerived.java
--- a/src/org/python/antlr/op/LtEDerived.java
+++ b/src/org/python/antlr/op/LtEDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/ModDerived.java b/src/org/python/antlr/op/ModDerived.java
--- a/src/org/python/antlr/op/ModDerived.java
+++ b/src/org/python/antlr/op/ModDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/MultDerived.java b/src/org/python/antlr/op/MultDerived.java
--- a/src/org/python/antlr/op/MultDerived.java
+++ b/src/org/python/antlr/op/MultDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/NotDerived.java b/src/org/python/antlr/op/NotDerived.java
--- a/src/org/python/antlr/op/NotDerived.java
+++ b/src/org/python/antlr/op/NotDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/NotEqDerived.java b/src/org/python/antlr/op/NotEqDerived.java
--- a/src/org/python/antlr/op/NotEqDerived.java
+++ b/src/org/python/antlr/op/NotEqDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/NotInDerived.java b/src/org/python/antlr/op/NotInDerived.java
--- a/src/org/python/antlr/op/NotInDerived.java
+++ b/src/org/python/antlr/op/NotInDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/OrDerived.java b/src/org/python/antlr/op/OrDerived.java
--- a/src/org/python/antlr/op/OrDerived.java
+++ b/src/org/python/antlr/op/OrDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/ParamDerived.java b/src/org/python/antlr/op/ParamDerived.java
--- a/src/org/python/antlr/op/ParamDerived.java
+++ b/src/org/python/antlr/op/ParamDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/PowDerived.java b/src/org/python/antlr/op/PowDerived.java
--- a/src/org/python/antlr/op/PowDerived.java
+++ b/src/org/python/antlr/op/PowDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/RShiftDerived.java b/src/org/python/antlr/op/RShiftDerived.java
--- a/src/org/python/antlr/op/RShiftDerived.java
+++ b/src/org/python/antlr/op/RShiftDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/StoreDerived.java b/src/org/python/antlr/op/StoreDerived.java
--- a/src/org/python/antlr/op/StoreDerived.java
+++ b/src/org/python/antlr/op/StoreDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/SubDerived.java b/src/org/python/antlr/op/SubDerived.java
--- a/src/org/python/antlr/op/SubDerived.java
+++ b/src/org/python/antlr/op/SubDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/UAddDerived.java b/src/org/python/antlr/op/UAddDerived.java
--- a/src/org/python/antlr/op/UAddDerived.java
+++ b/src/org/python/antlr/op/UAddDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/antlr/op/USubDerived.java b/src/org/python/antlr/op/USubDerived.java
--- a/src/org/python/antlr/op/USubDerived.java
+++ b/src/org/python/antlr/op/USubDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/ClasspathPyImporterDerived.java b/src/org/python/core/ClasspathPyImporterDerived.java
--- a/src/org/python/core/ClasspathPyImporterDerived.java
+++ b/src/org/python/core/ClasspathPyImporterDerived.java
@@ -536,6 +536,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyArrayDerived.java b/src/org/python/core/PyArrayDerived.java
--- a/src/org/python/core/PyArrayDerived.java
+++ b/src/org/python/core/PyArrayDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyBaseExceptionDerived.java b/src/org/python/core/PyBaseExceptionDerived.java
--- a/src/org/python/core/PyBaseExceptionDerived.java
+++ b/src/org/python/core/PyBaseExceptionDerived.java
@@ -536,6 +536,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyByteArrayDerived.java b/src/org/python/core/PyByteArrayDerived.java
--- a/src/org/python/core/PyByteArrayDerived.java
+++ b/src/org/python/core/PyByteArrayDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyClassMethodDerived.java b/src/org/python/core/PyClassMethodDerived.java
--- a/src/org/python/core/PyClassMethodDerived.java
+++ b/src/org/python/core/PyClassMethodDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyComplexDerived.java b/src/org/python/core/PyComplexDerived.java
--- a/src/org/python/core/PyComplexDerived.java
+++ b/src/org/python/core/PyComplexDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyDictionaryDerived.java b/src/org/python/core/PyDictionaryDerived.java
--- a/src/org/python/core/PyDictionaryDerived.java
+++ b/src/org/python/core/PyDictionaryDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyEnumerateDerived.java b/src/org/python/core/PyEnumerateDerived.java
--- a/src/org/python/core/PyEnumerateDerived.java
+++ b/src/org/python/core/PyEnumerateDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyFloatDerived.java b/src/org/python/core/PyFloatDerived.java
--- a/src/org/python/core/PyFloatDerived.java
+++ b/src/org/python/core/PyFloatDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyFrozenSetDerived.java b/src/org/python/core/PyFrozenSetDerived.java
--- a/src/org/python/core/PyFrozenSetDerived.java
+++ b/src/org/python/core/PyFrozenSetDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyIntegerDerived.java b/src/org/python/core/PyIntegerDerived.java
--- a/src/org/python/core/PyIntegerDerived.java
+++ b/src/org/python/core/PyIntegerDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyListDerived.java b/src/org/python/core/PyListDerived.java
--- a/src/org/python/core/PyListDerived.java
+++ b/src/org/python/core/PyListDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyLongDerived.java b/src/org/python/core/PyLongDerived.java
--- a/src/org/python/core/PyLongDerived.java
+++ b/src/org/python/core/PyLongDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyModuleDerived.java b/src/org/python/core/PyModuleDerived.java
--- a/src/org/python/core/PyModuleDerived.java
+++ b/src/org/python/core/PyModuleDerived.java
@@ -536,6 +536,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyPropertyDerived.java b/src/org/python/core/PyPropertyDerived.java
--- a/src/org/python/core/PyPropertyDerived.java
+++ b/src/org/python/core/PyPropertyDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PySetDerived.java b/src/org/python/core/PySetDerived.java
--- a/src/org/python/core/PySetDerived.java
+++ b/src/org/python/core/PySetDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyStringDerived.java b/src/org/python/core/PyStringDerived.java
--- a/src/org/python/core/PyStringDerived.java
+++ b/src/org/python/core/PyStringDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PySuperDerived.java b/src/org/python/core/PySuperDerived.java
--- a/src/org/python/core/PySuperDerived.java
+++ b/src/org/python/core/PySuperDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyTupleDerived.java b/src/org/python/core/PyTupleDerived.java
--- a/src/org/python/core/PyTupleDerived.java
+++ b/src/org/python/core/PyTupleDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyTypeDerived.java b/src/org/python/core/PyTypeDerived.java
--- a/src/org/python/core/PyTypeDerived.java
+++ b/src/org/python/core/PyTypeDerived.java
@@ -536,6 +536,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/core/PyUnicodeDerived.java b/src/org/python/core/PyUnicodeDerived.java
--- a/src/org/python/core/PyUnicodeDerived.java
+++ b/src/org/python/core/PyUnicodeDerived.java
@@ -560,6 +560,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/_collections/PyDefaultDictDerived.java b/src/org/python/modules/_collections/PyDefaultDictDerived.java
--- a/src/org/python/modules/_collections/PyDefaultDictDerived.java
+++ b/src/org/python/modules/_collections/PyDefaultDictDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/_collections/PyDequeDerived.java b/src/org/python/modules/_collections/PyDequeDerived.java
--- a/src/org/python/modules/_collections/PyDequeDerived.java
+++ b/src/org/python/modules/_collections/PyDequeDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/_csv/PyDialectDerived.java b/src/org/python/modules/_csv/PyDialectDerived.java
--- a/src/org/python/modules/_csv/PyDialectDerived.java
+++ b/src/org/python/modules/_csv/PyDialectDerived.java
@@ -537,6 +537,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/_functools/PyPartialDerived.java b/src/org/python/modules/_functools/PyPartialDerived.java
--- a/src/org/python/modules/_functools/PyPartialDerived.java
+++ b/src/org/python/modules/_functools/PyPartialDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/_io/PyFileIODerived.java b/src/org/python/modules/_io/PyFileIODerived.java
--- a/src/org/python/modules/_io/PyFileIODerived.java
+++ b/src/org/python/modules/_io/PyFileIODerived.java
@@ -2,33 +2,14 @@
 package org.python.modules._io;
 
 import java.io.Serializable;
-
-import org.python.core.Deriveds;
-import org.python.core.Py;
-import org.python.core.PyBoolean;
-import org.python.core.PyComplex;
-import org.python.core.PyException;
-import org.python.core.PyFloat;
-import org.python.core.PyInteger;
-import org.python.core.PyLong;
-import org.python.core.PyObject;
-import org.python.core.PySequenceIter;
-import org.python.core.PySlice;
-import org.python.core.PyString;
-import org.python.core.PyTuple;
-import org.python.core.PyType;
-import org.python.core.PyUnicode;
-import org.python.core.Slotted;
-import org.python.core.ThreadState;
+import org.python.core.*;
 
 public class PyFileIODerived extends PyFileIO implements Slotted {
 
-    @Override
     public PyObject getSlot(int index) {
         return slots[index];
     }
 
-    @Override
     public void setSlot(int index,PyObject value) {
         slots[index]=value;
     }
@@ -40,837 +21,714 @@
         slots=new PyObject[subtype.getNumSlots()];
     }
 
-    @Override
     public PyString __str__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__str__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__();
-            if (res instanceof PyString) {
+            if (res instanceof PyString)
                 return(PyString)res;
-            }
             throw Py.TypeError("__str__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__str__();
     }
 
-    @Override
     public PyString __repr__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__repr__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__();
-            if (res instanceof PyString) {
+            if (res instanceof PyString)
                 return(PyString)res;
-            }
             throw Py.TypeError("__repr__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__repr__();
     }
 
-    @Override
     public PyString __hex__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__hex__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__();
-            if (res instanceof PyString) {
+            if (res instanceof PyString)
                 return(PyString)res;
-            }
             throw Py.TypeError("__hex__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__hex__();
     }
 
-    @Override
     public PyString __oct__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__oct__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__();
-            if (res instanceof PyString) {
+            if (res instanceof PyString)
                 return(PyString)res;
-            }
             throw Py.TypeError("__oct__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__oct__();
     }
 
-    @Override
     public PyFloat __float__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__float__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__();
-            if (res instanceof PyFloat) {
+            if (res instanceof PyFloat)
                 return(PyFloat)res;
-            }
             throw Py.TypeError("__float__"+" returned non-"+"float"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__float__();
     }
 
-    @Override
     public PyComplex __complex__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__complex__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__();
-            if (res instanceof PyComplex) {
+            if (res instanceof PyComplex)
                 return(PyComplex)res;
-            }
             throw Py.TypeError("__complex__"+" returned non-"+"complex"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__complex__();
     }
 
-    @Override
     public PyObject __pos__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__pos__");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__();
-        }
         return super.__pos__();
     }
 
-    @Override
     public PyObject __neg__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__neg__");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__();
-        }
         return super.__neg__();
     }
 
-    @Override
     public PyObject __abs__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__abs__");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__();
-        }
         return super.__abs__();
     }
 
-    @Override
     public PyObject __invert__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__invert__");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__();
-        }
         return super.__invert__();
     }
 
-    @Override
     public PyObject __reduce__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__reduce__");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__();
-        }
         return super.__reduce__();
     }
 
-    @Override
     public PyObject __dir__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__dir__");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__();
-        }
         return super.__dir__();
     }
 
-    @Override
     public PyObject __add__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__add__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__add__(other);
     }
 
-    @Override
     public PyObject __radd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__radd__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__radd__(other);
     }
 
-    @Override
     public PyObject __sub__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__sub__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__sub__(other);
     }
 
-    @Override
     public PyObject __rsub__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rsub__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rsub__(other);
     }
 
-    @Override
     public PyObject __mul__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__mul__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__mul__(other);
     }
 
-    @Override
     public PyObject __rmul__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rmul__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rmul__(other);
     }
 
-    @Override
     public PyObject __div__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__div__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__div__(other);
     }
 
-    @Override
     public PyObject __rdiv__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rdiv__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rdiv__(other);
     }
 
-    @Override
     public PyObject __floordiv__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__floordiv__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__floordiv__(other);
     }
 
-    @Override
     public PyObject __rfloordiv__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rfloordiv__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rfloordiv__(other);
     }
 
-    @Override
     public PyObject __truediv__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__truediv__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__truediv__(other);
     }
 
-    @Override
     public PyObject __rtruediv__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rtruediv__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rtruediv__(other);
     }
 
-    @Override
     public PyObject __mod__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__mod__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__mod__(other);
     }
 
-    @Override
     public PyObject __rmod__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rmod__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rmod__(other);
     }
 
-    @Override
     public PyObject __divmod__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__divmod__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__divmod__(other);
     }
 
-    @Override
     public PyObject __rdivmod__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rdivmod__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rdivmod__(other);
     }
 
-    @Override
     public PyObject __rpow__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rpow__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rpow__(other);
     }
 
-    @Override
     public PyObject __lshift__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__lshift__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__lshift__(other);
     }
 
-    @Override
     public PyObject __rlshift__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rlshift__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rlshift__(other);
     }
 
-    @Override
     public PyObject __rshift__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rshift__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rshift__(other);
     }
 
-    @Override
     public PyObject __rrshift__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rrshift__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rrshift__(other);
     }
 
-    @Override
     public PyObject __and__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__and__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__and__(other);
     }
 
-    @Override
     public PyObject __rand__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rand__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rand__(other);
     }
 
-    @Override
     public PyObject __or__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__or__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__or__(other);
     }
 
-    @Override
     public PyObject __ror__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__ror__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ror__(other);
     }
 
-    @Override
     public PyObject __xor__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__xor__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__xor__(other);
     }
 
-    @Override
     public PyObject __rxor__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__rxor__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rxor__(other);
     }
 
-    @Override
     public PyObject __lt__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__lt__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__lt__(other);
     }
 
-    @Override
     public PyObject __le__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__le__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__le__(other);
     }
 
-    @Override
     public PyObject __gt__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__gt__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__gt__(other);
     }
 
-    @Override
     public PyObject __ge__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__ge__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ge__(other);
     }
 
-    @Override
     public PyObject __eq__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__eq__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__eq__(other);
     }
 
-    @Override
     public PyObject __ne__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__ne__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ne__(other);
     }
 
-    @Override
     public PyObject __format__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__format__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__format__(other);
     }
 
-    @Override
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__iadd__(other);
     }
 
-    @Override
     public PyObject __isub__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__isub__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__isub__(other);
     }
 
-    @Override
     public PyObject __imul__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__imul__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__imul__(other);
     }
 
-    @Override
     public PyObject __idiv__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__idiv__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__idiv__(other);
     }
 
-    @Override
     public PyObject __ifloordiv__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__ifloordiv__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ifloordiv__(other);
     }
 
-    @Override
     public PyObject __itruediv__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__itruediv__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__itruediv__(other);
     }
 
-    @Override
     public PyObject __imod__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__imod__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__imod__(other);
     }
 
-    @Override
     public PyObject __ipow__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__ipow__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ipow__(other);
     }
 
-    @Override
     public PyObject __ilshift__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__ilshift__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ilshift__(other);
     }
 
-    @Override
     public PyObject __irshift__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__irshift__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__irshift__(other);
     }
 
-    @Override
     public PyObject __iand__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iand__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__iand__(other);
     }
 
-    @Override
     public PyObject __ior__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__ior__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ior__(other);
     }
 
-    @Override
     public PyObject __ixor__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__ixor__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(other);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ixor__(other);
     }
 
-    @Override
     public PyObject __int__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__int__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__();
-            if (res instanceof PyInteger||res instanceof PyLong) {
+            if (res instanceof PyInteger||res instanceof PyLong)
                 return res;
-            }
             throw Py.TypeError("__int__"+" should return an integer");
         }
         return super.__int__();
     }
 
-    @Override
     public PyObject __long__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__long__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__();
-            if (res instanceof PyLong||res instanceof PyInteger) {
+            if (res instanceof PyLong||res instanceof PyInteger)
                 return res;
-            }
             throw Py.TypeError("__long__"+" returned non-"+"long"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__long__();
     }
 
-    @Override
     public int hashCode() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__hash__");
@@ -890,24 +748,20 @@
         return super.hashCode();
     }
 
-    @Override
     public PyUnicode __unicode__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__unicode__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__();
-            if (res instanceof PyUnicode) {
+            if (res instanceof PyUnicode)
                 return(PyUnicode)res;
-            }
-            if (res instanceof PyString) {
+            if (res instanceof PyString)
                 return new PyUnicode((PyString)res);
-            }
             throw Py.TypeError("__unicode__"+" should return a "+"unicode");
         }
         return super.__unicode__();
     }
 
-    @Override
     public int __cmp__(PyObject other) {
         PyType self_type=getType();
         PyObject[]where_type=new PyObject[1];
@@ -928,15 +782,13 @@
         return c<0?-1:c>0?1:0;
     }
 
-    @Override
     public boolean __nonzero__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__nonzero__");
         if (impl==null) {
             impl=self_type.lookup("__len__");
-            if (impl==null) {
+            if (impl==null)
                 return super.__nonzero__();
-            }
         }
         PyObject o=impl.__get__(this,self_type).__call__();
         Class c=o.getClass();
@@ -946,45 +798,37 @@
         return o.__nonzero__();
     }
 
-    @Override
     public boolean __contains__(PyObject o) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__contains__");
-        if (impl==null) {
+        if (impl==null)
             return super.__contains__(o);
-        }
         return impl.__get__(this,self_type).__call__(o).__nonzero__();
     }
 
-    @Override
     public int __len__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__len__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__();
-            if (res instanceof PyInteger) {
+            if (res instanceof PyInteger)
                 return((PyInteger)res).getValue();
-            }
             throw Py.TypeError("__len__ should return a int");
         }
         return super.__len__();
     }
 
-    @Override
     public PyObject __iter__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iter__");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__();
-        }
         impl=self_type.lookup("__getitem__");
-        if (impl==null) {
+        if (impl==null)
             return super.__iter__();
-        }
         return new PySequenceIter(this);
     }
 
-    @Override
     public PyObject __iternext__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("next");
@@ -992,50 +836,42 @@
             try {
                 return impl.__get__(this,self_type).__call__();
             } catch (PyException exc) {
-                if (exc.match(Py.StopIteration)) {
+                if (exc.match(Py.StopIteration))
                     return null;
-                }
                 throw exc;
             }
         }
         return super.__iternext__(); // ???
     }
 
-    @Override
     public PyObject __finditem__(PyObject key) { // ???
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__getitem__");
-        if (impl!=null) {
+        if (impl!=null)
             try {
                 return impl.__get__(this,self_type).__call__(key);
             } catch (PyException exc) {
-                if (exc.match(Py.LookupError)) {
+                if (exc.match(Py.LookupError))
                     return null;
-                }
                 throw exc;
             }
-        }
         return super.__finditem__(key);
     }
 
-    @Override
     public PyObject __finditem__(int key) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__getitem__");
-        if (impl!=null) {
+        if (impl!=null)
             try {
                 return impl.__get__(this,self_type).__call__(new PyInteger(key));
             } catch (PyException exc) {
-                if (exc.match(Py.LookupError)) {
+                if (exc.match(Py.LookupError))
                     return null;
-                }
                 throw exc;
             }
-        }
         return super.__finditem__(key);
     }
 
-    @Override
     public PyObject __getitem__(PyObject key) {
         // Same as __finditem__, without swallowing LookupErrors. This allows
         // __getitem__ implementations written in Python to raise custom
@@ -1057,13 +893,11 @@
 
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__getitem__");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__(key);
-        }
         return super.__getitem__(key);
     }
 
-    @Override
     public void __setitem__(PyObject key,PyObject value) { // ???
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__setitem__");
@@ -1074,7 +908,6 @@
         super.__setitem__(key,value);
     }
 
-    @Override
     public PyObject __getslice__(PyObject start,PyObject stop,PyObject step) { // ???
         if (step!=null) {
             return __getitem__(new PySlice(start,stop,step));
@@ -1088,7 +921,6 @@
         return super.__getslice__(start,stop,step);
     }
 
-    @Override
     public void __setslice__(PyObject start,PyObject stop,PyObject step,PyObject value) {
         if (step!=null) {
             __setitem__(new PySlice(start,stop,step),value);
@@ -1104,7 +936,6 @@
         super.__setslice__(start,stop,step,value);
     }
 
-    @Override
     public void __delslice__(PyObject start,PyObject stop,PyObject step) {
         if (step!=null) {
             __delitem__(new PySlice(start,stop,step));
@@ -1120,7 +951,6 @@
         super.__delslice__(start,stop,step);
     }
 
-    @Override
     public void __delitem__(PyObject key) { // ???
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__delitem__");
@@ -1131,30 +961,25 @@
         super.__delitem__(key);
     }
 
-    @Override
     public PyObject __call__(PyObject args[],String keywords[]) {
         ThreadState ts=Py.getThreadState();
-        if (ts.recursion_depth++>ts.systemState.getrecursionlimit()) {
+        if (ts.recursion_depth++>ts.systemState.getrecursionlimit())
             throw Py.RuntimeError("maximum __call__ recursion depth exceeded");
-        }
         try {
             PyType self_type=getType();
             PyObject impl=self_type.lookup("__call__");
-            if (impl!=null) {
+            if (impl!=null)
                 return impl.__get__(this,self_type).__call__(args,keywords);
-            }
             return super.__call__(args,keywords);
         } finally {
             --ts.recursion_depth;
         }
     }
 
-    @Override
     public PyObject __findattr_ex__(String name) {
         return Deriveds.__findattr_ex__(this,name);
     }
 
-    @Override
     public void __setattr__(String name,PyObject value) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__setattr__");
@@ -1165,7 +990,6 @@
         super.__setattr__(name,value);
     }
 
-    @Override
     public void __delattr__(String name) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__delattr__");
@@ -1176,23 +1000,19 @@
         super.__delattr__(name);
     }
 
-    @Override
     public PyObject __get__(PyObject obj,PyObject type) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__get__");
         if (impl!=null) {
-            if (obj==null) {
+            if (obj==null)
                 obj=Py.None;
-            }
-            if (type==null) {
+            if (type==null)
                 type=Py.None;
-            }
             return impl.__get__(this,self_type).__call__(obj,type);
         }
         return super.__get__(obj,type);
     }
 
-    @Override
     public void __set__(PyObject obj,PyObject value) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__set__");
@@ -1203,7 +1023,6 @@
         super.__set__(obj,value);
     }
 
-    @Override
     public void __delete__(PyObject obj) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__delete__");
@@ -1214,7 +1033,6 @@
         super.__delete__(obj);
     }
 
-    @Override
     public PyObject __pow__(PyObject other,PyObject modulo) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__pow__");
@@ -1225,20 +1043,17 @@
             } else {
                 res=impl.__get__(this,self_type).__call__(other,modulo);
             }
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__pow__(other,modulo);
     }
 
-    @Override
     public void dispatch__init__(PyObject[]args,String[]keywords) {
         Deriveds.dispatch__init__(this,args,keywords);
     }
 
-    @Override
     public PyObject __index__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__index__");
@@ -1252,7 +1067,6 @@
         return super.__index__();
     }
 
-    @Override
     public Object __tojava__(Class c) {
         // If we are not being asked by the "default" conversion to java, then
         // we can provide this as the result, as long as it is a instance of the
@@ -1265,52 +1079,43 @@
         // Otherwise, we call the derived __tojava__, if it exists:
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__tojava__");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__(Py.java2py(c)).__tojava__(Object.class);
-        }
         return super.__tojava__(c);
     }
 
-    @Override
     public Object __coerce_ex__(PyObject o) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__coerce__");
         if (impl!=null) {
             PyObject res=impl.__get__(this,self_type).__call__(o);
-            if (res==Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return Py.None;
-            }
-            if (!(res instanceof PyTuple)) {
+            if (!(res instanceof PyTuple))
                 throw Py.TypeError("__coerce__ didn't return a 2-tuple");
-            }
             return((PyTuple)res).getArray();
         }
         return super.__coerce_ex__(o);
     }
 
-    @Override
     public PyObject __enter__() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__enter__");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__();
-        }
         return super.__enter__();
     }
 
-    @Override
     public PyObject fileno() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("fileno");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__();
-        }
         return super.fileno();
     }
 
     // Hand-crafted in _io._IOBase.derived
 
-    @Override
     public long seek(long pos,int whence) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("seek");
@@ -1321,7 +1126,6 @@
         }
     }
 
-    @Override
     public long tell() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("tell");
@@ -1332,7 +1136,6 @@
         }
     }
 
-    @Override
     public long truncate(long size) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("truncate");
@@ -1343,7 +1146,6 @@
         }
     }
 
-    @Override
     public long truncate() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("truncate");
@@ -1354,7 +1156,6 @@
         }
     }
 
-    @Override
     public void flush() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("flush");
@@ -1365,7 +1166,6 @@
         }
     }
 
-    @Override
     public void close() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("close");
@@ -1376,7 +1176,6 @@
         }
     }
 
-    @Override
     public boolean seekable() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("seekable");
@@ -1387,7 +1186,6 @@
         }
     }
 
-    @Override
     public void _checkSeekable(String msg) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("_checkSeekable");
@@ -1399,7 +1197,6 @@
         }
     }
 
-    @Override
     public boolean readable() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("readable");
@@ -1410,7 +1207,6 @@
         }
     }
 
-    @Override
     public void _checkReadable(String msg) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("_checkReadable");
@@ -1422,7 +1218,6 @@
         }
     }
 
-    @Override
     public boolean writable() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("writable");
@@ -1433,7 +1228,6 @@
         }
     }
 
-    @Override
     public void _checkWritable(String msg) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("_checkWritable");
@@ -1447,7 +1241,6 @@
 
     // Note that closed is a property not a predicate, so no derived method.
 
-    @Override
     public void _checkClosed(String msg) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("_checkClosed");
@@ -1459,7 +1252,6 @@
         }
     }
 
-    @Override
     public boolean __exit__(PyObject type,PyObject value,PyObject traceback) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__exit__");
@@ -1470,7 +1262,6 @@
         }
     }
 
-    @Override
     public boolean isatty() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("isatty");
@@ -1481,7 +1272,6 @@
         }
     }
 
-    @Override
     public PyObject readline() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("readline");
@@ -1492,7 +1282,6 @@
         }
     }
 
-    @Override
     public PyObject readline(int limit) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("readline");
@@ -1503,7 +1292,6 @@
         }
     }
 
-    @Override
     public PyObject readlines(PyObject hint) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("readlines");
@@ -1515,7 +1303,6 @@
         }
     }
 
-    @Override
     public void writelines(PyObject lines) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("writelines");
@@ -1526,19 +1313,16 @@
         }
     }
 
-    @Override
     public PyObject readall() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("readall");
-        if (impl!=null) {
+        if (impl!=null)
             return impl.__get__(this,self_type).__call__();
-        }
         return super.readall();
     }
 
     // Hand-crafted in _io._RawIOBase.derived
 
-    @Override
     public PyObject read(int n) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("read");
@@ -1549,7 +1333,6 @@
         }
     }
 
-    @Override
     public PyObject readinto(PyObject b) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("readinto");
@@ -1560,7 +1343,6 @@
         }
     }
 
-    @Override
     public PyObject write(PyObject b) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("write");
diff --git a/src/org/python/modules/_io/PyIOBaseDerived.java b/src/org/python/modules/_io/PyIOBaseDerived.java
--- a/src/org/python/modules/_io/PyIOBaseDerived.java
+++ b/src/org/python/modules/_io/PyIOBaseDerived.java
@@ -1,1048 +1,877 @@
-/* Generated file, do not modify. See jython/src/templates/gderived.py. */
+/* Generated file, do not modify.  See jython/src/templates/gderived.py. */
 package org.python.modules._io;
 
 import java.io.Serializable;
-
-import org.python.core.Deriveds;
-import org.python.core.Py;
-import org.python.core.PyBoolean;
-import org.python.core.PyComplex;
-import org.python.core.PyException;
-import org.python.core.PyFloat;
-import org.python.core.PyInteger;
-import org.python.core.PyLong;
-import org.python.core.PyObject;
-import org.python.core.PySequenceIter;
-import org.python.core.PySlice;
-import org.python.core.PyString;
-import org.python.core.PyTuple;
-import org.python.core.PyType;
-import org.python.core.PyUnicode;
-import org.python.core.Slotted;
-import org.python.core.ThreadState;
+import org.python.core.*;
 
 public class PyIOBaseDerived extends PyIOBase implements Slotted {
 
-    @Override
     public PyObject getSlot(int index) {
         return slots[index];
     }
 
-    @Override
-    public void setSlot(int index, PyObject value) {
-        slots[index] = value;
+    public void setSlot(int index,PyObject value) {
+        slots[index]=value;
     }
 
-    private PyObject[] slots;
+    private PyObject[]slots;
 
     public PyIOBaseDerived(PyType subtype) {
         super(subtype);
-        slots = new PyObject[subtype.getNumSlots()];
+        slots=new PyObject[subtype.getNumSlots()];
     }
 
-    @Override
     public PyString __str__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__str__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyString) {
-                return (PyString)res;
-            }
-            throw Py.TypeError("__str__" + " returned non-" + "string" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__str__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__str__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__str__();
     }
 
-    @Override
     public PyString __repr__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__repr__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyString) {
-                return (PyString)res;
-            }
-            throw Py.TypeError("__repr__" + " returned non-" + "string" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__repr__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__repr__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__repr__();
     }
 
-    @Override
     public PyString __hex__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__hex__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyString) {
-                return (PyString)res;
-            }
-            throw Py.TypeError("__hex__" + " returned non-" + "string" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__hex__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__hex__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__hex__();
     }
 
-    @Override
     public PyString __oct__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__oct__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyString) {
-                return (PyString)res;
-            }
-            throw Py.TypeError("__oct__" + " returned non-" + "string" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__oct__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__oct__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__oct__();
     }
 
-    @Override
     public PyFloat __float__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__float__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyFloat) {
-                return (PyFloat)res;
-            }
-            throw Py.TypeError("__float__" + " returned non-" + "float" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__float__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyFloat)
+                return(PyFloat)res;
+            throw Py.TypeError("__float__"+" returned non-"+"float"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__float__();
     }
 
-    @Override
     public PyComplex __complex__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__complex__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyComplex) {
-                return (PyComplex)res;
-            }
-            throw Py.TypeError("__complex__" + " returned non-" + "complex" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__complex__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyComplex)
+                return(PyComplex)res;
+            throw Py.TypeError("__complex__"+" returned non-"+"complex"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__complex__();
     }
 
-    @Override
     public PyObject __pos__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__pos__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__pos__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__pos__();
     }
 
-    @Override
     public PyObject __neg__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__neg__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__neg__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__neg__();
     }
 
-    @Override
     public PyObject __abs__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__abs__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__abs__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__abs__();
     }
 
-    @Override
     public PyObject __invert__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__invert__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__invert__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__invert__();
     }
 
-    @Override
     public PyObject __reduce__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__reduce__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__reduce__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__reduce__();
     }
 
-    @Override
     public PyObject __dir__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__dir__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__dir__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__dir__();
     }
 
-    @Override
     public PyObject __add__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__add__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__add__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__add__(other);
     }
 
-    @Override
     public PyObject __radd__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__radd__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__radd__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__radd__(other);
     }
 
-    @Override
     public PyObject __sub__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__sub__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__sub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__sub__(other);
     }
 
-    @Override
     public PyObject __rsub__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rsub__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rsub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rsub__(other);
     }
 
-    @Override
     public PyObject __mul__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__mul__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__mul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__mul__(other);
     }
 
-    @Override
     public PyObject __rmul__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rmul__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rmul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rmul__(other);
     }
 
-    @Override
     public PyObject __div__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__div__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__div__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__div__(other);
     }
 
-    @Override
     public PyObject __rdiv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rdiv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rdiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rdiv__(other);
     }
 
-    @Override
     public PyObject __floordiv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__floordiv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__floordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__floordiv__(other);
     }
 
-    @Override
     public PyObject __rfloordiv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rfloordiv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rfloordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rfloordiv__(other);
     }
 
-    @Override
     public PyObject __truediv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__truediv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__truediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__truediv__(other);
     }
 
-    @Override
     public PyObject __rtruediv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rtruediv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rtruediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rtruediv__(other);
     }
 
-    @Override
     public PyObject __mod__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__mod__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__mod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__mod__(other);
     }
 
-    @Override
     public PyObject __rmod__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rmod__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rmod__(other);
     }
 
-    @Override
     public PyObject __divmod__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__divmod__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__divmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__divmod__(other);
     }
 
-    @Override
     public PyObject __rdivmod__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rdivmod__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rdivmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rdivmod__(other);
     }
 
-    @Override
     public PyObject __rpow__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rpow__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rpow__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rpow__(other);
     }
 
-    @Override
     public PyObject __lshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__lshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__lshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__lshift__(other);
     }
 
-    @Override
     public PyObject __rlshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rlshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rlshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rlshift__(other);
     }
 
-    @Override
     public PyObject __rshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rshift__(other);
     }
 
-    @Override
     public PyObject __rrshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rrshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rrshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rrshift__(other);
     }
 
-    @Override
     public PyObject __and__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__and__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__and__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__and__(other);
     }
 
-    @Override
     public PyObject __rand__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rand__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rand__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rand__(other);
     }
 
-    @Override
     public PyObject __or__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__or__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__or__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__or__(other);
     }
 
-    @Override
     public PyObject __ror__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ror__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ror__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ror__(other);
     }
 
-    @Override
     public PyObject __xor__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__xor__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__xor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__xor__(other);
     }
 
-    @Override
     public PyObject __rxor__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rxor__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rxor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rxor__(other);
     }
 
-    @Override
     public PyObject __lt__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__lt__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__lt__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__lt__(other);
     }
 
-    @Override
     public PyObject __le__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__le__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__le__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__le__(other);
     }
 
-    @Override
     public PyObject __gt__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__gt__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__gt__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__gt__(other);
     }
 
-    @Override
     public PyObject __ge__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ge__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ge__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ge__(other);
     }
 
-    @Override
     public PyObject __eq__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__eq__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__eq__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__eq__(other);
     }
 
-    @Override
     public PyObject __ne__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ne__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ne__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ne__(other);
     }
 
-    @Override
     public PyObject __format__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__format__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__format__(other);
     }
 
-    @Override
     public PyObject __iadd__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__iadd__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iadd__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__iadd__(other);
     }
 
-    @Override
     public PyObject __isub__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__isub__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__isub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__isub__(other);
     }
 
-    @Override
     public PyObject __imul__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__imul__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__imul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__imul__(other);
     }
 
-    @Override
     public PyObject __idiv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__idiv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__idiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__idiv__(other);
     }
 
-    @Override
     public PyObject __ifloordiv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ifloordiv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ifloordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ifloordiv__(other);
     }
 
-    @Override
     public PyObject __itruediv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__itruediv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__itruediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__itruediv__(other);
     }
 
-    @Override
     public PyObject __imod__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__imod__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__imod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__imod__(other);
     }
 
-    @Override
     public PyObject __ipow__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ipow__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ipow__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ipow__(other);
     }
 
-    @Override
     public PyObject __ilshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ilshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ilshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ilshift__(other);
     }
 
-    @Override
     public PyObject __irshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__irshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__irshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__irshift__(other);
     }
 
-    @Override
     public PyObject __iand__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__iand__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iand__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__iand__(other);
     }
 
-    @Override
     public PyObject __ior__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ior__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ior__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ior__(other);
     }
 
-    @Override
     public PyObject __ixor__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ixor__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ixor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ixor__(other);
     }
 
-    @Override
     public PyObject __int__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__int__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyInteger || res instanceof PyLong) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__int__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger||res instanceof PyLong)
                 return res;
-            }
-            throw Py.TypeError("__int__" + " should return an integer");
+            throw Py.TypeError("__int__"+" should return an integer");
         }
         return super.__int__();
     }
 
-    @Override
     public PyObject __long__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__long__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyLong || res instanceof PyInteger) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__long__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyLong||res instanceof PyInteger)
                 return res;
-            }
-            throw Py.TypeError("__long__" + " returned non-" + "long" + " (type "
-                    + res.getType().fastGetName() + ")");
+            throw Py.TypeError("__long__"+" returned non-"+"long"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__long__();
     }
 
-    @Override
     public int hashCode() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__hash__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__hash__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
             if (res instanceof PyInteger) {
-                return ((PyInteger)res).getValue();
-            } else if (res instanceof PyLong) {
-                return ((PyLong)res).getValue().intValue();
-            }
+                return((PyInteger)res).getValue();
+            } else
+                if (res instanceof PyLong) {
+                    return((PyLong)res).getValue().intValue();
+                }
             throw Py.TypeError("__hash__ should return a int");
         }
-        if (self_type.lookup("__eq__") != null || self_type.lookup("__cmp__") != null) {
-            throw Py.TypeError(String.format("unhashable type: '%.200s'", getType().fastGetName()));
+        if (self_type.lookup("__eq__")!=null||self_type.lookup("__cmp__")!=null) {
+            throw Py.TypeError(String.format("unhashable type: '%.200s'",getType().fastGetName()));
         }
         return super.hashCode();
     }
 
-    @Override
     public PyUnicode __unicode__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__unicode__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyUnicode) {
-                return (PyUnicode)res;
-            }
-            if (res instanceof PyString) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__unicode__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyUnicode)
+                return(PyUnicode)res;
+            if (res instanceof PyString)
                 return new PyUnicode((PyString)res);
-            }
-            throw Py.TypeError("__unicode__" + " should return a " + "unicode");
+            throw Py.TypeError("__unicode__"+" should return a "+"unicode");
         }
         return super.__unicode__();
     }
 
-    @Override
     public int __cmp__(PyObject other) {
-        PyType self_type = getType();
-        PyObject[] where_type = new PyObject[1];
-        PyObject impl = self_type.lookup_where("__cmp__", where_type);
+        PyType self_type=getType();
+        PyObject[]where_type=new PyObject[1];
+        PyObject impl=self_type.lookup_where("__cmp__",where_type);
         // Full Compatibility with CPython __cmp__:
         // If the derived type don't override __cmp__, the
         // *internal* super().__cmp__ should be called, not the
         // exposed one. The difference is that the exposed __cmp__
         // throws a TypeError if the argument is an instance of the same type.
-        if (impl == null || where_type[0] == TYPE || Py.isSubClass(TYPE, where_type[0])) {
+        if (impl==null||where_type[0]==TYPE||Py.isSubClass(TYPE,where_type[0])) {
             return super.__cmp__(other);
         }
-        PyObject res = impl.__get__(this, self_type).__call__(other);
-        if (res == Py.NotImplemented) {
-            return -2;
+        PyObject res=impl.__get__(this,self_type).__call__(other);
+        if (res==Py.NotImplemented) {
+            return-2;
         }
-        int c = res.asInt();
-        return c < 0 ? -1 : c > 0 ? 1 : 0;
+        int c=res.asInt();
+        return c<0?-1:c>0?1:0;
     }
 
-    @Override
     public boolean __nonzero__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__nonzero__");
-        if (impl == null) {
-            impl = self_type.lookup("__len__");
-            if (impl == null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__nonzero__");
+        if (impl==null) {
+            impl=self_type.lookup("__len__");
+            if (impl==null)
                 return super.__nonzero__();
-            }
         }
-        PyObject o = impl.__get__(this, self_type).__call__();
-        Class c = o.getClass();
-        if (c != PyInteger.class && c != PyBoolean.class) {
-            throw Py.TypeError(String.format("__nonzero__ should return bool or int, returned %s",
-                    self_type.getName()));
+        PyObject o=impl.__get__(this,self_type).__call__();
+        Class c=o.getClass();
+        if (c!=PyInteger.class&&c!=PyBoolean.class) {
+            throw Py.TypeError(String.format("__nonzero__ should return bool or int, returned %s",self_type.getName()));
         }
         return o.__nonzero__();
     }
 
-    @Override
     public boolean __contains__(PyObject o) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__contains__");
-        if (impl == null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__contains__");
+        if (impl==null)
             return super.__contains__(o);
-        }
-        return impl.__get__(this, self_type).__call__(o).__nonzero__();
+        return impl.__get__(this,self_type).__call__(o).__nonzero__();
     }
 
-    @Override
     public int __len__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__len__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyInteger) {
-                return ((PyInteger)res).getValue();
-            }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__len__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger)
+                return((PyInteger)res).getValue();
             throw Py.TypeError("__len__ should return a int");
         }
         return super.__len__();
     }
 
-    @Override
     public PyObject __iter__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__iter__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
-        impl = self_type.lookup("__getitem__");
-        if (impl == null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iter__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        impl=self_type.lookup("__getitem__");
+        if (impl==null)
             return super.__iter__();
-        }
         return new PySequenceIter(this);
     }
 
-    @Override
     public PyObject __iternext__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("next");
-        if (impl != null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("next");
+        if (impl!=null) {
             try {
-                return impl.__get__(this, self_type).__call__();
+                return impl.__get__(this,self_type).__call__();
             } catch (PyException exc) {
-                if (exc.match(Py.StopIteration)) {
+                if (exc.match(Py.StopIteration))
                     return null;
-                }
                 throw exc;
             }
         }
         return super.__iternext__(); // ???
     }
 
-    @Override
     public PyObject __finditem__(PyObject key) { // ???
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__getitem__");
-        if (impl != null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
             try {
-                return impl.__get__(this, self_type).__call__(key);
+                return impl.__get__(this,self_type).__call__(key);
             } catch (PyException exc) {
-                if (exc.match(Py.LookupError)) {
+                if (exc.match(Py.LookupError))
                     return null;
-                }
                 throw exc;
             }
-        }
         return super.__finditem__(key);
     }
 
-    @Override
     public PyObject __finditem__(int key) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__getitem__");
-        if (impl != null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
             try {
-                return impl.__get__(this, self_type).__call__(new PyInteger(key));
+                return impl.__get__(this,self_type).__call__(new PyInteger(key));
             } catch (PyException exc) {
-                if (exc.match(Py.LookupError)) {
+                if (exc.match(Py.LookupError))
                     return null;
-                }
                 throw exc;
             }
-        }
         return super.__finditem__(key);
     }
 
-    @Override
     public PyObject __getitem__(PyObject key) {
         // Same as __finditem__, without swallowing LookupErrors. This allows
         // __getitem__ implementations written in Python to raise custom
@@ -1053,7 +882,7 @@
         // in terms of __finditem__. Therefore, we would end with an infinite
         // loop when self_type.lookup("__getitem__") returns null:
         //
-        // __getitem__ -> super.__getitem__ -> __finditem__ -> __getitem__
+        //  __getitem__ -> super.__getitem__ -> __finditem__ -> __getitem__
         //
         // By duplicating the (short) lookup and call code, we are safe, because
         // the call chains will be:
@@ -1062,393 +891,349 @@
         //
         // __getitem__ -> super.__getitem__ -> __finditem__ -> super.__finditem__
 
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__getitem__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(key);
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__(key);
         return super.__getitem__(key);
     }
 
-    @Override
-    public void __setitem__(PyObject key, PyObject value) { // ???
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__setitem__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(key, value);
+    public void __setitem__(PyObject key,PyObject value) { // ???
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setitem__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(key,value);
             return;
         }
-        super.__setitem__(key, value);
+        super.__setitem__(key,value);
     }
 
-    @Override
-    public PyObject __getslice__(PyObject start, PyObject stop, PyObject step) { // ???
-        if (step != null) {
-            return __getitem__(new PySlice(start, stop, step));
+    public PyObject __getslice__(PyObject start,PyObject stop,PyObject step) { // ???
+        if (step!=null) {
+            return __getitem__(new PySlice(start,stop,step));
         }
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__getslice__");
-        if (impl != null) {
-            PyObject[] indices = PySlice.indices2(this, start, stop);
-            return impl.__get__(this, self_type).__call__(indices[0], indices[1]);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            return impl.__get__(this,self_type).__call__(indices[0],indices[1]);
         }
-        return super.__getslice__(start, stop, step);
+        return super.__getslice__(start,stop,step);
     }
 
-    @Override
-    public void __setslice__(PyObject start, PyObject stop, PyObject step, PyObject value) {
-        if (step != null) {
-            __setitem__(new PySlice(start, stop, step), value);
+    public void __setslice__(PyObject start,PyObject stop,PyObject step,PyObject value) {
+        if (step!=null) {
+            __setitem__(new PySlice(start,stop,step),value);
             return;
         }
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__setslice__");
-        if (impl != null) {
-            PyObject[] indices = PySlice.indices2(this, start, stop);
-            impl.__get__(this, self_type).__call__(indices[0], indices[1], value);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            impl.__get__(this,self_type).__call__(indices[0],indices[1],value);
             return;
         }
-        super.__setslice__(start, stop, step, value);
+        super.__setslice__(start,stop,step,value);
     }
 
-    @Override
-    public void __delslice__(PyObject start, PyObject stop, PyObject step) {
-        if (step != null) {
-            __delitem__(new PySlice(start, stop, step));
+    public void __delslice__(PyObject start,PyObject stop,PyObject step) {
+        if (step!=null) {
+            __delitem__(new PySlice(start,stop,step));
             return;
         }
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__delslice__");
-        if (impl != null) {
-            PyObject[] indices = PySlice.indices2(this, start, stop);
-            impl.__get__(this, self_type).__call__(indices[0], indices[1]);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            impl.__get__(this,self_type).__call__(indices[0],indices[1]);
             return;
         }
-        super.__delslice__(start, stop, step);
+        super.__delslice__(start,stop,step);
     }
 
-    @Override
     public void __delitem__(PyObject key) { // ???
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__delitem__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(key);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delitem__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(key);
             return;
         }
         super.__delitem__(key);
     }
 
-    @Override
-    public PyObject __call__(PyObject args[], String keywords[]) {
-        ThreadState ts = Py.getThreadState();
-        if (ts.recursion_depth++ > ts.systemState.getrecursionlimit()) {
+    public PyObject __call__(PyObject args[],String keywords[]) {
+        ThreadState ts=Py.getThreadState();
+        if (ts.recursion_depth++>ts.systemState.getrecursionlimit())
             throw Py.RuntimeError("maximum __call__ recursion depth exceeded");
-        }
         try {
-            PyType self_type = getType();
-            PyObject impl = self_type.lookup("__call__");
-            if (impl != null) {
-                return impl.__get__(this, self_type).__call__(args, keywords);
-            }
-            return super.__call__(args, keywords);
+            PyType self_type=getType();
+            PyObject impl=self_type.lookup("__call__");
+            if (impl!=null)
+                return impl.__get__(this,self_type).__call__(args,keywords);
+            return super.__call__(args,keywords);
         } finally {
             --ts.recursion_depth;
         }
     }
 
-    @Override
     public PyObject __findattr_ex__(String name) {
-        return Deriveds.__findattr_ex__(this, name);
+        return Deriveds.__findattr_ex__(this,name);
     }
 
-    @Override
-    public void __setattr__(String name, PyObject value) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__setattr__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(PyString.fromInterned(name), value);
+    public void __setattr__(String name,PyObject value) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setattr__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(PyString.fromInterned(name),value);
             return;
         }
-        super.__setattr__(name, value);
+        super.__setattr__(name,value);
     }
 
-    @Override
     public void __delattr__(String name) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__delattr__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(PyString.fromInterned(name));
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delattr__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(PyString.fromInterned(name));
             return;
         }
         super.__delattr__(name);
     }
 
-    @Override
-    public PyObject __get__(PyObject obj, PyObject type) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__get__");
-        if (impl != null) {
-            if (obj == null) {
-                obj = Py.None;
-            }
-            if (type == null) {
-                type = Py.None;
-            }
-            return impl.__get__(this, self_type).__call__(obj, type);
+    public PyObject __get__(PyObject obj,PyObject type) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__get__");
+        if (impl!=null) {
+            if (obj==null)
+                obj=Py.None;
+            if (type==null)
+                type=Py.None;
+            return impl.__get__(this,self_type).__call__(obj,type);
         }
-        return super.__get__(obj, type);
+        return super.__get__(obj,type);
     }
 
-    @Override
-    public void __set__(PyObject obj, PyObject value) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__set__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(obj, value);
+    public void __set__(PyObject obj,PyObject value) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__set__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(obj,value);
             return;
         }
-        super.__set__(obj, value);
+        super.__set__(obj,value);
     }
 
-    @Override
     public void __delete__(PyObject obj) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__delete__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(obj);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delete__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(obj);
             return;
         }
         super.__delete__(obj);
     }
 
-    @Override
-    public PyObject __pow__(PyObject other, PyObject modulo) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__pow__");
-        if (impl != null) {
+    public PyObject __pow__(PyObject other,PyObject modulo) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__pow__");
+        if (impl!=null) {
             PyObject res;
-            if (modulo == null) {
-                res = impl.__get__(this, self_type).__call__(other);
+            if (modulo==null) {
+                res=impl.__get__(this,self_type).__call__(other);
             } else {
-                res = impl.__get__(this, self_type).__call__(other, modulo);
+                res=impl.__get__(this,self_type).__call__(other,modulo);
             }
-            if (res == Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
-        return super.__pow__(other, modulo);
+        return super.__pow__(other,modulo);
     }
 
-    @Override
-    public void dispatch__init__(PyObject[] args, String[] keywords) {
-        Deriveds.dispatch__init__(this, args, keywords);
+    public void dispatch__init__(PyObject[]args,String[]keywords) {
+        Deriveds.dispatch__init__(this,args,keywords);
     }
 
-    @Override
     public PyObject __index__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__index__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyInteger || res instanceof PyLong) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__index__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger||res instanceof PyLong) {
                 return res;
             }
-            throw Py.TypeError(String.format("__index__ returned non-(int,long) (type %s)", res
-                    .getType().fastGetName()));
+            throw Py.TypeError(String.format("__index__ returned non-(int,long) (type %s)",res.getType().fastGetName()));
         }
         return super.__index__();
     }
 
-    @Override
     public Object __tojava__(Class c) {
         // If we are not being asked by the "default" conversion to java, then
         // we can provide this as the result, as long as it is a instance of the
         // specified class. Without this, derived.__tojava__(PyObject.class)
         // would broke. (And that's not pure speculation: PyReflectedFunction's
         // ReflectedArgs asks for things like that).
-        if ((c != Object.class) && (c != Serializable.class) && (c.isInstance(this))) {
+        if ((c!=Object.class)&&(c!=Serializable.class)&&(c.isInstance(this))) {
             return this;
         }
         // Otherwise, we call the derived __tojava__, if it exists:
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__tojava__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(Py.java2py(c)).__tojava__(Object.class);
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__tojava__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__(Py.java2py(c)).__tojava__(Object.class);
         return super.__tojava__(c);
     }
 
-    @Override
     public Object __coerce_ex__(PyObject o) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__coerce__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(o);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__coerce__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(o);
+            if (res==Py.NotImplemented)
                 return Py.None;
-            }
-            if (!(res instanceof PyTuple)) {
+            if (!(res instanceof PyTuple))
                 throw Py.TypeError("__coerce__ didn't return a 2-tuple");
-            }
-            return ((PyTuple)res).getArray();
+            return((PyTuple)res).getArray();
         }
         return super.__coerce_ex__(o);
     }
 
-    @Override
     public PyObject __enter__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__enter__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__enter__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__enter__();
     }
 
-    @Override
     public PyObject fileno() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("fileno");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("fileno");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.fileno();
     }
 
     // Hand-crafted in _io._IOBase.derived
 
-    @Override
-    public long seek(long pos, int whence) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("seek");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(Py.newLong(pos), Py.newInteger(whence))
-                    .asLong();
+    public long seek(long pos,int whence) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("seek");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(Py.newLong(pos),Py.newInteger(whence)).asLong();
         } else {
-            return super.seek(pos, whence);
+            return super.seek(pos,whence);
         }
     }
 
-    @Override
     public long tell() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("tell");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().asLong();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("tell");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().asLong();
         } else {
             return super.tell();
         }
     }
 
-    @Override
     public long truncate(long size) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("truncate");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(Py.newLong(size)).asLong();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("truncate");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(Py.newLong(size)).asLong();
         } else {
             return super.truncate(size);
         }
     }
 
-    @Override
     public long truncate() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("truncate");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().asLong();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("truncate");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().asLong();
         } else {
             return super.truncate();
         }
     }
 
-    @Override
     public void flush() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("flush");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("flush");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__();
         } else {
             super.flush();
         }
     }
 
-    @Override
     public void close() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("close");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("close");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__();
         } else {
             super.close();
         }
     }
 
-    @Override
     public boolean seekable() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("seekable");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().__nonzero__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("seekable");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().__nonzero__();
         } else {
             return super.seekable();
         }
     }
 
-    @Override
     public void _checkSeekable(String msg) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("_checkSeekable");
-        if (impl != null) {
-            PyObject pymsg = msg == null ? Py.None : new PyString(msg);
-            impl.__get__(this, self_type).__call__(pymsg);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("_checkSeekable");
+        if (impl!=null) {
+            PyObject pymsg=msg==null?Py.None:new PyString(msg);
+            impl.__get__(this,self_type).__call__(pymsg);
         } else {
             super._checkSeekable(msg);
         }
     }
 
-    @Override
     public boolean readable() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("readable");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().__nonzero__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("readable");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().__nonzero__();
         } else {
             return super.readable();
         }
     }
 
-    @Override
     public void _checkReadable(String msg) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("_checkReadable");
-        if (impl != null) {
-            PyObject pymsg = msg == null ? Py.None : new PyString(msg);
-            impl.__get__(this, self_type).__call__(pymsg);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("_checkReadable");
+        if (impl!=null) {
+            PyObject pymsg=msg==null?Py.None:new PyString(msg);
+            impl.__get__(this,self_type).__call__(pymsg);
         } else {
             super._checkReadable(msg);
         }
     }
 
-    @Override
     public boolean writable() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("writable");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().__nonzero__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("writable");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().__nonzero__();
         } else {
             return super.writable();
         }
     }
 
-    @Override
     public void _checkWritable(String msg) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("_checkWritable");
-        if (impl != null) {
-            PyObject pymsg = msg == null ? Py.None : new PyString(msg);
-            impl.__get__(this, self_type).__call__(pymsg);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("_checkWritable");
+        if (impl!=null) {
+            PyObject pymsg=msg==null?Py.None:new PyString(msg);
+            impl.__get__(this,self_type).__call__(pymsg);
         } else {
             super._checkWritable(msg);
         }
@@ -1456,80 +1241,73 @@
 
     // Note that closed is a property not a predicate, so no derived method.
 
-    @Override
     public void _checkClosed(String msg) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("_checkClosed");
-        if (impl != null) {
-            PyObject pymsg = msg == null ? Py.None : new PyString(msg);
-            impl.__get__(this, self_type).__call__(pymsg);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("_checkClosed");
+        if (impl!=null) {
+            PyObject pymsg=msg==null?Py.None:new PyString(msg);
+            impl.__get__(this,self_type).__call__(pymsg);
         } else {
             super._checkClosed(msg);
         }
     }
 
-    @Override
-    public boolean __exit__(PyObject type, PyObject value, PyObject traceback) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__exit__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(type, value, traceback).__nonzero__();
+    public boolean __exit__(PyObject type,PyObject value,PyObject traceback) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__exit__");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(type,value,traceback).__nonzero__();
         } else {
-            return super.__exit__(type, value, traceback);
+            return super.__exit__(type,value,traceback);
         }
     }
 
-    @Override
     public boolean isatty() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("isatty");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().__nonzero__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("isatty");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().__nonzero__();
         } else {
             return super.isatty();
         }
     }
 
-    @Override
     public PyObject readline() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("readline");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(Py.None);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("readline");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(Py.None);
         } else {
             return super.readline();
         }
     }
 
-    @Override
     public PyObject readline(int limit) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("readline");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(Py.newInteger(limit));
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("readline");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(Py.newInteger(limit));
         } else {
             return super.readline(limit);
         }
     }
 
-    @Override
     public PyObject readlines(PyObject hint) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("readlines");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(hint);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("readlines");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(hint);
             return res;
         } else {
             return super.readlines(hint);
         }
     }
 
-    @Override
     public void writelines(PyObject lines) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("writelines");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(lines);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("writelines");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(lines);
         } else {
             super.writelines(lines);
         }
diff --git a/src/org/python/modules/_io/PyRawIOBaseDerived.java b/src/org/python/modules/_io/PyRawIOBaseDerived.java
--- a/src/org/python/modules/_io/PyRawIOBaseDerived.java
+++ b/src/org/python/modules/_io/PyRawIOBaseDerived.java
@@ -1,1048 +1,877 @@
-/* Generated file, do not modify. See jython/src/templates/gderived.py. */
+/* Generated file, do not modify.  See jython/src/templates/gderived.py. */
 package org.python.modules._io;
 
 import java.io.Serializable;
-
-import org.python.core.Deriveds;
-import org.python.core.Py;
-import org.python.core.PyBoolean;
-import org.python.core.PyComplex;
-import org.python.core.PyException;
-import org.python.core.PyFloat;
-import org.python.core.PyInteger;
-import org.python.core.PyLong;
-import org.python.core.PyObject;
-import org.python.core.PySequenceIter;
-import org.python.core.PySlice;
-import org.python.core.PyString;
-import org.python.core.PyTuple;
-import org.python.core.PyType;
-import org.python.core.PyUnicode;
-import org.python.core.Slotted;
-import org.python.core.ThreadState;
+import org.python.core.*;
 
 public class PyRawIOBaseDerived extends PyRawIOBase implements Slotted {
 
-    @Override
     public PyObject getSlot(int index) {
         return slots[index];
     }
 
-    @Override
-    public void setSlot(int index, PyObject value) {
-        slots[index] = value;
+    public void setSlot(int index,PyObject value) {
+        slots[index]=value;
     }
 
-    private PyObject[] slots;
+    private PyObject[]slots;
 
     public PyRawIOBaseDerived(PyType subtype) {
         super(subtype);
-        slots = new PyObject[subtype.getNumSlots()];
+        slots=new PyObject[subtype.getNumSlots()];
     }
 
-    @Override
     public PyString __str__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__str__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyString) {
-                return (PyString)res;
-            }
-            throw Py.TypeError("__str__" + " returned non-" + "string" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__str__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__str__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__str__();
     }
 
-    @Override
     public PyString __repr__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__repr__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyString) {
-                return (PyString)res;
-            }
-            throw Py.TypeError("__repr__" + " returned non-" + "string" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__repr__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__repr__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__repr__();
     }
 
-    @Override
     public PyString __hex__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__hex__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyString) {
-                return (PyString)res;
-            }
-            throw Py.TypeError("__hex__" + " returned non-" + "string" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__hex__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__hex__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__hex__();
     }
 
-    @Override
     public PyString __oct__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__oct__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyString) {
-                return (PyString)res;
-            }
-            throw Py.TypeError("__oct__" + " returned non-" + "string" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__oct__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyString)
+                return(PyString)res;
+            throw Py.TypeError("__oct__"+" returned non-"+"string"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__oct__();
     }
 
-    @Override
     public PyFloat __float__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__float__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyFloat) {
-                return (PyFloat)res;
-            }
-            throw Py.TypeError("__float__" + " returned non-" + "float" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__float__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyFloat)
+                return(PyFloat)res;
+            throw Py.TypeError("__float__"+" returned non-"+"float"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__float__();
     }
 
-    @Override
     public PyComplex __complex__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__complex__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyComplex) {
-                return (PyComplex)res;
-            }
-            throw Py.TypeError("__complex__" + " returned non-" + "complex" + " (type "
-                    + res.getType().fastGetName() + ")");
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__complex__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyComplex)
+                return(PyComplex)res;
+            throw Py.TypeError("__complex__"+" returned non-"+"complex"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__complex__();
     }
 
-    @Override
     public PyObject __pos__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__pos__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__pos__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__pos__();
     }
 
-    @Override
     public PyObject __neg__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__neg__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__neg__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__neg__();
     }
 
-    @Override
     public PyObject __abs__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__abs__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__abs__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__abs__();
     }
 
-    @Override
     public PyObject __invert__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__invert__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__invert__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__invert__();
     }
 
-    @Override
     public PyObject __reduce__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__reduce__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__reduce__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__reduce__();
     }
 
-    @Override
     public PyObject __dir__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__dir__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__dir__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__dir__();
     }
 
-    @Override
     public PyObject __add__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__add__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__add__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__add__(other);
     }
 
-    @Override
     public PyObject __radd__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__radd__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__radd__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__radd__(other);
     }
 
-    @Override
     public PyObject __sub__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__sub__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__sub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__sub__(other);
     }
 
-    @Override
     public PyObject __rsub__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rsub__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rsub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rsub__(other);
     }
 
-    @Override
     public PyObject __mul__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__mul__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__mul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__mul__(other);
     }
 
-    @Override
     public PyObject __rmul__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rmul__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rmul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rmul__(other);
     }
 
-    @Override
     public PyObject __div__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__div__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__div__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__div__(other);
     }
 
-    @Override
     public PyObject __rdiv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rdiv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rdiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rdiv__(other);
     }
 
-    @Override
     public PyObject __floordiv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__floordiv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__floordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__floordiv__(other);
     }
 
-    @Override
     public PyObject __rfloordiv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rfloordiv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rfloordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rfloordiv__(other);
     }
 
-    @Override
     public PyObject __truediv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__truediv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__truediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__truediv__(other);
     }
 
-    @Override
     public PyObject __rtruediv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rtruediv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rtruediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rtruediv__(other);
     }
 
-    @Override
     public PyObject __mod__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__mod__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__mod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__mod__(other);
     }
 
-    @Override
     public PyObject __rmod__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rmod__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rmod__(other);
     }
 
-    @Override
     public PyObject __divmod__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__divmod__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__divmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__divmod__(other);
     }
 
-    @Override
     public PyObject __rdivmod__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rdivmod__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rdivmod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rdivmod__(other);
     }
 
-    @Override
     public PyObject __rpow__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rpow__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rpow__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rpow__(other);
     }
 
-    @Override
     public PyObject __lshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__lshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__lshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__lshift__(other);
     }
 
-    @Override
     public PyObject __rlshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rlshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rlshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rlshift__(other);
     }
 
-    @Override
     public PyObject __rshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rshift__(other);
     }
 
-    @Override
     public PyObject __rrshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rrshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rrshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rrshift__(other);
     }
 
-    @Override
     public PyObject __and__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__and__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__and__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__and__(other);
     }
 
-    @Override
     public PyObject __rand__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rand__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rand__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rand__(other);
     }
 
-    @Override
     public PyObject __or__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__or__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__or__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__or__(other);
     }
 
-    @Override
     public PyObject __ror__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ror__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ror__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ror__(other);
     }
 
-    @Override
     public PyObject __xor__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__xor__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__xor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__xor__(other);
     }
 
-    @Override
     public PyObject __rxor__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__rxor__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__rxor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__rxor__(other);
     }
 
-    @Override
     public PyObject __lt__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__lt__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__lt__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__lt__(other);
     }
 
-    @Override
     public PyObject __le__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__le__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__le__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__le__(other);
     }
 
-    @Override
     public PyObject __gt__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__gt__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__gt__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__gt__(other);
     }
 
-    @Override
     public PyObject __ge__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ge__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ge__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ge__(other);
     }
 
-    @Override
     public PyObject __eq__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__eq__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__eq__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__eq__(other);
     }
 
-    @Override
     public PyObject __ne__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ne__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ne__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ne__(other);
     }
 
-    @Override
     public PyObject __format__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__format__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__format__(other);
     }
 
-    @Override
     public PyObject __iadd__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__iadd__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iadd__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__iadd__(other);
     }
 
-    @Override
     public PyObject __isub__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__isub__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__isub__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__isub__(other);
     }
 
-    @Override
     public PyObject __imul__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__imul__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__imul__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__imul__(other);
     }
 
-    @Override
     public PyObject __idiv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__idiv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__idiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__idiv__(other);
     }
 
-    @Override
     public PyObject __ifloordiv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ifloordiv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ifloordiv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ifloordiv__(other);
     }
 
-    @Override
     public PyObject __itruediv__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__itruediv__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__itruediv__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__itruediv__(other);
     }
 
-    @Override
     public PyObject __imod__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__imod__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__imod__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__imod__(other);
     }
 
-    @Override
     public PyObject __ipow__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ipow__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ipow__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ipow__(other);
     }
 
-    @Override
     public PyObject __ilshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ilshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ilshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ilshift__(other);
     }
 
-    @Override
     public PyObject __irshift__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__irshift__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__irshift__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__irshift__(other);
     }
 
-    @Override
     public PyObject __iand__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__iand__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iand__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__iand__(other);
     }
 
-    @Override
     public PyObject __ior__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ior__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ior__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ior__(other);
     }
 
-    @Override
     public PyObject __ixor__(PyObject other) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__ixor__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(other);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__ixor__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
         return super.__ixor__(other);
     }
 
-    @Override
     public PyObject __int__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__int__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyInteger || res instanceof PyLong) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__int__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger||res instanceof PyLong)
                 return res;
-            }
-            throw Py.TypeError("__int__" + " should return an integer");
+            throw Py.TypeError("__int__"+" should return an integer");
         }
         return super.__int__();
     }
 
-    @Override
     public PyObject __long__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__long__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyLong || res instanceof PyInteger) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__long__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyLong||res instanceof PyInteger)
                 return res;
-            }
-            throw Py.TypeError("__long__" + " returned non-" + "long" + " (type "
-                    + res.getType().fastGetName() + ")");
+            throw Py.TypeError("__long__"+" returned non-"+"long"+" (type "+res.getType().fastGetName()+")");
         }
         return super.__long__();
     }
 
-    @Override
     public int hashCode() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__hash__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__hash__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
             if (res instanceof PyInteger) {
-                return ((PyInteger)res).getValue();
-            } else if (res instanceof PyLong) {
-                return ((PyLong)res).getValue().intValue();
-            }
+                return((PyInteger)res).getValue();
+            } else
+                if (res instanceof PyLong) {
+                    return((PyLong)res).getValue().intValue();
+                }
             throw Py.TypeError("__hash__ should return a int");
         }
-        if (self_type.lookup("__eq__") != null || self_type.lookup("__cmp__") != null) {
-            throw Py.TypeError(String.format("unhashable type: '%.200s'", getType().fastGetName()));
+        if (self_type.lookup("__eq__")!=null||self_type.lookup("__cmp__")!=null) {
+            throw Py.TypeError(String.format("unhashable type: '%.200s'",getType().fastGetName()));
         }
         return super.hashCode();
     }
 
-    @Override
     public PyUnicode __unicode__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__unicode__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyUnicode) {
-                return (PyUnicode)res;
-            }
-            if (res instanceof PyString) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__unicode__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyUnicode)
+                return(PyUnicode)res;
+            if (res instanceof PyString)
                 return new PyUnicode((PyString)res);
-            }
-            throw Py.TypeError("__unicode__" + " should return a " + "unicode");
+            throw Py.TypeError("__unicode__"+" should return a "+"unicode");
         }
         return super.__unicode__();
     }
 
-    @Override
     public int __cmp__(PyObject other) {
-        PyType self_type = getType();
-        PyObject[] where_type = new PyObject[1];
-        PyObject impl = self_type.lookup_where("__cmp__", where_type);
+        PyType self_type=getType();
+        PyObject[]where_type=new PyObject[1];
+        PyObject impl=self_type.lookup_where("__cmp__",where_type);
         // Full Compatibility with CPython __cmp__:
         // If the derived type don't override __cmp__, the
         // *internal* super().__cmp__ should be called, not the
         // exposed one. The difference is that the exposed __cmp__
         // throws a TypeError if the argument is an instance of the same type.
-        if (impl == null || where_type[0] == TYPE || Py.isSubClass(TYPE, where_type[0])) {
+        if (impl==null||where_type[0]==TYPE||Py.isSubClass(TYPE,where_type[0])) {
             return super.__cmp__(other);
         }
-        PyObject res = impl.__get__(this, self_type).__call__(other);
-        if (res == Py.NotImplemented) {
-            return -2;
+        PyObject res=impl.__get__(this,self_type).__call__(other);
+        if (res==Py.NotImplemented) {
+            return-2;
         }
-        int c = res.asInt();
-        return c < 0 ? -1 : c > 0 ? 1 : 0;
+        int c=res.asInt();
+        return c<0?-1:c>0?1:0;
     }
 
-    @Override
     public boolean __nonzero__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__nonzero__");
-        if (impl == null) {
-            impl = self_type.lookup("__len__");
-            if (impl == null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__nonzero__");
+        if (impl==null) {
+            impl=self_type.lookup("__len__");
+            if (impl==null)
                 return super.__nonzero__();
-            }
         }
-        PyObject o = impl.__get__(this, self_type).__call__();
-        Class c = o.getClass();
-        if (c != PyInteger.class && c != PyBoolean.class) {
-            throw Py.TypeError(String.format("__nonzero__ should return bool or int, returned %s",
-                    self_type.getName()));
+        PyObject o=impl.__get__(this,self_type).__call__();
+        Class c=o.getClass();
+        if (c!=PyInteger.class&&c!=PyBoolean.class) {
+            throw Py.TypeError(String.format("__nonzero__ should return bool or int, returned %s",self_type.getName()));
         }
         return o.__nonzero__();
     }
 
-    @Override
     public boolean __contains__(PyObject o) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__contains__");
-        if (impl == null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__contains__");
+        if (impl==null)
             return super.__contains__(o);
-        }
-        return impl.__get__(this, self_type).__call__(o).__nonzero__();
+        return impl.__get__(this,self_type).__call__(o).__nonzero__();
     }
 
-    @Override
     public int __len__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__len__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyInteger) {
-                return ((PyInteger)res).getValue();
-            }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__len__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger)
+                return((PyInteger)res).getValue();
             throw Py.TypeError("__len__ should return a int");
         }
         return super.__len__();
     }
 
-    @Override
     public PyObject __iter__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__iter__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
-        impl = self_type.lookup("__getitem__");
-        if (impl == null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__iter__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
+        impl=self_type.lookup("__getitem__");
+        if (impl==null)
             return super.__iter__();
-        }
         return new PySequenceIter(this);
     }
 
-    @Override
     public PyObject __iternext__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("next");
-        if (impl != null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("next");
+        if (impl!=null) {
             try {
-                return impl.__get__(this, self_type).__call__();
+                return impl.__get__(this,self_type).__call__();
             } catch (PyException exc) {
-                if (exc.match(Py.StopIteration)) {
+                if (exc.match(Py.StopIteration))
                     return null;
-                }
                 throw exc;
             }
         }
         return super.__iternext__(); // ???
     }
 
-    @Override
     public PyObject __finditem__(PyObject key) { // ???
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__getitem__");
-        if (impl != null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
             try {
-                return impl.__get__(this, self_type).__call__(key);
+                return impl.__get__(this,self_type).__call__(key);
             } catch (PyException exc) {
-                if (exc.match(Py.LookupError)) {
+                if (exc.match(Py.LookupError))
                     return null;
-                }
                 throw exc;
             }
-        }
         return super.__finditem__(key);
     }
 
-    @Override
     public PyObject __finditem__(int key) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__getitem__");
-        if (impl != null) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
             try {
-                return impl.__get__(this, self_type).__call__(new PyInteger(key));
+                return impl.__get__(this,self_type).__call__(new PyInteger(key));
             } catch (PyException exc) {
-                if (exc.match(Py.LookupError)) {
+                if (exc.match(Py.LookupError))
                     return null;
-                }
                 throw exc;
             }
-        }
         return super.__finditem__(key);
     }
 
-    @Override
     public PyObject __getitem__(PyObject key) {
         // Same as __finditem__, without swallowing LookupErrors. This allows
         // __getitem__ implementations written in Python to raise custom
@@ -1053,7 +882,7 @@
         // in terms of __finditem__. Therefore, we would end with an infinite
         // loop when self_type.lookup("__getitem__") returns null:
         //
-        // __getitem__ -> super.__getitem__ -> __finditem__ -> __getitem__
+        //  __getitem__ -> super.__getitem__ -> __finditem__ -> __getitem__
         //
         // By duplicating the (short) lookup and call code, we are safe, because
         // the call chains will be:
@@ -1062,393 +891,349 @@
         //
         // __getitem__ -> super.__getitem__ -> __finditem__ -> super.__finditem__
 
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__getitem__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(key);
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getitem__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__(key);
         return super.__getitem__(key);
     }
 
-    @Override
-    public void __setitem__(PyObject key, PyObject value) { // ???
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__setitem__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(key, value);
+    public void __setitem__(PyObject key,PyObject value) { // ???
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setitem__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(key,value);
             return;
         }
-        super.__setitem__(key, value);
+        super.__setitem__(key,value);
     }
 
-    @Override
-    public PyObject __getslice__(PyObject start, PyObject stop, PyObject step) { // ???
-        if (step != null) {
-            return __getitem__(new PySlice(start, stop, step));
+    public PyObject __getslice__(PyObject start,PyObject stop,PyObject step) { // ???
+        if (step!=null) {
+            return __getitem__(new PySlice(start,stop,step));
         }
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__getslice__");
-        if (impl != null) {
-            PyObject[] indices = PySlice.indices2(this, start, stop);
-            return impl.__get__(this, self_type).__call__(indices[0], indices[1]);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__getslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            return impl.__get__(this,self_type).__call__(indices[0],indices[1]);
         }
-        return super.__getslice__(start, stop, step);
+        return super.__getslice__(start,stop,step);
     }
 
-    @Override
-    public void __setslice__(PyObject start, PyObject stop, PyObject step, PyObject value) {
-        if (step != null) {
-            __setitem__(new PySlice(start, stop, step), value);
+    public void __setslice__(PyObject start,PyObject stop,PyObject step,PyObject value) {
+        if (step!=null) {
+            __setitem__(new PySlice(start,stop,step),value);
             return;
         }
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__setslice__");
-        if (impl != null) {
-            PyObject[] indices = PySlice.indices2(this, start, stop);
-            impl.__get__(this, self_type).__call__(indices[0], indices[1], value);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            impl.__get__(this,self_type).__call__(indices[0],indices[1],value);
             return;
         }
-        super.__setslice__(start, stop, step, value);
+        super.__setslice__(start,stop,step,value);
     }
 
-    @Override
-    public void __delslice__(PyObject start, PyObject stop, PyObject step) {
-        if (step != null) {
-            __delitem__(new PySlice(start, stop, step));
+    public void __delslice__(PyObject start,PyObject stop,PyObject step) {
+        if (step!=null) {
+            __delitem__(new PySlice(start,stop,step));
             return;
         }
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__delslice__");
-        if (impl != null) {
-            PyObject[] indices = PySlice.indices2(this, start, stop);
-            impl.__get__(this, self_type).__call__(indices[0], indices[1]);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delslice__");
+        if (impl!=null) {
+            PyObject[]indices=PySlice.indices2(this,start,stop);
+            impl.__get__(this,self_type).__call__(indices[0],indices[1]);
             return;
         }
-        super.__delslice__(start, stop, step);
+        super.__delslice__(start,stop,step);
     }
 
-    @Override
     public void __delitem__(PyObject key) { // ???
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__delitem__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(key);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delitem__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(key);
             return;
         }
         super.__delitem__(key);
     }
 
-    @Override
-    public PyObject __call__(PyObject args[], String keywords[]) {
-        ThreadState ts = Py.getThreadState();
-        if (ts.recursion_depth++ > ts.systemState.getrecursionlimit()) {
+    public PyObject __call__(PyObject args[],String keywords[]) {
+        ThreadState ts=Py.getThreadState();
+        if (ts.recursion_depth++>ts.systemState.getrecursionlimit())
             throw Py.RuntimeError("maximum __call__ recursion depth exceeded");
-        }
         try {
-            PyType self_type = getType();
-            PyObject impl = self_type.lookup("__call__");
-            if (impl != null) {
-                return impl.__get__(this, self_type).__call__(args, keywords);
-            }
-            return super.__call__(args, keywords);
+            PyType self_type=getType();
+            PyObject impl=self_type.lookup("__call__");
+            if (impl!=null)
+                return impl.__get__(this,self_type).__call__(args,keywords);
+            return super.__call__(args,keywords);
         } finally {
             --ts.recursion_depth;
         }
     }
 
-    @Override
     public PyObject __findattr_ex__(String name) {
-        return Deriveds.__findattr_ex__(this, name);
+        return Deriveds.__findattr_ex__(this,name);
     }
 
-    @Override
-    public void __setattr__(String name, PyObject value) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__setattr__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(PyString.fromInterned(name), value);
+    public void __setattr__(String name,PyObject value) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__setattr__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(PyString.fromInterned(name),value);
             return;
         }
-        super.__setattr__(name, value);
+        super.__setattr__(name,value);
     }
 
-    @Override
     public void __delattr__(String name) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__delattr__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(PyString.fromInterned(name));
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delattr__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(PyString.fromInterned(name));
             return;
         }
         super.__delattr__(name);
     }
 
-    @Override
-    public PyObject __get__(PyObject obj, PyObject type) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__get__");
-        if (impl != null) {
-            if (obj == null) {
-                obj = Py.None;
-            }
-            if (type == null) {
-                type = Py.None;
-            }
-            return impl.__get__(this, self_type).__call__(obj, type);
+    public PyObject __get__(PyObject obj,PyObject type) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__get__");
+        if (impl!=null) {
+            if (obj==null)
+                obj=Py.None;
+            if (type==null)
+                type=Py.None;
+            return impl.__get__(this,self_type).__call__(obj,type);
         }
-        return super.__get__(obj, type);
+        return super.__get__(obj,type);
     }
 
-    @Override
-    public void __set__(PyObject obj, PyObject value) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__set__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(obj, value);
+    public void __set__(PyObject obj,PyObject value) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__set__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(obj,value);
             return;
         }
-        super.__set__(obj, value);
+        super.__set__(obj,value);
     }
 
-    @Override
     public void __delete__(PyObject obj) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__delete__");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(obj);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__delete__");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(obj);
             return;
         }
         super.__delete__(obj);
     }
 
-    @Override
-    public PyObject __pow__(PyObject other, PyObject modulo) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__pow__");
-        if (impl != null) {
+    public PyObject __pow__(PyObject other,PyObject modulo) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__pow__");
+        if (impl!=null) {
             PyObject res;
-            if (modulo == null) {
-                res = impl.__get__(this, self_type).__call__(other);
+            if (modulo==null) {
+                res=impl.__get__(this,self_type).__call__(other);
             } else {
-                res = impl.__get__(this, self_type).__call__(other, modulo);
+                res=impl.__get__(this,self_type).__call__(other,modulo);
             }
-            if (res == Py.NotImplemented) {
+            if (res==Py.NotImplemented)
                 return null;
-            }
             return res;
         }
-        return super.__pow__(other, modulo);
+        return super.__pow__(other,modulo);
     }
 
-    @Override
-    public void dispatch__init__(PyObject[] args, String[] keywords) {
-        Deriveds.dispatch__init__(this, args, keywords);
+    public void dispatch__init__(PyObject[]args,String[]keywords) {
+        Deriveds.dispatch__init__(this,args,keywords);
     }
 
-    @Override
     public PyObject __index__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__index__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__();
-            if (res instanceof PyInteger || res instanceof PyLong) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__index__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__();
+            if (res instanceof PyInteger||res instanceof PyLong) {
                 return res;
             }
-            throw Py.TypeError(String.format("__index__ returned non-(int,long) (type %s)", res
-                    .getType().fastGetName()));
+            throw Py.TypeError(String.format("__index__ returned non-(int,long) (type %s)",res.getType().fastGetName()));
         }
         return super.__index__();
     }
 
-    @Override
     public Object __tojava__(Class c) {
         // If we are not being asked by the "default" conversion to java, then
         // we can provide this as the result, as long as it is a instance of the
         // specified class. Without this, derived.__tojava__(PyObject.class)
         // would broke. (And that's not pure speculation: PyReflectedFunction's
         // ReflectedArgs asks for things like that).
-        if ((c != Object.class) && (c != Serializable.class) && (c.isInstance(this))) {
+        if ((c!=Object.class)&&(c!=Serializable.class)&&(c.isInstance(this))) {
             return this;
         }
         // Otherwise, we call the derived __tojava__, if it exists:
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__tojava__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(Py.java2py(c)).__tojava__(Object.class);
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__tojava__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__(Py.java2py(c)).__tojava__(Object.class);
         return super.__tojava__(c);
     }
 
-    @Override
     public Object __coerce_ex__(PyObject o) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__coerce__");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(o);
-            if (res == Py.NotImplemented) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__coerce__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(o);
+            if (res==Py.NotImplemented)
                 return Py.None;
-            }
-            if (!(res instanceof PyTuple)) {
+            if (!(res instanceof PyTuple))
                 throw Py.TypeError("__coerce__ didn't return a 2-tuple");
-            }
-            return ((PyTuple)res).getArray();
+            return((PyTuple)res).getArray();
         }
         return super.__coerce_ex__(o);
     }
 
-    @Override
     public PyObject __enter__() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__enter__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__enter__");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.__enter__();
     }
 
-    @Override
     public PyObject fileno() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("fileno");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("fileno");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.fileno();
     }
 
     // Hand-crafted in _io._IOBase.derived
 
-    @Override
-    public long seek(long pos, int whence) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("seek");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(Py.newLong(pos), Py.newInteger(whence))
-                    .asLong();
+    public long seek(long pos,int whence) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("seek");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(Py.newLong(pos),Py.newInteger(whence)).asLong();
         } else {
-            return super.seek(pos, whence);
+            return super.seek(pos,whence);
         }
     }
 
-    @Override
     public long tell() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("tell");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().asLong();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("tell");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().asLong();
         } else {
             return super.tell();
         }
     }
 
-    @Override
     public long truncate(long size) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("truncate");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(Py.newLong(size)).asLong();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("truncate");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(Py.newLong(size)).asLong();
         } else {
             return super.truncate(size);
         }
     }
 
-    @Override
     public long truncate() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("truncate");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().asLong();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("truncate");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().asLong();
         } else {
             return super.truncate();
         }
     }
 
-    @Override
     public void flush() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("flush");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("flush");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__();
         } else {
             super.flush();
         }
     }
 
-    @Override
     public void close() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("close");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("close");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__();
         } else {
             super.close();
         }
     }
 
-    @Override
     public boolean seekable() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("seekable");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().__nonzero__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("seekable");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().__nonzero__();
         } else {
             return super.seekable();
         }
     }
 
-    @Override
     public void _checkSeekable(String msg) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("_checkSeekable");
-        if (impl != null) {
-            PyObject pymsg = msg == null ? Py.None : new PyString(msg);
-            impl.__get__(this, self_type).__call__(pymsg);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("_checkSeekable");
+        if (impl!=null) {
+            PyObject pymsg=msg==null?Py.None:new PyString(msg);
+            impl.__get__(this,self_type).__call__(pymsg);
         } else {
             super._checkSeekable(msg);
         }
     }
 
-    @Override
     public boolean readable() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("readable");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().__nonzero__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("readable");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().__nonzero__();
         } else {
             return super.readable();
         }
     }
 
-    @Override
     public void _checkReadable(String msg) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("_checkReadable");
-        if (impl != null) {
-            PyObject pymsg = msg == null ? Py.None : new PyString(msg);
-            impl.__get__(this, self_type).__call__(pymsg);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("_checkReadable");
+        if (impl!=null) {
+            PyObject pymsg=msg==null?Py.None:new PyString(msg);
+            impl.__get__(this,self_type).__call__(pymsg);
         } else {
             super._checkReadable(msg);
         }
     }
 
-    @Override
     public boolean writable() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("writable");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().__nonzero__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("writable");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().__nonzero__();
         } else {
             return super.writable();
         }
     }
 
-    @Override
     public void _checkWritable(String msg) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("_checkWritable");
-        if (impl != null) {
-            PyObject pymsg = msg == null ? Py.None : new PyString(msg);
-            impl.__get__(this, self_type).__call__(pymsg);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("_checkWritable");
+        if (impl!=null) {
+            PyObject pymsg=msg==null?Py.None:new PyString(msg);
+            impl.__get__(this,self_type).__call__(pymsg);
         } else {
             super._checkWritable(msg);
         }
@@ -1456,125 +1241,113 @@
 
     // Note that closed is a property not a predicate, so no derived method.
 
-    @Override
     public void _checkClosed(String msg) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("_checkClosed");
-        if (impl != null) {
-            PyObject pymsg = msg == null ? Py.None : new PyString(msg);
-            impl.__get__(this, self_type).__call__(pymsg);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("_checkClosed");
+        if (impl!=null) {
+            PyObject pymsg=msg==null?Py.None:new PyString(msg);
+            impl.__get__(this,self_type).__call__(pymsg);
         } else {
             super._checkClosed(msg);
         }
     }
 
-    @Override
-    public boolean __exit__(PyObject type, PyObject value, PyObject traceback) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("__exit__");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(type, value, traceback).__nonzero__();
+    public boolean __exit__(PyObject type,PyObject value,PyObject traceback) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__exit__");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(type,value,traceback).__nonzero__();
         } else {
-            return super.__exit__(type, value, traceback);
+            return super.__exit__(type,value,traceback);
         }
     }
 
-    @Override
     public boolean isatty() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("isatty");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__().__nonzero__();
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("isatty");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__().__nonzero__();
         } else {
             return super.isatty();
         }
     }
 
-    @Override
     public PyObject readline() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("readline");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(Py.None);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("readline");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(Py.None);
         } else {
             return super.readline();
         }
     }
 
-    @Override
     public PyObject readline(int limit) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("readline");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(Py.newInteger(limit));
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("readline");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(Py.newInteger(limit));
         } else {
             return super.readline(limit);
         }
     }
 
-    @Override
     public PyObject readlines(PyObject hint) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("readlines");
-        if (impl != null) {
-            PyObject res = impl.__get__(this, self_type).__call__(hint);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("readlines");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(hint);
             return res;
         } else {
             return super.readlines(hint);
         }
     }
 
-    @Override
     public void writelines(PyObject lines) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("writelines");
-        if (impl != null) {
-            impl.__get__(this, self_type).__call__(lines);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("writelines");
+        if (impl!=null) {
+            impl.__get__(this,self_type).__call__(lines);
         } else {
             super.writelines(lines);
         }
     }
 
-    @Override
     public PyObject readall() {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("readall");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__();
-        }
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("readall");
+        if (impl!=null)
+            return impl.__get__(this,self_type).__call__();
         return super.readall();
     }
 
     // Hand-crafted in _io._RawIOBase.derived
 
-    @Override
     public PyObject read(int n) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("read");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(Py.newInteger(n));
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("read");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(Py.newInteger(n));
         } else {
             return super.read(n);
         }
     }
 
-    @Override
     public PyObject readinto(PyObject b) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("readinto");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(b);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("readinto");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(b);
         } else {
             return super.readinto(b);
         }
     }
 
-    @Override
     public PyObject write(PyObject b) {
-        PyType self_type = getType();
-        PyObject impl = self_type.lookup("write");
-        if (impl != null) {
-            return impl.__get__(this, self_type).__call__(b);
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("write");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(b);
         } else {
             return super.write(b);
         }
diff --git a/src/org/python/modules/_weakref/ReferenceTypeDerived.java b/src/org/python/modules/_weakref/ReferenceTypeDerived.java
--- a/src/org/python/modules/_weakref/ReferenceTypeDerived.java
+++ b/src/org/python/modules/_weakref/ReferenceTypeDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/bz2/PyBZ2CompressorDerived.java b/src/org/python/modules/bz2/PyBZ2CompressorDerived.java
--- a/src/org/python/modules/bz2/PyBZ2CompressorDerived.java
+++ b/src/org/python/modules/bz2/PyBZ2CompressorDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/bz2/PyBZ2DecompressorDerived.java b/src/org/python/modules/bz2/PyBZ2DecompressorDerived.java
--- a/src/org/python/modules/bz2/PyBZ2DecompressorDerived.java
+++ b/src/org/python/modules/bz2/PyBZ2DecompressorDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/bz2/PyBZ2FileDerived.java b/src/org/python/modules/bz2/PyBZ2FileDerived.java
--- a/src/org/python/modules/bz2/PyBZ2FileDerived.java
+++ b/src/org/python/modules/bz2/PyBZ2FileDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/PyTeeIteratorDerived.java b/src/org/python/modules/itertools/PyTeeIteratorDerived.java
--- a/src/org/python/modules/itertools/PyTeeIteratorDerived.java
+++ b/src/org/python/modules/itertools/PyTeeIteratorDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/chainDerived.java b/src/org/python/modules/itertools/chainDerived.java
--- a/src/org/python/modules/itertools/chainDerived.java
+++ b/src/org/python/modules/itertools/chainDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/combinationsDerived.java b/src/org/python/modules/itertools/combinationsDerived.java
--- a/src/org/python/modules/itertools/combinationsDerived.java
+++ b/src/org/python/modules/itertools/combinationsDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/combinationsWithReplacementDerived.java b/src/org/python/modules/itertools/combinationsWithReplacementDerived.java
--- a/src/org/python/modules/itertools/combinationsWithReplacementDerived.java
+++ b/src/org/python/modules/itertools/combinationsWithReplacementDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/compressDerived.java b/src/org/python/modules/itertools/compressDerived.java
--- a/src/org/python/modules/itertools/compressDerived.java
+++ b/src/org/python/modules/itertools/compressDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/countDerived.java b/src/org/python/modules/itertools/countDerived.java
--- a/src/org/python/modules/itertools/countDerived.java
+++ b/src/org/python/modules/itertools/countDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/cycleDerived.java b/src/org/python/modules/itertools/cycleDerived.java
--- a/src/org/python/modules/itertools/cycleDerived.java
+++ b/src/org/python/modules/itertools/cycleDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/dropwhileDerived.java b/src/org/python/modules/itertools/dropwhileDerived.java
--- a/src/org/python/modules/itertools/dropwhileDerived.java
+++ b/src/org/python/modules/itertools/dropwhileDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/groupbyDerived.java b/src/org/python/modules/itertools/groupbyDerived.java
--- a/src/org/python/modules/itertools/groupbyDerived.java
+++ b/src/org/python/modules/itertools/groupbyDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/ifilterDerived.java b/src/org/python/modules/itertools/ifilterDerived.java
--- a/src/org/python/modules/itertools/ifilterDerived.java
+++ b/src/org/python/modules/itertools/ifilterDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/ifilterfalseDerived.java b/src/org/python/modules/itertools/ifilterfalseDerived.java
--- a/src/org/python/modules/itertools/ifilterfalseDerived.java
+++ b/src/org/python/modules/itertools/ifilterfalseDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/isliceDerived.java b/src/org/python/modules/itertools/isliceDerived.java
--- a/src/org/python/modules/itertools/isliceDerived.java
+++ b/src/org/python/modules/itertools/isliceDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/izipDerived.java b/src/org/python/modules/itertools/izipDerived.java
--- a/src/org/python/modules/itertools/izipDerived.java
+++ b/src/org/python/modules/itertools/izipDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/izipLongestDerived.java b/src/org/python/modules/itertools/izipLongestDerived.java
--- a/src/org/python/modules/itertools/izipLongestDerived.java
+++ b/src/org/python/modules/itertools/izipLongestDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/permutationsDerived.java b/src/org/python/modules/itertools/permutationsDerived.java
--- a/src/org/python/modules/itertools/permutationsDerived.java
+++ b/src/org/python/modules/itertools/permutationsDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/productDerived.java b/src/org/python/modules/itertools/productDerived.java
--- a/src/org/python/modules/itertools/productDerived.java
+++ b/src/org/python/modules/itertools/productDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/repeatDerived.java b/src/org/python/modules/itertools/repeatDerived.java
--- a/src/org/python/modules/itertools/repeatDerived.java
+++ b/src/org/python/modules/itertools/repeatDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/starmapDerived.java b/src/org/python/modules/itertools/starmapDerived.java
--- a/src/org/python/modules/itertools/starmapDerived.java
+++ b/src/org/python/modules/itertools/starmapDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/itertools/takewhileDerived.java b/src/org/python/modules/itertools/takewhileDerived.java
--- a/src/org/python/modules/itertools/takewhileDerived.java
+++ b/src/org/python/modules/itertools/takewhileDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/random/PyRandomDerived.java b/src/org/python/modules/random/PyRandomDerived.java
--- a/src/org/python/modules/random/PyRandomDerived.java
+++ b/src/org/python/modules/random/PyRandomDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/thread/PyLocalDerived.java b/src/org/python/modules/thread/PyLocalDerived.java
--- a/src/org/python/modules/thread/PyLocalDerived.java
+++ b/src/org/python/modules/thread/PyLocalDerived.java
@@ -537,6 +537,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");
diff --git a/src/org/python/modules/zipimport/zipimporterDerived.java b/src/org/python/modules/zipimport/zipimporterDerived.java
--- a/src/org/python/modules/zipimport/zipimporterDerived.java
+++ b/src/org/python/modules/zipimport/zipimporterDerived.java
@@ -561,6 +561,18 @@
         return super.__ne__(other);
     }
 
+    public PyObject __format__(PyObject other) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__format__");
+        if (impl!=null) {
+            PyObject res=impl.__get__(this,self_type).__call__(other);
+            if (res==Py.NotImplemented)
+                return null;
+            return res;
+        }
+        return super.__format__(other);
+    }
+
     public PyObject __iadd__(PyObject other) {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__iadd__");

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sun May 11 07:44:36 2014
From: jython-checkins at python.org (jim.baker)
Date: Sun, 11 May 2014 07:44:36 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Bumped_copyright_date_range?=
	=?utf-8?q?_to_include_2014?=
Message-ID: <3gRDkD3trlz7Lk1@mail.python.org>

http://hg.python.org/jython/rev/d9c4641be70a
changeset:   7242:d9c4641be70a
user:        Jim Baker <jim.baker at rackspace.com>
date:        Sat May 10 23:44:28 2014 -0600
summary:
  Bumped copyright date range to include 2014

files:
  src/org/python/core/PySystemState.java |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/src/org/python/core/PySystemState.java b/src/org/python/core/PySystemState.java
--- a/src/org/python/core/PySystemState.java
+++ b/src/org/python/core/PySystemState.java
@@ -94,7 +94,7 @@
      */
 
     public static final PyObject copyright = Py
-            .newString("Copyright (c) 2000-2013 Jython Developers.\n" + "All rights reserved.\n\n" +
+            .newString("Copyright (c) 2000-2014 Jython Developers.\n" + "All rights reserved.\n\n" +
 
             "Copyright (c) 2000 BeOpen.com.\n" + "All Rights Reserved.\n\n" +
 

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Tue May 13 15:59:16 2014
From: jython-checkins at python.org (jim.baker)
Date: Tue, 13 May 2014 15:59:16 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Add_Indra_Talip_to_ACKNOWLE?=
	=?utf-8?q?DGMENTS?=
Message-ID: <3gSgc46SVTz7LkD@mail.python.org>

http://hg.python.org/jython/rev/290759ab1d9c
changeset:   7243:290759ab1d9c
user:        Jim Baker <jim.baker at rackspace.com>
date:        Tue May 13 09:39:25 2014 -0400
summary:
  Add Indra Talip to ACKNOWLEDGMENTS

files:
  ACKNOWLEDGMENTS |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/ACKNOWLEDGMENTS b/ACKNOWLEDGMENTS
--- a/ACKNOWLEDGMENTS
+++ b/ACKNOWLEDGMENTS
@@ -105,6 +105,7 @@
     Santoso Wijaya
     Brandon Pedersen
     Chris Simpson
+    Indra Talip
 
 Local Variables:
 mode: indented-text

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Tue May 13 15:59:18 2014
From: jython-checkins at python.org (jim.baker)
Date: Tue, 13 May 2014 15:59:18 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Raise_ValueError_on_args_of?=
 =?utf-8?q?_invalid_domain_for_math=2Esqrt=2C_math=2Elog_to?=
Message-ID: <3gSgc641q4z7Lkd@mail.python.org>

http://hg.python.org/jython/rev/a5b000944a51
changeset:   7244:a5b000944a51
user:        Santoso Wijaya <santoso.wijaya at gmail.com>
date:        Tue May 13 09:41:50 2014 -0400
summary:
  Raise ValueError on args of invalid domain for math.sqrt, math.log to
conform to CPython. Fixes #2140.

files:
  ACKNOWLEDGMENTS                  |  1 +
  Lib/test/test_math_jy.py         |  9 +++++++++
  src/org/python/modules/math.java |  4 ++--
  3 files changed, 12 insertions(+), 2 deletions(-)


diff --git a/ACKNOWLEDGMENTS b/ACKNOWLEDGMENTS
--- a/ACKNOWLEDGMENTS
+++ b/ACKNOWLEDGMENTS
@@ -106,6 +106,7 @@
     Brandon Pedersen
     Chris Simpson
     Indra Talip
+    Michael B?sch
 
 Local Variables:
 mode: indented-text
diff --git a/Lib/test/test_math_jy.py b/Lib/test/test_math_jy.py
--- a/Lib/test/test_math_jy.py
+++ b/Lib/test/test_math_jy.py
@@ -28,6 +28,15 @@
         self.assertEqual(inf, math.hypot(ninf, 4))
         self.assertEqual(inf, math.hypot(4, ninf))
 
+    def test_math_domain_error(self):
+        self.assertRaises(ValueError, math.sqrt, -1)
+        self.assertRaises(ValueError, math.sqrt, -1.5)
+        self.assertRaises(ValueError, math.sqrt, -0.5)
+        self.assertRaises(ValueError, math.log, 0)
+        self.assertRaises(ValueError, math.log, -1)
+        self.assertRaises(ValueError, math.log, -1.5)
+        self.assertRaises(ValueError, math.log, -0.5)
+
 
 def test_main():
     test_support.run_unittest(MathTestCase)
diff --git a/src/org/python/modules/math.java b/src/org/python/modules/math.java
--- a/src/org/python/modules/math.java
+++ b/src/org/python/modules/math.java
@@ -318,7 +318,7 @@
         if (ispinf(v)) {
             return v;
         }
-        if (isninf(v) || v == MINUS_ONE) {
+        if (isninf(v) || v < MINUS_ZERO) {
             throwMathDomainValueError();
         }
         return Math.sqrt(v);
@@ -545,7 +545,7 @@
     }
 
     private static double log(double v) {
-        if (isninf(v)) {
+        if (isninf(v) || v <= ZERO) {
             throwMathDomainValueError();
         }
         if (isinf(v) || isnan(v)) {

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Tue May 13 18:32:18 2014
From: jython-checkins at python.org (frank.wierzbicki)
Date: Tue, 13 May 2014 18:32:18 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_=231896215_findResource=28s?=
	=?utf-8?q?=29_for_SyspathJavaLoader?=
Message-ID: <3gSl0f65V5z7Ljy@mail.python.org>

http://hg.python.org/jython/rev/ab99e96d9192
changeset:   7245:ab99e96d9192
parent:      7242:d9c4641be70a
user:        Richard Eckart de Castilho <richard.eckart at gmail.com>
date:        Tue May 13 16:30:39 2014 +0000
summary:
  #1896215 findResource(s) for SyspathJavaLoader

files:
  ACKNOWLEDGMENTS                            |   1 +
  NEWS                                       |   6 +
  src/org/python/core/SyspathJavaLoader.java |  52 ++++++++++
  3 files changed, 59 insertions(+), 0 deletions(-)


diff --git a/ACKNOWLEDGMENTS b/ACKNOWLEDGMENTS
--- a/ACKNOWLEDGMENTS
+++ b/ACKNOWLEDGMENTS
@@ -105,6 +105,7 @@
     Santoso Wijaya
     Brandon Pedersen
     Chris Simpson
+    Richard Eckart de Castilho
 
 Local Variables:
 mode: indented-text
diff --git a/NEWS b/NEWS
--- a/NEWS
+++ b/NEWS
@@ -1,5 +1,11 @@
 Jython NEWS
 
+Jython 2.7b3
+  Bugs Fixed
+
+  New Features
+    - [ 1896215 ] findResource(s) for SyspathJavaLoader
+
 Jython 2.7b2
   Bugs Fixed
     - [ 1753 ] zlib doesn't call end() on compress and decompress
diff --git a/src/org/python/core/SyspathJavaLoader.java b/src/org/python/core/SyspathJavaLoader.java
--- a/src/org/python/core/SyspathJavaLoader.java
+++ b/src/org/python/core/SyspathJavaLoader.java
@@ -10,6 +10,10 @@
 import java.io.InputStream;
 import java.net.MalformedURLException;
 import java.net.URL;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.Enumeration;
+import java.util.List;
 import java.util.StringTokenizer;
 import java.util.zip.ZipEntry;
 
@@ -172,7 +176,55 @@
         return null;
     }
 
+    @Override
+    protected Enumeration<URL> findResources(String res)
+        throws IOException
+    {
+        List<URL> resources = new ArrayList<URL>();
+        
+        PySystemState sys = Py.getSystemState();
+        
+        if (res.charAt(0) == SLASH_CHAR) {
+            res = res.substring(1);
+        }
+        String entryRes = res;
+        if (File.separatorChar != SLASH_CHAR) {
+            res = res.replace(SLASH_CHAR, File.separatorChar);
+            entryRes = entryRes.replace(File.separatorChar, SLASH_CHAR);
+        }
 
+        PyList path = sys.path;
+        for (int i = 0; i < path.__len__(); i++) {
+            PyObject entry = replacePathItem(sys, i, path);
+            if (entry instanceof SyspathArchive) {
+                SyspathArchive archive = (SyspathArchive) entry;
+                ZipEntry ze = archive.getEntry(entryRes);
+                if (ze != null) {
+                    try {
+                        resources.add(new URL("jar:file:" + archive.asUriCompatibleString() + "!/" + entryRes));
+                    } catch (MalformedURLException e) {
+                        throw new RuntimeException(e);
+                    }
+                }
+                continue;
+            }
+            if (!(entry instanceof PyUnicode)) {
+                entry = entry.__str__();
+            }
+            String dir = sys.getPath(entry.toString());
+            try {
+                File resource = new File(dir, res);
+                if (!resource.exists()) {
+                    continue;
+                }
+                resources.add(resource.toURI().toURL());
+            } catch (MalformedURLException e) {
+                throw new RuntimeException(e);
+            }
+        }
+        return Collections.enumeration(resources);
+    }
+    
     static PyObject replacePathItem(PySystemState sys, int idx, PyList paths) {
         PyObject path = paths.__getitem__(idx);
         if (path instanceof SyspathArchive) {

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Tue May 13 18:32:20 2014
From: jython-checkins at python.org (frank.wierzbicki)
Date: Tue, 13 May 2014 18:32:20 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython_=28merge_default_-=3E_default?=
	=?utf-8?b?KTogTWVyZ2Uu?=
Message-ID: <3gSl0h0qQgz7LkV@mail.python.org>

http://hg.python.org/jython/rev/2c30e000fbb4
changeset:   7246:2c30e000fbb4
parent:      7245:ab99e96d9192
parent:      7244:a5b000944a51
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Tue May 13 16:32:13 2014 +0000
summary:
  Merge.

files:
  ACKNOWLEDGMENTS                  |  2 ++
  Lib/test/test_math_jy.py         |  9 +++++++++
  src/org/python/modules/math.java |  4 ++--
  3 files changed, 13 insertions(+), 2 deletions(-)


diff --git a/ACKNOWLEDGMENTS b/ACKNOWLEDGMENTS
--- a/ACKNOWLEDGMENTS
+++ b/ACKNOWLEDGMENTS
@@ -105,6 +105,8 @@
     Santoso Wijaya
     Brandon Pedersen
     Chris Simpson
+    Indra Talip
+    Michael B?sch
     Richard Eckart de Castilho
 
 Local Variables:
diff --git a/Lib/test/test_math_jy.py b/Lib/test/test_math_jy.py
--- a/Lib/test/test_math_jy.py
+++ b/Lib/test/test_math_jy.py
@@ -28,6 +28,15 @@
         self.assertEqual(inf, math.hypot(ninf, 4))
         self.assertEqual(inf, math.hypot(4, ninf))
 
+    def test_math_domain_error(self):
+        self.assertRaises(ValueError, math.sqrt, -1)
+        self.assertRaises(ValueError, math.sqrt, -1.5)
+        self.assertRaises(ValueError, math.sqrt, -0.5)
+        self.assertRaises(ValueError, math.log, 0)
+        self.assertRaises(ValueError, math.log, -1)
+        self.assertRaises(ValueError, math.log, -1.5)
+        self.assertRaises(ValueError, math.log, -0.5)
+
 
 def test_main():
     test_support.run_unittest(MathTestCase)
diff --git a/src/org/python/modules/math.java b/src/org/python/modules/math.java
--- a/src/org/python/modules/math.java
+++ b/src/org/python/modules/math.java
@@ -318,7 +318,7 @@
         if (ispinf(v)) {
             return v;
         }
-        if (isninf(v) || v == MINUS_ONE) {
+        if (isninf(v) || v < MINUS_ZERO) {
             throwMathDomainValueError();
         }
         return Math.sqrt(v);
@@ -545,7 +545,7 @@
     }
 
     private static double log(double v) {
-        if (isninf(v)) {
+        if (isninf(v) || v <= ZERO) {
             throwMathDomainValueError();
         }
         if (isinf(v) || isnan(v)) {

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Thu May 15 06:19:55 2014
From: jython-checkins at python.org (jim.baker)
Date: Thu, 15 May 2014 06:19:55 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Fix_issue_=231982=3A_builti?=
 =?utf-8?q?n_function_enumerate=28=29_should_support_an_optional?=
Message-ID: <3gTffg19vjz7Lkh@mail.python.org>

http://hg.python.org/jython/rev/4aaec46e3ee5
changeset:   7247:4aaec46e3ee5
user:        Santoso Wijaya <santoso.wijaya at gmail.com>
date:        Tue May 13 21:03:06 2014 -0700
summary:
  Fix issue #1982: builtin function enumerate() should support an optional 'start' argument.

files:
  Lib/test/test_enumerate.py                  |   3 +-
  Lib/test/test_enumerate_jy.py               |  43 ++++++++++
  src/org/python/core/PyEnumerate.java        |  19 ++-
  src/org/python/core/PyEnumerateDerived.java |   4 +-
  4 files changed, 59 insertions(+), 10 deletions(-)


diff --git a/Lib/test/test_enumerate.py b/Lib/test/test_enumerate.py
--- a/Lib/test/test_enumerate.py
+++ b/Lib/test/test_enumerate.py
@@ -100,7 +100,8 @@
     def test_argumentcheck(self):
         self.assertRaises(TypeError, self.enum) # no arguments
         self.assertRaises(TypeError, self.enum, 1) # wrong type (not iterable)
-        self.assertRaises(TypeError, self.enum, 'abc', 2) # too many arguments
+        self.assertRaises(TypeError, self.enum, 'abc', 'a') # wrong type
+        self.assertRaises(TypeError, self.enum, 'abc', 2, 3) # too many arguments
 
     def test_tuple_reuse(self):
         # Tests an implementation detail where tuple is reused
diff --git a/Lib/test/test_enumerate_jy.py b/Lib/test/test_enumerate_jy.py
new file mode 100644
--- /dev/null
+++ b/Lib/test/test_enumerate_jy.py
@@ -0,0 +1,43 @@
+import unittest
+import sys
+
+from test import test_support
+
+
+class EnumerateJyTestCase(unittest.TestCase):
+
+    enum = enumerate
+    seq, start, res = 'abc', 5, [(5, 'a'), (6, 'b'), (7, 'c')]
+
+    def test_start_kwarg_1(self):
+        e = self.enum(self.seq, start=self.start)
+        self.assertEqual(iter(e), e)
+        self.assertEqual(list(e), self.res)
+
+    def test_start_kwarg_2(self):
+        e = self.enum(start=self.start, sequence=self.seq)
+        self.assertEqual(iter(e), e)
+        self.assertEqual(list(e), self.res)
+
+    def test_start_pos(self):
+        e = self.enum(self.seq, self.start)
+        self.assertEqual(iter(e), e)
+        self.assertEqual(list(e), self.res)
+
+
+def test_main(verbose=None):
+    testclasses = (EnumerateJyTestCase,)
+    test_support.run_unittest(*testclasses)
+
+    # verify reference counting
+    import sys
+    if verbose and hasattr(sys, "gettotalrefcount"):
+        counts = [None] * 5
+        for i in xrange(len(counts)):
+            test_support.run_unittest(*testclasses)
+            counts[i] = sys.gettotalrefcount()
+        print counts
+
+if __name__ == "__main__":
+    test_main(verbose=True)
+
diff --git a/src/org/python/core/PyEnumerate.java b/src/org/python/core/PyEnumerate.java
--- a/src/org/python/core/PyEnumerate.java
+++ b/src/org/python/core/PyEnumerate.java
@@ -23,14 +23,14 @@
         super(subType);
     }
 
-    public PyEnumerate(PyType subType, PyObject seq) {
+    public PyEnumerate(PyType subType, PyObject seq, long start) {
         super(subType);
-        index = 0;
+        index = start;
         sit = seq.__iter__();
     }
 
-    public PyEnumerate(PyObject seq) {
-        this(TYPE, seq);
+    public PyEnumerate(PyObject seq, long start) {
+        this(TYPE, seq, start);
     }
 
     public PyObject next() {
@@ -50,14 +50,19 @@
     @ExposedNew
     public final static PyObject enumerate_new(PyNewWrapper new_, boolean init, PyType subtype,
                                                PyObject[] args, String[] keywords) {
-        if (args.length != 1) {
+        if (args.length > 2 || args.length <= 0) {
             throw PyBuiltinCallable.DefaultInfo.unexpectedCall(args.length, false, "enumerate", 0,
                                                                1);
         }
+
+        ArgParser ap = new ArgParser("enumerate", args, keywords, new String[] {"sequence", "start"});
+        PyObject seq = ap.getPyObject(0);
+        long start = (long) ap.getInt(1, 0);
+
         if (new_.for_type == subtype) {
-            return new PyEnumerate(args[0]);
+            return new PyEnumerate(seq, start);
         } else {
-            return new PyEnumerateDerived(subtype, args[0]);
+            return new PyEnumerateDerived(subtype, seq, start);
         }
     }
 
diff --git a/src/org/python/core/PyEnumerateDerived.java b/src/org/python/core/PyEnumerateDerived.java
--- a/src/org/python/core/PyEnumerateDerived.java
+++ b/src/org/python/core/PyEnumerateDerived.java
@@ -38,8 +38,8 @@
         dict=new PyStringMap();
     }
 
-    public PyEnumerateDerived(PyType subtype,PyObject seq) {
-        super(subtype,seq);
+    public PyEnumerateDerived(PyType subtype,PyObject seq,long start) {
+        super(subtype,seq,start);
         slots=new PyObject[subtype.getNumSlots()];
         dict=subtype.instDict();
     }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Thu May 15 06:19:56 2014
From: jython-checkins at python.org (jim.baker)
Date: Thu, 15 May 2014 06:19:56 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Cleaned_up_test=5Fenumerate?=
 =?utf-8?q?=5Fjy_to_remove_dead_code_for_Jython?=
Message-ID: <3gTffh2XGGz7Lkv@mail.python.org>

http://hg.python.org/jython/rev/f6f7196d27a5
changeset:   7248:f6f7196d27a5
user:        Jim Baker <jim.baker at rackspace.com>
date:        Wed May 14 22:00:33 2014 -0600
summary:
  Cleaned up test_enumerate_jy to remove dead code for Jython

files:
  Lib/test/test_enumerate_jy.py |  8 --------
  1 files changed, 0 insertions(+), 8 deletions(-)


diff --git a/Lib/test/test_enumerate_jy.py b/Lib/test/test_enumerate_jy.py
--- a/Lib/test/test_enumerate_jy.py
+++ b/Lib/test/test_enumerate_jy.py
@@ -29,14 +29,6 @@
     testclasses = (EnumerateJyTestCase,)
     test_support.run_unittest(*testclasses)
 
-    # verify reference counting
-    import sys
-    if verbose and hasattr(sys, "gettotalrefcount"):
-        counts = [None] * 5
-        for i in xrange(len(counts)):
-            test_support.run_unittest(*testclasses)
-            counts[i] = sys.gettotalrefcount()
-        print counts
 
 if __name__ == "__main__":
     test_main(verbose=True)

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Fri May 16 00:18:54 2014
From: jython-checkins at python.org (frank.wierzbicki)
Date: Fri, 16 May 2014 00:18:54 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Support_for_indexer_code_no?=
 =?utf-8?q?w_part_of_code_generation=2E?=
Message-ID: <3gV6bf6ttVz7LjP@mail.python.org>

http://hg.python.org/jython/rev/507fbaad0fcc
changeset:   7249:507fbaad0fcc
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Thu May 15 22:17:18 2014 +0000
summary:
  Support for indexer code now part of code generation.

files:
  ast/asdl_antlr.py                         |   4 +-
  grammar/Python.g                          |   4 +-
  src/org/python/antlr/ast/Attribute.java   |  45 +++---
  src/org/python/antlr/ast/ClassDef.java    |  65 ++++----
  src/org/python/antlr/ast/FunctionDef.java |  53 +++---
  src/org/python/antlr/ast/Global.java      |  22 +-
  src/org/python/antlr/ast/ImportFrom.java  |  54 ++++--
  src/org/python/antlr/ast/Name.java        |   3 +-
  src/org/python/antlr/ast/alias.java       |  47 +++---
  src/org/python/antlr/ast/arguments.java   |  78 +++++-----
  10 files changed, 197 insertions(+), 178 deletions(-)


diff --git a/ast/asdl_antlr.py b/ast/asdl_antlr.py
--- a/ast/asdl_antlr.py
+++ b/ast/asdl_antlr.py
@@ -846,8 +846,8 @@
     // XXX: vararg and kwarg are deliberately moved to the end of the
     // method signature to avoid clashes with the (Token, List<expr>,
     // String, String, List<expr>) version of the constructor.
-    public arguments(Token token, java.util.List<expr> args,
-            java.util.List<expr> defaults, Name vararg, Name kwarg) {
+    public arguments(Token token, java.util.List<expr> args, Name vararg, Name kwarg,
+            java.util.List<expr> defaults) {
         super(token);
         this.args = args;
         if (args == null) {
diff --git a/grammar/Python.g b/grammar/Python.g
--- a/grammar/Python.g
+++ b/grammar/Python.g
@@ -511,7 +511,7 @@
         }
       |
         {
-            $args = new arguments($parameters.start, new ArrayList<expr>(), null, null, new ArrayList<expr>());
+            $args = new arguments($parameters.start, new ArrayList<expr>(), (Name)null, null, new ArrayList<expr>());
         }
       )
       RPAREN
@@ -1851,7 +1851,7 @@
       {
           arguments a = $varargslist.args;
           if (a == null) {
-              a = new arguments($LAMBDA, new ArrayList<expr>(), null, null, new ArrayList<expr>());
+              a = new arguments($LAMBDA, new ArrayList<expr>(), (Name)null, null, new ArrayList<expr>());
           }
           etype = new Lambda($LAMBDA, a, actions.castExpr($test.tree));
       }
diff --git a/src/org/python/antlr/ast/Attribute.java b/src/org/python/antlr/ast/Attribute.java
--- a/src/org/python/antlr/ast/Attribute.java
+++ b/src/org/python/antlr/ast/Attribute.java
@@ -45,10 +45,6 @@
     public String getInternalAttr() {
         return attr;
     }
-    private Name attrName;
-    public Name getInternalAttrName() {
-        return attrName;
-    }
     @ExposedGet(name = "attr")
     public PyObject getAttr() {
         if (attr == null) return Py.None;
@@ -123,24 +119,6 @@
         this.ctx = ctx;
     }
 
-    public Attribute(Token token, expr value, Name attr, expr_contextType ctx) {
-        super(token);
-        this.value = value;
-        addChild(value);
-        this.attr = attr.getText();
-        this.attrName = attr;
-        this.ctx = ctx;
-    }
-
-    public Attribute(Integer ttype, Token token, expr value, Name attr, expr_contextType ctx) {
-        super(ttype, token);
-        this.value = value;
-        addChild(value);
-        this.attr = attr.getText();
-        this.attrName = attr;
-        this.ctx = ctx;
-    }
-
     public Attribute(Integer ttype, Token token, expr value, String attr, expr_contextType ctx) {
         super(ttype, token);
         this.value = value;
@@ -218,4 +196,27 @@
         col_offset = num;
     }
 
+    // Support for indexer below
+
+    private Name attrName;
+    public Name getInternalAttrName() {
+        return attrName;
+    }
+    public Attribute(Token token, expr value, Name attr, expr_contextType ctx) {
+        super(token);
+        this.value = value;
+        addChild(value);
+        this.attr = attr.getText();
+        this.attrName = attr;
+        this.ctx = ctx;
+    }
+
+    public Attribute(Integer ttype, Token token, expr value, Name attr, expr_contextType ctx) {
+        super(ttype, token);
+        this.value = value;
+        addChild(value);
+        this.attr = attr.getText();
+        this.attrName = attr;
+        this.ctx = ctx;
+    }
 }
diff --git a/src/org/python/antlr/ast/ClassDef.java b/src/org/python/antlr/ast/ClassDef.java
--- a/src/org/python/antlr/ast/ClassDef.java
+++ b/src/org/python/antlr/ast/ClassDef.java
@@ -32,10 +32,6 @@
     public String getInternalName() {
         return name;
     }
-    private Name nameNode;
-    public Name getInternalNameNode() {
-        return nameNode;
-    }
     @ExposedGet(name = "name")
     public PyObject getName() {
         if (name == null) return Py.None;
@@ -158,34 +154,6 @@
         }
     }
 
-    public ClassDef(Token token, Name name, java.util.List<expr> bases, java.util.List<stmt>
-    body, java.util.List<expr> decorator_list) {
-        super(token);
-        this.name = name.getText();
-        this.nameNode = name;
-        this.bases = bases;
-        if (bases == null) {
-            this.bases = new ArrayList<expr>();
-        }
-        for(PythonTree t : this.bases) {
-            addChild(t);
-        }
-        this.body = body;
-        if (body == null) {
-            this.body = new ArrayList<stmt>();
-        }
-        for(PythonTree t : this.body) {
-            addChild(t);
-        }
-        this.decorator_list = decorator_list;
-        if (decorator_list == null) {
-            this.decorator_list = new ArrayList<expr>();
-        }
-        for(PythonTree t : this.decorator_list) {
-            addChild(t);
-        }
-    }
-
     public ClassDef(Integer ttype, Token token, String name, java.util.List<expr> bases,
     java.util.List<stmt> body, java.util.List<expr> decorator_list) {
         super(ttype, token);
@@ -316,4 +284,37 @@
         col_offset = num;
     }
 
+    // Support for indexer below
+
+    private Name nameNode;
+    public Name getInternalNameNode() {
+        return nameNode;
+    }
+    public ClassDef(Token token, Name name, java.util.List<expr> bases, java.util.List<stmt>
+    body, java.util.List<expr> decorator_list) {
+        super(token);
+        this.name = name.getText();
+        this.nameNode = name;
+        this.bases = bases;
+        if (bases == null) {
+            this.bases = new ArrayList<expr>();
+        }
+        for(PythonTree t : this.bases) {
+            addChild(t);
+        }
+        this.body = body;
+        if (body == null) {
+            this.body = new ArrayList<stmt>();
+        }
+        for(PythonTree t : this.body) {
+            addChild(t);
+        }
+        this.decorator_list = decorator_list;
+        if (decorator_list == null) {
+            this.decorator_list = new ArrayList<expr>();
+        }
+        for(PythonTree t : this.decorator_list) {
+            addChild(t);
+        }
+    }
 }
diff --git a/src/org/python/antlr/ast/FunctionDef.java b/src/org/python/antlr/ast/FunctionDef.java
--- a/src/org/python/antlr/ast/FunctionDef.java
+++ b/src/org/python/antlr/ast/FunctionDef.java
@@ -32,10 +32,6 @@
     public String getInternalName() {
         return name;
     }
-    private Name nameNode;
-    public Name getInternalNameNode() {
-        return nameNode;
-    }
     @ExposedGet(name = "name")
     public PyObject getName() {
         if (name == null) return Py.None;
@@ -152,28 +148,6 @@
         }
     }
 
-    public FunctionDef(Token token, Name name, arguments args, java.util.List<stmt> body,
-            java.util.List<expr> decorator_list) {
-        super(token);
-        this.name = name.getText();
-        this.nameNode = name;
-        this.args = args;
-        this.body = body;
-        if (body == null) {
-            this.body = new ArrayList<stmt>();
-        }
-        for(PythonTree t : this.body) {
-            addChild(t);
-        }
-        this.decorator_list = decorator_list;
-        if (decorator_list == null) {
-            this.decorator_list = new ArrayList<expr>();
-        }
-        for(PythonTree t : this.decorator_list) {
-            addChild(t);
-        }
-    }
-
     public FunctionDef(Integer ttype, Token token, String name, arguments args,
     java.util.List<stmt> body, java.util.List<expr> decorator_list) {
         super(ttype, token);
@@ -288,4 +262,31 @@
         col_offset = num;
     }
 
+    // Support for indexer below
+
+    private Name nameNode;
+    public Name getInternalNameNode() {
+        return nameNode;
+    }
+    public FunctionDef(Token token, Name name, arguments args, java.util.List<stmt> body,
+            java.util.List<expr> decorator_list) {
+        super(token);
+        this.name = name.getText();
+        this.nameNode = name;
+        this.args = args;
+        this.body = body;
+        if (body == null) {
+            this.body = new ArrayList<stmt>();
+        }
+        for(PythonTree t : this.body) {
+            addChild(t);
+        }
+        this.decorator_list = decorator_list;
+        if (decorator_list == null) {
+            this.decorator_list = new ArrayList<expr>();
+        }
+        for(PythonTree t : this.decorator_list) {
+            addChild(t);
+        }
+    }
 }
diff --git a/src/org/python/antlr/ast/Global.java b/src/org/python/antlr/ast/Global.java
--- a/src/org/python/antlr/ast/Global.java
+++ b/src/org/python/antlr/ast/Global.java
@@ -40,10 +40,7 @@
     public void setNames(PyObject names) {
         this.names = AstAdapters.py2identifierList(names);
     }
-    private java.util.List<Name> nameNodes;
-    public java.util.List<Name> getInternalNameNodes() {
-        return nameNodes;
-    }
+
 
     private final static PyString[] fields =
     new PyString[] {new PyString("names")};
@@ -88,12 +85,6 @@
         this.names = names;
     }
 
-    public Global(Token token, java.util.List<String> names, java.util.List<Name> nameNodes) {
-        super(token);
-        this.names = names;
-        this.nameNodes = nameNodes;
-    }
-
     public Global(Integer ttype, Token token, java.util.List<String> names) {
         super(ttype, token);
         this.names = names;
@@ -153,4 +144,15 @@
         col_offset = num;
     }
 
+    // Support for indexer below
+
+    private java.util.List<Name> nameNodes;
+    public java.util.List<Name> getInternalNameNodes() {
+        return nameNodes;
+    }
+    public Global(Token token, java.util.List<String> names, java.util.List<Name> nameNodes) {
+        super(token);
+        this.names = names;
+        this.nameNodes = nameNodes;
+    }
 }
diff --git a/src/org/python/antlr/ast/ImportFrom.java b/src/org/python/antlr/ast/ImportFrom.java
--- a/src/org/python/antlr/ast/ImportFrom.java
+++ b/src/org/python/antlr/ast/ImportFrom.java
@@ -1,4 +1,4 @@
-// Autogenerated AST node   -*- c-basic-offset:4 -*-
+// Autogenerated AST node
 package org.python.antlr.ast;
 import org.antlr.runtime.CommonToken;
 import org.antlr.runtime.Token;
@@ -24,7 +24,6 @@
 import java.io.DataOutputStream;
 import java.io.IOException;
 import java.util.ArrayList;
-import java.util.List;
 
 @ExposedType(name = "_ast.ImportFrom", base = AST.class)
 public class ImportFrom extends stmt {
@@ -43,8 +42,8 @@
         this.module = AstAdapters.py2identifier(module);
     }
 
-    private List<alias> names;
-    public List<alias> getInternalNames() {
+    private java.util.List<alias> names;
+    public java.util.List<alias> getInternalNames() {
         return names;
     }
     @ExposedGet(name = "names")
@@ -56,11 +55,6 @@
         this.names = AstAdapters.py2aliasList(names);
     }
 
-    private List<Name> moduleNames;
-    public List<Name> getInternalModuleNames() {
-        return moduleNames;
-    }
-
     private Integer level;
     public Integer getInternalLevel() {
         return level;
@@ -117,9 +111,7 @@
         setLevel(level);
     }
 
-    public ImportFrom(Token token,
-                      String module, List<Name> moduleNames,
-                      List<alias> names, Integer level) {
+    public ImportFrom(Token token, String module, java.util.List<alias> names, Integer level) {
         super(token);
         this.module = module;
         this.names = names;
@@ -129,17 +121,10 @@
         for(PythonTree t : this.names) {
             addChild(t);
         }
-        this.moduleNames = moduleNames;
-        if (moduleNames == null) {
-            this.moduleNames = new ArrayList<Name>();
-        }
-        for(PythonTree t : this.moduleNames) {
-            addChild(t);
-        }
         this.level = level;
     }
 
-    public ImportFrom(Integer ttype, Token token, String module, List<alias> names,
+    public ImportFrom(Integer ttype, Token token, String module, java.util.List<alias> names,
     Integer level) {
         super(ttype, token);
         this.module = module;
@@ -153,7 +138,7 @@
         this.level = level;
     }
 
-    public ImportFrom(PythonTree tree, String module, List<alias> names, Integer level) {
+    public ImportFrom(PythonTree tree, String module, java.util.List<alias> names, Integer level) {
         super(tree);
         this.module = module;
         this.names = names;
@@ -227,4 +212,31 @@
         col_offset = num;
     }
 
+    // Support for indexer below
+
+    private java.util.List<Name> moduleNames;
+    public java.util.List<Name> getInternalModuleNames() {
+        return moduleNames;
+    }
+    public ImportFrom(Token token,
+                      String module, java.util.List<Name> moduleNames,
+                      java.util.List<alias> names, Integer level) {
+        super(token);
+        this.module = module;
+        this.names = names;
+        if (names == null) {
+            this.names = new ArrayList<alias>();
+        }
+        for(PythonTree t : this.names) {
+            addChild(t);
+        }
+        this.moduleNames = moduleNames;
+        if (moduleNames == null) {
+            this.moduleNames = new ArrayList<Name>();
+        }
+        for(PythonTree t : this.moduleNames) {
+            addChild(t);
+        }
+        this.level = level;
+    }
 }
diff --git a/src/org/python/antlr/ast/Name.java b/src/org/python/antlr/ast/Name.java
--- a/src/org/python/antlr/ast/Name.java
+++ b/src/org/python/antlr/ast/Name.java
@@ -1,4 +1,4 @@
-// Autogenerated AST node  -*- c-basic-offset:4 -*-
+// Autogenerated AST node
 package org.python.antlr.ast;
 import org.antlr.runtime.CommonToken;
 import org.antlr.runtime.Token;
@@ -55,6 +55,7 @@
         this.ctx = AstAdapters.py2expr_context(ctx);
     }
 
+
     private final static PyString[] fields =
     new PyString[] {new PyString("id"), new PyString("ctx")};
     @ExposedGet(name = "_fields")
diff --git a/src/org/python/antlr/ast/alias.java b/src/org/python/antlr/ast/alias.java
--- a/src/org/python/antlr/ast/alias.java
+++ b/src/org/python/antlr/ast/alias.java
@@ -42,10 +42,6 @@
         this.name = AstAdapters.py2identifier(name);
     }
 
-    private java.util.List<Name> nameNodes;
-    public java.util.List<Name> getInternalNameNodes() {
-        return nameNodes;
-    }
     private String asname;
     public String getInternalAsname() {
         return asname;
@@ -60,10 +56,6 @@
         this.asname = AstAdapters.py2identifier(asname);
     }
 
-    private Name asnameNode;
-    public Name getInternalAsnameNode() {
-        return asnameNode;
-    }
 
     private final static PyString[] fields =
     new PyString[] {new PyString("name"), new PyString("asname")};
@@ -100,21 +92,6 @@
         this.asname = asname;
     }
 
-    // [import] name [as asname]
-    public alias(Name name, Name asname) {
-        this(java.util.Arrays.asList(new Name[]{name}), asname);
-    }
-
-    // [import] ...foo.bar.baz [as asname]
-    public alias(java.util.List<Name> nameNodes, Name asname) {
-        this.nameNodes = nameNodes;
-        this.name = dottedNameListToString(nameNodes);
-        if (asname != null) {
-            this.asnameNode = asname;
-            this.asname = asname.getInternalId();
-        }
-    }
-
     public alias(Integer ttype, Token token, String name, String asname) {
         super(ttype, token);
         this.name = name;
@@ -152,4 +129,28 @@
     public void traverse(VisitorIF<?> visitor) throws Exception {
     }
 
+    // Support for indexer below
+
+    private java.util.List<Name> nameNodes;
+    public java.util.List<Name> getInternalNameNodes() {
+        return nameNodes;
+    }
+    private Name asnameNode;
+    public Name getInternalAsnameNode() {
+        return asnameNode;
+    }
+    // [import] name [as asname]
+    public alias(Name name, Name asname) {
+        this(java.util.Arrays.asList(new Name[]{name}), asname);
+    }
+
+    // [import] ...foo.bar.baz [as asname]
+    public alias(java.util.List<Name> nameNodes, Name asname) {
+        this.nameNodes = nameNodes;
+        this.name = dottedNameListToString(nameNodes);
+        if (asname != null) {
+            this.asnameNode = asname;
+            this.asname = asname.getInternalId();
+        }
+    }
 }
diff --git a/src/org/python/antlr/ast/arguments.java b/src/org/python/antlr/ast/arguments.java
--- a/src/org/python/antlr/ast/arguments.java
+++ b/src/org/python/antlr/ast/arguments.java
@@ -45,12 +45,6 @@
     public String getInternalVararg() {
         return vararg;
     }
-
-    private Name varargName;
-    public Name getInternalVarargName() {
-        return varargName;
-    }
-
     @ExposedGet(name = "vararg")
     public PyObject getVararg() {
         if (vararg == null) return Py.None;
@@ -65,12 +59,6 @@
     public String getInternalKwarg() {
         return kwarg;
     }
-
-    private Name kwargName;
-    public Name getInternalKwargName() {
-        return kwargName;
-    }
-
     @ExposedGet(name = "kwarg")
     public PyObject getKwarg() {
         if (kwarg == null) return Py.None;
@@ -129,29 +117,8 @@
         setDefaults(defaults);
     }
 
-    // public arguments(Token token, java.util.List<expr> args, String vararg, String kwarg,
-    // java.util.List<expr> defaults) {
-    //     super(token);
-    //     this.args = args;
-    //     if (args == null) {
-    //         this.args = new ArrayList<expr>();
-    //     }
-    //     for(PythonTree t : this.args) {
-    //         addChild(t);
-    //     }
-    //     this.vararg = vararg;
-    //     this.kwarg = kwarg;
-    //     this.defaults = defaults;
-    //     if (defaults == null) {
-    //         this.defaults = new ArrayList<expr>();
-    //     }
-    //     for(PythonTree t : this.defaults) {
-    //         addChild(t);
-    //     }
-    // }
-
-    public arguments(Token token, java.util.List<expr> args, Name vararg, Name kwarg,
-            java.util.List<expr> defaults) {
+    public arguments(Token token, java.util.List<expr> args, String vararg, String kwarg,
+    java.util.List<expr> defaults) {
         super(token);
         this.args = args;
         if (args == null) {
@@ -160,10 +127,8 @@
         for(PythonTree t : this.args) {
             addChild(t);
         }
-        this.vararg = vararg == null ? null : vararg.getText();
-        this.varargName = vararg;
-        this.kwarg = kwarg == null ? null : kwarg.getText();
-        this.kwargName = kwarg;
+        this.vararg = vararg;
+        this.kwarg = kwarg;
         this.defaults = defaults;
         if (defaults == null) {
             this.defaults = new ArrayList<expr>();
@@ -258,4 +223,39 @@
         }
     }
 
+    // Support for indexer below
+
+    private Name varargName;
+    public Name getInternalVarargName() {
+        return varargName;
+    }
+    private Name kwargName;
+    public Name getInternalKwargName() {
+        return kwargName;
+    }
+    // XXX: vararg and kwarg are deliberately moved to the end of the
+    // method signature to avoid clashes with the (Token, List<expr>,
+    // String, String, List<expr>) version of the constructor.
+    public arguments(Token token, java.util.List<expr> args, Name vararg, Name kwarg,
+            java.util.List<expr> defaults) {
+        super(token);
+        this.args = args;
+        if (args == null) {
+            this.args = new ArrayList<expr>();
+        }
+        for(PythonTree t : this.args) {
+            addChild(t);
+        }
+        this.vararg = vararg == null ? null : vararg.getText();
+        this.varargName = vararg;
+        this.kwarg = kwarg == null ? null : kwarg.getText();
+        this.kwargName = kwarg;
+        this.defaults = defaults;
+        if (defaults == null) {
+            this.defaults = new ArrayList<expr>();
+        }
+        for(PythonTree t : this.defaults) {
+            addChild(t);
+        }
+    }
 }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Fri May 16 00:18:56 2014
From: jython-checkins at python.org (frank.wierzbicki)
Date: Fri, 16 May 2014 00:18:56 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Remove_print_statement=2E?=
Message-ID: <3gV6bh19Dgz7LjP@mail.python.org>

http://hg.python.org/jython/rev/524b03a50db5
changeset:   7250:524b03a50db5
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Thu May 15 22:18:34 2014 +0000
summary:
  Remove print statement.

files:
  ast/asdl_antlr.py |  1 -
  1 files changed, 0 insertions(+), 1 deletions(-)


diff --git a/ast/asdl_antlr.py b/ast/asdl_antlr.py
--- a/ast/asdl_antlr.py
+++ b/ast/asdl_antlr.py
@@ -616,7 +616,6 @@
         return jtype
     
     def indexerSupport(self, name, depth):
-        print "emitting indexer support for", name
 	self.emit("// Support for indexer below", depth + 1)
         self.file.write(indexer_support[name])
 

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Fri May 16 02:07:31 2014
From: jython-checkins at python.org (frank.wierzbicki)
Date: Fri, 16 May 2014 02:07:31 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_=232108_Added_dicts_to_all_?=
	=?utf-8?q?AST_classes=2E?=
Message-ID: <3gV90z5FyFz7Ljg@mail.python.org>

http://hg.python.org/jython/rev/ecb9475b7808
changeset:   7251:ecb9475b7808
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Fri May 16 00:07:24 2014 +0000
summary:
  #2108 Added dicts to all AST classes.

files:
  NEWS                                         |   2 +-
  ast/asdl_antlr.py                            |  24 ++++++++++
  src/org/python/antlr/ast/Assert.java         |  20 ++++++++
  src/org/python/antlr/ast/Assign.java         |  20 ++++++++
  src/org/python/antlr/ast/Attribute.java      |  20 ++++++++
  src/org/python/antlr/ast/AugAssign.java      |  20 ++++++++
  src/org/python/antlr/ast/BinOp.java          |  20 ++++++++
  src/org/python/antlr/ast/BoolOp.java         |  20 ++++++++
  src/org/python/antlr/ast/Break.java          |  20 ++++++++
  src/org/python/antlr/ast/Call.java           |  20 ++++++++
  src/org/python/antlr/ast/ClassDef.java       |  20 ++++++++
  src/org/python/antlr/ast/Compare.java        |  20 ++++++++
  src/org/python/antlr/ast/Continue.java       |  20 ++++++++
  src/org/python/antlr/ast/Delete.java         |  20 ++++++++
  src/org/python/antlr/ast/Dict.java           |  20 ++++++++
  src/org/python/antlr/ast/DictComp.java       |  20 ++++++++
  src/org/python/antlr/ast/Ellipsis.java       |  20 ++++++++
  src/org/python/antlr/ast/ExceptHandler.java  |  20 ++++++++
  src/org/python/antlr/ast/Exec.java           |  20 ++++++++
  src/org/python/antlr/ast/Expr.java           |  20 ++++++++
  src/org/python/antlr/ast/Expression.java     |  20 ++++++++
  src/org/python/antlr/ast/ExtSlice.java       |  20 ++++++++
  src/org/python/antlr/ast/For.java            |  20 ++++++++
  src/org/python/antlr/ast/FunctionDef.java    |  20 ++++++++
  src/org/python/antlr/ast/GeneratorExp.java   |  20 ++++++++
  src/org/python/antlr/ast/Global.java         |  20 ++++++++
  src/org/python/antlr/ast/If.java             |  20 ++++++++
  src/org/python/antlr/ast/IfExp.java          |  20 ++++++++
  src/org/python/antlr/ast/Import.java         |  20 ++++++++
  src/org/python/antlr/ast/ImportFrom.java     |  20 ++++++++
  src/org/python/antlr/ast/Index.java          |  20 ++++++++
  src/org/python/antlr/ast/Interactive.java    |  20 ++++++++
  src/org/python/antlr/ast/Lambda.java         |  20 ++++++++
  src/org/python/antlr/ast/List.java           |  20 ++++++++
  src/org/python/antlr/ast/ListComp.java       |  20 ++++++++
  src/org/python/antlr/ast/Module.java         |  20 ++++++++
  src/org/python/antlr/ast/Name.java           |  20 ++++++++
  src/org/python/antlr/ast/Num.java            |  20 ++++++++
  src/org/python/antlr/ast/Pass.java           |  20 ++++++++
  src/org/python/antlr/ast/Print.java          |  20 ++++++++
  src/org/python/antlr/ast/Raise.java          |  20 ++++++++
  src/org/python/antlr/ast/Repr.java           |  20 ++++++++
  src/org/python/antlr/ast/Return.java         |  20 ++++++++
  src/org/python/antlr/ast/Set.java            |  20 ++++++++
  src/org/python/antlr/ast/SetComp.java        |  20 ++++++++
  src/org/python/antlr/ast/Slice.java          |  20 ++++++++
  src/org/python/antlr/ast/Str.java            |  20 ++++++++
  src/org/python/antlr/ast/Subscript.java      |  20 ++++++++
  src/org/python/antlr/ast/Suite.java          |  20 ++++++++
  src/org/python/antlr/ast/TryExcept.java      |  20 ++++++++
  src/org/python/antlr/ast/TryFinally.java     |  20 ++++++++
  src/org/python/antlr/ast/Tuple.java          |  20 ++++++++
  src/org/python/antlr/ast/UnaryOp.java        |  20 ++++++++
  src/org/python/antlr/ast/VisitorBase.java    |   1 +
  src/org/python/antlr/ast/While.java          |  20 ++++++++
  src/org/python/antlr/ast/With.java           |  20 ++++++++
  src/org/python/antlr/ast/Yield.java          |  20 ++++++++
  src/org/python/antlr/ast/alias.java          |  20 ++++++++
  src/org/python/antlr/ast/arguments.java      |  20 ++++++++
  src/org/python/antlr/ast/comprehension.java  |  20 ++++++++
  src/org/python/antlr/ast/keyword.java        |  20 ++++++++
  src/org/python/antlr/base/excepthandler.java |   1 +
  src/org/python/antlr/base/expr.java          |   1 +
  src/org/python/antlr/base/mod.java           |   1 +
  src/org/python/antlr/base/slice.java         |   1 +
  src/org/python/antlr/base/stmt.java          |   1 +
  66 files changed, 1191 insertions(+), 1 deletions(-)


diff --git a/NEWS b/NEWS
--- a/NEWS
+++ b/NEWS
@@ -2,7 +2,7 @@
 
 Jython 2.7b3
   Bugs Fixed
-
+    - [ 2108 ] Cannot set attribute to instances of AST/PythonTree (blocks pyflakes)
   New Features
     - [ 1896215 ] findResource(s) for SyspathJavaLoader
 
diff --git a/ast/asdl_antlr.py b/ast/asdl_antlr.py
--- a/ast/asdl_antlr.py
+++ b/ast/asdl_antlr.py
@@ -73,6 +73,7 @@
             print >> self.file, 'import org.python.core.Py;'
             print >> self.file, 'import org.python.core.PyObject;'
             print >> self.file, 'import org.python.core.PyString;'
+            print >> self.file, 'import org.python.core.PyStringMap;'
             print >> self.file, 'import org.python.core.PyType;'
             print >> self.file, 'import org.python.expose.ExposedGet;'
             print >> self.file, 'import org.python.expose.ExposedMethod;'
@@ -466,6 +467,29 @@
         self.emit('}', depth)
         self.emit("", 0)
 
+        self.emit('public PyObject __dict__;', depth)
+        self.emit("", 0)
+        self.emit('@Override', depth)
+        self.emit('public PyObject fastGetDict() {', depth)
+        self.emit('ensureDict();', depth+1)
+        self.emit('return __dict__;', depth+1)
+        self.emit('}', depth)
+        self.emit("", 0)
+
+        self.emit('@ExposedGet(name = "__dict__")', depth)
+        self.emit('public PyObject getDict() {', depth)
+        self.emit('return fastGetDict();', depth+1)
+        self.emit('}', depth)
+        self.emit("", 0)
+
+        self.emit('private void ensureDict() {', depth)
+        self.emit('if (__dict__ == null) {', depth+1)
+        self.emit('__dict__ = new PyStringMap();', depth+2)
+        self.emit('}', depth+1)
+        self.emit('}', depth)
+        self.emit("", 0)
+
+
     def javaConstructors(self, type, name, clsname, is_product, fields, depth):
         self.emit("public %s(PyType subType) {" % (clsname), depth)
         self.emit("super(subType);", depth + 1)
diff --git a/src/org/python/antlr/ast/Assert.java b/src/org/python/antlr/ast/Assert.java
--- a/src/org/python/antlr/ast/Assert.java
+++ b/src/org/python/antlr/ast/Assert.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -147,6 +148,25 @@
             msg.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Assign.java b/src/org/python/antlr/ast/Assign.java
--- a/src/org/python/antlr/ast/Assign.java
+++ b/src/org/python/antlr/ast/Assign.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -166,6 +167,25 @@
             value.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Attribute.java b/src/org/python/antlr/ast/Attribute.java
--- a/src/org/python/antlr/ast/Attribute.java
+++ b/src/org/python/antlr/ast/Attribute.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -164,6 +165,25 @@
             value.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     public void setContext(expr_contextType c) {
         this.ctx = c;
     }
diff --git a/src/org/python/antlr/ast/AugAssign.java b/src/org/python/antlr/ast/AugAssign.java
--- a/src/org/python/antlr/ast/AugAssign.java
+++ b/src/org/python/antlr/ast/AugAssign.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -168,6 +169,25 @@
             value.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/BinOp.java b/src/org/python/antlr/ast/BinOp.java
--- a/src/org/python/antlr/ast/BinOp.java
+++ b/src/org/python/antlr/ast/BinOp.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -168,6 +169,25 @@
             right.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/BoolOp.java b/src/org/python/antlr/ast/BoolOp.java
--- a/src/org/python/antlr/ast/BoolOp.java
+++ b/src/org/python/antlr/ast/BoolOp.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -161,6 +162,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Break.java b/src/org/python/antlr/ast/Break.java
--- a/src/org/python/antlr/ast/Break.java
+++ b/src/org/python/antlr/ast/Break.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -91,6 +92,25 @@
     public void traverse(VisitorIF<?> visitor) throws Exception {
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Call.java b/src/org/python/antlr/ast/Call.java
--- a/src/org/python/antlr/ast/Call.java
+++ b/src/org/python/antlr/ast/Call.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -268,6 +269,25 @@
             kwargs.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/ClassDef.java b/src/org/python/antlr/ast/ClassDef.java
--- a/src/org/python/antlr/ast/ClassDef.java
+++ b/src/org/python/antlr/ast/ClassDef.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -256,6 +257,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Compare.java b/src/org/python/antlr/ast/Compare.java
--- a/src/org/python/antlr/ast/Compare.java
+++ b/src/org/python/antlr/ast/Compare.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -190,6 +191,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Continue.java b/src/org/python/antlr/ast/Continue.java
--- a/src/org/python/antlr/ast/Continue.java
+++ b/src/org/python/antlr/ast/Continue.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -91,6 +92,25 @@
     public void traverse(VisitorIF<?> visitor) throws Exception {
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Delete.java b/src/org/python/antlr/ast/Delete.java
--- a/src/org/python/antlr/ast/Delete.java
+++ b/src/org/python/antlr/ast/Delete.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -140,6 +141,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Dict.java b/src/org/python/antlr/ast/Dict.java
--- a/src/org/python/antlr/ast/Dict.java
+++ b/src/org/python/antlr/ast/Dict.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -186,6 +187,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/DictComp.java b/src/org/python/antlr/ast/DictComp.java
--- a/src/org/python/antlr/ast/DictComp.java
+++ b/src/org/python/antlr/ast/DictComp.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -194,6 +195,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Ellipsis.java b/src/org/python/antlr/ast/Ellipsis.java
--- a/src/org/python/antlr/ast/Ellipsis.java
+++ b/src/org/python/antlr/ast/Ellipsis.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -80,4 +81,23 @@
     public void traverse(VisitorIF<?> visitor) throws Exception {
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
 }
diff --git a/src/org/python/antlr/ast/ExceptHandler.java b/src/org/python/antlr/ast/ExceptHandler.java
--- a/src/org/python/antlr/ast/ExceptHandler.java
+++ b/src/org/python/antlr/ast/ExceptHandler.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -193,6 +194,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Exec.java b/src/org/python/antlr/ast/Exec.java
--- a/src/org/python/antlr/ast/Exec.java
+++ b/src/org/python/antlr/ast/Exec.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -173,6 +174,25 @@
             locals.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Expr.java b/src/org/python/antlr/ast/Expr.java
--- a/src/org/python/antlr/ast/Expr.java
+++ b/src/org/python/antlr/ast/Expr.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -121,6 +122,25 @@
             value.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Expression.java b/src/org/python/antlr/ast/Expression.java
--- a/src/org/python/antlr/ast/Expression.java
+++ b/src/org/python/antlr/ast/Expression.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -110,4 +111,23 @@
             body.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
 }
diff --git a/src/org/python/antlr/ast/ExtSlice.java b/src/org/python/antlr/ast/ExtSlice.java
--- a/src/org/python/antlr/ast/ExtSlice.java
+++ b/src/org/python/antlr/ast/ExtSlice.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -129,4 +130,23 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
 }
diff --git a/src/org/python/antlr/ast/For.java b/src/org/python/antlr/ast/For.java
--- a/src/org/python/antlr/ast/For.java
+++ b/src/org/python/antlr/ast/For.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -241,6 +242,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/FunctionDef.java b/src/org/python/antlr/ast/FunctionDef.java
--- a/src/org/python/antlr/ast/FunctionDef.java
+++ b/src/org/python/antlr/ast/FunctionDef.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -234,6 +235,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/GeneratorExp.java b/src/org/python/antlr/ast/GeneratorExp.java
--- a/src/org/python/antlr/ast/GeneratorExp.java
+++ b/src/org/python/antlr/ast/GeneratorExp.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -167,6 +168,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Global.java b/src/org/python/antlr/ast/Global.java
--- a/src/org/python/antlr/ast/Global.java
+++ b/src/org/python/antlr/ast/Global.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -116,6 +117,25 @@
     public void traverse(VisitorIF<?> visitor) throws Exception {
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/If.java b/src/org/python/antlr/ast/If.java
--- a/src/org/python/antlr/ast/If.java
+++ b/src/org/python/antlr/ast/If.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -212,6 +213,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/IfExp.java b/src/org/python/antlr/ast/IfExp.java
--- a/src/org/python/antlr/ast/IfExp.java
+++ b/src/org/python/antlr/ast/IfExp.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -173,6 +174,25 @@
             orelse.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Import.java b/src/org/python/antlr/ast/Import.java
--- a/src/org/python/antlr/ast/Import.java
+++ b/src/org/python/antlr/ast/Import.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -140,6 +141,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/ImportFrom.java b/src/org/python/antlr/ast/ImportFrom.java
--- a/src/org/python/antlr/ast/ImportFrom.java
+++ b/src/org/python/antlr/ast/ImportFrom.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -184,6 +185,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Index.java b/src/org/python/antlr/ast/Index.java
--- a/src/org/python/antlr/ast/Index.java
+++ b/src/org/python/antlr/ast/Index.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -110,4 +111,23 @@
             value.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
 }
diff --git a/src/org/python/antlr/ast/Interactive.java b/src/org/python/antlr/ast/Interactive.java
--- a/src/org/python/antlr/ast/Interactive.java
+++ b/src/org/python/antlr/ast/Interactive.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -129,4 +130,23 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
 }
diff --git a/src/org/python/antlr/ast/Lambda.java b/src/org/python/antlr/ast/Lambda.java
--- a/src/org/python/antlr/ast/Lambda.java
+++ b/src/org/python/antlr/ast/Lambda.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -144,6 +145,25 @@
             body.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/List.java b/src/org/python/antlr/ast/List.java
--- a/src/org/python/antlr/ast/List.java
+++ b/src/org/python/antlr/ast/List.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -161,6 +162,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     public void setContext(expr_contextType c) {
         this.ctx = c;
     }
diff --git a/src/org/python/antlr/ast/ListComp.java b/src/org/python/antlr/ast/ListComp.java
--- a/src/org/python/antlr/ast/ListComp.java
+++ b/src/org/python/antlr/ast/ListComp.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -167,6 +168,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Module.java b/src/org/python/antlr/ast/Module.java
--- a/src/org/python/antlr/ast/Module.java
+++ b/src/org/python/antlr/ast/Module.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -129,4 +130,23 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
 }
diff --git a/src/org/python/antlr/ast/Name.java b/src/org/python/antlr/ast/Name.java
--- a/src/org/python/antlr/ast/Name.java
+++ b/src/org/python/antlr/ast/Name.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -138,6 +139,25 @@
     public void traverse(VisitorIF<?> visitor) throws Exception {
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     public void setContext(expr_contextType c) {
         this.ctx = c;
     }
diff --git a/src/org/python/antlr/ast/Num.java b/src/org/python/antlr/ast/Num.java
--- a/src/org/python/antlr/ast/Num.java
+++ b/src/org/python/antlr/ast/Num.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -116,6 +117,25 @@
     public void traverse(VisitorIF<?> visitor) throws Exception {
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Pass.java b/src/org/python/antlr/ast/Pass.java
--- a/src/org/python/antlr/ast/Pass.java
+++ b/src/org/python/antlr/ast/Pass.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -91,6 +92,25 @@
     public void traverse(VisitorIF<?> visitor) throws Exception {
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Print.java b/src/org/python/antlr/ast/Print.java
--- a/src/org/python/antlr/ast/Print.java
+++ b/src/org/python/antlr/ast/Print.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -188,6 +189,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Raise.java b/src/org/python/antlr/ast/Raise.java
--- a/src/org/python/antlr/ast/Raise.java
+++ b/src/org/python/antlr/ast/Raise.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -173,6 +174,25 @@
             tback.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Repr.java b/src/org/python/antlr/ast/Repr.java
--- a/src/org/python/antlr/ast/Repr.java
+++ b/src/org/python/antlr/ast/Repr.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -121,6 +122,25 @@
             value.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Return.java b/src/org/python/antlr/ast/Return.java
--- a/src/org/python/antlr/ast/Return.java
+++ b/src/org/python/antlr/ast/Return.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -121,6 +122,25 @@
             value.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Set.java b/src/org/python/antlr/ast/Set.java
--- a/src/org/python/antlr/ast/Set.java
+++ b/src/org/python/antlr/ast/Set.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -140,6 +141,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/SetComp.java b/src/org/python/antlr/ast/SetComp.java
--- a/src/org/python/antlr/ast/SetComp.java
+++ b/src/org/python/antlr/ast/SetComp.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -166,6 +167,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Slice.java b/src/org/python/antlr/ast/Slice.java
--- a/src/org/python/antlr/ast/Slice.java
+++ b/src/org/python/antlr/ast/Slice.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -162,4 +163,23 @@
             step.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
 }
diff --git a/src/org/python/antlr/ast/Str.java b/src/org/python/antlr/ast/Str.java
--- a/src/org/python/antlr/ast/Str.java
+++ b/src/org/python/antlr/ast/Str.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -116,6 +117,25 @@
     public void traverse(VisitorIF<?> visitor) throws Exception {
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Subscript.java b/src/org/python/antlr/ast/Subscript.java
--- a/src/org/python/antlr/ast/Subscript.java
+++ b/src/org/python/antlr/ast/Subscript.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -165,6 +166,25 @@
             slice.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     public void setContext(expr_contextType c) {
         this.ctx = c;
     }
diff --git a/src/org/python/antlr/ast/Suite.java b/src/org/python/antlr/ast/Suite.java
--- a/src/org/python/antlr/ast/Suite.java
+++ b/src/org/python/antlr/ast/Suite.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -129,4 +130,23 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
 }
diff --git a/src/org/python/antlr/ast/TryExcept.java b/src/org/python/antlr/ast/TryExcept.java
--- a/src/org/python/antlr/ast/TryExcept.java
+++ b/src/org/python/antlr/ast/TryExcept.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -233,6 +234,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/TryFinally.java b/src/org/python/antlr/ast/TryFinally.java
--- a/src/org/python/antlr/ast/TryFinally.java
+++ b/src/org/python/antlr/ast/TryFinally.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -186,6 +187,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Tuple.java b/src/org/python/antlr/ast/Tuple.java
--- a/src/org/python/antlr/ast/Tuple.java
+++ b/src/org/python/antlr/ast/Tuple.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -161,6 +162,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     public void setContext(expr_contextType c) {
         this.ctx = c;
     }
diff --git a/src/org/python/antlr/ast/UnaryOp.java b/src/org/python/antlr/ast/UnaryOp.java
--- a/src/org/python/antlr/ast/UnaryOp.java
+++ b/src/org/python/antlr/ast/UnaryOp.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -142,6 +143,25 @@
             operand.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/VisitorBase.java b/src/org/python/antlr/ast/VisitorBase.java
--- a/src/org/python/antlr/ast/VisitorBase.java
+++ b/src/org/python/antlr/ast/VisitorBase.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
diff --git a/src/org/python/antlr/ast/While.java b/src/org/python/antlr/ast/While.java
--- a/src/org/python/antlr/ast/While.java
+++ b/src/org/python/antlr/ast/While.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -213,6 +214,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/With.java b/src/org/python/antlr/ast/With.java
--- a/src/org/python/antlr/ast/With.java
+++ b/src/org/python/antlr/ast/With.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -194,6 +195,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/Yield.java b/src/org/python/antlr/ast/Yield.java
--- a/src/org/python/antlr/ast/Yield.java
+++ b/src/org/python/antlr/ast/Yield.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -121,6 +122,25 @@
             value.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     private int lineno = -1;
     @ExposedGet(name = "lineno")
     public int getLineno() {
diff --git a/src/org/python/antlr/ast/alias.java b/src/org/python/antlr/ast/alias.java
--- a/src/org/python/antlr/ast/alias.java
+++ b/src/org/python/antlr/ast/alias.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -129,6 +130,25 @@
     public void traverse(VisitorIF<?> visitor) throws Exception {
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     // Support for indexer below
 
     private java.util.List<Name> nameNodes;
diff --git a/src/org/python/antlr/ast/arguments.java b/src/org/python/antlr/ast/arguments.java
--- a/src/org/python/antlr/ast/arguments.java
+++ b/src/org/python/antlr/ast/arguments.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -223,6 +224,25 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
     // Support for indexer below
 
     private Name varargName;
diff --git a/src/org/python/antlr/ast/comprehension.java b/src/org/python/antlr/ast/comprehension.java
--- a/src/org/python/antlr/ast/comprehension.java
+++ b/src/org/python/antlr/ast/comprehension.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -183,4 +184,23 @@
         }
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
 }
diff --git a/src/org/python/antlr/ast/keyword.java b/src/org/python/antlr/ast/keyword.java
--- a/src/org/python/antlr/ast/keyword.java
+++ b/src/org/python/antlr/ast/keyword.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
@@ -133,4 +134,23 @@
             value.accept(visitor);
     }
 
+    public PyObject __dict__;
+
+    @Override
+    public PyObject fastGetDict() {
+        ensureDict();
+        return __dict__;
+    }
+
+    @ExposedGet(name = "__dict__")
+    public PyObject getDict() {
+        return fastGetDict();
+    }
+
+    private void ensureDict() {
+        if (__dict__ == null) {
+            __dict__ = new PyStringMap();
+        }
+    }
+
 }
diff --git a/src/org/python/antlr/base/excepthandler.java b/src/org/python/antlr/base/excepthandler.java
--- a/src/org/python/antlr/base/excepthandler.java
+++ b/src/org/python/antlr/base/excepthandler.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
diff --git a/src/org/python/antlr/base/expr.java b/src/org/python/antlr/base/expr.java
--- a/src/org/python/antlr/base/expr.java
+++ b/src/org/python/antlr/base/expr.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
diff --git a/src/org/python/antlr/base/mod.java b/src/org/python/antlr/base/mod.java
--- a/src/org/python/antlr/base/mod.java
+++ b/src/org/python/antlr/base/mod.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
diff --git a/src/org/python/antlr/base/slice.java b/src/org/python/antlr/base/slice.java
--- a/src/org/python/antlr/base/slice.java
+++ b/src/org/python/antlr/base/slice.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;
diff --git a/src/org/python/antlr/base/stmt.java b/src/org/python/antlr/base/stmt.java
--- a/src/org/python/antlr/base/stmt.java
+++ b/src/org/python/antlr/base/stmt.java
@@ -15,6 +15,7 @@
 import org.python.core.Py;
 import org.python.core.PyObject;
 import org.python.core.PyString;
+import org.python.core.PyStringMap;
 import org.python.core.PyType;
 import org.python.expose.ExposedGet;
 import org.python.expose.ExposedMethod;

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 17 22:32:34 2014
From: jython-checkins at python.org (frank.wierzbicki)
Date: Sat, 17 May 2014 22:32:34 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Better_base_classes_for_ast?=
	=?utf-8?q?=2E?=
Message-ID: <3gWJ820kNMz7LjT@mail.python.org>

http://hg.python.org/jython/rev/213e1d9d051e
changeset:   7252:213e1d9d051e
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Fri May 16 22:50:27 2014 +0000
summary:
  Better base classes for ast.

files:
  ast/asdl_antlr.py                           |  3 ++-
  src/org/python/antlr/ast/Assert.java        |  2 +-
  src/org/python/antlr/ast/Assign.java        |  2 +-
  src/org/python/antlr/ast/Attribute.java     |  2 +-
  src/org/python/antlr/ast/AugAssign.java     |  2 +-
  src/org/python/antlr/ast/BinOp.java         |  2 +-
  src/org/python/antlr/ast/BoolOp.java        |  2 +-
  src/org/python/antlr/ast/Break.java         |  2 +-
  src/org/python/antlr/ast/Call.java          |  2 +-
  src/org/python/antlr/ast/ClassDef.java      |  2 +-
  src/org/python/antlr/ast/Compare.java       |  2 +-
  src/org/python/antlr/ast/Continue.java      |  2 +-
  src/org/python/antlr/ast/Delete.java        |  2 +-
  src/org/python/antlr/ast/Dict.java          |  2 +-
  src/org/python/antlr/ast/DictComp.java      |  2 +-
  src/org/python/antlr/ast/Ellipsis.java      |  2 +-
  src/org/python/antlr/ast/ExceptHandler.java |  2 +-
  src/org/python/antlr/ast/Exec.java          |  2 +-
  src/org/python/antlr/ast/Expr.java          |  2 +-
  src/org/python/antlr/ast/Expression.java    |  2 +-
  src/org/python/antlr/ast/ExtSlice.java      |  2 +-
  src/org/python/antlr/ast/For.java           |  2 +-
  src/org/python/antlr/ast/FunctionDef.java   |  2 +-
  src/org/python/antlr/ast/GeneratorExp.java  |  2 +-
  src/org/python/antlr/ast/Global.java        |  2 +-
  src/org/python/antlr/ast/If.java            |  2 +-
  src/org/python/antlr/ast/IfExp.java         |  2 +-
  src/org/python/antlr/ast/Import.java        |  2 +-
  src/org/python/antlr/ast/ImportFrom.java    |  2 +-
  src/org/python/antlr/ast/Index.java         |  2 +-
  src/org/python/antlr/ast/Interactive.java   |  2 +-
  src/org/python/antlr/ast/Lambda.java        |  2 +-
  src/org/python/antlr/ast/List.java          |  2 +-
  src/org/python/antlr/ast/ListComp.java      |  2 +-
  src/org/python/antlr/ast/Module.java        |  2 +-
  src/org/python/antlr/ast/Name.java          |  2 +-
  src/org/python/antlr/ast/Num.java           |  2 +-
  src/org/python/antlr/ast/Pass.java          |  2 +-
  src/org/python/antlr/ast/Print.java         |  2 +-
  src/org/python/antlr/ast/Raise.java         |  2 +-
  src/org/python/antlr/ast/Repr.java          |  2 +-
  src/org/python/antlr/ast/Return.java        |  2 +-
  src/org/python/antlr/ast/Set.java           |  2 +-
  src/org/python/antlr/ast/SetComp.java       |  2 +-
  src/org/python/antlr/ast/Slice.java         |  2 +-
  src/org/python/antlr/ast/Str.java           |  2 +-
  src/org/python/antlr/ast/Subscript.java     |  2 +-
  src/org/python/antlr/ast/Suite.java         |  2 +-
  src/org/python/antlr/ast/TryExcept.java     |  2 +-
  src/org/python/antlr/ast/TryFinally.java    |  2 +-
  src/org/python/antlr/ast/Tuple.java         |  2 +-
  src/org/python/antlr/ast/UnaryOp.java       |  2 +-
  src/org/python/antlr/ast/While.java         |  2 +-
  src/org/python/antlr/ast/With.java          |  2 +-
  src/org/python/antlr/ast/Yield.java         |  2 +-
  src/org/python/antlr/op/Add.java            |  1 +
  src/org/python/antlr/op/And.java            |  1 +
  src/org/python/antlr/op/AugLoad.java        |  1 +
  src/org/python/antlr/op/AugStore.java       |  1 +
  src/org/python/antlr/op/BitAnd.java         |  1 +
  src/org/python/antlr/op/BitOr.java          |  1 +
  src/org/python/antlr/op/BitXor.java         |  1 +
  src/org/python/antlr/op/Del.java            |  1 +
  src/org/python/antlr/op/Div.java            |  1 +
  src/org/python/antlr/op/Eq.java             |  1 +
  src/org/python/antlr/op/FloorDiv.java       |  1 +
  src/org/python/antlr/op/Gt.java             |  1 +
  src/org/python/antlr/op/GtE.java            |  1 +
  src/org/python/antlr/op/In.java             |  1 +
  src/org/python/antlr/op/Invert.java         |  1 +
  src/org/python/antlr/op/Is.java             |  1 +
  src/org/python/antlr/op/IsNot.java          |  1 +
  src/org/python/antlr/op/LShift.java         |  1 +
  src/org/python/antlr/op/Load.java           |  1 +
  src/org/python/antlr/op/Lt.java             |  1 +
  src/org/python/antlr/op/LtE.java            |  1 +
  src/org/python/antlr/op/Mod.java            |  1 +
  src/org/python/antlr/op/Mult.java           |  1 +
  src/org/python/antlr/op/Not.java            |  1 +
  src/org/python/antlr/op/NotEq.java          |  1 +
  src/org/python/antlr/op/NotIn.java          |  1 +
  src/org/python/antlr/op/Or.java             |  1 +
  src/org/python/antlr/op/Param.java          |  1 +
  src/org/python/antlr/op/Pow.java            |  1 +
  src/org/python/antlr/op/RShift.java         |  1 +
  src/org/python/antlr/op/Store.java          |  1 +
  src/org/python/antlr/op/Sub.java            |  1 +
  src/org/python/antlr/op/UAdd.java           |  1 +
  src/org/python/antlr/op/USub.java           |  1 +
  89 files changed, 90 insertions(+), 55 deletions(-)


diff --git a/ast/asdl_antlr.py b/ast/asdl_antlr.py
--- a/ast/asdl_antlr.py
+++ b/ast/asdl_antlr.py
@@ -180,6 +180,7 @@
             type = sum.types[i]
             self.open("op", type.name, refersToPythonTree=0)
             self.emit('import org.python.antlr.AST;', depth)
+            self.emit('import org.python.antlr.ast.%sType;' % name, depth)
             self.emit('import org.python.antlr.PythonTree;', depth)
             self.emit('import org.python.core.Py;', depth)
             self.emit('import org.python.core.PyObject;', depth)
@@ -328,7 +329,7 @@
             s = "implements %s " % ", ".join(ifaces)
         else:
             s = ""
-        self.emit('@ExposedType(name = "_ast.%s", base = AST.class)' % cons.name, depth);
+        self.emit('@ExposedType(name = "_ast.%s", base = %s.class)' % (cons.name, name), depth);
         self.emit("public class %s extends %s %s{" %
                     (cons.name, name, s), depth)
         self.emit("public static final PyType TYPE = PyType.fromClass(%s.class);" % cons.name, depth);
diff --git a/src/org/python/antlr/ast/Assert.java b/src/org/python/antlr/ast/Assert.java
--- a/src/org/python/antlr/ast/Assert.java
+++ b/src/org/python/antlr/ast/Assert.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Assert", base = AST.class)
+ at ExposedType(name = "_ast.Assert", base = stmt.class)
 public class Assert extends stmt {
 public static final PyType TYPE = PyType.fromClass(Assert.class);
     private expr test;
diff --git a/src/org/python/antlr/ast/Assign.java b/src/org/python/antlr/ast/Assign.java
--- a/src/org/python/antlr/ast/Assign.java
+++ b/src/org/python/antlr/ast/Assign.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Assign", base = AST.class)
+ at ExposedType(name = "_ast.Assign", base = stmt.class)
 public class Assign extends stmt {
 public static final PyType TYPE = PyType.fromClass(Assign.class);
     private java.util.List<expr> targets;
diff --git a/src/org/python/antlr/ast/Attribute.java b/src/org/python/antlr/ast/Attribute.java
--- a/src/org/python/antlr/ast/Attribute.java
+++ b/src/org/python/antlr/ast/Attribute.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Attribute", base = AST.class)
+ at ExposedType(name = "_ast.Attribute", base = expr.class)
 public class Attribute extends expr implements Context {
 public static final PyType TYPE = PyType.fromClass(Attribute.class);
     private expr value;
diff --git a/src/org/python/antlr/ast/AugAssign.java b/src/org/python/antlr/ast/AugAssign.java
--- a/src/org/python/antlr/ast/AugAssign.java
+++ b/src/org/python/antlr/ast/AugAssign.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.AugAssign", base = AST.class)
+ at ExposedType(name = "_ast.AugAssign", base = stmt.class)
 public class AugAssign extends stmt {
 public static final PyType TYPE = PyType.fromClass(AugAssign.class);
     private expr target;
diff --git a/src/org/python/antlr/ast/BinOp.java b/src/org/python/antlr/ast/BinOp.java
--- a/src/org/python/antlr/ast/BinOp.java
+++ b/src/org/python/antlr/ast/BinOp.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.BinOp", base = AST.class)
+ at ExposedType(name = "_ast.BinOp", base = expr.class)
 public class BinOp extends expr {
 public static final PyType TYPE = PyType.fromClass(BinOp.class);
     private expr left;
diff --git a/src/org/python/antlr/ast/BoolOp.java b/src/org/python/antlr/ast/BoolOp.java
--- a/src/org/python/antlr/ast/BoolOp.java
+++ b/src/org/python/antlr/ast/BoolOp.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.BoolOp", base = AST.class)
+ at ExposedType(name = "_ast.BoolOp", base = expr.class)
 public class BoolOp extends expr {
 public static final PyType TYPE = PyType.fromClass(BoolOp.class);
     private boolopType op;
diff --git a/src/org/python/antlr/ast/Break.java b/src/org/python/antlr/ast/Break.java
--- a/src/org/python/antlr/ast/Break.java
+++ b/src/org/python/antlr/ast/Break.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Break", base = AST.class)
+ at ExposedType(name = "_ast.Break", base = stmt.class)
 public class Break extends stmt {
 public static final PyType TYPE = PyType.fromClass(Break.class);
 
diff --git a/src/org/python/antlr/ast/Call.java b/src/org/python/antlr/ast/Call.java
--- a/src/org/python/antlr/ast/Call.java
+++ b/src/org/python/antlr/ast/Call.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Call", base = AST.class)
+ at ExposedType(name = "_ast.Call", base = expr.class)
 public class Call extends expr {
 public static final PyType TYPE = PyType.fromClass(Call.class);
     private expr func;
diff --git a/src/org/python/antlr/ast/ClassDef.java b/src/org/python/antlr/ast/ClassDef.java
--- a/src/org/python/antlr/ast/ClassDef.java
+++ b/src/org/python/antlr/ast/ClassDef.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.ClassDef", base = AST.class)
+ at ExposedType(name = "_ast.ClassDef", base = stmt.class)
 public class ClassDef extends stmt {
 public static final PyType TYPE = PyType.fromClass(ClassDef.class);
     private String name;
diff --git a/src/org/python/antlr/ast/Compare.java b/src/org/python/antlr/ast/Compare.java
--- a/src/org/python/antlr/ast/Compare.java
+++ b/src/org/python/antlr/ast/Compare.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Compare", base = AST.class)
+ at ExposedType(name = "_ast.Compare", base = expr.class)
 public class Compare extends expr {
 public static final PyType TYPE = PyType.fromClass(Compare.class);
     private expr left;
diff --git a/src/org/python/antlr/ast/Continue.java b/src/org/python/antlr/ast/Continue.java
--- a/src/org/python/antlr/ast/Continue.java
+++ b/src/org/python/antlr/ast/Continue.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Continue", base = AST.class)
+ at ExposedType(name = "_ast.Continue", base = stmt.class)
 public class Continue extends stmt {
 public static final PyType TYPE = PyType.fromClass(Continue.class);
 
diff --git a/src/org/python/antlr/ast/Delete.java b/src/org/python/antlr/ast/Delete.java
--- a/src/org/python/antlr/ast/Delete.java
+++ b/src/org/python/antlr/ast/Delete.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Delete", base = AST.class)
+ at ExposedType(name = "_ast.Delete", base = stmt.class)
 public class Delete extends stmt {
 public static final PyType TYPE = PyType.fromClass(Delete.class);
     private java.util.List<expr> targets;
diff --git a/src/org/python/antlr/ast/Dict.java b/src/org/python/antlr/ast/Dict.java
--- a/src/org/python/antlr/ast/Dict.java
+++ b/src/org/python/antlr/ast/Dict.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Dict", base = AST.class)
+ at ExposedType(name = "_ast.Dict", base = expr.class)
 public class Dict extends expr {
 public static final PyType TYPE = PyType.fromClass(Dict.class);
     private java.util.List<expr> keys;
diff --git a/src/org/python/antlr/ast/DictComp.java b/src/org/python/antlr/ast/DictComp.java
--- a/src/org/python/antlr/ast/DictComp.java
+++ b/src/org/python/antlr/ast/DictComp.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.DictComp", base = AST.class)
+ at ExposedType(name = "_ast.DictComp", base = expr.class)
 public class DictComp extends expr {
 public static final PyType TYPE = PyType.fromClass(DictComp.class);
     private expr key;
diff --git a/src/org/python/antlr/ast/Ellipsis.java b/src/org/python/antlr/ast/Ellipsis.java
--- a/src/org/python/antlr/ast/Ellipsis.java
+++ b/src/org/python/antlr/ast/Ellipsis.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Ellipsis", base = AST.class)
+ at ExposedType(name = "_ast.Ellipsis", base = slice.class)
 public class Ellipsis extends slice {
 public static final PyType TYPE = PyType.fromClass(Ellipsis.class);
 
diff --git a/src/org/python/antlr/ast/ExceptHandler.java b/src/org/python/antlr/ast/ExceptHandler.java
--- a/src/org/python/antlr/ast/ExceptHandler.java
+++ b/src/org/python/antlr/ast/ExceptHandler.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.ExceptHandler", base = AST.class)
+ at ExposedType(name = "_ast.ExceptHandler", base = excepthandler.class)
 public class ExceptHandler extends excepthandler {
 public static final PyType TYPE = PyType.fromClass(ExceptHandler.class);
     private expr type;
diff --git a/src/org/python/antlr/ast/Exec.java b/src/org/python/antlr/ast/Exec.java
--- a/src/org/python/antlr/ast/Exec.java
+++ b/src/org/python/antlr/ast/Exec.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Exec", base = AST.class)
+ at ExposedType(name = "_ast.Exec", base = stmt.class)
 public class Exec extends stmt {
 public static final PyType TYPE = PyType.fromClass(Exec.class);
     private expr body;
diff --git a/src/org/python/antlr/ast/Expr.java b/src/org/python/antlr/ast/Expr.java
--- a/src/org/python/antlr/ast/Expr.java
+++ b/src/org/python/antlr/ast/Expr.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Expr", base = AST.class)
+ at ExposedType(name = "_ast.Expr", base = stmt.class)
 public class Expr extends stmt {
 public static final PyType TYPE = PyType.fromClass(Expr.class);
     private expr value;
diff --git a/src/org/python/antlr/ast/Expression.java b/src/org/python/antlr/ast/Expression.java
--- a/src/org/python/antlr/ast/Expression.java
+++ b/src/org/python/antlr/ast/Expression.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Expression", base = AST.class)
+ at ExposedType(name = "_ast.Expression", base = mod.class)
 public class Expression extends mod {
 public static final PyType TYPE = PyType.fromClass(Expression.class);
     private expr body;
diff --git a/src/org/python/antlr/ast/ExtSlice.java b/src/org/python/antlr/ast/ExtSlice.java
--- a/src/org/python/antlr/ast/ExtSlice.java
+++ b/src/org/python/antlr/ast/ExtSlice.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.ExtSlice", base = AST.class)
+ at ExposedType(name = "_ast.ExtSlice", base = slice.class)
 public class ExtSlice extends slice {
 public static final PyType TYPE = PyType.fromClass(ExtSlice.class);
     private java.util.List<slice> dims;
diff --git a/src/org/python/antlr/ast/For.java b/src/org/python/antlr/ast/For.java
--- a/src/org/python/antlr/ast/For.java
+++ b/src/org/python/antlr/ast/For.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.For", base = AST.class)
+ at ExposedType(name = "_ast.For", base = stmt.class)
 public class For extends stmt {
 public static final PyType TYPE = PyType.fromClass(For.class);
     private expr target;
diff --git a/src/org/python/antlr/ast/FunctionDef.java b/src/org/python/antlr/ast/FunctionDef.java
--- a/src/org/python/antlr/ast/FunctionDef.java
+++ b/src/org/python/antlr/ast/FunctionDef.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.FunctionDef", base = AST.class)
+ at ExposedType(name = "_ast.FunctionDef", base = stmt.class)
 public class FunctionDef extends stmt {
 public static final PyType TYPE = PyType.fromClass(FunctionDef.class);
     private String name;
diff --git a/src/org/python/antlr/ast/GeneratorExp.java b/src/org/python/antlr/ast/GeneratorExp.java
--- a/src/org/python/antlr/ast/GeneratorExp.java
+++ b/src/org/python/antlr/ast/GeneratorExp.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.GeneratorExp", base = AST.class)
+ at ExposedType(name = "_ast.GeneratorExp", base = expr.class)
 public class GeneratorExp extends expr {
 public static final PyType TYPE = PyType.fromClass(GeneratorExp.class);
     private expr elt;
diff --git a/src/org/python/antlr/ast/Global.java b/src/org/python/antlr/ast/Global.java
--- a/src/org/python/antlr/ast/Global.java
+++ b/src/org/python/antlr/ast/Global.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Global", base = AST.class)
+ at ExposedType(name = "_ast.Global", base = stmt.class)
 public class Global extends stmt {
 public static final PyType TYPE = PyType.fromClass(Global.class);
     private java.util.List<String> names;
diff --git a/src/org/python/antlr/ast/If.java b/src/org/python/antlr/ast/If.java
--- a/src/org/python/antlr/ast/If.java
+++ b/src/org/python/antlr/ast/If.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.If", base = AST.class)
+ at ExposedType(name = "_ast.If", base = stmt.class)
 public class If extends stmt {
 public static final PyType TYPE = PyType.fromClass(If.class);
     private expr test;
diff --git a/src/org/python/antlr/ast/IfExp.java b/src/org/python/antlr/ast/IfExp.java
--- a/src/org/python/antlr/ast/IfExp.java
+++ b/src/org/python/antlr/ast/IfExp.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.IfExp", base = AST.class)
+ at ExposedType(name = "_ast.IfExp", base = expr.class)
 public class IfExp extends expr {
 public static final PyType TYPE = PyType.fromClass(IfExp.class);
     private expr test;
diff --git a/src/org/python/antlr/ast/Import.java b/src/org/python/antlr/ast/Import.java
--- a/src/org/python/antlr/ast/Import.java
+++ b/src/org/python/antlr/ast/Import.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Import", base = AST.class)
+ at ExposedType(name = "_ast.Import", base = stmt.class)
 public class Import extends stmt {
 public static final PyType TYPE = PyType.fromClass(Import.class);
     private java.util.List<alias> names;
diff --git a/src/org/python/antlr/ast/ImportFrom.java b/src/org/python/antlr/ast/ImportFrom.java
--- a/src/org/python/antlr/ast/ImportFrom.java
+++ b/src/org/python/antlr/ast/ImportFrom.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.ImportFrom", base = AST.class)
+ at ExposedType(name = "_ast.ImportFrom", base = stmt.class)
 public class ImportFrom extends stmt {
 public static final PyType TYPE = PyType.fromClass(ImportFrom.class);
     private String module;
diff --git a/src/org/python/antlr/ast/Index.java b/src/org/python/antlr/ast/Index.java
--- a/src/org/python/antlr/ast/Index.java
+++ b/src/org/python/antlr/ast/Index.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Index", base = AST.class)
+ at ExposedType(name = "_ast.Index", base = slice.class)
 public class Index extends slice {
 public static final PyType TYPE = PyType.fromClass(Index.class);
     private expr value;
diff --git a/src/org/python/antlr/ast/Interactive.java b/src/org/python/antlr/ast/Interactive.java
--- a/src/org/python/antlr/ast/Interactive.java
+++ b/src/org/python/antlr/ast/Interactive.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Interactive", base = AST.class)
+ at ExposedType(name = "_ast.Interactive", base = mod.class)
 public class Interactive extends mod {
 public static final PyType TYPE = PyType.fromClass(Interactive.class);
     private java.util.List<stmt> body;
diff --git a/src/org/python/antlr/ast/Lambda.java b/src/org/python/antlr/ast/Lambda.java
--- a/src/org/python/antlr/ast/Lambda.java
+++ b/src/org/python/antlr/ast/Lambda.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Lambda", base = AST.class)
+ at ExposedType(name = "_ast.Lambda", base = expr.class)
 public class Lambda extends expr {
 public static final PyType TYPE = PyType.fromClass(Lambda.class);
     private arguments args;
diff --git a/src/org/python/antlr/ast/List.java b/src/org/python/antlr/ast/List.java
--- a/src/org/python/antlr/ast/List.java
+++ b/src/org/python/antlr/ast/List.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.List", base = AST.class)
+ at ExposedType(name = "_ast.List", base = expr.class)
 public class List extends expr implements Context {
 public static final PyType TYPE = PyType.fromClass(List.class);
     private java.util.List<expr> elts;
diff --git a/src/org/python/antlr/ast/ListComp.java b/src/org/python/antlr/ast/ListComp.java
--- a/src/org/python/antlr/ast/ListComp.java
+++ b/src/org/python/antlr/ast/ListComp.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.ListComp", base = AST.class)
+ at ExposedType(name = "_ast.ListComp", base = expr.class)
 public class ListComp extends expr {
 public static final PyType TYPE = PyType.fromClass(ListComp.class);
     private expr elt;
diff --git a/src/org/python/antlr/ast/Module.java b/src/org/python/antlr/ast/Module.java
--- a/src/org/python/antlr/ast/Module.java
+++ b/src/org/python/antlr/ast/Module.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Module", base = AST.class)
+ at ExposedType(name = "_ast.Module", base = mod.class)
 public class Module extends mod {
 public static final PyType TYPE = PyType.fromClass(Module.class);
     private java.util.List<stmt> body;
diff --git a/src/org/python/antlr/ast/Name.java b/src/org/python/antlr/ast/Name.java
--- a/src/org/python/antlr/ast/Name.java
+++ b/src/org/python/antlr/ast/Name.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Name", base = AST.class)
+ at ExposedType(name = "_ast.Name", base = expr.class)
 public class Name extends expr implements Context {
 public static final PyType TYPE = PyType.fromClass(Name.class);
     private String id;
diff --git a/src/org/python/antlr/ast/Num.java b/src/org/python/antlr/ast/Num.java
--- a/src/org/python/antlr/ast/Num.java
+++ b/src/org/python/antlr/ast/Num.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Num", base = AST.class)
+ at ExposedType(name = "_ast.Num", base = expr.class)
 public class Num extends expr {
 public static final PyType TYPE = PyType.fromClass(Num.class);
     private Object n;
diff --git a/src/org/python/antlr/ast/Pass.java b/src/org/python/antlr/ast/Pass.java
--- a/src/org/python/antlr/ast/Pass.java
+++ b/src/org/python/antlr/ast/Pass.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Pass", base = AST.class)
+ at ExposedType(name = "_ast.Pass", base = stmt.class)
 public class Pass extends stmt {
 public static final PyType TYPE = PyType.fromClass(Pass.class);
 
diff --git a/src/org/python/antlr/ast/Print.java b/src/org/python/antlr/ast/Print.java
--- a/src/org/python/antlr/ast/Print.java
+++ b/src/org/python/antlr/ast/Print.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Print", base = AST.class)
+ at ExposedType(name = "_ast.Print", base = stmt.class)
 public class Print extends stmt {
 public static final PyType TYPE = PyType.fromClass(Print.class);
     private expr dest;
diff --git a/src/org/python/antlr/ast/Raise.java b/src/org/python/antlr/ast/Raise.java
--- a/src/org/python/antlr/ast/Raise.java
+++ b/src/org/python/antlr/ast/Raise.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Raise", base = AST.class)
+ at ExposedType(name = "_ast.Raise", base = stmt.class)
 public class Raise extends stmt {
 public static final PyType TYPE = PyType.fromClass(Raise.class);
     private expr type;
diff --git a/src/org/python/antlr/ast/Repr.java b/src/org/python/antlr/ast/Repr.java
--- a/src/org/python/antlr/ast/Repr.java
+++ b/src/org/python/antlr/ast/Repr.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Repr", base = AST.class)
+ at ExposedType(name = "_ast.Repr", base = expr.class)
 public class Repr extends expr {
 public static final PyType TYPE = PyType.fromClass(Repr.class);
     private expr value;
diff --git a/src/org/python/antlr/ast/Return.java b/src/org/python/antlr/ast/Return.java
--- a/src/org/python/antlr/ast/Return.java
+++ b/src/org/python/antlr/ast/Return.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Return", base = AST.class)
+ at ExposedType(name = "_ast.Return", base = stmt.class)
 public class Return extends stmt {
 public static final PyType TYPE = PyType.fromClass(Return.class);
     private expr value;
diff --git a/src/org/python/antlr/ast/Set.java b/src/org/python/antlr/ast/Set.java
--- a/src/org/python/antlr/ast/Set.java
+++ b/src/org/python/antlr/ast/Set.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Set", base = AST.class)
+ at ExposedType(name = "_ast.Set", base = expr.class)
 public class Set extends expr {
 public static final PyType TYPE = PyType.fromClass(Set.class);
     private java.util.List<expr> elts;
diff --git a/src/org/python/antlr/ast/SetComp.java b/src/org/python/antlr/ast/SetComp.java
--- a/src/org/python/antlr/ast/SetComp.java
+++ b/src/org/python/antlr/ast/SetComp.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.SetComp", base = AST.class)
+ at ExposedType(name = "_ast.SetComp", base = expr.class)
 public class SetComp extends expr {
 public static final PyType TYPE = PyType.fromClass(SetComp.class);
     private expr elt;
diff --git a/src/org/python/antlr/ast/Slice.java b/src/org/python/antlr/ast/Slice.java
--- a/src/org/python/antlr/ast/Slice.java
+++ b/src/org/python/antlr/ast/Slice.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Slice", base = AST.class)
+ at ExposedType(name = "_ast.Slice", base = slice.class)
 public class Slice extends slice {
 public static final PyType TYPE = PyType.fromClass(Slice.class);
     private expr lower;
diff --git a/src/org/python/antlr/ast/Str.java b/src/org/python/antlr/ast/Str.java
--- a/src/org/python/antlr/ast/Str.java
+++ b/src/org/python/antlr/ast/Str.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Str", base = AST.class)
+ at ExposedType(name = "_ast.Str", base = expr.class)
 public class Str extends expr {
 public static final PyType TYPE = PyType.fromClass(Str.class);
     private Object s;
diff --git a/src/org/python/antlr/ast/Subscript.java b/src/org/python/antlr/ast/Subscript.java
--- a/src/org/python/antlr/ast/Subscript.java
+++ b/src/org/python/antlr/ast/Subscript.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Subscript", base = AST.class)
+ at ExposedType(name = "_ast.Subscript", base = expr.class)
 public class Subscript extends expr implements Context {
 public static final PyType TYPE = PyType.fromClass(Subscript.class);
     private expr value;
diff --git a/src/org/python/antlr/ast/Suite.java b/src/org/python/antlr/ast/Suite.java
--- a/src/org/python/antlr/ast/Suite.java
+++ b/src/org/python/antlr/ast/Suite.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Suite", base = AST.class)
+ at ExposedType(name = "_ast.Suite", base = mod.class)
 public class Suite extends mod {
 public static final PyType TYPE = PyType.fromClass(Suite.class);
     private java.util.List<stmt> body;
diff --git a/src/org/python/antlr/ast/TryExcept.java b/src/org/python/antlr/ast/TryExcept.java
--- a/src/org/python/antlr/ast/TryExcept.java
+++ b/src/org/python/antlr/ast/TryExcept.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.TryExcept", base = AST.class)
+ at ExposedType(name = "_ast.TryExcept", base = stmt.class)
 public class TryExcept extends stmt {
 public static final PyType TYPE = PyType.fromClass(TryExcept.class);
     private java.util.List<stmt> body;
diff --git a/src/org/python/antlr/ast/TryFinally.java b/src/org/python/antlr/ast/TryFinally.java
--- a/src/org/python/antlr/ast/TryFinally.java
+++ b/src/org/python/antlr/ast/TryFinally.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.TryFinally", base = AST.class)
+ at ExposedType(name = "_ast.TryFinally", base = stmt.class)
 public class TryFinally extends stmt {
 public static final PyType TYPE = PyType.fromClass(TryFinally.class);
     private java.util.List<stmt> body;
diff --git a/src/org/python/antlr/ast/Tuple.java b/src/org/python/antlr/ast/Tuple.java
--- a/src/org/python/antlr/ast/Tuple.java
+++ b/src/org/python/antlr/ast/Tuple.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Tuple", base = AST.class)
+ at ExposedType(name = "_ast.Tuple", base = expr.class)
 public class Tuple extends expr implements Context {
 public static final PyType TYPE = PyType.fromClass(Tuple.class);
     private java.util.List<expr> elts;
diff --git a/src/org/python/antlr/ast/UnaryOp.java b/src/org/python/antlr/ast/UnaryOp.java
--- a/src/org/python/antlr/ast/UnaryOp.java
+++ b/src/org/python/antlr/ast/UnaryOp.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.UnaryOp", base = AST.class)
+ at ExposedType(name = "_ast.UnaryOp", base = expr.class)
 public class UnaryOp extends expr {
 public static final PyType TYPE = PyType.fromClass(UnaryOp.class);
     private unaryopType op;
diff --git a/src/org/python/antlr/ast/While.java b/src/org/python/antlr/ast/While.java
--- a/src/org/python/antlr/ast/While.java
+++ b/src/org/python/antlr/ast/While.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.While", base = AST.class)
+ at ExposedType(name = "_ast.While", base = stmt.class)
 public class While extends stmt {
 public static final PyType TYPE = PyType.fromClass(While.class);
     private expr test;
diff --git a/src/org/python/antlr/ast/With.java b/src/org/python/antlr/ast/With.java
--- a/src/org/python/antlr/ast/With.java
+++ b/src/org/python/antlr/ast/With.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.With", base = AST.class)
+ at ExposedType(name = "_ast.With", base = stmt.class)
 public class With extends stmt {
 public static final PyType TYPE = PyType.fromClass(With.class);
     private expr context_expr;
diff --git a/src/org/python/antlr/ast/Yield.java b/src/org/python/antlr/ast/Yield.java
--- a/src/org/python/antlr/ast/Yield.java
+++ b/src/org/python/antlr/ast/Yield.java
@@ -26,7 +26,7 @@
 import java.io.IOException;
 import java.util.ArrayList;
 
- at ExposedType(name = "_ast.Yield", base = AST.class)
+ at ExposedType(name = "_ast.Yield", base = expr.class)
 public class Yield extends expr {
 public static final PyType TYPE = PyType.fromClass(Yield.class);
     private expr value;
diff --git a/src/org/python/antlr/op/Add.java b/src/org/python/antlr/op/Add.java
--- a/src/org/python/antlr/op/Add.java
+++ b/src/org/python/antlr/op/Add.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/And.java b/src/org/python/antlr/op/And.java
--- a/src/org/python/antlr/op/And.java
+++ b/src/org/python/antlr/op/And.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.boolopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/AugLoad.java b/src/org/python/antlr/op/AugLoad.java
--- a/src/org/python/antlr/op/AugLoad.java
+++ b/src/org/python/antlr/op/AugLoad.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.expr_contextType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/AugStore.java b/src/org/python/antlr/op/AugStore.java
--- a/src/org/python/antlr/op/AugStore.java
+++ b/src/org/python/antlr/op/AugStore.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.expr_contextType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/BitAnd.java b/src/org/python/antlr/op/BitAnd.java
--- a/src/org/python/antlr/op/BitAnd.java
+++ b/src/org/python/antlr/op/BitAnd.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/BitOr.java b/src/org/python/antlr/op/BitOr.java
--- a/src/org/python/antlr/op/BitOr.java
+++ b/src/org/python/antlr/op/BitOr.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/BitXor.java b/src/org/python/antlr/op/BitXor.java
--- a/src/org/python/antlr/op/BitXor.java
+++ b/src/org/python/antlr/op/BitXor.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Del.java b/src/org/python/antlr/op/Del.java
--- a/src/org/python/antlr/op/Del.java
+++ b/src/org/python/antlr/op/Del.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.expr_contextType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Div.java b/src/org/python/antlr/op/Div.java
--- a/src/org/python/antlr/op/Div.java
+++ b/src/org/python/antlr/op/Div.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Eq.java b/src/org/python/antlr/op/Eq.java
--- a/src/org/python/antlr/op/Eq.java
+++ b/src/org/python/antlr/op/Eq.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.cmpopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/FloorDiv.java b/src/org/python/antlr/op/FloorDiv.java
--- a/src/org/python/antlr/op/FloorDiv.java
+++ b/src/org/python/antlr/op/FloorDiv.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Gt.java b/src/org/python/antlr/op/Gt.java
--- a/src/org/python/antlr/op/Gt.java
+++ b/src/org/python/antlr/op/Gt.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.cmpopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/GtE.java b/src/org/python/antlr/op/GtE.java
--- a/src/org/python/antlr/op/GtE.java
+++ b/src/org/python/antlr/op/GtE.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.cmpopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/In.java b/src/org/python/antlr/op/In.java
--- a/src/org/python/antlr/op/In.java
+++ b/src/org/python/antlr/op/In.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.cmpopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Invert.java b/src/org/python/antlr/op/Invert.java
--- a/src/org/python/antlr/op/Invert.java
+++ b/src/org/python/antlr/op/Invert.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.unaryopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Is.java b/src/org/python/antlr/op/Is.java
--- a/src/org/python/antlr/op/Is.java
+++ b/src/org/python/antlr/op/Is.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.cmpopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/IsNot.java b/src/org/python/antlr/op/IsNot.java
--- a/src/org/python/antlr/op/IsNot.java
+++ b/src/org/python/antlr/op/IsNot.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.cmpopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/LShift.java b/src/org/python/antlr/op/LShift.java
--- a/src/org/python/antlr/op/LShift.java
+++ b/src/org/python/antlr/op/LShift.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Load.java b/src/org/python/antlr/op/Load.java
--- a/src/org/python/antlr/op/Load.java
+++ b/src/org/python/antlr/op/Load.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.expr_contextType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Lt.java b/src/org/python/antlr/op/Lt.java
--- a/src/org/python/antlr/op/Lt.java
+++ b/src/org/python/antlr/op/Lt.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.cmpopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/LtE.java b/src/org/python/antlr/op/LtE.java
--- a/src/org/python/antlr/op/LtE.java
+++ b/src/org/python/antlr/op/LtE.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.cmpopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Mod.java b/src/org/python/antlr/op/Mod.java
--- a/src/org/python/antlr/op/Mod.java
+++ b/src/org/python/antlr/op/Mod.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Mult.java b/src/org/python/antlr/op/Mult.java
--- a/src/org/python/antlr/op/Mult.java
+++ b/src/org/python/antlr/op/Mult.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Not.java b/src/org/python/antlr/op/Not.java
--- a/src/org/python/antlr/op/Not.java
+++ b/src/org/python/antlr/op/Not.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.unaryopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/NotEq.java b/src/org/python/antlr/op/NotEq.java
--- a/src/org/python/antlr/op/NotEq.java
+++ b/src/org/python/antlr/op/NotEq.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.cmpopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/NotIn.java b/src/org/python/antlr/op/NotIn.java
--- a/src/org/python/antlr/op/NotIn.java
+++ b/src/org/python/antlr/op/NotIn.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.cmpopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Or.java b/src/org/python/antlr/op/Or.java
--- a/src/org/python/antlr/op/Or.java
+++ b/src/org/python/antlr/op/Or.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.boolopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Param.java b/src/org/python/antlr/op/Param.java
--- a/src/org/python/antlr/op/Param.java
+++ b/src/org/python/antlr/op/Param.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.expr_contextType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Pow.java b/src/org/python/antlr/op/Pow.java
--- a/src/org/python/antlr/op/Pow.java
+++ b/src/org/python/antlr/op/Pow.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/RShift.java b/src/org/python/antlr/op/RShift.java
--- a/src/org/python/antlr/op/RShift.java
+++ b/src/org/python/antlr/op/RShift.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Store.java b/src/org/python/antlr/op/Store.java
--- a/src/org/python/antlr/op/Store.java
+++ b/src/org/python/antlr/op/Store.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.expr_contextType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/Sub.java b/src/org/python/antlr/op/Sub.java
--- a/src/org/python/antlr/op/Sub.java
+++ b/src/org/python/antlr/op/Sub.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.operatorType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/UAdd.java b/src/org/python/antlr/op/UAdd.java
--- a/src/org/python/antlr/op/UAdd.java
+++ b/src/org/python/antlr/op/UAdd.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.unaryopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
diff --git a/src/org/python/antlr/op/USub.java b/src/org/python/antlr/op/USub.java
--- a/src/org/python/antlr/op/USub.java
+++ b/src/org/python/antlr/op/USub.java
@@ -2,6 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
+import org.python.antlr.ast.unaryopType;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 17 22:32:35 2014
From: jython-checkins at python.org (frank.wierzbicki)
Date: Sat, 17 May 2014 22:32:35 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_=231497_ast_classes_now_hav?=
 =?utf-8?q?e_appropriate_base_classes=2E?=
Message-ID: <3gWJ8361xFz7LjT@mail.python.org>

http://hg.python.org/jython/rev/b3f5c51c0a83
changeset:   7253:b3f5c51c0a83
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Sat May 17 20:32:23 2014 +0000
summary:
  #1497 ast classes now have appropriate base classes.

files:
  CoreExposed.includes                    |   5 ++++
  NEWS                                    |   3 ++
  ast/asdl_antlr.py                       |   4 +-
  src/org/python/antlr/ast/AstModule.java |  16 ++++--------
  src/org/python/antlr/op/Add.java        |   4 +-
  src/org/python/antlr/op/And.java        |   4 +-
  src/org/python/antlr/op/AugLoad.java    |   4 +-
  src/org/python/antlr/op/AugStore.java   |   4 +-
  src/org/python/antlr/op/BitAnd.java     |   4 +-
  src/org/python/antlr/op/BitOr.java      |   4 +-
  src/org/python/antlr/op/BitXor.java     |   4 +-
  src/org/python/antlr/op/Del.java        |   4 +-
  src/org/python/antlr/op/Div.java        |   4 +-
  src/org/python/antlr/op/Eq.java         |   4 +-
  src/org/python/antlr/op/FloorDiv.java   |   4 +-
  src/org/python/antlr/op/Gt.java         |   4 +-
  src/org/python/antlr/op/GtE.java        |   4 +-
  src/org/python/antlr/op/In.java         |   4 +-
  src/org/python/antlr/op/Invert.java     |   4 +-
  src/org/python/antlr/op/Is.java         |   4 +-
  src/org/python/antlr/op/IsNot.java      |   4 +-
  src/org/python/antlr/op/LShift.java     |   4 +-
  src/org/python/antlr/op/Load.java       |   4 +-
  src/org/python/antlr/op/Lt.java         |   4 +-
  src/org/python/antlr/op/LtE.java        |   4 +-
  src/org/python/antlr/op/Mod.java        |   4 +-
  src/org/python/antlr/op/Mult.java       |   4 +-
  src/org/python/antlr/op/Not.java        |   4 +-
  src/org/python/antlr/op/NotEq.java      |   4 +-
  src/org/python/antlr/op/NotIn.java      |   4 +-
  src/org/python/antlr/op/Or.java         |   4 +-
  src/org/python/antlr/op/Param.java      |   4 +-
  src/org/python/antlr/op/Pow.java        |   4 +-
  src/org/python/antlr/op/RShift.java     |   4 +-
  src/org/python/antlr/op/Store.java      |   4 +-
  src/org/python/antlr/op/Sub.java        |   4 +-
  src/org/python/antlr/op/UAdd.java       |   4 +-
  src/org/python/antlr/op/USub.java       |   4 +-
  38 files changed, 84 insertions(+), 80 deletions(-)


diff --git a/CoreExposed.includes b/CoreExposed.includes
--- a/CoreExposed.includes
+++ b/CoreExposed.includes
@@ -172,11 +172,16 @@
 org/python/antlr/ast/While.class
 org/python/antlr/ast/With.class
 org/python/antlr/ast/Yield.class
+org/python/antlr/base/boolop.class
+org/python/antlr/base/cmpop.class
 org/python/antlr/base/excepthandler.class
 org/python/antlr/base/expr.class
+org/python/antlr/base/expr_context.class
 org/python/antlr/base/mod.class
+org/python/antlr/base/operator.class
 org/python/antlr/base/slice.class
 org/python/antlr/base/stmt.class
+org/python/antlr/base/unaryop.class
 org/python/antlr/op/Add.class
 org/python/antlr/op/And.class
 org/python/antlr/op/AugLoad.class
diff --git a/NEWS b/NEWS
--- a/NEWS
+++ b/NEWS
@@ -3,6 +3,9 @@
 Jython 2.7b3
   Bugs Fixed
     - [ 2108 ] Cannot set attribute to instances of AST/PythonTree (blocks pyflakes)
+    - [ 1497 ] ast classes do not have appropiate base classes
+    - [ 1980 ] ast.Eq, ast.Gt, ast.GtE, ast.In, ast.Is, ast.IsNot, ast.Lt, ast.LtE, ast.NotEq and ast.NotIn should be subclasses of ast.cmpop
+    - [ 1981 ] ast.And and ast.Or should be subclasses of ast.boolop
   New Features
     - [ 1896215 ] findResource(s) for SyspathJavaLoader
 
diff --git a/ast/asdl_antlr.py b/ast/asdl_antlr.py
--- a/ast/asdl_antlr.py
+++ b/ast/asdl_antlr.py
@@ -180,7 +180,7 @@
             type = sum.types[i]
             self.open("op", type.name, refersToPythonTree=0)
             self.emit('import org.python.antlr.AST;', depth)
-            self.emit('import org.python.antlr.ast.%sType;' % name, depth)
+            self.emit('import org.python.antlr.base.%s;' % name, depth)
             self.emit('import org.python.antlr.PythonTree;', depth)
             self.emit('import org.python.core.Py;', depth)
             self.emit('import org.python.core.PyObject;', depth)
@@ -193,7 +193,7 @@
             self.emit('import org.python.expose.ExposedType;', depth)
             self.emit('', 0)
 
-            self.emit('@ExposedType(name = "_ast.%s", base = AST.class)' % type.name, depth)
+            self.emit('@ExposedType(name = "_ast.%s", base = %s.class)' % (type.name, name), depth)
             self.emit("public class %s extends PythonTree {" % type.name, depth)
             self.emit('public static final PyType TYPE = PyType.fromClass(%s.class);' % type.name, depth + 1)
             self.emit('', 0)
diff --git a/src/org/python/antlr/ast/AstModule.java b/src/org/python/antlr/ast/AstModule.java
--- a/src/org/python/antlr/ast/AstModule.java
+++ b/src/org/python/antlr/ast/AstModule.java
@@ -1,10 +1,6 @@
 package org.python.antlr.ast;
 
-import org.python.antlr.base.excepthandler;
-import org.python.antlr.base.expr;
-import org.python.antlr.base.mod;
-import org.python.antlr.base.slice;
-import org.python.antlr.base.stmt;
+import org.python.antlr.base.*;
 import org.python.antlr.op.*;
 import org.python.core.AstList;
 import org.python.core.ClassDictInit;
@@ -99,7 +95,7 @@
         dict.__setitem__("slice", slice.TYPE);
         dict.__setitem__("stmt", stmt.TYPE);
         
-        dict.__setitem__("operator", imp.importName("org.python.antlr.ast.operatorType", false));
+        dict.__setitem__("operator", operator.TYPE);
         dict.__setitem__("Add", Add.TYPE);
         dict.__setitem__("Sub", Sub.TYPE);
         dict.__setitem__("Mult", Mult.TYPE);
@@ -113,11 +109,11 @@
         dict.__setitem__("BitXor", BitXor.TYPE);
         dict.__setitem__("Pow", Pow.TYPE);
        
-        dict.__setitem__("boolop", imp.importName("org.python.antlr.ast.boolopType", false));
+        dict.__setitem__("boolop", boolop.TYPE);
         dict.__setitem__("And", And.TYPE);
         dict.__setitem__("Or", Or.TYPE);
       
-        dict.__setitem__("cmpop", imp.importName("org.python.antlr.ast.cmpopType", false));
+        dict.__setitem__("cmpop", cmpop.TYPE);
         dict.__setitem__("Eq", Eq.TYPE);
         dict.__setitem__("Gt", Gt.TYPE);
         dict.__setitem__("GtE", GtE.TYPE);
@@ -129,7 +125,7 @@
         dict.__setitem__("NotEq", NotEq.TYPE);
         dict.__setitem__("NotIn", NotIn.TYPE);
        
-        dict.__setitem__("expr_context", imp.importName("org.python.antlr.ast.expr_contextType", false));
+        dict.__setitem__("expr_context", expr_context.TYPE);
         dict.__setitem__("Load", Load.TYPE);
         dict.__setitem__("Store", Store.TYPE);
         dict.__setitem__("Del", Del.TYPE);
@@ -137,7 +133,7 @@
         dict.__setitem__("AugStore", AugStore.TYPE);
         dict.__setitem__("Param", Param.TYPE);
        
-        dict.__setitem__("unaryop", imp.importName("org.python.antlr.ast.unaryopType", false));
+        dict.__setitem__("unaryop", unaryop.TYPE);
         dict.__setitem__("Invert", Invert.TYPE);
         dict.__setitem__("Not", Not.TYPE);
         dict.__setitem__("UAdd", UAdd.TYPE);
diff --git a/src/org/python/antlr/op/Add.java b/src/org/python/antlr/op/Add.java
--- a/src/org/python/antlr/op/Add.java
+++ b/src/org/python/antlr/op/Add.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Add", base = AST.class)
+ at ExposedType(name = "_ast.Add", base = operator.class)
 public class Add extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Add.class);
 
diff --git a/src/org/python/antlr/op/And.java b/src/org/python/antlr/op/And.java
--- a/src/org/python/antlr/op/And.java
+++ b/src/org/python/antlr/op/And.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.boolopType;
+import org.python.antlr.base.boolop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.And", base = AST.class)
+ at ExposedType(name = "_ast.And", base = boolop.class)
 public class And extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(And.class);
 
diff --git a/src/org/python/antlr/op/AugLoad.java b/src/org/python/antlr/op/AugLoad.java
--- a/src/org/python/antlr/op/AugLoad.java
+++ b/src/org/python/antlr/op/AugLoad.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.expr_contextType;
+import org.python.antlr.base.expr_context;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.AugLoad", base = AST.class)
+ at ExposedType(name = "_ast.AugLoad", base = expr_context.class)
 public class AugLoad extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(AugLoad.class);
 
diff --git a/src/org/python/antlr/op/AugStore.java b/src/org/python/antlr/op/AugStore.java
--- a/src/org/python/antlr/op/AugStore.java
+++ b/src/org/python/antlr/op/AugStore.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.expr_contextType;
+import org.python.antlr.base.expr_context;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.AugStore", base = AST.class)
+ at ExposedType(name = "_ast.AugStore", base = expr_context.class)
 public class AugStore extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(AugStore.class);
 
diff --git a/src/org/python/antlr/op/BitAnd.java b/src/org/python/antlr/op/BitAnd.java
--- a/src/org/python/antlr/op/BitAnd.java
+++ b/src/org/python/antlr/op/BitAnd.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.BitAnd", base = AST.class)
+ at ExposedType(name = "_ast.BitAnd", base = operator.class)
 public class BitAnd extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(BitAnd.class);
 
diff --git a/src/org/python/antlr/op/BitOr.java b/src/org/python/antlr/op/BitOr.java
--- a/src/org/python/antlr/op/BitOr.java
+++ b/src/org/python/antlr/op/BitOr.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.BitOr", base = AST.class)
+ at ExposedType(name = "_ast.BitOr", base = operator.class)
 public class BitOr extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(BitOr.class);
 
diff --git a/src/org/python/antlr/op/BitXor.java b/src/org/python/antlr/op/BitXor.java
--- a/src/org/python/antlr/op/BitXor.java
+++ b/src/org/python/antlr/op/BitXor.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.BitXor", base = AST.class)
+ at ExposedType(name = "_ast.BitXor", base = operator.class)
 public class BitXor extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(BitXor.class);
 
diff --git a/src/org/python/antlr/op/Del.java b/src/org/python/antlr/op/Del.java
--- a/src/org/python/antlr/op/Del.java
+++ b/src/org/python/antlr/op/Del.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.expr_contextType;
+import org.python.antlr.base.expr_context;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Del", base = AST.class)
+ at ExposedType(name = "_ast.Del", base = expr_context.class)
 public class Del extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Del.class);
 
diff --git a/src/org/python/antlr/op/Div.java b/src/org/python/antlr/op/Div.java
--- a/src/org/python/antlr/op/Div.java
+++ b/src/org/python/antlr/op/Div.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Div", base = AST.class)
+ at ExposedType(name = "_ast.Div", base = operator.class)
 public class Div extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Div.class);
 
diff --git a/src/org/python/antlr/op/Eq.java b/src/org/python/antlr/op/Eq.java
--- a/src/org/python/antlr/op/Eq.java
+++ b/src/org/python/antlr/op/Eq.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.cmpopType;
+import org.python.antlr.base.cmpop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Eq", base = AST.class)
+ at ExposedType(name = "_ast.Eq", base = cmpop.class)
 public class Eq extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Eq.class);
 
diff --git a/src/org/python/antlr/op/FloorDiv.java b/src/org/python/antlr/op/FloorDiv.java
--- a/src/org/python/antlr/op/FloorDiv.java
+++ b/src/org/python/antlr/op/FloorDiv.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.FloorDiv", base = AST.class)
+ at ExposedType(name = "_ast.FloorDiv", base = operator.class)
 public class FloorDiv extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(FloorDiv.class);
 
diff --git a/src/org/python/antlr/op/Gt.java b/src/org/python/antlr/op/Gt.java
--- a/src/org/python/antlr/op/Gt.java
+++ b/src/org/python/antlr/op/Gt.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.cmpopType;
+import org.python.antlr.base.cmpop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Gt", base = AST.class)
+ at ExposedType(name = "_ast.Gt", base = cmpop.class)
 public class Gt extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Gt.class);
 
diff --git a/src/org/python/antlr/op/GtE.java b/src/org/python/antlr/op/GtE.java
--- a/src/org/python/antlr/op/GtE.java
+++ b/src/org/python/antlr/op/GtE.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.cmpopType;
+import org.python.antlr.base.cmpop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.GtE", base = AST.class)
+ at ExposedType(name = "_ast.GtE", base = cmpop.class)
 public class GtE extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(GtE.class);
 
diff --git a/src/org/python/antlr/op/In.java b/src/org/python/antlr/op/In.java
--- a/src/org/python/antlr/op/In.java
+++ b/src/org/python/antlr/op/In.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.cmpopType;
+import org.python.antlr.base.cmpop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.In", base = AST.class)
+ at ExposedType(name = "_ast.In", base = cmpop.class)
 public class In extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(In.class);
 
diff --git a/src/org/python/antlr/op/Invert.java b/src/org/python/antlr/op/Invert.java
--- a/src/org/python/antlr/op/Invert.java
+++ b/src/org/python/antlr/op/Invert.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.unaryopType;
+import org.python.antlr.base.unaryop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Invert", base = AST.class)
+ at ExposedType(name = "_ast.Invert", base = unaryop.class)
 public class Invert extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Invert.class);
 
diff --git a/src/org/python/antlr/op/Is.java b/src/org/python/antlr/op/Is.java
--- a/src/org/python/antlr/op/Is.java
+++ b/src/org/python/antlr/op/Is.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.cmpopType;
+import org.python.antlr.base.cmpop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Is", base = AST.class)
+ at ExposedType(name = "_ast.Is", base = cmpop.class)
 public class Is extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Is.class);
 
diff --git a/src/org/python/antlr/op/IsNot.java b/src/org/python/antlr/op/IsNot.java
--- a/src/org/python/antlr/op/IsNot.java
+++ b/src/org/python/antlr/op/IsNot.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.cmpopType;
+import org.python.antlr.base.cmpop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.IsNot", base = AST.class)
+ at ExposedType(name = "_ast.IsNot", base = cmpop.class)
 public class IsNot extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(IsNot.class);
 
diff --git a/src/org/python/antlr/op/LShift.java b/src/org/python/antlr/op/LShift.java
--- a/src/org/python/antlr/op/LShift.java
+++ b/src/org/python/antlr/op/LShift.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.LShift", base = AST.class)
+ at ExposedType(name = "_ast.LShift", base = operator.class)
 public class LShift extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(LShift.class);
 
diff --git a/src/org/python/antlr/op/Load.java b/src/org/python/antlr/op/Load.java
--- a/src/org/python/antlr/op/Load.java
+++ b/src/org/python/antlr/op/Load.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.expr_contextType;
+import org.python.antlr.base.expr_context;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Load", base = AST.class)
+ at ExposedType(name = "_ast.Load", base = expr_context.class)
 public class Load extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Load.class);
 
diff --git a/src/org/python/antlr/op/Lt.java b/src/org/python/antlr/op/Lt.java
--- a/src/org/python/antlr/op/Lt.java
+++ b/src/org/python/antlr/op/Lt.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.cmpopType;
+import org.python.antlr.base.cmpop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Lt", base = AST.class)
+ at ExposedType(name = "_ast.Lt", base = cmpop.class)
 public class Lt extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Lt.class);
 
diff --git a/src/org/python/antlr/op/LtE.java b/src/org/python/antlr/op/LtE.java
--- a/src/org/python/antlr/op/LtE.java
+++ b/src/org/python/antlr/op/LtE.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.cmpopType;
+import org.python.antlr.base.cmpop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.LtE", base = AST.class)
+ at ExposedType(name = "_ast.LtE", base = cmpop.class)
 public class LtE extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(LtE.class);
 
diff --git a/src/org/python/antlr/op/Mod.java b/src/org/python/antlr/op/Mod.java
--- a/src/org/python/antlr/op/Mod.java
+++ b/src/org/python/antlr/op/Mod.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Mod", base = AST.class)
+ at ExposedType(name = "_ast.Mod", base = operator.class)
 public class Mod extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Mod.class);
 
diff --git a/src/org/python/antlr/op/Mult.java b/src/org/python/antlr/op/Mult.java
--- a/src/org/python/antlr/op/Mult.java
+++ b/src/org/python/antlr/op/Mult.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Mult", base = AST.class)
+ at ExposedType(name = "_ast.Mult", base = operator.class)
 public class Mult extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Mult.class);
 
diff --git a/src/org/python/antlr/op/Not.java b/src/org/python/antlr/op/Not.java
--- a/src/org/python/antlr/op/Not.java
+++ b/src/org/python/antlr/op/Not.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.unaryopType;
+import org.python.antlr.base.unaryop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Not", base = AST.class)
+ at ExposedType(name = "_ast.Not", base = unaryop.class)
 public class Not extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Not.class);
 
diff --git a/src/org/python/antlr/op/NotEq.java b/src/org/python/antlr/op/NotEq.java
--- a/src/org/python/antlr/op/NotEq.java
+++ b/src/org/python/antlr/op/NotEq.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.cmpopType;
+import org.python.antlr.base.cmpop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.NotEq", base = AST.class)
+ at ExposedType(name = "_ast.NotEq", base = cmpop.class)
 public class NotEq extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(NotEq.class);
 
diff --git a/src/org/python/antlr/op/NotIn.java b/src/org/python/antlr/op/NotIn.java
--- a/src/org/python/antlr/op/NotIn.java
+++ b/src/org/python/antlr/op/NotIn.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.cmpopType;
+import org.python.antlr.base.cmpop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.NotIn", base = AST.class)
+ at ExposedType(name = "_ast.NotIn", base = cmpop.class)
 public class NotIn extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(NotIn.class);
 
diff --git a/src/org/python/antlr/op/Or.java b/src/org/python/antlr/op/Or.java
--- a/src/org/python/antlr/op/Or.java
+++ b/src/org/python/antlr/op/Or.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.boolopType;
+import org.python.antlr.base.boolop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Or", base = AST.class)
+ at ExposedType(name = "_ast.Or", base = boolop.class)
 public class Or extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Or.class);
 
diff --git a/src/org/python/antlr/op/Param.java b/src/org/python/antlr/op/Param.java
--- a/src/org/python/antlr/op/Param.java
+++ b/src/org/python/antlr/op/Param.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.expr_contextType;
+import org.python.antlr.base.expr_context;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Param", base = AST.class)
+ at ExposedType(name = "_ast.Param", base = expr_context.class)
 public class Param extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Param.class);
 
diff --git a/src/org/python/antlr/op/Pow.java b/src/org/python/antlr/op/Pow.java
--- a/src/org/python/antlr/op/Pow.java
+++ b/src/org/python/antlr/op/Pow.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Pow", base = AST.class)
+ at ExposedType(name = "_ast.Pow", base = operator.class)
 public class Pow extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Pow.class);
 
diff --git a/src/org/python/antlr/op/RShift.java b/src/org/python/antlr/op/RShift.java
--- a/src/org/python/antlr/op/RShift.java
+++ b/src/org/python/antlr/op/RShift.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.RShift", base = AST.class)
+ at ExposedType(name = "_ast.RShift", base = operator.class)
 public class RShift extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(RShift.class);
 
diff --git a/src/org/python/antlr/op/Store.java b/src/org/python/antlr/op/Store.java
--- a/src/org/python/antlr/op/Store.java
+++ b/src/org/python/antlr/op/Store.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.expr_contextType;
+import org.python.antlr.base.expr_context;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Store", base = AST.class)
+ at ExposedType(name = "_ast.Store", base = expr_context.class)
 public class Store extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Store.class);
 
diff --git a/src/org/python/antlr/op/Sub.java b/src/org/python/antlr/op/Sub.java
--- a/src/org/python/antlr/op/Sub.java
+++ b/src/org/python/antlr/op/Sub.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.operatorType;
+import org.python.antlr.base.operator;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.Sub", base = AST.class)
+ at ExposedType(name = "_ast.Sub", base = operator.class)
 public class Sub extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(Sub.class);
 
diff --git a/src/org/python/antlr/op/UAdd.java b/src/org/python/antlr/op/UAdd.java
--- a/src/org/python/antlr/op/UAdd.java
+++ b/src/org/python/antlr/op/UAdd.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.unaryopType;
+import org.python.antlr.base.unaryop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.UAdd", base = AST.class)
+ at ExposedType(name = "_ast.UAdd", base = unaryop.class)
 public class UAdd extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(UAdd.class);
 
diff --git a/src/org/python/antlr/op/USub.java b/src/org/python/antlr/op/USub.java
--- a/src/org/python/antlr/op/USub.java
+++ b/src/org/python/antlr/op/USub.java
@@ -2,7 +2,7 @@
 package org.python.antlr.op;
 
 import org.python.antlr.AST;
-import org.python.antlr.ast.unaryopType;
+import org.python.antlr.base.unaryop;
 import org.python.antlr.PythonTree;
 import org.python.core.Py;
 import org.python.core.PyObject;
@@ -14,7 +14,7 @@
 import org.python.expose.ExposedSet;
 import org.python.expose.ExposedType;
 
- at ExposedType(name = "_ast.USub", base = AST.class)
+ at ExposedType(name = "_ast.USub", base = unaryop.class)
 public class USub extends PythonTree {
     public static final PyType TYPE = PyType.fromClass(USub.class);
 

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sat May 17 23:54:42 2014
From: jython-checkins at python.org (frank.wierzbicki)
Date: Sat, 17 May 2014 23:54:42 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Forgot_to_check_these_in_-_?=
	=?utf-8?q?unbreak_build_=3A=28?=
Message-ID: <3gWKyp1yGqz7LjZ@mail.python.org>

http://hg.python.org/jython/rev/96732491fdfc
changeset:   7254:96732491fdfc
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Sat May 17 21:54:35 2014 +0000
summary:
  Forgot to check these in - unbreak build :(

files:
  src/org/python/antlr/base/boolop.java       |  58 ++++++++++
  src/org/python/antlr/base/cmpop.java        |  58 ++++++++++
  src/org/python/antlr/base/expr_context.java |  58 ++++++++++
  src/org/python/antlr/base/operator.java     |  58 ++++++++++
  src/org/python/antlr/base/unaryop.java      |  58 ++++++++++
  5 files changed, 290 insertions(+), 0 deletions(-)


diff --git a/src/org/python/antlr/base/boolop.java b/src/org/python/antlr/base/boolop.java
new file mode 100644
--- /dev/null
+++ b/src/org/python/antlr/base/boolop.java
@@ -0,0 +1,58 @@
+// Hand copied from stmt.
+// XXX: autogenerate this.
+package org.python.antlr.base;
+import org.antlr.runtime.CommonToken;
+import org.antlr.runtime.Token;
+import org.python.antlr.AST;
+import org.python.antlr.PythonTree;
+import org.python.antlr.adapter.AstAdapters;
+import org.python.antlr.base.excepthandler;
+import org.python.antlr.base.expr;
+import org.python.antlr.base.mod;
+import org.python.antlr.base.slice;
+import org.python.antlr.base.stmt;
+import org.python.core.ArgParser;
+import org.python.core.AstList;
+import org.python.core.Py;
+import org.python.core.PyObject;
+import org.python.core.PyString;
+import org.python.core.PyStringMap;
+import org.python.core.PyType;
+import org.python.expose.ExposedGet;
+import org.python.expose.ExposedMethod;
+import org.python.expose.ExposedNew;
+import org.python.expose.ExposedSet;
+import org.python.expose.ExposedType;
+
+ at ExposedType(name = "_ast.boolop", base = AST.class)
+public abstract class boolop extends PythonTree {
+
+    public static final PyType TYPE = PyType.fromClass(boolop.class);
+    private final static PyString[] fields = new PyString[0];
+    @ExposedGet(name = "_fields")
+    public PyString[] get_fields() { return fields; }
+
+    private final static PyString[] attributes =
+    new PyString[] {new PyString("lineno"), new PyString("col_offset")};
+    @ExposedGet(name = "_attributes")
+    public PyString[] get_attributes() { return attributes; }
+
+    public boolop() {
+    }
+
+    public boolop(PyType subType) {
+    }
+
+    public boolop(int ttype, Token token) {
+        super(ttype, token);
+    }
+
+    public boolop(Token token) {
+        super(token);
+    }
+
+    public boolop(PythonTree node) {
+        super(node);
+    }
+
+}
diff --git a/src/org/python/antlr/base/cmpop.java b/src/org/python/antlr/base/cmpop.java
new file mode 100644
--- /dev/null
+++ b/src/org/python/antlr/base/cmpop.java
@@ -0,0 +1,58 @@
+// Hand copied from stmt.
+// XXX: autogenerate this.
+package org.python.antlr.base;
+import org.antlr.runtime.CommonToken;
+import org.antlr.runtime.Token;
+import org.python.antlr.AST;
+import org.python.antlr.PythonTree;
+import org.python.antlr.adapter.AstAdapters;
+import org.python.antlr.base.excepthandler;
+import org.python.antlr.base.expr;
+import org.python.antlr.base.mod;
+import org.python.antlr.base.slice;
+import org.python.antlr.base.stmt;
+import org.python.core.ArgParser;
+import org.python.core.AstList;
+import org.python.core.Py;
+import org.python.core.PyObject;
+import org.python.core.PyString;
+import org.python.core.PyStringMap;
+import org.python.core.PyType;
+import org.python.expose.ExposedGet;
+import org.python.expose.ExposedMethod;
+import org.python.expose.ExposedNew;
+import org.python.expose.ExposedSet;
+import org.python.expose.ExposedType;
+
+ at ExposedType(name = "_ast.cmpop", base = AST.class)
+public abstract class cmpop extends PythonTree {
+
+    public static final PyType TYPE = PyType.fromClass(cmpop.class);
+    private final static PyString[] fields = new PyString[0];
+    @ExposedGet(name = "_fields")
+    public PyString[] get_fields() { return fields; }
+
+    private final static PyString[] attributes =
+    new PyString[] {new PyString("lineno"), new PyString("col_offset")};
+    @ExposedGet(name = "_attributes")
+    public PyString[] get_attributes() { return attributes; }
+
+    public cmpop() {
+    }
+
+    public cmpop(PyType subType) {
+    }
+
+    public cmpop(int ttype, Token token) {
+        super(ttype, token);
+    }
+
+    public cmpop(Token token) {
+        super(token);
+    }
+
+    public cmpop(PythonTree node) {
+        super(node);
+    }
+
+}
diff --git a/src/org/python/antlr/base/expr_context.java b/src/org/python/antlr/base/expr_context.java
new file mode 100644
--- /dev/null
+++ b/src/org/python/antlr/base/expr_context.java
@@ -0,0 +1,58 @@
+// Hand copied from stmt.
+// XXX: autogenerate this.
+package org.python.antlr.base;
+import org.antlr.runtime.CommonToken;
+import org.antlr.runtime.Token;
+import org.python.antlr.AST;
+import org.python.antlr.PythonTree;
+import org.python.antlr.adapter.AstAdapters;
+import org.python.antlr.base.excepthandler;
+import org.python.antlr.base.expr;
+import org.python.antlr.base.mod;
+import org.python.antlr.base.slice;
+import org.python.antlr.base.stmt;
+import org.python.core.ArgParser;
+import org.python.core.AstList;
+import org.python.core.Py;
+import org.python.core.PyObject;
+import org.python.core.PyString;
+import org.python.core.PyStringMap;
+import org.python.core.PyType;
+import org.python.expose.ExposedGet;
+import org.python.expose.ExposedMethod;
+import org.python.expose.ExposedNew;
+import org.python.expose.ExposedSet;
+import org.python.expose.ExposedType;
+
+ at ExposedType(name = "_ast.expr_context", base = AST.class)
+public abstract class expr_context extends PythonTree {
+
+    public static final PyType TYPE = PyType.fromClass(expr_context.class);
+    private final static PyString[] fields = new PyString[0];
+    @ExposedGet(name = "_fields")
+    public PyString[] get_fields() { return fields; }
+
+    private final static PyString[] attributes =
+    new PyString[] {new PyString("lineno"), new PyString("col_offset")};
+    @ExposedGet(name = "_attributes")
+    public PyString[] get_attributes() { return attributes; }
+
+    public expr_context() {
+    }
+
+    public expr_context(PyType subType) {
+    }
+
+    public expr_context(int ttype, Token token) {
+        super(ttype, token);
+    }
+
+    public expr_context(Token token) {
+        super(token);
+    }
+
+    public expr_context(PythonTree node) {
+        super(node);
+    }
+
+}
diff --git a/src/org/python/antlr/base/operator.java b/src/org/python/antlr/base/operator.java
new file mode 100644
--- /dev/null
+++ b/src/org/python/antlr/base/operator.java
@@ -0,0 +1,58 @@
+// Hand copied from stmt.
+// XXX: autogenerate this.
+package org.python.antlr.base;
+import org.antlr.runtime.CommonToken;
+import org.antlr.runtime.Token;
+import org.python.antlr.AST;
+import org.python.antlr.PythonTree;
+import org.python.antlr.adapter.AstAdapters;
+import org.python.antlr.base.excepthandler;
+import org.python.antlr.base.expr;
+import org.python.antlr.base.mod;
+import org.python.antlr.base.slice;
+import org.python.antlr.base.stmt;
+import org.python.core.ArgParser;
+import org.python.core.AstList;
+import org.python.core.Py;
+import org.python.core.PyObject;
+import org.python.core.PyString;
+import org.python.core.PyStringMap;
+import org.python.core.PyType;
+import org.python.expose.ExposedGet;
+import org.python.expose.ExposedMethod;
+import org.python.expose.ExposedNew;
+import org.python.expose.ExposedSet;
+import org.python.expose.ExposedType;
+
+ at ExposedType(name = "_ast.operator", base = AST.class)
+public abstract class operator extends PythonTree {
+
+    public static final PyType TYPE = PyType.fromClass(operator.class);
+    private final static PyString[] fields = new PyString[0];
+    @ExposedGet(name = "_fields")
+    public PyString[] get_fields() { return fields; }
+
+    private final static PyString[] attributes =
+    new PyString[] {new PyString("lineno"), new PyString("col_offset")};
+    @ExposedGet(name = "_attributes")
+    public PyString[] get_attributes() { return attributes; }
+
+    public operator() {
+    }
+
+    public operator(PyType subType) {
+    }
+
+    public operator(int ttype, Token token) {
+        super(ttype, token);
+    }
+
+    public operator(Token token) {
+        super(token);
+    }
+
+    public operator(PythonTree node) {
+        super(node);
+    }
+
+}
diff --git a/src/org/python/antlr/base/unaryop.java b/src/org/python/antlr/base/unaryop.java
new file mode 100644
--- /dev/null
+++ b/src/org/python/antlr/base/unaryop.java
@@ -0,0 +1,58 @@
+// Hand copied from stmt.
+// XXX: autogenerate this.
+package org.python.antlr.base;
+import org.antlr.runtime.CommonToken;
+import org.antlr.runtime.Token;
+import org.python.antlr.AST;
+import org.python.antlr.PythonTree;
+import org.python.antlr.adapter.AstAdapters;
+import org.python.antlr.base.excepthandler;
+import org.python.antlr.base.expr;
+import org.python.antlr.base.mod;
+import org.python.antlr.base.slice;
+import org.python.antlr.base.stmt;
+import org.python.core.ArgParser;
+import org.python.core.AstList;
+import org.python.core.Py;
+import org.python.core.PyObject;
+import org.python.core.PyString;
+import org.python.core.PyStringMap;
+import org.python.core.PyType;
+import org.python.expose.ExposedGet;
+import org.python.expose.ExposedMethod;
+import org.python.expose.ExposedNew;
+import org.python.expose.ExposedSet;
+import org.python.expose.ExposedType;
+
+ at ExposedType(name = "_ast.unaryop", base = AST.class)
+public abstract class unaryop extends PythonTree {
+
+    public static final PyType TYPE = PyType.fromClass(unaryop.class);
+    private final static PyString[] fields = new PyString[0];
+    @ExposedGet(name = "_fields")
+    public PyString[] get_fields() { return fields; }
+
+    private final static PyString[] attributes =
+    new PyString[] {new PyString("lineno"), new PyString("col_offset")};
+    @ExposedGet(name = "_attributes")
+    public PyString[] get_attributes() { return attributes; }
+
+    public unaryop() {
+    }
+
+    public unaryop(PyType subType) {
+    }
+
+    public unaryop(int ttype, Token token) {
+        super(ttype, token);
+    }
+
+    public unaryop(Token token) {
+        super(token);
+    }
+
+    public unaryop(PythonTree node) {
+        super(node);
+    }
+
+}

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sun May 18 02:46:29 2014
From: jython-checkins at python.org (jim.baker)
Date: Sun, 18 May 2014 02:46:29 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Backed_out_JNR_and_JFFI_upg?=
 =?utf-8?q?rade_in_7235=3A01460e803ef3=2C_because_this_was_causing?=
Message-ID: <3gWPn12lMhz7LjM@mail.python.org>

http://hg.python.org/jython/rev/05622c95c3d7
changeset:   7255:05622c95c3d7
user:        Jim Baker <jim.baker at rackspace.com>
date:        Sat May 17 18:46:28 2014 -0600
summary:
  Backed out JNR and JFFI upgrade in 7235:01460e803ef3, because this was causing problems on Windows

files:
  build.xml                                            |   58 +++++++--
  extlibs/jffi-1.2.6.jar                               |  Bin 
  extlibs/jffi-1.2.7-native.jar                        |  Bin 
  extlibs/jffi-1.2.7.jar                               |  Bin 
  extlibs/jffi-Darwin.jar                              |  Bin 
  extlibs/jffi-arm-Linux.jar                           |  Bin 
  extlibs/jffi-i386-FreeBSD.jar                        |  Bin 
  extlibs/jffi-i386-Linux.jar                          |  Bin 
  extlibs/jffi-i386-OpenBSD.jar                        |  Bin 
  extlibs/jffi-i386-SunOS.jar                          |  Bin 
  extlibs/jffi-i386-Windows.jar                        |  Bin 
  extlibs/jffi-ppc-AIX.jar                             |  Bin 
  extlibs/jffi-ppc-Linux.jar                           |  Bin 
  extlibs/jffi-ppc64-Linux.jar                         |  Bin 
  extlibs/jffi-s390x-Linux.jar                         |  Bin 
  extlibs/jffi-sparc-SunOS.jar                         |  Bin 
  extlibs/jffi-sparcv9-SunOS.jar                       |  Bin 
  extlibs/jffi-x86_64-FreeBSD.jar                      |  Bin 
  extlibs/jffi-x86_64-Linux.jar                        |  Bin 
  extlibs/jffi-x86_64-OpenBSD.jar                      |  Bin 
  extlibs/jffi-x86_64-SunOS.jar                        |  Bin 
  extlibs/jffi-x86_64-Windows.jar                      |  Bin 
  extlibs/jnr-constants-0.8.4.jar                      |  Bin 
  extlibs/jnr-constants-0.8.5.jar                      |  Bin 
  extlibs/jnr-ffi-0.7.10.jar                           |  Bin 
  extlibs/jnr-ffi-1.0.10.jar                           |  Bin 
  extlibs/jnr-netdb-1.1.1.jar                          |  Bin 
  extlibs/jnr-netdb-1.1.2.jar                          |  Bin 
  extlibs/jnr-posix-2.4.0.jar                          |  Bin 
  extlibs/jnr-posix-3.0.1.jar                          |  Bin 
  src/org/python/modules/posix/PythonPOSIXHandler.java |    8 +-
  31 files changed, 47 insertions(+), 19 deletions(-)


diff --git a/build.xml b/build.xml
--- a/build.xml
+++ b/build.xml
@@ -154,12 +154,29 @@
             <pathelement path="${extlibs.dir}/asm-util-4.0.jar" />
             <pathelement path="${extlibs.dir}/guava-17.0.jar" />
             <pathelement path="${extlibs.dir}/icu4j-52_1.jar" />
-            <pathelement path="${extlibs.dir}/jffi-1.2.7.jar"/>
-            <pathelement path="${extlibs.dir}/jffi-1.2.7-native.jar"/>
-            <pathelement path="${extlibs.dir}/jnr-constants-0.8.5.jar"/>
-            <pathelement path="${extlibs.dir}/jnr-ffi-1.0.10.jar"/>
-            <pathelement path="${extlibs.dir}/jnr-netdb-1.1.2.jar"/>
-            <pathelement path="${extlibs.dir}/jnr-posix-3.0.1.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-arm-Linux.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-Darwin.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-i386-FreeBSD.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-i386-Linux.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-i386-OpenBSD.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-i386-SunOS.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-i386-Windows.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-ppc-AIX.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-ppc-Linux.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-ppc64-Linux.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-s390x-Linux.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-sparc-SunOS.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-sparcv9-SunOS.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-x86_64-FreeBSD.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-x86_64-Linux.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-x86_64-OpenBSD.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-x86_64-SunOS.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-x86_64-Windows.jar"/>
+            <pathelement path="${extlibs.dir}/jffi-1.2.6.jar"/>
+            <pathelement path="${extlibs.dir}/jnr-ffi-0.7.10.jar"/>
+            <pathelement path="${extlibs.dir}/jnr-netdb-1.1.1.jar"/>
+            <pathelement path="${extlibs.dir}/jnr-posix-2.4.0.jar"/>
+            <pathelement path="${extlibs.dir}/jnr-constants-0.8.4.jar"/>
             <pathelement path="${extlibs.dir}/jline-1.0.jar" />
         </path>
 
@@ -576,12 +593,29 @@
             <rule pattern="io.netty.**" result="org.python.netty. at 1"/>
             <zipfileset src="extlibs/netty-transport-4.0.18.Final.jar"/>
             <rule pattern="io.netty.**" result="org.python.netty. at 1"/>
-            <zipfileset src="${extlibs.dir}/jffi-1.2.7.jar"/>
-            <zipfileset src="${extlibs.dir}/jffi-1.2.7-native.jar"/>
-            <zipfileset src="${extlibs.dir}/jnr-constants-0.8.5.jar"/>
-            <zipfileset src="${extlibs.dir}/jnr-ffi-1.0.10.jar"/>
-	    <zipfileset src="${extlibs.dir}/jnr-netdb-1.1.2.jar"/>
-	    <zipfileset src="${extlibs.dir}/jnr-posix-3.0.1.jar"/>
+            <zipfileset src="extlibs/jffi-arm-Linux.jar"/>
+            <zipfileset src="extlibs/jffi-Darwin.jar"/>
+            <zipfileset src="extlibs/jffi-i386-FreeBSD.jar"/>
+            <zipfileset src="extlibs/jffi-i386-Linux.jar"/>
+            <zipfileset src="extlibs/jffi-i386-OpenBSD.jar"/>
+            <zipfileset src="extlibs/jffi-i386-SunOS.jar"/>
+            <zipfileset src="extlibs/jffi-i386-Windows.jar"/>
+            <zipfileset src="extlibs/jffi-ppc-AIX.jar"/>
+            <zipfileset src="extlibs/jffi-ppc-Linux.jar"/>
+            <zipfileset src="extlibs/jffi-ppc64-Linux.jar"/>
+            <zipfileset src="extlibs/jffi-s390x-Linux.jar"/>
+            <zipfileset src="extlibs/jffi-sparc-SunOS.jar"/>
+            <zipfileset src="extlibs/jffi-sparcv9-SunOS.jar"/>
+            <zipfileset src="extlibs/jffi-x86_64-FreeBSD.jar"/>
+            <zipfileset src="extlibs/jffi-x86_64-Linux.jar"/>
+            <zipfileset src="extlibs/jffi-x86_64-OpenBSD.jar"/>
+            <zipfileset src="extlibs/jffi-x86_64-SunOS.jar"/>
+            <zipfileset src="extlibs/jffi-x86_64-Windows.jar"/>
+            <zipfileset src="extlibs/jffi-1.2.6.jar"/>
+            <zipfileset src="${extlibs.dir}/jnr-ffi-0.7.10.jar"/>
+	    <zipfileset src="${extlibs.dir}/jnr-netdb-1.1.1.jar"/>
+	    <zipfileset src="${extlibs.dir}/jnr-posix-2.4.0.jar"/>
+            <zipfileset src="${extlibs.dir}/jnr-constants-0.8.4.jar"/>
             <zipfileset src="extlibs/xml-apis-2.11.0.jar" excludes="META-INF/services/*"/>
             <zipfileset src="extlibs/xercesImpl-2.11.0.jar" excludes="META-INF/services/*"/>
             <rule pattern="org.apache.xml.**" result="org.python.apache.xml. at 1"/>
diff --git a/extlibs/jffi-1.2.6.jar b/extlibs/jffi-1.2.6.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..a6b2be6c39798f0fc5958aa309924506b74f0f9f
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-1.2.7-native.jar b/extlibs/jffi-1.2.7-native.jar
deleted file mode 100644
index 0b7404c256d1d4277b58c9c559bf24938f11404e..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-1.2.7.jar b/extlibs/jffi-1.2.7.jar
deleted file mode 100644
index 68c8db05c248ab54aba86d03d3c7904334468770..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-Darwin.jar b/extlibs/jffi-Darwin.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..009191ac8c0dcaf63267ea0103e73ea2a9895e43
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-arm-Linux.jar b/extlibs/jffi-arm-Linux.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..b3c5d977c1b7ad3fc196bfe2cd24fe56129dc8f9
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-i386-FreeBSD.jar b/extlibs/jffi-i386-FreeBSD.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8235a3484884243b1d17d5e19340e4166f3b0b55
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-i386-Linux.jar b/extlibs/jffi-i386-Linux.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..7e23b400cce5e3b4861141ee4f10730fbcbdb5de
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-i386-OpenBSD.jar b/extlibs/jffi-i386-OpenBSD.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8235a3484884243b1d17d5e19340e4166f3b0b55
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-i386-SunOS.jar b/extlibs/jffi-i386-SunOS.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..68bcacc39ac49a01929be37ec67c6784612bc966
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-i386-Windows.jar b/extlibs/jffi-i386-Windows.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..90a6baecf76870212dc4f8cb7aa56e0962d1f858
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-ppc-AIX.jar b/extlibs/jffi-ppc-AIX.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8235a3484884243b1d17d5e19340e4166f3b0b55
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-ppc-Linux.jar b/extlibs/jffi-ppc-Linux.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8235a3484884243b1d17d5e19340e4166f3b0b55
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-ppc64-Linux.jar b/extlibs/jffi-ppc64-Linux.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8235a3484884243b1d17d5e19340e4166f3b0b55
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-s390x-Linux.jar b/extlibs/jffi-s390x-Linux.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8235a3484884243b1d17d5e19340e4166f3b0b55
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-sparc-SunOS.jar b/extlibs/jffi-sparc-SunOS.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8235a3484884243b1d17d5e19340e4166f3b0b55
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-sparcv9-SunOS.jar b/extlibs/jffi-sparcv9-SunOS.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..306788c0810baeccb93f191316844d5d0d9e6254
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-x86_64-FreeBSD.jar b/extlibs/jffi-x86_64-FreeBSD.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..32824a7e209b57e06808d4de65de4443225a5349
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-x86_64-Linux.jar b/extlibs/jffi-x86_64-Linux.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..99deb014d458c5a42b5290832b8141acf42212dd
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-x86_64-OpenBSD.jar b/extlibs/jffi-x86_64-OpenBSD.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..8235a3484884243b1d17d5e19340e4166f3b0b55
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-x86_64-SunOS.jar b/extlibs/jffi-x86_64-SunOS.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..b8240422d442b4bcf0e95d6b6a0a02714aeac827
GIT binary patch
[stripped]
diff --git a/extlibs/jffi-x86_64-Windows.jar b/extlibs/jffi-x86_64-Windows.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..667e48db8193809b21118cff14b23184d558821a
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-constants-0.8.4.jar b/extlibs/jnr-constants-0.8.4.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..f0ff6bb1a1796560c09121b4366a5d04032156af
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-constants-0.8.5.jar b/extlibs/jnr-constants-0.8.5.jar
deleted file mode 100644
index fe11c5752485174311ae271f4ca8ce76a496ff63..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-ffi-0.7.10.jar b/extlibs/jnr-ffi-0.7.10.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..9e2e64f276334bed30c62d12823d5088b5672425
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-ffi-1.0.10.jar b/extlibs/jnr-ffi-1.0.10.jar
deleted file mode 100644
index 0e90154fcbf11a4ce289ec5b93cfe975cee6fa50..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-netdb-1.1.1.jar b/extlibs/jnr-netdb-1.1.1.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..731f6ec0fdc5c420ec55edec6c28092b06c70e32
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-netdb-1.1.2.jar b/extlibs/jnr-netdb-1.1.2.jar
deleted file mode 100644
index d082053606b4718b13648a1ca17c0ffc43eb66a6..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-posix-2.4.0.jar b/extlibs/jnr-posix-2.4.0.jar
new file mode 100644
index e69de29bb2d1d6434b8b29ae775ad8c2e48c5391..36f3fc1d217091941a1079e9ebccc70094a5ede6
GIT binary patch
[stripped]
diff --git a/extlibs/jnr-posix-3.0.1.jar b/extlibs/jnr-posix-3.0.1.jar
deleted file mode 100644
index 94f9bf3a7d19baf0071e12eb24f14a0d0cef6c0b..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
GIT binary patch
[stripped]
diff --git a/src/org/python/modules/posix/PythonPOSIXHandler.java b/src/org/python/modules/posix/PythonPOSIXHandler.java
--- a/src/org/python/modules/posix/PythonPOSIXHandler.java
+++ b/src/org/python/modules/posix/PythonPOSIXHandler.java
@@ -20,13 +20,7 @@
  */
 public class PythonPOSIXHandler implements POSIXHandler {
 
-    public void error(Errno error, String extraData) {
-        // XXX: extraData (filename) could have been unicode!
-        // http://bugs.jython.org/issue1825
-        throw Py.OSError(error, Py.newString(extraData));
-    }
-
-    public void error(Errno error, String methodName, String extraData) {
+	public void error(Errno error, String extraData) {
         // XXX: extraData (filename) could have been unicode!
         // http://bugs.jython.org/issue1825
         throw Py.OSError(error, Py.newString(extraData));

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Sun May 18 04:47:27 2014
From: jython-checkins at python.org (jim.baker)
Date: Sun, 18 May 2014 04:47:27 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Re-apply_overloaded_PythonP?=
 =?utf-8?q?OSIXHandler=23error_from_7235=3A01460e803ef3?=
Message-ID: <3gWSSb06vfz7Ljb@mail.python.org>

http://hg.python.org/jython/rev/8ceb0723b372
changeset:   7256:8ceb0723b372
user:        Jim Baker <jim.baker at rackspace.com>
date:        Sat May 17 19:10:18 2014 -0600
summary:
  Re-apply overloaded PythonPOSIXHandler#error from 7235:01460e803ef3

files:
  src/org/python/modules/posix/PythonPOSIXHandler.java |  8 +++++++-
  1 files changed, 7 insertions(+), 1 deletions(-)


diff --git a/src/org/python/modules/posix/PythonPOSIXHandler.java b/src/org/python/modules/posix/PythonPOSIXHandler.java
--- a/src/org/python/modules/posix/PythonPOSIXHandler.java
+++ b/src/org/python/modules/posix/PythonPOSIXHandler.java
@@ -20,7 +20,13 @@
  */
 public class PythonPOSIXHandler implements POSIXHandler {
 
-	public void error(Errno error, String extraData) {
+    public void error(Errno error, String extraData) {
+        // XXX: extraData (filename) could have been unicode!
+        // http://bugs.jython.org/issue1825
+        throw Py.OSError(error, Py.newString(extraData));
+    }
+
+    public void error(Errno error, String methodName, String extraData) {
         // XXX: extraData (filename) could have been unicode!
         // http://bugs.jython.org/issue1825
         throw Py.OSError(error, Py.newString(extraData));

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Mon May 19 00:49:33 2014
From: jython-checkins at python.org (jeff.allen)
Date: Mon, 19 May 2014 00:49:33 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Fix_imaginary_padding_probl?=
 =?utf-8?q?em_in_complex=2E=5F=5Fformat=5F=5F=2C_and_some_javadoc=2E?=
Message-ID: <3gWz7d065lzQPM@mail.python.org>

http://hg.python.org/jython/rev/805a6138224c
changeset:   7257:805a6138224c
parent:      7220:61ef07321554
user:        Jeff Allen <ja.py at farowl.co.uk>
date:        Sun May 04 16:56:16 2014 +0100
summary:
  Fix imaginary padding problem in complex.__format__, and some javadoc.
format(4j, "*>6") is '****4j' not '4j'. Cleaned up javadoc in stringlib.InternalFormat.

files:
  src/org/python/core/PyComplex.java                    |  24 +++--
  src/org/python/core/stringlib/InternalFormat.java     |  37 ++++-----
  src/org/python/core/stringlib/InternalFormatSpec.java |   6 +-
  3 files changed, 34 insertions(+), 33 deletions(-)


diff --git a/src/org/python/core/PyComplex.java b/src/org/python/core/PyComplex.java
--- a/src/org/python/core/PyComplex.java
+++ b/src/org/python/core/PyComplex.java
@@ -165,9 +165,10 @@
      * and <code>__repr__</code>, and none-format.
      * <p>
      * In general, the output is surrounded in parentheses, like <code>"(12.34+24.68j)"</code>.
-     * However, if the real part is zero, only the imaginary part is printed, and without
-     * parentheses like <code>"24.68j"</code>. The number format specification passed in is used
-     * without padding to width, for the real and imaginary parts individually.
+     * However, if the real part is zero (but not negative zero), only the imaginary part is
+     * printed, and without parentheses like <code>"24.68j"</code>. The number format specification
+     * passed in is used for the real and imaginary parts individually, with padding to width
+     * afterwards (if the specification requires it).
      *
      * @param spec parsed format specification string
      * @return formatted value
@@ -176,12 +177,13 @@
         FloatFormatter f = new FloatFormatter(spec, 2, 3); // Two elements + "(j)".length
         // Even in r-format, complex strips *all* the trailing zeros.
         f.setMinFracDigits(0);
-        if (real == 0.0) {
+        if (Double.doubleToLongBits(real) == 0L) {
+            // Real part is truly zero: show no real part.
             f.format(imag).append('j');
         } else {
-            f.append('(').format(real).format(imag, "+").append("j)").pad();
+            f.append('(').format(real).format(imag, "+").append("j)");
         }
-        return f.getResult();
+        return f.pad().getResult();
     }
 
     @Override
@@ -823,7 +825,7 @@
         try {
             String specString = formatSpecStr.getString();
             Spec spec = InternalFormat.fromText(specString);
-            if (spec.type!=Spec.NONE && "efgEFGn%".indexOf(spec.type) < 0) {
+            if (spec.type != Spec.NONE && "efgEFGn%".indexOf(spec.type) < 0) {
                 throw FloatFormatter.unknownFormat(spec.type, "complex");
             } else if (spec.alternate) {
                 throw FloatFormatter.alternateFormNotAllowed("complex");
@@ -838,12 +840,12 @@
                     // And then we use the __str__ mechanism to get parentheses or real 0 elision.
                     result = formatComplex(spec);
                 } else {
-                    // In any other format, the defaults those commonly used for numeric formats.
+                    // In any other format, defaults are those commonly used for numeric formats.
                     spec = spec.withDefaults(Spec.NUMERIC);
                     FloatFormatter f = new FloatFormatter(spec, 2, 1);// 2 floats + "j"
-                    // Convert as both parts per specification
-                    f.format(real).format(imag, "+").append('j').pad();
-                    result = f.getResult();
+                    // Convert both parts as per specification
+                    f.format(real).format(imag, "+").append('j');
+                    result = f.pad().getResult();
                 }
             }
         } catch (IllegalArgumentException e) {
diff --git a/src/org/python/core/stringlib/InternalFormat.java b/src/org/python/core/stringlib/InternalFormat.java
--- a/src/org/python/core/stringlib/InternalFormat.java
+++ b/src/org/python/core/stringlib/InternalFormat.java
@@ -108,7 +108,7 @@
          * method shows a '|' character between each section when it prints out the buffer. Override
          * this when you define more lengths in the subclass.
          *
-         * @return
+         * @return the lengths of the successive sections
          */
         protected int[] sectionLengths() {
             return new int[] {lenSign, lenWhole};
@@ -265,7 +265,6 @@
          * result you probably don't want. It is up to the client to disallow this (which
          * <code>complex</code> does).
          *
-         * @param value to pad
          * @return this object
          */
         public Formatter pad() {
@@ -468,8 +467,8 @@
      * during the construction of a new <code>Spec</code>, for attributes that are unspecified in a
      * primary source.
      * <p>
-     * This structure is returned by factory method {@link #fromText(CharSequence)}, and having
-     * public final members is freely accessed by formatters such as {@link FloatBuilder}, and the
+     * This structure is returned by factory method {@link #fromText(String)}, and having public
+     * final members is freely accessed by formatters such as {@link FloatFormatter}, and the
      * __format__ methods of client object types.
      * <p>
      * The fields correspond to the elements of a format specification. The grammar of a format
@@ -482,28 +481,28 @@
      * A typical idiom is:
      *
      * <pre>
-     *     private static final InternalFormatSpec FLOAT_DEFAULT = InternalFormatSpec.from(">");
+     *     private static final InternalFormatSpec FLOAT_DEFAULTS = InternalFormatSpec.from(">");
      *     ...
-     *         InternalFormatSpec spec = InternalFormatSpec.from(specString, FLOAT_DEFAULT);
+     *         InternalFormat.Spec spec = InternalFormat.fromText(specString);
+     *         spec = spec.withDefaults(FLOAT_DEFAULTS);
      *         ... // Validation of spec.type, and other attributes, for this type.
-     *         FloatBuilder buf = new FloatBuilder(spec);
-     *         buf.format(value);
-     *         String result = buf.getResult();
+     *         FloatFormatter f = new FloatFormatter(spec);
+     *         String result = f.format(value).getResult();
      *
      * </pre>
      */
     public static class Spec {
 
-        /** The fill character specified, or '\uffff' if unspecified. */
+        /** The fill character specified, or U+FFFF if unspecified. */
         public final char fill;
         /**
-         * Alignment indicator is one of {<code>'&lt;', '^', '>', '='</code>, or '\uffff' if
+         * Alignment indicator is one of {<code>'&lt;', '^', '>', '='</code>, or U+FFFF if
          * unspecified.
          */
         public final char align;
         /**
          * Sign-handling flag, one of <code>'+'</code>, <code>'-'</code>, or <code>' '</code>, or
-         * '\uffff' if unspecified.
+         * U+FFFF if unspecified.
          */
         public final char sign;
         /** The alternative format flag '#' was given. */
@@ -514,7 +513,7 @@
         public final boolean grouping;
         /** Precision decoded from the format, or -1 if unspecified. */
         public final int precision;
-        /** Type key from the format, or '\uffff' if unspecified. */
+        /** Type key from the format, or U+FFFF if unspecified. */
         public final char type;
 
         /** Non-character code point used to represent "no value" in <code>char</code> attributes. */
@@ -604,17 +603,17 @@
         }
 
         /**
-         * Return a merged <code>Spec</code> object, in which any attribute of this object, that is
-         * specified (or <code>true</code>) has the same value in the result, and any attribute of
-         * this object that is unspecified (or <code>false</code>) has the value that attribute
-         * takes in the other object. This the second object supplies default values. (These
+         * Return a merged <code>Spec</code> object, in which any attribute of this object that is
+         * specified (or <code>true</code>), has the same value in the result, and any attribute of
+         * this object that is unspecified (or <code>false</code>), has the value that attribute
+         * takes in the other object. Thus the second object supplies default values. (These
          * defaults may also be unspecified.) The use of this method is to allow a <code>Spec</code>
          * constructed from text to record exactly, and only, what was in the textual specification,
          * while the __format__ method of a client object supplies its type-specific defaults. Thus
          * "20" means "<20s" to a <code>str</code>, ">20.12" to a <code>float</code> and ">20.12g"
          * to a <code>complex</code>.
          *
-         * @param defaults to merge where this object does not specify the attribute.
+         * @param other defaults to merge where this object does not specify the attribute.
          * @return a new Spec object.
          */
         public Spec withDefaults(Spec other) {
@@ -646,7 +645,7 @@
          * @param precision (e.g. decimal places)
          * @param type indicator character
          */
-        public Spec(int width, int precision, char type) {
+        public Spec(int precision, char type) {
             this(' ', '>', Spec.NONE, false, UNSPECIFIED, false, precision, type);
         }
 
diff --git a/src/org/python/core/stringlib/InternalFormatSpec.java b/src/org/python/core/stringlib/InternalFormatSpec.java
--- a/src/org/python/core/stringlib/InternalFormatSpec.java
+++ b/src/org/python/core/stringlib/InternalFormatSpec.java
@@ -7,8 +7,8 @@
  * to a string assumed to be the result of formatting to the given precision.
  * <p>
  * This structure is returned by {@link InternalFormatSpecParser#parse()} and having public members
- * is freely used by {@link InternalFormatSpecParser}, {@link Formatter} and the __format__ methods
- * of client object types.
+ * is freely used by {@link InternalFormatSpecParser}, and the __format__ methods of client object
+ * types.
  * <p>
  * The fields correspond to the elements of a format specification. The grammar of a format
  * specification is:
@@ -45,7 +45,7 @@
      * @param defaultAlign to use if <code>this.align</code>=0 (one of <code>'&lt;'</code>,
      *            <code>'^'</code>, <code>'>'</code>, or <code>'='</code>).
      * @param leaveWidth to reduce effective <code>this.width</code> by
-     * @return
+     * @return padded value
      */
     public String pad(String value, char defaultAlign, int leaveWidth) {
 

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Mon May 19 00:49:34 2014
From: jython-checkins at python.org (jeff.allen)
Date: Mon, 19 May 2014 00:49:34 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Comment_the_=25-formatting_?=
 =?utf-8?q?mechanism_in_PyString=2Ejava=2E?=
Message-ID: <3gWz7f5HyJz7Ljd@mail.python.org>

http://hg.python.org/jython/rev/cfecb7862f21
changeset:   7258:cfecb7862f21
user:        Jeff Allen <ja.py at farowl.co.uk>
date:        Sun May 04 11:37:55 2014 +0100
summary:
  Comment the %-formatting mechanism in PyString.java.
No significant change to code. This is to aid comprehension of intended changes.

files:
  src/org/python/core/PyString.java |  256 ++++++++++++++++-
  1 files changed, 233 insertions(+), 23 deletions(-)


diff --git a/src/org/python/core/PyString.java b/src/org/python/core/PyString.java
--- a/src/org/python/core/PyString.java
+++ b/src/org/python/core/PyString.java
@@ -1296,8 +1296,8 @@
      * @param stripChars characters to strip from the left end of this str/bytes, or null
      * @return a new String, stripped of the specified characters/bytes
      */
-    public String lstrip(String sep) {
-        return _lstrip(sep);
+    public String lstrip(String stripChars) {
+        return _lstrip(stripChars);
     }
 
     /**
@@ -1310,8 +1310,8 @@
      * @return a new <code>PyString</code> (or {@link PyUnicode}), stripped of the specified
      *         characters/bytes
      */
-    public PyObject lstrip(PyObject sep) {
-        return str_lstrip(sep);
+    public PyObject lstrip(PyObject stripChars) {
+        return str_lstrip(stripChars);
     }
 
     @ExposedMethod(defaults = "null", doc = BuiltinDocs.str_lstrip_doc)
@@ -1385,8 +1385,8 @@
      * @param stripChars characters to strip from either end of this str/bytes, or null
      * @return a new String, stripped of the specified characters/bytes
      */
-    public String rstrip(String sep) {
-        return _rstrip(sep);
+    public String rstrip(String stripChars) {
+        return _rstrip(stripChars);
     }
 
     /**
@@ -1399,8 +1399,8 @@
      * @return a new <code>PyString</code> (or {@link PyUnicode}), stripped of the specified
      *         characters/bytes
      */
-    public PyObject rstrip(PyObject sep) {
-        return str_rstrip(sep);
+    public PyObject rstrip(PyObject stripChars) {
+        return str_rstrip(stripChars);
     }
 
     @ExposedMethod(defaults = "null", doc = BuiltinDocs.str_rstrip_doc)
@@ -1546,7 +1546,7 @@
      * the last element of the list contains the what is left over after the last split.
      * <p>
      * Implementation note: although a str contains only bytes, this method is also called by
-     * {@link PyUnicode#unicode_split(PyObject)}.
+     * {@link PyUnicode#unicode_split(PyObject, int)}.
      *
      * @param sep string to use as separator (or <code>null</code> if to split on whitespace)
      * @param maxsplit maximum number of splits to make (there may be <code>maxsplit+1</code>
@@ -1798,7 +1798,7 @@
      * left over after the last split.
      * <p>
      * Implementation note: although a str contains only bytes, this method is also called by
-     * {@link PyUnicode#unicode_rsplit(PyObject)} .
+     * {@link PyUnicode#unicode_rsplit(PyObject, int)} .
      *
      * @param sep string to use as separator (or <code>null</code> if to split on whitespace)
      * @param maxsplit maximum number of splits to make (there may be <code>maxsplit+1</code>
@@ -2931,11 +2931,10 @@
      *
      * @param oldPiece to replace where found.
      * @param newPiece replacement text.
-     * @param count maximum number of replacements to make, or -1 meaning all of them.
      * @return PyString (or PyUnicode if any string is one), this string after replacements.
      */
-    public PyString replace(PyObject oldPieceObj, PyObject newPieceObj) {
-        return str_replace(oldPieceObj, newPieceObj, -1);
+    public PyString replace(PyObject oldPiece, PyObject newPiece) {
+        return str_replace(oldPiece, newPiece, -1);
     }
 
     /**
@@ -2949,8 +2948,8 @@
      * @param count maximum number of replacements to make, or -1 meaning all of them.
      * @return PyString (or PyUnicode if any string is one), this string after replacements.
      */
-    public PyString replace(PyObject oldPieceObj, PyObject newPieceObj, int count) {
-        return str_replace(oldPieceObj, newPieceObj, count);
+    public PyString replace(PyObject oldPiece, PyObject newPiece, int count) {
+        return str_replace(oldPiece, newPiece, count);
     }
 
     @ExposedMethod(defaults = "-1", doc = BuiltinDocs.str_replace_doc)
@@ -3161,8 +3160,8 @@
      * @return <code>true</code> if this string slice starts with a specified prefix, otherwise
      *         <code>false</code>.
      */
-    public boolean startswith(PyObject prefix, PyObject offset) {
-        return str_startswith(prefix, offset, null);
+    public boolean startswith(PyObject prefix, PyObject start) {
+        return str_startswith(prefix, start, null);
     }
 
     /**
@@ -4002,13 +4001,24 @@
  */
 final class StringFormatter {
 
+    /** Index into {@link #format} being interpreted. */
     int index;
+    /** Format being interpreted. */
     String format;
+    /** Where the output is built. */
     StringBuilder buffer;
+    /** Remembers that the value currently converted is negative */
     boolean negative;
+    /** Precision from format specification. */
     int precision;
+    /**
+     * Index into args of argument currently being worked, or special values indicating -1: a single
+     * item that has not yet been used, -2: a single item that has already been used, -3: a mapping.
+     */
     int argIndex;
+    /** Arguments supplied to {@link #format(PyObject)} method. */
     PyObject args;
+    /** Indicate a <code>PyUnicode</code> result is expected. */
     boolean unicodeCoercion;
 
     final char pop() {
@@ -4027,6 +4037,11 @@
         index--;
     }
 
+    /**
+     * Initialise the interpreter with the given format string, ready for {@link #format(PyObject)}.
+     *
+     * @param format string to interpret
+     */
     public StringFormatter(String format) {
         this(format, false);
     }
@@ -4044,6 +4059,10 @@
         buffer = new StringBuilder(format.length() + 100);
     }
 
+    /**
+     * Read the next object from the argument list, taking special values of <code>argIndex</code>
+     * into account.
+     */
     PyObject getarg() {
         PyObject ret = null;
         switch (argIndex) {
@@ -4064,6 +4083,10 @@
         return ret;
     }
 
+    /**
+     * Parse a number from the format, except if the next thing is "*", read it from the argument
+     * list.
+     */
     int getNumber() {
         char c = pop();
         if (c == '*') {
@@ -4093,7 +4116,24 @@
 
     }
 
+    /**
+     * Format the argument interpreted as a long, using the argument's <code>__str__</code>,
+     * <code>__oct__</code>, or <code>__hex__</code> method according to <code>type</code>. If v is
+     * being treated as signed, the sign of v is transferred to {@link #negative} and the absolute
+     * value is converted. The <code>altFlag</code> argument controls the appearance of a "0x" or
+     * "0X" prefix in the hex case, or a "0" prefix in the octal case. The hexadecimal case, the
+     * case of characters and digits will match the type ('x' meaning lowercase, 'X' meaning
+     * uppercase).
+     *
+     * @param arg to convert
+     * @param type one of 'o' for octal, 'x' or 'X' for hex, anything else calls
+     *            <code>arg.__str__</code>.
+     * @param altFlag if true there will be a prefix
+     * @return converted value as <code>String</code>
+     */
     private String formatLong(PyObject arg, char type, boolean altFlag) {
+        // Convert using the appropriate type
+        // XXX Results in behaviour divergent from CPython when any of the methods is overridden.
         PyString argAsString;
         switch (type) {
             case 'o':
@@ -4107,29 +4147,37 @@
                 argAsString = arg.__str__();
                 break;
         }
+
         checkPrecision("long");
         String s = argAsString.toString();
         int end = s.length();
         int ptr = 0;
 
+        // In the hex case, the __hex__ return starts 0x
+        // XXX (we assume, perhaps falsely)
         int numnondigits = 0;
         if (type == 'x' || type == 'X') {
             numnondigits = 2;
         }
 
+        // Strip a "long" indicator
         if (s.endsWith("L")) {
             end--;
         }
 
+        // Strip a possible sign to member negative
         negative = s.charAt(0) == '-';
         if (negative) {
             ptr++;
         }
 
+        // The formatted number is s[ptr:end] and starts with numnondigits non-digits.
         int numdigits = end - numnondigits - ptr;
         if (!altFlag) {
+            // We should have no "base tag" '0' or "0x" on the front.
             switch (type) {
                 case 'o':
+                    // Strip the '0'
                     if (numdigits > 1) {
                         ++ptr;
                         --numdigits;
@@ -4137,27 +4185,36 @@
                     break;
                 case 'x':
                 case 'X':
+                    // Strip the "0x"
                     ptr += 2;
                     numnondigits -= 2;
                     break;
             }
         }
+
+        // If necessary, add leading zeros to the numerical digits part.
         if (precision > numdigits) {
+            // Recompose the formatted number in this buffer
             StringBuilder buf = new StringBuilder();
+            // The base indicator prefix
             for (int i = 0; i < numnondigits; ++i) {
                 buf.append(s.charAt(ptr++));
             }
+            // The extra zeros
             for (int i = 0; i < precision - numdigits; i++) {
                 buf.append('0');
             }
+            // The previously known digits
             for (int i = 0; i < numdigits; i++) {
                 buf.append(s.charAt(ptr++));
             }
             s = buf.toString();
         } else if (end < s.length() || ptr > 0) {
+            // It's only necessary to extract the formatted number from s
             s = s.substring(ptr, end);
         }
 
+        // And finally, deal with the case, so it matches x or X.
         switch (type) {
             case 'X':
                 s = s.toUpperCase();
@@ -4167,10 +4224,16 @@
     }
 
     /**
-     * Formats arg as an integer, with the specified radix
+     * Formats arg as an integer, with the specified radix. The integer value is obtained from the
+     * result of <code>arg.__int__()</code>. <code>type</code> and <code>altFlag</code> are passed
+     * to {@link #formatLong(PyObject, char, boolean)} in case the result is a PyLong.
      *
-     * type and altFlag are needed to be passed to {@link #formatLong(PyObject, char, boolean)} in
-     * case the result of <code>arg.__int__()</code> is a PyLong.
+     * @param arg to convert
+     * @param radix in which to express <code>arg</code>
+     * @param unsigned true if required to interpret a 32-bit integer as unsigned ('u' legacy?).
+     * @param type of conversion ('d', 'o', 'x', or 'X')
+     * @param altFlag '#' present in format (causes "0x" prefix in hex, and '0' prefix in octal)
+     * @return string form of the value
      */
     private String formatInteger(PyObject arg, int radix, boolean unsigned, char type,
             boolean altFlag) {
@@ -4202,25 +4265,44 @@
             }
         }
         if (argAsInt instanceof PyInteger) {
+            // This call does not provide the prefix and will be lowercase.
             return formatInteger(((PyInteger)argAsInt).getValue(), radix, unsigned);
         } else { // must be a PyLong (as per __int__ contract)
+            // This call provides the base prefix and case-matches with 'x' or 'X'.
             return formatLong(argAsInt, type, altFlag);
         }
     }
 
+    /**
+     * Convert a 32-bit integer (as from a {@link PyInteger}) to characters, signed or unsigned. The
+     * values is presented in a <code>long</code>. The string result is left-padded with zeros to
+     * the stated {@link #precision}. If v is being treated as signed, the sign of v is transferred
+     * to {@link #negative} and the absolute value is converted. Otherwise (unsigned case)
+     * <code>0x100000000L + v</code> is converted. This method does not provide the '0' or "0x"
+     * prefix, just the padded digit string.
+     *
+     * @param v value to convert
+     * @param radix of conversion
+     * @param unsigned if should be treated as unsigned
+     * @return string form
+     */
     private String formatInteger(long v, int radix, boolean unsigned) {
         checkPrecision("integer");
         if (unsigned) {
+            // If the high bit was set, this will have been sign-extended: correct that.
             if (v < 0) {
                 v = 0x100000000l + v;
             }
         } else {
+            // If the high bit was set, the sign extension was correct, but we need sign + abs(v).
             if (v < 0) {
                 negative = true;
                 v = -v;
             }
         }
+        // Use the method in java.lang.Long (lowercase, no prefix)
         String s = Long.toString(v, radix);
+        // But zero pad to the requested precision
         while (s.length() < precision) {
             s = "0" + s;
         }
@@ -4308,15 +4390,23 @@
         return buf.toString();
     }
 
+    /**
+     * Main service of this class: format one or more arguments with the format string supplied at
+     * construction.
+     *
+     * @param args tuple or map containing objects, or a single object, to convert
+     * @return result of formatting
+     */
     @SuppressWarnings("fallthrough")
     public PyString format(PyObject args) {
         PyObject dict = null;
         this.args = args;
         boolean needUnicode = unicodeCoercion;
         if (args instanceof PyTuple) {
+            // We will simply work through the tuple elements
             argIndex = 0;
         } else {
-            // special index indicating a single item rather than a tuple
+            // Not a tuple, but possibly still some kind of container: use special argIndex values.
             argIndex = -1;
             if (args instanceof PyDictionary || args instanceof PyStringMap
                     || (!(args instanceof PySequence) && args.__findattr__("__getitem__") != null)) {
@@ -4326,6 +4416,8 @@
         }
 
         while (index < format.length()) {
+
+            // Attributes to be parsed from the next format specifier
             boolean ljustFlag = false;
             boolean signFlag = false;
             boolean blankFlag = false;
@@ -4335,16 +4427,31 @@
             int width = -1;
             precision = -1;
 
+            // Read one character from the format string
             char c = pop();
             if (c != '%') {
                 buffer.append(c);
                 continue;
             }
+
+            // It's a %, so the beginning of a conversion specifier. Parse it.
+
+            // A conversion specifier contains the following components, in this order:
+            // + The '%' character, which marks the start of the specifier.
+            // + Mapping key (optional), consisting of a parenthesised sequence of characters.
+            // + Conversion flags (optional), which affect the result of some conversion types.
+            // + Minimum field width (optional), or an '*' (asterisk).
+            // + Precision (optional), given as a '.' (dot) followed by the precision or '*'.
+            // + Length modifier (optional).
+            // + Conversion type.
+
             c = pop();
             if (c == '(') {
+                // Mapping key, consisting of a parenthesised sequence of characters.
                 if (dict == null) {
                     throw Py.TypeError("format requires a mapping");
                 }
+                // Scan along until a matching close parenthesis is found
                 int parens = 1;
                 int keyStart = index;
                 while (parens > 0) {
@@ -4355,11 +4462,16 @@
                         parens++;
                     }
                 }
+                // Last c=pop() is the closing ')' while indexKey is just after the opening '('
                 String tmp = format.substring(keyStart, index - 1);
+                // Look it up using this extent as the (right type of) key.
                 this.args = dict.__getitem__(needUnicode ? new PyUnicode(tmp) : new PyString(tmp));
             } else {
+                // Not a mapping key: next clause will re-read c.
                 push();
             }
+
+            // Conversion flags (optional) that affect the result of some conversion types.
             while (true) {
                 switch (c = pop()) {
                     case '-':
@@ -4380,43 +4492,77 @@
                 }
                 break;
             }
+            // Push back c as next clause will re-read c.
             push();
+
+            /*
+             * Minimum field width (optional). If specified as an '*' (asterisk), the actual width
+             * is read from the next element of the tuple in values, and the object to convert comes
+             * after the minimum field width and optional precision. A custom getNumber() takes care
+             * of the '*' case.
+             */
             width = getNumber();
             if (width < 0) {
                 width = -width;
                 ljustFlag = true;
             }
+
+            /*
+             * Precision (optional), given as a '.' (dot) followed by the precision. If specified as
+             * '*' (an asterisk), the actual precision is read from the next element of the tuple in
+             * values, and the value to convert comes after the precision. A custom getNumber()
+             * takes care of the '*' case.
+             */
             c = pop();
             if (c == '.') {
                 precision = getNumber();
                 if (precision < -1) {
                     precision = 0;
                 }
-
                 c = pop();
             }
+
+            // Length modifier (optional). (Compatibility feature?) It has no effect.
             if (c == 'h' || c == 'l' || c == 'L') {
                 c = pop();
             }
+
+            // c is now the conversion type.
             if (c == '%') {
+                // It was just a percent sign after all
                 buffer.append(c);
                 continue;
             }
+
+            /*
+             * Process argument according to format specification decoded from the string. It is
+             * important we don't read the argumnent from the list until this point because of the
+             * possibility that width and precision were specified via the argument list.
+             */
             PyObject arg = getarg();
-            char fill = ' ';
             String string = null;
             negative = false;
+
+            // Independent of type, decide the padding character based on decoded flags.
+            char fill = ' ';
             if (zeroFlag) {
                 fill = '0';
             } else {
                 fill = ' ';
             }
+
+            // Perform the type-specific formatting
             switch (c) {
+
                 case 's':
+                    // String (converts any Python object using str()).
                     if (arg instanceof PyUnicode) {
                         needUnicode = true;
                     }
+                    // fall through ...
+
                 case 'r':
+                    // String (converts any Python object using repr()).
                     fill = ' ';
                     if (c == 's') {
                         if (needUnicode) {
@@ -4432,15 +4578,19 @@
                     }
 
                     break;
+
                 case 'i':
                 case 'd':
+                    // Signed integer decimal. Note floats accepted.
                     if (arg instanceof PyLong) {
                         string = formatLong(arg, c, altFlag);
                     } else {
                         string = formatInteger(arg, 10, false, c, altFlag);
                     }
                     break;
+
                 case 'u':
+                    // Obsolete type ? it is identical to 'd'. (Why not identical here?)
                     if (arg instanceof PyLong) {
                         string = formatLong(arg, c, altFlag);
                     } else if (arg instanceof PyInteger || arg instanceof PyFloat) {
@@ -4449,10 +4599,15 @@
                         throw Py.TypeError("int argument required");
                     }
                     break;
+
                 case 'o':
+                    // Signed octal value. Note floats accepted.
                     if (arg instanceof PyLong) {
+                        // This call provides the base prefix '0' if altFlag.
                         string = formatLong(arg, c, altFlag);
                     } else if (arg instanceof PyInteger || arg instanceof PyFloat) {
+                        // This call does not provide the '0' prefix and will be lowercase ...
+                        // ... except where arg.__int__ returns PyLong, then it's like formatLong.
                         string = formatInteger(arg, 8, false, c, altFlag);
                         if (altFlag && string.charAt(0) != '0') {
                             string = "0" + string;
@@ -4461,10 +4616,15 @@
                         throw Py.TypeError("int argument required");
                     }
                     break;
+
                 case 'x':
+                    // Signed hexadecimal (lowercase). Note floats accepted.
                     if (arg instanceof PyLong) {
+                        // This call provides the base prefix "0x" if altFlag and case-matches c.
                         string = formatLong(arg, c, altFlag);
                     } else if (arg instanceof PyInteger || arg instanceof PyFloat) {
+                        // This call does not provide the "0x" prefix and will be lowercase.
+                        // ... except where arg.__int__ returns PyLong, then it's like formatLong.
                         string = formatInteger(arg, 16, false, c, altFlag);
                         string = string.toLowerCase();
                         if (altFlag) {
@@ -4474,10 +4634,15 @@
                         throw Py.TypeError("int argument required");
                     }
                     break;
+
                 case 'X':
+                    // Signed hexadecimal (uppercase). Note floats accepted.
                     if (arg instanceof PyLong) {
+                        // This call provides the base prefix "0x" if altFlag and case-matches c.
                         string = formatLong(arg, c, altFlag);
                     } else if (arg instanceof PyInteger || arg instanceof PyFloat) {
+                        // This call does not provide the "0x" prefix and will be lowercase.
+                        // ... except where arg.__int__ returns PyLong, then it's like formatLong.
                         string = formatInteger(arg, 16, false, c, altFlag);
                         string = string.toUpperCase();
                         if (altFlag) {
@@ -4487,22 +4652,28 @@
                         throw Py.TypeError("int argument required");
                     }
                     break;
+
                 case 'e':
                 case 'E':
+                    // Floating point exponential format (+case).
                     string = formatFloatExponential(arg, c, false);
                     if (c == 'E') {
                         string = string.toUpperCase();
                     }
                     break;
+
                 case 'f':
                 case 'F':
+                    // Floating point decimal format (+case). Note ints accepted.
                     string = formatFloatDecimal(asDouble(arg), false);
                     if (c == 'F') {
                         string = string.toUpperCase();
                     }
                     break;
+
                 case 'g':
                 case 'G':
+                    // Value-adaptive floating point format (+case). Note ints accepted.
                     int origPrecision = precision;
                     if (precision == -1) {
                         precision = 6;
@@ -4540,7 +4711,9 @@
                         string = string.toUpperCase();
                     }
                     break;
+
                 case 'c':
+                    // Single character (accepts integer or single character string).
                     fill = ' ';
                     if (arg instanceof PyString) {
                         string = ((PyString)arg).toString();
@@ -4552,6 +4725,8 @@
                         }
                         break;
                     }
+
+                    // arg is not a str (or unicode)
                     int val;
                     try {
                         // Explicitly __int__ so we can look for an AttributeError (which is
@@ -4563,6 +4738,7 @@
                         }
                         throw e;
                     }
+                    // Range check, according to ultimate type of result as presentl;y known.
                     if (!needUnicode) {
                         if (val < 0) {
                             throw Py.OverflowError("unsigned byte integer is less than minimum");
@@ -4580,8 +4756,15 @@
                             + codecs.encode(Py.newString(c), null, "replace") + "' (0x"
                             + Integer.toHexString(c) + ") at index " + (index - 1));
             }
+
+            /*
+             * We have now dealt with the translation of the (absolute value of the) argument, in
+             * variable string[]. In the next sections we deal with sign, padding and base prefix.
+             */
             int length = string.length();
             int skip = 0;
+
+            // Decide how to represent the sign according to format and actual sign of argument.
             String signString = null;
             if (negative) {
                 signString = "-";
@@ -4593,34 +4776,47 @@
                 }
             }
 
+            // The width (from here on) will be the remaining width on the line.
             if (width < length) {
                 width = length;
             }
+
+            // Insert the sign in the buffer and adjust the width.
             if (signString != null) {
                 if (fill != ' ') {
+                    // When the fill is not space, the sign comes before the fill.
                     buffer.append(signString);
                 }
+                // Adjust width for sign.
                 if (width > length) {
                     width--;
                 }
             }
+
+            // Insert base prefix used with alternate mode for hexadecimal.
             if (altFlag && (c == 'x' || c == 'X')) {
                 if (fill != ' ') {
+                    // When the fill is not space, this base prefix comes before the fill.
                     buffer.append('0');
                     buffer.append(c);
                     skip += 2;
                 }
+                // Adjust width for base prefix.
                 width -= 2;
                 if (width < 0) {
                     width = 0;
                 }
                 length -= 2;
             }
+
+            // Fill on the left of the item.
             if (width > length && !ljustFlag) {
                 do {
                     buffer.append(fill);
                 } while (--width > length);
             }
+
+            // If the fill is spaces, we will have deferred the sign and hex base prefix
             if (fill == ' ') {
                 if (signString != null) {
                     buffer.append(signString);
@@ -4631,19 +4827,33 @@
                     skip += 2;
                 }
             }
+
+            // Now append the converted argument.
             if (skip > 0) {
+                // The string contains a hex-prefix, but we have already inserted one.
                 buffer.append(string.substring(skip));
             } else {
                 buffer.append(string);
             }
 
+            // If this hasn't filled the space required, add right-padding.
             while (--width >= length) {
                 buffer.append(' ');
             }
         }
+
+        /*
+         * All fields in the format string have been used to convert arguments (or used the argument
+         * as a width, etc.). This had better not leave any arguments unused. Note argIndex is an
+         * index into args or has a special value. If args is a 'proper' index, It should now be out
+         * of range; if a special value, it would be wrong if it were -1, indicating a single item
+         * that has not yet been used.
+         */
         if (argIndex == -1 || (argIndex >= 0 && args.__finditem__(argIndex) != null)) {
             throw Py.TypeError("not all arguments converted during string formatting");
         }
+
+        // Return the final buffer contents as a str or unicode as appropriate.
         if (needUnicode) {
             return new PyUnicode(buffer);
         }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Mon May 19 00:49:36 2014
From: jython-checkins at python.org (jeff.allen)
Date: Mon, 19 May 2014 00:49:36 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Fixes_for_=25-formatting_fl?=
	=?utf-8?q?oats=2E?=
Message-ID: <3gWz7h1sQqz7Ljc@mail.python.org>

http://hg.python.org/jython/rev/60838278e668
changeset:   7259:60838278e668
user:        Jeff Allen <ja.py at farowl.co.uk>
date:        Thu May 08 21:23:39 2014 +0100
summary:
  Fixes for %-formatting floats.
Replaces the float rendering in PyString StringFormatter with that used for float.__format__(). Skips removed from test_float. Some change to FloatFormatter to support alternative mode in %#e and %#f.

files:
  Lib/test/test_float.py                            |   40 --
  Lib/test/test_float_jy.py                         |    5 +-
  src/org/python/core/PyString.java                 |  149 +--------
  src/org/python/core/stringlib/FloatFormatter.java |   91 ++++-
  4 files changed, 83 insertions(+), 202 deletions(-)


diff --git a/Lib/test/test_float.py b/Lib/test/test_float.py
--- a/Lib/test/test_float.py
+++ b/Lib/test/test_float.py
@@ -617,8 +617,6 @@
         self.assertEqual('{0:f}'.format(NAN), 'nan')
         self.assertEqual('{0:F}'.format(NAN), 'NAN')
 
-    @unittest.skipIf(test_support.is_jython,
-                     "FIXME: not working on Jython")
     @requires_IEEE_754
     def test_format_testfile(self):
         with open(format_testfile) as testfile:
@@ -842,8 +840,6 @@
         self.assertAlmostEqual(round(1.5e22, -22), 2e22)
 
 
-    @unittest.skipIf(test_support.is_jython,
-                     "FIXME: %-formatting specials imperfect in Jython")
     @requires_IEEE_754
     def test_format_specials(self):
         # Test formatting of nans and infs.
@@ -878,42 +874,6 @@
             test(sfmt, NAN, ' nan')
             test(sfmt, -NAN, ' nan')
 
-    @requires_IEEE_754
-    def test_format_specials_jy(self):
-        # Test formatting of nans and infs (suppressing %-formatting).
-        # This is just a crudely restricted copy of test_format_specials.
-        # Delete this test when we no longer have to skip test_format_specials.
-
-        def test(fmt, value, expected):
-            # Test with only format().
-            #self.assertEqual(fmt % value, expected, fmt)
-            if not '#' in fmt:
-                # Until issue 7094 is implemented, format() for floats doesn't
-                #  support '#' formatting
-                fmt = fmt[1:] # strip off the %
-                self.assertEqual(format(value, fmt), expected, fmt)
-
-        for fmt in ['%e', '%f', '%g', '%.0e', '%.6f', '%.20g',
-                    '%#e', '%#f', '%#g', '%#.20e', '%#.15f', '%#.3g']:
-            pfmt = '%+' + fmt[1:]
-            sfmt = '% ' + fmt[1:]
-            test(fmt, INF, 'inf')
-            test(fmt, -INF, '-inf')
-            test(fmt, NAN, 'nan')
-            test(fmt, -NAN, 'nan')
-            # When asking for a sign, it's always provided. nans are
-            #  always positive.
-            test(pfmt, INF, '+inf')
-            test(pfmt, -INF, '-inf')
-            test(pfmt, NAN, '+nan')
-            test(pfmt, -NAN, '+nan')
-            # When using ' ' for a sign code, only infs can be negative.
-            #  Others have a space.
-            test(sfmt, INF, ' inf')
-            test(sfmt, -INF, '-inf')
-            test(sfmt, NAN, ' nan')
-            test(sfmt, -NAN, ' nan')
-
 
 # Beginning with Python 2.6 float has cross platform compatible
 # ways to create and represent inf and nan
diff --git a/Lib/test/test_float_jy.py b/Lib/test/test_float_jy.py
--- a/Lib/test/test_float_jy.py
+++ b/Lib/test/test_float_jy.py
@@ -40,11 +40,8 @@
 
     def test_float_str_formatting(self):
         self.assertEqual('%.13g' % 12345678.00005, '12345678.00005')
-        self.assertEqual('%.12g' % 12345678.00005,
-                         jython and '12345678' or '12345678.0001')
+        self.assertEqual('%.12g' % 12345678.00005, '12345678.0001')
         self.assertEqual('%.11g' % 12345678.00005, '12345678')
-        # XXX: The exponential formatter isn't totally correct, e.g. our
-        # output here is really .13g
         self.assertEqual('%.12g' % math.pi**-100, '1.92758141606e-50')
         self.assertEqual('%.5g' % 123.005, '123')
         self.assertEqual('%#.5g' % 123.005, '123.00')
diff --git a/src/org/python/core/PyString.java b/src/org/python/core/PyString.java
--- a/src/org/python/core/PyString.java
+++ b/src/org/python/core/PyString.java
@@ -4,17 +4,15 @@
 import java.lang.ref.Reference;
 import java.lang.ref.SoftReference;
 import java.math.BigInteger;
-import java.text.DecimalFormat;
-import java.text.DecimalFormatSymbols;
-
-import org.python.core.StringFormatter.DecimalFormatTemplate;
+
 import org.python.core.buffer.BaseBuffer;
 import org.python.core.buffer.SimpleStringBuffer;
 import org.python.core.stringlib.FieldNameIterator;
+import org.python.core.stringlib.FloatFormatter;
+import org.python.core.stringlib.InternalFormat.Spec;
 import org.python.core.stringlib.InternalFormatSpec;
 import org.python.core.stringlib.InternalFormatSpecParser;
 import org.python.core.stringlib.MarkupIterator;
-import org.python.core.util.ExtraMath;
 import org.python.core.util.StringUtil;
 import org.python.expose.ExposedMethod;
 import org.python.expose.ExposedNew;
@@ -4246,7 +4244,6 @@
                 // safe to call __int__:
                 argAsInt = arg.__int__();
             } else {
-                // Same case noted on formatFloatDecimal:
                 // We can't simply call arg.__int__() because PyString implements
                 // it without exposing it to python (i.e, str instances has no
                 // __int__ attribute). So, we would support strings as arguments
@@ -4255,7 +4252,7 @@
                 try {
                     argAsInt = arg.__getattr__("__int__").__call__();
                 } catch (PyException e) {
-                    // XXX: Swallow customs AttributeError throws from __float__ methods
+                    // XXX: Swallow custom AttributeError throws from __int__ methods
                     // No better alternative for the moment
                     if (e.match(Py.AttributeError)) {
                         throw Py.TypeError("int argument required");
@@ -4317,79 +4314,6 @@
         }
     }
 
-    static class DecimalFormatTemplate {
-
-        static DecimalFormat template;
-        static {
-            template =
-                    new DecimalFormat("#,##0.#####", new DecimalFormatSymbols(java.util.Locale.US));
-            DecimalFormatSymbols symbols = template.getDecimalFormatSymbols();
-            symbols.setNaN("nan");
-            symbols.setInfinity("inf");
-            template.setDecimalFormatSymbols(symbols);
-            template.setGroupingUsed(false);
-        }
-    }
-
-    private static final DecimalFormat getDecimalFormat() {
-        return (DecimalFormat)DecimalFormatTemplate.template.clone();
-    }
-
-    private String formatFloatDecimal(double v, boolean truncate) {
-        checkPrecision("decimal");
-        int prec = precision;
-        if (prec == -1) {
-            prec = 6;
-        }
-        if (v < 0) {
-            v = -v;
-            negative = true;
-        }
-
-        DecimalFormat decimalFormat = getDecimalFormat();
-        decimalFormat.setMaximumFractionDigits(prec);
-        decimalFormat.setMinimumFractionDigits(truncate ? 0 : prec);
-
-        String ret = decimalFormat.format(v);
-        return ret;
-    }
-
-    private String formatFloatExponential(PyObject arg, char e, boolean truncate) {
-        StringBuilder buf = new StringBuilder();
-        double v = asDouble(arg);
-        boolean isNegative = false;
-        if (v < 0) {
-            v = -v;
-            isNegative = true;
-        }
-        double power = 0.0;
-        if (v > 0) {
-            power = ExtraMath.closeFloor(Math.log10(v));
-        }
-        // System.err.println("formatExp: "+v+", "+power);
-        int savePrecision = precision;
-        precision = 2;
-
-        String exp = formatInteger((long)power, 10, false);
-        if (negative) {
-            negative = false;
-            exp = '-' + exp;
-        } else {
-            exp = '+' + exp;
-        }
-
-        precision = savePrecision;
-
-        double base = v / Math.pow(10, power);
-        buf.append(formatFloatDecimal(base, truncate));
-        buf.append(e);
-
-        buf.append(exp);
-        negative = isNegative;
-
-        return buf.toString();
-    }
-
     /**
      * Main service of this class: format one or more arguments with the format string supplied at
      * construction.
@@ -4655,61 +4579,26 @@
 
                 case 'e':
                 case 'E':
-                    // Floating point exponential format (+case).
-                    string = formatFloatExponential(arg, c, false);
-                    if (c == 'E') {
-                        string = string.toUpperCase();
-                    }
-                    break;
-
                 case 'f':
                 case 'F':
-                    // Floating point decimal format (+case). Note ints accepted.
-                    string = formatFloatDecimal(asDouble(arg), false);
-                    if (c == 'F') {
-                        string = string.toUpperCase();
-                    }
-                    break;
-
                 case 'g':
                 case 'G':
-                    // Value-adaptive floating point format (+case). Note ints accepted.
-                    int origPrecision = precision;
-                    if (precision == -1) {
-                        precision = 6;
-                    }
-
+                    // All floating point formats (+case).
+
+                    // Convert the flags (local variables) to the form needed in the Spec object.
+                    char align = ljustFlag ? '<' : '>';
+                    char sign = signFlag ? '+' : (blankFlag ? ' ' : Spec.NONE);
+                    int w = Spec.UNSPECIFIED;
+                    Spec spec = new Spec(fill, align, sign, altFlag, w, false, precision, c);
+
+                    // Format using this Spec the double form of the argument.
+                    FloatFormatter f = new FloatFormatter(spec);
                     double v = asDouble(arg);
-                    int exponent = (int)ExtraMath.closeFloor(Math.log10(Math.abs(v == 0 ? 1 : v)));
-                    if (v == Double.POSITIVE_INFINITY) {
-                        string = "inf";
-                    } else if (v == Double.NEGATIVE_INFINITY) {
-                        string = "-inf";
-                    } else if (exponent >= -4 && exponent < precision) {
-                        precision -= exponent + 1;
-                        string = formatFloatDecimal(v, !altFlag);
-
-                        // XXX: this block may be unnecessary now
-                        if (altFlag && string.indexOf('.') == -1) {
-                            int zpad = origPrecision - string.length();
-                            string += '.';
-                            if (zpad > 0) {
-                                char zeros[] = new char[zpad];
-                                for (int ci = 0; ci < zpad; zeros[ci++] = '0') {}
-                                string += new String(zeros);
-                            }
-                        }
-                    } else {
-                        // Exponential precision is the number of digits after the decimal
-                        // point, whereas 'g' precision is the number of significant digits --
-                        // and exponential always provides one significant digit before the
-                        // decimal point
-                        precision--;
-                        string = formatFloatExponential(arg, (char)(c - 2), !altFlag);
-                    }
-                    if (c == 'G') {
-                        string = string.toUpperCase();
-                    }
+                    f.format(v);
+                    string = f.getResult();
+
+                    // Suppress subsequent attempts to insert a correct sign, done already.
+                    signFlag = blankFlag = negative = false;
                     break;
 
                 case 'c':
diff --git a/src/org/python/core/stringlib/FloatFormatter.java b/src/org/python/core/stringlib/FloatFormatter.java
--- a/src/org/python/core/stringlib/FloatFormatter.java
+++ b/src/org/python/core/stringlib/FloatFormatter.java
@@ -16,7 +16,7 @@
 public class FloatFormatter extends InternalFormat.Formatter {
 
     /** The rounding mode dominant in the formatter. */
-    static final RoundingMode ROUND_PY = RoundingMode.HALF_UP; // Believed to be HALF_EVEN in Py3k
+    static final RoundingMode ROUND_PY = RoundingMode.HALF_EVEN;
 
     /** If it contains no decimal point, this length is zero, and 1 otherwise. */
     private int lenPoint;
@@ -325,10 +325,9 @@
             exp = lenFraction - vv.scale();
         }
 
-        // Finally add zeros, as necessary, and stick on the exponent.
-
+        // If the result is not already complete, add point and zeros as necessary, and exponent.
         if (lenMarker == 0) {
-            appendTrailingZeros(precision);
+            ensurePointAndTrailingZeros(precision);
             appendExponent(exp);
         }
     }
@@ -345,14 +344,7 @@
      */
     private void format_f(double value, String positivePrefix, int precision) {
 
-        if (signAndSpecialNumber(value, positivePrefix)) {
-
-            if (lenMarker == 0) {
-                // May be 0 or -0 so we still need to ...
-                appendTrailingZeros(precision);
-            }
-
-        } else {
+        if (!signAndSpecialNumber(value, positivePrefix)) {
             // Convert value to decimal exactly. (This can be very long.)
             BigDecimal vLong = new BigDecimal(Math.abs(value));
 
@@ -366,9 +358,53 @@
                 // There is a decimal point and some digits following
                 lenWhole = result.length() - (start + lenSign + (lenPoint = 1) + lenFraction);
             } else {
+                // There are no fractional digits and so no decimal point
                 lenWhole = result.length() - (start + lenSign);
             }
+        }
 
+        // Finally, ensure we have all the fractional digits we should.
+        if (lenMarker == 0) {
+            ensurePointAndTrailingZeros(precision);
+        }
+    }
+
+    /**
+     * Append a decimal point and trailing fractional zeros if necessary for 'e' and 'f' format.
+     * This should not be called if the result is not numeric ("inf" for example). This method deals
+     * with the following complexities: on return there will be at least the number of fractional
+     * digits specified in the argument <code>n</code>, and at least {@link #minFracDigits};
+     * further, if <code>minFracDigits&lt;0</code>, signifying the "alternate mode" of certain
+     * formats, the method will ensure there is a decimal point, even if there are no fractional
+     * digits to follow.
+     *
+     * @param n smallest number of fractional digits on return
+     */
+    private void ensurePointAndTrailingZeros(int n) {
+
+        // Set n to the number of fractional digits we should have.
+        if (n < minFracDigits) {
+            n = minFracDigits;
+        }
+
+        // Do we have a decimal point already?
+        if (lenPoint == 0) {
+            // No decimal point: add one if there will be any fractional digits or
+            if (n > 0 || minFracDigits < 0) {
+                // First need to add a decimal point.
+                result.append('.');
+                lenPoint = 1;
+            }
+        }
+
+        // Do we have enough fractional digits already?
+        int f = lenFraction;
+        if (n > f) {
+            // Make up the required number of zeros.
+            for (; f < n; f++) {
+                result.append('0');
+            }
+            lenFraction = f;
         }
     }
 
@@ -635,7 +671,7 @@
 
         // In no-truncate mode, the fraction is full precision. Otherwise trim it.
         if (minFracDigits >= 0) {
-            // Note minFracDigits only applies to fixed formats.
+            // Note positive minFracDigits only applies to fixed formats.
             removeTrailingZeros(0);
         }
 
@@ -757,16 +793,16 @@
 
     /**
      * Append the trailing fractional zeros, as required by certain formats, so that the total
-     * number of fractional digits is no less than specified. If <code>minFracDigits&lt;=0</code>,
-     * the method leaves the {@link #result} buffer unchanged.
+     * number of fractional digits is no less than specified. If <code>n&lt;=0</code>, the method
+     * leaves the {@link #result} buffer unchanged.
      *
-     * @param minFracDigits smallest number of fractional digits on return
+     * @param n smallest number of fractional digits on return
      */
-    private void appendTrailingZeros(int minFracDigits) {
+    private void appendTrailingZeros(int n) {
 
         int f = lenFraction;
 
-        if (minFracDigits > f) {
+        if (n > f) {
             if (lenPoint == 0) {
                 // First need to add a decimal point. (Implies lenFraction=0.)
                 result.append('.');
@@ -774,7 +810,7 @@
             }
 
             // Now make up the required number of zeros.
-            for (; f < minFracDigits; f++) {
+            for (; f < n; f++) {
                 result.append('0');
             }
             lenFraction = f;
@@ -787,9 +823,9 @@
      * originally (and therefore no fractional part), the method will add a decimal point, even if
      * it adds no zeros.
      *
-     * @param minFracDigits smallest number of fractional digits on return
+     * @param n smallest number of fractional digits on return
      */
-    private void appendPointAndTrailingZeros(int minFracDigits) {
+    private void appendPointAndTrailingZeros(int n) {
 
         if (lenPoint == 0) {
             // First need to add a decimal point. (Implies lenFraction=0.)
@@ -799,7 +835,7 @@
 
         // Now make up the required number of zeros.
         int f;
-        for (f = lenFraction; f < minFracDigits; f++) {
+        for (f = lenFraction; f < n; f++) {
             result.append('0');
         }
         lenFraction = f;
@@ -810,18 +846,17 @@
      * least the number of fractional digits specified. If the resultant number of fractional digits
      * is zero, this method will also remove the trailing decimal point (if there is one).
      *
-     * @param minFracDigits smallest number of fractional digits on return
+     * @param n smallest number of fractional digits on return
      */
-    private void removeTrailingZeros(int minFracDigits) {
-
-        int f = lenFraction;
+    private void removeTrailingZeros(int n) {
 
         if (lenPoint > 0) {
             // There's a decimal point at least, and there may be some fractional digits.
-            if (minFracDigits == 0 || f > minFracDigits) {
+            int f = lenFraction;
+            if (n == 0 || f > n) {
 
                 int fracStart = result.length() - f;
-                for (; f > minFracDigits; --f) {
+                for (; f > n; --f) {
                     if (result.charAt(fracStart - 1 + f) != '0') {
                         // Keeping this one as it isn't a zero
                         break;

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Mon May 19 00:49:38 2014
From: jython-checkins at python.org (jeff.allen)
Date: Mon, 19 May 2014 00:49:38 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython_=28merge_default_-=3E_default?=
 =?utf-8?q?=29=3A_Merge_of_formatting_changes?=
Message-ID: <3gWz7k1MTFz7Ljj@mail.python.org>

http://hg.python.org/jython/rev/f6768976582a
changeset:   7260:f6768976582a
parent:      7256:8ceb0723b372
parent:      7259:60838278e668
user:        Jeff Allen <ja.py at farowl.co.uk>
date:        Sun May 18 20:11:04 2014 +0100
summary:
  Merge of formatting changes

files:
  Lib/test/test_float.py                                |   40 -
  Lib/test/test_float_jy.py                             |    5 +-
  src/org/python/core/PyComplex.java                    |   24 +-
  src/org/python/core/PyString.java                     |  395 ++++++---
  src/org/python/core/stringlib/FloatFormatter.java     |   91 +-
  src/org/python/core/stringlib/InternalFormat.java     |   37 +-
  src/org/python/core/stringlib/InternalFormatSpec.java |    6 +-
  7 files changed, 345 insertions(+), 253 deletions(-)


diff --git a/Lib/test/test_float.py b/Lib/test/test_float.py
--- a/Lib/test/test_float.py
+++ b/Lib/test/test_float.py
@@ -617,8 +617,6 @@
         self.assertEqual('{0:f}'.format(NAN), 'nan')
         self.assertEqual('{0:F}'.format(NAN), 'NAN')
 
-    @unittest.skipIf(test_support.is_jython,
-                     "FIXME: not working on Jython")
     @requires_IEEE_754
     def test_format_testfile(self):
         with open(format_testfile) as testfile:
@@ -842,8 +840,6 @@
         self.assertAlmostEqual(round(1.5e22, -22), 2e22)
 
 
-    @unittest.skipIf(test_support.is_jython,
-                     "FIXME: %-formatting specials imperfect in Jython")
     @requires_IEEE_754
     def test_format_specials(self):
         # Test formatting of nans and infs.
@@ -878,42 +874,6 @@
             test(sfmt, NAN, ' nan')
             test(sfmt, -NAN, ' nan')
 
-    @requires_IEEE_754
-    def test_format_specials_jy(self):
-        # Test formatting of nans and infs (suppressing %-formatting).
-        # This is just a crudely restricted copy of test_format_specials.
-        # Delete this test when we no longer have to skip test_format_specials.
-
-        def test(fmt, value, expected):
-            # Test with only format().
-            #self.assertEqual(fmt % value, expected, fmt)
-            if not '#' in fmt:
-                # Until issue 7094 is implemented, format() for floats doesn't
-                #  support '#' formatting
-                fmt = fmt[1:] # strip off the %
-                self.assertEqual(format(value, fmt), expected, fmt)
-
-        for fmt in ['%e', '%f', '%g', '%.0e', '%.6f', '%.20g',
-                    '%#e', '%#f', '%#g', '%#.20e', '%#.15f', '%#.3g']:
-            pfmt = '%+' + fmt[1:]
-            sfmt = '% ' + fmt[1:]
-            test(fmt, INF, 'inf')
-            test(fmt, -INF, '-inf')
-            test(fmt, NAN, 'nan')
-            test(fmt, -NAN, 'nan')
-            # When asking for a sign, it's always provided. nans are
-            #  always positive.
-            test(pfmt, INF, '+inf')
-            test(pfmt, -INF, '-inf')
-            test(pfmt, NAN, '+nan')
-            test(pfmt, -NAN, '+nan')
-            # When using ' ' for a sign code, only infs can be negative.
-            #  Others have a space.
-            test(sfmt, INF, ' inf')
-            test(sfmt, -INF, '-inf')
-            test(sfmt, NAN, ' nan')
-            test(sfmt, -NAN, ' nan')
-
 
 # Beginning with Python 2.6 float has cross platform compatible
 # ways to create and represent inf and nan
diff --git a/Lib/test/test_float_jy.py b/Lib/test/test_float_jy.py
--- a/Lib/test/test_float_jy.py
+++ b/Lib/test/test_float_jy.py
@@ -40,11 +40,8 @@
 
     def test_float_str_formatting(self):
         self.assertEqual('%.13g' % 12345678.00005, '12345678.00005')
-        self.assertEqual('%.12g' % 12345678.00005,
-                         jython and '12345678' or '12345678.0001')
+        self.assertEqual('%.12g' % 12345678.00005, '12345678.0001')
         self.assertEqual('%.11g' % 12345678.00005, '12345678')
-        # XXX: The exponential formatter isn't totally correct, e.g. our
-        # output here is really .13g
         self.assertEqual('%.12g' % math.pi**-100, '1.92758141606e-50')
         self.assertEqual('%.5g' % 123.005, '123')
         self.assertEqual('%#.5g' % 123.005, '123.00')
diff --git a/src/org/python/core/PyComplex.java b/src/org/python/core/PyComplex.java
--- a/src/org/python/core/PyComplex.java
+++ b/src/org/python/core/PyComplex.java
@@ -165,9 +165,10 @@
      * and <code>__repr__</code>, and none-format.
      * <p>
      * In general, the output is surrounded in parentheses, like <code>"(12.34+24.68j)"</code>.
-     * However, if the real part is zero, only the imaginary part is printed, and without
-     * parentheses like <code>"24.68j"</code>. The number format specification passed in is used
-     * without padding to width, for the real and imaginary parts individually.
+     * However, if the real part is zero (but not negative zero), only the imaginary part is
+     * printed, and without parentheses like <code>"24.68j"</code>. The number format specification
+     * passed in is used for the real and imaginary parts individually, with padding to width
+     * afterwards (if the specification requires it).
      *
      * @param spec parsed format specification string
      * @return formatted value
@@ -176,12 +177,13 @@
         FloatFormatter f = new FloatFormatter(spec, 2, 3); // Two elements + "(j)".length
         // Even in r-format, complex strips *all* the trailing zeros.
         f.setMinFracDigits(0);
-        if (real == 0.0) {
+        if (Double.doubleToLongBits(real) == 0L) {
+            // Real part is truly zero: show no real part.
             f.format(imag).append('j');
         } else {
-            f.append('(').format(real).format(imag, "+").append("j)").pad();
+            f.append('(').format(real).format(imag, "+").append("j)");
         }
-        return f.getResult();
+        return f.pad().getResult();
     }
 
     @Override
@@ -823,7 +825,7 @@
         try {
             String specString = formatSpecStr.getString();
             Spec spec = InternalFormat.fromText(specString);
-            if (spec.type!=Spec.NONE && "efgEFGn%".indexOf(spec.type) < 0) {
+            if (spec.type != Spec.NONE && "efgEFGn%".indexOf(spec.type) < 0) {
                 throw FloatFormatter.unknownFormat(spec.type, "complex");
             } else if (spec.alternate) {
                 throw FloatFormatter.alternateFormNotAllowed("complex");
@@ -838,12 +840,12 @@
                     // And then we use the __str__ mechanism to get parentheses or real 0 elision.
                     result = formatComplex(spec);
                 } else {
-                    // In any other format, the defaults those commonly used for numeric formats.
+                    // In any other format, defaults are those commonly used for numeric formats.
                     spec = spec.withDefaults(Spec.NUMERIC);
                     FloatFormatter f = new FloatFormatter(spec, 2, 1);// 2 floats + "j"
-                    // Convert as both parts per specification
-                    f.format(real).format(imag, "+").append('j').pad();
-                    result = f.getResult();
+                    // Convert both parts as per specification
+                    f.format(real).format(imag, "+").append('j');
+                    result = f.pad().getResult();
                 }
             }
         } catch (IllegalArgumentException e) {
diff --git a/src/org/python/core/PyString.java b/src/org/python/core/PyString.java
--- a/src/org/python/core/PyString.java
+++ b/src/org/python/core/PyString.java
@@ -4,17 +4,15 @@
 import java.lang.ref.Reference;
 import java.lang.ref.SoftReference;
 import java.math.BigInteger;
-import java.text.DecimalFormat;
-import java.text.DecimalFormatSymbols;
-
-import org.python.core.StringFormatter.DecimalFormatTemplate;
+
 import org.python.core.buffer.BaseBuffer;
 import org.python.core.buffer.SimpleStringBuffer;
 import org.python.core.stringlib.FieldNameIterator;
+import org.python.core.stringlib.FloatFormatter;
+import org.python.core.stringlib.InternalFormat.Spec;
 import org.python.core.stringlib.InternalFormatSpec;
 import org.python.core.stringlib.InternalFormatSpecParser;
 import org.python.core.stringlib.MarkupIterator;
-import org.python.core.util.ExtraMath;
 import org.python.core.util.StringUtil;
 import org.python.expose.ExposedMethod;
 import org.python.expose.ExposedNew;
@@ -1296,8 +1294,8 @@
      * @param stripChars characters to strip from the left end of this str/bytes, or null
      * @return a new String, stripped of the specified characters/bytes
      */
-    public String lstrip(String sep) {
-        return _lstrip(sep);
+    public String lstrip(String stripChars) {
+        return _lstrip(stripChars);
     }
 
     /**
@@ -1310,8 +1308,8 @@
      * @return a new <code>PyString</code> (or {@link PyUnicode}), stripped of the specified
      *         characters/bytes
      */
-    public PyObject lstrip(PyObject sep) {
-        return str_lstrip(sep);
+    public PyObject lstrip(PyObject stripChars) {
+        return str_lstrip(stripChars);
     }
 
     @ExposedMethod(defaults = "null", doc = BuiltinDocs.str_lstrip_doc)
@@ -1385,8 +1383,8 @@
      * @param stripChars characters to strip from either end of this str/bytes, or null
      * @return a new String, stripped of the specified characters/bytes
      */
-    public String rstrip(String sep) {
-        return _rstrip(sep);
+    public String rstrip(String stripChars) {
+        return _rstrip(stripChars);
     }
 
     /**
@@ -1399,8 +1397,8 @@
      * @return a new <code>PyString</code> (or {@link PyUnicode}), stripped of the specified
      *         characters/bytes
      */
-    public PyObject rstrip(PyObject sep) {
-        return str_rstrip(sep);
+    public PyObject rstrip(PyObject stripChars) {
+        return str_rstrip(stripChars);
     }
 
     @ExposedMethod(defaults = "null", doc = BuiltinDocs.str_rstrip_doc)
@@ -1546,7 +1544,7 @@
      * the last element of the list contains the what is left over after the last split.
      * <p>
      * Implementation note: although a str contains only bytes, this method is also called by
-     * {@link PyUnicode#unicode_split(PyObject)}.
+     * {@link PyUnicode#unicode_split(PyObject, int)}.
      *
      * @param sep string to use as separator (or <code>null</code> if to split on whitespace)
      * @param maxsplit maximum number of splits to make (there may be <code>maxsplit+1</code>
@@ -1798,7 +1796,7 @@
      * left over after the last split.
      * <p>
      * Implementation note: although a str contains only bytes, this method is also called by
-     * {@link PyUnicode#unicode_rsplit(PyObject)} .
+     * {@link PyUnicode#unicode_rsplit(PyObject, int)} .
      *
      * @param sep string to use as separator (or <code>null</code> if to split on whitespace)
      * @param maxsplit maximum number of splits to make (there may be <code>maxsplit+1</code>
@@ -2931,11 +2929,10 @@
      *
      * @param oldPiece to replace where found.
      * @param newPiece replacement text.
-     * @param count maximum number of replacements to make, or -1 meaning all of them.
      * @return PyString (or PyUnicode if any string is one), this string after replacements.
      */
-    public PyString replace(PyObject oldPieceObj, PyObject newPieceObj) {
-        return str_replace(oldPieceObj, newPieceObj, -1);
+    public PyString replace(PyObject oldPiece, PyObject newPiece) {
+        return str_replace(oldPiece, newPiece, -1);
     }
 
     /**
@@ -2949,8 +2946,8 @@
      * @param count maximum number of replacements to make, or -1 meaning all of them.
      * @return PyString (or PyUnicode if any string is one), this string after replacements.
      */
-    public PyString replace(PyObject oldPieceObj, PyObject newPieceObj, int count) {
-        return str_replace(oldPieceObj, newPieceObj, count);
+    public PyString replace(PyObject oldPiece, PyObject newPiece, int count) {
+        return str_replace(oldPiece, newPiece, count);
     }
 
     @ExposedMethod(defaults = "-1", doc = BuiltinDocs.str_replace_doc)
@@ -3161,8 +3158,8 @@
      * @return <code>true</code> if this string slice starts with a specified prefix, otherwise
      *         <code>false</code>.
      */
-    public boolean startswith(PyObject prefix, PyObject offset) {
-        return str_startswith(prefix, offset, null);
+    public boolean startswith(PyObject prefix, PyObject start) {
+        return str_startswith(prefix, start, null);
     }
 
     /**
@@ -4002,13 +3999,24 @@
  */
 final class StringFormatter {
 
+    /** Index into {@link #format} being interpreted. */
     int index;
+    /** Format being interpreted. */
     String format;
+    /** Where the output is built. */
     StringBuilder buffer;
+    /** Remembers that the value currently converted is negative */
     boolean negative;
+    /** Precision from format specification. */
     int precision;
+    /**
+     * Index into args of argument currently being worked, or special values indicating -1: a single
+     * item that has not yet been used, -2: a single item that has already been used, -3: a mapping.
+     */
     int argIndex;
+    /** Arguments supplied to {@link #format(PyObject)} method. */
     PyObject args;
+    /** Indicate a <code>PyUnicode</code> result is expected. */
     boolean unicodeCoercion;
 
     final char pop() {
@@ -4027,6 +4035,11 @@
         index--;
     }
 
+    /**
+     * Initialise the interpreter with the given format string, ready for {@link #format(PyObject)}.
+     *
+     * @param format string to interpret
+     */
     public StringFormatter(String format) {
         this(format, false);
     }
@@ -4044,6 +4057,10 @@
         buffer = new StringBuilder(format.length() + 100);
     }
 
+    /**
+     * Read the next object from the argument list, taking special values of <code>argIndex</code>
+     * into account.
+     */
     PyObject getarg() {
         PyObject ret = null;
         switch (argIndex) {
@@ -4064,6 +4081,10 @@
         return ret;
     }
 
+    /**
+     * Parse a number from the format, except if the next thing is "*", read it from the argument
+     * list.
+     */
     int getNumber() {
         char c = pop();
         if (c == '*') {
@@ -4093,7 +4114,24 @@
 
     }
 
+    /**
+     * Format the argument interpreted as a long, using the argument's <code>__str__</code>,
+     * <code>__oct__</code>, or <code>__hex__</code> method according to <code>type</code>. If v is
+     * being treated as signed, the sign of v is transferred to {@link #negative} and the absolute
+     * value is converted. The <code>altFlag</code> argument controls the appearance of a "0x" or
+     * "0X" prefix in the hex case, or a "0" prefix in the octal case. The hexadecimal case, the
+     * case of characters and digits will match the type ('x' meaning lowercase, 'X' meaning
+     * uppercase).
+     *
+     * @param arg to convert
+     * @param type one of 'o' for octal, 'x' or 'X' for hex, anything else calls
+     *            <code>arg.__str__</code>.
+     * @param altFlag if true there will be a prefix
+     * @return converted value as <code>String</code>
+     */
     private String formatLong(PyObject arg, char type, boolean altFlag) {
+        // Convert using the appropriate type
+        // XXX Results in behaviour divergent from CPython when any of the methods is overridden.
         PyString argAsString;
         switch (type) {
             case 'o':
@@ -4107,29 +4145,37 @@
                 argAsString = arg.__str__();
                 break;
         }
+
         checkPrecision("long");
         String s = argAsString.toString();
         int end = s.length();
         int ptr = 0;
 
+        // In the hex case, the __hex__ return starts 0x
+        // XXX (we assume, perhaps falsely)
         int numnondigits = 0;
         if (type == 'x' || type == 'X') {
             numnondigits = 2;
         }
 
+        // Strip a "long" indicator
         if (s.endsWith("L")) {
             end--;
         }
 
+        // Strip a possible sign to member negative
         negative = s.charAt(0) == '-';
         if (negative) {
             ptr++;
         }
 
+        // The formatted number is s[ptr:end] and starts with numnondigits non-digits.
         int numdigits = end - numnondigits - ptr;
         if (!altFlag) {
+            // We should have no "base tag" '0' or "0x" on the front.
             switch (type) {
                 case 'o':
+                    // Strip the '0'
                     if (numdigits > 1) {
                         ++ptr;
                         --numdigits;
@@ -4137,27 +4183,36 @@
                     break;
                 case 'x':
                 case 'X':
+                    // Strip the "0x"
                     ptr += 2;
                     numnondigits -= 2;
                     break;
             }
         }
+
+        // If necessary, add leading zeros to the numerical digits part.
         if (precision > numdigits) {
+            // Recompose the formatted number in this buffer
             StringBuilder buf = new StringBuilder();
+            // The base indicator prefix
             for (int i = 0; i < numnondigits; ++i) {
                 buf.append(s.charAt(ptr++));
             }
+            // The extra zeros
             for (int i = 0; i < precision - numdigits; i++) {
                 buf.append('0');
             }
+            // The previously known digits
             for (int i = 0; i < numdigits; i++) {
                 buf.append(s.charAt(ptr++));
             }
             s = buf.toString();
         } else if (end < s.length() || ptr > 0) {
+            // It's only necessary to extract the formatted number from s
             s = s.substring(ptr, end);
         }
 
+        // And finally, deal with the case, so it matches x or X.
         switch (type) {
             case 'X':
                 s = s.toUpperCase();
@@ -4167,10 +4222,16 @@
     }
 
     /**
-     * Formats arg as an integer, with the specified radix
+     * Formats arg as an integer, with the specified radix. The integer value is obtained from the
+     * result of <code>arg.__int__()</code>. <code>type</code> and <code>altFlag</code> are passed
+     * to {@link #formatLong(PyObject, char, boolean)} in case the result is a PyLong.
      *
-     * type and altFlag are needed to be passed to {@link #formatLong(PyObject, char, boolean)} in
-     * case the result of <code>arg.__int__()</code> is a PyLong.
+     * @param arg to convert
+     * @param radix in which to express <code>arg</code>
+     * @param unsigned true if required to interpret a 32-bit integer as unsigned ('u' legacy?).
+     * @param type of conversion ('d', 'o', 'x', or 'X')
+     * @param altFlag '#' present in format (causes "0x" prefix in hex, and '0' prefix in octal)
+     * @return string form of the value
      */
     private String formatInteger(PyObject arg, int radix, boolean unsigned, char type,
             boolean altFlag) {
@@ -4183,7 +4244,6 @@
                 // safe to call __int__:
                 argAsInt = arg.__int__();
             } else {
-                // Same case noted on formatFloatDecimal:
                 // We can't simply call arg.__int__() because PyString implements
                 // it without exposing it to python (i.e, str instances has no
                 // __int__ attribute). So, we would support strings as arguments
@@ -4192,7 +4252,7 @@
                 try {
                     argAsInt = arg.__getattr__("__int__").__call__();
                 } catch (PyException e) {
-                    // XXX: Swallow customs AttributeError throws from __float__ methods
+                    // XXX: Swallow custom AttributeError throws from __int__ methods
                     // No better alternative for the moment
                     if (e.match(Py.AttributeError)) {
                         throw Py.TypeError("int argument required");
@@ -4202,25 +4262,44 @@
             }
         }
         if (argAsInt instanceof PyInteger) {
+            // This call does not provide the prefix and will be lowercase.
             return formatInteger(((PyInteger)argAsInt).getValue(), radix, unsigned);
         } else { // must be a PyLong (as per __int__ contract)
+            // This call provides the base prefix and case-matches with 'x' or 'X'.
             return formatLong(argAsInt, type, altFlag);
         }
     }
 
+    /**
+     * Convert a 32-bit integer (as from a {@link PyInteger}) to characters, signed or unsigned. The
+     * values is presented in a <code>long</code>. The string result is left-padded with zeros to
+     * the stated {@link #precision}. If v is being treated as signed, the sign of v is transferred
+     * to {@link #negative} and the absolute value is converted. Otherwise (unsigned case)
+     * <code>0x100000000L + v</code> is converted. This method does not provide the '0' or "0x"
+     * prefix, just the padded digit string.
+     *
+     * @param v value to convert
+     * @param radix of conversion
+     * @param unsigned if should be treated as unsigned
+     * @return string form
+     */
     private String formatInteger(long v, int radix, boolean unsigned) {
         checkPrecision("integer");
         if (unsigned) {
+            // If the high bit was set, this will have been sign-extended: correct that.
             if (v < 0) {
                 v = 0x100000000l + v;
             }
         } else {
+            // If the high bit was set, the sign extension was correct, but we need sign + abs(v).
             if (v < 0) {
                 negative = true;
                 v = -v;
             }
         }
+        // Use the method in java.lang.Long (lowercase, no prefix)
         String s = Long.toString(v, radix);
+        // But zero pad to the requested precision
         while (s.length() < precision) {
             s = "0" + s;
         }
@@ -4235,88 +4314,23 @@
         }
     }
 
-    static class DecimalFormatTemplate {
-
-        static DecimalFormat template;
-        static {
-            template =
-                    new DecimalFormat("#,##0.#####", new DecimalFormatSymbols(java.util.Locale.US));
-            DecimalFormatSymbols symbols = template.getDecimalFormatSymbols();
-            symbols.setNaN("nan");
-            symbols.setInfinity("inf");
-            template.setDecimalFormatSymbols(symbols);
-            template.setGroupingUsed(false);
-        }
-    }
-
-    private static final DecimalFormat getDecimalFormat() {
-        return (DecimalFormat)DecimalFormatTemplate.template.clone();
-    }
-
-    private String formatFloatDecimal(double v, boolean truncate) {
-        checkPrecision("decimal");
-        int prec = precision;
-        if (prec == -1) {
-            prec = 6;
-        }
-        if (v < 0) {
-            v = -v;
-            negative = true;
-        }
-
-        DecimalFormat decimalFormat = getDecimalFormat();
-        decimalFormat.setMaximumFractionDigits(prec);
-        decimalFormat.setMinimumFractionDigits(truncate ? 0 : prec);
-
-        String ret = decimalFormat.format(v);
-        return ret;
-    }
-
-    private String formatFloatExponential(PyObject arg, char e, boolean truncate) {
-        StringBuilder buf = new StringBuilder();
-        double v = asDouble(arg);
-        boolean isNegative = false;
-        if (v < 0) {
-            v = -v;
-            isNegative = true;
-        }
-        double power = 0.0;
-        if (v > 0) {
-            power = ExtraMath.closeFloor(Math.log10(v));
-        }
-        // System.err.println("formatExp: "+v+", "+power);
-        int savePrecision = precision;
-        precision = 2;
-
-        String exp = formatInteger((long)power, 10, false);
-        if (negative) {
-            negative = false;
-            exp = '-' + exp;
-        } else {
-            exp = '+' + exp;
-        }
-
-        precision = savePrecision;
-
-        double base = v / Math.pow(10, power);
-        buf.append(formatFloatDecimal(base, truncate));
-        buf.append(e);
-
-        buf.append(exp);
-        negative = isNegative;
-
-        return buf.toString();
-    }
-
+    /**
+     * Main service of this class: format one or more arguments with the format string supplied at
+     * construction.
+     *
+     * @param args tuple or map containing objects, or a single object, to convert
+     * @return result of formatting
+     */
     @SuppressWarnings("fallthrough")
     public PyString format(PyObject args) {
         PyObject dict = null;
         this.args = args;
         boolean needUnicode = unicodeCoercion;
         if (args instanceof PyTuple) {
+            // We will simply work through the tuple elements
             argIndex = 0;
         } else {
-            // special index indicating a single item rather than a tuple
+            // Not a tuple, but possibly still some kind of container: use special argIndex values.
             argIndex = -1;
             if (args instanceof PyDictionary || args instanceof PyStringMap
                     || (!(args instanceof PySequence) && args.__findattr__("__getitem__") != null)) {
@@ -4326,6 +4340,8 @@
         }
 
         while (index < format.length()) {
+
+            // Attributes to be parsed from the next format specifier
             boolean ljustFlag = false;
             boolean signFlag = false;
             boolean blankFlag = false;
@@ -4335,16 +4351,31 @@
             int width = -1;
             precision = -1;
 
+            // Read one character from the format string
             char c = pop();
             if (c != '%') {
                 buffer.append(c);
                 continue;
             }
+
+            // It's a %, so the beginning of a conversion specifier. Parse it.
+
+            // A conversion specifier contains the following components, in this order:
+            // + The '%' character, which marks the start of the specifier.
+            // + Mapping key (optional), consisting of a parenthesised sequence of characters.
+            // + Conversion flags (optional), which affect the result of some conversion types.
+            // + Minimum field width (optional), or an '*' (asterisk).
+            // + Precision (optional), given as a '.' (dot) followed by the precision or '*'.
+            // + Length modifier (optional).
+            // + Conversion type.
+
             c = pop();
             if (c == '(') {
+                // Mapping key, consisting of a parenthesised sequence of characters.
                 if (dict == null) {
                     throw Py.TypeError("format requires a mapping");
                 }
+                // Scan along until a matching close parenthesis is found
                 int parens = 1;
                 int keyStart = index;
                 while (parens > 0) {
@@ -4355,11 +4386,16 @@
                         parens++;
                     }
                 }
+                // Last c=pop() is the closing ')' while indexKey is just after the opening '('
                 String tmp = format.substring(keyStart, index - 1);
+                // Look it up using this extent as the (right type of) key.
                 this.args = dict.__getitem__(needUnicode ? new PyUnicode(tmp) : new PyString(tmp));
             } else {
+                // Not a mapping key: next clause will re-read c.
                 push();
             }
+
+            // Conversion flags (optional) that affect the result of some conversion types.
             while (true) {
                 switch (c = pop()) {
                     case '-':
@@ -4380,43 +4416,77 @@
                 }
                 break;
             }
+            // Push back c as next clause will re-read c.
             push();
+
+            /*
+             * Minimum field width (optional). If specified as an '*' (asterisk), the actual width
+             * is read from the next element of the tuple in values, and the object to convert comes
+             * after the minimum field width and optional precision. A custom getNumber() takes care
+             * of the '*' case.
+             */
             width = getNumber();
             if (width < 0) {
                 width = -width;
                 ljustFlag = true;
             }
+
+            /*
+             * Precision (optional), given as a '.' (dot) followed by the precision. If specified as
+             * '*' (an asterisk), the actual precision is read from the next element of the tuple in
+             * values, and the value to convert comes after the precision. A custom getNumber()
+             * takes care of the '*' case.
+             */
             c = pop();
             if (c == '.') {
                 precision = getNumber();
                 if (precision < -1) {
                     precision = 0;
                 }
-
                 c = pop();
             }
+
+            // Length modifier (optional). (Compatibility feature?) It has no effect.
             if (c == 'h' || c == 'l' || c == 'L') {
                 c = pop();
             }
+
+            // c is now the conversion type.
             if (c == '%') {
+                // It was just a percent sign after all
                 buffer.append(c);
                 continue;
             }
+
+            /*
+             * Process argument according to format specification decoded from the string. It is
+             * important we don't read the argumnent from the list until this point because of the
+             * possibility that width and precision were specified via the argument list.
+             */
             PyObject arg = getarg();
-            char fill = ' ';
             String string = null;
             negative = false;
+
+            // Independent of type, decide the padding character based on decoded flags.
+            char fill = ' ';
             if (zeroFlag) {
                 fill = '0';
             } else {
                 fill = ' ';
             }
+
+            // Perform the type-specific formatting
             switch (c) {
+
                 case 's':
+                    // String (converts any Python object using str()).
                     if (arg instanceof PyUnicode) {
                         needUnicode = true;
                     }
+                    // fall through ...
+
                 case 'r':
+                    // String (converts any Python object using repr()).
                     fill = ' ';
                     if (c == 's') {
                         if (needUnicode) {
@@ -4432,15 +4502,19 @@
                     }
 
                     break;
+
                 case 'i':
                 case 'd':
+                    // Signed integer decimal. Note floats accepted.
                     if (arg instanceof PyLong) {
                         string = formatLong(arg, c, altFlag);
                     } else {
                         string = formatInteger(arg, 10, false, c, altFlag);
                     }
                     break;
+
                 case 'u':
+                    // Obsolete type ? it is identical to 'd'. (Why not identical here?)
                     if (arg instanceof PyLong) {
                         string = formatLong(arg, c, altFlag);
                     } else if (arg instanceof PyInteger || arg instanceof PyFloat) {
@@ -4449,10 +4523,15 @@
                         throw Py.TypeError("int argument required");
                     }
                     break;
+
                 case 'o':
+                    // Signed octal value. Note floats accepted.
                     if (arg instanceof PyLong) {
+                        // This call provides the base prefix '0' if altFlag.
                         string = formatLong(arg, c, altFlag);
                     } else if (arg instanceof PyInteger || arg instanceof PyFloat) {
+                        // This call does not provide the '0' prefix and will be lowercase ...
+                        // ... except where arg.__int__ returns PyLong, then it's like formatLong.
                         string = formatInteger(arg, 8, false, c, altFlag);
                         if (altFlag && string.charAt(0) != '0') {
                             string = "0" + string;
@@ -4461,10 +4540,15 @@
                         throw Py.TypeError("int argument required");
                     }
                     break;
+
                 case 'x':
+                    // Signed hexadecimal (lowercase). Note floats accepted.
                     if (arg instanceof PyLong) {
+                        // This call provides the base prefix "0x" if altFlag and case-matches c.
                         string = formatLong(arg, c, altFlag);
                     } else if (arg instanceof PyInteger || arg instanceof PyFloat) {
+                        // This call does not provide the "0x" prefix and will be lowercase.
+                        // ... except where arg.__int__ returns PyLong, then it's like formatLong.
                         string = formatInteger(arg, 16, false, c, altFlag);
                         string = string.toLowerCase();
                         if (altFlag) {
@@ -4474,10 +4558,15 @@
                         throw Py.TypeError("int argument required");
                     }
                     break;
+
                 case 'X':
+                    // Signed hexadecimal (uppercase). Note floats accepted.
                     if (arg instanceof PyLong) {
+                        // This call provides the base prefix "0x" if altFlag and case-matches c.
                         string = formatLong(arg, c, altFlag);
                     } else if (arg instanceof PyInteger || arg instanceof PyFloat) {
+                        // This call does not provide the "0x" prefix and will be lowercase.
+                        // ... except where arg.__int__ returns PyLong, then it's like formatLong.
                         string = formatInteger(arg, 16, false, c, altFlag);
                         string = string.toUpperCase();
                         if (altFlag) {
@@ -4487,60 +4576,33 @@
                         throw Py.TypeError("int argument required");
                     }
                     break;
+
                 case 'e':
                 case 'E':
-                    string = formatFloatExponential(arg, c, false);
-                    if (c == 'E') {
-                        string = string.toUpperCase();
-                    }
-                    break;
                 case 'f':
                 case 'F':
-                    string = formatFloatDecimal(asDouble(arg), false);
-                    if (c == 'F') {
-                        string = string.toUpperCase();
-                    }
-                    break;
                 case 'g':
                 case 'G':
-                    int origPrecision = precision;
-                    if (precision == -1) {
-                        precision = 6;
-                    }
-
+                    // All floating point formats (+case).
+
+                    // Convert the flags (local variables) to the form needed in the Spec object.
+                    char align = ljustFlag ? '<' : '>';
+                    char sign = signFlag ? '+' : (blankFlag ? ' ' : Spec.NONE);
+                    int w = Spec.UNSPECIFIED;
+                    Spec spec = new Spec(fill, align, sign, altFlag, w, false, precision, c);
+
+                    // Format using this Spec the double form of the argument.
+                    FloatFormatter f = new FloatFormatter(spec);
                     double v = asDouble(arg);
-                    int exponent = (int)ExtraMath.closeFloor(Math.log10(Math.abs(v == 0 ? 1 : v)));
-                    if (v == Double.POSITIVE_INFINITY) {
-                        string = "inf";
-                    } else if (v == Double.NEGATIVE_INFINITY) {
-                        string = "-inf";
-                    } else if (exponent >= -4 && exponent < precision) {
-                        precision -= exponent + 1;
-                        string = formatFloatDecimal(v, !altFlag);
-
-                        // XXX: this block may be unnecessary now
-                        if (altFlag && string.indexOf('.') == -1) {
-                            int zpad = origPrecision - string.length();
-                            string += '.';
-                            if (zpad > 0) {
-                                char zeros[] = new char[zpad];
-                                for (int ci = 0; ci < zpad; zeros[ci++] = '0') {}
-                                string += new String(zeros);
-                            }
-                        }
-                    } else {
-                        // Exponential precision is the number of digits after the decimal
-                        // point, whereas 'g' precision is the number of significant digits --
-                        // and exponential always provides one significant digit before the
-                        // decimal point
-                        precision--;
-                        string = formatFloatExponential(arg, (char)(c - 2), !altFlag);
-                    }
-                    if (c == 'G') {
-                        string = string.toUpperCase();
-                    }
+                    f.format(v);
+                    string = f.getResult();
+
+                    // Suppress subsequent attempts to insert a correct sign, done already.
+                    signFlag = blankFlag = negative = false;
                     break;
+
                 case 'c':
+                    // Single character (accepts integer or single character string).
                     fill = ' ';
                     if (arg instanceof PyString) {
                         string = ((PyString)arg).toString();
@@ -4552,6 +4614,8 @@
                         }
                         break;
                     }
+
+                    // arg is not a str (or unicode)
                     int val;
                     try {
                         // Explicitly __int__ so we can look for an AttributeError (which is
@@ -4563,6 +4627,7 @@
                         }
                         throw e;
                     }
+                    // Range check, according to ultimate type of result as presentl;y known.
                     if (!needUnicode) {
                         if (val < 0) {
                             throw Py.OverflowError("unsigned byte integer is less than minimum");
@@ -4580,8 +4645,15 @@
                             + codecs.encode(Py.newString(c), null, "replace") + "' (0x"
                             + Integer.toHexString(c) + ") at index " + (index - 1));
             }
+
+            /*
+             * We have now dealt with the translation of the (absolute value of the) argument, in
+             * variable string[]. In the next sections we deal with sign, padding and base prefix.
+             */
             int length = string.length();
             int skip = 0;
+
+            // Decide how to represent the sign according to format and actual sign of argument.
             String signString = null;
             if (negative) {
                 signString = "-";
@@ -4593,34 +4665,47 @@
                 }
             }
 
+            // The width (from here on) will be the remaining width on the line.
             if (width < length) {
                 width = length;
             }
+
+            // Insert the sign in the buffer and adjust the width.
             if (signString != null) {
                 if (fill != ' ') {
+                    // When the fill is not space, the sign comes before the fill.
                     buffer.append(signString);
                 }
+                // Adjust width for sign.
                 if (width > length) {
                     width--;
                 }
             }
+
+            // Insert base prefix used with alternate mode for hexadecimal.
             if (altFlag && (c == 'x' || c == 'X')) {
                 if (fill != ' ') {
+                    // When the fill is not space, this base prefix comes before the fill.
                     buffer.append('0');
                     buffer.append(c);
                     skip += 2;
                 }
+                // Adjust width for base prefix.
                 width -= 2;
                 if (width < 0) {
                     width = 0;
                 }
                 length -= 2;
             }
+
+            // Fill on the left of the item.
             if (width > length && !ljustFlag) {
                 do {
                     buffer.append(fill);
                 } while (--width > length);
             }
+
+            // If the fill is spaces, we will have deferred the sign and hex base prefix
             if (fill == ' ') {
                 if (signString != null) {
                     buffer.append(signString);
@@ -4631,19 +4716,33 @@
                     skip += 2;
                 }
             }
+
+            // Now append the converted argument.
             if (skip > 0) {
+                // The string contains a hex-prefix, but we have already inserted one.
                 buffer.append(string.substring(skip));
             } else {
                 buffer.append(string);
             }
 
+            // If this hasn't filled the space required, add right-padding.
             while (--width >= length) {
                 buffer.append(' ');
             }
         }
+
+        /*
+         * All fields in the format string have been used to convert arguments (or used the argument
+         * as a width, etc.). This had better not leave any arguments unused. Note argIndex is an
+         * index into args or has a special value. If args is a 'proper' index, It should now be out
+         * of range; if a special value, it would be wrong if it were -1, indicating a single item
+         * that has not yet been used.
+         */
         if (argIndex == -1 || (argIndex >= 0 && args.__finditem__(argIndex) != null)) {
             throw Py.TypeError("not all arguments converted during string formatting");
         }
+
+        // Return the final buffer contents as a str or unicode as appropriate.
         if (needUnicode) {
             return new PyUnicode(buffer);
         }
diff --git a/src/org/python/core/stringlib/FloatFormatter.java b/src/org/python/core/stringlib/FloatFormatter.java
--- a/src/org/python/core/stringlib/FloatFormatter.java
+++ b/src/org/python/core/stringlib/FloatFormatter.java
@@ -16,7 +16,7 @@
 public class FloatFormatter extends InternalFormat.Formatter {
 
     /** The rounding mode dominant in the formatter. */
-    static final RoundingMode ROUND_PY = RoundingMode.HALF_UP; // Believed to be HALF_EVEN in Py3k
+    static final RoundingMode ROUND_PY = RoundingMode.HALF_EVEN;
 
     /** If it contains no decimal point, this length is zero, and 1 otherwise. */
     private int lenPoint;
@@ -325,10 +325,9 @@
             exp = lenFraction - vv.scale();
         }
 
-        // Finally add zeros, as necessary, and stick on the exponent.
-
+        // If the result is not already complete, add point and zeros as necessary, and exponent.
         if (lenMarker == 0) {
-            appendTrailingZeros(precision);
+            ensurePointAndTrailingZeros(precision);
             appendExponent(exp);
         }
     }
@@ -345,14 +344,7 @@
      */
     private void format_f(double value, String positivePrefix, int precision) {
 
-        if (signAndSpecialNumber(value, positivePrefix)) {
-
-            if (lenMarker == 0) {
-                // May be 0 or -0 so we still need to ...
-                appendTrailingZeros(precision);
-            }
-
-        } else {
+        if (!signAndSpecialNumber(value, positivePrefix)) {
             // Convert value to decimal exactly. (This can be very long.)
             BigDecimal vLong = new BigDecimal(Math.abs(value));
 
@@ -366,9 +358,53 @@
                 // There is a decimal point and some digits following
                 lenWhole = result.length() - (start + lenSign + (lenPoint = 1) + lenFraction);
             } else {
+                // There are no fractional digits and so no decimal point
                 lenWhole = result.length() - (start + lenSign);
             }
+        }
 
+        // Finally, ensure we have all the fractional digits we should.
+        if (lenMarker == 0) {
+            ensurePointAndTrailingZeros(precision);
+        }
+    }
+
+    /**
+     * Append a decimal point and trailing fractional zeros if necessary for 'e' and 'f' format.
+     * This should not be called if the result is not numeric ("inf" for example). This method deals
+     * with the following complexities: on return there will be at least the number of fractional
+     * digits specified in the argument <code>n</code>, and at least {@link #minFracDigits};
+     * further, if <code>minFracDigits&lt;0</code>, signifying the "alternate mode" of certain
+     * formats, the method will ensure there is a decimal point, even if there are no fractional
+     * digits to follow.
+     *
+     * @param n smallest number of fractional digits on return
+     */
+    private void ensurePointAndTrailingZeros(int n) {
+
+        // Set n to the number of fractional digits we should have.
+        if (n < minFracDigits) {
+            n = minFracDigits;
+        }
+
+        // Do we have a decimal point already?
+        if (lenPoint == 0) {
+            // No decimal point: add one if there will be any fractional digits or
+            if (n > 0 || minFracDigits < 0) {
+                // First need to add a decimal point.
+                result.append('.');
+                lenPoint = 1;
+            }
+        }
+
+        // Do we have enough fractional digits already?
+        int f = lenFraction;
+        if (n > f) {
+            // Make up the required number of zeros.
+            for (; f < n; f++) {
+                result.append('0');
+            }
+            lenFraction = f;
         }
     }
 
@@ -635,7 +671,7 @@
 
         // In no-truncate mode, the fraction is full precision. Otherwise trim it.
         if (minFracDigits >= 0) {
-            // Note minFracDigits only applies to fixed formats.
+            // Note positive minFracDigits only applies to fixed formats.
             removeTrailingZeros(0);
         }
 
@@ -757,16 +793,16 @@
 
     /**
      * Append the trailing fractional zeros, as required by certain formats, so that the total
-     * number of fractional digits is no less than specified. If <code>minFracDigits&lt;=0</code>,
-     * the method leaves the {@link #result} buffer unchanged.
+     * number of fractional digits is no less than specified. If <code>n&lt;=0</code>, the method
+     * leaves the {@link #result} buffer unchanged.
      *
-     * @param minFracDigits smallest number of fractional digits on return
+     * @param n smallest number of fractional digits on return
      */
-    private void appendTrailingZeros(int minFracDigits) {
+    private void appendTrailingZeros(int n) {
 
         int f = lenFraction;
 
-        if (minFracDigits > f) {
+        if (n > f) {
             if (lenPoint == 0) {
                 // First need to add a decimal point. (Implies lenFraction=0.)
                 result.append('.');
@@ -774,7 +810,7 @@
             }
 
             // Now make up the required number of zeros.
-            for (; f < minFracDigits; f++) {
+            for (; f < n; f++) {
                 result.append('0');
             }
             lenFraction = f;
@@ -787,9 +823,9 @@
      * originally (and therefore no fractional part), the method will add a decimal point, even if
      * it adds no zeros.
      *
-     * @param minFracDigits smallest number of fractional digits on return
+     * @param n smallest number of fractional digits on return
      */
-    private void appendPointAndTrailingZeros(int minFracDigits) {
+    private void appendPointAndTrailingZeros(int n) {
 
         if (lenPoint == 0) {
             // First need to add a decimal point. (Implies lenFraction=0.)
@@ -799,7 +835,7 @@
 
         // Now make up the required number of zeros.
         int f;
-        for (f = lenFraction; f < minFracDigits; f++) {
+        for (f = lenFraction; f < n; f++) {
             result.append('0');
         }
         lenFraction = f;
@@ -810,18 +846,17 @@
      * least the number of fractional digits specified. If the resultant number of fractional digits
      * is zero, this method will also remove the trailing decimal point (if there is one).
      *
-     * @param minFracDigits smallest number of fractional digits on return
+     * @param n smallest number of fractional digits on return
      */
-    private void removeTrailingZeros(int minFracDigits) {
-
-        int f = lenFraction;
+    private void removeTrailingZeros(int n) {
 
         if (lenPoint > 0) {
             // There's a decimal point at least, and there may be some fractional digits.
-            if (minFracDigits == 0 || f > minFracDigits) {
+            int f = lenFraction;
+            if (n == 0 || f > n) {
 
                 int fracStart = result.length() - f;
-                for (; f > minFracDigits; --f) {
+                for (; f > n; --f) {
                     if (result.charAt(fracStart - 1 + f) != '0') {
                         // Keeping this one as it isn't a zero
                         break;
diff --git a/src/org/python/core/stringlib/InternalFormat.java b/src/org/python/core/stringlib/InternalFormat.java
--- a/src/org/python/core/stringlib/InternalFormat.java
+++ b/src/org/python/core/stringlib/InternalFormat.java
@@ -108,7 +108,7 @@
          * method shows a '|' character between each section when it prints out the buffer. Override
          * this when you define more lengths in the subclass.
          *
-         * @return
+         * @return the lengths of the successive sections
          */
         protected int[] sectionLengths() {
             return new int[] {lenSign, lenWhole};
@@ -265,7 +265,6 @@
          * result you probably don't want. It is up to the client to disallow this (which
          * <code>complex</code> does).
          *
-         * @param value to pad
          * @return this object
          */
         public Formatter pad() {
@@ -468,8 +467,8 @@
      * during the construction of a new <code>Spec</code>, for attributes that are unspecified in a
      * primary source.
      * <p>
-     * This structure is returned by factory method {@link #fromText(CharSequence)}, and having
-     * public final members is freely accessed by formatters such as {@link FloatBuilder}, and the
+     * This structure is returned by factory method {@link #fromText(String)}, and having public
+     * final members is freely accessed by formatters such as {@link FloatFormatter}, and the
      * __format__ methods of client object types.
      * <p>
      * The fields correspond to the elements of a format specification. The grammar of a format
@@ -482,28 +481,28 @@
      * A typical idiom is:
      *
      * <pre>
-     *     private static final InternalFormatSpec FLOAT_DEFAULT = InternalFormatSpec.from(">");
+     *     private static final InternalFormatSpec FLOAT_DEFAULTS = InternalFormatSpec.from(">");
      *     ...
-     *         InternalFormatSpec spec = InternalFormatSpec.from(specString, FLOAT_DEFAULT);
+     *         InternalFormat.Spec spec = InternalFormat.fromText(specString);
+     *         spec = spec.withDefaults(FLOAT_DEFAULTS);
      *         ... // Validation of spec.type, and other attributes, for this type.
-     *         FloatBuilder buf = new FloatBuilder(spec);
-     *         buf.format(value);
-     *         String result = buf.getResult();
+     *         FloatFormatter f = new FloatFormatter(spec);
+     *         String result = f.format(value).getResult();
      *
      * </pre>
      */
     public static class Spec {
 
-        /** The fill character specified, or '\uffff' if unspecified. */
+        /** The fill character specified, or U+FFFF if unspecified. */
         public final char fill;
         /**
-         * Alignment indicator is one of {<code>'&lt;', '^', '>', '='</code>, or '\uffff' if
+         * Alignment indicator is one of {<code>'&lt;', '^', '>', '='</code>, or U+FFFF if
          * unspecified.
          */
         public final char align;
         /**
          * Sign-handling flag, one of <code>'+'</code>, <code>'-'</code>, or <code>' '</code>, or
-         * '\uffff' if unspecified.
+         * U+FFFF if unspecified.
          */
         public final char sign;
         /** The alternative format flag '#' was given. */
@@ -514,7 +513,7 @@
         public final boolean grouping;
         /** Precision decoded from the format, or -1 if unspecified. */
         public final int precision;
-        /** Type key from the format, or '\uffff' if unspecified. */
+        /** Type key from the format, or U+FFFF if unspecified. */
         public final char type;
 
         /** Non-character code point used to represent "no value" in <code>char</code> attributes. */
@@ -604,17 +603,17 @@
         }
 
         /**
-         * Return a merged <code>Spec</code> object, in which any attribute of this object, that is
-         * specified (or <code>true</code>) has the same value in the result, and any attribute of
-         * this object that is unspecified (or <code>false</code>) has the value that attribute
-         * takes in the other object. This the second object supplies default values. (These
+         * Return a merged <code>Spec</code> object, in which any attribute of this object that is
+         * specified (or <code>true</code>), has the same value in the result, and any attribute of
+         * this object that is unspecified (or <code>false</code>), has the value that attribute
+         * takes in the other object. Thus the second object supplies default values. (These
          * defaults may also be unspecified.) The use of this method is to allow a <code>Spec</code>
          * constructed from text to record exactly, and only, what was in the textual specification,
          * while the __format__ method of a client object supplies its type-specific defaults. Thus
          * "20" means "<20s" to a <code>str</code>, ">20.12" to a <code>float</code> and ">20.12g"
          * to a <code>complex</code>.
          *
-         * @param defaults to merge where this object does not specify the attribute.
+         * @param other defaults to merge where this object does not specify the attribute.
          * @return a new Spec object.
          */
         public Spec withDefaults(Spec other) {
@@ -646,7 +645,7 @@
          * @param precision (e.g. decimal places)
          * @param type indicator character
          */
-        public Spec(int width, int precision, char type) {
+        public Spec(int precision, char type) {
             this(' ', '>', Spec.NONE, false, UNSPECIFIED, false, precision, type);
         }
 
diff --git a/src/org/python/core/stringlib/InternalFormatSpec.java b/src/org/python/core/stringlib/InternalFormatSpec.java
--- a/src/org/python/core/stringlib/InternalFormatSpec.java
+++ b/src/org/python/core/stringlib/InternalFormatSpec.java
@@ -7,8 +7,8 @@
  * to a string assumed to be the result of formatting to the given precision.
  * <p>
  * This structure is returned by {@link InternalFormatSpecParser#parse()} and having public members
- * is freely used by {@link InternalFormatSpecParser}, {@link Formatter} and the __format__ methods
- * of client object types.
+ * is freely used by {@link InternalFormatSpecParser}, and the __format__ methods of client object
+ * types.
  * <p>
  * The fields correspond to the elements of a format specification. The grammar of a format
  * specification is:
@@ -45,7 +45,7 @@
      * @param defaultAlign to use if <code>this.align</code>=0 (one of <code>'&lt;'</code>,
      *            <code>'^'</code>, <code>'>'</code>, or <code>'='</code>).
      * @param leaveWidth to reduce effective <code>this.width</code> by
-     * @return
+     * @return padded value
      */
     public String pad(String value, char defaultAlign, int leaveWidth) {
 

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Mon May 19 05:43:38 2014
From: jython-checkins at python.org (frank.wierzbicki)
Date: Mon, 19 May 2014 05:43:38 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Prepare_for_release=2E?=
Message-ID: <3gX5fy4J1lz7LjQ@mail.python.org>

http://hg.python.org/jython/rev/2d4e0bbb87b5
changeset:   7261:2d4e0bbb87b5
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Mon May 19 03:43:31 2014 +0000
summary:
  Prepare for release.

files:
  README.txt |  15 ++++++---------
  build.xml  |   8 ++++----
  2 files changed, 10 insertions(+), 13 deletions(-)


diff --git a/README.txt b/README.txt
--- a/README.txt
+++ b/README.txt
@@ -1,16 +1,13 @@
-Welcome to Jython 2.7b2
+Welcome to Jython 2.7b3
 =======================
 
-This is the second beta release of the 2.7 version of Jython. Thanks to
-Adconion Media Group (http://www.adconion.com/) for sponsoring this release.
+This is the third beta release of the 2.7 version of Jython. Thanks to
+Rackspace (http://www.rackspace.com/) for sponsoring this release.
 Thanks to all who contribute to Jython.
 
-Jython 2.7b2 brings us up to language level compatibility with the 2.7 version
-of CPython. We have focused largely on CPython compatibility, and so this
-release of Jython can run more pure Python apps then any previous release.
-Please see the NEWS file for detailed release notes. This is primarily a bugfix
-release, with numerous improvements, including much improvement on Windows
-support.
+Jython 2.7b3 includes a total rework of ssl which will soon allow pip to work
+on Jython (once pip gets a bugfix as well). It also fixes many bugs including
+some in binascii support, ast support, and many others.
 
 As a beta release we are concentrating on bug fixing and stabilizion for a
 production release.
diff --git a/build.xml b/build.xml
--- a/build.xml
+++ b/build.xml
@@ -84,8 +84,8 @@
         <property name="PY_RELEASE_LEVEL_SNAPSHOT" value="170"/> <!-- 0xAA -->
 
         <!-- The current version info -->
-        <property name="jython.version" value="2.7b2+"/>
-        <property name="jython.version.noplus" value="2.7b2"/>
+        <property name="jython.version" value="2.7b3+"/>
+        <property name="jython.version.noplus" value="2.7b3"/>
         <property name="jython.major_version" value="2"/>
         <property name="jython.minor_version" value="7"/>
         <property name="jython.micro_version" value="0"/>
@@ -407,8 +407,8 @@
     <target name="brand-readme-version" depends="checkout" if="do.snapshot.build">
         <!-- change README.txt version string, if so defined: used for
         snapshot builds. XXX: a bit broken for now-->
-        <replace file="${jython.base.dir}/README.txt" token='2.7a1+'
-        value='2.7a${xxx.revision}' />
+        <replace file="${jython.base.dir}/README.txt" token='2.7b3+'
+        value='2.7b${xxx.revision}' />
         <replace file="${jython.base.dir}/README.txt">
             <replacetoken>=======================</replacetoken>
             <replacevalue>--------------------------

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Mon May 19 05:44:22 2014
From: jython-checkins at python.org (frank.wierzbicki)
Date: Mon, 19 May 2014 05:44:22 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Added_tag_v2=2E7b3_for_chan?=
	=?utf-8?q?geset_2d4e0bbb87b5?=
Message-ID: <3gX5gp6Rbqz7LjQ@mail.python.org>

http://hg.python.org/jython/rev/bca171e29f53
changeset:   7262:bca171e29f53
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Mon May 19 03:44:17 2014 +0000
summary:
  Added tag v2.7b3 for changeset 2d4e0bbb87b5

files:
  .hgtags |  1 +
  1 files changed, 1 insertions(+), 0 deletions(-)


diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -72,3 +72,4 @@
 45b3007473f9606be234eb64a5c98ad8ed160a31 v2.7b2
 45b3007473f9606be234eb64a5c98ad8ed160a31 v2.7b2
 4105892103e8db1bfb539757a399ce9b79a899e0 v2.7b2
+2d4e0bbb87b5d92dea257ef8d26596935e8feb2d v2.7b3

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Mon May 19 10:40:13 2014
From: jython-checkins at python.org (jeff.allen)
Date: Mon, 19 May 2014 10:40:13 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Restore_Jython-specific_Lib?=
 =?utf-8?q?/test/test=5Fhttpservers=2Epy_that_resolves_issue_=232109=2E?=
Message-ID: <3gXDF902TMz7LjQ@mail.python.org>

http://hg.python.org/jython/rev/d6646249f5b1
changeset:   7263:d6646249f5b1
user:        Jeff Allen <ja.py at farowl.co.uk>
date:        Mon May 19 08:44:39 2014 +0100
summary:
  Restore Jython-specific Lib/test/test_httpservers.py that resolves issue #2109.
Differs in Jython until CPython issue 20155 is resolved.

files:
  Lib/test/test_httpservers.py |  540 +++++++++++++++++++++++
  1 files changed, 540 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_httpservers.py b/Lib/test/test_httpservers.py
new file mode 100644
--- /dev/null
+++ b/Lib/test/test_httpservers.py
@@ -0,0 +1,540 @@
+"""Unittests for the various HTTPServer modules.
+
+Written by Cody A.W. Somerville <cody-somerville at ubuntu.com>,
+Josip Dzolonga, and Michael Otteneder for the 2007/08 GHOP contest.
+"""
+
+import os
+import sys
+import re
+import base64
+import shutil
+import urllib
+import httplib
+import tempfile
+import unittest
+import CGIHTTPServer
+
+
+from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
+from SimpleHTTPServer import SimpleHTTPRequestHandler
+from CGIHTTPServer import CGIHTTPRequestHandler
+from StringIO import StringIO
+from test import test_support
+
+
+threading = test_support.import_module('threading')
+
+
+class NoLogRequestHandler:
+    def log_message(self, *args):
+        # don't write log messages to stderr
+        pass
+
+class SocketlessRequestHandler(SimpleHTTPRequestHandler):
+    def __init__(self):
+        self.get_called = False
+        self.protocol_version = "HTTP/1.1"
+
+    def do_GET(self):
+        self.get_called = True
+        self.send_response(200)
+        self.send_header('Content-Type', 'text/html')
+        self.end_headers()
+        self.wfile.write(b'<html><body>Data</body></html>\r\n')
+
+    def log_message(self, fmt, *args):
+        pass
+
+
+class TestServerThread(threading.Thread):
+    def __init__(self, test_object, request_handler):
+        threading.Thread.__init__(self)
+        self.request_handler = request_handler
+        self.test_object = test_object
+
+    def run(self):
+        self.server = HTTPServer(('', 0), self.request_handler)
+        self.test_object.PORT = self.server.socket.getsockname()[1]
+        self.test_object.server_started.set()
+        self.test_object = None
+        try:
+            self.server.serve_forever(0.05)
+        finally:
+            self.server.server_close()
+
+    def stop(self):
+        self.server.shutdown()
+
+
+class BaseTestCase(unittest.TestCase):
+    def setUp(self):
+        self._threads = test_support.threading_setup()
+        os.environ = test_support.EnvironmentVarGuard()
+        self.server_started = threading.Event()
+        self.thread = TestServerThread(self, self.request_handler)
+        self.thread.start()
+        self.server_started.wait()
+
+    def tearDown(self):
+        self.thread.stop()
+        os.environ.__exit__()
+        test_support.threading_cleanup(*self._threads)
+
+    def request(self, uri, method='GET', body=None, headers={}):
+        self.connection = httplib.HTTPConnection('localhost', self.PORT)
+        self.connection.request(method, uri, body, headers)
+        return self.connection.getresponse()
+
+class BaseHTTPRequestHandlerTestCase(unittest.TestCase):
+    """Test the functionality of the BaseHTTPServer focussing on
+    BaseHTTPRequestHandler.
+    """
+
+    HTTPResponseMatch = re.compile('HTTP/1.[0-9]+ 200 OK')
+
+    def setUp (self):
+        self.handler = SocketlessRequestHandler()
+
+    def send_typical_request(self, message):
+        input_msg = StringIO(message)
+        output = StringIO()
+        self.handler.rfile = input_msg
+        self.handler.wfile = output
+        self.handler.handle_one_request()
+        output.seek(0)
+        return output.readlines()
+
+    def verify_get_called(self):
+        self.assertTrue(self.handler.get_called)
+
+    def verify_expected_headers(self, headers):
+        for fieldName in 'Server: ', 'Date: ', 'Content-Type: ':
+            self.assertEqual(sum(h.startswith(fieldName) for h in headers), 1)
+
+    def verify_http_server_response(self, response):
+        match = self.HTTPResponseMatch.search(response)
+        self.assertTrue(match is not None)
+
+    def test_http_1_1(self):
+        result = self.send_typical_request('GET / HTTP/1.1\r\n\r\n')
+        self.verify_http_server_response(result[0])
+        self.verify_expected_headers(result[1:-1])
+        self.verify_get_called()
+        self.assertEqual(result[-1], '<html><body>Data</body></html>\r\n')
+
+    def test_http_1_0(self):
+        result = self.send_typical_request('GET / HTTP/1.0\r\n\r\n')
+        self.verify_http_server_response(result[0])
+        self.verify_expected_headers(result[1:-1])
+        self.verify_get_called()
+        self.assertEqual(result[-1], '<html><body>Data</body></html>\r\n')
+
+    def test_http_0_9(self):
+        result = self.send_typical_request('GET / HTTP/0.9\r\n\r\n')
+        self.assertEqual(len(result), 1)
+        self.assertEqual(result[0], '<html><body>Data</body></html>\r\n')
+        self.verify_get_called()
+
+    def test_with_continue_1_0(self):
+        result = self.send_typical_request('GET / HTTP/1.0\r\nExpect: 100-continue\r\n\r\n')
+        self.verify_http_server_response(result[0])
+        self.verify_expected_headers(result[1:-1])
+        self.verify_get_called()
+        self.assertEqual(result[-1], '<html><body>Data</body></html>\r\n')
+
+    def test_request_length(self):
+        # Issue #10714: huge request lines are discarded, to avoid Denial
+        # of Service attacks.
+        result = self.send_typical_request(b'GET ' + b'x' * 65537)
+        self.assertEqual(result[0], b'HTTP/1.1 414 Request-URI Too Long\r\n')
+        self.assertFalse(self.handler.get_called)
+
+
+class BaseHTTPServerTestCase(BaseTestCase):
+    class request_handler(NoLogRequestHandler, BaseHTTPRequestHandler):
+        protocol_version = 'HTTP/1.1'
+        default_request_version = 'HTTP/1.1'
+
+        def do_TEST(self):
+            self.send_response(204)
+            self.send_header('Content-Type', 'text/html')
+            self.send_header('Connection', 'close')
+            self.end_headers()
+
+        def do_KEEP(self):
+            self.send_response(204)
+            self.send_header('Content-Type', 'text/html')
+            self.send_header('Connection', 'keep-alive')
+            self.end_headers()
+
+        def do_KEYERROR(self):
+            self.send_error(999)
+
+        def do_CUSTOM(self):
+            self.send_response(999)
+            self.send_header('Content-Type', 'text/html')
+            self.send_header('Connection', 'close')
+            self.end_headers()
+
+    def setUp(self):
+        BaseTestCase.setUp(self)
+        self.con = httplib.HTTPConnection('localhost', self.PORT)
+        self.con.connect()
+
+    def test_command(self):
+        self.con.request('GET', '/')
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 501)
+
+    def test_request_line_trimming(self):
+        # Issue 2109: differs in Jython until CPython issue 20155 resolved.
+        self.con._http_vsn_str = 'HTTP/1.1\n'
+        self.con.putrequest('XYZBOGUS', '/')
+        self.con.endheaders()
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 501)
+
+    def test_version_bogus(self):
+        self.con._http_vsn_str = 'FUBAR'
+        self.con.putrequest('GET', '/')
+        self.con.endheaders()
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 400)
+
+    def test_version_digits(self):
+        self.con._http_vsn_str = 'HTTP/9.9.9'
+        self.con.putrequest('GET', '/')
+        self.con.endheaders()
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 400)
+
+    def test_version_none_get(self):
+        self.con._http_vsn_str = ''
+        self.con.putrequest('GET', '/')
+        self.con.endheaders()
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 501)
+
+    def test_version_none(self):
+        # Issue 2109: differs in Jython until CPython issue 20155 resolved.
+        # Test that a valid method is rejected when not HTTP/1.x
+        self.con._http_vsn_str = ''
+        self.con.putrequest('CUSTOM', '/')
+        self.con.endheaders()
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 400)
+
+    def test_version_invalid(self):
+        self.con._http_vsn = 99
+        self.con._http_vsn_str = 'HTTP/9.9'
+        self.con.putrequest('GET', '/')
+        self.con.endheaders()
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 505)
+
+    def test_send_blank(self):
+        self.con._http_vsn_str = ''
+        self.con.putrequest('', '')
+        self.con.endheaders()
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 400)
+
+    def test_header_close(self):
+        self.con.putrequest('GET', '/')
+        self.con.putheader('Connection', 'close')
+        self.con.endheaders()
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 501)
+
+    def test_head_keep_alive(self):
+        self.con._http_vsn_str = 'HTTP/1.1'
+        self.con.putrequest('GET', '/')
+        self.con.putheader('Connection', 'keep-alive')
+        self.con.endheaders()
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 501)
+
+    def test_handler(self):
+        self.con.request('TEST', '/')
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 204)
+
+    def test_return_header_keep_alive(self):
+        self.con.request('KEEP', '/')
+        res = self.con.getresponse()
+        self.assertEqual(res.getheader('Connection'), 'keep-alive')
+        self.con.request('TEST', '/')
+        self.addCleanup(self.con.close)
+
+    def test_internal_key_error(self):
+        self.con.request('KEYERROR', '/')
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 999)
+
+    def test_return_custom_status(self):
+        self.con.request('CUSTOM', '/')
+        res = self.con.getresponse()
+        self.assertEqual(res.status, 999)
+
+
+class SimpleHTTPServerTestCase(BaseTestCase):
+    class request_handler(NoLogRequestHandler, SimpleHTTPRequestHandler):
+        pass
+
+    def setUp(self):
+        BaseTestCase.setUp(self)
+        self.cwd = os.getcwd()
+        basetempdir = tempfile.gettempdir()
+        os.chdir(basetempdir)
+        self.data = 'We are the knights who say Ni!'
+        self.tempdir = tempfile.mkdtemp(dir=basetempdir)
+        self.tempdir_name = os.path.basename(self.tempdir)
+        temp = open(os.path.join(self.tempdir, 'test'), 'wb')
+        temp.write(self.data)
+        temp.close()
+
+    def tearDown(self):
+        try:
+            os.chdir(self.cwd)
+            try:
+                shutil.rmtree(self.tempdir)
+            except OSError:
+                pass
+        finally:
+            BaseTestCase.tearDown(self)
+
+    def check_status_and_reason(self, response, status, data=None):
+        body = response.read()
+        self.assertTrue(response)
+        self.assertEqual(response.status, status)
+        self.assertIsNotNone(response.reason)
+        if data:
+            self.assertEqual(data, body)
+
+    def test_get(self):
+        #constructs the path relative to the root directory of the HTTPServer
+        response = self.request(self.tempdir_name + '/test')
+        self.check_status_and_reason(response, 200, data=self.data)
+        response = self.request(self.tempdir_name + '/')
+        self.check_status_and_reason(response, 200)
+        response = self.request(self.tempdir_name)
+        self.check_status_and_reason(response, 301)
+        response = self.request('/ThisDoesNotExist')
+        self.check_status_and_reason(response, 404)
+        response = self.request('/' + 'ThisDoesNotExist' + '/')
+        self.check_status_and_reason(response, 404)
+        f = open(os.path.join(self.tempdir_name, 'index.html'), 'w')
+        response = self.request('/' + self.tempdir_name + '/')
+        self.check_status_and_reason(response, 200)
+
+        # chmod() doesn't work as expected on Windows, and filesystem
+        # permissions are ignored by root on Unix.
+        if os.name == 'posix' and os.geteuid() != 0:
+            os.chmod(self.tempdir, 0)
+            response = self.request(self.tempdir_name + '/')
+            self.check_status_and_reason(response, 404)
+            os.chmod(self.tempdir, 0755)
+
+    def test_head(self):
+        response = self.request(
+            self.tempdir_name + '/test', method='HEAD')
+        self.check_status_and_reason(response, 200)
+        self.assertEqual(response.getheader('content-length'),
+                         str(len(self.data)))
+        self.assertEqual(response.getheader('content-type'),
+                         'application/octet-stream')
+
+    def test_invalid_requests(self):
+        response = self.request('/', method='FOO')
+        self.check_status_and_reason(response, 501)
+        # requests must be case sensitive,so this should fail too
+        response = self.request('/', method='get')
+        self.check_status_and_reason(response, 501)
+        response = self.request('/', method='GETs')
+        self.check_status_and_reason(response, 501)
+
+
+cgi_file1 = """\
+#!%s
+
+print "Content-type: text/html"
+print
+print "Hello World"
+"""
+
+cgi_file2 = """\
+#!%s
+import cgi
+
+print "Content-type: text/html"
+print
+
+form = cgi.FieldStorage()
+print "%%s, %%s, %%s" %% (form.getfirst("spam"), form.getfirst("eggs"),
+                          form.getfirst("bacon"))
+"""
+
+
+ at unittest.skipIf(hasattr(os, 'geteuid') and os.geteuid() == 0,
+        "This test can't be run reliably as root (issue #13308).")
+class CGIHTTPServerTestCase(BaseTestCase):
+    class request_handler(NoLogRequestHandler, CGIHTTPRequestHandler):
+        pass
+
+    def setUp(self):
+        BaseTestCase.setUp(self)
+        self.parent_dir = tempfile.mkdtemp()
+        self.cgi_dir = os.path.join(self.parent_dir, 'cgi-bin')
+        os.mkdir(self.cgi_dir)
+
+        # The shebang line should be pure ASCII: use symlink if possible.
+        # See issue #7668.
+        if hasattr(os, 'symlink'):
+            self.pythonexe = os.path.join(self.parent_dir, 'python')
+            os.symlink(sys.executable, self.pythonexe)
+        else:
+            self.pythonexe = sys.executable
+
+        self.file1_path = os.path.join(self.cgi_dir, 'file1.py')
+        with open(self.file1_path, 'w') as file1:
+            file1.write(cgi_file1 % self.pythonexe)
+        os.chmod(self.file1_path, 0777)
+
+        self.file2_path = os.path.join(self.cgi_dir, 'file2.py')
+        with open(self.file2_path, 'w') as file2:
+            file2.write(cgi_file2 % self.pythonexe)
+        os.chmod(self.file2_path, 0777)
+
+        self.cwd = os.getcwd()
+        os.chdir(self.parent_dir)
+
+    def tearDown(self):
+        try:
+            os.chdir(self.cwd)
+            if self.pythonexe != sys.executable:
+                os.remove(self.pythonexe)
+            os.remove(self.file1_path)
+            os.remove(self.file2_path)
+            os.rmdir(self.cgi_dir)
+            os.rmdir(self.parent_dir)
+        finally:
+            BaseTestCase.tearDown(self)
+
+    def test_url_collapse_path(self):
+        # verify tail is the last portion and head is the rest on proper urls
+        test_vectors = {
+            '': '//',
+            '..': IndexError,
+            '/.//..': IndexError,
+            '/': '//',
+            '//': '//',
+            '/\\': '//\\',
+            '/.//': '//',
+            'cgi-bin/file1.py': '/cgi-bin/file1.py',
+            '/cgi-bin/file1.py': '/cgi-bin/file1.py',
+            'a': '//a',
+            '/a': '//a',
+            '//a': '//a',
+            './a': '//a',
+            './C:/': '/C:/',
+            '/a/b': '/a/b',
+            '/a/b/': '/a/b/',
+            '/a/b/.': '/a/b/',
+            '/a/b/c/..': '/a/b/',
+            '/a/b/c/../d': '/a/b/d',
+            '/a/b/c/../d/e/../f': '/a/b/d/f',
+            '/a/b/c/../d/e/../../f': '/a/b/f',
+            '/a/b/c/../d/e/.././././..//f': '/a/b/f',
+            '../a/b/c/../d/e/.././././..//f': IndexError,
+            '/a/b/c/../d/e/../../../f': '/a/f',
+            '/a/b/c/../d/e/../../../../f': '//f',
+            '/a/b/c/../d/e/../../../../../f': IndexError,
+            '/a/b/c/../d/e/../../../../f/..': '//',
+            '/a/b/c/../d/e/../../../../f/../.': '//',
+        }
+        for path, expected in test_vectors.iteritems():
+            if isinstance(expected, type) and issubclass(expected, Exception):
+                self.assertRaises(expected,
+                                  CGIHTTPServer._url_collapse_path, path)
+            else:
+                actual = CGIHTTPServer._url_collapse_path(path)
+                self.assertEqual(expected, actual,
+                                 msg='path = %r\nGot:    %r\nWanted: %r' %
+                                 (path, actual, expected))
+
+    def test_headers_and_content(self):
+        res = self.request('/cgi-bin/file1.py')
+        self.assertEqual(('Hello World\n', 'text/html', 200),
+            (res.read(), res.getheader('Content-type'), res.status))
+
+    def test_post(self):
+        params = urllib.urlencode({'spam' : 1, 'eggs' : 'python', 'bacon' : 123456})
+        headers = {'Content-type' : 'application/x-www-form-urlencoded'}
+        res = self.request('/cgi-bin/file2.py', 'POST', params, headers)
+
+        self.assertEqual(res.read(), '1, python, 123456\n')
+
+    def test_invaliduri(self):
+        res = self.request('/cgi-bin/invalid')
+        res.read()
+        self.assertEqual(res.status, 404)
+
+    def test_authorization(self):
+        headers = {'Authorization' : 'Basic %s' %
+                   base64.b64encode('username:pass')}
+        res = self.request('/cgi-bin/file1.py', 'GET', headers=headers)
+        self.assertEqual(('Hello World\n', 'text/html', 200),
+                (res.read(), res.getheader('Content-type'), res.status))
+
+    def test_no_leading_slash(self):
+        # http://bugs.python.org/issue2254
+        res = self.request('cgi-bin/file1.py')
+        self.assertEqual(('Hello World\n', 'text/html', 200),
+             (res.read(), res.getheader('Content-type'), res.status))
+
+    def test_os_environ_is_not_altered(self):
+        signature = "Test CGI Server"
+        os.environ['SERVER_SOFTWARE'] = signature
+        res = self.request('/cgi-bin/file1.py')
+        self.assertEqual((b'Hello World\n', 'text/html', 200),
+                (res.read(), res.getheader('Content-type'), res.status))
+        self.assertEqual(os.environ['SERVER_SOFTWARE'], signature)
+
+
+class SimpleHTTPRequestHandlerTestCase(unittest.TestCase):
+    """ Test url parsing """
+    def setUp(self):
+        self.translated = os.getcwd()
+        self.translated = os.path.join(self.translated, 'filename')
+        self.handler = SocketlessRequestHandler()
+
+    def test_query_arguments(self):
+        path = self.handler.translate_path('/filename')
+        self.assertEqual(path, self.translated)
+        path = self.handler.translate_path('/filename?foo=bar')
+        self.assertEqual(path, self.translated)
+        path = self.handler.translate_path('/filename?a=b&spam=eggs#zot')
+        self.assertEqual(path, self.translated)
+
+    def test_start_with_double_slash(self):
+        path = self.handler.translate_path('//filename')
+        self.assertEqual(path, self.translated)
+        path = self.handler.translate_path('//filename?foo=bar')
+        self.assertEqual(path, self.translated)
+
+
+def test_main(verbose=None):
+    try:
+        cwd = os.getcwd()
+        test_support.run_unittest(BaseHTTPRequestHandlerTestCase,
+                                  SimpleHTTPRequestHandlerTestCase,
+                                  BaseHTTPServerTestCase,
+                                  SimpleHTTPServerTestCase,
+                                  CGIHTTPServerTestCase
+                                 )
+    finally:
+        os.chdir(cwd)
+
+if __name__ == '__main__':
+    test_main()

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Mon May 19 10:40:14 2014
From: jython-checkins at python.org (jeff.allen)
Date: Mon, 19 May 2014 10:40:14 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Add_skip_to_test=2Etest=5Fl?=
 =?utf-8?q?ogging_avoiding_hang_in_regrtest=2E?=
Message-ID: <3gXDFB1tCcz7LjQ@mail.python.org>

http://hg.python.org/jython/rev/06161ebf74ee
changeset:   7264:06161ebf74ee
user:        Jeff Allen <ja.py at farowl.co.uk>
date:        Mon May 19 09:02:22 2014 +0100
summary:
  Add skip to test.test_logging avoiding hang in regrtest.

files:
  Lib/test/test_logging.py |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py
--- a/Lib/test/test_logging.py
+++ b/Lib/test/test_logging.py
@@ -923,6 +923,8 @@
         self.tcpserver.finished.wait(2.0)
         return self.tcpserver.log_output
 
+    @unittest.skipIf(os.name=='java' and os._name=='nt', 
+        'Blocks test completion on Jython Windows.')
     def test_output(self):
         # The log message sent to the SocketHandler is properly received.
         logger = logging.getLogger("tcp")

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Thu May 22 22:02:41 2014
From: jython-checkins at python.org (jim.baker)
Date: Thu, 22 May 2014 22:02:41 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_=232088_Fix_defaultdict_so_?=
 =?utf-8?q?that_derived_classes_=5F=5Fmissing=5F=5F_methods_are_not?=
Message-ID: <3gZMFF1h95z7Ljg@mail.python.org>

http://hg.python.org/jython/rev/9c9fe8e61cf0
changeset:   7265:9c9fe8e61cf0
parent:      7256:8ceb0723b372
user:        Indra Talip <indra.talip at gmail.com>
date:        Sun May 18 21:11:38 2014 +1000
summary:
  #2088 Fix defaultdict so that derived classes __missing__ methods are not ignored.

files:
  Lib/test/test_defaultdict_jy.py                        |  32 +++++++++-
  src/org/python/modules/_collections/PyDefaultDict.java |   8 ++
  2 files changed, 39 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_defaultdict_jy.py b/Lib/test/test_defaultdict_jy.py
--- a/Lib/test/test_defaultdict_jy.py
+++ b/Lib/test/test_defaultdict_jy.py
@@ -94,8 +94,38 @@
         self.assertEquals(d.items(), [("vivify", [])]) 
 
 
+class KeyDefaultDict(defaultdict):
+    """defaultdict to pass the requested key to factory function."""
+    def __missing__(self, key):
+        if self.default_factory is None:
+            raise KeyError("Invalid key '{0}' and no default factory was set")
+        else:
+            val = self.default_factory(key)
+
+        self[key] = val
+        return val
+
+    @classmethod
+    def double(cls, k):
+        return k + k
+
+class OverrideMissingTestCase(unittest.TestCase):
+    def test_dont_call_derived_missing(self):
+        kdd = KeyDefaultDict(KeyDefaultDict.double)
+        kdd[3] = 5
+        self.assertEquals(kdd[3], 5)
+
+    #http://bugs.jython.org/issue2088
+    def test_override_missing(self):
+
+        kdd = KeyDefaultDict(KeyDefaultDict.double)
+        # line below causes KeyError in Jython, ignoring overridden __missing__ method
+        self.assertEquals(kdd[3], 6)
+        self.assertEquals(kdd['ab'], 'abab')
+
+
 def test_main():
-    test_support.run_unittest(PickleTestCase, ThreadSafetyTestCase, GetVariantsTestCase)
+    test_support.run_unittest(PickleTestCase, ThreadSafetyTestCase, GetVariantsTestCase, OverrideMissingTestCase)
 
 
 if __name__ == '__main__':
diff --git a/src/org/python/modules/_collections/PyDefaultDict.java b/src/org/python/modules/_collections/PyDefaultDict.java
--- a/src/org/python/modules/_collections/PyDefaultDict.java
+++ b/src/org/python/modules/_collections/PyDefaultDict.java
@@ -167,6 +167,14 @@
     @ExposedMethod(doc = BuiltinDocs.dict___getitem___doc)
     protected final PyObject defaultdict___getitem__(PyObject key) {
         try {
+            PyType type = getType();
+            if (!getMap().containsKey(key) && type != TYPE) {
+                // is a subclass. if it exists call the subclasses __missing__
+                PyObject missing = type.lookup("__missing__");
+                if (missing != null) {
+                    return missing.__get__(this, type).__call__(key);
+                }
+            }
             return backingMap.get(key);
         } catch (Exception ex) {
             throw Py.KeyError(key);

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Thu May 22 22:02:42 2014
From: jython-checkins at python.org (jim.baker)
Date: Thu, 22 May 2014 22:02:42 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Fix_race_condition_calling_?=
 =?utf-8?q?defaultdict=23=5F=5Fmissing=5F=5F_for_derived_classes=2E?=
Message-ID: <3gZMFG4SCFz7Ljg@mail.python.org>

http://hg.python.org/jython/rev/1a7fdc3681b2
changeset:   7266:1a7fdc3681b2
user:        Indra Talip <indra.talip at gmail.com>
date:        Wed May 21 11:05:26 2014 +1000
summary:
  Fix race condition calling defaultdict#__missing__ for derived classes.

Moves the test for deferring to the subclasses __missing__ to the
CacheLoader#load in order to have atomic behaviour with regards to
calling __missing__.

Adds a test to ThreadSafetyTestCase for subclasses of defaultdict.

files:
  Lib/test/test_defaultdict_jy.py                        |  114 ++++++---
  src/org/python/modules/_collections/PyDefaultDict.java |   17 +-
  2 files changed, 79 insertions(+), 52 deletions(-)


diff --git a/Lib/test/test_defaultdict_jy.py b/Lib/test/test_defaultdict_jy.py
--- a/Lib/test/test_defaultdict_jy.py
+++ b/Lib/test/test_defaultdict_jy.py
@@ -38,30 +38,29 @@
         for t in threads:
             self.assertFalse(t.isAlive())
 
+    class Counter(object):
+        def __init__(self, initial=0):
+            self.atomic = AtomicInteger(initial)
+             # waiting is important here to ensure that
+             # defaultdict factories can step on each other
+            time.sleep(0.001)
+
+        def decrementAndGet(self):
+            return self.atomic.decrementAndGet()
+
+        def incrementAndGet(self):
+            return self.atomic.incrementAndGet()
+
+        def get(self):
+            return self.atomic.get()
+
+        def __repr__(self):
+            return "Counter<%s>" % (self.atomic.get())
+
     def test_inc_dec(self):
+        counters = defaultdict(ThreadSafetyTestCase.Counter)
+        size = 17
 
-        class Counter(object):
-            def __init__(self):
-                self.atomic = AtomicInteger()
-                 # waiting is important here to ensure that
-                 # defaultdict factories can step on each other
-                time.sleep(0.001)
-
-            def decrementAndGet(self):
-                return self.atomic.decrementAndGet()
-
-            def incrementAndGet(self):
-                return self.atomic.incrementAndGet()
-
-            def get(self):
-                return self.atomic.get()
-
-            def __repr__(self):
-                return "Counter<%s>" % (self.atomic.get())
-
-        counters = defaultdict(Counter)
-        size = 17
-        
         def tester():
             for i in xrange(1000):
                 j = (i + randint(0, size)) % size
@@ -70,10 +69,36 @@
                 counters[j].incrementAndGet()
 
         self.run_threads(tester, 20)
-        
+
         for i in xrange(size):
             self.assertEqual(counters[i].get(), 0, counters)
 
+    def test_derived_inc_dec(self):
+        class DerivedDefaultDict(defaultdict):
+            def __missing__(self, key):
+                if self.default_factory is None:
+                    raise KeyError("Invalid key '{0}' and no default factory was set")
+
+                val = self.default_factory(key)
+
+                self[key] = val
+                return val
+
+        counters = DerivedDefaultDict(lambda key: ThreadSafetyTestCase.Counter(key))
+        size = 17
+
+        def tester():
+            for i in xrange(1000):
+                j = (i + randint(0, size)) % size
+                counters[j].decrementAndGet()
+                time.sleep(0.0001)
+                counters[j].incrementAndGet()
+
+        self.run_threads(tester, 20)
+
+        for i in xrange(size):
+            self.assertEqual(counters[i].get(), i, counters)
+
 class GetVariantsTestCase(unittest.TestCase):
 
     #http://bugs.jython.org/issue2133
@@ -94,34 +119,35 @@
         self.assertEquals(d.items(), [("vivify", [])]) 
 
 
-class KeyDefaultDict(defaultdict):
-    """defaultdict to pass the requested key to factory function."""
-    def __missing__(self, key):
-        if self.default_factory is None:
-            raise KeyError("Invalid key '{0}' and no default factory was set")
-        else:
-            val = self.default_factory(key)
-
-        self[key] = val
-        return val
-
-    @classmethod
-    def double(cls, k):
-        return k + k
 
 class OverrideMissingTestCase(unittest.TestCase):
+    class KeyDefaultDict(defaultdict):
+        """defaultdict to pass the requested key to factory function."""
+        def __missing__(self, key):
+            if self.default_factory is None:
+                raise KeyError("Invalid key '{0}' and no default factory was set")
+            else:
+                val = self.default_factory(key)
+
+            self[key] = val
+            return val
+
+        @classmethod
+        def double(cls, k):
+            return k + k
+
+    def setUp(self):
+        self.kdd = OverrideMissingTestCase.KeyDefaultDict(OverrideMissingTestCase.KeyDefaultDict.double)
+
     def test_dont_call_derived_missing(self):
-        kdd = KeyDefaultDict(KeyDefaultDict.double)
-        kdd[3] = 5
-        self.assertEquals(kdd[3], 5)
+        self.kdd[3] = 5
+        self.assertEquals(self.kdd[3], 5)
 
     #http://bugs.jython.org/issue2088
     def test_override_missing(self):
-
-        kdd = KeyDefaultDict(KeyDefaultDict.double)
         # line below causes KeyError in Jython, ignoring overridden __missing__ method
-        self.assertEquals(kdd[3], 6)
-        self.assertEquals(kdd['ab'], 'abab')
+        self.assertEquals(self.kdd[3], 6)
+        self.assertEquals(self.kdd['ab'], 'abab')
 
 
 def test_main():
diff --git a/src/org/python/modules/_collections/PyDefaultDict.java b/src/org/python/modules/_collections/PyDefaultDict.java
--- a/src/org/python/modules/_collections/PyDefaultDict.java
+++ b/src/org/python/modules/_collections/PyDefaultDict.java
@@ -58,6 +58,15 @@
         backingMap = CacheBuilder.newBuilder().build(
                 new CacheLoader<PyObject, PyObject>() {
                     public PyObject load(PyObject key) {
+                        PyType self_type = getType();
+                        if (self_type != TYPE) {
+                            // Is a subclass. If it exists call the subclasses __missing__.
+                            // Otherwise PyDefaultDic.defaultdict___missing__() will
+                            // be invoked.
+                            return PyDefaultDict.this.invoke("__missing__", key);
+                        }
+
+                        // in-lined __missing__
                         if (defaultFactory == Py.None) {
                             throw Py.KeyError(key);
                         }
@@ -167,14 +176,6 @@
     @ExposedMethod(doc = BuiltinDocs.dict___getitem___doc)
     protected final PyObject defaultdict___getitem__(PyObject key) {
         try {
-            PyType type = getType();
-            if (!getMap().containsKey(key) && type != TYPE) {
-                // is a subclass. if it exists call the subclasses __missing__
-                PyObject missing = type.lookup("__missing__");
-                if (missing != null) {
-                    return missing.__get__(this, type).__call__(key);
-                }
-            }
             return backingMap.get(key);
         } catch (Exception ex) {
             throw Py.KeyError(key);

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Thu May 22 22:02:43 2014
From: jython-checkins at python.org (jim.baker)
Date: Thu, 22 May 2014 22:02:43 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Enable_standard_java_resolu?=
 =?utf-8?q?tion_of_=5F=5Fmissing=5F=5F_in_subclasses_of_PyDefaultDict=2E?=
Message-ID: <3gZMFH6v0Sz7Lr8@mail.python.org>

http://hg.python.org/jython/rev/cf2289fafc98
changeset:   7267:cf2289fafc98
user:        Indra Talip <indra.talip at gmail.com>
date:        Thu May 22 21:30:06 2014 +1000
summary:
  Enable standard java resolution of __missing__ in subclasses of PyDefaultDict.

Creates PyDefaultDict#__missing__ that delegates to the exposed __missing__. Extend
defaultdict.derived/PyDefaultDictDerived to call back to PyDefaultDict#__missing__

files:
  src/org/python/modules/_collections/PyDefaultDict.java        |  36 +++------
  src/org/python/modules/_collections/PyDefaultDictDerived.java |   9 ++
  src/templates/defaultdict.derived                             |  11 ++-
  3 files changed, 30 insertions(+), 26 deletions(-)


diff --git a/src/org/python/modules/_collections/PyDefaultDict.java b/src/org/python/modules/_collections/PyDefaultDict.java
--- a/src/org/python/modules/_collections/PyDefaultDict.java
+++ b/src/org/python/modules/_collections/PyDefaultDict.java
@@ -58,22 +58,9 @@
         backingMap = CacheBuilder.newBuilder().build(
                 new CacheLoader<PyObject, PyObject>() {
                     public PyObject load(PyObject key) {
-                        PyType self_type = getType();
-                        if (self_type != TYPE) {
-                            // Is a subclass. If it exists call the subclasses __missing__.
-                            // Otherwise PyDefaultDic.defaultdict___missing__() will
-                            // be invoked.
-                            return PyDefaultDict.this.invoke("__missing__", key);
-                        }
-
-                        // in-lined __missing__
-                        if (defaultFactory == Py.None) {
-                            throw Py.KeyError(key);
-                        }
-                        return defaultFactory.__call__();
+                        return __missing__(key);
                     }
-                }
-        );
+                });
     }
 
     public PyDefaultDict(PyType subtype, Map<PyObject, PyObject> map) {
@@ -87,7 +74,7 @@
         int nargs = args.length - kwds.length;
         if (nargs != 0) {
             defaultFactory = args[0];
-            if (!defaultFactory.isCallable()) {
+            if (!(defaultFactory == Py.None || defaultFactory.isCallable())) {
                 throw Py.TypeError("first argument must be callable");
             }
             PyObject newargs[] = new PyObject[args.length - 1];
@@ -96,22 +83,21 @@
         }
     }
 
+    public PyObject __missing__(PyObject key) {
+        return defaultdict___missing__(key);
+    }
+
     /**
-     * This method is NOT called by the __getitem__ method of the dict class when the
-     * requested key is not found. It is simply here as an alternative to the atomic
-     * construction of that factory. (We actually inline it in.)
+     * This method is does NOT call __setitem__ instead it relies on the fact
+     * that it is called within the context of `CacheLoader#load` to actually
+     * insert the value into the dict.
      */
     @ExposedMethod
     final PyObject defaultdict___missing__(PyObject key) {
         if (defaultFactory == Py.None) {
             throw Py.KeyError(key);
         }
-        PyObject value = defaultFactory.__call__();
-        if (value == null) {
-            return value;
-        }
-        __setitem__(key, value);
-        return value;
+        return defaultFactory.__call__();
     }
 
     @Override
diff --git a/src/org/python/modules/_collections/PyDefaultDictDerived.java b/src/org/python/modules/_collections/PyDefaultDictDerived.java
--- a/src/org/python/modules/_collections/PyDefaultDictDerived.java
+++ b/src/org/python/modules/_collections/PyDefaultDictDerived.java
@@ -1122,6 +1122,15 @@
         return super.__coerce_ex__(o);
     }
 
+    public PyObject __missing__(PyObject key) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__missing__");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(key);
+        }
+        return super.__missing__(key);
+    }
+
     public String toString() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__repr__");
diff --git a/src/templates/defaultdict.derived b/src/templates/defaultdict.derived
--- a/src/templates/defaultdict.derived
+++ b/src/templates/defaultdict.derived
@@ -1,4 +1,13 @@
 base_class: PyDefaultDict
 want_dict: true
 ctr:
-incl: object
+incl: dict
+rest:
+    public PyObject __missing__(PyObject key) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__missing__");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(key);
+        }
+        return super.__missing__(key);
+    }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Thu May 22 22:02:45 2014
From: jython-checkins at python.org (jim.baker)
Date: Thu, 22 May 2014 22:02:45 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Fix_typo_in_comment_for_PyD?=
 =?utf-8?q?efaultDict=23defaultdict=5F=5F=5Fmissing=5F=5F?=
Message-ID: <3gZMFK16Ylz7Ljg@mail.python.org>

http://hg.python.org/jython/rev/e1ad4a0cd16e
changeset:   7268:e1ad4a0cd16e
user:        Indra Talip <indra.talip at gmail.com>
date:        Thu May 22 21:46:42 2014 +1000
summary:
  Fix typo in comment for PyDefaultDict#defaultdict___missing__

files:
  src/org/python/modules/_collections/PyDefaultDict.java |  6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/src/org/python/modules/_collections/PyDefaultDict.java b/src/org/python/modules/_collections/PyDefaultDict.java
--- a/src/org/python/modules/_collections/PyDefaultDict.java
+++ b/src/org/python/modules/_collections/PyDefaultDict.java
@@ -88,9 +88,9 @@
     }
 
     /**
-     * This method is does NOT call __setitem__ instead it relies on the fact
-     * that it is called within the context of `CacheLoader#load` to actually
-     * insert the value into the dict.
+     * This method does NOT call __setitem__ instead it relies on the fact that it is
+     * called within the context of `CacheLoader#load` to actually insert the value
+     * into the dict.
      */
     @ExposedMethod
     final PyObject defaultdict___missing__(PyObject key) {

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Thu May 22 22:02:46 2014
From: jython-checkins at python.org (jim.baker)
Date: Thu, 22 May 2014 22:02:46 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Update_test=5Fdefaultdict?=
 =?utf-8?q?=2Epy_to_cpython_version_and_add_skip_for?=
Message-ID: <3gZMFL3x9nz7Lr6@mail.python.org>

http://hg.python.org/jython/rev/374dd2f0a5ef
changeset:   7269:374dd2f0a5ef
user:        Indra Talip <indra.talip at gmail.com>
date:        Thu May 22 21:55:49 2014 +1000
summary:
  Update test_defaultdict.py to cpython version and add skip for test_recursive_repr

files:
  Lib/test/test_defaultdict.py |  59 ++++++++++++++++++-----
  1 files changed, 45 insertions(+), 14 deletions(-)


diff --git a/Lib/test/test_defaultdict.py b/Lib/test/test_defaultdict.py
--- a/Lib/test/test_defaultdict.py
+++ b/Lib/test/test_defaultdict.py
@@ -24,21 +24,21 @@
         d1[13]
         d1[14]
         self.assertEqual(d1, {12: [42, 24], 13: [], 14: []})
-        self.assert_(d1[12] is not d1[13] is not d1[14])
+        self.assertTrue(d1[12] is not d1[13] is not d1[14])
         d2 = defaultdict(list, foo=1, bar=2)
         self.assertEqual(d2.default_factory, list)
         self.assertEqual(d2, {"foo": 1, "bar": 2})
         self.assertEqual(d2["foo"], 1)
         self.assertEqual(d2["bar"], 2)
         self.assertEqual(d2[42], [])
-        self.assert_("foo" in d2)
-        self.assert_("foo" in d2.keys())
-        self.assert_("bar" in d2)
-        self.assert_("bar" in d2.keys())
-        self.assert_(42 in d2)
-        self.assert_(42 in d2.keys())
-        self.assert_(12 not in d2)
-        self.assert_(12 not in d2.keys())
+        self.assertIn("foo", d2)
+        self.assertIn("foo", d2.keys())
+        self.assertIn("bar", d2)
+        self.assertIn("bar", d2.keys())
+        self.assertIn(42, d2)
+        self.assertIn(42, d2.keys())
+        self.assertNotIn(12, d2)
+        self.assertNotIn(12, d2.keys())
         d2.default_factory = None
         self.assertEqual(d2.default_factory, None)
         try:
@@ -59,6 +59,7 @@
         d1 = defaultdict()
         self.assertEqual(d1.default_factory, None)
         self.assertEqual(repr(d1), "defaultdict(None, {})")
+        self.assertEqual(eval(repr(d1)), d1)
         d1[11] = 41
         self.assertEqual(repr(d1), "defaultdict(None, {11: 41})")
         d2 = defaultdict(int)
@@ -67,7 +68,7 @@
         self.assertEqual(repr(d2), "defaultdict(<type 'int'>, {12: 42})")
         def foo(): return 43
         d3 = defaultdict(foo)
-        self.assert_(d3.default_factory is foo)
+        self.assertTrue(d3.default_factory is foo)
         d3[13]
         self.assertEqual(repr(d3), "defaultdict(%s, {13: 43})" % repr(foo))
 
@@ -111,6 +112,12 @@
         d4[12]
         self.assertEqual(d4, {42: [], 12: []})
 
+        # Issue 6637: Copy fails for empty default dict
+        d = defaultdict()
+        d['a'] = 42
+        e = d.copy()
+        self.assertEqual(e['a'], 42)
+
     def test_shallow_copy(self):
         d1 = defaultdict(foobar, {1: 1})
         d2 = copy.copy(d1)
@@ -126,7 +133,7 @@
         d2 = copy.deepcopy(d1)
         self.assertEqual(d2.default_factory, foobar)
         self.assertEqual(d2, d1)
-        self.assert_(d1[1] is not d2[1])
+        self.assertTrue(d1[1] is not d2[1])
         d1.default_factory = list
         d2 = copy.deepcopy(d1)
         self.assertEqual(d2.default_factory, list)
@@ -137,12 +144,36 @@
         try:
             d1[(1,)]
         except KeyError, err:
-            # XXX: Exception.message is only supported in Python 2.5
-            #self.assertEqual(err.message, (1,))
-            pass
+            self.assertEqual(err.args[0], (1,))
         else:
             self.fail("expected KeyError")
 
+    @unittest.skipIf(test_support.is_jython, "FIXME: incorrect formatting of default_factory when it's a bound method")
+    def test_recursive_repr(self):
+        # Issue2045: stack overflow when default_factory is a bound method
+        class sub(defaultdict):
+            def __init__(self):
+                self.default_factory = self._factory
+            def _factory(self):
+                return []
+        d = sub()
+        self.assertTrue(repr(d).startswith(
+            "defaultdict(<bound method sub._factory of defaultdict(..."))
+
+        # NOTE: printing a subclass of a builtin type does not call its
+        # tp_print slot. So this part is essentially the same test as above.
+        tfn = tempfile.mktemp()
+        try:
+            f = open(tfn, "w+")
+            try:
+                print >>f, d
+            finally:
+                f.close()
+        finally:
+            os.remove(tfn)
+
+    def test_callable_arg(self):
+        self.assertRaises(TypeError, defaultdict, {})
 
 def test_main():
     test_support.run_unittest(TestDefaultDict)

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Thu May 22 22:02:47 2014
From: jython-checkins at python.org (jim.baker)
Date: Thu, 22 May 2014 22:02:47 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython_=28merge_default_-=3E_default?=
	=?utf-8?b?KTogTWVyZ2Vk?=
Message-ID: <3gZMFM6vZSz7Lr8@mail.python.org>

http://hg.python.org/jython/rev/77dcd0a92088
changeset:   7270:77dcd0a92088
parent:      7264:06161ebf74ee
parent:      7269:374dd2f0a5ef
user:        Jim Baker <jim.baker at rackspace.com>
date:        Thu May 22 13:50:00 2014 -0600
summary:
  Merged https://bitbucket.org/jython/jython/pull-request/39/2088-fix-defaultdict-so-that-derived/

files:
  Lib/test/test_defaultdict.py                                  |   59 ++++-
  Lib/test/test_defaultdict_jy.py                               |  104 +++++++--
  src/org/python/modules/_collections/PyDefaultDict.java        |   27 +-
  src/org/python/modules/_collections/PyDefaultDictDerived.java |    9 +
  src/templates/defaultdict.derived                             |   11 +-
  5 files changed, 155 insertions(+), 55 deletions(-)


diff --git a/Lib/test/test_defaultdict.py b/Lib/test/test_defaultdict.py
--- a/Lib/test/test_defaultdict.py
+++ b/Lib/test/test_defaultdict.py
@@ -24,21 +24,21 @@
         d1[13]
         d1[14]
         self.assertEqual(d1, {12: [42, 24], 13: [], 14: []})
-        self.assert_(d1[12] is not d1[13] is not d1[14])
+        self.assertTrue(d1[12] is not d1[13] is not d1[14])
         d2 = defaultdict(list, foo=1, bar=2)
         self.assertEqual(d2.default_factory, list)
         self.assertEqual(d2, {"foo": 1, "bar": 2})
         self.assertEqual(d2["foo"], 1)
         self.assertEqual(d2["bar"], 2)
         self.assertEqual(d2[42], [])
-        self.assert_("foo" in d2)
-        self.assert_("foo" in d2.keys())
-        self.assert_("bar" in d2)
-        self.assert_("bar" in d2.keys())
-        self.assert_(42 in d2)
-        self.assert_(42 in d2.keys())
-        self.assert_(12 not in d2)
-        self.assert_(12 not in d2.keys())
+        self.assertIn("foo", d2)
+        self.assertIn("foo", d2.keys())
+        self.assertIn("bar", d2)
+        self.assertIn("bar", d2.keys())
+        self.assertIn(42, d2)
+        self.assertIn(42, d2.keys())
+        self.assertNotIn(12, d2)
+        self.assertNotIn(12, d2.keys())
         d2.default_factory = None
         self.assertEqual(d2.default_factory, None)
         try:
@@ -59,6 +59,7 @@
         d1 = defaultdict()
         self.assertEqual(d1.default_factory, None)
         self.assertEqual(repr(d1), "defaultdict(None, {})")
+        self.assertEqual(eval(repr(d1)), d1)
         d1[11] = 41
         self.assertEqual(repr(d1), "defaultdict(None, {11: 41})")
         d2 = defaultdict(int)
@@ -67,7 +68,7 @@
         self.assertEqual(repr(d2), "defaultdict(<type 'int'>, {12: 42})")
         def foo(): return 43
         d3 = defaultdict(foo)
-        self.assert_(d3.default_factory is foo)
+        self.assertTrue(d3.default_factory is foo)
         d3[13]
         self.assertEqual(repr(d3), "defaultdict(%s, {13: 43})" % repr(foo))
 
@@ -111,6 +112,12 @@
         d4[12]
         self.assertEqual(d4, {42: [], 12: []})
 
+        # Issue 6637: Copy fails for empty default dict
+        d = defaultdict()
+        d['a'] = 42
+        e = d.copy()
+        self.assertEqual(e['a'], 42)
+
     def test_shallow_copy(self):
         d1 = defaultdict(foobar, {1: 1})
         d2 = copy.copy(d1)
@@ -126,7 +133,7 @@
         d2 = copy.deepcopy(d1)
         self.assertEqual(d2.default_factory, foobar)
         self.assertEqual(d2, d1)
-        self.assert_(d1[1] is not d2[1])
+        self.assertTrue(d1[1] is not d2[1])
         d1.default_factory = list
         d2 = copy.deepcopy(d1)
         self.assertEqual(d2.default_factory, list)
@@ -137,12 +144,36 @@
         try:
             d1[(1,)]
         except KeyError, err:
-            # XXX: Exception.message is only supported in Python 2.5
-            #self.assertEqual(err.message, (1,))
-            pass
+            self.assertEqual(err.args[0], (1,))
         else:
             self.fail("expected KeyError")
 
+    @unittest.skipIf(test_support.is_jython, "FIXME: incorrect formatting of default_factory when it's a bound method")
+    def test_recursive_repr(self):
+        # Issue2045: stack overflow when default_factory is a bound method
+        class sub(defaultdict):
+            def __init__(self):
+                self.default_factory = self._factory
+            def _factory(self):
+                return []
+        d = sub()
+        self.assertTrue(repr(d).startswith(
+            "defaultdict(<bound method sub._factory of defaultdict(..."))
+
+        # NOTE: printing a subclass of a builtin type does not call its
+        # tp_print slot. So this part is essentially the same test as above.
+        tfn = tempfile.mktemp()
+        try:
+            f = open(tfn, "w+")
+            try:
+                print >>f, d
+            finally:
+                f.close()
+        finally:
+            os.remove(tfn)
+
+    def test_callable_arg(self):
+        self.assertRaises(TypeError, defaultdict, {})
 
 def test_main():
     test_support.run_unittest(TestDefaultDict)
diff --git a/Lib/test/test_defaultdict_jy.py b/Lib/test/test_defaultdict_jy.py
--- a/Lib/test/test_defaultdict_jy.py
+++ b/Lib/test/test_defaultdict_jy.py
@@ -38,30 +38,29 @@
         for t in threads:
             self.assertFalse(t.isAlive())
 
+    class Counter(object):
+        def __init__(self, initial=0):
+            self.atomic = AtomicInteger(initial)
+             # waiting is important here to ensure that
+             # defaultdict factories can step on each other
+            time.sleep(0.001)
+
+        def decrementAndGet(self):
+            return self.atomic.decrementAndGet()
+
+        def incrementAndGet(self):
+            return self.atomic.incrementAndGet()
+
+        def get(self):
+            return self.atomic.get()
+
+        def __repr__(self):
+            return "Counter<%s>" % (self.atomic.get())
+
     def test_inc_dec(self):
+        counters = defaultdict(ThreadSafetyTestCase.Counter)
+        size = 17
 
-        class Counter(object):
-            def __init__(self):
-                self.atomic = AtomicInteger()
-                 # waiting is important here to ensure that
-                 # defaultdict factories can step on each other
-                time.sleep(0.001)
-
-            def decrementAndGet(self):
-                return self.atomic.decrementAndGet()
-
-            def incrementAndGet(self):
-                return self.atomic.incrementAndGet()
-
-            def get(self):
-                return self.atomic.get()
-
-            def __repr__(self):
-                return "Counter<%s>" % (self.atomic.get())
-
-        counters = defaultdict(Counter)
-        size = 17
-        
         def tester():
             for i in xrange(1000):
                 j = (i + randint(0, size)) % size
@@ -70,10 +69,36 @@
                 counters[j].incrementAndGet()
 
         self.run_threads(tester, 20)
-        
+
         for i in xrange(size):
             self.assertEqual(counters[i].get(), 0, counters)
 
+    def test_derived_inc_dec(self):
+        class DerivedDefaultDict(defaultdict):
+            def __missing__(self, key):
+                if self.default_factory is None:
+                    raise KeyError("Invalid key '{0}' and no default factory was set")
+
+                val = self.default_factory(key)
+
+                self[key] = val
+                return val
+
+        counters = DerivedDefaultDict(lambda key: ThreadSafetyTestCase.Counter(key))
+        size = 17
+
+        def tester():
+            for i in xrange(1000):
+                j = (i + randint(0, size)) % size
+                counters[j].decrementAndGet()
+                time.sleep(0.0001)
+                counters[j].incrementAndGet()
+
+        self.run_threads(tester, 20)
+
+        for i in xrange(size):
+            self.assertEqual(counters[i].get(), i, counters)
+
 class GetVariantsTestCase(unittest.TestCase):
 
     #http://bugs.jython.org/issue2133
@@ -94,8 +119,39 @@
         self.assertEquals(d.items(), [("vivify", [])]) 
 
 
+
+class OverrideMissingTestCase(unittest.TestCase):
+    class KeyDefaultDict(defaultdict):
+        """defaultdict to pass the requested key to factory function."""
+        def __missing__(self, key):
+            if self.default_factory is None:
+                raise KeyError("Invalid key '{0}' and no default factory was set")
+            else:
+                val = self.default_factory(key)
+
+            self[key] = val
+            return val
+
+        @classmethod
+        def double(cls, k):
+            return k + k
+
+    def setUp(self):
+        self.kdd = OverrideMissingTestCase.KeyDefaultDict(OverrideMissingTestCase.KeyDefaultDict.double)
+
+    def test_dont_call_derived_missing(self):
+        self.kdd[3] = 5
+        self.assertEquals(self.kdd[3], 5)
+
+    #http://bugs.jython.org/issue2088
+    def test_override_missing(self):
+        # line below causes KeyError in Jython, ignoring overridden __missing__ method
+        self.assertEquals(self.kdd[3], 6)
+        self.assertEquals(self.kdd['ab'], 'abab')
+
+
 def test_main():
-    test_support.run_unittest(PickleTestCase, ThreadSafetyTestCase, GetVariantsTestCase)
+    test_support.run_unittest(PickleTestCase, ThreadSafetyTestCase, GetVariantsTestCase, OverrideMissingTestCase)
 
 
 if __name__ == '__main__':
diff --git a/src/org/python/modules/_collections/PyDefaultDict.java b/src/org/python/modules/_collections/PyDefaultDict.java
--- a/src/org/python/modules/_collections/PyDefaultDict.java
+++ b/src/org/python/modules/_collections/PyDefaultDict.java
@@ -58,13 +58,9 @@
         backingMap = CacheBuilder.newBuilder().build(
                 new CacheLoader<PyObject, PyObject>() {
                     public PyObject load(PyObject key) {
-                        if (defaultFactory == Py.None) {
-                            throw Py.KeyError(key);
-                        }
-                        return defaultFactory.__call__();
+                        return __missing__(key);
                     }
-                }
-        );
+                });
     }
 
     public PyDefaultDict(PyType subtype, Map<PyObject, PyObject> map) {
@@ -78,7 +74,7 @@
         int nargs = args.length - kwds.length;
         if (nargs != 0) {
             defaultFactory = args[0];
-            if (!defaultFactory.isCallable()) {
+            if (!(defaultFactory == Py.None || defaultFactory.isCallable())) {
                 throw Py.TypeError("first argument must be callable");
             }
             PyObject newargs[] = new PyObject[args.length - 1];
@@ -87,22 +83,21 @@
         }
     }
 
+    public PyObject __missing__(PyObject key) {
+        return defaultdict___missing__(key);
+    }
+
     /**
-     * This method is NOT called by the __getitem__ method of the dict class when the
-     * requested key is not found. It is simply here as an alternative to the atomic
-     * construction of that factory. (We actually inline it in.)
+     * This method does NOT call __setitem__ instead it relies on the fact that it is
+     * called within the context of `CacheLoader#load` to actually insert the value
+     * into the dict.
      */
     @ExposedMethod
     final PyObject defaultdict___missing__(PyObject key) {
         if (defaultFactory == Py.None) {
             throw Py.KeyError(key);
         }
-        PyObject value = defaultFactory.__call__();
-        if (value == null) {
-            return value;
-        }
-        __setitem__(key, value);
-        return value;
+        return defaultFactory.__call__();
     }
 
     @Override
diff --git a/src/org/python/modules/_collections/PyDefaultDictDerived.java b/src/org/python/modules/_collections/PyDefaultDictDerived.java
--- a/src/org/python/modules/_collections/PyDefaultDictDerived.java
+++ b/src/org/python/modules/_collections/PyDefaultDictDerived.java
@@ -1122,6 +1122,15 @@
         return super.__coerce_ex__(o);
     }
 
+    public PyObject __missing__(PyObject key) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__missing__");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(key);
+        }
+        return super.__missing__(key);
+    }
+
     public String toString() {
         PyType self_type=getType();
         PyObject impl=self_type.lookup("__repr__");
diff --git a/src/templates/defaultdict.derived b/src/templates/defaultdict.derived
--- a/src/templates/defaultdict.derived
+++ b/src/templates/defaultdict.derived
@@ -1,4 +1,13 @@
 base_class: PyDefaultDict
 want_dict: true
 ctr:
-incl: object
+incl: dict
+rest:
+    public PyObject __missing__(PyObject key) {
+        PyType self_type=getType();
+        PyObject impl=self_type.lookup("__missing__");
+        if (impl!=null) {
+            return impl.__get__(this,self_type).__call__(key);
+        }
+        return super.__missing__(key);
+    }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Thu May 22 23:17:20 2014
From: jython-checkins at python.org (jim.baker)
Date: Thu, 22 May 2014 23:17:20 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Issue_2078=3A_Fix_tuple_cre?=
 =?utf-8?q?ation_for_inet6_addresses=2C_and_work_around_for_bound?=
Message-ID: <3gZNvN4ZK3z7LjV@mail.python.org>

http://hg.python.org/jython/rev/0e62bf96c773
changeset:   7271:0e62bf96c773
parent:      7264:06161ebf74ee
user:        Santoso Wijaya <santoso.wijaya at gmail.com>
date:        Wed May 21 18:40:34 2014 -0700
summary:
  Issue 2078: Fix tuple creation for inet6 addresses, and work around for bound inet4 socket local address.

files:
  Lib/_socket.py          |  21 +++++++++++++++++----
  Lib/test/test_socket.py |  16 ++++++++++++++++
  2 files changed, 33 insertions(+), 4 deletions(-)


diff --git a/Lib/_socket.py b/Lib/_socket.py
--- a/Lib/_socket.py
+++ b/Lib/_socket.py
@@ -605,6 +605,14 @@
     }
 }
 
+def _socktuple(addr):
+    port = addr.getPort()
+    inet_addr = addr.getAddress()
+    if isinstance(inet_addr, java.net.Inet6Address):
+        return str(inet_addr.getHostAddress()), port, 0, inet_addr.getScopeId()
+    else:
+        return str(inet_addr.getHostAddress()), port
+
 # actual socket support
 #######################
 
@@ -826,7 +834,7 @@
         self.child_handler = ChildSocketHandler(self)
         b.childHandler(self.child_handler)
 
-        future = b.bind(self.bind_addr)
+        future = b.bind(self.bind_addr.getAddress(), self.bind_addr.getPort())
         self._handle_channel_future(future, "listen")
         self.bind_timestamp = time.time()
         self.channel = future.channel()
@@ -1103,7 +1111,7 @@
             if self.bind_addr == _EPHEMERAL_ADDRESS:
                 raise error(errno.ENOTCONN, "Socket is not connected")
             else:
-                return self.bind_addr.getHostString(), self.bind_addr.getPort()
+                return _socktuple(self.bind_addr)
         # Netty 4 currently races between bind to ephemeral port and the availability
         # of the local address for the channel. Workaround for now is to poll.
         while True:
@@ -1116,14 +1124,19 @@
                 raise error(errno.ENOTCONN, "Socket is not connected")
             log.debug("Poll for local address", extra={"sock": self})
             time.sleep(0.1)  # completely arbitrary
-        return local_addr.getHostString(), local_addr.getPort()
+        if local_addr.getAddress().isAnyLocalAddress():
+            # Netty 4 will default to an IPv6 "any" address from a channel even if it was originally bound to an IPv4 "any" address
+            # so, as a workaround, let's construct a new "any" address using the port information gathered above
+            if type(self.bind_addr.getAddress()) != type(local_addr.getAddress()):
+                return _socktuple(java.net.InetSocketAddress(self.bind_addr.getAddress(), local_addr.getPort()))
+        return _socktuple(local_addr)
 
     def getpeername(self):
         self._verify_channel()
         remote_addr = self.channel.remoteAddress()
         if remote_addr is None:
             raise error(errno.ENOTCONN, "Socket is not connected")
-        return remote_addr.getHostString(), remote_addr.getPort()
+        return _socktuple(remote_addr)
 
 
 _socketmethods = (
diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -577,6 +577,22 @@
         name = sock.getsockname()
         self.assertEqual(name, ("0.0.0.0", PORT+1))
 
+    def testSockNameEphemeralV4(self):
+        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        sock.bind(('', 0))
+        sock.listen(1)
+        name = sock.getsockname()
+        self.assertEqual(len(name), 2)
+        self.assertNotEqual(name[1], 0)
+
+    def testSockNameEphemeralV6(self):
+        sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
+        sock.bind(('', 0, 0, 0))
+        sock.listen(1)
+        name = sock.getsockname()
+        self.assertEqual(len(name), 4)
+        self.assertNotEqual(name[1], 0)
+
     def testSockAttributes(self):
         # Testing required attributes
         for family in [socket.AF_INET, socket.AF_INET6]:

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Thu May 22 23:17:22 2014
From: jython-checkins at python.org (jim.baker)
Date: Thu, 22 May 2014 23:17:22 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython_=28merge_default_-=3E_default?=
	=?utf-8?b?KTogTWVyZ2Vk?=
Message-ID: <3gZNvQ05P9z7LjV@mail.python.org>

http://hg.python.org/jython/rev/291ff41c8317
changeset:   7272:291ff41c8317
parent:      7270:77dcd0a92088
parent:      7271:0e62bf96c773
user:        Jim Baker <jim.baker at rackspace.com>
date:        Thu May 22 14:40:35 2014 -0600
summary:
  Merged https://bitbucket.org/jython/jython/pull-request/40/issue-2078-fix-tuple-creation-for-inet6/

files:
  Lib/_socket.py          |  21 +++++++++++++++++----
  Lib/test/test_socket.py |  16 ++++++++++++++++
  2 files changed, 33 insertions(+), 4 deletions(-)


diff --git a/Lib/_socket.py b/Lib/_socket.py
--- a/Lib/_socket.py
+++ b/Lib/_socket.py
@@ -605,6 +605,14 @@
     }
 }
 
+def _socktuple(addr):
+    port = addr.getPort()
+    inet_addr = addr.getAddress()
+    if isinstance(inet_addr, java.net.Inet6Address):
+        return str(inet_addr.getHostAddress()), port, 0, inet_addr.getScopeId()
+    else:
+        return str(inet_addr.getHostAddress()), port
+
 # actual socket support
 #######################
 
@@ -826,7 +834,7 @@
         self.child_handler = ChildSocketHandler(self)
         b.childHandler(self.child_handler)
 
-        future = b.bind(self.bind_addr)
+        future = b.bind(self.bind_addr.getAddress(), self.bind_addr.getPort())
         self._handle_channel_future(future, "listen")
         self.bind_timestamp = time.time()
         self.channel = future.channel()
@@ -1103,7 +1111,7 @@
             if self.bind_addr == _EPHEMERAL_ADDRESS:
                 raise error(errno.ENOTCONN, "Socket is not connected")
             else:
-                return self.bind_addr.getHostString(), self.bind_addr.getPort()
+                return _socktuple(self.bind_addr)
         # Netty 4 currently races between bind to ephemeral port and the availability
         # of the local address for the channel. Workaround for now is to poll.
         while True:
@@ -1116,14 +1124,19 @@
                 raise error(errno.ENOTCONN, "Socket is not connected")
             log.debug("Poll for local address", extra={"sock": self})
             time.sleep(0.1)  # completely arbitrary
-        return local_addr.getHostString(), local_addr.getPort()
+        if local_addr.getAddress().isAnyLocalAddress():
+            # Netty 4 will default to an IPv6 "any" address from a channel even if it was originally bound to an IPv4 "any" address
+            # so, as a workaround, let's construct a new "any" address using the port information gathered above
+            if type(self.bind_addr.getAddress()) != type(local_addr.getAddress()):
+                return _socktuple(java.net.InetSocketAddress(self.bind_addr.getAddress(), local_addr.getPort()))
+        return _socktuple(local_addr)
 
     def getpeername(self):
         self._verify_channel()
         remote_addr = self.channel.remoteAddress()
         if remote_addr is None:
             raise error(errno.ENOTCONN, "Socket is not connected")
-        return remote_addr.getHostString(), remote_addr.getPort()
+        return _socktuple(remote_addr)
 
 
 _socketmethods = (
diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py
--- a/Lib/test/test_socket.py
+++ b/Lib/test/test_socket.py
@@ -577,6 +577,22 @@
         name = sock.getsockname()
         self.assertEqual(name, ("0.0.0.0", PORT+1))
 
+    def testSockNameEphemeralV4(self):
+        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        sock.bind(('', 0))
+        sock.listen(1)
+        name = sock.getsockname()
+        self.assertEqual(len(name), 2)
+        self.assertNotEqual(name[1], 0)
+
+    def testSockNameEphemeralV6(self):
+        sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
+        sock.bind(('', 0, 0, 0))
+        sock.listen(1)
+        name = sock.getsockname()
+        self.assertEqual(len(name), 4)
+        self.assertNotEqual(name[1], 0)
+
     def testSockAttributes(self):
         # Testing required attributes
         for family in [socket.AF_INET, socket.AF_INET6]:

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Fri May 23 01:26:17 2014
From: jython-checkins at python.org (jim.baker)
Date: Fri, 23 May 2014 01:26:17 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_ShutdownCloser_now_implemen?=
 =?utf-8?q?ts_Runnable_instead_of_extending_Thread_to?=
Message-ID: <3gZRm93w7Sz7LnK@mail.python.org>

http://hg.python.org/jython/rev/55f7eaddc954
changeset:   7273:55f7eaddc954
user:        Timoth?e Lecomte <lecomte.timothee at gmail.com>
date:        Thu May 22 17:06:36 2014 -0600
summary:
  ShutdownCloser now implements Runnable instead of extending Thread to
avoid subclass audit issues when running under a SecurityManager,
which causes issues with respect to finalization/shutdown.

See http://code.google.com/p/guava-libraries/issues/detail?id=92#c86
for a related issue seen in the Google Collections library.

files:
  ACKNOWLEDGMENTS                        |   1 +
  src/org/python/core/PySystemState.java |  10 +++-------
  2 files changed, 4 insertions(+), 7 deletions(-)


diff --git a/ACKNOWLEDGMENTS b/ACKNOWLEDGMENTS
--- a/ACKNOWLEDGMENTS
+++ b/ACKNOWLEDGMENTS
@@ -108,6 +108,7 @@
     Indra Talip
     Michael B?sch
     Richard Eckart de Castilho
+    Timoth?e Lecomte
 
 Local Variables:
 mode: indented-text
diff --git a/src/org/python/core/PySystemState.java b/src/org/python/core/PySystemState.java
--- a/src/org/python/core/PySystemState.java
+++ b/src/org/python/core/PySystemState.java
@@ -1644,7 +1644,7 @@
         // Python scripts expect that files are closed upon an orderly cleanup of the VM.
         private Thread initShutdownCloser() {
             try {
-                Thread shutdownHook = new ShutdownCloser();
+                Thread shutdownHook = new Thread(new ShutdownCloser(), "Jython Shutdown Closer");
                 Runtime.getRuntime().addShutdownHook(shutdownHook);
                 return shutdownHook;
             } catch (SecurityException se) {
@@ -1653,13 +1653,9 @@
             }
         }
 
-        private class ShutdownCloser extends Thread {
+        private class ShutdownCloser implements Runnable {
 
-            private ShutdownCloser() {
-                super("Jython Shutdown Closer");
-            }
-
-            @Override
+        	@Override
             public synchronized void run() {
                 runClosers(resourceClosers);
                 resourceClosers.clear();

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Fri May 23 20:53:22 2014
From: jython-checkins at python.org (jim.baker)
Date: Fri, 23 May 2014 20:53:22 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Add_test_on_subclassing_byt?=
 =?utf-8?q?earray=2C_then_using_=5F=5Flen=5F=5F?=
Message-ID: <3gZxfp4NTQz7LkS@mail.python.org>

http://hg.python.org/jython/rev/2f8b6070b213
changeset:   7274:2f8b6070b213
user:        Jim Baker <jim.baker at rackspace.com>
date:        Fri May 23 12:32:44 2014 -0600
summary:
  Add test on subclassing bytearray, then using __len__

files:
  Lib/test/test_bytes_jy.py |  23 +++++++++++++++++++++++
  1 files changed, 23 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_bytes_jy.py b/Lib/test/test_bytes_jy.py
new file mode 100644
--- /dev/null
+++ b/Lib/test/test_bytes_jy.py
@@ -0,0 +1,23 @@
+# Tests against problems we have seen in Jython's implementation of
+# buffer, bytes, bytearray, and memoryview to prevent possible
+# regression as well as integration with Java.
+
+import unittest
+import test.test_support
+
+
+class ByteArraySubclassTest(unittest.TestCase):
+
+    def test_len(self):
+        class Sub(bytearray): pass
+        s = Sub("abc123")
+        self.assertEqual(len(s), 6)
+ 
+
+def test_main():
+    test.test_support.run_unittest(
+        ByteArraySubclassTest)
+
+
+if __name__ == "__main__":
+    test_main()

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Fri May 23 20:53:23 2014
From: jython-checkins at python.org (jim.baker)
Date: Fri, 23 May 2014 20:53:23 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Fix_StackOverflowError_when?=
 =?utf-8?q?_calling_=5F=5Flen=5F=5F_on_a_subclassed_bytearray?=
Message-ID: <3gZxfq624Kz7Lkg@mail.python.org>

http://hg.python.org/jython/rev/c35aa984452f
changeset:   7275:c35aa984452f
user:        Indra Talip <indra.talip at gmail.com>
date:        Fri May 23 12:35:51 2014 -0600
summary:
  Fix StackOverflowError when calling __len__ on a subclassed bytearray

files:
  src/org/python/core/PyByteArray.java |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/src/org/python/core/PyByteArray.java b/src/org/python/core/PyByteArray.java
--- a/src/org/python/core/PyByteArray.java
+++ b/src/org/python/core/PyByteArray.java
@@ -1466,7 +1466,7 @@
 
     @ExposedMethod(doc = BuiltinDocs.bytearray___len___doc)
     final int bytearray___len__() {
-        return __len__();
+        return super.__len__();
     }
 
     /**

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Fri May 23 22:12:07 2014
From: jython-checkins at python.org (jim.baker)
Date: Fri, 23 May 2014 22:12:07 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Test_for_a_more_informative?=
 =?utf-8?q?_message_when_a_Java_StackOverflowError_is_raised?=
Message-ID: <3gZzPg5Cblz7LvK@mail.python.org>

http://hg.python.org/jython/rev/b3081e5fad92
changeset:   7276:b3081e5fad92
user:        Jim Baker <jim.baker at rackspace.com>
date:        Fri May 23 13:54:07 2014 -0600
summary:
  Test for a more informative message when a Java StackOverflowError is raised

files:
  Lib/test/test_exceptions_jy.py                   |  15 ++++++++++
  tests/java/javatests/StackOverflowErrorTest.java |  12 ++++++++
  2 files changed, 27 insertions(+), 0 deletions(-)


diff --git a/Lib/test/test_exceptions_jy.py b/Lib/test/test_exceptions_jy.py
--- a/Lib/test/test_exceptions_jy.py
+++ b/Lib/test/test_exceptions_jy.py
@@ -4,6 +4,8 @@
 """
 from test import test_support
 import unittest
+from javatests import StackOverflowErrorTest
+
 
 class C:
     def __str__(self):
@@ -41,6 +43,19 @@
             return
         unittest.fail("if __str__ raises an exception, re-raise")
 
+    def test_wrap_StackOverflowError(self):
+        with self.assertRaises(RuntimeError) as cm:
+            StackOverflowErrorTest.throwStackOverflowError()
+        self.assertEqual(
+            cm.exception.message,
+            "maximum recursion depth exceeded (Java StackOverflowError)")
+
+        with self.assertRaises(RuntimeError) as cm:
+            StackOverflowErrorTest.causeStackOverflowError()
+        self.assertEqual(
+            cm.exception.message,
+            "maximum recursion depth exceeded (Java StackOverflowError)")
+
 
 def test_main():
     test_support.run_unittest(ExceptionsTestCase)
diff --git a/tests/java/javatests/StackOverflowErrorTest.java b/tests/java/javatests/StackOverflowErrorTest.java
new file mode 100644
--- /dev/null
+++ b/tests/java/javatests/StackOverflowErrorTest.java
@@ -0,0 +1,12 @@
+package javatests;
+
+public class StackOverflowErrorTest {
+
+    public static void throwStackOverflowError() {
+	throw new StackOverflowError();
+    }
+
+    public static void causeStackOverflowError() {
+	causeStackOverflowError();
+    }
+}

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Fri May 23 22:12:08 2014
From: jython-checkins at python.org (jim.baker)
Date: Fri, 23 May 2014 22:12:08 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Provide_more_informative_me?=
 =?utf-8?q?ssage_when_a_Java_StackOverflowError_is_raised?=
Message-ID: <3gZzPh7190z7LvK@mail.python.org>

http://hg.python.org/jython/rev/5688f9c2b743
changeset:   7277:5688f9c2b743
user:        Michael B?sch <m at bues.ch>
date:        Fri May 23 13:54:14 2014 -0600
summary:
  Provide more informative message when a Java StackOverflowError is raised

files:
  src/org/python/core/Py.java |  2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)


diff --git a/src/org/python/core/Py.java b/src/org/python/core/Py.java
--- a/src/org/python/core/Py.java
+++ b/src/org/python/core/Py.java
@@ -508,7 +508,7 @@
         } else if (t instanceof InvocationTargetException) {
             return JavaError(((InvocationTargetException) t).getTargetException());
         } else if (t instanceof StackOverflowError) {
-            return Py.RuntimeError("maximum recursion depth exceeded");
+            return Py.RuntimeError("maximum recursion depth exceeded (Java StackOverflowError)");
         } else if (t instanceof OutOfMemoryError) {
             memory_error((OutOfMemoryError) t);
         }

-- 
Repository URL: http://hg.python.org/jython

From jython-checkins at python.org  Tue May 27 21:06:05 2014
From: jython-checkins at python.org (frank.wierzbicki)
Date: Tue, 27 May 2014 21:06:05 +0200 (CEST)
Subject: [Jython-checkins] =?utf-8?q?jython=3A_Added_tag_v2=2E7b3_for_chan?=
	=?utf-8?q?geset_5688f9c2b743?=
Message-ID: <3gdPld23YBzRgJ@mail.python.org>

http://hg.python.org/jython/rev/9cd9ab75eade
changeset:   7278:9cd9ab75eade
user:        Frank Wierzbicki <fwierzbicki at gmail.com>
date:        Tue May 27 19:05:44 2014 +0000
summary:
  Added tag v2.7b3 for changeset 5688f9c2b743

files:
  .hgtags |  2 ++
  1 files changed, 2 insertions(+), 0 deletions(-)


diff --git a/.hgtags b/.hgtags
--- a/.hgtags
+++ b/.hgtags
@@ -73,3 +73,5 @@
 45b3007473f9606be234eb64a5c98ad8ed160a31 v2.7b2
 4105892103e8db1bfb539757a399ce9b79a899e0 v2.7b2
 2d4e0bbb87b5d92dea257ef8d26596935e8feb2d v2.7b3
+2d4e0bbb87b5d92dea257ef8d26596935e8feb2d v2.7b3
+5688f9c2b7439654684e3e35cbecaffe413385c5 v2.7b3

-- 
Repository URL: http://hg.python.org/jython