[Python-bugs-list] [ python-Bugs-732299 ] Doctest chokes on '\x00'

[SourceForge.net]

Bugs item #732299, was opened at 2003-05-04 15:23
Message generated for change (Comment added) made by tim_one
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=732299&group_id=5470

Category: Python Library
>Group: Not a Bug
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Francis Avila (francisga)
>Assigned to: Tim Peters (tim_one)
Summary: Doctest chokes on '\x00'

Initial Comment:
$ cat ./dtbug.py
"""Demonstrate a python doctest bug."""

def main():
        """See here, I can't include any string literals with 
nulls in them:

        >>> x = '\x00'
        >>>

        """
        pass

if __name__ == '__main__':
        import doctest, dtbug
        doctest.testmod(dtbug)

$ python ./dtbug.py
*******************************************************
**********
Failure in example: x = ''
from line #2 of dtbug.main
Exception raised:
Traceback (most recent call last):
  File "/usr/lib/python2.2/doctest.py", line 499, in 
_run_examples_inner
    exec compile(source, "<string>", "single") in globs
TypeError: compile() argument 1 must be string 
without null bytes, not string
*******************************************************
**********
1 items had failures:
   1 of   1 in dtbug.main
***Test Failed*** 1 failures.


I'm not actually certain this is really a bug, but it 
certainly makes it impossible to demonstrate the use 
of functions that operate on binary data.


----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2003-05-04 23:07

Message:
Logged In: YES 
user_id=31435

This isn't a bug, can't be helped, but can be avoided.  You 
have to be aware that the doctest goes through *two* 
stages of string processing:  the Python interpreter does 
one when it's reading your source code.  At that time, the 
\x00 in your doctest string gets changed into a NUL byte.  
Similarly, all other backslash escape sequences are also 
processed.

The *result* of that is the *input* string doctest sees, and 
doctest has to pass its input on to the compiler again, in 
order to run your tests.

The easiest way to avoid the problem is to make the 
docstring a raw string (stick the letter 'r' -- without the 
quotes -- at the front of the docstring).  Another way is to 
double-up your backslashes, as the docs point out.  When 
Python sees

"\x00"

it produces a four-character string, a single backslash 
followed by "x00".  That's the one you want doctest to pass 
on to the compiler.  The raw string

r"\x00"

gets to the same end via a different mechanism.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=732299&group_id=5470